Add comprehensive validation tests with abstraction layer for future TypeScript migration

[nanasess]

Summary

Implement comprehensive validation tests for setup-chromedriver shell scripts (setup-chromedriver.sh and setup-chromedriver.ps1) with an abstraction layer that prepares for future TypeScript integration. This will enable unified validation of Chrome paths, versions, architectures, download URLs, actual binary hash verification, and enhanced CI/CD workflows.

Problem

Current situation:

1. Insufficient Testing: Complex logic in shell scripts lacks comprehensive test coverage
2. Missing Validation: No verification that downloaded binaries are correct
3. Future Extensibility: Design doesn't prepare for TypeScript migration
4. Platform-Specific Fragmentation: Platform-specific logic is scattered across different scripts
5. Limited CI/CD Coverage: Current workflows don't validate binary integrity or cross-platform equivalence

Both scripts perform critical operations without validation mechanisms:

• Chrome installation path detection
• Chrome version detection and API selection (Chrome 115 boundary)
• Platform/architecture mapping
• Download URL generation
• ChromeDriver binary integrity

Proposed Solution

1. Abstraction Layer Implementation

Define a common ChromeDriverInstaller interface supporting both current shell scripts and future TypeScript implementation:

interface ChromeDriverInstaller {
  detectChromePath(): Promise<string>;
  getChromeVersion(chromePath: string): Promise<string>;
  detectPlatform(): Promise<Platform>;
  resolveDownloadUrl(version: string, platform: Platform): Promise<string>;
  downloadChromeDriver(url: string, destination: string): Promise<void>;
  calculateHash(filePath: string): Promise<string>;
  install(options?: InstallOptions): Promise<InstallResult>;
}

2. Adapter Pattern Implementation

• BashScriptAdapter: Wrapper for setup-chromedriver.sh
• PowerShellAdapter: Wrapper for setup-chromedriver.ps1
• TypeScriptAdapter: Wrapper for future setup-chromedriver.ts

3. Actual Binary Verification

Rigorous validation through hash comparison:

1. Get expected URL from Chrome for Testing API
2. Direct download from that URL (expected value)
3. Execute script to download (actual value)
4. Compare SHA256 hashes of both binaries

4. Comprehensive Validation Tests

• Chrome Path Validation: Standard paths for each platform
• Version Validation: Exact match, fallback, API selection logic
• Architecture Validation: Platform-specific mappings
• URL Validation: Legacy API vs Chrome for Testing API
• Hash Validation: Actual binary integrity verification
• Cross-Platform Equivalence: Unified behavior verification across all platforms

5. Enhanced GitHub Actions Workflows

New Workflow: comprehensive-validation.yml

• Multi-platform testing: Ubuntu, macOS, Windows across multiple versions
• Hash verification jobs: Binary integrity validation for each platform
• Performance testing: Download speed and installation time benchmarks
• Cross-adapter equivalence: Bash vs PowerShell result comparison

Existing Workflow Enhancements

• test.yml expansion: Integration of new test suites, coverage reporting
• windows.yml enhancement: PowerShell-specific testing, Windows hash verification
• CI/CD integration: Automated test result reporting, failure diagnostics

Benefits

1. Gradual TypeScript Migration: Smooth transition path from shell scripts to TypeScript
2. Actual Binary Verification: Guarantee that downloaded files are correct
3. Unified Testing: All implementations share the same interface and tests
4. Type Safety: Future safety through TypeScript's type system
5. Cross-Platform Consistency: Uniform behavior across all platforms
6. Enhanced CI/CD Integration: Comprehensive automated testing with binary verification
7. Performance Monitoring: Automated detection of performance regressions
8. Quality Assurance: 95%+ test coverage with real-world validation

Acceptance Criteria

Core Implementation

• Implement ChromeDriverInstaller interface
• Chrome path validation tests (Linux/macOS/Windows)
• Chrome version detection and API selection logic tests
• Architecture mapping validation (linux64, mac-x64, mac-arm64, win32)
• Download URL validation tests (legacy/new API)
• Actual binary hash comparison tests
• Error handling tests (network failures, invalid versions, etc.)
• Cross-platform equivalence tests

GitHub Actions Integration

• Create comprehensive-validation.yml workflow
• Enhance test.yml with new test suites and coverage reporting
• Enhance windows.yml with PowerShell-specific testing
• Implement hash verification jobs for all platforms
• Add performance benchmarking and regression detection
• Integrate test result artifacts and failure diagnostics

Quality & Documentation

• Design compatible with future TypeScript implementation
• Achieve 95%+ test coverage
• Automated testing across all platforms in CI/CD
• Documentation updates for test implementation
• Performance monitoring and alerting

Technical Notes

Implementation Architecture

• Abstraction Layer: Unified implementation through common interface
• Adapter Pattern: Wrappers for each implementation (Bash/PowerShell/TypeScript)
• Hash Verification: SHA256-based actual binary integrity verification
• Mock Support: External dependency mocking for test environments
• CI/CD Integration: Multi-platform validation with performance monitoring

Gradual Migration Strategy

• Phase 1: Shell scripts + adapter tests + GitHub Actions enhancement
• Phase 2: Partial TypeScript implementation
• Phase 3: Complete TypeScript migration (maintaining backward compatibility)

Test Coverage Areas

• Platform-specific behavior (Linux/macOS vs Windows)
• Chrome for Testing API network requests
• Chrome version detection accuracy
• Fallback mechanism behavior
• Proper error handling under various conditions
• Binary integrity verification
• Cross-platform equivalence validation
• Performance benchmarking

GitHub Actions Strategy

• Comprehensive Validation: New dedicated workflow for thorough testing
• Enhanced Existing Workflows: Gradual integration of new capabilities
• Multi-Platform Matrix: Testing across Ubuntu, macOS, Windows variants
• Performance Monitoring: Automated benchmarking and regression detection
• Quality Gates: Coverage thresholds and test success requirements

This issue addresses both current shell script quality assurance and establishes the foundation for future TypeScript integration with robust CI/CD validation.

File Structure

__tests__/
├── lib/
│   ├── ChromeDriverInstallerInterface.ts  # Common interface
│   ├── adapters/
│   │   ├── BashScriptAdapter.ts           # .sh wrapper
│   │   ├── PowerShellAdapter.ts           # .ps1 wrapper
│   │   └── TypeScriptAdapter.ts           # .ts wrapper (future)
│   ├── core/
│   │   ├── HashValidator.ts               # Hash verification
│   │   ├── DownloadManager.ts             # Download processing
│   │   ├── PlatformDetector.ts            # Platform detection
│   │   └── VersionResolver.ts             # Version resolution
│   └── utils/
│       ├── TestEnvironment.ts             # Test environment
│       ├── MockServer.ts                  # Mock server
│       └── TestData.ts                    # Test data management
├── unit/
│   ├── chrome-detection.test.ts           # Chrome path detection
│   ├── version-resolution.test.ts         # Version resolution logic
│   ├── hash-validation.test.ts            # Hash verification
│   └── platform-mapping.test.ts          # Platform mapping
├── integration/
│   ├── installer.test.ts                  # End-to-end testing
│   ├── cross-platform.test.ts            # Cross-platform validation
│   └── performance.test.ts               # Performance benchmarks
├── equivalence/
│   └── cross-adapter.test.ts             # Adapter equivalence
└── fixtures/
    ├── test-binaries/                     # Test binary files
    └── api-responses/                     # Mock API responses

.github/workflows/
├── comprehensive-validation.yml           # New: Comprehensive testing
├── test.yml                              # Enhanced: Existing Unix testing
├── windows.yml                           # Enhanced: Existing Windows testing
├── claude.yml                            # Maintained: Claude integration
└── claude-code-review.yml                # Maintained: Code review

Implementation Priority

High Priority - Essential for meeting Issue #408 acceptance criteria

Complexity: Medium-High

• Cross-platform testing coordination is technically complex
• Hash verification requires careful binary comparison
• Adapter pattern implementation needs thorough design
• GitHub Actions workflow enhancement requires CI/CD expertise
• Clear implementation guidance available from existing patterns

Estimated Effort: 7.5-11.5 days

• Abstraction layer design and implementation: 2-3 days
• Adapter implementations: 2-3 days
• Comprehensive test suite: 2-3 days
• GitHub Actions workflow enhancement: 1-2 days
• Error handling and quality assurance: 1 day
• Documentation and future preparation: 0.5 day

Implementation Phases

Phase 1: Abstraction Layer (2-3 days)

• Common interface definition
• Adapter pattern implementation
• Basic test infrastructure

Phase 2: Hash Verification (1-2 days)

• SHA256 validation system
• Chrome for Testing API integration
• Binary comparison functionality

Phase 3: Comprehensive Testing (2-3 days)

• Unit tests for all components
• Integration and cross-platform tests
• Performance benchmarking

Phase 4: GitHub Actions Enhancement (1-2 days)

• New comprehensive validation workflow
• Existing workflow enhancements
• CI/CD integration features

Phase 5: Quality Assurance (1 day)

• Error handling robustness
• Security enhancements
• Performance optimization

Phase 6: Documentation (0.5 day)

• Updated documentation
• Migration preparation
• Implementation guides

Next Steps

1. Create abstraction layer: ChromeDriverInstallerInterface.ts
2. Implement adapters: Bash and PowerShell script wrappers
3. Build test suite: Comprehensive validation tests
4. Enhance GitHub Actions: Multi-platform automated testing with hash verification
5. Documentation: Update README with test execution procedures and CI/CD integration

[nanasess]

@claude Please handle tests/setup-chromedriver.sh.test.ts and create a PR.

[claude]

Claude finished @nanasess's task —— View job • claude/issue-408-20250709_010621 • Create PR ➔

---

I've successfully created comprehensive tests for the setup-chromedriver.sh script!

Todo List

• Analyze existing codebase and test patterns
• Examine the setup-chromedriver.sh script to understand its functionality
• Create comprehensive test file at __tests__/setup-chromedriver.sh.test.ts
• Install dependencies and run tests to ensure they work (Note: Unable to run bash commands due to permissions)
• Commit changes and provide PR link

Test Coverage Created

The test file covers all critical functionality:

• Argument parsing: VERSION, ARCH, CHROMEAPP parameters
• Platform detection: Linux vs macOS behavior
• Chrome version detection: From installed browser and provided version
• API selection: Chrome < 115 vs >= 115 (different APIs)
• Linux package management: apt-get installation logic
• Fallback mechanisms: When exact version not found
• Error handling: curl failures, missing Chrome, jq parsing errors, unzip failures
• macOS specific behavior: Architecture transformation (mac64 → mac-x64)

Implementation Details

• Uses Jest with TypeScript (consistent with existing tests)
• Comprehensive mocking of external dependencies (curl, jq, unzip, Chrome executable)
• Tests isolated in temporary directories
• Covers all major code paths and edge cases
• Error scenarios properly tested