|
| 1 | +# E2E Testing Guide |
| 2 | + |
| 3 | +E2E tests validate full workflows against real Cloudflare Workers and Docker containers. |
| 4 | + |
| 5 | +## Architecture |
| 6 | + |
| 7 | +All E2E tests share a **single sandbox container** for performance. Test isolation is achieved through **sessions** - each test file gets a unique session that provides isolated shell state (env vars, working directory) within the shared container. |
| 8 | + |
| 9 | +``` |
| 10 | +┌─────────────────────────────────────────────────────┐ |
| 11 | +│ Shared Sandbox │ |
| 12 | +│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ |
| 13 | +│ │ Session A │ │ Session B │ │ Session C │ │ |
| 14 | +│ │ (test 1) │ │ (test 2) │ │ (test 3) │ │ |
| 15 | +│ └─────────────┘ └─────────────┘ └─────────────┘ │ |
| 16 | +│ │ |
| 17 | +│ Shared filesystem & processes │ |
| 18 | +└─────────────────────────────────────────────────────┘ |
| 19 | +``` |
| 20 | + |
| 21 | +**Key files:** |
| 22 | + |
| 23 | +- `tests/e2e/global-setup.ts` - Creates sandbox before tests, warms containers |
| 24 | +- `tests/e2e/helpers/global-sandbox.ts` - Provides `getSharedSandbox()` API |
| 25 | +- `vitest.e2e.config.ts` - Configures parallel execution with global setup |
| 26 | + |
| 27 | +## Writing Tests |
| 28 | + |
| 29 | +### Basic Template |
| 30 | + |
| 31 | +```typescript |
| 32 | +import { describe, test, expect, beforeAll } from 'vitest'; |
| 33 | +import { |
| 34 | + getSharedSandbox, |
| 35 | + createUniqueSession |
| 36 | +} from './helpers/global-sandbox'; |
| 37 | + |
| 38 | +describe('My Feature', () => { |
| 39 | + let workerUrl: string; |
| 40 | + let headers: Record<string, string>; |
| 41 | + |
| 42 | + beforeAll(async () => { |
| 43 | + const sandbox = await getSharedSandbox(); |
| 44 | + workerUrl = sandbox.workerUrl; |
| 45 | + headers = sandbox.createHeaders(createUniqueSession()); |
| 46 | + }, 120000); |
| 47 | + |
| 48 | + test('should do something', async () => { |
| 49 | + const response = await fetch(`${workerUrl}/api/execute`, { |
| 50 | + method: 'POST', |
| 51 | + headers, |
| 52 | + body: JSON.stringify({ command: 'echo hello' }) |
| 53 | + }); |
| 54 | + expect(response.status).toBe(200); |
| 55 | + }, 60000); |
| 56 | +}); |
| 57 | +``` |
| 58 | + |
| 59 | +### Using Python Image |
| 60 | + |
| 61 | +For tests requiring Python (code interpreter, etc.): |
| 62 | + |
| 63 | +```typescript |
| 64 | +beforeAll(async () => { |
| 65 | + const sandbox = await getSharedSandbox(); |
| 66 | + workerUrl = sandbox.workerUrl; |
| 67 | + // Use createPythonHeaders instead of createHeaders |
| 68 | + headers = sandbox.createPythonHeaders(createUniqueSession()); |
| 69 | +}, 120000); |
| 70 | +``` |
| 71 | + |
| 72 | +### File Isolation |
| 73 | + |
| 74 | +Since the filesystem is shared, use unique paths to avoid conflicts: |
| 75 | + |
| 76 | +```typescript |
| 77 | +const sandbox = await getSharedSandbox(); |
| 78 | +const testDir = sandbox.uniquePath('my-feature'); // /workspace/test-abc123/my-feature |
| 79 | + |
| 80 | +await fetch(`${workerUrl}/api/file/write`, { |
| 81 | + method: 'POST', |
| 82 | + headers, |
| 83 | + body: JSON.stringify({ |
| 84 | + path: `${testDir}/config.json`, |
| 85 | + content: '{"key": "value"}' |
| 86 | + }) |
| 87 | +}); |
| 88 | +``` |
| 89 | + |
| 90 | +### Port Usage |
| 91 | + |
| 92 | +Ports must be exposed in the Dockerfile. Currently exposed: |
| 93 | + |
| 94 | +- `8080` - General testing |
| 95 | +- `9090`, `9091`, `9092` - Process readiness tests |
| 96 | +- `9998` - Process lifecycle tests |
| 97 | +- `9999` - WebSocket tests |
| 98 | + |
| 99 | +To use a new port: |
| 100 | + |
| 101 | +1. Add it to both `tests/e2e/test-worker/Dockerfile` and `Dockerfile.python` |
| 102 | +2. Document which test uses it |
| 103 | + |
| 104 | +### Process Cleanup |
| 105 | + |
| 106 | +Always clean up background processes: |
| 107 | + |
| 108 | +```typescript |
| 109 | +test('should start server', async () => { |
| 110 | + const startRes = await fetch(`${workerUrl}/api/process/start`, { |
| 111 | + method: 'POST', |
| 112 | + headers, |
| 113 | + body: JSON.stringify({ command: 'bun run server.js' }) |
| 114 | + }); |
| 115 | + const { id: processId } = await startRes.json(); |
| 116 | + |
| 117 | + // ... test logic ... |
| 118 | + |
| 119 | + // Cleanup |
| 120 | + await fetch(`${workerUrl}/api/process/${processId}`, { |
| 121 | + method: 'DELETE', |
| 122 | + headers |
| 123 | + }); |
| 124 | +}, 60000); |
| 125 | +``` |
| 126 | + |
| 127 | +## Test Organization |
| 128 | + |
| 129 | +| File | Purpose | |
| 130 | +| --------------------------------------- | ---------------------------- | |
| 131 | +| `comprehensive-workflow.test.ts` | Happy path integration tests | |
| 132 | +| `process-lifecycle-workflow.test.ts` | Error handling for processes | |
| 133 | +| `process-readiness-workflow.test.ts` | waitForLog/waitForPort tests | |
| 134 | +| `code-interpreter-workflow.test.ts` | Python/JS code execution | |
| 135 | +| `file-operations-workflow.test.ts` | File read/write/list | |
| 136 | +| `streaming-operations-workflow.test.ts` | Streaming command output | |
| 137 | +| `websocket-workflow.test.ts` | WebSocket connections | |
| 138 | +| `bucket-mounting.test.ts` | R2 bucket mounting (CI only) | |
| 139 | + |
| 140 | +## Running Tests |
| 141 | + |
| 142 | +```bash |
| 143 | +# All E2E tests |
| 144 | +npm run test:e2e |
| 145 | + |
| 146 | +# Single file |
| 147 | +npm run test:e2e -- -- tests/e2e/process-lifecycle-workflow.test.ts |
| 148 | + |
| 149 | +# Single test by name |
| 150 | +npm run test:e2e -- -- tests/e2e/git-clone-workflow.test.ts -t 'should clone repo' |
| 151 | +``` |
| 152 | + |
| 153 | +## Debugging |
| 154 | + |
| 155 | +- Tests auto-retry once on failure (`retry: 1` in config) |
| 156 | +- Global setup logs sandbox ID on startup - check for initialization errors |
| 157 | +- If tests fail on first run only, the container might not be warmed (check global-setup.ts initializes the right image type) |
| 158 | +- Port conflicts: check no other test uses the same port |
| 159 | + |
| 160 | +## What NOT to Do |
| 161 | + |
| 162 | +- **Don't create new sandboxes unless strictly necessary** - use `getSharedSandbox()` |
| 163 | +- **Don't skip cleanup** - leaked processes affect other tests |
| 164 | +- **Don't use hardcoded ports** without adding to Dockerfile |
| 165 | +- **Don't rely on filesystem state** from other tests - use unique paths |
0 commit comments