Project Hierarchy

Project Hierarchy

├─ project
│ ├─ tasks.jsonl
│ ├─ src-init
│ │ ├─ ...others
│ ├─ src
│ │ ├─ ...others
│ ├─ test
│ ├─ eval
│ ├─ .evalignore
│ ├─ assets
│ ├─ packages.json 

• tasks.jsonl: Task infomation, definition:

export interface RawTask {
  id: string
  date: string
  level: 'easy' | 'challenging' | 'moderate'
  description: string
}

• src-init: Fallback initial content for evaluation if init task retry fails.

• src: Initial files source for calibration when startTask is set.

• test: Test folder (naming: taskid.spec.js).

• eval: Local-only execution workspace for task runs.

• .evalignore: Ignore rules for rush eval (syntax like .gitignore).

• assets: Static resources copied to runtime when enabled.

Project Configuration

Eval

Define eval parameters in each project's package.json like:

{
  "name": "@web-bench/react",
  "version": "0.0.1",
  // others
  "eval": {
    "stable": true
  }
}

full:

export interface ProjectEvalConfig {
  /**
   * project is stable
   */
  stable?: boolean
   /**
   * should screenshot
   * default: true
   */
  screenshot?: boolean
}

Scripts

{
  "name": "@web-bench/react",
  "version": "0.0.1",
  // others
  "scripts": {
    "init": "node init.js",
    "build": "VITE_CJS_IGNORE_WARNING=true vite build",
    "test": "npx @web-bench/test-util"
  }
}

• init: Optional, Evaluation Workflow Step 5.
• build: Optional, Evaluation Workflow Step 6.
• test: Required, Evaluation Workflow Step 7.