v1

This GitHub Action automatically runs a before/after comparison of edited prompts and posts the result to your Pull Request.

promptfoo-action: v1.2.0

1.2.0 (2025-11-23)

Features

• add --no-table and --no-progress-bar flags (#583) (5b9ed14)
• add working-directory as an input (#359) (481aac1)
• Add comprehensive caching support for GitHub Actions (#625) (438d8e3)
• Add custom provider dependency detection with wildcard support (#615) (425c078)
• add max-concurrency option for rate limiting (#591) (6d0934e)
• add no-cache parameter to promptfoo-action (#669) (93f18ad)
• add option to disable comment (#584) (d660275)
• add support for loading .env files (#579) (68bfbb7)
• add workflow_dispatch support for manual triggering (#585) (30143f8)
• make prompts parameter optional (#108) (cb2158d)
• make prompts optional (#414) (7f9fc19)
• make prompts parameter truly optional (#592) (95bf94d)
• Skip sharing when auth is not available (#614) (71ad945)
• validate Promptfoo API key before running evaluation (#683) (b6d2253)

Bug Fixes

• Add build step before package in dependabot workflow (#679) (6bc088d)
• allow more git refs in validateGitRef (#636) (7c3955a)
• checkout repo to load script file in github-script action (#687) (de3682f)
• ci: rebuild dist files properly in dependabot PRs (#722) (41e299b)
• deps: update dependency glob to v13 (#750) (f4f1022)
• dist: rebuild after dependency updates (#721) (02fb87e)
• dist: rebuild distribution files after dependency update (#720) (2b9a209)
• Potential fix for code scanning alert no. 5: Workflow does not contain permissions (#542) (764baf4)
• skip empty commits in dependabot workflow (#682) (0c501d5)
• workflows: remove dependabot post-update automation workflow (#403) (05d17a4)
• workflows: update permissions in GitHub Actions workflows (#459) (edc671c)

Miscellaneous Chores

• update Node.js runtime from node16 to node20 (#588) (37e3eff)

v1.1.0 - Add workflow_dispatch and push event support

What's New in v1.1.0

This release adds support for workflow_dispatch and push events, along with many other features and improvements. All changes are fully backward compatible with v1.0.0.

New Features

Event Support

• workflow_dispatch: Manual triggering with custom inputs (#585)
• push: Automatic evaluation on push events

Performance & Caching

• Comprehensive caching support for GitHub Actions (#625)
• Custom provider dependency detection with wildcard support (#615)
• max-concurrency option for rate limiting (#591)
• no-cache flag to bypass caching (#669)

Configuration & Display

• no-table and no-progress-bar flags for minimal output (#583)
• disable-comment option to skip PR comments (#584)
• Support for loading .env files (#579)
• Made prompts parameter truly optional (#592)

Security & Validation

• API key validation before running evaluation (#683)
• Improved git ref validation (#636)

Bug Fixes

• Skip sharing when auth is not available (#614)
• Fix checkout repo to load script file (#687)
• Various security and permission fixes

Documentation

• Clarify version requirements for different event types
• Update all examples to use @v1
• Improve caching documentation

Breaking Changes

None - this release is fully backward compatible with v1.0.0.

Upgrade Instructions

If you're currently using @v1:

• No action needed! The v1 tag now points to v1.1.0
• All your existing workflows will automatically get these new features

If you're using @v1.0.0:

• Update to @v1 or @v1.1.0 to access new features
• No breaking changes - your workflows will continue to work

Full Changelog: v1.0.0...v1.1.0