Skip to content

hud runner adjustments for full evalsets #32

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
ericpts merged 12 commits into from
Aug 19, 2025
Merged

hud runner adjustments for full evalsets #32

ericpts merged 12 commits into from
Aug 19, 2025

Conversation

@lorenss-m
Copy link
Collaborator

@lorenss-m lorenss-m commented Aug 13, 2025

No description provided.

@lorenss-m
Copy link
Collaborator Author

lorenss-m commented Aug 13, 2025 •
edited
Loading

Add hud evaluation support

What changed

  • Added hud_eval.py for running hud tasksets with parallel execution support
  • Added support for hud environment in main.py
  • Added support for final response submission to evaluation

New features

  • Run evaluations: python hud_eval.py --taskset OSWorld-Verified --parallel
  • Use hud with main.py: python main.py --query "..." --env="hud"

@ericpts ericpts self-requested a review August 19, 2025 19:04
@ericpts ericpts merged commit 1083d1c into google-gemini:main Aug 19, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ericpts ericpts Awaiting requested review from ericpts

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@lorenss-m @ericpts