fix(dspy): exclude gpt-5-chat from reasoning model classification #9033
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The regex pattern for identifying reasoning models was incorrectly matching gpt-5-chat, causing it to be subject to reasoning model validation requirements and attempting to pass reasoning parameters that Azure's API doesn't support for this model. Updated the regex to use negative lookahead (?!-chat) to exclude gpt-5-chat while still allowing other gpt-5 variants like gpt-5-mini, gpt-5-nano, and gpt-5-pro to be recognized as reasoning models. - Updated regex pattern in dspy/clients/lm.py - Added test case for gpt-5-chat in test_reasoning_model_token_parameter - Added dedicated test test_gpt_5_chat_not_reasoning_model Fixes issue where gpt-5-chat (Azure AI Foundry) was incorrectly classified as a reasoning model, preventing normal usage with standard temperature and max_tokens values.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Fixes #9032
The
gpt-5-chatmodel (available through Azure AI Foundry) was being incorrectly classified as a reasoning model, causing it to be subject to reasoning model validation requirements and attempting to pass reasoning parameters that Azure's API doesn't support.Changes
Updated regex pattern in
dspy/clients/lm.py: Modified the reasoning model detection regex to use negative lookahead(?!-chat)to excludegpt-5-chatwhile still allowing othergpt-5variants (gpt-5-mini,gpt-5-nano,gpt-5-pro) to be recognized as reasoning models.Added test coverage:
gpt-5-chattest case totest_reasoning_model_token_parameterto verify it's not treated as a reasoning modeltest_gpt_5_chat_not_reasoning_modelto explicitly verifygpt-5-chatcan use standard temperature and max_tokens valuesTechnical Details
The original regex pattern
r"^(?:o[1345](?:-(?:mini|nano))?(?:-\d{4}-\d{2}-\d{2})?|gpt-5(?:-(?:mini|nano))?)$"would matchgpt-5-chatbecause the optional suffix allowed any text aftergpt-5-.The new pattern
r"^(?:o[1345](?:-(?:mini|nano|pro))?(?:-\d{4}-\d{2}-\d{2})?|gpt-5(?!-chat)(?:-.*)?)$"uses:(?!-chat)to explicitly exclude-chatsuffix(?:-.*)?to allow other suffixes (like-mini,-nano,-pro, or future variants)proto the o-model variants to supportgpt-5-proas a reasoning modelTesting
test_gpt_5_chat_not_reasoning_modelverifiesgpt-5-chatis not treated as a reasoning modeltest_reasoning_model_token_parameterincludesgpt-5-chatas a non-reasoning model caseincludesgpt-5-pro` to verify it's correctly recognized as a reasoning modelImpact
This fix allows users to use
gpt-5-chatwith standard conversational parameters (e.g.,temperature=0.7,max_tokens=1000) without validation errors, and prevents the runtime error where Azure's API rejects reasoning parameters for this non-reasoning model.Related
fix/reasoning-model-regex-pattern