Skip to content

Commit 74af711

Browse files
committed
fix: skip check unstreamed tool arg tokens when tool call name is present
Signed-off-by: Sungjae Lee <[email protected]>
1 parent d6704dd commit 74af711

File tree

1 file changed

+7
-1
lines changed

1 file changed

+7
-1
lines changed

vllm/entrypoints/openai/serving_chat.py

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1729,7 +1729,10 @@ def _should_check_for_unstreamed_tool_arg_tokens(
17291729
"""
17301730
Check to see if we should check for unstreamed tool arguments tokens.
17311731
This is only applicable when auto tool parsing is enabled, the delta
1732-
is a tool call with arguments.
1732+
is a tool call with arguments. We only enter this branch when arguments
1733+
are being streamed incrementally (i.e., when name is not present).
1734+
This branch is skipped if the complete tool call (including name) is present
1735+
or will be streamed.
17331736
"""
17341737

17351738
return bool(
@@ -1743,6 +1746,9 @@ def _should_check_for_unstreamed_tool_arg_tokens(
17431746
and delta_message.tool_calls[0]
17441747
and delta_message.tool_calls[0].function
17451748
and delta_message.tool_calls[0].function.arguments is not None
1749+
# Only enter this branch when name is not already streamed
1750+
# (i.e., when arguments are being streamed incrementally)
1751+
and not delta_message.tool_calls[0].function.name
17461752
)
17471753

17481754
def _make_request_with_harmony(

0 commit comments

Comments
 (0)