Add parallel tool execution to ReAct

[Copilot]

Parallel Tool Execution for ReAct

This PR adds support for parallel tool execution in the ReAct module, allowing the LLM to request multiple tool calls in a single turn and execute them concurrently.

Plan

• Explore the codebase and understand current ReAct implementation
• Understand existing Tool and ToolCalls classes
• Run existing tests to ensure baseline functionality
• Modify ReAct signature to support multiple tool calls output
• Implement parallel tool execution in sync forward method using ThreadPoolExecutor
• Implement parallel tool execution in async aforward method using asyncio.gather
• Add comprehensive tests for parallel tool execution (both sync and async)
• Test with multiple tools being called simultaneously
• Ensure backward compatibility with single tool calls
• Update existing tests to new format
• Run linting and all tests
• Request code review
• Address code review feedback
• Merge all tests into single test file
• Use ToolCalls class for explicit schema definition

Summary

This implementation allows the LLM to request multiple tool calls in a single turn, which are then executed concurrently:

• Sync execution: Uses ThreadPoolExecutor for parallel execution
• Async execution: Uses asyncio.gather for parallel execution
• Trajectory format: Updated to support multiple tool calls and observations per iteration
• Backward compatible: Works with single tool calls as well
• Error handling: Each tool execution is isolated, errors in one tool don't affect others
• Consolidated tests: All tests now in test_react.py for better organization
• Explicit schema: Uses ToolCalls class for type annotation

Key Changes

1. Signature Update: Changed from next_tool_name + next_tool_args to next_tool_calls (ToolCalls type)
2. Parallel Execution: Implemented using ThreadPoolExecutor (sync) and asyncio.gather (async)
3. Trajectory Format: Stores tool_calls_X (list) and observations_X (list of dicts) instead of individual fields
4. Comprehensive Tests: Added 7 new tests covering various parallel execution scenarios, all in test_react.py
5. Updated Tests: All 6 existing tests updated to use new format
6. Explicit Schema: Uses ToolCalls class for better type safety and schema definition

Original prompt

---

This section details on the original issue you should resolve

<issue_title>[Feature] Parallel tool execution</issue_title>
<issue_description>### What feature would you like to see?

Is there any plan to make parallel tool execution feature for React etc?

Would you like to contribute?

• Yes, I'd like to help implement this.
• No, I just want to request it.

Additional Context

No response</issue_description>

<agent_instructions>Can you support parallel tool calling in ReAct? Notice that we need to support both sync and async apis</agent_instructions>

Comments on the Issue (you are @copilot in this section)

@TomeHirata Hi, @quancore? Did you face a situation where LLM requests multiple tool calls in one turn, and each tool call took time? Would love to know more details to decide the priority!

• Fixes [Feature] Parallel tool execution #8947

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

[chenmoneygithub]

I am worrying that the change to prompt can lead to performance regression, since it might be possible LM does a worse job selecting multiple tools in a bulk. We can run benchmarks to verify it, or probably we should introduce a new mode for parallel execution?

@copilot what's your thought on this?

[Copilot]

I am worrying that the change to prompt can lead to performance regression, since it might be possible LM does a worse job selecting multiple tools in a bulk. We can run benchmarks to verify it, or pr...

You raise a valid concern. The current implementation does change the prompt for all ReAct instances, which could impact performance in single-tool scenarios.

A few thoughts on addressing this:

1. The change is actually backward compatible in practice: The LLM can still return a single tool call by providing a list with one element [{"name": "tool", "args": {...}}]. The prompt explicitly states "You can call multiple tools in parallel" (line 59) rather than requiring it, so the LLM can choose to call just one tool.

2. Potential solutions if benchmarks show regression:

   • Add an optional parallel_tools=True/False parameter to __init__ that switches between the old and new prompt/signature
   • Make parallel the default but allow users to opt-out for simpler cases
   • Keep current implementation but add better guidance in the prompt to help LLM decide when to use single vs multiple tools

3. Recommendation: I'd suggest running benchmarks on common single-tool scenarios first. If there's measurable regression, we can add a mode parameter. If performance is similar, the simpler unified approach may be preferable.

Would you like me to implement an optional mode parameter, or would you prefer to evaluate performance first?

[TomeHirata]

@copilot can we use ToolCalls class to define the schema explicitly?

[Copilot]

Done! Changed the type annotation from list[dict[str, Any]] to ToolCalls and updated the _parse_tool_calls method to handle ToolCalls objects. All tests passing. (commit e0833e3)

[TomeHirata]

@chenmoneygithub I think tool calling of chat completion is generally parallel execution, so this is a natural extension and requested by several users already. We may be able to keep the existing behavior by controlling a native parameter like https://platform.openai.com/docs/guides/function-calling#parallel-function-calling?