- 
                Notifications
    You must be signed in to change notification settings 
- Fork 7.9k
fix: stream tokens in agent instead of accumulated responses #10425
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
| Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the  You can disable this status message by setting the  WalkthroughThis PR adds streaming support and multimodal input handling to the agent framework. It refactors event handling to use callback-based workflows, introduces token streaming via callbacks, adds helper methods for message conversion with empty-text skipping, and includes tests for streaming behavior and multimodal content extraction. Changes
 Sequence Diagram(s)sequenceDiagram
    participant Agent as Agent (agent.py)
    participant EventMgr as Event Manager
    participant Events as Event Handler (events.py)
    participant TokenCB as Token Callback
    participant MsgCB as Message Callback
    Agent->>EventMgr: get on_token_callback()
    Agent->>Events: process_agent_events(..., send_message_callback, send_token_callback)
    
    Note over Events: Initialize message_id
    Events->>Events: detect streaming
    
    alt Streaming Path
        Events->>Events: on_chain_stream event
        rect rgb(200, 240, 255)
            Note over Events: Stream tokens
            loop Each token chunk
                Events->>TokenCB: send_token_callback(chunk, message_id)
            end
        end
        Events->>MsgCB: send_message_callback(final_message)
    else Non-Streaming Path
        Events->>Events: on_chain_end event
        Events->>MsgCB: send_message_callback(message)
    end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45–60 minutes Areas requiring extra attention: 
 Suggested labels
 Suggested reviewers
 Pre-merge checks and finishing touches❌ Failed checks (1 error, 3 warnings)
 ✅ Passed checks (3 passed)
 Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment  | 
| Codecov Report❌ Patch coverage is  ❌ Your patch check has failed because the patch coverage (10.29%) is below the target coverage (40.00%). You can increase the patch coverage or adjust the target coverage. Additional details and impacted files@@            Coverage Diff             @@
##             main   #10425      +/-   ##
==========================================
- Coverage   30.64%   30.63%   -0.01%     
==========================================
  Files        1318     1318              
  Lines       59710    59734      +24     
  Branches     8926     8934       +8     
==========================================
+ Hits        18298    18300       +2     
- Misses      40565    40584      +19     
- Partials      847      850       +3     
 Flags with carried forward coverage won't be shown. Click here to find out more. 
 🚀 New features to boost your workflow:
 | 
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 12
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️  Outside diff range comments (1)
src/lfx/src/lfx/base/agents/events.py (1)
75-79: BaseMessage handling uses non-existent .text(); extract content safely.Use .content and normalize to string.
Apply:
@@ - if isinstance(input_message, BaseMessage): - input_message = input_message.text() + if isinstance(input_message, BaseMessage): + content = getattr(input_message, "content", "") + if isinstance(content, list): + parts = [ + it.get("text", "") + for it in content + if isinstance(it, dict) and it.get("type") == "text" and (it.get("text") or "").strip() + ] + input_message = " ".join(parts) + else: + input_message = str(content)
🧹 Nitpick comments (4)
src/backend/tests/unit/components/agents/test_multimodal_agent.py (1)
31-62: Avoid duplicating production parsing logic in tests.These tests reimplement run_agent’s multimodal parsing. Extract a small helper (e.g., _extract_text_and_images) in LCAgentComponent and test that directly to prevent drift.
src/backend/tests/unit/components/agents/test_agent_events.py (2)
762-762: Prefer setting Message.id, not stuffing id into .data.process_agent_events should read message.id; update the test accordingly.
Apply:
@@ - agent_message.data["id"] = "test-message-id" + agent_message.id = "test-message-id"Do the same in other tests that set the id.
458-458: Remove stray no-op f-string.This expression has no effect.
Apply:
- f"{end_event['name']}_{end_event['run_id']}" + # tool_key = f"{end_event['name']}_{end_event['run_id']}" # not needed; relying on updated content lookupsrc/lfx/src/lfx/base/agents/events.py (1)
333-341: Tighten tool handler Protocol typing.Use ToolContent to match actual handlers and map.
Apply:
@@ -class ToolEventHandler(Protocol): +class ToolEventHandler(Protocol): async def __call__( self, event: dict[str, Any], agent_message: Message, - tool_blocks_map: dict[str, ContentBlock], + tool_blocks_map: dict[str, ToolContent], send_message_callback: SendMessageFunctionType, start_time: float, ) -> tuple[Message, float]: ...
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (5)
- src/backend/tests/unit/components/agents/test_agent_events.py(10 hunks)
- src/backend/tests/unit/components/agents/test_multimodal_agent.py(1 hunks)
- src/frontend/src/stores/messagesStore.ts(2 hunks)
- src/lfx/src/lfx/base/agents/agent.py(6 hunks)
- src/lfx/src/lfx/base/agents/events.py(16 hunks)
🧰 Additional context used
📓 Path-based instructions (9)
src/frontend/src/**/*.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (.cursor/rules/frontend_development.mdc)
src/frontend/src/**/*.{ts,tsx,js,jsx}: All frontend TypeScript and JavaScript code should be located under src/frontend/src/ and organized into components, pages, icons, stores, types, utils, hooks, services, and assets directories as per the specified directory layout.
Use React 18 with TypeScript for all UI components in the frontend.
Format all TypeScript and JavaScript code using the make format_frontend command.
Lint all TypeScript and JavaScript code using the make lint command.
Files:
- src/frontend/src/stores/messagesStore.ts
src/frontend/src/stores/**/*.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (.cursor/rules/frontend_development.mdc)
Use Zustand for state management in frontend stores.
Files:
- src/frontend/src/stores/messagesStore.ts
src/backend/tests/unit/components/**/*.py
📄 CodeRabbit inference engine (.cursor/rules/backend_development.mdc)
src/backend/tests/unit/components/**/*.py: Mirror the component directory structure for unit tests in src/backend/tests/unit/components/
Use ComponentTestBaseWithClient or ComponentTestBaseWithoutClient as base classes for component unit tests
Provide file_names_mapping for backward compatibility in component tests
Create comprehensive unit tests for all new components
Files:
- src/backend/tests/unit/components/agents/test_multimodal_agent.py
- src/backend/tests/unit/components/agents/test_agent_events.py
{src/backend/**/*.py,tests/**/*.py,Makefile}
📄 CodeRabbit inference engine (.cursor/rules/backend_development.mdc)
{src/backend/**/*.py,tests/**/*.py,Makefile}: Run make format_backend to format Python code before linting or committing changes
Run make lint to perform linting checks on backend Python code
Files:
- src/backend/tests/unit/components/agents/test_multimodal_agent.py
- src/backend/tests/unit/components/agents/test_agent_events.py
src/backend/tests/unit/**/*.py
📄 CodeRabbit inference engine (.cursor/rules/backend_development.mdc)
Test component integration within flows using create_flow, build_flow, and get_build_events utilities
Files:
- src/backend/tests/unit/components/agents/test_multimodal_agent.py
- src/backend/tests/unit/components/agents/test_agent_events.py
src/backend/tests/**/*.py
📄 CodeRabbit inference engine (.cursor/rules/testing.mdc)
src/backend/tests/**/*.py: Unit tests for backend code must be located in the 'src/backend/tests/' directory, with component tests organized by component subdirectory under 'src/backend/tests/unit/components/'.
Test files should use the same filename as the component under test, with an appropriate test prefix or suffix (e.g., 'my_component.py' → 'test_my_component.py').
Use the 'client' fixture (an async httpx.AsyncClient) for API tests in backend Python tests, as defined in 'src/backend/tests/conftest.py'.
When writing component tests, inherit from the appropriate base class in 'src/backend/tests/base.py' (ComponentTestBase, ComponentTestBaseWithClient, or ComponentTestBaseWithoutClient) and provide the required fixtures: 'component_class', 'default_kwargs', and 'file_names_mapping'.
Each test in backend Python test files should have a clear docstring explaining its purpose, and complex setups or mocks should be well-commented.
Test both sync and async code paths in backend Python tests, using '@pytest.mark.asyncio' for async tests.
Mock external dependencies appropriately in backend Python tests to isolate unit tests from external services.
Test error handling and edge cases in backend Python tests, including using 'pytest.raises' and asserting error messages.
Validate input/output behavior and test component initialization and configuration in backend Python tests.
Use the 'no_blockbuster' pytest marker to skip the blockbuster plugin in tests when necessary.
Be aware of ContextVar propagation in async tests; test both direct event loop execution and 'asyncio.to_thread' scenarios to ensure proper context isolation.
Test error handling by mocking internal functions using monkeypatch in backend Python tests.
Test resource cleanup in backend Python tests by using fixtures that ensure proper initialization and cleanup of resources.
Test timeout and performance constraints in backend Python tests using 'asyncio.wait_for' and timing assertions.
Test Langflow's Messag...
Files:
- src/backend/tests/unit/components/agents/test_multimodal_agent.py
- src/backend/tests/unit/components/agents/test_agent_events.py
src/backend/**/components/**/*.py
📄 CodeRabbit inference engine (.cursor/rules/icons.mdc)
In your Python component class, set the
iconattribute to a string matching the frontend icon mapping exactly (case-sensitive).
Files:
- src/backend/tests/unit/components/agents/test_multimodal_agent.py
- src/backend/tests/unit/components/agents/test_agent_events.py
**/{test_*.py,*.test.ts,*.test.tsx}
📄 CodeRabbit inference engine (coderabbit-custom-pre-merge-checks-unique-id-file-non-traceable-F7F2B60C-1728-4C9A-8889-4F2235E186CA.txt)
**/{test_*.py,*.test.ts,*.test.tsx}: Check if tests have too many mock objects that obscure what’s actually being tested
Warn when mocks are used instead of testing real behavior and interactions
Suggest using real objects or simpler test doubles when mocks become excessive
Ensure mocks are used only for external dependencies, not core business logic
Recommend integration tests when unit tests become overly mocked
Check that test files follow the project’s naming conventions (backend: test_*.py; frontend: *.test.ts/tsx)
Verify that tests actually exercise the new or changed functionality, not placeholder assertions
Test files should have descriptive test function names explaining what is being tested
Organize tests logically with proper setup and teardown
Include edge cases and error conditions for comprehensive coverage
Verify tests cover both positive (success) and negative (failure) scenarios
Ensure tests are not mere smoke tests; they should validate behavior thoroughly
Ensure tests follow the project’s testing frameworks (pytest for backend, Playwright for frontend)
Files:
- src/backend/tests/unit/components/agents/test_multimodal_agent.py
- src/backend/tests/unit/components/agents/test_agent_events.py
**/test_*.py
📄 CodeRabbit inference engine (coderabbit-custom-pre-merge-checks-unique-id-file-non-traceable-F7F2B60C-1728-4C9A-8889-4F2235E186CA.txt)
**/test_*.py: Backend tests must be named test_*.py and use proper pytest structure (fixtures, assertions)
For async backend code, use proper pytest async patterns (e.g., pytest-asyncio)
For API endpoints, include tests for both success and error responses
Files:
- src/backend/tests/unit/components/agents/test_multimodal_agent.py
- src/backend/tests/unit/components/agents/test_agent_events.py
🧬 Code graph analysis (4)
src/backend/tests/unit/components/agents/test_multimodal_agent.py (2)
src/lfx/src/lfx/base/agents/agent.py (1)
LCAgentComponent(35-278)src/lfx/src/lfx/schema/message.py (1)
Message(34-299)
src/lfx/src/lfx/base/agents/events.py (2)
src/lfx/src/lfx/schema/log.py (2)
OnTokenFunctionType(41-44)
SendMessageFunctionType(22-38)src/lfx/src/lfx/schema/message.py (1)
Message(34-299)
src/backend/tests/unit/components/agents/test_agent_events.py (3)
src/lfx/src/lfx/base/agents/events.py (4)
handle_on_chain_start(56-93)
handle_on_chain_end(150-182)
handle_on_chain_stream(292-329)
process_agent_events(374-428)src/backend/tests/unit/components/agents/test_multimodal_agent.py (1)
send_message(24-25)src/lfx/src/lfx/custom/custom_component/component.py (1)
send_message(1552-1603)
src/lfx/src/lfx/base/agents/agent.py (4)
src/backend/base/langflow/memory.py (1)
messages(340-345)src/lfx/src/lfx/custom/custom_component/component.py (1)
log(1480-1497)src/lfx/src/lfx/schema/message.py (1)
Message(34-299)src/lfx/src/lfx/schema/data.py (1)
Data(26-288)
🪛 GitHub Actions: Ruff Style Check
src/backend/tests/unit/components/agents/test_multimodal_agent.py
[error] 119-119: Ruff: W292 No newline at end of file.
🪛 GitHub Check: Ruff Style Check (3.13)
src/backend/tests/unit/components/agents/test_multimodal_agent.py
[failure] 119-119: Ruff (W292)
src/backend/tests/unit/components/agents/test_multimodal_agent.py:119:39: W292 No newline at end of file
src/lfx/src/lfx/base/agents/events.py
[failure] 123-123: Ruff (E501)
src/lfx/src/lfx/base/agents/events.py:123:121: E501 Line too long (129 > 120)
[failure] 117-117: Ruff (W293)
src/lfx/src/lfx/base/agents/events.py:117:1: W293 Blank line contains whitespace
src/lfx/src/lfx/base/agents/agent.py
[failure] 28-28: Ruff (I001)
src/lfx/src/lfx/base/agents/agent.py:28:5: I001 Import block is un-sorted or un-formatted
[failure] 195-195: Ruff (F821)
src/lfx/src/lfx/base/agents/agent.py:195:52: F821 Undefined name input_text
[failure] 195-195: Ruff (F821)
src/lfx/src/lfx/base/agents/agent.py:195:27: F821 Undefined name input_text
[failure] 178-178: Ruff (W291)
src/lfx/src/lfx/base/agents/agent.py:178:97: W291 Trailing whitespace
[failure] 175-175: Ruff (W293)
src/lfx/src/lfx/base/agents/agent.py:175:1: W293 Blank line contains whitespace
[failure] 167-167: Ruff (W293)
src/lfx/src/lfx/base/agents/agent.py:167:1: W293 Blank line contains whitespace
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (8)
- GitHub Check: Run Backend Tests / Unit Tests - Python 3.10 - Group 4
- GitHub Check: Run Backend Tests / Unit Tests - Python 3.10 - Group 2
- GitHub Check: Test Docker Images / Test docker images
- GitHub Check: Run Backend Tests / Unit Tests - Python 3.10 - Group 5
- GitHub Check: Run Backend Tests / Unit Tests - Python 3.10 - Group 3
- GitHub Check: Run Backend Tests / Unit Tests - Python 3.10 - Group 1
- GitHub Check: Run Backend Tests / Integration Tests - Python 3.10
- GitHub Check: Test Starter Templates
🔇 Additional comments (5)
src/frontend/src/stores/messagesStore.ts (2)
72-72: LGTM: Safe text concatenation.The nullish coalescing ensures that text concatenation works even when the current text is undefined or null, preventing potential runtime errors during streaming updates.
21-31: Frontend streaming logic is correct; verify backend integration to confirm final messages include complete text.Based on verification of the codebase:
The
updateMessagePartialmethod uses spread operators ({ ...updatedMessages[i], ...message }) which correctly allow the final message to overwrite accumulated properties. The backend ensures that whenstatetransitions from"partial"to"complete", the final message includes the complete accumulated text extracted via_extract_output_text()before sending (seeevents.pylines 161–165 and 302–307).The frontend logic correctly:
- Accumulates partial message text incrementally (line 24:
get().updateMessageText(...))- Removes the text property from partials before updating other fields (line 26:
const { text, ...messageWithoutText })- Replaces the entire message on completion (line 29:
get().updateMessagePartial(message)with full object)This design pattern properly mirrors the backend's behavior where streaming chunks are processed incrementally, then a complete final message containing the full accumulated output replaces all previous state.
src/backend/tests/unit/components/agents/test_multimodal_agent.py (1)
1-29: Consider adding a flow-level integration test.Per guidelines, include at least one create_flow/build_flow/get_build_events-based test that exercises multimodal input end-to-end through the agent.
src/lfx/src/lfx/base/agents/agent.py (2)
167-179: Clean up whitespace-only lines at 167, 175, and 178 (verified Ruff W293 violations detected).Run
make format_backendto resolve.
22-29: Remove duplicate OnTokenFunctionType from TYPE_CHECKING block.OnTokenFunctionType is correctly imported at runtime on line 22 but incorrectly duplicated in the TYPE_CHECKING block on line 28. Since it's used at runtime (lines 234–236), it belongs only in the runtime import. The TYPE_CHECKING block should contain only SendMessageFunctionType.
The proposed diff correctly removes the duplicate:
-if TYPE_CHECKING: - from lfx.schema.log import SendMessageFunctionType, OnTokenFunctionType +if TYPE_CHECKING: + from lfx.schema.log import SendMessageFunctionType
Ported over from release branch - #10216
Summary by CodeRabbit