Skip to content

[Chat][Store] Chat API & storage support for streaming #922

@marco-jouwweb

Description

@marco-jouwweb

Good morning,

I am creating an issue on this as I am about to implement this feature myself (today). I already talked through a proposal on Slack (see: https://symfony-devs.slack.com/archives/C09BAPA9BJP/p1763370944783239?thread_ts=1763369735.762229&cid=C09BAPA9BJP)

Proposed approach

  1. Add a finally {} block to StreamResult->getContent() with an optional callback.
  2. When the generator is exhausted, the callback fires and receives the fully accumulated message.
  3. Update the Chat API so that it defines this callback and passes it down to the StreamResult during streaming calls.
  4. This callback would then be responsible for writing the accumulated message to the StoreInterface once the stream completes.
  5. This approach preserves chunk-by-chunk yielding (so streaming behaviour stays intact) and updates the conversation store at the end.
  6. Ultimately this would allow the Chat API to support streaming while automatically persisting to the configured storage.

Limitations:
This callback functionality & SessionStorage will not work in unison. Due to the nature of how session/header lifecycles work in Symfony, it seems unavoidable that this will cause inherent issues with headers being already sent. This might be circumventable using low-level session functions, but this is very likely to have tons of side-effects in the future. Due to this, I plan to throw an error when it is attempted to combine streaming & SessionStorage .

Metadata

Metadata

Assignees

No one assigned

    Labels

    RFCRFC = Request For Comments (proposals about features that you want to be discussed)

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions