Skip to content

Conversation

@galkleinman
Copy link

@galkleinman galkleinman commented Nov 26, 2025

When using ThreadPoolExecutor to run LLM calls concurrently with
guardrails, contextvars were not being propagated to the worker thread.
This caused issues with trace context and other context-dependent
functionality being lost across thread boundaries.

The issue broke the traces of: https://github.com/traceloop/openllmetry

Changes:

  • Use copy_context() before submitting tasks to ThreadPoolExecutor
  • Apply fix to chat.completions.create, responses.create, and responses.parse
  • Add tests to verify contextvars propagate correctly across threads
  • Add tests for single and multiple contextvars propagation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant