v1.77.7.dev2

Full Changelog: v1.77.7-stable...v1.77.7.dev2

v1.79.0.rc.1

What's Changed

• [Bug Fix]: ErrorEvent ValidationError when OpenAI Responses API returns nested error structure by @ishaan-jaff in #15804
• [Fix] Azure AI Speech - Ensure voice is mapped from request body -> SSML body , allow sending role and style by @ishaan-jaff in #15810
• Use proper bedrock model name in health check by @nuernber in #15808
• [Feat] Add DataforSEO Search API by @ishaan-jaff in #15817
• [Feat] Add Google PSE Search Provider by @ishaan-jaff in #15816
• [Feat] Add cost tracking for Search API requests - Google PSE, Tavily, Parallel AI, Exa AI by @ishaan-jaff in #15821
• (feat) Vector Stores: support Vertex AI Search API as vector store through LiteLLM by @krrishdholakia in #15781
• fix(responses-api): simplify reasoning item handling for gpt-5-codex by @AlexsanderHamir in #15815
• fix: replace deprecated gemini-1.5-pro-preview-0514 by @AlexsanderHamir in #15852
• [Bug Fix]: Hooks broken on /bedrock passthrough due to missing metadata by @ishaan-jaff in #15849
• [Bug Fix] Exa Search API - ensure request params are sent to Exa AI by @ishaan-jaff in #15855
• fix(vertex-ai): cost tracking for search spend by @mythral in #15859
• fix(ui/): fix routing for custom server root path by @krrishdholakia in #15701
• [fix] Pass user-defined headers and extra_headers to image-edit calls by @byrongrogan in #15811
• [Feat] - [Backend] Search APIs - Allow storing configured Search APIs in DB by @ishaan-jaff in #15862
• [Feat] UI - Search Tools, allow adding search tools on UI + testing search by @ishaan-jaff in #15871
• [Feat] UI - Add logos for search providers by @ishaan-jaff in #15872
• docs: add responses api on openai docs by @mubashirosmani in #15866
• docs: add OpenAI responses api by @mubashirosmani in #15868
• feat(proxy): support absolute RPM/TPM in priority_reservation by @AlexsanderHamir in #15813
• Add documentation for enable_caching_on_provider_specific_optional_params setting by @Sameerlite in #15885
• Implement Bedrock Guardrail apply_guardrail endpoint support by @Sameerlite in #15892
• Lasso Security Guardrail: Add v3 API Support by @oroxenberg in #12452
• Litellm sameer oct staging by @Sameerlite in #15806
• fix: minimize the occurrence of deadlocks by @CAFxX in #15281
• Fix MLFlow tags - split request_tags into (key, val) if request_tag has colon by @reflection in #15914
• Add mistral medium 3 and Codestral 2 on vertex by @superpoussin22 in #15887
• fixed lasso import config, redis cluster hash tags for test keys by @shadielfares in #15917
• update vertex ai gemini costs by @otaviofbrito in #15911
• [Feat] New Guardrail - Dynamo AI Guardrail by @ishaan-jaff in #15920
• (feat) Organizations: allow org admins to create teams on UI + (feat) IBM Guardrails by @krrishdholakia in #15924
• fix(managed_files.py): don't raise error if managed object is not found + (Feat) Azure AI - Search Vector Stores + (Fix) Batches - “User default_user_id does not have access to the object” when object not in db + (fix) Vector Stores - show config.yaml vector stores on UI by @krrishdholakia in #15873
• Responses API - support tags in metadata by @krrishdholakia in #15867
• VertexAI Search Vector Store - Passthrough endpoint support + Vector store search Cost tracking support by @krrishdholakia in #15824
• Guardrails - Responses API, Image Gen, Text completions, Audio transcriptions, Audio Speech, Rerank, Anthropic Messages API support via the unified apply_guardrails function by @krrishdholakia in #15706
• Org level tpm/rpm limits + Team tpm/rpm validation when assigned to org by @krrishdholakia in #15549
• (security) Responses API - prevent User A from retrieving User B's response, if response.id is leaked by @krrishdholakia in #15757
• fix(opentelemetry.py): fix issue where headers were not being split correctly + feat(bedrock): add titan image generations w/ cost tracking by @krrishdholakia in #15916

New Contributors

• @mythral made their first contribution in #15859
• @mubashirosmani made their first contribution in #15866
• @CAFxX made their first contribution in #15281
• @reflection made their first contribution in #15914
• @shadielfares made their first contribution in #15917

Full Changelog: v1.78.7-nightly...v1.79.0.rc.1

v1.79.0-nightly

What's Changed

• [Bug Fix]: ErrorEvent ValidationError when OpenAI Responses API returns nested error structure by @ishaan-jaff in #15804
• [Fix] Azure AI Speech - Ensure voice is mapped from request body -> SSML body , allow sending role and style by @ishaan-jaff in #15810
• Use proper bedrock model name in health check by @nuernber in #15808
• [Feat] Add DataforSEO Search API by @ishaan-jaff in #15817
• [Feat] Add Google PSE Search Provider by @ishaan-jaff in #15816
• [Feat] Add cost tracking for Search API requests - Google PSE, Tavily, Parallel AI, Exa AI by @ishaan-jaff in #15821
• (feat) Vector Stores: support Vertex AI Search API as vector store through LiteLLM by @krrishdholakia in #15781
• fix(responses-api): simplify reasoning item handling for gpt-5-codex by @AlexsanderHamir in #15815
• fix: replace deprecated gemini-1.5-pro-preview-0514 by @AlexsanderHamir in #15852
• [Bug Fix]: Hooks broken on /bedrock passthrough due to missing metadata by @ishaan-jaff in #15849
• [Bug Fix] Exa Search API - ensure request params are sent to Exa AI by @ishaan-jaff in #15855
• fix(vertex-ai): cost tracking for search spend by @mythral in #15859
• fix(ui/): fix routing for custom server root path by @krrishdholakia in #15701
• [fix] Pass user-defined headers and extra_headers to image-edit calls by @byrongrogan in #15811
• [Feat] - [Backend] Search APIs - Allow storing configured Search APIs in DB by @ishaan-jaff in #15862
• [Feat] UI - Search Tools, allow adding search tools on UI + testing search by @ishaan-jaff in #15871
• [Feat] UI - Add logos for search providers by @ishaan-jaff in #15872
• docs: add responses api on openai docs by @mubashirosmani in #15866
• docs: add OpenAI responses api by @mubashirosmani in #15868
• feat(proxy): support absolute RPM/TPM in priority_reservation by @AlexsanderHamir in #15813
• Add documentation for enable_caching_on_provider_specific_optional_params setting by @Sameerlite in #15885
• Implement Bedrock Guardrail apply_guardrail endpoint support by @Sameerlite in #15892
• Lasso Security Guardrail: Add v3 API Support by @oroxenberg in #12452
• Litellm sameer oct staging by @Sameerlite in #15806
• fix: minimize the occurrence of deadlocks by @CAFxX in #15281
• Fix MLFlow tags - split request_tags into (key, val) if request_tag has colon by @reflection in #15914
• Add mistral medium 3 and Codestral 2 on vertex by @superpoussin22 in #15887
• fixed lasso import config, redis cluster hash tags for test keys by @shadielfares in #15917
• update vertex ai gemini costs by @otaviofbrito in #15911
• [Feat] New Guardrail - Dynamo AI Guardrail by @ishaan-jaff in #15920
• (feat) Organizations: allow org admins to create teams on UI + (feat) IBM Guardrails by @krrishdholakia in #15924
• fix(managed_files.py): don't raise error if managed object is not found + (Feat) Azure AI - Search Vector Stores + (Fix) Batches - “User default_user_id does not have access to the object” when object not in db + (fix) Vector Stores - show config.yaml vector stores on UI by @krrishdholakia in #15873
• Responses API - support tags in metadata by @krrishdholakia in #15867
• VertexAI Search Vector Store - Passthrough endpoint support + Vector store search Cost tracking support by @krrishdholakia in #15824
• Guardrails - Responses API, Image Gen, Text completions, Audio transcriptions, Audio Speech, Rerank, Anthropic Messages API support via the unified apply_guardrails function by @krrishdholakia in #15706
• Org level tpm/rpm limits + Team tpm/rpm validation when assigned to org by @krrishdholakia in #15549
• (security) Responses API - prevent User A from retrieving User B's response, if response.id is leaked by @krrishdholakia in #15757
• fix(opentelemetry.py): fix issue where headers were not being split correctly + feat(bedrock): add titan image generations w/ cost tracking by @krrishdholakia in #15916

New Contributors

• @mythral made their first contribution in #15859
• @mubashirosmani made their first contribution in #15866
• @CAFxX made their first contribution in #15281
• @reflection made their first contribution in #15914
• @shadielfares made their first contribution in #15917

Full Changelog: v1.78.7-nightly...v1.79.0-nightly

v1.78.7-nightly

What's Changed

• fix: Add response_type + PKCE parameters to OAuth authorization endpoint by @talalryz in #15720
• feat: add GraySwan Guardrails support by @uc4w6c in #15756
• [Docs] Add Azure AI - OCR to docs by @ishaan-jaff in #15768
• Auth Header Fix for MCP Tool Call by @1vinodsingh1 in #15736
• chore: rename GraySwan to Gray Swan by @uc4w6c in #15771
• Add AWS us-gov-west-1 Claude 3.7 Sonnet costs by @nuernber in #15775
• feat(ollama): set 'think' to False when reasoning effort is minimal/none/disable by @kowyo in #15763
• [Feat] Add SENTRY_ENVIRONMENT configuration for Sentry integration by @Thomas-Mildner in #15760
• Add details in docs by @javiergarciapleo in #15721
• [Feat ] /ocr - Add mode + Health check support for OCR models by @ishaan-jaff in #15767
• [Feat] Add def search() APIs for Web Search - Perplexity API by @ishaan-jaff in #15769
• fix(ollama): handle parsing ollama chunk error for issue #13333 by @lshgdut in #15717
• [Feat] Add Tavily Search API by @ishaan-jaff in #15770
• [Feat] Add Parallel AI - Search API by @ishaan-jaff in #15772
• [Feat] Add EXA AI Search API to LiteLLM by @ishaan-jaff in #15774
• feat: Add imageConfig parameter for gemini-2.5-flash-image by @kk-wangjifeng in #15530
• [Feat] Add /search endpoint on LiteLLM Gateway by @ishaan-jaff in #15780
• (feat) Passthrough - set auth on passthrough endpoints, on the UI by @krrishdholakia in #15778
• Fix reasoning item ID auto-generation causing encrypted content verification errors by @Sameerlite in #15782
• Feat: Allow prompt caching to be used for Anthropic Claude on Databricks by @anthonyivn2 in #15801
• fix the date for sonnet 3.7 in govcloud by @nuernber in #15800
• fix: Rename configured_cold_storage_logger to cold_storage_custom_logger by @hula-la in #15798
• fix: Apply max_connections configuration to Redis async client by @hula-la in #15797
• Support for embeddings_by_type Response Format in Bedrock Cohere Embed v1 by @romanglo in #15707
• Fix pass-through endpoint budget enforcement bug by @Sameerlite in #15805

New Contributors

• @talalryz made their first contribution in #15720
• @1vinodsingh1 made their first contribution in #15736
• @Thomas-Mildner made their first contribution in #15760
• @javiergarciapleo made their first contribution in #15721
• @lshgdut made their first contribution in #15717
• @kk-wangjifeng made their first contribution in #15530
• @anthonyivn2 made their first contribution in #15801
• @romanglo made their first contribution in #15707

Full Changelog: v1.78.6-nightly...v1.78.7-nightly

v1.78.6-nightly

What's Changed

• [Fix] Anthropic cache_control incorrectly applied to all content items instead of last item only by @ishaan-jaff in #15699
• [fix] - docs : update worker recommendation by @AlexsanderHamir in #15702
• [Fix] Forward anthropic-beta headers to Bedrock, VertexAI by @ishaan-jaff in #15700
• fix: Add pre and post call for list batches by @LucasSugi in #15673
• fix: Add function responsible to call precall by @LucasSugi in #15636
• Fix: Add gpt 4.1 pricing for response endpoint by @jlan-nl in #15593
• feat: read from custom-llm-provider header by @tlecomte in #15528
• fix the wrong request body in json mode doc by @Sameerlite in #15729
• (Bug) Fix JSON serialization error in Helicone logging by removing OpenTelemetry span from metadata by @Sameerlite in #15728
• [Fix] OpenAI Realtime API integration fails due to websockets.exceptions.PayloadTooBig error by @ishaan-jaff in #15751
• Change max_tokens value to match max_output_tokens for claude sonnet … by @akraines in #15715
• [Feat] Add Azure AVA TTS integration by @ishaan-jaff in #15749
• [Bug]: Fix Incorrect status value in responses api with gemini by @ishaan-jaff in #15753
• [Feat] Add Azure AVA (Speech AI) Cost Tracking by @ishaan-jaff in #15754

• Fix OpenTelemetry Logging functionality by @tomhaynes in #15645

New Contributors

• @tlecomte made their first contribution in #15528
• @tomhaynes made their first contribution in #15645

Full Changelog: v1.78.5-nightly...v1.78.6-nightly

v1.78.5.rc.1

What's Changed

• (feat) Team level model-specific tpm/rpm limits + working key-level validation of tpm/rpm limit when assigned to team by @krrishdholakia in #15513
• [Fix] Memory Leak - Ensure pass through routes only added when path does not exist by @ishaan-jaff in #15697
• [Docs] v1.78.5 notes by @ishaan-jaff in #15698
• fix(proxy_server.py): re-encrypt env var on config save + use original value on decrypt error by @krrishdholakia in #15671
• [Fix] Support service_tier in chat completion by @eycjur in #15693
• fix: add imagePullSecrets to migrations-job by @uc4w6c in #15681
• feat(guardrails): Add content masking and streaming support to PANW Prisma AIRS guardrail by @jroberts2600 in #15666

Full Changelog: v1.78.4.dev1...v1.78.5.rc.1

v1.78.5-stable

What's Changed

• (feat) Team level model-specific tpm/rpm limits + working key-level validation of tpm/rpm limit when assigned to team by @krrishdholakia in #15513
• [Fix] Memory Leak - Ensure pass through routes only added when path does not exist by @ishaan-jaff in #15697
• [Docs] v1.78.5 notes by @ishaan-jaff in #15698
• fix(proxy_server.py): re-encrypt env var on config save + use original value on decrypt error by @krrishdholakia in #15671
• [Fix] Support service_tier in chat completion by @eycjur in #15693
• fix: add imagePullSecrets to migrations-job by @uc4w6c in #15681
• feat(guardrails): Add content masking and streaming support to PANW Prisma AIRS guardrail by @jroberts2600 in #15666

Full Changelog: v1.78.4.dev1...v1.78.5-stable

v1.78.5-nightly

What's Changed

• (feat) Team level model-specific tpm/rpm limits + working key-level validation of tpm/rpm limit when assigned to team by @krrishdholakia in #15513
• [Fix] Memory Leak - Ensure pass through routes only added when path does not exist by @ishaan-jaff in #15697
• [Docs] v1.78.5 notes by @ishaan-jaff in #15698
• fix(proxy_server.py): re-encrypt env var on config save + use original value on decrypt error by @krrishdholakia in #15671
• [Fix] Support service_tier in chat completion by @eycjur in #15693
• fix: add imagePullSecrets to migrations-job by @uc4w6c in #15681
• feat(guardrails): Add content masking and streaming support to PANW Prisma AIRS guardrail by @jroberts2600 in #15666

Full Changelog: v1.78.4.dev1...v1.78.5-nightly

v1.78.4.dev1

What's Changed

• fix: bedrock-pricing-geo-inregion-cross-region / add Global Cross-Region Inference by @uc4w6c in #15685
• docs: improve budget clarity by @TeddyAmkie in #15682
• fix(token-counter): extract model_info from deployment for custom_tokenizer (#15657) by @Nikro in #15680
• litellm_Key Settings Max Budget Removal Error Fix by @yuneng-jiang in #15669
• Bedrock + MCP - working MCP calls to bedrock via Responses API + Log hidden params for OTEL calls by @krrishdholakia in #15677
• [Perf] Alexsander fixes round 2 - Oct 18th by @ishaan-jaff in #15695

New Contributors

• @Nikro made their first contribution in #15680

Full Changelog: v1.78.4-nightly...v1.78.4.dev1

v1.78.4-nightly

What's Changed

• [Fix] UI - Key Max Budget Removal Error Fix by @yuneng-jiang in #15672
• [Fix] GPT-OSS in Bedrock now supports streaming. Revert fake streaming by @ishaan-jaff in #15668
• [Fix (pricing)] - Fix pricing for watsonx model family for various models by @ishaan-jaff in #15670
• Fix: Gemini 2.5 Flash Image should not have supports_web_search=true by @jlan-nl in #15642
• Fix: Support us-gov prefix for AWS GovCloud Bedrock models by @ishaan-jaff in #15626
• [Bug Fix] Tags as metadata dicts were raising exceptions by @ishaan-jaff in #15625
• [Feat] Add Cost Tracking for /ocr endpoints by @ishaan-jaff in #15678
• fix: add missing context to benchmark docs by @AlexsanderHamir in #15688
• Fix: Separate OAuth M2M authentication from UI SSO + Handle Introspection endpoint for Oauth2 by @ishaan-jaff in #15667
• [Oct Staging Branch] by @ishaan-jaff in #15460
• [Feat] Add Guardrails for /v1/messages and /v1/responses API by @ishaan-jaff in #15686
• Add glm-4.6 model to pricing configuration by @Copilot in #15679

Full Changelog: v1.78.3-nightly...v1.78.4-nightly

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.78.4-nightly

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat