v1.31.14

What's Changed

• (feat) add groq/gemma-7b-it by @snekkenull in #2529

New Contributors

• @snekkenull made their first contribution in #2529

Full Changelog: v1.31.13...v1.31.14

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	88	97.8719389956135	1.5238205426081837	0.0	456	0	80.47983800003067	1042.1901939999998
/health/liveliness	Passed ✅	62	64.17129267745518	15.178054615189408	0.0	4542	0	59.447291000026325	1284.6712140000136
/health/readiness	Passed ✅	180.0	180.3285143601241	14.967526777065908	0.0	4479	0	123.18138000000545	1374.4275610000045
Aggregated	Passed ✅	90	120.690833738736	31.6694019348635	0.0	9477	0	59.447291000026325	1374.4275610000045

v1.31.13

What's Changed

• fix(utils.py): move to using litellm.modify_params to enable max output token trimming fix by @krrishdholakia in #2520
• (fix) multiplatform docker db builds by @ishaan-jaff in #2525

Full Changelog: v1.31.12...v1.31.13

v1.31.12

What's Changed

• (fix) run prisma generate in default dockerfile by @ishaan-jaff in #2519

Full Changelog: v1.31.10...v1.31.12

v1.31.10

Full Changelog: v1.31.9...v1.31.10

Stable Release: v1.30.2 👈 Recommended stable version of proxy.

v1.31.9

Stable Release: v1.30.2 👈 Recommended stable version of proxy.

What's Changed

• [Proxy] return 429 when no models available by @ishaan-jaff in #2493
• (fix) using enterprise folder on litellm Docker by @ishaan-jaff in #2501
• Add support for claude-3-haiku-20240307 (anthropic) by @bachya in #2505
• (feat) add claude-3-haiku by @ishaan-jaff in #2508
• fix(proxy_server.py): move to using UPDATE + SET for track_cost_callback by @krrishdholakia in #2506

New Contributors

• @bachya made their first contribution in #2505

Full Changelog: v1.31.8...v1.31.9

v1.31.8

What's Changed

• feat(prompt_injection_detection.py): support simple heuristic similarity check for prompt injection attacks by @krrishdholakia in #2498

Load Tests Run on every new release

Full Changelog: v1.31.7...v1.31.8

v1.31.7

What's Changed

• fix: add dependency for google-cloud-aiplatform by @tylerbrandt in #2470
• build(deps): bump orjson from 3.9.7 to 3.9.15 by @dependabot in #2204
• (fix) issue with using litellm enterprise on Admin UI by @ishaan-jaff in #2500

New Contributors

• @tylerbrandt made their first contribution in #2470

Full Changelog: v1.31.6...v1.31.7

v1.31.6

What's Changed

• [FEAT Cohere/command-r tool calling by @ishaan-jaff in https://github.com//pull/2479
• fix(anthropic.py): support claude-3 streaming with function calling by @krrishdholakia in #2472

Full Changelog: v1.31.5...v1.31.6

v1.31.5

What's Changed

• fix(proxy_server.py): cache master key check by @krrishdholakia in #2483
• fix(proxy_server.py): support cost tracking if general_settings is none by @krrishdholakia in #2382
• [Proxy-High Traffic] Fix using Langfuse in high traffic by @ishaan-jaff in #2484

Full Changelog: v1.31.4...v1.31.5

v1.31.4

[BETA] Thrilled to launch support for Cohere/Command-R on LiteLLM , LiteLLM Proxy Server 👉 Start here https://docs.litellm.ai/docs/providers/cohere

☎️ PR for using cohere tool calling in OpenAI format: #2479

⚡️ LiteLLM Proxy + @langfuse - High Traffic - support 80+/Requests per second with Proxy + Langfuse logging https://docs.litellm.ai/docs/proxy/logging

⚡️ New Models - Azure GPT-Instruct models https://docs.litellm.ai/docs/providers/azure#azure-instruct-models

🛠️ Fix for using DynamoDB + LiteLLM Virtual Keys

What's Changed

• (feat) support azure/gpt-instruct models by @ishaan-jaff in #2471
• [New-Model] Cohere/command-r by @ishaan-jaff in #2474
• (fix) patch dynamoDB team_model_alias bug by @ishaan-jaff in #2478
• fix(azure.py): support cost tracking for azure/dall-e-3 by @krrishdholakia in #2475
• fix(openai.py): return model name with custom llm provider for openai-compatible endpoints (e.g. mistral, together ai, etc.) by @krrishdholakia in #2473

Full Changelog: v1.30.2...v1.31.4