Skip to content

[Bug]: Model group weight: 0 config spills over into other groups #13878

@TeddyAmkie

Description

@TeddyAmkie

What happened?

Description

We’ve observed unexpected behavior when setting weight: 0 for a deployment in one model group.

Expected behavior

Setting weight: 0 should only remove that deployment from load balancing within that model group.
It should not impact other groups or direct calls to that same provider model elsewhere.

Actual behavior

  • When a deployment is configured with weight: 0 (e.g. Bedrock Sonnet 4), it not only triggered a division by zero crash (now patched by #13624) but also appeared to affect calls in other model groups that referenced the same provider model.
  • This makes direct calls to bedrock/converse/... unreliable if any weight: 0 entries exist in other groups.
  • In practice, config for one group unexpectedly influences behavior of other groups.

Example Config

- model_name: -multi-provider-claude-sonnet-4-20250514
  litellm_params:
    model: anthropic/claude-sonnet-4-20250514
    api_key: os.environ/ANTHROPIC_API_KEY
    weight: 85
- model_name:-multi-provider-claude-sonnet-4-20250514
  litellm_params:
    model: bedrock/converse/us.anthropic.claude-sonnet-4-20250514-v1:0
    weight: 0   # problematic entry
- model_name: -multi-provider-claude-sonnet-4-20250514
  litellm_params:
    model: vertex_ai/claude-sonnet-4@20250514
    vertex_project: os.environ/VERTEX_PROJECT
    vertex_location: us-east5
    vertex_credentials: os.environ/VERTEX_SERVICE_ACCOUNT
    weight: 15


### Relevant log output

```shell

Are you a ML Ops Team?

No

What LiteLLM version are you on ?

v1.75

Twitter / LinkedIn details

No response

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions