[Bug]: Requests without clientside credentials may use other clients' API keys

[clarence-inventive]

Potential Security Issue: Requests without clientside credentials may use other clients' API keys

Description

When a request is made without clientside credentials (e.g., no api_key in request body), the router appears to sometimes select a deployment that was previously created with another client's clientside credentials, causing the request to use the wrong API key.

Steps to Reproduce

1. Configure a model with configurable_clientside_auth_params: ["api_key"] in the proxy config, as well as a default api_key for the model
2. Make Request 1 with api_key: "client-a-key" in the request body
   • This should create a deployment with client-a-key and store it in the router
3. Make Request 2 without any api_key in the request body
   • Expected: Should use the default api_key from config
   • Actual: Appears to use client-a-key from Request 1's deployment

Observed Behavior

When testing, I noticed that:

• After making a request with a clientside api_key, subsequent requests without api_key seem to use that same clientside api_key
• This happens even though the second request doesn't include any clientside credentials

Hypothesis

Looking at the code flow, here's what I think might be happening:

1. When Request 1 includes api_key: "client-a-key":

   • _handle_clientside_credential() creates a new deployment with that api_key
   • upsert_deployment() adds this deployment to the router's model_list
   • This deployment has original_model_id set to track the original config deployment

2. When Request 2 has no api_key:

   • is_clientside_credential() returns False (no clientside credentials in request)
   • _update_kwargs_with_deployment() doesn't call _handle_clientside_credential()
   • The router selects from all available deployments via _get_all_deployments()
   • Since the clientside-credential deployment was added to model_list, it's included in the selection pool
   • The router may select this deployment, which still contains client-a-key

The issue seems to be that deployments created with clientside credentials remain in the router's deployment pool and can be selected even when no clientside credentials are provided in the current request.

Security Impact

• Severity: High - Potential API key leakage between clients
• If confirmed, this would mean clients' API keys could be used by other clients' requests
• This would violate credential isolation expectations

Questions

• Is this the intended behavior, or is this a bug?
• Should deployments created with clientside credentials be filtered out when no clientside credentials are provided?
• Is there a mechanism to ensure requests without clientside credentials only use the original config deployments?

Related

• PR fix(route_llm_request.py): move to using common router, for client-side credentials #8966 introduced clientside credential support - this might be related to that implementation

Relevant log output

Are you a ML Ops Team?

No

What LiteLLM version are you on ?

v1.80.0

Twitter / LinkedIn details

No response