-
-
Notifications
You must be signed in to change notification settings - Fork 4.9k
Description
What happened?
Mismatch litellm UI vs prometheus metrics on litellm_spend_metric_total and also on litellm_request_total_latency_metric_count (total number of requests) when using bedrock models via claude code, while all other models & keys track correctly (like when suing curl requests).
-
UI shows $3–4 of spend for that API key on the same day.
-
Prometheus /metrics only shows very small values (e.g. 0.002), and increase(litellm_spend_metric_total[1d]) returns 0.
-
This mismatch only happens for Bedrock model calls coming from Claude Code via litellm api key.
ENV
Litellm version = "0.1.815"
Prometheus callback enabled: Yes
Backend provider: AWS Bedrock
Prometheus scraping / Grafana queries are confirmed working correctly for other keys and models when not using claude code.
To reproduce:
- Configure LiteLLM with Bedrock models + Prometheus callback.
- Create a API key
- Use Claude Code to send several requests to a Bedrock Claude model via LiteLLM.
- LiteLLM UI for the 'key'($3.69) → Spend per day differs from Prometheus endpoint ($0.002) increase(litellm_spend_metric_total{api_key_alias="key"}[1d])
- Repeat the same process and use a curl request and then the Promethus spend matches perfectly.
- We can also observe that the total request counter is also having a mismatch when using bedrock models + claude code but matches perfectly when using a curl request.
Expected Behavior
For all models (including Bedrock Claude), spend should be counted identically in:
LiteLLM UI (DB-backed spend logs)
Prometheus metrics (litellm_spend_metric_total)
Impact
Grafana dashboards under-report cost for teams using Claude Code.
Please investigate whether:
Bedrock Claude requests triggered via Claude Code skip or partially skip Prometheus spend metric increments. Please let us know if you need any more information.
Relevant log output
Are you a ML Ops Team?
Yes
What LiteLLM version are you on ?
v0.1.815
Twitter / LinkedIn details
No response