6 views
-/https://github.com/berriai/litellm/issues/31594
GitHub Β· issue

#31594 [Bug]: cost_breakdown missing cache_read_cost/cache_creation_cost for DeepSeek / OpenAI-compatible

  • State: open
  • Author: @erdjiel
  • Labels: bug, proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

When prompting DeepSeek **directly**, the API response includes cache token information in `usage.prompt_tokens_details.cached_tokens`. For example, a cache-hit response from DeepSeek looks like:

```json { "usage": { "prompt_tokens": 136153, "completion_tokens": 418, "prompt_tokens_details": { "cached_tokens": 135936 } } } ```

When routing the **same request through LiteLLM Proxy**, the spend log's `cost_breakdown` only shows `input_cost`, `output_cost`, and `total_cost` β€” **no `cache_read_cost` field appears**, even though the proxy already applied the correct cache discount to the total. The breakdown is opaque: there's no way to see how much of `input_cost` came from cache hits vs cache misses.

The `completion_cost()` function only checks `usage.cache_read_input_tokens` (Anthropic convention). DeepSeek (OpenAI-compatible) returns cache tokens as `usage.prompt_tokens_details.cached_tokens`. The `Usage` Pydantic model does **not** have a direct `cache_read_input_tokens` field β€” so `getattr(obj, "cache_read_input_to…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 Β· 31d18h24m23s ago β€” entered Β· #import:https:::github.com:berriai:litellm post #2432
Cancellation-safe persistence spans asynchronous task lifecycle, queue locking, failure recovery, shutdown behavior, and race-condition testing; the other is a narrower usage-normalization and accounting change across provider schemas.
#0 of 0 Β· 31d18h23m31s ago β€” current Β· #import:https:::github.com:berriai:litellm post #2446
31594 requires changes across core cost accounting, usage normalization, provider-specific handling, and regression coverage, with greater compatibility and financial-correctness risk. 33007 is comparatively contained UI/link-generation and metadata handling work.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search