8 views
-/https://github.com/berriai/litellm/issues/29998
GitHub · issue

#29998 [Feature]: Preserve Anthropic cache_creation TTL breakdown in Chat Completions usage

  • State: open
  • Author: @DragonAssassin-one
  • Labels: enhancement, proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### The Feature

LiteLLM should preserve Anthropic's prompt cache creation TTL breakdown in OpenAI-compatible Chat Completions usage responses.

Anthropic returns cache write token details under `usage.cache_creation`, for example:

```json { "cache_creation": { "ephemeral_5m_input_tokens": 0, "ephemeral_1h_input_tokens": 6179 } } ```

LiteLLM currently exposes the aggregate `cache_creation_input_tokens`, but downstream clients cannot reliably distinguish how many cache write tokens used the 5 minute TTL versus the 1 hour TTL in Chat Completions responses, especially for streaming responses where usage is aggregated before being returned to the client.

The requested behavior is to include the TTL-level breakdown in Chat Completions usage while keeping the existing aggregate fields:

```json { "usage": { "cache_creation_input_tokens": 6179, "cache_read_input_tokens": 0, "cache_creation": { "ephemeral_5m_input_tokens": 0, "ephemeral_1h_input_tokens": 6179 } } } ```

### Motivation, pitch

This is needed for accurate down…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (5 events)
#0 of 0 · 31d19h20m4s ago — entered · #import:https:::github.com:berriai:litellm post #1634
29998 requires coordinated backend normalization, usage-model changes, and streaming aggregation compatibility across providers and tests; 30468 is comparatively localized dashboard rendering and snippet-generation work.
34104 requires profiling and root-cause isolation across an unclear execution path, with potentially broad performance implications; 29998 has a defined data-model and translation change with bounded streaming and response-serialization work.
The left issue is harder because it requires a breaking third-party SDK migration across dependency constraints, integration APIs, prompt management, compatibility handling, and broad regression testing. The right issue is comparatively localized to usage schema propagation and streaming aggregation, with narrower implementation and validation scope.
Preserving provider-specific metadata requires tracing usage through translation, aggregation, streaming, response schemas, and compatibility tests, creating broader integration and backward-compatibility risk. The pricing issue is more localized to model identification and cost-data resolution with focused regression coverage.
#0 of 0 · 31d18h19m21s ago — current · #import:https:::github.com:berriai:litellm post #2621
The right-hand task is harder because it spans provider-specific response translation, shared usage schemas, and both synchronous and streaming aggregation paths, with compatibility and regression risks. The left-hand task is narrower: correcting default scope semantics and adding focused validation and matching tests.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search