8 views
-/https://github.com/berriai/litellm/issues/35127
GitHub · issue

#35127 [Bug]: /v1/messages to an OpenAI Responses model drops cached tokens (reads Anthropic-only usage keys) — cache reads billed at the full input rate

  • State: open
  • Author: @sarveshb-p72
  • Labels: llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

Closely related but distinct: - #27763 (**closed, completed**) — the mirror-image defect: Anthropic-native `cache_read_input_tokens` not normalized into `prompt_tokens_details.cached_tokens` for Vertex/Bedrock on `/v1/messages`. Same end symptom ("cache reads billed as full-priced input tokens"), opposite direction of translation. This report is the **OpenAI-backed** path, which was not covered by that fix. - #11364 (`cached_tokens` not populated, Anthropic direct), #11789 (Anthropic streaming cost tracking ignores cache reads), #7790 (async logging callbacks drop cache fields when streaming).

### What happened?

When `/v1/messages` is routed to an OpenAI model (i.e. the Responses-API bridge, `_should_route_to_responses_api()` → `LiteLLMMessagesToResponsesAPIHandler` → `AnthropicResponsesStreamWrapper`), **cache-read tokens are always reported as 0**, even when OpenAI returns a large `cached_tokens` count.

Consequences:

1. The `message_delta` usage returned to the client omits `cache_read_input_tokens` entirely, so Anthropic-format clients (e.g. anything speaki…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (4 events)
#0 of 0 · 31d18h53m16s ago — entered · #import:https:::github.com:berriai:litellm post #1980
Model omitted braces; inferred difficulty from issue scope and surface area.
The left issue requires a broad security-sensitive integration spanning cryptographic signing, durable verification, policy enforcement, lifecycle hooks, and compliance-oriented validation. The right issue is comparatively localized to correcting usage-field translation and billing metadata across an existing API bridge.
The left issue is harder because it requires a security-sensitive hashing change, compatibility with existing credentials, migration or fallback behavior, and validation across FIPS and non-FIPS environments. The right issue is comparatively localized to usage-field translation and billing metadata tests.
#0 of 0 · 31d18h3m5s ago — current · #import:https:::github.com:berriai:litellm post #2801
The right issue is harder because it spans API translation, streaming usage normalization, cache accounting, and compatibility across response formats, requiring careful regression coverage. The left issue is comparatively contained to enriching metadata and request-tag propagation through logging.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search