6 views
-/https://github.com/berriai/litellm/issues/28354
GitHub · issue

#28354 [Bug]: Anthropic→Responses streaming adapter never extracts cache_read_input_tokens (always 0)

  • State: open
  • Author: @nbyula-engineering
  • Labels: llm translation, claude code

## Bug Description

When clients call `/v1/messages` (Anthropic format) and LiteLLM bridges to an OpenAI Responses API model (`mode: "responses"`), the streaming response emitted back to the client always reports `cache_read_input_tokens = 0` — even when OpenAI's underlying response correctly reports thousands of cached prompt tokens.

This breaks observability for any Anthropic-compatible client (Claude Code, Claude Agent SDK, etc.) routing OpenAI traffic through LiteLLM: dashboards, Sentry, Langfuse, billing readouts all see 0 cache reads and conclude "OpenAI prompt caching is broken" — when in fact only the *reporting* is broken.

## Root Cause

`litellm/llms/anthropic/experimental_pass_through/responses_adapters/streaming_iterator.py`, lines 264–270 in `AnthropicResponsesStreamWrapper._process_event`, on the `response.completed` event:

```python # Prefer direct cache fields if present cache_creation_tokens = int( getattr(usage, "cache_creation_input_tokens", 0) or 0 ) cache_read_tokens = int( getattr(usage, "cache_read_input_tokens", 0) or 0 ) ```

The translator reads `usage.cache_read_input_tokens` — an **Anthropic-only field name**. The OpenAI Responses API `usage` …

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d19h11m16s ago — entered · #import:https:::github.com:berriai:litellm post #1686
The right issue is harder because it crosses shared token-counting interfaces, provider-specific plumbing, request-shape compatibility, and regression testing across integrations. The left is comparatively localized to one streaming translation path with focused field mapping and event-level tests.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search