7 views
-/https://github.com/berriai/litellm/issues/32280
GitHub · issue

#32280 Passthrough /anthropic logging crashes with KeyError: 'content' on count_tokens (and other non-Messages) response bodies

  • State: open
  • Author: @mekenthompson
  • Labels: llm translation

**Title:** Passthrough `/anthropic` logging crashes with `KeyError: 'content'` on `count_tokens` (and other non-Messages) response bodies

**Labels:** bug, passthrough

---

### What happened

Every `/anthropic` passthrough request to `…/v1/messages/count_tokens` crashes the async logging worker with `LoggingWorker error: 'content'`. The client-facing response succeeds (the body is forwarded fine), but the spend-log write dies, so **both** the `LiteLLM_SpendLogs` row **and** the native `litellm_total_tokens_metric_total` Prometheus counter are dropped for that call. In our fleet this was ~663 errors / 5 min sustained (one per agent token-count preflight).

### Root cause

`AnthropicPassthroughLoggingHandler.anthropic_passthrough_handler` (unary path) unconditionally calls `AnthropicConfig.transform_response`, which does `completion_response["content"]` at `litellm/llms/anthropic/chat/transformation.py` (extract_response_content, ~line 1982 in v1.91.0).

The `count_tokens` endpoint returns `{"input_tokens": N}` — valid JSON, HTTP 200, but **not** a Messages response and with **no top-level `content`**. `_is_streaming_response` gates purely on `content-type: text/event-stream`, so an…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d18h33m58s ago — entered · #import:https:::github.com:berriai:litellm post #2487
The left issue spans passthrough logging, response-shape detection, transformation safeguards, metric persistence, and regression coverage across multiple endpoint variants. The right issue is comparatively localized to preserving cancellation semantics in one async provider path and validating router behavior. The left therefore carries greater integration and compatibility risk.
#0 of 0 · 31d17h46m39s ago — current · #import:https:::github.com:berriai:litellm post #3301
#27883 requires broader proxy architecture, configuration lifecycle, provider/tool integration, admin controls, and security validation; #32280 is a localized logging/response-shape fix with targeted regression coverage.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search