9 views
-/https://github.com/berriai/litellm/issues/32357
GitHub · issue

#32357 [Bug] /v1/messages adapter mis-encodes reasoning models: thinking_delta streamed inside a text block (+ duplicate message_start) → empty content in Anthropic SDK / Claude Code

  • State: open
  • Author: @ketor
  • Labels: llm translation, claude code

### What happened

When a LiteLLM proxy fronts an **OpenAI-compatible backend that returns `reasoning_content`** in streaming chunks (e.g. a GLM / DeepSeek-R1 style reasoning model served by vLLM/SGLang) and a client calls the **Anthropic `/v1/messages`** endpoint with `stream: true`, the experimental Anthropic adapter produces a **spec-violating SSE stream**:

1. **`message_start` is emitted twice** (byte-identical, same message `id`). 2. The reasoning is streamed as **`thinking_delta` events into a content block that was opened as `type: "text"`** — the block is never re-typed to `thinking` and no `signature_delta` is sent.

The official **Anthropic SDK** (and therefore **Claude Code**, which is built on it) only appends `text_delta` to a `text` block; a `thinking_delta` arriving in a `text` block is **silently dropped**. The assembled assistant message therefore ends up with **empty content**. In multi-turn / tool-use sessions where a turn produces only reasoning before `end_turn`, Claude Code sees `last_content_type=none` and aborts the turn with `error_during_execution` ("Your previous response had no visible output"). OpenAI-style clients hitting `/chat/completions` are unaff…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (4 events)
#0 of 0 · 31d19h22s ago — entered · #import:https:::github.com:berriai:litellm post #1954
The left requires coordinated streaming-state and protocol-correctness changes with broader compatibility risk; the right is a localized route-matching fix with a narrower test surface.
The right issue is harder because it requires correcting stateful streaming protocol behavior, preserving event ordering and content-block semantics, and validating compatibility across multiple client and model paths. The left is comparatively localized endpoint lookup logic with narrower regression testing.
The left task is harder because it requires security-boundary changes, careful failure-semantics decisions, compatibility review, and broader regression testing; the right task is more localized to protocol translation and streaming-state handling.
#0 of 0 · 31d18h19m23s ago — current · #import:https:::github.com:berriai:litellm post #2620
The left issue is harder because it spans application startup ordering, subprocess readiness detection, failure handling, background task coordination, and deployment-level regression testing. The right issue is more localized to protocol translation state management and stream conformance, with a narrower test surface.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search