8 views
-/https://github.com/berriai/litellm/issues/27671
GitHub Β· issue

#27671 Responses API streaming bridge: multi-step Anthropic tool calls emit text-delta with unregistered chatcmpl- ID

  • State: open
  • Author: @klly14
  • Labels: llm translation

## Description

When using LiteLLM's Responses API (`/v1/responses`) to proxy Anthropic models, multi-step tool call flows produce:

``` text part chatcmpl-<id> not found ```

The AI SDK tracks text parts via `text-start` β†’ `text-delta` β†’ `text-end` by ID. During step 2+ (text after tool use), the `text-delta` event arrives with a `chatcmpl-` response-level ID that was never registered via `text-start`, causing the AI SDK to drop the text.

Single-step text-only responses work fine. Only multi-step tool call sequences break.

**The Bug**

In _transform_chat_completion_chunk_to_response_api_chunk:

``` if self._cached_item_id is None and chunk.id: self._cached_item_id = chunk.id # ← sets cache to "chatcmpl-aaa" on first chunk item_id = self._cached_item_id or chunk.id # ← uses cached value for all events

``` Step 1 (tool call): First chunk arrives, _cached_item_id is None, so it caches chunk.id = "chatcmpl-aaa". The _ensure_output_item_for_chunk method then overwrites it to "msg_{uuid}" for the output item. Text events use "msg_{uuid}" β€” consistent. βœ…

Step 2 (text after tool result): LiteLLM internally makes a new Chat Completion call, which returns ch…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 Β· 31d18h17m33s ago β€” entered Β· #import:https:::github.com:berriai:litellm post #2569
Issue 35359 is harder because it requires changing fallback and retry semantics across batch routing, error classification, provider resource affinity, and regression coverage. Issue 27671 is narrower, primarily involving identifier lifecycle handling in one streaming translation path.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search