8 views
-/https://github.com/berriai/litellm/issues/29491
GitHub · issue

#29491 [Bug]: Anthropic streaming format drops input_json_delta for tool_use via custom OpenAI-compatible provider

  • State: open
  • Author: @YaoYufeng
  • Labels: llm translation, claude code

<h1 id="bug-report-litellm-streaming-response-causes-tool_use-input-loss-when-proxied-through-anthropic-compatible-gateways" class="atx">Bug Report: LiteLLM streaming response causes tool_use input loss when proxied through Anthropic-compatible gateways</h1> <h2 id="summary" class="atx">Summary</h2> <p>When LiteLLM proxies streaming responses from a custom OpenAI-compatible provider (SiliconFlow), its SSE output differs subtly from the upstream provider&#39;s native SSE. This causes downstream Anthropic-compatible gateways (e.g., cc-switch&#39;s <code>create_anthropic_sse_stream</code> converter) to lose <code>input_json_delta</code> events during OpenAI → Anthropic streaming format conversion. Claude Code receives <code>tool_use</code> blocks with empty <code>input: {}</code>, breaking the tool-use workflow. Multiple models are affected, with failure rates proportional to tool input payload size.</p> <h2 id="environment" class="atx">Environment</h2> <ul> <li><strong>LiteLLM version</strong>: 1.85.1 (main-stable Docker image, <code>ghcr.io/berriai/litellm:main-stable</code>)</li> <li><strong>Upstream provider</strong>: SiliconFlow (<code>api.siliconflow.cn</code>) — OpenAI-compatib…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (3 events)
#0 of 0 · 31d18h33m49s ago — entered · #import:https:::github.com:berriai:litellm post #2322
29491 requires tracing streaming event translation across provider and downstream compatibility boundaries, with timing, chunking, and tool-payload regression risks; 27946 is a localized response-field mapping change with a comparatively narrow test surface.
The right-hand task is harder because it crosses streaming protocols, incremental event handling, provider-specific translation, and compatibility testing with downstream consumers; the left-hand task is comparatively localized to PATCH update and JSON field-deletion semantics.
#0 of 0 · 31d18h17m31s ago — current · #import:https:::github.com:berriai:litellm post #2571
The right-side issue is harder because it likely requires tracing and correcting streaming event translation across provider formats, preserving incremental tool-call state, and adding end-to-end regression coverage. The left-side issue appears comparatively localized to router construction and a focused test.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search