6 views
-/https://github.com/berriai/litellm/issues/25869
GitHub · issue

#25869 [Bug]: `stream_chunk_builder` corrupts Gemini `server_side_tool_invocations` and `thought_signatures` in streaming, causing "Corrupted tool call context" on follow-up turns

  • State: open
  • Author: @jph00
  • Labels: bug, llm translation, SDK

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

When using Gemini models with streaming + web search + tool calls, follow-up turns fail with `400 Bad Request: "Corrupted tool call context"`. The root cause is in `stream_chunk_builder` — it incorrectly merges `provider_specific_fields` across streaming chunks.

### Root Cause

Gemini streams `server_side_tool_invocations` across multiple chunks: - **Chunk 1** contains `toolCall` with `args` and a correct `thought_signature` (~600 bytes) - **Chunk 2** contains `toolResponse` with `response` and a bloated `thought_signature` (~110KB, appears to contain the search HTML response)

In the non-streaming path, `_extract_server_side_tool_invocations` correctly handles this by merging toolCall and toolResponse by `id`, keeping the toolCall's `thought_signature` and only using the toolResponse's if the call doesn't already have one (lines 1411-1419 of `vertex_and_google_ai_studio_gemini.py`).

However, `stream_chunk_builder` (line ~7604 of `main.py`) simply takes the **last** list value for all list-type `provider_specific_fields`:

```python # For lis…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d18h30m28s ago — entered · #import:https:::github.com:berriai:litellm post #2439
The left issue is harder because it requires provider-specific streaming aggregation, preservation of structured metadata across chunk boundaries, compatibility with existing non-streaming behavior, and regression coverage for multi-turn tool workflows. The right issue is primarily an export aggregation discrepancy, likely localized to dashboard query or grouping logic and easier to validate against existing totals.
#0 of 0 · 31d17h48m30s ago — current · #import:https:::github.com:berriai:litellm post #3163
The right-hand task is harder because it requires carefully preserving provider-specific streaming state across incremental aggregation, maintaining parity with non-streaming behavior, and validating compatibility across affected model and tool-call paths. The left-hand task is comparatively contained to proxy logging configuration, filter registration, and lifecycle handling.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search