4 views
-/https://github.com/berriai/litellm/issues/30471
GitHub · issue

#30471 [Bug]: Gemini pass-through streaming endpoint (streamGenerateContent) returns truncated JSON

  • State: open
  • Author: @apujari-netapp
  • Labels: bug, proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

Gemini pass-through streaming endpoint (streamGenerateContent) returns truncated JSON. This issue seems intermittent when response contains large thoughtSignature.

When using the LiteLLM proxy's Gemini pass-through endpoint (/v1beta/models/{model}:streamGenerateContent), streaming responses from Gemini 3.x models with thinking enabled are intermittently truncated, resulting in invalid/incomplete JSON being delivered to the client.

This affects both google-genai SDK and the gemini-cli.

### Steps to Reproduce

1. LiteLLM version: [litellm[proxy]==1.89.0] 2. Models: Gemini Model with Vertex AI (global region) 3. Condition: Thinking/reasoning enabled (any thinkingLevel: minimal, low, medium, high)

### Relevant log output

```shell google.genai.errors.UnknownApiResponseError: Failed to parse response as JSON. ```

### What part of LiteLLM is this about?

Proxy

### What LiteLLM version are you on ?

v1.89.0

### Twitter / LinkedIn details

_No response_

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d17h43m52s ago — entered · #import:https:::github.com:berriai:litellm post #3242
The left entails diagnosing provider-specific streaming boundaries, incremental serialization, and intermittent large-payload behavior across SDK-compatible proxy paths, with substantial regression-testing risk. The right is comparatively localized accounting and entitlement-state logic with a narrower authentication/UI scope.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search