8 views
-/https://github.com/berriai/litellm/issues/31879
GitHub · issue

#31879 fix(vertex): accumulate JSON list buffer across stream chunks

  • State: open
  • Author: @deepanshululla
  • Labels: llm translation

## Problem

When Vertex AI returns a streaming response containing a JSON array split across multiple SSE chunks, the response parser does not accumulate the partial JSON across chunks. Each chunk is parsed independently, causing `json.JSONDecodeError` on partial array fragments and dropping content from all but the last chunk that completes the array.

## Proposed fix

Maintain a string buffer per stream that accumulates incoming JSON list content across chunks. Flush the buffer and parse when a complete JSON array is detected. This mirrors the existing partial-object accumulation logic and brings list handling to parity.

## Files

- `litellm/llms/vertex_ai/` (streaming response parser)

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d18h29m12s ago — entered · #import:https:::github.com:berriai:litellm post #2400
The right-hand change spans protocol interpretation, request lifecycle state, logging semantics, and regression coverage across integration paths. The left-hand change is a localized streaming-parser adjustment with a narrower test surface, so the right-hand issue carries greater implementation and compatibility risk.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search