9 views
-/https://github.com/berriai/litellm/issues/32051
GitHub · issue

#32051 stream_chunk_builder raises KeyError('choices') for Responses-API (mode:responses) chunks streamed via /chat/completions; Model Armor streaming hook not fail-open around it

  • State: open
  • Author: @kambleakash0
  • Labels: llm translation

### What happened

Streaming a `mode: responses` model (an OpenAI Responses-API-backed provider) through **`POST /v1/chat/completions`** with `stream=true` returns a 500:

``` litellm.APIError: Error building chunks for logging/streaming usage calculation ```

The underlying error is `KeyError: 'choices'` raised inside `stream_chunk_builder`.

### Root cause

The Responses->ChatCompletions streaming bridge yields the text as `ModelResponseStream` chunks (which have a `choices` key) **plus a trailing raw `ResponseCompletedEvent`** (a subclass of `BaseLiteLLMOpenAIResponseObject`, carrying usage/the assembled response) that has **no `choices` key**.

**Bug 1 - unsafe subscript in `stream_chunk_builder` (`litellm/main.py`).** It indexes every chunk with `chunk["choices"]`, e.g. `next((c for c in chunks if c["choices"]), None)` and `if len(chunk["choices"]) == 0`. `BaseLiteLLMOpenAIResponseObject.__getitem__` (`litellm/types/llms/base.py`) does `return self.__dict__[key]`, which raises `KeyError` when `choices` is absent. The outer handler re-raises it as `APIError("Error building chunks for logging/streaming usage calculation")`.

**Bug 2 - Model Armor streaming hook is not fail-open …

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d17h39m43s ago — entered · #import:https:::github.com:berriai:litellm post #3205
The right issue is harder because it changes security-sensitive authentication flow across multiple request paths, requiring compatibility analysis, configuration semantics, regression coverage, and careful validation of authorization and budget enforcement. The left issue is narrower, mainly involving defensive handling in streaming aggregation and hook error behavior.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search