6 views
-/https://github.com/berriai/litellm/issues/30761
GitHub · issue

#30761 [Bug]: Anthropic streaming bridge crashes on empty `choices` chunks from OpenAI/Azure-compatible backends

  • State: open
  • Author: @vrlo
  • Labels: bug, proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

When LiteLLM serves Anthropic-compatible `/v1/messages` requests for an OpenAI/Azure-compatible backend in streaming mode, the stream can crash mid-flight with:

```text IndexError: list index out of range ```

This happens in LiteLLM's Anthropic pass-through streaming adapter when an upstream streaming chunk has `choices=[]`.

The request still starts with `HTTP 200` and Anthropic-style SSE events such as:

- `message_start` - `content_block_start`

but then the stream fails and emits an SSE error payload instead of completing normally.

### Why this is happening

The Anthropic streaming adapter assumes every chunk has `choices[0]` and crashes instead of skipping or handling empty-choice chunks.

### Root cause

There are multiple unguarded `choices[0]` accesses in the Anthropic pass-through adapter.

Examples: - https://github.com/BerriAI/litellm/blob/v1.89.2/litellm/llms/anthropic/experimental_pass_through/adapters/streaming_iterator.py#L346 - https://github.com/BerriAI/litellm/blob/v1.89.2/litellm/llms/anthropic/experimental_pass_through/ad…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d18h37m27s ago — entered · #import:https:::github.com:berriai:litellm post #2431
The right-hand issue likely requires broader protocol, proxy, and compatibility investigation across request parsing and streaming behavior, with higher regression risk. The left-hand issue is comparatively localized to defensive handling in an existing adapter plus focused tests.
#0 of 0 · 31d18h22m22s ago — current · #import:https:::github.com:berriai:litellm post #2683
The right-side work is harder because it requires defensive handling across a streaming translation path, careful preservation of protocol behavior, and broader regression coverage. The left-side work appears more localized to model capability registration and validation.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search