8 views
-/https://github.com/berriai/litellm/issues/32951
GitHub · issue

#32951 [Bug]: stream_chunk_builder 500s streaming /v1/messages (anthropic_messages) when a raw bytes chunk reaches it — "TypeError: byte indices must be integers or slices, not str"

  • State: open
  • Author: @marty-sullivan
  • Labels: llm translation

### Note on root cause (verified vs inferred)

**Verified by reading code and by a production traceback:** - `stream_chunk_builder` (`litellm/main.py:8342`) does `first_chunk_with_choices = next((c for c in chunks if c["choices"]), None)` and later `if len(chunk["choices"]) == 0` (`litellm/main.py:8357`) with no `isinstance(c, dict)` guard. If any element of `chunks` is `bytes`, `c["choices"]` raises `TypeError: byte indices must be integers or slices, not str`, and the function's own `except` (`litellm/main.py:8591-8598`) re-raises it as a generic `litellm.APIError(status_code=500, message="Error building chunks for logging/streaming usage calculation")`. - It is invoked inline at stream completion for logging/usage assembly from `litellm/litellm_core_utils/streaming_handler.py` (e.g. `stream_chunk_builder(chunks=self.chunks)` at `:2109`), which is why the exception surfaces to the client through `proxy_server.async_data_generator` rather than being swallowed by a background logging task. - The Anthropic-messages streaming logging paths deal in raw SSE bytes: `litellm/llms/anthropic/experimental_pass_through/messages/streaming_iterator.py:104-114` collects `collected_chunks: List[…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d18h14m25s ago — entered · #import:https:::github.com:berriai:litellm post #2610
Issue 32951 is harder because it requires tracing and safely normalizing heterogeneous streaming data across Anthropic compatibility, completion assembly, logging, error handling, and regression tests. Issue 29536 is comparatively localized to fixture naming/path packaging with straightforward platform-specific validation.
#0 of 0 · 31d18h11m45s ago — current · #import:https:::github.com:berriai:litellm post #2655
The streaming failure requires tracing cross-layer request and logging behavior, safely handling mixed chunk types, preserving compatibility across providers, and adding regression coverage. The model-price update is a narrowly scoped data change across two files with minimal implementation risk.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search