7 views
-/https://github.com/berriai/litellm/issues/35589
GitHub · issue

#35589 [Bug]: amazon_nova streaming fails 100% - httpx zstd decoder reuses a finished decompressobj on Nova's multi-frame SSE

  • State: open
  • Author: @akashkokare2910
  • Labels: llm translation

### What happened?

Every **streaming** call to the `amazon_nova` provider fails. Non-streaming on the same account, model and prompt succeeds.

``` litellm.MidStreamFallbackError └─ httpx.DecodingError: cannot use a decompressobj multiple times └─ zstandard.backend_c.ZstdError: cannot use a decompressobj multiple times ```

Reproduced **5/5** streaming attempts against `amazon_nova/nova-micro-v1`.

### Root cause

`https://api.nova.amazon.com/v1` returns `content-encoding: zstd` for streaming SSE when the client offers it, and emits the stream as **independent zstd frames**. Verified with `curl` — a 785-byte capture contains 5 frame magics (`28b52ffd`) at offsets `[0, 210, 405, 584, 762]`.

`httpx.ZStandardDecoder.decode()` resets its decompressor only when `unused_data` is non-empty *within the same call*:

```python output.write(self.decompressor.decompress(data)) while self.decompressor.eof and self.decompressor.unused_data: ... # only resets if leftover bytes are present in THIS call ```

One SSE flush per network chunk delivers exactly one complete frame, so the decoder ends at `eof=True` with empty `unused_data`, and the next `decode()` reuses a finished decompressob…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d18h41m10s ago — entered · #import:https:::github.com:berriai:litellm post #2386
The left task is harder because it involves diagnosing and safely handling a low-level streaming decompression lifecycle across chunk boundaries, with provider-specific protocol behavior and dependency-level compatibility risks. The right task is comparatively contained to adding Unix-socket transport configuration to an existing observability client while preserving current HTTP modes.
#0 of 0 · 31d18h21s ago — current · #import:https:::github.com:berriai:litellm post #3067
The right-side task is harder because it involves streaming transport behavior, compression-state handling, provider-specific compatibility, and end-to-end regression testing. The left-side task is comparatively localized to propagating existing context through an error-metric path.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search