10 views
-/https://github.com/berriai/litellm/issues/33404
GitHub · issue

#33404 [Bug]: Streaming upstream reset is converted to synthetic finish_reason stop / [DONE]

  • State: open
  • Author: @sytianhe
  • Labels: proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

Related but not duplicate:

- #29602 discusses mid-stream timeout failures being logged/accounted as success. - #24929 discusses stream failures related to HTTP client TTL.

This issue is specifically about an upstream streaming TCP reset being converted into a synthetic successful terminal SSE event.

### What happened?

When LiteLLM Proxy streams from an OpenAI-compatible upstream provider, if the upstream provider:

1. returns `HTTP 200` with `content-type: text/event-stream`, 2. sends one or more valid SSE chat completion chunks, 3. then the TCP connection resets before the upstream sends a final `finish_reason` or `data: [DONE]`,

LiteLLM can synthesize a successful final stream event:

```text data: {"choices":[{"delta":{},"finish_reason":"stop"}]} data: [DONE] ```

This makes an abnormal upstream transport failure look like a valid model completion.

Observed variants:

1. If the upstream resets after only an initial assistant role chunk, the client receives an empty successful stream:

```text data: {"choices":[{"delta":{},"finish_reason":"stop"}]} data: […

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d18h52m25s ago — entered · #import:https:::github.com:berriai:litellm post #2092
#27949 requires broader architectural design, security-policy decisions, and integration across persistence and agent-related workflows, with substantial compatibility and validation risk. #33404 is comparatively localized to streaming error propagation and terminal-event handling, with a clearer implementation and testing boundary.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search