6 views
-/https://github.com/berriai/litellm/issues/26762
GitHub · issue

#26762 [Bug]: background Responses API stream resume on retrieve is not handled correctly via the proxy

  • State: open
  • Author: @chenyzan
  • Labels: proxy, llm translation, stale

## What happened?

I found this while using the OpenAI Responses API background mode through the LiteLLM proxy.

When resuming a background response via:

`GET /v1/responses/{response_id}?stream=true&starting_after=<sequence_number>`

LiteLLM did not correctly handle the retrieve streaming path.

I expected the proxy to support OpenAI-style cursor-based stream resume on the retrieve endpoint and return a valid SSE stream for background responses.

## Steps to Reproduce

1. Send a Responses API request through the LiteLLM proxy using OpenAI background mode. 2. Wait until the response can be retrieved by `response_id`. 3. Call `GET /v1/responses/{response_id}?stream=true&starting_after=<sequence_number>`. 4. Observe that the retrieve streaming path is not handled correctly.

## Relevant log output

No additional production logs attached.

I was able to reproduce this consistently through the proxy retrieve path and then narrowed it down to the retrieve streaming flow.

## What part of LiteLLM is this about?

Proxy

## What LiteLLM version are you on?

v1.83.14

## Additional context

This was discovered while trying to use background mode together with streaming resume, which is a va…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d18h31m43s ago — entered · #import:https:::github.com:berriai:litellm post #2332
The left issue has higher implementation risk because it involves stateful protocol semantics, cursor handling, resumable streaming, and compatibility-sensitive proxy behavior. The right issue is comparatively narrower, centered on provider resolution and mode-specific request routing.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search