6 views
-/https://github.com/berriai/litellm/issues/32561
GitHub · issue

#32561 [Bug]: MCP gateway streaming returns HTTP 200 + mcp_list_tools events with no response.created when the initial LLM call fails

  • State: open
  • Author: @thibault-linktree
  • Labels: llm translation

### What happened?

When a streaming Responses API request uses the MCP gateway (`server_url: litellm_proxy/...`) and the **initial LLM call fails** (e.g. an invalid/expired `previous_response_id` producing an OpenAI 400), the proxy still returns **HTTP 200** and the SSE stream emits the pre-generated `mcp_list_tools` discovery events with **no `response.created` before them**, then ends with `[DONE]`. No `error` event, no `response.failed`.

This violates the Responses API streaming contract (streams begin with `response.created` and end with a terminal event) and hard-crashes SDK stream accumulators. openai-node's `responses.stream()`:

``` Error: When snapshot hasn't been set yet, expected 'response.created' event, got response.mcp_list_tools.in_progress at ResponseStream._ResponseStream_accumulateResponse (.../openai/src/lib/responses/ResponseStream.ts:207:15) ```

The underlying provider error is only visible in proxy stderr (`mcp_streaming_iterator.py: Error creating initial response iterator: ...`); the client has no way to know what failed.

Expected: the request fails with a real 4xx carrying the provider error body (nothing has been streamed yet at that point), or at …

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d18h53m46s ago — entered · #import:https:::github.com:berriai:litellm post #1984
The left issue is harder because it spans MCP discovery, Responses API event sequencing, streaming lifecycle, HTTP status handling, and provider-error propagation across multiple layers. The right issue is more localized to preserving cancellation semantics and adding targeted regression coverage, although router fallback behavior requires care.
#0 of 0 · 31d17h34m11s ago — current · #import:https:::github.com:berriai:litellm post #3317
The left issue is harder because it crosses guardrail enforcement, pass-through response handling, provider-format normalization, and potentially streaming paths, with security-sensitive regression coverage. The right issue is comparatively localized to early-error propagation and protocol-correct SSE termination in one integration path.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search