8 views
-/https://github.com/berriai/litellm/issues/32562
GitHub · issue

#32562 [Bug]: MCP gateway streaming ends with no terminal event when tool execution or the follow-up LLM call fails

  • State: open
  • Author: @thibault-linktree

### What happened?

In MCP gateway streaming with auto-execution (`require_approval: never`), two mid-stream failure classes are swallowed by `MCPEnhancedStreamingIterator`:

1. **Tool execution fails as a batch** (`_generate_tool_execution_events` catches, sets `tool_results = []`): the iterator then proceeds to make a follow-up call carrying `function_call` items with **no `function_call_output` items**, which the provider rejects with `"No tool output found for function call ..."` — and that failure is swallowed too. 2. **The follow-up LLM call fails** (`_create_follow_up_iterator` catches, sets `base_iterator = None`): the stream just ends.

In both cases the client receives HTTP 200 and a stream that looks like a truncated success: `mcp_call` events, then silence. No `error` event, no `response.failed`, no final text. Client-side this is indistinguishable from a completed-but-empty answer; the actual failure only exists in proxy stderr.

Captured event sequence (harness driving the real iterator, main @ 999637883c), tools succeed then follow-up 400s:

``` response.output_item.added response.mcp_list_tools.in_progress response.mcp_list_tools.completed response.completed …

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (10 events)
#0 of 0 · 31d18h11m20s ago — entered · #import:https:::github.com:berriai:litellm post #2663
The left requires coordinated changes across asynchronous control flow, failure propagation, protocol-compliant terminal events, and regression coverage. The right is broad but primarily calls for profiling and targeted performance remediation, with less clearly defined implementation scope.
#32562 requires cross-path async stream error propagation, protocol-correct terminal events, and regression coverage; #26424 is a localized logging-severity and traceback-handling adjustment.
#34733 is harder: it requires coordinating atomic distributed state transitions, concurrency-safe accounting, and asynchronous persistence across replicas, with substantial race-condition testing. #32562 is more localized to propagating terminal failures consistently through an existing streaming iterator and adding protocol-focused tests.
Issue 23841 spans several translation paths and provider-specific compatibility cases, requiring coordinated transformation fixes and broad regression coverage. Issue 32562 is narrower, primarily involving failure propagation and terminal-event handling within one streaming iterator flow, so it carries less implementation scope.
The right issue is harder because it crosses streaming state management, exception propagation, protocol-compliant terminal signaling, and regression testing across multiple asynchronous failure paths. The left issue is more localized to request-shape translation and multipart image handling for one provider capability.
The left issue is harder because it requires coordinating asynchronous iterator failure handling, terminal-event guarantees, chained tool/LLM execution, and regression coverage across multiple streaming paths. The right issue is a broader input-translation enhancement, but it is comparatively bounded to multimodal normalization, MIME handling, and provider-specific tests.
The left issue requires coordinated changes to asynchronous streaming state, exception propagation, terminal event semantics, and regression coverage across multiple failure paths. The right issue is comparatively localized to provider routing and health-check input handling.
#26535 is harder because it spans coordinated test infrastructure, provider coverage, CI integration, and release-compatible validation, whereas #32562 is a focused runtime error-propagation fix within one streaming subsystem.
Recursive schema handling is harder because it requires safe, bounded expansion semantics across multiple translation paths, compatibility decisions, and broad regression testing; the streaming fix is comparatively localized to iterator error propagation and terminal-event behavior.
#0 of 0 · 31d17h24m20s ago — current · #import:https:::github.com:berriai:litellm post #3471
The left issue spans backend configuration discovery, secure redaction, API/UI changes, and compatibility across multiple declarative deployment paths. The right issue is narrower, mainly requiring robust failure propagation and event-stream tests within one execution flow.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search