7 views
-/https://github.com/berriai/litellm/issues/32031
GitHub · issue

#32031 [Bug]: Responses API + hosted MCP tool: `previous_response_id` is double-encoded, breaking multi-turn (upstream rejects it as "string too long", streaming returns an empty completion)

  • State: open
  • Author: @thibault-linktree
  • Labels: bug, proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

When a `/v1/responses` request is handled by the MCP gateway (a hosted MCP tool with `server_url: litellm_proxy/...`), the id LiteLLM returns to the client is base64-encoded twice. On the next turn LiteLLM only decodes one layer, so it forwards a still-encoded `previous_response_id` to the upstream provider. OpenAI rejects it:

``` litellm.BadRequestError: OpenAIException - { "error": { "message": "Invalid 'previous_response_id': string too long. Expected a string with maximum length 64, but got a string with length 205 instead.", "type": "invalid_request_error", "param": "previous_response_id", "code": "string_above_max_length" } } ```

On the non-streaming path this surfaces as a 400. On the streaming path the same error is swallowed inside the MCP streaming iterator, so the client gets HTTP 200, sees `response.mcp_list_tools.completed`, and then `[DONE]` with no model output at all (a silent empty completion). Multi-turn conversations that rely on `previous_response_id` together with a hosted MCP tool are therefore comple…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d18h1m3s ago — entered · #import:https:::github.com:berriai:litellm post #2850
The left issue is harder because it crosses response-ID encoding, hosted-tool gateway behavior, multi-turn state handling, and distinct streaming versus non-streaming error paths, requiring broader regression coverage. The right issue is comparatively localized to Ollama request/response transformation and reasoning-field extraction.
#0 of 0 · 31d17h46m47s ago — current · #import:https:::github.com:berriai:litellm post #3103
The right issue is harder because it spans protocol translation, state encoding across requests, hosted-tool integration, separate streaming and non-streaming control flows, and error propagation, requiring broader debugging and regression coverage than the narrower request-validation and UI contract change.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search