9 views
-/https://github.com/berriai/litellm/issues/24089
GitHub Β· issue

#24089 MCP: tool_name_to_mcp_server_name_mapping emptied by cancelled list_tools β†’ 404 on tool calls

  • State: open
  • Author: @madhu19991

## Description

When LiteLLM calls `list_tools` on an async/SSE-based MCP server (e.g. Databricks) via the MCP protocol endpoint, an anyio cancel scope fires before the SSE stream is fully processed. The call is aborted mid-execution β€” `_list_tools_and_add_prefix` never completes, so `tool_name_to_mcp_server_name_mapping` is **never populated** for that server. All subsequent `mcp-rest/tools/call` requests for that server's tools then return 404.

## Root Cause

### Step 1: Protocol `list_tools` is cancelled β€” mapping never written

For async/SSE MCP servers (e.g. Databricks responding 202 + SSE stream), the anyio cancel scope in `client.py` fires before the upstream response is processed. The entire `_list_tools_and_add_prefix` call is aborted β€” it doesn't write 0 tools, it writes **nothing**. The mapping for that server stays empty.

Synchronous servers (e.g. returning 200 directly) are unaffected because the response arrives before the cancel scope fires.

### Step 2: The 30s reload cycle doesn't fix it

`initialize_tool_name_to_mcp_server_name_mapping` is called on startup (after loading config) and when OpenAPI servers are registered. It calls `_get_tools_from_server` for each…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 Β· 31d18h6m23s ago β€” entered Β· #import:https:::github.com:berriai:litellm post #2756
The left issue requires diagnosing and safely correcting asynchronous cancellation, state initialization, protocol-specific behavior, and regression coverage across MCP transport variants. The right issue appears more localized to proxy callback wiring and should be easier to isolate and fix.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search