8 views
-/https://github.com/berriai/litellm/issues/33374
GitHub · issue

#33374 [Bug]: Latency issues in global mcp endpoint /mcp/

  • State: open
  • Author: @caldito
  • Labels: bug

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

LiteLLM’s /mcp/ endpoint has two latency issues:

1. tools/list unnecessarily fetched initialization instructions from upstream MCP servers, even though instructions are only needed during initialize. 2. LITELLM_MCP_TOOL_LISTING_TIMEOUT cancelled slow listings at the configured deadline, but still waited for MCP SDK cleanup. A five-second timeout could therefore take much longer to return.

### Steps to Reproduce

1. Configure LiteLLM with multiple MCP servers, including one slow server. 2. Set LITELLM_MCP_TOOL_LISTING_TIMEOUT=5. 3. Send an unscoped tools/list request to /mcp/. 4. Observe that upstream initialization requests are made and the response can take significantly longer than five seconds.

### Relevant log output

```shell

```

### What part of LiteLLM is this about?

_No response_

### What LiteLLM version are you on ?

v1.88.0

### Twitter / LinkedIn details

_No response_

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (3 events)
#0 of 0 · 31d19h3m11s ago — entered · #import:https:::github.com:berriai:litellm post #1911
The left issue requires coordinated asynchronous lifecycle, timeout, cancellation, cleanup, and MCP protocol behavior changes with concurrency-sensitive testing. The right issue is comparatively localized to provider request translation or endpoint construction, with narrower validation scope.
#32614 is harder because it involves subtle asynchronous consistency guarantees across shared cache, Redis synchronization, inherited call paths, and race-focused regression testing. #33374 is narrower endpoint-level latency work, mainly involving request planning and cancellation behavior.
#0 of 0 · 31d17h56m25s ago — current · #import:https:::github.com:berriai:litellm post #3025
The left issue is harder because it likely requires coordinating response-state lifecycle, persistence timing, provider abstraction, concurrency, and compatibility across proxy paths. The right issue is comparatively localized to MCP request preparation and timeout/cleanup behavior, with narrower testing and regression scope.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search