8 views
-/https://github.com/berriai/litellm/issues/34534
GitHub Ā· issue

#34534 [Bug]: Every MCP tool call permanently leaks a max_parallel_requests slot — key wedges after N calls until proxy restart

  • State: open
  • Author: @rschlek
  • Labels: claude code

## What happened

Any virtual key with `max_parallel_requests: N` that makes MCP tool calls through the proxy's `/mcp` endpoint gets permanently rate limited after ~N calls — even with **strictly sequential** calls (true concurrency never exceeding 1). Once wedged, every subsequent request on the key 429s:

``` Rate limit exceeded for api_key: <hash>. Limit type: max_parallel_requests. Current limit: 20, Remaining: 0. Limit resets at: <now + 1 min> ```

The "resets at" timestamp slides forward each minute but `Remaining` never recovers, because the in-flight gauge is a concurrency counter, not a window. Only a proxy restart clears it (in-memory cache), after which the countdown starts again.

## Repro

Proxy `main-latest` (image built 2026-07-18, self-reports v1.94.0), no Redis, v3 limiter active (default; `LEGACY_MULTI_INSTANCE_RATE_LIMITING` unset). Key with `max_parallel_requests: 20`, one MCP server configured (Graphiti, streamable HTTP).

1. MCP `initialize` handshake with the key 2. Call any MCP tool (`tools/call`) sequentially, one at a time, ~5/sec 3. Observed live: first 429 at call 15 (the key had ~5 slots consumed by earlier session traffic), 100% 429s after that, `ss` c…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 Ā· 31d18h9m7s ago — entered Ā· #import:https:::github.com:berriai:litellm post #2703
Budget rollover spans persistent budget semantics, reset scheduling, multiple entity types, API/config compatibility, accumulation caps, and concurrency/backward-compatibility concerns. The MCP issue is narrower, focused on tracing and correcting request-lifecycle cleanup in the limiter path, though it still requires careful regression testing.
#0 of 0 Ā· 31d18h2m19s ago — current Ā· #import:https:::github.com:berriai:litellm post #2820
34534 requires broader investigation across request lifecycle, limiter accounting, MCP integration, cleanup guarantees, and regression testing; 32412 is more localized to startup persistence/error handling.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search