8 views
-/https://github.com/berriai/litellm/issues/23544
GitHub · issue

#23544 MCP: list_tools called on every tool call for HTTP servers - should use cached tool list

  • State: open
  • Author: @madhu19991

## Describe the bug

When a `tools/call` request is made through LiteLLM's MCP proxy for an HTTP MCP server, LiteLLM calls `list_tools` on the upstream server on **every single tool call**. This causes:

1. An extra round-trip to the upstream MCP server on every call (doubles latency) 2. If `list_tools` is slow or gets cancelled (e.g. client disconnects), the `tools/call` is cancelled too 3. For OAuth2 M2M servers, this also means an extra token fetch/validation cycle

## Expected behavior

Tool metadata for HTTP MCP servers should be cached after the first successful `list_tools` (similar to how OpenAPI tools use `global_mcp_tool_registry`). The cache already exists in `global_mcp_tool_registry` — it's just not populated for HTTP servers.

Additionally, if `allowed_tools` is configured on the server (which is already stored in `MCPServer`), LiteLLM has enough information at startup to skip the runtime `list_tools` call entirely for validation purposes.

## Steps to reproduce

1. Configure an HTTP MCP server with `auth_type: oauth2` (M2M / client_credentials) 2. Make a `tools/call` request via the MCP protocol endpoint 3. Observe in logs: `list_tools` is called on the upstream serv…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d18h21m33s ago — entered · #import:https:::github.com:berriai:litellm post #2587
Issue 35109 is substantially harder because it requires cross-cutting proxy configuration, identity resolution, security-sensitive isolation behavior, and coordinated support across multiple vector and RAG backends. Issue 23544 is comparatively localized to MCP request flow, caching, and validation behavior.
#0 of 0 · 31d17h23m51s ago — current · #import:https:::github.com:berriai:litellm post #3557
The left task is harder because it requires diagnosing and safely eliminating a concurrency-dependent database deadlock, with transactional, locking, ordering, and migration/regression-test implications. The right task is comparatively localized cache integration with clearer behavior and bounded testing.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search