8 views
-/https://github.com/berriai/litellm/issues/33200
GitHub · issue

#33200 [Feature]: Adding concurrency on the REST /v1/mcp/tools endpoint

  • State: open
  • Author: @caldito
  • Labels: enhancement, proxy

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### The Feature

Make the tool listing fetch the tools from the available servers on the `/v1/mcp/tools` concurrent. Right now it's too slow as it fetches tools from each server sequentially.

### Motivation, pitch

I'm working on providing an MCP Gateway using LiteLLM, and already have a few MCPs registered.

When users use the endpoint for listing their tools they often think it's stuck since it takes long to respond. Also for us as admins is good to have a quick endpoint for monitoring.

### What part of LiteLLM is this about?

Proxy

### LiteLLM is hiring a founding backend engineer, are you interested in joining us and shipping to all our users?

No

### Twitter / LinkedIn details

_No response_

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d17h58m1s ago — entered · #import:https:::github.com:berriai:litellm post #2998
The left requires changes to asynchronous request orchestration, failure handling, bounded parallelism, and regression testing across a live proxy path. The right is a localized CI configuration correction with limited behavioral scope.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search