9 views
-/https://github.com/berriai/litellm/issues/26155
GitHub · issue

#26155 [Bug]: MCP semantic tool filter crashloops at proxy startup with a large MCP server registry

  • State: open
  • Author: @XyLearningProgramming
  • Labels: bug, proxy

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

When `litellm_settings.mcp_semantic_tool_filter.enabled: true` and the DB has a non-trivial number of MCP servers registered (total tools in the low thousands), the proxy enters a deterministic crash loop on every boot and never becomes ready due to OOM or readiness probe even with reasonable time gap.

Root cause is in `SemanticToolFilterHook.initialize_from_config()`:

```330:331:litellm/proxy/hooks/mcp_semantic_filter/hook.py # Build router from MCP registry on startup await semantic_filter.build_router_from_mcp_registry() ```

`build_router_from_mcp_registry()` iterates **every** MCP server in the DB, fetches **every** tool from each, and constructs a single `SemanticRouter(routes=[...], auto_sync="local")` whose construction eagerly embeds every route's utterances in one shot. This is the only caller of this method in the codebase — `grep` confirms it runs exclusively at startup.

Two design gaps make this a hard crash-loop rather than "slow but eventually healthy":

1. **No persistence / no incremental build.** Eve…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d17h50m53s ago — entered · #import:https:::github.com:berriai:litellm post #3031
The left requires redesigning a startup-scale data and embedding workflow, with lifecycle, memory, persistence, and readiness implications. The right is a comparatively localized request sanitization and provider-translation fix.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search