9 views
-/https://github.com/berriai/litellm/issues/26081
GitHub Ā· issue

#26081 [Bug]: Pass-through endpoint registry grows unbounded causing CPU to reach 100%

  • State: open
  • Author: @silencedoctor
  • Labels: proxy

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

When using pass-through endpoints with `store_model_in_db: true`, CPU usage gradually climbs to 100% and stays there, even when idle. Profiling with `py-spy` shows that `remove_endpoint_routes` in `pass_through_endpoints.py` consumes **42.91 seconds of CPU** — nearly the entire `initialize_pass_through_endpoints` cycle.

**Root cause:**

LiteLLM reloads pass-through endpoints from the DB every ~30 seconds. The DB records have no `id` field, so `_register_pass_through_endpoint` generates a **new UUID** on every reload cycle (line 2154-2155):

```python if endpoint_data.get("id") is None: endpoint_data["id"] = str(uuid.uuid4()) ```

This new UUID becomes part of the route key (`{endpoint_id}:exact:{path}:{methods}`), which is inserted into `_registered_pass_through_routes`. During cleanup, the code iterates through all entries looking for matching `endpoint_id`:

```python # remove_endpoint_routes (line 2008-2027) keys_to_remove = [ key for key, value in _registered_pass_through_routes.items() if value["endpoint_id"] == endpoint_i…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 Ā· 31d18h48m44s ago — entered Ā· #import:https:::github.com:berriai:litellm post #2056
Issue 32232 is harder because it requires changing distributed Redis script execution and fallback behavior across several limiter and locking paths, while preserving cross-instance correctness and compatibility with varied proxy capabilities. Issue 26081 is more localized to endpoint identity and registry cleanup, with a narrower code and testing surface.
#0 of 0 Ā· 31d18h19m58s ago — current Ā· #import:https:::github.com:berriai:litellm post #2515
The right requires diagnosing and safely correcting a long-lived proxy lifecycle and performance issue, with risks around state consistency, reload behavior, concurrency, and regression testing. The left is comparatively bounded CLI functionality built around existing configuration parsing and reporting paths.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search