8 views
-/https://github.com/berriai/litellm/issues/33168
GitHub ยท issue

#33168 [Bug]: DB-stored auto-router/complexity-router models permanently disappear from /v1/models & /model/info (strategy registries never cleaned on update/delete)

  • State: open
  • Author: @mihidumh
  • Labels: llm translation

### What happened?

DB-stored auto-router / complexity-router models (`litellm_params.model = "auto_router/..."`) permanently disappear from `/v1/models`, `/model/info`, and `/model_group/info` โ€” even for the master key โ€” while continuing to serve completions. Once delisted they cannot be re-listed without deleting the row AND restarting the proxy.

Verified lifecycle on a live proxy (1.92.0, `STORE_MODEL_IN_DB=true`), reproducible at will:

1. `POST /model/new` with a `complexity_router_config` row โ†’ listed, routes, and **stable** across the 30s DB sync cycles. โœ… 2. `PATCH /model/{id}/update` on that row โ†’ **instantly and permanently delisted**, and the patched config is silently ignored (the stale in-memory router keeps serving the OLD tiers). โŒ 3. `POST /model/delete` โ†’ removed from DB/listing but **keeps routing as a ghost** until restart. โŒ 4. Any replica restart โ†’ rows load at startup, then get delisted within one sync cycle. โŒ

Downstream impact: clients that build their model list from `/models` (LibreChat `fetch: true`, OpenWebUI, etc.) can never see or validate the auto-router model names.

### Root cause

Every deployment-removal path cleans `self.model_list` but never tโ€ฆ

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 ยท 31d18h26m21s ago โ€” entered ยท #import:https:::github.com:berriai:litellm post #2513
The left requires coordinated lifecycle handling across persistence synchronization, in-memory registries, updates, deletions, restarts, and multiple model-discovery paths, with broader regression and concurrency risk. The right is more likely an isolated provider-response parsing or non-streaming compatibility fix with narrower testing scope.
#0 of 0 ยท 31d18h25m42s ago โ€” current ยท #import:https:::github.com:berriai:litellm post #2520
The right-hand issue is substantially harder: it requires tracing proxy model synchronization, update/deletion lifecycle handling, in-memory routing registries, replica behavior, and regression coverage. The left-hand issue is a narrowly scoped documentation change with minimal implementation risk.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search