8 views
-/https://github.com/berriai/litellm/issues/35577
GitHub · issue

#35577 [Bug]: New DB deployments are dropped during router upsert when no existing deployment is found

  • State: open
  • Author: @emerzon
  • Labels: proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate

### What happened?

Database-backed deployments are dropped from a proxy pod's in-memory router when they are loaded for the first time through `Router.upsert_deployment()`

This affects startup and periodic database reloads. Static configuration deployments remain available, while database deployments disappear from model-list endpoints and cannot receive traffic. Model creation can persist the row successfully and then return an internal error because the new deployment is not live in the pod after reload

The regression was introduced by commit [300e710bc34](https://github.com/BerriAI/litellm/commit/300e710bc34). `upsert_deployment()` calls `get_deployment()`, which returns `None` for a deployment that is new to the current router. It then unconditionally passes that value to `_unregister_pre_routing_strategy_for_deployment()`, where `deployment.litellm_params` is dereferenced

With `ignore_invalid_deployments=True`, the resulting `AttributeError` is caught and the deployment is silently dropped

Expected behavior: a previously unseen database deployment is adde…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d18h42m33s ago — entered · #import:https:::github.com:berriai:litellm post #2179
The right-side task has broader protocol-translation implications, requiring careful handling of multiple message schemas, compatibility decisions, and regression coverage. The left-side task appears more localized, centered on defensive handling of an absent object and targeted lifecycle tests.
#0 of 0 · 31d18h34m26s ago — current · #import:https:::github.com:berriai:litellm post #2312
Issue 27122 is harder because it requires coordinating test-environment initialization and potentially many existing test paths, with CI and import-order risks. Issue 35577 appears more localized to router upsert handling, validation, and regression coverage.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search