8 views
-/https://github.com/berriai/litellm/issues/29397
GitHub · issue

#29397 [Bug] AdaptiveRouter reload crashes with "gammavariate: alpha and beta must be > 0.0" after restart

  • State: open
  • Author: @sha-ir

### What happened?

After a proxy restart, requests to an `auto_router/adaptive_router` model return **HTTP 500**:

``` ValueError: gammavariate: alpha and beta must be > 0.0 ```

### Root cause

`AdaptiveRouter.load_state_from_db` reloads persisted state by **overwriting** each cold-start prior with the raw DB row:

```python self._cells[(rt, row.model_name)] = BanditCell(alpha=row.alpha, beta=row.beta) ```

But `AdaptiveRouterUpdateQueue.flush_state_to_db` persists accumulated **deltas**, not absolute posteriors (Prisma `{"increment": ...}`, so multiple pods don't clobber each other), and `_compute_bandit_delta` maps satisfaction to `+alpha` only. So:

- a **satisfaction-only** cell persists `beta = 0` - a **negative-only** cell persists `alpha = 0`

On reload these become `BanditCell(alpha, 0)` / `BanditCell(0, beta)` — the cold-start prior (which guarantees `beta >= 0.5` via `initial_cell`) is discarded. Then `thompson_sample` → `random.betavariate(alpha, beta)` raises whenever `alpha <= 0` or `beta <= 0`. Because `pick_best` samples **every** pool model's cell for the classified `request_type`, one poisoned cell 500s **all** requests of that request type until the state table …

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d18h5m47s ago — entered · #import:https:::github.com:berriai:litellm post #2857
The left issue requires correcting persisted-state semantics across initialization, incremental updates, restart behavior, and multi-worker consistency, with broader regression and compatibility risk. The right issue is a localized streaming assembly condition with comparatively narrow test coverage and implementation scope.
#0 of 0 · 31d18h4m17s ago — current · #import:https:::github.com:berriai:litellm post #2885
32574 is harder because it involves coordinating retry/fallback state, shared logging lifecycle, callback semantics, and regression coverage across multiple execution paths; 29397 is comparatively localized to persisted-state reconstruction and validation.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search