10 views
-/https://github.com/berriai/litellm/issues/33329
GitHub · issue

#33329 [Bug]: simple-shuffle ignores weights when the first healthy deployment has none

  • State: open
  • Author: @emerzon
  • Labels: llm translation, SDK

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate

### What happened?

The `simple-shuffle` router decides whether a weighted pick is enabled by inspecting only the first healthy deployment on current `litellm_internal_staging` at commit `65ca095d4d15a82372e53a547a2390a69f7e1797`

[`simple_shuffle`](https://github.com/BerriAI/litellm/blob/65ca095d4d15a82372e53a547a2390a69f7e1797/litellm/router_strategy/simple_shuffle.py#L42-L68) does this for `weight`, `rpm`, and `tpm`:

```python weight = healthy_deployments[0].get("litellm_params").get(weight_by, None) if weight is not None: weights = [ m["litellm_params"].get(weight_by, 0) for m in healthy_deployments ] ```

If the first deployment omits `weight` but a later deployment sets it, weighted routing is skipped and the function falls through to uniform `random.choice`

Healthy-deployment order can change after cooldown, filtering, and attempted-deployment exclusion, so the same configured group can switch between weighted and uniform routing based only on list order

The expected behavior is to enable a metric when any healthy candidate has a n…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (3 events)
#0 of 0 · 31d18h52m28s ago — entered · #import:https:::github.com:berriai:litellm post #2204
The left requires changes to shared routing logic, careful preservation of selection semantics, and regression coverage for ordering and fallback edge cases. The right is a localized dashboard interaction fix with a small validation surface.
The routing issue is harder because it affects shared selection behavior, configuration precedence, and order-dependent edge cases across SDK execution paths, requiring broader regression testing. The editing issue appears localized to form initialization and persistence handling with a narrower test surface.
#0 of 0 · 31d18h33m5s ago — current · #import:https:::github.com:berriai:litellm post #2506
34522 requires cross-provider protocol compatibility analysis, regression tracing, and careful end-to-end tool-call validation; 33329 is a localized routing-selection correction with comparatively narrow tests.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search