6 views
-/https://github.com/berriai/litellm/issues/32223
GitHub · issue

#32223 [Question] dynamic_rate_limiter_v3: recommended default_priority config, over-allocation behavior, and per-model priority reservation

  • State: open
  • Author: @Haoxi2002

Hi team, I'm setting up dynamic_rate_limiter_v3 with priority reservation and have three short questions.

1. What happens when weights over-allocate? If priority_reservation sums to > 1.0, I understand named weights get auto-normalized. But since default_priority (default 0.5) is an independent pool and not part of that normalization, the effective total (named priorities + default pool) can exceed 100%.

My understanding is that this is fine because model capacity is always enforced at 100% via the model_saturation_check descriptor, so no real over-subscription happens — the pools just compete under that hard ceiling. Can you confirm this is the intended behavior?

2. Is default_priority: 0 + mandatory per-team priority the recommended pattern? There's an inconsistency I'd like clarified: the docs example uses default_priority: 0, while the code/README default is 0.5.

For production, is the recommendation to set default_priority: 0 and require every team/key to carry an explicit priority (so untagged traffic gets no reserved share under saturation), or is relying on the shared default pool the intended design (given spillover tests use default_priority = 0.25)? A note in the doc…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d19h15s ago — entered · #import:https:::github.com:berriai:litellm post #1961
The right issue requires a targeted provider-path code change, compatibility validation, and regression coverage across request transformations. The left issue is primarily configuration semantics and documentation clarification with comparatively limited engineering scope.
#0 of 0 · 31d18h24m9s ago — current · #import:https:::github.com:berriai:litellm post #2543
The left requires new proxy configuration plumbing, validation, lifecycle handling, and deployment-compatible integration, while the right is primarily behavioral clarification and documentation with little or no code change.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search