7 views
-/https://github.com/berriai/litellm/issues/24152
GitHub · issue

#24152 [Bug]: Key-level per-model rate limits (model_rpm_limit/model_tpm_limit) don't trigger fallbacks

  • State: open
  • Author: @sapirm-eleos
  • Labels: bug, proxy

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

When an API key has per-model rate limits (model_rpm_limit / model_tpm_limit) and a request exceeds the limit for a specific model, the configured fallback model is never attempted. Instead, the request returns a 429 without trying the fallback. This was reported in #10052 which was closed without full resolution.

### Steps to Reproduce

1. Create an API key with per-model rate limits: ``` { "model_rpm_limit": { "gpt-5.1": 1, "gpt-5.2": 100000 } } ``` 2. Make a successful request to exhaust the 1 RPM budget: ``` client.chat.completions.create( model="gpt-5.1", messages=[{"role": "user", "content": "Say hi"}], extra_body={"fallbacks": ["gpt-5.2"]} ) # ✅ Succeeds ``` …

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (3 events)
#0 of 0 · 31d19h1m45s ago — entered · #import:https:::github.com:berriai:litellm post #2046
24152 requires coordinated changes across proxy quota enforcement, model selection, fallback control flow, and regression tests, while 23339 is more localized redirect interoperability work.
#35568 is harder because it spans asynchronous failure-hook sequencing, reservation reconciliation, live counter consistency, and durable persistence, creating concurrency and regression risks. #24152 is more localized to fallback decision handling and rate-limit error propagation, with a narrower testing surface.
#0 of 0 · 31d18h2m18s ago — current · #import:https:::github.com:berriai:litellm post #3033
The right-hand task is harder because it likely requires coordinating proxy rate-limit accounting with fallback orchestration, retry semantics, and broad regression coverage. The left-hand task is a comparatively localized schema and provider-translation change with narrower testing scope.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search