6 views
-/https://github.com/berriai/litellm/issues/34140
GitHub Β· issue

#34140 v3 rate limiter double-counts team per-model limits (model_per_team) β†’ effective RPM/TPM is half configured

  • State: open
  • Author: @almogx3

Repository owner assessment aside β€” filing precise repro + root cause.

### What happened

Per-team per-model rate limits set via `POST /team/update {metadata: {model_rpm_limit: {...}}}` are enforced at **half** the configured value. A team-model limit of `N` starts returning 429 after ~`N/2` requests per window.

### Version `v1.83.14-stable` (also present on `main` as of filing β€” both code paths below still exist).

### Root cause

In `litellm/proxy/hooks/parallel_request_limiter_v3.py`, `async_pre_call_hook` builds the descriptor list via **two** paths that each append the *same* `model_per_team` descriptor from the *same* `team_metadata["model_rpm_limit"]`:

1. `_create_rate_limit_descriptors()` β†’ team block (~line 1011, via `get_team_model_rpm_limit()`), appends `RateLimitDescriptor(key="model_per_team", value=f"{team_id}:{model}")`. 2. `_add_team_model_rate_limit_descriptor_from_metadata()` (~line 1125, called from `async_pre_call_hook` ~line 1336), appends the **identical** descriptor.

`should_rate_limit()` then puts that counter key into `keys_to_fetch` **twice** (no dedup), and `BATCH_RATE_LIMITER_SCRIPT` (`for i = 1, #KEYS, 2`) does `INCR` per pair β†’ the counter is incre…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 Β· 31d18h27m58s ago β€” entered Β· #import:https:::github.com:berriai:litellm post #2418
The license-verification fix is harder because it involves parsing and compatibility of a signed wire format, with broader security-sensitive testing and migration risk. The rate-limiter issue appears comparatively localized to descriptor construction and regression coverage.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search