6 views
-/https://github.com/berriai/litellm/issues/34297
GitHub · issue

#34297 UI: per-model rate limits (model_rpm_limit/model_tpm_limit) not settable on internal users

  • State: open
  • Author: @devin-ai-integration[bot]

## What happened

Per model rate limits (`model_rpm_limit` / `model_tpm_limit`) can be set on keys (key edit view) and on teams (`TeamInfo.tsx`, "Set per-model TPM/RPM limits that apply across the whole team"), but there is no UI to set them on internal users

The backend already accepts `model_rpm_limit` / `model_tpm_limit` on `NewUserRequest` and `UpdateUserRequest` (via `GenerateRequestBase`), so this is a missing UI surface for users

(For context, a broader report claimed per model limits work "only on keys, not users or teams". Teams do have it, so the real gap is users only.)

## Possible secondary bug on teams

In `TeamInfo.tsx` the save path sends `model_tpm_limit` / `model_rpm_limit` at the top level of the update payload, but the display path reads them back from `info.metadata?.model_tpm_limit` / `metadata?.model_rpm_limit`. If the backend does not mirror these into `metadata`, saved per model limits will not render back in the team view. Worth verifying end to end

## Repro

1. Virtual Keys -> edit key: has Model TPM/RPM Limit fields 2. Teams -> team info: has a per model TPM/RPM editor 3. Internal Users -> create/edit: no per model rate limit field

Confirmed against …

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d17h58m20s ago — entered · #import:https:::github.com:berriai:litellm post #2894
The harder task spans provider-specific model translation, Azure integration behavior, compatibility handling, and broader test coverage. The easier task is primarily a localized frontend form addition with limited end-to-end payload verification.
#0 of 0 · 31d17h49m31s ago — current · #import:https:::github.com:berriai:litellm post #3050
31439 requires deeper provider-translation and serialization-path analysis, with compatibility-sensitive backend changes and regression testing across conversation modes; 34297 is primarily a bounded frontend form and payload integration with limited verification.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search