6 views
-/https://github.com/berriai/litellm/issues/31880
GitHub · issue

#31880 fix(rate-limiter): skip post-call Redis writes for keys with no rate limits configured

  • State: open
  • Author: @deepanshululla

## Problem

After every LLM call, the rate limiter unconditionally writes counters to Redis even for API keys, users, or teams that have no rate limits configured. These Redis writes are wasted work: the counters are never read for enforcement because there are no limits to enforce. At high throughput this generates unnecessary Redis load and latency on every request.

## Proposed fix

Before the post-call Redis increment, check whether the relevant key/user/team has any rate limit configured. If none is set, skip the write entirely. This is safe because the counter value is only meaningful when there is a limit to compare it against.

## Files

- `litellm/proxy/hooks/` or `litellm/proxy/utils.py` (rate limiter post-call logic)

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d19h4m32s ago — entered · #import:https:::github.com:berriai:litellm post #1787
The right-hand task spans streaming control flow, provider translation, interception behavior, and regression coverage across multiple execution paths. The left-hand task is comparatively localized: add a guarded post-call persistence decision and validate rate-limit edge cases.
#0 of 0 · 31d18h13m54s ago — current · #import:https:::github.com:berriai:litellm post #2616
The left task is harder because it requires a robust recursive transformation of arbitrary nested schemas, careful provider-specific behavior, and broad regression coverage. The right task is comparatively localized: add a guarded post-call persistence path while preserving existing limiter semantics.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search