9 views
-/https://github.com/berriai/litellm/issues/31821
GitHub · issue

#31821 [Feature]: Monthly call count limits with overflow buffer for virtual keys

  • State: open
  • Author: @limitjupyter-ops
  • Labels: enhancement, proxy

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### The Feature

I would like LiteLLM to support monthly call count limits for virtual keys, independent from spend budgets, RPM, and TPM limits.

The feature should allow admins to configure fields such as:

- `monthly_call_limit` - `monthly_call_overflow` - `current_month_call_count` - `monthly_call_warning_threshold` - `monthly_call_window_timezone`

Desired behavior:

1. A virtual key can have a fixed number of calls allowed per month. 2. LiteLLM checks the monthly call count before forwarding a request upstream. 3. LiteLLM rejects requests once the key exceeds `monthly_call_limit + monthly_call_overflow`. 4. LiteLLM can emit a warning status when usage is near the limit. 5. The monthly window can reset by calendar month, with configurable timezone support. 6. Admin APIs and the Admin UI expose used calls, remaining calls, warning status, and reset window.

This should be separate from token spend budgets and RPM/TPM limits, because some enterprise contracts are based on request counts rather than token cost.

### Motivation, pitch

Many enterprise AI gateway …

Open on GitHub

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (21 events)
#0 of 0 · 31d18h52m31s ago — entered · #import:https:::github.com:berriai:litellm post #2309
-/https://github.com/berriai/litellm/issues/31821⭐8:1-/https://github.com/berriai/litellm/issues/33780
The left issue is substantially harder because it spans persistent usage accounting, concurrent request enforcement, calendar/timezone semantics, rollover behavior, configuration surfaces, APIs, UI, and backward compatibility. The right issue is a narrowly scoped integration change affecting configuration, request signing, and focused tests.
#0 of 0 · 31d18h44m33s ago · #import:https:::github.com:berriai:litellm post #2423
-/https://github.com/berriai/litellm/issues/31821⭐5:1-/https://github.com/berriai/litellm/issues/26240
The right-hand task is substantially harder because it requires coordinated quota accounting, persistence, calendar-window and timezone semantics, concurrency-safe enforcement, warning behavior, and API/UI integration. The left-hand task is narrower, primarily involving request compatibility diagnosis and a targeted translation fix.
#0 of 0 · 31d17h56m20s ago · #import:https:::github.com:berriai:litellm post #3252
-/https://github.com/berriai/litellm/issues/3126320:1-/https://github.com/berriai/litellm/issues/31821⭐
The right issue is substantially harder because it entails a broad, cross-cutting systems migration with performance, compatibility, and operational risks. The left issue is a bounded proxy feature involving usage accounting, persistence, APIs, and UI integration.
#0 of 0 · 31d17h54m18s ago · #import:https:::github.com:berriai:litellm post #3286
-/https://github.com/berriai/litellm/issues/278833:2-/https://github.com/berriai/litellm/issues/31821⭐
The left item spans multiple external tool ecosystems and requires designing secure server-side configuration delivery and compatibility behavior, creating substantial integration and policy risk. The right item is broader than a simple counter because it touches persistence, concurrency, time windows, APIs, and UI, but it remains a more bounded gateway quota feature. Thus the left is moderately harder.
#0 of 0 · 31d17h54m12s ago · #import:https:::github.com:berriai:litellm post #3288
-/https://github.com/berriai/litellm/issues/31821⭐3:1-/https://github.com/berriai/litellm/issues/34733
The right issue is harder because it spans persistent data modeling, distributed request enforcement, calendar/time-zone semantics, API contracts, UI work, migrations, and broad testing. The left issue is a focused concurrency and write-order correction within an existing accounting path, though it carries meaningful distributed-systems risk.
#0 of 0 · 31d17h54m9s ago · #import:https:::github.com:berriai:litellm post #3289
-/https://github.com/berriai/litellm/issues/31821⭐4:1-/https://github.com/berriai/litellm/issues/31568
Monthly call limits require coordinated changes across quota state, persistence, concurrency-safe enforcement, calendar/timezone handling, warning semantics, APIs, and UI. The provider addition is narrower, mainly involving routing, model metadata, cost attribution, and integration tests.
#0 of 0 · 31d17h54m5s ago · #import:https:::github.com:berriai:litellm post #3290
-/https://github.com/berriai/litellm/issues/31821⭐5:3-/https://github.com/berriai/litellm/issues/30460
31821 is harder because it requires coordinated changes across persistence, request enforcement, calendar-window semantics, APIs, and UI, while 30460 is primarily a focused distributed-state debugging and correctness fix.
#0 of 0 · 31d17h53m44s ago · #import:https:::github.com:berriai:litellm post #3296
-/https://github.com/berriai/litellm/issues/304213:2-/https://github.com/berriai/litellm/issues/31821⭐
The right issue is harder because it spans CLI design, subprocess orchestration, authentication, remote capability discovery, interactive selection, and compatibility across several external agent ecosystems. The left issue is substantial but more contained within LiteLLM’s existing quota, persistence, API, and UI layers.
#0 of 0 · 31d17h52m58s ago · #import:https:::github.com:berriai:litellm post #3309
-/https://github.com/berriai/litellm/issues/31821⭐3:1-/https://github.com/berriai/litellm/issues/30641
The left task spans quota semantics, persistent usage accounting, calendar and timezone handling, concurrency safety, API changes, UI work, and distributed-system edge cases. The right task is primarily configuration discovery, safe presentation, and dashboard integration, with a narrower behavioral scope.
#0 of 0 · 31d17h52m54s ago · #import:https:::github.com:berriai:litellm post #3310
-/https://github.com/berriai/litellm/issues/293204:1-/https://github.com/berriai/litellm/issues/31821⭐
29320 has substantially greater implementation risk and scope: it introduces a new model-backed request-path subsystem spanning SDK and proxy flows, lifecycle hooks, configuration, isolation, failure handling, performance controls, and observability. 31821 is primarily a quota and accounting feature with distributed consistency, calendar-boundary, API, and UI considerations, but fits more naturally into existing proxy budget infrastructure.
#0 of 0 · 31d17h51m37s ago · #import:https:::github.com:berriai:litellm post #3331
-/https://github.com/berriai/litellm/issues/31821⭐5:1-/https://github.com/berriai/litellm/issues/26535
The left task is harder because it requires new stateful quota infrastructure, calendar-window semantics, concurrency-safe enforcement, persistence, API/UI changes, and broad regression coverage. The right task is primarily validation and test automation over an existing multi-provider stack, with less production-surface risk.
#0 of 0 · 31d17h51m34s ago · #import:https:::github.com:berriai:litellm post #3332
-/https://github.com/berriai/litellm/issues/31821⭐6:5-/https://github.com/berriai/litellm/issues/31824
Monthly call-limit enforcement is harder because it affects the request authorization path, requires accurate concurrent counting and reset semantics, and must integrate configuration, persistence, warnings, and administration surfaces. Self-service usage mainly extends existing reporting and UI capabilities with scoped authorization and aggregation.
#0 of 0 · 31d17h50m8s ago · #import:https:::github.com:berriai:litellm post #3357
-/https://github.com/berriai/litellm/issues/318355:4-/https://github.com/berriai/litellm/issues/31821⭐
Historical financial-data mutation carries greater correctness, auditability, idempotency, and backward-compatibility risk, especially across pricing versions and provider-specific outcomes. The other feature is broader across storage, enforcement, APIs, and UI, but follows a more straightforward request-gating pattern.
#0 of 0 · 31d17h48m58s ago · #import:https:::github.com:berriai:litellm post #3374
-/https://github.com/berriai/litellm/issues/31821⭐5:1-/https://github.com/berriai/litellm/issues/9753
The left issue spans data modeling, enforcement logic, calendar-boundary handling, concurrency, APIs, UI, and testing. The right issue is primarily an architectural/dependency evaluation with uncertain implementation scope.
#0 of 0 · 31d17h46m29s ago · #import:https:::github.com:berriai:litellm post #3416
-/https://github.com/berriai/litellm/issues/252973:2-/https://github.com/berriai/litellm/issues/31821⭐
The left requires a broader extension architecture across proxy request flow, strategy injection, compatibility, and operational behavior, creating more integration and regression risk. The right is a bounded quota feature with focused persistence, enforcement, API, and UI changes.
#0 of 0 · 31d17h46m22s ago · #import:https:::github.com:berriai:litellm post #3418
-/https://github.com/berriai/litellm/issues/31821⭐5:3-/https://github.com/berriai/litellm/issues/28125
Monthly quota enforcement spans persistent schema changes, atomic/distributed usage accounting, calendar-time and timezone edge cases, request-path integration, API/UI exposure, and backward compatibility. The composition change is primarily configuration parsing and model-group resolution, though it still carries routing and validation risks.
#0 of 0 · 31d17h44m30s ago · #import:https:::github.com:berriai:litellm post #3448
-/https://github.com/berriai/litellm/issues/254475:3-/https://github.com/berriai/litellm/issues/31821⭐
The right issue is harder because it requires diagnosing and correcting nondeterministic cross-request isolation in a distributed, concurrent Redis-backed deployment, with significant security and regression risk. The left issue spans quota accounting, calendar/timezone handling, APIs, and UI, but has a more bounded design and implementation path.
#0 of 0 · 31d17h42m14s ago · #import:https:::github.com:berriai:litellm post #3485
-/https://github.com/berriai/litellm/issues/31821⭐5:4-/https://github.com/berriai/litellm/issues/28235
The left task spans a new quota metric, calendar-based and timezone-aware lifecycle handling, overflow and warning semantics, enforcement, persistence, and coordinated API/UI exposure. The right task primarily extends existing budget-window, reservation, counter, and reset infrastructure to another entity scope, with concurrency and migration considerations but less new surface area.
#0 of 0 · 31d17h42m7s ago · #import:https:::github.com:berriai:litellm post #3487
-/https://github.com/berriai/litellm/issues/31821⭐3:1-/https://github.com/berriai/litellm/issues/29452
The left requires coordinated changes across request enforcement, durable usage accounting, concurrency safety, time-window semantics, APIs, and UI, creating substantially broader integration and migration risk. The right is primarily an authentication abstraction and provider-integration effort, with narrower runtime and data-consistency impact.
#0 of 0 · 31d17h40m41s ago · #import:https:::github.com:berriai:litellm post #3510
-/https://github.com/berriai/litellm/issues/31821⭐4:1-/https://github.com/berriai/litellm/issues/33772
The left issue is substantially harder because it requires coordinated quota state, calendar and timezone semantics, concurrency-safe enforcement, persistence changes, API/UI work, and broad integration testing. The right issue is comparatively localized to cost parsing, model metadata, and regression tests.
#0 of 0 · 31d17h38m40s ago — current · #import:https:::github.com:berriai:litellm post #3535
-/https://github.com/berriai/litellm/issues/31821⭐5:2-/https://github.com/berriai/litellm/issues/27755
#31821 requires coordinated persistence, atomic accounting, time-window semantics, API/UI changes, and broad testing, while #27755 is primarily an isolated provider-integration investigation with external-service uncertainty.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search