6 views
-/https://github.com/berriai/litellm/issues/34367
GitHub · issue

#34367 [Feature]: Model Group Budget Limits for API Key Usage

  • State: open
  • Author: @irfanhanip
  • Labels: enhancement, proxy

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### The Feature

Add support for defining a shared budget at the model_group level for an API key. All models within the group should draw from the same combined budget.

### Motivation, pitch

Teams may want to limit developer spending on a family of expensive models, such as multiple Claude Opus versions. Today, budgets can be set per individual model, but each model receives its own limit. This means developers can exceed the intended total cap by using several models in the same expensive family. A model-group budget would allow configuring one shared limit across models—for example, a single monthly budget covering all Opus model versions—so usage is enforced against the combined spend rather than per-model spend.

### What part of LiteLLM is this about?

Proxy

### LiteLLM is hiring a founding backend engineer, are you interested in joining us and shipping to all our users?

No

### Twitter / LinkedIn details

_No response_

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (4 events)
#0 of 0 · 31d18h58m55s ago — entered · #import:https:::github.com:berriai:litellm post #1985
#34367 requires cross-model spend aggregation, budget enforcement, persistence, configuration, and edge-case handling across proxy request flows. #26320 is a narrower provider-specific request translation fix with a smaller testing surface.
The harder item requires coordinated proxy-side accounting, configuration semantics, persistence, concurrency-safe enforcement, and broad testing; the other is primarily explanatory/support work with little implementation scope.
The right issue is substantially harder because it requires coordinated proxy configuration, budget aggregation, persistence, enforcement, edge-case handling, and broad tests. The left is comparatively localized diagnostic and mapping work.
#0 of 0 · 31d17h56m36s ago — current · #import:https:::github.com:berriai:litellm post #3021
Model-group budget limits require cross-model accounting, configuration semantics, persistence, concurrency-safe enforcement, and broad proxy/test integration. The cancellation bug is narrower in scope, though async cleanup and failure-path coverage add implementation risk.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search