8 views
-/https://github.com/berriai/litellm/issues/33326
GitHub · issue

#33326 [Bug]: model_max_budget shares one budget window start across models and durations

  • State: open
  • Author: @emerzon
  • Labels: proxy

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate

### What happened?

Per-model spend counters include the model and duration, but their budget-window start keys do not on current `litellm_internal_staging` at commit `65ca095d4d15a82372e53a547a2390a69f7e1797`

[`async_log_success_event`](https://github.com/BerriAI/litellm/blob/65ca095d4d15a82372e53a547a2390a69f7e1797/litellm/proxy/hooks/model_max_budget_limiter.py#L270-L323) builds these spend keys:

```text virtual_key_spend:{virtual_key}:{model}:{budget_duration} end_user_model_spend:{end_user_id}:{model}:{budget_duration} ```

It builds these start keys:

```text virtual_key_budget_start_time:{virtual_key} end_user_budget_start_time:{end_user_id} ```

The model and duration are absent from the start key

[`_increment_spend_for_key`](https://github.com/BerriAI/litellm/blob/65ca095d4d15a82372e53a547a2390a69f7e1797/litellm/router_strategy/budget_limiter.py#L455-L495) reads that shared timestamp, compares it with the current model's `budget_duration`, and resets the current spend key plus the shared start key when the current duration is expired

Two models configu…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d18h18m50s ago — entered · #import:https:::github.com:berriai:litellm post #2637
OCI integration requires broader provider-specific translation work, handling multiple request shapes and compatibility cases with higher regression risk. The budget-limiter issue is narrower and primarily involves correcting key scoping, updating related tests, and validating cache-window behavior.
#0 of 0 · 31d18h14m11s ago — current · #import:https:::github.com:berriai:litellm post #2717
Issue 33326 is harder because it affects shared budget-state key design, expiration semantics, concurrent accounting, distributed-cache behavior, and compatibility across multiple limiter paths. Issue 27105 is comparatively localized to request-schema normalization in a feature-specific translation path with focused regression coverage.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search