8 views
-/https://github.com/berriai/litellm/issues/33330
GitHub · issue

#33330 [Bug]: virtual-key soft budgets use stale spend and ignore a zero threshold

  • State: open
  • Author: @emerzon
  • Labels: proxy

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate

### What happened?

The virtual-key soft-budget check has two independent boundary/data-source defects on current `litellm_internal_staging` at commit `65ca095d4d15a82372e53a547a2390a69f7e1797`

[`_virtual_key_soft_budget_check`](https://github.com/BerriAI/litellm/blob/65ca095d4d15a82372e53a547a2390a69f7e1797/litellm/proxy/auth/auth_checks.py#L3599-L3628) uses:

```python if valid_token.soft_budget and valid_token.spend >= valid_token.soft_budget: ```

First, `soft_budget=0` is treated as unset because zero is falsy

The key and budget management validation accepts every non-negative finite soft budget, including zero, so zero is a valid threshold but can never alert on this path

Second, the comparison uses `valid_token.spend`, which is the spend value carried by the cached key object

The hard virtual-key budget check has moved to the Redis-first current-spend counter, but the soft-budget check does not read that counter. In a multi-replica deployment or during cache/DB lag, the soft alert can therefore remain below threshold after shared current spend has crosse…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d18h44m49s ago — entered · #import:https:::github.com:berriai:litellm post #2115
Issue #29292 spans shared persistence and retry/batching behavior across multiple spend dimensions, requiring broader correctness, load, and regression testing. Issue #33330 is comparatively localized to budget validation and choosing the authoritative spend source, though distributed-state behavior adds some risk.
#0 of 0 · 31d18h27m35s ago — current · #import:https:::github.com:berriai:litellm post #2399
Streaming response metadata must remain consistent across fallback routing, async chunk generation, and provider-specific wrappers, creating broader integration risk. The budget issue is more localized to validation, spend-source selection, and boundary-condition tests.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search