7 views
-/https://github.com/berriai/litellm/issues/35524
GitHub · issue

#35524 [Bug]: Budget reservation is skipped when request cost cannot be estimated

  • State: open
  • Author: @emerzon

### Summary

The optimistic budget reservation path returns without reserving anything when it cannot estimate a positive maximum request cost. This is explicit current behavior, but it leaves configured budgets exposed to concurrent overspend for routes or models whose cost cannot be estimated before dispatch

### Current behavior

`litellm/proxy/spend_tracking/budget_reservation.py` calls `estimate_request_max_cost()` from `reserve_budget_for_request()`. If the model is unknown to the cost map, the request has no token-priced cost fields, or the request is an image or audio route that the estimator cannot price, the function returns `None`. `reserve_budget_for_request()` then returns without creating a reservation

The source comment describes this as falling back to read-time enforcement. Read-time checks are not an atomic admission decision, so concurrent requests can all observe the same spend before any of their final spend updates are committed

The `fail_closed_budget_enforcement` setting does not change this missing-estimate branch because it handles failures while reading or reserving a known counter, not the absence of an estimate

### Reproduction

Configure a key, team…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (3 events)
#0 of 0 · 31d18h56m12s ago — entered · #import:https:::github.com:berriai:litellm post #1927
#35524 requires designing a reliable atomic reservation fallback across unknown-cost and multimodal requests, preserving accounting correctness under concurrency and adding broad regression coverage. #31260 is primarily a scoped propagation and cache-metadata consistency fix across a few synchronous paths.
Budget admission changes require designing safe semantics for unknown pricing across concurrent and distributed requests, preserving existing enforcement modes, and adding broad regression coverage. The streaming fix is narrower: contain the hook failure and emit a protocol-compliant terminal SSE response, mainly requiring careful async-generator and client-compatibility tests.
#0 of 0 · 31d17h24m49s ago — current · #import:https:::github.com:berriai:litellm post #3458
34328 is harder because it requires algorithmic safeguards, multiple integration-point changes, termination/resource guarantees, and cross-provider regression testing; 35524 is comparatively localized to budget-admission logic and its concurrency-focused tests.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search