8 views
-/https://github.com/berriai/litellm/issues/35568
GitHub · issue

#35568 [Bug]: Recovered partial stream cost is released from live budget counters

  • State: open
  • Author: @emerzon
  • Labels: proxy

### Check for existing issues

- [x] I searched open and closed issues before filing - [x] [#14457](https://github.com/BerriAI/litellm/issues/14457) covers missing usage recovery for terminated streams, not budget-counter reconciliation after cost has been recovered

### What happened?

The failure hook now recovers partial usage and cost from a stream that failed after delivering chunks, but the budget reservation is finalized as zero before that recovered cost is processed

`async_post_call_failure_hook` first calls `release_budget_reservation`, which reconciles every reserved counter to `actual_cost=0`. Later in the same hook, a valid `combined_usage_object` causes `recovered_response_cost` to be passed to `update_database`

The database writer does more than the original failure report suggested. It writes a failure spend log and schedules durable key, user, team, organization, end-user, and related entity updates. The missing part is the live enforcement path: the failure hook never calls `increment_spend_counters` or reconciles the reservation to `recovered_response_cost`

Until the asynchronous database updates flush and a later counter repair observes them, the shared budge…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d18h35m33s ago — entered · #import:https:::github.com:berriai:litellm post #2265
#35568 is harder because it spans asynchronous failure-hook sequencing, reservation reconciliation, live counter consistency, and durable persistence, creating concurrency and regression risks. #24152 is more localized to fallback decision handling and rate-limit error propagation, with a narrower testing surface.
#0 of 0 · 31d17h43m14s ago — current · #import:https:::github.com:berriai:litellm post #3144
The right issue is harder because it spans asynchronous accounting, reservation lifecycle, concurrency, and consistency across multiple enforcement paths. The left issue is comparatively localized to provider identification and targeted Bedrock coverage.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search