7 views
-/https://github.com/berriai/litellm/issues/31869
GitHub · issue

#31869 [Bug]: ResetBudgetJob loads all expired teams into memory at once -> OOMKills & scheduler failure with large number of teams

  • State: open
  • Author: @kevalmahajan
  • Labels: bug, proxy

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate. - Closely related (keys, closed as stale): #13210 - "ResetBudgetJob loads entire tables into memory -> OOM" - This report is specifically about the **team** reset path, which has the same defect and is still present in `v1.89.4`.

## What happened?

We set **team-level budgets on a large number of teams (20k+ roughly) all with the same `budget_duration` / reset duration of `7d`**.

Because every team shares the same reset duration, all of their `budget_reset_at` timestamps cluster together, so they all become "due for reset" within the same scheduler tick. When the `ResetBudgetJob` fires:

- Proxy process RSS spikes dramatically in a single reset tick. - The pod is **OOMKilled** and **restarts**. - The `reset_budget_job` scheduler job **fails to complete** (and after restart, the same set of teams is still due, so the crash loop repeats on the next tick).

**What I expected to happen:** The scheduled team budget reset should run without loading the entire set of expired teams into memory at once. It should page/stream through the teams in bounded chunks so mem…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d17h44m6s ago — entered · #import:https:::github.com:berriai:litellm post #3139
The left requires database pagination/streaming, scheduler resilience, memory-safety validation, and regression testing at scale; the right has no actionable engineering scope.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search