8 views
-/https://github.com/berriai/litellm/issues/34238
GitHub · issue

#34238 [Bug]: End-user budget can be bypassed via stale per-pod fallback spend after Redis counter clean-miss

  • State: open
  • Author: @FenterChen
  • Labels: bug, proxy

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

Budget enforcement for end users reads a cross-pod Redis counter, `spend:end_user:{user_id}`. When that counter is missing (default TTL is 60s, `BaseCache.default_ttl`), `SpendCounterReseed.from_db()` is supposed to reseed it from the persisted spend in `LiteLLM_EndUserTable` before enforcement runs, the same way it already does for key, team, team-member, user, and org counters. The end-user branch currently returns `None` unconditionally instead of querying the database: ```python elif counter_key.startswith("spend:end_user:"): return None ``` Because of this, enforcement falls back to `fallback_spend`, a per-pod cached value (`end_user_id:{user_id}.spend` in `user_api_key_cache`) governed by `general_settings.user_api_key_cache_ttl`. Each pod refreshes this cache independently, so at the moment the Redis counter disappears, whichever pod handles the next request may reconstruct it using a value that is lower than both the database and other pods' state.

### Steps to Reproduce

1. In a multi-pod deployment, create an end user with …

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d18h40m45s ago — entered · #import:https:::github.com:berriai:litellm post #2186
The orchestration feature is substantially harder because it spans gateway architecture, routing policy design, provider integrations, failure handling, configuration, observability, and extensive evaluation/testing. The other issue is a comparatively localized correctness fix involving an existing persistence and cache path.
#0 of 0 · 31d18h4m51s ago — current · #import:https:::github.com:berriai:litellm post #2766
The right issue is harder due to distributed-state consistency, persistence coordination, race conditions, security implications, and broader regression-testing requirements. The left issue is comparatively localized to response normalization and type validation within one provider integration.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search