6 views
-/https://github.com/berriai/litellm/issues/33872
GitHub · issue

#33872 [Bug]: Redis spend buffer loses dequeued transactions when the DB commit fails

  • State: open
  • Author: @emerzon
  • Labels: proxy

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate

### What happened?

With `general_settings.use_redis_transaction_buffer: true`, the leader pod removes buffered spend transactions from Redis before their database commit is acknowledged. If the database write fails after its retries, the removed spend is neither restored nor retried

The scalable path calls `get_all_transactions_from_redis_buffer_pipeline`, then commits each returned category to Postgres in [`_commit_spend_updates_to_db_with_redis`](https://github.com/BerriAI/litellm/blob/3f9b71c1a45e870d1789ee105bd59b9274bb0d74/litellm/proxy/db/db_spend_update_writer.py#L770-L878). The drain is a destructive `async_lpop_pipeline` across the entity and daily queues in [`RedisUpdateBuffer`](https://github.com/BerriAI/litellm/blob/3f9b71c1a45e870d1789ee105bd59b9274bb0d74/litellm/proxy/db/db_transaction_queue/redis_update_buffer.py#L464-L545)

The enclosing exception handler only emits `Data already popped from Redis may be lost` and releases the leader lock. There is no processing queue, acknowledgment, `LTRIM` after success, or requeue path for the popped payload

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d18h36m39s ago — entered · #import:https:::github.com:berriai:litellm post #2352
The left issue requires coordinated changes to distributed queue processing, failure recovery, idempotency, locking, and database integration, plus substantial testing. The right issue contains too little actionable detail and appears to require minimal implementation effort.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search