6 views
-/https://github.com/berriai/litellm/issues/34326
GitHub · issue

#34326 [Feature]: Partition DDL in SpendLogs cleanup causes service outage when it collides with routine PostgreSQL maintenance

  • State: open
  • Author: @oskar-dba
  • Labels: enhancement, proxy

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### The Feature

## Proposal

Allow the partition management path (`ensure_partitions` / `drop_partitions_older_than`) to run with `lock_timeout` and `statement_timeout` applied — scoped only to those DDL statements, so no other queries are affected. Sensible would be exposing them as env vars (or `general_settings` keys) with conservative defaults such as `lock_timeout=3s`, `statement_timeout=5s`.

### Motivation, pitch

## What happened in production

We run `vacuumdb --freeze` as a nightly cron on our Postgres cluster — standard practice to prevent transaction ID wraparound. On one night, `vacuumdb --freeze` on `LiteLLM_SpendLogs` overlapped with the litellm partition cleanup tick.

Timeline: 1. `vacuumdb --freeze` starts, holds `ShareUpdateExclusiveLock` on `LiteLLM_SpendLogs` for the duration of the freeze pass. 2. litellm's `SpendLogsPartitionManager` fires and issues `DROP TABLE IF EXISTS "LiteLLM_SpendLogs_pYYYYMMDD"`, which needs `AccessExclusiveLock` on the parent to update `pg_inherits`. 3. The DDL is blocked waiting for VACUUM. Postgres's lock queue n…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (3 events)
#0 of 0 · 31d19h43s ago — entered · #import:https:::github.com:berriai:litellm post #2064
The left requires careful database-session scoping, timeout configuration, failure handling, concurrency validation, and regression testing around production-sensitive DDL. The right is comparatively contained provider-model recognition and embedding translation work.
34326 is harder because it requires careful PostgreSQL lock/timeout scoping, concurrency-safe lifecycle handling, configurable behavior, and regression testing under production-like contention. 32736 is comparatively localized to configuration-precedence and refresh-merge logic.
#0 of 0 · 31d17h39m36s ago — current · #import:https:::github.com:berriai:litellm post #3420
#34733 requires distributed concurrency-safe accounting, atomic window transitions, and reliable async write coordination, with race-focused tests. #34326 is primarily scoped timeout configuration and application around existing PostgreSQL DDL operations.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search