6 views
-/https://github.com/berriai/litellm/issues/34479
GitHub · issue

#34479 [Bug]: All user API keys fail when PostgreSQL max_connections is exhausted in multi-worker proxy deployment

  • State: open
  • Author: @limitjupyter-ops
  • Labels: bug, proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

We run LiteLLM proxy with PostgreSQL as the database backend and multiple workers.

After the service had been running for some time under traffic, all normal user API keys started failing for every model call with:

```text openai.AuthenticationError: Error code: 401 - { "error": { "message": "Authentication Error, All connection attempts failed", "type": "auth_error", "code": "401" } }

### Steps to Reproduce

```markdown 1. Deploy LiteLLM proxy with PostgreSQL backend. 2. Configure PostgreSQL with `max_connections=300`. 3. Run LiteLLM with multiple workers, e.g. `--num_workers=8`. 4. Enable normal user API keys stored in PostgreSQL. 5. Generate concurrent chat completion traffic using normal user keys. 6. Run additional background services that also connect to the same DB, such as usage sync / log archive / custom quota callbacks. 7. Once PostgreSQL reaches `max_connections`, call:

```bash curl https://<proxy-host>/v1/chat/completions \ -H "Authorization: Bearer <normal-user-key>" \ -H "Content-Type: application/json" \ …

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d18h23s ago — entered · #import:https:::github.com:berriai:litellm post #2866
The left issue is harder because it involves cross-worker database connection management, resource exhaustion, failure isolation, and deployment-level testing. The right issue is more localized to retry classification and control flow.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search