7 views
-/https://github.com/berriai/litellm/issues/35131
GitHub · issue

#35131 [Bug]: CheckResponsesCost background job never clears LiteLLM_ManagedObjectTable rows stuck in queued

  • State: open
  • Author: @jimmychen-p72
  • Labels: proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

When using the Responses API with `background=true`, the resulting row in `LiteLLM_ManagedObjectTable` gets stuck in `queued` status **permanently**, even after the response has actually completed at the provider. Confirmed on both Azure and native OpenAI deployments — same failure on both. In our environment we have 20+ rows stuck as `queued`, some dating back over two months.

Since the rows never transition out of `queued`, the `CheckResponsesCost` background job keeps picking them up on every polling cycle and fails the same way each time. This means: - No cost is ever attributed for these background responses. - The number of stuck rows only grows as more `background=true` requests come in, so wasted DB queries and log noise compound over time.

### Steps to Reproduce

1. `POST /v1/responses` with `background=true` (any model). 2. Poll `GET /v1/responses/{id}` until `status=completed`. 3. Check the DB: ```sql SELECT status FROM "LiteLLM_ManagedObjectTable" WHERE file_purpose = 'response' ORDER BY created_at DESC LIMIT 5; ``` 4. Ob…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (5 events)
#0 of 0 · 31d19h14m30s ago — entered · #import:https:::github.com:berriai:litellm post #1631
The right-hand issue is harder because it requires protocol-aware routing across aggregated MCP backends, preserving existing multi-server behavior and adding integration coverage. The left-hand issue is more localized to persistence-state reconciliation and background cost-processing logic.
Issue #35131 is harder because it spans asynchronous response lifecycle handling, provider-specific completion behavior, background polling, persistence state transitions, cost calculation, retry/cleanup logic, and regression testing. Issue #30825 is comparatively localized to callback-specific metadata isolation and its surrounding tests.
The right issue is harder because it requires tracing an asynchronous lifecycle across API handling, persistence, polling, provider-specific completion states, cost accounting, retries, and regression coverage. The left issue is a localized performance refactor with a comparatively narrow implementation and validation surface.
Asynchronous state reconciliation spans response persistence, polling, cost attribution, and multiple provider paths, creating broader correctness and regression risk than a localized MCP header-forwarding fix.
#0 of 0 · 31d18h34m40s ago — current · #import:https:::github.com:berriai:litellm post #2288
The left issue spans multiple providers, model families, API mappings, parameter translation, multimodal behavior, cost metadata, and both SDK and proxy integration, creating substantially broader implementation and validation risk. The right issue is a focused proxy background-job state and persistence bug with a narrower investigation and fix surface.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search