8 views
-/https://github.com/berriai/litellm/issues/35354
GitHub · issue

#35354 [Bug]: CheckBatchCost never marks a terminal batch with no output file as processed — row re-polled forever

  • State: open
  • Author: @sudhanshugupta-p72
  • Labels: bug, proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

A managed batch whose request lines all failed reaches a terminal provider status (`completed`) with `output_file_id=None` and only an `error_file_id`. The `CheckBatchCost` background job never transitions that row: `batch_processed` stays `false` indefinitely, so the job query re-selects the same row on every polling cycle for the lifetime of the deployment.

There is no exception. The provider retrieve succeeds and returns a correct terminal status — the row is simply never handled, so nothing surfaces in the logs beyond the row being picked up again each cycle.

Two consequences: - Wasted DB queries and poller work that compound as more such batches accumulate. - Output-file deletion is gated on `batch_processed`, so the batch's output and error files can never be deleted through the managed API.

**Expected:** once the poller observes a terminal provider status, the row is marked terminal (`batch_processed=true`) exactly once. A batch with no output file has no billable completion output, so it should be marked terminal *without* being bill…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (4 events)
#0 of 0 · 31d18h31m15s ago — entered · #import:https:::github.com:berriai:litellm post #2529
The left issue is harder because it requires careful terminal-state handling, persistence semantics, idempotency, and regression coverage in a background workflow, while the right issue is primarily a narrowly scoped response-filtering change.
The left issue is harder because it spans database serialization, backward-compatible normalization, configuration update behavior, proxy reload integration, and regression testing. The right issue is comparatively localized to terminal-state handling in a background poller with focused edge-case tests.
17696 requires provider-aware request transformation, token-threshold handling, and careful interaction with retries and caching semantics; 35354 appears comparatively localized to terminal-state bookkeeping with focused regression coverage.
#0 of 0 · 31d17h52m10s ago — current · #import:https:::github.com:berriai:litellm post #3209
The right issue is harder because it requires changes to asynchronous batch-state handling, persistence semantics, cleanup behavior, and regression coverage, with greater risk of duplicate processing or lifecycle regressions. The left issue is comparatively localized to dependency packaging and image validation.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search