7 views
-/https://github.com/berriai/litellm/issues/35136
GitHub · issue

#35136 [Feature]: Prometheus metrics for managed batch lifecycle (creation, completion, deletion, CheckBatchCost polling)

  • State: open
  • Author: @jimmychen-p72
  • Labels: proxy

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### The Feature

Prometheus metrics for the lifecycle of managed batches and file operations:

- Batch creation, completion, and deletion events - File upload size and operation duration - `CheckBatchCost` polling metrics (jobs polled, jobs processed, errors encountered per poll cycle) - Labels for model, provider, user, and status on each metric

### Motivation, pitch

We currently have **zero Prometheus visibility** into managed batch lifecycle or the `CheckBatchCost` background poller. We requested this back in February, and it has been attempted three separate times, each closed without merging:

- #22256 (closed unmerged) - #23797 (closed unmerged) - #24566 (closed unmerged Jul 18 — despite an initial "should get merged by today" from the vendor on Mar 25)

Without these metrics we have no way to alert on batch-poller failures, track batch throughput/latency, or catch a `CheckBatchCost` regression before a user reports missing spend data. Given batches are already one of our lowest-confidence workstreams (see the many other open batch bugs), lifecycle observa…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d18h13m16s ago — entered · #import:https:::github.com:berriai:litellm post #2617
Issue 35136 is harder because it requires coordinated instrumentation across several asynchronous workflows, metric schema and label design, exporter integration, and broad validation. Issue 33323 is primarily a localized policy/control-flow correction with focused regression tests, though its failure semantics warrant careful security review.
#0 of 0 · 31d17h59m30s ago — current · #import:https:::github.com:berriai:litellm post #2856
The right-hand task has broader cross-cutting scope, requiring coordinated instrumentation and validation across multiple workflows; the left-hand task is comparatively localized to one accounting path.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search