8 views
-/https://github.com/berriai/litellm/issues/32246
GitHub · issue

#32246 fix(prometheus): skip budget metric DB/cache lookups when all budget gauges are NoOpMetric

  • State: open
  • Author: @deepanshululla

## Problem

\`PrometheusLogger._increment_remaining_budget_metrics\` is called on every successful LLM API request and always fans out four async tasks via \`asyncio.gather\` — one each for API key, team, user, and org budget metrics. Each task calls an \`_assemble_*_object\` helper that performs a DB or cache lookup (\`get_team_object\`, \`get_key_object\`, \`get_user_object\`, \`get_org_object\`) to hydrate the entity record, then calls \`.set()\` on the corresponding Prometheus gauge.

When an operator uses \`prometheus_metrics_config\` to restrict which metrics are published, any metric not in the allowlist is created as a \`NoOpMetric()\` instance whose \`.set()\` is a silent no-op. However, neither \`_increment_remaining_budget_metrics\` nor any of the four per-entity helpers checks whether the target gauge is a \`NoOpMetric\` before doing the I/O. The entire DB/cache lookup pipeline runs, and every retrieved object is immediately discarded.

In deployments that deliberately exclude the budget gauges (\`litellm_remaining_team_budget_metric\`, \`litellm_remaining_api_key_budget_metric\`, \`litellm_remaining_user_budget_metric\`) from the allowlist, this generates four async DB…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d18h7m37s ago — entered · #import:https:::github.com:berriai:litellm post #2939
Issue #34281 is harder because it likely requires tracing health-check orchestration, async failure boundaries, error classification, and regression coverage across multiple integrations. Issue #32246 is comparatively localized to conditional metric handling with focused tests.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search