8 views
-/https://github.com/berriai/litellm/issues/16773
GitHub Ā· issue

#16773 [Bug]: Incorrect label count in increment_deployment_cooled_down

  • State: open
  • Author: @micahjsmith
  • Labels: bug, mlops user request, stale

### What happened?

Router is failing to updated the litellm_deployment_cooled_down prometheus metric due to a "incorrect label count" error. As a result, we have no data for this metric at all.

Looks like the set of labels is coming from https://github.com/BerriAI/litellm/blob/v1.78.5-stable/litellm/types/integrations/prometheus.py#L352-L357 (four of them) whereas the call is providing 5 labels. The `exception_status` is the one that is presumably not supported.

Happy to make a PR to fix this. This appears to still be an issue on `main` branch, see https://github.com/BerriAI/litellm/blob/4dbbec233cebd126c6286d574839e4922bff4ff4/litellm/router_utils/cooldown_callbacks.py#L79

Similar to #7611

### Relevant log output

```shell Task exception was never retrieved future: <Task finished name='Task-13231120' coro=<router_cooldown_event_callback() done, defined at /usr/lib/python3.13/site-packages/litellm/router_utils/cooldown_callbacks.py:24> exception=ValueError('Incorrect label count')>

Traceback (most recent call last): File "/usr/lib/python3.13/site-packages/litellm/router_utils/cooldown_callbacks.py", line 77, in router_cooldown_event_callback prometheusLogger.increment_d…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 Ā· 31d18h19m27s ago — entered Ā· #import:https:::github.com:berriai:litellm post #2734
The right issue is harder because it spans authentication, key-rotation state, database persistence, cache behavior, expiration semantics, and enterprise compatibility, requiring broader investigation and integration testing. The left issue is comparatively localized to aligning metric definition and invocation labels.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search