8 views
-/https://github.com/berriai/litellm/issues/32425
GitHub · issue

#32425 [Bug]: Router cooldown policy silently ignores AllowedFailsPolicy.*AllowedFails=0 due to 'or' fallback

  • State: open
  • Author: @Harsh23Kashyap
  • Labels: llm translation

## Problem

`should_cooldown_based_on_allowed_fails_policy` in `litellm/router_utils/cooldown_handlers.py` combines the value returned by `get_allowed_fails_from_policy` with the router-level default using `or`. Because `or` treats `0` as falsy, configuring any `AllowedFailsPolicy.*AllowedFails` field to `0` (which a user would reasonably do to force cooldown on the first occurrence of that exception class) silently falls back to `router.allowed_fails`. The configured threshold is dropped without any warning, and the next comparison `updated_fails > allowed_fails` runs with the router default instead of `0`.

## Current behavior

```python allowed_fails = ( litellm_router_instance.get_allowed_fails_from_policy( exception=original_exception, ) or litellm_router_instance.allowed_fails ) ```

If `get_allowed_fails_from_policy` returns `0` (because the user set `AllowedFailsPolicy(RateLimitErrorAllowedFails=0)`), the expression evaluates to `litellm_router_instance.allowed_fails`, which is the router default (`None` unless explicitly set). The user's intent is silently discarded.

## Expected behavior

A user-configured value of `0` should be honored: cooldown on th…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (3 events)
#0 of 0 · 31d19h16m22s ago — entered · #import:https:::github.com:berriai:litellm post #1704
The right issue is substantially harder because it is an intermittent, long-running UI failure with unclear reproduction and potentially spans frontend assets, deployment/runtime behavior, caching, and observability. The left issue is a localized fallback-semantics defect with a narrowly scoped code change and targeted regression tests.
Issue #32425 is harder because it requires correcting fallback semantics in router cooldown logic, preserving explicit zero values, and adding regression coverage across policy/default combinations. Issue #32984 is primarily a scoped model-registry metadata and pricing update.
#0 of 0 · 31d18h33m7s ago — current · #import:https:::github.com:berriai:litellm post #2419
#32941 appears to involve broader, less-defined engineering scope, while #32425 is a localized logic correction with straightforward validation and regression testing.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search