7 views
-/https://github.com/berriai/litellm/issues/33960
GitHub · issue

#33960 Per-group allowed_fails / cooldown_time / enable_health_check_routing via routing_groups

  • State: open
  • Author: @roopantj
  • Labels: enhancement, proxy

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### The Feature

Extend RoutingGroup (**litellm/types/router.py**) with optional **allowed_fails**, **allowed_fails_policy**, **cooldown_time**, **enable_health_check_routing** fields, resolved via the **Router._model_to_group map** that **_init_routing_groups** already builds — falling back to the existing router-wide scalars when a group doesn't override them. I named the exact two functions in **cooldown_handlers.py** and the two filter methods in router.py that would need the group-lookup added, plus a sample config.

### Motivation, pitch

<html><head></head><body><h2>Background</h2><p>We run LiteLLM as a shared internal gateway in front of multiple services. While all services communicate through the same proxy, each service routes to a different model or model group depending on its use case:</p><ul><li><p><strong>Latency-sensitive inline scoring service</strong> → optimized for low latency and fast failover.</p></li><li><p><strong>Batch/analysis workloads</strong> → can tolerate transient failures and prioritize capacity.</p></li><li><p><strong>Chat as…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d18h36m34s ago — entered · #import:https:::github.com:berriai:litellm post #2554
The right issue spans authentication policy, key lifecycle behavior, notification delivery, user identity data, and error-handling changes across multiple proxy surfaces, creating substantially more integration and security risk. The left issue is comparatively localized: extending routing configuration and threading per-group values through existing lookup and filtering paths.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search