6 views
-/https://github.com/berriai/litellm/issues/31857
GitHub · issue

#31857 Router: `_unregister_router_selectors` drops old selectors from callbacks but never cancels their `_sync_task`, leaking asyncio tasks on `routing_groups`/`routing_strategy` updates

  • State: open
  • Author: @BooB90

### What happened

Repeated `Router.update_settings(routing_groups=...)` calls (or any `routing_strategy` change that goes through `routing_strategy_init`) rebuild the router's strategy selectors via `_init_routing_groups` / `routing_strategy_init`, both of which call `_unregister_router_selectors` (`litellm/router.py:930`) to drop the old selector instances from `litellm.callbacks` / `litellm.input_callback`.

That unregister step only removes the selector from the two callback lists by identity:

```python def _unregister_router_selectors(self, selectors: List[Any]) -> None: selector_ids = {id(s) for s in selectors if s is not None} if not selector_ids: return if isinstance(litellm.callbacks, list): litellm.callbacks = [c for c in litellm.callbacks if id(c) not in selector_ids] if isinstance(litellm.input_callback, list): litellm.input_callback = [c for c in litellm.input_callback if id(c) not in selector_ids] ```

It never calls `cleanup()` (or `.cancel()`) on the selector. Strategies built on `BaseRoutingStrategy` with `should_batch_redis_writes=True` (e.g. `LowestTPMLoggingHandler_v2`, `litellm/router_strategy/lowest_tpm_rpm_v2.py:56`) s…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (5 events)
#0 of 0 · 31d19h8m44s ago — entered · #import:https:::github.com:berriai:litellm post #1717
31857 is harder because it involves asynchronous resource lifecycle management, shared global callback state, repeated reinitialization, cancellation semantics, and concurrency-sensitive regression testing. 28232 is comparatively localized to translation behavior and provider-format compatibility tests.
The right issue requires careful asynchronous resource lifecycle handling, cancellation semantics, and regression coverage across router reconfiguration paths; the left is comparatively localized compatibility and validation work in request translation.
Issue #31857 is harder because it requires tracing router selector lifecycle, safely cancelling asynchronous tasks during dynamic reconfiguration, preserving callback behavior, and adding regression coverage. Issue #26669 provides no substantive technical scope and is therefore likely lower-effort or impossible to assess beyond a basic clarification.
The left issue requires cross-layer streaming behavior changes, careful protocol compatibility, timing-sensitive async handling, and end-to-end coverage across deployment configurations. The right issue is a comparatively localized lifecycle fix involving task cancellation and regression tests, with narrower behavioral risk.
#0 of 0 · 31d18h38m14s ago — current · #import:https:::github.com:berriai:litellm post #2217
The left issue spans a native Rust dependency upgrade, interpreter compatibility metadata, and cross-version wheel/build validation, creating broader release and CI risk. The right issue is more localized to asynchronous resource lifecycle handling, with targeted code changes and regression tests.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search