16 views
-/https://github.com/berriai/litellm/issues/18158
GitHub · issue

#18158 Document async_filter_deployments hook in CustomLogger for deployment filtering

  • State: open
  • Author: @praveenkanna
  • Labels: proxy

**Description:**

The async_filter_deployments method is an extremely useful hook in the CustomLogger class for filtering deployments before routing, but it's completely undocumented. This makes it difficult for users to discover and implement custom deployment filtering logic.

**Current Situation** The hook exists and is actively used in the router code (litellm/router.py around line ~4835), but:

❌ Not mentioned in the documentation ❌ Not in any examples ❌ Not in the API reference ❌ Difficult to discover without reading source code

**Use Case**

We're using async_filter_deployments to implement dynamic region failover in our LLM proxy. When a region experiences issues, we disable it in our database, and this hook filters out disabled regions from the healthy deployments list before routing occurs.

Example implementation:

``` from litellm.integrations.custom_logger import CustomLogger from typing import List, Optional, Any

class MyCustomHandler(CustomLogger): async def async_filter_deployments( self, model: str, healthy_deployments: List, messages: Optional[List], request_kwargs: Optional[dict] = None, parent_otel_span: Opt…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d17h39m44s ago — entered · #import:https:::github.com:berriai:litellm post #3202
The right issue requires changes across provider routing, request translation, feature-flag precedence, compatibility behavior, and regression coverage. The left issue is primarily documentation and examples, with minimal implementation risk.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search