8 views
-/https://github.com/berriai/litellm/issues/31826
GitHub · issue

#31826 [Feature]: Protocol-aware deployment selection in Router

  • State: open
  • Author: @limitjupyter-ops
  • Labels: enhancement, proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### The Feature

I would like LiteLLM Router to support protocol-aware deployment selection.

For the same logical model, different deployments may support different request protocols or endpoint styles, for example:

- OpenAI Chat Completions - OpenAI Responses API - Anthropic Messages - provider-specific OpenAI-compatible chat endpoints

Suggested behavior:

1. Each deployment can declare supported protocols or endpoint capabilities. 2. Router detects the incoming request protocol. 3. Router only selects deployments that are compatible with the request protocol. 4. Fallback and retry should preserve protocol compatibility. 5. If no compatible deployment exists, LiteLLM should return a clear error.

Example config direction:

```yaml model_list: - model_name: my-logical-model litellm_params: model: openai/my-provider-model model_info: supported_protocols: - openai_chat_completions

- model_name: my-logical-model litellm_params: model: anthropic/my-provider-model model_info: supported_protocols: - anthrop…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d18h13m20s ago — entered · #import:https:::github.com:berriai:litellm post #2629
The left task requires cross-cutting changes across request classification, deployment metadata, routing, retries, fallbacks, configuration validation, and compatibility testing. The right task is a comparatively localized diagnostic and regression fix in health-status aggregation or Vertex-specific handling.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search