10 views
-/https://github.com/berriai/litellm/issues/35223
GitHub · issue

#35223 [Bug]: Auto Router ignores vision capability when picking a deployment, so one image permanently breaks a session

  • State: open
  • Author: @Sonuzan
  • Labels: bug, proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

## What happens

If a model group (or an `auto_router` alias) mixes vision-capable and text-only deployments, one image kills the whole session, not just the turn it was sent on.

The client keeps the image in its history and resends it every turn. As soon as the router picks a text-only deployment, the provider rejects the entire request, so every following turn fails the same way. Client-side history compaction fails too, since it replays the full history, so there is no way out: the session has to be thrown away.

Hit this on Bedrock behind an `auto_router/complexity_router` alias, but nothing here is Bedrock-specific. Code below is the same on the `v1.94.0` tag and current `main`.

## Why

The router never looks at request modality when picking a deployment.

**1. `_pre_call_checks` has no modality filter.** `litellm/router.py:10163` filters candidates on context window, RPM, region, and unsupported params. That last one already does capability-based elimination via `litellm.get_supported_openai_params()`, but it is scoped to one param:

``…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d18h47m9s ago — entered · #import:https:::github.com:berriai:litellm post #2184
Issue #35223 is harder because it requires cross-cutting routing logic, capability detection, deployment filtering, session behavior, and regression coverage across heterogeneous providers. Issue #33202 is comparatively localized to Azure provider parameter translation and response handling.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search