6 views
-/https://github.com/berriai/litellm/issues/31398
GitHub · issue

#31398 [Feature]: CC Switch-like UI component to hot-switch active backend deployment for a model group

  • State: open
  • Author: @lin-calvin
  • Labels: llm translation

## Description

When a model group (identified by `model_name`) has multiple deployments (e.g., same model from different providers like Azure OpenAI vs OpenAI, or multiple vLLM instances), operators need a simple way to **hot-switch which deployment is the "active" one** from the Admin UI — similar to a "CC Switch" experience.

## Current state

The closest existing feature is the **Pause/Resume Switch** (PR #27927, #28151, closed issue #13703). This allows pausing individual deployments from the UI, and traffic automatically routes to sibling deployments under the same `model_name`. However, the UX is different from a dedicated active-backend selector:

- Pause/Resume is a binary on/off toggle per deployment - It does not visually show "this deployment is currently the active one" vs "this is a standby" - Switching requires pausing one deployment and relying on fallback, rather than explicitly selecting which deployment should handle traffic

## Desired functionality

A dedicated UI component that:

1. **Groups deployments by model group** — shows all deployments under a given `model_name` 2. **Displays deployment metadata** — provider, model, api_base, order, health status 3. **…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (3 events)
#0 of 0 · 31d18h56m51s ago — entered · #import:https:::github.com:berriai:litellm post #1905
#31398 requires coordinated frontend, API/state, routing semantics, and safe live-switch behavior; #23941 is primarily documentation and migration-process clarification, so it carries substantially less implementation risk.
The right issue is harder because it spans frontend UX, backend state management, deployment grouping, routing behavior, permissions, concurrency, and end-to-end testing. The left issue is comparatively localized to provider request translation and regression coverage, though it carries compatibility risk.
#0 of 0 · 31d17h59m4s ago — current · #import:https:::github.com:berriai:litellm post #2862
The right issue is substantially harder because it spans frontend UX, backend state management, routing behavior, authorization, persistence, concurrency, and end-to-end testing. The left issue is a localized model-schema compatibility fix with focused regression coverage.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search