8 views
-/https://github.com/berriai/litellm/issues/25503
GitHub · issue

#25503 [Feature]: Expose actual served model in response when router falls back to a different deployment

  • State: open
  • Author: @VANDRANKI
  • Labels: proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### The Feature

When the LiteLLM router falls back from one deployment to another (e.g., `gpt-4o` on Azure East ? `gpt-4o` on Azure West, or `gpt-4o` ? `gpt-4-turbo`), the `model` field in the response and the `x-litellm-model` response header still reflect the originally requested model group name, not the deployment that actually served the request.

Proposal: add a field to the response (or response header) that identifies the actual deployment/model that served the request after fallback resolution.

Suggested additions: - `x-litellm-actual-model` response header: the `litellm_params.model` of the deployment that served the call - `x-litellm-model-group` response header: the requested model group name (existing `x-litellm-model` behavior, renamed for clarity) - Alternatively, populate `usage.model` or a `_hidden_params.model_used` field with the resolved model

### Motivation, pitch

Without knowing which deployment actually served a request:

1. **Cost attribution breaks**: if the fallback lands on a more expensive model (e.g., GPT-4 instead of GPT-4o-mini),…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (4 events)
#0 of 0 · 31d19h1m27s ago — entered · #import:https:::github.com:berriai:litellm post #1834
Issue 33722 is harder because it spans persistence boundaries, API serialization, admin UI payload handling, backward compatibility, and performance-sensitive deployment copying, requiring coordinated fixes and regression coverage. Issue 25503 is comparatively localized to propagating resolved deployment metadata through fallback response paths and headers.
The right issue is harder because it requires propagating deployment-resolution metadata through router, proxy response, header, streaming, and compatibility paths, with broader API and regression-test impact. The left issue is more concentrated in tracing span ordering and processor lifecycle handling, though it carries integration-specific correctness risk.
The left is harder because it involves diagnosing and correcting time-boundary accounting across usage aggregation, persistence, and budget calculations, with significant data-integrity and regression risk. The right is comparatively contained interface work: propagating resolved deployment metadata through existing routing and response paths while preserving compatibility.
#0 of 0 · 31d18h49m20s ago — current · #import:https:::github.com:berriai:litellm post #2041
The left requires coordinated dashboard, authorization, team-context, and ephemeral-session changes with added state and security considerations. The right is broader across routing and response plumbing, but can likely be implemented as a focused metadata propagation change with targeted compatibility tests.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search