8 views
-/https://github.com/berriai/litellm/issues/31867
GitHub · issue

#31867 bug(litellm_proxy): tags and litellm_session_id silently dropped when upstream proxy forwards to downstream LiteLLM proxy in cascaded topology

  • State: open
  • Author: @deepanshululla
  • Labels: llm translation

## Problem

In a cascaded LiteLLM proxy topology (upstream proxy routing to a downstream LiteLLM proxy), two categories of operational metadata are silently dropped before the HTTP call reaches the downstream proxy: user-supplied tags and `litellm_session_id`.

When a client sends `{"metadata": {"tags": ["project-x", "team-a"]}}` to the upstream proxy, those tags are accumulated in `litellm_params["metadata"]["requester_metadata"]` during the upstream pipeline. `LiteLLMProxyChatConfig.transform_request()` builds the outbound request body from only `model`, `messages`, and `optional_params`, never inspecting `litellm_params`. The downstream proxy therefore receives no tags, and per-tag budget rules or spend attributions configured there are never evaluated.

`litellm_session_id` suffers the same fate. When a client includes `extra_body.litellm_session_id` for multi-turn session continuity, the upstream proxy extracts it into `litellm_params["litellm_session_id"]`. The transform discards it, so the downstream proxy creates sessions without the caller's intended identifier. Both issues are silent at runtime; no error is raised, the call succeeds, but the downstream proxy behaves as if…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d17h48m59s ago — entered · #import:https:::github.com:berriai:litellm post #3072
The model-integration work is harder because it may require provider-specific request/response handling, deployment and naming logic, capability mapping, and broader compatibility tests. The metadata-forwarding fix is narrower, centered on preserving existing context across one transformation boundary with regression coverage.
#0 of 0 · 31d17h39m16s ago — current · #import:https:::github.com:berriai:litellm post #3239
Issue #24677 is harder because it involves diagnosing and correcting distributed quota accounting and timing behavior, with concurrency, token estimation, storage consistency, and regression risk across proxy deployments. Issue #31867 is comparatively localized to request transformation and metadata propagation, with focused compatibility tests.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search