6 views
-/https://github.com/berriai/litellm/issues/27656
GitHub · issue

#27656 [Bug]: model_info cost override ignored when calling upstream LiteLLM proxy (litellm_proxy/ prefix)

  • State: open
  • Author: @fmanigrasso
  • Labels: bug, proxy

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

When a LiteLLM proxy instance (Y) is configured to call another LiteLLM proxy instance (X) using the `litellm_proxy/` model prefix, setting `input_cost_per_token: 0.0` / `output_cost_per_token: 0.0` in `model_info` or `litellm_params` on Y has **no effect**. Y always records the cost that X computed, ignoring the local override entirely.

**Expected behavior:** Local `model_info` cost overrides on the calling proxy (Y) should take precedence over the cost reported by the upstream proxy (X).

**Actual behavior:** The upstream proxy (X) always includes an `x-litellm-response-cost` response header (`common_request_processing.py:550`). When Y receives the response, `response_cost_calculator()` extracts this header and **returns immediately**, before `completion_cost()` or `cost_per_token()` are ever reached — so Y's local `model_info` overrides are never consulted.

Root cause — `cost_calculator.py`, lines 1749–1753: ```python provider_response_cost = get_response_cost_from_hidden_params( response_object._hidden_params ) if provider_response_co…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d18h35m46s ago — entered · #import:https:::github.com:berriai:litellm post #2461
The left issue is harder because it requires changing cost-precedence behavior across chained proxy requests, hidden response metadata, and multiple fallback calculators, with greater compatibility and regression-testing risk. The right issue is comparatively localized to dispatcher control flow and aggregation tests.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search