8 views
-/https://github.com/berriai/litellm/issues/31481
GitHub · issue

#31481 [Bug]: adaptive_router reads model cost from litellm_params.input_cost_per_token, not model_info — silently zeroes out cost-weighted routing

  • State: open
  • Author: @suranjanotti
  • Labels: bug, proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

When configuring `auto_router/adaptive_router` with `weights: {quality: X, cost: Y}`, the router's cost-aware scoring silently does nothing if `input_cost_per_token` is only declared under `model_info` (the conventional/documented location for cost tracking elsewhere in LiteLLM, e.g. cost logging, budget alerts).

Root cause (found by reading router.py and bandit.py inside the installed package):

`Router.init_adaptive_router_deployment` builds `model_to_cost` by reading `lp_dict.get("input_cost_per_token")` from each model's `litellm_params`, not from `model_info`:

``` lp = d.get("litellm_params") if isinstance(d, dict) else d.litellm_params lp_dict = lp if isinstance(lp, dict) else (lp.model_dump() if lp else {}) cost = lp_dict.get("input_cost_per_token") if cost is not None: model_to_cost[name] = float(cost) ```

### Steps to Reproduce

Configure model_list with at least two models behind auto_router/adaptive_router, declaring input_cost_per_token only under model_info (not litellm_params), e.g.: ``` model_list:

model_name: cheap-mode…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d17h38m4s ago — entered · #import:https:::github.com:berriai:litellm post #3245
#13245 is harder because it involves cancellation-safe asynchronous request handling, reliable post-request accounting, and provider-specific lifecycle testing; #31481 is comparatively localized to cost metadata resolution and regression tests.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search