7 views
-/https://github.com/berriai/litellm/issues/30081
GitHub · issue

#30081 [Bug]: Deployment custom pricing (`input_cost_per_token: 0`) silently ignored after model-cost-map reload; per-request pricing overwrites canonical model_cost entries

  • State: open
  • Author: @puneet-bst
  • Labels: bug, proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

## What happened?

We run wildcard BYOK routes alongside normally priced models:

```yaml model_list: - model_name: claude-opus-4-6 # normal, proxy-keyed, billed litellm_params: model: anthropic/claude-opus-4-6 api_key: os.environ/ANTHROPIC_API_KEY

- model_name: anthropic/* # BYOK: user supplies the key, cost must be 0 litellm_params: model: anthropic/* api_key: os.environ/ANTHROPIC_BYOK_PLACEHOLDER input_cost_per_token: 0 output_cost_per_token: 0 ```

We observed two intertwined billing bugs on v1.87.0 (code paths unchanged on current main as far as we can tell):

### Bug 1 — custom pricing lost when `litellm.model_cost` is replaced

`response_cost` for deployment custom pricing is resolved through `litellm.model_cost[<router_model_id>]`, registered by the router at deployment creation (`Router._create_deployment` / `add_deployment`). But:

- `_check_and_reload_model_cost_map` (and the admin force-reload endpoint) does `litellm.model_cost = get_model_cost_map(url=...…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (3 events)
#0 of 0 · 31d18h27m7s ago — entered · #import:https:::github.com:berriai:litellm post #2411
30081 is harder because it requires tracing and correcting shared pricing state across deployment registration, reload behavior, and request-time cost resolution, with significant regression risk in billing semantics. 29407 is comparatively bounded dependency maintenance, mainly involving version compatibility checks and CI/integration validation.
The left requires broader API-surface partitioning, schema ownership, compatibility handling, and integration across proxy exposure paths. The right is a more localized pricing-state consistency fix with targeted regression coverage.
#0 of 0 · 31d17h47m16s ago — current · #import:https:::github.com:berriai:litellm post #3076
Issue #30081 is harder because it spans shared pricing-state ownership, reload behavior, deployment registration, request-time cost resolution, and administrative flows, with higher regression and compatibility risk. Issue #23757 is comparatively localized to robust content-shape handling in a prompt transformation path plus focused regression tests.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search