9 views
-/https://github.com/berriai/litellm/issues/34378
GitHub · issue

#34378 Custom *_above_32k_tokens saved in UI but ignored in cost calc / spend model_map

  • State: open
  • Author: @zhouqq6

## Relevant issues Related to #30344 / #30880 — those fixed `get_model_info` dropping arbitrary `_above_*` keys from an existing cost-map entry. This issue is a remaining gap on the **Router custom-pricing registration** path. ## Describe the bug When configuring custom tiered pricing on a DB/UI deployment, e.g.: - `input_cost_per_token_above_32k_tokens` - `output_cost_per_token_above_32k_tokens` - `cache_read_input_token_cost_above_32k_tokens` these fields: 1. Appear correctly in Dashboard `litellm_params` (because `LiteLLM_Params` has `extra="allow"`) 2. Do **not** appear in spend-log `model_map_information.model_map_value` 3. Are **not** applied during cost calculation (requests in the 32k–128k range are billed at the base `input_cost_per_token`) Meanwhile `*_above_128k_tokens` works, because it is declared on `CustomPricingLiteLLMParams`. ## Root cause (code) Router only copies fields from `CustomPricingLiteLLMParams.model_fields` into the cost map: https://github.com/BerriAI/litellm/blob/v1.92.0/litellm/router.py#L7988-L8007 ```python for field in CustomPricingLiteLLMParams.model_fields.keys(): field_value = deployment.litellm_params.get(field) if field_value is not No…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (4 events)
#0 of 0 · 31d19h21m54s ago — entered · #import:https:::github.com:berriai:litellm post #1815
The left requires coordinated changes across schema handling, router registration, cost computation, persistence, and regression coverage, with compatibility risks in pricing behavior. The right requires little or no product engineering work.
Issue #34378 is harder because it spans configuration schema handling, router registration, persistence into spend metadata, and tiered cost-calculation behavior, with broader compatibility and regression-testing risk. Issue #26250 is comparatively localized to iterator lifecycle cleanup and accompanying async tests.
The right-hand change spans configuration modeling, registration, persistence, and billing behavior, requiring compatibility analysis and broad regression coverage; the left is a localized filesystem-safety fix with comparatively limited scope.
#0 of 0 · 31d19h1m20s ago — current · #import:https:::github.com:berriai:litellm post #2165
The right issue spans configuration schemas, router registration, persistence into spend metadata, and downstream pricing behavior, requiring coordinated changes and broader regression coverage. The left issue is a localized provider-specific calculation correction with a comparatively narrow test surface.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search