8 views
-/https://github.com/berriai/litellm/issues/32496
GitHub · issue

#32496 fireworks_ai cost calculator ignores cached_tokens / cache_read_input_token_cost

  • State: open
  • Author: @claudio-michel[bot]
  • Labels: llm translation

### What happened

`litellm/llms/fireworks_ai/cost_calculator.py :: cost_per_token()` bills **100% of prompt tokens at the full input price**, ignoring cached tokens entirely:

```python prompt_cost: float = usage["prompt_tokens"] * model_info["input_cost_per_token"] ```

It never reads `usage.prompt_tokens_details.cached_tokens`, and it never applies `model_info["cache_read_input_token_cost"]` — even though the docstring claims the usage block contains "caching information" and the model map for Fireworks models (e.g. `fireworks_ai/accounts/fireworks/models/glm-5p2`) already defines `cache_read_input_token_cost`. The generic cost calculator right next to it (`litellm/litellm_core_utils/llm_cost_calc/utils.py :: generic_cost_per_token`) handles both the cached-token split and the cache-read price; the Fireworks-specific path silently does not.

Fireworks serverless **does** cache and **does** return `prompt_tokens_details.cached_tokens` in usage (verified against the live API in both non-streaming and streaming), so on long agentic sessions with 80–95% cache hit rates this overbills input by up to ~4×.

### Repro (LiteLLM 1.91.0)

```python from litellm.types.utils import Usage, Pr…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d18h42m13s ago — entered · #import:https:::github.com:berriai:litellm post #2165
The right issue spans configuration schemas, router registration, persistence into spend metadata, and downstream pricing behavior, requiring coordinated changes and broader regression coverage. The left issue is a localized provider-specific calculation correction with a comparatively narrow test surface.
#0 of 0 · 31d18h18m20s ago — current · #import:https:::github.com:berriai:litellm post #2539
The left task spans multiple team-creation pathways and shared default-application semantics, with broader regression-testing and compatibility risk. The right task is a localized billing-calculation correction with a comparatively narrow implementation and test surface.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search