7 views
-/https://github.com/berriai/litellm/issues/31714
GitHub · issue

#31714 [Bug]: Fireworks cache-read tokens are not charged at cache-read rate

  • State: open
  • Author: @apshada

### What happened? Fireworks chat responses can include `usage.prompt_tokens_details.cached_tokens`, while model info can include `cache_read_input_token_cost`. The Fireworks cost calculator currently uses provider-specific prompt/output multiplication and does not apply the cache-read token price. Cached prompt tokens should be charged at `cache_read_input_token_cost`, with only uncached prompt tokens charged at `input_cost_per_token`

A representative usage has `prompt_tokens=27147`, `cached_tokens=27146`, `completion_tokens=351`, `input_cost_per_token=0.0000014`, `cache_read_input_token_cost=0.00000026`, and `output_cost_per_token=0.0000044`. Expected total cost is `0.00860376`

### Steps to Reproduce Register a Fireworks model with input, output, and cache-read token pricing. Build a `ModelResponse` whose usage has `prompt_tokens_details.cached_tokens`. Call `completion_cost(..., custom_llm_provider="fireworks_ai")` and compare the result with `uncached_prompt_tokens * input_cost_per_token + cached_tokens * cache_read_input_token_cost + completion_tokens * output_cost_per_token`

### Relevant log output `prompt_tokens_details.cached_tokens` is present in usage, and `cache_read_…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d18h12m37s ago — entered · #import:https:::github.com:berriai:litellm post #2648
Fireworks billing requires careful token-category accounting, compatibility with existing provider-specific cost logic, and regression coverage for edge cases. The Bedrock authentication fix is narrower: update the signing service identifier across the relevant paths and validate request authentication.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search