9 views
-/https://github.com/berriai/litellm/issues/35608
GitHub · issue

#35608 [Bug]: databricks cost_calculator ignores cached/audio/reasoning tokens (hand-rolled instead of generic_cost_per_token)

  • State: open
  • Author: @Anuj7411
  • Labels: llm translation

### What happened?

`litellm/llms/databricks/cost_calculator.py` computes cost by hand:

```python prompt_cost: float = usage["prompt_tokens"] * model_info["input_cost_per_token"] completion_cost = usage["completion_tokens"] * model_info["output_cost_per_token"] ```

This multiplies every prompt token by the flat `input_cost_per_token` and ignores the rest of the `Usage` block: `prompt_tokens_details.cached_tokens`, `cache_creation_tokens`, audio tokens, and reasoning tokens. It is the same shape as the Fireworks bug fixed in #33714, where the provider had a bespoke `cost_per_token` instead of delegating to `generic_cost_per_token`.

DeepSeek, xAI, Perplexity, Fireworks and other OpenAI-compatible providers delegate to `generic_cost_per_token`, which handles cache-read / cache-creation / audio / reasoning against the fields already on `model_info`. Databricks is an outlier.

### Impact

Latent today: no `litellm_provider: "databricks"` entry in `model_prices_and_context_window.json` currently publishes `cache_read_input_token_cost` or audio rates, so there is no active mis-billing. But Databricks serves models (e.g. Claude, Llama, DBRX) that can report cached tokens, so the moment …

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d18h13m44s ago — entered · #import:https:::github.com:berriai:litellm post #2831
The memory-security integration requires broader architectural design, new trust and persistence flows, external dependency evaluation, configuration, testing, and security review. The Databricks change is comparatively localized to shared token-cost calculation behavior with focused regression tests.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search