8 views
-/https://github.com/berriai/litellm/issues/33772
GitHub · issue

#33772 [Bug]: OpenAI cache-write (`cache_write_tokens`) is dropped from cost calculation — cached-input requests are mis-priced

  • State: open
  • Author: @sarveshb-p72
  • Labels: bug, proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

### What happened

For OpenAI models that report **cache-write** tokens, LiteLLM does not price the cache-write portion of the request, and does not honor tiered/priced cache-creation keys. As a result the `response_cost` for any request that writes to the prompt cache is **understated**, and the error compounds across three separate places in the cost path:

1. **Cost function drops `cache_write_tokens`.** In `litellm/litellm_core_utils/llm_cost_calc/utils.py`, `_parse_prompt_tokens_details` only reads Anthropic-style `cache_creation_tokens` and ignores OpenAI's `cache_write_tokens` field. OpenAI reports cache writes under `prompt_tokens_details.cache_write_tokens` (chat) / `input_tokens_details.cache_write_tokens` (responses), so the cache-write token count arrives as `0` and those tokens are billed at the plain input rate (or not at all), instead of the 1.25× cache-write rate.

2. **Tiered cache-creation cost keys are not registered.** `get_model_info` copies a fixed set of cost keys onto `ModelInfo`, but omits `cache_creation_input_token_co…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (7 events)
#0 of 0 · 31d18h8m19s ago — entered · #import:https:::github.com:berriai:litellm post #3046
33772 requires coordinated changes across token accounting, model metadata, pricing tiers, multiple API response shapes, and regression coverage, creating broader compatibility risk. 34503 is comparatively localized to provider dispatch/configuration and endpoint-specific tests.
#30043 requires coordinated changes across provider translation, streaming behavior, endpoint handling, and agent-workflow compatibility, creating substantially more integration and regression risk. #33772 is comparatively localized to token accounting, model metadata, and focused pricing tests.
The left issue is substantially harder because it requires coordinated quota state, calendar and timezone semantics, concurrency-safe enforcement, persistence changes, API/UI work, and broad integration testing. The right issue is comparatively localized to cost parsing, model metadata, and regression tests.
Issue 26535 is harder because it spans multi-provider integration coverage, test infrastructure, version-sensitive execution, and CI gating, creating substantially broader coordination and validation risk. Issue 33772 is more localized to cost parsing, metadata registration, and focused regression tests.
31296 is harder because it introduces a security-sensitive protocol feature spanning endpoint routing, authentication behavior, configuration, compatibility, and integration testing. 33772 is comparatively contained to billing-data propagation and model-cost metadata, despite requiring coverage across multiple response formats.
The left issue is harder because it requires diagnosing and safely redesigning concurrent database transaction behavior, with race-condition testing and possible cross-instance coordination. The right issue is primarily a bounded cost-model and metadata propagation change with focused unit and pricing tests.
#0 of 0 · 31d17h39m49s ago — current · #import:https:::github.com:berriai:litellm post #3521
The left task spans backend configuration discovery, Helm/environment precedence, secure secret handling, API exposure, and Admin UI integration with broad regression risk. The right task is a more localized cost-model and token-parsing correction with focused provider and billing tests.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search