7 views
-/https://github.com/berriai/litellm/issues/35011
GitHub · issue

#35011 `prompt_cache_min_tokens`: four `claude-fable-5` keys disagree with the direct entry, and 499 entries that claim caching support record no minimum at all

  • State: open
  • Author: @Tanisha-Katara
  • Labels: llm translation

Two related problems in `model_prices_and_context_window.json`, found while building a cache-cost analyzer that reads this file as a source of truth. The first is a wrong value, the second is a coverage gap. Both surface through `get_prompt_cache_min_tokens()`.

## 1. `claude-fable-5` contradicts itself

| key | `prompt_cache_min_tokens` | |---|---| | `claude-fable-5` | **512** | | `anthropic.claude-fable-5` | 1024 | | `us.anthropic.claude-fable-5` | 1024 | | `eu.anthropic.claude-fable-5` | 1024 | | `global.anthropic.claude-fable-5` | 1024 |

Anthropic publishes 512 for Fable 5, which the direct entry already has. The four platform entries say 1024.

Effect: on those four, a 700-token prefix is treated as too short to cache. `is_prompt_caching_valid_prompt()` returns `False`, the caller skips caching, and a prefix that would have cached is re-sent at full rate on every request.

I checked every other Anthropic model in the file for the same kind of internal disagreement. This is the only one. Every other recorded minimum is correct, including the non-monotonic ones (512 on Opus 5, 1024 on Opus 4.8 and Sonnet 5, 2048 on Opus 4.7 and Haiku 3.5, 4096 on Opus 4.6, Opus 4.5 and Haiku 4.…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (3 events)
#0 of 0 · 31d19h1m56s ago — entered · #import:https:::github.com:berriai:litellm post #2157
The right issue is harder because it requires modifying runtime guardrail translation behavior, handling multiple content shapes safely, and adding regression coverage for a security-sensitive path. The left issue is primarily a metadata consistency and completeness update, with comparatively limited code risk.
The left issue is harder because it involves diagnosing and correcting cross-provider request translation in a proxy integration, with compatibility behavior and end-to-end client testing. The right issue is primarily structured metadata correction and coverage expansion, requiring broader data validation but less runtime integration work.
#0 of 0 · 31d18h2m51s ago — current · #import:https:::github.com:berriai:litellm post #3143
#35011 is harder because it requires a broad, externally validated catalog audit across hundreds of model records, consistent semantics, and comprehensive regression coverage. #28983 is comparatively localized to one endpoint’s persistence mapping and its related tests.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search