18 views
-/https://github.com/berriai/litellm/issues/24987
GitHub · issue

#24987 [Bug]: Prompt Cache Not Working with GPT-5.4 via litellm proxy, but works with direct OpenAI API

  • State: open
  • Author: @s906903912
  • Labels: bug, llm translation, ui-dashboard

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

## Bug Description

Prompt caching does not work when calling GPT-5.4 through litellm proxy, while the same requests work correctly when using OpenCode/ClaudeCode CLI directly with sub2api or other third platform.

## Steps to Reproduce

1. Configure litellm proxy with GPT-5.4 model 2. Send requests with `cache_control` parameter in messages 3. Observe that cache is not being used on subsequent requests

## Expected Behavior

When sending identical requests with `cache_control`, the second request should hit the prompt cache and return faster with reduced token costs.

## Actual Behavior

- Direct API calls via OpenCode/ClaudeCode CLI → Cache works ✅ - Same requests through litellm proxy → Cache not triggered ❌

## Environment

- litellm version: v1.82.3 - Model: `gpt-5.4` (also tested with `gpt-5.4`) - Provider: OpenAI (via sub2api) - Configuration: - `use_in_pass_through: true` - `use_litellm_proxy: false`

Use Litellm: <img width="2221" height="798" alt="Image" src="https://github.com/user-attachments/assets/e25765a8-83d8-49bf-b525-b7ab4cfb2…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d18h37m13s ago — entered · #import:https:::github.com:berriai:litellm post #2437
#24987 requires tracing provider-specific request translation and cache metadata across proxy/pass-through paths, with compatibility and regression testing; #32226 is more narrowly scoped to correcting byte-boundary handling and adding UTF-8 coverage.
#0 of 0 · 31d17h45m41s ago — current · #import:https:::github.com:berriai:litellm post #3321
The left issue is harder because it requires a new provider integration spanning model discovery, pricing metadata, request-format routing, proxy behavior, and broad compatibility testing. The right issue is more narrowly scoped to diagnosing and correcting an existing request-path or caching regression.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search