8 views
-/https://github.com/berriai/litellm/issues/29810
GitHub · issue

#29810 [Bug] cache_control_injection_points on /v1/responses: no-op for caching AND triggers a deterministic Claude tool-call loop until MaxTurns

  • State: open
  • Author: @federicopascarella-CX
  • Labels: llm translation

## Body

### Summary

When `cache_control_injection_points` is configured for Anthropic Claude models on a LiteLLM proxy that serves the OpenAI Agents JS SDK via `/v1/responses`, two things happen — both bad:

1. **Cache injection is a silent no-op.** The hook iterates `messages[]` looking for `role: "system"` but the Responses API protocol places the system prompt in the top-level `instructions` string, not in the messages array. The hook finds nothing, returns empty (`cache_controls=[]` in DEBUG logs), and zero `cache_creation_input_tokens`/`cache_read_input_tokens` are returned by Anthropic on any subsequent turn — caching never engages. 2. **The mere presence of the hook in the request pipeline corrupts the conversation.** Even though the injection itself does nothing, on multi-turn tool calls Claude (Opus 4.6, but reproduced across reasoning effort levels and with extended thinking disabled) repeatedly re-issues the same tool call after receiving its result. Every turn returns `200 OK` — no protocol error — but the model behaves as if the prior tool result was never consumed. The orchestrator hits its `maxTurns` limit (default 15 in `@openai/agents`) and the request errors out…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d18h58m25s ago — entered · #import:https:::github.com:berriai:litellm post #1886
The Bayesian optimization request is substantially broader: it requires designing and integrating an experimentation and optimization system, defining objective and configuration interfaces, managing trials and persistence, and handling operational cost and safety concerns. The other item is a focused compatibility/debugging effort within an existing request transformation and caching pipeline, though reproducing the multi-turn behavior adds some risk.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search