8 views
-/https://github.com/berriai/litellm/issues/33281
GitHub · issue

#33281 [Bug]: Bedrock guardrailConfig silently drops cache_control on trailing user messages (breaks prompt caching)

  • State: open
  • Author: @superleesa
  • Labels: bug, llm translation, SDK

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

When a Bedrock Converse request includes `guardrailConfig`, any `cache_control` marker on the **trailing user message(s)** is silently lost during request transformation, so no `cachePoint` block is emitted for the latest message.

This affects both ways of placing the marker:

- `cache_control_injection_points` with `{location: message, index: -1}`, and - `cache_control` set directly on the last user message / its content blocks.

Because a typical chat request ends with a user message, every ordinary turn loses its conversation-prefix breakpoint whenever guardrails are enabled. Requests ending in a tool result are unaffected (tool messages are not rewritten), so cache behavior differs between plain turns and tool-loop turns of the same conversation.

**Root cause**

`AmazonConverseConfig._convert_consecutive_user_messages_to_guarded_text` (`litellm/llms/bedrock/chat/converse_transformation.py`) rewrites trailing user messages so the guardrail evaluates only the latest turn:

- list content: `{"type": "text", ...}` → a **new** `…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d17h58m16s ago — entered · #import:https:::github.com:berriai:litellm post #2906
The left task is harder because it requires carefully modifying provider-specific request transformation logic while preserving metadata across multiple message/content shapes and validating interactions with guardrail behavior and caching. The right task is more localized to proxy request-header extraction, configuration handling, and role-mapping validation, with a narrower integration surface. The left therefore carries greater implementation and regression risk.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search