8 views
-/https://github.com/berriai/litellm/issues/31863
GitHub · issue

#31863 bug(bedrock-invoke): native structured output blocked for Claude 4 models; model-rename workaround forces tool-call path for all models

  • State: open
  • Author: @deepanshululla
  • Labels: llm translation

## What happened

`AmazonAnthropicClaudeConfig.map_openai_params` in `litellm/llms/bedrock/chat/invoke_transformations/anthropic_claude3_transformation.py` (OSS 1.91.0, lines 76-104) contains a blanket workaround that renames the model to `claude-3-sonnet-20240229` whenever `response_format` appears in params. This forces the tool-based JSON shaping path for every Claude model on the Bedrock Invoke route, including modern Claude 4 models like `anthropic.claude-opus-4-6-v1` and `anthropic.claude-sonnet-4-6` that AWS has enabled native `output_format` support for.

A compounding issue is in `litellm/anthropic_beta_headers_config.json` (the `"bedrock"` section), where `structured-outputs-2025-11-13` is set to `null`. On Bedrock Invoke, Anthropic beta flags flow through the JSON request body as the `anthropic_beta` list. Setting the flag to `null` causes `filter_and_transform_beta_headers` to silently drop it, so even if native `output_format` were passed, the required beta signal would never reach the inference endpoint.

Together these two bugs mean every Bedrock Invoke structured-output request travels the tool-call emulation path regardless of model support, and the beta flag requi…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d18h3m40s ago — entered · #import:https:::github.com:berriai:litellm post #2794
31863 is harder because it spans provider-specific request transformation, model capability handling, beta-flag configuration, compatibility across multiple Claude generations, and comprehensive Bedrock regression coverage. 30617 is narrower and primarily involves repairing serialization lifecycle handling in shared streaming code, though it carries moderate cross-path regression risk.
#0 of 0 · 31d17h33m19s ago — current · #import:https:::github.com:berriai:litellm post #3322
Adding a new provider spans provider registration, model discovery, pricing integration, endpoint translation, proxy behavior, and broad compatibility testing. The Bedrock issue is narrower, centered on correcting existing model-specific structured-output routing and configuration, though it carries some API-compatibility risk.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search