8 views
-/https://github.com/berriai/litellm/issues/31113
GitHub · issue

#31113 [Bug]: bedrock-mantle using IAM Role/Policy for proxy?

  • State: open
  • Author: @hpcpony
  • Labels: bug, proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

I can't figure out how to setup bedrock-mantle models in the proxy while I'm using an IAM Role/Policy to give permissions to the EC2 instance running the litellm proxy.

### Steps to Reproduce

If I define the model as such:

- model_name: gemma-4-31b litellm_params: model: bedrock_mantle/google.gemma-4-31b aws_region_name: us-east-1

I get the following:

Error fetching response:Error: 500 litellm.APIConnectionError: Bedrock_mantleException - {"error":{"code":"invalid_api_key","message":"Missing 'authorization' or 'x-api-key' header","param":null,"type":"permission_denied_error"}}. Received Model Group=gemma-4-31b Available Model Group Fallbacks=None

GoogleAI suggested I add a bogus api_key and it would fail-over to the role/policy...

- model_name: gemma-4-31b litellm_params: model: bedrock_mantle/google.gemma-4-31b aws_region_name: us-east-1 api_key: "uses-iam-role"

but that didn't work as it seems to just fail because it's a bad key:

Error fetching response:Error: 500 litellm.APIConnectionError: …

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (4 events)
#0 of 0 · 31d19h5m6s ago — entered · #import:https:::github.com:berriai:litellm post #1792
#31113 requires deeper provider-specific authentication work, including AWS credential-chain handling, request signing, configuration semantics, and compatibility testing. #24771 is comparatively narrower: tracing the OAuth callback flow and correcting the route/redirect integration, though it still touches security-sensitive backend and UI behavior.
31113 requires cross-layer investigation of AWS credential discovery, Bedrock Mantle authentication, provider request construction, and compatibility testing; 28376 is comparatively localized identifier-selection and persistence handling with focused regression tests.
The right issue is harder because it involves provider authentication behavior, AWS credential resolution, request signing or header construction, and compatibility with proxy configuration. The left issue is a localized dashboard rendering fix with comparatively limited scope and risk.
#0 of 0 · 31d18h36m59s ago — current · #import:https:::github.com:berriai:litellm post #2263
The left issue requires tracing and correcting shared mutable request or deployment state, ensuring isolation across requests and clients, handling concurrency and lifecycle behavior, and adding regression coverage. The right issue is a narrower provider-authentication and configuration integration problem.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search