8 views
-/https://github.com/berriai/litellm/issues/30657
GitHub · issue

#30657 [Bug]: Add Gemma 4 which is reachable via mantle endpoint

  • State: open
  • Author: @Iana-Kasimova-TR
  • Labels: enhancement, llm translation, SDK

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### The Feature

I saw this merged PR(https://github.com/BerriAI/litellm/pull/30264). But gemma won't work on bedrock now via litellm, because it now is located on mantle endpoint and there should be another base url - https://aws.amazon.com/blogs/machine-learning/introducing-gemma-4-models-on-amazon-bedrock/

### Motivation, pitch

I am trying to use Gemma in our application from bedrock mantle - and now I created workaround for setting correct base url, but it would be nice to have production code clean

so this code:

`import litellm import os

response = litellm.responses( model="bedrock_mantle/google.gemma-4-31b", input="Hello! How can you help me today?", ) print(response)`

failed with the exception: MaskedHTTPStatusError: Client error '401 Unauthorized' for url 'https://bedrock-mantle.us-east-1.api.aws/v1/chat/completions' For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/401 litellm.APIConnectionError: Bedrock_mantleException - {"error":{"code":"access_denied","message":"Berm is not enabled for this account","par…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (4 events)
#0 of 0 · 31d19h19m5s ago — entered · #import:https:::github.com:berriai:litellm post #1656
The left task spans provider-specific routing, model translation, endpoint/authentication behavior, and compatibility testing across AWS integrations. The right task is a narrowly scoped parameter-propagation fix with focused regression coverage, so the left carries substantially greater integration risk.
The left issue is harder because it involves diagnosing an intermittent, time-delayed frontend failure with potentially broad interactions among asset delivery, caching, container lifecycle, and deployment state. The right issue is a comparatively contained provider-integration change involving endpoint routing, model translation, and focused regression tests.
Supporting a new provider-hosted model path requires broader integration work across routing, authentication/base-URL handling, translation, and regression coverage, with higher compatibility risk. The other issue is a localized type-handling and logging fix with limited behavioral scope.
#0 of 0 · 31d19h2m37s ago — current · #import:https:::github.com:berriai:litellm post #1922
12875 requires tracing configuration precedence, persistence, startup synchronization, and rollout behavior across proxy state and storage, with substantial regression risk; 30657 is comparatively localized provider/model routing work with focused tests.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search