6 views
-/https://github.com/berriai/litellm/issues/30371
GitHub · issue

#30371 [Bug]: Bedrock invoke path leaks LiteLLM-internal params into the request body (Converse already filters them)

  • State: open
  • Author: @kimnamu
  • Labels: llm translation

Thanks for LiteLLM — the unified gateway has been hugely useful for working across Bedrock and other providers.

## Problem

The Bedrock **invoke** transform leaks LiteLLM-internal `optional_params` into the provider request body. After filtering out AWS-auth keys, it splats the remaining `inference_params` straight into the request:

```python # litellm/llms/bedrock/chat/invoke_transformations/base_invoke_transformation.py (transform_request) inference_params = {k: v for k, v in inference_params.items() if k not in self.aws_authentication_params} ... request_data = {"prompt": prompt, **inference_params} # internal knobs leak through ```

Internal control flags (e.g. the MCP handler keys `skip_mcp_handler` / `_skip_mcp_handler`) are not valid Bedrock inference parameters, so leaking them into the body can make Bedrock reject the request.

## Why this is a gap in an existing pattern

The sibling **Converse** path already guards against exactly this:

```python # litellm/llms/bedrock/chat/converse_transformation.py additional_request_params = filter_internal_params(additional_request_params) ```

The invoke path never calls `filter_internal_params` — it relies …

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (5 events)
#0 of 0 · 31d19h9m2s ago — entered · #import:https:::github.com:berriai:litellm post #1720
The right-hand task is harder because it requires redesigning shared router state, defining safe data-retention boundaries, and validating concurrency and compatibility across retry and logging paths, whereas the left-hand task is a localized transformation fix with comparatively narrow tests.
The left task is harder because it requires reconciling structured-output semantics across LiteLLM’s Responses abstraction and Bedrock’s model-specific request formats, with broader compatibility and regression-testing risk. The right task is comparatively localized: applying an existing sanitization pattern to one transformation path and adding focused coverage.
The right issue requires a targeted provider-path code change, compatibility validation, and regression coverage across request transformations. The left issue is primarily configuration semantics and documentation clarification with comparatively limited engineering scope.
The left issue spans cache-control injection, prompt-management integration, and model registration behavior, requiring broader tracing across shared proxy and provider flows. The right issue is a localized parity fix with focused filtering and regression tests.
#0 of 0 · 31d18h30m49s ago — current · #import:https:::github.com:berriai:litellm post #2350
The left requires tracing and correcting cross-cutting routing, alias resolution, provider dispatch, and fallback behavior, with broader compatibility and regression risk. The right is a localized request-transformation fix with a comparatively narrow test surface.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search