10 views
-/https://github.com/berriai/litellm/issues/31118
GitHub · issue

#31118 extra_body can override the request `model` across all providers, bypassing proxy model-access/budget checks

  • State: open
  • Author: @mateo-berri
  • Labels: llm translation

## Summary

The shared completion path in `BaseLLMHTTPHandler` merges caller-controlled `extra_body` into the request body after `transform_request()` runs, at `litellm/llms/custom_httpx/llm_http_handler.py`:

```python if extra_body is not None: data = {**data, **extra_body} ```

Because this merge happens with `extra_body` taking precedence, a caller can set `extra_body: {"model": "<some-other-model>"}` and override the `model` field that the provider transform produced. For any OpenAI-compatible provider that reads `model` from the JSON body (OpenAI, Azure, Groq, Deepseek, Cloudflare's `/ai/v1` endpoint, and others), the upstream provider then executes the model named in `extra_body`, not the one the request was authorized and budgeted against.

Proxy model-access and budget enforcement runs at the auth layer on the top-level `model` parameter, before the request reaches any provider transform, and never inspects `extra_body`. So a key scoped to a cheap or allow-listed model could run a different, more expensive, or non-allowed model while accounting is tracked against the original `model`.

## Why this is filed separately

This came up as a review finding on #31053 (Cloudfl…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d18h10m25s ago — entered · #import:https:::github.com:berriai:litellm post #2685
31118 is harder because it requires a cross-cutting security fix in shared request construction, preserving provider-specific behavior while adding authorization-consistency checks and broad regression coverage. 27926 is comparatively localized to proxy metrics authentication and configuration handling.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search