7 views
-/https://github.com/berriai/litellm/issues/31827
GitHub · issue

#31827 [Feature]: Force override model parameters from proxy config

  • State: open
  • Author: @limitjupyter-ops
  • Labels: enhancement, proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### The Feature

I would like LiteLLM proxy model config to support forced parameter overrides.

LiteLLM already supports default parameters and provider transformations, but enterprise operators sometimes need to enforce parameters even if the client sends different values.

Example use cases:

- Force `max_tokens` for a model to avoid upstream errors. - Force `reasoning_effort=none` for models/providers where reasoning causes instability or unexpected cost. - Strip or override unsupported provider parameters. - Enforce safety or compatibility defaults at the gateway layer.

Suggested config direction:

```yaml model_list: - model_name: my-model litellm_params: model: openai/my-upstream-model model_info: forced_params: max_tokens: 4096 reasoning_effort: none

### Motivation, pitch

In production AI gateways, some upstream providers or models require strict parameter control.

Default parameters are useful, but they do not prevent clients from overriding them. Operators sometimes need a stronger policy layer to keep traffic st…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (3 events)
#0 of 0 · 31d18h55m40s ago — entered · #import:https:::github.com:berriai:litellm post #1951
31827 is harder because it requires a new cross-cutting configuration and precedence mechanism spanning proxy validation, request parameter merging, provider translation, security semantics, and broad compatibility testing. 27138 is comparatively localized to Bedrock request construction with focused regression coverage.
Policy enforcement requires cross-cutting request-pipeline changes, clear precedence rules, provider compatibility handling, and extensive regression coverage; the metadata addition is comparatively localized to model listing, pricing resolution, and response serialization.
#0 of 0 · 31d18h35m10s ago — current · #import:https:::github.com:berriai:litellm post #2294
The right issue is harder because it introduces a cross-cutting policy layer involving configuration schema, precedence rules, parameter validation, provider transformations, and broad regression coverage. The left issue is comparatively localized to correcting data extraction in an existing interception path.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search