8 views
-/https://github.com/berriai/litellm/issues/28940
GitHub · issue

#28940 [Feature]: Flexible configuration of context compression parameters for different models

  • State: open
  • Author: @chenwei0930
  • Labels: enhancement, proxy

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### The Feature

Would it be possible to support configuring different context-compression trigger thresholds and compression target thresholds for different models?

### Motivation, pitch

I have multiple models, each with a different context window—some are 128K, and some are 64K. The configuration in the current documentation seems to apply uniformly to all models (https://docs.litellm.ai/docs/completion/prompt_compression), which appears to be quite unfavorable for models with larger context windows.

### What part of LiteLLM is this about?

Proxy

### LiteLLM is hiring a founding backend engineer, are you interested in joining us and shipping to all our users?

No

### Twitter / LinkedIn details

_No response_

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d17h48m39s ago — entered · #import:https:::github.com:berriai:litellm post #3062
The right issue spans multiple startup paths, connection-management behavior, credential-refresh logic, URL preservation, and deployment-level testing, creating higher integration risk. The left issue is comparatively localized to configuration resolution and model-specific behavior.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search