6 views
-/https://github.com/berriai/litellm/issues/27900
GitHub · issue

#27900 [Bug]: set global_max_parallel_requests not work

  • State: open
  • Author: @coach00
  • Labels: bug, proxy

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

I set liteLLM global_max_parallel_requests

<img width="1324" height="185" alt="Image" src="https://github.com/user-attachments/assets/23d27165-3b79-46c9-8c02-47d47ad7cc94" />

It is hoped that the number of requests can be limited to maintain server stability.

but in fact,it has no use to protect my server, there still has too many requests

<img width="1676" height="47" alt="Image" src="https://github.com/user-attachments/assets/15106f37-2ee4-4a69-8b06-b3336f0c6f65" />

### Steps to Reproduce

1. set global_max_parallel_requests 2. just request llm 3. The number of concurrent requests can still exceed the limit.

### Relevant log output

```shell

```

### What part of LiteLLM is this about?

Proxy

### What LiteLLM version are you on ?

1.82.6

### Twitter / LinkedIn details

_No response_

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d18h33m49s ago — entered · #import:https:::github.com:berriai:litellm post #2491
The right-hand feature spans a new CLI workflow, multiple external agent integrations, model discovery, authentication, configuration, and cross-platform testing. The left-hand bug is narrower, centered on diagnosing and correcting concurrency-limit enforcement and adding regression coverage.
#0 of 0 · 31d18h27m56s ago — current · #import:https:::github.com:berriai:litellm post #2586
The left task requires diagnosing and safely correcting a subtle interaction between deferred Pydantic schema construction, SDK model parsing, streaming serialization, and tool-call/logprob object lifecycles, with regression coverage across intermittent stream paths. The right task is narrower: tracing proxy concurrency accounting and configuration enforcement, though it may require attention to async or distributed request handling. Overall implementation risk and cross-cutting scope are substantially higher on the left.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search