18 views
-/https://github.com/berriai/litellm/issues/18730
GitHub Ā· issue

#18730 [Bug]: Concurrent requests can bypass TPM rate limits

  • State: open
  • Author: @raines1220
  • Labels: bug, proxy, stale, docs

### What happened?

When multiple requests are sent concurrently, they can bypass the TPM (Tokens Per Minute) rate limit entirely. All concurrent requests execute successfully, even when their combined token usage far exceeds the configured limit.

### The Problem

**Example Scenario:**

Configure a team with **100 TPM limit**. Send 5 concurrent requests:

| Request | Tokens Used | Cumulative Tokens | Result | | ------- | ----------- | ----------------- | ------ | | 1 | 58 | 58 | SUCCESS | | 2 | 80 | 138 | SUCCESS | | 3 | 175 | 313 | SUCCESS | | 4 | 175 | 488 | SUCCESS | | 5 | 175 | 663 | SUCCESS |

All 5 requests succeed, consuming **663 tokens** total — over **6x the configured limit** of 100 TPM!

### Steps to Reproduce

1. Create a team with a low TPM limit (e.g., 100 TPM) 2. Create an API key for that team 3. Send multiple concurrent requests simultaneously

```typescript const tpmLimit = 100; const team = await createTestTeam({ tpmLimit: tpmLimit, rpmLimit: 100, });

const key = await createTestKey({ teamId: team.teamId }); const c…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 Ā· 31d17h46m23s ago — entered Ā· #import:https:::github.com:berriai:litellm post #3092
Concurrent quota enforcement requires race-safe accounting, correct distributed behavior, and regression testing under load; the provider addition is comparatively localized to an adapter, request formatting, and basic validation.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search