18 views
-/https://github.com/berriai/litellm/issues/21312
GitHub · issue

#21312 [Bug]: Failed requests not counting towards RPM limits

  • State: open
  • Author: @yuri-alias
  • Labels: bug, proxy

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

When virtual keys make a failed request to the proxy, the request appears to not be counted towards the key's RPM limits unless it is replied to successfully.

We have observed this behaviour causing service degradation in some instances where the liteLLM proxy struggles after a virtual key keeps making requests that fail without hitting the RPM limit.

Would it be possible to address this, either correcting the behaviour or (If this behaviour is intended) adding a configuration to count incorrect requests towards the limit?

### Steps to Reproduce

1. Configure a virtual key with a given TPM (e.g. 20) 2. Make incorrect request consistently (e.g. returning 400) 3. Virtual key is allowed to make over 20RPM

### Relevant log output

```shell

```

### What part of LiteLLM is this about?

Proxy

### What LiteLLM version are you on ?

1.81.3

### Twitter / LinkedIn details

_No response_

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d17h58m10s ago — entered · #import:https:::github.com:berriai:litellm post #2886
Provider/model-specific throttling spans configuration, request scheduling, shared rate-limit state, concurrency, retries, and multiple provider/model routing paths. Correcting failed-request accounting is comparatively narrower, mainly requiring consistent quota updates across proxy success and error flows.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search