8 views
-/https://github.com/berriai/litellm/issues/32620
GitHub · issue

#32620 [Feature]: Rate Limits for provider

  • State: open
  • Author: @wy6688
  • Labels: enhancement, claude code

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### The Feature

Many LLM Providers have rate limits, but client side such as Claude Code, Codex don't limits the request speed. I have got many RateLimit exceptions in LiteLLM because the provider's Rate Limits.

Is There a way to set a rate limits for per provider based or per model based?

### Motivation, pitch

I have got many RateLimit exceptions in LiteLLM because the provider's Rate Limits, and have no way to control it

### What part of LiteLLM is this about?

_No response_

### LiteLLM is hiring a founding backend engineer, are you interested in joining us and shipping to all our users?

No

### Twitter / LinkedIn details

_No response_

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d18h24m46s ago — entered · #import:https:::github.com:berriai:litellm post #2426
The left issue is substantially harder because it requires designing and integrating configurable throttling across provider/model routing, request scheduling, concurrency, configuration, and potentially distributed deployments. The right issue is a more localized dashboard data/filtering defect with a narrower validation surface.
#0 of 0 · 31d17h57m53s ago — current · #import:https:::github.com:berriai:litellm post #2886
Provider/model-specific throttling spans configuration, request scheduling, shared rate-limit state, concurrency, retries, and multiple provider/model routing paths. Correcting failed-request accounting is comparatively narrower, mainly requiring consistent quota updates across proxy success and error flows.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search