7 views
-/https://github.com/berriai/litellm/issues/34734
GitHub · issue

#34734 [Feature]: Add upstream quota and balance probing

  • State: open
  • Author: @neubig
  • Labels: SDK

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### The Feature

Add an optional, capability-based SDK interface for proactively querying quota or balance information reported by an upstream provider. This should be distinct from LiteLLM's configured budgets and spend observed by the proxy.

A possible interface:

```python snapshot = await litellm.aprobe_provider_quota( model=model, api_key=api_key, api_base=api_base, ) ```

with a normalized result along these lines:

```python ProviderQuotaSnapshot( provider="openrouter", scope="credential", # credential/account/project/subscription unit="usd", # usd/tokens/requests/percent limit=25.0, used=6.5, remaining=18.5, resets_at=None, as_of=..., ) ```

Suggested implementation:

- Add an optional `probe_quota` capability to provider configurations; unsupported providers return `None` or a typed `NotSupported` result. - Reuse LiteLLM's provider resolution and credential handling rather than requiring callers to detect providers. - Implement adapters only where providers expose a supported API, potentiall…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d17h39m42s ago — entered · #import:https:::github.com:berriai:litellm post #3223
#34734 requires a new cross-provider SDK abstraction, capability negotiation, credential/provider resolution integration, multiple adapter implementations, normalization semantics, and broad testing. #34681 is a localized cache-eviction correction with focused regression coverage, so it has substantially lower scope and risk.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search