8 views
-/https://github.com/berriai/litellm/issues/32895
GitHub · issue

#32895 Support max_retries for non-OpenAI/Azure providers

  • State: open
  • Author: @ammmanism
  • Labels: llm translation

### Description `litellm.completion(..., max_retries=N)` is only honored for OpenAI and Azure. For every other provider (Anthropic, Cohere, Bedrock, Vertex, etc.) the parameter is silently dropped. No retries happen and no error is raised.

### Reproduction ```python import litellm

# Works: OpenAI retries at the SDK layer litellm.completion(model="gpt-4o", messages=[...], max_retries=5)

# No effect: Anthropic never retries, param is ignored litellm.completion(model="anthropic/claude-3-5-sonnet-20241022", messages=[...], max_retries=5) ```

### Root cause In `litellm/utils.py` `get_optional_params`, `_check_valid_arg` explicitly skips `max_retries` for non-OpenAI providers (legacy TODO). OpenAI/Azure consume it via the SDK client constructor; other providers never receive it and have no retry transport configured.

### Expected behavior `max_retries` should be a supported, retry-honored parameter for all providers, consistent with OpenAI/Azure.

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d19h29s ago — entered · #import:https:::github.com:berriai:litellm post #1854
The security change is harder because it affects authorization boundaries, key and tenant attribution semantics, compatibility, and security-sensitive request-path behavior. The retry change has broader provider coverage, but can largely follow existing retry abstractions and provider-specific client behavior. Overall effort and risk favor the security issue as moderately harder.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search