8 views
-/https://github.com/berriai/litellm/issues/16068
GitHub · issue

#16068 [Feature]: Support Exponential Backoff for completion()

  • State: open
  • Author: @porteron
  • Labels: enhancement

### The Feature

Configurable retry delay and exponential backoff/jitter for `completion()` api.

### Motivation, pitch

The `completion()` api only allows to configure the `num_retries` for the `completion` and `acompletion` functions. It would be useful to be able to specify the retry delay and have exponential backoff/jitter ability. Currently this is only available via the litellm `Router`.

https://docs.litellm.ai/docs/completion/input#litellm-specific-params

Thank you

### LiteLLM is hiring a founding backend engineer, are you interested in joining us and shipping to all our users?

No

### Twitter / LinkedIn details

_No response_

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d19h48s ago — entered · #import:https:::github.com:berriai:litellm post #1949
A configurable retry policy spans synchronous and asynchronous execution paths, public parameter handling, timing and jitter semantics, backward compatibility, and broad test coverage. The telemetry failure is more likely a localized defensive fix in an existing callback path, with focused regression testing.
#0 of 0 · 31d18h58m40s ago — current · #import:https:::github.com:berriai:litellm post #1988
Core request execution semantics must be designed consistently across synchronous and asynchronous paths, with configuration, retry timing, jitter, backward compatibility, and broad regression testing. The other change is primarily localized dashboard form work plus documentation and an optional small API addition.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search