8 views
-/https://github.com/berriai/litellm/issues/33272
GitHub · issue

#33272 litellm.num_retries global is permanently set to None after any completion() exception, disabling retries process-wide

  • State: open
  • Author: @ErenAta16

### What happened?

`litellm.num_retries` (the module-level global) gets permanently set to `None` the first time any direct `litellm.completion()`/`acompletion()`/`responses()`/`aresponses()` call raises an exception, and it is never restored. Every later call in the same process that relies on the global default (does not pass `num_retries` per call) then silently gets zero retries for the rest of the process lifetime, even for completely unrelated requests and models.

### Where

`litellm/utils.py`, the exception branch inside the sync and async wrappers:

- sync `wrapper`: lines 1501-1530 (`completion` branch) and 1542-1554 (`responses` branch) - async `wrapper_async`: the equivalent branches around lines 1812-1864

```python except Exception as e: call_type = original_function.__name__ if call_type == CallTypes.completion.value: num_retries = kwargs.get("num_retries", None) or litellm.num_retries or None ... litellm.num_retries = None # set retries to None to prevent infinite loops ... if (num_retries and not _is_litellm_router_call): ... kwargs["num_retries"] = num_retries return litellm.comp…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d18h4m37s ago — entered · #import:https:::github.com:berriai:litellm post #2771
The left requires careful changes across shared sync and async retry state, regression tests, and concurrency-sensitive validation. The right is primarily a provider-quota explanation with little or no LiteLLM engineering scope.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search