6 views
-/https://github.com/berriai/litellm/issues/27363
GitHub · issue

#27363 [Bug]: aembedding missing num_retries kwarg causes zero retries and no failover for embedding model groups

  • State: open
  • Author: @dredwilliams
  • Labels: bug, proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

### Bug Description

When using the LiteLLM proxy router with multiple deployments of the same embedding `model_name` across different hosts, failover never occurs when a host is unreachable. The router attempts the selected deployment once, fails, and returns an error — no retries, no failover to other deployments in the model group.

### Root Cause

In `litellm/router.py`, the `aembedding()` method does not set `num_retries` in kwargs before calling `async_function_with_fallbacks()`. Compare `aembedding` to every other router method (`acompletion`, `acreate_file`, `atext_completion`, etc.) which all include this line:

```python kwargs["num_retries"] = kwargs.get("num_retries", self.num_retries) ```

**`aembedding` (missing the line — bug):** ```python async def aembedding(self, model, input, is_async=True, **kwargs): try: kwargs["model"] = model kwargs["input"] = input kwargs["original_function"] = self._aembedding # ✓ # kwargs["num_retries"] = ... # ✗ MISSING sel…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d17h47m25s ago — entered · #import:https:::github.com:berriai:litellm post #3181
The right-hand task is harder because it requires tracing model identity through Azure router selection and the spend-logging pipeline, preserving consistency across response metadata, callbacks, and persistence, with broader regression-testing risk. The left-hand task is a narrowly scoped retry-parameter propagation fix with an established pattern to follow.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search