10 views
-/https://github.com/berriai/litellm/issues/35359
GitHub · issue

#35359 [Bug]: batches.create falls back to another model group, returning the wrong provider's error

  • State: open
  • Author: @sudhanshugupta-p72
  • Labels: bug, proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

With a router fallback configured from an OpenAI model group to an Azure one, an invalid `batches.create` request against the OpenAI group is retried against the Azure group, and the caller receives **Azure's** error instead of OpenAI's.

Sending `completion_window="5m"` to the OpenAI group should surface OpenAI's validation error:

```json {"error": {"type": "invalid_request_error", "param": "completion_window", "message": "... must be one of: 24h"}} ```

Instead the response carries an Azure-shaped body with `code=quotaExceeded`, which says nothing about the actual mistake. The caller has no way to work out what was wrong with their request.

Beyond the misleading error, the same fallback means batch creation can land on a deployment other than the one that owns `input_file_id`. Batch and file resources are owned by the provider that created them, so creating batch state under a different provider's credentials than the input file's is not a safe retry target.

**Expected:** file and batch operations stay within the model group tha…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d18h54m1s ago — entered · #import:https:::github.com:berriai:litellm post #1964
The left issue is harder because it requires changing retry and fallback semantics across provider-specific, stateful operations while preserving error fidelity and resource ownership. The right issue is comparatively localized to streaming response framing and protocol compatibility, with a narrower implementation and test surface.
#0 of 0 · 31d18h16m28s ago — current · #import:https:::github.com:berriai:litellm post #2569
Issue 35359 is harder because it requires changing fallback and retry semantics across batch routing, error classification, provider resource affinity, and regression coverage. Issue 27671 is narrower, primarily involving identifier lifecycle handling in one streaming translation path.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search