9 views
-/https://github.com/berriai/litellm/issues/35133
GitHub Β· issue

#35133 [Bug]: Vertex batch create returns opaque HTTP 500 (list index out of range), real error/status dropped

  • State: open
  • Author: @jimmychen-p72
  • Labels: proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

`POST /v1/batches` create calls routed to `vertex_ai` (`custom_llm_provider=vertex_ai`) sometimes return an opaque **HTTP 500** with a Python `list index out of range` traceback, instead of surfacing the real Vertex provider error/status code. This makes it impossible to tell whether the failure is a client-side request problem (bad params) or a genuine provider-side error β€” we just see a generic 500 with no actionable detail.

**Best-effort root cause (please confirm/correct):** somewhere in the Vertex batch-create response-handling path, the code appears to index into a list (e.g. assuming a non-empty response/error array from the Vertex API) without guarding for an empty or unexpected-shape payload. When Vertex returns something outside that assumed shape, indexing raises `IndexError: list index out of range`, and that exception β€” rather than the real Vertex error β€” is what surfaces to the caller as a bare 500.

### Steps to Reproduce

1. Configure a Vertex AI model for batch use (`custom_llm_provider=vertex_ai`). 2. Create a batch via `POST…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 Β· 31d17h43m46s ago β€” entered Β· #import:https:::github.com:berriai:litellm post #3246
33955 is harder because it requires redesigning batch-output processing for bounded memory, safe concurrency, and regression-resistant cost tracking across workers, whereas 35133 is primarily localized defensive error handling and status propagation.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search