6 views
-/https://github.com/berriai/litellm/issues/32731
GitHub · issue

#32731 [Bug]: `:generateContent` rejects spec-compliant requests omitting `contents[].role` on the vertex_ai path

  • State: open
  • Author: @phuc2115
  • Labels: llm translation

### What happened?

On the Google-native endpoints `/v1beta/models/{model}:generateContent` and `:streamGenerateContent`, a request that omits `contents[].role` fails with **400** when the model is routed to `vertex_ai/*`.

Per the [Gemini API reference for `Content`](https://ai.google.dev/api/caching#Content), `role` is:

> **Optional.** The producer of the content. Must be either 'user' or 'model'. Useful to set for multi-turn conversations, otherwise can be left blank or unset.

So the request is spec-compliant against the API surface these endpoints expose, but LiteLLM forwards `contents` to Vertex unchanged, and Vertex (unlike Google AI Studio) requires the field.

This bit us on a real migration: switching a model group from `gemini/gemini-2.5-flash` to `vertex_ai/gemini-2.5-flash` — a change that should be invisible to clients, and is the entire value proposition of a gateway — broke every downstream caller that had left `role` unset. Nothing in the client changed; only the backend did.

### The inconsistency

LiteLLM already defaults this field, but only on one of the two paths.

`litellm/google_genai/adapters/transformation.py` (v1.88.0, line 384) — the Google → OpenAI ada…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d18h55m7s ago — entered · #import:https:::github.com:berriai:litellm post #1957
The right-hand issue is harder because it requires tracing and reconciling multiple proxy request paths, callback lifecycle behavior, async execution, and regression coverage across endpoint types. The left-hand issue is comparatively localized to request normalization for one provider path with focused transformation tests.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search