9 views
-/https://github.com/berriai/litellm/issues/33681
GitHub · issue

#33681 [Bug]: usage is all zeros (and inputs are fused) for gemini-embedding-2 multimodal embeddings via /v1/embeddings

  • State: open
  • Author: @huangdawei
  • Labels: llm translation

## What happens

Calling the proxy's OpenAI-compatible `/v1/embeddings` with `model=gemini/gemini-embedding-2` and an `input` array containing a media data URI (e.g. `data:image/png;base64,…`), optionally mixed with text strings:

1. The response `usage` is all zeros (`prompt_tokens: 0, total_tokens: 0`), even though the upstream native `embedContent` response includes `usageMetadata.promptTokenCount` and per-modality `promptTokenDetails` (e.g. `{"modality":"IMAGE","tokenCount":258}`). 2. The whole `input` array is fused into a single `content` and returns **one** embedding vector, whereas text-only arrays return one vector per input. The single-vector semantics may be intentional for multimodal fusion, but the asymmetry deserves a docs note.

## Why it matters

Google bills gemini-embedding-2 media input at modality-specific token rates (IMAGE 258 tokens/image, AUDIO 25 tokens/s, VIDEO 66 tokens/s — verifiable via `usageMetadata`). With `usage` zeroed, any cost tracking / spend logging / budget enforcement built on the proxy's usage fields silently sees zero cost for media embeddings.

## Expected

Propagate `usageMetadata.promptTokenCount` into `usage.prompt_tokens` / `total_toke…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (4 events)
#0 of 0 · 31d18h56m21s ago — entered · #import:https:::github.com:berriai:litellm post #2034
33681 requires provider-specific multimodal embedding translation, usage accounting across modalities, compatibility with existing per-input semantics, and likely tests/docs across response paths. 24516 is a localized observability hardening change involving type checks and attribute validation. Therefore 33681 carries substantially greater integration and regression risk.
The right issue spans provider-specific response translation, multimodal input semantics, usage accounting, cost propagation, and regression coverage, creating substantially more integration and compatibility risk. The left issue is comparatively localized to configuration-driven credential resolution and validation.
The right-hand task is harder because it crosses provider-specific multimodal request handling, response normalization, usage accounting, and compatibility tests. The left-hand task is comparatively localized parameter propagation across existing transformation paths, with narrower behavioral risk.
#0 of 0 · 31d18h34m7s ago — current · #import:https:::github.com:berriai:litellm post #2404
Model omitted braces; inferred difficulty from issue scope and surface area.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search