9 views
-/https://github.com/berriai/litellm/issues/29138
GitHub · issue

#29138 [Bug]: Anthropic input_audio base64 leaks into Gemini text tokens causing inflated input token counts

  • State: open
  • Author: @gurvkm
  • Labels: bug, proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

**Description** When using LiteLLM to proxy requests from Anthropic format to Gemini Flash (via Google AI Studio), we are intermittently seeing a massive spike in input text tokens. The base64-encoded audio string from the input_audio block appears to be leaking into the text content of the request sent to Gemini, causing it to be tokenized as text in addition to being correctly processed as audio.

**What happened?** We pass audio as inline base64 in the Anthropic input_audio block. LiteLLM translates this to Gemini's native format. Intermittently (~25% of calls on peak days), the translation fails and the raw base64 string leaks into the text content of the Gemini request. Gemini then charges both audio tokens AND text tokens for the same audio data.

Expected text tokens per call: ~1,750 (system prompt only) Observed text tokens on spiked calls: ~8,500–10,000 Spurious extra tokens per call: ~6,800–8,500 Output quality is NOT affected — extraction results remain accurate

### Steps to Reproduce

1. Use LiteLLM v1.84.0 with a Gemini Flash mode…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d18h46m56s ago — entered · #import:https:::github.com:berriai:litellm post #2079
The traffic-mirroring issue is harder because it likely spans proxy orchestration, configuration resolution, background execution, error handling, and cross-provider integration testing. The other issue is more localized to multimodal request translation and can likely be addressed with focused conversion logic and regression tests.
#0 of 0 · 31d17h40m0s ago — current · #import:https:::github.com:berriai:litellm post #3198
The right issue is harder because it involves intermittent, cross-provider multimodal translation behavior, requiring careful request normalization, regression coverage, and validation of token accounting without breaking audio handling. The left issue is more contained: integrating existing database-view creation into the migration lifecycle and verifying permissions and startup behavior.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search