8 views
-/https://github.com/berriai/litellm/issues/31178
GitHub · issue

#31178 [Bug]: LiteLLM proxy fails for vLLM qwen3-vl-embedding-8b with image inputs

  • State: open
  • Author: @cyysky
  • Labels: bug, proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

# LiteLLM proxy fails for `qwen3-vl-embedding-8b` with image inputs

## Summary

When proxying the multimodal embedding model `qwen3-vl-embedding-8b` through LiteLLM to a vLLM backend, image inputs are either rejected outright or cause an internal 500. The same requests work fine when sent **directly to vLLM**, which forces us to bypass the LiteLLM proxy for any image-embedding workloads.

## Environment

- vLLM serves `qwen3-vl-embedding-8b` with an OpenAI-compatible `/v1/embeddings` route. It also accepts vLLM-native `/v1/embeddings` requests shaped like a chat completion (`messages: [{role: "user", content: [{type:"image_url", image_url:{url:...}}]}]`). - LiteLLM proxy is fronting vLLM (env vars in our `.env`):

``` OPENAI_API_BASE=http://60.51.1.128:9999/v1 # LiteLLM proxy OPENAI_API_KEY=sk-... CHAT_MODEL=qwen3.5-397b INSTRUCT_MODEL=qwen3.5-397b-instruct EMBEDDING_OPENAI_API_BASE=http://10.0.151.2:14701/v1 # vLLM direct EMBEDDING_MODEL=qwen3-vl-embedding-8b ```

The proxy is reached on the default OpenAI por…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d18h30m50s ago — entered · #import:https:::github.com:berriai:litellm post #2342
Multimodal embedding translation across proxy boundaries requires coordinating request-shape handling, image normalization, provider capability detection, error paths, and regression coverage. The streaming discrepancy is more likely isolated to Azure model-specific response adaptation and stream-mode handling, with narrower scope and lower integration risk.
#0 of 0 · 31d17h56m11s ago — current · #import:https:::github.com:berriai:litellm post #2912
The right issue is harder because it likely requires tracing and adapting multimodal request handling across proxy validation, embedding translation, provider-specific behavior, and error paths, with broader integration testing. The left issue is comparatively localized to authorization matching logic, plus focused regression tests and consistency checks with existing wildcard semantics.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search