8 views
-/https://github.com/berriai/litellm/issues/31613
GitHub · issue

#31613 [Bug]: Realtime health check sets ssl= on ws:// connections (incompatible) + request to support self-hosted OpenAI-compatible (vLLM) realtime provider

  • State: open
  • Author: @doss22
  • Labels: bug, proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

I'm trying to use a self-hosted **vLLM** instance that exposes an OpenAI-compatible `/v1/realtime` WebSocket endpoint (model `Voxtral-Mini-4B-Realtime-2602`) behind LiteLLM Proxy.

There are two distinct problems:

**1) `Unsupported model` when using a non-listed provider (e.g. `hosted_vllm`)**

When configuring the model with the `hosted_vllm/` (or `vllm/`) prefix and `mode: realtime`, the realtime health check fails with:

``` ValueError: Unsupported model: Voxtral-Mini-4B-Realtime-2602 ```

This comes from `_realtime_health_check` in `litellm/realtime_api/main.py`, which only builds a WebSocket URL for `azure`, `openai`, `xai` and `vertex_ai`. Every other `custom_llm_provider` hits the `else: raise ValueError(f"Unsupported model: {model}")` branch. The same limitation exists in the actual forwarding path `_arealtime`.

**2) `ssl argument is incompatible with a ws:// URI` when using the `openai/` provider over plain HTTP**

Working around (1) by using `openai/Voxtral-Mini-4B-Realtime-2602` with `api_base: http://10.13.1.150:8030/v1` gets past…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d18h6m43s ago — entered · #import:https:::github.com:berriai:litellm post #2751
The right issue is harder because it introduces a cross-cutting observability integration, external service interaction, configuration, metric enrichment, and compatibility across multiple callback backends. The left is technically sensitive but more contained within realtime routing and WebSocket transport handling.
#0 of 0 · 31d17h56m16s ago — current · #import:https:::github.com:berriai:litellm post #2933
Supporting authenticated private-resource integrations spans credential handling, secure storage, proxy/API and UI changes, multiple repository access modes, error behavior, and documentation. The realtime issue is narrower: extending provider routing and correcting WebSocket transport options, though it still touches health checks and forwarding.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search