18 views
-/https://github.com/berriai/litellm/issues/27767
GitHub · issue

#27767 [Bug]: litellm.BadRequestError when using llm_as_a_judge guardrail with self-hosted vllm model

  • State: open
  • Author: @jeffoxenberg
  • Labels: bug, proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

I am running a self-hosted model using vLLM (google/gemma-4-31B-it). I have LiteLLM installed via helm chart in Kubernetes with the model added using the vllm provider. I have also tried the OpenAI compatible endpoint provider. In both cases, the model itself works fine in LiteLLM. I can query it programmatically and from the playground.

However, when I try to add a LLM as a Judge guardrail backed by the model, the guardrail execution fails with ___init__.py:177 - llm_as_a_judge guardrail: judge call failed, failing open. Error: litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=google/gemma-4-31B-it_.

### Steps to Reproduce

1. Install v1.83.14 2. Deploy self-hosted model using vllm 3. Models + Endpoints -> LLM Credentials -> Add Credential (use provider vllm) 4. Models + Endpoints -> Add Model (provider vllm, add name of model, choose existing creds from previous step) 5. Guardrails -> Add New Guardrail -> Add Provider Guardrail -> Guardrail Provider LiteLLM LLM as a Judge …

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d18h19m31s ago — entered · #import:https:::github.com:berriai:litellm post #2530
The left task spans cross-component provider resolution, guardrail execution, proxy configuration, and self-hosted backend compatibility, creating broader integration and regression risk. The right task is comparatively localized to request transformation logic with a focused unit-test fix.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search