7 views
-/https://github.com/berriai/litellm/issues/30730
GitHub · issue

#30730 [Bug]: Lakera v2 guardrail — flagged=True with empty payload takes mask-and-allow branch but masks nothing

  • State: open
  • Author: @AUTHENSOR

## Summary

In `litellm/proxy/guardrails/guardrail_hooks/lakera_ai_v2.py`, the ALLOW-vs-BLOCK decision and the masking use **different fields** of the Lakera response:

- the decision (`_is_only_pii_violation`, `:450-473`) inspects `breakdown` (true iff every detected item is `pii/*`) -> if PII-only, take the **mask-and-allow** branch instead of blocking; - the masking (`_mask_pii_in_messages`, `:143-200`) is driven by the separate `payload` field (the list of `{start, end, detector_type, ...}` spans), and **early-returns on an empty payload** (`:154`).

So when Lakera returns `flagged=True` with `breakdown=[{detector_type:"pii/email", detected:True}]` but `payload=[]` — the **exact empty-payload shape used in litellm's own test fixtures** (`test_lakera_v2.py:107`, `:272`) — `_is_only_pii_violation` returns True (mask-and-allow), `_mask_pii_in_messages` masks nothing, and the request is forwarded. The hook never verifies that masking actually happened before allowing.

## Reproduction (deterministic)

``` === Lakera verdict: flagged=True, breakdown=pii/email detected, payload=[] === FORWARDED TO LLM: 'My email test@example.com here' request allowed: True PII masked: False PI…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d18h42m8s ago — entered · #import:https:::github.com:berriai:litellm post #2373
The guardrail defect is harder because it requires safely reconciling verdict and redaction semantics, defining fail-open versus fail-closed behavior, and adding regression coverage for malformed or incomplete detector output. The Azure model addition is comparatively scoped to provider integration, capability metadata, and compatibility tests.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search