8 views
-/https://github.com/berriai/litellm/issues/28599
GitHub · issue

#28599 [Bug] Streaming requests bypass content_policy_fallbacks: 4xx filter in streaming_handler raises ContentPolicyViolationError directly

  • State: open
  • Author: @easyicetrusty
  • Labels: llm translation

## Summary

In streaming requests, `litellm.exceptions.ContentPolicyViolationError` is currently treated as a non-retriable client error and raised directly, bypassing both `fallbacks` and `content_policy_fallbacks`. The non-streaming path handles CPV correctly; only the streaming code path has the bug.

## Where

`litellm/litellm_core_utils/streaming_handler.py`, `CustomStreamWrapper._handle_stream_fallback_error`:

```python # Raise non-retriable client errors directly (skip fallback). # Exception: 429 (rate-limit) IS retriable/transient — allow it # through so the Router can switch to a different model group. if ( mapped_status_code is not None and 400 <= mapped_status_code < 500 and mapped_status_code != 429 ): raise mapped_exception ```

`ContentPolicyViolationError` has `status_code = 400`, so this branch fires and raises directly. The `raise MidStreamFallbackError(...)` further down is never reached, the router's fallback chain never engages, and the conversation 500s.

## Why this matters

`content_policy_fallbacks` is documented as the canonical way to route around provider content-moderation rejections (DashScope/Qwen "inappropriate content", Azure OpenAI …

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d18h58m59s ago — entered · #import:https:::github.com:berriai:litellm post #1898
The right issue is harder because it spans dashboard form state, request construction, and backend persistence semantics, with security-sensitive masking and backward-compatibility cases requiring coordinated changes and broader tests. The left issue is comparatively localized to streaming error classification and fallback behavior, mainly requiring a focused code change plus regression coverage.
#0 of 0 · 31d17h43m45s ago — current · #import:https:::github.com:berriai:litellm post #3161
Issue #28599 is harder because it crosses streaming error classification, mid-stream recovery, router fallback behavior, and regression testing across provider-specific paths. Issue #35334 is comparatively localized to request-parameter sanitization in one translation path with a focused regression test.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search