6 views
-/https://github.com/berriai/litellm/issues/31004
GitHub · issue

#31004 [Bug]: Guardrails are triggered on errors from provider's API and complicate troubleshooting

  • State: open
  • Author: @iavael
  • Labels: bug, proxy

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

Guardrails can trigger during handling of error response from provider which leads to "exception raised during exception handling" not being properly handled

### Steps to Reproduce

1. Create guardrais which would trigger on content of error message from provider 2. Make such request which would lead to error response from provider 3. Observe broken exception handling in logs

### Relevant log output

```shell 14:37:51 - LiteLLM Proxy:ERROR: proxy_server.py:7096 - litellm.proxy.proxy_server.async_data_generator(): Exception occured - litellm.APIError: APIError: Openrou terException - Message: Provider returned error, Metadata: {'error_type': 'permission_denied'}, User ID: Traceback (most recent call last): File "/app/.venv/lib/python3.13/site-packages/litellm/litellm_core_utils/streaming_handler.py", line 2087, in __anext__ async for chunk in self.completion_stream: # type: ignore[union-attr] ...<84 lines>... return processed_chunk File "/app/.venv/lib/python3.13/site-packages/litellm/llms/base_llm/base_model_iterator.py…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d18h11m57s ago — entered · #import:https:::github.com:berriai:litellm post #2664
31004 requires coordinating failure-path control flow across proxy, streaming, and guardrail integrations, with higher regression and observability risk; 30778 is comparatively localized parameter propagation through HTTP client construction and retries.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search