16 views
-/https://github.com/berriai/litellm/issues/26387
GitHub · issue

#26387 [Bug]: BedrockGuardrail streaming hook drops HTTPException after HTTP 200, producing broken SSE stream

  • State: open
  • Author: @orolega
  • Labels: bug, proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

## Description

When a Bedrock guardrail hard-blocks a streamed response, `async_post_call_streaming_iterator_hook` raises an `HTTPException` inside the async generator body. By the time the guardrail runs, the HTTP 200 header has already been written to the client — the exception cannot be converted to a 4xx response. Instead it propagates up, truncating the SSE stream mid-flight. The client receives a partial or malformed response with no indication of why.

## Expected behaviour

The client receives a well-formed SSE error frame followed by `[DONE]`:

data: {"error": {"message": "Violated guardrail policy", "code": 400, "type": "guardrail_violation"}}

data: [DONE]

## Actual behaviour

The `HTTPException` escapes the generator. The stream is truncated and the client has no structured error to parse.

## Root cause

`async_post_call_streaming_iterator_hook` has no `try/except` around the `make_bedrock_api_request` / `asyncio.gather` calls. Both the parallel INPUT+OUTPUT path (`should_validate_input=True`) …

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d17h47m39s ago — entered · #import:https:::github.com:berriai:litellm post #3094
Budget admission changes require designing safe semantics for unknown pricing across concurrent and distributed requests, preserving existing enforcement modes, and adding broad regression coverage. The streaming fix is narrower: contain the hook failure and emit a protocol-compliant terminal SSE response, mainly requiring careful async-generator and client-compatibility tests.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search