6 views
-/https://github.com/berriai/litellm/issues/27130
GitHub · issue

#27130 [Feature]: Forward stream flag to generic_guardrail_api via GenericGuardrailAPIRequest

  • State: open
  • Author: @bcyrill
  • Labels: enhancement, SDK

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### The Feature

### Summary

GenericGuardrailAPIRequest (the wire payload posted to a generic guardrail provider's apply_guardrail endpoint) doesn't carry the original chat-completion request's stream: true/false flag. This makes it impossible for a guardrail provider to tell at request time whether the LLM call it's running on is streaming. For some guardrail use cases that distinction matters.

### Proposed change

Add a single boolean field to the wire payload:

class GenericGuardrailAPIRequest(BaseModel): ... is_streaming: bool = False

Populate it in apply_guardrail() (in litellm/proxy/guardrails/guardrail_hooks/generic_guardrail_api/generic_guardrail_api.py) by reading request_body.get("stream", False). Construction site is around the existing GenericGuardrailAPIRequest(...) call (~line 436 on current main).

### Backwards compatibility

- Receiver side: Pydantic models on the receiving service ignore unknown fields by default, so older guardrail providers running against newer LiteLLM versions are unaffected. - Sender side: Default value False matc…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d18h19m43s ago — entered · #import:https:::github.com:berriai:litellm post #2846
The right-side task is harder because it spans MCP health-check control flow, OAuth token acquisition, failure handling, and regression coverage, while the left-side task is a localized schema and request-construction change.
#0 of 0 · 31d18h18m47s ago — current · #import:https:::github.com:berriai:litellm post #2863
The left requires tracing and correcting provider-specific exception translation across layered async HTTP handling, with regression coverage for failure semantics. The right is a localized schema and parameter-propagation change with straightforward compatibility testing.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search