8 views
-/https://github.com/berriai/litellm/issues/32201
GitHub · issue

#32201 [Bug]: `post_call` guardrails attached to a pass-through endpoint are consulted but never enforce (e.g. `tool_permission` does not block)

  • State: open
  • Author: @jt-helsinki
  • Labels: bug, proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

A `tool_permission` guardrail with `mode: post_call`, `default_action: deny`, `on_disallowed_action: block` attached to a `pass_through_endpoints` route is **loaded, attached, and considered** for the request (the debug logs confirm all three), but it **never inspects the upstream response body and never blocks**. A response containing a `tool_use`/tool-call that the rule is configured to deny is relayed back to the client unchanged, with HTTP 200.

The docs at https://docs.litellm.ai/docs/proxy/pass_through_guardrails state guardrails run on pass-through endpoints and support `pre_call`/`post_call`, so I expected a `post_call` `deny` rule to block. It does not — for a tool-call guardrail the tool call only exists in the *response*, and `post_call` is the guardrail's only supported hook, so there is no working mode to enforce it on a pass-through.

Root cause appears to be that `tool_permission`'s response inspection operates on LiteLLM's **normalized** `ModelResponse` (`choices[].message.tool_calls`), which is never produced for a raw pass-thr…

Open on GitHub

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (21 events)
#0 of 0 · 31d18h17m52s ago — entered · #import:https:::github.com:berriai:litellm post #2540
-/https://github.com/berriai/litellm/issues/32201⭐4:1-/https://github.com/berriai/litellm/issues/31882
The left issue is harder because it crosses proxy request/response lifecycle handling, guardrail enforcement semantics, raw pass-through payload inspection, and likely streaming and regression-test behavior. The right issue is comparatively localized to provider request transformation and capability-gated field mapping.
#0 of 0 · 31d17h43m41s ago · #import:https:::github.com:berriai:litellm post #3137
-/https://github.com/berriai/litellm/issues/32201⭐3:2-/https://github.com/berriai/litellm/issues/13786
The right issue is harder because it crosses pass-through routing, response-format normalization, guardrail enforcement semantics, and likely multiple provider and streaming paths. The left issue appears more localized to streaming error-state termination and callback lifecycle handling, with a narrower validation surface.
#0 of 0 · 31d17h42m59s ago · #import:https:::github.com:berriai:litellm post #3149
-/https://github.com/berriai/litellm/issues/342415:1-/https://github.com/berriai/litellm/issues/32201⭐
34241 requires a broad repository-wide architectural and licensing audit, relocation or redesign of shared gating mechanisms, compatibility review, and coordinated documentation/testing. 32201 is comparatively localized to pass-through response inspection and enforcement, with focused regression coverage.
#0 of 0 · 31d17h33m17s ago · #import:https:::github.com:berriai:litellm post #3312
-/https://github.com/berriai/litellm/issues/32201⭐5:3-/https://github.com/berriai/litellm/issues/26897
The right issue is harder because it spans proxy guardrail execution, pass-through response handling, provider-specific payload normalization, blocking semantics, and security-sensitive regression coverage. The left issue is narrower, mainly involving parameter propagation and Azure Responses bridge URL construction across an integration path.
#0 of 0 · 31d17h33m13s ago · #import:https:::github.com:berriai:litellm post #3313
-/https://github.com/berriai/litellm/issues/265353:1-/https://github.com/berriai/litellm/issues/32201⭐
Model omitted braces; inferred difficulty from issue scope and surface area.
#0 of 0 · 31d17h32m58s ago · #import:https:::github.com:berriai:litellm post #3317
-/https://github.com/berriai/litellm/issues/32201⭐3:1-/https://github.com/berriai/litellm/issues/32561
The left issue is harder because it crosses guardrail enforcement, pass-through response handling, provider-format normalization, and potentially streaming paths, with security-sensitive regression coverage. The right issue is comparatively localized to early-error propagation and protocol-correct SSE termination in one integration path.
#0 of 0 · 31d17h32m55s ago · #import:https:::github.com:berriai:litellm post #3318
-/https://github.com/berriai/litellm/issues/32201⭐4:1-/https://github.com/berriai/litellm/issues/25628
The left issue spans pass-through response handling, guardrail enforcement semantics, raw-to-normalized response translation, and likely multiple response formats, creating broader regression and testing risk. The right issue is comparatively localized to preserving the configured alias through the streaming callback path.
#0 of 0 · 31d17h32m52s ago · #import:https:::github.com:berriai:litellm post #3319
-/https://github.com/berriai/litellm/issues/347335:3-/https://github.com/berriai/litellm/issues/32201⭐
#34733 is harder because it requires correcting concurrent distributed budget accounting, atomic window transitions, and asynchronous persistence reliability across replicas. #32201 is narrower, mainly requiring a pass-through response normalization and guardrail enforcement integration.
#0 of 0 · 31d17h32m46s ago · #import:https:::github.com:berriai:litellm post #3320
-/https://github.com/berriai/litellm/issues/32201⭐3:2-/https://github.com/berriai/litellm/issues/30501
The left requires cross-cutting changes to the proxy’s response-processing and policy-enforcement pipeline, including raw-payload handling and preservation of existing guardrail semantics. The right is primarily a provider-specific multimodal translation addition with validation and targeted compatibility tests, making it narrower and lower risk.
#0 of 0 · 31d17h31m48s ago · #import:https:::github.com:berriai:litellm post #3337
-/https://github.com/berriai/litellm/issues/32201⭐7:4-/https://github.com/berriai/litellm/issues/30641
The left issue is harder because it crosses the pass-through execution path, raw provider response formats, guardrail enforcement semantics, and regression testing across endpoint variants. The right issue is primarily a configuration-discovery/API and dashboard presentation change with a narrower integration surface.
#0 of 0 · 31d17h30m14s ago · #import:https:::github.com:berriai:litellm post #3363
-/https://github.com/berriai/litellm/issues/262376:5-/https://github.com/berriai/litellm/issues/32201⭐
The right issue is harder because it requires cross-cutting worker lifecycle, readiness gating, retry recovery, state consistency, and race-condition testing across distributed dependencies. The left issue is narrower, mainly requiring response interception, format handling, and guardrail enforcement across pass-through paths.
#0 of 0 · 31d17h29m33s ago · #import:https:::github.com:berriai:litellm post #3373
-/https://github.com/berriai/litellm/issues/97534:1-/https://github.com/berriai/litellm/issues/32201⭐
The Prisma-related work would likely require a broad dependency replacement touching data access, generated clients, schema/migration workflows, compatibility behavior, and extensive regression testing. The guardrail issue appears narrower: adapting one pass-through response path so an existing enforcement mechanism can inspect it correctly, with focused proxy and integration-test changes.
#0 of 0 · 31d17h28m49s ago · #import:https:::github.com:berriai:litellm post #3385
-/https://github.com/berriai/litellm/issues/32201⭐3:1-/https://github.com/berriai/litellm/issues/32456
32201 is harder because it requires extending post-response guardrail enforcement across a raw pass-through path, including response interpretation, blocking semantics, and regression-safe integration with existing proxy behavior. 32456 is comparatively contained to provider-specific multipart input plumbing and request translation.
#0 of 0 · 31d17h28m35s ago · #import:https:::github.com:berriai:litellm post #3389
-/https://github.com/berriai/litellm/issues/238413:2-/https://github.com/berriai/litellm/issues/32201⭐
23841 spans multiple translation paths and compatibility cases, requiring broader behavioral changes and regression coverage; 32201 is a more localized enforcement-path integration fix.
#0 of 0 · 31d17h24m11s ago · #import:https:::github.com:berriai:litellm post #3462
-/https://github.com/berriai/litellm/issues/32201⭐3:1-/https://github.com/berriai/litellm/issues/26700
#32201 is harder because it spans pass-through request/response handling, normalization, guardrail enforcement semantics, and likely streaming and regression-test coverage. #26700 is comparatively narrower: isolating an asynchronous lifecycle incompatibility and applying or coordinating a targeted SDK/workaround change.
#0 of 0 · 31d17h23m16s ago · #import:https:::github.com:berriai:litellm post #3477
-/https://github.com/berriai/litellm/issues/32201⭐4:3-/https://github.com/berriai/litellm/issues/27989
The left issue is harder because it crosses pass-through response handling, guardrail enforcement, provider-specific payload interpretation, and regression coverage. The right issue is primarily a focused transactional/concurrency correction with a narrower implementation surface.
#0 of 0 · 31d17h22m34s ago · #import:https:::github.com:berriai:litellm post #3488
-/https://github.com/berriai/litellm/issues/32201⭐3:1-/https://github.com/berriai/litellm/issues/31296
Issue 32201 is harder because it requires integrating enforcement into pass-through response processing, including raw/streaming formats and guardrail execution paths, while issue 31296 is a more bounded authentication-metadata and response-header addition.
#0 of 0 · 31d17h22m30s ago · #import:https:::github.com:berriai:litellm post #3489
-/https://github.com/berriai/litellm/issues/343283:1-/https://github.com/berriai/litellm/issues/32201⭐
#34328 is harder because it requires correcting recursive schema expansion safeguards across multiple provider translation paths, addressing algorithmic termination and resource limits, and validating complex performance regressions. #32201 is narrower endpoint/response-processing integration work with a more localized enforcement fix.
#0 of 0 · 31d17h18m10s ago · #import:https:::github.com:berriai:litellm post #3548
-/https://github.com/berriai/litellm/issues/336663:1-/https://github.com/berriai/litellm/issues/32201⭐
The right issue is harder because it requires redesigning data retrieval and session reconstruction for bounded memory while preserving ordering and conversation correctness across large histories, with database-level performance validation. The left issue is more localized to adapting guardrail response inspection for an alternate response representation and adding focused enforcement tests.
#0 of 0 · 31d17h17m33s ago · #import:https:::github.com:berriai:litellm post #3555
-/https://github.com/berriai/litellm/issues/32201⭐3:2-/https://github.com/berriai/litellm/issues/27755
32201 is harder because it spans pass-through response handling, guardrail enforcement semantics, normalization boundaries, and regression testing across proxy execution paths. 27755 is more localized to provider-specific subscription/image-generation compatibility and error handling.
#0 of 0 · 31d17h16m3s ago — current · #import:https:::github.com:berriai:litellm post #3573
-/https://github.com/berriai/litellm/issues/32201⭐3:2-/https://github.com/berriai/litellm/issues/33325
Issue 32201 is harder because it requires integrating post-response policy enforcement with unnormalized pass-through traffic, preserving blocking semantics across response formats, and adding broad regression coverage. Issue 33325 appears more localized to limiter initialization and distributed-cache admission consistency.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search