8 views
-/https://github.com/berriai/litellm/issues/31881
GitHub · issue

#31881 feat(akto): add during_call support, streaming end-of-stream mode, and model-group scoping

  • State: open
  • Author: @deepanshululla

## Problem

The Akto guardrail integration currently only supports pre-call and post-call hooks. Three gaps: 1. No `during_call` mode — cannot inspect streaming chunks as they arrive 2. No end-of-stream accumulation mode for streaming — post-call fires before the full response is assembled 3. Model-group scoping is absent — Akto checks fire on every model regardless of which model group the request targets

## Proposed fix

Add `during_call` hook support to the Akto guardrail. Add an end-of-stream accumulation mode that buffers streaming chunks and fires the Akto check once the stream completes. Add a `model_group` scoping field so Akto checks can be restricted to specific model groups.

## Files

- `litellm/integrations/` (Akto guardrail implementation)

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (4 events)
#0 of 0 · 31d19h18m38s ago — entered · #import:https:::github.com:berriai:litellm post #1664
31881 spans multiple guardrail lifecycle paths, streaming state management, and configuration scoping, creating broader integration and regression risk; 24709 is comparatively localized to MCP authentication and health-check control flow.
Issue 31881 is harder because it spans guardrail lifecycle hooks, streaming state/buffering, end-of-stream behavior, model-group configuration, and compatibility across request modes. Issue 34296 is primarily concentrated in authorization decision logic with targeted regression tests and edge-case handling.
Akto requires coordinated guardrail lifecycle changes, streaming buffering/end-of-stream handling, hook semantics, model-group filtering, and integration tests. The Redis-enabled UI defect is narrower, likely involving cache event persistence/query behavior and frontend filtering.
#0 of 0 · 31d18h23m49s ago — current · #import:https:::github.com:berriai:litellm post #2550
Akto work spans multiple hook lifecycles, streaming state management, response buffering, and model-routing configuration, creating broader integration and regression risk. The onboarding bug is comparatively localized to authentication-header handling across the unauthenticated UI flow.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search