9 views
-/https://github.com/berriai/litellm/issues/31959
GitHub · issue

#31959 [Bug]: Presidio output_parse_pii gives the same entity different numbered tokens across messages in one request, corrupting unmasking

  • State: open
  • Author: @tuantranf
  • Labels: bug, proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

With `output_parse_pii: true` and a `/chat/completions` request that has more than one message, the Presidio guardrail numbers PII placeholders per message instead of per request. The sequence counter restarts at 1 for every message, so the same entity value gets a different token in different messages, and unrelated values collide onto the same token

The placeholder map used for unmasking lives in `request_data["metadata"]["pii_tokens"]` and is keyed by the token string. Every message is anonymized as its own `check_pii` task sharing that one dict, so when two messages both produce `<PERSON_1>` for different people, the later write overwrites the earlier one (last write wins). On the response side the unmasker then restores the wrong original value, and because the messages run concurrently via `asyncio.gather` the winner is nondeterministic from request to request

Expected behavior: within a single request each unique entity value maps to one stable token across all messages, and the token to value map stays one-to-one so unmasking always r…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (3 events)
#0 of 0 · 31d18h46m53s ago — entered · #import:https:::github.com:berriai:litellm post #2293
The left issue requires tracing shared request state across concurrent async guardrail execution, redesigning token allocation and reverse mapping invariants, and adding regression coverage for ordering and collisions. The right issue is a narrowly scoped catalog-data update with comparatively low implementation risk.
Issue #34399 is harder because it affects router-wide retry and failover behavior across deployment groups, requiring careful handling of shared upstream limits, health state, timing, and regression coverage. Issue #31959 is more localized to request-scoped token allocation and synchronization in the Presidio guardrail flow.
#0 of 0 · 31d18h18m46s ago — current · #import:https:::github.com:berriai:litellm post #2748
31959 requires coordinating shared state, concurrency behavior, token uniqueness, and regression coverage across guardrail processing and unmasking. 33075 is comparatively localized adapter mapping with focused provider-translation tests.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search