8 views
-/https://github.com/berriai/litellm/issues/32353
GitHub ยท issue

#32353 [Bug]: ReDoS in secret_redaction.redact_string() โ€” catastrophic regex backtracking on large exception strings blocks the event loop for minutes, kills liveness probes, and crash-loops the proxy

  • State: open
  • Author: @etatros
  • Labels: bug, proxy

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

## What happened?

All replicas of our LiteLLM proxy (Docker Swarm, 4 replicas) entered a continuous crash-loop: every container was killed with `task: non-zero exit (137): dockerexec: unhealthy container` every 4โ€“8 minutes, around the clock, for 26+ hours.

Root cause (confirmed with **five py-spy dumps across four different containers**): whenever a request with a **large payload fails** (provider 4xx/5xx), the exception string โ€” which embeds the original request/response body โ€” is passed through `litellm/litellm_core_utils/secret_redaction.py::redact_string()`. One of the compiled patterns exhibits **catastrophic backtracking** on large, brace-heavy input. The `re.sub()` call runs **synchronously on the asyncio event loop**, holding the GIL for **minutes**, during which:

- `/health/liveliness` cannot respond โ†’ orchestrator healthcheck fails โ†’ container is SIGKILLed (exit 137) - all in-flight async I/O stalls, producing a burst of *misleading* secondary errors when the loop resumes: Redis read timeouts, `httpx.ConnectTimeout` to the **local*โ€ฆ

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 ยท 31d18h39m36s ago โ€” entered ยท #import:https:::github.com:berriai:litellm post #2218
Python 3.14 support requires coordinated Rust dependency, ABI, packaging metadata, CI, and wheel-release changes with cross-version compatibility risk; the regex issue is more localized to redaction logic and regression testing.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search