7 views
-/https://github.com/berriai/litellm/issues/34727
GitHub · issue

#34727 [Docs]: REDIS_URL prod warning is stale; the perf gap it cites is fixed

  • State: open
  • Author: @mangabits

### What happened?

The caching docs still advise against `REDIS_URL` in production on performance grounds:

> we **don't** recommend using REDIS_URL in prod. We've noticed a performance difference between using it vs. redis_host, port, etc.

That guidance was correct when it was written, and it traces back to #3188. The cause was connection churn: the url path discarded the shared pool and opened roughly two new connections per request, which is ruinous over TLS because each one pays a fresh handshake.

Current code pools correctly, and the gap is gone. Measured against a local redis behind a TCP relay injecting a fixed one-way delay, with redis in the request path on every call (response caching with unique prompts, 500 requests, 50 concurrent workers):

| version | mode | req/s | connections opened | |---|---|---|---| | 1.35.17 | url | 540 | 1002 | | 1.35.17 | host/port | 742 | 50 | | current (1.95.0) | url | 650 | 50 | | current (1.95.0) | host/port | 690 | 50 |

Over TLS the old churn cost 83.9 req/s against 462.1 on current code, with 1007 connections against 52. On current code url and host/port are within run to run noise of each other, so the stated reason for the recommen…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d18h12m12s ago — entered · #import:https:::github.com:berriai:litellm post #2744
The right issue requires tracing model registration and provider-specific configuration across persistence, loading, request translation, and streaming behavior, with regression coverage across multiple configuration paths. The left issue is primarily a documentation correction based on already-fixed behavior, requiring minimal engineering work.
#0 of 0 · 31d17h57m9s ago — current · #import:https:::github.com:berriai:litellm post #3008
The left issue requires provider-specific streaming behavior changes, response parsing, compatibility handling, and regression tests. The right issue is primarily a documentation correction with comparatively little implementation risk.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search