9 views
-/https://github.com/berriai/litellm/issues/26233
GitHub · issue

#26233 [Bug]: Redis env vars ignored when cache_params has any keys - breaks spend tracking with multiple pods

  • State: open
  • Author: @dzbanek717
  • Labels: bug, proxy

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

Found a bug that broke our spend tracking in production. When you set `cache: true` with `cache_params` that has literally any key in it (like `mode: default_off`), the proxy completely ignores Redis environment variables and falls back to in-memory cache instead.

This is a problem because spend tracking between pods relies on Redis to sync counters. Without it, each pod tracks usage independently and the whole thing falls apart.

Took me a while to debug but the issue is in `proxy_server.py` around line 3138:

```python if ( cache_type == "redis" or cache_type == "redis-semantic" ) and len(cache_params.keys()) == 0: cache_host = get_secret("REDIS_HOST", None) # ... ```

The `len(cache_params.keys()) == 0` check means it only loads Redis from env vars when cache_params is completely empty. If you add `mode` or any other key, this condition fails and Redis config from env never gets added.

Then later when the code checks `isinstance(litellm.cache.cache, RedisCache)` to set up spend_counter_cache, it returns False because we got InM…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (3 events)
#0 of 0 · 31d19h13m43s ago — entered · #import:https:::github.com:berriai:litellm post #1648
#26233 is harder because it involves cache configuration precedence, initialization behavior, and distributed spend-accounting correctness across deployment modes, requiring broader regression testing. #24516 is comparatively localized to defensive handling and validation in an observability callback.
30534 requires tracing a regression across cache serialization, TTL handling, asynchronous Redis writes, and circuit-breaker behavior, with compatibility and deployment-specific testing risk. 26233 appears comparatively localized to configuration initialization and can be covered with focused proxy and multi-instance tests.
#0 of 0 · 31d19h1s ago — current · #import:https:::github.com:berriai:litellm post #1865
The right-side work is harder because it crosses configuration precedence, cache initialization, distributed state, and regression testing, while the left-side work is comparatively localized to provider-specific validation and endpoint handling.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search