7 views
-/https://github.com/berriai/litellm/issues/32106
GitHub · issue

#32106 [Bug] DB-backed Router rebuild omits cache_responses when store_model_in_db=True — Redis response cache never hits

  • State: open
  • Author: @phakkhaphong
  • Labels: llm translation

### Describe the bug

When LiteLLM Proxy runs with **`store_model_in_db: true`** and **Redis response caching** enabled, identical `/chat/completions` requests do **not** hit the cache after the router is (re)built from Postgres deployments.

The static router init path correctly passes `cache_responses=litellm.cache is not None`, but the **DB router rebuild path** creates `litellm.Router(...)` without `cache_responses`, leaving `router.cache_responses=False` even when `litellm.cache` is initialized.

`switch_on_llm_response_caching()` runs during cache settings init, but if the router is recreated later (model sync / deployment updates), caching is not re-enabled on the new Router instance.

### Affected versions

- Reproduced on **`v1.89.4`** and **`v1.90.3`** (upstream, unpatched) - Docker image: `ghcr.io/berriai/litellm:v1.90.3`

### Configuration

```yaml general_settings: store_model_in_db: true

litellm_settings: cache: true cache_params: type: redis host: redis port: 6379 mode: default_on ttl: 3600 supported_call_types: ["completion", "acompletion"] ```

Models and credentials are stored in Postgres (not in static `model_list`).

### Steps to r…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d18h22m19s ago — entered · #import:https:::github.com:berriai:litellm post #2571
The right-side issue is harder because it likely requires tracing and correcting streaming event translation across provider formats, preserving incremental tool-call state, and adding end-to-end regression coverage. The left-side issue appears comparatively localized to router construction and a focused test.
#0 of 0 · 31d17h47m0s ago — current · #import:https:::github.com:berriai:litellm post #3188
The left issue spans provider capability mapping, multimodal request handling, and possible upstream compatibility constraints, requiring broader investigation and integration testing. The right issue appears localized to configuration propagation during router reconstruction, with a more targeted code change and regression test.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search