7 views
-/https://github.com/berriai/litellm/issues/32068
GitHub · issue

#32068 [Bug]: reasoning_content lost on streaming cache hit; FIX included

  • State: open
  • Author: @FuxMak
  • Labels: bug, proxy

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

With LiteLLM Proxy response caching (Redis) enabled and a reasoning model behind it:

- Cache MISS (streamed): client receives delta.reasoning_content chunks as expected - Cache entry in Redis: contains reasoning_content in the stored message - Cache HIT (streamed): the response contains NO reasoning_content. Clients silently lose the reasoning on every cache hit

Helmfile release:

``` - name: litellm namespace: litellm chart: berriai/litellm-helm version: 1.89.1 ```

Helm values.yaml (shortened):

``` image: # Official LiteLLM image with database support repository: ghcr.io/berriai/litellm-non_root tag: "1.89.1"

proxy_config: litellm_settings: request_timeout: 600 json_logs: true

cache: true cache_params: ttl: 172800 # 2 days type: redis host: os.environ/REDIS_HOST port: os.environ/REDIS_PORT password: os.environ/REDIS_PASSWORD ```

### Steps to Reproduce

1. Send streamed request twice:

``` curl -k -v https://LITELLM_HOST/v1/chat/completions \ -H "Authorization: Bearer…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (3 events)
#0 of 0 · 31d19h2m43s ago — entered · #import:https:::github.com:berriai:litellm post #1824
The right issue is harder because it spans request-context propagation, asynchronous concurrency semantics, formatter behavior, and broad regression testing across logging paths. The left issue appears more localized to streamed cache serialization and reconstruction, with a supplied fix reducing implementation uncertainty.
The right issue spans proxy behavior, cache serialization/deserialization, streaming replay, and compatibility regression testing, creating broader integration risk. The left issue is comparatively localized to provider capability metadata and response-normalization logic.
#0 of 0 · 31d18h8m9s ago — current · #import:https:::github.com:berriai:litellm post #2721
#32068 requires cross-layer work across proxy streaming, cache serialization/deserialization, replay semantics, and regression coverage, while #32900 is comparatively localized to wizard/provider configuration and validation.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search