8 views
-/https://github.com/berriai/litellm/issues/32324
GitHub · issue

#32324 [Bug]: Valkey semantic cache failed as two self._get_async_embedding() call in `valkey_semantic_cache.py` forward `**kwargs` instead of `metadata`

  • State: open
  • Author: @D0wn10ad
  • Labels: bug, proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

Disclosure: found this while troubleshoot using AI.

related to #31610, in caching/valkey_semantic_cache.py 2 calls of ``` embedding = await self._get_async_embedding(prompt, **kwargs) ``` in lines 282 and 301 are not sending expected `metadata` but **kwargs

supposedly it should be something similar to

``` embedding = await self._get_async_embedding( prompt, metadata=kwargs.get("metadata"), ) ```

workaround applied for caching to work: patched caching/caching.py like #31610 patched caching/valkey_semantic_cache.py (attached) [valkey_semantic_cache.py](https://github.com/user-attachments/files/29734382/valkey_semantic_cache.py)

### Steps to Reproduce

1. docker pull litellm:v1.92.0-rc1 2. setup valkey semantic cache in config.yaml with proper embedding 3. monkey patch the cache not shown issue on litellm/_logging.py #32318 [_logging.py](https://github.com/user-attachments/files/29734028/_logging.py) 4. send 2 indentical chat_completion via curl 5. observe the log for errors

### Relevant log output

```shell 04:18:06 - LiteLLM:DE…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d17h52m26s ago — entered · #import:https:::github.com:berriai:litellm post #3095
Issue 34614 is harder because it requires resolving a dependency/API compatibility problem across multiple Redis and Valkey code paths, including cache operations and budget accounting, with broader regression testing. Issue 32324 is a localized argument-plumbing correction with a narrower behavioral scope.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search