6 views
-/https://github.com/berriai/litellm/issues/31862
GitHub · issue

#31862 bug(spend-tracking): get_cache_key called for call types not in supported_call_types, polluting spend log cache_key column

  • State: open
  • Author: @deepanshululla
  • Labels: llm translation

## Description

`get_logging_payload` in `litellm/proxy/spend_tracking/spend_tracking_utils.py` calls `litellm.cache.get_cache_key(**kwargs)` on every request whenever any cache backend is configured, regardless of whether the current call type is actually cacheable. The guard (OSS 1.91.0, lines 370-373) is a single boolean: `if litellm.cache is not None`. This is insufficient.

A cache backend can be configured with a `supported_call_types` list that excludes the current call type. A common example: Redis configured for routing state (cooldown tracking, model group keys) with `supported_call_types: []`. In that case, the cache singleton is not `None`, so `get_cache_key` runs for 100% of requests even though nothing is ever cached.

`get_cache_key` (defined in `litellm/caching/caching.py`, lines 329-379) is not a trivial lookup; it iterates the entire `kwargs` dict, concatenates API parameters, computes a SHA-256 hash, resolves a namespace prefix, and mutates `kwargs['litellm_params']['preset_cache_key']`. All of this work is wasted when the call type is not in `supported_call_types`.

The observable result is unnecessary CPU on the logging hot-path and a `cache_key` column in `Lit…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d19h4m47s ago — entered · #import:https:::github.com:berriai:litellm post #1789
The right issue is harder because it requires a new cloud-identity integration, dependency and configuration handling, secure token lifecycle coordination, and broad integration testing. The left issue is comparatively localized to request-path guarding and regression tests.
#0 of 0 · 31d19h4m25s ago — current · #import:https:::github.com:berriai:litellm post #1796
The right issue requires modifying a hot-path caching condition, handling call-type configuration safely, and adding regression coverage; the left is a narrowly scoped model metadata update.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search