8 views
-/https://github.com/berriai/litellm/issues/32250
GitHub · issue

#32250 [Bug]: Tuple dict keys in `metadata` crash `spend_tracking_utils.py` and cause unbounded memory growth

  • State: open
  • Author: @LaPetiteSouris
  • Labels: bug, proxy

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

## Summary

When the OpenTelemetry integration is enabled, `_otel_internal["spans_logged"]` stores **tuple keys** inside the request `metadata`. Because `litellm_pre_call_utils.py` shallow-copies `metadata` by reference into `proxy_server_request["body"]`, those tuple keys propagate into the spend-tracking path. `spend_tracking_utils.py` then calls `json.dumps` on the request body, which **crashes on every single request** with a `TypeError`.

The practical impact is catastrophic: every request fails to record its spend-log payload, so the in-memory `spend_log_transactions` queue grows without bound, leading to a severe memory leak and eventual OOM kills.

## Environment

- **LiteLLM version:** `1.85.0` - **Python version:** `3.13` - **OpenTelemetry integration:** Enabled (via `otel` callback) - **Proxy mode:** Yes

## Reproduction Steps

### 1. Minimal code trigger

```python import json from litellm.litellm_core_utils.safe_json_dumps import safe_dumps

# This is what opentelemetry.py injects into metadata via _otel_internal metadata = { …

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d18h44m55s ago — entered · #import:https:::github.com:berriai:litellm post #2141
The right issue is harder because it requires coordinated changes across virtual-key configuration, request validation, tag precedence, routing behavior, authentication responses, and compatibility tests. The left issue is serious but comparatively localized to metadata copying/serialization and spend-queue failure handling.
#0 of 0 · 31d18h38s ago — current · #import:https:::github.com:berriai:litellm post #2865
Salt-key rotation requires designing secure lifecycle semantics, configuration and backward compatibility, persistence/distribution behavior, and comprehensive operational testing. The other issue is a comparatively localized data-sanitization and regression-fix task, despite its serious production impact.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search