8 views
-/https://github.com/berriai/litellm/issues/34719
GitHub · issue

#34719 [Bug]: RuntimeError: dictionary changed size during iteration in success_handler/failure_handler under concurrent logging

  • State: open
  • Author: @yryzhan-vitech
  • Labels: bug, proxy

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

The synchronous `Logging.success_handler` and `Logging.failure_handler` (`litellm/litellm_core_utils/litellm_logging.py`) iterate the live `self.model_call_details` dict directly (`for k, v in self.model_call_details.items():`) while building the kwargs passed to sync callbacks (langfuse, logfire, greenscale, athina, traceloop).

These sync handlers run on a **worker thread** (dispatched via `executor.submit` / `threading.Thread`), while `async_success_handler` / `async_failure_handler` run on the **event loop** and insert **new keys** into the *same* `self.model_call_details` object (e.g. `async_complete_streaming_response`, `response_cost`, `standard_logging_object`, and — on failures — `log_event_type`, `exception`, `traceback_exception`, `end_time`). When the async path inserts a key while the sync loop is mid-iteration, Python raises `RuntimeError: dictionary changed size during iteration`, which aborts that callback's logging for the request (the exception is swallowed by the handler's non-blocking try/except, so the log is silently dropp…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (3 events)
#0 of 0 · 31d18h17m42s ago — entered · #import:https:::github.com:berriai:litellm post #2654
34719 is harder because it requires safely resolving cross-thread mutation and callback ordering, with concurrency-focused regression coverage and potential compatibility risks in logging. 34301 is comparatively localized to model capability metadata and parameter-validation tests.
Concurrent state mutation requires careful synchronization, compatibility review, and reliable race-condition regression tests across multiple execution paths, making it substantially riskier than exposing or clarifying a single configuration option in the UI.
#0 of 0 · 31d18h8m48s ago — current · #import:https:::github.com:berriai:litellm post #2807
The left issue spans identifier serialization, asynchronous endpoint state, deployment resolution, credential fallback, and compatibility with existing encoded IDs, requiring coordinated code and regression tests. The right issue is comparatively localized: make handler iteration operate on a stable snapshot and validate concurrent callback behavior. The left therefore carries substantially greater integration and regression risk.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search