8 views
-/https://github.com/berriai/litellm/issues/31873
GitHub · issue

#31873 feat(logging): add opt-in session_id/trace_id correlation to JSON log records via contextvars

  • State: open
  • Author: @deepanshululla
  • Labels: potential-duplicate

## Problem

LiteLLM's `JsonFormatter` in `_logging.py` emits structured JSON log records with no request-scoped identifiers. In production with concurrent async requests all writing to the same global logger singletons, log lines are interleaved with no way to group them per request. Operators cannot reconstruct a request's lifecycle from logs alone.

## Proposed fix

Introduce `session_id_var` and `trace_id_var` as `contextvars.ContextVar[str]` in `_logging.py`. Call `set_trace_id` and `set_session_id` in `Logging.__init__()` so the values are set once per request. `JsonFormatter.format()` reads both vars and injects them into the JSON record when non-empty — zero overhead on requests without a session ID, no changes to individual log call sites.

## Files

- `litellm/_logging.py` - `litellm/litellm_core_utils/litellm_logging.py`

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (7 events)
#0 of 0 · 31d19h8m21s ago — entered · #import:https:::github.com:berriai:litellm post #1736
The logging change is harder because it affects request-scoped state across concurrent execution, shared logging infrastructure, formatter behavior, and regression testing. The UI theme change is comparatively localized to presentation styling and theme-state integration.
The right issue is harder because it spans request-context propagation, asynchronous concurrency semantics, formatter behavior, and broad regression testing across logging paths. The left issue appears more localized to streamed cache serialization and reconstruction, with a supplied fix reducing implementation uncertainty.
The left issue is harder because it requires redesigning timing-sensitive streaming/concurrency behavior across iterator, thread-pool, and provider paths while preserving compatibility and validating regressions. The right issue is comparatively localized to request context propagation and structured log formatting.
The left issue is harder because it involves diagnosing and safely changing asynchronous HTTP transport timeout behavior across Azure-specific request paths, with risks around long-running responses, connection handling, streaming, and regression coverage. The right issue is comparatively localized to request context propagation and JSON formatting, with a smaller implementation surface.
The right-side change is harder because it affects shared asynchronous logging state, request lifecycle integration, backward compatibility, and concurrency-sensitive tests; the left-side change is a localized configuration-plumbing fix.
#35528 is harder because it changes failure semantics across asynchronous rate-limiting, Redis/cache integrations, exception classification, and enforcement tests, with significant risk of either bypassing limits or breaking intentional no-configuration and HTTP error paths. #31873 is comparatively contained to context propagation and JSON formatting, with limited call-site impact and straightforward opt-in tests.
#0 of 0 · 31d17h53m5s ago — current · #import:https:::github.com:berriai:litellm post #2977
The right issue is harder because it changes request-scoped state and shared logging behavior across concurrent execution paths, requiring careful compatibility validation and regression testing. The left issue is a narrowly scoped metadata/configuration update with limited code risk.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search