17 views
-/https://github.com/berriai/litellm/issues/26376
GitHub · issue

#26376 [Bug]: langfuse_otel callback drops OTEL spans under asyncio.gather() concurrency

  • State: open
  • Author: @shuttlesworthNEO

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

When multiple `litellm.acompletion()` calls run concurrently via `asyncio.gather()`, the `langfuse_otel` callback consistently drops ~1 in 3 OTEL spans. The spans are silently lost — no errors, no warnings. This means Langfuse traces are incomplete and cost tracking is unreliable for batch LLM operations.

This is related to #25978 (which covers the teardown/drain variant), but this is the **concurrent execution** variant — spans are dropped while the event loop is still alive, not during shutdown.

### Root cause

LiteLLM dispatches success callbacks via `asyncio.create_task()` through `GLOBAL_LOGGING_WORKER`. When multiple `acompletion()` calls complete near-simultaneously under `asyncio.gather()`, the callback tasks that create OTEL spans (`_handle_success` → `_start_primary_span`) interfere with each other. The result: some spans are never created or are lost before reaching the `BatchSpanProcessor`.

### Steps to Reproduce

```python import asyncio import litellm from langfuse import observe, get_client

litellm.success_callback = ["langfu…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d17h57m0s ago — entered · #import:https:::github.com:berriai:litellm post #2925
The right issue is substantially harder because it requires coordinated security response, forensic investigation, dependency and release-pipeline remediation, validation, and potentially broad stakeholder communication. The left issue is a contained concurrency and observability defect with a narrower code and test scope.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search