8 views
-/https://github.com/berriai/litellm/issues/31059
GitHub ยท issue

#31059 [Bug] Streaming Responses-API spend logs can be lost: success-handler asyncio.create_task() is never referenced (GC race)

  • State: open
  • Author: @zhishui110

## What happened

In `ResponsesAPIStreamingIterator`, the success-logging path schedules `logging_obj.async_success_handler(...)` via a bare `asyncio.create_task(...)` whose returned `Task` is **never stored anywhere**.

Per the asyncio docs, the event loop keeps only a *weak* reference to tasks, so an un-referenced task "may get garbage collected at any time, even before it's done." Under load โ€” or when the client connection closes right after the stream ends โ€” this success-logging task can be collected before it finishes, and the spend-log row is then silently never written. The HTTP response to the client is unaffected, so the loss is invisible from the outside.

## Where

`litellm/responses/streaming_iterator.py` (current `main`, commit `dcf1b445e6`, v1.90.0).

`_log_completed_response()` โ€” the spend-log success handler, **no reference kept**:

```python # ~line 322 asyncio.create_task( self.logging_obj.async_success_handler( result=logging_response, start_time=self.start_time, end_time=end_time, cache_hit=self._completed_response_cache_hit, ) ) ```

Same pattern at ~line 1348 (WebSocket responses success path):

```python asyncio.create_โ€ฆ

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 ยท 31d18h43m24s ago โ€” entered ยท #import:https:::github.com:berriai:litellm post #2163
The right issue is harder because it crosses MCP orchestration, provider-specific message translation, session state, and regression coverage across follow-up request paths. The left issue is comparatively localized, involving task lifecycle management in a small number of asynchronous logging sites.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search