8 views
-/https://github.com/berriai/litellm/issues/31619
GitHub ¡ issue

#31619 [Bug]: LangGraph streaming leaks the httpx connection — LangGraphSSEStreamIterator has no close()/aclose()

  • State: open
  • Author: @Vedant-Agarwal

## What happened?

LangGraph streaming leaks the underlying httpx connection whenever a stream is cleaned up (client disconnect, mid-stream error, or any early termination), because `LangGraphSSEStreamIterator` doesn't implement `close()` / `aclose()`.

`CustomStreamWrapper.aclose()` releases the underlying stream via: ```python if hasattr(stream_to_close, "aclose"): await stream_to_close.aclose() elif hasattr(stream_to_close, "close"): ... ``` `LangGraphSSEStreamIterator` (`litellm/llms/langgraph/chat/sse_iterator.py`) holds `self.response` (an httpx streaming response = a checked-out connection) but defines **neither** `aclose` **nor** `close` (nor `__del__`). So the cleanup contract silently no-ops and the connection is never returned to the pool. Additionally, `__next__`/`__anext__` swallow mid-stream exceptions and raise `Stop(Async)Iteration` without closing `self.response`.

This is the same failure class as #30271 (connection not released on early stream break). Other providers' stream iterators expose `aclose` (e.g. bare `response.aiter_lines()` generators), so cleanup propagates correctly — LangGraph breaks that chain.

## Repro (found via code review) ```python f…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d18h3m39s ago — entered · #import:https:::github.com:berriai:litellm post #3011
34819 has broader proxy-wide streaming behavior implications, requiring coordination across response generation, async timing, deployment timeout expectations, and compatibility testing. 31619 is comparatively localized to implementing and validating resource cleanup on one iterator.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search