8 views
-/https://github.com/berriai/litellm/issues/26250
GitHub · issue

#26250 [Feature]: Add aclose() to ResponsesAPIStreamingIterator to prevent upstream connection leak on error

  • State: open
  • Author: @thomasfloqs
  • Labels: enhancement, SDK

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### The Feature

Add an `aclose()` coroutine to `litellm.responses.streaming_iterator.BaseResponsesAPIStreamingIterator` (and by extension `ResponsesAPIStreamingIterator`) so that consumers can deterministically release the underlying `httpx.Response` when the stream is abandoned mid-iteration — matching the pattern introduced for chat-completions streaming in [PR #21213](https://github.com/BerriAI/litellm/pull/21213)).

Proposed implementation (mirrors `CustomStreamWrapper.aclose` from #21213):

```python class BaseResponsesAPIStreamingIterator: async def aclose(self) -> None: response = getattr(self, "response", None) if response is None: return with anyio.CancelScope(shield=True): try: await response.aclose() except BaseException as e: verbose_logger.debug( "ResponsesAPIStreamingIterator.aclose: error closing response: %s", e ) finally: self.finished = True ```

### Motivation, pitch

PR [#21213](https:/…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d18h56m19s ago — entered · #import:https:::github.com:berriai:litellm post #1916
Issue #34378 is harder because it spans configuration schema handling, router registration, persistence into spend metadata, and tiered cost-calculation behavior, with broader compatibility and regression-testing risk. Issue #26250 is comparatively localized to iterator lifecycle cleanup and accompanying async tests.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search