7 views
-/https://github.com/berriai/litellm/issues/34767
GitHub · issue

#34767 [Bug]: /v1/responses with stream=true + file_search (emulated, Anthropic) crashes with "'async for' requires an object with __aiter__ method, got ResponsesAPIResponse"

  • State: open
  • Author: @jega-ms
  • Labels: llm translation

### What happened?

On LiteLLM Proxy **v1.91.0**, calling `POST /v1/responses` with `stream: true` and a `file_search` tool pointing at a registered non-OpenAI vector store (`custom_llm_provider: s3_vectors`), against an **Anthropic** model, always returns HTTP 500:

```json {"error":{"message":"'async for' requires an object with __aiter__ method, got ResponsesAPIResponse","type":"None","param":"None","code":"500"}} ```

The response headers show the upstream Anthropic call **succeeded** before the crash (`llm_provider-anthropic-*` rate-limit headers present, `x-litellm-response-cost: 0.005721`, `x-litellm-response-duration-ms: 2406`), so the request is billed but still fails. The emulated file_search flow appears to produce a non-streaming `ResponsesAPIResponse` and hand it to the SSE bridge, which tries to `async for` over it.

- Same request with `stream: false` → no crash. - Same request with `stream: true` but **without** the `file_search` tool → streams normally.

So the crash is specific to emulated file_search + streaming.

### Reproduction

```bash curl -sS -N "$PROXY/v1/responses" \ -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \ -d '{ "mode…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (3 events)
#0 of 0 · 31d19h11m15s ago — entered · #import:https:::github.com:berriai:litellm post #1680
The left issue is harder because it crosses asynchronous streaming control flow, response-type handling, tool emulation, provider translation, and end-to-end regression testing. The right issue is more localized to persistence-model serialization and endpoint response mapping.
Issue #34767 is substantially harder because it requires diagnosing and safely fixing an interaction across response streaming, asynchronous iteration, provider translation, and tool emulation, with regression coverage across multiple execution paths. Issue #32888 appears to be a minimal test-only task with negligible implementation scope.
#0 of 0 · 31d17h41m31s ago — current · #import:https:::github.com:berriai:litellm post #3174
Enabling outbound HTTP/2 safely requires cross-cutting client, transport, dependency, configuration, and compatibility work across many provider paths. The other issue is narrower, centered on correcting an async streaming boundary in one emulated-tool flow.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search