6 views
-/https://github.com/berriai/litellm/issues/29187
GitHub Β· issue

#29187 Bedrock passthrough requests appear to hang then fail with "Internal server error" when Anthropic is overloaded β€” overloaded error is never surfaced to the client (e.g. Claude Code)

  • State: open
  • Author: @lukaso
  • Labels: llm translation, claude code

When using the Bedrock passthrough endpoint (`/bedrock/model/.../invoke-with-response-stream`) and Anthropic returns an overloaded error mid-stream, the LiteLLM gateway logs a full traceback as an unhandled asyncio task exception, and the client sees a generic "Internal server error" with no indication the request should be retried.

This leads to users believing there is something wrong with LiteLLM rather than seeing that the bedrock model is overloaded.

**Note** analysis done with AI.

**What is happening**

Bedrock streaming always opens with HTTP 200. If Anthropic becomes overloaded mid-stream, the error arrives inside the binary event stream β€” not as an HTTP non-200. After the stream closes, LiteLLM schedules a background task (`asyncio.create_task`) to flush buffered chunks for spend tracking. That task hits the overloaded error event in the buffered bytes and raises `AnthropicError('Overloaded')`. Because there is no `try/except` in `async_flush_passthrough_collected_chunks`, Python drops it as an unhandled task exception.

A secondary issue: `chunk_parser` hardcodes `status_code=500` for all in-stream error events regardless of `error.type` β€” so even if the exception prop…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 Β· 31d18h53m12s ago β€” entered Β· #import:https:::github.com:berriai:litellm post #1968
The harder task spans asynchronous streaming, provider-specific error propagation, background task safety, and status semantics across multiple layers, with higher regression risk. The easier task is a localized routing change with focused endpoint tests.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search