6 views
-/https://github.com/berriai/litellm/issues/30836
GitHub · issue

#30836 [Bug]: /v1/messages ignores timeout/stream_timeout, hard-capped at 600s (peak-time latency kills long streams)

  • State: open
  • Author: @andrey-dubnik
  • Labels: proxy, llm translation

### What happened?

The Anthropic `/v1/messages` passthrough route ignores the configured `timeout` and `stream_timeout`, so every request on this route is hard-capped at the built-in 600s default (`COMPLETION_HTTP_FALLBACK_SECONDS`). Per-deployment `litellm_params.timeout` / `stream_timeout` and `router_settings.timeout` are honored on `/chat/completions` and `/responses`, but silently dropped on `/v1/messages`.

This is increasingly painful in production. During peak hours the popular models (Claude on Bedrock and the native Anthropic API) slow down under load, and long-running streaming responses develop silent gaps between chunks. The gap is what matters here: with the aiohttp transport the read timeout maps to `ClientTimeout(sock_read=...)`, so a single inter-chunk gap longer than 600s kills the stream even though the connection is healthy and the model is still working. The request dies with `aiohttp.client_exceptions.SocketTimeoutError` surfaced as `httpx.ReadTimeout`.

Because the configured timeout is ignored, operators have no way to raise the ceiling for these slow-but-valid streams, and no way to lower it for fail-fast behavior either. Setting `timeout` / `stream_timeou…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d18h19m23s ago — entered · #import:https:::github.com:berriai:litellm post #2519
#30836 requires tracing configuration through multiple proxy and transport layers, preserving streaming timeout semantics, and validating regression-prone behavior across deployment types. #30919 has insufficient detail to indicate comparable scope or implementation risk.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search