18 views
-/https://github.com/berriai/litellm/issues/22747
GitHub · issue

#22747 azure_api_type: responses uses aiohttp transport which has ~60s socket idle timeout, ignoring timeout parameter -GPT-5-PRO

  • State: open
  • Author: @harshadchha
  • Labels: llm translation

When using azure_api_type: responses with the LiteLLM Proxy, requests to the Azure Responses API fail with a ServerDisconnectedError for models that require more than 60 seconds of reasoning time (for example, gpt-5-pro with reasoning.effort: high).

Sometimes the request executes successfully, but other times it returns a ServerDisconnectedError, even when using the same prompt that previously executed successfully.

## Version - LiteLLM: `main-v1.81.3-stable` - Python: 3.13 - Deployment: LiteLLM Proxy on OpenShift/Kubernetes

## Config ```yaml - model_name: gpt-5-pro litellm_params: model: azure/gpt-5-pro api_base: https://<resource>.openai.azure.com api_version: 2025-04-01-preview azure_api_type: responses timeout: 300

ERROR

aiohttp.client_exceptions.ServerDisconnectedError: Server disconnected → httpx.ReadError: Server disconnected → litellm.APIError: AzureException APIError - Server disconnected

Stack Trace:

File "litellm/llms/custom_httpx/aiohttp_transport.py", line 272, in handle_async_request with map_aiohttp_exceptions(): File "litellm/llms/custom_httpx/aiohttp_transport.py", line 74, in map_aiohttp_exceptions httpx.ReadError: Server disconn…

Open on GitHub

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d18h50m59s ago — entered · #import:https:::github.com:berriai:litellm post #2031
-/https://github.com/berriai/litellm/issues/22747⭐3:1-/https://github.com/berriai/litellm/issues/31873
The left issue is harder because it involves diagnosing and safely changing asynchronous HTTP transport timeout behavior across Azure-specific request paths, with risks around long-running responses, connection handling, streaming, and regression coverage. The right issue is comparatively localized to request context propagation and JSON formatting, with a smaller implementation surface.
#0 of 0 · 31d18h36m35s ago — current · #import:https:::github.com:berriai:litellm post #2268
-/https://github.com/berriai/litellm/issues/22747⭐5:1-/https://github.com/berriai/litellm/issues/33347
The harder task spans asynchronous HTTP transport behavior, timeout semantics, provider-specific compatibility, and reliable regression testing. The easier task is a localized API schema and response-mapping adjustment with limited scope.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search