7 views
-/https://github.com/berriai/litellm/issues/30362
GitHub · issue

#30362 [Feature]: Support HTTP/2 for outbound requests to upstream LLM providers

  • State: open
  • Author: @KevinZhao
  • Labels: llm translation

### The Feature

LiteLLM does not support **HTTP/2 for outbound requests** to upstream LLM providers.

When LiteLLM acts as a client calling provider APIs (Bedrock, Vertex, Anthropic, OpenAI, etc.), the httpx clients are created **without** `http2=True`:

- `litellm/llms/custom_httpx/http_handler.py:392` — `httpx.AsyncClient(...)` created without `http2=True` - `litellm/llms/custom_httpx/http_handler.py:943` — `httpx.Client(...)` created without `http2=True`

Since httpx defaults to `http2=False`, **all outbound calls use HTTP/1.1**. A repo-wide search for `http2=True` returns zero matches.

Additionally, the default async transport is **aiohttp** (`_should_use_aiohttp_transport()` returns `True` by default), and aiohttp does not support HTTP/2 at all — so even if the httpx flag were set, the default code path would still be HTTP/1.1.

> Note: The `h2` package present in the dependency tree is a transitive dependency of **hypercorn** (used for the *inbound* server side via `--run_hypercorn`, which already documents "supports HTTP/2"). It is not wired into the outbound client.

**Request:** Add opt-in HTTP/2 support for outbound provider requests, e.g. a `litellm.enable_http2` flag …

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d17h41m40s ago — entered · #import:https:::github.com:berriai:litellm post #3174
Enabling outbound HTTP/2 safely requires cross-cutting client, transport, dependency, configuration, and compatibility work across many provider paths. The other issue is narrower, centered on correcting an async streaming boundary in one emulated-tool flow.
#0 of 0 · 31d17h35m57s ago — current · #import:https:::github.com:berriai:litellm post #3270
30460 requires distributed-state debugging, failure-mode analysis, and cross-path correctness testing; 30362 is comparatively bounded transport/configuration work with compatibility testing.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search