8 views
-/https://github.com/berriai/litellm/issues/27213
GitHub · issue

#27213 [Feature]: Custom Transport support for acompletion calls

  • State: open
  • Author: @minherz
  • Labels: enhancement, SDK

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### The Feature

Allow customizing transport layer (HTTP) by passing a custom `httpx` client as a "client" argument. The customization will provide a simplified path to resolve simple tasks as an alternative to a more complex setup of litellm proxy.

### Motivation, pitch

Today in order to make many atomic manipulations such as: * Refresh bearer token * Inject an observability or analytic action before and after inference call * Dynamically change headers

a developer has to configure and deploy a proxy. While the proxy does this work, it is extra burden which rarely justifies the additional complexity and inherited costs. To be able to do these tasks in-process by implementing a standardized middleware using `httpx` package is a common way to solve these problems.

This FR proposes to add it *only* to `acompletion()` calls to narrow the scope of the change. However, if proved useful, it can be extended to other execution paths as well.

### What part of LiteLLM is this about?

SDK (litellm Python package)

### LiteLLM is hiring a founding backend engineer, ar…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d18h29m51s ago — entered · #import:https:::github.com:berriai:litellm post #2372
The right-side change is harder because it crosses authentication, token lifecycle, activity tracking, client behavior, persistence, configuration, and security-sensitive expiry semantics. The left-side change is narrower SDK plumbing confined to an async request path, though it still requires compatibility and lifecycle testing.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search