6 views
-/https://github.com/berriai/litellm/issues/30208
GitHub · issue

#30208 [Feature]: Support enabling fake streaming generally for all providers, or at least for OpenAI

  • State: open
  • Author: @segevfiner
  • Labels: enhancement, proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### The Feature

Some OpenAI compatible providers are poorly implemented and break when you try streaming. LiteLLM has internal code to handle faking streaming internally, but I couldn't find a way to enable it with just the plain OpenAI provider and an arbitrary model.

### Motivation, pitch

Being able to use the LiteLLM proxy with bad AI gateways that don't implement streaming properly

### What part of LiteLLM is this about?

Proxy

### LiteLLM is hiring a founding backend engineer, are you interested in joining us and shipping to all our users?

No

### Twitter / LinkedIn details

_No response_

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (3 events)
#0 of 0 · 31d19h9m54s ago — entered · #import:https:::github.com:berriai:litellm post #1714
The left task has broader cross-provider behavior, proxy integration, edge cases, and compatibility testing, while the right task appears localized to parameter propagation with focused regression coverage.
Issue 30208 requires broader cross-provider streaming behavior, configuration/API design, compatibility handling, and extensive testing, making it substantially riskier than the targeted dashboard authorization/logging adjustment in issue 30442.
#0 of 0 · 31d18h38m31s ago — current · #import:https:::github.com:berriai:litellm post #2229
The right item has broader cross-provider behavior, compatibility, and regression-testing scope, while the left is a more localized credential-to-request plumbing change with focused integration testing.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search