8 views
-/https://github.com/berriai/litellm/issues/34357
GitHub · issue

#34357 [Feature]:Add OpenInfer Provider

  • State: open
  • Author: @cbuffington-cell
  • Labels: enhancement, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### The Feature

OpenInfer would like to contribute a native LiteLLM provider integration. OpenInfer provides a production OpenAI-compatible inference API focused on cost-efficient, high-performance inference across heterogeneous compute infrastructure.

Current capabilities include: OpenAI-compatible /v1/chat/completions /v1/models endpoint Bearer authentication Streaming (SSE) Token usage reporting Enterprise deployments and dedicated infrastructure

A native provider would allow LiteLLM users to configure OpenInfer without relying on the generic OpenAI-compatible provider, providing a simpler setup and better user experience.

API Base https://api.openinfer.ai/v1

Website https://openinfer.io We're happy to contribute the provider implementation and work with the LiteLLM team on integration.

### Motivation, pitch

OpenInfer provides a production OpenAI-compatible inference API focused on delivering high-performance, cost-efficient AI inference across heterogeneous compute infrastructure. A native LiteLLM provider would allow developers to use OpenInfer as a fi…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d19h9m55s ago — entered · #import:https:::github.com:berriai:litellm post #1902
Adding a native provider requires broader integration across LiteLLM’s provider abstractions, request/response translation, authentication, streaming, usage accounting, configuration, documentation, and compatibility testing. The cache defect is narrower: tracing mutation paths, adding consistent invalidation, and covering the affected proxy workflows with regression tests.
#0 of 0 · 31d19h52s ago — current · #import:https:::github.com:berriai:litellm post #2059
A new provider integration carries broader scope and integration risk: provider routing, authentication, request/response translation, streaming and usage handling, tests, and documentation. The other issue is comparatively localized to diagnosing and correcting one non-streaming response path.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search