18 views
-/https://github.com/berriai/litellm/issues/20064
GitHub · issue

#20064 [Feature]: support model discovery with custom provider

  • State: open
  • Author: @cornmail
  • Labels: enhancement, proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### The Feature

Here is an example of how the desired config.yaml could look:

```yaml model_list: - model_name: my-custom-service/* # Use a wildcard for the custom provider group litellm_params: model: custom_openai/* # The litellm provider identifier api_key: os.environ/MY_CUSTOM_PROVIDER_KEY api_base: "http://my-custom-provider-url.com/v1" # The endpoint for the custom provider

litellm_settings: check_provider_endpoint: true # Enable model discovery ```

With this configuration, a call to the LiteLLM proxy's /v1/models endpoint should trigger a discovery call to http://my-custom-provider-url.com/v1/models and return the result in the expected format.

This enhancement would significantly improve the integration of LiteLLM with the rapidly growing ecosystem of self-hosted and third-party OpenAI-compatible APIs.

Thank you for considering this feature request

### Motivation, pitch

The new Model Discovery feature is an excellent addition for dynamically fetching a list of available models from a provider when using a wildcard in the …

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d17h49m56s ago — entered · #import:https:::github.com:berriai:litellm post #3023
The left task spans provider abstraction, configuration semantics, endpoint discovery, model normalization, and proxy integration, creating broader compatibility and testing risk. The right task is a more localized streaming-path correction with focused regression coverage.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search