6 views
-/https://github.com/berriai/litellm/issues/35599
GitHub · issue

#35599 /v1/rag/query endpoint does not resolve vector store credentials from vector_store_registry config

  • State: open
  • Author: @aaakinkunmi
  • Labels: llm translation

### Describe the Bug

The `/v1/rag/query` endpoint's internal search step calls `litellm.vector_stores.asearch()` **directly**, bypassing the `_update_request_data_with_litellm_managed_vector_store_registry()` function that the standalone `/v1/vector_stores/{id}/search` endpoint uses to resolve credentials from the `vector_store_registry` YAML config.

This means RAG query always defaults `custom_llm_provider` to `"openai"` and attempts to connect to `api.openai.com` for the search step — even when the vector store is configured as Azure AI Search (or any non-OpenAI provider) in the registry.

### To Reproduce

1. Configure a `vector_store_registry` with Azure AI Search in `litellm_config.yaml`:

```yaml vector_store_registry: - vector_store_name: "azure-ai-search" litellm_params: custom_llm_provider: "azure_ai" litellm_embedding_model: "azure/text-embedding-3-large" api_key: "os.environ/AZURE_SEARCH_API_KEY" api_base: "os.environ/AZURE_SEARCH_ENDPOINT" api_version: "2024-07-01" vector_store_id: "my-azure-index" ```

2. Verify standalone search works: ```bash # ✅ This works — resolves credentials from registry curl -X POST "http://localhost…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d18h7m57s ago — entered · #import:https:::github.com:berriai:litellm post #2723
The right issue is harder because it requires safely changing persistence-layer behavior and session reconstruction under large-scale workloads, with greater compatibility, performance, and regression risk. The left issue is a narrower request-path integration fix with a more localized validation surface.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search