14 views
-/https://github.com/berriai/litellm/issues/35109
GitHub · issue

#35109 [Feature]: Native Multi-Tenancy & Tenant Isolation for Vector Storage / RAG Endpoints

  • State: open
  • Author: @aaakinkunmi
  • Labels: enhancement, proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### The Feature

## 💡 Proposed Solution

We propose introducing **Vector Multi-Tenancy Configuration** in `config.yaml` and header-based tenant routing inside LiteLLM Proxy.

### 1. Header-Based Tenant Identification Allow LiteLLM Proxy to extract tenant identity from configurable incoming request headers or JWT claims: * `x-tenant-id` / `x-client-id` * Bearer token claims (`tenant_id`)

### 2. Tenant Vector Isolation Modes Support two primary industry-standard multi-tenancy models:

#### Mode A: Index / Collection / Corpus Name Prefixing (Hard Isolation) Automatically partition vector stores and cloud RAG engines by prefixing index, collection, or corpus names with the resolved `tenant_id`: ``` Target Index/Corpus = {tenant_prefix}_{requested_index} Examples: • OpenSearch / PGVector / Qdrant: tenant_52149_clinical_docs_index • Azure AI Search: tenant-52149-clinical-docs-index • Vertex AI RAG Engine: projects/{project}/locations/{region}/ragCorpora/tenant_52149_{corpus_id} ```

#### Mode B: Automatic Metadata Filter Injection (Soft Isolation / Shared Index)…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (10 events)
#0 of 0 · 31d18h52m0s ago — entered · #import:https:::github.com:berriai:litellm post #1999
The left issue is substantially harder because it spans proxy configuration, identity resolution, multiple vector backends, isolation semantics, compatibility, and extensive security and integration testing. The right issue is a narrower concurrency-control fix centered on atomic admission and spend accounting.
Issue #35109 is harder because it requires cross-cutting proxy, authentication, configuration, vector-provider, isolation, and security work, while #27591 is a localized environment-value parsing fix.
Issue 35109 is substantially harder because it requires cross-cutting proxy configuration, identity resolution, security-sensitive isolation behavior, and coordinated support across multiple vector and RAG backends. Issue 23544 is comparatively localized to MCP request flow, caching, and validation behavior.
The right-hand issue has substantially greater scope and risk: it requires cross-cutting proxy configuration, identity handling, isolation guarantees, and compatibility across multiple storage integrations. The left-hand issue is a localized provider request-path defect with a comparatively small fix and test surface.
Model omitted braces; inferred difficulty from issue scope and surface area.
Cross-cutting tenant resolution, authorization, isolation guarantees, and provider-specific request rewriting create greater integration and security risk than assembling a consistent export pipeline over existing proxy resources.
31595 entails a new adaptive routing lifecycle spanning request analysis, persistent training-data workflows, asynchronous trainer coordination, dynamic model updates, fallback behavior, and dashboard operations. 35109 has substantial cross-provider and security scope, but its core behavior is comparatively configuration and request-routing isolation. The former therefore carries greater architectural and integration risk.
The left demands cross-cutting runtime architecture, provider-specific behavior, security isolation, configuration, and extensive testing. The right is broad and legally sensitive but primarily involves repository restructuring and policy coordination rather than comparable runtime complexity.
Multi-tenant isolation spans authentication, request routing, configuration, and multiple storage/provider integrations, creating substantially broader security and compatibility risks. The validation work is also cross-cutting, but is more bounded as specification-driven test and tooling infrastructure.
#0 of 0 · 31d17h20m7s ago — current · #import:https:::github.com:berriai:litellm post #3528
The left requires a broad cross-language platform rewrite with extensive compatibility, performance validation, and staged migration risk, while the right is a narrower proxy and storage-isolation feature with bounded integration work.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search