8 views
-/https://github.com/berriai/litellm/issues/23766
GitHub Ā· issue

#23766 [Feature]: Support for mREP endpoint for vertex AI

  • State: open
  • Author: @Bharat23
  • Labels: enhancement, proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### The Feature

Vertex AI Multi-Region Endpoints (mREPs) generate an incorrect URL when using `api_base` with `vertex_ai/` models.

Google recently introduced for Vertex AI, which guarantee data residency within a jurisdiction. The mREP base URL format is `https://aiplatform.us.rep.googleapis.com` (note: no region prefix like `us-central1-`).

**Expected URL format** (per Google's docs): ``` https://aiplatform.us.rep.googleapis.com/v1/projects/{PROJECT}/locations/us/publishers/google/models/{MODEL}:generateContent ```

**Actual URL generated by LiteLLM:** ``` https://aiplatform.us.rep.googleapis.com:generateContent ```

### Relevant code

In `litellm/llms/vertex_ai/vertex_llm_base.py`, the `_check_custom_proxy` method handles custom `api_base` for Vertex AI. When `api_base` is set and `use_psc_endpoint_format` is `False` (the default), it falls through to:

```python url = "{}:{}".format(api_base, endpoint) ```

This strips the entire `/v1/projects/.../locations/.../publishers/google/models/...` path and just appends `:{endpoint}` to the base URL.

The `use_psc_e…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 Ā· 31d18h39m29s ago — entered Ā· #import:https:::github.com:berriai:litellm post #2210
The right issue is harder because it spans API response design, configuration and pricing resolution, access filtering, backward compatibility, and testing across proxy paths. The left issue is comparatively localized to provider URL construction and regression coverage.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search