6 views
-/https://github.com/berriai/litellm/issues/25748
GitHub · issue

#25748 [Bug]: Vertex AI Anthropic streaming broken when custom `api_base` is provided via `/v1/messages`

  • State: open
  • Author: @micahjsmith
  • Labels: bug, proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

Streaming requests to Vertex AI Anthropic models (e.g. `claude-sonnet-4-5@20250929`) via the `/v1/messages` endpoint fail with 404 when a custom `api_base` is configured in `litellm_params`. The required `:streamRawPredict` suffix is not appended to the URL.

The proxy logs an error ``` HTTPStatusError("Client error '404 Not Found' for url 'https://aiplatform.us.rep.googleapis.com/v1/projects/my-project/locations/us/publishers/anthropic/models/claude-sonnet-4-5@20250929'\nFor more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/404") ```

## Root cause

Commit `cdb4917185` (PR #21658) introduced a regression in `litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/experimental_pass_through/transformation.py`.

The fix moved the `get_complete_vertex_url()` call behind an `if api_base is None:` guard. This means when a user provides a custom `api_base` in their litellm config, `get_complete_vertex_url()` is never called and the required endpoint suffix (`:streamRawPredict`) is never appended.

Before the commit: `…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d18h56m1s ago — entered · #import:https:::github.com:berriai:litellm post #1923
The right issue is harder because it spans configuration persistence, runtime loading, and guardrail behavior across multiple execution paths, with compatibility and regression-testing risk. The left issue appears localized to endpoint construction and targeted coverage.
#0 of 0 · 31d18h34m35s ago — current · #import:https:::github.com:berriai:litellm post #2281
The left task has broader cross-provider request-normalization implications, involving interception behavior, tool-selection semantics, and compatibility testing. The right task is comparatively localized to endpoint construction and regression coverage.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search