9 views
-/https://github.com/berriai/litellm/issues/34872
GitHub · issue

#34872 [Bug]: Gemini context caching with custom api_base uses invalid cache URL and auth header

  • State: open
  • Author: @paulo037
  • Labels: llm translation

## Check for existing issues

- [x] I searched the existing issues and did not find a duplicate covering both the model-scoped `cachedContents` URL and malformed authentication header.

Related:

- #23846 - #23928

## What happened?

Gemini explicit context caching fails when the SDK uses a custom `api_base` implementing the standard Gemini REST API.

When a prompt contains an Anthropic-style `cache_control` marker and exceeds Gemini's minimum cache size, LiteLLM makes a secondary request to the named-cache API. For a custom Gemini base URL, LiteLLM currently:

1. sends the request to: ```text {api_base}/models/{model}:cachedContents ``` 2. converts the Gemini API-key header dictionary into a malformed Bearer value: ```text Authorization: Bearer {'x-goog-api-key': '<redacted>'} ```

The standard Gemini named-cache API uses the collection endpoint:

```text {api_base}/cachedContents ```

The model belongs in the create request body, for example:

```json {"model": "models/gemini-2.5-flash"} ```

This affects standards-compatible Gemini proxies such as Bifrost. The normal `generateContent` request succeeds through the same custom `api_base`; only the explicit context-…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (3 events)
#0 of 0 · 31d18h37m21s ago — entered · #import:https:::github.com:berriai:litellm post #2341
The left issue is harder because it spans request URL construction, authentication translation, provider-specific payload semantics, custom endpoint compatibility, and regression testing across the caching flow. The right issue is comparatively localized to model capability metadata and existing parameter-validation behavior.
The right issue is harder because it requires correcting provider-specific request construction, authentication handling, custom endpoint behavior, and regression coverage across caching flows. The left issue is broader visually but is primarily a frontend consistency and component-styling effort with lower integration risk.
#0 of 0 · 31d18h31m29s ago — current · #import:https:::github.com:berriai:litellm post #2420
The left issue is harder because it likely requires tracing authorization, model-list retrieval, and proxy UI behavior across multiple layers, with regression and role-specific testing. The right issue is comparatively localized to Gemini request construction and authentication handling, with a narrower provider-specific test surface.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search