6 views
-/https://github.com/berriai/litellm/issues/17696
GitHub · issue

#17696 [Bug]: Gemini request fails when cache size is too small

  • State: open
  • Author: @ahabhgk
  • Labels: bug, llm translation

### What happened?

Gemini models have minimum token count for caching: https://ai.google.dev/gemini-api/docs/caching?lang=rest#considerations

When using Claude Code + short custom system prompt, the proxied request to Gemini fails because there are too few tokens for caching

Suggestion: handle such cases gracefully and remove the caching label automatically

### Relevant log output

```shell INFO: 127.0.0.1:56097 - "POST /v1/messages?beta=true HTTP/1.1" 400 Bad Request 12:38:47 - LiteLLM Proxy:ERROR: endpoints.py:72 - litellm.proxy.proxy_server.anthropic_response(): Exception occured - litellm.BadRequestError: GeminiException BadRequestError - { "error": { "code": 400, "message": "Cached content is too small. total_token_count=896, min_total_token_count=1024", "status": "INVALID_ARGUMENT" } } . Received Model Group=gemini/gemini-2.5-flash Available Model Group Fallbacks=None LiteLLM Retried: 1 times, LiteLLM Max Retries: 2 Traceback (most recent call last): File "/Users/bytedance/GitHub/litellm-setup/.venv/lib/python3.12/site-packages/litellm/llms/vertex_ai/context_caching/vertex_ai_context_caching.py", line 480, in async_check_and_create_cache response…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d17h55m54s ago — entered · #import:https:::github.com:berriai:litellm post #2936
17696 requires provider-aware request transformation, token-threshold handling, and careful interaction with retries and caching semantics; 35354 appears comparatively localized to terminal-state bookkeeping with focused regression coverage.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search