5 views
-/https://github.com/berriai/litellm/issues/28530
GitHub · issue

#28530 [Bug]: Ollama Gemma 4 Infinite Tool Loop: Role mismatch ("tool" vs "tool_responses")

  • State: open
  • Author: @Najafu
  • Labels: bug, llm translation, SDK

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

### Description When using **Gemma 4 (cloud)** model (gemma4:31b-cloud) via the **Ollama** provider in LiteLLM, the model enters an infinite tool-calling loop. Even after the tool result is provided in the conversation history, the model fails to recognize the result and repeatedly triggers the same tool call.

This occurs because Gemma 4 (via Ollama) expects the tool result role to be `tool_responses`, but LiteLLM standardizes it to the OpenAI-compatible `tool` role. Because of this mismatch, the model ignores the result and hallucinates that the tool was never executed.

### Root Cause & Evidence This is a known semantic mismatch in how Gemma 4 handles tool roles compared to the OpenAI standard. The model specifically looks for `tool_responses` to terminate the tool-use phase. When it receives the standardized `tool` role, it does not recognize the output as a valid result, leading it to believe the tool was never called.

**Reference:** This issue was identified and fixed in the Google ADK project here: [https://github.com/google/adk-pytho…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (3 events)
#0 of 0 · 31d18h53m57s ago — entered · #import:https:::github.com:berriai:litellm post #1969
34917 is harder because it involves proxy streaming, request-boundary handling, UTF-8 correctness, and regression-sensitive routing behavior, whereas 28530 is more narrowly scoped to provider-specific message-role translation and validation.
The left item requires designing and integrating a new optimization subsystem across the SDK and proxy, including search orchestration, evaluation hooks, persistence, configuration, and operational safeguards. The right item is comparatively localized to provider-specific message translation and regression coverage.
#0 of 0 · 31d18h44m15s ago — current · #import:https:::github.com:berriai:litellm post #2136
The left issue requires cross-cutting audit architecture, cryptographic signing and canonicalization, chain/state persistence, callback reliability, verification tooling, and compliance-sensitive design. The right issue is comparatively localized provider-specific message translation with focused regression coverage.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search