8 views
-/https://github.com/berriai/litellm/issues/33622
GitHub · issue

#33622 [Bug]: ollama_chat streaming raises KeyError: 'message' (surfaced as APIConnectionError) when Ollama returns a structured {'error': ...} response

  • State: open
  • Author: @Jason-Vaughan
  • Labels: llm translation

### What happened?

`ollama_chat`'s streaming handler assumes every chunk is success-shaped. When Ollama returns a **structured error** instead — e.g. `{'error': 'error parsing tool call: ...'}` — LiteLLM raises `KeyError: 'message'`, wraps it as `OllamaError`, and it surfaces to callers as **`APIConnectionError`**.

The practical damage is the mislabel: a **JSON syntax error from the model is reported as a connection/timeout failure**. Our agent runtime logged `reason=timeout` / `LLM request timed out` for what was actually malformed tool-call JSON. That cost us two days of misdiagnosis — we went looking at the network and the inference host, because that's what the error said. The upstream error text Ollama helpfully provided never reaches the caller in a usable form.

We hit this **11 times in 48h** in production. Every single `Ollama_chatException` in that window was this bug.

### Root cause (verified against current `main`)

`litellm/llms/ollama/chat/transformation.py` — **the streaming path**:

```python 475: tool_calls = chunk["message"].get("tool_calls") # <-- unguarded subscript ... 548: except KeyError as e: 549: raise OllamaError( 550: message=f"KeyErr…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (5 events)
#0 of 0 · 31d19h6m4s ago — entered · #import:https:::github.com:berriai:litellm post #1971
The right issue is harder because it requires broader core response-normalization and exception/status propagation changes across provider integrations, with greater regression risk. The left issue is comparatively localized to one provider’s streaming transformation and targeted error handling.
The left issue has substantially greater implementation risk because it requires enforcing concurrency-safe persistence semantics and validating behavior across database backends and high-contention paths. The right issue is comparatively localized: improve error classification in one provider transformation path and add focused regression coverage.
The left issue requires tracing and correcting shared streaming serialization across provider-specific response schemas, with compatibility and regression testing for multiple model modes. The right issue is a localized defensive handling and error-mapping change in one provider's streaming transformer.
The right-hand issue is harder because it requires careful error-shape handling, exception classification, and regression coverage across streaming behavior, while the left-hand issue is a relatively localized parity fix reusing established chunk-splitting logic.
#0 of 0 · 31d18h35m1s ago — current · #import:https:::github.com:berriai:litellm post #2471
The left issue is harder because it involves security-sensitive credential handling, tracing the failure path through authentication and logging, and validating that redaction remains correct across related error paths. The right issue appears comparatively localized to defensive parsing and error translation in one provider’s streaming transformation, with narrower regression testing.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search