6 views
-/https://github.com/berriai/litellm/issues/33678
GitHub · issue

#33678 [Bug]: ollama_chat streaming emits index=0 for every tool call, merging parallel tool calls into malformed JSON

  • State: open
  • Author: @ndungx
  • Labels: llm translation

### What happened?

With `ollama_chat/*` + streaming + tools, **every** tool call delta is emitted with `index=0`. When a model emits 2+ tool calls in one turn, any client that accumulates `arguments` by `index` (the OpenAI streaming convention) merges them into a single malformed tool call:

``` {"path": "a.rs"}{"path": "b.rs"} ```

That string then goes into the client's conversation history as `tool_calls[0].function.arguments`. On the **next** turn it is sent back to the proxy, where `OllamaChatConfig.transform_request` calls a bare `json.loads()` on it ([`ollama/chat/transformation.py#L268`](https://github.com/BerriAI/litellm/blob/main/litellm/llms/ollama/chat/transformation.py#L268)) and raises. The proxy surfaces it as a 500:

``` litellm.APIConnectionError: Extra data: line 1 column 29 (char 28) File "/app/.venv/lib/python3.13/site-packages/litellm/main.py", line 5522, in completion response = _complete_ollama_chat(_dispatch_ctx) ```

Two things make this painful to diagnose:

1. The error surfaces **one turn after** the turn that corrupted the history. 2. It is reported as `APIConnectionError` (unmapped exceptions fall through to it), so it reads like a network probl…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (3 events)
#0 of 0 · 31d18h59m49s ago — entered · #import:https:::github.com:berriai:litellm post #1881
Issue 34704 spans multiple observability paths, metric semantics, label propagation, and compatibility/cardinality risks, requiring broader design and integration work. Issue 33678 is comparatively localized to streaming tool-call indexing and associated regression coverage.
#33678 requires debugging and correcting stateful streaming/tool-call assembly semantics, with cross-component behavior and regression coverage; #31872 is a localized parameter-allowlist update with low integration risk.
#0 of 0 · 31d18h8m15s ago — current · #import:https:::github.com:berriai:litellm post #2729
The right issue requires debugging and changing streaming transformation behavior, handling stateful edge cases, and adding regression coverage across multiple integration paths. The left issue is a localized configuration-data update with comparatively low implementation risk.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search