6 views
-/https://github.com/berriai/litellm/issues/31562
GitHub · issue

#31562 [Bug]: BaseModelResponseIterator falsely terminates stream when tool-call arguments contain "[DONE]"

  • State: open
  • Author: @chenbojian
  • Labels: llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

`BaseModelResponseIterator._handle_string_chunk` uses a substring check (`if "[DONE]" in str_line`) to detect the SSE termination signal. This causes false stream termination when a provider sends a valid JSON chunk whose string values happen to contain the literal "[DONE]".

For example, if a model generates a tool call with arguments like `{"command": "[DONE] reset"}`, or any JSON payload containing "[DONE]" as a substring, the stream is prematurely cut off. The user sees a truncated response with no error.

This affects all providers using `BaseModelResponseIterator`: Databricks, OpenRouter, OVHCloud, Cloudflare, Triton, A2A, Watsonx, Anthropic completion, Ollama, CometAPI, Bedrock DeepSeek, Vertex AI agent engine.

### Steps to Reproduce

Using any provider that routes through `BaseModelResponseIterator`, trigger a streaming response where the model's output contains the literal string "[DONE]" inside JSON content (e.g. inside tool-call arguments).

The simplest programmatic reproduction:

```python from litellm.llms.base_llm.base_model_ite…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d18h36m17s ago — entered · #import:https:::github.com:berriai:litellm post #2262
The right issue is harder because it involves tracing instrumentation, exporter behavior, event/span semantics, configuration interactions, and cross-provider validation. The left issue appears more localized to stream termination parsing with focused regression tests.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search