8 views
-/https://github.com/berriai/litellm/issues/13786
GitHub · issue

#13786 [Bug]: "The model is repeating the same chunk" may not terminate response processing

  • State: open
  • Author: @jeromeroussin
  • Labels: bug, enterprise

### What happened?

When the litellm proxy detects the issue "litellm.InternalServerError: The model is repeating the same chunk" it seems to call the custom logging callback thousands of times (and floods our telemetry system). Is it possible that the safety checker does not actually terminate the processing of the response?

### Relevant log output

```shell Traceback (most recent call last):\n File \"/usr/local/lib/python3.13/site-packages/litellm/litellm_core_utils/streaming_handler.py\", line 1708, in __anext__\n processed_chunk: Optional[ModelResponseStream] = self.chunk_creator(\n ~~~~~~~~~~~~~~~~~~^\n chunk=chunk\n ^^^^^^^^^^^\n )\n ^\n File \"/usr/local/lib/python3.13/site-packages/litellm/litellm_core_utils/streaming_handler.py\", line 1430, in chunk_creator\n raise exception_type(\n ...<3 lines>...\n )\n File \"/usr/local/lib/python3.13/site-packages/litellm/litellm_core_utils/streaming_handler.py\", line 1419, in chunk_creator\n return self.return_processed_chunk_logic(\n ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^\n completion_obj=completion_obj,\n ^^^^^^^^^^^^^^^^^…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (4 events)
#0 of 0 · 31d17h50m24s ago — entered · #import:https:::github.com:berriai:litellm post #3137
The right issue is harder because it crosses pass-through routing, response-format normalization, guardrail enforcement semantics, and likely multiple provider and streaming paths. The left issue appears more localized to streaming error-state termination and callback lifecycle handling, with a narrower validation surface.
The streaming lifecycle bug is harder because it requires tracing asynchronous response termination, exception propagation, callback invocation, and regression coverage across proxy execution paths. The provider addition is broader in integration points but follows established provider and metadata patterns, making its implementation more predictable.
The left task spans multimodal request normalization, provider-specific Gemini/Vertex translation, URL and MIME handling, metadata preservation, validation, and broad compatibility testing. The right task is more localized to streaming error termination and callback lifecycle control, with focused regression tests. The left therefore has substantially greater scope and integration risk.
#0 of 0 · 31d17h38m6s ago — current · #import:https:::github.com:berriai:litellm post #3344
Issue #30043 is harder because it spans cross-provider protocol compatibility, streaming semantics, token accounting, routing behavior, and broad regression coverage, while #13786 is primarily a localized streaming termination and callback-control defect.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search