8 views
-/https://github.com/berriai/litellm/issues/29286
GitHub · issue

#29286 [Bug]: `/v1/messages` streaming emits empty `input_json_delta` instead of `text_delta` when OpenAI-compatible vLLM stream returns `content` with empty `tool_calls: []`

  • State: open
  • Author: @jinbagi
  • Labels: bug, proxy, llm translation, claude code

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

I am using LiteLLM Proxy `v1.85.0` in front of a vLLM `v0.21.0` OpenAI-compatible backend serving **DeepSeek V4 Pro**. When I call the same model through LiteLLM's OpenAI-compatible `/v1/chat/completions` endpoint with `stream: true`, streaming works correctly. The backend returns normal OpenAI-style chunks with `delta.content`. However, when I call the model through LiteLLM's Anthropic-compatible `/v1/messages` endpoint with `stream: true`, the SSE stream is converted incorrectly. For a plain text response, LiteLLM emits repeated empty `input_json_delta` chunks instead of `text_delta` chunks. This causes Claude Code or other Anthropic-compatible clients to receive no visible streamed text, even though the OpenAI-compatible stream contains valid `delta.content`. Environment: ```text LiteLLM version: v1.85.0 vLLM version: v0.21.0 Backend model: DeepSeek V4 Pro Backend API type: OpenAI-compatible vLLM server Client path: Claude Code / Anthropic-compatible /v1/messages Proxy path: Claude Code -> NGINX -> APISIX -> LiteLLM v1.85.0 -> vLLM v0.21.0 -…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d18h28m19s ago — entered · #import:https:::github.com:berriai:litellm post #2416
The left issue requires diagnosing and correcting streaming translation logic across protocol formats, preserving chunk semantics, and adding regression coverage for backend-specific edge cases. The right issue is primarily a coordination and prioritization request without a defined implementation scope.
#0 of 0 · 31d17h49m27s ago — current · #import:https:::github.com:berriai:litellm post #3063
Issue 34364 is harder because it spans a database schema migration, transactional timezone handling, partition lifecycle logic, and backward-compatibility risks across deployments. Issue 29286 is more localized to stream-event translation and regression coverage.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search