7 views
-/https://github.com/berriai/litellm/issues/26428
GitHub · issue

#26428 [Bug]: Message role is omitted for zero-length messages (cases of immediate EOS).

  • State: open
  • Author: @abogovski
  • Labels: bug, SDK

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

Here's a valid sglang [cc_response_sse.txt](https://github.com/user-attachments/files/27054255/cc_response.json) albeit a corner-case. It consists of three server-sent events: 1. first one contains message role; 2. second one contains finish_reason; 3. third one contains token usage stats.

LiteLLM hides output message role from caller of `litellm.acocompletion` by simultaneously: - omitting first chunk; - not adding message role to other two chunks; - not adding extra chunk with role.

### Steps to Reproduce

Either run [mre.tar.gz](https://github.com/user-attachments/files/27054007/mre.tar.gz) or: 1. choose a chat-completion-request that would result in an empty message (e.g. via `continue_final_message=True`) 2. iterate over chunks from `litellm.acompletion(api_base=YOUR_SGLANG_API_BASE, stream=True)`

### Relevant log output

```shell

```

### What part of LiteLLM is this about?

SDK (litellm Python package)

### What LiteLLM version are you on ?

v1.83.11

### Twitter / LinkedIn details

https://www.linkedin.com/in/anton-bogovskiy-01b89…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d18h29m17s ago — entered · #import:https:::github.com:berriai:litellm post #2381
The right issue is harder because it spans billing-cycle state, aggregation correctness, persistence boundaries, and dashboard consistency, requiring broader investigation and regression coverage. The left issue is a narrowly scoped SDK streaming edge case with a more localized fix.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search