8 views
-/https://github.com/berriai/litellm/issues/28735
GitHub · issue

#28735 [Bug]: Synthetic include_usage chunk violates OpenAI spec — usage event has non-empty choices instead of choices: []`

  • State: open
  • Author: @dborysenko
  • Labels: bug, proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

Related: #8450 (same symptom, closed stale), PR #8751 (proposed fix, not merged).

With `stream: true` and `stream_options: {"include_usage": true}`, the OpenAI streaming spec requires the final usage chunk to have **`choices: []`**. LiteLLM emits a synthetic usage chunk at stream end with a **non-empty** `choices` array (see log output below).

OpenAI's documented shape for the final usage chunk:

```json { "choices": [], "usage": { "prompt_tokens": 13, "completion_tokens": 20, "total_tokens": 33 } } ```

Spec reference: [Chat Completions streaming — `include_usage`](https://platform.openai.com/docs/api-reference/chat/streaming)

> The usage field on this chunk shows the token usage statistics for the entire request, and **the choices field will always be an empty array**.

### Steps to Reproduce

```bash curl -N 'http://127.0.0.1:4000/v1/chat/completions' \ -H 'Authorization: Bearer sk-...' \ -H 'Content-Type: application/json' \ -d '{ "model": "bedrock-claude-sonnet-4-6", "messages": [{"role": "user", "content": "Say hello…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d18h51m19s ago — entered · #import:https:::github.com:berriai:litellm post #2003
The right issue is harder because it requires provider-specific request translation, nested message restructuring, preservation of existing caching semantics, and broader compatibility testing. The left issue is comparatively localized to final streaming-event construction and validation across affected response paths.
#0 of 0 · 31d18h35m1s ago — current · #import:https:::github.com:berriai:litellm post #2272
The right issue is harder because it requires tracing shared streaming and usage-aggregation paths, preserving protocol behavior across providers, and adding regression coverage without breaking existing clients. The left issue is comparatively localized to model metadata and provider recognition with targeted tests.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search