6 views
-/https://github.com/berriai/litellm/issues/34351
GitHub · issue

#34351 Chat completions bridged to Responses drops usage on response.incomplete

  • State: open
  • Author: @lizhen921
  • Labels: llm translation

## What happened?

When `/v1/chat/completions` is bridged to the Responses API (e.g. `azure/responses/gpt-5.5`), hitting `max_output_tokens` causes Azure/OpenAI to emit `response.incomplete` with the real usage (`output_tokens`, `reasoning_tokens`, etc.).

LiteLLM's chat stream bridge only maps usage from `response.completed`. `response.incomplete` (and `response.failed`) fall through as unhandled events, so upstream usage is dropped. `stream_chunk_builder` then falls back to local `token_counter` on the visible text, which:

- under-reports `completion_tokens` (e.g. 607 vs Azure `output_tokens` 1024) - loses `reasoning_tokens` (shows 0) - also mis-counts `prompt_tokens` for multimodal inputs

## Expected behavior

Terminal Responses events that carry usage (`response.completed`, `response.incomplete`, `response.failed`) should translate that usage into the chat stream chunk. Incomplete should map to `finish_reason=length`.

## Reproduction

1. Configure a model as `azure/responses/<deployment>` (or any chat path with `mode: responses`). 2. Call `/v1/chat/completions` with streaming and a low `max_tokens` so the model hits the cap. 3. Observe LiteLLM debug logs: `type=response.inco…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d19h2m49s ago — entered · #import:https:::github.com:berriai:litellm post #1805
The left requires tracing a streaming translation pipeline, handling multiple terminal states, preserving provider-reported accounting, mapping termination semantics, and adding regression coverage across modalities and failure paths. The right is a narrowly scoped configuration correction with straightforward validation.
#0 of 0 · 31d18h55m25s ago — current · #import:https:::github.com:berriai:litellm post #1930
The right-side change is harder because it spans multiple translation paths, terminal-state handling, metadata propagation, fallback behavior, and regression coverage. The left-side change is a comparatively localized pricing-precedence correction with narrower behavioral scope.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search