6 views
-/https://github.com/berriai/litellm/issues/16021
GitHub · issue

#16021 [Bug]: OpenRouter cost information lost in streaming responses

  • State: open
  • Author: @bbarwik
  • Labels: bug, llm translation

## Description

When using LiteLLM proxy with OpenRouter in streaming mode, the cost information returned by OpenRouter in the usage.cost field is not preserved in the final response. This works correctly in non-streaming mode.

## Reproduction

1. Configure LiteLLM proxy with OpenRouter 2. Make a streaming chat completion request: from openai import OpenAI

client = OpenAI(base_url="http://litellm-proxy-url", api_key="...") stream = client.chat.completions.create( model="openrouter/x-ai/grok-4-fast", messages=[{"role": "user", "content": "test"}], stream=True )

for chunk in stream: if chunk.usage: print(chunk.usage) # cost field is missing

## Expected Behavior

The final streaming chunk should include the cost in the usage object, similar to non-streaming responses: usage.cost = 0.00123 # actual cost from OpenRouter

## Actual Behavior

The cost field is missing from the usage object in streaming responses, even though OpenRouter returns it.

Example of last chunk received (from logs): usage=CompletionUsage( completion_tokens=255, prompt_tokens=63, total_tokens=318, # cost field is missing here, but present in OpenRouter's response )

##…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (4 events)
#0 of 0 · 31d19h32s ago — entered · #import:https:::github.com:berriai:litellm post #1857
The left issue is harder because it requires tracing provider metadata through LiteLLM’s streaming aggregation and serialization paths, while preserving compatibility across chunking and usage-finalization behavior. The right issue is narrower: extending an existing authentication option into the embedding client construction path with focused validation and tests.
The streaming-cost issue is harder because it spans provider response parsing, stream aggregation, usage-schema propagation, and compatibility tests, whereas the metadata issue is more likely isolated to the Anthropic request path and field propagation.
Preserving provider metadata through streaming requires tracing asynchronous chunk handling, aggregation, translation, and schema compatibility, with greater regression risk. The access-control correction is comparatively localized to model-visibility filtering and focused authorization tests.
#0 of 0 · 31d17h56m4s ago — current · #import:https:::github.com:berriai:litellm post #2929
The left task involves cross-system authentication, OAuth/MCP proxy integration, organization-level identity semantics, and likely documentation and compatibility work. The right task is a narrower response-translation and streaming-state preservation fix with focused regression tests.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search