9 views
-/https://github.com/berriai/litellm/issues/29913
GitHub · issue

#29913 [Bug]: Streaming /v1/responses success logger crashes with "'dict' object has no attribute 'usage'" → no spend log written, request goes uncharged

  • State: open
  • Author: @marktipo
  • Labels: bug, proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

On the proxy, **streaming** requests to `/v1/responses` (OpenAI Responses API surface, `stream=True`) never get a `LiteLLM_SpendLogs` row written, so they are not charged. The client still receives a correct 200 streamed body, but the success-logging callback crashes non-blockingly with:

``` LiteLLM.LoggingError: [Non-Blocking] ... 'dict' object has no attribute 'usage' ```

The crash happens **only** on the streaming Responses path:

| Path | Spend log written? | |---|---| | `/v1/responses`, `stream=True` | ❌ **crashes — no row** | | `/v1/responses`, `stream=False` (blocking) | ✅ fine | | `/v1/chat/completions`, `stream=True` | ✅ fine |

Because cost is never computed for the streamed responses request, the gateway silently under-bills that surface. The `x-litellm-response-cost` response header is also unreliable here (it's a pre-stream estimate flushed before the body), so the missing spend row is the real signal.

## Root cause

`litellm/litellm_core_utils/litellm_logging.py`, in `Logging._get_assembled_streaming_response`:

```python elif …

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (3 events)
#0 of 0 · 31d18h52m46s ago — entered · #import:https:::github.com:berriai:litellm post #2194
The left task spans asynchronous streaming assembly, logging callbacks, response-shape normalization, billing correctness, and regression coverage across API paths. The right task is narrower, likely centered on centralized client-side authentication error handling and navigation behavior, though it still requires validating interaction with existing session flows.
The right-side fix is harder because it touches shared streaming response assembly, asynchronous logging, usage normalization, and billing correctness, with broader regression risk across proxy endpoints. The left-side fix is comparatively contained to SAP-specific structured-message models and request transformation, with narrower testing scope.
#0 of 0 · 31d18h21m51s ago — current · #import:https:::github.com:berriai:litellm post #2690
The right issue is harder because it spans streaming response assembly, asynchronous logging, usage normalization, billing persistence, and regression coverage across multiple API paths. The left is primarily a dashboard capability addition with narrower UI and configuration changes.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search