8 views
-/https://github.com/berriai/litellm/issues/13245
GitHub · issue

#13245 [Bug]: Incomplete spend tracking for non-streaming Bedrock calls when client disconnects

  • State: open
  • Author: @dproud
  • Labels: bug

### What happened?

### Bug Description When a client disconnects during a non-streaming Bedrock API call, LiteLLM fails to accurately track the full cost of the request, even though AWS Bedrock continues processing and charges for the complete request.

### Expected Behavior - LiteLLM should track the full cost of all Bedrock requests, regardless of client connection status - Spend tracking should match AWS billing accurately - Cost logs should include both input and output token costs

### Actual Behavior - Pre-call logging captures request metadata and estimated input costs before sending to AWS - When clients disconnect during processing, post-call logging is skipped - AWS continues processing and charges for the full request (input + output tokens) - LiteLLM only records partial costs (input tokens only), creating discrepancy with AWS billing

### Technical Root Cause In the Bedrock provider implementation: 1. `pre_call()` logging occurs at `litellm/llms/custom_httpx/llm_http_handler.py:352` BEFORE the HTTP request to AWS 2. `post_call()` logging occurs after AWS response is received in the provider's `make_call()` function 3. Client disconnects trigger task cancellation, prev…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (3 events)
#0 of 0 · 31d18h21m9s ago — entered · #import:https:::github.com:berriai:litellm post #2495
The left issue is harder because it requires cancellation-safe asynchronous request handling, reliable deferred accounting, provider-specific behavior, and regression coverage under disconnect races. The right issue is a more bounded authentication change involving session expiry state, activity tracking, and coordinated UI/API tests.
The right issue is harder because it requires reliable cancellation-safe accounting across asynchronous request lifecycles, provider response handling, and billing metadata persistence, with significant risk of regressions in failure and disconnect paths. The left issue is comparatively localized to provider-specific capability validation or request translation.
#0 of 0 · 31d17h37m40s ago — current · #import:https:::github.com:berriai:litellm post #3245
#13245 is harder because it involves cancellation-safe asynchronous request handling, reliable post-request accounting, and provider-specific lifecycle testing; #31481 is comparatively localized to cost metadata resolution and regression tests.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search