9 views
-/https://github.com/berriai/litellm/issues/27588
GitHub · issue

#27588 [Feature]: Dynamic Cost Reconciliation for OpenRouter (Support for Service Tiers and Real-time Pricing)

  • State: open
  • Author: @alihossein
  • Labels: enhancement, proxy

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### The Feature

Implement a Dynamic Billing Reconciliation mechanism for OpenRouter provider to sync the response_cost with the actual billed amount.

Specifically, I propose adding a feature where LiteLLM can optionally fetch the final cost from OpenRouter's /api/v1/generation?id={id} endpoint after a request is completed, instead of solely relying on local static price maps. This is crucial for supporting OpenRouter's dynamic features like Service Tiers (Flex) and provider-specific discounts.

### Motivation, pitch

I am managing large-scale LLM infrastructure (Data Engineering at Digikala) and we use the /model/new endpoint to dynamically register hundreds of OpenRouter models.

**The Problem:** Currently, LiteLLM calculates costs locally based on static prices set at registration time. However, when we use extra_body={"service_tier": "flex"}, OpenRouter applies significant real-time discounts. Since LiteLLM is unaware of these dynamic billing changes, it logs the "standard" price in LiteLLM_SpendLogs, leading to a significant Cost Drift (e.g., LiteLLM logs $0…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d18h8m22s ago — entered · #import:https:::github.com:berriai:litellm post #2924
The left item spans provider integration, post-request lifecycle handling, external API reliability, pricing reconciliation, persistence, and compatibility concerns. The right item is comparatively localized to model metadata and existing parameter-validation behavior, with a smaller test surface and lower operational risk.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search