7 views
-/https://github.com/berriai/litellm/issues/33328
GitHub · issue

#33328 [Bug]: Replicate runtime fallback treats seconds as milliseconds

  • State: open
  • Author: @emerzon
  • Labels: SDK

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate

### What happened?

Replicate runtime pricing mixes seconds and milliseconds on current `litellm_internal_staging` at commit `65ca095d4d15a82372e53a547a2390a69f7e1797`

[`completion_cost`](https://github.com/BerriAI/litellm/blob/65ca095d4d15a82372e53a547a2390a69f7e1797/litellm/cost_calculator.py#L1110-L1150) documents `total_time` as the request duration in seconds

[`get_replicate_completion_pricing`](https://github.com/BerriAI/litellm/blob/65ca095d4d15a82372e53a547a2390a69f7e1797/litellm/cost_calculator.py#L696-L707) always divides the supplied duration by 1000:

```python return a100_80gb_price_per_second_public * total_time / 1000 ```

This conversion is correct when `completion_cost` sourced `total_time` from `_response_ms`

It is incorrect when a caller follows the documented API and passes seconds, and it is also incorrect when the helper derives `end_time - start_time`, which is a wall-clock duration in seconds

Those second-based paths undercount Replicate runtime cost by 1000 times

### Steps to Reproduce

Run:

```python from litellm.constants import DEF…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d19h7m33s ago — entered · #import:https:::github.com:berriai:litellm post #1833
The proxy change has broader scope: request-level option handling, response-path branching, provider compatibility, and end-to-end tests. The pricing correction is comparatively localized, mainly requiring consistent unit normalization and regression coverage.
#0 of 0 · 31d18h21m43s ago — current · #import:https:::github.com:berriai:litellm post #2578
The right issue is harder because it requires tracing and correcting duration-unit handling across multiple pricing call paths, preserving compatibility for existing callers, and adding targeted regression coverage. The left issue is comparatively localized: adjust unknown-model handling, emit a warning, and update tests for the changed contract.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search