6 views
-/https://github.com/berriai/litellm/issues/30635
GitHub ยท issue

#30635 [Bug]: Batch API cost never tracked โ€” background check_batch_cost poller gets 403 "User None does not have access" on managed batch retrieve (regression from #27004)

  • State: open
  • Author: @smartinov
  • Labels: llm translation

### What happened?

On a self-hosted LiteLLM proxy (OSS docker image, `STORE_MODEL_IN_DB=true`), per-request cost for OpenAI-style **managed batches** (`/v1/files` + `/v1/batches` with `custom_llm_provider: vertex_ai`) is **never tracked**. A virtual key that only runs batch jobs shows **$0 spend**, even after the batch completes successfully and produces an output file.

The background `check_batch_cost` scheduler job fires, but its retrieve of the completed managed batch is rejected by the managed-files access hook with `403 "User None does not have access to the object โ€ฆ"`. Instead of a `batch_success` cost row, a `failure` SpendLog row is written on each poll cycle.

### Root cause

`CheckBatchCost.check_batch_cost` (`enterprise/litellm_enterprise/proxy/common_utils/check_batch_cost.py`) calls `llm_router.aretrieve_batch(...)`, which routes through the proxy `retrieve_batch` endpoint โ†’ `pre_call_hook` โ†’ `litellm_enterprise/proxy/hooks/managed_files.py::async_pre_call_hook` โ†’ `can_user_call_unified_object_id` โ†’ `can_access_resource` (`litellm/llms/base_llm/managed_resources/isolation.py`).

The poller's internal retrieve runs with an empty auth context โ€” `user_api_key_dict.user_โ€ฆ

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 ยท 31d17h47m40s ago โ€” entered ยท #import:https:::github.com:berriai:litellm post #3184
30635 is harder because it requires tracing and safely repairing authentication context across background scheduling, router retrieval, enterprise hooks, resource isolation, and spend accounting, with regression coverage across managed batch flows. 35383 is comparatively narrower: adjusting ownership identity and configurable limits within the MCP session-management path, though it still needs concurrency and compatibility tests.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search