6 views
-/https://github.com/berriai/litellm/issues/26323
GitHub · issue

#26323 [Bug] acount_tokens: api_base silently dropped between public function and Anthropic handler

  • State: open
  • Author: @eumemic
  • Labels: llm translation

## Environment - litellm `1.83.4` and `1.83.12` - Provider: `anthropic` - Python 3.13

## What breaks

`litellm.acount_tokens(model='anthropic/…', api_base='https://my-proxy/…', api_key='…')` sends its request to the default `https://api.anthropic.com/v1/messages/count_tokens` instead of the provided `api_base`. Callers routing through proxies (local aggregators, OAuth-fronted gateways, custom rotation services) get 401/403/network-unreachable, not a count.

## Root cause

`litellm.acount_tokens` (`litellm/main.py`) correctly packs `api_base` into `deployment["litellm_params"]["api_base"]` before calling the provider's `token_counter_instance.count_tokens(...)`.

`AnthropicTokenCounter.count_tokens` (`litellm/llms/anthropic/count_tokens/token_counter.py`) then reads `api_key` out of `litellm_params` but **does not read `api_base`**, and calls `handle_count_tokens_request` without passing it:

```python litellm_params = deployment.get("litellm_params", {}) api_key = litellm_params.get("api_key") # pulled # api_base = litellm_params.get("api_base") # MISSING

result = await anthropic_count_tokens_handler.handle_count_tokens_request( model=model_to_use, messages=mess…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d18h8m31s ago — entered · #import:https:::github.com:berriai:litellm post #2704
26324 requires broader cross-layer integration, reuse or adaptation of existing request transformation logic, and comprehensive coverage of varied payload shapes and compatibility behavior. 26323 is comparatively localized parameter propagation with narrower regression testing.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search