8 views
-/https://github.com/berriai/litellm/issues/30217
GitHub Ā· issue

#30217 [Bug] Anthropic `/v1/messages/count_tokens` ignores `api_base` and hardcodes `api.anthropic.com`, breaking self-hosted backends

  • State: open
  • Author: @jinbagi
  • Labels: llm translation, claude code

## Core issue

This is a **routing bug**, not just a tokenizer fallback issue.

LiteLLM supports Anthropic-compatible custom backends using `custom_llm_provider: anthropic` with a custom `api_base`, for example `api_base: http://<self-hosted-backend>/v1`.

In this setup, Anthropic Messages API routes should respect the configured `api_base`.

The normal `/v1/messages` route does this correctly.

However, `/v1/messages/count_tokens` does not. It ignores the configured `api_base` and directly calls the hardcoded Anthropic SaaS endpoint: `https://api.anthropic.com/v1/messages/count_tokens`.

This means one Anthropic sub-route silently bypasses the user-configured backend.

## Why this matters

This breaks valid LiteLLM deployment patterns:

* **self-hosted Anthropic-compatible backends** * **vLLM** serving Anthropic-compatible `/v1/messages` * **air-gapped or firewall-restricted environments** * **Claude Code integrations** routed through LiteLLM to an internal backend

Expected behavior:

`/v1/messages/count_tokens` → LiteLLM → configured `api_base` → `<api_base>/messages/count_tokens`

Actual behavior:

`/v1/messages/count_tokens` → LiteLLM → `https://api.anthropic.com/v1/messages/c…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 Ā· 31d19h2m43s ago — entered Ā· #import:https:::github.com:berriai:litellm post #1920
The right issue is harder because it requires callback compatibility across distinct response schemas, token accounting, content extraction, and regression coverage; the left issue is comparatively localized URL routing and configuration propagation.
#0 of 0 Ā· 31d18h52m6s ago — current Ā· #import:https:::github.com:berriai:litellm post #2103
#35124 is harder because it spans two streaming adapter implementations and requires preserving logging state and callback behavior across asynchronous stream lifecycles, with broader regression-testing risk; #30217 is comparatively localized URL-resolution and routing work.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search