6 views
-/https://github.com/berriai/litellm/issues/31149
GitHub · issue

#31149 [Bug]: Anthropic `/v1/messages/count_tokens` ignores custom `api_base` despite repeated production reports

  • State: open
  • Author: @jinbagi
  • Labels: llm translation, claude code

## Summary

`/v1/messages/count_tokens` still does not consistently respect a custom `api_base` for Anthropic compatible deployments.

This has already been reported multiple times.

Related issues:

1. #26323 2. #29764 3. #30217

They all point to the same core problem.

## Problem

When `custom_llm_provider: anthropic` is configured with a custom `api_base`, every Anthropic native subrequest should use that same resolved backend.

That is not happening.

`/v1/messages` can route to the configured backend, while `/v1/messages/count_tokens` can still fall back to `api.anthropic.com`.

This is not a tokenizer issue. It is a proxy routing bug.

## Production impact

This breaks real production deployments using LiteLLM with self hosted Anthropic compatible backends.

This includes setups such as:

1. vLLM Anthropic compatible endpoints 2. Claude Code routed through LiteLLM 3. internal enterprise gateways 4. isolated or restricted network environments

Production users should not have to keep patching around the same `api_base` propagation bug.

## What is frustrating

This has been reported more than once with clear details.

The expected behavior is simple. If a deployment has a cus…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (3 events)
#0 of 0 · 31d18h8m44s ago — entered · #import:https:::github.com:berriai:litellm post #2710
31149 requires cross-cutting backend routing analysis, provider-specific behavior validation, and regression coverage; 31222 is primarily a contained frontend restoration using existing APIs and components.
The right issue is harder because it requires tracing provider and endpoint-specific routing state across shared request construction paths, with compatibility and regression risks across multiple Anthropic deployments. The left issue is comparatively localized to guardrail decision logic and targeted tests.
#0 of 0 · 31d17h52m16s ago — current · #import:https:::github.com:berriai:litellm post #2997
Issue #16060 is harder because it involves diagnosing state, concurrency, filtering, and fallback behavior across the router’s usage-based deployment-selection machinery. Issue #31149 appears narrower, centered on propagating a resolved endpoint through one Anthropic token-counting request path, with related reports likely providing a focused reproduction and fix boundary.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search