6 views
-/https://github.com/berriai/litellm/issues/29764
GitHub · issue

#29764 [Bug]: Anthropic `/v1/messages/count_tokens` ignores configured `api_base` and hardcodes `api.anthropic.com`, breaking vLLM Anthropic-compatible backends

  • State: open
  • Author: @jinbagi
  • Labels: bug, proxy, llm translation, claude code

## Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

## What happened?

I am using LiteLLM with a self-hosted vLLM backend that exposes the Anthropic-compatible Messages API.

The reason for using `custom_llm_provider: anthropic` is intentional: when using `hosted_vllm`, LiteLLM converts Anthropic `/v1/messages` requests into `/v1/chat/completions`, and that adapter path is not fully compatible with Claude Code / Anthropic Messages features such as content blocks, tool use, tool results, thinking blocks, and other Anthropic-native request shapes.

So the intended setup is:

```yaml model_list: - model_name: claude-opus-* litellm_params: custom_llm_provider: anthropic model: ultra api_base: http://<vllm-host>:<port>/v1 api_key: <vllm-api-key> ```

In this setup, `/v1/messages` should be forwarded to the configured Anthropic-compatible backend, not to Anthropic SaaS.

That works for normal message generation.

However, when Claude Code calls:

```text POST /v1/messages/count_tokens ```

LiteLLM does not forward the request to the configured `api_base`.

Instead, LiteLLM attempts to call:

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d18h38m29s ago — entered · #import:https:::github.com:berriai:litellm post #2325
The right issue is harder because it spans a new CLI workflow, multiple external agent integrations, remote discovery, authentication, configuration generation, and cross-platform testing, while the left issue is comparatively localized routing and regression-test work.
#0 of 0 · 31d18h18m2s ago — current · #import:https:::github.com:berriai:litellm post #2646
The right-hand task spans provider-specific endpoint routing, configuration propagation, compatibility behavior, and regression coverage, while the left is a localized metadata extraction fix with narrower testing scope. Therefore, the right issue is substantially harder.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search