19 views
-/https://github.com/berriai/litellm/issues/26324
GitHub · issue

#26324 [Bug] acount_tokens: Anthropic count path skips chat-completions → native translation; standard chat-format payloads rejected with 400

  • State: open
  • Author: @eumemic
  • Labels: llm translation

## Environment - litellm `1.83.4` and `1.83.12` - Provider: `anthropic` - Python 3.13

## What breaks

`litellm.acount_tokens(model='anthropic/…', messages=…, tools=…, system=…)` is documented as taking "standard chat format" (chat-completions / OpenAI-shape). For any realistic agent payload — tool-role messages, function-typed tools, system-role messages — Anthropic returns 400 and `acount_tokens` silently falls back to the local tiktoken counter.

Sample errors:

- `messages: Unexpected role "tool". Allowed roles are "user" or "assistant".` - `messages: Unexpected role "system". The Messages API accepts a top-level system parameter, not "system" as an input message role.` - `tools.0: Input tag 'function' found using 'type' does not match any of the expected tags: 'custom', 'bash_20250124', …`

Meanwhile the regular chat-completion path (`litellm.acompletion(...)`) handles these inputs fine because it runs them through `AnthropicConfig.transform_request` which does the chat-completions → Anthropic Messages translation (fold tool-role into user content blocks, strip the `function` wrapper off tools, lift system to top-level, etc.).

## Root cause

`AnthropicCountTokensHandler.handl…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d18h24m59s ago — entered · #import:https:::github.com:berriai:litellm post #2434
The right-side change spans proxy persistence, model-update mutation, router-specific state, and model-info filtering, requiring lifecycle tracing and regression coverage across API and database paths. The left-side change is more localized to the Anthropic token-counting adapter and shared request normalization, with narrower tests and lower integration risk.
#0 of 0 · 31d18h8m53s ago — current · #import:https:::github.com:berriai:litellm post #2704
26324 requires broader cross-layer integration, reuse or adaptation of existing request transformation logic, and comprehensive coverage of varied payload shapes and compatibility behavior. 26323 is comparatively localized parameter propagation with narrower regression testing.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search