6 views
-/https://github.com/berriai/litellm/issues/27113
GitHub · issue

#27113 [Bug]: `/v1/messages/count_tokens` ignores system prompt for vertex_ai partner models (Claude)

  • State: open
  • Author: @chanckben
  • Labels: bug, proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

When calling the `/v1/messages/count_tokens` endpoint with a system prompt against a Vertex AI Claude model (e.g. `vertex_ai/claude-sonnet-4-6`), the returned `input_tokens` value is identical to the value returned without a system prompt. The system prompt tokens are silently not counted.

The same request against `azure_ai/claude-sonnet-4-6` and `bedrock/global.anthropic.claude-sonnet-4-6` correctly accounts for the system prompt and returns a higher `input_tokens` value when a system prompt is included.

**Expected behaviour** `input_tokens` reflects tokens from both messages and system.

**Actual behaviour** `input_tokens` reflects only messages tokens; system is silently ignored.

**Root Cause (code-level)**

The system parameter is dropped at the Vertex AI partner-model layer. The call chain is:

1. `VertexAITokenCounter.count_tokens` (`litellm/llms/vertex_ai/common_utils.py:1087`) receives system but does not forward it to the next layer:

```python result = await partner_models_handler.count_tokens( model=model_to_use, messages=…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (7 events)
#0 of 0 · 31d19h11m27s ago — entered · #import:https:::github.com:berriai:litellm post #1683
The right issue is harder because it requires coordinated frontend and backend changes around secret-preserving update semantics, masking behavior, validation, and regression coverage. The left issue is comparatively localized to correcting parameter propagation through an existing provider token-counting path with focused tests.
The right issue is harder because it crosses shared token-counting interfaces, provider-specific plumbing, request-shape compatibility, and regression testing across integrations. The left is comparatively localized to one streaming translation path with focused field mapping and event-level tests.
Issue 31831 spans cross-cutting configuration, date-boundary semantics, persistence/aggregation behavior, exports, and UI consistency, creating substantially broader integration and regression risk. Issue 27113 appears localized to a model-specific request-processing path with focused tests and limited surface area.
Issue #34232 is harder because it spans schema/index design, migration and deduplication of existing data, null-safe conflict handling, concurrent writes, and spend-reporting correctness. Issue #27113 is comparatively localized to the Vertex token-counting call path with focused regression tests.
The right issue is harder because it involves tracing and correcting cross-layer request propagation in provider-specific token counting, with compatibility checks and regression tests. The left issue is comparatively localized to logging behavior and should require a smaller code change.
Issue 27924 is harder because it spans endpoint compatibility, model-routing semantics, provider translation, configuration behavior, and runtime failure modes. Issue 27113 has a localized, identified parameter-propagation defect with a comparatively narrow fix and targeted tests.
#0 of 0 · 31d18h13m45s ago — current · #import:https:::github.com:berriai:litellm post #2618
The left issue is harder because it affects shared routing state, cache-lifetime semantics, configuration behavior, and multi-deployment correctness, requiring broader regression coverage. The right issue is comparatively localized to parameter propagation and token-counting tests.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search