7 views
-/https://github.com/berriai/litellm/issues/34772
GitHub Ā· issue

#34772 [Feature]: Add `litellm token-count` CLI subcommand for offline token counting

  • State: open
  • Author: @Harsh23Kashyap
  • Labels: potential-duplicate

## Problem

`litellm cost-estimate` (#34686) and `litellm doctor` (#34513) give operators an offline CLI for spend planning and SDK diagnostics, but there is no matching one-shot way to count tokens for a given prompt without writing Python or hitting a provider. The internal `litellm.utils.token_counter()` is the canonical function, but it requires a Python import and a working litellm install to use.

This blocks three common offline workflows:

1. **CI cost gates** — fail the build if a prompt exceeds N tokens for a chosen model. 2. **Model selection** — quickly check whether a long prompt fits a small-context model (8K, 16K) before switching. 3. **Chunking strategy** — measure how many tokens a corpus chapter is, for a fixed tokenizer, before picking a chunk size.

Today, all three require either an inline Python call, a one-off `import litellm; litellm.token_counter(...)` in a REPL, or a round trip to `/v1/messages/count_tokens` (#34243 is a separate proxy endpoint issue, not a CLI).

## Motivation

`litellm.cli` already ships two offline subcommands that follow the same shape (single-purpose, no provider calls, table or JSON output, exit code 0/1/2). A `litellm token-count` s…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 Ā· 31d18h49m51s ago — entered Ā· #import:https:::github.com:berriai:litellm post #2042
The CLI feature spans command design, input parsing, tokenizer/model resolution, output modes, error handling, documentation, and cross-platform testing. The other change is comparatively localized to stream-aggregation state handling with focused regression coverage.
#0 of 0 Ā· 31d18h1m57s ago — current Ā· #import:https:::github.com:berriai:litellm post #2825
The CLI feature is harder because it requires designing and integrating a new user-facing command, defining input and output behavior, handling tokenizer/model edge cases, and adding broader tests and documentation. The pricing bug is comparatively narrower, centered on correcting an existing calculation path and validating provider-specific accounting.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search