7 views
-/https://github.com/berriai/litellm/issues/31954
GitHub · issue

#31954 [Bug]: Wrong values for max_tokens in some models

  • State: open
  • Author: @sats-23
  • Labels: bug, proxy

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

Certain models have wrong max_tokens value in model_prices_and_context_window.json

Example: https://github.com/BerriAI/litellm/blob/912ca6255ccb385abfa33ecececfe8cbbd108e34/model_prices_and_context_window.json#L37086

Has a value of 20,480, but actual value is 131,072 Refer - https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-models-ibm.html?context=wx#granite-4

### Steps to Reproduce

1. Setup WatsonX on IBM Cloud config for litellm instance 2. Do a query to /model/info 3. Notice the wrong values

### Relevant log output

```shell data: - model_name: ibm/granite-4-h-small litellm_params: use_in_pass_through: false use_litellm_proxy: false use_xai_oauth: false merge_reasoning_content_in_choices: false model: watsonx/ibm/granite-4-h-small project_id: 2ee3b103-xxxx base_url: 'https://' model_info: id: 11d243604726585c2d59ceab0e6d0e2db2dfd0ab2775d605a02dddcb8f951643 db_model: false key: watsonx/ibm/granite-4-h-small max_tokens: 20480 max_input_…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d18h6m34s ago — entered · #import:https:::github.com:berriai:litellm post #2747
The left issue is harder because correcting capability metadata may require tracing provider defaults, model-info propagation, validation, and regression coverage across proxy behavior. The right issue is comparatively contained to pricing data and its time-dependent update.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search