8 views
-/https://github.com/berriai/litellm/issues/34111
GitHub · issue

#34111 [Bug]: Cohere models incompatible with 'num_generations'

  • State: open
  • Author: @bchen1116
  • Labels: bug, llm translation, SDK

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

If I pass a completion request to a cohere model with 'n' supplied, it fails:

```python import litellm import os

os.environ["COHERE_API_KEY"] = "..." model_id = "cohere/command-r7b-12-2024" messages = [{"role": "user", "content": "In one word, what color is the sky?"}]

response = litellm.completion(model=model_id, messages=messages, n=1) ```

Expected successful completion. Actually got: ```python litellm.exceptions.APIConnectionError: litellm.APIConnectionError: Cohere_chatException - {"id":"3d71291b-8a91-44e9-8431-d1337b90a07b","message":"unknown field: parameter 'num_generations' is not a valid field. For proper usage, please refer to https://docs.cohere.com/reference/chat"} ```

Likely stemming from [this line](https://github.com/BerriAI/litellm/blob/c1b6c4062ef5372c1e5e0027c721a942d2bb66cb/litellm/llms/cohere/chat/transformation.py#L167)

### Steps to Reproduce

1. install litellm==1.93.0 2. Use above code example, include `n=<number>` in the args

### Relevant log output

```shell

```

### What part of LiteLLM is this about?

SDK (li…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (5 events)
#0 of 0 · 31d19h20m18s ago — entered · #import:https:::github.com:berriai:litellm post #1632
The UI change has broader cross-component scope, requiring theme tokens, state handling, accessibility checks, and regression coverage; the provider fix is comparatively localized to request translation and targeted tests.
The left issue is harder because it requires resilient per-item error handling across the batch reconciliation workflow, correct failure-state persistence, continued polling, and regression coverage for interaction with existing batch states. The right issue is comparatively localized to provider-specific request transformation and validation.
28902 is harder because it spans frontend form-state handling, backend credential-preservation semantics, sensitive-data safety, and regression testing across update paths; 34111 is more localized to provider-specific parameter translation and compatibility tests.
The left issue is harder because it involves cross-cutting proxy failure-path data propagation, attribution invariants, and regression coverage across multiple request and logging flows. The right is comparatively localized to a provider-specific parameter translation.
#0 of 0 · 31d18h23m10s ago — current · #import:https:::github.com:berriai:litellm post #2556
The right issue requires a bounded provider-specific SDK change plus regression coverage, while the left issue has no actionable technical specification and appears lower-scope. The right is therefore moderately harder to implement.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search