18 views
-/https://github.com/berriai/litellm/issues/28239
GitHub · issue

#28239 [Bug]: Default encoding_format list in model parameters triggers vLLM failure

  • State: open
  • Author: @ndjones
  • Labels: bug, proxy

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

Including encoding_format as a list within model parameters leads to a vLLM failure on model calls that don't explicitly set a specific encoding_format, as by default LiteLLM passes the whole list to vLLM as a string.

``` "encoding_format": [ "float", "base64", "ubyte", "int8" ]

```

The model Test Connection behaves in this default way, and displays this error

`Error testing connection: litellm.BadRequestError: Hosted_vllmException - {"error":{"message":"3 validation errors:\n {'type'... `

Using the Swagger API you can reproduce this error by not setting encoding_format or otherwise setting it.

### Steps to Reproduce

1. Set encoding_format to a list on an embedding model 2. Make a call to the embeddings endpoint without specifying an encoding_format 3. Make a call to the embeddings endpoint with specifying an encoding format

### Relevant log output

```shell

```

### What part of LiteLLM is this about?

Proxy

### What LiteLLM version are you on ?

v1.83.14

### Twitter / LinkedIn details

_No response_

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d18h31m22s ago — entered · #import:https:::github.com:berriai:litellm post #2353
#31851 is harder because it spans proxy data access, shared health-check caching, public API behavior, and dashboard consistency, with greater regression and integration-testing risk. #28239 is comparatively localized to parameter validation/serialization and provider-specific request handling.
#0 of 0 · 31d18h9m26s ago — current · #import:https:::github.com:berriai:litellm post #2701
Issue 33987 is harder because it spans asynchronous batch polling, terminal-state handling, logging-worker error propagation, provider-specific file semantics, and zero-cost accounting, requiring broader regression coverage. Issue 28239 is comparatively localized to request-parameter normalization for embedding calls and vLLM compatibility.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search