8 views
-/https://github.com/berriai/litellm/issues/30004
GitHub · issue

#30004 EmbeddingResponse(**response_json) fails with `must be a mapping, not list` when using openai_like/lm_studio provider with llama.cpp server

  • State: open
  • Author: @GDXbsv
  • Labels: llm translation

## Bug Description

When using `lm_studio/` (or `hosted_vllm/`, `openai/`) as the provider for an OpenAI-compatible embedding server (llama.cpp), LiteLLM raises a `TypeError` because `response_json` passed to `EmbeddingResponse(**response_json)` is a **list** instead of a **dict**.

The upstream server returns a **valid, standard OpenAI-format** response. The bug is in LiteLLM's internal handling of that response.

---

## Error

`lm_studio/` and `hosted_vllm/`:

``` File "litellm/llms/openai_like/embedding/handler.py", line 75, in aembedding return EmbeddingResponse(**response_json) TypeError: litellm.types.utils.EmbeddingResponse() argument after ** must be a mapping, not list ```

`openai/` — same root cause, different layer:

``` File "litellm/llms/openai/openai.py", line 1269, in aembedding stringified_response = response.model_dump() AttributeError: 'list' object has no attribute 'model_dump' ```

---

## Steps to Reproduce

1. Run a llama.cpp embedding server with `--embeddings` flag:

```bash docker run ghcr.io/ggml-org/llama.cpp:server-vulkan \ --embeddings --host 0.0.0.0 --port 8080 \ --hf-repo nomic-ai/nomic-embed-text-v2-moe-GGUF \ --hf-file nomic-embed-te…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (4 events)
#0 of 0 · 31d19h20m22s ago — entered · #import:https:::github.com:berriai:litellm post #1744
The right issue is harder because it spans configuration, timezone-aware aggregation and filtering, export formatting, and coordinated Admin UI behavior across multiple layers. The left issue is comparatively localized to response normalization and regression coverage in embedding handlers.
Issue 31451 is harder because it requires tracing and correcting cross-protocol request translation across proxy, client, and provider-specific parameter handling, with compatibility and regression risks across multiple model families. Issue 30004 appears more localized to defensive response normalization in embedding handlers, despite needing coverage across a few provider paths.
The left issue is highly underspecified, so implementation would require broad diagnosis, reproduction, and scope discovery. The right issue has a concrete failure path and localized provider-response handling, making its fix more bounded and testable.
#0 of 0 · 31d18h18m34s ago — current · #import:https:::github.com:berriai:litellm post #2750
Streaming error semantics require tracing asynchronous event translation, preserving exception behavior across SDK layers, and adding regression coverage without breaking valid terminal events. The embedding defect is more localized response-shape normalization, though it spans two adapter paths and needs compatibility tests.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search