7 views
-/https://github.com/berriai/litellm/issues/29156
GitHub · issue

#29156 [Bug]: SGLang rerank bug

  • State: open
  • Author: @DarkLorianPrime
  • Labels: bug, proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

A bug happened! When using SGLang's /v1/rerank endpoint, the response is a plain array/list: [ {"score": 0.95, "index": 0, "document": "..."} ]

However, litellm expects the response to have a results field (like OpenAI's format): { "results": [ {"index": 0, "relevance_score": 0.95, "document": {"text": "..."}} ], "model": "qwen3-reranker-8b", "id": "rerank-xxx" }

When using SGLang backend, litellm crashes with: ``` 'list' object has no attribute 'get' ```

### Steps to Reproduce

1. Start SGLang server with rerank model. 2. Call litellm rerank. 3. Except: litellm.exceptions.APIConnectionError: litellm.APIConnectionError: None - {"error":{"message":"'list' object has no attribute 'get'","type":"None","param":"None","code":"500"}}

### Relevant log output

```shell C:\Users\xxx\PycharmProjects\phoenix-gc\venv\Scripts\python.exe C:/Users/xxx/PycharmProjects/phoenix-gc/script.py 16:26:59 - LiteLLM:WARNING: get_model_cost_map.py:271 - LiteLLM: Failed to fetch remote model cost map from https://raw.githubusercontent.com/BerriAI/litell…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (4 events)
#0 of 0 · 31d18h56m13s ago — entered · #import:https:::github.com:berriai:litellm post #1942
#24929 requires diagnosing and safely changing shared HTTP-client lifecycle behavior under concurrency, streaming, timeout, and deployment conditions, with broader regression testing. #29156 is comparatively localized response-shape normalization in one provider adapter.
The left issue is harder because it involves correcting shared persistence, batching, retry, and data-integrity behavior across multiple spend paths, with higher concurrency and regression risk. The right issue is comparatively localized to response normalization in one provider integration and can likely be addressed with focused parsing and tests.
Issue #29156 is harder because it requires provider-specific response normalization, defensive handling of incompatible schemas, regression coverage, and validation across the reranking translation path. Issue #33434 is comparatively localized to deterministic role-priority selection in the SSO authorization flow, though it carries some permission semantics risk.
#0 of 0 · 31d18h1m39s ago — current · #import:https:::github.com:berriai:litellm post #2842
Issue 27362 is harder because it changes shared routing, retry, and deployment-health behavior, requiring careful compatibility analysis and regression coverage across cooldown policies. Issue 29156 is more localized to provider-specific response normalization and should have a narrower implementation and test surface.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search