6 views
-/https://github.com/berriai/litellm/issues/28882
GitHub · issue

#28882 bug(rerank): extra_headers silently dropped for hosted_vllm and other rerank providers

  • State: open
  • Author: @magicsong
  • Labels: llm translation

## Bug Report

**LiteLLM version**: current `litellm_internal_staging`

### What happened

When using the `hosted_vllm` rerank provider (and other non-Bedrock rerank providers) with `extra_headers` in the call parameters, the extra headers are silently dropped and never forwarded to the HTTP request. This breaks any deployment where a gateway or sidecar proxy requires custom headers for routing — for example an Istio HTTPRoute that matches on `x-model-name`.

### Root cause

In `litellm/rerank_api/main.py`, the `rerank()` function dispatches to provider branches. The **Bedrock** branch correctly merges `extra_headers`:

```python # Bedrock branch (correct) merged_headers = headers or litellm.headers or {} extra_headers_from_kwargs = kwargs.get("extra_headers") if extra_headers_from_kwargs: merged_headers = {**merged_headers, **extra_headers_from_kwargs} response = bedrock_rerank.rerank(..., extra_headers=merged_headers) ```

But **all other providers** (hosted_vllm, cohere, azure_ai, infinity, together_ai, deepinfra, watsonx, huggingface, fireworks_ai, jina, nvidia) pass only:

```python headers=headers or litellm.headers or {}, # ← extra_headers silently dropped ```

### Imp…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d17h53m42s ago — entered · #import:https:::github.com:berriai:litellm post #2970
Issue 28882 spans a shared dispatch path and numerous provider integrations, requiring consistent behavior, compatibility checks, and broad regression coverage. Issue 31861 is localized to one buffering implementation with a comparatively contained optimization and validation surface.
#0 of 0 · 31d17h40m28s ago — current · #import:https:::github.com:berriai:litellm post #3204
Changing the authorization boundary requires tracing authentication and provisioning paths, defining safe backward-compatible semantics, and adding security-focused regression coverage; the other is primarily localized parameter plumbing across provider branches with focused tests.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search