6 views
-/https://github.com/berriai/litellm/issues/31871
GitHub · issue

#31871 fix(anthropic): use model capability flag for response_format routing + eager_input_streaming + citation offsets

  • State: open
  • Author: @deepanshululla
  • Labels: llm translation

## Problem

`AnthropicConfig` routes `response_format` requests by checking whether the model name contains a hardcoded substring set. When `claude-opus-4-8` was added to the model-cost map with `supports_output_config: true`, the hardcoded routing set was never updated, so every request to `claude-opus-4-8` with `response_format` falls into the tool-call emulation path instead of the native `output_format` API.

Two additional gaps: `eager_input_streaming` is silently dropped from tool definitions because it is absent from both the `_map_tool_helper` passthrough blocks and the `AnthropicMessagesTool` TypedDict. Citation-enriched responses omit character offset fields (`supported_text_start_char_index` / `supported_text_end_char_index`) that downstream consumers need to locate cited text.

## Proposed fix

Replace the hardcoded substring set with a call to `_supports_model_capability(model, "supports_output_config")` (already exists, already correct). Add `eager_input_streaming` passthrough in `_map_tool_helper` and to `AnthropicMessagesTool`. Compute and attach char offsets in `extract_response_content`.

## Files

- `litellm/llms/anthropic/chat/transformation.py` - `litellm/types…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d18h53m58s ago — entered · #import:https:::github.com:berriai:litellm post #2074
#27917 is harder because it likely requires tracing state across billing-period boundaries, aggregation logic, persistence, and regression coverage. #31871 is comparatively bounded to several localized Anthropic transformation and type-mapping changes with an existing capability helper.
#0 of 0 · 31d18h29m48s ago — current · #import:https:::github.com:berriai:litellm post #2455
Issue 20294 is harder because it spans endpoint behavior, persisted metadata, YAML-derived defaults, and likely UI/API integration, requiring clarification of configuration precedence and broader regression coverage. Issue 31871 is comparatively contained within the Anthropic transformation layer, with an existing capability helper and a few focused type, mapping, and response-processing changes.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search