6 views
-/https://github.com/berriai/litellm/issues/31385
GitHub ยท issue

#31385 [Bug]: completion_start_time (TTFT) falls back to end_time for streaming /v1/messages (agentic-hook path) and /v1/responses (always)

  • State: open
  • Author: @WellMaxWang
  • Labels: bug, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

Summary

/chat/completions streaming correctly records completion_start_time (time of the first chunk), so TTFT in the Admin UI / Prometheus / OTEL is accurate.

/v1/messages and /v1/responses do not, in two different ways:

/v1/messages โ€” the fix from #25599 (merged via #25828) only patches the plain streaming path (BaseAnthropicMessagesStreamingIterator.async_sse_wrapper). If any registered callback overrides async_should_run_agentic_loop (e.g. the built-in websearch_interception / compression_interception callbacks, or any custom CustomLogger implementing that hook), the stream is wrapped in AgenticAnthropicStreamingIterator instead, which never sets completion_start_time at all. /v1/responses โ€” completion_start_time is never set anywhere in the Responses API streaming path. This triggers the end_time fallback on every streaming request, unconditionally โ€” no special config required.

In both cases, litellm_logging.py's fallback kicks in:

python# litellm/litellm_core_utils/litellm_logging.py if self.completion_start_time is None: self.โ€ฆ

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 ยท 31d18h39m33s ago โ€” entered ยท #import:https:::github.com:berriai:litellm post #2224
31385 spans multiple streaming architectures and callback-dependent execution paths, requiring coordinated instrumentation and regression tests across APIs. 34388 is narrower provider-specific request normalization with targeted compatibility tests, though it carries model-behavior risk.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search