6 views
-/https://github.com/berriai/litellm/issues/30617
GitHub · issue

#30617 [Bug]: Streaming tool-call responses crash with TypeError: 'MockValSer' object is not an instance of 'SchemaSerializer'

  • State: open
  • Author: @lizzzcai
  • Labels: llm translation

## Describe the bug

When a streaming response contains tool calls (or logprobs), LiteLLM crashes mid-stream with:

``` TypeError: 'MockValSer' object is not an instance of 'SchemaSerializer' ```

The stream terminates early and the client receives no tool call content.

## Root cause

`openai._models.BaseModel` sets `defer_build=True`, so all openai SDK subclasses start with `MockValSer` as their pydantic serializer. The deferred build fires on `__init__`, but the SDK always parses streaming responses via `model_validate()`, which does not trigger the build. So objects like `ChoiceLogprobs` and `ChoiceDeltaToolCall` retain `MockValSer` after parsing.

`streaming_handler.py` stores these raw SDK objects as extra field values in `ModelResponseStream`. When `proxy_server._serialize_streaming_chunk()` calls `model_dump_json()`, pydantic's Rust core looks up `type(value).__pydantic_serializer__` at the C level (no Python `__getattr__` fallback), finds `MockValSer`, and raises `TypeError`.

The crash is intermittent: `model_dump()` accidentally self-heals via `MockValSer.__getattr__` -> `attempt_rebuild()`. If a usage chunk triggers `model_dump()` first, the class is rebuilt as a side e…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (3 events)
#0 of 0 · 31d18h21m23s ago — entered · #import:https:::github.com:berriai:litellm post #2586
The left task requires diagnosing and safely correcting a subtle interaction between deferred Pydantic schema construction, SDK model parsing, streaming serialization, and tool-call/logprob object lifecycles, with regression coverage across intermittent stream paths. The right task is narrower: tracing proxy concurrency accounting and configuration enforcement, though it may require attention to async or distributed request handling. Overall implementation risk and cross-cutting scope are substantially higher on the left.
31863 is harder because it spans provider-specific request transformation, model capability handling, beta-flag configuration, compatibility across multiple Claude generations, and comprehensive Bedrock regression coverage. 30617 is narrower and primarily involves repairing serialization lifecycle handling in shared streaming code, though it carries moderate cross-path regression risk.
#0 of 0 · 31d18h7m34s ago — current · #import:https:::github.com:berriai:litellm post #2824
26897 is harder because it spans agent integrations, router parameter propagation, Azure endpoint selection, and Responses-bridge behavior, requiring broader compatibility testing; 30617 appears more localized to streaming serialization and a focused SDK/Pydantic workaround.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search