6 views
-/https://github.com/berriai/litellm/issues/31888
GitHub · issue

#31888 [Bug]: Proxy A/B Testing - Traffic Mirroring Not Working

  • State: open
  • Author: @garystafford
  • Labels: bug, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

A bug happened! I attempted to use 'A/B Testing - Traffic Mirroring', as documented here: https://docs.litellm.ai/docs/traffic_mirroring. The call to the primary model was successful, but the call to the secondary model appears to have never happened and fails silently. I tested this with multiple model combinations from OpenAI and Anthropic. The results were the same. No log entries are found for the secondary/mirrored model call. Direct calls to the secondary/mirrored model succeed as expected.

Also, note that the docs state: "Use the `is_silent_experiment: True` flag in your logs to filter and compare results between the primary and mirrored calls." Using `DEBUG` level logging, the term `is_silent_experiment` never appears.

My LiteLLM Proxy configuration (snippet):

```ini - model_name: bedrock-mantle-openai-gpt-5-5 litellm_params: model: bedrock_mantle/openai.gpt-5.5 api_key: os.environ/AWS_BEARER_TOKEN_BEDROCK silent_model: openai-gpt-5-5

- model_name: openai-gpt-5-5 litellm_params: model: openai/gpt-5.5 api_key…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (4 events)
#0 of 0 · 31d19h10m26s ago — entered · #import:https:::github.com:berriai:litellm post #1703
Traffic mirroring spans proxy request orchestration, asynchronous execution, provider-specific routing, failure handling, and logging/observability, creating broader integration and regression risk. The budget-window defect is more localized to update semantics, validation, persistence, and UI/API synchronization.
The traffic-mirroring issue is harder because it likely spans proxy orchestration, configuration resolution, background execution, error handling, and cross-provider integration testing. The other issue is more localized to multimodal request translation and can likely be addressed with focused conversion logic and regression tests.
The left issue is harder because it requires careful reconciliation of persistent and startup configuration, preservation of precedence, failure-safe synchronization, and security-focused regression coverage. The right issue is more likely a localized execution-path and observability fix.
#0 of 0 · 31d18h20m36s ago — current · #import:https:::github.com:berriai:litellm post #2507
A one-field response-schema correction is narrowly scoped and low risk. The other requires tracing proxy routing, background execution, configuration handling, observability, and provider-specific behavior, making investigation and regression testing substantially broader.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search