7 views
-/https://github.com/berriai/litellm/issues/35346
GitHub · issue

#35346 tests/test_litellm/interactions/test_openapi_compliance.py fails: Google Interactions API spec dropped Content discriminator and added "queued" status

  • State: open
  • Author: @hadasgo

## Bug Description

`tests/test_litellm/interactions/test_openapi_compliance.py` fetches Google's live Interactions API OpenAPI spec (`https://ai.google.dev/static/api/interactions.openapi.json`) on every CI run and asserts against exact snapshots of specific schema fields. Google has changed the live spec in two ways that now fail these pinned assertions:

1. `TestRequestCompliance::test_content_schema_uses_discriminator` - asserts `"discriminator" in content_schema` for the `Content` schema. The live spec no longer includes a `discriminator` key on `Content`; it's now a plain `oneOf` with no discriminator. 2. `TestResponseCompliance::test_status_enum_values` - asserts `Interaction.properties.status.enum == ["in_progress", "requires_action", "completed", "failed", "cancelled", "incomplete", "budget_exceeded"]`. The live spec now includes an additional `"queued"` value at the end of that enum.

This is expected/by-design breakage per the test's own comments (`test_status_enum_values` notes "this test intentionally breaks CI when Google changes the live spec"), but it currently fails on the base branch itself, independent of any application code change, so every PR's CI run is red o…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d17h50m18s ago — entered · #import:https:::github.com:berriai:litellm post #3018
The right issue is substantially harder: it requires designing and integrating new runtime metrics, configuration and accounting flows, provider or hardware data handling, validation, and likely broader testing. The left issue is a narrowly scoped maintenance change to expectations in an existing compliance test.
#0 of 0 · 31d17h42m35s ago — current · #import:https:::github.com:berriai:litellm post #3155
32484 likely requires tracing a version regression across model registration, cost resolution, and environment-specific routing while preserving backward compatibility. 35346 appears comparatively localized to updating brittle external-contract tests and validating CI behavior.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search