### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

When using `googleMaps` tool with `response_format` (json_schema) on a `vertex_ai-` prefixed model, LiteLLM v1.89.0+ converts the request to use `responseFormat` in `generationConfig`. However, `responseFormat` is not a documented field in the Vertex AI API (the official docs only list `responseMimeType` + `responseSchema`). Under concurrent load this causes ~65-75% of requests to fail with:

"Unable to submit request because Both legacy structured output fields and new response format fields are set. Please use only one of the two approaches." The request body only contains `responseFormat` (no legacy fields), yet Vertex AI returns this error intermittently, indicating Google's backend is adding legacy fields internally on some requests before validation. Expected behavior: Vertex AI requests should use `responseMimeType` + `responseJsonSchema` (OLD format), which works 100% reliably. The `responseFormat` rewrite introduced in PR #29582 should only apply to the `gemini/` provider path (Gemini API), not `vertex_a…