Quick answer
Start troubleshooting with four hard Preview boundaries:
- every backend in one router shares one Vertex AI hostname;
- every client request includes a valid
modeltag or name; - targets are configured pre-deployed Vertex AI Model Garden MaaS models;
- the request is text-based OpenAI-compatible JSON.
Response streaming through server-sent events is supported. Request streaming, gRPC, WebSockets, and Gemini Live are not.
Symptom matrix
| Symptom | First check | Safe response |
|---|---|---|
| Config deployment rejected | Backend references, router names, OAS 3.x syntax | Validate and deploy a new version; keep the prior config |
| Wrong default model | defaultModel backend and target | Make the default explicit and test an unknown virtual name |
| Route does not match | Request model value versus rules[].model | Normalize only documented values; do not guess aliases |
| Cross-host error | Global and regional hosts mixed in one router | Split routers or align all backends to one supported host |
| Unexpected missing-model behavior | Client omitted model | Validate and reject before the gateway |
| Feature mismatch | Tool, schema, modality, stream, or parameter differs | Test the exact target; expose incompatibility to callers |
Host checklist
Compare the full hostname, not just the provider or project. aiplatform.googleapis.com and us-central1-aiplatform.googleapis.com are different hosts. A single router cannot mix them. Model routing also does not turn arbitrary third-party endpoints into Vertex AI backends.
Protocol checklist
Log the content type, path, method, model field, streaming flag, API config revision, response code, and selected backend where available. Reproduce with a minimal text request before adding tools, long context, or streaming. Do not retry non-idempotent surrounding actions simply because a model response failed.
Use the OpenAPI guide for reference resolution and the model target guide for publisher-specific paths.
Frequently asked questions
Why can’t one model router use global and regional hosts together?
Google requires every backend referenced by one router to share a single hostname: either the global Vertex AI host or one regional Vertex AI host.
What happens if the model field is missing?
Google documents a Public Preview issue where a missing model field can be processed incorrectly instead of rejected. Validate that field before requests reach the gateway.
Does model routing support WebSockets or Gemini Live?
No. The Preview supports server-sent response streaming but not request-side streaming, gRPC, WebSockets, or Gemini Live.
Official sources
- API Gateway model routing overview and limitations
- Google model routing launch
- OpenAPI 3.x feature limitations
Source check: August 5, 2026. Recheck Preview limitations and known issues before each deployment.