Analysis and planned orchestration
Standard preview
Use when structured output, code execution, file or URL context, batch work, or richer non-streaming analysis matters.
Quick answer
Use ER 2 as a high-level embodied-reasoning layer: stream or upload physical-world context, plan work, call bounded tools, track progress, and hand execution to a VLA model or robot API. Keep motor control and safety-rated behavior outside the model.
New sibling layer · August 2026
Gemini Robotics ER 2 interprets multimodal context, plans, calls declared tools, and tracks progress. Anthropic's Model Hardware Standard is a separate, model-agnostic research preview for making programmable devices discoverable through standardized drivers and read/write primitives, with access through MCP, CLI, or APIs.
A team could evaluate an MHS-backed device capability as one bounded tool beneath ER 2, but neither provider source establishes native compatibility. Keep device identity, command authorization, deterministic limits, controller behavior, independent interlocks, expert oversight, and real-hardware validation outside the model.
Use the MHS versus MCP architecture guide for the layer map and the MHS readiness check before designing a combined proof of concept.
High-level embodied reasoning: understand a physical scene, plan a multi-step task, call tools, and hand motor execution to a lower-level VLA model or robot API.
`gemini-robotics-er-2-preview` for spatial reasoning, video understanding, structured output, code execution, grounding, and tool orchestration.
`gemini-robotics-er-2-streaming-preview` for low-latency Live API sessions with continuous multimodal input and blocking function calls.
Both preview endpoints accept text, images, video, and audio and return text. They do not directly return motor commands unless your application exposes a structured robot tool.
Public preview in Google AI Studio and the Gemini API; private preview on Gemini Enterprise Agent Platform.
Not established by Google's launch. Check the current Flowith model selector independently.
Layer decision
Analysis and planned orchestration
Use when structured output, code execution, file or URL context, batch work, or richer non-streaming analysis matters.
Reactive robot-agent loop
Use when a persistent Live API session, continuous audio or camera input, and low-latency function calling matter.
Motor execution
Keep navigation, grasping, movement, and actuator control behind explicit tools, independent safety controls, and validated hardware adapters.
Physical-system boundary
Google reports improved instruction-following and human-proximity evaluations. Those results do not certify a robot, site, task, actuator, sensor stack, or operating procedure.