A coordinating brain for robots and their actions

Gemini Robotics ER 2 organizes machine actions continuously through video analysis. What this model from Google DeepMind changes for robotics.

Conceived as a reasoning layer placed on top of motor commands, Gemini Robotics ER 2 allows a robot to interpret a request, observe its environment, and organize a task into multiple steps. The model does not directly control every movement. It transmits its decisions to a vision-language-action model, a navigation API, or any other interface declared by the developer.

Google DeepMind adds continuous video understanding. While a robot acts, ER 2 tracks its progress, verifies that a step is actually completed, and adapts its plan when a problem arises. This approach is intended to limit the pauses during which the machine stops to analyze what to do next, thanks to a bidirectional connection with the Gemini Live API.

On evaluations published by the company, the model achieves 57.4% accuracy in classifying task progress across five levels. It also scores 91.3% on detecting the exact moment an event occurs in a video, with an average deviation of 0.96 seconds. This capability can be used to determine when to stop pouring a liquid, move to the next step, or retry an incomplete action.

Collaboration between multiple machines is also part of this release. Robots with different characteristics can share a common understanding of the mission, divide operations among themselves, and hand off a task. Google DeepMind illustrates this feature with equipment from Apptronik, Boston Dynamics, and Agile Robots, particularly for object retrieval and scenarios requiring multiple systems.

Safety progresses in parallel with better human proximity detection. In Google's demonstrations, the system can pause a humanoid's movement when a person enters its workspace, then resume once the area is clear. Gemini Robotics ER 2 is accessible to developers via the Gemini API and Google AI Studio, as well as in private preview on the Gemini Enterprise Agent Platform.