AW-0 teaches robots to program their movements and anticipate their consequences

With AW-0, Berkeley, NVIDIA, and Penn combine programming, video prediction, and robotic control into a model trained on 3,000 hours of demonstrations.

A robot capable of predicting what will happen, producing the corresponding code, and executing the necessary movements, all within a single artificial intelligence model. This is the principle behind AW-0, an architecture developed by researchers from the University of California, Berkeley, NVIDIA, and the University of Pennsylvania. Their approach combines three forms of generation that were previously often treated separately: code to structure tasks, video to anticipate environmental changes, and motor commands to act physically.

Dubbed Agentic World Action Models, the system was trained on approximately 3,000 hours of human and robotic demonstrations. The researchers report gains on concrete manipulation tasks, such as sorting fruit, folding towels, using tongs, or solving a Rubik's cube. Across eight physical robot trials, AW-0 achieves an average score of 71.1, compared to 46 for the best baseline model using the same evaluation grid.

The study, titled Agentic World Action Models, accompanies the release of the project's code and resources. Programming, imagining, acting

So-called vision-language-action (VLA) robotics models typically combine what a robot sees, the instructions it receives, and the actions it must perform. Other architectures, known as World Action Models (WAM), add the prediction of future visual states to anticipate the consequences of a movement.

AW-0 takes this approach further by integrating programming directly into the generation process. A first component produces code describing the skills to be deployed, the target objects, and their order of execution. A second component predicts upcoming video frames of the scene, while a third generates continuous motor commands for the robot's actuators.

These three components operate within a Mixture-of-Transformers architecture, in which information flows between experts during a unified denoising generation process. The code thus influences future frames and predicted actions, rather than being appended as an external instruction to the model.

This difference is illustrated in one of the demonstrations provided by the researchers. When a robot is instructed to leave only a mango in the left bowl, the model featuring a programming component predicts an interaction with the correct fruit. A version lacking this component moves toward an apple.

Here, the program provides an explicit representation of intent: which object to manipulate, in what order, and using which operation. A 50-skill library built from 3,000 hours of demonstrations

To learn these behaviors, the researchers collected roughly 2,000 hours of data recorded with dual-arm YAM robots and 1,000 hours of egocentric videos showing human manipulation.

These sequences are not used merely as visual examples. An automated annotation pipeline converts them into programs tied to various steps of the observed gestures.

Coding agents identify elementary operations, draft the corresponding instructions, and group recurring behaviors into a reusable skill library. The system notably leverages Gemini 3.1 Pro and Claude Code with Claude Opus 4.8 to generate and organize these annotations.

The final library contains 50 skills, 23 of which derive from robotic data and 27 from human demonstrations. The movements described this way can be reused during training and adapted to new tasks.

To fold a towel, for example, the model can rely on routines for grasping, moving, aligning, and folding. The program describes the sequence of operations, while the video and motor components learn to account for the fabric's actual position and deformation. Up to 80.8% success rate in simulation

AW-0 was benchmarked against four baseline architectures—π0.5, DreamZero, EgoScale, and WLA—on 50 tasks from the RoboTwin 2.0 benchmark. Each task underwent 50 trials across standardized and subsequently perturbed visual environments.

Under standard conditions, AW-0 achieves an 80.8% success rate, compared to 64.8% for WLA, the top-performing competitor in this setup. This represents a 16 percentage point gap.

When researchers alter lighting, textures, table heights, or add distractor objects, performance drops sharply. AW-0 falls to a 20.6% success rate, behind π0.5, which maintains 27.7% under these conditions.

The authors also evaluated a hybrid mode in which the robot can directly execute certain generated programs, provided they pass validity and progress checks. Otherwise, it falls back on the motor commands predicted by the model.

This configuration improves the success rate in visually perturbed environments, rising from 20.6% to 29.2%. Conversely, it reduces performance in standardized scenes, dropping from 80.8% to 63.1%.

The hybrid mode was also trained on specific demonstrations generated from executable programs. Consequently, it is not possible to attribute the entire performance difference solely to the switching mechanism between code and learned movements. Physical tasks evaluated on two types of robots

Experiments were not limited to RoboTwin. The team also evaluated AW-0 on eight real-world tasks using two distinct robotic setups: YAM, equipped with dual arms and parallel-jaw grippers, and Tianji–Sharpa, featuring articulated hands with 22 degrees of freedom each.

The tasks cover a variety of manipulations, from organizing fruit and packing a bag to replacing a lightbulb, dealing cards, or solving a cube.

Across these eight tests, AW-0 achieves an average score of 71.1, compared to 46 for DreamZero and 43.6 for π0.5. Detailed results include 91.7 for fruit sorting, 84.4 for towel folding, and 72.1 for Rubik's cube manipulation.

These scores must be distinguished from RoboTwin success rates. The physical experiments use evaluation rubrics that award partial credit for completed intermediate steps. A score of 71.1 does not mean the robot fully completes 71.1% of the tasks.

For each real-world task, researchers used 100 post-training demonstrations and conducted 48 evaluation trials split across three