A video model that teaches physics to robots: LingBot-Video

Robbyant open-sources LingBot-Video, a 30B MoE video foundation model that trains robots in physical world interactions, beating Cosmos3 on the RBench.

Robbyant, Ant Group's embodied AI subsidiary, is open-sourcing LingBot-Video, touted as the first MoE (mixture of experts)-based video foundation model designed for embodied AI—machines that act in the physical world. The model boasts 30 billion parameters, with only 3 billion activated during inference.

While most video generators aim for aesthetically pleasing shots, LingBot-Video focuses on the physical grounding a robot needs. Its architecture blends a Diffusion Transformer (DiT) with an MoE layer, claiming approximately three times the efficiency of a dense model, a prerequisite for use in real-time control loops. For data, the developer adds 70,000 hours of embodied sequences (fine manipulation, navigation, first-person interaction) to pre-training on internet videos, teaching the model how an action modifies its environment, not just what a video looks like. Post-training utilizes multi-criteria reinforcement rewarding physical plausibility and task completion.

This foundation serves as a video simulator for robotic arms, humanoids, quadrupeds, or mobile platforms, generating action progression, contact states, and environmental feedback for data synthesis, policy evaluation, and planning purposes. On the RBench benchmark (co-produced by Peking University and ByteDance), Robbyant claims the top spot ahead of competitors like Wan2.6, Seedance 1.5 Pro, and Cosmos3 Super for generating physically credible robotic behaviors.

Weights, code, technical report, and project page are accessible on Hugging Face, GitHub, and ModelScope.