NVIDIA founds the Cosmos Coalition and opens the Cosmos 3 model

NVIDIA launched the Cosmos Coalition and Cosmos 3, an open omnimodel for physical AI co-developed with Runway, LTX, and Black Forest Labs.

NVIDIA is forming a consortium called Cosmos Coalition to develop and open world models for Physical AI, the AI designed to act in the real world, such as robots or autonomous vehicles. These models simulate the physical environment: they predict its interactions and generate realistic scenes, allowing for the production of synthetic data to train robots and vehicles without filming everything and to shorten development cycles.

The initiative operates as an open ecosystem: members contribute models, research, and evaluations, and share NVIDIA's training tools and infrastructure. Weights, code, and datasets are published under the Linux Foundation's OpenMDW 1.1 license. Founding members include Runway, LTX, Black Forest Labs, Agile Robots, Generalist, and Skild AI, with Runway also co-developing a foundational model with NVIDIA.

The technical foundation is Cosmos 3, presented as the first open omnimodel, which combines text, image, video, ambient sound, and action. It relies on a so-called mixture-of-transformers architecture, separating reasoning from generation, and comes in Super versions (maximum precision), Nano (fast, for edge devices), and soon Edge (real-time). The model is available on Hugging Face with its weights, datasets, and fine-tuning recipes, and NVIDIA claims it leads physical AI benchmarks.