CADRE: kyutai reduces the response time of AI-generated virtual worlds

Kyutai releases CADRE, seven open video autoencoders for building interactive worlds at 20 frames per second on a single GPU.

An AI-generated world can produce convincing video while responding too slowly to user actions. That delay may be relatively unimportant in conventional video generation, but it becomes a problem when controlling a robot, driving through a simulated environment, or manipulating objects in real time. With CADRE, French research laboratory Kyutai introduces seven open video autoencoders designed to reduce this latency while preserving motion quality. The associated experimental world models run at 20 frames per second on a single NVIDIA H100 GPU, with action response latency below 230 milliseconds in selected interactive configurations. Kyutai is also releasing the training and inference code, model weights, and an interactive keyboard-controlled demonstration. Why Convincing Video Can Make a Poor Simulator

A world model is a learned simulation system that predicts how an environment changes based on previous observations and actions. It can generate the visual consequences of a robot movement, steering input, or interaction with an object. These models can support agent training, policy evaluation, planning, and interactive simulations.

To reduce computation, world models typically operate on compressed video representations rather than directly generating every pixel. A video autoencoder converts frames into a compact latent space, where the world model predicts what happens next. A decoder then reconstructs the visible video. Compression therefore influences both the amount of computation required and the delay before an action appears on screen.

Many world models reuse autoencoders from text-to-video systems such as Wan 2.2, LTX-2, and NVIDIA Cosmos. These systems, however, were primarily designed to produce complete video sequences, not to respond immediately to interactive commands. They often combine several consecutive frames into a single latent representation, reducing processing costs but requiring all corresponding actions to be available before generating their effects.

At 20 frames per second, grouping eight frames introduces at least 400 milliseconds of delay, before adding generation and decoding time. The researchers cite earlier findings suggesting that people begin noticing delays between their actions and visible consequences at around 230 milliseconds. Compression that works well for offline video generation can therefore create considerable input lag in interactive simulations. Seven Autoencoders With Different Compression Strategies

CADRE stands for Causal Diffusion Representation Autoencoders. The project extends Kyutai's earlier work on MIRA, an interactive world model developed around Rocket League, to a broader set of environments.

The team trained seven autoencoders using a common training approach, varying the degree of spatial and temporal compression. This allows researchers to isolate how each factor affects visual quality, motion, and responsiveness. Spatial compression represents larger image regions with fewer latent elements, while temporal compression groups multiple consecutive frames into a single representation.

CADRE's encoder uses visual features from DINOv3, with the feature extractor frozen during training. A separately trained bottleneck determines the compression level. The decoder uses causal diffusion to reconstruct video progressively without relying on future frames that have not yet been generated.

To make decoding efficient, the researchers apply self-distillation so that a reconstruction requires only one or two decoding steps. This retains some of the visual detail associated with diffusion while avoiding the lengthy sampling processes commonly used in image and video generation.

The autoencoders were trained on approximately 76,000 hours of video from 23 public datasets. The researchers then trained world models with a shared architecture and training procedure, allowing differences in performance to be examined primarily through the compression strategy. Testing Manipulation, Driving, and Indoor Navigation

The study evaluates four environments: robot manipulation, human hand manipulation, autonomous driving, and indoor navigation. The corresponding datasets are ABC, EgoDex, NVIDIA Physical AI Autonomous Vehicles, and RealEstate10K.

Each world model receives two seconds of real video and must generate the following 4.8 seconds, conditioned only on recorded actions. Experiments use a resolution of 288 × 512 pixels at 20 frames per second and evaluate sequences held out from training.

The researchers measure three complementary properties. FID assesses image quality, FVD evaluates video distributions and temporal consistency, and ARR measures how accurately the actions underlying a sequence can be recovered from the generated video. The last metric distinguishes footage that merely looks plausible from footage that follows the provided controls.

Across the evaluation environments, CADRE configurations occupy much of the best-performing latency-quality trade-off, particularly when the camera is moving. Comparisons include autoencoders from Wan 2.2, Cosmos, LTX-2, DC-AE, and MIRA. Selected CADRE configurations sustain real-time output on one H100 while keeping action latency below the study's perceptibility threshold.

Different CADRE variants nevertheless make different trade-offs. Some preserve more visual detail, while others minimize computation and response time. No single compression setting provides the best performance for every environment. Diffusion Decoding Preserves Sharper Motion

One of the study's findings concerns reconstruction from heavily compressed representations. When a latent contains limited information, several possible videos may correspond to it. A decoder trained through regression can average those possibilities together, producing blurred textures and motion.

CADRE instead uses a diffusion decoder to produce a plausible detailed sequence. At equivalent compression to MIRA's autoencoder, its one-step decoder achieves 1.4 to 2.5