Runway open-sources the Rust decoder that feeds data to its video models

Runway open-sources AVTensor, a Rust-written media decoder accelerating video training pipelines up to six times faster than Meta’s PyTorch torchcodec.

Runway is open-sourcing AVTensor, the Rust-written media decoder that powers its own training pipelines. Its function: to transform video and audio files into PyTorch tensors, the step through which a video generation model receives its data.

The tool processes video and audio on a single, synchronized timeline and sends them directly into PyTorch, without any intermediate library. It also performs resizing during decoding rather than after the fact, an operation that Runway claims is up to six times faster than torchcodec, the reference decoder of the PyTorch ecosystem by Meta.

The issue is a frequently overlooked bottleneck. At scale, training GPUs can sit idle waiting for data to be decoded and formatted, wasting compute time. By accelerating this step, AVTensor improved the MFU (Model FLOPs Utilization, the share of computing power actually utilized) of Runway's training by 1.8 percentage points, according to the company. The code is now public.