Runway shares the construction of Runway Characters

Runway Characters animates single images into real-time 24fps HD conversational videos using the GWM-1 model, achieving a low 1.75-second server latency.

Characters is a video agent that transforms a single reference image, whether photorealistic, drawn, or imaginary, into an animated conversational character in real-time. The output achieves 24 frames per second in HD, without requiring prior fine-tuning. The model handles lip-sync, facial expressions, and head movements based on the incoming audio.

The system relies on GWM-1, the publisher's General World Model, and employs frame-by-frame autoregressive generation, streamed to the client as it occurs. Runway claims 37 milliseconds of effective model time per frame and a server latency of 1.75 seconds between the end of a user's utterance and the start of the character's response. This delay breaks down into 1,185 milliseconds for the voice agent and 567 for the video pipeline, with the client-server network adding approximately 200 milliseconds per trip.

Several optimizations make execution possible within the 42-millisecond per-frame budget imposed by 24fps: Distribution Matching Distillation to reduce the number of denoising steps, shared tensor parallelism between the diffusion transformer and VAE decoder whose overlapping execution eliminates most of the decoding from the critical path, fine-grained KV cache management, CUDA Graph capture of the forward pass, and tuning of attention and matmul kernels.

Around the model, Runway exposes a set of peripheral functions: webcam vision or screen sharing, customizable voice via text or cloning from a sample, tool calling to trigger UI actions or backend calls, a document-based knowledge base, an embeddable widget with a single line of code, and integration with video conferencing tools (Zoom, Google Meet, Teams). Characters is immediately accessible via the Runway API and web and mobile applications.