End-to-End Character Animation: The Wan-Animate-2 Approach

Wan-Animate-2 animates multiple characters, separates motion from camera angles, and includes a real-time version running at 24 FPS.

Wan-Animate-2 is a 14-billion-parameter framework for character animation using a reference image and a driving video. Unlike approaches that rely on extracting motion beforehand, the system directly processes the reference video within its Diffusion Transformer architecture to preserve character appearance and motion details.

The model can animate multiple characters within the same scene, with each maintaining its own identity and motion. It also adds camera control that is independent from the source video: a text instruction such as “top view” can change the viewing angle without being constrained by the perspective of the sequence used to drive the animation.

A Wan-Animate-2-Lite variant targets real-time use cases. Alibaba reports generation at 24 frames per second at 400 × 720 resolution on four H100 GPUs, with output produced in chunks to extend sequences over time. The team says it observed no visible accumulation of errors across those chunks.

The architecture combines mechanisms for temporal alignment between the reference and generated video, along with selective use of relevant visual information from the source image. The Distillation version reduces the process to 10 inference steps, compared with 40 in the example provided for the Base version.

The weights and inference code are available on Hugging Face, ModelScope, and GitHub under the Apache 2.0 license. The repository lists default settings for 720p generation on eight A800 GPUs and also mentions tests at 480p on two A800 GPUs.