ByteDance launches Bernini, an open-source framework for video generation and editing

ByteDance open-sources Bernini, an Apache-2.0 licensed video generation and editing framework separating semantic planning from image rendering.

ByteDance open-sources Bernini, a unified framework dedicated to diffusion video generation and editing. Its distinguishing feature lies in an architecture that separates reasoning from rendering. A first module, a semantic planner leveraging a multimodal language model (MLLM), interprets text, image, or video inputs and directly predicts the targets to be produced, while a second module, the renderer, then generates the actual images.

To maintain temporal and spatial consistency between video segments, the team introduces a technique called Segment-Aware 3D RoPE. This preliminary planning logic, presented as a form of latent reasoning, targets the most complex tasks and better generalization beyond cases seen during training. Bernini unifies several use cases: prompt-driven video editing, which allows modifying style, scenery, weather, or objects; reference image-guided editing; inserting an image or video into an existing scene; and generating a video from one to five reference images, useful for maintaining a character or object from one shot to another.

The tool aligns with the same trend as Gemini Omni, Google's recently unveiled video generation and editing model, from which it primarily distinguishes itself by its openness: whereas Google's offering remains proprietary and tied to its ecosystem, Bernini's code and models are distributed under an Apache-2.0 license on GitHub and Hugging Face, with the renderer's weights having been recently open-sourced.