The interface no longer resizes, it rebuilds the scene

Runway Labs is experimenting with video interfaces that can recompose a scene, its objects, and its interactions in real time as the window changes shape.

A narrow room does not simply display a cropped version of the same image. Furniture moves closer together, volumes are rearranged, and a straight staircase becomes a spiral staircase so it can fit inside the available space. When the window expands again, the scene returns to a more open composition.

Runway Labs is experimenting with generative video interfaces that no longer treat screen dimensions as a constraint applied after the content has been created. Width and height become direct model inputs, much like a click, a drag, or another user action.

The prototype presented on September 21, 2026, renders a new composition when the window is resized. It does not merely enlarge, reduce, or crop the previous version. Instead, the system reinterprets the content of the scene to make different use of the available area.

This goes beyond placing a video inside a responsive frame. On a conventional website, responsive design largely relies on predefined rules. A grid may reduce its number of columns, a menu may collapse, and elements may move beneath one another as the display becomes narrower. The browser rearranges components that retain their identity and function.

In Runway’s demonstrations, the representation of the objects can change as well. A piece of furniture may adopt different proportions, a passage can widen, and a structure can take on another shape. The composition is not only rearranged. Parts of it are recreated.

The staircase example captures this approach particularly well. When the scene becomes too narrow to preserve a straight staircase, the system does not simply shrink it. It turns it into a spiral staircase with a smaller footprint. The object keeps its purpose, connecting two levels, while changing form to satisfy a new spatial constraint.

That behavior requires the model to establish relationships between elements in the scene. It must recognize that one object is in front of another, that furniture rests on the floor, or that a passage needs enough room to remain visually credible. Runway describes this as an understanding of the scene in three dimensions.

That description still requires caution. The company has not published a depth map, geometric mesh, or scene graph explicitly describing the position and function of each object. The videos show apparent spatial consistency, but they do not establish that the prototype maintains a measurable three-dimensional representation comparable to one found in design software or a game engine.

The system could produce a convincing scene by learning common visual relationships between volumes without precisely reconstructing the underlying space. A staircase that appears to fit inside a room does not guarantee that its steps remain consistent, that its incline would be practical, or that its structure would hold up when viewed from several angles.

Runway Labs also says objects settle into the available space instead of overlapping or spilling out of the frame. This moves the prototype closer to a dynamic composition tool. In a conventional interface, alignment, overflow, and hierarchy constraints are written by designers and developers. Here, some of those decisions are produced at the moment of interaction.

The demonstrations extend beyond resizing. They show interactions taking place directly inside the scenes, without relying only on fixed menus or predefined views. A click or drag can alter the environment, move an element, or trigger a visual transformation.

The displayed surface becomes both the content and the interface. Instead of selecting an action from a toolbar and viewing the result somewhere else, the user manipulates the representation itself. The scene responds like an interactive environment.

This approach could fit product configurators, real-estate tours, cultural experiences, or interior-design applications. A room could reorganize itself according to the screen format, requested furniture, or selected area. A retailer could show a product in several contexts without manually preparing every composition.

It also suggests another direction for creative tools. The same project could be explored in vertical, square, or panoramic formats without simply cropping an initial creation. The system would seek a distinct composition for each proportion and could transform certain elements to preserve the broader intent.

That flexibility may conflict with the level of control designers expect. In a deterministic interface, the same screen should normally produce the same result as long as the data and rules remain unchanged. A generated composition can introduce unexpected variation in object shapes, colors, placement, or visual importance.

Moving between formats could therefore alter more than intended. A secondary element could become dominant, a visual control could change appearance, or an object used as a landmark could disappear. That instability may support exploration in an artistic prototype. In a banking, medical, or government interface, it would become an immediate problem.

The demonstrations do not indicate whether a scene returns to exactly the same state when the window returns to its previous dimensions. They also do not show how the system preserves a visual identity, hierarchy, or essential placement through a long sequence of transformations.

This directly affects quality assurance. Current interfaces are tested across a limited range of formats, states, and browsers. A continuously reconstructed composition creates a much larger number of possible outcomes. It becomes more difficult to verify in advance that every label remains readable, every control remains accessible, and no critical element is hidden.

Accessibility presents another major limitation. A traditional web interface has a structure that assistive technologies can navigate. Headings, buttons, fields, and links can be identified independently of their visual appearance. A generated video scene does not automatically provide that structure.

To become