From silent editing to multi-track soundtrack with Instant Soundtracks

ElevenCreative launches Instant Soundtracks, a tool that generates synchronized music and sound effects from a video, complete with multitrack editing.

A finished edit, defined transitions, but no soundtrack yet. With Instant Soundtracks, ElevenCreative offers the ability to automatically generate original music and sound effects from the footage of an existing sequence. The system analyzes the edit, identifies key moments to attach sound to, and places the generated elements on a timeline. Creators can then tweak each track in the music editor without having to restart the entire generation process. Video becomes the starting point for sound creation

Instant Soundtracks works from a video imported into ElevenCreative. Unlike music generation based solely on a text prompt, the system directly uses the visual information from the footage.

The model analyzes the frames to build a musical accompaniment and identify events that might require sound effects.

A logo reveal can thus receive a transition effect, while a scene change can be accompanied by a sound impact. The effects are generated and then automatically placed at the corresponding moments.

The process therefore combines two operations: composing original music tailored to the edit and creating sound effects synchronized with the detected actions.

ElevenCreative presents this combined generation as a way to produce an initial sound pass without having to manually select each effect from a library or determine its placement. Music that follows changes in mood

The new feature also takes into account narrative progression and mood variations between scenes.

An energetic sequence might be paired with electronic or R&B music, before shifting to acoustic instrumentation as a shot becomes quieter.

ElevenCreative states it has improved the precision of these changes, particularly on short videos where transitions can follow one another quickly.

The music is built based on the visual content, but also on the pacing of the cuts and the events present in the edit.

This approach can be used for commercials, promotional videos, or social media content, where sequences frequently alternate between different shots.

However, musical relevance remains a creative judgment. The automatic analysis of a scene does not guarantee that the result will match the director’s or editor’s exact intent. Using the original sound as a reference

Instant Soundtracks can also take into account the audio track already present in a video.

The model analyzes certain characteristics of this audio, including musical genre, intensity variations, tempo, or the instruments used.

This information is then used to guide the new composition.

A user can thus preserve the general musical characteristics of a sequence while requesting the creation of a new sound accompaniment.

The feature is not simply about replacing existing music with an unrelated track. It allows the generation to be guided by the properties of the original sound.

ElevenCreative does not elaborate, in its announcement, on the specifics regarding dialogue preservation or the separation of various sources present in the initial recording. Separated sound effects in the editor

After generation, the soundtrack can be modified in ElevenCreative's Music Video Editor.

Sound effects appear as separate tracks from the music. The user can review them individually, change their placement, or add new ones.

They can also swap out the music, adjust specific passages, and tweak transitions before exporting.

This separation makes it possible to keep the elements that are already satisfactory without having to generate a complete new soundtrack for every correction.

For instance, an effect can be moved to better align with a cut, while the music remains unchanged.

The tool thus retains a manual editing step after automatic generation, offering distinct control over the musical composition and sound effects. Processing tailored to short-form content

ElevenCreative particularly highlights short videos, especially ad content and assets intended for social platforms.

In these formats, a few frames can be enough to mark a transition, a scene change, or a product reveal.

Instant Soundtracks seeks to identify these events to place effects at the right moment and adapt the music to the rhythm of the edit.

The company promises more precise synchronization on cuts, transitions, and on-screen actions.

This enhancement also applies to mood shifts, preventing the same musical intent from persisting when the video content changes significantly.

However, ElevenCreative does not provide benchmark figures to measure the temporal accuracy of these effects or to compare this version with other video-to-audio generation systems. An API with separate track exports

Instant Soundtracks is also integrated into the ElevenCreative Studio API, for developers and teams looking to automate sound creation.

Three additional parameters have been announced:

`addsfx` allows adding sound effects generated from the video.

`separatetracks` allows retrieving the music and sound effects as distinct files.

`useaudioasreference` enables the use of the original audio as a reference for the new composition.

When `separatetracks` is enabled, the API returns a ZIP archive containing the separated audio stems.

These files can then be integrated into an editing workflow, a mixing software, or another production environment.

The API supports up to ten videos per request, up to a combined limit of 200 MB and ten minutes of video.

These caps apply to files submitted for generation. They do not specify the maximum duration of a single composition or any restrictions specific to individual subscription tiers. Available in ElevenCreative and ElevenMusic

Instant Soundtracks is available in ElevenCreative from the music editor, where generated effects can be edited separately.

The feature is also offered in Creative Engine to accompany ad assets, as well as on the ElevenMusic platform and in the ElevenCreative Studio API.

Users can import an existing video, run the generation for its musical accompaniment and effects, and then tweak the result prior to export.

The