Violin, an open-source end-to-end video translation pipeline

Violin is an open-source video translation pipeline by Shang Zhu, Kevin Lin, and James Zou, using Together AI, Whisper V3, Deepseek V4 Pro, and Cartesia.

Violin is a fully open-source video translation tool, developed by Shang Zhu, Kevin Qinghong Lin (Oxford), and James Zou, powered by Together AI's API. The premise behind the project: according to a study cited by the authors, 66% of videos from the 250 most-followed YouTube channels are in English, compared to 15% in Spanish, which keeps a large portion of global audiences out of reach for high-value informational content.

The pipeline consists of three steps. Speech recognition relies on Whisper V3 large hosted by Together, which produces a timestamped multilingual transcription. Translation is then handled by Deepseek V4 Pro by default, with the option to inject a list of custom rules to preserve fidelity and accuracy. Speech synthesis is based on Cartesia Sonic 3, which covers a wide range of native voices, from Korean to Chinese, including Dutch and Italian, and accepts a natural language description to adjust the timbre. Voice cloning is excluded: the generated voice remains distinct from the original speaker and is superimposed at low volume over the original track.

A video discussion module completes the set, powered by the Qwen3.5-397B-A17B vision-language model, which answers questions rooted in the video's audio and images. Violin comes in three formats: a no-code web application, a command-line interface for batch processing, and an agent skill that can be integrated into major frameworks. The entire code is published under an MIT license on GitHub, and the online demo deletes uploaded videos after twenty-four hours.