Translating while the other speaks, Tencent reduces the delay to three seconds

Tencent launches Hy-SimulMT, a Chinese-English simultaneous translation model featuring voice cloning and a measured lag of about three seconds.

A three-second lag to translate an ongoing speech, without waiting for the speaker to finish their sentence. This is the performance announced by Tencent Hunyuan for Hy-SimulMT, its simultaneous translation model between Chinese and English, whose weights have just been made public. The system processes a continuous audio stream, produces a translation on the fly, and can deliver the spoken output while preserving certain characteristics of the original voice.

Intended for meetings, conferences, and interviews, Hy-SimulMT has several variants accessible on Hugging Face. Tencent is also releasing the tools necessary for its deployment and continued training, along with a series of comparisons against competing solutions and professional interpreters. Translating before knowing the end of the sentence

Simultaneous translation poses a specific challenge for artificial intelligence models. To properly interpret a sentence, it is often preferable to know its complete structure. However, waiting for this information increases the delay between the original speech and its translation.

This difficulty is particularly noticeable when the two languages organize their sentences differently. A system must sometimes begin rendering a statement before having all the elements necessary to understand it.

Hy-SimulMT addresses this constraint by progressively analyzing the incoming audio. It generates the translation while the speaker continues talking, without imposing a systematic division into completed sentences.

The model is designed for bidirectional translation, from Chinese to English and from English to Chinese. Its family of checkpoints notably includes Hy-SimulMT-30B-A3B, a mixture-of-experts architecture containing approximately 30 billion parameters, 3 billion of which are activated per token. A three-second lag on long speeches

On the RealSI benchmark, dedicated in particular to evaluating simultaneous translation on long recordings, Tencent reports a lag of about three seconds. This is the lowest result among the systems presented in its comparison.

The measurement relies on StreamLAAL, an indicator that quantifies the accumulated delay between the original speech and its translation in a continuous stream. It should therefore not be confused with the delay preceding the very first audio output, nor with a guarantee of a systematic three-second response.

Translation quality is evaluated separately using the GEMBA score. According to the results provided by Tencent, Hy-SimulMT surpasses the open models compared, as well as four of the five commercial APIs included in this test.

The trade-off between speed and fidelity is the central element of this evaluation. Translating earlier reduces waiting time, but may also force the system to rephrase or correct a translation as new information arrives. Results that vary depending on the datasets

Tencent also tested its model on four benchmarks for shorter audio segment translation: FLEURS, CoVoST2, ACL6060, and RealSI-Sentence.

Hy-SimulMT scores 90.3 on FLEURS, 75.5 on CoVoST2, 81.2 on ACL6060, and 84.7 on RealSI-Sentence. On this last dataset, it posts the highest score among all systems in the table, ahead of Qwen3.8-LiveTranslate at 84.4.

Results are less uniform elsewhere. On FLEURS, Gemini3.5-LiveTranslate reaches 92.4, compared to 90.3 for Tencent. On CoVoST2, Qwen3.8-LiveTranslate achieves 82.1, ahead of Hy-SimulMT at 75.5.

These figures show that Hy-SimulMT does not dominate all evaluations. They also stem from the comparison protocol published by Tencent and do not, on their own, guarantee identical performance under all real-world conditions. A comparison with six professional interpreters

To complement the automated evaluations, the researchers compiled a set of 18 audio recordings, approximately six minutes each, covering ten domains, including medicine, law, science, education, industry, and culture.

Six professional interpreters participated in this comparison. Three held a level-1 Chinese CATTI certification, and three held a level-2 certification. For each level, Tencent retained the best result among the three participants.

The lab indicates that it achieved a translation quality comparable to that of the evaluated interpreters, often with a shorter delay.

The scope of the experiment remains limited by the sample size, the choice of recordings, and the number of participants. Tencent itself specifies that this comparison does not constitute a comprehensive evaluation of voice quality, voice similarity, or all professional interpreting scenarios. Preserving the voice of the translated speaker

Beyond text translation, Hy-SimulMT supports a real-time voice cloning feature. The goal is to retain recognizable characteristics of the original voice when the speech is rendered in the other language.

The published demonstrations notably include an English-to-Chinese sequence featuring three speakers, whose voices remain distinguishable in the generated output.

However, this feature relies on an additional speech synthesis service. The Hy-SimulMT repository provides the translation model and inference tools, but voice rendering with cloning requires separately configuring a compatible TTS engine.

The demonstration videos are pre-recorded playbacks. Their duration and playback speed therefore do not serve as a direct measurement of live conversation latency. An open model, complete with deployment tools

Tencent is distributing several versions of Hy-SimulMT, notably BF16, FP8, and GGUF variants. An Interleave variant can output both the source text and its translation, while the standard configuration focuses on the translated output.

The GitHub repository contains SimulInfer, intended for inference, and SimulTrain, dedicated to further training of the model. SimulInfer notably offers a web interface, audio file processing, and a WebSocket connection for real-time streams.

Deployment relies on environments compatible with vLLM. Tencent notes, however, that support