Speaker tracking goes real time with Scribe v3 Live

Two transcription models target recordings and live conversations, with reported improvements in word accuracy and speaker identification.

In a meeting, a transcript can contain the right words while attributing them to the wrong person. On the phone, a hesitation can be mistaken for the end of an answer. These are among the challenges addressed by Scribe v3 and Scribe v3 Live, ElevenLabs’ speech-to-text models.

The first processes recordings; the second transcribes conversations in real time and adds speaker tracking, known as diarization. Both are accessible through ElevenAPI. The Live version is also available in ElevenAgents, while Scribe v3 becomes the default transcription model in the application. Different gains depending on what is measured

For recordings, the company reports a 27% relative reduction in word error rate compared with Scribe v2. This metric, known as WER, accounts for words substituted, omitted, or added compared with a reference transcript.

For real-time transcription, the 43% figure highlighted in the announcement refers to a different metric: semantic word error rate, which focuses on differences that affect meaning. The reduction in conventional word error rate is 31% compared with Scribe v2 Realtime. These results therefore mean neither a 43-percentage-point increase in accuracy nor an equivalent reduction across all types of errors.

The launch post places these comparisons on an internal version of FLEURS covering 102 languages, with each language weighted equally in the average. This benchmark uses short clips of clean, read speech. On its own, it cannot establish performance in a noisy meeting. A second internal dataset contains meetings, calls, and voice notes in 16 languages; the company says it plans to release it. Attributing each passage to the right person

On recordings involving at least three people, diarization errors reportedly fell by 17% compared with Scribe v2, according to the company’s evaluations.

For the new real-time diarization feature, the reported error rate is 3.1% on the English portion of the conversational dataset, compared with 5.9% to 7.3% for the other real-time models tested. However, the evaluation aligns transcripts with a reference before scoring them to isolate whether words are assigned to the correct speakers.

This result measures a specific task. It does not capture every error in a complete transcript and cannot be directly compared with the scores obtained for recorded audio. A pause is no longer the only signal that a turn has ended

For voice agents, the Live version uses conversational context to estimate whether someone has finished speaking. The goal is to better distinguish a completed answer from a pause in the middle of a number or sentence.

The documentation on transcription strategies specifies that this feature must be explicitly enabled in the API. Simply switching models is not enough. Silence detection also remains active: context provides an additional signal for the decision.

Vocabulary customization accompanies these changes. For recordings, Scribe v3 accepts up to 1,000 terms supplied in advance, such as proper names, acronyms, or jargon. Instructions written in plain English can also apply formatting rules to the text, such as using a consistent date format. The Live version has its own library of specialized terms. Early results for Dutch postal codes

In trials involving energy company Eneco’s voice agent, Dutch postal codes were reportedly recognized correctly on the first attempt 40% more often. For street numbers, the reported improvement was 15%.

These figures concern first-attempt success for specific information, rather than the model’s overall accuracy. The launch post does not provide baseline success rates, the number of calls evaluated, or a detailed methodology. This example illustrates a concrete use case without establishing that the same gains apply across all calls.