When a voice assistant listens to itself speak, speech recognition loses the thread

Soniox’s STT v6 version distinguishes the assistant’s voice from the user’s and leverages conversation context in more than 60 languages.

A voice assistant that answers its own question, cuts off its sentence because it hears its own voice in the microphone, or mishears an answer as simple as “the second one.” These malfunctions remain common in conversational interfaces, especially when the speaker and microphone operate simultaneously. To reduce them, Soniox STT v6 introduces a speech recognition method that takes into account not only what the user says, but also the words spoken by the assistant.

The innovation is based on two complementary functions, Agent text input and Agent voice echo cancellation. The first sends the transcription engine the text that the assistant is currently uttering. The second uses that information to distinguish the user’s speech from the speech that comes back into the microphone. Presented in the technical post dedicated to STT v6, they are intended for conversational agents capable of listening while they speak, without confusing a real interruption with their own voice. Understanding an answer from the question asked

Speech recognition engines usually receive the sound captured by the microphone, without necessarily knowing what the assistant has just said. This separation deprives the system of information that is nonetheless essential to understanding certain short answers.

When an agent offers two destinations, two time slots, or several products, an answer of a few words takes its meaning from the previous question. Without that context, a proper noun can be confused with a phonetically similar expression. The problem becomes especially acute with numbers, technical terms, or place names.

The Agent text input function sends the assistant’s text to the recognition engine as it is being spoken. The user’s words and the agent’s words are thus taken into account within the same conversational context, even if only the former appear in the final transcript.

This integration requires precise synchronization. The documentation recommends sending text fragments about 500 milliseconds to one second before they are actually spoken, rather than as soon as they are generated by the language model. If an interruption occurs, transmission must stop at the last passage that was actually spoken. The context thus remains consistent with what the user actually heard. Eliminating echo without preventing interruptions

The second problem concerns acoustic feedback. When an assistant speaks on a laptop, in a car, or through a speaker, part of its voice may be recaptured by the microphone. The transcription system may then interpret it as a new user intervention, triggering a response or cutting off speech synthesis prematurely.

The Agent voice echo cancellation function uses the text transmitted by the assistant to recognize elements of its own voice present in the audio signal. It seeks to exclude them from the transcription while continuing to identify the user’s speech, including when the user speaks before the end of an answer.

This technique complements traditional acoustic echo cancellation mechanisms, without replacing them. The Soniox documentation explicitly recommends keeping these audio processing steps, with the new function handling the residuals that escape them.

The goal is to make simultaneous exchanges less fragile. A user can interrupt a confirmation or correct information without the assistant interpreting its own words as a new command. The provider claims a reduction in erroneous interruptions and clarification requests, but does not publish, in its announcement, detailed comparative measurements allowing these improvements to be quantified. Dictation, meetings, and translation in more than 60 languages

The changes are not limited to voice assistants. The sixth generation of the model also brings changes to dictation, meeting transcription, and real-time speech translation, with support for more than 60 languages.

For dictation, the work focuses especially on recognizing amounts, units, dates, alphanumeric identifiers, proper nouns, and specialized vocabulary. These elements are often difficult to reproduce faithfully, especially when information is dictated quickly or in a noisy environment. Developers can provide product names, professional terms, or other contextual elements to improve transcription.

Speaker separation, called speaker diarization, also benefits from training that includes meetings with interruptions, rapid changes of speaker, overlapping voices, and varied recording conditions. The announced improvements also concern language identification and end-of-speech detection, two important functions for preventing an assistant from responding too early or attributing a sentence to the wrong participant.

In translation, the engine dynamically adjusts the amount of speech collected before producing the translated text. This context management is aimed in particular at languages whose grammatical structure sometimes requires waiting until the end of a clause to render its meaning correctly. Multilingual conversations, including those that switch languages in the middle of a sentence, are also supported without having to select a separate model for each language.

All of these functions are announced in a unified voice architecture, but the claimed performance remains that of the provider. No complete comparative protocol with quantified results by language or by use case accompanies this publication. Two versions accessible via API

The model documentation confirms the availability of STT v6 under two references: `stt-rt-v6` for streaming transcription and `stt-async-v6` for delayed processing of recordings. The new functions for contextualizing the agent’s speech and filtering its voice apply to the real-time version. The other transcription improvements are distributed between the two modes according to their respective features.

The transition from the previous generation mainly requires changing the model identifier in API requests. STT v5 versions remain temporarily available, but their