Voice AI learns to interpret its texts before speaking them

Inworld AI enhances Realtime TTS-2 with a reasoning system capable of adapting emotion, intonation, and rhythm to the context of each line.

A single phrase can express joy, anger, or deep disappointment. Yet text-to-speech models have long required additional instructions to distinguish these intentions. With Reasoning for Realtime TTS-2, Inworld AI introduces a reasoning step prior to audio generation. The system analyzes the text and its context, then automatically determines how to pronounce it. This evolution comes alongside improvements in speed, voice fidelity, and stability during extended conversations. Understanding an intention before producing a voice

The example chosen by Inworld relies on a intentionally ambiguous phrase: "I can't believe you did that." Depending on the situation, it can express enthusiasm facing a surprise or the pain caused by a betrayal. Without additional information, a text-to-speech system risks producing a neutral interpretation, regardless of the emotion the scene should convey.

The new operation of Realtime TTS-2 adds an interpretation step to the synthesis request. A specialized language model examines the phrase as well as the contextual information transmitted by the application. It then drafts a vocal performance direction specifying the desired emotion, tone, intensity, or rhythm. The synthesis engine uses this instruction to produce the audio recording.

In the demonstration presented by the company, an unexpected reunion between friends leads to a warm and enthusiastic, almost smiling voice. The same phrase, placed in the context of a romantic betrayal, is interpreted with restraint, pain, and disbelief.

Context intervenes solely in the preparation of the interpretation: it is not spoken in the final recording. The generated direction is also returned in the API response, allowing developers to monitor how the system interpreted their request. The end of hand-programmed emotions?

Until now, voice applications could rely on explicit tags like "happy", "sad", or "angry" to guide synthesis. These indications had to be added manually or generated by the language model responsible for the conversation.

The operation detailed by Inworld replaces this selection from predefined categories with a freeform prompt capable of combining multiple nuances. A voice can thus be described as calm, slightly hurt, and marked by contained disbelief, rather than simply classified in the "sad" category.

The benefit for developers also lies in the simplification of instructions. It is no longer essential to maintain a long list of rules explaining to the conversational model when to use each emotion. The latter can focus on text generation, while the voice layer determines its interpretation.

This automation does not take away user control. Explicit directions and emotional tags inserted directly into the text retain priority over decisions made by Reasoning. The system can therefore interpret a line independently, while respecting an artistic direction imposed by the developer. Up to 13% faster generation

The update does not solely concern expressiveness. The Realtime TTS-2 engine also benefits from performance optimizations. According to Inworld's internal measurements, short phrases are now generated approximately 13% faster than on the previous version, compared to 9% for medium-length sequences.

These gains pertain to voice generation duration, not the total time required for a conversational application to respond. The latter also depends on end-of-speech detection, the language model, network exchanges, and audio playback.

The results from the independent benchmark by Cekura shed another light on the performance of the TTS-2 family. In tests published before this update, Realtime TTS-2 Flash achieves a median latency of 46 milliseconds to first audible sound, with the entire text provided from the start. It holds first place among 27 models evaluated in this specific category.

For its part, Realtime TTS-2 correctly pronounces 93.4% of elements considered difficult in Cekura's protocol, notably numbers, abbreviations, codes, addresses, and certain medical terms. The evaluation relies on 104 phrases spoken three times, then analyzed using automated transcriptions.

These results do not directly measure the emotional quality of the new Reasoning system. They concern the versions tested before its introduction and should not be interpreted as an independent validation of this feature. More stable voices, even in long conversations

Adjustments also concern the reproduction of custom voices. Inworld states it has improved fidelity to reference recordings in Stable mode, particularly for voices whose characteristics were difficult to reproduce.

The company also addressed voice drift observed during long sessions. Over the course of exchanges, a synthetic voice can progressively alter its timbre or sonic identity. The new version seeks to limit these variations in order to preserve perceptible continuity across different turns.

Other fixes address pronunciation accuracy. Certain words could be omitted in extended passages, while final consonants lost clarity. Inworld notably mentions the sounds corresponding to the letters p, t, and k, whose rendering is important for intelligibility.

Creative mode also benefits from adjustments designed to make interpretations more natural. Developers thus have two complementary approaches: prioritize the stability of a voice in conversational applications or seek greater expressiveness for characters and narrative content. A feature accessible without changing models

Reasoning is available with Realtime TTS-2 via Inworld's text-to-speech API. Its activation requires adding a parameter to the request, optionally accompanied by contextual information. No additional service is necessary to generate the interpretation direction.

The system currently works with standard synthesis, streaming synthesis, and asynchronous requests. These modes notably allow for the production of voice messages, narratives, audiobooks, or prepared lines for video game characters.

Certain integrations remain under