Voice AI learns to stay quiet, listen, and work at the same time

With MOSS-Live, MOSS introduces a real-time voice AI that can listen while speaking, handle interruptions, and carry out complex tasks in parallel.

A pause to think, a simple “mm-hmm” to show you're listening, a comment directed at someone else, or an interruption in the middle of a sentence. These small details shape human conversations, yet they remain surprisingly difficult for voice assistants to interpret. Some interrupt too early, others continue speaking even when someone tries to stop them, while complex tasks still regularly force conversations to pause until processing is complete. With MOSS-Live, the MOSS team is addressing these limitations through a voice system designed to listen, speak, perceive its surroundings, and carry out certain operations simultaneously.

Introduced in early November 2026, MOSS-Live uses a full-duplex architecture in which listening and speaking are no longer strictly sequential operations. The assistant can continue receiving information while responding, determine whether someone is actually addressing it, and delegate research to another model without interrupting the conversation. MOSS also highlights simultaneous translation, live video understanding, and proactive assistance. The objective isn't simply to generate a more natural-sounding voice, but to make interactions flexible enough for an assistant to participate in an ongoing activity rather than function as a separate tool that must be addressed each time. The Problem With Assistants That Wait Their Turn

Most interactions with voice AI still follow a relatively rigid pattern. The user makes a request, the system determines that the user has finished speaking, and then generates a response. This works well enough for short questions, but becomes less satisfactory when conversations involve hesitation, changes of direction, or multiple participants.

MOSS-Live attempts to move beyond this constraint through continuous two-way interaction. While producing a spoken response, the system keeps listening and analyzing incoming signals. In particular, it must distinguish a genuine attempt to take over the conversation from a simple acknowledgment indicating that the user is following along.

The distinction may seem minor. But when someone says “yeah,” “right,” or “I see” during an explanation, they don't necessarily want to interrupt the speaker. An assistant that stops every time it hears one of these expressions turns an ordinary conversation into a series of artificial interruptions. Conversely, an assistant that ignores all of them risks continuing to speak when the user genuinely wants to change the subject.

MOSS-Live also claims to handle pauses more intelligently. When someone hesitates while searching for the right words, the system can wait rather than immediately assuming the speaker has finished. This requires interpreting conversational context rather than simply detecting silence in the audio signal. Understanding an Interruption Without Losing the Thread

Another challenge involves resuming an interrupted conversation. In many systems, interrupting a response effectively abandons the content being generated. The assistant must then reformulate its answer or resume an explanation from an approximate understanding of where it stopped.

MOSS-Live introduces awareness of how far its spoken response has actually progressed. The system tracks the words it has already said and can resume an explanation from the point where it was interrupted. This distinction matters because the text a model intends to produce doesn't necessarily match what the user has already heard.

The team also claims the model can recognize speech that isn't directed at it. During a meeting, several participants might talk among themselves while an assistant carries out a task. The system must determine whether a comment requires its participation or simply belongs to a separate conversation.

Managing conversational turns presents a different technical challenge from traditional speech recognition. Understanding the words being spoken isn't enough. The system must also interpret their role in the exchange, identify who is being addressed, and decide whether to respond, wait, or continue speaking. Two Models to Keep Conversations Going While Research Continues

The architecture presented by MOSS separates a foreground model responsible for immediate interaction from a background model dedicated to more complex operations.

The foreground model handles speech, audiovisual perception, and the decisions required to maintain conversational rhythm. It must react quickly, listen while speaking, and interpret new information coming from the environment. The background model takes over when a request requires deeper reasoning, tool use, or a sequence of operations that would take too long to execute directly within the live voice interaction.

Both components share information about context, ongoing tasks, and the current state of the conversation. The assistant can therefore continue talking with the user while research progresses, then deliver the results at an appropriate moment.

This arrangement addresses an important difficulty in real-time AI: the speed requirements for a spoken reaction are not the same as those for complex research. A system responding to an interruption must make a decision almost immediately. An operation involving several tools, by contrast, may require multiple processing stages whose duration is difficult to predict.

By separating these functions, MOSS aims to prevent the time required for a complex task from turning into a prolonged silence. Conversation becomes a continuous interface to multiple activities rather than an exchange suspended whenever a new operation begins. A Shanghai Demonstration Combining Tourism and Live Research

To illustrate the architecture, MOSS presents several usage scenarios. In a demonstration involving a visit to Shanghai, the assistant organizes an itinerary, identifies the Peace Hotel through a live video feed, and provides information about its history. The visitor can then ask about room rates or dining options while the