Speaking and working at the same time in Higgs Realtime Duplex

Boson AI’s Higgs Realtime Duplex preview combines voice, video, and background tasks. Its API remains waitlisted.

In a meeting demonstration, Boson AI describes an assistant that lays out an architecture, takes questions in the middle of a sentence, and updates a diagram. The work continues during the exchange, with images from a camera or a shared screen as input information. The animated face on the screen accompanies a conversation during which tasks can keep running.

The Higgs Realtime Duplex research preview is based on this simultaneous operation: receiving speech and images, producing a response, while also using tools. API access is announced for later, with a waitlist. The Higgs Realtime service already offered by the company remains a separate offering from this new preview. Listening while tools work

The user’s direct interlocutor is a 45-billion-parameter mixture-of-experts model. According to the described architecture, it can reason and initiate tool calls itself, without systematically handing these operations off to another model. For more difficult requests, it delegates to a larger model, to which it passes the shared context.

Results come back while the exchange continues. A search or a calculation can therefore progress while the user clarifies their request, or even changes the subject. Tools can also deliver partial results, from which the assistant begins to respond. The promise concerns continuity of the interaction during the wait, without necessarily shortening the duration of the delegated task.

“Full duplex” refers to this simultaneous receiving and sending. The streams are organized into 160-millisecond segments: at each step, the model reads the new inputs and prepares the next output. This duration describes the chunking of the streams, not a guaranteed delay between a complete question and its answer. Silences are also part of the information being processed. Choosing when to speak, and when to stay silent

The behaviors presented go beyond responding to a finished question. The model can briefly acknowledge, start responding before the end of a sentence, or stop when the user speaks again. The team also claims that it distinguishes remarks addressed to it from surrounding conversations.

A demonstration described in the announcement places it in a car: it waits while the driver places an order with an employee, then resumes its report. This sequence illustrates the intended behavior; it does not measure its reliability in all situations involving noise, overlapping voices, or multi-party conversation.

The voice and the face are rendered by two generation components. A short audio excerpt serves as the voice reference, while a still image defines the face. According to Boson AI, these references are provided at deployment, without retraining. The face is then generated throughout the exchange, while preserving recent visual context to maintain continuity. Capabilities trained together, mixed results

Development starts from a model already pretrained on text. The team continues training it on about 2,000 billion tokens combining text, audio, and video. It then trains several specialized teacher models in different aspects of interaction, and transfers their capabilities into a single student model. Simulations are also used to produce targeted data on the errors observed.

In the published comparison, the score of 98.9 on Big Bench Audio is the highest in the table. The advantage does not carry over to all benchmarks: on Full-Duplex-Bench v1.5, the reported average is 75.5, compared with 77.8 for Thinking Machines Lab’s Interaction Small and 80.1 for GPT-Live-1.

The competing figures come from public reports and rankings, which the team says it reproduced by more than half. The reasoning configurations vary depending on the table entries. These comparisons published by Boson AI therefore do not constitute an independent evaluation of the preview. Compressing streams to extend exchanges

A model that continuously processes the conversation accumulates information even when no one is speaking. To limit this volume, the system compresses audio and visual streams. The video input notably preserves a camera image every six slices, or about one second. Audio and visual streams together would account for some 2,600 tokens per minute.

The total budget announced is rather 2,600 to 5,000 tokens per minute once reasoning, tool calls, and their results are added. This estimate already assumes progressive compression of the work traces; it does not correspond to a published API rate for Duplex.

Historical management typically retains the last five minutes of audio and visual streams together with the condensed working text, or about 13,000 to 25,000 tokens depending on activity. Older history remains stored as text, in addition to this recent window.