One AI keeps the conversation going while another searches the web

Built for a virtual character, Shizuku-Duplex can listen and speak at the same time in Japanese, even while a second model looks up information.

Two people are planning a trip to Kyoto on a budget of ¥100,000. In an example published by Shizuku AI, the character responds to the idea of trying matcha sweets, then says it’ll look up a few options. The search takes 7.1 seconds. During that time, the voice model keeps listening and can still respond. When the results come in, it picks up the thread and explains that lodging will likely be the biggest expense for a three-night stay.

Two models share the work. One handles the conversation. When it needs more information, it sends a request to a second model with access to tools such as web search. The search still takes time, but the person can clarify what they’re looking for while it runs. Shizuku AI draws on an approach described by Thinking Machines Lab, which separates real-time conversation from tasks running in the background. Listening, speaking, and knowing when to stop

The model processes incoming audio in 80-millisecond chunks and decides at each step whether to speak or stay quiet. It generates the text of what it’s about to say slightly ahead of the audio while continuing to process sound from the microphone. That’s what makes the conversation full-duplex. The character can say “mm-hmm” while the other person is speaking. If they cut in mid-response, the model can stop talking. The same system also generates movement instructions for the Live2D avatar.

Full-duplex voice interaction in Japanese had already been demonstrated with J-Moshi, presented at Interspeech in 2025. For its virtual character, Shizuku AI also addresses what happens as conversations grow longer. About every five minutes, the system replaces older parts of the conversation with a summary and keeps the most recent exchanges intact. The conversation can continue with a shorter history. Earlier details remain available to the model if the summary captures them. Interruptions are still hard to interpret

In Shizuku AI’s internal evaluation in Japanese, Shizuku-Duplex stopped speaking within two seconds in 89% of cases when the user cut in, compared with 82% for GPT-Live-1. It also responded to background speech in 75% of cases. Yielding when someone wants to speak and recognizing when a voice is talking to someone else are two different challenges.

The 80 milliseconds refer to how often the model processes audio. In the same evaluation, the median time from the end of the user’s speech to the start of a reply was 0.57 seconds, compared with 0.53 for GPT-Live-1, with network delay excluded in both cases. On questions that unfolded over several turns, Shizuku-Duplex answered correctly 79% of the time, compared with 95% for OpenAI’s model. That test measures what the character says separately from when it speaks.