A compelling voice isn’t enough to succeed on a call

More than 50 models and voice platforms are compared in Cekura Bench, which evaluates conversation quality, reliability, speed, and cost.

Between simulated scenarios and recorded human interactions, the rankings of voice agents change. In Cekura Bench, Phonic v1 ranks first in a reliability test on predefined tasks. On calls based on recorded human voices, first place goes to Grok Voice 3.0. These results describe different capabilities, under distinct testing conditions.

The new benchmark suite brings together more than 50 models, according to its announcement, and also compares complete platforms. It covers transcription, text-to-speech, and systems capable of directly receiving and producing speech. The tables separate errors, response time, cost, and the ability to complete a task. Succeeding three times in the same scenario

For the agent performance test, a simulated caller puts the systems through the same 82 scenarios, each repeated three times. They involve appointment scheduling and requests related to Medicare, the U.S. health insurance program. The model changes, but the reference agent, its instructions, and its tools remain the same. An execution succeeds if the expected outcome is achieved and the recorded data are correct.

The pass³ score counts scenarios that succeed in all three runs. Phonic v1’s 90.2% therefore indicates repeated success, not the proportion of responses that are pleasant to hear. For complete platforms, a separate ranking places PolyAI at 87.8% on this measure. The configurations of these platforms include several components: their result should not be conflated with that of a single voice model.

The Real Human Calls test, for its part, uses 70 calls based on recordings of real people, presented three times to each model. It covers banking and healthcare, mainly in English, with calls in Spanish and Hindi. No French sample appears there. Hesitations, code-switching, and turn-taking are part of the evaluation.

On this protocol, Grok Voice 3.0 tops the Elo ranking and reaches 42.9% pass³, compared with 31.4% for Phonic-1. Fully completing the task is yet another metric: it reaches 34.8% for Grok Voice 3.0. A call can thus exceed the quality threshold set by the benchmark without every requested step being completed. The best transcriber also depends on the recording

The speech recognition results separate public clips and phone conversations. Word error rate, or WER, counts substitutions, omissions, and insertions relative to a reference transcription. The lower it is, the better the result on the corpus in question.

On the Pipecat clips, AssemblyAI’s Universal 3.6 Pro achieves the lowest rate, at 1.77%. On the Ocular phone recordings, Fish Audio Transcribe Stream 1 takes the lead with 2.78%. It also posts the best overall published result, at 2.21%, ahead of Nemotron 3.5 Transcribe’s 2.28%.

The site notes that several models remain within the error margin of the leaders. The phone corpus is also limited to eight recordings: a first-place finish on this set does not establish superiority across all conversations. Milliseconds do not all measure the same wait

The text-to-speech comparison covers 104 agent phrases. For Inworld TTS-2 Flash, the reported 46 ms correspond to audio startup when the text is sent in full. This measure does not include the work needed to understand a request and draft the assistant’s response.

When text arrives word by word, Gradium TTS begins speaking, on average, 64 ms before the last word is received. The negative value comes from the chosen reference point: the speech starts while the sentence is still coming in.

Soniox STT-RT v5’s 38 ms indicate something else again: the time to finalize the text after the speech ends. This figure describes neither the appearance of the first transcribed words nor the response time of a complete agent. Results that can be consulted, independence that still needs to be clarified

The governance statement says that Cekura sells testing and monitoring tools for voice agents. Some evaluated providers may be its clients or partners. The company designs and runs the benchmarks itself.

The code, scenarios, and some recordings are published to make the results verifiable. The documentation nevertheless specifies that the judge instructions used to evaluate the expected outcome and some scoring rubrics are not yet public. The corresponding scores therefore cannot be reproduced exactly from the published materials alone.