To search wide and dig deep, no one can do it yet

Perplexity releases WANDR to test search agents. Between 57% and 87% of the cited sources do not back up the claims made by these tools.

Perplexity is launching WANDR, an open-source benchmark for research agents, complete with its evaluation toolkit. Five hundred tasks replicate the work already entrusted to them: competitive mapping, due diligence, literature reviews, market analysis, and candidate sourcing.

The difficulty arises from an intersection. Agents must go broad—discovering an extensive and often open-ended set of entities meeting a specific criterion—and go deep, supporting every claim about each of them with evidence. For instance, one published task requires seventy US companies that announced the appointment of a CEO or CFO within a two-month window, with an authoritative page for each, plus a second piece of evidence establishing its status as a publicly traded company. That is one hundred and forty verifiable records for a single task; the median task in the benchmark requires two hundred and forty-five.

The evaluation does not rely on a static answer key, which ages poorly as answers change over time. Instead, each claim is judged against the evidence the agent itself cites: the grader reloads the page, verifies that it is usable, confirms that the selected snippets actually appear on it, and checks that they fully support all the stated requirements.

The verdict is harsh for all tested systems, and the nature of the failures is surprising. Finding a usable page is hardly an issue: with one exception, less than a tenth of the submitted pages prove unusable. Discovery forms an initial bottleneck, as a portion of the requested cohort is never gathered. However, the dominant loss comes from the evidence itself. Between one-third and two-thirds of the pages fail to satisfy at least one substantive requirement, and most notably, between 57% and 87% of the cited snippets are insufficient to support everything the record claims. The agent does produce a source, but it does not prove its assertion.

A caveat is in order: the benchmark is published by Perplexity, which also subjects its own research system to it. The tasks, evaluator, and technical report are all publicly accessible.