Perplexity Launches WANDR Benchmark With 500 Research Tasks to Evaluate AI Agent Capabilities

Perplexity AI launched WANDR (Wide ANd Deep Research), an open-source benchmark on July 15 to measure how effectively AI systems perform large-scale research tasks requiring broad information discovery and detailed evidence verification. The benchmark contains 500 realistic data-gathering tasks modeled on professional knowledge work, including market analysis, due diligence, literature reviews, and competitive intelligence, backed by over 170,000 source-verified records.

Unlike traditional benchmarks that evaluate single answers, WANDR measures an AI system's ability to identify large numbers of relevant entities and verify each result with supporting evidence. In evaluation of six production AI systems, Perplexity's Search as Code platform achieved the highest performance with a soft F1 score of 0.363 and hard F1 score of 0.133, followed by Anthropic at 0.249 and 0.072, indicating that wide-scale, evidence-backed research remains far from fully automated.

Disclaimer: The information on this page may come from third-party sources and is for reference only. It does not represent the views or opinions of Gate and does not constitute any financial, investment, or legal advice. Virtual asset trading involves high risk. Please do not rely solely on the information on this page when making decisions. For details, see the Disclaimer.
Comment
0/400
No comments