Just how smart are smart speakers? Loup Ventures has released the results of its annual Smart Speaker IQ Test, and the report found that Google Assistant is currently the brainiest of the bunch. In a test of 800 questions, Google’s voice platform answered 88% correctly. Apple’s Siri had 75% correct answers, Amazon’s Alexa had 73%, and Microsoft’s Cortana had 63%.
All of the voice assistants reported near-perfect comprehension. Google Assistant understood 100% of the queries, and the other three understood at least 99%. Nearly every misunderstood questions included a proper noun, such as the name of a town or local business.
For this iteration of the test, Loup Ventures tested the functions of voice assistants on smart speakers. The tests were conducted on second generation Amazon Echo, Google Home Mini, Apple HomePod, and Harman Kardon Invoke (which runs Cortana). It also examined third-party smart home devices.
Google maintained its lead in almost every category of questions. The exception was the command category, which Apple’s HomePod topped with an 85% share followed by 73% for the Google Home. The commerce category had the largest disparity across platforms with Google leading at 86% and Microsoft correctly answering just 31%. Loup Ventures noted that real-world use cases for commerce are likely focused on product research and local business discovery rather than making actual purchases.
All four of the voice assistants posted improved scores over the last time Loup Ventures tested them. Google Home posted an improvement of 7 percentage points from the past 12 months, Amazon Echo’s score rose by 9 percentage points and Cortana by 7 points. Siri’s score rose 22 percentage points over the past 9 months.