Loup Ventures has published their latest report on personal assistants and it is good news for Apple’s Siri. The company tested all four of the major digital assistants with a series of 800 questions and Siri was by far the most improved. It answered 74.6% of the questions correctly, up from the 52.3% success rate from the December 2017 report. That test was composed of 782 questions.
The test involved asking Amazon Alexa on an Amazon Echo, Siri on a HomePod, Google Assistant on a Google Home, and Microsoft Cortana on a Harmon Kardon Invoke. The questions were divided into four categories:
- Local – Where is the nearest coffee shop?
- Commerce – Can you order me more paper towels?
- Navigation – How do I get to uptown on the bus?
- Information – Who do the Twins play tonight?
- Command – Remind me to call Steve at 2 pm today.
Overall Siri was second behind Google Assistant on the Google Home. It was able to answer 87.9% of the questions correctly on the test. Alexa was third with 72.5% accuracy while Microsoft’s Cortana was last at 63.4%.
Breaking down the results further, the Loup Ventures report shows the strengths and weaknesses of each of the digital assistants. HomePod with Siri was able to 89% of the Local questions asked of it correctly while it struggled a bit with Commerce questions. It was only able to answer 56% of those questions correctly where as Google Assistant was at 86%. However, when it came to Command questions, Siri led the pack at 85% while Google Assistant was only able to answer 73% of them accurately.
What is clear in the report is that Apple has been working hard to improve their digital assistant. Take a look at the chart below from the report. HomePod substantially improved in the 2018 report in every question category except Commerce.
You can view the full report over at Loup Ventures.