Last weekend we did a guest post for VentureBeat comparing Siri, Cortana and Google Now in five different languages – English, French, Italian, German and Mandarin.
We received a number of requests asking us about our methodology. So, we’re sharing our specific test cases and results here.
Our test cases consisted of eighteen questions that users would be likely to ask for from a personal voice assistant. All requests were executed through voice commands and were targeted to test the usefulness of the assistant (rather than accuracy in pure dictation or search).
Defaulting to search for a query which could have generated a direct response (i.e. “call the nearest Chinese restaurant”) was penalized.
Query Results by Language
Scroll through the slideshow below to see detailed results for each language.
We scored each question on a scale of 1.0, 0.5 and 0.0. Questions that scored a full pass (1.0) were successfully executed by the voice assistant returning a direct response.
In contrast, a 0.5 rating was given to questions that displayed the right answer in a web search query.
0.0 ratings were given to inaccurate answers or to questions which the voice assistant could not successfully execute.
See below for an accuracy comparison across all five languages.
5 AI SEO Tools to Help You Rank Up
AI can give your SEO efforts – and your sales numbers, by extension - quite a boost. Here are 5 AI SEO too...
What You Need to Know About AI in SEO
The use of AI in SEO can boost your website’s ranking and therefore boost sales. But don’t rely on AI to c...