Thanks for putting the effort into this great article. I find that I am not surprised with Siri's recognition rate, which is so far below the 97% or greater rate necessary to make a recognition engine useful that it's no longer useful. I have been using siri voice recognition since the iPhone 4s came out and I would say that if I had recorded my voice back then and tested siri every year, the rate would be on a constant down trend due to a strong desire to recognize proper nowns and other problems that crop up when you don't know how to work with recognition engines- something I did as a developer in the past.
I want to call out two inaccuracies in the article that will probably be helpful.
First, actually, apple has intentionally crippled homekit remotely. Have you tried it? It turns out, you can't use siri to control your home remotely UNLESS you own an apple TV. If you own one, you may not have realized this. Since my lutron caseta application works just fine remotely, I'm having trouble believing this is necessary, but it's definitely a limitation. Of course the echo can't hear you when you're outside the house so it doesn't work, of course! Not the same as intentionally crippling it. My alexa isn't even on the same wifi network as my smart home devices so amazon is doing everything through the internet and associated accounts. Apple may be using wifi to interact. Both have their pros and cons from a security and reliability perspective.
The other inaccuracy: Actually you need to try siri with apple music enabled. With apple music enabled, you'll find that you can voice call out pretty much any song in the library of congress (close exaggeration). Realize: Amazon has a similar benefit when you are a prime subscriber, as I have been for years. I have both an echo / amazon prime... and the iphone/apple music combo. I have been pleasantly surprised by Amazon (they even have the Partridge family- don't ask)... but the Siri/Apple music combo is actually scary amazing. And of course if you have thousands of songs you've "ripped", those are all available too.
Anyway, thanks again for doing this important research, which I hope will drive more competition to get this important function right. I have become a big Alexa fan because besides the accuracy, is a "hands off" technology. So coming in the house with hands full, she's turning on the lights for me and doing a few other things before the grocery bags are even down. Love it.