Siri on HomePod Asked 800 Questions and Answered 74% Correctly vs. Just 52% Earlier This Year

Apple analyst Gene Munster of Loup Ventures recently tested the accuracy of digital assistants on four smart speakers by asking Alexa, Siri, Google Assistant, and Cortana a series of 800 questions each on the Amazon Echo, HomePod, Google Home Mini, and Harmon Kardon Invoke respectively.


The results indicate that Siri on the HomePod correctly answered 74.6 percent of the questions, a dramatic improvement over the speaker's 52.3 percent success rate when Loup Ventures asked it a similar 782 questions in December 2017.


Siri on the HomePod remained less accurate than Google Assistant on the Google Home, which correctly answered 87.9 percent of questions in the test. Meanwhile, Alexa on the Echo and Cortana on the Invoke trailed Siri on the HomePod, correctly answering 72.5 percent and 63.4 percent of questions in the test.


Munster attributed the HomePod's improved accuracy to "the enabling of more domains in the past year," as a series of software updates in recent months have enabled the speaker to make and receive phone calls, schedule calendar events, set multiple timers, search for songs by lyrics, and more.

Methodology


Loup Ventures says it asked each smart speaker the same 800 questions, and they were graded on two metrics: whether the query was understood and whether a correct response was provided. The question set was designed to "comprehensively test a smart speaker's ability and utility" based on five categories:
  • Local – Where is the nearest coffee shop?
  • Commerce – Can you order me more paper towels?
  • Navigation – How do I get to uptown on the bus?
  • Information – Who do the Twins play tonight?
  • Command – Remind me to call Steve at 2 p.m. today.
The venture capital firm said it continues to modify its question set in order to reflect the changing abilities of digital assistants. "As voice computing becomes more versatile and assistants become more capable, we will continue to alter our test so that it remains exhaustive," said Munster.

Results by Category



Google Assistant on the Google Home correctly answered the most questions in four out of the five categories in the test, but fell short of Siri on the HomePod in the "command" category, according to Loup Ventures:
HomePod's lead in this category may come from the fact that the HomePod will pass on full SiriKit requests like those regarding messaging, lists, and basically anything other than music to the iOS device paired to the speaker. Siri on iPhone has deep integration with email, calendar, messaging, and other areas of focus in our Command category. Our question set also contains a fair amount of music-related queries, which HomePod specializes in.
Loup Ventures found the HomePod and Google Home stood "head and shoulders above the others" in both the "local" and "navigation" categories due to propriety data in Apple Maps and Google Maps respectively. "This data is a potential long-term comparative advantage for Siri and Google Assistant," said Munster.

What's Next


Siri's shortcomings were brutally exposed on the HomePod, which relies almost entirely on the assistant to function, so anecdotal evidence of improvement is a good sign, but Apple still has significant ground to make up against its competitors.

Loup Ventures acknowledged that Siri remains limited on the HomePod compared to its expanded capabilities on the iPhone. "This is partially due to Apple's apparent positioning of HomePod not as a 'smart speaker,' but as a home speaker you can interact with using your voice with Siri onboard," said Munster.

The venture capital firm said it will continue to compare the HomePod to other smart speakers over time amid rapid advancements in artificial intelligence.

Related Roundup: HomePod
Buyer's Guide: HomePod (Neutral)

Discuss this article in our forums

Amazon Confirms More Alexa-Enabled Speakers Will Eventually Support Apple Music

Amazon has confirmed that Apple Music will eventually be supported on additional Alexa-enabled speakers, according to a tweet from Mashable's Raymond Wong spotted by AppleInsider. Amazon didn't provide a timeframe for the rollout.


A variety of third-party Alexa speakers and devices are available from brands such as Sonos, JBL, Ultimate Ears, and First Alert.


Apple Music went live on Amazon's line of Echo speakers last Friday in the United States. This allows users to link Apple Music with their Amazon account in the Alexa app for iOS and use Alexa voice commands to control playback of Apple Music songs and playlists and Beats 1 radio on Echo speakers.

To access this feature, simply use a voice command such as "Alexa, play music by Ed Sheeran on Apple Music" or "Alexa, play today's hits on Apple Music." Apple Music can also be set as the default music service in the Alexa app so that "Apple Music" does not need to be specified each time.

Apple and Amazon announced this new partnership in late November, with Amazon saying it is "committed to offering great music providers to our customers," and referring to Apple Music as "one of the most popular music services."


Discuss this article in our forums

Apple Music Now Playable on Amazon Echo Speakers via Alexa in United States

Apple Music can now be streamed on Amazon Echo speakers via Alexa in the United States, a few days ahead of schedule.


As spotted by 9to5Mac, it is now possible to link Apple Music with your Amazon account in the Alexa app for iPhone and use Alexa voice commands to control playback of songs, playlists, and Beats 1 on Apple Music on Amazon Echo speakers.

To access this feature, simply use a voice command such as "Alexa, play music by Ed Sheeran on Apple Music" or "Alexa, play today's hits on Apple Music." Apple Music can also be set as the default music service in the Alexa app, so that "Apple Music" does not need to be specified each time.


Other streaming music services supported on Echo speakers include Spotify, Deezer, Vevo, SiriusXM, Tidal, and Pandora.

Apple and Amazon announced this new partnership in late November, with Amazon saying it is "committed to offering great music providers to our customers," and referring to Apple Music as "one of the most popular music services."

Apple Music playback on Amazon Echo speakers is currently limited to the United States.


Discuss this article in our forums

Amazon Alexa App Gains Redesigned Interface for Controlling Devices and Groups

Amazon is currently rolling out an updated version of its companion app for Alexa-enabled devices that includes a significantly redesigned user interface.

The visual changes are immediately apparent on firing up the app, and center around a new Devices and Groups tab located in the lower right of the screen.


With the new menu selected, the devices are listed in a horizontal strip along the top of the interface, allowing users to control individual smart lights, audio devices, and power outlets all from the same screen.

Below that, Alexa devices are also grouped by room indicated by colored cards, which have on/off buttons and can also be tapped to control and edit included devices.

Elsewhere, users can add new devices by tapping a plus button in the upper right of the interface, while the original clunky sidebar menu remains available via a button in the upper right.

The changes to the app come on the heels of Amazon's recent additions to its ecosystem of Alexa-powered devices, which include new Echo speakers, a subwoofer, amplifiers, a microwave, wall clock, and more.

The Amazon Alexa app is a free download on the App Store for iPhone and iPad. [Direct Link]

Tags: Amazon, Alexa

Discuss this article in our forums

Skype Calling Features Coming to Alexa Devices Later This Year

Microsoft has revealed that Skype is coming to Alexa devices. The partnership between Microsoft and Amazon means that owners of devices in Amazon's Echo range will soon be able to make outgoing Skype voice and video calls, accept incoming Skype calls, and also make SkypeOut calls to most phone numbers around the world.


Announcing the upcoming feature in a blog post, Microsoft said device owners will be able to say something like "Alexa, call Jimmy on Skype," or if Jimmy is calling you on Skype, they'll be able to say, "Alexa, answer."

This isn't the first time Amazon and Microsoft has seen fit to merge existing products and services. The two companies partnered on Alexa and Cortana integration last year.
"Since then, we've added Alexa integration on Xbox and are continuing to work together to bring the best of Skype and Alexa together to enable intelligent communications for our users," said Gaurav Sareen, corporate vice president for Microsoft. "We're excited to continue bringing the best of Microsoft and Amazon together."
Skype calling on Alexa will begin rolling out later this year.


Discuss this article in our forums

Amazon Working to Further Lessen Reliance on Power Cords for Future Echo Speakers

In an effort that would give future Echo speakers with Alexa a new advantage over Apple's HomePod, Reuters today reports that Amazon is working with chipmakers to build Alexa speakers that would not require a constant power source. As it stands, most smart assistant devices -- Amazon's main Echo, Apple HomePod, Google Home, etc -- require a constant draw of power due to their always-on microphones listening for the user's voice commands, among other reasons.

Now, Amazon has partnered with chipmaker DSP Group to build a standardized chip that is "a little smaller than a U.S. 10-cent coin" and optimized for lower power consumption. Amazon and other companies will be able to use the chip to place Alexa into their own smart devices that offer greater portability to users.


Amazon already has the Amazon Tap, a portable Bluetooth speaker that doesn't need to be plugged in to use all of its features. With a "hands-free mode" users can ask Alexa questions and command her to play music, which slightly affects battery life: 9 hours of continuous playback compared to 8 hours with hands-free mode activated.

There are also third-party companies like Fabriq that sell portable Alexa solutions, but these devices are still a small portion of the market in comparison to AC-powered speakers. With Amazon and DSP Group's new partnership, it appears that Amazon is looking to greatly bolster the amount of cord-free Alexa devices available to buy, while also improving the user experience with internal chips that can run always-on listening features without as much of a hit to battery life.

Amazon also has numerous other chipmakers offering solutions for Alexa -- including Intel and Qualcomm -- but as of now DSP is said to be the sole supplier competing with a chip that offers low power consumption. DSP's current client roster includes making chips for Samsung smart watches, cameras for GoPro, and a wireless speaker for Logitech.
Those are all large companies with many engineers, but by working with Amazon to make its chip easier to use in a device, DSP is hoping to spur more products from gadget makers of all sizes to include Alexa, Elyakim said.

“We are at a fairly early level in this market, but I think over the next 12 months we’ll see a lot of announcements,” he said.
For Apple's HomePod, the main rumor for the next-generation speaker centers around its price tag, with numerous reports earlier this year suggesting Apple is "mulling" and "low-cost version" of the HomePod that will run between $150-$200. What this means for specific features remains unclear, but since the original speaker is believed to have underperformed due to ecosystem limitations and Siri issues, some theorize Apple could focus on combating these problems in the cheaper HomePod.


The only time that portability and a future version of HomePod were tied together came from a questionable report earlier this spring, which claimed that the "low-priced" HomePod will be placed under the Beats by Dre brand, which only sells portable rechargeable speakers. Apple has yet to introduce its Siri assistant into any non-Apple product, and one reason Beats Pill devices don't have "Hey Siri" is likely due to their portability and lack of constant power.

Although unlikely at this point, the speculative HomePod/Beats speaker was described in a way that suggested a more portable smart speaker than the first generation, and could be similar to the proposed future versions of Amazon Echo from today's report. As it stands, there are many pros and cons to both the HomePod and Echo, as Apple's more expensive speaker focuses on sound quality first, while Amazon's speaker provides one of the most robust smart home hubs on the market thanks to its numerous Alexa Skills.

Tags: Amazon, Alexa

Discuss this article in our forums

Amazon Alexa App for iOS Gains Voice Controls

Amazon today updated its Alexa app for iOS to add a promised voice control feature that's designed to allow iOS users to tap the Alexa button and use voice commands to ask Alexa to play music, answer questions, check the weather, and more.

As noted by TechCrunch, Amazon introduced voice control to its Alexa app for Android back in January, and at the time, said the feature would be expanded to iOS devices "soon."


Soon turned out to mean six months, but starting today, iOS users can speak with Alexa in the Alexa iOS app and ask standard Alexa questions. There is no native functionality, so pushing an in-app button to activate Alexa is required rather than just saying the "Alexa" trigger phrase.

The new voice control feature will be rolled out to all iOS users "in the coming days."

The Amazon Alexa app can be downloaded from the iOS App Store for free. [Direct Link]

Tags: Amazon, Alexa

Discuss this article in our forums

Amazon’s Alexa Recorded a Woman’s Private Conversation and Sent it to a Contact

A woman in Portland recently had an alarming experience with her Alexa-enabled devices after a private conversation was recorded and sent to a random contact, according to a news report from Seattle's Kiro7 news.

The woman, Danielle, and her family had Amazon devices situated in each room for home control, and two weeks ago, one of those devices apparently recorded a conversation about hardwood floors and sent it to a person on their contact list. There are no details on how the recording was delivered to the contact.

But Danielle said two weeks ago their love for Alexa changed with an alarming phone call. "The person on the other line said, 'unplug your Alexa devices right now,'" she said. "'You're being hacked.'"

That person was one of her husband's employees, calling from Seattle.

"We unplugged all of them and he proceeded to tell us that he had received audio files of recordings from inside our house," she said. "At first, my husband was, like, 'no you didn't!' And the (recipient of the message) said 'You sat there talking about hardwood floors.' And we said, 'oh gosh, you really did hear us.'"
Danielle confirmed that the recordings received by the contact were indeed conversations picked up by her Alexa device, and in no way was she informed that Alexa was sending the recording to a contact. She contacted Amazon and was told that the "device just guessed what we were saying." Amazon apologized and told her it would fix the issue.

Alexa has an option to send a message to a contact name using a voice recording, but Alexa is supposed to vocally confirm such requests and does not appear to have done so in this instance.

In a statement to the Kiro7, Amazon said that it "takes privacy very seriously" and that the event was an "extremely rare occurrence" that it is taking steps to prevent in the future.

This is not the first strange Alexa behavior that Amazon has had to deal with. Back in March, Alexa made headlines after multiple customers with Alexa-enabled devices reported hearing creepy, unsolicited laughter.


Discuss this article in our forums

Amazon’s Alexa App Finally Gains Support for iPhone X

Amazon today updated its Alexa app for iOS devices to introduce iPhone X compatibility, allowing the app to be displayed full screen on the longer display of the iPhone X.

Prior to today, the Amazon Alexa app displayed unattractive black bars at the bottom and top of the app, so this should be a welcome change for Amazon users who have an iPhone X. The addition of Alexa support comes just over a month after Amazon promised the update was in the works.


The Alexa app for iOS is designed to be a companion to Alexa devices, allowing iPhone users to set up and manage Alexa-enabled equipment.

While many popular apps have been updated with support for the display of the iPhone X, there are still some notable holdouts even months after the release of the new smartphone. Inbox by Gmail, for example, continues to lack iPhone X support, as do several Apple apps, including iMovie, iTunes Connect, iTunes Remote, and AirPort Utility.

Tags: Amazon, Alexa

Discuss this article in our forums

New Survey Says Amazon and Google Have a More ‘Positive Impact on Society’ Than Apple

In a new survey conducted by SurveyMonkey and shared by Recode, 11 percent of respondents said that they believe Apple has the most positive impact on society today. Apple came in third place, however, falling in line behind Google at 15 percent and Amazon in the top spot at 20 percent.


Survey respondents were given the option of choosing between a number of technology companies between April 8-9, and it totaled 2,772 adults based in the United States. When asked which company they believe has the most positive impact on society, 20 percent elected to choose "none of the above."

Despite the ongoing Cambridge Analytica scandal, 10 percent of respondents chose Facebook as having the most positive impact today. Microsoft rounded out the top 5 slots at 7 percent, followed by Tesla at 6 percent, Uber at 3 percent, Netflix at 2 percent, and a list of companies at 1 percent.


The survey also dived into the CEOs of the companies and asked respondents which leaders had the greatest impact on people's daily lives. Amazon came in first place again, with 22 percent saying CEO Jeff Bezos and his decisions impact them in some way each day. Google's Sundar Pichai came in second at 18 percent, and Facebook's Mark Zuckerberg was third at 17 percent.

Amazon has been a competitor for Apple in a few areas over the last couple of years, particularly in areas like AI assistants and smart home accessory support. Most recently, The Information reported that an increasing number of home builders have opted to go with Alexa-enabled smart homes, rather than install HomeKit products during construction.

A Bloomberg report today claims that Amazon's next big push into consumer products will be a robot that can navigate around the user's home "like a self-driving car." The ultimate purpose of the robot is still unclear, but people familiar with the project believe it to be a "mobile Alexa" that would follow customers into areas of the home that don't have an Echo device.

Tags: Amazon, Alexa

Discuss this article in our forums