Amazon employees are listening to what people tell Alexa — the company's voice recognition-based AI software — in the intimacy of their homes, a Bloomberg report revealed Thursday.
In order to improve machine learning algorithms, thousands of people are listening to snippets of audio detected by Alexa, checking whether the digital assistant correctly parsed the request and provided the desired solution. The audio is checked, annotated and then fed back into the system, thus training the voice recognition neural network. Working nine hours per day, Amazon employees check up to 1,000 audio clips daily, two workers from Amazon's Budapest division told Bloomberg.
"You don't necessarily think of another human listening to what you're telling your smart speaker in the intimacy of your home," Florian Schaub, a professor at the University of Michigan who has researched privacy issues related to smart speakers, told Bloomberg. "I think we've been conditioned to the [assumption] that these machines are just doing magic machine learning. But the fact is there is still manual processing involved."
The way Amazon's system is designed, Alexa (or its physical embodiment, the Echo smart speaker) listens to everything that happens in the room, including background conversations. Officially, the system does not store any audio until the wake word is detected ("Alexa" or "Echo" by default). However, the system often activates by mistake, triggered by mondegreen, background noise or even a working TV set. Regardless of what woke the system, Amazon employees are required to analyse and annotate input into it.
"We take the security and privacy of our customers' personal information seriously," an Amazon spokesman said in an emailed statement to Bloomberg. "We only annotate an extremely small sample of Alexa voice recordings in order [to] improve the customer experience."
The employees utilize group chats, in which they share the audio clips and seek assistance with particularly unintelligible words. As a way of relieving stress, possibly criminal or simply embarrassing clips often end up in those chats as well, the employees say.
Can Amazon track users by abusing this system? It is unclear, but the possibility is there. Screenshots of the review process provided to Bloomberg indicate each audio clip is tied to an account number and device serial number. While the reviewers have no access to any person's last name and personal address, their superiors might.
Interestingly, a certain percentage of consumers actually had their suspicions confirmed regarding unwanted ears listening in while they talk to Alexa. According to Bloomberg's sources, the employees often pick up questions like "Do you work for the NSA?" or "Alexa, is someone else listening to us?"