Vice reported that contractors working for Microsoft have listened to audio of Xbox users speaking in their homes in order to improve the console’s voice command features, Motherboard has learned. The audio was supposed to be captured following a voice command like “Xbox” or “Hey Cortana,” but contractors said that recordings were sometimes triggered and recorded by mistake.The news is the latest in a string of revelations that show contractors working on behalf of Microsoft listen to audio captured by several of its products. Motherboard previously reported that human contractors were listening to some Skype calls as well as audio recorded by Cortana, Microsoft’s Siri-like virtual assistant.
Incidents like this are a good reminder of the risk inherent in virtually all cloud-driven IoT systems. The fact that data traverses vendor infrastructure allows it to be exposed not only to authorized contractors as was the case here. It also potentially puts it within the reach of malicious hackers or rogue insiders. Despite the immense resources these organizations may have to implement security, their size is also a weakness as it makes them more attractive and can greatly expand attack surface.
The best way to avoid incidents like this is to use technologies which can function without relying on someone else’s computer (e.g. vendor infrastructure) for each and every interaction. At the present, this is somewhat daunting for the complexities of a voice assistant, but as the field and the underlying compute technology evolves, I think it will open new possibilities for products with the advanced features we’ve come to expect but without the sacrifice of privacy.