Guoming Zhang, Chen Yan, Xiaoyu Ji
Taimin Zhang, Tianchen Zhang, Wenyuan Xu
2017 · arXiv
Speech recognition (SR) systems such as Siri or Google Now have become an increasingly popular human-computer interaction method, and have turned various systems into voice controllable systems(VCS). Prior work on attacking VCS shows that the hidden voice commands that are incomprehensible to people can control the systems. Hidden voice commands, though hidden, are nonetheless audible. In this work, we design a completely inaudible attack, DolphinAttack, that modulates voice commands on ultrasonic carriers (e.g., f > 20 kHz) to achieve inaudibility. By leveraging the nonlinearity of the microphone circuits, the modulated low frequency audio commands can be successfully demodulated, recovered, and more importantly interpreted by the speech recognition systems. We validate DolphinAttack on popular speech recognition systems, including Siri, Google Now, Samsung S Voice, Huawei HiVoice, Cortana and Alexa. By injecting a sequence of inaudible voice commands, we show a few proof-of-concept attacks, which include activating Siri to initiate a FaceTime call on iPhone, activating Google Now to switch the phone to the airplane mode, and even manipulating the navigation system in an Audi automobile. We propose hardware and software defense solutions. We validate that it is feasible to detect DolphinAttack by classifying the audios using supported vector machine (SVM), and suggest to re-design voice controllable systems to be resilient to inaudible voice command attacks.
Metadata: arXiv. Abstracts retain their original authors’ rights.
Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels
5 selected excerpts · 5 total mentions
Dolphin Attack used ultrasonic amplitude modulation against a variety of devices with some success [0]. Page 9 for the results table. I believe they were taking advantage of the non linearity of microphones. 0: https://arxiv.org/pdf/1708.09537.pdf
Completely agree. I use multiple ad- and tracker-blocking browser extensions, but mostly due to the tracking. Incidentally, while blocking ads online can be noticed by the serving party (to e.g. display an annoying warning) those behind the goods/services being verbally advertised are none the wiser. > There are ways to overlay inaudible tones Are you referencing prior work or was that hypothetical? There is actual prior work on this. They were able to engineer a POC that could trigger "Hey Siri" and "OK Google" inaudibly by broadcasting it with ultrasound on a harmonic of the correct frequency, but at a pitch inaudible to humans [1] [1] https://arxiv.org/pdf/1708.09537.pdf
That's clever but that's so many dB down with any sane anti-aliasing filter that it would require quite the sound source. Based on flipping through the pages of the paper (https://arxiv.org/pdf/1708.09537.pdf), it looks like they're taking advantage of the non-linearity in the response at high frequencies to effectively demodulate a lower-frequency signal that was mixed up to ~22 KHz. Which, if that's what they're doing, is totally awesome!
In case anyone missed it, the commands don't have to be audible: From https://arxiv.org/pdf/1708.09537.pdf > DolphinAttack ... modulates voice commands on ultrasonic carriers ... to achieve inaudibility. By leveraging the nonlinearity of the microphone circuits the modulated ... commands can be successfully demodulated
Filter won’t necessarily help. It’s possible to use ultra sonic for amplitude modulation signal to affect the non linearity of a microphone itself. [0] Although it does require knowing what the underlying natural sound to be modulated is (some processing required). Robustness to background noise doesn’t necessarily help either. 0: https://arxiv.org/pdf/1708.09537.pdf
More from the same topics.
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan +11
Artificial intelligence (AI) researchers have been developing and refining large language models (LLMs) that exhibit remarkable capabilities across a variety of domains and tasks, challenging our understanding of learning and cognition.
A. Pluchino. A. E. Biondo, A. Rapisarda
The largely dominant meritocratic paradigm of highly competitive Western cultures is rooted on the belief that success is due mainly, if not exclusively, to personal qualities such as talent, intelligence, skills, efforts or risk taking.
David Manheim, Scott Garrabrant
There are several distinct failure modes for overoptimization of systems on the basis of metrics.
Luca Beurer-Kellner, Beat Buesser, Ana-Maria Creţu +11
As AI agents powered by Large Language Models (LLMs) become increasingly versatile and capable of addressing a broad spectrum of tasks, ensuring their security has become a critical challenge.