Anh Nguyen, Jason Yosinski, Jeff Clune
2014 · arXiv
Deep neural networks (DNNs) have recently been achieving state-of-the-art performance on a variety of pattern-recognition tasks, most notably visual classification problems. Given that DNNs are now able to classify objects in images with near-human-level performance, questions naturally arise as to what differences remain between computer and human vision. A recent study revealed that changing an image (e.g. of a lion) in a way imperceptible to humans can cause a DNN to label the image as something else entirely (e.g. mislabeling a lion a library). Here we show a related result: it is easy to produce images that are completely unrecognizable to humans, but that state-of-the-art DNNs believe to be recognizable objects with 99.99% confidence (e.g. labeling with certainty that white noise static is a lion). Specifically, we take convolutional neural networks trained to perform well on either the ImageNet or MNIST datasets and then find images with evolutionary algorithms or gradient ascent that DNNs label with high confidence as belonging to each dataset class. It is possible to produce images totally unrecognizable to human eyes that DNNs believe with near certainty are familiar objects, which we call "fooling images" (more generally, fooling examples). Our results shed light on interesting differences between human vision and current DNNs, and raise questions about the generality of DNN computer vision.
Metadata: arXiv. Abstracts retain their original authors’ rights.
Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels
12 selected excerpts · 12 total mentions
For some background, see https://arxiv.org/pdf/1412.1897.pdf which has some cool visuals of pictures that make neural networks to strange things. Looks like this group might have solved adversarial examples (or rather, found that they're not a problem in practice after all) for self driving cars! Yann LeCun seems confident about it: https://mobile.twitter.com/ylecun/status/885384106418294784
TLDR: The point is to correct a known flaw with image recognition algorithms. I'm not a professional researcher, but here's what I gather from reading other articles: One major problem with image recognition in machines is that while they are generally able to recognize real images correctly, they are 1) Easily fooled 2) Unable to have human-type image understanding. For example, you can recognize an animated character as a human, even though you may never have seen that particular style of drawing before. One major problem that people realized is that deep neural networks have a tendency to recognize individual facets, for example, a nose. So if you want a neural network…
to believe something is a human, pepper it with as many noses as you can. It knows that humans have noses so the more noses the more human it must be. Of course, if we saw an image like this we wouldn't think that it was a human because we know a human has only one nose. To me it seems the primary thrust of this research is to generate a NN that can recognize, like a human, that if an entity has 10 eyes, it's probably not a cat. This is alluded to with the house of horrors cat photographs at the beginning of the article. You can see that when passed an image of a cat with 3 eyes, this neural network correctly removed one of the eyes to make it more realistic. Here is an article explaining this problem: https://arxiv.org/pdf/1412.1897.pdf
> I don't understand how Google's algorithm can be misled into finding sexiness in those. I'm reminded of a paper for which the authors generated different pictures of static that fooled neural network image classifiers into confidently identifying them as different objects: https://arxiv.org/abs/1412.1897 Wired summary: https://www.wired.com/2015/01/simple-pictures-state-art-ai-s... > Computer vision and human vision are nothing alike. And yet, since it increasingly relies on neural networks that teach themselves to see, we’re not sure precisely how computer vision differs from our own. As Jeff Clune, one of the researchers who conducted the study, puts it, when it comes to AI, “we can get the results without knowing how we’re getting those results.”
Basically this video ignores the history of machine learning in general. Jumping from Expert Systems to Neural Networks and Deep Learning is actually ignoring 36 years (and billions of dollars) of research http://machinelearning.org/icml.html (Breiman, Quinlan, Mitchell, Dietterich, Domingos, etc). Calling 2012 the seminal moment of Deep Learning is quite hard to digest. Maybe it means that 2012 is the point in time when the VC community discovered machine learning? Even harder to digest is calling Deep Learning the most productive and accurate machine learning system. What about more business oriented domains (without unstructured inputs), the extreme difficulties and…
expertise required to fine tune a network for a specific problem, or some drawbacks like the ones explained by http://arxiv.org/pdf/1412.1897v2.pdf or http://cs.nyu.edu/~zaremba/docs/understanding.pdf. Those who ignore history are doomed to repeat it. As Roger Schank pointed out recently http://www.rogerschank.com/fraudulent-claims-made-by-IBM-abo..., another AI winter is coming soon! Funny that the video details the three first AI winters but the author doesn't realize that this excessive enthusiasm in one particular technique is contributing to a new one!
That's a good point and it also brings to mind a way to fight it. This http://arxiv.org/pdf/1412.1897v4.pdf could be one way. You could generate images which are not of someone who wants to protect his privacy, but tweak them to strongly correlate to an image of that person by the neural net, whereas it could be a picture of anything you have just optimized (the images could be just noise, or another person, etc). You could generate batches of those and upload them, and confirm to Facebook that they are indeed photos of the person who wants their privacy back. You could repeat this (automatically) and corrupt the weights in FB's neural net, which would overcome their face detection abilities for the individual in question.
> given that we now have programs that look at a photograph and say "Women wearing a hat, sitting on a bar stool and drinking wine" That may be; but then there's also this: http://arxiv.org/abs/1412.1897
I find it less interesting in terms of applications. I am sure finding the causes of these illusions will help in understanding aspects of how the system, human or machine, subject to the illusion operates; however, so many of them depend on either a static image or a fixed perspective that it just doesn't seem to be all that big of an issue to me. The visual illusions humans are subject to are frequently defeated by some simple motion to reposition the viewer and there seem to be few illusions that can sustain being in motion themselves and still fool people. From that it doesn't seem all that revolutionary that a fixed position view of a three dimensional reality, with…
no fourth dimension of time, is not omniscient. It only has one bias sample or view of the scene, so why would we expect it to be omniscient? The paper "Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images"[0] is interesting in its explorations of the issue but I still don't get the hype around "fooling" DNNs. Even if someone gets an actual video scene, a timeseries of frame after frame, that still fools some kind of DNN (perhaps a LSTM => softmax classification), it's still not all that interesting as that occasionally seems to happen to humans too. [0]http://arxiv.org/abs/1412.1897 (web, but the pdf linked from there is ~9.5MB)
Definitely agree here, want to add a link to this paper which shows how far we still have to go (and questions whether the current models will ever replicate human vision): http://arxiv.org/abs/1412.1897
It's entirely possible for biometrics to spoofed. For example, folks have already figured out how to trick deep neural nets: http://arxiv.org/abs/1412.1897 Not to mention bio-engineering possibilities on the horizon. Of course, just because we can use a certain technology doesn't mean we should...
Algorithmic biases often reflect human biases I've been doing this for a while and I've literally never met a human who told an algorithm to overweight x[23] ("good looking"), x[48] ("is white") and x[873] ("is wealthy"), for x a 1,100-dimensional feature vector. Algorithms do have biases, but they are almost always orthogonal to the human ones. Witness, for example, all the recent "we can fool deep learning image recognition systems" papers. http://arxiv.org/abs/1412.1897 http://arxiv.org/abs/1312.6199 At this point I'm 90% sure you are a layperson who's never actually programmed such a system.
In (2)'s paper [2], the images are really neat. Staring at it for a few moments, I feel like I can see where the computer is coming from. A bit. The robin, armadillo, centipede, peacock, and bubble all actually have a little swirl of features that - to me at least - resemble the labels provided. But from afar, and you've got to basically damp the noise. I did this by taking my glasses off and leaning about 8 inches from the screen (with the grid taking about 3 inches wide). I've got about -7.5 diopter near-sightedness, so this cleaned the images right up. At least the armadillo I would have guessed, as it's absolutely a little critter walking to the lower right, and it…
has the demi-circle body and long face. And it might make sense: it was told to make an armadillo, so it did. And nothing else. (I also think the baseball was super clever - if this were abstract art, I would totally have fallen for that classification, as well as a few of the others. There's something cool going on there.) I'm printing and reading the rest of the paper now (about a quarter done). To say it's both fascinating and interesting really understates it. Really neat stuff! [2] http://arxiv.org/pdf/1412.1897v2.pdf
Automatic feature choice can actually lead to a set of features that resembles V1 receptive fields (as demonstrated by Olshausen and Field in https://courses.cs.washington.edu/courses/cse528/11sp/Olshau...). I recently attended a talk by Geoff Hinton on "capsules." He pointed out that the max pooling used in convolutional neural networks effectively disregards information about relationships among features. Instead, he propose a network composed of "capsules" that each estimate whether an implicitly defined intermediate feature is present and its pose. The idea is that an object is present only if its intermediate features are present and their poses agree. He showed some…
neat results from these models (some published in http://arxiv.org/pdf/1412.1897v1.pdf, and some from http://www.cs.utoronto.ca/~tijmen/tijmen_thesis.pdf). Notably, these models can evidently learn to classify MNIST with >98% accuracy given only 25 labeled examples. (I am not sure how many unlabeled examples were used.) I don't have any experience with these models, but given that most of these images look like a single feature embedded in noise or as a texture, I would not be surprised if a capsule-based network would not be so susceptible to these images.
More from the same topics.
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan +11
Artificial intelligence (AI) researchers have been developing and refining large language models (LLMs) that exhibit remarkable capabilities across a variety of domains and tasks, challenging our understanding of learning and cognition.
Tom B. Brown, Benjamin Mann, Nick Ryder +28
Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task.
Chiyuan Zhang, Samy Bengio, Moritz Hardt +2
Despite their massive size, successful deep artificial neural networks can exhibit a remarkably small difference between training and test performance.
A. Pluchino. A. E. Biondo, A. Rapisarda
The largely dominant meritocratic paradigm of highly competitive Western cultures is rooted on the belief that success is due mainly, if not exclusively, to personal qualities such as talent, intelligence, skills, efforts or risk taking.