Robert Geirhos, Patricia Rubisch, Claudio Michaelis
Matthias Bethge, Felix A. Wichmann, Wieland Brendel
2018 · arXiv
Convolutional Neural Networks (CNNs) are commonly thought to recognise objects by learning increasingly complex representations of object shapes. Some recent studies suggest a more important role of image textures. We here put these conflicting hypotheses to a quantitative test by evaluating CNNs and human observers on images with a texture-shape cue conflict. We show that ImageNet-trained CNNs are strongly biased towards recognising textures rather than shapes, which is in stark contrast to human behavioural evidence and reveals fundamentally different classification strategies. We then demonstrate that the same standard architecture (ResNet-50) that learns a texture-based representation on ImageNet is able to learn a shape-based representation instead when trained on "Stylized-ImageNet", a stylized version of ImageNet. This provides a much better fit for human behavioural performance in our well-controlled psychophysical lab setting (nine experiments totalling 48,560 psychophysical trials across 97 observers) and comes with a number of unexpected emergent benefits such as improved object detection performance and previously unseen robustness towards a wide range of image distortions, highlighting advantages of a shape-based representation.
Metadata: arXiv. Abstracts retain their original authors’ rights.
Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels
3 selected excerpts · 3 total mentions
For everyone interested in what makes a classifier put an image in a certain class I can recommend the excellent paper by Robert Geirhos et al. "ImageNet-trained CNNs are biased towards texture" [1]. The paper is a bit older but it's an excellent read. [1] https://arxiv.org/pdf/1811.12231
You can make the CNN focus more on shape rather than photographic texture using style transfer, hence, Stylized-Imagenet: https://arxiv.org/abs/1811.12231 I've wondered if you could do the same thing with anime images as well, and if a joint classifier (using, say, https://www.gwern.net/Danbooru2020 ?) would be much better for it. Of course, the big trend now is CLIP/ALIGN https://openai.com/blog/clip/ https://arxiv.org/abs/2102.05918#google which are trained on web images on such a scale that they do quite fine on anime images. Look on Twitter or Reddit for 'Big Sleep' examples using the ThisAnimeDoesNotExist.ai anime model: CLIP can generate anime images reasonably well, along with everything else it can do so well. (The 'blessings of scale', I call it: https://www.gwern.net/newsletter/2020/05#blessings-of-scale Just train on as much data as possible, and a lot of these weaknesses fix themselves.)
Note that the leading image classification algorithms are apparently trained to recognize texture more than shapes, because that's the easiest way to win at current benchmarks. But that can be fixed: https://arxiv.org/abs/1811.12231
More from the same topics.
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan +11
Artificial intelligence (AI) researchers have been developing and refining large language models (LLMs) that exhibit remarkable capabilities across a variety of domains and tasks, challenging our understanding of learning and cognition.
Tom B. Brown, Benjamin Mann, Nick Ryder +28
Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task.
Chiyuan Zhang, Samy Bengio, Moritz Hardt +2
Despite their massive size, successful deep artificial neural networks can exhibit a remarkably small difference between training and test performance.
A. Pluchino. A. E. Biondo, A. Rapisarda
The largely dominant meritocratic paradigm of highly competitive Western cultures is rooted on the belief that success is due mainly, if not exclusively, to personal qualities such as talent, intelligence, skills, efforts or risk taking.