Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jonathon Shlens, Zbigniew Wojna
2015 · arXiv
Convolutional networks are at the core of most state-of-the-art computer vision solutions for a wide variety of tasks. Since 2014 very deep convolutional networks started to become mainstream, yielding substantial gains in various benchmarks. Although increased model size and computational cost tend to translate to immediate quality gains for most tasks (as long as enough labeled data is provided for training), computational efficiency and low parameter count are still enabling factors for various use cases such as mobile vision and big-data scenarios. Here we explore ways to scale up networks in ways that aim at utilizing the added computation as efficiently as possible by suitably factorized convolutions and aggressive regularization. We benchmark our methods on the ILSVRC 2012 classification challenge validation set demonstrate substantial gains over the state of the art: 21.2% top-1 and 5.6% top-5 error for single frame evaluation using a network with a computational cost of 5 billion multiply-adds per inference and with using less than 25 million parameters. With an ensemble of 4 models and multi-crop evaluation, we report 3.5% top-5 error on the validation set (3.6% error on the test set) and 17.3% top-1 error on the validation set.
Metadata: arXiv. Abstracts retain their original authors’ rights.
Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels
4 selected excerpts · 4 total mentions
The latest deep learning models are indeed quite large. For comparison, Inception clocks in at "only" 5M parameters, itself a 12x reduction over AlexNet (60M) and VGGNet (180M)! (source: https://arxiv.org/abs/1512.00567) A further point is that even if the model has relatively few parameters, there are advantages to having more memory--- namely, you can do inference on larger batch sizes in one go.
> It's 2016 and our best of the best image recognition algorithms can't even tell a zebra and a sofa in a zebra print apart. State of the art offline CNNs have been doing classification quite well recently, arguably better than humans. See the Inception-v3 paper: http://arxiv.org/pdf/1512.00567v3.pdf
Looking up info on the inception architecture. Here is a paper if others are wondering: http://arxiv.org/pdf/1512.00567v3 From late 2015, so I guess the most up to date info? It only mentions v2. Is there a paper for v3? Edit: Sure enough v3 is right there. I saw the early focus on v2 and jumped to the big dumb assumption that v3 wasn't in there.
[1] is a Google-authored demo running Inception-v3[2] (about twice as accurate as VGG) on Android phones. I don't know how many parameters Inception v3 has, but I know Google considers it more efficient than VGG ("Although our network is 42 layers deep, our computation cost is only about 2.5 higher than that of GoogLeNet and it is still much more efficient than VGGNet".) Yes, being able to run big networks is great. But ultimately it's what you do with it, and a 3% error rate on ImageNet is a pretty compelling argument that size isn't the only factor. [1] https://github.com/tensorflow/tensorflow/tree/master/tensorf... [2] http://arxiv.org/abs/1512.00567
More from the same topics.
Chiyuan Zhang, Samy Bengio, Moritz Hardt +2
Despite their massive size, successful deep artificial neural networks can exhibit a remarkably small difference between training and test performance.
Mariusz Bojarski, Davide Del Testa, Daniel Dworakowski +10
We trained a convolutional neural network (CNN) to map raw pixels from a single front-facing camera directly to steering commands.
Ashish Vaswani, Noam Shazeer, Niki Parmar +5
The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration.
Robert Geirhos, Patricia Rubisch, Claudio Michaelis +3
Convolutional Neural Networks (CNNs) are commonly thought to recognise objects by learning increasingly complex representations of object shapes.