Skip to content
The methodOpen data
Back to the collection

Very Deep Convolutional Networks for Large-Scale Image Recognition

Karen Simonyan, Andrew Zisserman

2014 · arXiv

Computer visionPsychology
Read the paper
HN score
0
Mentions
6
Readers
6
Months
6
What’s in a score?

From the abstract

In this work we investigate the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting. Our main contribution is a thorough evaluation of networks of increasing depth using an architecture with very small (3x3) convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers. These findings were the basis of our ImageNet Challenge 2014 submission, where our team secured the first and the second places in the localisation and classification tracks respectively. We also show that our representations generalise well to other datasets, where they achieve state-of-the-art results. We have made our two best-performing ConvNet models publicly available to facilitate further research on the use of deep visual representations in computer vision.

Metadata: arXiv. Abstracts retain their original authors’ rights.

In the conversation

Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels

6 selected excerpts · 6 total mentions

godelski
Neutral mention · rule
For more context to people, we have the universal approximation theorem for neural nets that basically says if a network is wide enough it can approximate anything (with at least 2 layers). So a lot of stuff was really wide. Then VGG[0] came out and showed that deep networks were very effective (along with other papers, things happen in unison. Leibniz and Newton). Then you get ResNets[1] with skip connections and move forward to today. Today we've started looking more at what networks are doing and where their biases lie. This is because we're running into some roadblocks with CNNs vs Transformers. They have different inductive biases. Vision transformers still aren't…
Read the full comment
defeating CNNs, but they are close and it is clear they learn different things. So we're seeing more papers doing these types of analyses. ML will likely never be fully interpretable, but we're getting better at understanding. This is good because a lot of times picking your model and network architecture is more art than science (especially when choosing hyper parameters). [0] https://arxiv.org/abs/1409.1556 [1] https://arxiv.org/abs/1512.03385
Original comment Do wide and deep networks learn the same things?
nuclai
Neutral mention · rule
It's a pre-trained network on image classification dataset from 2014 called ImageNet. The network is called VGG, paper is here: http://arxiv.org/abs/1409.1556 There's no additional training apart from that. The neural network is used to extract patterns (grain/texture/style) and a separate optimization tries to reproduce them as appropriate.
Original comment Turning two-bit doodles into fine artworks with deep neural networks
varelse
Neutral mention · rule
The bigger the convolution, the faster it gets because a convolution in real space is a multiplication in frequency space. It breaks even at 5x5 or so and gets dramatically better shortly thereafter. However, most of the convolutional nets in use rely on 3x3 convolutions because I guess reasons: http://arxiv.org/pdf/1409.1556.pdf (all 3x3) http://www.cs.unc.edu/~wliu/papers/GoogLeNet.pdf (3x3 and 5x5) There's probably a new Imagenet winner in this somewhere IMO...
Original comment Deep learning
gcr
Neutral mention · rule
The current trend in convolutional neural networks seems to be moving toward more convolutions with smaller kernels. This year's second-place ILSVRC winner (VGG) is essentially a giant stack of 19 convolution layers with super tiny kernels, sandwiched between nonlinearities and pooling. "To reduce the number of parameters in such very deep networks, we use very small 3×3 filters in all convolutional layers" http://arxiv.org/pdf/1409.1556.pdf
Original comment Why GEMM is at the heart of deep learning
avallet
Neutral mention · rule
Hi! Author here. The blog post and source code refer to an older algorithm than the one in the demo. Currently it is using a convolutional network for prediction not dissimilar to [1] or [2]. I'm writing a paper about it, if there is interest I'll put it online :) . Thank you for the feedback on incorrectly identified characters! The algorithm is not quite perfect - 70% accuracy on a randomly chosen test set. [1]: http://arxiv.org/abs/1409.4842 [2]: http://arxiv.org/abs/1409.1556
Original comment Show HN: Anime character recognizer
karpathy
Neutral mention · rule
Google hasn't released details about their model yet, but the VGG team in close 2nd place had a very simple and beautiful "vanilla" CovnNet model, but with more careful hyperparameter settings and training/testing protocol. In other words, the source of improvement are tweaks on the original Krizhevsky architecture from 2012, not completely new and unexpected ideas. That is not to belittle the contribution - these experiments take forever to run and require very good practices and intuitions for what to try next. Karen released the details of the model only yesterday on arXiv (http://arxiv.org/pdf/1409.1556v1.pdf) More specifically, as can be seen in the paper, it seems…
Read the full comment
that very deep stacks of conv/conv/pool modules with tiny 3x3 filters work well (which is satisfying because it's really simple and beautiful), and from being more thorough with the training and testing protocol (data augmentations, averaging, multiscale approaches in both train/test time, etc). Google will release details of their method on Sept 12 so we'll know more. From their abstract, it seems they have a more significant departure from a basic convnet architecture.
Original comment Building a deeper understanding of images

Keep following the thread.

More in Computer vision

More from the same topics.

2018arXiv

Talent vs Luck: the role of randomness in success and failure

A. Pluchino. A. E. Biondo, A. Rapisarda

The largely dominant meritocratic paradigm of highly competitive Western cultures is rooted on the belief that success is due mainly, if not exclusively, to personal qualities such as talent, intelligence, skills, efforts or risk taking.

Psychology
16 mentions14 months
2.5HN score