We propose a deep learning method for single image super-resolution (SR). Our method directly learns an end-to-end mapping between the low/high-resolution images. The mapping is represented as a deep convolutional neural network (CNN) that takes the low-resolution image as the input and outputs the high-resolution one. We further show that traditional sparse-coding-based SR methods can also be viewed as a deep convolutional network. But unlike traditional methods that handle each component separately, our method jointly optimizes all layers. Our deep CNN has a lightweight structure, yet demonstrates state-of-the-art restoration quality, and achieves fast speed for practical on-line usage. We explore different network structures and parameter settings to achieve trade-offs between performance and speed. Moreover, we extend our network to cope with three color channels simultaneously, and show better overall reconstruction quality.
Metadata: arXiv. Abstracts retain their original authors’ rights.
In the conversation
Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels
I think it's important to note that all of these methods have been largely superseded by deep learning techniques. For example, we can now directly learn algorithms such as gradient descent[1] and classical inverse problems like superresolution are solvable with deep networks [2]. While there still may be a role for tools like CVX, I anticipate all future progress will come from end-to-end differentiable systems.
[1] https://arxiv.org/abs/1606.04474
[2] https://arxiv.org/abs/1501.00092
They aren't. They compare to techniques "A+" from R. Timofte, V. De Smet, and L. Van Gool, “A+: Adjusted anchored neighborhood regression for fast super-resolution" http://www.vision.ee.ethz.ch/~timofter/ACCV2014_ID820_SUPPLE...
and "SRCNN" C. Dong, C. C. Loy, K. He, and X. Tang, “Learning a deep convolutional network for image super-resolution” https://arxiv.org/abs/1501.00092
Their result seems not to score as well in quantitative metrics (signal to noise ratio), but is faster. (And seems to have good qualitative results, in my opinion.)
There is a more recent version of this paper published here: http://arxiv.org/abs/1501.00092v3
At Flipboard, we did not have time to do a full comparison of related upscaling research, but we were happy with the low amount of error our CNN achieved.
FWIW, waifu2x was inspired by this paper from researchers at Chinese University of Hong Kong and Microsoft Research Asia http://arxiv.org/abs/1501.00092v3
waifu2x is a great demonstration of this approach applied to a specific domain.
By coincidence, Flipboard's DNN approach was developed around the same time as the MSRA research in summer 2014.
I'm excited to see future research in applying deep learning to generative tasks. Some of the CNN music composition work is quite impressive.
Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task.
Despite their massive size, successful deep artificial neural networks can exhibit a remarkably small difference between training and test performance.