Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network

Because that isn't true? Looking at the first thing (GANs): developed by Ian Goodfellow, who is now at OpenAI. The first linked paper (https://arxiv.org/pdf/1605.05396v2.pdf) is out of University of Michigan and Max Planck Institute. The second (https://phillipi.github.io/pix2pix/) is out of Berkeley, the third (https://arxiv.org/pdf/1609.04802v3.pdf) is out of Twitter (which I guess counts as a rich American tech company). Google Brain/DeepMind/FAIR/MS Research do great work, but there is plenty of great work elsewhere too. For example, just yesterday an implementation[1] of Fast Layer Normalization for TensorFlow was posted on Reddit[2] but someone, who says they don't really do C++ ("I am really new to CUDA and C++"). This can speed up training (sometimes) by 5-10X (!). That's democratization. [1] https://github.com/MycChiu/fast-LayerNorm-TF [2] https://www.reddit.com/r/MachineLearning/comments/5gt0wm/p_f...

Some of the deeper - and wider - convolutional neural network-based SR methods do this, implicitly. We have a few papers, like SRGAN (https://arxiv.org/abs/1609.04802) or AffGAN (https://arxiv.org/abs/1610.04490) that generate pretty interesting context-aware textures. Sophisticated hallucination of missing details comes at a cost: As one would expect, in some cases the algorithm guesses incorrectly. See for example Figure 5 of AffGAN paper, column 3: when superresolving an image of a snake, the algorithm mistakenly thought it's a body of water, and produces textures accordingly. Overall we found that most people find the results of GAN-based approaches superior to other ML-based techniques.

(Author here.) My biggest insight from this project is that super-resolution with neural networks benefits significantly from being domain specific. If you train on broader datasets, it does pretty well but has to make compromises. Many recent papers do a comparison in terms of pixel similarity (PSNR/SSIM), and using those metrics the quality drops because high-frequency detail is punished under those criteria (even though it may look better perceptually). Reference: http://arxiv.org/abs/1609.04802 On GitHub, below each GIF there's a demo comparison, but on the site you can also submit your own to try it out (click on title or restart button). Takes about 60s currently; running on CPU as GPUs are busy training ;-)

Agreed, it would have been nice to show other upscaling algorithms. But neural net super resolution generators can still have significantly more detail at 4-8x, as shown here http://arxiv.org/abs/1609.04802

from trustswz' comment: https://arxiv.org/abs/1609.04802 The pic with the boat on page 13 is interesting. In the SRGAN version I would take the shore for some sort of cliff, while the original shows separated boulders.

If you are interested in how it compares to bicubic or the original. Check these papers using the sub pixel convolutional layer: https://arxiv.org/abs/1609.05158 https://arxiv.org/abs/1609.04802.