Skip to content
The methodOpen data
Back to the collection

Deep Image Prior

Dmitry Ulyanov, Andrea Vedaldi, Victor Lempitsky

2017 · arXiv

Computer visionStatistics & probabilityPsychology
Read the paper
HN score
0
Mentions
3
Readers
3
Months
3
What’s in a score?

From the abstract

Deep convolutional networks have become a popular tool for image generation and restoration. Generally, their excellent performance is imputed to their ability to learn realistic image priors from a large number of example images. In this paper, we show that, on the contrary, the structure of a generator network is sufficient to capture a great deal of low-level image statistics prior to any learning. In order to do so, we show that a randomly-initialized neural network can be used as a handcrafted prior with excellent results in standard inverse problems such as denoising, super-resolution, and inpainting. Furthermore, the same prior can be used to invert deep neural representations to diagnose them, and to restore images based on flash-no flash input pairs. Apart from its diverse applications, our approach highlights the inductive bias captured by standard generator network architectures. It also bridges the gap between two very popular families of image restoration methods: learning-based methods using deep convolutional networks and learning-free methods based on handcrafted image priors such as self-similarity. Code and supplementary material are available at https://dmitryulyanov.github.io/deep_image_prior .

Metadata: arXiv. Abstracts retain their original authors’ rights.

In the conversation

Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels

3 selected excerpts · 3 total mentions

vintermann
Neutral mention · rule
> Up until now learning to replicate the input exactly was called overfitting and considered a bug, not a feature, but they showed a completely new way to wield neural nets. Well, for that particular ting there was a predecessor of sort in Deep Image Priors from 2017. https://arxiv.org/abs/1711.10925 That was all about overfitting a neural net on a single image, which they used to get impressive inpainting, noise removal and superresolution results without any training at all (though of course it did not beat state of the art training-based approaches, even then.) I had a lot of fun playing around with it when it came. The idea is dead simple, within reach to implement yourself with no complex mathematical understanding.
Original comment HyperNeRF
loser777
Neutral mention · rule
What happens when the perturbed images are processed by some noise removal method? On the crude end, even something like aggressive JPEG compression will tend to remove high frequency noise. There's also more sophisticated work like Deep Image Prior [1], which can reconstruct images while discarding noise in a more "natural" way. Finally, on the most extreme end, what happens when someone hires an artist or builds a sufficiently good robot artist to create a photorealistic "painting" of the perturbed image? There's a lot of work on compressing/denoising images so that only the human-salient parts are preserved, and without seeing this working past that I think it's better to interpret "adversarial" in the machine learning sense only. Where "adversarial" means useful for understanding how models work, but not with any strong security implications. [1] https://arxiv.org/abs/1711.10925
Original comment Adversarial.io – Fighting mass image recognition
blt
Neutral mention · rule
(two comments in this thread:) > What always fascinated me is how those images look almost exactly like the hallucinations you get on some psychedelics... The AI must be pretty close if they can already match the output of a confused human brain. > Traditionally, with computer generated stuff, you could clearly see the math in the algorithms (the sine waves and fractals and whatnot). With AI generated stuff it looks... natural... It's a computer no longer letting you see how he thinks. The paper "Deep Image Prior" by Dmitry Ulyanov et al. gives compelling evidence that the structure of convolutional neural networks already encodes strong knowledge about the appearance of…
Read the full comment
natural images, independent of any specific parameters (learned weights). Independence from parameters means it's independent from what task the network was trained to accomplish, and of the training algorithm. This helps explain (IMO) why a neural network with "wrong" weights (meaning, the training process did not fully meet the goal of the project) still produces images that like plausible activations of the human visual cortex, rather than harsh mathematical patterns. The convolutional network structure is biased towards natural-looking images. paper: https://arxiv.org/abs/1711.10925 third-party blog post: http://mlexplained.com/2018/01/18/paper-dissected-deep-image...
Original comment This AI is bad at drawing but will try anyways

Keep following the thread.

More in Computer vision

More from the same topics.

2018arXiv

Talent vs Luck: the role of randomness in success and failure

A. Pluchino. A. E. Biondo, A. Rapisarda

The largely dominant meritocratic paradigm of highly competitive Western cultures is rooted on the belief that success is due mainly, if not exclusively, to personal qualities such as talent, intelligence, skills, efforts or risk taking.

Psychology
16 mentions14 months
2.5HN score