Justin Johnson, Alexandre Alahi, Li Fei-Fei
2016 · arXiv
We consider image transformation problems, where an input image is transformed into an output image. Recent methods for such problems typically train feed-forward convolutional neural networks using a \emph{per-pixel} loss between the output and ground-truth images. Parallel work has shown that high-quality images can be generated by defining and optimizing \emph{perceptual} loss functions based on high-level features extracted from pretrained networks. We combine the benefits of both approaches, and propose the use of perceptual loss functions for training feed-forward networks for image transformation tasks. We show results on image style transfer, where a feed-forward network is trained to solve the optimization problem proposed by Gatys et al in real-time. Compared to the optimization-based method, our network gives similar qualitative results but is three orders of magnitude faster. We also experiment with single-image super-resolution, where replacing a per-pixel loss with a perceptual loss gives visually pleasing results.
Metadata: arXiv. Abstracts retain their original authors’ rights.
Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels
4 selected excerpts · 4 total mentions
Ahh I see, these are some resources that should help you. The original Neural Style Transfer paper should help you understand the loss functions involved: https://arxiv.org/abs/1508.06576 The paper introducing Real-Time Style Transfer is basically the algorithm used here: https://arxiv.org/abs/1603.08155
Slightly off-topic, but there's some pretty fascinating work that's been done recently on using neural networks for style transfer: https://arxiv.org/pdf/1603.08155v1.pdf http://www.anishathalye.com/2015/12/19/an-ai-that-can-mimic-...
This approach surprised me. Why are they doing feature extraction and then feeding that into a DNN? It seems much more straightforward to have the input of the network be noisy samples and the output be clean samples a la super resolution[0] in images. They probably wouldn't want to use fully-connected layers in that instance, but I don't see any fundamental barriers if they have enough computational power to run a neural network already. Am I missing something? [0] https://arxiv.org/pdf/1603.08155.pdf
IIRC you only get to choose from a fixed set of styles to process your image, right? Then they're probably using this algorithm to precompute the style model and seriously speed up rendering: http://arxiv.org/abs/1603.08155
More from the same topics.
Tom B. Brown, Benjamin Mann, Nick Ryder +28
Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task.
Chiyuan Zhang, Samy Bengio, Moritz Hardt +2
Despite their massive size, successful deep artificial neural networks can exhibit a remarkably small difference between training and test performance.
Haoran Xu, Fredrik Kjolstad
Fast compilation is important when compilation occurs at runtime, such as query compilers in modern database systems and WebAssembly virtual machines in modern browsers.
Mariusz Bojarski, Davide Del Testa, Daniel Dworakowski +10
We trained a convolutional neural network (CNN) to map raw pixels from a single front-facing camera directly to steering commands.