Wasserstein GAN

Hmm, I'm not sure what you mean by applicable loss functions? I'll answer what I think you're asking and you can tell me if I got it wrong: There's been a lot of effort spent on coming up with different loss functions for GANs, but https://arxiv.org/abs/1711.10337 shows that, according to the metrics in problem 5, they don't really improve results compared to the original GAN loss function. There's something called a Wasserstein GAN https://arxiv.org/abs/1701.07875 that you may have heard about, but IMO the useful thing to take away from that paper is their 'gradient penalty' technique and not the new loss function.

Quote: “One of our assigned projects was to implement and train a Wasserstein GAN in TensorFlow. At this point, I had taken only an introductory statistics class that was a required general elective, and then promptly forgotten most of it. Needless to say, my statistical skills were not very strong. Yet, I was able to read and understand a paper on a state-of-the-art generative machine learning model...Yet, if you had asked me, or most of the students in that class, how to calculate the variance of a population, or to define marginal probability, you likely would have gotten blank stares. ” I agree that the GAN might be implementable without much stats knowledge, but I would be very surprised if someone who did not know population variances and marginal probabilities would be able to follow the Wasserstein GAN paper. Just see for yourself: https://arxiv.org/abs/1701.07875

There is a paper that indicates Earth mover, aka Wasserstein-1 is better for convergence of GANs: https://arxiv.org/pdf/1701.07875.pdf I tried it out on one of the Udacity Deep learning assignments using the Wasserstein loss functions built into tensorflow. I was unsuccessful in my limited use. The discriminator always ‘won out’ rather than the combo finding a saddle point. I eventually got my project to work without it, and did not go back to compare against just swapping EM back in.

The kind of math that would be necessary to understand papers like [1][2][3][4]. [1]: https://arxiv.org/abs/1703.04933 [2]: https://arxiv.org/abs/1701.07875 [3]: https://arxiv.org/abs/1706.01350 [4]: https://arxiv.org/abs/1512.04860

It's not so much that there's just one or two specific results, but rather that there's just far more researchers working on this now, and quite a few are working on more theoretical stuff. Sometimes, that theory results in really practical outcomes - a good example would be https://arxiv.org/abs/1701.07875 . Or another by the same researcher: https://arxiv.org/abs/1506.00059 . We've seen, in the last year or two, interesting results in nearly every area of theoretical research of deep learning, including generalization, optimization, generative modelling, bayesian models, and network architecture.