Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation

[FLAGGED] This very misguided article seems to be a complete misunderstanding of two papers: https://arxiv.org/abs/1706.05125 and https://arxiv.org/abs/1611.04558 None of the dramatic claims have any basis in real science, or real world events (e.g.: "it prompted concerns we could lose control of AI"? I'm 100% certain this came from the author's imagination). This kind of AI apocalyptic alarmism with no real basis is the same kind that created the first AI winter, and the complete misinterpretation of research in popular culture. Unless there's a very deep layer of irony I'm not seeing, this article is not worth your click.

I admittedly understand very little about ML and how Google is using it in Translate, but this seems relevant. In “Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation”, we address this challenge by extending our previous GNMT system, allowing for a single system to translate between multiple languages. Our proposed architecture requires no change in the base GNMT system, but instead uses an additional “token” at the beginning of the input sentence to specify the required target language to translate to. In addition to improving translation quality, our method also enables “Zero-Shot Translation” — translation between language pairs never seen explicitly by the system. https://research.googleblog.com/2016/11/zero-shot-translatio... The related research paper: https://arxiv.org/abs/1611.04558

Why do you say German is notoriously bad? I don't speak German, but that linked page reads fine to me, and their neural translation system has state-of-the-art performance for both English->German and German->English at least[1]. I would have thought that the well-structured nature of German makes it reasonably easy to translate, assuming you have a NN architecture with sufficient memory range. [1] https://arxiv.org/abs/1611.04558

He makes a good point about continuous functions. But actually neural nets are quite good at handling discrete elements like words, likes, atoms in a molecule, etc. Deep learning is part of enormous advances in NLP[0], just as it has set records to accuracy in almost every field of machine perception, from vision to audio. Neural word embeddings like those produced by word2vec[1] make for very useful feature vectors when fed into other neural nets. The headline of this post should be that NLP is harder than, say, image processing. In fact, for non-specialists, none of it is easy, because tuning hyperparameters is hard. The kind of NLP that tries to reproduce human-level sentences and understanding is simply a more complex problem, given the plasticity of language. [0] https://arxiv.org/abs/1611.04558 [1] https://deeplearning4j.org/word2vec

Hey HN, I'm author of the article. If there's anything confusing, I'd be happy to help :) The GNMT architecture is likely to keep popping up in Google's papers due to the amount of engineering that's been dedicated to make it scale well. Other than chewing up massive machine translation datasets, from single to multiple language pairs, it's also been trained over the entirety of Reddit for a conversational model system. Just this week a new paper[1] was released using the GNMT architecture which teaches a machine translation system how to translate between language pairs it has never been taught before by using bridging from other language pair knowledge. It's all pretty fascinating stuff :) [1]: https://arxiv.org/abs/1611.04558