Deep Speech 2: End-to-End Speech Recognition in English and Mandarin

https://arxiv.org/abs/1603.05027 Recurrent Neural Network Regularization - https://arxiv.org/abs/1409.2329 Deep Speech 2: End-to-End Speech Recognition in English and Mandarin - https://arxiv.org/abs/1512.02595 Order Matters: Sequence to Sequence for Sets - https://arxiv.org/abs/1511.06391 Neural Machine Translation by Jointly Learning to Align and Translate - https://arxiv.org/abs/1409.0473 Pointer Networks - https://arxiv.org/abs/1506.03134 Attention Is All You Need - https://arxiv.org/abs/1706.03762 The Annotated Transformer - https://nlp.seas.harvard.edu/annotated-transformer/ Neural Turing Machines - https://arxiv.org/abs/1410.5401 A Simple Neural Network Module for Relational Reasoning - https://arxiv.org/abs/1706.01427 Relational Recurrent Neural Networks - https://arxiv.org/abs/1806.01822 Neural Message Passing for Quantum Chemistry - https://arxiv.org/abs/1704.01212 Scaling Laws for Neural Language Models - https://arxiv.org/abs/2001.08361 GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism - https://arxiv.org/abs/1811.06965 Keeping Neural Networks Simple by Minimizing the Description Length of the Weights - https://www.cs.toronto.edu/~hinton/absps/colt93.pdf A Tutorial Introduction to the Minimum Description Length Principle - https://arxiv.org/abs/math/0406077 The First Law of Complexodynamics - https://scottaaronson.blog/?p=762 Quantifying the Rise and Fall of Complexity in Closed Systems: The Coffee Automaton - https://arxiv.org/abs/1405.6903 Kolmogorov Complexity - https://onlinelibrary.wiley.com/doi/book/10.1002/047174882X Variational Lossy Autoencoder - https://arxiv.org/abs/1611.02731 Machine Super Intelligence - https://www.vetta.org/documents/Machine_Super_Intelligence.p...

are various Arbeitgeberanteile which you have to pay as well. Let's say your per-employee expense would amount to €15/hour. Let's assume you can verify and record at 70% efficiency and you use two people to verify. Then you need to expend 4.29 employee-hours per final result hour. You couldn't just get German at this rate: Berlin is one of the towns with the largest language diversities in Germany. This would give you a price of €64.35 per result hour. You'd have to pay €64k for 1000 hours of validated training data, and 128k for the 2 thousand hour figure of currently achieved data. These €128k are probably on the same order of magnitude that Mozilla pays for the project (employee time to design, build, and run it), and if the project scales it will look even better. From a business POV, going open source was thus a great idea. To put the 2000 hours into comparison, the deepspeech 2 paper [1] used 10k hour datasets (per language, while the 2k hours are distributed amongst multiple languages) [1]. Record holder is probably Amazon with 1 million hours (although it's unlabeled) [2]. It's possible though that future breakthroughs will remove the need for tons of training data. So even if the restricted amount of training data can't create practical models in niche languages for now, it might very well be able to in the future. [1]: https://arxiv.org/pdf/1512.02595.pdf [1]: https://arxiv.org/pdf/1904.01624.pdf