Deep Voice: Real-time Neural Text-to-Speech

Sercan O. Arik, Mike Chrzanowski, Adam Coates

Show all 12 authors

Gregory Diamos, Andrew Gibiansky, Yongguo Kang, Xian Li, John Miller, Andrew Ng, Jonathan Raiman, Shubho Sengupta, Mohammad Shoeybi

2017 · arXiv

Machine learning Computer vision Economics

Read the paper

HN score: 0
Mentions: 3
Readers: 3
Months: 3

What’s in a score?

From the abstract

We present Deep Voice, a production-quality text-to-speech system constructed entirely from deep neural networks. Deep Voice lays the groundwork for truly end-to-end neural speech synthesis. The system comprises five major building blocks: a segmentation model for locating phoneme boundaries, a grapheme-to-phoneme conversion model, a phoneme duration prediction model, a fundamental frequency prediction model, and an audio synthesis model. For the segmentation model, we propose a novel way of performing phoneme boundary detection with deep neural networks using connectionist temporal classification (CTC) loss. For the audio synthesis model, we implement a variant of WaveNet that requires fewer parameters and trains faster than the original. By using a neural network for each component, our system is simpler and more flexible than traditional text-to-speech systems, where each component requires laborious feature engineering and extensive domain expertise. Finally, we show that inference with our system can be performed faster than real time and describe optimized WaveNet inference kernels on both CPU and GPU that achieve up to 400x speedups over existing implementations.

Metadata: arXiv. Abstracts retain their original authors’ rights.

In the conversation

Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels

3 selected excerpts · 3 total mentions

Show

SmerityOct 4, 2024

Neutral mention · rule

Excited to see more people working on RNNs but wish their citations were better. In 2016 my team from Salesforce Research published our work on the Quasi-Recurrent Neural Network[1] (QRNN). The QRNN variants we describe are near identical (minGRU) or highly similar (minLSTM) to the work here. The QRNN was used, many years ago now, in the first version of Baidu's speech recognition system (Deep Voice [6]) and as part of Google's handwriting recognition system in Gboard[5] (2019). Even if there are expressivity trade-offs when using parallelizable RNNs they've shown historically they can work well and are low resource and incredibly fast. Very few of the possibilities…

Read the full comment

regarding distillation, hardware optimization, etc, have been explored. Even if you need "exact" recall, various works have shown that even a single layer of attention with a parallelizable RNN can yield strong results. Distillation down to such a model is quite promising. Other recent fast RNN variants such as the RWKV, S4, Mamba et al. include citations to QRNN (2016) and SRU (2017) for a richer history + better context. The SRU work has also had additions in recent years (SRU++), doing well in speech recognition and LM tasks where they found similar speed benefits over Transformers. I note this primarily as the more data points, especially when strongly relevant, the better positioned the research is. A number of the "new" findings from this paper have been previously explored - and do certainly show promise! This makes sure we're asking new questions with new insights (with all the benefit of additional research from ~8 years ago) versus missing the work from those earlier. [1] QRNN paper: https://arxiv.org/abs/1611.01576 [2] SRU paper: https://arxiv.org/abs/1709.02755 [3]: SRU++ for speech recognition: https://arxiv.org/abs/2110.05571 [4]: SRU++ for language modeling: https://arxiv.org/abs/2102.12459 [5]: https://research.google/blog/rnn-based-handwriting-recogniti... [6]: https://arxiv.org/abs/1702.07825

Original comment Were RNNs all we needed?

methouMar 1, 2017

Neutral mention · rule

It's in the abstract. "... For the audio synthesis model, we implement a variant of WaveNet that requires fewer parameters and trains faster than the original ..."[1] [1]: https://arxiv.org/abs/1702.07825

Original comment Deep Voice: Real-Time Neural Text-To-Speech

PieSquaredFeb 28, 2017

Neutral mention · rule

Hey there! Our paper is on Arxiv [0] and we have a blog post about this [1]. I am one of the authors on this paper and am happy to answer any questions you may have. [0] https://arxiv.org/abs/1702.07825 [1] http://research.baidu.com/deep-voice-production-quality-text...

Original comment Deep Voice: Real-Time Neural Text-To-Speech for Production

Keep following the thread.

Language Models are Few-Shot Learners

Tom B. Brown, Benjamin Mann, Nick Ryder +28

Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task.

Language models Machine learning

83 mentions35 months

3HN score

2016arXiv

Understanding deep learning requires rethinking generalization

Chiyuan Zhang, Samy Bengio, Moritz Hardt +2

Despite their massive size, successful deep artificial neural networks can exhibit a remarkably small difference between training and test performance.

Machine learning Computer vision Statistics & probability

16 mentions14 months

2.5HN score

2016arXiv

End to End Learning for Self-Driving Cars

Mariusz Bojarski, Davide Del Testa, Daniel Dworakowski +10

We trained a convolutional neural network (CNN) to map raw pixels from a single front-facing camera directly to steering commands.

Machine learning Computer vision

8 mentions7 months

2.5HN score

2018arXiv

Categorizing Variants of Goodhart's Law

David Manheim, Scott Garrabrant

There are several distinct failure modes for overoptimization of systems on the basis of metrics.

Algorithms Psychology Machine learning

8 mentions7 months

2.5HN score