English is singled out because its orthography really is atrociously bad even when compared with other languages. https://arxiv.org/pdf/1912.13321
OTEANN: Estimating the Transparency of Orthographies with an Artificial Neural Network
Xavier Marjou
2019 · arXiv
- HN score
- 0
- Mentions
- 5
- Readers
- 3
- Months
- 3
From the abstract
To transcribe spoken language to written medium, most alphabets enable an unambiguous sound-to-letter rule. However, some writing systems have distanced themselves from this simple concept and little work exists in Natural Language Processing (NLP) on measuring such distance. In this study, we use an Artificial Neural Network (ANN) model to evaluate the transparency between written words and their pronunciation, hence its name Orthographic Transparency Estimation with an ANN (OTEANN). Based on datasets derived from Wikimedia dictionaries, we trained and tested this model to score the percentage of correct predictions in phoneme-to-grapheme and grapheme-to-phoneme translation tasks. The scores obtained on 17 orthographies were in line with the estimations of other studies. Interestingly, the model also provided insight into typical mistakes made by learners who only consider the phonemic rule in reading and writing.
Metadata: arXiv. Abstracts retain their original authors’ rights.
In the conversation
Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels
5 selected excerpts · 5 total mentions
According to this paper [0] and my own experience, it's way easier to pronounce a word in French given the spelled word than in English. It's slightly harder to spell French than English for the model of the study, but it's really close. Now, in my personal experience, I feel like French has a lot of rules while English has a lot of outliers which do not follow any rules. But my native language is French, so I am obviously biased. [0]: https://arxiv.org/pdf/1912.13321
This is more generally the concept of Orthographic depth [1] A recent study attempted to categorize languages and found English and French to be among the most opaque, Finnish and Korean to be the most transparent, and German in the middle. [2] [1] https://en.wikipedia.org/wiki/Orthographic_depth [2] https://arxiv.org/abs/1912.13321
The complexity of these rules, and the number of exceptions that you need to learn notwithstanding the rules, can be roughly estimated for any given language by training a language model on word <-> IPA correspondence for that language (using a subset of the vocabulary as a training set), and then seeing how well it can predict the remaining words. You can run it in either direction, too, to separately measure the difficulty of reading (word -> IPA) and writing (IPA -> word) that language. This was actually done for a number of languages including English: https://arxiv.org/abs/1912.13321 You can see how languages with true phonemic spellings tend to be in the >90% range on both reading and writing, with Esperanto at 99%. Spanish and German are in 60-80% range. English is dismal at ~30% for both, though, with only French and Chinese being harder to write, and all other languages tested being easier to read.
There was an amusing study where they tried to measure how phonemic various language orthographies are by training an ANN model on some data to convert back and forth between spelling and pronunciation, and then checking the error rate on more inputs: https://arxiv.org/abs/1912.13321. Here's the table from that paper; numbers are percentages of accurate guesses. Orthography Write Read ----------- ---------- ---------- ar 84.3 ± 0.8 99.4 ± 0.3 br 80.6 ± 0.6 77.2 ± 1.6 de 69.1 ± 1.0 78.0 ± 1.5 en 36.1 ± 1.5 31.1 ± 1.3 eo 99.3 ± 0.2 99.7 ± 0.1 es 66.9 ± 2.0 85.3 ± 1.3 fi 97.7 ± 0.3 92.3 ± 0.8 fr 28.0 ± 1.4 79.6 ± 1.7 it 94.5 ± 0.8 71.6 ± 0.9 ko 81.9 ± 1.0 97.5 ±…
Read the full comment
0.5 nl 72.9 ± 1.7 55.7 ± 2.2 pt 75.8 ± 1.0 82.4 ± 0.9 ru 41.3 ± 1.6 97.2 ± 0.5 sh 99.2 ± 0.3 99.3 ± 0.3 tr 95.4 ± 0.7 95.9 ± 0.6 zh 19.9 ± 1.4 78.7 ± 0.9 It's easy to see here that English is truly dismal, so being better than it is not a great achievement. Spanish is okay but far from the best. Esperanto (eo) tops the chart for obvious reasons, but note how Serbo-Croatian (sh) is an example of truly outstanding phonemic orthography for an already existing language. Turkish (tr) and Finnish (fi) are also impressive.
Keep following the thread.
More in Machine learningMore from the same topics.
Language Models are Few-Shot Learners
Tom B. Brown, Benjamin Mann, Nick Ryder +28
Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task.
Understanding deep learning requires rethinking generalization
Chiyuan Zhang, Samy Bengio, Moritz Hardt +2
Despite their massive size, successful deep artificial neural networks can exhibit a remarkably small difference between training and test performance.
End to End Learning for Self-Driving Cars
Mariusz Bojarski, Davide Del Testa, Daniel Dworakowski +10
We trained a convolutional neural network (CNN) to map raw pixels from a single front-facing camera directly to steering commands.
Categorizing Variants of Goodhart's Law
David Manheim, Scott Garrabrant
There are several distinct failure modes for overoptimization of systems on the basis of metrics.