Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference

not justified. On the one hand, it's a language model and most of the tasks it's applied to are tasks for which we don't have very good metrics or benchmarks. Like I say in my earliest comment above, natural language understanding metrics are very bad at measuring "understanding" and we don't even have a commonly agreed definition of what that means. Basically, many benchmarks are defined as classification tasks, e.g. with multiple choice questions supposedly testing a model's understanding, but without any way to ensure that a system is not overfitting to statistical regularities in the dataset - and, indeed, language models have often been shown to do exactly that (e.g. see [1]). On the other hand, OpenAI very aggressively promotes its systems (not just GPT-3) to users outside of AI research and those users have no way to perform a systematic evaluation of such claims, so they are left with good old eyballing [2] of stuff like language generation or translation, etc. It's all too easy for such users to be impressed by a few hand-picked examples provided by OpenAI itself, or by other users who also don't have the capacity for systematic evaluation (and who hand-pick their results out of undue excitement, rather than for any other reason). The result is that there is a public perception that OpenAI's language models are much better than they really are. If memory serves, OpenAI made a very big todo about how GPT-2 was so good it was dangerous, etc. Well, now we have GPT-3 which is reportedly even more better- but it's served as an API. Doesn't sound that dangerous and it all sounds a lot more as hype than actual progress. ____________ [1] Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference https://arxiv.org/abs/1902.01007 [2] I keep saying that word, but it's actually semi-formal terminology. There was a paper about evaluating the results of grammar induction algorithms that used it. I'll see if I can find it.

not. Consider the philosophical problem of the existence of other minds. We believe other humans are intelligent because they consistently show intelligent behavior. Things that people claim to be examples of AI right now lack this consistency (possibly excluding a few very specific examples such as AlphaZero). It is quite annoying to see all these senior researchers along with graduate students spend so much time pushing numbers on those datasets without paying enough attention to the fact that pushing numbers is all they are doing. [1]: As a concrete example, consider the textual entailment (TE) task. In the deep learning era of TE there are two commonly used datasets on which the current state-of-the-art has been claimed to be near or exceeding human performance. What these models are performing seemingly exceptionally well is not the general task of TE, it is the task of TE evaluated on these fixed datasets. A recent paper by McCoy, Pavlick, and Linzen (https://arxiv.org/abs/1902.01007) shows how brittle these systems are that at this point the only sensible response to those insistent on claiming we are nearing human performance in AI is to laugh.