We are surprised to find that BERT's peak performance of 77% on the Argument Reasoning Comprehension Task reaches just three points below the average untrained human baseline. However, we show that this result is entirely accounted for by exploitation of spurious statistical cues in the dataset. We analyze the nature of these cues and demonstrate that a range of models all exploit them. This analysis informs the construction of an adversarial dataset on which all models achieve random accuracy. Our adversarial dataset provides a more robust assessment of argument comprehension and should be adopted as the standard in future work.
Metadata: arXiv. Abstracts retain their original authors’ rights.
In the conversation
Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels
I think I read this paper because someone posted it on HN (or linked to it in a comment):
https://arxiv.org/abs/1907.07355
We are surprised to find that BERT's peak performance of 77% on the Argument Reasoning Comprehension Task reaches just three points below the average untrained human baseline. However, we show that this result is entirely accounted for by exploitation of spurious statistical cues in the dataset. We analyze the nature of these cues and demonstrate that a range of models all exploit them. This analysis informs the construction of an adversarial dataset on which all models achieve random accuracy. Our adversarial dataset provides a more robust assessment…
Read the full comment
of argument comprehension and should be adopted as the standard in future work.
So it seems to be the benchmarks that are flawed and not BERT and friends who are that good at text comprehension. And that is not surprising. It would be surprising if language understanding just arose spontaneously by training a very big net on a very big corpus.
AI is not even close to having true logical reasoning, that's probably decades away. The issue is that cognitive scientists are clueless. Scientists have a good model for associative reasoning, which is the basis of modern neural networks, but we don't have a clue how abstract reasoning actually works. All birds and mammals have advanced abstract reasoning and are far more intelligent than GPT-4:
- birds and mammals are inherently able to count in almost any context because they understand what numbers actually mean; GPT-4 can only be trained to count in certain contexts. GPT-4 would be like a pigeon that could count apples, but not oranges, yet biological pigeons can…
Read the full comment
count anything they can see, touch, or hear. There's a profound gap in true quantitative reasoning, even if GPT-4 can fake this reasoning on specific human math problems.
- Relatedly, birds and mammals are far faster at general pattern recognition than GPT-4, unless it has been trained to recognize that specific pattern.
- Birds and mammals can spontaneously form highly complex plans; GPT-4 struggles with even the simplest plans, unless it has been trained to execute that specific plan.
The "trained to do that specific thing" is what makes GPT-4 so much dumber than warm-blooded vertebrates. When we test the intelligence of an animal in a lab, we make sure to test them on a problem they've never seen before. If you test AI like you test an animal, AI looks incredibly stupid - because it is!
There was a devastating paper back in 2019[1] proving that Google's BERT model - which at the time was world-class at "logical reasoning" - was entirely cheating on its benchmarks. And another paper from this year[2] demonstrates that LLMs definitely don't have "emergent" abilities, AI researchers are just sloppy with stats. It is amazing how much bad science and wishful thinking has been accepted by the AI community.
[1] https://arxiv.org/abs/1907.07355
[2] https://arxiv.org/abs/2304.15004
Similar to this post is this:
https://arxiv.org/abs/1907.07355
"Probing Neural Network Comprehension of Natural Language Arguments
Timothy Niven, Hung-Yu Kao
(Submitted on 17 Jul 2019)
We are surprised to find that BERT's peak performance of 77% on the Argument Reasoning Comprehension Task reaches just three points below the average untrained human baseline. However, we show that this result is entirely accounted for by exploitation of spurious statistical cues in the dataset. We analyze the nature of these cues and demonstrate that a range of models all exploit them. This analysis informs the construction of an adversarial dataset on which all models achieve random…
Read the full comment
accuracy. Our adversarial dataset provides a more robust assessment of argument comprehension and should be adopted as the standard in future work."
In the field of computer vision, there are similar suspicions that state of the art computer vision models are overfit to ImageNet datasets and the like. The issue being that even average research labs do not have hundreds of thousands of dollars to reproduce extremely expensive and highly tuned models. Reproducing and advancing the field of deep learning is quickly becoming inaccessible to almost everyone except for a few of the highest funded industrial labs (Google, FB, OpenAI, Microsoft Research among a few others). This is not all negative, it's just that not everything can be taken as gospel.
Regarding the "failed paradigm" (i.e: of logic- and knowledge-based AI) and
how neural nets scale, specifically on language modelling tasks.
I study a class of machine learning algorithms that learn logic programs (as
in Prolog, or ASP) from examples. The field is Inductive Logic Programming.
ILP algorithms are characterised by their ability to incorporate background
knowledge and impose strong inductive biases on the hypothesis space to learn
accurately from a handful of examples.
My favourite example of this is learning a grammar of the aⁿbⁿ language. My
implementation of one of the algorithms that my research group studies (I'm a
Phd research student) is…
Read the full comment
Thelma:
https://github.com/stassa/thelma
In Thelma's README on github you will find an example of Thelma learning a
grammar of the aⁿbⁿ language from three positive examples and no negative
examples. The learned grammar generalises to any n. In typical CFG notation
it's this grammar:
S → AB
S → AS₁
S₁ → B
A → a
B → b
(The nonterminal S₁ is invented, i.e. it was not given in the original
learning problem defintion. The preterminals A and B are background
knowledge.)
By contrast, neural networks can only learn fragments of this grammar up to
some limited n and that only if they're given tens of thousands of training
examples.
For instance, a classic paper by Gers and Schmidhüber [1] claims LSTM
"generalisation" from training sets of 22 to 42 thousand aⁿbⁿ strings, but
their average generalisaion is, e.g. from n in [1,50] (i.e. that's the value
of n in training strings) to n in [1,430] (n in testing strings) and their
"best" generalisation is at most to n in [1,1000].
On Thelma's README on github I have a small testing query that tests how the
aⁿbⁿ grammar it learned from 3 positive examples with n in [1,3] generalises
to an aⁿbⁿ string of e.g. n = 100,000:
?- _N = 100_000, findall(a, between(1,_N,_), _As), findall(b, between(1,_N,_),_Bs), append(_As,_Bs,_AsBs), anbn:'S'(_AsBs,[]).
true .
Well, it's a correct grammar so it generalises perfectly. You can bind _N to
any number your computer memory will allow and it will still parse.
So, besides the plug of my work (sorry) I would say that the earlier paradigm
that you say has "failed" with respect to recent connectionist success is
alive and well and it still gets many things right that the connectionist
paradigm struggles with mightily, in particular robust generalisation from
very few examples (without expensive pre-training etc), and of course any task
that requires reasoning [2].
______________
[1] "LSTM recurrent networks learn simple context free and context sensitive
languages": https://ieeexplore.ieee.org/document/963769 See table 2 on page 7
for the results I quote above.
[2] The examples of good performance on reasoning tasks you bring up have been
strongly criticised, e.g. in https://arxiv.org/abs/1907.07355 (Probing Neural
Network Comprehension of Natural Language Arguments) or Gary Marcus' recent
paper on a critical appraisal of deep learning etc.
P.S. I'm very sorry to see you're being downvoted.
Artificial intelligence (AI) researchers have been developing and refining large language models (LLMs) that exhibit remarkable capabilities across a variety of domains and tasks, challenging our understanding of learning and cognition.
Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task.
The Fermi paradox is the conflict between an expectation of a high {\em ex ante} probability of intelligent life elsewhere in the universe and the apparently lifeless universe we in fact observe.
Despite their massive size, successful deep artificial neural networks can exhibit a remarkably small difference between training and test performance.