Kyle Mahowald, Anna A. Ivanova, Idan A. Blank
Nancy Kanwisher, Joshua B. Tenenbaum, Evelina Fedorenko
2023 · arXiv
Large Language Models (LLMs) have come closest among all models to date to mastering human language, yet opinions about their linguistic and cognitive capabilities remain split. Here, we evaluate LLMs using a distinction between formal linguistic competence -- knowledge of linguistic rules and patterns -- and functional linguistic competence -- understanding and using language in the world. We ground this distinction in human neuroscience, which has shown that formal and functional competence rely on different neural mechanisms. Although LLMs are surprisingly good at formal competence, their performance on functional competence tasks remains spotty and often requires specialized fine-tuning and/or coupling with external modules. We posit that models that use language in human-like ways would need to master both of these competence types, which, in turn, could require the emergence of mechanisms specialized for formal linguistic competence, distinct from functional competence.
Metadata: arXiv. Abstracts retain their original authors’ rights.
Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels
6 selected excerpts · 6 total mentions
Here's the problem with LLMs: https://arxiv.org/abs/2301.06627 Basically, they're missing a lot of the brain machinery required to function. For example, if you ask them to solve a math problem, they do just fine ... until you ask them to apply an inference rule on top of it that takes them outside of their training set. The result is something that LOOKS like AGI until you realize it's read the entire Internet.
There's a decent amount of research that supports this view [1][2]. How else do you explain the unreasonable effectiveness of transformer-based language models? 1. https://journals.sagepub.com/doi/pdf/10.1177/014272371986973... 2. https://arxiv.org/pdf/2301.06627.pdf
> the idea that you can't get a very good statistical approximation to knowing and reasoning via a computer seem naively prone anthropocentrism. I don't think this is being questioned in general right now, but rather the claim is: You can't get a very good statistical approximation to knowing and reasoning via _just analyzing the language_. Language is evidently not enough on its own [1]. According to some researchers [2], the system needs to be "grounded" (think of it as being given common sense). Although there's apparently no consensus [3] among scientists on how to _fundamentally_ solve the shortcomings of current systems. [1] https://arxiv.org/abs/2301.06627 [2] https://drive.google.com/file/d/1BU5bV3X5w65DwSMapKcsr0ZvrMR... [3] https://www.youtube.com/watch?v=x10964w00zk edit: formatting
Amusing dialog, and quite germane - Yorick claims to hold a cognitivist perspective (that meaning comes from 'handling symbols', see also [1]) while the final paragraph ironically expresses a healthy skepticism of cognitivism which (IMO) is justified: we all have to 'take it on faith' that things mean what we want them to mean. That is, as Wittgenstein famously pointed out, the meaning of utterances is given by their use and not by some mysterious psychical accompaniment or the grinding of symbolic gears in the brain: > There must surely be a further, different connexion between my talk and N, for otherwise I should still not have meant HIM. > Certainly such a connexion…
exists. Only not as you imagine it: namely by means of a mental mechanism. - Philosophical Investigations §689 This is a drum I've been quietly banging for some time! [2],[3] 1: https://arxiv.org/abs/2301.06627 (but see [4]) 2: https://news.ycombinator.com/item?id=10158214 3: https://news.ycombinator.com/item?id=18880260 4: https://news.ycombinator.com/item?id=34638196
To properly explain this would take longer than the length of the comment limit (is there a length limit? I don't know, but even if there isn't I don't feel like explaining this for the 70th time), but here's why: chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://arxiv.org/pdf/2301.06627.pdf To sum up: a human can think outside of their training distribution, an LLM cannot. A larger training distribution simply means you have to go farther outside the norm. In order to solve this problem would require multiple other processing architectures besides an LLM, and a human-like AGI cannot be reached by simply predicting upcoming words. Functional language processing (formal reasoning, social cognition, etc) require other modules in the brain. This explains the various pejorative names given to LLMs - stochastic parrots, Chinese Rooms - etc, etc, etc.
> … but never any data to substantiate the claim There is! https://arxiv.org/abs/2301.06627 From the paper: > Based on this evidence, we argue that (1) contemporary LLMs should be taken seriously as models of formal linguistic skills; (2) models that master real-life language use would need to incorporate or develop not only a core language module, but also multiple non-language-specific cognitive capacities required for modeling thought.
More from the same topics.
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan +11
Artificial intelligence (AI) researchers have been developing and refining large language models (LLMs) that exhibit remarkable capabilities across a variety of domains and tasks, challenging our understanding of learning and cognition.
Tom B. Brown, Benjamin Mann, Nick Ryder +28
Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task.
Scott Aaronson
In honor of Alan Turing's hundredth birthday, I unwisely set out some thoughts about one of Turing's obsessions throughout his life, the question of physics and free will.
Luca Beurer-Kellner, Beat Buesser, Ana-Maria Creţu +11
As AI agents powered by Large Language Models (LLMs) become increasingly versatile and capable of addressing a broad spectrum of tasks, ensuring their security has become a critical challenge.