Skip to content
The methodOpen data
Back to the collection

Alice in Wonderland: Simple Tasks Showing Complete Reasoning Breakdown in State-Of-the-Art Large Language Models

Marianna Nezhurina, Lucia Cipolina-Kun, Mehdi Cherti, Jenia Jitsev

2024 · arXiv

Machine learningLanguage models
Read the paper
HN score
0
Mentions
7
Readers
7
Months
5
What’s in a score?

From the abstract

Large Language Models (LLMs) are often described as instances of foundation models that possess strong generalization obeying scaling laws, and therefore transfer robustly across various conditions in few- or zero-shot manner. Such claims rely on standardized benchmarks that suppose to measure generalization and reasoning, where state-of-the-art (SOTA) models score high. We demonstrate here a dramatic breakdown of generalization and basic reasoning of all SOTA models claiming strong function, including large scale advanced models like GPT-4 or Claude 3 Opus, using a simple, short common sense math problem formulated in concise natural language, easily solvable by humans (AIW problem). The breakdown is dramatic as it manifests on a simple problem in both low average performance and strong performance fluctuations on natural variations in problem template that do not change either problem structure or its difficulty at all. By testing models on further control problems with similar form, we rule out that breakdown might be rooted in minor low-level issues like natural language or numbers parsing. We also observe strong overconfidence in the wrong solutions, expressed in form of plausible sounding explanation-like confabulations. Various standard interventions in an attempt to get the right solution, like chain-of-thought prompting, or urging the models to reconsider the wrong solutions again by multi step re-evaluation, fail. We use these observations to stimulate re-assessment of the capabilities of current generation of LLMs as claimed by standardized benchmarks. Such re-assessment also requires common action to create standardized benchmarks that would allow proper detection of such deficits in generalization and reasoning that obviously remain undiscovered by current state-of-the-art evaluation procedures, where SOTA LLMs manage to score high. Code: https://github.com/LAION-AI/AIW

Metadata: arXiv. Abstracts retain their original authors’ rights.

In the conversation

Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels

7 selected excerpts · 7 total mentions

s-macke
Neutral mention · rule
These results are very similar to the "Alice in Wonderland" problem [1, 2], which was already discussed a few months ago. However the authors of the other paper are much more critical and call it a "Complete Reasoning Breakdown". You could argue that the issue lies in the models being in an intermediate state between pattern matching and reasoning. To me, such results indicate that you can't trust any LLM benchmark results related to math and reasoning when you see, that changing the characters, numbers or the sentence structure in a problem alter the outcome by more than 20 percentage points. [1] https://arxiv.org/html/2406.02061v1 [2] https://news.ycombinator.com/item?id=40811329
Original comment Understanding the Limitations of Mathematical Reasoning in LLMs
smusamashah
Neutral mention · rule
We need results from these harder/different benchmarks which give pretty bad scores to current top LLMs. https://www.wolfram.com/llm-benchmarking-project/ https://help.kagi.com/kagi/ai/llm-benchmark.html Edit : There are few other benchmarks that give pretty low scores (<20%) to top LLMs. Can't find them atm. There was a benchmark with common sense easy looking questions. Edit: found two more papers https://arxiv.org/html/2405.19616 https://arxiv.org/html/2406.02061v1 Edit: How about Wordle? https://www.strangeloopcanon.com/p/what-can-llms-never-do https://news.ycombinator.com/item?id=40179232
Original comment Reflection 70B, the top open-source model
discreteevent
Neutral mention · rule
Well we don't have an understanding of how the brain works so we can't be fully sure but it's clear why they have this intuition: 1) Many people have had to cram for some exam where they didn't have time to fully understand the material. So for those parts they memorized as much as they could and got through the exam by pattern matching. But they knew there was a difference because they knew what it was like to fully understand something where that they could reason about it and play with it in their mind. 2) Crucially, if they understand the key mechanism early then they often don't need to memorize anything (the opposite of LLM's which need millions of examples) 3) LLM's display attributes of someone who has crammed for an exam and when it is probed further [1] it starts to break down in exactly the same way a crammer does. [1] https://arxiv.org/abs/2406.02061
Original comment Reasoning in Large Language Models: A Geometric Perspective
aadityaubhat
Neutral mention · rule
https://arxiv.org/abs/2406.02061v1 research paper shows the reasoning breakdown in SOTA LLMs by asking a simple question, “Alice has N brothers and she also has M sisters. How many sisters does Alice’s brother have?” I investigated performance of different prompts on this question, and show that 'Expand-then solve' prompt significantly outperforms standard and chain-of-thought prompts.
Original comment Novel prompting approach for Alice in Wonderland problem
salamo
Neutral mention · rule
What transformers have taught us is that gobbling up the whole internet can get you pretty far, but it doesn't change the fundamentals. Hallucinations still are trivial [1] to elicit. On the other hand, transformers are a software breakthrough that made training on the whole internet even feasible. Other methods (RNNs and LSTMs) were simply too slow. Transformers are a new hammer for an internet-sized nail. If I can make an analogy to chess engines, a type of AI so old that people don't call it AI anymore [2], if we had started out by applying transformers to available chess games, we would never be able to create an engine that could reliably beat a grandmaster.…
Read the full comment
Predicting the next chess move based on some representation of the game and its relationship with previously-seen positions only gets you so far. Deep Mind tried [3] this. One route forward, which doesn't require a fundamental breakthrough, would be synthetic data. To stick with chess, we could create huge synthetic datasets of a huge amount of chess games, and simply continue training as we were. However, this is very expensive, and can be tricky if the synthetic distribution isn't the same as the real one. And the end result is still going to require incredible computation power to use. What an LLM really lacks is the ability to search. And search is really important [4]: without it you'll probably need orders of magnitude more data and orders of magnitude larger models. At a fundamental level, search means we don't just make a snap judgement about e.g. the likely next token, but, as Daniel Kahneman would say, "thinking slowly" about what to do/say next. Assuming we will not have search is how you get projections like needing 100 GW power plants to supply dedicated training data centers. The human brain uses 12 watts. 100 GW is the energy requirements of all human brains on the planet. And yet, if we put all human brains on the planet together, our collective capabilities would be outclassed by a chess engine I can run on my phone. [1] https://arxiv.org/pdf/2406.02061 [2] https://en.wikipedia.org/wiki/AI_effect [3] https://arxiv.org/pdf/2402.04494 [4] https://www.youtube.com/watch?v=ceCg90Q9N6Y&t=1200s
Original comment AI Appears to Rapidly Be Approaching Brick Wall Where It Can't Get Smarter

Keep following the thread.

More in Machine learning

More from the same topics.