Lukas Berglund, Meg Tong, Max Kaufmann
Mikita Balesni, Asa Cooper Stickland, Tomasz Korbak, Owain Evans
2023 · arXiv
We expose a surprising failure of generalization in auto-regressive large language models (LLMs). If a model is trained on a sentence of the form "A is B", it will not automatically generalize to the reverse direction "B is A". This is the Reversal Curse. For instance, if a model is trained on "Valentina Tereshkova was the first woman to travel to space", it will not automatically be able to answer the question, "Who was the first woman to travel to space?". Moreover, the likelihood of the correct answer ("Valentina Tershkova") will not be higher than for a random name. Thus, models do not generalize a prevalent pattern in their training set: if "A is B" occurs, "B is A" is more likely to occur. It is worth noting, however, that if "A is B" appears in-context, models can deduce the reverse relationship. We provide evidence for the Reversal Curse by finetuning GPT-3 and Llama-1 on fictitious statements such as "Uriah Hawthorne is the composer of Abyssal Melodies" and showing that they fail to correctly answer "Who composed Abyssal Melodies?". The Reversal Curse is robust across model sizes and model families and is not alleviated by data augmentation. We also evaluate ChatGPT (GPT-3.5 and GPT-4) on questions about real-world celebrities, such as "Who is Tom Cruise's mother? [A: Mary Lee Pfeiffer]" and the reverse "Who is Mary Lee Pfeiffer's son?". GPT-4 correctly answers questions like the former 79% of the time, compared to 33% for the latter. Code available at: https://github.com/lukasberglund/reversal_curse.
Metadata: arXiv. Abstracts retain their original authors’ rights.
Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels
8 selected excerpts · 8 total mentions
My take is when the model can't do the things that you would expect it to do if it were reasoning, and instead does the things that you would expect it to do if it were an approximate retrieval system with an enormous knowledge base. this is a good paper: https://arxiv.org/abs/2309.12288
What a sloppy reply. You've hijacked a thread on mathematics first to complain that your incompetent attempt to use ChatGPT to find a job failed, but it seems now that this was a ruse to instead begin arguments unrelated to the article at all where you just spam arxiv links you've never read to "prove" that AI is a scam. This comes across, frankly, as either Dunning-Kruger (classic illusory superiority), or potentially as mental illness. The slop dump is highly reminiscent of how a schizophrenic friend of mine communicates. Do you really think slopping down a bunch of random arxiv links "proves" that AI is a scam and you're so smart and everyone else isn't? Most awkwardly…
for your arxiv slop -- most of this is irrelevant to your central claim, and you've missed papers that are much closer. For example your LogicGraph paper: "Can't exhaustively enumerate all minimal proofs" is not "can't distinguish Ireland from London". Or your "Do VLMs Understand 3D Scenes..." is nothing more than citation decoration, completely irrelevant to our discussion. Or your "Frontier LLMs Still Struggle with Simple Reasoning Tasks" which is potentially your pièce de résistance, it supports brittle multi-step constraint handling, but isn't remotely an eval of a modern web-search agent. For example, VibeSearchBench would have been far more relevant to your claims https://arxiv.org/html/2605.27882v1 (but still obviously not proof that AI is "a parlour trick") Going further: my point that we need to discuss your beginner's approach to the harness is substantiated clearly here: https://arxiv.org/html/2605.23950v1 Finally, failure to exhibit human-like generality is not evidence of absence of intelligence. It is evidence that whatever cognitive machinery LLMs possess has a very different error distribution from ours. General365, LLMEval-Logic and the Reversal Curse are actually fascinating evidence for that jaggedness. https://arxiv.org/html/2604.11778v1 https://arxiv.org/abs/2309.12288 https://arxiv.org/html/2605.19597v1 Don't worry, I don't expect you to read all of that research I posted, since you clearly didn't read your own linkspam, since most of it is irrelevant to the claim you made.
Its the tools. Sell those RSUs while they last. Most of these are 2026.... Frontier LLMs Still Struggle with Simple Reasoning Tasks - https://arxiv.org/abs/2507.07313 General365: Benchmarking General Reasoning in Large Language Models Across Diverse and Challenging Tasks - https://arxiv.org/abs/2604.11778 LLMEval-Logic: A Solver-Verified Chinese Benchmark for Logical Reasoning of LLMs with Adversarial Hardening - https://arxiv.org/abs/2605.19597 LogicGraph: Benchmarking Multi-Path Logical Reasoning via Neuro-Symbolic Generation and Verification - https://arxiv.org/abs/2602.21044 Blind-Spots-Bench: Evaluating Blind Spots in Multimodal Models -…
https://arxiv.org/abs/2607.08317 Vision-Language Models Lag Human Performance on Physical Dynamics and Intent Reasoning - https://arxiv.org/abs/2601.01547 Do Vision-Language Models Understand 3D Scenes or Just Catalogue Objects? - https://arxiv.org/abs/2605.20448 The Reversal Curse: LLMs Trained on “A is B” Fail to Learn “B is A” - https://arxiv.org/abs/2309.12288 Large Language Model Reasoning Failures - https://arxiv.org/abs/2602.06176
I don't have explanations but I can point you to one of the papers: https://arxiv.org/pdf/2309.12288 which calls it "the reversal curse" and does a bunch of experiments showing models that are successful at questions like "Who is Tom Cruise’s mother?" (Mary Lee Pfeiffer) will not be equally successful at answering "Who is Mary Lee Pfeiffer’s son?"
I've written about the cognitive aspect the author talks about before: https://blog.jtoy.net/examining-concepts-through-different-f... https://blog.jtoy.net/on-the-wondrous-human-simulation-engin... https://arxiv.org/abs/2210.12068 The human brain has at least over 12 different coordinate systems. I like to reference this image: https://cln.sh/gpqhwrw2Hg92MQz62TY6 Humans have the ability to inspect anything from any point of view that we want, like a real time video game simulation system in our minds. I suspect that artificial neural networks are missing this core ability to encode and translate its point of view when processing data. Just look at basic LLMs failing at…
the reversal curse: https://arxiv.org/abs/2309.12288 No amount of more GPUs and more training data will fix it. The entorhinal cortex in humans has a special kind of neuron called grid cells that works with hippocampus cells to do generalized coordinate translations. The discovery of grid cells won a nobel peace prize ( https://www.nobelprize.org/prizes/medicine/2014/advanced-inf... ). It is a prime suspect to incorporate for a coordinate transformation layer in my opinion. I want to build an ANN that incorporates this "coordinate transformation system", but I'm not sure how to translate it into code, this is my dream project.
>Ilya claims the transformer architecture, with some modification for efficiency, is actually sufficient for AGI. I thought this guy was supposed to know what he's talking about? There was a paper that shows LLMs cannot generalise[0]. Anybody who's used ChatGPT can see there's imperfections. [0] https://arxiv.org/abs/2309.12288
> Numbers (LLMs still can't do math and the ones that "can" are very brute forced and don't generalize) These seem to generalize. https://arxiv.org/abs/2211.09066 https://arxiv.org/abs/2308.00304 > Commutative properties (The (idk why anyone was surprised) "reversal curse": https://arxiv.org/abs/2309.12288) This was a retrieval from training issue not an inference one. and make no mistake, people will fail these sort of answers too (perhaps not to the same extent). Very often someone will learn a fact in a certain why and fail to recall that when asked in a reverse or roundabout way. Communicative properties are not something that occur in the vast majority of text or…
language. It's not "generalization" for training to treat it as such lol. Since they can answer such questions just fine during inference, It makes little sense to say an LLM doesn't understand communicative properties. > How to say "I don't know" There's quite a lot of indication that the computation can distinguishing hallucinations. It just has no incentive to communicate this. Don't blame the LLM here. Blame the very human data that doesn't encourage this. GPT-4 logits calibration pre RLHF - https://imgur.com/a/3gYel9r Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - https://arxiv.org/abs/2305.14975 Teaching Models to Express Their Uncertainty in Words - https://arxiv.org/abs/2205.14334 Language Models (Mostly) Know What They Know - https://arxiv.org/abs/2207.05221 I'm not trying to say that LLMs are perfect. and as long as people understand that saying an LLM doesn't understand x doesn't mean an LLM doesn't understand anything at all, i'll happily state weaknesses.
To add to this list of things LLMs don't understand: - Numbers (LLMs still can't do math and the ones that "can" are very brute forced and don't generalize) - Commutative properties (The (idk why anyone was surprised) "reversal curse": https://arxiv.org/abs/2309.12288) - Associative properties - World models (i.e. can understand some basic physics and causality. Not mathematically, but the same way a child understands that letting go of something in their hand falls to the ground and not up) - How to say "I don't know" - Any form of meaningful generalization (GPT 3.5 and most LLMs still have difficulties with "Which weighs more, a pound of feathers or a kilogram of…
bricks". GPT4 gets it, but I'm not convinced it isn't because I've asked it that question too many times and they specifically trained on it.) And a whole load of things. With basically all these things being quite relevant to the collective interpretation of "understanding." They aren't hard to tease out either. You ever have a problem that isn't easy to google because the search terms are too close to a different problem? The LLM will give you very similar results, even if you probe with corrections and specify that your problem is different. Like layer8, many many times I've corrected (many different) LLMs and cannot tease out the correct answer. Because the correct answer is too low likelihood and often the RLHF has decreased the distribution in that region to get to it. Because this is exactly what RLHF does, on purpose.
More from the same topics.
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan +11
Artificial intelligence (AI) researchers have been developing and refining large language models (LLMs) that exhibit remarkable capabilities across a variety of domains and tasks, challenging our understanding of learning and cognition.
Tom B. Brown, Benjamin Mann, Nick Ryder +28
Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task.
Chiyuan Zhang, Samy Bengio, Moritz Hardt +2
Despite their massive size, successful deep artificial neural networks can exhibit a remarkably small difference between training and test performance.
Mariusz Bojarski, Davide Del Testa, Daniel Dworakowski +10
We trained a convolutional neural network (CNN) to map raw pixels from a single front-facing camera directly to steering commands.