Skip to content
The methodOpen data
Back to the collection

Banishing LLM Hallucinations Requires Rethinking Generalization

Johnny Li, Saksham Consul, Eda Zhou

Show all 12 authors

James Wong, Naila Farooqui, Yuxin Ye, Nithyashree Manohar, Zhuxiaona Wei, Tian Wu, Ben Echols, Sharon Zhou, Gregory Diamos

2024 · arXiv

Language modelsMachine learningNetworks
Read the paper
HN score
0
Mentions
5
Readers
3
Months
4
What’s in a score?

From the abstract

Despite their powerful chat, coding, and reasoning abilities, Large Language Models (LLMs) frequently hallucinate. Conventional wisdom suggests that hallucinations are a consequence of a balance between creativity and factuality, which can be mitigated, but not eliminated, by grounding the LLM in external knowledge sources. Through extensive systematic experiments, we show that these traditional approaches fail to explain why LLMs hallucinate in practice. Specifically, we show that LLMs augmented with a massive Mixture of Memory Experts (MoME) can easily memorize large datasets of random numbers. We corroborate these experimental findings with a theoretical construction showing that simple neural networks trained to predict the next token hallucinate when the training loss is above a threshold as it usually does in practice when training on internet scale data. We interpret our findings by comparing against traditional retrieval methods for mitigating hallucinations. We use our findings to design a first generation model for removing hallucinations -- Lamini-1 -- that stores facts in a massive mixture of millions of memory experts that are retrieved dynamically.

Metadata: arXiv. Abstracts retain their original authors’ rights.

In the conversation

Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels

5 selected excerpts · 5 total mentions

nickpsecurity
Neutral mention · rule
I forgot one tech that’s actually commercialized by lamini.ai: https://arxiv.org/abs/2406.17642 https://medium.com/pythons-gurus/mixture-of-memory-experts-l... Re your reply I’ll add two things to “drive us to models that have emergent, reasoning abilities:” 1. It didn’t have that until they spent around $30 million pre-training a specific architecture (GPT3-176B) on TB’s of data with portions having built-in, reasoning steps it could extrapolate. Doing the same with the other architectures might have led to similar, better, or worse results. GPT3’s training investment, not Transformer architecture itself, led to seeing them as reasoning engines. 2. The memory-less…
Read the full comment
architectures also had lots of hallucinations. Another submission said they’re like professional liars who also don’t know when they’re lying. It used to hurt me a lot with even GPT4-generated code. What reduces that, especially simply using huge models, drives pre-training cost back up to exorbitant levels. So, if you have tens of millions, you can pretrain models for reasoning that also confidently lie to us. The open question is whether that’s built into the prevailing architectures. I think it is given brains’ performance. Also, how specific components of brains being damaged causes hallucinations. Brain architecture has layers of mitigations for it. If integrating memory, we might see architectures with few to no hallucinations. That can be combined with other techniques already in use. They might also be cheaper to train, too, if they build on what they remember. That’s a bigger maybe.
Original comment Memory makes computation universal, remember?
Animats
Neutral mention · rule
Comments on that paper? PDF: [1] What they are measuring, it seems, is whether LLMs can be built which will retrieve a reliable known correct answer on request. That's an information retrieval problem, and, in fact, they solve it by adding "Memory Experts" which are basically data storage. It's not clear that this helps either replies which require synthesizing disparate information, or detecting that the training data does not contain info needed to construct a reply. [1] https://arxiv.org/pdf/2406.17642
Original comment OpenAI, Google and Anthropic are struggling to build more advanced AI
gdiamos
Neutral mention · rule
Disagree - https://arxiv.org/abs/2406.17642 We cover halting problem and intractable problems in the related work. Of course LLMs cannot give answers to intractable problems. I also don’t see why you should call an answer of “I cannot compute that” to a halting problem question a hallucination.
Original comment LLMs Will Always Hallucinate, and We Need to Live with This
nickpsecurity
Neutral mention · rule
The brain solves that problem. It seems to involve memory and specialized regions. I found a few groups building hippocampus-like, research models. One had content-addressable memory. There was another one that claimed to get rid of hallucinations. They also said it takes 50-100 epochs for regular architectures to actually memorize something. Their paper is below in case people qualified to review it want to. https://arxiv.org/abs/2406.17642 Like the brain, I believe the problem will be solved by a mix of specialized components working together. One of those components will be a memory (or series of them) that the others reference to keep processing grounded in reality.
Original comment OpenAI, Google and Anthropic are struggling to build more advanced AI
gdiamos
Neutral mention · rule
We know that LLMs hallucinate, but we can also remove them. I’d love to see a future generation of a model that doesn’t hallucinate on key facts that are peer and expert reviewed. Like the Wikipedia of LLMs https://arxiv.org/pdf/2406.17642 That’s a paper we wrote digging into why LLMs hallucinate and how to fix it. It turns out to be a technical problem with how the LLM is trained.
Original comment Anthropic publishes the 'system prompts' that make Claude tick

Keep following the thread.

More in Language models

More from the same topics.