Neutral mention · rule
That works if the LLM has adequate external feedback from a terminal and browser in context with the past trial etc. It can't self-correct its own reasoning: https://arxiv.org/abs/2310.01798
Back to the collection
Large Language Models Cannot Self-Correct Reasoning Yet
Jie Huang, Xinyun Chen, Swaroop Mishra
Show all 7 authors
Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, Denny Zhou
2023 · arXiv
- HN score
- 0
- Mentions
- 4
- Readers
- 4
- Months
- 3
From the abstract
Large Language Models (LLMs) have emerged as a groundbreaking technology with their unparalleled text generation capabilities across various applications. Nevertheless, concerns persist regarding the accuracy and appropriateness of their generated content. A contemporary methodology, self-correction, has been proposed as a remedy to these issues. Building upon this premise, this paper critically examines the role and efficacy of self-correction within LLMs, shedding light on its true potential and limitations. Central to our investigation is the notion of intrinsic self-correction, whereby an LLM attempts to correct its initial responses based solely on its inherent capabilities, without the crutch of external feedback. In the context of reasoning, our research indicates that LLMs struggle to self-correct their responses without external feedback, and at times, their performance even degrades after self-correction. Drawing from these insights, we offer suggestions for future research and practical applications in this field.
Metadata: arXiv. Abstracts retain their original authors’ rights.
In the conversation
Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels
4 selected excerpts · 4 total mentions
Neutral mention · rule
Self-reflection and chain-of-thought are different things; CoT is about writing out the intermediate steps to an answer it can't immediately reach. Self-reflection doesn't work well though: https://arxiv.org/abs/2310.01798
Neutral mention · rule
> The implications are vast. We may be able to translate between languages that have never had a “Rosetta Stone”. Any animals that have a true language could have it decoded. And while an LLM that’s gotten an 8 year old’s understanding of balancing assorted items isn’t that useful, an LLM that’s got a baby whale’s grasp on whale language would be revolutionary. Given that almost no-one trusts these LLMs for anything serious beyond creative / BS generating uses, this article does a lot of glossing over the reliability of their outputs and focuses more on such utopian promises with claims of 'understanding' which is in this article is beyond absurd and nonsensical. The…
Read the full comment
reality of LLMs is that they cannot reason. [0] It is known that LLMs perform incredibly worse in mathematical reasoning and understanding which it consistently hallucinates the wrong answers [1] and even when you tell it to correct itself, it pretends to identify its mistake and regurgitates another incorrect answer and its answers get even worse further down the conversation. [2] Of course to the typical user, it's easy to be fooled by a stochastic parrot pretending to demonstrate reasoning. But experts understand that these systems cannot transparently explain themselves and are AI sophists which fundamentally cannot be trusted. [0] https://arxiv.org/pdf/2302.08399.pdf [1] https://arxiv.org/pdf/2301.13867.pdf [2] https://arxiv.org/pdf/2310.01798.pdf
Neutral mention · rule
The paper: https://arxiv.org/pdf/2310.01798.pdf > Our research shows that LLMs are not yet capable of self-correcting their reasoning The paper actually just shows that the particular "self-correction" strategy and set of prompts they used doesn't help for the tasks they looked at, for the models they looked at. It may be the case in general, but it may not. > it is plausible that there exist specific prompts or strategies that could enhance the reasoning performance of models for particular benchmarks Seems they agree. So the wording of the title/conclusion is too strong. > searching such prompts or strategies may inadvertently rely on external feedback, either from…
Read the full comment
human insights or training data I'm not sure this justifies picking a single prompting strategy, and not looking at the impact of different prompting strategies. Even just writing a few different prompts in advance with different wordings and showing the variation in results would have been helpful.
Keep following the thread.
More in Language modelsMore from the same topics.
2023arXiv
Sparks of Artificial General Intelligence: Early experiments with GPT-4
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan +11
Artificial intelligence (AI) researchers have been developing and refining large language models (LLMs) that exhibit remarkable capabilities across a variety of domains and tasks, challenging our understanding of learning and cognition.
60 mentions14 months
3.5HN score
2020arXiv
Language Models are Few-Shot Learners
Tom B. Brown, Benjamin Mann, Nick Ryder +28
Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task.
83 mentions35 months
3HN score
2025arXiv
Design Patterns for Securing LLM Agents against Prompt Injections
Luca Beurer-Kellner, Beat Buesser, Ana-Maria Creţu +11
As AI agents powered by Large Language Models (LLMs) become increasingly versatile and capable of addressing a broad spectrum of tasks, ensuring their security has become a critical challenge.
4 mentions4 months
2.5HN score
2023arXiv
Toolformer: Language Models Can Teach Themselves to Use Tools
Timo Schick, Jane Dwivedi-Yu, Roberto Dessì +5
Language models (LMs) exhibit remarkable abilities to solve new tasks from just a few examples or textual instructions, especially at scale.
24 mentions5 months
2HN score