We explore how generating a chain of thought -- a series of intermediate reasoning steps -- significantly improves the ability of large language models to perform complex reasoning. In particular, we show how such reasoning abilities emerge naturally in sufficiently large language models via a simple method called chain of thought prompting, where a few chain of thought demonstrations are provided as exemplars in prompting. Experiments on three large language models show that chain of thought prompting improves performance on a range of arithmetic, commonsense, and symbolic reasoning tasks. The empirical gains can be striking. For instance, prompting a 540B-parameter language model with just eight chain of thought exemplars achieves state of the art accuracy on the GSM8K benchmark of math word problems, surpassing even finetuned GPT-3 with a verifier.
Metadata: arXiv. Abstracts retain their original authors’ rights.
In the conversation
Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels
It's not impossible to see that LLMs can make finite step "reasoning" between input and output, as each block of the transformer can model probabilistic causality. Transformers are sometimes considered a fully connected graph neural network, which can be used for modeling causal graphs. One additional supporting evidence on the finite step reasoning hypothesis is that "train of thought" improves the performance of LLMs[1], meaning that letting the model explain itself explicitly reduces the amount of implicit "reasoning" steps that need to happen within the model.
Additionally, while we don't know how us humans functionally reason, it's believed that the predictive nature…
Read the full comment
of our brain is central to our reasoning abilities. Maybe in some way, the autoregressive nature of LLMs is similar to our predictive brain.
For a collection of emergent properties in LLMs, I recommend this paper [2]
[1]https://arxiv.org/abs/2201.11903.pdf
[2]https://arxiv.org/pdf/2206.07682.pdf
Reasoning improves the output of the model. Originally it was a prompting method called chain of thought but later models were trained to do the reasoning steps without the human prompting for it. Being able to see the reasoning steps is just an accidental benefit.
https://arxiv.org/abs/2201.11903
My biggest gripe with the discourse around AI, especially by programmers with hubris about Machine Learning, is the idea that LLMs can’t come up with “novel solutions”. They can, and they have. CoT[0] is how LLMs can output tokens in “reasoning space” to guide their “thinking” to produce absolutely novel solutions. You can imagine reasoning being multi-layered, where the top layer is an abstract heuristic (examples of which can be “try special cases”, “try solving a part of the problem with relaxed constraints”). The lower layers become more and more concrete with the details of the problem, and the result is a solution of the problem.
You don’t even have to understand…
Read the full comment
how modern reasoning LLMs work to be able to tell that your perception is warped and doesn’t reflect reality - there’s plenty of news to the contrary - OpenAI resolving a major Erdos problem[1], the First Proof endeavour[2], amongst others [3].
[0]: https://arxiv.org/abs/2201.11903
[1]: https://openai.com/index/model-disproves-discrete-geometry-c...
[2]: https://1stproof.org/assets/docs/report.pdf
[3]: https://archive.ph/2w4fi
Isn’t this sort of just “chain of thought” (i.e. the seminal https://arxiv.org/abs/2201.11903 ) where the user is helping the model 1-shot or k-shot the solution instead of 0-shot? I’ve used a similar technique to great effect. I feel things are so new / moving so fast that it’s hard to have common lingo. So very helpful to have a blog / example! But I wonder if the phenomena has been seen / understood before and just in smaller circles / different name.
AI from first principles has not changed. Fundamentally it is: neural nets, transformers and RL. The most important paper in recent years is on CoT [https://arxiv.org/pdf/2201.11903] and I'm not even sure what comes close.
And I think what's more important these days is knowing how to filter the noise from the signal.
This is probably one of the better blogs I have read recently that shows the general direction currently in AI which are improvements on the generator / verifier loop: https://www.julian.ac/blog/2025/11/13/alphaproof-paper/
Yeah. One example of people mindlessly mass citing some random paper is this: Chain of thought (CoT) prompting was used in the past to greatly enhance the reasoning ability of LLMs. Usually this paper is cited when CoT is discussed:
https://arxiv.org/abs/2201.11903
It has over 20,000 citations according to Google Scholar. But clearly the technique was not invented by these authors. It was known 1.5 years earlier, just after GPT-3 came out:
https://xcancel.com/kleptid/status/1284069270603866113#m
Perhaps even longer. But the paper above is cited nonetheless. Probably because there is pressure to cite something and the title of that paper sounds like they pioneered it. I…
Read the full comment
doubt many people who cite it have even read it.
Another funny example is that in machine learning and some other fields, a success measure named "Matthews Correlation Coefficient" (MCC) is used. It's named after some biochemist, Brian Matthews, who used it in a paper from 1975. Needless to say, he didn't invent it at all, he just used the well-known binary version of the well-known correlation coefficient. People who named the measure "MCC" apparently thought he invented it. Matthews probably just didn't bother to cite any sources himself.
I'm not much of an ML engineer but I can point you to the original chain of thought paper [0] and Anthropic's docs on how to enable their official thinking scratchpad [1].
[0] https://arxiv.org/pdf/2201.11903
[1] https://docs.anthropic.com/en/docs/build-with-claude/extende...
Chain of Thought was known since 2022 (https://arxiv.org/abs/2201.11903), we just were stuck in a world where we were dumping more data and compute at the training instead of looking at other improvements.
Of course Prolog is no magic but I'm sure you know the argument in favor of having LLMs generate Prolog are based on the observation that prompting or otherwise making an LLM to perform chain-of-thought reasoning results in demonstratable experimental improvements, with [1] the canonical paper, and using Prolog with its unique characteristics and roots in NLP and logic an extension of that idea termed program-as-thought. OpenAI's latest o1 model makes heavy use of CoD internally until it returns an answer to the user.
[1]: https://arxiv.org/abs/2201.11903
No. CoT has been around for several years (Jan 2022) https://arxiv.org/abs/2201.11903. And so has Reflection (March 2023) https://arxiv.org/abs/2303.11366. This approach that was taken by Reflection is nothing new.
That prompts aren't science means little. If anything it makes them more important because you can't systematically arrive at good ones.
If one spends a lot of time building an application to achieve an actual goal they'll realize the prompts make a gigantic difference and it takes an enormous amount of fiddly, annoying work to improve. I do this (and I built an agent system, which was more straightforward to do...) in financial markets. It so much so that people build systems just to be able to iterate on prompts (https://www.promptlayer.com/).
I may be wrong - but I'll speculate you work on infra and have never had to build a (real) application that is trying to achieve…
Read the full comment
a business outcome. I expect if you did, you'd know how much (non sexy) work is involved on prompting that is hard to replicate.
Hell, papers get published that are just about prompting!
https://arxiv.org/abs/2201.11903
This line of thought effectively led to Gpt-4-o1. Good prompts -> good output -> good training data -> good model.
> as part of/after the main training
I take this to mean during weight updates, e.g. training.
> "runtime train of thought"
I take runtime here to mean inference, not during RL. What does runtime mean to you?
Previous approaches [0] successfully used inference time chain of thought to improve model responses. That has nothing to do with RL though.
The grandparent is wrong about the paper. They are doing chain of thought responses during training and doing RL on that to update the weights, not just during inference/runtime.
[0] https://arxiv.org/abs/2201.11903
My understanding is this used to be the case[1] but isn't really true any longer due to things like the "star" method for model training[2]. Empirically it absolutely (circa GPT3) used to be the case that if you prompted with "Explain all your reasoning step by step and then give the answer at the end" or similar it would give you a better answer for a complex question than if you said "Just give me the answer and nothing else" or similar, or asked for the answer first, and then circa gpt-4 answers started getting much longer even if you asked the model to be concise.
That doesn't seem to be the case any more and there has been speculation this is down to the star method…
Read the full comment
being used for training newer models. I say speculation because I don't believe people have come out and said they are using star for training. OpenAI referred to Q* somewhere but they wouldn't be drawn on whether that * is this "star" and although google were involved in publishing the star paper they haven't said gemini uses it (I don't think).
[1] https://arxiv.org/abs/2201.11903
[2] https://arxiv.org/pdf/2203.14465
Justification before decision still works out better in practice, though, because of chain of thought [1]. You'll tend to get more accurate and better-justified decisions.
With decision before justification, you tend to have a greater risk of the output being a wrong decision followed by convincing BS justifying it.
(edit: Another way you could think of it is, LLMs still can't violate causality. Attention heads' ability to look in both directions with respect to a particular token's position in the sequence does not enable them to see into the future and observe tokens that don't exist yet.)
1: https://arxiv.org/abs/2201.11903
I've found that for tasks like this you need to ask the model to output its reasoning prior to outputting the "answer".
See "chain of thought" e.g
https://arxiv.org/abs/2201.11903
I didn't have time to read this, but it is a single author paper, the author is not affiliated with a research group, it is not peer reviewed, it was published on a preprint server that I have never heard of.
LLMs can definitely perform some kinds of reasoning. For example GSM8K is a dataset of grade school math problem requiring reasoning that LLMs are typically evaluated at. We talk about one method for this in our chain of thought paper [1]
[1] https://arxiv.org/abs/2201.11903
Chain of thought: https://arxiv.org/abs/2201.11903
ReAct: https://arxiv.org/abs/2210.03629
Reflexion: https://arxiv.org/abs/2303.11366
Tree of thoughts: https://arxiv.org/abs/2305.10601
Good video on "Tree of thoughts" which also reviews / puts it in the context of other methods: https://www.youtube.com/watch?v=ut5kp56wW_4
Completion vs conversational interface is something you can read about in the OpenAI API documentation.
For the remaining things I don't have single specific pointer at hand.
They’re referring to a method of prompting the model that encourages it to think through something step by step rather than spit out the answer.
I think this is the paper that really kicked off this technique: https://arxiv.org/abs/2201.11903
Just speculating, but this looks like precisely the sort of result one might expect from fine-tuning models on chain-of-thought-prompted interactions like those described by Wei et al. in 2022[1]. Alternatively, as Madaan et al. show in their 2022 paper[2], this may simply be the result of larger language models having seen more code, and thus structured reasoning, in their training data.
[1] https://arxiv.org/abs/2201.11903
[2] https://arxiv.org/abs/2210.07128
That is important, but asking in certain ways can make a big difference. E.g. https://twitter.com/mitchellh/status/1645562198935347205. And just putting “let’s think step by step” at the end can really help some of the larger GPT models. https://arxiv.org/abs/2201.11903
Artificial intelligence (AI) researchers have been developing and refining large language models (LLMs) that exhibit remarkable capabilities across a variety of domains and tasks, challenging our understanding of learning and cognition.
Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task.
As AI agents powered by Large Language Models (LLMs) become increasingly versatile and capable of addressing a broad spectrum of tasks, ensuring their security has become a critical challenge.