Teaching Algorithmic Reasoning via In-context Learning

> But it fails at simple arithmetic. Does it though? When allowing LLMs to use their outputs as a form of state they can very much succeed up to 14 digits with > 99.9% accuracy, and it goes up to 18 without deteriorating significantly [1]. That really isn't a good argument because you are asking it to do one-shot something that 99.999% of humans can't. https://arxiv.org/abs/2211.09066

Yeah the issue tends to be as the length increases the performance drops off. Your approach reminds me of Hattie's work: https://arxiv.org/pdf/2211.09066.pdf I do wonder if the observable drop-off in performance on these formal languages / arithmetic tasks is something more fundamental, and related to the attention mechanism; namely that it can't attend further back as the input gets longer (possibly due to the way attention scores are computed).

> Yet if you define the algorithmic steps to perform addition on 2 numbers, accuracy on addition arithmetic shoots up to 98% even on very large numbers. https://arxiv.org/abs/2211.09066 Think about what that means. That means that even with the giant model, you need to stuff even the most basic knowledge for dealing with problems of that class into the prompt space to get it to work, cutting into conversation depth and per-response size? The advantage of GPT-4’s big window and the opportunity it provides for things like retrieval and deep iterative context shrinks if I’ve got to stuff a domain textbook into the system prompt so it isn’t just BSing me.

> “LLMs can’t even do addition” > We show that they CAN add! To teach algos to LLMs, the trick is to describe the algo in enough detail so that there is no room for misinterpretation Algorithmic prompting paper https://arxiv.org/pdf/2211.09066.pdf

LLMs are a different beast in the ML world. Finetuned Palm for Medicine and Finetuned Minerva for Math all perform a good deal worse than GPT-4. A fine-tuned smaller model is by no means guaranteed to beat a larger more general one (though of course you may get acceptable performance). And then the necessity of fine-tuning itself is called into question plenty with LLMs. https://huggingface.co/papers/2308.00304 https://huggingface.co/papers/2308.07921 https://arxiv.org/abs/2211.09066

>No, but I believe this is a different question. arithmetic but in a different base is one of the counterfactual examples in the paper. That's why i mentioned that. and yes it can answer them with worse performance. You can juice arithmetic performance as is with algorithmic instructions. https://arxiv.org/abs/2211.09066. I see no reason the same for other bases wouldn't work. Even if you gave a human substantial time for it (say a week of study), i believe he/she almost certainly reach the same accuracy unless he had access to specific instructions for working in base 8 he/she could call upon when taking the test.

It's alright with algorithmic prompts - https://arxiv.org/abs/2211.09066 also it knows when to use a calculator if it has access to one so it's not a big deal

Post moving. "Reasonable" is just an arbitrary line. Especially since most if not all would make some mistake somewhere along the line. You can greatly increase GPT's arithmetic capabilities tackling it like a problem to solve "on paper" in context. And this was done on 3.5 not 4. https://arxiv.org/abs/2211.09066

I don't think you took more than a passing glance, if any at those papers. What you describe is impossible with these 3. https://arxiv.org/abs/2212.09196 - new evaluation set introduced with the paper. modelled after tests that previously only had visual equivalents. contamination literally impossible https://arxiv.org/abs/2204.02329 - effect of explanations on questions introduced with the paper. dataset concerns make no sense. https://arxiv.org/abs/2211.09066 - new prompting method introduced to improve algorithmic calculations. dataset concerns make no sense. The Casual paper is the only one where worries about dataset contamination makes any sense at all.

>It's not better at reasoning. It's barely even capable of it You are wrong. and there's many papers to show otherwise. Algorithmic, Casual, Inference, Analogical LLMs reason just fine https://arxiv.org/abs/2212.09196 https://arxiv.org/abs/2305.00050 https://arxiv.org/abs/2204.02329 https://arxiv.org/abs/2211.09066

No world model ? A world model is so obvious papers like these are more confirmation than surprise https://arxiv.org/abs/2305.11169 https://arxiv.org/abs/2210.13382 There a certain sentiment that AGI however you wish to define it won't infact be a "We'll know it when we see it" situation but rather a "AGI will arrive long before consensus reaches its AGI". LLMs have made me believe this will 100% be the case, either way. It's one thing to argue over things we can't evaluate even now but man the 100th "They can't reason!" every week is pretty funny when you can basically take your pick of reasonong type - Algorithmic, Casual, Inference, Analogical and read a paper showing strong performance. https://arxiv.org/abs/2212.09196 https://arxiv.org/abs/2305.00050 https://arxiv.org/abs/2204.02329 https://arxiv.org/abs/2211.09066 People refuse to see even what is staring right at them.

LLMs are not very bad at reasoning. People say this but struggle to demonstrate how. There are many types of reasoning. For some like abstract analogical reasoning, LLMs are top % performers. https://arxiv.org/abs/2212.09196 There's many papers at this point demonstrating pretty strong reasoning. From Casual Reasoning, and explanatory reasoning to algorithmic reasoning. https://arxiv.org/abs/2305.00050 https://arxiv.org/abs/2204.02329 https://arxiv.org/abs/2211.09066 The only kind of reasoning that's somewhat lacking is logical reasoning. And even then it's not awful, just lagging behind equivalent human performance. So someone really needs to tell where this oft repeated "can't reason" rhetoric is coming from. Because the numbers sure as hell don't support it.

Not too shocking for me after this paper. https://arxiv.org/abs/2211.09066 You can teach GPT-3 arithmetic - https://imgur.com/a/w3DAYOi 100% accuracy up to about 13 digit addition and >90 after that. What else can you teach GPT without changing weights ?

We are already there. 100% accuracy on up to 13 digit addition can be taught to 3.5 as is. https://arxiv.org/abs/2211.09066 And 4 has little need for such out the box

In context learning is hands down the biggest breakthrough of LLMs. The flexibility the model displays without updating weights is genuinely mind blowing. See here - https://imgur.com/a/w3DAYOi from the paper - https://arxiv.org/abs/2211.09066 GPT's addition accuracy tanks after 2 digits. However, simply by describing arithmetic as an algorithm to be performed, you can supercharge accuracy to basically 100% for up to 13 digit addition and >90% after.

I didn't say it's not there. or that you couldn't communicate the ideas of math/arithmetic with language. You can get GPT-3 (yes 3) to have 98.5% accuracy on addition arithmetic (even very large numbers) by..simply describing the algorithm of addition to performed on 2 numbers. https://arxiv.org/abs/2211.09066 The basic idea i'm communicating is that not everything that can be extracted from self supervisory token prediction can be extracted with the same ease. and it is very easy to see why math is one the higher difficulty things. It is extremely easy comparatively to infere "happy" from the sentence - John is smiling therefore he is ----. all the information you need to make that inference is packed tightly in the preceding words. it is not the same for arithmetic at all.

Oh boy, if you want to fling papers/research, i can do that just fine. https://arxiv.org/abs/2212.09196v1 https://arxiv.org/abs/2211.09066 https://thegradient.pub/othello/ Moreover, "have trouble reasoning" and "can't reason at all" are two very different statements. Not that i particularly agree with either but you should keep that in mind.