Faith and Fate: Limits of Transformers on Compositionality

the o series of models came out, and basically proved this point moot. Turns out RL + long context mitigate this fairly well. And a year later, we have all SotA models being able to "solve" problems 100k+ tokens deep. If you go hands on in any decent size codebase with an agent session length and context size become noticeable issues. Again, mathematically error propagation eventually leads to a 100% chance of error. Yann isn't wrong here, we've just kicked the can a little further down the road. What happens at 200k+ tokens? 500k+ tokens? 1M tokens? The underlying issue of a stochastic system isn't addressed. >While Yann is clearly brilliant, and has a deeper understanding of the roots of the filed than many of us mortals, I think he's been on a debbie downer trend lately As he should be. Nothing he said was wrong at a fundamental level. The transformer architecture we have now cannot scale with task complexity. Which is fine, by nature it was not designed for such tasks. The problem is that people see these models work on a subset of small scope complex projects and make claims that go against the underlying architecture. If a model is 'solving' complex or planning tasks but then fails to do similar tasks at a higher complexity it's a sign that there is no underlying deterministic process. What is more likely: the model is genuinely 'planning' or 'solving' complex tasks, or that the model has been trained with enough planning and task related examples that it can make a high probability guess? > So, yeah, I'd take everything any one singular person says with a huge grain of salt. No matter how brilliant said individual is. If anything, a guy like Yann with a role such as his at a Mag7 company being realistic (bearish if you are a LLM evangelist) about what the transformer architecture can do is a relief. I'm more inclined to listen to him than a guy like Altman who touts LLMs as the future of humanity meanwhile is path to profitability is AI Tik-Tok, sex chatbots, and a third party way to purchase things from Walmart during a recession.

you're working on, but while I'm not curing cancer, I am solving problems that aren't in the training data and can't be found on Google. Just a few days ago, Gemini 2.5 Pro literally told me it didn’t know what to do and asked me for help. The other models hallucinated incorrect answers. I solved the problem in 15 minutes. If you're working on yet another CRUD app, and you've never implemented transformers yourself or understood how they work internally, then I understand why LLMs might seem like magic to you.

and GNNs have foundational limits to expressivity [3, 4, 5], if the scaling crowd could show strong generalization results on these sorts of problems they would have the last laugh... [1]: https://arxiv.org/pdf/2210.10749.pdf [2]: https://arxiv.org/pdf/2305.18654.pdf [3]: https://arxiv.org/pdf/2106.16213.pdf [4]: https://arxiv.org/pdf/2301.10743.pdf [5]: https://arxiv.org/pdf/2303.04613.pdf

intermediate sub-procedures. Our empirical findings suggest that Transformers solve compositional tasks by reducing multi-step compositional reasoning into linearized subgraph matching, without necessarily developing systematic problem-solving skills. https://arxiv.org/abs/2305.18654

do is to create a concept of programming language, I'm not talking about a new programming language, I'm talking about the concepts. > I'm sure that won't work for every problem of this sort, but the quality of tests people do on LLMs is really awful, at least because people tend to do very narrow tests like that and make broad pronouncements about what LLM's "can't" do based on it. Here, a few papers that show they can't reason: https://arxiv.org/abs/2311.00871 https://arxiv.org/abs/2309.13638 https://arxiv.org/abs/2311.09247 https://arxiv.org/abs/2305.18654 https://arxiv.org/abs/2309.01809