Skip to content
The methodOpen data
Back to the collection

Less is More: Recursive Reasoning with Tiny Networks

Alexia Jolicoeur-Martineau

2025 · arXiv

Machine learningLanguage modelsBiology & medicine
Read the paper
HN score
0
Mentions
3
Readers
3
Months
3
What’s in a score?

From the abstract

Hierarchical Reasoning Model (HRM) is a novel approach using two small neural networks recursing at different frequencies. This biologically inspired method beats Large Language models (LLMs) on hard puzzle tasks such as Sudoku, Maze, and ARC-AGI while trained with small models (27M parameters) on small data (around 1000 examples). HRM holds great promise for solving hard problems with small networks, but it is not yet well understood and may be suboptimal. We propose Tiny Recursive Model (TRM), a much simpler recursive reasoning approach that achieves significantly higher generalization than HRM, while using a single tiny network with only 2 layers. With only 7M parameters, TRM obtains 45% test-accuracy on ARC-AGI-1 and 8% on ARC-AGI-2, higher than most LLMs (e.g., Deepseek R1, o3-mini, Gemini 2.5 Pro) with less than 0.01% of the parameters.

Metadata: arXiv. Abstracts retain their original authors’ rights.

In the conversation

Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels

3 selected excerpts · 3 total mentions

SamCoding
Neutral mention · rule
Referring to this paper: https://arxiv.org/pdf/2510.04871 My understanding: The Q-head predicts whether to stop early or not to reduce training times (by not always having to complete the full number of deep supervision steps). The complication: When working in batches: - Either, the Q-head is computed over the entire batch and averaged (which could early stopping of reasoning steps when most items in batch are complete but some aren't, meaning that those incomplete samples aren't trained on fully) - Or, the Q-head is computed over entire batch and samples that are complete are dropped. The first option has issues obvious as discussed. The second option is problematic…
Read the full comment
because it's complex to implement and frameworks tend to want fixed sizes for batches. If we go with just running for the maximum number of deep supervision steps every time to avoid these issues, the Q-head becomes useless (assuming you don't want to use it during inference). I really appreciate any help anyone can give me in understanding how they get around this, either in the original paper or in practice in more recent research.
Original comment How does Q-head work in batch training in Tiny Recursive Networks?
m_ke
Neutral mention · rule
Yes it would require completely new hardware and most likely ditching gradient descent for alternative optimization methods, though I'm not convinced that we'd need to turn to discrete optimization. Some recent works that people might find interesting: - Evolution Strategies at the Hyperscale - https://eshyperscale.github.io/ - Introducing Nested Learning: A new ML paradigm for continual learning - https://research.google/blog/introducing-nested-learning-a-n... - Less is More: Recursive Reasoning with Tiny Networks - https://arxiv.org/abs/2510.04871 - Nemotron Elastic: Towards Efficient Many-in-One Reasoning LLMs - https://arxiv.org/abs/2511.16664
Original comment Weight-sparse transformers have interpretable circuits [pdf]

Keep following the thread.

More in Machine learning

More from the same topics.