Skip to content
The methodOpen data
Back to the collection

Towards Understanding Grokking: An Effective Theory of Representation Learning

Ziming Liu, Ouail Kitouni, Niklas Nolte

Show all 6 authors

Eric J. Michaud, Max Tegmark, Mike Williams

2022 · arXiv

PhysicsMachine learningPsychology
Read the paper
HN score
1
Mentions
4
Readers
3
Months
4
What’s in a score?

From the abstract

We aim to understand grokking, a phenomenon where models generalize long after overfitting their training set. We present both a microscopic analysis anchored by an effective theory and a macroscopic analysis of phase diagrams describing learning performance across hyperparameters. We find that generalization originates from structured representations whose training dynamics and dependence on training set size can be predicted by our effective theory in a toy setting. We observe empirically the presence of four learning phases: comprehension, grokking, memorization, and confusion. We find representation learning to occur only in a "Goldilocks zone" (including comprehension and grokking) between memorization and confusion. We find on transformers the grokking phase stays closer to the memorization phase (compared to the comprehension phase), leading to delayed generalization. The Goldilocks phase is reminiscent of "intelligence from starvation" in Darwinian evolution, where resource limitations drive discovery of more efficient solutions. This study not only provides intuitive explanations of the origin of grokking, but also highlights the usefulness of physics-inspired tools, e.g., effective theories and phase diagrams, for understanding deep learning.

Metadata: arXiv. Abstracts retain their original authors’ rights.

In the conversation

Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels

4 selected excerpts · 4 total mentions

sinenomine
Recommendation · rule
Ironic, since the relatively recently discovered double descent makes it clear that bias-variance tradeoff as we know it from statistical learning theory simply doesn't apply to "overparameterized" deep models. Much of old theory is barely applicable and people are, understandably, bewildered and in denial. If someone were to be inclined to theory, I'd just recommend reading papers that don't try oversimplify the domain: https://arxiv.org/abs/2006.15191 https://arxiv.org/abs/2210.10749 https://arxiv.org/abs/2205.10343 https://arxiv.org/abs/2105.04026
Original comment Ask HN: What are the foundational texts for learning about AI/ML/NN?
Imnimo
Neutral mention · rule
Grokking may not even occur for datasets of that scale. Even the MNIST experiments require dropping the training data size from 50k examples to 1k. The reason for this is that the phenomenon seems to occur at a critical zone of having just barely enough training data to make generalization possible. See https://arxiv.org/abs/2205.10343 for details. Even figuring out how to induce grokking behavior on a 100M model or OpenWebText would be a big leap in the understanding of grokking. It's perfectly reasonable for a paper like this to show results on the standard tasks for which grokking has already been characterized.
Original comment Grokfast: Accelerated Grokking by Amplifying Slow Gradients
versteegen
Neutral mention · rule
> It will never learn math this way, no matter how much training you give it. Not so. Actually, (for example) the phenomenon of "grokking" is when with enough training a NN eventually experiences a phase-change from memorising data to learning the general rules underlying it [1]. Grokking isn't actually desirable, it's better that the model go more directly and quickly to learning the general rule, which is achievable in toy problems (called "comprehension" in [2]). I feel that people seem to have forgotten that deep learning is so powerful because it performs feature/representation learning, not because it can memorise, although that's powerful too. IMO that is the…
Read the full comment
proper definition of 'deep learning'. [1] Power &al. Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets https://arxiv.org/abs/2201.02177 [2] Liu &al. Towards Understanding Grokking: An Effective Theory of Representation Learning https://arxiv.org/abs/2205.10343
Original comment Understanding the limits of large language models
sinenomine
Neutral mention · rule
It is common to switch to a conservative mode of thought upon entering one's domain of competence; The question is, though, if the expertise and intuition developed during, say, running XGboost classifiers at scale in AdTech really of much relevance when thinking about large transformer models trained with a self-supervised objective and RLHF? If you try to study this in depth, these models can do something the usual "datascience"-tier ones commonly cannot: https://arxiv.org/abs/2205.10343 https://moultano.wordpress.com/2020/10/18/why-deep-learning-...
Original comment How Does GPT Obtain Its Ability? Tracing Emergent Abilities of Language Models

Keep following the thread.

More in Physics

More from the same topics.

2018arXiv

Talent vs Luck: the role of randomness in success and failure

A. Pluchino. A. E. Biondo, A. Rapisarda

The largely dominant meritocratic paradigm of highly competitive Western cultures is rooted on the belief that success is due mainly, if not exclusively, to personal qualities such as talent, intelligence, skills, efforts or risk taking.

Psychology
16 mentions14 months
2.5HN score