Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets

>There are grammar rules, they are just very weak because the structure of human language is generally quite weak. When presented with languages which have strong consistent grammars the weights are very easily interpretable as a grammar: https://arxiv.org/abs/2201.02177 That paper did not train the models on 'a language with strong consistent grammars'. Mathematical Operation tables are not a language by any definition of the word. Grammar itself is a post-hoc rationalization and there's no evidence LLMs follow 'grammar rules' anymore than the brain follows grammar rules.

It's not often I see something that's fractally wrong but here we are. There is a dictionary, it's called the tokenizer. There are grammar rules, they are just very weak because the structure of human language is generally quite weak. When presented with languages which have strong consistent grammars the weights are very easily interpretable as a grammar: https://arxiv.org/abs/2201.02177 The point of the original short story is that the computational substrate doesn't matter when you have Turing completeness. This one seems to think that you don't need structure and interpretability just because you change substrates.

Yes I did find that paper, I did not find which one of the 5 authors it was, or someone not listed as an author. The word 'vacation' is not in the paper. https://arxiv.org/pdf/2201.02177.

I don't think that is true? As far as I know the grokking phenomenon was first observed (and the name coined) in this paper, not in any blog post: https://arxiv.org/abs/2201.02177

> once a company creates a proto AGI where the path to a functional AGI is entirely predictable with more compute, I find it hard to believe this will happen. I expect AGI training to be more like a phase transition (or a bit like grokking https://arxiv.org/pdf/2201.02177.pdf)

I can see that. But are we sure that a rank-based difference that doesn't manifest early in the training process won't manifest as you get further along? See also 'grokking' [0] [0]: https://arxiv.org/abs/2201.02177

To some limited extent, they are already good at maths: https://ar5iv.labs.arxiv.org/html/2201.02177 This paper makes me believe it's less about the lack of ability to understand maths, but the power to learn more of it.

> ChatGPT doesn’t “comprehend” anything. You’re anthropomorphizing it. It wouldn't be a ChatGPT HN thread without someone claiming that LLMs are stupid predictors incapable of "real" understanding. Of course ChatGPT comprehends things. It does so under any useful definition of the word "comprehend". The "grokking" paper [1] shows that it learns fundamental principles of various algorithms and isn't just predicting text. If this isn't comprehension, humans aren't capable of comprehension. [1] https://arxiv.org/abs/2201.02177 (check out the citation list too)

This is not the consensus among ML researchers. Transformers are showing strong generalisation[1] and their performance continues to surprise us as they scale[2]. The Socratic paper is not about “higher intelligence”, it’s about demonstrating useful behaviour purely by connecting several large models via language. [1] https://arxiv.org/abs/2201.02177 [2] https://arxiv.org/abs/2204.02311

Sure: Deep Symbolic Regression for Recurrent Sequences https://arxiv.org/abs/2201.04600 (Interactive demo: http://recur-env.eba-rm3fchmn.us-east-2.elasticbeanstalk.com... ) Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets https://arxiv.org/abs/2201.02177 Both of these models can generalize to numbers it have not seen.

Take a look at this paper: Deep Symbolic Regression for Recurrent Sequences https://arxiv.org/abs/2201.04600 If you look at embedding visualization it is very clear that the model learns order of numbers. (Interactive demo: http://recur-env.eba-rm3fchmn.us-east-2.elasticbeanstalk.com... ) There is also: Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets https://arxiv.org/abs/2201.02177 Again, looking at visualizations the model very clearly grasps the structure of the function it models.