The notion of interpolation and extrapolation is fundamental in various fields from deep learning to function approximation. Interpolation occurs for a sample $x$ whenever this sample falls inside or on the boundary of the given dataset's convex hull. Extrapolation occurs when $x$ falls outside of that convex hull. One fundamental (mis)conception is that state-of-the-art algorithms work so well because of their ability to correctly interpolate training data. A second (mis)conception is that interpolation happens throughout tasks and datasets, in fact, many intuitions and theories rely on that assumption. We empirically and theoretically argue against those two points and demonstrate that on any high-dimensional ($>$100) dataset, interpolation almost surely never happens. Those results challenge the validity of our current interpolation/extrapolation definition as an indicator of generalization performances.
Metadata: arXiv. Abstracts retain their original authors’ rights.
In the conversation
Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels
The tricky part is that LLMs aren't just spewing outputs from the distribution (or "near" learned manifolds), but also extrapolating / interpolating (depending on how much you care about the semantics of these terms https://arxiv.org/abs/2110.09485).
There are genuine creative insights that come from connecting two known semantic spaces in a way that wasn't obvious before (e.g, novel isomorphism). It is very conceivable that LLMs could make this kind of connection, but we haven't really seen a dramatic form of this yet. This kind of connection can lead to deep, non-trivial insights, but whether or not it is "out-of-distribution" is harder to answer in this case.
I don't care much for the concept of AGI since it's ill defined.
However, out of distribution generation and generalization are a better more useful metric. In these Yann Lecun has argued that interpolation is meaningless in the high dimensional spaces these "curves" are embedded in.(https://arxiv.org/abs/2110.09485)
ARC has proven they can generalize, and Alpha Go (not an llm but a deep network) has proven it can generate novel/creative solutions. We don't need AI to have a sense of self "I" for them to beat us at every human skill and activity. Infact it might be detrimental and not useful for us if AI developed a sense of self.
The "statistical parrot" parrots have been demonstrably wrong for years (see e.g. LeCun et al[1]). It's just harder to ignore reality with hundreds of millions of people now using incredible new AI tools. We're approaching "don't believe your lying eyes" territory. Deniers will continue pretending that LLMs are just an NFT-level fad or bubble or whatever. The AI revolution will continue to pass them by. More's the pity.
[1] https://arxiv.org/abs/2110.09485
If you like ML, this is also related with the results of this paper [0], where they show that learning in high dimensions amounts to extrapolation, as opposed to interpolation. Intuitively I think of this as the fact that points in the sphere are convexly independent, and most of the volume of the ball is near the boundary.
[0] https://arxiv.org/abs/2110.09485
I hate this as being too wooish.
There first example, arithmetic, we know actually benefits from over fitting, but even then it isn't great.
We now know that over Parameterization and double decent is less lossy compression.
We know that feed forward networks are TC^0 and we know that CoT etc... are limited to polynomial reductions.
The fact that we can't predict exactly what is learned in a specific instance is purely a function of computational complexity theory.
Attention we know is equivalent to exhaustive search.
Yes interpretability is hard and math explains that.
We know that the curse of dimensionality means LLMs are using extrapolation as an…
Read the full comment
example.
https://arxiv.org/abs/2110.09485
This isn't meant to diminish the very impressive advances that have been made.
But it is not a complete god box, we can understand the underlying properties that lead to models being found.
But IMHO pretending that it is something that is magical seriously diminishes the hard work that was required to get where we are today.
"Supercharged Interpolation" is not a thing that exists sorry.
Learning in high dimensions always results in extrapolation.
https://arxiv.org/abs/2110.09485
And I would love to see the human that can generalize beyond their training distribution.
These models are pretty much always extrapolating [0]
Whether the extrapolation is crude/low-rank or astute/high-rank is a question of memorization vs generalization. That gets into the question of whether or not the model is over-fitted or under-fitted. There are certain heuristics borrowed from high dimensional statistical physics that can be used to guess how good the test performance of a model will be on a typical task without even knowing what the test data is [1].
Originality for me means finding better answers to sub-tasks, and then combining those answers together in a better way. This is the nirvana of cross-entropy minimization - the emergence of capability…
Read the full comment
results from gaining the ability to amass a wider range of skills, improving upon them, and percolating those improvements towards multiplying the leverage of other skills.
How long such a thing can keep improving with current tech, who knows, but you should really think critically about whether that sounds just like interpolation through the corpus.
[0] Learning in High Dimension Always Amounts to Extrapolation - https://arxiv.org/abs/2110.09485
[1] https://github.com/CalculatedContent/WeightWatcher
I wish I could do a good job at that.
Let me rob a definition from this paper "Learning in High Dimension Always Amounts to Extrapolation" https://arxiv.org/pdf/2110.09485.pdf :
Definition 1. Interpolation occurs for a sample x whenever this sample belongs to the convex hull of a set of samples X,
{x1, . . . , xN }, if not, extrapolation occurs.
There's a neat argument against these models doing interpolation: the manifold of the data is so sparse that it's infinitesimally unlikely for a good predictor to be doing interpolation between existing points on the manifold.
https://arxiv.org/abs/2110.09485
Depends on how you define interpolation and extrapolation.
In high dimensional spaces basically everything is extrapolation including in pixel space and embedding space.
> The notion of interpolation and extrapolation is fundamental in various fields from deep learning to function approximation. Interpolation occurs for a sample x whenever this sample falls inside or on the boundary of the given dataset's convex hull. Extrapolation occurs when x falls outside of that convex hull. One fundamental (mis)conception is that state-of-the-art algorithms work so well because of their ability to correctly interpolate training data. A second (mis)conception is that interpolation…
Read the full comment
happens throughout tasks and datasets, in fact, many intuitions and theories rely on that assumption. We empirically and theoretically argue against those two points and demonstrate that on any high-dimensional (>100) dataset, interpolation almost surely never happens. Those results challenge the validity of our current interpolation/extrapolation definition as an indicator of generalization performances.
https://arxiv.org/abs/2110.09485
Also:
> The location of decision boundaries inside the convex hull of training set can be investigated in relation to the training samples. However, our analysis shows that in standard image classification datasets, all testing images are considerably outside that convex hull, in the pixel space, in the wavelet space, and in the internal representations learned by deep networks. Therefore, the performance of a trained model partially depends on how its decision boundaries are extended outside
the convex hull of its training data.
https://arxiv.org/abs/2101.09849
Learning in High Dimension Always Amounts to Extrapolation
https://arxiv.org/abs/2110.09485
"Supercharged Interpolation" is not a real thing.
They generalize fine when the data incentivizes that.
"Supercharged Interpolation" is not something that actually exists.
Learning in High Dimension Always Amounts to Extrapolation
https://arxiv.org/abs/2110.09485
What you're asking for is not "generalization" but magic and humans would also fail.
>Since it's incapable of actually summarising financial data. It's only capable of selecting combinations of pieces of its training set.
Third completely off misconception from you today.
This is not at all what it is doing. "Supercharged Interpolation" is false and makes no sense. It's not a lookup table either. It doesn't memorize enough of what it needs to to make your assertion possible.
https://arxiv.org/abs/2110.09485
LLMs are not lookup tables. They aren't "matching" in that way.
There's this idea of "supercharged interpolation", the idea that they just take out certain texts and switch out words or whatever, that is not true.
https://arxiv.org/abs/2110.09485
>That is what LLMs can do, they don't write something inspired by it they write down a copy of it with switched out words and grammar structure.
That's not at all what they do. This myth of "supercharged interpolation" really needs to die. https://arxiv.org/abs/2110.09485
>It doesn't matter if you personally aren't using them like that, the fact that they can be used like that is the problem.
Of course it matters. You penalize people when they have done something wrong not if they have the capacity to.
Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task.
Despite their massive size, successful deep artificial neural networks can exhibit a remarkably small difference between training and test performance.
Fast compilation is important when compilation occurs at runtime, such as query compilers in modern database systems and WebAssembly virtual machines in modern browsers.