Studying Large Language Model Generalization with Influence Functions

> LLMs (I believe) _can_ be trained to not plagiarize. They do not have to be original, nor to use public domain training-data, they just have to give attribution (which, as far as I can tell, requires little more than a corpus and a stack of similarity-measures). And something like attributions can be determined even if the LLM is paraphrasing, if I'm understanding Cosma Shalizi correctly[1]. He gives Grosse et al.[2] as a reference for doing so in a scalable manner. [1] https://bactra.org/research/2026-03-10.html [2] https://arxiv.org/abs/2308.03296

And ideally if someone is proposing new prompting techniques they should test it across both the most capable models (which are unfortunately proprietary) and the best open models. The problem is that what works on small LLMs does not necessarily scale to larger ones. See page 35 of [1] for example. A researcher only using the models of a few years ago (where the open models had <1B parameters) could come to a completely incorrect conclusion: that language models are incapable of generalising facts learned in one language to another. [1] https://arxiv.org/pdf/2308.03296.pdf

Some work, yeah. It's still an open problem to do it well, but I think the folks at Anthropic have made a reasonable start with their work[1] on influence functions ("tracing model outputs to the training data"). Basically their work attempts to answer the question "what particular training data most strongly influenced the model to give the answer it did", by doing some fancy math that I think is equivalent to taking the gradient produced by each piece of training data, computing the derivative of loss on the output of interest as the gradient is applied to the model, and then using that as the answer. Though it sounds like even their much cheaper clever approach is still very expensive. [1] paper at https://arxiv.org/abs/2308.03296, post at https://www.anthropic.com/index/influence-functions

Anthropic has a recent paper that talks about this [1]. See figure 16 in particular. The larger the model, the more it generalizes between languages. On the other hand, no matter how large the model they struggle to generalize between sentences of the form "George Washington was the first US president" and "The first US president was George Washington". Clearly generalization behavior is unintuitive, you can come up with a post hoc explanation (weights are independent between early and late layers) but I doubt anyone would have thought ahead of time that models would have an easier time generalizing between languages than between slightly different ways of saying the same thing in English. [1] https://arxiv.org/pdf/2308.03296.pdf