How much do language models memorize?

... If you want to do this seriously, I'd recommend that you hire someone who already did that kind of integration, at least as a consultant. Someone whose first reflex won't be to just tell you LLMs are fixed and can't learn but will also add this isn't a limitation since RAG pipelines are better suited for this task than fine-tuning [1]. Also RAG isn't a monolithic solutions, there are many, many variations. For your use-case, I'd consider more elaborate solutions than just baseline RAG, such as GraphRAG [2]. For the box problem above, you might want to consider integrating symbolic reasoning tools such as prolog, or consider using reasoning models and developing your own reinforcement learning environments. Needless to say, all of these aspects need to be carefully balanced and optimized to work together, and you need to follow a benchmark/dataset centric-approach to developing your solution. For this task consider frameworks that were designed to optimize llm/agentic workflows as a whole [3][4]. Shit is complex really. [1] https://arxiv.org/abs/2505.24832 tells us generalization happens in LLM once their capacity for remembering things is saturated, and this might explain why fine-tuning has been less efficient than RAG so far. [2] https://microsoft.github.io/graphrag/ [3] https://github.com/microsoft/Trace [4] https://github.com/SylphAI-Inc/AdalFlow

model essentially loselessly compared to full bf16 weights) takes ~255GB of space, so the model is only 0.18% of its training dataset. And this is the best case, because we took the biggest model, and the actual capacity of a model to memorize is only at most ~4 bit per parameter[1] instead of full ~8 bits we assumed here. For reference, the best loseless compression we can achieve for text is around ~15% of the original size (e.g. Fabrice Bellard's NNCP), which is two orders of magnitude worse. So purely from information theoretic perspective saying that those models memorize the whole datasets on which they were trained is nonsense. They can't do that, because there's just not enough bits to store all of this data. They extract patterns, the same way that I can take the very same ~137TB dataset and build a frequency table of all of the bigrams appearing in it and build a hidden Markov model out of it to generate text. Would that also be "stealing"? And what if I extend my frequency table to trigrams? Where exactly do we draw the line? [1] -- https://arxiv.org/pdf/2505.24832

spin out GenAI. Of course, it sounds like your metric is "AI/LLM competitiveness" and we have no disagreements that FAIR is failing on that end (I just don't think its only important or right metric to be judging FAIR). * Normatively, I think that it's good to have monopolistic big tech firms be funding basic open research as a counterbalance to academia and also because good basic research requires lots of compute these days. It feels somewhat shortsighted to reallocate all resources to LLM research. * DINO and JEPA aren't particularly useful for language modeling, but are still important for embodiment/robotics/3D, which indeed seems to be the "next big thing." Also, to their credit, FAIR is still doing interesting and useful work on LLMs for encoders [1], training dynamics [2], and multimodality [3], just not training frontier models. ** GenAI took the money, scope, and resources, but not sure about the good people lol, that seems to be their problem. [1] https://arxiv.org/abs/2504.01017 [2] https://arxiv.org/abs/2505.24832 [3] https://arxiv.org/abs/2412.14164