Skip to content
The methodOpen data
Back to the collection

Extracting books from production language models

Ahmed Ahmed, A. Feder Cooper, Sanmi Koyejo, Percy Liang

2026 · arXiv

Language models
Read the paper
HN score
0
Mentions
21
Readers
13
Months
6
What’s in a score?

From the abstract

Many unresolved legal questions over LLMs and copyright center on memorization: whether specific training data have been encoded in the model's weights during training, and whether those memorized data can be extracted in the model's outputs. While many believe that LLMs do not memorize much of their training data, recent work shows that substantial amounts of copyrighted text can be extracted from open-weight models. However, it remains an open question if similar extraction is feasible for production LLMs, given the safety measures these systems implement. We investigate this question using a two-phase procedure: (1) an initial probe to test for extraction feasibility, which sometimes uses a Best-of-N (BoN) jailbreak, followed by (2) iterative continuation prompts to attempt to extract the book. We evaluate our procedure on four production LLMs -- Claude 3.7 Sonnet, GPT-4.1, Gemini 2.5 Pro, and Grok 3 -- and we measure extraction success with a score computed from a block-based approximation of longest common substring (nv-recall). With different per-LLM experimental configurations, we were able to extract varying amounts of text. For the Phase 1 probe, it was unnecessary to jailbreak Gemini 2.5 Pro and Grok 3 to extract text (e.g, nv-recall of 76.8% and 70.3%, respectively, for Harry Potter and the Sorcerer's Stone), while it was necessary for Claude 3.7 Sonnet and GPT-4.1. In some cases, jailbroken Claude 3.7 Sonnet outputs entire books near-verbatim (e.g., nv-recall=95.8%). GPT-4.1 requires significantly more BoN attempts (e.g., 20X), and eventually refuses to continue (e.g., nv-recall=4.0%). Taken together, our work highlights that, even with model- and system-level safeguards, extraction of (in-copyright) training data remains a risk for production LLMs.

Metadata: arXiv. Abstracts retain their original authors’ rights.

In the conversation

Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels

20 selected excerpts · 21 total mentions

philipportner
Neutral mention · rule
> I'm not sure you can prompt a full, accurate, copy of a nontrivial codebase out of them. Even with zero temperature their accuracy is just not that high. Granted, these are some of the most widely spread texts, and not codebases, but just fyi: https://arxiv.org/pdf/2601.02671 > For Claude 3.7 Sonnet, we were able to extract four whole books near-verbatim, including two books under copyright in the U.S.: Harry Potter and the Sorcerer’s Stone and 1984 (Section 4).
Original comment Grit: Rewriting Git in Rust with Agents
Topfi
Neutral mention · rule
> […] fed large chunks of an article multiple times to an LLM […] So they had to prompt? An LLM? I got this argument before and still don’t get what it’s trying to say. These models do not output anything unless prompted, that’s not any kind of gotcha. On the code outputting front there is a lot of relevant evidence beyond the NYC lawsuit [0]. [0] https://arxiv.org/html/2601.02671?amp=&amp= and https://arxiv.org/abs/2506.12286 and https://ai.stanford.edu/blog/verbatim-memorization/
Original comment Did Claude Increase Bugs in Rsync?
logicprog
Neutral mention · rule
Also from that exact same study (why not cite the actual study? It's quite readable) the LLMs couldn't recite more than a small fraction of many other books, often ones just as well known[0] — in fact, from the bar charts shown in the exact news article you cited, it's pretty clear that Sonnet 3.7 was a massive outlier, and so was Harry Potter and the Sorcerer's Stone, so it really seems to me like that's an extremely unrepresentative example, and if all the other LLMs couldn't recite even a small fraction of all the other books except that one outlier pairing, despite them being widely reproduced classics, why would we expect LLMs to actually regurgitate regularly,…
Read the full comment
especially a relatively unknown open source project that probably hasn't been separately reproduced that many times? Not to mention the fact that, as the other commenters mention, that appears to just... not have happened at all in this case, so it's a moot point. [0]: https://arxiv.org/pdf/2601.02671
Original comment AI and the Ship of Theseus
msdz
Neutral mention · rule
They’d probably get the farthest, but they won’t pursue that because they don’t want to end up leaking the original data from training. It is possible in regular language/text subsets of models to reconstruct massive consecutive parts of the training data [1], so it ought to be possible for their internal code, too. [1] https://arxiv.org/abs/2601.02671
Original comment Relicensing with AI-Assisted Rewrite
Alifatisk
Neutral mention · rule
From the paper [1]: > While we needed to jailbreak Claude 3.7 Sonnet and GPT-4.1 to facilitate extraction, Gemini 2.5 Pro and Grok 3 directly complied with text continuation requests. For Claude 3.7 Sonnet, we were able to extract four whole books near-verbatim, including two books under copyright... I am just thinking loudly here. Can't one argue that because they had to jailbreak the models, they are circumventing the system that protects the copyright? So the llms that reproduce the copyrighted material without any jailbreaking required is infringing the copyright. 1. https://arxiv.org/pdf/2601.02671
Original comment AIs can generate near-verbatim copies of novels from training data
hodgehog11
Neutral mention · rule
> "...it would sometimes regurgitate training data verbatim. That’s been patched in the years since..." > "They are robots. Programs. Fancy robots and big complicated programs, to be sure — but computer programs, nonetheless." This is totally misleading to anyone with less familiarity with how LLMs work. They are only programs in as much as they perform inference from a fixed, stored, statistical model. It turns out that treating them theoretically in the same way as other computer programs gives a poor representation of their behaviour. This distinction is important, because no, "regurgitating data" is not something that was "patched out", like a bug in a computer…
Read the full comment
program. The internal representations became more differentially private as newer (subtly different) training techniques were discovered. There is an objective metric by which one can measure this "plagiarism" in the theory, and it isn't nearly as simple as "copying" vs "not copying". It's also still an ongoing issue and an active area of research, see [1] for example. It is impossible for the models to never "plagiarize" in the sense we think of while remaining useful. But humans repeat things verbatim too in little snippets, all the time. So there is some threshold where no-one seems to care anymore; think of it like the % threshold in something like Turnitin. That's the point that researchers would like to target. Of course, this is separate from all of the ethical issues around training on data collected without explicit consent, and I would argue that's where the real issues lie. [1] https://arxiv.org/abs/2601.02671
Original comment The Problem with LLMs
zephen
Neutral mention · rule
> It's a bit disappointing that people are still re-hashing the same "it's in the training data" old thing from 3 years ago. They only have to keep reiterating this because people keep lying about this. > It's not like any LLM could 1for1 regurgitate millions of LoC from any training set... This is not how it works. Maybe not any old LLM, but Claude sure as shit can. https://arxiv.org/pdf/2601.02671v1
Original comment We tasked Opus 4.6 using agent teams to build a C Compiler
empath75
Neutral mention · rule
They do memorize some books. You can test this trivially by asking ChatGPT to produce the first chapter of something in the public domain -- for example a Tale of Two Cities. It may not be word for word exact, but it'll be very close. These academics were able to get multiple LLMs to produce large amounts of text from Harry Potter: https://arxiv.org/abs/2601.02671
Original comment Nvidia Contacted Anna's Archive to Access Books
flir
Neutral mention · rule
There was a paper a while back where (from memory) they managed to coax 75% of the original text of some internet-popular books out of an LLM. Harry Potter, 1984, etc. That's why I said it was possible for some texts. My assumption is that multiple copies in the training data "wear a deeper groove". I believe those are infringing, and should be dealt with on a case-by-case basis. But the vast majority of text doesn't wear that groove. (Edit: Think it was this one https://arxiv.org/abs/2601.02671)
Original comment Judge approves $1.5B Anthropic settlement for pirated books used to train Claude
maplethorpe
Neutral mention · rule
Researchers have. The idea that the data is unrecoverable after training is incorrect. "Extracting books from production language models While many believe that LLMs do not memorize much of their training data, recent work shows that substantial amounts of copyrighted text can be extracted from open-weight models. However, it remains an open question if similar extraction is feasible for production LLMs, given the safety measures these systems implement. We evaluate our procedure on four production LLMs -- Claude 3.7 Sonnet, GPT-4.1, Gemini 2.5 Pro, and Grok 3 -- and we measure extraction success with a score computed from a block-based approximation of longest common…
Read the full comment
substring (nv-recall). For the Phase 1 probe, it was unnecessary to jailbreak Gemini 2.5 Pro and Grok 3 to extract text (e.g, nv-recall of 76.8% and 70.3%, respectively, for Harry Potter and the Sorcerer's Stone), while it was necessary for Claude 3.7 Sonnet and GPT-4.1. In some cases, jailbroken Claude 3.7 Sonnet outputs entire books near-verbatim (e.g., nv-recall=95.8%)." https://arxiv.org/abs/2601.02671
Original comment An AI Hate Wave Is Here
Topfi
Neutral mention · rule
> Modern LLMs don't memorize harry potter like GPT3 did. [...] You can't reliably do this with Opus or GPT5. If you try any modern LLM, you will find that you can. Easily [0], reliably [1], consistently [2]. All these examples are with models released in 2025/26. [0] https://arxiv.org/html/2601.02671?amp=&amp= [1] https://arxiv.org/abs/2506.12286 [2] https://ai.stanford.edu/blog/verbatim-memorization/
Original comment FSF Threatens Anthropic over Infringed Copyright: Share Your LLMs Freel
logicprog
Neutral mention · rule
Regurgitation is pretty rare, and very difficult to coax out, if not even impossible, for things that aren't massively overrepresented in the training set relative to the size of the training set. Even the famous regurgitation paper showed this: while they got most of the models to regurgitate the first book of the Harry Potter series, only Claude 3.7 Sonnet was able to regurgitate any significant portion of any of the other books that had a high nv-recall rate, and basically all of them dropped off precipitously for works like GoT, The Catcher in the Rye, Beloved, and remembered almost nothing about the Da Vinci Code or Catch-22[0]. So you really need huge amounts of examples to get any kind of meaningful regurgitation on any kind of reliable basis. Thus, you'd have to prove that hypothesis. [0]: https://arxiv.org/pdf/2601.02671
Original comment Claude's Cycles: Claude Opus 4.6 solves a problem posed by Don Knuth [pdf]
philipportner
Neutral mention · rule
This seems related, it may not be a codebase but they are able to extract "near" verbatim books out of Claude Sonnet. https://arxiv.org/pdf/2601.02671 > For Claude 3.7 Sonnet, we were able to extract four whole books near-verbatim, including two books under copyright in the U.S.: Harry Potter and the Sorcerer’s Stone and 1984 (Section 4).
Original comment We tasked Opus 4.6 using agent teams to build a C Compiler

Keep following the thread.

More in Language models

More from the same topics.