Gowthami Somepalli, Vasu Singla, Micah Goldblum, Jonas Geiping, Tom Goldstein
2022 · arXiv
Cutting-edge diffusion models produce images with high quality and customizability, enabling them to be used for commercial art and graphic design purposes. But do diffusion models create unique works of art, or are they replicating content directly from their training sets? In this work, we study image retrieval frameworks that enable us to compare generated images with training samples and detect when content has been replicated. Applying our frameworks to diffusion models trained on multiple datasets including Oxford flowers, Celeb-A, ImageNet, and LAION, we discuss how factors such as training set size impact rates of content replication. We also identify cases where diffusion models, including the popular Stable Diffusion model, blatantly copy from their training data.
Metadata: arXiv. Abstracts retain their original authors’ rights.
Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels
11 selected excerpts · 11 total mentions
what are the opinions on [0]? what's the scene for language rather than image? also what are the opinions on turning generative AI (that doesn't ask permission to creators) public domain? donation money that surpasses the cost of hosting the work to people/groups "creating" with AI, should be a violation of the license? are you allowed to play with the models in hardware made by for-profit entities, like Nvidia? [0] https://arxiv.org/abs/2212.03860
is this (1) the lawsuit you're referring to? 'cause those images are not the same. Sports events are just easy to fake, because they're boring - all sports pictures look roughly the same. Edited to add: There's another lawsuit (a class action - 2), and after a little light reading, I came across section 5: 'Do diffusion models copy?', and my stomach jumped. What they're doing, to make a point at trial that stable diffusion copies images, is _training images into the model, then using that trained model to prove that stable diffusion is a compression algorithm_. This is a patent fabrication. If you train a model hard enough, yeah, it will produce the image you trained it…
on. And become useless for all other images. Congrats, you've just compressed your 7kb image to a 7gb diffusion model. What scares me about this, is that the average court in the US is absolutely dumb enough to fall for it. 1 - https://www.theverge.com/2023/2/6/23587393/ai-art-copyright-... 2 - https://arxiv.org/pdf/2212.03860.pdf
I do want to clarify that I think stable diffusion and tools like it can engage in illegal copying. For example it will happily produce infringing images of logos and even somewhat random other images https://arxiv.org/pdf/2212.03860.pdf. It seems like it’s devoting an uneven amount of its weights to different images, but I remain unconvinced that’s all it can do, or at least anymore all it can do than for a human artist
Also another previous study that has Stable Diffusion (SD) emitting images in the training set [0] It is now clear that SD is treading on thin ice: training on watermarked and copyrighted images without their author's permission, then attempting to commercialize it even when the model emits images that resemble a high similarity of the original training data including watermarks or copyrighted images: (Mickey Mouse, Getty Images watermarks, Bloodborne art cover, etc). This weakens their fair use argument, especially with Getty Images also threatening to sue SD for the same reason. If OpenAI was able to get permission to train on shutterstock images [1], then SD could have…
done the same, but chose not to. Perhaps SD thought they could get away with it and launch their grift (DreamStudio) on digital images and artists. It turns out that now SD creates an opt-out system afterwards but artists can already find out if their images are in the training set. [2]. [0] https://arxiv.org/pdf/2212.03860.pdf [1] https://www.prnewswire.com/news-releases/shutterstock-partne... [2] https://haveibeentrained.com/
Afaik theoretically you could reproduce any image in the training set using the full weights (not a fraction of them) and the correct prompt. In practice since this is an extremely lossy process, some or most of them aren't reproducible. For this specific case, I suspect it'll come down to whether somebody in the class can pass a test like this: https://arxiv.org/abs/2212.03860
Honestly, can people stop speaking in absolutes regarding these systems? We (researchers and non-researchers alike) are gradually trying to comprehend exactly how much they generalise and memorise, but this is darn hard work and it is not our fault that several major tech giants decided to deploy and profit from these models long before the scientific and legal landscape was clear. Somepalli et al. (2022) [1] for example is a fairly strong argument against your statement above. [1]: https://arxiv.org/abs/2212.03860 The fact is that these systems are complex, new, and interesting. However, it is not the fault of small-time programmers and artists that modern copyright law is a major, overreaching mess that is now finally greatly affecting what the big corporations want to do. They are getting sued? Cry me a river… Perhaps they will finally stop backing the American-led copyright lobby then?
> Could you provide evidence of your last statement? I haven’t seen these models produce actual copies of any art (can’t imagine that’s an option in general). Here's a whole paper, complete with citations: https://arxiv.org/pdf/2212.03860.pdf > These models do not contain copies. One way to describe the data is they contain a statistical breakdown of the artwork, which is substantially different from a JPEG -> PNG conversion you mention. I don't understand the distinction you're making. What legally separates a "statistical breakdown" representation from a zip file representation, JPEG representation, PNG representation?
> when doing paintovers on copyrighted images (VERY common) What are you talking about? I've been doing drawing and digital painting as a hobby for a long time and tracing is absolutely not "VERY common". I don't know anybody who has ever done this. > fan art where they paint trademarked characters (also VERY common) This is true in the sense that many artists do it (besides confusing trademark law and copyright law: the character designs are copyright-protected, trademarks protect brand names and logos). However, it is not fair use (as far as I'm aware at least, I'm not a lawyer). A rightholder can request for fanart to be removed and the artist would have to remove it.…
Rightsholders almost never do, because fanart doesn't hurt them. There's also more examples of it reproducing copyright-protected images, I pulled the "bloodborne box art" prompt from this article: https://arxiv.org/pdf/2212.03860.pdf But I agree with you that reproducing images is very much not the intention of Stable Diffusion, and it's already very rare. The way I see it, the cases of Stable Diffusion reproducing images too closely is just a gotcha for establishing a court case.
> The actual issue is that Stable Diffusion does create derivatives of copyrighted works. Nothing points to that, in fact even in this website they had to lie on how stablediffusion actually works, maybe a sign that their argument isn't really solid enough. > [1] https://arxiv.org/pdf/2212.03860.pdf You realize those are considered defects of the model right? Sure, this model isn't perfect and will be improved.
There is no need for rhetorical games. The actual issue is that Stable Diffusion does create derivatives of copyrighted works. In some cases the produced images contain pixel level details from the originals. [1] [1] https://arxiv.org/pdf/2212.03860.pdf
From fact checking myself it is worth noting that stable diffusion will absolutely copy things in ways that would probably infringe a copyright (not a layer and not legal advice), but I found this paper interesting both because some of the examples it gives of clear copying don't look all that copied to me, and because some do and they find overall a relatively high rate of copying. 1.8% https://arxiv.org/abs/2212.03860. I believe this is still not peer reviewed.
More from the same topics.
Martin Kleppmann
The CAP Theorem is a frequently cited impossibility result in distributed systems, especially among NoSQL distributed databases.
Tom B. Brown, Benjamin Mann, Nick Ryder +28
Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task.
Chiyuan Zhang, Samy Bengio, Moritz Hardt +2
Despite their massive size, successful deep artificial neural networks can exhibit a remarkably small difference between training and test performance.
Scott Aaronson
In honor of Alan Turing's hundredth birthday, I unwisely set out some thoughts about one of Turing's obsessions throughout his life, the question of physics and free will.