Scalable watermarking for identifying large language model outputs

My understanding is that you get back a percentage from SynthID regarding how confident it thinks the text is generated by AI or not. The question is what percentage would be considered undeniable that it is AI generated or require more than just your testimony to prove that you actually did the work yourself. https://www.nature.com/articles/s41586-024-08025-4 (Google Deepmind) was using a "Watermark detectability is measured using the true-positive rate (TPR) when the false-positive rate (FPR) is set to 1%" for instance in their research paper. This 1% was set arbitrary just for the research paper but what percentage should it be set to?

> “By definition it must make text worse … because the nature of the watermarking algorithm requires it to sometimes increase the probability of selecting a worse word choice and decrease the probability of selecting the model’s best choice.” Gruber made an effort to but doesn't fully understand how SynthID works. LLMs select the next word randomly from a set probability distribution, so there is no "best choice" unless you run the LLM with a temperature of 0 which would give out terrible results. Anthropic runs a non-distorting version of SynthID that doesn't change the probabilities of the underlying distribution of tokens. It makes the watermark less likely to work over smaller samples but preserves text quality. I encourage the mathematically inclined to read the paper: https://www.nature.com/articles/s41586-024-08025-4

Google's SynthID is embedded into the content itself. Google open sourced their SynthID for text. Repo: https://github.com/google-deepmind/synthid-text Paper: https://www.nature.com/articles/s41586-024-08025-4 With images and video, it's less clear exactly what they're doing, but it's watermarking on the pixel leve. From one of their blog posts: Videos are composed of individual frames or still images. So we developed a watermarking technique inspired by our SynthID for image tool. This technique embeds a watermark directly into the pixels of every video frame, making it imperceptible to the human eye, but detectable for identification. https://deepmind.google/discover/blog/watermarking-ai-genera... Elevenlab's audio watermarking is trivial to shake off with compression, but google claims that synthid is resilient to such manipulation.

"Since launching in 2023, SynthID has watermarked over 10 billion images, videos, audio files and texts, helping identify them as AI-generated and reduce the chances of misinformation and misattribution. Outputs generated by Veo 3, Imagen 4 and Lyria 2 will continue to have SynthID watermarks. Today, we’re launching SynthID Detector, a verification portal to help people identify AI-generated content. Upload a piece of content and the SynthID Detector will identify if either the entire file or just a part of it has SynthID in it. With all our generative AI models, we aim to unleash human creativity and enable artists and creators to bring their ideas to life faster and more easily than ever before." From the page linked in the post.... So there's different ways to detect AI generated content (videos/images atleast). (https://www.nature.com/articles/s41586-024-08025-4 <-- paper on synthID / watermarking and detecting it with LLMs)

There's already research into stenography for LLM generated text for fingerprinting and identifying source: https://www.nature.com/articles/s41586-024-08025-4 The idea seems similar enough that I wanted to share. The same way you can hide information in the text to prove it was generated by a specific model and version, of course you can use this for secrets as well.

The academic paper: https://www.nature.com/articles/s41586-024-08025-4 They use the last N prefix tokens, hash them (with a keyed hash), and use the random value to sample the next token by doing an 8-wise tournament, by assigning random bits to each of the top 8 preferred tokens, making pairwise comparisons, and keeping the token with a larger bit. (Yes, it seems complicated, but apparently it increases the watermarking accuracy compared to a straightforward nucleus9 sampling.) The negative of this approach is that you need to rerun the LLM, so you must keep all versions of all LLMs that you trained, forever.

huh, thanks for commenting this! I trusted the linked explanation post https://declaude.org/watermarking/ but actually reading the the synthid paper showed me that my understanding was wrong: https://www.nature.com/articles/s41586-024-08025-4 It is definitely blurrier whether you can say this approach changes the distribution then. By definition, it _has_ to change the probabilities of output tokens, but it's not totally clear that the pseudorandomly generated scoring functions does affect the learned distribution. put another way, I think it's safer to do: compute distribution -> sample -> watermark from sampled options than it would be to do: compute distribution -> watermark distribution -> sample

Not sure why Anthropic is getting all the attention here. Google been doing this since at least late 2024[1] and openai I believe is doing this around 9 days ago[2]. [1]https://www.nature.com/articles/s41586-024-08025-4 [2]https://help.openai.com/en/articles/8912793-provenance-signa...