The Hidden Watermarks in Your AI: A Conversation About Privacy, Consent, and Control
Every time you generate an image with Google's tools, create a voice clone for a podcast, or use text-to-speech software, something invisible is happening to your content. Digital watermarks—hidden signatures embedded deep within the files—are being added without most users' knowledge or explicit consent. In a recent episode of My Weird Prompts, hosts Corn and Herman Poppleberry dive into this murky intersection of technology, privacy, and regulation, exploring what these watermarks really mean for creators and consumers alike.
The Discovery: Watermarks in Plain Sight
The conversation began when podcast producer Daniel Rosehill stumbled upon something unexpected while reviewing API documentation for Chatterbox, a text-to-speech tool. Buried in the technical specifications was a reference to "neural timestamping" using something called Perth—a hidden watermark that survives editing, compression, and reformatting. It was a casual discovery that opened a much larger door.
This practice isn't isolated. Google DeepMind has been actively embedding watermarks into images generated by their Imagen model through an initiative called SynthID. The technology embeds invisible data into AI-generated images that persists even after the content is edited or compressed. The stated goal is admirable: create a way to identify deepfakes and prevent misuse of generative AI technology. But as Corn and Herman explore, the reality is far more complicated.
Two Different Things: Identification vs. Identification
Herman makes a crucial distinction that often gets lost in discussions about AI watermarking. There's a meaningful difference between two concepts: marking content as "AI-generated" versus embedding data that could potentially identify the individual user who created it.
"Most people would agree that saying 'this content is AI-generated' is reasonable," Herman explains. But when a watermark contains encrypted or hidden information that could theoretically trace the content back to a specific person, that's a different proposition entirely. One is about content authentication; the other ventures into personal identification and tracking.
For Corn, who uses these tools professionally to create voice clones for his podcast, the distinction matters deeply. He's comfortable with a watermark declaring that his audio is artificially generated—that seems like a fair and transparent practice. But the idea that the watermark might also contain personal information that could identify him as the creator, even if encrypted or obscured, feels invasive. As he points out, this discomfort exists even when he's doing nothing wrong and has nothing to hide.
The Transparency Problem
What troubles Herman most isn't necessarily the watermarks themselves, but the lack of transparency surrounding them. Most users have no idea these watermarks exist. Rosehill only discovered it by accident while reading technical documentation—something the vast majority of people using these tools will never do.
This raises a fundamental question about informed consent. When you sign up for a service and generate content, do you deserve to know exactly what's being embedded in that content? Who can access it? How is it protected? Herman argues that the answer is unequivocally yes, and that this isn't paranoia—it's basic informed consent.
The ambiguity surrounding what information is actually embedded makes the situation worse. Google has been relatively public about SynthID embedding metadata about the image itself, but they've been less clear about whether user-identifying data is included. This lack of clarity is precisely what should concern people. If companies won't clearly explain what they're embedding, users have no way to make informed decisions about whether they're comfortable with the practice.
An Industry-Wide Trend
While Google has been the most aggressive about watermarking, the practice is becoming increasingly common across the generative AI industry. It's not yet universal, but the trend is clear. As more companies adopt watermarking technology, the lack of standardized transparency becomes a more pressing issue.
The persistence of these watermarks presents its own challenge. Because the watermarks survive editing and compression, they represent an incredibly durable form of digital marking. This raises an important question: if someone wants to remove a watermark, what options do they have? And what happens when removal tools become widely available?
The Arms Race: Watermarks vs. Removal Tools
Corn identifies what he sees as an inevitable outcome: an arms race between watermarking technology and watermark removal tools. As soon as companies embed watermarks, people develop tools to strip them out. This isn't theoretical—academic papers have already been published on adversarial attacks against watermarking systems. People are already generating images specifically designed to fool detection algorithms.
This echoes the long history of digital rights management (DRM) battles, where technology companies and circumvention specialists engage in an endless escalation. The watermark becomes more sophisticated, then removal tools become more sophisticated, and the cycle continues.
Herman acknowledges that bad actors will find ways around watermarks anyway. Someone determined to create deepfakes of celebrities or political figures won't be using official tools with watermarks—they'll use open-source models or tools without embedded watermarking. So does the watermark really matter?
What Watermarks Actually Protect
According to Herman, watermarks aren't primarily designed to stop determined bad actors. They're designed for the 99.9% of people using these tools legitimately. They serve as a deterrent and, more importantly, as a verification tool. If someone posts an image online claiming it's a photograph, a watermark proving it's AI-generated can be powerful evidence if you're trying to debunk misinformation.
But this assumes the watermark is actually detectable and verifiable by regular people. Currently, detection requires specific tools and technical expertise. Most users don't have the knowledge to check for invisible watermarks. So practically speaking, how does this help?
Herman suggests this is a long-term infrastructure play. Eventually, platforms like Twitter, Facebook, and news organizations could automatically scan for watermarks using backend infrastructure. They'd have the capability to verify authenticity at scale. But that future infrastructure doesn't exist yet, and in the meantime, users are being watermarked without knowing it.
The Slippery Slope of Scope Creep
Another concern Herman raises is the potential for scope creep. Today, watermarks identify content as AI-generated. But what prevents tomorrow's watermarks from including metadata about usage patterns, location data, account type, or subscription level?
Corn pushes back, noting that this is a slippery slope argument—we don't actually know that this is happening. But Herman's response is telling: "We don't, which is exactly my point. We should know before we agree to it."
This gets at the heart of the consent issue. The problem isn't necessarily what companies are doing right now; it's the lack of clarity about what they could do, and the absence of explicit user agreement about what information is embedded in generated content.
What Good Transparency Looks Like
When asked what adequate transparency would actually look like, Herman proposes a clear standard: "All content generated using this tool will be embedded with a watermark that identifies it as AI-generated. This watermark is designed to [specific purpose]. It will survive [specific types of modifications]. The watermark may contain the following information: [list]. You can [options for removal/modification, if any]. This data is stored [location] and accessed by [who]. You can request deletion by [method]."
Corn suggests this level of detail might be overkill for most users, and that a simpler approach might be better. But Herman counters that the basics must include information affecting privacy and rights. You don't need to understand the algorithm, but you absolutely need to know what data about you might be embedded or tracked. That's not overkill—that's baseline.
The challenge, as Corn notes, is that verbose disclosures often get ignored anyway. Nobody reads terms of service because they're walls of text. But Herman's response is pragmatic: the solution is to make disclosures clear and concise, not to skip them entirely. That's a design problem, not a reason to avoid transparency.
Balancing Safety and Privacy
The fundamental tension at the heart of this conversation is how to balance protecting against genuine misuse—voice cloning, deepfakes, unauthorized impersonation—without invading everyone's privacy.
Herman argues that watermarking isn't actually the right tool for preventing misuse. The people who want to create unauthorized deepfakes aren't using official tools with watermarks; they're using open-source models that don't have embedded watermarking. Watermarking legitimate users doesn't stop bad actors—it's more about security theater than actual prevention.
What would actually work, according to Herman, is a combination of better regulation, stronger authentication systems, legal consequences for misuse, and a cultural shift around consent. Voice cloning technology is incredibly powerful—you can impersonate someone based on just a ten-second audio sample. That should require explicit consent from the person being cloned, not a silent watermark embedded in the creator's files.
Conclusion: The Need for Clear Rules
The conversation between Corn and Herman reveals a fundamental gap between the technology companies are deploying and the transparency they're providing to users. Watermarking AI-generated content isn't inherently wrong, but doing it without clear, upfront disclosure about what information is embedded and how it's used violates basic principles of informed consent.
As generative AI becomes more powerful and more prevalent, these questions about watermarking, tracking, and transparency will only become more urgent. Users deserve to know what's happening to their content, and companies need to be explicit about the data they're collecting and embedding. Until that transparency exists, the hidden watermarks in our AI-generated content remain a privacy concern worth taking seriously.
BLOG_POST
