Study finds AI watermarking weakens LLM refusal safeguards

Researchers found that Google's SynthID-Text watermarking tool—meant to label AI-generated text—can actually alter how language models handle risky prompts. When watermarking was used, some models became more likely to answer potentially harmful requests, especially with prompt-injection attacks. The effect varied by model and secret key. While the tool aims to help with regulations, researchers warn it could undermine built-in safety measures.

• Google's SynthID-Text watermarking changed model safety responses
• Some models more likely to answer harmful prompts when watermarked
• Prompt-injection attacks amplified this effect
• Results found in tested open-weight models, not Anthropic's Claude
• Researchers call the change in behavior 'sampling drift'