UMD Researchers Organize NeurIPS Competition to Improve AI Watermarks
This image was generated using artificial intelligence.
As AI-generated images become increasingly more realistic, they present both opportunities and challenges. While such advancements can enable creative applications and enhance visual storytelling, they can also fuel the spread of disinformation or cast doubt on the validity of authentic photographs. This glaring dichotomy highlights the high-stakes challenge of determining the origin and authenticity of online images.
One approach for establishing whether an image is AI-generated is through watermarking—the process of embedding a unique signal into an image to indicate its origin. While this method can be applied to any image, it is often proposed as a promising solution for identifying AI-generated content. These digital watermarks, however, can be altered or removed by malicious users, raising questions about their reliability and long-term efficacy.
To address these concerns, a team led by University of Maryland researchers recently held an international AI watermark-removal competition. The contest —held last December during the annual Conference on Neural Information Processing Systems (NeurIPS 2024)—was open to researchers worldwide, challenging participants to remove invisible watermarks from AI-generated images. The goal was to rigorously test the robustness of various watermarking techniques while also identifying potential vulnerabilities.
“We wanted to determine how well watermarks could withstand real-world manipulation—to unearth their vulnerabilities and weaknesses,” says Furong Huang, an associate professor of computer science at the University of Maryland who was a lead organizer of the event.
Huang says the idea for the competition came about through a casual chat between academics, including Tom Goldstein, a professor of computer science at UMD, and other researchers and faculty at Carnegie Mellon University, the University of Illinois Urbana-Champaign, and Google DeepMind.
Both Huang and Goldstein have appointments in the University of Maryland Institute for Advanced Computer Studies (UMIACS), which provided technical support for the event and partnered with watermarking startup Trufo to offer cash prizes for contestants totaling $7,000.
The contest—officially known as Erasing the Invisible: A Stress-Test Challenge for Image Watermarks—received an impressive 2,722 submissions, a turnout that exceeded Huang’s expectations, demonstrating a strong interest from the AI community in tackling the challenges of watermarking.
Contestants competed in two separate tracks: The black-box track, which simulates a real-world scenario where attackers must operate without knowledge of the watermarking technique employed; and the beige-box track, which provides participants with images alongside labels indicating the watermarking methodology used.
Submissions were evaluated using a metric system to autonomously evaluate entries using a formula. A rolling leaderboard provided feedback in real-time, with human judges evaluating the submissions to ensure fairness and prevent gaming of the evaluation metrics.
The winners were announced on Dec. 15 at a special NeurIPS workshop. First-place honors (for both tracks) went to a team from Mohamed bin Zayed University of Artificial Intelligence in the United Arab Emirates. Second place in the beige-box category as well as a third-place finish in the black-box category went to a team of researchers representing the Sony Group Corporation. Third place in the beige-box category went to a solo student representing the Sharif University of Technology in Iran.
The contest organizers had high praise for the winners, noting that while they all demonstrated creative and unique methods, they shared a common strategy—having a tailored approach to removing the AI watermarks.
The beige-box teams matched specific attack methods to different watermarks, which proved highly effective, says Huang, while within black box track, teams faced a wider variety of watermarks, including some complex compositions that required clustering and fine-tuning AI algorithms to effectively apply different attacks to different images.
The competition also revealed significant weaknesses in existing watermarking techniques, organizers said. The event featured eight different watermarking pipelines in the competition, ranging from older, basic models to advanced state-of-the-art methods, but most weren’t robust enough against the creative removal tactics deployed by the top teams.
The UMD researchers plan on sharing their findings in a forthcoming paper, which will discuss the watermarking and removal techniques showcased during the competition.
Both Huang and Goldstein acknowledged the outstanding support they received in organizing the event from Ph.D. students in UMD’s Department of Computer Science, as well as the technical support from UMIACS in managing the large amount of data tied to the contest.
One of the UMD students, Bang An, said that helping organize such a large-scale event was both a challenging and rewarding journey, and one that went beyond just completing a research project or solving a technical challenge.
Another, Tahseen Rabbani, says that the lead student organizers—himself, An, Mucong Ding and Chenghao Deng—quickly realized that putting together a high-quality experience for a major conference like NeurIPS required more than technical knowledge.
“We spent many late nights poring over the tiniest of details to ensure that our participants would have a seamless, enjoyable experience,” he says.
—Story by Aleena Haroon, UMIACS communications group
