Generative AI is at risk of falling into the hands of radical groups who see it as the perfect tool for producing propaganda according to experts from big tech firms quoted by a WIRED report on Thursday.
Traditionally such platform providers have used databases of known extremists, called hashing databases, to track violent and extremist content and can be shared quickly between platforms to remove the content across social media.
Adam Hadley, the executive director of Tech Against Terrorism explained that there has been a sudden surge in such AI-generated content, particularly surrounding the Israel-Hamas War since October 7.
“Our biggest concern is that if terrorists start using gen AI to manipulate imagery at scale, this could well destroy hash-sharing as a solution, This is a massive risk,” Hadley told WIRED.
Hamas/Hezbollah using generative AI propaganda
“Give it six months or so, the possibility that [they] are manipulating imagery to break hashing is really concerning,” Hadley says. “The tech sector has done so well to build automated technology, terrorists could well start using gen AI to evade what's already been done.”
Examples of the type of content that has started to evade previous defenses are the Islamic State publishing a tech support guide on how to securely use generative AI tools, a pro-al-Qaeda outlet publishing several posters with images highly likely to have been created using generative AI, and far-right figures producing a “guide to memetic warfare” advising others on how to use AI-generated image tools to create extremist memes.