AI could be used to create wide-scale disinformation campaigns has long been a fear of computer scientists.
Facebook said Friday that it had removed hundreds of accounts with ties to the Epoch Media Group, parent company of the Falun Gong-related publication and conservative news outlet The Epoch Times.
The accounts—including pages, groups and Instagram feeds meant to be seen in both the United States and Vietnam—presented a new wrinkle to researchers: fake profile photos generated with the help of artificial intelligence.
The idea that artificial intelligence could be used to create wide-scale disinformation campaigns has long been a fear of computer scientists. And they said it was worrying to see it already being used in a coordinated effort on Facebook.
While the technology used to create the fake profile photos was most likely a far cry from the sophisticated AI systems being created in labs at big tech companies like Google, the network of fake accounts showed “an eerie, tech-enabled future of disinformation,” said Graham Brookie, director of the Atlantic Council’s Digital Forensic Research Lab.
Scientists have already shown that machines can generate images and sounds that are indistinguishable from the real thing or spew vast volumes of fake text, which could accelerate the creation of false and misleading information. This year, researchers at a Canadian company even built a system that learned to imitate the voice of podcaster Joe Rogan by analysing audio from his old podcasts. It was a shockingly accurate imitation.
The people behind the network of 610 Facebook accounts, 89 Facebook Pages, 156 Groups and 72 Instagram accounts posted about political news and issues in the United States, including President Donald Trump’s impeachment, conservative ideology, political candidates, trade and religion.
“This was a large, brazen network that had multiple layers of fake accounts and automation that systematically posted content with two ideological focuses: support of Donald Trump and opposition to the Chinese government,” Brookie said in an interview.
The Atlantic Council’s lab and another company, Graphika, which also studies disinformation, released a joint report analysing the Facebook takedown.
The Epoch Media Group denied in an email sent to The New York Times that it was linked to the network targeted by Facebook and said that Facebook had not contacted the company before publishing its conclusions.
The people behind the network used artificial intelligence to generate profile pictures, Facebook said. They relied on a type of artificial intelligence called generative adversarial networks. These networks can, through a process called machine learning, teach themselves to create realistic images of faces, even though they do not belong to a real person.
Nathaniel Gleicher, Facebook’s head of security policy, said in an interview that “using AI-generated photos for profiles” has been talked about for several months, but for Facebook, this is “the first time we’ve seen a systemic use of this by actors or a group of actors to make accounts look more authentic.”
He added that this AI technique did not actually make it harder for the company’s automated systems to detect the fakes, because the systems focus on patterns of behavior among accounts.
Ben Nimmo, director of investigations at Graphika, said that “we need more research into AI-generated imagery like this, but it takes a lot more to hide a fake network than just the profile pictures.”
Facebook said the accounts masked their activities by using a combination of fake and authentic US accounts to manage pages and groups on the platforms. The coordinated, inauthentic activity, Facebook said, revolved around the media outlet The BL—short for “The Beauty of Life”—which the fact-checking outlet Snopes said in November was “building a fake empire on Facebook and getting away with it.”
Gleicher said Facebook began its investigation into The BL in July and accelerated its efforts when the network became more aggressive in posting this fall. It is continuing to investigate “other links and networks” tied to The BL, he said.
Facebook said the network had spent less than $9.5 million on Facebook and Instagram ads. On Friday, Facebook said The BL would be banned from the social network.
The Epoch Times and The BL have denied being linked, but Facebook said it had found coordinated, inauthentic behavior from the network to the Epoch Media Group and individuals in Vietnam working on its behalf.
The Epoch Media Group said in its email that The BL was founded by a former employee and employs some of its former employees. “However, that some of our former employees work for BL is not evidence of any connection between the two organisations,” the company said.
In August, Facebook banned advertising from The Epoch Times after it found it had obscured its connection to Facebook ads promoting Trump and conspiracy content.
© 2019 The New York Times