China's deepfake anchors spread disinformation on social media, Graphika says

US research group says AI anchors are bolstering Beijing’s propaganda and badmouthing Washington.
By RFA Staff
2023.02.08
China's deepfake anchors spread disinformation on social media, Graphika says AI-generated people acting as ‘news anchors’ in Wolf News videos.
Graphika

Artificial intelligence-generated news anchors have been deployed for the first time to propagandize political content on social media, a U.S. research firm said in its latest report.

The New York-based Graphika said in the report ‘Deepfake It Till You Make It’ that Spamouflage, a China state-aligned influence operation, has been using AI-generated fictitious people to promote China’s global role and spreading disinformation against the United States since late 2022.

Spamouflage, a combination of Spam and Camouflage, is a tactic used by spammers to evade email spam filters by replacing certain letters with numbers. Cyber security researchers use the term, as well as other terms such as Dragonbridge or Spamouflage Dragon, to refer to the pro-Chinese propaganda network on the internet.

Spamouflage is responsible for posting, via fake accounts targeting social media users, thousands of assets that praise China, criticize the United States, and attack the Hong Kong pro-democracy movement as well as Taiwan independence. 

Recently it has been “promoting a new and distinctive form of video content on social media platforms including Facebook, Twitter, and YouTube,” Graphika said.

The firm’s researchers spotted and analyzed two “unique” videos featuring a male and a female anchor, both speaking English and appearing to be Caucasian.

The videos used the logo of a “likely fictitious media company” called ‘Wolf News’ with the accompanying slogan “Focus on hot spots and broadcast in real time.” 

The male anchor criticized the U.S. government for its “hypocritical repetition of empty rhetoric” in tackling gun violence while the female highlighted the importance of China-U.S. cooperation for the global economic recovery.

Both videos mirrored “past Spamouflage efforts to pass as legitimate news outlets,” said the Graphika report.

Alex, Jason and James

The abundant similarities spotted in the videos and other Spamouflage assets on the internet led to the conclusion about the connection between the two.

China has yet to respond to the Graphika report which points to the alarming potential misuse of sophisticated AI technology in disinformation campaigns by state actors. 

Until now, online influence operations have been limited to computer-generated fake faces and fabricated videos but the new videos featured AI-generated fictitious people that looked almost real. 

“At first glance, the Wolf News anchors present as real people. Our initial hypothesis was that they were paid actors that had been recruited to appear in the videos,” the team behind the report said.

After further investigation they spotted the anchors’ robotic speech that doesn’t sync with the lips’ movements, as well as numerous grammatical mistakes in the subtitles.  

Graphika team said the anchors were likely created using technology provided by Synthesia, a British AI video company.

They found several marketing videos using the same AI-generated duo speaking a number of languages besides English. The male avatar, named Alex in the Wolf News video, was also called Mr Curtis, Jason and James.

“The main benefit of this technology to the creators of the Spamouflage videos appears to be increased efficiency, specifically high-speed, low-cost content production,” the report said, adding that Synthesia’s products can create AI-generated videos in a matter of minutes and subscriptions start at just U.S. $30 per month.

‘Deepfakes, Real Problems’

Influence operations actors “will continue to experiment with AI technologies, producing increasingly convincing media artifacts that are harder to detect and verify,” warned Graphika.

In 2018, the Chinese state news agency Xinhua revealed the world’s first AI news anchor, which was a male image with a voice, facial expressions and actions of a real person.

The avatar was jointly developed by Xinhua and Chinese search engine company Sogou.com.

Xinhua said the English-speaking anchor “can work 24 hours a day on its official website and various social media platforms, reducing news production costs and improving efficiency.”

Chinese AI anchor.JPG
Xinhua revealed the world’s first AI news anchor in 2018. Credit: Xinhua

In the past decade, China has placed great attention on developing AI technology. 

The Stanford University’s 2022 AI Index, which assesses AI advancements worldwide, ranks China second in total private investment in AI and the number of newly funded AI companies, only after the United States. 

Beijing introduced first-of-its-kind regulation on AI-generated images and videos, or “deepfakes,” in January. According to it, “deep synthesis services cannot use the technology to disseminate fake news.”

The new regulation also said content that endangers national security and interests, damages the national image or disrupts the economy is banned.

The regulation however doesn’t apply to deepfakes generated outside of the country. Critics say its priority is to tighten online censorship and stifle dissent.

POST A COMMENT

Add your comment by filling out the form below in plain text. Comments are approved by a moderator and can be edited in accordance with RFAs Terms of Use. Comments will not appear in real time. RFA is not responsible for the content of the postings. Please, be respectful of others' point of view and stick to the facts.

COMMENTS

Prithi Deepfaek
Feb 08, 2023 10:59 AM

Badmouthing America, eh. Sounds fun. I bet it's a good thing everyone in the civilized world can get behind doing. :)