Twitter bots surfaced during Chinese protests. Who's behind them remains a mystery

In late November, as anti-COVID lockdown protests broke out across cities in China and photos and videos were shared over social media, researchers noticed something odd on Twitter. When they searched for the names of large cities in China, the results included scads of suggestive images and posts advertising escort services. Some observers accused the Chinese government of attempting to drown out reporting on the protests.

Using irrelevant spam content from automated accounts (known as bots) to drown out material targeted for suppression - or "flooding" - is a known tactic that the Chinese government has used during protests in Hong Kong and COVID lockdowns, say researchers at the Atlantic Council's Digital Forensic Research Lab. One of the hallmarks of such information operations is the activation of long-dormant accounts, which has been observed during this round of protests.

Researchers at the DFR Lab have suggested that tweeting over 72 times a day is bot-like behavior. NPR identified over 3,500 accounts that have done so and mentioned China's three largest cities at least once a day from Nov 21, 2022 to Nov. 30. The data shows an uptick in the number of these accounts, peaking on Nov. 28.

The seeming surge in spam accounts also comes as Twitter's new owner, billionaire Elon Musk, has slashed the company's teams that worked in non-English languages and monitored the site for disinformation, manipulation and government-sponsored propaganda campaigns. Musk dissolved Twitter's outside Trust and Safety Council on Tuesday.

But researchers caution that the narrative of government-sponsored spam accounts trying to drown out news of the protests online is not cut and dried. Attributing bot activities to the Chinese government typically requires more concrete evidence, and bots advertising sexual content and mentioning city names were active and pervasive on Twitter for at least several weeks before a deadly fire in the Chinese city of Urumqi set off the protests.

Whose spambots?

Researchers say that spamming activity alone isn't conclusive evidence to suggest a government information operation. It could simply be what social media watchers call hashtag hijacking, in which organizations identify trending topics - sometimes using bots - and incorporate them into their tweets to drive traffic to their accounts.

When House Speaker Nancy Pelosi visited Taiwan, raising ire from the Chinese government and generating significant discussion online, fan groups of Korean and Chinese entertainers used hashtags related to the visit to boost their idols' social media popularity, even when there was no relationship between the pop stars and the hashtags, DFR Lab researchers told NPR.

Chinese information operations tend to target more specific topics, individuals or small groups rather than city names, the researchers say.

Rather than focus on Shanghai, the Chinese government would more likely try to flood mentions of locations where the protests happened, say Darren Linvill and Patrick Warren at Clemson University's Media Forensics Hub.

They also say that other known information operations considered linked to the Chinese government tend to not only engage in flooding, but also amplify messages aligned with the state's agenda.

A prominent example is from 2019 when Twitter identified over 900 accounts the company said were linked to the Chinese government. While Twitter was never specific about how it zeroed in on those accounts, researchers at the analytics firm Graphika identified patterns of behavior and unearthed a network of related accounts across other social media platforms like YouTube and Twitter. Graphika's report identified narrative themes the accounts would coalesce around, ranging from personal attacks to support for the police.

Search results of major cities outside of China also turn up similar escort ads, wrote Ray Serrato, a former member of Twitter's safety and integrity team, in a blog post.

Some of the bots could also just be advertising sex services, which are banned in China, researchers say. A reporter for Semafor reached out to one of the advertised accounts and received a response asking where in Beijing the potential client is.

Preparing for unrest?

It's also possible that the bots were created in anticipation of unrest tied to the 20th Party Congress, where Chinese President Xi Jinping solidified his precedent-breaking third-term rule, DFR Lab's Kenton Thibaut says.

About half the bot-like accounts NPR identified, both before and after the fire, were created in 2022 - recent creation is a major sign of inauthentic activity. NPR shared a random sample of tweets with researchers at the Social Media Research Foundation, a non-profit that analyzes social media content. Their network analysis showed a large group of accounts that repeatedly post escort ads - not all at a bot-like level - and do not otherwise interact with other users. The escort ad group of accounts was the largest group in the search results before the fire and initially after the fire, and they were mostly created from September to October of 2022.

"They would want to have that infrastructure in place to be able to roll out quickly in case something needed to happen," says Thibaut.

Researchers, activists and policy makers have raised concerns that government-backed influence operations could flourish on Twitter after Musk's takeover and subsequent cuts to its trust and safety teams. The company says it is committed to providing a safe environment to users and will rely more heavily on automated tools.

The spamming doesn't seem tied to Twitter's change in management. Social Media Foundation researchers pulled search results days before and after Musk's takeover and showed that spam accounts were already the largest cluster of accounts at that point.

As NPR has reported, Twitter, like other major social networks, has struggled with moderating content outside of the US, facing challenges in navigating non-English languages, politics and culture. With prior mechanisms of international content moderation now degraded, many worry that the situation is going to worsen.

Ultimately, researchers say it wouldn't be surprising if some government-linked bot accounts were part of the activity in November. "I bet there is something in that data, but separating the wheat from the chaff is really hard." Linvill says.

As November turned into December, the number of active bot-like accounts returned to pre-protest level. Local governments in China relaxed COVID restrictions, authorities tracked down protest participants and the on-the-ground protests in China subsided.

Methodology

NPR downloaded Twitter search results mentioning either Beijing, Shanghai and Guangzhou in Chinese from Nov 21, 2022 to Dec. 1, broke up the dataset into three tranches of equal amount of time - Nov. 21 to Nov.24, Nov. 24 to Nov. 27, and Nov. 27 to Nov. 29, and shared samples containing 5% of these tranches with the Social Media Research Foundation. View NPR's analysis here. [Copyright 2022 NPR]