DIGITALISATION

Can you spot the bots? New research says not

Until recently it has been a challenge to make convincing fake social media profiles at scale because images could be traced back to their source, and the text often didn't sound human-like.

Can you spot this white robot holding a tablet and looking into the camera?

Today with rapid advances in artificial intelligence it is increasingly becoming difficult to tell the difference. Researchers from Copenhagen Business School decided to conduct an experiment with 375 participants to test the difficulty in distinguishing between real and fake social media profiles.

They found participants were unable to differentiate between artificially generated fake Twitter accounts and real ones, and in fact, perceived the fake accounts to be less likely to be fake than the genuine ones.

The researchers created their own mock twitter feed where the topic was the war in Ukraine. The feed included real and generated profiles with tweets supporting both sides. The fake profiles used computer-generated synthetic profile pictures created with StyleGAN, and posts generated by GPT-3, the same language model that is behind ChatGPT.

“Interestingly, the most divisive accounts on questions of accuracy and likelihood belonged to the genuine humans. One of the real profiles was mislabelled as fake by 41.5% of the participants who saw it. Meanwhile, one of the best-performing fake profiles was only labelled as a bot by 10%,” says Sippo Rossi, a PhD Fellow from the Centre for Business Data Analytics at the Department of Digitalisation at Copenhagen Business School.

“Our findings suggest that the technology for creating generated fake profiles has advanced to such a point that it is difficult to distinguish them from real profiles,” he adds.

The research was published in the Hawaii International Conference on System Sciences (HICSS)

“Our findings suggest that the technology for creating generated fake profiles has advanced to such a point that it is difficult to distinguish them from real profiles”

Sippo Rossi, PhD Fellow from the Centre for Business Data Analytics at the Department of Digitalization at Copenhagen Business School

Potential for misuse

“Previously it was a lot of work to create realistic fake profiles. Five years ago the average user did not have the technology to create fake profiles at this scale and easiness. Today it is very accessible and available to the many not just the few,” says co-author Raghava Rao Mukkamala, the Director of the Centre for Business Data Analytics at Department of Digitalisation at Copenhagen Business School.

From political manipulation to misinformation to cyberbullying and cybercrime, the proliferation of deep learning-generated social media profiles has significant implications for society and democracy as a whole.

“Authoritarian governments are flooding social media with seemingly supportive people to manipulate information so it’s essential to consider the potential consequences of these technologies carefully and work towards mitigating these negative impacts,” adds Raghava Rao Mukkamala.

Future research

The researchers used a simplified setting where the participants saw one tweet and the profile information of the account that posted it, the next research step will be to see if bots can be correctly identified from a news feed discussion where different fake and real profile are commenting on a specific news article in the same thread.

“We need new ways and new methods to deal with this as putting the genie back in the lamp is now virtually impossible. If humans are unable to detect fake profile and posts and to report them then it will have to be the role of automated detection, like removing accounts and ID verification and the development of other safeguards by the companies operating these social networking sites,” adds Sippo Rossi.

“Right now my advice would be to only trust people on social media that you know,” concludes Sippo Rossi.

Illustration picturing ChatGPT as a black box producing content read by people

Everyone uses ChatGPT, but it is a black box: here's how to make the most of it

CBS Associate Professor Abayomi Baiyere and Director of Digital Innovation at Lundbeck Sanne Thougaard at AI mindset event with play button

What's next? Developing an AI-agile mindset

Podcast Big question little time on sustainable development cover

Can ChatGPT contribute to sustainable learning?

A little girl is playing with a rubik's cube, illustrating the idea that AI is not magic

Artificial Intelligence is not magic

Did you find this content interesting? Sign up for our newsletter and gain access to more of our research findings and events.

With the world’s grand challenges awaiting us, we need to continuously evolve, gain new knowledge and insights, and upskill.

Dive into a world of knowledge by signing up below. You will get:

Personal event invitations
Pertinent research-based insights on business and society
Information about the next intakes of our Executive Education programmes and courses

Executive MBA

CBS EMBA's unique programme design and real-world applicability provides the platform for you to navigate a turbulent global business environment and become a leader in business who can steer your organisation to sustainable impact and success.

Learn more

Master of Business Development

The degree for those who are passionate about creating new, innovative solutions and want to take responsibility for elevating and developing the business - for those who want more.

Learn more

Digital media strategy

In Denmark, two-thirds of the advertising budget is spent digitally, and digital media are well on their way to becoming the largest item on many companies’ marketing budget. Nevertheless, many still lack a strategic approach to managing the contact created with customers and the outside world on paid, earned, and owned digital media. This is what you will learn in the course. The teaching is in Danish 🇩🇰.

Learn more

Can you spot the bots? New research says not

“Our findings suggest that the technology for creating generated fake profiles has advanced to such a point that it is difficult to distinguish them from real profiles”

Keep reading

Your gateway to more knowledge

Keep learning