TikTok and YouTube bombarding boys and men with misogynistic content within minutes of signing up, Irish study finds

TikTok and YouTube Shorts algorithms are bombarding boys and men with misogynistic and male supremacist content within minutes of signing up, a study conducted by DCU has found.

The Dublin City University Anti-Bullying Centre study shows that the recommended algorithms used by social media platforms are “rapidly amplifying misogynistic and male supremacist content”.

Overall, YouTube Shorts accounts were recommending a larger amount of toxic content, 61.5pc of the total recommended content, than TikTok accounts which recommended 34.7pc of toxic content overall.

The study recorded and coded the content recommended to 10 experimental or ‘sockpuppet’ accounts on 10 blank smartphones – five on YouTube Shorts and five on TikTok.

Content featuring ‘Manfluencers’ or male influencers, accounted for the vast majority of recommended videos in the dataset.

By far the most prevalent of these was Andrew Tate, who featured 582 times on the YouTube Shorts accounts and 93 times on the TikTok accounts.

Professor Debbie Ging, who conducted the study with Dr Catherine Baker and Dr Maja Andreasen said it shows that shutting down influencers’ accounts does not necessarily remove their content.

“The overwhelming presence of Andrew Tate content in our dataset at a time when he was de-platformed means that social media companies must tackle harmful content in more sophisticated ways,” she said.

Researchers found that all of the male-identified accounts were fed masculinist, anti-feminist and other extremist content, irrespective of whether they sought out this material or not.

They all received this content within the first 23 minutes of the experiment.

Once the account showed interest by watching this sort of content, the amount rapidly increased.

By the last round of the experiment, after 400 videos or two to three hours viewing, the vast majority of the content being recommended to the phones was toxic (TikTok 76pc and YouTube Shorts 78pc), with content primarily falling into the alpha male and anti-feminist category.

The other toxic categories were reactionary right wing views and conspiracy, which accounted for 13.6pc of recommended content on TikTok and 5.2pc of recommended content on YouTube Shorts.

Much of this was anti-transgender content, the DCU’s Anti-Bullying Centre report noted, with the centre known worldwide for its research excellence in bullying and online safety.

The findings of the report point to urgent and concerning issues for parents, teachers, policy makers, and society as a whole, the study’s authors warned.

Among the authors’ recommendations are better content moderation, turning off recommender algorithms by default and cooperation with trusted flaggers to highlight illegal, harmful, and borderline content.

They also stress the need for teacher education and the teaching of critical digital literacy skills in schools to equip young people with a better understanding of how influencer culture and algorithms work.

“Ultimately, girls and women are the most severely impacted by these beliefs, but they are also damaging to the boys and men who consume them, especially in relation to mental wellbeing.

“The social media companies must come under increased pressure from the government to prioritise the safety and wellbeing of young people over profit,” Prof Ging added.

TikTok declined to comment on the study.

However, the company said the study is not reflective of how the broad user base would experience the platform, as the study relies on the experience of 12 hours of content viewed across five TikTok accounts which is a limited sample size.

TikTok said it does not allow hate speech or hateful behaviour on its platform, including misogyny and transphobia.

It insisted it takes action against any content or behaviour that violates its community guidelines, by removing videos, comments, accounts, and hashtags. It also prevents similar content from being recommended in the For You feed.

YouTube has also been contacted for a comment.

Meanwhile, Childline has said more than 140 children have contacted it to talk about self-harm in the wake of revelations regarding the use of TikTok on RTÉ’s Prime Time programme.

The chief executive of the The Irish Society for the Prevention of Cruelty to Children (ISPCC) John Church, said the findings of the report, ‘13 on TikTok’ broadcast last night “will come as no surprise to those of us working to safeguard children and young people in Ireland”.

Mr Church said the “particularly frightening aspect of this investigation” is the fact that the Prime Time team did not seek out topics, “like” , comment, or engage with any particular content.

“However, within minutes the accounts which TikTok assumed were those of a 13-year-old were brought down a mental health rabbit hole,” he said.

Mr Church added that while Childline hears from children and young people every day about the effect that technology can have on their mental health, “to see it played out in real time was truly shocking”.

“Since February alone, 145 children have reached out to us at Childline to talk about self-harm, and 65 of those children are repeat contacts who have then gone on to reach out to us for help with suicide ideation,” Mr Church said.

“Remember this is more than mere numbers. Behind all of our contacts is a child or young person desperate for help. So I ask, with these shocking statistics in mind what are the technology platforms really doing to safeguard children?

His comments follow an investigation by RTÉ’s Prime Time which showed users as young as 13 are being exposed to videos relating to self-harm and suicide on TikTok.

Referring to the social media companies, Mr Church stressed they are “for-profit businesses that are not in the business of child protection”.

“It is painfully evident from RTÉ’s investigation that they are pumping out the most dangerous messages possible to already vulnerable children, and it is high time steps are taken to address dangerous algorithmic amplification,” he added.

He also insisted that Ireland must have comprehensive regulations and legislation to tackle the issue.

TikTok yesterday told an Oireachtas committee that it removes 80 million accounts of under-age users every year.

The revelation came during a Children’s Committee hearing on child protection in the context of artificial intelligence.

Representatives from Facebook and Instagram parent Meta, TikTok, and X (formerly Twitter) were told by TDs and senators that “social media is a cesspit” and their companies were not doing enough to protect children.

One of the issues discussed at committee was age verification of users on apps to protect children.

Meta’s head of public policy in Ireland, Dualta Ó Broin, suggested a solution to concerns over age verification could be done at App Store level, taking the burden off individual apps – particularly newer companies that see rapid rises in users.

“That would be a step forward,” he said. “It would be a resolution of the age verification question. We would still have huge responsibilities to ensure that all of these users are then placed into an age-appropriate experience.”

He said other solutions included the process being done by telecommunications companies or by device.

The social media giant, which owns Facebook, WhatsApp and Instagram, said it dismantled 27 abusive networks and banned almost half a million accounts for child safety violations between 2020 and 2022.

Fine Gael senator Mary Seery Kearney raised concern about social media platforms’ “deliberate manipulation” of users and resultant “behaviour modification”.

She said the companies at the committee had a business model based on the capture of attention, adding that smartphones should be banned for young people.

Ms Seery Kearney said she wanted to see more time limits on app use, adding: “Social media needs to come with a mental health warning.”

TikTok’s public policy lead for child safety, Chloe Setter, said she “totally appreciates” the senator’s concerns, but added there is no agreement among experts on what amount of time is considered “good”.

She said TikTok had take-a-break reminders, usage limits and push alert cut-offs associated with age.

Meta’s director of safety policy, David Miles, told the politicians their concerns were justified and the company was working with safety experts.

He said the industry had seen a dramatic rise in the youth demographic and that “things need to change”.

Echoing recent comments from Tánaiste Micheál Martin, Fianna Fáil TD Jennifer Murnane O’Connor said the impact of social media on children is “the new public health crisis of our time”.

She said there would soon be funding for schools to support the banning of smartphones during class time.

Susan Moss, head of public policy at TikTok, replied: “I agree with you. Schools are a place for education. They’re not a place for smartphones and the internet.”

More generally, she said TikTok would invest two billion euro in trust and safety in 2024.

The committee was told that more than two million people in Ireland use the platform every month.

Ms Moss said said TikTok “meticulously monitors” for child sexual abuse material.

She said all content on the platform undergoes some form of moderation, including by automated systems, to detect harmful material.

Ms Setter: “To give you a sense of the effort we’re putting in, we remove on average 20 million suspected under-age accounts every quarter globally.”

The platform is designed for people aged 13 and over.

Fianna Fáil senator Erin McGreehan told the companies: “Social media is a cesspit and X is the worst.”

Her party colleague, senator Malcolm Byrne, said it was a “serious problem” that the social media company had more than halved the number of human moderators it employs under the ownership of Elon Musk, down from 5,500.

He said young people told him the content on X was “far more gratuitous, far more violent, and far more sexual” than other platforms.

He added: “You see far more trolls and bots and misinformation and disinformation on your platform and your AI model is not picking it up.

“And… not picking it up for the wider population, not picking it up for children and teenagers, is really dangerous.”

Claire Dile, X’s director for government affairs in Europe, said the company could make more effort to detect and remove harmful content as quickly as possible.

She said the company had launched a new moderation centre and had begun to rehire moderators as part of a variety of enforcement actions, including AI processes.

She also said fighting child sexual exploitation material, including AI-generated images, is the company’s “number one priority”.

Get ahead of the day with the morning headlines at 7.30am and Fionnán Sheahan’s exclusive take on the day’s news every afternoon, with our free daily newsletter.