A face-swapping app shows how easily deep fake tech can be used as a 'weapon' against women

Deepfakes use AI to edit the likenesses of one person’s face in a video and swap it with another face. Getty Images

• App stores removed Facemega, a deepfake app that put Emma Watson’s face in a suggestive video.
• The suggestive ad was promoted on Meta, which later removed it following reporting from NBC News.
• An attorney said new deepfake tech could become a “weapon” of harassment against women and girls.

Multiple online stores and Meta have removed a controversial face swap app that promoted a sexually suggestive ad featuring the face of the “Harry Potter” actor Emma Watson imposed onto someone else.

The app for creating deepfakes, called Facemega, showed a woman with Watson’s face smiling flirtatiously and then getting down on her knees for what appears to be a man. The caption read: “Swap any face into the video!” A screen recording of the ad posted to Twitter got 3.2 million views.

Deepfakes use artificial intelligence to replace the likeness of one person with another in videos and other digital media, and its use has raised ethical and privacy concerns.

An attorney told Insider that this is just one illustration of how emerging deepfake technology could be used as a “weapon” against women and girls.

Michael Farhi, an attorney who has published material on the subject before, told Insider that the use of AI “is going to result in a huge jump in it being a weapon against women online all across the world.” He added that women are the most popular demographic to target with such harassment.

Deepfakes and revenge porn are already threats to women, and AI technology, which can make fake videos look convincingly real, is going to “quadruple” the negative impacts, Farhi said.

The Facemega ad ran on Facebook until the site removed it following coverage by NBC News, who reported that the app also published advertisements with Scarlett Johansson’s face in provocative videos. Facemega was listed for free on Apple and Google Play, who have both since removed it, but similar apps remain listed in both stores.

A spokesperson for Apple told Insider that the company removed the app from the Apple Store and that the company does not allow apps that include defamatory, pornographic, or mean-spirited content intended to humiliate others. A spokesperson for Google said the company took “appropriate action” and removed the app for “violations of our policies.”

A Meta spokesperson told Insider that their “policies prohibit adult content regardless of whether it is generated by AI or not,” and that FaceMega’s page was restricted on Facebook.

Watson’s legal representation could not be reached for comment.

Stills of the advertisement for a deepfake app that used Emma Watson’s face. Twitter

The suggestive ad appeared to violate Facemega’s own terms and conditions of service, which prohibit users from uploading content that is defamatory or sexually explicit.

The Chinese software company Wondershare owns Facemega, a spokesperson for the company confirmed to Insider. When asked about the advertisement, the Wondershare spokesperson said: “Our legal department is already following up on this matter, and the advertising content has also been removed from the shelves.”

Deepfake technology will be used as a ‘weapon’ against women, attorney said

In the US, victims of deepfake porn “have potential claims like defamation, invasion of privacy, infliction of emotional distress,” Farhi told Insider. But he also said that initiating legal action internationally against a Chinese company would be a “hurdle.”

“A lawyer wherever in the US sending a cease-and-desist type letter to an entity in China, even if it’s translated appropriately, it’s going to have little to no effect as a practical matter. So where do you go next? You go to Facebook, which has challenging rules and regulations of its own in terms of what it posts and what ads it puts up,” Farhi said.

Public pressure against companies who share the content may be more effective than legal actions, Farhi said. He added that “the advances in technology are happening too fast for it to be legislated.” Deepfake technology, like all tech advancements, comes with “pluses and minuses,” but Farhi said the potential damages are being “swept under the rug.”

“The advocates of AI are, of course, pushing the idea that you’ll get instant research, instant assistance at work. For creators, it’s a great asset and a great tool,” Farhi said. “It’s going to be a frigging nightmare, even more than what’s out there now, as a tool or a weapon to harm women.”

Farhi said an especially vulnerable group is likely to be teen girls, whom revenge and deepfake porn already bombard on a regular basis.

“This is going to have real harm to untold numbers of girls and women. Not just in high school, before high school at the elementary-school or middle-school level, all the way up to women in their twenties, thirties, and beyond,” Farhi said. “And the easier it is to do, or the easier it is for a predator to do — and it’s going to be a lot easier — the more it’s going to be done.”