In Arizona, election workers trained with deepfakes to prepare for 2024

08/05/2024

The video message from Arizona Secretary of State Adrian Fontes warmly greeted the scores of election workers who had gathered at a Phoenix-area hotel in December for a first-of-its-kind drill: “We are very excited that all of you are here,” Fontes, a Democrat, began. “You are on the front lines, and this exercise is a prime opportunity for you to hone your skills by experiencing new challenges as a team.” He wished them luck.

Over two days, the election workers from around the state maneuvered through a training exercise involving the kinds of attacks generated by artificial intelligence that they might face in this politically competitive state during the coming election cycle. They tackled situations that plunged them into AI-generated scenarios ranging from law enforcement operations to attempts to infiltrate technology systems.

It was not until the end that the real Adrian Fontes told the shocked group that the person in the welcome video was a fake, an AI creation built out of publicly available images.

Dana Lewis, the county recorder in Pinal, a conservative area that borders metro Phoenix and has dealt with many false claims about elections, said the simulations in the exercise were both “mind blowing” and unsettling.

“By the end of the second day, you’re like: Trust no one,” Lewis recalled in an interview.

For years, election officials across the country have faced extraordinary pressure from rampant conspiracy theories and violent threats. But 2024 presents something more: widely available and highly convincing AI-generated deepfakes. FBI Director Christopher A. Wray recently told a national security conference that while the United States had faced malign foreign influence campaigns in the past, this election cycle’s adversaries would move faster, enabled by new technologies such as generative AI. And in the United States, elections are administered by the states, which means local officials like the ones who gathered in Arizona are on the front lines.

The Brennan Center, a nonpartisan law and policy group, and the Elections Group, which consults with election officials across the country, helped develop parts of the tabletop exercise for Arizona after studying and writing about the impact of AI on elections and hearing from election officials that they didn’t feel they had enough direction about how to deal with the threats from this new technology, said Larry Norden, senior director for elections and government at the Brennan Center.

The Arizona tabletop exercise — spearheaded by Fontes — was an effort to give one state’s election workers an insight into the types of attacks they might be up against. Following the event, organizers created a training document they plan to make available to election workers across the country, titled, “How Election Officials Can Identify, Prepare for, and Respond to AI Threats.” On Thursday, they plan to host an event in Phoenix for media to witness a similar simulation and are hoping to organize another in Michigan in the coming weeks. “But there is no way we are going to work with all 50 states, or that every election official in the country is going to be able to go through this kind of training before November,” said Norden. “That’s why we wrote the report.” (Officials say they destroyed the deepfakes used in the exercise, like the one described in this article, so they wouldn’t be misused later. The AI video accompanying the article, also featuring a fake Fontes, was created to warn the public of AI threats.)

In December, 10 teams of roughly eight to 10 employees from 14 of the 15 Arizona counties — representing the secretary of state, law enforcement, private vendors, as well as some federal employees — came together. To start, organizers assigned each group a fictitious county and a limited budget, one that could not possibly pay for all the offered security measures. The constraints were meant to mimic the real-life restrictions facing election workers today. A team could purchase anti-phishing training, for example, and then, as a result, would not face a simulated phishing attack but also might not have the budget to purchase a backup communications system.

“It’s very much like role-playing games, like Dungeons and Dragons,” said Fontes. “You’ve got to make decisions in the moment. And there’s an element of surreality. It’s kind of real, but it’s not.”

Even before the sessions — during which a deepfake of Lewis speaking in German and Chinese was played — she was careful about her social media presence and avoided posting on Twitter. Afterward, she culled her Facebook friends and began cropping photos to show only a corner of her face and blurring the faces of others. Lewis hopes her extreme edits reduce the chances that outsiders could deceptively manipulate her digital content during the election.

“I only post from the eyes up,” she said. “You can see my nose and one of my eyes and my famous little hair bob that I do.”

The last AI-generated video some of the election officials remembered seeing was a deepfake of Barack Obama from 2018, according to Toshi Hoo, director of the emerging media lab at the Institute for the Future, whose organization created the deepfakes for the Arizona exercise. But the Obama video “took a week of computer scientists and a super computer, and no one else could do that at the time,” said Hoo. “Now, anybody with a web browser and a web cam can go and make a clone of themselves.”

Indeed, in January, the weekend before the New Hampshire primary, voters received a call from what sounded like a digitally generated voice impersonating President Biden that advised them not to vote that Tuesday. New Hampshire’s attorney general announced a criminal investigation into a Texas-based company that was allegedly behind the calls.

To give the assembled Arizona election workers direct experience with the possibilities of AI, the Phoenix attendees listened to AI-generated voice-mail messages from staffers in the secretary of state’s office telling them to keep polling locations open due to nonexistent court orders. They heard calls from AI-generated voices of voters saying they had not received their ballots. They saw faked social media posts designed to suppress turnout, like one that appeared to come from Fontes’s X account that featured a photograph of a worried-looking Latino man along with a warning that immigration officials would be near polling places on election day and that “anyone in the vicinity may be subject to search and possible detention if you are unable to show proof of citizenship.”

Constance Hargrove, the elections director of Tucson-area Pima County who has worked in elections for three decades, recalled the experience as “surreal.” By the end of the training, she said, she and her colleagues “began to question everything,” from the identities of people and offices featured in social media posts to those on the other end of emails, phone calls, voice mails and text messages.

Many of the practical suggestions that came out of the exercise were relatively simple: Reinforce basic online security measures such as multifactor authentication. Use .gov web domains and secure and update your digital presence. Establish strong relationships with local media so you can alert them to fake information circulating in their county. Maintain a social media presence where correct information can be disseminated quickly. Call on law enforcement to address major issues.

Workers trained for potential impersonations and learned to develop code words and to call back to verify someone’s identity if an audio or email message seemed suspect.

Hargrove returned home from the training and immediately shared her concerns about deepfakes with her election staff, emphasizing the need to verify information before acting on it.

“You get an email and you feel like this is not something Constance would send in an email, then get out of your chair and walk to my office and ask me, ‘Did you really send this email?’”

“Much of the training involved reminding folks of best practices,” said Norden. “They had already been dealing with false information and bad foreign actors,” he added, “but it may be coming at you faster this time around.”

Fontes acknowledges there are private chatrooms and messaging apps where AI-generated fakes could flourish. “There’s a lot of spaces we just don’t have control over, and there’s a lot of spaces we just don’t have access to,” he said. “We’re just doing what we can in our lane.”

At the end of the second day of the exercise, the AI video of Fontes returned on screen to congratulate the assembled group.

“I’m going to call it a day, relax a little bit on our back patio, and watch my son practice ice hockey on the frozen lake behind our house here in Phoenix.”

This time, the election workers weren’t fooled by the deepfake Fontes. Ice in Arizona gave it away. A malicious actor would not leave such a helpful clue.