AI-generated exam papers go undetected in 94% of cases, study finds

Nearly all submissions generated by artificial intelligence (AI) went undetected in a recent test of UK universities' exam systems.

Nearly all submissions generated by artificial intelligence (AI) went undetected in a recent test of UK universities' exam systems.

The AI-generated papers also received on average higher grades compared to those written by real students, researchers at the University of Reading found. They published their findings in the open-access journal PLOS ONE.

Artificial intelligence already has proven that it can pass exams which has led some schools and universities to ban students from using available AI tools such as ChatGPT.

However, applying this rule proved to be challenging, according to the new study.

Researchers submitted exams written by AI for five different courses for an undergraduate degree in psychology at the university.

They found that 94 per cent of these submissions went undetected in what they called a “Turing test” case study.

Named after the British mathematician and computer scientist Alan Turing in 1950, this measures a machine's ability to exhibit intelligent behaviour akin to that of a human.

‘Extremely concerning’

The authors said the finding was “extremely concerning”, especially as the “content of the AI-generated answers” was unmodified by researchers.

“Overall, our 6 per cent detection rate likely overestimates our ability to detect real-world use of AI to cheat in exams,” they noted in the study, adding that it would be more plausible that students would modify the AI output to make it less detectable.

Moreover, in 83.4 per cent of instances, the AI-generated submissions achieved higher grades compared to a randomly chosen group of the same number of exams from actual students.

One exception was a module which involved more abstract reasoning, which is something AI can struggle to manage compared to real students.

“The results of the ‘Examinations Turing Test’ invite the global education sector to accept a new normal and this is exactly what we are doing at the University of Reading,” the study’s authors said in a statement.

“New policies and advice to our staff and students acknowledge both the risks and the opportunities afforded by tools that employ artificial intelligence,” they added.

The researchers expressed concern regarding academic integrity in light of these findings and suggested that supervised, in-person exams could mitigate the issue.

However, as AI tools continue to evolve and become prevalent in professional environments, universities might need to explore ways to integrate AI into education as part of the "new normal".