Text- and image-generating tools present a new hurdle for efforts to tackle the growing number of fake papers making their way into the academic literature.
Advances in artificial intelligence (AI) are complicating publishers’ efforts to tackle the growing problem of paper mills — companies that produce fake scientific papers to order. Generative AI tools, including chatbots such as ChatGPT and image-generating software, provide new ways of producing paper-mill content, which could prove particularly difficult to detect. These were among the challenges discussed by research-integrity experts at a summit on 24 May, which focused on the paper-mill problem.
Paper mills and their fraudulent papers were already a scourge just a few months ago. Still, generative Large Language Models (LLM) and AI image creation are making them increasingly hard to detect. Institutions, publishers and funding bodies are facing an almost impossible challenge. It seems unlikely that the sharp-eyed sleuths that have been previously detecting manipulated images, shonky papers and the work of paper mills will still be able to detect these problems. This is a frightening state of affairs with wide potential impact.
“I have seen fake microscopy images that were just generated by AI,” says Jana Christopher, an image-data-integrity analyst at the publisher FEBS Press in Heidelberg, Germany. But being able to prove beyond suspicion that images are AI-generated remains a challenge, she says.
Language-generating AI tools such as ChatGPT pose a similar problem. “As soon as you have something that can show that something’s generated by ChatGPT, there’ll be some other tool to scramble that,” says Christopher.
A stream of papers
Anna Abalkina, a social scientist at the Free University of Berlin and an independent research-integrity analyst, suspects that there might be a delay in these AI tools becoming more apparent in the academic literature because of the length of the peer-review process. Perhaps in the next few months, “we will see the first stream of papers”, she says.
