Tools like ChatGPT can help, but transparency is vital, say Mohammad Hosseini and Serge Horbach
In December 2022, we asked ChatGPT to write a cynical review of the first preprint covering Covid-19 research. It responded by calling the preprint “yet another example of the questionable research coming out of China”, which could not be trusted because of “the lack of transparency and credibility of the Chinese research community”.
There are a few obvious ways in which the use of AI in peer review could be a concern. It can replicate bias, merely replicate flawed thinking and just regurgitate past practice. But used well, it can complement human decision-making, and supplement an overstressed system from centuries ago, that no longer matches modern inclusive society. In other fields we have seen that AI working in cooperation with skilled professionals, can achieve remarkable results. We hope this will be true of AI in peer review.
The two interactions show the incredible speed with which generative artificial intelligence is developing. They also highlight both the potential for Large Language Models (LLMs) such as ChatGPT and Bard to aid peer review, alleviating some of the problems that have undermined the system in recent years, and the possibility of creating new pitfalls—something we discuss in a recent paper.
Automation in peer review predates generative AI. Computer assistance with specific tasks, such as screening references, plagiarism detection and checking compliance with journal policies, has become commonplace. Generative AI, however, could significantly increase both the number of automated tasks and the degree to which they can be automated, benefiting specific parties within the peer review system.
