Amid bans and restrictions on their use, artificial intelligence tools are creating interest among those who see a solution to systemic peer-review woes.
Debate over the use of artificial intelligence, already touching everything from admissions to grading, has reached peer reviewing, as academics balance technological uncertainty and ethical concerns with potential solutions for persistent peer-review problems.It is not hard to see the signs of peer review in trouble. It is struggling with a crippling workload as the number of new papers for the review grows at an exponential rate. There are also indications that a lack of diversity among reviewers seriously undermines the process. Even though artificial intelligence tools offer a tempting solution, their bloopers and hallucinations are troubling. To date, most of the conversation about peer review and artificial intelligence has related to detecting the undisclosed use of AI tools in preparing outputs, but with much less said about its use in the conduct of peer reviews. This thought-provoking piece, published by Inside Higher Ed in October 2023, looks at whether AI can be helpful if it is used to support the work of peer reviewers.
Zou, along with Laurie Schintler, an associate professor at the Schar School of Policy and Government at George Mason University, both tackled the potential use of AI in peer review in recent papers. The verdict on the usefulness of the technology: the overlap between human and AI feedback is “comparable,” according to Zou, especially when it comes to less “mature” papers—either those in the early stages or ones that were never published.
Many academics are not thinking of using the technology for peer review just yet, Schintler said, but she expects it will become more widespread.
“I don’t think a lot of people are aware of this trend in using AI in peer review,” she said. “Some colleagues can’t even believe it’s being used at all, but definitely the trend is going in that direction and we’ll see more and more of it.”
While both scholars said they have yet to see AI in peer reviewing take off, scholarly journals and researchers are already trying to get ahead of the curve, laying the groundwork to restrict or outright ban AI in submissions.
