Text scans by artificial intelligence will flag inconsistent or unusual patterns in reviewer behaviour, says Wolverhampton professor

The scornful comments of “reviewer 2” have become a running joke in academia. But a new artificial intelligence system – dubbed a “TripAdvisor for peer review” – may soon be able to test whether the scathing remarks of anonymous referees are being handed out fairly or not.

Amid concerns that peer reviewers are harsher in their criticisms of female researchers or those from less prestigious institutions, PeerJudge scans peer review reports for keywords – either positive or negative – to see whether reviewers are unduly tough when assessing certain types of researchers.

The program, which was created by technology company F1000 and researchers at the University of Wolverhampton, also checks whether reviewer comments correspond with the final recommendation to accept or reject the paper – an area of grievance for researchers when broadly positive comments on their manuscripts are followed by a call to reject.

Read the rest of this discussion piece