One of the first megajournals, PLOS ONE, has played a significant role in changing scholarly communication and in particular peer review, by placing an emphasis on soundness, as opposed to novelty, in published research. Drawing on a study of peer review reports from PLOS ONE recently published as an open-access book, Martin Paul Eve, Daniel Paul O’Donnell, Cameron Neylon, Sam Moore, Robert Gadie, Victoria Odeniyi, and Shahina Parvin¸ assess PLOS ONE’s impact on the culture of peer review and what it can tell us about efforts to change academic culture more broadly.

Most scholars are familiar with peer review; the sometimes brutal system by which academic work is judged. The COVID-19 pandemic has even mainstreamed this process, with media outlets commenting on unreviewed preprints with warnings that work has “not yet undergone peer review”. Yet few venues, over the past two decades, have done more to challenge the way that peer review works than the Public Library of Science’s PLOS ONE title.

PLOS ONE initiated a revolution in peer-review that was predicated on the principle of ‘technical soundness’. The idea being that papers are judged according to whether or not their methods and procedures are thought to be solid, rather than whether their contents are judged to be important. A fairly radical idea from its inception, PLOS ONE was designed to foster Kuhnian “normal science”, rather than shooting for high-profile ‘discoveries’. Detractors worried PLOS ONE’s review process would let through a raft of useless, insignificant (non-)results. Proponents believed it could usher in an era of better science.

As this change represented a fundamental shift in a core research practice, we wanted to understand how peer reviewers behave at PLOS ONE, and were generously given access to an anonymised set of peer review reports from the journal. What did we find? In purely quantitative terms, the mean length of reviews was 3,081 characters, while the longest report we found was just under 14,000 words(!)