Robustness of evidence reported in preprints during peer review (Papers: Lindsay Nelson et. al.

Posted by Connar Allen in Human Research Ethics, Research Integrity on May 5, 2023
Keywords: Institutional responsibilities, Journal, Peer review, Research results

The Linked Original Item was Posted On November, 1 2022 10:26:23

Summary

This open access paper (published in November 2022) and the research it reports looked at the degree to which the scientific details of preprint papers were altered when they appeared in a peer-reviewed journal. This highlights that the value of rapid publication and open/immediate peer review does not entail a loss of quality and vigour. It does indicate that preprints serve a valuable role during pandemics and other global medical emergencies. Perhaps, certain grant funding bodies need to take a more nuanced approach to preprints.

Scientists have expressed concern that the risk of flawed decision making is increased through the use of preprint data that might change after undergoing peer review. This Health Policy paper assesses how COVID-19 evidence presented in preprints changes after review. We quantified attrition dynamics of more than 1000 epidemiological estimates first reported in 100 preprints matched to their subsequent peer-reviewed journal publication. Point estimate values changed an average of 6% during review; the correlation between estimate values before and after review was high (0·99) and there was no systematic trend. Expert peer-review scores of preprint quality were not related to eventual publication in a peer-reviewed journal. Uncertainty was reduced during peer review, with CIs reducing by 7% on average. These results support the use of preprints, a component of biomedical research literature, in decision making. These results can also help inform the use of preprints during the ongoing COVID-19 pandemic and future disease outbreaks.

Nelson, L., Ye. H., Schwenn A, Lee S, Arabi S, Hutchins BI. Robustness of evidence reported in preprints during peer review. Lancet Global Health. 2022 Nov;10(11):e1684-e1687. doi: 10.1016/S2214-109X 00368-0. PMID: 36240832; PMCID: PMC9553196.

Publisher (Open Access): https://www.thelancet.com/journals/langlo/article/PIIS2214-109X(22)00368-0/fulltext

Robustness of evidence reported in preprints during peer review

Researchers push preprint reviews to improve scientific communications – Science (Jeffrey Brainard | December 2022)

Preprints and open peer review come of age – Research Information (February 2022)

Point of View: Promoting constructive feedback on preprints with the FAST principles – eLife (Sandra Franco Iborra, et al | April 2022)

Science in motion: A qualitative analysis of journalists’ use and perception of preprints (Preprint Papers: Alice Fleerackers, et al | February 2022)

Open peer review is the key to tackling public health misinformation – Times Higher Education (Rebecca Lawrence | June 2022)

Preprints ‘largely unchanged’ by peer review, even during Covid – Times Higher Education (Simon Baker | February 2022)

Publishing of COVID-19 preprints in peer-reviewed journals, preprinting trends, public discussion and quality issues (Papers: Ivan Kodvanj, et al | January 2022)

Opinion: In Defense of Preprints – The Scientist (Richard Sever & John Inglis | November 2021)

(UK) Covid 19: How harm reduction advocates and the tobacco industry capitalised on the pandemic to promote nicotine – BMJ Blog (June 2021)

Reading Peer Review – What a dataset of peer review reports can teach us about changing research culture – LSE Impact Blog (Martin Eve, et al | March 2021)

(EU) How pandemic-driven preprints are driving open scrutiny of research – Horizon (Rex Merrifield | April 2021)

Changes in the Scientific Information Environment During the COVID-19 Pandemic: The Importance of Scientific Situational Awareness in Responding to the Infodemic – Mary Ann Liebert, Inc (John K. Iskander | December 2020)

ACN - 101321555 | ABN - 39101321555

Australasian Human Research Ethics Consultancy Services Pty Ltd (AHRECS)