I’m not a significant contributor to the academic scientific literature but I am a big user of it. What I see has me worried, and itchy for the obvious solution. Science isn’t broken, but gatekeeping journals expected to give any kind of stamp of quality are.
After the Surgisphere debacle (see my blog post, the big piece in The Guardian that blew it all sky high a few days later, and fallout from retractions from the world’s top two medical journals), I’ve been pondering scientific publishing and the failings of pre-publication peer review. Then a bunch of other examples emerged, including this retraction from Nature on mental health impact of gaming in Africa (87% of insomnia variance explained by gaming habits!), and this statistical-model-free piece on the impact of wearing masks in Proceedings of the Natural Academy of Sciences (not retracted yet but I have hopes). And that’s not even a complete list of the bad, published, peer reviewed scientific articles I’ve seen taken up with considerable publicity in the media recently.
One thing is clear – pre-publication peer review says very little about quality. I certainly never want to hear again an article defended on the grounds that “it is in a peer reviewed journal”, or critiqued on the grounds that it isn’t. But review of some sort is essential. It’s just that this isn’t coming from journals. So how to do it better?
