Highlights
- The overall quality of medical research remains poor, despite longstanding criticisms.
- The scientific enterprise is business-like and consistently undervalues its own backbone, methodology.
- Despite great initiatives to improve research quality, progress is modest.
- Top-down action from journals, funding agencies, universities and governments is needed to break the cycle. These actions should give methodology a central place in funding acquisition as well as study design, conduct, and reporting.
This open access paper reflects on the awful scientific and patient consequences of how we have all allowed science and scientific publication to be distorted by the obsession with metrics. Tinkering with components of the system isn’t working. We need a radical systemic reset. We have included links to nine related items.
Covid-19 research made it painfully clear that the scandal of poor medical research, as denounced by Altman in 1994, persists today. The overall quality of medical research remains poor, despite longstanding criticisms. The problems are well known, but the research community fails to properly address them. We suggest most problems stem from an underlying paradox: although methodology is undeniably the backbone of qualitative and responsible research, science consistently undervalues methodology. The focus remains more on the destination (research claims and metrics) than on the journey. Notwithstanding, research should serve society more than the reputation of those involved. While we notice that many initiatives are being established to improve components of the research cycle, these initiatives are too disjointed. The overall system is monolithic and slow to adapt. We assert that a top-down action is needed from journals, universities, funders and governments to break the cycle and put methodology first. These actions should involve the widespread adoption of registered reports, balanced research funding between innovative, incremental and methodological research projects, full recognition and demystification of peer review, mandatory statistical review of reports, adherence to reporting guidelines, and investment in methodological education and research. Currently, the scientific enterprise is doing a major disservice to patients and society.
Keywords
Research quality, Methodology, Reporting
Calster, B.V., Wynants, L., Riley, R.D., Smeden, M.V. & Gary S Collins, G.S. (2021) Methodology over metrics: Current scientific standards are a disservice to patients and society. Journal of Clinical Epidemiology. https://doi.org/10.1016/j.jclinepi.2021.05.018
Publisher (Open Access): https://www.jclinepi.com/article/S0895-4356(21)00170-0/fulltext
Methodology over metrics: Current scientific standards are a disservice to patients and society
The academic world quickly responded to the covid-19 situation, and produced a staggering
amount of research publications. Whilst nobody would disagree that we need organized
collaborative research efforts to study disease prevention, management and treatment,
the reality is that large swathes of re…
amount of research publications. Whilst nobody would disagree that we need organized
collaborative research efforts to study disease prevention, management and treatment,
the reality is that large swathes of re…
