ACN - 101321555 Australasian Human Research Ethics Consultancy Services Pty Ltd (AHRECS)

Resource Library

Research Ethics MonthlyAbout Us

ResourcesResearch IntegrityHARKing: How Badly Can Cherry-Picking and Question Trolling Produce Bias in Published Results? (Kevin R. Murphy & Herman Aguinis | 2017)

Australasian Human Research Ethics Consultancy Services Pty Ltd (AHRECS)

HARKing: How Badly Can Cherry-Picking and Question Trolling Produce Bias in Published Results? (Kevin R. Murphy & Herman Aguinis | 2017)

Published/Released on December 11, 2017 | Posted by Admin on January 15, 2018 / , , ,
 


View full details | Go to resource #1, resource #2


Abstract

An interesting and informative take on this important issue.

The practice of hypothesizing after results are known (HARKing) has been identified as a potential threat to the credibility of research results. We conducted simulations using input values based on comprehensive meta-analyses and reviews in applied psychology and management (e.g., strategic management studies) to determine the extent to which two forms of HARKing behaviors might plausibly bias study outcomes and to examine the determinants of the size of this effect. When HARKing involves cherry-picking, which consists of searching through data involving alternative measures or samples to find the results that offer the strongest possible support for a particular hypothesis or research question, HARKing has only a small effect on estimates of the population effect size. When HARKing involves question trolling, which consists of searching through data involving several different constructs, measures of those constructs, interventions, or relationships to find seemingly notable results worth writing about, HARKing produces substantial upward bias particularly when it is prevalent and there are many effects from which to choose. Results identify the precise circumstances under which different forms of HARKing behaviors are more or less likely to have a substantial impact on a study’s substantive conclusions and the field’s cumulative knowledge. We offer suggestions for authors, consumers of research, and reviewers and editors on how to understand, minimize, detect, and deter detrimental forms of HARKing in future research.
.
Keywords
HARKing, Simulation Publication bias, Data snooping
.

Murphy, K.R. & Aguinis, H. J Bus Psychol (2017). HARKing: How Badly Can Cherry-Picking and Question Trolling Produce Bias in Published Results? Journal of Business and Psychology. https://doi.org/10.1007/s10869-017-9524-7
Publisher: https://link.springer.com/article/10.1007/s10869-017-9524-7
ResearchGate: https://www.researchgate.net/publication/321734216_HARKing_How_Badly_Can_Cherry-Picking_and_Question_Trolling_Produce_Bias_in_Published_Results



Resources Menu

Research Integrity


Human Research Ethics

0