In Spring 2020, early during the COVID-19 pandemic, 2 prominent studies that used a database from Surgisphere, a little-known company, were quickly retracted because of concerns about fraudulent data. One study1 purported to study the safety and efficacy of hydroxychloroquine for treating hospitalized patients with COVID-19. Published online in the Lancet on May 22, 2020, it was retracted on June 13, 2020.2 As of June 11, 2021, the study had an Altmetric attention score of 23 084. The second study3 claimed to investigate the association between cardiovascular disease, renin-angiotensin–aldosterone system inhibitor therapy, and COVID-19 outcomes. Published online May 1, 2020, in the New England Journal of Medicine (NEJM), it was retracted on June 4, 2020.3 As of June 11, 2021, the study had an Altmetric attention score of 3727. We sought to evaluate the association of the widely publicized retraction4,5 with further citations of the NEJM study.3
A concerning topic that is not discussed often enough, especially for medical research, is that the continued citation of retracted papers perpetuates the toxic impact of papers that have been retracted. The consequences of the longevity of these toxic papers (sometimes referred to as zombie papers) has been especially apparent for COVID research. Is only a question of the validity of the scientific record, it is about patient safety and the degree in which crazy/loopy theories are perpetuated by continued citation. We have included links to two related items.
On April 2, 2021, and with an update on May 31, 2021, we used Google Scholar to identify works that cited the retracted NEJM article from Mehra et al.3 We verified the citations by reviewing the full text of the articles. For each citing article, we determined the initial date of publication, whether the retraction was cited or otherwise noted, and if a study that was reported in the article used the data in a secondary analysis. We also determined the country of origin, article language, type of article, and the number of secondary citations (ie, citations of the article that cited the retracted study).3 Because our study did not involve human participants, it was deemed exempt from institutional board review.
We identified 934 articles that cited the retracted study; of these, 146 (15.6%) were preprints or not from peer reviewed journals, 63 (6.7%) were duplicate citations in Google Scholar, and 2 (0.2%) were false links. We could not locate the full text of 10 articles (1.1%). Of the remaining 713, a citation was verified in 652 (91.4%) (http://individual.utoronto.ca/leet/citingpapers.pdf) (Table). Of the 652 verified citations, 70 citations (10.7%) preceded the month of retraction and 227 (34.8%) of the citations occurred within the first 2 months of the retraction. Most citing articles (355 [54.4%]) were published 3 months or later after the retraction; 181 (27.8%) were published 6 months or later (Table and Figure). In May 2021, 11 months after the article was retracted, it was cited 21 times. Of the 652 verified citations, 115 (17.6%) cited or otherwise noted the retraction. Most citations were used to support a statement in the main text in the article; however, in 17 articles (2.6%) the authors incorporated the data into a new analysis. Two of these articles noted the retraction. Of these 17 articles, 11 (64.7%) were published 3 months or later and 7 (41.2%) were published 6 months or later after the retraction; as of May 31, 2021, the 17 articles had a median of 19 secondary citations (interquartile range, 2-49).
Lee TC., Senecal J., Hsu JM. & McDonald EG. (2021) Ongoing Citations of a Retracted Study Involving Cardiovascular Disease, Drug Therapy, and Mortality in COVID-19. JAMA Intern Med. 2021;181(11):1535–1537. doi:10.1001/jamainternmed.2021.4112
Publisher (Open Access): https://jamanetwork.com/journals/jamainternalmedicine/fullarticle/2782460
