ACN - 101321555 Australasian Human Research Ethics Consultancy Services Pty Ltd (AHRECS)

Resource Library

Research Ethics MonthlyAbout Us

ResourcesPeer review

Australasian Human Research Ethics Consultancy Services Pty Ltd (AHRECS)

(US) NIH rejected a study of alcohol advertising while pursuing industry funding for other research – STAT (Sharon Begley | April 2018)0

Posted by Admin in on June 23, 2018
 

It’s rare for officials at the National Institutes of Health to summon university scientists from hundreds of miles away. So when Dr. Michael Siegel of Boston University and a colleague got the call to meet with the director of NIH’s Institute on Alcohol Abuse and Alcoholism, he said, “I knew we were in trouble.”

He never imagined, however, that at the 2015 meeting the director, George Koob, would leap out of his seat and scream at the scientists after their PowerPoint presentation on research the agency had eagerly funded on the association between alcohol marketing and underage drinking. “I don’t f***ing care!” Koob yelled, referring to alcohol advertising, according to the scientists.

Koob also made clear that NIAAA would pull back from such research, recalled Siegel and his colleague, David Jernigan of Johns Hopkins University, who described the previously undisclosed meeting in Bethesda, Md., in separate interviews with STAT. Shocked by the encounter, they retreated to an NIH cafeteria, asking each other what had just happened — and why.

(UPDATE – 19/06/2018 – KHN –165-Page Internal NIH Report Lays Bare Just How Cozy Scientists Were With Alcohol Industry)

Read the rest of this discussion piece

Advocating for publishing peer review – ASAbio (Iain Cheeseman | April 2018)0

Posted by Admin in on June 13, 2018
 

Journals play a critical role in the scientific process, refining research through peer review and disseminating it to appropriate communities. At its best, the publishing process is a partnership among editors, staff, authors, reviewers, and readers. Each group has a vested interest in working together to ensure a robust and fair editorial evaluation, rigorous and constructive review, and the broadest visibility of the work. This process functions best when each group communicates openly with the others as we strive to refine the way that science is conducted and disseminated.

One of the advantages of publishing peer reviews is that it will expose illegitimate publishers for what they are (vanity/pay-to-play publishers) but further study would be prudent on the impact of reputational concerns on reviews and to address the potential for compounded bias in reviews.

The last few years have seen a revolution in life sciences publishing with the adoption of preprints and increasingly diverse experiments from traditional journals. A recent meeting organized by ASAPbio, HHMI, and Wellcome focused on bringing more transparency and innovation to our peer review system. One of the most important tangible recommendations to come from the meeting, in my opinion, was the proposal that journals should widely adopt more transparency in the peer review process. There was near unanimous consensus from the participants—which included scientists, representatives of journals, scientific societies, and funders—that more journals should commit to publishing the peer reviews of every paper that appears in the journal. Similar to the peer review processes offered by EMBO Journal, eLife, and others, the reviewers could still remain anonymous, but this transparency would provide much better insight into the assessment process and nature of the paper.
.
Make your voice heard: writing letters to the editor
.

Read the rest of this discussion piece

(US) NIH moves to punish researchers who violate confidentiality in proposal reviews – Science (Jeffrey Brainard | March 2018)0

Posted by Admin in on June 9, 2018
 

When a scientist sends a grant application to the U.S. National Institutes of Health (NIH) in Bethesda, Maryland, and it goes through peer review, the entire process is supposed to be shrouded in secrecy. But late last year, NIH officials disclosed that they had discovered that someone involved in the proposal review process had violated confidentiality rules designed to protect its integrity. As a result, the agency announced in December 2017 that it would rereview dozens of applications that might have been compromised.

Now, NIH says it has completed re-evaluating 60 applications and has also begun taking disciplinary action against researchers who broke its rules. “We are beginning a process of really coming down on reviewers and applicants who do anything to break confidentiality of review,” Richard Nakamura, director of NIH’s Center for Scientific Review (CSR), said at a meeting of the center’s advisory council earlier this week. (CSR manages most of NIH’s peer reviews.) Targets could include “applicants who try to influence reviewers … [or] try to get favors from reviewers.”

“We hope that in the next few months we will have several cases” of violations that can be shared publicly, Nakamura told ScienceInsider. He said these cases are “rare, but it is very important that we make it even more rare.”

Read the rest of this discussion piece

Why all randomised controlled trials produce biased results (Papers: Alexander Krauss | March 2018)0

Posted by Admin in on June 7, 2018
 

Abstract

Background: Randomised controlled trials (RCTs) are commonly viewed as the best research method to inform public health and social policy. Usually they are thought of as providing the most rigorous evidence of a treatment’s effectiveness without strong assumptions, biases and limitations.

Objective: This is the first study to examine that hypothesis by assessing the 10 most cited RCT studies worldwide.

Data sources: These 10 RCT studies with the highest number of citations in any journal (up to June 2016) were identified by searching Scopus (the largest database of peer-reviewed journals).

Results: This study shows that these world-leading RCTs that have influenced policy produce biased results by illustrating that participants’ background traits that affect outcomes are often poorly distributed between trial groups, that the trials often neglect alternative factors contributing to their main reported outcome and, among many other issues, that the trials are often only partially blinded or unblinded. The study here also identifies a number of novel and important assumptions, biases and limitations not yet thoroughly discussed in existing studies that arise when designing, implementing and analysing trials.

Conclusions: Researchers and policymakers need to become better aware of the broader set of assumptions, biases and limitations in trials. Journals need to also begin requiring researchers to outline them in their studies. We need to furthermore better use RCTs together with other research methods.

Key messages

  • RCTs face a range of strong assumptions, biases and limitations that have not yet all been thoroughly discussed in the literature.
  • This study assesses the 10 most cited RCTs worldwide and shows that trials inevitably produce bias.
  • Trials involve complex processes – from randomising, blinding and controlling, to implementing treatments, monitoring participants etc. – that require many decisions and steps at different levels that bring their own assumptions and degree of bias to results.

Keywords: Randomised controlled trial, RCT, reproducibility crisis, replication crisis, bias, statistical bias, evidence-based medicine, evidence-based practice, reproducibility of results, clinical medicine, research design

Krauss, A. (2018) Why all randomised controlled trials produce biased results. Annals of Medicine. 50:4, 312-322, DOI: 10.1080/07853890.2018.1453233
Publisher (Open Access): https://www.tandfonline.com/doi/full/10.1080/07853890.2018.1453233

0