ACN - 101321555 Australasian Human Research Ethics Consultancy Services Pty Ltd (AHRECS)

Resource Library

Research Ethics MonthlyAbout Us

Medical research

Australasian Human Research Ethics Consultancy Services Pty Ltd (AHRECS)

The inclusion of retracted trials in systematic reviews: implications for patients’ safety1


After a paper has been through peer review and has been published it is the obligation of the scientific community to scrutinise an author’s work. If a serious error or misconduct is spotted the paper should be retracted and the work is removed from the evidence base. Over the past ten years there has been an exponential growth in the number of retracted papers. Much of the increase may be explained by the use of technology that has made it easier to spot duplicate publications, or fabricated data, for example. Once a paper is retracted researchers should not cite this work in future publications; this is, however, not the case. Many papers continue to be cited long after they have been retracted. Retraction Watch has a list of the ten most highly cited retracted papers. The paper that currently holds the number one spot has been cited a total of 942 times, after retraction. It is plausible that researchers are using retracted work to justify further study. This may be the scientific equivalent of “fruit of the poisonous tree”. That is to say, if the research is based on tainted work then that work is itself tainted. Authors may also include retracted work in systematic reviews and meta-analyses. In clinical disciplines – such as nursing or medicine – this is particularly worrisome.

Clinical practice should be based on the best available evidence, i.e. from systematic reviews. If a review were to include a retracted paper then the resulting meta-analysis would be contaminated and recommendations for practice emerging from the study would be unsound; ipso facto putting patients at risk because a clinician is using evidence that is flawed. To date we have found five examples in the nursing literature where this has happened. We have written to the journal editors to advise then of the error that authors have made. In our minds this is a cut and dry issue. The author has clearly made an error, potentially a serious error and one that will need to be resolved. Either the editor will need to issue an erratum or potentially retract the review (and there are examples in the literature where this has happened).

There is a second way in which a systematic review may include research that is retracted. This is when the authors of the review cite a paper that is retracted after the review is published. A more nuanced debate is perhaps required given that the review author has not made a mistake. Would it not be punitive to the author – potentially damaging their career prospects – to retract a review when they have not made a mistake? However, the inclusion of a paper that has subsequently been retracted has the potential to impact effect sizes in meta-analysis and/or review conclusions. My group undertook a study to explore how often retracted clinical trials were included in systematic reviews. The answer; more common than you might think. We followed up the citations of eleven retracted nursing trials and determined that they were included in 23 systematic reviews. Currently there is no mechanism that will alert authors (or publishing editors) that their systematic review includes a study that has subsequently been retracted. We suspect, but don’t know for certain, that in medicine and the allied health professions there are many more systematic reviews that include retracted studies. Clinical practice guidelines, such as those produced by the National Institute of Health and Care Excellence (NICE) rely on evidence from systematic reviews. And this is where our observation flips from being an interesting intellectual exercise to one that may impact patient safety. Could it be that patients are being exposed to ineffective treatments because guidelines are based on flawed reviews?

Journal editors, reviewers and researchers need to be aware and mindful that systematic reviews may contain citations that have been retracted. There is a compelling argument that the editor who issues a retraction notice for a paper also has a duty to alert authors citing this work of the retraction decision. Part of the peer review process should be checking that included references (particularly those included in meta-analysis) are not retracted, it might also be argued. Finally, not only do review authors need to ensure that they have not cited retracted papers, but they also have a responsibility to periodically check (something the Cochrane collaboration encourage authors to do) the status of included studies.

The inclusion of retracted trials is a threat to the integrity of systematic reviews. Consideration needs to be given to how the scientific community responds to the issue with the ultimate goal of keeping patients safe.

Professor Richard Gray is the editor of the Journal of Psychiatric and Mental Health Nursing. No other conflict of interest declared.

Richard Gray PhD
Professor of Clinical Nursing Practice, La Trobe University, Melbourne, Australia
Richard’s University profile |

This post may be cited as:
Gray R. (26 May 2018) The inclusion of retracted trials in systematic reviews: implications for patients’ safety. Research Ethics Monthly. Retrieved from:

Research Ethics in the Philippines: a personal journey0


My recall of the earliest encounter I had with research ethics is when, as a newly appointed faculty member of the department of obstetrics and gynecology of the College of Medicine (CM) of the University of the Philippines (UP) and concurrent attending at the Philippine General Hospital (PGH), I rushed to the office of the ethics research committee (known as the Research Implementation and Development Office or RIDO) of CM before the end of office hours one Friday. In my hand was a letter, addressed to the then chairman requesting approval of a study I was about to conduct. Attached to the letter was a one page synopsis of the research protocol. I was fortunate enough to catch him on his way out of the office, and doubly lucky he agreed to quickly browse through the papers I pushed in front of him. He then instructed the office secretary to stamp the letter “APPROVED”, and proceeded to affix his signature. It was in the early 1990’s!

His stamp of approval went a long way towards legitimizing the outcome of my research. I was able to collect and isolate N. gonorrhoea from commercial sex workers in Manila and Cebu, freeze dried all 92, and transport them hand-carried to the laboratory of at the University of Washington. It turned out, almost all the isolates were resistant to the standard first line drug (ciprofloxacin) at that time. Interestingly, a few months before, my collaborator from the US walked into our office looking for someone to work with. Apparently, a US male citizen had been diagnosed to have ciprofloxacin resistant gonorrhea infection. He admitted to having paid sex in Manila and Cebu prior to flying back home. Fate would have it that I was in the office when my collaborator walked in. And since my sub-specialty in obstetrics and gynecology is in infectious disease, the rest was history. I am including this in my narrative because ordinarily, researches with no international collaboration and/or funding would not warrant a mandatory ethics research committee approval. If the process I went through at that time could be construed as a legitimate one today!

Soon after I finished the gonorrhea study, I found myself being appointed by our department chairman to be the representative to the same ethics research committee (RIDO)! By then, in the early years of 1990, all basic science departments of CM and all clinical departments of PGH appointed representatives to RIDO. Meetings were conducted almost monthly to discuss and evaluate research protocols of faculty members who cared to submit their protocols. In those early times, these usually were those with external funding such as clinical trials, or those with international collaborations. I seem to remember the chair of RIDO would present a brief summary of the protocol at hand for the consideration of the members present. If there were no major objections, the research protocol gets approved, and the study will proceed. There didn’t exist written guidelines and standard operating procedures for RIDO. That was in the later years of 1990 and early years of 2000.

When the chair of RIDO retired from the College of Medicine, she recommended me as her replacement. By then, the beginnings of guidelines and standard operating procedures have been put in place. The developments in the interest and commitment to research ethics were being fueled not only within the walls of the academe (CM, PGH and UP), but also in the scientific community. The creation of the National Institutes of Health in UP Manila, whose mandate is to spearhead research at par with the international community, played a big role in upgrading the standards of research, and along with it, compliance to international standards of conducting ethical review of research involving human participants. A Fogarty International grant to UP Manila, whose prime mover was Professor Leonardo D. de Castro, PhD of the College of Social Science and Philosophy of UP Diliman, made it possible to create training programs which empowered the academe in bioethics. In fact in the early years of 2000, a Diploma course in Bioethics was approved and offered through the collaborative efforts of UP Manila and Diliman campuses. Unwittingly, for what I consider to be totally less noble reasons, I took the Diploma course. My main reason was not to help promote research ethics specifically. It was really more for my professional development. At that time, I was already a tenured faculty member. But the trend in the academe was for younger members, even though not yet tenured, to proceed to obtain masteral and even doctoral degrees. My thought at that time was I didn’t want to be upended by younger colleagues. So I enrolled and finished the Diploma course in Bioethics. A year thereafter, the full Masteral course was approved and offered. I proceeded to re-enroll for the same main reason and motivation. It took me several years and 3 extensions of the maximum residency rules of the University before I was able to finish and defend my thesis, and get my Masteral degree in Bioethics!

The prime movers of the bioethics program UP Manila were from the College of Medicine headed jointly by Dra Marita Reyes and Dra Cecilia Tomas. Equal collaborators of the program were Professor Leonardo D. de Castro of the Department of Philosophy, College of Social Science and Philosophy in UP Diliman, among others. The multi-disciplinary collaboration made it possible for many others to establish the Social Medicine Unit (SMU) of the College of Medicine to administer to the MS Bioethics program. It also paved the way for the establishment of a coordinated and integrated system of research ethics review in UP Manila, called the UP Manila Research Ethics Board (UPMREB).

The UPMREB created several panels, each one practically a research ethics committee, with jurisdiction over various sectors of UP Manila: faculty of the College of Medicine (who conduct most of the basic and clinical trials); resident and fellow doctors of Philippine General Hospital; and faculty and students of the various other colleges. Using the same guidelines and standard operating procedures, all the panels of the UPMREB are able to review, approve and monitor all researches in UP Manila. It was also around this time, after my few years as chair of RIDO, that intense preparations were made for the accreditation of RIDO by the Forum for Ethics Research Committees in the Asia Pacific (FERCAP). With the efforts of Dra Evangeline Santos, professor of Ophthalmology and co-graduate of mine from the Diploma in Bioethics program, assisted by other staff of the College of Medicine, FERCAP accreditation was achieved. Subsequently, UPMREB and all its panels achieved the same accreditation.

In the meantime, a law (Republic Act No. 10532), called the Philippine National Health Research System (PNHRS) was enacted in May 2013 to coordinate and integrate all stakeholders in health research in the Philippines. It is through the force of this law that the scientific community outside the University through the Department of Science and Technology (DOST) of the Philippine government, in collaboration with the Department of Education and Culture through the Commission on Higher Education (CHED) and the Department of Health, asked the NIH of UP Manila to implement a Memorandum Order which mandates that all research involving human participants shall undergo review by accredited ethics research committees (by December 2015). The DOST, through the Philippine Council for Health Research and Development (PCHRD), designated the Philippine Health Research Ethics Board (PHREB) as the policy-making body with regards to the establishment, registration, accreditation and regulation of research ethics committees in the country. Henceforth, all academic institutions, all hospitals and health care facilities, and all entities doing health and health-related research involving human participants should submit their studies to PHREB-accredited research ethics committees for review, approval and monitoring. The main objectives are to assure that research participants are not harmed (but benefitted), and that research outcomes are credible.

The PHREB, under the chairmanship of Professor Leonardo D. De Castro, created two important committees: 1). Committee on Information Dissemination, Training and Advocacy (CIDTA); and 2). Committee on Standards and Accreditation (CSA). CIDTA was initially chaired by Dra Rosario Tan-Alora, professor of internal medicine, bioethics and former dean of the college of medicine in the University of Santo Tomas. I had the privilege of being a member of her committee, which conducted trainings for nearly all hospitals and academic institutions in the country. Trainings programs were on Basic Research Ethics, on Good Research Practice, and on Standard Operating Procedures. The objective was to enable participants to create and work in research ethics committees of their respective institutions, be they hospitals or academes. Very recently, Dra Alora decided to turn over the chairmanship of CIDTA to me, although she continues to be an invaluable member/mentor. And more recently, CIDTA is preparing to embark on including a Good Clinical Practice module among its training programs.

The other committee (CSA), initially chaired by Dra Cecilia Tomas, has been in charge of setting standards for research ethics committees all over the country, registering them, and assessing them for accreditation. Three levels of accreditation have been established by CSA: Level 1 are research ethics committees capable of reviewing all types of protocols, except clinical trials; Level 2 are research ethics committees capable of reviewing even clinical trials but not those for products intended for registration with the Philippine Food and Drug Administration (FDA). Both committees had been busy the past 2 years. From a few accredited research ethics committees a couple of years ago, there are now 72 all over the country, many of them Level 3! (See for a complete listing)

The Philippine Council for Health Research and Development (PCHRD), recognizing the existence of research projects whose proponents may not be affiliated with institutions with accredited research ethics committees, and in fact providing funds for some such projects, reactivated the National Ethics Committee (NEC). Chaired by Dra Marita Reyes, the NEC is essentially a research ethics committee composed of multi-sectoral recruited volunteers, myself recently included representing the medical field, which reviews research proposals referred to it by the Department of Health and PCHRD. In 2011, the PHREB published the National Ethical Guidelines for Health Research, providing written, country-specific guidelines on the ethical conduct of researches on various fields. (See Currently, a technical working group headed by Dra Marita V.T. Reyes, with me as one of the members, is in the final stages of updating the guidelines for 2017!

My personal journey in the world of research ethics continues in my newly-assigned tasks of handling classes in the MS Bioethics graduate program, specifically handling Research Ethics and Research Ethics Review classes. From the one-man, practically ambush approval of my very first international research collaboration, to the current legislated and well-established research ethics system, I have been a privileged witness, albeit by twists of fate more than intent design on my part in many instances, to the evolving developmental history of research ethics in the Philippines. This narration is by no means the complete accurate picture. It is a humble and modest attempt to share a part of my career in the academe as a professional doctor taking care of patients, teaching younger colleagues, doing research on the side, and performing administrative functions.

I am grateful to Dr Gary Allen for the opportunity.

Submitted 16 April 2017.
Revised 24 April 2017 after obtaining permission (and more accurate inputs) from the persons whose names were included in the article.

Ricardo Manalastas, Jr., MD, MSc (Bioethics) is a professor of Obstetrics & Gynecology, Infectious Diseases and Bioethics at the College of Medicine, University of the Philippines, Manila, and Attending obstetrician gynecologist at the Philippine General Hospital.
He can be reached by email at  |

This post may be cited as:
Manalastas R. (2017, 24 April) Research Ethics in the Philippines: a personal journey. Research Ethics Monthly. Retrieved from:

Applying Place to Research Ethics and Cultural Competence Training0


In the 1990s, I worked with many community groups and Native American/African-American communities on the difficult challenges of understanding environmental health risks from low-level radiation contamination. These place-based communities and cultural groups were downwind from nuclear weapons production facilities which had massive deliberate and accidental releases of radiation since their operations began during and after World War II. In the health organizing work I had conducted, I was not aware of the potential of research ethics guidelines to bring more beneficence and protection to these populations and their geographic communities. Soon after formal ethical investigations produced findings of cultural ignorance and a lack of knowledge of research ethics by many researchers involved with human radiation experiments, I decided to pursue doctoral studies to promote ethical protections for place-based communities. After receiving my PhD and doing some extensive studies of bioethical principles and their potential to be applied to groups/communities and place, I have been able to publish new studies/practices in this area. With much support both from National Institute for Health and the National Science Foundation and their grant programs on research ethics training, I have worked with several collaborators to promote research ethics training for graduate students in environmental health/sciences, natural resource sciences and engineering (Quigley et al 2015, see NEEP website

In this blog, I provide a discussion of human subjects protections being extended to the protection of the spatial setting, the place-based identities and meanings of individual and group human subjects in their local communities. In a recent paper (Quigley 2016), I argued for this protection both from recommendations that already exist in bioethical guidelines (National Bioethics Advisory Committee (NBAC) and Council for the International Organization of Medical Sciences (CIOMS) and from field studies that demonstrate important lessons for protection of place and place-based identities. The bioethical principles of beneficence, nonmaleficence, respect for persons/respect for communities and justice are reviewed in this article with detailed guidance about each principle as it relates to protecting place and place-based identities.

  • Regulatory guidance exists in terms of the need for researchers to provide benefits to researched populations, to reduce exploitation particularly to racial/cultural and resource-poor groups who are vulnerable subjects, and to allow community consultation on the risks and benefits of research designs. Many resource-poor and politically powerless communities are directly dependent on the subsistence resources of their local spatial settings. Research interventions should not harm these conditions but instead produce beneficial change. Reasonably available benefits should be determined with local representatives (health care providers, community representatives, advocacy groups, scientists and government officials). Such consultation will help to reduce harms, particularly relevant to indigenous groups when the social risks of research can cause disrespect of cultural beliefs, traditions, world views, the violation of local protocols, social stigmatization, and discriminatory harms. For example, in studies of landscape planning, academic researchers co-collaborated with Native community leaders to adopt community-based designs on walking/bike paths, community gardens, mixed use and conservation with housing needs (Thering 2011). Dangles et al (2010) worked with community consultation to ensure that environmental monitoring for control of pests in Andean potato farming and for climate and soil conditions was conducted with community members and particularly with the youth who received training on monitoring technologies which helped to improve youth training opportunities and reduce youth migration. With community collaboration, local community-based benefits can be identified and integrated into technical research plans to improve beneficence.

, I have described how research interventions with cultural groups do require a deep study and practice of an “environmental” cultural competence by researchers, particularly for place-based identities, meanings and past conditions (Quigley 2016b).

There are abundant field studies on new participatory approaches to field research with local communities (see Bibliographies on NEEP website), many of which incorporate collaborative learning about place-based meanings which then lead to research designs which produce local benefits along with technical research activities (capacity-building, skills development, youth outreach, access to critical services, local knowledge guidance about local conditions/resources) These community-based and culturally-competent interventions help to promote the “justice” principle, achieving fair representation, recruitment and fair benefits/burdens for these place-based settings. IRBs are learning more about social risks and community-based protections to ensure more fair treatment, fair benefits and to reduce unintended harms to researched communities.


Quigley, D. D. Sonnenfeld, P. Brown, L. Silka, Q. Tian. L. He. Research Ethics Training on Place-based Communities and Cultural Groups. Journal of Environmental Studies and Sciences, DOI 10.1007/s13412-015-0236-x , published online, March 29, 2015.

Quigley, D. (2016a) Applying Place to Research Ethics and Cultural Competence/Humility Training. Journal of Academic Ethics, published online 13 January, Springer

Quigley, D. (2016b) “Building Cultural Competence in Environmental Studies and Natural Resource Sciences”. Society and Natural Resources, 29:6, 725-737.

Dianne Quigley, PhD is an Adjunct Assistant Professor at Brown University’s Science and Technology Studies Program and can be contacted at

This post may be cited as:
Quigley D. (2016, 22 August) Applying Place to Research Ethics and Cultural Competence Training.Research Ethics Monthly. Retrieved from:

When is research not research?0


Most institutions have processes for differentiating between Quality Assurance/Quality Improvement (QA/QI) activities and those that can be considered to be research. Unfortunately, much of the debate about which is which has been driven by regulatory needs, as a categorization of QA/QI leads to a project not requiring ethics committee review, a preference for many where the low risk pathway is still considered burdensome. Avoidance of ethics review for bureaucratic reasons though is a less than satisfactory motive.

In large scale genomics projects a vast amount of the work being done is in the enabling technologies, that is, the sequencing itself as well as the computational methodologies that are at the heart of the bioinformatics that makes sense of the vast quantities of raw data generated. To develop robust and reliable informatics approaches one can run simulations but ultimately they must be done on real data to ensure they are fit for purpose. The question arises then, is using the data generated from a person’s cancer as well as their normal DNA sequence for the purposes of establishing valid computational tools research? On this topic Joly et al (EJHG 2016) provide a perspective with regard to the International Cancer Genome Consortium (ICGC), which has sequenced more than 10000 patient’s cancers from across 17 jurisdictions. The authors of the paper, of which I am one, are members of the ICGC Ethics and Policy Committee (EPC), which provides advice to ICGC member jurisdictions on matters of ethics relating to the program.

Using two activities, both of which are effectively a means to benchmark how variants and mutations are identified in the genome, we explored how a variety of international jurisdictions viewed the activity and whether they were helpful in defining whether it was a QA/QI activity or one that was more properly regarded as research. Both were identified as having potential risks to confidentiality and both wished to publish their findings. For these reasons they ended up being called ‘research’ and underwent appropriate review. However, recognizing that this may create hurdles for such work that are disproportionate to the true risk of the activity, we reviewed jurisdictional approaches to this topic as well as the literature to see if a more helpful framework could be established to guide appropriate review.

The exercise proved particularly useful as it shone a critical light on some of the more widely used criteria, such as generalizability, which whilst being used by many organizations and jurisdictions as a key distinction between research and QA/QI is in fact a flawed criterion if not used carefully. In contrast, risk to a participant stands up as an important factor that must be evaluated in all activities. Four other criteria (novelty of comparison, speed of implementation, methodology, and scope of involvement), were also reviewed for their utility in developing a useful algorithm for triaging an appropriate review pathway.

The paper proposes that a two step process be implemented in which the six identified criteria are first used to determine whether a project is more QA/QI, research or has elements of both, followed by a risk-based assessment process to determine which review pathways is used. Expedited review, or exemption from review, are options for very low risk projects but, as the paper highlighted from a review of the pathways in four ICGC member countries (UK, USA, Canada and Australia), there is no consensus on how to apply this. The challenge therefore remains establishing more uniformity between jurisdictions on the policies that apply to risk-based evaluation of research. Nevertheless, simple categorization into QA/QI vs Research is not particularly useful and a greater emphasis on evaluation based on criteria that define risk of harm to participants is the way forward.

Further reading

Joly Y, So D, Osien G, Crimi L, Bobrow M, Chalmers D, Wallace S E, Zeps N and Knoppers B (2016) A decision tool to guide the ethics review of a challenging breed of emerging genomic projects. European Journal of Human Genetics advance. Online publication. doi:10.1038/ejhg.2015.279

NHMRC (2014) Ethical Considerations in Quality Assurance and Evaluation Activities

Dr. Nik Zeps
Dr. Zeps is Director of Research at St John of God Subiaco, Murdoch and Midland Hospitals. He was a member of the Australian Health Ethics Committee from 2006-2012 and the Research Committee of the National Health and Medical Research Council (NHMRC) from 2009-2015. He is a board member of the Australian Clinical Trials Alliance and co-chair of the international Cancer Genome Consortium communication committee. His objective as Director of Research is to integrate clinical research and teaching into routine healthcare delivery to improve the lives of patients and their families.

This post may be cited as: Zeps N. (2016, 30 June) When is research not research?. Research Ethics Monthly. Retrieved from:

Page 1 of 212