In a notorious scene from Pirates of the Caribbean: The Curse of the Black Pearl, Captain Barbarossa refers to the Pirate’s Code cynically as ‘what you’d call guidelines’ suggesting that conformity is merely a matter of choice:
Elizabeth: Wait! You have to take me to shore. According to the Code of the Order of the Brethren…
Captain Barbarossa: First, your return to shore was not part of our negotiations nor our agreement so I must do nothing. And secondly, you must be a pirate for the Pirate’s Code to apply and you’re not. And thirdly, the Code is more what you’d call ‘guidelines’ than actual rules. Welcome aboard the Black Pearl, Miss Turner.
Recently, some evidence has emerged that the same observation could be made about another set of guidelines, namely, those relating to the ethics review and conduct of human research in Australia: the National Statement on Ethical Conduct in Human Research issued by the National Health and Medical Research Council, the Australian Research Council and Universities Australia in 2007 and modified to the current version of May 2015. These guidelines set out the principles and processes for ethics review by human research ethics committees (HRECs) and conduct of research in which people are participants. The guidelines also set out requirements for the establishment, membership and operation of HRECs and assign obligations to institutions to see that these are followed. Since 2001, the NHMRC has established and maintained a register on which institutions list their HRECs and agree to operate them according to the National Statement.
Annually, these institutions provide to the NHMRC, on request, reports on the conduct of the HRECs they have established. It is these reports, covering 2014, 2015 and 2016 that provide revealing evidence about the extent to which HRECs and institutions in fact conform to the National Statement.
.
ACTIVITY
‘When an institution has established an HREC, the institution is responsible for ensuring that…
- review of research proposal is thorough;
- review processes and procedures are expeditious;
- the workload of an HREC does not compromise the quality or timeliness of the ethical review;’ (National Statement 5.1.28 (c), (d) & (i))
Some reasons for these guidelines are that ensuring an adequate workload maintains review skills and that avoiding an excessive workload weakens and prolongs reviews.
Evidence:
In 2014, of the 216 HRECs that reported, 11 did not review any proposals and 59 met between 1 to 5 times and 41 HRECs considered not more than 10 proposals.
In 2015, of the 212 HRECs that reported, 10 did not review any proposals and 54 met between 1 to 5 times, and 42 HRECs considered not more than 10 new proposals.
In 2016, of the 210 HRECs that reported, 15 did not review any proposals and 50 met not more than 1 to 5 times and 42 HRECs considered not more than 10 new proposals.
The published data support the conclusion that in each of the last three years about 5% of the HRECs did not review any proposals and about 20% did not review more than 10 new proposals. What would be the effect of such light workloads on the review expertise of HREC members?
.
TRAINING
One other means by which committees can maintain their review skills is to undertake regular continuing education or training. The National Statement contains two relevant guidelines that recognise this:
‘where an institution has established an HREC, the institution is responsible for ensuring that… (b) members undertake: (ii) continuing education;’ (5.1.28 (b)(ii)), and
‘…each member of a review body should: …(c) attend continuing education or training programs in research ethics at least every three years.’ (5.2.3 (c)).
Evidence:
In 2016, 161 of 210 HRECs reported that ‘1 or more’ members had attended training during the year.
In 2015, 185 of the 212 HRECs reported that their institutions made opportunities for training available to members, but only 160 reported that 1 or more members had attended relevant training during the year.
In 2014, although 179 of the 217 HRECs indicted that their responsible institution provided opportunities for members to attend training, only 149 of the committees reported that 1 or more members had attended relevant training during the year.
The data published by the NHMRC does not allow us to identify whether the same HRECs failed to take advantage of training possibilities in different years nor to work out what kinds of institutions were more or less likely to ensure professional development of committee members. Nevertheless, in each of these years, between 30 and 40 HRECs undertook no training at all and the data for those that did could mean that not more than one member attended one training opportunity in that year.
While some HRECs may be investing in the professional development of their members, it is difficult to conclude that the requirements are being taken seriously across the sector.
Indeed, it is doubtful that the sector as a whole is engaging with capacity building of HREC members. Since 2008, the NHMRC has not devoted any resources to training of HRECs, in contrast with its counterparts in the United Kingdom. Canada and the United States. The provision of such professional development as is available has fallen to voluntary national gatherings, such as the Australasian Ethics Network or to commercial providers.
.
MEMBERSHIP
There is a minimum membership for HRECs of eight constituted by specified categories, namely,
- a chair,
- at least two community members,
- a member with experience in counselling or treatment of people,
- a person who performs a community pastoral role
- a lawyer
- at least two researchers. (National Statement 5.1.30)
The specification relates to HREC decision-making, as the guidelines provide that HREC decisions ‘must be informed by an exchange of opinions from each of those who constitute the minimum membership…’
Evidence:
In the 2016 year, 16 of the 210 HRECs reported that they did not have the minimum membership during the year.
In the 2015 year, 42 of the 212 HRECs reported during the year that they made decisions on proposals when there was a vacancy in their membership, conduct that the report noted as being ‘contrary to the National Statement’.
In the 2014 year, 34 of the 216 HRECs reported that they had continued meeting when there was a vacancy in their membership, again noted in the report as being ‘contrary to the National Statement’.
Accordingly, there were numerous decisions – it is impossible to calculate how many – made by HRECs during each of the last three years that lacked the range of input that the guidelines require. Further, as the reports in 2014 and 2015 note, such decision-making is contrary to the National Statement and, in turn, a failure by the responsible institution to fulfil its responsibility to ‘see that any human research for which they are responsible is… ethically reviewed and monitor in accordance with this National Statement.’ (National Statement 5.1.1(b)).
.
GENDER BALANCE
One other guideline about membership provides that, ‘as far as possible, there should be equal numbers of men and women’ (National Statement 5.1.29(a)).
Evidence:
In the 2016 report, it was stated that because ‘It is recognised that this may be difficult to attain’, the ‘NHMRC considered instances in which there was at least an 80:20 gender imbalance as significant and requiring attention.’ Only 5 of the 210 HRECs reported such an imbalance. No data were reported of the number of HRECs in which there was a lesser gender imbalance.
In the 2015 report, the same statements appear and only 3 of the 212 HRECs reported an imbalance of 80:20 or more.
In the 2014 report, that statement does not appear and 24 of the 216 HRECs are reported to have ‘less than a 70:30 gender balance in either direction.’
Accordingly, gender imbalances of anything less than 1 in 5 are not regarded as in need of attention, an interpretation of the guideline that provides very little incentive to correct imbalances that, in many other contexts in 2018, would be regarded as unacceptable. Were the sector taking gender seriously in HREC membership, it would be far better to create more meaningful targets and, if necessary, to phase these in over time. It would also be sensible to track where gender imbalance lies within committee membership and compare that to broader patterns within the host institutions. For example, if the small number of female academics on a committee reflected a broader problem in an institution, the latter may need to be addressed first rather than placing an increased burden on a small number of more senior female academics.
| Year | Reviewed applications | Complaints about research | Complaints about review |
| 2007 | 10777 | 138 | 49 |
| 2008 | 21087 | 96 | 19 |
| 2009 | 22306 | 100 | 11 |
| 2010 | 23696 | 121 | 21 |
| 2011 | 25022 | n/a | n/a |
| 2012 | 26257 | 161 | 19 |
| 2013 | 24882 | 145 | 20 |
| 2014 | 20892 | 226 | 58 |
| 2015 | 18768 | 229 | 34 |
| 2016 | 18039 | 237 | 37 |
.
IMPLICATIONS
Does any of this matter? If so, why: do deficiencies in the processes by which HRECs reach their conclusions and decisions contribute to more of those decisions being inappropriate or unacceptable? Trends in complaints data may be some indication, and these data, available from 2007 to the current year (except 2011), tabulated below, do show some increase in both kinds of complaints. However, it is notoriously difficult to generate reliable conclusions from complaints data.
Implications for NHMRC Administering Institutions
Institutions take responsibility for annual HREC reports and many of those institutions will be administering institutions with NHMRC grants, the conditions for which are contained in a standard Funding Agreement. Clause 24 of that agreement provides that ‘in carrying out this Agreement, the Administering Institution must comply… with… the NHMRC Approved Standards and Guidelines.’ (which are defined to include the National Statement). Clause 30.4 of the same agreement requires Administering Institutions to ‘immediately notify the NHMRC in writing if it ceases fully to comply with… the NHMRC Approved Standard and Guidelines. Such failures are among the grounds on which the NHMRC can suspend or terminate research funding’ (Funding Agreement clause 15).
Should reporting of a non-compliant HREC be treated as such a breach? Given that an institution is required to notify the NHMRC when it ceases ‘fully to comply’, any of the deficiencies recorded above from the last three annual reports of HRECs would appear to be sufficient.
.
Some wider implications and questions
These data raise the question: when is an HREC decision sufficiently defective as not to merit respect or recognition? When there is no input from any one of the minimum members? When the gender balance is lower than 1 in 5 in either direction? When the decision is the only one or only one of 5 that the committee has made in a year? When none of the committee members have attended training in the last three years? When the decision is one of 30 made at the same meeting? When there is no reference in the decision to any one of the four key review criteria: research merit, justice, beneficence or respect?
Will tolerance of nonconforming practice lead to declining support for and recognition of HRECs and, in turn, of ethics review itself? If that recognition declines, will the need for ethics review be questioned? Would recognition that Australian ethics review lacks accountability and conformity to national guidelines threaten the reputation of Australian research, researchers and research institutions?
Doing nothing in the face of this evidence condones the nonconforming practices and risks breeding an indifference to ethics review and, in turn, to seeing it as irrelevant and unnecessary.
However, if a response to these data of deficiency is needed, at what level should that response be made: that of the HRECs, institutions or all the sector stakeholders? In short, who should take responsibility for the reliability of ethics review and how should that responsibility be implemented?
Contributors
Colin Thomson – Senior Consultant, AHRECS | AHRECS bio| co***********@****cs.com
This post may be cited as:
Thomson C. (2017, 22 March 2018) “More what you’d call guidelines”. Research Ethics Monthly. Retrieved from: https://ahrecs.com/human-research-ethics/more-what-youd-call-guidelines