ACN - 101321555 Australasian Human Research Ethics Consultancy Services Pty Ltd (AHRECS)

Resource Library

Research Ethics MonthlyAbout Us

Outputs

Australasian Human Research Ethics Consultancy Services Pty Ltd (AHRECS)

Stop centring Western academic ethics: deidentification in social science research – Anna Denejkina0

 

This blog will provide a discussion of issues present in deidentifying marginalised research participants, or research participants who request to be identified, in the publication of qualitative research. As my research is mixed-method (quantitative and multi-method qualitative) it included several data collection techniques and analyses. For this discussion, I will specifically focus on the face-to-face and Skype interviews I conducted with participants in Russia and the United States.

My PhD study investigates intergenerational transmission of combat-related trauma from parent to child, focusing on the Soviet–Afghan war, 1979–89. This research includes interviews with Soviet veterans and family members of veterans; it was these interviews that raised questions of participant erasure and agency. From 12 face-to-face and Skype interview participants, one participant requested complete deidentification; one requested that their real name not be used but their location and other identifying details remain; two participants requested that only their first names be used and their location and other identifying details remain; the eight remaining participants requested that they be fully identified, with some participants sending me photographs of them and their families for inclusion in research publications. Given the social and political sensitivity that persists in Eastern Europe around the discussion of the Soviet invasion into Afghanistan, I had to consider and discuss with participants that requested they be identified the issue of their safety.

My research participants are marginalized participants by virtue of the topic of my research, the Soviet–Afghan war, and the ongoing silencing treatment they’ve received during and following the war by the state:

To take just two examples: in the hope of obscuring the true impact of the war, some local authorities refused to allow special areas in cemeteries to be set apart for the graves of soldiers killed in Afghanistan; while others forbade the cause and place of death to be stated on gravestones or memorial shields. (Aleksievich, Whitby & Whitby 1992, p.5–6)

Given academic broad-stroke standards of deidentifying research participants, we must review the ethics of this practice as it can promote and perpetuate erasure of marginalised participants and the silencing of their voices. Some textbooks on the topic of ethics in the social sciences approach anonymity and deidentification of participants from the angle that anonymity is part of the basic expectations of a research participant, without elaborating that anonymity is not always desirable nor ethical (see for example Ransome 2013), essentially replicating the medical model of human research ethics developed for the regulation of biomedical research in the United States (Dingwall 2016, p.25). Such an approach does not address the issues of presenting anonymity as a status-quo in social research, and makes a sweeping – and a Western academic – generalisation that anonymity is one of the vital assurances researchers must give to their participants to keep within their duty of care (that is, that researchers have at least some obligation to care for their research participants).

This approach to research ethics negates participant agency, particularly those participants that request they be identified in research. Furthermore, forced anonymity can be an act of disrespecting participants (Mattingly 2005, p.455–456) who may have already experienced invisibility and who are then further erased through anonymity by researchers (Scarth & Schafer 2016, p.86); for example, “in some Australian and, in particular, some Indigenous cultures, failing to name sources is both a mark of disrespect and a sign of poor research practice” (Israel, Allen & Thomson 2016, p.296).

As researchers, we must also question if presenting this approach as a vital tenet of social research can become a damaging rule-of-thumb for new researchers who might, therefore, not question the potential undermining of participant agency, and use deidentification unethically as a sweeping regulation within their research without consideration for the individual situations of their research participants. This is part of the issue created by applying a medical model of ethics assessment processes to the social sciences, in which the prevailing interpretation is that deidentification is also required within social research, whereas the reality is that specific agreements between the researcher and the research participant must be honoured.

The ethical dilemma, therefore is: can researchers ethically deidentify participants at the expense of the participants’ agency, potentially perpetuating the historical and symbolic erasure of their voices and experiences? I argue that, based on research design and data collection methods, this decision-making process is an ‘ethics in practice’ and must be approached in context, individually for each study, and for each individual participant.
As scholars, we want to minimise or eradicate harm that might come to our participants through our research. While we think “in advance about how to protect those who are brought into the study” (Tolich 2016, p.30) this must be a continual process throughout our project, in which we “work out the meaning of what constitutes ethical research and human rights in a particular context” (Breckenridge, James & Jops 2016, p.169; also see Ntseane 2009). This is important to note, because protection does not only refer to participants but also to others connected to them. For example, the use of a real name at the request of a participant may expose their family member(s) who were not part of the research.

Consequentialist approaches to ethics suggest that “an action can be considered morally right or obligatory if it will produce the greater possible balance of good over evil” (Israel, 2015: 10; also see Reynolds, 1979). This is an approach we could take to issues around deidentification; however, this also means that researchers must know what is good or bad. In studies like mine, this would mean knowing (or making an attempt, or an assumption to know) what is good or bad for my research participants. This action is infantilising, and places the researcher above the research participant by making the final call ourselves, which is to remove participant agency – if we can assume participants are autonomous during the research consent process, we must also assume that they are autonomous in making decisions with respect to their identification (Said 2016, p.212). Additionally, this action may be culturally insensitive given that Western human research ethics committees follow Western cultural guidelines, centring the dominance of Western academia.

The ethical issues I faced during my PhD research highlight why researchers cannot take a sweeping approach to deidentification in qualitative research – not even for a single study. ‘Ethics in practice’ means that each participant’s situation is analysed individually, and issues around erasure, safety, and their agency weighed against each other to reach a conclusion. I propose that if this conclusion is at odds with the preference of the participant, that it must then be taken back to the participant for further discussion. Not implementing this aspect of ‘ethics in practice’ goes against social science ethics, that we must avoid doing long-term and systemic harm, both of which come through erasure and silencing. We must also remember that “any research project has the potential to further disenfranchise vulnerable groups” (Breckenridge, James & Jops 2016, p.169), and ignoring the wishes of participants regarding their identification due to a Western model of ethics can cause further damage to these groups.

References:
Aleksievich, S., Whitby, J. & Whitby, R. 1992, Zinky Boys: Soviet voices from a forgotten war, Chatto & Windus, London.

Breckenridge, J., James, K. & Jops, P. 2016, ‘Rights, relationship and reciprocity: Ethical research practice with refugee women from Burma and New Delhi, India’, in K. Nakray, M. Alston & K. Whittenbury (eds), Social Sciences Research Ethics for a Globalizing World: Interdisciplinary and Cross-Cultural Perspectives, Routledge, New York, pp. 167–186.

Dingwall, R. 2016, ‘The social costs of ethics regulation’, in W.C. van den Hoonaard & A. Hamilton (eds),The Ethics Rupture, University of Toronto Press, Toronto, pp. 25–42.

Israel, M., Allen, G. & Thomson, C. 2016, ‘Australian research ethics governance: Plotting the demise of the adversarial culture’, in W.C. van der Hoonaard & A. Hamilton (eds),The Ethics Rupture, University of Toronto Press, Toronto, pp. 285–216.

Mattingly, C. 2005, ‘Toward a vulnerable ethics of research practice’, Health: An Inderdisciplinary Journal for the Social Study of Health, Illness and Medicine, vol. 9, no. 4, pp. 453–471.

Ntseane, P.G. 2009, ‘The ethics of the researcher-subject relationship: Experiences from the field’, in D.M. Mertens & P.E. Ginsberg (eds), The Handbook of Social Research Ethics, 1st edn, Sage, Thousand Oaks, pp. 295–307.
Ransome, P. 2013, ‘Social research and professional codes of ethics’, Ethics and Values in Social Research, Palgrave Macmillan, Basingstoke, pp. 24–53.

Said, D.G. 2016, ‘Transforming the lens of vulnerability: Human agency as an ethical consideration in research with refugees’, in K. Nakray, M. Alston & K. Whittenbury (eds),Social Sciences Research Ethics for a Globalizing World: Interdisciplinary and Cross-Cultural Perspectives, Routledge, New York, pp. 208–222.

Scarth, B. & Schafer, C. 2016, ‘Resilient Vulnerabilities: Bereaved Persons Discuss Their Experience of Participating in Thanatology Research’, in M. Tolich (ed.), Qualitative Ethics in Practice, Left Coast Press, Walnut Creek, CA, pp. 85–98.

‘Tolich, M. 2016, ‘Contemporary Ethical Dilemmas in Qualitative Research’, in M. Tolich (ed.), Qualitative Ethics in Practice, Left Coast Press, Walnut Creek, CA, pp. 25–32.

Statement of interest
No interests to declare.

Contributor
Anna Denejkina | Casual Academic and PhD  candidate in the Faculty of Arts and Social Sciences, researching intergenerational trauma transmission UTS | Staff profileAnna.Denejkina@uts.edu.au

This post may be cited as:
Denejkina A. (24 May 2018) Stop centring Western academic ethics: deidentification in social science research. Research Ethics Monthly. Retrieved from: https://ahrecs.com/human-research-ethics/stop-centring-western-academic-ethics-deidentification-in-social-science-research-anna-denejkina

A Model for the Participation of Indigenous Children and Young People in Research0

 

Following my September 2017 piece: Ethics and the Participation of Indigenous Children and Young People in Research, this article briefly overviews the research model I developed in my PhD. The model is based on a children’s rights-based approach (CRBA) to research informed by Indigenous research methodologies. It combines Laura Lundy’s[1] analysis of Article 12 of the Convention on the Rights of the Child (CRC) with aspects of Indigenous research methodologies articulated by Ray (Indigenous convergence methodology)[2] and Nakata (Indigenous standpoint theory).[3] The field research methods sought to engage with children and young people in a culturally appropriate and child friendly way by using Bessarab and Ng’andu’s[4] ‘yarning’ approach, as well as a range of other child friendly and play based methods such as drawing, modelling with playdough, as well as photography and peer-to-peer video interviewing using iPads[5].

Lundy’s diagram below highlights the interpretation of Article 12 of the CRC adopted in the research. This interpretation emphasises that Article 12 requires governments to ensure children and young people not only have the opportunity to voice their views about matters affecting them, but that their views are taken into consideration and influence the decisions that are made.

Lundy’s Conceptualisation of Article 12[6]

The child rights-based model used prioritised child-centred play in the research process and engaged with children, rather than doing research on or about children[7]. Some of these interactions are depicted below in the photographs.

10-Year-Old Child Modelling Something that is Important to Him—‘My Family’[8]

10-Year-Old Child Modelling Something that is Important to Him—‘I Like Toys, and Robots … and Dreamtime and Culture Dance’[9]

Experimental Photography, Testing the Functionality of the iPads [10]

Making an iPad Video [11]

Taking part in social research can expose Indigenous children and young people to varying degrees of risk however ‘the line between gate-keeping intended for the protection of participants and their communities and the risk of sliding into paternalism is a thin one.’[12] Research that is carried out in an ethically robust, age appropriate and culturally sensitive way can present avenues for Indigenous children and young people to express their views and have these views taken into consideration in accordance with Article 12 of the CRC.

This research suggests Indigenous children and young people are ready, willing and able to voice their perspectives about matters affecting them, if given the opportunity in appropriate circumstances and in an appropriate setting. The findings of this research debunk conceptualisations of Indigenous children and young people as passive and vulnerable. The implications of viewing and defining Indigenous children and young people in this way limits their civic participation and reduces opportunities for their voice to be heard about matters affecting them.

A children’s rights-based approach to research positions children and young people as empowered co-researchers, with expertise and valuable perspectives capable of leading and informing the research process. It is an approach which engages children and young people in research in a collaborative way that fulfils, promotes and protects a range of rights provided for by the CRC, in particular, their rights to participate in decision making processes.

For more information about the research model see Doel-Mackaway, Holly, ‘I think it’s Okay … But it’s Racist, it’s Bad Racism’: Aboriginal Children and Young People’s Views about the Intervention’ (2017) 43(1) Monash University Law Review 76.

In 2018 Routledge is publishing a book about this PhD research.

References

Barker, John and Susie Weller, ‘“Is It Fun?” Developing Children Centred Research Methods’ (2003) 23(1/2) International Journal of Sociology and Social Policy 33.

Bat, Melodie et al, ‘Ethical Moves: Innovation in Qualitative Research: An Example of an Ethical and Effective Cross-Cultural Research Methodology Using Video’ (Paper presented at the AARE Annual Conference, Canberra, 2009);

Bessarab, Dawn and Bridget Ng’andu, ‘Yarning about Yarning as a Legitimate Method in Indigenous Research’ (2010) 3(1) International Journal of Critical Indigenous Studies 37.

Coram, Stella, ‘Rethinking Indigenous Research Approval: The Perspective of a “Stranger”’ (2011) 11(2) Qualitative Research Journal 38.

Kral, Inge (2010) ‘Plugged In: Remote Australian Indigenous Youth and Digital Culture’ (Working Paper No 69/2010, Centre for Aboriginal Economic Policy Research, May 2010). http://caepr.anu.edu.au/Publications/WP/2010WP69.php

Lundy, Laura, ‘“Voice” Is Not Enough: Conceptualising Article 12 of the United Nations Convention on the Rights of the Child’ (2007) 33 British Educational Research Journal 927.

Nakata, Martin, Disciplining the Savages: Savaging the Disciplines (Aboriginal Studies Press, 2007), chapter 11.

Ray, Lana, ‘Deciphering the “Indigenous” in Indigenous Methodologies’ (2012) 8(1) AlterNative: An International Journal of Indigenous Peoples 85, 88, 88. See also Lester-Irabinna Rigney, ‘Indigenist Research and Aboriginal Australia’ in Julian Kunnie and Nomalungelo Ivy Goduka (eds), Indigenous Peoples’ Wisdom and Power: Affirming Our Knowledge Through Narratives (Ashgate Publishing, 2006) 32.

Contributor
Dr Holly Doel-Mackaway | Lecturer | Macquarie Law School |
Dr Doel-Mackaway’s Macquarie staff page | holly.doel-mackaway@mq.edu.au

This post may be cited as:
Doel-Mackaway H. (2017, 20 October 2017) A Model for the Participation of Indigenous Children and Young People in Research Research Ethics Monthly. Retrieved from: https://ahrecs.com/human-research-ethics/model-participation-indigenous-children-young-people-research

Footnotes

[1]Laura Lundy, ‘“Voice” Is Not Enough: Conceptualising Article 12 of the United Nations Convention on the Rights of the Child’ (2007) 33 British Educational Research Journal 927.
.
[2]Lana Ray, ‘Deciphering the “Indigenous” in Indigenous Methodologies’ (2012) 8(1) AlterNative: An International Journal of Indigenous Peoples 85, 88, 88. See also Lester-Irabinna Rigney, ‘Indigenist Research and Aboriginal Australia’ in Julian Kunnie and Nomalungelo Ivy Goduka (eds), Indigenous Peoples’ Wisdom and Power: Affirming Our Knowledge Through Narratives (Ashgate Publishing, 2006) 32.
.
[3]Martin Nakata, Disciplining the Savages: Savaging the Disciplines (Aboriginal Studies Press, 2007), chapter 11
.
[4]Dawn Bessarab and Bridget Ng’andu, ‘Yarning about Yarning as a Legitimate Method in Indigenous Research’ (2010) 3(1) International Journal of Critical Indigenous Studies 37.
.
[5]Melodie Bat et al, ‘Ethical Moves: Innovation in Qualitative Research: An Example of an Ethical and Effective Cross-Cultural Research Methodology Using Video’ (Paper presented at the AARE Annual Conference, Canberra, 2009); Inge Kral, ‘Plugged In: Remote Australian Indigenous Youth and Digital Culture’ (Working Paper No 69/2010, Centre for Aboriginal Economic Policy Research, May 2010).
.
[6] Laura Lundy, ‘“Voice” is Not Enough: Conceptualising Article 12 of the United Nations Convention on the Rights of the Child’ (2007) 33(6) British Educational Research Journal 927, 932
.
[7]John Barker and Susie Weller, ‘“Is It Fun?” Developing Children Centred Research Methods’ (2003) 23(1/2) International Journal of Sociology and Social Policy 33, 33.
.
[8]10-year-old male, Primary Class Group Discussion, Field Research Session 1 (of 4) (Northern Territory, 13 May 2014).
.
[9] 10-year-old male, Primary Class Group Discussion, Field Research Session 1 (of 4) (Northern Territory, 13 May 2014).
.
[10]14-year-old male, Secondary Class Group Discussion, Field Research Session 3 (of 4) (Northern Territory, 20 May 2014).
.
[11] 10-year-old male, Primary Class Group Discussion, Field Research Session 1 (of 4) (Northern Territory, 13 May 2014
.
[12]Stella Coram, ‘Rethinking Indigenous Research Approval: The Perspective of a “Stranger”’ (2011) 11(2) Qualitative Research Journal 38, 45.

Terms and conditions apply0

 

Kids tell us that making decisions can sometimes be hard (anyone who has taken a child to an ice cream shop can attest to this). Adults don’t often give children choices and kids tell us that when they do it can be confusing: ‘what am I being asked?’, ‘can I really say no?’, ‘do they seriously care?’ and ‘what will happen if I make a decision the adult doesn’t like?’ are questions that might spring to mind. After all, they tell us that it’s not usual for adults to seek out children’s views, to let them make big decisions or to give up some of their ‘adult power’ and act on children’s wishes.

And yet children are required to ‘assent’ to research, often with little information about what research actually is and what they will be required to do. In most cases, they know that their parents have already given permission for them to be involved – which may be reassuring but also a bit daunting (how often is it that kids can say ‘no’ when their parents have already said ‘yes’?) – but in most cases a complete stranger comes into their home or schoolroom and pulls out a note pad and asks them whether their happy to answer a few questions. “Um OK?”

Since the Helsinki Declaration there has been an expectation that children provide assent to their participation in research. Often this entails providing them with a long-winded, legalistic and ‘pretty boring’ information letter, telling them that if they agree they might get a movie voucher or at least a packet of chips and a can of soft drink before asking them to tick a box to show that they agree. As a child in one of my studies reported, the process is ‘kinda like’ the terms and conditions process they go through when downloading a new app from i-Tunes. Like 73% of Terms and Conditions non-readers, kids in research often have no idea what they are signing up for and what their rights are when things go wrong.

In a recent study my colleagues and I conducted for the Australian Royal Commission into Institutional Responses to Child Sexual Abuse we got advice from children and young people about how to best help kids understand and consent to participating in research on a fairly sensitive topic. Based on this advice we conceptualized consent as an ongoing process that included six steps: (1) present information in a child-friendly and accessible way; (2) make sure they understand what research is and what they are expected to do; (3) give them the choice to participate (or not) and ask them to formally agree; (4) give them lots of opportunities (and the skills or tools) to bow out of the research (particularly after they’ve got a ‘feel’ for what they are being asked to do) or to change the way that they are participating; (5) be aware of the ways that kids resist or ‘dissent’ (yawning or sneaking out of the room might give it away) and constantly ‘check in’ with them in child-friendly ways (6) Get an agreement with them at the end of the research activity that they are still happy for their input to be included in the study and negotiate what, if anything, they’re happy for the researcher to share with their parent, teacher or older sibling who is standing behind the door.

In the paper “More a marathon than a hurdle: towards children’s consent in a safety study” my colleagues and I outline how we went through these steps with kids, we describe how we used felt toys, ‘stop signs’ and ‘rights posters’ to help children and young people consent and, most importantly, quote advice and feedback from children and young people on how adult researchers might best help kids to make decisions within the research context.

One tool we feature in the article is our “Charter of Rights” poster which we provide kids in our studies. The poster informs them of what they should expect from us, as researchers, and what to do if they are unhappy. The poster is given to the children prior to them meeting with our staff and is further explained before assent is sought. On the advice of children and young people who have advised our projects, the rights charter has also been used as the basis of a series of games and activities that can be used to help children understand their rights in research (and in welfare practice). More detail about these can be found here. My team at the Institute of Child Protection Studies are working with peers from the Centre from Children and Young People (Southern Cross University), UNSW and the University of Melbourne to progress ethical research with children and young people. We’re currently hosting a survey on ethical decision-making – take a minute to fill it in! We’re keen to chat with others who are grappling with how to meaningfully engage kids in research (and support them to make good decisions) and would love to hear from you. *Terms and Conditions Apply.

Contributor
Dr Tim Moore
Senior Research Fellow | Institute of Child Protection Studies, ACU
Bio page at ACUTim.Moore@acu.edu.au

This post may be cited as:
Moore T. (2017, 21 July) Terms and conditions apply; Research Ethics Monthly. Retrieved from: https://ahrecs.com/human-research-ethics/terms-conditions-apply

PID Power: Persistent Identifiers as Part of a Trusted Information Infrastructure0

 

We live in a world where fake news and alternative facts are, unfortunately, part of how we share information. Expertise is becoming less valued and, in some cases, is even seen as a liability. In this environment, how do we engender trust in scholarly communications?

Developing a strong and sustainable information infrastructure, which enables reliable connections between researchers, their contributions, and their organizations, is critical to building this trust. Many of the pieces we require are already in place, but work is still needed to ensure that they operate the way we need them to, and that all sectors – funders, publishers, and universities, as well as vendors and other third parties – understand the vital role each plays.

Persistent identifiers (PIDs) play an important part in making the research infrastructure work, and doing so in a transparent way, which builds trust.  Wikipedia describes persistent identifiers as: “a long-lasting reference to a document, file, web page, or other object … usually used in the context of digital objects that are accessible over the Internet. Typically, such an identifier is not only persistent but actionable … you can plug it into a web browser and be taken to the identified source.”

In the scholarly communications world, PIDs enable clear identification of and reliable connections between people (researchers), places (their organizations), and things (their research contributions and works). Examples of PIDs in common use in research and scholarship include ORCID iDs, ResearcherID, and Scopus IDs for people; GRID, Ringgold, and Crossref Funder Registry IDs for organizations; and DOIs (Digital Object Identifiers) such as those minted by Crossref and DataCite for publications and datasets.

So, how exactly can PIDs help build trust in the research infrastructure – and the scholarship supported by that infrastructure?

Tackling the problem of fake reviews and reviewers is a good example of the power of persistent identifiers in practice. While the vast majority of reviews and reviewers are legitimate, unfortunately some individuals and organizations deliberately attempt to manipulate the system to their own, or their client’s, advantage. Industry organizations such as COPE – the Committee on Publication Ethics – recognize that this as an issue and it’s also found its way into mainstream media, where it’s often seen as more ‘evidence’ that science isn’t working.

But imagine a world where all research institutions routinely connect their organization ID to their researchers’ ORCID records and, at the same time, assert their affiliation. That institutional validation makes information about those researchers significantly more trustworthy.

And now imagine a world where researchers routinely use their ORCID iD during the manuscript submission/review process. Where publishers routinely include those iDs in the metadata for DOIs for the papers/open peer review reports authored by those researchers. And where that information is automatically pushed back into the author’s ORCID record, for example by Crossref or DataCite. Those trusted connections (assertions)  between each researcher and her/his publications and reviews could help editors and publishers build up an authoritative picture of each researcher, creating an even higher level of confidence that they are who they say they are. Adding in information from funders about the reviewer’s awards would add an even higher level of certainty. Taken together, the use of PIDs in this way could be a powerful tool in combatting the fake author and reviewer problem.

This scenario clearly shows that tackling the issue of trust in scholarly communication requires a community approach.. Each sector plays a role: institutions connect and assert affiliations to ORCID records; publishers connect and assert works; funders connect assert awards; and PID organizations including Crossref, DataCIte, and ORCID provide the “plumbing” that enables those assertions and connections to be made, easily and reliably.

Of course, researchers themselves also need to be involved in improving trust in scholarly communications. Using PIDs is a good (and easy!) first step – the technology is already in place across hundreds of systems that researchers interact with.  So, for example, researchers who use their ORCID iD when publishing or reviewing a paper, can authorize Crossref or DataCite to automatically update their ORCID record every time a DOI for one of their works is minted (provided that their publisher includes the iD in the metadata). Likewise some funders are already collecting ORCID iDs during grant application and then connecting information about awards granted back to the applicant’s ORCID record. And, in an exciting new opportunity, it’s now possible for researchers to sign into ORCID using their institutional credentials and, at the same time, grant their university permission to update their ORCID record, including asserting their affiliation. Vendor systems across all sectors – grant application, manuscript submission, CRIS systems, and more – are supporting all these efforts.

As Simon Porter of Digital Science pointed out in his keynote at PIDapalooza 2016, the challenges of achieving this goal are at least as much social as technical. Understanding why PIDs are important is every bit as critical as implementing them in researcher systems. So, if you’d like to  play your own part in making  our vision of a trustworthy PID-enabled research infrastructure a reality, please help us spread the word about the power of PIDs in your own organizations!

Contributor
Alice Meadows | Director of Community Engagement & Support, ORCID
Alice’s ORCID staff page and Alice’s LinkedIn page
a.meadows@orcid.org

This post may be cited as:
Meadows A. (2017, 27 July) PID Power: Persistent Identifiers as Part of a Trusted Information Infrastructure Research Ethics Monthly. Retrieved from: https://ahrecs.com/research-integrity/pid-power-persistent-identifiers-part-trusted-information-infrastructure

 

Page 1 of 3123