Multiple accounts from researchers suggest that feedback for Discovery Project grant funding was written by artificial intelligence
The Australian Research Council has faced allegations that some of its peer reviewers may have used ChatGPT to assess research proposals, prompting a warning from the education minister and concerns about possible academic misconduct.
Have you ever felt a grant reviewer was soulless, didn’t understand the significance of your research question, or did not understand what the research meant? If this story in The Guardian is accurate, when this happens, it might literally be true. The reviewer didn’t understand because it wasn’t genuinely human. As we have observed recently, LLMs like ChatGPT do not genuinely understand the information it is provided or the text it produces. Given the way in which such text is produced (compression plagiarism from existing text) and someone claiming it as their work is a form of misconduct.
One academic, who wished to remain anonymous, told Guardian Australia that one of the assessor reports they received included the words “Regenerate response” – text which appears as a prompt button in the ChatGPT interface.
“It’s quite a positive report, but it’s quite bland also, and it quotes back the proposal at you,” the researcher said. “It’s almost like reading something you’ve written yourself.”
After they submitted a complaint to the ARC, the report was removed.
The researcher said the apparent use of AI pointed to the time pressures faced by academics in Australia and also a possible lack of quality control internally by the ARC.
“I think it’s a sign of someone being overworked and trying to cut corners … If you’ve used artificial intelligence to generate a response, you lose the ability to engage in a proper academic cut and thrust.”
Detailed assessor reports are typically written by academics in closely related fields and are used by the ARC’s College of Experts to decide which grant proposals should ultimately receive government funding. Only 19% of Discovery Projects in last year’s funding round were ultimately successful. The outcomes of the 2023 grant round have not yet been announced.
