ChatGPT’s ability to churn out mediocre papers should lead us to reappraise how research is carried out, reported and evaluated, says Martyn Hammersley
Much current concern about the implications of large-language AI models, such as ChatGPT, has focused on their use by students in producing essays for assessment. But some attention also needs to be given to the prospect of research articles being produced by, or with the aid of, such technology. And this raises questions about the functions these articles have come to serve in the institutional conditions that prevail today in universities.
For some time, it has been known that research metrics have mutated research into something akin to a sausage factory, where researchers are merely drones that churn out as many publications as possible and as quickly as possible. Volume is celebrated rather than genuine quality. With the maturation of artificial intelligence tools like ChatGPT and other LLMs, we are likely to see a fast-moving avalanche of mediocre papers. This piece that Times Higher Education published muses on whether this will it last forces a shift away from quantity and towards quality. Hope springs eternal.
Somewhat later, Berkeley sociologist Philip Selznick developed the same argument in relation to the Tennessee Valley Authority, established in 1933 to promote economic development in the Tennessee Valley basin. Here, again, the bureaucracy increasingly came to serve its own interests, as well as those of local power-holders.
Over the past few decades, displacement of goals appears to have occurred within universities too. This has taken place on many fronts, but particularly as regards the production of research. The original goal was to contribute to the body of collective knowledge, and much research still does this. But research is increasingly evaluated in terms that bear little effective relationship to that purpose.
Institutionally, the primary function of research papers now appears to be to boost universities’ research profiles (in the UK, for example, in the context of the Research Excellence Framework), as well as to enhance the prospects of individual academics getting jobs or gaining promotion. Furthermore, the value of articles is increasingly measured in terms of citation impact, either that of the articles themselves or of the journals in which they appear. Yet the relationship between these measures and an article’s contribution to knowledge is weak at best, especially outside the natural sciences.
SUBSCRIPTION REQUIRED
