As researchers dive into the brave new world of advanced AI chatbots, publishers need to acknowledge their legitimate uses and lay down clear guidelines to avoid abuse.
It has been clear for several years that artificial intelligence (AI) is gaining the ability to generate fluent language, churning out sentences that are increasingly hard to distinguish from text written by people. Last year, Nature reported that some scientists were already using chatbots as research assistants — to help organize their thinking, generate feedback on their work, assist with writing code and summarize research literature (Nature 611, 192–193; 2022).
First of all, let’s set aside the science-fiction nightmares of machines taking over the world and the hyperbole around ChatGPT. The NLP and AI systems that are currently available do not possess general AI (AGI). But they are becoming increasingly hard to spot and they produce their output by harvesting text from around the web, without attribution. It is dishonest for a researcher to use systems such as ChatGPT to produce an output and claim it as their own work. Without careful review, editing and paraphrasing, the resulting text is likely to have serious errors, plagiarism and be impossible for a future researcher to replicate.
ChatGPT can write presentable student essays, summarize research papers, answer questions well enough to pass medical exams and generate helpful computer code. It has produced research abstracts good enough that scientists found it hard to spot that a computer had written them. Worryingly for society, it could also make spam, ransomware and other malicious outputs easier to produce. Although OpenAI has tried to put guard rails on what the chatbot will do, users are already finding ways around them.
The big worry in the research community is that students and scientists could deceitfully pass off LLM-written text as their own, or use LLMs in a simplistic fashion (such as to conduct an incomplete literature review) and produce work that is unreliable. Several preprints and published articles have already credited ChatGPT with formal authorship.