The impact of large language models on computer science student writing - Katerina Zdravkova; Bojan Ilijoski (International Journal of Educational Technology in Higher Education)
Abstract The new version of ACM and IEEE-CS Computer Science Curricula envisages the preparation of a dozen white papers within more than a third of the Body of Knowledge courses. Lack of experience in conducting research, insufficient ability to articulate thoughts, non-observance of defined recommendations, as well as forgotten verification of what has been done are key challenges in the implementation of these recommendations. Large language models (LLMs) can significantly support the creation of anticipated white papers, encouraging initial directions and inspiration for research, as well as a focused presentation of key elements of the work. Unfortunately, they can also hinder the creation of expected academic texts, promoting the superficiality of research and the presentation of unverified information. In order to facilitate progress in acquiring the necessary writing skills without discriminating against those students who have already acquired them and prefer traditional writing, we have defined a series of strict rules and strategies for preparing the briefing reports with complete, partial, or no reliance on the LLM. They were embedded into the faculty learning management system, implemented, and evaluated twice with more than 150 students during the academic year 2023/24. This paper presents a new approach along with the lessons learned from the first attempt and the changes incorporated into the second. Based on these two carefully designed, conducted and analyzed studies, teachers and students gained a very positive opinion about the impact of LLM on the creation of quality student reports. The effectiveness of the new approach is a guarantee that optimization of the integration of AI-generated content can become a benchmark for the successful preparation of student reports that can be recommended to the wider academic community.