Evaluating simulated teaching audio for teacher trainees using RAG and local LLMs
Abstract In the training of teacher students, simulated teaching is a key method for enhancing teaching skills. However, traditional evaluations of simulated teaching typically rely on direct teacher involvement and guidance, increasing teachers’ workload and limiting the opportunities for teacher s...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-01-01
|
| Series: | Scientific Reports |
| Subjects: | |
| Online Access: | https://doi.org/10.1038/s41598-025-87898-5 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832571822100447232 |
|---|---|
| author | Ke Fang Ci Tang Jing Wang |
| author_facet | Ke Fang Ci Tang Jing Wang |
| author_sort | Ke Fang |
| collection | DOAJ |
| description | Abstract In the training of teacher students, simulated teaching is a key method for enhancing teaching skills. However, traditional evaluations of simulated teaching typically rely on direct teacher involvement and guidance, increasing teachers’ workload and limiting the opportunities for teacher students to practice independently. This paper introduces a Retrieval-Augmented Generation (RAG) framework constructed using various open-source tools (such as FastChat for model inference and Whisper for speech-to-text) combined with a local large language model (LLM) for audio analysis of simulated teaching. We then selected three leading 7B-parameter open-source Chinese LLMs from the ModelScope community to analyze their generalizability and adaptability in simulated teaching voice evaluation tasks. The results show that the internlm2 model more effectively analyzes teacher students’ teaching audio, providing key educational feedback. Finally, we conducted a system analysis of the simulated teaching of 10 participants in a teaching ability competition and invited three experts to score manually, verifying the system’s application potential. This research demonstrates a potential approach to improving educational evaluation methods using advanced language technology. |
| format | Article |
| id | doaj-art-c31f4dc8ddb74c748ee0943370ffef1b |
| institution | Kabale University |
| issn | 2045-2322 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj-art-c31f4dc8ddb74c748ee0943370ffef1b2025-02-02T12:17:51ZengNature PortfolioScientific Reports2045-23222025-01-0115111110.1038/s41598-025-87898-5Evaluating simulated teaching audio for teacher trainees using RAG and local LLMsKe Fang0Ci Tang1Jing Wang2Network and Information Center, Chengdu Normal UniversityOffice for the Advancement of Educational Information, Chengdu Normal UniversityOffice for the Advancement of Educational Information, Chengdu Normal UniversityAbstract In the training of teacher students, simulated teaching is a key method for enhancing teaching skills. However, traditional evaluations of simulated teaching typically rely on direct teacher involvement and guidance, increasing teachers’ workload and limiting the opportunities for teacher students to practice independently. This paper introduces a Retrieval-Augmented Generation (RAG) framework constructed using various open-source tools (such as FastChat for model inference and Whisper for speech-to-text) combined with a local large language model (LLM) for audio analysis of simulated teaching. We then selected three leading 7B-parameter open-source Chinese LLMs from the ModelScope community to analyze their generalizability and adaptability in simulated teaching voice evaluation tasks. The results show that the internlm2 model more effectively analyzes teacher students’ teaching audio, providing key educational feedback. Finally, we conducted a system analysis of the simulated teaching of 10 participants in a teaching ability competition and invited three experts to score manually, verifying the system’s application potential. This research demonstrates a potential approach to improving educational evaluation methods using advanced language technology.https://doi.org/10.1038/s41598-025-87898-5Simulated teachingOpen-source toolsRAG frameworkTeacher student trainingLLMs |
| spellingShingle | Ke Fang Ci Tang Jing Wang Evaluating simulated teaching audio for teacher trainees using RAG and local LLMs Scientific Reports Simulated teaching Open-source tools RAG framework Teacher student training LLMs |
| title | Evaluating simulated teaching audio for teacher trainees using RAG and local LLMs |
| title_full | Evaluating simulated teaching audio for teacher trainees using RAG and local LLMs |
| title_fullStr | Evaluating simulated teaching audio for teacher trainees using RAG and local LLMs |
| title_full_unstemmed | Evaluating simulated teaching audio for teacher trainees using RAG and local LLMs |
| title_short | Evaluating simulated teaching audio for teacher trainees using RAG and local LLMs |
| title_sort | evaluating simulated teaching audio for teacher trainees using rag and local llms |
| topic | Simulated teaching Open-source tools RAG framework Teacher student training LLMs |
| url | https://doi.org/10.1038/s41598-025-87898-5 |
| work_keys_str_mv | AT kefang evaluatingsimulatedteachingaudioforteachertraineesusingragandlocalllms AT citang evaluatingsimulatedteachingaudioforteachertraineesusingragandlocalllms AT jingwang evaluatingsimulatedteachingaudioforteachertraineesusingragandlocalllms |