AI-teacher agreement in evaluating learning diaries
Learning diaries are reflective tools, often used as formative assessments in adult education with the aim to promote cognitive and metacognitive learning strategies. As grading of and feedback on learning diaries is effortful for teachers, artificial intelligence (AI) may assist teachers in evaluat...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2025-07-01
|
| Series: | Frontiers in Education |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/feduc.2025.1601789/full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849319436044992512 |
|---|---|
| author | Lhea Reinhold Marion Händel Nick Naujoks-Schober |
| author_facet | Lhea Reinhold Marion Händel Nick Naujoks-Schober |
| author_sort | Lhea Reinhold |
| collection | DOAJ |
| description | Learning diaries are reflective tools, often used as formative assessments in adult education with the aim to promote cognitive and metacognitive learning strategies. As grading of and feedback on learning diaries is effortful for teachers, artificial intelligence (AI) may assist teachers in evaluating learning diaries. A prerequisite is that AI's ratings show high accordance with the teachers' ratings. AI accuracy, measured via absolute accuracy and bias, is the focus of the current study with N = 540 learning diary entries focusing on learning strategies, seven teachers, and ChatGPT-4o. Findings revealed that AI evaluations align closely with teacher assessments, indicated by high overall accuracy and low bias. Interestingly, the accuracy varied based on the types of learning strategies assessed in the diaries. Additionally, individual teacher assessments influenced the alignment between human and AI evaluations, suggesting that teachers applied their profession-specific expertise to the assessment process while AI produced somewhat generic evaluations. Overall, the study results indicate that AI can enhance the efficiency of formative assessments while providing timely feedback to learners. |
| format | Article |
| id | doaj-art-e3e044a522da497f84f802f4cf3b1c06 |
| institution | Kabale University |
| issn | 2504-284X |
| language | English |
| publishDate | 2025-07-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Education |
| spelling | doaj-art-e3e044a522da497f84f802f4cf3b1c062025-08-20T03:50:26ZengFrontiers Media S.A.Frontiers in Education2504-284X2025-07-011010.3389/feduc.2025.16017891601789AI-teacher agreement in evaluating learning diariesLhea Reinhold0Marion Händel1Nick Naujoks-Schober2Faculty of Humanities, Social Sciences, and Theology, Friedrich-Alexander-Universität Erlangen-Nürnberg, Nürnberg, GermanyFaculty of Media, Ansbach University of Applied Sciences, Ansbach, GermanyFaculty of Media, Ansbach University of Applied Sciences, Ansbach, GermanyLearning diaries are reflective tools, often used as formative assessments in adult education with the aim to promote cognitive and metacognitive learning strategies. As grading of and feedback on learning diaries is effortful for teachers, artificial intelligence (AI) may assist teachers in evaluating learning diaries. A prerequisite is that AI's ratings show high accordance with the teachers' ratings. AI accuracy, measured via absolute accuracy and bias, is the focus of the current study with N = 540 learning diary entries focusing on learning strategies, seven teachers, and ChatGPT-4o. Findings revealed that AI evaluations align closely with teacher assessments, indicated by high overall accuracy and low bias. Interestingly, the accuracy varied based on the types of learning strategies assessed in the diaries. Additionally, individual teacher assessments influenced the alignment between human and AI evaluations, suggesting that teachers applied their profession-specific expertise to the assessment process while AI produced somewhat generic evaluations. Overall, the study results indicate that AI can enhance the efficiency of formative assessments while providing timely feedback to learners.https://www.frontiersin.org/articles/10.3389/feduc.2025.1601789/fulllearning diaryAI assessmentAI accuracyAI-teacher agreementadult education |
| spellingShingle | Lhea Reinhold Marion Händel Nick Naujoks-Schober AI-teacher agreement in evaluating learning diaries Frontiers in Education learning diary AI assessment AI accuracy AI-teacher agreement adult education |
| title | AI-teacher agreement in evaluating learning diaries |
| title_full | AI-teacher agreement in evaluating learning diaries |
| title_fullStr | AI-teacher agreement in evaluating learning diaries |
| title_full_unstemmed | AI-teacher agreement in evaluating learning diaries |
| title_short | AI-teacher agreement in evaluating learning diaries |
| title_sort | ai teacher agreement in evaluating learning diaries |
| topic | learning diary AI assessment AI accuracy AI-teacher agreement adult education |
| url | https://www.frontiersin.org/articles/10.3389/feduc.2025.1601789/full |
| work_keys_str_mv | AT lheareinhold aiteacheragreementinevaluatinglearningdiaries AT marionhandel aiteacheragreementinevaluatinglearningdiaries AT nicknaujoksschober aiteacheragreementinevaluatinglearningdiaries |