Subject-Specialized Chatbot in Higher Education as a Tutor for Autonomous Exam Preparation: Analysis of the Impact on Academic Performance and Students’ Perception of Its Usefulness
This study evaluates the impact of an AI chatbot as a support tool for second-year students in the Bachelor’s Degree in Early Childhood Education program during final exam preparation. Over 1-month, 42 students used the chatbot, generating 704 interactions across 186 conversations. The study aimed t...
Saved in:
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-12-01
|
| Series: | Education Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2227-7102/15/1/26 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832588608622559232 |
|---|---|
| author | Fulgencio Sánchez-Vera |
| author_facet | Fulgencio Sánchez-Vera |
| author_sort | Fulgencio Sánchez-Vera |
| collection | DOAJ |
| description | This study evaluates the impact of an AI chatbot as a support tool for second-year students in the Bachelor’s Degree in Early Childhood Education program during final exam preparation. Over 1-month, 42 students used the chatbot, generating 704 interactions across 186 conversations. The study aimed to assess the chatbot’s effectiveness in resolving specific questions, enhancing concept comprehension, and preparing for exams. Methods included surveys, in-depth interviews, and analysis of chatbot interactions. Results showed that the chatbot was highly effective in clarifying doubts (91.4%) and aiding concept understanding (95.7%), although its perceived usefulness was lower in content review (42.9%) and exam simulations (45.4%). Students with moderate chatbot use achieved better academic outcomes, while excessive use did not lead to further improvements. The study also identified challenges in students’ ability to formulate effective questions, limiting the chatbot’s potential in some areas. Overall, the chatbot was valued for fostering study autonomy, though improvements are needed in features supporting motivation and study organization. These findings highlight the potential of chatbots as complementary learning tools but underscore the need for better user training in “prompt engineering” to maximize their effectiveness. |
| format | Article |
| id | doaj-art-1f52aade53764403b4857f5ea7d26f43 |
| institution | Kabale University |
| issn | 2227-7102 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Education Sciences |
| spelling | doaj-art-1f52aade53764403b4857f5ea7d26f432025-01-24T13:30:12ZengMDPI AGEducation Sciences2227-71022024-12-011512610.3390/educsci15010026Subject-Specialized Chatbot in Higher Education as a Tutor for Autonomous Exam Preparation: Analysis of the Impact on Academic Performance and Students’ Perception of Its UsefulnessFulgencio Sánchez-Vera0Department of Didactics and Educational Research, University of La Laguna, 38200 San Cristóbal de La Laguna, SpainThis study evaluates the impact of an AI chatbot as a support tool for second-year students in the Bachelor’s Degree in Early Childhood Education program during final exam preparation. Over 1-month, 42 students used the chatbot, generating 704 interactions across 186 conversations. The study aimed to assess the chatbot’s effectiveness in resolving specific questions, enhancing concept comprehension, and preparing for exams. Methods included surveys, in-depth interviews, and analysis of chatbot interactions. Results showed that the chatbot was highly effective in clarifying doubts (91.4%) and aiding concept understanding (95.7%), although its perceived usefulness was lower in content review (42.9%) and exam simulations (45.4%). Students with moderate chatbot use achieved better academic outcomes, while excessive use did not lead to further improvements. The study also identified challenges in students’ ability to formulate effective questions, limiting the chatbot’s potential in some areas. Overall, the chatbot was valued for fostering study autonomy, though improvements are needed in features supporting motivation and study organization. These findings highlight the potential of chatbots as complementary learning tools but underscore the need for better user training in “prompt engineering” to maximize their effectiveness.https://www.mdpi.com/2227-7102/15/1/26educational technologyeducational chatbotartificial intelligencehigher educationprompt engineering |
| spellingShingle | Fulgencio Sánchez-Vera Subject-Specialized Chatbot in Higher Education as a Tutor for Autonomous Exam Preparation: Analysis of the Impact on Academic Performance and Students’ Perception of Its Usefulness Education Sciences educational technology educational chatbot artificial intelligence higher education prompt engineering |
| title | Subject-Specialized Chatbot in Higher Education as a Tutor for Autonomous Exam Preparation: Analysis of the Impact on Academic Performance and Students’ Perception of Its Usefulness |
| title_full | Subject-Specialized Chatbot in Higher Education as a Tutor for Autonomous Exam Preparation: Analysis of the Impact on Academic Performance and Students’ Perception of Its Usefulness |
| title_fullStr | Subject-Specialized Chatbot in Higher Education as a Tutor for Autonomous Exam Preparation: Analysis of the Impact on Academic Performance and Students’ Perception of Its Usefulness |
| title_full_unstemmed | Subject-Specialized Chatbot in Higher Education as a Tutor for Autonomous Exam Preparation: Analysis of the Impact on Academic Performance and Students’ Perception of Its Usefulness |
| title_short | Subject-Specialized Chatbot in Higher Education as a Tutor for Autonomous Exam Preparation: Analysis of the Impact on Academic Performance and Students’ Perception of Its Usefulness |
| title_sort | subject specialized chatbot in higher education as a tutor for autonomous exam preparation analysis of the impact on academic performance and students perception of its usefulness |
| topic | educational technology educational chatbot artificial intelligence higher education prompt engineering |
| url | https://www.mdpi.com/2227-7102/15/1/26 |
| work_keys_str_mv | AT fulgenciosanchezvera subjectspecializedchatbotinhighereducationasatutorforautonomousexampreparationanalysisoftheimpactonacademicperformanceandstudentsperceptionofitsusefulness |