A mixed methods crossover randomized controlled trial exploring the experiences, perceptions, and usability of artificial intelligence (ChatGPT) in health sciences education
Background Generative artificial intelligence (AI) integrated programs such as Chat Generative Pre-trained Transformers (ChatGPT) are becoming more widespread in educational settings, with mounting ethical and reliability concerns regarding its usage. This paper explores the experiences, perceptions...
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SAGE Publishing
2024-12-01
|
| Series: | Digital Health |
| Online Access: | https://doi.org/10.1177/20552076241298485 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832096202382901248 |
|---|---|
| author | Mirella Veras Joseph-Omer Dyer Holly Shannon Bryce J M Bogie Morgan Ronney Harmehr Sekhon Derek Rutherford Paulo Goberlânio Barros Silva Dahlia Kairy |
| author_facet | Mirella Veras Joseph-Omer Dyer Holly Shannon Bryce J M Bogie Morgan Ronney Harmehr Sekhon Derek Rutherford Paulo Goberlânio Barros Silva Dahlia Kairy |
| author_sort | Mirella Veras |
| collection | DOAJ |
| description | Background Generative artificial intelligence (AI) integrated programs such as Chat Generative Pre-trained Transformers (ChatGPT) are becoming more widespread in educational settings, with mounting ethical and reliability concerns regarding its usage. This paper explores the experiences, perceptions, and usability of ChatGPT in undergraduate health sciences students. Methods Twenty-seven students at Carleton University (Canada) were enrolled in a crossover randomized controlled trial study from a Health Sciences course during the Fall 2023 academic term. The intervention condition involved the use of ChatGPT-3.5, whereas the control condition involved using conventional web-based tools. Technology usability was compared between ChatGPT-3.5 and the traditional tools using questionnaires. Focus group discussions were conducted with seven students to further elaborate on student perceptions and experiences. Reflexive thematic analysis was employed to identify themes from the focus group data. Results Easiness of learnability for personal use and a perception of quick learnability towards ChatGPT-3.5 were significantly higher, compared to conventional online tools from the Systems Usability Scale. Qualitative results highlighted strong benefits of ChatGPT-3.5, such as being a tool for increased overall productivity and brainstorming. However, students identified challenges associated with reliability and accuracy, and concerns about academic integrity. Conclusions Despite the benefits and positive usability of ChatGPT-3.5 identified by students, an explicit need for the development of policies, procedures and regulations remains. An established framework of best practices for the usage of AI within health science education is necessary. This will ensure accountability of users and lead to a more effective integration of AI technologies into academic settings. |
| format | Article |
| id | doaj-art-133a3814dd1240e499a7e734d2222693 |
| institution | Kabale University |
| issn | 2055-2076 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | SAGE Publishing |
| record_format | Article |
| series | Digital Health |
| spelling | doaj-art-133a3814dd1240e499a7e734d22226932025-02-05T16:03:21ZengSAGE PublishingDigital Health2055-20762024-12-011010.1177/20552076241298485A mixed methods crossover randomized controlled trial exploring the experiences, perceptions, and usability of artificial intelligence (ChatGPT) in health sciences educationMirella Veras0Joseph-Omer Dyer1Holly Shannon2 Bryce J M Bogie3Morgan Ronney4Harmehr Sekhon5Derek Rutherford6Paulo Goberlânio Barros Silva7Dahlia Kairy8 Department of Health Sciences, Carleton University, Ottawa, ON, Canada Groupe Interdisciplinaire de Recherche sur la Cognition et le Raisonnement Professionnel, Faculty of Medicine, , Montréal, QC, Canada Department of Neuroscience, , Ottawa, ON, Canada Department of Cellular and Molecular Medicine, Faculty of Medicine, , Ottawa, ON, Canada Teaching and Learning Services, , Ottawa, ON, Canada , Toronto, ON, Canada School of Physiotherapy, , Halifax, NS, Canada , Fortaleza, Ceará, Brazil Institut Universitaire sur la Réadaptation en Déficience Physique de Montréal, Centre Intégré Universitaire de Santé et Services Sociaux du Centre-Sud-de-l'Île-de-Montréal, Montréal, QC, CanadaBackground Generative artificial intelligence (AI) integrated programs such as Chat Generative Pre-trained Transformers (ChatGPT) are becoming more widespread in educational settings, with mounting ethical and reliability concerns regarding its usage. This paper explores the experiences, perceptions, and usability of ChatGPT in undergraduate health sciences students. Methods Twenty-seven students at Carleton University (Canada) were enrolled in a crossover randomized controlled trial study from a Health Sciences course during the Fall 2023 academic term. The intervention condition involved the use of ChatGPT-3.5, whereas the control condition involved using conventional web-based tools. Technology usability was compared between ChatGPT-3.5 and the traditional tools using questionnaires. Focus group discussions were conducted with seven students to further elaborate on student perceptions and experiences. Reflexive thematic analysis was employed to identify themes from the focus group data. Results Easiness of learnability for personal use and a perception of quick learnability towards ChatGPT-3.5 were significantly higher, compared to conventional online tools from the Systems Usability Scale. Qualitative results highlighted strong benefits of ChatGPT-3.5, such as being a tool for increased overall productivity and brainstorming. However, students identified challenges associated with reliability and accuracy, and concerns about academic integrity. Conclusions Despite the benefits and positive usability of ChatGPT-3.5 identified by students, an explicit need for the development of policies, procedures and regulations remains. An established framework of best practices for the usage of AI within health science education is necessary. This will ensure accountability of users and lead to a more effective integration of AI technologies into academic settings.https://doi.org/10.1177/20552076241298485 |
| spellingShingle | Mirella Veras Joseph-Omer Dyer Holly Shannon Bryce J M Bogie Morgan Ronney Harmehr Sekhon Derek Rutherford Paulo Goberlânio Barros Silva Dahlia Kairy A mixed methods crossover randomized controlled trial exploring the experiences, perceptions, and usability of artificial intelligence (ChatGPT) in health sciences education Digital Health |
| title | A mixed methods crossover randomized controlled trial exploring the experiences, perceptions, and usability of artificial intelligence (ChatGPT) in health sciences education |
| title_full | A mixed methods crossover randomized controlled trial exploring the experiences, perceptions, and usability of artificial intelligence (ChatGPT) in health sciences education |
| title_fullStr | A mixed methods crossover randomized controlled trial exploring the experiences, perceptions, and usability of artificial intelligence (ChatGPT) in health sciences education |
| title_full_unstemmed | A mixed methods crossover randomized controlled trial exploring the experiences, perceptions, and usability of artificial intelligence (ChatGPT) in health sciences education |
| title_short | A mixed methods crossover randomized controlled trial exploring the experiences, perceptions, and usability of artificial intelligence (ChatGPT) in health sciences education |
| title_sort | mixed methods crossover randomized controlled trial exploring the experiences perceptions and usability of artificial intelligence chatgpt in health sciences education |
| url | https://doi.org/10.1177/20552076241298485 |
| work_keys_str_mv | AT mirellaveras amixedmethodscrossoverrandomizedcontrolledtrialexploringtheexperiencesperceptionsandusabilityofartificialintelligencechatgptinhealthscienceseducation AT josephomerdyer amixedmethodscrossoverrandomizedcontrolledtrialexploringtheexperiencesperceptionsandusabilityofartificialintelligencechatgptinhealthscienceseducation AT hollyshannon amixedmethodscrossoverrandomizedcontrolledtrialexploringtheexperiencesperceptionsandusabilityofartificialintelligencechatgptinhealthscienceseducation AT brycejmbogie amixedmethodscrossoverrandomizedcontrolledtrialexploringtheexperiencesperceptionsandusabilityofartificialintelligencechatgptinhealthscienceseducation AT morganronney amixedmethodscrossoverrandomizedcontrolledtrialexploringtheexperiencesperceptionsandusabilityofartificialintelligencechatgptinhealthscienceseducation AT harmehrsekhon amixedmethodscrossoverrandomizedcontrolledtrialexploringtheexperiencesperceptionsandusabilityofartificialintelligencechatgptinhealthscienceseducation AT derekrutherford amixedmethodscrossoverrandomizedcontrolledtrialexploringtheexperiencesperceptionsandusabilityofartificialintelligencechatgptinhealthscienceseducation AT paulogoberlaniobarrossilva amixedmethodscrossoverrandomizedcontrolledtrialexploringtheexperiencesperceptionsandusabilityofartificialintelligencechatgptinhealthscienceseducation AT dahliakairy amixedmethodscrossoverrandomizedcontrolledtrialexploringtheexperiencesperceptionsandusabilityofartificialintelligencechatgptinhealthscienceseducation AT mirellaveras mixedmethodscrossoverrandomizedcontrolledtrialexploringtheexperiencesperceptionsandusabilityofartificialintelligencechatgptinhealthscienceseducation AT josephomerdyer mixedmethodscrossoverrandomizedcontrolledtrialexploringtheexperiencesperceptionsandusabilityofartificialintelligencechatgptinhealthscienceseducation AT hollyshannon mixedmethodscrossoverrandomizedcontrolledtrialexploringtheexperiencesperceptionsandusabilityofartificialintelligencechatgptinhealthscienceseducation AT brycejmbogie mixedmethodscrossoverrandomizedcontrolledtrialexploringtheexperiencesperceptionsandusabilityofartificialintelligencechatgptinhealthscienceseducation AT morganronney mixedmethodscrossoverrandomizedcontrolledtrialexploringtheexperiencesperceptionsandusabilityofartificialintelligencechatgptinhealthscienceseducation AT harmehrsekhon mixedmethodscrossoverrandomizedcontrolledtrialexploringtheexperiencesperceptionsandusabilityofartificialintelligencechatgptinhealthscienceseducation AT derekrutherford mixedmethodscrossoverrandomizedcontrolledtrialexploringtheexperiencesperceptionsandusabilityofartificialintelligencechatgptinhealthscienceseducation AT paulogoberlaniobarrossilva mixedmethodscrossoverrandomizedcontrolledtrialexploringtheexperiencesperceptionsandusabilityofartificialintelligencechatgptinhealthscienceseducation AT dahliakairy mixedmethodscrossoverrandomizedcontrolledtrialexploringtheexperiencesperceptionsandusabilityofartificialintelligencechatgptinhealthscienceseducation |