Application of Conversational AI Models in Decision Making for Clinical Periodontology: Analysis and Predictive Modeling
(1) Background: Language represents a crucial ability of humans, enabling communication and collaboration. ChatGPT is an AI chatbot utilizing the GPT (Generative Pretrained Transformer) language model architecture, enabling the generation of human-like text. The aim of the research was to assess the...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-01-01
|
| Series: | AI |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2673-2688/6/1/3 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832589386542219264 |
|---|---|
| author | Albert Camlet Aida Kusiak Dariusz Świetlik |
| author_facet | Albert Camlet Aida Kusiak Dariusz Świetlik |
| author_sort | Albert Camlet |
| collection | DOAJ |
| description | (1) Background: Language represents a crucial ability of humans, enabling communication and collaboration. ChatGPT is an AI chatbot utilizing the GPT (Generative Pretrained Transformer) language model architecture, enabling the generation of human-like text. The aim of the research was to assess the effectiveness of ChatGPT-3.5 and the latest version, ChatGPT-4, in responding to questions posed within the scope of a periodontology specialization exam. (2) Methods: Two certification examinations in periodontology, available in both English and Polish, comprising 120 multiple-choice questions, each in a single-best-answer format. The questions were additionally assigned to five types in accordance with the subject covered. These exams were utilized to evaluate the performance of ChatGPT-3.5 and ChatGPT-4. Logistic regression models were used to estimate the chances of correct answers regarding the type of question, exam session, AI model, and difficulty index. (3) Results: The percentages of correct answers obtained by ChatGPT-3.5 and ChatGPT-4 in the Spring 2023 session in Polish and English were 40.3% vs. 55.5% and 45.4% vs. 68.9%, respectively. The periodontology specialty examination test accuracy of ChatGPT-4 was significantly better than that of ChatGPT-3.5 for both sessions (<i>p</i> < 0.05). For the ChatGPT-4 spring session, it was significantly more effective in the English language (<i>p</i> = 0.0325) due to the lack of statistically significant differences for ChatGPT-3.5. In the case of ChatGPT-3.5 and ChatGPT-4, incorrect responses showed notably lower difficulty index values during the Spring 2023 session in English and Polish (<i>p</i> < 0.05). (4) Conclusions: ChatGPT-4 exceeded the 60% threshold and passed the examination in the Spring 2023 session in the English version. In general, ChatGPT-4 performed better than ChatGPT-3.5, achieving significantly better results in the Spring 2023 test in the Polish and English versions. |
| format | Article |
| id | doaj-art-3066c1b1145f4b9f99a718f6a693e168 |
| institution | Kabale University |
| issn | 2673-2688 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | AI |
| spelling | doaj-art-3066c1b1145f4b9f99a718f6a693e1682025-01-24T13:17:21ZengMDPI AGAI2673-26882025-01-0161310.3390/ai6010003Application of Conversational AI Models in Decision Making for Clinical Periodontology: Analysis and Predictive ModelingAlbert Camlet0Aida Kusiak1Dariusz Świetlik2Department of Periodontology and Oral Mucosa Diseases, Medical University of Gdansk, Orzeszkowej 18 St., 80-208 Gdansk, PolandDepartment of Periodontology and Oral Mucosa Diseases, Medical University of Gdansk, Orzeszkowej 18 St., 80-208 Gdansk, PolandDivision of Biostatistics and Neural Networks, Medical University of Gdansk, Debinki 1 St., 80-211 Gdansk, Poland(1) Background: Language represents a crucial ability of humans, enabling communication and collaboration. ChatGPT is an AI chatbot utilizing the GPT (Generative Pretrained Transformer) language model architecture, enabling the generation of human-like text. The aim of the research was to assess the effectiveness of ChatGPT-3.5 and the latest version, ChatGPT-4, in responding to questions posed within the scope of a periodontology specialization exam. (2) Methods: Two certification examinations in periodontology, available in both English and Polish, comprising 120 multiple-choice questions, each in a single-best-answer format. The questions were additionally assigned to five types in accordance with the subject covered. These exams were utilized to evaluate the performance of ChatGPT-3.5 and ChatGPT-4. Logistic regression models were used to estimate the chances of correct answers regarding the type of question, exam session, AI model, and difficulty index. (3) Results: The percentages of correct answers obtained by ChatGPT-3.5 and ChatGPT-4 in the Spring 2023 session in Polish and English were 40.3% vs. 55.5% and 45.4% vs. 68.9%, respectively. The periodontology specialty examination test accuracy of ChatGPT-4 was significantly better than that of ChatGPT-3.5 for both sessions (<i>p</i> < 0.05). For the ChatGPT-4 spring session, it was significantly more effective in the English language (<i>p</i> = 0.0325) due to the lack of statistically significant differences for ChatGPT-3.5. In the case of ChatGPT-3.5 and ChatGPT-4, incorrect responses showed notably lower difficulty index values during the Spring 2023 session in English and Polish (<i>p</i> < 0.05). (4) Conclusions: ChatGPT-4 exceeded the 60% threshold and passed the examination in the Spring 2023 session in the English version. In general, ChatGPT-4 performed better than ChatGPT-3.5, achieving significantly better results in the Spring 2023 test in the Polish and English versions.https://www.mdpi.com/2673-2688/6/1/3artificial intelligenceperiodontologydental educationChatGPTdeep learning-based language model (LLM) |
| spellingShingle | Albert Camlet Aida Kusiak Dariusz Świetlik Application of Conversational AI Models in Decision Making for Clinical Periodontology: Analysis and Predictive Modeling AI artificial intelligence periodontology dental education ChatGPT deep learning-based language model (LLM) |
| title | Application of Conversational AI Models in Decision Making for Clinical Periodontology: Analysis and Predictive Modeling |
| title_full | Application of Conversational AI Models in Decision Making for Clinical Periodontology: Analysis and Predictive Modeling |
| title_fullStr | Application of Conversational AI Models in Decision Making for Clinical Periodontology: Analysis and Predictive Modeling |
| title_full_unstemmed | Application of Conversational AI Models in Decision Making for Clinical Periodontology: Analysis and Predictive Modeling |
| title_short | Application of Conversational AI Models in Decision Making for Clinical Periodontology: Analysis and Predictive Modeling |
| title_sort | application of conversational ai models in decision making for clinical periodontology analysis and predictive modeling |
| topic | artificial intelligence periodontology dental education ChatGPT deep learning-based language model (LLM) |
| url | https://www.mdpi.com/2673-2688/6/1/3 |
| work_keys_str_mv | AT albertcamlet applicationofconversationalaimodelsindecisionmakingforclinicalperiodontologyanalysisandpredictivemodeling AT aidakusiak applicationofconversationalaimodelsindecisionmakingforclinicalperiodontologyanalysisandpredictivemodeling AT dariuszswietlik applicationofconversationalaimodelsindecisionmakingforclinicalperiodontologyanalysisandpredictivemodeling |