Large language models for pretreatment education in pediatric radiation oncology: A comparative evaluation study
Background and purpose: Pediatric radiotherapy patients and their parents are usually aware of their need for radiotherapy early on, but they meet with a radiation oncologist later in their treatment. Consequently, they search for information online, often encountering unreliable sources. Large lang...
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2025-03-01
|
| Series: | Clinical and Translational Radiation Oncology |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2405630825000047 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832582146314731520 |
|---|---|
| author | Dominik Wawrzuta Aleksandra Napieralska Katarzyna Ludwikowska Laimonas Jaruševičius Anastasija Trofimoviča-Krasnorucka Gints Rausis Agata Szulc Katarzyna Pędziwiatr Kateřina Poláchová Justyna Klejdysz Marzanna Chojnacka |
| author_facet | Dominik Wawrzuta Aleksandra Napieralska Katarzyna Ludwikowska Laimonas Jaruševičius Anastasija Trofimoviča-Krasnorucka Gints Rausis Agata Szulc Katarzyna Pędziwiatr Kateřina Poláchová Justyna Klejdysz Marzanna Chojnacka |
| author_sort | Dominik Wawrzuta |
| collection | DOAJ |
| description | Background and purpose: Pediatric radiotherapy patients and their parents are usually aware of their need for radiotherapy early on, but they meet with a radiation oncologist later in their treatment. Consequently, they search for information online, often encountering unreliable sources. Large language models (LLMs) have the potential to serve as an educational pretreatment tool, providing reliable answers to their questions. We aimed to evaluate the responses provided by generative pre-trained transformers (GPT), the most popular subgroup of LLMs, to questions about pediatric radiation oncology. Materials and methods: We collected pretreatment questions regarding radiotherapy from patients and parents. Responses were generated using GPT-3.5, GPT-4, and fine-tuned GPT-3.5, with fine-tuning based on pediatric radiotherapy guides from various institutions. Additionally, a radiation oncologist prepared answers to these questions. Finally, a multi-institutional group of nine pediatric radiotherapy experts conducted a blind review of responses, assessing reliability, concision, and comprehensibility. Results: The radiation oncologist and GPT-4 provided the highest-quality responses, though GPT-4′s answers were often excessively verbose. While fine-tuned GPT-3.5 generally outperformed basic GPT-3.5, it often provided overly simplistic answers. Inadequate responses were rare, occurring in 4% of GPT-generated responses across all models, primarily due to GPT-3.5 generating excessively long responses. Conclusions: LLMs can be valuable tools for educating patients and their families before treatment in pediatric radiation oncology. Among them, only GPT-4 provides information of a quality comparable to that of a radiation oncologist, although it still occasionally generates poor-quality responses. GPT-3.5 models should be used cautiously, as they are more likely to produce inadequate answers to patient questions. |
| format | Article |
| id | doaj-art-1640f44bec1543a8b524c2a3d5564f5f |
| institution | Kabale University |
| issn | 2405-6308 |
| language | English |
| publishDate | 2025-03-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Clinical and Translational Radiation Oncology |
| spelling | doaj-art-1640f44bec1543a8b524c2a3d5564f5f2025-01-30T05:14:29ZengElsevierClinical and Translational Radiation Oncology2405-63082025-03-0151100914Large language models for pretreatment education in pediatric radiation oncology: A comparative evaluation studyDominik Wawrzuta0Aleksandra Napieralska1Katarzyna Ludwikowska2Laimonas Jaruševičius3Anastasija Trofimoviča-Krasnorucka4Gints Rausis5Agata Szulc6Katarzyna Pędziwiatr7Kateřina Poláchová8Justyna Klejdysz9Marzanna Chojnacka10Department of Radiation Oncology, Maria Sklodowska-Curie National Research Institute of Oncology, Wawelska 15B, 02-034 Warsaw, Poland; Corresponding author.Radiotherapy Department, Maria Sklodowska-Curie National Research Institute of Oncology, Wybrzeże Armii Krajowej 15, 44-100 Gliwice, Poland; Department of Oncology, Maria Sklodowska-Curie National Research Institute of Oncology, Garncarska 11, 31-115 Cracow, Poland; Faculty of Medicine & Health Sciences, Andrzej Frycz Modrzewski Krakow University, Gustawa Herlinga-Grudzińskiego 1, 30-705 Cracow, PolandDepartment of Radiation Oncology, Maria Sklodowska-Curie National Research Institute of Oncology, Wawelska 15B, 02-034 Warsaw, PolandOncology Institute, Lithuanian University of Health Sciences, A. Mickevičiaus g. 9, LT-44307, Kaunas, LithuaniaDepartment of Radiation Oncology, Riga East University Hospital, Hipokrāta iela 2, LV-1038 Riga, Latvia; Department of Internal Diseases, Riga Stradiņš University, Dzirciema iela 16, LV-1007 Riga, LatviaDepartment of Radiation Oncology, Riga East University Hospital, Hipokrāta iela 2, LV-1038 Riga, LatviaDepartment of Radiation Oncology, Lower Silesian Center of Oncology, Pulmonology and Hematology, Hirszfelda 12, 53-413 Wroclaw, PolandDepartment of Radiation Oncology, Maria Sklodowska-Curie National Research Institute of Oncology, Wawelska 15B, 02-034 Warsaw, PolandDepartment of Radiation Oncology, Masaryk Memorial Cancer Institute, Žlutý kopec 7, 656 53 Brno, Czech Republic; Department of Radiation Oncology, Faculty of Medicine, Masaryk University, Kamenice 5, 625 00 Brno, Czech RepublicDepartment of Economics, Ludwig Maximilian University of Munich (LMU), Geschwister-Scholl-Platz 1, 80539 Munich, Germany; ifo Institute, Poschinger Straße 5, 81679 Munich, GermanyDepartment of Radiation Oncology, Maria Sklodowska-Curie National Research Institute of Oncology, Wawelska 15B, 02-034 Warsaw, PolandBackground and purpose: Pediatric radiotherapy patients and their parents are usually aware of their need for radiotherapy early on, but they meet with a radiation oncologist later in their treatment. Consequently, they search for information online, often encountering unreliable sources. Large language models (LLMs) have the potential to serve as an educational pretreatment tool, providing reliable answers to their questions. We aimed to evaluate the responses provided by generative pre-trained transformers (GPT), the most popular subgroup of LLMs, to questions about pediatric radiation oncology. Materials and methods: We collected pretreatment questions regarding radiotherapy from patients and parents. Responses were generated using GPT-3.5, GPT-4, and fine-tuned GPT-3.5, with fine-tuning based on pediatric radiotherapy guides from various institutions. Additionally, a radiation oncologist prepared answers to these questions. Finally, a multi-institutional group of nine pediatric radiotherapy experts conducted a blind review of responses, assessing reliability, concision, and comprehensibility. Results: The radiation oncologist and GPT-4 provided the highest-quality responses, though GPT-4′s answers were often excessively verbose. While fine-tuned GPT-3.5 generally outperformed basic GPT-3.5, it often provided overly simplistic answers. Inadequate responses were rare, occurring in 4% of GPT-generated responses across all models, primarily due to GPT-3.5 generating excessively long responses. Conclusions: LLMs can be valuable tools for educating patients and their families before treatment in pediatric radiation oncology. Among them, only GPT-4 provides information of a quality comparable to that of a radiation oncologist, although it still occasionally generates poor-quality responses. GPT-3.5 models should be used cautiously, as they are more likely to produce inadequate answers to patient questions.http://www.sciencedirect.com/science/article/pii/S2405630825000047 |
| spellingShingle | Dominik Wawrzuta Aleksandra Napieralska Katarzyna Ludwikowska Laimonas Jaruševičius Anastasija Trofimoviča-Krasnorucka Gints Rausis Agata Szulc Katarzyna Pędziwiatr Kateřina Poláchová Justyna Klejdysz Marzanna Chojnacka Large language models for pretreatment education in pediatric radiation oncology: A comparative evaluation study Clinical and Translational Radiation Oncology |
| title | Large language models for pretreatment education in pediatric radiation oncology: A comparative evaluation study |
| title_full | Large language models for pretreatment education in pediatric radiation oncology: A comparative evaluation study |
| title_fullStr | Large language models for pretreatment education in pediatric radiation oncology: A comparative evaluation study |
| title_full_unstemmed | Large language models for pretreatment education in pediatric radiation oncology: A comparative evaluation study |
| title_short | Large language models for pretreatment education in pediatric radiation oncology: A comparative evaluation study |
| title_sort | large language models for pretreatment education in pediatric radiation oncology a comparative evaluation study |
| url | http://www.sciencedirect.com/science/article/pii/S2405630825000047 |
| work_keys_str_mv | AT dominikwawrzuta largelanguagemodelsforpretreatmenteducationinpediatricradiationoncologyacomparativeevaluationstudy AT aleksandranapieralska largelanguagemodelsforpretreatmenteducationinpediatricradiationoncologyacomparativeevaluationstudy AT katarzynaludwikowska largelanguagemodelsforpretreatmenteducationinpediatricradiationoncologyacomparativeevaluationstudy AT laimonasjarusevicius largelanguagemodelsforpretreatmenteducationinpediatricradiationoncologyacomparativeevaluationstudy AT anastasijatrofimovicakrasnorucka largelanguagemodelsforpretreatmenteducationinpediatricradiationoncologyacomparativeevaluationstudy AT gintsrausis largelanguagemodelsforpretreatmenteducationinpediatricradiationoncologyacomparativeevaluationstudy AT agataszulc largelanguagemodelsforpretreatmenteducationinpediatricradiationoncologyacomparativeevaluationstudy AT katarzynapedziwiatr largelanguagemodelsforpretreatmenteducationinpediatricradiationoncologyacomparativeevaluationstudy AT katerinapolachova largelanguagemodelsforpretreatmenteducationinpediatricradiationoncologyacomparativeevaluationstudy AT justynaklejdysz largelanguagemodelsforpretreatmenteducationinpediatricradiationoncologyacomparativeevaluationstudy AT marzannachojnacka largelanguagemodelsforpretreatmenteducationinpediatricradiationoncologyacomparativeevaluationstudy |