ChatGPT, Google, or PINK? Who Provides the Most Reliable Information on Side Effects of Systemic Therapy for Early Breast Cancer?
<b>Introduction:</b> The survival in early breast cancer (BC) has been significantly improved thanks to numerous new drugs. Nevertheless, the information about the need for systemic therapy, especially chemotherapy, represents an additional stress factor for patients. A common coping str...
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-12-01
|
| Series: | Clinics and Practice |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2039-7283/15/1/8 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832588746031104000 |
|---|---|
| author | Stefan Lukac Sebastian Griewing Elena Leinert Davut Dayan Benedikt Heitmeir Markus Wallwiener Wolfgang Janni Visnja Fink Florian Ebner |
| author_facet | Stefan Lukac Sebastian Griewing Elena Leinert Davut Dayan Benedikt Heitmeir Markus Wallwiener Wolfgang Janni Visnja Fink Florian Ebner |
| author_sort | Stefan Lukac |
| collection | DOAJ |
| description | <b>Introduction:</b> The survival in early breast cancer (BC) has been significantly improved thanks to numerous new drugs. Nevertheless, the information about the need for systemic therapy, especially chemotherapy, represents an additional stress factor for patients. A common coping strategy is searching for further information, traditionally via search engines or websites, but artificial intelligence (AI) is also increasingly being used. Who provides the most reliable information is now unclear. <b>Material and Methods</b>: AI in the form of ChatGPT 3.5 and 4.0, Google, and the website of PINK, a provider of a prescription-based mobile health app for patients with BC, were compared to determine the validity of the statements on the five most common side effects of nineteen approved drugs and one drug with pending approval (Ribociclib) for the systemic treatment of BC. For this purpose, the drugs were divided into three groups: chemotherapy, targeted therapy, and endocrine therapy. The reference for the comparison was the prescribing information of the respective drug. A congruence score was calculated for the information on side effects: correct information (2 points), generally appropriate information (1 point), and otherwise no point. The information sources were then compared using a Friedmann test and a Bonferroni-corrected post-hoc test. <b>Results:</b> In the overall comparison, ChatGPT 3.5 received the best score with a congruence of 67.5%, followed by ChatGPT 4.0 with 67.0%, PINK with 59.5%, and with Google 40.0% (<i>p</i> < 0.001). There were also significant differences when comparing the individual subcategories, with the best congruence achieved by PINK (73.3%, <i>p</i> = 0.059) in the chemotherapy category, ChatGPT 4.0 (77.5%; <i>p</i> < 0.001) in the targeted therapy category, and ChatGPT 3.5 (<i>p</i> = 0.002) in the endocrine therapy category. <b>Conclusions:</b> Artificial intelligence and professional online information websites provide the most reliable information on the possible side effects of the systemic treatment of early breast cancer, but congruence with prescribing information is limited. The medical consultation should still be considered the best source of information. |
| format | Article |
| id | doaj-art-17780f39de7c4b98a6a5ece56afb4ae1 |
| institution | Kabale University |
| issn | 2039-7283 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Clinics and Practice |
| spelling | doaj-art-17780f39de7c4b98a6a5ece56afb4ae12025-01-24T13:27:40ZengMDPI AGClinics and Practice2039-72832024-12-01151810.3390/clinpract15010008ChatGPT, Google, or PINK? Who Provides the Most Reliable Information on Side Effects of Systemic Therapy for Early Breast Cancer?Stefan Lukac0Sebastian Griewing1Elena Leinert2Davut Dayan3Benedikt Heitmeir4Markus Wallwiener5Wolfgang Janni6Visnja Fink7Florian Ebner8Department of Gynaecology and Obstetrics, University Hospital Ulm, Prittwitzstr. 43, 89075 Ulm, GermanyDepartment of Gynaecology and Obstetrics, University Hospital Marburg, Baldingerstraße, 35043 Marburg, GermanyDepartment of Gynaecology and Obstetrics, University Hospital Ulm, Prittwitzstr. 43, 89075 Ulm, GermanyDepartment of Gynaecology and Obstetrics, University Hospital Ulm, Prittwitzstr. 43, 89075 Ulm, GermanyDepartment of Gynaecology and Obstetrics, University Hospital Ulm, Prittwitzstr. 43, 89075 Ulm, GermanyUniversity Clinic and Polyclinic for Gynaecology, University Hosptial Halle (Saale), Ernst-Grube-Straße 40, 06120 Halle (Saale), GermanyDepartment of Gynaecology and Obstetrics, University Hospital Ulm, Prittwitzstr. 43, 89075 Ulm, GermanyDepartment of Gynaecology and Obstetrics, University Hospital Ulm, Prittwitzstr. 43, 89075 Ulm, GermanyDepartment of Gynaecology and Obstetrics, University Hospital Ulm, Prittwitzstr. 43, 89075 Ulm, Germany<b>Introduction:</b> The survival in early breast cancer (BC) has been significantly improved thanks to numerous new drugs. Nevertheless, the information about the need for systemic therapy, especially chemotherapy, represents an additional stress factor for patients. A common coping strategy is searching for further information, traditionally via search engines or websites, but artificial intelligence (AI) is also increasingly being used. Who provides the most reliable information is now unclear. <b>Material and Methods</b>: AI in the form of ChatGPT 3.5 and 4.0, Google, and the website of PINK, a provider of a prescription-based mobile health app for patients with BC, were compared to determine the validity of the statements on the five most common side effects of nineteen approved drugs and one drug with pending approval (Ribociclib) for the systemic treatment of BC. For this purpose, the drugs were divided into three groups: chemotherapy, targeted therapy, and endocrine therapy. The reference for the comparison was the prescribing information of the respective drug. A congruence score was calculated for the information on side effects: correct information (2 points), generally appropriate information (1 point), and otherwise no point. The information sources were then compared using a Friedmann test and a Bonferroni-corrected post-hoc test. <b>Results:</b> In the overall comparison, ChatGPT 3.5 received the best score with a congruence of 67.5%, followed by ChatGPT 4.0 with 67.0%, PINK with 59.5%, and with Google 40.0% (<i>p</i> < 0.001). There were also significant differences when comparing the individual subcategories, with the best congruence achieved by PINK (73.3%, <i>p</i> = 0.059) in the chemotherapy category, ChatGPT 4.0 (77.5%; <i>p</i> < 0.001) in the targeted therapy category, and ChatGPT 3.5 (<i>p</i> = 0.002) in the endocrine therapy category. <b>Conclusions:</b> Artificial intelligence and professional online information websites provide the most reliable information on the possible side effects of the systemic treatment of early breast cancer, but congruence with prescribing information is limited. The medical consultation should still be considered the best source of information.https://www.mdpi.com/2039-7283/15/1/8artificial intelligenceChatGPTGooglePINKbreast cancerside effects |
| spellingShingle | Stefan Lukac Sebastian Griewing Elena Leinert Davut Dayan Benedikt Heitmeir Markus Wallwiener Wolfgang Janni Visnja Fink Florian Ebner ChatGPT, Google, or PINK? Who Provides the Most Reliable Information on Side Effects of Systemic Therapy for Early Breast Cancer? Clinics and Practice artificial intelligence ChatGPT PINK breast cancer side effects |
| title | ChatGPT, Google, or PINK? Who Provides the Most Reliable Information on Side Effects of Systemic Therapy for Early Breast Cancer? |
| title_full | ChatGPT, Google, or PINK? Who Provides the Most Reliable Information on Side Effects of Systemic Therapy for Early Breast Cancer? |
| title_fullStr | ChatGPT, Google, or PINK? Who Provides the Most Reliable Information on Side Effects of Systemic Therapy for Early Breast Cancer? |
| title_full_unstemmed | ChatGPT, Google, or PINK? Who Provides the Most Reliable Information on Side Effects of Systemic Therapy for Early Breast Cancer? |
| title_short | ChatGPT, Google, or PINK? Who Provides the Most Reliable Information on Side Effects of Systemic Therapy for Early Breast Cancer? |
| title_sort | chatgpt google or pink who provides the most reliable information on side effects of systemic therapy for early breast cancer |
| topic | artificial intelligence ChatGPT PINK breast cancer side effects |
| url | https://www.mdpi.com/2039-7283/15/1/8 |
| work_keys_str_mv | AT stefanlukac chatgptgoogleorpinkwhoprovidesthemostreliableinformationonsideeffectsofsystemictherapyforearlybreastcancer AT sebastiangriewing chatgptgoogleorpinkwhoprovidesthemostreliableinformationonsideeffectsofsystemictherapyforearlybreastcancer AT elenaleinert chatgptgoogleorpinkwhoprovidesthemostreliableinformationonsideeffectsofsystemictherapyforearlybreastcancer AT davutdayan chatgptgoogleorpinkwhoprovidesthemostreliableinformationonsideeffectsofsystemictherapyforearlybreastcancer AT benediktheitmeir chatgptgoogleorpinkwhoprovidesthemostreliableinformationonsideeffectsofsystemictherapyforearlybreastcancer AT markuswallwiener chatgptgoogleorpinkwhoprovidesthemostreliableinformationonsideeffectsofsystemictherapyforearlybreastcancer AT wolfgangjanni chatgptgoogleorpinkwhoprovidesthemostreliableinformationonsideeffectsofsystemictherapyforearlybreastcancer AT visnjafink chatgptgoogleorpinkwhoprovidesthemostreliableinformationonsideeffectsofsystemictherapyforearlybreastcancer AT florianebner chatgptgoogleorpinkwhoprovidesthemostreliableinformationonsideeffectsofsystemictherapyforearlybreastcancer |