Evaluating AI performance in nephrology triage and subspecialty referrals
Abstract Artificial intelligence (AI) has shown promise in revolutionizing medical triage, particularly in the context of the rising prevalence of kidney-related conditions with the aging global population. This study evaluates the utility of ChatGPT, a large language model, in triaging nephrology c...
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-01-01
|
| Series: | Scientific Reports |
| Subjects: | |
| Online Access: | https://doi.org/10.1038/s41598-025-88074-5 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832571655717650432 |
|---|---|
| author | Priscilla Koirala Charat Thongprayoon Jing Miao Oscar A. Garcia Valencia Mohammad S. Sheikh Supawadee Suppadungsuk Michael A. Mao Justin H. Pham Iasmina M. Craici Wisit Cheungpasitporn |
| author_facet | Priscilla Koirala Charat Thongprayoon Jing Miao Oscar A. Garcia Valencia Mohammad S. Sheikh Supawadee Suppadungsuk Michael A. Mao Justin H. Pham Iasmina M. Craici Wisit Cheungpasitporn |
| author_sort | Priscilla Koirala |
| collection | DOAJ |
| description | Abstract Artificial intelligence (AI) has shown promise in revolutionizing medical triage, particularly in the context of the rising prevalence of kidney-related conditions with the aging global population. This study evaluates the utility of ChatGPT, a large language model, in triaging nephrology cases through simulated real-world scenarios. Two nephrologists created 100 patient cases that encompassed various aspects of nephrology. ChatGPT’s performance in determining the appropriateness of nephrology consultations and identifying suitable nephrology subspecialties was assessed. The results demonstrated high accuracy; ChatGPT correctly determined the need for nephrology in 99–100% of cases, and it accurately identified the most suitable nephrology subspecialty triage in 96–99% of cases across two evaluation rounds. The agreement between the two rounds was 97%. While ChatGPT showed promise in improving medical triage efficiency and accuracy, the study also identified areas for refinement. This included the need for better integration of multidisciplinary care for patients with complex, intersecting medical conditions. This study’s findings highlight the potential of AI in enhancing decision-making processes in clinical workflow, and it can inform the development of AI-assisted triage systems tailored to institution-specific practices including multidisciplinary approaches. |
| format | Article |
| id | doaj-art-5985b9a30ec34c2e9ca73db1bb7d6104 |
| institution | Kabale University |
| issn | 2045-2322 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj-art-5985b9a30ec34c2e9ca73db1bb7d61042025-02-02T12:24:36ZengNature PortfolioScientific Reports2045-23222025-01-011511910.1038/s41598-025-88074-5Evaluating AI performance in nephrology triage and subspecialty referralsPriscilla Koirala0Charat Thongprayoon1Jing Miao2Oscar A. Garcia Valencia3Mohammad S. Sheikh4Supawadee Suppadungsuk5Michael A. Mao6Justin H. Pham7Iasmina M. Craici8Wisit Cheungpasitporn9Internal Medicine, Mayo ClinicDivision of Nephrology and Hypertension, Mayo ClinicDivision of Nephrology and Hypertension, Mayo ClinicDivision of Nephrology and Hypertension, Mayo ClinicDivision of Nephrology and Hypertension, Mayo ClinicDivision of Nephrology and Hypertension, Mayo ClinicDivision of Nephrology and Hypertension, Department of Medicine, Mayo ClinicInternal Medicine, Mayo ClinicDivision of Nephrology and Hypertension, Mayo ClinicDivision of Nephrology and Hypertension, Mayo ClinicAbstract Artificial intelligence (AI) has shown promise in revolutionizing medical triage, particularly in the context of the rising prevalence of kidney-related conditions with the aging global population. This study evaluates the utility of ChatGPT, a large language model, in triaging nephrology cases through simulated real-world scenarios. Two nephrologists created 100 patient cases that encompassed various aspects of nephrology. ChatGPT’s performance in determining the appropriateness of nephrology consultations and identifying suitable nephrology subspecialties was assessed. The results demonstrated high accuracy; ChatGPT correctly determined the need for nephrology in 99–100% of cases, and it accurately identified the most suitable nephrology subspecialty triage in 96–99% of cases across two evaluation rounds. The agreement between the two rounds was 97%. While ChatGPT showed promise in improving medical triage efficiency and accuracy, the study also identified areas for refinement. This included the need for better integration of multidisciplinary care for patients with complex, intersecting medical conditions. This study’s findings highlight the potential of AI in enhancing decision-making processes in clinical workflow, and it can inform the development of AI-assisted triage systems tailored to institution-specific practices including multidisciplinary approaches.https://doi.org/10.1038/s41598-025-88074-5Artificial IntelligenceChatGPTNephrology triageMedical triageIdentify subspecialty |
| spellingShingle | Priscilla Koirala Charat Thongprayoon Jing Miao Oscar A. Garcia Valencia Mohammad S. Sheikh Supawadee Suppadungsuk Michael A. Mao Justin H. Pham Iasmina M. Craici Wisit Cheungpasitporn Evaluating AI performance in nephrology triage and subspecialty referrals Scientific Reports Artificial Intelligence ChatGPT Nephrology triage Medical triage Identify subspecialty |
| title | Evaluating AI performance in nephrology triage and subspecialty referrals |
| title_full | Evaluating AI performance in nephrology triage and subspecialty referrals |
| title_fullStr | Evaluating AI performance in nephrology triage and subspecialty referrals |
| title_full_unstemmed | Evaluating AI performance in nephrology triage and subspecialty referrals |
| title_short | Evaluating AI performance in nephrology triage and subspecialty referrals |
| title_sort | evaluating ai performance in nephrology triage and subspecialty referrals |
| topic | Artificial Intelligence ChatGPT Nephrology triage Medical triage Identify subspecialty |
| url | https://doi.org/10.1038/s41598-025-88074-5 |
| work_keys_str_mv | AT priscillakoirala evaluatingaiperformanceinnephrologytriageandsubspecialtyreferrals AT charatthongprayoon evaluatingaiperformanceinnephrologytriageandsubspecialtyreferrals AT jingmiao evaluatingaiperformanceinnephrologytriageandsubspecialtyreferrals AT oscaragarciavalencia evaluatingaiperformanceinnephrologytriageandsubspecialtyreferrals AT mohammadssheikh evaluatingaiperformanceinnephrologytriageandsubspecialtyreferrals AT supawadeesuppadungsuk evaluatingaiperformanceinnephrologytriageandsubspecialtyreferrals AT michaelamao evaluatingaiperformanceinnephrologytriageandsubspecialtyreferrals AT justinhpham evaluatingaiperformanceinnephrologytriageandsubspecialtyreferrals AT iasminamcraici evaluatingaiperformanceinnephrologytriageandsubspecialtyreferrals AT wisitcheungpasitporn evaluatingaiperformanceinnephrologytriageandsubspecialtyreferrals |