Harnessing GPT-4 for automated error detection in pathology reports: Implications for oncology diagnostics
Objective Accurate pathology reports are crucial for the diagnosis and treatment planning of cancer patients. However, these reports are prone to errors due to time pressures, subjective interpretation, and inconsistencies among professionals. Addressing these errors is vital for improving oncology...
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SAGE Publishing
2025-05-01
|
| Series: | Digital Health |
| Online Access: | https://doi.org/10.1177/20552076251346703 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850123067317026816 |
|---|---|
| author | Xiongwen Yang Yun Zhang Jinyan Jiang Zhijun Chen Rinasu Bai Zihao Yuan Longyan Dong Yi Xiao Di Liu Huiyin Deng Jian Huang Huiyou Shi Dan Liu Maoli Liang WeiJuan Tang Chuan Xu |
| author_facet | Xiongwen Yang Yun Zhang Jinyan Jiang Zhijun Chen Rinasu Bai Zihao Yuan Longyan Dong Yi Xiao Di Liu Huiyin Deng Jian Huang Huiyou Shi Dan Liu Maoli Liang WeiJuan Tang Chuan Xu |
| author_sort | Xiongwen Yang |
| collection | DOAJ |
| description | Objective Accurate pathology reports are crucial for the diagnosis and treatment planning of cancer patients. However, these reports are prone to errors due to time pressures, subjective interpretation, and inconsistencies among professionals. Addressing these errors is vital for improving oncology care outcomes. Artificial intelligence (AI) systems, such as GPT-4, offer the potential to enhance diagnostic accuracy and efficiency. Methods A total of 700 malignant tumor pathology reports were collected from four hospitals. Of these, 350 reports had deliberate errors introduced by a senior pathologist, mimicking real-world reporting challenges. Error detection performance was evaluated by comparing GPT-4 to six human pathologists (two seniors, two attending pathologists, and two residents). Key metrics included error detection rates with Wilson confidence intervals and processing time per report. Results GPT-4 detected 88% of errors (350/400; 95% CI: [84, 91]), compared to a 95% detection rate by the top senior pathologist (382/400; 95% CI: [93, 97]). GPT-4 significantly reduced the average processing time to 4.03 seconds per report, compared to 65.64 seconds for the fastest human pathologist. However, GPT-4 exhibited a higher rate of false positives (2.3%; 95% CI: [1.52, 3.01]) compared to the best-performing senior pathologist (0.3%; 95% CI: [0.01, 0.91]). Conclusions GPT-4 demonstrates substantial potential in improving the efficiency and accuracy of pathology error detection, which could accelerate clinical workflows and enhance cancer diagnostics. However, its higher false-positive rate emphasizes the need for human oversight to ensure safe implementation in clinical practice. |
| format | Article |
| id | doaj-art-f2bc26cf86b8428b991761c6ce2cb78f |
| institution | OA Journals |
| issn | 2055-2076 |
| language | English |
| publishDate | 2025-05-01 |
| publisher | SAGE Publishing |
| record_format | Article |
| series | Digital Health |
| spelling | doaj-art-f2bc26cf86b8428b991761c6ce2cb78f2025-08-20T02:34:42ZengSAGE PublishingDigital Health2055-20762025-05-011110.1177/20552076251346703Harnessing GPT-4 for automated error detection in pathology reports: Implications for oncology diagnosticsXiongwen Yang0Yun Zhang1Jinyan Jiang2Zhijun Chen3Rinasu Bai4Zihao Yuan5Longyan Dong6Yi Xiao7Di Liu8Huiyin Deng9Jian Huang10Huiyou Shi11Dan Liu12Maoli Liang13WeiJuan Tang14Chuan Xu15 NHC Key Laboratory of Pulmonary Immunological Diseases, , Guiyang, Guizhou, China Department of Pathology, , Guiyang, Guizhou, China Department of Pathology, , Chenzhou, Hunan, China Department of Pathology, , Chenzhou, Hunan, China Department of Pathology, , Beijing, China The Second Clinical Medical College, , Dongguan, Guangdong, China The Second Clinical Medical College, , Dongguan, Guangdong, China Department of Cardio-Thoracic Surgery, , Guangzhou, Guangdong, China NHC Key Laboratory of Pulmonary Immunological Diseases, , Guiyang, Guizhou, China Department of Anesthesiology, , Changsha, Hunan, China Department of Thoracic Surgery, , Nanchang, Jiangxi, China Department of Radiology, , Guiyang, Guizhou, China Department of Medical Records and Statistics, , Guiyang, Guizhou, China Department of Respiratory Medicine, , Guiyang, Guizhou, China NHC Key Laboratory of Pulmonary Immunological Diseases, , Guiyang, Guizhou, China NHC Key Laboratory of Pulmonary Immunological Diseases, , Guiyang, Guizhou, ChinaObjective Accurate pathology reports are crucial for the diagnosis and treatment planning of cancer patients. However, these reports are prone to errors due to time pressures, subjective interpretation, and inconsistencies among professionals. Addressing these errors is vital for improving oncology care outcomes. Artificial intelligence (AI) systems, such as GPT-4, offer the potential to enhance diagnostic accuracy and efficiency. Methods A total of 700 malignant tumor pathology reports were collected from four hospitals. Of these, 350 reports had deliberate errors introduced by a senior pathologist, mimicking real-world reporting challenges. Error detection performance was evaluated by comparing GPT-4 to six human pathologists (two seniors, two attending pathologists, and two residents). Key metrics included error detection rates with Wilson confidence intervals and processing time per report. Results GPT-4 detected 88% of errors (350/400; 95% CI: [84, 91]), compared to a 95% detection rate by the top senior pathologist (382/400; 95% CI: [93, 97]). GPT-4 significantly reduced the average processing time to 4.03 seconds per report, compared to 65.64 seconds for the fastest human pathologist. However, GPT-4 exhibited a higher rate of false positives (2.3%; 95% CI: [1.52, 3.01]) compared to the best-performing senior pathologist (0.3%; 95% CI: [0.01, 0.91]). Conclusions GPT-4 demonstrates substantial potential in improving the efficiency and accuracy of pathology error detection, which could accelerate clinical workflows and enhance cancer diagnostics. However, its higher false-positive rate emphasizes the need for human oversight to ensure safe implementation in clinical practice.https://doi.org/10.1177/20552076251346703 |
| spellingShingle | Xiongwen Yang Yun Zhang Jinyan Jiang Zhijun Chen Rinasu Bai Zihao Yuan Longyan Dong Yi Xiao Di Liu Huiyin Deng Jian Huang Huiyou Shi Dan Liu Maoli Liang WeiJuan Tang Chuan Xu Harnessing GPT-4 for automated error detection in pathology reports: Implications for oncology diagnostics Digital Health |
| title | Harnessing GPT-4 for automated error detection in pathology reports: Implications for oncology diagnostics |
| title_full | Harnessing GPT-4 for automated error detection in pathology reports: Implications for oncology diagnostics |
| title_fullStr | Harnessing GPT-4 for automated error detection in pathology reports: Implications for oncology diagnostics |
| title_full_unstemmed | Harnessing GPT-4 for automated error detection in pathology reports: Implications for oncology diagnostics |
| title_short | Harnessing GPT-4 for automated error detection in pathology reports: Implications for oncology diagnostics |
| title_sort | harnessing gpt 4 for automated error detection in pathology reports implications for oncology diagnostics |
| url | https://doi.org/10.1177/20552076251346703 |
| work_keys_str_mv | AT xiongwenyang harnessinggpt4forautomatederrordetectioninpathologyreportsimplicationsforoncologydiagnostics AT yunzhang harnessinggpt4forautomatederrordetectioninpathologyreportsimplicationsforoncologydiagnostics AT jinyanjiang harnessinggpt4forautomatederrordetectioninpathologyreportsimplicationsforoncologydiagnostics AT zhijunchen harnessinggpt4forautomatederrordetectioninpathologyreportsimplicationsforoncologydiagnostics AT rinasubai harnessinggpt4forautomatederrordetectioninpathologyreportsimplicationsforoncologydiagnostics AT zihaoyuan harnessinggpt4forautomatederrordetectioninpathologyreportsimplicationsforoncologydiagnostics AT longyandong harnessinggpt4forautomatederrordetectioninpathologyreportsimplicationsforoncologydiagnostics AT yixiao harnessinggpt4forautomatederrordetectioninpathologyreportsimplicationsforoncologydiagnostics AT diliu harnessinggpt4forautomatederrordetectioninpathologyreportsimplicationsforoncologydiagnostics AT huiyindeng harnessinggpt4forautomatederrordetectioninpathologyreportsimplicationsforoncologydiagnostics AT jianhuang harnessinggpt4forautomatederrordetectioninpathologyreportsimplicationsforoncologydiagnostics AT huiyoushi harnessinggpt4forautomatederrordetectioninpathologyreportsimplicationsforoncologydiagnostics AT danliu harnessinggpt4forautomatederrordetectioninpathologyreportsimplicationsforoncologydiagnostics AT maoliliang harnessinggpt4forautomatederrordetectioninpathologyreportsimplicationsforoncologydiagnostics AT weijuantang harnessinggpt4forautomatederrordetectioninpathologyreportsimplicationsforoncologydiagnostics AT chuanxu harnessinggpt4forautomatederrordetectioninpathologyreportsimplicationsforoncologydiagnostics |