Enhancing Diagnostic Accuracy of Lung Nodules in Chest Computed Tomography Using Artificial Intelligence: Retrospective Analysis
BackgroundUncertainty in the diagnosis of lung nodules is a challenge for both patients and physicians. Artificial intelligence (AI) systems are increasingly being integrated into medical imaging to assist diagnostic procedures. However, the accuracy of AI systems in identify...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2025-01-01
|
| Series: | Journal of Medical Internet Research |
| Online Access: | https://www.jmir.org/2025/1/e64649 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832584058673037312 |
|---|---|
| author | Weiqi Liu You Wu Zhuozhao Zheng Mark Bittle Wei Yu Hadi Kharrazi |
| author_facet | Weiqi Liu You Wu Zhuozhao Zheng Mark Bittle Wei Yu Hadi Kharrazi |
| author_sort | Weiqi Liu |
| collection | DOAJ |
| description |
BackgroundUncertainty in the diagnosis of lung nodules is a challenge for both patients and physicians. Artificial intelligence (AI) systems are increasingly being integrated into medical imaging to assist diagnostic procedures. However, the accuracy of AI systems in identifying and measuring lung nodules on chest computed tomography (CT) scans remains unclear, which requires further evaluation.
ObjectiveThis study aimed to evaluate the impact of an AI-assisted diagnostic system on the diagnostic efficiency of radiologists. It specifically examined the report modification rates and missed and misdiagnosed rates of junior radiologists with and without AI assistance.
MethodsWe obtained effective data from 12,889 patients in 2 tertiary hospitals in Beijing before and after the implementation of the AI system, covering the period from April 2018 to March 2022. Diagnostic reports written by both junior and senior radiologists were included in each case. Using reports by senior radiologists as a reference, we compared the modification rates of reports written by junior radiologists with and without AI assistance. We further evaluated alterations in lung nodule detection capability over 3 years after the integration of the AI system. Evaluation metrics of this study include lung nodule detection rate, accuracy, false negative rate, false positive rate, and positive predictive value. The statistical analyses included descriptive statistics and chi-square, Cochran-Armitage, and Mann-Kendall tests.
ResultsThe AI system was implemented in Beijing Anzhen Hospital (Hospital A) in January 2019 and Tsinghua Changgung Hospital (Hospital C) in June 2021. The modification rate of diagnostic reports in the detection of lung nodules increased from 4.73% to 7.23% (χ21=12.15; P<.001) at Hospital A. In terms of lung nodule detection rates postimplementation, Hospital C increased from 46.19% to 53.45% (χ21=25.48; P<.001) and Hospital A increased from 39.29% to 55.22% (χ21=122.55; P<.001). At Hospital A, the false negative rate decreased from 8.4% to 5.16% (χ21=9.85; P=.002), while the false positive rate increased from 2.36% to 9.77% (χ21=53.48; P<.001). The detection accuracy demonstrated a decrease from 93.33% to 92.23% for Hospital A and from 95.27% to 92.77% for Hospital C. Regarding the changes in lung nodule detection capability over a 3-year period following the integration of the AI system, the detection rates for lung nodules exhibited a modest increase from 54.6% to 55.84%, while the overall accuracy demonstrated a slight improvement from 92.79% to 93.92%.
ConclusionsThe AI system enhanced lung nodule detection, offering the possibility of earlier disease identification and timely intervention. Nevertheless, the initial reduction in accuracy underscores the need for standardized diagnostic criteria and comprehensive training for radiologists to maximize the effectiveness of AI-enabled diagnostic systems. |
| format | Article |
| id | doaj-art-3cb756ffe7e74b1dba1dc4eed2e8fa73 |
| institution | Kabale University |
| issn | 1438-8871 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | Journal of Medical Internet Research |
| spelling | doaj-art-3cb756ffe7e74b1dba1dc4eed2e8fa732025-01-27T17:00:36ZengJMIR PublicationsJournal of Medical Internet Research1438-88712025-01-0127e6464910.2196/64649Enhancing Diagnostic Accuracy of Lung Nodules in Chest Computed Tomography Using Artificial Intelligence: Retrospective AnalysisWeiqi Liuhttps://orcid.org/0009-0002-6221-7136You Wuhttps://orcid.org/0000-0001-9672-6129Zhuozhao Zhenghttps://orcid.org/0000-0001-8547-6670Mark Bittlehttps://orcid.org/0000-0001-7258-4445Wei Yuhttps://orcid.org/0009-0007-2896-6236Hadi Kharrazihttps://orcid.org/0000-0003-1481-4323 BackgroundUncertainty in the diagnosis of lung nodules is a challenge for both patients and physicians. Artificial intelligence (AI) systems are increasingly being integrated into medical imaging to assist diagnostic procedures. However, the accuracy of AI systems in identifying and measuring lung nodules on chest computed tomography (CT) scans remains unclear, which requires further evaluation. ObjectiveThis study aimed to evaluate the impact of an AI-assisted diagnostic system on the diagnostic efficiency of radiologists. It specifically examined the report modification rates and missed and misdiagnosed rates of junior radiologists with and without AI assistance. MethodsWe obtained effective data from 12,889 patients in 2 tertiary hospitals in Beijing before and after the implementation of the AI system, covering the period from April 2018 to March 2022. Diagnostic reports written by both junior and senior radiologists were included in each case. Using reports by senior radiologists as a reference, we compared the modification rates of reports written by junior radiologists with and without AI assistance. We further evaluated alterations in lung nodule detection capability over 3 years after the integration of the AI system. Evaluation metrics of this study include lung nodule detection rate, accuracy, false negative rate, false positive rate, and positive predictive value. The statistical analyses included descriptive statistics and chi-square, Cochran-Armitage, and Mann-Kendall tests. ResultsThe AI system was implemented in Beijing Anzhen Hospital (Hospital A) in January 2019 and Tsinghua Changgung Hospital (Hospital C) in June 2021. The modification rate of diagnostic reports in the detection of lung nodules increased from 4.73% to 7.23% (χ21=12.15; P<.001) at Hospital A. In terms of lung nodule detection rates postimplementation, Hospital C increased from 46.19% to 53.45% (χ21=25.48; P<.001) and Hospital A increased from 39.29% to 55.22% (χ21=122.55; P<.001). At Hospital A, the false negative rate decreased from 8.4% to 5.16% (χ21=9.85; P=.002), while the false positive rate increased from 2.36% to 9.77% (χ21=53.48; P<.001). The detection accuracy demonstrated a decrease from 93.33% to 92.23% for Hospital A and from 95.27% to 92.77% for Hospital C. Regarding the changes in lung nodule detection capability over a 3-year period following the integration of the AI system, the detection rates for lung nodules exhibited a modest increase from 54.6% to 55.84%, while the overall accuracy demonstrated a slight improvement from 92.79% to 93.92%. ConclusionsThe AI system enhanced lung nodule detection, offering the possibility of earlier disease identification and timely intervention. Nevertheless, the initial reduction in accuracy underscores the need for standardized diagnostic criteria and comprehensive training for radiologists to maximize the effectiveness of AI-enabled diagnostic systems.https://www.jmir.org/2025/1/e64649 |
| spellingShingle | Weiqi Liu You Wu Zhuozhao Zheng Mark Bittle Wei Yu Hadi Kharrazi Enhancing Diagnostic Accuracy of Lung Nodules in Chest Computed Tomography Using Artificial Intelligence: Retrospective Analysis Journal of Medical Internet Research |
| title | Enhancing Diagnostic Accuracy of Lung Nodules in Chest Computed Tomography Using Artificial Intelligence: Retrospective Analysis |
| title_full | Enhancing Diagnostic Accuracy of Lung Nodules in Chest Computed Tomography Using Artificial Intelligence: Retrospective Analysis |
| title_fullStr | Enhancing Diagnostic Accuracy of Lung Nodules in Chest Computed Tomography Using Artificial Intelligence: Retrospective Analysis |
| title_full_unstemmed | Enhancing Diagnostic Accuracy of Lung Nodules in Chest Computed Tomography Using Artificial Intelligence: Retrospective Analysis |
| title_short | Enhancing Diagnostic Accuracy of Lung Nodules in Chest Computed Tomography Using Artificial Intelligence: Retrospective Analysis |
| title_sort | enhancing diagnostic accuracy of lung nodules in chest computed tomography using artificial intelligence retrospective analysis |
| url | https://www.jmir.org/2025/1/e64649 |
| work_keys_str_mv | AT weiqiliu enhancingdiagnosticaccuracyoflungnodulesinchestcomputedtomographyusingartificialintelligenceretrospectiveanalysis AT youwu enhancingdiagnosticaccuracyoflungnodulesinchestcomputedtomographyusingartificialintelligenceretrospectiveanalysis AT zhuozhaozheng enhancingdiagnosticaccuracyoflungnodulesinchestcomputedtomographyusingartificialintelligenceretrospectiveanalysis AT markbittle enhancingdiagnosticaccuracyoflungnodulesinchestcomputedtomographyusingartificialintelligenceretrospectiveanalysis AT weiyu enhancingdiagnosticaccuracyoflungnodulesinchestcomputedtomographyusingartificialintelligenceretrospectiveanalysis AT hadikharrazi enhancingdiagnosticaccuracyoflungnodulesinchestcomputedtomographyusingartificialintelligenceretrospectiveanalysis |