A comparison of CXR-CAD software to radiologists in identifying COVID-19 in individuals evaluated for Sars CoV-2 infection in Malawi and Zambia.
AI based software, including computer aided detection software for chest radiographs (CXR-CAD), was developed during the pandemic to improve COVID-19 case finding and triage. In high burden TB countries, the use of highly portable CXR and computer aided detection software has been adopted more broad...
Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2025-01-01
|
| Series: | PLOS Digital Health |
| Online Access: | https://doi.org/10.1371/journal.pdig.0000535 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832539989634711552 |
|---|---|
| author | Sam Linsen Aurélie Kamoun Andrews Gunda Tamara Mwenifumbo Chancy Chavula Lindiwe Nchimunya Yucheng Tsai Namwaka Mulenga Godfrey Kadewele Eunice Nahache Kajombo Veronica Sunkutu Jane Shawa Rigveda Kadam Matthew Arentz |
| author_facet | Sam Linsen Aurélie Kamoun Andrews Gunda Tamara Mwenifumbo Chancy Chavula Lindiwe Nchimunya Yucheng Tsai Namwaka Mulenga Godfrey Kadewele Eunice Nahache Kajombo Veronica Sunkutu Jane Shawa Rigveda Kadam Matthew Arentz |
| author_sort | Sam Linsen |
| collection | DOAJ |
| description | AI based software, including computer aided detection software for chest radiographs (CXR-CAD), was developed during the pandemic to improve COVID-19 case finding and triage. In high burden TB countries, the use of highly portable CXR and computer aided detection software has been adopted more broadly to improve the screening and triage of individuals for TB, but there is little evidence in these settings regarding COVID-19 CAD performance. We performed a multicenter, retrospective cross-over study evaluating CXRs from individuals at risk for COVID-19. We evaluated performance of CAD software and radiologists in comparison to COVID-19 laboratory results in 671 individuals evaluated for COVID-19 at sites in Zambia and Malawi between January 2021 and June 2022. All CXRs were interpreted by an expert radiologist and two commercially available COVID-19 CXR-CAD software. Radiologists interpreted CXRs for COVID-19 with a sensitivity of 73% (95% CI: 69%- 76%) and specificity of 49% (95% CI: 40%-58%). One CAD software (CAD2) showed performance in diagnosing COVID-19 that was comparable to that of radiologists, (AUC-ROC of 0.70 (95% CI: 0.65-0.75)), while a second (CAD1) showed inferior performance (AUC-ROC of 0.57 (95% CI: 0.52-0.63)). Agreement between CAD software and radiologists was moderate for diagnosing COVID-19, and agreement was very good in differentiating normal and abnormal CXRs in this high prevalent population. The study highlights the potential of CXR-CAD as a tool to support effective triage of individuals in Malawi and Zambia during the pandemic, particularly for distinguishing normal from abnormal CXRs. These findings suggest that while current AI-based diagnostics like CXR-CAD show promise, their effectiveness varies significantly. In order to better prepare for future pandemics, there is a need for representative training data to optimize performance in key populations, and ongoing data collection to maintain diagnostic accuracy, especially as new disease strains emerge. |
| format | Article |
| id | doaj-art-b7ccb52c19f34468afafcbb3f57dfc22 |
| institution | Kabale University |
| issn | 2767-3170 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | Public Library of Science (PLoS) |
| record_format | Article |
| series | PLOS Digital Health |
| spelling | doaj-art-b7ccb52c19f34468afafcbb3f57dfc222025-02-05T05:33:36ZengPublic Library of Science (PLoS)PLOS Digital Health2767-31702025-01-0141e000053510.1371/journal.pdig.0000535A comparison of CXR-CAD software to radiologists in identifying COVID-19 in individuals evaluated for Sars CoV-2 infection in Malawi and Zambia.Sam LinsenAurélie KamounAndrews GundaTamara MwenifumboChancy ChavulaLindiwe NchimunyaYucheng TsaiNamwaka MulengaGodfrey KadeweleEunice Nahache KajomboVeronica SunkutuJane ShawaRigveda KadamMatthew ArentzAI based software, including computer aided detection software for chest radiographs (CXR-CAD), was developed during the pandemic to improve COVID-19 case finding and triage. In high burden TB countries, the use of highly portable CXR and computer aided detection software has been adopted more broadly to improve the screening and triage of individuals for TB, but there is little evidence in these settings regarding COVID-19 CAD performance. We performed a multicenter, retrospective cross-over study evaluating CXRs from individuals at risk for COVID-19. We evaluated performance of CAD software and radiologists in comparison to COVID-19 laboratory results in 671 individuals evaluated for COVID-19 at sites in Zambia and Malawi between January 2021 and June 2022. All CXRs were interpreted by an expert radiologist and two commercially available COVID-19 CXR-CAD software. Radiologists interpreted CXRs for COVID-19 with a sensitivity of 73% (95% CI: 69%- 76%) and specificity of 49% (95% CI: 40%-58%). One CAD software (CAD2) showed performance in diagnosing COVID-19 that was comparable to that of radiologists, (AUC-ROC of 0.70 (95% CI: 0.65-0.75)), while a second (CAD1) showed inferior performance (AUC-ROC of 0.57 (95% CI: 0.52-0.63)). Agreement between CAD software and radiologists was moderate for diagnosing COVID-19, and agreement was very good in differentiating normal and abnormal CXRs in this high prevalent population. The study highlights the potential of CXR-CAD as a tool to support effective triage of individuals in Malawi and Zambia during the pandemic, particularly for distinguishing normal from abnormal CXRs. These findings suggest that while current AI-based diagnostics like CXR-CAD show promise, their effectiveness varies significantly. In order to better prepare for future pandemics, there is a need for representative training data to optimize performance in key populations, and ongoing data collection to maintain diagnostic accuracy, especially as new disease strains emerge.https://doi.org/10.1371/journal.pdig.0000535 |
| spellingShingle | Sam Linsen Aurélie Kamoun Andrews Gunda Tamara Mwenifumbo Chancy Chavula Lindiwe Nchimunya Yucheng Tsai Namwaka Mulenga Godfrey Kadewele Eunice Nahache Kajombo Veronica Sunkutu Jane Shawa Rigveda Kadam Matthew Arentz A comparison of CXR-CAD software to radiologists in identifying COVID-19 in individuals evaluated for Sars CoV-2 infection in Malawi and Zambia. PLOS Digital Health |
| title | A comparison of CXR-CAD software to radiologists in identifying COVID-19 in individuals evaluated for Sars CoV-2 infection in Malawi and Zambia. |
| title_full | A comparison of CXR-CAD software to radiologists in identifying COVID-19 in individuals evaluated for Sars CoV-2 infection in Malawi and Zambia. |
| title_fullStr | A comparison of CXR-CAD software to radiologists in identifying COVID-19 in individuals evaluated for Sars CoV-2 infection in Malawi and Zambia. |
| title_full_unstemmed | A comparison of CXR-CAD software to radiologists in identifying COVID-19 in individuals evaluated for Sars CoV-2 infection in Malawi and Zambia. |
| title_short | A comparison of CXR-CAD software to radiologists in identifying COVID-19 in individuals evaluated for Sars CoV-2 infection in Malawi and Zambia. |
| title_sort | comparison of cxr cad software to radiologists in identifying covid 19 in individuals evaluated for sars cov 2 infection in malawi and zambia |
| url | https://doi.org/10.1371/journal.pdig.0000535 |
| work_keys_str_mv | AT samlinsen acomparisonofcxrcadsoftwaretoradiologistsinidentifyingcovid19inindividualsevaluatedforsarscov2infectioninmalawiandzambia AT aureliekamoun acomparisonofcxrcadsoftwaretoradiologistsinidentifyingcovid19inindividualsevaluatedforsarscov2infectioninmalawiandzambia AT andrewsgunda acomparisonofcxrcadsoftwaretoradiologistsinidentifyingcovid19inindividualsevaluatedforsarscov2infectioninmalawiandzambia AT tamaramwenifumbo acomparisonofcxrcadsoftwaretoradiologistsinidentifyingcovid19inindividualsevaluatedforsarscov2infectioninmalawiandzambia AT chancychavula acomparisonofcxrcadsoftwaretoradiologistsinidentifyingcovid19inindividualsevaluatedforsarscov2infectioninmalawiandzambia AT lindiwenchimunya acomparisonofcxrcadsoftwaretoradiologistsinidentifyingcovid19inindividualsevaluatedforsarscov2infectioninmalawiandzambia AT yuchengtsai acomparisonofcxrcadsoftwaretoradiologistsinidentifyingcovid19inindividualsevaluatedforsarscov2infectioninmalawiandzambia AT namwakamulenga acomparisonofcxrcadsoftwaretoradiologistsinidentifyingcovid19inindividualsevaluatedforsarscov2infectioninmalawiandzambia AT godfreykadewele acomparisonofcxrcadsoftwaretoradiologistsinidentifyingcovid19inindividualsevaluatedforsarscov2infectioninmalawiandzambia AT eunicenahachekajombo acomparisonofcxrcadsoftwaretoradiologistsinidentifyingcovid19inindividualsevaluatedforsarscov2infectioninmalawiandzambia AT veronicasunkutu acomparisonofcxrcadsoftwaretoradiologistsinidentifyingcovid19inindividualsevaluatedforsarscov2infectioninmalawiandzambia AT janeshawa acomparisonofcxrcadsoftwaretoradiologistsinidentifyingcovid19inindividualsevaluatedforsarscov2infectioninmalawiandzambia AT rigvedakadam acomparisonofcxrcadsoftwaretoradiologistsinidentifyingcovid19inindividualsevaluatedforsarscov2infectioninmalawiandzambia AT matthewarentz acomparisonofcxrcadsoftwaretoradiologistsinidentifyingcovid19inindividualsevaluatedforsarscov2infectioninmalawiandzambia AT samlinsen comparisonofcxrcadsoftwaretoradiologistsinidentifyingcovid19inindividualsevaluatedforsarscov2infectioninmalawiandzambia AT aureliekamoun comparisonofcxrcadsoftwaretoradiologistsinidentifyingcovid19inindividualsevaluatedforsarscov2infectioninmalawiandzambia AT andrewsgunda comparisonofcxrcadsoftwaretoradiologistsinidentifyingcovid19inindividualsevaluatedforsarscov2infectioninmalawiandzambia AT tamaramwenifumbo comparisonofcxrcadsoftwaretoradiologistsinidentifyingcovid19inindividualsevaluatedforsarscov2infectioninmalawiandzambia AT chancychavula comparisonofcxrcadsoftwaretoradiologistsinidentifyingcovid19inindividualsevaluatedforsarscov2infectioninmalawiandzambia AT lindiwenchimunya comparisonofcxrcadsoftwaretoradiologistsinidentifyingcovid19inindividualsevaluatedforsarscov2infectioninmalawiandzambia AT yuchengtsai comparisonofcxrcadsoftwaretoradiologistsinidentifyingcovid19inindividualsevaluatedforsarscov2infectioninmalawiandzambia AT namwakamulenga comparisonofcxrcadsoftwaretoradiologistsinidentifyingcovid19inindividualsevaluatedforsarscov2infectioninmalawiandzambia AT godfreykadewele comparisonofcxrcadsoftwaretoradiologistsinidentifyingcovid19inindividualsevaluatedforsarscov2infectioninmalawiandzambia AT eunicenahachekajombo comparisonofcxrcadsoftwaretoradiologistsinidentifyingcovid19inindividualsevaluatedforsarscov2infectioninmalawiandzambia AT veronicasunkutu comparisonofcxrcadsoftwaretoradiologistsinidentifyingcovid19inindividualsevaluatedforsarscov2infectioninmalawiandzambia AT janeshawa comparisonofcxrcadsoftwaretoradiologistsinidentifyingcovid19inindividualsevaluatedforsarscov2infectioninmalawiandzambia AT rigvedakadam comparisonofcxrcadsoftwaretoradiologistsinidentifyingcovid19inindividualsevaluatedforsarscov2infectioninmalawiandzambia AT matthewarentz comparisonofcxrcadsoftwaretoradiologistsinidentifyingcovid19inindividualsevaluatedforsarscov2infectioninmalawiandzambia |