Impact of uncertainty quantification through conformal prediction on volume assessment from deep learning-based MRI prostate segmentation
Abstract Objectives To estimate the uncertainty of a deep learning (DL)-based prostate segmentation algorithm through conformal prediction (CP) and to assess its effect on the calculation of the prostate volume (PV) in patients at risk of prostate cancer (PC). Methods Three-hundred seventy-seven mul...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SpringerOpen
2024-11-01
|
| Series: | Insights into Imaging |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s13244-024-01863-w |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850058051745218560 |
|---|---|
| author | Marius Gade Kevin Mekhaphan Nguyen Sol Gedde Alvaro Fernandez-Quilez |
| author_facet | Marius Gade Kevin Mekhaphan Nguyen Sol Gedde Alvaro Fernandez-Quilez |
| author_sort | Marius Gade |
| collection | DOAJ |
| description | Abstract Objectives To estimate the uncertainty of a deep learning (DL)-based prostate segmentation algorithm through conformal prediction (CP) and to assess its effect on the calculation of the prostate volume (PV) in patients at risk of prostate cancer (PC). Methods Three-hundred seventy-seven multi-center 3-Tesla axial T2-weighted exams from biopsied males (66.64 $$\pm$$ ± 7.47 years) at risk of PC were retrospectively included in the study. Assessment of PV based on PI-RADS 2.1 ellipsoid formula ( $${{{\rm{PV}}}}_{{ref}}$$ PV r e f ) was available for included patients. Prostate segmentations were obtained from a DL model and used to calculate the PV ( $${{{\rm{PV}}}}_{{DL}}$$ PV D L ). CP was applied at a confidence level of 85% to flag unreliable pixel segmentations of the DL model. Subsequently, the PV ( $${{{\rm{PV}}}}_{{CP}}$$ PV C P ) was calculated when disregarding uncertain pixel segmentations. Agreement between $${{{\rm{PV}}}}_{{DL}}$$ PV D L and $${{{\rm{PV}}}}_{{CP}}$$ PV C P was evaluated against the reference standard $${{{\rm{PV}}}}_{{ref}}$$ PV r e f . Intraclass correlation coefficient (ICC) and Bland–Altman plots were used to assess the agreement. The relative volume difference (RVD) was used to evaluate the PV calculation accuracy, and the Wilcoxon Signed-Rank Test was used to assess statistical differences. A p-value < 0.05 was considered statistically significant. Results Conformal prediction significantly reduced RVD when compared to the DL algorithm (RVD = − 2.81 $$\pm$$ ± 8.85 and RVD = −8.01 $$\pm$$ ± 11.50). $${{{\rm{PV}}}}_{{CP}}$$ PV C P showed a significantly larger agreement than $${{{\rm{PV}}}}_{{DL}}$$ PV D L when using the reference standard $${{{\rm{PV}}}}_{{ref}}$$ PV r e f (mean difference (95% limits of agreement) $${{{\rm{PV}}}}_{{CP}}$$ PV C P : 1.27 mL (− 13.64; 16.17 mL) $${{{\rm{PV}}}}_{{DL}}$$ PV D L : 6.07 mL (− 14.29; 26.42 mL)), with an excellent ICC ( $${{{\rm{PV}}}}_{{CP}}$$ PV C P : 0.97 (95% CI: 0.97 to 0.98)). Conclusion Uncertainty quantification through CP increases the accuracy and reliability of DL-based PV assessment in patients at risk of PC. Critical relevance statement Conformal prediction can flag uncertain pixel predictions of DL-based prostate MRI segmentation at a desired confidence level, increasing the reliability and safety of prostate volume assessment in patients at risk of prostate cancer. Key Points Conformal prediction can flag uncertain pixel predictions of prostate segmentations at a user-defined confidence level. Deep learning with conformal prediction shows high accuracy in prostate volumetric assessment. Agreement between automatic and ellipsoid-derived volume was significantly larger with conformal prediction. Graphical Abstract |
| format | Article |
| id | doaj-art-ab522de416e844ef9e6af9a45db59473 |
| institution | DOAJ |
| issn | 1869-4101 |
| language | English |
| publishDate | 2024-11-01 |
| publisher | SpringerOpen |
| record_format | Article |
| series | Insights into Imaging |
| spelling | doaj-art-ab522de416e844ef9e6af9a45db594732025-08-20T02:51:15ZengSpringerOpenInsights into Imaging1869-41012024-11-0115111310.1186/s13244-024-01863-wImpact of uncertainty quantification through conformal prediction on volume assessment from deep learning-based MRI prostate segmentationMarius Gade0Kevin Mekhaphan Nguyen1Sol Gedde2Alvaro Fernandez-Quilez3Department of Electrical Engineering and Computer Science, University of StavangerDepartment of Electrical Engineering and Computer Science, University of StavangerStavanger Medical Imaging Laboratory (SMIL), Department of Radiology, Stavanger University HospitalDepartment of Electrical Engineering and Computer Science, University of StavangerAbstract Objectives To estimate the uncertainty of a deep learning (DL)-based prostate segmentation algorithm through conformal prediction (CP) and to assess its effect on the calculation of the prostate volume (PV) in patients at risk of prostate cancer (PC). Methods Three-hundred seventy-seven multi-center 3-Tesla axial T2-weighted exams from biopsied males (66.64 $$\pm$$ ± 7.47 years) at risk of PC were retrospectively included in the study. Assessment of PV based on PI-RADS 2.1 ellipsoid formula ( $${{{\rm{PV}}}}_{{ref}}$$ PV r e f ) was available for included patients. Prostate segmentations were obtained from a DL model and used to calculate the PV ( $${{{\rm{PV}}}}_{{DL}}$$ PV D L ). CP was applied at a confidence level of 85% to flag unreliable pixel segmentations of the DL model. Subsequently, the PV ( $${{{\rm{PV}}}}_{{CP}}$$ PV C P ) was calculated when disregarding uncertain pixel segmentations. Agreement between $${{{\rm{PV}}}}_{{DL}}$$ PV D L and $${{{\rm{PV}}}}_{{CP}}$$ PV C P was evaluated against the reference standard $${{{\rm{PV}}}}_{{ref}}$$ PV r e f . Intraclass correlation coefficient (ICC) and Bland–Altman plots were used to assess the agreement. The relative volume difference (RVD) was used to evaluate the PV calculation accuracy, and the Wilcoxon Signed-Rank Test was used to assess statistical differences. A p-value < 0.05 was considered statistically significant. Results Conformal prediction significantly reduced RVD when compared to the DL algorithm (RVD = − 2.81 $$\pm$$ ± 8.85 and RVD = −8.01 $$\pm$$ ± 11.50). $${{{\rm{PV}}}}_{{CP}}$$ PV C P showed a significantly larger agreement than $${{{\rm{PV}}}}_{{DL}}$$ PV D L when using the reference standard $${{{\rm{PV}}}}_{{ref}}$$ PV r e f (mean difference (95% limits of agreement) $${{{\rm{PV}}}}_{{CP}}$$ PV C P : 1.27 mL (− 13.64; 16.17 mL) $${{{\rm{PV}}}}_{{DL}}$$ PV D L : 6.07 mL (− 14.29; 26.42 mL)), with an excellent ICC ( $${{{\rm{PV}}}}_{{CP}}$$ PV C P : 0.97 (95% CI: 0.97 to 0.98)). Conclusion Uncertainty quantification through CP increases the accuracy and reliability of DL-based PV assessment in patients at risk of PC. Critical relevance statement Conformal prediction can flag uncertain pixel predictions of DL-based prostate MRI segmentation at a desired confidence level, increasing the reliability and safety of prostate volume assessment in patients at risk of prostate cancer. Key Points Conformal prediction can flag uncertain pixel predictions of prostate segmentations at a user-defined confidence level. Deep learning with conformal prediction shows high accuracy in prostate volumetric assessment. Agreement between automatic and ellipsoid-derived volume was significantly larger with conformal prediction. Graphical Abstracthttps://doi.org/10.1186/s13244-024-01863-wMagnetic resonance imagingDeep learningProstateConformal predictionUncertainty |
| spellingShingle | Marius Gade Kevin Mekhaphan Nguyen Sol Gedde Alvaro Fernandez-Quilez Impact of uncertainty quantification through conformal prediction on volume assessment from deep learning-based MRI prostate segmentation Insights into Imaging Magnetic resonance imaging Deep learning Prostate Conformal prediction Uncertainty |
| title | Impact of uncertainty quantification through conformal prediction on volume assessment from deep learning-based MRI prostate segmentation |
| title_full | Impact of uncertainty quantification through conformal prediction on volume assessment from deep learning-based MRI prostate segmentation |
| title_fullStr | Impact of uncertainty quantification through conformal prediction on volume assessment from deep learning-based MRI prostate segmentation |
| title_full_unstemmed | Impact of uncertainty quantification through conformal prediction on volume assessment from deep learning-based MRI prostate segmentation |
| title_short | Impact of uncertainty quantification through conformal prediction on volume assessment from deep learning-based MRI prostate segmentation |
| title_sort | impact of uncertainty quantification through conformal prediction on volume assessment from deep learning based mri prostate segmentation |
| topic | Magnetic resonance imaging Deep learning Prostate Conformal prediction Uncertainty |
| url | https://doi.org/10.1186/s13244-024-01863-w |
| work_keys_str_mv | AT mariusgade impactofuncertaintyquantificationthroughconformalpredictiononvolumeassessmentfromdeeplearningbasedmriprostatesegmentation AT kevinmekhaphannguyen impactofuncertaintyquantificationthroughconformalpredictiononvolumeassessmentfromdeeplearningbasedmriprostatesegmentation AT solgedde impactofuncertaintyquantificationthroughconformalpredictiononvolumeassessmentfromdeeplearningbasedmriprostatesegmentation AT alvarofernandezquilez impactofuncertaintyquantificationthroughconformalpredictiononvolumeassessmentfromdeeplearningbasedmriprostatesegmentation |