Development and validation of an explainable machine learning model for mortality prediction among patients with infected pancreatic necrosisResearch in context
Summary: Background: Infected pancreatic necrosis (IPN) represents a severe complication of acute pancreatitis, commonly linked with mortality rates ranging from 15% to 35%. However, the present mortality prediction tools for IPN are limited and lack sufficient sensitivity and specificity. This stu...
Saved in:
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2025-02-01
|
| Series: | EClinicalMedicine |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2589537025000069 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832589997894533120 |
|---|---|
| author | Caihong Ning Hui Ouyang Jie Xiao Di Wu Zefang Sun Baiqi Liu Dingcheng Shen Xiaoyue Hong Chiayan Lin Jiarong Li Lu Chen Shuai Zhu Xinying Li Fada Xia Gengwen Huang |
| author_facet | Caihong Ning Hui Ouyang Jie Xiao Di Wu Zefang Sun Baiqi Liu Dingcheng Shen Xiaoyue Hong Chiayan Lin Jiarong Li Lu Chen Shuai Zhu Xinying Li Fada Xia Gengwen Huang |
| author_sort | Caihong Ning |
| collection | DOAJ |
| description | Summary: Background: Infected pancreatic necrosis (IPN) represents a severe complication of acute pancreatitis, commonly linked with mortality rates ranging from 15% to 35%. However, the present mortality prediction tools for IPN are limited and lack sufficient sensitivity and specificity. This study aims to develop and validate an explainable machine learning (ML) model for death prediction among patients with IPN. Methods: We performed a prospective cohort study of 344 patients with IPN consecutively enrolled from a large Chinese tertiary hospital from January 2011 to January 2023. Ten ML models were developed to predict 90-day mortality in these patients. A benchmarking test, involving nested resampling, automatic hyperparameter tuning and random search techniques, was conducted to select the ML model. Sequential forward selection method was employed to select the optimal feature subset from 31 candidate subsets to simplify the model and maximize predictive performance. The final model was internally validated with the 1000 bootstrap method and externally validated using an independent cohort of 132 patients with IPN retrospectively collected from another Chinese tertiary hospital from January 2018 to January 2023. The SHapley Additive exPlanations (SHAP) method was employed to interpret the model in terms of features importance and features effect. The final model constructed with optimal feature subset was deployed as an interactive web-based Shiny app. Findings: Random survival forest (RSF) model showed the best predictive performance than other 9 ML models (internal validation, C-index = 0.863 [95% CI: 0.854–0.875]; external validation, C-index = 0.857 [95% CI: 0.850–0.865]). Multiple organ failure, Acute Physiology and Chronic Health Examination II (APACHE II) score ≥20, duration of organ failure ≥21 days, bloodstream infection, time from onset to first intervention <30 days, Bedside Index of Severity in Acute Pancreatitis score ≥3, critical acute pancreatitis, age ≥ 50 years, and hemorrhage were 9 most important features associated with mortality. Furthermore, SHAP algorithm revealed insightful nonlinear interactive associations between important predictors and mortality, identifying 9 features pairs with high interaction SHAP value and clinical significance. Two interactive web-based Shiny apps were developed to enhance clinical practicability: https://rsfmodels.shinyapps.io/IPN_app/ for cases where the APACHE II score was available and https://rsfmodels.shinyapps.io/IPNeasy/ for cases where it was not. Interpretation: An explainable ML model for death prediction among IPN patients was feasible and effective, suggesting its superior potential in guiding clinical management and improving patient outcomes. Two publicly accessible web tools generated for the optimized model facilitated its utility in clinical settings. Funding: The Natural Science Foundation of Hunan Province (2023JJ30885), Postdoctoral Fellowship Program of CPSF (GZB20230872), The Youth Science Foundation of Xiangya Hospital (2023Q13), The Project Program of National Clinical Research Center for Geriatric Disorders of Xiangya Hospital (2021LNJJ19). |
| format | Article |
| id | doaj-art-a504bac909a747d89387111c835de201 |
| institution | Kabale University |
| issn | 2589-5370 |
| language | English |
| publishDate | 2025-02-01 |
| publisher | Elsevier |
| record_format | Article |
| series | EClinicalMedicine |
| spelling | doaj-art-a504bac909a747d89387111c835de2012025-01-24T04:45:35ZengElsevierEClinicalMedicine2589-53702025-02-0180103074Development and validation of an explainable machine learning model for mortality prediction among patients with infected pancreatic necrosisResearch in contextCaihong Ning0Hui Ouyang1Jie Xiao2Di Wu3Zefang Sun4Baiqi Liu5Dingcheng Shen6Xiaoyue Hong7Chiayan Lin8Jiarong Li9Lu Chen10Shuai Zhu11Xinying Li12Fada Xia13Gengwen Huang14Department of General Surgery, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, China; National Clinical Research Center for Geriatric Disorders, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, China; Division of Pancreatic Surgery, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, China; Division of Hernia and Abdominal Wall Surgery, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, China; FuRong Laboratory, Changsha, Hunan Province 410078, ChinaDepartment of General Surgery, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, China; National Clinical Research Center for Geriatric Disorders, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, ChinaDepartment of Emergency, Third Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, ChinaDepartment of Emergency, Third Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, ChinaDepartment of General Surgery, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, China; National Clinical Research Center for Geriatric Disorders, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, China; Division of Pancreatic Surgery, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, China; Division of Hernia and Abdominal Wall Surgery, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, ChinaDepartment of General Surgery, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, China; National Clinical Research Center for Geriatric Disorders, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, China; Division of Pancreatic Surgery, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, China; Division of Hernia and Abdominal Wall Surgery, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, ChinaDepartment of General Surgery, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, China; National Clinical Research Center for Geriatric Disorders, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, China; Division of Pancreatic Surgery, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, China; Division of Hernia and Abdominal Wall Surgery, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, ChinaDepartment of General Surgery, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, China; National Clinical Research Center for Geriatric Disorders, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, China; Division of Pancreatic Surgery, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, China; Division of Hernia and Abdominal Wall Surgery, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, ChinaDepartment of General Surgery, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, China; National Clinical Research Center for Geriatric Disorders, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, China; Division of Pancreatic Surgery, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, China; Division of Hernia and Abdominal Wall Surgery, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, ChinaDepartment of General Surgery, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, China; National Clinical Research Center for Geriatric Disorders, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, China; Division of Pancreatic Surgery, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, China; Division of Hernia and Abdominal Wall Surgery, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, ChinaDepartment of General Surgery, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, China; National Clinical Research Center for Geriatric Disorders, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, China; Division of Pancreatic Surgery, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, China; Division of Hernia and Abdominal Wall Surgery, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, ChinaDepartment of General Surgery, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, China; National Clinical Research Center for Geriatric Disorders, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, China; Division of Pancreatic Surgery, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, China; Division of Hernia and Abdominal Wall Surgery, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, ChinaDepartment of General Surgery, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, China; National Clinical Research Center for Geriatric Disorders, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, ChinaDepartment of General Surgery, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, China; National Clinical Research Center for Geriatric Disorders, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, China; Corresponding author. Department of General Surgery, Xiangya Hospital, Central South University, 87 Xiangya Rd, Changsha, Hunan Province 410008, China.Department of General Surgery, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, China; National Clinical Research Center for Geriatric Disorders, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, China; Division of Pancreatic Surgery, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, China; Division of Hernia and Abdominal Wall Surgery, Xiangya Hospital, Central South University, Changsha, Hunan Province 410008, China; FuRong Laboratory, Changsha, Hunan Province 410078, China; Corresponding author. Department of General Surgery, Xiangya Hospital, Central South University, 87 Xiangya Rd, Changsha, Hunan Province 410008, China.Summary: Background: Infected pancreatic necrosis (IPN) represents a severe complication of acute pancreatitis, commonly linked with mortality rates ranging from 15% to 35%. However, the present mortality prediction tools for IPN are limited and lack sufficient sensitivity and specificity. This study aims to develop and validate an explainable machine learning (ML) model for death prediction among patients with IPN. Methods: We performed a prospective cohort study of 344 patients with IPN consecutively enrolled from a large Chinese tertiary hospital from January 2011 to January 2023. Ten ML models were developed to predict 90-day mortality in these patients. A benchmarking test, involving nested resampling, automatic hyperparameter tuning and random search techniques, was conducted to select the ML model. Sequential forward selection method was employed to select the optimal feature subset from 31 candidate subsets to simplify the model and maximize predictive performance. The final model was internally validated with the 1000 bootstrap method and externally validated using an independent cohort of 132 patients with IPN retrospectively collected from another Chinese tertiary hospital from January 2018 to January 2023. The SHapley Additive exPlanations (SHAP) method was employed to interpret the model in terms of features importance and features effect. The final model constructed with optimal feature subset was deployed as an interactive web-based Shiny app. Findings: Random survival forest (RSF) model showed the best predictive performance than other 9 ML models (internal validation, C-index = 0.863 [95% CI: 0.854–0.875]; external validation, C-index = 0.857 [95% CI: 0.850–0.865]). Multiple organ failure, Acute Physiology and Chronic Health Examination II (APACHE II) score ≥20, duration of organ failure ≥21 days, bloodstream infection, time from onset to first intervention <30 days, Bedside Index of Severity in Acute Pancreatitis score ≥3, critical acute pancreatitis, age ≥ 50 years, and hemorrhage were 9 most important features associated with mortality. Furthermore, SHAP algorithm revealed insightful nonlinear interactive associations between important predictors and mortality, identifying 9 features pairs with high interaction SHAP value and clinical significance. Two interactive web-based Shiny apps were developed to enhance clinical practicability: https://rsfmodels.shinyapps.io/IPN_app/ for cases where the APACHE II score was available and https://rsfmodels.shinyapps.io/IPNeasy/ for cases where it was not. Interpretation: An explainable ML model for death prediction among IPN patients was feasible and effective, suggesting its superior potential in guiding clinical management and improving patient outcomes. Two publicly accessible web tools generated for the optimized model facilitated its utility in clinical settings. Funding: The Natural Science Foundation of Hunan Province (2023JJ30885), Postdoctoral Fellowship Program of CPSF (GZB20230872), The Youth Science Foundation of Xiangya Hospital (2023Q13), The Project Program of National Clinical Research Center for Geriatric Disorders of Xiangya Hospital (2021LNJJ19).http://www.sciencedirect.com/science/article/pii/S2589537025000069Infected pancreatic necrosisMachine learningPrediction modelSHapley additive exPlanationsShiny app |
| spellingShingle | Caihong Ning Hui Ouyang Jie Xiao Di Wu Zefang Sun Baiqi Liu Dingcheng Shen Xiaoyue Hong Chiayan Lin Jiarong Li Lu Chen Shuai Zhu Xinying Li Fada Xia Gengwen Huang Development and validation of an explainable machine learning model for mortality prediction among patients with infected pancreatic necrosisResearch in context EClinicalMedicine Infected pancreatic necrosis Machine learning Prediction model SHapley additive exPlanations Shiny app |
| title | Development and validation of an explainable machine learning model for mortality prediction among patients with infected pancreatic necrosisResearch in context |
| title_full | Development and validation of an explainable machine learning model for mortality prediction among patients with infected pancreatic necrosisResearch in context |
| title_fullStr | Development and validation of an explainable machine learning model for mortality prediction among patients with infected pancreatic necrosisResearch in context |
| title_full_unstemmed | Development and validation of an explainable machine learning model for mortality prediction among patients with infected pancreatic necrosisResearch in context |
| title_short | Development and validation of an explainable machine learning model for mortality prediction among patients with infected pancreatic necrosisResearch in context |
| title_sort | development and validation of an explainable machine learning model for mortality prediction among patients with infected pancreatic necrosisresearch in context |
| topic | Infected pancreatic necrosis Machine learning Prediction model SHapley additive exPlanations Shiny app |
| url | http://www.sciencedirect.com/science/article/pii/S2589537025000069 |
| work_keys_str_mv | AT caihongning developmentandvalidationofanexplainablemachinelearningmodelformortalitypredictionamongpatientswithinfectedpancreaticnecrosisresearchincontext AT huiouyang developmentandvalidationofanexplainablemachinelearningmodelformortalitypredictionamongpatientswithinfectedpancreaticnecrosisresearchincontext AT jiexiao developmentandvalidationofanexplainablemachinelearningmodelformortalitypredictionamongpatientswithinfectedpancreaticnecrosisresearchincontext AT diwu developmentandvalidationofanexplainablemachinelearningmodelformortalitypredictionamongpatientswithinfectedpancreaticnecrosisresearchincontext AT zefangsun developmentandvalidationofanexplainablemachinelearningmodelformortalitypredictionamongpatientswithinfectedpancreaticnecrosisresearchincontext AT baiqiliu developmentandvalidationofanexplainablemachinelearningmodelformortalitypredictionamongpatientswithinfectedpancreaticnecrosisresearchincontext AT dingchengshen developmentandvalidationofanexplainablemachinelearningmodelformortalitypredictionamongpatientswithinfectedpancreaticnecrosisresearchincontext AT xiaoyuehong developmentandvalidationofanexplainablemachinelearningmodelformortalitypredictionamongpatientswithinfectedpancreaticnecrosisresearchincontext AT chiayanlin developmentandvalidationofanexplainablemachinelearningmodelformortalitypredictionamongpatientswithinfectedpancreaticnecrosisresearchincontext AT jiarongli developmentandvalidationofanexplainablemachinelearningmodelformortalitypredictionamongpatientswithinfectedpancreaticnecrosisresearchincontext AT luchen developmentandvalidationofanexplainablemachinelearningmodelformortalitypredictionamongpatientswithinfectedpancreaticnecrosisresearchincontext AT shuaizhu developmentandvalidationofanexplainablemachinelearningmodelformortalitypredictionamongpatientswithinfectedpancreaticnecrosisresearchincontext AT xinyingli developmentandvalidationofanexplainablemachinelearningmodelformortalitypredictionamongpatientswithinfectedpancreaticnecrosisresearchincontext AT fadaxia developmentandvalidationofanexplainablemachinelearningmodelformortalitypredictionamongpatientswithinfectedpancreaticnecrosisresearchincontext AT gengwenhuang developmentandvalidationofanexplainablemachinelearningmodelformortalitypredictionamongpatientswithinfectedpancreaticnecrosisresearchincontext |