Development and external validation of machine learning-based models to predict patients with cellulitis developing sepsis during hospitalisation
Objective Cellulitis is the most common cause of skin-related hospitalisations, and the mortality of patients with sepsis remains high. Some stratification models have been developed, but their performance in external validation has been unsatisfactory. This study was designed to develop and compare...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMJ Publishing Group
2024-07-01
|
| Series: | BMJ Open |
| Online Access: | https://bmjopen.bmj.com/content/14/7/e084183.full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832570215759609856 |
|---|---|
| author | Li Hu Xilingyuan Chen Rentao Yu |
| author_facet | Li Hu Xilingyuan Chen Rentao Yu |
| author_sort | Li Hu |
| collection | DOAJ |
| description | Objective Cellulitis is the most common cause of skin-related hospitalisations, and the mortality of patients with sepsis remains high. Some stratification models have been developed, but their performance in external validation has been unsatisfactory. This study was designed to develop and compare different models for predicting patients with cellulitis developing sepsis during hospitalisation.Design This is a retrospective cohort study.Setting This study included both the development and the external-validation phases from two independent large cohorts internationally.Participants and methods A total of 6695 patients with cellulitis in the Medical Information Mart for Intensive care (MIMIC)-IV database were used to develop models with different machine-learning algorithms. The best models were selected and then externally validated in 2506 patients with cellulitis from the YiduCloud database of our university. The performances and robustness of selected models were further compared in the external-validation group by area under the curve (AUC), diagnostic accuracy, sensitivity, specificity and diagnostic OR.Primary outcome measures The primary outcome of interest in this study was the development based on the Sepsis-3.0 criteria during hospitalisation.Results Patient characteristics were significantly different between the two groups. In internal validation, XGBoost was the best model, with an AUC of 0.780, and AdaBoost was the worst model, with an AUC of 0.585. In external validation, the AUC of the artificial neural network (ANN) model was the highest, 0.830, while the AUC of the logistic regression (LR) model was the lowest, 0.792. The AUC values changed less in the boosting and ANN models than in the LR model when variables were deleted.Conclusions Boosting and neural network models performed slightly better than the LR model and were more robust in complex clinical situations. The results could provide a tool for clinicians to detect hospitalised patients with cellulitis developing sepsis early. |
| format | Article |
| id | doaj-art-7bd9a4d8eb814332970663b052e31e1e |
| institution | Kabale University |
| issn | 2044-6055 |
| language | English |
| publishDate | 2024-07-01 |
| publisher | BMJ Publishing Group |
| record_format | Article |
| series | BMJ Open |
| spelling | doaj-art-7bd9a4d8eb814332970663b052e31e1e2025-02-02T16:00:10ZengBMJ Publishing GroupBMJ Open2044-60552024-07-0114710.1136/bmjopen-2024-084183Development and external validation of machine learning-based models to predict patients with cellulitis developing sepsis during hospitalisationLi Hu0Xilingyuan Chen1Rentao Yu22 Department of Dermatology, The First Affiliated Hospital of Chongqing Medical University, Chongqing, China1 Chongqing Medical University, Chongqing, China2 Department of Dermatology, The First Affiliated Hospital of Chongqing Medical University, Chongqing, ChinaObjective Cellulitis is the most common cause of skin-related hospitalisations, and the mortality of patients with sepsis remains high. Some stratification models have been developed, but their performance in external validation has been unsatisfactory. This study was designed to develop and compare different models for predicting patients with cellulitis developing sepsis during hospitalisation.Design This is a retrospective cohort study.Setting This study included both the development and the external-validation phases from two independent large cohorts internationally.Participants and methods A total of 6695 patients with cellulitis in the Medical Information Mart for Intensive care (MIMIC)-IV database were used to develop models with different machine-learning algorithms. The best models were selected and then externally validated in 2506 patients with cellulitis from the YiduCloud database of our university. The performances and robustness of selected models were further compared in the external-validation group by area under the curve (AUC), diagnostic accuracy, sensitivity, specificity and diagnostic OR.Primary outcome measures The primary outcome of interest in this study was the development based on the Sepsis-3.0 criteria during hospitalisation.Results Patient characteristics were significantly different between the two groups. In internal validation, XGBoost was the best model, with an AUC of 0.780, and AdaBoost was the worst model, with an AUC of 0.585. In external validation, the AUC of the artificial neural network (ANN) model was the highest, 0.830, while the AUC of the logistic regression (LR) model was the lowest, 0.792. The AUC values changed less in the boosting and ANN models than in the LR model when variables were deleted.Conclusions Boosting and neural network models performed slightly better than the LR model and were more robust in complex clinical situations. The results could provide a tool for clinicians to detect hospitalised patients with cellulitis developing sepsis early.https://bmjopen.bmj.com/content/14/7/e084183.full |
| spellingShingle | Li Hu Xilingyuan Chen Rentao Yu Development and external validation of machine learning-based models to predict patients with cellulitis developing sepsis during hospitalisation BMJ Open |
| title | Development and external validation of machine learning-based models to predict patients with cellulitis developing sepsis during hospitalisation |
| title_full | Development and external validation of machine learning-based models to predict patients with cellulitis developing sepsis during hospitalisation |
| title_fullStr | Development and external validation of machine learning-based models to predict patients with cellulitis developing sepsis during hospitalisation |
| title_full_unstemmed | Development and external validation of machine learning-based models to predict patients with cellulitis developing sepsis during hospitalisation |
| title_short | Development and external validation of machine learning-based models to predict patients with cellulitis developing sepsis during hospitalisation |
| title_sort | development and external validation of machine learning based models to predict patients with cellulitis developing sepsis during hospitalisation |
| url | https://bmjopen.bmj.com/content/14/7/e084183.full |
| work_keys_str_mv | AT lihu developmentandexternalvalidationofmachinelearningbasedmodelstopredictpatientswithcellulitisdevelopingsepsisduringhospitalisation AT xilingyuanchen developmentandexternalvalidationofmachinelearningbasedmodelstopredictpatientswithcellulitisdevelopingsepsisduringhospitalisation AT rentaoyu developmentandexternalvalidationofmachinelearningbasedmodelstopredictpatientswithcellulitisdevelopingsepsisduringhospitalisation |