Anatomy-Informed Multimodal Learning for Myocardial Infarction Prediction
<italic>Goal:</italic> In patients with coronary artery disease, the prediction of future cardiac events such as myocardial infarction (MI) remains a major challenge. In this work, we propose a novel anatomy-informed multimodal deep learning framework to predict future MI from clinical d...
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Open Journal of Engineering in Medicine and Biology |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10540036/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832583268322508800 |
|---|---|
| author | Ivan-Daniel Sievering Ortal Senouf Thabo Mahendiran David Nanchen Stephane Fournier Olivier Muller Pascal Frossard Emmanuel Abbe Dorina Thanou |
| author_facet | Ivan-Daniel Sievering Ortal Senouf Thabo Mahendiran David Nanchen Stephane Fournier Olivier Muller Pascal Frossard Emmanuel Abbe Dorina Thanou |
| author_sort | Ivan-Daniel Sievering |
| collection | DOAJ |
| description | <italic>Goal:</italic> In patients with coronary artery disease, the prediction of future cardiac events such as myocardial infarction (MI) remains a major challenge. In this work, we propose a novel anatomy-informed multimodal deep learning framework to predict future MI from clinical data and Invasive Coronary Angiography (ICA) images. <italic>Methods:</italic> The images are analyzed by Convolutional Neural Networks (CNNs) guided by anatomical information, and the clinical data by an Artificial Neural Network (ANN). Embeddings from both sources are then merged to provide a patient-level prediction. <italic>Results:</italic> The results of our framework on a clinical study of 445 patients admitted with acute coronary syndromes confirms that multimodal learning increases the predictive power and achieves good performance (AUC: <inline-formula><tex-math notation="LaTeX">$0.67\pm 0.04$</tex-math></inline-formula> & F1-Score: <inline-formula><tex-math notation="LaTeX">$0.36\pm 0.12$</tex-math></inline-formula>), which outperforms the prediction obtained by each modality independently as well as that of interventional cardiologists (AUC: 0.54 & F1-Score: 0.18). <italic>Conclusions:</italic> To the best of our knowledge, this is the first attempt towards combining multimodal data through a deep learning framework for future MI prediction. Although it demonstrates the superiority of multi-modal approaches over single modality, the results do not yet meet the necessary criteria for practical application. |
| format | Article |
| id | doaj-art-f1b5d03763aa4bbfa0907a72340d40eb |
| institution | Kabale University |
| issn | 2644-1276 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Open Journal of Engineering in Medicine and Biology |
| spelling | doaj-art-f1b5d03763aa4bbfa0907a72340d40eb2025-01-29T00:01:32ZengIEEEIEEE Open Journal of Engineering in Medicine and Biology2644-12762024-01-01583784510.1109/OJEMB.2024.340394810540036Anatomy-Informed Multimodal Learning for Myocardial Infarction PredictionIvan-Daniel Sievering0https://orcid.org/0009-0009-5423-6178Ortal Senouf1Thabo Mahendiran2https://orcid.org/0000-0002-0025-8162David Nanchen3https://orcid.org/0000-0002-2493-3505Stephane Fournier4https://orcid.org/0000-0002-9422-9521Olivier Muller5https://orcid.org/0000-0003-2441-5799Pascal Frossard6https://orcid.org/0000-0002-4010-714XEmmanuel Abbe7Dorina Thanou8https://orcid.org/0000-0003-2319-4832Signal Processing Laboratory 4, EPFL, Lausanne, SwitzerlandSignal Processing Laboratory 4, EPFL, Lausanne, SwitzerlandDepartment of Cardiology, CHUV, Lausanne, SwitzerlandUnisante, Lausanne, SwitzerlandDepartment of Cardiology, CHUV, Lausanne, SwitzerlandDepartment of Cardiology, CHUV, Lausanne, SwitzerlandSignal Processing Laboratory 4, EPFL, Lausanne, SwitzerlandChair of Mathematical Data Science, EPFL, Lausanne, SwitzerlandCenter for Intelligent Systems, EPFL, Lausanne, Switzerland<italic>Goal:</italic> In patients with coronary artery disease, the prediction of future cardiac events such as myocardial infarction (MI) remains a major challenge. In this work, we propose a novel anatomy-informed multimodal deep learning framework to predict future MI from clinical data and Invasive Coronary Angiography (ICA) images. <italic>Methods:</italic> The images are analyzed by Convolutional Neural Networks (CNNs) guided by anatomical information, and the clinical data by an Artificial Neural Network (ANN). Embeddings from both sources are then merged to provide a patient-level prediction. <italic>Results:</italic> The results of our framework on a clinical study of 445 patients admitted with acute coronary syndromes confirms that multimodal learning increases the predictive power and achieves good performance (AUC: <inline-formula><tex-math notation="LaTeX">$0.67\pm 0.04$</tex-math></inline-formula> & F1-Score: <inline-formula><tex-math notation="LaTeX">$0.36\pm 0.12$</tex-math></inline-formula>), which outperforms the prediction obtained by each modality independently as well as that of interventional cardiologists (AUC: 0.54 & F1-Score: 0.18). <italic>Conclusions:</italic> To the best of our knowledge, this is the first attempt towards combining multimodal data through a deep learning framework for future MI prediction. Although it demonstrates the superiority of multi-modal approaches over single modality, the results do not yet meet the necessary criteria for practical application.https://ieeexplore.ieee.org/document/10540036/Coronary artery diseasedeep learninginvasive coronary angiographymultimodal datamyocardial infarction |
| spellingShingle | Ivan-Daniel Sievering Ortal Senouf Thabo Mahendiran David Nanchen Stephane Fournier Olivier Muller Pascal Frossard Emmanuel Abbe Dorina Thanou Anatomy-Informed Multimodal Learning for Myocardial Infarction Prediction IEEE Open Journal of Engineering in Medicine and Biology Coronary artery disease deep learning invasive coronary angiography multimodal data myocardial infarction |
| title | Anatomy-Informed Multimodal Learning for Myocardial Infarction Prediction |
| title_full | Anatomy-Informed Multimodal Learning for Myocardial Infarction Prediction |
| title_fullStr | Anatomy-Informed Multimodal Learning for Myocardial Infarction Prediction |
| title_full_unstemmed | Anatomy-Informed Multimodal Learning for Myocardial Infarction Prediction |
| title_short | Anatomy-Informed Multimodal Learning for Myocardial Infarction Prediction |
| title_sort | anatomy informed multimodal learning for myocardial infarction prediction |
| topic | Coronary artery disease deep learning invasive coronary angiography multimodal data myocardial infarction |
| url | https://ieeexplore.ieee.org/document/10540036/ |
| work_keys_str_mv | AT ivandanielsievering anatomyinformedmultimodallearningformyocardialinfarctionprediction AT ortalsenouf anatomyinformedmultimodallearningformyocardialinfarctionprediction AT thabomahendiran anatomyinformedmultimodallearningformyocardialinfarctionprediction AT davidnanchen anatomyinformedmultimodallearningformyocardialinfarctionprediction AT stephanefournier anatomyinformedmultimodallearningformyocardialinfarctionprediction AT oliviermuller anatomyinformedmultimodallearningformyocardialinfarctionprediction AT pascalfrossard anatomyinformedmultimodallearningformyocardialinfarctionprediction AT emmanuelabbe anatomyinformedmultimodallearningformyocardialinfarctionprediction AT dorinathanou anatomyinformedmultimodallearningformyocardialinfarctionprediction |