An Improved ConvNeXt With Multimodal Transformer for Physiological Signal Classification
With escalating mortality rates associated with cardiovascular disease, the early detection of arrhythmias assumes ever-increasing significance. This study introduces a novel multimodal network that concurrently classifies electrocardiogram (ECG) and wrist pulse signal (WPS). Both ECG and WPS, as hu...
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10401880/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832592893282353152 |
|---|---|
| author | Jiajian Zhu Yue Feng Qichao Liu Hong Xu Yuan Miao Zhuosheng Lin Jia Li Huilin Liu Ying Xu Fufeng Li |
| author_facet | Jiajian Zhu Yue Feng Qichao Liu Hong Xu Yuan Miao Zhuosheng Lin Jia Li Huilin Liu Ying Xu Fufeng Li |
| author_sort | Jiajian Zhu |
| collection | DOAJ |
| description | With escalating mortality rates associated with cardiovascular disease, the early detection of arrhythmias assumes ever-increasing significance. This study introduces a novel multimodal network that concurrently classifies electrocardiogram (ECG) and wrist pulse signal (WPS). Both ECG and WPS, as human physiological signals, share closely related distributions and characteristics, holding potential to accurately reflect underlying cardiovascular conditions. The proposed ICMT-Net utilizes continuous wavelet transform to partition 5-second ECG and WPS segments into spectrograms. It incorporates an improved ConvNeXt, a multimodal transformer layer, and a fused multi-layer perceptron to extract and fuse multimodal features for ECG classification. Subsequently, the network is adapted to WPS and coronary heart disease classification tasks through transfer learning techniques. In comparison to existing methods, our approach achieves heightened sensitivity in detecting supraventricular and ventricular ectopic segments, while also outperforming established WPS classification methodologies. Importantly, the proposed network adeptly handles multimodal signals and excels in classification accuracy, particularly within the realm of physiological signals. |
| format | Article |
| id | doaj-art-d20ea76f3d1f47888d3a157eef700488 |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-d20ea76f3d1f47888d3a157eef7004882025-01-21T00:02:31ZengIEEEIEEE Access2169-35362024-01-0112112171122910.1109/ACCESS.2024.335527310401880An Improved ConvNeXt With Multimodal Transformer for Physiological Signal ClassificationJiajian Zhu0https://orcid.org/0000-0003-4500-8885Yue Feng1Qichao Liu2Hong Xu3https://orcid.org/0000-0002-2968-9804Yuan Miao4https://orcid.org/0000-0002-6712-3465Zhuosheng Lin5https://orcid.org/0000-0001-5963-8525Jia Li6Huilin Liu7Ying Xu8Fufeng Li9https://orcid.org/0000-0002-0566-3589Faculty of Intelligent Manufacturing, Wuyi University, Jiangmen, Guangdong, ChinaFaculty of Intelligent Manufacturing, Wuyi University, Jiangmen, Guangdong, ChinaFaculty of Intelligent Manufacturing, Wuyi University, Jiangmen, Guangdong, ChinaFaculty of Intelligent Manufacturing, Wuyi University, Jiangmen, Guangdong, ChinaCollege of Arts, Business, Law, Education and iT,, Victoria University, Melbourne, VIC, AustraliaFaculty of Intelligent Manufacturing, Wuyi University, Jiangmen, Guangdong, ChinaFaculty of Intelligent Manufacturing, Wuyi University, Jiangmen, Guangdong, ChinaBasic Medical College, Shanghai University of Traditional Chinese Medicine, Shanghai, ChinaLaboratory of TCM Four Processing, Shanghai University of TCM, Shanghai, ChinaLaboratory of TCM Four Processing, Shanghai University of TCM, Shanghai, ChinaWith escalating mortality rates associated with cardiovascular disease, the early detection of arrhythmias assumes ever-increasing significance. This study introduces a novel multimodal network that concurrently classifies electrocardiogram (ECG) and wrist pulse signal (WPS). Both ECG and WPS, as human physiological signals, share closely related distributions and characteristics, holding potential to accurately reflect underlying cardiovascular conditions. The proposed ICMT-Net utilizes continuous wavelet transform to partition 5-second ECG and WPS segments into spectrograms. It incorporates an improved ConvNeXt, a multimodal transformer layer, and a fused multi-layer perceptron to extract and fuse multimodal features for ECG classification. Subsequently, the network is adapted to WPS and coronary heart disease classification tasks through transfer learning techniques. In comparison to existing methods, our approach achieves heightened sensitivity in detecting supraventricular and ventricular ectopic segments, while also outperforming established WPS classification methodologies. Importantly, the proposed network adeptly handles multimodal signals and excels in classification accuracy, particularly within the realm of physiological signals.https://ieeexplore.ieee.org/document/10401880/Electrocardiogramwrist pulse signaltransfer learningcoronary heart disease classification |
| spellingShingle | Jiajian Zhu Yue Feng Qichao Liu Hong Xu Yuan Miao Zhuosheng Lin Jia Li Huilin Liu Ying Xu Fufeng Li An Improved ConvNeXt With Multimodal Transformer for Physiological Signal Classification IEEE Access Electrocardiogram wrist pulse signal transfer learning coronary heart disease classification |
| title | An Improved ConvNeXt With Multimodal Transformer for Physiological Signal Classification |
| title_full | An Improved ConvNeXt With Multimodal Transformer for Physiological Signal Classification |
| title_fullStr | An Improved ConvNeXt With Multimodal Transformer for Physiological Signal Classification |
| title_full_unstemmed | An Improved ConvNeXt With Multimodal Transformer for Physiological Signal Classification |
| title_short | An Improved ConvNeXt With Multimodal Transformer for Physiological Signal Classification |
| title_sort | improved convnext with multimodal transformer for physiological signal classification |
| topic | Electrocardiogram wrist pulse signal transfer learning coronary heart disease classification |
| url | https://ieeexplore.ieee.org/document/10401880/ |
| work_keys_str_mv | AT jiajianzhu animprovedconvnextwithmultimodaltransformerforphysiologicalsignalclassification AT yuefeng animprovedconvnextwithmultimodaltransformerforphysiologicalsignalclassification AT qichaoliu animprovedconvnextwithmultimodaltransformerforphysiologicalsignalclassification AT hongxu animprovedconvnextwithmultimodaltransformerforphysiologicalsignalclassification AT yuanmiao animprovedconvnextwithmultimodaltransformerforphysiologicalsignalclassification AT zhuoshenglin animprovedconvnextwithmultimodaltransformerforphysiologicalsignalclassification AT jiali animprovedconvnextwithmultimodaltransformerforphysiologicalsignalclassification AT huilinliu animprovedconvnextwithmultimodaltransformerforphysiologicalsignalclassification AT yingxu animprovedconvnextwithmultimodaltransformerforphysiologicalsignalclassification AT fufengli animprovedconvnextwithmultimodaltransformerforphysiologicalsignalclassification AT jiajianzhu improvedconvnextwithmultimodaltransformerforphysiologicalsignalclassification AT yuefeng improvedconvnextwithmultimodaltransformerforphysiologicalsignalclassification AT qichaoliu improvedconvnextwithmultimodaltransformerforphysiologicalsignalclassification AT hongxu improvedconvnextwithmultimodaltransformerforphysiologicalsignalclassification AT yuanmiao improvedconvnextwithmultimodaltransformerforphysiologicalsignalclassification AT zhuoshenglin improvedconvnextwithmultimodaltransformerforphysiologicalsignalclassification AT jiali improvedconvnextwithmultimodaltransformerforphysiologicalsignalclassification AT huilinliu improvedconvnextwithmultimodaltransformerforphysiologicalsignalclassification AT yingxu improvedconvnextwithmultimodaltransformerforphysiologicalsignalclassification AT fufengli improvedconvnextwithmultimodaltransformerforphysiologicalsignalclassification |