Multi-modal feature fusion with multi-head self-attention for epileptic EEG signals
It is important to classify electroencephalography (EEG) signals automatically for the diagnosis and treatment of epilepsy. Currently, the dominant single-modal feature extraction methods cannot cover the information of different modalities, resulting in poor classification performance of existing m...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
AIMS Press
2024-08-01
|
| Series: | Mathematical Biosciences and Engineering |
| Subjects: | |
| Online Access: | https://www.aimspress.com/article/doi/10.3934/mbe.2024304 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832590816578633728 |
|---|---|
| author | Ning Huang Zhengtao Xi Yingying Jiao Yudong Zhang Zhuqing Jiao Xiaona Li |
| author_facet | Ning Huang Zhengtao Xi Yingying Jiao Yudong Zhang Zhuqing Jiao Xiaona Li |
| author_sort | Ning Huang |
| collection | DOAJ |
| description | It is important to classify electroencephalography (EEG) signals automatically for the diagnosis and treatment of epilepsy. Currently, the dominant single-modal feature extraction methods cannot cover the information of different modalities, resulting in poor classification performance of existing methods, especially the multi-classification problem. We proposed a multi-modal feature fusion (MMFF) method for epileptic EEG signals. First, the time domain features were extracted by kernel principal component analysis, the frequency domain features were extracted by short-time Fourier extracted transform, and the nonlinear dynamic features were extracted by calculating sample entropy. On this basis, the features of these three modalities were interactively learned through the multi-head self-attention mechanism, and the attention weights were trained simultaneously. The fused features were obtained by combining the value vectors of feature representations, while the time, frequency, and nonlinear dynamics information were retained to screen out more representative epileptic features and improve the accuracy of feature extraction. Finally, the feature fusion method was applied to epileptic EEG signal classifications. The experimental results demonstrated that the proposed method achieves a classification accuracy of 92.76 ± 1.64% across the five-category classification task for epileptic EEG signals. The multi-head self-attention mechanism promotes the fusion of multi-modal features and offers an efficient and novel approach for diagnosing and treating epilepsy. |
| format | Article |
| id | doaj-art-64b8dc40d9f84a51b397a5aeca93f085 |
| institution | Kabale University |
| issn | 1551-0018 |
| language | English |
| publishDate | 2024-08-01 |
| publisher | AIMS Press |
| record_format | Article |
| series | Mathematical Biosciences and Engineering |
| spelling | doaj-art-64b8dc40d9f84a51b397a5aeca93f0852025-01-23T07:47:47ZengAIMS PressMathematical Biosciences and Engineering1551-00182024-08-012186918693510.3934/mbe.2024304Multi-modal feature fusion with multi-head self-attention for epileptic EEG signalsNing Huang0Zhengtao Xi1Yingying Jiao2Yudong Zhang3Zhuqing Jiao4Xiaona Li5School of Computer Science and Artificial Intelligence, Changzhou University, Changzhou 213164, ChinaSchool of Wangzheng Microelectronics, Changzhou University, Changzhou 213164, ChinaSchool of Computer Science and Artificial Intelligence, Changzhou University, Changzhou 213164, ChinaSchool of Computing and Mathematical Sciences, University of Leicester, Leicester, UKSchool of Computer Science and Artificial Intelligence, Changzhou University, Changzhou 213164, ChinaDepartment of Nursing, The Third Affiliated Hospital with Nanjing Medical University, Changzhou 213003, ChinaIt is important to classify electroencephalography (EEG) signals automatically for the diagnosis and treatment of epilepsy. Currently, the dominant single-modal feature extraction methods cannot cover the information of different modalities, resulting in poor classification performance of existing methods, especially the multi-classification problem. We proposed a multi-modal feature fusion (MMFF) method for epileptic EEG signals. First, the time domain features were extracted by kernel principal component analysis, the frequency domain features were extracted by short-time Fourier extracted transform, and the nonlinear dynamic features were extracted by calculating sample entropy. On this basis, the features of these three modalities were interactively learned through the multi-head self-attention mechanism, and the attention weights were trained simultaneously. The fused features were obtained by combining the value vectors of feature representations, while the time, frequency, and nonlinear dynamics information were retained to screen out more representative epileptic features and improve the accuracy of feature extraction. Finally, the feature fusion method was applied to epileptic EEG signal classifications. The experimental results demonstrated that the proposed method achieves a classification accuracy of 92.76 ± 1.64% across the five-category classification task for epileptic EEG signals. The multi-head self-attention mechanism promotes the fusion of multi-modal features and offers an efficient and novel approach for diagnosing and treating epilepsy.https://www.aimspress.com/article/doi/10.3934/mbe.2024304epilepsyelectroencephalogramfeature fusionmulti-modal feature fusionmulti-head self-attention mechanism |
| spellingShingle | Ning Huang Zhengtao Xi Yingying Jiao Yudong Zhang Zhuqing Jiao Xiaona Li Multi-modal feature fusion with multi-head self-attention for epileptic EEG signals Mathematical Biosciences and Engineering epilepsy electroencephalogram feature fusion multi-modal feature fusion multi-head self-attention mechanism |
| title | Multi-modal feature fusion with multi-head self-attention for epileptic EEG signals |
| title_full | Multi-modal feature fusion with multi-head self-attention for epileptic EEG signals |
| title_fullStr | Multi-modal feature fusion with multi-head self-attention for epileptic EEG signals |
| title_full_unstemmed | Multi-modal feature fusion with multi-head self-attention for epileptic EEG signals |
| title_short | Multi-modal feature fusion with multi-head self-attention for epileptic EEG signals |
| title_sort | multi modal feature fusion with multi head self attention for epileptic eeg signals |
| topic | epilepsy electroencephalogram feature fusion multi-modal feature fusion multi-head self-attention mechanism |
| url | https://www.aimspress.com/article/doi/10.3934/mbe.2024304 |
| work_keys_str_mv | AT ninghuang multimodalfeaturefusionwithmultiheadselfattentionforepilepticeegsignals AT zhengtaoxi multimodalfeaturefusionwithmultiheadselfattentionforepilepticeegsignals AT yingyingjiao multimodalfeaturefusionwithmultiheadselfattentionforepilepticeegsignals AT yudongzhang multimodalfeaturefusionwithmultiheadselfattentionforepilepticeegsignals AT zhuqingjiao multimodalfeaturefusionwithmultiheadselfattentionforepilepticeegsignals AT xiaonali multimodalfeaturefusionwithmultiheadselfattentionforepilepticeegsignals |