Multi-branch convolutional neural network with cross-attention mechanism for emotion recognition
Abstract Research on emotion recognition is an interesting area because of its wide-ranging applications in education, marketing, and medical fields. This study proposes a multi-branch convolutional neural network model based on cross-attention mechanism (MCNN-CA) for accurate recognition of differe...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-02-01
|
| Series: | Scientific Reports |
| Subjects: | |
| Online Access: | https://doi.org/10.1038/s41598-025-88248-1 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832571654384910336 |
|---|---|
| author | Fei Yan Zekai Guo Abdullah M. Iliyasu Kaoru Hirota |
| author_facet | Fei Yan Zekai Guo Abdullah M. Iliyasu Kaoru Hirota |
| author_sort | Fei Yan |
| collection | DOAJ |
| description | Abstract Research on emotion recognition is an interesting area because of its wide-ranging applications in education, marketing, and medical fields. This study proposes a multi-branch convolutional neural network model based on cross-attention mechanism (MCNN-CA) for accurate recognition of different emotions. The proposed model provides automated extraction of relevant features from multimodal data and fusion of feature maps from diverse sources as modules for the subsequent emotion recognition. In the feature extraction stage, various convolutional neural networks were designed to extract critical information from multiple dimensional features. The feature fusion module was used to enhance the inter-correlation between features based on channel-efficient attention mechanism. This innovation proves effective in fusing distinctive features within a single mode and across different modes. The model was assessed based on EEG emotion recognition experiments on the SEED and SEED-IV datasets. Furthermore, the efficiency of the proposed model was evaluated via multimodal emotion experiments using EEG and text data from the ZuCo dataset. Comparative analysis alongside contemporary studies shows that our model excels in terms of accuracy, precision, recall, and F1-score. |
| format | Article |
| id | doaj-art-55ef23cf2e0542238dd853e2b8805cb5 |
| institution | Kabale University |
| issn | 2045-2322 |
| language | English |
| publishDate | 2025-02-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj-art-55ef23cf2e0542238dd853e2b8805cb52025-02-02T12:24:10ZengNature PortfolioScientific Reports2045-23222025-02-0115111810.1038/s41598-025-88248-1Multi-branch convolutional neural network with cross-attention mechanism for emotion recognitionFei Yan0Zekai Guo1Abdullah M. Iliyasu2Kaoru Hirota3School of Computer Science and Technology, Changchun University of Science and TechnologySchool of Computer Science and Technology, Changchun University of Science and TechnologyCollege of Engineering, Prince Sattam Bin Abdulaziz UniversitySchool of Computing, Tokyo Institute of TechnologyAbstract Research on emotion recognition is an interesting area because of its wide-ranging applications in education, marketing, and medical fields. This study proposes a multi-branch convolutional neural network model based on cross-attention mechanism (MCNN-CA) for accurate recognition of different emotions. The proposed model provides automated extraction of relevant features from multimodal data and fusion of feature maps from diverse sources as modules for the subsequent emotion recognition. In the feature extraction stage, various convolutional neural networks were designed to extract critical information from multiple dimensional features. The feature fusion module was used to enhance the inter-correlation between features based on channel-efficient attention mechanism. This innovation proves effective in fusing distinctive features within a single mode and across different modes. The model was assessed based on EEG emotion recognition experiments on the SEED and SEED-IV datasets. Furthermore, the efficiency of the proposed model was evaluated via multimodal emotion experiments using EEG and text data from the ZuCo dataset. Comparative analysis alongside contemporary studies shows that our model excels in terms of accuracy, precision, recall, and F1-score.https://doi.org/10.1038/s41598-025-88248-1Biomedical engineeringEEG signalEmotion recognitionFeature fusionConvolutional neural network |
| spellingShingle | Fei Yan Zekai Guo Abdullah M. Iliyasu Kaoru Hirota Multi-branch convolutional neural network with cross-attention mechanism for emotion recognition Scientific Reports Biomedical engineering EEG signal Emotion recognition Feature fusion Convolutional neural network |
| title | Multi-branch convolutional neural network with cross-attention mechanism for emotion recognition |
| title_full | Multi-branch convolutional neural network with cross-attention mechanism for emotion recognition |
| title_fullStr | Multi-branch convolutional neural network with cross-attention mechanism for emotion recognition |
| title_full_unstemmed | Multi-branch convolutional neural network with cross-attention mechanism for emotion recognition |
| title_short | Multi-branch convolutional neural network with cross-attention mechanism for emotion recognition |
| title_sort | multi branch convolutional neural network with cross attention mechanism for emotion recognition |
| topic | Biomedical engineering EEG signal Emotion recognition Feature fusion Convolutional neural network |
| url | https://doi.org/10.1038/s41598-025-88248-1 |
| work_keys_str_mv | AT feiyan multibranchconvolutionalneuralnetworkwithcrossattentionmechanismforemotionrecognition AT zekaiguo multibranchconvolutionalneuralnetworkwithcrossattentionmechanismforemotionrecognition AT abdullahmiliyasu multibranchconvolutionalneuralnetworkwithcrossattentionmechanismforemotionrecognition AT kaoruhirota multibranchconvolutionalneuralnetworkwithcrossattentionmechanismforemotionrecognition |