Develop an emotion recognition system using jointly connectivity between electroencephalogram and electrocardiogram signals
This study pioneers an innovative approach to improve the accuracy and dependability of emotion recognition (ER) systems by integrating electroencephalogram (EEG) with electrocardiogram (ECG) data. We propose a novel method of estimating effective connectivity (EC) to capture the dynamic interplay b...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2025-01-01
|
| Series: | Heliyon |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2405844025001471 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832573189192941568 |
|---|---|
| author | Javid Farhadi Sedehi Nader Jafarnia Dabanloo Keivan Maghooli Ali Sheikhani |
| author_facet | Javid Farhadi Sedehi Nader Jafarnia Dabanloo Keivan Maghooli Ali Sheikhani |
| author_sort | Javid Farhadi Sedehi |
| collection | DOAJ |
| description | This study pioneers an innovative approach to improve the accuracy and dependability of emotion recognition (ER) systems by integrating electroencephalogram (EEG) with electrocardiogram (ECG) data. We propose a novel method of estimating effective connectivity (EC) to capture the dynamic interplay between the heart and brain during emotions of happiness, disgust, fear, and sadness. Leveraging three EC estimation techniques (Granger causality (GC), partial directed coherence (PDC) and directed transfer function (DTF)), we feed the resulting EC representations as inputs into convolutional neural networks (CNNs), namely ResNet-18 and MobileNetV2, known for their swift and superior performance. To evaluate this approach, we have used EEG and ECG data from the public MAHNOB-HCI database through 5-fold cross-validation criterion. Remarkably, our approach achieves an average accuracy of 97.34 ± 1.19 and 96.53 ± 3.54 for DTF images within the alpha frequency band using ResNet-18 and MobileNetV2, respectively. Comparative analyses in comparison to cutting-edge research unequivocally prove the efficacy of augmenting ECG with EEG, showcasing substantial improvements in ER performance across the spectrum of emotions studied. |
| format | Article |
| id | doaj-art-fd859d3df60446a49a2f358262f2d986 |
| institution | Kabale University |
| issn | 2405-8440 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Heliyon |
| spelling | doaj-art-fd859d3df60446a49a2f358262f2d9862025-02-02T05:28:08ZengElsevierHeliyon2405-84402025-01-01112e41767Develop an emotion recognition system using jointly connectivity between electroencephalogram and electrocardiogram signalsJavid Farhadi Sedehi0Nader Jafarnia Dabanloo1Keivan Maghooli2Ali Sheikhani3Department of Biomedical Engineering, Science and Research Branch, Islamic Azad University, Tehran, IranCorresponding author.; Department of Biomedical Engineering, Science and Research Branch, Islamic Azad University, Tehran, IranDepartment of Biomedical Engineering, Science and Research Branch, Islamic Azad University, Tehran, IranDepartment of Biomedical Engineering, Science and Research Branch, Islamic Azad University, Tehran, IranThis study pioneers an innovative approach to improve the accuracy and dependability of emotion recognition (ER) systems by integrating electroencephalogram (EEG) with electrocardiogram (ECG) data. We propose a novel method of estimating effective connectivity (EC) to capture the dynamic interplay between the heart and brain during emotions of happiness, disgust, fear, and sadness. Leveraging three EC estimation techniques (Granger causality (GC), partial directed coherence (PDC) and directed transfer function (DTF)), we feed the resulting EC representations as inputs into convolutional neural networks (CNNs), namely ResNet-18 and MobileNetV2, known for their swift and superior performance. To evaluate this approach, we have used EEG and ECG data from the public MAHNOB-HCI database through 5-fold cross-validation criterion. Remarkably, our approach achieves an average accuracy of 97.34 ± 1.19 and 96.53 ± 3.54 for DTF images within the alpha frequency band using ResNet-18 and MobileNetV2, respectively. Comparative analyses in comparison to cutting-edge research unequivocally prove the efficacy of augmenting ECG with EEG, showcasing substantial improvements in ER performance across the spectrum of emotions studied.http://www.sciencedirect.com/science/article/pii/S2405844025001471Coupling EEG-ECGConvolutional neural network (CNN)Effective connectivityEmotion recognitionTransfer learning |
| spellingShingle | Javid Farhadi Sedehi Nader Jafarnia Dabanloo Keivan Maghooli Ali Sheikhani Develop an emotion recognition system using jointly connectivity between electroencephalogram and electrocardiogram signals Heliyon Coupling EEG-ECG Convolutional neural network (CNN) Effective connectivity Emotion recognition Transfer learning |
| title | Develop an emotion recognition system using jointly connectivity between electroencephalogram and electrocardiogram signals |
| title_full | Develop an emotion recognition system using jointly connectivity between electroencephalogram and electrocardiogram signals |
| title_fullStr | Develop an emotion recognition system using jointly connectivity between electroencephalogram and electrocardiogram signals |
| title_full_unstemmed | Develop an emotion recognition system using jointly connectivity between electroencephalogram and electrocardiogram signals |
| title_short | Develop an emotion recognition system using jointly connectivity between electroencephalogram and electrocardiogram signals |
| title_sort | develop an emotion recognition system using jointly connectivity between electroencephalogram and electrocardiogram signals |
| topic | Coupling EEG-ECG Convolutional neural network (CNN) Effective connectivity Emotion recognition Transfer learning |
| url | http://www.sciencedirect.com/science/article/pii/S2405844025001471 |
| work_keys_str_mv | AT javidfarhadisedehi developanemotionrecognitionsystemusingjointlyconnectivitybetweenelectroencephalogramandelectrocardiogramsignals AT naderjafarniadabanloo developanemotionrecognitionsystemusingjointlyconnectivitybetweenelectroencephalogramandelectrocardiogramsignals AT keivanmaghooli developanemotionrecognitionsystemusingjointlyconnectivitybetweenelectroencephalogramandelectrocardiogramsignals AT alisheikhani developanemotionrecognitionsystemusingjointlyconnectivitybetweenelectroencephalogramandelectrocardiogramsignals |