Selective Auditory Attention Detection Using Combined Transformer and Convolutional Graph Neural Networks
Attention is one of many human cognitive functions that are essential in everyday life. Given our limited processing capacity, attention helps us focus only on what matters. Focusing attention on one speaker in an environment with many speakers is a critical ability of the human auditory system. Thi...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-11-01
|
| Series: | Bioengineering |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2306-5354/11/12/1216 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850036654860926976 |
|---|---|
| author | Masoud Geravanchizadeh Amir Shaygan Asl Sebelan Danishvar |
| author_facet | Masoud Geravanchizadeh Amir Shaygan Asl Sebelan Danishvar |
| author_sort | Masoud Geravanchizadeh |
| collection | DOAJ |
| description | Attention is one of many human cognitive functions that are essential in everyday life. Given our limited processing capacity, attention helps us focus only on what matters. Focusing attention on one speaker in an environment with many speakers is a critical ability of the human auditory system. This paper proposes a new end-to-end method based on the combined transformer and graph convolutional neural network (TraGCNN) that can effectively detect auditory attention from electroencephalograms (EEGs). This approach eliminates the need for manual feature extraction, which is often time-consuming and subjective. Here, the first EEG signals are converted to graphs. We then extract attention information from these graphs using spatial and temporal approaches. Finally, our models are trained with these data. Our model can detect auditory attention in both the spatial and temporal domains. Here, the EEG input is first processed by transformer layers to obtain a sequential representation of EEG based on attention onsets. Then, a family of graph convolutional layers is used to find the most active electrodes using the spatial position of electrodes. Finally, the corresponding EEG features of active electrodes are fed into the graph attention layers to detect auditory attention. The Fuglsang 2020 dataset is used in the experiments to train and test the proposed and baseline systems. The new TraGCNN approach, as compared with state-of-the-art attention classification methods from the literature, yields the highest performance in terms of accuracy (80.12%) as a classification metric. Additionally, the proposed model results in higher performance than our previously graph-based model for different lengths of EEG segments. The new TraGCNN approach is advantageous because attenuation detection is achieved from EEG signals of subjects without requiring speech stimuli, as is the case with conventional auditory attention detection methods. Furthermore, examining the proposed model for different lengths of EEG segments shows that the model is faster than our previous graph-based detection method in terms of computational complexity. The findings of this study have important implications for the understanding and assessment of auditory attention, which is crucial for many applications, such as brain–computer interface (BCI) systems, speech separation, and neuro-steered hearing aid development. |
| format | Article |
| id | doaj-art-ebea3e7b70fd456fa96bfbd689771ca3 |
| institution | DOAJ |
| issn | 2306-5354 |
| language | English |
| publishDate | 2024-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Bioengineering |
| spelling | doaj-art-ebea3e7b70fd456fa96bfbd689771ca32025-08-20T02:57:05ZengMDPI AGBioengineering2306-53542024-11-011112121610.3390/bioengineering11121216Selective Auditory Attention Detection Using Combined Transformer and Convolutional Graph Neural NetworksMasoud Geravanchizadeh0Amir Shaygan Asl1Sebelan Danishvar2Faculty of Electrical & Computer Engineering, University of Tabriz, Tabriz 51666-15813, IranFaculty of Electrical & Computer Engineering, University of Tabriz, Tabriz 51666-15813, IranCollege of Engineering, Design and Physical Sciences, Brunel University London, London UB8 3PH, UKAttention is one of many human cognitive functions that are essential in everyday life. Given our limited processing capacity, attention helps us focus only on what matters. Focusing attention on one speaker in an environment with many speakers is a critical ability of the human auditory system. This paper proposes a new end-to-end method based on the combined transformer and graph convolutional neural network (TraGCNN) that can effectively detect auditory attention from electroencephalograms (EEGs). This approach eliminates the need for manual feature extraction, which is often time-consuming and subjective. Here, the first EEG signals are converted to graphs. We then extract attention information from these graphs using spatial and temporal approaches. Finally, our models are trained with these data. Our model can detect auditory attention in both the spatial and temporal domains. Here, the EEG input is first processed by transformer layers to obtain a sequential representation of EEG based on attention onsets. Then, a family of graph convolutional layers is used to find the most active electrodes using the spatial position of electrodes. Finally, the corresponding EEG features of active electrodes are fed into the graph attention layers to detect auditory attention. The Fuglsang 2020 dataset is used in the experiments to train and test the proposed and baseline systems. The new TraGCNN approach, as compared with state-of-the-art attention classification methods from the literature, yields the highest performance in terms of accuracy (80.12%) as a classification metric. Additionally, the proposed model results in higher performance than our previously graph-based model for different lengths of EEG segments. The new TraGCNN approach is advantageous because attenuation detection is achieved from EEG signals of subjects without requiring speech stimuli, as is the case with conventional auditory attention detection methods. Furthermore, examining the proposed model for different lengths of EEG segments shows that the model is faster than our previous graph-based detection method in terms of computational complexity. The findings of this study have important implications for the understanding and assessment of auditory attention, which is crucial for many applications, such as brain–computer interface (BCI) systems, speech separation, and neuro-steered hearing aid development.https://www.mdpi.com/2306-5354/11/12/1216selective auditory attention detectiongraph neural networktransformerconvolutional neural networksbrain connectivityhybrid neural networks |
| spellingShingle | Masoud Geravanchizadeh Amir Shaygan Asl Sebelan Danishvar Selective Auditory Attention Detection Using Combined Transformer and Convolutional Graph Neural Networks Bioengineering selective auditory attention detection graph neural network transformer convolutional neural networks brain connectivity hybrid neural networks |
| title | Selective Auditory Attention Detection Using Combined Transformer and Convolutional Graph Neural Networks |
| title_full | Selective Auditory Attention Detection Using Combined Transformer and Convolutional Graph Neural Networks |
| title_fullStr | Selective Auditory Attention Detection Using Combined Transformer and Convolutional Graph Neural Networks |
| title_full_unstemmed | Selective Auditory Attention Detection Using Combined Transformer and Convolutional Graph Neural Networks |
| title_short | Selective Auditory Attention Detection Using Combined Transformer and Convolutional Graph Neural Networks |
| title_sort | selective auditory attention detection using combined transformer and convolutional graph neural networks |
| topic | selective auditory attention detection graph neural network transformer convolutional neural networks brain connectivity hybrid neural networks |
| url | https://www.mdpi.com/2306-5354/11/12/1216 |
| work_keys_str_mv | AT masoudgeravanchizadeh selectiveauditoryattentiondetectionusingcombinedtransformerandconvolutionalgraphneuralnetworks AT amirshayganasl selectiveauditoryattentiondetectionusingcombinedtransformerandconvolutionalgraphneuralnetworks AT sebelandanishvar selectiveauditoryattentiondetectionusingcombinedtransformerandconvolutionalgraphneuralnetworks |