Electromyography-Based Gesture Recognition With Explainable AI (XAI): Hierarchical Feature Extraction for Enhanced Spatial-Temporal Dynamics
Hand gesture recognition using multichannel surface electromyography (sEMG) is challenging due to unstable predictions and inefficient time-varying feature enhancement. To overcome the lack of signal-based time-varying feature problems, we propose a lightweight squeeze-excitation deep learning-based...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/11004042/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850269653114290176 |
|---|---|
| author | Jungpil Shin Abu Saleh Musa Miah Sota Konnai Shu Hoshitaka Pankoo Kim |
| author_facet | Jungpil Shin Abu Saleh Musa Miah Sota Konnai Shu Hoshitaka Pankoo Kim |
| author_sort | Jungpil Shin |
| collection | DOAJ |
| description | Hand gesture recognition using multichannel surface electromyography (sEMG) is challenging due to unstable predictions and inefficient time-varying feature enhancement. To overcome the lack of signal-based time-varying feature problems, we propose a lightweight squeeze-excitation deep learning-based multi-stream spatial-temporal dynamics time-varying feature extraction approach to build an effective sEMG-based hand gesture recognition system. Each branch of the proposed model was designed to extract hierarchical features, capturing both global and detailed spatial-temporal relationships to ensure feature effectiveness. The first branch, utilizing a Bidirectional-TCN (Bi-TCN), focuses on capturing long-term temporal dependencies by modelling past and future temporal contexts, providing a holistic view of gesture dynamics. The second branch, incorporating a 1D Convolutional layer, separable CNN, and Squeeze-and-Excitation (SE) block, efficiently extracts spatial-temporal features while emphasizing critical feature channels, enhancing feature relevance. The third branch, combining a Temporal Convolutional Network (TCN) and Bidirectional LSTM (BiLSTM), captures bidirectional temporal relationships and time-varying patterns. Outputs from all branches are fused using concatenation to capture subtle variations in the data and then refined with a channel attention module, selectively focusing on the most informative features while improving computational efficiency. The proposed model was tested on the Ninapro DB2, DB4, and DB5 datasets, achieving accuracy rates of 95.31%, 92.40%, and 93.34%, respectively. Additionally, we visualize the attention maps across various classes. These results demonstrate the system’s capability to handle complex sEMG dynamics, offering advancements in prosthetic limb control and human-machine interface technologies with significant implications for assistive technologies. |
| format | Article |
| id | doaj-art-47f1d7e1f5054abba2234a40a3d4ad0e |
| institution | OA Journals |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-47f1d7e1f5054abba2234a40a3d4ad0e2025-08-20T01:53:00ZengIEEEIEEE Access2169-35362025-01-0113889308895110.1109/ACCESS.2025.356989911004042Electromyography-Based Gesture Recognition With Explainable AI (XAI): Hierarchical Feature Extraction for Enhanced Spatial-Temporal DynamicsJungpil Shin0https://orcid.org/0000-0002-7476-2468Abu Saleh Musa Miah1https://orcid.org/0000-0002-1238-0464Sota Konnai2Shu Hoshitaka3https://orcid.org/0009-0001-9358-9824Pankoo Kim4https://orcid.org/0000-0003-0111-5152School of Computer Science and Engineering, The University of Aizu, Aizuwakamatsu, JapanSchool of Computer Science and Engineering, The University of Aizu, Aizuwakamatsu, JapanSchool of Computer Science and Engineering, The University of Aizu, Aizuwakamatsu, JapanSchool of Computer Science and Engineering, The University of Aizu, Aizuwakamatsu, JapanDepartment of Computer Engineering, Chosun University, Gwangju, South KoreaHand gesture recognition using multichannel surface electromyography (sEMG) is challenging due to unstable predictions and inefficient time-varying feature enhancement. To overcome the lack of signal-based time-varying feature problems, we propose a lightweight squeeze-excitation deep learning-based multi-stream spatial-temporal dynamics time-varying feature extraction approach to build an effective sEMG-based hand gesture recognition system. Each branch of the proposed model was designed to extract hierarchical features, capturing both global and detailed spatial-temporal relationships to ensure feature effectiveness. The first branch, utilizing a Bidirectional-TCN (Bi-TCN), focuses on capturing long-term temporal dependencies by modelling past and future temporal contexts, providing a holistic view of gesture dynamics. The second branch, incorporating a 1D Convolutional layer, separable CNN, and Squeeze-and-Excitation (SE) block, efficiently extracts spatial-temporal features while emphasizing critical feature channels, enhancing feature relevance. The third branch, combining a Temporal Convolutional Network (TCN) and Bidirectional LSTM (BiLSTM), captures bidirectional temporal relationships and time-varying patterns. Outputs from all branches are fused using concatenation to capture subtle variations in the data and then refined with a channel attention module, selectively focusing on the most informative features while improving computational efficiency. The proposed model was tested on the Ninapro DB2, DB4, and DB5 datasets, achieving accuracy rates of 95.31%, 92.40%, and 93.34%, respectively. Additionally, we visualize the attention maps across various classes. These results demonstrate the system’s capability to handle complex sEMG dynamics, offering advancements in prosthetic limb control and human-machine interface technologies with significant implications for assistive technologies.https://ieeexplore.ieee.org/document/11004042/Hand gesture recognitionelectromyography (EMG)deep learningtemporal convolutional network (TCN) |
| spellingShingle | Jungpil Shin Abu Saleh Musa Miah Sota Konnai Shu Hoshitaka Pankoo Kim Electromyography-Based Gesture Recognition With Explainable AI (XAI): Hierarchical Feature Extraction for Enhanced Spatial-Temporal Dynamics IEEE Access Hand gesture recognition electromyography (EMG) deep learning temporal convolutional network (TCN) |
| title | Electromyography-Based Gesture Recognition With Explainable AI (XAI): Hierarchical Feature Extraction for Enhanced Spatial-Temporal Dynamics |
| title_full | Electromyography-Based Gesture Recognition With Explainable AI (XAI): Hierarchical Feature Extraction for Enhanced Spatial-Temporal Dynamics |
| title_fullStr | Electromyography-Based Gesture Recognition With Explainable AI (XAI): Hierarchical Feature Extraction for Enhanced Spatial-Temporal Dynamics |
| title_full_unstemmed | Electromyography-Based Gesture Recognition With Explainable AI (XAI): Hierarchical Feature Extraction for Enhanced Spatial-Temporal Dynamics |
| title_short | Electromyography-Based Gesture Recognition With Explainable AI (XAI): Hierarchical Feature Extraction for Enhanced Spatial-Temporal Dynamics |
| title_sort | electromyography based gesture recognition with explainable ai xai hierarchical feature extraction for enhanced spatial temporal dynamics |
| topic | Hand gesture recognition electromyography (EMG) deep learning temporal convolutional network (TCN) |
| url | https://ieeexplore.ieee.org/document/11004042/ |
| work_keys_str_mv | AT jungpilshin electromyographybasedgesturerecognitionwithexplainableaixaihierarchicalfeatureextractionforenhancedspatialtemporaldynamics AT abusalehmusamiah electromyographybasedgesturerecognitionwithexplainableaixaihierarchicalfeatureextractionforenhancedspatialtemporaldynamics AT sotakonnai electromyographybasedgesturerecognitionwithexplainableaixaihierarchicalfeatureextractionforenhancedspatialtemporaldynamics AT shuhoshitaka electromyographybasedgesturerecognitionwithexplainableaixaihierarchicalfeatureextractionforenhancedspatialtemporaldynamics AT pankookim electromyographybasedgesturerecognitionwithexplainableaixaihierarchicalfeatureextractionforenhancedspatialtemporaldynamics |