Micro-Expression Recognition Using Convolutional Variational Attention Transformer (ConVAT) With Multihead Attention Mechanism
Micro-Expression Recognition is crucial in various fields such as behavioral analysis, security, and psychological studies, offering valuable insights into subtle and often concealed emotional states. Despite significant advancements in deep learning models, challenges persist in accurately handling...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10843702/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832542576013475840 |
|---|---|
| author | Hafiz Khizer Bin Talib Kaiwei Xu Yanlong Cao Yuan Ping Xu Zhijie Xu Muhammad Zaman Adnan Akhunzada |
| author_facet | Hafiz Khizer Bin Talib Kaiwei Xu Yanlong Cao Yuan Ping Xu Zhijie Xu Muhammad Zaman Adnan Akhunzada |
| author_sort | Hafiz Khizer Bin Talib |
| collection | DOAJ |
| description | Micro-Expression Recognition is crucial in various fields such as behavioral analysis, security, and psychological studies, offering valuable insights into subtle and often concealed emotional states. Despite significant advancements in deep learning models, challenges persist in accurately handling the nuanced and fleeting nature of micro-expressions, particularly when applied across diverse datasets with varied expressions. Existing models often struggle with precision and adaptability, leading to inconsistent recognition performance. To address these limitations, we propose the Convolutional Variational Attention Transformer (ConVAT), a novel model that leverages a multi-head attention mechanism integrated with convolutional networks, optimized specifically for detailed micro-expression analysis. Our methodology employs the Leave-One-Subject-Out (LOSO) cross-validation technique across three widely used datasets: SAMM, CASME II, and SMIC. The results demonstrate the effectiveness of ConVAT, achieving impressive performance with 98.73% accuracy on the SAMM dataset, 97.95% on the SMIC dataset, and 97.65% on CASME II. These outcomes not only surpass current state-of-the-art benchmarks but also highlight ConVAT’s robustness and reliability in capturing micro-expressions, marking a significant advancement toward developing sophisticated automated systems for real-world applications in micro-expression recognition. |
| format | Article |
| id | doaj-art-47402f4856984c2d941082a57bdb8b52 |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-47402f4856984c2d941082a57bdb8b522025-02-04T00:00:46ZengIEEEIEEE Access2169-35362025-01-0113200542007010.1109/ACCESS.2025.353011410843702Micro-Expression Recognition Using Convolutional Variational Attention Transformer (ConVAT) With Multihead Attention MechanismHafiz Khizer Bin Talib0https://orcid.org/0009-0006-7683-8389Kaiwei Xu1Yanlong Cao2https://orcid.org/0000-0003-0383-6586Yuan Ping Xu3Zhijie Xu4https://orcid.org/0000-0002-0524-5926Muhammad Zaman5https://orcid.org/0000-0001-6831-3589Adnan Akhunzada6https://orcid.org/0000-0001-8370-9290State Key Laboratory of Fluid Power and Mechatronic Systems, School of Mechanical Engineering, Zhejiang University, Hangzhou, ChinaState Key Laboratory of Fluid Power and Mechatronic Systems, School of Mechanical Engineering, Zhejiang University, Hangzhou, ChinaState Key Laboratory of Fluid Power and Mechatronic Systems, School of Mechanical Engineering, Zhejiang University, Hangzhou, ChinaSchool of Software Engineering, Chengdu University of Information Technology, Chengdu, ChinaSchool of Computing and Engineering, University of Huddersfield, Huddersfield, U.K.Department of Computer Science, COMSATS University Islamabad (CUI), Islamabad, PakistanDepartment of Data and Cybersecurity, College of Computing and IT, University of Doha for Science and Technology, Doha, QatarMicro-Expression Recognition is crucial in various fields such as behavioral analysis, security, and psychological studies, offering valuable insights into subtle and often concealed emotional states. Despite significant advancements in deep learning models, challenges persist in accurately handling the nuanced and fleeting nature of micro-expressions, particularly when applied across diverse datasets with varied expressions. Existing models often struggle with precision and adaptability, leading to inconsistent recognition performance. To address these limitations, we propose the Convolutional Variational Attention Transformer (ConVAT), a novel model that leverages a multi-head attention mechanism integrated with convolutional networks, optimized specifically for detailed micro-expression analysis. Our methodology employs the Leave-One-Subject-Out (LOSO) cross-validation technique across three widely used datasets: SAMM, CASME II, and SMIC. The results demonstrate the effectiveness of ConVAT, achieving impressive performance with 98.73% accuracy on the SAMM dataset, 97.95% on the SMIC dataset, and 97.65% on CASME II. These outcomes not only surpass current state-of-the-art benchmarks but also highlight ConVAT’s robustness and reliability in capturing micro-expressions, marking a significant advancement toward developing sophisticated automated systems for real-world applications in micro-expression recognition.https://ieeexplore.ieee.org/document/10843702/ConVATconvolutional neural networksLOSO cross-validationmicro-expression recognitionmulti-head attention |
| spellingShingle | Hafiz Khizer Bin Talib Kaiwei Xu Yanlong Cao Yuan Ping Xu Zhijie Xu Muhammad Zaman Adnan Akhunzada Micro-Expression Recognition Using Convolutional Variational Attention Transformer (ConVAT) With Multihead Attention Mechanism IEEE Access ConVAT convolutional neural networks LOSO cross-validation micro-expression recognition multi-head attention |
| title | Micro-Expression Recognition Using Convolutional Variational Attention Transformer (ConVAT) With Multihead Attention Mechanism |
| title_full | Micro-Expression Recognition Using Convolutional Variational Attention Transformer (ConVAT) With Multihead Attention Mechanism |
| title_fullStr | Micro-Expression Recognition Using Convolutional Variational Attention Transformer (ConVAT) With Multihead Attention Mechanism |
| title_full_unstemmed | Micro-Expression Recognition Using Convolutional Variational Attention Transformer (ConVAT) With Multihead Attention Mechanism |
| title_short | Micro-Expression Recognition Using Convolutional Variational Attention Transformer (ConVAT) With Multihead Attention Mechanism |
| title_sort | micro expression recognition using convolutional variational attention transformer convat with multihead attention mechanism |
| topic | ConVAT convolutional neural networks LOSO cross-validation micro-expression recognition multi-head attention |
| url | https://ieeexplore.ieee.org/document/10843702/ |
| work_keys_str_mv | AT hafizkhizerbintalib microexpressionrecognitionusingconvolutionalvariationalattentiontransformerconvatwithmultiheadattentionmechanism AT kaiweixu microexpressionrecognitionusingconvolutionalvariationalattentiontransformerconvatwithmultiheadattentionmechanism AT yanlongcao microexpressionrecognitionusingconvolutionalvariationalattentiontransformerconvatwithmultiheadattentionmechanism AT yuanpingxu microexpressionrecognitionusingconvolutionalvariationalattentiontransformerconvatwithmultiheadattentionmechanism AT zhijiexu microexpressionrecognitionusingconvolutionalvariationalattentiontransformerconvatwithmultiheadattentionmechanism AT muhammadzaman microexpressionrecognitionusingconvolutionalvariationalattentiontransformerconvatwithmultiheadattentionmechanism AT adnanakhunzada microexpressionrecognitionusingconvolutionalvariationalattentiontransformerconvatwithmultiheadattentionmechanism |