Feasibility study of emotion mimicry analysis in human–machine interaction
Abstract Health apps have increased in popularity as people increasingly follow the advice these apps provide to enhance physical and mental well-being. One key aspect of improving neurosensory health is identifying and expressing emotions. Emotional intelligence is crucial for maintaining and enhan...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-01-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-025-87688-z |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832571754196762624 |
|---|---|
| author | Herag Arabian Tamer Abdulbaki Alshirbaji Ashish Bhave Verena Wagner-Hartl Marcel Igel J. Geoffrey Chase Knut Moeller |
| author_facet | Herag Arabian Tamer Abdulbaki Alshirbaji Ashish Bhave Verena Wagner-Hartl Marcel Igel J. Geoffrey Chase Knut Moeller |
| author_sort | Herag Arabian |
| collection | DOAJ |
| description | Abstract Health apps have increased in popularity as people increasingly follow the advice these apps provide to enhance physical and mental well-being. One key aspect of improving neurosensory health is identifying and expressing emotions. Emotional intelligence is crucial for maintaining and enhancing social interactions. In this context, a preliminary closed-loop feedback system has been developed to help people project specific emotions by altering their facial expressions. This system is part of a research intervention aimed at therapeutic applications for individuals with autism spectrum disorder. The proposed system functions as a digital mirror, initially displaying an animated avatar’s face expressing a predefined emotion. Users are then asked to mimic the avatar’s expression. During this process, a custom emotion recognition model analyzes the user’s facial expressions and provides feedback on the accuracy of their projection. A small experimental study involving 8 participants tested the system for feasibility, with avatars projecting the six basic emotions and a neutral expression. The study results indicated a positive correlation between the projected facial expressions and the emotions identified by participants. Participants effectively recognized the emotions, with 85.40% accuracy demonstrating the system’s potential in enhancing the well-being of individuals. The participants were also able to mimic the given expression effectively with an accuracy of 46.67%. However, a deficiency in the performance of one of the expressions, surprise, was noticed. In the post processing, this issue was addressed and model enhancements were tailored to boost the performance by ~ 30%. This approach shows promise for therapeutic use and emotional skill development. A further wider experimental study is still required to validate the findings of this study and analyze the impact of modifications made. |
| format | Article |
| id | doaj-art-c6ec9ab6adc74dcbad0c3b07aa88f705 |
| institution | Kabale University |
| issn | 2045-2322 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj-art-c6ec9ab6adc74dcbad0c3b07aa88f7052025-02-02T12:21:13ZengNature PortfolioScientific Reports2045-23222025-01-0115111810.1038/s41598-025-87688-zFeasibility study of emotion mimicry analysis in human–machine interactionHerag Arabian0Tamer Abdulbaki Alshirbaji1Ashish Bhave2Verena Wagner-Hartl3Marcel Igel4J. Geoffrey Chase5Knut Moeller6Institute of Technical Medicine (ITeM), Furtwangen UniversityInstitute of Technical Medicine (ITeM), Furtwangen UniversityInstitute of Technical Medicine (ITeM), Furtwangen UniversityInstitute of Technical Medicine (ITeM), Furtwangen UniversityImsimity GmbHDepartment of Mechanical Engineering, University of CanterburyInstitute of Technical Medicine (ITeM), Furtwangen UniversityAbstract Health apps have increased in popularity as people increasingly follow the advice these apps provide to enhance physical and mental well-being. One key aspect of improving neurosensory health is identifying and expressing emotions. Emotional intelligence is crucial for maintaining and enhancing social interactions. In this context, a preliminary closed-loop feedback system has been developed to help people project specific emotions by altering their facial expressions. This system is part of a research intervention aimed at therapeutic applications for individuals with autism spectrum disorder. The proposed system functions as a digital mirror, initially displaying an animated avatar’s face expressing a predefined emotion. Users are then asked to mimic the avatar’s expression. During this process, a custom emotion recognition model analyzes the user’s facial expressions and provides feedback on the accuracy of their projection. A small experimental study involving 8 participants tested the system for feasibility, with avatars projecting the six basic emotions and a neutral expression. The study results indicated a positive correlation between the projected facial expressions and the emotions identified by participants. Participants effectively recognized the emotions, with 85.40% accuracy demonstrating the system’s potential in enhancing the well-being of individuals. The participants were also able to mimic the given expression effectively with an accuracy of 46.67%. However, a deficiency in the performance of one of the expressions, surprise, was noticed. In the post processing, this issue was addressed and model enhancements were tailored to boost the performance by ~ 30%. This approach shows promise for therapeutic use and emotional skill development. A further wider experimental study is still required to validate the findings of this study and analyze the impact of modifications made.https://doi.org/10.1038/s41598-025-87688-z |
| spellingShingle | Herag Arabian Tamer Abdulbaki Alshirbaji Ashish Bhave Verena Wagner-Hartl Marcel Igel J. Geoffrey Chase Knut Moeller Feasibility study of emotion mimicry analysis in human–machine interaction Scientific Reports |
| title | Feasibility study of emotion mimicry analysis in human–machine interaction |
| title_full | Feasibility study of emotion mimicry analysis in human–machine interaction |
| title_fullStr | Feasibility study of emotion mimicry analysis in human–machine interaction |
| title_full_unstemmed | Feasibility study of emotion mimicry analysis in human–machine interaction |
| title_short | Feasibility study of emotion mimicry analysis in human–machine interaction |
| title_sort | feasibility study of emotion mimicry analysis in human machine interaction |
| url | https://doi.org/10.1038/s41598-025-87688-z |
| work_keys_str_mv | AT heragarabian feasibilitystudyofemotionmimicryanalysisinhumanmachineinteraction AT tamerabdulbakialshirbaji feasibilitystudyofemotionmimicryanalysisinhumanmachineinteraction AT ashishbhave feasibilitystudyofemotionmimicryanalysisinhumanmachineinteraction AT verenawagnerhartl feasibilitystudyofemotionmimicryanalysisinhumanmachineinteraction AT marceligel feasibilitystudyofemotionmimicryanalysisinhumanmachineinteraction AT jgeoffreychase feasibilitystudyofemotionmimicryanalysisinhumanmachineinteraction AT knutmoeller feasibilitystudyofemotionmimicryanalysisinhumanmachineinteraction |