Real-time fear emotion recognition in mice based on multimodal data fusion
Abstract A multimodal emotion recognition method that utilizes facial expressions, body postures, and movement trajectories to detect fear in mice is proposed in this study. By integrating and analyzing these distinct data sources through feature encoders and attention classifiers, we developed a ro...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-04-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-025-95483-z |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849737736355840000 |
|---|---|
| author | Hao Wang Zhanpeng Shi Ruijie Hu Xinyi Wang Jian Chen Haoyuan Che |
| author_facet | Hao Wang Zhanpeng Shi Ruijie Hu Xinyi Wang Jian Chen Haoyuan Che |
| author_sort | Hao Wang |
| collection | DOAJ |
| description | Abstract A multimodal emotion recognition method that utilizes facial expressions, body postures, and movement trajectories to detect fear in mice is proposed in this study. By integrating and analyzing these distinct data sources through feature encoders and attention classifiers, we developed a robust emotion classification model. The performance of the model was evaluated by comparing it with single-modal methods, and the results showed significant accuracy improvements. Our findings indicate that the multimodal fusion emotion recognition model enhanced the precision of emotion detection, achieving a fear recognition accuracy of 86.7%. Additionally, the impacts of different monitoring durations and frame sampling rates on the achieved recognition accuracy were investigated in this study. The proposed method provides an efficient and simple solution for conducting real-time, comprehensive emotion monitoring in animal research, with potential applications in neuroscience and psychiatric studies. |
| format | Article |
| id | doaj-art-e8852f5a1d5045bfb341e089e94ed0ee |
| institution | DOAJ |
| issn | 2045-2322 |
| language | English |
| publishDate | 2025-04-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj-art-e8852f5a1d5045bfb341e089e94ed0ee2025-08-20T03:06:50ZengNature PortfolioScientific Reports2045-23222025-04-011511910.1038/s41598-025-95483-zReal-time fear emotion recognition in mice based on multimodal data fusionHao Wang0Zhanpeng Shi1Ruijie Hu2Xinyi Wang3Jian Chen4Haoyuan Che5Public Computer Teaching and Research Center, Jilin UniversityCollege of Veterinary Medicine, Jilin UniversityCollege of Veterinary Medicine, Jilin UniversityCollege of Computer Science and Technology, Jilin UniversityCollege of Animal Science, Jilin UniversityPublic Computer Teaching and Research Center, Jilin UniversityAbstract A multimodal emotion recognition method that utilizes facial expressions, body postures, and movement trajectories to detect fear in mice is proposed in this study. By integrating and analyzing these distinct data sources through feature encoders and attention classifiers, we developed a robust emotion classification model. The performance of the model was evaluated by comparing it with single-modal methods, and the results showed significant accuracy improvements. Our findings indicate that the multimodal fusion emotion recognition model enhanced the precision of emotion detection, achieving a fear recognition accuracy of 86.7%. Additionally, the impacts of different monitoring durations and frame sampling rates on the achieved recognition accuracy were investigated in this study. The proposed method provides an efficient and simple solution for conducting real-time, comprehensive emotion monitoring in animal research, with potential applications in neuroscience and psychiatric studies.https://doi.org/10.1038/s41598-025-95483-z |
| spellingShingle | Hao Wang Zhanpeng Shi Ruijie Hu Xinyi Wang Jian Chen Haoyuan Che Real-time fear emotion recognition in mice based on multimodal data fusion Scientific Reports |
| title | Real-time fear emotion recognition in mice based on multimodal data fusion |
| title_full | Real-time fear emotion recognition in mice based on multimodal data fusion |
| title_fullStr | Real-time fear emotion recognition in mice based on multimodal data fusion |
| title_full_unstemmed | Real-time fear emotion recognition in mice based on multimodal data fusion |
| title_short | Real-time fear emotion recognition in mice based on multimodal data fusion |
| title_sort | real time fear emotion recognition in mice based on multimodal data fusion |
| url | https://doi.org/10.1038/s41598-025-95483-z |
| work_keys_str_mv | AT haowang realtimefearemotionrecognitioninmicebasedonmultimodaldatafusion AT zhanpengshi realtimefearemotionrecognitioninmicebasedonmultimodaldatafusion AT ruijiehu realtimefearemotionrecognitioninmicebasedonmultimodaldatafusion AT xinyiwang realtimefearemotionrecognitioninmicebasedonmultimodaldatafusion AT jianchen realtimefearemotionrecognitioninmicebasedonmultimodaldatafusion AT haoyuanche realtimefearemotionrecognitioninmicebasedonmultimodaldatafusion |