Facial expression recognition based on adaptation of the classifier to videos of the user
In this paper, an approach that can significantly increase the accuracy of facial emotion recognition by adapting the model to the emotions of a particular user (e.g., smartphone owner) is considered. At the first stage, a neural network model, which was previously trained to recognize facial expres...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Samara National Research University
2023-10-01
|
| Series: | Компьютерная оптика |
| Subjects: | |
| Online Access: | https://www.computeroptics.ru/eng/KO/Annot/KO47-5/470515e.html |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832590904822595584 |
|---|---|
| author | E.N. Churaev A.V. Savchenko |
| author_facet | E.N. Churaev A.V. Savchenko |
| author_sort | E.N. Churaev |
| collection | DOAJ |
| description | In this paper, an approach that can significantly increase the accuracy of facial emotion recognition by adapting the model to the emotions of a particular user (e.g., smartphone owner) is considered. At the first stage, a neural network model, which was previously trained to recognize facial expressions in static photos, is used to extract visual features of faces in each frame. Next, the face features of video frames are aggregated into a single descriptor for a short video fragment. After that a neural network classifier is trained. At the second stage, it is proposed that adaptation (fine-tuning) to this classifier should be performed using a small set of video data with the facial expressions of a particular user. After emotion classification, the user can adjust the predicted emotions to further improve the accuracy of a personal model. As part of an experimental study for the RAVDESS dataset, it has been shown that the approach with model adaptation to a specific user can significantly (up to 20 – 50 %) improve the accuracy of facial expression recognition in the video. |
| format | Article |
| id | doaj-art-b2ed01403c3f4806b25bc3bbcc525702 |
| institution | Kabale University |
| issn | 0134-2452 2412-6179 |
| language | English |
| publishDate | 2023-10-01 |
| publisher | Samara National Research University |
| record_format | Article |
| series | Компьютерная оптика |
| spelling | doaj-art-b2ed01403c3f4806b25bc3bbcc5257022025-01-23T05:58:49ZengSamara National Research UniversityКомпьютерная оптика0134-24522412-61792023-10-0147580681510.18287/2412-6179-CO-1269Facial expression recognition based on adaptation of the classifier to videos of the userE.N. Churaev0A.V. Savchenko1HSE University, Laboratory of Algorithms and Technologies for Networks AnalysisHSE University, Laboratory of Algorithms and Technologies for Networks Analysis; Sber AIIn this paper, an approach that can significantly increase the accuracy of facial emotion recognition by adapting the model to the emotions of a particular user (e.g., smartphone owner) is considered. At the first stage, a neural network model, which was previously trained to recognize facial expressions in static photos, is used to extract visual features of faces in each frame. Next, the face features of video frames are aggregated into a single descriptor for a short video fragment. After that a neural network classifier is trained. At the second stage, it is proposed that adaptation (fine-tuning) to this classifier should be performed using a small set of video data with the facial expressions of a particular user. After emotion classification, the user can adjust the predicted emotions to further improve the accuracy of a personal model. As part of an experimental study for the RAVDESS dataset, it has been shown that the approach with model adaptation to a specific user can significantly (up to 20 – 50 %) improve the accuracy of facial expression recognition in the video.https://www.computeroptics.ru/eng/KO/Annot/KO47-5/470515e.htmlfacial expression classificationneural network classifier adaptationspeaker-dependent emotion recognition |
| spellingShingle | E.N. Churaev A.V. Savchenko Facial expression recognition based on adaptation of the classifier to videos of the user Компьютерная оптика facial expression classification neural network classifier adaptation speaker-dependent emotion recognition |
| title | Facial expression recognition based on adaptation of the classifier to videos of the user |
| title_full | Facial expression recognition based on adaptation of the classifier to videos of the user |
| title_fullStr | Facial expression recognition based on adaptation of the classifier to videos of the user |
| title_full_unstemmed | Facial expression recognition based on adaptation of the classifier to videos of the user |
| title_short | Facial expression recognition based on adaptation of the classifier to videos of the user |
| title_sort | facial expression recognition based on adaptation of the classifier to videos of the user |
| topic | facial expression classification neural network classifier adaptation speaker-dependent emotion recognition |
| url | https://www.computeroptics.ru/eng/KO/Annot/KO47-5/470515e.html |
| work_keys_str_mv | AT enchuraev facialexpressionrecognitionbasedonadaptationoftheclassifiertovideosoftheuser AT avsavchenko facialexpressionrecognitionbasedonadaptationoftheclassifiertovideosoftheuser |