Mf-net: multi-feature fusion network based on two-stream extraction and multi-scale enhancement for face forgery detection
Abstract Due to the increasing sophistication of face forgery techniques, the images generated are becoming more and more realistic and difficult for human eyes to distinguish. These face forgery techniques can cause problems such as fraud and social engineering attacks in facial recognition and ide...
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Springer
2024-11-01
|
| Series: | Complex & Intelligent Systems |
| Subjects: | |
| Online Access: | https://doi.org/10.1007/s40747-024-01634-6 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832571153789485056 |
|---|---|
| author | Hanxian Duan Qian Jiang Xin Jin Michal Wozniak Yi Zhao Liwen Wu Shaowen Yao Wei Zhou |
| author_facet | Hanxian Duan Qian Jiang Xin Jin Michal Wozniak Yi Zhao Liwen Wu Shaowen Yao Wei Zhou |
| author_sort | Hanxian Duan |
| collection | DOAJ |
| description | Abstract Due to the increasing sophistication of face forgery techniques, the images generated are becoming more and more realistic and difficult for human eyes to distinguish. These face forgery techniques can cause problems such as fraud and social engineering attacks in facial recognition and identity verification areas. Therefore, researchers have worked on face forgery detection studies and have made significant progress. Current face forgery detection algorithms achieve high detection accuracy within-dataset. However, it is difficult to achieve satisfactory generalization performance in cross-dataset scenarios. In order to improve the cross-dataset detection performance of the model, this paper proposes a multi-feature fusion network based on two-stream extraction and multi-scale enhancement. First, we design a two-stream feature extraction module to obtain richer feature information. Secondly, the multi-scale feature enhancement module is proposed to focus the model more on information related to the current sub-region from different scales. Finally, the forgery detection module calculates the overlap between the features of the input image and real images during the training phase to determine the forgery regions. The method encourages the model to mine forgery features and learns generic and robust features not limited to a particular feature. Thus, the model achieves high detection accuracy and performance. We achieve the AUC of 99.70% and 90.71% on FaceForensics++ and WildDeepfake datasets. The generalization experiments on Celeb-DF-v2 and WildDeepfake datasets achieve the AUC of 80.16% and 65.15%. Comparison experiments with multiple methods on other benchmark datasets confirm the superior generalization performance of our proposed method while ensuring model detection accuracy. Our code can be found at: https://github.com/1241128239/MFNet . |
| format | Article |
| id | doaj-art-2d5b748f6b924effb6a5cddacfadc652 |
| institution | Kabale University |
| issn | 2199-4536 2198-6053 |
| language | English |
| publishDate | 2024-11-01 |
| publisher | Springer |
| record_format | Article |
| series | Complex & Intelligent Systems |
| spelling | doaj-art-2d5b748f6b924effb6a5cddacfadc6522025-02-02T12:49:38ZengSpringerComplex & Intelligent Systems2199-45362198-60532024-11-0111111510.1007/s40747-024-01634-6Mf-net: multi-feature fusion network based on two-stream extraction and multi-scale enhancement for face forgery detectionHanxian Duan0Qian Jiang1Xin Jin2Michal Wozniak3Yi Zhao4Liwen Wu5Shaowen Yao6Wei Zhou7Engineering Research Center of Cyberspace, Yunnan UniversityEngineering Research Center of Cyberspace, Yunnan UniversityEngineering Research Center of Cyberspace, Yunnan UniversityInformation and Communication Technology, Wroclaw University of Science and TechnologyEngineering Research Center of Cyberspace, Yunnan UniversityEngineering Research Center of Cyberspace, Yunnan UniversityEngineering Research Center of Cyberspace, Yunnan UniversityEngineering Research Center of Cyberspace, Yunnan UniversityAbstract Due to the increasing sophistication of face forgery techniques, the images generated are becoming more and more realistic and difficult for human eyes to distinguish. These face forgery techniques can cause problems such as fraud and social engineering attacks in facial recognition and identity verification areas. Therefore, researchers have worked on face forgery detection studies and have made significant progress. Current face forgery detection algorithms achieve high detection accuracy within-dataset. However, it is difficult to achieve satisfactory generalization performance in cross-dataset scenarios. In order to improve the cross-dataset detection performance of the model, this paper proposes a multi-feature fusion network based on two-stream extraction and multi-scale enhancement. First, we design a two-stream feature extraction module to obtain richer feature information. Secondly, the multi-scale feature enhancement module is proposed to focus the model more on information related to the current sub-region from different scales. Finally, the forgery detection module calculates the overlap between the features of the input image and real images during the training phase to determine the forgery regions. The method encourages the model to mine forgery features and learns generic and robust features not limited to a particular feature. Thus, the model achieves high detection accuracy and performance. We achieve the AUC of 99.70% and 90.71% on FaceForensics++ and WildDeepfake datasets. The generalization experiments on Celeb-DF-v2 and WildDeepfake datasets achieve the AUC of 80.16% and 65.15%. Comparison experiments with multiple methods on other benchmark datasets confirm the superior generalization performance of our proposed method while ensuring model detection accuracy. Our code can be found at: https://github.com/1241128239/MFNet .https://doi.org/10.1007/s40747-024-01634-6Deepfake detectionFace forgery detectionDeep learningAttention mechanismFeature extractionFeature enhancement |
| spellingShingle | Hanxian Duan Qian Jiang Xin Jin Michal Wozniak Yi Zhao Liwen Wu Shaowen Yao Wei Zhou Mf-net: multi-feature fusion network based on two-stream extraction and multi-scale enhancement for face forgery detection Complex & Intelligent Systems Deepfake detection Face forgery detection Deep learning Attention mechanism Feature extraction Feature enhancement |
| title | Mf-net: multi-feature fusion network based on two-stream extraction and multi-scale enhancement for face forgery detection |
| title_full | Mf-net: multi-feature fusion network based on two-stream extraction and multi-scale enhancement for face forgery detection |
| title_fullStr | Mf-net: multi-feature fusion network based on two-stream extraction and multi-scale enhancement for face forgery detection |
| title_full_unstemmed | Mf-net: multi-feature fusion network based on two-stream extraction and multi-scale enhancement for face forgery detection |
| title_short | Mf-net: multi-feature fusion network based on two-stream extraction and multi-scale enhancement for face forgery detection |
| title_sort | mf net multi feature fusion network based on two stream extraction and multi scale enhancement for face forgery detection |
| topic | Deepfake detection Face forgery detection Deep learning Attention mechanism Feature extraction Feature enhancement |
| url | https://doi.org/10.1007/s40747-024-01634-6 |
| work_keys_str_mv | AT hanxianduan mfnetmultifeaturefusionnetworkbasedontwostreamextractionandmultiscaleenhancementforfaceforgerydetection AT qianjiang mfnetmultifeaturefusionnetworkbasedontwostreamextractionandmultiscaleenhancementforfaceforgerydetection AT xinjin mfnetmultifeaturefusionnetworkbasedontwostreamextractionandmultiscaleenhancementforfaceforgerydetection AT michalwozniak mfnetmultifeaturefusionnetworkbasedontwostreamextractionandmultiscaleenhancementforfaceforgerydetection AT yizhao mfnetmultifeaturefusionnetworkbasedontwostreamextractionandmultiscaleenhancementforfaceforgerydetection AT liwenwu mfnetmultifeaturefusionnetworkbasedontwostreamextractionandmultiscaleenhancementforfaceforgerydetection AT shaowenyao mfnetmultifeaturefusionnetworkbasedontwostreamextractionandmultiscaleenhancementforfaceforgerydetection AT weizhou mfnetmultifeaturefusionnetworkbasedontwostreamextractionandmultiscaleenhancementforfaceforgerydetection |