Localization and detection of deepfake videos based on self-blending method
Abstract Deepfake technology, which encompasses various video manipulation techniques implemented through deep learning algorithms-such as face swapping and expression alteration-has advanced to generate fake videos that are increasingly difficult for human observers to detect, posing significant th...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-01-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-025-88523-1 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832571755753897984 |
|---|---|
| author | Junfeng Xu Xintao Liu Weiguo Lin Wenqing Shang Yuefeng Wang |
| author_facet | Junfeng Xu Xintao Liu Weiguo Lin Wenqing Shang Yuefeng Wang |
| author_sort | Junfeng Xu |
| collection | DOAJ |
| description | Abstract Deepfake technology, which encompasses various video manipulation techniques implemented through deep learning algorithms-such as face swapping and expression alteration-has advanced to generate fake videos that are increasingly difficult for human observers to detect, posing significant threats to societal security. Existing methods for detecting deepfake videos aim to identify such manipulated content to effectively prevent the spread of misinformation. However, these methods often suffer from limited generalization capabilities, exhibiting poor performance when detecting fake videos outside of their training datasets. Moreover, research on the precise localization of manipulated regions within deepfake videos is limited, primarily due to the absence of datasets with fine-grained annotations that specify which regions have been manipulated.To address these challenges, this paper proposes a novel spatial-based training method that does not require fake samples to detect spatial manipulations in deepfake videos. By employing a technique that combines multi-part local displacement deformation and fusion, we generate more diverse deepfake feature data, enhancing the detection accuracy of specific manipulation methods while producing mixed-region labels to guide manipulation localization. We utilize the Swin-Unet model for manipulation localization detection, incorporating classification loss functions, local difference loss functions, and manipulation localization loss functions to effectively improve the precision of localization and detection.Experimental results demonstrate that the proposed spatial-based training method without fake samples effectively simulates the features present in real datasets. Our method achieves satisfactory detection accuracy on datasets such as FF++, Celeb-DF, and DFDC, while accurately localizing the manipulated regions. These findings indicate the effectiveness of the proposed self-blending method and model in deepfake video detection and manipulation localization. |
| format | Article |
| id | doaj-art-cf032ffbd8d54563ad386b4ea9b0584d |
| institution | Kabale University |
| issn | 2045-2322 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj-art-cf032ffbd8d54563ad386b4ea9b0584d2025-02-02T12:23:33ZengNature PortfolioScientific Reports2045-23222025-01-0115111510.1038/s41598-025-88523-1Localization and detection of deepfake videos based on self-blending methodJunfeng Xu0Xintao Liu1Weiguo Lin2Wenqing Shang3Yuefeng Wang4Communication University of China, School of Computer & Cyber SciencesCommunication University of China, School of Computer & Cyber SciencesCommunication University of China, School of Computer & Cyber SciencesCommunication University of China, School of Computer & Cyber SciencesCommunication University of China, School of Computer & Cyber SciencesAbstract Deepfake technology, which encompasses various video manipulation techniques implemented through deep learning algorithms-such as face swapping and expression alteration-has advanced to generate fake videos that are increasingly difficult for human observers to detect, posing significant threats to societal security. Existing methods for detecting deepfake videos aim to identify such manipulated content to effectively prevent the spread of misinformation. However, these methods often suffer from limited generalization capabilities, exhibiting poor performance when detecting fake videos outside of their training datasets. Moreover, research on the precise localization of manipulated regions within deepfake videos is limited, primarily due to the absence of datasets with fine-grained annotations that specify which regions have been manipulated.To address these challenges, this paper proposes a novel spatial-based training method that does not require fake samples to detect spatial manipulations in deepfake videos. By employing a technique that combines multi-part local displacement deformation and fusion, we generate more diverse deepfake feature data, enhancing the detection accuracy of specific manipulation methods while producing mixed-region labels to guide manipulation localization. We utilize the Swin-Unet model for manipulation localization detection, incorporating classification loss functions, local difference loss functions, and manipulation localization loss functions to effectively improve the precision of localization and detection.Experimental results demonstrate that the proposed spatial-based training method without fake samples effectively simulates the features present in real datasets. Our method achieves satisfactory detection accuracy on datasets such as FF++, Celeb-DF, and DFDC, while accurately localizing the manipulated regions. These findings indicate the effectiveness of the proposed self-blending method and model in deepfake video detection and manipulation localization.https://doi.org/10.1038/s41598-025-88523-1 |
| spellingShingle | Junfeng Xu Xintao Liu Weiguo Lin Wenqing Shang Yuefeng Wang Localization and detection of deepfake videos based on self-blending method Scientific Reports |
| title | Localization and detection of deepfake videos based on self-blending method |
| title_full | Localization and detection of deepfake videos based on self-blending method |
| title_fullStr | Localization and detection of deepfake videos based on self-blending method |
| title_full_unstemmed | Localization and detection of deepfake videos based on self-blending method |
| title_short | Localization and detection of deepfake videos based on self-blending method |
| title_sort | localization and detection of deepfake videos based on self blending method |
| url | https://doi.org/10.1038/s41598-025-88523-1 |
| work_keys_str_mv | AT junfengxu localizationanddetectionofdeepfakevideosbasedonselfblendingmethod AT xintaoliu localizationanddetectionofdeepfakevideosbasedonselfblendingmethod AT weiguolin localizationanddetectionofdeepfakevideosbasedonselfblendingmethod AT wenqingshang localizationanddetectionofdeepfakevideosbasedonselfblendingmethod AT yuefengwang localizationanddetectionofdeepfakevideosbasedonselfblendingmethod |