Abstract With the continuous development of artificial intelligence-generated content technology, the diversity of forgery techniques presents significant challenges to existing detection methods. Most current detection methods are based on facial forgery features extracted by different advanced convolutional neural networks. However, these methods are trained on datasets containing known forgery techniques, and their generalization capabilities are inadequate to handle images forged by unknown methods. Therefore, a face forgery detection method combined with deep forgery features comparison is proposed, and it exhibits excellent adaptability to unknown forgery techniques. The proposed approach consists of two stages. First, similar features of different forgery techniques are explored, and a meta-learning-based similar feature fusion network is introduced. This network leverages the learning capabilities of meta-learning to capture the similar features among different forgery methods. Second, unique forgery features specific to individual task are taken into account, and a task-specific uniqueness fine-tuning method is proposed to enhance the adaptability of the model to unknown forgery techniques.Cross-manipulation testing demonstrates that the proposed method improves the performance with superior detection capability against attacks from unknown forgery techniques.
Fund:General Project of National Natural Science Foundation of China(No.62272141)
Corresponding Authors:
GAO Xinjian, Ph.D., associate professor. His research interests include image processing, deep learning, artificial intelligence and machine learning.
About author:: GAO Jun, Ph.D., professor. His research interests include image processing, pattern recognition, neural network theory and applications, optoelectronic information processing and intelligent information proce-ssing. DA Zikai, Ph.D. candidate. His research interests include image processing, deep lear-ning, artificial intelligence and machine lear-ning. LI Zhaowei, Master student. His research interests include face forgery detection, computer vision and graphic processing.
LI Zhaowei,GAO Xinjian,DA Zikai等. Face Forgery Detection Combined with Deep Forgery Features Comparison[J]. Pattern Recognition and Artificial Intelligence, 2024, 37(9): 786-797.
[1] GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Genera-tive Adversarial Networks. Communications of the ACM, 2020, 63(11): 139-144. [2] KINGMA D P, WELLING M.Auto-Encoding Variational Bayes[C/OL].[2024-05-23].https://arxiv.org/pdf/1312.6114. [3] LI J M, XIE H T, LI J H, et al. Frequency-Aware Discriminative Feature Learning Supervised by Single-Center Loss for Face Forgery Detection//Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2021: 6454-6463. [4] 朱新同,唐云祁,耿鹏志.基于特征融合的篡改与深度伪造图像检测算法.信息网络安全, 2021, 21(8): 70-81. (ZHU X T, TANG Y Q, GENG P Z.Detection Algorithm of Tamper and Deepfake Image Based on Feature Fusion. Netinfo Security, 2021, 21(8): 70-81.) [5] ZHAO H Q, WEI T, ZHOU W B, et al. Multi-attentional Deepfake Detection//Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2021: 2185-2194. [6] NGUYEN H H, FANG F M, YAMAGISHI J, et al. Multi-task Lear-ning for Detecting and Segmenting Manipulated Facial Images and Videos//Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2019. DOI: 10.1109/BTAS46853.2019.9185974. [7] CHEN L, ZHANG Y, SONG Y B, et al. Self-Supervised Learning of Adversarial Example: Towards Good Generalizations for Deepfake Detection//Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2022: 18689-18698. [8] HUANG P H, HAN Y H, CHU E, et al. Multi-task Self-Blended Images for Face Forgery Detection//Proc of the 5th ACM International Conference on Multimedia in Asia. New York,USA: ACM, 2023. DOI: 10.1145/3595916.3626426. [9] LIN Y K, YEN T Y.A Meta-Learning Approach for Few-Shot Face Forgery Segmentation and Classification. Sensors, 2023, 23(7). DOI: 10.3390/s23073647. [10] WANG C R, DENG W H.Representative Forgery Mining for Fake Face Detection//Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2021: 14918-14927. [11] WODAJO D, ATNAFU S.Deepfake Video Detection Using Convolutional Vision Transformer[C/OL].[2024-05-23].https://arxiv.org/pdf/2102.11126. [12] BAYAR B, STAMM M C.A Deep Learning Approach to Universal Image Manipulation Detection Using a New Convolutional Layer//Proc of the 4th ACM Workshop on Information Hiding and Multimedia Security. New York, USA: ACM, 2016: 5-10. [13] CAO J Y, MA C, YAO T P, et al. End-to-End Reconstruction-Classification Learning for Face Forgery Detection//Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2022: 4103-4112. [14] BAI W M, LIU Y F, ZHANG Z P, et al. AUNet: Learning Relations between Action Units for Face Forgery Detection//Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2023: 24709-24719. [15] HUANG Z Q, CHAN K C K, JIANG Y M, et al. Collaborative Diffusion for Multi-modal Face Generation and Editing//Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2023: 6080-6090. [16] KUMAR A, BHAVSAR A, VERMA R.Detecting Deepfakes with Metric Learning//Proc of the 8th International Workshop on Bio-metrics and Forensics. Washington, USA: IEEE, 2020. DOI: 10.1109/IWBF49977.2020.9107962. [17] SELVARAJU R R, COGSWELL M, DAS A, et al. Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Loca-lization//Proc of the IEEE International Conference on Computer Vision. Washington, USA: IEEE, 2017: 618-626. [18] CHOLLET F.Xception: Deep Learning with Depthwise Separable Convolutions//Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 1800-1807. [19] LI Y Z, YANG X, SUN P, et al. Celeb-DF: A Large-Scale Cha-llenging Dataset for DeepFake Forensics//Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2020: 3204-3213. [20] ZHAO W L, RAO Y M, SHI W K, et al. DiffSwap: High-Fidelity and Controllable Face Swapping via 3D-Aware Masked Diffusion//Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2023: 8568-8577. [21] FINN C, ABBEEL P, LEVINE S.Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. Journal of Machine Learning Research, 2017, 70: 1126-1135. [22] ANTONIOU A, EDWARDS H, STORKEY A.How to Train Your MAML[C/OL].[2024-05-23].https://arxiv.org/pdf/1810.09502. [23] VANSCHOREN J.Meta-Learning: A Survey[C/OL].[2024-05-23].https://arxiv.org/pdf/1810.03548. [24] HO J, JAIN A, ABBEEL P.Denoising Diffusion Probabilistic Mo-dels//Proc of the 34th International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2020: 6840-6851. [25] SONG J M, MENG C L, ERMON S.Denoising Diffusion Implicit Models[C/OL].[2024-05-23].https://arxiv.org/pdf/2010.02502. [26] THIES J, ZOLLHOFER M, STAMMINGER M, et al. Face2Face: Real-Time Face Capture and Reenactment of RGB Videos//Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2016: 2387-2395. [27] THIES J, ZOLLHÖFER M, NIEβNER M. Deferred Neural Rendering: Image Synthesis Using Neural Textures. ACM Transactions on Graphics, 2019, 38(4). DOI: 10.1145/3306346.3323035. [28] GUO Z Q, JIA Z H, WANG L J, et al. Constructing New Backbone Networks via Space-Frequency Interactive Convolution for Deepfake Detection. IEEE Transactions on Information Forensics and Security, 2024, 19: 401-413. [29] ZHOU P, HAN X T, MORARIU V I, et al. Two-Stream Neural Networks for Tampered Face Detection//Proc of the IEEE Confe-rence on Computer Vision and Pattern Recognition Workshops. Washington, USA: IEEE, 2017: 1831-1839. [30] AFCHAR D, NOZICK V, YAMAGISHI J, et al. MesoNet: A Compact Facial Video Forgery Detection Network//Proc of the IEEE International Workshop on Information Forensics and Securi-ty. Washington, USA: IEEE, 2018. DOI: 10.1109/WIFS.2018.8630761. [31] LI Y Z, LÜ S W.Exposing DeepFake Videos by Detecting Face Warping Artifacts[C/OL].[2024-05-23].https://arxiv.org/pdf/1811.00656v3. [32] TAN M X, LE Q.EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. Journal of Machine Learning Research, 2019, 97: 6105-6114. [33] MASI I, KILLEKAR A, MASCARENHAS R M, et al. Two-Branch Recurrent Network for Isolating Deepfakes in Videos//Proc of the 16th European Conference on Computer Vision. Berlin, Germany: Springer, 2020: 667-684.
2024, Vol. 37 
