Analysis and comparison of deepfakes detection methods for cross-library generalisation

The rise of generative artificial intelligence (GenAI) has made it increasingly possible to use Deepfakes technology to generate fake pictures and videos. While this technology has benefits, it also has downsides such as spreading misinformation and endangering public interests. To address this issue, researchers have proposed various deep forgery detection algorithms and have achieved remarkable results. However, a common problem regarding these detection methods is that while in-library detection can usually achieve high accuracy, their performance is significantly degraded in cross-library detection. This indicates a severe problem of insufficient generalisation ability.

To better compare the performance differences between various detection methods, this paper analyses the detection performance of the six established models of Two-stream, MesoNet, HeadPose, FWA, VA, and Multi-task. To ensure consistency, we employ a uniform evaluation framework as a benchmark for comparison. We conduct extensive intra-library and cross-library tests to evaluate these methods' generalisation ability by utilising accuracy and error rate as key evaluation criteria for our experiments. Additionally, we further explore areas for improvement by analysing the impact of data augmentation, dataset partitioning, and threshold selection on the performance of these detection methods. 

Our comparative experiments are conducted on three existing fake face video datasets, including FaceForensics++, DeepfakeTIMIT, and Celeb-DF.Our research findings indicate the database partitioning method has a direct impact on the detector's performance, and to enhance generalisation performance, the database should be divided person-based manually. The effectiveness of data augmentation techniques in improving cross-library performance is generally limited, and setting the threshold directly using source domain data often leads to a high error rate in the target domain. The findings of this paper provide insights into the development of more effective detection methods to combat the harmful effects of Deepfakes.