INFORMATION SYSTEM FOR DETECTING AI-GENERATED FACE FORGERY VIDEOS

Abstract

This study addresses the challenge of detecting manipulated videos featuring forged faces generated by deepfake technologies. We propose an information system based on a hybrid neural network architecture that integrates Convolutional Neural Networks (CNN) for extracting spatial features with Long Short-Term Memory (LSTM) networks for analyzing temporal dependencies. By incorporating the Grad-CAM technique, the system offers explainability by visualizing the key regions influencing the classification decision. Experiments conducted on the DFDC and FakeAVCeleb datasets demonstrate valuable effectiveness of the proposed approach, underlining its potential in the fight against disinformation and cybercrime.

The results on the DFDC set are slightly better than on FakeAVCeleb,  because the model was trained on data from this set and is better adapted to its features. The results on the FakeAVCeleb set are slightly lower. The Precision, Recall, and F1-score metrics on the DFDC set for both classes are quite close, which indicates a balance in recognizing both real and fake videos. Compared to the results on DFDC, on FakeAVCeleb there is a decrease in the metrics (Accuracy, Precision, Recall, F1-score, AUC) and an increase in the loss value (BCE). For DFDC, the matrix shows an almost symmetric distribution of errors, as for FakeAVCeleb. The high AUC indicator (0.94 for DFDC and 0.83 for FakeAVCeleb) demonstrates that the model separates classes well regardless of the chosen classification threshold.

Explanatory AI (XAI) makes the model’s work clearer by providing explanations for its decisions. As mentioned above, the system implements the model’s explanations using Grad Cam. Grad-CAM visualizes the regions of the image that are important for classification using a heat map (red is high activity, blue is low). In real videos, we can see more “active” areas on the face, while in the case of a fake, the model focuses on various artifacts that occur in deepfakes. Grad Cam shows high activity in these areas (red).