Detecting deepfakes in multimedia content

Authors: Roberto Caldelli (MICC – University of Florence); Fabrizio Falchi (National Council Research – CNR); Adrian Popescu (French Alternative Energies and Atomic Energy Commission- CEA

When created by malevolent entities, deepfakes pollute the online space and have deleterious effects in users’ real lives, especially when aimed to interfere with debates related to polarizing situations. Deep learning has enabled the generation of credible deepfakes for different types of multimedia content, such as texts, videos and images. AI4Media proposes tools for efficient deepfake detection, regardless of the nature of forged documents.

Promising results were already obtained for texts and videos. Fake texts are difficult to distinguish from human-generated texts for short sequences. An efficient method was designed for the detection of fake tweets generated by specific accounts by learning adapted deep language models based per account. Deepfake videos are hard to detect when models are not specifically trained for a specific type of forgery. An algorithm that leverages the optical flow in videos was introduced and it successfully generalizes the detection capabilities to unlearnt forgeries.

Deep language models can be used to generate short texts, such as Tweets, which are difficult to distinguish from real tweets. Fake tweets are written by bots that mimic specific users by exploiting language models fine-tuned on the user’s past contributions.  The more refined the language models are, the more credible the generated fake tweets will be. The proposed fake tweet detection method is designed to match these generation practices and thus successfully distinguish fake from real tweets. A wide array of detection models were tested and the best results were obtained using a RoBERTa, a recent algorithm whose objective is to produce deep language models. It provides a detection accuracy of over 90%. The method can be used for an effective flagging of fake texts on Twitter. Importantly, it is easy to deploy for a large number of users who are of interest in AI4Media (or beyond) since language models are created per account. The work also led to the creation of TweepFake, a public dataset dedicated to the detection of deepfake tweets. The availability of this dataset will facilitate future research in the area and ensure the proposal of comparable and replicable results (Read the paper).

AI-technologies can be used in various ways to generate realistic fake videos. While the detection of known forgeries is well handled, the same is not true for forgeries that are not known to the detection algorithms and thus cannot be learned. The proposed model exploits the optical flow fields of the videos in order to improve the robustness of the detection of unlearnt forgeries. It also has competitive performance for the detection of learned forgeries. The main novelty is the integration of the bi-dimensional optical flow fields with the pre-trained network which usually receives the inputs of the three channels. This allows the detection of temporal inconsistencies which complement the information obtained from the usual frame-based analysis of content. This work paves the way toward the proposal of deepfake detection methods which are exploitable in practice since it generalizes to forgeries that are unknown and thus learned by detectors (Read the paper).

Work is currently ongoing to propose methods that combine different cues available in documents in order to seamlessly detect forgeries in multimedia documents. Early results obtained for deepfake videos which combine the visual and audio channels are particularly promising.

References:

Caldelli, R., Galteri, L., Amerini, I., & Del Bimbo, A. (2021). Optical Flow based CNN for detection of unlearnt deepfake manipulations. Pattern Recognition Letters, 146, 31-37.

Fagni, T., Falchi, F., Gambini, M., Martella, A., & Tesconi, M. (2021). TweepFake: About detecting deepfake tweets. Plos one, 16(5), e0251415.