|
A Tool Prototype for Deepfake Detection Based on the Enhancement and Integration of Pre-trained Neural Networks
|
Morteza Ziabakhsh1   , Seyed Amirhossein Tabatabaei *1 , Sadegh Eskandari1 |
| 1- Department of Computer Science, Faculty of Mathematical Sciences, University of Guilan, Iran |
|
|
Abstract: (68 Views) |
| Deepfakes, as synthetic data generated using deep learning technologies, enables the manipulation or replacement of individuals’ identities, actions, and attributes in existing image and video content. As an example, deepfake technology utilizing deep learning technology enables contents generation wherein a subjected individual has been removed or swapped with another targeted subject. Generating fake contents is not a recent or newborn technology, however deepfake which uses high-tech Machin Learning techniques elaborates the generated fake content to appear much more realistic than before. By producing highly convincing and realistic imitative media, this technology introduces significant technical challenges. While deepfakes have beneficial applications in domains such as art, filmmaking, and advertising, their malicious use for spreading misinformation, defaming individuals or organizations, fostering political instability, and facilitating cybercrime poses serious political, social, and economic threats. These risks are amplified by scale and velocity of disseminated and spread information on social media platforms. These concerns have rendered the development of accurate and efficient tools for deepfake detection and classification a critical and even vital necessity. To address this point, developing tools which are able to detect and classify the originality of subjected contents and verifies whether they are real have been concentrated since recent past years. Despite of this trend, development of such tools or systems is bound with many challenges. One of the main challenges is generalization problem wherein the developed tools fail while facing significantly different fake contents. This concern is of great importance wherein, fake contents are generated using diverse technologies or tools. To address this point, recent approaches have been biased towards using engineered datasets which let the models more concentrated on general and common features of deep contents rather than specialized ones dedicated to a specific fake dataset. However, the aforementioned approach shows some restrictions while dominating on all features in synthesized objects is not a trivial task for the discriminating model. As a result, the generated model might show a weak performance subjected to the fake contents with unknown features. This problem introduces a serious obstacle in design and development of generalizable approaches which outperform the classical approaches in discriminating fake contents from real ones significantly. In order to address this important problem, in this work we have tried to combine two different learning models to capture the features not only by concentrating on generated fake data but also on synthesized data with generic fake features. As the first step, we have used a pretrained model, whose initial training was based on a dataset in which fake negatives were generated by applying pre-identified fake artifacts to real images. This step helps the model learn general and common features among deep fakes, without getting trapped in the specific features of a dataset. In the second step, we retrained the pretrained model using a fine-tuning method on a new set of real negatives. In this step, by using a low learning rate as a trick, we allowed the model to learn more specific, complex, and unknown features present in the fake data, without completely overwriting or forgetting the learnings from the first step. To implement and assess the aforementioned idea of combined learning paradigms, two modern deepfake detection have been employed. The first model is based on Self-Blended Images wherein the source and target subjects (images) are combined by each other using extracted masks based on simple transformation like color adjustments or frequency transformation. The second model is based on a Localized Artifact Attention Network (LAA-Net) aiming at discriminating high-quality deepfake images focusing on local and fine features in fake images. The latter model utilizes EfficientNet-B4 as its base architecture pretrained on ImageNet dataset. Using this approach, the developed model integrates and refines multiple prominent pre-trained models enabling the identification of deepfake-induced face swap manipulation, with accuracy surpassing existing methods. The proposed system achieves an accuracy of approximately 0.99 in deepfake detection. These results represent a meaningful step toward mitigating the growing threats posed by synthetic fake media. In order to provide a platform for practical, commercial and industrial use of the AI model developed in this research, a prototype of a deep fake (currently dedicated for face swap deepfakes) detection tool has been designed and implemented. This tool has been developed in the form of a bilingual (Persian and English) website and allows users to upload and analyze images and videos. To implement the developed model, a workstation equipped with Ubuntu 24.04.1 LTS as its OS was used to run the experiments and train the models. The hardware of this system includes an Intel Xeon Gold 6148 processor with 40 cores and a frequency of 3.7 GHz, an NVIDIA GeForce RTX 4090 graphics card with 24 GB of dedicated memory, and 160 GB of main memory. This hardware configuration allowed for training deep models and processing large amounts of data with reasonable efficiency. During training, the GPU consumption was on average close to 100%, indicating full utilization of its computational capacity. Also, the execution time of each epoch lasted on average about 310 seconds. Although achieving interesting results in this work, some restrictions exist which might impact on generalizability and usability of developing such systems. One of the main restrictions is large sample size of benchmark data and their accessibility issue. The other concern is requirements to high-performance processing resources. Finally, generalizing the detection capability to other categories of deepfake rather than just face swap introduces also another restriction which impact on the usability as well. |
|
| Keywords: Deepfake, Generative adversarial network, Classification, Cybercrime, Transfer learning |
|
|
Full-Text [PDF 1164 kb]
(82 Downloads)
|
Type of Study: Research Article |
Subject:
Cryptology and Information Security
Received: 2025/12/22 | Accepted: 2026/01/21 | Published: 2026/03/19
|
|
|
|
|
|
|
| Send email to the article author |
|
|
|
| Add your comments about this article |
|
|
|
