Deep-Fake-Detector-v2-Model-ONNX

Deep-Fake-Detector-v2-Model (ONNX)#

This is an ONNX version of prithivMLmods/Deep-Fake-Detector-v2-Model. It was automatically converted and uploaded using this Hugging Face Space.

Usage with Transformers.js#

See the pipeline documentation for image-classification: https://huggingface.co/docs/transformers.js/api/pipelines#module_pipelines.ImageClassificationPipeline

Image: fake q.gif

Deep-Fake-Detector-v2-Model#

Overview#

The Deep-Fake-Detector-v2-Model is a state-of-the-art deep learning model designed to detect deepfake images. It leverages the Vision Transformer (ViT) architecture, specifically the google/vit-base-patch16-224-in21k model, fine-tuned on a dataset of real and deepfake images. The model is trained to classify images as either "Realism" or "Deepfake" with high accuracy, making it a powerful tool for detecting manipulated media.

Classification report:

              precision    recall  f1-score   support

     Realism     0.9683    0.8708    0.9170     28001
    Deepfake     0.8826    0.9715    0.9249     28000

    accuracy                         0.9212     56001
   macro avg     0.9255    0.9212    0.9210     56001
weighted avg     0.9255    0.9212    0.9210     56001

Confusion Matrix:

  [[True Positives, False Negatives],
   [False Positives, True Negatives]]

Image: download.png

Update : The previous model checkpoint was obtained using a smaller classification dataset. Although it performed well in evaluation scores, its real-time performance was average due to limited variations in the training set. The new update includes a larger dataset to improve the detection of fake images.

Key Features#

• Architecture: Vision Transformer (ViT) - google/vit-base-patch16-224-in21k.
• Input: RGB images resized to 224x224 pixels.
• Output: Binary classification ("Realism" or "Deepfake").
• Training Dataset: A curated dataset of real and deepfake images.
• Fine-Tuning: The model is fine-tuned using Hugging Face's Trainer API with advanced data augmentation techniques.
• Performance: Achieves high accuracy and F1 score on validation and test datasets.

Model Architecture#

The model is based on the Vision Transformer (ViT), which treats images as sequences of patches and applies a transformer encoder to learn spatial relationships. Key components include:

• Patch Embedding: Divides the input image into fixed-size patches (16x16 pixels).
• Transformer Encoder: Processes patch embeddings using multi-head self-attention mechanisms.
• Classification Head: A fully connected layer for binary classification.

Training Details#

• Optimizer: AdamW with a learning rate of 1e-6.
• Batch Size: 32 for training, 8 for evaluation.
• Epochs: 2.
• Data Augmentation:
  • Random rotation (±90 degrees).
  • Random sharpness adjustment.
  • Random resizing and cropping.
• Loss Function: Cross-Entropy Loss.
• Evaluation Metrics: Accuracy, F1 Score, and Confusion Matrix.

Inference with Hugging Face Pipeline#

from transformers import pipeline

# Load the model
pipe = pipeline('image-classification', model="prithivMLmods/Deep-Fake-Detector-v2-Model", device=0)

# Predict on an image
result = pipe("path_to_image.jpg")
print(result)

Inference with PyTorch#

from transformers import ViTForImageClassification, ViTImageProcessor
from PIL import Image
import torch

# Load the model and processor
model = ViTForImageClassification.from_pretrained("prithivMLmods/Deep-Fake-Detector-v2-Model")
processor = ViTImageProcessor.from_pretrained("prithivMLmods/Deep-Fake-Detector-v2-Model")

# Load and preprocess the image
image = Image.open("path_to_image.jpg").convert("RGB")
inputs = processor(images=image, return_tensors="pt")

# Perform inference
with torch.no_grad():
    outputs = model(**inputs)
    logits = outputs.logits
    predicted_class = torch.argmax(logits, dim=1).item()

# Map class index to label
label = model.config.id2label[predicted_class]
print(f"Predicted Label: {label}")

Dataset#

The model is fine-tuned on the dataset, which contains:

• Real Images: Authentic images of human faces.
• Fake Images: Deepfake images generated using advanced AI techniques.

Limitations#

The model is trained on a specific dataset and may not generalize well to other deepfake datasets or domains.

• Performance may degrade on low-resolution or heavily compressed images.
• The model is designed for image classification and does not detect deepfake videos directly.

Ethical Considerations#

Misuse: This model should not be used for malicious purposes, such as creating or spreading deepfakes. Bias: The model may inherit biases from the training dataset. Care should be taken to ensure fairness and inclusivity. Transparency: Users should be informed when deepfake detection tools are used to analyze their content.

Future Work#

• Extend the model to detect deepfake videos.
• Improve generalization by training on larger and more diverse datasets.
• Incorporate explainability techniques to provide insights into model predictions.

Citation#

@misc{Deep-Fake-Detector-v2-Model,
  author = {prithivMLmods},
  title = {Deep-Fake-Detector-v2-Model},
  initial = {21 Mar 2024},
  second_updated = {31 Jan 2025},
  latest_updated = {02 Feb 2025}
}