AIGIDetectionBenchmark

The official code of "Generalizable and Adaptive Continual Learning Framework for AI-generated Image Detection"!

This repository provides a comprehensive benchmark for AI-generated image detection, integrating advanced methods to ensure generalization and adaptability in continually evolving tasks.

🔗 Overview

The AIGIDetectionBenchmark integrates the following components:

• Existing AI-generated image detection models : A comprehensive suite of mainstream detection methods for benchmarking across various datasets and architectures.
• Continual learning methods : Ensures detection models stay adaptive to new generative methods while retaining knowledge of previous tasks.
• Training and evaluation workflows : Ready-to-use code and pre-trained model weights for seamless implementation.

🧩 Supported Models for AI-generated Image Detection

The following state-of-the-art AI-generated image detection models are integrated into the benchmark:

Method	Paper	Training Code	Testing Code	Model Weights
CNNSpot	Cnn-generated images are surprisingly easy to spot... for now	✅	✅	✅
FreDect	Leveraging frequency analysis for deep fake image recognition	✅	✅	✅
GramNet	Global texture enhancement for fake face detection in the wild	✅	✅	✅
LGrad	Learning on gradients: Generalized artifacts representation	✅	✅	✅
LNP	Detecting generated images by real images	✅	✅	✅
UnivFD	Towards universal fake image detectors that generalize across models	✅	✅	✅
FatFormer	Forgery-aware adaptive transformer for generalizable synthetic detection	✅	✅	✅
NPR	Rethinking up-sampling operations in generative networks	✅	✅	✅
SAFE	Improving synthetic image detection towards generalization	✅	✅	✅
SimE (Ours)	Generalizable and Adaptive Continual Learning Framework for AI-generated Image Detection	✅	✅	✅

🔄 Supported Continual Learning Methods

The following prominent continual learning techniques are integrated to ensure adaptability over time:

Method	Paper	Training Code	Testing Code
Seq	/	✅	✅
Joint	/	✅	✅
ER	On tiny episodic memories in continual learning	✅	✅
EWC	Overcoming catastrophic forgetting in neural networks	✅	✅
OSLA	Online structured laplace approx- imations for overcoming catastrophic forgetting	✅	✅
A-GEM	Efficient lifelong learning with a-gem	✅	✅
SI	Continual learning through synaptic intelligence	✅	✅
iCaRL	icarl: Incremental classifier and representation learning	✅	✅
Linear (Ours)	Generalizable and Adaptive Continual Learning Framework for AI-generated Image Detection	✅	✅

These methods effectively combat catastrophic forgetting and ensure detection models retain previous performance while adapting to new AI-generated image data.

🗂️ Datasets

Training Datasets

To train the detection model for AI-generated images, we follow the widely recognized protocol adopted by most methods. Specifically, we train on the ProGAN dataset, which is provided by CNNSpot.

Evaluation Datasets

The evaluation datasets are organized into two main categories:

• GAN-generated images: For GAN-based methods, we employ 8 types of GANs for testing, including:

  ProGAN, StyleGAN, StyleGAN2, BigGAN, CycleGAN, StarGAN, GauGAN, and DeepFake.

  These datasets are sourced from benchmarks: CNNSpot

• Diffusion model-generated images: For the DMs dataset, we collect 19 types of SOTA DMs, including:

  DDPM, iDDPM, ADM, DALLE, GLIDE, LDM, PNDM, Wukong, SDv1.4, Midjourney, SDv1.5, VQDM, SDV2.1, SDXL1.0, SD-Turbo, SDXL-Turbo, SDv3.0, PixArt-sigma, FLUX.1 from UniversalFakeDetect, GenImage, DRCT-2M, DiTFake.

  These datasets are sourced from benchmarks: UniversalFakeDetect, GenImage, DRCT-2M, and DiTFake.

Continual Learning Datasets

For continual learning experiments, We meticulously organize these datasets to align with the chronological emergence of that our approach accurately reflects real-world scenarios in the continual learning context.

Data Folder Formulation

We expect the directory structure to be the following:

./datasets/
├── train/                      # Training datasets
│   ├── ProGAN/                 # Sub-category for GAN-generated images
│   │   ├── car/                # Example class (car)
│   │   │   ├── 0_real/         # Real images
│   │   │   └── 1_fake/         # Fake images
│   │   └── ...                 # Additional classes
│
├── test/                       # Testing datasets
│   ├── DDPM/                   # Sub-category for Diffusion Models
│   │   ├── 0_real/             # Real images
│   │   └── 1_fake/             # Fake images
│   ├── GLIDE/                  # Another Diffusion-based generator
│   │   ├── 0_real/             # Real images
│   │   └── 1_fake/             # Fake images
│   └── ...                     # Additional testing datasets

🏛 Model Zoo

The Model Zoo provides pre-trained models weights for download:

• Baseline Detection Models: Models trained on ProGAN datasets.
• Re-implemented Models: We also re-implemented the FatFormer, NPR, and SAFE models using their official code. The re-implemented code is available in the Reimplemented_models/ directory.

All pre-trained model weights can be downloaded from Google_Drive. Please place the downloaded model weights into the weights directory of our project.

🚀 Quick Start

1️⃣ Clone the Repository

First, clone this repository to your local machine:

git clone https://github.com/Mark-Dou/AIGIDetectionBenchmark.git
cd AIGIDetectionBenchmark

2️⃣ Install Dependencies

Install the necessary libraries and frameworks:

pip install -r requirements.txt

3️⃣ Train the Detection Model

Start training the AI-generated image detection model:

cd bash_scripts
bash detection_model/train.sh

For example, to train the SimE model:

bash SimE/train.sh

4️⃣ Evaluate the Detection Model

Evaluate the trained AI-generated image detection model:

cd bash_scripts
bash detection_model/test.sh

For example, to evaluate the SimE model:

bash SimE/test.sh

5️⃣ Continual Learning for Detection Models

Start training models with continual learning methods. In the script, specify the --CL_method parameter to define the continual learning method:

For example, to train the SimE model using the Linear method, specify the continual learning method in the script:

--CL_method Linear

Then execute the following command:

bash SimE/CL_train.sh

6️⃣ Evaluate Continual Learning Results

Evaluate the performance of continual learning models:

cd bash_scripts
bash detection_model/CL_test_AA_AF.sh

For example, to evaluate the SimE model using the Linear method:

bash SimE/CL_test_AA_AF.sh

🙏 Acknowledgments

This project is built on the open-source repositories AIGCDetectBenchmark. Thank them for their well-organized codes and datasets!