Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
@CauchyComplete
CauchyComplete authored Dec 16, 2024
1 parent f34e2ec commit 5a27420
Showing 1 changed file with 106 additions and 97 deletions.
203 changes: 106 additions & 97 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,30 +1,62 @@
# CAT-Net
This is the official repository for Compression Artifact Tracing Network (CAT-Net). Given a possibly manipulated image, this network outputs a probability map of each pixel being manipulated.
This repo provides <B>codes, pretrained/trained weights, and our five custom datasets</B>. For more details, see the papers below.
The IJCV paper is an extension of the WACV paper and it covers almost all contents provided by the WACV paper.
# 😺 CAT-Net

Example input / output:
Welcome to the official repository for **Compression Artifact Tracing Network (CAT-Net)**. CAT-Net specializes in detecting and localizing manipulated regions in images by analyzing compression artifacts. This repository provides **code, pretrained/trained weights, and five custom datasets** for image forensics research.

<img src="https://github.com/mjkwon2021/CAT-Net/blob/main/github_images/example_input.jpg" width="400">
<img src="https://github.com/mjkwon2021/CAT-Net/blob/main/github_images/example_output_pred.png" width="400">
CAT-Net has two versions:
- **CAT-Net v1**: Targets only splicing forgery (WACV 2021).
- **CAT-Net v2**: Extends to both splicing and copy-move forgery (IJCV 2022).

For more details, refer to the papers below.

* CAT-Net v1: WACV 2021 [[link to the paper]](https://openaccess.thecvf.com/content/WACV2021/html/Kwon_CAT-Net_Compression_Artifact_Tracing_Network_for_Detection_and_Localization_of_WACV_2021_paper.html)
---

Myung-Joon Kwon, In-Jae Yu, Seung-Hun Nam, and Heung-Kyu Lee, “CAT-Net: Compression Artifact Tracing Network for Detection and Localization of Image Splicing”, Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2021, pp. 375–384
## 📄 Papers

* CAT-Net v2: International Journal of Computer Vision (IJCV), 2022 [[link to the paper]](https://link.springer.com/article/10.1007/s11263-022-01617-5) [[arXiv]](https://arxiv.org/abs/2108.12947)
### CAT-Net v1: WACV 2021
- **Title**: CAT-Net: Compression Artifact Tracing Network for Detection and Localization of Image Splicing
- **Authors**: Myung-Joon Kwon, In-Jae Yu, Seung-Hun Nam, and Heung-Kyu Lee
- **Publication**: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2021, pp. 375–384
- **Links**: [WACV Paper](https://openaccess.thecvf.com/content/WACV2021/html/Kwon_CAT-Net_Compression_Artifact_Tracing_Network_for_Detection_and_Localization_of_WACV_2021_paper.html)

Myung-Joon Kwon, Seung-Hun Nam, In-Jae Yu, Heung-Kyu Lee, and Changick Kim, “Learning JPEG Compression Artifacts for Image Manipulation Detection and Localization”, International Journal of Computer Vision, 2022, vol. 130, no. 8, pp. 1875–1895, Aug. 2022.
### CAT-Net v2: IJCV 2022
- **Title**: Learning JPEG Compression Artifacts for Image Manipulation Detection and Localization
- **Authors**: Myung-Joon Kwon, Seung-Hun Nam, In-Jae Yu, Heung-Kyu Lee, and Changick Kim
- **Publication**: International Journal of Computer Vision, vol. 130, no. 8, pp. 1875–1895, Aug. 2022
- **Links**: [IJCV Paper](https://link.springer.com/article/10.1007/s11263-022-01617-5), [arXiv](https://arxiv.org/abs/2108.12947)

---

## 🎨 Example Input / Output

<div style="display: flex; justify-content: space-between; gap: 5px;">
<figure style="text-align: center; width: 400px;">
<img src="https://github.com/mjkwon2021/CAT-Net/blob/main/github_images/example_input.jpg" width="400px">
<figcaption>Input Image</figcaption>
</figure>

## Setup
##### 1. Clone this repo.
<figure style="text-align: center; width: 400px;">
<img src="https://github.com/mjkwon2021/CAT-Net/blob/main/github_images/example_output_pred.png" width="400px">
<figcaption>Output Probability Map</figcaption>
</figure>
</div>

##### 2. Download the weights from: [[Google Drive Link]](https://drive.google.com/drive/folders/1hBEfnFtGG6q_srBHVEmbF3fTq0IhP8jq?usp=sharing) or [[Baiduyun Link]](https://pan.baidu.com/s/1hecZC0IZXdgh5WRbRoAytQ) (extract code: ycft).
````
---

## ⚙️ Setup

### 1. Clone this repository
```bash
git clone https://github.com/mjkwon2021/CAT-Net.git
cd CAT-Net
```

### 2. Download weights
Pretrained and trained weights can be downloaded from:
- [Google Drive](https://drive.google.com/drive/folders/1hBEfnFtGG6q_srBHVEmbF3fTq0IhP8jq?usp=sharing)
- [Baiduyun Link](https://pan.baidu.com/s/1hecZC0IZXdgh5WRbRoAytQ) (Extract code: `ycft`)

Place the weights as follows:
```
CAT-Net
├── pretrained_models (pretrained weights for each stream)
│ ├── DCT_djpeg.pth.tar
Expand All @@ -34,124 +66,101 @@ CAT-Net
│ ├── CAT_DCT_only
│ │ └── DCT_only_v2.pth.tar
│ └── CAT_full
│ └── CAT_full_v1.pth.tar
│ ├── CAT_full_v1.pth.tar
│ └── CAT_full_v2.pth.tar
````
If you are trying to test the network, you only need CAT_full_v1.pth.tar or CAT_full_v2.pth.tar.
```
- **CAT_full_v1**: WACV model (splicing forgery only).
- **CAT_full_v2**: IJCV model (splicing + copy-move forgery).

The weights v1 indicates the WACV model while v2 indicates the journal model. Both models have the same architecture but the trained weights are different. CAT-Net v1 targets only splicing but v2 also targets copy-move forgery. If you are planning to train from scratch, you can skip downloading.

##### 3. Setup environment.
````
### 3. Setup environment
```bash
conda create -n cat python=3.6
conda activate cat
conda install pytorch==1.1.0 torchvision==0.3.0 cudatoolkit=10.0 -c pytorch
pip install -r requirements.txt
````

##### 4. Modify configuration files.
Set paths properly in 'project_config.py'.

Set settings properly in 'experiments/CAT_full.yaml'. If you are using single GPU, set GPU=(0,) not (0).
```

### 4. Modify configuration files
- Set paths in `project_config.py`.
- Update GPU settings in `experiments/CAT_full.yaml` (e.g., `GPU=(0,)` for single GPU).

## Inference
Put input images in 'input' directory. Use English file names.
---

Choose between full CAT-Net and the DCT stream by commenting/uncommenting lines 65-66 and 75-76 in `tools/infer.py`. Also, choose between v1 and v2 in the lines 65-66 by modifying the strings.
## 🚀 Inference

At the root of this repo, run:
````
### Steps
1. **Prepare Input Images**: Place images in the `input` directory. Use English filenames.
2. **Select Model and Stream**: Modify `tools/infer.py`:
- Comment/uncomment lines 65-66 and 75-76 to select full CAT-Net or DCT stream.
- Update lines 65-66 to select v1 or v2 weights.
3. **Run Inference**: At the root of this repository, run:
```bash
python tools/infer.py
````
The predictions are saved in 'output_pred' directory as heatmaps.

## Train

##### 1. Downloading tampCOCO / compRAISE

You can download tampCOCO datasets on
[[Kaggle Link]](https://www.kaggle.com/datasets/qsii24/tampcoco)
or
[[Baiduyun Link]](https://pan.baidu.com/s/1n9nN6cB0FGxsl6VH53CRwQ?pwd=ycft).


Note that tampCOCO consists of four datasets: cm_COCO, sp_COCO, bcm_COCO (=CM RAISE), bcmc_COCO (=CM-JPEG RAISE).

As this dataset was created using MS COCO, you must follow the licensing terms of MS COCO.
```
4. **View Results**: Predictions are saved in the `output_pred` directory as heatmaps.

You can download compRAISE dataset on
[[Kaggle Link]](https://www.kaggle.com/datasets/qsii24/compraise).
---

Also, compRAISE is an alias of JPEG RAISE in the journal paper.
## 😺 Training

As this dataset was created using RAISE, you must follow the licensing terms of RAISE.
### 1. Download tampCOCO / compRAISE datasets
- **tampCOCO**: [Kaggle Link](https://www.kaggle.com/datasets/qsii24/tampcoco) or [Baiduyun Link](https://pan.baidu.com/s/1n9nN6cB0FGxsl6VH53CRwQ?pwd=ycft) (Extract code: `ycft`)
- Contains: `cm_COCO`, `sp_COCO`, `bcm_COCO`, `bcmc_COCO`.
- Follows MS COCO licensing terms.
- **compRAISE**: [Kaggle Link](https://www.kaggle.com/datasets/qsii24/compraise)
- Also referred to as JPEG RAISE in the IJCV paper.
- Follows RAISE licensing terms.

**Note**: Use datasets for research purposes only.

You are allowed to use the datasets for <B>research purpose only</B>.
### 2. Prepare datasets
- Obtain the required datasets.
- Configure training/validation paths in `Splicing/data/data_core.py`.
- JPEG-compress non-JPEG images before training. Run dataset-specific scripts (e.g., `Splicing/data/dataset_IMD2020.py`) for automatic compression.
- To add custom datasets, create dataset class files similar to the existing ones.

[2023-03-28 update] Now tampCOCO and compRAISE are uploaded to Kaggle.

##### 2. Prepare datasets.
Obtain datasets you want to use for training.

Set training and validation set configuration in Splicing/data/data_core.py [[Link]](
https://github.com/mjkwon2021/CAT-Net/blob/main/Splicing/data/data_core.py).

[2024-01-22 Update] Note on obtaining FantasticReality dataset: see [[This issue]](https://github.com/mjkwon2021/CAT-Net/issues/51).

CAT-Net only allows JPEG images for training.
So non-JPEG images in each dataset must be JPEG compressed (with Q100 and no chroma subsampling) before you start training.
You may run each dataset file (EX: Splicing/data/dataset_IMD2020.py), for automatic compression.

If you wish to add additional datasets, you should create dataset class files similar to the existing ones.



##### 3. Train.
At the root of this repo, run:
````
### 3. Start training
Run the following command at the root of the repository:
```bash
python tools/train.py
````
Training starts from the pretrained weight if you place it properly.

## Licence
This code is built on top of [HRNet](https://github.com/HRNet/HRNet-Semantic-Segmentation). You need to follow their licence.
```
Training starts from pretrained weights if they are placed properly.

For CAT-Net, you may freely use it for research purpose.
---

Commercial usage is strictly prohibited.
## 📚 Citation

If you use CAT-Net or its resources, please cite the following papers:


## Citation
If you use some resources provided by this repo, please cite these papers.
* CAT-Net v1 (WACV2021)
````
### CAT-Net v1 (WACV 2021)
```bibtex
@inproceedings{kwon2021cat,
title={CAT-Net: Compression Artifact Tracing Network for Detection and Localization of Image Splicing},
author={Kwon, Myung-Joon and Yu, In-Jae and Nam, Seung-Hun and Lee, Heung-Kyu},
booktitle={Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision},
pages={375--384},
year={2021}
}
````
* CAT-Net v2 (IJCV)
````
```

### CAT-Net v2 (IJCV 2022)
```bibtex
@article{kwon2022learning,
title={Learning JPEG Compression Artifacts for Image Manipulation Detection and Localization},
author={Kwon, Myung-Joon and Nam, Seung-Hun and Yu, In-Jae and Lee, Heung-Kyu and Kim, Changick},
journal={International Journal of Computer Vision},
volume = {130},
number = {8},
volume={130},
number={8},
pages={1875--1895},
month = aug,
month={aug},
year={2022},
publisher={Springer},
doi = {10.1007/s11263-022-01617-5}
doi={10.1007/s11263-022-01617-5}
}
````
```

---

## 🔑 Keywords
CAT-Net, Image Forensics, Multimedia Forensics, Image Manipulation Detection, Image Manipulation Localization, Image Processing

##### Keywords
CAT-Net, Image forensics, Multimedia forensics, Image manipulation detection, Image manipulation localization, Image processing

0 comments on commit 5a27420

Please sign in to comment.