├── .gitignore ├── README.md ├── data └── transform.py ├── images ├── artifact-fake-v4.png ├── artifact-real-v4.png ├── header.png └── visual_summary2.jpg └── requirements.txt /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # C extensions 7 | *.so 8 | 9 | # Distribution / packaging 10 | .Python 11 | build/ 12 | develop-eggs/ 13 | dist/ 14 | downloads/ 15 | eggs/ 16 | .eggs/ 17 | lib/ 18 | lib64/ 19 | parts/ 20 | sdist/ 21 | var/ 22 | wheels/ 23 | pip-wheel-metadata/ 24 | share/python-wheels/ 25 | *.egg-info/ 26 | .installed.cfg 27 | *.egg 28 | MANIFEST 29 | 30 | # PyInstaller 31 | # Usually these files are written by a python script from a template 32 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 33 | *.manifest 34 | *.spec 35 | 36 | # Installer logs 37 | pip-log.txt 38 | pip-delete-this-directory.txt 39 | 40 | # Unit test / coverage reports 41 | htmlcov/ 42 | .tox/ 43 | .nox/ 44 | .coverage 45 | .coverage.* 46 | .cache 47 | nosetests.xml 48 | coverage.xml 49 | *.cover 50 | .hypothesis/ 51 | .pytest_cache/ 52 | 53 | # Translations 54 | *.mo 55 | *.pot 56 | 57 | # Django stuff: 58 | *.log 59 | local_settings.py 60 | db.sqlite3 61 | db.sqlite3-journal 62 | 63 | # Flask stuff: 64 | instance/ 65 | .webassets-cache 66 | 67 | # Scrapy stuff: 68 | .scrapy 69 | 70 | # Sphinx documentation 71 | docs/_build/ 72 | 73 | # PyBuilder 74 | target/ 75 | 76 | # Jupyter Notebook 77 | .ipynb_checkpoints 78 | 79 | # IPython 80 | profile_default/ 81 | ipython_config.py 82 | 83 | # pyenv 84 | .python-version 85 | 86 | # pipenv 87 | # According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control. 88 | # However, in case of collaboration, if having platform-specific dependencies or dependencies 89 | # having no cross-platform support, pipenv may install dependencies that don't work, or not 90 | # install all needed dependencies. 91 | #Pipfile.lock 92 | 93 | # celery beat schedule file 94 | celerybeat-schedule 95 | 96 | # SageMath parsed files 97 | *.sage.py 98 | 99 | # Environments 100 | .env 101 | .venv 102 | env/ 103 | venv/ 104 | ENV/ 105 | env.bak/ 106 | venv.bak/ 107 | 108 | # Spyder project settings 109 | .spyderproject 110 | .spyproject 111 | 112 | # Rope project settings 113 | .ropeproject 114 | 115 | # mkdocs documentation 116 | /site 117 | 118 | # mypy 119 | .mypy_cache/ 120 | .dmypy.json 121 | dmypy.json 122 | 123 | # Pyre type checker 124 | .pyre/ 125 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # ArtiFact: A Large-Scale Dataset with Artificial and Factual Images for Generalizable and Robust Synthetic Image Detection [ICIP 2023] 2 | 3 | 4 | 5 | **Paper:** 6 | * IEEE Xplore: https://ieeexplore.ieee.org/document/10222083 7 | * ArXiv: https://arxiv.org/abs/2302.11970 8 | 9 | **Abstract:** Synthetic image generation has opened up new opportunities but has also created threats in regard to privacy, authenticity, and security. Detecting fake images is of paramount importance to prevent illegal activities, and previous research has shown that generative models leave unique patterns in their synthetic images that can be exploited to detect them. However, the fundamental problem of generalization remains, as even state-of-the-art detectors encounter difficulty when facing generators never seen during training. To assess the generalizability and robustness of synthetic image detectors in the face of real-world impairments, this paper presents a large-scale dataset named ArtiFact, comprising diverse generators, object categories, and real-world challenges. Moreover, the proposed multi-class classification scheme, combined with a filter stride reduction strategy addresses social platform impairments and effectively detects synthetic images from both seen and unseen generators. The proposed solution significantly outperforms other top teams by 8.34% on Test 1, 1.26% on Test 2, and 15.08% on Test 3 in the IEEE VIP Cup challenge at ICIP 2022, as measured by the accuracy metric. 10 | 11 | **Presentation**: 12 | [![YouTube](https://badges.aleen42.com/src/youtube.svg)](https://youtu.be/hGXufeubNME) 13 | 14 | **Visual Summary**: 15 | 16 |
17 | 18 | # Update 19 | * [22 June 2023] - The work has been accepted to IEEE ICIP 2023 conference. 20 | 21 | # Result on [IEEE VIP Cup at ICIP 2022](https://grip-unina.github.io/vipcup2022/) 22 | 23 | Accuracy (%) of Top3 Teams on Leaderboard, 24 | 25 | | Team Names | Test 1 | Test 2 | Test 3 | 26 | | :-------------------- | :--------: | :--------: | :--------: | 27 | | Sherlock | 87\.70 | 77\.52 | 73\.45 | 28 | | FAU Erlangen-Nürnberg | 87\.14 | 81\.74 | 75\.52 | 29 | | **Megatron (Ours)** | **96\.04** | **83\.00** | **90\.60** | 30 | 31 | > **Note:** A small portion of the proposed ArtiFact dataset, totaling 222K images of 71K real images and 151K fake images from only 13 generators is used in the IEEE VIP Cup. Here all the Test data is kept confidential from all participating teams. Additionally, the generators used for the Test 1 data are known to all teams, whereas the generators for Test 2 and Test 3 are kept undisclosed. 32 | 33 | # Dataset Description 34 | * Total number of images: $2,496,738$ 35 | * Number of real images: $964,989$ 36 | * Number of fake images: $1,531,749$ 37 | * Number of generators used for fake images: $25$ (including $13$ GANs, $7$ Diffusion, and $5$ miscellaneous generators) 38 | * Number of sources used for real images: $8$ 39 | * Categories included in the dataset: `Human/Human Faces`, `Animal/Animal Faces`, `Places`, `Vehicles`, `Art`, and other real-life objects 40 | * Image Resolution: $200 \times 200$ 41 | 42 | ## Data Distribution 43 | 44 | * Real 45 | 46 |
47 | 48 | * Fake 49 | 50 |
51 | 52 | # Download Dataset 53 | 54 | The dataset is hosted on Kaggle. The dataset can be downloaded i) directly from the browser using the link below or ii) can be downloaded using [kaggle-api](https://github.com/Kaggle/kaggle-api). 55 | 56 | ## i) Directly from Browser 57 | Link: [ArtiFact Dataset](https://www.kaggle.com/datasets/awsaf49/artifact-dataset) 58 | 59 | 60 | ## ii) Kaggle API 61 | ```shell 62 | !kaggle datasets download -d awsaf49/artifact-dataset 63 | ``` 64 | 65 | # How to Use 66 | The dataset is organized into folders, each of which corresponds to a specific generator of synthetic images or source of real images. Each folder contains a `metadata.csv` file, which provides information about the images in the folder. It contains following columns, 67 | 68 | * `image_path` : The relative path of the image file. 69 | * `target` : The label for the image, which is either 0 for real or 1 for fake. 70 | * `category` : The category (cat or dog etc) of the image 71 | 72 | 73 | # Data Generation 74 | 75 | * Images are randomly sampled from different methods then transformed using impairments. The methods are listed below, 76 |
77 | Methods 78 | 79 | 80 | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 81 | | :-------------- | :--------------------------------------------- | :----------------------------------------- | :---------------------------------- | :-------------------------------------------- | :--------------------------------------------- | :------------------------------------------------- | :---------------------------------------------------------------- | :------------------------------------------------ | :---------------------------------------------- | :------------------------------------------ | :------------------------------------------ | :----------------------------------------------------------- | :---------------------------------------------------- | :----------------------------------------- | :---------------------------------------- | :----------------------------------------------- | :------------------------------------------------ | :-------------------------------------------------------------------------- | :----------------------------------------- | :---------------------------------------------------------------------------------------- | :---------------------------------------- | :---------------------------------------- | :---------------------------------------------------------------------------- | :-------------------------------------------- | :----------------------------------------------------------- | :-------------------------------------------------- | :-------------------------------------------------- | :-------------------------------------------------------- | :------------------------------------------------ | :----------------------------------------------------- | :-------------------------------------- | :---------------------------------------------------------------- | :-------------------------------------------------------------- | 82 | | **Method** | ImageNet | COCO | LSUN | AFHQ | FFHQ | Metfaces | CelebAHQ | Landscape | Glide | StyleGAN2 | StyleGAN3 | Generative Inpainting | Taming Transformer | MAT | LaMa | Stable Diffusion | VQ Diffusion | Palette | StyleGAN1 | Latent Diffusion | CIPS | StarGAN | BigGAN | GANformer | ProjectedGAN | SFHQ | FaceSynthetics | Denoising Diffusion GAN | DDPM | DiffusionGAN | GauGAN | ProGAN | CycleGAN | 83 | | **Reference** | [link](https://www.image-net.org/download.php) | [link](https://cocodataset.org/\#download) | [link](https://github.com/fyu/lsun) | [link](https://github.com/clovaai/stargan-v2) | [link](https://github.com/NVlabs/ffhq-dataset) | [link](https://github.com/NVlabs/metfaces-dataset) | [link](https://github.com/tkarras/progressive*growing*of*gans) | [link](https://github.com/mahmoudnafifi/HistoGAN) | [link](https://github.com/openai/glide-text2im) | [link](https://github.com/NVlabs/stylegan2) | [link](https://github.com/NVlabs/stylegan3) | [link](https://github.com/JiahuiYu/generative*inpainting) | [link](https://github.com/CompVis/taming-transformer) | [link](https://github.com/fenglinglwb/mat) | [link](https://github.com/saic-mdal/lama) | [link](https://github.com/huggingface/diffusers) | [link](https://github.com/microsoft/VQ-Diffusion) | [link](https://github.com/Janspiry/Palette-Image-to-Image-Diffusion-Models) | [link](https://github.com/NVlabs/stylegan) | [link](https://github.com/compvis/latent-diffusion\#retrieval-augmented-diffusion-models) | [link](https://github.com/saic-mdal/CIPS) | [link](https://github.com/yunjey/StarGAN) | [link](https://github.com/open-mmlab/mmgeneration/tree/master/configs/biggan) | [link](https://github.com/dorarad/gansformer) | [link](https://github.com/autonomousvision/projected*gan) | [link](https://github.com/SelfishGene/SFHQ-dataset) | [link](https://github.com/microsoft/FaceSynthetics) | [link](https://github.com/NVlabs/denoising-diffusion-gan) | [link](https://github.com/hojonathanho/diffusion) | [link](https://github.com/Zhendong-Wang/Diffusion-GAN) | [link](https://github.com/NVlabs/SPADE) | [link](https://github.com/tkarras/progressive*growing*of*gans) | [link](https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix) | 84 | 85 |
86 | 87 | * All images went through RandomCrop and Random Impairments (Jpeg Compression & Downscale). To apply these transformation use [data/transform.py](https://github.com/awsaf49/artifact/blob/main/data/transform.py) which applies random transformation. All images are cropped and resized to $200 \times 200$ pixels and then compressed using JPEG at a random quality level. 88 | 89 | ```shell 90 | !python data/transform.py 91 | ``` 92 | 93 | # Citation 94 | ```bibtex 95 | @INPROCEEDINGS{artifact, 96 | author={Rahman, Md Awsafur and Paul, Bishmoy and Sarker, Najibul Haque and Hakim, Zaber Ibn Abdul and Fattah, Shaikh Anowarul}, 97 | booktitle={2023 IEEE International Conference on Image Processing (ICIP)}, 98 | title={Artifact: A Large-Scale Dataset With Artificial And Factual Images For Generalizable And Robust Synthetic Image Detection}, 99 | year={2023}, 100 | volume={}, 101 | number={}, 102 | pages={2200-2204}, 103 | doi={10.1109/ICIP49359.2023.10222083}} 104 | ``` 105 | 106 | # License 107 | 108 | ArtiFact dataset takes leverage of data from multiple methods thus different parts of the dataset come with different licenses. All the methods and their associated licenses are mentioned in the table, 109 | 110 |
111 | Data License 112 | 113 | 114 | | Method | License | 115 | |:------------------------|:---------------------------------------------------------------------------------------| 116 | | ImageNet | Non Commercial | 117 | | COCO | Creative Commons Attribution 4.0 License | 118 | | LSUN | Unknown | 119 | | AFHQ | Creative Commons Attribution-NonCommercial 4.0 International Public | 120 | | FFHQ | Creative Commons BY-NC-SA 4.0 license | 121 | | Metfaces | Creative Commons BY-NC 2.0 | 122 | | CelebAHQ | Creative Commons Attribution-NonCommercial 4.0 International Public | 123 | | Landscape | MIT license | 124 | | Glide | MIT license | 125 | | StyleGAN2 | Nvidia Source Code License | 126 | | StyleGAN3 | Nvidia Source Code License | 127 | | Generative Inpainting | Creative Commons Public Licenses | 128 | | Taming Transformer | MIT License | 129 | | MAT | Creative Commons Public Licenses | 130 | | LaMa | Apache-2.0 License | 131 | | Stable Diffusion | Apache-2.0 License | 132 | | VQ Diffusion | MIT License | 133 | | Palette | MIT License | 134 | | StyleGAN1 | Creative Commons Public Licenses | 135 | | Latent Diffusion | MIT License | 136 | | CIPS | MIT License | 137 | | StarGAN | MIT License | 138 | | BigGAN | MIT License | 139 | | GANformer | MIT License | 140 | | ProjectedGAN | MIT License | 141 | | SFHQ | MIT License | 142 | | FaceSynthetics | Research Use of Data Agreement v1.0 | 143 | | Denoising Diffusion GAN | NVIDIA License | 144 | | DDPM | Unknown | 145 | | DiffusionGAN | MIT License | 146 | | GauGAN | Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Public License | 147 | | ProGAN | Attribution-NonCommercial 4.0 International | 148 | | CycleGAN | BSD | 149 | 150 | 151 |
152 | 153 | # Acknowledgment 154 | 155 | * The authors would like to express their gratitude to the IEEE Signal Processing Society, GRIP of the University Federico II of Naples (Italy), and NVIDIA (USA) for hosting the [IEEE Video and Image Processing (VIP) Cup](https://grip-unina.github.io/vipcup2022/) competition at ICIP 2022. This competition provided a platform for the authors to showcase their work and motivated them to push their boundaries to deliver a state-of-the-art solution. 156 | 157 | * The authors also would like to express their gratitude to the authors of the methods that is used for creating ArtiFact dataset. All the methods and their reference is added below, 158 |
159 | Data Reference 160 | 161 | | Method | Reference | 162 | |:------------------------|:-----------------------------------------------------------------------------------------| 163 | | ImageNet | [link](https://www.image-net.org/download.php) | 164 | | COCO | [link](https://cocodataset.org/#download) | 165 | | LSUN | [link](https://github.com/fyu/lsun) | 166 | | AFHQ | [link](https://github.com/clovaai/stargan-v2) | 167 | | FFHQ | [link](https://github.com/NVlabs/ffhq-dataset) | 168 | | Metfaces | [link](https://github.com/NVlabs/metfaces-dataset) | 169 | | CelebAHQ | [link](https://github.com/tkarras/progressive_growing_of_gans) | 170 | | Landscape | [link](https://github.com/mahmoudnafifi/HistoGAN) | 171 | | Glide | [link](https://github.com/openai/glide-text2im) | 172 | | StyleGAN2 | [link](https://github.com/NVlabs/stylegan2) | 173 | | StyleGAN3 | [link](https://github.com/NVlabs/stylegan3) | 174 | | Generative Inpainting | [link](https://github.com/JiahuiYu/generative_inpainting) | 175 | | Taming Transformer | [link](https://github.com/CompVis/taming-transformer) | 176 | | MAT | [link](https://github.com/fenglinglwb/mat) | 177 | | LaMa | [link](https://github.com/saic-mdal/lama) | 178 | | Stable Diffusion | [link](https://github.com/huggingface/diffusers) | 179 | | VQ Diffusion | [link](https://github.com/microsoft/VQ-Diffusion) | 180 | | Palette | [link](https://github.com/Janspiry/Palette-Image-to-Image-Diffusion-Models) | 181 | | StyleGAN1 | [link](https://github.com/NVlabs/stylegan) | 182 | | Latent Diffusion | [link](https://github.com/compvis/latent-diffusion#retrieval-augmented-diffusion-models) | 183 | | CIPS | [link](https://github.com/saic-mdal/CIPS) | 184 | | StarGAN | [link](https://github.com/yunjey/StarGAN) | 185 | | BigGAN | [link](https://github.com/open-mmlab/mmgeneration/tree/master/configs/biggan) | 186 | | GANformer | [link](https://github.com/dorarad/gansformer) | 187 | | ProjectedGAN | [link](https://github.com/autonomousvision/projected_gan) | 188 | | SFHQ | [link](https://github.com/SelfishGene/SFHQ-dataset) | 189 | | FaceSynthetics | [link](https://github.com/microsoft/FaceSynthetics) | 190 | | Denoising Diffusion GAN | [link](https://github.com/NVlabs/denoising-diffusion-gan) | 191 | | DDPM | [link](https://github.com/hojonathanho/diffusion) | 192 | | DiffusionGAN | [link](https://github.com/Zhendong-Wang/Diffusion-GAN) | 193 | | GauGAN | [link](https://github.com/NVlabs/SPADE) | 194 | | ProGAN | [link](https://github.com/tkarras/progressive_growing_of_gans) | 195 | | CycleGAN | [link](https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix) | 196 | 197 |
198 | -------------------------------------------------------------------------------- /data/transform.py: -------------------------------------------------------------------------------- 1 | # Adapted from: IEEE VIP Cup 2022 - https://www.grip.unina.it/download/vipcup2022/random_operations.py 2 | 3 | import os 4 | from PIL import Image 5 | from tqdm import tqdm 6 | import shutil 7 | import glob 8 | from random import Random 9 | from joblib import Parallel, delayed 10 | import pandas as pd 11 | import warnings 12 | 13 | # Configuirations 14 | OUTPUT_SIZE = 200 15 | CROPSIZE_MIN = 160 16 | CROPSIZE_MAX = 2048 17 | CROPSIZE_RATIO = (5,8) 18 | QF_RANGE = (65, 100) 19 | 20 | def check_img(filename): 21 | return filename.lower().endswith(('.png', '.jpg', '.jpeg', '.tif', '.tiff', '.bmp', '.gif')) 22 | 23 | def transform(input_dir, output_dir, seed, maximg=None): 24 | """ 25 | Transforms images in the input directory by performing random operations such as cropping, resizing, and jpeg compression. 26 | 27 | Args: 28 | input_dir (str): The directory path containing the source images to be transformed. It can include wildcards (*) to match multiple directories. 29 | output_dir (str): The directory path to save the transformed images. 30 | seed (int): The seed to use for random operations. 31 | maximg (int, optional): The maximum number of images to be transformed. If None, all images in the input directory will be transformed. Default is None. 32 | 33 | Returns: 34 | None 35 | """ 36 | print('Random Operations from ', input_dir, 'to', output_dir, flush=True) 37 | if os.path.isdir(output_dir): 38 | shutil.rmtree(output_dir) # remove existing output directory 39 | os.makedirs(output_dir) # create output directory 40 | 41 | random = Random(seed) # set seed 42 | 43 | # list fo images 44 | input_dir = input_dir if '*' in input_dir else os.path.join(input_dir, '**/*') 45 | list_src = [_ for _ in sorted(glob.glob(input_dir, recursive=True)) if check_img(_)] 46 | 47 | if maximg is not None: 48 | random.shuffle(list_src) # shuffle the list of images 49 | list_src = list_src[:maximg] # limit the number of images 50 | 51 | def transform(index, src): 52 | filename_dst = 'img%06d.jpg' % index 53 | if (len(src.split("/"))<=4 and "/kaggle" not in src) or (len(src.split("/"))<=5 and "/kaggle" in src): 54 | sub_dir = '' 55 | else : 56 | sub_dir = src.split("/",4)[-1] if '/kaggle' in src else src.split("/",3)[-1] 57 | sub_dir = sub_dir.rsplit('/',1)[0] # remove name 58 | os.makedirs(os.path.join(output_dir, sub_dir), exist_ok=True) 59 | dst = os.path.join(output_dir, sub_dir, filename_dst) 60 | 61 | # open image 62 | with Image.open(src) as img: 63 | img = img.convert('RGB') 64 | height = img.size[1] 65 | width = img.size[0] 66 | 67 | # select the size of crop 68 | cropmax = min(min(width, height), CROPSIZE_MAX) 69 | if cropmaxOUTPUT_SIZE else Image.CUBIC 78 | 79 | # select the position of the crop 80 | x1 = random.randint(0, width - cropsize) 81 | y1 = random.randint(0, height - cropsize) 82 | 83 | # select the jpeg quality factor 84 | qf = random.randint(*QF_RANGE) 85 | 86 | # make cropping 87 | img = img.crop((x1, y1, x1+cropsize, y1+cropsize)) 88 | assert img.size[0]==cropsize 89 | assert img.size[1]==cropsize 90 | 91 | # make resizing 92 | img = img.resize((OUTPUT_SIZE, OUTPUT_SIZE), interp) 93 | assert img.size[0]==OUTPUT_SIZE 94 | assert img.size[1]==OUTPUT_SIZE 95 | 96 | # make jpeg compression 97 | img.save(dst, "JPEG", quality = qf) 98 | 99 | return [filename_dst,dst,src,cropsize,x1,y1,qf] 100 | 101 | metainfo = Parallel(n_jobs=-1, backend='threading')( 102 | delayed(transform)(index, src) for index, src in enumerate(tqdm(list_src)) 103 | ) 104 | metainfo = pd.DataFrame(metainfo, columns=['filename','dst','src','cropsize','x1','y1','qf']) 105 | output_csv = os.path.join(output_dir, "metadata.csv") 106 | metainfo.to_csv(output_csv, index=False) 107 | 108 | if __name__=='__main__': 109 | from sys import argv 110 | input_dir = argv[1] 111 | output_dir = argv[2] 112 | seed = int(argv[3]) 113 | maximg = int(argv[4]) if len(argv)>4 else None 114 | transform(input_dir, output_dir, seed, maximg) -------------------------------------------------------------------------------- /images/artifact-fake-v4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/awsaf49/artifact/994ea5cf73285ccd8adeb29654490d58addac358/images/artifact-fake-v4.png -------------------------------------------------------------------------------- /images/artifact-real-v4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/awsaf49/artifact/994ea5cf73285ccd8adeb29654490d58addac358/images/artifact-real-v4.png -------------------------------------------------------------------------------- /images/header.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/awsaf49/artifact/994ea5cf73285ccd8adeb29654490d58addac358/images/header.png -------------------------------------------------------------------------------- /images/visual_summary2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/awsaf49/artifact/994ea5cf73285ccd8adeb29654490d58addac358/images/visual_summary2.jpg -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | Pillow>=9.0.1 2 | tqdm>=4.63.0 3 | joblib 4 | pandas 5 | --------------------------------------------------------------------------------