├── img ├── placeholder ├── 12.png ├── 733.png ├── Kate.png ├── kate2.jpg ├── download1.png ├── download2.png ├── download3.png ├── download4.png ├── download5.png ├── download6.png ├── download7.png ├── download8.png ├── download9.png └── download10.png └── README.md /img/placeholder: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /img/12.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Knightzjz/NCL-IML/HEAD/img/12.png -------------------------------------------------------------------------------- /img/733.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Knightzjz/NCL-IML/HEAD/img/733.png -------------------------------------------------------------------------------- /img/Kate.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Knightzjz/NCL-IML/HEAD/img/Kate.png -------------------------------------------------------------------------------- /img/kate2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Knightzjz/NCL-IML/HEAD/img/kate2.jpg -------------------------------------------------------------------------------- /img/download1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Knightzjz/NCL-IML/HEAD/img/download1.png -------------------------------------------------------------------------------- /img/download2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Knightzjz/NCL-IML/HEAD/img/download2.png -------------------------------------------------------------------------------- /img/download3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Knightzjz/NCL-IML/HEAD/img/download3.png -------------------------------------------------------------------------------- /img/download4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Knightzjz/NCL-IML/HEAD/img/download4.png -------------------------------------------------------------------------------- /img/download5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Knightzjz/NCL-IML/HEAD/img/download5.png -------------------------------------------------------------------------------- /img/download6.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Knightzjz/NCL-IML/HEAD/img/download6.png -------------------------------------------------------------------------------- /img/download7.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Knightzjz/NCL-IML/HEAD/img/download7.png -------------------------------------------------------------------------------- /img/download8.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Knightzjz/NCL-IML/HEAD/img/download8.png -------------------------------------------------------------------------------- /img/download9.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Knightzjz/NCL-IML/HEAD/img/download9.png -------------------------------------------------------------------------------- /img/download10.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Knightzjz/NCL-IML/HEAD/img/download10.png -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | Open In Colab 2 | ![Repository Views](https://komarev.com/ghpvc/?username=Knightzjz&label=Views&color=yellow) 3 | ![GitHub repo size](https://img.shields.io/github/repo-size/Knightzjz/NCL-IML?logo=hack%20the%20box&color=pink) 4 | ![last commit](https://img.shields.io/github/last-commit/Knightzjz/NCL-IML) 5 | [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black) 6 | [![jupyter](https://img.shields.io/badge/Jupyter-Lab-F37626.svg?style=flat&logo=Jupyter)](https://jupyterlab.readthedocs.io/en/stable) 7 | [![License: GPLv3](https://img.shields.io/badge/License-GPLv3-purple.svg)](https://www.gnu.org/licenses/gpl-3.0) 8 | # **Pre-training-free Image Manipulation Localization through Non-Mutually Contrastive Learning (ICCV2023)** 9 | ### 🏀Jizhe Zhou, 👨‍🎓[Xiaochen Ma](https://me.xiaochen.world/), 💪[Xia Du](https://cs.xmut.edu.cn/info/1085/4510.htm), 🇦🇪Ahemd Y.Alhammadi, 🏎️[Wentao Feng*](https://cs.scu.edu.cn/info/1359/17839.htm) 10 | #### _Sichuan University_ & _Xiamen University of Technology_ & _Mohamed Bin Zayed University for Humanities_ 11 | **** 12 | This is the official repo of our paper [Pre-training-free Image Manipulation Localization through Non-Mutually Contrastive Learning](https://openaccess.thecvf.com/content/ICCV2023/papers/Zhou_Pre-Training-Free_Image_Manipulation_Localization_through_Non-Mutually_Exclusive_Contrastive_Learning_ICCV_2023_paper.pdf). 13 | >### 🌳[Spring Specialties] 🔥(April 3rd) We also tested the 2'14'' [video](https://news.sky.com/story/kates-cancer-diagnosis-watch-princess-of-waless-video-message-in-full-13100006) from Kate and NCL discovers strong manipulation traces around the entire body of Kate, indicating this video is faked through the "reenactment" technique. Video reenactment commonly drives a still image (Kate's photo) by another human actor (driven video). This is also the reason why the background in this 2'14'' video is frozen-still. 14 | >### (March 19th) The Mother's Day photo from Kate Middleton, i.e., Princess of Whales, has been tested by our latest model IML-ViT. Reach the results here. For high-resolution localization results, please visit the IML-ViT model below and upload the photo by yourself. 🌳 15 | ![png](img/kate2.jpg) 16 | >### 🎅[Christmas Specialties] Training Code for our [IML-ViT](https://github.com/SunnyHaze/IML-ViT) is now released. Do note that IML-ViT is the latest benchmark (till Mar. 2024) and possibly the only fully open-source benchmark on Git. 🎄🎄 17 | >📸 If you feel helpful, please cite our work and **star this repo**. Version `#DG773`, updated on 8th Apr. 18 | **** 19 | ### >**Play Tips:** 20 | 1. Due to Google Cloud Disk reasons, the "Preparation" section may not be running. But it WON'T break this notebook; just ignore it and execute the rest sections in sequence. 21 | 2. The loaded CaCL-Net is the NCL model proposed in our paper. The nickname "CaCL-Net" comes from a local Macau restaurant called "CaCL", where we came up with the NCL idea. 22 | 3. The 4th "Result Display" section shows some representative results of NCL on those frequently-compared images. Scroll down the right slide in this section to view all pictures. Random selection from the pic pool will be involving soon. Stay tuned. 23 | 4. We built a small playground in the 5th "Test Samples From Web" section. Substituting the default image URLs stored in "urls=[...]" with your own ones and then re-execute this section, you will get the results of TCL on your customized input! Hope you will enjoy it, and please contact us if any exception occurs. 24 | 25 | **** 26 | ## **Visit our code directly through colab by clicking** Open In Colab 27 | 28 | ### >**A Bit More About Us:** 29 | >We are the Special Interest Group on IML, led by Associate Researcher 🏀 _Jizhe Zhou_ and Professor 👨‍🏫 _Jiancheng Lv_, under Sichuan University🇨🇳. Please refer to [here](https://dicalab.cn/) for more information. 30 | >Also, here are some of our other works. 🀄 Feel free to cite and star them. 31 | ------------------------------------------------------------------------------------------------- 32 | ### Latest Update (Dec. 25th) 33 | * ➡️ [🎄Christmas Special Update🎅] Training Code for IML-ViT is Now Released[IML-ViT](https://github.com/SunnyHaze/IML-ViT). Do note that IML-ViT is the latest benchmark (till Dec. 2024) and possibly the only fully opensourced bechmark you can find. Feel free to test it and conduct us for any exceptions or questions!!!!! 34 | * ➡️ Our corrected NIST16 dataset, duplicated images and the label leakage problem are fully addressed. Training and testing splits are also provided. [NIST16-Deduplicated](https://github.com/DSLJDI/NIST16-data-set-deduplication/tree/main)(Nov. 22th) 35 | --------------------------------------------------------------------------------------------------- 36 | * 1🥇 Our latest benchmark and the first pure ViT-based IML build. [IML-ViT](https://github.com/SunnyHaze/IML-ViT) 37 | * 2🥇 Our implementation of the MVSS-Net (ICCV 2021 by Dong et al.) in Pytorch, with training code embedded. [MVSS-NetPytorch-withTraining](https://github.com/dddb11/MVSS-Net) 38 | * 3🥈 Our corrected CASIAv2 dataset, with ground-truth mask correctly aligned. [Casia2.0-Corrected](https://github.com/SunnyHaze/CASIA2.0-Corrected-Groundtruth) 39 | * 4🥉 Our implementation of the Mile-stone Mantran-Net (CVPR 2019 by Wu, et al.) in Pytorch, with training code embedded. [Mantra-NetPytorch-withTraining](https://github.com/SunnyHaze/ManTraNet-Pytorch) 40 | * 5🥈 Our Manipulation Mask Generator (PRML 2023 by Yang, et al.), which crawls real-life, open-posted tampered images and auto-generates high-quality IML datasets in Python, with code embedded. [Manipulation-Mask-Generator](https://github.com/ndyysheep/MTVNR) 41 | * 6🏅 Our attempts of MAE on image manipulation localization in pytorch. [Manipulation-MAE](https://github.com/chelton-niu/Manipulation-MAE) 42 | >This repo will be under consistent construction. You will be teleported to our latest work right from here. Stay tuned. 43 | **** 44 | **** 45 | 46 | ## **Preparation** 47 | 48 | 49 | ```python 50 | import shutil 51 | shutil.rmtree("SampleData") 52 | ``` 53 | 54 | ## **1. Environment Configuration** 55 | 56 | 57 | ```python 58 | !gdown https://drive.google.com/uc?id=13G-Ay5Sx7o2jpG_AdjVA2drpFXZ0R2kJ 59 | !unzip CaCLNet.zip 60 | Sample_path = 'SampleData' 61 | 62 | !gdown https://drive.google.com/uc?id=1urqD-AqGiHSB8k3ruz2HJuu-mBxWqqf0 63 | CaCLNet_path = '20230319-010.pth' 64 | 65 | !rm CaCLNet.zip 66 | ``` 67 | 68 | ## **2. Load Sample Dataset** 69 | 70 | 71 | ```python 72 | import os 73 | SampleList=[] 74 | for file in os.listdir(Sample_path): 75 | name = os.path.join(Sample_path, file) 76 | SampleList.append(name) 77 | print(SampleList) 78 | ``` 79 | 80 | 81 | 82 | ## **3. Load a Pre-trained CaCL-Net Model** 83 | 84 | 85 | ```python 86 | import torch 87 | CaCLNet = torch.load('/content/20230319-010.pth') 88 | 89 | ``` 90 | 91 | ## **4. Result Display** 92 | 93 | 94 | ```python 95 | import cv2 96 | import custom_transforms as tr 97 | from PIL import Image 98 | from torchvision import transforms 99 | from torchvision.utils import make_grid, save_image 100 | from utils import decode_seg_map_sequence 101 | import torch 102 | from matplotlib import pyplot 103 | 104 | 105 | composed_transforms = transforms.Compose([ 106 | tr.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)), 107 | tr.ToTensor()]) 108 | 109 | for name in SampleList: 110 | im = cv2.imread(name) 111 | b, g, r = cv2.split(im) 112 | rgb = cv2.merge([r, g, b]) 113 | image = Image.fromarray(rgb) 114 | image = image.resize((512, 512), Image.BILINEAR) 115 | 116 | target = 0.0 # Consistent with training process 117 | sample = {'image': image, 'label': target} 118 | 119 | tensor_in = composed_transforms(sample)['image'].unsqueeze(0) 120 | tensor_in = tensor_in.cuda() 121 | CaCLNet.eval() 122 | _, _, _, output = CaCLNet(tensor_in) 123 | grid_image = make_grid(decode_seg_map_sequence(torch.max(output[:3], 1)[1].detach().cpu().numpy()), 124 | 3, normalize=False, range=(0, 255)) 125 | save_image(grid_image, "mask.png") 126 | img = cv2.imread("mask.png") 127 | pyplot.figure( figsize=(15,5) ) 128 | pyplot.subplot(131) 129 | pyplot.imshow( rgb ) 130 | pyplot.subplot(132) 131 | pyplot.imshow( img ) 132 | 133 | ``` 134 | 135 | 136 | 137 | ![png](img/download1.png) 138 | 139 | 140 | 141 | 142 | 143 | ![png](img/download2.png) 144 | 145 | 146 | 147 | 148 | 149 | ![png](img/download3.png) 150 | 151 | 152 | 153 | 154 | 155 | ![png](img/download4.png) 156 | 157 | 158 | 159 | 160 | 161 | ![png](img/download5.png) 162 | 163 | 164 | 165 | 166 | 167 | ![png](img/download6.png) 168 | 169 | 170 | 171 | 172 | 173 | ![png](img/download7.png) 174 | 175 | 176 | 177 | # **5. Test Samples From Web** 178 | 179 | 180 | ```python 181 | import requests 182 | from io import BytesIO 183 | from PIL import Image 184 | import numpy as np 185 | 186 | def test_image_from_web(): 187 | 188 | for url in urls: 189 | response = requests.get(url) 190 | rgb = np.asarray(Image.open(BytesIO(response.content))) # pil->numpy->bgr 191 | image = Image.fromarray(rgb) 192 | image = image.resize((512, 512), Image.BILINEAR) 193 | 194 | target = 0.0 # Consistent with training process 195 | sample = {'image': image, 'label': target} 196 | tensor_in = composed_transforms(sample)['image'].unsqueeze(0) 197 | tensor_in = tensor_in.cuda() 198 | CaCLNet.eval() 199 | _, _, _, output = CaCLNet(tensor_in) 200 | grid_image = make_grid(decode_seg_map_sequence(torch.max(output[:3], 1)[1].detach().cpu().numpy()), 201 | 3, normalize=False, range=(0, 255)) 202 | save_image(grid_image, "mask.png") 203 | img = cv2.imread("mask.png") 204 | pyplot.figure( figsize=(15,5) ) 205 | pyplot.subplot(131) 206 | pyplot.imshow( rgb ) 207 | pyplot.subplot(132) 208 | pyplot.imshow( img ) 209 | 210 | ``` 211 | 212 | 213 | 214 | * Images from Internet 215 | * you can replace the url with our own data for testing ! 216 | 217 | 218 | ```python 219 | 220 | urls = [ 221 | 'http://nadignewspapers.com/wp-content/uploads/2019/11/Kit-11.jpg', 222 | 'https://www.digitalforensics.com/blog/wp-content/uploads/2016/09/digital_image_forgery_detection.jpg', 223 | 'https://assets.hongkiat.com/uploads/amazing-photoshop-skills/topps20.jpg' 224 | ] 225 | 226 | test_image_from_web() 227 | ``` 228 | 229 | 230 | 231 | ![png](img/download8.png) 232 | 233 | 234 | 235 | 236 | 237 | ![png](img/download9.png) 238 | 239 | 240 | 241 | 242 | 243 | ![png](img/download10.png) 244 | 245 | **** 246 | 247 | ## Citation 248 | 249 | ``` 250 | @InProceedings{Zhou_2023_ICCV, 251 | author = {Zhou, Jizhe and Ma, Xiaochen and Du, Xia and Alhammadi, Ahmed Y. and Feng, Wentao}, 252 | title = {Pre-Training-Free Image Manipulation Localization through Non-Mutually Exclusive Contrastive Learning}, 253 | booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)}, 254 | month = {October}, 255 | year = {2023}, 256 | pages = {22346-22356} 257 | } 258 | ``` 259 | 260 | **** 261 | 262 |
Flag Counter
263 | 264 | --------------------------------------------------------------------------------