├── README.assert └── relationship.png └── README.md /README.assert/relationship.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nebula-beta/awesome-adversarial-deep-learning/HEAD/README.assert/relationship.png -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | [TOC] 2 | 3 | # Awesome Adversarial Examples for Deep Learning 4 | 5 | 6 | 7 | ## Table of Contents 8 | 9 | - [Survey](#Survey) 10 | - [Attack](#Attack) 11 | - [Defense](#Defense) 12 | - [Competition](#Competition) 13 | - [ToolBox](#ToolBox) 14 | 15 | 16 | 17 | 18 | 19 | ## Survey 20 | 21 | [Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey](https://arxiv.org/abs/1801.00553) 22 | 23 | 24 | 25 | ## Attack 26 | 27 | ### Gradient-base method 28 | 29 | - **Box-constrained L-BFGS :** [Intriguing properties of neural networks](https://arxiv.org/pdf/1312.6199.pdf). Szegedy, Christian, et al. ICLR(Poster) 2014. [[blogs](https://www.cnblogs.com/lainey/p/8552422.html)] 30 | 31 | * **FGSM :** [Explaining and harnessing adversarial examples](https://arxiv.org/abs/1412.6572). Goodfellow, Ian J., Jonathon Shlens, and Christian Szegedy. ICLR(Poster) 2015. [[code](https://github.com/1Konny/FGSM), ] 32 | * **I-FGSM :** [Adversarial examples in the physical world](https://arxiv.org/abs/1607.02533). Kurakin, Alexey, Ian Goodfellow, and Samy Bengio. ICLR(Workshop) 2017. [[code](https://github.com/1Konny/FGSM), ] 33 | * **MI-FGSM :** [Boosting Adversarial Attacks with Momentum](http://openaccess.thecvf.com/content_cvpr_2018/html/Dong_Boosting_Adversarial_Attacks_CVPR_2018_paper.html). Dong Y , Liao F , Pang T , et al. CVPR 2017. [[poster](http://ml.cs.tsinghua.edu.cn/~yinpeng/poster/Attack-CVPR2018.pdf), [code]()] 34 | * **DI^2-FGSM and M-DI^2FGSM :** [Improving Transferability of Adversarial Examples with Input Diversity](https://arxiv.org/abs/1803.06978). Xie, Cihang, et al. CVPR 2019. [[code](https://github.com/cihangxie/DI-2-FGSM), ] 35 | 36 | 37 | 38 | **Relationships between above different attacks：** 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | * **JSMA :** [The limitations of deep learning in adversarial settings](https://ieeexplore.ieee.org/document/7467366). Papernot, Nicolas, et al. (EuroS&P)*. IEEE, 2016. 47 | * **One Pixel Attack :** [One pixel attack for fooling deep neural networks](https://ieeexplore.ieee.org/abstract/document/8601309/). J. Su, D. V. Vargas, S. Kouichi. arXiv preprint arXiv:1710.08864, 2017. 48 | * **DeepFool :** [DeepFool: a simple and accurate method to fool deep neural networks](https://arxiv.org/abs/1511.04599). S. Moosavi-Dezfooli et al., CVPR 2016. 49 | * **C&W :** [Towards Evaluating the Robustness of Neural Networks](https://ieeexplore.ieee.org/abstract/document/7958570). N. Carlini, D. Wagner. arXiv preprint arXiv:1608.04644, 2016. 50 | * **ATNs :**[Adversarial Transformation Networks: Learning to Generate Adversarial Examples](https://arxiv.org/abs/1703.09387). S. Baluja, I. Fischer. arXiv preprint arXiv:1703.09387, 2017. 51 | * **UPSET and ANGRI :** [UPSET and ANGRI: Breaking High Performance Image Classifiers](https://arxiv.org/abs/1707.01159). Sarkar, A. Bansal, U. Mahbub, and R. Chellappa. arXiv preprint arXiv:1707.01159, 2017. 52 | 53 | 54 | 55 | 56 | 57 | - [Intriguing properties of neural networks](https://arxiv.org/pdf/1312.6199.pdf) Szegedy, Christian, et al. arXiv preprint arXiv:1312.6199 (2013). 58 | - [Explaining and harnessing adversarial examples](https://arxiv.org/abs/1412.6572) Goodfellow, Ian J., Jonathon Shlens, and Christian Szegedy. arXiv preprint arXiv:1412.6572 (2014). 59 | - [Deep neural networks are easily fooled: High confidence predictions for unrecognizable images](https://www.cv-foundation.org/openaccess/content_cvpr_2015/html/Nguyen_Deep_Neural_Networks_2015_CVPR_paper.html) Nguyen, Anh, Jason Yosinski, and Jeff Clune. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015. 60 | - [Adversarial examples in the physical world](https://arxiv.org/abs/1607.02533) Kurakin, Alexey, Ian Goodfellow, and Samy Bengio. arXiv preprint arXiv:1607.02533 (2016). 61 | - [Adversarial diversity and hard positive generation](https://www.cv-foundation.org/openaccess/content_cvpr_2016_workshops/w12/html/Rozsa_Adversarial_Diversity_and_CVPR_2016_paper.html) Rozsa, Andras, Ethan M. Rudd, and Terrance E. Boult. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2016. 62 | - [The limitations of deep learning in adversarial settings](http://ieeexplore.ieee.org/abstract/document/7467366/) Papernot, Nicolas, et al. Security and Privacy (EuroS&P), 2016 IEEE European Symposium on. IEEE, 2016. 63 | - [Adversarial manipulation of deep representations](https://arxiv.org/abs/1511.05122) Sabour, Sara, et al. ICLR. 2016. 64 | - [Deepfool: a simple and accurate method to fool deep neural networks](https://www.cv-foundation.org/openaccess/content_cvpr_2016/html/Moosavi-Dezfooli_DeepFool_A_Simple_CVPR_2016_paper.html) Moosavi-Dezfooli, Seyed-Mohsen, Alhussein Fawzi, and Pascal Frossard. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016. 65 | - [Universal adversarial perturbations](https://arxiv.org/abs/1610.08401) Moosavi-Dezfooli, Seyed-Mohsen, et al. IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2017. 66 | - [Towards evaluating the robustness of neural networks](https://arxiv.org/abs/1608.04644) Carlini, Nicholas, and David Wagner. Security and Privacy (S&P). 2017. 67 | - [Machine Learning as an Adversarial Service: Learning Black-Box Adversarial Examples](https://arxiv.org/abs/1708.05207) Hayes, Jamie, and George Danezis. arXiv preprint arXiv:1708.05207 (2017). 68 | - [Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models](https://arxiv.org/abs/1708.03999) Chen, Pin-Yu, et al. 10th ACM Workshop on Artificial Intelligence and Security (AISEC) with the 24th ACM Conference on Computer and Communications Security (CCS). 2017. 69 | - [Ground-Truth Adversarial Examples](https://arxiv.org/abs/1709.10207) Carlini, Nicholas, et al. arXiv preprint arXiv:1709.10207. 2017. 70 | - [Generating Natural Adversarial Examples](https://arxiv.org/abs/1710.11342) Zhao, Zhengli, Dheeru Dua, and Sameer Singh. arXiv preprint arXiv:1710.11342. 2017. 71 | - [Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples](https://arxiv.org/abs/1802.00420) Anish Athalye, Nicholas Carlini, David Wagner. arXiv preprint arXiv:1802.00420. 2018. 72 | 73 | 74 | 75 | 76 | 77 | ## Defense 78 | 79 | ### Network Ditillation 80 | 81 | - [Distillation as a defense to adversarial perturbations against deep neural networks](http://ieeexplore.ieee.org/abstract/document/7546524/) Papernot, Nicolas, et al.Security and Privacy (SP), 2016 IEEE Symposium on. IEEE, 2016. 82 | 83 | ### Adversarial (Re)Training 84 | - [Learning with a strong adversary](https://arxiv.org/abs/1511.03034) Huang, Ruitong, et al. arXiv preprint arXiv:1511.03034 (2015). 85 | - [Adversarial machine learning at scale](https://arxiv.org/abs/1611.01236) Kurakin, Alexey, Ian Goodfellow, and Samy Bengio. ICLR. 2017. 86 | - [Ensemble Adversarial Training: Attacks and Defenses](https://arxiv.org/abs/1705.07204) Tramèr, Florian, et al. arXiv preprint arXiv:1705.07204 (2017). 87 | - [Adversarial training for relation extraction](http://www.aclweb.org/anthology/D17-1187) Wu, Yi, David Bamman, and Stuart Russell. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 2017. 88 | - [Adversarial Logit Pairing](https://arxiv.org/abs/1803.06373) Harini Kannan, Alexey Kurakin, Ian Goodfellow. arXiv preprint arXiv:1803.06373 (2018). 89 | 90 | ### Adversarial Detecting 91 | - [Detecting Adversarial Samples from Artifacts](https://arxiv.org/abs/1703.00410) Feinman, Reuben, et al. arXiv preprint arXiv:1703.00410 (2017). 92 | - [Adversarial and Clean Data Are Not Twins](https://arxiv.org/abs/1704.04960) Gong, Zhitao, Wenlu Wang, and Wei-Shinn Ku. arXiv preprint arXiv:1704.04960 (2017). 93 | - [Safetynet: Detecting and rejecting adversarial examples robustly](https://arxiv.org/abs/1704.00103) Lu, Jiajun, Theerasit Issaranon, and David Forsyth. ICCV (2017). 94 | - [On the (statistical) detection of adversarial examples](https://arxiv.org/abs/1702.06280) Grosse, Kathrin, et al. arXiv preprint arXiv:1702.06280 (2017). 95 | - [On detecting adversarial perturbations](https://arxiv.org/abs/1702.04267) Metzen, Jan Hendrik, et al. ICLR Poster. 2017. 96 | - [Early Methods for Detecting Adversarial Images](https://openreview.net/forum?id=B1dexpDug¬eId=B1dexpDug) Hendrycks, Dan, and Kevin Gimpel. ICLR Workshop (2017). 97 | - [Dimensionality Reduction as a Defense against Evasion Attacks on Machine Learning Classifiers](https://arxiv.org/abs/1704.02654) Bhagoji, Arjun Nitin, Daniel Cullina, and Prateek Mittal. arXiv preprint arXiv:1704.02654 (2017). 98 | - [Detecting Adversarial Attacks on Neural Network Policies with Visual Foresight](https://arxiv.org/abs/1710.00814) Lin, Yen-Chen, et al. arXiv preprint arXiv:1710.00814 (2017). 99 | - [PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examples](https://arxiv.org/abs/1710.10766) Song, Yang, et al. arXiv preprint arXiv:1710.10766 (2017). 100 | 101 | ### Input Reconstruction 102 | - [PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examples](https://arxiv.org/abs/1710.10766) Song, Yang, et al. arXiv preprint arXiv:1710.10766 (2017). 103 | - [MagNet: a Two-Pronged Defense against Adversarial Examples](https://arxiv.org/abs/1705.09064) Meng, Dongyu, and Hao Chen. CCS (2017). 104 | - [Towards deep neural network architectures robust to adversarial examples](https://arxiv.org/abs/1412.5068) Gu, Shixiang, and Luca Rigazio. arXiv preprint arXiv:1412.5068 (2014). 105 | 106 | ### Classifier Robustifying 107 | - [Adversarial Examples, Uncertainty, and Transfer Testing Robustness in Gaussian Process Hybrid Deep Networks](https://arxiv.org/abs/1707.02476) Bradshaw, John, Alexander G. de G. Matthews, and Zoubin Ghahramani.arXiv preprint arXiv:1707.02476 (2017). 108 | - [Robustness to Adversarial Examples through an Ensemble of Specialists](https://arxiv.org/abs/1702.06856) Abbasi, Mahdieh, and Christian Gagné. arXiv preprint arXiv:1702.06856 (2017). 109 | 110 | ### Network Verification 111 | - [Reluplex: An efficient SMT solver for verifying deep neural networks](https://arxiv.org/abs/1702.01135) Katz, Guy, et al. CAV 2017. 112 | - [Safety verification of deep neural networks](https://link.springer.com/chapter/10.1007/978-3-319-63387-9_1) Huang, Xiaowei, et al. International Conference on Computer Aided Verification. Springer, Cham, 2017. 113 | - [Towards proving the adversarial robustness of deep neural networks](https://arxiv.org/abs/1709.02802) Katz, Guy, et al. arXiv preprint arXiv:1709.02802 (2017). 114 | - [Deepsafe: A data-driven approach for checking adversarial robustness in neural networks](https://arxiv.org/abs/1710.00486) Gopinath, Divya, et al. arXiv preprint arXiv:1710.00486 (2017). 115 | - [DeepXplore: Automated Whitebox Testing of Deep Learning Systems](https://arxiv.org/abs/1705.06640) Pei, Kexin, et al. arXiv preprint arXiv:1705.06640 (2017). 116 | 117 | ### Others 118 | - [Adversarial Example Defenses: Ensembles of Weak Defenses are not Strong](https://arxiv.org/abs/1706.04701) He, Warren, et al. 11th USENIX Workshop on Offensive Technologies (WOOT 17). (2017). 119 | - [Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods](https://arxiv.org/abs/1705.07263) Carlini, Nicholas, and David Wagner. AISec. 2017. 120 | 121 | 122 | 123 | ## Competition 124 | 125 | * [NIPS 2017: Defense Against Adversarial Attack](https://www.kaggle.com/c/nips-2017-defense-against-adversarial-attack/data) 126 | * [NIPS 2018 : Adversarial Vision Challenge](https://www.crowdai.org/challenges) 127 | * [GeekPwn CAAD 2018](http://2018.geekpwn.org/en/index.html#4). [Winners](http://hof.geekpwn.org/caad/en/index2.html) 128 | 129 | * [IJCAI-19 Alibaba Adversarial AI Challenge](https://tianchi.aliyun.com/markets/tianchi/ijcai19_en) 130 | * [GeekPwn CAAD 2019](http://www.geekpwn.org/zh/index.html) 131 | 132 | 133 | 134 | 135 | 136 | ## ToolBox 137 | 138 | * [**advertorch**](https://github.com/BorealisAI/advertorch) 139 | * [**foolbox**](https://github.com/bethgelab/foolbox) 140 | * [**cleverhans**](https://github.com/tensorflow/cleverhans) 141 | * [**Adversarial-Face-Attack**](https://github.com/ppwwyyxx/Adversarial-Face-Attack) 142 | * [**adversarial-robustness-toolbox**](https://github.com/IBM/adversarial-robustness-toolbox) --------------------------------------------------------------------------------