├── README.md ├── database ├── readme.md └── test.md ├── evaluation ├── backdoor │ └── readme.md ├── membership │ └── readme.md ├── modelsteal │ └── readme.md └── readme.md └── figures └── SecurityNet.png /README.md: -------------------------------------------------------------------------------- 1 | # SecurityNet 2 | This is the official repository for our USENIX Security Symposium 2024 paper ["SecurityNet: Assessing Machine Learning Vulnerabilities on Public Models"](https://www.usenix.org/conference/usenixsecurity24/presentation/zhang-boyang). 3 | 4 | ## Introduction 5 | 6 | While advanced machine learning (ML) models are deployed in numerous real-world applications, previous works demonstrate these models have security and privacy vulnerabilities. 7 | Various empirical research has been done in this field. 8 | However, most of the experiments are performed on target ML models trained by the security researchers themselves. 9 | Due to the high computational resource requirement for training advanced models with complex architectures, researchers generally choose to train a few target models using relatively simple architectures on typical experiment datasets. 10 | We argue that to understand ML models' vulnerabilities comprehensively, experiments should be performed on a large set of models trained with various purposes (not just the purpose of evaluating ML attacks and defenses). 11 | To this end, we propose using publicly available models with weights from the Internet (public models) for evaluating attacks and defenses on ML models. 12 | We establish a database, namely SecurityNet, containing 910 annotated image classification models. 13 | We then analyze the effectiveness of several representative attacks/defenses, including model stealing attacks, membership inference attacks, and backdoor detection on these public models. 14 | Our evaluation empirically shows the performance of these attacks/defenses can vary significantly on public models compared to self-trained models. 15 | We advocate researchers to perform experiments on public models to better demonstrate their proposed methods' effectiveness in the future. 16 | 17 | ![SecurityNet Statistics](./figures/SecurityNet.png) 18 | 19 | ## Code and database coming soon 20 | -------------------------------------------------------------------------------- /database/readme.md: -------------------------------------------------------------------------------- 1 | # Database 2 | SecurityNet database. 3 | The models are annotated with relevant information. See model link for download details. 4 | -------------------------------------------------------------------------------- /database/test.md: -------------------------------------------------------------------------------- 1 | | | model_name | data_name | data_category | num_class | training_size | conference | year | country | num_authors | test accuracy | model stealing accuracy | model stealing agreement | num_conv | num_fc | max_kernel | max_channel | pooling | dropout | train accuracy | correct AUC | m_entropy AUC | MLP AUC | overfit gap | AUC diff_c | AUC diff_m | MLP top-3 AUC | MLP Large AUC | num_params | wrn accuracy | wrn agree | imgnet accuracy | imgnet agree | model architecture | model type | purpose | conference type | 2 | |----:|:---------------------------------|:------------|:----------------|------------:|----------------:|:-------------|-------:|:-----------|--------------:|----------------:|--------------------------:|---------------------------:|-----------:|---------:|-------------:|--------------:|:------------------|:----------|-----------------:|--------------:|----------------:|----------:|--------------:|-------------:|-------------:|----------------:|----------------:|-----------------:|---------------:|------------:|------------------:|---------------:|:---------------------|:-----------------|:----------|:------------------| 3 | | 200 | seresnet542bn_cifar10 | CIFAR-10 | natural | 10 | 50000 | CVPR | 2018 | China | 5 | 0.9646 | 0.7655 | 0.7678 | 544 | 0 | 3 | 256 | AdaptiveAvgPool2d | False | 1 | 0.516 | 0.54764 | 0.4999 | 0.0354 | -0.0161 | -0.0477402 | 0 | 0 | 6.27275e+06 | 0.5968 | 0.5997 | 0.63 | 0.6348 | seresnet542bn | seresnet | benchmark | CV | 4 | | 201 | sepreresnet20_cifar10 | CIFAR-10 | natural | 10 | 50000 | CVPR | 2018 | China, UK | 5 | 0.9363 | 0.7942 | 0.798 | 19 | 0 | 3 | 64 | AdaptiveAvgPool2d | False | 0.99336 | 0.5249 | 0.556093 | 0.532671 | 0.05706 | 0.00777144 | -0.0234213 | 0 | 0 | 274559 | 0.5951 | 0.5992 | 0.686 | 0.697 | sepreresnet20 | sepreresnet | benchmark | CV | 5 | | 202 | sepreresnet56_cifar10 | CIFAR-10 | natural | 10 | 50000 | CVPR | 2018 | China, UK | 5 | 0.9532 | 0.7516 | 0.7532 | 55 | 0 | 3 | 64 | AdaptiveAvgPool2d | False | 0.9994 | 0.5228 | 0.561786 | 0.551033 | 0.0462 | 0.0282325 | -0.0107531 | 0 | 0 | 862601 | 0.54 | 0.5425 | 0.6187 | 0.6246 | sepreresnet56 | sepreresnet | benchmark | CV | 6 | | 203 | sepreresnet110_cifar10 | CIFAR-10 | natural | 10 | 50000 | CVPR | 2018 | China, UK | 5 | 0.9535 | 0.7422 | 0.7467 | 109 | 0 | 3 | 64 | AdaptiveAvgPool2d | False | 0.99952 | 0.5215 | 0.561053 | 0.5 | 0.04602 | -0.0215 | -0.0610533 | 0 | 0 | 1.74466e+06 | 0.5168 | 0.5199 | 0.5636 | 0.569 | sepreresnet110 | sepreresnet | benchmark | CV | 7 | | 204 | sepreresnet164bn_cifar10 | CIFAR-10 | natural | 10 | 50000 | CVPR | 2018 | China, UK | 5 | 0.9621 | 0.7701 | 0.7723 | 163 | 0 | 3 | 256 | AdaptiveAvgPool2d | False | 0.99984 | 0.5183 | 0.547973 | 0.5 | 0.03774 | -0.0183 | -0.047973 | 0 | 0 | 1.90488e+06 | 0.581 | 0.5823 | 0.6238 | 0.6298 | sepreresnet164bn | sepreresnet | benchmark | CV | 8 | | 205 | sepreresnet272bn_cifar10 | CIFAR-10 | natural | 10 | 50000 | CVPR | 2018 | China, UK | 5 | 0.9651 | 0.7579 | 0.7608 | 271 | 0 | 3 | 256 | AdaptiveAvgPool2d | False | 0.99988 | 0.5169 | 0.542131 | 0.5 | 0.03478 | -0.0169 | -0.0421308 | 0 | 0 | 3.15245e+06 | 0.5967 | 0.5973 | 0.6196 | 0.6227 | sepreresnet272bn | sepreresnet | benchmark | CV | 9 | | 206 | sepreresnet542bn_cifar10 | CIFAR-10 | natural | 10 | 50000 | CVPR | 2018 | China, UK | 5 | 0.9687 | 0.7558 | 0.7571 | 541 | 0 | 3 | 256 | AdaptiveAvgPool2d | False | 1 | 0.514 | 0.531066 | 0.5 | 0.0313 | -0.014 | -0.0310662 | 0 | 0 | 6.27137e+06 | 0.5915 | 0.5906 | 0.6109 | 0.6118 | sepreresnet542bn | sepreresnet | benchmark | CV | 10 | | 207 | pyramidnet110_a48_cifar10 | CIFAR-10 | natural | 10 | 50000 | CVPR | 2017 | Korea | 3 | 0.9618 | 0.7585 | 0.7594 | 109 | 0 | 3 | 64 | None | False | 1 | 0.5175 | 0.546012 | 0.5 | 0.0382 | -0.0175 | -0.0460116 | 0 | 0 | 1.77271e+06 | 0.5824 | 0.5824 | 0.5775 | 0.5822 | pyramidnet110 | pyramidnet | benchmark | CV | 11 | | 208 | pyramidnet110_a84_cifar10 | CIFAR-10 | natural | 10 | 50000 | CVPR | 2017 | Korea | 3 | 0.9697 | 0.7548 | 0.7549 | 109 | 0 | 3 | 100 | None | False | 1 | 0.5144 | 0.545051 | 0.5 | 0.0303 | -0.0144 | -0.0450508 | 0 | 0 | 3.90445e+06 | 0.5884 | 0.5884 | 0.6022 | 0.6059 | pyramidnet110 | pyramidnet | benchmark | CV | 12 | | 209 | pyramidnet110_a270_cifar10 | CIFAR-10 | natural | 10 | 50000 | CVPR | 2017 | Korea | 3 | 0.9741 | 0.7611 | 0.7618 | 109 | 0 | 3 | 286 | None | False | 1 | 0.5115 | 0.52887 | 0.5 | 0.0259 | -0.0115 | -0.0288698 | 0 | 0 | 2.84855e+07 | 0.5878 | 0.5883 | 0.5642 | 0.5683 | pyramidnet110 | pyramidnet | benchmark | CV | 13 | | 210 | pyramidnet164_a270_bn_cifar10 | CIFAR-10 | natural | 10 | 50000 | CVPR | 2017 | Korea | 3 | 0.9756 | 0.7585 | 0.7617 | 163 | 0 | 3 | 1144 | None | False | 1 | 0.511 | 0.543582 | 0.5 | 0.0244 | -0.011 | -0.0435819 | 0 | 0 | 2.7216e+07 | 0.6007 | 0.6025 | 0.5788 | 0.5797 | pyramidnet164 | pyramidnet | benchmark | CV | 14 | | 211 | pyramidnet200_a240_bn_cifar10 | CIFAR-10 | natural | 10 | 50000 | CVPR | 2017 | Korea | 3 | 0.9755 | 0.7716 | 0.7727 | 199 | 0 | 3 | 1024 | None | False | 1 | 0.5116 | 0.546567 | 0.4999 | 0.0245 | -0.0117 | -0.0466672 | 0 | 0 | 2.67527e+07 | 0.6192 | 0.6189 | 0.5927 | 0.5942 | pyramidnet200 | pyramidnet | benchmark | CV | 15 | | 212 | pyramidnet236_a220_bn_cifar10 | CIFAR-10 | natural | 10 | 50000 | CVPR | 2017 | Korea | 3 | 0.9761 | 0.7647 | 0.7692 | 235 | 0 | 3 | 944 | None | False | 0.99988 | 0.5116 | 0.541843 | 0.5 | 0.02378 | -0.0116 | -0.0418428 | 0 | 0 | 2.6969e+07 | 0.6042 | 0.6053 | 0.5865 | 0.5871 | pyramidnet236 | pyramidnet | benchmark | CV | 16 | | 213 | pyramidnet272_a200_bn_cifar10 | CIFAR-10 | natural | 10 | 50000 | CVPR | 2017 | Korea | 3 | 0.9763 | 0.7621 | 0.763 | 271 | 0 | 3 | 864 | None | False | 0.99996 | 0.5117 | 0.54587 | 0.5 | 0.02366 | -0.0117 | -0.0458704 | 0 | 0 | 2.62108e+07 | 0.6191 | 0.6204 | 0.5687 | 0.5718 | pyramidnet272 | pyramidnet | benchmark | CV | 17 | | 214 | densenet40_k12_cifar10 | CIFAR-10 | natural | 10 | 50000 | CVPR | 2017 | USA, China | 4 | 0.9449 | 0.7881 | 0.7907 | 76 | 0 | 3 | 114 | AvgPool2d | False | 0.99832 | 0.5276 | 0.565164 | 0.537232 | 0.05342 | 0.0096318 | -0.0279318 | 0 | 0 | 599050 | 0.6073 | 0.6091 | 0.6488 | 0.656 | densenet40 | densenet | benchmark | CV | 18 | | 215 | densenet40_k12_bc_cifar10 | CIFAR-10 | natural | 10 | 50000 | CVPR | 2017 | USA, China | 4 | 0.9341 | 0.7843 | 0.7886 | 40 | 0 | 3 | 60 | AvgPool2d | False | 0.98732 | 0.5254 | 0.550027 | 0.5 | 0.05322 | -0.0254 | -0.0500271 | 0 | 0 | 176122 | 0.5901 | 0.592 | 0.668 | 0.6753 | densenet40 | densenet | benchmark | CV | 19 | | 216 | densenet40_k24_bc_cifar10 | CIFAR-10 | natural | 10 | 50000 | CVPR | 2017 | USA, China | 4 | 0.9548 | 0.7758 | 0.7783 | 40 | 0 | 3 | 120 | AvgPool2d | False | 0.9996 | 0.5216 | 0.567022 | 0.554488 | 0.0448 | 0.0328883 | -0.0125335 | 0 | 0 | 690346 | 0.593 | 0.5951 | 0.6273 | 0.6307 | densenet40 | densenet | benchmark | CV | 20 | | 217 | densenet40_k36_bc_cifar10 | CIFAR-10 | natural | 10 | 50000 | CVPR | 2017 | USA, China | 4 | 0.9588 | 0.75 | 0.755 | 40 | 0 | 3 | 180 | AvgPool2d | False | 0.99992 | 0.5197 | 0.554043 | 0.5 | 0.04112 | -0.0197 | -0.0540427 | 0 | 0 | 1.54268e+06 | 0.5311 | 0.5324 | 0.5934 | 0.5969 | densenet40 | densenet | benchmark | CV | 21 | | 218 | densenet100_k12_cifar10 | CIFAR-10 | natural | 10 | 50000 | CVPR | 2017 | USA, China | 4 | 0.9617 | 0.7482 | 0.751 | 196 | 0 | 3 | 294 | AvgPool2d | False | 1 | 0.5182 | 0.562561 | 0.5 | 0.0383 | -0.0182 | -0.0625609 | 0 | 0 | 4.06849e+06 | 0.5201 | 0.5211 | 0.5726 | 0.5778 | densenet100 | densenet | benchmark | CV | 22 | | 219 | densenet100_k24_cifar10 | CIFAR-10 | natural | 10 | 50000 | CVPR | 2017 | USA, China | 4 | 0.969 | 0.7514 | 0.756 | 196 | 0 | 3 | 588 | AvgPool2d | False | 1 | 0.5142 | 0.533206 | 0.5 | 0.031 | -0.0142 | -0.0332062 | 0 | 0 | 1.61141e+07 | 0.5749 | 0.5765 | 0.5668 | 0.5693 | densenet100 | densenet | benchmark | CV | 23 | | 220 | densenet100_k12_bc_cifar10 | CIFAR-10 | natural | 10 | 50000 | CVPR | 2017 | USA, China | 4 | 0.9566 | 0.7479 | 0.7502 | 100 | 0 | 3 | 150 | AvgPool2d | False | 0.99948 | 0.5206 | 0.552568 | 0.5 | 0.04288 | -0.0206 | -0.0525675 | 0 | 0 | 769162 | 0.5215 | 0.5242 | 0.5851 | 0.5901 | densenet100 | densenet | benchmark | CV | 24 | | 221 | densenet190_k40_bc_cifar10 | CIFAR-10 | natural | 10 | 50000 | CVPR | 2017 | USA, China | 4 | 0.9746 | 0.7581 | 0.7605 | 190 | 0 | 3 | 950 | AvgPool2d | False | 1 | 0.5128 | 0.564433 | 0.5002 | 0.0254 | -0.0126 | -0.0642328 | 0 | 0 | 2.56244e+07 | 0.5891 | 0.5906 | 0.586 | 0.5879 | densenet190 | densenet | benchmark | CV | 25 | | 222 | densenet250_k24_bc_cifar10 | CIFAR-10 | natural | 10 | 50000 | CVPR | 2017 | USA, China | 4 | 0.9726 | 0.762 | 0.7636 | 250 | 0 | 3 | 750 | AvgPool2d | False | 1 | 0.5125 | 0.539006 | 0.4999 | 0.0274 | -0.0126 | -0.0391063 | 0 | 0 | 1.53244e+07 | 0.6093 | 0.6135 | 0.5803 | 0.5818 | densenet250 | densenet | benchmark | CV | 26 | | 223 | xdensenet40_2_k24_bc_cifar10 | CIFAR-10 | natural | 10 | 50000 | ECCV | 2018 | India | 3 | 0.9449 | 0.7867 | 0.7844 | 40 | 0 | 3 | 120 | AvgPool2d | False | 0.99736 | 0.5268 | 0.564037 | 0.4998 | 0.05246 | -0.027 | -0.0642374 | 0 | 0 | 1.31934e+06 | 0.5779 | 0.5776 | 0.642 | 0.6473 | xdensenet40 | xdensenet | benchmark | CV | 27 | | 224 | xdensenet40_2_k36_bc_cifar10 | CIFAR-10 | natural | 10 | 50000 | ECCV | 2018 | India | 3 | 0.9559 | 0.7496 | 0.7508 | 40 | 0 | 3 | 180 | AvgPool2d | False | 0.99932 | 0.5209 | 0.559954 | 0.551516 | 0.04342 | 0.030616 | -0.00843748 | 0 | 0 | 2.95791e+06 | 0.5296 | 0.5316 | 0.5987 | 0.6024 | xdensenet40 | xdensenet | benchmark | CV | 28 | | 225 | wrn16_10_cifar10 | CIFAR-10 | natural | 10 | 50000 | arXiv | 2017 | France | 2 | 0.9695 | 0.7655 | 0.7677 | 12 | 0 | 3 | 640 | None | False | 1 | 0.5143 | 0.60435 | 0.5 | 0.0305 | -0.0143 | -0.10435 | 0 | 0 | 1.71166e+07 | 0.6011 | 0.6027 | 0.6138 | 0.6194 | wrn16 | wrn | benchmark | arXiv | 29 | | 226 | wrn28_10_cifar10 | CIFAR-10 | natural | 10 | 50000 | arXiv | 2017 | France | 2 | 0.9753 | 0.7572 | 0.7582 | 24 | 0 | 3 | 640 | None | False | 1 | 0.5119 | 0.5418 | 0.5 | 0.0247 | -0.0119 | -0.0417998 | 0 | 0 | 3.64792e+07 | 0.6026 | 0.602 | 0.5889 | 0.5899 | wrn28 | wrn | benchmark | arXiv | 30 | | 227 | wrn40_8_cifar10 | CIFAR-10 | natural | 10 | 50000 | arXiv | 2017 | France | 2 | 0.9756 | 0.7463 | 0.7459 | 36 | 0 | 3 | 512 | None | False | 1 | 0.5108 | 0.55411 | 0.4999 | 0.0244 | -0.0109 | -0.0542104 | 0 | 0 | 3.57483e+07 | 0.5299 | 0.5313 | 0.5647 | 0.5656 | wrn40 | wrn | benchmark | arXiv | 31 | | 228 | wrn20_10_1bit_cifar10 | CIFAR-10 | natural | 10 | 50000 | arXiv | 2017 | France | 2 | 0.9664 | 0.7622 | 0.7639 | 19 | 0 | 3 | 640 | None | False | 0.99996 | 0.5146 | 0.593732 | 0.5 | 0.03356 | -0.0146 | -0.0937317 | 0 | 0 | 2.67371e+07 | 0.5966 | 0.5993 | 0.6499 | 0.6544 | wrn20 | wrn | benchmark | arXiv | 32 | | 229 | wrn20_10_32bit_cifar10 | CIFAR-10 | natural | 10 | 50000 | arXiv | 2017 | France | 2 | 0.9684 | 0.7466 | 0.7482 | 19 | 0 | 3 | 640 | None | False | 1 | 0.5134 | 0.556283 | 0.4999 | 0.0316 | -0.0135 | -0.0563832 | 0 | 0 | 2.67371e+07 | 0.5264 | 0.5288 | 0.5839 | 0.5876 | wrn20 | wrn | benchmark | arXiv | 33 | | 230 | ror3_56_cifar10 | CIFAR-10 | natural | 10 | 50000 | IEEE | 2017 | China, USA | 6 | 0.9448 | 0.7635 | 0.7683 | 55 | 0 | 3 | 64 | MaxPool2d | False | 0.99784 | 0.5271 | 0.549626 | 0.5 | 0.05304 | -0.0271 | -0.0496263 | 0 | 0 | 762746 | 0.5463 | 0.5472 | 0.6437 | 0.6517 | ror3 | ror | benchmark | other | 34 | | 231 | ror3_110_cifar10 | CIFAR-10 | natural | 10 | 50000 | IEEE | 2017 | China, USA | 6 | 0.9551 | 0.7594 | 0.7598 | 109 | 0 | 3 | 64 | MaxPool2d | False | 0.99964 | 0.5216 | 0.553916 | 0.5 | 0.04454 | -0.0216 | -0.0539157 | 0 | 0 | 1.63769e+06 | 0.5324 | 0.533 | 0.6299 | 0.6361 | ror3 | ror | benchmark | other | 35 | | 232 | ror3_164_cifar10 | CIFAR-10 | natural | 10 | 50000 | IEEE | 2017 | China, USA | 6 | 0.96 | 0.761 | 0.7636 | 163 | 0 | 3 | 64 | MaxPool2d | False | 0.9998 | 0.5192 | 0.558881 | 0.531546 | 0.0398 | 0.0123457 | -0.0273351 | 0 | 0 | 2.51263e+06 | 0.5977 | 0.599 | 0.629 | 0.6329 | ror3 | ror | benchmark | other | 36 | | 233 | rir_cifar10 | CIFAR-10 | natural | 10 | 50000 | arXiv | 2016 | USA | 3 | 0.966 | 0.7697 | 0.7733 | 3 | 0 | 3 | 48 | None | False | 1 | 0.5163 | 0.553535 | 0.5 | 0.034 | -0.0163 | -0.0535353 | 0 | 0 | 9.49298e+06 | 0.6045 | 0.6039 | 0.6223 | 0.625 | rir | rir | benchmark | arXiv | 37 | | 234 | shakeshakeresnet20_2x16d_cifar10 | CIFAR-10 | natural | 10 | 50000 | arXiv | 2017 | UK | 1 | 0.9462 | 0.7885 | 0.7978 | 37 | 0 | 3 | 64 | AvgPool2d | False | 0.98564 | 0.5174 | 0.549713 | 0.4999 | 0.03944 | -0.0175 | -0.0498131 | 0 | 0 | 541082 | 0.5789 | 0.5842 | 0.7079 | 0.7186 | shakeshakeresnet20 | shakeshakeresnet | benchmark | arXiv | 38 | | 235 | shakeshakeresnet26_2x32d_cifar10 | CIFAR-10 | natural | 10 | 50000 | arXiv | 2017 | UK | 1 | 0.9684 | 0.7584 | 0.7609 | 49 | 0 | 3 | 128 | AvgPool2d | False | 0.99976 | 0.5154 | 0.559366 | 0.535081 | 0.03136 | 0.0196806 | -0.0242856 | 0 | 0 | 2.92316e+06 | 0.584 | 0.5863 | 0.6451 | 0.6489 | shakeshakeresnet26 | shakeshakeresnet | benchmark | arXiv | 39 | | 236 | diaresnet20_cifar10 | CIFAR-10 | natural | 10 | 50000 | arXiv | 2019 | Singapore | 4 | 0.9356 | 0.7521 | 0.7591 | 21 | 0 | 3 | 64 | AdaptiveAvgPool2d | True | 0.99252 | 0.5281 | 0.561917 | 0.5 | 0.05692 | -0.0281 | -0.0619166 | 0 | 0 | 286866 | 0.5233 | 0.5265 | 0.6391 | 0.6467 | diaresnet20 | diaresnet | benchmark | arXiv | 40 | | 237 | diaresnet56_cifar10 | CIFAR-10 | natural | 10 | 50000 | arXiv | 2019 | Singapore | 4 | 0.9494 | 0.7667 | 0.7682 | 57 | 0 | 3 | 64 | AdaptiveAvgPool2d | True | 0.9994 | 0.5237 | 0.5755 | 0.5 | 0.0500001 | -0.0237 | -0.0755002 | 0 | 0 | 870162 | 0.5729 | 0.5729 | 0.65 | 0.6561 | diaresnet56 | diaresnet | benchmark | arXiv | 41 | | 238 | diaresnet110_cifar10 | CIFAR-10 | natural | 10 | 50000 | arXiv | 2019 | Singapore | 4 | 0.9583 | 0.7632 | 0.7654 | 111 | 0 | 3 | 64 | AdaptiveAvgPool2d | True | 0.9998 | 0.5194 | 0.557977 | 0.5 | 0.0415 | -0.0194 | -0.0579774 | 0 | 0 | 1.74511e+06 | 0.5404 | 0.5421 | 0.6099 | 0.6156 | diaresnet110 | diaresnet | benchmark | arXiv | 42 | | 239 | diaresnet164bn_cifar10 | CIFAR-10 | natural | 10 | 50000 | arXiv | 2019 | Singapore | 4 | 0.9651 | 0.7661 | 0.7654 | 166 | 0 | 3 | 256 | AdaptiveAvgPool2d | True | 0.99992 | 0.5163 | 0.559349 | 0.5001 | 0.03482 | -0.0162 | -0.0592487 | 0 | 0 | 1.923e+06 | 0.591 | 0.5922 | 0.6156 | 0.6198 | diaresnet164bn | diaresnet | benchmark | arXiv | 43 | | 240 | diapreresnet20_cifar10 | CIFAR-10 | natural | 10 | 50000 | arXiv | 2019 | Singapore | 4 | 0.9344 | 0.7908 | 0.7944 | 18 | 0 | 3 | 64 | AdaptiveAvgPool2d | True | 0.99348 | 0.5285 | 0.554267 | 0.538311 | 0.05908 | 0.00981062 | -0.015956 | 0 | 0 | 286674 | 0.5857 | 0.5901 | 0.6658 | 0.6757 | diapreresnet20 | diapreresnet | benchmark | arXiv | 44 | | 241 | diapreresnet56_cifar10 | CIFAR-10 | natural | 10 | 50000 | arXiv | 2019 | Singapore | 4 | 0.9527 | 0.7712 | 0.7736 | 54 | 0 | 3 | 64 | AdaptiveAvgPool2d | True | 0.99952 | 0.5216 | 0.565179 | 0.5 | 0.04682 | -0.0216 | -0.0651791 | 0 | 0 | 869970 | 0.5878 | 0.5873 | 0.6604 | 0.6654 | diapreresnet56 | diapreresnet | benchmark | arXiv | 45 | | 242 | diapreresnet110_cifar10 | CIFAR-10 | natural | 10 | 50000 | arXiv | 2019 | Singapore | 4 | 0.9557 | 0.7657 | 0.7713 | 108 | 0 | 3 | 64 | AdaptiveAvgPool2d | True | 0.99992 | 0.5224 | 0.547978 | 0.5 | 0.04422 | -0.0224 | -0.0479781 | 0 | 0 | 1.74491e+06 | 0.5902 | 0.5911 | 0.641 | 0.6491 | diapreresnet110 | diapreresnet | benchmark | arXiv | 46 | | 243 | diapreresnet164bn_cifar10 | CIFAR-10 | natural | 10 | 50000 | arXiv | 2019 | Singapore | 4 | 0.9644 | 0.7645 | 0.767 | 162 | 0 | 3 | 256 | AdaptiveAvgPool2d | True | 0.99992 | 0.5168 | 0.544901 | 0.5 | 0.03552 | -0.0168 | -0.0449011 | 0 | 0 | 1.92211e+06 | 0.6047 | 0.6042 | 0.6342 | 0.6388 | diapreresnet164bn | diapreresnet | benchmark | arXiv | 47 | | 244 | nin_cifar100 | CIFAR-100 | natural | 100 | 50000 | ICLR | 2014 | Singapore | 3 | 0.7114 | 0.5144 | 0.5675 | 9 | 0 | 5 | 192 | None | False | 0.8871 | 0.5827 | 0.611663 | 0.587583 | 0.1757 | 0.00488264 | -0.0240801 | 0.6004 | 0 | 984356 | 0.2727 | 0.2937 | 0.3656 | 0.414 | nin | nin | benchmark | ML | 48 | | 245 | resnet20_cifar100 | CIFAR-100 | natural | 100 | 50000 | CVPR | 2016 | USA | 4 | 0.7005 | 0.51 | 0.5602 | 21 | 0 | 3 | 64 | None | False | 0.840575 | 0.5645 | 0.577739 | 0.537171 | 0.140075 | -0.0273295 | -0.0405681 | 0.563852 | 0 | 278324 | 0.2831 | 0.3055 | 0.3673 | 0.408 | resnet20 | resnet | benchmark | CV | 49 | | 246 | resnet56_cifar100 | CIFAR-100 | natural | 100 | 50000 | CVPR | 2016 | USA | 4 | 0.7509 | 0.478 | 0.5026 | 57 | 0 | 3 | 64 | None | False | 0.974875 | 0.6077 | 0.665123 | 0.635746 | 0.223975 | 0.0280461 | -0.029377 | 0.646575 | 0 | 861620 | 0.2589 | 0.2685 | 0.3139 | 0.3348 | resnet56 | resnet | benchmark | CV | 50 | | 247 | resnet110_cifar100 | CIFAR-100 | natural | 100 | 50000 | CVPR | 2016 | USA | 4 | 0.7714 | 0.4602 | 0.4721 | 111 | 0 | 3 | 64 | None | False | 0.993975 | 0.6077 | 0.697375 | 0.679035 | 0.222575 | 0.0713354 | -0.0183392 | 0.680381 | 0 | 1.73656e+06 | 0.223 | 0.2283 | 0.2767 | 0.29 | resnet110 | resnet | benchmark | CV | 51 | | 248 | resnet164bn_cifar100 | CIFAR-100 | natural | 100 | 50000 | CVPR | 2016 | USA | 4 | 0.7936 | 0.4688 | 0.4857 | 166 | 0 | 3 | 256 | None | False | 0.997 | 0.5998 | 0.7019 | 0.682774 | 0.2034 | 0.0829744 | -0.019126 | 0.689727 | 0 | 1.72728e+06 | 0.2336 | 0.2361 | 0.2789 | 0.2943 | resnet164bn | resnet | benchmark | CV | 52 | | 249 | resnet272bn_cifar100 | CIFAR-100 | natural | 100 | 50000 | CVPR | 2016 | USA | 4 | 0.7986 | 0.4719 | 0.4834 | 274 | 0 | 3 | 256 | None | False | 0.998975 | 0.596 | 0.712265 | 0.694759 | 0.200375 | 0.098759 | -0.017506 | 0.702546 | 0 | 2.84012e+06 | 0.2282 | 0.2331 | 0.2698 | 0.2785 | resnet272bn | resnet | benchmark | CV | 53 | | 250 | resnet542bn_cifar100 | CIFAR-100 | natural | 100 | 50000 | CVPR | 2016 | USA | 4 | 0.8057 | 0.4498 | 0.4623 | 544 | 0 | 3 | 256 | None | False | 0.999125 | 0.5935 | 0.730286 | 0.678164 | 0.193425 | 0.0846641 | -0.0521217 | 0.729295 | 0 | 5.6222e+06 | 0.2157 | 0.221 | 0.2467 | 0.2642 | resnet542bn | resnet | benchmark | CV | 54 | | 251 | resnet1001_cifar100 | CIFAR-100 | natural | 100 | 50000 | CVPR | 2016 | USA | 4 | 0.8003 | 0.4417 | 0.4464 | 1003 | 0 | 3 | 256 | None | False | 0.999475 | 0.5968 | 0.724086 | 0.668408 | 0.199175 | 0.0716081 | -0.0556782 | 0.730516 | 0 | 1.03517e+07 | 0.2207 | 0.2223 | 0.2542 | 0.2621 | resnet1001 | resnet | benchmark | CV | 55 | | 252 | resnet1202_cifar100 | CIFAR-100 | natural | 100 | 50000 | CVPR | 2016 | USA | 4 | 0.7852 | 0.4479 | 0.4544 | 1203 | 0 | 3 | 64 | None | False | 0.999525 | 0.6029 | 0.72507 | 0.697106 | 0.214325 | 0.094206 | -0.0279636 | 0.734459 | 0 | 1.94299e+07 | 0.2237 | 0.2285 | 0.2771 | 0.2895 | resnet1202 | resnet | benchmark | CV | 56 | | 253 | preresnet20_cifar100 | CIFAR-100 | natural | 100 | 50000 | ECCV | 2016 | USA | 4 | 0.6967 | 0.5154 | 0.5648 | 18 | 0 | 3 | 64 | None | False | 0.847225 | 0.5697 | 0.582448 | 0.537403 | 0.150525 | -0.0322966 | -0.0450449 | 0.561095 | 0 | 278132 | 0.2841 | 0.3061 | 0.3772 | 0.4204 | preresnet20 | preresnet | benchmark | CV | 57 | | 254 | preresnet56_cifar100 | CIFAR-100 | natural | 100 | 50000 | ECCV | 2016 | USA | 4 | 0.7488 | 0.4837 | 0.5121 | 54 | 0 | 3 | 64 | None | False | 0.970475 | 0.6064 | 0.659997 | 0.629191 | 0.221675 | 0.0227911 | -0.0308055 | 0.639779 | 0 | 861428 | 0.2561 | 0.265 | 0.3175 | 0.3384 | preresnet56 | preresnet | benchmark | CV | 58 | | 255 | preresnet110_cifar100 | CIFAR-100 | natural | 100 | 50000 | ECCV | 2016 | USA | 4 | 0.7727 | 0.4646 | 0.4777 | 108 | 0 | 3 | 64 | None | False | 0.993775 | 0.6079 | 0.700955 | 0.67086 | 0.221075 | 0.0629598 | -0.0300953 | 0.68399 | 0 | 1.73637e+06 | 0.2311 | 0.2357 | 0.2846 | 0.3009 | preresnet110 | preresnet | benchmark | CV | 59 | | 256 | preresnet164bn_cifar100 | CIFAR-100 | natural | 100 | 50000 | ECCV | 2016 | USA | 4 | 0.7958 | 0.4571 | 0.4686 | 162 | 0 | 3 | 256 | None | False | 0.99695 | 0.6013 | 0.691288 | 0.667859 | 0.20115 | 0.0665589 | -0.0234287 | 0.678803 | 0 | 1.72639e+06 | 0.2298 | 0.235 | 0.2573 | 0.2752 | preresnet164bn | preresnet | benchmark | CV | 60 | | 257 | preresnet272bn_cifar100 | CIFAR-100 | natural | 100 | 50000 | ECCV | 2016 | USA | 4 | 0.8035 | 0.4338 | 0.4428 | 270 | 0 | 3 | 256 | None | False | 0.99895 | 0.597 | 0.682814 | 0.63975 | 0.19545 | 0.04275 | -0.0430645 | 0.68689 | 0 | 2.83922e+06 | 0.2189 | 0.2205 | 0.2551 | 0.266 | preresnet272bn | preresnet | benchmark | CV | 61 | | 258 | preresnet542bn_cifar100 | CIFAR-100 | natural | 100 | 50000 | ECCV | 2016 | USA | 4 | 0.8124 | 0.3841 | 0.3888 | 540 | 0 | 3 | 256 | None | False | 0.998975 | 0.5939 | 0.652887 | 0.626884 | 0.186575 | 0.032984 | -0.026003 | 0.683382 | 0 | 5.6213e+06 | 0.1559 | 0.1569 | 0.2016 | 0.2084 | preresnet542bn | preresnet | benchmark | CV | 62 | | 259 | preresnet1001_cifar100 | CIFAR-100 | natural | 100 | 50000 | ECCV | 2016 | USA | 4 | 0.8172 | 0.4108 | 0.4158 | 999 | 0 | 3 | 256 | None | False | 0.999625 | 0.5891 | 0.652603 | 0.578015 | 0.182425 | -0.0110846 | -0.0745875 | 0.694508 | 0 | 1.03508e+07 | 0.198 | 0.1995 | 0.2173 | 0.2222 | preresnet1001 | preresnet | benchmark | CV | 63 | | 260 | resnext29_32x4d_cifar100 | CIFAR-100 | natural | 100 | 50000 | CVPR | 2017 | USA | 5 | 0.8013 | 0.4846 | 0.498 | 31 | 0 | 3 | 1024 | None | False | 0.999725 | 0.6005 | 0.799214 | 0.796031 | 0.198425 | 0.195531 | -0.00318296 | 0.793407 | 0 | 4.868e+06 | 0.2512 | 0.2566 | 0.2492 | 0.2585 | resnext29 | resnext | benchmark | CV | 64 | | 261 | resnext29_16x64d_cifar100 | CIFAR-100 | natural | 100 | 50000 | CVPR | 2017 | USA | 5 | 0.8312 | 0.4344 | 0.4435 | 31 | 0 | 3 | 4096 | None | False | 0.999825 | 0.5846 | 0.828358 | 0.787312 | 0.168625 | 0.202712 | -0.0410457 | 0.829289 | 0 | 6.82475e+07 | 0.2095 | 0.2153 | 0.1974 | 0.2058 | resnext29 | resnext | benchmark | CV | 65 | | 262 | resnext272_1x64d_cifar100 | CIFAR-100 | natural | 100 | 50000 | CVPR | 2017 | USA | 5 | 0.8086 | 0.4021 | 0.4093 | 274 | 0 | 3 | 1024 | None | False | 0.999775 | 0.593 | 0.671725 | 0.594859 | 0.191175 | 0.00185896 | -0.0768657 | 0.711573 | 0 | 4.4633e+07 | 0.2018 | 0.2027 | 0.2296 | 0.2366 | resnext272 | resnext | benchmark | CV | 66 | | 263 | resnext272_2x32d_cifar100 | CIFAR-100 | natural | 100 | 50000 | CVPR | 2017 | USA | 5 | 0.8172 | 0.3881 | 0.3962 | 274 | 0 | 3 | 1024 | None | False | 0.999575 | 0.5908 | 0.679502 | 0.676551 | 0.182375 | 0.0857514 | -0.00295096 | 0.687306 | 0 | 3.30208e+07 | 0.1761 | 0.1784 | 0.1938 | 0.1979 | resnext272 | resnext | benchmark | CV | 67 | | 264 | seresnet20_cifar100 | CIFAR-100 | natural | 100 | 50000 | CVPR | 2018 | China | 5 | 0.7151 | 0.4892 | 0.5337 | 21 | 0 | 3 | 64 | AdaptiveAvgPool2d | False | 0.873375 | 0.5716 | 0.593091 | 0.548527 | 0.158275 | -0.0230735 | -0.0445641 | 0.573683 | 0 | 280697 | 0.2711 | 0.2917 | 0.3434 | 0.379 | seresnet20 | seresnet | benchmark | CV | 68 | | 265 | seresnet56_cifar100 | CIFAR-100 | natural | 100 | 50000 | CVPR | 2018 | China | 5 | 0.7694 | 0.4725 | 0.4896 | 57 | 0 | 3 | 64 | AdaptiveAvgPool2d | False | 0.98625 | 0.6053 | 0.678951 | 0.645255 | 0.21685 | 0.0399549 | -0.0336963 | 0.659311 | 0 | 868739 | 0.2385 | 0.2456 | 0.2922 | 0.3119 | seresnet56 | seresnet | benchmark | CV | 69 | | 266 | seresnet110_cifar100 | CIFAR-100 | natural | 100 | 50000 | CVPR | 2018 | China | 5 | 0.7908 | 0.4583 | 0.4688 | 111 | 0 | 3 | 64 | AdaptiveAvgPool2d | False | 0.996725 | 0.5981 | 0.696642 | 0.667773 | 0.205925 | 0.0696725 | -0.0288695 | 0.683888 | 0 | 1.7508e+06 | 0.237 | 0.2416 | 0.277 | 0.2936 | seresnet110 | seresnet | benchmark | CV | 70 | | 267 | seresnet164bn_cifar100 | CIFAR-100 | natural | 100 | 50000 | CVPR | 2018 | China | 5 | 0.8004 | 0.4579 | 0.4696 | 166 | 0 | 3 | 256 | AdaptiveAvgPool2d | False | 0.997625 | 0.5969 | 0.700925 | 0.676347 | 0.197225 | 0.0794469 | -0.0245778 | 0.677651 | 0 | 1.92939e+06 | 0.2284 | 0.2333 | 0.2411 | 0.2567 | seresnet164bn | seresnet | benchmark | CV | 71 | | 268 | seresnet272bn_cifar100 | CIFAR-100 | natural | 100 | 50000 | CVPR | 2018 | China | 5 | 0.8108 | 0.4494 | 0.4579 | 274 | 0 | 3 | 256 | AdaptiveAvgPool2d | False | 0.9992 | 0.5949 | 0.719846 | 0.682662 | 0.1884 | 0.0877616 | -0.0371843 | 0.709524 | 0 | 3.17696e+06 | 0.2174 | 0.2196 | 0.2145 | 0.2266 | seresnet272bn | seresnet | benchmark | CV | 72 | | 269 | seresnet542bn_cifar100 | CIFAR-100 | natural | 100 | 50000 | CVPR | 2018 | China | 5 | 0.8106 | 0.428 | 0.4366 | 544 | 0 | 3 | 256 | AdaptiveAvgPool2d | False | 0.999425 | 0.5936 | 0.700769 | 0.669824 | 0.188825 | 0.0762245 | -0.0309443 | 0.707285 | 0 | 6.29588e+06 | 0.2169 | 0.2218 | 0.2224 | 0.2312 | seresnet542bn | seresnet | benchmark | CV | 73 | | 270 | sepreresnet20_cifar100 | CIFAR-100 | natural | 100 | 50000 | CVPR | 2018 | China, UK | 5 | 0.7176 | 0.5119 | 0.5532 | 19 | 0 | 3 | 64 | AdaptiveAvgPool2d | False | 0.87705 | 0.5725 | 0.584723 | 0.563118 | 0.15945 | -0.00938168 | -0.0216045 | 0.564571 | 0 | 280409 | 0.2964 | 0.3133 | 0.3476 | 0.379 | sepreresnet20 | sepreresnet | benchmark | CV | 74 | | 271 | sepreresnet56_cifar100 | CIFAR-100 | natural | 100 | 50000 | CVPR | 2018 | China, UK | 5 | 0.7685 | 0.4884 | 0.5048 | 55 | 0 | 3 | 64 | AdaptiveAvgPool2d | False | 0.982375 | 0.6041 | 0.671403 | 0.635888 | 0.213875 | 0.031788 | -0.0355154 | 0.65433 | 0 | 868451 | 0.2606 | 0.2645 | 0.3127 | 0.3308 | sepreresnet56 | sepreresnet | benchmark | CV | 75 | | 272 | sepreresnet110_cifar100 | CIFAR-100 | natural | 100 | 50000 | CVPR | 2018 | China, UK | 5 | 0.7735 | 0.4778 | 0.4905 | 109 | 0 | 3 | 64 | AdaptiveAvgPool2d | False | 0.99325 | 0.6076 | 0.70542 | 0.666721 | 0.21975 | 0.0591211 | -0.0386989 | 0.686985 | 0 | 1.75051e+06 | 0.2503 | 0.2542 | 0.3118 | 0.3232 | sepreresnet110 | sepreresnet | benchmark | CV | 76 | | 273 | sepreresnet164bn_cifar100 | CIFAR-100 | natural | 100 | 50000 | CVPR | 2018 | China, UK | 5 | 0.7998 | 0.4748 | 0.4891 | 163 | 0 | 3 | 256 | AdaptiveAvgPool2d | False | 0.9963 | 0.5936 | 0.708769 | 0.679216 | 0.1965 | 0.0856164 | -0.0295525 | 0.695111 | 0 | 1.92801e+06 | 0.2541 | 0.2581 | 0.2826 | 0.295 | sepreresnet164bn | sepreresnet | benchmark | CV | 77 | | 274 | sepreresnet272bn_cifar100 | CIFAR-100 | natural | 100 | 50000 | CVPR | 2018 | China, UK | 5 | 0.8095 | 0.4536 | 0.4643 | 271 | 0 | 3 | 256 | AdaptiveAvgPool2d | False | 0.998975 | 0.594 | 0.71414 | 0.689273 | 0.189475 | 0.0952729 | -0.024867 | 0.707148 | 0 | 3.17558e+06 | 0.225 | 0.2274 | 0.2585 | 0.2655 | sepreresnet272bn | sepreresnet | benchmark | CV | 78 | | 275 | sepreresnet542bn_cifar100 | CIFAR-100 | natural | 100 | 50000 | CVPR | 2018 | China, UK | 5 | 0.8059 | 0.4366 | 0.4413 | 541 | 0 | 3 | 256 | AdaptiveAvgPool2d | False | 0.999375 | 0.5951 | 0.69253 | 0.65104 | 0.193475 | 0.0559403 | -0.0414894 | 0.705435 | 0 | 6.2945e+06 | 0.2192 | 0.2223 | 0.2327 | 0.2355 | sepreresnet542bn | sepreresnet | benchmark | CV | 79 | | 276 | pyramidnet110_a48_cifar100 | CIFAR-100 | natural | 100 | 50000 | CVPR | 2017 | Korea | 3 | 0.7892 | 0.4677 | 0.4806 | 109 | 0 | 3 | 64 | None | False | 0.995475 | 0.5982 | 0.686859 | 0.664087 | 0.206275 | 0.0658875 | -0.0227714 | 0.67526 | 0 | 1.77856e+06 | 0.2383 | 0.2417 | 0.267 | 0.2761 | pyramidnet110 | pyramidnet | benchmark | CV | 80 | | 277 | pyramidnet110_a84_cifar100 | CIFAR-100 | natural | 100 | 50000 | CVPR | 2017 | Korea | 3 | 0.8092 | 0.4369 | 0.4443 | 109 | 0 | 3 | 100 | None | False | 0.999325 | 0.5937 | 0.713368 | 0.674718 | 0.190125 | 0.0810175 | -0.0386503 | 0.71118 | 0 | 3.91354e+06 | 0.235 | 0.2368 | 0.2267 | 0.2355 | pyramidnet110 | pyramidnet | benchmark | CV | 81 | | 278 | pyramidnet110_a270_cifar100 | CIFAR-100 | natural | 100 | 50000 | CVPR | 2017 | Korea | 3 | 0.8286 | 0.3831 | 0.3853 | 109 | 0 | 3 | 286 | None | False | 0.99985 | 0.5839 | 0.656641 | 0.585052 | 0.17125 | 0.00115216 | -0.0715893 | 0.715497 | 0 | 2.85113e+07 | 0.1856 | 0.1883 | 0.1837 | 0.1854 | pyramidnet110 | pyramidnet | benchmark | CV | 82 | | 279 | pyramidnet164_a270_bn_cifar100 | CIFAR-100 | natural | 100 | 50000 | CVPR | 2017 | Korea | 3 | 0.8317 | 0.4085 | 0.4126 | 163 | 0 | 3 | 1144 | None | False | 0.9998 | 0.5813 | 0.670954 | 0.586765 | 0.1681 | 0.0054653 | -0.0841887 | 0.716869 | 0 | 2.73191e+07 | 0.1952 | 0.1962 | 0.2158 | 0.2247 | pyramidnet164 | pyramidnet | benchmark | CV | 83 | | 280 | pyramidnet200_a240_bn_cifar100 | CIFAR-100 | natural | 100 | 50000 | CVPR | 2017 | Korea | 3 | 0.8369 | 0.3901 | 0.3951 | 199 | 0 | 3 | 1024 | None | False | 0.999825 | 0.5796 | 0.696951 | 0.539343 | 0.162925 | -0.0402567 | -0.157608 | 0.731073 | 0 | 2.6845e+07 | 0.2061 | 0.2093 | 0.2029 | 0.2114 | pyramidnet200 | pyramidnet | benchmark | CV | 84 | | 281 | pyramidnet236_a220_bn_cifar100 | CIFAR-100 | natural | 100 | 50000 | CVPR | 2017 | Korea | 3 | 0.8348 | 0.4113 | 0.4116 | 235 | 0 | 3 | 944 | None | False | 0.99975 | 0.5813 | 0.655019 | 0.50372 | 0.16495 | -0.0775796 | -0.151298 | 0.705333 | 0 | 2.70541e+07 | 0.1993 | 0.1999 | 0.2111 | 0.2163 | pyramidnet236 | pyramidnet | benchmark | CV | 85 | | 282 | pyramidnet272_a200_bn_cifar100 | CIFAR-100 | natural | 100 | 50000 | CVPR | 2017 | Korea | 3 | 0.8361 | 0.3633 | 0.3715 | 271 | 0 | 3 | 864 | None | False | 0.9998 | 0.581 | 0.659095 | 0.632474 | 0.1637 | 0.0514737 | -0.0266216 | 0.663015 | 0 | 2.62887e+07 | 0.1434 | 0.1451 | 0.1762 | 0.1801 | pyramidnet272 | pyramidnet | benchmark | CV | 86 | | 283 | densenet40_k12_cifar100 | CIFAR-100 | natural | 100 | 50000 | CVPR | 2017 | USA, China | 4 | 0.7502 | 0.5004 | 0.5354 | 76 | 0 | 3 | 114 | AvgPool2d | False | 0.92575 | 0.5842 | 0.606827 | 0.578072 | 0.17555 | -0.0061283 | -0.0287548 | 0.597002 | 0 | 622360 | 0.2725 | 0.2839 | 0.31 | 0.3416 | densenet40 | densenet | benchmark | CV | 87 | | 284 | densenet40_k12_bc_cifar100 | CIFAR-100 | natural | 100 | 50000 | CVPR | 2017 | USA, China | 4 | 0.7172 | 0.4978 | 0.5434 | 40 | 0 | 3 | 60 | AvgPool2d | False | 0.84745 | 0.559 | 0.56928 | 0.538422 | 0.13025 | -0.0205783 | -0.0308584 | 0.552722 | 0 | 188092 | 0.2783 | 0.2989 | 0.3312 | 0.37 | densenet40 | densenet | benchmark | CV | 88 | | 285 | densenet40_k24_bc_cifar100 | CIFAR-100 | natural | 100 | 50000 | CVPR | 2017 | USA, China | 4 | 0.7721 | 0.4898 | 0.5111 | 40 | 0 | 3 | 120 | AvgPool2d | False | 0.98015 | 0.6011 | 0.663386 | 0.654078 | 0.20805 | 0.0529783 | -0.00930806 | 0.660533 | 0 | 714196 | 0.2781 | 0.2856 | 0.2868 | 0.3092 | densenet40 | densenet | benchmark | CV | 89 | | 286 | densenet40_k36_bc_cifar100 | CIFAR-100 | natural | 100 | 50000 | CVPR | 2017 | USA, China | 4 | 0.793 | 0.4812 | 0.4946 | 40 | 0 | 3 | 180 | AvgPool2d | False | 0.99525 | 0.6001 | 0.702701 | 0.694023 | 0.20225 | 0.0939226 | -0.00867864 | 0.693722 | 0 | 1.57841e+06 | 0.2547 | 0.261 | 0.2682 | 0.2817 | densenet40 | densenet | benchmark | CV | 90 | | 287 | densenet100_k12_cifar100 | CIFAR-100 | natural | 100 | 50000 | CVPR | 2017 | USA, China | 4 | 0.8006 | 0.4629 | 0.4799 | 196 | 0 | 3 | 294 | AvgPool2d | False | 0.998625 | 0.5977 | 0.725647 | 0.716717 | 0.198025 | 0.119017 | -0.00892972 | 0.71791 | 0 | 4.1296e+06 | 0.2393 | 0.2433 | 0.2381 | 0.2496 | densenet100 | densenet | benchmark | CV | 91 | | 288 | densenet100_k24_cifar100 | CIFAR-100 | natural | 100 | 50000 | CVPR | 2017 | USA, China | 4 | 0.8173 | 0.4285 | 0.4323 | 196 | 0 | 3 | 588 | AvgPool2d | False | 0.999775 | 0.5897 | 0.739842 | 0.70302 | 0.182475 | 0.11332 | -0.0368222 | 0.747752 | 0 | 1.62363e+07 | 0.2087 | 0.2111 | 0.2099 | 0.2179 | densenet100 | densenet | benchmark | CV | 92 | | 289 | densenet100_k12_bc_cifar100 | CIFAR-100 | natural | 100 | 50000 | CVPR | 2017 | USA, China | 4 | 0.7857 | 0.4903 | 0.5088 | 100 | 0 | 3 | 150 | AvgPool2d | False | 0.986825 | 0.5993 | 0.679106 | 0.663346 | 0.201125 | 0.0640459 | -0.0157605 | 0.668854 | 0 | 800032 | 0.2631 | 0.272 | 0.2854 | 0.2995 | densenet100 | densenet | benchmark | CV | 93 | | 290 | densenet250_k24_bc_cifar100 | CIFAR-100 | natural | 100 | 50000 | CVPR | 2017 | USA, China | 4 | 0.8247 | 0.4028 | 0.4068 | 250 | 0 | 3 | 750 | AvgPool2d | False | 0.9998 | 0.586 | 0.698773 | 0.618294 | 0.1751 | 0.0322936 | -0.0804789 | 0.632012 | 0 | 1.54806e+07 | 0.2073 | 0.2079 | 0.2185 | 0.2253 | densenet250 | densenet | benchmark | CV | 94 | | 291 | xdensenet40_2_k24_bc_cifar100 | CIFAR-100 | natural | 100 | 50000 | ECCV | 2018 | India | 3 | 0.7583 | 0.4971 | 0.5262 | 40 | 0 | 3 | 120 | AvgPool2d | False | 0.941025 | 0.5887 | 0.615944 | 0.586737 | 0.182725 | -0.00196344 | -0.029207 | 0.598461 | 0 | 1.34319e+06 | 0.2673 | 0.28 | 0.302 | 0.3313 | xdensenet40 | xdensenet | benchmark | CV | 95 | | 292 | xdensenet40_2_k36_bc_cifar100 | CIFAR-100 | natural | 100 | 50000 | ECCV | 2018 | India | 3 | 0.7816 | 0.4878 | 0.5103 | 40 | 0 | 3 | 180 | AvgPool2d | False | 0.9856 | 0.5982 | 0.67192 | 0.66524 | 0.204 | 0.0670398 | -0.0066798 | 0.662579 | 0 | 2.99364e+06 | 0.256 | 0.2628 | 0.295 | 0.3122 | xdensenet40 | xdensenet | benchmark | CV | 96 | | 293 | wrn16_10_cifar100 | CIFAR-100 | natural | 100 | 50000 | arXiv | 2017 | France | 2 | 0.8104 | 0.4602 | 0.4735 | 12 | 0 | 3 | 640 | None | False | 0.999875 | 0.5938 | 0.804307 | 0.800158 | 0.189475 | 0.206359 | -0.00414846 | 0.797398 | 0 | 1.71743e+07 | 0.2215 | 0.23 | 0.2346 | 0.2488 | wrn16 | wrn | benchmark | arXiv | 97 | | 294 | wrn28_10_cifar100 | CIFAR-100 | natural | 100 | 50000 | arXiv | 2017 | France | 2 | 0.8194 | 0.369 | 0.3757 | 24 | 0 | 3 | 640 | None | False | 0.9998 | 0.59 | 0.693119 | 0.691371 | 0.1804 | 0.101371 | -0.0017475 | 0.736118 | 0 | 3.65369e+07 | 0.1596 | 0.1603 | 0.1831 | 0.1923 | wrn28 | wrn | benchmark | arXiv | 98 | | 295 | wrn40_8_cifar100 | CIFAR-100 | natural | 100 | 50000 | arXiv | 2017 | France | 2 | 0.8185 | 0.3607 | 0.3678 | 36 | 0 | 3 | 512 | None | False | 0.999825 | 0.5887 | 0.662958 | 0.655019 | 0.181325 | 0.066319 | -0.00793874 | 0.717039 | 0 | 3.57945e+07 | 0.1515 | 0.1572 | 0.1864 | 0.1924 | wrn40 | wrn | benchmark | arXiv | 99 | | 296 | wrn20_10_1bit_cifar100 | CIFAR-100 | natural | 100 | 50000 | arXiv | 2017 | France | 2 | 0.8096 | 0.4721 | 0.489 | 19 | 0 | 3 | 640 | None | False | 0.99965 | 0.5916 | 0.78311 | 0.788352 | 0.19005 | 0.196752 | 0.00524144 | 0.777234 | 0 | 2.67949e+07 | 0.2108 | 0.2177 | 0.2567 | 0.2683 | wrn20 | wrn | benchmark | arXiv | 100 | | 297 | wrn20_10_32bit_cifar100 | CIFAR-100 | natural | 100 | 50000 | arXiv | 2017 | France | 2 | 0.8188 | 0.4274 | 0.4393 | 19 | 0 | 3 | 640 | None | False | 0.999825 | 0.5891 | 0.819226 | 0.815257 | 0.181025 | 0.226157 | -0.0039692 | 0.817305 | 0 | 2.67949e+07 | 0.2122 | 0.2193 | 0.2317 | 0.24 | wrn20 | wrn | benchmark | arXiv | 101 | | 298 | ror3_56_cifar100 | CIFAR-100 | natural | 100 | 50000 | IEEE | 2017 | China, USA | 6 | 0.7456 | 0.4923 | 0.5111 | 55 | 0 | 3 | 64 | MaxPool2d | False | 0.96135 | 0.6015 | 0.647528 | 0.614778 | 0.21575 | 0.0132782 | -0.0327498 | 0.627412 | 0 | 768596 | 0.257 | 0.2647 | 0.3185 | 0.3399 | ror3 | ror | benchmark | other | 102 | | 299 | ror3_110_cifar100 | CIFAR-100 | natural | 100 | 50000 | IEEE | 2017 | China, USA | 6 | 0.7597 | 0.4805 | 0.4917 | 109 | 0 | 3 | 64 | MaxPool2d | False | 0.992575 | 0.6127 | 0.697416 | 0.672546 | 0.232875 | 0.0598456 | -0.02487 | 0.680961 | 0 | 1.64354e+06 | 0.2563 | 0.2622 | 0.2982 | 0.3178 | ror3 | ror | benchmark | other | 103 | -------------------------------------------------------------------------------- /evaluation/backdoor/readme.md: -------------------------------------------------------------------------------- 1 | # Backdoor Detection 2 | -------------------------------------------------------------------------------- /evaluation/membership/readme.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /evaluation/modelsteal/readme.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /evaluation/readme.md: -------------------------------------------------------------------------------- 1 | # Evaluation Code 2 | This directory contains all codes used for evaluating the models. 3 | -------------------------------------------------------------------------------- /figures/SecurityNet.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/SecurityNet-Research/SecurityNet/e55bb9d8c1b9882c654d302bda10b6668af9dee8/figures/SecurityNet.png --------------------------------------------------------------------------------