├── .gitignore ├── README.md ├── all_in_one_cifar10.py ├── all_in_one_imagenet.py ├── attacks.py ├── cifar10_config.py ├── cifar10models.py ├── demo.py ├── imagenet_config.py ├── imagenet_labels.txt ├── test_images ├── bear_test_image_label_296.JPEG └── soccer_ball_test_image_label_805.JPEG └── visualize.ipynb /.gitignore: -------------------------------------------------------------------------------- 1 | .DS_Store 2 | **/.DS_Store 3 | data/ 4 | __pycache__/ 5 | .ipynb_checkpoints/ 6 | 7 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Enhancing Adversarial Example Transferability with an Intermediate Level Attack 2 | This repository includes the official PyTorch implementation of [Enhancing 3 | Adversarial Example Transferability with an Intermediate Level Attack 4 | ](https://arxiv.org/abs/1907.10823) (ICCV 2019). 5 | 6 | Summary: We fine-tune adversarial perturbations for better transferability by 7 | maximizing projection onto the perturbation at an intermediate layer. We 8 | demonstrate improved transferability across a wide range of attacks, 9 | including SOTA ones. In addition, we demonstrate that the choice of layer 10 | makes a substantial impact on transferability. 11 | 12 | ## Software Requirements 13 | This codebase requires Python 3, PyTorch 1.0+, Torchvision 0.2+, and pretrainedmodels ([Cadene's repo](https://github.com/Cadene/pretrained-models.pytorch), installed via `pip install pretrainedmodels`). In principle, this code can be run on CPU but we assume GPU utilization throughout the codebase. 14 | 15 | ## Usage 16 | 17 | ### Demo 18 | 19 | To generate an adversarial example with enhanced transferability using our method (using I-FGSM as the baseline attack), please run a command such as the following: 20 | 21 | ``` 22 | python demo.py --modeltype ResNet18 --layerindex 4 --imagepath test_images/bear_test_image_label_296.JPEG --imagelabel 296 --outpath adv_out.jpg --epsilon 0.03 23 | ``` 24 | 25 | This command uses ILA to generate a transferable adversarial (with epsilon=0.03) for the given bear image. 26 | 27 | The output is: 28 | ```color 29 | True label: 296 (ice bear, polar bear, Ursus Maritimus, Thalarctos maritimus) 30 | ResNet18 (source model) 31 | Prediction on original: 296 (ice bear, polar bear, Ursus Maritimus, Thalarctos maritimus) 32 | Prediction on I-FGSM: 257 (Great Pyrenees) 33 | Prediction on ILA: 155 (Shih-Tzu) 34 | 35 | ---Transfer Results Follow--- 36 | DenseNet121 (transfer model) 37 | Prediction on original: 296 (ice bear, polar bear, Ursus Maritimus, Thalarctos maritimus) 38 | Prediction on I-FGSM: 296 (ice bear, polar bear, Ursus Maritimus, Thalarctos maritimus) 39 | Prediction on ILA: 222 (kuvasz) 40 | 41 | alexnet (transfer model) 42 | Prediction on original: 296 (ice bear, polar bear, Ursus Maritimus, Thalarctos maritimus) 43 | Prediction on I-FGSM: 296 (ice bear, polar bear, Ursus Maritimus, Thalarctos maritimus) 44 | Prediction on ILA: 222 (kuvasz) 45 | 46 | SqueezeNet1.0 (transfer model) 47 | Prediction on original: 296 (ice bear, polar bear, Ursus Maritimus, Thalarctos maritimus) 48 | Prediction on I-FGSM: 257 (Great Pyrenees) 49 | Prediction on ILA: 257 (Great Pyrenees) 50 | ``` 51 | 52 | As can be seen from the output, the original I-FGSM perturbation on ResNet18 transfers to only 1 model (SqueezeNet), whereas the ILA perturbation transfers to all 3 models. 53 | 54 | ### Evaluate on All Images of a Dataset 55 | 56 | To evaluate the performance of the attack on cifar10, run 57 | 58 | ``` 59 | python all_in_one_cifar10.py --source_models ResNet18 --transfer_models ResNet18 DenseNet121 GoogLeNet SENet18 --out_name=test.csv --attacks ifgsm --num_batches=50 --batch_size=32 60 | ``` 61 | [Checkpoints](https://drive.google.com/drive/folders/1RGtlPCc2vTqeQc5utOgLb1Y3_vIO5JVi?usp=sharing) of our models. You can replace the checkpoints paths in `cifar10_config.py`. 62 | 63 | Full usage: 64 | 65 | ``` 66 | usage: all_in_one_cifar10.py [-h] --source_models SOURCE_MODELS 67 | [SOURCE_MODELS ...] --transfer_models 68 | TRANSFER_MODELS [TRANSFER_MODELS ...] --attacks 69 | ATTACKS [ATTACKS ...] --num_batches NUM_BATCHES 70 | --batch_size BATCH_SIZE --out_name OUT_NAME 71 | 72 | optional arguments: 73 | -h, --help show this help message and exit 74 | --source_models SOURCE_MODELS [SOURCE_MODELS ...] 75 | source models 76 | --transfer_models TRANSFER_MODELS [TRANSFER_MODELS ...] 77 | transfer models 78 | --attacks ATTACKS [ATTACKS ...] 79 | base attacks 80 | --num_batches NUM_BATCHES 81 | number of batches 82 | --batch_size BATCH_SIZE 83 | batch size 84 | --out_name OUT_NAME out file name 85 | ``` 86 | 87 | Run `all_in_one_imagenet.py` to evaluate on imagenet with similar usage, with the imagenet val folder path. 88 | 89 | To visualize the output csv of any of the above runs, simply open the `visualize.ipynb` notebook and change `df = pd.read_csv('ifgsm_imagenet_0.03_new.csv')` to read from your specific csv. Then run all the cells in the notebook to obtain a concise summary. 90 | 91 | ## Attribution 92 | 93 | If you use this code or our results in your research, please cite: 94 | 95 | ``` 96 | @article{Huang2019EnhancingAE, 97 | title={Enhancing Adversarial Example Transferability with an Intermediate Level Attack}, 98 | author={Qian Huang and Isay Katsman and Horace He and Zeqi Gu and Serge J. Belongie and Ser-Nam Lim}, 99 | journal={ArXiv}, 100 | year={2019}, 101 | volume={abs/1907.10823} 102 | } 103 | ``` 104 | -------------------------------------------------------------------------------- /all_in_one_cifar10.py: -------------------------------------------------------------------------------- 1 | import argparse 2 | from attacks import wrap_attack, wrap_cw_linf, ifgsm, momentum_ifgsm, deepfool, CW_Linf, Transferable_Adversarial_Perturbations, ILA 3 | from cifar10models import * 4 | from cifar10_config import * 5 | import numpy as np 6 | import pandas as pd 7 | from tqdm import tqdm 8 | 9 | def get_args(): 10 | parser = argparse.ArgumentParser() 11 | parser.add_argument('--source_models', nargs='+', help=' source models', required=True) 12 | parser.add_argument('--transfer_models', nargs='+', help=' transfer models', required=True) 13 | parser.add_argument('--attacks', nargs='+', help=' base attacks', required=True) 14 | parser.add_argument('--num_batches', type=int, help=' number of batches', required=True) 15 | parser.add_argument('--batch_size', type=int, help=' batch size', required=True) 16 | parser.add_argument('--out_name', help=' out file name', required=True) 17 | args = parser.parse_args() 18 | return args 19 | 20 | def log(out_df, source_model, source_model_file, target_model, target_model_file, batch_index, layer_index, layer_name, fool_method, with_ILA, fool_rate, acc_after_attack, original_acc): 21 | return out_df.append({ 22 | 'source_model':model_name(source_model), 23 | 'source_model_file': source_model_file, 24 | 'target_model':model_name(target_model), 25 | 'target_model_file': target_model_file, 26 | 'batch_index':batch_index, 27 | 'layer_index':layer_index, 28 | 'layer_name':layer_name, 29 | 'fool_method':fool_method, 30 | 'with_ILA':with_ILA, 31 | 'fool_rate':fool_rate, 32 | 'acc_after_attack':acc_after_attack, 33 | 'original_acc':original_acc},ignore_index=True) 34 | 35 | 36 | def get_data(batch_size, mean, stddev): 37 | transform = transforms.Compose([transforms.ToTensor(), transforms.Normalize(mean, stddev)]) 38 | trainset = torchvision.datasets.CIFAR10(root='./data', train=True, download=True, transform=transform) 39 | trainloader = torch.utils.data.DataLoader(trainset, batch_size=batch_size, shuffle=True, num_workers=0) 40 | 41 | testset = torchvision.datasets.CIFAR10(root='./data', train=False, download=True, transform=transform) 42 | testloader = torch.utils.data.DataLoader(testset, batch_size=batch_size, shuffle=False, num_workers=0) 43 | return trainloader, testloader 44 | 45 | def get_fool_adv_orig(model, adversarial_xs, originals, labels): 46 | total = adversarial_xs.size(0) 47 | correct_orig = 0 48 | correct_adv = 0 49 | fooled = 0 50 | 51 | advs, ims, lbls = adversarial_xs.cuda(), originals.cuda(), labels.cuda() 52 | outputs_adv = model(advs) 53 | outputs_orig = model(ims) 54 | _, predicted_adv = torch.max(outputs_adv.data, 1) 55 | _, predicted_orig = torch.max(outputs_orig.data, 1) 56 | 57 | correct_adv += (predicted_adv == lbls).sum() 58 | correct_orig += (predicted_orig == lbls).sum() 59 | fooled += (predicted_adv != predicted_orig).sum() 60 | return [100.0 * float(fooled.item())/total, 100.0 * float(correct_adv.item())/total, 100.0 * float(correct_orig.item())/total] 61 | 62 | 63 | def test_adv_examples_across_models(transfer_models, adversarial_xs, originals, labels): 64 | accum = [] 65 | for (network, weights_path) in transfer_models: 66 | net = network().cuda() 67 | net.load_state_dict(torch.load(weights_path, map_location=lambda storage, loc: storage)) 68 | net.eval() 69 | res = get_fool_adv_orig(net, adversarial_xs, originals, labels) 70 | res.append(weights_path) 71 | accum.append(res) 72 | return accum 73 | 74 | 75 | def complete_loop(sample_num, batch_size, attacks, source_models, transfer_models, out_name): 76 | out_df = pd.DataFrame(columns=['source_model', 'source_model_file', 'target_model','target_model_file', 'batch_index','layer_index', 'layer_name', 'fool_method', 'with_ILA', 'fool_rate', 'acc_after_attack', 'original_acc']) 77 | 78 | trainloader, testloader = get_data(batch_size, *data_preprocess) 79 | for model_class, source_weight_path in source_models: 80 | model = model_class().cuda() 81 | model.load_state_dict(torch.load(source_weight_path)) 82 | model.eval() 83 | dic = model._modules 84 | for attack_name, attack in attacks: 85 | print('using source model {0} attack {1}'.format(model_name(model_class), attack_name)) 86 | iterator = tqdm(enumerate(testloader, 0)) 87 | for batch_i, data in iterator: 88 | if batch_i == sample_num: 89 | iterator.close() 90 | break 91 | images, labels = data 92 | images, labels = images.cuda(), labels.cuda() 93 | 94 | 95 | #### baseline 96 | 97 | ### generate 98 | adversarial_xs = attack(model, images, labels, niters= 20) 99 | 100 | ### eval 101 | transfer_list = test_adv_examples_across_models(transfer_models, adversarial_xs, images, labels) 102 | for i, (target_fool_rate, target_acc_attack, target_acc_original, target_weight_path) in enumerate(transfer_list): 103 | out_df = log(out_df,model_class, source_weight_path,transfer_models[i][0], 104 | target_weight_path, batch_i, np.nan, "", attack_name, False, 105 | target_fool_rate, target_acc_attack, target_acc_original) 106 | 107 | 108 | #### ILA 109 | 110 | ### generate 111 | ## step1: reference 112 | ILA_input_xs = attack(model, images, labels, niters= 10) 113 | 114 | ## step2: ILA target at different layers 115 | for layer_ind, layer_name in source_layers[model_name(model_class)]: 116 | ILA_adversarial_xs = ILA(model, images, X_attack=ILA_input_xs, y=labels, feature_layer=model._modules.get(layer_name), **(ILA_params[attack_name])) 117 | 118 | ### eval 119 | ILA_transfer_list = test_adv_examples_across_models(transfer_models, ILA_adversarial_xs, images, labels) 120 | for i, (fooling_ratio, accuracy_perturbed, accuracy_original, attacked_model_path) in enumerate(ILA_transfer_list): 121 | out_df = log(out_df,model_class,attacked_model_path, transfer_models[i][0], source_weight_path, batch_i, layer_ind, layer_name, attack_name, True, fooling_ratio, accuracy_perturbed, accuracy_original) 122 | 123 | 124 | #save csv 125 | out_df.to_csv(out_name, sep=',', encoding='utf-8') 126 | 127 | 128 | if __name__ == "__main__": 129 | args = get_args() 130 | attacks = list(map(lambda attack_name: (attack_name, attack_configs[attack_name]), args.attacks)) 131 | source_models = list(map(lambda model_name: model_configs[model_name], args.source_models)) 132 | transfer_models = list(map(lambda model_name: model_configs[model_name], args.transfer_models)) 133 | 134 | complete_loop(args.num_batches, args.batch_size, attacks, source_models, transfer_models, args.out_name); 135 | 136 | 137 | 138 | 139 | 140 | 141 | 142 | 143 | 144 | 145 | 146 | 147 | 148 | 149 | 150 | 151 | 152 | 153 | 154 | -------------------------------------------------------------------------------- /all_in_one_imagenet.py: -------------------------------------------------------------------------------- 1 | import argparse 2 | import numpy as np 3 | import pandas as pd 4 | import torch 5 | import torchvision 6 | import torchvision.models as models 7 | import torchvision.transforms as transforms 8 | from imagenet_config import * 9 | from attacks import wrap_attack_imagenet, momentum_ifgsm, Transferable_Adversarial_Perturbations, ILA, ifgsm 10 | 11 | def get_args(): 12 | parser = argparse.ArgumentParser() 13 | parser.add_argument('--source_models', nargs='+', help=' source models', required=True) 14 | parser.add_argument('--transfer_models', nargs='+', help=' transfer models', required=True) 15 | parser.add_argument('--attacks', nargs='+', help=' base attacks', required=True) 16 | parser.add_argument('--num_batches', type=int, help=' number of batches', required=True) 17 | parser.add_argument('--batch_size', type=int, help=' batch size', required=True) 18 | parser.add_argument('--out_name', help=' out file name', required=True) 19 | parser.add_argument('--use_Inc_model', action='store_true', help=' use Inception models group') 20 | args = parser.parse_args() 21 | return args 22 | 23 | def log(out_df, source_model_name, target_model_name, batch_index, layer_index, layer_name, fool_method, with_ILA, fool_rate, acc_after_attack, original_acc): 24 | return out_df.append({ 25 | 'source_model':source_model_name, 26 | 'target_model':target_model_name, 27 | 'batch_index':batch_index, 28 | 'layer_index':layer_index, 29 | 'layer_name':layer_name, 30 | 'fool_method':fool_method, 31 | 'with_ILA':with_ILA, 32 | 'fool_rate':fool_rate, 33 | 'acc_after_attack':acc_after_attack, 34 | 'original_acc':original_acc},ignore_index=True) 35 | 36 | 37 | def get_data(batch_size, use_Inc_model = False): 38 | 39 | if use_Inc_model: 40 | transform_test = transforms.Compose([ 41 | transforms.Resize(299), 42 | transforms.CenterCrop(299), 43 | transforms.ToTensor(), 44 | transforms.Normalize([0.5, 0.5, 0.5],[0.5, 0.5, 0.5]), 45 | ]) 46 | else: 47 | transform_test = transforms.Compose([ 48 | transforms.Resize(256), 49 | transforms.CenterCrop(224), 50 | transforms.ToTensor(), 51 | transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]), 52 | ]) 53 | 54 | testset = torchvision.datasets.ImageFolder(root='/share/cuvl/datasets/imagenet/val', 55 | transform=transform_test) 56 | testloader = torch.utils.data.DataLoader(testset, batch_size=batch_size, shuffle=True, 57 | num_workers=8, pin_memory=True) 58 | return testloader 59 | 60 | def get_fool_adv_orig(model, adversarial_xs, originals, labels): 61 | total = adversarial_xs.size(0) 62 | correct_orig = 0 63 | correct_adv = 0 64 | fooled = 0 65 | 66 | advs, ims, lbls = adversarial_xs.cuda(), originals.cuda(), labels.cuda() 67 | outputs_adv = model(advs) 68 | outputs_orig = model(ims) 69 | _, predicted_adv = torch.max(outputs_adv.data, 1) 70 | _, predicted_orig = torch.max(outputs_orig.data, 1) 71 | 72 | correct_adv += (predicted_adv == lbls).sum() 73 | correct_orig += (predicted_orig == lbls).sum() 74 | fooled += (predicted_adv != predicted_orig).sum() 75 | return [100.0 * float(fooled.item())/total, 100.0 * float(correct_adv.item())/total, 100.0 * float(correct_orig.item())/total] 76 | 77 | 78 | def test_adv_examples_across_models(transfer_models, adversarial_xs, originals, labels, use_Inc_model): 79 | accum = [] 80 | for name, net_class in transfer_models: 81 | if use_Inc_model: 82 | net = net_class(num_classes=1000, pretrained='imagenet').cuda() 83 | else: 84 | net = net_class(pretrained=True).cuda() 85 | net.eval() 86 | res = get_fool_adv_orig(net, adversarial_xs, originals, labels) 87 | res.append(name) 88 | accum.append(res) 89 | return accum 90 | 91 | 92 | def complete_loop(sample_num, batch_size, attacks, source_models, transfer_models, out_name, use_Inc_model): 93 | labels_file = open('labels', 'r').readlines() 94 | out_df = pd.DataFrame(columns=['source_model','target_model','batch_index','layer_index', 'layer_name', 'fool_method', 'with_ILA', 'fool_rate', 'acc_after_attack', 'original_acc']) 95 | testloader = get_data(batch_size, use_Inc_model) 96 | for source_model_name, model_class in source_models: 97 | if use_Inc_model: 98 | model = model_class(num_classes=1000, pretrained='imagenet').cuda() 99 | else: 100 | model = model_class(pretrained=True).cuda() 101 | model.eval() 102 | for attack_name, attack in attacks: 103 | print('using source model {0} attack {1}'.format(source_model_name, attack_name)) 104 | 105 | for batch_i, data in enumerate(testloader, 0): 106 | 107 | if batch_i%100 == 0: 108 | print("batch" , batch_i) 109 | save_to_csv(out_df, out_name) 110 | if batch_i == sample_num: 111 | break 112 | images, labels = data 113 | images, labels = images.cuda(), labels.cuda() 114 | 115 | #### baseline 116 | 117 | ### generate 118 | adversarial_xs = attack(model, images, labels, niters=20, use_Inc_model=use_Inc_model) 119 | 120 | ### eval 121 | transfer_list = test_adv_examples_across_models(transfer_models, adversarial_xs, images, labels, use_Inc_model) 122 | for target_fool_rate, target_acc_attack, target_acc_original, transfer_model_name in transfer_list: 123 | out_df = log(out_df,source_model_name, transfer_model_name, 124 | batch_i, np.nan, "", attack_name, False, 125 | target_fool_rate, target_acc_attack, target_acc_original) 126 | 127 | #### ILA 128 | 129 | ### generate 130 | ## step1: reference 131 | ILA_input_xs = attack(model, images, labels, niters=10, use_Inc_model=use_Inc_model) 132 | 133 | ## step2: ILA target at different layers 134 | for layer_ind, (layer_name, layer) in get_source_layers(source_model_name, model): 135 | ILA_adversarial_xs = ILA(model, images, X_attack=ILA_input_xs, y=labels, feature_layer=layer, use_Inc_model=use_Inc_model, **(ILA_params[attack_name])) 136 | 137 | ### eval 138 | ILA_transfer_list = test_adv_examples_across_models(transfer_models, ILA_adversarial_xs, images, labels, use_Inc_model) 139 | for target_fool_rate, target_acc_attack, target_acc_original, transfer_model_name in ILA_transfer_list: 140 | out_df = log(out_df,source_model_name, transfer_model_name, batch_i, layer_ind, layer_name, attack_name, True, target_fool_rate, target_acc_attack, target_acc_original) 141 | 142 | save_to_csv(out_df, out_name) 143 | 144 | 145 | def save_to_csv(out_df, out_name): 146 | #save csv 147 | out_df.to_csv(out_name, sep=',', encoding='utf-8') 148 | 149 | 150 | if __name__ == "__main__": 151 | args = get_args() 152 | attacks = list(map(lambda attack_name: (attack_name, attack_configs[attack_name]), args.attacks)) 153 | source_models = list(map(lambda model_name: model_configs[model_name], args.source_models)) 154 | transfer_models = list(map(lambda model_name: model_configs[model_name], args.transfer_models)) 155 | 156 | complete_loop(args.num_batches, args.batch_size, attacks, source_models, transfer_models, args.out_name, args.use_Inc_model); 157 | 158 | 159 | 160 | 161 | 162 | 163 | 164 | 165 | 166 | 167 | 168 | 169 | 170 | 171 | 172 | 173 | 174 | 175 | 176 | -------------------------------------------------------------------------------- /attacks.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | import random 3 | 4 | import matplotlib.pyplot as plt 5 | import numpy as np 6 | import torch 7 | import torch.nn as nn 8 | import torchvision 9 | from torch import optim 10 | from torch.autograd import Variable 11 | from scipy.misc import imread, imresize, imsave 12 | 13 | 14 | # helpers 15 | def avg(l): 16 | return sum(l) / len(l) 17 | 18 | 19 | def sow_images(images): 20 | """Sow batch of torch images (Bx3xWxH) into a grid PIL image (BWxHx3) 21 | 22 | Args: 23 | images: batch of torch images. 24 | 25 | Returns: 26 | The grid of images, as a numpy array in PIL format. 27 | """ 28 | images = torchvision.utils.make_grid( 29 | images 30 | ) # sow our batch of images e.g. (4x3x32x32) into a grid e.g. (3x32x128) 31 | 32 | mean_arr, stddev_arr = [0.5, 0.5, 0.5], [0.5, 0.5, 0.5] 33 | # denormalize 34 | for c in range(3): 35 | images[c, :, :] = images[c, :, :] * stddev_arr[c] + mean_arr[c] 36 | 37 | images = images.cpu().numpy() # go from Tensor to numpy array 38 | # switch channel order back from 39 | # torch Tensor to PIL image: going from 3x32x128 - to 32x128x3 40 | images = np.transpose(images, (1, 2, 0)) 41 | return images 42 | 43 | 44 | # normalization (L-inf norm projection) code for output delta 45 | def normalize_and_scale_imagenet(delta_im, epsilon, use_Inc_model): 46 | """Normalize and scale imagenet perturbation according to epsilon Linf norm 47 | 48 | Args: 49 | delta_im: perturbation on imagenet images 50 | epsilon: Linf norm 51 | 52 | Returns: 53 | The re-normalized perturbation 54 | """ 55 | 56 | if use_Inc_model: 57 | stddev_arr = [0.5, 0.5, 0.5] 58 | else: 59 | stddev_arr = [0.229, 0.224, 0.225] 60 | 61 | for ci in range(3): 62 | mag_in_scaled = epsilon / stddev_arr[ci] 63 | delta_im[:,ci] = delta_im[:,ci].clone().clamp(-mag_in_scaled, mag_in_scaled) 64 | 65 | return delta_im 66 | 67 | 68 | def renormalization(X, X_pert, epsilon, dataset="cifar10", use_Inc_model = False): 69 | """Normalize and scale perturbations according to epsilon Linf norm 70 | 71 | Args: 72 | X: original images 73 | X_pert: adversarial examples corresponding to X 74 | epsilon: Linf norm 75 | dataset: dataset images are from, 'cifar10' | 'imagenet' 76 | 77 | Returns: 78 | The re-normalized perturbation 79 | """ 80 | # make sure you don't modify the original image beyond epsilon, also clamp 81 | if dataset == "cifar10": 82 | eps_added = (X_pert.detach() - X.clone()).clamp(-epsilon, epsilon) + X.clone() 83 | # clamp 84 | return eps_added.clamp(-1.0, 1.0) 85 | elif dataset == "imagenet": 86 | eps_added = normalize_and_scale_imagenet(X_pert.detach() - X.clone(), epsilon, use_Inc_model) + X.clone() 87 | # clamp 88 | mean, stddev = [0.485, 0.456, 0.406], [0.229, 0.224, 0.225] 89 | for i in range(3): 90 | min_clamp = (0 - mean[i]) / stddev[i] 91 | max_clamp = (1 - mean[i]) / stddev[i] 92 | eps_added[:,i] = eps_added[:,i].clone().clamp(min_clamp, max_clamp) 93 | return eps_added 94 | 95 | 96 | def ifgsm( 97 | model, 98 | X, 99 | y, 100 | niters=10, 101 | epsilon=0.03, 102 | visualize=False, 103 | learning_rate=0.005, 104 | display=4, 105 | defense_model=False, 106 | setting="regular", 107 | dataset="cifar10", 108 | use_Inc_model = False, 109 | ): 110 | """Perform ifgsm attack with respect to model on images X with labels y 111 | 112 | Args: 113 | model: torch model with respect to which attacks will be computed 114 | X: batch of torch images 115 | y: labels corresponding to the batch of images 116 | niters: number of iterations of ifgsm to perform 117 | epsilon: Linf norm of resulting perturbation; scale of images is -1..1 118 | visualize: whether you want to visualize the perturbations or not 119 | learning_rate: learning rate of ifgsm 120 | display: number of images to display in visualization 121 | defense_model: set to true if you are using a defended model, 122 | e.g. ResNet18Defended, instead of the usual ResNet18 123 | setting: 'regular' is usual ifgsm, 'll' is least-likely ifgsm, and 124 | 'nonleaking' is non-label-leaking ifgsm 125 | dataset: dataset the images are from, 'cifar10' | 'imagenet' 126 | 127 | Returns: 128 | The batch of adversarial examples corresponding to the original images 129 | """ 130 | model.eval() 131 | out = None 132 | if defense_model: 133 | out = model(X)[0] 134 | else: 135 | out = model(X) 136 | y_ll = out.min(1)[1] # least likely model output 137 | y_ml = out.max(1)[1] # model label 138 | 139 | X_pert = X.clone() 140 | X_pert.requires_grad = True 141 | for i in range(niters): 142 | output_perturbed = None 143 | if defense_model: 144 | output_perturbed = model(X_pert)[0] 145 | else: 146 | output_perturbed = model(X_pert) 147 | 148 | y_used = y 149 | ll_factor = 1 150 | if setting == "ll": 151 | y_used = y_ll 152 | ll_factor = -1 153 | elif setting == "noleaking": 154 | y_used = y_ml 155 | 156 | loss = nn.CrossEntropyLoss()(output_perturbed, y_used) 157 | loss.backward() 158 | pert = ll_factor * learning_rate * X_pert.grad.detach().sign() 159 | 160 | # perform visualization 161 | if visualize is True and i == niters - 1: 162 | np_image = sow_images(X[:display].detach()) 163 | np_delta = sow_images(pert[:display].detach()) 164 | np_recons = sow_images( 165 | (X_pert.detach() + pert.detach()).clamp(-1, 1)[:display] 166 | ) 167 | 168 | fig = plt.figure(figsize=(8, 8)) 169 | fig.add_subplot(3, 1, 1) 170 | plt.axis("off") 171 | plt.imshow(np_recons) 172 | fig.add_subplot(3, 1, 2) 173 | plt.axis("off") 174 | plt.imshow(np_image) 175 | fig.add_subplot(3, 1, 3) 176 | plt.axis("off") 177 | plt.imshow(np_delta) 178 | plt.show() 179 | # end visualization 180 | 181 | # add perturbation 182 | X_pert = X_pert.detach() + pert 183 | X_pert.requires_grad = True 184 | 185 | # make sure we don't modify the original image beyond epsilon and clamp 186 | X_pert = renormalization(X, X_pert, epsilon, dataset=dataset, use_Inc_model=use_Inc_model) 187 | X_pert.requires_grad = True 188 | 189 | return X_pert 190 | 191 | 192 | def fgsm(model, X, y, epsilon=0.01, **args): 193 | """Perform cifar 10 fgsm attack with respect to model on images X with labels y 194 | 195 | Args: 196 | model: torch model with respect to which attacks will be computed 197 | X: batch of torch images 198 | y: labels corresponding to the batch of images 199 | epsilon: Linf norm of resulting perturbation; scale of images is -1..1 200 | 201 | Returns: 202 | The batch of adversarial examples corresponding to the original images 203 | """ 204 | if dataset != "cifar10": 205 | raise "fgsm as now does not support " + dataset 206 | 207 | X_pert = X.clone() 208 | X_pert.requires_grad = True 209 | output_perturbed = model(X_pert) 210 | loss = nn.CrossEntropyLoss()(output_perturbed, y) 211 | loss.backward() 212 | 213 | pert = epsilon * X_pert.grad.detach().sign() 214 | X_pert = X_pert.detach() + pert 215 | X_pert = X_pert.detach().clamp(X.min(), X.max()) 216 | return X_pert 217 | 218 | 219 | def momentum_ifgsm( 220 | model, 221 | X, 222 | y, 223 | niters=10, 224 | epsilon=0.03, 225 | visualize=False, 226 | learning_rate=0.005, 227 | decay=0.9, 228 | dataset="cifar10", 229 | use_Inc_model = False, 230 | ): 231 | """Perform momentum ifgsm attack with respect to model on images 232 | X with labels y 233 | 234 | Args: 235 | model: torch model with respect to which attacks will be computed 236 | X: batch of torch images 237 | y: labels corresponding to the batch of images 238 | niters: number of iterations of ifgsm to perform 239 | epsilon: Linf norm of resulting perturbation; scale of images is -1..1 240 | visualize: whether you want to visualize the perturbations or not 241 | learning_rate: learning rate of ifgsm 242 | decay: decay of momentum in the momentum ifgsm attack 243 | dataset: dataset the images are from, 'cifar10' | 'imagenet' 244 | 245 | Returns: 246 | The batch of adversarial examples corresponding to the original images 247 | """ 248 | X_pert = X.clone() 249 | X_pert.requires_grad = True 250 | 251 | momentum = 0 252 | for _ in range(niters): 253 | output_perturbed = model(X_pert) 254 | loss = nn.CrossEntropyLoss()(output_perturbed, y) 255 | loss.backward() 256 | 257 | momentum = decay * momentum + X_pert.grad / torch.sum(torch.abs(X_pert.grad)) 258 | pert = learning_rate * momentum.sign() 259 | 260 | # add perturbation 261 | X_pert = X_pert.detach() + pert 262 | X_pert.requires_grad = True 263 | 264 | # make sure we don't modify the original image beyond epsilon 265 | X_pert = renormalization(X, X_pert, epsilon, dataset=dataset, use_Inc_model=use_Inc_model) 266 | X_pert.requires_grad = True 267 | 268 | return X_pert 269 | 270 | 271 | def deepfool_single(net, image, num_classes=10, overshoot=0.02, max_iter=20): 272 | """Perform cifar10 deepfool attack with respect to net on image, 273 | pushing towards easiest target class as defined in the deepfool paper 274 | 275 | Args: 276 | net: torch model with respect to which attacks will be computed 277 | image: single image 278 | num_classes: number of classes net classifies 279 | overshoot: overshoot parameter of deepfool attack 280 | max_iter: maximum number of iterations of deepfool attack 281 | 282 | Returns: 283 | The adversarial examples corresponding to image 284 | """ 285 | import copy 286 | from torch.autograd.gradcheck import zero_gradients 287 | from torch.autograd import Variable 288 | 289 | f_image = ( 290 | net.forward(Variable(image[None, :, :, :], requires_grad=True)) 291 | .data.cpu() 292 | .numpy() 293 | .flatten() 294 | ) 295 | out = (np.array(f_image)).flatten().argsort()[::-1] 296 | 297 | out = out[0:num_classes] 298 | label = out[0] 299 | 300 | input_shape = image.cpu().numpy().shape 301 | pert_image = copy.deepcopy(image) 302 | w = np.zeros(input_shape) 303 | r_tot = np.zeros(input_shape) 304 | 305 | loop_i = 0 306 | 307 | x = Variable(pert_image[None, :], requires_grad=True) 308 | fs = net.forward(x) 309 | k_i = label 310 | 311 | while k_i == label and loop_i < max_iter: 312 | 313 | pert = np.inf 314 | fs[0, out[0]].backward(retain_graph=True) 315 | grad_orig = x.grad.data.cpu().numpy().copy() 316 | 317 | for k in range(1, num_classes): 318 | zero_gradients(x) 319 | 320 | fs[0, out[k]].backward(retain_graph=True) 321 | cur_grad = x.grad.data.cpu().numpy().copy() 322 | 323 | # set new w_k and new f_k 324 | w_k = cur_grad - grad_orig 325 | f_k = (fs[0, out[k]] - fs[0, out[0]]).data.cpu().numpy() 326 | 327 | pert_k = abs(f_k) / np.linalg.norm(w_k.flatten()) 328 | 329 | # determine which w_k to use 330 | if pert_k < pert: 331 | pert = pert_k 332 | w = w_k 333 | 334 | # compute r_i and r_tot 335 | # Added 1e-4 for numerical stability 336 | r_i = (pert + 1e-4) * w / np.linalg.norm(w) 337 | r_tot = np.float32(r_tot + r_i) 338 | 339 | pert_image = image + (1 + overshoot) * torch.from_numpy(r_tot).cuda() 340 | 341 | x = Variable(pert_image, requires_grad=True) 342 | fs = net.forward(x) 343 | k_i = np.argmax(fs.data.cpu().numpy().flatten()) 344 | 345 | loop_i += 1 346 | 347 | r_tot = (1 + overshoot) * r_tot 348 | return pert_image 349 | 350 | 351 | 352 | def deepfool(model, images, labels, num_classes=10, niters=50, epsilon=0.03, dataset="cifar10"): 353 | """Perform cifar 10 deepfool attack with respect to net on images with specified 354 | labels, pushing towards easiest target class as defined in the 355 | deepfool paper 356 | 357 | Args: 358 | model: torch model with respect to which attacks will be computed 359 | images: batch of torch images 360 | labels: groundtruth labels of corresponding images 361 | num_classes: number of classes net classifies 362 | niters: number of iterations 363 | epsilon: Linf norm of resulting perturbation; scale of images is -1..1 364 | 365 | Returns: 366 | The batch of adversarial examples corresponding to the original images 367 | """ 368 | if dataset != "cifar10": 369 | raise "deepfool as now does not support " + dataset 370 | r = torch.zeros(images.size()).cuda() 371 | for i in range(images.size(0)): 372 | r[i] = deepfool_single(model, images[i], num_classes, max_iter=niters) 373 | 374 | adv = (r - images).clamp(-epsilon, epsilon) + images 375 | adv = adv.clamp(-1.0, 1.0) 376 | return adv 377 | 378 | 379 | # CW 380 | 381 | # cifar only, -1..1 image range 382 | def tanh(x): 383 | return torch.nn.Tanh()(x) 384 | 385 | 386 | def return_max(x_tensor, y): 387 | if x_tensor.data > y: 388 | return x_tensor.data 389 | else: 390 | return y 391 | 392 | 393 | def torch_arctanh(x, eps=1e-6): 394 | x *= 1.0 - eps 395 | return (torch.log((1 + x) / (1 - x))) * 0.5 396 | 397 | 398 | class CW_Linf: 399 | def __init__( 400 | self, 401 | targeted=True, 402 | learning_rate=5e-3, 403 | max_iterations=1000, 404 | abort_early=True, 405 | initial_const=1e-5, 406 | largest_const=2e1, 407 | reduce_const=False, 408 | decrease_factor=0.9, 409 | const_factor=2.0, 410 | num_classes=10, 411 | ): 412 | self.TARGETED = targeted 413 | self.LEARNING_RATE = learning_rate 414 | self.MAX_ITERATIONS = max_iterations 415 | self.ABORT_EARLY = abort_early 416 | self.INITIAL_CONST = initial_const 417 | self.LARGEST_CONST = largest_const 418 | self.DECREASE_FACTOR = decrease_factor 419 | self.REDUCE_CONST = reduce_const 420 | self.const_factor = const_factor 421 | self.num_classes = num_classes 422 | self.cuda = True 423 | 424 | self.I_KNOW_WHAT_I_AM_DOING_AND_WANT_TO_OVERRIDE_THE_PRESOFTMAX_CHECK = False 425 | 426 | def gradient_descent(self, model): 427 | def compare(x, y): 428 | if self.TARGETED: 429 | return x == y 430 | else: 431 | return x != y 432 | 433 | # TODO replace hardcode 434 | shape = (1, 3, 32, 32) 435 | 436 | def doit(oimgs, labs, starts_temp, tt, CONST): 437 | # convert to tanh space 438 | imgs = torch_arctanh(oimgs * 1.999999) 439 | starts = torch_arctanh(starts_temp * 1.999999) 440 | 441 | # initial tau 442 | tau = tt 443 | timg = imgs 444 | tlab = labs 445 | const = CONST 446 | 447 | # changing tlab to one hot 448 | target_onehot = torch.zeros( 449 | (1, self.num_classes), requires_grad=True, device="cuda" 450 | ) 451 | tlab = target_onehot.scatter(1, tlab.unsqueeze(1), 1.0) 452 | # iterate through constants, try to get highest one 453 | while CONST < self.LARGEST_CONST: 454 | # setup the modifier variable, 455 | # this is the variable we are optimizing over 456 | modifier = torch.zeros(shape, requires_grad=True, device="cuda") 457 | optimizer = optim.Adam([modifier], lr=self.LEARNING_RATE) 458 | 459 | # starting point for simg 460 | simg = starts.clone() 461 | 462 | for _ in range(self.MAX_ITERATIONS): 463 | newimg = tanh(modifier + simg) / 2 464 | 465 | output = model(newimg) 466 | orig_output = model(tanh(timg) / 2) # assumes -0.5..0.5 467 | 468 | real = torch.mean((tlab) * output) 469 | other = torch.max((1 - tlab) * output - (tlab * 10000)) 470 | 471 | if self.TARGETED: 472 | # if targetted, optimize for making the other class most likely 473 | loss1 = torch.max(other - real, torch.zeros_like(real)) 474 | else: 475 | # if untargeted, optimize for making this class least likely. 476 | loss1 = torch.max(real - other, torch.zeros_like(real)) 477 | 478 | loss2 = torch.sum( 479 | torch.max( 480 | torch.abs(newimg - tanh(timg) / 2) - tau, 481 | torch.zeros_like(newimg), 482 | ) 483 | ) 484 | loss = const * loss1 + loss2 485 | 486 | optimizer.zero_grad() 487 | loss.backward(retain_graph=True) 488 | optimizer.step() 489 | 490 | # it worked 491 | if loss < 0.0001 * CONST and self.ABORT_EARLY: 492 | works = compare(torch.argmax(output), torch.argmax(tlab)) 493 | if works: 494 | return output, orig_output, newimg, CONST 495 | 496 | C = CONST * self.const_factor 497 | CONST = C 498 | 499 | return doit 500 | 501 | def attack(self, model, imgs, targets): 502 | """ 503 | Perform the L_inf attack on the given images for the given targets. 504 | If self.targeted is true, then the targets represents the target labels. 505 | If self.targeted is false, then targets are the original class labels. 506 | """ 507 | r = [] 508 | i = 0 509 | for img, target in zip(imgs, targets): 510 | orig_lab = target.cpu().data.numpy() 511 | target_lab = random.choice([j for j in range(10) if j != orig_lab]) 512 | x = self.attack_single( 513 | model, img.unsqueeze(0), torch.tensor([target_lab]).cuda() 514 | ) 515 | print(target.data, torch.max(model(x), 1)[1]) 516 | r.extend(x) 517 | i += 1 518 | return r 519 | 520 | def attack_single(self, model, img, target): 521 | """ 522 | Run the attack on a single image and label 523 | """ 524 | prev = img.clone() 525 | tau = 1.0 526 | const = self.INITIAL_CONST 527 | 528 | while tau > 1.0 / 256: 529 | # try to solve given this tau value 530 | res = self.gradient_descent(model)( 531 | img.clone(), target, prev.clone(), tau, const 532 | ) 533 | if res is None: 534 | return prev 535 | 536 | scores, origscores, nimg, const = res 537 | 538 | if self.REDUCE_CONST: 539 | const = torch.div(const, 2) 540 | 541 | # the attack succeeded, reduce tau and try again 542 | actualtau = torch.max(torch.abs(nimg - img)) 543 | 544 | if actualtau < tau: 545 | tau = actualtau 546 | 547 | prev = nimg.clone() 548 | t = tau * self.DECREASE_FACTOR 549 | tau = t 550 | return prev 551 | 552 | 553 | # function to wrap/instantiate and call CW_linf 554 | def wrap_cw_linf(attack, params): 555 | """Perform cifar10 cw linf attack using model on "images" with "labels" 556 | 557 | Args: 558 | model: torch model with respect to which attacks will be computed 559 | images: batch of torch images 560 | labels: labels corresponding to the batch of images 561 | niters: number of iterations of cw to perform 562 | epsilon: Linf norm of resulting perturbation; scale of images is -1..1 563 | 564 | Returns: 565 | The batch of adversarial examples corresponding to the original images 566 | """ 567 | def wrap_f(model, images, labels, niters, use_Inc_model=False): 568 | def net_hacked(x): 569 | return model(x * 2) 570 | 571 | i2 = images.clone() / 2 # CW optimization happens on -0.5..0.5 range 572 | cw = CW_Linf(max_iterations=niters) 573 | res_unclip = torch.stack(cw.attack(net_hacked, i2, labels)) 574 | res = (res_unclip - i2).detach().clamp(-epsilon / 2, epsilon / 2) + i2 575 | r = res.clamp(-0.5, 0.5) 576 | return torch.mul(r, 2).cuda() # return normal -1..1 range adversarial example 577 | return wrap_f 578 | 579 | 580 | def wrap_attack(attack, params, dataset='cifar10'): 581 | """ 582 | A wrap for attack functions 583 | attack: an attack function 584 | params: kwargs for the attack func 585 | """ 586 | def wrap_f(model, images, labels, niters, use_Inc_model=False): 587 | # attack should process by batch 588 | return attack(model, images, labels, niters=niters, dataset=dataset, use_Inc_model=use_Inc_model, **params) 589 | 590 | return wrap_f 591 | 592 | 593 | def wrap_attack_imagenet(attack, params): 594 | return wrap_attack(attack, params, dataset='imagenet') 595 | 596 | 597 | # TAP (transferable adversairal perturbation ECCV 2018) 598 | class Transferable_Adversarial_Perturbations_Loss(torch.nn.Module): 599 | def __init__(self): 600 | super(Transferable_Adversarial_Perturbations_Loss, self).__init__() 601 | 602 | def forward( 603 | self, 604 | X, 605 | X_pert, 606 | original_mids, 607 | new_mids, 608 | y, 609 | output_perturbed, 610 | lam, 611 | alpha, 612 | s, 613 | yita, 614 | ): 615 | 616 | l1 = nn.CrossEntropyLoss()(output_perturbed, y) 617 | 618 | l2 = 0 619 | for i, new_mid in enumerate(new_mids): 620 | a = torch.sign(original_mids[i]) * torch.pow( 621 | torch.abs(original_mids[i]), alpha 622 | ) 623 | b = torch.sign(new_mid) * torch.pow(torch.abs(new_mid), alpha) 624 | l2 += lam * (a - b).norm() ** 2 625 | 626 | l3 = yita * torch.abs(nn.AvgPool2d(s)(X - X_pert)).sum() 627 | 628 | return l1 + l2 + l3 629 | 630 | 631 | mid_outputs = [] 632 | 633 | 634 | def Transferable_Adversarial_Perturbations( 635 | model, 636 | X, 637 | y, 638 | niters=10, 639 | epsilon=0.03, 640 | lam=0.005, 641 | alpha=0.5, 642 | s=3, 643 | yita=0.01, 644 | learning_rate=0.006, 645 | dataset="cifar10", 646 | use_Inc_model = False, 647 | ): 648 | """Perform cifar10 TAP attack using model on images X with labels y 649 | 650 | Args: 651 | model: torch model with respect to which attacks will be computed 652 | X: batch of torch images 653 | y: labels corresponding to the batch of images 654 | niters: number of iterations of TAP to perform 655 | epsilon: Linf norm of resulting perturbation; scale of images is -1..1 656 | lam: lambda parameter of TAP 657 | alpha: alpha parameter of TAP 658 | s: s parameter of TAP 659 | yita: yita parameter of TAP 660 | learning_rate: learning rate of TAP attack 661 | 662 | Returns: 663 | The batch of adversarial examples corresponding to the original images 664 | """ 665 | feature_layers = list(model._modules.keys()) 666 | global mid_outputs 667 | X = X.detach() 668 | X_pert = torch.zeros(X.size()).cuda() 669 | X_pert.copy_(X).detach() 670 | X_pert.requires_grad = True 671 | 672 | def get_mid_output(m, i, o): 673 | global mid_outputs 674 | mid_outputs.append(o) 675 | 676 | hs = [] 677 | for layer_name in feature_layers: 678 | hs.append(model._modules.get(layer_name).register_forward_hook(get_mid_output)) 679 | 680 | out = model(X) 681 | 682 | mid_originals = [] 683 | for mid_output in mid_outputs: 684 | mid_original = torch.zeros(mid_output.size()).cuda() 685 | mid_originals.append(mid_original.copy_(mid_output)) 686 | 687 | mid_outputs = [] 688 | 689 | for _ in range(niters): 690 | output_perturbed = model(X_pert) 691 | # generate adversarial example by max middle 692 | # layer pertubation in the direction of increasing loss 693 | mid_originals_ = [] 694 | for mid_original in mid_originals: 695 | mid_originals_.append(mid_original.detach()) 696 | 697 | loss = Transferable_Adversarial_Perturbations_Loss()( 698 | X, 699 | X_pert, 700 | mid_originals_, 701 | mid_outputs, 702 | y, 703 | output_perturbed, 704 | lam, 705 | alpha, 706 | s, 707 | yita, 708 | ) 709 | loss.backward() 710 | pert = learning_rate * X_pert.grad.detach().sign() 711 | 712 | # minimize loss 713 | X_pert = X_pert.detach() + pert 714 | X_pert.requires_grad = True 715 | 716 | # make sure we don't modify the original image beyond epsilon 717 | X_pert = renormalization(X, X_pert, epsilon, dataset=dataset, use_Inc_model=use_Inc_model) 718 | X_pert.requires_grad = True 719 | 720 | mid_outputs = [] 721 | 722 | for h in hs: 723 | h.remove() 724 | return X_pert 725 | 726 | 727 | # ILA attack 728 | 729 | # square sum of dot product 730 | class Proj_Loss(torch.nn.Module): 731 | def __init__(self): 732 | super(Proj_Loss, self).__init__() 733 | 734 | def forward(self, old_attack_mid, new_mid, original_mid, coeff): 735 | x = (old_attack_mid - original_mid).view(1, -1) 736 | y = (new_mid - original_mid).view(1, -1) 737 | x_norm = x / x.norm() 738 | 739 | proj_loss = torch.mm(y, x_norm.transpose(0, 1)) / x.norm() 740 | return proj_loss 741 | 742 | 743 | # square sum of dot product 744 | class Mid_layer_target_Loss(torch.nn.Module): 745 | def __init__(self): 746 | super(Mid_layer_target_Loss, self).__init__() 747 | 748 | def forward(self, old_attack_mid, new_mid, original_mid, coeff): 749 | x = (old_attack_mid - original_mid).view(1, -1) 750 | y = (new_mid - original_mid).view(1, -1) 751 | 752 | x_norm = x / x.norm() 753 | if (y == 0).all(): 754 | y_norm = y 755 | else: 756 | y_norm = y / y.norm() 757 | angle_loss = torch.mm(x_norm, y_norm.transpose(0, 1)) 758 | magnitude_gain = y.norm() / x.norm() 759 | return angle_loss + magnitude_gain * coeff 760 | 761 | 762 | """Return: perturbed x""" 763 | mid_output = None 764 | 765 | 766 | def ILA( 767 | model, 768 | X, 769 | X_attack, 770 | y, 771 | feature_layer, 772 | niters=10, 773 | epsilon=0.01, 774 | coeff=1.0, 775 | learning_rate=1, 776 | dataset="cifar10", 777 | use_Inc_model = False, 778 | with_projection=True, 779 | ): 780 | """Perform ILA attack with respect to model on images X with labels y 781 | 782 | Args: 783 | with_projection: boolean, specifies whether projection should happen 784 | in the attack 785 | model: torch model with respect to which attacks will be computed 786 | X: batch of torch images 787 | X_attack: starting adversarial examples of ILA that will be modified 788 | to become more transferable 789 | y: labels corresponding to the batch of images 790 | feature_layer: layer of model to project on in ILA attack 791 | niters: number of iterations of the attack to perform 792 | epsilon: Linf norm of resulting perturbation; scale of images is -1..1 793 | coeff: coefficient of magnitude loss in ILA attack 794 | visualize: whether you want to visualize the perturbations or not 795 | learning_rate: learning rate of the attack 796 | dataset: dataset the images are from, 'cifar10' | 'imagenet' 797 | 798 | Returns: 799 | The batch of modified adversarial examples, examples have been 800 | augmented from X_attack to become more transferable 801 | """ 802 | X = X.detach() 803 | X_pert = torch.zeros(X.size()).cuda() 804 | X_pert.copy_(X).detach() 805 | X_pert.requires_grad = True 806 | 807 | def get_mid_output(m, i, o): 808 | global mid_output 809 | mid_output = o 810 | 811 | h = feature_layer.register_forward_hook(get_mid_output) 812 | 813 | out = model(X) 814 | mid_original = torch.zeros(mid_output.size()).cuda() 815 | mid_original.copy_(mid_output) 816 | 817 | out = model(X_attack) 818 | mid_attack_original = torch.zeros(mid_output.size()).cuda() 819 | mid_attack_original.copy_(mid_output) 820 | 821 | for _ in range(niters): 822 | output_perturbed = model(X_pert) 823 | 824 | # generate adversarial example by max middle layer pertubation 825 | # in the direction of increasing loss 826 | if with_projection: 827 | loss = Proj_Loss()( 828 | mid_attack_original.detach(), mid_output, mid_original.detach(), coeff 829 | ) 830 | else: 831 | loss = Mid_layer_target_Loss()( 832 | mid_attack_original.detach(), mid_output, mid_original.detach(), coeff 833 | ) 834 | 835 | loss.backward() 836 | pert = learning_rate * X_pert.grad.detach().sign() 837 | 838 | # minimize loss 839 | X_pert = X_pert.detach() + pert 840 | X_pert.requires_grad = True 841 | 842 | # make sure we don't modify the original image beyond epsilon 843 | X_pert = renormalization(X, X_pert, epsilon, dataset=dataset, use_Inc_model=use_Inc_model) 844 | X_pert.requires_grad = True 845 | 846 | h.remove() 847 | return X_pert 848 | 849 | 850 | def input_diversity(X, p, image_width, image_resize): 851 | rnd = torch.randint(image_width, image_resize, ()) 852 | rescaled = nn.functional.interpolate(X, [rnd, rnd]) 853 | h_rem = image_resize - rnd 854 | w_rem = image_resize - rnd 855 | pad_top = torch.randint(0, h_rem, ()) 856 | pad_bottom = h_rem - pad_top 857 | pad_left = torch.randint(0, w_rem,()) 858 | pad_right = w_rem - pad_left 859 | padded = nn.ConstantPad2d((pad_left, pad_right, pad_top, pad_bottom), 0.)(rescaled) 860 | padded = nn.functional.interpolate(padded, [image_width, image_width]) 861 | return padded if torch.rand(()) < p else X 862 | 863 | 864 | def DI_2_fgsm( 865 | model, 866 | X, 867 | y, 868 | niters=10, 869 | epsilon=0.01, 870 | p = 0.5, 871 | image_width=32, 872 | image_resize=36, 873 | visualize=False, 874 | learning_rate=0.005, 875 | dataset="cifar10", 876 | use_Inc_model = False, 877 | ): 878 | X_pert = X.clone() 879 | X_pert.requires_grad = True 880 | 881 | for _ in range(niters): 882 | output_perturbed = model(input_diversity(X_pert, p, image_width, image_resize)) 883 | loss = nn.CrossEntropyLoss()(output_perturbed, y) 884 | loss.backward() 885 | pert = learning_rate * X_pert.grad.detach().sign() 886 | # add perturbation 887 | X_pert = X_pert.detach() + pert 888 | X_pert.requires_grad = True 889 | 890 | # make sure we don't modify the original image beyond epsilon 891 | X_pert = renormalization(X, X_pert, epsilon, dataset=dataset, use_Inc_model=use_Inc_model) 892 | X_pert.requires_grad = True 893 | 894 | return X_pert 895 | 896 | TMP_PATH = '/home/qh53/Intermediate-Level-Attack-test' 897 | 898 | 899 | def DI_2_fgsm_tf(model, batch_i, y, niters=10, epsilon=0.01, p = 0.5, image_width=299, image_resize=330, learning_rate=0.005, dataset="cifar10", 900 | use_Inc_model = False,): 901 | 902 | images = [] 903 | batch_size = 8 904 | for i in range(batch_size): 905 | image = imread(TMP_PATH + '/tmp_{}_iter={}_momentum=0/{}_{}.png'.format(model, niters, batch_i, i), mode='RGB') 906 | images.append(image) 907 | 908 | return torch.tensor(images).permute(0, 3, 1, 2).cuda().float() / 255.0 * 2.0 - 1.0 909 | -------------------------------------------------------------------------------- /cifar10_config.py: -------------------------------------------------------------------------------- 1 | from attacks import wrap_attack, wrap_cw_linf, ifgsm, momentum_ifgsm, fgsm, deepfool, CW_Linf, Transferable_Adversarial_Perturbations, ILA 2 | from cifar10models import * 3 | 4 | data_preprocess = ([0.5, 0.5, 0.5], [0.5, 0.5, 0.5]) 5 | 6 | def model_name(model): 7 | return model.__name__ 8 | 9 | model_configs = { 10 | model_name(ResNet18): (ResNet18, '/share/cuvl/weights/cifar10/resnet18_epoch_347_acc_94.77.pth'), 11 | model_name(DenseNet121): (DenseNet121,'/share/cuvl/weights/cifar10/densenet121_epoch_315_acc_95.61.pth'), 12 | model_name(GoogLeNet): (GoogLeNet, '/share/cuvl/weights/cifar10/googlenet_epoch_227_acc_94.86.pth'), 13 | model_name(SENet18): (SENet18, '/share/cuvl/weights/cifar10/senet18_epoch_279_acc_94.59.pth') 14 | } 15 | 16 | 17 | def attack_name(attack): 18 | return attack.__name__ 19 | 20 | attack_configs ={ 21 | attack_name(ifgsm): wrap_attack(ifgsm, {'learning_rate': 0.002, 'epsilon': 0.03}), 22 | attack_name(momentum_ifgsm): wrap_attack(momentum_ifgsm, {'learning_rate': 0.002, 'epsilon': 0.03, 'decay': 0.9}), 23 | attack_name(fgsm): wrap_attack(fgsm, {}), 24 | attack_name(deepfool): wrap_attack(deepfool, {}), 25 | attack_name(CW_Linf): wrap_cw_linf(CW_Linf, {'targeted': False} ), 26 | attack_name(Transferable_Adversarial_Perturbations): wrap_attack(Transferable_Adversarial_Perturbations, {'learning_rate': 0.002, 'epsilon': 0.03, 'lam' : 0.005, 'alpha' : 0.5, 's' : 3, 'yita' : 0.01} ), 27 | } 28 | 29 | 30 | use_projection = True 31 | 32 | ILA_params = { 33 | attack_name(ifgsm): {'niters': 10, 'learning_rate': 0.006, 'epsilon':0.03, 'coeff': 5.0}, 34 | attack_name(momentum_ifgsm): {'niters': 10, 'learning_rate': 0.006, 'epsilon':0.03, 'coeff': 5.0}, 35 | attack_name(fgsm): {'niters': 10, 'learning_rate': 0.006, 'epsilon':0.03, 'coeff': 5.0}, 36 | attack_name(deepfool): {'niters': 10, 'learning_rate': 1.0, 'epsilon':0.03, 'coeff': 5.0}, 37 | attack_name(CW_Linf): {'niters': 10, 'learning_rate': 0.006, 'epsilon':0.03, 'coeff': 5.0}, 38 | attack_name(Transferable_Adversarial_Perturbations): {'niters': 10, 'learning_rate': 0.006, 'epsilon':0.03, 'coeff': 5.0}, 39 | } 40 | 41 | source_layers = { 42 | model_name(ResNet18): list(enumerate(ResNet18()._modules.keys())), 43 | model_name(DenseNet121): list(enumerate(DenseNet121()._modules.keys())), 44 | model_name(GoogLeNet): list(enumerate(GoogLeNet()._modules.keys())), 45 | model_name(SENet18):list(enumerate(SENet18()._modules.keys())), 46 | } 47 | 48 | 49 | 50 | 51 | -------------------------------------------------------------------------------- /cifar10models.py: -------------------------------------------------------------------------------- 1 | import torch 2 | import torchvision 3 | import torchvision.transforms as transforms 4 | import torchvision.models as models 5 | import torch.nn as nn 6 | import torch.nn.functional as F 7 | import torch.optim as optim 8 | import math 9 | 10 | # Models you can import from this file: ResNet18, VGG19, DenseNet121, GoogLeNet 11 | # View the file for some of the models you can import 12 | # These models are from https://github.com/kuangliu/pytorch-cifar 13 | 14 | ''' ResNet ''' 15 | class BasicBlock(nn.Module): 16 | expansion = 1 17 | def __init__(self, in_planes, planes, stride=1): 18 | super(BasicBlock, self).__init__() 19 | self.conv1 = nn.Conv2d(in_planes, planes, kernel_size=3, stride=stride, padding=1, bias=False) 20 | self.bn1 = nn.BatchNorm2d(planes) 21 | self.conv2 = nn.Conv2d(planes, planes, kernel_size=3, stride=1, padding=1, bias=False) 22 | self.bn2 = nn.BatchNorm2d(planes) 23 | 24 | self.shortcut = nn.Sequential() 25 | if stride != 1 or in_planes != self.expansion*planes: 26 | self.shortcut = nn.Sequential( 27 | nn.Conv2d(in_planes, self.expansion*planes, kernel_size=1, stride=stride, bias=False), 28 | nn.BatchNorm2d(self.expansion*planes) 29 | ) 30 | 31 | def forward(self, x): 32 | out = F.relu(self.bn1(self.conv1(x))) 33 | out = self.bn2(self.conv2(out)) 34 | out += self.shortcut(x) 35 | out = F.relu(out) 36 | return out 37 | 38 | 39 | class Bottleneck(nn.Module): 40 | expansion = 4 41 | 42 | def __init__(self, in_planes, planes, stride=1): 43 | super(Bottleneck, self).__init__() 44 | self.conv1 = nn.Conv2d(in_planes, planes, kernel_size=1, bias=False) 45 | self.bn1 = nn.BatchNorm2d(planes) 46 | self.conv2 = nn.Conv2d(planes, planes, kernel_size=3, stride=stride, padding=1, bias=False) 47 | self.bn2 = nn.BatchNorm2d(planes) 48 | self.conv3 = nn.Conv2d(planes, self.expansion*planes, kernel_size=1, bias=False) 49 | self.bn3 = nn.BatchNorm2d(self.expansion*planes) 50 | 51 | self.shortcut = nn.Sequential() 52 | if stride != 1 or in_planes != self.expansion*planes: 53 | self.shortcut = nn.Sequential( 54 | nn.Conv2d(in_planes, self.expansion*planes, kernel_size=1, stride=stride, bias=False), 55 | nn.BatchNorm2d(self.expansion*planes) 56 | ) 57 | 58 | def forward(self, x): 59 | out = F.relu(self.bn1(self.conv1(x))) 60 | out = F.relu(self.bn2(self.conv2(out))) 61 | out = self.bn3(self.conv3(out)) 62 | out += self.shortcut(x) 63 | out = F.relu(out) 64 | return out 65 | 66 | 67 | class ResNet(nn.Module): 68 | def __init__(self, block, num_blocks, num_classes=10): 69 | super(ResNet, self).__init__() 70 | self.in_planes = 64 71 | 72 | self.conv1 = nn.Conv2d(3, 64, kernel_size=3, stride=1, padding=1, bias=False) 73 | self.bn1 = nn.BatchNorm2d(64) 74 | self.layer1 = self._make_layer(block, 64, num_blocks[0], stride=1) 75 | self.layer2 = self._make_layer(block, 128, num_blocks[1], stride=2) 76 | self.layer3 = self._make_layer(block, 256, num_blocks[2], stride=2) 77 | self.layer4 = self._make_layer(block, 512, num_blocks[3], stride=2) 78 | self.linear = nn.Linear(512*block.expansion, num_classes) 79 | 80 | def _make_layer(self, block, planes, num_blocks, stride): 81 | strides = [stride] + [1]*(num_blocks-1) 82 | layers = [] 83 | for stride in strides: 84 | layers.append(block(self.in_planes, planes, stride)) 85 | self.in_planes = planes * block.expansion 86 | return nn.Sequential(*layers) 87 | 88 | def forward(self, x): 89 | out = F.relu(self.bn1(self.conv1(x))) 90 | out = self.layer1(out) 91 | out = self.layer2(out) 92 | out = self.layer3(out) 93 | out = self.layer4(out) 94 | out = F.avg_pool2d(out, 4) 95 | out = out.view(out.size(0), -1) 96 | out = self.linear(out) 97 | return out 98 | 99 | 100 | def ResNet18(): 101 | return ResNet(BasicBlock, [2,2,2,2]) 102 | 103 | def ResNet34(): 104 | return ResNet(BasicBlock, [3,4,6,3]) 105 | 106 | def ResNet50(): 107 | return ResNet(Bottleneck, [3,4,6,3]) 108 | 109 | def ResNet101(): 110 | return ResNet(Bottleneck, [3,4,23,3]) 111 | 112 | def ResNet152(): 113 | return ResNet(Bottleneck, [3,8,36,3]) 114 | 115 | '''PreAct ResNet''' 116 | class PreActBlock(nn.Module): 117 | '''Pre-activation version of the BasicBlock.''' 118 | expansion = 1 119 | 120 | def __init__(self, in_planes, planes, stride=1): 121 | super(PreActBlock, self).__init__() 122 | self.bn1 = nn.BatchNorm2d(in_planes) 123 | self.conv1 = nn.Conv2d(in_planes, planes, kernel_size=3, stride=stride, padding=1, bias=False) 124 | self.bn2 = nn.BatchNorm2d(planes) 125 | self.conv2 = nn.Conv2d(planes, planes, kernel_size=3, stride=1, padding=1, bias=False) 126 | 127 | if stride != 1 or in_planes != self.expansion*planes: 128 | self.shortcut = nn.Sequential( 129 | nn.Conv2d(in_planes, self.expansion*planes, kernel_size=1, stride=stride, bias=False) 130 | ) 131 | 132 | def forward(self, x): 133 | out = F.relu(self.bn1(x)) 134 | shortcut = self.shortcut(out) if hasattr(self, 'shortcut') else x 135 | out = self.conv1(out) 136 | out = self.conv2(F.relu(self.bn2(out))) 137 | out += shortcut 138 | return out 139 | 140 | 141 | class PreActBottleneck(nn.Module): 142 | '''Pre-activation version of the original Bottleneck module.''' 143 | expansion = 4 144 | 145 | def __init__(self, in_planes, planes, stride=1): 146 | super(PreActBottleneck, self).__init__() 147 | self.bn1 = nn.BatchNorm2d(in_planes) 148 | self.conv1 = nn.Conv2d(in_planes, planes, kernel_size=1, bias=False) 149 | self.bn2 = nn.BatchNorm2d(planes) 150 | self.conv2 = nn.Conv2d(planes, planes, kernel_size=3, stride=stride, padding=1, bias=False) 151 | self.bn3 = nn.BatchNorm2d(planes) 152 | self.conv3 = nn.Conv2d(planes, self.expansion*planes, kernel_size=1, bias=False) 153 | 154 | if stride != 1 or in_planes != self.expansion*planes: 155 | self.shortcut = nn.Sequential( 156 | nn.Conv2d(in_planes, self.expansion*planes, kernel_size=1, stride=stride, bias=False) 157 | ) 158 | 159 | def forward(self, x): 160 | out = F.relu(self.bn1(x)) 161 | shortcut = self.shortcut(out) if hasattr(self, 'shortcut') else x 162 | out = self.conv1(out) 163 | out = self.conv2(F.relu(self.bn2(out))) 164 | out = self.conv3(F.relu(self.bn3(out))) 165 | out += shortcut 166 | return out 167 | 168 | 169 | class PreActResNet(nn.Module): 170 | def __init__(self, block, num_blocks, num_classes=10): 171 | super(PreActResNet, self).__init__() 172 | self.in_planes = 64 173 | 174 | self.conv1 = nn.Conv2d(3, 64, kernel_size=3, stride=1, padding=1, bias=False) 175 | self.layer1 = self._make_layer(block, 64, num_blocks[0], stride=1) 176 | self.layer2 = self._make_layer(block, 128, num_blocks[1], stride=2) 177 | self.layer3 = self._make_layer(block, 256, num_blocks[2], stride=2) 178 | self.layer4 = self._make_layer(block, 512, num_blocks[3], stride=2) 179 | self.linear = nn.Linear(512*block.expansion, num_classes) 180 | 181 | def _make_layer(self, block, planes, num_blocks, stride): 182 | strides = [stride] + [1]*(num_blocks-1) 183 | layers = [] 184 | for stride in strides: 185 | layers.append(block(self.in_planes, planes, stride)) 186 | self.in_planes = planes * block.expansion 187 | return nn.Sequential(*layers) 188 | 189 | def forward(self, x): 190 | out = self.conv1(x) 191 | out = self.layer1(out) 192 | out = self.layer2(out) 193 | out = self.layer3(out) 194 | out = self.layer4(out) 195 | out = F.avg_pool2d(out, 4) 196 | out = out.view(out.size(0), -1) 197 | out = self.linear(out) 198 | return out 199 | 200 | 201 | def PreActResNet18(): 202 | return PreActResNet(PreActBlock, [2,2,2,2]) 203 | 204 | def PreActResNet34(): 205 | return PreActResNet(PreActBlock, [3,4,6,3]) 206 | 207 | def PreActResNet50(): 208 | return PreActResNet(PreActBottleneck, [3,4,6,3]) 209 | 210 | def PreActResNet101(): 211 | return PreActResNet(PreActBottleneck, [3,4,23,3]) 212 | 213 | def PreActResNet152(): 214 | return PreActResNet(PreActBottleneck, [3,8,36,3]) 215 | 216 | 217 | ''' SENet ''' 218 | class SEBasicBlock(nn.Module): 219 | def __init__(self, in_planes, planes, stride=1): 220 | super(SEBasicBlock, self).__init__() 221 | self.conv1 = nn.Conv2d(in_planes, planes, kernel_size=3, stride=stride, padding=1, bias=False) 222 | self.bn1 = nn.BatchNorm2d(planes) 223 | self.conv2 = nn.Conv2d(planes, planes, kernel_size=3, stride=1, padding=1, bias=False) 224 | self.bn2 = nn.BatchNorm2d(planes) 225 | 226 | self.shortcut = nn.Sequential() 227 | if stride != 1 or in_planes != planes: 228 | self.shortcut = nn.Sequential( 229 | nn.Conv2d(in_planes, planes, kernel_size=1, stride=stride, bias=False), 230 | nn.BatchNorm2d(planes) 231 | ) 232 | 233 | # SE layers 234 | self.fc1 = nn.Conv2d(planes, planes//16, kernel_size=1) # Use nn.Conv2d instead of nn.Linear 235 | self.fc2 = nn.Conv2d(planes//16, planes, kernel_size=1) 236 | 237 | def forward(self, x): 238 | out = F.relu(self.bn1(self.conv1(x))) 239 | out = self.bn2(self.conv2(out)) 240 | 241 | # Squeeze 242 | w = F.avg_pool2d(out, out.size(2)) 243 | w = F.relu(self.fc1(w)) 244 | w = F.sigmoid(self.fc2(w)) 245 | # Excitation 246 | out = out * w # New broadcasting feature from v0.2! 247 | 248 | out += self.shortcut(x) 249 | out = F.relu(out) 250 | return out 251 | 252 | class SE_PreActBlock(nn.Module): 253 | def __init__(self, in_planes, planes, stride=1): 254 | super(SE_PreActBlock, self).__init__() 255 | self.bn1 = nn.BatchNorm2d(in_planes) 256 | self.conv1 = nn.Conv2d(in_planes, planes, kernel_size=3, stride=stride, padding=1, bias=False) 257 | self.bn2 = nn.BatchNorm2d(planes) 258 | self.conv2 = nn.Conv2d(planes, planes, kernel_size=3, stride=1, padding=1, bias=False) 259 | 260 | if stride != 1 or in_planes != planes: 261 | self.shortcut = nn.Sequential( 262 | nn.Conv2d(in_planes, planes, kernel_size=1, stride=stride, bias=False) 263 | ) 264 | 265 | # SE layers 266 | self.fc1 = nn.Conv2d(planes, planes//16, kernel_size=1) 267 | self.fc2 = nn.Conv2d(planes//16, planes, kernel_size=1) 268 | 269 | def forward(self, x): 270 | out = F.relu(self.bn1(x)) 271 | shortcut = self.shortcut(out) if hasattr(self, 'shortcut') else x 272 | out = self.conv1(out) 273 | out = self.conv2(F.relu(self.bn2(out))) 274 | 275 | # Squeeze 276 | w = F.avg_pool2d(out, out.size(2)) 277 | w = F.relu(self.fc1(w)) 278 | w = F.sigmoid(self.fc2(w)) 279 | # Excitation 280 | out = out * w 281 | 282 | out += shortcut 283 | return out 284 | 285 | class SENet(nn.Module): 286 | def __init__(self, block, num_blocks, num_classes=10): 287 | super(SENet, self).__init__() 288 | self.in_planes = 64 289 | 290 | self.conv1 = nn.Conv2d(3, 64, kernel_size=3, stride=1, padding=1, bias=False) 291 | self.bn1 = nn.BatchNorm2d(64) 292 | self.layer1 = self._make_layer(block, 64, num_blocks[0], stride=1) 293 | self.layer2 = self._make_layer(block, 128, num_blocks[1], stride=2) 294 | self.layer3 = self._make_layer(block, 256, num_blocks[2], stride=2) 295 | self.layer4 = self._make_layer(block, 512, num_blocks[3], stride=2) 296 | self.linear = nn.Linear(512, num_classes) 297 | 298 | def _make_layer(self, block, planes, num_blocks, stride): 299 | strides = [stride] + [1]*(num_blocks-1) 300 | layers = [] 301 | for stride in strides: 302 | layers.append(block(self.in_planes, planes, stride)) 303 | self.in_planes = planes 304 | return nn.Sequential(*layers) 305 | 306 | def forward(self, x): 307 | out = F.relu(self.bn1(self.conv1(x))) 308 | out = self.layer1(out) 309 | out = self.layer2(out) 310 | out = self.layer3(out) 311 | out = self.layer4(out) 312 | out = F.avg_pool2d(out, 4) 313 | out = out.view(out.size(0), -1) 314 | out = self.linear(out) 315 | return out 316 | 317 | 318 | def SENet18(): 319 | return SENet(SE_PreActBlock, [2,2,2,2]) 320 | 321 | 322 | ''' DenseNet ''' 323 | class BottleneckDense(nn.Module): 324 | def __init__(self, in_planes, growth_rate): 325 | super(BottleneckDense, self).__init__() 326 | self.bn1 = nn.BatchNorm2d(in_planes) 327 | self.conv1 = nn.Conv2d(in_planes, 4*growth_rate, kernel_size=1, bias=False) 328 | self.bn2 = nn.BatchNorm2d(4*growth_rate) 329 | self.conv2 = nn.Conv2d(4*growth_rate, growth_rate, kernel_size=3, padding=1, bias=False) 330 | 331 | def forward(self, x): 332 | out = self.conv1(F.relu(self.bn1(x))) 333 | out = self.conv2(F.relu(self.bn2(out))) 334 | out = torch.cat([out,x], 1) 335 | return out 336 | 337 | 338 | class Transition(nn.Module): 339 | def __init__(self, in_planes, out_planes): 340 | super(Transition, self).__init__() 341 | self.bn = nn.BatchNorm2d(in_planes) 342 | self.conv = nn.Conv2d(in_planes, out_planes, kernel_size=1, bias=False) 343 | 344 | def forward(self, x): 345 | out = self.conv(F.relu(self.bn(x))) 346 | out = F.avg_pool2d(out, 2) 347 | return out 348 | 349 | 350 | class DenseNet(nn.Module): 351 | def __init__(self, block, nblocks, growth_rate=12, reduction=0.5, num_classes=10): 352 | super(DenseNet, self).__init__() 353 | self.growth_rate = growth_rate 354 | 355 | num_planes = 2*growth_rate 356 | self.conv1 = nn.Conv2d(3, num_planes, kernel_size=3, padding=1, bias=False) 357 | 358 | self.dense1 = self._make_dense_layers(block, num_planes, nblocks[0]) 359 | num_planes += nblocks[0]*growth_rate 360 | out_planes = int(math.floor(num_planes*reduction)) 361 | self.trans1 = Transition(num_planes, out_planes) 362 | num_planes = out_planes 363 | 364 | self.dense2 = self._make_dense_layers(block, num_planes, nblocks[1]) 365 | num_planes += nblocks[1]*growth_rate 366 | out_planes = int(math.floor(num_planes*reduction)) 367 | self.trans2 = Transition(num_planes, out_planes) 368 | num_planes = out_planes 369 | 370 | self.dense3 = self._make_dense_layers(block, num_planes, nblocks[2]) 371 | num_planes += nblocks[2]*growth_rate 372 | out_planes = int(math.floor(num_planes*reduction)) 373 | self.trans3 = Transition(num_planes, out_planes) 374 | num_planes = out_planes 375 | 376 | self.dense4 = self._make_dense_layers(block, num_planes, nblocks[3]) 377 | num_planes += nblocks[3]*growth_rate 378 | 379 | self.bn = nn.BatchNorm2d(num_planes) 380 | self.linear = nn.Linear(num_planes, num_classes) 381 | 382 | def _make_dense_layers(self, block, in_planes, nblock): 383 | layers = [] 384 | for i in range(nblock): 385 | layers.append(block(in_planes, self.growth_rate)) 386 | in_planes += self.growth_rate 387 | return nn.Sequential(*layers) 388 | 389 | def forward(self, x): 390 | out = self.conv1(x) 391 | out = self.trans1(self.dense1(out)) 392 | out = self.trans2(self.dense2(out)) 393 | out = self.trans3(self.dense3(out)) 394 | out = self.dense4(out) 395 | out = F.avg_pool2d(F.relu(self.bn(out)), 4) 396 | out = out.view(out.size(0), -1) 397 | out = self.linear(out) 398 | return out 399 | 400 | def DenseNet121(): 401 | return DenseNet(BottleneckDense, [6,12,24,16], growth_rate=32) 402 | 403 | def DenseNet169(): 404 | return DenseNet(BottleneckDense, [6,12,32,32], growth_rate=32) 405 | 406 | def DenseNet201(): 407 | return DenseNet(BottleneckDense, [6,12,48,32], growth_rate=32) 408 | 409 | def DenseNet161(): 410 | return DenseNet(BottleneckDense, [6,12,36,24], growth_rate=48) 411 | 412 | def densenet_cifar(): 413 | return DenseNet(BottleneckDense, [6,12,24,16], growth_rate=12) 414 | 415 | ''' GoogLeNet ''' 416 | class Inception(nn.Module): 417 | def __init__(self, in_planes, n1x1, n3x3red, n3x3, n5x5red, n5x5, pool_planes): 418 | super(Inception, self).__init__() 419 | # 1x1 conv branch 420 | self.b1 = nn.Sequential( 421 | nn.Conv2d(in_planes, n1x1, kernel_size=1), 422 | nn.BatchNorm2d(n1x1), 423 | nn.ReLU(True), 424 | ) 425 | 426 | # 1x1 conv -> 3x3 conv branch 427 | self.b2 = nn.Sequential( 428 | nn.Conv2d(in_planes, n3x3red, kernel_size=1), 429 | nn.BatchNorm2d(n3x3red), 430 | nn.ReLU(True), 431 | nn.Conv2d(n3x3red, n3x3, kernel_size=3, padding=1), 432 | nn.BatchNorm2d(n3x3), 433 | nn.ReLU(True), 434 | ) 435 | 436 | # 1x1 conv -> 5x5 conv branch 437 | self.b3 = nn.Sequential( 438 | nn.Conv2d(in_planes, n5x5red, kernel_size=1), 439 | nn.BatchNorm2d(n5x5red), 440 | nn.ReLU(True), 441 | nn.Conv2d(n5x5red, n5x5, kernel_size=3, padding=1), 442 | nn.BatchNorm2d(n5x5), 443 | nn.ReLU(True), 444 | nn.Conv2d(n5x5, n5x5, kernel_size=3, padding=1), 445 | nn.BatchNorm2d(n5x5), 446 | nn.ReLU(True), 447 | ) 448 | 449 | # 3x3 pool -> 1x1 conv branch 450 | self.b4 = nn.Sequential( 451 | nn.MaxPool2d(3, stride=1, padding=1), 452 | nn.Conv2d(in_planes, pool_planes, kernel_size=1), 453 | nn.BatchNorm2d(pool_planes), 454 | nn.ReLU(True), 455 | ) 456 | 457 | def forward(self, x): 458 | y1 = self.b1(x) 459 | y2 = self.b2(x) 460 | y3 = self.b3(x) 461 | y4 = self.b4(x) 462 | return torch.cat([y1,y2,y3,y4], 1) 463 | 464 | 465 | class GoogLeNet(nn.Module): 466 | def __init__(self): 467 | super(GoogLeNet, self).__init__() 468 | self.pre_layers = nn.Sequential( 469 | nn.Conv2d(3, 192, kernel_size=3, padding=1), 470 | nn.BatchNorm2d(192), 471 | nn.ReLU(True), 472 | ) 473 | 474 | self.a3 = Inception(192, 64, 96, 128, 16, 32, 32) 475 | self.b3 = Inception(256, 128, 128, 192, 32, 96, 64) 476 | 477 | self.maxpool = nn.MaxPool2d(3, stride=2, padding=1) 478 | 479 | self.a4 = Inception(480, 192, 96, 208, 16, 48, 64) 480 | self.b4 = Inception(512, 160, 112, 224, 24, 64, 64) 481 | self.c4 = Inception(512, 128, 128, 256, 24, 64, 64) 482 | self.d4 = Inception(512, 112, 144, 288, 32, 64, 64) 483 | self.e4 = Inception(528, 256, 160, 320, 32, 128, 128) 484 | 485 | self.a5 = Inception(832, 256, 160, 320, 32, 128, 128) 486 | self.b5 = Inception(832, 384, 192, 384, 48, 128, 128) 487 | 488 | self.avgpool = nn.AvgPool2d(8, stride=1) 489 | self.linear = nn.Linear(1024, 10) 490 | 491 | def forward(self, x): 492 | out = self.pre_layers(x) 493 | out = self.a3(out) 494 | out = self.b3(out) 495 | out = self.maxpool(out) 496 | out = self.a4(out) 497 | out = self.b4(out) 498 | out = self.c4(out) 499 | out = self.d4(out) 500 | out = self.e4(out) 501 | out = self.maxpool(out) 502 | out = self.a5(out) 503 | out = self.b5(out) 504 | out = self.avgpool(out) 505 | out = out.view(out.size(0), -1) 506 | out = self.linear(out) 507 | return out 508 | 509 | '''VGG''' 510 | cfg = { 511 | 'VGG11': [64, 'M', 128, 'M', 256, 256, 'M', 512, 512, 'M', 512, 512, 'M'], 512 | 'VGG13': [64, 64, 'M', 128, 128, 'M', 256, 256, 'M', 512, 512, 'M', 512, 512, 'M'], 513 | 'VGG16': [64, 64, 'M', 128, 128, 'M', 256, 256, 256, 'M', 512, 512, 512, 'M', 512, 512, 512, 'M'], 514 | 'VGG19': [64, 64, 'M', 128, 128, 'M', 256, 256, 256, 256, 'M', 512, 512, 512, 512, 'M', 512, 512, 512, 512, 'M'], 515 | } 516 | 517 | class VGG(nn.Module): 518 | def __init__(self, vgg_name): 519 | super(VGG, self).__init__() 520 | self.features = self._make_layers(cfg[vgg_name]) 521 | self.classifier = nn.Linear(512, 10) 522 | 523 | def forward(self, x): 524 | out = self.features(x) 525 | out = out.view(out.size(0), -1) 526 | out = self.classifier(out) 527 | return out 528 | 529 | def _make_layers(self, cfg): 530 | layers = [] 531 | in_channels = 3 532 | for x in cfg: 533 | if x == 'M': 534 | layers += [nn.MaxPool2d(kernel_size=2, stride=2)] 535 | else: 536 | layers += [nn.Conv2d(in_channels, x, kernel_size=3, padding=1), 537 | nn.BatchNorm2d(x), 538 | nn.ReLU(inplace=True)] 539 | in_channels = x 540 | layers += [nn.AvgPool2d(kernel_size=1, stride=1)] 541 | return nn.Sequential(*layers) 542 | -------------------------------------------------------------------------------- /demo.py: -------------------------------------------------------------------------------- 1 | import torch 2 | import torchvision 3 | import torchvision.transforms as transforms 4 | import numpy as np 5 | from PIL import Image 6 | 7 | import argparse 8 | 9 | from all_in_one_imagenet import get_source_layers 10 | from attacks import ifgsm, ILA 11 | 12 | # Training settings 13 | parser = argparse.ArgumentParser(description='Demo for ILA on ImageNet') 14 | parser.add_argument('--modeltype', type=str, default='resnet18', help='ResNet18 | DenseNet121 | alexnet | SqueezeNet1.0') 15 | parser.add_argument('--layerindex', type=int, default='4', help='layer index to emphasize with ila projection') 16 | parser.add_argument('--imagepath', type=str, default='eft_image_test.jpg', help='path to image to test') 17 | parser.add_argument('--outpath', type=str, default='adv_example.jpg', help='path for output image') 18 | parser.add_argument('--imagelabel', type=int, default=27, help='imagenet label (0-999)') 19 | parser.add_argument('--niters_baseline', type=int, default=20, help='number of iterations of baseline attack') 20 | parser.add_argument('--niters_ila', type=int, default=10, help='number of iterations of ILA') 21 | parser.add_argument('--epsilon', type=float, default=0.03, help='epsilon on 0..1 range, 0.03 corresponds to ~8 in the imagenet scale') 22 | opt = parser.parse_args() 23 | 24 | # load pretrained model to attack 25 | def load_model(model_name): 26 | if model_name == 'ResNet18': 27 | return torchvision.models.resnet18(pretrained=True).cuda() 28 | elif model_name == 'DenseNet121': 29 | return torchvision.models.densenet121(pretrained=True).cuda() 30 | elif model_name == 'alexnet': 31 | return torchvision.models.alexnet(pretrained=True).cuda() 32 | elif model_name == 'SqueezeNet1.0': 33 | return torchvision.models.squeezenet1_0(pretrained=True).cuda() 34 | else: 35 | print('Not supported model') 36 | 37 | # load source model 38 | model = load_model(opt.modeltype) 39 | 40 | # load transfer models 41 | all_model_names = ['ResNet18', 'DenseNet121', 'alexnet', 'SqueezeNet1.0'] 42 | transfer_model_names = [x for x in all_model_names if x != opt.modeltype] 43 | transfer_models = [load_model(x) for x in transfer_model_names] 44 | 45 | print('Loaded model...') 46 | 47 | # pre-process input image 48 | mean, stddev = [0.485, 0.456, 0.406], [0.229, 0.224, 0.225] 49 | transform_resize = transforms.Compose([transforms.Resize(256), transforms.CenterCrop(224)]) 50 | transform_norm = transforms.Compose([transforms.ToTensor(), transforms.Normalize(mean, stddev)]) 51 | img_pil = Image.open(opt.imagepath) 52 | img_pil_resize = transform_resize(img_pil.copy()) 53 | img = transform_norm(img_pil_resize.copy()).cuda().unsqueeze(0) 54 | lbl = torch.tensor([opt.imagelabel]).cuda() 55 | 56 | # run attack on source model 57 | img_ifgsm = ifgsm(model, img, lbl, niters=opt.niters_baseline, dataset='imagenet') 58 | 59 | source_layers = get_source_layers(opt.modeltype, model) 60 | ifgsm_guide = ifgsm(model, img, lbl, learning_rate=0.008, epsilon=opt.epsilon, niters=opt.niters_ila, dataset='imagenet') 61 | img_ila = ILA(model, img, ifgsm_guide, lbl, source_layers[opt.layerindex][1][1], learning_rate=0.01, epsilon=opt.epsilon, niters=opt.niters_ila, dataset='imagenet') 62 | 63 | # get labels for source 64 | model.eval() 65 | orig_pred_label, ifgsm_pred_label, ila_pred_label = model(img).max(dim=1)[1].item(), model(img_ifgsm).max(dim=1)[1].item(), model(img_ila).max(dim=1)[1].item() 66 | 67 | # get labels for transfer 68 | transfer_labs = [] 69 | for mod in transfer_models: 70 | mod.eval() 71 | o, f, i = mod(img).max(dim=1)[1].item(), mod(img_ifgsm).max(dim=1)[1].item(), mod(img_ila).max(dim=1)[1].item() 72 | transfer_labs.append((o, f, i)) 73 | 74 | print('Ran attacks...') 75 | 76 | # display results 77 | imagenet_labels = eval(open('imagenet_labels.txt').read()) 78 | 79 | print(f'True label: {opt.imagelabel} ({imagenet_labels[opt.imagelabel]})') 80 | 81 | print(f'{opt.modeltype} (source model)') 82 | print(f'Prediction on original: {orig_pred_label} ({imagenet_labels[orig_pred_label]})') 83 | print(f'Prediction on I-FGSM: {ifgsm_pred_label} ({imagenet_labels[ifgsm_pred_label]})') 84 | print(f'Prediction on ILA: {ila_pred_label} ({imagenet_labels[ila_pred_label]})') 85 | print() 86 | 87 | print('---Transfer Results Follow---') 88 | 89 | for j, name in enumerate(transfer_model_names): 90 | o, f, i = transfer_labs[j] 91 | print(f'{name} (transfer model)') 92 | print(f'Prediction on original: {o} ({imagenet_labels[o]})') 93 | print(f'Prediction on I-FGSM: {f} ({imagenet_labels[f]})') 94 | print(f'Prediction on ILA: {i} ({imagenet_labels[i]})') 95 | print() 96 | 97 | # helpers 98 | class NormalizeInverse(torchvision.transforms.Normalize): 99 | def __init__(self, mean, std): 100 | mean = torch.as_tensor(mean).cuda() 101 | std = torch.as_tensor(std).cuda() 102 | std_inv = 1 / (std + 1e-7) 103 | mean_inv = -mean * std_inv 104 | super().__init__(mean=mean_inv, std=std_inv) 105 | 106 | def __call__(self, tensor): 107 | return super().__call__(tensor.clone()) 108 | 109 | # save output image 110 | invTrans = NormalizeInverse(mean, stddev) 111 | to_pil = transforms.ToPILImage() 112 | out_img = np.array(to_pil(invTrans(img_ila[0]).clamp(0,1).cpu())).clip(0, 255).astype(np.uint8) 113 | 114 | print(f'Image L-inf modification: {np.abs(out_img.astype(np.int32) - np.array(img_pil_resize).astype(np.int32)).max()}') 115 | Image.fromarray(out_img).save(opt.outpath) 116 | 117 | print('Saved image.') 118 | -------------------------------------------------------------------------------- /imagenet_config.py: -------------------------------------------------------------------------------- 1 | import sys 2 | import numpy as np 3 | import pandas as pd 4 | import torch 5 | import torchvision 6 | import torchvision.models as models 7 | import torchvision.transforms as transforms 8 | import pretrainedmodels 9 | from attacks import wrap_attack_imagenet, ifgsm, momentum_ifgsm, ILA, Transferable_Adversarial_Perturbations 10 | 11 | 12 | model_configs = { 13 | "ResNet18": ("ResNet18", models.resnet18), 14 | "DenseNet121": ("DenseNet121", models.densenet121), 15 | "SqueezeNet1.0": ("SqueezeNet1.0", models.squeezenet1_0), 16 | "alexnet": ("alexnet", models.alexnet), 17 | 'Inc-v3': ('Inc-v3',pretrainedmodels.__dict__['inceptionv3']), 18 | 'IncRes-v2': ('IncRes-v2',pretrainedmodels.__dict__['inceptionresnetv2']), 19 | 'Inc-v4': ('Inc-v4',pretrainedmodels.__dict__['inceptionv4']), 20 | } 21 | 22 | def attack_name(attack): 23 | return attack.__name__ 24 | 25 | TAP_params ={'learning_rate': 0.008, 'epsilon': 0.06, 'lam' : 0.005, 'alpha' : 0.5, 's' : 3, 'yita' : 0.01} 26 | attack_configs ={ 27 | attack_name(ifgsm): wrap_attack_imagenet(ifgsm, {'learning_rate': 0.008, 'epsilon': 0.03}), 28 | attack_name(momentum_ifgsm): wrap_attack_imagenet(momentum_ifgsm, {'learning_rate': 0.018, 'epsilon': 0.03, 'decay': 0.9}), 29 | attack_name(Transferable_Adversarial_Perturbations): wrap_attack_imagenet(Transferable_Adversarial_Perturbations, TAP_params), 30 | } 31 | 32 | 33 | use_projection = True 34 | 35 | ILA_params = { 36 | attack_name(ifgsm): {'niters': 10, 'learning_rate': 0.01, 'epsilon':0.03, 'coeff': 0.8, 'dataset': 'imagenet'}, 37 | attack_name(momentum_ifgsm): {'niters': 10, 'learning_rate': 0.018, 'epsilon':0.03, 'coeff': 0.8, 'dataset': 'imagenet'}, 38 | attack_name(Transferable_Adversarial_Perturbations): {'niters': 10, 'learning_rate': 0.01, 'epsilon':0.06, 'coeff': 5.0, 'dataset': 'imagenet'}, 39 | } 40 | 41 | 42 | 43 | def get_source_layers(model_name, model): 44 | if model_name == 'ResNet18': 45 | # exclude relu, maxpool 46 | return list(enumerate(map(lambda name: (name, model._modules.get(name)), ['conv1', 'bn1', 'layer1', 'layer2','layer3','layer4','fc']))) 47 | 48 | elif model_name == 'DenseNet121': 49 | # exclude relu, maxpool 50 | layer_list = list(map(lambda name: (name, model._modules.get('features')._modules.get(name)), ['conv0', 'denseblock1', 'transition1', 'denseblock2', 'transition2', 'denseblock3', 'transition3', 'denseblock4', 'norm5'])) 51 | layer_list.append(('classifier', model._modules.get('classifier'))) 52 | return list(enumerate(layer_list)) 53 | 54 | elif model_name == 'SqueezeNet1.0': 55 | # exclude relu, maxpool 56 | layer_list = list(map(lambda name: ('layer '+name, model._modules.get('features')._modules.get(name)), ['0','3','4','5','7','8','9','10','12'])) 57 | layer_list.append(('classifier', model._modules.get('classifier')._modules.get('1'))) 58 | return list(enumerate(layer_list)) 59 | 60 | elif model_name == 'alexnet': 61 | # exclude avgpool 62 | layer_list = list(map(lambda name: ('layer '+name, model._modules.get('features')._modules.get(name)), ['0','3','6','8','10'])) 63 | layer_list += list(map(lambda name: ('layer '+name, model._modules.get('classifier')._modules.get(name)), ['1','4','6'])) 64 | return list(enumerate(layer_list)) 65 | 66 | elif model_name == 'IncRes-v2': 67 | # exclude relu, maxpool 68 | return list(enumerate(map(lambda name: (name, model._modules.get(name)), ['conv2d_1a', 'conv2d_2a', 'conv2d_2b', 'maxpool_3a', 'conv2d_3b', 'conv2d_4a', 'maxpool_5a', 'mixed_5b', 'repeat', 'mixed_6a','repeat_1', 'mixed_7a', 'repeat_2', 'block8', 'conv2d_7b', 'avgpool_1a', 'last_linear']))) 69 | 70 | elif model_name == 'Inc-v4': 71 | # exclude relu, maxpool 72 | layer_list = list(map(lambda name: (name, model._modules.get('features')._modules.get(name)), ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '10', '11', '12', '13', '14', '15', '16', '17', '18', '19', '20', '21'])) 73 | return list(enumerate(layer_list)) 74 | 75 | elif model_name == 'Inc-v3': 76 | # exclude relu, maxpool 77 | layer_list = list(map(lambda name: (name, model._modules.get(name)), ['Conv2d_1a_3x3', 'Conv2d_2a_3x3', 'Conv2d_2b_3x3', 'Conv2d_3b_1x1', 'Conv2d_4a_3x3', 'Mixed_5b', 'Mixed_5c', 'Mixed_5d', 'Mixed_6a', 'Mixed_6b', 'Mixed_6c'])) 78 | return list(enumerate(layer_list)) 79 | 80 | else: 81 | # model is not supported 82 | assert False 83 | -------------------------------------------------------------------------------- /imagenet_labels.txt: -------------------------------------------------------------------------------- 1 | {0: 'tench, Tinca tinca', 2 | 1: 'goldfish, Carassius auratus', 3 | 2: 'great white shark, white shark, man-eater, man-eating shark, Carcharodon carcharias', 4 | 3: 'tiger shark, Galeocerdo cuvieri', 5 | 4: 'hammerhead, hammerhead shark', 6 | 5: 'electric ray, crampfish, numbfish, torpedo', 7 | 6: 'stingray', 8 | 7: 'cock', 9 | 8: 'hen', 10 | 9: 'ostrich, Struthio camelus', 11 | 10: 'brambling, Fringilla montifringilla', 12 | 11: 'goldfinch, Carduelis carduelis', 13 | 12: 'house finch, linnet, Carpodacus mexicanus', 14 | 13: 'junco, snowbird', 15 | 14: 'indigo bunting, indigo finch, indigo bird, Passerina cyanea', 16 | 15: 'robin, American robin, Turdus migratorius', 17 | 16: 'bulbul', 18 | 17: 'jay', 19 | 18: 'magpie', 20 | 19: 'chickadee', 21 | 20: 'water ouzel, dipper', 22 | 21: 'kite', 23 | 22: 'bald eagle, American eagle, Haliaeetus leucocephalus', 24 | 23: 'vulture', 25 | 24: 'great grey owl, great gray owl, Strix nebulosa', 26 | 25: 'European fire salamander, Salamandra salamandra', 27 | 26: 'common newt, Triturus vulgaris', 28 | 27: 'eft', 29 | 28: 'spotted salamander, Ambystoma maculatum', 30 | 29: 'axolotl, mud puppy, Ambystoma mexicanum', 31 | 30: 'bullfrog, Rana catesbeiana', 32 | 31: 'tree frog, tree-frog', 33 | 32: 'tailed frog, bell toad, ribbed toad, tailed toad, Ascaphus trui', 34 | 33: 'loggerhead, loggerhead turtle, Caretta caretta', 35 | 34: 'leatherback turtle, leatherback, leathery turtle, Dermochelys coriacea', 36 | 35: 'mud turtle', 37 | 36: 'terrapin', 38 | 37: 'box turtle, box tortoise', 39 | 38: 'banded gecko', 40 | 39: 'common iguana, iguana, Iguana iguana', 41 | 40: 'American chameleon, anole, Anolis carolinensis', 42 | 41: 'whiptail, whiptail lizard', 43 | 42: 'agama', 44 | 43: 'frilled lizard, Chlamydosaurus kingi', 45 | 44: 'alligator lizard', 46 | 45: 'Gila monster, Heloderma suspectum', 47 | 46: 'green lizard, Lacerta viridis', 48 | 47: 'African chameleon, Chamaeleo chamaeleon', 49 | 48: 'Komodo dragon, Komodo lizard, dragon lizard, giant lizard, Varanus komodoensis', 50 | 49: 'African crocodile, Nile crocodile, Crocodylus niloticus', 51 | 50: 'American alligator, Alligator mississipiensis', 52 | 51: 'triceratops', 53 | 52: 'thunder snake, worm snake, Carphophis amoenus', 54 | 53: 'ringneck snake, ring-necked snake, ring snake', 55 | 54: 'hognose snake, puff adder, sand viper', 56 | 55: 'green snake, grass snake', 57 | 56: 'king snake, kingsnake', 58 | 57: 'garter snake, grass snake', 59 | 58: 'water snake', 60 | 59: 'vine snake', 61 | 60: 'night snake, Hypsiglena torquata', 62 | 61: 'boa constrictor, Constrictor constrictor', 63 | 62: 'rock python, rock snake, Python sebae', 64 | 63: 'Indian cobra, Naja naja', 65 | 64: 'green mamba', 66 | 65: 'sea snake', 67 | 66: 'horned viper, cerastes, sand viper, horned asp, Cerastes cornutus', 68 | 67: 'diamondback, diamondback rattlesnake, Crotalus adamanteus', 69 | 68: 'sidewinder, horned rattlesnake, Crotalus cerastes', 70 | 69: 'trilobite', 71 | 70: 'harvestman, daddy longlegs, Phalangium opilio', 72 | 71: 'scorpion', 73 | 72: 'black and gold garden spider, Argiope aurantia', 74 | 73: 'barn spider, Araneus cavaticus', 75 | 74: 'garden spider, Aranea diademata', 76 | 75: 'black widow, Latrodectus mactans', 77 | 76: 'tarantula', 78 | 77: 'wolf spider, hunting spider', 79 | 78: 'tick', 80 | 79: 'centipede', 81 | 80: 'black grouse', 82 | 81: 'ptarmigan', 83 | 82: 'ruffed grouse, partridge, Bonasa umbellus', 84 | 83: 'prairie chicken, prairie grouse, prairie fowl', 85 | 84: 'peacock', 86 | 85: 'quail', 87 | 86: 'partridge', 88 | 87: 'African grey, African gray, Psittacus erithacus', 89 | 88: 'macaw', 90 | 89: 'sulphur-crested cockatoo, Kakatoe galerita, Cacatua galerita', 91 | 90: 'lorikeet', 92 | 91: 'coucal', 93 | 92: 'bee eater', 94 | 93: 'hornbill', 95 | 94: 'hummingbird', 96 | 95: 'jacamar', 97 | 96: 'toucan', 98 | 97: 'drake', 99 | 98: 'red-breasted merganser, Mergus serrator', 100 | 99: 'goose', 101 | 100: 'black swan, Cygnus atratus', 102 | 101: 'tusker', 103 | 102: 'echidna, spiny anteater, anteater', 104 | 103: 'platypus, duckbill, duckbilled platypus, duck-billed platypus, Ornithorhynchus anatinus', 105 | 104: 'wallaby, brush kangaroo', 106 | 105: 'koala, koala bear, kangaroo bear, native bear, Phascolarctos cinereus', 107 | 106: 'wombat', 108 | 107: 'jellyfish', 109 | 108: 'sea anemone, anemone', 110 | 109: 'brain coral', 111 | 110: 'flatworm, platyhelminth', 112 | 111: 'nematode, nematode worm, roundworm', 113 | 112: 'conch', 114 | 113: 'snail', 115 | 114: 'slug', 116 | 115: 'sea slug, nudibranch', 117 | 116: 'chiton, coat-of-mail shell, sea cradle, polyplacophore', 118 | 117: 'chambered nautilus, pearly nautilus, nautilus', 119 | 118: 'Dungeness crab, Cancer magister', 120 | 119: 'rock crab, Cancer irroratus', 121 | 120: 'fiddler crab', 122 | 121: 'king crab, Alaska crab, Alaskan king crab, Alaska king crab, Paralithodes camtschatica', 123 | 122: 'American lobster, Northern lobster, Maine lobster, Homarus americanus', 124 | 123: 'spiny lobster, langouste, rock lobster, crawfish, crayfish, sea crawfish', 125 | 124: 'crayfish, crawfish, crawdad, crawdaddy', 126 | 125: 'hermit crab', 127 | 126: 'isopod', 128 | 127: 'white stork, Ciconia ciconia', 129 | 128: 'black stork, Ciconia nigra', 130 | 129: 'spoonbill', 131 | 130: 'flamingo', 132 | 131: 'little blue heron, Egretta caerulea', 133 | 132: 'American egret, great white heron, Egretta albus', 134 | 133: 'bittern', 135 | 134: 'crane', 136 | 135: 'limpkin, Aramus pictus', 137 | 136: 'European gallinule, Porphyrio porphyrio', 138 | 137: 'American coot, marsh hen, mud hen, water hen, Fulica americana', 139 | 138: 'bustard', 140 | 139: 'ruddy turnstone, Arenaria interpres', 141 | 140: 'red-backed sandpiper, dunlin, Erolia alpina', 142 | 141: 'redshank, Tringa totanus', 143 | 142: 'dowitcher', 144 | 143: 'oystercatcher, oyster catcher', 145 | 144: 'pelican', 146 | 145: 'king penguin, Aptenodytes patagonica', 147 | 146: 'albatross, mollymawk', 148 | 147: 'grey whale, gray whale, devilfish, Eschrichtius gibbosus, Eschrichtius robustus', 149 | 148: 'killer whale, killer, orca, grampus, sea wolf, Orcinus orca', 150 | 149: 'dugong, Dugong dugon', 151 | 150: 'sea lion', 152 | 151: 'Chihuahua', 153 | 152: 'Japanese spaniel', 154 | 153: 'Maltese dog, Maltese terrier, Maltese', 155 | 154: 'Pekinese, Pekingese, Peke', 156 | 155: 'Shih-Tzu', 157 | 156: 'Blenheim spaniel', 158 | 157: 'papillon', 159 | 158: 'toy terrier', 160 | 159: 'Rhodesian ridgeback', 161 | 160: 'Afghan hound, Afghan', 162 | 161: 'basset, basset hound', 163 | 162: 'beagle', 164 | 163: 'bloodhound, sleuthhound', 165 | 164: 'bluetick', 166 | 165: 'black-and-tan coonhound', 167 | 166: 'Walker hound, Walker foxhound', 168 | 167: 'English foxhound', 169 | 168: 'redbone', 170 | 169: 'borzoi, Russian wolfhound', 171 | 170: 'Irish wolfhound', 172 | 171: 'Italian greyhound', 173 | 172: 'whippet', 174 | 173: 'Ibizan hound, Ibizan Podenco', 175 | 174: 'Norwegian elkhound, elkhound', 176 | 175: 'otterhound, otter hound', 177 | 176: 'Saluki, gazelle hound', 178 | 177: 'Scottish deerhound, deerhound', 179 | 178: 'Weimaraner', 180 | 179: 'Staffordshire bullterrier, Staffordshire bull terrier', 181 | 180: 'American Staffordshire terrier, Staffordshire terrier, American pit bull terrier, pit bull terrier', 182 | 181: 'Bedlington terrier', 183 | 182: 'Border terrier', 184 | 183: 'Kerry blue terrier', 185 | 184: 'Irish terrier', 186 | 185: 'Norfolk terrier', 187 | 186: 'Norwich terrier', 188 | 187: 'Yorkshire terrier', 189 | 188: 'wire-haired fox terrier', 190 | 189: 'Lakeland terrier', 191 | 190: 'Sealyham terrier, Sealyham', 192 | 191: 'Airedale, Airedale terrier', 193 | 192: 'cairn, cairn terrier', 194 | 193: 'Australian terrier', 195 | 194: 'Dandie Dinmont, Dandie Dinmont terrier', 196 | 195: 'Boston bull, Boston terrier', 197 | 196: 'miniature schnauzer', 198 | 197: 'giant schnauzer', 199 | 198: 'standard schnauzer', 200 | 199: 'Scotch terrier, Scottish terrier, Scottie', 201 | 200: 'Tibetan terrier, chrysanthemum dog', 202 | 201: 'silky terrier, Sydney silky', 203 | 202: 'soft-coated wheaten terrier', 204 | 203: 'West Highland white terrier', 205 | 204: 'Lhasa, Lhasa apso', 206 | 205: 'flat-coated retriever', 207 | 206: 'curly-coated retriever', 208 | 207: 'golden retriever', 209 | 208: 'Labrador retriever', 210 | 209: 'Chesapeake Bay retriever', 211 | 210: 'German short-haired pointer', 212 | 211: 'vizsla, Hungarian pointer', 213 | 212: 'English setter', 214 | 213: 'Irish setter, red setter', 215 | 214: 'Gordon setter', 216 | 215: 'Brittany spaniel', 217 | 216: 'clumber, clumber spaniel', 218 | 217: 'English springer, English springer spaniel', 219 | 218: 'Welsh springer spaniel', 220 | 219: 'cocker spaniel, English cocker spaniel, cocker', 221 | 220: 'Sussex spaniel', 222 | 221: 'Irish water spaniel', 223 | 222: 'kuvasz', 224 | 223: 'schipperke', 225 | 224: 'groenendael', 226 | 225: 'malinois', 227 | 226: 'briard', 228 | 227: 'kelpie', 229 | 228: 'komondor', 230 | 229: 'Old English sheepdog, bobtail', 231 | 230: 'Shetland sheepdog, Shetland sheep dog, Shetland', 232 | 231: 'collie', 233 | 232: 'Border collie', 234 | 233: 'Bouvier des Flandres, Bouviers des Flandres', 235 | 234: 'Rottweiler', 236 | 235: 'German shepherd, German shepherd dog, German police dog, alsatian', 237 | 236: 'Doberman, Doberman pinscher', 238 | 237: 'miniature pinscher', 239 | 238: 'Greater Swiss Mountain dog', 240 | 239: 'Bernese mountain dog', 241 | 240: 'Appenzeller', 242 | 241: 'EntleBucher', 243 | 242: 'boxer', 244 | 243: 'bull mastiff', 245 | 244: 'Tibetan mastiff', 246 | 245: 'French bulldog', 247 | 246: 'Great Dane', 248 | 247: 'Saint Bernard, St Bernard', 249 | 248: 'Eskimo dog, husky', 250 | 249: 'malamute, malemute, Alaskan malamute', 251 | 250: 'Siberian husky', 252 | 251: 'dalmatian, coach dog, carriage dog', 253 | 252: 'affenpinscher, monkey pinscher, monkey dog', 254 | 253: 'basenji', 255 | 254: 'pug, pug-dog', 256 | 255: 'Leonberg', 257 | 256: 'Newfoundland, Newfoundland dog', 258 | 257: 'Great Pyrenees', 259 | 258: 'Samoyed, Samoyede', 260 | 259: 'Pomeranian', 261 | 260: 'chow, chow chow', 262 | 261: 'keeshond', 263 | 262: 'Brabancon griffon', 264 | 263: 'Pembroke, Pembroke Welsh corgi', 265 | 264: 'Cardigan, Cardigan Welsh corgi', 266 | 265: 'toy poodle', 267 | 266: 'miniature poodle', 268 | 267: 'standard poodle', 269 | 268: 'Mexican hairless', 270 | 269: 'timber wolf, grey wolf, gray wolf, Canis lupus', 271 | 270: 'white wolf, Arctic wolf, Canis lupus tundrarum', 272 | 271: 'red wolf, maned wolf, Canis rufus, Canis niger', 273 | 272: 'coyote, prairie wolf, brush wolf, Canis latrans', 274 | 273: 'dingo, warrigal, warragal, Canis dingo', 275 | 274: 'dhole, Cuon alpinus', 276 | 275: 'African hunting dog, hyena dog, Cape hunting dog, Lycaon pictus', 277 | 276: 'hyena, hyaena', 278 | 277: 'red fox, Vulpes vulpes', 279 | 278: 'kit fox, Vulpes macrotis', 280 | 279: 'Arctic fox, white fox, Alopex lagopus', 281 | 280: 'grey fox, gray fox, Urocyon cinereoargenteus', 282 | 281: 'tabby, tabby cat', 283 | 282: 'tiger cat', 284 | 283: 'Persian cat', 285 | 284: 'Siamese cat, Siamese', 286 | 285: 'Egyptian cat', 287 | 286: 'cougar, puma, catamount, mountain lion, painter, panther, Felis concolor', 288 | 287: 'lynx, catamount', 289 | 288: 'leopard, Panthera pardus', 290 | 289: 'snow leopard, ounce, Panthera uncia', 291 | 290: 'jaguar, panther, Panthera onca, Felis onca', 292 | 291: 'lion, king of beasts, Panthera leo', 293 | 292: 'tiger, Panthera tigris', 294 | 293: 'cheetah, chetah, Acinonyx jubatus', 295 | 294: 'brown bear, bruin, Ursus arctos', 296 | 295: 'American black bear, black bear, Ursus americanus, Euarctos americanus', 297 | 296: 'ice bear, polar bear, Ursus Maritimus, Thalarctos maritimus', 298 | 297: 'sloth bear, Melursus ursinus, Ursus ursinus', 299 | 298: 'mongoose', 300 | 299: 'meerkat, mierkat', 301 | 300: 'tiger beetle', 302 | 301: 'ladybug, ladybeetle, lady beetle, ladybird, ladybird beetle', 303 | 302: 'ground beetle, carabid beetle', 304 | 303: 'long-horned beetle, longicorn, longicorn beetle', 305 | 304: 'leaf beetle, chrysomelid', 306 | 305: 'dung beetle', 307 | 306: 'rhinoceros beetle', 308 | 307: 'weevil', 309 | 308: 'fly', 310 | 309: 'bee', 311 | 310: 'ant, emmet, pismire', 312 | 311: 'grasshopper, hopper', 313 | 312: 'cricket', 314 | 313: 'walking stick, walkingstick, stick insect', 315 | 314: 'cockroach, roach', 316 | 315: 'mantis, mantid', 317 | 316: 'cicada, cicala', 318 | 317: 'leafhopper', 319 | 318: 'lacewing, lacewing fly', 320 | 319: "dragonfly, darning needle, devil's darning needle, sewing needle, snake feeder, snake doctor, mosquito hawk, skeeter hawk", 321 | 320: 'damselfly', 322 | 321: 'admiral', 323 | 322: 'ringlet, ringlet butterfly', 324 | 323: 'monarch, monarch butterfly, milkweed butterfly, Danaus plexippus', 325 | 324: 'cabbage butterfly', 326 | 325: 'sulphur butterfly, sulfur butterfly', 327 | 326: 'lycaenid, lycaenid butterfly', 328 | 327: 'starfish, sea star', 329 | 328: 'sea urchin', 330 | 329: 'sea cucumber, holothurian', 331 | 330: 'wood rabbit, cottontail, cottontail rabbit', 332 | 331: 'hare', 333 | 332: 'Angora, Angora rabbit', 334 | 333: 'hamster', 335 | 334: 'porcupine, hedgehog', 336 | 335: 'fox squirrel, eastern fox squirrel, Sciurus niger', 337 | 336: 'marmot', 338 | 337: 'beaver', 339 | 338: 'guinea pig, Cavia cobaya', 340 | 339: 'sorrel', 341 | 340: 'zebra', 342 | 341: 'hog, pig, grunter, squealer, Sus scrofa', 343 | 342: 'wild boar, boar, Sus scrofa', 344 | 343: 'warthog', 345 | 344: 'hippopotamus, hippo, river horse, Hippopotamus amphibius', 346 | 345: 'ox', 347 | 346: 'water buffalo, water ox, Asiatic buffalo, Bubalus bubalis', 348 | 347: 'bison', 349 | 348: 'ram, tup', 350 | 349: 'bighorn, bighorn sheep, cimarron, Rocky Mountain bighorn, Rocky Mountain sheep, Ovis canadensis', 351 | 350: 'ibex, Capra ibex', 352 | 351: 'hartebeest', 353 | 352: 'impala, Aepyceros melampus', 354 | 353: 'gazelle', 355 | 354: 'Arabian camel, dromedary, Camelus dromedarius', 356 | 355: 'llama', 357 | 356: 'weasel', 358 | 357: 'mink', 359 | 358: 'polecat, fitch, foulmart, foumart, Mustela putorius', 360 | 359: 'black-footed ferret, ferret, Mustela nigripes', 361 | 360: 'otter', 362 | 361: 'skunk, polecat, wood pussy', 363 | 362: 'badger', 364 | 363: 'armadillo', 365 | 364: 'three-toed sloth, ai, Bradypus tridactylus', 366 | 365: 'orangutan, orang, orangutang, Pongo pygmaeus', 367 | 366: 'gorilla, Gorilla gorilla', 368 | 367: 'chimpanzee, chimp, Pan troglodytes', 369 | 368: 'gibbon, Hylobates lar', 370 | 369: 'siamang, Hylobates syndactylus, Symphalangus syndactylus', 371 | 370: 'guenon, guenon monkey', 372 | 371: 'patas, hussar monkey, Erythrocebus patas', 373 | 372: 'baboon', 374 | 373: 'macaque', 375 | 374: 'langur', 376 | 375: 'colobus, colobus monkey', 377 | 376: 'proboscis monkey, Nasalis larvatus', 378 | 377: 'marmoset', 379 | 378: 'capuchin, ringtail, Cebus capucinus', 380 | 379: 'howler monkey, howler', 381 | 380: 'titi, titi monkey', 382 | 381: 'spider monkey, Ateles geoffroyi', 383 | 382: 'squirrel monkey, Saimiri sciureus', 384 | 383: 'Madagascar cat, ring-tailed lemur, Lemur catta', 385 | 384: 'indri, indris, Indri indri, Indri brevicaudatus', 386 | 385: 'Indian elephant, Elephas maximus', 387 | 386: 'African elephant, Loxodonta africana', 388 | 387: 'lesser panda, red panda, panda, bear cat, cat bear, Ailurus fulgens', 389 | 388: 'giant panda, panda, panda bear, coon bear, Ailuropoda melanoleuca', 390 | 389: 'barracouta, snoek', 391 | 390: 'eel', 392 | 391: 'coho, cohoe, coho salmon, blue jack, silver salmon, Oncorhynchus kisutch', 393 | 392: 'rock beauty, Holocanthus tricolor', 394 | 393: 'anemone fish', 395 | 394: 'sturgeon', 396 | 395: 'gar, garfish, garpike, billfish, Lepisosteus osseus', 397 | 396: 'lionfish', 398 | 397: 'puffer, pufferfish, blowfish, globefish', 399 | 398: 'abacus', 400 | 399: 'abaya', 401 | 400: "academic gown, academic robe, judge's robe", 402 | 401: 'accordion, piano accordion, squeeze box', 403 | 402: 'acoustic guitar', 404 | 403: 'aircraft carrier, carrier, flattop, attack aircraft carrier', 405 | 404: 'airliner', 406 | 405: 'airship, dirigible', 407 | 406: 'altar', 408 | 407: 'ambulance', 409 | 408: 'amphibian, amphibious vehicle', 410 | 409: 'analog clock', 411 | 410: 'apiary, bee house', 412 | 411: 'apron', 413 | 412: 'ashcan, trash can, garbage can, wastebin, ash bin, ash-bin, ashbin, dustbin, trash barrel, trash bin', 414 | 413: 'assault rifle, assault gun', 415 | 414: 'backpack, back pack, knapsack, packsack, rucksack, haversack', 416 | 415: 'bakery, bakeshop, bakehouse', 417 | 416: 'balance beam, beam', 418 | 417: 'balloon', 419 | 418: 'ballpoint, ballpoint pen, ballpen, Biro', 420 | 419: 'Band Aid', 421 | 420: 'banjo', 422 | 421: 'bannister, banister, balustrade, balusters, handrail', 423 | 422: 'barbell', 424 | 423: 'barber chair', 425 | 424: 'barbershop', 426 | 425: 'barn', 427 | 426: 'barometer', 428 | 427: 'barrel, cask', 429 | 428: 'barrow, garden cart, lawn cart, wheelbarrow', 430 | 429: 'baseball', 431 | 430: 'basketball', 432 | 431: 'bassinet', 433 | 432: 'bassoon', 434 | 433: 'bathing cap, swimming cap', 435 | 434: 'bath towel', 436 | 435: 'bathtub, bathing tub, bath, tub', 437 | 436: 'beach wagon, station wagon, wagon, estate car, beach waggon, station waggon, waggon', 438 | 437: 'beacon, lighthouse, beacon light, pharos', 439 | 438: 'beaker', 440 | 439: 'bearskin, busby, shako', 441 | 440: 'beer bottle', 442 | 441: 'beer glass', 443 | 442: 'bell cote, bell cot', 444 | 443: 'bib', 445 | 444: 'bicycle-built-for-two, tandem bicycle, tandem', 446 | 445: 'bikini, two-piece', 447 | 446: 'binder, ring-binder', 448 | 447: 'binoculars, field glasses, opera glasses', 449 | 448: 'birdhouse', 450 | 449: 'boathouse', 451 | 450: 'bobsled, bobsleigh, bob', 452 | 451: 'bolo tie, bolo, bola tie, bola', 453 | 452: 'bonnet, poke bonnet', 454 | 453: 'bookcase', 455 | 454: 'bookshop, bookstore, bookstall', 456 | 455: 'bottlecap', 457 | 456: 'bow', 458 | 457: 'bow tie, bow-tie, bowtie', 459 | 458: 'brass, memorial tablet, plaque', 460 | 459: 'brassiere, bra, bandeau', 461 | 460: 'breakwater, groin, groyne, mole, bulwark, seawall, jetty', 462 | 461: 'breastplate, aegis, egis', 463 | 462: 'broom', 464 | 463: 'bucket, pail', 465 | 464: 'buckle', 466 | 465: 'bulletproof vest', 467 | 466: 'bullet train, bullet', 468 | 467: 'butcher shop, meat market', 469 | 468: 'cab, hack, taxi, taxicab', 470 | 469: 'caldron, cauldron', 471 | 470: 'candle, taper, wax light', 472 | 471: 'cannon', 473 | 472: 'canoe', 474 | 473: 'can opener, tin opener', 475 | 474: 'cardigan', 476 | 475: 'car mirror', 477 | 476: 'carousel, carrousel, merry-go-round, roundabout, whirligig', 478 | 477: "carpenter's kit, tool kit", 479 | 478: 'carton', 480 | 479: 'car wheel', 481 | 480: 'cash machine, cash dispenser, automated teller machine, automatic teller machine, automated teller, automatic teller, ATM', 482 | 481: 'cassette', 483 | 482: 'cassette player', 484 | 483: 'castle', 485 | 484: 'catamaran', 486 | 485: 'CD player', 487 | 486: 'cello, violoncello', 488 | 487: 'cellular telephone, cellular phone, cellphone, cell, mobile phone', 489 | 488: 'chain', 490 | 489: 'chainlink fence', 491 | 490: 'chain mail, ring mail, mail, chain armor, chain armour, ring armor, ring armour', 492 | 491: 'chain saw, chainsaw', 493 | 492: 'chest', 494 | 493: 'chiffonier, commode', 495 | 494: 'chime, bell, gong', 496 | 495: 'china cabinet, china closet', 497 | 496: 'Christmas stocking', 498 | 497: 'church, church building', 499 | 498: 'cinema, movie theater, movie theatre, movie house, picture palace', 500 | 499: 'cleaver, meat cleaver, chopper', 501 | 500: 'cliff dwelling', 502 | 501: 'cloak', 503 | 502: 'clog, geta, patten, sabot', 504 | 503: 'cocktail shaker', 505 | 504: 'coffee mug', 506 | 505: 'coffeepot', 507 | 506: 'coil, spiral, volute, whorl, helix', 508 | 507: 'combination lock', 509 | 508: 'computer keyboard, keypad', 510 | 509: 'confectionery, confectionary, candy store', 511 | 510: 'container ship, containership, container vessel', 512 | 511: 'convertible', 513 | 512: 'corkscrew, bottle screw', 514 | 513: 'cornet, horn, trumpet, trump', 515 | 514: 'cowboy boot', 516 | 515: 'cowboy hat, ten-gallon hat', 517 | 516: 'cradle', 518 | 517: 'crane', 519 | 518: 'crash helmet', 520 | 519: 'crate', 521 | 520: 'crib, cot', 522 | 521: 'Crock Pot', 523 | 522: 'croquet ball', 524 | 523: 'crutch', 525 | 524: 'cuirass', 526 | 525: 'dam, dike, dyke', 527 | 526: 'desk', 528 | 527: 'desktop computer', 529 | 528: 'dial telephone, dial phone', 530 | 529: 'diaper, nappy, napkin', 531 | 530: 'digital clock', 532 | 531: 'digital watch', 533 | 532: 'dining table, board', 534 | 533: 'dishrag, dishcloth', 535 | 534: 'dishwasher, dish washer, dishwashing machine', 536 | 535: 'disk brake, disc brake', 537 | 536: 'dock, dockage, docking facility', 538 | 537: 'dogsled, dog sled, dog sleigh', 539 | 538: 'dome', 540 | 539: 'doormat, welcome mat', 541 | 540: 'drilling platform, offshore rig', 542 | 541: 'drum, membranophone, tympan', 543 | 542: 'drumstick', 544 | 543: 'dumbbell', 545 | 544: 'Dutch oven', 546 | 545: 'electric fan, blower', 547 | 546: 'electric guitar', 548 | 547: 'electric locomotive', 549 | 548: 'entertainment center', 550 | 549: 'envelope', 551 | 550: 'espresso maker', 552 | 551: 'face powder', 553 | 552: 'feather boa, boa', 554 | 553: 'file, file cabinet, filing cabinet', 555 | 554: 'fireboat', 556 | 555: 'fire engine, fire truck', 557 | 556: 'fire screen, fireguard', 558 | 557: 'flagpole, flagstaff', 559 | 558: 'flute, transverse flute', 560 | 559: 'folding chair', 561 | 560: 'football helmet', 562 | 561: 'forklift', 563 | 562: 'fountain', 564 | 563: 'fountain pen', 565 | 564: 'four-poster', 566 | 565: 'freight car', 567 | 566: 'French horn, horn', 568 | 567: 'frying pan, frypan, skillet', 569 | 568: 'fur coat', 570 | 569: 'garbage truck, dustcart', 571 | 570: 'gasmask, respirator, gas helmet', 572 | 571: 'gas pump, gasoline pump, petrol pump, island dispenser', 573 | 572: 'goblet', 574 | 573: 'go-kart', 575 | 574: 'golf ball', 576 | 575: 'golfcart, golf cart', 577 | 576: 'gondola', 578 | 577: 'gong, tam-tam', 579 | 578: 'gown', 580 | 579: 'grand piano, grand', 581 | 580: 'greenhouse, nursery, glasshouse', 582 | 581: 'grille, radiator grille', 583 | 582: 'grocery store, grocery, food market, market', 584 | 583: 'guillotine', 585 | 584: 'hair slide', 586 | 585: 'hair spray', 587 | 586: 'half track', 588 | 587: 'hammer', 589 | 588: 'hamper', 590 | 589: 'hand blower, blow dryer, blow drier, hair dryer, hair drier', 591 | 590: 'hand-held computer, hand-held microcomputer', 592 | 591: 'handkerchief, hankie, hanky, hankey', 593 | 592: 'hard disc, hard disk, fixed disk', 594 | 593: 'harmonica, mouth organ, harp, mouth harp', 595 | 594: 'harp', 596 | 595: 'harvester, reaper', 597 | 596: 'hatchet', 598 | 597: 'holster', 599 | 598: 'home theater, home theatre', 600 | 599: 'honeycomb', 601 | 600: 'hook, claw', 602 | 601: 'hoopskirt, crinoline', 603 | 602: 'horizontal bar, high bar', 604 | 603: 'horse cart, horse-cart', 605 | 604: 'hourglass', 606 | 605: 'iPod', 607 | 606: 'iron, smoothing iron', 608 | 607: "jack-o'-lantern", 609 | 608: 'jean, blue jean, denim', 610 | 609: 'jeep, landrover', 611 | 610: 'jersey, T-shirt, tee shirt', 612 | 611: 'jigsaw puzzle', 613 | 612: 'jinrikisha, ricksha, rickshaw', 614 | 613: 'joystick', 615 | 614: 'kimono', 616 | 615: 'knee pad', 617 | 616: 'knot', 618 | 617: 'lab coat, laboratory coat', 619 | 618: 'ladle', 620 | 619: 'lampshade, lamp shade', 621 | 620: 'laptop, laptop computer', 622 | 621: 'lawn mower, mower', 623 | 622: 'lens cap, lens cover', 624 | 623: 'letter opener, paper knife, paperknife', 625 | 624: 'library', 626 | 625: 'lifeboat', 627 | 626: 'lighter, light, igniter, ignitor', 628 | 627: 'limousine, limo', 629 | 628: 'liner, ocean liner', 630 | 629: 'lipstick, lip rouge', 631 | 630: 'Loafer', 632 | 631: 'lotion', 633 | 632: 'loudspeaker, speaker, speaker unit, loudspeaker system, speaker system', 634 | 633: "loupe, jeweler's loupe", 635 | 634: 'lumbermill, sawmill', 636 | 635: 'magnetic compass', 637 | 636: 'mailbag, postbag', 638 | 637: 'mailbox, letter box', 639 | 638: 'maillot', 640 | 639: 'maillot, tank suit', 641 | 640: 'manhole cover', 642 | 641: 'maraca', 643 | 642: 'marimba, xylophone', 644 | 643: 'mask', 645 | 644: 'matchstick', 646 | 645: 'maypole', 647 | 646: 'maze, labyrinth', 648 | 647: 'measuring cup', 649 | 648: 'medicine chest, medicine cabinet', 650 | 649: 'megalith, megalithic structure', 651 | 650: 'microphone, mike', 652 | 651: 'microwave, microwave oven', 653 | 652: 'military uniform', 654 | 653: 'milk can', 655 | 654: 'minibus', 656 | 655: 'miniskirt, mini', 657 | 656: 'minivan', 658 | 657: 'missile', 659 | 658: 'mitten', 660 | 659: 'mixing bowl', 661 | 660: 'mobile home, manufactured home', 662 | 661: 'Model T', 663 | 662: 'modem', 664 | 663: 'monastery', 665 | 664: 'monitor', 666 | 665: 'moped', 667 | 666: 'mortar', 668 | 667: 'mortarboard', 669 | 668: 'mosque', 670 | 669: 'mosquito net', 671 | 670: 'motor scooter, scooter', 672 | 671: 'mountain bike, all-terrain bike, off-roader', 673 | 672: 'mountain tent', 674 | 673: 'mouse, computer mouse', 675 | 674: 'mousetrap', 676 | 675: 'moving van', 677 | 676: 'muzzle', 678 | 677: 'nail', 679 | 678: 'neck brace', 680 | 679: 'necklace', 681 | 680: 'nipple', 682 | 681: 'notebook, notebook computer', 683 | 682: 'obelisk', 684 | 683: 'oboe, hautboy, hautbois', 685 | 684: 'ocarina, sweet potato', 686 | 685: 'odometer, hodometer, mileometer, milometer', 687 | 686: 'oil filter', 688 | 687: 'organ, pipe organ', 689 | 688: 'oscilloscope, scope, cathode-ray oscilloscope, CRO', 690 | 689: 'overskirt', 691 | 690: 'oxcart', 692 | 691: 'oxygen mask', 693 | 692: 'packet', 694 | 693: 'paddle, boat paddle', 695 | 694: 'paddlewheel, paddle wheel', 696 | 695: 'padlock', 697 | 696: 'paintbrush', 698 | 697: "pajama, pyjama, pj's, jammies", 699 | 698: 'palace', 700 | 699: 'panpipe, pandean pipe, syrinx', 701 | 700: 'paper towel', 702 | 701: 'parachute, chute', 703 | 702: 'parallel bars, bars', 704 | 703: 'park bench', 705 | 704: 'parking meter', 706 | 705: 'passenger car, coach, carriage', 707 | 706: 'patio, terrace', 708 | 707: 'pay-phone, pay-station', 709 | 708: 'pedestal, plinth, footstall', 710 | 709: 'pencil box, pencil case', 711 | 710: 'pencil sharpener', 712 | 711: 'perfume, essence', 713 | 712: 'Petri dish', 714 | 713: 'photocopier', 715 | 714: 'pick, plectrum, plectron', 716 | 715: 'pickelhaube', 717 | 716: 'picket fence, paling', 718 | 717: 'pickup, pickup truck', 719 | 718: 'pier', 720 | 719: 'piggy bank, penny bank', 721 | 720: 'pill bottle', 722 | 721: 'pillow', 723 | 722: 'ping-pong ball', 724 | 723: 'pinwheel', 725 | 724: 'pirate, pirate ship', 726 | 725: 'pitcher, ewer', 727 | 726: "plane, carpenter's plane, woodworking plane", 728 | 727: 'planetarium', 729 | 728: 'plastic bag', 730 | 729: 'plate rack', 731 | 730: 'plow, plough', 732 | 731: "plunger, plumber's helper", 733 | 732: 'Polaroid camera, Polaroid Land camera', 734 | 733: 'pole', 735 | 734: 'police van, police wagon, paddy wagon, patrol wagon, wagon, black Maria', 736 | 735: 'poncho', 737 | 736: 'pool table, billiard table, snooker table', 738 | 737: 'pop bottle, soda bottle', 739 | 738: 'pot, flowerpot', 740 | 739: "potter's wheel", 741 | 740: 'power drill', 742 | 741: 'prayer rug, prayer mat', 743 | 742: 'printer', 744 | 743: 'prison, prison house', 745 | 744: 'projectile, missile', 746 | 745: 'projector', 747 | 746: 'puck, hockey puck', 748 | 747: 'punching bag, punch bag, punching ball, punchball', 749 | 748: 'purse', 750 | 749: 'quill, quill pen', 751 | 750: 'quilt, comforter, comfort, puff', 752 | 751: 'racer, race car, racing car', 753 | 752: 'racket, racquet', 754 | 753: 'radiator', 755 | 754: 'radio, wireless', 756 | 755: 'radio telescope, radio reflector', 757 | 756: 'rain barrel', 758 | 757: 'recreational vehicle, RV, R.V.', 759 | 758: 'reel', 760 | 759: 'reflex camera', 761 | 760: 'refrigerator, icebox', 762 | 761: 'remote control, remote', 763 | 762: 'restaurant, eating house, eating place, eatery', 764 | 763: 'revolver, six-gun, six-shooter', 765 | 764: 'rifle', 766 | 765: 'rocking chair, rocker', 767 | 766: 'rotisserie', 768 | 767: 'rubber eraser, rubber, pencil eraser', 769 | 768: 'rugby ball', 770 | 769: 'rule, ruler', 771 | 770: 'running shoe', 772 | 771: 'safe', 773 | 772: 'safety pin', 774 | 773: 'saltshaker, salt shaker', 775 | 774: 'sandal', 776 | 775: 'sarong', 777 | 776: 'sax, saxophone', 778 | 777: 'scabbard', 779 | 778: 'scale, weighing machine', 780 | 779: 'school bus', 781 | 780: 'schooner', 782 | 781: 'scoreboard', 783 | 782: 'screen, CRT screen', 784 | 783: 'screw', 785 | 784: 'screwdriver', 786 | 785: 'seat belt, seatbelt', 787 | 786: 'sewing machine', 788 | 787: 'shield, buckler', 789 | 788: 'shoe shop, shoe-shop, shoe store', 790 | 789: 'shoji', 791 | 790: 'shopping basket', 792 | 791: 'shopping cart', 793 | 792: 'shovel', 794 | 793: 'shower cap', 795 | 794: 'shower curtain', 796 | 795: 'ski', 797 | 796: 'ski mask', 798 | 797: 'sleeping bag', 799 | 798: 'slide rule, slipstick', 800 | 799: 'sliding door', 801 | 800: 'slot, one-armed bandit', 802 | 801: 'snorkel', 803 | 802: 'snowmobile', 804 | 803: 'snowplow, snowplough', 805 | 804: 'soap dispenser', 806 | 805: 'soccer ball', 807 | 806: 'sock', 808 | 807: 'solar dish, solar collector, solar furnace', 809 | 808: 'sombrero', 810 | 809: 'soup bowl', 811 | 810: 'space bar', 812 | 811: 'space heater', 813 | 812: 'space shuttle', 814 | 813: 'spatula', 815 | 814: 'speedboat', 816 | 815: "spider web, spider's web", 817 | 816: 'spindle', 818 | 817: 'sports car, sport car', 819 | 818: 'spotlight, spot', 820 | 819: 'stage', 821 | 820: 'steam locomotive', 822 | 821: 'steel arch bridge', 823 | 822: 'steel drum', 824 | 823: 'stethoscope', 825 | 824: 'stole', 826 | 825: 'stone wall', 827 | 826: 'stopwatch, stop watch', 828 | 827: 'stove', 829 | 828: 'strainer', 830 | 829: 'streetcar, tram, tramcar, trolley, trolley car', 831 | 830: 'stretcher', 832 | 831: 'studio couch, day bed', 833 | 832: 'stupa, tope', 834 | 833: 'submarine, pigboat, sub, U-boat', 835 | 834: 'suit, suit of clothes', 836 | 835: 'sundial', 837 | 836: 'sunglass', 838 | 837: 'sunglasses, dark glasses, shades', 839 | 838: 'sunscreen, sunblock, sun blocker', 840 | 839: 'suspension bridge', 841 | 840: 'swab, swob, mop', 842 | 841: 'sweatshirt', 843 | 842: 'swimming trunks, bathing trunks', 844 | 843: 'swing', 845 | 844: 'switch, electric switch, electrical switch', 846 | 845: 'syringe', 847 | 846: 'table lamp', 848 | 847: 'tank, army tank, armored combat vehicle, armoured combat vehicle', 849 | 848: 'tape player', 850 | 849: 'teapot', 851 | 850: 'teddy, teddy bear', 852 | 851: 'television, television system', 853 | 852: 'tennis ball', 854 | 853: 'thatch, thatched roof', 855 | 854: 'theater curtain, theatre curtain', 856 | 855: 'thimble', 857 | 856: 'thresher, thrasher, threshing machine', 858 | 857: 'throne', 859 | 858: 'tile roof', 860 | 859: 'toaster', 861 | 860: 'tobacco shop, tobacconist shop, tobacconist', 862 | 861: 'toilet seat', 863 | 862: 'torch', 864 | 863: 'totem pole', 865 | 864: 'tow truck, tow car, wrecker', 866 | 865: 'toyshop', 867 | 866: 'tractor', 868 | 867: 'trailer truck, tractor trailer, trucking rig, rig, articulated lorry, semi', 869 | 868: 'tray', 870 | 869: 'trench coat', 871 | 870: 'tricycle, trike, velocipede', 872 | 871: 'trimaran', 873 | 872: 'tripod', 874 | 873: 'triumphal arch', 875 | 874: 'trolleybus, trolley coach, trackless trolley', 876 | 875: 'trombone', 877 | 876: 'tub, vat', 878 | 877: 'turnstile', 879 | 878: 'typewriter keyboard', 880 | 879: 'umbrella', 881 | 880: 'unicycle, monocycle', 882 | 881: 'upright, upright piano', 883 | 882: 'vacuum, vacuum cleaner', 884 | 883: 'vase', 885 | 884: 'vault', 886 | 885: 'velvet', 887 | 886: 'vending machine', 888 | 887: 'vestment', 889 | 888: 'viaduct', 890 | 889: 'violin, fiddle', 891 | 890: 'volleyball', 892 | 891: 'waffle iron', 893 | 892: 'wall clock', 894 | 893: 'wallet, billfold, notecase, pocketbook', 895 | 894: 'wardrobe, closet, press', 896 | 895: 'warplane, military plane', 897 | 896: 'washbasin, handbasin, washbowl, lavabo, wash-hand basin', 898 | 897: 'washer, automatic washer, washing machine', 899 | 898: 'water bottle', 900 | 899: 'water jug', 901 | 900: 'water tower', 902 | 901: 'whiskey jug', 903 | 902: 'whistle', 904 | 903: 'wig', 905 | 904: 'window screen', 906 | 905: 'window shade', 907 | 906: 'Windsor tie', 908 | 907: 'wine bottle', 909 | 908: 'wing', 910 | 909: 'wok', 911 | 910: 'wooden spoon', 912 | 911: 'wool, woolen, woollen', 913 | 912: 'worm fence, snake fence, snake-rail fence, Virginia fence', 914 | 913: 'wreck', 915 | 914: 'yawl', 916 | 915: 'yurt', 917 | 916: 'web site, website, internet site, site', 918 | 917: 'comic book', 919 | 918: 'crossword puzzle, crossword', 920 | 919: 'street sign', 921 | 920: 'traffic light, traffic signal, stoplight', 922 | 921: 'book jacket, dust cover, dust jacket, dust wrapper', 923 | 922: 'menu', 924 | 923: 'plate', 925 | 924: 'guacamole', 926 | 925: 'consomme', 927 | 926: 'hot pot, hotpot', 928 | 927: 'trifle', 929 | 928: 'ice cream, icecream', 930 | 929: 'ice lolly, lolly, lollipop, popsicle', 931 | 930: 'French loaf', 932 | 931: 'bagel, beigel', 933 | 932: 'pretzel', 934 | 933: 'cheeseburger', 935 | 934: 'hotdog, hot dog, red hot', 936 | 935: 'mashed potato', 937 | 936: 'head cabbage', 938 | 937: 'broccoli', 939 | 938: 'cauliflower', 940 | 939: 'zucchini, courgette', 941 | 940: 'spaghetti squash', 942 | 941: 'acorn squash', 943 | 942: 'butternut squash', 944 | 943: 'cucumber, cuke', 945 | 944: 'artichoke, globe artichoke', 946 | 945: 'bell pepper', 947 | 946: 'cardoon', 948 | 947: 'mushroom', 949 | 948: 'Granny Smith', 950 | 949: 'strawberry', 951 | 950: 'orange', 952 | 951: 'lemon', 953 | 952: 'fig', 954 | 953: 'pineapple, ananas', 955 | 954: 'banana', 956 | 955: 'jackfruit, jak, jack', 957 | 956: 'custard apple', 958 | 957: 'pomegranate', 959 | 958: 'hay', 960 | 959: 'carbonara', 961 | 960: 'chocolate sauce, chocolate syrup', 962 | 961: 'dough', 963 | 962: 'meat loaf, meatloaf', 964 | 963: 'pizza, pizza pie', 965 | 964: 'potpie', 966 | 965: 'burrito', 967 | 966: 'red wine', 968 | 967: 'espresso', 969 | 968: 'cup', 970 | 969: 'eggnog', 971 | 970: 'alp', 972 | 971: 'bubble', 973 | 972: 'cliff, drop, drop-off', 974 | 973: 'coral reef', 975 | 974: 'geyser', 976 | 975: 'lakeside, lakeshore', 977 | 976: 'promontory, headland, head, foreland', 978 | 977: 'sandbar, sand bar', 979 | 978: 'seashore, coast, seacoast, sea-coast', 980 | 979: 'valley, vale', 981 | 980: 'volcano', 982 | 981: 'ballplayer, baseball player', 983 | 982: 'groom, bridegroom', 984 | 983: 'scuba diver', 985 | 984: 'rapeseed', 986 | 985: 'daisy', 987 | 986: "yellow lady's slipper, yellow lady-slipper, Cypripedium calceolus, Cypripedium parviflorum", 988 | 987: 'corn', 989 | 988: 'acorn', 990 | 989: 'hip, rose hip, rosehip', 991 | 990: 'buckeye, horse chestnut, conker', 992 | 991: 'coral fungus', 993 | 992: 'agaric', 994 | 993: 'gyromitra', 995 | 994: 'stinkhorn, carrion fungus', 996 | 995: 'earthstar', 997 | 996: 'hen-of-the-woods, hen of the woods, Polyporus frondosus, Grifola frondosa', 998 | 997: 'bolete', 999 | 998: 'ear, spike, capitulum', 1000 | 999: 'toilet tissue, toilet paper, bathroom tissue'} -------------------------------------------------------------------------------- /test_images/bear_test_image_label_296.JPEG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CUAI/Intermediate-Level-Attack/25271f84f74a9ee02b9ba0e707cc02680ab236dd/test_images/bear_test_image_label_296.JPEG -------------------------------------------------------------------------------- /test_images/soccer_ball_test_image_label_805.JPEG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CUAI/Intermediate-Level-Attack/25271f84f74a9ee02b9ba0e707cc02680ab236dd/test_images/soccer_ball_test_image_label_805.JPEG -------------------------------------------------------------------------------- /visualize.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "metadata": {}, 7 | "outputs": [], 8 | "source": [ 9 | "import pandas as pd\n", 10 | "import numpy as np\n", 11 | "import torch\n", 12 | "import matplotlib.pyplot as plt\n", 13 | "%matplotlib inline \n", 14 | "from matplotlib.font_manager import FontProperties" 15 | ] 16 | }, 17 | { 18 | "cell_type": "markdown", 19 | "metadata": {}, 20 | "source": [ 21 | "## Usage\n", 22 | "\n", 23 | "This notebook is to visualize the output of `all_in_one_cifar10.py` and `all_in_one_imagenet.py` simply change the `df = pd.read_csv('ifgsm_imagenet_0.03_new.csv')` definition below to read_csv from your specific output file." 24 | ] 25 | }, 26 | { 27 | "cell_type": "code", 28 | "execution_count": 2, 29 | "metadata": {}, 30 | "outputs": [ 31 | { 32 | "name": "stdout", 33 | "output_type": "stream", 34 | "text": [ 35 | "num batches: 29\n" 36 | ] 37 | } 38 | ], 39 | "source": [ 40 | "df = pd.read_csv('ifgsm_imagenet_0.03_new.csv')\n", 41 | "print(\"num batches: \" + str(df[df.with_ILA == True]['batch_index'].max()))" 42 | ] 43 | }, 44 | { 45 | "cell_type": "code", 46 | "execution_count": 3, 47 | "metadata": {}, 48 | "outputs": [], 49 | "source": [ 50 | "def visualize_result(df):\n", 51 | " source_models = df.source_model.unique()\n", 52 | " target_models = df.target_model.unique()\n", 53 | " attacks = df.fool_method.unique()\n", 54 | " metrics = ['acc_after_attack']\n", 55 | " for metric in metrics:\n", 56 | "\n", 57 | " for source_model in source_models:\n", 58 | " fig = plt.figure(figsize = [6*len(attacks), 4])\n", 59 | " shared_ax = None\n", 60 | " for i, attack in enumerate(attacks):\n", 61 | "\n", 62 | " if shared_ax == None:\n", 63 | " shared_ax = plt.subplot(1,len(attacks),i+1)\n", 64 | " else:\n", 65 | " plt.subplot(1,len(attacks),i+1, sharey=shared_ax)\n", 66 | "\n", 67 | " layer_indexs = df[df.source_model == source_model]['layer_index'].unique()[1:]\n", 68 | " layer_names = df[df.source_model == source_model]['layer_name'].unique()[1:]\n", 69 | " print(layer_names)\n", 70 | " for target_model in target_models: \n", 71 | " r = df[(df.source_model == source_model) & (df.target_model == target_model) & (df.fool_method == attack)]\n", 72 | "\n", 73 | " fla = r[r.with_ILA].groupby('layer_index')\n", 74 | " other = r[r.with_ILA == False]\n", 75 | " xs = fla.layer_index.unique()\n", 76 | " baseline = other[metric].mean()\n", 77 | " fla_r = fla[metric].mean()\n", 78 | " names = fla.layer_name.unique()\n", 79 | " p = plt.plot(layer_indexs, [baseline for i in xs], linestyle = '--', label = target_model)\n", 80 | " plt.plot(fla_r, label = target_model + \" ILAP\", color = p[0]._color)\n", 81 | " if source_model != target_model and ((metric == 'fool_rate' and not(fla_r > baseline).any())or (metric == 'acc_after_attack' and not(fla_r < baseline).any())):\n", 82 | " print(\"never perform absolutely better: \" + source_model + \" \" + target_model)\n", 83 | "\n", 84 | " plt.ylim([0,75])\n", 85 | " plt.ylabel('Accuracy after Attack')\n", 86 | " plt.xlabel('Layer Index')\n", 87 | " fontP = FontProperties()\n", 88 | " fontP.set_size('small')\n", 89 | " \n", 90 | " plt.legend(ncol = 3, loc=2, prop=fontP)\n", 91 | " plt.show()\n" 92 | ] 93 | }, 94 | { 95 | "cell_type": "code", 96 | "execution_count": 4, 97 | "metadata": {}, 98 | "outputs": [ 99 | { 100 | "name": "stdout", 101 | "output_type": "stream", 102 | "text": [ 103 | "['conv1' 'bn1' 'layer1' 'layer2' 'layer3' 'layer4' 'fc']\n" 104 | ] 105 | }, 106 | { 107 | "data": { 108 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEKCAYAAAAfGVI8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzs3Xd4FNX6wPHvSS+UJIQSCFV6EwSBgJAiKkVErIggYMGCgqIolqvIvVLkZ70qqNeL5VoRKyAWIBDpRXqVZoJJIIGEkJ7s+f0xmyWbTdmUzSbZ9/M8++zunJ2Zk2Rz3plzzryjtNYIIYRwXW7OroAQQgjnkkAghBAuTgKBEEK4OAkEQgjh4iQQCCGEi5NAIIQQLk4CgRBCuDgJBEII4eIkEAghhIvzcHYF7BEcHKzbtGnj7GoIIUStsmPHjiStdeOyPlcrAkGbNm3Yvn27s6shhBC1ilLqlD2fk64hIYRwcRIIhBDCxUkgEEIIF1crxgiKSklJISkpidzcXGdXRQghKszT05Pg4GACAgKcWo9aGQji4+Np06YNPj4+KKWcXR0hhCg3rTVZWVmcPHlSAkFF+fr6OrsKQghRYUqpGtOOyRiBHU6ePEnjxo2JiIhgwIAB7Nixw+51IyIiePDBBy3vBwwYUOp+fvnlF8v7yZMn07hxY9566y3LspiYGPr3709YWBgvvvhiOX+SuqHg7xEVFUVkZCQvvPCCw7oJZ8+eTbdu3Si4k9/YsWM5efJkiZ9/7733LK/ffvtt2rRpwy233GJZtmTJEgYOHEhYWBjz588HIC0tjf79+1OvXj327dtns83o6GieeOIJwPg+Xbx40eYzAwcOZN68eZb3H374IR06dGDIkCGMHj2a7Ozs8v3gNdzJkyetfq9VrfDf0RVIILBTeHg40dHRvPrqq5Z/YHtt376dxMTEMj9XNBDMnTuXhQsXWn1m4cKFfPTRR2zatIlff/2VlJSUctWlrggPD2fNmjWsWbMGk8nE66+/7rB9KaX48ccf7fps4Qbk1ltvZfXq1VblERERbNiwgY0bN7J8+XLOnDmDr68vy5cvr3DDFhsbS+vWrW32NX36dNavX0+rVq34+uuvK7RtVyWBQJQqJSUFrTVJSUnceOONREVFMX78ePLz89m0aRP9+vUjPDyc559/3rLOww8/zBtvvGG1neLWX7RoEV9++SURERGkpqYSEhJis/+uXbuSmppKbm4u7u7u+Pj4OPxnrsmUUjz33HN8//33AKxatYrBgwczcOBAPv/8cwAmTZrEfffdx9ChQxk9ejRa62L/VsWtC0aD+tprr1ntV2vNI488QmRkJNdccw1xcXEsWrSIw4cPExERwbp162jSpAnu7u5W67Vt2xalFEopPD09cXd3x8PDg8aNy7z4s0Rff/0148ePp127dhw7dsymvHv37sTFxVV4+zVBYmIiQ4cOZciQIdxyyy3k5+dbyrZv305kZCSDBw/m//7v/wB49tln+eSTT8jIyOCqq64iKSmp2O+BPX9HV+CwMQKlVCfgy0KL2gHPAx+bl7cBTgK3aa3PV3Q/t7+7yWbZ9T1DmBDWhsycfCYt2WpTfkufUG7t25Jz6Tk8+L8dfHl/WJn7WbduHf379+fYsWOsWbOG+fPnM23aNKKionjllVf49ttv2b17N88//zzXX389JpPJsu6YMWO45pprSEtLsywrbv0HH3yQli1bWr7Mxbnxxhu5+eab8fb2Zvz48TUzEPw0CxL2Vm4bzXrAcPvOvLy9vcnOzsZkMjFnzhyio6Px8PAgMjKS2267DYDBgwfz/vvvc+edd7J3715Wrlxp9bcqbd2QkBAuu+wyYmJiLPtcsWIFgYGBrF27lh07djB//nzeeustPvjgA6Kjo8us8w8//ECHDh1o1KhR+X83RaxevZqHH34YPz8/vv76a5566imr8piYGId2owCcmnCXzbL6w4cRNG4cpsxMYqfcb1PecMwYAm4aQ97585yeNp3Wn3xc4vYDAwNZtWoVHh4ezJgxgzVr1ljKnnrqKb755hsCAwMZM2YMEyZM4IUXXuDaa6/lp59+YubMmQQHBwO234O//vqrUn/HusJhgUBrfRjoBaCUcgdOA98Cs4DVWuv5SqlZ5vdPlbihGiI8PJyvv/6aN954gy1btnDgwAG2bNnCnDlzyMzMZMKECUydOpV58+bx1VdfMXbsWEaMGAGAm5sb9957L4sXL7Zsr7j1C76spXn88ceJiYmhdevW3HzzzRw8eJAuXbo47OeuDXJycvDy8iIpKYmjR49y7bXXAsZZ19mzZwHo3bs3AC1btuT8+fM2f6u+ffuWuC7AzJkzmTlzJn5+foDx9/v2229Zv349Wmtatmxpd3337NnD66+/zooVKyr9s8fFxbFnzx5GjRqFyWQiMzPTEgjeeOMNvvrqK3r16sUNN9xQ6X0507lz53jggQc4f/488fHxNGzY0FK2d+9exowZA8D58+eJjY2ladOm3H777bz55pt89tlnls8W/R5U5u9Yl1TXrKGrgWNa61NKqdFAhHn5R0A0lQgEpR3N+3q5l1oe5O9l19lAYQ899BADBw5k8ODBjBkzhsGDBwOQm5tLXl4eb7zxBjk5OfTp08cSCADuuusuBg0aZBl07Ny5s836W7dutTrlLY6bmxsNGzbEzc2N+vXrk5qaWq76Vws7j+Sryty5cxk9ejTBwcF06dKFX3/9FU9PT3Jzc/H09ASwmmastaZhw4ZWf6vdu3eXuC5Ap06d8PT0tAzmdu7cmdtuu41//OMfAJbB6rKmM8fHx3P//fezbNmyKpkxUnBwUtAQ3nvvvZw4cQIwurQefvjhSu/DHqUdzbv5+pZa7hEYWGo5wKeffsq1117LQw89xIwZM2jdujV79xpnnZdffjlff/01DRs2JD8/Hzc3N5KSkvj888+5+eab+eCDD7jnnnsA2+9BRf+OdU11jRGMBQo6XZtqreMBzM9NqqkOVcLT05OhQ4fSv39/XnvtNaKiooiKimL37t28++67DBkyhLCwMCZNmmS1nre3N7feeivJycmA0YdZdP0ePXqwY8cObrnlFtLS0nj66adZuHAhb731Fo8//jgAL7zwAiNGjOCqq67C09OT/v37V/evoEZYt24dUVFRREREkJeXx6OPPoqbmxvPPvssQ4cOJTIykjvvvLPE9Yv+rexZ98knn2T//v0AjBo1iuTkZCIjI4mMjOTjj42GrFOnTtx8881s3ryZL774gvHjxxMTE8PQoUMBYxbSmTNnGDduHBERERw+fBiAESNG8Msvv3DfffdZtlWSUaNGMXToUIYOHcqyZcsIDw+3lEVFRdXJgeGrr76aRYsWMXr0aBISEqzK5s+fz0033URkZCQjRowgKyuLqVOnsmDBAl588UU+//xzTp0qPveaPX9HV6AKjlAdtgOlvIC/gW5a60SlVIrWOqBQ+XmtdWAx600BpgC0atWqT+E/pHSHCCHqCke2Z0qpHVrrvmV9rjrOCIYDO7XWBfMnE5VSIQDm5zPFraS1fk9r3Vdr3bcyMyqEEEKUrjoCwR1c6hYC+AGYaH49Efi+GuoghBCiBA4NBEopP+Aa4JtCi+cD1yiljprLqndkUQghhBWHzhrSWmcAjYosS8aYRSSEEKIGkCuLhRDCxUkgKENlEs5B9SedmzRpEvv27SsxKdeRI0dwd3fn9OnTVnUcPHgwffv2rTU5Vi5cuMCoUaOIiIigX79+ducCchSlFF9+aVxIf+jQIZvpw4VFR0dz5MgRoOyEc3l5eUyePJnBgwczffp0h9S9tnJW4rm+fY1JOLNnz2b58uU25U8//TTDhg2zvC/chgwaNIg///zTMRWuBAkEdqhMwjmoWUnnvvrqK8sFTYX99NNPxMTEMHfu3Fpxw59PPvmEYcOGER0dzZYtWywX5jlL27ZtrQJ2aQoHgrISzv3444+0aNGCmJgYMjIy2LhxY5XVWZSuogdFW7duxcfHx3LNEFxqQ2bMmMGCBQuqqopVRgJBORQknIPik8bVhqRz69evZ+HChVYBp4Cvry8hISGcO3euXNt0Bj8/P7Zu3UpiYiJKKQICApg7dy5hYWE8/PDDXHnllcClMySAWbNmER0dXWyiMTBSNxcknVuzZg35+flEREQQERFB586defzxx8nKymL8+PFERUVxww03cOHCBQCCgoLo1auXze+1aEK0zMxMPvzwQ55++mkmT55cZsK5TZs2WdJeDBs2zGUDQWlJ56DmJJ77448/uOKKK7jtttv47rvvbMpragLAWntjGoAXf9zPgb8vVHo7XZs34IVR3UosL5pwDopPGlfTk84dOXKETp064e/vT+PGjYmPj7cKNklJSSQmJlYqE2aByasm2yy7rs11jO08lsy8TB767SGb8tHtR3Nj+xs5n3WeQB+bawytTJgwgfj4eK677jr8/PxYsGABP//8Mxs3buTo0aNWp+ZFFZcwbvbs2Xz++eesX7+ezMxMRo0aRVRUFNHR0aSkpDBmzBgee+wx/vOf/xAVFcXdd9/NsmXLeO+99yz3Cnj88ce5//77rYJ+cQnRJk2aRN++fbn++uvL/D2mpKTQoEEDABo2bFgjg3TC3LlkHzxU6e14d+lMs2eeKbasuKRzHTp0sJTXlMRzS5cu5fbbb6dLly7cddddltQWBWJiYujUqVP5fjHVoFYHgupSNOFcz549a2XSuaVLl7J161aGDRtGUlISy5Yts+SiGT58OG5ubrzxxhu4udX8E0UPDw+eeeYZnnnmGdauXcvzzz9Pz549UUrRsWNHS1KyorlloPiEccePH+fAgQNERkYCWBLO5efnM3HiRBYsWEBoaCgHDhxg27ZtfPzxx+Tm5lp1SbVp04bg4GC2bdtmWVZcQrTyCAwMtJx1pKSkEBQUVN5fVZ1QNOnc5ZdfbhUIakriuZUrV7Jz507A+J6dP28kVl63bh0REREEBQWxaNGiCm/fUWp1ICjtKN4RChLOTZ48udikcTU96dxvv/3Gpk2bcHNzIzs7mzFjxlgCwU8//US9evXK9fsozZJhS0os8/XwLbW8rLMBgFOnThESEoKXlxdNmjTBy8uLffv2obXm2LFjlt9LYGAgsbGxdO/enZ07dzJ8+PBiE42lpqbSs2dPli9fjlLKMk7y6KOPMnbsWPr16wcYf7ewsDAmTJhgWbewJ598kvHjx9OnTx+g+IRoP//8c5l/5wIDBgzgl19+YciQIfz888/cfffddq1XnUo6iq9KRZPOFU2NUxMSz+3atYvw8HDLGeEnn3zC999/T0REhOVgsqaq+Yd+NUhBwrlly5YVmzSupiWdK0h2NnToUBYuXGgJIAX10VrbJPCqLfbu3cuQIUOIiIhg6tSpvPLKK1xzzTWEhYXx6quvWvL8T5o0iVmzZnHDDTdYsokWl2gsODiYsWPHEh4eTmRkJI8//jixsbEsWbKEd999l4iICN566y2mTJnCr7/+avm7FR0TuPzyywkNDbW8Ly4hWkF34IwZM4DiE87df//9lrrGxcUxePBgfH19CQsrX7bcuqK0pHPgvMRzzz33nOV/7MUXX7ScURbUeenSpZX8yauHw5POVYW+ffvq7du3W95L0jlRlgEDBrhM5khRu7lK0jkhhBA1mAQCUSfJ2YAQ9pNAIIQQLk4CgRBCuDgJBEII4eIkEAghhIuTQGCHguyBUVFRREZG8sILLzgkMdvs2bPp1q2b5WKZsWPHcvLkyRI/Xzgp1ttvv02bNm2skpctWbKEgQMHEhYWZkmWV1a2y9qgpmUeBedlH/3www8tye4KsmIWprUmNDSUzz+/dJPA2bNn06NHDwYNGsS9995brp9T1E0SCOwUHh7OmjVrWLNmDSaTiddff90h+1FK2d2wFQ4Et956K6tXr7Yqj4iIYMOGDWzcuJHly5dz5syZMrNd1gY1LfMo1Nzsoxs2bGDEiBF8++23VsvnzZvHhg0bSExMZMOGDeXapqh7anWKCQCWjLRd1u1G6Hcf5GTAp7falvcaB73vhPRk+OoumLzC7t0ppXjuuee4+uqr6dGjBy+99BL5+fk88sgj3HHHHUyaNAlPT09OnDiBv78/3333HZs3b2b69On4+voSHh7OnDlzWLVqlc26ANOnT+e1117jhhtusOxTa820adPYt28fHh4eLFmyhB9//NGSHfHFF18kPDycjIwMq7q2bdvW8trT0xN3d/cys11WhQVbF3DoXOWSkHUO6sxT/Z4qtszPz4/o6GhuueUWmjZtSkBAAGCk7f7xxx/p06cPW7ZsYdu2bUyaNIknnniC7t27M2vWLIYNG0Z4eLjN7zM0NJQPP/yQDz74gPz8fP71r38RHh7O1VcbN9NLSEhg5MiRvPTSS9x77738/fff1KtXj//97380aNDAKvtoq1atLHXdvn07M2fOJC8vj9GjRzN16lQ+/PBDli1bxrJly1iyZEmZ2UcLktMVZB8dOHCg3b/HpUuXMn36dJ555hkuXrxok0akpmbDFNVLzggqwNvbm+zsbObMmcPq1av5/fffWbx4sSV/zODBg/ntt9+oV68ee/fuZeXKlTz//POsW7eO2bNnYzKZSlw3JCSEyy67jJiYGMv+CmfLnD9/PvPnz+fBBx+kU6dOREdHEx4eXmp9f/jhBzp06GBJu1DbTZgwgU6dOnHdddcxcOBAjh49SkJCgiX76LRp06xywRdV3O+zIC/N+vXr+e2333jppZdwd3cnOjqa7777jpCQEKvso2vWrGHixIlWZ2WPP/44r7zyitW+CrJixsTEsGHDBi5cuMCkSZOYN28eS5aUnG+pQGWyj2qtOXz4MN26dePGG29kxQrrA578/Hw2btxYI7NhiupV+88ISjua9/Irvdy/UbnOBgrk5ORw8eJFkpKSLLnik5KSLBkri2Y3LJqVtG/fvhw9erTYdQFmzpzJzJkz8fPzA4rPlmmvPXv28Prrr9s0Ao5U0pF8VSku8+j06dMl+2gRGzdu5NixYwwbNoycnBxLNk4w7qL18ssvc+2119KrV69y1UnUPQ4NBEqpAOA/QHdAA3cDh4EvgTbASeA2rfV5R9ajqs2dO5fJkyezfPlyfv31Vzw9PcnNzbUkNSva+DRs2NAqK+nu3bvp0qVLseuCkezK09PTMnhY0eyI8fHxlruR+fr6VunvwJmKZh41mUy0adNGso8WsXTpUr766ivLgcmoUaNIT08HjDECe+6HIFyDo7uG3gBWaa07A5cDB4FZwGqtdQdgtfl9jbdu3TqioqKIiIggLy+PRx99lGeffZahQ4cSGRnJnXfeWeK6RbOSurm5lbnuk08+yf79+wH7siN+8cUXjB8/3pJxFIzZIWfOnGHcuHFERERw+PBhoPhsl7VJ0cyj//jHP2jWrJnLZx+9cOGCJRPmnXfeyZo1a7j88sst5WFhYaxcubIK/gKirnFY9lGlVANgN9BOF9qJUuowEKG1jldKhQDRWutSOykl+6goL8k+KmqLup59tB1wFliilPpDKfUfpZQ/0FRrHQ9gfm7iwDoIIYQogyMDgQdwBbBIa90bSKcc3UBKqSlKqe1Kqe2FB1KFsIecDQhhP0cGgjggTmu9xfz+a4zAkGjuEsL8fKa4lbXW72mt+2qt+xY3z7o23FBHCCFKU1PaMYcFAq11AhCrlCro/78aOAD8AEw0L5sIfF/ebfv4+JCcnFxjfolCCFFeWmuSk5Px8fFxdlUcfh3BI8CnSikv4DgwGSP4fKWUugf4Cyjm0t/ShYaGEhcXh3QZCSFqMx8fH6tZZs7i0ECgtd4FFDdifXVltuvp6WmVPkEIIUTFSYoJIYRwcRIIhBDCxUkgEEIIFyeBQAghXJwEAiGEcHESCIQQwsVJIBBCCBdXZiAwX/hVdNl8x1RHCCFEdbPngrJblFJZWutPAZRS7wDejq2WEEK4mLxsOH8Szh2H5GPG87ljcPMH4B/s0F3bEwhuAn5QSpmA4cA5rfVDDq2VEELURSU19snHITUW40aOZj4B0OgyyEp1XiBQShW+Oeq9wHfABmCOUipIa23/XbSFEMJVVKSxb9UfgsZBUDvjfVA78LP//tSVVdoZwQ6MGqtCzyPND41x4xkhhHA9No29ucGvwY19aUoMBFpryeomhHBddayxL02ZYwRKqanAp1rrFPP7QOAOrfU7jq6cEEI4lBMae52Xh87NLfmRU/Ccg87Nxa/PFbj5+VX5j15YmTevV0rt0lr3KrLsD/PtJ6tF0ZvXCyFESUyZmZgyMi41rJnp6ORTxuPcKfS5WHTK3+jz8eiLyWgT5odCu/mifYLRPkFor0C0ZwO0Rz20uz8ad3RuLhTXaBdquMt6YDKV6+dpt3IF3u0q1hNv783r7Zk15KaUUtocMZRS7oBXhWolhBBVKD81lawDB4zH/gNk7d9PzqlT5dhCYDHL0syPU+DujvL0tOvhVq+e7XIv6/cU/ryXl9V764eX5bVnSEjV/LJKYU8g+BnjjmKLMc6THgBWObRWQghRRN65c0Zjf8Bo8LMOHCA3Ls5S7tm8OT5NvWhQ/wLu9bxR9RujGjZFNWiGCghBBYWiAkNR9RvZNLZFG2zLw93diT9x9bEnEDwF3A88iDFz6BfgP46slBDCteWeOWNp7LMOHCRr/37yEhIs5Z6tWuHTozsBt9+GT9eu+HTtiseZLfD5WOg2Bm75LyjlxJ+gdikzEGitTcAi80MIIaqM1pq8+HiyDhwg09LwHyD/bJLxAaXwatsWv759jQa/Wzd8unTGvUED6w2dOQTL7oWQnjD6bQkC5WTPrKEOwDygK+BTsFxrXSuuIzg14S6bZfWHDyNo3DhMmZnETrnfprzhmDEE3DSGvPPnOT1tuk154B1jaTBiBLnx8fz95FM25UGTJ1M/KpLs4ydIeOEFm/LgBx/Af+BAsg4eJHHuPJvyxo89ht8VvcnY+QdnX3vNprzpM0/j06UL6Rs3krRosU15sxdfxLtdW9LWrOXckiU25c1fXoBnSAgXVq7k/Odf2JS3ePMNPAIDSfnmW1K//damvOV77+Lm68u5zz4j7SfbXsLWn3wMQPIH/+VidLRVmfLxodX77wFw9p13yNi02arcPSCA0H+/CcCZV14lc9cuq3KPZs1osfBlABLmziX74CGrcq82bQj55xwA4v/xPDknT1qVe3fpTLNnngHg9MwnrY4yAXx79aLJ4zMAiHtkGvkpKVblfmEDaPyQcWH9X/dNQWdlWZXXi4ig0T13A/LdK/rd01rT5LFHyTt7ltQffiRj+3ZMGRmQl2f5jFe7dtQbdBU6L4/sP//Ezc8P5e5OXmIiFxMTCbxjrO13z5QH8bvAVI/W338GXn515rtX8L/kaPZ0DS0BXgBeAyKByRhdRGVSSp3EGHXJB/K01n3NVyx/CbQBTgK3aa3Pl7fiQoiaS5tMltk7powMTOnpmDIyOHXneOMD7u4ob2/cAwJw8/fHzc8PN19fQt9+y3IQkpeYaM+e4OwhYxposx7QMNShP1ddZc/00R1a6z5Kqb1a6x7mZTFa68FlbtwIBH211kmFlr2Mka9ovlJqFhCotbY9tClEpo8KUXPpvDxyTpyw6trJPnDQONoHlJcX3p064dOtq7k/vxveHTsYs2Yq66dZsGUR3PAWXDGh8turY6py+miWUsoNOKqUehg4DTSpRN1GAxHm1x8B0RgD0kKIGk7n5JB97JjVdM2sw4ctXWTK1xefzp1pOGaM0eh374Z3u3bG1MmqtvMTIwgMeEiCQCXZEwgeBfyAacA/MbqHbDs/i6eBX5RSGnhXa/0e0FRrHQ+gtY5XSlUmqAghHMSUnU32kaOFZu8cIPvwYeOiKMDN3x+fLl0IvP12y9G+V9u21TPl8q/NsPwxaBcJ1/zT8fur4+wJBG201tuAixjjAyilbgW22LHuIK313+bG/lel1KEy1zBTSk0BpgC0atXK3tWEEOWgtUbn5GBKTyfn1Cmri7Oy//zTMpDr1rAhPl27EHjXBHy7dcOna1c8W7VCuTnhJocpsfDleAhoaUwTdbenGROlsec3+DSw1I5lNrTWf5ufzyilvgX6AYlKqRDz2UAIcKaEdd8D3gNjjMCOegpRp1ka7YLB1/QMTBnpNgOy2vLeernNw7yc/Hyr/bgHBeHTrRv1wsPNUza74tmiBaomTMnMyYAvxkFuFkxaUSsSutUGpd2PYDgwAmihlHqzUFEDIK/4tazW9wfctNZp5tfXAnOAH4CJwHzz8/cVr74QNVPhI+1LjXK6TcNs3WinX2q8M4pvxIs22iVSCjdfX8uMHOXvh5ufH+5BgXi2DDVm6fiZZ+uYH57NQ/Dp1g2Ppk1rRqNflNbw/VRI2AvjvoTGnZxdozqjtDOCv4HtwA0Y9yYokAY8Zse2mwLfmr9QHsBnWutVSqltGCkr7gH+Am6tSMVF7ZV/4QI6OxttMkF+/qXnfBOYzM/5edbvi33OB5Op+Od8E9qUb95u4fclP5e6r7z84pfn5pV41F2uRrtQg2xptBsF2Tba/taNt5u/X7HLlY+Pc7ptHCnmFdj/DQx9ETpe5+za1Cml3Y9gN7BbKdVUa/1R4TKl1HTgjdI2rLU+DlxezPJk4OqKVVfURvkpKaRv2Ur65k1kbNpsc6GN0yllzGt3cyv92d3ddrmnB+5+/rgHN8LTr+WlRtvf9ojbzb+YhtzPD+XrWzOPwGuSQytgzT+hx20wyPZCO1E59owRjAVeLrJsEmUEAuG6TJmZZOzYScbmTaRv3ETWwYOgNcrPD78r+9Lwpptwr18P3NxR7m7FPuPuZjS8bsaz8dpcXqgxLii3fS6yzVK2IY1wDZd4AL6ZAs17ww1vSvoIByhtjOAOYBzQVin1Q6GiBkCyoysmag+dm0vm3n2WI/7MXbuMKYaenvhe3pPgqVPxHxiGb48ejplPLuqujHNGIjkvfxj7GXj6OrtGdVJpZwQbgXggGHil0PI0YLcjKyVqNm0ykX30KOmbjIY/Y9s2o09cKby7dCZwwgT8wwbg16ePw++sJOqw/Fz46i5IS4DJK6FBc2fXqM4qbYzgFHAKCCu8XCk1CHgTmOrYqomaJCc21mj4N28mffMW8s8k6l1UAAAgAElEQVSdA8CrdWsa3DAK/wFh+PXvh0dgcTf6EKICfn4GTsbAjYshtMwsCaIS7LoSQynVC6Ob6DbgBPCNIyslnC8vOZn0zZstR/25p08D4NG4Mf5XDcJ/QBj+YQOq5e5JwgVtXwJb34Owh6HXHc6uTZ1X2hhBR4yB4jswxgS+xEhSF1lNdRPVKP/iRTK2bTOO+DdtJvvIEQDc6tfHr38/giZPxj9sAF7t2sngqnCsUxth5RPQfihcM8fZtXEJpZ0RHAJigFFa6z8BlFL2XD8gagFTTg6Zf+y6NMC7dy/k56O8vfG9ojeNZ8zAP2wAPl27uszt+kQNkPIXfDkBAtvAzR8YM8iEw5UWCG7GOCNYq5RaBXyBnfchEDWPzs8n68BBS8OfsXOnkTHSzQ2fHt1pdO+9+IcNwLd3b9y8vZ1dXeGKctLh83HGIPEdX4BvgLNr5DJKGyz+FuPKYH/gRoyriZsqpRYB32qtf6mmOooK0FqTc+Ik6Zs2Gt09W7dhSk0FwLtDewJuvdWY2XPllbjXr+/k2gqXZzLBtw/Amf0w7isI7uDsGrkUe+5ZnA58CnxqvrvYrcAsjJvYixokNzHRMribvnmz5Q5Pns2bU3/o1cYA74D+eDRu7OSaClHE+oVw8Ae49l/Q4Rpn18bllCt/q9b6HPCu+SGcLD8lhfStWy0DvDknTgDgHhiI34D+l2b2tGwpA7yi5jr4I0TPhcvvMGYJiWonibxrEavUDZs2k3XgwKXUDX37EHDbbfiHDcC7Y8e6l3BM1E0J++Cb+6FFX7j+dUkf4SR1PxAsGWm7rNuN0O8+I7f5p8UkP+01DnrfCenJxpWNRV15N3S/GVLjjC9xUQMfhk7DIeko/PiobfmQJ+CySIjfA6ueti2/+nnyg7qT/ft3ZP+0mOyzOWQlZpOVkI3OBzzc8e3Vi+BxI/DnD3xDvFHup4GlsGkpBL9u9LEe/gk2vmW7/ZveNW7yvW8ZbPuvbfltH4N/I/jjU9j1mW35nUvByw+2vg/7v7Mtn7zCeN7wJhz52brM0wfGLzNer3sZjq+zLvcLhNv/Z7z+bTbEbrMub9Acbn7feP3TLCMlcWGNLjPy0QD8MA2Sj1mXN+sBw+cbr5fdBxf+ti5veSUMnW28/nI8ZJy3Lm8XDuFPGq//d7ORF7+wjtfBoGnG61r63aNVf/hrC6wuZurmsHkQ0hOOrYX1/2dbPqoc373N70L8LowbGSrj9ynfPeN1wXev4OdxsFIDgVLKHfhZaz20WmrjgrRJk5OSS/aZHKPBP5tD9sczyY0/a/mMm5fCu5EHgb398W9bH7/JC3Br18/8z3jQibUXooJMeXD2IOTnQLOe4CEz1ZxJaV36zb/MCecmaK1Tq6dKtvr27au3b9/urN1Xmbzz58k+fJjsI0fIOnyY7MNHyP7zT8uNv3Fzw6tNG7w7dcSnUye8L7sMb/dTeB79Hyr+D+MzAa2gzWBoc5XxCJDbeIpaaPljsP2/cNP70PM2Z9emzlJK7dBal5mfw56uoSxgr1LqVyC9YKHWelol6lenmXJyyDl+nOzDh8k6fITsI0fIPnyYvLOXjvLdg4Lw6dyJwNtvx7tTJ7w7dcT7sstw8/GBrFTY+TFseQxSYyHoMhi+ENBG7pXDP8GuT40NSWAQtc22/xhBYNCjEgQKycrLIjkrmeRM88P8elyXcdT3cuwUb3sCwQrzQxShtSYvIcE4uj9y1Hy0f5jsEyctN/1Wnp54dWiP/6BBRoPfsQM+nTrhERxsu8HzpyD6XSMI5KRB66tg+MvQcRgUDP72v9+Yc332IJz8XQKDqF1OxMBPT0GH64zxiDpMa016bjrJWcmcyzpn08AXfj6XdY703PRitxPVKsrhgaDMriEApZQv0EprfdihtSlBTegaMqWnk330qHGEX9C9c+QIpgsXLJ/xbN7c3Nh3xKdTR7w7dcKrdWuURxnxNm47bPy3MY9auUG3MRA21bgRh12VKxIYTm6ATCM7qAQGUWOcPwnvRYJ/MNz7G/g0dHaNyk1rzYWcC8U26JbGvtDy7Pxsm20oFAHeATTybUQjn0YE+QbRyKeR5X3h5yCfILzcvSpcX3u7huwZIxgF/B/gpbVua85EOkdrfUOFa1dO1RkIdH4+OX/9ZfTfHzlC1hGjLz83NtbyGTd/f7w7drzUl9+xI94dOuDeoIH9OzLlG7ff2/QWxG4B74bQdxL0m2LMqqgMCQyipsm+CB9cCxfi4L61xgybGiLflM/57POXGvPijtgzz1mO3PNMeTbbcFfuBPoEltqgF7wP9AnEw616JmxWZSDYAUQB0Vrr3uZle7XWPaqkpnZwVCAwBm+PkH3kcOmDt4WO8L07dsKzRfOKX6CVnWZMjdv8DqScgoDWMOAhY8qgt4NO/yQwCGcymeCrCXB4pTF987Ioh+8y15RrabxtjtqLLEvJTsGkTTbb8HTztG3MS2joG3o3xE3VvGt3qnKwOE9rnVqk4Su7P+lSRdyB7cBprfX1Sqm2GAnsgoCdGDOScuzdXkVYDd4eOWI0+MUM3np36nhp8LZjR7zbmwdvq0Lqadj6Lmz/ELJToWV/uPaf0Pl6x2dYdHODpt2Mh4wxiOq2bj4cWg7XzXN4ENh9djfPxDzDX2l/FVvu6+FrOToPrRfK5Y0vL76h921Efc/6LnNFvj2BYJ9SahzgrpTqAEzDuI2lvaYDBzHudQywAHhNa/2FUmoxcA+wqBzbs9uZ117n4po1ZJ84YTt4O3CgZbZOiYO3VeHvXUb3z/5vQZug62gYMNW4eMRZJDCI6rL/W1i3AHqNhwEPOnRXPx77kdkbZ9PErwkP9Xqo2KN3P0+5dWpx7Oka8gOeBa41L/oZ+KfW2nYUxHbdUOAj4CVgBjAKOAs001rnKaXCgNla6+tK205Fu4YSF7xMzokTVrN1vFq3dvwN1E0mOLIKNr0Np34Hr/pwxV1GoxvY2rH7rgrSlSSqQvwe+O910LQ7TFrusIvG8k35vPnHm/x333/p16wfr4S/QoCPpLCGqh0juFVrvbSsZSWs+zUwD6gPPAFMAjZrrduby1sCP2mtu5e2nZowa8guORmw+zPY9A6cOwYNQmHAA0YQqIUzJCxsAsPvkGlOvSCBQRTn4ll4P9I4C75vLdRv6pDdpOemM2v9LKLjorm90+081e8pPN0cfKBXi1TlGMHTQNFGv7hlRStwPXBGa71DKRVRsLiYjxYbiZRSU4ApAK1a1fDGJS3ByH2y/QOjgWx+BdzyX+gyGtzrQDqn4rqSzhwo1JW0UrqSxCV5OcbgcPpZuHuVw4JAbFos09ZM40TqCZ7t/yxjO491yH5cQWn3LB4OjABaKKXeLFTUALCdP2VrEHCDUmoE4GNe73UgQCnlobXOA0KBv4tbWWv9HvAeGGcEduyv+iXsNY7+9y41cqd0Hmmk0W01oG5nUXRzg2bdjceAByQwiEu0hpWPw1+bjFtN2nstTDltS9jGjOgZmLSJxdcsZkDIAIfsx1WUdrh6GmO2zw3AjkLL0zDuVlYqrfXTGGcOmM8IntBa36mUWgrcgjFzaCLwfYVq7iwmExxbbQwAH48GTz/oO9kYCAtq5+zaOUd5A0PbIdDvfiOLpahbtr5vXBk/+HHocYtDdrH0yFLmbp5LywYteSvqLVo1kIOLyiotELymtb5aKXW51vqjKtznU8AXSql/AX8AH1Thth0nNwv2fGkMACcdhvohRsrYPpPAN9DJlathygoM+7+HP/5nTJ0Nf0oCQl1xfB2smgUdh0Pkc1W++TxTHgu3LeSzQ58xqMUgFg5Z6PDUC66ixMFipdQB4EFgMTCOIv37WuudDq+dmVMHiy+eNZJkbfsPZCQZOcXDHjHSQHhU/NJvl5aZAlsWG91q2akSEOqCc8fh/Sio1xTu+RV8ynGVvR1Ss1OZuW4mm+I3cVfXu5jRZwbujr7+pg6o9KwhpdQtGHP8r8LoIipMa60df3mgmVMCwZlDsPlt2P0l5GcbRzlhU41+7rrc/1+dJCDUDVkX4INrjEkTU9ZWeRfpidQTTFszjbiLcTw/4HnGdBhTpduvy6py+ug/tNb/rLKaVUC1BQKtjX7/TW/Bn7+Bh49xx6gBDxl3XRKOIQGh9jKZ4ItxcPQXmPANtIuo0s1vPL2RJ9Y9gae7J69FvMYVTa+o0u3XdVUWCMwbCwQ6YMz+AUBrvb5SNSwHhweCvGzY+7XR/39mP/g3MZK/9b3buG2eqB4SEGqf1f+EmP8z7pfRf0qVbVZrzWeHPuPlbS/TPqA9/476N83rNa+y7buKKruOQCl1L0aaiFBgFzAA2ISRiK52yzhnzP3f+j5cTIQm3WD0O8ZsB7l1XvXzDYCIWdD/gUsB4dByCQg11b5lRhC4YqJxH+Yqkpufy0tbXmLZ0WVEtYxi3uB5khrCwezpGtoLXIlxRXAvpVRn4EWt9e3VUUFwwBlB0p9G9s9dn0FeJrQfavT/t4uU/v+aRM4Qaq6//4D/DofmveCuH6ps4sT5rPM8Fv0YOxJ3cF+P+3i498M1MqtnbVGlt6rUWmcppVBKeWutDymlOlVBHauX1nBqA2x8y8gD5O5l3CYvbCo06eLs2oniyBlCzZSWCF/cCX6N4LZPqiwIHDl/hGlrppGUmcT8wfMZ2W5klWxXlM2eQBCnlAoAvgN+VUqdp4SrgWuk/FwjA+KmtyB+t/HlDX8KrrwH6jVxdu2EPSQg1Bx52Ub6iMzzcPfPUK9xlWx27V9rmRUzC39Pfz4c9iHdg0tNPyaqWJmBQGtdMFdrtlJqLdAQWOXQWlWVzYuYvO9t48vr6QvdBkK9JlwX0pax9ZqQmZfJQ789ZLPa6PajubH9jZzPOs+M6Bk25bd3up1hbYeRkJ7A0zFP25RP7DaRiJYRnEg9wZxNc2zKp/ScQljzMA6dO8SCrQtsyqdfMZ1eTXqx68wu3tj5hk35U/2eonNQZzb9vYn39rxnU/582PO0bdiW6NhoPtpvey3gvMHzaObfjFUnVvHl4S9tyl+NeJVAn0C++/M7vv/T9sLvd4a+g6+HL18c+oKfT/5sU75k2BIAPtz3Ievi1lmVeXt4s3joYgAW717MlvgtVuUB3gG8FvkaAK/veJ3dZ3dblTf1b8r8R/fAlsUs2P9fDn1/sxHcA1qBlz+tG7Rm9sDZAMzeOJtTF05Zrd85qDNP9XsKgFkxs0hMT7Qqv7zx5Tza51EAHlv7GCnZKVbl/UP688DlDwDwwG8PkJ1nnYQ3PDScSd0nATB51WSb3811ba5jbOextfe7l26ic+wWNl33D97b/RpY/3nK/d3TWpOQkcDpi6fx8/Bj8dDFdAzqWHO/e4PnA7Bg6wIOnTtkVe6I717Bz+No5cqIprVeV/anapDE/UYAaNRerv6tSwrOELxy4K91cOFvo8/arxF4yJWmDnPhb/hzBwx5EloPgj37K7U5kzZx8sJJzmWdI8g7iDYN29DYr2rOMET52DV91NkqPFicnwvukpK2zpNBZcf7czV8egt0GmGMC7hVbgD3bMZZHl37KHuS9vBI70e4r8d9LnM3sOpUpdcROFutuR+BcC4JCI6RfMy4t0CDULjnF/CuV6nNHUg+wLQ107iQc4F5V83j6tZXV1FFRVH2BoIyw7pS6mHzBWVC1GwFXUaP7oGIp+FEDLw72JjhEr/H2bWrnbJS4fM7QLnDHZ9VOgj8fPJnJv40ETflxifDP5EgUEPYc37XDNimlPpKKTVMyfmbqOkkIFQNUz4su8+4295tH0Ngm4pvSpt4Z9c7PLHuCbo06sLnIz+nU1Dtm4VeV5UZCLTWz2Gkl/gA41aTR5VSc5VSlzm4bkJUjgSEylk9B47+DMMXQNvBFd5MRm4GT6x7gkW7F3Fj+xv5z7X/oZGvpG6pSewa8dHGQEKC+ZEHBAJfK6VedmDdhKgaEhDKb89XsOF1I9/WlfdWeDMJ6QlMWjWJ1X+t5om+TzBn4By83CV9e01jT4qJaRh3EksC/gN8p7XOVUq5AUe11g4/M5DBYlGlZFC5dKd3wJIR0KIvTPi2wlcO7zqzi0fXPkp2fjYLhixgSOiQKq6oKEuVDRYDwcBNWuvrtNZLtda5AFprE3B9JespRPWTM4SSpSUYvwf/JnDbRxUOAj8c+4G7f74bP08//jfifxIEajh7AsFK4FzBG6VUfaVUfwCt9UFHVUwIh5OAYC03y/jZsy7AHZ+Df3C5N5FvyufVHa/y7O/PckWTK/hsxGdcFiDDiTWdPYFgEXCx0Pt08zIh6gYJCEZSxuWPwuntMGaxcb/pcrqYc5Hpa6ezZN8Sbu90O4uuWUSAT4ADKiuqmj0pJpQuNJCgtTYppcqVmkKIWqEuJ7czmYzxkMwUyEoxrg8oeJ2ZAmcPwe7PIeIZ6HpDuTcfmxbLtDXTOJF6gmf7P8vYzmMd8EMIR7GnQT9uHjAuOAt4CDhe1kpKKR9gPeBt3s/XWusXlFJtgS+AIGAnMEFrnVORygvhEDU1IOTnmhvv1EsNeFaK9etiG/pUyL4AlDIxxM0Deo+HITPLXa1tCduYET0Dkzbx7jXv0j+kf8V/RuEU9swaagK8iXFHMg2sBh7VWp8pYz0F+GutLyqlPIHfMe50NgP4Rmv9hVJqMbBba11qV5Orzxq6kHOB3+N+p7FfY3o36Y2Hm5yQVauqnGWUm1mk0S7aqKeWXJ6bXvq2PXzAJ8AIZD4B4NPw0mtf8/uSyr38K3RTpqVHljJ381xaNmjJW1Fv0apBq/L/ToTD1KhcQ0opP4xA8CCwAmimtc5TSoUBs7XW15W2visGgpz8HGLiYlhxYgXrYteRYzJOmhp4NWBw6GAiQiMY1GIQ9b0k22a1KS4gXDER8rOLacBLaNTzs0vfh1f9ijXkPg3B06f0bVehPFMeC7ct5LNDn3FVi6t4ecjL8l2sgarynsU+wD1AN6xvXn+3Heu6AzuA9sDbwDEgRWudZ/5IHNCirO24CpM2sSNxByuOr+CXU7+QlpNGkE8Qt3a6lWFthpGUmcTa2LVGgDi+Ag/lQd9mfYloGUFEywha1JNfpUOV1GVUmHIzN9iFGu0GzYtp1Is29IHg3QDca/7ZXmp2Kk+se4LN8ZuZ2HUij/V5DHc3d2dXS1SCPV1DS4FDwDhgDnAncFBrPd3unRh3OPsWeB5YorVub17eElipte5RzDpTgCkArVq16nPq1KmiH6kzjp4/yvLjy1l5YiUJ6Qn4evhydaurub7d9fQP6W/TFZRvymdP0h7Wxq4lOjaaE6knAOgQ2IGI0AgiW0bSLbib3OvV0TJTIHGf0YAXNOpe9SudorkmO5F6gkfWPMLpi6d5fsDzjOkwpuyVhNNUWdeQUuoPrXVvpdQerXVPc3//z1rrqHJW6AUgA3gK6RoiIT2Bn078xPLjyzly/gjuyp2BzQcyst1IIltG4ufpZ/e2Tl04RXRsNNGx0fxx5g/ydT7BvsGEh4YT0TKC/iH98fXwdeBPI1zBhtMbmLluJp7unrwe+Tq9m/R2dpVEGaoyEGzVWvdTSq3HmDGUAGzVWrcrY73GQK7WOkUp5Qv8AizASFexrNBg8R6t9TulbauuBIILORf47dRvrDi+gm0J29Boejbuyci2I7muzXVVkogrNTuVmNMxRMdG8/vp30nPTcfb3ZuwkDAiWkYQ3jKcYN/yXygkXJfWmk8PfsrC7QtpH9Cef0f9m+b1mju7WsIOVRkI7gWWAT2AD4F6wD+01u+WsV5P4CPAHePCta+01nOUUu24NH30D2C81rrUEbTaHAiKG/Rt3aA1I9uNZGTbkQ6dZZGbn8v2xO2Ws4W/0/8GoEdwD8u4QoeADnJnKFGi3PxcXtryEsuOLiOqZRTzBs8r19mqcK4qCQTmxHK3aK2/qsrKlVdtCwQlDfoObzuc69tdT7dG3aq98dVaczTlqCUo7E3aC0Bz/+aWoNC3aV885daewuxc1jkeW/sYO8/sZErPKUztNVXGnWqZqjwjWK+1dmrGqNoSCEoa9B3ZbiQDQgbUqPn/ZzPOsj5uPdGx0WyK30R2fjb1POsxqMUgIlpGMLjFYBp6N3R2NYWTHDl/hGlrppGUmcScgXMY0W6Es6skKqAqA8E/gEzgS4w8QwBorc+VuFIVq8mBoCoHfZ0lMy+TLfFbLGcLyVnJuCt3rmh6BeGh4US2jJQLhVzI2r/WMitmFv6e/rwZ9Sbdg8ufd0jUDFUZCE4Us1iXNVhclWpaICh20De4JyPbVd2gr7OYtIl9SfuMoBAXzdHzRwFo17Ad4S2NoNAzuKfMG6+DtNZ8sO8D3tz5Jl0bdeWNyDdo6t/U2dUSlVCjriyurJoQCHLyc4g5bVzIZTXo23YkI9s5dtDXmeLS4lgXt47o2Gi2J2wnT+cR6B3IkNAhRLSMYGDzgbXirEeULjs/mxc2vsCK4ysY3mY4cwbNwcej+q5UFo5RlWcEdxW3XGv9cQXrVm7OCgQmbWJn4k6WH19uM+g7su1Iugd3d6kZN2k5aWw4vYHouGjWx60nLScNTzdP+of0J7JlJENCh9DMv5mzqynskJaTxv7k/exL2sf+pP3sPrubs5lnmdZ7Gvf2uNelvtd1WVUGgn8XeusDXA3s1FrfUrkq2q+6A8HR80dZcXwFK0+sJD49vkYP+jpLrimXXWd2ER0bzdrYtcSmxQLQJagLkS0jCW8ZTpegLtKg1ADZ+dkcOneIfUn7LI+TF05aylvVb0W34G6MajeKwaEVv0m9qHkc1jWklGoIfKK1Ln/S8gqqjkBQ3KBvWPMwrm93fa0Z9HUWrTUnUk+wNnYt6+LWsevMLjSapn5NjYvYQsPpF9IPb3dvZ1e1zss35XMs9ZhVo3/0/FHyzOm9Gvs2pltwN3oE96B7o+50C+4ms8PqMEcGAk+Mq4G7VLRy5eWoQFDSoO+IdiMY1mZYrR70dabkzGRiTsewLnYdG/7eQGZeJr4evgxqbp6aGjqYIJ8gZ1ez1tNaE3cxzqrRP3juIJl5mQDU96xPt+BudA/uTvdG3eke3F0Gf11MVXYN/cilO1q4AV0xrhKeVela2qkqA0Fpg74j2o2gdYPWVbIfYcjOz2Zr/FbLLKQzGcZtLIJ8ggitF0qL+i0IrRdKaP1QWtRrQWj9UJr6NZXut2IkZSZZNfr7k/eTkp0CgLe7N52DOhuNvrnhb9WglVwA5uKqMhCEF3qbB5zSWsdVsn7lUtlAIIO+NYPWmoPnDrLp703EpsUSdzGOuLQ4EtITyNf5ls95KA+a+TezChKFg0WAd0Cd/3ul5aRxIPkAe5P2sj9pP/uS95GQngCAm3KjfUB7q0a/fWB7PN3kqnBhrcruRwD8BcRrrbPMG/ZVSrXRWp+sZB0d7s/zf1qu9C0Y9I1qFcX17a6XQV8nUErRtVFXujbqarU8z5RHQnoCpy+eJi4tzup5bexazmVZX7vo7+lvnD0UOaMIrRdK83rNa920x+z8bA6fO2xp9Pcm7bUazG1ZvyW9m/S2dO90DuosY1aiStlzRrAdGFhwX2GllBewQWt9ZTXUD6j4GcF9v9zHtoRtMuhby2XkZljOHooLFln5WVafb+zb2KqrqSBohNYPpbFvY6deDFcwmLs/yZi6uTdpr9VgbrBvsOUov0dwDxnMFZVSlV1Du7TWvYos2621vrySdbRbRQPB8ZTjNPRuKIO+dZjWmuSsZOLS4oi7GMfptNPGszlQJGYkYtImy+c93TxpXq+5cTZROFDUNwJFA68GVVq3uItxlqP84gZzuwZ3tZrB09SvaZ3v9hLVpyq7hs4qpW7QWv9g3vBoIKmyFawO7QKqLQuGcBKlFMG+wQT7BtOrSS+b8tz8XOLT4y2BonCw2Je8j9TsVKvP1/eqb9XVVDhYNK/XHC93rxLrYjWYm2xcqFUwmOvl5kWXRl24qcNNdGtkTN+UwVxRU9hzRnAZ8ClQcCeKOOAurfWfDq6bRU1IMSHqprScNKtupti0WMv7vy/+TY4px/JZhaKJXxOrswhPN0/LoG5xg7kFjb4M5gpnqPLrCJRS9cyfT6ts5cpLAoFwBpM2cTbjrFVXU8Fz3MU4y1TYlvVbWgZyZTBX1CRV1jWklJoLvKy1TjG/DwQe11o/V/lqClFzuSk3mvo3pal/U/o07WNTnp2fTU5+DvW96juhdkJUHXs6KIcXBAEArfV5QO5SIVyet7u3BAFRJ9gTCNyVUpYkMeYb0UvSGCGEqCPsmTX0P2C1UmoJRqqJu4FqS0EthBDCscoMBFrrl5VSe4ChgAL+qbX+uaz1lFItMQJGM8AEvKe1fkMpFYRx28s2wEngNnN3kxBCCCewaxKz1nqV1voJrfXjwEWl1Nt2rJaHMajcBRgATFVKdQVmAau11h2A1eb3QgghnMSuQKCU6qWUWqCUOgn8CzhU1jpa63it9U7z6zTgINACGA18ZP7YR8CNFai3EEKIKlJi15BSqiMwFrgDSMbozlFa68jy7kQp1QboDWwBmmqt48EIFkqpJuWvthBCiKpS2hjBISAGGFVwFbFS6rHy7sB8Idoy4FGt9QV786gopaYAUwBataqbN4YXQoiaoLSuoZuBBGCtUup9pdTVGIPFdjPfzWwZ8KnW+hvz4kSlVIi5PAQ4U9y6Wuv3tNZ9tdZ9GzduXJ7dCiGEKIcSA4HW+lut9e1AZyAaeAxoqpRapJS6tqwNK+PQ/wPgoNb61UJFPwATza8nAt9XsO5CCCGqQJmDxVrrdK31p1rr64FQYBf2zfQZBEwAopRSu8yPEcB84Bql1FHgGvN7IYQQTlKuW3Rprc8B75ofZX32d0ruSrq6PPsVQgjhOJIMXQghXJwEAiGEcHESCIQQwsVJIBBCCBcngUAIIVycBAIhhHBxEgiEEMLFSSAQQggXJ4FACCFcnAQCIYRwcRIIhBDCxUkgEEIIFyeBQAghXJwEAiGEcHgDAuwAAAksSURBVHESCIQQwsVJIBBCCBcngUAIIVycBAIhhHBxEgiEEMLFSSAQQggX57BAoJT6r1LqjFJqX6FlQUqpX5VSR83PgY7avxBCCPs48ozgQ2BYkWWzgNVa6w7AavN7IYQQTuSwQKC1Xg+cK7J4NPCR+fVHwI2O2r8QQgj7VPcYQVOtdTyA+blJNe9fCCFEETV2sFgpNUUptV0ptf3s2bPOro4QQtRZ1R0IEpVSIQDm5zMlfVBr/Z7Wuq/Wum/jxo2rrYJCCOFqqjsQ/ABMNL+eCHxfzfsXQghRhCOnj34ObAI6KaXilFL3APOBa5RSR4FrzO+FEEI4kYejNqy1vqOEoqsdtU8hhBDlV2MHi4UQQlQPCQRCCOHiJBAIIYSLk0AghBAuTgKBEEK4OAkEQgjh4iQQCCGEi5NAIIQQLk4CgRBCuDgJBEII4eIkEAghhIuTQCCEEC5OAoEQQrg4CQRCCOHiJBAIIYSLk0AghBAuTgKBEEK4OAkEQgjh4iQQCCGEi5NAIIQQLs4pgUApNUwpdVgp9adSapYz6iCEEMJQ7YFAKeUOvA0MB7oCdyilulZ3PYQQQhiccUbQD/hTa31ca50DfAGMdkI9hBBCAB5O2GcLILbQ+zigvyN29OKP+/l6R5zN8kb+XjRt4INJaw4lpNmUN67nTeP63uSZNEcSbcubNvChkb8XOXkm/jx70aY8pKEPgX5eZOXmczwp3aa8RYAvDX09ycjJ42Ryhk15qyA/6nl7cDE7j7/O2Za3aeSHn5cHqZm5nE7JtClvF+yPj6c75zNyiE/Nsilv37geXh5uJKfnkHjBtrxj0/p4uCnOpmVz9mK2TXnnZvVxU4rEC1kkp+fYlHcNaQBAfGom5zNyrcrclKJzs/oAnE7JJDXTutzDTdGxqVEeey6DtOw8q3IvdzfaN6kHwKnkdNJz8q3KfT3daRvsD8CJpHQyc63L/b3cad3IKP/zzEVy8k1W5fW9PWgZ5AfAkcQ08kzaqryhryctAnwBOJSQhklblwf6eRLS0Cg/EH+BouS7J989sP+7d0ufUF4Y1c3m56xqShf5Ijt8h//f3v3GyFXVYRz/PuVPgGIloU1TU2glUSLBpMCmSIqkwSoq1fCGRKMEUCQQwPLHEPAFwltfoICJSbMtgbQo0EJsCIGKAqUJ1tICIpQ/UUm6gBYwEkoMhPr44p61m7Xtbmd29s7c+3ySSe/cOTvnd9LM/O49957fSOcD59i+pDy/AFhs+6px7S4FLi1PTwRe6bDL2cA7Hf5tv2nKWJoyDshY+lVTxtLtOBbYnjNRozrOCEaA48Y8nw+8Ob6R7ZXAym47k/SM7aFu36cfNGUsTRkHZCz9qiljma5x1HGNYCvwGUmflnQ48C1gQw1xREQENZwR2P5Y0pXAo8AhwGrbL053HBERUaljagjbDwMPT1N3XU8v9ZGmjKUp44CMpV81ZSzTMo5pv1gcERH9JSUmIiJartGJoCmlLCStlrRL0p/rjqUbko6T9LikHZJelLSi7pg6JekISX+U9HwZyy11x9QNSYdIelbSQ3XH0g1Jr0t6QdJzkp6pO55uSDpG0jpJL5fPzBk966upU0OllMWrwJepblndCnzb9ku1BtYBSWcBu4G7bZ9cdzydkjQPmGd7u6RPANuA8wb0/0TATNu7JR0GbAZW2P5DzaF1RNK1wBAwy/byuuPplKTXgSHbA7+GQNJdwFO2h8sdlkfZ/lcv+mryGUFjSlnY3gT8s+44umX7Ldvby/b7wA6qleYDx5XRpb2HlcdAHlVJmg+cCwzXHUtUJM0CzgJWAdj+qFdJAJqdCPZVymIgv3SaSNJC4BRgS72RdK5MpzwH7AJ+a3tQx/Jz4HrgPxM1HAAGNkraVqoTDKoTgLeBO8uU3bCkmb3qrMmJQPvYN5BHbE0j6WhgPXC17f8vyDMgbO+xvYhqdfxiSQM3bSdpObDL9ra6Y5kiS2yfSlXd+IoyrTqIDgVOBX5p+xTgA6Bn1zmbnAgmVcoipleZT18PrLX9QN3xTIVyyv4E8NWaQ+nEEuCbZW7918DZktbUG1LnbL9Z/t0FPEg1RTyIRoCRMWeZ66gSQ080ORGklEWfKRdYVwE7bN9adzzdkDRH0jFl+0hgGfByvVEdPNs32p5veyHVZ+T3tr9bc1gdkTSz3IRAmUb5CjCQd9rZ/juwU9KJZdeXgJ7dVFHLyuLp0KRSFpJ+BSwFZksaAX5ie1W9UXVkCXAB8EKZWwf4cVlpPmjmAXeVu9NmAPfZHuhbLxtgLvBgdbzBocA9th+pN6SuXAWsLQeyfwUu7lVHjb19NCIiJqfJU0MRETEJSQQRES2XRBAR0XJJBBERLZdEEBHRckkE0SiSdk/cqif9LjzY6rCSLpL0i17FFDFZSQQRHZDU2DU40T5JBNF4kr4haUsp3vWYpLmSZkh6TdKc0mZG+d2K2WXV8HpJW8tjSWlzs6SVkjYCdx+gv4skPSDpkdLHT8e8drGkVyU9SbXAbnT//vq8XdJNZfscSZsk5XMbUypHNdEGm4Ev2LakS4DrbV9Xaup8h6r65jLgedvvSLoH+JntzZKOp1qd/rnyXqcBZ9r+9wR9LqKqrvoh8IqkO4CPgVvKe7wHPA48W9rftp8+bwC2SnoKuB34uu0mVAmNPpJEEG0wH7i3/DDO4cDfyv7VwG+oEsH3gDvL/mXASaVUAcCs0Ro2wIZJJAGA39l+D0DSS8ACYDbwhO23y/57gc8eqE/b70v6AbAJuMb2Xw5u6BETSyKINrgDuNX2BklLgZsBbO+U9A9JZwOnU50dQDVlesb4L/zyJf3BJPv8cMz2HvZ+1vZX02WffRafB94FPjXJviMOSuYaow0+CbxRti8c99owsIaqaNyesm8jcOVoA0mLpiiOLcBSSceWctznj3ltn31KWgBcRzXN9DVJp09RLBH/k0QQTXOUpJExj2upzgDuL/Ps43/LdgNwNHunhQB+CAxJ+lOZ1rlsKgKz/VaJ5WngMWD7gfocU7b7R6XO/veBYUlHTEU8EaNSfTRaTdIQ1UXaL9YdS0Rdco0gWkvSDcDl7L02ENFKOSOIiGi5XCOIiGi5JIKIiJZLIoiIaLkkgoiIlksiiIhouSSCiIiW+y+3jF1eKJVgigAAAABJRU5ErkJggg==\n", 109 | "text/plain": [ 110 | "
" 111 | ] 112 | }, 113 | "metadata": { 114 | "needs_background": "light" 115 | }, 116 | "output_type": "display_data" 117 | } 118 | ], 119 | "source": [ 120 | "visualize_result(df)" 121 | ] 122 | }, 123 | { 124 | "cell_type": "code", 125 | "execution_count": 5, 126 | "metadata": {}, 127 | "outputs": [ 128 | { 129 | "data": { 130 | "text/html": [ 131 | "
\n", 132 | "\n", 145 | "\n", 146 | " \n", 147 | " \n", 148 | " \n", 149 | " \n", 150 | " \n", 151 | " \n", 152 | " \n", 153 | " \n", 154 | " \n", 155 | " \n", 156 | " \n", 157 | " \n", 158 | " \n", 159 | " \n", 160 | " \n", 161 | " \n", 162 | " \n", 163 | " \n", 164 | " \n", 165 | " \n", 166 | " \n", 167 | " \n", 168 | " \n", 169 | " \n", 170 | " \n", 171 | " \n", 172 | " \n", 173 | " \n", 174 | " \n", 175 | " \n", 176 | " \n", 177 | " \n", 178 | " \n", 179 | " \n", 180 | " \n", 181 | " \n", 182 | " \n", 183 | " \n", 184 | " \n", 185 | "
source_modeltarget_modelfool_methodoriginal_acc
0ResNet18DenseNet121ifgsm75.000000
1ResNet18ResNet18ifgsm69.687500
2ResNet18SqueezeNet1.0ifgsm57.187500
3ResNet18alexnetifgsm58.020833
\n", 186 | "
" 187 | ], 188 | "text/plain": [ 189 | " source_model target_model fool_method original_acc\n", 190 | "0 ResNet18 DenseNet121 ifgsm 75.000000\n", 191 | "1 ResNet18 ResNet18 ifgsm 69.687500\n", 192 | "2 ResNet18 SqueezeNet1.0 ifgsm 57.187500\n", 193 | "3 ResNet18 alexnet ifgsm 58.020833" 194 | ] 195 | }, 196 | "execution_count": 5, 197 | "metadata": {}, 198 | "output_type": "execute_result" 199 | } 200 | ], 201 | "source": [ 202 | "# Original model accuracies\n", 203 | "df.groupby(['source_model', 'target_model','fool_method'])['original_acc'].mean().reset_index()" 204 | ] 205 | }, 206 | { 207 | "cell_type": "code", 208 | "execution_count": 6, 209 | "metadata": {}, 210 | "outputs": [ 211 | { 212 | "data": { 213 | "text/html": [ 214 | "
\n", 215 | "\n", 228 | "\n", 229 | " \n", 230 | " \n", 231 | " \n", 232 | " \n", 233 | " \n", 234 | " \n", 235 | " \n", 236 | " \n", 237 | " \n", 238 | " \n", 239 | " \n", 240 | " \n", 241 | " \n", 242 | " \n", 243 | " \n", 244 | " \n", 245 | " \n", 246 | " \n", 247 | " \n", 248 | " \n", 249 | " \n", 250 | " \n", 251 | " \n", 252 | " \n", 253 | " \n", 254 | " \n", 255 | " \n", 256 | " \n", 257 | " \n", 258 | " \n", 259 | " \n", 260 | " \n", 261 | " \n", 262 | " \n", 263 | " \n", 264 | " \n", 265 | " \n", 266 | " \n", 267 | " \n", 268 | "
source_modeltarget_modelfool_methodacc_after_attack
0ResNet18DenseNet121ifgsm43.854167
1ResNet18ResNet18ifgsm0.000000
2ResNet18SqueezeNet1.0ifgsm35.416667
3ResNet18alexnetifgsm51.458333
\n", 269 | "
" 270 | ], 271 | "text/plain": [ 272 | " source_model target_model fool_method acc_after_attack\n", 273 | "0 ResNet18 DenseNet121 ifgsm 43.854167\n", 274 | "1 ResNet18 ResNet18 ifgsm 0.000000\n", 275 | "2 ResNet18 SqueezeNet1.0 ifgsm 35.416667\n", 276 | "3 ResNet18 alexnet ifgsm 51.458333" 277 | ] 278 | }, 279 | "execution_count": 6, 280 | "metadata": {}, 281 | "output_type": "execute_result" 282 | } 283 | ], 284 | "source": [ 285 | "baseline = df[df.with_ILA == False].groupby(['source_model', 'target_model','fool_method'])['acc_after_attack'].mean().reset_index()\n", 286 | "baseline" 287 | ] 288 | }, 289 | { 290 | "cell_type": "code", 291 | "execution_count": 7, 292 | "metadata": {}, 293 | "outputs": [ 294 | { 295 | "data": { 296 | "text/html": [ 297 | "
\n", 298 | "\n", 311 | "\n", 312 | " \n", 313 | " \n", 314 | " \n", 315 | " \n", 316 | " \n", 317 | " \n", 318 | " \n", 319 | " \n", 320 | " \n", 321 | " \n", 322 | " \n", 323 | " \n", 324 | " \n", 325 | " \n", 326 | " \n", 327 | " \n", 328 | " \n", 329 | " \n", 330 | " \n", 331 | " \n", 332 | " \n", 333 | " \n", 334 | " \n", 335 | " \n", 336 | " \n", 337 | " \n", 338 | " \n", 339 | " \n", 340 | " \n", 341 | " \n", 342 | " \n", 343 | " \n", 344 | " \n", 345 | " \n", 346 | " \n", 347 | " \n", 348 | " \n", 349 | " \n", 350 | " \n", 351 | " \n", 352 | " \n", 353 | " \n", 354 | " \n", 355 | " \n", 356 | " \n", 357 | " \n", 358 | " \n", 359 | " \n", 360 | " \n", 361 | " \n", 362 | " \n", 363 | " \n", 364 | " \n", 365 | " \n", 366 | " \n", 367 | " \n", 368 | " \n", 369 | " \n", 370 | " \n", 371 | " \n", 372 | " \n", 373 | " \n", 374 | " \n", 375 | " \n", 376 | " \n", 377 | " \n", 378 | " \n", 379 | " \n", 380 | " \n", 381 | " \n", 382 | " \n", 383 | " \n", 384 | " \n", 385 | " \n", 386 | " \n", 387 | " \n", 388 | " \n", 389 | " \n", 390 | " \n", 391 | " \n", 392 | " \n", 393 | " \n", 394 | " \n", 395 | " \n", 396 | " \n", 397 | " \n", 398 | " \n", 399 | " \n", 400 | " \n", 401 | " \n", 402 | " \n", 403 | " \n", 404 | " \n", 405 | " \n", 406 | " \n", 407 | " \n", 408 | " \n", 409 | " \n", 410 | " \n", 411 | " \n", 412 | " \n", 413 | " \n", 414 | " \n", 415 | " \n", 416 | " \n", 417 | " \n", 418 | " \n", 419 | " \n", 420 | " \n", 421 | " \n", 422 | " \n", 423 | " \n", 424 | " \n", 425 | " \n", 426 | " \n", 427 | " \n", 428 | " \n", 429 | " \n", 430 | " \n", 431 | " \n", 432 | " \n", 433 | " \n", 434 | " \n", 435 | " \n", 436 | " \n", 437 | " \n", 438 | " \n", 439 | " \n", 440 | " \n", 441 | " \n", 442 | " \n", 443 | " \n", 444 | " \n", 445 | " \n", 446 | " \n", 447 | " \n", 448 | " \n", 449 | " \n", 450 | " \n", 451 | " \n", 452 | " \n", 453 | " \n", 454 | " \n", 455 | " \n", 456 | " \n", 457 | " \n", 458 | " \n", 459 | " \n", 460 | " \n", 461 | " \n", 462 | " \n", 463 | " \n", 464 | " \n", 465 | " \n", 466 | " \n", 467 | " \n", 468 | " \n", 469 | " \n", 470 | " \n", 471 | " \n", 472 | " \n", 473 | " \n", 474 | " \n", 475 | " \n", 476 | " \n", 477 | " \n", 478 | " \n", 479 | " \n", 480 | " \n", 481 | " \n", 482 | " \n", 483 | " \n", 484 | " \n", 485 | " \n", 486 | " \n", 487 | " \n", 488 | " \n", 489 | " \n", 490 | " \n", 491 | " \n", 492 | " \n", 493 | " \n", 494 | " \n", 495 | " \n", 496 | " \n", 497 | " \n", 498 | " \n", 499 | " \n", 500 | " \n", 501 | " \n", 502 | " \n", 503 | " \n", 504 | " \n", 505 | " \n", 506 | " \n", 507 | " \n", 508 | " \n", 509 | " \n", 510 | " \n", 511 | " \n", 512 | " \n", 513 | " \n", 514 | " \n", 515 | " \n", 516 | " \n", 517 | " \n", 518 | " \n", 519 | " \n", 520 | " \n", 521 | " \n", 522 | " \n", 523 | " \n", 524 | " \n", 525 | " \n", 526 | " \n", 527 | " \n", 528 | " \n", 529 | " \n", 530 | " \n", 531 | " \n", 532 | " \n", 533 | " \n", 534 | " \n", 535 | " \n", 536 | " \n", 537 | " \n", 538 | " \n", 539 | " \n", 540 | " \n", 541 | " \n", 542 | " \n", 543 | " \n", 544 | " \n", 545 | " \n", 546 | " \n", 547 | " \n", 548 | "
source_modeltarget_modelfool_methodlayer_indexacc_after_attack
0ResNet18DenseNet121ifgsm0.035.416667
1ResNet18DenseNet121ifgsm1.043.229167
2ResNet18DenseNet121ifgsm2.038.854167
3ResNet18DenseNet121ifgsm3.030.104167
4ResNet18DenseNet121ifgsm4.031.250000
5ResNet18DenseNet121ifgsm5.052.604167
6ResNet18DenseNet121ifgsm6.055.104167
7ResNet18ResNet18ifgsm0.00.000000
8ResNet18ResNet18ifgsm1.00.000000
9ResNet18ResNet18ifgsm2.00.000000
10ResNet18ResNet18ifgsm3.00.000000
11ResNet18ResNet18ifgsm4.00.000000
12ResNet18ResNet18ifgsm5.00.000000
13ResNet18ResNet18ifgsm6.00.000000
14ResNet18SqueezeNet1.0ifgsm0.024.270833
15ResNet18SqueezeNet1.0ifgsm1.030.520833
16ResNet18SqueezeNet1.0ifgsm2.025.208333
17ResNet18SqueezeNet1.0ifgsm3.022.916667
18ResNet18SqueezeNet1.0ifgsm4.026.562500
19ResNet18SqueezeNet1.0ifgsm5.041.041667
20ResNet18SqueezeNet1.0ifgsm6.042.604167
21ResNet18alexnetifgsm0.040.625000
22ResNet18alexnetifgsm1.043.750000
23ResNet18alexnetifgsm2.048.333333
24ResNet18alexnetifgsm3.048.541667
25ResNet18alexnetifgsm4.049.375000
26ResNet18alexnetifgsm5.052.812500
27ResNet18alexnetifgsm6.053.229167
\n", 549 | "
" 550 | ], 551 | "text/plain": [ 552 | " source_model target_model fool_method layer_index acc_after_attack\n", 553 | "0 ResNet18 DenseNet121 ifgsm 0.0 35.416667\n", 554 | "1 ResNet18 DenseNet121 ifgsm 1.0 43.229167\n", 555 | "2 ResNet18 DenseNet121 ifgsm 2.0 38.854167\n", 556 | "3 ResNet18 DenseNet121 ifgsm 3.0 30.104167\n", 557 | "4 ResNet18 DenseNet121 ifgsm 4.0 31.250000\n", 558 | "5 ResNet18 DenseNet121 ifgsm 5.0 52.604167\n", 559 | "6 ResNet18 DenseNet121 ifgsm 6.0 55.104167\n", 560 | "7 ResNet18 ResNet18 ifgsm 0.0 0.000000\n", 561 | "8 ResNet18 ResNet18 ifgsm 1.0 0.000000\n", 562 | "9 ResNet18 ResNet18 ifgsm 2.0 0.000000\n", 563 | "10 ResNet18 ResNet18 ifgsm 3.0 0.000000\n", 564 | "11 ResNet18 ResNet18 ifgsm 4.0 0.000000\n", 565 | "12 ResNet18 ResNet18 ifgsm 5.0 0.000000\n", 566 | "13 ResNet18 ResNet18 ifgsm 6.0 0.000000\n", 567 | "14 ResNet18 SqueezeNet1.0 ifgsm 0.0 24.270833\n", 568 | "15 ResNet18 SqueezeNet1.0 ifgsm 1.0 30.520833\n", 569 | "16 ResNet18 SqueezeNet1.0 ifgsm 2.0 25.208333\n", 570 | "17 ResNet18 SqueezeNet1.0 ifgsm 3.0 22.916667\n", 571 | "18 ResNet18 SqueezeNet1.0 ifgsm 4.0 26.562500\n", 572 | "19 ResNet18 SqueezeNet1.0 ifgsm 5.0 41.041667\n", 573 | "20 ResNet18 SqueezeNet1.0 ifgsm 6.0 42.604167\n", 574 | "21 ResNet18 alexnet ifgsm 0.0 40.625000\n", 575 | "22 ResNet18 alexnet ifgsm 1.0 43.750000\n", 576 | "23 ResNet18 alexnet ifgsm 2.0 48.333333\n", 577 | "24 ResNet18 alexnet ifgsm 3.0 48.541667\n", 578 | "25 ResNet18 alexnet ifgsm 4.0 49.375000\n", 579 | "26 ResNet18 alexnet ifgsm 5.0 52.812500\n", 580 | "27 ResNet18 alexnet ifgsm 6.0 53.229167" 581 | ] 582 | }, 583 | "execution_count": 7, 584 | "metadata": {}, 585 | "output_type": "execute_result" 586 | } 587 | ], 588 | "source": [ 589 | "# ILA\n", 590 | "q = df[(df.with_ILA == True)].groupby(['source_model', 'target_model','fool_method','layer_index'])['acc_after_attack'].mean().reset_index()\n", 591 | "baseline.head()\n", 592 | "q\n" 593 | ] 594 | }, 595 | { 596 | "cell_type": "code", 597 | "execution_count": null, 598 | "metadata": {}, 599 | "outputs": [], 600 | "source": [] 601 | }, 602 | { 603 | "cell_type": "code", 604 | "execution_count": null, 605 | "metadata": {}, 606 | "outputs": [], 607 | "source": [] 608 | } 609 | ], 610 | "metadata": { 611 | "kernelspec": { 612 | "display_name": "Python 3", 613 | "language": "python", 614 | "name": "python3" 615 | }, 616 | "language_info": { 617 | "codemirror_mode": { 618 | "name": "ipython", 619 | "version": 3 620 | }, 621 | "file_extension": ".py", 622 | "mimetype": "text/x-python", 623 | "name": "python", 624 | "nbconvert_exporter": "python", 625 | "pygments_lexer": "ipython3", 626 | "version": "3.6.8" 627 | } 628 | }, 629 | "nbformat": 4, 630 | "nbformat_minor": 2 631 | } 632 | --------------------------------------------------------------------------------