├── README.md ├── download_files.py ├── experiments ├── entropy_based_mia.py └── lira_mia_offline_meta.py ├── mico_competition ├── __init__.py ├── challenge_datasets.py ├── mico.py └── scoring │ ├── __init__.py │ ├── metadata │ ├── score.py │ └── score_html.py ├── run_lira_mia.py ├── save_preds.py └── save_zip.py /README.md: -------------------------------------------------------------------------------- 1 | # Microsoft Membership Inference Competition: 2nd Place Solution 2 | 3 | ### Solution 4 | 5 | We follow the methodology used by Carlini et. al. in their work [Membership Inference Attacks From First Principles](https://arxiv.org/abs/2112.03570). To achieve this, we reproduce the code from scratch to fit in the data provided by the competition authors. Carlini et. al. summarize a few tricks which helped them achieve state of the art TPR at low FPRs. We implemented these 'tricks' but were not able to benefit from most of them. We believe that this is because of a very low number of shadow models available (more details below). 6 | 7 | The 'tricks' mentioned by Carlini et. al. are listed below. Alongside, we also state the obervations on applying / removing these. 8 | 9 | - [ ] _Logit Scaling / Stable Logit Scaling_: This scales the logit scores to help them resemble a gaussian distribution. However, since the number of `models_out` (models in 'train' phase not trained on a certain challenge point, refer paper and code for more details) were around 20 compared to the results in the paper which have 256, the gaussian could not be well formed. Note that we did not train any additional models other than those provided in the data files. 10 | - [ ] _Online Attack_: This uses both `models_out` and `models_in` to predict inference scores. On turning this feature on, our TPR was impacted negatively. We again reason this to poorly formed gaussians due to low and uneven number of `models_out` and `models_in`. 11 | - [x] _Global Variance_: This uses global variance of all challenge points for a model to fit the gaussians. This helped us improve our results as expected (global variance helps with small number of shadow models according to the paper). 12 | - [ ] _Augmentations_: Uses inference on augmented versions of challenge points to fit a multivariate gaussian. Since models trained by the challenge authors did not include random augmentations, this would not help. Our claim is supported by experiments. 13 | 14 | Further, we include some of the methods which we tried but are not part of the final solution. These methods include one using the entropy of the outputs and another using a neural network classifier which analyses the output and loss distributions to determine membership of a sample. These methods are possible directions for further research in the area. 15 | 16 | ### Steps to reproduce submitted results 17 | 18 | **For CIFAR-10 Track** 19 | 20 | | Command | Purpose | 21 | | --- | --- | 22 | | `python download_files.py --challenge 'cifar10'` | Downloads files provided by the challenge authors | 23 | | `python save_preds.py --challenge 'cifar10'` | Stores predictions of 'train' models on challenge points offline for faster membership inference predictions | 24 | | `python run_lira_mia.py --challenge 'cifar10' --global_variance`| Stores membership inference scores for all challenge points in respective 'prediction.csv' | 25 | | `python save_zip.py --challenge 'cifar10'` | Saves inference scores as zip file for submission | 26 | 27 | **For Purchase-100 Track** 28 | 29 | | Command | Purpose | 30 | | --- | --- | 31 | | `python download_files.py --challenge 'purchase100'` | Downloads files provided by the challenge authors | 32 | | `python save_preds.py --challenge 'purchase100'` | Stores predictions of 'train' models on challenge points offline for faster membership inference predictions | 33 | | `python run_lira_mia.py --challenge 'purchase100' --global_variance`| Stores membership inference scores for all challenge points in respective 'prediction.csv' | 34 | | `python save_zip.py --challenge 'purchase100'` | Saves inference scores as zip file for submission | 35 | 36 | _Note_: 37 | 38 | 1. `run_lira_mia.py` has other options: `--global_variance`, `--logit_scaling`, `--stable_logit_scaling`, `--online_attack` 39 | 2. `--logit_scaling` and `--stable_logit_scaling` are not implemented with `--online_attack` 40 | -------------------------------------------------------------------------------- /download_files.py: -------------------------------------------------------------------------------- 1 | import os 2 | import urllib 3 | import argparse 4 | 5 | from torchvision.datasets.utils import download_and_extract_archive 6 | 7 | parser = argparse.ArgumentParser() 8 | parser.add_argument('--challenge', type=str, required=True, choices=['cifar10', 'purchase100']) 9 | 10 | args = parser.parse_args() 11 | 12 | CHALLENGE = args.challenge 13 | 14 | url = "https://membershipinference.blob.core.windows.net/mico/cifar10.zip?si=cifar10&spr=https&sv=2021-06-08&sr=b&sig=d7lmXZ7SFF4ZWusbueK%2Bnssm%2BsskRXsovy2%2F5RBzylg%3D." if CHALLENGE == 'cifar10' else 'https://membershipinference.blob.core.windows.net/mico/purchase100.zip?si=purchase100&spr=https&sv=2021-06-08&sr=b&sig=YzJUTPoNndtIy0y2666XnPXS4WBF%2BbN7kbVM2soQNoU%3D' 15 | filename = "cifar10.zip" if CHALLENGE == 'cifar10' else 'purchase100.zip' 16 | md5 = "c615b172eb42aac01f3a0737540944b1" if CHALLENGE == 'cifar10' else '67eba1f88d112932fe722fef85fb95fd' 17 | 18 | try: 19 | download_and_extract_archive(url=url, download_root=os.curdir, extract_root=None, filename=filename, md5=md5, remove_finished=False) 20 | except urllib.error.HTTPError as e: 21 | print(e) 22 | print("Have you replaced the URL above with the one you got after registering?") 23 | -------------------------------------------------------------------------------- /experiments/entropy_based_mia.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import torch 3 | import csv 4 | import os 5 | import urllib 6 | import scipy 7 | 8 | from tqdm.auto import tqdm 9 | from mico_competition import ChallengeDataset, load_cifar10, load_model 10 | 11 | CHALLENGE = "cifar10" 12 | LEN_TRAINING = 50000 13 | LEN_CHALLENGE = 100 14 | 15 | scenarios = os.listdir(CHALLENGE) 16 | phases = ['train', 'dev', 'final'] 17 | 18 | dataset = load_cifar10(dataset_dir="..") 19 | 20 | criterion = torch.nn.CrossEntropyLoss(reduction='none') 21 | 22 | for scenario in tqdm(scenarios, desc="scenario"): 23 | for phase in tqdm(phases, desc="phase"): 24 | root = os.path.join(CHALLENGE, scenario, phase) 25 | for model_folder in tqdm(sorted(os.listdir(root), key=lambda d: int(d.split('_')[1])), desc="model"): 26 | path = os.path.join(root, model_folder) 27 | challenge_dataset = ChallengeDataset.from_path(path, dataset=dataset, len_training=LEN_TRAINING) 28 | challenge_points = challenge_dataset.get_challenges() 29 | 30 | # This is where you plug in your membership inference attack 31 | # As an example, here is a simple loss threshold attack 32 | 33 | # Loss Threshold Attack 34 | model = load_model('cifar10', path) 35 | challenge_dataloader = torch.utils.data.DataLoader(challenge_points, batch_size=2*LEN_CHALLENGE) 36 | features, labels = next(iter(challenge_dataloader)) 37 | output = model(features) 38 | output = torch.nn.functional.softmax(output, dim=-1) 39 | 40 | # predictions = -criterion(output, labels).detach().numpy() 41 | # print(output) 42 | predictions = -scipy.stats.entropy(output.detach().numpy(), axis=-1) 43 | # print(predictions) 44 | # Normalize to unit interval 45 | min_prediction = np.min(predictions) 46 | max_prediction = np.max(predictions) 47 | predictions = (predictions - min_prediction) / (max_prediction - min_prediction) 48 | 49 | assert np.all((0 <= predictions) & (predictions <= 1)) 50 | 51 | with open(os.path.join(path, "prediction.csv"), "w") as f: 52 | csv.writer(f).writerow(predictions) 53 | 54 | from mico_competition.scoring import tpr_at_fpr, score, generate_roc, generate_table 55 | from sklearn.metrics import roc_curve, roc_auc_score 56 | 57 | FPR_THRESHOLD = 0.1 58 | 59 | all_scores = {} 60 | phases = ['train'] 61 | 62 | for scenario in tqdm(scenarios, desc="scenario"): 63 | all_scores[scenario] = {} 64 | for phase in tqdm(phases, desc="phase"): 65 | predictions = [] 66 | solutions = [] 67 | 68 | root = os.path.join(CHALLENGE, scenario, phase) 69 | for model_folder in tqdm(sorted(os.listdir(root), key=lambda d: int(d.split('_')[1])), desc="model"): 70 | path = os.path.join(root, model_folder) 71 | predictions.append(np.loadtxt(os.path.join(path, "prediction.csv"), delimiter=",")) 72 | solutions.append(np.loadtxt(os.path.join(path, "solution.csv"), delimiter=",")) 73 | 74 | predictions = np.concatenate(predictions) 75 | solutions = np.concatenate(solutions) 76 | 77 | scores = score(solutions, predictions) 78 | all_scores[scenario][phase] = scores 79 | 80 | 81 | # import matplotlib.pyplot as plt 82 | # import matplotlib 83 | 84 | for scenario in scenarios: 85 | fpr = all_scores[scenario]['train']['fpr'] 86 | tpr = all_scores[scenario]['train']['tpr'] 87 | # fig = generate_roc(fpr, tpr) 88 | # fig.suptitle(f"{scenario}", x=-0.1, y=0.5) 89 | # fig.tight_layout(pad=1.0) 90 | 91 | 92 | import pandas as pd 93 | 94 | for scenario in scenarios: 95 | print(scenario) 96 | scores = all_scores[scenario]['train'] 97 | scores.pop('fpr', None) 98 | scores.pop('tpr', None) 99 | # display(pd.DataFrame([scores])) 100 | 101 | import zipfile 102 | 103 | phases = ['dev', 'final'] 104 | 105 | with zipfile.ZipFile("predictions_cifar10.zip", 'w') as zipf: 106 | for scenario in tqdm(scenarios, desc="scenario"): 107 | for phase in tqdm(phases, desc="phase"): 108 | root = os.path.join(CHALLENGE, scenario, phase) 109 | for model_folder in tqdm(sorted(os.listdir(root), key=lambda d: int(d.split('_')[1])), desc="model"): 110 | path = os.path.join(root, model_folder) 111 | file = os.path.join(path, "prediction.csv") 112 | if os.path.exists(file): 113 | zipf.write(file) 114 | else: 115 | raise FileNotFoundError(f"`prediction.csv` not found in {path}. You need to provide predictions for all challenges") 116 | -------------------------------------------------------------------------------- /experiments/lira_mia_offline_meta.py: -------------------------------------------------------------------------------- 1 | from scipy import special 2 | from scipy import stats 3 | import os 4 | import numpy as np 5 | import torch 6 | import csv 7 | import torch.nn as nn 8 | import zipfile 9 | from tqdm.auto import tqdm 10 | from mico_competition import ChallengeDataset, load_cifar10, load_model 11 | import math 12 | 13 | class meta_network(nn.Module): 14 | def __init__(self): 15 | super().__init__() 16 | self.bn1 = nn.BatchNorm1d(13) 17 | self.l1 = nn.Linear(13, 10) 18 | self.relu = nn.ReLU() 19 | self.bn2 = nn.BatchNorm1d(10) 20 | self.l2 = nn.Linear(10, 5) 21 | self.bn3 = nn.BatchNorm1d(5) 22 | self.l3 = nn.Linear(5, 3) 23 | self.bn4 = nn.BatchNorm1d(3) 24 | self.l4 = nn.Linear(3, 1) 25 | # self.bn5 = nn.BatchNorm1d(7) 26 | # self.l5 = nn.Linear(7, 3) 27 | # self.bn6 = nn.BatchNorm1d(3) 28 | # self.l6 = nn.Linear(3, 1) 29 | self.sigmoid = nn.Sigmoid() 30 | 31 | def forward(self, input): 32 | input = self.bn1(input) 33 | x = self.l1(input) 34 | x = self.relu(x) 35 | x = self.l2(self.bn2(x)) 36 | x = self.relu(x) 37 | x = self.l3(self.bn3(x)) 38 | x = self.relu(x) 39 | x = self.l4(self.bn4(x)) 40 | # x = self.relu(x) 41 | # x = self.l5(self.bn5(x)) 42 | # x = self.relu(x) 43 | # x = self.l6(self.bn6(x)) 44 | return self.sigmoid(x) 45 | 46 | def init_weights(m): 47 | if isinstance(m, nn.Linear): 48 | torch.nn.init.xavier_uniform(m.weight) 49 | m.bias.data.fill_(0.01) 50 | 51 | meta_net = meta_network() 52 | meta_net.apply(init_weights) 53 | meta_net.train() 54 | 55 | CHALLENGE = "cifar10" 56 | LEN_TRAINING = 50000 57 | LEN_CHALLENGE = 100 58 | 59 | dataset = load_cifar10(dataset_dir="..") 60 | criterion = torch.nn.CrossEntropyLoss(reduction='none') 61 | 62 | ###### EDITABLE ####### 63 | 64 | scenarios = ['cifar10_inf', 'cifar10_hi', 'cifar10_lo'] 65 | phases = ['train'] 66 | 67 | ####################### 68 | 69 | # store training indices for train phase 70 | 71 | from collections import defaultdict 72 | train_sets = defaultdict(dict) 73 | for scenario in tqdm(scenarios, desc="scenario"): 74 | root = os.path.join(CHALLENGE, scenario, 'train') 75 | for model_folder in tqdm(sorted(os.listdir(root), key=lambda d: int(d.split('_')[1])), desc="model"): 76 | path = os.path.join(root, model_folder) 77 | challenge_dataset = ChallengeDataset.from_path(path, dataset=dataset, len_training=LEN_TRAINING) 78 | challenge_points = challenge_dataset.get_challenges() 79 | 80 | train_sets[scenario][model_folder] = challenge_dataset.member.indices + challenge_dataset.training.indices 81 | 82 | # loading stored predictions 83 | # scenarios = ['cifar10_inf'] 84 | 85 | predictions = defaultdict(lambda: defaultdict(dict)) 86 | for scenario in tqdm(scenarios, desc="scenario"): 87 | for phase in tqdm(phases, desc="phase"): 88 | for i in range(100): 89 | path = os.path.join(f'predictions_{phase}_{scenario[8:]}', f'model_{i}.npy') 90 | predictions[phase][scenario][f'model_{i}'] = np.load(path, allow_pickle=True)[()] 91 | 92 | loss_fn = nn.BCELoss(reduction = 'mean') 93 | optimizer = torch.optim.Adam(meta_net.parameters(), lr = 0.0001) 94 | pr_out = [] 95 | num_epochs = 20 96 | for epoch in range(num_epochs): 97 | for scenario in tqdm(scenarios, desc="scenario"): 98 | for phase in tqdm(phases, desc="phase"): 99 | root = os.path.join(CHALLENGE, scenario, phase) 100 | for model_folder in tqdm(sorted(os.listdir(root), key=lambda d: int(d.split('_')[1])), desc="model"): 101 | path = os.path.join(root, model_folder) 102 | meta_labels = torch.tensor(np.loadtxt(os.path.join(path, "solution.csv"), delimiter=",")) 103 | challenge_dataset = ChallengeDataset.from_path(path, dataset=dataset, len_training=LEN_TRAINING) 104 | challenge_points = challenge_dataset.get_challenges() 105 | challenge_dataloader = torch.utils.data.DataLoader(challenge_points, batch_size=2*LEN_CHALLENGE) 106 | 107 | features, labels = next(iter(challenge_dataloader)) 108 | model_scores = load_model('cifar10', path)(features).detach().numpy() 109 | model_scores = special.softmax(model_scores, axis=-1) 110 | 111 | pr_in = [] 112 | 113 | for i, cp in tqdm(enumerate(challenge_points.indices), desc=f"challenge_points for {model_folder}"): 114 | models_out = [key for key, val in train_sets[scenario].items() if cp not in val] 115 | inp = dataset.__getitem__(cp)[0].unsqueeze(0) 116 | preds_out = np.array([predictions[phase][scenario][m][model_folder][i] for m in models_out]) 117 | preds_out = special.softmax(preds_out, axis=-1) 118 | preds_out = [p[labels[i]] for p in preds_out] 119 | preds_out = list(map(lambda x: np.log(x / (1 - x + 10e-30)), preds_out)) 120 | 121 | mean_out = np.mean(preds_out) 122 | std_out = np.std(preds_out) 123 | 124 | score = model_scores[i][labels[i]] 125 | score = np.log(score / (1 - score)) 126 | 127 | stats.norm.cdf(score, mean_out, std_out+1e-30) 128 | input = torch.Tensor(np.concatenate((np.mean(model_scores, axis=0), np.expand_dims(mean_out, axis = 0), np.expand_dims(std_out, axis = 0), np.expand_dims(score, axis = 0)))).float() 129 | pr_in.append(input) 130 | inputs = torch.stack(pr_in) 131 | preds = torch.squeeze(meta_net(inputs)) 132 | pr_out.append(preds) 133 | preds = torch.nan_to_num(preds, nan = 0.5) 134 | loss = loss_fn(preds, meta_labels.float()) 135 | print(f"[ epoch: {epoch} | loss: {loss.item()} ]") 136 | loss.backward() 137 | optimizer.step() 138 | optimizer.zero_grad() 139 | 140 | ######################################################################################################################################################################################################################################################################################################## 141 | torch.save(meta_net, 'latest_meta_net.pt') 142 | meta_net.eval() 143 | # exit() 144 | scenarios = ['cifar10_inf', 'cifar10_hi', 'cifar10_lo'] 145 | phases = ['dev', 'final'] 146 | 147 | ####################### 148 | 149 | # store training indices for train phase 150 | 151 | from collections import defaultdict 152 | train_sets = defaultdict(dict) 153 | for scenario in tqdm(scenarios, desc="scenario"): 154 | root = os.path.join(CHALLENGE, scenario, 'train') 155 | for model_folder in tqdm(sorted(os.listdir(root), key=lambda d: int(d.split('_')[1])), desc="model"): 156 | path = os.path.join(root, model_folder) 157 | challenge_dataset = ChallengeDataset.from_path(path, dataset=dataset, len_training=LEN_TRAINING) 158 | challenge_points = challenge_dataset.get_challenges() 159 | 160 | train_sets[scenario][model_folder] = challenge_dataset.member.indices + challenge_dataset.training.indices 161 | 162 | # loading stored predictions 163 | # scenarios = ['cifar10_inf'] 164 | 165 | predictions = defaultdict(lambda: defaultdict(dict)) 166 | for scenario in tqdm(scenarios, desc="scenario"): 167 | for phase in tqdm(phases, desc="phase"): 168 | for i in range(100): 169 | path = os.path.join(f'predictions_{phase}_{scenario[8:]}', f'model_{i}.npy') 170 | predictions[phase][scenario][f'model_{i}'] = np.load(path, allow_pickle=True)[()] 171 | 172 | 173 | # generating membership predictions 174 | phases = ['dev', 'final'] 175 | preds_all = [] 176 | for scenario in tqdm(scenarios, desc="scenario"): 177 | for phase in tqdm(phases, desc="phase"): 178 | root = os.path.join(CHALLENGE, scenario, phase) 179 | for model_folder in tqdm(sorted(os.listdir(root), key=lambda d: int(d.split('_')[1])), desc="model"): 180 | path = os.path.join(root, model_folder) 181 | 182 | challenge_dataset = ChallengeDataset.from_path(path, dataset=dataset, len_training=LEN_TRAINING) 183 | challenge_points = challenge_dataset.get_challenges() 184 | challenge_dataloader = torch.utils.data.DataLoader(challenge_points, batch_size=2*LEN_CHALLENGE) 185 | 186 | features, labels = next(iter(challenge_dataloader)) 187 | model_scores = load_model('cifar10', path)(features).detach().numpy() 188 | model_scores = special.softmax(model_scores, axis=-1) 189 | 190 | pr_out = [] 191 | 192 | for i, cp in tqdm(enumerate(challenge_points.indices), desc=f"challenge_points for {model_folder}"): 193 | models_out = [key for key, val in train_sets[scenario].items() if cp not in val] 194 | inp = dataset.__getitem__(cp)[0].unsqueeze(0) 195 | 196 | preds_out = np.array([predictions[phase][scenario][m][model_folder][i] for m in models_out]) 197 | preds_out = special.softmax(preds_out, axis=-1) 198 | preds_out = [p[labels[i]] for p in preds_out] 199 | preds_out = list(map(lambda x: np.log(x / (1 - x + 10e-30)), preds_out)) 200 | 201 | mean_out = np.mean(preds_out) 202 | std_out = np.std(preds_out) 203 | 204 | score = model_scores[i][labels[i]] 205 | score = np.log(score / (1 - score)) 206 | 207 | stats.norm.cdf(score, mean_out, std_out+1e-30) 208 | pr_out.append(torch.squeeze(meta_net(torch.unsqueeze(torch.Tensor(np.concatenate((np.mean(model_scores, axis=0), np.expand_dims(mean_out, axis = 0), np.expand_dims(std_out, axis = 0), np.expand_dims(score, axis = 0)))).float(), dim=0))).detach().numpy()) 209 | 210 | preds = np.array(pr_out) 211 | preds = np.nan_to_num(preds, nan = 0.5) 212 | preds_out.append(preds) 213 | for k in range(len(preds)): 214 | if preds[k] < 0.6: 215 | preds[k] = 0 216 | assert np.all((0 <= preds) & (preds <= 1)) 217 | with open(os.path.join(path, "prediction.csv"), "w") as f: 218 | csv.writer(f).writerow(preds) 219 | 220 | print(preds_out) 221 | 222 | phases = ['dev', 'final'] 223 | 224 | with zipfile.ZipFile("predictions_meta.zip", 'w') as zipf: 225 | for scenario in tqdm(scenarios, desc="scenario"): 226 | for phase in tqdm(phases, desc="phase"): 227 | root = os.path.join(CHALLENGE, scenario, phase) 228 | for model_folder in tqdm(sorted(os.listdir(root), key=lambda d: int(d.split('_')[1])), desc="model"): 229 | path = os.path.join(root, model_folder) 230 | file = os.path.join(path, "prediction.csv") 231 | if os.path.exists(file): 232 | zipf.write(file) 233 | else: 234 | raise FileNotFoundError(f"`prediction.csv` not found in {path}. You need to provide predictions for all challenges") 235 | -------------------------------------------------------------------------------- /mico_competition/__init__.py: -------------------------------------------------------------------------------- 1 | from .mico import ChallengeDataset, CNN, MLP, load_model 2 | from .challenge_datasets import load_cifar10, load_purchase100, load_sst2 3 | 4 | __all__ = [ 5 | "ChallengeDataset", 6 | "load_model", 7 | "load_cifar10", 8 | "load_purchase100", 9 | "load_sst2", 10 | "CNN", 11 | "MLP" 12 | ] -------------------------------------------------------------------------------- /mico_competition/challenge_datasets.py: -------------------------------------------------------------------------------- 1 | import os 2 | import numpy as np 3 | import torch 4 | 5 | from torch.utils.data import Dataset, ConcatDataset 6 | 7 | 8 | def load_cifar10(dataset_dir: str = ".", download=True) -> Dataset: 9 | """Loads the CIFAR10 dataset. 10 | """ 11 | from torchvision.datasets import CIFAR10 12 | import torchvision.transforms as transforms 13 | 14 | # Precomputed statistics of CIFAR10 dataset 15 | # Exact values are assumed to be known, but can be estimated with a modest privacy budget 16 | # Opacus wrongly uses CIFAR10_STD = (0.2023, 0.1994, 0.2010) 17 | # This is the _average_ std across all images (see https://github.com/kuangliu/pytorch-cifar/issues/8) 18 | CIFAR10_MEAN = (0.49139968, 0.48215841, 0.44653091) 19 | CIFAR10_STD = (0.24703223, 0.24348513, 0.26158784) 20 | 21 | transform = transforms.Compose([ 22 | transforms.ToTensor(), 23 | transforms.Normalize(CIFAR10_MEAN, CIFAR10_STD) 24 | ]) 25 | 26 | # NB: torchvision checks the integrity of downloaded files 27 | train_dataset = CIFAR10( 28 | root=f"{dataset_dir}/cifar10", 29 | train=True, 30 | download=download, 31 | transform=transform 32 | ) 33 | 34 | test_dataset = CIFAR10( 35 | root=f"{dataset_dir}/cifar10", 36 | train=False, 37 | download=download, 38 | transform=transform 39 | ) 40 | 41 | return ConcatDataset([train_dataset, test_dataset]) 42 | 43 | 44 | def load_sst2() -> Dataset: 45 | """Loads the SST2 dataset. 46 | """ 47 | import datasets 48 | 49 | # Specify cache_dir as argument? 50 | ds = datasets.load_dataset("glue", "sst2") 51 | return ConcatDataset([ds['train'], ds['validation']]) 52 | 53 | 54 | class Purchase100(Dataset): 55 | """ 56 | Purchase100 dataset pre-processed by Shokri et al. 57 | (https://github.com/privacytrustlab/datasets/blob/master/dataset_purchase.tgz). 58 | We save the dataset in a .pickle version because it is much faster to load 59 | than the original file. 60 | """ 61 | def __init__(self, dataset_dir: str) -> None: 62 | import pickle 63 | 64 | dataset_path = os.path.join(dataset_dir, 'purchase100', 'dataset_purchase') 65 | 66 | # Saving the dataset in pickle format because it is quicker to load. 67 | dataset_path_pickle = dataset_path + '.pickle' 68 | 69 | if not os.path.exists(dataset_path) and not os.path.exists(dataset_path_pickle): 70 | raise ValueError("Purchase-100 dataset not found.\n" 71 | "You may download the dataset from https://www.comp.nus.edu.sg/~reza/files/datasets.html\n" 72 | f"and unzip it in the {dataset_dir}/purchase100 directory") 73 | 74 | if not os.path.exists(dataset_path_pickle): 75 | print('Found the dataset. Saving it in a pickle file that takes less time to load...') 76 | purchase = np.loadtxt(dataset_path, dtype=int, delimiter=',') 77 | with open(dataset_path_pickle, 'wb') as f: 78 | pickle.dump({'dataset': purchase}, f) 79 | 80 | with open(dataset_path_pickle, 'rb') as f: 81 | dataset = pickle.load(f)['dataset'] 82 | 83 | self.labels = list(dataset[:, 0] - 1) 84 | self.records = torch.FloatTensor(dataset[:, 1:]) 85 | assert len(self.labels) == len(self.records), f'ERROR: {len(self.labels)} and {len(self.records)}' 86 | print('Successfully loaded the Purchase-100 dataset consisting of', 87 | f'{len(self.records)} records and {len(self.records[0])}', 'attributes.') 88 | 89 | def __len__(self) -> int: 90 | return len(self.records) 91 | 92 | def __getitem__(self, idx: int): 93 | return self.records[idx], self.labels[idx] 94 | 95 | 96 | def load_purchase100(dataset_dir: str = ".") -> Dataset: 97 | """Loads the Purchase-100 dataset. 98 | """ 99 | return Purchase100(dataset_dir) 100 | -------------------------------------------------------------------------------- /mico_competition/mico.py: -------------------------------------------------------------------------------- 1 | from __future__ import annotations 2 | 3 | import os 4 | import torch 5 | import torch.nn as nn 6 | 7 | from collections import OrderedDict 8 | from typing import List, Optional, Union, Type, TypeVar 9 | from torch.utils.data import Dataset, ConcatDataset, random_split 10 | 11 | D = TypeVar("D", bound="ChallengeDataset") 12 | 13 | LEN_CHALLENGE = 100 14 | 15 | class ChallengeDataset: 16 | """Reconstructs the data splits associated with a model from stored seeds. 17 | 18 | Given a `torch.utils.Dataset`, the desired length of the training dataset `n`, 19 | and the desired number of members/non-member challenge examples `m`, it uses 20 | `torch.utils.data.random_split` with the stored seeds to produce: 21 | 22 | - `challenge` : `2m` challenge examples 23 | - `nonmember` : `m` non-members challenge examples from `challenge` 24 | - `member` : `m` member challenge examples, from `challenge` 25 | - `training` : non-challenge examples to use for model training 26 | - `evaluation`: non-challenge examples to use for model evaluation 27 | 28 | Use `get_training_dataset` to construct the full training dataset 29 | (the concatenation of `member` and `training`) to train a model. 30 | 31 | Use `get_eval_dataset` to retrieve `evaluation`. Importantly, do not 32 | attempt to use `nonmember` for model evaluation, as releasing the 33 | evaluation results would leak membership information. 34 | 35 | The diagram below details the process, where arrows denote calls to 36 | `torch.utils.data.random_split` and `N = len(dataset)`: 37 | 38 | ┌────────────────────────────────────────────────────────────┐ 39 | │ dataset │ 40 | └──────────────────────────────┬─────────────────────────────┘ 41 | │N 42 | seed_challenge │ 43 | ┌────────────────────┴────────┐ 44 | │2m │N - 2m 45 | ▼ ▼ 46 | ┌───────────────────┬────────────────────────────────────────┐ 47 | │ challenge │ rest │ 48 | └─────────┬─────────┴───────────────────┬────────────────────┘ 49 | │2m │N - 2m 50 | seed_membership │ seed_training │ 51 | ┌────┴────┐ ┌─────────┴────────┐ 52 | │m │m │n - m │N - n - m 53 | ▼ ▼ ▼ ▼ 54 | ┌─────────┬─────────┬───────────────────┬────────────────────┐ 55 | │nonmember│ member │ training │ evaluation │ 56 | └─────────┴─────────┴───────────────────┴────────────────────┘ 57 | 58 | - Models are trained on `member + training` and evaluated on `evaluation` 59 | - Standard scenarios disclose `challenge` (equivalently, `seed_challenge`) 60 | - DP distinguisher scenarios also disclose `training` and `evaluation` (equivalently, `seed_training`) 61 | - To disclose ground truth, disclose `nonmember` and `member` (equivalently, `seed_membership`) 62 | """ 63 | def __init__(self, dataset: Dataset, len_training: int, len_challenge: int, 64 | seed_challenge: int, seed_training: Optional[int], seed_membership: Optional[int]) -> None: 65 | """Pseudorandomly select examples for `challenge`, `non-member`, `member`, `training`, and `evaluation` 66 | splits from given seeds. Only the seed for `challenge` is mandatory. 67 | 68 | Args: 69 | dataset (Dataset): Dataset to select examples from. 70 | len_training (int): Length of the training dataset. 71 | len_challenge (int): Number of challenge examples (`len_challenge` members and `len_challenge` non-members). 72 | seed_challenge (int): Seed to select challenge examples. 73 | seed_training (Optional[int]): Seed to select non-challenge training examples. 74 | seed_membership (Optional[int]): Seed to split challenge examples into members/non-members. 75 | """ 76 | from torchcsprng import create_mt19937_generator 77 | 78 | challenge_gen = create_mt19937_generator(seed_challenge) 79 | self.challenge, self.rest = random_split( 80 | dataset, 81 | [2 * len_challenge, len(dataset) - 2 * len_challenge], 82 | generator = challenge_gen) 83 | 84 | if seed_training is not None: 85 | training_gen = create_mt19937_generator(seed_training) 86 | self.training, self.evaluation = random_split( 87 | self.rest, 88 | [len_training - len_challenge, len(dataset) - len_training - len_challenge], 89 | generator = training_gen) 90 | 91 | if seed_membership is not None: 92 | membership_gen = create_mt19937_generator(seed_membership) 93 | self.nonmember, self.member = random_split( 94 | self.challenge, 95 | [len_challenge, len_challenge], 96 | generator = membership_gen) 97 | 98 | def get_challenges(self) -> Dataset: 99 | """Returns the challenge dataset. 100 | 101 | Returns: 102 | Dataset: The challenge examples. 103 | """ 104 | return self.challenge 105 | 106 | def get_train_dataset(self) -> Dataset: 107 | """Returns the training dataset. 108 | 109 | Raises: 110 | ValueError: If the seed to select non-challenge training examples has not been set. 111 | ValueError: If the seed to split challenges into members/non-members has not been set. 112 | 113 | Returns: 114 | Dataset: The training dataset. 115 | """ 116 | if self.training is None: 117 | raise ValueError("The seed to generate the training dataset has not been set.") 118 | 119 | if self.member is None: 120 | raise ValueError("The seed to split challenges into members/non-members has not been set.") 121 | 122 | return ConcatDataset([self.member, self.training]) 123 | 124 | def get_eval_dataset(self) -> Dataset: 125 | """Returns the evaluation dataset. 126 | 127 | Raises: 128 | ValueError: If the seed to generate the evaluation dataset has not been set. 129 | 130 | Returns: 131 | Dataset: The evaluation dataset. 132 | """ 133 | if self.evaluation is None: 134 | raise ValueError("The seed to generate the evaluation dataset has not been set.") 135 | 136 | return self.evaluation 137 | 138 | def get_solutions(self) -> List: 139 | """Returns the membership labels of the challenges. 140 | 141 | Raises: 142 | ValueError: If the seed to generate the evaluation dataset has not been set. 143 | 144 | Returns: 145 | List: The list of membership labels for challenges, indexed as in the 146 | Dataset returned by `get_challenges()`. 147 | """ 148 | if self.member is None: 149 | raise ValueError("The seed to split challenges into members/non-members has not been set.") 150 | 151 | member_indices = set(self.challenge.indices[i] for i in self.member.indices) 152 | 153 | labels = [1 if i in member_indices else 0 for i in self.challenge.indices] 154 | 155 | return labels 156 | 157 | @classmethod 158 | def from_path(cls: Type[D], path: Union[str, os.PathLike], dataset: Dataset, len_training: int, len_challenge: int=LEN_CHALLENGE) -> D: 159 | """Loads a ChallengeDataset from a directory `path`. 160 | The directory must contain, at a minimum, the file `seed_challenge`. 161 | 162 | Args: 163 | path (str): Path to the folder containing the dataset. 164 | 165 | Returns: 166 | ChallengeDataset: The loaded ChallengeDataset. 167 | """ 168 | # Load the seeds. 169 | if os.path.exists(os.path.join(path, "seed_challenge")): 170 | with open(os.path.join(path, "seed_challenge"), "r") as f: 171 | seed_challenge = int(f.read()) 172 | else: 173 | raise Exception(f"`seed_challenge` was not found in {path}") 174 | 175 | seed_training = None 176 | if os.path.exists(os.path.join(path, "seed_training")): 177 | with open(os.path.join(path, "seed_training"), "r") as f: 178 | seed_training = int(f.read()) 179 | 180 | seed_membership = None 181 | if os.path.exists(os.path.join(path, "seed_membership")): 182 | with open(os.path.join(path, "seed_membership"), "r") as f: 183 | seed_membership = int(f.read()) 184 | 185 | return cls( 186 | dataset=dataset, 187 | len_training=len_training, 188 | len_challenge=len_challenge, 189 | seed_challenge=seed_challenge, 190 | seed_training=seed_training, 191 | seed_membership=seed_membership 192 | ) 193 | 194 | 195 | X = TypeVar("X", bound="CNN") 196 | 197 | class CNN(nn.Module): 198 | def __init__(self): 199 | super().__init__() 200 | self.cnn = nn.Sequential( 201 | nn.Conv2d(3, 128, kernel_size=8, stride=2, padding=3), nn.Tanh(), 202 | nn.MaxPool2d(kernel_size=3, stride=1), 203 | nn.Conv2d(128, 256, kernel_size=3), nn.Tanh(), 204 | nn.Conv2d(256, 256, kernel_size=3), nn.Tanh(), 205 | nn.AvgPool2d(kernel_size=2, stride=2), 206 | nn.Flatten(), 207 | nn.Linear(in_features=6400, out_features=10) 208 | ) 209 | 210 | def forward(self, x: torch.Tensor) -> torch.Tensor: 211 | # shape of x is [B, 3, 32, 32] for CIFAR10 212 | logits = self.cnn(x) 213 | return logits 214 | 215 | @classmethod 216 | def load(cls: Type[X], path: Union[str, os.PathLike]) -> X: 217 | model = cls() 218 | state_dict = torch.load(path) 219 | new_state_dict = OrderedDict((k.replace('_module.', ''), v) for k, v in state_dict.items()) 220 | model.load_state_dict(new_state_dict) 221 | model.eval() 222 | return model 223 | 224 | 225 | Y = TypeVar("Y", bound="MLP") 226 | 227 | class MLP(nn.Module): 228 | """ 229 | The fully-connected network architecture from Bao et al. (2022). 230 | """ 231 | def __init__(self): 232 | super().__init__() 233 | self.mlp = nn.Sequential( 234 | nn.Linear(600, 128), nn.Tanh(), 235 | nn.Linear(128, 100) 236 | ) 237 | 238 | def forward(self, x: torch.Tensor) -> torch.Tensor: 239 | return self.mlp(x) 240 | 241 | @classmethod 242 | def load(cls: Type[Y], path: Union[str, os.PathLike]) -> Y: 243 | model = cls() 244 | state_dict = torch.load(path) 245 | new_state_dict = OrderedDict((k.replace('_module.', ''), v) for k, v in state_dict.items()) 246 | model.load_state_dict(new_state_dict) 247 | model.eval() 248 | return model 249 | 250 | 251 | def load_model(task: str, path: Union[str, os.PathLike]) -> nn.Module: 252 | if task == 'cifar10': 253 | return CNN.load(os.path.join(path, 'model.pt')) 254 | elif task == 'purchase100': 255 | return MLP.load(os.path.join(path, 'model.pt')) 256 | elif task == 'sst2': 257 | from transformers import AutoModelForSequenceClassification 258 | # tokenizer = AutoTokenizer.from_pretrained('roberta-base') 259 | model = AutoModelForSequenceClassification.from_pretrained(path, num_labels=2) 260 | model.eval() 261 | return model 262 | else: 263 | raise ValueError("`task` must be one of {'cifar10', 'purchase100', 'sst2'}") 264 | -------------------------------------------------------------------------------- /mico_competition/scoring/__init__.py: -------------------------------------------------------------------------------- 1 | from .score import tpr_at_fpr, score 2 | from .score_html import generate_roc, generate_table, generate_html 3 | 4 | __all__ = [ 5 | "tpr_at_fpr", 6 | "score", 7 | "generate_roc", 8 | "generate_table", 9 | "generate_html", 10 | ] -------------------------------------------------------------------------------- /mico_competition/scoring/metadata: -------------------------------------------------------------------------------- 1 | command: python $program/score.py $input $output 2 | description: Compute scores for the competition 3 | -------------------------------------------------------------------------------- /mico_competition/scoring/score.py: -------------------------------------------------------------------------------- 1 | """Scoring program for the CodaLab competition platform. 2 | 3 | Usage: 4 | score.py 5 | 6 | This program expects the following directory structure for : 7 | - /ref/: Contains the solutions directories 8 | (e.g., cifar10/cifar10_lo, cifar10/cifar10_hi, cifar10/cifar10_inf) for either the dev or the final phase. 9 | - /res/: Contains the predictions directories (containing prediction.csv) for both 10 | phases. 11 | """ 12 | import os 13 | import numpy as np 14 | 15 | from sklearn.metrics import roc_curve, roc_auc_score 16 | from typing import List, Dict 17 | 18 | FPR_THRESHOLD = 0.1 19 | FPR_THRESHOLD_LIST = [0.001, 0.01, 0.05, 0.1, 0.15, 0.2] 20 | 21 | def tpr_at_fpr(true_membership: List, predictions: List, max_fpr=FPR_THRESHOLD) -> float: 22 | """Calculates the best True Positive Rate when the False Positive Rate is 23 | at most `max_fpr`. 24 | 25 | Args: 26 | true_membership (List): A list of values in {0,1} indicating the membership of a 27 | challenge point. 0: "non-member", 1: "member". 28 | predictions (List): A list of values in the range [0,1] indicating the confidence 29 | that a challenge point is a member. The closer the value to 1, the more 30 | confident the predictor is about the hypothesis that the challenge point is 31 | a member. 32 | max_fpr (float, optional): Threshold on the FPR. Defaults to 0.1. 33 | 34 | Returns: 35 | float: The TPR @ `max_fpr` FPR. 36 | """ 37 | fpr, tpr, _ = roc_curve(true_membership, predictions) 38 | 39 | return max(tpr[fpr < max_fpr]) 40 | 41 | 42 | def score(solutions: List, predictions: List) -> Dict: 43 | scores = {} 44 | for max_fpr in FPR_THRESHOLD_LIST: 45 | scores[f"TPR_FPR_{int(1e4 * max_fpr)}"] = tpr_at_fpr(solutions, predictions, max_fpr=max_fpr) 46 | fpr, tpr, _ = roc_curve(solutions, predictions) 47 | scores["fpr"] = fpr 48 | scores["tpr"] = tpr 49 | scores["AUC"] = roc_auc_score(solutions, predictions) 50 | scores["MIA"] = np.max(tpr - fpr) 51 | # This is the balanced accuracy, which coincides with accuracy for balanced classes 52 | scores["accuracy"] = np.max(1 - (fpr + (1 - tpr)) / 2) 53 | 54 | return scores 55 | 56 | 57 | if __name__ == "__main__": 58 | from score_html import generate_html 59 | 60 | # Parse arguments. 61 | assert len(os.sys.argv) == 3, "Usage: score.py " 62 | solutions_dir = os.path.join(os.sys.argv[1], "ref") 63 | predictions_dir = os.path.join(os.sys.argv[1], "res") 64 | output_dir = os.sys.argv[2] 65 | 66 | current_phase = None 67 | 68 | # Which competition? 69 | dataset = os.listdir(solutions_dir) 70 | assert len(dataset) == 1, f"Wrong content: {solutions_dir}: {dataset}" 71 | dataset = dataset[0] 72 | print(f"[*] Competition: {dataset}") 73 | 74 | # Update solutions and predictions directories. 75 | solutions_dir = os.path.join(solutions_dir, dataset) 76 | assert os.path.exists(solutions_dir), f"Couldn't find soultions directory: {solutions_dir}" 77 | 78 | predictions_dir = os.path.join(predictions_dir, dataset) 79 | assert os.path.exists(predictions_dir), f"Couldn't find predictions directory: {predictions_dir}" 80 | 81 | scenarios = sorted(os.listdir(solutions_dir)) 82 | assert len(scenarios) == 3, f"Found spurious directories in solutions directory: {solutions_dir}: {scenarios}" 83 | 84 | found_scenarios = sorted(os.listdir(predictions_dir)) 85 | assert scenarios == found_scenarios, f"Found spurious directories in predictions directory {solutions_dir}: {found_scenarios}" 86 | 87 | # Compute the scores for each scenario 88 | all_scores = {} 89 | for scenario in scenarios: 90 | print(f"[*] Processing {scenario}...") 91 | 92 | # What phase are we in? 93 | phase = os.listdir(os.path.join(solutions_dir, scenario)) 94 | assert len(phase) == 1, "Corrupted solutions directory" 95 | assert phase[0] in ["dev", "final"], "Corrupted solutions directory" 96 | current_phase = phase[0] 97 | print(f"[**] Scoring `{current_phase}` phase...") 98 | 99 | # We compute the scores globally, across the models. This is somewhat equivalent to having 100 | # one attack (threshold) for all the attacks. 101 | # Load the predictions. 102 | predictions = [] 103 | solutions = [] 104 | for model_id in os.listdir(os.path.join(solutions_dir, scenario, current_phase)): 105 | basedir = os.path.join(scenario, current_phase, model_id) 106 | solutions.append(np.loadtxt(os.path.join(solutions_dir, basedir, "solution.csv"), delimiter=",")) 107 | predictions.append(np.loadtxt(os.path.join(predictions_dir, basedir, "prediction.csv"), delimiter=",")) 108 | 109 | solutions = np.concatenate(solutions) 110 | predictions = np.concatenate(predictions) 111 | 112 | # Verify that the predictions are valid. 113 | assert len(predictions) == len(solutions) 114 | assert np.all(predictions >= 0), "Some predictions are < 0" 115 | assert np.all(predictions <= 1), "Some predictions are > 1" 116 | 117 | scores = score(solutions, predictions) 118 | 119 | print(f"[*] Scores: {scores}") 120 | all_scores[scenario] = scores 121 | 122 | # Store the scores. 123 | os.makedirs(output_dir, exist_ok=True) 124 | with open(os.path.join(output_dir, "scores.txt"), "w") as f: 125 | for i, scenario in enumerate(scenarios): 126 | assert scenario in all_scores, f"Score for scenario {scenario} not found. Corrupted ref/?" 127 | for score in {"AUC", "MIA", "accuracy"}: 128 | f.write(f"scenario{i+1}_{score}: {all_scores[scenario][score]}\n") 129 | for max_fpr in FPR_THRESHOLD_LIST: 130 | score = f"TPR_FPR_{int(1e4 * max_fpr)}" 131 | f.write(f"scenario{i+1}_{score}: {all_scores[scenario][score]}\n") 132 | 133 | # Average TPR@0.1FPR (used for ranking) 134 | avg = np.mean([all_scores[scenario]["TPR_FPR_1000"] for scenario in scenarios]) 135 | f.write(f"average_TPR_FPR_1000: {avg}") 136 | 137 | # Detailed scoring (HTML) 138 | html = generate_html(all_scores) 139 | with open(os.path.join(output_dir, "scores.html"), "w") as f: 140 | f.write(html) 141 | -------------------------------------------------------------------------------- /mico_competition/scoring/score_html.py: -------------------------------------------------------------------------------- 1 | import io 2 | import matplotlib 3 | import pandas as pd 4 | import matplotlib.pyplot as plt 5 | 6 | 7 | def image_to_html(fig): 8 | """Converts a matplotlib plot to SVG""" 9 | iostring = io.StringIO() 10 | fig.savefig(iostring, format="svg", bbox_inches=0, dpi=300) 11 | iostring.seek(0) 12 | 13 | return iostring.read() 14 | 15 | 16 | def generate_roc(fpr, tpr): 17 | fig, (ax1, ax2) = plt.subplots(ncols=2, figsize=(8,3.5)) 18 | 19 | ax2.semilogx() 20 | ax2.semilogy() 21 | ax2.set_xlim(1e-5,1) 22 | ax2.set_ylim(1e-5,1) 23 | ax2.set_xlabel("False Positive Rate") 24 | #ax2.set_ylabel("True Positive Rate") 25 | ax2.plot([0, 1], [0, 1], ls=':', color='grey') 26 | 27 | ax1.set_xlim(0,1) 28 | ax1.set_ylim(0,1) 29 | ax1.set_xlabel("False Positive Rate") 30 | ax1.set_ylabel("True Positive Rate") 31 | ax1.plot([0,1], [0,1], ls=':', color='grey') 32 | 33 | ax1.plot(fpr, tpr) 34 | ax2.plot(fpr, tpr) 35 | 36 | return fig 37 | 38 | 39 | def generate_table(scores): 40 | table = pd.DataFrame(scores).T 41 | table.drop(["fpr", "tpr"], axis=1, inplace=True) 42 | # replace = { 43 | # "inf": "No DP", 44 | # "hi": "High ε", 45 | # "lo": "Low ε", 46 | # } 47 | # table.index = [replace[i] for i in table.index] 48 | replace_column = { 49 | "accuracy": "Accuracy", 50 | "AUC": "AUC-ROC", 51 | "MIA": "MIA", 52 | "TPR_FPR_10": "TPR @ 0.001 FPR", 53 | "TPR_FPR_100": "TPR @ 0.01 FPR", 54 | "TPR_FPR_500": "TPR @ 0.05 FPR", 55 | "TPR_FPR_1000": "TPR @ 0.1 FPR", 56 | "TPR_FPR_1500": "TPR @ 0.15 FPR", 57 | "TPR_FPR_2000": "TPR @ 0.2 FPR", 58 | } 59 | table.columns = [replace_column[c] for c in table.columns] 60 | 61 | return table 62 | 63 | 64 | def generate_html(scores): 65 | """Generates the HTML document as a string, containing the various detailed scores""" 66 | matplotlib.use('Agg') 67 | 68 | img = {} 69 | for scenario in scores: 70 | fpr = scores[scenario]["fpr"] 71 | tpr = scores[scenario]["tpr"] 72 | fig = generate_roc(fpr, tpr) 73 | fig.tight_layout(pad=1.0) 74 | 75 | img[scenario] = f"

{scenario}

{image_to_html(fig)}
" 76 | 77 | table = generate_table(scores) 78 | 79 | # Generate the HTML document. 80 | css = ''' 81 | body { 82 | background-color: #ffffff; 83 | } 84 | h1 { 85 | text-align: center; 86 | } 87 | h2 { 88 | text-align: center; 89 | } 90 | div { 91 | white-space: normal; 92 | text-align: center; 93 | } 94 | table { 95 | border-collapse: collapse; 96 | margin: auto; 97 | } 98 | table > :is(thead, tbody) > tr > :is(th, td) { 99 | padding: 5px; 100 | } 101 | table > thead > tr > :is(th, td) { 102 | border-top: 2px solid; /* \toprule */ 103 | border-bottom: 1px solid; /* \midrule */ 104 | } 105 | table > tbody > tr:last-child > :is(th, td) { 106 | border-bottom: 2px solid; /* \bottomrule */ 107 | }''' 108 | 109 | html = f''' 110 | 111 | 112 | MICO - Detailed scores 113 | 116 | 117 | 118 | 119 |
120 | {table.to_html(border=0, float_format='{:0.4f}'.format, escape=False)} 121 |
''' 122 | 123 | for scenario in scores: 124 | html += img[scenario] 125 | 126 | html += "" 127 | 128 | return html -------------------------------------------------------------------------------- /run_lira_mia.py: -------------------------------------------------------------------------------- 1 | import scipy 2 | from scipy import stats 3 | import os 4 | import numpy as np 5 | import torch 6 | import csv 7 | from torchvision import transforms 8 | from tqdm.auto import tqdm 9 | from mico_competition import ChallengeDataset, load_purchase100, load_model, load_cifar10 10 | import argparse 11 | 12 | parser = argparse.ArgumentParser() 13 | parser.add_argument('--challenge', type=str, required=True,choices=['cifar10', 'purchase100']) 14 | parser.add_argument('--logit_scaling', action='store_true') 15 | parser.add_argument('--stable_logit_scaling', action='store_true') 16 | parser.add_argument('--global_variance', action='store_true') 17 | parser.add_argument('--online_attack', action='store_true') 18 | 19 | args = parser.parse_args() 20 | 21 | CHALLENGE = args.challenge 22 | LEN_TRAINING = 50000 23 | LEN_CHALLENGE = 100 24 | 25 | dataset = load_cifar10(dataset_dir=".") if CHALLENGE == 'cifar10' else load_purchase100(dataset_dir='.') 26 | criterion = torch.nn.CrossEntropyLoss(reduction='none') 27 | 28 | scenarios = os.listdir(CHALLENGE) 29 | phases = ['dev', 'final', 'train'] 30 | 31 | ########################################## 32 | # store training indices for train phase # 33 | ########################################## 34 | 35 | from collections import defaultdict 36 | train_sets = defaultdict(dict) 37 | for scenario in tqdm(scenarios, desc="scenario"): 38 | root = os.path.join(CHALLENGE, scenario, 'train') 39 | for model_folder in tqdm(sorted(os.listdir(root), key=lambda d: int(d.split('_')[1])), desc="model"): 40 | path = os.path.join(root, model_folder) 41 | challenge_dataset = ChallengeDataset.from_path(path, dataset=dataset, len_training=LEN_TRAINING) 42 | challenge_points = challenge_dataset.get_challenges() 43 | 44 | train_sets[scenario][model_folder] = challenge_dataset.member.indices + challenge_dataset.training.indices 45 | 46 | ############################## 47 | # loading stored predictions # 48 | ############################## 49 | 50 | predictions = defaultdict(lambda: defaultdict(dict)) 51 | trans_predictions = defaultdict(lambda: defaultdict(dict)) 52 | 53 | for scenario in tqdm(scenarios, desc="scenario"): 54 | for phase in tqdm(phases, desc="phase"): 55 | for i in range(100): 56 | path = os.path.join(f'predictions_{phase}_{scenario}', f'model_{i}.npy') 57 | predictions[phase][scenario][f'model_{i}'] = np.load(path, allow_pickle=True)[()] 58 | 59 | ##################################### 60 | # generating membership predictions # 61 | ##################################### 62 | 63 | for scenario in tqdm(scenarios, desc="scenario"): 64 | for phase in tqdm(phases, desc="phase"): 65 | root = os.path.join(CHALLENGE, scenario, phase) 66 | for model_folder in tqdm(sorted(os.listdir(root), key=lambda d: int(d.split('_')[1])), desc="model"): 67 | path = os.path.join(root, model_folder) 68 | challenge_dataset = ChallengeDataset.from_path(path, dataset=dataset, len_training=LEN_TRAINING) 69 | challenge_points = challenge_dataset.get_challenges() 70 | challenge_dataloader = torch.utils.data.DataLoader(challenge_points, batch_size=2*LEN_CHALLENGE) 71 | 72 | features, labels = next(iter(challenge_dataloader)) 73 | model_scores = load_model(CHALLENGE, path)(features).detach().numpy() 74 | 75 | scores = [] 76 | means = [] 77 | predicted_scores = [] 78 | 79 | scores_in = [] 80 | means_in = [] 81 | predicted_scores_in = [] 82 | 83 | pr_out = [] 84 | 85 | for i, cp in tqdm(enumerate(challenge_points.indices), desc=f"challenge_points for {model_folder}"): 86 | models_out = [key for key, val in train_sets[scenario].items() if cp not in val] 87 | preds_out = np.array([predictions[phase][scenario][m][model_folder][i] for m in models_out]) 88 | 89 | ############################################################ 90 | # stable logit scaling / logit scaling / no transformation # 91 | ############################################################ 92 | 93 | if args.stable_logit_scaling: 94 | preds_out = preds_out - np.max(preds_out, axis=-1, keepdims=True) 95 | preds_out = np.array(np.exp(preds_out), dtype=np.float64) 96 | preds_out = preds_out / np.sum(preds_out, axis=-1, keepdims=True) 97 | 98 | y_true = np.array([p[labels[i]] for p in preds_out]) 99 | y_wrong = np.sum(preds_out, axis=-1) - y_true 100 | 101 | preds_out = np.log(y_true + 1e-45) - np.log(y_wrong + 1e-45) 102 | mean_out = np.mean(preds_out) 103 | std_out = np.std(preds_out) 104 | 105 | model_pred = model_scores[i] 106 | model_pred = model_pred - np.max(model_pred, keepdims=True) 107 | model_pred = np.array(np.exp(model_pred), dtype=np.float64) 108 | model_pred = model_pred / np.sum(model_pred, keepdims=True) 109 | 110 | model_pred_true = model_pred[labels[i]] 111 | model_pred_wrong = np.sum(model_pred, axis=-1) - model_pred_true 112 | 113 | model_pred_score = np.log(model_pred_true + 1e-45) - np.log(model_pred_wrong + 1e-45) 114 | score = model_pred_score 115 | elif args.logit_scaling: 116 | preds_out = scipy.special.softmax(preds_out, axis=-1) 117 | preds_out = [p[labels[i]] for p in preds_out] 118 | preds_out = list(map(lambda x: np.log(x / (1 - x + 10e-30)), preds_out)) 119 | mean_out = np.mean(preds_out, axis=-1) 120 | std_out = np.std(preds_out) 121 | 122 | score = model_scores[i] 123 | score = scipy.special.softmax(score)[labels[i]] 124 | score = np.log(score / (1 - score + 10e-30)) 125 | else: 126 | preds_out = scipy.special.softmax(preds_out, axis=-1) 127 | preds_out = [p[labels[i]] for p in preds_out] 128 | mean_out = np.mean(preds_out, axis=-1) 129 | std_out = np.std(preds_out) 130 | 131 | score = model_scores[i] 132 | score = scipy.special.softmax(score)[labels[i]] 133 | 134 | ################# 135 | # online attack # 136 | ################# 137 | 138 | if args.online_attack: 139 | models_in = [key for key, val in train_sets[scenario].items() if cp in val] 140 | preds_in = np.array([predictions[phase][scenario][m][model_folder][i] for m in models_in]) 141 | 142 | if args.stable_logit_scaling: 143 | raise NotImplementedError('online attack with logit scaling not implemented yet') 144 | elif args.logit_scaling: 145 | raise NotImplementedError('online attack with logit scaling not implemented yet') 146 | else: 147 | if len(models_in) == 0: 148 | mean_in = mean_out 149 | std_in = std_out 150 | else: 151 | preds_in = scipy.special.softmax(preds_in, axis=-1) 152 | preds_in = [p[labels[i]] for p in preds_in] 153 | mean_in = np.mean(preds_in, axis=-1) 154 | std_in = np.std(preds_in) 155 | 156 | score_in = model_scores[i] 157 | score_in = scipy.special.softmax(score_in)[labels[i]] 158 | 159 | 160 | ################### 161 | # global variance # 162 | ################### 163 | 164 | if args.global_variance: 165 | scores.append(score) 166 | means.append(mean_out) 167 | predicted_scores.extend(preds_out) 168 | 169 | if args.online_attack: 170 | scores_in.append(score_in) 171 | means_in.append(mean_in) 172 | predicted_scores_in.extend(preds_in) 173 | else: 174 | if args.online_attack: 175 | test_score = scipy.stats.norm.pdf(score, mean_in, std_in+1e-30) / scipy.stats.norm.pdf(score, mean_out, std_out+1e-30) 176 | else: 177 | test_score = scipy.stats.norm.cdf(score, mean_out, std_out+1e-30) 178 | 179 | pr_out.append(test_score) 180 | 181 | 182 | if args.global_variance: 183 | if args.online_attack: 184 | preds = scipy.stats.norm.pdf(scores_in, means_in, np.std(predicted_scores_in)+1e-30) / scipy.stats.norm.pdf(scores, means, np.std(predicted_scores)+1e-30) 185 | else: 186 | preds = scipy.stats.norm.cdf(scores, means, np.std(predicted_scores)+1e-30) 187 | else: 188 | preds = np.array(pr_out) 189 | 190 | if not args.online_attack: 191 | assert np.all((0 <= preds) & (preds <= 1)) 192 | 193 | with open(os.path.join(path, "prediction.csv"), "w") as f: 194 | csv.writer(f).writerow(preds) 195 | -------------------------------------------------------------------------------- /save_preds.py: -------------------------------------------------------------------------------- 1 | import os 2 | import numpy as np 3 | import torch 4 | from tqdm.auto import tqdm 5 | from mico_competition import ChallengeDataset, load_cifar10, load_purchase100, load_model 6 | import argparse 7 | 8 | parser = argparse.ArgumentParser() 9 | parser.add_argument('--challenge', type=str, required=True, choices=['cifar10', 'purchase100']) 10 | 11 | args = parser.parse_args() 12 | 13 | CHALLENGE = args.challenge 14 | LEN_TRAINING = 50000 15 | LEN_CHALLENGE = 100 16 | 17 | scenarios = os.listdir(CHALLENGE) 18 | phases = ['dev', 'final', 'train'] 19 | 20 | dataset = load_cifar10(dataset_dir=".") if CHALLENGE == 'cifar10' else load_purchase100(dataset_dir='.') 21 | criterion = torch.nn.CrossEntropyLoss(reduction='none') 22 | 23 | for scenario in tqdm(scenarios, desc="scenario"): 24 | for phase in tqdm(phases, desc="phase"): 25 | root = os.path.join(CHALLENGE, scenario, 'train') 26 | for model_folder in tqdm(sorted(os.listdir(root), key=lambda d: int(d.split('_')[1])), desc=f"train models"): 27 | path = os.path.join(root, model_folder) 28 | model = load_model(CHALLENGE, path) 29 | 30 | predictions = dict() 31 | phase_path = os.path.join(CHALLENGE, scenario, phase) 32 | 33 | for mf in tqdm(sorted(os.listdir(phase_path), key=lambda d: int(d.split('_')[1])), desc=f"challenge points in {phase}"): 34 | phase_path_model = os.path.join(phase_path, mf) 35 | 36 | challenge_dataset = ChallengeDataset.from_path(phase_path_model, dataset=dataset, len_training=LEN_TRAINING) 37 | challenge_points = challenge_dataset.get_challenges() 38 | 39 | challenge_dataloader = torch.utils.data.DataLoader( 40 | torch.utils.data.ConcatDataset(challenge_points), 41 | batch_size=2*LEN_CHALLENGE 42 | ) 43 | 44 | features, labels = next(iter(challenge_dataloader)) 45 | output = model(features).detach().numpy() 46 | 47 | predictions[mf] = output 48 | 49 | np.save(f'predictions_{phase}_{scenario}/{model_folder}', predictions) 50 | -------------------------------------------------------------------------------- /save_zip.py: -------------------------------------------------------------------------------- 1 | import zipfile 2 | import os 3 | from tqdm.auto import tqdm 4 | import argparse 5 | 6 | parser = argparse.ArgumentParser() 7 | parser.add_argument('--challenge', type=str, required=True, choices=['cifar10', 'purchase100']) 8 | 9 | args = parser.parse_args() 10 | 11 | CHALLENGE = args.challenge 12 | scenarios = os.listdir(CHALLENGE) 13 | phases = ['dev', 'final'] 14 | 15 | with zipfile.ZipFile(f"predictions_{CHALLENGE}.zip", 'w') as zipf: 16 | for scenario in tqdm(scenarios, desc="scenario"): 17 | for phase in tqdm(phases, desc="phase"): 18 | root = os.path.join(CHALLENGE, scenario, phase) 19 | for model_folder in tqdm(sorted(os.listdir(root), key=lambda d: int(d.split('_')[1])), desc="model"): 20 | path = os.path.join(root, model_folder) 21 | file = os.path.join(path, "prediction.csv") 22 | if os.path.exists(file): 23 | zipf.write(file) 24 | else: 25 | raise FileNotFoundError( 26 | f"`prediction.csv` not found in {path}. You need to provide predictions for all challenges") 27 | --------------------------------------------------------------------------------