├── figs ├── 1_7 │ ├── point.png │ ├── al_entropy.png │ ├── ep_entropy.png │ └── total_entropy.png ├── 3_8 │ ├── point.png │ ├── al_entropy.png │ ├── ep_entropy.png │ └── total_entropy.png └── 4_9 │ ├── point.png │ ├── al_entropy.png │ ├── ep_entropy.png │ └── total_entropy.png ├── README.md ├── CONTRIBUTING.md ├── requirements.txt ├── NLP └── BERT.py ├── CODE_OF_CONDUCT.md ├── MCDropout ├── dropoutNetMulti.py └── dropoutNet.py ├── xai_uncertainty.R └── LICENSE /figs/1_7/point.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/facebookresearch/infoshap/HEAD/figs/1_7/point.png -------------------------------------------------------------------------------- /figs/3_8/point.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/facebookresearch/infoshap/HEAD/figs/3_8/point.png -------------------------------------------------------------------------------- /figs/4_9/point.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/facebookresearch/infoshap/HEAD/figs/4_9/point.png -------------------------------------------------------------------------------- /figs/1_7/al_entropy.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/facebookresearch/infoshap/HEAD/figs/1_7/al_entropy.png -------------------------------------------------------------------------------- /figs/1_7/ep_entropy.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/facebookresearch/infoshap/HEAD/figs/1_7/ep_entropy.png -------------------------------------------------------------------------------- /figs/3_8/al_entropy.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/facebookresearch/infoshap/HEAD/figs/3_8/al_entropy.png -------------------------------------------------------------------------------- /figs/3_8/ep_entropy.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/facebookresearch/infoshap/HEAD/figs/3_8/ep_entropy.png -------------------------------------------------------------------------------- /figs/4_9/al_entropy.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/facebookresearch/infoshap/HEAD/figs/4_9/al_entropy.png -------------------------------------------------------------------------------- /figs/4_9/ep_entropy.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/facebookresearch/infoshap/HEAD/figs/4_9/ep_entropy.png -------------------------------------------------------------------------------- /figs/1_7/total_entropy.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/facebookresearch/infoshap/HEAD/figs/1_7/total_entropy.png -------------------------------------------------------------------------------- /figs/3_8/total_entropy.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/facebookresearch/infoshap/HEAD/figs/3_8/total_entropy.png -------------------------------------------------------------------------------- /figs/4_9/total_entropy.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/facebookresearch/infoshap/HEAD/figs/4_9/total_entropy.png -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Explaining predictive uncertainty 2 | 3 | Experiments in information theoretic Shapley values. 4 | 5 | ## References 6 | * Watson, D., O'Hara, J., Tax, N., Mudd, R., & Guy, I. (2023). Explaining predictive uncertainty with information theoretic Shapley values. In: *Advances in Neural Information Processing Systems (NeurIPS) 37*. Link [herewith](https://proceedings.neurips.cc/paper_files/paper/2023/hash/16e4be78e61a3897665fa01504e9f452-Abstract-Conference.html). 7 | 8 | ## License 9 | 10 | InfoSHAP is licensed under CC-BY-NC (see LICENSE file) 11 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | Contributing to InfoSHAP 2 | In order to accept your pull request, we need you to submit a CLA. You only need to do this once to work on any of Meta's open source projects. 3 | 4 | Complete your CLA here: https://code.facebook.com/cla 5 | 6 | InfoSHAP reproduces results of a research paper, so we welcome any bug fixes. 7 | 8 | Issues 9 | We use GitHub issues to track public bugs. Please ensure your description is clear and has sufficient instructions to be able to reproduce the issue. 10 | 11 | License 12 | By contributing to this repository, you agree that your contributions will be licensed under the LICENSE file in the root directory of this source tree. 13 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | cloudpickle==3.0.0 2 | contourpy==1.2.0 3 | cycler==0.12.1 4 | filelock==3.13.3 5 | fonttools==4.50.0 6 | fsspec==2024.3.1 7 | Jinja2==3.1.3 8 | joblib==1.3.2 9 | kiwisolver==1.4.5 10 | llvmlite==0.42.0 11 | MarkupSafe==2.1.5 12 | matplotlib==3.8.3 13 | mpmath==1.3.0 14 | networkx==3.2.1 15 | numba==0.59.1 16 | numpy==1.26.4 17 | nvidia-cublas-cu12==12.1.3.1 18 | nvidia-cuda-cupti-cu12==12.1.105 19 | nvidia-cuda-nvrtc-cu12==12.1.105 20 | nvidia-cuda-runtime-cu12==12.1.105 21 | nvidia-cudnn-cu12==8.9.2.26 22 | nvidia-cufft-cu12==11.0.2.54 23 | nvidia-curand-cu12==10.3.2.106 24 | nvidia-cusolver-cu12==11.4.5.107 25 | nvidia-cusparse-cu12==12.1.0.106 26 | nvidia-nccl-cu12==2.19.3 27 | nvidia-nvjitlink-cu12==12.4.99 28 | nvidia-nvtx-cu12==12.1.105 29 | packaging==24.0 30 | pandas==2.2.1 31 | pillow==10.2.0 32 | pyparsing==3.1.2 33 | python-dateutil==2.9.0.post0 34 | pytz==2024.1 35 | scikit-learn==1.4.1.post1 36 | scipy==1.12.0 37 | shap==0.42.0 38 | six==1.16.0 39 | slicer==0.0.7 40 | sympy==1.12 41 | threadpoolctl==3.4.0 42 | torch==2.2.1 43 | torchaudio==2.2.1 44 | torchvision==0.17.1 45 | tqdm==4.66.2 46 | triton==2.2.0 47 | typing_extensions==4.10.0 48 | tzdata==2024.1 49 | -------------------------------------------------------------------------------- /NLP/BERT.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) Meta Platforms, Inc. and affiliates. 2 | # All rights reserved. 3 | 4 | # This source code is licensed under the license found in the 5 | # LICENSE file in the root directory of this source tree. 6 | 7 | 8 | import shap 9 | import transformers 10 | import nlp 11 | import torch 12 | import numpy as np 13 | import scipy as sp 14 | import matplotlib.pyplot as plt 15 | import webbrowser 16 | 17 | # load a BERT sentiment analysis model 18 | tokenizer = transformers.DistilBertTokenizerFast.from_pretrained("distilbert-base-uncased") 19 | model = transformers.DistilBertForSequenceClassification.from_pretrained( 20 | "distilbert-base-uncased-finetuned-sst-2-english" 21 | ).cuda() 22 | 23 | def entropy(x): 24 | _x = x 25 | logp = np.log(_x) 26 | plogp = np.multiply(_x, logp) 27 | out = np.sum(plogp, axis=1) 28 | return -out 29 | 30 | 31 | def f(x): 32 | tv = torch.tensor([tokenizer.encode(v, pad_to_max_length=True, max_length=500) for v in x]).cuda() 33 | outputs = model(tv)[0].detach().cpu().numpy() 34 | scores = (np.exp(outputs).T / np.exp(outputs).sum(-1)).T 35 | val = entropy(scores) 36 | #val = sp.special.logit(scores[:,1]) # use one vs rest logit units 37 | return val 38 | 39 | imdb_train = nlp.load_dataset("imdb")["train"] 40 | 41 | background = 1000 42 | test_reviews = [] 43 | for i in range(background): 44 | if len(imdb_train[i]['text']) < 300: 45 | test_reviews.append(i) 46 | 47 | subset = imdb_train.select(test_reviews) 48 | 49 | #build an explainer using a token masker 50 | explainer = shap.Explainer(f, tokenizer) 51 | 52 | # explain the model's predictions on IMDB reviews 53 | shap_values = explainer(subset) 54 | 55 | for i in range(len(shap_values)): 56 | uncertainty = shap_values[i].base_values + np.sum(shap_values[i].values) 57 | if uncertainty > 0.5: 58 | file = open(str(i) + '.html','w') 59 | file.write(shap.plots.text(shap_values[i], display=False)) 60 | file.close 61 | -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | # Code of Conduct 2 | 3 | ## Our Pledge 4 | 5 | In the interest of fostering an open and welcoming environment, we as 6 | contributors and maintainers pledge to make participation in our project and 7 | our community a harassment-free experience for everyone, regardless of age, body 8 | size, disability, ethnicity, sex characteristics, gender identity and expression, 9 | level of experience, education, socio-economic status, nationality, personal 10 | appearance, race, religion, or sexual identity and orientation. 11 | 12 | ## Our Standards 13 | 14 | Examples of behavior that contributes to creating a positive environment 15 | include: 16 | 17 | * Using welcoming and inclusive language 18 | * Being respectful of differing viewpoints and experiences 19 | * Gracefully accepting constructive criticism 20 | * Focusing on what is best for the community 21 | * Showing empathy towards other community members 22 | 23 | Examples of unacceptable behavior by participants include: 24 | 25 | * The use of sexualized language or imagery and unwelcome sexual attention or 26 | advances 27 | * Trolling, insulting/derogatory comments, and personal or political attacks 28 | * Public or private harassment 29 | * Publishing others' private information, such as a physical or electronic 30 | address, without explicit permission 31 | * Other conduct which could reasonably be considered inappropriate in a 32 | professional setting 33 | 34 | ## Our Responsibilities 35 | 36 | Project maintainers are responsible for clarifying the standards of acceptable 37 | behavior and are expected to take appropriate and fair corrective action in 38 | response to any instances of unacceptable behavior. 39 | 40 | Project maintainers have the right and responsibility to remove, edit, or 41 | reject comments, commits, code, wiki edits, issues, and other contributions 42 | that are not aligned to this Code of Conduct, or to ban temporarily or 43 | permanently any contributor for other behaviors that they deem inappropriate, 44 | threatening, offensive, or harmful. 45 | 46 | ## Scope 47 | 48 | This Code of Conduct applies within all project spaces, and it also applies when 49 | an individual is representing the project or its community in public spaces. 50 | Examples of representing a project or community include using an official 51 | project e-mail address, posting via an official social media account, or acting 52 | as an appointed representative at an online or offline event. Representation of 53 | a project may be further defined and clarified by project maintainers. 54 | 55 | This Code of Conduct also applies outside the project spaces when there is a 56 | reasonable belief that an individual's behavior may have a negative impact on 57 | the project or its community. 58 | 59 | ## Enforcement 60 | 61 | Instances of abusive, harassing, or otherwise unacceptable behavior may be 62 | reported by contacting the project team at . All 63 | complaints will be reviewed and investigated and will result in a response that 64 | is deemed necessary and appropriate to the circumstances. The project team is 65 | obligated to maintain confidentiality with regard to the reporter of an incident. 66 | Further details of specific enforcement policies may be posted separately. 67 | 68 | Project maintainers who do not follow or enforce the Code of Conduct in good 69 | faith may face temporary or permanent repercussions as determined by other 70 | members of the project's leadership. 71 | 72 | ## Attribution 73 | 74 | This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, 75 | available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html 76 | 77 | [homepage]: https://www.contributor-covenant.org 78 | 79 | For answers to common questions about this code of conduct, see 80 | https://www.contributor-covenant.org/faq 81 | -------------------------------------------------------------------------------- /MCDropout/dropoutNetMulti.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) Meta Platforms, Inc. and affiliates. 2 | # All rights reserved. 3 | 4 | # This source code is licensed under the license found in the 5 | # LICENSE file in the root directory of this source tree. 6 | 7 | 8 | import torch 9 | from torchvision import datasets, transforms 10 | from torch import nn, optim, Tensor 11 | from torch.utils.data import DataLoader 12 | from torch.nn import functional as F 13 | from torch.optim import Optimizer 14 | import matplotlib.pyplot as plt 15 | from typing import Tuple 16 | 17 | import numpy as np 18 | import sys 19 | import shap 20 | 21 | batch_size = 128 22 | num_epochs = 40 23 | device = torch.device('cuda' if torch.cuda.is_available() else 'cpu') 24 | inference_mode = 'all' 25 | train_mode = 'test' 26 | forward_passes = 40 27 | path = "..\models\multi" + ".pth" 28 | 29 | 30 | class Net(nn.Module): 31 | def __init__(self, forward_passes: int = 20, mode: str = 'point') -> None: 32 | """Initialize the network.""" 33 | super(Net, self).__init__() 34 | 35 | self.forward_passes = forward_passes 36 | self.mode = mode 37 | 38 | # Convolutional layers 39 | self.conv_layers = nn.Sequential( 40 | nn.Conv2d(1, 10, kernel_size=5), 41 | nn.MaxPool2d(2), 42 | nn.ReLU(), 43 | nn.Dropout(p=0.3), 44 | nn.Conv2d(10, 20, kernel_size=5), 45 | nn.Dropout(), 46 | nn.MaxPool2d(2), 47 | nn.ReLU(), 48 | nn.Dropout(p=0.3), 49 | ) 50 | 51 | # Fully connected layers 52 | self.fc_layers = nn.Sequential( 53 | nn.Linear(320, 50), 54 | nn.ReLU(), 55 | nn.Dropout(), 56 | nn.Linear(50, 10), 57 | nn.Softmax(dim=1) 58 | ) 59 | 60 | def forward(self, x: torch.Tensor) -> torch.Tensor: 61 | """Forward pass through the network.""" 62 | if self.mode == 'point': 63 | x = self.conv_layers(x) 64 | x = x.view(-1, 320) 65 | x = self.fc_layers(x) 66 | 67 | elif self.mode == 'total_entropy': 68 | total_entropy = self.get_monte_carlo_predictions(x, x.size()[0], self.forward_passes, 'total_entropy') 69 | x = total_entropy 70 | 71 | elif self.mode == 'al_entropy': 72 | al_entropy = self.get_monte_carlo_predictions(x, x.size()[0], self.forward_passes, 'al_entropy') 73 | x = al_entropy 74 | 75 | elif self.mode == 'ep_entropy': 76 | total_entropy = self.get_monte_carlo_predictions(x, x.size()[0], self.forward_passes, 'total_entropy') 77 | al_entropy = self.get_monte_carlo_predictions(x, x.size()[0], self.forward_passes, 'al_entropy') 78 | ep_entropy = total_entropy - al_entropy 79 | x = ep_entropy 80 | 81 | else: 82 | x = self.get_monte_carlo_predictions(x, x.size()[0], self.forward_passes, self.mode) 83 | 84 | return x 85 | 86 | def total_entropy(self, x: torch.Tensor) -> torch.Tensor: 87 | """Calculate total entropy.""" 88 | entropy = -torch.sum(x * torch.log(x), dim=-1) 89 | return entropy[:, None] 90 | 91 | def enable_dropout(self) -> None: 92 | """Enable dropout layers during test-time.""" 93 | for m in self.modules(): 94 | if m.__class__.__name__.startswith('Dropout'): 95 | m.train() 96 | 97 | def get_monte_carlo_predictions(self, x: torch.Tensor, batch: int, forward_passes: int, mode: str) -> torch.Tensor: 98 | """Get Monte Carlo predictions.""" 99 | n_classes = 10 100 | n_samples = batch 101 | device = x.device 102 | dropout_predictions = torch.empty((0, n_samples, n_classes), device=device) 103 | 104 | for i in range(forward_passes): 105 | self.enable_dropout() 106 | conv_out = self.conv_layers(x) 107 | reshape_out = conv_out.view(-1, 320) 108 | predictions = self.fc_layers(reshape_out) 109 | dropout_predictions = torch.cat((dropout_predictions, predictions.unsqueeze(0)), dim=0) 110 | 111 | # Calculate mean across multiple MCD forward passes 112 | mean = dropout_predictions.mean(dim=0) 113 | 114 | # Calculate entropy across multiple MCD forward passes 115 | entropy = -torch.sum(dropout_predictions * torch.log(dropout_predictions), dim=-1) 116 | entropy = entropy.mean(dim=0) 117 | 118 | if mode == 'al_entropy': 119 | return entropy[:, None] 120 | 121 | if mode == 'total_entropy': 122 | return self.total_entropy(mean) 123 | 124 | if mode == "mean": 125 | return mean 126 | 127 | 128 | def train(model: torch.nn.Module, 129 | device: torch.device, 130 | train_loader: DataLoader, 131 | optimizer: optim.Optimizer, 132 | epoch: int) -> None: 133 | 134 | model.train() 135 | total = 0 136 | correct = 0 137 | 138 | for batch_idx, (data, target) in enumerate(train_loader): 139 | data, target = data.to(device), target.to(device) 140 | optimizer.zero_grad() 141 | output = model(data) 142 | loss = F.nll_loss(output.log(), target) 143 | loss.backward() 144 | optimizer.step() 145 | 146 | # Calculate number of correctly classified samples in the current batch 147 | pred = output.argmax(dim=1, keepdim=True) 148 | correct += pred.eq(target.view_as(pred)).sum().item() 149 | total += data.size(0) 150 | 151 | if batch_idx % 100 == 0: 152 | print(f'Train Epoch: {epoch} ' 153 | f'Loss: {loss.item():.6f} Correct: {correct}/{total}') 154 | 155 | def create_shap_plots(model: torch.nn.Module, 156 | background: Tensor, 157 | test_images: Tensor, 158 | mode: str) -> None: 159 | """ 160 | Create SHAP plots for the model predictions. 161 | 162 | Parameters: 163 | - model: The trained neural network model. 164 | - background: Background data for the SHAP explainer. 165 | - test_images: Images to be explained. 166 | - mode: The mode for SHAP plotting. 167 | 168 | Returns: 169 | - None 170 | """ 171 | explainer = shap.DeepExplainer(model, background) 172 | shap_values = np.asarray(explainer.shap_values(test_images)) 173 | 174 | test_images_cpu = test_images.to(torch.device('cpu')) 175 | 176 | plt.ioff() 177 | 178 | if mode in ['al_entropy', 'total_entropy', 'ep_entropy']: 179 | shap_entropy_numpy = np.swapaxes(np.swapaxes(shap_values, 1, -1), 1, 2) 180 | test_entropy_numpy = np.swapaxes(np.swapaxes(np.asarray(test_images_cpu), 1, -1), 1, 2) 181 | shap.image_plot(shap_entropy_numpy, -test_entropy_numpy, show=False) 182 | fig = plt.gcf() 183 | fig.savefig(f"../figs/{mode}.png") 184 | plt.close(fig) 185 | 186 | elif mode in ['mean', 'point']: 187 | shap_numpy = [np.swapaxes(np.swapaxes(s, 1, -1), 1, 2) for s in shap_values] 188 | test_numpy = np.swapaxes(np.swapaxes(test_images_cpu.numpy(), 1, -1), 1, 2) 189 | shap.image_plot(shap_numpy, -test_numpy, show=False) 190 | fig = plt.gcf() 191 | fig.savefig(f"../figs/{mode}.png") 192 | plt.close(fig) 193 | 194 | plt.ion() 195 | 196 | def create_dataset(x: int = 1, y: int = 7) -> Tuple[DataLoader, DataLoader, str]: 197 | """ 198 | Create a dataset for training and testing. 199 | 200 | Parameters: 201 | - x: Label 1 for the dataset. 202 | - y: Label 2 for the dataset. 203 | 204 | Returns: 205 | - Tuple containing train_loader, test_loader, and model path. 206 | """ 207 | train_dataset = datasets.MNIST('mnist_data', train=True, download=True, transform=transforms.Compose([ 208 | transforms.ToTensor() 209 | ])) 210 | train_idx1 = torch.tensor(train_dataset.targets) == x 211 | train_idx2 = torch.tensor(train_dataset.targets) == y 212 | 213 | train_indices_1 = train_idx1.nonzero().reshape(-1) 214 | train_indices_2 = train_idx2.nonzero().reshape(-1) 215 | 216 | for i in train_indices_1: 217 | train_dataset.targets[i] = 0 218 | for j in train_indices_2: 219 | train_dataset.targets[j] = 1 220 | 221 | train_mask = train_idx1 | train_idx2 222 | train_indices = train_mask.nonzero().reshape(-1) 223 | train_subset = torch.utils.data.Subset(train_dataset, train_indices) 224 | train_loader = torch.utils.data.DataLoader(train_subset, batch_size=batch_size, shuffle=True) 225 | 226 | test_dataset = datasets.MNIST('mnist_data', train=False, transform=transforms.Compose([ 227 | transforms.ToTensor() 228 | ])) 229 | test_idx1 = torch.tensor(test_dataset.targets) == x 230 | test_idx2 = torch.tensor(test_dataset.targets) == y 231 | 232 | test_indices_1 = test_idx1.nonzero().reshape(-1) 233 | test_indices_2 = test_idx2.nonzero().reshape(-1) 234 | 235 | for i in test_indices_1: 236 | test_dataset.targets[i] = 0 237 | for j in test_indices_2: 238 | test_dataset.targets[j] = 1 239 | 240 | test_mask = test_idx1 | test_idx2 241 | test_indices = test_mask.nonzero().reshape(-1) 242 | test_subset = torch.utils.data.Subset(test_dataset, test_indices) 243 | test_loader = torch.utils.data.DataLoader(test_subset, batch_size=batch_size, shuffle=True) 244 | path = "../models/MCDropout" + str(x) + str(y) + ".pth" 245 | return train_loader, test_loader, path 246 | 247 | 248 | if __name__ == '__main__': 249 | path = "~/infoshap/models/multi.pth" 250 | 251 | train_dataset = datasets.MNIST('mnist_data', train=True, download=True, transform=transforms.Compose([ 252 | transforms.ToTensor() 253 | ])) 254 | test_dataset = datasets.MNIST('mnist_data', train=False, transform=transforms.Compose([ 255 | transforms.ToTensor() 256 | ])) 257 | train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=10000, shuffle=True) 258 | test_loader = torch.utils.data.DataLoader(test_dataset, batch_size=10000, shuffle=True) 259 | 260 | ################################################################################# 261 | 262 | trainPointModel = Net(mode="point").to(device) 263 | # optimizer = optim.SGD(trainPointModel.parameters(), lr=0.01, momentum=0.5) 264 | optimizer = optim.Adam(trainPointModel.parameters(), lr=0.001) 265 | 266 | batch = next(iter(test_loader)) 267 | images, _ = batch 268 | 269 | background = images[0:1000].to(device) 270 | test_images = images[0:5] 271 | 272 | if train_mode == 'train': 273 | for epoch in range(1, num_epochs + 1): 274 | train(trainPointModel, device, train_loader, optimizer, epoch) 275 | torch.save(trainPointModel.state_dict(), path) 276 | 277 | if train_mode == 'test': 278 | if inference_mode == 'point': 279 | #Initialise model(s) 280 | pointModel = Net(forward_passes=forward_passes, mode=inference_mode).to(device) 281 | pointModel.load_state_dict(torch.load(path)) 282 | out = pointModel(test_images.to(device)) 283 | 284 | create_shap_plots(pointModel, background, test_images, inference_mode) 285 | 286 | elif inference_mode == 'mean': 287 | #Initialise model(s) 288 | meanModel = Net(forward_passes=forward_passes, mode=inference_mode).to(device) 289 | meanModel.load_state_dict(torch.load(path)) 290 | out = meanModel(test_images.to(device)) 291 | 292 | create_shap_plots(meanModel, background, test_images, inference_mode) 293 | 294 | elif inference_mode == 'al_entropy': 295 | #Initialise model(s) 296 | alEntropyModel = Net(forward_passes=forward_passes, mode=inference_mode).to(device) 297 | alEntropyModel.load_state_dict(torch.load(path)) 298 | 299 | entropies = alEntropyModel(background) 300 | mean_entropy = torch.mean(entropies) 301 | max_entropy = torch.topk(entropies.flatten(), 5) 302 | indices_cpu = max_entropy.indices.to(torch.device('cpu')) 303 | test_images = images[indices_cpu].to(device) 304 | 305 | create_shap_plots(alEntropyModel, background, test_images, inference_mode) 306 | 307 | elif inference_mode == 'total_entropy': 308 | #Initialise model(s) 309 | totalEntropyModel = Net(forward_passes=forward_passes, mode=inference_mode).to(device) 310 | totalEntropyModel.load_state_dict(torch.load(path)) 311 | 312 | entropies = totalEntropyModel(background) 313 | mean_entropy = torch.mean(entropies) 314 | max_entropy = torch.topk(entropies.flatten(), 5) 315 | indices_cpu = max_entropy.indices.to(torch.device('cpu')) 316 | test_images = images[indices_cpu].to(device) 317 | 318 | create_shap_plots(totalEntropyModel, background, test_images, inference_mode) 319 | 320 | elif inference_mode == 'all': 321 | #Initialise model(s) 322 | totalEntropyModel = Net(forward_passes=forward_passes, mode='total_entropy').to(device) 323 | totalEntropyModel.load_state_dict(torch.load(path)) 324 | alEntropyModel = Net(forward_passes=forward_passes, mode='al_entropy').to(device) 325 | alEntropyModel.load_state_dict(torch.load(path)) 326 | epEntropyModel = Net(forward_passes=forward_passes, mode='ep_entropy').to(device) 327 | epEntropyModel.load_state_dict(torch.load(path)) 328 | pointModel = Net(forward_passes=forward_passes, mode='point').to(device) 329 | pointModel.load_state_dict(torch.load(path)) 330 | 331 | #Filter subsample for high entropy 332 | entropies = totalEntropyModel(background) 333 | mean_entropy = torch.mean(entropies) 334 | 335 | max_entropy = torch.topk(entropies.flatten(), 5) 336 | indices_cpu = max_entropy.indices.to(torch.device('cpu')) 337 | test_images = images[indices_cpu].to(device) 338 | 339 | create_shap_plots(totalEntropyModel, background, test_images, 'total_entropy') 340 | create_shap_plots(alEntropyModel, background, test_images, 'al_entropy') 341 | create_shap_plots(epEntropyModel, background, test_images, 'ep_entropy') 342 | create_shap_plots(pointModel, background, test_images, 'point') 343 | 344 | else: 345 | print("Invalid inference mode selected!") 346 | 347 | print("DONE") 348 | 349 | -------------------------------------------------------------------------------- /xai_uncertainty.R: -------------------------------------------------------------------------------- 1 | # Copyright (c) Meta Platforms, Inc. and affiliates. 2 | # All rights reserved. 3 | 4 | # This source code is licensed under the license found in the 5 | # LICENSE file in the root directory of this source tree. 6 | 7 | 8 | # Load libraries, register cores 9 | library(data.table) 10 | library(mlbench) 11 | library(xgboost) 12 | library(precrec) 13 | library(Rfast) 14 | library(ggplot2) 15 | library(viridis) 16 | library(doMC) 17 | registerDoMC(8) 18 | 19 | # Set seed 20 | set.seed(123) 21 | 22 | ### Friedman benchmark ### 23 | 24 | # Simulate data 25 | n <- 2000 26 | dat1 <- mlbench.friedman1(n) 27 | dat2 <- mlbench.friedman1(n) 28 | dat3 <- mlbench.friedman1(n/2) 29 | x1 <- dat1$x 30 | x2 <- dat2$x 31 | x3 <- dat3$x 32 | colnames(x1) <- colnames(x2) <- colnames(x3) <- paste0('x', 1:10) 33 | y1 <- dat1$y 34 | y2 <- dat2$y 35 | y3 <- dat3$y 36 | y1 <- scale(y1)^2 37 | y2 <- scale(y2)^2 38 | y3 <- scale(y3)^2 39 | z_fn <- function(x, y) { 40 | 10 * sin(pi * x[, 6] * x[, 7]) + 20 * (x[, 8] - 1/2)^2 + 10 * x[, 9] + 41 | 5 * x[, 10] + rnorm(nrow(x), sd = sqrt(y)) 42 | } 43 | z1 <- z_fn(x1, y1) 44 | z2 <- z_fn(x2, y2) 45 | z3 <- z_fn(x3, y3) 46 | 47 | # Train XGBoost model for conditional mean, compute Shapley values on test set 48 | f <- xgboost(data = x1, label = z1, nrounds = 25, verbose = 0) 49 | phi_f <- predict(f, x3, predcontrib = TRUE)[, 1:10] 50 | colnames(phi_f) <- paste0('x', 1:10) 51 | 52 | # Train XGBoost model for conditional variance, compute Shapley values on test set 53 | eps <- z2 - predict(f, x2) 54 | #h <- xgboost(data = x2, label = log(eps^2), nrounds = 25) 55 | h <- xgboost(data = x2, label = y2, nrounds = 25, verbose = 0) 56 | phi_h <- predict(h, x3, predcontrib = TRUE)[, 1:10] 57 | colnames(phi_h) <- paste0('x', 1:10) 58 | 59 | # Plot results 60 | df <- melt(data.table(x3), measure.vars = 1:10, variable.name = 'feature') 61 | tmp1 <- melt(data.table(phi_f), measure.vars = 1:10, variable.name = 'feature', 62 | value.name = 'phi')[, 'moment' := 'Mean'][, feature := NULL] 63 | tmp2 <- melt(data.table(phi_h), measure.vars = 1:10, variable.name = 'feature', 64 | value.name = 'phi')[, 'moment' := 'Variance'][, feature := NULL] 65 | df <- rbind(cbind(df, tmp1), cbind(df, tmp2)) 66 | ggplot(df, aes(phi, feature, color = value)) + 67 | geom_jitter(size = .5, width = 0, height = 0.1, alpha = 0.25) + 68 | geom_vline(xintercept = 0, color = 'red', linetype = 'dashed') + 69 | scale_color_viridis('Feature\nValue', option = 'C') + 70 | labs(x = 'Shapley Value', y = 'Feature') + 71 | theme_bw() + 72 | theme(text = element_text(size = 8), legend.key.size = unit(0.4, 'cm')) + 73 | facet_wrap(~ moment, scales = 'free_x') 74 | ggsave('friedman.pdf', width = 5, height = 3) 75 | 76 | 77 | # Coverage 78 | tst_n <- 1000 79 | alpha <- 0.1 80 | inner_loop <- function(b, j, moment) { 81 | if (moment == 1) { 82 | phi <- phi_f 83 | } else if (moment == 2) { 84 | phi <- phi_h 85 | } 86 | i <- sample(tst_n, 1) 87 | s <- phi[i, j] 88 | x <- phi[-i, j] 89 | #mu <- mean(x) 90 | #R <- abs(x - mu) 91 | #q <- ceiling(tst_n * (1 - alpha)) 92 | #r <- sort(R)[q] 93 | #lo <- mu - r 94 | #hi <- mu + r 95 | q_lo <- floor(tst_n * alpha / 2) 96 | q_hi <- ceiling(tst_n * (1 - alpha / 2)) 97 | lo <- sort(x)[q_lo] 98 | hi <- sort(x)[q_hi] 99 | cvg <- 1 - ((sum(x < lo) + sum(x > hi)) / (tst_n - 1)) 100 | data.table( 101 | 'Feature' = paste0('x', j), 102 | 'Moment' = moment, 103 | 'Covered' = ifelse(s >= lo & s <= hi, 1, 0), 104 | 'hi' = hi, 'lo' = lo 105 | ) 106 | } 107 | outer_loop <- function(m) { 108 | df <- foreach(bb = 1:1000, .combine = rbind) %:% 109 | foreach(jj = 1:10, .combine = rbind) %:% 110 | foreach(mm = c(1, 2), .combine = rbind) %do% inner_loop(bb, jj, mm) 111 | df[, Cvg := mean(Covered), by = .(Feature, Moment)] 112 | df[, hi_avg := mean(hi), by = .(Feature, Moment)] 113 | df[, lo_avg := mean(lo), by = .(Feature, Moment)] 114 | unique(df[, .(Feature, Moment, Cvg, hi_avg, lo_avg)]) 115 | } 116 | df <- foreach(mm = 1:20, .combine = rbind) %dopar% 117 | outer_loop(mm) 118 | 119 | df1 <- df[Moment == 1, mean(Cvg), by = Feature] 120 | setnames(df1, 'V1', 'Coverage') 121 | df1[, hi := df[Moment == 1, mean(hi_avg), by = Feature]$V1] 122 | df1[, lo := df[Moment == 1, mean(lo_avg), by = Feature]$V1] 123 | df2 <- df[Moment == 2, mean(Cvg), by = Feature] 124 | setnames(df2, 'V1', 'Coverage') 125 | df2[, hi := df[Moment == 2, mean(hi_avg), by = Feature]$V1] 126 | df2[, lo := df[Moment == 2, mean(lo_avg), by = Feature]$V1] 127 | 128 | 129 | # Feature value acquisition 130 | zero_out <- function(x, frac = 0.1) { 131 | n <- nrow(x) 132 | out <- foreach(j = 1:10, .combine = cbind) %do% { 133 | tmp <- x[, j] 134 | tmp[sample(n, frac * n)] <- NA_real_ 135 | return(tmp) 136 | } 137 | colnames(out) <- colnames(x) 138 | return(out) 139 | } 140 | df <- foreach(ff = c(0, 0.05, 0.1, 0.2, 0.5), .combine = rbind) %dopar% { 141 | x2_tilde <- zero_out(x2, ff) 142 | h <- xgboost(data = x2_tilde, label = y2, nrounds = 10, verbose = 0) 143 | x3_tilde <- zero_out(x3) 144 | phi_h <- predict(h, x3_tilde, predcontrib = TRUE)[, 1:10] 145 | phi <- foreach(i = 1:nrow(phi_h), .combine = rbind) %do% { 146 | phi_h[i, ] / sum(phi_h[i, ]) 147 | } 148 | df <- rbindlist( 149 | lapply(1:10, function(j) { 150 | na_idx <- is.na(x3_tilde[, j]) 151 | data.table('feature' = paste0('x', j), 152 | 'phi' = abs(phi[na_idx, j]), 153 | 'impt' = ifelse(j <= 5L, 1L, 0L)) 154 | }) 155 | ) 156 | rocs <- evalmod(scores = df$phi, labels = df$impt)$rocs 157 | out <- data.table(FPR = rocs[[1]]$x, TPR = rocs[[1]]$y, frac = ff * 100, 158 | 'AUC' = attr(rocs[[1]], 'auc')) 159 | return(out) 160 | } 161 | ggplot(df, aes(FPR, TPR, color = as.factor(frac))) + 162 | geom_line(linewidth = 0.75) + 163 | geom_abline(intercept = 0L, slope = 1L, 164 | linetype = 'dashed', color = 'grey') + 165 | lims(x = c(0L, 1L), y = c(0L, 1L)) + 166 | labs(x = 'False Positive Rate', y = 'True Positive Rate') + 167 | scale_color_d3(name = 'Percent\nMissing') + 168 | theme_bw() + 169 | theme(text = element_text(size = 8), legend.key.size = unit(0.4, 'cm')) 170 | ggsave('fva.pdf', width = 5, height = 3) 171 | 172 | 173 | 174 | 175 | ### COVARIATE SHIFT ### 176 | 177 | ## CASE 1: BREASTCANCER ## 178 | df <- data.table(BreastCancer)[, c('Id', 'Class') := NULL] 179 | df <- as.data.table(lapply(df, as.numeric)) 180 | df[, Class := BreastCancer$Class] 181 | x <- df[, -c('Class')] 182 | x <- as.data.table(lapply(x, as.numeric)) 183 | n <- x[, .N] 184 | idx <- sample(1:n, n/5) 185 | trn_x <- as.matrix(x[-idx, ]) 186 | tst_x <- as.matrix(x[idx, ]) 187 | trn_y <- df$Class[-idx] 188 | trn_y <- ifelse(trn_y == 'malignant', 1L, 0L) 189 | tst_y <- df$Class[idx] 190 | f0 <- xgboost(data = trn_x, label = trn_y, nrounds = 20, 191 | objective = 'binary:logistic', verbose = 0) 192 | y1_hat <- predict(f0, trn_x) 193 | y0_hat <- 1 - y1_hat 194 | entropy <- -(y1_hat * log(y1_hat) + y0_hat * log(y0_hat)) 195 | f1 <- xgboost(data = trn_x, label = entropy, nrounds = 20, verbose = 0) 196 | phi0 <- as.data.table(predict(f1, tst_x, predcontrib = TRUE)) 197 | xgb.importance(colnames(trn_x), model = f0) 198 | 199 | # Perturb cell size 200 | x_tilde <- x 201 | x_tilde[, Cell.size := Cell.size + rnorm(.N)] 202 | x_tilde_mat <- as.matrix(x_tilde) 203 | y1_tilde <- predict(f0, x_tilde_mat) 204 | y0_tilde <- 1 - y1_tilde 205 | h_tilde <- -(y1_tilde * log(y1_tilde) + y1_tilde * log(y0_tilde)) 206 | f1_tilde <- xgboost(data = x_tilde_mat[-idx, ], label = h_tilde[-idx], nrounds = 20, 207 | verbose = 0) 208 | phi1 <- as.data.table(predict(f1_tilde, x_tilde_mat[idx, ], predcontrib = TRUE)) 209 | 210 | # Plot: feature value vs. Shapley value, colored by shift indicator 211 | tmp1 <- data.table('phi' = c(phi0$Cell.size, phi1$Cell.size), 212 | 'value' = c(df[idx, Cell.size], x_tilde[idx, Cell.size])) 213 | tmp1[, Data := rep(c('Original', 'Perturbed'), each = length(idx))] 214 | tmp1[, Dataset := 'BreastCancer'] 215 | 216 | ## CASE 2: Diabetes 217 | data(PimaIndiansDiabetes2) 218 | df <- data.table(PimaIndiansDiabetes2) 219 | x <- df[, -c('diabetes')] 220 | n <- x[, .N] 221 | idx <- sample(1:n, n/5) 222 | trn_x <- as.matrix(x[-idx, ]) 223 | tst_x <- as.matrix(x[idx, ]) 224 | trn_y <- df$diabetes[-idx] 225 | trn_y <- ifelse(trn_y == 'pos', 1L, 0L) 226 | f0 <- xgboost(data = trn_x, label = trn_y, nrounds = 20, 227 | objective = 'binary:logistic', verbose = 0) 228 | y1_hat <- predict(f0, trn_x) 229 | y0_hat <- 1 - y1_hat 230 | entropy <- -(y1_hat * log(y1_hat) + y0_hat * log(y0_hat)) 231 | f1 <- xgboost(data = trn_x, label = entropy, nrounds = 20, verbose = 0) 232 | phi0 <- as.data.table(predict(f1, tst_x, predcontrib = TRUE)) 233 | xgb.importance(colnames(trn_x), model = f0) 234 | 235 | # Perturb glucose 236 | x_tilde <- x 237 | x_tilde[, glucose := glucose + rnorm(.N)] 238 | x_tilde_mat <- as.matrix(x_tilde) 239 | y1_tilde <- predict(f0, x_tilde_mat) 240 | y0_tilde <- 1 - y1_tilde 241 | h_tilde <- -(y1_tilde * log(y1_tilde) + y1_tilde * log(y0_tilde)) 242 | f1_tilde <- xgboost(data = x_tilde_mat[-idx, ], label = h_tilde[-idx], nrounds = 20, 243 | verbose = 0) 244 | phi1 <- as.data.table(predict(f1_tilde, x_tilde_mat[idx, ], predcontrib = TRUE)) 245 | 246 | # Plot: feature value vs. Shapley value, colored by shift indicator 247 | tmp2 <- data.table('phi' = c(phi0$glucose, phi1$glucose), 248 | 'value' = c(df[idx, glucose], x_tilde[idx, glucose])) 249 | tmp2[, Data := rep(c('Original', 'Perturbed'), each = length(idx))] 250 | tmp2[, Dataset := 'Diabetes'] 251 | 252 | ## CASE 3: ## 253 | data(Sonar) 254 | df <- data.table(Sonar) 255 | x <- df[, -c('Class')] 256 | n <- x[, .N] 257 | idx <- sample(1:n, n/5) 258 | trn_x <- as.matrix(x[-idx, ]) 259 | tst_x <- as.matrix(x[idx, ]) 260 | trn_y <- df$Class[-idx] 261 | trn_y <- ifelse(trn_y == 'M', 1L, 0L) 262 | f0 <- xgboost(data = trn_x, label = trn_y, nrounds = 20, 263 | objective = 'binary:logistic', verbose = 0) 264 | y1_hat <- predict(f0, trn_x) 265 | y0_hat <- 1 - y1_hat 266 | entropy <- -(y1_hat * log(y1_hat) + y0_hat * log(y0_hat)) 267 | f1 <- xgboost(data = trn_x, label = entropy, nrounds = 20, verbose = 0) 268 | phi0 <- as.data.table(predict(f1, tst_x, predcontrib = TRUE)) 269 | xgb.importance(colnames(trn_x), model = f0) 270 | 271 | # Perturb V12 272 | x_tilde <- x 273 | x_tilde[, V12 := V12 + rnorm(.N, sd = 0.01)] 274 | x_tilde_mat <- as.matrix(x_tilde) 275 | y1_tilde <- predict(f0, x_tilde_mat) 276 | y0_tilde <- 1 - y1_tilde 277 | h_tilde <- -(y1_tilde * log(y1_tilde) + y1_tilde * log(y0_tilde)) 278 | f1_tilde <- xgboost(data = x_tilde_mat[-idx, ], label = h_tilde[-idx], nrounds = 20, 279 | verbose = 0) 280 | phi1 <- as.data.table(predict(f1_tilde, x_tilde_mat[idx, ], predcontrib = TRUE)) 281 | 282 | # Plot: feature value vs. Shapley value, colored by shift indicator 283 | tmp3 <- data.table('phi' = c(phi0$V12, phi1$V12), 284 | 'value' = c(df[idx, V12], x_tilde[idx, V12])) 285 | tmp3[, Data := rep(c('Original', 'Perturbed'), each = length(idx))] 286 | tmp3[, Dataset := 'Sonar'] 287 | 288 | 289 | ## CASE 4: Ionosphere ### 290 | 291 | data(Ionosphere) 292 | df <- data.table(Ionosphere)[, V2 := NULL][, V1 := as.numeric(V1)] 293 | x <- df[, -c('Class')] 294 | n <- x[, .N] 295 | idx <- sample(1:n, n/5) 296 | trn_x <- as.matrix(x[-idx, ]) 297 | tst_x <- as.matrix(x[idx, ]) 298 | trn_y <- df$Class[-idx] 299 | trn_y <- ifelse(trn_y == 'good', 1L, 0L) 300 | f0 <- xgboost(data = trn_x, label = trn_y, nrounds = 20, 301 | objective = 'binary:logistic', verbose = 0) 302 | y1_hat <- predict(f0, trn_x) 303 | y0_hat <- 1 - y1_hat 304 | entropy <- -(y1_hat * log(y1_hat) + y0_hat * log(y0_hat)) 305 | f1 <- xgboost(data = trn_x, label = entropy, nrounds = 20, verbose = 0) 306 | phi0 <- as.data.table(predict(f1, tst_x, predcontrib = TRUE)) 307 | xgb.importance(colnames(trn_x), model = f0) 308 | 309 | # Perturb V5 310 | x_tilde <- x 311 | x_tilde[, V5 := V5 + rnorm(.N, sd = 0.25)] 312 | x_tilde_mat <- as.matrix(x_tilde) 313 | y1_tilde <- predict(f0, x_tilde_mat) 314 | y0_tilde <- 1 - y1_tilde 315 | h_tilde <- -(y1_tilde * log(y1_tilde) + y1_tilde * log(y0_tilde)) 316 | f1_tilde <- xgboost(data = x_tilde_mat[-idx, ], label = h_tilde[-idx], nrounds = 20, 317 | verbose = 0) 318 | phi1 <- as.data.table(predict(f1_tilde, x_tilde_mat[idx, ], predcontrib = TRUE)) 319 | 320 | # Plot: feature value vs. Shapley value, colored by shift indicator 321 | tmp4 <- data.table('phi' = c(phi0$V5, phi1$V5), 322 | 'value' = c(df[idx, V5], x_tilde[idx, V5])) 323 | tmp4[, Data := rep(c('Original', 'Perturbed'), each = length(idx))] 324 | tmp4[, Dataset := 'Ionosphere'] 325 | 326 | tmp <- rbind(tmp1, tmp2, tmp3, tmp4) 327 | ggplot(tmp, aes(value, phi, color = Data, shape = Data)) + 328 | geom_point(size = 0.5, alpha = 0.75) + 329 | scale_color_d3() + 330 | labs(x = 'Feature Value', y = 'Shapley Value') + 331 | facet_wrap(~ Dataset, nrow = 2, scales = 'free') + 332 | theme_bw() + 333 | theme(text = element_text(size = 8), legend.key.size = unit(0.4, 'cm')) 334 | ggsave('ood.pdf', width = 5, height = 3) 335 | 336 | 337 | ### Convergence experiments ### 338 | 339 | # Hyperparameters 340 | n <- 2000 341 | d <- 4 342 | sparsity <- 0 343 | rho <- 0.5 344 | mu <- rep(0, d + 1) 345 | sigma <- toeplitz(rho^(0:(d-1))) 346 | 347 | # Effects 348 | beta <- double(length = d) 349 | k <- round((1 - sparsity) * d) 350 | if (k > 0) { 351 | beta[1:k] <- 1L 352 | } 353 | 354 | # Create data 355 | x <- matrix(rmvnorm(n + 1000, mu = mu, sigma = sigma), ncol = d + 1, 356 | dimnames = list(NULL, paste0('x', 1:d))) 357 | log_var <- x %*% beta 358 | s2 <- exp(log_var) 359 | y <- rnorm(n + 1000, sd = sqrt(s2)) 360 | y2 <- log(y^2) 361 | 362 | # Split train/test 363 | trn_x <- x[1:n, ] 364 | trn_y <- y2[1:n] 365 | tst_x <- x[(n + 1):(n + 1000), ] 366 | tst_y <- y2[(n + 1):(n + 1000)] 367 | 368 | # Fit heteroskedastic error model 369 | h <- lm(trn_y ~ trn_x) 370 | summary(h) 371 | 372 | # Plot 373 | ggplot(df1, aes(n, MAE, color = Method, fill = Method)) + 374 | geom_point() + 375 | geom_line() + 376 | geom_ribbon(aes(ymin = MAE - std_dev, ymax = MAE + std_dev), alpha = 0.4) + 377 | scale_y_log10() + 378 | scale_color_d3() + 379 | scale_fill_d3() + 380 | labs(x = 'Sample Size', y = 'MAE (log scale)') + 381 | theme_bw() + 382 | theme(text = element_text(size = 8), legend.key.size = unit(0.4, 'cm')) 383 | ggsave('n_vs_mae.pdf', width = 5, height = 3) 384 | 385 | ggplot(df2, aes(rho, MAE, color = Method, fill = Method)) + 386 | geom_point() + 387 | geom_line() + 388 | geom_ribbon(aes(ymin = MAE - std_dev, ymax = MAE + std_dev), alpha = 0.4) + 389 | scale_y_log10() + 390 | scale_color_d3() + 391 | scale_fill_d3() + 392 | labs(x = 'Correlation', y = 'MAE (log scale)') + 393 | theme_bw() + 394 | theme(text = element_text(size = 8), legend.key.size = unit(0.4, 'cm')) 395 | ggsave('rho_vs_mae.pdf', width = 5, height = 3) 396 | 397 | 398 | 399 | 400 | 401 | 402 | 403 | 404 | 405 | 406 | 407 | 408 | 409 | 410 | 411 | 412 | 413 | 414 | 415 | 416 | 417 | 418 | 419 | 420 | 421 | 422 | 423 | 424 | 425 | 426 | 427 | 428 | 429 | 430 | 431 | 432 | 433 | 434 | 435 | 436 | 437 | 438 | -------------------------------------------------------------------------------- /MCDropout/dropoutNet.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) Meta Platforms, Inc. and affiliates. 2 | # All rights reserved. 3 | 4 | # This source code is licensed under the license found in the 5 | # LICENSE file in the root directory of this source tree. 6 | 7 | import torch 8 | from torchvision import datasets, transforms 9 | from torch import nn, optim, Tensor 10 | from torch.utils.data import DataLoader 11 | from torch.nn import functional as F 12 | from torch.optim import Optimizer 13 | import matplotlib.pyplot as plt 14 | from typing import Tuple 15 | import argparse 16 | 17 | import numpy as np 18 | import sys 19 | import shap 20 | 21 | class Net(nn.Module): 22 | def __init__(self, forward_passes: int = 20, mode: str = 'point') -> None: 23 | """Initialize the network.""" 24 | super(Net, self).__init__() 25 | 26 | self.forward_passes = forward_passes 27 | self.mode = mode 28 | 29 | # Convolutional layers 30 | self.conv_layers = nn.Sequential( 31 | nn.Conv2d(1, 10, kernel_size=5), 32 | nn.MaxPool2d(2), 33 | nn.ReLU(), 34 | nn.Dropout(p=0.3), 35 | nn.Conv2d(10, 20, kernel_size=5), 36 | nn.Dropout(), 37 | nn.MaxPool2d(2), 38 | nn.ReLU(), 39 | nn.Dropout(p=0.3), 40 | ) 41 | 42 | # Fully connected layers 43 | self.fc_layers = nn.Sequential( 44 | nn.Linear(320, 50), 45 | nn.ReLU(), 46 | nn.Dropout(), 47 | nn.Linear(50, 2), 48 | nn.Softmax(dim=1) 49 | ) 50 | 51 | def forward(self, x: torch.Tensor) -> torch.Tensor: 52 | """Forward pass through the network.""" 53 | if self.mode == 'point': 54 | x = self.conv_layers(x) 55 | x = x.view(-1, 320) 56 | x = self.fc_layers(x) 57 | 58 | elif self.mode == 'total_entropy': 59 | total_entropy = self.get_monte_carlo_predictions(x, x.size()[0], self.forward_passes, 'total_entropy') 60 | x = total_entropy 61 | 62 | elif self.mode == 'al_entropy': 63 | al_entropy = self.get_monte_carlo_predictions(x, x.size()[0], self.forward_passes, 'al_entropy') 64 | x = al_entropy 65 | 66 | elif self.mode == 'ep_entropy': 67 | total_entropy = self.get_monte_carlo_predictions(x, x.size()[0], self.forward_passes, 'total_entropy') 68 | al_entropy = self.get_monte_carlo_predictions(x, x.size()[0], self.forward_passes, 'al_entropy') 69 | ep_entropy = total_entropy - al_entropy 70 | x = ep_entropy 71 | 72 | else: 73 | x = self.get_monte_carlo_predictions(x, x.size()[0], self.forward_passes, self.mode) 74 | 75 | return x 76 | 77 | def total_entropy(self, x: torch.Tensor) -> torch.Tensor: 78 | """Calculate total entropy.""" 79 | entropy = -torch.sum(x * torch.log(x), dim=-1) 80 | return entropy[:, None] 81 | 82 | def enable_dropout(self) -> None: 83 | """Enable dropout layers during test-time.""" 84 | for m in self.modules(): 85 | if m.__class__.__name__.startswith('Dropout'): 86 | m.train() 87 | 88 | def get_monte_carlo_predictions(self, x: torch.Tensor, batch: int, forward_passes: int, mode: str) -> torch.Tensor: 89 | """Get Monte Carlo predictions.""" 90 | n_classes = 2 91 | n_samples = batch 92 | device = x.device 93 | dropout_predictions = torch.empty((0, n_samples, n_classes), device=device) 94 | 95 | for i in range(forward_passes): 96 | self.enable_dropout() 97 | conv_out = self.conv_layers(x) 98 | reshape_out = conv_out.view(-1, 320) 99 | predictions = self.fc_layers(reshape_out) 100 | dropout_predictions = torch.cat((dropout_predictions, predictions.unsqueeze(0)), dim=0) 101 | 102 | # Calculate mean across multiple MCD forward passes 103 | mean = dropout_predictions.mean(dim=0) 104 | 105 | # Calculate entropy across multiple MCD forward passes 106 | entropy = -torch.sum(dropout_predictions * torch.log(dropout_predictions), dim=-1) 107 | entropy = entropy.mean(dim=0) 108 | 109 | if mode == 'al_entropy': 110 | return entropy[:, None] 111 | 112 | if mode == 'total_entropy': 113 | return self.total_entropy(mean) 114 | 115 | if mode == "mean": 116 | return mean 117 | 118 | 119 | def train(model: torch.nn.Module, 120 | device: torch.device, 121 | train_loader: DataLoader, 122 | optimizer: optim.Optimizer, 123 | epoch: int) -> None: 124 | 125 | model.train() 126 | total = 0 127 | correct = 0 128 | 129 | for batch_idx, (data, target) in enumerate(train_loader): 130 | data, target = data.to(device), target.to(device) 131 | optimizer.zero_grad() 132 | output = model(data) 133 | loss = F.nll_loss(output.log(), target) 134 | loss.backward() 135 | optimizer.step() 136 | 137 | # Calculate number of correctly classified samples in the current batch 138 | pred = output.argmax(dim=1, keepdim=True) 139 | correct += pred.eq(target.view_as(pred)).sum().item() 140 | total += data.size(0) 141 | 142 | if batch_idx % 100 == 0: 143 | print(f'Train Epoch: {epoch} ' 144 | f'Loss: {loss.item():.6f} Correct: {correct}/{total}') 145 | 146 | def create_shap_plots(model: torch.nn.Module, 147 | background: Tensor, 148 | test_images: Tensor, 149 | mode: str, 150 | class1: str, 151 | class2: str) -> None: 152 | """ 153 | Create SHAP plots for the model predictions. 154 | 155 | Parameters: 156 | - model: The trained neural network model. 157 | - background: Background data for the SHAP explainer. 158 | - test_images: Images to be explained. 159 | - mode: The mode for SHAP plotting. 160 | 161 | Returns: 162 | - None 163 | """ 164 | explainer = shap.DeepExplainer(model, background) 165 | shap_values = np.asarray(explainer.shap_values(test_images)) 166 | 167 | test_images_cpu = test_images.to(torch.device('cpu')) 168 | 169 | plt.ioff() 170 | 171 | if mode in ['al_entropy', 'total_entropy', 'ep_entropy']: 172 | shap_entropy_numpy = np.swapaxes(np.swapaxes(shap_values, 1, -1), 1, 2) 173 | test_entropy_numpy = np.swapaxes(np.swapaxes(np.asarray(test_images_cpu), 1, -1), 1, 2) 174 | shap.image_plot(shap_entropy_numpy, -test_entropy_numpy, show=False) 175 | fig = plt.gcf() 176 | fig.savefig(f"../figs/{class1}_{class2}/{mode}.png") 177 | plt.close(fig) 178 | 179 | elif mode in ['mean', 'point']: 180 | # transformed_shap_values = [] 181 | # shap_values_transposed = np.transpose(shap_values, (0, 4, 1, 2, 3)) 182 | 183 | # # Iterate over each model output and reshape accordingly 184 | # for i in range(shap_values_transposed.shape[1]): 185 | # shap_for_output = shap_values_transposed[:, i, :, :, :] 186 | # shap_for_output = np.reshape(shap_for_output, (shap_for_output.shape[0], 28, 28, 1)) 187 | # transformed_shap_values.append(shap_for_output) 188 | # test_numpy = np.swapaxes(np.swapaxes(test_images_cpu.numpy(), 1, -1), 1, 2) 189 | shap_numpy = [np.swapaxes(np.swapaxes(s, 1, -1), 1, 2) for s in shap_values] 190 | test_numpy = np.swapaxes(np.swapaxes(test_images_cpu.numpy(), 1, -1), 1, 2) 191 | shap.image_plot(shap_numpy, -test_numpy, show=False) 192 | fig = plt.gcf() 193 | fig.savefig(f"../figs/{class1}_{class2}/{mode}.png") 194 | plt.close(fig) 195 | 196 | plt.ion() 197 | 198 | 199 | def create_dataset(x: int = 1, y: int = 7) -> tuple: 200 | # Create and return dataset 201 | train_dataset = datasets.MNIST('mnist_data', train=True, download=True, transform=transforms.Compose([ 202 | transforms.ToTensor() 203 | ])) 204 | train_idx1 = torch.tensor(train_dataset.targets) == x 205 | train_idx2 = torch.tensor(train_dataset.targets) == y 206 | 207 | train_indices_1 = train_idx1.nonzero().reshape(-1) 208 | train_indices_2 = train_idx2.nonzero().reshape(-1) 209 | 210 | for i in train_indices_1: 211 | train_dataset.targets[i] = 0 212 | for j in train_indices_2: 213 | train_dataset.targets[j] = 1 214 | 215 | train_mask = train_idx1 | train_idx2 216 | train_indices = train_mask.nonzero().reshape(-1) 217 | train_subset = torch.utils.data.Subset(train_dataset, train_indices) 218 | train_loader = torch.utils.data.DataLoader(train_subset, batch_size=args.batch_size, shuffle=True) 219 | 220 | test_dataset = datasets.MNIST('mnist_data', train=False, transform=transforms.Compose([ 221 | transforms.ToTensor() 222 | ])) 223 | test_idx1 = torch.tensor(test_dataset.targets) == x 224 | test_idx2 = torch.tensor(test_dataset.targets) == y 225 | 226 | test_indices_1 = test_idx1.nonzero().reshape(-1) 227 | test_indices_2 = test_idx2.nonzero().reshape(-1) 228 | 229 | for i in test_indices_1: 230 | test_dataset.targets[i] = 0 231 | for j in test_indices_2: 232 | test_dataset.targets[j] = 1 233 | 234 | test_mask = test_idx1 | test_idx2 235 | test_indices = test_mask.nonzero().reshape(-1) 236 | test_subset = torch.utils.data.Subset(test_dataset, test_indices) 237 | test_loader = torch.utils.data.DataLoader(test_subset, batch_size=args.batch_size, shuffle=True) 238 | path = "../models/MCDropout" + str(x) + str(y) + ".pth" 239 | return train_loader, test_loader, path 240 | 241 | 242 | def main(args): 243 | device = torch.device(args.device) 244 | train_loader, test_loader, path = create_dataset(args.class1, args.class2) 245 | 246 | # Initialize and train/test model based on specified mode 247 | trainPointModel = Net(mode="point").to(device) 248 | optimizer = optim.SGD(trainPointModel.parameters(), lr=0.01, momentum=0.5) 249 | 250 | batch = next(iter(test_loader)) 251 | images, _ = batch 252 | 253 | background = images[0:2000].to(device) 254 | test_images = images[0:5] 255 | 256 | if args.train_mode == 'train': 257 | for epoch in range(1, args.num_epochs + 1): 258 | train(trainPointModel, device, train_loader, optimizer, epoch) 259 | #test(meanModel, device, test_loader) 260 | torch.save(trainPointModel.state_dict(), path) 261 | print(f"Model saved at {path}") 262 | 263 | if args.train_mode == 'test': 264 | if args.inference_mode == 'point': 265 | #Initialise model(s) 266 | pointModel = Net(forward_passes=args.forward_passes, mode=args.inference_mode).to(device) 267 | pointModel.load_state_dict(torch.load(path)) 268 | out = pointModel(test_images.to(device)) 269 | 270 | create_shap_plots(pointModel, background, test_images, args.inference_mode, args.class1, args.class2) 271 | 272 | elif args.inference_mode == 'mean': 273 | #Initialise model(s) 274 | meanModel = Net(forward_passes=args.forward_passes, mode=args.inference_mode).to(device) 275 | meanModel.load_state_dict(torch.load(path)) 276 | out = meanModel(test_images.to(device)) 277 | 278 | create_shap_plots(meanModel, background, test_images, args.inference_mode, args.class1, args.class2) 279 | 280 | elif args.inference_mode == 'al_entropy': 281 | #Initialise model(s) 282 | alEntropyModel = Net(forward_passes=args.forward_passes, mode=args.inference_mode).to(device) 283 | alEntropyModel.load_state_dict(torch.load(path)) 284 | 285 | entropies = alEntropyModel(background) 286 | mean_entropy = torch.mean(entropies) 287 | max_entropy = torch.topk(entropies.flatten(), 5) 288 | indices_cpu = max_entropy.indices.to(torch.device('cpu')) 289 | test_images = images[indices_cpu].to(device) 290 | 291 | create_shap_plots(alEntropyModel, background, test_images, args.inference_mode, args.class1, args.class2) 292 | 293 | elif args.inference_mode == 'total_entropy': 294 | #Initialise model(s) 295 | totalEntropyModel = Net(forward_passes=args.forward_passes, mode=args.inference_mode).to(device) 296 | totalEntropyModel.load_state_dict(torch.load(path)) 297 | 298 | entropies = totalEntropyModel(background) 299 | mean_entropy = torch.mean(entropies) 300 | max_entropy = torch.topk(entropies.flatten(), 5) 301 | indices_cpu = max_entropy.indices.to(torch.device('cpu')) 302 | test_images = images[indices_cpu].to(device) 303 | 304 | create_shap_plots(totalEntropyModel, background, test_images, args.inference_mode, args.class1, args.class2) 305 | 306 | elif args.inference_mode == 'all': 307 | #Initialise model(s) 308 | totalEntropyModel = Net(forward_passes=args.forward_passes, mode='total_entropy').to(device) 309 | totalEntropyModel.load_state_dict(torch.load(path)) 310 | alEntropyModel = Net(forward_passes=args.forward_passes, mode='al_entropy').to(device) 311 | alEntropyModel.load_state_dict(torch.load(path)) 312 | epEntropyModel = Net(forward_passes=args.forward_passes, mode='ep_entropy').to(device) 313 | epEntropyModel.load_state_dict(torch.load(path)) 314 | pointModel = Net(forward_passes=args.forward_passes, mode='point').to(device) 315 | pointModel.load_state_dict(torch.load(path)) 316 | 317 | #Filter subsample for high entropy 318 | entropies = totalEntropyModel(background) 319 | mean_entropy = torch.mean(entropies) 320 | max_entropy = torch.topk(entropies.flatten(), 5) 321 | indices_cpu = max_entropy.indices.to(torch.device('cpu')) 322 | test_images = images[indices_cpu].to(device) 323 | test_images.to(device) 324 | print('breakdown') 325 | create_shap_plots(totalEntropyModel, background, test_images, 'total_entropy', args.class1, args.class2) 326 | create_shap_plots(alEntropyModel, background, test_images, 'al_entropy', args.class1, args.class2) 327 | create_shap_plots(epEntropyModel, background, test_images, 'ep_entropy', args.class1, args.class2) 328 | create_shap_plots(pointModel, background, test_images, 'point', args.class1, args.class2) 329 | else: 330 | print("Invalid inference mode selected!") 331 | 332 | if __name__ == '__main__': 333 | parser = argparse.ArgumentParser(description='NeurIPS Paper Code') 334 | parser.add_argument('--batch_size', type=int, default=128, help='Batch size for training.') 335 | parser.add_argument('--num_epochs', type=int, default=20, help='Number of epochs for training.') 336 | parser.add_argument('--device', type=str, default='cuda', help='Device to use for training/testing.') 337 | parser.add_argument('--inference_mode', type=str, default='all', help='Mode for inference.') 338 | parser.add_argument('--train_mode', type=str, default='test', help='Mode for training.') 339 | parser.add_argument('--forward_passes', type=int, default=50, help='Number of forward passes.') 340 | parser.add_argument('--class1', type=int, default=4, help='Class 1 for the dataset.') 341 | parser.add_argument('--class2', type=int, default=9, help='Class 2 for the dataset.') 342 | 343 | args = parser.parse_args() 344 | main(args) 345 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Attribution-NonCommercial 4.0 International 2 | 3 | ======================================================================= 4 | 5 | Creative Commons Corporation ("Creative Commons") is not a law firm and 6 | does not provide legal services or legal advice. Distribution of 7 | Creative Commons public licenses does not create a lawyer-client or 8 | other relationship. Creative Commons makes its licenses and related 9 | information available on an "as-is" basis. Creative Commons gives no 10 | warranties regarding its licenses, any material licensed under their 11 | terms and conditions, or any related information. Creative Commons 12 | disclaims all liability for damages resulting from their use to the 13 | fullest extent possible. 14 | 15 | Using Creative Commons Public Licenses 16 | 17 | Creative Commons public licenses provide a standard set of terms and 18 | conditions that creators and other rights holders may use to share 19 | original works of authorship and other material subject to copyright 20 | and certain other rights specified in the public license below. The 21 | following considerations are for informational purposes only, are not 22 | exhaustive, and do not form part of our licenses. 23 | 24 | Considerations for licensors: Our public licenses are 25 | intended for use by those authorized to give the public 26 | permission to use material in ways otherwise restricted by 27 | copyright and certain other rights. Our licenses are 28 | irrevocable. Licensors should read and understand the terms 29 | and conditions of the license they choose before applying it. 30 | Licensors should also secure all rights necessary before 31 | applying our licenses so that the public can reuse the 32 | material as expected. Licensors should clearly mark any 33 | material not subject to the license. This includes other CC- 34 | licensed material, or material used under an exception or 35 | limitation to copyright. More considerations for licensors: 36 | wiki.creativecommons.org/Considerations_for_licensors 37 | 38 | Considerations for the public: By using one of our public 39 | licenses, a licensor grants the public permission to use the 40 | licensed material under specified terms and conditions. If 41 | the licensor's permission is not necessary for any reason--for 42 | example, because of any applicable exception or limitation to 43 | copyright--then that use is not regulated by the license. Our 44 | licenses grant only permissions under copyright and certain 45 | other rights that a licensor has authority to grant. Use of 46 | the licensed material may still be restricted for other 47 | reasons, including because others have copyright or other 48 | rights in the material. A licensor may make special requests, 49 | such as asking that all changes be marked or described. 50 | Although not required by our licenses, you are encouraged to 51 | respect those requests where reasonable. More_considerations 52 | for the public: 53 | wiki.creativecommons.org/Considerations_for_licensees 54 | 55 | ======================================================================= 56 | 57 | Creative Commons Attribution-NonCommercial 4.0 International Public 58 | License 59 | 60 | By exercising the Licensed Rights (defined below), You accept and agree 61 | to be bound by the terms and conditions of this Creative Commons 62 | Attribution-NonCommercial 4.0 International Public License ("Public 63 | License"). To the extent this Public License may be interpreted as a 64 | contract, You are granted the Licensed Rights in consideration of Your 65 | acceptance of these terms and conditions, and the Licensor grants You 66 | such rights in consideration of benefits the Licensor receives from 67 | making the Licensed Material available under these terms and 68 | conditions. 69 | 70 | Section 1 -- Definitions. 71 | 72 | a. Adapted Material means material subject to Copyright and Similar 73 | Rights that is derived from or based upon the Licensed Material 74 | and in which the Licensed Material is translated, altered, 75 | arranged, transformed, or otherwise modified in a manner requiring 76 | permission under the Copyright and Similar Rights held by the 77 | Licensor. For purposes of this Public License, where the Licensed 78 | Material is a musical work, performance, or sound recording, 79 | Adapted Material is always produced where the Licensed Material is 80 | synched in timed relation with a moving image. 81 | 82 | b. Adapter's License means the license You apply to Your Copyright 83 | and Similar Rights in Your contributions to Adapted Material in 84 | accordance with the terms and conditions of this Public License. 85 | 86 | c. Copyright and Similar Rights means copyright and/or similar rights 87 | closely related to copyright including, without limitation, 88 | performance, broadcast, sound recording, and Sui Generis Database 89 | Rights, without regard to how the rights are labeled or 90 | categorized. For purposes of this Public License, the rights 91 | specified in Section 2(b)(1)-(2) are not Copyright and Similar 92 | Rights. 93 | d. Effective Technological Measures means those measures that, in the 94 | absence of proper authority, may not be circumvented under laws 95 | fulfilling obligations under Article 11 of the WIPO Copyright 96 | Treaty adopted on December 20, 1996, and/or similar international 97 | agreements. 98 | 99 | e. Exceptions and Limitations means fair use, fair dealing, and/or 100 | any other exception or limitation to Copyright and Similar Rights 101 | that applies to Your use of the Licensed Material. 102 | 103 | f. Licensed Material means the artistic or literary work, database, 104 | or other material to which the Licensor applied this Public 105 | License. 106 | 107 | g. Licensed Rights means the rights granted to You subject to the 108 | terms and conditions of this Public License, which are limited to 109 | all Copyright and Similar Rights that apply to Your use of the 110 | Licensed Material and that the Licensor has authority to license. 111 | 112 | h. Licensor means the individual(s) or entity(ies) granting rights 113 | under this Public License. 114 | 115 | i. NonCommercial means not primarily intended for or directed towards 116 | commercial advantage or monetary compensation. For purposes of 117 | this Public License, the exchange of the Licensed Material for 118 | other material subject to Copyright and Similar Rights by digital 119 | file-sharing or similar means is NonCommercial provided there is 120 | no payment of monetary compensation in connection with the 121 | exchange. 122 | 123 | j. Share means to provide material to the public by any means or 124 | process that requires permission under the Licensed Rights, such 125 | as reproduction, public display, public performance, distribution, 126 | dissemination, communication, or importation, and to make material 127 | available to the public including in ways that members of the 128 | public may access the material from a place and at a time 129 | individually chosen by them. 130 | 131 | k. Sui Generis Database Rights means rights other than copyright 132 | resulting from Directive 96/9/EC of the European Parliament and of 133 | the Council of 11 March 1996 on the legal protection of databases, 134 | as amended and/or succeeded, as well as other essentially 135 | equivalent rights anywhere in the world. 136 | 137 | l. You means the individual or entity exercising the Licensed Rights 138 | under this Public License. Your has a corresponding meaning. 139 | 140 | Section 2 -- Scope. 141 | 142 | a. License grant. 143 | 144 | 1. Subject to the terms and conditions of this Public License, 145 | the Licensor hereby grants You a worldwide, royalty-free, 146 | non-sublicensable, non-exclusive, irrevocable license to 147 | exercise the Licensed Rights in the Licensed Material to: 148 | 149 | a. reproduce and Share the Licensed Material, in whole or 150 | in part, for NonCommercial purposes only; and 151 | 152 | b. produce, reproduce, and Share Adapted Material for 153 | NonCommercial purposes only. 154 | 155 | 2. Exceptions and Limitations. For the avoidance of doubt, where 156 | Exceptions and Limitations apply to Your use, this Public 157 | License does not apply, and You do not need to comply with 158 | its terms and conditions. 159 | 160 | 3. Term. The term of this Public License is specified in Section 161 | 6(a). 162 | 163 | 4. Media and formats; technical modifications allowed. The 164 | Licensor authorizes You to exercise the Licensed Rights in 165 | all media and formats whether now known or hereafter created, 166 | and to make technical modifications necessary to do so. The 167 | Licensor waives and/or agrees not to assert any right or 168 | authority to forbid You from making technical modifications 169 | necessary to exercise the Licensed Rights, including 170 | technical modifications necessary to circumvent Effective 171 | Technological Measures. For purposes of this Public License, 172 | simply making modifications authorized by this Section 2(a) 173 | (4) never produces Adapted Material. 174 | 175 | 5. Downstream recipients. 176 | 177 | a. Offer from the Licensor -- Licensed Material. Every 178 | recipient of the Licensed Material automatically 179 | receives an offer from the Licensor to exercise the 180 | Licensed Rights under the terms and conditions of this 181 | Public License. 182 | 183 | b. No downstream restrictions. You may not offer or impose 184 | any additional or different terms or conditions on, or 185 | apply any Effective Technological Measures to, the 186 | Licensed Material if doing so restricts exercise of the 187 | Licensed Rights by any recipient of the Licensed 188 | Material. 189 | 190 | 6. No endorsement. Nothing in this Public License constitutes or 191 | may be construed as permission to assert or imply that You 192 | are, or that Your use of the Licensed Material is, connected 193 | with, or sponsored, endorsed, or granted official status by, 194 | the Licensor or others designated to receive attribution as 195 | provided in Section 3(a)(1)(A)(i). 196 | 197 | b. Other rights. 198 | 199 | 1. Moral rights, such as the right of integrity, are not 200 | licensed under this Public License, nor are publicity, 201 | privacy, and/or other similar personality rights; however, to 202 | the extent possible, the Licensor waives and/or agrees not to 203 | assert any such rights held by the Licensor to the limited 204 | extent necessary to allow You to exercise the Licensed 205 | Rights, but not otherwise. 206 | 207 | 2. Patent and trademark rights are not licensed under this 208 | Public License. 209 | 210 | 3. To the extent possible, the Licensor waives any right to 211 | collect royalties from You for the exercise of the Licensed 212 | Rights, whether directly or through a collecting society 213 | under any voluntary or waivable statutory or compulsory 214 | licensing scheme. In all other cases the Licensor expressly 215 | reserves any right to collect such royalties, including when 216 | the Licensed Material is used other than for NonCommercial 217 | purposes. 218 | 219 | Section 3 -- License Conditions. 220 | 221 | Your exercise of the Licensed Rights is expressly made subject to the 222 | following conditions. 223 | 224 | a. Attribution. 225 | 226 | 1. If You Share the Licensed Material (including in modified 227 | form), You must: 228 | 229 | a. retain the following if it is supplied by the Licensor 230 | with the Licensed Material: 231 | 232 | i. identification of the creator(s) of the Licensed 233 | Material and any others designated to receive 234 | attribution, in any reasonable manner requested by 235 | the Licensor (including by pseudonym if 236 | designated); 237 | 238 | ii. a copyright notice; 239 | 240 | iii. a notice that refers to this Public License; 241 | 242 | iv. a notice that refers to the disclaimer of 243 | warranties; 244 | 245 | v. a URI or hyperlink to the Licensed Material to the 246 | extent reasonably practicable; 247 | 248 | b. indicate if You modified the Licensed Material and 249 | retain an indication of any previous modifications; and 250 | 251 | c. indicate the Licensed Material is licensed under this 252 | Public License, and include the text of, or the URI or 253 | hyperlink to, this Public License. 254 | 255 | 2. You may satisfy the conditions in Section 3(a)(1) in any 256 | reasonable manner based on the medium, means, and context in 257 | which You Share the Licensed Material. For example, it may be 258 | reasonable to satisfy the conditions by providing a URI or 259 | hyperlink to a resource that includes the required 260 | information. 261 | 262 | 3. If requested by the Licensor, You must remove any of the 263 | information required by Section 3(a)(1)(A) to the extent 264 | reasonably practicable. 265 | 266 | 4. If You Share Adapted Material You produce, the Adapter's 267 | License You apply must not prevent recipients of the Adapted 268 | Material from complying with this Public License. 269 | 270 | Section 4 -- Sui Generis Database Rights. 271 | 272 | Where the Licensed Rights include Sui Generis Database Rights that 273 | apply to Your use of the Licensed Material: 274 | 275 | a. for the avoidance of doubt, Section 2(a)(1) grants You the right 276 | to extract, reuse, reproduce, and Share all or a substantial 277 | portion of the contents of the database for NonCommercial purposes 278 | only; 279 | 280 | b. if You include all or a substantial portion of the database 281 | contents in a database in which You have Sui Generis Database 282 | Rights, then the database in which You have Sui Generis Database 283 | Rights (but not its individual contents) is Adapted Material; and 284 | 285 | c. You must comply with the conditions in Section 3(a) if You Share 286 | all or a substantial portion of the contents of the database. 287 | 288 | For the avoidance of doubt, this Section 4 supplements and does not 289 | replace Your obligations under this Public License where the Licensed 290 | Rights include other Copyright and Similar Rights. 291 | 292 | Section 5 -- Disclaimer of Warranties and Limitation of Liability. 293 | 294 | a. UNLESS OTHERWISE SEPARATELY UNDERTAKEN BY THE LICENSOR, TO THE 295 | EXTENT POSSIBLE, THE LICENSOR OFFERS THE LICENSED MATERIAL AS-IS 296 | AND AS-AVAILABLE, AND MAKES NO REPRESENTATIONS OR WARRANTIES OF 297 | ANY KIND CONCERNING THE LICENSED MATERIAL, WHETHER EXPRESS, 298 | IMPLIED, STATUTORY, OR OTHER. THIS INCLUDES, WITHOUT LIMITATION, 299 | WARRANTIES OF TITLE, MERCHANTABILITY, FITNESS FOR A PARTICULAR 300 | PURPOSE, NON-INFRINGEMENT, ABSENCE OF LATENT OR OTHER DEFECTS, 301 | ACCURACY, OR THE PRESENCE OR ABSENCE OF ERRORS, WHETHER OR NOT 302 | KNOWN OR DISCOVERABLE. WHERE DISCLAIMERS OF WARRANTIES ARE NOT 303 | ALLOWED IN FULL OR IN PART, THIS DISCLAIMER MAY NOT APPLY TO YOU. 304 | 305 | b. TO THE EXTENT POSSIBLE, IN NO EVENT WILL THE LICENSOR BE LIABLE 306 | TO YOU ON ANY LEGAL THEORY (INCLUDING, WITHOUT LIMITATION, 307 | NEGLIGENCE) OR OTHERWISE FOR ANY DIRECT, SPECIAL, INDIRECT, 308 | INCIDENTAL, CONSEQUENTIAL, PUNITIVE, EXEMPLARY, OR OTHER LOSSES, 309 | COSTS, EXPENSES, OR DAMAGES ARISING OUT OF THIS PUBLIC LICENSE OR 310 | USE OF THE LICENSED MATERIAL, EVEN IF THE LICENSOR HAS BEEN 311 | ADVISED OF THE POSSIBILITY OF SUCH LOSSES, COSTS, EXPENSES, OR 312 | DAMAGES. WHERE A LIMITATION OF LIABILITY IS NOT ALLOWED IN FULL OR 313 | IN PART, THIS LIMITATION MAY NOT APPLY TO YOU. 314 | 315 | c. The disclaimer of warranties and limitation of liability provided 316 | above shall be interpreted in a manner that, to the extent 317 | possible, most closely approximates an absolute disclaimer and 318 | waiver of all liability. 319 | 320 | Section 6 -- Term and Termination. 321 | 322 | a. This Public License applies for the term of the Copyright and 323 | Similar Rights licensed here. However, if You fail to comply with 324 | this Public License, then Your rights under this Public License 325 | terminate automatically. 326 | 327 | b. Where Your right to use the Licensed Material has terminated under 328 | Section 6(a), it reinstates: 329 | 330 | 1. automatically as of the date the violation is cured, provided 331 | it is cured within 30 days of Your discovery of the 332 | violation; or 333 | 334 | 2. upon express reinstatement by the Licensor. 335 | 336 | For the avoidance of doubt, this Section 6(b) does not affect any 337 | right the Licensor may have to seek remedies for Your violations 338 | of this Public License. 339 | 340 | c. For the avoidance of doubt, the Licensor may also offer the 341 | Licensed Material under separate terms or conditions or stop 342 | distributing the Licensed Material at any time; however, doing so 343 | will not terminate this Public License. 344 | 345 | d. Sections 1, 5, 6, 7, and 8 survive termination of this Public 346 | License. 347 | 348 | Section 7 -- Other Terms and Conditions. 349 | 350 | a. The Licensor shall not be bound by any additional or different 351 | terms or conditions communicated by You unless expressly agreed. 352 | 353 | b. Any arrangements, understandings, or agreements regarding the 354 | Licensed Material not stated herein are separate from and 355 | independent of the terms and conditions of this Public License. 356 | 357 | Section 8 -- Interpretation. 358 | 359 | a. For the avoidance of doubt, this Public License does not, and 360 | shall not be interpreted to, reduce, limit, restrict, or impose 361 | conditions on any use of the Licensed Material that could lawfully 362 | be made without permission under this Public License. 363 | 364 | b. To the extent possible, if any provision of this Public License is 365 | deemed unenforceable, it shall be automatically reformed to the 366 | minimum extent necessary to make it enforceable. If the provision 367 | cannot be reformed, it shall be severed from this Public License 368 | without affecting the enforceability of the remaining terms and 369 | conditions. 370 | 371 | c. No term or condition of this Public License will be waived and no 372 | failure to comply consented to unless expressly agreed to by the 373 | Licensor. 374 | 375 | d. Nothing in this Public License constitutes or may be interpreted 376 | as a limitation upon, or waiver of, any privileges and immunities 377 | that apply to the Licensor or You, including from the legal 378 | processes of any jurisdiction or authority. 379 | 380 | ======================================================================= 381 | 382 | Creative Commons is not a party to its public 383 | licenses. Notwithstanding, Creative Commons may elect to apply one of 384 | its public licenses to material it publishes and in those instances 385 | will be considered the “Licensor.” The text of the Creative Commons 386 | public licenses is dedicated to the public domain under the CC0 Public 387 | Domain Dedication. Except for the limited purpose of indicating that 388 | material is shared under a Creative Commons public license or as 389 | otherwise permitted by the Creative Commons policies published at 390 | creativecommons.org/policies, Creative Commons does not authorize the 391 | use of the trademark "Creative Commons" or any other trademark or logo 392 | of Creative Commons without its prior written consent including, 393 | without limitation, in connection with any unauthorized modifications 394 | to any of its public licenses or any other arrangements, 395 | understandings, or agreements concerning use of licensed material. For 396 | the avoidance of doubt, this paragraph does not form part of the 397 | public licenses. 398 | 399 | Creative Commons may be contacted at creativecommons.org. 400 | --------------------------------------------------------------------------------