├── Makefile ├── README.org └── sbin ├── README.template.org ├── cache.jsonl └── gener_readme.py /Makefile: -------------------------------------------------------------------------------- 1 | README.org: sbin/README.template.org sbin/gener_readme.py 2 | cd sbin; ./gener_readme.py --readme-template README.template.org --cache-fname cache.jsonl > ../README.org 3 | -------------------------------------------------------------------------------- /README.org: -------------------------------------------------------------------------------- 1 | * Awesome Interpretable Machine Learning [[https://awesome.re][https://awesome.re/badge.svg]] 2 | 3 | Opinionated list of resources facilitating model interpretability 4 | (introspection, simplification, visualization, explanation). 5 | 6 | ** Interpretable Models 7 | + Interpretable models 8 | + Simple decision trees 9 | + Rules 10 | + (Regularized) linear regression 11 | + k-NN 12 | 13 | + (2008) Predictive learning via rule ensembles by Jerome H. Friedman, Bogdan E. Popescu 14 | + https://dx.doi.org/10.1214/07-AOAS148 15 | 16 | + (2014) Comprehensible classification models by Alex A. Freitas 17 | + https://dx.doi.org/10.1145/2594473.2594475 18 | + http://www.kdd.org/exploration_files/V15-01-01-Freitas.pdf 19 | + Interesting discussion of interpretability for a few classification models 20 | (decision trees, classification rules, decision tables, nearest neighbors and Bayesian network classifier) 21 | 22 | + (2015) Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model by Benjamin Letham, Cynthia Rudin, Tyler H. McCormick, David Madigan 23 | + https://arxiv.org/pdf/1511.01644 24 | + https://dx.doi.org/10.1214/15-AOAS848 25 | 26 | + (2017) Learning Explanatory Rules from Noisy Data by Richard Evans, Edward Grefenstette 27 | + https://arxiv.org/pdf/1711.04574 28 | 29 | + (2019) Transparent Classification with Multilayer Logical Perceptrons and Random Binarization by Zhuo Wang, Wei Zhang, Ning Liu, Jianyong Wang 30 | + https://arxiv.org/pdf/1912.04695 31 | + Code: https://github.com/12wang3/mllp 32 | 33 | ** Feature Importance 34 | + Models offering feature importance measures 35 | + Random forest 36 | + Boosted trees 37 | + Extremely randomized trees 38 | + (2006) Extremely randomized trees by Pierre Geurts, Damien Ernst, Louis Wehenkel 39 | + https://dx.doi.org/10.1007/s10994-006-6226-1 40 | + Random ferns 41 | + (2015) rFerns: An Implementation of the Random Ferns Method for General-Purpose Machine Learning by Miron B. Kursa 42 | + https://dx.doi.org/10.18637/jss.v061.i10 43 | + https://cran.r-project.org/web/packages/rFerns 44 | + https://notabug.org/mbq/rFerns 45 | + Linear regression (with a grain of salt) 46 | 47 | + (2007) Bias in random forest variable importance measures: Illustrations, sources and a solution by Carolin Strobl, Anne-Laure Boulesteix, Achim Zeileis, Torsten Hothorn 48 | + https://dx.doi.org/10.1186/1471-2105-8-25 49 | 50 | + (2008) Conditional Variable Importance for Random Forests by Carolin Strobl, Anne-Laure Boulesteix, Thomas Kneib, Thomas Augustin, Achim Zeileis 51 | + https://dx.doi.org/10.1186/1471-2105-9-307 52 | 53 | + (2018) Model Class Reliance: Variable Importance Measures for any Machine Learning Model Class, from the "Rashomon" Perspective by Aaron Fisher, Cynthia Rudin, Francesca Dominici 54 | + https://arxiv.org/pdf/1801.01489 55 | + https://github.com/aaronjfisher/mcr 56 | + Universal (model agnostic) variable importance measure 57 | 58 | + (2019) Please Stop Permuting Features: An Explanation and Alternatives by Giles Hooker, Lucas Mentch 59 | + https://arxiv.org/pdf/1905.03151 60 | + Paper advocating against feature permutation for importance 61 | 62 | + (2018) Visualizing the Feature Importance for Black Box Models by Giuseppe Casalicchio, Christoph Molnar, Bernd Bischl 63 | + https://arxiv.org/pdf/1804.06620 64 | + https://github.com/giuseppec/featureImportance 65 | + Global and local (model agnostic) variable importance measure (based on Model Reliance) 66 | 67 | + Very good blog post describing deficiencies of random forest feature importance and the permutation importance 68 | + http://explained.ai/rf-importance/index.html 69 | 70 | + Permutation importance - simple model agnostic approach is described in Eli5 documentation 71 | + https://eli5.readthedocs.io/en/latest/blackbox/permutation_importance.html 72 | 73 | ** Feature Selection 74 | + Classification of feature selection methods 75 | + Filters 76 | + Wrappers 77 | + Embedded methods 78 | 79 | + (2003) An Introduction to Variable and Feature Selection by Isabelle Guyon, André Elisseeff 80 | + http://www.jmlr.org/papers/volume3/guyon03a/guyon03a.pdf 81 | + Be sure to read this very illustrative introduction to feature selection 82 | 83 | + Filter Methods 84 | 85 | + (2006) On the Use of Variable Complementarity for Feature Selection in Cancer Classification by Patrick Meyer, Gianluca Bontempi 86 | + https://dx.doi.org/10.1007/11732242_9 87 | + https://pdfs.semanticscholar.org/d72f/f5063520ce4542d6d9b9e6a4f12aafab6091.pdf 88 | + Introduces information theoretic methods - double input symmetrical relevance (DISR) 89 | 90 | + (2012) Conditional Likelihood Maximisation: A Unifying Framework for Information Theoretic Feature Selection by Gavin Brown, Adam Pocock, Ming-Jie Zhao, Mikel Luján 91 | + http://www.jmlr.org/papers/volume13/brown12a/brown12a.pdf 92 | + Code: https://github.com/Craigacp/FEAST 93 | + Discusses various approaches based on mutual information (MIM, mRMR, MIFS, CMIM, JMI, DISR, ICAP, CIFE, CMI) 94 | 95 | + (2012) Feature selection via joint likelihood by Adam Pocock 96 | + http://www.cs.man.ac.uk/~gbrown/publications/pocockPhDthesis.pdf 97 | 98 | + (2017) Relief-Based Feature Selection: Introduction and Review by Ryan J. Urbanowicz, Melissa Meeker, William LaCava, Randal S. Olson, Jason H. Moore 99 | + https://arxiv.org/pdf/1711.08421 100 | 101 | + (2017) Benchmarking Relief-Based Feature Selection Methods for Bioinformatics Data Mining by Ryan J. Urbanowicz, Randal S. Olson, Peter Schmitt, Melissa Meeker, Jason H. Moore 102 | + https://arxiv.org/pdf/1711.08477 103 | 104 | + Wrapper methods 105 | 106 | + (2015) Feature Selection with theBorutaPackage by Miron B. Kursa, Witold R. Rudnicki 107 | + https://dx.doi.org/10.18637/jss.v036.i11 108 | + https://cran.r-project.org/web/packages/Boruta/ 109 | + Code (official, R): https://notabug.org/mbq/Boruta/ 110 | + Code (Python): https://github.com/scikit-learn-contrib/boruta_py 111 | 112 | + Boruta for those in a hurry 113 | + https://cran.r-project.org/web/packages/Boruta/vignettes/inahurry.pdf 114 | 115 | + General 116 | 117 | + (1994) Irrelevant Features and the Subset Selection Problem by George John, Ron Kohavi, Karl Pfleger 118 | + https://pdfs.semanticscholar.org/a83b/ddb34618cc68f1014ca12eef7f537825d104.pdf 119 | + Classic paper discussing weakly relevant features, irrelevant features, strongly relevant features 120 | 121 | + (2003) Special issue of JMLR of feature selection - oldish (2003) 122 | + http://www.jmlr.org/papers/special/feature03.html 123 | 124 | + (2004) Result Analysis of the NIPS 2003 Feature Selection Challenge by Isabelle Guyon, Steve Gunn, Asa Ben-Hur, Gideon Dror 125 | + Paper: https://papers.nips.cc/paper/2728-result-analysis-of-the-nips-2003-feature-selection-challenge.pdf 126 | + Website http://clopinet.com/isabelle/Projects/NIPS2003/ 127 | 128 | + (2007) Consistent Feature Selection for Pattern Recognition in Polynomial Time by Roland Nilsson, José Peña, Johan Björkegren, Jesper Tegnér 129 | + http://www.jmlr.org/papers/volume8/nilsson07a/nilsson07a.pdf 130 | + Discusses minimal optimal vs all-relevant approaches to feature selection 131 | 132 | + Feature Engineering and Selection by Kuhn & Johnson 133 | + Sligtly off-topic, but very interesting book 134 | + http://www.feat.engineering/index.html 135 | + https://bookdown.org/max/FES/ 136 | + https://github.com/topepo/FES 137 | 138 | + Feature Engineering presentation by H. J. van Veen 139 | + Slightly off-topicm but very interesting deck of slides 140 | + Slides: https://www.slideshare.net/HJvanVeen/feature-engineering-72376750 141 | 142 | ** Model Explanations 143 | *** Philosophy 144 | + Magnets by R. P. Feynman 145 | https://www.youtube.com/watch?v=wMFPe-DwULM 146 | 147 | + (2002) Looking Inside the Black Box, presentation of Leo Breiman 148 | + https://www.stat.berkeley.edu/users/breiman/wald2002-2.pdf 149 | 150 | + (2011) To Explain or to Predict? by Galit Shmueli 151 | + https://arxiv.org/pdf/1101.0891 152 | + https://dx.doi.org/10.1214/10-STS330 153 | 154 | + (2016) The Mythos of Model Interpretability by Zachary C. Lipton 155 | + https://arxiv.org/pdf/1606.03490 156 | + https://www.youtube.com/watch?v=mvzBQci04qA 157 | 158 | + (2017) Towards A Rigorous Science of Interpretable Machine Learning by Finale Doshi-Velez, Been Kim 159 | + https://arxiv.org/pdf/1702.08608 160 | 161 | + (2017) The Promise and Peril of Human Evaluation for Model Interpretability by Bernease Herman 162 | + https://arxiv.org/pdf/1711.07414 163 | 164 | + (2018) [[http://bayes.cs.ucla.edu/WHY/why-intro.pdf][The Book of Why: The New Science of Cause and Effect]] by Judea Pearl 165 | 166 | + (2018) Please Stop Doing the "Explainable" ML by Cynthia Rudin 167 | + Video (starts 17:30, lasts 10 min): https://zoom.us/recording/play/0y-iI9HamgyDzzP2k_jiTu6jB7JgVVXnjWZKDMbnyRTn3FsxTDZy6Wkrj3_ekx4J 168 | + Linked at: https://users.cs.duke.edu/~cynthia/mediatalks.html 169 | 170 | + (2018) Explaining Explanations: An Approach to Evaluating Interpretability of Machine Learning by Leilani H. Gilpin, David Bau, Ben Z. Yuan, Ayesha Bajwa, Michael Specter, Lalana Kagal 171 | + https://arxiv.org/pdf/1806.00069 172 | 173 | + (2019) Interpretable machine learning: definitions, methods, and applications by W. James Murdoch, Chandan Singh, Karl Kumbier, Reza Abbasi-Asl, Bin Yu 174 | + https://arxiv.org/pdf/1901.04592 175 | 176 | + (2019) On Explainable Machine Learning Misconceptions A More Human-Centered Machine Learning by Patrick Hall 177 | + https://github.com/jphall663/xai_misconceptions/blob/master/xai_misconceptions.pdf 178 | + https://github.com/jphall663/xai_misconceptions 179 | 180 | + (2019) An Introduction to Machine Learning Interpretability. An Applied Perspective on Fairness, Accountability, Transparency, and Explainable AI by Patrick Hall and Navdeep Gill 181 | + https://www.h2o.ai/wp-content/uploads/2019/08/An-Introduction-to-Machine-Learning-Interpretability-Second-Edition.pdf 182 | 183 | *** Model Agnostic Explanations 184 | + (2009) How to Explain Individual Classification Decisions by David Baehrens, Timon Schroeter, Stefan Harmeling, Motoaki Kawanabe, Katja Hansen, Klaus-Robert Mueller 185 | + https://arxiv.org/pdf/0912.1128 186 | 187 | + (2013) Peeking Inside the Black Box: Visualizing Statistical Learning with Plots of Individual Conditional Expectation by Alex Goldstein, Adam Kapelner, Justin Bleich, Emil Pitkin 188 | + https://arxiv.org/pdf/1309.6392 189 | 190 | + (2016) "Why Should I Trust You?": Explaining the Predictions of Any Classifier by Marco Tulio Ribeiro, Sameer Singh, Carlos Guestrin 191 | + https://arxiv.org/pdf/1602.04938 192 | + Code: https://github.com/marcotcr/lime 193 | + https://github.com/marcotcr/lime-experiments 194 | + https://www.youtube.com/watch?v=bCgEP2zuYxI 195 | + Introduces the LIME method (Local Interpretable Model-agnostic Explanations) 196 | 197 | + (2016) A Model Explanation System: Latest Updates and Extensions by Ryan Turner 198 | + https://arxiv.org/pdf/1606.09517 199 | + http://www.blackboxworkshop.org/pdf/Turner2015_MES.pdf 200 | 201 | + (2017) Understanding Black-box Predictions via Influence Functions by Pang Wei Koh, Percy Liang 202 | + https://arxiv.org/pdf/1703.04730 203 | 204 | + (2017) A Unified Approach to Interpreting Model Predictions by Scott Lundberg, Su-In Lee 205 | + https://arxiv.org/pdf/1705.07874 206 | + Code: https://github.com/slundberg/shap 207 | + Introduces the SHAP method (SHapley Additive exPlanations), generalizing LIME 208 | 209 | + (2018) Anchors: High-Precision Model-Agnostic Explanations by Marco Ribeiro, Sameer Singh, Carlos Guestrin 210 | + https://homes.cs.washington.edu/~marcotcr/aaai18.pdf 211 | + Code: https://github.com/marcotcr/anchor-experiments 212 | 213 | + (2018) Learning to Explain: An Information-Theoretic Perspective on Model Interpretation by Jianbo Chen, Le Song, Martin J. Wainwright, Michael I. Jordan 214 | + https://arxiv.org/pdf/1802.07814 215 | 216 | + (2018) Explanations of model predictions with live and breakDown packages by Mateusz Staniak, Przemyslaw Biecek 217 | + https://arxiv.org/pdf/1804.01955 218 | + Docs: https://mi2datalab.github.io/live/ 219 | + Code: https://github.com/MI2DataLab/live 220 | + Docs: https://pbiecek.github.io/breakDown 221 | + Code: https://github.com/pbiecek/breakDown 222 | 223 | + (2018) A review book - Interpretable Machine Learning. A Guide for Making Black Box 224 | Models Explainable by Christoph Molnar 225 | 226 | + https://christophm.github.io/interpretable-ml-book/ 227 | + (2018) Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead by Cynthia Rudin 228 | + https://arxiv.org/pdf/1811.10154 229 | + (2019) Quantifying Interpretability of Arbitrary Machine Learning Models Through Functional Decomposition by Christoph Molnar, Giuseppe Casalicchio, Bernd Bischl 230 | + https://arxiv.org/pdf/1904.03867 231 | 232 | *** Model Specific Explanations - Neural Networks 233 | + (2013) Visualizing and Understanding Convolutional Networks by Matthew D Zeiler, Rob Fergus 234 | + https://arxiv.org/pdf/1311.2901 235 | 236 | + (2013) Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps by Karen Simonyan, Andrea Vedaldi, Andrew Zisserman 237 | + https://arxiv.org/pdf/1312.6034 238 | 239 | + (2015) Understanding Neural Networks Through Deep Visualization by Jason Yosinski, Jeff Clune, Anh Nguyen, Thomas Fuchs, Hod Lipson 240 | + https://arxiv.org/pdf/1506.06579 241 | + https://github.com/yosinski/deep-visualization-toolbox 242 | 243 | + (2016) Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization by Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, Dhruv Batra 244 | + https://arxiv.org/pdf/1610.02391 245 | 246 | + (2016) Generating Visual Explanations by Lisa Anne Hendricks, Zeynep Akata, Marcus Rohrbach, Jeff Donahue, Bernt Schiele, Trevor Darrell 247 | + https://arxiv.org/pdf/1603.08507 248 | 249 | + (2016) Rationalizing Neural Predictions by Tao Lei, Regina Barzilay, Tommi Jaakkola 250 | + https://arxiv.org/pdf/1606.04155 251 | + https://people.csail.mit.edu/taolei/papers/emnlp16_rationale_slides.pdf 252 | + Code: https://github.com/taolei87/rcnn/tree/master/code/rationale 253 | 254 | + (2016) Gradients of Counterfactuals by Mukund Sundararajan, Ankur Taly, Qiqi Yan 255 | + https://arxiv.org/pdf/1611.02639 256 | 257 | + Pixel entropy can be used to detect relevant picture regions (for CovNets) 258 | + See Visualization section and Fig. 5 of the paper 259 | + (2017) High-Resolution Breast Cancer Screening with Multi-View Deep Convolutional Neural Networks by Krzysztof J. Geras, Stacey Wolfson, Yiqiu Shen, Nan Wu, S. Gene Kim, Eric Kim, Laura Heacock, Ujas Parikh, Linda Moy, Kyunghyun Cho 260 | + https://arxiv.org/pdf/1703.07047 261 | 262 | + (2017) SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability by Maithra Raghu, Justin Gilmer, Jason Yosinski, Jascha Sohl-Dickstein 263 | + https://arxiv.org/pdf/1706.05806 264 | + https://research.googleblog.com/2017/11/interpreting-deep-neural-networks-with.html 265 | 266 | + (2017) Visual Explanation by Interpretation: Improving Visual Feedback Capabilities of Deep Neural Networks by Jose Oramas, Kaili Wang, Tinne Tuytelaars 267 | + https://arxiv.org/pdf/1712.06302 268 | 269 | + (2017) Axiomatic Attribution for Deep Networks by Mukund Sundararajan, Ankur Taly, Qiqi Yan 270 | + https://arxiv.org/pdf/1703.01365 271 | + Code: https://github.com/ankurtaly/Integrated-Gradients 272 | + Proposes Integrated Gradients Method 273 | + See also: Gradients of Counterfactuals https://arxiv.org/pdf/1611.02639.pdf 274 | 275 | + (2017) Learning Important Features Through Propagating Activation Differences by Avanti Shrikumar, Peyton Greenside, Anshul Kundaje 276 | + https://arxiv.org/pdf/1704.02685 277 | 278 | + Proposes Deep Lift method 279 | 280 | + Code: https://github.com/kundajelab/deeplift 281 | 282 | + Videos: https://www.youtube.com/playlist?list=PLJLjQOkqSRTP3cLB2cOOi_bQFw6KPGKML 283 | 284 | + (2017) The (Un)reliability of saliency methods by Pieter-Jan Kindermans, Sara Hooker, Julius Adebayo, Maximilian Alber, Kristof T. Schütt, Sven Dähne, Dumitru Erhan, Been Kim 285 | + https://arxiv.org/pdf/1711.0867 286 | + Review of failures for methods extracting most important pixels for prediction 287 | 288 | + (2018) Classifier-agnostic saliency map extraction by Konrad Zolna, Krzysztof J. Geras, Kyunghyun Cho 289 | + https://arxiv.org/pdf/1805.08249 290 | + Code: https://github.com/kondiz/casme 291 | 292 | + (2018) A Benchmark for Interpretability Methods in Deep Neural Networks by Sara Hooker, Dumitru Erhan, Pieter-Jan Kindermans, Been Kim 293 | + https://arxiv.org/pdf/1806.10758 294 | 295 | + (2018) The Building Blocks of Interpretability by Chris Olah, Arvind Satyanarayan, Ian Johnson, Shan Carter, Ludwig Schubert, Katherine Ye, Alexander Mordvintsev 296 | + https://dx.doi.org/10.23915/distill.00010 297 | + Has some embeded links to notebooks 298 | + Uses Lucid library https://github.com/tensorflow/lucid 299 | 300 | + (2018) Hierarchical interpretations for neural network predictions by Chandan Singh, W. James Murdoch, Bin Yu 301 | + https://arxiv.org/pdf/1806.05337 302 | + Code: https://github.com/csinva/hierarchical_dnn_interpretations 303 | 304 | + (2018) iNNvestigate neural networks! by Maximilian Alber, Sebastian Lapuschkin, Philipp Seegerer, Miriam Hägele, Kristof T. Schütt, Grégoire Montavon, Wojciech Samek, Klaus-Robert Müller, Sven Dähne, Pieter-Jan Kindermans 305 | + https://arxiv.org/pdf/1808.04260 306 | + Code: https://github.com/albermax/innvestigate 307 | 308 | + (2018) YASENN: Explaining Neural Networks via Partitioning Activation Sequences by Yaroslav Zharov, Denis Korzhenkov, Pavel Shvechikov, Alexander Tuzhilin 309 | + https://arxiv.org/pdf/1811.02783 310 | 311 | + (2019) Attention is not Explanation by Sarthak Jain, Byron C. Wallace 312 | + https://arxiv.org/pdf/1902.10186 313 | 314 | + (2019) Attention Interpretability Across NLP Tasks by Shikhar Vashishth, Shyam Upadhyay, Gaurav Singh Tomar, Manaal Faruqui 315 | + https://arxiv.org/pdf/1909.11218 316 | 317 | + (2019) GRACE: Generating Concise and Informative Contrastive Sample to Explain Neural Network Model's Prediction by Thai Le, Suhang Wang, Dongwon Lee 318 | + https://arxiv.org/pdf/1911.02042 319 | + Code: https://github.com/lethaiq/GRACE_KDD20 320 | 321 | ** Extracting Interpretable Models From Complex Ones 322 | 323 | + (2017) Extracting Automata from Recurrent Neural Networks Using Queries and Counterexamples by Gail Weiss, Yoav Goldberg, Eran Yahav 324 | + https://arxiv.org/pdf/1711.09576 325 | 326 | + (2017) Distilling a Neural Network Into a Soft Decision Tree by Nicholas Frosst, Geoffrey Hinton 327 | + https://arxiv.org/pdf/1711.09784 328 | 329 | + (2017) Detecting Bias in Black-Box Models Using Transparent Model Distillation by Sarah Tan, Rich Caruana, Giles Hooker, Yin Lou 330 | + http://www.aies-conference.com/2018/contents/papers/main/AIES_2018_paper_96.pdf 331 | 332 | ** Model Visualization 333 | + Visualizing Statistical Models: Removing the blindfold 334 | + http://had.co.nz/stat645/model-vis.pdf 335 | 336 | + Partial dependence plots 337 | + http://scikit-learn.org/stable/auto_examples/ensemble/plot_partial_dependence.html 338 | + pdp: An R Package for Constructing Partial Dependence Plots 339 | https://journal.r-project.org/archive/2017/RJ-2017-016/RJ-2017-016.pdf 340 | https://cran.r-project.org/web/packages/pdp/index.html 341 | 342 | + ggfortify: Unified Interface to Visualize Statistical Results of Popular R Packages 343 | + https://journal.r-project.org/archive/2016-2/tang-horikoshi-li.pdf 344 | + CRAN https://cran.r-project.org/web/packages/ggfortify/index.html 345 | 346 | + RandomForestExplainer 347 | + Master thesis https://rawgit.com/geneticsMiNIng/BlackBoxOpener/master/randomForestExplainer_Master_thesis.pdf 348 | + R code 349 | + CRAN https://cran.r-project.org/web/packages/randomForestExplainer/index.html 350 | + Code: https://github.com/MI2DataLab/randomForestExplainer 351 | 352 | + ggRandomForest 353 | + Paper (vignette) https://github.com/ehrlinger/ggRandomForests/raw/master/vignettes/randomForestSRC-Survival.pdf 354 | + R code 355 | + CRAN https://cran.r-project.org/web/packages/ggRandomForests/index.html 356 | + Code: https://github.com/ehrlinger/ggRandomForests 357 | 358 | ** Selected Review Talks and Tutorials 359 | + Tutorial on Interpretable machine learning at ICML 2017 360 | + Slides: http://people.csail.mit.edu/beenkim/papers/BeenK_FinaleDV_ICML2017_tutorial.pdf 361 | 362 | + P. Biecek, Show Me Your Model - Tools for Visualisation of Statistical Models 363 | + Video: https://channel9.msdn.com/Events/useR-international-R-User-conferences/useR-International-R-User-2017-Conference/Show-Me-Your-Model-tools-for-visualisation-of-statistical-models 364 | 365 | + S. Ritchie, Just-So Stories of AI 366 | + Video: https://www.youtube.com/watch?v=DiWkKqZChF0 367 | + Slides: https://speakerdeck.com/sritchie/just-so-stories-for-ai-explaining-black-box-predictions 368 | 369 | + C. Jarmul, Towards Interpretable Accountable Models 370 | + Video: https://www.youtube.com/watch?v=B3PtcF-6Dtc 371 | + Slides: https://docs.google.com/presentation/d/e/2PACX-1vR05kpagAbL5qo1QThxwu44TI5SQAws_UFVg3nUAmKp39uNG0xdBjcMA-VyEeqZRGGQtt0CS5h2DMTS/embed?start=false&loop=false&delayms=3000 372 | 373 | + I. Oszvald, Machine Learning Libraries You'd Wish You'd Known About 374 | + A large part of the talk covers model explanation and visualization 375 | + Video: https://www.youtube.com/watch?v=nDF7_8FOhpI 376 | + Associated notebook on explaining regression predictions: https://github.com/ianozsvald/data_science_delivered/blob/master/ml_explain_regression_prediction.ipynb 377 | 378 | + G. Varoquaux, Understanding and diagnosing your machine-learning models (covers PDP and Lime among others) 379 | + Video: https://www.youtube.com/watch?v=kbj3llSbaVA 380 | + Slides: http://gael-varoquaux.info/interpreting_ml_tuto/ 381 | 382 | ** Venues 383 | + Interpretable ML Symposium (NIPS 2017) (contains links to *papers*, *slides* and *videos*) 384 | + http://interpretable.ml/ 385 | + Debate, Interpretability is necessary in machine learning 386 | + https://www.youtube.com/watch?v=2hW05ZfsUUo 387 | + Workshop on Human Interpretability in Machine Learning (WHI), organised in conjunction with ICML 388 | + 2018 (contains links to *papers* and *slides*) 389 | + https://sites.google.com/view/whi2018 390 | + Proceedings https://arxiv.org/html/1807.01308 391 | + 2017 (contains links to *papers* and *slides*) 392 | + https://sites.google.com/view/whi2017/home 393 | + Proceedings https://arxiv.org/html/1708.02666 394 | + 2016 (contains links to *papers*) 395 | + https://sites.google.com/site/2016whi/ 396 | + Proceedings https://arxiv.org/html/1607.02531 or [[https://drive.google.com/open?id=0B9mGJ4F63iKGZWk0cXZraTNjRVU][here]] 397 | + Analyzing and interpreting neural networks for NLP (BlackboxNLP), organised in conjunction with EMNLP 398 | + 2019 (links below may get prefixed by 2019 later on) 399 | + https://blackboxnlp.github.io/ 400 | + https://blackboxnlp.github.io/program.html 401 | + Papers should be available on arXiv 402 | + 2018 403 | + https://blackboxnlp.github.io/2018 404 | + https://blackboxnlp.github.io/program.html 405 | + [[https://arxiv.org/search/advanced?advanced=&terms-0-operator=AND&terms-0-term=BlackboxNLP&terms-0-field=comments&terms-1-operator=OR&terms-1-term=Analyzing+interpreting+neural+networks+NLP&terms-1-field=comments&classification-physics_archives=all&date-filter_by=all_dates&date-year=&date-from_date=&date-to_date=&date-date_type=submitted_date&abstracts=show&size=200&order=-announced_date_first][List of papers]] 406 | + FAT/ML Fairness, Accountability, and Transparency in Machine Learning [[https://www.fatml.org/]] 407 | + 2018 408 | + https://www.fatml.org/schedule/2018 409 | + 2017 410 | + https://www.fatml.org/schedule/2017 411 | + 2016 412 | + https://www.fatml.org/schedule/2016 413 | + 2016 414 | + https://www.fatml.org/schedule/2016 415 | + 2015 416 | + https://www.fatml.org/schedule/2015 417 | + 2014 418 | + https://www.fatml.org/schedule/2014 419 | + AAAI/ACM Annual Conferenceon AI, Ethics, and Society 420 | + 2019 (links below may get prefixed by 2019 later on) 421 | + http://www.aies-conference.com/accepted-papers/ 422 | + 2018 423 | + http://www.aies-conference.com/2018/accepted-papers/ 424 | + http://www.aies-conference.com/2018/accepted-student-papers/ 425 | ** Software 426 | Software related to papers is mentioned along with each publication. 427 | Here only standalone software is included. 428 | 429 | + DALEX - R package, Descriptive mAchine Learning EXplanations 430 | + CRAN https://cran.r-project.org/web/packages/DALEX/DALEX.pdf 431 | + Code: https://github.com/pbiecek/DALEX 432 | 433 | + ELI5 - Python package dedicated to debugging machine learning classifiers 434 | and explaining their predictions 435 | + Code: https://github.com/TeamHG-Memex/eli5 436 | + https://eli5.readthedocs.io/en/latest/ 437 | 438 | + forestmodel - R package visualizing coefficients of different models with the so called forest plot 439 | + CRAN https://cran.r-project.org/web/packages/forestmodel/index.html 440 | + Code: https://github.com/NikNakk/forestmodel 441 | 442 | + fscaret - R package with automated Feature Selection from 'caret' 443 | + CRAN https://cran.r-project.org/web/packages/fscaret/ 444 | + Tutorial: https://cran.r-project.org/web/packages/fscaret/vignettes/fscaret.pdf 445 | 446 | + iml - R package for Interpretable Machine Learning 447 | + CRAN https://cran.r-project.org/web/packages/iml/ 448 | + Code: https://github.com/christophM/iml 449 | + Publication: http://joss.theoj.org/papers/10.21105/joss.00786 450 | 451 | + interpret - Python package package for training interpretable models and explaining blackbox systems by Microsoft 452 | + Code: https://github.com/microsoft/interpret 453 | 454 | + lime - R package implementing LIME 455 | + https://github.com/thomasp85/lime 456 | 457 | + lofo-importance - Python package feature importance by Leave One Feature Out Importance method 458 | + Code: https://github.com/aerdem4/lofo-importance 459 | 460 | + Lucid - a collection of infrastructure and tools for research in neural network interpretability 461 | + Code: https://github.com/tensorflow/lucid 462 | 463 | + praznik - R package with a collection of feature selection filters performing greedy optimisation of mutual information-based usefulness criteria, see JMLR 13, 27−66 (2012) 464 | + CRAN https://cran.r-project.org/web/packages/praznik/index.html 465 | + Code: https://notabug.org/mbq/praznik 466 | 467 | + yellowbrick - Python package offering visual analysis and diagnostic tools to facilitate machine learning model selection 468 | + Code: https://github.com/DistrictDataLabs/yellowbrick 469 | + http://www.scikit-yb.org/en/latest/ 470 | 471 | ** Other Resources 472 | + *Awesome* list of resources by Patrick Hall 473 | + https://github.com/jphall663/awesome-machine-learning-interpretability 474 | + *Awesome* XAI resources by Przemysław Biecek 475 | + https://github.com/pbiecek/xai_resources 476 | -------------------------------------------------------------------------------- /sbin/README.template.org: -------------------------------------------------------------------------------- 1 | * Awesome Interpretable Machine Learning [[https://awesome.re][https://awesome.re/badge.svg]] 2 | 3 | Opinionated list of resources facilitating model interpretability 4 | (introspection, simplification, visualization, explanation). 5 | 6 | ** Interpretable Models 7 | + Interpretable models 8 | + Simple decision trees 9 | + Rules 10 | + (Regularized) linear regression 11 | + k-NN 12 | 13 | + { "doi" : "10.1214/07-AOAS148" } 14 | 15 | + { "doi" : "10.1145/2594473.2594475" } 16 | + http://www.kdd.org/exploration_files/V15-01-01-Freitas.pdf 17 | + Interesting discussion of interpretability for a few classification models 18 | (decision trees, classification rules, decision tables, nearest neighbors and Bayesian network classifier) 19 | 20 | + { "arxiv_id": "1511.01644" } 21 | 22 | + { "arxiv_id" : "1711.04574" } 23 | 24 | + { "arxiv_id" : "1912.04695" } 25 | + Code: https://github.com/12wang3/mllp 26 | 27 | ** Feature Importance 28 | + Models offering feature importance measures 29 | + Random forest 30 | + Boosted trees 31 | + Extremely randomized trees 32 | + { "doi" : "10.1007/s10994-006-6226-1" } 33 | + Random ferns 34 | + { "doi" : "10.18637/jss.v061.i10" } 35 | + https://cran.r-project.org/web/packages/rFerns 36 | + https://notabug.org/mbq/rFerns 37 | + Linear regression (with a grain of salt) 38 | 39 | + { "doi" : "10.1186/1471-2105-8-25", "title" : "Bias in random forest variable importance measures: Illustrations, sources and a solution" } 40 | 41 | + { "doi" : "10.1186/1471-2105-9-307" } 42 | 43 | + { "arxiv_id" : "1801.01489" } 44 | + https://github.com/aaronjfisher/mcr 45 | + Universal (model agnostic) variable importance measure 46 | 47 | + { "arxiv_id": "1905.03151" } 48 | + Paper advocating against feature permutation for importance 49 | 50 | + { "arxiv_id" : "1804.06620" } 51 | + https://github.com/giuseppec/featureImportance 52 | + Global and local (model agnostic) variable importance measure (based on Model Reliance) 53 | 54 | + Very good blog post describing deficiencies of random forest feature importance and the permutation importance 55 | + http://explained.ai/rf-importance/index.html 56 | 57 | + Permutation importance - simple model agnostic approach is described in Eli5 documentation 58 | + https://eli5.readthedocs.io/en/latest/blackbox/permutation_importance.html 59 | 60 | ** Feature Selection 61 | + Classification of feature selection methods 62 | + Filters 63 | + Wrappers 64 | + Embedded methods 65 | 66 | + { "sems_id" : "5264ae4ea4411426ddd91dc780c2892c3ff933d3", "skip_doi" : true } 67 | + http://www.jmlr.org/papers/volume3/guyon03a/guyon03a.pdf 68 | + Be sure to read this very illustrative introduction to feature selection 69 | 70 | + Filter Methods 71 | 72 | + { "sems_id" : "d72ff5063520ce4542d6d9b9e6a4f12aafab6091" } 73 | + https://pdfs.semanticscholar.org/d72f/f5063520ce4542d6d9b9e6a4f12aafab6091.pdf 74 | + Introduces information theoretic methods - double input symmetrical relevance (DISR) 75 | 76 | + { "sems_id" : "2629632a8032c02ad25021dbea82184077117d78" } 77 | + http://www.jmlr.org/papers/volume13/brown12a/brown12a.pdf 78 | + Code: https://github.com/Craigacp/FEAST 79 | + Discusses various approaches based on mutual information (MIM, mRMR, MIFS, CMIM, JMI, DISR, ICAP, CIFE, CMI) 80 | 81 | + { "sems_id" : "5a2f4bdab1499314a24e3a8d3c88dbfc6f6ecaf6" } 82 | + http://www.cs.man.ac.uk/~gbrown/publications/pocockPhDthesis.pdf 83 | 84 | + { "arxiv_id" : "1711.08421" } 85 | 86 | + { "arxiv_id" : "1711.08477" } 87 | 88 | + Wrapper methods 89 | 90 | + { "doi" : "10.18637/jss.v036.i11" } 91 | + https://cran.r-project.org/web/packages/Boruta/ 92 | + Code (official, R): https://notabug.org/mbq/Boruta/ 93 | + Code (Python): https://github.com/scikit-learn-contrib/boruta_py 94 | 95 | + Boruta for those in a hurry 96 | + https://cran.r-project.org/web/packages/Boruta/vignettes/inahurry.pdf 97 | 98 | + General 99 | 100 | + { "sems_id" : "a93a1d9c9a1f46dc5aaadabedf906823d5043173" } 101 | + https://pdfs.semanticscholar.org/a83b/ddb34618cc68f1014ca12eef7f537825d104.pdf 102 | + Classic paper discussing weakly relevant features, irrelevant features, strongly relevant features 103 | 104 | + (2003) Special issue of JMLR of feature selection - oldish (2003) 105 | + http://www.jmlr.org/papers/special/feature03.html 106 | 107 | + { "sems_id" : "b14dea76cafede81c6ff5478d4221fce3aec9284" } 108 | + Paper: https://papers.nips.cc/paper/2728-result-analysis-of-the-nips-2003-feature-selection-challenge.pdf 109 | + Website http://clopinet.com/isabelle/Projects/NIPS2003/ 110 | 111 | + { "sems_id" : "00bc2153cd21001860e43758273945110f0ae40d" } 112 | + http://www.jmlr.org/papers/volume8/nilsson07a/nilsson07a.pdf 113 | + Discusses minimal optimal vs all-relevant approaches to feature selection 114 | 115 | + Feature Engineering and Selection by Kuhn & Johnson 116 | + Sligtly off-topic, but very interesting book 117 | + http://www.feat.engineering/index.html 118 | + https://bookdown.org/max/FES/ 119 | + https://github.com/topepo/FES 120 | 121 | + Feature Engineering presentation by H. J. van Veen 122 | + Slightly off-topicm but very interesting deck of slides 123 | + Slides: https://www.slideshare.net/HJvanVeen/feature-engineering-72376750 124 | 125 | ** Model Explanations 126 | *** Philosophy 127 | + Magnets by R. P. Feynman 128 | https://www.youtube.com/watch?v=wMFPe-DwULM 129 | 130 | + (2002) Looking Inside the Black Box, presentation of Leo Breiman 131 | + https://www.stat.berkeley.edu/users/breiman/wald2002-2.pdf 132 | 133 | + { "arxiv_id" : "1101.0891" } 134 | 135 | + { "arxiv_id" : "1606.03490" } 136 | + https://www.youtube.com/watch?v=mvzBQci04qA 137 | 138 | + { "arxiv_id" : "1702.08608" } 139 | 140 | + { "arxiv_id" : "1711.07414" } 141 | 142 | + (2018) [[http://bayes.cs.ucla.edu/WHY/why-intro.pdf][The Book of Why: The New Science of Cause and Effect]] by Judea Pearl 143 | 144 | + (2018) Please Stop Doing the "Explainable" ML by Cynthia Rudin 145 | + Video (starts 17:30, lasts 10 min): https://zoom.us/recording/play/0y-iI9HamgyDzzP2k_jiTu6jB7JgVVXnjWZKDMbnyRTn3FsxTDZy6Wkrj3_ekx4J 146 | + Linked at: https://users.cs.duke.edu/~cynthia/mediatalks.html 147 | 148 | + { "arxiv_id" : "1806.00069" } 149 | 150 | + { "arxiv_id" : "1901.04592" } 151 | 152 | + (2019) On Explainable Machine Learning Misconceptions A More Human-Centered Machine Learning by Patrick Hall 153 | + https://github.com/jphall663/xai_misconceptions/blob/master/xai_misconceptions.pdf 154 | + https://github.com/jphall663/xai_misconceptions 155 | 156 | + (2019) An Introduction to Machine Learning Interpretability. An Applied Perspective on Fairness, Accountability, Transparency, and Explainable AI by Patrick Hall and Navdeep Gill 157 | + https://www.h2o.ai/wp-content/uploads/2019/08/An-Introduction-to-Machine-Learning-Interpretability-Second-Edition.pdf 158 | 159 | *** Model Agnostic Explanations 160 | + { "arxiv_id" : "0912.1128" } 161 | 162 | + { "arxiv_id" : "1309.6392" } 163 | 164 | + { "arxiv_id" : "1602.04938" } 165 | + Code: https://github.com/marcotcr/lime 166 | + https://github.com/marcotcr/lime-experiments 167 | + https://www.youtube.com/watch?v=bCgEP2zuYxI 168 | + Introduces the LIME method (Local Interpretable Model-agnostic Explanations) 169 | 170 | + { "arxiv_id" : "1606.09517" } 171 | + http://www.blackboxworkshop.org/pdf/Turner2015_MES.pdf 172 | 173 | + { "arxiv_id" : "1703.04730" } 174 | 175 | + { "arxiv_id" : "1705.07874" } 176 | + Code: https://github.com/slundberg/shap 177 | + Introduces the SHAP method (SHapley Additive exPlanations), generalizing LIME 178 | 179 | + { "sems_id" : "1c6690ab404b23d5026dd3ad0c7a49ce2875c1b3" } 180 | + https://homes.cs.washington.edu/~marcotcr/aaai18.pdf 181 | + Code: https://github.com/marcotcr/anchor-experiments 182 | 183 | + { "arxiv_id" : "1802.07814" } 184 | 185 | + { "arxiv_id" : "1804.01955" } 186 | + Docs: https://mi2datalab.github.io/live/ 187 | + Code: https://github.com/MI2DataLab/live 188 | + Docs: https://pbiecek.github.io/breakDown 189 | + Code: https://github.com/pbiecek/breakDown 190 | 191 | + (2018) A review book - Interpretable Machine Learning. A Guide for Making Black Box 192 | Models Explainable by Christoph Molnar 193 | 194 | + https://christophm.github.io/interpretable-ml-book/ 195 | + { "arxiv_id" : "1811.10154" } 196 | + { "arxiv_id" : "1904.03867" } 197 | 198 | *** Model Specific Explanations - Neural Networks 199 | + { "arxiv_id" : "1311.2901" } 200 | 201 | + { "arxiv_id" : "1312.6034" } 202 | 203 | + { "arxiv_id" : "1506.06579" } 204 | + https://github.com/yosinski/deep-visualization-toolbox 205 | 206 | + { "arxiv_id" : "1610.02391" } 207 | 208 | + { "arxiv_id" : "1603.08507" } 209 | 210 | + { "arxiv_id" : "1606.04155" } 211 | + https://people.csail.mit.edu/taolei/papers/emnlp16_rationale_slides.pdf 212 | + Code: https://github.com/taolei87/rcnn/tree/master/code/rationale 213 | 214 | + { "arxiv_id" : "1611.02639" } 215 | 216 | + Pixel entropy can be used to detect relevant picture regions (for CovNets) 217 | + See Visualization section and Fig. 5 of the paper 218 | + { "arxiv_id" : "1703.07047" } 219 | 220 | + { "arxiv_id" : "1706.05806" } 221 | + https://research.googleblog.com/2017/11/interpreting-deep-neural-networks-with.html 222 | 223 | + { "arxiv_id" : "1712.06302" } 224 | 225 | + { "arxiv_id" : "1703.01365" } 226 | + Code: https://github.com/ankurtaly/Integrated-Gradients 227 | + Proposes Integrated Gradients Method 228 | + See also: Gradients of Counterfactuals https://arxiv.org/pdf/1611.02639.pdf 229 | 230 | + { "arxiv_id" : "1704.02685" } 231 | 232 | + Proposes Deep Lift method 233 | 234 | + Code: https://github.com/kundajelab/deeplift 235 | 236 | + Videos: https://www.youtube.com/playlist?list=PLJLjQOkqSRTP3cLB2cOOi_bQFw6KPGKML 237 | 238 | + { "arxiv_id" : "1711.0867" } 239 | + Review of failures for methods extracting most important pixels for prediction 240 | 241 | + { "arxiv_id" : "1805.08249" } 242 | + Code: https://github.com/kondiz/casme 243 | 244 | + { "arxiv_id" : "1806.10758" } 245 | 246 | + { "doi" : "10.23915/distill.00010" } 247 | + Has some embeded links to notebooks 248 | + Uses Lucid library https://github.com/tensorflow/lucid 249 | 250 | + { "arxiv_id" : "1806.05337"} 251 | + Code: https://github.com/csinva/hierarchical_dnn_interpretations 252 | 253 | + { "arxiv_id" : "1808.04260" } 254 | + Code: https://github.com/albermax/innvestigate 255 | 256 | + { "arxiv_id" : "1811.02783" } 257 | 258 | + { "arxiv_id" : "1902.10186" } 259 | 260 | + { "arxiv_id": "1909.11218" } 261 | 262 | + { "arxiv_id": "1911.02042" } 263 | + Code: https://github.com/lethaiq/GRACE_KDD20 264 | 265 | ** Extracting Interpretable Models From Complex Ones 266 | 267 | + { "arxiv_id" : "1711.09576" } 268 | 269 | + { "arxiv_id" : "1711.09784" } 270 | 271 | + { "sems_id" : "94770d1ea1192c05a6922712200689344a742d81" } 272 | + http://www.aies-conference.com/2018/contents/papers/main/AIES_2018_paper_96.pdf 273 | 274 | ** Model Visualization 275 | + Visualizing Statistical Models: Removing the blindfold 276 | + http://had.co.nz/stat645/model-vis.pdf 277 | 278 | + Partial dependence plots 279 | + http://scikit-learn.org/stable/auto_examples/ensemble/plot_partial_dependence.html 280 | + pdp: An R Package for Constructing Partial Dependence Plots 281 | https://journal.r-project.org/archive/2017/RJ-2017-016/RJ-2017-016.pdf 282 | https://cran.r-project.org/web/packages/pdp/index.html 283 | 284 | + ggfortify: Unified Interface to Visualize Statistical Results of Popular R Packages 285 | + https://journal.r-project.org/archive/2016-2/tang-horikoshi-li.pdf 286 | + CRAN https://cran.r-project.org/web/packages/ggfortify/index.html 287 | 288 | + RandomForestExplainer 289 | + Master thesis https://rawgit.com/geneticsMiNIng/BlackBoxOpener/master/randomForestExplainer_Master_thesis.pdf 290 | + R code 291 | + CRAN https://cran.r-project.org/web/packages/randomForestExplainer/index.html 292 | + Code: https://github.com/MI2DataLab/randomForestExplainer 293 | 294 | + ggRandomForest 295 | + Paper (vignette) https://github.com/ehrlinger/ggRandomForests/raw/master/vignettes/randomForestSRC-Survival.pdf 296 | + R code 297 | + CRAN https://cran.r-project.org/web/packages/ggRandomForests/index.html 298 | + Code: https://github.com/ehrlinger/ggRandomForests 299 | 300 | ** Selected Review Talks and Tutorials 301 | + Tutorial on Interpretable machine learning at ICML 2017 302 | + Slides: http://people.csail.mit.edu/beenkim/papers/BeenK_FinaleDV_ICML2017_tutorial.pdf 303 | 304 | + P. Biecek, Show Me Your Model - Tools for Visualisation of Statistical Models 305 | + Video: https://channel9.msdn.com/Events/useR-international-R-User-conferences/useR-International-R-User-2017-Conference/Show-Me-Your-Model-tools-for-visualisation-of-statistical-models 306 | 307 | + S. Ritchie, Just-So Stories of AI 308 | + Video: https://www.youtube.com/watch?v=DiWkKqZChF0 309 | + Slides: https://speakerdeck.com/sritchie/just-so-stories-for-ai-explaining-black-box-predictions 310 | 311 | + C. Jarmul, Towards Interpretable Accountable Models 312 | + Video: https://www.youtube.com/watch?v=B3PtcF-6Dtc 313 | + Slides: https://docs.google.com/presentation/d/e/2PACX-1vR05kpagAbL5qo1QThxwu44TI5SQAws_UFVg3nUAmKp39uNG0xdBjcMA-VyEeqZRGGQtt0CS5h2DMTS/embed?start=false&loop=false&delayms=3000 314 | 315 | + I. Oszvald, Machine Learning Libraries You'd Wish You'd Known About 316 | + A large part of the talk covers model explanation and visualization 317 | + Video: https://www.youtube.com/watch?v=nDF7_8FOhpI 318 | + Associated notebook on explaining regression predictions: https://github.com/ianozsvald/data_science_delivered/blob/master/ml_explain_regression_prediction.ipynb 319 | 320 | + G. Varoquaux, Understanding and diagnosing your machine-learning models (covers PDP and Lime among others) 321 | + Video: https://www.youtube.com/watch?v=kbj3llSbaVA 322 | + Slides: http://gael-varoquaux.info/interpreting_ml_tuto/ 323 | 324 | ** Venues 325 | + Interpretable ML Symposium (NIPS 2017) (contains links to *papers*, *slides* and *videos*) 326 | + http://interpretable.ml/ 327 | + Debate, Interpretability is necessary in machine learning 328 | + https://www.youtube.com/watch?v=2hW05ZfsUUo 329 | + Workshop on Human Interpretability in Machine Learning (WHI), organised in conjunction with ICML 330 | + 2018 (contains links to *papers* and *slides*) 331 | + https://sites.google.com/view/whi2018 332 | + Proceedings https://arxiv.org/html/1807.01308 333 | + 2017 (contains links to *papers* and *slides*) 334 | + https://sites.google.com/view/whi2017/home 335 | + Proceedings https://arxiv.org/html/1708.02666 336 | + 2016 (contains links to *papers*) 337 | + https://sites.google.com/site/2016whi/ 338 | + Proceedings https://arxiv.org/html/1607.02531 or [[https://drive.google.com/open?id=0B9mGJ4F63iKGZWk0cXZraTNjRVU][here]] 339 | + Analyzing and interpreting neural networks for NLP (BlackboxNLP), organised in conjunction with EMNLP 340 | + 2019 (links below may get prefixed by 2019 later on) 341 | + https://blackboxnlp.github.io/ 342 | + https://blackboxnlp.github.io/program.html 343 | + Papers should be available on arXiv 344 | + 2018 345 | + https://blackboxnlp.github.io/2018 346 | + https://blackboxnlp.github.io/program.html 347 | + [[https://arxiv.org/search/advanced?advanced=&terms-0-operator=AND&terms-0-term=BlackboxNLP&terms-0-field=comments&terms-1-operator=OR&terms-1-term=Analyzing+interpreting+neural+networks+NLP&terms-1-field=comments&classification-physics_archives=all&date-filter_by=all_dates&date-year=&date-from_date=&date-to_date=&date-date_type=submitted_date&abstracts=show&size=200&order=-announced_date_first][List of papers]] 348 | + FAT/ML Fairness, Accountability, and Transparency in Machine Learning [[https://www.fatml.org/]] 349 | + 2018 350 | + https://www.fatml.org/schedule/2018 351 | + 2017 352 | + https://www.fatml.org/schedule/2017 353 | + 2016 354 | + https://www.fatml.org/schedule/2016 355 | + 2016 356 | + https://www.fatml.org/schedule/2016 357 | + 2015 358 | + https://www.fatml.org/schedule/2015 359 | + 2014 360 | + https://www.fatml.org/schedule/2014 361 | + AAAI/ACM Annual Conferenceon AI, Ethics, and Society 362 | + 2019 (links below may get prefixed by 2019 later on) 363 | + http://www.aies-conference.com/accepted-papers/ 364 | + 2018 365 | + http://www.aies-conference.com/2018/accepted-papers/ 366 | + http://www.aies-conference.com/2018/accepted-student-papers/ 367 | ** Software 368 | Software related to papers is mentioned along with each publication. 369 | Here only standalone software is included. 370 | 371 | + DALEX - R package, Descriptive mAchine Learning EXplanations 372 | + CRAN https://cran.r-project.org/web/packages/DALEX/DALEX.pdf 373 | + Code: https://github.com/pbiecek/DALEX 374 | 375 | + ELI5 - Python package dedicated to debugging machine learning classifiers 376 | and explaining their predictions 377 | + Code: https://github.com/TeamHG-Memex/eli5 378 | + https://eli5.readthedocs.io/en/latest/ 379 | 380 | + forestmodel - R package visualizing coefficients of different models with the so called forest plot 381 | + CRAN https://cran.r-project.org/web/packages/forestmodel/index.html 382 | + Code: https://github.com/NikNakk/forestmodel 383 | 384 | + fscaret - R package with automated Feature Selection from 'caret' 385 | + CRAN https://cran.r-project.org/web/packages/fscaret/ 386 | + Tutorial: https://cran.r-project.org/web/packages/fscaret/vignettes/fscaret.pdf 387 | 388 | + iml - R package for Interpretable Machine Learning 389 | + CRAN https://cran.r-project.org/web/packages/iml/ 390 | + Code: https://github.com/christophM/iml 391 | + Publication: http://joss.theoj.org/papers/10.21105/joss.00786 392 | 393 | + interpret - Python package package for training interpretable models and explaining blackbox systems by Microsoft 394 | + Code: https://github.com/microsoft/interpret 395 | 396 | + lime - R package implementing LIME 397 | + https://github.com/thomasp85/lime 398 | 399 | + lofo-importance - Python package feature importance by Leave One Feature Out Importance method 400 | + Code: https://github.com/aerdem4/lofo-importance 401 | 402 | + Lucid - a collection of infrastructure and tools for research in neural network interpretability 403 | + Code: https://github.com/tensorflow/lucid 404 | 405 | + praznik - R package with a collection of feature selection filters performing greedy optimisation of mutual information-based usefulness criteria, see JMLR 13, 27−66 (2012) 406 | + CRAN https://cran.r-project.org/web/packages/praznik/index.html 407 | + Code: https://notabug.org/mbq/praznik 408 | 409 | + yellowbrick - Python package offering visual analysis and diagnostic tools to facilitate machine learning model selection 410 | + Code: https://github.com/DistrictDataLabs/yellowbrick 411 | + http://www.scikit-yb.org/en/latest/ 412 | 413 | ** Other Resources 414 | + *Awesome* list of resources by Patrick Hall 415 | + https://github.com/jphall663/awesome-machine-learning-interpretability 416 | + *Awesome* XAI resources by Przemysław Biecek 417 | + https://github.com/pbiecek/xai_resources 418 | -------------------------------------------------------------------------------- /sbin/gener_readme.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | import argparse 4 | import collections 5 | import json 6 | import re 7 | import os 8 | import sys 9 | import time 10 | 11 | import urllib.request 12 | import xml.etree.ElementTree as ET 13 | 14 | KEY_FIELD_NAME = '0KEY_' 15 | 16 | def parse_argv(argv): 17 | parser = argparse.ArgumentParser() 18 | parser.add_argument('--readme-template', dest='readme_template', 19 | action='store', required=True) 20 | parser.add_argument('--cache-fname', dest='cache_fname', 21 | action='store', required=False, default=None) 22 | return parser.parse_args(argv) 23 | 24 | 25 | def load_cache(cache_fname): 26 | cache = {} 27 | if os.path.isfile(cache_fname): 28 | with open(cache_fname, 'rt') as f: 29 | for line in f: 30 | r = json.loads(line) 31 | key = r[KEY_FIELD_NAME] 32 | cache[key] = r 33 | return cache 34 | 35 | 36 | def save_cache(cache_fname, cache): 37 | 38 | with open(cache_fname, 'wt') as f: 39 | for k in sorted(cache.keys()): 40 | f.write(json.dumps(cache[k], sort_keys=True)+'\n') 41 | 42 | 43 | def extract_metadata(line): 44 | ENUMERATION_CHARS = { '+' } 45 | BEG_REFERENCE_CHAR = '{' 46 | END_REFERENCE_CHAR = '}' 47 | 48 | def _get_line_prefix(line, enumerator_char): 49 | return line.split(sep=enumerator_char, maxsplit=1)[0] 50 | 51 | res = None 52 | line_stripped = line.strip() 53 | if line_stripped and line_stripped[0] in ENUMERATION_CHARS: 54 | line_reference = line_stripped[1:].strip() 55 | if (line_reference[0] == BEG_REFERENCE_CHAR and line_reference[-1] == END_REFERENCE_CHAR): 56 | enumerator_char = line_stripped[0] 57 | res = json.loads(line_reference) 58 | res['enumerator_char'] = enumerator_char 59 | res['line_prefix'] = _get_line_prefix(line, enumerator_char) 60 | return res 61 | 62 | 63 | def get_cache_key(src_code, src_id): 64 | return src_code + ':' + src_id 65 | 66 | 67 | def parse_generic_xml(root): 68 | 69 | ATTRIB_KEY = '_attrib' 70 | 71 | def _normalize_tag(s): 72 | i = s.rfind('}') 73 | return s[(i+1):] 74 | 75 | def _insert_and_listify(res, key, val): 76 | 77 | if key in res: 78 | val_cur = res[key] 79 | if isinstance(val_cur, list): 80 | res[key].append(val) 81 | else: 82 | res[key] = [val_cur] + [val] 83 | else: 84 | res[key] = val 85 | return res 86 | 87 | res = {} 88 | 89 | for child in root: 90 | if len(child) > 0: 91 | key = _normalize_tag(child.tag) 92 | val = parse_generic_xml(child) 93 | if len(child.attrib) > 0: 94 | val[ATTRIB_KEY] = child.attrib 95 | res = _insert_and_listify(res, key, val) 96 | else: 97 | key = _normalize_tag(child.tag) 98 | val = child.text 99 | if len(child.attrib) > 0: 100 | val_new = { 'val': val, ATTRIB_KEY: child.attrib } 101 | val = val_new 102 | res = _insert_and_listify(res, key, val) 103 | return res 104 | 105 | 106 | def fetch_raw_metadata_arxiv(src_id): 107 | OAI_PMH_URL = 'http://export.arxiv.org/oai2' 108 | QUERY_FORMAT = '?verb=GetRecord&identifier=oai:arXiv.org:%s&metadataPrefix=arXiv' 109 | 110 | query = QUERY_FORMAT % src_id 111 | finished = False 112 | while not finished: 113 | try: 114 | url = OAI_PMH_URL + query 115 | #print('Fetching %s' % url, file=sys.stderr) 116 | result = urllib.request.urlopen(url).read() 117 | time.sleep(5) 118 | finished = True 119 | except urllib.error.HTTPError as e: 120 | if e.code == 503: 121 | retry_after = int(e.headers['Retry-After']) 122 | print('Sleeping %d...' % retry_after, file=sys.stderr) 123 | time.sleep(retry_after) 124 | else: 125 | raise e 126 | res=parse_generic_xml(ET.fromstring(result)) 127 | return res 128 | 129 | 130 | def clean_raw_metadata_arxiv(met_raw): 131 | 132 | def _normalize_title(title): 133 | title_norm = title.replace('\n', ' ') 134 | title_norm = re.sub(r' +', ' ', title_norm) 135 | return title_norm 136 | 137 | 138 | def _normalize_authors(authors): 139 | return [ [ a['keyname'], a['forenames'] ] for a in authors ] 140 | 141 | 142 | def _normalize_arxiv_id(arxiv_id): 143 | return arxiv_id.replace('oai:arXiv.org:', '') 144 | 145 | met = {} 146 | title_raw = met_raw['GetRecord']['record']['metadata']['arXiv']['title'] 147 | authors_raw = met_raw['GetRecord']['record']['metadata']['arXiv']['authors']['author'] 148 | if not isinstance(authors_raw, collections.Sequence): 149 | authors_raw = [ authors_raw ] 150 | met['title'] = _normalize_title(title_raw) 151 | met['authors'] = _normalize_authors(authors_raw) 152 | arxiv_id_raw = met_raw['GetRecord']['record']['header']['identifier'] 153 | met['arxiv_id'] = _normalize_arxiv_id(arxiv_id_raw) 154 | created = met_raw['GetRecord']['record']['metadata']['arXiv']['created'] 155 | 156 | if created is not None: 157 | met['year'] = created[0:4] 158 | try: 159 | doi = met_raw['GetRecord']['record']['metadata']['arXiv']['doi'] 160 | met['doi'] = doi 161 | except Exception as e: 162 | met['doi'] = None 163 | return met 164 | 165 | 166 | def fetch_raw_metadata_doi(src_id): 167 | CROSSREF_API_URL = 'http://api.crossref.org/works/' 168 | 169 | url = CROSSREF_API_URL + src_id 170 | #print('Fetching %s' % url, file=sys.stderr) 171 | result = urllib.request.urlopen(url).read() 172 | time.sleep(5) 173 | finished = True 174 | return json.loads(result.decode('utf-8')) 175 | 176 | 177 | def clean_raw_metadata_doi(met_raw): 178 | title_parts = met_raw['message']['title'] 179 | 180 | met = {} 181 | 182 | title='' 183 | for t in title_parts: 184 | title += ' ' + t 185 | 186 | authors =[ [ a['family'], a['given'] ] for a in met_raw['message']['author'] ] 187 | 188 | met['title'] = title[1:] 189 | met['authors'] = authors 190 | met['doi'] = met_raw['message']['DOI'] 191 | met['year'] = str(met_raw['message']['created']['date-parts'][0][0]) 192 | return met 193 | 194 | 195 | def fetch_raw_metadata_sems(src_id): 196 | SEMS_API_URL = 'http://api.semanticscholar.org/v1/paper/' 197 | 198 | url = SEMS_API_URL + src_id 199 | #print('Fetching %s' % url, file=sys.stderr) 200 | result = urllib.request.urlopen(url).read() 201 | time.sleep(5) 202 | finished = True 203 | res = json.loads(result.decode('utf-8')) 204 | res['sems_id'] = src_id 205 | return res 206 | 207 | 208 | def clean_raw_metadata_sems(met_raw): 209 | met = {} 210 | 211 | authors = [] 212 | for a in met_raw['authors']: 213 | a_split = a['name'].split() 214 | a = [a_split[-1]] + a_split[:-1] 215 | authors.append(a) 216 | met['authors'] = authors 217 | met['title'] = met_raw['title'] 218 | met['sems_id'] = met_raw['sems_id'] 219 | 220 | if 'arxivId' in met_raw: 221 | met['arxiv_id'] = met_raw['arxivId'] 222 | if 'doi' in met_raw: 223 | met['doi'] = met_raw['doi'] 224 | if 'year' in met_raw: 225 | met['year'] = str(met_raw['year']) 226 | return met 227 | 228 | 229 | def get_year(m): 230 | return m.get('year', None) 231 | 232 | 233 | def convert_metadata_to_lines(m): 234 | year = get_year(m) 235 | 236 | if year is not None: 237 | year = ' (' + year + ') ' 238 | else: 239 | year = ' ' 240 | content = m['line_prefix'] + m['enumerator_char'] + year \ 241 | + m['title'] 242 | authors = '' 243 | 244 | for a in m['authors']: 245 | authors += ', ' + a[1] + ' ' + a[0] 246 | content += ' by ' + authors[2:] + '\n' 247 | prefix = m['line_prefix'] + ' ' + m['enumerator_char'] + ' ' 248 | arxiv_id = m.get('arxiv_id', None) 249 | if arxiv_id is not None: 250 | content += prefix + 'https://arxiv.org/pdf/' + arxiv_id + '\n' 251 | doi = m.get('doi', None) 252 | if doi is not None and not m.get('skip_doi', False): 253 | content += prefix + 'https://dx.doi.org/' + doi + '\n' 254 | return content 255 | 256 | 257 | def fetch_metadata_cached(src_code, src_id, cache, fetch_raw, clean_raw): 258 | cache_key = get_cache_key(src_code, src_id) 259 | met_raw = cache.get(cache_key, None) 260 | 261 | if met_raw is None: 262 | met_raw = fetch_raw(src_id) 263 | met_raw[KEY_FIELD_NAME] = cache_key 264 | cache[cache_key] = met_raw 265 | met = clean_raw(met_raw) 266 | return met 267 | 268 | 269 | def main(args): 270 | 271 | def merge_dicts(m1, m2): 272 | for k,v in m2.items(): 273 | if k not in m1: 274 | m1[k] = v 275 | return m1 276 | 277 | cache = {} 278 | if args.cache_fname is not None: 279 | cache = load_cache(args.cache_fname) 280 | n0_cache = len(cache) 281 | 282 | with open(args.readme_template, 'r') as f: 283 | for line in f: 284 | 285 | m = extract_metadata(line) 286 | m_arx, m_doi, m_sems = {}, {}, {} 287 | 288 | if m is not None: 289 | if 'arxiv_id' in m: 290 | m_arx = fetch_metadata_cached( 291 | src_code='a', src_id=m['arxiv_id'], 292 | fetch_raw=fetch_raw_metadata_arxiv, 293 | clean_raw=clean_raw_metadata_arxiv, 294 | cache=cache) 295 | if 'doi' in m: 296 | m_doi = fetch_metadata_cached( 297 | src_code='d', src_id=m['doi'], 298 | fetch_raw=fetch_raw_metadata_doi, 299 | clean_raw=clean_raw_metadata_doi, 300 | cache=cache) 301 | if 'sems_id' in m: 302 | m_sems = fetch_metadata_cached( 303 | src_code='s', src_id=m['sems_id'], 304 | fetch_raw=fetch_raw_metadata_sems, 305 | clean_raw=clean_raw_metadata_sems, 306 | cache=cache) 307 | 308 | m = merge_dicts(m, m_arx) 309 | m = merge_dicts(m, m_doi) 310 | m = merge_dicts(m, m_sems) 311 | 312 | lines = convert_metadata_to_lines(m) 313 | print(lines, end='', flush=True) 314 | else: 315 | print(line, end='', flush=True) 316 | 317 | if args.cache_fname is not None and n0_cache != len(cache): 318 | save_cache(args.cache_fname, cache) 319 | 320 | 321 | if __name__ == '__main__': 322 | args = parse_argv(sys.argv[1:]) 323 | main(args) 324 | --------------------------------------------------------------------------------