└── README.md /README.md: -------------------------------------------------------------------------------- 1 | # Bayesian Neural Network Recent Papers 2 | a repo sharing Bayesian Neural Network recent papers 3 | # Methods 4 | ## Variational Inference (VI) 5 | [1] [Variational Bayesian Phylogenetic Inference ](https://openreview.net/forum?id=SJVmjjR9FX), ICLR 2019 6 | 7 | [2] [FUNCTIONAL VARIATIONAL BAYESIAN NEURAL NETWORKS](https://openreview.net/forum?id=rkxacs0qY7), ICLR 2019 8 | 9 | [3] [Deterministic Variational Inference for Robust Bayesian Neural Networks ](https://openreview.net/forum?id=B1l08oAct7), ICLR 2019 10 | 11 | [4] [Structured Variational Learning of Bayesian Neural Networks with Horseshoe Priors](https://arxiv.org/abs/1806.05975), ICML 2018 12 | 13 | [5] [Projected BNNs: Avoiding weight-space pathologies by learning latent representations of neural network weights](https://arxiv.org/abs/1811.07006), arXiv 2018 14 | 15 | [6] [Noisy Natural Gradient as Variational Inference](http://proceedings.mlr.press/v80/zhang18l/zhang18l.pdf), ICML 2018 16 | 17 | [7] [Neural Control Variates for Variance Reduction](https://arxiv.org/abs/1806.00159), arXiv 2018 18 | 19 | [8] [Message Passing Stein Variational Gradient Descent](https://arxiv.org/abs/1711.04425), ICML 2018 20 | 21 | [9] [KERNEL IMPLICIT VARIATIONAL INFERENCE](https://openreview.net/forum?id=r1l4eQW0Z), ICLR 2018 22 | 23 | [10] [Gradient Estimators for Implicit Models](https://openreview.net/forum?id=SJi9WOeRb), ICLR 2018 24 | 25 | [11] [Fixing Variational Bayes: Deterministic Variational Inference for Bayesian Neural Networks](https://arxiv.org/abs/1810.03958), arXiv 2018 26 | 27 | [12] [Reducing Reparameterization Gradient Variance](https://papers.nips.cc/paper/6961-reducing-reparameterization-gradient-variance), NIPS 2017 28 | 29 | [13] [Multiplicative Normalizing Flows for Variational Bayesian Neural Networks](https://arxiv.org/abs/1703.01961), ICML 2017 30 | 31 | [14] [Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm](https://arxiv.org/abs/1608.04471), NIPS 2016 32 | 33 | [15] [Known Unknowns: Uncertainty Quality in Bayesian Neural Networks](https://arxiv.org/abs/1612.01251), NIPS 2016 34 | 35 | [16] [Practical Variational Inference for Neural Networks](https://papers.nips.cc/paper/4329-practical-variational-inference-for-neural-networks), NIPS 2011 36 | 37 | [17] [Mixed Variational Inference](https://arxiv.org/abs/1901.04791), arXiv 2019 38 | 39 | [18] [Radial and Directional Posteriors for Bayesian Neural Networks](https://arxiv.org/abs/1902.02603), arXiv 2019 40 | 41 | [19] [Unbiased Implicit Variational Inference](https://arxiv.org/abs/1808.02078), arXiv 2019 42 | 43 | [20] [Semi-implicit variational inference](https://arxiv.org/abs/1805.11183), ICML 2018 44 | 45 | [21] [Doubly Semi-Implicit Variational Inference](https://arxiv.org/abs/1810.02789), arXiv 2018 46 | 47 | [22] [Automated Variational Inference in Probabilistic Programming](https://arxiv.org/abs/1301.1299), arXiv 2013 48 | 49 | [23] [Black Box Variational Inference](https://arxiv.org/abs/1401.0118), arXiv 2014 50 | 51 | [24] [Stochastic Variational Inference](http://www.jmlr.org/papers/volume14/hoffman13a/hoffman13a.pdf), JMLR 2013 52 | 53 | [25] [Weight Uncertainty in Neural Networks](https://arxiv.org/pdf/1505.05424.pdf), ICML 2015 54 | 55 | [26] [Functional Variational Bayesian Neural Networks](https://arxiv.org/abs/1903.05779) 56 | 57 | ## Markov Chain Monte Carlo 58 | [1] [Meta-Learning For Stochastic Gradient MCMC](https://openreview.net/forum?id=HkeoOo09YX), ICLR 2019 59 | 60 | [2] [Adversarial Distillation of Bayesian Neural Network Posteriors](https://arxiv.org/abs/1806.10317), ICML 2018 61 | 62 | [3] [Preconditioned Stochastic Gradient Langevin Dynamics for Deep Neural Networks](http://www.aaai.org/ocs/index.php/AAAI/AAAI16/paper/view/11835), AAAI 2016 63 | 64 | [4] [Bayesian Dark Knowledge](https://arxiv.org/abs/1506.04416), NIPS 2015 65 | 66 | [5] [Stochastic Gradient Hamiltonian Monte Carlo](https://arxiv.org/abs/1402.4102), ICML 2014 67 | 68 | [6] [Bayesian learning via stochastic gradient langevin dynamics](https://www.ics.uci.edu/~welling/publications/papers/stoclangevin_v6.pdf), ICML 2011 69 | 70 | [7] [Cyclical Stochastic Gradient MCMC for Bayesian Deep Learning](https://arxiv.org/abs/1902.03932), arXiv 2019 71 | 72 | [8] [Communication-Efficient Stochastic Gradient MCMC for Neural Networks](http://chunyuan.li/papers/aaai_2019_distributed.pdf), AAAI 2019 73 | 74 | [9] [Stochastic Gradient MCMC with Stale Gradients](http://people.ee.duke.edu/~lcarin/Changyou_nips_2016.pdf), NIPS 2016 75 | 76 | [10] [Markov Chain Monte Carlo and Variational Inference: Bridging the Gap](http://proceedings.mlr.press/v37/salimans15.pdf), ICML 2015 77 | 78 | [11] [CPSG-MCMC: Clustering-Based Preprocessing method for Stochastic Gradient MCMC](http://proceedings.mlr.press/v54/fu17a.html), AISTATS 2017 79 | 80 | ## MCMC + VI 81 | 82 | [1] [Bridging the Gap between Stochastic Gradient MCMC and Stochastic Optimization](http://people.ee.duke.edu/~lcarin/Santa_aistats16.pdf), AISTATS 2016 83 | 84 | [2] [Auxiliary Variational MCMC](https://openreview.net/pdf?id=r1NJqsRctX), ICLR 2019 85 | 86 | [3] [Variational MCMC](https://arxiv.org/abs/1301.2266), UAI 2011 87 | 88 | [4] [Variational Hamiltonian Monte Carlo via Score Matching](https://projecteuclid.org/euclid.ba/1500948232), 2018 89 | 90 | ## Ensembling Sampling (ES) 91 | [1] [Uncertainty in Neural Networks: Bayesian Ensembling](https://arxiv.org/abs/1810.05546), arXiv 2019 92 | 93 | [2] [A Simple Baseline for Bayesian Uncertainty in Deep Learning](https://arxiv.org/abs/1902.02476), arXiv 2019 94 | 95 | [3] [Fast and Scalable Bayesian Deep Learning by Weight-Perturbation in Adam](https://arxiv.org/abs/1806.04854), ICML 2018 96 | 97 | [4] [Bayesian Neural Network Ensembles](https://arxiv.org/abs/1811.12188), NIPS 2018 98 | 99 | [5] [Averaging Weights Leads to Wider Optima and Better Generalization](https://arxiv.org/abs/1803.05407), UAI 2018 100 | 101 | [6] [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://arxiv.org/abs/1612.01474), NIPS 2017 102 | 103 | ## Particle Optimization 104 | [1] [Function Space Particle Optimization for Bayesian Neural Networks](https://openreview.net/forum?id=BkgtDsCcKQ), ICLR 2019 105 | 106 | [2] [A Unified Particle-Optimization Framework for Scalable Bayesian Sampling](https://arxiv.org/abs/1805.11659), UAI 2018 107 | 108 | [3] [Bayesian posterior approximation via greedy particle optimization](https://arxiv.org/abs/1805.07912), arXiv 2019 109 | 110 | ## Laplace Approximation 111 | [1] [Online Structured Laplace Approximations For Overcoming Catastrophic Forgetting](https://arxiv.org/abs/1805.07810), arXiv 2018 112 | 113 | [2] [A Scalable Laplace Approximation for Neural Networks](https://openreview.net/forum?id=Skdvd2xAZ), ICLR 2018 114 | ## Expectation Propgation (EP) 115 | [1] [Assumed Density Filtering Methods for Learning Bayesian Neural Networks](https://www.aaai.org/ocs/index.php/AAAI/AAAI16/paper/download/12391/11777), AAAi 2016 116 | 117 | [2] [Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks](https://arxiv.org/abs/1502.05336), ICML 2016 118 | ## Others 119 | [1] [Practical Bayesian Learning of Neural Networks via Adaptive Subgradient Methods](https://arxiv.org/abs/1811.03679), Under Review AISTATS 2019 120 | 121 | [2] [Learning Structured Weight Uncertainty in Bayesian Neural Networks](http://proceedings.mlr.press/v54/sun17b.html), AISTATS 2017 122 | 123 | # Theory 124 | ## Gaussian Process 125 | [1] [Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processes ](https://openreview.net/forum?id=B1g30j0qF7), ICLR 2019 126 | 127 | [2] [Closed Form Variational Objectives For Bayesian Neural Networks with a Single Hidden Layer](https://arxiv.org/abs/1811.00686), NIPS 2018 128 | 129 | [3] [Building Bayesian Neural Networks with Blocks: On Structure, Interpretability and Uncertainty](https://arxiv.org/abs/1806.03563), arXiv 2018 130 | 131 | [4] [Mapping Gaussian Process Priors to Bayesian Neural Networks](http://bayesiandeeplearning.org/2017/papers/65.pdf), NIPS 2017 132 | ## Dropout 133 | [1] [Variational Bayesian dropout: pitfalls and fixes](https://arxiv.org/abs/1807.01969), ICML 2018 134 | 135 | [2] [Loss-Calibrated Approximate Inference in Bayesian Neural Networks](https://arxiv.org/abs/1805.03901), arXiv 2018 136 | 137 | [3] [Variational Dropout Sparsifies Deep Neural Networks](https://arxiv.org/abs/1701.05369), ICML 2017 138 | 139 | [4] [Dropout Inference in Bayesian Neural Networks with Alpha-divergences](https://arxiv.org/abs/1703.02914), ICML 2017 140 | 141 | [5] [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://arxiv.org/abs/1506.02142), ICML 2016 142 | ## Issues 143 | [1] [Overpruning in Variational Bayesian Neural Networks](https://arxiv.org/abs/1801.06230), NIPS 2017 144 | 145 | [2] [Bayesian neural networks increasingly sparsify their units with depth](https://arxiv.org/abs/1810.05193), arXiv 2018 146 | 147 | [3] [Accelerated First-order Methods on the Wasserstein Space for Bayesian Inference](https://arxiv.org/abs/1807.01750), arXiv 2018 148 | 149 | ## Others 150 | 151 | [1] [Statistical Guarantees for the Robustness of Bayesian Neural Networks](https://arxiv.org/abs/1903.01980), arXiv 2019 152 | 153 | [2] [Performance Measurement for Deep Bayesian Neural Network](https://arxiv.org/abs/1903.08674) 154 | 155 | # Applications 156 | ## Adversarial Defense 157 | [1] [Understanding Measures of Uncertainty for Adversarial Example Detection](https://arxiv.org/abs/1803.08533), UAI 2018 158 | 159 | [2] [Sufficient Conditions for Robustness to Adversarial Examples: a Theoretical and Empirical Study with Bayesian Neural Networks](https://openreview.net/forum?id=B1eZRiC9YX), ICLR 2019 160 | 161 | [3] [Adv-BNN: Improved Adversarial Defense through Robust Bayesian Neural Network](https://openreview.net/forum?id=rk4Qso0cKm), ICLR 2019 162 | 163 | [4] [Predictive Uncertainty Quantification with Compound Density Networks](https://arxiv.org/abs/1902.01080), arXiv 2019 164 | 165 | ## Bayesian Optmization 166 | [1] [Learning Curve Prediction with Bayesian Neural Networks](https://openreview.net/forum?id=S11KBYclx), ICLR 2017 167 | 168 | [2] [Bayesian Optimization with Robust Bayesian Neural Networks](https://papers.nips.cc/paper/6117-bayesian-optimization-with-robust-bayesian-neural-networks), NIPS 2016 169 | ## Hardware Acceleration 170 | [1] [VIBNN Hardware Acceleration of Bayesian Neural Networks](https://arxiv.org/abs/1802.00822), ASPLOS 2018 171 | ## Regression 172 | [1] [Applying SVGD to Bayesian Neural Networks for Cyclical Time-Series Prediction and Inference](https://arxiv.org/abs/1901.05906), NIPS 2018 173 | 174 | [2] [Informed MCMC with Bayesian Neural Networks for Facial Image Analysis](https://arxiv.org/abs/1811.07969), NIPS 2018 175 | 176 | [3] [Accurate Uncertainties for Deep Learning Using Calibrated Regression](https://arxiv.org/abs/1807.00263), ICML 2018 177 | 178 | [4] [Personalizing Gesture Recognition Using Hierarchical Bayesian Neural Networks](http://openaccess.thecvf.com/content_cvpr_2017/papers/Joshi_Personalizing_Gesture_Recognition_CVPR_2017_paper.pdf), CVPR 2017 179 | 180 | ## Implicit Multivariate Prior 181 | [1] [Variational Implicit Processes](https://arxiv.org/abs/1806.02390), NIPS 2018 182 | 183 | ## Classification 184 | [1] [Uncertainty quantification using Bayesian neural networks in classification: Application to ischemic stroke lesion segmentation](https://openreview.net/forum?id=Sk_P2Q9sG), MIDL 2018 185 | 186 | [2] [Hierarchical Bayesian Neural Networks for Personalized Classification](http://bayesiandeeplearning.org/2016/papers/BDL_28.pdf), NIPS 2016 187 | ## Reinforcement Learning 188 | [1] [Randomized Prior Functions for Deep Reinforcement Learning](https://arxiv.org/abs/1806.03335), NIPS 2018 189 | 190 | [2] [Learning Structural Weight Uncertainty for Sequential Decision-Making](https://arxiv.org/abs/1801.00085), AISTATS 2018 191 | 192 | [3] [VIME: Variational Information Maximizing Exploration](https://arxiv.org/abs/1605.09674), arXiv 2017 193 | 194 | [4] [Learning and Policy Search in Stochastic Dynamical Systems with Bayesian Neural Networks](https://openreview.net/forum?id=H1fl8S9ee), ICLR 2017 195 | 196 | [5] [Uncertainty Aware Learning from Demonstrations in Multiple Contexts using Bayesian Neural Networks](https://arxiv.org/abs/1903.05697) 197 | 198 | [6] [Stein Variational Policy Gradient](http://auai.org/uai2017/proceedings/papers/239.pdf), UAI 2018 199 | 200 | [7] [Variational Inference for Policy Gradient](https://arxiv.org/pdf/1802.07833.pdf), arXiv 2018 201 | 202 | ## Recurrent/Convolutional Neural Networks 203 | [1] [Bayesian Recurrent Neural Networks](https://arxiv.org/abs/1704.02798), arXiv 2018 204 | 205 | [2] [Bayesian Convolutional Neural Networks with Variational Inference](https://arxiv.org/abs/1806.05978), arXiv 2018 206 | 207 | [3] [Scalable Bayesian Learning of Recurrent Neural Networks for Language Modeling](https://arxiv.org/abs/1611.08034), ACL 2017 208 | 209 | [4] [Propagating Uncertainty in Multi-Stage Bayesian Convolutional Neural Networks with Application to Pulmonary Nodule Detection](https://arxiv.org/abs/1712.00497), NIPS 2017 210 | 211 | [5] [Bayesian Sparsification of Recurrent Neural Networks](https://arxiv.org/pdf/1708.00077), arXiv 2017 212 | 213 | [6] [BAYESIAN CONVOLUTIONAL NEURAL NETWORKS WITH BERNOULLI APPROXIMATE VARIATIONAL INFERENCE](https://arxiv.org/abs/1506.02158), ICLR 2016 214 | 215 | [7] [Sparse Bayesian Recurrent Neural Networks](https://link.springer.com/chapter/10.1007/978-3-319-23525-7_22), ECML PKDD 2015 216 | 217 | [8] [Learning Weight Uncertainty with SG-MCMC for Shape Classification](http://people.duke.edu/~cl319/doc/papers/dbnn_shape_cvpr.pdf), CVPR 2016 218 | 219 | ## Incremental Learning 220 | [1] [BAYESIAN INCREMENTAL LEARNING FOR DEEP NEURAL NETWORKS](https://openreview.net/forum?id=ByZzFPJDG), ICLR 2018 221 | 222 | ## GAN 223 | [1] [Bayesian GAN](https://arxiv.org/abs/1705.09558), NIPS 2017 224 | 225 | ## ODE 226 | [1] [ODE2VAE: Deep generative second order ODEs with Bayesian neural networks](https://arxiv.org/abs/1905.10994), arXiv 2019 227 | 228 | ## Interpretable AI 229 | [1] [Interpretable Outcome Prediction with Sparse Bayesian Neural Networks in Intensive Care](https://arxiv.org/abs/1905.02599), arXiv 2019 230 | 231 | ## Computational Fluid Dynamics 232 | 233 | [1] [Quantifying model form uncertainty in Reynolds-averaged turbulence models with Bayesian deep neural networks](https://arxiv.org/abs/1807.02901), arXiv 2019 234 | 235 | # Survey 236 | [1] [Towards Bayesian Deep Learning A Survey](https://arxiv.org/abs/1604.01662), arXiv 2016 237 | 238 | [2] [Advances in Variational Inference](https://arxiv.org/abs/1711.05597), arXiv 2017 239 | 240 | [3] [Variational Inference: A Review for Statisticians](https://arxiv.org/abs/1601.00670), arXiv 2018 241 | 242 | # Thesis 243 | 244 | [1] [UNCERTAINTY IN DEEP LEARNING](http://www.cs.ox.ac.uk/people/yarin.gal/website/thesis/thesis.pdf), Yarin Gal 2016 245 | 246 | [2] [Approximate Inference: New Visions](http://yingzhenli.net/home/pdf/phd_thesis.pdf), Yingzhen Li 2018 247 | 248 | [3] [Towards Better Representations with Deep/Bayes Learning](http://chunyuan.li/doc/PhD_research_cli.pdf), Chunyuan Li 2018 (slide) 249 | 250 | [4] [Variational Inference & Deep Learning: A new Synthesis](https://dare.uva.nl/search?identifier=8e55e07f-e4be-458f-a929-2f9bc2d169e8), D.P. Kingma 2017 251 | 252 | [5] [Bayesian Convolutional Neural Network](https://github.com/kumar-shridhar/Master-Thesis-BayesianCNN), Shridhar Kumar 2018 253 | 254 | [6] [BAYESIAN LEARNING FOR NEURAL NETWORKS](http://www.cs.toronto.edu/~radford/bnn.book.html), Radford M. Neal 1995 255 | 256 | [7] [Stochastic Gradient MCMC: Algorithms and Applications](https://escholarship.org/content/qt4k8039zm/qt4k8039zm.pdf), Sungjin Ahn 2015 257 | 258 | [8] [Large-Scale Bayesian Computation Using Stochastic Gradient Markov Chain Monte Carlo](http://59.80.44.99/eprints.lancs.ac.uk/131012/1/2019JackBakerPhD.pdf), Jack Baker 2018 259 | 260 | # Researcher 261 | 262 | [1] [Max Welling](https://staff.fnwi.uva.nl/m.welling/) 263 | 264 | [2] [Zoubin Ghahramani](http://mlg.eng.cam.ac.uk/zoubin/) 265 | 266 | [3] [Radford M. Neal](https://www.cs.toronto.edu/~radford/homepage.html) 267 | 268 | [4] [Yee Whye Teh](http://www.stats.ox.ac.uk/~teh/index.html) 269 | 270 | [5] [Ryan P. Adams](http://www.cs.princeton.edu/~rpa/) 271 | 272 | [6] [David M. Blei](http://www.cs.columbia.edu/~blei/) 273 | 274 | [7] [Jun Zhu](http://ml.cs.tsinghua.edu.cn/~jun/index.shtml) 275 | 276 | [8] [Lawrence Carin](http://people.ee.duke.edu/~lcarin/) 277 | 278 | [9] [Andrew Gordon Wilson](https://people.orie.cornell.edu/andrew/) 279 | 280 | [10] [Roger Grosse](http://www.cs.toronto.edu/~rgrosse/) 281 | 282 | [11] [Yarin Gal](http://www.cs.ox.ac.uk/people/yarin.gal/website/) 283 | 284 | [12] [Yingzhen Li](http://yingzhenli.net/home/en/) 285 | 286 | [13] [Yutian Chen](http://yutianchen.com/) 287 | 288 | [14] [Christos Louizos](https://christoslouizos.wordpress.com/) 289 | 290 | [15] [Charles Blundell](http://www.gatsby.ucl.ac.uk/~ucgtcbl/index.html) 291 | 292 | [16] [Shengyang sun](http://www.cs.toronto.edu/~ssy/) 293 | 294 | [17] [Jiaxin Shi](http://jiaxins.me/) 295 | 296 | [18] [Chunyuan Li](http://chunyuan.li/) 297 | 298 | [19] [Mingyuan Zhou](https://mingyuanzhou.github.io/) 299 | 300 | [20] [Sungjin Ahn](http://www.sungjinahn.com/) 301 | 302 | [21] [Yian Ma](https://sites.google.com/view/yianma) 303 | 304 | [22] [Emily Fox](https://homes.cs.washington.edu/~ebfox/) 305 | 306 | [23] [Qiang Liu](https://www.cs.utexas.edu/~lqiang/) 307 | 308 | [24] [eric nalisnick](http://people.ds.cam.ac.uk/etn22/) 309 | 310 | # Events 311 | 312 | [1] [NIPS Bayesian Deep Learning Workshop](http://bayesiandeeplearning.org/) (2016, 2017, 2018) 313 | 314 | [2] [NIPS Symposium on Advances in Approximate Bayesian Inference](http://approximateinference.org/) (2014 - 2018) 315 | --------------------------------------------------------------------------------