├── LICENSE
└── README.md


/LICENSE:
--------------------------------------------------------------------------------
 1 | MIT License
 2 | 
 3 | Copyright (c) 2022-2024 Gianni Franchi
 4 | 
 5 | Permission is hereby granted, free of charge, to any person obtaining a copy
 6 | of this software and associated documentation files (the "Software"), to deal
 7 | in the Software without restriction, including without limitation the rights
 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 9 | copies of the Software, and to permit persons to whom the Software is
10 | furnished to do so, subject to the following conditions:
11 | 
12 | The above copyright notice and this permission notice shall be included in all
13 | copies or substantial portions of the Software.
14 | 
15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21 | SOFTWARE.
22 | 


--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
  1 | # Awesome Uncertainty in Deep learning
  2 | 
  3 | <div align="center">
  4 | 
  5 | [![MIT License](https://img.shields.io/badge/license-MIT-green.svg)](https://opensource.org/licenses/MIT)
  6 | [![Awesome](https://awesome.re/badge.svg)](https://awesome.re)
  7 | 
  8 | </div>
  9 | 
 10 | This repo is a collection of *awesome* papers, codes, books, and blogs about Uncertainty and Deep learning. 
 11 | 
 12 | :star: Feel free to star and fork. :star:
 13 | 
 14 | If you think we missed a paper, please open a pull request or send a message on the corresponding [GitHub discussion](https://github.com/ENSTA-U2IS-AI/awesome-uncertainty-deeplearning/discussions). Tell us where the article was published and when, and send us GitHub and ArXiv links if they are available.
 15 | 
 16 | We are also open to any ideas for improvements!
 17 | 
 18 | <h2>
 19 | Table of Contents
 20 | </h2>
 21 | 
 22 | - [Awesome Uncertainty in Deep learning](#awesome-uncertainty-in-deep-learning)
 23 | - [Papers](#papers)
 24 |   - [Surveys](#surveys)
 25 |   - [Theory](#theory)
 26 |   - [Bayesian-Methods](#bayesian-methods)
 27 |   - [Ensemble-Methods](#ensemble-methods)
 28 |   - [Sampling/Dropout-based-Methods](#samplingdropout-based-methods)
 29 |   - [Post-hoc-Methods/Auxiliary-Networks](#post-hoc-methodsauxiliary-networks)
 30 |   - [Data-augmentation/Generation-based-methods](#data-augmentationgeneration-based-methods)
 31 |   - [Output-Space-Modeling/Evidential-deep-learning](#output-space-modelingevidential-deep-learning)
 32 |   - [Deterministic-Uncertainty-Methods](#deterministic-uncertainty-methods)
 33 |   - [Quantile-Regression/Predicted-Intervals](#quantile-regressionpredicted-intervals)
 34 |   - [Conformal Predictions](#conformal-predictions)
 35 |   - [Calibration/Evaluation-Metrics](#calibrationevaluation-metrics)
 36 |   - [Misclassification Detection \& Selective Classification](#misclassification-detection--selective-classification)
 37 |   - [Applications](#applications)
 38 |     - [Classification and Semantic-Segmentation](#classification-and-semantic-segmentation)
 39 |     - [Regression](#regression)
 40 |     - [Anomaly-detection and Out-of-Distribution-Detection](#anomaly-detection-and-out-of-distribution-detection)
 41 |     - [Object detection](#object-detection)
 42 |     - [Domain adaptation](#domain-adaptation)
 43 |     - [Semi-supervised](#semi-supervised)
 44 |     - [Natural Language Processing](#natural-language-processing)
 45 |     - [Others](#others)
 46 | - [Datasets and Benchmarks](#datasets-and-benchmarks)
 47 | - [Libraries](#libraries)
 48 |   - [Python](#python)
 49 |   - [PyTorch](#pytorch)
 50 |   - [JAX](#jax)
 51 |   - [TensorFlow](#tensorflow)
 52 | - [Lectures and tutorials](#lectures-and-tutorials)
 53 | - [Books](#books)
 54 | - [Other Resources](#other-resources)
 55 | 
 56 | # Papers
 57 | 
 58 | ## Surveys
 59 | 
 60 | **Conference**
 61 | 
 62 | - A Comparison of Uncertainty Estimation Approaches in Deep Learning Components for Autonomous Vehicle Applications [[AISafety Workshop 2020]](<https://arxiv.org/abs/2006.15172>)
 63 | 
 64 | **Journal**
 65 | 
 66 | - A survey of uncertainty in deep neural networks [[Artificial Intelligence Review 2023]](<https://arxiv.org/abs/2107.03342>) - [[GitHub]](<https://github.com/JakobCode/UncertaintyInNeuralNetworks_Resources>) 
 67 | - Prior and Posterior Networks: A Survey on Evidential Deep Learning Methods For Uncertainty Estimation [[TMLR2023]](<https://arxiv.org/abs/2110.03051>)
 68 | - A Survey on Uncertainty Estimation in Deep Learning Classification Systems from a Bayesian Perspective [[ACM2021]](<https://dl.acm.org/doi/pdf/10.1145/3477140?casa_token=6fozCYTovlIAAAAA:t5vcjuXCMem1b8iFwaMG4o_YJHTe0wArLtoy9KCbL8Cow0aGEoxSiJans2Kzpm2FSKOg-4ZCDkBa>)
 69 | - Ensemble deep learning: A review [[Engineering Applications of AI 2021]](<https://arxiv.org/abs/2104.02395>)
 70 | - A review of uncertainty quantification in deep learning: Techniques, applications and challenges [[Information Fusion 2021]](<https://www.sciencedirect.com/science/article/pii/S1566253521001081>)
 71 | - Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods [[Machine Learning 2021]](<https://link.springer.com/article/10.1007/s10994-021-05946-3>)
 72 | - Predictive inference with the jackknife+ [[The Annals of Statistics 2021]](<https://arxiv.org/abs/1905.02928>)
 73 | - Uncertainty in big data analytics: survey, opportunities, and challenges [[Journal of Big Data 2019]](<https://journalofbigdata.springeropen.com/articles/10.1186/s40537-019-0206-3?cv=1>)
 74 | 
 75 | **Arxiv**
 76 | 
 77 | - Benchmarking Uncertainty Disentanglement: Specialized Uncertainties for Specialized Tasks [[ArXiv2024]](<https://arxiv.org/pdf/2402.19460.pdf>) - [[PyTorch]](<https://github.com/bmucsanyi/bud>)
 78 | - A System-Level View on Out-of-Distribution Data in Robotics [[arXiv2022]](<https://arxiv.org/abs/2212.14020>)
 79 | - A Survey on Uncertainty Reasoning and Quantification for Decision Making: Belief Theory Meets Deep Learning [[arXiv2022]](<https://arxiv.org/abs/2206.05675>)
 80 | 
 81 | ## Theory
 82 | 
 83 | **Conference**
 84 | 
 85 | - A Rigorous Link between Deep Ensembles and (Variational) Bayesian Methods [[NeurIPS2023]](<https://arxiv.org/pdf/2305.15027>)
 86 | - Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning [[ICLR2023]](<https://arxiv.org/pdf/2012.09816.pdf>)
 87 | - Unmasking the Lottery Ticket Hypothesis: What's Encoded in a Winning Ticket's Mask? [[ICLR2023]](<https://arxiv.org/pdf/2210.03044.pdf>)
 88 | - Probabilistic Contrastive Learning Recovers the Correct Aleatoric Uncertainty of Ambiguous Inputs [[ICML2023]](<https://arxiv.org/pdf/2302.02865.pdf>) - [[PyTorch]](<https://github.com/mkirchhof/Probabilistic_Contrastive_Learning>)
 89 | - On Second-Order Scoring Rules for Epistemic Uncertainty Quantification [[ICML2023]](<https://arxiv.org/pdf/2301.12736.pdf>)
 90 | - Neural Variational Gradient Descent [[AABI2022]](<https://openreview.net/forum?id=oG0vTBw58ic>)
 91 | - Top-label calibration and multiclass-to-binary reductions [[ICLR2022]](<https://openreview.net/forum?id=WqoBaaPHS->)
 92 | - Bayesian Model Selection, the Marginal Likelihood, and Generalization [[ICML2022]](<https://arxiv.org/abs/2202.11678>)
 93 | - With malice towards none: Assessing uncertainty via equalized coverage [[AIES 2021]](<https://arxiv.org/abs/1908.05428>)
 94 | - Uncertainty in Gradient Boosting via Ensembles [[ICLR2021]](<https://arxiv.org/abs/2006.10562>) - [[PyTorch]](<https://github.com/yandex-research/GBDT-uncertainty>)
 95 | - Repulsive Deep Ensembles are Bayesian [[NeurIPS2021]](<https://arxiv.org/abs/2106.11642>) - [[PyTorch]](<https://github.com/ratschlab/repulsive_ensembles>)
 96 | - Bayesian Optimization with High-Dimensional Outputs [[NeurIPS2021]](<https://arxiv.org/abs/2106.12997>)
 97 | - Residual Pathway Priors for Soft Equivariance Constraints [[NeurIPS2021]](<https://arxiv.org/abs/2112.01388>)
 98 | - Dangers of Bayesian Model Averaging under Covariate Shift [[NeurIPS2021]](<https://arxiv.org/abs/2106.11905>) - [[TensorFlow]](<https://github.com/izmailovpavel/bnn_covariate_shift>)
 99 | - A Mathematical Analysis of Learning Loss for Active Learning in Regression [[CVPR Workshop2021]](<https://openaccess.thecvf.com/content/CVPR2021W/TCV/html/Shukla_A_Mathematical_Analysis_of_Learning_Loss_for_Active_Learning_in_CVPRW_2021_paper.html>)
100 | - Why Are Bootstrapped Deep Ensembles Not Better? [[NeurIPS Workshop]](<https://openreview.net/forum?id=dTCir0ceyv0>)
101 | - Deep Convolutional Networks as shallow Gaussian Processes [[ICLR2019]](<https://arxiv.org/abs/1808.05587>)
102 | - On the accuracy of influence functions for measuring group effects [[NeurIPS2018]](<https://proceedings.neurips.cc/paper/2019/hash/a78482ce76496fcf49085f2190e675b4-Abstract.html>)
103 | - To Trust Or Not To Trust A Classifier [[NeurIPS2018]](<https://arxiv.org/abs/1805.11783>) - [[Python]](<https://github.com/google/TrustScore>)
104 | - Understanding Measures of Uncertainty for Adversarial Example Detection [[UAI2018]](<https://arxiv.org/abs/1803.08533>)
105 | 
106 | **Journal**
107 | 
108 | - Martingale posterior distributions [[Royal Statistical Society Series B]](<https://arxiv.org/abs/2103.15671>)
109 | - A Unified Theory of Diversity in Ensemble Learning [[JMLR2023]](<https://jmlr.org/papers/volume24/23-0041/23-0041.pdf>)
110 | - Multivariate Uncertainty in Deep Learning [[TNNLS2021]](<https://arxiv.org/abs/1910.14215>)
111 | - A General Framework for Uncertainty Estimation in Deep Learning [[RAL2020]](<https://arxiv.org/abs/1907.06890>)
112 | - Adaptive nonparametric confidence sets [[Ann. Statist. 2006]](<https://arxiv.org/abs/math/0605473>)
113 | 
114 | **Arxiv**
115 | 
116 | - Ensembles for Uncertainty Estimation: Benefits of Prior Functions and Bootstrapping [[arXiv2022]](<https://arxiv.org/pdf/2206.03633.pdf>)
117 | - Efficient Gaussian Neural Processes for Regression [[arXiv2021]](<https://arxiv.org/abs/2108.09676>)
118 | - Dense Uncertainty Estimation [[arXiv2021]](<https://arxiv.org/abs/2110.06427>) - [[PyTorch]](<https://github.com/JingZhang617/UncertaintyEstimation>)
119 | - A higher-order swiss army infinitesimal jackknife [[arXiv2019]](<https://arxiv.org/abs/1907.12116>)
120 | 
121 | ## Bayesian-Methods
122 | 
123 | **Conference**
124 | 
125 | - Training Bayesian Neural Networks with Sparse Subspace Variational Inference [[ICLR2024]](<https://arxiv.org/abs/2402.11025>)
126 | - Variational Bayesian Last Layers [[ICLR2024]](https://arxiv.org/abs/2404.11599)
127 | - A Symmetry-Aware Exploration of Bayesian Neural Network Posteriors [[ICLR2024]](<https://arxiv.org/abs/2310.08287>)
128 | - Gradient-based Uncertainty Attribution for Explainable Bayesian Deep Learning [[CVPR2023]](<https://arxiv.org/abs/2304.04824>)
129 | - Robustness to corruption in pre-trained Bayesian neural networks [[ICLR2023]](<https://arxiv.org/pdf/2206.12361.pdf>)
130 | - Beyond Deep Ensembles: A Large-Scale Evaluation of Bayesian Deep Learning under Distribution Shift [[NeurIPS2023]](<https://arxiv.org/abs/2306.12306>) - [[PyTorch]](<https://github.com/Feuermagier/Beyond_Deep_Ensembles>)
131 | - Transformers Can Do Bayesian Inference [[ICLR2022]](<https://arxiv.org/abs/2112.10510>) - [[PyTorch]](<https://github.com/automl/PFNs?tab=readme-ov-file>)
132 | - Uncertainty Estimation for Multi-view Data: The Power of Seeing the Whole Picture [[NeurIPS2022]](<https://arxiv.org/abs/2210.02676>)
133 | - On Batch Normalisation for Approximate Bayesian Inference [[AABI2021]](<https://openreview.net/pdf?id=SH2tfpm_0LE>)
134 | - Activation-level uncertainty in deep neural networks [[ICLR2021]](<https://openreview.net/forum?id=UvBPbpvHRj->)
135 | - Laplace Redux – Effortless Bayesian Deep Learning [[NeurIPS2021]](<https://arxiv.org/abs/2106.14806>) - [[PyTorch]](<https://github.com/AlexImmer/Laplace>)
136 | - On the Effects of Quantisation on Model Uncertainty in Bayesian Neural Networks [[UAI2021]](<https://arxiv.org/abs/2102.11062>)
137 | - Learnable uncertainty under Laplace approximations [[UAI2021]](<https://proceedings.mlr.press/v161/kristiadi21a.html>)
138 | - Bayesian Neural Networks with Soft Evidence [[ICML Workshop2021]](<https://arxiv.org/abs/2010.09570>) - [[PyTorch]](<https://github.com/edwardyu/soft-evidence-bnn>)
139 | - TRADI: Tracking deep neural network weight distributions for uncertainty estimation [[ECCV2020]](<https://arxiv.org/abs/1912.11316>) - [[PyTorch]](<https://github.com/giannifranchi/TRADI_Tracking_DNN_weights>)
140 | - How Good is the Bayes Posterior in Deep Neural Networks Really? [[ICML2020]](<http://proceedings.mlr.press/v119/wenzel20a.html>)
141 | - Efficient and Scalable Bayesian Neural Nets with Rank-1 Factors [[ICML2020]](<http://proceedings.mlr.press/v119/dusenberry20a/dusenberry20a.pdf>) - [[TensorFlow]](<https://github.com/google/edward2>)
142 | - Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networks [[ICML2020]](<http://proceedings.mlr.press/v119/kristiadi20a/kristiadi20a.pdf>) - [[PyTorch]](<https://github.com/AlexImmer/Laplace>)
143 | - Bayesian Deep Learning and a Probabilistic Perspective of Generalization [[NeurIPS2020]](<https://proceedings.neurips.cc/paper/2020/file/322f62469c5e3c7dc3e58f5a4d1ea399-Paper.pdf>)
144 | - A Simple Baseline for Bayesian Uncertainty in Deep Learning [[NeurIPS2019]](<https://arxiv.org/abs/1902.02476>) - [[PyTorch]](<https://github.com/wjmaddox/swa_gaussian>) - [[TorchUncertainty]](<https://github.com/ENSTA-U2IS-AI/torch-uncertainty>)
145 | - Bayesian Uncertainty Estimation for Batch Normalized Deep Networks [[ICML2018]](<http://proceedings.mlr.press/v80/teye18a.html>) - [[TensorFlow]](<https://github.com/icml-mcbn/mcbn>) - [[TorchUncertainty]](<https://github.com/ENSTA-U2IS-AI/torch-uncertainty>)
146 | - Lightweight Probabilistic Deep Networks [[CVPR2018]](<https://github.com/ezjong/lightprobnets>) - [[PyTorch]](<https://github.com/ezjong/lightprobnets>)
147 | - A Scalable Laplace Approximation for Neural Networks [[ICLR2018]](<https://openreview.net/pdf?id=Skdvd2xAZ>) - [[Theano]](<https://github.com/BB-UCL/Lasagne>)
148 | - Decomposition of Uncertainty in Bayesian Deep Learning for Efficient and Risk-sensitive Learning [[ICML2018]](<http://proceedings.mlr.press/v80/depeweg18a.html>)
149 | - Weight Uncertainty in Neural Networks [[ICML2015]](<https://proceedings.mlr.press/v37/blundell15.html>)
150 | 
151 | **Journal**
152 | 
153 | - Analytically Tractable Hidden-States Inference in Bayesian Neural Networks [[JMLR2024]](https://jmlr.org/papers/v23/21-0758.html)
154 | - Encoding the latent posterior of Bayesian Neural Networks for uncertainty quantification [[TPAMI2023]](<https://arxiv.org/abs/2012.02818>) - [[PyTorch]](<https://github.com/giannifranchi/LP_BNN>)
155 | - Bayesian modeling of uncertainty in low-level vision [[IJCV1990]](<https://link.springer.com/article/10.1007%2FBF00126502>)
156 | 
157 | **Arxiv**
158 | 
159 | - Density Uncertainty Layers for Reliable Uncertainty Estimation [[arXiv2023]](<https://arxiv.org/abs/2306.12497>)
160 | 
161 | ## Ensemble-Methods
162 | 
163 | **Conference**
164 | 
165 | - Input-gradient space particle inference for neural network ensembles [[ICLR2024]](<https://arxiv.org/abs/2306.02775>)
166 | - Fast Ensembling with Diffusion Schrödinger Bridge [[ICLR2024]](<https://arxiv.org/abs/2404.15814>)
167 | - Pathologies of Predictive Diversity in Deep Ensembles [[ICLR2024]](<https://arxiv.org/abs/2302.00704>)
168 | - Model Ratatouille: Recycling Diverse Models for Out-of-Distribution Generalization [[ICML2023]](<https://arxiv.org/pdf/2212.10445.pdf>)
169 | - Bayesian Posterior Approximation With Stochastic Ensembles [[CVPR2023]](<https://openaccess.thecvf.com/content/CVPR2023/papers/Balabanov_Bayesian_Posterior_Approximation_With_Stochastic_Ensembles_CVPR_2023_paper.pdf>)
170 | - Normalizing Flow Ensembles for Rich Aleatoric and Epistemic Uncertainty Modeling [[AAAI2023]](<https://arxiv.org/abs/2302.01312>)
171 | - Window-Based Early-Exit Cascades for Uncertainty Estimation: When Deep Ensembles are More Efficient than Single Models [[ICCV2023]](<https://arxiv.org/abs/2303.08010>) - [[PyTorch]](<https://github.com/guoxoug/window-early-exit>)
172 | - Weighted Ensemble Self-Supervised Learning [[ICLR2023]](<https://arxiv.org/pdf/2211.09981.pdf>)
173 | - Agree to Disagree: Diversity through Disagreement for Better Transferability [[ICLR2023]](<https://arxiv.org/pdf/2202.04414.pdf>) - [[PyTorch]](<https://github.com/mpagli/Agree-to-Disagree>)
174 | - Packed-Ensembles for Efficient Uncertainty Estimation [[ICLR2023]](<https://arxiv.org/abs/2210.09184>) - [[TorchUncertainty]](<https://github.com/ENSTA-U2IS-AI/torch-uncertainty>)
175 | - Sub-Ensembles for Fast Uncertainty Estimation in Neural Networks [[ICCV Workshop2023]](<https://openaccess.thecvf.com/content/ICCV2023W/LXCV/papers/Valdenegro-Toro_Sub-Ensembles_for_Fast_Uncertainty_Estimation_in_Neural_Networks_ICCVW_2023_paper.pdf>)
176 | - Prune and Tune Ensembles: Low-Cost Ensemble Learning With Sparse Independent Subnetworks [[AAAI2022]](<https://arxiv.org/abs/2202.11782>)
177 | - Deep Ensembles Work, But Are They Necessary? [[NeurIPS2022]](<https://arxiv.org/abs/2202.06985>)
178 | - FiLM-Ensemble: Probabilistic Deep Learning via Feature-wise Linear Modulation [[NeurIPS2022]](<https://arxiv.org/abs/2206.00050>)
179 | - Deep Ensembling with No Overhead for either Training or Testing: The All-Round Blessings of Dynamic Sparsity [[ICLR2022]](<https://arxiv.org/abs/2106.14568>) - [[PyTorch]](<https://github.com/VITA-Group/FreeTickets>)
180 | - On the Usefulness of Deep Ensemble Diversity for Out-of-Distribution Detection [[ECCV Workshop2022]](<https://arxiv.org/abs/2207.07517>)
181 | - Masksembles for Uncertainty Estimation [[CVPR2021]](<https://nikitadurasov.github.io/projects/masksembles/>) - [[PyTorch/TensorFlow]](<https://github.com/nikitadurasov/masksembles>)
182 | - Robustness via Cross-Domain Ensembles [[ICCV2021]](<https://arxiv.org/abs/2103.10919>) - [[PyTorch]](<https://github.com/EPFL-VILAB/XDEnsembles>)
183 | - Uncertainty in Gradient Boosting via Ensembles [[ICLR2021]](<https://arxiv.org/abs/2006.10562>) - [[PyTorch]](<https://github.com/yandex-research/GBDT-uncertainty>)
184 | - Uncertainty Quantification and Deep Ensembles [[NeurIPS2021]](<https://openreview.net/forum?id=wg_kD_nyAF>)
185 | - Maximizing Overall Diversity for Improved Uncertainty Estimates in Deep Ensembles [[AAAI2020]](<https://ojs.aaai.org/index.php/AAAI/article/view/5849>)
186 | - Uncertainty in Neural Networks: Approximately Bayesian Ensembling [[AISTATS 2020]](<https://arxiv.org/abs/1810.05546>)
187 | - Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning [[ICLR2020]](<https://arxiv.org/abs/2002.06470>) - [[PyTorch]](<https://github.com/SamsungLabs/pytorch-ensembles>)
188 | - BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learning [[ICLR2020]](<https://arxiv.org/abs/2002.06715>) - [[TensorFlow]](<https://github.com/google/edward2>) - [[TorchUncertainty]](<https://github.com/ENSTA-U2IS-AI/torch-uncertainty>)
189 | - Hyperparameter Ensembles for Robustness and Uncertainty Quantification [[NeurIPS2020]](<https://proceedings.neurips.cc/paper/2020/hash/481fbfa59da2581098e841b7afc122f1-Abstract.html>)
190 | - Bayesian Deep Ensembles via the Neural Tangent Kernel [[NeurIPS2020]](<https://proceedings.neurips.cc/paper/2020/hash/0b1ec366924b26fc98fa7b71a9c249cf-Abstract.html>)
191 | - Diversity with Cooperation: Ensemble Methods for Few-Shot Classification [[ICCV2019]](<https://arxiv.org/abs/1903.11341>)
192 | - Accurate Uncertainty Estimation and Decomposition in Ensemble Learning [[NeurIPS2019]](<https://papers.nips.cc/paper/2019/hash/1cc8a8ea51cd0adddf5dab504a285915-Abstract.html>)
193 | - High-Quality Prediction Intervals for Deep Learning: A Distribution-Free, Ensembled Approach [[ICML2018]](<https://arxiv.org/abs/1802.07167>) - [[TensorFlow]](<https://github.com/TeaPearce/Deep_Learning_Prediction_Intervals>)
194 | - Snapshot Ensembles: Train 1, get M for free [[ICLR2017]](https://arxiv.org/abs/1704.00109) - [[TorchUncertainty]](<https://github.com/ENSTA-U2IS-AI/torch-uncertainty>)
195 | - Simple and scalable predictive uncertainty estimation using deep ensembles [[NeurIPS2017]](<https://arxiv.org/abs/1612.01474>) - [[TorchUncertainty]](<https://github.com/ENSTA-U2IS-AI/torch-uncertainty>)
196 | 
197 | **Journal**
198 | 
199 | - One Versus all for deep Neural Network for uncertainty (OVNNI) quantification [[IEEE Access2021]](<https://arxiv.org/abs/2006.00954>)
200 | 
201 | **Arxiv**
202 | 
203 | - Split-Ensemble: Efficient OOD-aware Ensemble via Task and Model Splitting [[arXiv2023]](<https://arxiv.org/abs/2312.09148>)
204 | - Deep Ensemble as a Gaussian Process Approximate Posterior [[arXiv2022]](<https://arxiv.org/abs/2205.00163>)
205 | - Sequential Bayesian Neural Subnetwork Ensembles [[arXiv2022]](<https://arxiv.org/abs/2206.00794>)
206 | - Confident Neural Network Regression with Bootstrapped Deep Ensembles [[arXiv2022]](<https://arxiv.org/abs/2202.10903>) - [[TensorFlow]](<https://github.com/LaurensSluyterman/Bootstrapped_Deep_Ensembles>)
207 | - Dense Uncertainty Estimation via an Ensemble-based Conditional Latent Variable Model [[arXiv2021]](<https://arxiv.org/abs/2111.11055>)
208 | - Deep Ensembles: A Loss Landscape Perspective [[arXiv2019]](<https://arxiv.org/abs/1912.02757>)
209 | - Checkpoint ensembles: Ensemble methods from a single training process [[arXiv2017]](<https://arxiv.org/abs/1710.03282>) - [[TorchUncertainty]](<https://github.com/ENSTA-U2IS-AI/torch-uncertainty>)
210 | 
211 | ## Sampling/Dropout-based-Methods
212 | 
213 | **Conference**
214 | 
215 | - Enabling Uncertainty Estimation in Iterative Neural Networks [[ICML2024]](<https://arxiv.org/pdf/2403.16732>) - [[Pytorch]](<https://github.com/cvlab-epfl/iter_unc>)
216 | - Make Me a BNN: A Simple Strategy for Estimating Bayesian Uncertainty from Pre-trained Models [[CVPR2024]](<https://arxiv.org/abs/2312.15297>) - [[TorchUncertainty]](<https://github.com/ENSTA-U2IS-AI/torch-uncertainty>)
217 | - Training-Free Uncertainty Estimation for Dense Regression: Sensitivity as a Surrogate [[AAAI2022]](<https://arxiv.org/abs/1910.04858v3>)
218 | - Efficient Bayesian Uncertainty Estimation for nnU-Net [[MICCAI2022]](<https://link.springer.com/chapter/10.1007/978-3-031-16452-1_51>)
219 | - Dropout Sampling for Robust Object Detection in Open-Set Conditions [[ICRA2018]](<https://arxiv.org/abs/1710.06677>)
220 | - Test-time data augmentation for estimation of heteroscedastic aleatoric uncertainty in deep neural networks [[MIDL2018]](<https://openreview.net/forum?id=rJZz-knjz>)
221 | - Concrete Dropout [[NeurIPS2017]](<https://arxiv.org/abs/1705.07832>)
222 | - Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning [[ICML2016]](<https://arxiv.org/abs/1506.02142>) - [[TorchUncertainty]](<https://github.com/ENSTA-U2IS-AI/torch-uncertainty>)
223 | 
224 | **Journal**
225 | 
226 | - A General Framework for Uncertainty Estimation in Deep Learning [[Robotics and Automation Letters2020]](<https://arxiv.org/pdf/1907.06890.pdf>)
227 | 
228 | **Arxiv**
229 | 
230 | - SoftDropConnect (SDC) – Effective and Efficient Quantification of the Network Uncertainty in Deep MR Image Analysis [[arXiv2022]](<https://arxiv.org/abs/2201.08418>)
231 | 
232 | ## Post-hoc-Methods/Auxiliary-Networks
233 | 
234 | **Conference**
235 | 
236 | - On the Limitations of Temperature Scaling for Distributions with Overlaps [[ICLR2024]](https://arxiv.org/abs/2306.00740)
237 | - Post-hoc Uncertainty Learning using a Dirichlet Meta-Model [[AAAI2023]](<https://arxiv.org/abs/2212.07359>) - [[PyTorch]](<https://github.com/maohaos2/PosthocUQ>)
238 | - ProbVLM: Probabilistic Adapter for Frozen Vision-Language Models [[ICCV2023]](<https://openaccess.thecvf.com/content/ICCV2023/html/Upadhyay_ProbVLM_Probabilistic_Adapter_for_Frozen_Vison-Language_Models_ICCV_2023_paper.html>)
239 | - Out-of-Distribution Detection for Monocular Depth Estimation [[ICCV2023]](<https://arxiv.org/abs/2308.06072>)
240 | - Detecting Misclassification Errors in Neural Networks with a Gaussian Process Model [[AAAI2022]](<https://ojs.aaai.org/index.php/AAAI/article/view/20773>)
241 | - Learning Structured Gaussians to Approximate Deep Ensembles [[CVPR2022]](<https://arxiv.org/abs/2203.15485>)
242 | - Improving the reliability for confidence estimation [[ECCV2022]](<https://arxiv.org/abs/2210.06776>)
243 | - Gradient-based Uncertainty for Monocular Depth Estimation [[ECCV2022]](<https://arxiv.org/abs/2208.02005>) - [[PyTorch]](<https://github.com/jhornauer/GrUMoDepth>)
244 | - BayesCap: Bayesian Identity Cap for Calibrated Uncertainty in Frozen Neural Networks [[ECCV2022]](<https://arxiv.org/abs/2207.06873>) - [[PyTorch]](<https://github.com/ExplainableML/BayesCap>)
245 | - Learning Uncertainty For Safety-Oriented Semantic Segmentation In Autonomous Driving [[ICIP2022]](<https://arxiv.org/abs/2105.13688>)
246 | - SLURP: Side Learning Uncertainty for Regression Problems [[BMVC2021]](<https://arxiv.org/abs/2104.02395>) - [[PyTorch]](<https://github.com/xuanlongORZ/SLURP_uncertainty_estimate>)
247 | - Triggering Failures: Out-Of-Distribution detection by learning from local adversarial attacks in Semantic Segmentation [[ICCV2021]](<https://arxiv.org/abs/2108.01634>) - [[PyTorch]](<https://github.com/valeoai/obsnet>)
248 | - Learning to Predict Error for MRI Reconstruction [[MICCAI2021]](<https://arxiv.org/abs/2002.05582>)
249 | - A Mathematical Analysis of Learning Loss for Active Learning in Regression [[CVPR Workshop2021]](<https://openaccess.thecvf.com/content/CVPR2021W/TCV/html/Shukla_A_Mathematical_Analysis_of_Learning_Loss_for_Active_Learning_in_CVPRW_2021_paper.html>)
250 | - Real-time uncertainty estimation in computer vision via uncertainty-aware distribution distillation [[WACV2021]](<https://arxiv.org/abs/2007.15857>)
251 | - On the uncertainty of self-supervised monocular depth estimation [[CVPR2020]](<https://arxiv.org/abs/2005.06209>) - [[PyTorch]](<https://github.com/mattpoggi/mono-uncertainty>)
252 | - Quantifying Point-Prediction Uncertainty in Neural Networks via Residual Estimation with an I/O Kernel [[ICLR2020]](<https://arxiv.org/abs/1906.00588>) - [[TensorFlow]](<https://github.com/cognizant-ai-labs/rio-paper>)
253 | - Gradients as a Measure of Uncertainty in Neural Networks [[ICIP2020]](<https://arxiv.org/abs/2008.08030>)
254 | - Learning Loss for Test-Time Augmentation [[NeurIPS2020]](<https://proceedings.neurips.cc/paper/2020/hash/2ba596643cbbbc20318224181fa46b28-Abstract.html>)
255 | - Learning loss for active learning [[CVPR2019]](<https://arxiv.org/abs/1905.03677>) - [[PyTorch]](<https://github.com/Mephisto405/Learning-Loss-for-Active-Learning>) (unofficial codes)
256 | - Addressing failure prediction by learning model confidence [[NeurIPS2019]](<https://papers.NeurIPS.cc/paper/2019/file/757f843a169cc678064d9530d12a1881-Paper.pdf>) - [[PyTorch]](<https://github.com/valeoai/ConfidNet>)
257 | - Structured Uncertainty Prediction Networks [[CVPR2018]](<https://arxiv.org/abs/1802.07079>) - [[TensorFlow]](<https://github.com/Era-Dorta/tf_mvg>)
258 | - Classification uncertainty of deep neural networks based on gradient information [[IAPR Workshop2018]](<https://arxiv.org/abs/1805.08440>)
259 | 
260 | **Journal**
261 | 
262 | - Towards More Reliable Confidence Estimation [[TPAMI2023]](<https://ieeexplore.ieee.org/abstract/document/10172026/>)
263 | - Confidence Estimation via Auxiliary Models [[TPAMI2021]](<https://arxiv.org/abs/2012.06508>)
264 | 
265 | **Arxiv**
266 | 
267 | - Instance-Aware Observer Network for Out-of-Distribution Object Segmentation [[arXiv2022]](<https://arxiv.org/abs/2207.08782>)
268 | - DEUP: Direct Epistemic Uncertainty Prediction [[arXiv2020]](<https://arxiv.org/abs/2102.08501>)
269 | - Learning Confidence for Out-of-Distribution Detection in Neural Networks [[arXiv2018]](<https://arxiv.org/abs/1802.04865>)
270 | 
271 | ## Data-augmentation/Generation-based-methods
272 | 
273 | **Conference**
274 | 
275 | - Posterior Uncertainty Quantification in Neural Networks using Data Augmentation [[AISTATS2024]](<https://arxiv.org/abs/2403.12729>)
276 | - Learning to Generate Training Datasets for Robust Semantic Segmentation [[WACV2024]](<https://arxiv.org/abs/2308.02535>)
277 | - OpenMix: Exploring Outlier Samples for Misclassification Detection [[CVPR2023]](<https://arxiv.org/abs/2303.17093>) - [[PyTorch]](<https://github.com/Impression2805/OpenMix>)
278 | - On the Pitfall of Mixup for Uncertainty Calibration [[CVPR2023]](<https://openaccess.thecvf.com/content/CVPR2023/html/Wang_On_the_Pitfall_of_Mixup_for_Uncertainty_Calibration_CVPR_2023_paper.html>)
279 | - Diverse, Global and Amortised Counterfactual Explanations for Uncertainty Estimates [[AAAI2022]](<https://arxiv.org/abs/2112.02646>)
280 | - Out-of-distribution Detection with Implicit Outlier Transformation [[ICLR2023]](<https://arxiv.org/abs/2303.05033>) - [[PyTorch]](<https://github.com/qizhouwang/doe>)
281 | - PixMix: Dreamlike Pictures Comprehensively Improve Safety Measures [[CVPR2022]](<https://arxiv.org/abs/2112.05135>)
282 | - RegMixup: Mixup as a Regularizer Can Surprisingly Improve Accuracy & Out-of-Distribution Robustness [[NeurIPS2022]](<https://arxiv.org/abs/2206.14502>) - [[PyTorch]](<https://github.com/francescopinto/regmixup>)
283 | - Towards efficient feature sharing in MIMO architectures [[CVPR Workshop2022]](<https://openaccess.thecvf.com/content/CVPR2022W/ECV/html/Sun_Towards_Efficient_Feature_Sharing_in_MIMO_Architectures_CVPRW_2022_paper.html>)
284 | - Robust Semantic Segmentation with Superpixel-Mix [[BMVC2021]](<https://arxiv.org/abs/2108.00968>) - [[PyTorch]](<https://github.com/giannifranchi/deeplabv3-superpixelmix>)
285 | - MixMo: Mixing Multiple Inputs for Multiple Outputs via Deep Subnetworks [[ICCV2021]](<https://arxiv.org/abs/2103.06132>) - [[PyTorch]](<https://github.com/alexrame/mixmo-pytorch>)
286 | - Training independent subnetworks for robust prediction [[ICLR2021]](<https://arxiv.org/abs/2010.06610>)
287 | - Regularizing Variational Autoencoder with Diversity and Uncertainty Awareness [[IJCAI2021]](<https://arxiv.org/abs/2110.12381>) - [[PyTorch]](<https://github.com/smilesdzgk/du-vae>)
288 | - Uncertainty-aware GAN with Adaptive Loss for Robust MRI Image Enhancement  [[ICCV Workshop2021]](<https://arxiv.org/pdf/2110.03343.pdf>)
289 | - Uncertainty-Aware Deep Classifiers using Generative Models [[AAAI2020]](<https://arxiv.org/abs/2006.04183>)
290 | - Synthesize then Compare: Detecting Failures and Anomalies for Semantic Segmentation [[ECCV2020]](<https://arxiv.org/abs/2003.08440>) - [[PyTorch]](<https://github.com/YingdaXia/SynthCP>)
291 | - Detecting the Unexpected via Image Resynthesis [[ICCV2019]](<https://arxiv.org/abs/1904.07595>) - [[PyTorch]](<https://github.com/cvlab-epfl/detecting-the-unexpected>)
292 | - Mix-n-match: Ensemble and compositional methods for uncertainty calibration in deep learning [[ICML2020]](<http://proceedings.mlr.press/v119/zhang20k/zhang20k.pdf>)
293 | - Deep Anomaly Detection with Outlier Exposure [[ICLR2019]](<https://arxiv.org/pdf/1812.04606.pdf>)
294 | - On Mixup Training: Improved Calibration and Predictive Uncertainty for Deep Neural Networks [[NeurIPS2019]](<https://arxiv.org/abs/1905.11001>)
295 | 
296 | **Arxiv**
297 | 
298 | - Reliability in Semantic Segmentation: Can We Use Synthetic Data? [[arXiv2023]](<https://arxiv.org/pdf/2312.09231.pdf>)
299 | - Quantifying uncertainty with GAN-based priors [[arXiv2019]](<https://openreview.net/forum?id=HyeAPeBFwS>) - [[TensorFlow]](<https://github.com/dhruvpatel108/GANPriors>)
300 | 
301 | ## Output-Space-Modeling/Evidential-deep-learning
302 | 
303 | **Conference**
304 | 
305 | - Hyper Evidential Deep Learning to Quantify Composite Classification Uncertainty [[ICLR2024]](https://arxiv.org/abs/2404.10980)
306 | - The Evidence Contraction Issue in Deep Evidential Regression: Discussion and Solution [[AAAI2024]](<https://ojs.aaai.org/index.php/AAAI/article/view/30172>)
307 | - Discretization-Induced Dirichlet Posterior for Robust Uncertainty Quantification on Regression [[AAAI2024]](<https://arxiv.org/abs/2308.09065>) - [[PyTorch]](<https://github.com/ENSTA-U2IS-AI/DIDO>)
308 | - The Unreasonable Effectiveness of Deep Evidential Regression [[AAAI2023]](<https://arxiv.org/abs/2205.10060>) - [[PyTorch]](<https://github.com/pasteurlabs/unreasonable_effective_der>) - [[TorchUncertainty]](https://github.com/ENSTA-U2IS-AI/torch-uncertainty)
309 | - Exploring and Exploiting Uncertainty for Incomplete Multi-View Classification [[CVPR2023]](https://arxiv.org/abs/2304.05165)
310 | - Plausible Uncertainties for Human Pose Regression [[ICCV2023]](https://openaccess.thecvf.com/content/ICCV2023/papers/Bramlage_Plausible_Uncertainties_for_Human_Pose_Regression_ICCV_2023_paper.pdf) - [[PyTorch]](<https://github.com/biggzlar/plausible-uncertainties>)
311 | - Uncertainty Estimation by Fisher Information-based Evidential Deep Learning [[ICML2023]](https://arxiv.org/pdf/2303.02045.pdf) - [[PyTorch]](<https://github.com/danruod/iedl>)
312 | - Improving Evidential Deep Learning via Multi-task Learning [[AAAI2022]](<https://arxiv.org/abs/2112.09368>) - [[PyTorch]](https://github.com/deargen/MT-ENet)
313 | - An Evidential Neural Network Model for Regression Based on Random Fuzzy Numbers [[BELIEF2022]](<https://arxiv.org/abs/2208.00647>)
314 | - On the Pitfalls of Heteroscedastic Uncertainty Estimation with Probabilistic Neural Networks [[ICLR2022]](<https://arxiv.org/abs/2203.09168>) - [[PyTorch]](https://github.com/martius-lab/beta-nll)
315 | - Natural Posterior Network: Deep Bayesian Uncertainty for Exponential Family Distributions [[ICLR2022]](<https://arxiv.org/abs/2105.04471>) - [[PyTorch]](<https://github.com/borchero/natural-posterior-network>)
316 | - Pitfalls of Epistemic Uncertainty Quantification through Loss Minimisation [[NeurIPS2022]](<https://openreview.net/pdf?id=epjxT_ARZW5>)
317 | - Fast Predictive Uncertainty for Classification with Bayesian Deep Networks [[UAI2022]](<https://arxiv.org/abs/2003.01227>) - [[PyTorch]](<https://github.com/mariushobbhahn/LB_for_BNNs_official>)
318 | - Evaluating robustness of predictive uncertainty estimation: Are Dirichlet-based models reliable? [[ICML2021]](<http://proceedings.mlr.press/v139/kopetzki21a/kopetzki21a.pdf>)
319 | - Trustworthy multimodal regression with mixture of normal-inverse gamma distributions [[NeurIPS2021]](<https://arxiv.org/abs/2111.08456>)
320 | - Misclassification Risk and Uncertainty Quantification in Deep Classifiers [[WACV2021]](<https://openaccess.thecvf.com/content/WACV2021/html/Sensoy_Misclassification_Risk_and_Uncertainty_Quantification_in_Deep_Classifiers_WACV_2021_paper.html>)
321 | - Ensemble Distribution Distillation [[ICLR2020]](<https://arxiv.org/abs/1905.00076>)
322 | - Conservative Uncertainty Estimation By Fitting Prior Networks [[ICLR2020]](<https://openreview.net/forum?id=BJlahxHYDS>)
323 | - Being Bayesian about Categorical Probability [[ICML2020]](<https://arxiv.org/abs/2002.07965>) - [[PyTorch]](<https://github.com/tjoo512/belief-matching-framework>)
324 | - Posterior Network: Uncertainty Estimation without OOD Samples via Density-Based Pseudo-Counts  [[NeurIPS2020]](<https://proceedings.neurips.cc/paper/2020/hash/0eac690d7059a8de4b48e90f14510391-Abstract.html>) - [[PyTorch]](<https://github.com/sharpenb/Posterior-Network>)
325 | - Deep Evidential Regression [[NeurIPS2020]](<https://arxiv.org/abs/1910.02600>) - [[TensorFlow]](<https://github.com/aamini/evidential-deep-learning>) - [[TorchUncertainty]](<https://github.com/ENSTA-U2IS-AI/torch-uncertainty>)
326 | - Noise Contrastive Priors for Functional Uncertainty [[UAI2020]](<https://proceedings.mlr.press/v115/hafner20a.html>)
327 | - Towards Maximizing the Representation Gap between In-Domain & Out-of-Distribution Examples [[NeurIPS Workshop2020]](<https://arxiv.org/abs/2010.10474>)
328 | - Uncertainty on Asynchronous Time Event Prediction [[NeurIPS2019]](<https://arxiv.org/abs/1911.05503>) - [[TensorFlow]](<https://github.com/sharpenb/Uncertainty-Event-Prediction>)
329 | - Reverse KL-Divergence Training of Prior Networks: Improved Uncertainty and Adversarial Robustness [[NeurIPS2019]](<https://proceedings.neurips.cc/paper/2019/hash/7dd2ae7db7d18ee7c9425e38df1af5e2-Abstract.html>)
330 | - Quantifying Classification Uncertainty using Regularized Evidential Neural Networks [[AAAI FSS2019]](<https://arxiv.org/abs/1910.06864>)
331 | - Uncertainty estimates and multi-hypotheses networks for optical flow [[ECCV2018]](<https://arxiv.org/abs/1802.07095>) - [[TensorFlow]](<https://github.com/lmb-freiburg/netdef_models>)
332 | - Evidential Deep Learning to Quantify Classification Uncertainty [[NeurIPS2018]](<https://arxiv.org/abs/1806.01768>) - [[PyTorch]](<https://github.com/dougbrion/pytorch-classification-uncertainty>)
333 | - Predictive uncertainty estimation via prior networks [[NeurIPS2018]](<https://proceedings.neurips.cc/paper/2018/hash/3ea2db50e62ceefceaf70a9d9a56a6f4-Abstract.html>)
334 | - What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? [[NeurIPS2017]](<https://arxiv.org/abs/1703.04977>)
335 | - Estimating the Mean and Variance of the Target Probability Distribution [[(ICNN1994)]](<https://ieeexplore.ieee.org/document/374138>)
336 | 
337 | **Journal**
338 | 
339 | - Prior and Posterior Networks: A Survey on Evidential Deep Learning Methods For Uncertainty Estimation [[TMLR2023]](<https://arxiv.org/abs/2110.03051>)
340 | - Region-Based Evidential Deep Learning to Quantify Uncertainty and Improve Robustness of Brain Tumor Segmentation [[NCA2022]](<http://arxiv.org/abs/2208.06038>)
341 | - An evidential classifier based on Dempster-Shafer theory and deep learning [[Neurocomputing2021]](<https://www.sciencedirect.com/science/article/pii/S0925231221004525>) - [[TensorFlow]](<https://github.com/tongzheng1992/E-CNN-classifier>)
342 | - Evidential fully convolutional network for semantic segmentation [[AppliedIntelligence2021]](<https://link.springer.com/article/10.1007/s10489-021-02327-0>) - [[TensorFlow]](<https://github.com/tongzheng1992/E-FCN>)
343 | - Information Aware max-norm Dirichlet networks for predictive uncertainty estimation [[NeuralNetworks2021]](<https://arxiv.org/abs/1910.04819#:~:text=Information%20Aware%20Max%2DNorm%20Dirichlet%20Networks%20for%20Predictive%20Uncertainty%20Estimation,-Theodoros%20Tsiligkaridis&text=Precise%20estimation%20of%20uncertainty%20in,prone%20to%20over%2Dconfident%20predictions>)
344 | - A neural network classifier based on Dempster-Shafer theory [[IEEETransSMC2000]](<https://ieeexplore.ieee.org/abstract/document/833094/>)
345 | 
346 | **Arxiv**
347 | 
348 | - Evidential Uncertainty Quantification: A Variance-Based Perspective [[arXiv2023]](<https://arxiv.org/pdf/2311.11367.pdf>)
349 | - Effective Uncertainty Estimation with Evidential Models for Open-World Recognition [[arXiv2022]](<https://openreview.net/pdf?id=NrB52z3eOTY>)
350 | - Multivariate Deep Evidential Regression [[arXiv2022]](<https://arxiv.org/abs/2104.06135>)
351 | - Regression Prior Networks [[arXiv2020]](<https://arxiv.org/abs/2006.11590>)
352 | - A Variational Dirichlet Framework for Out-of-Distribution Detection [[arXiv2019]](<https://arxiv.org/abs/1811.07308>)
353 | - Uncertainty estimation in deep learning with application to spoken language assessment [[PhDThesis2019]](<https://www.repository.cam.ac.uk/handle/1810/298857>)
354 | - Inhibited softmax for uncertainty estimation in neural networks [[arXiv2018]](<https://arxiv.org/abs/1810.01861>)
355 | - Quantifying Intrinsic Uncertainty in Classification via Deep Dirichlet Mixture Networks [[arXiv2018]](<https://arxiv.org/abs/1906.04450>)
356 | 
357 | ## Deterministic-Uncertainty-Methods
358 | 
359 | **Conference**
360 | - A Rate-Distortion View of Uncertainty Quantification [[ICML2024]](https://arxiv.org/abs/2406.10775) - [[Tensorflow]](https://github.com/ifiaposto/Distance_Aware_Bottleneck)
361 | - Deep Deterministic Uncertainty: A Simple Baseline [[CVPR2023]](<https://arxiv.org/abs/2102.11582>) - [[PyTorch]](<https://github.com/omegafragger/DDU>)
362 | - Gaussian Latent Representations for Uncertainty Estimation using Mahalanobis Distance in Deep Classifiers [[ICCV Workshop2023]](<https://openaccess.thecvf.com/content/ICCV2023W/UnCV/papers/Venkataramanan_Gaussian_Latent_Representations_for_Uncertainty_Estimation_Using_Mahalanobis_Distance_in_ICCVW_2023_paper.pdf>) - [[PyTorch]](<https://github.com/vaishwarya96/MAPLE-uncertainty-estimation>)
363 | - A Simple and Explainable Method for Uncertainty Estimation using Attribute Prototype Networks [[ICCV Workshop2023]](<https://openaccess.thecvf.com/content/ICCV2023W/UnCV/papers/Zelenka_A_Simple_and_Explainable_Method_for_Uncertainty_Estimation_Using_Attribute_ICCVW_2023_paper.pdf>)
364 | - Training, Architecture, and Prior for Deterministic Uncertainty Methods [[ICLR Workshop2023]](<https://arxiv.org/abs/2303.05796>) - [[PyTorch]](<https://github.com/orientino/dum-components>)
365 | - Latent Discriminant deterministic Uncertainty [[ECCV2022]](<https://arxiv.org/abs/2207.10130>) - [[PyTorch]](<https://github.com/ENSTA-U2IS-AI/LDU>)
366 | - On the Practicality of Deterministic Epistemic Uncertainty [[ICML2022]](<https://arxiv.org/abs/2107.00649>)
367 | - Improving Deterministic Uncertainty Estimation in Deep Learning for Classification and Regression [[CoRR2021]](<https://arxiv.org/abs/2102.11409>)
368 | - Uncertainty Estimation Using a Single Deep Deterministic Neural Network [[ICML2020]](<https://arxiv.org/abs/2003.02037>) - [[PyTorch]](<https://github.com/y0ast/deterministic-uncertainty-quantification>)
369 | - Training normalizing flows with the information bottleneck for competitive generative classification [[NeurIPS2020]](<https://arxiv.org/abs/2001.06448>)
370 | - Simple and principled uncertainty estimation with deterministic deep learning via distance awareness [[NeurIPS2020]](<https://proceedings.neurips.cc/paper/2020/hash/543e83748234f7cbab21aa0ade66565f-Abstract.html>)
371 | - Revisiting One-vs-All Classifiers for Predictive Uncertainty and Out-of-Distribution Detection in Neural Networks [[ICML Workshop2020]](<https://arxiv.org/abs/2007.05134>)
372 | - Sampling-Free Epistemic Uncertainty Estimation Using Approximated Variance Propagation [[ICCV2019]](<https://openaccess.thecvf.com/content_ICCV_2019/html/Postels_Sampling-Free_Epistemic_Uncertainty_Estimation_Using_Approximated_Variance_Propagation_ICCV_2019_paper.html>) - [[PyTorch]](<https://github.com/janisgp/Sampling-free-Epistemic-Uncertainty>)
373 | - Single-Model Uncertainties for Deep Learning [[NeurIPS2019]](<https://arxiv.org/abs/1811.00908>) - [[PyTorch]](<https://github.com/facebookresearch/SingleModelUncertainty/>)
374 | 
375 | **Journal**
376 | 
377 | - ZigZag: Universal Sampling-free Uncertainty Estimation Through Two-Step Inference [[TMLR2024]](<https://arxiv.org/pdf/2211.11435>) - [[Pytorch]](<https://github.com/cvlab-epfl/zigzag>)
378 | - Density estimation in representation space [[EDSMLS2020]](<https://arxiv.org/abs/1908.07235>)
379 | 
380 | **Arxiv**
381 | 
382 | - The Hidden Uncertainty in a Neural Network’s Activations [[arXiv2020]](<https://arxiv.org/abs/2012.03082>)
383 | - A simple framework for uncertainty in contrastive learning [[arXiv2020]](<https://arxiv.org/abs/2010.02038>)
384 | - Distance-based Confidence Score for Neural Network Classifiers [[arXiv2017]](<https://arxiv.org/abs/1709.09844>)
385 | 
386 | ## Quantile-Regression/Predicted-Intervals
387 | 
388 | **Conference**
389 | 
390 | - Image-to-Image Regression with Distribution-Free Uncertainty Quantification and Applications in Imaging [[ICML2022]](<https://arxiv.org/abs/2202.05265>) - [[PyTorch]](<https://github.com/aangelopoulos/im2im-uq>)
391 | - Prediction Intervals: Split Normal Mixture from Quality-Driven Deep Ensembles [[UAI2020]](<http://proceedings.mlr.press/v124/saleh-salem20a.html>) - [[PyTorch]](<https://github.com/tarik/pi-snm-qde>)
392 | - Classification with Valid and Adaptive Coverage [[NeurIPS2020]](<https://proceedings.neurips.cc/paper/2020/hash/244edd7e85dc81602b7615cd705545f5-Abstract.html>)
393 | - Single-Model Uncertainties for Deep Learning [[NeurIPS2019]](<https://arxiv.org/abs/1811.00908>) - [[PyTorch]](<https://github.com/facebookresearch/SingleModelUncertainty/>)
394 | - High-Quality Prediction Intervals for Deep Learning: A Distribution-Free, Ensembled Approach [[ICML2018]](<https://arxiv.org/abs/1802.07167>) - [[TensorFlow]](<https://github.com/TeaPearce/Deep_Learning_Prediction_Intervals>)
395 | 
396 | **Journal**
397 | 
398 | - Scalable Uncertainty Quantification for Deep Operator Networks using Randomized Priors [[CMAME2022]](<https://arxiv.org/abs/2203.03048>)
399 | - Exploring uncertainty in regression neural networks for construction of prediction intervals [[Neurocomputing2022]](<https://www.sciencedirect.com/science/article/abs/pii/S0925231222001102>)
400 | 
401 | **Arxiv**
402 | 
403 | - Interval Neural Networks: Uncertainty Scores [[arXiv2020]](<https://arxiv.org/abs/2003.11566>)
404 | - Tight Prediction Intervals Using Expanded Interval Minimization [[arXiv2018]](<https://arxiv.org/abs/1806.11222>)
405 | 
406 | ## Conformal Predictions
407 | 
408 | Awesome Conformal Prediction [[GitHub]](<https://github.com/valeman/awesome-conformal-prediction>)
409 | 
410 | <!-- **Conference**
411 | 
412 | - Testing for Outliers with Conformal p-values  [[Ann. Statist. 2023]](<https://arxiv.org/abs/2104.08279>) - [[Python]](<https://github.com/msesia/conditional-conformal-pvalues>)
413 | - Uncertainty sets for image classifiers using conformal prediction [[ICLR2021]](https://arxiv.org/pdf/2009.14193.pdf) - [[GitHub]](https://github.com/aangelopoulos/conformal_classification)
414 | - Conformal Prediction Under Covariate Shift [[NeurIPS2019]](<https://proceedings.neurips.cc/paper/2019/hash/8fb21ee7a2207526da55a679f0332de2-Abstract.html>)
415 | - Conformalized Quantile Regression [[NeurIPS2019]](<https://proceedings.neurips.cc/paper/2019/hash/5103c3584b063c431bd1268e9b5e76fb-Abstract.html>) -->
416 | 
417 | ## Calibration/Evaluation-Metrics
418 | 
419 | **Conference**
420 | 
421 | - Smooth ECE: Principled Reliability Diagrams via Kernel Smoothing [[ICLR2024]](<https://arxiv.org/abs/2309.12236>)
422 | - Calibrating Transformers via Sparse Gaussian Processes [[ICLR2023]](<https://arxiv.org/abs/2303.02444>) - [[PyTorch]](<https://github.com/chenw20/sgpa>)
423 | - Beyond calibration: estimating the grouping loss of modern neural networks [[ICLR2023]](<https://openreview.net/pdf?id=6w1k-IixnL8>) - [[Python]](<https://github.com/aperezlebel/beyond_calibration>)
424 | - Dual Focal Loss for Calibration [[ICML 2023]](https://arxiv.org/abs/2305.13665)
425 | - What Are Effective Labels for Augmented Data? Improving Calibration and Robustness with AutoLabel [[SaTML2023]](https://arxiv.org/abs/2302.11188)
426 | - The Devil is in the Margin: Margin-based Label Smoothing for Network Calibration [[CVPR2022]](<https://arxiv.org/abs/2111.15430>) - [[PyTorch]](<https://github.com/by-liu/mbls>)
427 | - AdaFocal: Calibration-aware Adaptive Focal Loss [[NeurIPS2022]](https://arxiv.org/abs/2211.11838)
428 | - Calibrating Deep Neural Networks by Pairwise Constraints [[CVPR2022]](<https://openaccess.thecvf.com/content/CVPR2022/html/Cheng_Calibrating_Deep_Neural_Networks_by_Pairwise_Constraints_CVPR_2022_paper.html>)
429 | - Top-label calibration and multiclass-to-binary reductions [[ICLR2022]](<https://openreview.net/forum?id=WqoBaaPHS->)
430 | - From label smoothing to label relaxation [[AAAI2021]](<https://www.aaai.org/AAAI21Papers/AAAI-2191.LienenJ.pdf>)
431 | - Diagnostic Uncertainty Calibration: Towards Reliable Machine Predictions in Medical Domain [[AIStats2021]](https://arxiv.org/pdf/2007.01659)
432 | - Rethinking Calibration of Deep Neural Networks: Do Not Be Afraid of Overconfidence [[NeurIPS2021]](<https://proceedings.neurips.cc/paper/2021/hash/61f3a6dbc9120ea78ef75544826c814e-Abstract.html>)
433 | - Beyond Pinball Loss: Quantile Methods for Calibrated Uncertainty Quantification [[NeurIPS2021]](<https://arxiv.org/abs/2011.09588>)
434 | - Confidence-Aware Learning for Deep Neural Networks [[ICML2020]](<https://arxiv.org/abs/2007.01458>) - [[PyTorch]](<https://github.com/daintlab/confidence-aware-learning>)
435 | - Mix-n-match: Ensemble and compositional methods for uncertainty calibration in deep learning [[ICML2020]](<http://proceedings.mlr.press/v119/zhang20k/zhang20k.pdf>)
436 | - Regularization via structural label smoothing [[ICML2020]](<https://proceedings.mlr.press/v108/li20e.html>)
437 | - Well-Calibrated Regression Uncertainty in Medical Imaging with Deep Learning [[MIDL2020]](<http://proceedings.mlr.press/v121/laves20a.html>) - [[PyTorch]](<https://github.com/mlaves/well-calibrated-regression-uncertainty>)
438 | - Calibrating Deep Neural Networks using Focal Loss [[NeurIPS2020]](<https://arxiv.org/abs/2002.09437>) - [[PyTorch]](<https://github.com/torrvision/focal_calibration>)
439 | - Stationary activations for uncertainty calibration in deep learning [[NeurIPS2020]](<https://proceedings.neurips.cc/paper/2020/hash/18a411989b47ed75a60ac69d9da05aa5-Abstract.html>)
440 | - Revisiting the evaluation of uncertainty estimation and its application to explore model complexity-uncertainty trade-off [[CVPR Workshop2020]](<https://openaccess.thecvf.com/content_CVPRW_2020/html/w1/Ding_Revisiting_the_Evaluation_of_Uncertainty_Estimation_and_Its_Application_to_CVPRW_2020_paper.html>)
441 | - Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Vision [[CVPR Workshop2020]](<https://arxiv.org/abs/1906.01620>) - [[PyTorch]](<https://github.com/fregu856/evaluating_bdl>)
442 | - Bias-Reduced Uncertainty Estimation for Deep Neural Classifiers [[ICLR2019]](<https://arxiv.org/abs/1805.08206>)
443 | - Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibration [[NeurIPS2019]](<https://arxiv.org/pdf/1910.12656.pdf>) - [[GitHub]](<https://github.com/dirichletcal>)
444 | - When does label smoothing help? [[NeurIPS2019]](<https://proceedings.neurips.cc/paper/2019/hash/f1748d6b0fd9d439f71450117eba2725-Abstract.html>)
445 | - Verified Uncertainty Calibration [[NeurIPS2019]](<https://papers.NeurIPS.cc/paper/2019/hash/f8c0c968632845cd133308b1a494967f-Abstract.html>) - [[GitHub]](<https://github.com/p-lambda/verified_calibration>)
446 | - Measuring Calibration in Deep Learning [[CVPR Workshop2019]](<https://arxiv.org/abs/1904.01685>)
447 | - Accurate Uncertainties for Deep Learning Using Calibrated Regression [[ICML2018]](<https://arxiv.org/abs/1807.00263>)
448 | - Generalized zero-shot learning with deep calibration network [[NeurIPS2018]](<https://proceedings.neurips.cc/paper/2018/hash/1587965fb4d4b5afe8428a4a024feb0d-Abstract.html>)
449 | - On calibration of modern neural networks [[ICML2017]](<https://arxiv.org/abs/1706.04599>) - [[TorchUncertainty]](https://github.com/ENSTA-U2IS-AI/torch-uncertainty)
450 | - On Fairness and Calibration [[NeurIPS2017]](<https://arxiv.org/abs/1709.02012>)
451 | - Obtaining Well Calibrated Probabilities Using Bayesian Binning [[AAAI2015]](<https://ojs.aaai.org/index.php/AAAI/article/view/9602/9461>)
452 | 
453 | **Journal**
454 | 
455 | - Meta-Calibration: Learning of Model Calibration Using Differentiable Expected Calibration Error [[TMLR2023]](<https://arxiv.org/abs/2106.09613>) - [[PyTorch]](<https://github.com/ondrejbohdal/meta-calibration>)
456 | - Evaluating and Calibrating Uncertainty Prediction in Regression Tasks [[Sensors2022]](<https://arxiv.org/abs/1905.11659>)
457 | - Calibrated Prediction Intervals for Neural Network Regressors [[IEEE Access 2018]](<https://arxiv.org/abs/1803.09546>) - [[Python]](<https://github.com/cruvadom/Prediction_Intervals>)
458 | 
459 | **Arxiv**
460 | 
461 | - Towards Understanding Label Smoothing [[arXiv2020]](<https://arxiv.org/abs/2006.11653>)
462 | - An Investigation of how Label Smoothing Affects Generalization [[arXiv2020]](<https://arxiv.org/abs/2010.12648>)
463 |   
464 | ## Misclassification Detection & Selective Classification
465 | 
466 | - A Data-Driven Measure of Relative Uncertainty for Misclassification Detection [[ICLR2024]](https://arxiv.org/abs/2306.01710)
467 | - Plugin estimators for selective classification with out-of-distribution detection [[ICLR2024]](https://arxiv.org/abs/2301.12386)
468 | - SURE: SUrvey REcipes for building reliable and robust deep networks [[CVPR2024]](https://arxiv.org/abs/2403.00543) - [[PyTorch]](https://yutingli0606.github.io/SURE/)
469 | - Augmenting Softmax Information for Selective Classification with Out-of-Distribution Data [[ACCV2022]](<https://openaccess.thecvf.com/content/ACCV2022/html/Xia_Augmenting_Softmax_Information_for_Selective_Classification_with_Out-of-Distribution_Data_ACCV_2022_paper.html>)
470 | - Anomaly Detection via Reverse Distillation from One-Class Embedding [[CVPR2022]](<https://arxiv.org/abs/2201.10703>)
471 | - Rethinking Confidence Calibration for Failure Prediction [[ECCV2022]](<https://link.springer.com/chapter/10.1007/978-3-031-19806-9_30>) - [[PyTorch]](<https://github.com/Impression2805/FMFP>)
472 | 
473 | ## Applications
474 | 
475 | ### Classification and Semantic-Segmentation
476 | 
477 | **Conference**
478 | 
479 | - Modeling Multimodal Aleatoric Uncertainty in Segmentation with Mixture of Stochastic Experts [[ICLR2023]](<https://arxiv.org/abs/2212.07328>) - [[PyTorch]](<https://github.com/gaozhitong/mose-auseg>)
480 | - Anytime Dense Prediction with Confidence Adaptivity [[ICLR2022]](<https://openreview.net/forum?id=kNKFOXleuC>) - [[PyTorch]](<https://github.com/liuzhuang13/anytime>)
481 | - CRISP - Reliable Uncertainty Estimation for Medical Image Segmentation [[MICCAI2022]](<https://arxiv.org/abs/2206.07664>)
482 | - TBraTS: Trusted Brain Tumor Segmentation [[MICCAI2022]](<https://arxiv.org/abs/2206.09309>) - [[PyTorch]](<https://github.com/cocofeat/tbrats>)
483 | - Robust Semantic Segmentation with Superpixel-Mix [[BMVC2021]](<https://arxiv.org/abs/2108.00968>) - [[PyTorch]](<https://github.com/giannifranchi/deeplabv3-superpixelmix>)
484 | - Deep Deterministic Uncertainty for Semantic Segmentation [[ICMLW2021]](<https://arxiv.org/abs/2111.00079>)
485 | - DEAL: Difficulty-aware Active Learning for Semantic Segmentation [[ACCV2020]](<https://openaccess.thecvf.com/content/ACCV2020/html/Xie_DEAL_Difficulty-aware_Active_Learning_for_Semantic_Segmentation_ACCV_2020_paper.html>)
486 | - Classification with Valid and Adaptive Coverage [[NeurIPS2020]](<https://proceedings.neurips.cc/paper/2020/hash/244edd7e85dc81602b7615cd705545f5-Abstract.html>)
487 | - Guided Curriculum Model Adaptation and Uncertainty-Aware Evaluation for Semantic Nighttime Image Segmentation [[ICCV2019]](<https://openaccess.thecvf.com/content_ICCV_2019/html/Sakaridis_Guided_Curriculum_Model_Adaptation_and_Uncertainty-Aware_Evaluation_for_Semantic_Nighttime_ICCV_2019_paper.html>)
488 | - Human Uncertainty Makes Classification More Robust [[ICCV2019]](<https://openaccess.thecvf.com/content_ICCV_2019/html/Peterson_Human_Uncertainty_Makes_Classification_More_Robust_ICCV_2019_paper.html>)
489 | - Uncertainty-aware self-ensembling model for semi-supervised 3D left atrium segmentation [[MICCAI2019]](<https://arxiv.org/abs/1806.05034>) - [[PyTorch]](<https://github.com/yulequan/UA-MT>)
490 | - Lightweight Probabilistic Deep Networks [[CVPR2018]](<https://arxiv.org/abs/1805.11327>) - [[PyTorch]](<https://github.com/ezjong/lightprobnets>)
491 | - A Probabilistic U-Net for Segmentation of Ambiguous Images [[NeurIPS2018]](<https://arxiv.org/abs/1806.05034>) - [[PyTorch]](<https://github.com/stefanknegt/Probabilistic-Unet-Pytorch>)
492 | - Evidential Deep Learning to Quantify Classification Uncertainty [[NeurIPS2018]](<https://arxiv.org/abs/1806.01768>) - [[PyTorch]](<https://github.com/dougbrion/pytorch-classification-uncertainty>)
493 | - To Trust Or Not To Trust A Classifier [[NeurIPS2018]](<https://proceedings.neurips.cc/paper/2018/hash/7180cffd6a8e829dacfc2a31b3f72ece-Abstract.html>)
494 | - Classification uncertainty of deep neural networks based on gradient information [[IAPR Workshop2018]](<https://arxiv.org/abs/1805.08440>)
495 | - Bayesian segnet: Model uncertainty in deep convolutional encoder-decoder architectures for scene understanding [[BMVC2017]](<https://arxiv.org/abs/1511.02680>)
496 | 
497 | **Journal**
498 | 
499 | - Explainable machine learning in image classification models: An uncertainty quantification perspective." [[KnowledgeBased2022]](<https://www.sciencedirect.com/science/article/pii/S095070512200168X>)
500 | - Region-Based Evidential Deep Learning to Quantify Uncertainty and Improve Robustness of Brain Tumor Segmentation [[NCA2022]](<https://arxiv.org/abs/2208.06038>)
501 | 
502 | **Arxiv**
503 | 
504 | - Leveraging Uncertainty Estimates to Improve Classifier Performance [[arXiv2023]](<https://arxiv.org/pdf/2311.11723.pdf>)
505 | - Evaluating Bayesian Deep Learning Methods for Semantic Segmentation [[arXiv2018]](<https://arxiv.org/abs/1811.12709>)
506 | 
507 | ### Regression
508 | 
509 | **Conference**
510 | 
511 | - Learning the Distribution of Errors in Stereo Matching for Joint Disparity and Uncertainty Estimation [[CVPR2023]](<https://arxiv.org/abs/2304.00152>) - [[PyTorch]](<https://github.com/lly00412/sednet>)
512 | - Probabilistic MIMO U-Net: Efficient and Accurate Uncertainty Estimation for Pixel-wise Regression [[ICCV Workshop2023]](<https://arxiv.org/abs/2308.07477>) - [[PyTorch]](<https://github.com/antonbaumann/mimo-unet>)
513 | - Training-Free Uncertainty Estimation for Dense Regression: Sensitivity as a Surrogate [[AAAI2022]](<https://arxiv.org/abs/1910.04858v3>)
514 | - Learning Structured Gaussians to Approximate Deep Ensembles [[CVPR2022]](<https://arxiv.org/abs/2203.15485>)
515 | - Uncertainty Quantification in Depth Estimation via Constrained Ordinal Regression [[ECCV2022]](<https://www.ecva.net/papers/eccv_2022/papers_ECCV/papers/136620229.pdf>)
516 | - On Monocular Depth Estimation and Uncertainty Quantification using Classification Approaches for Regression [[ICIP2022]](<https://arxiv.org/abs/2202.12369>)
517 | - Anytime Dense Prediction with Confidence Adaptivity [[ICLR2022]](<https://openreview.net/forum?id=kNKFOXleuC>) - [[PyTorch]](<https://github.com/liuzhuang13/anytime>)
518 | - Variational Depth Networks: Uncertainty-Aware Monocular Self-supervised Depth Estimation [[ECCV Workshop2022]](<https://link.springer.com/chapter/10.1007/978-3-031-25085-9_3>)
519 | - SLURP: Side Learning Uncertainty for Regression Problems [[BMVC2021]](<https://arxiv.org/abs/2104.02395>) - [[PyTorch]](<https://github.com/xuanlongORZ/SLURP_uncertainty_estimate>)
520 | - Robustness via Cross-Domain Ensembles [[ICCV2021]](<https://arxiv.org/abs/2103.10919>) - [[PyTorch]](<https://github.com/EPFL-VILAB/XDEnsembles>)
521 | - Learning to Predict Error for MRI Reconstruction [[MICCAI2021]](<https://arxiv.org/abs/2002.05582>)
522 | - On the uncertainty of self-supervised monocular depth estimation [[CVPR2020]](<https://arxiv.org/abs/2005.06209>) - [[PyTorch]](<https://github.com/mattpoggi/mono-uncertainty>)
523 | - Quantifying Point-Prediction Uncertainty in Neural Networks via Residual Estimation with an I/O Kernel [[ICLR2020]](<https://arxiv.org/abs/1906.00588>) - [[TensorFlow]](<https://github.com/cognizant-ai-labs/rio-paper>)
524 | - Fast Uncertainty Estimation for Deep Learning Based Optical Flow [[IROS2020]](<https://authors.library.caltech.edu/104758/>)
525 | - Well-Calibrated Regression Uncertainty in Medical Imaging with Deep Learning [[MIDL2020]](<http://proceedings.mlr.press/v121/laves20a.html>) - [[PyTorch]](<https://github.com/mlaves/well-calibrated-regression-uncertainty>)
526 | - Deep Evidential Regression [[NeurIPS2020]](<https://arxiv.org/abs/1910.02600>) - [[TensorFlow]](<https://github.com/aamini/evidential-deep-learning>)
527 | - Inferring Distributions Over Depth from a Single Image [[IROS2019]](<https://arxiv.org/abs/1912.06268>) - [[TensorFlow]](<https://github.com/gengshan-y/monodepth-uncertainty>)
528 | - Multi-Task Learning based on Separable Formulation of Depth Estimation and its Uncertainty [[CVPR Workshop2019]](<https://openaccess.thecvf.com/content_CVPRW_2019/html/Uncertainty_and_Robustness_in_Deep_Visual_Learning/Asai_Multi-Task_Learning_based_on_Separable_Formulation_of_Depth_Estimation_and_CVPRW_2019_paper.html>)
529 | - Lightweight Probabilistic Deep Networks [[CVPR2018]](<https://arxiv.org/abs/1805.11327>) - [[PyTorch]](<https://github.com/ezjong/lightprobnets>)
530 | - Structured Uncertainty Prediction Networks [[CVPR2018]](<https://arxiv.org/abs/1802.07079>) - [[TensorFlow]](<https://github.com/Era-Dorta/tf_mvg>)
531 | - Uncertainty estimates and multi-hypotheses networks for optical flow [[ECCV2018]](<https://arxiv.org/abs/1802.07095>) - [[TensorFlow]](<https://github.com/lmb-freiburg/netdef_models>)
532 | - Accurate Uncertainties for Deep Learning Using Calibrated Regression [[ICML2018]](<https://arxiv.org/abs/1807.00263>)
533 | 
534 | **Journal**
535 | 
536 | - How Reliable is Your Regression Model's Uncertainty Under Real-World Distribution Shifts? [[TMLR2023]](<https://arxiv.org/abs/2302.03679>) - [[PyTorch]](<https://github.com/fregu856/regression_uncertainty>)
537 | - Evaluating and Calibrating Uncertainty Prediction in Regression Tasks [[Sensors2022]](<https://arxiv.org/abs/1905.11659>)
538 | - Exploring uncertainty in regression neural networks for construction of prediction intervals [[Neurocomputing2022]](<https://www.sciencedirect.com/science/article/abs/pii/S0925231222001102>)
539 | - Wasserstein Dropout [[Machine Learning 2022]](<https://arxiv.org/abs/2012.12687>) - [[PyTorch]](<https://github.com/fraunhofer-iais/second-moment-loss>)
540 | - Deep Distribution Regression [[Computational Statistics & Data Analysis2021]](<https://arxiv.org/abs/1903.06023>)
541 | - Calibrated Prediction Intervals for Neural Network Regressors [[IEEE Access 2018]](<https://arxiv.org/abs/1803.09546>) - [[Python]](<https://github.com/cruvadom/Prediction_Intervals>)
542 | - Learning a Confidence Measure for Optical Flow [[TPAMI2013]](<https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=6261321&casa_token=fYVGhK2pa40AAAAA:XWJdS8zJ4JRw1brCIGiYpzEqMidXTTYVkcKTYnnhSl4ys5pUoHzHO6xsVeGZII9Ir1LAI_3YyfI&tag=1>)
543 | 
544 | **Arxiv**
545 | 
546 | - Understanding pathologies of deep heteroskedastic regression [[arxiv2024]](<https://arxiv.org/abs/2306.16717>)
547 | - Measuring and Modeling Uncertainty Degree for Monocular Depth Estimation [[arXiv2023]](<https://arxiv.org/abs/2307.09929>)
548 | - UncertaINR: Uncertainty Quantification of End-to-End Implicit Neural Representations for Computed Tomographaphy [[arXiv2022]](<https://arxiv.org/abs/2202.10847>)
549 | - Efficient Gaussian Neural Processes for Regression [[arXiv2021]](<https://arxiv.org/abs/2108.09676>)
550 | 
551 | ### Anomaly-detection and Out-of-Distribution-Detection
552 | 
553 | **Conference**
554 | 
555 | - Learning Transferable Negative Prompts for Out-of-Distribution Detection [[CVPR2024]](<https://arxiv.org/abs/2404.03248>) - [[PyTorch]](<https://github.com/mala-lab/negprompt>)
556 | - Epistemic Uncertainty Quantification For Pre-trained Neural Networks [[CVPR2024]](<https://arxiv.org/abs/2404.10124>)
557 | - NECO: NEural Collapse Based Out-of-distribution Detection [[ICLR2024]](<https://arxiv.org/abs/2310.06823>)
558 | - When and How Does In-Distribution Label Help Out-of-Distribution Detection? [[ICML2024]](<https://arxiv.org/abs/2405.18635>) - [[PyTorch]](<https://github.com/deeplearning-wisc/id_label>)
559 | - Anomaly Detection under Distribution Shift [[ICCV2023]](<https://arxiv.org/abs/2303.13845>) - [[PyTorch]](<https://github.com/mala-lab/ADShift>)
560 | - Normalizing Flows for Human Pose Anomaly Detection [[ICCV2023]](https://orhir.github.io/STG_NF/) - [[PyTorch]](https://github.com/orhir/stg-nf)
561 | - RbA: Segmenting Unknown Regions Rejected by All [[ICCV2023]](https://openaccess.thecvf.com/content/ICCV2023/papers/Nayal_RbA_Segmenting_Unknown_Regions_Rejected_by_All_ICCV_2023_paper.pdf) - [[PyTorch]](https://github.com/NazirNayal8/RbA)
562 | - Uncertainty-Aware Optimal Transport for Semantically Coherent Out-of-Distribution Detection [[CVPR2023]](<https://arxiv.org/abs/2303.10449>) - [[PyTorch]](<https://github.com/lufan31/et-ood>)
563 | - Modeling the Distributional Uncertainty for Salient Object Detection Models [[CVPR2023]](https://npucvr.github.io/Distributional_uncer/) - [[PyTorch]](https://github.com/txynwpu/Distributional_uncertainty_SOD)
564 | - SQUID: Deep Feature In-Painting for Unsupervised Anomaly Detection [[CVPR2023]](<https://arxiv.org/abs/2111.13495>) - [[PyTorch]](<https://github.com/tiangexiang/SQUID>)
565 | - How to Exploit Hyperspherical Embeddings for Out-of-Distribution Detection? [[ICLR2023]](<https://arxiv.org/pdf/2203.04450.pdf>) - [[PyTorch]](<https://github.com/deeplearning-wisc/cider>)
566 | - Modeling the Data-Generating Process is Necessary for Out-of-Distribution Generalization [[ICLR2023]](<https://arxiv.org/pdf/2206.07837.pdf>)
567 | - Can CNNs Be More Robust Than Transformers? [[ICLR2023]](<https://arxiv.org/pdf/2206.03452.pdf>)
568 | - A framework for benchmarking class-out-of-distribution detection and its application to ImageNet [[ICLR2023]](<https://arxiv.org/pdf/2302.11893.pdf>)
569 | - Extremely Simple Activation Shaping for Out-of-Distribution Detection [[ICLR2023]](<https://arxiv.org/abs/2209.09858>) - [[PyTorch]](<https://github.com/andrijazz/ash>)
570 | - Quantification of Uncertainty with Adversarial Models [[NeurIPS2023]](<https://arxiv.org/abs/2307.03217>)
571 | - The Robust Semantic Segmentation UNCV2023 Challenge Results [[ICCV Workshop2023]](https://arxiv.org/abs/2309.15478)
572 | - Continual Evidential Deep Learning for Out-of-Distribution Detection [[ICCV Workshop2023]](https://openaccess.thecvf.com/content/ICCV2023W/VCL/html/Aguilar_Continual_Evidential_Deep_Learning_for_Out-of-Distribution_Detection_ICCVW_2023_paper.html)
573 | - Far Away in the Deep Space: Nearest-Neighbor-Based Dense Out-of-Distribution Detection [[ICCV Workshop2023]](<https://arxiv.org/abs/2211.06660>)
574 | - Gaussian Latent Representations for Uncertainty Estimation using Mahalanobis Distance in Deep Classifiers [[ICCV Workshop2023]](<https://arxiv.org/abs/2305.13849>)
575 | - Calibrated Out-of-Distribution Detection with a Generic Representation [[ICCV Workshop2023]](<https://arxiv.org/abs/2303.13148>) - [[PyTorch]](<https://github.com/vojirt/grood>)
576 | - Detecting Misclassification Errors in Neural Networks with a Gaussian Process Model [[AAAI2022]](<https://ojs.aaai.org/index.php/AAAI/article/view/20773>)
577 | - Towards Total Recall in Industrial Anomaly Detection [[CVPR2022]](<https://arxiv.org/abs/2106.08265>) - [[PyTorch]](<https://github.com/hcw-00/PatchCore_anomaly_detection>)
578 | - POEM: Out-of-Distribution Detection with Posterior Sampling [[ICML2022]](<https://arxiv.org/abs/2206.13687>) - [[PyTorch]](<https://github.com/deeplearning-wisc/poem>)
579 | - VOS: Learning What You Don't Know by Virtual Outlier Synthesis [[ICLR2022]](<https://arxiv.org/abs/2202.01197>) - [[PyTorch]](<https://github.com/deeplearning-wisc/vos>)
580 | - Fully Convolutional Cross-Scale-Flows for Image-based Defect Detection [[WACV2022]](<https://arxiv.org/abs/2110.02855>) - [[PyTorch]](<https://github.com/marco-rudolph/cs-flow>)
581 | - Out-of-Distribution Detection Using Union of 1-Dimensional Subspaces [[CVPR2021]](<https://openaccess.thecvf.com/content/CVPR2021/html/Zaeemzadeh_Out-of-Distribution_Detection_Using_Union_of_1-Dimensional_Subspaces_CVPR_2021_paper.html>) - [[PyTorch]](<https://github.com/zaeemzadeh/OOD>)
582 | - NAS-OoD: Neural Architecture Search for Out-of-Distribution Generalization [[ICCV2021]](<https://arxiv.org/abs/2109.02038>)
583 | - On the Importance of Gradients for Detecting Distributional Shifts in the Wild [[NeurIPS2021]](<https://arxiv.org/abs/2110.00218>)
584 | - Exploring the Limits of Out-of-Distribution Detection [[NeurIPS2021]](<https://arxiv.org/abs/2106.03004>)
585 | - Detecting out-of-distribution image without learning from out-of-distribution data. [[CVPR2020]](<https://openaccess.thecvf.com/content_CVPR_2020/html/Hsu_Generalized_ODIN_Detecting_Out-of-Distribution_Image_Without_Learning_From_Out-of-Distribution_Data_CVPR_2020_paper.html>)
586 | - Learning Open Set Network with Discriminative Reciprocal Points [[ECCV2020]](<https://arxiv.org/abs/2011.00178>)
587 | - Synthesize then Compare: Detecting Failures and Anomalies for Semantic Segmentation [[ECCV2020]](<https://arxiv.org/abs/2003.08440>) - [[PyTorch]](<https://github.com/YingdaXia/SynthCP>)
588 | - NADS: Neural Architecture Distribution Search for Uncertainty Awareness [[ICML2020]](<https://arxiv.org/abs/2006.06646>)
589 | - PaDiM: a Patch Distribution Modeling Framework for Anomaly Detection and Localization [[ICPR2020]](<https://arxiv.org/abs/2011.08785>) - [[PyTorch]](<https://github.com/openvinotoolkit/anomalib>)
590 | - Energy-based Out-of-distribution Detection [[NeurIPS2020]](<https://arxiv.org/abs/2010.03759?context=cs>)
591 | - Towards Maximizing the Representation Gap between In-Domain & Out-of-Distribution Examples [[NeurIPS Workshop2020]](<https://arxiv.org/abs/2010.10474>)
592 | - Memorizing Normality to Detect Anomaly: Memory-Augmented Deep Autoencoder for Unsupervised Anomaly Detection [[ICCV2019]](<https://arxiv.org/abs/1904.02639>) - [[PyTorch]](<https://github.com/donggong1/memae-anomaly-detection>)
593 | - Detecting the Unexpected via Image Resynthesis [[ICCV2019]](<https://arxiv.org/abs/1904.07595>) - [[PyTorch]](<https://github.com/cvlab-epfl/detecting-the-unexpected>)
594 | - Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks [[ICLR2018]](<https://arxiv.org/abs/1706.02690>)
595 | - A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks [[ICLR2017]](<https://arxiv.org/abs/1610.02136>) - [[TensorFlow]](<https://github.com/hendrycks/error-detection>)
596 | 
597 | **Journal**
598 | 
599 | - Revisiting Confidence Estimation: Towards Reliable Failure Prediction [[TPAMI2023]](https://www.computer.org/csdl/journal/tp/5555/01/10356834/1SQHDHvGg9i) - [[PyTorch]](<https://github.com/Impression2805/FMFP>)
600 | - One Versus all for deep Neural Network for uncertaInty (OVNNI) quantification [[IEEE Access2021]](<https://arxiv.org/abs/2006.00954>)
601 | 
602 | **Arxiv**
603 | 
604 | - Neuron Activation Coverage: Rethinking Out-of-distribution Detection and Generalization [[arXiv2023]](<https://arxiv.org/abs/2306.02879>) - [[PyTorch]](<https://github.com/bierone/ood_coverage>)
605 | - A Simple Fix to Mahalanobis Distance for Improving Near-OOD Detection [[arXiv2021]](<https://arxiv.org/abs/2106.09022>)
606 | - Generalized out-of-distribution detection: A survey [[arXiv2021]](<https://arxiv.org/abs/2110.11334>)
607 | - Do We Really Need to Learn Representations from In-domain Data for Outlier Detection? [[arXiv2021]](<https://arxiv.org/abs/2105.09270>)
608 | - Frequentist uncertainty estimates for deep learning [[arXiv2018]](<http://bayesiandeeplearning.org/2018/papers/31.pdf>)
609 | 
610 | ### Object detection
611 | 
612 | **Conference**
613 | 
614 | - Bridging Precision and Confidence: A Train-Time Loss for Calibrating Object Detection [[CVPR2023]](<https://arxiv.org/pdf/2303.14404.pdf>) - [[PyTorch]](<https://github.com/akhtarvision/bpc_calibration?tab=readme-ov-file>)
615 | - Parametric and Multivariate Uncertainty Calibration for Regression and Object Detection [[ECCV Workshop2022]](<https://arxiv.org/abs/2207.01242>) - [[PyTorch]](<https://github.com/EFS-OpenSource/calibration-framework>)
616 | - Estimating and Evaluating Regression Predictive Uncertainty in Deep Object Detectors [[ICLR2021]](<https://openreview.net/forum?id=YLewtnvKgR7>) - [[PyTorch]](<https://github.com/asharakeh/probdet?tab=readme-ov-file>)
617 | - Multivariate Confidence Calibration for Object Detection [[CVPR Workshop2020]](<https://arxiv.org/abs/2004.13546>) - [[PyTorch]](<https://github.com/EFS-OpenSource/calibration-framework>)
618 | - Gaussian YOLOv3: An Accurate and Fast Object Detector Using Localization Uncertainty for Autonomous Driving [[ICCV2019]](<https://openaccess.thecvf.com/content_ICCV_2019/papers/Choi_Gaussian_YOLOv3_An_Accurate_and_Fast_Object_Detector_Using_Localization_ICCV_2019_paper.pdf>) - [[CUDA]](<https://github.com/jwchoi384/Gaussian_YOLOv3>) - [[PyTorch]](<https://github.com/motokimura/PyTorch_Gaussian_YOLOv3>) - [[Keras]](<https://github.com/xuannianz/keras-GaussianYOLOv3>)
619 | 
620 | ### Domain adaptation
621 | 
622 | **Conference**
623 | 
624 | - Guiding Pseudo-labels with Uncertainty Estimation for Source-free Unsupervised Domain Adaptation [[CVPR2023]](<https://arxiv.org/abs/2303.03770>) - [[PyTorch]](https://github.com/mattialitrico/guiding-pseudo-labels-with-uncertainty-estimation-for-source-free-unsupervised-domain-adaptation)
625 | - Uncertainty-guided Source-free
626 | Domain Adaptation [[ECCV2022]](<https://arxiv.org/pdf/2208.07591.pdf>) - [[PyTorch]](<https://github.com/roysubhankar/uncertainty-sfda>)
627 | 
628 | ### Semi-supervised
629 | 
630 | **Conference**
631 | 
632 | - Confidence Estimation Using Unlabeled Data [[ICLR2023]](<https://openreview.net/pdf?id=sOXU-PEJSgQ>) - [[PyTorch]](<https://github.com/TopoXLab/consistency-ranking-loss>)
633 | 
634 | ### Natural Language Processing
635 | 
636 | Awesome LLM Uncertainty, Reliability, & Robustness [[GitHub]](<https://github.com/jxzhangjhu/Awesome-LLM-Uncertainty-Reliability-Robustness>)
637 | 
638 | 
639 | **Conference**
640 | 
641 | - R-U-SURE? Uncertainty-Aware Code Suggestions By Maximizing Utility Across Random User Intents [[ICML2023]](<https://arxiv.org/pdf/2303.00732.pdf>) - [[GitHub]](https://github.com/google-research/r_u_sure)
642 | - Strength in Numbers: Estimating Confidence of Large Language Models by Prompt Agreement [[TrustNLP2023]](<https://aclanthology.org/2023.trustnlp-1.28/>) - [[GitHub]](https://github.com/JHU-CLSP/Confidence-Estimation-TrustNLP2023)
643 | - Disentangling Uncertainty in Machine Translation Evaluation [[EMNLP2022]](<https://arxiv.org/abs/2204.06546>) - [[PyTorch]](<https://github.com/deep-spin/uncertainties_mt_eval>)
644 | - Investigating Ensemble Methods for Model Robustness Improvement of Text Classifiers [[EMNLP2022 Findings]](<https://arxiv.org/abs/2210.16298>)
645 | - DATE: Detecting Anomalies in Text via Self-Supervision of Transformers [[NAACL2021]](<https://arxiv.org/abs/2104.05591>)
646 | - Calibrating Structured Output Predictors for Natural Language Processing [[ACL2020]](<https://aclanthology.org/2020.acl-main.188/>)
647 | - Calibrated Language Model Fine-Tuning for In- and Out-of-Distribution Data [[EMNLP2020]](<https://aclanthology.org/2020.emnlp-main.102/>) - [[PyTorch]](https://github.com/Lingkai-Kong/Calibrated-BERT-Fine-Tuning)
648 | 
649 | **Journal**
650 | - How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering [[TACL2021]](https://arxiv.org/abs/2012.00955) - [[PyTorch]](https://github.com/jzbjyb/lm-calibration)
651 | 
652 | **Arxiv**
653 | 
654 | - Gaussian Stochastic Weight Averaging for Bayesian Low-Rank Adaptation of Large Language Models [[arXiv2024]](<https://arxiv.org/pdf/2405.03425>)
655 | - To Believe or Not to Believe Your LLM [[arXiv2024]](<https://arxiv.org/abs/2406.02543>)
656 | - Decomposing Uncertainty for Large Language Models through Input Clarification Ensembling [[arXiv2023]](<https://arxiv.org/abs/2311.08718>)
657 | 
658 | ### Others
659 | 
660 | **Conference**
661 | 
662 | - PaSCo: Urban 3D Panoptic Scene Completion with Uncertainty Awareness [[CVPR2024]](<https://arxiv.org/pdf/2312.02158.pdf>) - [[Website]](<https://astra-vision.github.io/PaSCo/>)
663 | - Uncertainty Quantification via Stable Distribution Propagation [[ICLR2024]](<https://arxiv.org/abs/2402.08324>)
664 | - Assessing Uncertainty in Similarity Scoring: Performance & Fairness in Face Recognition [[ICLR2024]](<https://arxiv.org/abs/2211.07245>)
665 | 
666 | **Arxiv**
667 | 
668 | - Shaving Weights with Occam's Razor: Bayesian Sparsification for Neural Networks Using the Marginal Likelihood - [[arxiv2024]](<https://arxiv.org/pdf/2402.15978>)
669 | - Urban 3D Panoptic Scene Completion with Uncertainty Awareness [[arXiv2023]](<https://astra-vision.github.io/PaSCo/>) - [[PyTorch]](<https://github.com/astra-vision/PaSCo>)
670 | 
671 | # Datasets and Benchmarks
672 | 
673 | - SHIFT: A Synthetic Driving Dataset for Continuous Multi-Task Domain Adaptation [[CVPR2022]](<https://openaccess.thecvf.com/content/CVPR2022/html/Sun_SHIFT_A_Synthetic_Driving_Dataset_for_Continuous_Multi-Task_Domain_Adaptation_CVPR_2022_paper.html>)
674 | - MUAD: Multiple Uncertainties for Autonomous Driving, a benchmark for multiple uncertainty types and tasks [[BMVC2022]](<https://arxiv.org/abs/2203.01437>) - [[PyTorch]](<https://github.com/ENSTA-U2IS-AI/MUAD-Dataset>)
675 | - ACDC: The Adverse Conditions Dataset with Correspondences for Semantic Driving Scene Understanding [[ICCV2021]](<https://arxiv.org/abs/2104.13395>)
676 | - The MVTec Anomaly Detection Dataset: A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection [[IJCV2021]](<https://link.springer.com/content/pdf/10.1007/s11263-020-01400-4.pdf>)
677 | - SegmentMeIfYouCan: A Benchmark for Anomaly Segmentation [[NeurIPS2021]](<https://arxiv.org/abs/2104.14812>)
678 | - Uncertainty Baselines: Benchmarks for Uncertainty & Robustness in Deep Learning [[arXiv2021]](<https://arxiv.org/abs/2106.04015>) - [[TensorFlow]](<https://github.com/google/uncertainty-baselines>)
679 | - Curriculum Model Adaptation with Synthetic and Real Data for Semantic Foggy Scene Understanding [[IJCV2020]](<https://people.ee.ethz.ch/~csakarid/Model_adaptation_SFSU_dense/>)
680 | - Benchmarking the Robustness of Semantic Segmentation Models [[CVPR2020]](<https://arxiv.org/abs/1908.05005>)
681 | - Fishyscapes: A Benchmark for Safe Semantic Segmentation in Autonomous Driving [[ICCV Workshop2019]](<https://openaccess.thecvf.com/content_ICCVW_2019/html/ADW/Blum_Fishyscapes_A_Benchmark_for_Safe_Semantic_Segmentation_in_Autonomous_Driving_ICCVW_2019_paper.html>)
682 | - Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming [[NeurIPS Workshop2019]](<https://arxiv.org/abs/1907.07484>) - [[GitHub]](<https://github.com/bethgelab/robust-detection-benchmark>)
683 | - Semantic Foggy Scene Understanding with Synthetic Data [[IJCV2018]](<https://people.ee.ethz.ch/~csakarid/SFSU_synthetic/>)
684 | - Lost and Found: Detecting Small Road Hazards for Self-Driving Vehicles [[IROS2016]](<https://arxiv.org/abs/1609.04653>)
685 | 
686 | # Libraries
687 | 
688 | ## Python
689 | 
690 | - Uncertainty Calibration Library [[GitHub]](<https://github.com/p-lambda/verified_calibration>)
691 | - MAPIE: Model Agnostic Prediction Interval Estimator [[Sklearn]](https://github.com/scikit-learn-contrib/MAPIE)
692 | - Uncertainty Toolbox [[GitHub]](<https://uncertainty-toolbox.github.io/>)
693 | - OpenOOD: Benchmarking Generalized OOD Detection [[GitHub]](<https://github.com/jingkang50/openood>)
694 | - Darts: Forecasting and anomaly detection on time series [[GitHub]](<https://github.com/unit8co/darts>)
695 | - Mixture Density Networks (MDN) for distribution and uncertainty estimation [[GitHub]](<https://github.com/axelbrando/Mixture-Density-Networks-for-distribution-and-uncertainty-estimation>)
696 | 
697 | ## PyTorch
698 | 
699 | - TorchUncertainty [[GitHub]](<https://github.com/ENSTA-U2IS-AI/torch-uncertainty>)
700 | - Bayesian Torch [[GitHub]](<https://github.com/IntelLabs/bayesian-torch>)
701 | - Blitz: A Bayesian Neural Network library for PyTorch [[GitHub]](<https://github.com/piEsposito/blitz-bayesian-deep-learning>)
702 | 
703 | ## JAX
704 | 
705 | - Fortuna [[GitHub - JAX]](<https://github.com/awslabs/fortuna>)
706 | 
707 | ## TensorFlow
708 | 
709 | - TensorFlow Probability [[Website]](<https://www.tensorflow.org/probability>)
710 | 
711 | # Lectures and tutorials
712 | 
713 | - Dan Hendrycks: Intro to ML Safety course [[Website]](<https://course.mlsafety.org/>)
714 | - Uncertainty and Robustness in Deep Learning Workshop in ICML (2020, 2021) [[SlidesLive]](<https://slideslive.com/icml-2020/icml-workshop-on-uncertainty-and-robustness-in-deep-learning-udl>)
715 | - Yarin Gal: Bayesian Deep Learning 101 [[Website]](<http://www.cs.ox.ac.uk/people/yarin.gal/website/bdl101/>)
716 | - MIT 6.S191: Evidential Deep Learning and Uncertainty (2021) [[Youtube]](<https://www.youtube.com/watch?v=toTcf7tZK8c>)
717 | - Hands-on Bayesian Neural Networks - a Tutorial for Deep Learning Users [[IEEE Computational Intelligence Magazine]](https://arxiv.org/pdf/2007.06823.pdf)
718 | 
719 | # Books
720 | 
721 | - The "Probabilistic Machine-Learning" book series by Kevin Murphy [[Book]](<https://probml.github.io/pml-book/>)
722 | 
723 | # Other Resources
724 | 
725 | Uncertainty Quantification in Deep Learning [[GitHub]](<https://github.com/ahmedmalaa/deep-learning-uncertainty>)
726 | 
727 | Awesome Out-of-distribution Detection [[GitHub]](<https://github.com/continuousml/Awesome-Out-Of-Distribution-Detection>)
728 | 
729 | Anomaly Detection Learning Resources [[GitHub]](<https://github.com/yzhao062/anomaly-detection-resources>)
730 | 
731 | Awesome Conformal Prediction [[GitHub]](<https://github.com/valeman/awesome-conformal-prediction>)
732 | 
733 | Awesome LLM Uncertainty, Reliability, & Robustness [[GitHub]](<https://github.com/jxzhangjhu/Awesome-LLM-Uncertainty-Reliability-Robustness>)
734 | 
735 | UQSay - Seminars on Uncertainty Quantification (UQ), Design and Analysis of Computer Experiments (DACE) and related topics @ Paris Saclay [[Website]](<https://www.uqsay.org/p/welcome.html/>)
736 | 
737 | ProbAI summer school [[Website]](<https://probabilistic.ai/>)
738 | 
739 | Gaussian process summer school [[Website]](<https://gpss.cc/>)
740 | 


--------------------------------------------------------------------------------