├── README.md ├── calibrators.py ├── cifar10h ├── densenet-bc-L190-k40.csv ├── human_model_truth_cifar10h.csv ├── preresnet-110.csv └── resnet-110.csv ├── combination_methods.py ├── data_utils.py ├── ensemble_ts.py ├── experiments ├── ablation_experiment_v2.py ├── calibrate_combo_experiment.py ├── calibrate_first_experiment.py ├── calibration_experiment.py ├── calibration_method_experiment.py ├── em_experiment.py ├── semisup_em_experiment.py └── weighted_semisup_em_experiment.py ├── imax_calib ├── __init__.py ├── calibration.py ├── calibrators │ ├── __init__.py │ ├── binners.py │ └── scalers_np.py ├── clustering.py ├── evaluations │ ├── __init__.py │ └── calibration_metrics.py ├── hb_utils.py ├── io.py └── utils.py ├── metrics.py └── utils.py /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GavinKerrigan/conf_matrix_and_calibration/HEAD/README.md -------------------------------------------------------------------------------- /calibrators.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GavinKerrigan/conf_matrix_and_calibration/HEAD/calibrators.py -------------------------------------------------------------------------------- /cifar10h/densenet-bc-L190-k40.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GavinKerrigan/conf_matrix_and_calibration/HEAD/cifar10h/densenet-bc-L190-k40.csv -------------------------------------------------------------------------------- /cifar10h/human_model_truth_cifar10h.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GavinKerrigan/conf_matrix_and_calibration/HEAD/cifar10h/human_model_truth_cifar10h.csv -------------------------------------------------------------------------------- /cifar10h/preresnet-110.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GavinKerrigan/conf_matrix_and_calibration/HEAD/cifar10h/preresnet-110.csv -------------------------------------------------------------------------------- /cifar10h/resnet-110.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GavinKerrigan/conf_matrix_and_calibration/HEAD/cifar10h/resnet-110.csv -------------------------------------------------------------------------------- /combination_methods.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GavinKerrigan/conf_matrix_and_calibration/HEAD/combination_methods.py -------------------------------------------------------------------------------- /data_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GavinKerrigan/conf_matrix_and_calibration/HEAD/data_utils.py -------------------------------------------------------------------------------- /ensemble_ts.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GavinKerrigan/conf_matrix_and_calibration/HEAD/ensemble_ts.py -------------------------------------------------------------------------------- /experiments/ablation_experiment_v2.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GavinKerrigan/conf_matrix_and_calibration/HEAD/experiments/ablation_experiment_v2.py -------------------------------------------------------------------------------- /experiments/calibrate_combo_experiment.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GavinKerrigan/conf_matrix_and_calibration/HEAD/experiments/calibrate_combo_experiment.py -------------------------------------------------------------------------------- /experiments/calibrate_first_experiment.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GavinKerrigan/conf_matrix_and_calibration/HEAD/experiments/calibrate_first_experiment.py -------------------------------------------------------------------------------- /experiments/calibration_experiment.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GavinKerrigan/conf_matrix_and_calibration/HEAD/experiments/calibration_experiment.py -------------------------------------------------------------------------------- /experiments/calibration_method_experiment.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GavinKerrigan/conf_matrix_and_calibration/HEAD/experiments/calibration_method_experiment.py -------------------------------------------------------------------------------- /experiments/em_experiment.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GavinKerrigan/conf_matrix_and_calibration/HEAD/experiments/em_experiment.py -------------------------------------------------------------------------------- /experiments/semisup_em_experiment.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GavinKerrigan/conf_matrix_and_calibration/HEAD/experiments/semisup_em_experiment.py -------------------------------------------------------------------------------- /experiments/weighted_semisup_em_experiment.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GavinKerrigan/conf_matrix_and_calibration/HEAD/experiments/weighted_semisup_em_experiment.py -------------------------------------------------------------------------------- /imax_calib/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GavinKerrigan/conf_matrix_and_calibration/HEAD/imax_calib/__init__.py -------------------------------------------------------------------------------- /imax_calib/calibration.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GavinKerrigan/conf_matrix_and_calibration/HEAD/imax_calib/calibration.py -------------------------------------------------------------------------------- /imax_calib/calibrators/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /imax_calib/calibrators/binners.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GavinKerrigan/conf_matrix_and_calibration/HEAD/imax_calib/calibrators/binners.py -------------------------------------------------------------------------------- /imax_calib/calibrators/scalers_np.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GavinKerrigan/conf_matrix_and_calibration/HEAD/imax_calib/calibrators/scalers_np.py -------------------------------------------------------------------------------- /imax_calib/clustering.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GavinKerrigan/conf_matrix_and_calibration/HEAD/imax_calib/clustering.py -------------------------------------------------------------------------------- /imax_calib/evaluations/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /imax_calib/evaluations/calibration_metrics.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GavinKerrigan/conf_matrix_and_calibration/HEAD/imax_calib/evaluations/calibration_metrics.py -------------------------------------------------------------------------------- /imax_calib/hb_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GavinKerrigan/conf_matrix_and_calibration/HEAD/imax_calib/hb_utils.py -------------------------------------------------------------------------------- /imax_calib/io.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GavinKerrigan/conf_matrix_and_calibration/HEAD/imax_calib/io.py -------------------------------------------------------------------------------- /imax_calib/utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GavinKerrigan/conf_matrix_and_calibration/HEAD/imax_calib/utils.py -------------------------------------------------------------------------------- /metrics.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GavinKerrigan/conf_matrix_and_calibration/HEAD/metrics.py -------------------------------------------------------------------------------- /utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GavinKerrigan/conf_matrix_and_calibration/HEAD/utils.py --------------------------------------------------------------------------------