├── README.md ├── cfgs ├── dataset │ └── KS.yaml ├── encoder_a │ └── resnet18_a.yaml ├── encoder_v │ └── resnet18_v.yaml ├── methods │ ├── CRMT_AT.yaml │ ├── CRMT_JT.yaml │ └── CRMT_mix.yaml ├── optimizer │ └── SGD.yaml ├── scheduler │ └── step_decay.yaml └── train.yaml ├── data ├── ks_test_overlap.txt └── ks_train_overlap.txt ├── datasets ├── KS_dataset.py └── __init__.py ├── main.py ├── models ├── __init__.py ├── backbone.py ├── fusion_model.py └── multimodal.py ├── pics ├── framework.jpg ├── rob.jpg └── step1.png ├── pre-processing ├── README.md ├── extract_audio.py ├── extract_rgb_of.py └── extract_video_frames.py ├── requirements.txt ├── rob_test.py ├── train.py └── utils.py /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeWu-Lab/Certifiable-Robust-Multi-modal-Training/HEAD/README.md -------------------------------------------------------------------------------- /cfgs/dataset/KS.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeWu-Lab/Certifiable-Robust-Multi-modal-Training/HEAD/cfgs/dataset/KS.yaml -------------------------------------------------------------------------------- /cfgs/encoder_a/resnet18_a.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeWu-Lab/Certifiable-Robust-Multi-modal-Training/HEAD/cfgs/encoder_a/resnet18_a.yaml -------------------------------------------------------------------------------- /cfgs/encoder_v/resnet18_v.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeWu-Lab/Certifiable-Robust-Multi-modal-Training/HEAD/cfgs/encoder_v/resnet18_v.yaml -------------------------------------------------------------------------------- /cfgs/methods/CRMT_AT.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeWu-Lab/Certifiable-Robust-Multi-modal-Training/HEAD/cfgs/methods/CRMT_AT.yaml -------------------------------------------------------------------------------- /cfgs/methods/CRMT_JT.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeWu-Lab/Certifiable-Robust-Multi-modal-Training/HEAD/cfgs/methods/CRMT_JT.yaml -------------------------------------------------------------------------------- /cfgs/methods/CRMT_mix.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeWu-Lab/Certifiable-Robust-Multi-modal-Training/HEAD/cfgs/methods/CRMT_mix.yaml -------------------------------------------------------------------------------- /cfgs/optimizer/SGD.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeWu-Lab/Certifiable-Robust-Multi-modal-Training/HEAD/cfgs/optimizer/SGD.yaml -------------------------------------------------------------------------------- /cfgs/scheduler/step_decay.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeWu-Lab/Certifiable-Robust-Multi-modal-Training/HEAD/cfgs/scheduler/step_decay.yaml -------------------------------------------------------------------------------- /cfgs/train.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeWu-Lab/Certifiable-Robust-Multi-modal-Training/HEAD/cfgs/train.yaml -------------------------------------------------------------------------------- /data/ks_test_overlap.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeWu-Lab/Certifiable-Robust-Multi-modal-Training/HEAD/data/ks_test_overlap.txt -------------------------------------------------------------------------------- /data/ks_train_overlap.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeWu-Lab/Certifiable-Robust-Multi-modal-Training/HEAD/data/ks_train_overlap.txt -------------------------------------------------------------------------------- /datasets/KS_dataset.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeWu-Lab/Certifiable-Robust-Multi-modal-Training/HEAD/datasets/KS_dataset.py -------------------------------------------------------------------------------- /datasets/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /main.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeWu-Lab/Certifiable-Robust-Multi-modal-Training/HEAD/main.py -------------------------------------------------------------------------------- /models/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /models/backbone.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeWu-Lab/Certifiable-Robust-Multi-modal-Training/HEAD/models/backbone.py -------------------------------------------------------------------------------- /models/fusion_model.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeWu-Lab/Certifiable-Robust-Multi-modal-Training/HEAD/models/fusion_model.py -------------------------------------------------------------------------------- /models/multimodal.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeWu-Lab/Certifiable-Robust-Multi-modal-Training/HEAD/models/multimodal.py -------------------------------------------------------------------------------- /pics/framework.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeWu-Lab/Certifiable-Robust-Multi-modal-Training/HEAD/pics/framework.jpg -------------------------------------------------------------------------------- /pics/rob.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeWu-Lab/Certifiable-Robust-Multi-modal-Training/HEAD/pics/rob.jpg -------------------------------------------------------------------------------- /pics/step1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeWu-Lab/Certifiable-Robust-Multi-modal-Training/HEAD/pics/step1.png -------------------------------------------------------------------------------- /pre-processing/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeWu-Lab/Certifiable-Robust-Multi-modal-Training/HEAD/pre-processing/README.md -------------------------------------------------------------------------------- /pre-processing/extract_audio.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeWu-Lab/Certifiable-Robust-Multi-modal-Training/HEAD/pre-processing/extract_audio.py -------------------------------------------------------------------------------- /pre-processing/extract_rgb_of.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeWu-Lab/Certifiable-Robust-Multi-modal-Training/HEAD/pre-processing/extract_rgb_of.py -------------------------------------------------------------------------------- /pre-processing/extract_video_frames.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeWu-Lab/Certifiable-Robust-Multi-modal-Training/HEAD/pre-processing/extract_video_frames.py -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeWu-Lab/Certifiable-Robust-Multi-modal-Training/HEAD/requirements.txt -------------------------------------------------------------------------------- /rob_test.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeWu-Lab/Certifiable-Robust-Multi-modal-Training/HEAD/rob_test.py -------------------------------------------------------------------------------- /train.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeWu-Lab/Certifiable-Robust-Multi-modal-Training/HEAD/train.py -------------------------------------------------------------------------------- /utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeWu-Lab/Certifiable-Robust-Multi-modal-Training/HEAD/utils.py --------------------------------------------------------------------------------