├── CLIP4IDC.png ├── LICENSE ├── README.md ├── dataloaders ├── data_dataloaders.py ├── dataloader_clevr_caption.py ├── dataloader_clevr_retrieval.py ├── dataloader_spot_caption.py ├── dataloader_spot_retrieval.py └── rawimage_util.py ├── gt ├── clevr_total_change_captions_reformat.json └── spot_total_change_captions_reformat.json ├── main_task_caption.py ├── main_task_retrieval.py ├── metrics.py ├── modules ├── __init__.py ├── beam.py ├── bpe_simple_vocab_16e6.txt.gz ├── cross-base │ └── cross_config.json ├── decoder-base │ └── decoder_config.json ├── file_utils.py ├── modeling.py ├── module_clip.py ├── module_cross.py ├── module_decoder.py ├── optimization.py ├── tokenization_clip.py ├── until_config.py └── until_module.py ├── preprocess ├── eval_utils.py └── reformat_dataset.py ├── scripts ├── caption_clevr.sh ├── caption_clevr_multigpu.sh ├── caption_spot.sh ├── eval_caption_clevr.sh ├── retrieve_clevr.sh └── retrieve_spot.sh ├── util.py └── visualize.py /CLIP4IDC.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sushizixin/CLIP4IDC/HEAD/CLIP4IDC.png -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sushizixin/CLIP4IDC/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sushizixin/CLIP4IDC/HEAD/README.md -------------------------------------------------------------------------------- /dataloaders/data_dataloaders.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sushizixin/CLIP4IDC/HEAD/dataloaders/data_dataloaders.py -------------------------------------------------------------------------------- /dataloaders/dataloader_clevr_caption.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sushizixin/CLIP4IDC/HEAD/dataloaders/dataloader_clevr_caption.py -------------------------------------------------------------------------------- /dataloaders/dataloader_clevr_retrieval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sushizixin/CLIP4IDC/HEAD/dataloaders/dataloader_clevr_retrieval.py -------------------------------------------------------------------------------- /dataloaders/dataloader_spot_caption.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sushizixin/CLIP4IDC/HEAD/dataloaders/dataloader_spot_caption.py -------------------------------------------------------------------------------- /dataloaders/dataloader_spot_retrieval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sushizixin/CLIP4IDC/HEAD/dataloaders/dataloader_spot_retrieval.py -------------------------------------------------------------------------------- /dataloaders/rawimage_util.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sushizixin/CLIP4IDC/HEAD/dataloaders/rawimage_util.py -------------------------------------------------------------------------------- /gt/clevr_total_change_captions_reformat.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sushizixin/CLIP4IDC/HEAD/gt/clevr_total_change_captions_reformat.json -------------------------------------------------------------------------------- /gt/spot_total_change_captions_reformat.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sushizixin/CLIP4IDC/HEAD/gt/spot_total_change_captions_reformat.json -------------------------------------------------------------------------------- /main_task_caption.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sushizixin/CLIP4IDC/HEAD/main_task_caption.py -------------------------------------------------------------------------------- /main_task_retrieval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sushizixin/CLIP4IDC/HEAD/main_task_retrieval.py -------------------------------------------------------------------------------- /metrics.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sushizixin/CLIP4IDC/HEAD/metrics.py -------------------------------------------------------------------------------- /modules/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /modules/beam.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sushizixin/CLIP4IDC/HEAD/modules/beam.py -------------------------------------------------------------------------------- /modules/bpe_simple_vocab_16e6.txt.gz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sushizixin/CLIP4IDC/HEAD/modules/bpe_simple_vocab_16e6.txt.gz -------------------------------------------------------------------------------- /modules/cross-base/cross_config.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sushizixin/CLIP4IDC/HEAD/modules/cross-base/cross_config.json -------------------------------------------------------------------------------- /modules/decoder-base/decoder_config.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sushizixin/CLIP4IDC/HEAD/modules/decoder-base/decoder_config.json -------------------------------------------------------------------------------- /modules/file_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sushizixin/CLIP4IDC/HEAD/modules/file_utils.py -------------------------------------------------------------------------------- /modules/modeling.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sushizixin/CLIP4IDC/HEAD/modules/modeling.py -------------------------------------------------------------------------------- /modules/module_clip.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sushizixin/CLIP4IDC/HEAD/modules/module_clip.py -------------------------------------------------------------------------------- /modules/module_cross.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sushizixin/CLIP4IDC/HEAD/modules/module_cross.py -------------------------------------------------------------------------------- /modules/module_decoder.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sushizixin/CLIP4IDC/HEAD/modules/module_decoder.py -------------------------------------------------------------------------------- /modules/optimization.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sushizixin/CLIP4IDC/HEAD/modules/optimization.py -------------------------------------------------------------------------------- /modules/tokenization_clip.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sushizixin/CLIP4IDC/HEAD/modules/tokenization_clip.py -------------------------------------------------------------------------------- /modules/until_config.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sushizixin/CLIP4IDC/HEAD/modules/until_config.py -------------------------------------------------------------------------------- /modules/until_module.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sushizixin/CLIP4IDC/HEAD/modules/until_module.py -------------------------------------------------------------------------------- /preprocess/eval_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sushizixin/CLIP4IDC/HEAD/preprocess/eval_utils.py -------------------------------------------------------------------------------- /preprocess/reformat_dataset.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sushizixin/CLIP4IDC/HEAD/preprocess/reformat_dataset.py -------------------------------------------------------------------------------- /scripts/caption_clevr.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sushizixin/CLIP4IDC/HEAD/scripts/caption_clevr.sh -------------------------------------------------------------------------------- /scripts/caption_clevr_multigpu.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sushizixin/CLIP4IDC/HEAD/scripts/caption_clevr_multigpu.sh -------------------------------------------------------------------------------- /scripts/caption_spot.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sushizixin/CLIP4IDC/HEAD/scripts/caption_spot.sh -------------------------------------------------------------------------------- /scripts/eval_caption_clevr.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sushizixin/CLIP4IDC/HEAD/scripts/eval_caption_clevr.sh -------------------------------------------------------------------------------- /scripts/retrieve_clevr.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sushizixin/CLIP4IDC/HEAD/scripts/retrieve_clevr.sh -------------------------------------------------------------------------------- /scripts/retrieve_spot.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sushizixin/CLIP4IDC/HEAD/scripts/retrieve_spot.sh -------------------------------------------------------------------------------- /util.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sushizixin/CLIP4IDC/HEAD/util.py -------------------------------------------------------------------------------- /visualize.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sushizixin/CLIP4IDC/HEAD/visualize.py --------------------------------------------------------------------------------