├── .gitignore ├── README.md ├── X-CLIP ├── LICENSE ├── README.md ├── XCLIP.png ├── dataloaders │ ├── data_dataloaders.py │ ├── dataloader_activitynet_retrieval.py │ ├── dataloader_didemo_retrieval.py │ ├── dataloader_lsmdc_retrieval.py │ ├── dataloader_msrvtt_retrieval.py │ ├── dataloader_msvd_retrieval.py │ ├── dataloader_vitatecs.py │ └── rawvideo_util.py ├── eval_vitatecs.sh ├── eval_vitatecs_clip4clip.py ├── eval_vitatecs_xclip.py ├── main_clip4clip.py ├── main_xclip.py ├── metrics.py ├── modules │ ├── __init__.py │ ├── bpe_simple_vocab_16e6.txt.gz │ ├── cross-base │ │ └── cross_config.json │ ├── file_utils.py │ ├── modeling.py │ ├── modeling_xclip.py │ ├── module_clip.py │ ├── module_cross.py │ ├── optimization.py │ ├── tokenization_clip.py │ ├── until_config.py │ └── until_module.py ├── preprocess │ └── compress_video.py ├── requirements.txt ├── scripts │ ├── run_xclip_actnet_vit16.sh │ ├── run_xclip_actnet_vit32.sh │ ├── run_xclip_didemo_vit16.sh │ ├── run_xclip_didemo_vit32.sh │ ├── run_xclip_lsmdc_vit16.sh │ ├── run_xclip_lsmdc_vit32.sh │ ├── run_xclip_msrvtt_vit16.sh │ ├── run_xclip_msrvtt_vit32.sh │ ├── run_xclip_msvd_vit16.sh │ └── run_xclip_msvd_vit32.sh └── util.py ├── alpro ├── alpro_pretrain.yaml └── eval_alpro.py ├── data ├── Compositionality.jsonl ├── Direction.jsonl ├── Intensity.jsonl ├── Localization.jsonl ├── Sequence.jsonl └── Type.jsonl └── data_opt ├── Compositionality.jsonl ├── Direction.jsonl ├── Intensity.jsonl ├── Localization.jsonl ├── Sequence.jsonl └── Type.jsonl /.gitignore: -------------------------------------------------------------------------------- 1 | videos 2 | ckpts 3 | *.pt 4 | __pycache__ -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/README.md -------------------------------------------------------------------------------- /X-CLIP/LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/LICENSE -------------------------------------------------------------------------------- /X-CLIP/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/README.md -------------------------------------------------------------------------------- /X-CLIP/XCLIP.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/XCLIP.png -------------------------------------------------------------------------------- /X-CLIP/dataloaders/data_dataloaders.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/dataloaders/data_dataloaders.py -------------------------------------------------------------------------------- /X-CLIP/dataloaders/dataloader_activitynet_retrieval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/dataloaders/dataloader_activitynet_retrieval.py -------------------------------------------------------------------------------- /X-CLIP/dataloaders/dataloader_didemo_retrieval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/dataloaders/dataloader_didemo_retrieval.py -------------------------------------------------------------------------------- /X-CLIP/dataloaders/dataloader_lsmdc_retrieval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/dataloaders/dataloader_lsmdc_retrieval.py -------------------------------------------------------------------------------- /X-CLIP/dataloaders/dataloader_msrvtt_retrieval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/dataloaders/dataloader_msrvtt_retrieval.py -------------------------------------------------------------------------------- /X-CLIP/dataloaders/dataloader_msvd_retrieval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/dataloaders/dataloader_msvd_retrieval.py -------------------------------------------------------------------------------- /X-CLIP/dataloaders/dataloader_vitatecs.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/dataloaders/dataloader_vitatecs.py -------------------------------------------------------------------------------- /X-CLIP/dataloaders/rawvideo_util.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/dataloaders/rawvideo_util.py -------------------------------------------------------------------------------- /X-CLIP/eval_vitatecs.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/eval_vitatecs.sh -------------------------------------------------------------------------------- /X-CLIP/eval_vitatecs_clip4clip.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/eval_vitatecs_clip4clip.py -------------------------------------------------------------------------------- /X-CLIP/eval_vitatecs_xclip.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/eval_vitatecs_xclip.py -------------------------------------------------------------------------------- /X-CLIP/main_clip4clip.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/main_clip4clip.py -------------------------------------------------------------------------------- /X-CLIP/main_xclip.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/main_xclip.py -------------------------------------------------------------------------------- /X-CLIP/metrics.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/metrics.py -------------------------------------------------------------------------------- /X-CLIP/modules/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /X-CLIP/modules/bpe_simple_vocab_16e6.txt.gz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/modules/bpe_simple_vocab_16e6.txt.gz -------------------------------------------------------------------------------- /X-CLIP/modules/cross-base/cross_config.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/modules/cross-base/cross_config.json -------------------------------------------------------------------------------- /X-CLIP/modules/file_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/modules/file_utils.py -------------------------------------------------------------------------------- /X-CLIP/modules/modeling.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/modules/modeling.py -------------------------------------------------------------------------------- /X-CLIP/modules/modeling_xclip.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/modules/modeling_xclip.py -------------------------------------------------------------------------------- /X-CLIP/modules/module_clip.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/modules/module_clip.py -------------------------------------------------------------------------------- /X-CLIP/modules/module_cross.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/modules/module_cross.py -------------------------------------------------------------------------------- /X-CLIP/modules/optimization.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/modules/optimization.py -------------------------------------------------------------------------------- /X-CLIP/modules/tokenization_clip.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/modules/tokenization_clip.py -------------------------------------------------------------------------------- /X-CLIP/modules/until_config.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/modules/until_config.py -------------------------------------------------------------------------------- /X-CLIP/modules/until_module.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/modules/until_module.py -------------------------------------------------------------------------------- /X-CLIP/preprocess/compress_video.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/preprocess/compress_video.py -------------------------------------------------------------------------------- /X-CLIP/requirements.txt: -------------------------------------------------------------------------------- 1 | ftfy 2 | regex 3 | tqdm 4 | opencv-python 5 | boto3 6 | requests 7 | pandas -------------------------------------------------------------------------------- /X-CLIP/scripts/run_xclip_actnet_vit16.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/scripts/run_xclip_actnet_vit16.sh -------------------------------------------------------------------------------- /X-CLIP/scripts/run_xclip_actnet_vit32.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/scripts/run_xclip_actnet_vit32.sh -------------------------------------------------------------------------------- /X-CLIP/scripts/run_xclip_didemo_vit16.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/scripts/run_xclip_didemo_vit16.sh -------------------------------------------------------------------------------- /X-CLIP/scripts/run_xclip_didemo_vit32.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/scripts/run_xclip_didemo_vit32.sh -------------------------------------------------------------------------------- /X-CLIP/scripts/run_xclip_lsmdc_vit16.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/scripts/run_xclip_lsmdc_vit16.sh -------------------------------------------------------------------------------- /X-CLIP/scripts/run_xclip_lsmdc_vit32.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/scripts/run_xclip_lsmdc_vit32.sh -------------------------------------------------------------------------------- /X-CLIP/scripts/run_xclip_msrvtt_vit16.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/scripts/run_xclip_msrvtt_vit16.sh -------------------------------------------------------------------------------- /X-CLIP/scripts/run_xclip_msrvtt_vit32.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/scripts/run_xclip_msrvtt_vit32.sh -------------------------------------------------------------------------------- /X-CLIP/scripts/run_xclip_msvd_vit16.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/scripts/run_xclip_msvd_vit16.sh -------------------------------------------------------------------------------- /X-CLIP/scripts/run_xclip_msvd_vit32.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/scripts/run_xclip_msvd_vit32.sh -------------------------------------------------------------------------------- /X-CLIP/util.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/X-CLIP/util.py -------------------------------------------------------------------------------- /alpro/alpro_pretrain.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/alpro/alpro_pretrain.yaml -------------------------------------------------------------------------------- /alpro/eval_alpro.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/alpro/eval_alpro.py -------------------------------------------------------------------------------- /data/Compositionality.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/data/Compositionality.jsonl -------------------------------------------------------------------------------- /data/Direction.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/data/Direction.jsonl -------------------------------------------------------------------------------- /data/Intensity.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/data/Intensity.jsonl -------------------------------------------------------------------------------- /data/Localization.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/data/Localization.jsonl -------------------------------------------------------------------------------- /data/Sequence.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/data/Sequence.jsonl -------------------------------------------------------------------------------- /data/Type.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/data/Type.jsonl -------------------------------------------------------------------------------- /data_opt/Compositionality.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/data_opt/Compositionality.jsonl -------------------------------------------------------------------------------- /data_opt/Direction.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/data_opt/Direction.jsonl -------------------------------------------------------------------------------- /data_opt/Intensity.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/data_opt/Intensity.jsonl -------------------------------------------------------------------------------- /data_opt/Localization.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/data_opt/Localization.jsonl -------------------------------------------------------------------------------- /data_opt/Sequence.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/data_opt/Sequence.jsonl -------------------------------------------------------------------------------- /data_opt/Type.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lscpku/VITATECS/HEAD/data_opt/Type.jsonl --------------------------------------------------------------------------------