├── LICENSE ├── README.md ├── data_preprocessor ├── GQA.py ├── OKVQA.py ├── TDIUC.py ├── VAW+visual_attribute.py ├── VG_splits.json ├── VQAv2+open-domain_VQA.py ├── VQAv2+question_image_match.py ├── __init__.py ├── coco_splits.json ├── conceptual_splits.json ├── hateful_memes.py ├── label_mapping.csv ├── medic.py ├── mocheg.py ├── mscoco2014+ITM.py ├── mscoco2014+caption.py ├── mscoco2014+missing_object_selection.py ├── mscoco2014+object_grounding.py ├── mscoco2014+object_match.py ├── mscoco2014+object_region_selection.py ├── mscoco2014+region_generation.py ├── mscoco2014+region_object_match.py ├── mscoco2014+region_object_selection.py ├── mscoco_text.py ├── nlvr.py ├── refcoco+descriptive_object_identification.py ├── snli_ve.py ├── text_vqa.py ├── vcr.py ├── visdial.py ├── visual7W.py ├── visual7w_VQA.py ├── visual_spatial_reasoning.py ├── visualgenome+object_relationship.py ├── visualgenome+open-domain_VQA.py ├── visualgenome+region_caption.py ├── vizwiz+image_quality.py ├── wikihow.py └── wikihow_split.json ├── download_data.sh ├── download_scripts ├── README.md ├── TDIUC.sh ├── coco_text.sh ├── gqa.sh ├── medic.sh ├── mocheg.sh ├── mscoco2014.sh ├── nlvr.sh ├── ok_vqa.sh ├── refcoco+.sh ├── refcoco.sh ├── refcocog.sh ├── snli_ve.sh ├── text_vqa.sh ├── vaw.sh ├── visdial.sh ├── visual7w.sh ├── visual_spatial_reasoning.sh ├── vizwiz_image_quality.sh ├── vqa_v2.sh └── wikihow.sh ├── instruction_templates.py ├── multi_instruct_tasks_fig.png ├── process_data.sh ├── test.jsonl ├── train.jsonl └── util.py /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/README.md -------------------------------------------------------------------------------- /data_preprocessor/GQA.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/data_preprocessor/GQA.py -------------------------------------------------------------------------------- /data_preprocessor/OKVQA.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/data_preprocessor/OKVQA.py -------------------------------------------------------------------------------- /data_preprocessor/TDIUC.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/data_preprocessor/TDIUC.py -------------------------------------------------------------------------------- /data_preprocessor/VAW+visual_attribute.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/data_preprocessor/VAW+visual_attribute.py -------------------------------------------------------------------------------- /data_preprocessor/VG_splits.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/data_preprocessor/VG_splits.json -------------------------------------------------------------------------------- /data_preprocessor/VQAv2+open-domain_VQA.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/data_preprocessor/VQAv2+open-domain_VQA.py -------------------------------------------------------------------------------- /data_preprocessor/VQAv2+question_image_match.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/data_preprocessor/VQAv2+question_image_match.py -------------------------------------------------------------------------------- /data_preprocessor/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /data_preprocessor/coco_splits.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/data_preprocessor/coco_splits.json -------------------------------------------------------------------------------- /data_preprocessor/conceptual_splits.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/data_preprocessor/conceptual_splits.json -------------------------------------------------------------------------------- /data_preprocessor/hateful_memes.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/data_preprocessor/hateful_memes.py -------------------------------------------------------------------------------- /data_preprocessor/label_mapping.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/data_preprocessor/label_mapping.csv -------------------------------------------------------------------------------- /data_preprocessor/medic.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/data_preprocessor/medic.py -------------------------------------------------------------------------------- /data_preprocessor/mocheg.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/data_preprocessor/mocheg.py -------------------------------------------------------------------------------- /data_preprocessor/mscoco2014+ITM.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/data_preprocessor/mscoco2014+ITM.py -------------------------------------------------------------------------------- /data_preprocessor/mscoco2014+caption.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/data_preprocessor/mscoco2014+caption.py -------------------------------------------------------------------------------- /data_preprocessor/mscoco2014+missing_object_selection.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/data_preprocessor/mscoco2014+missing_object_selection.py -------------------------------------------------------------------------------- /data_preprocessor/mscoco2014+object_grounding.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/data_preprocessor/mscoco2014+object_grounding.py -------------------------------------------------------------------------------- /data_preprocessor/mscoco2014+object_match.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/data_preprocessor/mscoco2014+object_match.py -------------------------------------------------------------------------------- /data_preprocessor/mscoco2014+object_region_selection.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/data_preprocessor/mscoco2014+object_region_selection.py -------------------------------------------------------------------------------- /data_preprocessor/mscoco2014+region_generation.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/data_preprocessor/mscoco2014+region_generation.py -------------------------------------------------------------------------------- /data_preprocessor/mscoco2014+region_object_match.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/data_preprocessor/mscoco2014+region_object_match.py -------------------------------------------------------------------------------- /data_preprocessor/mscoco2014+region_object_selection.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/data_preprocessor/mscoco2014+region_object_selection.py -------------------------------------------------------------------------------- /data_preprocessor/mscoco_text.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/data_preprocessor/mscoco_text.py -------------------------------------------------------------------------------- /data_preprocessor/nlvr.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/data_preprocessor/nlvr.py -------------------------------------------------------------------------------- /data_preprocessor/refcoco+descriptive_object_identification.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/data_preprocessor/refcoco+descriptive_object_identification.py -------------------------------------------------------------------------------- /data_preprocessor/snli_ve.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/data_preprocessor/snli_ve.py -------------------------------------------------------------------------------- /data_preprocessor/text_vqa.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/data_preprocessor/text_vqa.py -------------------------------------------------------------------------------- /data_preprocessor/vcr.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/data_preprocessor/vcr.py -------------------------------------------------------------------------------- /data_preprocessor/visdial.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/data_preprocessor/visdial.py -------------------------------------------------------------------------------- /data_preprocessor/visual7W.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/data_preprocessor/visual7W.py -------------------------------------------------------------------------------- /data_preprocessor/visual7w_VQA.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/data_preprocessor/visual7w_VQA.py -------------------------------------------------------------------------------- /data_preprocessor/visual_spatial_reasoning.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/data_preprocessor/visual_spatial_reasoning.py -------------------------------------------------------------------------------- /data_preprocessor/visualgenome+object_relationship.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/data_preprocessor/visualgenome+object_relationship.py -------------------------------------------------------------------------------- /data_preprocessor/visualgenome+open-domain_VQA.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/data_preprocessor/visualgenome+open-domain_VQA.py -------------------------------------------------------------------------------- /data_preprocessor/visualgenome+region_caption.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/data_preprocessor/visualgenome+region_caption.py -------------------------------------------------------------------------------- /data_preprocessor/vizwiz+image_quality.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/data_preprocessor/vizwiz+image_quality.py -------------------------------------------------------------------------------- /data_preprocessor/wikihow.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/data_preprocessor/wikihow.py -------------------------------------------------------------------------------- /data_preprocessor/wikihow_split.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/data_preprocessor/wikihow_split.json -------------------------------------------------------------------------------- /download_data.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/download_data.sh -------------------------------------------------------------------------------- /download_scripts/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/download_scripts/README.md -------------------------------------------------------------------------------- /download_scripts/TDIUC.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/download_scripts/TDIUC.sh -------------------------------------------------------------------------------- /download_scripts/coco_text.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/download_scripts/coco_text.sh -------------------------------------------------------------------------------- /download_scripts/gqa.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/download_scripts/gqa.sh -------------------------------------------------------------------------------- /download_scripts/medic.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/download_scripts/medic.sh -------------------------------------------------------------------------------- /download_scripts/mocheg.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/download_scripts/mocheg.sh -------------------------------------------------------------------------------- /download_scripts/mscoco2014.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/download_scripts/mscoco2014.sh -------------------------------------------------------------------------------- /download_scripts/nlvr.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/download_scripts/nlvr.sh -------------------------------------------------------------------------------- /download_scripts/ok_vqa.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/download_scripts/ok_vqa.sh -------------------------------------------------------------------------------- /download_scripts/refcoco+.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/download_scripts/refcoco+.sh -------------------------------------------------------------------------------- /download_scripts/refcoco.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/download_scripts/refcoco.sh -------------------------------------------------------------------------------- /download_scripts/refcocog.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/download_scripts/refcocog.sh -------------------------------------------------------------------------------- /download_scripts/snli_ve.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/download_scripts/snli_ve.sh -------------------------------------------------------------------------------- /download_scripts/text_vqa.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/download_scripts/text_vqa.sh -------------------------------------------------------------------------------- /download_scripts/vaw.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/download_scripts/vaw.sh -------------------------------------------------------------------------------- /download_scripts/visdial.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/download_scripts/visdial.sh -------------------------------------------------------------------------------- /download_scripts/visual7w.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/download_scripts/visual7w.sh -------------------------------------------------------------------------------- /download_scripts/visual_spatial_reasoning.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/download_scripts/visual_spatial_reasoning.sh -------------------------------------------------------------------------------- /download_scripts/vizwiz_image_quality.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/download_scripts/vizwiz_image_quality.sh -------------------------------------------------------------------------------- /download_scripts/vqa_v2.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/download_scripts/vqa_v2.sh -------------------------------------------------------------------------------- /download_scripts/wikihow.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/download_scripts/wikihow.sh -------------------------------------------------------------------------------- /instruction_templates.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/instruction_templates.py -------------------------------------------------------------------------------- /multi_instruct_tasks_fig.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/multi_instruct_tasks_fig.png -------------------------------------------------------------------------------- /process_data.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/process_data.sh -------------------------------------------------------------------------------- /test.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/test.jsonl -------------------------------------------------------------------------------- /train.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/train.jsonl -------------------------------------------------------------------------------- /util.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PLUM-Lab/MultiInstruct/HEAD/util.py --------------------------------------------------------------------------------