├── .gitattributes ├── .gitignore ├── FROMAGe_example_notebook.ipynb ├── LICENSE ├── README.md ├── datasets ├── cc3m_train.tsv └── cc3m_val.tsv ├── demo ├── app.py └── share_btn.py ├── evals ├── VIST_Contextual_Image_Retrieval.ipynb ├── VisDial_Inference_IT2T_Generation.ipynb ├── VisDial_Inference_T2I_Retrieval.ipynb ├── eval_visdial_generation.py ├── eval_visdial_retrieval.py └── eval_vist_retrieval.py ├── fromage ├── __init__.py ├── data.py ├── evaluate.py ├── extract_img_embs.py ├── losses.py ├── models.py ├── prune_model_ckpt.py └── utils.py ├── fromage_model ├── fromage_vis4 │ ├── model_args.json │ └── pretrained_ckpt.pth.tar ├── model_args.json └── pretrained_ckpt.pth.tar ├── main.py ├── requirements.txt ├── teaser.png ├── teaser_gif.gif └── test_main.py /.gitattributes: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kohjingyu/fromage/HEAD/.gitignore -------------------------------------------------------------------------------- /FROMAGe_example_notebook.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kohjingyu/fromage/HEAD/FROMAGe_example_notebook.ipynb -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kohjingyu/fromage/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kohjingyu/fromage/HEAD/README.md -------------------------------------------------------------------------------- /datasets/cc3m_train.tsv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kohjingyu/fromage/HEAD/datasets/cc3m_train.tsv -------------------------------------------------------------------------------- /datasets/cc3m_val.tsv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kohjingyu/fromage/HEAD/datasets/cc3m_val.tsv -------------------------------------------------------------------------------- /demo/app.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kohjingyu/fromage/HEAD/demo/app.py -------------------------------------------------------------------------------- /demo/share_btn.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kohjingyu/fromage/HEAD/demo/share_btn.py -------------------------------------------------------------------------------- /evals/VIST_Contextual_Image_Retrieval.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kohjingyu/fromage/HEAD/evals/VIST_Contextual_Image_Retrieval.ipynb -------------------------------------------------------------------------------- /evals/VisDial_Inference_IT2T_Generation.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kohjingyu/fromage/HEAD/evals/VisDial_Inference_IT2T_Generation.ipynb -------------------------------------------------------------------------------- /evals/VisDial_Inference_T2I_Retrieval.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kohjingyu/fromage/HEAD/evals/VisDial_Inference_T2I_Retrieval.ipynb -------------------------------------------------------------------------------- /evals/eval_visdial_generation.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kohjingyu/fromage/HEAD/evals/eval_visdial_generation.py -------------------------------------------------------------------------------- /evals/eval_visdial_retrieval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kohjingyu/fromage/HEAD/evals/eval_visdial_retrieval.py -------------------------------------------------------------------------------- /evals/eval_vist_retrieval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kohjingyu/fromage/HEAD/evals/eval_vist_retrieval.py -------------------------------------------------------------------------------- /fromage/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /fromage/data.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kohjingyu/fromage/HEAD/fromage/data.py -------------------------------------------------------------------------------- /fromage/evaluate.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kohjingyu/fromage/HEAD/fromage/evaluate.py -------------------------------------------------------------------------------- /fromage/extract_img_embs.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kohjingyu/fromage/HEAD/fromage/extract_img_embs.py -------------------------------------------------------------------------------- /fromage/losses.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kohjingyu/fromage/HEAD/fromage/losses.py -------------------------------------------------------------------------------- /fromage/models.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kohjingyu/fromage/HEAD/fromage/models.py -------------------------------------------------------------------------------- /fromage/prune_model_ckpt.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kohjingyu/fromage/HEAD/fromage/prune_model_ckpt.py -------------------------------------------------------------------------------- /fromage/utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kohjingyu/fromage/HEAD/fromage/utils.py -------------------------------------------------------------------------------- /fromage_model/fromage_vis4/model_args.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kohjingyu/fromage/HEAD/fromage_model/fromage_vis4/model_args.json -------------------------------------------------------------------------------- /fromage_model/fromage_vis4/pretrained_ckpt.pth.tar: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kohjingyu/fromage/HEAD/fromage_model/fromage_vis4/pretrained_ckpt.pth.tar -------------------------------------------------------------------------------- /fromage_model/model_args.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kohjingyu/fromage/HEAD/fromage_model/model_args.json -------------------------------------------------------------------------------- /fromage_model/pretrained_ckpt.pth.tar: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kohjingyu/fromage/HEAD/fromage_model/pretrained_ckpt.pth.tar -------------------------------------------------------------------------------- /main.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kohjingyu/fromage/HEAD/main.py -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kohjingyu/fromage/HEAD/requirements.txt -------------------------------------------------------------------------------- /teaser.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kohjingyu/fromage/HEAD/teaser.png -------------------------------------------------------------------------------- /teaser_gif.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kohjingyu/fromage/HEAD/teaser_gif.gif -------------------------------------------------------------------------------- /test_main.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kohjingyu/fromage/HEAD/test_main.py --------------------------------------------------------------------------------