├── .gitignore ├── LICENSE ├── README.md ├── code ├── dataset_generation.ipynb ├── image_guided │ ├── chart_caption.py │ ├── chart_caption_data.py │ ├── chart_caption_model.py │ ├── dist_utils.py │ ├── modeling_bart.py │ ├── modeling_t5.py │ ├── param.py │ ├── tokenization.py │ ├── trainer_base.py │ └── utils.py ├── run_metrics.py ├── text_only │ └── run_train_eval_predict.py ├── utils_gen.py └── utils_metrics.py ├── data ├── README.md ├── feature_extraction │ ├── README.md │ ├── detectron2_proposal_maxnms.py │ └── vistext_proposal.py ├── loading_script_noprefixtuning │ └── loading_script_noprefixtuning.py ├── loading_script_prefixtuning │ └── loading_script_prefixtuning.py ├── splits.json └── statista_mappings.json ├── download_data.sh ├── environment-image_guided.yml ├── environment.yml ├── metrics ├── README.md └── vistext_test_predictions_scenegraph_t5_prefixtuningtrue_seed16983_scores.txt ├── models └── README.md ├── run.sh ├── run_test.sh └── teaser.png /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mitvis/vistext/HEAD/.gitignore -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mitvis/vistext/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mitvis/vistext/HEAD/README.md -------------------------------------------------------------------------------- /code/dataset_generation.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mitvis/vistext/HEAD/code/dataset_generation.ipynb -------------------------------------------------------------------------------- /code/image_guided/chart_caption.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mitvis/vistext/HEAD/code/image_guided/chart_caption.py -------------------------------------------------------------------------------- /code/image_guided/chart_caption_data.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mitvis/vistext/HEAD/code/image_guided/chart_caption_data.py -------------------------------------------------------------------------------- /code/image_guided/chart_caption_model.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mitvis/vistext/HEAD/code/image_guided/chart_caption_model.py -------------------------------------------------------------------------------- /code/image_guided/dist_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mitvis/vistext/HEAD/code/image_guided/dist_utils.py -------------------------------------------------------------------------------- /code/image_guided/modeling_bart.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mitvis/vistext/HEAD/code/image_guided/modeling_bart.py -------------------------------------------------------------------------------- /code/image_guided/modeling_t5.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mitvis/vistext/HEAD/code/image_guided/modeling_t5.py -------------------------------------------------------------------------------- /code/image_guided/param.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mitvis/vistext/HEAD/code/image_guided/param.py -------------------------------------------------------------------------------- /code/image_guided/tokenization.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mitvis/vistext/HEAD/code/image_guided/tokenization.py -------------------------------------------------------------------------------- /code/image_guided/trainer_base.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mitvis/vistext/HEAD/code/image_guided/trainer_base.py -------------------------------------------------------------------------------- /code/image_guided/utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mitvis/vistext/HEAD/code/image_guided/utils.py -------------------------------------------------------------------------------- /code/run_metrics.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mitvis/vistext/HEAD/code/run_metrics.py -------------------------------------------------------------------------------- /code/text_only/run_train_eval_predict.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mitvis/vistext/HEAD/code/text_only/run_train_eval_predict.py -------------------------------------------------------------------------------- /code/utils_gen.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mitvis/vistext/HEAD/code/utils_gen.py -------------------------------------------------------------------------------- /code/utils_metrics.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mitvis/vistext/HEAD/code/utils_metrics.py -------------------------------------------------------------------------------- /data/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mitvis/vistext/HEAD/data/README.md -------------------------------------------------------------------------------- /data/feature_extraction/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mitvis/vistext/HEAD/data/feature_extraction/README.md -------------------------------------------------------------------------------- /data/feature_extraction/detectron2_proposal_maxnms.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mitvis/vistext/HEAD/data/feature_extraction/detectron2_proposal_maxnms.py -------------------------------------------------------------------------------- /data/feature_extraction/vistext_proposal.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mitvis/vistext/HEAD/data/feature_extraction/vistext_proposal.py -------------------------------------------------------------------------------- /data/loading_script_noprefixtuning/loading_script_noprefixtuning.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mitvis/vistext/HEAD/data/loading_script_noprefixtuning/loading_script_noprefixtuning.py -------------------------------------------------------------------------------- /data/loading_script_prefixtuning/loading_script_prefixtuning.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mitvis/vistext/HEAD/data/loading_script_prefixtuning/loading_script_prefixtuning.py -------------------------------------------------------------------------------- /data/splits.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mitvis/vistext/HEAD/data/splits.json -------------------------------------------------------------------------------- /data/statista_mappings.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mitvis/vistext/HEAD/data/statista_mappings.json -------------------------------------------------------------------------------- /download_data.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mitvis/vistext/HEAD/download_data.sh -------------------------------------------------------------------------------- /environment-image_guided.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mitvis/vistext/HEAD/environment-image_guided.yml -------------------------------------------------------------------------------- /environment.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mitvis/vistext/HEAD/environment.yml -------------------------------------------------------------------------------- /metrics/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mitvis/vistext/HEAD/metrics/README.md -------------------------------------------------------------------------------- /metrics/vistext_test_predictions_scenegraph_t5_prefixtuningtrue_seed16983_scores.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mitvis/vistext/HEAD/metrics/vistext_test_predictions_scenegraph_t5_prefixtuningtrue_seed16983_scores.txt -------------------------------------------------------------------------------- /models/README.md: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /run.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mitvis/vistext/HEAD/run.sh -------------------------------------------------------------------------------- /run_test.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mitvis/vistext/HEAD/run_test.sh -------------------------------------------------------------------------------- /teaser.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mitvis/vistext/HEAD/teaser.png --------------------------------------------------------------------------------