├── .gitignore ├── LICENSE ├── README.md ├── config.py ├── eval ├── data │ ├── eval_object_vocab.json │ ├── tvqa_plus_val.json │ ├── val_sample_prediction.json │ └── word2idx.json ├── eval_sample.sh ├── eval_tvqa_plus.py ├── maskrcnn_voc │ ├── __init__.py │ ├── bounding_box.py │ ├── boxlist_ops.py │ └── voc_eval.py ├── readme.txt └── utils.py ├── imgs ├── model_overview.png ├── model_prediction.png ├── qa_example.png └── qa_example_pair.png ├── inference.py ├── main.py ├── model ├── __init__.py ├── cnn.py ├── context_query_attention.py ├── encoder.py ├── model_utils.py ├── position_encoding.py ├── self_attention.py └── stage.py ├── run_inference.sh ├── run_main.sh ├── tvqa_dataset.py └── utils.py /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayleicn/TVQAplus/HEAD/.gitignore -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayleicn/TVQAplus/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayleicn/TVQAplus/HEAD/README.md -------------------------------------------------------------------------------- /config.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayleicn/TVQAplus/HEAD/config.py -------------------------------------------------------------------------------- /eval/data/eval_object_vocab.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayleicn/TVQAplus/HEAD/eval/data/eval_object_vocab.json -------------------------------------------------------------------------------- /eval/data/tvqa_plus_val.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayleicn/TVQAplus/HEAD/eval/data/tvqa_plus_val.json -------------------------------------------------------------------------------- /eval/data/val_sample_prediction.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayleicn/TVQAplus/HEAD/eval/data/val_sample_prediction.json -------------------------------------------------------------------------------- /eval/data/word2idx.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayleicn/TVQAplus/HEAD/eval/data/word2idx.json -------------------------------------------------------------------------------- /eval/eval_sample.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayleicn/TVQAplus/HEAD/eval/eval_sample.sh -------------------------------------------------------------------------------- /eval/eval_tvqa_plus.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayleicn/TVQAplus/HEAD/eval/eval_tvqa_plus.py -------------------------------------------------------------------------------- /eval/maskrcnn_voc/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayleicn/TVQAplus/HEAD/eval/maskrcnn_voc/__init__.py -------------------------------------------------------------------------------- /eval/maskrcnn_voc/bounding_box.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayleicn/TVQAplus/HEAD/eval/maskrcnn_voc/bounding_box.py -------------------------------------------------------------------------------- /eval/maskrcnn_voc/boxlist_ops.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayleicn/TVQAplus/HEAD/eval/maskrcnn_voc/boxlist_ops.py -------------------------------------------------------------------------------- /eval/maskrcnn_voc/voc_eval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayleicn/TVQAplus/HEAD/eval/maskrcnn_voc/voc_eval.py -------------------------------------------------------------------------------- /eval/readme.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayleicn/TVQAplus/HEAD/eval/readme.txt -------------------------------------------------------------------------------- /eval/utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayleicn/TVQAplus/HEAD/eval/utils.py -------------------------------------------------------------------------------- /imgs/model_overview.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayleicn/TVQAplus/HEAD/imgs/model_overview.png -------------------------------------------------------------------------------- /imgs/model_prediction.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayleicn/TVQAplus/HEAD/imgs/model_prediction.png -------------------------------------------------------------------------------- /imgs/qa_example.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayleicn/TVQAplus/HEAD/imgs/qa_example.png -------------------------------------------------------------------------------- /imgs/qa_example_pair.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayleicn/TVQAplus/HEAD/imgs/qa_example_pair.png -------------------------------------------------------------------------------- /inference.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayleicn/TVQAplus/HEAD/inference.py -------------------------------------------------------------------------------- /main.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayleicn/TVQAplus/HEAD/main.py -------------------------------------------------------------------------------- /model/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /model/cnn.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayleicn/TVQAplus/HEAD/model/cnn.py -------------------------------------------------------------------------------- /model/context_query_attention.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayleicn/TVQAplus/HEAD/model/context_query_attention.py -------------------------------------------------------------------------------- /model/encoder.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayleicn/TVQAplus/HEAD/model/encoder.py -------------------------------------------------------------------------------- /model/model_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayleicn/TVQAplus/HEAD/model/model_utils.py -------------------------------------------------------------------------------- /model/position_encoding.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayleicn/TVQAplus/HEAD/model/position_encoding.py -------------------------------------------------------------------------------- /model/self_attention.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayleicn/TVQAplus/HEAD/model/self_attention.py -------------------------------------------------------------------------------- /model/stage.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayleicn/TVQAplus/HEAD/model/stage.py -------------------------------------------------------------------------------- /run_inference.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayleicn/TVQAplus/HEAD/run_inference.sh -------------------------------------------------------------------------------- /run_main.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayleicn/TVQAplus/HEAD/run_main.sh -------------------------------------------------------------------------------- /tvqa_dataset.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayleicn/TVQAplus/HEAD/tvqa_dataset.py -------------------------------------------------------------------------------- /utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayleicn/TVQAplus/HEAD/utils.py --------------------------------------------------------------------------------