├── .gitignore ├── README.md ├── qa ├── basic_tokenizer.py ├── bert_retrieve_qa.py ├── config.py ├── datasets.py ├── eval_utils.py ├── msmarco_process.py ├── official_eval.py ├── online_sampler.py ├── prepro_dense.py ├── prepro_utils.py ├── tokenizer.py ├── train.py ├── train_dense_qa.sh ├── train_retrieve_qa.py └── utils.py ├── requirements.txt └── retrieval ├── basic_tokenizer.py ├── config.py ├── datasets.py ├── eval_retrieval.py ├── gen_index_id_map.py ├── get_embed.py ├── get_para_embed.sh ├── group_paras.py ├── retriever.py ├── tokenizer.py ├── train_retriever.py ├── train_retriever_cluster.sh ├── train_retriever_single.sh ├── trec_process.py └── utils.py /.gitignore: -------------------------------------------------------------------------------- 1 | *.pyc 2 | /data 3 | /pretrained_models 4 | *.zip 5 | retrieval/logs/ 6 | __MACOSX/ 7 | 8 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xwhan/ProQA/HEAD/README.md -------------------------------------------------------------------------------- /qa/basic_tokenizer.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xwhan/ProQA/HEAD/qa/basic_tokenizer.py -------------------------------------------------------------------------------- /qa/bert_retrieve_qa.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xwhan/ProQA/HEAD/qa/bert_retrieve_qa.py -------------------------------------------------------------------------------- /qa/config.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xwhan/ProQA/HEAD/qa/config.py -------------------------------------------------------------------------------- /qa/datasets.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xwhan/ProQA/HEAD/qa/datasets.py -------------------------------------------------------------------------------- /qa/eval_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xwhan/ProQA/HEAD/qa/eval_utils.py -------------------------------------------------------------------------------- /qa/msmarco_process.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xwhan/ProQA/HEAD/qa/msmarco_process.py -------------------------------------------------------------------------------- /qa/official_eval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xwhan/ProQA/HEAD/qa/official_eval.py -------------------------------------------------------------------------------- /qa/online_sampler.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xwhan/ProQA/HEAD/qa/online_sampler.py -------------------------------------------------------------------------------- /qa/prepro_dense.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xwhan/ProQA/HEAD/qa/prepro_dense.py -------------------------------------------------------------------------------- /qa/prepro_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xwhan/ProQA/HEAD/qa/prepro_utils.py -------------------------------------------------------------------------------- /qa/tokenizer.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xwhan/ProQA/HEAD/qa/tokenizer.py -------------------------------------------------------------------------------- /qa/train.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xwhan/ProQA/HEAD/qa/train.py -------------------------------------------------------------------------------- /qa/train_dense_qa.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xwhan/ProQA/HEAD/qa/train_dense_qa.sh -------------------------------------------------------------------------------- /qa/train_retrieve_qa.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xwhan/ProQA/HEAD/qa/train_retrieve_qa.py -------------------------------------------------------------------------------- /qa/utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xwhan/ProQA/HEAD/qa/utils.py -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xwhan/ProQA/HEAD/requirements.txt -------------------------------------------------------------------------------- /retrieval/basic_tokenizer.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xwhan/ProQA/HEAD/retrieval/basic_tokenizer.py -------------------------------------------------------------------------------- /retrieval/config.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xwhan/ProQA/HEAD/retrieval/config.py -------------------------------------------------------------------------------- /retrieval/datasets.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xwhan/ProQA/HEAD/retrieval/datasets.py -------------------------------------------------------------------------------- /retrieval/eval_retrieval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xwhan/ProQA/HEAD/retrieval/eval_retrieval.py -------------------------------------------------------------------------------- /retrieval/gen_index_id_map.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xwhan/ProQA/HEAD/retrieval/gen_index_id_map.py -------------------------------------------------------------------------------- /retrieval/get_embed.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xwhan/ProQA/HEAD/retrieval/get_embed.py -------------------------------------------------------------------------------- /retrieval/get_para_embed.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xwhan/ProQA/HEAD/retrieval/get_para_embed.sh -------------------------------------------------------------------------------- /retrieval/group_paras.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xwhan/ProQA/HEAD/retrieval/group_paras.py -------------------------------------------------------------------------------- /retrieval/retriever.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xwhan/ProQA/HEAD/retrieval/retriever.py -------------------------------------------------------------------------------- /retrieval/tokenizer.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xwhan/ProQA/HEAD/retrieval/tokenizer.py -------------------------------------------------------------------------------- /retrieval/train_retriever.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xwhan/ProQA/HEAD/retrieval/train_retriever.py -------------------------------------------------------------------------------- /retrieval/train_retriever_cluster.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xwhan/ProQA/HEAD/retrieval/train_retriever_cluster.sh -------------------------------------------------------------------------------- /retrieval/train_retriever_single.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xwhan/ProQA/HEAD/retrieval/train_retriever_single.sh -------------------------------------------------------------------------------- /retrieval/trec_process.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xwhan/ProQA/HEAD/retrieval/trec_process.py -------------------------------------------------------------------------------- /retrieval/utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xwhan/ProQA/HEAD/retrieval/utils.py --------------------------------------------------------------------------------