├── .gitignore ├── Config ├── TrainConfig.py └── __init__.py ├── DataLoader ├── VectorDataSet.py └── __init__.py ├── Evaluator ├── VectorEvaluator.py └── __init__.py ├── InfoLogger ├── VectorInfoLogger.py └── __init__.py ├── LICENSE ├── LossModel ├── VectorLossModel.py └── __init__.py ├── Model ├── VectorModel.py └── __init__.py ├── ModelSaver ├── GeneralModelSaver.py └── __init__.py ├── README.md ├── Trainer ├── VectorTrainer.py └── __init__.py ├── Utils └── LoggerUtil.py ├── backbone_model_configs └── bert_config.json ├── convert_tf_roformer_to_pt.py ├── get_vocab.py ├── pretrain_roformer.py ├── run_finetune.py ├── run_predict.py └── train_configs ├── conf.yml ├── conf_B128_L128.yml ├── conf_B128_L256.yml ├── conf_B256_L128.yml └── conf_B256_L256.yml /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DunZhang/DFPassageRetrieve/HEAD/.gitignore -------------------------------------------------------------------------------- /Config/TrainConfig.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DunZhang/DFPassageRetrieve/HEAD/Config/TrainConfig.py -------------------------------------------------------------------------------- /Config/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /DataLoader/VectorDataSet.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DunZhang/DFPassageRetrieve/HEAD/DataLoader/VectorDataSet.py -------------------------------------------------------------------------------- /DataLoader/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /Evaluator/VectorEvaluator.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DunZhang/DFPassageRetrieve/HEAD/Evaluator/VectorEvaluator.py -------------------------------------------------------------------------------- /Evaluator/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /InfoLogger/VectorInfoLogger.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DunZhang/DFPassageRetrieve/HEAD/InfoLogger/VectorInfoLogger.py -------------------------------------------------------------------------------- /InfoLogger/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DunZhang/DFPassageRetrieve/HEAD/LICENSE -------------------------------------------------------------------------------- /LossModel/VectorLossModel.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DunZhang/DFPassageRetrieve/HEAD/LossModel/VectorLossModel.py -------------------------------------------------------------------------------- /LossModel/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /Model/VectorModel.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DunZhang/DFPassageRetrieve/HEAD/Model/VectorModel.py -------------------------------------------------------------------------------- /Model/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /ModelSaver/GeneralModelSaver.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DunZhang/DFPassageRetrieve/HEAD/ModelSaver/GeneralModelSaver.py -------------------------------------------------------------------------------- /ModelSaver/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DunZhang/DFPassageRetrieve/HEAD/README.md -------------------------------------------------------------------------------- /Trainer/VectorTrainer.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DunZhang/DFPassageRetrieve/HEAD/Trainer/VectorTrainer.py -------------------------------------------------------------------------------- /Trainer/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /Utils/LoggerUtil.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DunZhang/DFPassageRetrieve/HEAD/Utils/LoggerUtil.py -------------------------------------------------------------------------------- /backbone_model_configs/bert_config.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DunZhang/DFPassageRetrieve/HEAD/backbone_model_configs/bert_config.json -------------------------------------------------------------------------------- /convert_tf_roformer_to_pt.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DunZhang/DFPassageRetrieve/HEAD/convert_tf_roformer_to_pt.py -------------------------------------------------------------------------------- /get_vocab.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DunZhang/DFPassageRetrieve/HEAD/get_vocab.py -------------------------------------------------------------------------------- /pretrain_roformer.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DunZhang/DFPassageRetrieve/HEAD/pretrain_roformer.py -------------------------------------------------------------------------------- /run_finetune.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DunZhang/DFPassageRetrieve/HEAD/run_finetune.py -------------------------------------------------------------------------------- /run_predict.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DunZhang/DFPassageRetrieve/HEAD/run_predict.py -------------------------------------------------------------------------------- /train_configs/conf.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DunZhang/DFPassageRetrieve/HEAD/train_configs/conf.yml -------------------------------------------------------------------------------- /train_configs/conf_B128_L128.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DunZhang/DFPassageRetrieve/HEAD/train_configs/conf_B128_L128.yml -------------------------------------------------------------------------------- /train_configs/conf_B128_L256.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DunZhang/DFPassageRetrieve/HEAD/train_configs/conf_B128_L256.yml -------------------------------------------------------------------------------- /train_configs/conf_B256_L128.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DunZhang/DFPassageRetrieve/HEAD/train_configs/conf_B256_L128.yml -------------------------------------------------------------------------------- /train_configs/conf_B256_L256.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DunZhang/DFPassageRetrieve/HEAD/train_configs/conf_B256_L256.yml --------------------------------------------------------------------------------