├── .deepspeed_env ├── .gitignore ├── LICENSE ├── README.md ├── configs ├── ds_config_basic.json ├── ds_config_fp16.json ├── ds_config_zero1.json ├── ds_config_zero1_fp16.json └── hostfile ├── data ├── verb-form-vocab.txt └── vocabulary │ ├── d_tags.txt │ ├── labels.txt │ ├── labels_zh.txt │ └── non_padded_namespaces.txt ├── gector.yaml ├── predict.py ├── requirements.txt ├── scripts ├── predict.sh ├── prepare_data.sh └── train.sh ├── src ├── dataset.py ├── model.py ├── predictor.py └── trainer.py ├── train.py └── utils ├── data_utils.py ├── generate_labels.py ├── helpers.py ├── mismatched_utils.py ├── preprocess_data.py ├── segment.py └── tokenization.py /.deepspeed_env: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cofe-ai/fast-gector/HEAD/.deepspeed_env -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cofe-ai/fast-gector/HEAD/.gitignore -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cofe-ai/fast-gector/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cofe-ai/fast-gector/HEAD/README.md -------------------------------------------------------------------------------- /configs/ds_config_basic.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cofe-ai/fast-gector/HEAD/configs/ds_config_basic.json -------------------------------------------------------------------------------- /configs/ds_config_fp16.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cofe-ai/fast-gector/HEAD/configs/ds_config_fp16.json -------------------------------------------------------------------------------- /configs/ds_config_zero1.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cofe-ai/fast-gector/HEAD/configs/ds_config_zero1.json -------------------------------------------------------------------------------- /configs/ds_config_zero1_fp16.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cofe-ai/fast-gector/HEAD/configs/ds_config_zero1_fp16.json -------------------------------------------------------------------------------- /configs/hostfile: -------------------------------------------------------------------------------- 1 | localhost slots=8 2 | -------------------------------------------------------------------------------- /data/verb-form-vocab.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cofe-ai/fast-gector/HEAD/data/verb-form-vocab.txt -------------------------------------------------------------------------------- /data/vocabulary/d_tags.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cofe-ai/fast-gector/HEAD/data/vocabulary/d_tags.txt -------------------------------------------------------------------------------- /data/vocabulary/labels.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cofe-ai/fast-gector/HEAD/data/vocabulary/labels.txt -------------------------------------------------------------------------------- /data/vocabulary/labels_zh.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cofe-ai/fast-gector/HEAD/data/vocabulary/labels_zh.txt -------------------------------------------------------------------------------- /data/vocabulary/non_padded_namespaces.txt: -------------------------------------------------------------------------------- 1 | *tags 2 | *labels 3 | -------------------------------------------------------------------------------- /gector.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cofe-ai/fast-gector/HEAD/gector.yaml -------------------------------------------------------------------------------- /predict.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cofe-ai/fast-gector/HEAD/predict.py -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cofe-ai/fast-gector/HEAD/requirements.txt -------------------------------------------------------------------------------- /scripts/predict.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cofe-ai/fast-gector/HEAD/scripts/predict.sh -------------------------------------------------------------------------------- /scripts/prepare_data.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cofe-ai/fast-gector/HEAD/scripts/prepare_data.sh -------------------------------------------------------------------------------- /scripts/train.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cofe-ai/fast-gector/HEAD/scripts/train.sh -------------------------------------------------------------------------------- /src/dataset.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cofe-ai/fast-gector/HEAD/src/dataset.py -------------------------------------------------------------------------------- /src/model.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cofe-ai/fast-gector/HEAD/src/model.py -------------------------------------------------------------------------------- /src/predictor.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cofe-ai/fast-gector/HEAD/src/predictor.py -------------------------------------------------------------------------------- /src/trainer.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cofe-ai/fast-gector/HEAD/src/trainer.py -------------------------------------------------------------------------------- /train.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cofe-ai/fast-gector/HEAD/train.py -------------------------------------------------------------------------------- /utils/data_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cofe-ai/fast-gector/HEAD/utils/data_utils.py -------------------------------------------------------------------------------- /utils/generate_labels.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cofe-ai/fast-gector/HEAD/utils/generate_labels.py -------------------------------------------------------------------------------- /utils/helpers.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cofe-ai/fast-gector/HEAD/utils/helpers.py -------------------------------------------------------------------------------- /utils/mismatched_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cofe-ai/fast-gector/HEAD/utils/mismatched_utils.py -------------------------------------------------------------------------------- /utils/preprocess_data.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cofe-ai/fast-gector/HEAD/utils/preprocess_data.py -------------------------------------------------------------------------------- /utils/segment.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cofe-ai/fast-gector/HEAD/utils/segment.py -------------------------------------------------------------------------------- /utils/tokenization.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cofe-ai/fast-gector/HEAD/utils/tokenization.py --------------------------------------------------------------------------------