├── .github └── workflows │ └── pr-arena-workflow.yml ├── .gitignore ├── LICENSE ├── README.md ├── figures ├── overview.png └── results.png ├── index_el_datasets.py ├── index_parsing_datasets.py ├── index_pos_datasets.py ├── index_ted_datasets.py ├── langrank.py ├── langrank_predict.py ├── pretrained ├── DEP │ └── lgbm_model_dep_all.txt ├── EL │ └── lgbm_model_el_all.txt ├── MT │ ├── lgbm_model_mt_all.txt │ ├── lgbm_model_mt_aze.txt │ ├── lgbm_model_mt_ben.txt │ └── lgbm_model_mt_fin.txt └── POS │ └── lgbm_model_pos_all.txt ├── requirements.txt ├── sample-data ├── ell.tok ├── ell.tok.bpe ├── ted-train.orig.aze ├── ted-train.orig.ben ├── ted-train.orig.fin ├── ted-train.orig.spm8000.aze ├── ted-train.orig.spm8000.ben └── ted-train.orig.spm8000.fin └── tests └── test_train_file.py /.github/workflows/pr-arena-workflow.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/neulab/langrank/HEAD/.github/workflows/pr-arena-workflow.yml -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | *.swp 2 | __pycache__ 3 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/neulab/langrank/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/neulab/langrank/HEAD/README.md -------------------------------------------------------------------------------- /figures/overview.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/neulab/langrank/HEAD/figures/overview.png -------------------------------------------------------------------------------- /figures/results.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/neulab/langrank/HEAD/figures/results.png -------------------------------------------------------------------------------- /index_el_datasets.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/neulab/langrank/HEAD/index_el_datasets.py -------------------------------------------------------------------------------- /index_parsing_datasets.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/neulab/langrank/HEAD/index_parsing_datasets.py -------------------------------------------------------------------------------- /index_pos_datasets.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/neulab/langrank/HEAD/index_pos_datasets.py -------------------------------------------------------------------------------- /index_ted_datasets.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/neulab/langrank/HEAD/index_ted_datasets.py -------------------------------------------------------------------------------- /langrank.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/neulab/langrank/HEAD/langrank.py -------------------------------------------------------------------------------- /langrank_predict.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/neulab/langrank/HEAD/langrank_predict.py -------------------------------------------------------------------------------- /pretrained/DEP/lgbm_model_dep_all.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/neulab/langrank/HEAD/pretrained/DEP/lgbm_model_dep_all.txt -------------------------------------------------------------------------------- /pretrained/EL/lgbm_model_el_all.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/neulab/langrank/HEAD/pretrained/EL/lgbm_model_el_all.txt -------------------------------------------------------------------------------- /pretrained/MT/lgbm_model_mt_all.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/neulab/langrank/HEAD/pretrained/MT/lgbm_model_mt_all.txt -------------------------------------------------------------------------------- /pretrained/MT/lgbm_model_mt_aze.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/neulab/langrank/HEAD/pretrained/MT/lgbm_model_mt_aze.txt -------------------------------------------------------------------------------- /pretrained/MT/lgbm_model_mt_ben.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/neulab/langrank/HEAD/pretrained/MT/lgbm_model_mt_ben.txt -------------------------------------------------------------------------------- /pretrained/MT/lgbm_model_mt_fin.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/neulab/langrank/HEAD/pretrained/MT/lgbm_model_mt_fin.txt -------------------------------------------------------------------------------- /pretrained/POS/lgbm_model_pos_all.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/neulab/langrank/HEAD/pretrained/POS/lgbm_model_pos_all.txt -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | lightgbm 2 | -------------------------------------------------------------------------------- /sample-data/ell.tok: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/neulab/langrank/HEAD/sample-data/ell.tok -------------------------------------------------------------------------------- /sample-data/ell.tok.bpe: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/neulab/langrank/HEAD/sample-data/ell.tok.bpe -------------------------------------------------------------------------------- /sample-data/ted-train.orig.aze: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/neulab/langrank/HEAD/sample-data/ted-train.orig.aze -------------------------------------------------------------------------------- /sample-data/ted-train.orig.ben: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/neulab/langrank/HEAD/sample-data/ted-train.orig.ben -------------------------------------------------------------------------------- /sample-data/ted-train.orig.fin: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/neulab/langrank/HEAD/sample-data/ted-train.orig.fin -------------------------------------------------------------------------------- /sample-data/ted-train.orig.spm8000.aze: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/neulab/langrank/HEAD/sample-data/ted-train.orig.spm8000.aze -------------------------------------------------------------------------------- /sample-data/ted-train.orig.spm8000.ben: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/neulab/langrank/HEAD/sample-data/ted-train.orig.spm8000.ben -------------------------------------------------------------------------------- /sample-data/ted-train.orig.spm8000.fin: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/neulab/langrank/HEAD/sample-data/ted-train.orig.spm8000.fin -------------------------------------------------------------------------------- /tests/test_train_file.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/neulab/langrank/HEAD/tests/test_train_file.py --------------------------------------------------------------------------------