├── LICENSE ├── README.md ├── assets ├── images │ ├── glotlid_logo.svg │ └── zxx-und.png ├── inference │ ├── customlid.py │ ├── language_names.json │ ├── metrics.py │ ├── throughput.py │ └── vis_contribution.py ├── sources │ └── wiki.py ├── train │ ├── dedup_split.py │ ├── gen_und.py │ └── upsample.py └── utils │ └── baselines.py ├── languages-v3.md └── sources.md /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cisnlp/GlotLID/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cisnlp/GlotLID/HEAD/README.md -------------------------------------------------------------------------------- /assets/images/glotlid_logo.svg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cisnlp/GlotLID/HEAD/assets/images/glotlid_logo.svg -------------------------------------------------------------------------------- /assets/images/zxx-und.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cisnlp/GlotLID/HEAD/assets/images/zxx-und.png -------------------------------------------------------------------------------- /assets/inference/customlid.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cisnlp/GlotLID/HEAD/assets/inference/customlid.py -------------------------------------------------------------------------------- /assets/inference/language_names.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cisnlp/GlotLID/HEAD/assets/inference/language_names.json -------------------------------------------------------------------------------- /assets/inference/metrics.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cisnlp/GlotLID/HEAD/assets/inference/metrics.py -------------------------------------------------------------------------------- /assets/inference/throughput.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cisnlp/GlotLID/HEAD/assets/inference/throughput.py -------------------------------------------------------------------------------- /assets/inference/vis_contribution.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cisnlp/GlotLID/HEAD/assets/inference/vis_contribution.py -------------------------------------------------------------------------------- /assets/sources/wiki.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cisnlp/GlotLID/HEAD/assets/sources/wiki.py -------------------------------------------------------------------------------- /assets/train/dedup_split.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cisnlp/GlotLID/HEAD/assets/train/dedup_split.py -------------------------------------------------------------------------------- /assets/train/gen_und.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cisnlp/GlotLID/HEAD/assets/train/gen_und.py -------------------------------------------------------------------------------- /assets/train/upsample.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cisnlp/GlotLID/HEAD/assets/train/upsample.py -------------------------------------------------------------------------------- /assets/utils/baselines.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cisnlp/GlotLID/HEAD/assets/utils/baselines.py -------------------------------------------------------------------------------- /languages-v3.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cisnlp/GlotLID/HEAD/languages-v3.md -------------------------------------------------------------------------------- /sources.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cisnlp/GlotLID/HEAD/sources.md --------------------------------------------------------------------------------