├── .gitignore ├── README.md ├── ds_zero2_1gpu.json ├── ds_zero3_1gpu.json ├── gpu_run_mlm.sh ├── requirements.txt ├── run_clm.py ├── run_clm_no_trainer.py ├── run_mlm.py ├── run_mlm_no_trainer.py ├── run_plm.py ├── sampled_20190101_20200611_v2.txt └── zero3_gpu_run_mlm.sh /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Beomi/transformers-language-modeling/HEAD/.gitignore -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Beomi/transformers-language-modeling/HEAD/README.md -------------------------------------------------------------------------------- /ds_zero2_1gpu.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Beomi/transformers-language-modeling/HEAD/ds_zero2_1gpu.json -------------------------------------------------------------------------------- /ds_zero3_1gpu.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Beomi/transformers-language-modeling/HEAD/ds_zero3_1gpu.json -------------------------------------------------------------------------------- /gpu_run_mlm.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Beomi/transformers-language-modeling/HEAD/gpu_run_mlm.sh -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | datasets >= 1.1.3 2 | sentencepiece != 0.1.92 3 | protobuf 4 | -------------------------------------------------------------------------------- /run_clm.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Beomi/transformers-language-modeling/HEAD/run_clm.py -------------------------------------------------------------------------------- /run_clm_no_trainer.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Beomi/transformers-language-modeling/HEAD/run_clm_no_trainer.py -------------------------------------------------------------------------------- /run_mlm.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Beomi/transformers-language-modeling/HEAD/run_mlm.py -------------------------------------------------------------------------------- /run_mlm_no_trainer.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Beomi/transformers-language-modeling/HEAD/run_mlm_no_trainer.py -------------------------------------------------------------------------------- /run_plm.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Beomi/transformers-language-modeling/HEAD/run_plm.py -------------------------------------------------------------------------------- /sampled_20190101_20200611_v2.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Beomi/transformers-language-modeling/HEAD/sampled_20190101_20200611_v2.txt -------------------------------------------------------------------------------- /zero3_gpu_run_mlm.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Beomi/transformers-language-modeling/HEAD/zero3_gpu_run_mlm.sh --------------------------------------------------------------------------------