├── LICENSE ├── README.md ├── configs ├── ds_config.json └── ds_config_zero1.json ├── data └── alpaca_data_sample_oneline_format.json ├── examples ├── train.py ├── train_llama_deepspeed.sh └── train_slurm.sh ├── requirements.txt ├── scripts ├── check_data.py ├── convert2ckpt.py ├── convert2dialog_mode_data_format.py ├── convert2hf.py ├── data_utils.py └── pretokenize.py ├── setup.py └── src └── transpeeder ├── feeder.py ├── models ├── llama_pipeline_model.py ├── modeling_llama.py └── patching.py └── utils.py /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HuangLK/transpeeder/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HuangLK/transpeeder/HEAD/README.md -------------------------------------------------------------------------------- /configs/ds_config.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HuangLK/transpeeder/HEAD/configs/ds_config.json -------------------------------------------------------------------------------- /configs/ds_config_zero1.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HuangLK/transpeeder/HEAD/configs/ds_config_zero1.json -------------------------------------------------------------------------------- /data/alpaca_data_sample_oneline_format.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HuangLK/transpeeder/HEAD/data/alpaca_data_sample_oneline_format.json -------------------------------------------------------------------------------- /examples/train.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HuangLK/transpeeder/HEAD/examples/train.py -------------------------------------------------------------------------------- /examples/train_llama_deepspeed.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HuangLK/transpeeder/HEAD/examples/train_llama_deepspeed.sh -------------------------------------------------------------------------------- /examples/train_slurm.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HuangLK/transpeeder/HEAD/examples/train_slurm.sh -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HuangLK/transpeeder/HEAD/requirements.txt -------------------------------------------------------------------------------- /scripts/check_data.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HuangLK/transpeeder/HEAD/scripts/check_data.py -------------------------------------------------------------------------------- /scripts/convert2ckpt.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HuangLK/transpeeder/HEAD/scripts/convert2ckpt.py -------------------------------------------------------------------------------- /scripts/convert2dialog_mode_data_format.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HuangLK/transpeeder/HEAD/scripts/convert2dialog_mode_data_format.py -------------------------------------------------------------------------------- /scripts/convert2hf.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HuangLK/transpeeder/HEAD/scripts/convert2hf.py -------------------------------------------------------------------------------- /scripts/data_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HuangLK/transpeeder/HEAD/scripts/data_utils.py -------------------------------------------------------------------------------- /scripts/pretokenize.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HuangLK/transpeeder/HEAD/scripts/pretokenize.py -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HuangLK/transpeeder/HEAD/setup.py -------------------------------------------------------------------------------- /src/transpeeder/feeder.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HuangLK/transpeeder/HEAD/src/transpeeder/feeder.py -------------------------------------------------------------------------------- /src/transpeeder/models/llama_pipeline_model.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HuangLK/transpeeder/HEAD/src/transpeeder/models/llama_pipeline_model.py -------------------------------------------------------------------------------- /src/transpeeder/models/modeling_llama.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HuangLK/transpeeder/HEAD/src/transpeeder/models/modeling_llama.py -------------------------------------------------------------------------------- /src/transpeeder/models/patching.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HuangLK/transpeeder/HEAD/src/transpeeder/models/patching.py -------------------------------------------------------------------------------- /src/transpeeder/utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HuangLK/transpeeder/HEAD/src/transpeeder/utils.py --------------------------------------------------------------------------------