├── .gitignore ├── README.md ├── example └── create_dataset.md ├── llm_trainer ├── __init__.py ├── checkpoint.py ├── dataset.py ├── dpo_trainer.py ├── ds_checkpoint.py ├── eval.py ├── generate_utils.py ├── grpo_trainer.py ├── log.py ├── loss.py ├── parallel.py ├── partition_utils.py ├── ppo_trainer.py ├── scheduler.py ├── sft_trainer.py ├── tokenizer.py ├── tools.py ├── train_configs.py ├── trainer.py └── utils.py ├── mk ├── scripts ├── calc_intermediate_size ├── ddp_train ├── ds_train ├── plot_log ├── plot_lr ├── py_train └── smart_train └── setup.py /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qibin0506/llm_trainer/HEAD/.gitignore -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qibin0506/llm_trainer/HEAD/README.md -------------------------------------------------------------------------------- /example/create_dataset.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qibin0506/llm_trainer/HEAD/example/create_dataset.md -------------------------------------------------------------------------------- /llm_trainer/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qibin0506/llm_trainer/HEAD/llm_trainer/__init__.py -------------------------------------------------------------------------------- /llm_trainer/checkpoint.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qibin0506/llm_trainer/HEAD/llm_trainer/checkpoint.py -------------------------------------------------------------------------------- /llm_trainer/dataset.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qibin0506/llm_trainer/HEAD/llm_trainer/dataset.py -------------------------------------------------------------------------------- /llm_trainer/dpo_trainer.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qibin0506/llm_trainer/HEAD/llm_trainer/dpo_trainer.py -------------------------------------------------------------------------------- /llm_trainer/ds_checkpoint.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qibin0506/llm_trainer/HEAD/llm_trainer/ds_checkpoint.py -------------------------------------------------------------------------------- /llm_trainer/eval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qibin0506/llm_trainer/HEAD/llm_trainer/eval.py -------------------------------------------------------------------------------- /llm_trainer/generate_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qibin0506/llm_trainer/HEAD/llm_trainer/generate_utils.py -------------------------------------------------------------------------------- /llm_trainer/grpo_trainer.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qibin0506/llm_trainer/HEAD/llm_trainer/grpo_trainer.py -------------------------------------------------------------------------------- /llm_trainer/log.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qibin0506/llm_trainer/HEAD/llm_trainer/log.py -------------------------------------------------------------------------------- /llm_trainer/loss.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qibin0506/llm_trainer/HEAD/llm_trainer/loss.py -------------------------------------------------------------------------------- /llm_trainer/parallel.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qibin0506/llm_trainer/HEAD/llm_trainer/parallel.py -------------------------------------------------------------------------------- /llm_trainer/partition_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qibin0506/llm_trainer/HEAD/llm_trainer/partition_utils.py -------------------------------------------------------------------------------- /llm_trainer/ppo_trainer.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qibin0506/llm_trainer/HEAD/llm_trainer/ppo_trainer.py -------------------------------------------------------------------------------- /llm_trainer/scheduler.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qibin0506/llm_trainer/HEAD/llm_trainer/scheduler.py -------------------------------------------------------------------------------- /llm_trainer/sft_trainer.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qibin0506/llm_trainer/HEAD/llm_trainer/sft_trainer.py -------------------------------------------------------------------------------- /llm_trainer/tokenizer.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qibin0506/llm_trainer/HEAD/llm_trainer/tokenizer.py -------------------------------------------------------------------------------- /llm_trainer/tools.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qibin0506/llm_trainer/HEAD/llm_trainer/tools.py -------------------------------------------------------------------------------- /llm_trainer/train_configs.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qibin0506/llm_trainer/HEAD/llm_trainer/train_configs.py -------------------------------------------------------------------------------- /llm_trainer/trainer.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qibin0506/llm_trainer/HEAD/llm_trainer/trainer.py -------------------------------------------------------------------------------- /llm_trainer/utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qibin0506/llm_trainer/HEAD/llm_trainer/utils.py -------------------------------------------------------------------------------- /mk: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qibin0506/llm_trainer/HEAD/mk -------------------------------------------------------------------------------- /scripts/calc_intermediate_size: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qibin0506/llm_trainer/HEAD/scripts/calc_intermediate_size -------------------------------------------------------------------------------- /scripts/ddp_train: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qibin0506/llm_trainer/HEAD/scripts/ddp_train -------------------------------------------------------------------------------- /scripts/ds_train: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qibin0506/llm_trainer/HEAD/scripts/ds_train -------------------------------------------------------------------------------- /scripts/plot_log: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qibin0506/llm_trainer/HEAD/scripts/plot_log -------------------------------------------------------------------------------- /scripts/plot_lr: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qibin0506/llm_trainer/HEAD/scripts/plot_lr -------------------------------------------------------------------------------- /scripts/py_train: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qibin0506/llm_trainer/HEAD/scripts/py_train -------------------------------------------------------------------------------- /scripts/smart_train: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qibin0506/llm_trainer/HEAD/scripts/smart_train -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/qibin0506/llm_trainer/HEAD/setup.py --------------------------------------------------------------------------------