├── .gitattributes ├── .gitignore ├── LICENSE ├── README.md ├── configs ├── config.yaml ├── config_iter1.yaml ├── config_iter2.yaml ├── config_iter3.yaml ├── deepspeed_zero3.yaml ├── multi_gpu.yaml └── tulu-70B │ ├── config.yaml │ └── deepspeed_zero3.yaml ├── images ├── dpo_compare.png ├── iter_openllm.png └── spin_dalle.png ├── scripts ├── finetune.sh ├── finetune_iter1.sh ├── finetune_iter2.sh ├── finetune_iter3.sh ├── generate.sh ├── generate_vllm.sh ├── generate_vllm_batched.sh └── tulu-70B │ ├── README.md │ ├── finetune.sh │ ├── generate_training_data.sh │ └── generate_vllm_batched.sh ├── setup.cfg ├── setup.py └── spin ├── alignment ├── __init__.py ├── configs.py ├── data.py ├── model_utils.py ├── release.py ├── trainer.py └── utils.py ├── batched_generate_vllm.py ├── convert_data.py ├── generate.py ├── generate_vllm.py ├── reformat.py └── run_spin.py /.gitattributes: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/.gitattributes -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/.gitignore -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/README.md -------------------------------------------------------------------------------- /configs/config.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/configs/config.yaml -------------------------------------------------------------------------------- /configs/config_iter1.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/configs/config_iter1.yaml -------------------------------------------------------------------------------- /configs/config_iter2.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/configs/config_iter2.yaml -------------------------------------------------------------------------------- /configs/config_iter3.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/configs/config_iter3.yaml -------------------------------------------------------------------------------- /configs/deepspeed_zero3.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/configs/deepspeed_zero3.yaml -------------------------------------------------------------------------------- /configs/multi_gpu.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/configs/multi_gpu.yaml -------------------------------------------------------------------------------- /configs/tulu-70B/config.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/configs/tulu-70B/config.yaml -------------------------------------------------------------------------------- /configs/tulu-70B/deepspeed_zero3.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/configs/tulu-70B/deepspeed_zero3.yaml -------------------------------------------------------------------------------- /images/dpo_compare.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/images/dpo_compare.png -------------------------------------------------------------------------------- /images/iter_openllm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/images/iter_openllm.png -------------------------------------------------------------------------------- /images/spin_dalle.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/images/spin_dalle.png -------------------------------------------------------------------------------- /scripts/finetune.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/scripts/finetune.sh -------------------------------------------------------------------------------- /scripts/finetune_iter1.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/scripts/finetune_iter1.sh -------------------------------------------------------------------------------- /scripts/finetune_iter2.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/scripts/finetune_iter2.sh -------------------------------------------------------------------------------- /scripts/finetune_iter3.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/scripts/finetune_iter3.sh -------------------------------------------------------------------------------- /scripts/generate.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/scripts/generate.sh -------------------------------------------------------------------------------- /scripts/generate_vllm.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/scripts/generate_vllm.sh -------------------------------------------------------------------------------- /scripts/generate_vllm_batched.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/scripts/generate_vllm_batched.sh -------------------------------------------------------------------------------- /scripts/tulu-70B/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/scripts/tulu-70B/README.md -------------------------------------------------------------------------------- /scripts/tulu-70B/finetune.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/scripts/tulu-70B/finetune.sh -------------------------------------------------------------------------------- /scripts/tulu-70B/generate_training_data.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/scripts/tulu-70B/generate_training_data.sh -------------------------------------------------------------------------------- /scripts/tulu-70B/generate_vllm_batched.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/scripts/tulu-70B/generate_vllm_batched.sh -------------------------------------------------------------------------------- /setup.cfg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/setup.cfg -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/setup.py -------------------------------------------------------------------------------- /spin/alignment/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/spin/alignment/__init__.py -------------------------------------------------------------------------------- /spin/alignment/configs.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/spin/alignment/configs.py -------------------------------------------------------------------------------- /spin/alignment/data.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/spin/alignment/data.py -------------------------------------------------------------------------------- /spin/alignment/model_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/spin/alignment/model_utils.py -------------------------------------------------------------------------------- /spin/alignment/release.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/spin/alignment/release.py -------------------------------------------------------------------------------- /spin/alignment/trainer.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/spin/alignment/trainer.py -------------------------------------------------------------------------------- /spin/alignment/utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/spin/alignment/utils.py -------------------------------------------------------------------------------- /spin/batched_generate_vllm.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/spin/batched_generate_vllm.py -------------------------------------------------------------------------------- /spin/convert_data.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/spin/convert_data.py -------------------------------------------------------------------------------- /spin/generate.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/spin/generate.py -------------------------------------------------------------------------------- /spin/generate_vllm.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/spin/generate_vllm.py -------------------------------------------------------------------------------- /spin/reformat.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/spin/reformat.py -------------------------------------------------------------------------------- /spin/run_spin.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/uclaml/SPIN/HEAD/spin/run_spin.py --------------------------------------------------------------------------------