├── .gitignore ├── README.md ├── assets └── prefix_sharing.png ├── benchmark ├── __init__.py └── utils.py ├── config.py ├── configs └── zero3.yaml ├── data ├── __init__.py ├── collators.py ├── packing.py ├── patch_datasets.py └── utils.py ├── modeling ├── dpo_flex_attn_masks.py ├── llama_patches.py └── mistral_patches.py ├── requirements.txt ├── scripts ├── run_capybara_no_packing.sh ├── run_capybara_packing.sh ├── run_ultrafeedback_no_packing.sh └── run_ultrafeedback_packing.sh ├── train_dpo.py └── trainer.py /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/frankxwang/dpo-prefix-sharing/HEAD/.gitignore -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/frankxwang/dpo-prefix-sharing/HEAD/README.md -------------------------------------------------------------------------------- /assets/prefix_sharing.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/frankxwang/dpo-prefix-sharing/HEAD/assets/prefix_sharing.png -------------------------------------------------------------------------------- /benchmark/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /benchmark/utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/frankxwang/dpo-prefix-sharing/HEAD/benchmark/utils.py -------------------------------------------------------------------------------- /config.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/frankxwang/dpo-prefix-sharing/HEAD/config.py -------------------------------------------------------------------------------- /configs/zero3.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/frankxwang/dpo-prefix-sharing/HEAD/configs/zero3.yaml -------------------------------------------------------------------------------- /data/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /data/collators.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/frankxwang/dpo-prefix-sharing/HEAD/data/collators.py -------------------------------------------------------------------------------- /data/packing.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/frankxwang/dpo-prefix-sharing/HEAD/data/packing.py -------------------------------------------------------------------------------- /data/patch_datasets.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/frankxwang/dpo-prefix-sharing/HEAD/data/patch_datasets.py -------------------------------------------------------------------------------- /data/utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/frankxwang/dpo-prefix-sharing/HEAD/data/utils.py -------------------------------------------------------------------------------- /modeling/dpo_flex_attn_masks.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/frankxwang/dpo-prefix-sharing/HEAD/modeling/dpo_flex_attn_masks.py -------------------------------------------------------------------------------- /modeling/llama_patches.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/frankxwang/dpo-prefix-sharing/HEAD/modeling/llama_patches.py -------------------------------------------------------------------------------- /modeling/mistral_patches.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/frankxwang/dpo-prefix-sharing/HEAD/modeling/mistral_patches.py -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/frankxwang/dpo-prefix-sharing/HEAD/requirements.txt -------------------------------------------------------------------------------- /scripts/run_capybara_no_packing.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/frankxwang/dpo-prefix-sharing/HEAD/scripts/run_capybara_no_packing.sh -------------------------------------------------------------------------------- /scripts/run_capybara_packing.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/frankxwang/dpo-prefix-sharing/HEAD/scripts/run_capybara_packing.sh -------------------------------------------------------------------------------- /scripts/run_ultrafeedback_no_packing.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/frankxwang/dpo-prefix-sharing/HEAD/scripts/run_ultrafeedback_no_packing.sh -------------------------------------------------------------------------------- /scripts/run_ultrafeedback_packing.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/frankxwang/dpo-prefix-sharing/HEAD/scripts/run_ultrafeedback_packing.sh -------------------------------------------------------------------------------- /train_dpo.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/frankxwang/dpo-prefix-sharing/HEAD/train_dpo.py -------------------------------------------------------------------------------- /trainer.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/frankxwang/dpo-prefix-sharing/HEAD/trainer.py --------------------------------------------------------------------------------