├── .github └── workflows │ └── python-publish.yml ├── .gitignore ├── LICENSE ├── Makefile ├── README.md ├── docs ├── perf.png ├── prefix_sharing.excalidraw └── prefix_sharing.png ├── examples ├── demo.py ├── dpo_trl.py └── reward_modeling_trl.py ├── flash_pref ├── __init__.py ├── flash_dpo_trainer.py ├── flash_reward_trainer.py └── shared_prefix.py ├── pyproject.toml └── tests ├── benchmark.py ├── perf.csv ├── plot.py └── test_flash_pref.py /.github/workflows/python-publish.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/li-plus/flash-preference/HEAD/.github/workflows/python-publish.yml -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/li-plus/flash-preference/HEAD/.gitignore -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/li-plus/flash-preference/HEAD/LICENSE -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/li-plus/flash-preference/HEAD/Makefile -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/li-plus/flash-preference/HEAD/README.md -------------------------------------------------------------------------------- /docs/perf.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/li-plus/flash-preference/HEAD/docs/perf.png -------------------------------------------------------------------------------- /docs/prefix_sharing.excalidraw: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/li-plus/flash-preference/HEAD/docs/prefix_sharing.excalidraw -------------------------------------------------------------------------------- /docs/prefix_sharing.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/li-plus/flash-preference/HEAD/docs/prefix_sharing.png -------------------------------------------------------------------------------- /examples/demo.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/li-plus/flash-preference/HEAD/examples/demo.py -------------------------------------------------------------------------------- /examples/dpo_trl.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/li-plus/flash-preference/HEAD/examples/dpo_trl.py -------------------------------------------------------------------------------- /examples/reward_modeling_trl.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/li-plus/flash-preference/HEAD/examples/reward_modeling_trl.py -------------------------------------------------------------------------------- /flash_pref/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/li-plus/flash-preference/HEAD/flash_pref/__init__.py -------------------------------------------------------------------------------- /flash_pref/flash_dpo_trainer.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/li-plus/flash-preference/HEAD/flash_pref/flash_dpo_trainer.py -------------------------------------------------------------------------------- /flash_pref/flash_reward_trainer.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/li-plus/flash-preference/HEAD/flash_pref/flash_reward_trainer.py -------------------------------------------------------------------------------- /flash_pref/shared_prefix.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/li-plus/flash-preference/HEAD/flash_pref/shared_prefix.py -------------------------------------------------------------------------------- /pyproject.toml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/li-plus/flash-preference/HEAD/pyproject.toml -------------------------------------------------------------------------------- /tests/benchmark.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/li-plus/flash-preference/HEAD/tests/benchmark.py -------------------------------------------------------------------------------- /tests/perf.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/li-plus/flash-preference/HEAD/tests/perf.csv -------------------------------------------------------------------------------- /tests/plot.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/li-plus/flash-preference/HEAD/tests/plot.py -------------------------------------------------------------------------------- /tests/test_flash_pref.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/li-plus/flash-preference/HEAD/tests/test_flash_pref.py --------------------------------------------------------------------------------