├── .gitignore ├── LICENSE ├── README.md ├── figures ├── dataset.png ├── logo.png ├── ultracm.png ├── ultraf.png └── ultrarm.png └── src ├── comparison_data_generation ├── fastchat.py ├── main.py ├── main_vllm.py ├── run.sh ├── run_vllm.sh └── sampling.py └── data_annotation ├── annotate_critique.py ├── annotate_preference.py ├── fix_overall_score_issue.py └── preference_templates.py /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenBMB/UltraFeedback/HEAD/.gitignore -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenBMB/UltraFeedback/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenBMB/UltraFeedback/HEAD/README.md -------------------------------------------------------------------------------- /figures/dataset.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenBMB/UltraFeedback/HEAD/figures/dataset.png -------------------------------------------------------------------------------- /figures/logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenBMB/UltraFeedback/HEAD/figures/logo.png -------------------------------------------------------------------------------- /figures/ultracm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenBMB/UltraFeedback/HEAD/figures/ultracm.png -------------------------------------------------------------------------------- /figures/ultraf.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenBMB/UltraFeedback/HEAD/figures/ultraf.png -------------------------------------------------------------------------------- /figures/ultrarm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenBMB/UltraFeedback/HEAD/figures/ultrarm.png -------------------------------------------------------------------------------- /src/comparison_data_generation/fastchat.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenBMB/UltraFeedback/HEAD/src/comparison_data_generation/fastchat.py -------------------------------------------------------------------------------- /src/comparison_data_generation/main.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenBMB/UltraFeedback/HEAD/src/comparison_data_generation/main.py -------------------------------------------------------------------------------- /src/comparison_data_generation/main_vllm.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenBMB/UltraFeedback/HEAD/src/comparison_data_generation/main_vllm.py -------------------------------------------------------------------------------- /src/comparison_data_generation/run.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenBMB/UltraFeedback/HEAD/src/comparison_data_generation/run.sh -------------------------------------------------------------------------------- /src/comparison_data_generation/run_vllm.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenBMB/UltraFeedback/HEAD/src/comparison_data_generation/run_vllm.sh -------------------------------------------------------------------------------- /src/comparison_data_generation/sampling.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenBMB/UltraFeedback/HEAD/src/comparison_data_generation/sampling.py -------------------------------------------------------------------------------- /src/data_annotation/annotate_critique.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenBMB/UltraFeedback/HEAD/src/data_annotation/annotate_critique.py -------------------------------------------------------------------------------- /src/data_annotation/annotate_preference.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenBMB/UltraFeedback/HEAD/src/data_annotation/annotate_preference.py -------------------------------------------------------------------------------- /src/data_annotation/fix_overall_score_issue.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenBMB/UltraFeedback/HEAD/src/data_annotation/fix_overall_score_issue.py -------------------------------------------------------------------------------- /src/data_annotation/preference_templates.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenBMB/UltraFeedback/HEAD/src/data_annotation/preference_templates.py --------------------------------------------------------------------------------