├── .gitignore ├── README.md ├── assets ├── Figure_WPRMCollection.pdf ├── Figure_WPRMCollection.png ├── Figure_WebShepherd.pdf ├── Figure_WebShepherd.png ├── Figure_performance.pdf ├── Figure_performance.png ├── WebShepherd.pdf ├── huggingface_logo-noborder.svg └── shepherd_emoji.png ├── evaluate_webprm.sh ├── evaluate_webshepherd.sh ├── run_vllm.sh └── src ├── evaluate_web_reward_bench.py ├── utils ├── eval_utils.py ├── inference_utils.py └── utils.py └── webprm ├── __init__.py ├── models.py └── prompts ├── __init__.py ├── action.py ├── checklist_prompt.py ├── construct_messages.py ├── eval_type.py ├── image_utils.py ├── input_information.py └── judge_prompt.py /.gitignore: -------------------------------------------------------------------------------- 1 | **/__pycache__/ 2 | results_benchmark/ -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyle8581/Web-Shepherd/HEAD/README.md -------------------------------------------------------------------------------- /assets/Figure_WPRMCollection.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyle8581/Web-Shepherd/HEAD/assets/Figure_WPRMCollection.pdf -------------------------------------------------------------------------------- /assets/Figure_WPRMCollection.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyle8581/Web-Shepherd/HEAD/assets/Figure_WPRMCollection.png -------------------------------------------------------------------------------- /assets/Figure_WebShepherd.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyle8581/Web-Shepherd/HEAD/assets/Figure_WebShepherd.pdf -------------------------------------------------------------------------------- /assets/Figure_WebShepherd.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyle8581/Web-Shepherd/HEAD/assets/Figure_WebShepherd.png -------------------------------------------------------------------------------- /assets/Figure_performance.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyle8581/Web-Shepherd/HEAD/assets/Figure_performance.pdf -------------------------------------------------------------------------------- /assets/Figure_performance.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyle8581/Web-Shepherd/HEAD/assets/Figure_performance.png -------------------------------------------------------------------------------- /assets/WebShepherd.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyle8581/Web-Shepherd/HEAD/assets/WebShepherd.pdf -------------------------------------------------------------------------------- /assets/huggingface_logo-noborder.svg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyle8581/Web-Shepherd/HEAD/assets/huggingface_logo-noborder.svg -------------------------------------------------------------------------------- /assets/shepherd_emoji.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyle8581/Web-Shepherd/HEAD/assets/shepherd_emoji.png -------------------------------------------------------------------------------- /evaluate_webprm.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyle8581/Web-Shepherd/HEAD/evaluate_webprm.sh -------------------------------------------------------------------------------- /evaluate_webshepherd.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyle8581/Web-Shepherd/HEAD/evaluate_webshepherd.sh -------------------------------------------------------------------------------- /run_vllm.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyle8581/Web-Shepherd/HEAD/run_vllm.sh -------------------------------------------------------------------------------- /src/evaluate_web_reward_bench.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyle8581/Web-Shepherd/HEAD/src/evaluate_web_reward_bench.py -------------------------------------------------------------------------------- /src/utils/eval_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyle8581/Web-Shepherd/HEAD/src/utils/eval_utils.py -------------------------------------------------------------------------------- /src/utils/inference_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyle8581/Web-Shepherd/HEAD/src/utils/inference_utils.py -------------------------------------------------------------------------------- /src/utils/utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyle8581/Web-Shepherd/HEAD/src/utils/utils.py -------------------------------------------------------------------------------- /src/webprm/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /src/webprm/models.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyle8581/Web-Shepherd/HEAD/src/webprm/models.py -------------------------------------------------------------------------------- /src/webprm/prompts/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyle8581/Web-Shepherd/HEAD/src/webprm/prompts/__init__.py -------------------------------------------------------------------------------- /src/webprm/prompts/action.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyle8581/Web-Shepherd/HEAD/src/webprm/prompts/action.py -------------------------------------------------------------------------------- /src/webprm/prompts/checklist_prompt.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyle8581/Web-Shepherd/HEAD/src/webprm/prompts/checklist_prompt.py -------------------------------------------------------------------------------- /src/webprm/prompts/construct_messages.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyle8581/Web-Shepherd/HEAD/src/webprm/prompts/construct_messages.py -------------------------------------------------------------------------------- /src/webprm/prompts/eval_type.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyle8581/Web-Shepherd/HEAD/src/webprm/prompts/eval_type.py -------------------------------------------------------------------------------- /src/webprm/prompts/image_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyle8581/Web-Shepherd/HEAD/src/webprm/prompts/image_utils.py -------------------------------------------------------------------------------- /src/webprm/prompts/input_information.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyle8581/Web-Shepherd/HEAD/src/webprm/prompts/input_information.py -------------------------------------------------------------------------------- /src/webprm/prompts/judge_prompt.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyle8581/Web-Shepherd/HEAD/src/webprm/prompts/judge_prompt.py --------------------------------------------------------------------------------