├── README.md ├── compare_with_local_model.py ├── conversation.py ├── figures ├── comparison_reported_matheval.md ├── corner_case_extraction_0.png ├── corner_case_extraction_1.png ├── corner_case_extraction_2.png ├── corner_case_extraction_3.png ├── corner_case_verification_0.png ├── corner_case_verification_1.png ├── corner_case_verification_2.png ├── corner_case_verification_3.png ├── cost.png ├── extraction_comparison.png └── verification_comparison.png ├── generate_inference_shell_scripts └── generate_infer_compare_shell.py ├── generate_shell_config.py ├── generate_shell_config_matheval.py ├── gpu_scheduler.py ├── inference_matheval.py ├── json_utils.py ├── preprocess_dataset └── build_dataset.py ├── prompt_builder.py ├── prompts ├── extraction_prompts │ ├── examples0.md │ ├── examples1.md │ ├── examples2.md │ ├── examples3.md │ └── instruction.md └── verification_prompts │ ├── examples0.md │ ├── examples1.md │ ├── examples2.md │ └── instruction.md ├── run.py ├── run_gpt4_extraction.py └── run_gpt4_verification.py /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/math-eval/MathEval/HEAD/README.md -------------------------------------------------------------------------------- /compare_with_local_model.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/math-eval/MathEval/HEAD/compare_with_local_model.py -------------------------------------------------------------------------------- /conversation.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/math-eval/MathEval/HEAD/conversation.py -------------------------------------------------------------------------------- /figures/comparison_reported_matheval.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/math-eval/MathEval/HEAD/figures/comparison_reported_matheval.md -------------------------------------------------------------------------------- /figures/corner_case_extraction_0.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/math-eval/MathEval/HEAD/figures/corner_case_extraction_0.png -------------------------------------------------------------------------------- /figures/corner_case_extraction_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/math-eval/MathEval/HEAD/figures/corner_case_extraction_1.png -------------------------------------------------------------------------------- /figures/corner_case_extraction_2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/math-eval/MathEval/HEAD/figures/corner_case_extraction_2.png -------------------------------------------------------------------------------- /figures/corner_case_extraction_3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/math-eval/MathEval/HEAD/figures/corner_case_extraction_3.png -------------------------------------------------------------------------------- /figures/corner_case_verification_0.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/math-eval/MathEval/HEAD/figures/corner_case_verification_0.png -------------------------------------------------------------------------------- /figures/corner_case_verification_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/math-eval/MathEval/HEAD/figures/corner_case_verification_1.png -------------------------------------------------------------------------------- /figures/corner_case_verification_2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/math-eval/MathEval/HEAD/figures/corner_case_verification_2.png -------------------------------------------------------------------------------- /figures/corner_case_verification_3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/math-eval/MathEval/HEAD/figures/corner_case_verification_3.png -------------------------------------------------------------------------------- /figures/cost.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/math-eval/MathEval/HEAD/figures/cost.png -------------------------------------------------------------------------------- /figures/extraction_comparison.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/math-eval/MathEval/HEAD/figures/extraction_comparison.png -------------------------------------------------------------------------------- /figures/verification_comparison.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/math-eval/MathEval/HEAD/figures/verification_comparison.png -------------------------------------------------------------------------------- /generate_inference_shell_scripts/generate_infer_compare_shell.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/math-eval/MathEval/HEAD/generate_inference_shell_scripts/generate_infer_compare_shell.py -------------------------------------------------------------------------------- /generate_shell_config.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/math-eval/MathEval/HEAD/generate_shell_config.py -------------------------------------------------------------------------------- /generate_shell_config_matheval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/math-eval/MathEval/HEAD/generate_shell_config_matheval.py -------------------------------------------------------------------------------- /gpu_scheduler.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/math-eval/MathEval/HEAD/gpu_scheduler.py -------------------------------------------------------------------------------- /inference_matheval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/math-eval/MathEval/HEAD/inference_matheval.py -------------------------------------------------------------------------------- /json_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/math-eval/MathEval/HEAD/json_utils.py -------------------------------------------------------------------------------- /preprocess_dataset/build_dataset.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/math-eval/MathEval/HEAD/preprocess_dataset/build_dataset.py -------------------------------------------------------------------------------- /prompt_builder.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/math-eval/MathEval/HEAD/prompt_builder.py -------------------------------------------------------------------------------- /prompts/extraction_prompts/examples0.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/math-eval/MathEval/HEAD/prompts/extraction_prompts/examples0.md -------------------------------------------------------------------------------- /prompts/extraction_prompts/examples1.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/math-eval/MathEval/HEAD/prompts/extraction_prompts/examples1.md -------------------------------------------------------------------------------- /prompts/extraction_prompts/examples2.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/math-eval/MathEval/HEAD/prompts/extraction_prompts/examples2.md -------------------------------------------------------------------------------- /prompts/extraction_prompts/examples3.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/math-eval/MathEval/HEAD/prompts/extraction_prompts/examples3.md -------------------------------------------------------------------------------- /prompts/extraction_prompts/instruction.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/math-eval/MathEval/HEAD/prompts/extraction_prompts/instruction.md -------------------------------------------------------------------------------- /prompts/verification_prompts/examples0.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/math-eval/MathEval/HEAD/prompts/verification_prompts/examples0.md -------------------------------------------------------------------------------- /prompts/verification_prompts/examples1.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/math-eval/MathEval/HEAD/prompts/verification_prompts/examples1.md -------------------------------------------------------------------------------- /prompts/verification_prompts/examples2.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/math-eval/MathEval/HEAD/prompts/verification_prompts/examples2.md -------------------------------------------------------------------------------- /prompts/verification_prompts/instruction.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/math-eval/MathEval/HEAD/prompts/verification_prompts/instruction.md -------------------------------------------------------------------------------- /run.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/math-eval/MathEval/HEAD/run.py -------------------------------------------------------------------------------- /run_gpt4_extraction.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/math-eval/MathEval/HEAD/run_gpt4_extraction.py -------------------------------------------------------------------------------- /run_gpt4_verification.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/math-eval/MathEval/HEAD/run_gpt4_verification.py --------------------------------------------------------------------------------