├── .gitignore ├── LICENSE ├── README.md ├── USE_POLICY.md ├── assets └── q_eg.png ├── configs ├── few-shot.yaml ├── few-shot_cot.yaml ├── few-shot_cot_sc.yaml ├── few-shot_sc.yaml ├── few-shot_sys.yaml ├── few-shot_sys_cot.yaml ├── few-shot_tot.yaml ├── zero-shot.yaml ├── zero-shot_cot.yaml ├── zero-shot_cot_sc.yaml ├── zero-shot_sc.yaml ├── zero-shot_sys.yaml ├── zero-shot_sys_cot.yaml └── zero-shot_tot.yaml ├── environment.yml ├── evaluate.py ├── prompts ├── __init__.py ├── qa_prompt.py ├── sys_prompt.py └── tot_prompt.py ├── requirements.txt ├── run.py ├── scripts ├── run.sh ├── run_ctx.sh ├── run_os.sh └── serving.sh ├── solve ├── __init__.py └── solver.py ├── tokenizer.model ├── tokenizer_checklist.chk └── utils ├── __init__.py ├── arg_parser.py ├── tot_solver.py └── utils.py /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LinxinS97/NLPBench/HEAD/.gitignore -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LinxinS97/NLPBench/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LinxinS97/NLPBench/HEAD/README.md -------------------------------------------------------------------------------- /USE_POLICY.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LinxinS97/NLPBench/HEAD/USE_POLICY.md -------------------------------------------------------------------------------- /assets/q_eg.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LinxinS97/NLPBench/HEAD/assets/q_eg.png -------------------------------------------------------------------------------- /configs/few-shot.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LinxinS97/NLPBench/HEAD/configs/few-shot.yaml -------------------------------------------------------------------------------- /configs/few-shot_cot.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LinxinS97/NLPBench/HEAD/configs/few-shot_cot.yaml -------------------------------------------------------------------------------- /configs/few-shot_cot_sc.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LinxinS97/NLPBench/HEAD/configs/few-shot_cot_sc.yaml -------------------------------------------------------------------------------- /configs/few-shot_sc.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LinxinS97/NLPBench/HEAD/configs/few-shot_sc.yaml -------------------------------------------------------------------------------- /configs/few-shot_sys.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LinxinS97/NLPBench/HEAD/configs/few-shot_sys.yaml -------------------------------------------------------------------------------- /configs/few-shot_sys_cot.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LinxinS97/NLPBench/HEAD/configs/few-shot_sys_cot.yaml -------------------------------------------------------------------------------- /configs/few-shot_tot.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LinxinS97/NLPBench/HEAD/configs/few-shot_tot.yaml -------------------------------------------------------------------------------- /configs/zero-shot.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LinxinS97/NLPBench/HEAD/configs/zero-shot.yaml -------------------------------------------------------------------------------- /configs/zero-shot_cot.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LinxinS97/NLPBench/HEAD/configs/zero-shot_cot.yaml -------------------------------------------------------------------------------- /configs/zero-shot_cot_sc.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LinxinS97/NLPBench/HEAD/configs/zero-shot_cot_sc.yaml -------------------------------------------------------------------------------- /configs/zero-shot_sc.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LinxinS97/NLPBench/HEAD/configs/zero-shot_sc.yaml -------------------------------------------------------------------------------- /configs/zero-shot_sys.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LinxinS97/NLPBench/HEAD/configs/zero-shot_sys.yaml -------------------------------------------------------------------------------- /configs/zero-shot_sys_cot.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LinxinS97/NLPBench/HEAD/configs/zero-shot_sys_cot.yaml -------------------------------------------------------------------------------- /configs/zero-shot_tot.yaml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LinxinS97/NLPBench/HEAD/configs/zero-shot_tot.yaml -------------------------------------------------------------------------------- /environment.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LinxinS97/NLPBench/HEAD/environment.yml -------------------------------------------------------------------------------- /evaluate.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LinxinS97/NLPBench/HEAD/evaluate.py -------------------------------------------------------------------------------- /prompts/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LinxinS97/NLPBench/HEAD/prompts/__init__.py -------------------------------------------------------------------------------- /prompts/qa_prompt.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LinxinS97/NLPBench/HEAD/prompts/qa_prompt.py -------------------------------------------------------------------------------- /prompts/sys_prompt.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LinxinS97/NLPBench/HEAD/prompts/sys_prompt.py -------------------------------------------------------------------------------- /prompts/tot_prompt.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LinxinS97/NLPBench/HEAD/prompts/tot_prompt.py -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LinxinS97/NLPBench/HEAD/requirements.txt -------------------------------------------------------------------------------- /run.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LinxinS97/NLPBench/HEAD/run.py -------------------------------------------------------------------------------- /scripts/run.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LinxinS97/NLPBench/HEAD/scripts/run.sh -------------------------------------------------------------------------------- /scripts/run_ctx.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LinxinS97/NLPBench/HEAD/scripts/run_ctx.sh -------------------------------------------------------------------------------- /scripts/run_os.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LinxinS97/NLPBench/HEAD/scripts/run_os.sh -------------------------------------------------------------------------------- /scripts/serving.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LinxinS97/NLPBench/HEAD/scripts/serving.sh -------------------------------------------------------------------------------- /solve/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LinxinS97/NLPBench/HEAD/solve/__init__.py -------------------------------------------------------------------------------- /solve/solver.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LinxinS97/NLPBench/HEAD/solve/solver.py -------------------------------------------------------------------------------- /tokenizer.model: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LinxinS97/NLPBench/HEAD/tokenizer.model -------------------------------------------------------------------------------- /tokenizer_checklist.chk: -------------------------------------------------------------------------------- 1 | eeec4125e9c7560836b4873b6f8e3025 tokenizer.model 2 | -------------------------------------------------------------------------------- /utils/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LinxinS97/NLPBench/HEAD/utils/__init__.py -------------------------------------------------------------------------------- /utils/arg_parser.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LinxinS97/NLPBench/HEAD/utils/arg_parser.py -------------------------------------------------------------------------------- /utils/tot_solver.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LinxinS97/NLPBench/HEAD/utils/tot_solver.py -------------------------------------------------------------------------------- /utils/utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LinxinS97/NLPBench/HEAD/utils/utils.py --------------------------------------------------------------------------------