├── .gitignore ├── compress.sh ├── config ├── default_hf.json ├── default_openai.json └── ljp_prompt.json ├── download_data.sh ├── lm_eval ├── __init__.py ├── evaluate.py ├── models │ ├── __init__.py │ ├── base.py │ ├── hf_model.py │ └── openai_model.py ├── parser.py ├── tasks │ ├── __init__.py │ ├── base.py │ └── ljp.py └── utils.py ├── main.py ├── readme.md ├── requirements.txt ├── resources ├── paper_version_acc.csv ├── paper_version_f1.csv └── static │ ├── fig_setting_example_v2.jpg │ └── fig_setting_example_v2.pdf ├── run.sh ├── scripts ├── copy_paper_results.sh ├── csv2htlm.py ├── get_result_table.py └── parse_all_output.py └── tests ├── __init__.py ├── ljp_task.py └── model_generate.py /.gitignore: -------------------------------------------------------------------------------- 1 | __pycache__/ 2 | *.env* 3 | wandb/ 4 | outputs/ 5 | data_hub/ 6 | runs/ -------------------------------------------------------------------------------- /compress.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/srhthu/LM-CompEval-Legal/HEAD/compress.sh -------------------------------------------------------------------------------- /config/default_hf.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/srhthu/LM-CompEval-Legal/HEAD/config/default_hf.json -------------------------------------------------------------------------------- /config/default_openai.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/srhthu/LM-CompEval-Legal/HEAD/config/default_openai.json -------------------------------------------------------------------------------- /config/ljp_prompt.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/srhthu/LM-CompEval-Legal/HEAD/config/ljp_prompt.json -------------------------------------------------------------------------------- /download_data.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/srhthu/LM-CompEval-Legal/HEAD/download_data.sh -------------------------------------------------------------------------------- /lm_eval/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/srhthu/LM-CompEval-Legal/HEAD/lm_eval/__init__.py -------------------------------------------------------------------------------- /lm_eval/evaluate.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/srhthu/LM-CompEval-Legal/HEAD/lm_eval/evaluate.py -------------------------------------------------------------------------------- /lm_eval/models/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/srhthu/LM-CompEval-Legal/HEAD/lm_eval/models/__init__.py -------------------------------------------------------------------------------- /lm_eval/models/base.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/srhthu/LM-CompEval-Legal/HEAD/lm_eval/models/base.py -------------------------------------------------------------------------------- /lm_eval/models/hf_model.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/srhthu/LM-CompEval-Legal/HEAD/lm_eval/models/hf_model.py -------------------------------------------------------------------------------- /lm_eval/models/openai_model.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/srhthu/LM-CompEval-Legal/HEAD/lm_eval/models/openai_model.py -------------------------------------------------------------------------------- /lm_eval/parser.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/srhthu/LM-CompEval-Legal/HEAD/lm_eval/parser.py -------------------------------------------------------------------------------- /lm_eval/tasks/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/srhthu/LM-CompEval-Legal/HEAD/lm_eval/tasks/__init__.py -------------------------------------------------------------------------------- /lm_eval/tasks/base.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/srhthu/LM-CompEval-Legal/HEAD/lm_eval/tasks/base.py -------------------------------------------------------------------------------- /lm_eval/tasks/ljp.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/srhthu/LM-CompEval-Legal/HEAD/lm_eval/tasks/ljp.py -------------------------------------------------------------------------------- /lm_eval/utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/srhthu/LM-CompEval-Legal/HEAD/lm_eval/utils.py -------------------------------------------------------------------------------- /main.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/srhthu/LM-CompEval-Legal/HEAD/main.py -------------------------------------------------------------------------------- /readme.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/srhthu/LM-CompEval-Legal/HEAD/readme.md -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/srhthu/LM-CompEval-Legal/HEAD/requirements.txt -------------------------------------------------------------------------------- /resources/paper_version_acc.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/srhthu/LM-CompEval-Legal/HEAD/resources/paper_version_acc.csv -------------------------------------------------------------------------------- /resources/paper_version_f1.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/srhthu/LM-CompEval-Legal/HEAD/resources/paper_version_f1.csv -------------------------------------------------------------------------------- /resources/static/fig_setting_example_v2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/srhthu/LM-CompEval-Legal/HEAD/resources/static/fig_setting_example_v2.jpg -------------------------------------------------------------------------------- /resources/static/fig_setting_example_v2.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/srhthu/LM-CompEval-Legal/HEAD/resources/static/fig_setting_example_v2.pdf -------------------------------------------------------------------------------- /run.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/srhthu/LM-CompEval-Legal/HEAD/run.sh -------------------------------------------------------------------------------- /scripts/copy_paper_results.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/srhthu/LM-CompEval-Legal/HEAD/scripts/copy_paper_results.sh -------------------------------------------------------------------------------- /scripts/csv2htlm.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/srhthu/LM-CompEval-Legal/HEAD/scripts/csv2htlm.py -------------------------------------------------------------------------------- /scripts/get_result_table.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/srhthu/LM-CompEval-Legal/HEAD/scripts/get_result_table.py -------------------------------------------------------------------------------- /scripts/parse_all_output.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/srhthu/LM-CompEval-Legal/HEAD/scripts/parse_all_output.py -------------------------------------------------------------------------------- /tests/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /tests/ljp_task.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/srhthu/LM-CompEval-Legal/HEAD/tests/ljp_task.py -------------------------------------------------------------------------------- /tests/model_generate.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/srhthu/LM-CompEval-Legal/HEAD/tests/model_generate.py --------------------------------------------------------------------------------