├── .gitignore ├── CITATION ├── Evaluation ├── automatic_eval.py ├── automatic_metrics.py ├── llm_eval.py └── llm_score.py ├── LICENSE ├── LooGLE-testdata ├── longdep_qa.jsonl ├── longdep_summarization.jsonl ├── shortdep_cloze.jsonl └── shortdep_qa.jsonl ├── Output ├── longdep_qa_reorder_gpt4-32k.jsonl ├── longdep_summarization_llama-index.jsonl ├── shortdep_qa_chatglm2-6b-32k.jsonl ├── shortdep_qa_gpt-3.5-turbo-16k.jsonl └── shortdep_qa_llama-index.jsonl ├── Prediction ├── pred_gpt_models.py ├── pred_llamaindex.py └── pred_opensource_models.py ├── README.md ├── Reorder ├── get_max_deviation.py ├── get_reorder_deviation.py └── reorder_eval.py ├── Retrieval └── pred_retrieval_based_method.py ├── Tools ├── Labeling_form_A_annotator.html └── Labeling_form_Q_annotator.html ├── assets ├── LooGle_logo.png ├── overview.png ├── overview_page1.png ├── overview_performance.png └── table.png ├── config ├── task2maxlen.json └── task2prompt.json ├── docs ├── .gitignore ├── 404.html ├── Gemfile ├── _config.yml ├── _includes │ └── head-custom.html ├── _layouts │ └── default.html ├── assets │ ├── css │ │ └── style.scss │ └── js │ │ └── main.js ├── backup.html └── index.md └── requirements.txt /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/.gitignore -------------------------------------------------------------------------------- /CITATION: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/CITATION -------------------------------------------------------------------------------- /Evaluation/automatic_eval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/Evaluation/automatic_eval.py -------------------------------------------------------------------------------- /Evaluation/automatic_metrics.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/Evaluation/automatic_metrics.py -------------------------------------------------------------------------------- /Evaluation/llm_eval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/Evaluation/llm_eval.py -------------------------------------------------------------------------------- /Evaluation/llm_score.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/Evaluation/llm_score.py -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/LICENSE -------------------------------------------------------------------------------- /LooGLE-testdata/longdep_qa.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/LooGLE-testdata/longdep_qa.jsonl -------------------------------------------------------------------------------- /LooGLE-testdata/longdep_summarization.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/LooGLE-testdata/longdep_summarization.jsonl -------------------------------------------------------------------------------- /LooGLE-testdata/shortdep_cloze.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/LooGLE-testdata/shortdep_cloze.jsonl -------------------------------------------------------------------------------- /LooGLE-testdata/shortdep_qa.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/LooGLE-testdata/shortdep_qa.jsonl -------------------------------------------------------------------------------- /Output/longdep_qa_reorder_gpt4-32k.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/Output/longdep_qa_reorder_gpt4-32k.jsonl -------------------------------------------------------------------------------- /Output/longdep_summarization_llama-index.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/Output/longdep_summarization_llama-index.jsonl -------------------------------------------------------------------------------- /Output/shortdep_qa_chatglm2-6b-32k.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/Output/shortdep_qa_chatglm2-6b-32k.jsonl -------------------------------------------------------------------------------- /Output/shortdep_qa_gpt-3.5-turbo-16k.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/Output/shortdep_qa_gpt-3.5-turbo-16k.jsonl -------------------------------------------------------------------------------- /Output/shortdep_qa_llama-index.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/Output/shortdep_qa_llama-index.jsonl -------------------------------------------------------------------------------- /Prediction/pred_gpt_models.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/Prediction/pred_gpt_models.py -------------------------------------------------------------------------------- /Prediction/pred_llamaindex.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/Prediction/pred_llamaindex.py -------------------------------------------------------------------------------- /Prediction/pred_opensource_models.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/Prediction/pred_opensource_models.py -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/README.md -------------------------------------------------------------------------------- /Reorder/get_max_deviation.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/Reorder/get_max_deviation.py -------------------------------------------------------------------------------- /Reorder/get_reorder_deviation.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/Reorder/get_reorder_deviation.py -------------------------------------------------------------------------------- /Reorder/reorder_eval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/Reorder/reorder_eval.py -------------------------------------------------------------------------------- /Retrieval/pred_retrieval_based_method.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/Retrieval/pred_retrieval_based_method.py -------------------------------------------------------------------------------- /Tools/Labeling_form_A_annotator.html: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/Tools/Labeling_form_A_annotator.html -------------------------------------------------------------------------------- /Tools/Labeling_form_Q_annotator.html: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/Tools/Labeling_form_Q_annotator.html -------------------------------------------------------------------------------- /assets/LooGle_logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/assets/LooGle_logo.png -------------------------------------------------------------------------------- /assets/overview.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/assets/overview.png -------------------------------------------------------------------------------- /assets/overview_page1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/assets/overview_page1.png -------------------------------------------------------------------------------- /assets/overview_performance.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/assets/overview_performance.png -------------------------------------------------------------------------------- /assets/table.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/assets/table.png -------------------------------------------------------------------------------- /config/task2maxlen.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/config/task2maxlen.json -------------------------------------------------------------------------------- /config/task2prompt.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/config/task2prompt.json -------------------------------------------------------------------------------- /docs/.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/docs/.gitignore -------------------------------------------------------------------------------- /docs/404.html: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/docs/404.html -------------------------------------------------------------------------------- /docs/Gemfile: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/docs/Gemfile -------------------------------------------------------------------------------- /docs/_config.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/docs/_config.yml -------------------------------------------------------------------------------- /docs/_includes/head-custom.html: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/docs/_includes/head-custom.html -------------------------------------------------------------------------------- /docs/_layouts/default.html: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/docs/_layouts/default.html -------------------------------------------------------------------------------- /docs/assets/css/style.scss: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/docs/assets/css/style.scss -------------------------------------------------------------------------------- /docs/assets/js/main.js: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/docs/assets/js/main.js -------------------------------------------------------------------------------- /docs/backup.html: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/docs/backup.html -------------------------------------------------------------------------------- /docs/index.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/docs/index.md -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bigai-nlco/LooGLE/HEAD/requirements.txt --------------------------------------------------------------------------------