├── .gitignore ├── LICENSE ├── README.md ├── assets └── longmemeval_examples.png ├── data └── custom_history │ └── sample_haystack_and_timestamp.py ├── requirements-full.txt ├── requirements-lite.txt └── src ├── evaluation ├── evaluate_qa.py ├── print_qa_metrics.py └── print_retrieval_metrics.py ├── generation ├── run_generation.py └── run_generation.sh ├── index_expansion ├── batch_expansion_session_keyphrases.py ├── batch_expansion_session_summ.py ├── batch_expansion_session_temp_event.py ├── batch_expansion_session_userfact.py ├── batch_expansion_turn_keyphrases.py ├── batch_expansion_turn_userfact.py └── temp_query_search_pruning.py ├── retrieval ├── eval_utils.py ├── index_expansion_utils.py ├── run_retrieval.py └── run_retrieval.sh └── utils ├── serve_vllm.sh └── serve_vllm_with_maxlen.sh /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xiaowu0162/LongMemEval/HEAD/.gitignore -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xiaowu0162/LongMemEval/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xiaowu0162/LongMemEval/HEAD/README.md -------------------------------------------------------------------------------- /assets/longmemeval_examples.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xiaowu0162/LongMemEval/HEAD/assets/longmemeval_examples.png -------------------------------------------------------------------------------- /data/custom_history/sample_haystack_and_timestamp.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xiaowu0162/LongMemEval/HEAD/data/custom_history/sample_haystack_and_timestamp.py -------------------------------------------------------------------------------- /requirements-full.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xiaowu0162/LongMemEval/HEAD/requirements-full.txt -------------------------------------------------------------------------------- /requirements-lite.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xiaowu0162/LongMemEval/HEAD/requirements-lite.txt -------------------------------------------------------------------------------- /src/evaluation/evaluate_qa.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xiaowu0162/LongMemEval/HEAD/src/evaluation/evaluate_qa.py -------------------------------------------------------------------------------- /src/evaluation/print_qa_metrics.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xiaowu0162/LongMemEval/HEAD/src/evaluation/print_qa_metrics.py -------------------------------------------------------------------------------- /src/evaluation/print_retrieval_metrics.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xiaowu0162/LongMemEval/HEAD/src/evaluation/print_retrieval_metrics.py -------------------------------------------------------------------------------- /src/generation/run_generation.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xiaowu0162/LongMemEval/HEAD/src/generation/run_generation.py -------------------------------------------------------------------------------- /src/generation/run_generation.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xiaowu0162/LongMemEval/HEAD/src/generation/run_generation.sh -------------------------------------------------------------------------------- /src/index_expansion/batch_expansion_session_keyphrases.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xiaowu0162/LongMemEval/HEAD/src/index_expansion/batch_expansion_session_keyphrases.py -------------------------------------------------------------------------------- /src/index_expansion/batch_expansion_session_summ.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xiaowu0162/LongMemEval/HEAD/src/index_expansion/batch_expansion_session_summ.py -------------------------------------------------------------------------------- /src/index_expansion/batch_expansion_session_temp_event.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xiaowu0162/LongMemEval/HEAD/src/index_expansion/batch_expansion_session_temp_event.py -------------------------------------------------------------------------------- /src/index_expansion/batch_expansion_session_userfact.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xiaowu0162/LongMemEval/HEAD/src/index_expansion/batch_expansion_session_userfact.py -------------------------------------------------------------------------------- /src/index_expansion/batch_expansion_turn_keyphrases.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xiaowu0162/LongMemEval/HEAD/src/index_expansion/batch_expansion_turn_keyphrases.py -------------------------------------------------------------------------------- /src/index_expansion/batch_expansion_turn_userfact.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xiaowu0162/LongMemEval/HEAD/src/index_expansion/batch_expansion_turn_userfact.py -------------------------------------------------------------------------------- /src/index_expansion/temp_query_search_pruning.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xiaowu0162/LongMemEval/HEAD/src/index_expansion/temp_query_search_pruning.py -------------------------------------------------------------------------------- /src/retrieval/eval_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xiaowu0162/LongMemEval/HEAD/src/retrieval/eval_utils.py -------------------------------------------------------------------------------- /src/retrieval/index_expansion_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xiaowu0162/LongMemEval/HEAD/src/retrieval/index_expansion_utils.py -------------------------------------------------------------------------------- /src/retrieval/run_retrieval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xiaowu0162/LongMemEval/HEAD/src/retrieval/run_retrieval.py -------------------------------------------------------------------------------- /src/retrieval/run_retrieval.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xiaowu0162/LongMemEval/HEAD/src/retrieval/run_retrieval.sh -------------------------------------------------------------------------------- /src/utils/serve_vllm.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xiaowu0162/LongMemEval/HEAD/src/utils/serve_vllm.sh -------------------------------------------------------------------------------- /src/utils/serve_vllm_with_maxlen.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xiaowu0162/LongMemEval/HEAD/src/utils/serve_vllm_with_maxlen.sh --------------------------------------------------------------------------------