├── CODE_OF_CONDUCT.md ├── CONTRIBUTING.md ├── LICENSE ├── NOTICE ├── README.md ├── code ├── arguments.py ├── compute_correlation.py ├── data_processors.py ├── evaluate_pair_responses.py ├── generate_responses.py ├── models.py ├── report_results.py ├── template.py └── utils.py ├── data ├── annotations_fiqa_with_citation.jsonl ├── annotations_lifestyle_with_citation.jsonl ├── annotations_recreation_with_citation.jsonl ├── annotations_science_with_citation.jsonl ├── annotations_technology_with_citation.jsonl └── annotations_writing_with_citation.jsonl ├── generate_response_gpt.sh ├── generate_response_hf.sh ├── requirements.txt ├── run_pairwise_eval_lfrqa.sh ├── run_pairwise_eval_llms.sh └── templates ├── ans_generation_v1.cfg ├── ans_generation_v2.cfg ├── pairwise_lfrqa.cfg ├── pairwise_lfrqa_examples.json └── pairwise_lfrqa_system.txt /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/awslabs/rag-qa-arena/HEAD/CODE_OF_CONDUCT.md -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/awslabs/rag-qa-arena/HEAD/CONTRIBUTING.md -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/awslabs/rag-qa-arena/HEAD/LICENSE -------------------------------------------------------------------------------- /NOTICE: -------------------------------------------------------------------------------- 1 | Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/awslabs/rag-qa-arena/HEAD/README.md -------------------------------------------------------------------------------- /code/arguments.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/awslabs/rag-qa-arena/HEAD/code/arguments.py -------------------------------------------------------------------------------- /code/compute_correlation.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/awslabs/rag-qa-arena/HEAD/code/compute_correlation.py -------------------------------------------------------------------------------- /code/data_processors.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/awslabs/rag-qa-arena/HEAD/code/data_processors.py -------------------------------------------------------------------------------- /code/evaluate_pair_responses.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/awslabs/rag-qa-arena/HEAD/code/evaluate_pair_responses.py -------------------------------------------------------------------------------- /code/generate_responses.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/awslabs/rag-qa-arena/HEAD/code/generate_responses.py -------------------------------------------------------------------------------- /code/models.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/awslabs/rag-qa-arena/HEAD/code/models.py -------------------------------------------------------------------------------- /code/report_results.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/awslabs/rag-qa-arena/HEAD/code/report_results.py -------------------------------------------------------------------------------- /code/template.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/awslabs/rag-qa-arena/HEAD/code/template.py -------------------------------------------------------------------------------- /code/utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/awslabs/rag-qa-arena/HEAD/code/utils.py -------------------------------------------------------------------------------- /data/annotations_fiqa_with_citation.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/awslabs/rag-qa-arena/HEAD/data/annotations_fiqa_with_citation.jsonl -------------------------------------------------------------------------------- /data/annotations_lifestyle_with_citation.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/awslabs/rag-qa-arena/HEAD/data/annotations_lifestyle_with_citation.jsonl -------------------------------------------------------------------------------- /data/annotations_recreation_with_citation.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/awslabs/rag-qa-arena/HEAD/data/annotations_recreation_with_citation.jsonl -------------------------------------------------------------------------------- /data/annotations_science_with_citation.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/awslabs/rag-qa-arena/HEAD/data/annotations_science_with_citation.jsonl -------------------------------------------------------------------------------- /data/annotations_technology_with_citation.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/awslabs/rag-qa-arena/HEAD/data/annotations_technology_with_citation.jsonl -------------------------------------------------------------------------------- /data/annotations_writing_with_citation.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/awslabs/rag-qa-arena/HEAD/data/annotations_writing_with_citation.jsonl -------------------------------------------------------------------------------- /generate_response_gpt.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/awslabs/rag-qa-arena/HEAD/generate_response_gpt.sh -------------------------------------------------------------------------------- /generate_response_hf.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/awslabs/rag-qa-arena/HEAD/generate_response_hf.sh -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/awslabs/rag-qa-arena/HEAD/requirements.txt -------------------------------------------------------------------------------- /run_pairwise_eval_lfrqa.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/awslabs/rag-qa-arena/HEAD/run_pairwise_eval_lfrqa.sh -------------------------------------------------------------------------------- /run_pairwise_eval_llms.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/awslabs/rag-qa-arena/HEAD/run_pairwise_eval_llms.sh -------------------------------------------------------------------------------- /templates/ans_generation_v1.cfg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/awslabs/rag-qa-arena/HEAD/templates/ans_generation_v1.cfg -------------------------------------------------------------------------------- /templates/ans_generation_v2.cfg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/awslabs/rag-qa-arena/HEAD/templates/ans_generation_v2.cfg -------------------------------------------------------------------------------- /templates/pairwise_lfrqa.cfg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/awslabs/rag-qa-arena/HEAD/templates/pairwise_lfrqa.cfg -------------------------------------------------------------------------------- /templates/pairwise_lfrqa_examples.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/awslabs/rag-qa-arena/HEAD/templates/pairwise_lfrqa_examples.json -------------------------------------------------------------------------------- /templates/pairwise_lfrqa_system.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/awslabs/rag-qa-arena/HEAD/templates/pairwise_lfrqa_system.txt --------------------------------------------------------------------------------