├── .gitignore ├── LICENSE ├── README.md ├── evaluation_scripts ├── evaluate_multirc_predictions.py ├── evaluate_openbookqa_predictions.py ├── lib │ ├── __init__.py │ └── multirc_measures.py ├── multirc_predictions_to_official_format.py └── openbookqa_predictions_to_official_format.py ├── experiment_configs ├── final_esim_glove_snli_multinli_for_multirc.json ├── final_esim_glove_snli_multinli_for_openbookqa.json ├── final_multee_glove_multirc.json └── final_multee_glove_openbookqa.json ├── images ├── main-diagram-with-caption.png ├── main-diagram.png └── multee-icon.png ├── lib ├── __init__.py ├── commands │ └── predict_with_vocab_expansion.py ├── dataset_readers │ ├── __init__.py │ ├── entailment_pair.py │ ├── multiple_correct_mcq_entailment.py │ └── single_correct_mcq_entailment.py ├── models │ ├── __init__.py │ ├── multee_esim.py │ ├── multiple_correct_mcq_multee_esim.py │ └── single_correct_mcq_multee_esim.py ├── modules │ ├── __init__.py │ ├── coverage_loss.py │ ├── each_output_timedistributed.py │ └── esim_comparator.py ├── nn │ ├── __init__.py │ └── util.py └── predictors │ ├── __init__.py │ ├── entailment_pair.py │ ├── multiple_correct_mcq_entailment.py │ └── single_correct_mcq_entailment.py ├── predictions └── .keep ├── preprocessing ├── combine_snli_multinli.sh ├── construct_answerwise_multirc.py ├── lib │ ├── __init__.py │ ├── coref_replace.py │ ├── question_answer_to_explicit_hypothesis.py │ └── question_to_implicit_hypothesis.py ├── preprocess_multirc.py ├── preprocess_openbookqa.py └── rank_openbookqa_knowledge_sources.sh ├── requirements.txt ├── run.py ├── run_tests.py ├── scripts ├── download_trained_models.sh ├── install_requirements.sh └── setup_data.sh ├── tests ├── dataset_readers │ ├── multiple_correct_mcq_entailment_test.py │ └── single_correct_mcq_entailment_test.py ├── fixtures │ ├── datasets │ │ ├── multiple-correct-mcq-entailment-dataset-fixture.jsonl │ │ └── single-correct-mcq-entailment-dataset-fixture.jsonl │ └── experiment_configs │ │ ├── multiple_correct_mcq_multee.json │ │ └── single_correct_mcq_multee.json ├── models │ ├── multiple_correct_mcq_multee_esim_test.py │ └── single_correct_mcq_multee_esim_test.py └── nn │ └── util_test.py ├── vocab_for_multirc ├── labels.txt ├── non_padded_namespaces.txt └── tokens.txt └── vocab_for_openbookqa ├── labels.txt ├── non_padded_namespaces.txt └── tokens.txt /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/.gitignore -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/README.md -------------------------------------------------------------------------------- /evaluation_scripts/evaluate_multirc_predictions.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/evaluation_scripts/evaluate_multirc_predictions.py -------------------------------------------------------------------------------- /evaluation_scripts/evaluate_openbookqa_predictions.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/evaluation_scripts/evaluate_openbookqa_predictions.py -------------------------------------------------------------------------------- /evaluation_scripts/lib/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /evaluation_scripts/lib/multirc_measures.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/evaluation_scripts/lib/multirc_measures.py -------------------------------------------------------------------------------- /evaluation_scripts/multirc_predictions_to_official_format.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/evaluation_scripts/multirc_predictions_to_official_format.py -------------------------------------------------------------------------------- /evaluation_scripts/openbookqa_predictions_to_official_format.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/evaluation_scripts/openbookqa_predictions_to_official_format.py -------------------------------------------------------------------------------- /experiment_configs/final_esim_glove_snli_multinli_for_multirc.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/experiment_configs/final_esim_glove_snli_multinli_for_multirc.json -------------------------------------------------------------------------------- /experiment_configs/final_esim_glove_snli_multinli_for_openbookqa.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/experiment_configs/final_esim_glove_snli_multinli_for_openbookqa.json -------------------------------------------------------------------------------- /experiment_configs/final_multee_glove_multirc.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/experiment_configs/final_multee_glove_multirc.json -------------------------------------------------------------------------------- /experiment_configs/final_multee_glove_openbookqa.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/experiment_configs/final_multee_glove_openbookqa.json -------------------------------------------------------------------------------- /images/main-diagram-with-caption.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/images/main-diagram-with-caption.png -------------------------------------------------------------------------------- /images/main-diagram.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/images/main-diagram.png -------------------------------------------------------------------------------- /images/multee-icon.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/images/multee-icon.png -------------------------------------------------------------------------------- /lib/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /lib/commands/predict_with_vocab_expansion.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/lib/commands/predict_with_vocab_expansion.py -------------------------------------------------------------------------------- /lib/dataset_readers/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/lib/dataset_readers/__init__.py -------------------------------------------------------------------------------- /lib/dataset_readers/entailment_pair.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/lib/dataset_readers/entailment_pair.py -------------------------------------------------------------------------------- /lib/dataset_readers/multiple_correct_mcq_entailment.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/lib/dataset_readers/multiple_correct_mcq_entailment.py -------------------------------------------------------------------------------- /lib/dataset_readers/single_correct_mcq_entailment.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/lib/dataset_readers/single_correct_mcq_entailment.py -------------------------------------------------------------------------------- /lib/models/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/lib/models/__init__.py -------------------------------------------------------------------------------- /lib/models/multee_esim.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/lib/models/multee_esim.py -------------------------------------------------------------------------------- /lib/models/multiple_correct_mcq_multee_esim.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/lib/models/multiple_correct_mcq_multee_esim.py -------------------------------------------------------------------------------- /lib/models/single_correct_mcq_multee_esim.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/lib/models/single_correct_mcq_multee_esim.py -------------------------------------------------------------------------------- /lib/modules/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/lib/modules/__init__.py -------------------------------------------------------------------------------- /lib/modules/coverage_loss.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/lib/modules/coverage_loss.py -------------------------------------------------------------------------------- /lib/modules/each_output_timedistributed.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/lib/modules/each_output_timedistributed.py -------------------------------------------------------------------------------- /lib/modules/esim_comparator.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/lib/modules/esim_comparator.py -------------------------------------------------------------------------------- /lib/nn/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /lib/nn/util.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/lib/nn/util.py -------------------------------------------------------------------------------- /lib/predictors/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/lib/predictors/__init__.py -------------------------------------------------------------------------------- /lib/predictors/entailment_pair.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/lib/predictors/entailment_pair.py -------------------------------------------------------------------------------- /lib/predictors/multiple_correct_mcq_entailment.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/lib/predictors/multiple_correct_mcq_entailment.py -------------------------------------------------------------------------------- /lib/predictors/single_correct_mcq_entailment.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/lib/predictors/single_correct_mcq_entailment.py -------------------------------------------------------------------------------- /predictions/.keep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /preprocessing/combine_snli_multinli.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/preprocessing/combine_snli_multinli.sh -------------------------------------------------------------------------------- /preprocessing/construct_answerwise_multirc.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/preprocessing/construct_answerwise_multirc.py -------------------------------------------------------------------------------- /preprocessing/lib/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /preprocessing/lib/coref_replace.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/preprocessing/lib/coref_replace.py -------------------------------------------------------------------------------- /preprocessing/lib/question_answer_to_explicit_hypothesis.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/preprocessing/lib/question_answer_to_explicit_hypothesis.py -------------------------------------------------------------------------------- /preprocessing/lib/question_to_implicit_hypothesis.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/preprocessing/lib/question_to_implicit_hypothesis.py -------------------------------------------------------------------------------- /preprocessing/preprocess_multirc.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/preprocessing/preprocess_multirc.py -------------------------------------------------------------------------------- /preprocessing/preprocess_openbookqa.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/preprocessing/preprocess_openbookqa.py -------------------------------------------------------------------------------- /preprocessing/rank_openbookqa_knowledge_sources.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/preprocessing/rank_openbookqa_knowledge_sources.sh -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | allennlp==0.8.4 2 | spacy==2.1.0 3 | neuralcoref==4.0 4 | -------------------------------------------------------------------------------- /run.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/run.py -------------------------------------------------------------------------------- /run_tests.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/run_tests.py -------------------------------------------------------------------------------- /scripts/download_trained_models.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/scripts/download_trained_models.sh -------------------------------------------------------------------------------- /scripts/install_requirements.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/scripts/install_requirements.sh -------------------------------------------------------------------------------- /scripts/setup_data.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/scripts/setup_data.sh -------------------------------------------------------------------------------- /tests/dataset_readers/multiple_correct_mcq_entailment_test.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/tests/dataset_readers/multiple_correct_mcq_entailment_test.py -------------------------------------------------------------------------------- /tests/dataset_readers/single_correct_mcq_entailment_test.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/tests/dataset_readers/single_correct_mcq_entailment_test.py -------------------------------------------------------------------------------- /tests/fixtures/datasets/multiple-correct-mcq-entailment-dataset-fixture.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/tests/fixtures/datasets/multiple-correct-mcq-entailment-dataset-fixture.jsonl -------------------------------------------------------------------------------- /tests/fixtures/datasets/single-correct-mcq-entailment-dataset-fixture.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/tests/fixtures/datasets/single-correct-mcq-entailment-dataset-fixture.jsonl -------------------------------------------------------------------------------- /tests/fixtures/experiment_configs/multiple_correct_mcq_multee.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/tests/fixtures/experiment_configs/multiple_correct_mcq_multee.json -------------------------------------------------------------------------------- /tests/fixtures/experiment_configs/single_correct_mcq_multee.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/tests/fixtures/experiment_configs/single_correct_mcq_multee.json -------------------------------------------------------------------------------- /tests/models/multiple_correct_mcq_multee_esim_test.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/tests/models/multiple_correct_mcq_multee_esim_test.py -------------------------------------------------------------------------------- /tests/models/single_correct_mcq_multee_esim_test.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/tests/models/single_correct_mcq_multee_esim_test.py -------------------------------------------------------------------------------- /tests/nn/util_test.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/tests/nn/util_test.py -------------------------------------------------------------------------------- /vocab_for_multirc/labels.txt: -------------------------------------------------------------------------------- 1 | entailment 2 | contradiction 3 | neutral 4 | -------------------------------------------------------------------------------- /vocab_for_multirc/non_padded_namespaces.txt: -------------------------------------------------------------------------------- 1 | *labels 2 | *tags 3 | -------------------------------------------------------------------------------- /vocab_for_multirc/tokens.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/vocab_for_multirc/tokens.txt -------------------------------------------------------------------------------- /vocab_for_openbookqa/labels.txt: -------------------------------------------------------------------------------- 1 | entailment 2 | contradiction 3 | neutral 4 | -------------------------------------------------------------------------------- /vocab_for_openbookqa/non_padded_namespaces.txt: -------------------------------------------------------------------------------- 1 | *tags 2 | *labels 3 | -------------------------------------------------------------------------------- /vocab_for_openbookqa/tokens.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/StonyBrookNLP/multee/HEAD/vocab_for_openbookqa/tokens.txt --------------------------------------------------------------------------------