├── .gitignore ├── LICENSE ├── README.md ├── analysis ├── compute_savings.ipynb ├── deletion_pattern_analysis.ipynb ├── figures │ └── macs_relative.pdf └── get_deletion_patterns.py ├── data ├── data_collator_finetuning.py ├── data_collator_for_t5_mlm.py ├── preprocess_char_dataset.py ├── preprocess_diagnostic_dataset.py └── preprocess_lm_dataset.py ├── eval ├── character_task_eval.py ├── diagnostic_task_eval.ipynb ├── diagnostic_task_eval.py ├── figures │ ├── gate_placement_analysis_pctrl50.pdf │ ├── span_corruption_per_sample_mrt5.pdf │ └── span_corruption_per_sample_randomt5.pdf ├── finetune_task_eval.ipynb ├── gate_placement_analysis.ipynb ├── qa_eval.py ├── span_corruption_eval.ipynb ├── span_corruption_eval.py ├── span_corruption_per_sample_eval.ipynb └── xnli_eval.py ├── icons └── MrT5.png ├── models ├── configuration_mrt5.py ├── model_upload.py ├── modeling_bpt5.py ├── modeling_canine.py ├── modeling_mrt5.py ├── modeling_t5.py └── shortening.py ├── requirements.txt ├── training ├── train.py └── trainer.py └── utils.py /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jkallini/mrt5/HEAD/.gitignore -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jkallini/mrt5/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jkallini/mrt5/HEAD/README.md -------------------------------------------------------------------------------- /analysis/compute_savings.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jkallini/mrt5/HEAD/analysis/compute_savings.ipynb -------------------------------------------------------------------------------- /analysis/deletion_pattern_analysis.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jkallini/mrt5/HEAD/analysis/deletion_pattern_analysis.ipynb -------------------------------------------------------------------------------- /analysis/figures/macs_relative.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jkallini/mrt5/HEAD/analysis/figures/macs_relative.pdf -------------------------------------------------------------------------------- /analysis/get_deletion_patterns.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jkallini/mrt5/HEAD/analysis/get_deletion_patterns.py -------------------------------------------------------------------------------- /data/data_collator_finetuning.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jkallini/mrt5/HEAD/data/data_collator_finetuning.py -------------------------------------------------------------------------------- /data/data_collator_for_t5_mlm.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jkallini/mrt5/HEAD/data/data_collator_for_t5_mlm.py -------------------------------------------------------------------------------- /data/preprocess_char_dataset.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jkallini/mrt5/HEAD/data/preprocess_char_dataset.py -------------------------------------------------------------------------------- /data/preprocess_diagnostic_dataset.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jkallini/mrt5/HEAD/data/preprocess_diagnostic_dataset.py -------------------------------------------------------------------------------- /data/preprocess_lm_dataset.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jkallini/mrt5/HEAD/data/preprocess_lm_dataset.py -------------------------------------------------------------------------------- /eval/character_task_eval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jkallini/mrt5/HEAD/eval/character_task_eval.py -------------------------------------------------------------------------------- /eval/diagnostic_task_eval.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jkallini/mrt5/HEAD/eval/diagnostic_task_eval.ipynb -------------------------------------------------------------------------------- /eval/diagnostic_task_eval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jkallini/mrt5/HEAD/eval/diagnostic_task_eval.py -------------------------------------------------------------------------------- /eval/figures/gate_placement_analysis_pctrl50.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jkallini/mrt5/HEAD/eval/figures/gate_placement_analysis_pctrl50.pdf -------------------------------------------------------------------------------- /eval/figures/span_corruption_per_sample_mrt5.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jkallini/mrt5/HEAD/eval/figures/span_corruption_per_sample_mrt5.pdf -------------------------------------------------------------------------------- /eval/figures/span_corruption_per_sample_randomt5.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jkallini/mrt5/HEAD/eval/figures/span_corruption_per_sample_randomt5.pdf -------------------------------------------------------------------------------- /eval/finetune_task_eval.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jkallini/mrt5/HEAD/eval/finetune_task_eval.ipynb -------------------------------------------------------------------------------- /eval/gate_placement_analysis.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jkallini/mrt5/HEAD/eval/gate_placement_analysis.ipynb -------------------------------------------------------------------------------- /eval/qa_eval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jkallini/mrt5/HEAD/eval/qa_eval.py -------------------------------------------------------------------------------- /eval/span_corruption_eval.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jkallini/mrt5/HEAD/eval/span_corruption_eval.ipynb -------------------------------------------------------------------------------- /eval/span_corruption_eval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jkallini/mrt5/HEAD/eval/span_corruption_eval.py -------------------------------------------------------------------------------- /eval/span_corruption_per_sample_eval.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jkallini/mrt5/HEAD/eval/span_corruption_per_sample_eval.ipynb -------------------------------------------------------------------------------- /eval/xnli_eval.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jkallini/mrt5/HEAD/eval/xnli_eval.py -------------------------------------------------------------------------------- /icons/MrT5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jkallini/mrt5/HEAD/icons/MrT5.png -------------------------------------------------------------------------------- /models/configuration_mrt5.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jkallini/mrt5/HEAD/models/configuration_mrt5.py -------------------------------------------------------------------------------- /models/model_upload.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jkallini/mrt5/HEAD/models/model_upload.py -------------------------------------------------------------------------------- /models/modeling_bpt5.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jkallini/mrt5/HEAD/models/modeling_bpt5.py -------------------------------------------------------------------------------- /models/modeling_canine.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jkallini/mrt5/HEAD/models/modeling_canine.py -------------------------------------------------------------------------------- /models/modeling_mrt5.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jkallini/mrt5/HEAD/models/modeling_mrt5.py -------------------------------------------------------------------------------- /models/modeling_t5.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jkallini/mrt5/HEAD/models/modeling_t5.py -------------------------------------------------------------------------------- /models/shortening.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jkallini/mrt5/HEAD/models/shortening.py -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jkallini/mrt5/HEAD/requirements.txt -------------------------------------------------------------------------------- /training/train.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jkallini/mrt5/HEAD/training/train.py -------------------------------------------------------------------------------- /training/trainer.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jkallini/mrt5/HEAD/training/trainer.py -------------------------------------------------------------------------------- /utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jkallini/mrt5/HEAD/utils.py --------------------------------------------------------------------------------