├── .gitignore ├── CODE_OF_CONDUCT.md ├── CONTRIBUTING.md ├── LICENSE ├── README.md ├── adagrad_with_grad_clip.py ├── adaptive_span.py ├── config.py ├── data.py ├── experiments ├── enwik8_large.sh └── text8_large.sh ├── get_data.sh ├── get_pretrained.sh ├── main.py ├── models.py ├── trainer.py └── utils.py /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ofirpress/sandwich_transformer/HEAD/.gitignore -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ofirpress/sandwich_transformer/HEAD/CODE_OF_CONDUCT.md -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ofirpress/sandwich_transformer/HEAD/CONTRIBUTING.md -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ofirpress/sandwich_transformer/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ofirpress/sandwich_transformer/HEAD/README.md -------------------------------------------------------------------------------- /adagrad_with_grad_clip.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ofirpress/sandwich_transformer/HEAD/adagrad_with_grad_clip.py -------------------------------------------------------------------------------- /adaptive_span.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ofirpress/sandwich_transformer/HEAD/adaptive_span.py -------------------------------------------------------------------------------- /config.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ofirpress/sandwich_transformer/HEAD/config.py -------------------------------------------------------------------------------- /data.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ofirpress/sandwich_transformer/HEAD/data.py -------------------------------------------------------------------------------- /experiments/enwik8_large.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ofirpress/sandwich_transformer/HEAD/experiments/enwik8_large.sh -------------------------------------------------------------------------------- /experiments/text8_large.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ofirpress/sandwich_transformer/HEAD/experiments/text8_large.sh -------------------------------------------------------------------------------- /get_data.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ofirpress/sandwich_transformer/HEAD/get_data.sh -------------------------------------------------------------------------------- /get_pretrained.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ofirpress/sandwich_transformer/HEAD/get_pretrained.sh -------------------------------------------------------------------------------- /main.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ofirpress/sandwich_transformer/HEAD/main.py -------------------------------------------------------------------------------- /models.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ofirpress/sandwich_transformer/HEAD/models.py -------------------------------------------------------------------------------- /trainer.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ofirpress/sandwich_transformer/HEAD/trainer.py -------------------------------------------------------------------------------- /utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ofirpress/sandwich_transformer/HEAD/utils.py --------------------------------------------------------------------------------