├── .github └── workflows │ └── python-publish.yml ├── .gitignore ├── LICENSE ├── README.md ├── assert.py ├── data ├── README.md └── enwik8.gz ├── images ├── layernorm-forward-backward.png ├── layernorm.png └── softmax.png ├── setup.py ├── train.py └── triton_transformer ├── __init__.py ├── autoregressive_wrapper.py ├── bmm.py ├── cross_entropy.py ├── dropout.py ├── layernorm.py ├── softmax.py ├── transformer.py └── utils.py /.github/workflows/python-publish.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lucidrains/triton-transformer/HEAD/.github/workflows/python-publish.yml -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lucidrains/triton-transformer/HEAD/.gitignore -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lucidrains/triton-transformer/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lucidrains/triton-transformer/HEAD/README.md -------------------------------------------------------------------------------- /assert.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lucidrains/triton-transformer/HEAD/assert.py -------------------------------------------------------------------------------- /data/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lucidrains/triton-transformer/HEAD/data/README.md -------------------------------------------------------------------------------- /data/enwik8.gz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lucidrains/triton-transformer/HEAD/data/enwik8.gz -------------------------------------------------------------------------------- /images/layernorm-forward-backward.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lucidrains/triton-transformer/HEAD/images/layernorm-forward-backward.png -------------------------------------------------------------------------------- /images/layernorm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lucidrains/triton-transformer/HEAD/images/layernorm.png -------------------------------------------------------------------------------- /images/softmax.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lucidrains/triton-transformer/HEAD/images/softmax.png -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lucidrains/triton-transformer/HEAD/setup.py -------------------------------------------------------------------------------- /train.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lucidrains/triton-transformer/HEAD/train.py -------------------------------------------------------------------------------- /triton_transformer/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lucidrains/triton-transformer/HEAD/triton_transformer/__init__.py -------------------------------------------------------------------------------- /triton_transformer/autoregressive_wrapper.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lucidrains/triton-transformer/HEAD/triton_transformer/autoregressive_wrapper.py -------------------------------------------------------------------------------- /triton_transformer/bmm.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lucidrains/triton-transformer/HEAD/triton_transformer/bmm.py -------------------------------------------------------------------------------- /triton_transformer/cross_entropy.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lucidrains/triton-transformer/HEAD/triton_transformer/cross_entropy.py -------------------------------------------------------------------------------- /triton_transformer/dropout.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lucidrains/triton-transformer/HEAD/triton_transformer/dropout.py -------------------------------------------------------------------------------- /triton_transformer/layernorm.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lucidrains/triton-transformer/HEAD/triton_transformer/layernorm.py -------------------------------------------------------------------------------- /triton_transformer/softmax.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lucidrains/triton-transformer/HEAD/triton_transformer/softmax.py -------------------------------------------------------------------------------- /triton_transformer/transformer.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lucidrains/triton-transformer/HEAD/triton_transformer/transformer.py -------------------------------------------------------------------------------- /triton_transformer/utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lucidrains/triton-transformer/HEAD/triton_transformer/utils.py --------------------------------------------------------------------------------