├── .gitignore ├── README.md ├── __init__.py ├── __pycache__ ├── datasets.cpython-37.pyc ├── datasets.cpython-39.pyc ├── model.cpython-37.pyc └── model.cpython-39.pyc ├── data ├── div_97.npy ├── div_odd_97.npy ├── minus_97.npy ├── perm_xy_5.npy ├── perm_xyx1_5.npy ├── perm_xyx_5.npy ├── plus_5.npy ├── plus_97.npy ├── s5.npy ├── x2xyy2_97.npy ├── x2xyy2x_97.npy ├── x2y2_97.npy ├── x3xy2y_97.npy └── x3xy_97.npy ├── datasets.py ├── environment.yml ├── model.py └── train.py /.gitignore: -------------------------------------------------------------------------------- 1 | lightning_logs/ 2 | wandb/ 3 | grokking-transformer/ 4 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/TomFrederik/grokking/HEAD/README.md -------------------------------------------------------------------------------- /__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /__pycache__/datasets.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/TomFrederik/grokking/HEAD/__pycache__/datasets.cpython-37.pyc -------------------------------------------------------------------------------- /__pycache__/datasets.cpython-39.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/TomFrederik/grokking/HEAD/__pycache__/datasets.cpython-39.pyc -------------------------------------------------------------------------------- /__pycache__/model.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/TomFrederik/grokking/HEAD/__pycache__/model.cpython-37.pyc -------------------------------------------------------------------------------- /__pycache__/model.cpython-39.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/TomFrederik/grokking/HEAD/__pycache__/model.cpython-39.pyc -------------------------------------------------------------------------------- /data/div_97.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/TomFrederik/grokking/HEAD/data/div_97.npy -------------------------------------------------------------------------------- /data/div_odd_97.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/TomFrederik/grokking/HEAD/data/div_odd_97.npy -------------------------------------------------------------------------------- /data/minus_97.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/TomFrederik/grokking/HEAD/data/minus_97.npy -------------------------------------------------------------------------------- /data/perm_xy_5.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/TomFrederik/grokking/HEAD/data/perm_xy_5.npy -------------------------------------------------------------------------------- /data/perm_xyx1_5.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/TomFrederik/grokking/HEAD/data/perm_xyx1_5.npy -------------------------------------------------------------------------------- /data/perm_xyx_5.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/TomFrederik/grokking/HEAD/data/perm_xyx_5.npy -------------------------------------------------------------------------------- /data/plus_5.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/TomFrederik/grokking/HEAD/data/plus_5.npy -------------------------------------------------------------------------------- /data/plus_97.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/TomFrederik/grokking/HEAD/data/plus_97.npy -------------------------------------------------------------------------------- /data/s5.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/TomFrederik/grokking/HEAD/data/s5.npy -------------------------------------------------------------------------------- /data/x2xyy2_97.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/TomFrederik/grokking/HEAD/data/x2xyy2_97.npy -------------------------------------------------------------------------------- /data/x2xyy2x_97.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/TomFrederik/grokking/HEAD/data/x2xyy2x_97.npy -------------------------------------------------------------------------------- /data/x2y2_97.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/TomFrederik/grokking/HEAD/data/x2y2_97.npy -------------------------------------------------------------------------------- /data/x3xy2y_97.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/TomFrederik/grokking/HEAD/data/x3xy2y_97.npy -------------------------------------------------------------------------------- /data/x3xy_97.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/TomFrederik/grokking/HEAD/data/x3xy_97.npy -------------------------------------------------------------------------------- /datasets.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/TomFrederik/grokking/HEAD/datasets.py -------------------------------------------------------------------------------- /environment.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/TomFrederik/grokking/HEAD/environment.yml -------------------------------------------------------------------------------- /model.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/TomFrederik/grokking/HEAD/model.py -------------------------------------------------------------------------------- /train.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/TomFrederik/grokking/HEAD/train.py --------------------------------------------------------------------------------