├── README.md ├── activations_check.ipynb ├── hessian.ipynb ├── image-1.png ├── results ├── compare_results.ipynb ├── loss_vs_log2lr_SPMLP_1.0_Adam.csv ├── loss_vs_log2lr_SPMLP_1.0_Adam.png ├── loss_vs_log2lr_SPMLP_1.0_SGD.csv ├── loss_vs_log2lr_SPMLP_1.0_SGD.png ├── loss_vs_log2lr_muMLP_1.0_Adam.csv ├── loss_vs_log2lr_muMLP_1.0_Adam.png ├── loss_vs_log2lr_muMLP_1.0_SGD.csv ├── loss_vs_log2lr_muMLP_1.0_SGD.png ├── mlp_cifar10.png └── mlp_cifar10_ready.png ├── run_mlp.sh ├── toy_cifar.ipynb └── train_mlp.py /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Laz4rz/mup/HEAD/README.md -------------------------------------------------------------------------------- /activations_check.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Laz4rz/mup/HEAD/activations_check.ipynb -------------------------------------------------------------------------------- /hessian.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Laz4rz/mup/HEAD/hessian.ipynb -------------------------------------------------------------------------------- /image-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Laz4rz/mup/HEAD/image-1.png -------------------------------------------------------------------------------- /results/compare_results.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Laz4rz/mup/HEAD/results/compare_results.ipynb -------------------------------------------------------------------------------- /results/loss_vs_log2lr_SPMLP_1.0_Adam.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Laz4rz/mup/HEAD/results/loss_vs_log2lr_SPMLP_1.0_Adam.csv -------------------------------------------------------------------------------- /results/loss_vs_log2lr_SPMLP_1.0_Adam.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Laz4rz/mup/HEAD/results/loss_vs_log2lr_SPMLP_1.0_Adam.png -------------------------------------------------------------------------------- /results/loss_vs_log2lr_SPMLP_1.0_SGD.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Laz4rz/mup/HEAD/results/loss_vs_log2lr_SPMLP_1.0_SGD.csv -------------------------------------------------------------------------------- /results/loss_vs_log2lr_SPMLP_1.0_SGD.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Laz4rz/mup/HEAD/results/loss_vs_log2lr_SPMLP_1.0_SGD.png -------------------------------------------------------------------------------- /results/loss_vs_log2lr_muMLP_1.0_Adam.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Laz4rz/mup/HEAD/results/loss_vs_log2lr_muMLP_1.0_Adam.csv -------------------------------------------------------------------------------- /results/loss_vs_log2lr_muMLP_1.0_Adam.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Laz4rz/mup/HEAD/results/loss_vs_log2lr_muMLP_1.0_Adam.png -------------------------------------------------------------------------------- /results/loss_vs_log2lr_muMLP_1.0_SGD.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Laz4rz/mup/HEAD/results/loss_vs_log2lr_muMLP_1.0_SGD.csv -------------------------------------------------------------------------------- /results/loss_vs_log2lr_muMLP_1.0_SGD.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Laz4rz/mup/HEAD/results/loss_vs_log2lr_muMLP_1.0_SGD.png -------------------------------------------------------------------------------- /results/mlp_cifar10.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Laz4rz/mup/HEAD/results/mlp_cifar10.png -------------------------------------------------------------------------------- /results/mlp_cifar10_ready.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Laz4rz/mup/HEAD/results/mlp_cifar10_ready.png -------------------------------------------------------------------------------- /run_mlp.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Laz4rz/mup/HEAD/run_mlp.sh -------------------------------------------------------------------------------- /toy_cifar.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Laz4rz/mup/HEAD/toy_cifar.ipynb -------------------------------------------------------------------------------- /train_mlp.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Laz4rz/mup/HEAD/train_mlp.py --------------------------------------------------------------------------------