├── .gitignore ├── .project ├── .pydevproject ├── BuildWeightWatcher.ipynb ├── BuildWeightWatcherOnColab.ipynb ├── Changelog.txt ├── LICENSE.txt ├── README.md ├── checks ├── check_mac_M12.py ├── check_scipy_svd.py ├── check_svd_perf.py └── check_tf.py ├── examples ├── CV-models.png ├── Epoch-Wise-DoubleDescent.ipynb ├── LSA.ipynb ├── MLP3-MNIST-AdamW.ipynb ├── MLP3-MNIST-Muon.ipynb ├── Marchenko_Pastur_Plots.ipynb ├── ModelPlots.ipynb ├── README.md ├── TestBulkEdge.ipynb ├── TestFrameworks.ipynb ├── VGG16.1.png ├── VGG16.2.png ├── WW-Albert-DetX.ipynb ├── WW-Albert-Paper-Example.ipynb ├── WW-Albert.ipynb ├── WW-BERT-BlogExample.ipynb ├── WW-Calibrate_Alpha.ipynb ├── WW-Custom-Plot-Title.ipynb ├── WW-DenseNet.ipynb ├── WW-Full-PyTorchCV.ipynb ├── WW-GPT-Debug.ipynb ├── WW-GPT-TPL.ipynb ├── WW-GPT.ipynb ├── WW-GloveVectors.ipynb ├── WW-Inverse.ipynb ├── WW-Laplacian.ipynb ├── WW-LayerIterator.ipynb ├── WW-LegalNER.ipynb ├── WW-Lightning-Flash.ipynb ├── WW-ONNX.ipynb ├── WW-RandMats.ipynb ├── WW-ResNet.ipynb ├── WW-SVDSharpness-VGG11.ipynb ├── WW-SVDSmoothing-VGG16-Keras.ipynb ├── WW-SVDSmoothing-VGG16.ipynb ├── WW-SVDSmoothing.ipynb ├── WW-TopologicalEntropy.ipynb ├── WW-VGG-RandDistance.ipynb ├── WW-VGG.ipynb ├── WW-VGG11-Debug.ipynb ├── WW-VGG11-SoftRank.ipynb ├── WWLayerIteratorB.ipynb ├── WWPyStateDictFileIterator.ipynb ├── WW_Sentence_Transformers.ipynb ├── WeightWatcher.ipynb ├── all_sentence_transformers.txt ├── bvlcalexnet-9.onnx ├── densenet.png ├── example.onnx ├── pytorchcv.html ├── resnet18.png ├── tempNetwork.onnx └── test.sh ├── img ├── ESD-plots.png ├── WW-logo-long.jpg ├── correlation_trap.jpeg ├── early_stopping.png ├── sample-ww-details.png └── ww0.7.4.jpeg ├── presentations ├── icml19_poster.pdf ├── icml19_workshop_poster.pdf ├── kdd2019.pdf ├── pytorch2021_poster.pdf ├── ww_5min_talk.pdf ├── ww_short_talk.pdf └── ww_short_talk3.pdf ├── setup.cfg ├── setup.mac ├── setup.py ├── tests ├── __init__.py ├── test.py └── test_svd.py └── weightwatcher ├── RMT_Util.py ├── WW_powerlaw.py ├── __init__.py ├── __main__.py ├── constants.py └── weightwatcher.py /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/.gitignore -------------------------------------------------------------------------------- /.project: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/.project -------------------------------------------------------------------------------- /.pydevproject: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/.pydevproject -------------------------------------------------------------------------------- /BuildWeightWatcher.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/BuildWeightWatcher.ipynb -------------------------------------------------------------------------------- /BuildWeightWatcherOnColab.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/BuildWeightWatcherOnColab.ipynb -------------------------------------------------------------------------------- /Changelog.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/Changelog.txt -------------------------------------------------------------------------------- /LICENSE.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/LICENSE.txt -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/README.md -------------------------------------------------------------------------------- /checks/check_mac_M12.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/checks/check_mac_M12.py -------------------------------------------------------------------------------- /checks/check_scipy_svd.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/checks/check_scipy_svd.py -------------------------------------------------------------------------------- /checks/check_svd_perf.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/checks/check_svd_perf.py -------------------------------------------------------------------------------- /checks/check_tf.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/checks/check_tf.py -------------------------------------------------------------------------------- /examples/CV-models.png: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /examples/Epoch-Wise-DoubleDescent.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/Epoch-Wise-DoubleDescent.ipynb -------------------------------------------------------------------------------- /examples/LSA.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/LSA.ipynb -------------------------------------------------------------------------------- /examples/MLP3-MNIST-AdamW.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/MLP3-MNIST-AdamW.ipynb -------------------------------------------------------------------------------- /examples/MLP3-MNIST-Muon.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/MLP3-MNIST-Muon.ipynb -------------------------------------------------------------------------------- /examples/Marchenko_Pastur_Plots.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/Marchenko_Pastur_Plots.ipynb -------------------------------------------------------------------------------- /examples/ModelPlots.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/ModelPlots.ipynb -------------------------------------------------------------------------------- /examples/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/README.md -------------------------------------------------------------------------------- /examples/TestBulkEdge.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/TestBulkEdge.ipynb -------------------------------------------------------------------------------- /examples/TestFrameworks.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/TestFrameworks.ipynb -------------------------------------------------------------------------------- /examples/VGG16.1.png: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /examples/VGG16.2.png: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /examples/WW-Albert-DetX.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/WW-Albert-DetX.ipynb -------------------------------------------------------------------------------- /examples/WW-Albert-Paper-Example.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/WW-Albert-Paper-Example.ipynb -------------------------------------------------------------------------------- /examples/WW-Albert.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/WW-Albert.ipynb -------------------------------------------------------------------------------- /examples/WW-BERT-BlogExample.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/WW-BERT-BlogExample.ipynb -------------------------------------------------------------------------------- /examples/WW-Calibrate_Alpha.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/WW-Calibrate_Alpha.ipynb -------------------------------------------------------------------------------- /examples/WW-Custom-Plot-Title.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/WW-Custom-Plot-Title.ipynb -------------------------------------------------------------------------------- /examples/WW-DenseNet.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/WW-DenseNet.ipynb -------------------------------------------------------------------------------- /examples/WW-Full-PyTorchCV.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/WW-Full-PyTorchCV.ipynb -------------------------------------------------------------------------------- /examples/WW-GPT-Debug.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/WW-GPT-Debug.ipynb -------------------------------------------------------------------------------- /examples/WW-GPT-TPL.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/WW-GPT-TPL.ipynb -------------------------------------------------------------------------------- /examples/WW-GPT.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/WW-GPT.ipynb -------------------------------------------------------------------------------- /examples/WW-GloveVectors.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/WW-GloveVectors.ipynb -------------------------------------------------------------------------------- /examples/WW-Inverse.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/WW-Inverse.ipynb -------------------------------------------------------------------------------- /examples/WW-Laplacian.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/WW-Laplacian.ipynb -------------------------------------------------------------------------------- /examples/WW-LayerIterator.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/WW-LayerIterator.ipynb -------------------------------------------------------------------------------- /examples/WW-LegalNER.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/WW-LegalNER.ipynb -------------------------------------------------------------------------------- /examples/WW-Lightning-Flash.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/WW-Lightning-Flash.ipynb -------------------------------------------------------------------------------- /examples/WW-ONNX.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/WW-ONNX.ipynb -------------------------------------------------------------------------------- /examples/WW-RandMats.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/WW-RandMats.ipynb -------------------------------------------------------------------------------- /examples/WW-ResNet.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/WW-ResNet.ipynb -------------------------------------------------------------------------------- /examples/WW-SVDSharpness-VGG11.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/WW-SVDSharpness-VGG11.ipynb -------------------------------------------------------------------------------- /examples/WW-SVDSmoothing-VGG16-Keras.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/WW-SVDSmoothing-VGG16-Keras.ipynb -------------------------------------------------------------------------------- /examples/WW-SVDSmoothing-VGG16.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/WW-SVDSmoothing-VGG16.ipynb -------------------------------------------------------------------------------- /examples/WW-SVDSmoothing.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/WW-SVDSmoothing.ipynb -------------------------------------------------------------------------------- /examples/WW-TopologicalEntropy.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/WW-TopologicalEntropy.ipynb -------------------------------------------------------------------------------- /examples/WW-VGG-RandDistance.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/WW-VGG-RandDistance.ipynb -------------------------------------------------------------------------------- /examples/WW-VGG.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/WW-VGG.ipynb -------------------------------------------------------------------------------- /examples/WW-VGG11-Debug.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/WW-VGG11-Debug.ipynb -------------------------------------------------------------------------------- /examples/WW-VGG11-SoftRank.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/WW-VGG11-SoftRank.ipynb -------------------------------------------------------------------------------- /examples/WWLayerIteratorB.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/WWLayerIteratorB.ipynb -------------------------------------------------------------------------------- /examples/WWPyStateDictFileIterator.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/WWPyStateDictFileIterator.ipynb -------------------------------------------------------------------------------- /examples/WW_Sentence_Transformers.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/WW_Sentence_Transformers.ipynb -------------------------------------------------------------------------------- /examples/WeightWatcher.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/WeightWatcher.ipynb -------------------------------------------------------------------------------- /examples/all_sentence_transformers.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/all_sentence_transformers.txt -------------------------------------------------------------------------------- /examples/bvlcalexnet-9.onnx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/bvlcalexnet-9.onnx -------------------------------------------------------------------------------- /examples/densenet.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/densenet.png -------------------------------------------------------------------------------- /examples/example.onnx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/example.onnx -------------------------------------------------------------------------------- /examples/pytorchcv.html: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/pytorchcv.html -------------------------------------------------------------------------------- /examples/resnet18.png: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /examples/tempNetwork.onnx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/tempNetwork.onnx -------------------------------------------------------------------------------- /examples/test.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/examples/test.sh -------------------------------------------------------------------------------- /img/ESD-plots.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/img/ESD-plots.png -------------------------------------------------------------------------------- /img/WW-logo-long.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/img/WW-logo-long.jpg -------------------------------------------------------------------------------- /img/correlation_trap.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/img/correlation_trap.jpeg -------------------------------------------------------------------------------- /img/early_stopping.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/img/early_stopping.png -------------------------------------------------------------------------------- /img/sample-ww-details.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/img/sample-ww-details.png -------------------------------------------------------------------------------- /img/ww0.7.4.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/img/ww0.7.4.jpeg -------------------------------------------------------------------------------- /presentations/icml19_poster.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/presentations/icml19_poster.pdf -------------------------------------------------------------------------------- /presentations/icml19_workshop_poster.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/presentations/icml19_workshop_poster.pdf -------------------------------------------------------------------------------- /presentations/kdd2019.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/presentations/kdd2019.pdf -------------------------------------------------------------------------------- /presentations/pytorch2021_poster.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/presentations/pytorch2021_poster.pdf -------------------------------------------------------------------------------- /presentations/ww_5min_talk.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/presentations/ww_5min_talk.pdf -------------------------------------------------------------------------------- /presentations/ww_short_talk.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/presentations/ww_short_talk.pdf -------------------------------------------------------------------------------- /presentations/ww_short_talk3.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/presentations/ww_short_talk3.pdf -------------------------------------------------------------------------------- /setup.cfg: -------------------------------------------------------------------------------- 1 | [metadata] 2 | description-file = README.md 3 | -------------------------------------------------------------------------------- /setup.mac: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/setup.mac -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/setup.py -------------------------------------------------------------------------------- /tests/__init__.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | 4 | -------------------------------------------------------------------------------- /tests/test.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/tests/test.py -------------------------------------------------------------------------------- /tests/test_svd.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/tests/test_svd.py -------------------------------------------------------------------------------- /weightwatcher/RMT_Util.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/weightwatcher/RMT_Util.py -------------------------------------------------------------------------------- /weightwatcher/WW_powerlaw.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/weightwatcher/WW_powerlaw.py -------------------------------------------------------------------------------- /weightwatcher/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/weightwatcher/__init__.py -------------------------------------------------------------------------------- /weightwatcher/__main__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/weightwatcher/__main__.py -------------------------------------------------------------------------------- /weightwatcher/constants.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/weightwatcher/constants.py -------------------------------------------------------------------------------- /weightwatcher/weightwatcher.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CalculatedContent/WeightWatcher/HEAD/weightwatcher/weightwatcher.py --------------------------------------------------------------------------------