├── LICENSE ├── README.md ├── moonlight_toy_results.png ├── moonlight_toy_train.py ├── mudamw_results.png ├── optims ├── __init__.py ├── adafactor.py ├── lion.py ├── mu_lion.py ├── mu_soap.py ├── mudafactor.py ├── mudamw.py ├── muon.py ├── mupoo.py ├── shampoo.py ├── sharded_muon.py ├── sign_muon.py ├── soap.py └── utils.py └── sign_muon.jpg /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyleliang919/Super_Muon/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyleliang919/Super_Muon/HEAD/README.md -------------------------------------------------------------------------------- /moonlight_toy_results.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyleliang919/Super_Muon/HEAD/moonlight_toy_results.png -------------------------------------------------------------------------------- /moonlight_toy_train.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyleliang919/Super_Muon/HEAD/moonlight_toy_train.py -------------------------------------------------------------------------------- /mudamw_results.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyleliang919/Super_Muon/HEAD/mudamw_results.png -------------------------------------------------------------------------------- /optims/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /optims/adafactor.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyleliang919/Super_Muon/HEAD/optims/adafactor.py -------------------------------------------------------------------------------- /optims/lion.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyleliang919/Super_Muon/HEAD/optims/lion.py -------------------------------------------------------------------------------- /optims/mu_lion.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyleliang919/Super_Muon/HEAD/optims/mu_lion.py -------------------------------------------------------------------------------- /optims/mu_soap.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyleliang919/Super_Muon/HEAD/optims/mu_soap.py -------------------------------------------------------------------------------- /optims/mudafactor.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyleliang919/Super_Muon/HEAD/optims/mudafactor.py -------------------------------------------------------------------------------- /optims/mudamw.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyleliang919/Super_Muon/HEAD/optims/mudamw.py -------------------------------------------------------------------------------- /optims/muon.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyleliang919/Super_Muon/HEAD/optims/muon.py -------------------------------------------------------------------------------- /optims/mupoo.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyleliang919/Super_Muon/HEAD/optims/mupoo.py -------------------------------------------------------------------------------- /optims/shampoo.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyleliang919/Super_Muon/HEAD/optims/shampoo.py -------------------------------------------------------------------------------- /optims/sharded_muon.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyleliang919/Super_Muon/HEAD/optims/sharded_muon.py -------------------------------------------------------------------------------- /optims/sign_muon.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyleliang919/Super_Muon/HEAD/optims/sign_muon.py -------------------------------------------------------------------------------- /optims/soap.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyleliang919/Super_Muon/HEAD/optims/soap.py -------------------------------------------------------------------------------- /optims/utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyleliang919/Super_Muon/HEAD/optims/utils.py -------------------------------------------------------------------------------- /sign_muon.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kyleliang919/Super_Muon/HEAD/sign_muon.jpg --------------------------------------------------------------------------------