├── .gitignore ├── LICENSE ├── README.md ├── assets ├── img.png └── zero.png └── training ├── cifar ├── LICENSE ├── NOTICE.txt ├── README.md ├── cifar10_deepspeed.py ├── cifar10_tutorial.py ├── ds_config.json ├── requirements.txt ├── run_ds.sh ├── run_ds_moe.sh └── run_ds_prmoe.sh └── pipeline_parallelism ├── alexnet.py ├── ds_config.json ├── run.sh └── train.py /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bobo0810/LearnDeepSpeed/HEAD/.gitignore -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bobo0810/LearnDeepSpeed/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bobo0810/LearnDeepSpeed/HEAD/README.md -------------------------------------------------------------------------------- /assets/img.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bobo0810/LearnDeepSpeed/HEAD/assets/img.png -------------------------------------------------------------------------------- /assets/zero.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bobo0810/LearnDeepSpeed/HEAD/assets/zero.png -------------------------------------------------------------------------------- /training/cifar/LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bobo0810/LearnDeepSpeed/HEAD/training/cifar/LICENSE -------------------------------------------------------------------------------- /training/cifar/NOTICE.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bobo0810/LearnDeepSpeed/HEAD/training/cifar/NOTICE.txt -------------------------------------------------------------------------------- /training/cifar/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bobo0810/LearnDeepSpeed/HEAD/training/cifar/README.md -------------------------------------------------------------------------------- /training/cifar/cifar10_deepspeed.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bobo0810/LearnDeepSpeed/HEAD/training/cifar/cifar10_deepspeed.py -------------------------------------------------------------------------------- /training/cifar/cifar10_tutorial.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bobo0810/LearnDeepSpeed/HEAD/training/cifar/cifar10_tutorial.py -------------------------------------------------------------------------------- /training/cifar/ds_config.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bobo0810/LearnDeepSpeed/HEAD/training/cifar/ds_config.json -------------------------------------------------------------------------------- /training/cifar/requirements.txt: -------------------------------------------------------------------------------- 1 | torchvision==0.4.0 2 | pillow>=7.1.0 3 | matplotlib 4 | -------------------------------------------------------------------------------- /training/cifar/run_ds.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bobo0810/LearnDeepSpeed/HEAD/training/cifar/run_ds.sh -------------------------------------------------------------------------------- /training/cifar/run_ds_moe.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bobo0810/LearnDeepSpeed/HEAD/training/cifar/run_ds_moe.sh -------------------------------------------------------------------------------- /training/cifar/run_ds_prmoe.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bobo0810/LearnDeepSpeed/HEAD/training/cifar/run_ds_prmoe.sh -------------------------------------------------------------------------------- /training/pipeline_parallelism/alexnet.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bobo0810/LearnDeepSpeed/HEAD/training/pipeline_parallelism/alexnet.py -------------------------------------------------------------------------------- /training/pipeline_parallelism/ds_config.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bobo0810/LearnDeepSpeed/HEAD/training/pipeline_parallelism/ds_config.json -------------------------------------------------------------------------------- /training/pipeline_parallelism/run.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bobo0810/LearnDeepSpeed/HEAD/training/pipeline_parallelism/run.sh -------------------------------------------------------------------------------- /training/pipeline_parallelism/train.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bobo0810/LearnDeepSpeed/HEAD/training/pipeline_parallelism/train.py --------------------------------------------------------------------------------