├── .gitignore ├── LICENSE ├── README.md ├── caffe └── README.md ├── chainer ├── README.md ├── mnist_dataset.py ├── mnist_helper.py ├── train_mnist.py ├── train_mnist_gpu.py └── train_mnist_multi.py ├── imgs └── pytorch │ ├── data_parallel_gpu.PNG │ ├── data_parallel_time.PNG │ ├── multi_node_distribute_gpu.PNG │ ├── multi_node_distribute_gpu_node2.PNG │ ├── multi_node_distribute_time.PNG │ ├── sg_gpu.PNG │ ├── sg_time.PNG │ ├── single_node_distribute.PNG │ ├── single_node_distribute_rank0_time.PNG │ └── single_node_distribute_rank1_time.PNG ├── mxnet └── README.md ├── pytorch ├── README.md ├── data_parallel.py ├── distributed_data_parallel.py ├── model.py └── single_gpu.py ├── tensorflow └── README.md └── tensorflow2 ├── README.md ├── mnist_mirror_strategy.py ├── mnist_multi_worker_strategy.py └── mnist_single.py /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Crisescode/distributed-training-dl/HEAD/.gitignore -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Crisescode/distributed-training-dl/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Crisescode/distributed-training-dl/HEAD/README.md -------------------------------------------------------------------------------- /caffe/README.md: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /chainer/README.md: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /chainer/mnist_dataset.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Crisescode/distributed-training-dl/HEAD/chainer/mnist_dataset.py -------------------------------------------------------------------------------- /chainer/mnist_helper.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Crisescode/distributed-training-dl/HEAD/chainer/mnist_helper.py -------------------------------------------------------------------------------- /chainer/train_mnist.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Crisescode/distributed-training-dl/HEAD/chainer/train_mnist.py -------------------------------------------------------------------------------- /chainer/train_mnist_gpu.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Crisescode/distributed-training-dl/HEAD/chainer/train_mnist_gpu.py -------------------------------------------------------------------------------- /chainer/train_mnist_multi.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Crisescode/distributed-training-dl/HEAD/chainer/train_mnist_multi.py -------------------------------------------------------------------------------- /imgs/pytorch/data_parallel_gpu.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Crisescode/distributed-training-dl/HEAD/imgs/pytorch/data_parallel_gpu.PNG -------------------------------------------------------------------------------- /imgs/pytorch/data_parallel_time.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Crisescode/distributed-training-dl/HEAD/imgs/pytorch/data_parallel_time.PNG -------------------------------------------------------------------------------- /imgs/pytorch/multi_node_distribute_gpu.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Crisescode/distributed-training-dl/HEAD/imgs/pytorch/multi_node_distribute_gpu.PNG -------------------------------------------------------------------------------- /imgs/pytorch/multi_node_distribute_gpu_node2.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Crisescode/distributed-training-dl/HEAD/imgs/pytorch/multi_node_distribute_gpu_node2.PNG -------------------------------------------------------------------------------- /imgs/pytorch/multi_node_distribute_time.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Crisescode/distributed-training-dl/HEAD/imgs/pytorch/multi_node_distribute_time.PNG -------------------------------------------------------------------------------- /imgs/pytorch/sg_gpu.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Crisescode/distributed-training-dl/HEAD/imgs/pytorch/sg_gpu.PNG -------------------------------------------------------------------------------- /imgs/pytorch/sg_time.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Crisescode/distributed-training-dl/HEAD/imgs/pytorch/sg_time.PNG -------------------------------------------------------------------------------- /imgs/pytorch/single_node_distribute.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Crisescode/distributed-training-dl/HEAD/imgs/pytorch/single_node_distribute.PNG -------------------------------------------------------------------------------- /imgs/pytorch/single_node_distribute_rank0_time.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Crisescode/distributed-training-dl/HEAD/imgs/pytorch/single_node_distribute_rank0_time.PNG -------------------------------------------------------------------------------- /imgs/pytorch/single_node_distribute_rank1_time.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Crisescode/distributed-training-dl/HEAD/imgs/pytorch/single_node_distribute_rank1_time.PNG -------------------------------------------------------------------------------- /mxnet/README.md: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /pytorch/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Crisescode/distributed-training-dl/HEAD/pytorch/README.md -------------------------------------------------------------------------------- /pytorch/data_parallel.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Crisescode/distributed-training-dl/HEAD/pytorch/data_parallel.py -------------------------------------------------------------------------------- /pytorch/distributed_data_parallel.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Crisescode/distributed-training-dl/HEAD/pytorch/distributed_data_parallel.py -------------------------------------------------------------------------------- /pytorch/model.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Crisescode/distributed-training-dl/HEAD/pytorch/model.py -------------------------------------------------------------------------------- /pytorch/single_gpu.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Crisescode/distributed-training-dl/HEAD/pytorch/single_gpu.py -------------------------------------------------------------------------------- /tensorflow/README.md: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /tensorflow2/README.md: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /tensorflow2/mnist_mirror_strategy.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Crisescode/distributed-training-dl/HEAD/tensorflow2/mnist_mirror_strategy.py -------------------------------------------------------------------------------- /tensorflow2/mnist_multi_worker_strategy.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Crisescode/distributed-training-dl/HEAD/tensorflow2/mnist_multi_worker_strategy.py -------------------------------------------------------------------------------- /tensorflow2/mnist_single.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Crisescode/distributed-training-dl/HEAD/tensorflow2/mnist_single.py --------------------------------------------------------------------------------