├── .gitignore ├── LICENSE ├── README.md ├── apex_distributed.py ├── assets ├── fig1_experimental_result.jpg └── fig2_allreduce.jpg ├── dataparallel.py ├── distributed.py ├── distributed_slurm_main.py ├── horovod_distributed.py ├── multiprocessing_distributed.py ├── requirements.txt ├── start.sh └── statistics.sh /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tczhangzhi/pytorch-distributed/HEAD/.gitignore -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tczhangzhi/pytorch-distributed/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tczhangzhi/pytorch-distributed/HEAD/README.md -------------------------------------------------------------------------------- /apex_distributed.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tczhangzhi/pytorch-distributed/HEAD/apex_distributed.py -------------------------------------------------------------------------------- /assets/fig1_experimental_result.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tczhangzhi/pytorch-distributed/HEAD/assets/fig1_experimental_result.jpg -------------------------------------------------------------------------------- /assets/fig2_allreduce.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tczhangzhi/pytorch-distributed/HEAD/assets/fig2_allreduce.jpg -------------------------------------------------------------------------------- /dataparallel.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tczhangzhi/pytorch-distributed/HEAD/dataparallel.py -------------------------------------------------------------------------------- /distributed.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tczhangzhi/pytorch-distributed/HEAD/distributed.py -------------------------------------------------------------------------------- /distributed_slurm_main.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tczhangzhi/pytorch-distributed/HEAD/distributed_slurm_main.py -------------------------------------------------------------------------------- /horovod_distributed.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tczhangzhi/pytorch-distributed/HEAD/horovod_distributed.py -------------------------------------------------------------------------------- /multiprocessing_distributed.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tczhangzhi/pytorch-distributed/HEAD/multiprocessing_distributed.py -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tczhangzhi/pytorch-distributed/HEAD/requirements.txt -------------------------------------------------------------------------------- /start.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tczhangzhi/pytorch-distributed/HEAD/start.sh -------------------------------------------------------------------------------- /statistics.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tczhangzhi/pytorch-distributed/HEAD/statistics.sh --------------------------------------------------------------------------------