├── DistributedSgd.py ├── Readme.md ├── UnshuffleSampler.py ├── figure ├── lenet_identical.pdf ├── lenet_non_identical.pdf ├── textCNN_identical.pdf ├── textCNN_non_identical.pdf ├── transfer_learning_identical.pdf └── transfer_learning_non_identical.pdf ├── main.py ├── myutils.py ├── plot.py ├── plot_all.sh ├── process_text.py ├── process_tiny_magenet.py ├── record ├── LeNet │ ├── identical │ │ ├── localsgd_lenet5_mnist_restart_20_extra_0_100.csv │ │ ├── minisgd_lenet5_mnist_restart_1_extra_0_100.csv │ │ └── vrlsgdsgd_lenet5_mnist_restart_20_extra_1_100.csv │ └── non-identical │ │ ├── localsgd_lenet5_mnist_restart_20_extra_0_eta_0.005_b_32_100.csv │ │ ├── minisgd_lenet5_mnist_restart_1_extra_0_eta_0.005_b_32_100.csv │ │ └── vrlsgd_lenet5_mnist_restart_20_extra_1_eta_0.005_b_32_100.csv ├── TextCNN │ ├── identical │ │ ├── localsgd_text_cnn_DB_Pedia_restart_50_extra_0_eta_0.01_b_64_cluster_data_False_100.csv │ │ ├── minisgd_text_cnn_DB_Pedia_restart_1_extra_0_eta_0.01_b_64_cluster_data_False_100.csv │ │ └── vrlsgd_text_cnn_DB_Pedia_restart_50_extra_1_eta_0.01_b_64_cluster_data_False_100.csv │ └── non-identical │ │ ├── localsgd_text_cnn_DB_Pedia_restart_50_extra_0_eta_0.01_b_64_cluster_data_True_100.csv │ │ ├── minisgd_text_cnn_DB_Pedia_restart_1_extra_0_eta_0.01_b_64_cluster_data_False_100.csv │ │ └── vrlsgd_text_cnn_DB_Pedia_restart_50_extra_1_eta_0.01_b_64_cluster_data_True_100.csv └── TransferLearning │ ├── identical │ ├── localsgd_mlp_tiny_imagenet_restart_20_extra_0_eta_0.025_b_32_cluster_data_False_300.csv │ ├── minisgd_mlp_tiny_imagenet_restart_1_extra_0_eta_0.025_b_32_cluster_data_False_300.csv │ └── vrlsgd_mlp_tiny_imagenet_restart_20_extra_1_eta_0.025_b_32_cluster_data_False_300.csv │ └── non-identical │ ├── localsgd_mlp_tiny_imagenet_restart_20_extra_0_eta_0.025_b_32_cluster_data_True_300.csv │ ├── minisgd_mlp_tiny_imagenet_restart_1_extra_0_eta_0.025_b_32_cluster_data_True_300.csv │ └── vrlsgd_mlp_tiny_imagenet_restart_20_extra_1_eta_0.025_b_32_cluster_data_True_300.csv ├── restarted.py ├── textCNN.py ├── train.sh └── trainer.py /DistributedSgd.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zerolxf/VRL-SGD/HEAD/DistributedSgd.py -------------------------------------------------------------------------------- /Readme.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zerolxf/VRL-SGD/HEAD/Readme.md -------------------------------------------------------------------------------- /UnshuffleSampler.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zerolxf/VRL-SGD/HEAD/UnshuffleSampler.py -------------------------------------------------------------------------------- /figure/lenet_identical.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zerolxf/VRL-SGD/HEAD/figure/lenet_identical.pdf -------------------------------------------------------------------------------- /figure/lenet_non_identical.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zerolxf/VRL-SGD/HEAD/figure/lenet_non_identical.pdf -------------------------------------------------------------------------------- /figure/textCNN_identical.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zerolxf/VRL-SGD/HEAD/figure/textCNN_identical.pdf -------------------------------------------------------------------------------- /figure/textCNN_non_identical.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zerolxf/VRL-SGD/HEAD/figure/textCNN_non_identical.pdf -------------------------------------------------------------------------------- /figure/transfer_learning_identical.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zerolxf/VRL-SGD/HEAD/figure/transfer_learning_identical.pdf -------------------------------------------------------------------------------- /figure/transfer_learning_non_identical.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zerolxf/VRL-SGD/HEAD/figure/transfer_learning_non_identical.pdf -------------------------------------------------------------------------------- /main.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zerolxf/VRL-SGD/HEAD/main.py -------------------------------------------------------------------------------- /myutils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zerolxf/VRL-SGD/HEAD/myutils.py -------------------------------------------------------------------------------- /plot.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zerolxf/VRL-SGD/HEAD/plot.py -------------------------------------------------------------------------------- /plot_all.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zerolxf/VRL-SGD/HEAD/plot_all.sh -------------------------------------------------------------------------------- /process_text.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zerolxf/VRL-SGD/HEAD/process_text.py -------------------------------------------------------------------------------- /process_tiny_magenet.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zerolxf/VRL-SGD/HEAD/process_tiny_magenet.py -------------------------------------------------------------------------------- /record/LeNet/identical/localsgd_lenet5_mnist_restart_20_extra_0_100.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zerolxf/VRL-SGD/HEAD/record/LeNet/identical/localsgd_lenet5_mnist_restart_20_extra_0_100.csv -------------------------------------------------------------------------------- /record/LeNet/identical/minisgd_lenet5_mnist_restart_1_extra_0_100.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zerolxf/VRL-SGD/HEAD/record/LeNet/identical/minisgd_lenet5_mnist_restart_1_extra_0_100.csv -------------------------------------------------------------------------------- /record/LeNet/identical/vrlsgdsgd_lenet5_mnist_restart_20_extra_1_100.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zerolxf/VRL-SGD/HEAD/record/LeNet/identical/vrlsgdsgd_lenet5_mnist_restart_20_extra_1_100.csv -------------------------------------------------------------------------------- /record/LeNet/non-identical/localsgd_lenet5_mnist_restart_20_extra_0_eta_0.005_b_32_100.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zerolxf/VRL-SGD/HEAD/record/LeNet/non-identical/localsgd_lenet5_mnist_restart_20_extra_0_eta_0.005_b_32_100.csv -------------------------------------------------------------------------------- /record/LeNet/non-identical/minisgd_lenet5_mnist_restart_1_extra_0_eta_0.005_b_32_100.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zerolxf/VRL-SGD/HEAD/record/LeNet/non-identical/minisgd_lenet5_mnist_restart_1_extra_0_eta_0.005_b_32_100.csv -------------------------------------------------------------------------------- /record/LeNet/non-identical/vrlsgd_lenet5_mnist_restart_20_extra_1_eta_0.005_b_32_100.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zerolxf/VRL-SGD/HEAD/record/LeNet/non-identical/vrlsgd_lenet5_mnist_restart_20_extra_1_eta_0.005_b_32_100.csv -------------------------------------------------------------------------------- /record/TextCNN/identical/localsgd_text_cnn_DB_Pedia_restart_50_extra_0_eta_0.01_b_64_cluster_data_False_100.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zerolxf/VRL-SGD/HEAD/record/TextCNN/identical/localsgd_text_cnn_DB_Pedia_restart_50_extra_0_eta_0.01_b_64_cluster_data_False_100.csv -------------------------------------------------------------------------------- /record/TextCNN/identical/minisgd_text_cnn_DB_Pedia_restart_1_extra_0_eta_0.01_b_64_cluster_data_False_100.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zerolxf/VRL-SGD/HEAD/record/TextCNN/identical/minisgd_text_cnn_DB_Pedia_restart_1_extra_0_eta_0.01_b_64_cluster_data_False_100.csv -------------------------------------------------------------------------------- /record/TextCNN/identical/vrlsgd_text_cnn_DB_Pedia_restart_50_extra_1_eta_0.01_b_64_cluster_data_False_100.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zerolxf/VRL-SGD/HEAD/record/TextCNN/identical/vrlsgd_text_cnn_DB_Pedia_restart_50_extra_1_eta_0.01_b_64_cluster_data_False_100.csv -------------------------------------------------------------------------------- /record/TextCNN/non-identical/localsgd_text_cnn_DB_Pedia_restart_50_extra_0_eta_0.01_b_64_cluster_data_True_100.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zerolxf/VRL-SGD/HEAD/record/TextCNN/non-identical/localsgd_text_cnn_DB_Pedia_restart_50_extra_0_eta_0.01_b_64_cluster_data_True_100.csv -------------------------------------------------------------------------------- /record/TextCNN/non-identical/minisgd_text_cnn_DB_Pedia_restart_1_extra_0_eta_0.01_b_64_cluster_data_False_100.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zerolxf/VRL-SGD/HEAD/record/TextCNN/non-identical/minisgd_text_cnn_DB_Pedia_restart_1_extra_0_eta_0.01_b_64_cluster_data_False_100.csv -------------------------------------------------------------------------------- /record/TextCNN/non-identical/vrlsgd_text_cnn_DB_Pedia_restart_50_extra_1_eta_0.01_b_64_cluster_data_True_100.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zerolxf/VRL-SGD/HEAD/record/TextCNN/non-identical/vrlsgd_text_cnn_DB_Pedia_restart_50_extra_1_eta_0.01_b_64_cluster_data_True_100.csv -------------------------------------------------------------------------------- /record/TransferLearning/identical/localsgd_mlp_tiny_imagenet_restart_20_extra_0_eta_0.025_b_32_cluster_data_False_300.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zerolxf/VRL-SGD/HEAD/record/TransferLearning/identical/localsgd_mlp_tiny_imagenet_restart_20_extra_0_eta_0.025_b_32_cluster_data_False_300.csv -------------------------------------------------------------------------------- /record/TransferLearning/identical/minisgd_mlp_tiny_imagenet_restart_1_extra_0_eta_0.025_b_32_cluster_data_False_300.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zerolxf/VRL-SGD/HEAD/record/TransferLearning/identical/minisgd_mlp_tiny_imagenet_restart_1_extra_0_eta_0.025_b_32_cluster_data_False_300.csv -------------------------------------------------------------------------------- /record/TransferLearning/identical/vrlsgd_mlp_tiny_imagenet_restart_20_extra_1_eta_0.025_b_32_cluster_data_False_300.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zerolxf/VRL-SGD/HEAD/record/TransferLearning/identical/vrlsgd_mlp_tiny_imagenet_restart_20_extra_1_eta_0.025_b_32_cluster_data_False_300.csv -------------------------------------------------------------------------------- /record/TransferLearning/non-identical/localsgd_mlp_tiny_imagenet_restart_20_extra_0_eta_0.025_b_32_cluster_data_True_300.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zerolxf/VRL-SGD/HEAD/record/TransferLearning/non-identical/localsgd_mlp_tiny_imagenet_restart_20_extra_0_eta_0.025_b_32_cluster_data_True_300.csv -------------------------------------------------------------------------------- /record/TransferLearning/non-identical/minisgd_mlp_tiny_imagenet_restart_1_extra_0_eta_0.025_b_32_cluster_data_True_300.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zerolxf/VRL-SGD/HEAD/record/TransferLearning/non-identical/minisgd_mlp_tiny_imagenet_restart_1_extra_0_eta_0.025_b_32_cluster_data_True_300.csv -------------------------------------------------------------------------------- /record/TransferLearning/non-identical/vrlsgd_mlp_tiny_imagenet_restart_20_extra_1_eta_0.025_b_32_cluster_data_True_300.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zerolxf/VRL-SGD/HEAD/record/TransferLearning/non-identical/vrlsgd_mlp_tiny_imagenet_restart_20_extra_1_eta_0.025_b_32_cluster_data_True_300.csv -------------------------------------------------------------------------------- /restarted.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zerolxf/VRL-SGD/HEAD/restarted.py -------------------------------------------------------------------------------- /textCNN.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zerolxf/VRL-SGD/HEAD/textCNN.py -------------------------------------------------------------------------------- /train.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zerolxf/VRL-SGD/HEAD/train.sh -------------------------------------------------------------------------------- /trainer.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zerolxf/VRL-SGD/HEAD/trainer.py --------------------------------------------------------------------------------