├── .gitignore ├── .travis.yml ├── LICENSE ├── README.md ├── img ├── backprop.png ├── checkpoint.png ├── output.gif ├── output2.gif ├── output_poor.gif ├── resnet_test.png └── sqrtn.png ├── memory_saving_gradients.py ├── requirements.txt └── test ├── data ├── __init__.py ├── __pycache__ │ ├── __init__.cpython-35.pyc │ └── cifar10_data.cpython-35.pyc ├── cifar10_data.py ├── imagenet_data.py └── pixelcnn_samples.png ├── deep_imagenet_benchmark.py ├── deep_resnet_benchmark.py ├── graphs ├── cifar_10.pbtxt ├── cifar_14.pbtxt ├── imagenet_101.pbtxt ├── imagenet_200.pbtxt ├── imagenet_34.pbtxt ├── imagenet_50.pbtxt ├── resnet_grad.pbtxt ├── resnet_grad2.pbtxt ├── simple2_stepstats.pbtxt ├── simple2_timeline_graph.pbtxt ├── simple_stepstats.pbtxt └── simple_timeline_graph.pbtxt ├── imagenet_correctness_test.py ├── imagenet_test.py ├── keras_test.py ├── linearize.py ├── linearize_test.py ├── mem_util.py ├── mem_util_test.py ├── memory_test.py ├── mnist_correctness_test.py ├── multiple_memory_obtain_test.py ├── pixel_cnn_pp ├── __init__.py ├── model.py └── nn.py ├── pixel_cnn_test.py ├── resnet_correctness_test.py ├── resnet_model.py ├── resnet_test.py ├── run_all_tests.sh ├── simple_timeline.png ├── tf.sh ├── util.py └── util_test.py /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/.gitignore -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/.travis.yml -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/README.md -------------------------------------------------------------------------------- /img/backprop.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/img/backprop.png -------------------------------------------------------------------------------- /img/checkpoint.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/img/checkpoint.png -------------------------------------------------------------------------------- /img/output.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/img/output.gif -------------------------------------------------------------------------------- /img/output2.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/img/output2.gif -------------------------------------------------------------------------------- /img/output_poor.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/img/output_poor.gif -------------------------------------------------------------------------------- /img/resnet_test.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/img/resnet_test.png -------------------------------------------------------------------------------- /img/sqrtn.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/img/sqrtn.png -------------------------------------------------------------------------------- /memory_saving_gradients.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/memory_saving_gradients.py -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | tensorflow-gpu 2 | pytest 3 | networkx 4 | toposort 5 | -------------------------------------------------------------------------------- /test/data/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /test/data/__pycache__/__init__.cpython-35.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/data/__pycache__/__init__.cpython-35.pyc -------------------------------------------------------------------------------- /test/data/__pycache__/cifar10_data.cpython-35.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/data/__pycache__/cifar10_data.cpython-35.pyc -------------------------------------------------------------------------------- /test/data/cifar10_data.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/data/cifar10_data.py -------------------------------------------------------------------------------- /test/data/imagenet_data.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/data/imagenet_data.py -------------------------------------------------------------------------------- /test/data/pixelcnn_samples.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/data/pixelcnn_samples.png -------------------------------------------------------------------------------- /test/deep_imagenet_benchmark.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/deep_imagenet_benchmark.py -------------------------------------------------------------------------------- /test/deep_resnet_benchmark.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/deep_resnet_benchmark.py -------------------------------------------------------------------------------- /test/graphs/cifar_10.pbtxt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/graphs/cifar_10.pbtxt -------------------------------------------------------------------------------- /test/graphs/cifar_14.pbtxt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/graphs/cifar_14.pbtxt -------------------------------------------------------------------------------- /test/graphs/imagenet_101.pbtxt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/graphs/imagenet_101.pbtxt -------------------------------------------------------------------------------- /test/graphs/imagenet_200.pbtxt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/graphs/imagenet_200.pbtxt -------------------------------------------------------------------------------- /test/graphs/imagenet_34.pbtxt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/graphs/imagenet_34.pbtxt -------------------------------------------------------------------------------- /test/graphs/imagenet_50.pbtxt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/graphs/imagenet_50.pbtxt -------------------------------------------------------------------------------- /test/graphs/resnet_grad.pbtxt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/graphs/resnet_grad.pbtxt -------------------------------------------------------------------------------- /test/graphs/resnet_grad2.pbtxt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/graphs/resnet_grad2.pbtxt -------------------------------------------------------------------------------- /test/graphs/simple2_stepstats.pbtxt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/graphs/simple2_stepstats.pbtxt -------------------------------------------------------------------------------- /test/graphs/simple2_timeline_graph.pbtxt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/graphs/simple2_timeline_graph.pbtxt -------------------------------------------------------------------------------- /test/graphs/simple_stepstats.pbtxt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/graphs/simple_stepstats.pbtxt -------------------------------------------------------------------------------- /test/graphs/simple_timeline_graph.pbtxt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/graphs/simple_timeline_graph.pbtxt -------------------------------------------------------------------------------- /test/imagenet_correctness_test.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/imagenet_correctness_test.py -------------------------------------------------------------------------------- /test/imagenet_test.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/imagenet_test.py -------------------------------------------------------------------------------- /test/keras_test.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/keras_test.py -------------------------------------------------------------------------------- /test/linearize.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/linearize.py -------------------------------------------------------------------------------- /test/linearize_test.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/linearize_test.py -------------------------------------------------------------------------------- /test/mem_util.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/mem_util.py -------------------------------------------------------------------------------- /test/mem_util_test.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/mem_util_test.py -------------------------------------------------------------------------------- /test/memory_test.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/memory_test.py -------------------------------------------------------------------------------- /test/mnist_correctness_test.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/mnist_correctness_test.py -------------------------------------------------------------------------------- /test/multiple_memory_obtain_test.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/multiple_memory_obtain_test.py -------------------------------------------------------------------------------- /test/pixel_cnn_pp/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /test/pixel_cnn_pp/model.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/pixel_cnn_pp/model.py -------------------------------------------------------------------------------- /test/pixel_cnn_pp/nn.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/pixel_cnn_pp/nn.py -------------------------------------------------------------------------------- /test/pixel_cnn_test.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/pixel_cnn_test.py -------------------------------------------------------------------------------- /test/resnet_correctness_test.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/resnet_correctness_test.py -------------------------------------------------------------------------------- /test/resnet_model.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/resnet_model.py -------------------------------------------------------------------------------- /test/resnet_test.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/resnet_test.py -------------------------------------------------------------------------------- /test/run_all_tests.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/run_all_tests.sh -------------------------------------------------------------------------------- /test/simple_timeline.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/simple_timeline.png -------------------------------------------------------------------------------- /test/tf.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/tf.sh -------------------------------------------------------------------------------- /test/util.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/util.py -------------------------------------------------------------------------------- /test/util_test.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cybertronai/gradient-checkpointing/HEAD/test/util_test.py --------------------------------------------------------------------------------