├── Dockerfile ├── LICENSE ├── README.md └── entrypoint.sh /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM nvidia/cuda:11.3.0-cudnn8-devel-ubuntu20.04 2 | 3 | ENV TORCHVER=v1.8.1 4 | ENV VISIONVER=v0.9.1 5 | ENV AUDIOVER=v0.8.1 6 | 7 | ENV DEBIAN_FRONTEND=noninteractive 8 | 9 | # Install dependencies (1) 10 | RUN apt-get update && apt-get install -y \ 11 | automake autoconf libpng-dev nano wget npm \ 12 | curl zip unzip libtool swig zlib1g-dev pkg-config \ 13 | git wget xz-utils python3-mock libpython3-dev \ 14 | libpython3-all-dev python3-pip g++ gcc make \ 15 | pciutils cpio gosu git liblapack-dev liblapacke-dev 16 | 17 | # Install dependencies (2) 18 | RUN pip3 install --upgrade pip \ 19 | && pip3 install --upgrade onnx \ 20 | && pip3 install --upgrade onnxruntime \ 21 | && pip3 install --upgrade gdown \ 22 | && pip3 install cmake==3.18.4 \ 23 | && pip3 install --upgrade pyyaml \ 24 | && pip3 install --upgrade ninja \ 25 | && pip3 install --upgrade yapf \ 26 | && pip3 install --upgrade six \ 27 | && pip3 install --upgrade wheel \ 28 | && pip3 install --upgrade moc \ 29 | && ldconfig 30 | 31 | # Build 32 | # Bypass CUDA and CUB version checks. 33 | RUN git clone -b ${TORCHVER} --recursive https://github.com/pytorch/pytorch 34 | RUN cd /pytorch \ 35 | && sed -i -e "/^#ifndef THRUST_IGNORE_CUB_VERSION_CHECK$/i #define THRUST_IGNORE_CUB_VERSION_CHECK" \ 36 | /usr/local/cuda/targets/x86_64-linux/include/thrust/system/cuda/config.h \ 37 | && cat /usr/local/cuda/targets/x86_64-linux/include/thrust/system/cuda/config.h \ 38 | && sed -i -e "/^if(DEFINED GLIBCXX_USE_CXX11_ABI)/i set(GLIBCXX_USE_CXX11_ABI 1)" \ 39 | CMakeLists.txt \ 40 | && pip3 install -r requirements.txt \ 41 | && USE_NCCL=OFF python3 setup.py build \ 42 | && python3 setup.py bdist_wheel 43 | 44 | RUN git clone -b ${VISIONVER} https://github.com/pytorch/vision.git 45 | RUN cd /vision \ 46 | && git submodule update --init --recursive \ 47 | && pip3 install /pytorch/dist/*.whl \ 48 | && python3 setup.py build \ 49 | && python3 setup.py bdist_wheel 50 | 51 | RUN git clone -b ${AUDIOVER} https://github.com/pytorch/audio.git 52 | RUN cd /audio \ 53 | && git submodule update --init --recursive \ 54 | && apt-get install -y sox libsox-dev \ 55 | && python3 setup.py build \ 56 | && python3 setup.py bdist_wheel 57 | 58 | COPY entrypoint.sh /usr/local/bin/entrypoint.sh 59 | RUN chmod +x /usr/local/bin/entrypoint.sh 60 | ENTRYPOINT ["/usr/local/bin/entrypoint.sh"] 61 | 62 | WORKDIR /workspace 63 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2021 Katsuya Hyodo 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # PyTorch-build 2 | Provide Docker build sequences of PyTorch for various environments. 3 | **https://github.com/pytorch/pytorch** 4 | 5 | ## 1. Docker Image Environment 6 | - Ubuntu 20.04 x86_64 7 | - CUDA 11.3 8 | - cuDNN 8.2 9 | - PyTorch v1.8.1 (Build from source code. It will be downloaded automatically during docker build.) 10 | - TorchVision v0.9.1 11 | - TorchAudio v0.8.1 12 | 13 | ## 2. Default build parameters 14 | 15 | ``` 16 | -- 17 | -- ******** Summary ******** 18 | -- General: 19 | -- CMake version : 3.18.4 20 | -- CMake command : /usr/local/lib/python3.8/dist-packages/cmake/data/bin/cmake 21 | -- System : Linux 22 | -- C++ compiler : /usr/bin/c++ 23 | -- C++ compiler id : GNU 24 | -- C++ compiler version : 9.3.0 25 | -- CXX flags : -D_GLIBCXX_USE_CXX11_ABI=1 -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow 26 | -- Build type : Release 27 | -- Compile definitions : ONNX_ML=1;ONNXIFI_ENABLE_EXT=1;ONNX_NAMESPACE=onnx_torch;HAVE_MMAP=1;_FILE_OFFSET_BITS=64;HAVE_SHM_OPEN=1;HAVE_SHM_UNLINK=1;HAVE_MALLOC_USABLE_SIZE=1;USE_EXTERNAL_MZCRC;MINIZ_DISABLE_ZIP_READER_CRC32_CHECKS 28 | -- CMAKE_PREFIX_PATH : /usr/lib/python3/dist-packages;/usr/local/cuda;/usr/local/cuda 29 | -- CMAKE_INSTALL_PREFIX : /pytorch/torch 30 | -- 31 | -- TORCH_VERSION : 1.8.0 32 | -- CAFFE2_VERSION : 1.8.0 33 | -- BUILD_CAFFE2 : ON 34 | -- BUILD_CAFFE2_OPS : ON 35 | -- BUILD_CAFFE2_MOBILE : OFF 36 | -- BUILD_STATIC_RUNTIME_BENCHMARK: OFF 37 | -- BUILD_TENSOREXPR_BENCHMARK: OFF 38 | -- BUILD_BINARY : OFF 39 | -- BUILD_CUSTOM_PROTOBUF : ON 40 | -- Link local protobuf : ON 41 | -- BUILD_DOCS : OFF 42 | -- BUILD_PYTHON : True 43 | -- Python version : 3.8.5 44 | -- Python executable : /usr/bin/python3 45 | -- Pythonlibs version : 3.8.5 46 | -- Python library : /usr/lib/libpython3.8.so.1.0 47 | -- Python includes : /usr/include/python3.8 48 | -- Python site-packages: lib/python3/dist-packages 49 | -- BUILD_SHARED_LIBS : ON 50 | -- CAFFE2_USE_MSVC_STATIC_RUNTIME : OFF 51 | -- BUILD_TEST : True 52 | -- BUILD_JNI : OFF 53 | -- BUILD_MOBILE_AUTOGRAD : OFF 54 | -- INTERN_BUILD_MOBILE : 55 | -- USE_BLAS : 1 56 | -- BLAS : generic 57 | -- USE_LAPACK : 1 58 | -- LAPACK : generic 59 | -- USE_ASAN : OFF 60 | -- USE_CPP_CODE_COVERAGE : OFF 61 | -- USE_CUDA : ON 62 | -- Split CUDA : OFF 63 | -- CUDA static link : OFF 64 | -- USE_CUDNN : ON 65 | -- CUDA version : 11.3 66 | -- cuDNN version : 8.2.0 67 | -- CUDA root directory : /usr/local/cuda 68 | -- CUDA library : /usr/local/cuda/lib64/stubs/libcuda.so 69 | -- cudart library : /usr/local/cuda/lib64/libcudart.so 70 | -- cublas library : /usr/local/cuda/lib64/libcublas.so 71 | -- cufft library : /usr/local/cuda/lib64/libcufft.so 72 | -- curand library : /usr/local/cuda/lib64/libcurand.so 73 | -- cuDNN library : /usr/lib/x86_64-linux-gnu/libcudnn.so 74 | -- nvrtc : /usr/local/cuda/lib64/libnvrtc.so 75 | -- CUDA include path : /usr/local/cuda/include 76 | -- NVCC executable : /usr/local/cuda/bin/nvcc 77 | -- NVCC flags : -Xfatbin;-compress-all;-DONNX_NAMESPACE=onnx_torch;-gencode;arch=compute_35,code=sm_35;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_52,code=sm_52;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_86,code=compute_86;-Xcudafe;--diag_suppress=cc_clobber_ignored,--diag_suppress=integer_sign_change,--diag_suppress=useless_using_declaration,--diag_suppress=set_but_not_used,--diag_suppress=field_without_dll_interface,--diag_suppress=base_class_has_different_dll_interface,--diag_suppress=dll_interface_conflict_none_assumed,--diag_suppress=dll_interface_conflict_dllexport_assumed,--diag_suppress=implicit_return_from_non_void_function,--diag_suppress=unsigned_compare_with_zero,--diag_suppress=declared_but_not_referenced,--diag_suppress=bad_friend_decl;-std=c++14;-Xcompiler;-fPIC;--expt-relaxed-constexpr;--expt-extended-lambda;-Wno-deprecated-gpu-targets;--expt-extended-lambda;-Xcompiler;-fPIC;-DCUDA_HAS_FP16=1;-D__CUDA_NO_HALF_OPERATORS__;-D__CUDA_NO_HALF_CONVERSIONS__;-D__CUDA_NO_BFLOAT16_CONVERSIONS__;-D__CUDA_NO_HALF2_OPERATORS__ 78 | -- CUDA host compiler : /usr/bin/cc 79 | -- NVCC --device-c : OFF 80 | -- USE_TENSORRT : OFF 81 | -- USE_ROCM : OFF 82 | -- USE_EIGEN_FOR_BLAS : ON 83 | -- USE_FBGEMM : ON 84 | -- USE_FAKELOWP : OFF 85 | -- USE_KINETO : ON 86 | -- USE_FFMPEG : OFF 87 | -- USE_GFLAGS : OFF 88 | -- USE_GLOG : OFF 89 | -- USE_LEVELDB : OFF 90 | -- USE_LITE_PROTO : OFF 91 | -- USE_LMDB : OFF 92 | -- USE_METAL : OFF 93 | -- USE_PYTORCH_METAL : OFF 94 | -- USE_FFTW : OFF 95 | -- USE_MKL : OFF 96 | -- USE_MKLDNN : ON 97 | -- USE_MKLDNN_CBLAS : OFF 98 | -- USE_NCCL : OFF 99 | -- USE_SYSTEM_NCCL : OFF 100 | -- USE_NNPACK : ON 101 | -- USE_NUMPY : ON 102 | -- USE_OBSERVERS : ON 103 | -- USE_OPENCL : OFF 104 | -- USE_OPENCV : OFF 105 | -- USE_OPENMP : ON 106 | -- USE_TBB : OFF 107 | -- USE_VULKAN : OFF 108 | -- USE_PROF : OFF 109 | -- USE_QNNPACK : ON 110 | -- USE_PYTORCH_QNNPACK : ON 111 | -- USE_REDIS : OFF 112 | -- USE_ROCKSDB : OFF 113 | -- USE_ZMQ : OFF 114 | -- USE_DISTRIBUTED : ON 115 | -- USE_MPI : OFF 116 | -- USE_GLOO : ON 117 | -- USE_TENSORPIPE : ON 118 | -- USE_DEPLOY : OFF 119 | -- Public Dependencies : Threads::Threads;caffe2::mkldnn 120 | -- Private Dependencies : pthreadpool;cpuinfo;qnnpack;pytorch_qnnpack;nnpack;XNNPACK;fbgemm;fp16;gloo;tensorpipe;aten_op_header_gen;foxi_loader;rt;fmt::fmt-header-only;kineto;gcc_s;gcc;dl 121 | -- Configuring done 122 | -- Generating done 123 | -- Build files have been written to: /pytorch/build 124 | ``` 125 | 126 | ## 3. Usage - Docker Build 127 | You can customize the Dockerfile to build and run your own container images on your own. 128 | ```bash 129 | $ version=1.8.1 130 | $ git clone -b ${version} https://github.com/PINTO0309/PyTorch-build.git 131 | $ cd PyTorch-build 132 | $ docker build -t pinto0309/pytorch-build:11.3.0-cudnn8-devel-ubuntu20.04 . 133 | 134 | $ docker run --gpus all -it --rm \ 135 | -v `pwd`:/workspace \ 136 | -e LOCAL_UID=$(id -u $USER) \ 137 | -e LOCAL_GID=$(id -g $USER) \ 138 | pinto0309/pytorch-build:11.3.0-cudnn8-devel-ubuntu20.04 bash 139 | ``` 140 | 141 | ## 4. Usage - Docker Pull / Run 142 | You can download and run a pre-built container image from Docker Hub. 143 | ```bash 144 | $ docker run --gpus all -it --rm \ 145 | -v `pwd`:/workspace \ 146 | -e LOCAL_UID=$(id -u $USER) \ 147 | -e LOCAL_GID=$(id -g $USER) \ 148 | pinto0309/pytorch-build:11.3.0-cudnn8-devel-ubuntu20.04 bash 149 | ``` -------------------------------------------------------------------------------- /entrypoint.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | USER_ID=${LOCAL_UID:-62000} 4 | GROUP_ID=${LOCAL_GID:-62000} 5 | 6 | echo "Starting with UID : $USER_ID, GID: $GROUP_ID" 7 | useradd -u $USER_ID -o -m user 8 | groupmod -g $GROUP_ID user 9 | export HOME=/home/user 10 | 11 | exec /usr/sbin/gosu user "$@" --------------------------------------------------------------------------------