├── LICENSE ├── README.md ├── apt_essentials.sh ├── bitcoin_dependencies.sh ├── compile_tf.sh ├── compile_xgboost.sh ├── get_latest_anaconda.sh ├── get_latest_miniconda.sh ├── install_cuda.sh ├── install_cudnn.sh ├── install_discord.sh ├── install_mkl_dnn.sh ├── install_spotify.sh ├── python_essentials.sh └── windows ├── explorer_rightclick_bash.reg └── show_seconds_in_taskbar_clock.reg /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2018 Mikel Bober-Irizar 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # shell_scripts 2 | Various bash scripts I've written and I use for setting machines up. These have all been tested on Ubuntu. 3 | 4 | The windows/ folder includes various powershell scripts and registry edits I use. 5 | 6 | Everything in this repo is provided without warranty of any kind - I recommend reading the scripts before running them as they may not be tailored for your use case. 7 | 8 | ## get_latest_anaconda.sh 9 | Fetches the latest build of Anaconda3, installs it (without user prompts), adds it to bashrc, reloads the shell, and cleans up after itself. 10 | Basically takes you from a fresh system to one with Python/Anaconda setup and ready to use. **Also supports MacOS** 11 | 12 | Easily run from any machine: ```curl https://conda.ml | bash``` 13 | 14 | ## get_latest_miniconda.sh 15 | Same as get_latest_anaconda.sh, just installing Miniconda instead. 16 | 17 | Easily run from any machine: ```curl https://mini.conda.ml | bash``` 18 | 19 | ## install_mkl_dnn.sh 20 | Downloads, compiles and installs [mkl-dnn](https://github.com/01org/mkl-dnn). This is a requirement for using TensorFlow with MKL (eg. from compile_tf.sh) 21 | 22 | ## compile_tf.sh 23 | Installs bazel, then downloads, compiles and installs TensorFlow v1.4.1 from source. Uses CUDA and MKL (requires CUDA, CuDNN and mkl-dnn to be installed). 24 | 25 | Unlike tensorflow-gpu from pip, this will work with the non-recommended versions of CUDA and CuDNN (tested with CUDA 9.0 and CuDNN 7.0.5). This script is NOT completely unattened, one or two minutes in you will be presented with the Tensorflow interactive configuration. 26 | 27 | ## compile_xgboost.sh 28 | Downloads, compiles and installs the bleeding edge XGBoost for Python, with **CUDA and AVX support**. 29 | 30 | ## install_cuda.sh 31 | Downloads and installs CUDA 9.0 (requires NVIDIA drivers to be installed), and adds it to path. After installation, it also locks the CUDA packages to prevent it from auto-updating and breaking your kernel (as nvidia updates tend to do). 32 | 33 | ## install_cudnn.sh 34 | Installs CuDNN **from a downloaded cudnn.tgz** file into your local CUDA installation. I can't download CuDNN in the script because it's locked behind a download wall - instead the location of the downloaded file is in line 1 of the script. 35 | 36 | ## install_spotify.sh 37 | Downloads and installs Spotify. 38 | 39 | ## install_discord.sh 40 | Downloads and installs Discord. 41 | 42 | ## bitcoin_dependencies.sh 43 | Installs dependencies required to compile Bitcoin Core or most other cryptocurrency full node clients from source. 44 | 45 | ## python_essentials.sh 46 | Installs various packages which I commonly use with Python (Anaconda). Also installs MKL and updates all packages using conda. 47 | 48 | ## apt_essentials.sh 49 | Installs various linux "essentials" such as git and compilers. 50 | -------------------------------------------------------------------------------- /apt_essentials.sh: -------------------------------------------------------------------------------- 1 | sudo apt-get install -y git build-essential sshfs nmon htop pigz p7zip-full pwgen cmake screen net-tools curl wget && 2 | 3 | echo "Done! :)" 4 | -------------------------------------------------------------------------------- /bitcoin_dependencies.sh: -------------------------------------------------------------------------------- 1 | # Install various dependencies needed to compile Bitcoin Core and it's derivatives (basically every other coin's full node) 2 | 3 | sudo apt-get update && 4 | sudo apt-get install -y build-essential libtool autotools-dev automake pkg-config libssl-dev libevent-dev bsdmainutils python3 && 5 | sudo apt-get install -y libboost-system-dev libboost-filesystem-dev libboost-chrono-dev libboost-program-options-dev libboost-test-dev libboost-thread-dev && 6 | sudo apt-get install -y software-properties-common && 7 | yes | sudo add-apt-repository ppa:bitcoin/bitcoin && 8 | sudo apt-get update && 9 | sudo apt-get install -y libdb4.8-dev libdb4.8++-dev && 10 | sudo apt-get install -y libminiupnpc-dev libzmq3-dev && 11 | sudo apt-get install -y libqt5gui5 libqt5core5a libqt5dbus5 qttools5-dev qttools5-dev-tools libprotobuf-dev protobuf-compiler && 12 | 13 | wget https://download.libsodium.org/libsodium/releases/libsodium-1.0.16.tar.gz -O libsodium-1.0.16.tar.gz && 14 | tar xvf libsodium-1.0.16.tar.gz && 15 | rm libsodium-1.0.16.tar.gz && 16 | cd libsodium-1.0.16/ && 17 | ./configure && 18 | make -j8 && 19 | make check -j8 && 20 | sudo make install && 21 | cd .. && 22 | rm -rf libsodium-1.0.16/ && 23 | 24 | echo "Done! :)" 25 | -------------------------------------------------------------------------------- /compile_tf.sh: -------------------------------------------------------------------------------- 1 | echo "[STATUS] Cloning TF repo" 2 | 3 | git clone https://github.com/tensorflow/tensorflow.git && 4 | 5 | echo "[STATUS] Installing Java" 6 | 7 | sudo apt install -y openjdk-8-jdk 8 | yes | sudo add-apt-repository ppa:webupd8team/java 9 | sudo apt-get update && sudo apt-get -y install oracle-java8-installer 10 | 11 | echo "[STATUS] Installing other dependencies" 12 | 13 | sudo apt-get install -y libcupti-dev && 14 | sudo apt-get install -y pkg-config zip g++ zlib1g-dev unzip python && 15 | 16 | echo "[STATUS] Installing Bazel 0.8.1" 17 | 18 | wget https://github.com/bazelbuild/bazel/releases/download/0.8.1/bazel_0.8.1-linux-x86_64.deb -O bazel.deb && 19 | sudo dpkg -i bazel.deb && 20 | rm bazel.deb && 21 | 22 | cd tensorflow && 23 | git checkout v1.5.0 && 24 | 25 | echo "[STATUS] !!!!!!!!! TENSORFLOW CONFIGURING:" 26 | # Build with cuda support => y 27 | # Cuda version => 9.0 28 | # CuDNN version => 7.0.5 29 | # All other configure options default (this will vary depending on your system). 30 | ./configure && 31 | 32 | echo "[STATUS] Building Tensorflow!" 33 | 34 | bazel clean && 35 | bazel build --config=mkl --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package && 36 | 37 | echo "[STATUS] Installing TF Package" 38 | 39 | bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg && 40 | pip install /tmp/tensorflow_pkg/*.whl && 41 | rm /tmp/tensorflow_pkg/*.whl && 42 | 43 | echo "Done! :)" 44 | -------------------------------------------------------------------------------- /compile_xgboost.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # This installs the bleeding-edge version of xgboost with GPU support - pip install is fine if you want the latest release. 3 | 4 | NUM_THREADS=12 5 | 6 | sudo apt-get install -y cmake 7 | 8 | git clone --recursive https://github.com/dmlc/xgboost.git && 9 | cd xgboost && 10 | mkdir build && 11 | cd build && 12 | cmake .. -DUSE_CUDA=ON -DUSE_AVX=ON && 13 | 14 | make -j${NUM_THREADS} && 15 | 16 | cd ../python-package && 17 | python setup.py install && 18 | 19 | echo "Done! :)" 20 | -------------------------------------------------------------------------------- /get_latest_anaconda.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | BLUE='\033[0;34m' 4 | NC='\033[0m' 5 | 6 | echo -e "${BLUE}Installing Anaconda3. https://github.com/mxbi/shell_scripts/ ${NC}" 7 | 8 | REPO="https://repo.continuum.io/archive/" 9 | 10 | if [ "$(uname)" == "Darwin" ]; then 11 | echo "Detected MacOS" 12 | RC=".bash_profile" 13 | FILENAME="$(curl -s ${REPO} | grep -Eoi ']+>' | grep -Eo 'href="[^\"]+"' | grep -Eo '[^/"]+' | grep MacOSX-x86_64.sh | grep Anaconda3 | head -n 1)" # Do something under Mac OS X platform 14 | elif [ "$(expr substr $(uname -s) 1 5)" == "Linux" ]; then 15 | echo "Detected Linux" 16 | RC=".bashrc" 17 | FILENAME="$(curl -s ${REPO} | grep -Eoi ']+>' | grep -Eo 'href="[^\"]+"' | grep -Eo '[^/"]+' | grep Linux-x86_64.sh | grep Anaconda3 | head -n 1)" # Do something under GNU/Linux platform 18 | fi 19 | 20 | # Get filename of latest Anaconda 21 | DL_URL=$REPO$FILENAME && 22 | 23 | echo -e "${BLUE}Downloading $DL_URL${NC}" && 24 | if [ -x "$(command -v wget)" ] 25 | then 26 | wget $DL_URL -O $FILENAME 27 | else 28 | curl $DL_URL > $FILENAME 29 | fi && 30 | 31 | chmod u+x $FILENAME && 32 | 33 | echo -e "${BLUE}Installing $FILENAME${NC}" && 34 | ./$FILENAME -b -u && 35 | 36 | echo -e "${BLUE}Deleting anaconda file${NC}" && 37 | rm $FILENAME && 38 | 39 | echo -e "${BLUE}Adding to ${RC}${NC}" && 40 | echo '' >> $HOME/${RC} && 41 | echo '# Anaconda3 installation' >> $HOME/${RC} && 42 | echo 'export PATH="$HOME/anaconda3/bin:$PATH"' >> $HOME/${RC} && 43 | 44 | echo -e "${BLUE}Sourcing ${RC}${NC}" && 45 | source $HOME/${RC} && 46 | 47 | echo -e "${BLUE}DONE! :)${NC}" && 48 | python --version 49 | -------------------------------------------------------------------------------- /get_latest_miniconda.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | BLUE='\033[0;34m' 4 | NC='\033[0m' 5 | 6 | echo -e "${BLUE}Installing Miniconda3. https://github.com/mxbi/shell_scripts/ ${NC}" 7 | 8 | REPO="https://repo.continuum.io/miniconda/" 9 | 10 | if [ "$(uname)" == "Darwin" ]; then 11 | echo "Detected MacOS" 12 | RC=".bash_profile" 13 | FILENAME="$(curl -s ${REPO} | grep -Eoi ']+>' | grep -Eo 'href="[^\"]+"' | grep -Eo '[^/"]+' | grep MacOSX-x86_64.sh | grep Miniconda3 | head -n 1)" # Do something under Mac OS X platform 14 | elif [ "$(expr substr $(uname -s) 1 5)" == "Linux" ]; then 15 | echo "Detected Linux" 16 | RC=".bashrc" 17 | FILENAME="$(curl -s ${REPO} | grep -Eoi ']+>' | grep -Eo 'href="[^\"]+"' | grep -Eo '[^/"]+' | grep Linux-x86_64.sh | grep Miniconda3 | head -n 1)" # Do something under GNU/Linux platform 18 | fi 19 | 20 | # Get filename of latest Anaconda 21 | DL_URL=$REPO$FILENAME && 22 | 23 | echo -e "${BLUE}Downloading $DL_URL${NC}" && 24 | if [ -x "$(command -v wget)" ] 25 | then 26 | wget $DL_URL -O $FILENAME 27 | else 28 | curl $DL_URL > $FILENAME 29 | fi && 30 | 31 | chmod u+x $FILENAME && 32 | 33 | echo -e "${BLUE}Installing $FILENAME${NC}" && 34 | ./$FILENAME -b -u && 35 | 36 | echo -e "${BLUE}Deleting anaconda file${NC}" && 37 | rm $FILENAME && 38 | 39 | echo -e "${BLUE}Adding to ${RC}${NC}" && 40 | echo '' >> $HOME/${RC} && 41 | echo '# Miniconda3 installation' >> $HOME/${RC} && 42 | echo 'export PATH="$HOME/miniconda3/bin:$PATH"' >> $HOME/${RC} && 43 | 44 | echo -e "${BLUE}Sourcing ${RC}${NC}" && 45 | source $HOME/${RC} && 46 | 47 | echo -e "${BLUE}DONE! :)${NC}" && 48 | python --version 49 | -------------------------------------------------------------------------------- /install_cuda.sh: -------------------------------------------------------------------------------- 1 | CUDA_URL="https://developer.nvidia.com/compute/cuda/9.0/Prod/local_installers/cuda-repo-ubuntu1604-9-0-local_9.0.176-1_amd64-deb" 2 | 3 | wget $CUDA_URL -o cuda_repo.deb && 4 | 5 | sudo dpkg -i cuda_repo.deb && 6 | sudo apt-key adv --fetch-keys http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/7fa2af80.pub && 7 | sudo apt-get update && 8 | sudo apt-get install cuda && 9 | 10 | echo "CUDA installed. Preventing updates..." 11 | sudo apt-mark hold cuda && 12 | sudo apt-mark hold cuda cuda-9-0 cuda-command-line-tools-9-0 cuda-core-9-0 cuda-cublas-9-0 cuda-cublas-dev-9-0 cuda-cudart-9-0 \ 13 | cuda-cudart-dev-9-0 cuda-cufft-9-0 cuda-cufft-dev-9-0 cuda-curand-9-0 cuda-curand-dev-9-0 cuda-cusolver-9-0 cuda-cusolver-dev-9-0 \ 14 | cuda-cusparse-9-0 cuda-cusparse-dev-9-0 cuda-demo-suite-9-0 cuda-documentation-9-0 cuda-driver-dev-9-0 cuda-drivers cuda-libraries-9-0 \ 15 | cuda-libraries-dev-9-0 cuda-license-9-0 cuda-misc-headers-9-0 cuda-npp-9-0 cuda-npp-dev-9-0 cuda-nvgraph-9-0 cuda-nvgraph-dev-9-0 \ 16 | cuda-nvml-dev-9-0 cuda-nvrtc-9-0 cuda-nvrtc-dev-9-0 cuda-runtime-9-0 cuda-samples-9-0 cuda-toolkit-9-0 cuda-visual-tools-9-0 && 17 | 18 | echo "Adding to path..." 19 | echo "" >> $HOME/.bashrc 20 | echo "# CUDA (automatically added)" >> $HOME/.bashrc && 21 | echo "export PATH='/usr/local/cuda/bin/:$PATH" >> $HOME/.bashrc && 22 | echo "export LD_LIBRARY_PATH='/usr/local/cuda/lib64:$LD_LIBRARY_PATH" >> $HOME/.bashrc && 23 | sudo ldconfig && 24 | 25 | echo "Done! :)" 26 | -------------------------------------------------------------------------------- /install_cudnn.sh: -------------------------------------------------------------------------------- 1 | CUDNN_TAR = "cudnn-9.0-linux-x64-v7.tgz" 2 | 3 | tar xvf $CUDNN_TAR && 4 | sudo cp cuda/include/* /usr/local/cuda/include/ && 5 | sudo cp cuda/lib64/* /usr/local/cuda/lib64/ && 6 | 7 | rm -rf cuda && 8 | 9 | echo "Done! :)" 10 | -------------------------------------------------------------------------------- /install_discord.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | wget "https://discordapp.com/api/download?platform=linux&format=deb" -O discord.deb && 4 | 5 | sudo dpkg -i discord.deb 6 | 7 | if [[ $? -eq 1 ]] 8 | then 9 | echo "Failed to install discord, installing missing dependencies." 10 | sudo apt-get install -f -y && 11 | sudo dpkg -i discord.deb 12 | fi && 13 | 14 | rm discord.deb && 15 | 16 | echo "Done :)" 17 | -------------------------------------------------------------------------------- /install_mkl_dnn.sh: -------------------------------------------------------------------------------- 1 | # More info: https://github.com/01org/mkl-dnn#installation 2 | 3 | NUM_THREADS=12 4 | 5 | git clone https://github.com/01org/mkl-dnn.git && 6 | cd mkl-dnn && 7 | git checkout v0.12 && 8 | 9 | sudo apt-get install -y cmake && 10 | 11 | cd scripts && 12 | ./prepare_mkl.sh && 13 | cd .. && 14 | 15 | mkdir -p build && 16 | cd build && 17 | cmake .. && 18 | make -j${NUM_THREADS} && 19 | make -j12 test && 20 | sudo make install && 21 | sudo ldconfig && 22 | 23 | echo "Done! :)" 24 | -------------------------------------------------------------------------------- /install_spotify.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 0DF731E45CE24F27EEEB1450EFDC8610341D9410 && 4 | echo deb http://repository.spotify.com stable non-free | sudo tee /etc/apt/sources.list.d/spotify.list && 5 | sudo apt-get update && 6 | sudo apt-get -y install spotify-client && 7 | 8 | echo "Done! :)" 9 | -------------------------------------------------------------------------------- /python_essentials.sh: -------------------------------------------------------------------------------- 1 | pip install tqdm xgboost pathos ml_metrics mlcrate && 2 | yes | conda install -c menpo opencv && 3 | yes | conda install mkl && 4 | yes | conda update --all && 5 | 6 | echo "Done! :)" 7 | -------------------------------------------------------------------------------- /windows/explorer_rightclick_bash.reg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mxbi/shell_scripts/HEAD/windows/explorer_rightclick_bash.reg -------------------------------------------------------------------------------- /windows/show_seconds_in_taskbar_clock.reg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mxbi/shell_scripts/HEAD/windows/show_seconds_in_taskbar_clock.reg --------------------------------------------------------------------------------