├── .github └── FUNDING.yml ├── LICENSE ├── README.md ├── debian-2-proxmox ├── debian10-2-proxmox6.sh ├── debian11-2-proxmox7.sh └── debian9-2-proxmox5.sh ├── helpers ├── pve-edege-kernel.sh └── pve-enable-lxc-docker.sh ├── hetzner ├── README.md ├── installimage-proxmox.sh ├── pbs ├── pve └── vnc-install-proxmox.sh ├── install-post.sh ├── networking ├── README.md ├── network-addiprange.sh ├── network-configure.sh └── tincvpn.sh ├── nvidia ├── nvidia-docker.sh └── readme.md ├── ovh └── README.md ├── xs-install-post.env.sample └── zfs ├── README.md ├── benchmark_zfs.sh ├── createzfs.sh ├── lvm-2-zfs.sh └── xshok_slog_cache-2-zfs.sh /.github/FUNDING.yml: -------------------------------------------------------------------------------- 1 | # These are supported funding model platforms 2 | 3 | github: extremeshok 4 | custom: ['https://paypal.me/AdrianKriel', 'https://www.extremeshok.com'] 5 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | This is property of eXtremeSHOK.com 2 | You are free to use, modify and distribute, however you may not remove this notice. 3 | Copyright (c) Adrian Jon Kriel :: admin@extremeshok.com 4 | License: BSD (Berkeley Software Distribution) 5 | 6 | All rights reserved. 7 | 8 | Redistribution and use in source and binary forms, with or without modification, 9 | are permitted provided that the following conditions are met: 10 | 11 | * Redistributions of source code must retain the above copyright 12 | notice, this list of conditions and the following disclaimer. 13 | 14 | * Redistributions in binary form must reproduce the above copyright 15 | notice, this list of conditions and the following disclaimer in the 16 | documentation and/or other materials provided with the distribution. 17 | 18 | * Neither the name of the author/copyright holder nor the names of its 19 | contributors may be used to endorse or promote products derived from 20 | this software without specific prior written permission. 21 | 22 | THIS SOFTWARE IS PROVIDED BY AUTHOR/COPYRIGHT HOLDER "AS IS" AND ANY EXPRESS OR 23 | IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF 24 | MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT 25 | SHALL AUTHOR/COPYRIGHT HOLDER BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 26 | SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, 27 | PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR 28 | BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN 29 | CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING 30 | IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY 31 | OF SUCH DAMAGE. 32 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # xshok-proxmox :: eXtremeSHOK.com Proxmox (pve) 2 | 3 | Scripts for working with and optimizing proxmox 4 | 5 | ## Maintained and provided by 6 | 7 | ### Please Submit Patches / Pull requests 8 | 9 | ## Optimization / Post Install Script (install-post.sh aka postinstall.sh) *run once* 10 | Turns a fresh proxmox install into an optimised proxmox host 11 | *not required if server setup with installimage-proxmox.sh* 12 | 13 | 'reboot-quick' command which uses kexec to boot the latest kernel, its a fast method of rebooting, without needing to do a hardware reboot 14 | 15 | * Disable the enterprise repo, enable the public repo, Add non-free sources 16 | * Fixes known bugs (public key missing, max user watches, etc) 17 | * Update the system 18 | * Detect AMD EPYC CPU and Apply Fixes 19 | * Force APT to use IPv4 20 | * Update proxmox and install various system utils 21 | * Customise bashrc 22 | * add the latest ceph provided by proxmox 23 | * Disable portmapper / rpcbind (security) 24 | * Ensure Entropy Pools are Populated, prevents slowdowns whilst waiting for entropy 25 | * Protect the web interface with fail2ban 26 | * Detect if is running in a virtual machine and install the relavant guest agent 27 | * Install ifupdown2 for a virtual internal network allows rebootless networking changes (not compatible with openvswitch-switch) 28 | * Limit the size and optimise journald 29 | * Install kernel source headers 30 | * Install kexec, allows for quick reboots into the latest updated kernel set as primary in the boot-loader. 31 | * Ensure ksmtuned (ksm-control-daemon) is enabled and optimise according to ram size 32 | * Set language, if chnaged will disable XS_NOAPTLANG 33 | * Increase max user watches, FD limit, FD ulimit, max key limit, ulimits 34 | * Optimise logrotate 35 | * Lynis security scan tool by Cisofy 36 | * Increase Max FS open files 37 | * Optimise Memory 38 | * Pretty MOTD BANNER 39 | * Enable Network optimising 40 | * Save bandwidth and skip downloading additional languages, requires XS_LANG="en_US.UTF-8" 41 | * Disable enterprise proxmox repo 42 | * Remove subscription banner 43 | * Install openvswitch for a virtual internal network 44 | * Detect if this is an OVH server and install OVH Real Time Monitoring 45 | * Set pigz to replace gzip, 2x faster gzip compression 46 | * Bugfix: high swap usage with low memory usage 47 | * Enable TCP BBR congestion control 48 | * Enable TCP fastopen 49 | * Enable testing proxmox repo 50 | * Automatically Synchronize the time 51 | * Set Timezone, empty = set automatically by IP 52 | * Install common system utilities 53 | * Increase vzdump backup speed 54 | * Optimise ZFS arc size accoring to memory size 55 | * Install zfs-auto-snapshot 56 | 57 | https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/install-post.sh 58 | 59 | return value is 0 60 | 61 | ``` 62 | wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/install-post.sh -c -O install-post.sh && bash install-post.sh && rm install-post.sh 63 | ``` 64 | 65 | ## TO SET AND USE YOUR OWN OPTIONS (using xs-post-install.env) 66 | User Defined Options for (install-post.sh) post-installation script for Proxmox are set in the xs-install-post.env, see the sample : xs-install-post.env.sample 67 | ``` 68 | wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/xs-install-post.env.sample -c -O xs-install-post.env 69 | wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/install-post.sh -c -O install-post.sh 70 | nano xs-install-post.env 71 | bash install-post.sh 72 | ``` 73 | ## TO SET AND USE YOUR OWN OPTIONS (using ENV) 74 | Examnple to disable the MOTD banner 75 | ``` 76 | wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/install-post.sh -c -O install-post.sh 77 | export XS_MOTD="no" 78 | bash install-post.sh 79 | ``` 80 | 81 | ## Install Proxmox Recommendations 82 | Recommeneded partitioning scheme: 83 | * Raid 1 (mirror) 40 000MB ext4 / 84 | * Raid 1 (mirror) 30 000MB ext4 /xshok/zfs-cache *only create if an ssd and there is 1+ unused hdd which will be made into a zfspool* 85 | * Raid 1 (mirror) 5 000MB ext4 /xshok/zfs-slog *only create if an ssd and there is 1+ unused hdd which will be made into a zfspool* 86 | * SWAP 87 | * * HDD less than 130gb = 16GB swap 88 | * * HDD more than 130GB and RAM less than 64GB = 32GB swap 89 | * * HDD more than 130GB and RAM more than 64GB = 64GB swap 90 | * Remaining for lv xfs /var/lib/vz (LVM) 91 | 92 | # Hetzner Proxmox Installation Guide # 93 | see *hetzner* folder 94 | 95 | # OVH Proxmox Installation Guide # 96 | see *ovh* folder 97 | 98 | # ------- SCRIPTS ------ 99 | 100 | ## Convert from Debian 11 to Proxmox 7 (debian11-2-proxmox7.sh) *optional* 101 | Assumptions: Debian11 installed with a valid FQDN hostname set 102 | * Tested on KVM, VirtualBox and Dedicated Server 103 | * Will automatically detect cloud-init and disable. 104 | * Will automatically generate a correct /etc/hosts 105 | * Note: will automatically run the install-post.sh script 106 | ``` 107 | curl -O https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/debian-2-proxmox/debian11-2-proxmox7.sh && chmod +x debian11-2-proxmox7.sh 108 | ./debian11-2-proxmox7.sh 109 | ``` 110 | ## Convert from Debian 10 to Proxmox 6 (debian10-2-proxmox6.sh) *optional* 111 | ``` 112 | curl -O https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/debian-2-proxmox/debian10-2-proxmox6.sh && chmod +x debian10-2-proxmox6.sh 113 | ./debian10-2-proxmox6.sh 114 | ``` 115 | ## Convert from Debian 9 to Proxmox 5 (debian9-2-proxmox5.sh) *optional* 116 | ``` 117 | curl -O https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/debian-2-proxmox/debian9-2-proxmox5.sh && chmod +x debian9-2-proxmox5.sh 118 | ./debian9-2-proxmox5.sh 119 | ``` 120 | 121 | ## Enable Docker support for an LXC container (pve-enable-lxc-docker.sh) *optional* 122 | There can be security implications as the LXC container is running in a higher privileged mode. 123 | ``` 124 | curl https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/helpers/pve-enable-lxc-docker.sh --output /usr/sbin/pve-enable-lxc-docker && chmod +x /usr/sbin/pve-enable-lxc-docker 125 | pve-enable-lxc-docker container_id 126 | ``` 127 | 128 | ## Convert from LVM to ZFS (lvm-2-zfs.sh) *run once* 129 | Converts the a MDADM BASED LVM into a ZFS raid 1 (mirror) 130 | * Defaults to mount point: /var/lib/vz 131 | * Optional: specify the LVM_MOUNT_POINT ( ./lvm-2-zfs.sh LVM_MOUNT_POINT ) 132 | * Creates the following storage/rpools 133 | * zfsbackup (rpool/backup) 134 | * zfsvmdata (rpool/vmdata) 135 | * /var/lib/vz/tmp_backup (rpool/tmp_backup) 136 | * 137 | * Will automatically detect the required raid level and optimise. 138 | * 1 Drive = zfs 139 | * 2 Drives = mirror 140 | * 3-5 Drives = raidz-1 141 | * 6-11 Drives = raidz-2 142 | * 11+ Drives = raidz-3 143 | 144 | **NOTE: WILL DESTROY ALL DATA ON LVM_MOUNT_POINT** 145 | ``` 146 | wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/zfs/lvm-2-zfs.sh -c -O lvm-2-zfs.sh && chmod +x lvm-2-zfs.sh 147 | ./lvm-2-zfs.sh 148 | ``` 149 | 150 | ## Create ZFS from devices (createzfs.sh) *optional* 151 | Creates a zfs pool from specified devices 152 | * Will automatically detect the required raid level and optimise 153 | * 1 Drive = zfs (single) 154 | * 2 Drives = mirror (raid1) 155 | * 3-5 Drives = raidz-1 (raid5) 156 | * 6-11 Drives = raidz-2 (raid6) 157 | * 11+ Drives = raidz-3 (raid7) 158 | 159 | **NOTE: WILL DESTROY ALL DATA ON SPECIFIED DEVICES** 160 | ``` 161 | wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/zfs/createzfs.sh -c -O createzfs.sh && chmod +x createzfs.sh 162 | ./createzfs.sh poolname /dev/device1 /dev/device2 163 | ``` 164 | 165 | ## Create ZFS cache and slog from /xshok/zfs-cache and /xshok/zfs-slog partitions and adds them to a zpool (xshok_slog_cache-2-zfs.sh) *optional* 166 | Creates a zfs pool from specified devices 167 | * Will automatically mirror the slog and stripe the cache if there are multiple drives 168 | 169 | **NOTE: WILL DESTROY ALL DATA ON SPECIFIED PARTITIONS** 170 | ``` 171 | wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/zfs/xshok_slog_cache-2-zfs.sh -c -O xshok_slog_cache-2-zfs.sh && chmod +x xshok_slog_cache-2-zfs.sh 172 | ./xshok_slog_cache-2-zfs.sh poolname 173 | ``` 174 | 175 | ## CREATES A ROUTED vmbr0 AND NAT vmbr1 NETWORK CONFIGURATION FOR PROXMOX (network-configure.sh) **run once** 176 | Autodetects the correct settings (interface, gatewat, netmask, etc) 177 | Supports IPv4 and IPv6, Private Network uses 10.10.10.1/24 178 | Also installs and properly configures the isc-dhcp-server to allow for DHCP on the vmbr1 (NAT) 179 | ROUTED (vmbr0): 180 | All traffic is routed via the main IP address and uses the MAC address of the physical interface. 181 | VM's can have multiple IP addresses and they do NOT require a MAC to be set for the IP via service provider 182 | 183 | NAT (vmbr1): 184 | Allows a VM to have internet connectivity without requiring its own IP address 185 | Assignes 10.10.10.100 - 10.10.10.200 via DHCP 186 | 187 | Public IP's can be assigned via DHCP, adding a host define to the /etc/dhcp/hosts.public file 188 | 189 | Tested on OVH and Hetzner based servers 190 | 191 | ALSO CREATES A NAT Private Network as vmbr1 192 | 193 | NOTE: WILL OVERWRITE /etc/network/interfaces 194 | A backup will be created as /etc/network/interfaces.timestamp 195 | ``` 196 | wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/networking/network-configure.sh -c -O network-configure.sh && chmod +x network-configure.sh 197 | ./network-configure.sh && rm network-configure.sh 198 | ``` 199 | 200 | ## Creates default routes to allow for extra ip ranges to be used (network-addiprange.sh) *optional* 201 | If no interface is specified the default gateway interface will be detected and used. 202 | ``` 203 | wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/networking/network-addiprange.sh -c -O network-addiprange.sh && chmod +x network-addiprange.sh 204 | ./network-addiprange.sh ip.xx.xx.xx/cidr interface_optional 205 | ``` 206 | 207 | ## Create Private mesh vpn/network (tincvpn.sh) 208 | tinc private mesh vpn/network which supports multicast, ideal for private cluster communication 209 | ``` 210 | wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/networking/tincvpn.sh -c -O tincvpn.sh && chmod +x tincvpn.sh 211 | ./tincvpn.sh -h 212 | ``` 213 | ### Example for 3 node Cluster 214 | # cat /etc/hosts 215 | # global ips for tinc servers 216 | # 11.11.11.11 host1 217 | # 22.22.22.22 host2 218 | # 33.33.33.33 host3 219 | #### First Host (hostname: host1) 220 | ``` 221 | bash tincvpn.sh -i 1 -c host2 222 | ``` 223 | #### Second Host (hostname: host2) 224 | ``` 225 | bash tincvpn.sh -i 2 -c host3 226 | ``` 227 | #### Third Host (hostname: host3) 228 | ``` 229 | bash tincvpn.sh -i 3 -c host1 230 | ``` 231 | 232 | # NOTES 233 | 234 | ## Alpine Linux KVM / Qemu Agent Client Fix 235 | Run the following on the guest alpine linux 236 | ``` 237 | apk update && apk add qemu-guest-agent acpi 238 | echo 'GA_PATH="/dev/vport2p1"' >> /etc/conf.d/qemu-guest-agent 239 | rc-update add qemu-guest-agent default 240 | rc-update add acpid default 241 | /etc/init.d/qemu-guest-agent restart 242 | ``` 243 | 244 | ## Proxmox ACME / Letsencrypt 245 | Run the following on the proxmox server, ensure you have a valid DNS for the server which resolves 246 | ``` 247 | pvenode acme account register default mail@example.invalid 248 | pvenode config set --acme domains=example.invalid 249 | pvenode acme cert order 250 | ``` 251 | 252 | ## ZFS Snapshot Usage 253 | ``` 254 | # list all snapshots 255 | zfs list -t snapshot 256 | # create a pre-rollback snapshot 257 | zfs-auto-snapshot --verbose --label=prerollback -r // 258 | # rollback to a specific snapshot 259 | zfs rollback 260 | ``` 261 | -------------------------------------------------------------------------------- /debian-2-proxmox/debian10-2-proxmox6.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | ################################################################################ 3 | # This is property of eXtremeSHOK.com 4 | # You are free to use, modify and distribute, however you may not remove this notice. 5 | # Copyright (c) Adrian Jon Kriel :: admin@extremeshok.com 6 | ################################################################################ 7 | # 8 | # Script updates can be found at: https://github.com/extremeshok/xshok-proxmox 9 | # 10 | # Debian 10 to Proxmox 6 conversion script 11 | # 12 | # License: BSD (Berkeley Software Distribution) 13 | # 14 | ################################################################################ 15 | # 16 | # Assumptions: Debian10 installed with a valid FQDN hostname set 17 | # 18 | # Tested on KVM, VirtualBox and Dedicated Server 19 | # 20 | # Will automatically detect cloud-init and disable. 21 | # Will automatically generate a correct /etc/hosts 22 | # 23 | # Note: will automatically run the install-post.sh script 24 | # 25 | # Thank you @floco 26 | # 27 | # Usage: 28 | # curl -O https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/debian-2-proxmox/debian10-2-proxmox6.sh && chmod +x debian10-2-proxmox6.sh 29 | # ./debian10-2-proxmox6.sh 30 | # 31 | # 32 | ################################################################################ 33 | # 34 | # THERE ARE NO USER CONFIGURABLE OPTIONS IN THIS SCRIPT 35 | # 36 | ################################################################################ 37 | 38 | #todo : verify and check 39 | 40 | # Set the local 41 | export LANG="en_US.UTF-8" 42 | export LC_ALL="C" 43 | sh -c "echo -e 'LANG=en_US.UTF-8\nLC_ALL=en_US.UTF-8' > /etc/default/locale" 44 | 45 | #create lock dir for aptitude 46 | if [ -d "/run/lock" ] ; then 47 | mkdir /run/lock 48 | chmod a+rwxt /run/lock 49 | fi 50 | 51 | echo "Deinstalling any linux firmware packages " 52 | firmware="$(dpkg -l | grep -i 'firmware-')" 53 | if [ -n "$firmware" ]; then 54 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' purge firmware-bnx2x firmware-realtek firmware-linux firmware-linux-free firmware-linux-nonfree 55 | else 56 | echo "No firmware packages loaded" 57 | fi 58 | 59 | echo "Deinstalling the Debian standard kernel packages " 60 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' purge linux-image-amd64 61 | 62 | echo "Removing conflicting packages" 63 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' purge os-prober 64 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' autoremove 65 | apt-get clean all 66 | 67 | echo "Auto detecting existing network settings" 68 | default_interface="$(ip route | awk '/default/ { print $5 }' | grep -v "vmbr")" 69 | if [ "$default_interface" == "" ]; then 70 | #filter the interfaces to get the default interface and which is not down and not a virtual bridge 71 | default_interface="$(ip link | sed -e '/state DOWN / { N; d; }' | sed -e '/veth[0-9].*:/ { N; d; }' | sed -e '/vmbr[0-9].*:/ { N; d; }' | sed -e '/tap[0-9].*:/ { N; d; }' | sed -e '/lo:/ { N; d; }' | head -n 1 | cut -d':' -f 2 | xargs)" 72 | fi 73 | if [ "$default_interface" == "" ]; then 74 | echo "ERROR: Could not detect default interface" 75 | exit 1 76 | fi 77 | default_v4="$(ip -4 addr show dev "$default_interface" | awk '/inet/ { print $2 }' )" 78 | default_v4ip=${default_v4%/*} 79 | if [ "$default_v4ip" == "" ] ; then 80 | echo "ERROR: Could not detect default IPv4 address" 81 | echo "IP: ${default_v4ip}" 82 | exit 1 83 | fi 84 | 85 | echo "Configure /etc/hosts" 86 | if [ -f /etc/cloud/cloud.cfg ] ; then 87 | echo 'manage_etc_hosts: False' | tee --append /etc/cloud/cloud.cfg 88 | fi 89 | sed -i "s/^ - update_etc_hosts/# - update_etc_hosts/" /etc/cloud/cloud.cfg 90 | cat < /etc/hosts 91 | 127.0.0.1 localhost.localdomain localhost 92 | ${default_v4ip} $(hostname -f) $(hostname -s) pvelocalhost 93 | # The following lines are desirable for IPv6 capable hosts 94 | ::1 ip6-localhost ip6-loopback 95 | fe00::0 ip6-localnet 96 | ff00::0 ip6-mcastprefix 97 | ff02::1 ip6-allnodes 98 | ff02::2 ip6-allrouters 99 | ff02::3 ip6-allhosts 100 | EOF 101 | 102 | echo "Add Proxmox repo to APT sources" 103 | cat <> /etc/apt/sources.list.d/proxmox.list 104 | # PVE packages provided by proxmox.com" 105 | deb http://download.proxmox.com/debian/pve buster pve-no-subscription 106 | EOF 107 | wget -q "http://download.proxmox.com/debian/proxmox-ve-release-6.x.gpg" -O /etc/apt/trusted.gpg.d/proxmox-ve-release-6.x.gpg 108 | apt-get update > /dev/null 109 | 110 | echo "Upgrading system" 111 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' dist-upgrade 112 | 113 | echo "Installing postfix" 114 | cat < /etc/default/locale" 44 | 45 | #create lock dir for aptitude 46 | if [ -d "/run/lock" ] ; then 47 | mkdir /run/lock 48 | chmod a+rwxt /run/lock 49 | fi 50 | 51 | echo "Deinstalling any linux firmware packages " 52 | firmware="$(dpkg -l | grep -i 'firmware-')" 53 | if [ -n "$firmware" ]; then 54 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' purge firmware-bnx2x firmware-realtek firmware-linux firmware-linux-free firmware-linux-nonfree 55 | else 56 | echo "No firmware packages loaded" 57 | fi 58 | 59 | echo "Deinstalling the Debian standard kernel packages " 60 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' purge linux-image-amd64 61 | 62 | echo "Removing conflicting packages" 63 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' purge os-prober 64 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' autoremove 65 | apt-get clean all 66 | 67 | echo "Auto detecting existing network settings" 68 | default_interface="$(ip route | awk '/default/ { print $5 }' | grep -v "vmbr")" 69 | if [ "$default_interface" == "" ]; then 70 | #filter the interfaces to get the default interface and which is not down and not a virtual bridge 71 | default_interface="$(ip link | sed -e '/state DOWN / { N; d; }' | sed -e '/veth[0-9].*:/ { N; d; }' | sed -e '/vmbr[0-9].*:/ { N; d; }' | sed -e '/tap[0-9].*:/ { N; d; }' | sed -e '/lo:/ { N; d; }' | head -n 1 | cut -d':' -f 2 | xargs)" 72 | fi 73 | if [ "$default_interface" == "" ]; then 74 | echo "ERROR: Could not detect default interface" 75 | exit 1 76 | fi 77 | default_v4="$(ip -4 addr show dev "$default_interface" | awk '/inet/ { print $2 }' )" 78 | default_v4ip=${default_v4%/*} 79 | if [ "$default_v4ip" == "" ] ; then 80 | echo "ERROR: Could not detect default IPv4 address" 81 | echo "IP: ${default_v4ip}" 82 | exit 1 83 | fi 84 | 85 | echo "Configure /etc/hosts" 86 | if [ -f /etc/cloud/cloud.cfg ] ; then 87 | echo 'manage_etc_hosts: False' | tee --append /etc/cloud/cloud.cfg 88 | fi 89 | sed -i "s/^ - update_etc_hosts/# - update_etc_hosts/" /etc/cloud/cloud.cfg 90 | cat < /etc/hosts 91 | 127.0.0.1 localhost.localdomain localhost 92 | ${default_v4ip} $(hostname -f) $(hostname -s) pvelocalhost 93 | # The following lines are desirable for IPv6 capable hosts 94 | ::1 ip6-localhost ip6-loopback 95 | fe00::0 ip6-localnet 96 | ff00::0 ip6-mcastprefix 97 | ff02::1 ip6-allnodes 98 | ff02::2 ip6-allrouters 99 | ff02::3 ip6-allhosts 100 | EOF 101 | 102 | echo "Add Proxmox repo to APT sources" 103 | cat <> /etc/apt/sources.list.d/proxmox.list 104 | # PVE packages provided by proxmox.com" 105 | deb [arch=amd64] http://download.proxmox.com/debian/pve bullseye pve-no-subscription 106 | EOF 107 | wget -q "https://enterprise.proxmox.com/debian/proxmox-release-bullseye.gpg" -O /etc/apt/trusted.gpg.d/proxmox-release-bullseye.gpg 108 | apt-get update > /dev/null 109 | 110 | echo "Upgrading system" 111 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' full-upgrade 112 | 113 | echo "Installing postfix" 114 | cat < /etc/hosts 84 | 127.0.0.1 localhost.localdomain localhost 85 | ${default_v4ip} $(hostname -f) $(hostname) pvelocalhost 86 | 87 | # The following lines are desirable for IPv6 capable hosts 88 | 89 | ::1 ip6-localhost ip6-loopback 90 | fe00::0 ip6-localnet 91 | ff00::0 ip6-mcastprefix 92 | ff02::1 ip6-allnodes 93 | ff02::2 ip6-allrouters 94 | ff02::3 ip6-allhosts 95 | EOF 96 | 97 | echo "Add Proxmox repo to APT sources" 98 | cat <> /etc/apt/sources.list.d/proxmox.list 99 | 100 | # PVE packages provided by proxmox.com" 101 | deb http://mirror.hetzner.de/debian/pve stretch pve-no-subscription 102 | deb http://download.proxmox.com/debian/pve stretch pve-no-subscription 103 | EOF 104 | wget -q "http://download.proxmox.com/debian/proxmox-ve-release-5.x.gpg" -O /etc/apt/trusted.gpg.d/proxmox-ve-release-5.x.gpg 105 | apt-get update > /dev/null 106 | 107 | echo "Upgrading system" 108 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' dist-upgrade 109 | 110 | echo "Installing postfix" 111 | cat < $filename" 54 | echo "$linecontent" >> "$filename" 55 | fi 56 | } 57 | 58 | #add cgroups support 59 | if [ "$(command -v cgroupfs-mount)" == "" ] ; then 60 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install cgroupfs-mount 61 | fi 62 | 63 | if [ -f "$container_config" ]; then 64 | 65 | addlineifnotfound "$container_config" "lxc.apparmor.profile: unconfined" 66 | addlineifnotfound "$container_config" "lxc.cgroup.devices.allow: a" 67 | addlineifnotfound "$container_config" "lxc.cap.drop:" 68 | addlineifnotfound "$container_config" "linux.kernel_modules: aufs ip_tables" 69 | addlineifnotfound "$container_config" "lxc.mount.auto: proc:rw sys:rw" 70 | 71 | #pve is missing the lxc binary 72 | #lxc config set "$container_id" security.nesting true 73 | #lxc config set "$container_id" security.privileged true 74 | #lxc restart "$container_id" 75 | 76 | #pve lxc container restart 77 | lxc-stop --name "$container_id" 78 | lxc-start --name "$container_id" 79 | 80 | echo "Docker support added to $container_id" 81 | 82 | else 83 | echo "Error: Config $container_config could not be found" 84 | exit 1 85 | fi 86 | -------------------------------------------------------------------------------- /hetzner/README.md: -------------------------------------------------------------------------------- 1 | # Hetzner Proxmox Installation Guide # 2 | 3 | ## Assumptions: 4 | Run this script from the hetzner rescue system 5 | Operating system=Linux, Architecture=64 bit, Public key=*optional* 6 | 7 | Will automatically detect nvme, ssd and hdd and configure accordingly. 8 | 9 | # Semi-Automated Using VNC 10 | 11 | ## VNC Install (Native install Proxmox from ISO on systems without ipmi) 12 | ## Notes: 13 | Will automatically detect nvme, ssd and hdd and configure accordingly. 14 | sata ssd is used (boot and root) instead of nvme 15 | will use nvme, if sda is a spinning disk 16 | 17 | ### Proxmox VE (PVE) 18 | ``` 19 | curl -O https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/hetzner/vnc-install-proxmox.sh && chmod +x vnc-install-proxmox.sh 20 | ./vnc-install-proxmox.sh 21 | ``` 22 | 23 | ### Proxmox Backup Server (PBS) 24 | ``` 25 | curl -O https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/hetzner/vnc-install-proxmox.sh && chmod +x vnc-install-proxmox.sh 26 | ./vnc-install-proxmox.sh pbs 27 | ``` 28 | 29 | # Automated Using Installimage 30 | 31 | ## Notes: 32 | ext3 boot partition of 1GB 33 | ext4 root partition adjusted according to available drive space, upto 128GB 34 | 35 | sata ssd is used (boot and root) instead of nvme 36 | will use nvme as target, if sda is a spinning disk 37 | slog and L2ARC if nvme is present, no ssd and hdd is present 38 | slog and L2ARC if ssd is present, no nvme and hdd is present 39 | 40 | *includes and runs the (install-post.sh) script* 41 | * Select the Rescue tab for the specific server, via the hetzner robot manager 42 | * * Operating system=Linux 43 | * * Architecture=64 bit 44 | * * Public key=*optional* 45 | * --> Activate rescue system 46 | * Select the Reset tab for the specific server, 47 | * Check: Execute an automatic hardware reset 48 | * --> Send 49 | * Wait a few mins 50 | * Connect via ssh/terminal to the rescue system running on your server and run either of the following 51 | * To Install Proxmox VE (PVE) 52 | ```` 53 | wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/hetzner/installimage-proxmox.sh -c -O installimage-proxmox.sh && chmod +x installimage-proxmox.sh 54 | ./installimage-proxmox.sh "your.hostname.here" 55 | ```` 56 | * To Install Proxmox Backup Server (PBS) 57 | ```` 58 | wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/hetzner/installimage-proxmox.sh -c -O installimage-proxmox.sh && chmod +x installimage-proxmox.sh 59 | ./installimage-proxmox.sh "your.hostname.here" pbs 60 | ```` 61 | * Reboot 62 | * Connect via ssh/terminal to the new Proxmox system running on your server and run the following 63 | 64 | ## LVM to ZFS 65 | ```` 66 | wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/zfs/lvm-2-zfs.sh -c -O lvm-2-zfs.sh && chmod +x lvm-2-zfs.sh 67 | ./lvm-2-zfs.sh && rm lvm-2-zfs.sh 68 | ```` 69 | * Reboot 70 | * Connect via ssh/terminal to the new Proxmox system running on your server and run the following 71 | 72 | ## NETWORKING (vmbr0 vmbr1) 73 | ``` 74 | wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/networking/network-configure.sh -c -O network-configure.sh && chmod +x network-configure.sh 75 | ./network-configure.sh && rm network-configure.sh 76 | ``` 77 | * Reboot 78 | * Post Install: Now login via ssh as root and create a password, which will be used for the webinterface when logging in with pam authentication 79 | 80 | ## OPTIONAL: POST INSTALL OPTIMISATION 81 | ``` 82 | wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/install-post.sh -c -O install-post.sh && chmod +x install-post.sh 83 | ./install-post.sh && rm install-post.sh 84 | -------------------------------------------------------------------------------- /hetzner/installimage-proxmox.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | ################################################################################ 3 | # This is property of eXtremeSHOK.com 4 | # You are free to use, modify and distribute, however you may not remove this notice. 5 | # Copyright (c) Adrian Jon Kriel :: admin@extremeshok.com 6 | ################################################################################ 7 | # 8 | # Script updates can be found at: https://github.com/extremeshok/xshok-proxmox 9 | # 10 | # hetzner installation script for Proxmox v2 11 | # 12 | # License: BSD (Berkeley Software Distribution) 13 | # 14 | ############################################################################## 15 | # Usage : 16 | ########## Proxmox VE 17 | # vnc-install-proxmox.sh hostname.fqd.com 18 | ########## Backup Server 19 | # vnc-install-proxmox.sh hostname.fqd.com pbs 20 | # 21 | ############################################################################### 22 | # Assumptions: 23 | # Run this script from the hetzner rescue system 24 | # Operating system=Linux, Architecture=64 bit, Public key=*optional* 25 | # 26 | # Will automatically detect nvme, ssd and hdd and configure accordingly. 27 | # 28 | # Notes: 29 | # ext3 boot partition of 1GB 30 | # ext4 root partition adjusted according to available drive space, upto 128GB 31 | # 32 | # sata ssd is used (boot and root) instead of nvme 33 | # will use nvme as target, if sda is a spinning disk 34 | # slog and L2ARC if nvme is present, no ssd and hdd is present 35 | # slog and L2ARC if ssd is present, no nvme and hdd is present 36 | # 37 | ################################################################################ 38 | # 39 | # ALL CONFIGURATION OPTIONS ARE LOCATED BELOW THIS MESSAGE 40 | # 41 | ################################################################################ 42 | # Ensure NVME devices use 4K block size and not 512 block size, can cause problems with some devices 43 | NVME_FORCE_4K="FALSE" 44 | # Will create a new GPT partition table on the install target drives. 45 | # this will wipe all patition information on the drives 46 | WIPE_PARTITION_TABLE="TRUE" 47 | # FQDN Hostname 48 | MY_HOSTNAME="" 49 | # Select the OS to install "PVE" "PBS", default is PVE 50 | MY_OS="" 51 | #set size of boot partition or leave blank for autoconfig, will be in gbytes, 1GB or larger 52 | MY_BOOT="" 53 | #set size of root partition, will be in gbytes, 10GB or larger 54 | MY_ROOT="" 55 | #set size of swap partition or leave blank for autoconfig, USE NUMBER ONLY, will be in gbytes, 0 to disable 56 | MY_SWAP="" 57 | #set size of L2ARC partition or leave blank for autoconfig, USE NUMBER ONLY, will be in gbytes, 0 to disable 58 | MY_ZFS_L2ARC="" 59 | #set size of slog partition or leave blank for autoconfig, USE NUMBER ONLY, will be in gbytes, 0 to disable 60 | MY_ZFS_SLOG="" 61 | ################################################################################ 62 | 63 | # Set the local 64 | export LANG="en_US.UTF-8" 65 | export LC_ALL="C" 66 | 67 | # Reconnect to screen, incase of a disconnect 68 | screen -r proxmox-install && exit 0 69 | 70 | #Check for Installimage 71 | installimage_bin="/root/.oldroot/nfs/install/installimage" 72 | if [ ! -f "$installimage_bin" ]; then 73 | echo "$installimage_bin does not exist" 74 | echo "Please report the issue with the following" 75 | ls -laFh "/root/.oldroot/nfs/install/" 76 | exit 1 77 | fi 78 | 79 | #Check for base image 80 | installimage_file="/root/images/Debian-bullseye-latest-amd64-base.tar.gz" 81 | # Detect the latest installimage file to use 82 | #installimage_file=$(find /root/images/ -iname 'Debian-*-bullseye-amd64-base.tar.gz.tar.gz' | sort --version-sort --field-separator=- --key=2,2 -r | head -n1) 83 | if [ ! -f $installimage_file ] ; then 84 | echo "Error: Image file was not found: ${installimage_file}" 85 | echo "Please log an issue on the github repo with the following" 86 | ls -laFh "/root/images/" 87 | exit 1 88 | fi 89 | 90 | #Hostname 91 | if [ "$MY_HOSTNAME" == "" ] ; then 92 | MY_HOSTNAME="$1" 93 | fi 94 | if [[ "$MY_HOSTNAME" != *.* ]] ; then 95 | echo "ERROR: Please set a FQDN hostname" 96 | echo "$0 host.name" 97 | exit 1 98 | echo "Hostname: ${MY_HOSTNAME}" 99 | fi 100 | 101 | if [ "$MY_USE_LVM" != "" ] && [ "${MY_USE_LVM,,}" != "true" ] && [ "${MY_USE_LVM,,}" != "yes" ] ; then 102 | LVM="FALSE" 103 | else 104 | LVM="TRUE" 105 | fi 106 | 107 | # Validate Custom partition sizes 108 | if [[ ! $MY_ZFS_SLOG =~ ^[0-9]+$ ]] && [ "$MY_ZFS_SLOG" != "" ] ; then 109 | echo "ERROR: ${MY_ZFS_SLOG} is Not a number, specify in GB" 110 | exit 1 111 | fi 112 | if [[ ! $MY_ZFS_L2ARC =~ ^[0-9]+$ ]] && [ "$MY_ZFS_L2ARC" != "" ] ; then 113 | echo "ERROR: ${MY_ZFS_L2ARC} is Not a number, specify in GB" 114 | exit 1 115 | fi 116 | if [[ ! $MY_SWAP =~ ^[0-9]+$ ]] && [ "$MY_SWAP" != "" ]; then 117 | echo "ERROR: ${MY_SWAP} is Not a number, specify in GB" 118 | exit 1 119 | fi 120 | if [[ ! $MY_BOOT =~ ^[0-9]+$ ]] && [ "$MY_BOOT" != "" ] ; then 121 | echo "ERROR: MY_BOOT is Not a number, specify in GB" 122 | exit 1 123 | fi 124 | if [[ $MY_BOOT -lt 1 ]] && [ "$MY_BOOT" != "" ]; then 125 | echo "ERROR: MY_BOOT cannot be less than 1 GB" 126 | exit 1 127 | fi 128 | if [[ ! $MY_ROOT =~ ^[0-9]+$ ]] && [ "$MY_ROOT" != "" ] ; then 129 | echo "ERROR: MY_ROOT is Not a number, specify in GB" 130 | exit 1 131 | fi 132 | if [[ $MY_ROOT -lt 10 ]] && [ "$MY_ROOT" != "" ] ; then 133 | echo "ERROR: MY_ROOT cannot be less than 10 GB" 134 | exit 1 135 | fi 136 | 137 | #OS to install 138 | if [ "$MY_OS" == "" ]; then 139 | OS="$2" 140 | fi 141 | if [ "${OS,,}" == "pve" ] ; then 142 | OS="PVE" 143 | elif [ "${OS,,}" == "pbs" ] ; then 144 | OS="PBS" 145 | else 146 | OS="PVE" 147 | fi 148 | 149 | # Generate NVME Device Arrays 150 | # shellcheck disable=SC2010 151 | mapfile -t NVME_ARRAY < <( ls -1 /sys/block | grep ^nvme | sort -d ) 152 | NVME_COUNT=${#NVME_ARRAY[@]} 153 | NVME_TARGET="" 154 | NVME_TARGET_FIRST="" 155 | NVME_TARGET_COUNT=0 156 | if [[ $NVME_COUNT -ge 1 ]] ; then 157 | for nvme_device in "${NVME_ARRAY[@]}"; do 158 | if [ "$NVME_FORCE_4K" == "yes" ] ; then 159 | if [[ $(nvme id-ns "/dev/${nvme_device}" -H | grep "LBA Format" | grep "(in use)" | grep -oP "Data Size\K.*" | cut -d" " -f 2) -ne 4096 ]] ; then 160 | echo "Appling 4K block size to NVME: ${nvme_device}" 161 | nvme format "/dev/${nvme_device}" -b 4096 -f || exit 1 162 | sleep 5 163 | echo "Reset NVME controller: ${nvme_device::-2}" 164 | nvme reset "/dev/${nvme_device::-2}" || exit 1 165 | sleep 5 166 | fi 167 | fi 168 | if [ "${NVME_TARGET}" == "" ] ; then 169 | NVME_TARGET="${nvme_device}" 170 | NVME_TARGET_FIRST="${nvme_device}" 171 | NVME_TARGET_COUNT=1 172 | else 173 | if [[ $(grep "${NVME_TARGET_FIRST}" -m1 /proc/partitions | xargs | cut -d" " -f3) -eq $(grep "${nvme_device}" -m1 /proc/partitions | xargs | cut -d" " -f3) ]]; then 174 | NVME_TARGET="${NVME_TARGET},${nvme_device}" 175 | NVME_TARGET_COUNT=$((NVME_TARGET_COUNT+1)) 176 | fi 177 | fi 178 | done 179 | fi 180 | 181 | # Generate SCSI (HDD/SSD) Device Arrays 182 | # shellcheck disable=SC2010 183 | mapfile -t SCSI_ARRAY < <( ls -1 /sys/block | grep ^sd | sort -d ) 184 | SCSI_COUNT=${#SCSI_ARRAY[@]} 185 | SSD_COUNT=0 186 | HDD_COUNT=0 187 | SSD_TARGET="" 188 | HDD_TARGET="" 189 | SSD_TARGET_COUNT=0 190 | HDD_TARGET_COUNT=0 191 | SSD_TARGET_FIRST="" 192 | HDD_TARGET_FIRST="" 193 | if [[ $SCSI_COUNT -ge 1 ]] ; then 194 | for scsi_device in "${SCSI_ARRAY[@]}"; do 195 | if [ "$(lsblk -d -o rota "/dev/${scsi_device}" | tail -n 1 | xargs)" -ne "1" ] ; then 196 | SSD_COUNT=$((SSD_COUNT+1)) 197 | if [ "${SSD_TARGET}" == "" ] ; then 198 | SSD_TARGET="${scsi_device}" 199 | SSD_TARGET_FIRST="${scsi_device}" 200 | SSD_TARGET_COUNT=1 201 | else 202 | if [[ $(grep "${SSD_TARGET_FIRST}" -m1 /proc/partitions | xargs | cut -d" " -f3) -eq $(grep "${scsi_device}" -m1 /proc/partitions | xargs | cut -d" " -f3) ]]; then 203 | SSD_TARGET="${SSD_TARGET},${scsi_device}" 204 | SSD_TARGET_COUNT=$((SSD_TARGET_COUNT+1)) 205 | fi 206 | fi 207 | else 208 | HDD_COUNT=$((HDD_COUNT+1)) 209 | if [ "${HDD_TARGET}" == "" ] ; then 210 | HDD_TARGET="${scsi_device}" 211 | HDD_TARGET_FIRST="${scsi_device}" 212 | HDD_TARGET_COUNT=1 213 | else 214 | if [[ $(grep "${HDD_TARGET_FIRST}" -m1 /proc/partitions | xargs | cut -d" " -f3) -eq $(grep "${scsi_device}" -m1 /proc/partitions | xargs | cut -d" " -f3) ]]; then 215 | HDD_TARGET="${HDD_TARGET},${scsi_device}" 216 | HDD_TARGET_COUNT=$((HDD_TARGET_COUNT+1)) 217 | fi 218 | fi 219 | fi 220 | done 221 | fi 222 | 223 | # Calculate Install Target 224 | RAID="" 225 | INSTALL_TARGET="" 226 | ZFS_L2ARC="" 227 | ZFS_SLOG="" 228 | #NVME only 229 | if [[ $NVME_TARGET_COUNT -ge 1 ]] && [[ $SSD_TARGET_COUNT -eq 0 ]] && [[ $HDD_TARGET_COUNT -eq 0 ]]; then 230 | if [[ $NVME_TARGET_COUNT -eq 4 ]] ; then 231 | RAID="10" 232 | elif [[ $NVME_TARGET_COUNT -eq 2 ]] ; then 233 | RAID="1" 234 | else 235 | RAID="" 236 | fi 237 | SWAP="yes" 238 | INSTALL_TARGET="${NVME_TARGET}" 239 | #SSD Only 240 | elif [[ $NVME_TARGET_COUNT -eq 0 ]] && [[ $SSD_TARGET_COUNT -ge 1 ]] && [[ $HDD_TARGET_COUNT -eq 0 ]] ; then 241 | if [[ $SSD_TARGET_COUNT -eq 4 ]] ; then 242 | RAID="10" 243 | elif [[ $SSD_TARGET_COUNT -eq 2 ]] ; then 244 | RAID="1" 245 | else 246 | RAID="" 247 | fi 248 | SWAP="yes" 249 | INSTALL_TARGET="${SSD_TARGET}" 250 | #HDD Only 251 | elif [[ $NVME_TARGET_COUNT -eq 0 ]] && [[ $SSD_TARGET_COUNT -eq 0 ]] && [[ $HDD_TARGET_COUNT -ge 1 ]] ; then 252 | if [[ $HDD_TARGET_COUNT -eq 4 ]] ; then 253 | RAID="10" 254 | elif [[ $HDD_TARGET_COUNT -eq 2 ]] ; then 255 | RAID="1" 256 | else 257 | RAID="" 258 | fi 259 | SWAP="no" 260 | INSTALL_TARGET="${HDD_TARGET}" 261 | #NVME with SSD, OS on SSD 262 | elif [[ $NVME_TARGET_COUNT -ge 1 ]] && [[ $SSD_TARGET_COUNT -ge 1 ]] && [[ $HDD_TARGET_COUNT -eq 0 ]] ; then 263 | if [[ $SSD_TARGET_COUNT -eq 4 ]] ; then 264 | RAID="10" 265 | elif [[ $SSD_TARGET_COUNT -eq 2 ]] ; then 266 | RAID="1" 267 | else 268 | RAID="" 269 | fi 270 | SWAP="yes" 271 | INSTALL_TARGET="${SSD_TARGET}" 272 | #SSD with HDD, OS on SSD with ZFS L2ARC and slog on SSD 273 | elif [[ $NVME_TARGET_COUNT -eq 0 ]] && [[ $SSD_TARGET_COUNT -ge 1 ]] && [[ $HDD_TARGET_COUNT -ge 1 ]] ; then 274 | if [[ $SSD_TARGET_COUNT -eq 4 ]] ; then 275 | RAID="10" 276 | elif [[ $SSD_TARGET_COUNT -eq 2 ]] ; then 277 | RAID="1" 278 | else 279 | RAID="" 280 | fi 281 | ZFS_L2ARC="yes" 282 | ZFS_SLOG="yes" 283 | SWAP="yes" 284 | INSTALL_TARGET="${SSD_TARGET}" 285 | #NVME with SSD and HDD, OS on SSD 286 | elif [[ $NVME_TARGET_COUNT -ge 1 ]] && [[ $SSD_TARGET_COUNT -ge 1 ]] && [[ $HDD_TARGET_COUNT -ge 1 ]] ; then 287 | if [[ $SSD_TARGET_COUNT -eq 4 ]] ; then 288 | RAID="10" 289 | elif [[ $SSD_TARGET_COUNT -eq 2 ]] ; then 290 | RAID="1" 291 | else 292 | RAID="" 293 | fi 294 | INSTALL_TARGET="${SSD_TARGET}" 295 | #NVME with HDD, OS on NVME with ZFS L2ARC and slog on NVME 296 | elif [[ $NVME_TARGET_COUNT -ge 1 ]] && [[ $SSD_TARGET_COUNT -eq 0 ]] && [[ $HDD_TARGET_COUNT -ge 1 ]] ; then 297 | if [[ $NVME_TARGET_COUNT -eq 4 ]] ; then 298 | RAID="10" 299 | elif [[ $NVME_TARGET_COUNT -eq 2 ]] ; then 300 | RAID="1" 301 | else 302 | RAID="" 303 | fi 304 | ZFS_L2ARC="yes" 305 | ZFS_SLOG="yes" 306 | SWAP="yes" 307 | INSTALL_TARGET="${NVME_TARGET}" 308 | fi 309 | 310 | # check for ram size 311 | #MEMORY_SIZE_GB=$(( $(vmstat -s | grep -i "total memory" | xargs | cut -d" " -f 1) / 1024 / 1000)) 312 | TARGET_SIZE_GB=$(( $(grep "${INSTALL_TARGET//,*}" -m1 /proc/partitions | xargs | cut -d" " -f3) / 1024 / 1024)) 313 | 314 | ##### CONFIGURE BOOT PARTITION SIZE 315 | if [ "$MY_BOOT" != "" ] ; then 316 | BOOT="${MY_BOOT}" 317 | else 318 | BOOT=1 319 | fi 320 | 321 | ##### CONFIGURE ROOT PARTITION SIZE 322 | if [[ $MY_ROOT != "" ]] ; then 323 | ROOT=$MY_ROOT 324 | elif [[ $TARGET_SIZE_GB -gt 800 ]] ; then 325 | ROOT=100 326 | elif [[ $TARGET_SIZE_GB -gt 400 ]] ; then 327 | ROOT=60 328 | elif [[ $TARGET_SIZE_GB -gt 200 ]] ; then 329 | ROOT=50 330 | elif [[ $TARGET_SIZE_GB -gt 100 ]] ; then 331 | ROOT=40 332 | elif [[ $TARGET_SIZE_GB -gt 50 ]] ; then 333 | ROOT=20 334 | else 335 | ROOT=10 336 | fi 337 | 338 | #### CONFIGURE SWAP PARTITION SIZE& 339 | if [ "$MY_SWAP" != "" ] ; then 340 | SWAP="${MY_SWAP}" 341 | elif [ "${SWAP,,}" == "yes" ] || [ "${SWAP,,}" == "true" ] ; then 342 | if [[ $TARGET_SIZE_GB -gt 800 ]] ; then 343 | SWAP=128 344 | elif [[ $TARGET_SIZE_GB -gt 400 ]] ; then 345 | SWAP=64 346 | elif [[ $TARGET_SIZE_GB -gt 200 ]] ; then 347 | SWAP=32 348 | elif [[ $TARGET_SIZE_GB -gt 100 ]] ; then 349 | SWAP=16 350 | elif [[ $TARGET_SIZE_GB -gt 50 ]] ; then 351 | SWAP=8 352 | else 353 | SWAP=4 354 | fi 355 | else 356 | SWAP=0 357 | fi 358 | 359 | #### CONFIGURE ZFS SLOG PARTITION SIZE& 360 | if [ "$MY_ZFS_SLOG" != "" ] ; then 361 | ZFS_SLOG="${MY_ZFS_SLOG}" 362 | elif [ "${ZFS_SLOG,,}" == "yes" ] || [ "${ZFS_SLOG,,}" == "true" ] ; then 363 | if [[ $TARGET_SIZE_GB -gt 800 ]] ; then 364 | ZFS_SLOG=64 365 | elif [[ $TARGET_SIZE_GB -gt 400 ]] ; then 366 | ZFS_SLOG=32 367 | elif [[ $TARGET_SIZE_GB -gt 200 ]] ; then 368 | ZFS_SLOG=16 369 | elif [[ $TARGET_SIZE_GB -gt 100 ]] ; then 370 | ZFS_SLOG=8 371 | elif [[ $TARGET_SIZE_GB -gt 50 ]] ; then 372 | ZFS_SLOG=4 373 | else 374 | ZFS_SLOG=2 375 | fi 376 | else 377 | ZFS_SLOG=0 378 | fi 379 | 380 | #### CONFIGURE ZFS SLOG PARTITION SIZE& 381 | if [ "$MY_ZFS_L2ARC" != "" ] ; then 382 | ZFS_SLOG="${MY_ZFS_L2ARC}" 383 | elif [ "${ZFS_L2ARC,,}" == "yes" ] || [ "${ZFS_L2ARC,,}" == "true" ] ; then 384 | if [[ $TARGET_SIZE_GB -gt 800 ]] ; then 385 | ZFS_L2ARC=128 386 | elif [[ $TARGET_SIZE_GB -gt 400 ]] ; then 387 | ZFS_L2ARC=64 388 | elif [[ $TARGET_SIZE_GB -gt 200 ]] ; then 389 | ZFS_L2ARC=32 390 | elif [[ $TARGET_SIZE_GB -gt 100 ]] ; then 391 | ZFS_L2ARC=16 392 | elif [[ $TARGET_SIZE_GB -gt 50 ]] ; then 393 | ZFS_L2ARC=8 394 | else 395 | ZFS_L2ARC=4 396 | fi 397 | else 398 | ZFS_L2ARC=0 399 | fi 400 | 401 | #### CHECK PARTITIONS WILL FIT ON DISK 402 | if [[ $(( BOOT + ROOT + SWAP + ZFS_L2ARC + ZFS_SLOG + 1 )) -gt $TARGET_SIZE_GB ]] ; then 403 | echo "ERROR: Drive of ${TARGET_SIZE_GB} is too small" 404 | exit 1 405 | fi 406 | 407 | echo "--------------------------------" 408 | echo "OS: ${OS}" 409 | echo "LVM: ${LVM}" 410 | echo "RAID: ${RAID}" 411 | echo "BOOT: ${BOOT}" 412 | echo "ROOT: ${ROOT}" 413 | echo "SWAP: ${SWAP}" 414 | echo "ZFS_L2ARC: ${ZFS_L2ARC}" 415 | echo "ZFS_SLOG: ${ZFS_SLOG}" 416 | echo "Total+1: $(( BOOT + ROOT + SWAP + ZFS_L2ARC + ZFS_SLOG + 1 ))" 417 | echo "TARGET_SIZE_GB: ${TARGET_SIZE_GB}" 418 | echo "INSTALL_TARGET: ${INSTALL_TARGET}" 419 | echo "NVME_COUNT: ${NVME_COUNT}" 420 | echo "NVME_TARGET: ${NVME_TARGET}" 421 | echo "NVME_TARGET_COUNT: ${NVME_TARGET_COUNT}" 422 | echo "SSD_COUNT: ${SSD_COUNT}" 423 | echo "SSD_TARGET: ${SSD_TARGET}" 424 | echo "SSD_TARGET_COUNT: ${SSD_TARGET_COUNT}" 425 | echo "HDD_COUNT: ${HDD_COUNT}" 426 | echo "HDD_TARGET: ${HDD_TARGET}" 427 | echo "HDD_TARGET_COUNT: ${HDD_TARGET_COUNT}" 428 | echo "--------------------------------" 429 | 430 | #wait 10 seconds 431 | sleep 10 432 | 433 | 434 | # GENERATE PARTITION STRINGS 435 | 436 | if [ "$SWAP" != "0" ]; then 437 | SWAP=",swap:swap:${SWAP}G" 438 | else 439 | SWAP="" 440 | fi 441 | if [ "$ZFS_L2ARC" != "0" ]; then 442 | ZFS_L2ARC=",/xshok/zfs-L2ARC:ext4:${ZFS_L2ARC}G" 443 | else 444 | ZFS_L2ARC="" 445 | fi 446 | if [ "$ZFS_SLOG" != "0" ]; then 447 | ZFS_SLOG=",/xshok/zfs-slog:ext4:${ZFS_SLOG}G" 448 | else 449 | ZFS_SLOG="" 450 | fi 451 | if [ "$RAID" != "" ]; then 452 | RAID="-r yes -l ${RAID}" 453 | else 454 | RAID="" 455 | fi 456 | 457 | if [ "$OS" == "PBS" ] ; then 458 | if [ ! -f postinstall_file="/root/pbs" ] ; then 459 | wget "https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/hetzner/pbs" -c -O /root/pbs 460 | fi 461 | postinstall_file="/root/pbs" 462 | else 463 | if [ ! -f postinstall_file="/root/pve" ] ; then 464 | wget "https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/hetzner/pve" -c -O /root/pve 465 | fi 466 | postinstall_file="/root/pve" 467 | fi 468 | 469 | if [ ! -f "$postinstall_file" ] ; then 470 | echo "Error: postinstall file was not found: ${postinstall_file}" 471 | exit 1 472 | fi 473 | cp -f "$postinstall_file" /post-install-proxmox 474 | chmod 777 /post-install-proxmox 475 | 476 | 477 | if [ "${WIPE_PARTITION_TABLE,,}" == "yes" ] || [ "${WIPE_PARTITION_TABLE,,}" == "true" ] ; then 478 | IFS=', ' read -r -a INSTALL_TARGET_ARRAY <<< "${INSTALL_TARGET}" 479 | for install_device in "${INSTALL_TARGET_ARRAY[@]}"; do 480 | echo "Creating NEW GPT table: ${install_device}" 481 | printf "Yes\n" | parted "/dev/${install_device}" mklabel gpt ---pretend-input-tty || exit 1 482 | sleep 5 483 | done 484 | fi 485 | 486 | 487 | # INSTALL 488 | if [ "$OS" == "PVE" ]; then 489 | INSTALL_COMMAND="${installimage_bin} -a -t yes -i ${installimage_file} -g -s en -x /post-install-proxmox -n ${MY_HOSTNAME} -b grub -d ${INSTALL_TARGET} ${RAID} -p /boot:ext3:${BOOT}G,/:ext4:${ROOT}G${SWAP}${ZFS_L2ARC}${ZFS_SLOG},lvm:pve:all -v pve:data:/var/lib/vz:xfs:all" 490 | else 491 | INSTALL_COMMAND="${installimage_bin} -a -t yes -i ${installimage_file} -g -s en -x /post-install-proxmox -n ${MY_HOSTNAME} -b grub -d ${INSTALL_TARGET} ${RAID} -p /boot:ext3:${BOOT}G,/:ext4:${ROOT}G${SWAP}${ZFS_L2ARC}${ZFS_SLOG},/backup:xfs:all" 492 | fi 493 | 494 | echo "Starting Installer ...." 495 | echo "launching via a screen process, incase your connection is disconnected" 496 | echo "run this script again to automatically reconnect to it." 497 | 498 | screen -mS proxmox-install /usr/bin/bash -c "$INSTALL_COMMAND" 499 | 500 | echo "Please reboot to load proxmox" 501 | 502 | # usage: installimage [options] 503 | # -a automatic mode / batch mode 504 | # -x Use this file as post-install script, that will be executed after installation inside the chroot. 505 | # -n set the specified hostNAME. 506 | # -r activate software RAID or not. 507 | # -l <0|1|5|6|10> set the specified raid LEVEL. 508 | # -i use the specified IMAGE to install (full path to the OS image) 509 | # -g Use this to force validation of the image file with detached GPG signature. 510 | # -p define the PARTITIONS to create, example: 511 | # - regular partitions: swap:swap:4G,/:ext3:all 512 | # - lvm setup example: /boot:ext2:256M,lvm:vg0:all 513 | # -v define the logical VOLUMES you want to be created 514 | # - example: vg0:root:/:ext3:20G,vg0:swap:swap:swap:4G 515 | # -d /dev names of DRIVES to use, e.g.: sda or sda,sdb 516 | # -f FORMAT the second drive (if not used for raid)? 517 | # -s Language to use for different things (e.g.PLESK) 518 | # -z PLESK_ Install optional software like PLESK with version 519 | # -K Install SSH-Keys from file/URL 520 | # -t Take over rescue system SSH public keys 521 | # -u Allow usb drives 522 | # -G Generate new SSH host keys (default: yes) 523 | -------------------------------------------------------------------------------- /hetzner/pbs: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Proxmox Backup Server installer script based on Debian Bullseye 4 | 5 | echo -e "\n----- Update and upgrade system, please wait ... -----\n" 6 | 7 | export LANG="en_US.UTF-8" 8 | export LC_ALL="C" 9 | 10 | #create lock dir for apt 11 | mkdir /run/lock 12 | chmod a+rwxt /run/lock 13 | 14 | apt update >/dev/null 15 | apt upgrade -y 16 | 17 | echo -e "\n----------------- Deinstalling any linux firmware packages ------------------\n" 18 | 19 | firmware="$(dpkg -l | grep -i 'firmware-')" 20 | 21 | if [ -n "$firmware" ]; then 22 | apt -q -y purge firmware-bnx2x firmware-realtek firmware-linux firmware-linux-free firmware-linux-nonfree 23 | else 24 | echo "no firmware packages installed" 25 | fi 26 | 27 | echo -e "\n----- Add Proxmox repo to APT sources -----\n" 28 | 29 | echo -e "\n\n# PBS packages provided by proxmox.com" >/etc/apt/sources.list.d/proxmox.list 30 | echo -e "deb http://download.proxmox.com/debian/pbs bullseye pbs-no-subscription\n" >>/etc/apt/sources.list.d/proxmox.list 31 | wget -q "http://download.proxmox.com/debian/proxmox-release-bullseye.gpg" -O /etc/apt/trusted.gpg.d/proxmox-release-bullseye.gpg 32 | 33 | 34 | apt update >/dev/null 35 | 36 | echo -e "\n----- Install Proxmox packages -----\n" 37 | 38 | cat </dev/null 15 | apt upgrade -y 16 | 17 | echo -e "\n----------------- Deinstalling any linux firmware packages ------------------\n" 18 | 19 | firmware="$(dpkg -l | grep -i 'firmware-')" 20 | 21 | if [ -n "$firmware" ]; then 22 | apt -q -y purge firmware-bnx2x firmware-realtek firmware-linux firmware-linux-free firmware-linux-nonfree 23 | else 24 | echo "no firmware packages installed" 25 | fi 26 | 27 | echo -e "\n----- Add Proxmox repo to APT sources -----\n" 28 | 29 | echo -e "\n\n# PVE packages provided by proxmox.com" >>/etc/apt/sources.list.d/proxmox.list 30 | echo -e "deb http://download.proxmox.com/debian/pve bullseye pve-no-subscription\n" >>/etc/apt/sources.list.d/proxmox.list 31 | wget -q "http://download.proxmox.com/debian/proxmox-release-bullseye.gpg" -O /etc/apt/trusted.gpg.d/proxmox-release-bullseye.gpg 32 | 33 | apt update >/dev/null 34 | 35 | echo -e "\n----- Install Proxmox packages -----\n" 36 | 37 | cat <> CONNECT VIA VNC TO ${MY_IP4_AND_NETMASK%/*} WITH PASSWORD ${MY_RANDOM_PASS}" 241 | echo "** Please use the following, install options **" 242 | echo "Target Harddisk: [OPTIONS]" 243 | if [[ $INSTALL_COUNT -ge 8 ]] ; then 244 | echo "Filesystem: zfs (RAIDZ-2)" 245 | elif [[ $INSTALL_COUNT -ge 3 ]] ; then 246 | echo "Filesystem: zfs (RAIDZ-1)" 247 | elif [[ $INSTALL_COUNT -ge 2 ]] ; then 248 | echo "Filesystem: zfs (RAID1)" 249 | fi 250 | echo "Keyboard Layout: U.S. English" 251 | 252 | echo "IP Address (CIDR): ${MY_IP4_AND_NETMASK}" 253 | echo "Gateway: ${MY_IP4_GATEWAY}" 254 | echo "DNS Server: ${MY_DNS_SERVER}" 255 | echo "********************************" 256 | 257 | echo ">> CONNECT VIA VNC TO ${MY_IP4_AND_NETMASK%/*} WITH PASSWORD ${MY_RANDOM_PASS}" 258 | 259 | printf "change vnc password\n%s\n" ${MY_RANDOM_PASS} | qemu-system-x86_64 -machine type=q35,accel=kvm -cpu host -enable-kvm -smp 4 -m 4096 -boot d -cdrom ${INSTALL_IMAGE} ${DISKS} -vnc :0,password -monitor stdio -no-reboot 260 | 261 | #https://blogs.oracle.com/linux/post/how-to-emulate-block-devices-with-qemu 262 | 263 | printf "n\n" | zfsonlinux_install 2&> /dev/null 264 | retVal=$? 265 | if [ $retVal -eq 1 ]; then 266 | echo "Installing zfsonlinux" 267 | printf "y\n" | zfsonlinux_install 268 | echo "Correcting network config" 269 | zpool import -f -R /mnt rpool 270 | sed -i -e "s/enp[0-9]s[0-9]/${MY_IFACE}/g" /mnt/etc/network/interfaces 271 | #cat /mnt/etc/network/interfaces 272 | zpool export -f rpool 273 | else 274 | echo "zfsonlinux_install not detected, launching vnc to complete networking config." 275 | echo ">> SERVER SHOULD BE INSTALLED, RESTARTING <<" 276 | echo "" 277 | echo ">> RE-CONNECT VIA VNC TO ${MY_IP4_AND_NETMASK%/*} WITH PASSWORD ${MY_RANDOM_PASS}" 278 | echo "" 279 | echo "Login as root, with the password which was set during install" 280 | echo ">> run the following command below" 281 | echo "nano /etc/network/interfaces" 282 | echo "** edit the file to have the following **" 283 | echo " 284 | auto lo 285 | iface lo inet loopback 286 | 287 | iface ${MY_IFACE} inet manual 288 | 289 | auto vmbr0 290 | iface vmbr0 inet static 291 | address ${MY_IP4_AND_NETMASK} 292 | gateway ${MY_IP4_GATEWAY} 293 | bridge_ports ${MY_IFACE} 294 | bridge_stp off 295 | bridge_fd 0 296 | " 297 | echo ">> save the file <<" 298 | echo "export the zfs pool, so the machine will boot correctly " 299 | echo ">> run the following command below" 300 | echo "zpool export -f rpool" 301 | printf "change vnc password\n%s\n" ${MY_RANDOM_PASS} | qemu-system-x86_64 -enable-kvm -smp 4 -m 4096 $DISKS -vnc :0,password -monitor stdio -no-reboot -serial telnet:localhost:4321,server,nowait 302 | 303 | fi 304 | 305 | echo ">> COMPLETED PROXMOX IS NOW INSTALLED" 306 | echo "" 307 | echo ">> PLEASE REBOOT and connect to https://${MY_IP4_AND_NETMASK%/*}:8006" 308 | -------------------------------------------------------------------------------- /install-post.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | ################################################################################ 3 | # This is property of eXtremeSHOK.com 4 | # You are free to use, modify and distribute, however you may not remove this notice. 5 | # Copyright (c) Adrian Jon Kriel :: admin@extremeshok.com 6 | ################################################################################ 7 | # 8 | # Script updates can be found at: https://github.com/extremeshok/xshok-proxmox 9 | # 10 | # post-installation script for Proxmox 11 | # 12 | # License: BSD (Berkeley Software Distribution) 13 | # 14 | ################################################################################ 15 | # 16 | # Tested on Proxmox Version: 7.1 17 | # 18 | # Assumptions: Proxmox installed 19 | # 20 | # Notes: 21 | # openvswitch will be disabled (removed) when ifupdown2 is enabled 22 | # ifupdown2 will be disabled (removed) when openvswitch is enabled 23 | # 24 | # Docker : not advisable to run docker on the Hypervisor(proxmox) directly. 25 | # Correct way is to create a VM which will be used exclusively for docker. 26 | # ie. fresh ubuntu lts server with https://github.com/extremeshok/xshok-docker 27 | ################################################################################ 28 | # 29 | # THERE ARE NO USER CONFIGURABLE OPTIONS IN THIS SCRIPT 30 | # 31 | ################################################################################ 32 | 33 | ##### T O S E T Y O U R O P T I O N S ###### 34 | # User Defined Options for (install-post.sh) post-installation script for Proxmox 35 | # are set in the xs-install-post.env, see the sample : xs-install-post.env.sample 36 | ## Alternatively, set the varible via the export 37 | # Example to disable to motd 38 | # export XS_MOTD="no" ; bash install-post.sh 39 | ############################### 40 | ##### D O N O T E D I T B E L O W ###### 41 | 42 | #### VARIABLES / options 43 | # Detect AMD EPYC and Ryzen CPU and Apply Fixes 44 | if [ -z "$XS_AMDFIXES" ] ; then 45 | XS_AMDFIXES="yes" 46 | fi 47 | # Force APT to use IPv4 48 | if [ -z "$XS_APTIPV4" ] ; then 49 | XS_APTIPV4="yes" 50 | fi 51 | # Update proxmox and install various system utils 52 | if [ -z "$XS_APTUPGRADE" ] ; then 53 | XS_APTUPGRADE="yes" 54 | fi 55 | # Customise bashrc 56 | if [ -z "$XS_BASHRC" ] ; then 57 | XS_BASHRC="yes" 58 | fi 59 | # Add the latest ceph provided by proxmox 60 | if [ -z "$XS_CEPH" ] ; then 61 | XS_CEPH="no" 62 | fi 63 | # Disable portmapper / rpcbind (security) 64 | if [ -z "$XS_DISABLERPC" ] ; then 65 | XS_DISABLERPC="yes" 66 | fi 67 | # Ensure Entropy Pools are Populated, prevents slowdowns whilst waiting for entropy 68 | if [ -z "$XS_ENTROPY" ] ; then 69 | XS_ENTROPY="yes" 70 | fi 71 | # Protect the web interface with fail2ban 72 | if [ -z "$XS_FAIL2BAN" ] ; then 73 | XS_FAIL2BAN="yes" 74 | fi 75 | # Detect if is a virtual machine and install the relavant guest agent 76 | if [ -z "$XS_GUESTAGENT" ] ; then 77 | XS_GUESTAGENT="yes" 78 | fi 79 | # Install ifupdown2 for a virtual internal network allows rebootless networking changes (not compatible with openvswitch-switch) 80 | if [ -z "$XS_IFUPDOWN2" ] ; then 81 | XS_IFUPDOWN2="yes" 82 | fi 83 | # Limit the size and optimise journald 84 | if [ -z "$XS_JOURNALD" ] ; then 85 | XS_JOURNALD="yes" 86 | fi 87 | # Install kernel source headers 88 | if [ -z "$XS_KERNELHEADERS" ] ; then 89 | XS_KERNELHEADERS="yes" 90 | fi 91 | # Ensure ksmtuned (ksm-control-daemon) is enabled and optimise according to ram size 92 | if [ -z "$XS_KSMTUNED" ] ; then 93 | XS_KSMTUNED="yes" 94 | fi 95 | # Set language, if changed will disable XS_NOAPTLANG 96 | if [ -z "$XS_LANG" ] ; then 97 | XS_LANG="en_US.UTF-8" 98 | fi 99 | # Enable restart on kernel panic, kernel oops and hardlockup 100 | if [ -z "$XS_KERNELPANIC" ] ; then 101 | XS_KERNELPANIC="yes" 102 | fi 103 | # Increase max user watches, FD limit, FD ulimit, max key limit, ulimits 104 | if [ -z "$XS_LIMITS" ] ; then 105 | XS_LIMITS="yes" 106 | fi 107 | # Optimise logrotate 108 | if [ -z "$XS_LOGROTATE" ] ; then 109 | XS_LOGROTATE="yes" 110 | fi 111 | # Lynis security scan tool by Cisofy 112 | if [ -z "$XS_LYNIS" ] ; then 113 | XS_LYNIS="yes" 114 | fi 115 | # Increase Max FS open files 116 | if [ -z "$XS_MAXFS" ] ; then 117 | XS_MAXFS="yes" 118 | fi 119 | # Optimise Memory 120 | if [ -z "$XS_MEMORYFIXES" ] ; then 121 | XS_MEMORYFIXES="yes" 122 | fi 123 | # Pretty MOTD BANNER 124 | if [ -z "$XS_MOTD" ] ; then 125 | XS_MOTD="yes" 126 | fi 127 | # Enable Network optimising 128 | if [ -z "$XS_NET" ] ; then 129 | XS_NET="yes" 130 | fi 131 | # Save bandwidth and skip downloading additional languages, requires XS_LANG="en_US.UTF-8" 132 | if [ -z "$XS_NOAPTLANG" ] ; then 133 | XS_NOAPTLANG="yes" 134 | fi 135 | # Disable enterprise proxmox repo 136 | if [ -z "$XS_NOENTREPO" ] ; then 137 | XS_NOENTREPO="yes" 138 | fi 139 | # Remove subscription banner 140 | if [ -z "$XS_NOSUBBANNER" ] ; then 141 | XS_NOSUBBANNER="yes" 142 | fi 143 | # Install openvswitch for a virtual internal network 144 | if [ -z "$XS_OPENVSWITCH" ] ; then 145 | XS_OPENVSWITCH="no" 146 | fi 147 | # Detect if this is an OVH server and install OVH Real Time Monitoring 148 | if [ -z "$XS_OVHRTM" ] ; then 149 | XS_OVHRTM="yes" 150 | fi 151 | # Set pigz to replace gzip, 2x faster gzip compression 152 | if [ -z "$XS_PIGZ" ] ; then 153 | XS_PIGZ="yes" 154 | fi 155 | # Bugfix: high swap usage with low memory usage 156 | if [ -z "$XS_SWAPPINESS" ] ; then 157 | XS_SWAPPINESS="yes" 158 | fi 159 | # Enable TCP BBR congestion control 160 | if [ -z "$XS_TCPBBR" ] ; then 161 | XS_TCPBBR="yes" 162 | fi 163 | # Enable TCP fastopen 164 | if [ -z "$XS_TCPFASTOPEN" ] ; then 165 | XS_TCPFASTOPEN="yes" 166 | fi 167 | # Enable testing proxmox repo 168 | if [ -z "$XS_TESTREPO" ] ; then 169 | XS_TESTREPO="no" 170 | fi 171 | # Automatically Synchronize the time 172 | if [ -z "$XS_TIMESYNC" ] ; then 173 | XS_TIMESYNC="yes" 174 | fi 175 | # Set Timezone, empty = set automatically by IP 176 | if [ -z "$XS_TIMEZONE" ] ; then 177 | XS_TIMEZONE="" 178 | fi 179 | # Install common system utilities 180 | if [ -z "$XS_UTILS" ] ; then 181 | XS_UTILS="yes" 182 | fi 183 | # Increase vzdump backup speed 184 | if [ -z "$XS_VZDUMP" ] ; then 185 | XS_VZDUMP="yes" 186 | fi 187 | # Optimise ZFS arc size accoring to memory size 188 | if [ -z "$XS_ZFSARC" ] ; then 189 | XS_ZFSARC="yes" 190 | fi 191 | # Install zfs-auto-snapshot 192 | if [ -z "$XS_ZFSAUTOSNAPSHOT" ] ; then 193 | XS_ZFSAUTOSNAPSHOT="no" 194 | fi 195 | # Enable VFIO IOMMU support for PCIE passthrough 196 | if [ -z "$XS_VFIO_IOMMU" ] ; then 197 | XS_VFIO_IOMMU="yes" 198 | fi 199 | ################# D O N O T E D I T ###################################### 200 | 201 | echo "Processing .... " 202 | 203 | # VARIABLES are overrideen with xs-install-post.env 204 | if [ -f "xs-install-post.env" ] ; then 205 | echo "Loading variables from xs-install-post.env ..." 206 | # shellcheck disable=SC1091 207 | source xs-install-post.env; 208 | fi 209 | 210 | # Set the local 211 | if [ "$XS_LANG" == "" ] ; then 212 | XS_LANG="en_US.UTF-8" 213 | fi 214 | export LANG="$XS_LANG" 215 | export LC_ALL="C" 216 | 217 | # enforce proxmox 218 | if [ ! -f "/etc/pve/.version" ] ; then 219 | echo "ERROR: This script only supports Proxmox" 220 | exit 1 221 | fi 222 | 223 | if [ -f "/etc/extremeshok" ] ; then 224 | echo "ERROR: Script can only be run once" 225 | exit 1 226 | fi 227 | 228 | # SET VARIBLES 229 | 230 | OS_CODENAME="$(grep "VERSION_CODENAME=" /etc/os-release | cut -d"=" -f 2 | xargs )" 231 | RAM_SIZE_GB=$(( $(vmstat -s | grep -i "total memory" | xargs | cut -d" " -f 1) / 1024 / 1000)) 232 | 233 | if [ "${XS_LANG}" == "en_US.UTF-8" ] && [ "${XS_NOAPTLANG,,}" == "yes" ] ; then 234 | # save bandwidth and skip downloading additional languages 235 | echo -e "Acquire::Languages \"none\";\\n" > /etc/apt/apt.conf.d/99-xs-disable-translations 236 | fi 237 | 238 | if [ "${XS_APTIPV4,,}" == "yes" ] ; then 239 | # force APT to use IPv4 240 | echo -e "Acquire::ForceIPv4 \"true\";\\n" > /etc/apt/apt.conf.d/99-xs-force-ipv4 241 | fi 242 | 243 | if [ "${XS_NOENTREPO,,}" == "yes" ] ; then 244 | # disable enterprise proxmox repo 245 | if [ -f /etc/apt/sources.list.d/pve-enterprise.list ]; then 246 | sed -i "s/^deb/#deb/g" /etc/apt/sources.list.d/pve-enterprise.list 247 | fi 248 | # enable free public proxmox repo 249 | if [ ! -f /etc/apt/sources.list.d/proxmox.list ] && [ ! -f /etc/apt/sources.list.d/pve-public-repo.list ] && [ ! -f /etc/apt/sources.list.d/pve-install-repo.list ] ; then 250 | echo -e "deb http://download.proxmox.com/debian/pve ${OS_CODENAME} pve-no-subscription\\n" > /etc/apt/sources.list.d/pve-public-repo.list 251 | fi 252 | if [ "${XS_TESTREPO,,}" == "yes" ] ; then 253 | # enable testing proxmox repo 254 | echo -e "deb http://download.proxmox.com/debian/pve ${OS_CODENAME} pvetest\\n" > /etc/apt/sources.list.d/pve-testing-repo.list 255 | fi 256 | fi 257 | 258 | # rebuild and add non-free to /etc/apt/sources.list 259 | cat < /etc/apt/sources.list 260 | deb https://ftp.debian.org/debian ${OS_CODENAME} main contrib 261 | deb https://ftp.debian.org/debian ${OS_CODENAME}-updates main contrib 262 | # non-free 263 | deb https://httpredir.debian.org/debian/ ${OS_CODENAME} main contrib non-free 264 | # security updates 265 | deb https://security.debian.org/debian-security ${OS_CODENAME}/updates main contrib 266 | EOF 267 | 268 | # Refresh the package lists 269 | apt-get update > /dev/null 2>&1 270 | 271 | # Remove conflicting utilities 272 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' purge ntp openntpd systemd-timesyncd 273 | 274 | # Fixes for common apt repo errors 275 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install apt-transport-https debian-archive-keyring ca-certificates curl 276 | 277 | if [ "${XS_APTUPGRADE,,}" == "yes" ] ; then 278 | # update proxmox and install various system utils 279 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' dist-upgrade 280 | pveam update 281 | fi 282 | 283 | # Install packages which are sometimes missing on some Proxmox installs. 284 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install zfsutils-linux proxmox-backup-restore-image chrony 285 | 286 | if [ "${XS_UTILS,,}" == "yes" ] ; then 287 | # Install common system utilities 288 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install \ 289 | axel \ 290 | build-essential \ 291 | curl \ 292 | dialog \ 293 | dnsutils \ 294 | dos2unix \ 295 | git \ 296 | gnupg-agent \ 297 | grc \ 298 | htop \ 299 | iftop \ 300 | iotop \ 301 | iperf \ 302 | ipset \ 303 | iptraf \ 304 | mlocate \ 305 | msr-tools \ 306 | nano \ 307 | net-tools \ 308 | omping \ 309 | software-properties-common \ 310 | sshpass \ 311 | tmux \ 312 | unzip \ 313 | vim \ 314 | vim-nox \ 315 | wget \ 316 | whois \ 317 | zip 318 | fi 319 | 320 | if [ "${XS_CEPH,,}" == "yes" ] ; then 321 | # Add the latest ceph provided by proxmox 322 | echo "deb http://download.proxmox.com/debian/ceph-pacific ${OS_CODENAME} main" > /etc/apt/sources.list.d/ceph-pacific.list 323 | ## Refresh the package lists 324 | apt-get update > /dev/null 2>&1 325 | ## Install ceph support 326 | echo "Y" | pveceph install 327 | fi 328 | 329 | if [ "${XS_LYNIS,,}" == "yes" ] ; then 330 | # Lynis security scan tool by Cisofy 331 | wget -O - https://packages.cisofy.com/keys/cisofy-software-public.key | apt-key add - 332 | ## Add the latest lynis 333 | echo "deb https://packages.cisofy.com/community/lynis/deb/ stable main" > /etc/apt/sources.list.d/cisofy-lynis.list 334 | ## Refresh the package lists 335 | apt-get update > /dev/null 2>&1 336 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install lynis 337 | fi 338 | 339 | if [ "${XS_OPENVSWITCH,,}" == "yes" ] && [ "${XS_IFUPDOWN2}" == "no" ] ; then 340 | ## Install openvswitch for a virtual internal network 341 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install ifenslave ifupdown 342 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' remove ifupdown2 343 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install openvswitch-switch 344 | else 345 | ## Install ifupdown2 for a virtual internal network allows rebootless networking changes (not compatible with openvswitch-switch) 346 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' purge openvswitch-switch 347 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install ifupdown2 348 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' remove ifenslave ifupdown 349 | fi 350 | 351 | if [ "${XS_ZFSAUTOSNAPSHOT,,}" == "yes" ] ; then 352 | ## Install zfs-auto-snapshot 353 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install zfs-auto-snapshot 354 | # make 5min snapshots , keep 12 5min snapshots 355 | if [ -f "/etc/cron.d/zfs-auto-snapshot" ] ; then 356 | sed -i 's|--keep=[0-9]*|--keep=12|g' /etc/cron.d/zfs-auto-snapshot 357 | sed -i 's|*/[0-9]*|*/5|g' /etc/cron.d/zfs-auto-snapshot 358 | fi 359 | # keep 24 hourly snapshots 360 | if [ -f "/etc/cron.hourly/zfs-auto-snapshot" ] ; then 361 | sed -i 's|--keep=[0-9]*|--keep=24|g' /etc/cron.hourly/zfs-auto-snapshot 362 | fi 363 | # keep 7 daily snapshots 364 | if [ -f "/etc/cron.daily/zfs-auto-snapshot" ] ; then 365 | sed -i 's|--keep=[0-9]*|--keep=7|g' /etc/cron.daily/zfs-auto-snapshot 366 | fi 367 | # keep 4 weekly snapshots 368 | if [ -f "/etc/cron.weekly/zfs-auto-snapshot" ] ; then 369 | sed -i 's|--keep=[0-9]*|--keep=4|g' /etc/cron.weekly/zfs-auto-snapshot 370 | fi 371 | # keep 3 monthly snapshots 372 | if [ -f "/etc/cron.monthly/zfs-auto-snapshot" ] ; then 373 | sed -i 's|--keep=[0-9]*|--keep=3|g' /etc/cron.monthly/zfs-auto-snapshot 374 | fi 375 | fi 376 | 377 | if [ "${XS_KSMTUNED,,}" == "yes" ] ; then 378 | ## Ensure ksmtuned (ksm-control-daemon) is enabled and optimise according to ram size 379 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install ksm-control-daemon 380 | if [[ RAM_SIZE_GB -le 16 ]] ; then 381 | # start at 50% full 382 | KSM_THRES_COEF=50 383 | KSM_SLEEP_MSEC=80 384 | elif [[ RAM_SIZE_GB -le 32 ]] ; then 385 | # start at 60% full 386 | KSM_THRES_COEF=40 387 | KSM_SLEEP_MSEC=60 388 | elif [[ RAM_SIZE_GB -le 64 ]] ; then 389 | # start at 70% full 390 | KSM_THRES_COEF=30 391 | KSM_SLEEP_MSEC=40 392 | elif [[ RAM_SIZE_GB -le 128 ]] ; then 393 | # start at 80% full 394 | KSM_THRES_COEF=20 395 | KSM_SLEEP_MSEC=20 396 | else 397 | # start at 90% full 398 | KSM_THRES_COEF=10 399 | KSM_SLEEP_MSEC=10 400 | fi 401 | sed -i -e "s/\# KSM_THRES_COEF=.*/KSM_THRES_COEF=${KSM_THRES_COEF}/g" /etc/ksmtuned.conf 402 | sed -i -e "s/\# KSM_SLEEP_MSEC=.*/KSM_SLEEP_MSEC=${KSM_SLEEP_MSEC}/g" /etc/ksmtuned.conf 403 | systemctl enable ksmtuned 404 | fi 405 | 406 | if [ "${XS_AMDFIXES,,}" == "yes" ] ; then 407 | ## Detect AMD EPYC and Ryzen CPU and Apply Fixes 408 | if [ "$(grep -i -m 1 "model name" /proc/cpuinfo | grep -i "EPYC")" != "" ]; then 409 | echo "AMD EPYC detected" 410 | elif [ "$(grep -i -m 1 "model name" /proc/cpuinfo | grep -i "Ryzen")" != "" ]; then 411 | echo "AMD Ryzen detected" 412 | else 413 | XS_AMDFIXES="no" 414 | fi 415 | 416 | if [ "${XS_AMDFIXES,,}" == "yes" ] ; then 417 | #Apply fix to kernel : Fixes random crashing and instability 418 | if ! grep "GRUB_CMDLINE_LINUX_DEFAULT" /etc/default/grub | grep -q "idle=nomwait" ; then 419 | echo "Setting kernel idle=nomwait" 420 | sed -i 's/GRUB_CMDLINE_LINUX_DEFAULT="/GRUB_CMDLINE_LINUX_DEFAULT="idle=nomwait /g' /etc/default/grub 421 | update-grub 422 | fi 423 | ## Add msrs ignore to fix Windows guest on EPIC/Ryzen host 424 | echo "options kvm ignore_msrs=Y" >> /etc/modprobe.d/kvm.conf 425 | echo "options kvm report_ignored_msrs=N" >> /etc/modprobe.d/kvm.conf 426 | 427 | echo "Installing kernel 5.15" 428 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install pve-kernel-5.15 429 | fi 430 | fi 431 | 432 | if [ "${XS_KERNELHEADERS,,}" == "yes" ] ; then 433 | ## Install kernel source headers 434 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install pve-headers module-assistant 435 | fi 436 | 437 | # if [ "$XS_KEXEC" == "yes" ] ; then 438 | # ## Install kexec, allows for quick reboots into the latest updated kernel set as primary in the boot-loader. 439 | # # use command 'reboot-quick' 440 | # echo "kexec-tools kexec-tools/load_kexec boolean false" | debconf-set-selections 441 | # /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install kexec-tools 442 | # cat <<'EOF' > /etc/systemd/system/kexec-pve.service 443 | # [Unit] 444 | # Description=Loading new kernel into memory 445 | # Documentation=man:kexec(8) 446 | # DefaultDependencies=no 447 | # Before=reboot.target 448 | # RequiresMountsFor=/boot 449 | # #Before=shutdown.target umount.target final.target 450 | 451 | # [Service] 452 | # Type=oneshot 453 | # RemainAfterExit=yes 454 | # ExecStart=/sbin/kexec -d -l /boot/pve/vmlinuz --initrd=/boot/pve/initrd.img --reuse-cmdline 455 | 456 | # [Install] 457 | # WantedBy=default.target 458 | # EOF 459 | # systemctl enable kexec-pve.service 460 | # echo "alias reboot-quick='systemctl kexec'" >> /root/.bash_profile 461 | # fi 462 | 463 | if [ "${XS_DISABLERPC,,}" == "yes" ] ; then 464 | ## Disable portmapper / rpcbind (security) 465 | systemctl disable rpcbind 466 | systemctl stop rpcbind 467 | fi 468 | 469 | if [ "${XS_TIMEZONE}" == "" ] ; then 470 | ## Set Timezone, empty = set automatically by ip 471 | this_ip="$(dig +short myip.opendns.com @resolver1.opendns.com)" 472 | timezone="$(curl "https://ipapi.co/${this_ip}/timezone")" 473 | if [ "$timezone" != "" ] ; then 474 | echo "Found $timezone for ${this_ip}" 475 | timedatectl set-timezone "$timezone" 476 | else 477 | echo "WARNING: Timezone not found for ${this_ip}, set to UTC" 478 | timedatectl set-timezone UTC 479 | fi 480 | else 481 | ## Set Timezone to XS_TIMEZONE 482 | timedatectl set-timezone "$XS_TIMEZONE" 483 | fi 484 | 485 | if [ "${XS_TIMESYNC,,}" == "yes" ] ; then 486 | timedatectl set-ntp true 487 | fi 488 | 489 | if [ "${XS_GUESTAGENT,,}" == "yes" ] ; then 490 | ## Detect if is running in a virtual machine and install the relavant guest agent 491 | if [ "$(dmidecode -s system-manufacturer | xargs)" == "QEMU" ] || [ "$(systemd-detect-virt | xargs)" == "kvm" ] ; then 492 | echo "QEMU Detected, installing guest agent" 493 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install qemu-guest-agent 494 | elif [ "$(systemd-detect-virt | xargs)" == "vmware" ] ; then 495 | echo "VMware Detected, installing vm-tools" 496 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install open-vm-tools 497 | elif [ "$(systemd-detect-virt | xargs)" == "oracle" ] ; then 498 | echo "Virtualbox Detected, installing guest-utils" 499 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install virtualbox-guest-utils 500 | fi 501 | fi 502 | 503 | if [ "${XS_PIGZ,,}" == "yes" ] ; then 504 | ## Set pigz to replace gzip, 2x faster gzip compression 505 | sed -i "s/#pigz:.*/pigz: 1/" /etc/vzdump.conf 506 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install pigz 507 | cat < /bin/pigzwrapper 508 | #!/bin/sh 509 | # eXtremeSHOK.com 510 | PATH=/bin:\$PATH 511 | GZIP="-1" 512 | exec /usr/bin/pigz "\$@" 513 | EOF 514 | mv -f /bin/gzip /bin/gzip.original 515 | cp -f /bin/pigzwrapper /bin/gzip 516 | chmod +x /bin/pigzwrapper 517 | chmod +x /bin/gzip 518 | fi 519 | 520 | if [ "${XS_OVHRTM,,}" == "yes" ] ; then 521 | ## Detect if this is an OVH server by getting the global IP and checking the ASN, then install OVH RTM (real time monitoring)" 522 | if [ "$(whois -h v4.whois.cymru.com " -t $(curl ipinfo.io/ip 2> /dev/null)" | tail -n 1 | cut -d'|' -f3 | grep -i "ovh")" != "" ] ; then 523 | echo "Deteted OVH Server, installing OVH RTM (real time monitoring)" 524 | # http://help.ovh.co.uk/RealTimeMonitoring 525 | # https://docs.ovh.com/gb/en/dedicated/install-rtm/ 526 | wget -qO - https://last-public-ovh-infra-yak.snap.mirrors.ovh.net/yak/archives/apply.sh | OVH_PUPPET_MANIFEST=distribyak/catalog/master/puppet/manifests/common/rtmv2.pp bash 527 | fi 528 | fi 529 | 530 | if [ "${XS_FAIL2BAN,,}" == "yes" ] ; then 531 | ## Protect the web interface with fail2ban 532 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install fail2ban 533 | # shellcheck disable=1117 534 | cat < /etc/fail2ban/filter.d/proxmox.conf 535 | [Definition] 536 | failregex = pvedaemon\[.*authentication failure; rhost= user=.* msg=.* 537 | ignoreregex = 538 | EOF 539 | 540 | cat < /etc/fail2ban/jail.d/proxmox.conf 541 | [proxmox] 542 | enabled = true 543 | port = https,http,8006,8007 544 | filter = proxmox 545 | logpath = /var/log/daemon.log 546 | maxretry = 3 547 | # 1 hour 548 | bantime = 3600 549 | findtime = 600 550 | EOF 551 | 552 | # cat < /etc/fail2ban/jail.local 553 | # [DEFAULT] 554 | # banaction = iptables-ipset-proto4 555 | # EOF 556 | 557 | systemctl enable fail2ban 558 | 559 | # ##testing 560 | # #fail2ban-regex /var/log/daemon.log /etc/fail2ban/filter.d/proxmox.conf 561 | fi 562 | 563 | if [ "${XS_NOSUBBANNER,,}" == "yes" ] ; then 564 | ## Remove subscription banner 565 | if [ -f "/usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js" ] ; then 566 | # create a daily cron to make sure the banner does not re-appear 567 | cat <<'EOF' > /etc/cron.daily/xs-pve-nosub 568 | #!/bin/sh 569 | # eXtremeSHOK.com Remove subscription banner 570 | sed -i "s/data.status !== 'Active'/false/g" /usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js 571 | sed -i "s/checked_command: function(orig_cmd) {/checked_command: function() {} || function(orig_cmd) {/g" /usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js 572 | EOF 573 | chmod 755 /etc/cron.daily/xs-pve-nosub 574 | bash /etc/cron.daily/xs-pve-nosub 575 | fi 576 | # Remove nag @tinof 577 | echo "DPkg::Post-Invoke { \"dpkg -V proxmox-widget-toolkit | grep -q '/proxmoxlib\.js$'; if [ \$? -eq 1 ]; then { echo 'Removing subscription nag from UI...'; sed -i '/data.status/{s/\!//;s/Active/NoMoreNagging/}' /usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js; }; fi\"; };" > /etc/apt/apt.conf.d/xs-pve-no-nag && apt --reinstall install proxmox-widget-toolkit 578 | fi 579 | 580 | if [ "${XS_MOTD,,}" == "yes" ] ; then 581 | ## Pretty MOTD BANNER 582 | if ! grep -q https "/etc/motd" ; then 583 | cat << 'EOF' > /etc/motd.new 584 | This system is optimised by: eXtremeSHOK.com 585 | EOF 586 | 587 | cat /etc/motd >> /etc/motd.new 588 | mv /etc/motd.new /etc/motd 589 | fi 590 | fi 591 | 592 | if [ "${XS_KERNELPANIC,,}" == "yes" ] ; then 593 | # Enable restart on kernel panic 594 | cat < /etc/sysctl.d/99-xs-kernelpanic.conf 595 | # eXtremeSHOK.com 596 | # Enable restart on kernel panic, kernel oops and hardlockup 597 | kernel.core_pattern=/var/crash/core.%t.%p 598 | # Reboot on kernel panic afetr 10s 599 | kernel.panic=10 600 | # Panic on kernel oops, kernel exploits generally create an oops 601 | kernel.panic_on_oops=1 602 | # Panic on a hardlockup 603 | kernel.hardlockup_panic=1 604 | EOF 605 | fi 606 | 607 | if [ "${XS_LIMITS,,}" == "yes" ] ; then 608 | ## Increase max user watches 609 | # BUG FIX : No space left on device 610 | cat < /etc/sysctl.d/99-xs-maxwatches.conf 611 | # eXtremeSHOK.com 612 | # Increase max user watches 613 | fs.inotify.max_user_watches=1048576 614 | fs.inotify.max_user_instances=1048576 615 | fs.inotify.max_queued_events=1048576 616 | EOF 617 | ## Increase max FD limit / ulimit 618 | cat <> /etc/security/limits.d/99-xs-limits.conf 619 | # eXtremeSHOK.com 620 | # Increase max FD limit / ulimit 621 | * soft nproc 1048576 622 | * hard nproc 1048576 623 | * soft nofile 1048576 624 | * hard nofile 1048576 625 | root soft nproc unlimited 626 | root hard nproc unlimited 627 | root soft nofile unlimited 628 | root hard nofile unlimited 629 | EOF 630 | ## Increase kernel max Key limit 631 | cat < /etc/sysctl.d/99-xs-maxkeys.conf 632 | # eXtremeSHOK.com 633 | # Increase kernel max Key limit 634 | kernel.keys.root_maxkeys=1000000 635 | kernel.keys.maxkeys=1000000 636 | EOF 637 | ## Set systemd ulimits 638 | echo "DefaultLimitNOFILE=256000" >> /etc/systemd/system.conf 639 | echo "DefaultLimitNOFILE=256000" >> /etc/systemd/user.conf 640 | 641 | echo 'session required pam_limits.so' >> /etc/pam.d/common-session 642 | echo 'session required pam_limits.so' >> /etc/pam.d/runuser-l 643 | 644 | ## Set ulimit for the shell user 645 | echo "ulimit -n 256000" >> /root/.profile 646 | fi 647 | 648 | if [ "${XS_LOGROTATE,,}" == "yes" ] ; then 649 | ## Optimise logrotate 650 | cat < /etc/logrotate.conf 651 | # eXtremeSHOK.com 652 | daily 653 | su root adm 654 | rotate 7 655 | create 656 | compress 657 | size=10M 658 | delaycompress 659 | copytruncate 660 | 661 | include /etc/logrotate.d 662 | EOF 663 | systemctl restart logrotate 664 | fi 665 | 666 | if [ "${XS_JOURNALD,,}" == "yes" ] ; then 667 | ## Limit the size and optimise journald 668 | cat < /etc/systemd/journald.conf 669 | # eXtremeSHOK.com 670 | [Journal] 671 | # Store on disk 672 | Storage=persistent 673 | # Don't split Journald logs by user 674 | SplitMode=none 675 | # Disable rate limits 676 | RateLimitInterval=0 677 | RateLimitIntervalSec=0 678 | RateLimitBurst=0 679 | # Disable Journald forwarding to syslog 680 | ForwardToSyslog=no 681 | # Journald forwarding to wall /var/log/kern.log 682 | ForwardToWall=yes 683 | # Disable signing of the logs, save cpu resources. 684 | Seal=no 685 | Compress=yes 686 | # Fix the log size 687 | SystemMaxUse=64M 688 | RuntimeMaxUse=60M 689 | # Optimise the logging and speed up tasks 690 | MaxLevelStore=warning 691 | MaxLevelSyslog=warning 692 | MaxLevelKMsg=warning 693 | MaxLevelConsole=notice 694 | MaxLevelWall=crit 695 | EOF 696 | systemctl restart systemd-journald.service 697 | journalctl --vacuum-size=64M --vacuum-time=1d; 698 | journalctl --rotate 699 | fi 700 | 701 | if [ "${XS_ENTROPY,,}" == "yes" ] ; then 702 | ## Ensure Entropy Pools are Populated, prevents slowdowns whilst waiting for entropy 703 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install haveged 704 | ## Net optimising 705 | cat < /etc/default/haveged 706 | # eXtremeSHOK.com 707 | # -w sets low entropy watermark (in bits) 708 | DAEMON_ARGS="-w 1024" 709 | EOF 710 | systemctl daemon-reload 711 | systemctl enable haveged 712 | fi 713 | 714 | if [ "${XS_VZDUMP,,}" == "yes" ] ; then 715 | ## Increase vzdump backup speed 716 | sed -i "s/#bwlimit:.*/bwlimit: 0/" /etc/vzdump.conf 717 | sed -i "s/#ionice:.*/ionice: 5/" /etc/vzdump.conf 718 | fi 719 | 720 | if [ "${XS_MEMORYFIXES,,}" == "yes" ] ; then 721 | ## Optimise Memory 722 | cat < /etc/sysctl.d/99-xs-memory.conf 723 | # eXtremeSHOK.com 724 | # Memory Optimising 725 | ## Bugfix: reserve 1024MB memory for system 726 | vm.min_free_kbytes=1048576 727 | vm.nr_hugepages=72 728 | # (Redis/MongoDB) 729 | vm.max_map_count=262144 730 | vm.overcommit_memory = 1 731 | EOF 732 | fi 733 | 734 | if [ "${XS_TCPBBR,,}" == "yes" ] ; then 735 | ## Enable TCP BBR congestion control 736 | cat < /etc/sysctl.d/99-xs-kernel-bbr.conf 737 | # eXtremeSHOK.com 738 | # TCP BBR congestion control 739 | net.core.default_qdisc=fq 740 | net.ipv4.tcp_congestion_control=bbr 741 | EOF 742 | fi 743 | 744 | if [ "${XS_TCPFASTOPEN,,}" == "yes" ] ; then 745 | ## Enable TCP fastopen 746 | cat < /etc/sysctl.d/99-xs-tcp-fastopen.conf 747 | # eXtremeSHOK.com 748 | # TCP fastopen 749 | net.ipv4.tcp_fastopen=3 750 | EOF 751 | fi 752 | 753 | if [ "${XS_NET,,}" == "yes" ] ; then 754 | ## Enable Network optimising 755 | cat < /etc/sysctl.d/99-xs-net.conf 756 | # eXtremeSHOK.com 757 | net.core.netdev_max_backlog=8192 758 | net.core.optmem_max=8192 759 | net.core.rmem_max=16777216 760 | net.core.somaxconn=8151 761 | net.core.wmem_max=16777216 762 | net.ipv4.conf.all.accept_redirects = 0 763 | net.ipv4.conf.all.accept_source_route = 0 764 | net.ipv4.conf.all.log_martians = 0 765 | net.ipv4.conf.all.rp_filter = 1 766 | net.ipv4.conf.all.secure_redirects = 0 767 | net.ipv4.conf.all.send_redirects = 0 768 | net.ipv4.conf.default.accept_redirects = 0 769 | net.ipv4.conf.default.accept_source_route = 0 770 | net.ipv4.conf.default.log_martians = 0 771 | net.ipv4.conf.default.rp_filter = 1 772 | net.ipv4.conf.default.secure_redirects = 0 773 | net.ipv4.conf.default.send_redirects = 0 774 | net.ipv4.icmp_echo_ignore_broadcasts = 1 775 | net.ipv4.icmp_ignore_bogus_error_responses = 1 776 | net.ipv4.ip_local_port_range=1024 65535 777 | net.ipv4.tcp_base_mss = 1024 778 | net.ipv4.tcp_challenge_ack_limit = 999999999 779 | net.ipv4.tcp_fin_timeout=10 780 | net.ipv4.tcp_keepalive_intvl=30 781 | net.ipv4.tcp_keepalive_probes=3 782 | net.ipv4.tcp_keepalive_time=240 783 | net.ipv4.tcp_limit_output_bytes=65536 784 | net.ipv4.tcp_max_syn_backlog=8192 785 | net.ipv4.tcp_max_tw_buckets = 1440000 786 | net.ipv4.tcp_mtu_probing = 1 787 | net.ipv4.tcp_rfc1337=1 788 | net.ipv4.tcp_rmem=8192 87380 16777216 789 | net.ipv4.tcp_sack=1 790 | net.ipv4.tcp_slow_start_after_idle=0 791 | net.ipv4.tcp_syn_retries=3 792 | net.ipv4.tcp_synack_retries = 2 793 | net.ipv4.tcp_tw_recycle = 0 794 | net.ipv4.tcp_tw_reuse = 0 795 | net.ipv4.tcp_wmem=8192 65536 16777216 796 | net.netfilter.nf_conntrack_generic_timeout = 60 797 | net.netfilter.nf_conntrack_helper=0 798 | net.netfilter.nf_conntrack_max = 524288 799 | net.netfilter.nf_conntrack_tcp_timeout_established = 28800 800 | net.unix.max_dgram_qlen = 4096 801 | EOF 802 | fi 803 | 804 | if [ "${XS_SWAPPINESS,,}" == "yes" ] ; then 805 | ## Bugfix: high swap usage with low memory usage 806 | cat < /etc/sysctl.d/99-xs-swap.conf 807 | # eXtremeSHOK.com 808 | # Bugfix: high swap usage with low memory usage 809 | vm.swappiness=10 810 | EOF 811 | fi 812 | 813 | if [ "${XS_MAXFS,,}" == "yes" ] ; then 814 | ## Increase Max FS open files 815 | cat < /etc/sysctl.d/99-xs-fs.conf 816 | # eXtremeSHOK.com 817 | # Max FS Optimising 818 | fs.nr_open=12000000 819 | fs.file-max=9000000 820 | fs.aio-max-nr=524288 821 | EOF 822 | fi 823 | 824 | if [ "${XS_BASHRC,,}" == "yes" ] ; then 825 | ## Customise bashrc (thanks broeckca) 826 | cat <> /root/.bashrc 827 | export HISTTIMEFORMAT="%d/%m/%y %T " 828 | export PS1='\u@\h:\W \$ ' 829 | alias l='ls -CF' 830 | alias la='ls -A' 831 | alias ll='ls -alF' 832 | alias ls='ls --color=auto' 833 | source /etc/profile.d/bash_completion.sh 834 | export PS1="\[\e[31m\][\[\e[m\]\[\e[38;5;172m\]\u\[\e[m\]@\[\e[38;5;153m\]\h\[\e[m\] \[\e[38;5;214m\]\W\[\e[m\]\[\e[31m\]]\[\e[m\]\\$ " 835 | EOF 836 | echo "source /root/.bashrc" >> /root/.bash_profile 837 | fi 838 | 839 | if [ "${XS_ZFSARC,,}" == "yes" ] ; then 840 | ## Optimise ZFS arc size accoring to memory size 841 | if [ "$(command -v zfs)" != "" ] ; then 842 | if [[ RAM_SIZE_GB -le 16 ]] ; then 843 | MY_ZFS_ARC_MIN=536870911 844 | MY_ZFS_ARC_MAX=536870912 845 | elif [[ RAM_SIZE_GB -le 32 ]] ; then 846 | # 1GB/1GB 847 | MY_ZFS_ARC_MIN=1073741823 848 | MY_ZFS_ARC_MAX=1073741824 849 | else 850 | MY_ZFS_ARC_MIN=$((RAM_SIZE_GB * 1073741824 / 16)) 851 | MY_ZFS_ARC_MAX=$((RAM_SIZE_GB * 1073741824 / 8)) 852 | fi 853 | # Enforce the minimum, incase of a faulty vmstat 854 | if [[ MY_ZFS_ARC_MIN -lt 536870911 ]] ; then 855 | MY_ZFS_ARC_MIN=536870911 856 | fi 857 | if [[ MY_ZFS_ARC_MAX -lt 536870912 ]] ; then 858 | MY_ZFS_ARC_MAX=536870912 859 | fi 860 | cat < /etc/modprobe.d/99-xs-zfsarc.conf 861 | # eXtremeSHOK.com ZFS tuning 862 | 863 | # Use 1/8 RAM for MAX cache, 1/16 RAM for MIN cache, or 1GB 864 | options zfs zfs_arc_min=$MY_ZFS_ARC_MIN 865 | options zfs zfs_arc_max=$MY_ZFS_ARC_MAX 866 | 867 | # use the prefetch method 868 | options zfs l2arc_noprefetch=0 869 | 870 | # max write speed to l2arc 871 | # tradeoff between write/read and durability of ssd (?) 872 | # default : 8 * 1024 * 1024 873 | # setting here : 500 * 1024 * 1024 874 | options zfs l2arc_write_max=524288000 875 | options zfs zfs_txg_timeout=60 876 | EOF 877 | fi 878 | fi 879 | 880 | 881 | # Fix missing /etc/network/interfaces.d include 882 | if ! grep -q 'source /etc/network/interfaces.d/*' "/etc/network/interfaces" ; then 883 | echo "Added missing include to /etc/network/interfaces" 884 | echo "source /etc/network/interfaces.d/*" >> /etc/network/interfaces 885 | fi 886 | 887 | if [ "${XS_VFIO_IOMMU,,}" == "yes" ] ; then 888 | # Enable IOMMU 889 | cpu=$(cat /proc/cpuinfo) 890 | if [[ $cpu == *"GenuineIntel"* ]]; then 891 | echo "Detected Intel CPU" 892 | sed -i 's/quiet/quiet intel_iommu=on iommu=pt/g' /etc/default/grub 893 | elif [[ $cpu == *"AuthenticAMD"* ]]; then 894 | echo "Detected AMD CPU" 895 | sed -i 's/quiet/quiet amd_iommu=on iommu=pt/g' /etc/default/grub 896 | else 897 | echo "Unknown CPU" 898 | fi 899 | 900 | cat <> /etc/modules 901 | # eXtremeSHOK.com 902 | vfio 903 | vfio_iommu_type1 904 | vfio_pci 905 | vfio_virqfd 906 | 907 | EOF 908 | cat <> /etc/modprobe.d/blacklist.conf 909 | # eXtremeSHOK.com 910 | blacklist nouveau 911 | blacklist lbm-nouveau 912 | options nouveau modeset=0 913 | blacklist amdgpu 914 | blacklist radeon 915 | blacklist nvidia 916 | blacklist nvidiafb 917 | 918 | EOF 919 | 920 | fi 921 | 922 | # propagate the settings 923 | update-initramfs -u -k all 924 | update-grub 925 | pve-efiboot-tool refresh 926 | 927 | # cleanup 928 | ## Remove no longer required packages and purge old cached updates 929 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' autoremove 930 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' autoclean 931 | 932 | echo "# eXtremeSHOK.com" > /etc/extremeshok 933 | date >> /etc/extremeshok 934 | 935 | ## Script Finish 936 | echo -e '\033[1;33m Finished....please restart the system \033[0m' 937 | echo "Optimisations by https://eXtremeSHOK.com" 938 | -------------------------------------------------------------------------------- /networking/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/extremeshok/xshok-proxmox/35c323b8fb2f7fefb0e6dc214426db1230cbea3a/networking/README.md -------------------------------------------------------------------------------- /networking/network-addiprange.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | ################################################################################ 3 | # This is property of eXtremeSHOK.com 4 | # You are free to use, modify and distribute, however you may not remove this notice. 5 | # Copyright (c) Adrian Jon Kriel :: admin@extremeshok.com 6 | ################################################################################ 7 | # 8 | # Script updates can be found at: https://github.com/extremeshok/xshok-proxmox 9 | # 10 | # License: BSD (Berkeley Software Distribution) 11 | # 12 | ################################################################################ 13 | # 14 | # Creates default routes to allow for extra ip ranges to be used. 15 | # Tested on OVH servers with VLAN 16 | # 17 | # NOTE: WILL APPLY CHANGES TO /etc/network/interfaces 18 | # 19 | # Usage: 20 | # curl -O https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/networking/network-addiprange.sh && chmod +x network-addiprange.sh 21 | # ./network-addiprange.sh ip.xx.xx.xx/cidr interface_optional 22 | # ./network-addiprange.sh ip.xx.xx.xx / cidr interface_optional 23 | # ./network-addiprange.sh ip.xx.xx.xx interface_optional 24 | # 25 | # If no interface is specified the default gateway interface will be used. 26 | # 27 | ################################################################################ 28 | # 29 | # THERE ARE NO USER CONFIGURABLE OPTIONS IN THIS SCRIPT 30 | # 31 | ############################################################## 32 | 33 | # Set the local 34 | export LANG="en_US.UTF-8" 35 | export LC_ALL="C" 36 | 37 | #assign and check arguments 38 | if [ $# -lt "1" ] ; then 39 | echo "ERROR: missing aguments" 40 | echo "Usage: $(basename "$0") ip/mask optional_gateway_interface" 41 | exit 1 42 | else 43 | ipwithcidr="$1" 44 | fi 45 | myroute="$(command -v route)" 46 | if [ "$myroute" == "" ] ; then 47 | echo "ERROR: route command is missing" 48 | exit 1 49 | fi 50 | 51 | 52 | if [[ "$ipwithcidr" =~ "/" ]] ; then 53 | networkip=${ipwithcidr%/*} 54 | cidr=${ipwithcidr##*/} 55 | gatewaydev="$2" 56 | else 57 | networkip="$1" 58 | if [ "$2" == "/" ] ; then #assume xx.xx.xx.xx / 29 eth0 59 | cidr="$3" 60 | gatewaydev="$4" 61 | else 62 | echo "Info: IP missing cidr, assigning default: 32" 63 | cidr="32" 64 | gatewaydev="$3" 65 | fi 66 | fi 67 | if ! [[ $networkip =~ ^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$ ]]; then 68 | echo "ERROR: Invaid IP, use xxx.xxx.xxx.xxx/xx format: $networkip" 69 | exit 1 70 | fi 71 | if ! [ "$cidr" -eq "$cidr" ] 2> /dev/null ; then 72 | echo "Error: Invalid CIDR must be an integer $cidr" 73 | exit 1 74 | 75 | fi 76 | if [ "$cidr" -lt "1" ] || [ "$cidr" -gt "32" ] ; then 77 | echo "ERROR: Invalid CIDR $cidr" 78 | exit 1 79 | else 80 | totalip=$((2**(32-cidr) )) # y = 2^(32-x), x = CIDR class 81 | fi 82 | if [ "$gatewaydev" == "" ] ; then 83 | if ip -br link show | grep -v 'UNKNOWN' | grep -v '@' | grep -v ':' | cut -d' ' -f1 | grep -q vmbr0 ; then 84 | gatewaydev="vmbr0" 85 | else 86 | gatewaydev="$(route -4 | grep default | awk '{ print $NF }')" 87 | fi 88 | fi 89 | echo "Set gatewaydev to ${gatewaydev}" 90 | 91 | usableip=$((totalip - 2)) 92 | if [ "$usableip" -eq "0" ] ; then 93 | echo "ERROR: No usable IP ($totalip - 2 = $usableip)" 94 | exit 1 95 | fi 96 | ## Not used because this is slower than the if/else block 97 | #cdr2mask () { 98 | # set -- $(( 5 - ($1 / 8) )) 255 255 255 255 $(( (255 << (8 - ($1 % 8))) & 255 )) 0 0 0 99 | # if [[ "$1" -gt 1 ]] ; then shift "$1" ; else shift ; fi ; echo "${1-0}.${2-0}.${3-0}.${4-0}" 100 | #} 101 | #netmask="$(cdr2mask ""$cidr")" 102 | # get the netmask 103 | if ! [ "$((totalip/256**0))" -gt "256" ]; then # y 104 | netmask="255.255.255.$((256-(totalip/256**0) ))" # 256-(totalip) 105 | elif ! [ "$((totalip/256**1))" -gt "256" ]; then # totalip/256 106 | netmask="255.255.$((256-(totalip/256**1) )).0" # 256-(totalip/256) 107 | elif ! [ "$((totalip/256**2))" -gt "256" ]; then # totalip/256/256 108 | netmask="255.$((256-(totalip/256**2) )).0.0" # 256-(totalip/256/256) 109 | elif ! [ "$((totalip/256**3))" -gt "256" ]; then # totalip/256/256/256 110 | netmask="$((256-(totalip/256**3) )).0.0.0" # 256-(totalip/256/256/256) 111 | fi 112 | 113 | #informationmyroute="$(command -v route)" 114 | echo "UsableIP $usableip | TotalIP $totalip | Netmask $netmask | CIDR $cidr | NetworkIP $networkip | GatewayDev $gatewaydev" 115 | 116 | # Check if the route is currently in use, otherwise add it. 117 | res="$(route | grep "$networkip" | grep "$netmask" | grep "$gatewaydev")" 118 | if [ "$res" == "" ] ; then 119 | echo "Activating the route until restart" 120 | $myroute add -net "$networkip" netmask "$netmask" dev "$gatewaydev" 121 | else 122 | echo "Route is already active" 123 | fi 124 | 125 | #if_filename="${networkip}_${cidr}" 126 | #if_filename=${if_filename//./-} 127 | #if_filename="xs_${if_filename}.cfg" 128 | 129 | 130 | if [ ! -f "/etc/network/if-up.d/route" ] ; then 131 | echo '#!/usr/bin/env bash' > /etc/network/if-up.d/route 132 | chmod +x /etc/network/if-up.d/route 133 | fi 134 | if ! grep -q "${myroute} add -net ${networkip} netmask ${netmask} dev ${gatewaydev}" "/etc/network/if-up.d/route" ; then 135 | echo "Permantly added the route" 136 | echo "${myroute} add -net ${networkip} netmask ${netmask} dev ${gatewaydev}" >> "/etc/network/if-up.d/route" 137 | else 138 | echo "Route is already exists in /etc/network/if-up.d/route" 139 | fi 140 | 141 | #script Finish 142 | echo -e '\033[1;33m Finished \033[0m' 143 | -------------------------------------------------------------------------------- /networking/network-configure.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | ################################################################################ 3 | # This is property of eXtremeSHOK.com 4 | # You are free to use, modify and distribute, however you may not remove this notice. 5 | # Copyright (c) Adrian Jon Kriel :: admin@extremeshok.com 6 | ################################################################################ 7 | # 8 | # Script updates can be found at: https://github.com/extremeshok/xshok-proxmox 9 | # 10 | # License: BSD (Berkeley Software Distribution) 11 | # 12 | ################################################################################ 13 | # 14 | ## CREATES A ROUTED vmbr0 AND NAT vmbr1 NETWORK CONFIGURATION FOR PROXMOX 15 | # Autodetects the correct settings (interface, gatewat, netmask, etc) 16 | # Supports IPv4 and IPv6, Private Network uses 10.10.10.1/24 17 | # 18 | # Also installs and properly configures the isc-dhcp-server to allow for DHCP on the vmbr1 (NAT) 19 | # 20 | # ROUTED (vmbr0): 21 | # All traffic is routed via the main IP address and uses the MAC address of the physical interface. 22 | # VM's can have multiple IP addresses and they do NOT require a MAC to be set for the IP via service provider 23 | # 24 | # NAT (vmbr1): 25 | # Allows a VM to have internet connectivity without requiring its own IP address 26 | # Assignes 10.10.10.100 - 10.10.10.200 via DHCP 27 | # 28 | # Public IP's can be assigned via DHCP, adding a host define to the /etc/dhcp/hosts.public file 29 | # 30 | # Tested on OVH and Hetzner based servers 31 | # 32 | # ALSO CREATES A NAT Private Network as vmbr1 33 | # 34 | # NOTE: WILL OVERWRITE /etc/network/interfaces 35 | # A backup will be created as /etc/network/interfaces.timestamp 36 | # 37 | ################################################################################ 38 | # 39 | # THERE ARE NO USER CONFIGURABLE OPTIONS IN THIS SCRIPT 40 | # 41 | ################################################################################ 42 | 43 | # Set the local 44 | export LANG="en_US.UTF-8" 45 | export LC_ALL="C" 46 | 47 | network_interfaces_file="/etc/network/interfaces" 48 | 49 | #Detect and install dependencies 50 | if ! type "dhcpd" >& /dev/null; then 51 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install isc-dhcp-server 52 | fi 53 | 54 | if ! [ -f "network-addiprange.sh" ]; then 55 | echo "Downloading network-addiprange.sh script" 56 | curl -O https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/networking/network-addiprange.sh && chmod +x network-addiprange.sh 57 | fi 58 | if ! grep -q '#!/usr/bin/env bash' "network-addiprange.sh"; then 59 | echo "ERROR: network-addiprange.sh is invalid" 60 | fi 61 | 62 | if ! [ -f "/etc/sysctl.d/99-networking.conf" ]; then 63 | echo "Creating /etc/sysctl.d/99-networking.conf" 64 | cat > /etc/sysctl.d/99-networking.conf < "$network_interfaces_file" <> /etc/pve/datacenter.cfg 173 | 174 | ### Load extra files, ie for extra gateways 175 | source /etc/network/interfaces.d/* 176 | 177 | EOF 178 | 179 | default_v6="$(ip -6 addr show dev "$default_interface" | awk '/global/ { print $2}')" 180 | default_v6ip=${default_v6%/*} 181 | default_v6mask=${default_v6#*/} 182 | default_v6gateway="$(ip -6 route | awk '/default/ { print $3 }')" 183 | 184 | if [ "$default_v6ip" != "" ] && [ "$default_v6mask" != "" ] && [ "$default_v6gateway" != "" ]; then 185 | cat >> "$network_interfaces_file" << EOF 186 | ### IPv6 ### 187 | iface ${default_interface} inet6 static 188 | address ${default_v6ip} 189 | netmask ${default_v6mask} 190 | gateway ${default_v6gateway} 191 | 192 | iface vmbr0 inet6 static 193 | address ${default_v6ip} 194 | netmask 64 195 | 196 | EOF 197 | fi 198 | 199 | 200 | cat >> "$network_interfaces_file" << EOF 201 | ### Extra IP/IP Ranges ### 202 | # Use ./network-addiprange.sh script to add ip/ip ranges or edit the examples below 203 | # 204 | ## Example add IP range 176.9.216.192/27 205 | # up route add -net 94.130.239.192 netmask 255.255.255.192 dev vmbr0 206 | ## Example add IP 176.9.123.158 207 | # up route add -net 176.9.123.158 netmask 255.255.255.255 dev vmbr0 208 | 209 | EOF 210 | 211 | 212 | # Configure isc-dhcp-server 213 | if [ -f "/etc/default/isc-dhcp-server" ] ; then 214 | cp /etc/default/isc-dhcp-server "/etc/default/isc-dhcp-server.$(date +"%Y-%m-%d_%H-%M-%S")" 215 | fi 216 | if [ -f "/etc/dhcp/dhcpd.conf" ] ; then 217 | cp /etc/dhcp/dhcpd.conf "/etc/dhcp/dhcpd.conf.$(date +"%Y-%m-%d_%H-%M-%S")" 218 | fi 219 | 220 | cat > /etc/default/isc-dhcp-server < /etc/dhcp/dhcpd.conf < "/etc/dhcp/hosts.public" < -p -c -a -r (reset) -u (uninstall)" ; exit ;; 59 | esac 60 | done 61 | 62 | if [ "$reset" == "yes" ] || [ "$uninstall" == "yes" ] ; then 63 | echo "Stopping Tinc" 64 | systemctl stop tinc-xsvpn.service 65 | pkill -9 tincd 66 | 67 | echo "Removing configs" 68 | rm -rf /etc/tinc/my_default_v4ip 69 | rm -rf /etc/tinc/xsvpn 70 | mv -f /etc/tinc/nets.boot.orig /etc/tinc/nets.boot 71 | rm -f /etc/network/interfaces.d/tinc-vpn.cfg 72 | rm -f /etc/systemd/system/tinc-xsvpn.service 73 | 74 | if [ "$uninstall" == "yes" ] ; then 75 | systemctl disable tinc.service 76 | echo "Tinc uninstalled" 77 | exit 0 78 | fi 79 | fi 80 | 81 | if [ "$(command -v tinc)" == "" ] ; then 82 | if [ "$(command -v apt-get)" != "" ] ; then 83 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install tinc 84 | else 85 | echo "ERROR: tinc not installed" 86 | exit 1 87 | fi 88 | fi 89 | 90 | 91 | if [ "$my_default_v4ip" == "" ] ; then 92 | #detect default ipv4 and default interface 93 | default_interface="$(ip route | awk '/default/ { print $5 }' | grep -v "vmbr")" 94 | if [ "$default_interface" == "" ]; then 95 | #filter the interfaces to get the default interface and which is not down and not a virtual bridge 96 | default_interface="$(ip link | sed -e '/state DOWN / { N; d; }' | sed -e '/veth[0-9].*:/ { N; d; }' | sed -e '/vmbr[0-9].*:/ { N; d; }' | sed -e '/tap[0-9].*:/ { N; d; }' | sed -e '/lo:/ { N; d; }' | head -n 1 | cut -d':' -f 2 | xargs)" 97 | fi 98 | if [ "$default_interface" == "" ]; then 99 | echo "ERROR: Could not detect default interface" 100 | exit 1 101 | fi 102 | default_v4="$(ip -4 addr show dev "$default_interface" | awk '/inet/ { print $2 }' )" 103 | my_default_v4ip=${default_v4%/*} 104 | if [ "$my_default_v4ip" == "" ] ; then 105 | echo "ERROR: Could not detect default IPv4 address" 106 | echo "IP: ${my_default_v4ip}" 107 | exit 1 108 | fi 109 | fi 110 | 111 | # Assign and Fix varibles 112 | 113 | my_name=$(uname -n) 114 | my_name=${my_name//-/_} 115 | 116 | if [ "$vpn_connect_to" != "${vpn_connect_to//-/_}" ]; then 117 | echo "ERROR: - character is not allowed in hostname for vpn_connect_to" 118 | exit 1 119 | fi 120 | 121 | echo "Options:" 122 | echo "VPN IP: 10.10.1.${vpn_ip_last}" 123 | echo "VPN PORT: ${vpn_port}" 124 | echo "VPN Connect to host: ${vpn_connect_to}" 125 | echo "Public Address: ${my_default_v4ip}" 126 | 127 | # Detect and Install 128 | if [ "$(command -v tincd)" == "" ] ; then 129 | echo "Tinc not found, installing...." 130 | if [ "$(command -v yum)" != "" ] ; then 131 | yum install -y tinc 132 | elif [ "$(command -v apt-get)" != "" ] ; then 133 | apt-get install -y tinc 134 | fi 135 | fi 136 | 137 | #Create the DIR and key files 138 | mkdir -p /etc/tinc/xsvpn/hosts 139 | touch /etc/tinc/xsvpn/rsa_key.pub 140 | touch /etc/tinc/xsvpn/rsa_key.priv 141 | 142 | if [ "$(grep "BEGIN RSA PUBLIC KEY" /etc/tinc/xsvpn/rssa_key.pub 2> /dev/null)" != "" ] ; then 143 | if [ "$(grep "BEGIN RSA PRIVATE KEY" /etc/tinc/xsvpn/rssa_key.priv 2> /dev/null)" != "" ] ; then 144 | echo "Using Previous RSA Keys" 145 | else 146 | echo "Generating New RSA Keys" 147 | tincd -K4096 -c /etc/tinc/xsvpn /dev/null 148 | fi 149 | else 150 | echo "Generating New 4096 bit RSA Keys" 151 | tincd -K4096 -c /etc/tinc/xsvpn /dev/null 152 | fi 153 | 154 | #Generate Configs 155 | cat < /etc/tinc/xsvpn/tinc.conf 156 | Name = $my_name 157 | AddressFamily = ipv4 158 | Interface = Tun0 159 | Mode = switch 160 | # Switch: Unicast, multicast and broadcast packaets 161 | ConnectTo = $vpn_connect_to 162 | EOF 163 | 164 | cat < "/etc/tinc/xsvpn/hosts/$my_name" 165 | Address = ${my_default_v4ip} 166 | Subnet = 10.10.1.${vpn_ip_last} 167 | Port = ${vpn_port} 168 | Compression = 10 #LZO 169 | EOF 170 | cat /etc/tinc/xsvpn/rsa_key.pub >> "/etc/tinc/xsvpn/hosts/${my_name}" 171 | 172 | cat < /etc/tinc/xsvpn/tinc-up 173 | #!/usr/bin/env bash 174 | ip link set \$INTERFACE up 175 | ip addr add 10.10.1.${vpn_ip_last}/24 dev \$INTERFACE 176 | ip route add 10.10.1.0/24 dev \$INTERFACE 177 | 178 | # Set a multicast route over interface 179 | route add -net 224.0.0.0 netmask 240.0.0.0 dev \$INTERFACE 180 | 181 | # To allow IP forwarding: 182 | #echo 1 > /proc/sys/net/ipv4/ip_forward 183 | 184 | # To limit the chance of Corosync Totem re-transmission issues: 185 | #echo 0 > /sys/devices/virtual/net/\$INTERFACE/bridge/multicast_snooping 186 | EOF 187 | 188 | chmod 755 /etc/tinc/xsvpn/tinc-up 189 | 190 | cat < /etc/tinc/xsvpn/tinc-down 191 | #!/usr/bin/env bash 192 | ip route del 10.10.1.0/24 dev \$INTERFACE 193 | ip addr del 10.10.1.${vpn_ip_last}/24 dev \$INTERFACE 194 | ip link set \$INTERFACE down 195 | 196 | # Set a multicast route over interface 197 | route del -net 224.0.0.0 netmask 240.0.0.0 dev \$INTERFACE 198 | 199 | #echo 0 > /proc/sys/net/ipv4/ip_forward 200 | EOF 201 | 202 | chmod 755 /etc/tinc/xsvpn/tinc-down 203 | 204 | # Set which VPN to start 205 | #cp -f /etc/tinc/nets.boot /etc/tinc/nets.boot.orig 206 | #echo "vpn" >> /etc/tinc/nets.boot 207 | 208 | cat < /etc/systemd/system/tinc-xsvpn.service 209 | [Unit] 210 | Description=eXtremeSHOK.com Tinc VPN 211 | After=network.target 212 | 213 | [Service] 214 | Type=simple 215 | WorkingDirectory=/etc/tinc/xsvpn 216 | ExecStart=$(command -v tincd) -n xsvpn -D -d2 217 | ExecReload=$(command -v tincd) -n xsvpn -kHUP 218 | TimeoutStopSec=5 219 | Restart=always 220 | RestartSec=60 221 | 222 | [Install] 223 | WantedBy=multi-user.target 224 | EOF 225 | 226 | # Enable at Boot 227 | systemctl enable tinc-xsvpn.service 228 | 229 | # Add a Tun0 entry to /etc/network/interfaces to allow for ceph suport over the VPN 230 | if [ "$(grep "source /etc/network/interfaces.d/*" /etc/network/interfaces 2> /dev/null)" == "" ] ; then 231 | echo "source /etc/network/interfaces.d/*" >> /etc/network/interfaces 232 | mkdir -p /etc/network/interfaces.d/ 233 | fi 234 | if [ ! -f /etc/network/interfaces.d/tinc-vpn.cfg ]; then 235 | cat < /etc/network/interfaces.d/tinc-vpn.cfg 236 | # tinc vpn 237 | iface Tun0 inet static 238 | address 10.10.1.${vpn_ip_last} 239 | netmask 255.255.255.0 240 | broadcast 0.0.0.0 241 | EOF 242 | fi 243 | 244 | #Display the Host config for simple cpy-paste to another node 245 | echo "" 246 | echo "Run the following on the other VPN nodes:" 247 | echo "The following information is stored in /etc/tinc/xsvpn/this_host.info" 248 | 249 | echo 'cat <> /etc/tinc/xsvpn/hosts/'"${my_name}" > /etc/tinc/xsvpn/this_host.info 250 | cat "/etc/tinc/xsvpn/hosts/${my_name}" >> /etc/tinc/xsvpn/this_host.info 251 | echo "EOF" >> /etc/tinc/xsvpn/this_host.info 252 | 253 | echo "" 254 | echo 'cat <> /etc/tinc/xsvpn/hosts/'"${my_name}" 255 | cat "/etc/tinc/xsvpn/hosts/${my_name}" 256 | echo "EOF" 257 | echo "" 258 | -------------------------------------------------------------------------------- /nvidia/nvidia-docker.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # thanks @88plug 3 | curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | \ 4 | apt-key add - 5 | distribution=$(. /etc/os-release;echo $ID$VERSION_ID) 6 | curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | \ 7 | tee /etc/apt/sources.list.d/nvidia-docker.list 8 | apt-get update 9 | 10 | # Install nvidia-docker2 and reload the Docker daemon configuration 11 | apt-get install -y nvidia-docker2 12 | pkill -SIGHUP dockerd 13 | 14 | reboot 15 | -------------------------------------------------------------------------------- /nvidia/readme.md: -------------------------------------------------------------------------------- 1 | # initial work by @88plug 2 | 3 | xserver-xorg-dev dkms 4 | 5 | #!/bin/bash 6 | apt-get install build-essential pve-headers-$(uname -r) pkg-config libgtk-3-0 libglvnd-dev 7 | update-grub 8 | REBOOT 9 | 10 | 11 | #Old Driver 12 | #wget https://us.download.nvidia.com/XFree86/Linux-x86_64/455.38/NVIDIA-Linux-x86_64-455.38.run 13 | #chmod +x NVIDIA-Linux-x86_64-455.38.run 14 | #./NVIDIA-Linux-x86_64-455.38.run 15 | 16 | wget https://us.download.nvidia.com/XFree86/Linux-x86_64/460.56/NVIDIA-Linux-x86_64-460.56.run 17 | chmod +x NVIDIA-Linux-x86_64-460.56.run 18 | ./NVIDIA-Linux-x86_64-460.56.run 19 | 20 | Installer will ask to create modeprobe file, say YES! 21 | Reboot 22 | Run ./NVIDIA-Linux-x86_64-455.38.run again 23 | 24 | WARNING: nvidia-installer was forced to guess the X library path '/usr/lib' and X module path '/usr/lib/xorg/modules'; these paths were not queryable from the system. If X fails to find the NVIDIA X driver module, please 25 | install the `pkg-config` utility and the X.Org SDK/development package for your distribution and reinstall the driver 26 | 27 | YES to 32 bit dependencies 28 | 29 | Would you like to run the nvidia-xconfig utility to automatically update your X configuration file so that the NVIDIA X driver will be used when you restart X? Any pre-existing X configuration file will be backed up. 30 | 31 | NO 32 | 33 | 34 | REBOOT 35 | nvidia-smi! 36 | 37 | Now run ./docker.sh to install nvidia-docker! 38 | 39 | Unlock card with 40 | sudo nvidia-xconfig -a --cool-bits=31 --allow-empty-initial-configuration 41 | nvidia-smi -pl 200 -i 0 42 | 43 | Now if you want to overclock you need a Xauthority/gdm 44 | 45 | tasksel > install gnome desktop 46 | then run this command 47 | systemctl mask sleep.target suspend.target hibernate.target hybrid-sleep.target 48 | as gnome stupidly enables sleep by default! 49 | finally reboot...you should now have a folder 50 | /run/user/$SOMENUMBER/gdm/Xauthority 51 | 52 | replace $SOMENUBMER in lines below! :) 53 | 54 | DISPLAY=:0 XAUTHORITY=/run/user/121/gdm/Xauthority sudo nvidia-settings -a [gpu:0]/GPUFanControlState=1 -a [fan-0]/GPUTargetFanSpeed=80 55 | sleep 3 56 | DISPLAY=:0 XAUTHORITY=/run/user/121/gdm/Xauthority sudo nvidia-settings -a [gpu:1]/GPUFanControlState=1 -a [fan-1]/GPUTargetFanSpeed=80 57 | sleep 3 58 | DISPLAY=:0 XAUTHORITY=/run/user/121/gdm/Xauthority sudo nvidia-settings -a [gpu:2]/GPUFanControlState=1 -a [fan-2]/GPUTargetFanSpeed=80 59 | sleep 3 60 | DISPLAY=:0 XAUTHORITY=/run/user/121/gdm/Xauthority sudo nvidia-settings -a [gpu:3]/GPUFanControlState=1 -a [fan-3]/GPUTargetFanSpeed=85 61 | 62 | DISPLAY=:0 XAUTHORITY=/run/user/121/gdm/Xauthority nvidia-settings -a '[gpu:0]/GPUGraphicsClockOffset[3]=150' 63 | DISPLAY=:0 XAUTHORITY=/run/user/121/gdm/Xauthority nvidia-settings -a '[gpu:0]/GPUMemoryTransferRateOffset[3]=600' 64 | DISPLAY=:0 XAUTHORITY=/run/user/121/gdm/Xauthority nvidia-settings -a '[gpu:1]/GPUGraphicsClockOffset[3]=150' 65 | DISPLAY=:0 XAUTHORITY=/run/user/121/gdm/Xauthority nvidia-settings -a '[gpu:1]/GPUMemoryTransferRateOffset[3]=600' 66 | DISPLAY=:0 XAUTHORITY=/run/user/121/gdm/Xauthority nvidia-settings -a '[gpu:2]/GPUGraphicsClockOffset[3]=150' 67 | DISPLAY=:0 XAUTHORITY=/run/user/121/gdm/Xauthority nvidia-settings -a '[gpu:2]/GPUMemoryTransferRateOffset[3]=600' 68 | DISPLAY=:0 XAUTHORITY=/run/user/121/gdm/Xauthority nvidia-settings -a '[gpu:3]/GPUGraphicsClockOffset[3]=150' 69 | DISPLAY=:0 XAUTHORITY=/run/user/121/gdm/Xauthority nvidia-settings -a '[gpu:3]/GPUMemoryTransferRateOffset[3]=600' 70 | -------------------------------------------------------------------------------- /ovh/README.md: -------------------------------------------------------------------------------- 1 | # OVH Proxmox Installation Guide # 2 | Select install for the specific server, via the ovh manager 3 | * --INSTALL--> 4 | * Install from an OVH template 5 | * --NEXT--> 6 | * Type of OS: Ready-to-go (graphical user interface) 7 | * VPS Proxmox VE *(pick the latest non zfs version)* 8 | * Language: EN 9 | * Target disk arrray: *(always select the SSD array if you have ssd and hdd arrays)* 10 | * Enable/Tick: Customise the partition configuration 11 | * --NEXT--> 12 | * Disks used for this installation: *(All of them)* 13 | * (Remove all the partitions and do the following) 14 | * Type: Filesystem: Mount Point: LVM Name: RAID: Size: 15 | * * 1 primary Ext4 / - 1 20.0 GB 16 | * * 2 primary Swap swap - - 2 x 8.0 GB *(minimum 16GB total, set recommended swap size)* 17 | * * 3 LV xfs /var/lib/vz data 1 REMAINING GB *(use all the remaining space)* 18 | * --NEXT--> 19 | * Hostname: server.fqdn.com 20 | * Installation script (URL): https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/install-post.sh 21 | * Script return value: 0 22 | * SSH keys: *(always suggested, however if this value is used a webinterface login will not work without setting a root password in shell)* 23 | * --CONFIRM--> 24 | After installation, Connect via ssh/terminal to the new Proxmox system running on your server and run the following 25 | ## LVM to ZFS 26 | ```` 27 | wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/zfs/lvm-2-zfs.sh -c -O lvm-2-zfs.sh && chmod +x lvm-2-zfs.sh 28 | ./lvm-2-zfs.sh && rm lvm-2-zfs.sh 29 | ```` 30 | * Reboot 31 | * Connect via ssh/terminal to the new Proxmox system running on your server and run the following 32 | ## NETWORKING (vmbr0 vmbr1) 33 | ``` 34 | wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/networking/network-configure.sh -c -O network-configure.sh && chmod +x network-configure.sh 35 | ./network-configure.sh && rm network-configure.sh 36 | ``` 37 | * Reboot 38 | * Post Install: Now login via ssh as root and create a password, which will be used for the webinterface when logging in with pam authentication 39 | 40 | # Advance Installation Options # 41 | Assumptions: Proxmox installed, SSD raid1 partitions mounted as /xshok/zfs-slog and /xshok/zfs-cache, 1+ unused hdd which will be made into a zfspool 42 | 43 | * Connect via ssh/terminal to the new Proxmox system running on your server and run the follow 44 | ## Create ZFS from unused devices (createzfs.sh) 45 | 46 | **NOTE: WILL DESTROY ALL DATA ON SPECIFIED DEVICES** 47 | ``` 48 | wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/zfs/createzfs.sh -c -O createzfs.sh && chmod +x createzfs.sh 49 | ./createzfs.sh poolname /dev/device1 /dev/device2 50 | ``` 51 | ## Create ZFS cache and slog from /xshok/zfs-cache and /xshok/zfs-slog partitions and adds them to a zpool (xshok_slog_cache-2-zfs.sh) *optional* 52 | 53 | **NOTE: WILL DESTROY ALL DATA ON SPECIFIED PARTITIONS** 54 | ``` 55 | wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/zfs/xshok_slog_cache-2-zfs.sh -c -O xshok_slog_cache-2-zfs.sh && chmod +x xshok_slog_cache-2-zfs.sh 56 | ./xshok_slog_cache-2-zfs.sh poolname 57 | ``` 58 | * Reboot 59 | -------------------------------------------------------------------------------- /xs-install-post.env.sample: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | # shellcheck disable=2034 3 | ################################################################################ 4 | # This is property of eXtremeSHOK.com 5 | # You are free to use, modify and distribute, however you may not remove this notice. 6 | # Copyright (c) Adrian Jon Kriel :: admin@extremeshok.com 7 | ################################################################################ 8 | # 9 | # Script updates can be found at: https://github.com/extremeshok/xshok-proxmox 10 | # 11 | ################################################################################ 12 | # 13 | # User Defined Options for (install-post.sh) post-installation script for Proxmox 14 | # 15 | # Usage: place in the same folder as install-post.sh and rename to xs-install-post.env 16 | # 17 | #### VARIABLES 18 | # 19 | # yes = enabled 20 | # no = disabled 21 | # 22 | #### VARIABLES / options 23 | # Detect AMD EPYC and Ryzen CPU and Apply Fixes 24 | XS_AMDFIXES="yes" 25 | # Force APT to use IPv4 26 | XS_APTIPV4="yes" 27 | # Update proxmox and install various system utils 28 | XS_APTUPGRADE="yes" 29 | # Customise bashrc 30 | XS_BASHRC="yes" 31 | # Add the latest ceph provided by proxmox 32 | XS_CEPH="no" 33 | # Disable portmapper / rpcbind (security) 34 | XS_DISABLERPC="yes" 35 | # Ensure Entropy Pools are Populated, prevents slowdowns whilst waiting for entropy 36 | XS_ENTROPY="yes" 37 | # Protect the web interface with fail2ban 38 | XS_FAIL2BAN="yes" 39 | # Detect if is a virtual machine and install the relavant guest agent 40 | XS_GUESTAGENT="yes" 41 | # Install ifupdown2 for a virtual internal network allows rebootless networking changes (not compatible with openvswitch-switch) 42 | XS_IFUPDOWN2="yes" 43 | # Limit the size and optimise journald 44 | XS_JOURNALD="yes" 45 | # Install kernel source headers 46 | XS_KERNELHEADERS="yes" 47 | # Ensure ksmtuned (ksm-control-daemon) is enabled and optimise according to ram size 48 | XS_KSMTUNED="yes" 49 | # Set language, if changed will disable XS_NOAPTLANG 50 | XS_LANG="en_US.UTF-8" 51 | # Enable restart on kernel panic, kernel oops and hardlockup 52 | XS_KERNELPANIC="yes" 53 | # Increase max user watches, FD limit, FD ulimit, max key limit, ulimits 54 | XS_LIMITS="yes" 55 | # Optimise logrotate 56 | XS_LOGROTATE="yes" 57 | # Lynis security scan tool by Cisofy 58 | XS_LYNIS="yes" 59 | # Increase Max FS open files 60 | XS_MAXFS="yes" 61 | # Optimise Memory 62 | XS_MEMORYFIXES="yes" 63 | # Pretty MOTD BANNER 64 | XS_MOTD="yes" 65 | # Enable Network optimising 66 | XS_NET="yes" 67 | # Save bandwidth and skip downloading additional languages, requires XS_LANG="en_US.UTF-8" 68 | XS_NOAPTLANG="yes" 69 | # Disable enterprise proxmox repo 70 | XS_NOENTREPO="yes" 71 | # Remove subscription banner 72 | XS_NOSUBBANNER="yes" 73 | # Install openvswitch for a virtual internal network 74 | XS_OPENVSWITCH="no" 75 | # Detect if this is an OVH server and install OVH Real Time Monitoring 76 | XS_OVHRTM="yes" 77 | # Set pigz to replace gzip, 2x faster gzip compression 78 | XS_PIGZ="yes" 79 | # Bugfix: high swap usage with low memory usage 80 | XS_SWAPPINESS="yes" 81 | # Enable TCP BBR congestion control 82 | XS_TCPBBR="yes" 83 | # Enable TCP fastopen 84 | XS_TCPFASTOPEN="yes" 85 | # Enable testing proxmox repo 86 | XS_TESTREPO="no" 87 | # Automatically Synchronize the time 88 | XS_TIMESYNC="yes" 89 | # Set Timezone, empty = set automatically by IP 90 | XS_TIMEZONE="" 91 | # Install common system utilities 92 | XS_UTILS="yes" 93 | # Increase vzdump backup speed 94 | XS_VZDUMP="yes" 95 | # Optimise ZFS arc size accoring to memory size 96 | XS_ZFSARC="yes" 97 | # Install zfs-auto-snapshot 98 | XS_ZFSAUTOSNAPSHOT="no" 99 | # Enable VFIO IOMMU support for PCIE passthrough 100 | XS_VFIO_IOMMU="yes" 101 | -------------------------------------------------------------------------------- /zfs/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/extremeshok/xshok-proxmox/35c323b8fb2f7fefb0e6dc214426db1230cbea3a/zfs/README.md -------------------------------------------------------------------------------- /zfs/benchmark_zfs.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | ################################################################################ 3 | # This is property of eXtremeSHOK.com 4 | # You are free to use, modify and distribute, however you may not remove this notice. 5 | # Copyright (c) Adrian Jon Kriel :: admin@extremeshok.com 6 | ################################################################################ 7 | # 8 | # Script updates can be found at: https://github.com/extremeshok/xshok-proxmox 9 | # Based on https://blog.programster.org/zfs-add-intent-log-device 10 | # 11 | # License: BSD (Berkeley Software Distribution) 12 | # 13 | ################################################################################ 14 | 15 | # Set the local 16 | export LANG="en_US.UTF-8" 17 | export LC_ALL="C" 18 | 19 | if [ ! -e /usr/bin/time ] ; then 20 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install time 21 | fi 22 | 23 | echo "Performing cached write of 1,000,000 4k blocks..." 24 | /usr/bin/time -f "%e" sh -c 'dd if=/dev/zero of=4k-test.img bs=4k count=1000000 2> /dev/null' 25 | 26 | rm 4k-test.img 27 | echo "" 28 | sleep 3 29 | 30 | 31 | echo "Performing cached write of 10,000 1M blocks..." 32 | /usr/bin/time -f "%e" sh -c 'dd if=/dev/zero of=1GB.img bs=1M count=10000 2> /dev/null' 33 | 34 | rm 1GB.img 35 | echo "" 36 | sleep 3 37 | 38 | 39 | echo "Performing non-cached write of 1,000,000 4k blocks..." 40 | /usr/bin/time -f "%e" sh -c 'dd if=/dev/zero of=4k-test.img bs=4k count=1000000 conv=fdatasync 2> /dev/null' 41 | 42 | rm 4k-test.img 43 | echo "" 44 | sleep 3 45 | 46 | 47 | echo "Performing non-cached write of 10,000 1M blocks..." 48 | /usr/bin/time -f "%e" sh -c 'dd if=/dev/zero of=1GB.img bs=1M count=10000 conv=fdatasync 2> /dev/null' 49 | 50 | rm 1GB.img 51 | echo "" 52 | sleep 3 53 | 54 | 55 | echo "Performing sequential write of 10,000 4k blocks..." 56 | /usr/bin/time -f "%e" sh -c 'dd if=/dev/zero of=4k-test.img bs=4k count=10000 oflag=dsync 2> /dev/null' 57 | 58 | rm 4k-test.img 59 | echo "" 60 | sleep 3 61 | 62 | echo "Performing sequential write of 10,000 1M blocks..." 63 | /usr/bin/time -f "%e" sh -c 'dd if=/dev/zero of=1GB.img bs=1M count=10000 oflag=dsync 2> /dev/null' 64 | 65 | rm 1GB.img 66 | -------------------------------------------------------------------------------- /zfs/createzfs.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | ################################################################################ 3 | # This is property of eXtremeSHOK.com 4 | # You are free to use, modify and distribute, however you may not remove this notice. 5 | # Copyright (c) Adrian Jon Kriel :: admin@extremeshok.com 6 | ################################################################################ 7 | # 8 | # Script updates can be found at: https://github.com/extremeshok/xshok-proxmox 9 | # 10 | # Will create a ZFS pool from the devices specified with the correct raid level 11 | # 12 | # Note: compatible with all debian based distributions 13 | # If proxmox is detected, it will add the pools to the storage system 14 | # 15 | # License: BSD (Berkeley Software Distribution) 16 | # 17 | ################################################################################ 18 | # 19 | # Creates the following storage/rpools 20 | # poolnamebackup (poolname/backup) 21 | # poolnamevmdata (poolname/vmdata) 22 | # 23 | # zfs-auto-snapshot is disabled on the backup (poolname/backup) 24 | # 25 | # Will automatically detect the required raid level and optimise. 26 | # 27 | # Will automatically resolve device names (eg. /dev/sda) to device id (eg. SSD1_16261489FFCA) 28 | # 29 | # 1 Drive = zfs 30 | # 2 Drives = mirror 31 | # 3-5 Drives = raidz-1 32 | # 6-11 Drives = raidz-2 33 | # 11+ Drives = raidz-3 34 | # 35 | # NOTE: WILL DESTROY ALL DATA ON DEVICES SPECIFED 36 | # 37 | # Usage: 38 | # curl -O https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/zfs/createzfs.sh && chmod +x createzfs.sh 39 | # ./createzfs.sh poolname /dev/sda /dev/sdb 40 | # 41 | ################################################################################ 42 | # 43 | # THERE ARE USER CONFIGURABLE OPTIONS IN THIS SCRIPT 44 | # ALL CONFIGURATION OPTIONS ARE LOCATED BELOW THIS MESSAGE 45 | # 46 | ############################################################## 47 | 48 | #/dev/md3 4.9G 20M 4.6G 1% /xshok/zfs-slog 49 | #/dev/md2 59G 53M 56G 1% /xshok/zfs-cache 50 | 51 | # Set the local 52 | export LANG="en_US.UTF-8" 53 | export LC_ALL="C" 54 | 55 | poolname=${1} 56 | zfsdevicearray=("${@:2}") 57 | 58 | #Detect and install dependencies 59 | if ! type "zpool" >& /dev/null; then 60 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install zfsutils-linux 61 | modprobe zfs 62 | fi 63 | if ! type "parted" >& /dev/null; then 64 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install parted 65 | fi 66 | 67 | #check arguments 68 | if [ $# -lt "2" ] ; then 69 | echo "Usage: $(basename "$0") poolname /list/of /dev/device1 /dev/device2" 70 | echo "Note will append 'pool' to the poolname, eg. hdd -> hddpool" 71 | echo "Will automatically resolve device names (eg. /dev/sda) to device id (eg. SSD1_16261489FFCA)" 72 | echo "Device names, /dev/disk/by-id" 73 | # shellcheck disable=2010 74 | ls -l /dev/disk/by-id/ | grep -v "\\-part*" | grep -v "wwn\\-*" | grep -v "usb\\-*" | cut -d" " -f10-20 | sed 's|../../|/dev/|' | awk NF 75 | exit 1 76 | fi 77 | if [[ "$poolname" =~ "/" ]] ; then 78 | echo "ERROR: invalid poolname: $poolname" 79 | exit 1 80 | fi 81 | if [ "${#zfsdevicearray[@]}" -lt "1" ] ; then 82 | echo "ERROR: less than 1 devices were detected" 83 | exit 1 84 | fi 85 | 86 | #add the suffix pool to the poolname, prevent namepoolpool 87 | poolprefix=${poolname/pool/} 88 | poolname="${poolprefix}pool" 89 | 90 | INDEX=0 91 | for zfsdevice in "${zfsdevicearray[@]}" ; do 92 | if ! [[ "$zfsdevice" =~ "/" ]] ; then 93 | if ! [[ "$zfsdevice" =~ "-" ]] ; then 94 | echo "ERROR: Invalid device specified: $zfsdevice" 95 | exit 1 96 | fi 97 | fi 98 | if ! [ -e "$zfsdevice" ]; then 99 | if ! [ -e "/dev/disk/by-id/$zfsdevice" ]; then 100 | if ! [ -e "/dev/disk/by-uuid/$zfsdevice" ]; then 101 | echo "ERROR: Device $zfsdevice does not exist" 102 | exit 1 103 | fi 104 | fi 105 | fi 106 | if grep -q "$zfsdevice" "/proc/mounts" ; then 107 | echo "ERROR: Device is mounted $zfsdevice" 108 | exit 1 109 | fi 110 | echo "Clearing partitions: ${zfsdevice}" 111 | for v_partition in $(parted -s "${zfsdevice}" print|awk '/^ / {print $1}') ; do 112 | parted -s "${zfsdevice}" rm "${v_partition}" 2> /dev/null 113 | done 114 | 115 | if [[ "$zfsdevice" =~ "/" ]] ; then 116 | MY_DEV="${zfsdevice/*\//}" 117 | # shellcheck disable=2010 118 | MY_DEV="$(ls -l /dev/disk/by-id/ | grep -i "/${MY_DEV}\$" | grep -o 'ata[^ ]*')" 119 | if ! [ -e "/dev/disk/by-id/${MY_DEV}" ]; then 120 | echo "ERROR: Device $zfsdevice does not exist" 121 | exit 1 122 | else 123 | echo "${zfsdevice} -> ${MY_DEV}" 124 | #replace current value 125 | zfsdevicearray[$INDEX]="${MY_DEV}" 126 | fi 127 | fi 128 | ((INDEX++)) 129 | done 130 | 131 | echo "Enable ZFS to autostart and mount" 132 | systemctl enable zfs.target 133 | systemctl enable zfs-mount 134 | systemctl enable zfs-import-cache 135 | 136 | echo "Ensure ZFS is started" 137 | systemctl start zfs.target 138 | modprobe zfs 139 | 140 | if [ "$(zpool import 2> /dev/null | grep -m 1 -o "\\s$poolname\\b")" == "$poolname" ] ; then 141 | echo "ERROR: $poolname already exists as an exported pool" 142 | zpool import 143 | exit 1 144 | fi 145 | if [ "$(zpool list 2> /dev/null | grep -m 1 -o "\\s$poolname\\b")" == "$poolname" ] ; then 146 | echo "ERROR: $poolname already exists as a listed pool" 147 | zpool list 148 | exit 1 149 | fi 150 | 151 | echo "Creating the array" 152 | if [ "${#zfsdevicearray[@]}" -eq "1" ] ; then 153 | echo "Creating ZFS single" 154 | # shellcheck disable=2068 155 | zpool create -f -o ashift=12 -O compression=lz4 -O checksum=on "$poolname" ${zfsdevicearray[@]} 156 | ret=$? 157 | elif [ "${#zfsdevicearray[@]}" -eq "2" ] ; then 158 | echo "Creating ZFS mirror (raid1)" 159 | # shellcheck disable=2068 160 | zpool create -f -o ashift=12 -O compression=lz4 -O checksum=on "$poolname" mirror ${zfsdevicearray[@]} 161 | ret=$? 162 | elif [ "${#zfsdevicearray[@]}" -ge "3" ] && [ "${#zfsdevicearray[@]}" -le "5" ] ; then 163 | echo "Creating ZFS raidz-1 (raid5)" 164 | # shellcheck disable=2068 165 | zpool create -f -o ashift=12 -O compression=lz4 -O checksum=on "$poolname" raidz ${zfsdevicearray[@]} 166 | ret=$? 167 | elif [ "${#zfsdevicearray[@]}" -ge "6" ] && [ "${#zfsdevicearray[@]}" -lt "11" ] ; then 168 | echo "Creating ZFS raidz-2 (raid6)" 169 | # shellcheck disable=2068 170 | zpool create -f -o ashift=12 -O compression=lz4 -O checksum=on "$poolname" raidz2 ${zfsdevicearray[@]} 171 | ret=$? 172 | elif [ "${#zfsdevicearray[@]}" -ge "11" ] ; then 173 | echo "Creating ZFS raidz-3 (raid7)" 174 | # shellcheck disable=2068 175 | zpool create -f -o ashift=12 -O compression=lz4 -O checksum=on "$poolname" raidz3 ${zfsdevicearray[@]} 176 | ret=$? 177 | fi 178 | 179 | if [ $ret != 0 ] ; then 180 | echo "ERROR: creating ZFS" 181 | exit 1 182 | fi 183 | 184 | if [ "$( zpool list | grep "$poolname" | cut -f 1 -d " ")" != "$poolname" ] ; then 185 | echo "ERROR: $poolname pool not found" 186 | zpool list 187 | exit 1 188 | fi 189 | 190 | echo "Creating Secondary ZFS volumes" 191 | echo "-- ${poolname}/vmdata" 192 | zfs create "${poolname}/vmdata" 193 | echo "-- ${poolname}/backup (/backup_${poolprefix})" 194 | 195 | #export the pool 196 | zpool export "${poolname}" 197 | sleep 10 198 | zpool import "${poolname}" 199 | sleep 5 200 | 201 | echo "Optimising ${poolname}" 202 | zfs set compression=on "${poolname}" 203 | zfs set compression=lz4 "${poolname}" 204 | zfs set primarycache=all "${poolname}" 205 | zfs set atime=off "${poolname}" 206 | zfs set relatime=off "${poolname}" 207 | zfs set checksum=on "${poolname}" 208 | zfs set dedup=off "${poolname}" 209 | zfs set xattr=sa "${poolname}" 210 | 211 | # disable zfs-auto-snapshot on backup pools 212 | zfs set com.sun:auto-snapshot=false "${poolname}/backup" 213 | 214 | #check we do not already have a cron for zfs 215 | if [ ! -f "/etc/cron.d/zfsutils-linux" ] ; then 216 | if [ -f /usr/lib/zfs-linux/scrub ] ; then 217 | cat <<'EOF' > /etc/cron.d/zfsutils-linux 218 | PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin 219 | 220 | # Scrub the pool every second Sunday of every month. 221 | 24 0 8-14 * * root [ $(date +\%w) -eq 0 ] && [ -x /usr/lib/zfs-linux/scrub ] && /usr/lib/zfs-linux/scrub 222 | EOF 223 | else 224 | echo "Scrub the pool every second Sunday of every month ${poolname}" 225 | if [ ! -f "/etc/cron.d/zfs-scrub" ] ; then 226 | echo "PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin" > "/etc/cron.d/zfs-scrub" 227 | fi 228 | echo "24 0 8-14 * * root [ \$(date +\\%w) -eq 0 ] && zpool scrub ${poolname}" >> "/etc/cron.d/zfs-scrub" 229 | fi 230 | fi 231 | 232 | # pvesm (proxmox) is optional 233 | if type "pvesm" >& /dev/null; then 234 | # https://pve.proxmox.com/pve-docs/pvesm.1.html 235 | echo "Adding the ZFS storage pools to Proxmox GUI" 236 | echo "-- ${poolname}-vmdata" 237 | pvesm add zfspool "${poolname}-vmdata" --pool "${poolname}/vmdata" --sparse 1 238 | echo "-- ${poolname}-backup" 239 | pvesm add dir "${poolname}-backup" --path "/backup_${poolprefix}" 240 | fi 241 | 242 | ### Work in progress , create specialised pools ### 243 | # echo "ZFS 8GB swap partition" 244 | # zfs create -V 8G -b $(getconf PAGESIZE) -o logbias=throughput -o sync=always -o primarycache=metadata -o com.sun:auto-snapshot=false "$poolname"/swap 245 | # mkswap -f /dev/zvol/"$poolname"/swap 246 | # swapon /dev/zvol/"$poolname"/swap 247 | # /dev/zvol/"$poolname"/swap none swap discard 0 0 248 | # 249 | # echo "ZFS tmp partition" 250 | # zfs create -o setuid=off -o devices=off -o sync=disabled -o mountpoint=/tmp -o atime=off "$poolname"/tmp 251 | ## note: if you want /tmp on ZFS, mask (disable) systemd's automatic tmpfs-backed /tmp 252 | # systemctl mask tmp.mount 253 | # 254 | # echo "RDBMS partition (MySQL/PostgreSQL/Oracle)" 255 | # zfs create -o recordsize=8K -o primarycache=metadata -o mountpoint=/rdbms -o logbias=throughput "$poolname"/rdbms 256 | 257 | zpool iostat -v "${poolname}" -L -T d 258 | 259 | #script Finish 260 | echo -e '\033[1;33m Finished....please restart the server \033[0m' 261 | -------------------------------------------------------------------------------- /zfs/lvm-2-zfs.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | ################################################################################ 3 | # This is property of eXtremeSHOK.com 4 | # You are free to use, modify and distribute, however you may not remove this notice. 5 | # Copyright (c) Adrian Jon Kriel :: admin@extremeshok.com 6 | ################################################################################ 7 | # 8 | # Script updates can be found at: https://github.com/extremeshok/xshok-proxmox 9 | # 10 | # post-installation script for Proxmox 11 | # 12 | # License: BSD (Berkeley Software Distribution) 13 | # 14 | ################################################################################ 15 | # 16 | # Assumptions: proxmox installed via OVH manager (non zfs) 17 | # Remaining for /var/lib/vz (LVM) 18 | # 19 | # Creates the following storage/rpools 20 | # zfsvmdata (rpool/vmdata) 21 | # zfsbackup (rpool/backup) 22 | # /var/lib/vz/tmp_backup (rpool/tmp_backup) 23 | # zfs-auto-snapshot is disabled on the rpool/backup and rpool/tmp_backup 24 | # 25 | # Will automatically detect the required raid level and optimise. 26 | # 27 | # 1 Drive = zfs 28 | # 2 Drives = mirror 29 | # 3-5 Drives = raidz-1 30 | # 6-11 Drives = raidz-2 31 | # 11+ Drives = raidz-3 32 | # 33 | # NOTE: WILL DESTROY ALL DATA ON LVM_MOUNT_POINT, default is /var/lib/vz 34 | # 35 | # Assumes LVM on top of a MD raid (linux software raid) 36 | # 37 | # Usage: 38 | # curl -O https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/lvm2zfs.sh && chmod +x lvm-2-zfs.sh 39 | # ./lvm-2-zfs.sh LVM_MOUNT_POINT 40 | # 41 | ################################################################################ 42 | # 43 | # THERE ARE NO USER CONFIGURABLE OPTIONS IN THIS SCRIPT 44 | # 45 | ################################################################################ 46 | 47 | # Set the local 48 | export LANG="en_US.UTF-8" 49 | export LC_ALL="C" 50 | 51 | LVM_MOUNT_POINT="$1" 52 | 53 | if [ "$LVM_MOUNT_POINT" == "" ]; then 54 | # The default LVM mount which will be replaced with ZFS 55 | LVM_MOUNT_POINT="/var/lib/vz" 56 | fi 57 | 58 | echo "+++++++++++++++++++++++++" 59 | echo "WILL DESTROY ALL DATA ON" 60 | echo "$LVM_MOUNT_POINT" 61 | echo "+++++++++++++++++++++++++" 62 | echo "[CTRL]+[C] to exit" 63 | echo "+++++++++++++++++++++++++" 64 | sleep 1 65 | echo "5.." ; sleep 1 66 | echo "4.." ; sleep 1 67 | echo "3.." ; sleep 1 68 | echo "2.." ; sleep 1 69 | echo "1.." ; sleep 1 70 | echo "STARTING CONVERSION" 71 | sleep 1 72 | 73 | #check mountpiont exists and is a device 74 | MY_LVM_DEV=$(mount | grep -i "$LVM_MOUNT_POINT" | cut -d " " -f 1) 75 | ret=$? 76 | if [ $ret == 0 ] && [ "$MY_LVM_DEV" != "" ] ; then 77 | echo "Found partition, continuing" 78 | echo "MY_LVM_DEV=$MY_LVM_DEV" #/dev/mapper/pve-data 79 | else 80 | echo "ERROR: $LVM_MOUNT_POINT not found" 81 | exit 1 82 | fi 83 | 84 | #Detect and install dependencies 85 | if [ "$(command -v zpool)" == "" ] ; then 86 | if [ "$(command -v apt-get)" != "" ] ; then 87 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install zfsutils-linux 88 | modprobe zfs 89 | else 90 | echo "ERROR: ZFS not installed" 91 | exit 1 92 | fi 93 | fi 94 | if [ "$(command -v zpool)" == "" ] ; then 95 | echo "ERROR: ZFS not installed" 96 | exit 1 97 | fi 98 | 99 | MY_MD_RAID=$(pvdisplay 2> /dev/null | sed -n -e 's/^.*\/dev\///p') 100 | ret=$? 101 | if [ $ret == 0 ] ; then 102 | echo "Found raid, continuing" 103 | echo "MY_MD_RAID=$MY_MD_RAID" #md5 104 | else 105 | echo "ERROR: $MY_MD_RAID not found" 106 | exit 1 107 | fi 108 | 109 | #pve/data 110 | MY_LV=$(lvdisplay "$MY_LVM_DEV" 2> /dev/null | sed -n -e 's/^.*\/dev\///p') 111 | ret=$? 112 | if [ $ret == 0 ] ; then 113 | echo "Found lv, continuing" 114 | echo "MY_LV=$MY_LV" #sda1 115 | else 116 | echo "ERROR: $MY_LV not found" 117 | exit 1 118 | fi 119 | 120 | IFS=' ' read -r -a mddevarray <<< "$(grep "$MY_MD_RAID :" /proc/mdstat | cut -d ' ' -f5- | xargs)" 121 | 122 | if [ "${mddevarray[0]}" == "" ] ; then 123 | echo "ERROR: no devices found for $MY_MD_RAID in /proc/mdstat" 124 | exit 1 125 | fi 126 | #check there is a minimum of 1 drives detected, not needed, but i rather have it. 127 | if [ "${#mddevarray[@]}" -lt "1" ] ; then 128 | echo "ERROR: less than 1 devices were detected" 129 | exit 1 130 | fi 131 | 132 | if [ "$MY_LVM_DEV" != "" ] && [ "$MY_MD_RAID" != "" ] && [ "$MY_LV" != "" ] ; then 133 | echo "All required varibles detected" 134 | else 135 | echo "ERROR: required varible not found or the server is already converted to zfs" 136 | exit 1 137 | fi 138 | 139 | # remove [*] and /dev/ to each record 140 | echo "Creating the device array" 141 | for index in "${!mddevarray[@]}" ; do 142 | tempmddevarraystring="${mddevarray[index]}" 143 | mddevarray[$index]="/dev/${tempmddevarraystring%\[*\]}" 144 | done 145 | 146 | echo "Destroying LV (logical volume)" 147 | echo umount -l "$LVM_MOUNT_POINT" 148 | umount -l "$LVM_MOUNT_POINT" 149 | echo lvremove "/dev/$MY_LV" -y 2> /dev/null 150 | lvremove "/dev/$MY_LV" -y 2> /dev/null 151 | 152 | echo "Destroying MD (linux raid)" 153 | echo mdadm --stop "/dev/$MY_MD_RAID" 154 | mdadm --stop "/dev/$MY_MD_RAID" 155 | echo mdadm --remove "/dev/$MY_MD_RAID" 156 | mdadm --remove "/dev/$MY_MD_RAID" 157 | 158 | for MY_LVM_DEV in "${mddevarray[@]}" ; do 159 | echo "zeroing $MY_LVM_DEV" 160 | echo mdadm --zero-superblock "$MY_LVM_DEV" 161 | mdadm --zero-superblock "$MY_LVM_DEV" 162 | done 163 | 164 | # #used to make a max free space lvm 165 | # #lvcreate -n ZFS pve -l 100%FREE -y 166 | if [ "${#mddevarray[@]}" -eq "1" ] ; then 167 | echo "Creating ZFS single" 168 | echo zpool create -f -o ashift=12 -O compression=lz4 -O checksum=on rpool "${mddevarray[@]}" 169 | zpool create -f -o ashift=12 -O compression=lz4 -O checksum=on rpool "${mddevarray[@]}" 170 | ret=$? 171 | elif [ "${#mddevarray[@]}" -eq "2" ] ; then 172 | echo "Creating ZFS mirror (raid1)" 173 | echo zpool create -f -o ashift=12 -O compression=lz4 -O checksum=on rpool mirror "${mddevarray[@]}" 174 | zpool create -f -o ashift=12 -O compression=lz4 -O checksum=on rpool mirror "${mddevarray[@]}" 175 | ret=$? 176 | elif [ "${#mddevarray[@]}" -ge "3" ] && [ "${#mddevarray[@]}" -le "5" ] ; then 177 | echo "Creating ZFS raidz-1 (raid5)" 178 | echo zpool create -f -o ashift=12 -O compression=lz4 -O checksum=on rpool raidz "${mddevarray[@]}" 179 | zpool create -f -o ashift=12 -O compression=lz4 -O checksum=on rpool raidz "${mddevarray[@]}" 180 | ret=$? 181 | elif [ "${#mddevarray[@]}" -ge "6" ] && [ "${#mddevarray[@]}" -lt "11" ] ; then 182 | echo "Creating ZFS raidz-2 (raid6)" 183 | echo zpool create -f -o ashift=12 -O compression=lz4 -O checksum=on rpool raidz2 "${mddevarray[@]}" 184 | zpool create -f -o ashift=12 -O compression=lz4 -O checksum=on rpool raidz2 "${mddevarray[@]}" 185 | ret=$? 186 | elif [ "${#mddevarray[@]}" -ge "11" ] ; then 187 | echo "Creating ZFS raidz-3 (raid7)" 188 | echo zpool create -f -o ashift=12 -O compression=lz4 -O checksum=on rpool raidz3 "${mddevarray[@]}" 189 | zpool create -f -o ashift=12 -O compression=lz4 -O checksum=on rpool raidz3 "${mddevarray[@]}" 190 | ret=$? 191 | fi 192 | 193 | if [ $ret != 0 ] ; then 194 | echo "ERROR: creating ZFS" 195 | exit 1 196 | fi 197 | 198 | echo "Creating Secondary ZFS volumes" 199 | echo "-- rpool/vmdata" 200 | echo zfs create rpool/vmdata 201 | zfs create rpool/vmdata 202 | echo "-- rpool/backup (/backup_rpool)" 203 | echo zfs create -o mountpoint=/backup_rpool rpool/backup 204 | zfs create -o mountpoint=/backup_rpool rpool/backup 205 | echo "ZFS tmp_backup partition" 206 | zfs create -o setuid=off -o devices=off -o sync=disabled -o mountpoint=/var/lib/vz/tmp_backup -o atime=off rpool/tmp_backup 207 | 208 | 209 | #export the pool 210 | echo zpool export rpool 211 | zpool export rpool 212 | echo sleep 5 213 | sleep 5 214 | echo zpool import rpool 215 | zpool import rpool 216 | echo sleep 5 217 | sleep 5 218 | 219 | echo "Cleaning up fstab / mounts" 220 | #/dev/pve/data /var/lib/vz ext3 defaults 1 2 221 | grep -v "$LVM_MOUNT_POINT" /etc/fstab > /tmp/fstab.new && mv /tmp/fstab.new /etc/fstab 222 | 223 | echo "Optimising rpool" 224 | zfs set compression=on "rpool" 225 | zfs set compression=lz4 "rpool" 226 | zfs set primarycache=all "rpool" 227 | zfs set atime=off "rpool" 228 | zfs set relatime=off "rpool" 229 | zfs set checksum=on "rpool" 230 | zfs set dedup=off "rpool" 231 | zfs set xattr=sa "rpool" 232 | # disable zfs-auto-snapshot on backup pools 233 | zfs set com.sun:auto-snapshot=false rpool/backup 234 | zfs set com.sun:auto-snapshot=false rpool/tmp_backup 235 | 236 | 237 | #check we do not already have a cron for zfs 238 | if [ ! -f "/etc/cron.d/zfsutils-linux" ] ; then 239 | if [ -f /usr/lib/zfs-linux/scrub ] ; then 240 | cat <<'EOF' > /etc/cron.d/zfsutils-linux 241 | PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin 242 | 243 | # Scrub the pool every second Sunday of every month. 244 | 24 0 8-14 * * root [ $(date +\%w) -eq 0 ] && [ -x /usr/lib/zfs-linux/scrub ] && /usr/lib/zfs-linux/scrub 245 | EOF 246 | else 247 | echo "Scrub the pool every second Sunday of every month rpool" 248 | if [ ! -f "/etc/cron.d/zfs-scrub" ] ; then 249 | echo "PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin" > "/etc/cron.d/zfs-scrub" 250 | fi 251 | echo "24 0 8-14 * * root [ \$(date +\\%w) -eq 0 ] && zpool scrub rpool" >> "/etc/cron.d/zfs-scrub" 252 | fi 253 | fi 254 | 255 | if type "pvesm" >& /dev/null; then 256 | # https://pve.proxmox.com/pve-docs/pvesm.1.html 257 | echo "Adding the ZFS storage pools to Proxmox GUI" 258 | echo "-- rpool-vmdata" 259 | echo pvesm add zfspool rpool-vmdata --pool rpool/vmdata --sparse 1 260 | pvesm add zfspool rpool-vmdata --pool rpool/vmdata --sparse 1 261 | echo "-- rpool-backup" 262 | echo pvesm add dir rpool-backup --path /backup_rpool 263 | pvesm add dir rpool-backup --path /backup_rpool 264 | fi 265 | 266 | zpool iostat -v "rpool" -L -T d 267 | 268 | #script Finish 269 | echo -e '\033[1;33m Finished....please restart the server \033[0m' 270 | -------------------------------------------------------------------------------- /zfs/xshok_slog_cache-2-zfs.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | ################################################################################ 3 | # This is property of eXtremeSHOK.com 4 | # You are free to use, modify and distribute, however you may not remove this notice. 5 | # Copyright (c) Adrian Jon Kriel :: admin@extremeshok.com 6 | ################################################################################ 7 | # 8 | # Script updates can be found at: https://github.com/extremeshok/xshok-proxmox 9 | # 10 | # post-installation script for Proxmox 11 | # 12 | # License: BSD (Berkeley Software Distribution) 13 | # 14 | ################################################################################ 15 | # 16 | # Assumptions: /xshok/zfs-cache and/or /xshok/zfs-slog are mounted. 17 | # 18 | # Assumes mounted MD raid partitions (linux software raid) 19 | # 20 | # Usage: 21 | # curl -O https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/zfs/xshok_slog_cache-2-zfs.sh && chmod +x xshok_slog_cache-2-zfs.sh 22 | # ./xshok_slog_cache-2-zfs.sh MY_ZFS_POOL 23 | # 24 | # NOTES: remove slog with 25 | # zpool remove MYPOOL mirror-1 26 | # NOTES: remove cache with 27 | # zpool remove DEVICE 28 | # 29 | ################################################################################ 30 | # 31 | # THERE ARE NO USER CONFIGURABLE OPTIONS IN THIS SCRIPT 32 | # 33 | ################################################################################ 34 | 35 | # Set the local 36 | export LANG="en_US.UTF-8" 37 | export LC_ALL="C" 38 | 39 | MY_ZFS_POOL="$1" 40 | 41 | if [ "$MY_ZFS_POOL" == "" ]; then 42 | #DEFAULT ZFS POOL 43 | MY_ZFS_POOL="hddpool" 44 | fi 45 | 46 | declare -a XSHOK_MOUNTS=('/xshok/zfs-cache' '/xshok/zfs-slog'); 47 | 48 | echo "+++++++++++++++++++++++++" 49 | echo "WILL DESTROY ALL DATA ON" 50 | echo "${XSHOK_MOUNTS[@]}" 51 | echo "+++++++++++++++++++++++++" 52 | echo "[CTRL]+[C] to exit" 53 | echo "+++++++++++++++++++++++++" 54 | sleep 1 55 | echo "5.." ; sleep 1 56 | echo "4.." ; sleep 1 57 | echo "3.." ; sleep 1 58 | echo "2.." ; sleep 1 59 | echo "1.." ; sleep 1 60 | echo "STARTING CONVERSION" 61 | sleep 1 62 | 63 | for XSHOK_MOUNT_POINT in "${XSHOK_MOUNTS[@]}" ; do 64 | echo "$XSHOK_MOUNT_POINT" 65 | #check mountpiont exists and is a device 66 | XSHOK_MOUNT_POINT_DEV=$(mount | grep -i "$XSHOK_MOUNT_POINT" | cut -d " " -f 1) 67 | ret=$? 68 | if [ $ret == 0 ] && [ "$XSHOK_MOUNT_POINT_DEV" != "" ] ; then 69 | echo "Found partition, continuing" 70 | echo "XSHOK_MOUNT_POINT_DEV=$XSHOK_MOUNT_POINT_DEV" #/dev/mapper/pve-data 71 | else 72 | echo "SKIPPING: $XSHOK_MOUNT_POINT not found" 73 | break 74 | fi 75 | 76 | #Detect and install dependencies 77 | if [ "$(command -v zpool)" == "" ] ; then 78 | if [ "$(command -v apt-get)" != "" ] ; then 79 | /usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install zfsutils-linux 80 | modprobe zfs 81 | else 82 | echo "ERROR: ZFS not installed" 83 | exit 1 84 | fi 85 | fi 86 | if [ "$(command -v zpool)" == "" ] ; then 87 | echo "ERROR: ZFS not installed" 88 | exit 1 89 | fi 90 | if [ "$(command -v tune2fs)" == "" ] ; then 91 | echo "ERROR: tune2fs not installed" 92 | exit 1 93 | fi 94 | 95 | if ! zpool status "$MY_ZFS_POOL" 2> /dev/null ; then 96 | echo "ERROR: ZFS pool ${MY_ZFS_POOL} not found" 97 | exit 1 98 | fi 99 | 100 | XSHOK_MOUNT_POINT_MD_RAID=${XSHOK_MOUNT_POINT_DEV/\/dev\//} 101 | 102 | IFS=' ' read -r -a mddevarray <<< "$(grep "$XSHOK_MOUNT_POINT_MD_RAID :" /proc/mdstat | cut -d ' ' -f5- | xargs)" 103 | 104 | if [ "${mddevarray[0]}" == "" ] ; then 105 | echo "ERROR: no devices found for $XSHOK_MOUNT_POINT_DEV in /proc/mdstat" 106 | #exit 1 107 | fi 108 | #check there is a minimum of 1 drives detected, not needed, but i rather have it. 109 | if [ "${#mddevarray[@]}" -lt "1" ] ; then 110 | echo "ERROR: less than 1 devices were detected" 111 | #exit 1 112 | fi 113 | 114 | # remove [*] and /dev/ to each record 115 | echo "Creating the device array" 116 | for index in "${!mddevarray[@]}" ; do 117 | tempmddevarraystring="${mddevarray[index]}" 118 | mddevarray[$index]="/dev/${tempmddevarraystring%\[*\]}" 119 | done 120 | 121 | echo "Destroying MD (linux raid)" 122 | echo umount -f "${XSHOK_MOUNT_POINT_DEV}" 123 | umount -f "${XSHOK_MOUNT_POINT_DEV}" 124 | echo mdadm --stop "${XSHOK_MOUNT_POINT_DEV}" 125 | mdadm --stop "${XSHOK_MOUNT_POINT_DEV}" 126 | echo mdadm --remove "${XSHOK_MOUNT_POINT_DEV}" 127 | mdadm --remove "${XSHOK_MOUNT_POINT_DEV}" 128 | echo "Cleaning up fstab / mounts" 129 | grep -v "$XSHOK_MOUNT_POINT" /etc/fstab > /tmp/fstab.new && mv /tmp/fstab.new /etc/fstab 130 | 131 | MY_MD_DEV_UUID_LIST="" 132 | for MY_MD_DEV in "${mddevarray[@]}" ; do 133 | echo "zeroing $MY_MD_DEV" 134 | echo mdadm --zero-superblock "$MY_MD_DEV" 135 | mdadm --zero-superblock "$MY_MD_DEV" 136 | #MY_MD_DEV_UUID="$(uuidgen)" 137 | #tune2fs "$MY_MD_DEV" -U "$MY_MD_DEV_UUID" 138 | MY_MD_DEV_UUID="$(blkid | grep "$MY_MD_DEV" | cut -d= -f2 | xargs)" 139 | MY_MD_DEV_UUID_LIST="${MY_MD_DEV_UUID_LIST} ${MY_MD_DEV_UUID}" 140 | done 141 | 142 | if [ "$XSHOK_MOUNT_POINT" == "/xshok/zfs-cache" ] ; then 143 | echo "Adding ${mddevarray[*]} to ${MY_ZFS_POOL} as CACHE" 144 | echo "$MY_MD_DEV_UUID_LIST" 145 | zpool add ${MY_ZFS_POOL} cache "$MY_MD_DEV_UUID_LIST" 146 | elif [ "$XSHOK_MOUNT_POINT" == "/xshok/zfs-slog" ] ; then 147 | echo "Adding ${mddevarray[*]} to ${MY_ZFS_POOL} as SLOG" 148 | echo "$MY_MD_DEV_UUID_LIST" 149 | if [ "${#mddevarray[@]}" -eq "1" ] ; then 150 | zpool add ${MY_ZFS_POOL} log "$MY_MD_DEV_UUID_LIST" 151 | else 152 | zpool add ${MY_ZFS_POOL} log mirror "$MY_MD_DEV_UUID_LIST" 153 | fi 154 | else 155 | echo "SKIPPING: Nothing todo with the partions" 156 | echo "${mddevarray[@]}" 157 | echo "$MY_MD_DEV_UUID_LIST" 158 | fi 159 | done 160 | 161 | zpool iostat -v "$MY_ZFS_POOL" -L -T d 162 | 163 | 164 | #script Finish 165 | echo -e '\033[1;33m Finished....please restart the server \033[0m' 166 | --------------------------------------------------------------------------------