├── README.md └── hosters ├── hetzner-cloud ├── README.md └── nixos-install-hetzner-cloud.sh ├── hetzner-dedicated ├── hetzner-dedicated-wipe-and-install-nixos.sh └── zfs-uefi-nvme-nixos.sh ├── leaseweb-dedicated └── leaseweb-dedicated-wipe-and-install-nixos.sh └── ovh-dedicated └── ovh-dedicated-wipe-and-install-nixos.sh /README.md: -------------------------------------------------------------------------------- 1 | # nixos-install-scripts 2 | 3 | A collection of one-shot scripts to install NixOS on various server hosters and other hardware. 4 | 5 | They are designed to get NixOS onto your machine with minimal effort, usually requiring only a single command and waiting a few minutes. 6 | 7 | See the `hosters` directory for available hosters. 8 | 9 | 10 | ## Usage 11 | 12 | Each script contains instructions at the top. 13 | 14 | You must slighly modify the script, most importantly, to put your login credentials (SSH key) into it. 15 | -------------------------------------------------------------------------------- /hosters/hetzner-cloud/README.md: -------------------------------------------------------------------------------- 1 | # Instructions 2 | 3 | 1. Create a server with any distribution (Ubuntu for example) from the Hetzner UI. 4 | 2. In your server options on the Hetzner UI click on `Mount image`. Find the NixOS image and mount it. Reboot the server into it. 5 | 3. You won't be able to ssh into your server. You will have to do the following steps by logging in with the Hetzner UI. 6 | - On the top right of the server details click on the icon `>_`. 7 | - Fork this repo and replace your SSH public key in the script. For exapmle, make a commit on a new branch. 8 | - Get the URL of the script you created (might look like `https://raw.github.com/YOUR_USER_NAME/nixos-install-scripts/YOUR_BRANCH_NAME/hosters/hetzner-cloud/nixos-install-hetzner-cloud.sh`). 9 | - Paste the following command into your console gui `curl -L YOUR_URL | sudo bash`. 10 | This will install NixOS and turn the server off. 11 | 4. In your server options on the Hetzner UI click on `Unmount`, and turn the server back on using the big power-switch button in the top right. 12 | 5. On your own computer you can now ssh into the newly created machine. 13 | 14 | # Troubleshooting 15 | 16 | ## SSH `WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!` 17 | 18 | if you have already ssh-ed into your server before installing NixOS, you will have to remove the server from your .know-hosts file. 19 | Open the file (located in your .ssh directory) and delete the entry corresponding to the server. 20 | 21 | ## Curl 404 error 22 | 23 | if you copy and paste a url from the Hetzner console GUI, some characters can get replaced (If you are using an English/US keyboard layout for example). Just replace the offending characters (check the punctuation especially, @:_ and the likes). 24 | -------------------------------------------------------------------------------- /hosters/hetzner-cloud/nixos-install-hetzner-cloud.sh: -------------------------------------------------------------------------------- 1 | #! /usr/bin/env bash 2 | 3 | # Script to install NixOS from the Hetzner Cloud NixOS bootable ISO image. 4 | # (tested with Hetzner's `NixOS 20.03 (amd64/minimal)` ISO image). 5 | # 6 | # This script wipes the disk of the server! 7 | # 8 | # Instructions: 9 | # 10 | # 1. Mount the above mentioned ISO image from the Hetzner Cloud GUI 11 | # and reboot the server into it; do not run the default system (e.g. Ubuntu). 12 | # 2. To be able to SSH straight in (recommended), you must replace hardcoded pubkey 13 | # further down in the section labelled "Replace this by your SSH pubkey" by you own, 14 | # and host the modified script way under a URL of your choosing 15 | # (e.g. gist.github.com with git.io as URL shortener service). 16 | # 3. Run on the server: 17 | # 18 | # # Replace this URL by your own that has your pubkey in 19 | # curl -L https://raw.githubusercontent.com/nix-community/nixos-install-scripts/master/hosters/hetzner-cloud/nixos-install-hetzner-cloud.sh | sudo bash 20 | # 21 | # This will install NixOS and power off the server. 22 | # 4. Unmount the ISO image from the Hetzner Cloud GUI. 23 | # 5. Turn the server back on from the Hetzner Cloud GUI. 24 | # 25 | # To run it from the Hetzner Cloud web terminal without typing it down, 26 | # you can either select it and then middle-click onto the web terminal, (that pastes 27 | # to it), or use `xdotool` (you have e.g. 3 seconds to focus the window): 28 | # 29 | # sleep 3 && xdotool type --delay 50 'curl YOUR_URL_HERE | sudo bash' 30 | # 31 | # (In the xdotool invocation you may have to replace chars so that 32 | # the right chars appear on the US-English keyboard.) 33 | # 34 | # If you do not replace the pubkey, you'll be running with my pubkey, but you can 35 | # change it afterwards by logging in via the Hetzner Cloud web terminal as `root` 36 | # with empty password. 37 | 38 | set -e 39 | 40 | # Hetzner Cloud OS images grow the root partition to the size of the local 41 | # disk on first boot. In case the NixOS live ISO is booted immediately on 42 | # first powerup, that does not happen. Thus we need to grow the partition 43 | # by deleting and re-creating it. 44 | sgdisk -d 1 /dev/sda 45 | sgdisk -N 1 /dev/sda 46 | partprobe /dev/sda 47 | 48 | mkfs.ext4 -F /dev/sda1 # wipes all data! 49 | 50 | mount /dev/sda1 /mnt 51 | 52 | nixos-generate-config --root /mnt 53 | 54 | # Delete trailing `}` from `configuration.nix` so that we can append more to it. 55 | sed -i -E 's:^\}\s*$::g' /mnt/etc/nixos/configuration.nix 56 | 57 | # Extend/override default `configuration.nix`: 58 | echo ' 59 | boot.loader.grub.devices = [ "/dev/sda" ]; 60 | 61 | # Initial empty root password for easy login: 62 | users.users.root.initialHashedPassword = ""; 63 | services.openssh.permitRootLogin = "prohibit-password"; 64 | 65 | services.openssh.enable = true; 66 | 67 | users.users.root.openssh.authorizedKeys.keys = [ 68 | # Replace this by your SSH pubkey! 69 | "ssh-rsa AAAAAAAAAAA..." 70 | ]; 71 | } 72 | ' >> /mnt/etc/nixos/configuration.nix 73 | 74 | nixos-install --no-root-passwd 75 | 76 | poweroff 77 | -------------------------------------------------------------------------------- /hosters/hetzner-dedicated/hetzner-dedicated-wipe-and-install-nixos.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Installs NixOS on a Hetzner server, wiping the server. 4 | # 5 | # This is for a specific server configuration; adjust where needed. 6 | # 7 | # Prerequisites: 8 | # * Update the script wherever FIXME is present 9 | # 10 | # Usage: 11 | # ssh root@YOUR_SERVERS_IP bash -s < hetzner-dedicated-wipe-and-install-nixos.sh 12 | # 13 | # When the script is done, make sure to boot the server from HD, not rescue mode again. 14 | 15 | # Explanations: 16 | # 17 | # * Adapted from https://gist.github.com/nh2/78d1c65e33806e7728622dbe748c2b6a 18 | # * Following largely https://nixos.org/nixos/manual/index.html#sec-installing-from-other-distro. 19 | # * **Important:** We boot in legacy-BIOS mode, not UEFI, because that's what Hetzner uses. 20 | # * NVMe devices aren't supported for booting (those require EFI boot) 21 | # * We set a custom `configuration.nix` so that we can connect to the machine afterwards, 22 | # inspired by https://nixos.wiki/wiki/Install_NixOS_on_Hetzner_Online 23 | # * This server has 2 HDDs. 24 | # We put everything on RAID1. 25 | # Storage scheme: `partitions -> RAID -> LVM -> ext4`. 26 | # * A root user with empty password is created, so that you can just login 27 | # as root and press enter when using the Hetzner spider KVM. 28 | # Of course that empty-password login isn't exposed to the Internet. 29 | # Change the password afterwards to avoid anyone with physical access 30 | # being able to login without any authentication. 31 | # * The script reboots at the end. 32 | 33 | set -eu 34 | set -o pipefail 35 | 36 | set -x 37 | 38 | # Inspect existing disks 39 | lsblk 40 | 41 | # Undo existing setups to allow running the script multiple times to iterate on it. 42 | # We allow these operations to fail for the case the script runs the first time. 43 | set +e 44 | umount /mnt 45 | vgchange -an 46 | set -e 47 | 48 | # Stop all mdadm arrays that the boot may have activated. 49 | mdadm --stop --scan 50 | 51 | # Prevent mdadm from auto-assembling arrays. 52 | # Otherwise, as soon as we create the partition tables below, it will try to 53 | # re-assemple a previous RAID if any remaining RAID signatures are present, 54 | # before we even get the chance to wipe them. 55 | # From: 56 | # https://unix.stackexchange.com/questions/166688/prevent-debian-from-auto-assembling-raid-at-boot/504035#504035 57 | # We use `>` because the file may already contain some detected RAID arrays, 58 | # which would take precedence over our ``. 59 | echo 'AUTO -all 60 | ARRAY UUID=00000000:00000000:00000000:00000000' > /etc/mdadm/mdadm.conf 61 | 62 | # Create wrapper for parted >= 3.3 that does not exit 1 when it cannot inform 63 | # the kernel of partitions changing (we use partprobe for that). 64 | echo -e "#! /usr/bin/env bash\nset -e\n" 'parted $@ 2> parted-stderr.txt || grep "unable to inform the kernel of the change" parted-stderr.txt && echo "This is expected, continuing" || echo >&2 "Parted failed; stderr: $(< parted-stderr.txt)"' > parted-ignoring-partprobe-error.sh && chmod +x parted-ignoring-partprobe-error.sh 65 | 66 | # Create partition tables (--script to not ask) 67 | ./parted-ignoring-partprobe-error.sh --script /dev/sda mklabel gpt 68 | ./parted-ignoring-partprobe-error.sh --script /dev/sdb mklabel gpt 69 | 70 | # Create partitions (--script to not ask) 71 | # 72 | # We create the 1MB BIOS boot partition at the front. 73 | # 74 | # Note we use "MB" instead of "MiB" because otherwise `--align optimal` has no effect; 75 | # as per documentation https://www.gnu.org/software/parted/manual/html_node/unit.html#unit: 76 | # > Note that as of parted-2.4, when you specify start and/or end values using IEC 77 | # > binary units like "MiB", "GiB", "TiB", etc., parted treats those values as exact 78 | # 79 | # Note: When using `mkpart` on GPT, as per 80 | # https://www.gnu.org/software/parted/manual/html_node/mkpart.html#mkpart 81 | # the first argument to `mkpart` is not a `part-type`, but the GPT partition name: 82 | # ... part-type is one of 'primary', 'extended' or 'logical', and may be specified only with 'msdos' or 'dvh' partition tables. 83 | # A name must be specified for a 'gpt' partition table. 84 | # GPT partition names are limited to 36 UTF-16 chars, see https://en.wikipedia.org/wiki/GUID_Partition_Table#Partition_entries_(LBA_2-33). 85 | ./parted-ignoring-partprobe-error.sh --script --align optimal /dev/sda -- mklabel gpt mkpart 'BIOS-boot-partition' 1MB 2MB set 1 bios_grub on mkpart 'data-partition' 2MB '100%' 86 | ./parted-ignoring-partprobe-error.sh --script --align optimal /dev/sdb -- mklabel gpt mkpart 'BIOS-boot-partition' 1MB 2MB set 1 bios_grub on mkpart 'data-partition' 2MB '100%' 87 | 88 | # Reload partitions 89 | partprobe 90 | 91 | # Wait for all devices to exist 92 | udevadm settle --timeout=5 --exit-if-exists=/dev/sda1 93 | udevadm settle --timeout=5 --exit-if-exists=/dev/sda2 94 | udevadm settle --timeout=5 --exit-if-exists=/dev/sdb1 95 | udevadm settle --timeout=5 --exit-if-exists=/dev/sdb2 96 | 97 | # Wipe any previous RAID signatures 98 | mdadm --zero-superblock --force /dev/sda2 99 | mdadm --zero-superblock --force /dev/sdb2 100 | 101 | # Create RAIDs 102 | # Note that during creating and boot-time assembly, mdadm cares about the 103 | # host name, and the existence and contents of `mdadm.conf`! 104 | # This also affects the names appearing in /dev/md/ being different 105 | # before and after reboot in general (but we take extra care here 106 | # to pass explicit names, and set HOMEHOST for the rebooting system further 107 | # down, so that the names appear the same). 108 | # Almost all details of this are explained in 109 | # https://bugzilla.redhat.com/show_bug.cgi?id=606481#c14 110 | # and the followup comments by Doug Ledford. 111 | mdadm --create --run --verbose /dev/md0 --level=1 --raid-devices=2 --homehost=hetzner --name=root0 /dev/sda2 /dev/sdb2 112 | 113 | # Assembling the RAID can result in auto-activation of previously-existing LVM 114 | # groups, preventing the RAID block device wiping below with 115 | # `Device or resource busy`. So disable all VGs first. 116 | vgchange -an 117 | 118 | # Wipe filesystem signatures that might be on the RAID from some 119 | # possibly existing older use of the disks (RAID creation does not do that). 120 | # See https://serverfault.com/questions/911370/why-does-mdadm-zero-superblock-preserve-file-system-information 121 | wipefs -a /dev/md0 122 | 123 | # Disable RAID recovery. We don't want this to slow down machine provisioning 124 | # in the rescue mode. It can run in normal operation after reboot. 125 | echo 0 > /proc/sys/dev/raid/speed_limit_max 126 | 127 | # LVM 128 | # PVs 129 | pvcreate /dev/md0 130 | # VGs 131 | vgcreate vg0 /dev/md0 132 | # LVs (--yes to automatically wipe detected file system signatures) 133 | lvcreate --yes --extents 95%FREE -n root0 vg0 # 5% slack space 134 | 135 | # Filesystems (-F to not ask on preexisting FS) 136 | mkfs.ext4 -F -L root /dev/mapper/vg0-root0 137 | 138 | # Creating file systems changes their UUIDs. 139 | # Trigger udev so that the entries in /dev/disk/by-uuid get refreshed. 140 | # `nixos-generate-config` depends on those being up-to-date. 141 | # See https://github.com/NixOS/nixpkgs/issues/62444 142 | udevadm trigger 143 | 144 | # Wait for FS labels to appear 145 | udevadm settle --timeout=5 --exit-if-exists=/dev/disk/by-label/root 146 | 147 | # NixOS pre-installation mounts 148 | 149 | # Mount target root partition 150 | mount /dev/disk/by-label/root /mnt 151 | 152 | # Installing nix 153 | 154 | # Installing nix requires `sudo`; the Hetzner rescue mode doesn't have it. 155 | apt-get install -y sudo 156 | 157 | # Allow installing nix as root, see 158 | # https://github.com/NixOS/nix/issues/936#issuecomment-475795730 159 | mkdir -p /etc/nix 160 | echo "build-users-group =" > /etc/nix/nix.conf 161 | 162 | curl -L https://nixos.org/nix/install | sh 163 | set +u +x # sourcing this may refer to unset variables that we have no control over 164 | . $HOME/.nix-profile/etc/profile.d/nix.sh 165 | set -u -x 166 | 167 | # FIXME Keep in sync with `system.stateVersion` set below! 168 | nix-channel --add https://nixos.org/channels/nixos-24.11 nixpkgs 169 | nix-channel --update 170 | 171 | # Getting NixOS installation tools. 172 | # In NixOS 24.11, `nixos-enter` is deprecated from `system.build` and should come from `pkgs`, 173 | # but to also support <= 24.05, we keep taking it from `system.build` for now.` 174 | nix-env -iE "_: with import { configuration = {}; }; (with config.system.build; [ nixos-generate-config nixos-install nixos-enter ]) ++ (with pkgs; [ ])" 175 | 176 | nixos-generate-config --root /mnt 177 | 178 | # Find the name of the network interface that connects us to the Internet. 179 | # Inspired by https://unix.stackexchange.com/questions/14961/how-to-find-out-which-interface-am-i-using-for-connecting-to-the-internet/302613#302613 180 | RESCUE_INTERFACE=$(ip route get 8.8.8.8 | grep -Po '(?<=dev )(\S+)') 181 | 182 | # Find what its name will be under NixOS, which uses stable interface names. 183 | # See https://major.io/2015/08/21/understanding-systemds-predictable-network-device-names/#comment-545626 184 | # NICs for most Hetzner servers are not onboard, which is why we use 185 | # `ID_NET_NAME_PATH`otherwise it would be `ID_NET_NAME_ONBOARD`. 186 | INTERFACE_DEVICE_PATH=$(udevadm info -e | grep -Po "(?<=^P: )(.*${RESCUE_INTERFACE})") 187 | UDEVADM_PROPERTIES_FOR_INTERFACE=$(udevadm info --query=property "--path=$INTERFACE_DEVICE_PATH") 188 | NIXOS_INTERFACE=$(echo "$UDEVADM_PROPERTIES_FOR_INTERFACE" | grep -o -E 'ID_NET_NAME_PATH=\w+' | cut -d= -f2) 189 | echo "Determined NIXOS_INTERFACE as '$NIXOS_INTERFACE'" 190 | 191 | IP_V4=$(ip route get 8.8.8.8 | grep -Po '(?<=src )(\S+)') 192 | echo "Determined IP_V4 as $IP_V4" 193 | 194 | # Determine Internet IPv6 by checking route, and using ::1 195 | # (because Hetzner rescue mode uses ::2 by default). 196 | # The `ip -6 route get` output on Hetzner looks like: 197 | # # ip -6 route get 2001:4860:4860:0:0:0:0:8888 198 | # 2001:4860:4860::8888 via fe80::1 dev eth0 src 2a01:4f8:151:62aa::2 metric 1024 pref medium 199 | IP_V6="$(ip route get 2001:4860:4860:0:0:0:0:8888 | head -1 | cut -d' ' -f7 | cut -d: -f1-4)::1" 200 | echo "Determined IP_V6 as $IP_V6" 201 | 202 | 203 | # From https://stackoverflow.com/questions/1204629/how-do-i-get-the-default-gateway-in-linux-given-the-destination/15973156#15973156 204 | read _ _ DEFAULT_GATEWAY _ < <(ip route list match 0/0); echo "$DEFAULT_GATEWAY" 205 | echo "Determined DEFAULT_GATEWAY as $DEFAULT_GATEWAY" 206 | 207 | 208 | # Generate `configuration.nix`. Note that we splice in shell variables. 209 | cat > /mnt/etc/nixos/configuration.nix <' (using the system hostname). 234 | # This results mdadm considering such disks as "foreign" as opposed to 235 | # "local", and showing them as e.g. '/dev/md/hetzner:root0' 236 | # instead of '/dev/md/root0'. 237 | # This is mdadm's protection against accidentally putting a RAID disk 238 | # into the wrong machine and corrupting data by accidental sync, see 239 | # https://bugzilla.redhat.com/show_bug.cgi?id=606481#c14 and onward. 240 | # We do not worry about plugging disks into the wrong machine because 241 | # we will never exchange disks between machines, so we tell mdadm to 242 | # ignore the homehost entirely. 243 | boot.swraid.mdadmConf = '' 244 | HOMEHOST 245 | ''; 246 | 247 | # Network (Hetzner uses static IP assignments, and we don't use DHCP here) 248 | networking.useDHCP = false; 249 | networking.interfaces."$NIXOS_INTERFACE".ipv4.addresses = [ 250 | { 251 | address = "$IP_V4"; 252 | # Hetzner requires /32, see: 253 | # https://docs.hetzner.com/robot/dedicated-server/network/net-config-debian-ubuntu/#ipv4. 254 | # NixOS automatically sets up a route to the gateway 255 | # (but only because we set "networking.defaultGateway.interface" below), see 256 | # https://github.com/NixOS/nixops/pull/1032#issuecomment-2763497444 257 | prefixLength = 32; 258 | } 259 | ]; 260 | networking.interfaces."$NIXOS_INTERFACE".ipv6.addresses = [ 261 | { 262 | address = "$IP_V6"; 263 | prefixLength = 64; 264 | } 265 | ]; 266 | networking.defaultGateway = { 267 | address = "$DEFAULT_GATEWAY"; 268 | # Interface must be given for Hetzner networking to work, see comment above. 269 | interface = "$NIXOS_INTERFACE"; 270 | }; 271 | networking.defaultGateway6 = { address = "fe80::1"; interface = "$NIXOS_INTERFACE"; }; 272 | networking.nameservers = [ "8.8.8.8" ]; 273 | 274 | # Initial empty root password for easy login: 275 | users.users.root.initialHashedPassword = ""; 276 | services.openssh.settings = { 277 | PermitRootLogin = "prohibit-password"; 278 | }; 279 | 280 | users.users.root.openssh.authorizedKeys.keys = [ 281 | # FIXME Replace this by your SSH pubkey! 282 | "ssh-rsa AAAAAAAAAAA..." 283 | ]; 284 | 285 | services.openssh.enable = true; 286 | 287 | # FIXME 288 | # This value determines the NixOS release with which your system is to be 289 | # compatible, in order to avoid breaking some software such as database 290 | # servers. You should change this only after NixOS release notes say you 291 | # should. 292 | system.stateVersion = "24.11"; # Did you read the comment? 293 | 294 | } 295 | EOF 296 | 297 | # Install NixOS 298 | PATH="$PATH" `which nixos-install` --no-root-passwd --root /mnt --max-jobs 40 299 | 300 | umount /mnt 301 | 302 | reboot 303 | -------------------------------------------------------------------------------- /hosters/hetzner-dedicated/zfs-uefi-nvme-nixos.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Installs NixOS on a Hetzner server, wiping the server. 4 | # 5 | # This is for a specific server configuration; adjust where needed. 6 | # 7 | # Prerequisites: 8 | # * Update the script wherever FIXME is present 9 | # 10 | # Usage: 11 | # ssh root@YOUR_SERVERS_IP bash -s < hetzner-dedicated-wipe-and-install-nixos.sh 12 | # 13 | # When the script is done, make sure to boot the server from HD, not rescue mode again. 14 | 15 | # Explanations: 16 | # 17 | # * Following largely https://nixos.org/nixos/manual/index.html#sec-installing-from-other-distro. 18 | # * and https://nixos.wiki/wiki/NixOS_on_ZFS 19 | # * **Important:** First you need to boot in legacy-BIOS mode. Then ask for 20 | # hetzner support to enable UEFI for you. 21 | # * We set a custom `configuration.nix` so that we can connect to the machine afterwards, 22 | # inspired by https://nixos.wiki/wiki/Install_NixOS_on_Hetzner_Online 23 | # * This server has 2 SSDs. 24 | # We put everything on mirror (RAID1 equivalent). 25 | # * A root user with empty password is created, so that you can just login 26 | # as root and press enter when using the Hetzner spider KVM. 27 | # Of course that empty-password login isn't exposed to the Internet. 28 | # Change the password afterwards to avoid anyone with physical access 29 | # being able to login without any authentication. 30 | # * The script reboots at the end. 31 | # * exports of env vars are added throughout the script in case you want to run it manually 32 | export LC_ALL=C 33 | 34 | # WARNING: on 2023/07/16 the rescue system of hetzner boots with kernel 6.3.7 which 35 | # is by default not supported by the latest debian package. You need to update to debian 36 | # unstable to proceed with the zfs installation. 37 | 38 | set -euox pipefail 39 | 40 | # Install zfs via Hetzner's install helper. 41 | # The helper impersonates the `zfs` and `zpool` commands in the Rescue system. 42 | # It asks for confirmation; provide it with `y`. 43 | if grep -q -i hetzner /usr/local/sbin/zfs "$(which zfs)" ]; then 44 | echo "Installing ZFS from Hetzner Rescue system"; 45 | echo y | zfs 46 | fi 47 | 48 | # Inspect existing disks 49 | # Should give you something like 50 | # NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT 51 | # nvme0n1 259:0 0 476.9G 0 disk 52 | # ├─nvme0n1p1 259:2 0 32G 0 part 53 | # │ └─md0 9:0 0 32G 0 raid1 [SWAP] 54 | # ├─nvme0n1p2 259:3 0 512M 0 part 55 | # │ └─md1 9:1 0 511M 0 raid1 /boot 56 | # └─nvme0n1p3 259:4 0 444.4G 0 part 57 | # └─md2 9:2 0 444.3G 0 raid1 / 58 | # nvme1n1 259:1 0 476.9G 0 disk 59 | # ├─nvme1n1p1 259:5 0 32G 0 part 60 | # │ └─md0 9:0 0 32G 0 raid1 [SWAP] 61 | # ├─nvme1n1p2 259:6 0 512M 0 part 62 | # │ └─md1 9:1 0 511M 0 raid1 /boot 63 | # └─nvme1n1p3 259:7 0 444.4G 0 part 64 | # └─md2 9:2 0 444.3G 0 raid1 / 65 | lsblk 66 | 67 | # check the disks that you have available 68 | # you have to use disks by ID with zfs 69 | # see https://openzfs.github.io/openzfs-docs/Getting%20Started/Ubuntu/Ubuntu%2020.04%20Root%20on%20ZFS.html#step-2-disk-formatting 70 | ls -1 /dev/disk/by-id 71 | # should give you something like this 72 | # md-name-rescue:0 73 | # md-name-rescue:1 74 | # md-name-rescue:2 75 | # md-uuid-15391820:32e070f6:ecbfb99e:e983e018 76 | # md-uuid-48379d14:3c44fe11:e6528eec:ad784ade 77 | # md-uuid-f2a894fc:9e90e3af:9af81d28:b120ae1f 78 | # nvme-eui.0025388a01051b55 79 | # nvme-eui.0025388a01051b55-part1 80 | # nvme-eui.0025388a01051b55-part2 81 | # nvme-eui.0025388a01051b55-part3 82 | # nvme-eui.0025388a01051b58 83 | # nvme-eui.0025388a01051b58-part1 84 | # nvme-eui.0025388a01051b58-part2 85 | # nvme-eui.0025388a01051b58-part3 86 | # nvme-SAMSUNG_MZVLB512HBJQ-00000_S4GENA0NA00424 87 | # nvme-SAMSUNG_MZVLB512HBJQ-00000_S4GENA0NA00424-part1 88 | # nvme-SAMSUNG_MZVLB512HBJQ-00000_S4GENA0NA00424-part2 89 | # nvme-SAMSUNG_MZVLB512HBJQ-00000_S4GENA0NA00424-part3 90 | # nvme-SAMSUNG_MZVLB512HBJQ-00000_S4GENA0NA00427 91 | # nvme-SAMSUNG_MZVLB512HBJQ-00000_S4GENA0NA00427-part1 92 | # nvme-SAMSUNG_MZVLB512HBJQ-00000_S4GENA0NA00427-part2 93 | # nvme-SAMSUNG_MZVLB512HBJQ-00000_S4GENA0NA00427-part3 94 | # 95 | # we will use the two disks 96 | # nvme-SAMSUNG_MZVLB512HBJQ-00000_S4GENA0NA00424 97 | # nvme-SAMSUNG_MZVLB512HBJQ-00000_S4GENA0NA00427 98 | 99 | # FIXME The following variables should be replaced 100 | export DISK1=/dev/disk/by-id/nvme-SAMSUNG_MZQLB3T8HALS-00007_S438NC0R804840 101 | export DISK2=/dev/disk/by-id/nvme-SAMSUNG_MZQLB3T8HALS-00007_S438NC0R811800 102 | # FIXME Replace this by your SSH pubkey! 103 | export SSH_PUB_KEY="AAAAAAAAAAA..." 104 | # choose whatever you want, it doesn't matter 105 | export MY_HOSTNAME=htz 106 | # this has to be a number in this format exactly. You can replace the numbers though 107 | export MY_HOSTID=00000001 108 | 109 | # Undo existing setups to allow running the script multiple times to iterate on it. 110 | # We allow these operations to fail for the case the script runs the first time. 111 | umount /mnt || true 112 | vgchange -an || true 113 | 114 | # Stop all mdadm arrays that the boot may have activated. 115 | mdadm --stop --scan 116 | 117 | # Prevent mdadm from auto-assembling arrays. 118 | # Otherwise, as soon as we create the partition tables below, it will try to 119 | # re-assemple a previous RAID if any remaining RAID signatures are present, 120 | # before we even get the chance to wipe them. 121 | # From: 122 | # https://unix.stackexchange.com/questions/166688/prevent-debian-from-auto-assembling-raid-at-boot/504035#504035 123 | # We use `>` because the file may already contain some detected RAID arrays, 124 | # which would take precedence over our ``. 125 | echo 'AUTO -all 126 | ARRAY UUID=00000000:00000000:00000000:00000000' > /etc/mdadm/mdadm.conf 127 | 128 | # Create wrapper for parted >= 3.3 that does not exit 1 when it cannot inform 129 | # the kernel of partitions changing (we use partprobe for that). 130 | echo -e "#! /usr/bin/env bash\nset -e\n" 'parted $@ 2> parted-stderr.txt || grep "unable to inform the kernel of the change" parted-stderr.txt && echo "This is expected, continuing" || echo >&2 "Parted failed; stderr: $(< parted-stderr.txt)"' > parted-ignoring-partprobe-error.sh && chmod +x parted-ignoring-partprobe-error.sh 131 | 132 | # Create partition tables (--script to not ask) 133 | ./parted-ignoring-partprobe-error.sh --script $DISK1 mklabel gpt 134 | ./parted-ignoring-partprobe-error.sh --script $DISK2 mklabel gpt 135 | 136 | # Create partitions (--script to not ask) 137 | # 138 | # We create the 1MB BIOS boot partition at the front. 139 | # 140 | # Note we use "MB" instead of "MiB" because otherwise `--align optimal` has no effect; 141 | # as per documentation https://www.gnu.org/software/parted/manual/html_node/unit.html#unit: 142 | # > Note that as of parted-2.4, when you specify start and/or end values using IEC 143 | # > binary units like "MiB", "GiB", "TiB", etc., parted treats those values as exact 144 | # 145 | # Note: When using `mkpart` on GPT, as per 146 | # https://www.gnu.org/software/parted/manual/html_node/mkpart.html#mkpart 147 | # the first argument to `mkpart` is not a `part-type`, but the GPT partition name: 148 | # ... part-type is one of 'primary', 'extended' or 'logical', and may be specified only with 'msdos' or 'dvh' partition tables. 149 | # A name must be specified for a 'gpt' partition table. 150 | # GPT partition names are limited to 36 UTF-16 chars, see https://en.wikipedia.org/wiki/GUID_Partition_Table#Partition_entries_(LBA_2-33). 151 | # TODO the bios partition should not be this big 152 | # however if it's less the installation fails with 153 | # cannot copy /nix/store/d4xbrrailkn179cdp90v4m57mqd73hvh-linux-5.4.100/bzImage to /boot/kernels/d4xbrrailkn179cdp90v4m57mqd73hvh-linux-5.4.100-bzImage.tmp: No space left on device 154 | ./parted-ignoring-partprobe-error.sh --script --align optimal $DISK1 -- mklabel gpt \ 155 | mkpart 'BIOS-boot-partition' 1MB 2MB set 1 bios_grub on \ 156 | mkpart 'EFI-system-partition' 2MB 512MB set 2 esp on \ 157 | mkpart 'data-partition' 512MB '100%' 158 | 159 | ./parted-ignoring-partprobe-error.sh --script --align optimal $DISK2 -- mklabel gpt \ 160 | mkpart 'BIOS-boot-partition' 1MB 2MB set 1 bios_grub on \ 161 | mkpart 'EFI-system-partition' 2MB 512MB set 2 esp on \ 162 | mkpart 'data-partition' 512MB '100%' 163 | 164 | # Reload partitions 165 | partprobe 166 | 167 | # Wait for all devices to exist 168 | udevadm settle --timeout=5 --exit-if-exists=$DISK1-part1 169 | udevadm settle --timeout=5 --exit-if-exists=$DISK1-part2 170 | udevadm settle --timeout=5 --exit-if-exists=$DISK1-part3 171 | udevadm settle --timeout=5 --exit-if-exists=$DISK2-part1 172 | udevadm settle --timeout=5 --exit-if-exists=$DISK2-part2 173 | udevadm settle --timeout=5 --exit-if-exists=$DISK2-part3 174 | 175 | # Wipe any previous RAID signatures 176 | # sometimes they are not on a specific disk for some reason 177 | mdadm --zero-superblock --force $DISK1-part1 || true 178 | mdadm --zero-superblock --force $DISK1-part2 || true 179 | mdadm --zero-superblock --force $DISK1-part3 || true 180 | mdadm --zero-superblock --force $DISK2-part1 || true 181 | mdadm --zero-superblock --force $DISK2-part2 || true 182 | mdadm --zero-superblock --force $DISK2-part3 || true 183 | 184 | # Creating file systems changes their UUIDs. 185 | # Trigger udev so that the entries in /dev/disk/by-uuid get refreshed. 186 | # `nixos-generate-config` depends on those being up-to-date. 187 | # See https://github.com/NixOS/nixpkgs/issues/62444 188 | udevadm trigger 189 | 190 | # taken from https://nixos.wiki/wiki/NixOS_on_ZFS 191 | # somehow there is a weird symlink in the default zfs 192 | # 193 | # `-o compatibility=grub2` is needed because this 194 | # script makes GRUB boot from root-on-ZFS (there's 195 | # no separate non-ZFS `/boot` that contains kernels 196 | # and initrds). 197 | # Without it,`nixos-install` fails with 198 | # grub-install: error: unknown filesystem 199 | # since GRUB's installer iteslf already tries to read the ZFS 200 | # filesystem, and that fails if ZFS uses features that 201 | # GRUB's ZFS support does not understand. 202 | # See https://discourse.nixos.org/t/failing-to-successfully-install-grub-on-zfs-please-help/55387/9 203 | # TODO: It would be better to not do it this way, 204 | # and instead install kernels/initrd outside 205 | # of ZFS because GRUB does not do error 206 | # recovery using mirrors, see 207 | # https://discourse.nixos.org/t/disko-partition-setup-on-uefi-system-which-has-fallback-boot-partition/51968/2?u=nh2 208 | zpool create -O mountpoint=none \ 209 | -O atime=off \ 210 | -O compression=lz4 \ 211 | -O xattr=sa \ 212 | -O acltype=posixacl \ 213 | -o ashift=12 \ 214 | -o compatibility=grub2 \ 215 | -f \ 216 | root_pool mirror $DISK1-part3 $DISK2-part3 217 | 218 | # Create the filesystems. This layout is designed so that /home is separate from the root 219 | # filesystem, as you'll likely want to snapshot it differently for backup purposes. It also 220 | # makes a "nixos" filesystem underneath the root, to support installing multiple OSes if 221 | # that's something you choose to do in future. 222 | zfs create -o mountpoint=legacy root_pool/root 223 | zfs create -o mountpoint=legacy root_pool/root/nixos 224 | zfs create -o mountpoint=legacy root_pool/home 225 | # add 1G of reseved space in case the disk gets full 226 | # zfs needs space to delete files 227 | zfs create -o refreservation=1G -o mountpoint=none root_pool/reserved 228 | # this creates a special volume for db data see https://wiki.archlinux.org/index.php/ZFS#Databases 229 | zfs create -o mountpoint=legacy \ 230 | -o recordsize=8K \ 231 | -o primarycache=metadata \ 232 | -o logbias=throughput \ 233 | root_pool/postgres 234 | 235 | # NixOS pre-installation mounts 236 | # 237 | # Mount the filesystems manually. The nixos installer will detect these mountpoints 238 | # and save them to /mnt/nixos/hardware-configuration.nix during the install process. 239 | mount -t zfs root_pool/root/nixos /mnt 240 | mkdir /mnt/home 241 | mount -t zfs root_pool/home /mnt/home 242 | mkdir -p /mnt/var/lib/postgres 243 | mount -t zfs root_pool/postgres /mnt/var/lib/postgres 244 | 245 | # Create a raid mirror for the efi boot 246 | # see https://docs.hetzner.com/robot/dedicated-server/operating-systems/efi-system-partition/ 247 | # TODO check this though the following article says it doesn't work properly 248 | # https://outflux.net/blog/archives/2018/04/19/uefi-booting-and-raid1/ 249 | mdadm --create --run --verbose /dev/md127 \ 250 | --level 1 \ 251 | --raid-disks 2 \ 252 | --metadata 1.0 \ 253 | --homehost=$MY_HOSTNAME \ 254 | --name=boot_efi \ 255 | $DISK1-part2 $DISK2-part2 256 | 257 | # Assembling the RAID can result in auto-activation of previously-existing LVM 258 | # groups, preventing the RAID block device wiping below with 259 | # `Device or resource busy`. So disable all VGs first. 260 | vgchange -an 261 | 262 | # Wipe filesystem signatures that might be on the RAID from some 263 | # possibly existing older use of the disks (RAID creation does not do that). 264 | # See https://serverfault.com/questions/911370/why-does-mdadm-zero-superblock-preserve-file-system-information 265 | wipefs -a /dev/md127 266 | 267 | # Disable RAID recovery. We don't want this to slow down machine provisioning 268 | # in the rescue mode. It can run in normal operation after reboot. 269 | echo 0 > /proc/sys/dev/raid/speed_limit_max 270 | 271 | # Filesystems (-F to not ask on preexisting FS) 272 | mkfs.vfat -F 32 /dev/md127 273 | 274 | # Creating file systems changes their UUIDs. 275 | # Trigger udev so that the entries in /dev/disk/by-uuid get refreshed. 276 | # `nixos-generate-config` depends on those being up-to-date. 277 | # See https://github.com/NixOS/nixpkgs/issues/62444 278 | udevadm trigger 279 | 280 | mkdir -p /mnt/boot/efi 281 | mount /dev/md127 /mnt/boot/efi 282 | 283 | # Installing nix 284 | 285 | # Installing nix requires `sudo`; the Hetzner rescue mode doesn't have it. 286 | apt-get install -y sudo 287 | 288 | # Allow installing nix as root, see 289 | # https://github.com/NixOS/nix/issues/936#issuecomment-475795730 290 | mkdir -p /etc/nix 291 | echo "build-users-group =" > /etc/nix/nix.conf 292 | 293 | # Allow installing nix as root, see 294 | # https://github.com/NixOS/nix/issues/936#issuecomment-475795730 295 | mkdir -p /etc/nix 296 | echo "build-users-group =" > /etc/nix/nix.conf 297 | 298 | curl -L https://nixos.org/nix/install | sh 299 | set +u +x # sourcing this may refer to unset variables that we have no control over 300 | . $HOME/.nix-profile/etc/profile.d/nix.sh 301 | set -u -x 302 | 303 | # FIXME Keep in sync with `system.stateVersion` set below! 304 | nix-channel --add https://nixos.org/channels/nixos-24.11 nixpkgs 305 | nix-channel --update 306 | 307 | # TODO use something like nix shell nixpkgs#nixos-generate-config nixpkgs#nixos-install nixpkgs#nixos-enter nixpkgs#manual.manpages 308 | # Getting NixOS installation tools. 309 | # In NixOS 24.11, `nixos-enter` is deprecated from `system.build` and should come from `pkgs`, 310 | # but to also support <= 24.05, we keep taking it from `system.build` for now.` 311 | nix-env -iE "_: with import { configuration = {}; }; (with config.system.build; [ nixos-generate-config nixos-install nixos-enter ]) ++ (with pkgs; [ ])" 312 | 313 | # TODO 314 | # perl: warning: Please check that your locale settings: 315 | # LANGUAGE = (unset), 316 | # LC_ALL = "en_US.UTF-8", 317 | # LANG = "en_US.UTF-8" 318 | # are supported and installed on your system. 319 | nixos-generate-config --root /mnt 320 | 321 | # Find the name of the network interface that connects us to the Internet. 322 | # Inspired by https://unix.stackexchange.com/questions/14961/how-to-find-out-which-interface-am-i-using-for-connecting-to-the-internet/302613#302613 323 | export RESCUE_INTERFACE=$(ip route get 8.8.8.8 | grep -Po '(?<=dev )(\S+)') 324 | # Find what its name will be under NixOS, which uses stable interface names. 325 | # See https://major.io/2015/08/21/understanding-systemds-predictable-network-device-names/#comment-545626 326 | # NICs for most Hetzner servers are not onboard, which is why we use 327 | # `ID_NET_NAME_PATH`otherwise it would be `ID_NET_NAME_ONBOARD`. 328 | export INTERFACE_DEVICE_PATH=$(udevadm info -e | grep -Po "(?<=^P: )(.*${RESCUE_INTERFACE})") 329 | export UDEVADM_PROPERTIES_FOR_INTERFACE=$(udevadm info --query=property "--path=$INTERFACE_DEVICE_PATH") 330 | export NIXOS_INTERFACE=$(echo "$UDEVADM_PROPERTIES_FOR_INTERFACE" | grep -o -E 'ID_NET_NAME_PATH=\w+' | cut -d= -f2) 331 | echo "Determined NIXOS_INTERFACE as '$NIXOS_INTERFACE'" 332 | export IP_V4=$(ip route get 8.8.8.8 | grep -Po '(?<=src )(\S+)') 333 | echo "Determined IP_V4 as $IP_V4" 334 | # Find what its name will be under NixOS, which uses stable interface names. 335 | # See https://major.io/2015/08/21/understanding-systemds-predictable-network-device-names/#comment-545626 336 | # NICs for most Hetzner servers are not onboard, which is why we use 337 | # `ID_NET_NAME_PATH`otherwise it would be `ID_NET_NAME_ONBOARD`. 338 | export INTERFACE_DEVICE_PATH=$(udevadm info -e | grep -Po "(?<=^P: )(.*${RESCUE_INTERFACE})") 339 | export UDEVADM_PROPERTIES_FOR_INTERFACE=$(udevadm info --query=property "--path=$INTERFACE_DEVICE_PATH") 340 | export NIXOS_INTERFACE=$(echo "$UDEVADM_PROPERTIES_FOR_INTERFACE" | grep -o -E 'ID_NET_NAME_PATH=\w+' | cut -d= -f2) 341 | echo "Determined NIXOS_INTERFACE as '$NIXOS_INTERFACE'" 342 | 343 | # Determine Internet IPv6 by checking route, and using ::1 344 | # (because Hetzner rescue mode uses ::2 by default). 345 | # The `ip -6 route get` output on Hetzner looks like: 346 | # # ip -6 route get 2001:4860:4860:0:0:0:0:8888 347 | # 2001:4860:4860::8888 via fe80::1 dev eth0 src 2a01:4f8:151:62aa::2 metric 1024 pref medium 348 | export IP_V6="$(ip route get 2001:4860:4860::8888 | head -1 | cut -d' ' -f7 | cut -d: -f1-4)::1" 349 | echo "Determined IP_V6 as $IP_V6" 350 | 351 | # From https://stackoverflow.com/questions/1204629/how-do-i-get-the-default-gateway-in-linux-given-the-destination/15973156#15973156 352 | read _ _ DEFAULT_GATEWAY _ < <(ip route list match 0/0); echo "$DEFAULT_GATEWAY" 353 | echo "Determined DEFAULT_GATEWAY as $DEFAULT_GATEWAY" 354 | 355 | # Generate `configuration.nix`. Note that we splice in shell variables. 356 | cat > /mnt/etc/nixos/configuration.nix < RAID -> LVM -> ext4`. 28 | # * A root user with empty password is created, so that you can just login 29 | # as root and press enter when using a KVM. 30 | # Of course that empty-password login isn't exposed to the Internet. 31 | # Change the password afterwards to avoid anyone with physical access 32 | # being able to login without any authentication. 33 | # * The script reboots at the end. 34 | 35 | set -eu 36 | set -o pipefail 37 | 38 | set -x 39 | 40 | # Inspect existing disks 41 | lsblk 42 | 43 | # Undo existing setups to allow running the script multiple times to iterate on it. 44 | # We allow these operations to fail for the case the script runs the first time. 45 | set +e 46 | umount /mnt 47 | vgchange -an 48 | set -e 49 | 50 | # Stop all mdadm arrays that the boot may have activated. 51 | mdadm --stop --scan 52 | 53 | # Prevent mdadm from auto-assembling arrays. 54 | # Otherwise, as soon as we create the partition tables below, it will try to 55 | # re-assemple a previous RAID if any remaining RAID signatures are present, 56 | # before we even get the chance to wipe them. 57 | # From: 58 | # https://unix.stackexchange.com/questions/166688/prevent-debian-from-auto-assembling-raid-at-boot/504035#504035 59 | # We use `>` because the file may already contain some detected RAID arrays, 60 | # which would take precedence over our ``. 61 | echo 'AUTO -all 62 | ARRAY UUID=00000000:00000000:00000000:00000000' > /etc/mdadm/mdadm.conf 63 | 64 | # Create wrapper for parted >= 3.3 that does not exit 1 when it cannot inform 65 | # the kernel of partitions changing (we use partprobe for that). 66 | echo -e "#! /usr/bin/env bash\nset -e\n" 'parted $@ 2> parted-stderr.txt || grep "unable to inform the kernel of the change" parted-stderr.txt && echo "This is expected, continuing" || echo >&2 "Parted failed; stderr: $(< parted-stderr.txt)"' > parted-ignoring-partprobe-error.sh && chmod +x parted-ignoring-partprobe-error.sh 67 | 68 | # Create partition tables (--script to not ask) 69 | ./parted-ignoring-partprobe-error.sh /dev/sda mklabel gpt 70 | ./parted-ignoring-partprobe-error.sh /dev/sdb mklabel gpt 71 | 72 | # Create partitions (--script to not ask) 73 | # 74 | # We create the 1MB BIOS boot partition at the front. 75 | # 76 | # Note we use "MB" instead of "MiB" because otherwise `--align optimal` has no effect; 77 | # as per documentation https://www.gnu.org/software/parted/manual/html_node/unit.html#unit: 78 | # > Note that as of parted-2.4, when you specify start and/or end values using IEC 79 | # > binary units like "MiB", "GiB", "TiB", etc., parted treats those values as exact 80 | # 81 | # Note: When using `mkpart` on GPT, as per 82 | # https://www.gnu.org/software/parted/manual/html_node/mkpart.html#mkpart 83 | # the first argument to `mkpart` is not a `part-type`, but the GPT partition name: 84 | # ... part-type is one of 'primary', 'extended' or 'logical', and may be specified only with 'msdos' or 'dvh' partition tables. 85 | # A name must be specified for a 'gpt' partition table. 86 | # GPT partition names are limited to 36 UTF-16 chars, see https://en.wikipedia.org/wiki/GUID_Partition_Table#Partition_entries_(LBA_2-33). 87 | ./parted-ignoring-partprobe-error.sh --align optimal /dev/sda -- mklabel gpt mkpart 'BIOS-boot-partition' 1MB 2MB set 1 bios_grub on mkpart 'data-partition' 2MB '100%' 88 | ./parted-ignoring-partprobe-error.sh --align optimal /dev/sdb -- mklabel gpt mkpart 'BIOS-boot-partition' 1MB 2MB set 1 bios_grub on mkpart 'data-partition' 2MB '100%' 89 | 90 | # Relaod partitions 91 | partprobe 92 | 93 | # Wait for all devices to exist 94 | udevadm settle --timeout=5 --exit-if-exists=/dev/sda1 95 | udevadm settle --timeout=5 --exit-if-exists=/dev/sda2 96 | udevadm settle --timeout=5 --exit-if-exists=/dev/sdb1 97 | udevadm settle --timeout=5 --exit-if-exists=/dev/sdb2 98 | 99 | # Wipe any previous RAID signatures 100 | mdadm --zero-superblock --force /dev/sda2 101 | mdadm --zero-superblock --force /dev/sdb2 102 | 103 | # Create RAIDs 104 | # Note that during creating and boot-time assembly, mdadm cares about the 105 | # host name, and the existence and contents of `mdadm.conf`! 106 | # This also affects the names appearing in /dev/md/ being different 107 | # before and after reboot in general (but we take extra care here 108 | # to pass explicit names, and set HOMEHOST for the rebooting system further 109 | # down, so that the names appear the same). 110 | # Almost all details of this are explained in 111 | # https://bugzilla.redhat.com/show_bug.cgi?id=606481#c14 112 | # and the followup comments by Doug Ledford. 113 | mdadm --create --run --verbose /dev/md0 --level=1 --raid-devices=2 --homehost=leaseweb --name=root0 /dev/sda2 /dev/sdb2 114 | 115 | # Assembling the RAID can result in auto-activation of previously-existing LVM 116 | # groups, preventing the RAID block device wiping below with 117 | # `Device or resource busy`. So disable all VGs first. 118 | vgchange -an 119 | 120 | # Wipe filesystem signatures that might be on the RAID from some 121 | # possibly existing older use of the disks (RAID creation does not do that). 122 | # See https://serverfault.com/questions/911370/why-does-mdadm-zero-superblock-preserve-file-system-information 123 | wipefs -a /dev/md0 124 | 125 | # Disable RAID recovery. We don't want this to slow down machine provisioning 126 | # in the rescue mode. It can run in normal operation after reboot. 127 | echo 0 > /proc/sys/dev/raid/speed_limit_max 128 | 129 | # LVM 130 | # PVs 131 | pvcreate /dev/md0 132 | # VGs 133 | vgcreate vg0 /dev/md0 134 | # LVs (--yes to automatically wipe detected file system signatures) 135 | lvcreate --yes --extents 95%FREE -n root0 vg0 # 5% slack space 136 | 137 | # Filesystems (-F to not ask on preexisting FS) 138 | mkfs.ext4 -F -L root /dev/mapper/vg0-root0 139 | 140 | # Creating file systems changes their UUIDs. 141 | # Trigger udev so that the entries in /dev/disk/by-uuid get refreshed. 142 | # `nixos-generate-config` depends on those being up-to-date. 143 | # See https://github.com/NixOS/nixpkgs/issues/62444 144 | udevadm trigger 145 | 146 | # Wait for FS labels to appear 147 | udevadm settle --timeout=5 --exit-if-exists=/dev/disk/by-label/root 148 | 149 | # NixOS pre-installation mounts 150 | 151 | # Mount target root partition 152 | mount /dev/disk/by-label/root /mnt 153 | 154 | # Installing nix 155 | 156 | # Allow installing nix as root, see 157 | # https://github.com/NixOS/nix/issues/936#issuecomment-475795730 158 | mkdir -p /etc/nix 159 | echo "build-users-group =" > /etc/nix/nix.conf 160 | 161 | curl -L https://nixos.org/nix/install | sh 162 | set +u +x # sourcing this may refer to unset variables that we have no control over 163 | . $HOME/.nix-profile/etc/profile.d/nix.sh 164 | set -u -x 165 | 166 | # Keep in sync with `system.stateVersion` set below! 167 | # nix-channel --add https://nixos.org/channels/nixos-20.03 nixpkgs 168 | nix-channel --add https://nixos.org/channels/nixos-20.03 nixpkgs 169 | nix-channel --update 170 | 171 | # Getting NixOS installation tools 172 | nix-env -iE "_: with import { configuration = {}; }; with config.system.build; [ nixos-generate-config nixos-install nixos-enter manual.manpages ]" 173 | 174 | nixos-generate-config --root /mnt 175 | 176 | # Find the name of the network interface that connects us to the Internet. 177 | # Inspired by https://unix.stackexchange.com/questions/14961/how-to-find-out-which-interface-am-i-using-for-connecting-to-the-internet/302613#302613 178 | RESCUE_INTERFACE=$(ip route get 8.8.8.8 | grep -Po '(?<=dev )(\S+)') 179 | 180 | # Find what its name will be under NixOS, which uses stable interface names. 181 | # See https://major.io/2015/08/21/understanding-systemds-predictable-network-device-names/#comment-545626 182 | # 183 | # IMPORTANT: 184 | # There is a known complication in that Linux somewhere between 4.19 and 5.4.27 185 | # switched from classifying only 1 of the 2 network interfaces of the server as 186 | # "onboard" to classifying both as "onboard", thus "enp2s0" shows up as "eno0" 187 | # instead in newer kernels. 188 | # See: 189 | # https://gist.github.com/nh2/71854c40a1a1a7c15bc8a8105e854f88#file-analysis-md 190 | # So once the Leaseweb GRML rescue mode upgrades to a newer kernel, the value of 191 | # `NIXOS_INTERFACE` should be successfully found from `RESCUE_INTERFACE` using 192 | # the `ID_NET_NAME_ONBOARD` grep below; but until then (when the grep is empty) 193 | # we have to detect this situation, turning `enp2s0` into `eno0` ourselves, 194 | # because we want to boot a NixOS that uses the new kernel (>= 5.4.27) of which 195 | # we know that it will detect the card as "onboard" and thus call it "eno". 196 | INTERFACE_DEVICE_PATH=$(udevadm info -e | grep -Po "(?<=^P: )(.*${RESCUE_INTERFACE})") 197 | UDEVADM_PROPERTIES_FOR_INTERFACE=$(udevadm info --query=property "--path=$INTERFACE_DEVICE_PATH") 198 | set +o pipefail # allow the grep to fail, see comment above 199 | NIXOS_INTERFACE=$(echo "$UDEVADM_PROPERTIES_FOR_INTERFACE" | grep -o -E 'ID_NET_NAME_ONBOARD=\w+' | cut -d= -f2) 200 | set -o pipefail 201 | # The following `if` logic can be deleted once versions < 20.03 are no longer relevant. 202 | if [ -z "$NIXOS_INTERFACE" ]; then 203 | echo "Could not determine NIXOS_INTERFACE from udevadm, RESCUE_INTERFACE is '$RESCUE_INTERFACE'" 204 | # Set this to 1 iff you are installing a newer kernel as described in the comment above: 205 | INSTALLING_NEWER_KERNEL=1 206 | if [ "$INSTALLING_NEWER_KERNEL" == "1" ]; then 207 | echo "INSTALLING_NEWER_KERNEL=1 is active, setting NIXOS_INTERFACE=eno0" 208 | NIXOS_INTERFACE="eno0" 209 | else 210 | echo "INSTALLING_NEWER_KERNEL=1 is NOT active, setting NIXOS_INTERFACE=$RESCUE_INTERFACE" 211 | NIXOS_INTERFACE="$RESCUE_INTERFACE" 212 | fi 213 | else 214 | echo "Determined NIXOS_INTERFACE as '$NIXOS_INTERFACE'" 215 | fi 216 | 217 | IP_V4=$(ip route get 8.8.8.8 | grep -Po '(?<=src )(\S+)') 218 | echo "Determined IP_V4 as $IP_V4" 219 | 220 | # From https://stackoverflow.com/questions/1204629/how-do-i-get-the-default-gateway-in-linux-given-the-destination/15973156#15973156 221 | read _ _ DEFAULT_GATEWAY _ < <(ip route list match 0/0); echo "$DEFAULT_GATEWAY" 222 | echo "Determined DEFAULT_GATEWAY as $DEFAULT_GATEWAY" 223 | 224 | # The Leaseweb GRML Rescue mode as of writing has no IPv6 connectivity, 225 | # so we cannot get the IPv6 address here. 226 | 227 | 228 | # Generate `configuration.nix`. Note that we splice in shell variables. 229 | cat > /mnt/etc/nixos/configuration.nix <' (using the system hostname). 304 | # This results mdadm considering such disks as "foreign" as opposed to 305 | # "local", and showing them as e.g. '/dev/md/leaseweb:root0' 306 | # instead of '/dev/md/root0'. 307 | # This is mdadm's protection against accidentally putting a RAID disk 308 | # into the wrong machine and corrupting data by accidental sync, see 309 | # https://bugzilla.redhat.com/show_bug.cgi?id=606481#c14 and onward. 310 | # We set the HOMEHOST manually go get the short '/dev/md' names, 311 | # and so that things look and are configured the same on all such 312 | # machines irrespective of host names. 313 | # We do not worry about plugging disks into the wrong machine because 314 | # we will never exchange disks between machines. 315 | environment.etc."mdadm.conf".text = '' 316 | HOMEHOST leaseweb 317 | ''; 318 | # The RAIDs are assembled in stage1, so we need to make the config 319 | # available there. 320 | boot.initrd.mdadmConf = config.environment.etc."mdadm.conf".text; 321 | 322 | # Network 323 | # Leaseweb uses static IP assignments only, see: 324 | # https://kb.leaseweb.com/network/ipv4-address-assignment-and-usage-guidelines#IPv4addressassignmentandusageguidelines-DHCP 325 | networking.useDHCP = false; 326 | networking.interfaces."$NIXOS_INTERFACE".ipv4.addresses = [ 327 | { 328 | address = "$IP_V4"; 329 | prefixLength = 24; 330 | } 331 | ]; 332 | networking.defaultGateway = "$DEFAULT_GATEWAY"; 333 | networking.nameservers = [ "8.8.8.8" ]; 334 | 335 | # Initial empty root password for easy login: 336 | users.users.root.initialHashedPassword = ""; 337 | services.openssh.permitRootLogin = "prohibit-password"; 338 | 339 | users.users.root.openssh.authorizedKeys.keys = [ 340 | # Replace this by your pubkey! 341 | "ssh-rsa AAAAAAAAAAA..." 342 | ]; 343 | 344 | services.openssh.enable = true; 345 | 346 | # This value determines the NixOS release with which your system is to be 347 | # compatible, in order to avoid breaking some software such as database 348 | # servers. You should change this only after NixOS release notes say you 349 | # should. 350 | system.stateVersion = "20.03"; # Did you read the comment? 351 | 352 | } 353 | EOF 354 | 355 | # TODO Remove once https://github.com/NixOS/nixpkgs/pull/85895 is merged and 356 | # backported to 20.03, or this script installs a newer version that has it. 357 | rm -f extra-grub-install-flags-20.03.tar.gz 358 | wget 'https://github.com/nh2/nixpkgs/archive/extra-grub-install-flags-20.03.tar.gz' 359 | rm -rf nixpkgs-extra-grub-install-flags-20.03 360 | tar xf extra-grub-install-flags-20.03.tar.gz 361 | NIX_PATH=nixpkgs=$PWD/nixpkgs-extra-grub-install-flags-20.03 362 | 363 | 364 | # Install NixOS 365 | PATH="$PATH" NIX_PATH="$NIX_PATH" `which nixos-install` --no-root-passwd --root /mnt --max-jobs 40 366 | 367 | umount /mnt 368 | 369 | reboot 370 | -------------------------------------------------------------------------------- /hosters/ovh-dedicated/ovh-dedicated-wipe-and-install-nixos.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Installs NixOS on an OVH server, wiping the server. 4 | # 5 | # This is for a specific server configuration; adjust where needed. 6 | # Originally written for an OVH STOR-1 server. 7 | # 8 | # Prerequisites: 9 | # * Create a LUKS key file at $LUKS_KEYFILE_PATH 10 | # e.g. by copying it up. 11 | # See https://wiki.archlinux.org/title/Dm-crypt/Device_encryption#Keyfiles 12 | # * Update the script to put in your SSH pubkey, adjust hostname, NixOS version etc. 13 | # 14 | # Usage: 15 | # ssh root@YOUR_SERVERS_IP bash -s < ovh-dedicated-wipe-and-install-nixos.sh 16 | # 17 | # When the script is done, make sure to boot the server from HD, not rescue mode again. 18 | 19 | # Explanations: 20 | # 21 | # * Following largely https://nixos.org/nixos/manual/index.html#sec-installing-from-other-distro. 22 | # * **Important:** We boot in UEFI mode, thus requiring an ESP. 23 | # Booting in LEGACY mode (non-UEFI boot, without ESP) would require that: 24 | # * `/boot` is on the same device as GRUB 25 | # * NVMe devices aren not used for booting (those require EFI boot) 26 | # We also did not manage to boot our OVH server in LEGACY mode on our SuperMicro mainboard, even when we installed `/` (including `/boot`) directly to a simple RAID1ed GPT partition. The screen just stayed black. 27 | # * We set a custom `configuration.nix` so that we can connect to the machine afterwards. 28 | # * This server has 1 SSD and 4 HDDs. 29 | # We'll ignore the SSD, putting the OS on the HDDs as well, so that everything is on RAID1. 30 | # We wipe the SSD though, so that if it had some boot partitions on it, they don't interfere. 31 | # Storage scheme: `partitions -> RAID -> LUKS -> LVM -> ext4`. 32 | # * A root user with empty password is created, so that you can just login 33 | # as root and press enter when using the OVH KVM. 34 | # Of course that empty-password login isn't exposed to the Internet. 35 | # Change the password afterwards to avoid anyone with physical access 36 | # being able to login without any authentication. 37 | # * The script reboots at the end. 38 | 39 | # Edit those variables to your need 40 | LUKS_KEYFILE_PATH="/root/benacofs-luks-key" 41 | FS_NAME="benacofs" 42 | HOSTNAME="benaco-cdn-na1" 43 | HOMEHOST="benaco-cdn" 44 | 45 | set -eu 46 | set -o pipefail 47 | 48 | set -x 49 | 50 | # Inspect existing disks 51 | lsblk 52 | 53 | # Undo existing setups to allow running the script multiple times to iterate on it. 54 | # We allow these operations to fail for the case the script runs the first time. 55 | set +e 56 | umount /mnt/boot/ESP* 57 | umount /mnt 58 | vgchange -an 59 | cryptsetup luksClose data0-unencrypted 60 | cryptsetup luksClose data1-unencrypted 61 | set -e 62 | 63 | # Stop all mdadm arrays that the boot may have activated. 64 | mdadm --stop --scan 65 | 66 | # Create wrapper for parted >= 3.3 that does not exit 1 when it cannot inform 67 | # the kernel of partitions changing (we use partprobe for that). 68 | echo -e "#! /usr/bin/env bash\nset -e\n" 'parted $@ 2> parted-stderr.txt || grep "unable to inform the kernel of the change" parted-stderr.txt && echo "This is expected, continuing" || echo >&2 "Parted failed; stderr: $(< parted-stderr.txt)"' > parted-ignoring-partprobe-error.sh && chmod +x parted-ignoring-partprobe-error.sh 69 | 70 | # Create partition tables (--script to not ask) 71 | ./parted-ignoring-partprobe-error.sh --script /dev/sda mklabel gpt 72 | ./parted-ignoring-partprobe-error.sh --script /dev/sdb mklabel gpt 73 | ./parted-ignoring-partprobe-error.sh --script /dev/sdc mklabel gpt 74 | ./parted-ignoring-partprobe-error.sh --script /dev/sdd mklabel gpt 75 | ./parted-ignoring-partprobe-error.sh --script /dev/nvme0n1 mklabel gpt 76 | 77 | # Create partitions (--script to not ask) 78 | # 79 | # Create EFI system partition (ESP) and main partition for each boot device. 80 | # We make it 550 M as recommended by the author of gdisk (https://www.rodsbooks.com/linux-uefi/); 81 | # using 550 ensures it's greater than 512 MiB, no matter if Mi or M were used. 82 | # For the non-boot devices, we still make space for an ESP partition 83 | # (in case the disks get repurposed for that at some point) but mark 84 | # it as `off` and label it `*-unused` to avoid confusion. 85 | # 86 | # Note we use "MB" instead of "MiB" because otherwise `--align optimal` has no effect; 87 | # as per documentation https://www.gnu.org/software/parted/manual/html_node/unit.html#unit: 88 | # > Note that as of parted-2.4, when you specify start and/or end values using IEC 89 | # > binary units like "MiB", "GiB", "TiB", etc., parted treats those values as exact 90 | # 91 | # Note: When using `mkpart` on GPT, as per 92 | # https://www.gnu.org/software/parted/manual/html_node/mkpart.html#mkpart 93 | # the first argument to `mkpart` is not a `part-type`, but the GPT partition name: 94 | # ... part-type is one of 'primary', 'extended' or 'logical', and may be specified only with 'msdos' or 'dvh' partition tables. 95 | # A name must be specified for a 'gpt' partition table. 96 | # GPT partition names are limited to 36 UTF-16 chars, see https://en.wikipedia.org/wiki/GUID_Partition_Table#Partition_entries_(LBA_2-33). 97 | ./parted-ignoring-partprobe-error.sh --script --align optimal /dev/sda -- mklabel gpt mkpart 'ESP-partition0' fat32 1MB 551MB set 1 esp on mkpart 'OS-partition0' 551MB 500GB mkpart 'data-partition0' 500GB '100%' 98 | ./parted-ignoring-partprobe-error.sh --script --align optimal /dev/sdb -- mklabel gpt mkpart 'ESP-partition1' fat32 1MB 551MB set 1 esp on mkpart 'OS-partition1' 551MB 500GB mkpart 'data-partition1' 500GB '100%' 99 | ./parted-ignoring-partprobe-error.sh --script --align optimal /dev/sdc -- mklabel gpt mkpart 'ESP-partition2-unused' fat32 1MB 551MB set 1 esp off mkpart 'data-partition2' 551MB '100%' 100 | ./parted-ignoring-partprobe-error.sh --script --align optimal /dev/sdd -- mklabel gpt mkpart 'ESP-partition3-unused' fat32 1MB 551MB set 1 esp off mkpart 'data-partition3' 551MB '100%' 101 | 102 | # Relaod partitions 103 | partprobe 104 | 105 | # Wait for all devices to exist 106 | udevadm settle --timeout=5 --exit-if-exists=/dev/sda1 107 | udevadm settle --timeout=5 --exit-if-exists=/dev/sda2 108 | udevadm settle --timeout=5 --exit-if-exists=/dev/sda3 109 | udevadm settle --timeout=5 --exit-if-exists=/dev/sdb1 110 | udevadm settle --timeout=5 --exit-if-exists=/dev/sdb2 111 | udevadm settle --timeout=5 --exit-if-exists=/dev/sdb3 112 | udevadm settle --timeout=5 --exit-if-exists=/dev/sdc1 113 | udevadm settle --timeout=5 --exit-if-exists=/dev/sdc2 114 | udevadm settle --timeout=5 --exit-if-exists=/dev/sdd1 115 | udevadm settle --timeout=5 --exit-if-exists=/dev/sdd2 116 | 117 | # Array gets created automatically 118 | # Stop it so mdadm can zero the superblock 119 | for f in $(ls -p /dev | grep -v / | grep md); do 120 | mdadm --stop /dev/$f 121 | done 122 | 123 | # Wipe any previous RAID signatures 124 | mdadm --zero-superblock /dev/sda2 125 | mdadm --zero-superblock /dev/sda3 126 | mdadm --zero-superblock /dev/sdb2 127 | mdadm --zero-superblock /dev/sdb3 128 | mdadm --zero-superblock /dev/sdc2 129 | mdadm --zero-superblock /dev/sdd2 130 | 131 | # Create RAIDs 132 | # Note that during creating and boot-time assembly, mdadm cares about the 133 | # host name, and the existence and contents of `mdadm.conf`! 134 | # This also affects the names appearing in /dev/md/ being different 135 | # before and after reboot in general (but we take extra care here 136 | # to pass explicit names, and set HOMEHOST for the rebooting system further 137 | # down, so that the names appear the same). 138 | # Almost all details of this are explained in 139 | # https://bugzilla.redhat.com/show_bug.cgi?id=606481#c14 140 | # and the followup comments by Doug Ledford. 141 | mdadm --create --run --verbose /dev/md/root0 --level=1 --raid-devices=2 --homehost=benaco-cdn --name=root0 /dev/sda2 /dev/sdb2 142 | mdadm --create --run --verbose /dev/md/data0-encrypted --level=1 --raid-devices=2 --homehost=benaco-cdn --name=data0-encrypted /dev/sda3 /dev/sdb3 143 | mdadm --create --run --verbose /dev/md/data1-encrypted --level=1 --raid-devices=2 --homehost=benaco-cdn --name=data1-encrypted /dev/sdc2 /dev/sdd2 144 | 145 | # Assembling the RAID can result in auto-activation of previously-existing LVM 146 | # groups, preventing the RAID block device wiping below with 147 | # `Device or resource busy`. So disable all VGs first. 148 | vgchange -an 149 | 150 | # Wipe filesystem signatures that might be on the RAID from some 151 | # possibly existing older use of the disks (RAID creation does not do that). 152 | # See https://serverfault.com/questions/911370/why-does-mdadm-zero-superblock-preserve-file-system-information 153 | wipefs -a /dev/md/root0 154 | wipefs -a /dev/md/data0-encrypted 155 | wipefs -a /dev/md/data1-encrypted 156 | 157 | # Disable RAID recovery. We don't want this to slow down machine provisioning 158 | # in the rescue mode. It can run in normal operation after reboot. 159 | echo 0 > /proc/sys/dev/raid/speed_limit_max 160 | 161 | # LUKS encryption (--batch-mode to not ask) 162 | cryptsetup --batch-mode luksFormat /dev/md/data0-encrypted $LUKS_KEYFILE_PATH 163 | cryptsetup --batch-mode luksFormat /dev/md/data1-encrypted $LUKS_KEYFILE_PATH 164 | 165 | # Decrypt 166 | cryptsetup luksOpen /dev/md/data0-encrypted data0-unencrypted --key-file $LUKS_KEYFILE_PATH 167 | cryptsetup luksOpen /dev/md/data1-encrypted data1-unencrypted --key-file $LUKS_KEYFILE_PATH 168 | 169 | # LVM 170 | # PVs 171 | pvcreate /dev/mapper/data0-unencrypted 172 | pvcreate /dev/mapper/data1-unencrypted 173 | # VGs 174 | vgcreate vg0 /dev/mapper/data0-unencrypted /dev/mapper/data1-unencrypted 175 | # LVs 176 | lvcreate --extents 95%FREE -n $FS_NAME vg0 # 5% slack space 177 | 178 | # Filesystems (-F to not ask on preexisting FS) 179 | mkfs.fat -F 32 -n esp0 /dev/disk/by-partlabel/ESP-partition0 180 | mkfs.fat -F 32 -n esp1 /dev/disk/by-partlabel/ESP-partition1 181 | mkfs.ext4 -F -L root /dev/md/root0 182 | mkfs.ext4 -F -L $FS_NAME /dev/mapper/vg0-$FS_NAME 183 | 184 | # Creating file systems changes their UUIDs. 185 | # Trigger udev so that the entries in /dev/disk/by-uuid get refreshed. 186 | # `nixos-generate-config` depends on those being up-to-date. 187 | # See https://github.com/NixOS/nixpkgs/issues/62444 188 | udevadm trigger 189 | 190 | # Wait for FS labels to appear 191 | udevadm settle --timeout=5 --exit-if-exists=/dev/disk/by-label/root 192 | udevadm settle --timeout=5 --exit-if-exists=/dev/disk/by-label/$FS_NAME 193 | 194 | # NixOS pre-installation mounts 195 | 196 | # Mount target root partition 197 | mount /dev/disk/by-label/root /mnt 198 | # Mount efivars unless already mounted 199 | # (OVH rescue doesn't have them by default and the NixOS installer needs this) 200 | mount | grep efivars || mount -t efivarfs efivarfs /sys/firmware/efi/efivars 201 | # Mount our ESP partitions 202 | mkdir -p /mnt/boot/ESP0 203 | mkdir -p /mnt/boot/ESP1 204 | mount /dev/disk/by-label/esp0 /mnt/boot/ESP0 205 | mount /dev/disk/by-label/esp1 /mnt/boot/ESP1 206 | 207 | # Installing nix 208 | 209 | # Allow installing nix as root, see 210 | # https://github.com/NixOS/nix/issues/936#issuecomment-475795730 211 | mkdir -p /etc/nix 212 | echo "build-users-group =" > /etc/nix/nix.conf 213 | echo "sandbox = false" >> /etc/nix/nix.conf 214 | 215 | # https://github.com/NixOS/nix/issues/7790#issuecomment-1451990482 216 | mount --bind / / 217 | 218 | curl -L https://nixos.org/nix/install | sh -s -- --daemon --yes 219 | set +u +x # sourcing this may refer to unset variables that we have no control over 220 | . $HOME/.nix-profile/etc/profile.d/nix.sh 221 | set -u -x 222 | 223 | nix-channel --add https://nixos.org/channels/nixos-24.05 nixpkgs 224 | nix-channel --update 225 | 226 | # Getting NixOS installation tools 227 | nix-env -iE "_: with import { configuration = {}; }; with pkgs; [ nixos-install-tools ]" 228 | 229 | nixos-generate-config --root /mnt 230 | 231 | # On the OVH rescue mode, the default Internet interface is called `eth0`. 232 | # Find what its name will be under NixOS, which uses stable interface names. 233 | # See https://major.io/2015/08/21/understanding-systemds-predictable-network-device-names/#comment-545626 234 | INTERFACE=$(udevadm info -e | grep -A 11 ^P.*eth0 | grep -o -E 'ID_NET_NAME_ONBOARD=\w+' | cut -d= -f2) 235 | echo "Determined INTERFACE as $INTERFACE" 236 | 237 | IP_V4=$(ip route get 8.8.8.8 | head -1 | cut -d' ' -f7) 238 | echo "Determined IP_V4 as $IP_V4" 239 | 240 | # From https://stackoverflow.com/questions/1204629/how-do-i-get-the-default-gateway-in-linux-given-the-destination/15973156#15973156 241 | read _ _ DEFAULT_GATEWAY _ < <(ip route list match 0/0); echo "$DEFAULT_GATEWAY" 242 | echo "Determined DEFAULT_GATEWAY as $DEFAULT_GATEWAY" 243 | 244 | 245 | # Generate `configuration.nix`. Note that we splice in shell variables. 246 | cat > /mnt/etc/nixos/configuration.nix <