├── .gitignore ├── LICENSE ├── README.md └── corezfs /.gitignore: -------------------------------------------------------------------------------- 1 | coreos_developer_container.bin* 2 | *.swp 3 | corezfs*.tar.gz 4 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | This is free and unencumbered software released into the public domain. 2 | 3 | Anyone is free to copy, modify, publish, use, compile, sell, or 4 | distribute this software, either in source code form or as a compiled 5 | binary, for any purpose, commercial or non-commercial, and by any 6 | means. 7 | 8 | In jurisdictions that recognize copyright laws, the author or authors 9 | of this software dedicate any and all copyright interest in the 10 | software to the public domain. We make this dedication for the benefit 11 | of the public at large and to the detriment of our heirs and 12 | successors. We intend this dedication to be an overt act of 13 | relinquishment in perpetuity of all present and future rights to this 14 | software under copyright law. 15 | 16 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 17 | EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 18 | MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. 19 | IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR 20 | OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, 21 | ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR 22 | OTHER DEALINGS IN THE SOFTWARE. 23 | 24 | For more information, please refer to 25 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # corezfs 2 | 3 | ## ZFS on CoreOS 4 | This is a script to compile and install ZFS on CoreOS. It is meant to be installed on a fresh clean CoreOS instance. Although it can be run manually, it is envisioned that it is typically used as part of an automated provisioning process. 5 | 6 | ## Installation 7 | To build and install ZFS on Linux on CoreOS download the script to the target machine and run it with the install command: 8 | ```bash 9 | sudo ./corezfs install 10 | ``` 11 | 12 | After the script completes zfs kernel modules should be loaded, and user tools installed. 13 | 14 | The script will create an archive file which can be used to install zfs on other CoreOS instances without having to rebuild it on each instance. The archive filename will indicate which CoreOS version the archive file is targeted for. 15 | 16 | To install zfs on another instance using the archive file, copy the corezfs script and archive file to the target machine and run: 17 | ```bash 18 | sudo ./corezfs install 19 | ``` 20 | 21 | The script will automatically look for an archive file in the current directory with the target CoreOS version in the filename and install it if found, otherwise it will build a new archive file for the target system and install it. 22 | 23 | 24 | ## Introduction 25 | [CoreOS](https://coreos.com/os/docs/latest) is a light weight linux distribution designed specifically for running containers but does not currently come with support for ZFS. The idea is the only thing CoreOS does is coordinate containers and all work is performed inside containers. CoreOS is also very focused on reliability, clustering and federating with support for things like kubernetes baked in. 26 | 27 | [ZFS](http://zfsonlinux.org) is a very performant filesystem that supports error checking, snapshots, clones, native nfs and cifs support, and incremental backups. The ZFS on Linux project is a port of OpenZFS for Linux. 28 | 29 | Because the design philosophy of CoreOS is to be a minimal "container orchestration" tool, it is locked down very tightly and most of the file system is read only, which presents a problem for software which requires kernel modules (both because containers need special permissions to communicate with the kernel, and because the CoreOS kernel modules folder is read only). 30 | 31 | This script downloads a CoreOS development environment and runs it in a container to build zfs and create the archive file. The archive file is a tar file containing an /etc/ directory containing zfs configuration information, and an /opt/corezfs/usr directory which is overlaid over the /usr filesystem and contains the kernel modules and binaries. A systemd unit file called zfs-overlay.service (included in the /etc/ directory of the archive file) mounts the overlay at startup. 32 | 33 | This script was written on CoreOS stable (1465.7.0), but in theory, will work on any version / channel (ie. stable, beta, alpha). It installs the latest release of ZFS on Linux which is based on OpenZFS and consists of a repository [zfs](https://github.com/zfsonlinux/zfs) which includes the upstream OpenZFS implementation and a repository [spl](https://github.com/zfsonlinux/spl) which is a shim to run OpenZFS on Linux. 34 | 35 | Note that the script does not effect the filesystem that CoreOS is mounted on, it allows additional block devices to be mounted using the ZFS file system. 36 | 37 | Hopefully this script will allow more people to experiment with ZFS on CoreOS to gain enough support that the CoreOS developers will bake it into CoreOS (note that this implementation does not gracefully handle updates to the CoreOS OS). 38 | 39 | ## References 40 | This script is adapted from the instructions from: 41 | 42 | 1. https://coreos.com/os/docs/latest/kernel-modules.html 43 | 2. https://github.com/zfsonlinux/zfs/wiki/Building-ZFS 44 | 45 | ## Issues 46 | 47 | 1. This should really be baked into CoreOS (so hopefully this script is just a temporary stop-gap solution until the CoreOS developers include ZFS support natively) 48 | 2. It is uncertain whether the kernel drivers will continue to work after a CoreOS update, or whether the script needs to be re-run to re-build them (further support for why it should be baked into CoreOS). It is recommended to turn off CoreOS automatic updates to ensure that an automatic update does not result an incompatibility with the kernel drivers. 49 | 50 | ## Using ZFS 51 | 52 | There are some good resources for using zfs at: 53 | - https://www.csparks.com/ZFS%20Without%20Tears.html. 54 | - https://pthree.org/2012/04/17/install-zfs-on-debian-gnulinux 55 | -------------------------------------------------------------------------------- /corezfs: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | version="v2.0.6" 4 | 5 | export PS1="host-debug# " 6 | yell() { echo "$(basename $0): $*" >&2; } 7 | die() { yell "FATAL $*"; [ ${debug} ] && sh || exit 111; } 8 | try() { yell ${1}; shift; "$@" || die "ERROR executing: $*"; } 9 | 10 | [ ${ZFS_VERSION:=$(curl -s https://api.github.com/repos/zfsonlinux/zfs/releases/latest | jq -r ".tag_name" 2> /dev/null)} ] \ 11 | || die "failed to query latest zfs version" 12 | 13 | ZFS_VERSION=${ZFS_VERSION#zfs-} 14 | yell "building ZFS on Linux v${ZFS_VERSION}" 15 | 16 | . /usr/share/coreos/release 17 | . /usr/share/coreos/update.conf 18 | 19 | VERSION="${COREOS_RELEASE_BOARD}_${COREOS_RELEASE_VERSION}_${GROUP:=stable}" 20 | ARCHIVE="corezfs_${VERSION}.tar.gz" 21 | 22 | read -d '' help_message <<-EOF 23 | 24 | corezfs (${version}) https://github.com/varasys/corezfs 25 | 26 | Bash script to build ZFS on Linux (ZoL) on CoreOS 27 | 28 | Usage: 29 | corezfs build [archive_file] 30 | corezfs install [archive_file] 31 | corezfs -h | -v 32 | 33 | Options: 34 | -h ) echo this help file to stdout 35 | -v ) echo corezfs version to stdout 36 | 37 | The build command will build zfs and create the archive file which is an 38 | overlay for the "/usr" filesystem. The default archive_file is 39 | "./${ARCHIVE}". The archive file should be installed to the target system 40 | using the install command. The install command builds an archive file 41 | automatically if it doesn't exist, so the build command is only required 42 | to build the archive on a system it will not be installed on. 43 | 44 | The archive file can be used to install the zfs binaries on other systems 45 | instead of building it each time. 46 | 47 | The install command will build the archive file if it doesn't exist and 48 | then extracts the archive file including the overlay to /opt/corezfs and 49 | some configuration files into /etc, and starts the relevant systemd 50 | services (including a zfs-overlay.service to mount the overlay). 51 | 52 | By default, the script will query the ZoL github repository for the latest 53 | released version. The ZoL version may be specified manually by setting the 54 | ZFS_VERSION environment variable (ie. "ZFS_VERSION=0.7.1 corezfs install"). 55 | 56 | This is meant to be built and installed on a clean CoreOS instance. 57 | 58 | This script was originally written and tested on CoreOS 1465.7.0 and 59 | ZoL v0.7.1, but should work on other versions unless upstream 60 | changes break the script. It has been updated for CoreOS 1520.6.0 and 61 | ZoL v0.7.2. 62 | 63 | EOF 64 | 65 | while getopts ":dhv" flag; do 66 | case ${flag} in 67 | d ) yell "running in debug mode"; debug="true";; 68 | h ) echo "${help_message}"; exit 0;; 69 | v ) echo "${version}"; exit 0;; 70 | \? ) die "Invalid option: -${OPTARG}";; 71 | esac 72 | done 73 | shift $((OPTIND-1)) 74 | [ ${2} ] && ARCHIVE=${2} 75 | 76 | [ ${EUID} -eq 0 ] || die "must be run as root user" 77 | 78 | read -d '' build_script <<-'EOF' 79 | #!/usr/bin/env bash 80 | 81 | export PS1="container-debug# " 82 | yell() { echo "$(basename $0): $*" >&2; } 83 | die() { yell "FATAL $*"; [ ${debug} ] && sh || exit 111; } 84 | try() { mesg=${1}; yell ${mesg}; shift; "$@" || die "ERROR executing: $*"; } 85 | 86 | try "sourcing /usr/share/coreos/release" \ 87 | . /usr/share/coreos/release 88 | 89 | try "emerging gitclone" 90 | emerge-gitclone 91 | 92 | try "checking out coreos-overlay" \ 93 | git -C /var/lib/portage/coreos-overlay checkout build-${COREOS_RELEASE_VERSION%%.*} 94 | 95 | try "emerging coreos-sources" \ 96 | emerge -v coreos-sources 97 | 98 | try "copying /proc/config.gz to /usr/src/linux/.config" \ 99 | gzip -cd /proc/config.gz > /usr/src/linux/.config 100 | 101 | try "preparing modules" \ 102 | make -C /usr/src/linux modules_prepare 103 | 104 | try "creating /usr/local/portage/sys-fs/{zfs,spl}" \ 105 | mkdir -p /usr/local/portage/{metadata,profiles,sys-fs/{zfs,spl}} 106 | 107 | cat > /etc/portage/repos.conf/localrepo.conf <<-EOF2 108 | [localrepo] 109 | location = /usr/local/portage 110 | EOF2 111 | 112 | echo "localrepo" > /usr/local/portage/profiles/repo_name 113 | 114 | cat > /usr/local/portage/metadata/layout.conf <<-EOF2 115 | masters = portage-stable 116 | auto-sync = false 117 | EOF2 118 | 119 | cat > /usr/local/portage/sys-fs/spl/spl-${ZFS_VERSION}.ebuild <<-'EOF2' 120 | EAPI=6 121 | 122 | DESCRIPTION="SPL shim for ZFS on Linux (ZoL)" 123 | HOMEPAGE="http://zfsonlinux.org" 124 | SRC_URI="https://github.com/zfsonlinux/zfs/releases/download/zfs-${PV}/${PF}.tar.gz" 125 | 126 | LICENSE="CCDL" 127 | SLOT="0" 128 | KEYWORDS="amd64 x86" 129 | #TODO ZoL requires the following ARCH hack to compile modules as of 0.7.1 (may not be needed for future releases) 130 | ARCH="x86" 131 | EOF2 132 | 133 | cat > /usr/local/portage/sys-fs/zfs/zfs-${ZFS_VERSION}.ebuild <<-'EOF2' 134 | EAPI=6 135 | 136 | DESCRIPTION="ZFS on Linux (ZoL)" 137 | HOMEPAGE="http://zfsonlinux.org" 138 | SRC_URI="https://github.com/zfsonlinux/zfs/releases/download/zfs-${PV}/${PF}.tar.gz" 139 | 140 | LICENSE="CCDL" 141 | SLOT="0" 142 | KEYWORDS="amd64 x86" 143 | #TODO ZoL requires the following ARCH hack to compile modules as of 0.7.1 (may not be needed for future releases) 144 | ARCH="x86" 145 | 146 | DEPEND="sys-fs/spl" 147 | RDEPEND="${DEPEND}" 148 | 149 | source_configure() { 150 | econf \ 151 | --disable-sysvinit \ 152 | --with-systemdunitdir=/etc/systemd/system \ 153 | --with-systemdpresetdir=/etc/systemd/system-preset 154 | } 155 | EOF2 156 | 157 | yell "Creating Systemd Unit Files" 158 | cat > /corezfs/usr/lib64/systemd/system/zfs-overlay.service <<-EOF2 159 | [Unit] 160 | Description=CoreZFS /usr overlay 161 | DefaultDependencies=no 162 | ConditionPathExists=/opt/corezfs/usr 163 | Requires=zfs.target 164 | Before=zfs-import-cache.service 165 | Before=zfs-import-scan.service 166 | Before=zfs-mount.service 167 | Before=zfs-share.service 168 | Before=zfs-zed.service 169 | 170 | [Service] 171 | Type=oneshot 172 | RemainAfterExit=yes 173 | ExecStart=/usr/bin/mount -t overlay overlay -o lowerdir=/opt/corezfs/usr:/usr /usr 174 | ExecStartPost=/usr/sbin/modprobe zfs 175 | 176 | [Install] 177 | RequiredBy=local-fs.target 178 | EOF2 179 | 180 | cat > /corezfs/usr/lib64/systemd/system-preset/30-corezfs.preset <<-EOF2 181 | enable zfs-overlay.service 182 | EOF2 183 | 184 | try "generating spl manifest" \ 185 | ebuild /usr/local/portage/sys-fs/spl/spl-${ZFS_VERSION}.ebuild manifest 186 | try "generating zfs manifest" \ 187 | ebuild /usr/local/portage/sys-fs/zfs/zfs-${ZFS_VERSION}.ebuild manifest 188 | 189 | try "building ZFS on Linux (ZoL) for CoreOS" emerge --root=/corezfs --oneshot zfs 190 | 191 | try "running depmod" chroot /corezfs depmod 192 | try "running ldconfig" chroot /corezfs ldconfig -v 193 | 194 | yell "finished building ZFS on Linux (ZoL) overlay for CoreOS" 195 | EOF 196 | 197 | case "$1" in 198 | build) 199 | yell "Building ZFS on Linux (ZoL) for CoreOS ($VERSION)" 200 | yell "target archive file: ${ARCHIVE}" 201 | 202 | try "creating build directory ${CZ_BUILD_DIR:="/opt/corezfs_build"}" \ 203 | mkdir -p ${CZ_BUILD_DIR} && CZ_BUILD_DIR=$(cd ${CZ_BUILD_DIR}; pwd) 204 | try "creating overlay directory ${CZ_OVERLAY_DIR:=${CZ_BUILD_DIR}/overlay}" \ 205 | mkdir -p ${CZ_OVERLAY_DIR}/opt/corezfs/usr && CZ_OVERLAY_DIR=$(cd ${CZ_OVERLAY_DIR}; pwd) 206 | 207 | try "preparing build script" \ 208 | echo "${build_script}" > "${CZ_OVERLAY_DIR}/opt/corezfs/build-zfs.sh" 209 | try "setting build script executable" \ 210 | chmod +x "${CZ_OVERLAY_DIR}/opt/corezfs/build-zfs.sh" 211 | 212 | DEVENV="coreos_${VERSION}.bin" 213 | if [ ! -r "${CZ_BUILD_DIR}/${DEVENV}" ]; then 214 | try "refreshing CoreOS signing key" \ 215 | curl -s "https://coreos.com/security/image-signing-key/CoreOS_Image_Signing_Key.asc" -o - | gpg2 --import 216 | 217 | DEVREPO="https://${GROUP}.release.core-os.net/${COREOS_RELEASE_BOARD}/${COREOS_RELEASE_VERSION}/coreos_developer_container.bin.bz2" 218 | if [ ! -r "${CZ_BUILD_DIR}/${DEVENV}.bz2" ]; then 219 | try "downloading CoreOS development container" \ 220 | curl -L "${DEVREPO}" > "${CZ_BUILD_DIR}/${DEVENV}.bz2" 221 | fi 222 | 223 | try "verifying development container signature" \ 224 | gpg2 --verify <(try "downloading signature" curl -L "${DEVREPO}.sig") "${CZ_BUILD_DIR}/${DEVENV}.bz2" 225 | try "decompressing development container (this may take awhile)" \ 226 | bunzip2 -fv "${CZ_BUILD_DIR}/${DEVENV}.bz2" > "${CZ_BUILD_DIR}/${DEVENV}" 227 | fi 228 | 229 | try "allocating swap file" fallocate -l 1024m ${CZ_BUILD_DIR}/swap 230 | trap "{ swapoff ${CZ_BUILD_DIR}/swap; rm ${CZ_BUILD_DIR}/swap; }" EXIT 231 | try "setting swap file permissions" chmod 600 ${CZ_BUILD_DIR}/swap 232 | try "initializing swap file" mkswap ${CZ_BUILD_DIR}/swap 233 | try "activating swap" swapon ${CZ_BUILD_DIR}/swap 234 | try "creating tmp dir" mkdir -p "${CZ_BUILD_DIR}/tmp" 235 | try "launching development container" \ 236 | systemd-nspawn \ 237 | --image="${CZ_BUILD_DIR}/${DEVENV}" \ 238 | --ephemeral \ 239 | --setenv="debug=${debug}" \ 240 | --setenv="ZFS_VERSION=${ZFS_VERSION}" \ 241 | --setenv="CZ_SPLREPO=${CZ_SPLREPO:=https://api.github.com/repos/zfsonlinux/spl/tarball}" \ 242 | --setenv="CZ_ZFSREPO=${CZ_ZFSREPO:=https://api.github.com/repos/zfsonlinux/zfs/tarball}" \ 243 | --bind="${CZ_BUILD_DIR}/tmp:/var/tmp/portage" \ 244 | --overlay="+/var/lib/portage::/var/lib/portage" \ 245 | --overlay="/:${CZ_OVERLAY_DIR}/opt/corezfs:/corezfs" \ 246 | --overlay="/usr:${CZ_OVERLAY_DIR}/opt/corezfs/usr:/corezfs/usr" \ 247 | --as-pid2 \ 248 | /corezfs/build-zfs.sh 249 | 250 | try "transferring systemd unit files to /etc/systemd" \ 251 | cp --update --recursive ${CZ_OVERLAY_DIR}/opt/corezfs/usr/lib64/systemd ${CZ_OVERLAY_DIR}/opt/corezfs/etc/ 252 | 253 | try "creating ${ARCHIVE} tarball" \ 254 | tar -zcvf "${ARCHIVE}" -C "${CZ_OVERLAY_DIR}" opt/corezfs/usr -C "${CZ_OVERLAY_DIR}/opt/corezfs" etc 255 | 256 | [ ${debug} ] || try "cleaning up temporary build directories" \ 257 | rm -rf "${CZ_OVERLAY_DIR}" "${CZ_BUILD_DIR}" 258 | 259 | cat >&2 <<-EOF 260 | 261 | Finished Building ZFS on Linux (ZoL) for CoreOS ($VERSION) 262 | 263 | Archive file: ${ARCHIVE} 264 | 265 | Run "corezfs install ${ARCHIVE}" to install 266 | 267 | EOF 268 | ;; 269 | install) 270 | yell "Installing ZFS on Linux (ZoL) for CoreOS ($VERSION)" 271 | 272 | [ -r ${ARCHIVE} ] || try "no archive file available; building one ..." ${0} ${debug:+"-d"} build ${ARCHIVE} 273 | 274 | try "extracting ${ARCHIVE}" \ 275 | tar -zxvf ${ARCHIVE} -C / 276 | try "enabling zfs-overlay.service" \ 277 | systemctl enable --now zfs-overlay.service 278 | try "presetting systemd units" \ 279 | systemctl preset-all --now 280 | 281 | yell "Finished Installing ZFS on Linux (ZoL) for CoreOS ($VERSION)" 282 | ;; 283 | *) 284 | die "unknown sub-command \"${1}\" (run \"$(basename ${0}) -h\" for help)" 285 | ;; 286 | esac 287 | --------------------------------------------------------------------------------