├── .gitignore ├── anita-2.11.tar.gz.sha1 ├── AUTHORS ├── CONTRIBUTORS ├── CONTRIBUTING.md ├── LICENSE ├── mkvm.py ├── make.bash └── README.md /.gitignore: -------------------------------------------------------------------------------- 1 | anita-2.* 2 | !*.sha1 3 | -------------------------------------------------------------------------------- /anita-2.11.tar.gz.sha1: -------------------------------------------------------------------------------- 1 | 3156faf4e1b309bf093190d0e2c47b4b7a635ab5 anita-2.11.tar.gz 2 | -------------------------------------------------------------------------------- /AUTHORS: -------------------------------------------------------------------------------- 1 | # This source code refers to The Go Authors for copyright purposes. 2 | # The master list of authors is in the main Go distribution, 3 | # visible at http://tip.golang.org/AUTHORS. 4 | -------------------------------------------------------------------------------- /CONTRIBUTORS: -------------------------------------------------------------------------------- 1 | # This source code was written by the Go contributors. 2 | # The master list of contributors is in the main Go distribution, 3 | # visible at http://tip.golang.org/CONTRIBUTORS. 4 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # How to contribute 2 | 3 | We would love to accept your patches and contributions to this project. There 4 | are just a few small guidelines you need to follow. 5 | 6 | ## Contributor License Agreement 7 | 8 | Contributions to this project must be accompanied by a Contributor License 9 | Agreement. You (or your employer) retain the copyright to your contribution, 10 | this simply gives us permission to use and redistribute your contributions as 11 | part of the project. Head over to to see 12 | your current agreements on file or to sign a new one. 13 | 14 | You generally only need to submit a CLA once, so if you have already submitted 15 | one (even if it was for a different project), you probably do not need to do it 16 | again. 17 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) 2009 The Go Authors. All rights reserved. 2 | 3 | Redistribution and use in source and binary forms, with or without 4 | modification, are permitted provided that the following conditions are 5 | met: 6 | 7 | * Redistributions of source code must retain the above copyright 8 | notice, this list of conditions and the following disclaimer. 9 | * Redistributions in binary form must reproduce the above 10 | copyright notice, this list of conditions and the following disclaimer 11 | in the documentation and/or other materials provided with the 12 | distribution. 13 | * Neither the name of Google Inc. nor the names of its 14 | contributors may be used to endorse or promote products derived from 15 | this software without specific prior written permission. 16 | 17 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 18 | "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 19 | LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 20 | A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 21 | OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 22 | SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 23 | LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 24 | DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 25 | THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 26 | (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 27 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 28 | -------------------------------------------------------------------------------- /mkvm.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # Copyright 2016 The Go Authors. All rights reserved. 3 | # Use of this source code is governed by a BSD-style 4 | # license that can be found in the LICENSE file. 5 | 6 | import anita 7 | import ftplib 8 | import sys 9 | 10 | 11 | def find_latest_release(branch, arch): 12 | """Find the latest NetBSD-current release for the given arch. 13 | 14 | Returns: 15 | the full path to the release. 16 | """ 17 | conn = ftplib.FTP("nyftp.netbsd.org") 18 | conn.login() 19 | conn.cwd("/pub/NetBSD-daily/%s" % branch) 20 | releases = conn.nlst() 21 | releases.sort(reverse=True) 22 | for r in releases: 23 | archs = conn.nlst(r) 24 | if not archs: 25 | next 26 | has_arch = [a for a in archs if a.endswith(arch)] 27 | if has_arch: 28 | return "https://nycdn.netbsd.org/pub/NetBSD-daily/%s/%s/" % ( 29 | branch, 30 | has_arch[0], 31 | ) 32 | 33 | 34 | arch = sys.argv[1] 35 | branch = sys.argv[2] 36 | disk_size = sys.argv[3] 37 | 38 | commands = [ 39 | """cat > /etc/ifconfig.vioif0 << EOF 40 | !dhcpcd vioif0 41 | mtu 1460 42 | EOF""", 43 | "dhcpcd", 44 | """ed /etc/fstab << EOF 45 | H 46 | %s/wd0/sd0/ 47 | wq 48 | EOF""", 49 | "sync; shutdown -hp now", 50 | ] 51 | 52 | 53 | a = anita.Anita( 54 | anita.URL(find_latest_release(branch, arch)), 55 | workdir="work-%s-%s" % (branch, arch), 56 | disk_size=disk_size, 57 | memory_size="1G", 58 | persist=True, 59 | ) 60 | child = a.boot() 61 | anita.login(child) 62 | 63 | for cmd in commands: 64 | anita.shell_cmd(child, cmd, 1200) 65 | 66 | # Sometimes, the halt command times out, even though it has completed 67 | # successfully. 68 | try: 69 | a.halt() 70 | except: 71 | pass 72 | -------------------------------------------------------------------------------- /make.bash: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # Copyright 2016 The Go Authors. All rights reserved. 3 | # Use of this source code is governed by a BSD-style 4 | # license that can be found in the LICENSE file. 5 | 6 | # This script uses Anita (an automated NetBSD installer) for setting up 7 | # the VM. It needs the following things on the build host: 8 | # - qemu 9 | # - cdrtools 10 | # - GNU tar (not BSD tar) 11 | # - Python 3 12 | # - python-pexpect 13 | # - coreutils (for sha1sum) 14 | 15 | set -e -x 16 | 17 | ANITA_VERSION=2.11 18 | ARCH=${1:-amd64} 19 | RELEASE=${2:-netbsd-10} 20 | DISK_SIZE=${3:-4G} 21 | 22 | # Must use GNU tar. On NetBSD, tar is BSD tar and gtar is GNU. 23 | TAR=tar 24 | if which gtar > /dev/null; then 25 | TAR=gtar 26 | fi 27 | 28 | SHA1SUM=sha1sum 29 | if which gsha1sum > /dev/null; then 30 | SHA1SUM=gsha1sum 31 | fi 32 | 33 | PYTHON= 34 | for cmd in python3 python; do 35 | if which ${cmd} > /dev/null; then 36 | PYTHON=${cmd} 37 | break 38 | fi 39 | done 40 | 41 | WORKDIR=work-${RELEASE}-${ARCH} 42 | 43 | # Remove WORKDIR unless -k (keep) is given. 44 | if [ "$1" != "-k" ]; then 45 | rm -rf ${WORKDIR} 46 | fi 47 | 48 | # Download and build anita (automated NetBSD installer). 49 | if ! ${SHA1SUM} -c anita-${ANITA_VERSION}.tar.gz.sha1; then 50 | curl -vO http://www.gson.org/netbsd/anita/download/anita-${ANITA_VERSION}.tar.gz 51 | ${SHA1SUM} -c anita-${ANITA_VERSION}.tar.gz.sha1 || exit 1 52 | fi 53 | 54 | ${TAR} xfz anita-${ANITA_VERSION}.tar.gz 55 | cd anita-${ANITA_VERSION} 56 | ${PYTHON} setup.py build 57 | cd .. 58 | 59 | env PYTHONPATH=${PWD}/anita-${ANITA_VERSION} ${PYTHON} mkvm.py ${ARCH} ${RELEASE} ${DISK_SIZE} 60 | 61 | echo "Archiving wd0.img (this may take a while)" 62 | ${TAR} --format=oldgnu -Szcf netbsd-${ARCH}-gce.tar.gz --transform s,${WORKDIR}/wd0.img,disk.raw, ${WORKDIR}/wd0.img 63 | echo "Done. GCE image is netbsd-${ARCH}-gce.tar.gz." 64 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Creating NetBSD images for Google Compute Engine 2 | 3 | This repository holds tools to build a NetBSD image for use on Google Compute 4 | Engine (GCE). GCE is part of the Google Cloud Platform. 5 | 6 | ## Running `make.bash` 7 | 8 | `make.bash` can be run under a GNU/Linux, BSD or macOS operating system. To run 9 | the script, you need a few things to be installed: 10 | 11 | * bash 12 | * qemu 13 | * cdrtools 14 | * GNU tar (http://pkgsrc.se/archivers/gtar) 15 | * GNU coreutils (http://pkgsrc.se/sysutils/coreutils) 16 | * Python 3 17 | * python-pexpect (http://pkgsrc.se/devel/py-pexpect) 18 | 19 | When you run 20 | 21 | ``` 22 | bash ./make.bash 23 | ``` 24 | 25 | it will download a distfile for Anita (an automated NetBSD installation tool), 26 | which will download and install NetBSD 9_STABLE in a virtual machine on the local 27 | host. It then adds several tweaks to ensure that networking and storage will 28 | work on GCE and packs the image into a tar.gz file. 29 | 30 | Optionally, you can give the script an architecture (`i386` or `amd64`), a 31 | branch name and the desired root disk size as parameters, for example 32 | 33 | ``` 34 | bash ./make.bash amd64 HEAD 10G 35 | ``` 36 | 37 | to install a 64-bit version of NetBSD-current. 38 | 39 | ## How to use the created image (i.e. how to get started on GCE) 40 | 41 | The how-to below describes how to do the required operations in a web browser. 42 | You can also use the [Google Cloud SDK](https://cloud.google.com/sdk/) and its 43 | `gcloud` command line tool. 44 | 45 | 1. Run `make.bash` as described above. 46 | 2. Go to https://cloud.google.com/. Log in with your Google account or 47 | create a new one. If you never used Google Cloud Platform before, you will 48 | need to enter a credit card for billing. Yes, running stuff on GCP costs 49 | money. 50 | 3. Create a new Cloud project. A project is a collection of resources, such as 51 | VMs, storage, logs, etc. 52 | 4. In the left hand menu, click "Storage" and create a new bucket. 53 | 5. Click on the bucket, then click the "Upload files" button and select the 54 | output file that was created in step 1 (e.g. `netbsd-amd64-gce.tar.gz`). 55 | 6. In the left hand menu, select "Compute Engine", then "Images". Click "Create 56 | Image", choose a name, and select "Cloud Storage file" as the source. Browse 57 | to the file you just uploaded. 58 | 7. Select "Instances" from the left hand menu, then "Create Instance". Select a 59 | zone and the amount of CPU and memory. Under the "Boot Disk" heading, click 60 | "Change", select "Custom images" and choose the image you just created. 61 | Click the "Create" button at the bottom. 62 | 8. You will be transported back to the list of instances, where the instance 63 | you created is just starting up. Congratulations! To see the console output, 64 | click on the instance name, scroll down and click on "View serial port". 65 | 66 | ## Using the interactive serial console 67 | 68 | You will soon notice that you cannot use the SSH button to connect to the VM. 69 | Unfortunately, transferring of SSH keys to the machine does not work yet. To 70 | connect to the instance now, you can use the interactive serial console. Click 71 | "Edit" on the instance details page, scroll all the way to the bottom and tick 72 | the "Enable connecting to serial ports" box. 73 | 74 | Now you can click the button labeled "Connect to serial port". You will get a 75 | terminal window in the browser. Log in as `root` with no password. (This is the 76 | first thing you should change!) 77 | 78 | Now you can create user accounts and copy SSH keys as you wish. For example, to 79 | create a user named `myuser`, use the following commands: 80 | 81 | ``` 82 | useradd -m myuser 83 | passwd myuser 84 | ``` 85 | 86 | To enable the SSH daemon, use the following commands: 87 | 88 | ``` 89 | echo sshd=YES >> /etc/rc.conf 90 | /etc/rc.d/sshd start 91 | ``` 92 | 93 | Once sshd is running, you should be able to connect to the instance by pointing 94 | your ssh client to the instance's external IP address. 95 | --------------------------------------------------------------------------------