├── .github └── workflows │ └── go.yml ├── .gitignore ├── LICENSE ├── README.md ├── config.go ├── config_test.go ├── docs ├── container.md ├── environment.md └── tips.md ├── file.go ├── file_test.go ├── go.mod ├── go.sum ├── image.go ├── image_test.go ├── init.go ├── init_test.go ├── logo.png ├── main.go ├── main_test.go ├── proc.go ├── qemu.go ├── qemu_test.go ├── rpc.go ├── static_cgo.go ├── static_nocgo.go └── testdata ├── coredumps.txt ├── default.toml ├── gdb.txt ├── go-test.txt ├── help.txt ├── kvm.txt ├── lifecycle.txt ├── oci.txt ├── overlay.txt └── vimto.txt /.github/workflows/go.yml: -------------------------------------------------------------------------------- 1 | name: Go 2 | 3 | on: 4 | push: 5 | branches: [ "main" ] 6 | pull_request: 7 | branches: [ "main" ] 8 | 9 | concurrency: 10 | group: ${{ github.workflow }}-${{ github.ref }} 11 | cancel-in-progress: true 12 | 13 | jobs: 14 | amd64: 15 | runs-on: buildjet-2vcpu-ubuntu-2204 16 | strategy: 17 | matrix: 18 | kernel-version: [ "stable", "4.9" ] 19 | env: 20 | CI_KERNEL: ghcr.io/cilium/ci-kernels:${{ matrix.kernel-version }} 21 | 22 | steps: 23 | - uses: actions/checkout@v3 24 | 25 | - name: Set up Go 26 | uses: actions/setup-go@v4 27 | with: 28 | go-version-file: 'go.mod' 29 | 30 | - name: Install dependencies 31 | run: | 32 | sudo apt-get update 33 | sudo apt-get install -y --no-install-recommends gdb qemu-system-x86 34 | 35 | - run: sudo chmod 0666 /dev/kvm 36 | 37 | - name: Build 38 | run: go build -v ./... 39 | 40 | - name: Test 41 | run: go test -v ./... 42 | 43 | arm64: 44 | runs-on: buildjet-2vcpu-ubuntu-2204-arm 45 | strategy: 46 | matrix: 47 | kernel-version: [ "stable" ] 48 | env: 49 | CI_KERNEL: ghcr.io/cilium/ci-kernels:${{ matrix.kernel-version }} 50 | VIMTO_DISABLE_KVM: "true" 51 | 52 | steps: 53 | - uses: actions/checkout@v3 54 | 55 | - name: Set up Go 56 | uses: actions/setup-go@v4 57 | with: 58 | go-version-file: 'go.mod' 59 | 60 | - name: Install dependencies 61 | run: | 62 | sudo apt-get update 63 | sudo apt-get install -y --no-install-recommends gdb qemu-system-aarch64 64 | 65 | - name: Build 66 | run: go build -v ./... 67 | 68 | - name: Test 69 | run: go test -v ./... 70 | 71 | results: 72 | if: ${{ always() }} 73 | runs-on: ubuntu-latest 74 | name: Results 75 | needs: 76 | - amd64 77 | - arm64 78 | steps: 79 | - run: exit 1 80 | if: >- 81 | ${{ 82 | contains(needs.*.result, 'failure') 83 | || contains(needs.*.result, 'cancelled') 84 | }} -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | 2 | # If you prefer the allow list template instead of the deny list, see community template: 3 | # https://github.com/github/gitignore/blob/main/community/Golang/Go.AllowList.gitignore 4 | # 5 | # Binaries for programs and plugins 6 | *.exe 7 | *.exe~ 8 | *.dll 9 | *.so 10 | *.dylib 11 | /vimto 12 | 13 | # Test binary, built with `go test -c` 14 | *.test 15 | 16 | # Output of the go coverage tool, specifically when used with LiteIDE 17 | *.out 18 | 19 | # Dependency directories (remove the comment below to include it) 20 | # vendor/ 21 | 22 | # Go workspace file 23 | go.work 24 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright © 2023, 2024 Lorenz Bauer 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: 4 | 5 | The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. 6 | 7 | THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 8 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # vimto 2 | 3 | ![vimto logo](logo.png "vimto: virtual machine testing orchestrator") 4 | 5 | `vimto` is a virtual machine testing orchestrator for the Go toolchain. It allows you to easily run Go unit tests using a specific Linux kernel. 6 | 7 | ```shell 8 | # With .vimto.toml in place: 9 | vimto -- go test . 10 | # Otherwise: 11 | vimto -kernel /path/to/vmlinuz -- go test . 12 | ``` 13 | 14 | The tests are executed inside an ephemeral VM, with an [execution environment](docs/environment.md) which mimics the host. 15 | 16 | It's possible to obtain the kernel from a container image. 17 | 18 | ```shell 19 | vimto -kernel example.org/reg/image:tag -- go test . 20 | ``` 21 | 22 | Finally, you can also use a path to a directory: 23 | 24 | ```shell 25 | vimto -kernel /path/to/dir -- go test . 26 | ``` 27 | 28 | `vimto` expects the kernel to be at `/boot/vmlinuz` for containers and directories. 29 | See also [Container format](docs/container.md). 30 | 31 | ## Installation 32 | 33 | Install using the Go toolchain: 34 | 35 | ```shell 36 | CGO_ENABLED=0 go install lmb.io/vimto@latest 37 | ``` 38 | 39 | ## Configuration 40 | 41 | `vimto` reads a configuration file `.vimto.toml` in [TOML] format, either from the current directory or from the root of the git repository enclosing the current directory. 42 | 43 | All available options and their values are in [testdata/default.toml](./testdata/default.toml). 44 | 45 | ## Currently not supported 46 | 47 | * Networking 48 | * Interactive shell sessions 49 | * Cross-arch tests (running `arm64` on `amd64` host for example) 50 | 51 | ## Requirements 52 | 53 | * An `amd64` or `arm64` host running Linux 54 | * A recent version of `qemu` (8.1.3 is known to work) 55 | * A Linux kernel with the necessary configuration (>= 4.9 is known to work) 56 | * KVM (optional, see [VIMTO_DISABLE_KVM](docs/tips.md)) 57 | 58 | Here is a non-exhaustive list of required Linux options: 59 | 60 | * `CONFIG_9P_FS=y` 61 | * `CONFIG_DEVTMPFS=y` 62 | * `CONFIG_NET_9P_VIRTIO=y` 63 | * `CONFIG_NET_9P=y` 64 | * `CONFIG_NET_CORE=y` 65 | * `CONFIG_OVERLAY_FS=y` 66 | * `CONFIG_PCI=y` 67 | * `CONFIG_SYSFS=y` 68 | * `CONFIG_TMPFS=y` 69 | * `CONFIG_TTY=y` 70 | * `CONFIG_UNIX=y` 71 | * `CONFIG_UNIX98_PTYS=y` 72 | * `CONFIG_VIRTIO_CONSOLE=y` 73 | * `CONFIG_VIRTIO_MMIO=y` 74 | * `CONFIG_VIRTIO_NET=y` 75 | * `CONFIG_VIRTIO_PCI=y` 76 | * `CONFIG_VIRTIO=y` 77 | * `CONFIG_VT=y` 78 | 79 | [TOML]: https://toml.io/en/v1.0.0 80 | -------------------------------------------------------------------------------- /config.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "bytes" 5 | "errors" 6 | "flag" 7 | "fmt" 8 | "os" 9 | "os/exec" 10 | "path/filepath" 11 | "strings" 12 | 13 | "github.com/BurntSushi/toml" 14 | "github.com/kballard/go-shellquote" 15 | ) 16 | 17 | type config struct { 18 | Kernel string `toml:"kernel"` 19 | Memory string `toml:"memory"` 20 | SMP string `toml:"smp"` 21 | User string `toml:"user"` 22 | GDB string `toml:"-"` 23 | Setup []configCommand `toml:"setup"` 24 | Teardown []configCommand `toml:"teardown"` 25 | } 26 | 27 | type configCommand []string 28 | 29 | func (cc *configCommand) UnmarshalText(text []byte) error { 30 | words, err := shellquote.Split(string(text)) 31 | if err != nil { 32 | return err 33 | } 34 | 35 | *cc = configCommand(words) 36 | return nil 37 | } 38 | 39 | func (cc *configCommand) MarshalText() ([]byte, error) { 40 | return []byte(shellquote.Join(*cc...)), nil 41 | } 42 | 43 | var defaultConfig = &config{ 44 | Memory: "size=128M", 45 | SMP: "cpus=1", 46 | Setup: []configCommand{}, 47 | Teardown: []configCommand{}, 48 | } 49 | 50 | const configFileName = ".vimto.toml" 51 | 52 | var errUnrecognisedKeys = errors.New("unrecognised key(s)") 53 | 54 | func parseConfigFromTOML(dir string, cfg *config) error { 55 | f, err := findConfigFile(dir) 56 | if errors.Is(err, os.ErrNotExist) { 57 | return nil 58 | } 59 | if err != nil { 60 | return nil 61 | } 62 | defer f.Close() 63 | 64 | md, err := toml.NewDecoder(f).Decode(cfg) 65 | if err != nil { 66 | return fmt.Errorf("read %q: %w", f.Name(), err) 67 | } 68 | 69 | if len(md.Undecoded()) == 0 { 70 | return nil 71 | } 72 | 73 | var keys []string 74 | for _, key := range md.Undecoded() { 75 | keys = append(keys, strings.Join(key, ".")) 76 | } 77 | 78 | return fmt.Errorf("%q: %w: %s", f.Name(), errUnrecognisedKeys, strings.Join(keys, ", ")) 79 | } 80 | 81 | func configFlags(name string, cfg *config) *flag.FlagSet { 82 | fs := flag.NewFlagSet(name, flag.ContinueOnError) 83 | fs.Func("kernel", "`path or url` to the Linux image (use ':tag' to substitute tag in url)", func(s string) error { 84 | if !strings.HasPrefix(s, ":") { 85 | cfg.Kernel = s 86 | return nil 87 | } 88 | 89 | tag := s[1:] 90 | if strings.Contains(tag, ":") { 91 | return fmt.Errorf("tag %q contains colons", tag) 92 | } 93 | image, _, found := strings.Cut(cfg.Kernel, ":") 94 | if !found { 95 | return fmt.Errorf("no tag in image %q (missing colon)", cfg.Kernel) 96 | } 97 | 98 | cfg.Kernel = fmt.Sprintf("%s:%s", image, tag) 99 | return nil 100 | }) 101 | fs.Func("memory", "memory to give to the VM", func(s string) error { 102 | cfg.Memory = s 103 | return nil 104 | }) 105 | fs.Func("smp", "", func(s string) error { 106 | cfg.SMP = s 107 | return nil 108 | }) 109 | fs.BoolFunc("sudo", "execute as root", func(s string) error { 110 | if s != "true" { 111 | return errors.New("flag only accepts true") 112 | } 113 | 114 | cfg.User = "root" 115 | return nil 116 | }) 117 | fs.BoolFunc("gdb", "enable GDB server", func(s string) error { 118 | if s != "true" { 119 | return errors.New("flag only accepts true") 120 | } 121 | 122 | cfg.GDB = "localhost:1234" 123 | return nil 124 | }) 125 | 126 | return fs 127 | } 128 | 129 | func findConfigFile(dir string) (*os.File, error) { 130 | dirs := []string{dir} 131 | root, err := findGitRoot(dir) 132 | if err != nil { 133 | return nil, err 134 | } 135 | if root != "" { 136 | dirs = append(dirs, root) 137 | } 138 | 139 | for _, dir := range dirs { 140 | f, err := os.Open(filepath.Join(dir, configFileName)) 141 | if err == nil { 142 | return f, nil 143 | } 144 | if errors.Is(err, os.ErrNotExist) { 145 | continue 146 | } 147 | return nil, err 148 | } 149 | 150 | return nil, os.ErrNotExist 151 | } 152 | 153 | func findGitRoot(dir string) (string, error) { 154 | git := exec.Command("git", "rev-parse", "--show-toplevel") 155 | git.Dir = dir 156 | output, err := git.CombinedOutput() 157 | if errors.Is(err, exec.ErrNotFound) { 158 | return "", nil 159 | } 160 | 161 | var exitErr *exec.ExitError 162 | if errors.As(err, &exitErr) { 163 | // Not a git directory 164 | return "", nil 165 | } 166 | 167 | if err != nil { 168 | return "", err 169 | } 170 | 171 | path := string(bytes.TrimSpace(output)) 172 | if !filepath.IsAbs(path) { 173 | path = filepath.Join(dir, path) 174 | } 175 | 176 | return path, nil 177 | } 178 | -------------------------------------------------------------------------------- /config_test.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "io" 5 | "os" 6 | "os/exec" 7 | "path/filepath" 8 | "testing" 9 | 10 | "github.com/BurntSushi/toml" 11 | "github.com/go-quicktest/qt" 12 | ) 13 | 14 | func TestParseConfigFromTOML(t *testing.T) { 15 | tmp := t.TempDir() 16 | 17 | want := &config{"A", "B", "C", "", "", nil, nil} 18 | 19 | have := *want 20 | qt.Assert(t, qt.IsNil(parseConfigFromTOML(tmp, &have))) 21 | qt.Assert(t, qt.DeepEquals(&have, want), 22 | qt.Commentf("config shouldn't change if file doesn't exist")) 23 | 24 | toml := []byte(` 25 | kernel = "foo" 26 | memory = "bar" 27 | `) 28 | 29 | err := os.WriteFile(filepath.Join(tmp, configFileName), toml, 0644) 30 | qt.Assert(t, qt.IsNil(err)) 31 | 32 | qt.Assert(t, qt.IsNil(parseConfigFromTOML(tmp, &have))) 33 | qt.Assert(t, qt.DeepEquals(&have, &config{ 34 | Kernel: "foo", Memory: "bar", SMP: want.SMP, 35 | })) 36 | } 37 | 38 | func TestRefuseExtraneousConfig(t *testing.T) { 39 | tmp := t.TempDir() 40 | 41 | mustWriteConfig(t, tmp, `bazbar = 1`) 42 | qt.Assert(t, qt.ErrorIs(parseConfigFromTOML(tmp, &config{}), errUnrecognisedKeys)) 43 | } 44 | 45 | func TestFindGitRoot(t *testing.T) { 46 | root, err := findGitRoot(t.TempDir()) 47 | qt.Assert(t, qt.IsNil(err)) 48 | qt.Assert(t, qt.Equals(root, "")) 49 | 50 | wd, err := os.Getwd() 51 | qt.Assert(t, qt.IsNil(err)) 52 | 53 | root, err = findGitRoot(".") 54 | qt.Assert(t, qt.IsNil(err)) 55 | qt.Assert(t, qt.Equals(root, wd)) 56 | } 57 | 58 | func TestFindConfigFile(t *testing.T) { 59 | root := t.TempDir() 60 | output, err := exec.Command("git", "-C", root, "init").CombinedOutput() 61 | qt.Assert(t, qt.IsNil(err), qt.Commentf("output: %s", string(output))) 62 | 63 | _, err = findConfigFile(root) 64 | qt.Assert(t, qt.ErrorIs(err, os.ErrNotExist)) 65 | 66 | subdir := filepath.Join(root, "subdir") 67 | qt.Assert(t, qt.IsNil(os.Mkdir(subdir, 0755))) 68 | 69 | rootCfg := mustWriteConfig(t, root, `foo=1`) 70 | f, err := findConfigFile(subdir) 71 | qt.Assert(t, qt.IsNil(err)) 72 | qt.Assert(t, qt.Equals(f.Name(), rootCfg)) 73 | 74 | subdirCfg := mustWriteConfig(t, subdir, `foo=2`) 75 | f, err = findConfigFile(subdir) 76 | qt.Assert(t, qt.IsNil(err)) 77 | f.Close() 78 | qt.Assert(t, qt.Equals(f.Name(), subdirCfg)) 79 | 80 | f, err = findConfigFile(root) 81 | qt.Assert(t, qt.IsNil(err)) 82 | f.Close() 83 | qt.Assert(t, qt.Equals(f.Name(), rootCfg)) 84 | } 85 | 86 | func TestWriteConfig(t *testing.T) { 87 | f, err := os.Create("testdata/default.toml") 88 | qt.Assert(t, qt.IsNil(err)) 89 | defer f.Close() 90 | qt.Assert(t, qt.IsNil(toml.NewEncoder(f).Encode(defaultConfig))) 91 | } 92 | 93 | func TestConfigFlags(t *testing.T) { 94 | var cfg config 95 | fs := configFlags("test", &cfg) 96 | fs.SetOutput(io.Discard) 97 | 98 | err := fs.Parse([]string{"-kernel", ":foo"}) 99 | qt.Assert(t, qt.IsNotNil(err)) 100 | 101 | cfg.Kernel = "example.org/image:bar" 102 | err = fs.Parse([]string{"-kernel", ":foo"}) 103 | qt.Assert(t, qt.IsNil(err)) 104 | qt.Assert(t, qt.Equals(cfg.Kernel, "example.org/image:foo")) 105 | } 106 | 107 | func mustWriteConfig(tb testing.TB, dir, contents string) string { 108 | tb.Helper() 109 | filename := filepath.Join(dir, configFileName) 110 | qt.Assert(tb, qt.IsNil(os.WriteFile(filename, []byte(contents), 0644))) 111 | return filename 112 | } 113 | -------------------------------------------------------------------------------- /docs/container.md: -------------------------------------------------------------------------------- 1 | # Container format 2 | 3 | The container must contain one of these files in order to boot the VM. 4 | 5 | 1. `/boot/vmlinux` 6 | 2. `/boot/vmlinuz` 7 | 8 | The first existing file is used. 9 | 10 | Other files and directories in the container are merged with the host filesystem 11 | using an overlayfs mount inside the VM. 12 | 13 | ## Error: directory /lib: shadows symlink on host 14 | 15 | This error is generated if the image contains a directory that would shadow 16 | important directories in the host: 17 | 18 | * /lib 19 | * /lib64 20 | * /bin 21 | * /sbin 22 | 23 | This happens when running on distributions that have completed a /usr merge. In 24 | this case these directories are symlinks on the host. Overlaying a directory from 25 | the image will make the symlink disappear. 26 | 27 | To work around the issue, place files in `/usr/lib`, ... and include your own 28 | `/lib -> /usr/lib` symlink in the image. 29 | -------------------------------------------------------------------------------- /docs/environment.md: -------------------------------------------------------------------------------- 1 | # Execution environment 2 | 3 | When developing new code for cilium/ebpf I usually iterate as follows: 4 | 5 | 1. Run tests using `go test -exec sudo` on the host kernel. 6 | Step through code using dlv (only works for some tests, sudo is cumbersome). 7 | 2. Run tests using `go test -exec vimto` on a specific kernel. This is mostly to 8 | mimic CI without having to push to GitHub. 9 | 3. Very rarely, debug using dlv or gdb using a specific kernel. This is to catch 10 | those pesky kernel bugs. 11 | 12 | vimto should therefore support the following tasks with minimal setup or interaction 13 | required by the user: 14 | 15 | - Execute a set of Go unit tests against a pre-compiled kernel. 16 | - Debugging Go unit tests with delve using a pre-comiled kernel. 17 | - Execute Go unit tests using a kernel which is debugged by gdb. 18 | 19 | The execution environment should be as close as possible to using `-exec sudo` 20 | so that the "ladder of iteration" doesn't require much context switching. 21 | 22 | - The host operating system already has all the necessary dependencies for unit tests. 23 | - Debug tooling like gdb and delve are taken from the host filesystem. 24 | - Kernel and modules are retrieved from an OCI image. 25 | 26 | ## Filesystems 27 | 28 | The root filesystem is a merge of the following host paths. 29 | 30 | | Path | Permissions | Comment | 31 | |-------------------|-------------|-----------------------------------| 32 | | / | r/o | Root of the host operating system | 33 | | /boot, /usr, /lib | r/o | Kernel and modules from OCI image | 34 | 35 | The root filesystem inside the VM is writable, but changes are ephemeral and not 36 | carried over to the host. This is because the root fs is shared by multiple VMs. 37 | Some system relevant filesystems like /sys, /proc and so on are private to the VM. 38 | 39 | The following paths are shared between the host and the VM: 40 | 41 | | Path | Comment | 42 | |----------------------|--------------------------------------------------| 43 | | /tmp/path/to/workdir | Temporary files used by the Go toolchain (-work) | 44 | | /path/to/repository | Root of the VCS repository | 45 | 46 | ## User and group 47 | 48 | By default, commands execute as the current user and group. This works because 49 | /etc is shared between the host and the VM. 50 | 51 | It's possible to opt into running a command inside the VM as if it was invoked 52 | via `sudo --preserve-env`. This is not straight up sudo since the user might not 53 | have sudo privileges on the host. 54 | 55 | ## Environment variables 56 | 57 | All variables from the environment are copied verbatim into the VM. PATH from 58 | outside the VM does take effect when looking up binaries. 59 | 60 | ## Working directory 61 | 62 | The working directory is preserved in the VM. 63 | -------------------------------------------------------------------------------- /docs/tips.md: -------------------------------------------------------------------------------- 1 | # Tips 2 | 3 | ## Disable KVM 4 | 5 | You can disable KVM by setting an environment variable: 6 | 7 | ``` 8 | VIMTO_DISABLE_KVM=true vimto ... 9 | ``` 10 | 11 | This can be useful on hosts where nested virtualisation is not available. It 12 | will be rather slow though. 13 | 14 | ## Enable Go coredumps 15 | 16 | Sometimes a test inside the VM crashes or panics and debugging may be difficult. 17 | In that case you can enable collecting a core dump like so: 18 | 19 | ``` 20 | GOTRACEBACK=crash vimto -- go test ... 21 | ``` 22 | 23 | [`GOTRACEBACK`](https://pkg.go.dev/runtime) is interpreted by the Go runtime. 24 | `vimto` will preserve the test binaries if it detects a core dump. This allows 25 | you to collect the binary and the core dump in CI for later debugging. 26 | 27 | ## Debug the kernel using GDB 28 | 29 | You can debug the kernel by passing the `-gdb` flag: 30 | 31 | ``` 32 | $ go test -c . 33 | $ vimto -gdb -kernel :4.9 exec -- pkg.test 34 | Starting GDB server with CPU halted, connect using: 35 | gdb -ex 'target remote localhost:1234' -ex '[...]' 36 | ``` 37 | 38 | This works best if the image you are using contains an uncompressed `vmlinux` 39 | which includes debug symbols. 40 | -------------------------------------------------------------------------------- /file.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "errors" 5 | "fmt" 6 | "os" 7 | "runtime" 8 | "syscall" 9 | 10 | "golang.org/x/sys/unix" 11 | ) 12 | 13 | func replaceFdWithFile(sys syscaller, fd int, file *os.File) error { 14 | _, err := fileControl(file, func(replacement uintptr) (struct{}, error) { 15 | // dup2 overwrites fd with the newly opened file. 16 | return struct{}{}, sys.dup2(int(replacement), fd) 17 | }) 18 | if err != nil { 19 | return fmt.Errorf("dup2: %w", err) 20 | } 21 | return nil 22 | } 23 | 24 | func flock(f *os.File, how int) error { 25 | _, err := fileControl(f, func(fd uintptr) (struct{}, error) { 26 | return struct{}{}, unix.Flock(int(fd), how) 27 | }) 28 | return err 29 | } 30 | 31 | func unixSocketpair() (*os.File, *os.File, error) { 32 | fds, err := unix.Socketpair(unix.AF_UNIX, unix.SOCK_STREAM, 0) 33 | if err != nil { 34 | return nil, nil, fmt.Errorf("create unix socket pair: %w", err) 35 | } 36 | 37 | return os.NewFile(uintptr(fds[0]), ""), os.NewFile(uintptr(fds[1]), ""), nil 38 | } 39 | 40 | func fileIsDevZero(f *os.File) (bool, error) { 41 | info, err := f.Stat() 42 | if err != nil { 43 | return false, err 44 | } 45 | 46 | fStat, ok := info.Sys().(*syscall.Stat_t) 47 | if !ok { 48 | return false, fmt.Errorf("GOOS not supported: %s", runtime.GOOS) 49 | } 50 | 51 | nullInfo, err := os.Stat(os.DevNull) 52 | if err != nil { 53 | return false, err 54 | } 55 | 56 | nullStat, ok := nullInfo.Sys().(*syscall.Stat_t) 57 | if !ok { 58 | return false, fmt.Errorf("GOOS not supported: %s", runtime.GOOS) 59 | } 60 | 61 | return fStat.Rdev == nullStat.Rdev, nil 62 | } 63 | 64 | func fileIsTTY(f *os.File) (bool, error) { 65 | return fileControl(f, func(fd uintptr) (bool, error) { 66 | _, err := unix.IoctlGetTermios(int(fd), unix.TCGETS) 67 | if err != nil && !errors.Is(err, unix.ENOTTY) { 68 | return false, err 69 | } 70 | return err == nil, nil 71 | }) 72 | } 73 | 74 | func fileControl[T any](f *os.File, fn func(fd uintptr) (T, error)) (T, error) { 75 | var result T 76 | sys, err := f.SyscallConn() 77 | if err != nil { 78 | return result, err 79 | } 80 | 81 | var opErr error 82 | err = sys.Control(func(fd uintptr) { 83 | result, opErr = fn(fd) 84 | }) 85 | if err != nil { 86 | return result, fmt.Errorf("control fd: %w", err) 87 | } 88 | if opErr != nil { 89 | return result, opErr 90 | } 91 | 92 | return result, nil 93 | } 94 | -------------------------------------------------------------------------------- /file_test.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "errors" 5 | "os" 6 | "testing" 7 | 8 | "github.com/creack/pty/v2" 9 | "github.com/go-quicktest/qt" 10 | "golang.org/x/sys/unix" 11 | ) 12 | 13 | func TestFileControl(t *testing.T) { 14 | f, err := os.CreateTemp("", "") 15 | qt.Assert(t, qt.IsNil(err)) 16 | defer f.Close() 17 | defer os.Remove(f.Name()) 18 | 19 | sentinel := errors.New("sentinel") 20 | 21 | _, err = fileControl(f, func(fd uintptr) (struct{}, error) { 22 | return struct{}{}, sentinel 23 | }) 24 | 25 | qt.Assert(t, qt.ErrorIs(err, sentinel)) 26 | } 27 | 28 | func TestFlock(t *testing.T) { 29 | tmp := t.TempDir() 30 | f1, err := os.Open(tmp) 31 | qt.Assert(t, qt.IsNil(err)) 32 | defer f1.Close() 33 | 34 | f2, err := os.Open(tmp) 35 | qt.Assert(t, qt.IsNil(err)) 36 | defer f2.Close() 37 | 38 | qt.Assert(t, qt.IsNil(flock(f1, unix.LOCK_SH))) 39 | qt.Assert(t, qt.ErrorIs(flock(f2, unix.LOCK_EX|unix.LOCK_NB), unix.EWOULDBLOCK)) 40 | 41 | f1.Close() 42 | qt.Assert(t, qt.IsNil(flock(f2, unix.LOCK_EX|unix.LOCK_NB))) 43 | } 44 | 45 | func TestFileIsTTY(t *testing.T) { 46 | pty, tty, err := pty.Open() 47 | qt.Assert(t, qt.IsNil(err)) 48 | defer pty.Close() 49 | defer tty.Close() 50 | 51 | ok, err := fileIsTTY(pty) 52 | qt.Assert(t, qt.IsNil(err)) 53 | qt.Assert(t, qt.Equals(ok, true)) 54 | 55 | ok, err = fileIsTTY(tty) 56 | qt.Assert(t, qt.IsNil(err)) 57 | qt.Assert(t, qt.Equals(ok, true)) 58 | 59 | dir, err := os.Open(t.TempDir()) 60 | qt.Assert(t, qt.IsNil(err)) 61 | defer dir.Close() 62 | 63 | ok, err = fileIsTTY(dir) 64 | qt.Assert(t, qt.IsNil(err)) 65 | qt.Assert(t, qt.Equals(ok, false)) 66 | } 67 | -------------------------------------------------------------------------------- /go.mod: -------------------------------------------------------------------------------- 1 | module lmb.io/vimto 2 | 3 | go 1.23.0 4 | 5 | require ( 6 | github.com/BurntSushi/toml v1.3.2 7 | github.com/creack/pty/v2 v2.0.1 8 | github.com/docker/docker v27.5.0+incompatible 9 | github.com/go-quicktest/qt v1.101.0 10 | github.com/google/go-containerregistry v0.20.3 11 | github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 12 | github.com/schollz/progressbar/v3 v3.18.0 13 | github.com/u-root/u-root v0.11.0 14 | golang.org/x/sync v0.10.0 15 | golang.org/x/sys v0.29.0 16 | rsc.io/script v0.0.2-0.20231205190631-334f6c18cff3 17 | ) 18 | 19 | require ( 20 | github.com/AdaLogics/go-fuzz-headers v0.0.0-20240806141605-e8a1dd7889d6 // indirect 21 | github.com/adrg/xdg v0.5.3 // indirect 22 | github.com/containerd/log v0.1.0 // indirect 23 | github.com/containerd/stargz-snapshotter/estargz v0.16.3 // indirect 24 | github.com/docker/cli v27.5.0+incompatible // indirect 25 | github.com/docker/distribution v2.8.3+incompatible // indirect 26 | github.com/docker/docker-credential-helpers v0.8.2 // indirect 27 | github.com/google/go-cmp v0.6.0 // indirect 28 | github.com/google/goexpect v0.0.0-20191001010744-5b6988669ffa // indirect 29 | github.com/google/goterm v0.0.0-20200907032337-555d40f16ae2 // indirect 30 | github.com/klauspost/compress v1.17.11 // indirect 31 | github.com/kr/pretty v0.3.1 // indirect 32 | github.com/kr/text v0.2.0 // indirect 33 | github.com/mitchellh/colorstring v0.0.0-20190213212951-d06e56a500db // indirect 34 | github.com/mitchellh/go-homedir v1.1.0 // indirect 35 | github.com/moby/patternmatcher v0.6.0 // indirect 36 | github.com/moby/sys/sequential v0.6.0 // indirect 37 | github.com/moby/sys/user v0.3.0 // indirect 38 | github.com/moby/sys/userns v0.1.0 // indirect 39 | github.com/opencontainers/go-digest v1.0.0 // indirect 40 | github.com/opencontainers/image-spec v1.1.0 // indirect 41 | github.com/pkg/errors v0.9.1 // indirect 42 | github.com/rivo/uniseg v0.4.7 // indirect 43 | github.com/rogpeppe/go-internal v1.13.1 // indirect 44 | github.com/sirupsen/logrus v1.9.3 // indirect 45 | github.com/stretchr/testify v1.10.0 // indirect 46 | github.com/vbatts/tar-split v0.11.6 // indirect 47 | golang.org/x/crypto v0.0.0-20210921155107-089bfa567519 // indirect 48 | golang.org/x/term v0.28.0 // indirect 49 | golang.org/x/tools v0.29.0 // indirect 50 | google.golang.org/genproto/googleapis/rpc v0.0.0-20250102185135-69823020774d // indirect 51 | google.golang.org/grpc v1.69.4 // indirect 52 | gotest.tools/v3 v3.5.1 // indirect 53 | ) 54 | -------------------------------------------------------------------------------- /go.sum: -------------------------------------------------------------------------------- 1 | github.com/AdaLogics/go-fuzz-headers v0.0.0-20240806141605-e8a1dd7889d6 h1:He8afgbRMd7mFxO99hRNu+6tazq8nFF9lIwo9JFroBk= 2 | github.com/AdaLogics/go-fuzz-headers v0.0.0-20240806141605-e8a1dd7889d6/go.mod h1:8o94RPi1/7XTJvwPpRSzSUedZrtlirdB3r9Z20bi2f8= 3 | github.com/BurntSushi/toml v1.3.2 h1:o7IhLm0Msx3BaB+n3Ag7L8EVlByGnpq14C4YWiu/gL8= 4 | github.com/BurntSushi/toml v1.3.2/go.mod h1:CxXYINrC8qIiEnFrOxCa7Jy5BFHlXnUU2pbicEuybxQ= 5 | github.com/adrg/xdg v0.5.3 h1:xRnxJXne7+oWDatRhR1JLnvuccuIeCoBu2rtuLqQB78= 6 | github.com/adrg/xdg v0.5.3/go.mod h1:nlTsY+NNiCBGCK2tpm09vRqfVzrc2fLmXGpBLF0zlTQ= 7 | github.com/chengxilo/virtualterm v1.0.4 h1:Z6IpERbRVlfB8WkOmtbHiDbBANU7cimRIof7mk9/PwM= 8 | github.com/chengxilo/virtualterm v1.0.4/go.mod h1:DyxxBZz/x1iqJjFxTFcr6/x+jSpqN0iwWCOK1q10rlY= 9 | github.com/containerd/log v0.1.0 h1:TCJt7ioM2cr/tfR8GPbGf9/VRAX8D2B4PjzCpfX540I= 10 | github.com/containerd/log v0.1.0/go.mod h1:VRRf09a7mHDIRezVKTRCrOq78v577GXq3bSa3EhrzVo= 11 | github.com/containerd/stargz-snapshotter/estargz v0.16.3 h1:7evrXtoh1mSbGj/pfRccTampEyKpjpOnS3CyiV1Ebr8= 12 | github.com/containerd/stargz-snapshotter/estargz v0.16.3/go.mod h1:uyr4BfYfOj3G9WBVE8cOlQmXAbPN9VEQpBBeJIuOipU= 13 | github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= 14 | github.com/creack/pty/v2 v2.0.1 h1:RDY1VY5b+7m2mfPsugucOYPIxMp+xal5ZheSyVzUA+k= 15 | github.com/creack/pty/v2 v2.0.1/go.mod h1:2dSssKp3b86qYEMwA/FPwc3ff+kYpDdQI8osU8J7gxQ= 16 | github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= 17 | github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= 18 | github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= 19 | github.com/docker/cli v27.5.0+incompatible h1:aMphQkcGtpHixwwhAXJT1rrK/detk2JIvDaFkLctbGM= 20 | github.com/docker/cli v27.5.0+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8= 21 | github.com/docker/distribution v2.8.3+incompatible h1:AtKxIZ36LoNK51+Z6RpzLpddBirtxJnzDrHLEKxTAYk= 22 | github.com/docker/distribution v2.8.3+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w= 23 | github.com/docker/docker v27.5.0+incompatible h1:um++2NcQtGRTz5eEgO6aJimo6/JxrTXC941hd05JO6U= 24 | github.com/docker/docker v27.5.0+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk= 25 | github.com/docker/docker-credential-helpers v0.8.2 h1:bX3YxiGzFP5sOXWc3bTPEXdEaZSeVMrFgOr3T+zrFAo= 26 | github.com/docker/docker-credential-helpers v0.8.2/go.mod h1:P3ci7E3lwkZg6XiHdRKft1KckHiO9a2rNtyFbZ/ry9M= 27 | github.com/go-quicktest/qt v1.101.0 h1:O1K29Txy5P2OK0dGo59b7b0LR6wKfIhttaAhHUyn7eI= 28 | github.com/go-quicktest/qt v1.101.0/go.mod h1:14Bz/f7NwaXPtdYEgzsx46kqSxVwTbzVZsDC26tQJow= 29 | github.com/golang/glog v1.2.2 h1:1+mZ9upx1Dh6FmUTFR1naJ77miKiXgALjWOZ3NVFPmY= 30 | github.com/golang/glog v1.2.2/go.mod h1:6AhwSGph0fcJtXVM/PEHPqZlFeoLxhs7/t5UDAwmO+w= 31 | github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI= 32 | github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= 33 | github.com/google/go-containerregistry v0.20.3 h1:oNx7IdTI936V8CQRveCjaxOiegWwvM7kqkbXTpyiovI= 34 | github.com/google/go-containerregistry v0.20.3/go.mod h1:w00pIgBRDVUDFM6bq+Qx8lwNWK+cxgCuX1vd3PIBDNI= 35 | github.com/google/goexpect v0.0.0-20191001010744-5b6988669ffa h1:PMkmJA8ju9DjqAJjIzrBdrmhuuPsoNnNLYgKQBopWL0= 36 | github.com/google/goexpect v0.0.0-20191001010744-5b6988669ffa/go.mod h1:qtE5aAEkt0vOSA84DBh8aJsz6riL8ONfqfULY7lBjqc= 37 | github.com/google/goterm v0.0.0-20200907032337-555d40f16ae2 h1:CVuJwN34x4xM2aT4sIKhmeib40NeBPhRihNjQmpJsA4= 38 | github.com/google/goterm v0.0.0-20200907032337-555d40f16ae2/go.mod h1:nOFQdrUlIlx6M6ODdSpBj1NVA+VgLC6kmw60mkw34H4= 39 | github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 h1:Z9n2FFNUXsshfwJMBgNA0RU6/i7WVaAegv3PtuIHPMs= 40 | github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51/go.mod h1:CzGEWj7cYgsdH8dAjBGEr58BoE7ScuLd+fwFZ44+/x8= 41 | github.com/klauspost/compress v1.17.11 h1:In6xLpyWOi1+C7tXUUWv2ot1QvBjxevKAaI6IXrJmUc= 42 | github.com/klauspost/compress v1.17.11/go.mod h1:pMDklpSncoRMuLFrf1W9Ss9KT+0rH90U12bZKk7uwG0= 43 | github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= 44 | github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= 45 | github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= 46 | github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= 47 | github.com/mattn/go-runewidth v0.0.16 h1:E5ScNMtiwvlvB5paMFdw9p4kSQzbXFikJ5SQO6TULQc= 48 | github.com/mattn/go-runewidth v0.0.16/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w= 49 | github.com/mitchellh/colorstring v0.0.0-20190213212951-d06e56a500db h1:62I3jR2EmQ4l5rM/4FEfDWcRD+abF5XlKShorW5LRoQ= 50 | github.com/mitchellh/colorstring v0.0.0-20190213212951-d06e56a500db/go.mod h1:l0dey0ia/Uv7NcFFVbCLtqEBQbrT4OCwCSKTEv6enCw= 51 | github.com/mitchellh/go-homedir v1.1.0 h1:lukF9ziXFxDFPkA1vsr5zpc1XuPDn/wFntq5mG+4E0Y= 52 | github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0= 53 | github.com/moby/patternmatcher v0.6.0 h1:GmP9lR19aU5GqSSFko+5pRqHi+Ohk1O69aFiKkVGiPk= 54 | github.com/moby/patternmatcher v0.6.0/go.mod h1:hDPoyOpDY7OrrMDLaYoY3hf52gNCR/YOUYxkhApJIxc= 55 | github.com/moby/sys/sequential v0.6.0 h1:qrx7XFUd/5DxtqcoH1h438hF5TmOvzC/lspjy7zgvCU= 56 | github.com/moby/sys/sequential v0.6.0/go.mod h1:uyv8EUTrca5PnDsdMGXhZe6CCe8U/UiTWd+lL+7b/Ko= 57 | github.com/moby/sys/user v0.3.0 h1:9ni5DlcW5an3SvRSx4MouotOygvzaXbaSrc/wGDFWPo= 58 | github.com/moby/sys/user v0.3.0/go.mod h1:bG+tYYYJgaMtRKgEmuueC0hJEAZWwtIbZTB+85uoHjs= 59 | github.com/moby/sys/userns v0.1.0 h1:tVLXkFOxVu9A64/yh59slHVv9ahO9UIev4JZusOLG/g= 60 | github.com/moby/sys/userns v0.1.0/go.mod h1:IHUYgu/kao6N8YZlp9Cf444ySSvCmDlmzUcYfDHOl28= 61 | github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U= 62 | github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM= 63 | github.com/opencontainers/image-spec v1.1.0 h1:8SG7/vwALn54lVB/0yZ/MMwhFrPYtpEHQb2IpWsCzug= 64 | github.com/opencontainers/image-spec v1.1.0/go.mod h1:W4s4sFTMaBeK1BQLXbG4AdM2szdn85PY75RI83NrTrM= 65 | github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA= 66 | github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4= 67 | github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= 68 | github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= 69 | github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= 70 | github.com/rivo/uniseg v0.4.7 h1:WUdvkW8uEhrYfLC4ZzdpI2ztxP1I582+49Oc5Mq64VQ= 71 | github.com/rivo/uniseg v0.4.7/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88= 72 | github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs= 73 | github.com/rogpeppe/go-internal v1.13.1 h1:KvO1DLK/DRN07sQ1LQKScxyZJuNnedQ5/wKSR38lUII= 74 | github.com/rogpeppe/go-internal v1.13.1/go.mod h1:uMEvuHeurkdAXX61udpOXGD/AzZDWNMNyH2VO9fmH0o= 75 | github.com/schollz/progressbar/v3 v3.18.0 h1:uXdoHABRFmNIjUfte/Ex7WtuyVslrw2wVPQmCN62HpA= 76 | github.com/schollz/progressbar/v3 v3.18.0/go.mod h1:IsO3lpbaGuzh8zIMzgY3+J8l4C8GjO0Y9S69eFvNsec= 77 | github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ= 78 | github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ= 79 | github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= 80 | github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= 81 | github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA= 82 | github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= 83 | github.com/u-root/u-root v0.11.0 h1:6gCZLOeRyevw7gbTwMj3fKxnr9+yHFlgF3N7udUVNO8= 84 | github.com/u-root/u-root v0.11.0/go.mod h1:DBkDtiZyONk9hzVEdB/PWI9B4TxDkElWlVTHseglrZY= 85 | github.com/vbatts/tar-split v0.11.6 h1:4SjTW5+PU11n6fZenf2IPoV8/tz3AaYHMWjf23envGs= 86 | github.com/vbatts/tar-split v0.11.6/go.mod h1:dqKNtesIOr2j2Qv3W/cHjnvk9I8+G7oAkFDFN6TCBEI= 87 | golang.org/x/crypto v0.0.0-20210921155107-089bfa567519 h1:7I4JAnoQBe7ZtJcBaYHi5UtiO8tQHbUSXxL+pnGRANg= 88 | golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc= 89 | golang.org/x/sync v0.10.0 h1:3NQrjDixjgGwUOCaF8w2+VYHv0Ve/vGYSbdkTa98gmQ= 90 | golang.org/x/sync v0.10.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= 91 | golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= 92 | golang.org/x/sys v0.29.0 h1:TPYlXGxvx1MGTn2GiZDhnjPA9wZzZeGKHHmKhHYvgaU= 93 | golang.org/x/sys v0.29.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= 94 | golang.org/x/term v0.28.0 h1:/Ts8HFuMR2E6IP/jlo7QVLZHggjKQbhu/7H0LJFr3Gg= 95 | golang.org/x/term v0.28.0/go.mod h1:Sw/lC2IAUZ92udQNf3WodGtn4k/XoLyZoh8v/8uiwek= 96 | golang.org/x/tools v0.29.0 h1:Xx0h3TtM9rzQpQuR4dKLrdglAmCEN5Oi+P74JdhdzXE= 97 | golang.org/x/tools v0.29.0/go.mod h1:KMQVMRsVxU6nHCFXrBPhDB8XncLNLM0lIy/F14RP588= 98 | google.golang.org/genproto/googleapis/rpc v0.0.0-20250102185135-69823020774d h1:xJJRGY7TJcvIlpSrN3K6LAWgNFUILlO+OMAqtg9aqnw= 99 | google.golang.org/genproto/googleapis/rpc v0.0.0-20250102185135-69823020774d/go.mod h1:3ENsm/5D1mzDyhpzeRi1NR784I0BcofWBoSc5QqqMK4= 100 | google.golang.org/grpc v1.69.4 h1:MF5TftSMkd8GLw/m0KM6V8CMOCY6NZ1NQDPGFgbTt4A= 101 | google.golang.org/grpc v1.69.4/go.mod h1:vyjdE6jLBI76dgpDojsFGNaHlxdjXN9ghpnd2o7JGZ4= 102 | google.golang.org/protobuf v1.36.3 h1:82DV7MYdb8anAVi3qge1wSnMDrnKK7ebr+I0hHRN1BU= 103 | google.golang.org/protobuf v1.36.3/go.mod h1:9fA7Ob0pmnwhb644+1+CVWFRbNajQ6iRojtC/QF5bRE= 104 | gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= 105 | gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= 106 | gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= 107 | gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= 108 | gotest.tools/v3 v3.5.1 h1:EENdUnS3pdur5nybKYIh2Vfgc8IUNBjxDPSjtiJcOzU= 109 | gotest.tools/v3 v3.5.1/go.mod h1:isy3WKz7GK6uNw/sbHzfKBLvlvXwUyV06n6brMxxopU= 110 | rsc.io/script v0.0.2-0.20231205190631-334f6c18cff3 h1:2vM6uMBq2/Dou/Wzu2p+yUFkuI3lgMbX0UYfVnzh0ck= 111 | rsc.io/script v0.0.2-0.20231205190631-334f6c18cff3/go.mod h1:cKBjCtFBBeZ0cbYFRXkRoxP+xGqhArPa9t3VWhtXfzU= 112 | -------------------------------------------------------------------------------- /image.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "context" 5 | "crypto/sha256" 6 | "errors" 7 | "fmt" 8 | "io" 9 | "os" 10 | "path/filepath" 11 | "runtime" 12 | "slices" 13 | "time" 14 | 15 | "github.com/adrg/xdg" 16 | "github.com/docker/docker/pkg/archive" 17 | "github.com/google/go-containerregistry/pkg/name" 18 | v1 "github.com/google/go-containerregistry/pkg/v1" 19 | "github.com/google/go-containerregistry/pkg/v1/mutate" 20 | "github.com/google/go-containerregistry/pkg/v1/remote" 21 | "github.com/schollz/progressbar/v3" 22 | "golang.org/x/sys/unix" 23 | ) 24 | 25 | // imageCache ensures that multiple invocations of vimto don't download the 26 | // same images over and over again. 27 | // 28 | // The main concern is startup speed of vimto. 29 | type imageCache struct { 30 | baseDir string 31 | } 32 | 33 | var userCacheDir = filepath.Join(xdg.CacheHome, "vimto") 34 | 35 | func newImageCache() (*imageCache, error) { 36 | if err := os.MkdirAll(userCacheDir, 0700); err != nil && !errors.Is(err, os.ErrExist) { 37 | return nil, fmt.Errorf("create cache directory: %w", err) 38 | } 39 | 40 | return &imageCache{userCacheDir}, nil 41 | } 42 | 43 | // Acquire an image from the cache. 44 | // 45 | // The image remains valid even after closing the cache. 46 | func (ic *imageCache) Acquire(ctx context.Context, refStr string, status io.Writer) (_ *image, err error) { 47 | ref, err := name.ParseReference(refStr) 48 | if err != nil { 49 | return nil, fmt.Errorf("parsing reference %q: %w", refStr, err) 50 | } 51 | 52 | // Use the sha256 of the canonical reference as the cache key. This means 53 | // that images / tags pointing at the blob will have separate cache entries. 54 | dir := fmt.Sprintf("%x", sha256.Sum256([]byte(ref.Name()))) 55 | 56 | lock, path, err := populateDirectoryOnce(filepath.Join(ic.baseDir, dir), func(path string) error { 57 | err := fetchImage(ctx, refStr, path, status) 58 | if err != nil { 59 | return fmt.Errorf("fetch image: %w", err) 60 | } 61 | 62 | return nil 63 | }) 64 | if err != nil { 65 | return nil, err 66 | } 67 | 68 | return &image{refStr, path, lock}, nil 69 | } 70 | 71 | func populateDirectoryOnce(path string, fn func(string) error) (lock *os.File, _ string, err error) { 72 | closeOnError := func(c io.Closer) { 73 | if err != nil { 74 | c.Close() 75 | } 76 | } 77 | 78 | if err := os.Mkdir(path, 0755); err != nil && !errors.Is(err, os.ErrExist) { 79 | return nil, "", fmt.Errorf("create cache directory: %w", err) 80 | } 81 | 82 | dir, err := os.Open(path) 83 | if err != nil { 84 | return nil, "", err 85 | } 86 | defer closeOnError(dir) 87 | 88 | if err := flock(dir, unix.LOCK_SH); err != nil { 89 | return nil, "", fmt.Errorf("lock %q: %w", dir.Name(), err) 90 | } 91 | 92 | contents := filepath.Join(dir.Name(), "contents") 93 | if _, err := os.Stat(contents); err == nil { 94 | // We have a cached copy of the image. 95 | return dir, contents, nil 96 | } 97 | 98 | // Need to extract the image, acquire exclusive lock. 99 | if err := flock(dir, unix.LOCK_EX); err != nil { 100 | return nil, "", fmt.Errorf("lock %q: %w", dir.Name(), err) 101 | } 102 | 103 | // Changing lock mode is not atomic, revalidate. 104 | if _, err := os.Stat(contents); err == nil { 105 | if err := flock(dir, unix.LOCK_SH); err != nil { 106 | return nil, "", fmt.Errorf("lock %q: %w", dir.Name(), err) 107 | } 108 | return dir, contents, nil 109 | } 110 | 111 | tmpdir, err := os.MkdirTemp(path, "") 112 | if err != nil { 113 | return nil, "", err 114 | } 115 | defer os.RemoveAll(tmpdir) 116 | 117 | if err := fn(tmpdir); err != nil { 118 | return nil, "", fmt.Errorf("populate %s: %w", contents, err) 119 | } 120 | 121 | if err := os.Rename(tmpdir, contents); err != nil { 122 | return nil, "", err 123 | } 124 | 125 | // Drop the exclusive lock. 126 | if err := flock(dir, unix.LOCK_SH); err != nil { 127 | return nil, "", fmt.Errorf("drop exclusive lock: %w", err) 128 | } 129 | 130 | return dir, contents, nil 131 | } 132 | 133 | type image struct { 134 | // The image name in OCIspeak, for example "example.com/foo:latest". 135 | Name string 136 | 137 | // Path to directory containing the contents of the image. 138 | Directory string 139 | 140 | // The directory file descriptor holding a cache lock. 141 | lock *os.File 142 | } 143 | 144 | func (img *image) Close() error { 145 | if img != nil { 146 | return img.lock.Close() 147 | } 148 | return nil 149 | } 150 | 151 | type bootFiles struct { 152 | // Path to the kernel to boot. 153 | Kernel string 154 | 155 | // Path to a directory to be overlaid over the root filesystem. Optional. 156 | Overlay string 157 | 158 | // Source OCI image. Optional. 159 | Image *image 160 | } 161 | 162 | func newBootFiles(path string) (*bootFiles, error) { 163 | for _, kernel := range []string{ 164 | "boot/vmlinux", 165 | "boot/vmlinuz", 166 | } { 167 | kernelPath := filepath.Join(path, kernel) 168 | if _, err := os.Stat(kernelPath); errors.Is(err, os.ErrNotExist) { 169 | continue 170 | } else if err != nil { 171 | return nil, err 172 | } 173 | 174 | return &bootFiles{ 175 | Kernel: kernelPath, 176 | Overlay: path, 177 | }, nil 178 | } 179 | 180 | return nil, fmt.Errorf("no kernel found in %s", path) 181 | } 182 | 183 | func newBootFilesFromImage(img *image) (*bootFiles, error) { 184 | bf, err := newBootFiles(img.Directory) 185 | if err != nil { 186 | return nil, fmt.Errorf("image %s: %w", img.Name, err) 187 | } 188 | 189 | bf.Image = img 190 | return bf, nil 191 | } 192 | 193 | var remoteOptions = []remote.Option{ 194 | remote.WithUserAgent("vimto"), 195 | remote.WithPlatform(v1.Platform{ 196 | OS: "linux", 197 | Architecture: runtime.GOARCH, 198 | }), 199 | } 200 | 201 | func fetchImage(ctx context.Context, refStr, dst string, status io.Writer) error { 202 | ref, err := name.ParseReference(refStr) 203 | if err != nil { 204 | return fmt.Errorf("parsing reference %q: %w", refStr, err) 205 | } 206 | 207 | bar := progressbar.NewOptions64( 208 | -1, 209 | progressbar.OptionSetDescription(fmt.Sprintf("Caching %s", ref.Name())), 210 | progressbar.OptionSetWriter(status), 211 | progressbar.OptionShowBytes(true), 212 | progressbar.OptionShowTotalBytes(false), 213 | progressbar.OptionThrottle(65*time.Millisecond), 214 | progressbar.OptionShowCount(), 215 | progressbar.OptionSpinnerType(14), 216 | progressbar.OptionFullWidth(), 217 | progressbar.OptionSetRenderBlankState(true), 218 | progressbar.OptionClearOnFinish(), 219 | ) 220 | defer bar.Finish() 221 | 222 | options := append(slices.Clone(remoteOptions), 223 | remote.WithContext(ctx), 224 | ) 225 | 226 | rmt, err := remote.Get(ref, options...) 227 | if err != nil { 228 | return fmt.Errorf("get from remote: %w", err) 229 | } 230 | 231 | image, err := rmt.Image() 232 | if err != nil { 233 | return err 234 | } 235 | 236 | if err := os.MkdirAll(dst, 0755); err != nil { 237 | return fmt.Errorf("create destination directory: %w", err) 238 | } 239 | 240 | rc := mutate.Extract(image) 241 | defer rc.Close() 242 | 243 | reader := readProxy{rc, bar} 244 | 245 | return archive.UntarUncompressed(reader, dst, &archive.TarOptions{NoLchown: true}) 246 | } 247 | 248 | type readProxy struct { 249 | io.Reader 250 | *progressbar.ProgressBar 251 | } 252 | 253 | func (rp readProxy) Read(p []byte) (int, error) { 254 | n, err := rp.Reader.Read(p) 255 | rp.ProgressBar.Add(n) 256 | return n, err 257 | } 258 | -------------------------------------------------------------------------------- /image_test.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "context" 5 | "errors" 6 | "io" 7 | "os" 8 | "path/filepath" 9 | "runtime" 10 | "testing" 11 | "time" 12 | 13 | "github.com/go-quicktest/qt" 14 | ) 15 | 16 | func TestCacheAcquire(t *testing.T) { 17 | cache := imageCache{t.TempDir()} 18 | 19 | img1, err := cache.Acquire(context.Background(), "busybox", io.Discard) 20 | qt.Assert(t, qt.IsNil(err)) 21 | defer img1.Close() 22 | 23 | start := time.Now() 24 | img2, err := cache.Acquire(context.Background(), "busybox", io.Discard) 25 | delta := time.Since(start) 26 | qt.Assert(t, qt.IsTrue(delta < 100*time.Millisecond)) 27 | qt.Assert(t, qt.IsNil(err)) 28 | defer img2.Close() 29 | 30 | qt.Assert(t, qt.Equals(img2.Directory, img1.Directory)) 31 | } 32 | 33 | func TestFetchAndExtractImage(t *testing.T) { 34 | tmp := t.TempDir() 35 | 36 | err := fetchImage(context.Background(), "busybox", tmp, io.Discard) 37 | qt.Assert(t, qt.IsNil(err)) 38 | 39 | _, err = os.Stat(filepath.Join(tmp, "bin", "sh")) 40 | qt.Assert(t, qt.IsNil(err)) 41 | } 42 | 43 | func TestPopulateDirectoryOnce(t *testing.T) { 44 | tmp := t.TempDir() 45 | 46 | waiting := make(chan struct{}) 47 | quit := make(chan struct{}) 48 | errs := make(chan error, 2) 49 | go func() { 50 | f, _, err := populateDirectoryOnce(tmp, func(s string) error { 51 | close(waiting) 52 | <-quit 53 | return nil 54 | }) 55 | if err == nil { 56 | f.Close() 57 | } 58 | errs <- err 59 | }() 60 | 61 | select { 62 | case <-waiting: 63 | case err := <-errs: 64 | t.Fatal("Got error from first invoke:", err) 65 | } 66 | 67 | go func() { 68 | f, _, err := populateDirectoryOnce(tmp, func(s string) error { 69 | return errors.New("invoked second fn") 70 | }) 71 | if err == nil { 72 | f.Close() 73 | } 74 | errs <- err 75 | }() 76 | 77 | runtime.Gosched() 78 | close(quit) 79 | 80 | qt.Assert(t, qt.IsNil(<-errs)) 81 | qt.Assert(t, qt.IsNil(<-errs)) 82 | } 83 | -------------------------------------------------------------------------------- /init.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "errors" 5 | "fmt" 6 | "io" 7 | "os" 8 | "os/exec" 9 | "path/filepath" 10 | "strings" 11 | "syscall" 12 | "time" 13 | 14 | "golang.org/x/sys/unix" 15 | ) 16 | 17 | // Default options used to mount a 9pfs. 18 | // 19 | // - cache=mmap: required to avoid mmap returning EINVAL. 20 | const default9POptions = "version=9p2000.L,trans=virtio,access=any,msize=1048576,cache=mmap" 21 | 22 | type syscaller interface { 23 | mount(*mountPoint) error 24 | sync() 25 | reboot(int) error 26 | dup2(int, int) error 27 | } 28 | 29 | type realSyscaller struct{} 30 | 31 | func (rs realSyscaller) mount(mp *mountPoint) error { 32 | err := unix.Mount(mp.source, mp.target, mp.fstype, mp.flags, mp.options) 33 | if err != nil { 34 | return fmt.Errorf("mount %s: %w", mp.target, err) 35 | } 36 | return nil 37 | } 38 | 39 | func (rs realSyscaller) sync() { 40 | unix.Sync() 41 | } 42 | 43 | func (rs realSyscaller) reboot(cmd int) error { 44 | return unix.Reboot(cmd) 45 | } 46 | 47 | func (rs realSyscaller) dup2(old, new int) error { 48 | return unix.Dup2(old, new) 49 | } 50 | 51 | func mount(source, target, fstype string, flags uintptr, data string) error { 52 | err := unix.Mount(source, target, fstype, flags, data) 53 | if err != nil { 54 | return fmt.Errorf("mount %s on %s (%s): %w", source, target, fstype, err) 55 | } 56 | return nil 57 | } 58 | 59 | func mountOverlay(target, upperdir, workdir string, lowerdirs []string) error { 60 | var options strings.Builder 61 | fmt.Fprintf(&options, "upperdir=%s,workdir=%s,", upperdir, workdir) 62 | options.WriteString("lowerdir=") 63 | for i, dir := range lowerdirs { 64 | if i > 0 { 65 | options.WriteRune(':') 66 | } 67 | options.WriteString(strings.ReplaceAll(dir, `:`, `\:`)) 68 | } 69 | 70 | return unix.Mount("overlay", target, "overlay", 0, options.String()) 71 | } 72 | 73 | type mountPoint struct { 74 | source, target string 75 | fstype string 76 | options string 77 | flags uintptr 78 | required bool 79 | } 80 | 81 | func (mp *mountPoint) String() string { 82 | return fmt.Sprintf("type %s on %s", mp.fstype, mp.target) 83 | } 84 | 85 | type mountTable []*mountPoint 86 | 87 | // Reference is https://github.com/systemd/systemd/blob/307b6a4dab21c854b141b53d9bdd05c8af0abc78/src/shared/mount-setup.c#L79 88 | var earlyMounts = mountTable{ 89 | {"sys", "/sys", "sysfs", "", unix.MS_NOSUID | unix.MS_NOEXEC | unix.MS_NODEV, true}, 90 | {"proc", "/proc", "proc", "", unix.MS_NOSUID | unix.MS_NOEXEC | unix.MS_NODEV, true}, 91 | {"devtmpfs", "/dev", "devtmpfs", "mode=0755", unix.MS_NOSUID | unix.MS_STRICTATIME, true}, 92 | {"securityfs", "/sys/kernel/security", "securityfs", "", unix.MS_NOSUID | unix.MS_NOEXEC | unix.MS_NODEV, false}, 93 | // We don't need tmpfs mounts since the root filesystem is already ephemeral. 94 | // See prepareRoot(). 95 | // {"tmpfs", "/dev/shm", "tmpfs", "mode=01777", unix.MS_NOSUID | unix.MS_NODEV | unix.MS_STRICTATIME, true}, 96 | // {"tmpfs", "/tmp", "tmpfs", "mode=01777", unix.MS_NOSUID | unix.MS_NOEXEC | unix.MS_NODEV | unix.MS_STRICTATIME, true}, 97 | // {"tmpfs", "/run", "tmpfs", "mode=01777", unix.MS_NOSUID | unix.MS_NODEV | unix.MS_STRICTATIME, true}, 98 | // {"cgroup2", "/sys/fs/cgroup", "cgroup2", "nsdelegate,memory_recursiveprot", unix.MS_NOSUID | unix.MS_NOEXEC | unix.MS_NODEV, false}, 99 | {"bpf", "/sys/fs/bpf", "bpf", "mode=0700", unix.MS_NOSUID | unix.MS_NOEXEC | unix.MS_NODEV, false}, 100 | {"debugfs", "/sys/kernel/debug", "debugfs", "", unix.MS_NOSUID | unix.MS_NOEXEC | unix.MS_NODEV | unix.MS_RELATIME, false}, 101 | {"tracefs", "/sys/kernel/tracing", "tracefs", "", unix.MS_NOSUID | unix.MS_NOEXEC | unix.MS_NODEV | unix.MS_RELATIME, false}, 102 | } 103 | 104 | // Directories from the host root filesystem which should not be accessible in 105 | // the VM. They are replaced with an empty directory. 106 | var opaqueDirectories = []string{ 107 | "/dev/shm", 108 | "/tmp", 109 | "/run", 110 | } 111 | 112 | // From man 2 statfs. 113 | var fsMagic = map[string]int64{ 114 | "bpf": unix.BPF_FS_MAGIC, 115 | "cgroup2": unix.CGROUP2_SUPER_MAGIC, 116 | "debugfs": unix.DEBUGFS_MAGIC, 117 | "devtmpfs": unix.TMPFS_MAGIC, 118 | "overlay": unix.OVERLAYFS_SUPER_MAGIC, 119 | "proc": unix.PROC_SUPER_MAGIC, 120 | "securityfs": unix.SECURITYFS_MAGIC, 121 | "sysfs": unix.SYSFS_MAGIC, 122 | "tmpfs": unix.TMPFS_MAGIC, 123 | "tracefs": unix.TRACEFS_MAGIC, 124 | } 125 | 126 | // Mount all mount points contained in the table. 127 | // 128 | // Returns a list of optional mount points which failed to mount. 129 | func (mt mountTable) mountAll(sys syscaller) ([]*mountPoint, error) { 130 | var ignored []*mountPoint 131 | for _, mp := range mt { 132 | if _, err := os.Stat(mp.target); errors.Is(err, unix.ENOENT) { 133 | if err := os.MkdirAll(mp.target, 0755); err != nil { 134 | return nil, fmt.Errorf("mount %s: %w", mp, err) 135 | } 136 | } else if err != nil { 137 | return nil, fmt.Errorf("mount %s: %w", mp, err) 138 | } 139 | 140 | // TODO: Check /proc/self/mountinfo or similar whether the mountpoint 141 | // already exists. statfs doesn't work since 9pfs will happily forward 142 | // the statfs call to the host mount. 143 | 144 | err := sys.mount(mp) 145 | if errors.Is(err, unix.ENODEV) && !mp.required { 146 | ignored = append(ignored, mp) 147 | continue 148 | } else if errors.Is(err, unix.EBUSY) { 149 | // Already mounted. From man 2 mount: 150 | // An attempt was made to stack a new mount directly on top of an 151 | // existing mount point that was created in this mount namespace 152 | // with the same source and target. 153 | continue 154 | } else if err != nil { 155 | return nil, fmt.Errorf("mount %s: %w", mp, err) 156 | } 157 | } 158 | return ignored, nil 159 | } 160 | 161 | func (mt mountTable) pathIsBelowMount(path string) (string, bool) { 162 | for _, mp := range mt { 163 | target := mp.target 164 | if len(target) > 0 && target[len(target)-1] != filepath.Separator { 165 | target += string(filepath.Separator) 166 | } 167 | 168 | // TODO: This ignores case insensitivity of the filesystem. 169 | if strings.HasPrefix(path, mp.target) { 170 | return mp.fstype, true 171 | } 172 | } 173 | return "", false 174 | } 175 | 176 | func minimalInit(sys syscaller, args []string) error { 177 | err := func() error { 178 | if len(args) != 2 { 179 | return fmt.Errorf("expected two arguments, got %q", args) 180 | } 181 | 182 | if err := prepareRoot(); err != nil { 183 | return err 184 | } 185 | 186 | stdioPort := args[0] 187 | controlPort := args[1] 188 | 189 | ignored, err := earlyMounts.mountAll(sys) 190 | if err != nil { 191 | return fmt.Errorf("mount: %w", err) 192 | } 193 | 194 | for _, mp := range ignored { 195 | fmt.Fprintf(os.Stderr, "Mounting %s failed, ignoring\n", mp) 196 | } 197 | 198 | ports, err := readVirtioPorts() 199 | if err != nil { 200 | return err 201 | } 202 | 203 | stdio, err := os.OpenFile(ports[stdioPort], os.O_RDWR, 0) 204 | if err != nil { 205 | return fmt.Errorf("open stdio: %w", err) 206 | } 207 | defer stdio.Close() 208 | delete(ports, stdioPort) 209 | 210 | control, err := os.OpenFile(ports[controlPort], os.O_RDWR, 0) 211 | if err != nil { 212 | return err 213 | } 214 | comm := newRPC(control) 215 | defer comm.Close() 216 | delete(ports, controlPort) 217 | 218 | var cmd execCommand 219 | if err := comm.Read(&cmd, time.Now().Add(time.Second)); err != nil { 220 | return fmt.Errorf("read command: %w", err) 221 | } 222 | 223 | for tag, path := range cmd.MountTags { 224 | path = filepath.Clean(path) 225 | fmt.Println("Mounting", path) 226 | 227 | for _, opaque := range opaqueDirectories { 228 | if strings.HasPrefix(path, opaque) { 229 | if err := os.MkdirAll(path, 0755); err != nil { 230 | return fmt.Errorf("mount %q: %w", path, err) 231 | } 232 | 233 | break 234 | } 235 | } 236 | 237 | // TODO: Investigate dfltuid, dfltgid, noxattr options. 238 | err = sys.mount(&mountPoint{ 239 | tag, path, 240 | "9p", 241 | default9POptions, 242 | 0, 243 | false, // ignored 244 | }) 245 | if err != nil { 246 | return fmt.Errorf("mount %q: %w", path, err) 247 | } 248 | } 249 | 250 | // Setup environment variables. 251 | for _, env := range cmd.Env { 252 | key, value, _ := strings.Cut(env, "=") 253 | if key == "PATH" { 254 | err := os.Setenv("PATH", value) 255 | if err != nil { 256 | return err 257 | } 258 | // NB: Must check all entries since there might be duplicates. 259 | } 260 | } 261 | 262 | // Apply sysctls. 263 | for _, sysctl := range cmd.Sysctls { 264 | err = writeSysctl(sysctl.Name, sysctl.Value) 265 | if err != nil { 266 | return fmt.Errorf("set sysctl %s: %w", sysctl.Name, err) 267 | } 268 | } 269 | 270 | if err := executeSimpleCommands([]configCommand{{"ip", "link", "set", "dev", "lo", "up"}}, cmd.Dir, cmd.Env); err != nil { 271 | fmt.Fprintln(os.Stderr, "Set up lo:", err) 272 | } 273 | 274 | if err := executeSimpleCommands(cmd.Setup, cmd.Dir, cmd.Env); err != nil { 275 | return fmt.Errorf("setup: %w", err) 276 | } 277 | 278 | // Set resource limits 279 | for resource, limit := range cmd.Rlimits { 280 | err = unix.Setrlimit(resource, &limit) 281 | if err != nil { 282 | return fmt.Errorf("raise resource limit 0x%x: %w", resource, err) 283 | } 284 | } 285 | 286 | proc := exec.Command(cmd.Path) 287 | proc.Args = cmd.Args 288 | proc.Dir = cmd.Dir 289 | proc.Env = cmd.Env 290 | proc.Stdin = stdio 291 | proc.Stdout = stdio 292 | proc.Stderr = stdio 293 | proc.SysProcAttr = &syscall.SysProcAttr{ 294 | Credential: &syscall.Credential{ 295 | Uid: uint32(cmd.Uid), 296 | Gid: uint32(cmd.Gid), 297 | NoSetGroups: true, 298 | }, 299 | } 300 | 301 | if err := unix.Access(proc.Path, unix.R_OK); err == unix.EACCES { 302 | // QEMU limitation: the 9p implementation needs to be able to read the 303 | // executable to be able to execute it in the VM. 304 | // TODO: Might be able to avoid this using CAP_DAC_OVERRIDE or similar. 305 | fmt.Fprintf(stdio, "%s is not readable, execution might fail with %q\n", cmd.Path, err) 306 | } 307 | 308 | result := proc.Run() 309 | 310 | if err := executeSimpleCommands(cmd.Teardown, cmd.Dir, cmd.Env); err != nil { 311 | return fmt.Errorf("teardown: %w", err) 312 | } 313 | 314 | var exitError *exec.ExitError 315 | if errors.As(result, &exitError) && exitError.Exited() { 316 | result = &guestExitError{exitError.ExitCode()} 317 | } else if result != nil { 318 | result = &genericGuestError{result.Error()} 319 | } 320 | 321 | return comm.Write(&result, time.Now().Add(time.Second)) 322 | }() 323 | 324 | if err != nil { 325 | fmt.Fprintln(os.Stderr, "Error:", err) 326 | } 327 | 328 | sys.sync() 329 | return sys.reboot(unix.LINUX_REBOOT_CMD_POWER_OFF) 330 | } 331 | 332 | func executeSimpleCommands(cmds []configCommand, dir string, env []string) error { 333 | for _, args := range cmds { 334 | if len(args) < 1 { 335 | return fmt.Errorf("invalid empty command") 336 | } 337 | 338 | cmd := exec.Command(args[0], args[1:]...) 339 | cmd.Dir = dir 340 | cmd.Env = env 341 | cmd.Stderr = os.Stderr 342 | cmd.Stdout = os.Stdout 343 | cmd.WaitDelay = time.Second 344 | if err := cmd.Run(); err != nil { 345 | return fmt.Errorf("%s: %w", cmd.Path, err) 346 | } 347 | } 348 | return nil 349 | } 350 | 351 | func prepareRoot() error { 352 | const ( 353 | hostDir = "/host" 354 | overlayDir = "/overlay" 355 | upperDir = "/upper" 356 | workDir = "/work" 357 | mergedDir = "/merged" 358 | ) 359 | 360 | // The current mount table looks something like this: 361 | // / 9pfs mount of host 362 | // /dev devtmpfs (automounted) 363 | 364 | // Remove unnecessary /dev, we're going to mount our own later on. 365 | if err := unix.Unmount("/dev", 0); err != nil && !errors.Is(err, unix.ENOENT) { 366 | return fmt.Errorf("unmount automounted /dev: %w", err) 367 | } 368 | 369 | // Mount a tmpfs so that we can create files, etc. Doesn't have to be on 370 | // /tmp, but why not? 371 | // Limit its size to 25% of total RAM to avoid triggering oomkills, instead 372 | // turning them into ENOSPC or similar. 373 | // TODO: flags? 374 | if err := mount("tmpfs", "/tmp", "tmpfs", 0, "size=25%"); err != nil { 375 | return err 376 | } 377 | 378 | // Create mountpoints in our own tmpfs. 379 | for _, dir := range []string{hostDir, overlayDir, upperDir, workDir, mergedDir} { 380 | if err := os.Mkdir(filepath.Join("/tmp", dir), 0755); err != nil { 381 | return err 382 | } 383 | } 384 | 385 | // Switch the root to the tmpfs. 386 | if err := unix.PivotRoot("/tmp", filepath.Join("/tmp", hostDir)); err != nil { 387 | return fmt.Errorf("pivot root: %w", err) 388 | } 389 | 390 | // The mount table is now: 391 | // / tmpfs 392 | // /host 9pfs 393 | 394 | // Remove access to some temporary directories on the host root fs. 395 | // 396 | // See https://docs.kernel.org/filesystems/overlayfs.html#whiteouts-and-opaque-directories 397 | for _, opaque := range opaqueDirectories { 398 | path := filepath.Join(upperDir, opaque) 399 | if err := os.MkdirAll(path, 0755); err != nil { 400 | return fmt.Errorf("make directory opaque: %w", err) 401 | } 402 | 403 | if err := unix.Setxattr(path, "trusted.overlay.opaque", []byte("y"), 0); err != nil { 404 | return fmt.Errorf("set opaque xattr: %w", err) 405 | } 406 | } 407 | 408 | // Constituent parts of the merged filesystem. 409 | // 410 | // Directories with lower indices take precedence. 411 | var lowerDirs []string 412 | 413 | // Optionally mount an overlay for the root filesystem. 414 | err := mount(p9OverlayTag, overlayDir, "9p", unix.MS_RDONLY, default9POptions) 415 | if err != nil { 416 | if !errors.Is(err, unix.ENOENT) { 417 | return fmt.Errorf("mount overlay: %w", err) 418 | } 419 | 420 | fmt.Fprintln(os.Stderr, "Not mounting overlay:", err) 421 | } else { 422 | lowerDirs = append(lowerDirs, overlayDir) 423 | 424 | if err := checkHostShadowing(hostDir, overlayDir); err != nil { 425 | return err 426 | } 427 | } 428 | 429 | // Mount the overlayfs which we'll use as the root. 430 | lowerDirs = append(lowerDirs, hostDir) 431 | if err := mountOverlay(mergedDir, upperDir, workDir, lowerDirs); err != nil { 432 | return fmt.Errorf("mount root overlay: %w", err) 433 | } 434 | 435 | // The mount table is now: 436 | // / tmpfs 437 | // /host 9pfs 438 | // /overlay 9pfs 439 | // /merged overlayfs 440 | 441 | if err := os.Chdir(mergedDir); err != nil { 442 | return err 443 | } 444 | 445 | err = unix.Mount(".", "/", "", unix.MS_MOVE, "") 446 | if err != nil { 447 | return fmt.Errorf("move root mount: %w", err) 448 | } 449 | 450 | // The mount table is now: 451 | // / overlayfs 452 | // /host 9pfs (shadowed by /) 453 | 454 | if err := unix.Chroot("."); err != nil { 455 | return fmt.Errorf("chroot: %w", err) 456 | } 457 | 458 | if err := unix.Chdir("/"); err != nil { 459 | return fmt.Errorf("chdir: %w", err) 460 | } 461 | 462 | if err := unix.Chmod("/tmp", 01777); err != nil { 463 | return fmt.Errorf("chmod /tmp: %w", err) 464 | } 465 | 466 | return nil 467 | } 468 | 469 | var errShadowedDirectory = errors.New("shadows symlink on host") 470 | 471 | // checkHostShadowing ensures that some important directories in the host aren't 472 | // shadowed by the overlay. 473 | // 474 | // For example, a /lib directory in the overlay will shadow a /lib symlink on 475 | // the host mount since overlay fs only ever merges two directories, not a 476 | // directory and a symlink. 477 | func checkHostShadowing(host, overlay string) error { 478 | dirs := []string{ 479 | "/lib", "/lib64", 480 | "/bin", "/sbin", 481 | } 482 | 483 | for _, dir := range dirs { 484 | ovlInfo, err := os.Lstat(filepath.Join(overlay, dir)) 485 | if errors.Is(err, os.ErrNotExist) { 486 | continue 487 | } 488 | if err != nil { 489 | return err 490 | } 491 | if !ovlInfo.IsDir() { 492 | continue 493 | } 494 | 495 | hostInfo, err := os.Lstat(filepath.Join(host, dir)) 496 | if errors.Is(err, os.ErrNotExist) { 497 | continue 498 | } 499 | if err != nil { 500 | return err 501 | } 502 | if hostInfo.Mode().Type() == os.ModeSymlink { 503 | return fmt.Errorf("directory %s: %w", dir, errShadowedDirectory) 504 | } 505 | } 506 | 507 | return nil 508 | } 509 | 510 | // Read the names of virtio ports from /sys. 511 | // 512 | // Based on https://gitlab.com/qemu-project/qemu/-/issues/506 513 | func readVirtioPorts() (map[string]string, error) { 514 | const base = "/sys/class/virtio-ports" 515 | 516 | files, err := os.ReadDir(base) 517 | if err != nil { 518 | return nil, err 519 | } 520 | 521 | ports := make(map[string]string) 522 | for _, file := range files { 523 | // NB: file.IsDir() returns false even though it behaves like a directory. 524 | // Oh well! 525 | name, err := os.ReadFile(filepath.Join(base, file.Name(), "name")) 526 | if err != nil { 527 | return nil, err 528 | } 529 | 530 | ports[strings.TrimSpace(string(name))] = filepath.Join("/dev/", file.Name()) 531 | } 532 | 533 | return ports, nil 534 | } 535 | 536 | func read9PMountTags() ([]string, error) { 537 | files, err := filepath.Glob("/sys/bus/virtio/drivers/9pnet_virtio/virtio*/mount_tag") 538 | if err != nil { 539 | return nil, err 540 | } 541 | 542 | var tags []string 543 | for _, file := range files { 544 | rawTag, err := os.ReadFile(file) 545 | if err != nil { 546 | return nil, err 547 | } 548 | 549 | tags = append(tags, unix.ByteSliceToString(rawTag)) 550 | } 551 | 552 | return tags, nil 553 | } 554 | 555 | func writeSysctl(name, value string) error { 556 | path := filepath.Join("/proc/sys", strings.ReplaceAll(name, ".", string(filepath.Separator))) 557 | // No O_CREAT, so that we don't create non-existent sysctls. 558 | f, err := os.OpenFile(path, os.O_WRONLY|os.O_TRUNC, 0644) 559 | if err != nil { 560 | return err 561 | } 562 | _, err = io.WriteString(f, value) 563 | if err1 := f.Close(); err1 != nil && err == nil { 564 | err = err1 565 | } 566 | return err 567 | } 568 | -------------------------------------------------------------------------------- /init_test.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "os" 5 | "path/filepath" 6 | "testing" 7 | 8 | "github.com/go-quicktest/qt" 9 | ) 10 | 11 | func TestFsMagic(t *testing.T) { 12 | for _, mp := range earlyMounts { 13 | _, ok := fsMagic[mp.fstype] 14 | qt.Check(t, qt.IsTrue(ok), qt.Commentf("Unknown magic for fstype %q", mp.fstype)) 15 | } 16 | } 17 | 18 | func TestCheckHostShadowing(t *testing.T) { 19 | root := t.TempDir() 20 | mustMkdirAll(t, root, "usr/lib") 21 | qt.Assert(t, qt.IsNil(os.Symlink("usr/lib", filepath.Join(root, "lib")))) 22 | 23 | overlay := t.TempDir() 24 | mustMkdirAll(t, overlay, "usr/lib") 25 | qt.Assert(t, qt.IsNil(checkHostShadowing(root, overlay)), qt.Commentf("Nothing shadows")) 26 | 27 | qt.Assert(t, qt.IsNil(os.Symlink("usr/lib", filepath.Join(overlay, "lib")))) 28 | qt.Assert(t, qt.IsNil(checkHostShadowing(root, overlay)), qt.Commentf("Symlink shadow")) 29 | 30 | qt.Assert(t, qt.IsNil(os.Remove(filepath.Join(overlay, "lib")))) 31 | mustMkdirAll(t, overlay, "lib") 32 | qt.Assert(t, qt.ErrorIs(checkHostShadowing(root, overlay), errShadowedDirectory), qt.Commentf("Directory shadows symlink")) 33 | } 34 | 35 | func mustMkdirAll(tb testing.TB, parts ...string) { 36 | tb.Helper() 37 | 38 | qt.Assert(tb, qt.IsNil(os.MkdirAll(filepath.Join(parts...), 0755))) 39 | } 40 | -------------------------------------------------------------------------------- /logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lmb/vimto/f07f29b9a49024cfa6f3f2e20678f234a1f870fd/logo.png -------------------------------------------------------------------------------- /main.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "context" 5 | "crypto/rand" 6 | "errors" 7 | "flag" 8 | "fmt" 9 | "io" 10 | "os" 11 | "os/signal" 12 | "path/filepath" 13 | "slices" 14 | "strings" 15 | 16 | "github.com/kballard/go-shellquote" 17 | "golang.org/x/sys/unix" 18 | ) 19 | 20 | func main() { 21 | args := os.Args[1:] 22 | 23 | var err error 24 | if os.Getpid() == 1 { 25 | err = minimalInit(realSyscaller{}, args) 26 | } else { 27 | err = run(args) 28 | } 29 | 30 | if err == nil || errors.Is(err, flag.ErrHelp) { 31 | return 32 | } 33 | 34 | var exitError *guestExitError 35 | if errors.As(err, &exitError) { 36 | os.Exit(exitError.ExitCode) 37 | } 38 | 39 | fmt.Fprintln(os.Stderr, "Error:", err) 40 | os.Exit(128) 41 | } 42 | 43 | var usage = ` 44 | Usage: %s [flags] [command] [--] ... 45 | 46 | Available commands: 47 | exec Execute a command inside a VM 48 | flush-cache Clear the image cache 49 | 50 | Flags: 51 | ` 52 | 53 | func run(args []string) error { 54 | cfg := *defaultConfig 55 | fs := configFlags("vimto", &cfg) 56 | fs.Usage = func() { 57 | o := fs.Output() 58 | fmt.Fprintf(o, strings.TrimSpace(usage), fs.Name()) 59 | fs.PrintDefaults() 60 | } 61 | if err := parseConfigFromTOML(".", &cfg); err != nil { 62 | return fmt.Errorf("read config: %w", err) 63 | } 64 | if err := fs.Parse(args); err != nil { 65 | return err 66 | } 67 | 68 | if fs.NArg() < 1 { 69 | return fmt.Errorf("expected at least one argument") 70 | } 71 | 72 | var err error 73 | switch cmd := fs.Arg(0); { 74 | case cmd == "exec": 75 | err = execCmd(&cfg, fs.Args()[1:]) 76 | 77 | case cmd == "flush-cache": 78 | err = flushCacheCmd(fs.Args()[1:]) 79 | 80 | case strings.HasPrefix(cmd, "go"): 81 | // This is an invocation of go test, possibly via a pre-relase binary 82 | // like go1.21rc2. 83 | var flags []string 84 | flags, err = splitFlagsFromArgs(args) 85 | if err != nil { 86 | return err 87 | } 88 | 89 | err = goTestCmd(&cfg, flags, cmd, fs.Args()[1:]) 90 | 91 | default: 92 | fs.Usage() 93 | return fmt.Errorf("unknown command %q", fs.Arg(0)) 94 | } 95 | 96 | if err != nil { 97 | return fmt.Errorf("%s: %w", fs.Arg(0), err) 98 | } 99 | 100 | return nil 101 | } 102 | 103 | // goTestCmd executes a go test command inside a VM. 104 | func goTestCmd(cfg *config, flags []string, goBinary string, goArgs []string) error { 105 | if len(goArgs) < 1 || goArgs[0] != "test" { 106 | return fmt.Errorf("first argument to go binary must be 'test'") 107 | } 108 | 109 | for _, arg := range goArgs { 110 | if strings.HasPrefix(arg, "-exec") { 111 | return fmt.Errorf("specifying -exec on the go command line is not supported") 112 | } 113 | } 114 | 115 | if cfg.GDB != "" { 116 | return fmt.Errorf("can't enable gdb integration when invoking via go toolchain") 117 | } 118 | 119 | exe, err := findExecutable() 120 | if err != nil { 121 | return err 122 | } 123 | 124 | // Prime the cache before invoking the go binary. 125 | bf, err := findBootFiles(cfg.Kernel) 126 | if err != nil { 127 | return err 128 | } 129 | defer bf.Image.Close() 130 | 131 | execArgs := []string{exe} 132 | // Retain command line arguments 133 | // TODO: Make -kernel parameter absolute? 134 | execArgs = append(execArgs, flags...) 135 | // Execute exec command and ignore all test flags. 136 | execArgs = append(execArgs, "exec", "--") 137 | 138 | goArgs = slices.Insert(goArgs, 1, "-exec", shellquote.Join(execArgs...)) 139 | 140 | ctx, cancel := signal.NotifyContext(context.Background(), os.Interrupt) 141 | defer cancel() 142 | 143 | cmd := commandWithGracefulTermination(ctx, goBinary, goArgs...) 144 | cmd.Stdin = os.Stdin 145 | cmd.Stdout = os.Stdout 146 | cmd.Stderr = os.Stderr 147 | return cmd.Run() 148 | } 149 | 150 | func execCmd(cfg *config, args []string) error { 151 | fs := flag.NewFlagSet("exec", flag.ContinueOnError) 152 | fs.Usage = func() { 153 | fmt.Fprintf(fs.Output(), "Usage: %s [--] [flags of binary]\n", fs.Name()) 154 | fmt.Fprintln(fs.Output()) 155 | fs.PrintDefaults() 156 | fmt.Fprintln(fs.Output()) 157 | } 158 | 159 | if err := fs.Parse(args); err != nil { 160 | return err 161 | } 162 | 163 | if fs.NArg() < 1 { 164 | fs.Usage() 165 | return fmt.Errorf("missing arguments") 166 | } 167 | 168 | if !staticBuild { 169 | return errors.New("binary is not statically linked (did you build with CGO_ENABLED=0?)") 170 | } 171 | 172 | bf, err := findBootFiles(cfg.Kernel) 173 | if err != nil { 174 | return err 175 | } 176 | defer bf.Image.Close() 177 | 178 | if cfg.GDB != "" { 179 | fmt.Println("Starting GDB server with CPU halted, connect using:") 180 | args := []string{ 181 | "-ex", fmt.Sprintf("target remote %s", cfg.GDB), 182 | } 183 | if bf.Overlay != "" { 184 | if strings.Contains(bf.Overlay, ":") { 185 | // Can't figure out how to avoid gdb interpreting the colon 186 | // as a directory separator. 187 | return fmt.Errorf("path %q contains a colon", bf.Overlay) 188 | } 189 | args = append(args, "-ex", fmt.Sprintf("dir %q", bf.Overlay)) 190 | } 191 | fmt.Printf("\tgdb %s %s\n", shellquote.Join(args...), bf.Kernel) 192 | } 193 | 194 | ctx, cancel := signal.NotifyContext(context.Background(), os.Interrupt) 195 | defer cancel() 196 | 197 | // Ensure that the repository root is available in the VM. 198 | var sharedDirectories []string 199 | if repo, err := findGitRoot("."); err != nil { 200 | return err 201 | } else if repo != "" { 202 | sharedDirectories = append(sharedDirectories, repo) 203 | } 204 | 205 | // Ensure that the binary is available in the VM. 206 | path := fs.Arg(0) 207 | sharedDirectories = append(sharedDirectories, filepath.Dir(path)) 208 | 209 | // Ensure that the working directory is available. 210 | wd, err := os.Getwd() 211 | if err != nil { 212 | return err 213 | } 214 | sharedDirectories = append(sharedDirectories, wd) 215 | 216 | tmp := make([]byte, 2) 217 | rand.Read(tmp) 218 | corePrefix := fmt.Sprintf("core-%x-", tmp) 219 | 220 | cmd := &command{ 221 | Kernel: bf.Kernel, 222 | Memory: cfg.Memory, 223 | SMP: cfg.SMP, 224 | Path: path, 225 | Args: fs.Args(), 226 | Dir: wd, 227 | GDB: cfg.GDB, 228 | User: cfg.User, 229 | Stdin: os.Stdin, 230 | Stdout: os.Stdout, 231 | Stderr: os.Stderr, 232 | RootOverlay: bf.Overlay, 233 | Sysctls: []sysctl{ 234 | {"kernel.core_pattern", filepath.Join(wd, corePrefix+"%e.%p.%t")}, 235 | }, 236 | Rlimits: map[int]unix.Rlimit{ 237 | unix.RLIMIT_CORE: {Cur: unix.RLIM_INFINITY, Max: unix.RLIM_INFINITY}, 238 | }, 239 | Setup: cfg.Setup, 240 | Teardown: cfg.Teardown, 241 | SharedDirectories: slices.Compact(sharedDirectories), 242 | } 243 | 244 | if err := cmd.Start(ctx); err != nil { 245 | return err 246 | } 247 | 248 | waitErr := cmd.Wait() 249 | if waitErr == nil { 250 | return nil 251 | } 252 | 253 | // Something went wrong, try to retain the go test binary if appropriate. 254 | if err := preserveTestBinary(cmd.Path, wd, corePrefix); err != nil { 255 | return err 256 | } 257 | 258 | return waitErr 259 | } 260 | 261 | func findBootFiles(kernel string) (_ *bootFiles, err error) { 262 | closeOnError := func(c io.Closer) { 263 | if err != nil { 264 | c.Close() 265 | } 266 | } 267 | 268 | if kernel == "" { 269 | return nil, errors.New("no kernel specified") 270 | } 271 | 272 | info, err := os.Stat(kernel) 273 | if errors.Is(err, os.ErrNotExist) { 274 | // Assume that kernel is a reference to an image. 275 | cache, err := newImageCache() 276 | if err != nil { 277 | return nil, fmt.Errorf("image cache: %w", err) 278 | } 279 | 280 | img, err := cache.Acquire(context.Background(), kernel, os.Stdout) 281 | if err != nil { 282 | return nil, fmt.Errorf("retrieve kernel from OCI image: %w", err) 283 | } 284 | defer closeOnError(img) 285 | 286 | return newBootFilesFromImage(img) 287 | } else if err != nil { 288 | // Unexpected error from stat, maybe not allowed to access it? 289 | return nil, err 290 | } 291 | 292 | if info.IsDir() { 293 | return newBootFiles(kernel) 294 | } 295 | 296 | // Kernel is a file on disk. 297 | return &bootFiles{Kernel: kernel}, nil 298 | } 299 | 300 | func findExecutable() (string, error) { 301 | // https://man7.org/linux/man-pages/man5/proc.5.html 302 | buf := make([]byte, unix.NAME_MAX) 303 | n, err := unix.Readlink("/proc/self/exe", buf) 304 | if err != nil { 305 | return "", fmt.Errorf("readlink /proc/self/exe: %w", err) 306 | } 307 | 308 | if n == unix.NAME_MAX { 309 | return "", fmt.Errorf("readlink returned truncated name") 310 | } 311 | 312 | path := unix.ByteSliceToString(buf) 313 | if _, err := os.Stat(path); err != nil { 314 | // Make sure the symlink doesn't reference a deleted file. 315 | return "", err 316 | } 317 | 318 | return path, nil 319 | } 320 | 321 | func splitFlagsFromArgs(args []string) ([]string, error) { 322 | var flags []string 323 | for _, arg := range args { 324 | if arg == "--" { 325 | return flags, nil 326 | } 327 | 328 | flags = append(flags, arg) 329 | } 330 | 331 | return nil, fmt.Errorf("missing '--' in arguments") 332 | } 333 | 334 | func preserveTestBinary(path, wd, corePrefix string) error { 335 | if isGoTest := strings.HasPrefix(path, os.TempDir()); !isGoTest { 336 | return nil 337 | } 338 | 339 | if files, err := filepath.Glob(filepath.Join(wd, corePrefix+"*")); err != nil { 340 | return err 341 | } else if len(files) == 0 { 342 | return nil 343 | } 344 | 345 | src, err := os.Open(path) 346 | if err != nil { 347 | return err 348 | } 349 | defer src.Close() 350 | 351 | dst, err := os.OpenFile(filepath.Join(wd, filepath.Base(path)), os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0644) 352 | if err != nil { 353 | return err 354 | } 355 | defer dst.Close() 356 | 357 | _, err = io.Copy(dst, src) 358 | return err 359 | } 360 | 361 | // flushCacheCmd deletes the image cache directory by first renaming it 362 | // and then removing it to handle concurrent access. 363 | func flushCacheCmd(args []string) error { 364 | if len(args) > 0 { 365 | return fmt.Errorf("flush-cache command takes no arguments") 366 | } 367 | 368 | // Generate a random suffix for the temporary directory 369 | tmp := make([]byte, 4) 370 | if _, err := rand.Read(tmp); err != nil { 371 | return fmt.Errorf("generate random suffix: %w", err) 372 | } 373 | tmpDir := userCacheDir + fmt.Sprintf(".%x", tmp) 374 | 375 | // Rename the cache directory to prevent another process from seeing 376 | // a partially delete cache. 377 | if err := os.Rename(userCacheDir, tmpDir); err != nil { 378 | if os.IsNotExist(err) { 379 | return nil 380 | } 381 | return fmt.Errorf("rename cache directory: %w", err) 382 | } 383 | 384 | // Remove the renamed directory 385 | if err := os.RemoveAll(tmpDir); err != nil { 386 | return fmt.Errorf("remove cache directory: %w", err) 387 | } 388 | 389 | return nil 390 | } 391 | -------------------------------------------------------------------------------- /main_test.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "bufio" 5 | "context" 6 | "encoding/hex" 7 | "errors" 8 | "fmt" 9 | "io" 10 | "net" 11 | "os" 12 | "os/exec" 13 | "path/filepath" 14 | "strings" 15 | "sync" 16 | "testing" 17 | "time" 18 | 19 | "github.com/go-quicktest/qt" 20 | "golang.org/x/sys/unix" 21 | "rsc.io/script" 22 | "rsc.io/script/scripttest" 23 | ) 24 | 25 | func TestMain(m *testing.M) { 26 | if os.Getpid() == 1 { 27 | err := minimalInit(realSyscaller{}, os.Args[1:]) 28 | if err != nil { 29 | fmt.Fprintln(os.Stderr, "Error from minimalInit:", err) 30 | os.Exit(1) 31 | } 32 | } 33 | 34 | os.Exit(m.Run()) 35 | } 36 | 37 | func TestExecutable(t *testing.T) { 38 | path := t.TempDir() 39 | cmd := exec.Command("go", "build", "-o", path, ".") 40 | cmd.Env = append(os.Environ(), "CGO_ENABLED=0") 41 | if output, err := cmd.CombinedOutput(); err != nil { 42 | t.Log(string(output)) 43 | t.Fatal("Failed to compile binary:", err) 44 | } 45 | 46 | t.Setenv("PATH", fmt.Sprintf("%s:%s", path, os.Getenv("PATH"))) 47 | 48 | e := script.NewEngine() 49 | e.Cmds["glob-exists"] = globExists 50 | e.Cmds["gdb"] = gdbStub 51 | e.Cmds["new-tmp"] = script.Command(script.CmdUsage{ 52 | Summary: "use a distinct temp directory", 53 | Detail: []string{ 54 | "Create a new TMPDIR which is not a subdirectory of WORK.", 55 | }, 56 | }, func(s *script.State, args ...string) (script.WaitFunc, error) { 57 | if len(args) != 0 { 58 | return nil, script.ErrUsage 59 | } 60 | 61 | s.Setenv("TMPDIR", t.TempDir()) 62 | return nil, nil 63 | }) 64 | e.Cmds["vimto"] = script.Program("vimto", nil, time.Second) 65 | e.Cmds["config"] = script.Command(script.CmdUsage{ 66 | Summary: "Write to the configuration file", 67 | Args: "items...", 68 | }, func(s *script.State, args ...string) (script.WaitFunc, error) { 69 | contents := strings.Join(args, "\n") 70 | return nil, os.WriteFile(filepath.Join(s.Getwd(), configFileName), []byte(contents), 0644) 71 | }) 72 | 73 | var env []string 74 | for _, v := range os.Environ() { 75 | for _, prefix := range []string{ 76 | "GO", 77 | "XDG_", 78 | "PATH=", 79 | "HOME=", 80 | "VIMTO_", 81 | } { 82 | if strings.HasPrefix(v, prefix) { 83 | env = append(env, v) 84 | break 85 | } 86 | } 87 | } 88 | 89 | bf := mustFetchKernelImage(t) 90 | env = append(env, "IMAGE="+bf.Image.Name) 91 | env = append(env, "KERNEL="+bf.Kernel) 92 | env = append(env, fmt.Sprintf("UID=%d", os.Geteuid())) 93 | 94 | scripttest.Test(t, context.Background(), e, env, "testdata/*.txt") 95 | } 96 | 97 | func kernelImage() string { 98 | image := os.Getenv("CI_KERNEL") 99 | if image == "" { 100 | image = "ghcr.io/cilium/ci-kernels:stable" 101 | } 102 | return image 103 | } 104 | 105 | var fetchKernelImage = sync.OnceValues(func() (*bootFiles, error) { 106 | cache, err := newImageCache() 107 | if err != nil { 108 | return nil, err 109 | } 110 | 111 | img, err := cache.Acquire(context.Background(), kernelImage(), io.Discard) 112 | if err != nil { 113 | return nil, err 114 | } 115 | 116 | return newBootFilesFromImage(img) 117 | }) 118 | 119 | // mustFetchKernelImage fetches a kernel image once for the entire lifetime 120 | // of the test binary. 121 | func mustFetchKernelImage(tb testing.TB) *bootFiles { 122 | bf, err := fetchKernelImage() 123 | qt.Assert(tb, qt.IsNil(err)) 124 | // NB: Do not call image.Close()! 125 | return bf 126 | } 127 | 128 | var globExists = script.Command( 129 | script.CmdUsage{ 130 | Summary: "check that files exist", 131 | Args: "pattern...", 132 | }, 133 | func(s *script.State, patterns ...string) (script.WaitFunc, error) { 134 | if len(patterns) == 0 { 135 | return nil, script.ErrUsage 136 | } 137 | 138 | for _, pattern := range patterns { 139 | files, err := filepath.Glob(s.Path(pattern)) 140 | if err != nil { 141 | return nil, err 142 | } 143 | 144 | if len(files) == 0 { 145 | return nil, fmt.Errorf("no file(s) matched pattern %q", pattern) 146 | } 147 | 148 | for _, file := range files { 149 | file, err := filepath.Rel(s.Getwd(), file) 150 | if err != nil { 151 | return nil, err 152 | } 153 | 154 | s.Logf("pattern %q matched %v", pattern, file) 155 | } 156 | 157 | } 158 | 159 | return nil, nil 160 | }, 161 | ) 162 | 163 | var gdbStub = script.Command( 164 | script.CmdUsage{ 165 | Summary: "send raw commands to a gdb stub", 166 | Args: "target packets...", 167 | }, 168 | func(s *script.State, args ...string) (script.WaitFunc, error) { 169 | if len(args) < 1 { 170 | return nil, script.ErrUsage 171 | } 172 | 173 | target := args[0] 174 | 175 | var packets []string 176 | for _, cmd := range args[1:] { 177 | packets = append(packets, fmt.Sprintf("$%s#%02x", cmd, gdbChecksum(cmd))) 178 | } 179 | 180 | var stdout, stderr strings.Builder 181 | errs := make(chan error, 1) 182 | go func() { 183 | defer close(errs) 184 | 185 | ctx, cancel := context.WithTimeout(s.Context(), 5*time.Second) 186 | defer cancel() 187 | 188 | conn, err := tryDial(ctx, "tcp", target) 189 | if err != nil { 190 | errs <- fmt.Errorf("connect to gdb stub: %w", err) 191 | return 192 | } 193 | defer conn.Close() 194 | 195 | buf := bufio.NewReader(conn) 196 | for _, packet := range packets { 197 | if err := conn.SetWriteDeadline(time.Now().Add(100 * time.Millisecond)); err != nil { 198 | errs <- err 199 | return 200 | } 201 | 202 | _, err := io.WriteString(conn, packet) 203 | if err != nil { 204 | errs <- err 205 | return 206 | } 207 | 208 | response, err := gdbReadResponse(buf) 209 | if err != nil { 210 | errs <- fmt.Errorf("read response: %w", err) 211 | return 212 | } 213 | 214 | switch { 215 | case response == "": 216 | // https://sourceware.org/gdb/current/onlinedocs/gdb.html/Standard-Replies.html#Standard-Replies 217 | errs <- fmt.Errorf("packet %q is not implemented", packet) 218 | return 219 | case strings.HasPrefix("E ", response), 220 | strings.HasPrefix("E.", response): 221 | fmt.Fprintln(&stderr, response) 222 | default: 223 | fmt.Fprintln(&stdout, response) 224 | } 225 | } 226 | }() 227 | 228 | return func(s *script.State) (string, string, error) { 229 | err := <-errs 230 | return stdout.String(), stderr.String(), err 231 | }, nil 232 | }, 233 | ) 234 | 235 | func tryDial(ctx context.Context, network, addr string) (net.Conn, error) { 236 | var d net.Dialer 237 | for { 238 | conn, err := d.DialContext(ctx, network, addr) 239 | if errors.Is(err, unix.ECONNREFUSED) { 240 | select { 241 | case <-ctx.Done(): 242 | return nil, ctx.Err() 243 | case <-time.After(50 * time.Millisecond): 244 | continue 245 | } 246 | } else if err != nil { 247 | return nil, err 248 | } 249 | 250 | return conn, nil 251 | } 252 | } 253 | 254 | func gdbChecksum[T interface{ ~[]byte | ~string }](data T) (sum byte) { 255 | for _, d := range []byte(data) { 256 | sum += d 257 | } 258 | return 259 | } 260 | 261 | func gdbReadResponse(r *bufio.Reader) (string, error) { 262 | const ( 263 | ack byte = '+' 264 | start = '$' 265 | end = '#' 266 | checksum = iota 267 | ) 268 | 269 | var packet, csumBytes []byte 270 | next := ack 271 | read: 272 | for { 273 | c, err := r.ReadByte() 274 | if err != nil { 275 | return "", err 276 | } 277 | 278 | switch { 279 | case next == ack && c == ack: 280 | next = start 281 | continue 282 | 283 | case next == start && c == start: 284 | next = end 285 | continue 286 | 287 | case next == end && c == end: 288 | next = checksum 289 | continue 290 | 291 | case next == end: 292 | packet = append(packet, c) 293 | continue 294 | 295 | case next == checksum: 296 | csumBytes = append(csumBytes, c) 297 | if len(csumBytes) == 2 { 298 | break read 299 | } 300 | continue 301 | } 302 | 303 | return "", fmt.Errorf("expected %v got %v", rune(next), rune(c)) 304 | } 305 | 306 | tmp := make([]byte, 1) 307 | if _, err := hex.Decode(tmp, csumBytes); err != nil { 308 | return "", fmt.Errorf("decode checksum: %w", err) 309 | } 310 | 311 | haveCsum := tmp[0] 312 | wantCsum := gdbChecksum(packet) 313 | if wantCsum != haveCsum { 314 | return "", fmt.Errorf("invalid checksum 0x%x for %q (want 0x%2x)", haveCsum, string(packet), wantCsum) 315 | } 316 | 317 | return string(packet), nil 318 | } 319 | -------------------------------------------------------------------------------- /proc.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "context" 5 | "os" 6 | "os/exec" 7 | "time" 8 | ) 9 | 10 | // send SIGINT and wait for a while instead of SIGKILL. 11 | func commandWithGracefulTermination(ctx context.Context, name string, args ...string) *exec.Cmd { 12 | cmd := exec.CommandContext(ctx, name, args...) 13 | cmd.Cancel = func() error { 14 | return cmd.Process.Signal(os.Interrupt) 15 | } 16 | cmd.WaitDelay = 500 * time.Millisecond 17 | return cmd 18 | } 19 | -------------------------------------------------------------------------------- /qemu.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "bytes" 5 | "context" 6 | "errors" 7 | "fmt" 8 | "io" 9 | "net" 10 | "os" 11 | "os/exec" 12 | "os/user" 13 | "path/filepath" 14 | "regexp" 15 | "runtime" 16 | "slices" 17 | "strconv" 18 | "strings" 19 | "time" 20 | 21 | "github.com/u-root/u-root/pkg/qemu" 22 | "golang.org/x/sync/errgroup" 23 | "golang.org/x/sys/unix" 24 | ) 25 | 26 | const p9OverlayTag = "overlay" 27 | 28 | const envDisableKVM = "VIMTO_DISABLE_KVM" 29 | 30 | type sysctl struct { 31 | Name string 32 | Value string 33 | } 34 | 35 | // command is a binary to be executed under a different kernel. 36 | // 37 | // Mirrors exec.Cmd. 38 | type command struct { 39 | Kernel string 40 | // Memory to give to the VM. Passed verbatim as the QEMU -m flag. 41 | Memory string 42 | // SMP is passed verbatim as the QEMU -smp flag. 43 | SMP string 44 | // Enabled the GDB server on the specified address. The VM will be 45 | // paused on startup and wait for a "continue" from a gdb client. 46 | GDB string 47 | // Path to the binary to execute. 48 | Path string 49 | // Arguments passed to the binary. The first element is conventionally Path. 50 | Args []string 51 | // The directory to execute the binary in. Defaults to the current working 52 | // directory. 53 | Dir string 54 | // User to execute the command under. Defaults to the current user. 55 | User string 56 | // Env works like exec.Cmd.Env. 57 | Env []string 58 | Stdin io.Reader 59 | Stdout io.Writer 60 | Stderr io.Writer 61 | // A directory to overlay over the root filesystem. 62 | RootOverlay string 63 | 64 | // Sysctls set before the program is executed. 65 | Sysctls []sysctl 66 | 67 | // Resource limits. 68 | Rlimits map[int]unix.Rlimit 69 | 70 | // Commands to execute before and after Path is executed. 71 | Setup, Teardown []configCommand 72 | 73 | SerialPorts map[string]*os.File 74 | SharedDirectories []string 75 | 76 | cmd *exec.Cmd 77 | tasks errgroup.Group 78 | // Console contains boot diagnostics may only be read once tasks.Wait() has returned. 79 | console bytes.Buffer 80 | // Results may contain the result of an execution after tasks.Wait() has returned. 81 | results chan error 82 | // Write end of a pipe used to provide a blocking stdin to qemu. 83 | fakeStdin *os.File 84 | } 85 | 86 | func (cmd *command) Start(ctx context.Context) (err error) { 87 | closeOnError := func(c io.Closer) { 88 | if err != nil { 89 | c.Close() 90 | } 91 | } 92 | 93 | const controlPortName = "ctrl" 94 | const stdioPortName = "stdio" 95 | 96 | if cmd.cmd != nil { 97 | return errors.New("qemu: already started") 98 | } 99 | 100 | fds := &fdSets{} 101 | cds := &chardevs{} 102 | ports := &serialPorts{} 103 | virtioPorts := &virtioSerialPorts{make(map[chardev]string)} 104 | 105 | // The first serial port is always console, earlyprintk and SeaBIOS (on amd64) 106 | // output. SeaBIOS seems to always write to the first serial port. 107 | // Some platforms like the arm64 virt board only have a single console. 108 | consoleHost, consoleGuest, err := unixSocketpair() 109 | if err != nil { 110 | return err 111 | } 112 | defer closeOnError(consoleHost) 113 | defer consoleGuest.Close() 114 | 115 | consolePort := ports.add(cds.addFdSet(fds.addFile(consoleGuest))) 116 | 117 | // The second serial port is used for communication between host and guest. 118 | controlHost, controlGuest, err := unixSocketpair() 119 | if err != nil { 120 | return err 121 | } 122 | defer controlHost.Close() 123 | defer controlGuest.Close() 124 | 125 | virtioPorts.Chardevs[cds.addFdSet(fds.addFile(controlGuest))] = controlPortName 126 | 127 | // The third serial port is always stdio. The init process executes 128 | // subprocesses with this port as stdin, stdout and stderr. 129 | virtioPorts.Chardevs[chardev("stdio")] = stdioPortName 130 | 131 | devices := []qemu.Device{ 132 | qemu.ArbitraryArgs{ 133 | "-nodefaults", 134 | "-display", "none", 135 | "-cpu", "max", 136 | "-chardev", "stdio,id=stdio", 137 | "-m", cmd.Memory, 138 | "-smp", cmd.SMP, 139 | }, 140 | qemu.VirtioRandom{}, 141 | readOnlyRootfs{}, 142 | exitOnPanic{}, 143 | disablePS2Probing{}, 144 | disableRaidAutodetect{}, 145 | &p9Root{ 146 | "/", 147 | }, 148 | fds, 149 | cds, 150 | ports, 151 | virtioPorts, 152 | consoleOnSerialPort{consolePort}, 153 | } 154 | 155 | disableKVM := false 156 | if env := os.Getenv(envDisableKVM); env != "" { 157 | disableKVM, err = strconv.ParseBool(env) 158 | if err != nil { 159 | return fmt.Errorf("%s: %w", envDisableKVM, err) 160 | } 161 | } 162 | 163 | if !disableKVM { 164 | devices = append(devices, qemu.ArbitraryArgs{"-enable-kvm"}) 165 | } else if cmd.Stderr != nil { 166 | fmt.Fprintln(cmd.Stderr, "Warning: KVM disabled, performance will be limited.") 167 | } 168 | 169 | var binary string 170 | switch runtime.GOARCH { 171 | case "amd64": 172 | binary = "qemu-system-x86_64" 173 | devices = append(devices, earlyprintkOnSerialPort{consolePort}) 174 | 175 | case "arm64": 176 | binary = "qemu-system-aarch64" 177 | devices = append(devices, 178 | qemu.ArbitraryArgs{"-machine", "virt,gic-version=max"}, 179 | earlycon{}, 180 | ) 181 | 182 | default: 183 | return fmt.Errorf("unsupported GOARCH %s", runtime.GOARCH) 184 | } 185 | 186 | for name, port := range cmd.SerialPorts { 187 | virtioPorts.Chardevs[cds.addFdSet(fds.addFile(port))] = name 188 | } 189 | 190 | mountTags := make(map[string]string) 191 | for i, dir := range cmd.SharedDirectories { 192 | fstype, found := earlyMounts.pathIsBelowMount(dir) 193 | if found && fstype != "tmpfs" { 194 | return fmt.Errorf("directory %s is shadowed by %s mount in the guest", dir, fstype) 195 | } 196 | 197 | id := fmt.Sprintf("sd-9p-%d", i) 198 | mountTags[id] = dir 199 | devices = append(devices, &p9SharedDirectory{ 200 | ID: fsdev(id), 201 | Tag: id, 202 | Path: dir, 203 | }) 204 | } 205 | 206 | var rootOverlay string 207 | if cmd.RootOverlay != "" { 208 | rootOverlay, err = filepath.Abs(cmd.RootOverlay) 209 | if err != nil { 210 | return err 211 | } 212 | 213 | devices = append(devices, &p9SharedDirectory{ 214 | fsdev(p9OverlayTag), 215 | p9OverlayTag, 216 | rootOverlay, 217 | true, 218 | }) 219 | } 220 | 221 | if cmd.GDB != "" { 222 | devices = append(devices, gdbServer{cmd.GDB}) 223 | } 224 | 225 | uid := os.Geteuid() 226 | gid := os.Getegid() 227 | if cmd.User != "" { 228 | usr, err := user.Lookup(cmd.User) 229 | if err != nil { 230 | return err 231 | } 232 | 233 | uid, err = strconv.Atoi(usr.Uid) 234 | if err != nil { 235 | return fmt.Errorf("parse uid: %w", err) 236 | } 237 | 238 | gid, err = strconv.Atoi(usr.Gid) 239 | if err != nil { 240 | return fmt.Errorf("parse gid: %w", err) 241 | } 242 | } 243 | 244 | // Sanitize environment 245 | env := slices.Clone(cmd.Env) 246 | if env == nil { 247 | env = os.Environ() 248 | } 249 | 250 | env = slices.DeleteFunc(env, func(env string) bool { 251 | // Don't allow overriding TMPDIR from outside the VM. 252 | return strings.HasPrefix(env, "TMPDIR=") 253 | }) 254 | 255 | execCmd := execCommand{ 256 | cmd.Path, 257 | cmd.Args, 258 | cmd.Dir, 259 | uid, gid, 260 | env, 261 | cmd.Sysctls, 262 | cmd.Rlimits, 263 | cmd.Setup, cmd.Teardown, 264 | mountTags, 265 | } 266 | 267 | init, err := findExecutable() 268 | if err != nil { 269 | return err 270 | } 271 | 272 | // init has to go last since we stop processing of KArgs after. 273 | devices = append(devices, initWithArgs{ 274 | init, 275 | []string{stdioPortName, controlPortName}, 276 | }) 277 | 278 | qemuPath, err := exec.LookPath(binary) 279 | if err != nil { 280 | return err 281 | } 282 | 283 | qemuOpts := qemu.Options{ 284 | QEMUPath: qemuPath, 285 | Kernel: cmd.Kernel, 286 | Devices: devices, 287 | } 288 | 289 | qemuArgs, err := qemuOpts.Cmdline() 290 | if err != nil { 291 | return err 292 | } 293 | 294 | stdinIsDevZero := false 295 | if f, ok := cmd.Stdin.(*os.File); ok { 296 | stdinIsDevZero, err = fileIsDevZero(f) 297 | if err != nil { 298 | return fmt.Errorf("stdin: %w", err) 299 | } 300 | } 301 | 302 | stdin := cmd.Stdin 303 | if stdin == nil || stdinIsDevZero { 304 | // Writing to stdio in the guest hangs when stdin is /dev/zero. 305 | // Use an empty pipe instead. 306 | fakeStdinGuest, fakeStdinHost, err := os.Pipe() 307 | if err != nil { 308 | return fmt.Errorf("create fake stdin: %w", err) 309 | } 310 | defer fakeStdinGuest.Close() 311 | defer closeOnError(fakeStdinHost) 312 | 313 | stdin = fakeStdinGuest 314 | cmd.fakeStdin = fakeStdinHost 315 | } 316 | 317 | proc := commandWithGracefulTermination(ctx, qemuArgs[0], qemuArgs[1:]...) 318 | proc.Stdin = stdin 319 | proc.Stdout = cmd.Stdout 320 | proc.Stderr = cmd.Stderr 321 | proc.WaitDelay = time.Second 322 | proc.ExtraFiles = fds.Files 323 | 324 | control, err := net.FileConn(controlHost) 325 | if err != nil { 326 | return err 327 | } 328 | defer closeOnError(control) 329 | 330 | if err := proc.Start(); err != nil { 331 | return err 332 | } 333 | 334 | cmd.tasks.Go(func() error { 335 | defer consoleHost.Close() 336 | 337 | _, err = io.Copy(&cmd.console, consoleHost) 338 | return err 339 | }) 340 | 341 | results := make(chan error, 1) 342 | cmd.tasks.Go(func() error { 343 | comm := newRPC(control) 344 | defer comm.Close() 345 | 346 | if err := comm.Write(&execCmd, time.Now().Add(time.Second)); err != nil { 347 | return fmt.Errorf("write command: %w", err) 348 | } 349 | 350 | var result error 351 | if err := comm.Read(&result, time.Time{}); err != nil { 352 | return fmt.Errorf("decode execution result: %w", err) 353 | } 354 | 355 | results <- result 356 | return nil 357 | }) 358 | 359 | cmd.cmd = proc 360 | cmd.results = results 361 | return nil 362 | } 363 | 364 | func (cmd *command) Wait() error { 365 | defer cmd.fakeStdin.Close() 366 | 367 | if err := cmd.cmd.Wait(); err != nil { 368 | return fmt.Errorf("qemu: %w", err) 369 | } 370 | 371 | if err := cmd.tasks.Wait(); err != nil { 372 | if cmd.Stderr != nil { 373 | _, _ = io.Copy(cmd.Stderr, controlCodeStripper{&cmd.console}) 374 | } 375 | return err 376 | } 377 | 378 | return <-cmd.results 379 | } 380 | 381 | // Control codes emitted by the SeaBIOS boot sequence. 382 | // 383 | // See https://www.man7.org/linux/man-pages/man4/console_codes.4.html 384 | var seBIOSEscapeCodes = regexp.MustCompile("\x1b(c|\\[\\?7l|\\[2J|\\[0m)") 385 | 386 | type controlCodeStripper struct { 387 | io.Reader 388 | } 389 | 390 | func (s controlCodeStripper) Read(buf []byte) (int, error) { 391 | n, err := s.Reader.Read(buf) 392 | n = copy(buf, seBIOSEscapeCodes.ReplaceAll(buf[:n], nil)) 393 | return n, err 394 | } 395 | 396 | type execCommand struct { 397 | Path string 398 | Args []string 399 | Dir string 400 | Uid, Gid int 401 | Env []string 402 | Sysctls []sysctl 403 | Rlimits map[int]unix.Rlimit 404 | Setup, Teardown []configCommand 405 | MountTags map[string]string // map[tag]path 406 | } 407 | 408 | type guestExitError struct { 409 | ExitCode int 410 | } 411 | 412 | func (gee *guestExitError) Error() string { 413 | return fmt.Sprintf("guest: exit %d", gee.ExitCode) 414 | } 415 | 416 | type genericGuestError struct { 417 | Message string 418 | } 419 | 420 | func (gge *genericGuestError) Error() string { 421 | return fmt.Sprintf("guest: %s", gge.Message) 422 | } 423 | 424 | type initWithArgs struct { 425 | path string 426 | args []string 427 | } 428 | 429 | func (i initWithArgs) Cmdline() []string { 430 | return nil 431 | } 432 | 433 | func (i initWithArgs) KArgs() []string { 434 | kargs := []string{"init=" + i.path, "--"} 435 | for _, arg := range i.args { 436 | if arg == "" { 437 | kargs = append(kargs, `""`) 438 | } else { 439 | kargs = append(kargs, arg) 440 | } 441 | } 442 | return kargs 443 | } 444 | 445 | type fdSet string 446 | 447 | // fdSets manages fdsets. 448 | // 449 | // Assumes that files is passed in exec.Cmd.ExtraFiles. 450 | type fdSets struct { 451 | Files []*os.File 452 | } 453 | 454 | // addFile a fd backed chardev. 455 | // 456 | // Returns the chardev id allocated for the file. 457 | func (cds *fdSets) addFile(f *os.File) fdSet { 458 | const idFmt = "/dev/fdset/%d" 459 | 460 | for i, file := range cds.Files { 461 | if f == file { 462 | return fdSet(fmt.Sprintf(idFmt, i)) 463 | } 464 | } 465 | 466 | id := fdSet(fmt.Sprintf(idFmt, len(cds.Files))) 467 | cds.Files = append(cds.Files, f) 468 | return id 469 | } 470 | 471 | func (cds *fdSets) Cmdline() []string { 472 | const execFirstExtraFd = 3 473 | 474 | var args []string 475 | for i := range cds.Files { 476 | fd := execFirstExtraFd + i 477 | args = append(args, 478 | "-add-fd", fmt.Sprintf("fd=%d,set=%d", fd, i), 479 | ) 480 | } 481 | return args 482 | } 483 | 484 | func (*fdSets) KArgs() []string { return nil } 485 | 486 | type chardev string 487 | 488 | // chardevs manages character devices. 489 | type chardevs struct { 490 | Pipes []fdSet 491 | } 492 | 493 | func (cds *chardevs) addFdSet(fds fdSet) chardev { 494 | id := chardev(fmt.Sprintf("cd-%d", len(cds.Pipes))) 495 | cds.Pipes = append(cds.Pipes, fds) 496 | return id 497 | } 498 | 499 | func (cds *chardevs) Cmdline() []string { 500 | mux := make(map[fdSet]int) 501 | for _, fds := range cds.Pipes { 502 | mux[fds]++ 503 | } 504 | 505 | var args []string 506 | for i, fds := range cds.Pipes { 507 | id := chardev(fmt.Sprintf("cd-%d", i)) 508 | 509 | arg := fmt.Sprintf("pipe,id=%s,path=%s", id, fds) 510 | if mux[fds] > 1 { 511 | arg += ",mux=on" 512 | } 513 | 514 | args = append(args, "-chardev", arg) 515 | } 516 | return args 517 | } 518 | 519 | func (*chardevs) KArgs() []string { return nil } 520 | 521 | // A "simple" serial port using a character device. 522 | // 523 | // Probably more portable than virtio-serial, but doesn't allow naming. 524 | type serialPorts struct { 525 | Chardevs []chardev 526 | } 527 | 528 | func (ports *serialPorts) add(cd chardev) string { 529 | pattern := "ttyS%d" 530 | if runtime.GOARCH == "arm64" { 531 | pattern = "ttyAMA%d" 532 | } 533 | 534 | port := fmt.Sprintf(pattern, len(ports.Chardevs)) 535 | ports.Chardevs = append(ports.Chardevs, cd) 536 | return port 537 | } 538 | 539 | func (ports *serialPorts) Cmdline() []string { 540 | var args []string 541 | for _, chardev := range ports.Chardevs { 542 | args = append(args, "-serial", fmt.Sprintf("chardev:%s", chardev)) 543 | } 544 | return args 545 | } 546 | 547 | func (*serialPorts) KArgs() []string { return nil } 548 | 549 | type virtioSerialPorts struct { 550 | // A map of character devices to serial port names. 551 | // 552 | // Inside the VM, port names can be accessed via /sys/class/virtio-ports. 553 | Chardevs map[chardev]string 554 | } 555 | 556 | func (vios *virtioSerialPorts) Cmdline() []string { 557 | if len(vios.Chardevs) == 0 { 558 | return nil 559 | } 560 | 561 | args := []string{ 562 | // There seems to be an off by one error with max_ports. 563 | "-device", fmt.Sprintf("virtio-serial,max_ports=%d", len(vios.Chardevs)+1), 564 | } 565 | for dev, name := range vios.Chardevs { 566 | args = append(args, 567 | "-device", fmt.Sprintf("virtserialport,chardev=%s,name=%s", dev, name), 568 | ) 569 | } 570 | return args 571 | } 572 | 573 | func (*virtioSerialPorts) KArgs() []string { return nil } 574 | 575 | // Force the root fs to be read-only. 576 | type readOnlyRootfs struct{} 577 | 578 | func (readOnlyRootfs) Cmdline() []string { 579 | return nil 580 | } 581 | 582 | func (readOnlyRootfs) KArgs() []string { 583 | return []string{"ro"} 584 | } 585 | 586 | // Make qemu exit on panic instead of pausing. 587 | type exitOnPanic struct{} 588 | 589 | func (exitOnPanic) Cmdline() []string { 590 | return []string{"-no-reboot"} 591 | } 592 | 593 | func (exitOnPanic) KArgs() []string { 594 | return []string{"panic=-1"} 595 | } 596 | 597 | type consoleOnSerialPort struct { 598 | // Linux name of the serial port, e.g. ttyS0. 599 | Port string 600 | } 601 | 602 | func (consoleOnSerialPort) Cmdline() []string { return nil } 603 | 604 | func (cs consoleOnSerialPort) KArgs() []string { 605 | return []string{ 606 | fmt.Sprintf("console=%s,115200", cs.Port), 607 | } 608 | } 609 | 610 | type earlyprintkOnSerialPort struct { 611 | Port string 612 | } 613 | 614 | func (earlyprintkOnSerialPort) Cmdline() []string { return nil } 615 | 616 | func (epk earlyprintkOnSerialPort) KArgs() []string { 617 | return []string{ 618 | fmt.Sprintf("earlyprintk=serial,%s,115200", epk.Port), 619 | } 620 | } 621 | 622 | type earlycon struct { 623 | } 624 | 625 | func (earlycon) Cmdline() []string { return nil } 626 | 627 | func (epk earlycon) KArgs() []string { 628 | return []string{ 629 | "earlycon=pl011,0x9000000", 630 | } 631 | } 632 | 633 | // Disable PS/2 protocol probing to speed up booting. 634 | type disablePS2Probing struct{} 635 | 636 | func (disablePS2Probing) Cmdline() []string { 637 | return nil 638 | } 639 | 640 | func (disablePS2Probing) KArgs() []string { 641 | return []string{"psmouse.proto=exps"} 642 | } 643 | 644 | // Disable RAID autodetection to speed up booting. 645 | type disableRaidAutodetect struct{} 646 | 647 | func (disableRaidAutodetect) Cmdline() []string { 648 | return nil 649 | } 650 | 651 | func (disableRaidAutodetect) KArgs() []string { 652 | return []string{"raid=noautodetect"} 653 | } 654 | 655 | type gdbServer struct { 656 | Listen string 657 | } 658 | 659 | func (gs gdbServer) Cmdline() []string { 660 | return []string{"-gdb", "tcp:" + gs.Listen, "-S"} 661 | } 662 | 663 | func (gdbServer) KArgs() []string { 664 | return []string{"nokaslr"} 665 | } 666 | 667 | type fsdev string 668 | 669 | type p9Root struct { 670 | Path string 671 | } 672 | 673 | func (p9r *p9Root) Cmdline() []string { 674 | return []string{ 675 | // Need security_model=none due to https://gitlab.com/qemu-project/qemu/-/issues/173 676 | "-fsdev", fmt.Sprintf("local,id=rootdrv,path=%s,readonly=on,security_model=none,multidevs=remap", p9r.Path), 677 | "-device", "virtio-9p-pci,fsdev=rootdrv,mount_tag=/dev/root", 678 | } 679 | } 680 | 681 | func (*p9Root) KArgs() []string { 682 | return []string{ 683 | "root=/dev/root", 684 | "rootfstype=9p", 685 | "rootflags=" + default9POptions, 686 | } 687 | } 688 | 689 | type p9SharedDirectory struct { 690 | ID fsdev 691 | Tag string 692 | Path string 693 | ReadOnly bool 694 | } 695 | 696 | func (p9sd *p9SharedDirectory) Cmdline() []string { 697 | readOnly := "off" 698 | if p9sd.ReadOnly { 699 | readOnly = "on" 700 | } 701 | 702 | return []string{ 703 | // Need security_model=none due to https://gitlab.com/qemu-project/qemu/-/issues/173 704 | "-fsdev", fmt.Sprintf("local,id=%s,path=%s,readonly=%s,security_model=none,multidevs=remap", p9sd.ID, p9sd.Path, readOnly), 705 | "-device", fmt.Sprintf("virtio-9p-pci,fsdev=%s,mount_tag=%s", p9sd.ID, p9sd.Tag), 706 | } 707 | } 708 | 709 | func (*p9SharedDirectory) KArgs() []string { 710 | return nil 711 | } 712 | -------------------------------------------------------------------------------- /qemu_test.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "bytes" 5 | "context" 6 | "os" 7 | "testing" 8 | "time" 9 | 10 | "github.com/creack/pty/v2" 11 | "github.com/go-quicktest/qt" 12 | "golang.org/x/sys/unix" 13 | ) 14 | 15 | func TestQemuTTY(t *testing.T) { 16 | t.Parallel() 17 | 18 | pty, tty, err := pty.Open() 19 | qt.Assert(t, qt.IsNil(err)) 20 | defer pty.Close() 21 | defer tty.Close() 22 | 23 | get := func(f *os.File) (*unix.Termios, error) { 24 | return fileControl(f, func(fd uintptr) (*unix.Termios, error) { 25 | return unix.IoctlGetTermios(int(fd), unix.TCGETS) 26 | }) 27 | } 28 | 29 | old, err := get(tty) 30 | qt.Assert(t, qt.IsNil(err)) 31 | 32 | image := mustFetchKernelImage(t) 33 | 34 | r, w, err := os.Pipe() 35 | qt.Assert(t, qt.IsNil(err)) 36 | defer r.Close() 37 | defer w.Close() 38 | 39 | var stderr bytes.Buffer 40 | cmd := command{ 41 | Kernel: image.Kernel, 42 | Memory: "128M", 43 | SMP: "cpus=1", 44 | Path: "sh", 45 | Args: []string{"sh", "-c", "echo a; sleep 60"}, 46 | Stdin: tty, 47 | Stdout: w, 48 | Stderr: &stderr, 49 | } 50 | 51 | ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) 52 | qt.Assert(t, qt.IsNil(cmd.Start(ctx))) 53 | 54 | // Make sure we get EOF if the VM exits. 55 | qt.Assert(t, qt.IsNil(w.Close())) 56 | 57 | // qemu changes the tty settings before changing signal handlers. Cancelling 58 | // too early means that the process is killed immediately, which aborts 59 | // the shutdown. 60 | // Wait for the vm to write to stdout as a proxy for signal handlers 61 | // being up. 62 | _, err = r.Read(make([]byte, 1)) 63 | qt.Assert(t, qt.IsNil(err)) 64 | 65 | new, err := get(tty) 66 | qt.Assert(t, qt.IsNil(err)) 67 | qt.Assert(t, qt.Not(qt.Equals(*new, *old)), qt.Commentf("termios should change")) 68 | 69 | cancel() 70 | 71 | err = cmd.Wait() 72 | if stderr.Len() > 0 { 73 | t.Log(stderr.String()) 74 | } 75 | qt.Assert(t, qt.ErrorIs(err, context.Canceled)) 76 | 77 | // Ensure that the tty settings were restored. 78 | new, err = get(tty) 79 | qt.Assert(t, qt.IsNil(err)) 80 | qt.Assert(t, qt.Equals(*new, *old), qt.Commentf("termios should be restored")) 81 | } 82 | -------------------------------------------------------------------------------- /rpc.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "encoding/gob" 5 | "io" 6 | "time" 7 | ) 8 | 9 | func init() { 10 | gob.Register((*guestExitError)(nil)) 11 | gob.Register((*genericGuestError)(nil)) 12 | } 13 | 14 | type conn interface { 15 | io.Reader 16 | io.Writer 17 | io.Closer 18 | SetReadDeadline(time.Time) error 19 | SetWriteDeadline(time.Time) error 20 | } 21 | 22 | type rpc struct { 23 | conn conn 24 | enc *gob.Encoder 25 | dec *gob.Decoder 26 | } 27 | 28 | func newRPC(conn conn) *rpc { 29 | return &rpc{ 30 | conn, 31 | gob.NewEncoder(conn), 32 | gob.NewDecoder(conn), 33 | } 34 | } 35 | 36 | func (r *rpc) Close() error { 37 | return r.conn.Close() 38 | } 39 | 40 | func (r *rpc) Read(v any, deadline time.Time) error { 41 | if err := r.conn.SetReadDeadline(deadline); err != nil { 42 | return err 43 | } 44 | 45 | return r.dec.Decode(v) 46 | } 47 | 48 | func (r *rpc) Write(v any, deadline time.Time) error { 49 | if err := r.conn.SetWriteDeadline(deadline); err != nil { 50 | return err 51 | } 52 | 53 | return r.enc.Encode(v) 54 | } 55 | -------------------------------------------------------------------------------- /static_cgo.go: -------------------------------------------------------------------------------- 1 | //go:build cgo 2 | 3 | package main 4 | 5 | const staticBuild = false 6 | -------------------------------------------------------------------------------- /static_nocgo.go: -------------------------------------------------------------------------------- 1 | //go:build !cgo 2 | 3 | package main 4 | 5 | const staticBuild = true 6 | -------------------------------------------------------------------------------- /testdata/coredumps.txt: -------------------------------------------------------------------------------- 1 | config kernel="${KERNEL}" 2 | 3 | # Ensure that ulimit inside the VM is raised. 4 | vimto exec -- sh -c 'ulimit -c' 5 | stdout unlimited 6 | 7 | # Trigger a core dump inside the VM. 8 | ! vimto exec -- timeout -s QUIT 0.1s sleep 10 9 | stdout 'dumped core' 10 | glob-exists core-* 11 | -------------------------------------------------------------------------------- /testdata/default.toml: -------------------------------------------------------------------------------- 1 | kernel = "" 2 | memory = "size=128M" 3 | smp = "cpus=1" 4 | user = "" 5 | setup = [] 6 | teardown = [] 7 | -------------------------------------------------------------------------------- /testdata/gdb.txt: -------------------------------------------------------------------------------- 1 | config kernel="${KERNEL}" 2 | 3 | vimto -gdb exec true & 4 | 5 | # The gdbstub is reachable and accepts commands. 6 | gdb localhost:1234 c 7 | stdout W00 8 | 9 | # Output contains instructions how to connect. 10 | wait 11 | stdout 'target remote' 12 | -------------------------------------------------------------------------------- /testdata/go-test.txt: -------------------------------------------------------------------------------- 1 | config kernel="${KERNEL}" 2 | new-tmp 3 | 4 | # Exit status and stdout, stderr of tests is correctly forwarded. 5 | vimto -- go test -run Success -v . 6 | stdout TestSuccess 7 | 8 | ! vimto -- go test -run Failure . 9 | stdout TestFailure 10 | 11 | # We can pass various command line arguments through the test runner. 12 | vimto -smp 2 -- go test -run Success . 13 | vimto -smp=2 -- go test -run Success . 14 | 15 | # Running a test with coverage enabled works. 16 | vimto -- go test -run Success -coverprofile=cover.out . 17 | exists cover.out 18 | 19 | # Running a test with race enabled works. 20 | vimto -- go test -run Success -race . 21 | 22 | # The test binary is preserved if a test crashes with a coredump. 23 | env GOTRACEBACK=crash 24 | ! vimto -- go test -run Panic . 25 | exists test.test 26 | 27 | # Building packages within the VM works. 28 | vimto exec go build . 29 | 30 | -- go.mod -- 31 | 32 | module test 33 | 34 | go 1.21 35 | 36 | -- main.go -- 37 | 38 | package main 39 | 40 | func main() {} 41 | 42 | -- main_test.go -- 43 | 44 | package main 45 | 46 | import "testing" 47 | 48 | func TestSuccess(t *testing.T) { 49 | t.Log("Zaphod Beeblebrox is a hoopy frood") 50 | } 51 | 52 | func TestFailure(t *testing.T) { 53 | t.Error("Groop, I implore thee, my foonting turlingdromes") 54 | } 55 | 56 | func TestPanic(t *testing.T) { 57 | panic("oh no!") 58 | } 59 | -------------------------------------------------------------------------------- /testdata/help.txt: -------------------------------------------------------------------------------- 1 | vimto -help 2 | stderr Usage 3 | 4 | vimto exec -help 5 | stderr Usage 6 | -------------------------------------------------------------------------------- /testdata/kvm.txt: -------------------------------------------------------------------------------- 1 | config kernel="${KERNEL}" 2 | 3 | # Ensure that running without KVM is supported. 4 | env VIMTO_DISABLE_KVM=true 5 | vimto exec true 6 | stderr 'KVM disabled' 7 | -------------------------------------------------------------------------------- /testdata/lifecycle.txt: -------------------------------------------------------------------------------- 1 | config kernel="${KERNEL}" 'setup=["touch ''foo bar''"]' 'teardown=["rm ''foo bar''"]' 2 | 3 | # Setup and teardown programs are executed. 4 | vimto exec -- stat 'foo bar' 5 | ! exists foo 6 | 7 | config kernel="${KERNEL}" 'setup=["/bin/false"]' 8 | ! vimto exec -- true 9 | stderr /bin/false 10 | 11 | config kernel="${KERNEL}" 'teardown=["/bin/false"]' 12 | ! vimto exec -- true 13 | stderr /bin/false 14 | -------------------------------------------------------------------------------- /testdata/oci.txt: -------------------------------------------------------------------------------- 1 | # Load the kernel from an OCI image. 2 | config kernel="${IMAGE}" 3 | vimto exec true 4 | -------------------------------------------------------------------------------- /testdata/overlay.txt: -------------------------------------------------------------------------------- 1 | mkdir ./kernel/boot 2 | cp ${KERNEL} ./kernel/boot/ 3 | config kernel="./kernel" 4 | 5 | # Booting via a kernel image or directory adds additonal files as an overlay. 6 | vimto exec /bin/cat /vimto-test 7 | stdout ^IRnRnZPbNY$ 8 | 9 | -- kernel/vimto-test -- 10 | 11 | IRnRnZPbNY 12 | -------------------------------------------------------------------------------- /testdata/vimto.txt: -------------------------------------------------------------------------------- 1 | config kernel="${KERNEL}" 2 | 3 | # We can execute binary, with the kernel being read from the config. Exit status 4 | # is forwarded correctly. 5 | vimto exec true 6 | ! vimto exec false 7 | ! stderr warning 8 | 9 | # Flag overrides config file. 10 | ! vimto -kernel ./bogus exec true 11 | 12 | # Output is forwarded correctly. 13 | vimto exec sh -c 'echo testing' 14 | stdout testing 15 | 16 | vimto exec sh -c 'echo testing >&2' 17 | stdout testing # we don't have separate stderr at the moment. 18 | 19 | # We can pass various command line arguments. Include flags which contain 20 | # their value separated with a '='. 21 | vimto -smp 2 exec true 22 | vimto -memory 96M exec true 23 | 24 | # Binaries are executed with the appropriate user. 25 | vimto exec -- id -u 26 | stdout ^${UID}$ 27 | 28 | vimto -sudo exec -- id -u 29 | stdout ^0$ 30 | 31 | # vm.sudo doesn't accept arguments. 32 | ! vimto -sudo=false exec -- true 33 | 34 | # Current working directory is preserved. 35 | vimto exec -- pwd 36 | stdout ^${WORKDIR} 37 | --------------------------------------------------------------------------------