├── containers ├── register │ └── .gitkeep └── latest │ ├── Dockerfile │ └── register.sh ├── docs ├── images │ └── forked-repo-travis-ci-settings.png ├── examples.md ├── compatible_images.md └── developers_guide.md ├── .gitignore ├── .github ├── ISSUE_TEMPLATE.md └── workflows │ └── actions.yml ├── LICENSE ├── CONTRIBUTING.md ├── run.sh ├── test.sh └── README.md /containers/register/.gitkeep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /docs/images/forked-repo-travis-ci-settings.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/multiarch/qemu-user-static/HEAD/docs/images/forked-repo-travis-ci-settings.png -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | *.swp 2 | *~ 3 | # Ignore generated files 4 | /containers/latest/qemu-*-static 5 | /containers/register/Dockerfile 6 | /containers/register/register.sh 7 | /containers/x86_64_qemu-*/ 8 | /releases 9 | -------------------------------------------------------------------------------- /containers/latest/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM busybox 2 | ENV QEMU_BIN_DIR=/usr/bin 3 | ADD ./register.sh /register 4 | ADD https://raw.githubusercontent.com/qemu/qemu/e75941331e4cdc05878119e08635ace437aae721/scripts/qemu-binfmt-conf.sh /qemu-binfmt-conf.sh 5 | RUN chmod +x /qemu-binfmt-conf.sh 6 | COPY qemu-*-static /usr/bin/ 7 | ENTRYPOINT ["/register"] 8 | -------------------------------------------------------------------------------- /containers/latest/register.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | QEMU_BIN_DIR=${QEMU_BIN_DIR:-/usr/bin} 4 | 5 | 6 | if [ ! -d /proc/sys/fs/binfmt_misc ]; then 7 | echo "No binfmt support in the kernel." 8 | echo " Try: '/sbin/modprobe binfmt_misc' from the host" 9 | exit 1 10 | fi 11 | 12 | 13 | if [ ! -f /proc/sys/fs/binfmt_misc/register ]; then 14 | mount binfmt_misc -t binfmt_misc /proc/sys/fs/binfmt_misc 15 | fi 16 | 17 | 18 | if [ "${1}" = "--reset" ]; then 19 | shift 20 | find /proc/sys/fs/binfmt_misc -type f -name 'qemu-*' -exec sh -c 'echo -1 > {}' \; 21 | fi 22 | 23 | exec /qemu-binfmt-conf.sh --qemu-suffix "-static" --qemu-path "${QEMU_BIN_DIR}" $@ 24 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE.md: -------------------------------------------------------------------------------- 1 | 6 | 7 | **Is this a bug report, feature (enhancement) request or question? (leave only one on its own line)** 8 | 9 | /kind bug 10 | 11 | /kind enhancement 12 | 13 | /kind question 14 | 15 | **Description:** 16 | 17 | 20 | 21 | **Steps to reproduce the issue:** 22 | 23 | 1. 24 | 25 | 2. 26 | 27 | 3. 28 | 29 | **Describe the results you received:** 30 | 31 | **Describe the results you expected:** 32 | 33 | **Environment:** 34 | 35 | 36 | * QEMU version: (if you can know it): 37 | * Container application: Docker/Podman/Singularity (Leave only one) 38 | 39 | **Output of `docker version`, `podman version` or `singularity version`** 40 | 41 | ``` 42 | (paste your output here) 43 | ``` 44 | 45 | **Additional information optionally:** 46 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2015-2016 Manfred Touron 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /docs/examples.md: -------------------------------------------------------------------------------- 1 | # Sister projects, examples & articles 2 | 3 | ## Sister projects 4 | 5 | * [qemu-user-static (qus) and docker](https://github.com/dbhi/qus) 6 | 7 | ## Examples and articles 8 | 9 | * [junaruga/ci-multi-arch-test](https://github.com/junaruga/ci-multi-arch-test) and the [Travis CI](https://travis-ci.org/junaruga/ci-multi-arch-test) 10 | 11 | ## Examples and articles [DEPRECATED]? 12 | 13 | * Scaleway's build system: 14 | * [scaleway/image-tools](https://github.com/scaleway/image-tools) 15 | * [scaleway/image-builder](https://github.com/scaleway/image-builder) 16 | * [Docker + multiarch = <3](https://manfredtouron.com/2016/01/28/docker-multiarch/) (Release blog post) 17 | * Introduction article: [eyskens.me/multiarch-docker-images](https://eyskens.me/multiarch-docker-images/) 18 | * Dockerized C benchmarks for both ARM and amd64 hardware: [luxas/benchmark](https://github.com/luxas/benchmark) 19 | * Standalone image example: [meyskens/multiarch-nodejs](https://github.com/meyskens/multiarch-nodejs) 20 | * RaspberryPI + haskell hacks: 21 | * [TGOlson/rpi-haskell](https://github.com/TGOlson/rpi-haskell) 22 | * [TGOlson/rpi-haskell-classy](https://github.com/TGOlson/rpi-haskell-classy) 23 | * Music notation software: [musescore/MuseScore](https://github.com/musescore/MuseScore) 24 | -------------------------------------------------------------------------------- /docs/compatible_images.md: -------------------------------------------------------------------------------- 1 | # Compatible images [DEPRECATED] 2 | 3 | * [hub.docker.com/r/multiarch/](https://hub.docker.com/r/multiarch/) 4 | * alpine: [Docker Hub](https://hub.docker.com/r/multiarch/alpine/), [GitHub](https://github.com/multiarch/alpine) 5 | * debian-debootstrap: [Docker Hub](https://hub.docker.com/r/multiarch/debian-debootstrap/), [GitHub](https://github.com/multiarch/debian-debootstrap) 6 | * ubuntu-core: [Docker Hub](https://hub.docker.com/r/multiarch/ubuntu-core/), [GitHub](https://github.com/multiarch/ubuntu-core) 7 | * ubuntu-debootstrap: [Docker Hub](https://hub.docker.com/r/multiarch/ubuntu-debootstrap/), [GitHub](https://github.com/multiarch/ubuntu-debootstrap) 8 | * fedora: [Docker Hub](https://hub.docker.com/r/multiarch/fedora/), [GitHub](https://github.com/multiarch/fedora) 9 | * centos: [Docker Hub](https://hub.docker.com/r/multiarch/centos/), [GitHub](https://github.com/multiarch/centos) 10 | * busybox: [Docker Hub](https://hub.docker.com/r/multiarch/busybox/), [GitHub](https://github.com/multiarch/busybox) 11 | 12 | Organizations with some (if not all) multiarch images: 13 | 14 | * [hub.docker.com/u/multiarch](https://hub.docker.com/u/multiarch/) 15 | * [hub.docker.com/u/scaleway](https://hub.docker.com/u/scaleway/) 16 | * [hub.docker.com/u/meyskens](https://hub.docker.com/u/meyskens/) 17 | -------------------------------------------------------------------------------- /.github/workflows/actions.yml: -------------------------------------------------------------------------------- 1 | name: actions 2 | on: 3 | push: 4 | branches: 5 | - master 6 | paths: 7 | - containers/** 8 | - .github/workflows/actions.yml 9 | - "*.sh" 10 | pull_request: 11 | branches: 12 | - master 13 | paths: 14 | - containers/** 15 | - .github/workflows/actions.yml 16 | - "*.sh" 17 | jobs: 18 | qemu-user-static: 19 | runs-on: ubuntu-latest 20 | env: 21 | VERSION: 7.2.0-1 22 | ORIGIN_VERSION: 7.2+dfsg-1~bpo11+2 23 | steps: 24 | - uses: actions/checkout@v2 25 | - name: Set variables 26 | run: | 27 | echo "DOCKER_REPO=docker.io/$GITHUB_REPOSITORY" >> $GITHUB_ENV 28 | - name: Build 29 | run: | 30 | wget --content-disposition http://ftp.de.debian.org/debian/pool/main/q/qemu/qemu-user-static_${{ env.ORIGIN_VERSION }}_amd64.deb 31 | dpkg-deb -R qemu-user-static_*.deb releases 32 | ./run.sh -t "${{ env.VERSION }}" -r "${{ env.GITHUB_REPOSITORY }}" -d "${{ env.DOCKER_REPO }}" 33 | - name: Test 34 | run: ./test.sh -d "${{ env.DOCKER_REPO }}" 35 | - name: Publish 36 | if: github.ref == 'refs/heads/master' 37 | run: | 38 | docker login -u ${{ secrets.DOCKER_USERNAME }} -p ${{ secrets.DOCKER_TOKEN }} 39 | docker push --all-tags ${{ env.DOCKER_REPO }} 40 | - name: Create Release 41 | if: github.ref == 'refs/heads/master' 42 | uses: ncipollo/release-action@v1 43 | with: 44 | name: "v${{ env.VERSION }}" 45 | tag: "v${{ env.VERSION }}" 46 | body: | 47 | # `qemu-*-static` @ ${{ env.VERSION }} 48 | artifacts: "releases/usr/bin/*.tar.gz,releases/usr/bin/qemu-*-static" 49 | draft: false 50 | prerelease: false 51 | token: ${{ secrets.GITHUB_TOKEN }} 52 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing to multiarch/qemu-user-static 2 | 3 | Your contributions such as reporting a bug and sending pull-request are very wellcome! Thank you. 4 | 5 | ## Did you find a bug? 6 | 7 | * Ensure the bug was not already reported by searching on GitHub under Issues. 8 | * multiarch/qemu-user-static is a collection of containers to enable people to emulate multi-architecture containers by using qemu-user-static (= a collection of QEMU's user mode static binaries `qemu-$arch-static`) [1] and binfmt_misc [2]. This repository is not QEMU project's one. If you find a bug about them, you can visit the website [1][2] to report an issue on the projects. 9 | 10 | ## How to send pull-request 11 | 12 | 1. Fork the repository: https://github.com/multiarch/qemu-user-static . Ex. https://github.com/junaruga/qemu-user-static 13 | 2. This repository is using Github Actions. You can test your modified code on your forked repository before sending a pull-requeste. 14 | 3. If you want to test pushing created container images, 15 | * You need to have your own container repository such as DockerHub or Quay.io. Ex. https://quay.io/repository/junaruga/qemu-user-static 16 | * You need to set environment variables `DOCKER_USERNAME` and `DOCKER_PASSWORD` on your repository's Settings page. Please remember it is better to set `DOCKER_PASSWORD` without displaying the value for your security. 17 | 4. You also need to have your https://quay.io/repository/junaruga/qemu-user-static 18 | 5. You need to edit `.github/workflows/actions.yml` with your container repository. This step can be improved in the future. 19 | 6. Below is an example of how to test with your container repository. 20 | 7. Check Github Actions's log, and ensure the container images are created. 21 | 8. You are ready to send the pull-request! 22 | 23 | ## References 24 | 25 | * [1] QEMU: https://www.qemu.org/ 26 | * [2] binfmt-misc: https://www.kernel.org/doc/html/latest/admin-guide/binfmt-misc.html 27 | 28 | -------------------------------------------------------------------------------- /run.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -xe 3 | 4 | # A POSIX variable 5 | OPTIND=1 # Reset in case getopts has been used previously in the shell. 6 | 7 | while getopts "r:t:d:" opt; do 8 | case "$opt" in 9 | r) REPO=$OPTARG 10 | ;; 11 | t) TAG_VER=$OPTARG 12 | ;; 13 | d) DOCKER_REPO=$OPTARG 14 | ;; 15 | esac 16 | done 17 | 18 | shift $((OPTIND-1)) 19 | 20 | [ "$1" = "--" ] && shift 21 | 22 | # Build container images creating the directory. 23 | # containers/ 24 | # latest/ - An image including /usr/bin/qemu-$arch-status and /register script. 25 | # ${from_arch}_qemu-${to_arch}/ - Images including /usr/bin/qemu-$arch-status 26 | # register/ - An image including /register script. 27 | 28 | from_arch="x86_64" 29 | root_dir=$(pwd) 30 | out_dir="containers" 31 | releases_dir="releases/usr/bin/" 32 | 33 | cd ${releases_dir} 34 | for file in *; do 35 | tar -czf $file.tar.gz $file; 36 | cp $file.tar.gz x86_64_$file.tar.gz 37 | done 38 | cd ${root_dir} 39 | 40 | # Generate register files. 41 | cp -p "${out_dir}/latest/register.sh" "${out_dir}/register/" 42 | cp -p "${out_dir}/latest/Dockerfile" "${out_dir}/register/" 43 | # Comment out the line to copy qemu-*-static not to provide those. 44 | sed -i '/^COPY qemu/ s/^/#/' "${out_dir}/register/Dockerfile" 45 | 46 | for file in ${releases_dir}* 47 | do 48 | if [[ $file =~ qemu-(.+)-static ]]; then 49 | to_arch=${BASH_REMATCH[1]} 50 | if [ "$from_arch" != "$to_arch" ]; then 51 | work_dir="${out_dir}/${from_arch}_qemu-${to_arch}" 52 | mkdir -p "${work_dir}" 53 | cp -p "${releases_dir}qemu-${to_arch}-static" ${work_dir} 54 | cp -p "${work_dir}/qemu-${to_arch}-static" "${out_dir}/latest/" 55 | cat > ${work_dir}/Dockerfile -<&2 16 | exit 1 17 | fi 18 | 19 | # Test cases 20 | 21 | # ------------------------------------------------ 22 | # multiarch/qemu-user-static image 23 | 24 | # It should register binfmt_misc entry with 'flags: F' 25 | # by given "-p yes" option. 26 | docker run --rm --privileged ${DOCKER_REPO} --reset -p yes 27 | cat /proc/sys/fs/binfmt_misc/qemu-aarch64 28 | grep -q '^flags: F$' /proc/sys/fs/binfmt_misc/qemu-aarch64 29 | 30 | # It should output the result of "uname -m". 31 | docker pull arm64v8/ubuntu 32 | docker run --rm -t arm64v8/ubuntu uname -m 33 | # It should install a package. 34 | docker build --rm -t "test/latest/ubuntu" -< /proc/sys/fs/binfmt_misc/register` to add a binary format entry. 71 | * `# echo -1 > /proc/sys/fs/binfmt_misc/qemu-$arch` to remove a qemu binary format entry. 72 | * If the entry file's `flags` is empty, the exsistance of the interpreter is checked at run time. 73 | * If the entry file's `flags` is `flags: F`, the existance of the interpreter is checked when registering the entry. 74 | 75 | ### qemu-user-static, binfmt_misc and container 76 | 77 | A point to keep in mind when using qemu-user-static and binfmt_misc in container, is binfmt_misc `/proc/sys/fs/binfmt_misc` files `register`, `status` and `qemu-$arch` are shared and commonly used between host and inside of container. As a result, a script executed in container can modify `/proc/sys/fs/binfmt_misc` files on host OS. 78 | binfmt_misc is a feature of kernel. A container uses the host OS's kernel. 79 | 80 | ## Programs input & output 81 | 82 | In this section, we describe a program's input and output. 83 | This repository is a pipeline system by using Github Actions. 84 | 85 | First, we describe the entire pipelne system's input and output. `.github/workflows/actions.yml` is the top level file. 86 | 87 | * Input of the pipeline: `qemu-user-static-X.Y.Z-R.fcNN.$arch.rpm` RPM file under [Fedora Project URL](https://kojipkgs.fedoraproject.org/packages/qemu). Right now `$arch` is only x86_64. 88 | * Output of the pipeline: 89 | * [GitHub Releases page](https://github.com/multiarch/qemu-user-static/releases): `qemu-$arch-static` binary files, `qemu-$arch-static.tar.gz` and `x86_64_qemu-$arch-static.tar.gz` (`$from_arch_qemu-$arch-statc.tar.gz`). `qemu-$arch-static.tar.gz` files are same content with `x86_64_qemu-$arch-static.tar.gz`. It is an implementation to add supported host architectures `$from_arch` in the future. 90 | * Images on [Docker Hub](https://hub.docker.com/r/multiarch/qemu-user-static/). For actual images, see `README.md` Usage - multiarch/qemu-user-static images section. 91 | 92 | Second, we describe each program's input and output by sequence. 93 | 94 | | Step | Name | Description | Input | Output | 95 | | ---- | ---- | ----------- | ----- | ------ | 96 | | 1 | `generate_tarballs.sh` | Create tar.gz files to upload GitHub Releases page. | `qemu-$arch-static` files in qemu-user-static RPM | `qemu-$arch-static.tar.gz` and `x86_64_qemu-$arch-static.tar.gz` files | 97 | | 2 | `publish.sh` | Upload the tar.gz files by [GitHub API](https://developer.github.com/) | `qemu-$arch-static.tar.gz` and `x86_64_qemu-$arch-static.tar.gz` files | `qemu-$arch-static.tar.gz` and `x86_64_qemu-$arch-static.tar.gz` files on GitHub Releases page. | 98 | | 3 | `update.sh` | Create container images on local | `x86_64_qemu-$arch-static.tar.gz` files on GitHub Releases page. | Container images on local | 99 | | 4 | `test.sh` | Test created container images on local | Container images on local | `test/*` container images created as a result of tests | 100 | | 5 | `docker push $DOCKER_REPO` | Push the container images to DockerHub | `multiarch/qemu-user-static:*` container images on local | `multiarch/qemu-user-static:*` container images on Docker Hub | 101 | 102 | ## References 103 | 104 | * [1] QEMU: https://www.qemu.org/ 105 | * [2] Fedora qemu RPM: https://src.fedoraproject.org/rpms/qemu 106 | * [3] binfmt_misc: https://www.kernel.org/doc/html/latest/admin-guide/binfmt-misc.html 107 | * [4] binfmt_misc C language level manual: https://lwn.net/Articles/630727/ 108 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # qemu-user-static 2 | 3 | [![License](https://img.shields.io/github/license/multiarch/qemu-user-static.svg?style=flat-square)](./LICENSE) ![actions](https://github.com/multiarch/qemu-user-static/workflows/actions/badge.svg) [![Releases](https://img.shields.io/github/commits-since/multiarch/qemu-user-static/latest.svg?style=flat-square)](https://github.com/multiarch/qemu-user-static/releases) [![Docker Hub](https://img.shields.io/docker/pulls/multiarch/qemu-user-static.svg?style=flat-square)](https://hub.docker.com/r/multiarch/qemu-user-static/) 4 | 5 | ![](https://raw.githubusercontent.com/multiarch/dockerfile/master/logo.jpg) 6 | 7 | **multiarch/qemu-user-static** is to enable an execution of different multi-architecture containers by QEMU [1](#ref-1) and binfmt_misc [2](#ref-2). 8 | Here are examples with Docker [3](#ref-3). 9 | 10 | ## Getting started 11 | 12 | ``` 13 | $ uname -m 14 | x86_64 15 | 16 | $ docker run --rm -t arm64v8/ubuntu uname -m 17 | standard_init_linux.go:211: exec user process caused "exec format error" 18 | 19 | $ docker run --rm --privileged multiarch/qemu-user-static --reset -p yes 20 | 21 | $ docker run --rm -t arm64v8/ubuntu uname -m 22 | aarch64 23 | ``` 24 | 25 | It works on many architectures and OS container images. 26 | 27 | ``` 28 | $ docker run --rm -t arm32v6/alpine uname -m 29 | armv7l 30 | 31 | $ docker run --rm -t ppc64le/debian uname -m 32 | ppc64le 33 | 34 | $ docker run --rm -t s390x/ubuntu uname -m 35 | s390x 36 | 37 | $ docker run --rm -t arm64v8/fedora uname -m 38 | aarch64 39 | 40 | $ docker run --rm -t arm32v7/centos uname -m 41 | armv7l 42 | 43 | $ docker run --rm -t ppc64le/busybox uname -m 44 | ppc64le 45 | 46 | $ docker run --rm -t i386/ubuntu uname -m 47 | x86_64 48 | ``` 49 | 50 | Podman [4](#ref-4) also works. 51 | 52 | ``` 53 | $ sudo podman run --rm --privileged multiarch/qemu-user-static --reset -p yes 54 | 55 | $ podman run --rm -t arm64v8/fedora uname -m 56 | aarch64 57 | ``` 58 | 59 | Singularity [5](#ref-5) also works. 60 | 61 | ``` 62 | $ sudo singularity run docker://multiarch/qemu-user-static --reset -p yes 63 | 64 | $ singularity run --cleanenv docker://arm64v8/fedora uname -m 65 | aarch64 66 | ``` 67 | 68 | ## Usage 69 | 70 | ### multiarch/qemu-user-static images 71 | 72 | multiarch/qemu-user-static images are managed on the [Docker Hub](https://hub.docker.com/r/multiarch/qemu-user-static/) container repository. 73 | The images have below tags. 74 | 75 | **Images** 76 | 77 | 1. `multiarch/qemu-user-static` image 78 | 2. `multiarch/qemu-user-static:$version` images 79 | 3. `multiarch/qemu-user-static:$from_arch-$to_arch` images 80 | 4. `multiarch/qemu-user-static:$from_arch-$to_arch-$version` images 81 | 5. `multiarch/qemu-user-static:$to_arch` images 82 | 6. `multiarch/qemu-user-static:$to_arch-$version` images 83 | 7. `multiarch/qemu-user-static:register` image 84 | 85 | **Variables** 86 | 87 | * `$version`: Based QEMU's version. 88 | * `$from_arch`: Host architecture 89 | * `$to_arch`: Guest architecture 90 | 91 | **Description** 92 | 93 | * `multiarch/qemu-user-static` image container includes both a register script to register binfmt_misc entries and all the `/usr/bin/qemu-$arch-static` binary files in the container in it. `multiarch/qemu-user-static` image is an alias of the latest version of `multiarch/qemu-user-static:$version` images. 94 | * `multiarch/qemu-user-static:$to_arch` images are aliases of `multiarch/qemu-user-static:x86_64-$to_arch`. `multiarch/qemu-user-static:$to_arch` images only include the `$to_arch`'s `/usr/bin/qemu-$to_arch-static` binary file in it. `multiarch/qemu-user-static:$to_arch` image is an alias of the latest version of `multiarch/qemu-user-static:$to_arch-$version` images. 95 | * `multiarch/qemu-user-static:register` image has only the register script binfmt_misc entries. 96 | 97 | `multiarch/qemu-user-static` and `multiarch/qemu-user-static:register` images execute the register script that registers below kind of `/proc/sys/fs/binfmt_misc/qemu-$arch` files for all supported processors except the current one in it when running the container. See binfmt_misc manual [2] for detail of the files. 98 | As the `/proc/sys/fs/binfmt_misc` are common between host and inside of container, the register script modifies the file on host. 99 | 100 | ``` 101 | $ cat /proc/sys/fs/binfmt_misc/qemu-$arch 102 | enabled 103 | interpreter /usr/bin/qemu-$arch-static 104 | flags: F 105 | offset 0 106 | magic 7f454c460201010000000000000000000200b700 107 | mask ffffffffffffff00fffffffffffffffffeffffff 108 | ``` 109 | 110 | The `--reset` option is implemented at the register script that executes `find /proc/sys/fs/binfmt_misc -type f -name 'qemu-*' -exec sh -c 'echo -1 > {}' \;` to remove binfmt_misc entry files before register the entry. 111 | When same name's file `/proc/sys/fs/binfmt_misc/qemu-$arch` exists, the register command is failed with an error message "sh: write error: File exists". 112 | 113 | ``` 114 | $ docker run --rm --privileged multiarch/qemu-user-static [--reset][--help][-p yes][options] 115 | ``` 116 | 117 | On below image, we can not specify `-p yes` (`--persistent yes`) option. Because an interpreter's existance is checked when registering a binfmt_misc entry. As the interpreter does not exist in the container, the register script finishes with an error. 118 | 119 | ``` 120 | $ docker run --rm --privileged multiarch/qemu-user-static:register [--reset][--help][options] 121 | ``` 122 | 123 | Then the register script executes QEMU's [scripts/qemu-binfmt-conf.sh](https://github.com/qemu/qemu/blob/master/scripts/qemu-binfmt-conf.sh) script with options. 124 | You can check `usage()` in the file about the options. 125 | 126 | ``` 127 | Usage: qemu-binfmt-conf.sh [--qemu-path PATH][--debian][--systemd CPU] 128 | [--help][--credential yes|no][--exportdir PATH] 129 | [--persistent yes|no][--qemu-suffix SUFFIX] 130 | Configure binfmt_misc to use qemu interpreter 131 | --help: display this usage 132 | --qemu-path: set path to qemu interpreter ($QEMU_PATH) 133 | --qemu-suffix: add a suffix to the default interpreter name 134 | --debian: don't write into /proc, 135 | instead generate update-binfmts templates 136 | --systemd: don't write into /proc, 137 | instead generate file for systemd-binfmt.service 138 | for the given CPU. If CPU is "ALL", generate a 139 | file for all known cpus 140 | --exportdir: define where to write configuration files 141 | (default: $SYSTEMDDIR or $DEBIANDIR) 142 | --credential: if yes, credential and security tokens are 143 | calculated according to the binary to interpret 144 | --persistent: if yes, the interpreter is loaded when binfmt is 145 | configured and remains in memory. All future uses 146 | are cloned from the open file. 147 | ``` 148 | 149 | You can run `/usr/bin/qemu-$arch-static` binary file` in the container. 150 | 151 | ``` 152 | $ docker run --rm -t multiarch/qemu-user-static:x86_64-aarch64 /usr/bin/qemu-aarch64-static -help 153 | usage: qemu-aarch64 [options] program [arguments...] 154 | Linux CPU emulator (compiled for aarch64 emulation) 155 | ... 156 | 157 | $ docker run --rm -t multiarch/qemu-user-static:x86_64-aarch64 /usr/bin/qemu-aarch64-static -version 158 | qemu-aarch64 version 4.0.0 (qemu-4.0.0-5.fc31) 159 | Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers 160 | 161 | 162 | $ docker run --rm -t multiarch/qemu-user-static:aarch64 /usr/bin/qemu-aarch64-static -help 163 | usage: qemu-aarch64 [options] program [arguments...] 164 | Linux CPU emulator (compiled for aarch64 emulation) 165 | ... 166 | 167 | $ docker run --rm -t multiarch/qemu-user-static:aarch64 /usr/bin/qemu-aarch64-static -version 168 | qemu-aarch64 version 4.0.0 (qemu-4.0.0-5.fc31) 169 | Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers 170 | ``` 171 | 172 | `multiarch/qemu-user-static:$from_arch-$to_arch` images are used with `multiarch/qemu-user-static:register` image. 173 | Because when the binfmt_misc entry is registered without `-p` option, the interpreter needs to be put in the container. 174 | 175 | ``` 176 | $ docker run --rm --privileged multiarch/qemu-user-static:register --reset 177 | 178 | $ docker build --rm -t "test/integration/ubuntu" -<[1] QEMU: https://www.qemu.org/ 230 | * [2] binfmt_misc: https://www.kernel.org/doc/html/latest/admin-guide/binfmt-misc.html 231 | * [3] Docker: https://www.docker.com/ 232 | * [4] Podman: https://podman.io/ 233 | * [5] Singularity: https://sylabs.io/singularity/ 234 | --------------------------------------------------------------------------------