├── .github └── workflows │ └── quic-organization-repolinter.yml ├── LICENSE ├── README.md ├── debugging.md ├── quickstart.md ├── scripts ├── .dockr-env-rc ├── bashrc-extn ├── build-docker-img.sh ├── check-and-migrate.sh ├── core-utils │ ├── build-linux.sh │ ├── build-llvm.sh │ ├── build-py-vdev.sh │ ├── build-qemu.sh │ ├── build-ramdisk.sh │ ├── build-sysroot.sh │ ├── clone-linux.sh │ ├── clone-llvm.sh │ ├── clone-qcbor-dtc.sh │ ├── clone-qemu.sh │ └── gunyah-qcbor.patch ├── dockerfile-hyp ├── install-core-tools.sh ├── install-wsp-imgs.sh ├── migrate-tools-to-vol.sh ├── new-term.sh ├── run-docker.sh ├── utils │ ├── bldextfs.sh │ ├── build-crosvm.sh │ ├── build-gunyah.sh │ ├── build-rootfs-img.sh │ ├── clone-crosvm.sh │ ├── clone-gunyah.sh │ ├── gdb-start.sh │ ├── kern-test.sh │ ├── mnt-img.sh │ └── run-qemu.sh └── version.sh └── svm_booting_linux.md /.github/workflows/quic-organization-repolinter.yml: -------------------------------------------------------------------------------- 1 | name: QuIC Organization Repolinter 2 | 3 | on: 4 | push: 5 | branches: [ develop ] 6 | pull_request: 7 | branches: [ develop ] 8 | 9 | jobs: 10 | repolinter: 11 | runs-on: ubuntu-latest 12 | steps: 13 | - name: Checkout Repo 14 | uses: actions/checkout@v2 15 | - name: Verify repolinter config file is present 16 | id: check_files 17 | uses: andstor/file-existence-action@v1 18 | with: 19 | files: "repolint.json" 20 | - name: Run Repolinter with local repolint.json 21 | if: steps.check_files.outputs.files_exists == 'true' 22 | uses: newrelic/repolinter-action@v1 23 | - name: Run Repolinter with default ruleset 24 | if: steps.check_files.outputs.files_exists == 'false' 25 | uses: newrelic/repolinter-action@v1 26 | with: 27 | config_url: "https://raw.githubusercontent.com/quic/.github/main/repolint.json" 28 | 29 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | © 2021 Qualcomm Innovation Center, Inc. All rights reserved. 2 | 3 | Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 4 | 5 | 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 6 | 7 | 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 8 | 9 | 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. 10 | 11 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 12 | 13 | ==================================================================================================================== 14 | Note: Individual files contain the following tag instead of the full license text. 15 | 16 | SPDX-License-Identifier: BSD-3-Clause 17 | 18 | This enables machine processing of license information based on the SPDX License Identifiers that are available here: http://spdx.org/licenses/ 19 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Toolchain and support scripts 2 | 3 | This repository stores scripts and Dockerfiles needed for building and testing Gunyah Hypervisor. 4 | 5 | ## Quick start: 6 | 7 | - [Quick Setup instructions](quickstart.md) 8 | - [Debug instructions and notes](debugging.md) 9 | 10 | Gunyah related repositories: 11 | - [Setup Tools and Scripts](https://github.com/quic/gunyah-support-scripts) (This repository) 12 | - [Hypervisor](https://github.com/quic/gunyah-hypervisor.git) (Gunyah core hypervisor) 13 | - [Resource Manager](https://github.com/quic/gunyah-resource-manager.git) (Platform policy engine) 14 | - [C Runtime](https://github.com/quic/gunyah-c-runtime.git) (C runtime environment for Resource Manager) 15 | 16 | > See https://github.com/quic/gunyah-hypervisor for additional documentation. 17 | 18 | ## Release notes 19 | 20 | #### Dec 2023 (1.20) 21 | - Moved all generated images out of docker image (folder re-arch) 22 | - Using docker volumes for all generated images, so all changes will be persistent 23 | - Faster and simpler updates to latest versions of scripts in future 24 | - Added version info to file ```scripts/version.sh``` (previous releases are implied as 1.00 and 1.10) 25 | - Also set in env variable ```$ENV_VERSION``` in docker environment 26 | 27 | #### Sep 2023 28 | - Added scripts to generate all images required for SVM linux booting 29 | - Updated instructions to demo SVM loading and execution 30 | 31 | #### June 2023 32 | - Major update from previous releases 33 | - Simplified docker script 34 | - Moved all commands into simple scripts 35 | - Updated and simplified all instructions 36 | 37 | ## License 38 | 39 | SPDX-License-Identifier: BSD-3-Clause 40 | 41 | See [LICENSE](LICENSE) for the full license text. 42 | -------------------------------------------------------------------------------- /debugging.md: -------------------------------------------------------------------------------- 1 | 2 | ## Debugging with QEMU and Gunyah hypervisor 3 | 4 | Gunyah hypervisor can be debugged using either gdb-multiarch or other debug 5 | software such as VSCode. Helpful initial scripts are provided to get started. 6 | 7 | ### Disable ASLR 8 | 9 | The Gunyah Hypervisor standard build enables ASLR by default, and while it can 10 | be debugged with ASLR enabled, it is easier to debug with it disabled. 11 | 12 | With ASLR disabled, the EL2 virtual addresses used will be constant across 13 | multiple runs, simplifying symbol relocation. 14 | 15 | For example, the debug configuration of Gunyah defaults with ASLR enabled, it 16 | can be disabled by un-commenting the line ```#include include/debug_no_kaslr``` 17 | in ```/hyp/config/quality/debug.conf```. 18 | 19 | E.g. change the config to: 20 | ``` 21 | include include/debug_no_kaslr 22 | ``` 23 | 24 | Re-build the hypervisor with ```build-gunyah.sh``` as per instructions above. 25 | 26 | ### The Gunyah trace buffer 27 | 28 | It is possible to extract and decode the Gunyah trace buffer. 29 | 30 | - In QEMU, stop the target. E.g. ```CTRL-A + c``` to enter the monitor. 31 | - Find the symbol address in ```hyp.elf``` 32 | + ```000000000020d190 l O .bss 0000000000000008 trace_buffer_global``` 33 | - Make sure the current vcpu is in EL2. 34 | + QEMU cmd ```info registers``` - will show the current EL. 35 | + There is no reliable way to stop in EL2. You can try ```cont``` followed by ```stop```. 36 | + Or, you can try find the physical address of the buffer, and use ```pmemsave``` instead, 37 | + When stopped in EL2, check the PC address, and calulate the ```trace_buffer_global``` address.l 38 | + E.g. PC ```0xffffffd5ffe102c4``` 39 | + Set bottom 20-bits to zero => ```0xffffffd5ffe00000``` 40 | + Add trace_buffer_global symbol address (e.g. ``0x20d190``))=> ```0xffffffd60000d190``` 41 | + Read the pointer: ```x /1gx 0xffffffd60000d190```. Substitute the address into memsave below. 42 | + Save the trace: ```memsave 0xffffffd3f6100000 33554432 trace.bin``` 43 | + Exit QEMU, or from another docker shell: 44 | + ```~/share/hyp/tools/debug/tracebuf.py --elf ~/share/hyp/build/qemu/gunyah-rm-qemu/debug/hyp.elf -o trace.txt trace.bin``` 45 | 46 | ### Debug using gdb 47 | 48 | We need two terminals for debugging with gdb. 49 | 50 | In the Docker container shell, launch qemu with debug: 51 | 52 | ```bash 53 | cd ~/tools/qemu 54 | ./run-qemu.sh dbg 55 | ``` 56 | 57 | Start a second shell in another host terminal and attach to the running container. 58 | 59 | In the second terminal, run: 60 | ```bash 61 | cd ~/gunyah/gunyah-support-scripts/scripts 62 | ./new-term.sh 63 | ``` 64 | 65 | Connect to gdb from the second Docker shell: 66 | ```bash 67 | cd ~/share 68 | ~/utils/gdb-start.sh 69 | ``` 70 | 71 | Expected output:: 72 | ```bash 73 | (gunyah-venv) nemo@hyp-dev-env:~/share/$ ~/utils/gdb-start.sh 74 | The target architecture is assumed to be aarch64 75 | Remote debugging using localhost:1234 76 | warning: No executable has been specified and target does not support 77 | determining executable automatically. Try using the "file" command. 78 | 0x0000000080000000 in ?? () 79 | add symbol table from file "hyp/build/qemu/gunyah-rm-qemu/debug/hyp.elf" at 80 | .text_addr = 0xffffffd5ffe00000 81 | ... 82 | Breakpoint 4 at 0x80482a80: file src/exit.c, line 39. 83 | add symbol table from file "resource-manager/build/qemu/debug/resource-manager" at 84 | .text_addr = 0x804A0000 85 | Reading symbols from resource-manager/build/qemu/debug/resource-manager... 86 | (gdb) continue 87 | 88 | ``` 89 | 90 | At this point, either ```run``` to continue, or place additional breakpoints. 91 | 92 | Note: A GUI or ```tui``` frontend can be used for better debugging experience, 93 | but it can affect the stability of the tool. 94 | 95 | --- 96 | 97 | ## Early Linux kernel debugging 98 | 99 | The ```vmlinux``` elf file from the Linux build can be used for debugging 100 | symbols if required. 101 | 102 | --- 103 | -------------------------------------------------------------------------------- /quickstart.md: -------------------------------------------------------------------------------- 1 | # Quick Start 2 | 3 | Gunyah Hypervisor - Docker based development environment: 4 | 5 | _The following configuration is tested_ 6 | 7 | - Linux machine (Ubuntu 20.04) 8 | - 8GB RAM 9 | - 40+ GB free disk space (for docker images) 10 | - Quick internet connection with sufficient quota (downloads 3-5+ GB) 11 | - Docker (20.10) 12 | - [Installing Docker](https://docs.docker.com/engine/install/) 13 | - Note, version as reported by ```docker -v``` 14 | - LLVM (15.x) + musl C (1.1) 15 | - QEMU (7.2) 16 | - gdb-multiarch (9.2) 17 | 18 | A Docker container environment containing the above tools is supported. The 19 | scripts prepare an Ubuntu 22.04 based docker environment, download and build 20 | LLVM, QEMU, a reference Linux kernel and Busybox ramdisk and a the Python 21 | virtual environment to ease the development process. 22 | 23 | A similar setup may be created without Docker directly on an Ubuntu 22.04 24 | machine, however this has not been tested. 25 | 26 | -------- 27 | 28 | All the required scripts are available at 29 | [Gunyah support scripts](https://github.com/quic/gunyah-support-scripts). Clone 30 | the files to any local folder on the host machine: 31 | 32 | These commands are exected in the host machine terminal 33 | ```bash 34 | mkdir -p ~/gunyah 35 | cd ~/gunyah 36 | git clone https://github.com/quic/gunyah-support-scripts 37 | ``` 38 | 39 | Scripts referenced from here on are located in the scripts folder of this repository. 40 | 41 | -------- 42 | 43 | ## Build the Docker image 44 | ### Note about updating from the previous versions 45 | Latest versions of scripts (>= 1.20) have changed the folder layout compared to previous versions. The build script detects older images if present and migrate the images to latest version. Newer version scripts install all tools and images into docker volumes which are writable. So any modifications in those volumes to tools/images would persist, so any broken changes can be fixed by re-generating those tools/images. This new method helps to avoid time consuming process of download/build and storage space. 46 | 47 | If the previous version of docker image was older than when SVM support was added (implied as 1.10 in [release notes](README.md#release-notes)), then its recommended that linux image be rebuilt in the latest package. 48 | 49 | There are 2 volumes created ```tools-vol``` and ```wsp-vol```. 50 | 51 | ```tools-vol``` mounted at ```/usr/local/mnt/tools``` would contain llvm and qemu. LLVM toolchain typically don't need much updates but Qemu might need updates to add networking support in future. 52 | 53 | ```wsp-vol``` mounted in home folder at ```~/mnt/workspace``` will have all the rest of the images needed. These include linux, crosvm, rootfs etc. 54 | 55 | Host folder is still mapped at ```~/share``` in order to share the gunyah hypervisor source/builds between docker and host environment so that it would be easy to handle edit/debugging code using tools on the host machine. 56 | 57 | -------- 58 | ### Some common pitfalls to avoid: 59 | 60 | - Ensure _your userid_ is a member of the docker group (check with ```id```) 61 | + To add the group, run: ```sudo usermod -a -G docker $USER```, then logout and log back in. 62 | - You may need to symlink /var/lib/docker to a filesystem with sufficient space (e.g. 40+ GB) 63 | + e.g. ```lrwxrwxrwx 1 root root 12 Jun 9 11:34 /var/lib/docker -> /home/docker``` 64 | - If the following errors are seen during Docker build: 65 | 66 | ```shell 67 | E: Failed to fetch, 68 | E: Unable to fetch some archives 69 | ``` 70 | then uncomment the following option in the file ```dockerfile-hyp``` so that a fresh copy of the apt cache will be downloaded instead of using the stale ones. 71 | ```bash 72 | DOCKER_OPTIONS+=" --no-cache " 73 | ``` 74 | 75 | -------- 76 | ### Building the Docker Image: 77 | 78 | This method works for both upgrading from previous image as well as first time creation. 79 | 80 | While building docker image do not set the environment variable ```HOST_TO_DOCKER_SHARED_DIR```, since the build operation will be executed from within the docker environment and it needs access to rest of the scripts which are mapped at the shared folder. 81 | 82 | Change directory to ```gunyah-support-scripts/scripts``` 83 | 84 | Run the ```build-docker-img.sh``` script to generate the Docker image. 85 | 86 | Since the tools are built within the docker environment, and when a password is asked for ```sudo``` password, in order to write to some folders and to execute some commands needs root access. So provide the password created during the docker build process (default ```sudo``` password was set to ```1234```, unless its changed) 87 | 88 | During the build, Docker will download Ubuntu 22.04 packages, LLVM, QEMU, Linux 89 | and other libraries and build them. _This process may take a few hours 90 | depending on your internet connection system specs_. 91 | 92 | Following commands are exected in the host machine terminal 93 | ```bash 94 | cd ~/gunyah/gunyah-support-scripts/scripts 95 | ./build-docker-img.sh 96 | ``` 97 | If any errors occur, refer to above pitfalls and remedies. After this script completes successfully the docker image and other volumes are ready to be used. 98 | 99 | -------- 100 | 101 | ## Launch Docker environment 102 | 103 | Launch the Docker container using the ```run-docker.sh``` script from the scripts folder. 104 | 105 | A host machine folder can be shared with the docker environment. The 106 | environment variable ```HOST_TO_DOCKER_SHARED_DIR``` can be set before running 107 | the script or the script can be run from a folder that needs to be mapped from 108 | host into the docker environment. If the variable is not set, then current folder is used to share. It is recommended to keep a known folder to share with docker and optionally set the above variable within ```.bashrc``` file so that every shell session started will have the variable set. 109 | 110 | The choosen host folder is mounted in the docker container at 111 | ```~/share``` to facilitate copying files in or out of the container. 112 | The environment variable ```HOST_SHARED_DIR``` can be set to change the shared folder mountpoint if desired. 113 | 114 | Following commands are exected in the host machine terminal 115 | ```bash 116 | export HOST_TO_DOCKER_SHARED_DIR= 117 | ~/gunyah/gunyah-support-scripts/scripts/run-docker.sh 118 | ``` 119 | OR 120 | 121 | ```bash 122 | mkdir -p ~/gunyah 123 | cd ~/gunyah 124 | ~/gunyah/gunyah-support-scripts/scripts/run-docker.sh 125 | ``` 126 | 127 | Running this script starts a shell session in the docker container, in which all the 128 | following development/testing processes can be performed. A set of scripts are provided to assist 129 | building and running Gunyah based systems. 130 | 131 | Note: `the Docker environment has ```sudo``` privileges for the user with the default password ```1234```. This was set in the docker-hyp when building the container. 132 | 133 | -------- 134 | 135 |

All the following commands are run within the docker container! 136 | 137 | ## Test QEMU and Linux image 138 | 139 | Firstly, it is useful to test that the built QEMU and Linux images work correctly, without Gunyah. 140 | 141 | ```bash 142 | cd ~/mnt/workspace 143 | kern-test.sh 144 | ``` 145 | Note: *It may take several seconds for Linux to boot on the simulator before any console output.* 146 | 147 | Above script should boot linux kernel into a shell running in Qemu emulator. Within a couple of mins, all the linux log messages should show up. Press any key go into shell and test the linux environment. 148 | 149 | Output: 150 | ``` 151 | [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0x000f0510] 152 | [ 0.000000] Linux version 6.2.0-rc7-g0983f6bf2bfc (nemo@9a231c81b3c5) (aarch64-linux-gnu-gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0, GNU ld (GNU Binutils for Ubuntu) 2.34) #1 SMP PREEMPT Wed Feb 8 14:39:22 PST 2023 153 | [ 0.000000] random: crng init done 154 | [ 0.000000] Machine model: linux,dummy-virt 155 | [ 0.000000] efi: UEFI not found. 156 | [ 0.000000] NUMA: No NUMA configuration found 157 | [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x00000000bfffffff] 158 | [ 0.000000] NUMA: NODE_DATA [mem 0xbfbeca00-0xbfbeefff] 159 | ... 160 | [ 53.783957] uart-pl011 9000000.pl011: no DMA platform data 161 | [ 55.779296] Freeing unused kernel memory: 7936K 162 | [ 55.804200] Run /sbin/init as init process 163 | 164 | Please press Enter to activate this console. 165 | / # ls 166 | bin etc proc sbin usr 167 | dev linuxrc root sys 168 | / # 169 | ``` 170 | 171 | Note: Since QEMU is run interactively with ```-nographic```, the **CTRL-A** 172 | escape key is required to interact with QEMU. 173 | 174 | To exit QEMU: **CTRL-A** + ```x``` 175 | 176 | Note: the qemu run scripts can be modified to start ```qemu-system-aarch64``` 177 | to open a socket (for telnet) for the UART instead of running interactive on 178 | the shell and/or open a separate monitor telnet session to interact with QEMU 179 | and control/debug any resources. 180 | 181 | -------- 182 | 183 | ## Clone and build a Gunyah Hypervisor image 184 | 185 | The Gunyah hypervisor sources can be cloned from the host machine or from within the docker environment to 186 | the shared folder. 187 | 188 | It is strongly recommended to keep sources on the host machine's filesystem, 189 | since the sources and binary can be used on host to debug. But the builds need to be done within the docker environment only. 190 | 191 | To clone the sources run the provided script 192 | 193 | ```bash 194 | cd ~/share/gunyah 195 | clone-gunyah.sh 196 | ``` 197 | 198 | This will clone the required source repositories into the following 199 | directory structure. (All scripts assume these directory names) 200 | 201 | ```bash 202 | (gunyah-venv) nemo@hyp-dev-env:~/share/gunyah$ ls 203 | hyp musl-c-runtime resource-manager 204 | ``` 205 | 206 | This script may also be used to clone sources in the shared folder from your host machine. 207 | 208 | -------- 209 | 210 | ## Building a Gunyah Hypervisor image 211 | 212 | The Gunyah image build requires the above three source repositories to have been cloned. 213 | 214 | The build script *must be run from within the docker container*. 215 | 216 | This example assumes that we are building the sources cloned in the shared folder 217 | ```~/share/gunyah``` as visible from the docker container. 218 | 219 | To build the Gunyah image for the ```qemu``` target: 220 | 221 | ```bash 222 | cd ~/share/gunyah 223 | build-gunyah.sh qemu 224 | ``` 225 | 226 | Note: The docker container setup adds ```~/utils``` (containing build-gunyah.sh) 227 | to the PATH. 228 | 229 | When the build script completes successfully, it produces an output image 230 | ```/hypvm.elf```. This is also copied to the path in the environment 231 | variable ```QEMU_IMGS_DIR```, which has default path set to 232 | ```~/mnt/workspace/imgs```. 233 | 234 | -------- 235 | 236 | ## Test Hypervisor and Linux booting 237 | 238 | The Gunyah image can now be run on the QEMU system simulator. The reference 239 | mainline Linux kernel can be booted as a primary VM under the Gunyah 240 | hypervisor. 241 | 242 | First, we need to run QEMU to generate a devicetree binary used for booting with the hypervisor. 243 | 244 | ```bash 245 | cd ~/mnt/workspace 246 | run-qemu.sh dtb 247 | ``` 248 | We can now run QEMU with Gunyah and the Linux primary VM. This will boot with 249 | the Gunyah Hypervisor and a single Linux VM. This should boot up with the same 250 | Linux kernel, as run previously without Gunyah. 251 | 252 | ```bash 253 | run-qemu.sh 254 | ``` 255 | 256 | This should produce output below: 257 | ``` 258 | (gunyah-venv) nemo@hyp-dev-env:~/mnt/workspace$ run-qemu.sh 259 | [HYP] debug disabled 260 | [HYP] No spectre-BHB mitigation registered for unknown core 0:2 261 | [HYP] added heap: partition 0xffffffd60000a908, virt 0xffffffeeb0000000, phys 0xbdc00000, size 0x100000 262 | [HYP] added heap: partition 0xffffffd600016b60, virt 0xffffff938c100000, phys 0xbdd00000, size 0x2300000 263 | . . . 264 | [ 7.315612] uart-pl011 9000000.pl011: no DMA platform data 265 | [ 8.675942] Freeing unused kernel memory: 7936K 266 | [ 8.687593] Run /sbin/init as init process 267 | 268 | Please press Enter to activate this console. 269 | / # ls 270 | bin etc proc sbin usr 271 | dev linuxrc root sys 272 | / # 273 | ``` 274 | 275 | -------- 276 | 277 | ## SVM booting Linux 278 | 279 | This page describes how to prepare the host environment and the guest environment to showcase the capability of SVM booting Linux HLOS. 280 | 281 | [Linux booting on SVM in Gunyah](svm_booting_linux.md) 282 | 283 | -------- 284 | 285 | # Non-Docker setup 286 | 287 | If for any reason a Docker environment is not preferred, a native host 288 | environment can be prepared using the same scripts that were used to prepare the 289 | docker environment (refer to the docker config file ```dockerfile-hyp```). Some 290 | modifications may be required, your milage may vary. 291 | 292 | Since this brings too many variations into the mix that could potentially break 293 | the toolchain or other builds etc. in an unpredictable way, we are not 294 | supporting this use case. It's included here for information purposes only. 295 | -------------------------------------------------------------------------------- /scripts/.dockr-env-rc: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # © 2022 Qualcomm Innovation Center, Inc. All rights reserved. 4 | # 5 | # SPDX-License-Identifier: BSD-3-Clause 6 | 7 | 8 | # This file is sourced and run from within docker shell environment 9 | # Can be used for any local or user customization of the environment 10 | 11 | # Host folder mount point in docker env 12 | HOST_SHARED_DIR=~/share/docker-share 13 | 14 | -------------------------------------------------------------------------------- /scripts/bashrc-extn: -------------------------------------------------------------------------------- 1 | # © 2022 Qualcomm Innovation Center, Inc. All rights reserved. 2 | # 3 | # SPDX-License-Identifier: BSD-3-Clause 4 | 5 | alias c='clear' 6 | alias l='ls -la' 7 | 8 | if [[ -z "${USER}" ]]; then 9 | export USER=`whoami` 10 | fi 11 | 12 | # If any custom adjustments needed in docker environment runtime without 13 | # generating new image, add the following file in the shared host folder 14 | # This script is sourced in docker shell environment 15 | if [[ ! -f ~/.init-done ]] ; then 16 | if [[ -f ~/share/.dockr-env-rc ]]; then 17 | echo "Applying .dockr-env-rc customizations.." 18 | source ~/share/.dockr-env-rc 19 | fi 20 | touch ~/.init-done 21 | fi 22 | -------------------------------------------------------------------------------- /scripts/build-docker-img.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # © 2022 Qualcomm Innovation Center, Inc. All rights reserved. 4 | # 5 | # SPDX-License-Identifier: BSD-3-Clause 6 | 7 | set -e 8 | 9 | # See if we can migrate data from Old docker image if existing 10 | ./check-and-migrate.sh 11 | 12 | VERSION_SCRIPT=$(dirname "${BASH_SOURCE[0]}")/version.sh 13 | . ${VERSION_SCRIPT} 14 | 15 | CURRENT_DOCKER_IMAGE_TAG=hyp-dev-term:${CURRENT_VER} 16 | 17 | # Use public docker registry 18 | REGISTRY_SERVER="ubuntu:22.04" 19 | 20 | # For internal Qcom network, set the environment variable using 21 | # LOCAL_DOCKER_REGISTRY=docker-registry-ro.qualcomm.com/library 22 | # Use local registry if set 23 | if [[ ! -z "${LOCAL_DOCKER_REGISTRY}" ]]; then 24 | REGISTRY_SERVER="${LOCAL_DOCKER_REGISTRY}/${REGISTRY_SERVER}" 25 | echo "Using local registry server as : ${LOCAL_DOCKER_REGISTRY}" 26 | fi 27 | 28 | echo "Registry server is set to ${REGISTRY_SERVER}" 29 | 30 | if [[ -z "${USE_THIS_DOCKERFILE}" ]]; then 31 | USE_THIS_DOCKERFILE="dockerfile-hyp" ; 32 | fi 33 | 34 | echo "Building Docker file ${USE_THIS_DOCKERFILE}" 35 | 36 | DOCKER_OPTIONS=" build . " 37 | 38 | #DOCKER_OPTIONS+=" --progress=plain " 39 | 40 | # no-cache alleviates some install errors for not finding some packages 41 | #DOCKER_OPTIONS+=" --no-cache " 42 | 43 | # user environment related so the permissions will same as the host machine 44 | DOCKER_OPTIONS+=" --build-arg UID=$(id -u) " 45 | DOCKER_OPTIONS+=" --build-arg GID=$(id -g) " 46 | DOCKER_OPTIONS+=" --build-arg USER=${USER} " 47 | DOCKER_OPTIONS+=" --build-arg REGISTRY=${REGISTRY_SERVER} " 48 | DOCKER_OPTIONS+=" --build-arg ENV_VER=${CURRENT_VER} " 49 | 50 | # Dockerfile name 51 | DOCKER_OPTIONS+=" -f ${USE_THIS_DOCKERFILE} " 52 | 53 | # Docker image Tag 54 | DOCKER_OPTIONS+=" -t $CURRENT_DOCKER_IMAGE_TAG" 55 | 56 | # Build docker image 57 | docker $DOCKER_OPTIONS $* 58 | 59 | echo -e "\nBuilt or updated the docker image, now installing or configuring Core tools\n" 60 | 61 | echo "" 62 | echo "*********** Sudo password Note..!! *********************************" 63 | echo "" 64 | echo " Following commands are executed within docker environment, so when a" 65 | echo " prompt for sudo password showsup, type 1234 which is the default password" 66 | echo " set when building the docker image (unless it was changed)" 67 | echo "" 68 | echo "********************************************************************" 69 | echo "" 70 | 71 | HOST_TO_DOCKER_SHARED_DIR="" . ./run-docker.sh /bin/bash -c 'sudo -E $BASE_DIR/share/install-core-tools.sh' 72 | 73 | # Now install workspace images 74 | HOST_TO_DOCKER_SHARED_DIR="" . ./run-docker.sh /bin/bash -c '$BASE_DIR/share/install-wsp-imgs.sh' 75 | 76 | echo -e "Building docker image completed\n" 77 | -------------------------------------------------------------------------------- /scripts/check-and-migrate.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # © 2022 Qualcomm Innovation Center, Inc. All rights reserved. 4 | # 5 | # SPDX-License-Identifier: BSD-3-Clause 6 | 7 | # This script is run from the host machine to execute in old docker image to 8 | # migrate the data from previous docker image into volumes to be used in the 9 | # newer version of docker image. This will save time and space rebuilding all 10 | # the tools and images again (like llvm, qemu, linux etc) 11 | 12 | RENAME_TAG=hyp-dev-term:1.10 13 | OLD_TAG=hyp:dev-term 14 | 15 | OLD_TAG_ID=`docker images --format="{{.Repository}}:{{.Tag}} {{.ID}}" | grep "hyp:dev-term" | cut -d " " -f 2` 16 | NEW_TAG_ID=`docker images --format="{{.Repository}}:{{.Tag}} {{.ID}}" | grep "hyp-dev-term" | cut -d " " -f 2` 17 | 18 | # If old tag exists then extract the volumes and discard it 19 | if [[ ! -z $OLD_TAG_ID ]]; then 20 | # Tag the old image to newer format 21 | docker image tag $OLD_TAG $RENAME_TAG 22 | 23 | # Remove old tag 24 | docker image rm $OLD_TAG 25 | echo "Renamed old tag to new format, now the tag for old image is $RENAME_TAG" 26 | 27 | echo "" 28 | echo "*********** Sudo password Note..!! *********************************" 29 | echo "" 30 | echo " Following commands are executed within docker environment, so when a" 31 | echo " prompt for sudo password showsup, type 1234 which is the default password" 32 | echo " set when building the docker image (unless it was changed)" 33 | echo "" 34 | echo "********************************************************************" 35 | echo "" 36 | 37 | 38 | HOST_TO_DOCKER_SHARED_DIR="" DOCKER_TAG=$RENAME_TAG . ./run-docker.sh /bin/bash -c '$BASE_DIR/share/migrate-tools-to-vol.sh' 39 | 40 | echo "Old docker image can be removed *after* the new docker image has been verified to work" 41 | echo "Use the following command to remove stoped containers and delete the dangling images" 42 | echo ' docker rm $(docker ps -a -q) ; docker image rm hyp-dev-term:1.10 ; docker image prune ' 43 | fi 44 | -------------------------------------------------------------------------------- /scripts/core-utils/build-linux.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # © 2022 Qualcomm Innovation Center, Inc. All rights reserved. 4 | # 5 | # SPDX-License-Identifier: BSD-3-Clause 6 | 7 | set -e 8 | 9 | if [[ -z "${LINUX_DIR}" ]]; then 10 | echo "LINUX_DIR location not provided" 11 | return 12 | fi 13 | 14 | KERNEL_IMAGE_FILE=${LINUX_DIR}/build/arch/arm64/boot/Image 15 | 16 | if [[ -f ${KERNEL_IMAGE_FILE} ]]; then 17 | echo "Linux kernel is already built, skipping building again..." 18 | echo "to rebuild again, delete the kernel image file : ${KERNEL_IMAGE_FILE} and rebuild" 19 | return 20 | fi 21 | 22 | ADDITIONAL_KERNEL_CONFIGS=" " 23 | 24 | echo -e "\nBuild linux from LINUX_DIR : ${LINUX_DIR}" 25 | 26 | mkdir -p ${LINUX_DIR}/build 27 | cd ${LINUX_DIR}/src/linux 28 | 29 | if [[ -f "./arch/arm64/configs/gunyah.config" ]]; then 30 | echo -e "Found Gunyah config, enabling virt and gunyah kernel config" 31 | ADDITIONAL_KERNEL_CONFIGS+=" virt.config gunyah.config" 32 | fi 33 | 34 | CPU_CNT=$(grep -c ^processor /proc/cpuinfo) 35 | 36 | echo "Building Linux" 37 | 38 | make ARCH=arm64 O=${LINUX_DIR}/build CROSS_COMPILE=aarch64-linux-gnu- defconfig ${ADDITIONAL_KERNEL_CONFIGS} 39 | 40 | make ARCH=arm64 O=${LINUX_DIR}/build CROSS_COMPILE=aarch64-linux-gnu- -j${CPU_CNT} 41 | 42 | if [[ ! -z "${QEMU_IMGS_DIR}" ]]; then 43 | cp ${LINUX_DIR}/build/arch/arm64/boot/Image "${QEMU_IMGS_DIR}/" 44 | echo "Copied Linux kernel Image to ${QEMU_IMGS_DIR}/" 45 | fi 46 | -------------------------------------------------------------------------------- /scripts/core-utils/build-llvm.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # © 2022 Qualcomm Innovation Center, Inc. All rights reserved. 4 | # 5 | # SPDX-License-Identifier: BSD-3-Clause 6 | 7 | set -e 8 | 9 | echo -e "\nBuild llvm from TOOLS_DIR : $TOOLS_DIR" 10 | 11 | if [[ -z "${LLVM}" ]]; then 12 | echo "LLVM environment variable not set" 13 | return 14 | fi 15 | 16 | echo "LLVM : ${LLVM}" 17 | 18 | TOOLS_SRC_DIR=${TOOLS_DIR}/src 19 | LLVM_TOOLS_SRC_DIR=${TOOLS_SRC_DIR}/llvm-tools 20 | LLVM_SRC_DIR="${LLVM_TOOLS_SRC_DIR}/llvm-project" 21 | MUSL_SRC_DIR="${LLVM_TOOLS_SRC_DIR}/musl" 22 | 23 | LLVM_INSTALL_DIR="${LLVM}" 24 | CPU_CNT=$(grep -c ^processor /proc/cpuinfo) 25 | 26 | if [[ ! -d ${LLVM_TOOLS_SRC_DIR} ]]; then 27 | echo -e "\nllvm Sources not found at : ${LLVM_TOOLS_SRC_DIR}" 28 | return 29 | fi 30 | 31 | pushd ${LLVM_SRC_DIR}/llvm 32 | mkdir -p build 33 | cd build 34 | 35 | echo "Building LLVM $LLVM_SRC_DIR" 36 | 37 | cmake .. \ 38 | -G Ninja \ 39 | -DCMAKE_BUILD_TYPE=Release \ 40 | -DCMAKE_INSTALL_PREFIX=${LLVM_INSTALL_DIR} \ 41 | -DLLVM_TARGETS_TO_BUILD="ARM;AArch64;X86" \ 42 | -DLLVM_ENABLE_PROJECTS="llvm;clang;lld" \ 43 | -DLLVM_ENABLE_BINDINGS=OFF \ 44 | -DLLVM_LINK_LLVM_DYLIB=ON \ 45 | -DLLVM_INSTALL_TOOLCHAIN_ONLY=ON && 46 | ninja -j${CPU_CNT} && 47 | rm -rf ${LLVM_INSTALL_DIR}/* && 48 | ninja install || { 49 | echo "Failed building LLVM" 50 | return 51 | } 52 | 53 | echo "Done building LLVM, installed at ${LLVM_INSTALL_DIR}" 54 | popd 55 | 56 | LLVM_VERSION=$(${LLVM_SRC_DIR}/llvm/build/bin/llvm-config --version | tr -d git) 57 | 58 | 59 | echo "LLVM_VERSION : ${LLVM_VERSION}" 60 | 61 | pushd ${MUSL_SRC_DIR} 62 | mkdir -p build 63 | cd build 64 | echo "Building musl lib" 65 | 66 | CROSS_COMPILE=aarch64-linux-gnu- \ 67 | CFLAGS="-march=armv8.5-a+rng" \ 68 | LDFLAGS="-static" \ 69 | ../configure \ 70 | --prefix=${LLVM_INSTALL_DIR}/aarch64-linux-gnu/libc \ 71 | --build=aarch64-linux-gnu \ 72 | --target=aarch64-linux-gnu \ 73 | --enable-debug \ 74 | --disable-optimize && 75 | make && 76 | make install || { 77 | echo "Failed building musl lib" 78 | return 79 | } 80 | 81 | echo "Done building musl lib" 82 | popd 83 | 84 | export PATH=${LLVM_INSTALL_DIR}/bin:${PATH} 85 | 86 | pushd ${LLVM_SRC_DIR}/compiler-rt 87 | mkdir -p build 88 | cd build 89 | echo "Install c runtime lib to llvm" 90 | 91 | cmake .. \ 92 | -G Ninja \ 93 | -DCMAKE_AR=${LLVM_INSTALL_DIR}/bin/llvm-ar \ 94 | -DCMAKE_ASM_COMPILER_TARGET=aarch64-linux-gnu \ 95 | -DCMAKE_C_COMPILER=${LLVM_INSTALL_DIR}/bin/clang \ 96 | -DCMAKE_C_COMPILER_TARGET=aarch64-linux-gnu \ 97 | -DCMAKE_EXE_LINKER_FLAGS="-fuse-ld=lld" \ 98 | -DCMAKE_NM=${LLVM_INSTALL_DIR}/bin/llvm-nm \ 99 | -DCMAKE_RANLIB=${LLVM_INSTALL_DIR}/bin/llvm-ranlib \ 100 | -DCOMPILER_RT_BUILD_BUILTINS=ON \ 101 | -DCOMPILER_RT_BUILD_LIBFUZZER=OFF \ 102 | -DCOMPILER_RT_BUILD_MEMPROF=OFF \ 103 | -DCOMPILER_RT_BUILD_PROFILE=OFF \ 104 | -DCOMPILER_RT_BUILD_SANITIZERS=OFF \ 105 | -DCOMPILER_RT_BUILD_XRAY=OFF \ 106 | -DCOMPILER_RT_DEFAULT_TARGET_ONLY=ON \ 107 | -DLLVM_CONFIG_PATH=${LLVM_SRC_DIR}/llvm/build/bin/llvm-config \ 108 | -DCMAKE_BUILD_TYPE=Release \ 109 | -DCMAKE_INSTALL_PREFIX=${LLVM_INSTALL_DIR}/lib/clang/${LLVM_VERSION}/ && 110 | ninja -j${CPU_CNT} && 111 | ninja install || { 112 | echo "Failed to install runtime lib to llvm" 113 | return 114 | } 115 | 116 | echo "Install c runtime lib to llvm done" 117 | popd 118 | 119 | # Delete the sources and build folders to save disk space 120 | rm -rf ${LLVM_TOOLS_SRC_DIR} 121 | -------------------------------------------------------------------------------- /scripts/core-utils/build-py-vdev.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # © 2022 Qualcomm Innovation Center, Inc. All rights reserved. 4 | # 5 | # SPDX-License-Identifier: BSD-3-Clause 6 | 7 | set -e 8 | 9 | # versions of the tools lark_parser, cheetah and pyelftools are listed 10 | # in the requirements.txt file in the Gunyah hypervisor source repository 11 | # in the path hyp/tools/requirements.txt 12 | 13 | if [[ ! -f ${WORKSPACE}/gunyah-venv/bin/activate ]]; then 14 | 15 | pushd ${WORKSPACE} 16 | echo "Preparing python venv for gunyah" 17 | python3 -m venv gunyah-venv && 18 | chmod a+x gunyah-venv/bin/activate && 19 | . gunyah-venv/bin/activate && 20 | pip3 install --upgrade pip && 21 | pip3 install wheel && 22 | pip3 install pexpect && 23 | pip3 install lark_parser==0.8.9 && 24 | pip3 install Cheetah3==3.2.6 && 25 | pip3 install pyelftools==0.26 || { 26 | echo "Faild to setup python venv" 27 | # popd 28 | return 29 | } 30 | popd 31 | 32 | echo "" >> $WORKSPACE/.wsp-env 33 | echo 'source ${WORKSPACE}/gunyah-venv/bin/activate' \ 34 | >> $WORKSPACE/.wsp-env 35 | else 36 | echo "Gunyah python venv is already installed" 37 | fi 38 | -------------------------------------------------------------------------------- /scripts/core-utils/build-qemu.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # © 2022 Qualcomm Innovation Center, Inc. All rights reserved. 4 | # 5 | # SPDX-License-Identifier: BSD-3-Clause 6 | 7 | set -e 8 | 9 | echo -e "\nBuild Qemu from TOOLS_DIR : $TOOLS_DIR" 10 | echo "QEMU_INSTALL_DIR : $QEMU_INSTALL_DIR" 11 | 12 | CPU_CNT=$(grep -c ^processor /proc/cpuinfo) 13 | echo -e "\nCPU_CNT:${CPU_CNT}" 14 | 15 | TOOLS_SRC_DIR="${TOOLS_DIR}/src" 16 | 17 | SLIRPLIB_FLAGS=" " 18 | 19 | if [[ -d ${TOOLS_SRC_DIR}/libslirp ]]; then 20 | echo "Found libslirp, build and installing" 21 | 22 | SLIRPLIB_FLAGS=" --enable-slirp --static " 23 | 24 | cd ${TOOLS_SRC_DIR}/libslirp 25 | meson build --prefix=${QEMU_INSTALL_DIR} 26 | ninja -C build install 27 | cd .. 28 | else 29 | echo "libslirp is not found..!!" 30 | fi 31 | 32 | if [[ ! -d ${TOOLS_SRC_DIR}/qemu ]]; then 33 | echo -e "\nqemu Sources do not exist here: ${TOOLS_SRC_DIR}" 34 | echo -e "\nrun clone-qemu.sh in utils" 35 | return 36 | fi 37 | 38 | cd $TOOLS_SRC_DIR 39 | 40 | pushd qemu 41 | 42 | # TODO: Fix it later and enable it back 43 | SLIRPLIB_FLAGS=" " 44 | 45 | mkdir -p build 46 | cd build 47 | echo -e "[BUILD] building and installing ''qemu''." 48 | ../configure \ 49 | --prefix=${QEMU_INSTALL_DIR} \ 50 | --target-list=aarch64-softmmu \ 51 | --extra-cflags=-Wno-error \ 52 | ${SLIRPLIB_FLAGS} \ 53 | --enable-debug && 54 | make -j${CPU_CNT} && 55 | make install || { 56 | echo "Failed." 57 | return 58 | } 59 | popd >/dev/null 60 | 61 | # Delete sources to save space, comment to retain sources 62 | #rm -rf ${TOOLS_SRC_DIR}/qemu 63 | -------------------------------------------------------------------------------- /scripts/core-utils/build-ramdisk.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # © 2022 Qualcomm Innovation Center, Inc. All rights reserved. 4 | # 5 | # SPDX-License-Identifier: BSD-3-Clause 6 | 7 | set -e 8 | 9 | CPU_CNT=$(grep -c ^processor /proc/cpuinfo) 10 | 11 | BUSYBOX_VER=busybox-1.33.0 12 | BUSYBOX_FILE=${BUSYBOX_VER}.tar.bz2 13 | 14 | echo "Building Busybox" 15 | 16 | if [[ -z "${LINUX_DIR}" ]]; then 17 | echo "LINUX_DIR location not provided" 18 | return 19 | fi 20 | 21 | RAMDISK_DIR="${LINUX_DIR}/${BUSYBOX_VER}" 22 | 23 | echo -e "\nClone linux sources to LINUX_DIR : ${LINUX_DIR}" 24 | 25 | if [[ ! -d ${RAMDISK_DIR} ]]; then 26 | 27 | echo -e "\nDownloading Busybox:" 28 | cd ${LINUX_DIR} 29 | wget -c https://busybox.net/downloads/${BUSYBOX_FILE} || { 30 | echo "Unable to Download busybox" 31 | return 32 | } 33 | 34 | tar xf ${BUSYBOX_FILE} || { 35 | echo "Unable to Uncompress busybox" 36 | return 37 | } 38 | fi 39 | 40 | if [[ ! -f ${RAMDISK_DIR}/initrd.img ]]; then 41 | cd ${RAMDISK_DIR} 42 | 43 | make ARCH=arm CROSS_COMPILE=aarch64-linux-gnu- defconfig 44 | #make ARCH=arm CROSS_COMPILE=aarch64-linux-gnu- menuconfig 45 | sed -i '/# CONFIG_STATIC is not set/c\CONFIG_STATIC=y' .config 46 | ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make -j${CPU_CNT} 47 | ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make install 48 | 49 | cd _install 50 | mkdir proc sys dev etc etc/init.d 51 | 52 | cat < etc/init.d/rcS 53 | #!bin/sh 54 | mount -t proc none /proc 55 | mount -t sysfs none /sys 56 | EOF 57 | 58 | chmod u+x etc/init.d/rcS 59 | grep -v tty ../examples/inittab > ./etc/inittab 60 | find . | cpio -o -H newc | gzip > ${RAMDISK_DIR}/initrd.img 61 | cp ${RAMDISK_DIR}/initrd.img "${LINUX_DIR}/" 62 | fi 63 | 64 | if [[ ! -z "${QEMU_IMGS_DIR}" ]] && [[ ! -f ${QEMU_IMGS_DIR}/initrd.img ]]; then 65 | cp ${RAMDISK_DIR}/initrd.img "${QEMU_IMGS_DIR}/" 66 | echo "Copied Linux kernel Image to ${QEMU_IMGS_DIR}" 67 | fi 68 | -------------------------------------------------------------------------------- /scripts/core-utils/build-sysroot.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # © 2022 Qualcomm Innovation Center, Inc. All rights reserved. 4 | # 5 | # SPDX-License-Identifier: BSD-3-Clause 6 | 7 | set -e 8 | 9 | echo -e "\nBuilding sysroot from LIB_DIR : $LIB_DIR" 10 | echo "LLVM : $LLVM" 11 | echo "LOCAL_SYSROOT : ${LOCAL_SYSROOT}" 12 | 13 | SRC_DIR="${LIB_DIR}/src" 14 | 15 | if [[ ! -d ${SRC_DIR}/dtc ]] || [[ ! -d ${SRC_DIR}/QCBOR ]]; then 16 | echo -e "\ndtc/QCBOR Sources not found in : ${SRC_DIR}" 17 | return 18 | fi 19 | 20 | cd $SRC_DIR 21 | 22 | echo "[BUILD] building and installing ''dtc''." 23 | 24 | pushd dtc 25 | 26 | CC=aarch64-linux-gnu-gcc make && 27 | CC=aarch64-linux-gnu-gcc make install libfdt \ 28 | PREFIX=${LOCAL_SYSROOT} || { 29 | echo "Failed building app-sysroot" 30 | return 31 | } 32 | 33 | popd >/dev/null 34 | 35 | echo "[BUILD] building and installing ''QCBOR''." 36 | 37 | pushd QCBOR 38 | 39 | make -f Makefile.hyp sysroot_decoder_lib 40 | 41 | echo "Built decoder lib" 42 | 43 | popd >/dev/null 44 | -------------------------------------------------------------------------------- /scripts/core-utils/clone-linux.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # © 2022 Qualcomm Innovation Center, Inc. All rights reserved. 4 | # 5 | # SPDX-License-Identifier: BSD-3-Clause 6 | 7 | set -e 8 | 9 | if [[ -z "${LINUX_DIR}" ]]; then 10 | echo "LINUX_DIR location not provided" 11 | return 12 | fi 13 | 14 | echo -e "\nClone linux sources to LINUX_DIR : ${LINUX_DIR}" 15 | 16 | mkdir -p ${LINUX_DIR}/src 17 | 18 | if [[ -d ${LINUX_DIR}/src/linux ]]; then 19 | echo "Linux folder already exists..!!" 20 | return 21 | fi 22 | 23 | cd ${LINUX_DIR}/src 24 | 25 | # LINUX 26 | LINUX_VER="v6.5" 27 | echo -e "\nCloning Linux ${LINUX_VER}:" 28 | git clone \ 29 | --depth=1 --progress -c advice.detachedHead=false \ 30 | -b ${LINUX_VER} \ 31 | https://github.com/torvalds/linux.git || { 32 | echo "Unable to clone Linux" 33 | return 34 | } 35 | 36 | # Enable Gunyah drivers in linux, static linking for now 37 | echo "Applying gunyah drivers patch to linux" 38 | 39 | # Install functional b4 first 40 | # Instructions from https://b4.docs.kernel.org/en/latest/installing.html 41 | echo "Installing b4 to download patches" 42 | 43 | # Set git global config for b4 to work 44 | git config --global user.email "$USER@test.com" 45 | git config --global user.name "$USER" 46 | git config --global color.ui auto 47 | 48 | # Now get b4 installed to local folder 49 | mkdir -p ${LINUX_DIR}/tools 50 | cd ${LINUX_DIR}/tools 51 | git clone https://git.kernel.org/pub/scm/utils/b4/b4.git b4 52 | cd b4 53 | git switch stable-0.9.y 54 | git submodule update --init 55 | pip install -r requirements.txt 56 | 57 | echo "Installed b4 to ${LINUX_DIR}/tools/b4" 58 | 59 | cd ${LINUX_DIR}/src/linux 60 | 61 | # Latest version of the patches do not support crosvm, so pick v14 instead 62 | ${LINUX_DIR}/tools/b4/b4.sh shazam -v 14 https://lore.kernel.org/all/20230613172054.3959700-1-quic_eberman@quicinc.com/ 63 | echo "Applied gunyah drivers patch successfully" 64 | 65 | echo "Generate gunyah.config" 66 | echo "CONFIG_VIRT_DRIVERS=y" > ./arch/arm64/configs/gunyah.config 67 | echo "CONFIG_GUNYAH=y" >> ./arch/arm64/configs/gunyah.config 68 | echo "CONFIG_GUNYAH_VCPU=y" >> ./arch/arm64/configs/gunyah.config 69 | echo "CONFIG_GUNYAH_IRQFD=y" >> ./arch/arm64/configs/gunyah.config 70 | echo "CONFIG_GUNYAH_IOEVENTFD=y" >> ./arch/arm64/configs/gunyah.config 71 | echo "Created gunyah.config" 72 | 73 | echo "Linux kernel clone and gunyah config completed" 74 | -------------------------------------------------------------------------------- /scripts/core-utils/clone-llvm.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # © 2022 Qualcomm Innovation Center, Inc. All rights reserved. 4 | # 5 | # SPDX-License-Identifier: BSD-3-Clause 6 | 7 | set -e 8 | 9 | echo -e "\nLLVM TOOLS_DIR : $TOOLS_DIR" 10 | 11 | TOOLS_SRC_DIR=$TOOLS_DIR/src 12 | LLVM_TOOLS_SRC_DIR=${TOOLS_SRC_DIR}/llvm-tools 13 | 14 | if [[ ! -d "${LLVM_TOOLS_SRC_DIR}/llvm-project" ]] ; then 15 | 16 | mkdir -p ${LLVM_TOOLS_SRC_DIR} 17 | cd ${LLVM_TOOLS_SRC_DIR} 18 | 19 | # LLVM 20 | echo -e "\nCloning LLVM into : ${LLVM_TOOLS_SRC_DIR}" 21 | git clone \ 22 | --single-branch --depth=1 --progress -c advice.detachedHead=false -b \ 23 | release/15.x \ 24 | https://github.com/llvm/llvm-project.git || { 25 | echo "Unable to clone LLVM" 26 | return 27 | } 28 | 29 | # MUSL 30 | echo -e "\nCloning MUSL into ${LLVM_TOOLS_SRC_DIR}" 31 | git clone -c advice.detachedHead=false -b v1.1.24 \ 32 | https://git.musl-libc.org/git/musl || { 33 | echo "Unable to clone Musl" 34 | return 35 | } 36 | fi 37 | -------------------------------------------------------------------------------- /scripts/core-utils/clone-qcbor-dtc.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # © 2022 Qualcomm Innovation Center, Inc. All rights reserved. 4 | # 5 | # SPDX-License-Identifier: BSD-3-Clause 6 | 7 | set -e 8 | 9 | echo -e "\nCloning QCBOR and fdt lib for sys_root into LIB_DIR : $LIB_DIR" 10 | echo "BASE_DIR : $BASE_DIR" 11 | 12 | mkdir -p ${LIB_DIR}/src 13 | cd ${LIB_DIR}/src 14 | rm -rf QCBOR dtc 15 | 16 | # QCBOR 17 | echo -e "\nCloning QCBOR:" 18 | git clone \ 19 | --single-branch --depth=1 --progress --quiet -c advice.detachedHead=false \ 20 | -b v1.2 \ 21 | https://github.com/laurencelundblade/QCBOR.git || { 22 | echo "Unable to clone QCBOR" 23 | return 24 | } 25 | 26 | pushd QCBOR 27 | 28 | echo -e "\napplying QCBOR patch" 29 | 30 | git apply "${BASE_DIR}/core-utils/gunyah-qcbor.patch" || { 31 | echo "Unable to apply qcbor gunyah patch to QCBOR" 32 | return 33 | } 34 | echo -e "\nDone applying patch" 35 | 36 | popd >/dev/null 37 | 38 | # DTC 39 | echo -e "\nCloning DTC:" 40 | git clone \ 41 | --single-branch --progress --quiet -c advice.detachedHead=false -b \ 42 | v1.6.1 \ 43 | https://github.com/dgibson/dtc.git || { 44 | echo "Unable to clone DTC" 45 | return 46 | } 47 | -------------------------------------------------------------------------------- /scripts/core-utils/clone-qemu.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # © 2022 Qualcomm Innovation Center, Inc. All rights reserved. 4 | # 5 | # SPDX-License-Identifier: BSD-3-Clause 6 | 7 | set -e 8 | 9 | echo -e "\nClone QEMU to TOOLS_DIR : $TOOLS_DIR" 10 | 11 | TOOLS_SRC_DIR=${TOOLS_DIR}/src 12 | 13 | mkdir -p ${TOOLS_SRC_DIR} 14 | 15 | if [[ ! -d ${TOOLS_SRC_DIR}/qemu ]] ; then 16 | 17 | cd ${TOOLS_SRC_DIR} 18 | 19 | # QEMU 20 | echo -e "\nCloning QEMU:" 21 | git clone \ 22 | --single-branch --depth=1 --progress -c advice.detachedHead=false \ 23 | -b v7.2.0 \ 24 | https://git.qemu.org/git/qemu.git || { 25 | echo "Unable to clone QEMU" 26 | return 27 | } 28 | 29 | pushd qemu 30 | 31 | git submodule init || { 32 | echo "Unable to do submodule init" 33 | return 34 | } 35 | 36 | git submodule update --recursive --depth=1 || { 37 | echo "Unable to do submodule update" 38 | return 39 | } 40 | 41 | popd 42 | fi 43 | 44 | if [[ ! -d ${TOOLS_SRC_DIR}/libslirp ]] ; then 45 | # download libslirp to enable networking 46 | git clone https://gitlab.freedesktop.org/slirp/libslirp.git 47 | fi 48 | -------------------------------------------------------------------------------- /scripts/core-utils/gunyah-qcbor.patch: -------------------------------------------------------------------------------- 1 | diff --git a/Makefile.hyp b/Makefile.hyp 2 | new file mode 100644 3 | index 0000000..14c2111 4 | --- /dev/null 5 | +++ b/Makefile.hyp 6 | @@ -0,0 +1,64 @@ 7 | +QCBOR_INC_PATH = inc 8 | +SYSROOT_INC_PATH = $(LOCAL_SYSROOT)/include 9 | + 10 | +ifneq ($(wildcard $(QCBOR_INC_PATH)),) 11 | + BUILD_INC_PATH = $(QCBOR_INC_PATH) 12 | + LIB_INSTALL_PATH = lib 13 | + #$(info From QCBOR sources folder) 14 | +else 15 | + BUILD_INC_PATH = $(SYSROOT_INC_PATH) 16 | + LIB_INSTALL_PATH = ../../lib 17 | + #$(info From sysroot sources folder) 18 | +endif 19 | + 20 | +CC = ${LLVM}/bin/clang 21 | +SYSROOT_SRC_PATH = src/qcbor 22 | +AR = ${LLVM}/bin/llvm-ar 23 | + 24 | +HDR_FILES = qcbor/qcbor.h \ 25 | + qcbor/qcbor_common.h \ 26 | + qcbor/qcbor_decode.h \ 27 | + qcbor/qcbor_private.h \ 28 | + qcbor/qcbor_spiffy_decode.h \ 29 | + qcbor/UsefulBuf.h \ 30 | + UsefulBuf.h 31 | + 32 | +all: $(LIB_INSTALL_PATH)/libqcbor.a 33 | + 34 | +CFLAGS = -target aarch64-linux-gnu -Os -march=armv8-a -mcpu=cortex-a72 35 | +CFLAGS += -g -gdwarf-4 -std=gnu18 -Weverything -Werror 36 | +CFLAGS += -Wno-covered-switch-default -Wno-c++98-compat -Wno-c++-compat 37 | +CFLAGS += -Wno-gcc-compat -Wno-undef -Wno-declaration-after-statement 38 | +CFLAGS += -ffunction-sections -fdata-sections -fpie 39 | +CFLAGS += -DQCBOR_DISABLE_INDEFINITE_LENGTH_ARRAYS 40 | +CFLAGS += -c -I $(BUILD_INC_PATH) 41 | + 42 | +QCBOR_OBJS = src/UsefulBuf.o src/qcbor_decode.o 43 | + 44 | +$(LIB_INSTALL_PATH)/libqcbor.a: $(QCBOR_OBJS) 45 | + @mkdir -p $(LIB_INSTALL_PATH) 46 | + $(AR) -rc $@ $^ 47 | + @echo "Installed QCBOR decode lib at $(PWD)/$(LIB_INSTALL_PATH)" 48 | + 49 | +src/UsefulBuf.o: $(BUILD_INC_PATH)/qcbor/UsefulBuf.h 50 | +src/qcbor_decode.o: $(foreach sf, $(HDR_FILES), $(BUILD_INC_PATH)/$(sf)) 51 | + 52 | +# First create a sysroot qcbor decoder source copy from original QCBOR repo 53 | +# Then build in sysroot folder 54 | +sysroot_decoder_lib: 55 | + @echo -e "Copying QCBOR decoder sources to sysroot folder: $(LOCAL_SYSROOT)/$(SYSROOT_SRC_PATH)" 56 | + @$(foreach sf, $(HDR_FILES), \ 57 | + mkdir -p $(dir $(SYSROOT_INC_PATH)/$(sf)); \ 58 | + cp $(QCBOR_INC_PATH)/$(sf) $(SYSROOT_INC_PATH)/$(sf); \ 59 | + ) 60 | + @$(foreach sf, $(QCBOR_OBJS:.o=.c), \ 61 | + mkdir -p $(dir $(LOCAL_SYSROOT)/$(SYSROOT_SRC_PATH)/$(sf)) ; \ 62 | + cp $(sf) $(LOCAL_SYSROOT)/$(SYSROOT_SRC_PATH)/$(sf); \ 63 | + ) 64 | + @cp ./Makefile.hyp $(LOCAL_SYSROOT)/$(SYSROOT_SRC_PATH)/Makefile 65 | + @echo -e "Copying completed, building the lib..." 66 | + @cd $(LOCAL_SYSROOT)/$(SYSROOT_SRC_PATH) ; make 67 | + 68 | + 69 | +clean: 70 | + rm -f $(QCBOR_OBJS) $(LIB_INSTALL_PATH)/libqcbor.a 71 | diff --git a/inc/qcbor/UsefulBuf.h b/inc/qcbor/UsefulBuf.h 72 | index aa24507..ddd20e0 100644 73 | --- a/inc/qcbor/UsefulBuf.h 74 | +++ b/inc/qcbor/UsefulBuf.h 75 | @@ -71,8 +71,8 @@ IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 76 | 77 | =============================================================================*/ 78 | 79 | -#ifndef _UsefulBuf_h 80 | -#define _UsefulBuf_h 81 | +#ifndef UsefulBuf_h_ 82 | +#define UsefulBuf_h_ 83 | 84 | 85 | /* 86 | @@ -832,6 +832,7 @@ typedef struct useful_out_buf { 87 | uint16_t magic; /* Used to detect corruption and lack 88 | * of initialization */ 89 | uint8_t err; 90 | + uint8_t padding_[5]; // explicit padding 91 | } UsefulOutBuf; 92 | 93 | 94 | @@ -1416,6 +1417,7 @@ typedef struct useful_input_buf { 95 | size_t cursor; /* Current offset in data being parse */ 96 | uint16_t magic; /* Check for corrupted or uninitialized UsefulInputBuf */ 97 | uint8_t err; /* Set request goes off end or magic number is bad */ 98 | + uint8_t padding_[5]; // explicit padding 99 | } UsefulInputBuf; 100 | 101 | #define UIB_MAGIC (0xB00F) 102 | @@ -2459,6 +2461,4 @@ static inline void UsefulInputBuf_SetBufferLength(UsefulInputBuf *pMe, size_t uN 103 | } 104 | #endif 105 | 106 | -#endif /* _UsefulBuf_h */ 107 | - 108 | - 109 | +#endif /* UsefulBuf_h_ */ 110 | diff --git a/inc/qcbor/qcbor.h b/inc/qcbor/qcbor.h 111 | index 7b8096b..e4aea87 100644 112 | --- a/inc/qcbor/qcbor.h 113 | +++ b/inc/qcbor/qcbor.h 114 | @@ -37,5 +37,17 @@ 115 | * into four include files). 116 | */ 117 | 118 | -#include "qcbor_encode.h" 119 | +#if defined(ARCH_ENDIAN_LITTLE) && ARCH_ENDIAN_LITTLE 120 | +#define USEFULBUF_CONFIG_LITTLE_ENDIAN 1 121 | +#elif defined(ARCH_ENDIAN_BIG) && ARCH_ENDIAN_BIG 122 | +#define USEFULBUF_CONFIG_BIG_ENDIAN 1 123 | +#endif 124 | + 125 | +//#include "qcbor_encode.h" 126 | #include "qcbor_decode.h" 127 | + 128 | +typedef UsefulBuf useful_buff_t; 129 | +typedef UsefulBufC const_useful_buff_t; 130 | + 131 | +typedef QCBORDecodeContext qcbor_dec_ctxt_t; 132 | +typedef QCBORItem qcbor_item_t; 133 | diff --git a/inc/qcbor/qcbor_common.h b/inc/qcbor/qcbor_common.h 134 | index 127537d..b2692e5 100644 135 | --- a/inc/qcbor/qcbor_common.h 136 | +++ b/inc/qcbor/qcbor_common.h 137 | @@ -62,8 +62,12 @@ IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 138 | some aspects of tag handling that are different. This define can be 139 | used handle these variances. 140 | */ 141 | -#define QCBOR_SPIFFY_DECODE 142 | +// Disabled for Gunyah usage 143 | +// #define QCBOR_SPIFFY_DECODE 144 | 145 | +#define QCBOR_CONFIG_DISABLE_EXP_AND_MANTISSA 146 | +#define QCBOR_DISABLE_TAGS 147 | +#define USEFULBUF_DISABLE_ALL_FLOAT 148 | 149 | /* It was originally defined as QCBOR_CONFIG_DISABLE_EXP_AND_MANTISSA, 150 | * but this is incosistent with all the other QCBOR_DISABLE_ 151 | diff --git a/inc/qcbor/qcbor_decode.h b/inc/qcbor/qcbor_decode.h 152 | index bf30e6d..a67cc40 100644 153 | --- a/inc/qcbor/qcbor_decode.h 154 | +++ b/inc/qcbor/qcbor_decode.h 155 | @@ -48,6 +48,9 @@ extern "C" { 156 | #endif 157 | #endif 158 | 159 | +#pragma GCC diagnostic push 160 | +#pragma GCC diagnostic ignored "-Wreserved-identifier" 161 | + 162 | 163 | /** 164 | * @file qcbor_decode.h 165 | @@ -392,6 +395,7 @@ typedef struct _QCBORItem { 166 | * up by calling QCBORDecode_SetMemPool() or 167 | * QCBORDecode_SetUpAllocator(). */ 168 | uint8_t uLabelAlloc; 169 | + uint8_t padding_[2]; // explicit padding 170 | 171 | /** The union holding the item's value. Select union member based 172 | * on @c uDataType. */ 173 | @@ -1327,6 +1331,7 @@ static inline int QCBOR_Int64ToUInt64(int64_t src, uint64_t *dest) 174 | */ 175 | typedef struct { 176 | uint8_t uNumTags; 177 | + uint8_t padding_[7]; // explicit padding 178 | const uint64_t *puTags; 179 | } QCBORTagListIn; 180 | 181 | @@ -1347,6 +1352,7 @@ typedef struct { 182 | typedef struct { 183 | uint8_t uNumUsed; 184 | uint8_t uNumAllocated; 185 | + uint8_t padding_[6]; // explicit padding 186 | uint64_t *puTags; 187 | } QCBORTagListOut; 188 | 189 | @@ -1479,4 +1485,5 @@ static inline bool QCBORDecode_IsUnrecoverableError(QCBORError uErr) 190 | } 191 | #endif 192 | 193 | +#pragma GCC diagnostic pop 194 | #endif /* qcbor_decode_h */ 195 | diff --git a/inc/qcbor/qcbor_private.h b/inc/qcbor/qcbor_private.h 196 | index 9a2a720..acefe18 100644 197 | --- a/inc/qcbor/qcbor_private.h 198 | +++ b/inc/qcbor/qcbor_private.h 199 | @@ -47,6 +47,8 @@ extern "C" { 200 | #endif 201 | #endif 202 | 203 | +#pragma GCC diagnostic push 204 | +#pragma GCC diagnostic ignored "-Wreserved-identifier" 205 | 206 | /* 207 | The maxium nesting of arrays and maps when encoding or decoding. 208 | @@ -147,6 +149,7 @@ typedef struct __QCBORTrackNesting { 209 | uint16_t uCount; // Number of items in the arrary or map; counts items 210 | // in a map, not pairs of items 211 | uint8_t uMajorType; // Indicates if item is a map or an array 212 | + uint8_t padding_[1]; // explicit padding 213 | } pArrays[QCBOR_MAX_ARRAY_NESTING1+1], // stored state for the nesting levels 214 | *pCurrentNesting; // the current nesting level 215 | } QCBORTrackNesting; 216 | @@ -225,6 +228,7 @@ typedef struct __QCBORDecodeNesting { 217 | padding for alignment will be saved. 218 | */ 219 | uint8_t uLevelType; 220 | + uint8_t padding_[3]; // explicit padding 221 | union { 222 | struct { 223 | #define QCBOR_COUNT_INDICATES_INDEFINITE_LENGTH UINT16_MAX 224 | @@ -307,6 +311,7 @@ struct _QCBORDecodeContext { 225 | uint8_t uDecodeMode; 226 | uint8_t bStringAllocateAll; 227 | uint8_t uLastError; // QCBORError stuffed into a uint8_t 228 | + uint8_t padding_[1]; // explicit padding 229 | 230 | /* See MapTagNumber() for description of how tags are mapped. */ 231 | uint64_t auMappedTags[QCBOR_NUM_MAPPED_TAGS]; 232 | @@ -348,4 +353,6 @@ struct _QCBORDecodeContext { 233 | } 234 | #endif 235 | 236 | +#pragma GCC diagnostic pop 237 | + 238 | #endif /* qcbor_private_h */ 239 | diff --git a/inc/qcbor/qcbor_spiffy_decode.h b/inc/qcbor/qcbor_spiffy_decode.h 240 | index 0faddc3..7f306df 100644 241 | --- a/inc/qcbor/qcbor_spiffy_decode.h 242 | +++ b/inc/qcbor/qcbor_spiffy_decode.h 243 | @@ -232,7 +232,7 @@ static void QCBORDecode_GetInt64InMapSZ(QCBORDecodeContext *pCtx, 244 | Please see @ref Decode-Errors-Overview "Decode Errors Overview". 245 | 246 | If the CBOR data type can never be convered by this function or the 247 | - conversion was not selected in @c uConversionTypes @ref 248 | + conversion was not selected in @c uConversionTypes 249 | @ref QCBOR_ERR_UNEXPECTED_TYPE is set. 250 | 251 | When converting floating-point values, the integer is rounded to the 252 | @@ -2469,6 +2469,9 @@ QCBORDecode_GetRegexInMapSZ(QCBORDecodeContext *pMe, 253 | QCBORDecode_GetTaggedStringInMapSZ(pMe, szLabel, TagSpec, pRegex); 254 | } 255 | 256 | +// Casting function returning enum QCBORError 257 | +#pragma clang diagnostic push 258 | +#pragma clang diagnostic ignored "-Wbad-function-cast" 259 | 260 | static inline void 261 | QCBORDecode_GetMIMEMessage(QCBORDecodeContext *pMe, 262 | @@ -2530,6 +2533,7 @@ QCBORDecode_GetMIMEMessageInMapSZ(QCBORDecodeContext *pMe, 263 | } 264 | } 265 | 266 | +#pragma clang diagnostic pop 267 | 268 | static inline void 269 | QCBORDecode_GetBinaryUUID(QCBORDecodeContext *pMe, 270 | diff --git a/src/qcbor_decode.c b/src/qcbor_decode.c 271 | index 8a547ee..9f49b58 100644 272 | --- a/src/qcbor_decode.c 273 | +++ b/src/qcbor_decode.c 274 | @@ -47,9 +47,6 @@ IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 275 | 276 | 277 | 278 | -#define SIZEOF_C_ARRAY(array,type) (sizeof(array)/sizeof(type)) 279 | - 280 | - 281 | 282 | 283 | static inline bool 284 | @@ -385,7 +382,7 @@ DecodeNesting_DescendMapOrArray(QCBORDecodeNesting *pNesting, 285 | DecodeNesting_ClearBoundedMode(pNesting); 286 | 287 | Done: 288 | - return uError;; 289 | + return uError; 290 | } 291 | 292 | 293 | @@ -435,7 +432,7 @@ DecodeNesting_DescendIntoBstrWrapped(QCBORDecodeNesting *pNesting, 294 | pNesting->pCurrentBounded = pNesting->pCurrent; 295 | 296 | Done: 297 | - return uError;; 298 | + return uError; 299 | } 300 | 301 | 302 | @@ -656,6 +653,7 @@ void QCBORDecode_SetCallerConfiguredTagList(QCBORDecodeContext *pMe, 303 | * analyzers. 304 | */ 305 | 306 | +#pragma clang diagnostic ignored "-Wdocumentation" 307 | 308 | /** 309 | * @brief Decode the CBOR head, the type and argument. 310 | @@ -920,7 +918,7 @@ DecodeType7(int nAdditionalInfo, uint64_t uArgument, QCBORItem *pDecodedItem) 311 | goto Done; 312 | } 313 | /* FALLTHROUGH */ 314 | - 315 | + __attribute__((fallthrough)); 316 | default: /* 0-19 */ 317 | pDecodedItem->uDataType = QCBOR_TYPE_UKNOWN_SIMPLE; 318 | /* DecodeHead() will make uArgument equal to 319 | @@ -1013,7 +1011,7 @@ Done: 320 | * @brief Map the CBOR major types for strings to the QCBOR types. 321 | * 322 | * @param[in] nCBORMajorType The CBOR major type to convert. 323 | - * @retturns QCBOR type number. 324 | + * @returns QCBOR type number. 325 | * 326 | * This only works for the two string types. 327 | */ 328 | @@ -1035,7 +1033,7 @@ static inline uint8_t ConvertStringMajorTypes(int nCBORMajorType) 329 | * @brief Map the CBOR major types for arrays/maps to the QCBOR types. 330 | * 331 | * @param[in] nCBORMajorType The CBOR major type to convert. 332 | - * @retturns QCBOR type number. 333 | + * @returns QCBOR type number. 334 | * 335 | * This only works for the two aggregate types. 336 | */ 337 | @@ -1948,7 +1946,6 @@ static QCBORError DecodeDateEpoch(QCBORItem *pDecodedItem) 338 | */ 339 | uReturn = QCBOR_ERR_DATE_OVERFLOW; 340 | goto Done; 341 | - break; 342 | 343 | case QCBOR_TYPE_DOUBLE: 344 | case QCBOR_TYPE_FLOAT: 345 | @@ -1993,6 +1990,7 @@ static QCBORError DecodeDateEpoch(QCBORItem *pDecodedItem) 346 | pDecodedItem->val.epochDate.nSeconds = (int64_t)d; 347 | pDecodedItem->val.epochDate.fSecondsFraction = 348 | d - (double)pDecodedItem->val.epochDate.nSeconds; 349 | + break; 350 | } 351 | #else /* QCBOR_DISABLE_FLOAT_HW_USE */ 352 | 353 | @@ -2000,7 +1998,6 @@ static QCBORError DecodeDateEpoch(QCBORItem *pDecodedItem) 354 | goto Done; 355 | 356 | #endif /* QCBOR_DISABLE_FLOAT_HW_USE */ 357 | - break; 358 | 359 | default: 360 | /* It's the arrays and maps that are unrecoverable because 361 | @@ -2047,7 +2044,6 @@ static QCBORError DecodeDaysEpoch(QCBORItem *pDecodedItem) 362 | */ 363 | uReturn = QCBOR_ERR_DATE_OVERFLOW; 364 | goto Done; 365 | - break; 366 | 367 | default: 368 | /* It's the arrays and maps that are unrecoverable because 369 | @@ -2056,7 +2052,6 @@ static QCBORError DecodeDaysEpoch(QCBORItem *pDecodedItem) 370 | * recoverable for non-arrays and maps like strings. */ 371 | uReturn = QCBOR_ERR_UNRECOVERABLE_TAG_CONTENT; 372 | goto Done; 373 | - break; 374 | } 375 | 376 | pDecodedItem->uDataType = QCBOR_TYPE_DAYS_EPOCH; 377 | @@ -2229,6 +2224,9 @@ static inline QCBORError DecodeMIME(QCBORItem *pDecodedItem) 378 | } 379 | #endif /* QCBOR_DISABLE_UNCOMMON_TAGS */ 380 | 381 | +// Missing padding 382 | +#pragma clang diagnostic push 383 | +#pragma clang diagnostic ignored "-Wpadded" 384 | /** 385 | * Table of CBOR tags whose content is either a text string or a byte 386 | * string. The table maps the CBOR tag to the QCBOR type. The high-bit 387 | @@ -2240,6 +2238,8 @@ struct StringTagMapEntry { 388 | uint8_t uQCBORtype; 389 | }; 390 | 391 | +#pragma clang diagnostic pop 392 | + 393 | #define IS_BYTE_STRING_BIT 0x80 394 | #define QCBOR_TYPE_MASK ~IS_BYTE_STRING_BIT 395 | 396 | @@ -2470,6 +2470,9 @@ QCBORDecode_PeekNext(QCBORDecodeContext *pMe, QCBORItem *pDecodedItem) 397 | return uErr; 398 | } 399 | 400 | +// Casting function returning enum QCBORError 401 | +#pragma clang diagnostic push 402 | +#pragma clang diagnostic ignored "-Wbad-function-cast" 403 | 404 | /* 405 | * Public function, see header qcbor/qcbor_decode.h file 406 | @@ -4742,10 +4745,10 @@ ConvertInt64(const QCBORItem *pItem, uint32_t uConvertTypes, int64_t *pnValue) 407 | } else { 408 | return QCBOR_ERR_UNEXPECTED_TYPE; 409 | } 410 | + break; 411 | #else 412 | return QCBOR_ERR_HW_FLOAT_DISABLED; 413 | #endif /* QCBOR_DISABLE_FLOAT_HW_USE */ 414 | - break; 415 | 416 | case QCBOR_TYPE_INT64: 417 | if(uConvertTypes & QCBOR_CONVERT_TYPE_XINT64) { 418 | @@ -4852,7 +4855,7 @@ Int64ConvertAll(const QCBORItem *pItem, uint32_t uConvertTypes, int64_t *pnValue 419 | } else { 420 | return QCBOR_ERR_UNEXPECTED_TYPE; 421 | } 422 | - break; 423 | +// break; 424 | 425 | case QCBOR_TYPE_NEGBIGNUM: 426 | if(uConvertTypes & QCBOR_CONVERT_TYPE_BIG_NUM) { 427 | @@ -4860,7 +4863,7 @@ Int64ConvertAll(const QCBORItem *pItem, uint32_t uConvertTypes, int64_t *pnValue 428 | } else { 429 | return QCBOR_ERR_UNEXPECTED_TYPE; 430 | } 431 | - break; 432 | +// break; 433 | 434 | #ifndef QCBOR_DISABLE_EXP_AND_MANTISSA 435 | case QCBOR_TYPE_DECIMAL_FRACTION: 436 | @@ -5093,10 +5096,10 @@ static QCBORError ConvertUInt64(const QCBORItem *pItem, uint32_t uConvertTypes, 437 | } else { 438 | return QCBOR_ERR_UNEXPECTED_TYPE; 439 | } 440 | + break; 441 | #else 442 | return QCBOR_ERR_HW_FLOAT_DISABLED; 443 | #endif /* QCBOR_DISABLE_FLOAT_HW_USE */ 444 | - break; 445 | 446 | case QCBOR_TYPE_INT64: 447 | if(uConvertTypes & QCBOR_CONVERT_TYPE_XINT64) { 448 | @@ -5197,7 +5200,7 @@ UInt64ConvertAll(const QCBORItem *pItem, uint32_t uConvertTypes, uint64_t *puVal 449 | } else { 450 | return QCBOR_ERR_UNEXPECTED_TYPE; 451 | } 452 | - break; 453 | +// break; 454 | 455 | case QCBOR_TYPE_NEGBIGNUM: 456 | if(uConvertTypes & QCBOR_CONVERT_TYPE_BIG_NUM) { 457 | @@ -5205,7 +5208,7 @@ UInt64ConvertAll(const QCBORItem *pItem, uint32_t uConvertTypes, uint64_t *puVal 458 | } else { 459 | return QCBOR_ERR_UNEXPECTED_TYPE; 460 | } 461 | - break; 462 | +// break; 463 | 464 | #ifndef QCBOR_DISABLE_EXP_AND_MANTISSA 465 | 466 | @@ -5618,6 +5621,7 @@ DoubleConvertAll(const QCBORItem *pItem, uint32_t uConvertTypes, double *pdValue 467 | 468 | } 469 | 470 | +#pragma clang diagnostic pop 471 | 472 | /* 473 | Public function, see header qcbor/qcbor_decode.h file 474 | -------------------------------------------------------------------------------- /scripts/dockerfile-hyp: -------------------------------------------------------------------------------- 1 | # © 2022 Qualcomm Innovation Center, Inc. All rights reserved. 2 | # 3 | # SPDX-License-Identifier: BSD-3-Clause 4 | 5 | # FROM ubuntu:22.04 6 | ARG REGISTRY 7 | FROM "$REGISTRY" 8 | 9 | RUN apt-get update 10 | 11 | ARG DEBIAN_FRONTEND=noninteractive 12 | ENV TZ=America/Los_Angeles \ 13 | LANG=en_US.UTF-8 \ 14 | LANGUAGE=en_US:en \ 15 | LC_ALL=en_US.UTF-8 16 | 17 | # These defaults below are overridden by DOCKER_OPTIONS 18 | ARG USER=test 19 | ARG UID=50000 20 | ARG GID=200 21 | 22 | # Setup the user with default docker password (change if required) 23 | RUN groupadd -g "$GID" "$USER" && \ 24 | useradd -m -u "$UID" -g "$GID" -s /bin/bash "$USER" && \ 25 | echo "$USER:1234" | chpasswd && adduser "$USER" sudo 26 | 27 | RUN apt-get update && apt-get install -y apt-utils 28 | 29 | RUN apt-get install -y \ 30 | software-properties-common \ 31 | build-essential \ 32 | tzdata \ 33 | python3 \ 34 | python3-pip \ 35 | sudo \ 36 | git \ 37 | make \ 38 | gcc \ 39 | g++ \ 40 | ninja-build \ 41 | gcc-aarch64-linux-gnu \ 42 | g++-aarch64-linux-gnu \ 43 | cpio \ 44 | device-tree-compiler \ 45 | wget \ 46 | lsb-release \ 47 | # QEMU dependencies 48 | libglib2.0-dev \ 49 | libfdt-dev \ 50 | libpixman-1-dev \ 51 | zlib1g-dev \ 52 | meson \ 53 | # gdb 9.2 dependencies 54 | libreadline-dev \ 55 | texinfo \ 56 | libexpat1-dev \ 57 | libmpfr-dev \ 58 | # Linux dependencies 59 | bc \ 60 | libssl-dev \ 61 | flex \ 62 | bison \ 63 | # Toolchain dependencies 64 | libz3-dev \ 65 | cmake \ 66 | vim \ 67 | scons \ 68 | gdb-multiarch \ 69 | telnet \ 70 | podman \ 71 | curl \ 72 | fdisk \ 73 | locales \ 74 | chrpath diffstat file gawk lz4 zstd \ 75 | # Repos dependencies 76 | python3-venv 77 | 78 | RUN locale-gen en_US.UTF-8 && \ 79 | dpkg-reconfigure locales 80 | 81 | ENV HOME="/home/$USER" 82 | RUN mkdir -p ${HOME} && \ 83 | chown -R $USER:$USER ${HOME} && \ 84 | echo "Creating Docker image for user : $USER" 85 | 86 | ENV BASE_DIR="${HOME}" 87 | ENV WORKSPACE="${BASE_DIR}/mnt/workspace" 88 | ENV TOOLS_DIR="/usr/local/mnt/tools" 89 | 90 | # Set version, so that we can take action for updates 91 | ARG ENV_VER="" 92 | ENV ENV_VERSION=$ENV_VER 93 | 94 | RUN mkdir -p ${BASE_DIR}/utils ${BASE_DIR}/core-utils 95 | 96 | # Copy core utils (used to build docker image) files from host machine. 97 | COPY --chown=$USER:$USER ./utils/* "${BASE_DIR}/utils/" 98 | COPY --chown=$USER:$USER ./core-utils/* "${BASE_DIR}/core-utils/" 99 | 100 | # Add tools paths to search path 101 | ENV PATH="${PATH}:${BASE_DIR}/utils:${BASE_DIR}/core-utils" 102 | 103 | WORKDIR ${BASE_DIR} 104 | 105 | ## Since all files/dirs created so far are owned by root, change the permissions/owner 106 | RUN touch "${HOME}/.bashrc" && \ 107 | chown -R $USER:$USER "${BASE_DIR}" && \ 108 | ln -s python3 /usr/bin/python 109 | 110 | # Switch to user account 111 | USER "$USER" 112 | 113 | ## Add any volume based environment scripts env updates to .bashrc 114 | RUN echo "" >> "${HOME}/.bashrc" && \ 115 | echo 'if [[ -f ${TOOLS_DIR}/.tools-env ]] ; then' >> "${HOME}/.bashrc" && \ 116 | echo ' source ${TOOLS_DIR}/.tools-env' >> "${HOME}/.bashrc" && \ 117 | echo ' #echo \"sourced from ${TOOLS_DIR}/.tools-env\"' >> "${HOME}/.bashrc" && \ 118 | echo "fi" >> "${HOME}/.bashrc" && \ 119 | echo "" >> "${HOME}/.bashrc" && \ 120 | echo 'if [[ -f ${WORKSPACE}/.wsp-env ]] ; then' >> "${HOME}/.bashrc" && \ 121 | echo ' source ${WORKSPACE}/.wsp-env' >> "${HOME}/.bashrc" && \ 122 | echo ' #echo \"sourced from ${WORKSPACE}/.wsp-env\"' >> "${HOME}/.bashrc" && \ 123 | echo "fi" >> "${HOME}/.bashrc" && \ 124 | echo "" >> "${HOME}/.bashrc" 125 | 126 | ## Add any .bashrc user customizations to .bashrc 127 | COPY --chown=$USER:$USER bashrc-extn . 128 | RUN cat bashrc-extn >> "${HOME}/.bashrc" && rm -f bashrc-extn 129 | 130 | CMD /bin/bash 131 | -------------------------------------------------------------------------------- /scripts/install-core-tools.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # © 2022 Qualcomm Innovation Center, Inc. All rights reserved. 4 | # 5 | # SPDX-License-Identifier: BSD-3-Clause 6 | 7 | set -e 8 | 9 | echo "Generate/build the tools" 10 | 11 | echo "BASE_DIR : ${BASE_DIR}" 12 | echo "TOOLS_DIR : ${TOOLS_DIR}" 13 | 14 | if [[ -z "$USER" ]]; then 15 | echo "User is not set in Environment" 16 | export USER=`whoami` 17 | fi 18 | 19 | if [[ ! -d $(dirname "${TOOLS_DIR}") ]]; then 20 | echo "Tools volume not mounted" 21 | return 22 | fi 23 | 24 | if [[ ! -d ${TOOLS_DIR} ]]; then 25 | echo "Creating tools mount folder" 26 | sudo mkdir -p ${TOOLS_DIR} 27 | sudo chown $USER:$USER ${TOOLS_DIR} 28 | 29 | touch ${TOOLS_DIR}/.tools-env 30 | chmod 0775 ${TOOLS_DIR}/.tools-env 31 | fi 32 | 33 | if [[ ! -d ${TOOLS_DIR}/llvm ]]; then 34 | echo -e "\nLLVM toolchain not found..." 35 | 36 | echo -e 'export LLVM=${TOOLS_DIR}/llvm' >> ${TOOLS_DIR}/.tools-env 37 | echo -e 'export PATH=$PATH:$LLVM/bin' >> ${TOOLS_DIR}/.tools-env 38 | 39 | . ${TOOLS_DIR}/.tools-env 40 | 41 | . ${BASE_DIR}/core-utils/clone-llvm.sh 42 | . ${BASE_DIR}/core-utils/build-llvm.sh 43 | fi 44 | 45 | if [[ ! -d ${TOOLS_DIR}/qemu ]]; then 46 | echo -e "\nBuilding qemu tools" 47 | 48 | echo -e 'export QEMU_INSTALL_DIR=${TOOLS_DIR}/qemu' >> ${TOOLS_DIR}/.tools-env 49 | echo -e 'export QEMU=${QEMU_INSTALL_DIR}/bin/qemu-system-aarch64' >> ${TOOLS_DIR}/.tools-env 50 | echo -e 'export PATH=$PATH:$QEMU_INSTALL_DIR/bin' >> ${TOOLS_DIR}/.tools-env 51 | 52 | . ${TOOLS_DIR}/.tools-env 53 | 54 | . ${BASE_DIR}/core-utils/clone-qemu.sh 55 | . ${BASE_DIR}/core-utils/build-qemu.sh 56 | 57 | fi 58 | 59 | echo -e "Installation of the core tools is complete\n" 60 | -------------------------------------------------------------------------------- /scripts/install-wsp-imgs.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # © 2022 Qualcomm Innovation Center, Inc. All rights reserved. 4 | # 5 | # SPDX-License-Identifier: BSD-3-Clause 6 | 7 | set -e 8 | 9 | echo "Building the workspace images" 10 | 11 | echo "BASE_DIR : ${BASE_DIR}" 12 | echo "TOOLS_DIR : ${TOOLS_DIR}" 13 | echo "WORKSPACE : ${WORKSPACE}" 14 | echo "" 15 | 16 | if [[ -z "$USER" ]]; then 17 | echo "User is not set in Environment" 18 | export USER=`whoami` 19 | fi 20 | 21 | if [[ ! -d $(dirname "${WORKSPACE}") ]]; then 22 | echo "Workspace volume not mounted" 23 | return 24 | fi 25 | 26 | if [[ ! -d ${WORKSPACE} ]]; then 27 | 28 | echo "Creating workspace folder" 29 | sudo mkdir -p ${WORKSPACE} 30 | sudo chmod 0775 ${WORKSPACE} 31 | sudo chown $USER:$USER ${WORKSPACE} 32 | 33 | touch ${WORKSPACE}/.wsp-env 34 | chmod 0775 ${WORKSPACE}/.wsp-env 35 | 36 | echo -e 'if [[ -z "${USER}" ]]; then\n export USER=`whoami`\nfi' >> ${WORKSPACE}/.wsp-env 37 | . ${WORKSPACE}/.wsp-env 38 | fi 39 | 40 | # Source the tools environment, otherwise builds fail which need LLVM 41 | if [[ -f ${TOOLS_DIR}/.tools-env ]]; then 42 | echo "Sourcing tools environment" 43 | . ${TOOLS_DIR}/.tools-env 44 | fi 45 | 46 | # Source the workspace environment if its there 47 | if [[ -f ${WORKSPACE}/.wsp-env ]]; then 48 | echo "Sourcing workspace environment" 49 | . ${WORKSPACE}/.wsp-env 50 | fi 51 | 52 | if [[ ! -d ${WORKSPACE}/imgs ]]; then 53 | mkdir -p ${WORKSPACE}/imgs 54 | 55 | echo -e 'export QEMU_IMGS_DIR=${WORKSPACE}/imgs' >> ${WORKSPACE}/.wsp-env 56 | . $WORKSPACE/.wsp-env 57 | fi 58 | 59 | if [[ ! -d ${WORKSPACE}/gunyah-venv ]]; then 60 | echo -e \n"Building gunyah venv" 61 | 62 | . ${BASE_DIR}/core-utils/build-py-vdev.sh 63 | 64 | . $WORKSPACE/.wsp-env 65 | fi 66 | 67 | if [[ ! -d ${WORKSPACE}/lib/app-sysroot ]]; then 68 | echo -e "\nBuilding app sysroot" 69 | 70 | echo -e 'export LIB_DIR=${WORKSPACE}/lib' >> ${WORKSPACE}/.wsp-env 71 | echo -e 'export LOCAL_SYSROOT=${LIB_DIR}/app-sysroot' >> ${WORKSPACE}/.wsp-env 72 | 73 | . $WORKSPACE/.wsp-env 74 | 75 | . ${BASE_DIR}/core-utils/clone-qcbor-dtc.sh 76 | . ${BASE_DIR}/core-utils/build-sysroot.sh 77 | fi 78 | 79 | if [[ -z "${LINUX_DIR}" ]]; then 80 | echo -e 'export LINUX_DIR=${WORKSPACE}/linux' >> ${WORKSPACE}/.wsp-env 81 | echo -e 'export RAMDISK_FILE_PATH=${LINUX_DIR}/initrd.img' >> ${WORKSPACE}/.wsp-env 82 | 83 | . $WORKSPACE/.wsp-env 84 | fi 85 | 86 | if [[ ! -d $LINUX_DIR ]]; then 87 | echo "Building Linux kernel and Ramdisk image" 88 | 89 | . ${BASE_DIR}/core-utils/clone-linux.sh 90 | . ${BASE_DIR}/core-utils/build-linux.sh 91 | . ${BASE_DIR}/core-utils/build-ramdisk.sh 92 | fi 93 | 94 | # Copy scripts to destinations 95 | if [[ ! -f ${WORKSPACE}/kern-test.sh ]]; then 96 | cp ${BASE_DIR}/utils/kern-test.sh ${WORKSPACE}/kern-test.sh 97 | fi 98 | 99 | if [[ ! -f ${WORKSPACE}/run-qemu.sh ]]; then 100 | cp ${BASE_DIR}/utils/run-qemu.sh ${WORKSPACE}/run-qemu.sh 101 | fi 102 | 103 | if [[ ! -f ${WORKSPACE}/crosvm/crosvm ]]; then 104 | mkdir -p ${WORKSPACE}/crosvm 105 | cd ${WORKSPACE}/crosvm 106 | . clone-crosvm.sh 107 | . build-crosvm.sh 108 | 109 | echo -e 'export CROSVM_FILE_PATH=${WORKSPACE}/crosvm/crosvm' >> ${WORKSPACE}/.wsp-env 110 | . ${WORKSPACE}/.wsp-env 111 | fi 112 | 113 | if [[ ! -f ${WORKSPACE}/rootfs/rootfs-extfs-disk.img ]]; then 114 | echo -e "\nrootfs image not found, creating new one" 115 | . build-rootfs-img.sh 116 | 117 | echo -e 'export VIRTIO_DEVICE_FILE=${WORKSPACE}/rootfs/rootfs-extfs-disk.img' >> ${WORKSPACE}/.wsp-env 118 | . ${WORKSPACE}/.wsp-env 119 | fi 120 | 121 | echo "Installation of the workspace images is completed" 122 | -------------------------------------------------------------------------------- /scripts/migrate-tools-to-vol.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # © 2022 Qualcomm Innovation Center, Inc. All rights reserved. 4 | # 5 | # SPDX-License-Identifier: BSD-3-Clause 6 | 7 | # This script will run in the Old docker image instance to migrate the tools 8 | # that were built during original old docker image generation, thus saving 9 | # time and space to generate newer version of docker images using new scripts 10 | 11 | set -e 12 | 13 | echo "Migrating the old tools/data into volumes for usage in the new container" 14 | 15 | WORKSPACE="${BASE_DIR}/mnt/workspace" 16 | NEW_TOOLS_DIR="/usr/local/mnt/tools" 17 | 18 | if [[ ! -d ${NEW_TOOLS_DIR} ]]; then 19 | echo "Creating tools mount folder" 20 | sudo mkdir -p ${NEW_TOOLS_DIR} 21 | 22 | sudo touch ${NEW_TOOLS_DIR}/.tools-env 23 | sudo chmod 0777 ${NEW_TOOLS_DIR}/.tools-env 24 | fi 25 | 26 | if [[ ! -d ${NEW_TOOLS_DIR}/llvm ]]; then 27 | echo "Copying llvm folder to tools volume" 28 | sudo cp -r /usr/local/llvm ${NEW_TOOLS_DIR}/ 29 | 30 | sudo echo -e 'export LLVM=${TOOLS_DIR}/llvm' >> ${NEW_TOOLS_DIR}/.tools-env 31 | sudo echo -e 'export PATH=$PATH:$LLVM/bin' >> ${NEW_TOOLS_DIR}/.tools-env 32 | fi 33 | 34 | if [[ ! -d ${NEW_TOOLS_DIR}/qemu ]]; then 35 | echo "Copying qemu folder to tools volume" 36 | sudo cp -r ${BASE_DIR}/tools/qemu ${NEW_TOOLS_DIR}/ 37 | 38 | sudo rm -rf ${NEW_TOOLS_DIR}/qemu/imgs ${NEW_TOOLS_DIR}/qemu/*.sh 39 | 40 | sudo echo -e 'export QEMU_INSTALL_DIR=${TOOLS_DIR}/qemu' >> ${NEW_TOOLS_DIR}/.tools-env 41 | sudo echo -e 'export QEMU=${QEMU_INSTALL_DIR}/bin/qemu-system-aarch64' >> ${NEW_TOOLS_DIR}/.tools-env 42 | sudo echo -e 'export PATH=$PATH:$QEMU_INSTALL_DIR/bin' >> ${NEW_TOOLS_DIR}/.tools-env 43 | fi 44 | 45 | if [[ ! -d ${WORKSPACE} ]]; then 46 | if [[ -z "${USER}" ]]; then 47 | USER=`whoami` 48 | fi 49 | 50 | echo "Creating workspace folder" 51 | sudo mkdir -p ${WORKSPACE} 52 | sudo chmod 0777 ${WORKSPACE} 53 | sudo chown $USER:$USER ${WORKSPACE} 54 | 55 | touch ${WORKSPACE}/.wsp-env 56 | chmod 0775 ${WORKSPACE}/.wsp-env 57 | fi 58 | 59 | if [[ ! -d ${WORKSPACE}/imgs ]]; then 60 | cp -r ${QEMU_IMGS_DIR} ${WORKSPACE}/ 61 | 62 | echo -e 'export QEMU_IMGS_DIR=${WORKSPACE}/imgs' >> ${WORKSPACE}/.wsp-env 63 | fi 64 | 65 | if [[ ! -d ${WORKSPACE}/linux ]]; then 66 | echo "Copying linux folder" 67 | 68 | OLD_RAMDISK_FILE_PATH=`grep "RAMDISK_FILE_PATH" ~/.bashrc` 69 | 70 | echo -e 'export LINUX_DIR=${WORKSPACE}/linux' >> ${WORKSPACE}/.wsp-env 71 | BUSYBOX_PATH=`echo ${OLD_RAMDISK_FILE_PATH} | sed -re 's/.+(busybox.*$)/\$LINUX_DIR\/\1/'` 72 | echo -e "export RAMDISK_FILE_PATH=$BUSYBOX_PATH" >> ${WORKSPACE}/.wsp-env 73 | 74 | cp -r ${LINUX_DIR} ${WORKSPACE}/ 75 | 76 | echo "Done copying linux files" 77 | fi 78 | 79 | if [[ -d ~/share/docker-share/crosvm ]]; then 80 | mv ~/share/docker-share/crosvm ${WORKSPACE}/ 81 | echo "Found crosvm, moved into workspace folder" 82 | mv ${WORKSPACE}/crosvm/crosvm ${WORKSPACE}/crosvm/crosvm-src 83 | cp ${WORKSPACE}/crosvm/crosvm-src/crosvm ${WORKSPACE}/crosvm/crosvm 84 | rm -rf ${WORKSPACE}/crosvm/crosvm-src 85 | echo -e 'export CROSVM_FILE_PATH=${WORKSPACE}/crosvm/crosvm' >> ${WORKSPACE}/.wsp-env 86 | fi 87 | 88 | if [[ -d ~/share/docker-share/rootfs ]]; then 89 | mv ~/share/docker-share/rootfs ${WORKSPACE}/ 90 | rm -rf ${WORKSPACE}/rootfs/oe-rpb 91 | echo "Moved rootfs to workspace folder" 92 | echo -e 'export VIRTIO_DEVICE_FILE=${WORKSPACE}/rootfs/rootfs-extfs-disk.img' >> ${WORKSPACE}/.wsp-env 93 | fi 94 | 95 | echo "Successfully populated the volumes with data from previous docker image" 96 | echo "Exiting the old docker image" 97 | -------------------------------------------------------------------------------- /scripts/new-term.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # © 2022 Qualcomm Innovation Center, Inc. All rights reserved. 4 | # 5 | # SPDX-License-Identifier: BSD-3-Clause 6 | 7 | set -e 8 | 9 | VERSION_SCRIPT=$(dirname "${BASH_SOURCE[0]}")/version.sh 10 | . ${VERSION_SCRIPT} 11 | 12 | if [[ -z "$DOCKER_TAG" ]]; then 13 | DOCKER_TAG=" hyp-dev-term:${CURRENT_VER} " 14 | fi 15 | 16 | docker exec -it `docker ps | grep "${DOCKER_TAG}" | cut -d ' ' -f 1` /bin/bash 17 | -------------------------------------------------------------------------------- /scripts/run-docker.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # © 2022 Qualcomm Innovation Center, Inc. All rights reserved. 4 | # 5 | # SPDX-License-Identifier: BSD-3-Clause 6 | 7 | set -e 8 | 9 | VERSION_SCRIPT=$(dirname "${BASH_SOURCE[0]}")/version.sh 10 | . ${VERSION_SCRIPT} 11 | 12 | if [[ -z "${HOST_SHARED_DIR}" ]]; then 13 | HOST_SHARED_DIR="/home/$USER/share" 14 | fi 15 | 16 | echo "Host shared folder : ${HOST_TO_DOCKER_SHARED_DIR}" 17 | 18 | if [[ -z "${HOST_TO_DOCKER_SHARED_DIR}" ]]; then 19 | HOST_TO_DOCKER_SHARED_DIR=`pwd` 20 | fi 21 | 22 | # add as many port mappings required as a single entry or a range 23 | if [[ -z "$NO_PORT_MAP" ]]; then 24 | #PORT_MAPPINGS=" -p 1234-1235:1234-1235 " 25 | PORT_MAPPINGS=" -p 1234:1234 " 26 | PORT_MAPPINGS+=" -p 1235:1235 " 27 | fi 28 | 29 | DOCKER_MACHINE_NAME="hyp-dev-env" 30 | 31 | if [[ -z "$DOCKER_TAG" ]]; then 32 | DOCKER_TAG=" hyp-dev-term:${CURRENT_VER} " 33 | fi 34 | 35 | docker run --privileged -h ${DOCKER_MACHINE_NAME} -it \ 36 | ${PORT_MAPPINGS} \ 37 | ${ADDITIONAL_DOCKER_ARGS} \ 38 | -v tools-vol:/usr/local/mnt \ 39 | -v wsp-vol:/home/$USER/mnt \ 40 | -v ${HOST_TO_DOCKER_SHARED_DIR}:${HOST_SHARED_DIR} \ 41 | ${DOCKER_TAG} "$@" 42 | -------------------------------------------------------------------------------- /scripts/utils/bldextfs.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # © 2023 Qualcomm Innovation Center, Inc. All rights reserved. 4 | # 5 | # SPDX-License-Identifier: BSD-3-Clause 6 | 7 | set -e 8 | 9 | # Build extfs image to mount to linux as virtio blk device disk 10 | 11 | VALID_ARGS=$(getopt -o pf:o:s: --long folder:,outfile:,size:,parti -- "$@") 12 | 13 | if [[ $# -le 2 ]]; then 14 | echo -e "Usage: $(basename $0) \n args:" 15 | echo -e "\t -f|--folder path : Input folder path" 16 | echo -e "\t -o|--outfile path : Path name of file to create" 17 | echo -e "\t -s|--size xxx : Image size to create. KMG can be used for units" 18 | echo -e "\t -p|--parti : Create partition in the disk" 19 | return 20 | fi 21 | 22 | eval set -- "$VALID_ARGS" 23 | while [ : ]; do 24 | case "$1" in 25 | -f | --folder) 26 | IN_FOLDER=$2 27 | #echo "Input folder: ${IN_FOLDER}" 28 | shift 2 29 | ;; 30 | -o | --outfile) 31 | OUT_FILE=$2 32 | #echo "Output file: ${OUT_FILE}" 33 | shift 2 34 | ;; 35 | -s | --size) 36 | DISK_SIZE=$2 37 | #echo "Disk size: ${DISK_SIZE}" 38 | shift 2 39 | ;; 40 | -p | --parti) 41 | DISK_PARTI="YES" 42 | #echo "Partition Disk" 43 | shift 44 | ;; 45 | --) shift; 46 | break 47 | ;; 48 | esac 49 | done 50 | 51 | if [[ ! -d ${IN_FOLDER} ]]; then 52 | echo "${IN_FOLDER} not found!" 53 | return 54 | fi 55 | 56 | if [[ -f ${OUT_FILE} ]]; then 57 | echo "File ${OUT_FILE} already exists..!!" 58 | return 59 | fi 60 | 61 | #dd if=/dev/zero of=extfs-disk.img bs=1k count=512k 62 | qemu-img create -f raw ${OUT_FILE} ${DISK_SIZE} 63 | 64 | TEMP_MOUNT_FOLDER=$(dirname ${OUT_FILE})/tmp-ext-fs 65 | mkdir -p ${TEMP_MOUNT_FOLDER} 66 | 67 | if [[ -z ${DISK_PARTI} ]]; then 68 | 69 | # Disk image, NO partitioning 70 | echo "Creating Disk image without partition" 71 | 72 | mkfs.ext4 ${OUT_FILE} 73 | sudo mount -o loop ${OUT_FILE} ${TEMP_MOUNT_FOLDER} 74 | 75 | else 76 | # Disk image, with partitioning 77 | 78 | echo "Creating Partition on the Disk" 79 | 80 | (echo n; echo p; echo 1; echo "" ; echo "" ; echo p; echo w) | fdisk ${OUT_FILE} 81 | 82 | PARTI_OFFSET=`fdisk -l ${OUT_FILE} | grep "${OUT_FILE}1" | xargs | cut -d " " -f 2` 83 | mkfs.ext4 ${OUT_FILE} -E offset=$(( 512 * ${PARTI_OFFSET} )) 84 | 85 | sudo mount -o loop,offset=$(( 512 * $PARTI_OFFSET )) ${OUT_FILE} ${TEMP_MOUNT_FOLDER} 86 | 87 | fi 88 | 89 | echo "Mounted the extfs formatted disk, starting to copy" 90 | 91 | sudo cp -v -r -p ${IN_FOLDER}/* ${TEMP_MOUNT_FOLDER} 92 | 93 | echo "Copying completed, syncing..." 94 | sync 95 | 96 | echo "Sync completed, Unmounting..." 97 | sudo umount ${TEMP_MOUNT_FOLDER} 98 | 99 | echo "Unmounting completed, removing the temp folder..." 100 | rm -rf ${TEMP_MOUNT_FOLDER} 101 | -------------------------------------------------------------------------------- /scripts/utils/build-crosvm.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # © 2023 Qualcomm Innovation Center, Inc. All rights reserved. 4 | # 5 | # SPDX-License-Identifier: BSD-3-Clause 6 | 7 | set -e 8 | 9 | if [[ ! -f ./crosvm ]]; then 10 | echo "Building crosvm from folder `pwd`" 11 | cd ./crosvm-src 12 | ./tools/dev_container sh -c "cargo build --features gunyah --target aarch64-unknown-linux-gnu --release --no-default-features && cp /scratch/cargo_target/aarch64-unknown-linux-gnu/release/crosvm ." 13 | 14 | echo "Completed building crosvm binary, copying to parent folder" 15 | cp ./crosvm ../ 16 | cd .. 17 | 18 | # Delete sources to save on space 19 | rm -rf ./crosvm-src 20 | else 21 | echo "Crosvm binary already exists, skipping building" 22 | fi 23 | -------------------------------------------------------------------------------- /scripts/utils/build-gunyah.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # © 2022 Qualcomm Innovation Center, Inc. All rights reserved. 4 | # 5 | # SPDX-License-Identifier: BSD-3-Clause 6 | 7 | set -e 8 | 9 | if [[ -z "$LOCAL_SYSROOT" ]]; then 10 | echo "LOCAL_SYSROOT is not found" 11 | return 12 | fi 13 | 14 | if [[ $# -eq 0 ]]; then 15 | PLATFORM=qemu 16 | echo "Building qemu as default platform" 17 | #return 18 | else 19 | PLATFORM=$1 20 | fi 21 | 22 | CLEAN=0 23 | if [ ! -z $2 ]; then 24 | if [[ $2 == "clean" ]]; then 25 | CLEAN=1 26 | else 27 | echo "Second argument should be clean or not provided" 28 | return 29 | fi 30 | fi 31 | 32 | if [[ -f ./${PLATFORM}/hypvm.elf ]]; then 33 | rm -f ./${PLATFORM}/hypvm.elf 34 | echo "Deleted the ${PLATFORM} hypvm.elf file" 35 | fi 36 | 37 | if [[ ${PLATFORM} == "qemu" ]]; then 38 | FEATURE=gunyah-rm-qemu 39 | fi 40 | 41 | QUALITY=debug 42 | 43 | if [[ ${CLEAN} == 1 ]]; then 44 | rm -rf hyp/build/${PLATFORM} 45 | rm -rf resource-manager/build/${PLATFORM} 46 | rm -rf musl-c-runtime/build 47 | rm -rf ${PLATFORM}/* 48 | fi 49 | 50 | cd hyp 51 | echo "./configure.py platform=${PLATFORM} featureset=${FEATURE} quality=${QUALITY} ; ninja" 52 | ./configure.py platform=${PLATFORM} featureset=${FEATURE} quality=${QUALITY} ; ninja 53 | cp build/${PLATFORM}/${FEATURE}/${QUALITY}/hyp.elf build/${PLATFORM}/${FEATURE}/${QUALITY}/hyp.strip.elf 54 | $LLVM/bin/llvm-strip -d build/${PLATFORM}/${FEATURE}/${QUALITY}/hyp.strip.elf 55 | 56 | cd ../resource-manager 57 | echo "./configure.py platform=${PLATFORM} featureset=${FEATURE} quality=${QUALITY} ; ninja" 58 | ./configure.py platform=${PLATFORM} featureset=${FEATURE} quality=${QUALITY} ; ninja 59 | cp build/${PLATFORM}/${QUALITY}/resource-manager build/${PLATFORM}/${QUALITY}/resource-manager.elf 60 | cp build/${PLATFORM}/${QUALITY}/resource-manager.elf build/${PLATFORM}/${QUALITY}/resource-manager.strip.elf 61 | $LLVM/bin/llvm-strip -d build/${PLATFORM}/${QUALITY}/resource-manager.strip.elf 62 | 63 | cd ../musl-c-runtime 64 | echo "./configure.py platform=${PLATFORM} featureset=${FEATURE} quality=${QUALITY} ; ninja" 65 | ./configure.py platform=${PLATFORM} featureset=${FEATURE} quality=${QUALITY} ; ninja 66 | cp build/runtime build/runtime.strip.elf 67 | $LLVM/bin/llvm-strip -d build/runtime.strip.elf 68 | 69 | cd .. 70 | 71 | if [[ ! -d ${PLATFORM} ]]; then 72 | mkdir -p ${PLATFORM} 73 | fi 74 | 75 | python3 hyp/tools/elf/package_apps.py \ 76 | -a resource-manager/build/${PLATFORM}/debug/resource-manager.strip.elf \ 77 | -r musl-c-runtime/build/runtime.strip.elf \ 78 | hyp/build/${PLATFORM}/${FEATURE}/debug/hyp.strip.elf \ 79 | -o ${PLATFORM}/hypvm.elf 80 | 81 | echo "created ${PLATFORM}/hypvm.elf" 82 | 83 | 84 | if [[ -f ${PLATFORM}/hypvm.elf ]]; then 85 | if [[ -z ${QEMU_IMGS_DIR} ]]; then 86 | echo "Environment variable QEMU_IMGS_DIR is not set, to copy image to test destination" 87 | else 88 | cp ${PLATFORM}/hypvm.elf ${QEMU_IMGS_DIR}/hypvm.elf 89 | echo "Copied hypvm.elf to ${QEMU_IMGS_DIR}/hypvm.elf" 90 | fi 91 | fi 92 | 93 | CRT_ENTP=`readelf -e -W musl-c-runtime/build/runtime | grep "Entry point address" | cut -d ':' -f 2 | tr -d "[:space:]"` 94 | echo "C Runtime entry point : ${CRT_ENTP}" 95 | 96 | RM_ENTP=`readelf -e -W resource-manager/build/qemu/debug/resource-manager | grep "Entry point address" | cut -d ':' -f 2 | tr -d "[:space:]"` 97 | echo "Resource Manager entry point : ${RM_ENTP}" 98 | -------------------------------------------------------------------------------- /scripts/utils/build-rootfs-img.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # © 2023 Qualcomm Innovation Center, Inc. All rights reserved. 4 | # 5 | # SPDX-License-Identifier: BSD-3-Clause 6 | 7 | set -e 8 | 9 | # ---------------------------------------------------------------------------- 10 | # This file fetches and builds the rootfs extfs image that can be mounted as 11 | # a virtio disk image to host VM in Qemu. It Re-uses the already built 12 | # binaries, downloads few from internet and builds them. 13 | # 14 | # NOTE: Script uses sudo to mount the fs image to extract the files. 15 | # 16 | # Use built images: 17 | # Image : From linux kernel build in docker container 18 | # initrd.img : RAM Disk image built in docker container 19 | # *.ko : Kernel object modules built in linux kernel in docker container 20 | # 21 | # Download and build: 22 | # crosvm : User space VMM running Host HLOS 23 | # initramfs-*rootfs.ext4.gz : Stock linaro rootfs extfs image used as base 24 | # libgcc_s.so.1 : dependency for crosvm, built using bitbake from Open 25 | # Embedded rpb image. 26 | # This could use some optimization to just build one image 27 | # instead of the whole package 28 | # ---------------------------------------------------------------------------- 29 | 30 | IN_ERROR="NO" 31 | 32 | if [[ -z "${LINUX_DIR}" ]] || [[ ! -f "${LINUX_DIR}/build/arch/arm64/boot/Image" ]]; then 33 | echo "Linux build folder not set or kernel image not found in LINUX_DIR : ${LINUX_DIR}" 34 | IN_ERROR="YES" 35 | fi 36 | 37 | if [[ -z "${RAMDISK_FILE_PATH}" ]] || [[ ! -f "${RAMDISK_FILE_PATH}" ]]; then 38 | echo "Ramdisk file path is not set or not found RAMDISK_FILE_PATH : ${RAMDISK_FILE_PATH}" 39 | IN_ERROR="YES" 40 | fi 41 | 42 | if [[ -z ${CROSVM_FILE_PATH} ]] || [[ ! -f ${CROSVM_FILE_PATH} ]]; then 43 | echo "crosvm file path is not set or not found in CROSVM_FILE_PATH : ${CROSVM_FILE_PATH}" 44 | IN_ERROR="YES" 45 | fi 46 | 47 | if [[ -z ${WORKSPACE} ]]; then 48 | echo "workspace path is not set WORKSPACE : ${WORKSPACE}" 49 | IN_ERROR="YES" 50 | fi 51 | 52 | if [[ "$IN_ERROR" = "YES" ]]; then 53 | echo "Resolve above errors and run the script again" 54 | return 55 | fi 56 | 57 | ROOTFS_BASE="${WORKSPACE}/rootfs" 58 | ROOTFS_REFERENCE_DIR="${ROOTFS_BASE}/reference" 59 | 60 | mkdir -p ${ROOTFS_REFERENCE_DIR} 61 | cd ${ROOTFS_BASE} 62 | 63 | # 64 | # First prepare the folder structure in reference folder, then copy the whole 65 | # file tree into the created and mounted root fs. This provides an opportunity 66 | # to update anything else required in the reference folder 67 | # 68 | 69 | 70 | # ---------------------------------------------------------------------------- 71 | # Linaro stock rootfs image 72 | 73 | echo "Now preparing Linaro stock rootfs image" 74 | 75 | ROOTFS_LINARO_STOCK="${ROOTFS_BASE}/linaro-stock" 76 | LINARO_ROOTFS_IMAGE_FILE_NAME=initramfs-tiny-image-qemuarm64-20230321073831-1379.rootfs.ext4 77 | LINARO_ROOTFS_IMAGE=${LINARO_ROOTFS_IMAGE_FILE_NAME}.gz 78 | LINARO_ROOTFS_URL=https://snapshots.linaro.org/member-builds/qcomlt/testimages/arm64/1379 79 | 80 | if [[ -d ${ROOTFS_REFERENCE_DIR}/etc/systemd/system ]]; then 81 | echo "Reference folder already exists in ${ROOTFS_REFERENCE_DIR}" 82 | else 83 | mkdir -p ${ROOTFS_LINARO_STOCK} 84 | 85 | cd ${ROOTFS_LINARO_STOCK} 86 | 87 | echo "Now downloading Linaro reference rootfs image" 88 | 89 | # Download the rootfs image $LINARO_ROOTFS_IMAGE from linaro website 90 | if [[ ! -f ${ROOTFS_LINARO_STOCK}/${LINARO_ROOTFS_IMAGE_FILE_NAME} ]]; then 91 | wget ${LINARO_ROOTFS_URL}/${LINARO_ROOTFS_IMAGE} 92 | 93 | echo "Download completed, decompressing the image" 94 | 95 | # Decompress the image LINARO_ROOTFS_IMAGE as LINARO_ROOTFS_IMAGE_FILE_NAME 96 | gunzip ${LINARO_ROOTFS_IMAGE} 97 | fi 98 | 99 | # It would be nice if resize works, but newer e2fsck is needed TBD later!! 100 | #resize2fs initramfs-tiny-image-qemuarm64-20230321073831-1379.rootfs.ext4 512M 101 | #e2fsck -f initramfs-tiny-image-qemuarm64-20230321073831-1379.rootfs.ext4 102 | 103 | echo "Decompression completed, mount the image to ${ROOTFS_LINARO_STOCK}/mnt" 104 | 105 | mkdir -p ${ROOTFS_LINARO_STOCK}/mnt 106 | 107 | # mount the Linaro stock rootfs image to extract all the files 108 | sudo mount -o loop ${LINARO_ROOTFS_IMAGE_FILE_NAME} ${ROOTFS_LINARO_STOCK}/mnt 109 | 110 | echo "Copy the file tree to reference tree" 111 | 112 | sudo cp -r -v -p ${ROOTFS_LINARO_STOCK}/mnt/* ${ROOTFS_REFERENCE_DIR} 113 | 114 | sudo umount ${ROOTFS_LINARO_STOCK}/mnt 115 | 116 | # Retain if needed later 117 | #rm -rf ${ROOTFS_LINARO_STOCK} 118 | fi 119 | 120 | # ---------------------------------------------------------------------------- 121 | # Linux kernel built modules 122 | # 123 | # Copy the linux build generated .ko files into the release folder 124 | # ./lib/modules/6.3.0-rc1-00035-g937b9453a2f3-dirty/kernel/ 125 | UTS_RELEASE=`cat ${LINUX_DIR}/build/include/config/kernel.release` 126 | KO_FILES_DST="${ROOTFS_REFERENCE_DIR}/lib/modules/${UTS_RELEASE}/kernel" 127 | 128 | if [[ -d ${KO_FILES_DST} ]]; then 129 | echo "Kernel files already copied to reference destination" 130 | else 131 | echo "Copying the linux ko files to destination dir ${KO_FILES_DST}" 132 | cd ${LINUX_DIR}/build 133 | 134 | for f in $(find . -iname "*.ko"); 135 | do 136 | DST_DIR=$(dirname "${KO_FILES_DST}/$f") 137 | #echo "mkdir ${DST_DIR}" 138 | sudo mkdir -p ${DST_DIR} 139 | sudo cp -v -p $f ${DST_DIR} 140 | done 141 | 142 | echo "Done copying linux kernel object files to reference rootfs tree" 143 | fi 144 | cd ${ROOTFS_BASE} 145 | 146 | # ---------------------------------------------------------------------------- 147 | # crosvm and SVM related files 148 | 149 | # Now copy the crosvm and SVM related files to destination 150 | # these include, crosvm binary, SVM linux kernel image, ramdisk 151 | 152 | SVM_DESTINATION=${ROOTFS_REFERENCE_DIR}/usr/gunyah 153 | 154 | if [[ -f ${SVM_DESTINATION}/$(basename ${CROSVM_FILE_PATH}) ]]; then 155 | echo "Crosvm file is already copied to reference folder" 156 | else 157 | sudo mkdir -p ${SVM_DESTINATION} 158 | if [[ ! -z ${CROSVM_FILE_PATH} ]] && [[ -f ${CROSVM_FILE_PATH} ]]; then 159 | echo "Copying crosvm file to rootfs reference tree" 160 | sudo cp -v -p ${CROSVM_FILE_PATH} ${SVM_DESTINATION} 161 | fi 162 | fi 163 | 164 | if [[ ! -f ${SVM_DESTINATION}/Image ]]; then 165 | sudo cp -v -p ${LINUX_DIR}/build/arch/arm64/boot/Image ${SVM_DESTINATION} 166 | fi 167 | 168 | if [[ ! -f ${SVM_DESTINATION}/$(basename ${RAMDISK_FILE_PATH}) ]]; then 169 | sudo cp -v -p ${RAMDISK_FILE_PATH} ${SVM_DESTINATION} 170 | fi 171 | 172 | if [[ ! -f ${SVM_DESTINATION}/svm.sh ]]; then 173 | echo -e '#!/bin/sh\n\n/usr/gunyah/crosvm --no-syslog run --disable-sandbox \\' > ./svm.sh 174 | echo -e '--serial=type=stdout,hardware=virtio-console,console,stdin,num=1 \\' >> ./svm.sh 175 | echo -e '--serial=type=stdout,hardware=serial,earlycon,num=1 \\' >> ./svm.sh 176 | echo -e '--initrd /usr/gunyah/initrd.img --no-balloon --no-rng \\' >> ./svm.sh 177 | echo -e '--params "rw root=/dev/ram rdinit=/sbin/init earlyprintk=serial panic=0" \\' >> ./svm.sh 178 | echo -e ' /usr/gunyah/Image $@\n' >> ./svm.sh 179 | 180 | sudo cp ./svm.sh ${SVM_DESTINATION} 181 | rm -f ./svm.sh 182 | sudo chmod 0775 ${SVM_DESTINATION}/svm.sh 183 | fi 184 | 185 | echo "Completed copying crosvm and SVM kernel files to rootfs reference tree" 186 | 187 | 188 | # ---------------------------------------------------------------------------- 189 | # Generate and copy libgcc_s.so.1 file 190 | 191 | # crosvm has dependency on libgcc_s.so.1 file. For now a very long approach is 192 | # taken to generate this file, but we can optimize this step later to use the 193 | # required recipe only to generate this binary 194 | 195 | # Following Reference commands are derived from files fetch.log, build.log at 196 | # https://snapshots.linaro.org/member-builds/qcomlt/testimages/arm64/1379/ 197 | 198 | if [[ ! -f ${ROOTFS_REFERENCE_DIR}/lib/libgcc_s.so.1 ]]; then 199 | if [[ ! -f ~/bin/repo ]]; then 200 | echo "Installing repo into local bin folder" 201 | mkdir -p ~/bin 202 | #export PATH=~/bin:$PATH 203 | #echo "$PATH" 204 | curl http://commondatastorage.googleapis.com/git-repo-downloads/repo > ~/bin/repo 205 | chmod a+x ~/bin/repo 206 | fi 207 | 208 | if [[ ! -f ~/.gitconfig ]]; then 209 | echo "Warning: setting the git global config to default values..!!" 210 | git config --global user.name "$USER" 211 | git config --global user.email "$USER@local.com" 212 | git config --global color.ui auto 213 | fi 214 | 215 | ROOTFS_IMAGE_TO_BUILD="rpb-console-image" 216 | export MACHINE=qemuarm64 217 | export DISTRO=rpb 218 | 219 | mkdir ${ROOTFS_BASE}/oe-rpb 220 | cd ${ROOTFS_BASE}/oe-rpb 221 | 222 | # fetch 223 | ~/bin/repo init -u https://github.com/96boards/oe-rpb-manifest.git -b qcom/master 224 | ~/bin/repo sync 225 | 226 | # add config for libgcc and other virtualization options 227 | echo -e "\n" > ./extra_local.conf 228 | echo "INHERIT += 'buildstats buildstats-summary'" >> ./extra_local.conf 229 | echo "PREFERRED_PROVIDER_virtual/kernel = 'linux-dummy'" >> ./extra_local.conf 230 | echo "PREFERRED_PROVIDER_android-tools-conf = 'android-tools-conf-configfs'" >> ./extra_local.conf 231 | echo "CORE_IMAGE_EXTRA_INSTALL += 'openssh libgcc'" >> ./extra_local.conf 232 | echo "PACKAGE_INSTALL:append = ' openssh libgcc'" >> ./extra_local.conf 233 | echo "DISTRO_FEATURES:append = ' virtualization'" >> ./extra_local.conf 234 | echo -e "\n" >> ./extra_local.conf 235 | 236 | echo -e "\n\n" > ./bblayers.conf 237 | 238 | 239 | source setup-environment build 240 | 241 | cat ../extra_local.conf >> conf/local.conf 242 | cat ../bblayers.conf >> conf/bblayers.conf 243 | 244 | echo '"Dumping local.conf.."' 245 | cat conf/local.conf 246 | 247 | bitbake -e > bitbake-environment 248 | 249 | bitbake ${ROOTFS_IMAGE_TO_BUILD} 250 | 251 | # Completed the build. The file libgcc_s.so.1 should be available at path 252 | # ${ROOTFS_BASE}/oe-rpb/build/tmp-rpb-glibc/sysroots-components/cortexa57/libgcc/usr/lib/libgcc_s.so.1 253 | LIBGCC_OUT_PATH="build/tmp-rpb-glibc/sysroots-components/cortexa57/libgcc/usr/lib" 254 | 255 | sudo cp ${ROOTFS_BASE}/oe-rpb/${LIBGCC_OUT_PATH}/libgcc_s.so.1 ${ROOTFS_REFERENCE_DIR}/lib 256 | sudo chmod 0755 ${ROOTFS_REFERENCE_DIR}/lib/libgcc_s.so.1 257 | 258 | rm -rf ${ROOTFS_BASE}/oe-rpb 259 | fi 260 | 261 | echo "Successfully created the reference files folder for rootfs image at : `pwd`" 262 | 263 | # ----------------------------------------------------------------------------- 264 | # Create a extfs device image of required size 265 | 266 | if [[ -f ${WORKSPACE}/rootfs/rootfs-extfs-disk.img ]]; then 267 | echo "Rootfs image already exists, delete this file if need to create" 268 | echo " new file with any modified content from reference folder" 269 | else 270 | echo "Creating rootfs image file from reference : `pwd`" 271 | cd ${WORKSPACE}/rootfs 272 | . ~/utils/bldextfs.sh -f ${WORKSPACE}/rootfs/reference -o ${WORKSPACE}/rootfs/rootfs-extfs-disk.img -s 800M 273 | fi 274 | -------------------------------------------------------------------------------- /scripts/utils/clone-crosvm.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # © 2023 Qualcomm Innovation Center, Inc. All rights reserved. 4 | # 5 | # SPDX-License-Identifier: BSD-3-Clause 6 | 7 | set -e 8 | 9 | if [[ ! -d ./crosvm-src ]]; then 10 | echo "Local sources copy doesn't exist, cloning now" 11 | git clone --recurse-submodules https://chromium.googlesource.com/crosvm/crosvm crosvm-src 12 | else 13 | echo "crosvm sources already exists, skipping cloning" 14 | fi 15 | -------------------------------------------------------------------------------- /scripts/utils/clone-gunyah.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # © 2022 Qualcomm Innovation Center, Inc. All rights reserved. 4 | # 5 | # SPDX-License-Identifier: BSD-3-Clause 6 | 7 | set -e 8 | 9 | if [[ -z "${REPO_URL}" ]]; then 10 | REPO_URL="https://github.com/quic" 11 | fi 12 | 13 | echo -e "\nCloning from repo : ${REPO_URL}\n" 14 | 15 | git clone ${REPO_URL}/gunyah-hypervisor.git hyp 16 | echo "" 17 | 18 | git clone ${REPO_URL}/gunyah-resource-manager.git resource-manager 19 | echo "" 20 | 21 | git clone ${REPO_URL}/gunyah-c-runtime.git musl-c-runtime 22 | echo "" 23 | -------------------------------------------------------------------------------- /scripts/utils/gdb-start.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # © 2022 Qualcomm Innovation Center, Inc. All rights reserved. 4 | # 5 | # SPDX-License-Identifier: BSD-3-Clause 6 | 7 | gdb-multiarch -q \ 8 | -ex 'set arch aarch64' \ 9 | -ex 'set confirm off' \ 10 | -ex 'target remote localhost:1234' \ 11 | -ex 'add-symbol-file hyp/build/qemu/gunyah-rm-qemu/debug/hyp.elf 0xffffffd5ffe00000' \ 12 | -ex 'directory ./hyp' \ 13 | -ex 'break rootvm_init' \ 14 | -ex 'add-symbol-file musl-c-runtime/build/runtime 0x80481000' \ 15 | -ex 'directory ./musl-c-runtime' \ 16 | -ex 'break sys_exit' \ 17 | -ex 'add-symbol-file resource-manager/build/qemu/debug/resource-manager 0x804A0000' \ 18 | -ex 'directory ./resource-manager' \ 19 | -ex 'tui enable' \ 20 | -ex 'layout src' \ 21 | -ex 'layout regs' \ 22 | -ex 'focus cmd' \ 23 | -ex 'p/a &hyp_log_buffer' \ 24 | -ex 'p/a (uint32_t*)((0x8049D000-0x12000+0x1000)+(uint64_t)(&rm_log_area))' \ 25 | -ex 'break hlos_vm_create' \ 26 | -ex 'break vm_dt_create_hlos' \ 27 | -ex 'break memparcel_do_accept' \ 28 | " " 29 | 30 | # -ex 'break soc_qemu_handle_rootvm_init' \ 31 | # -ex 'break hypercall_msgqueue_send' \ 32 | # -ex 'break _start' \ 33 | # -ex 'break hyp_smc_handler' \ 34 | # -ex 'break smccc_handle_vcpu_trap_hvc64' 35 | # -ex 'break smccc_handle_vcpu_trap_smc64' \ 36 | # -ex 'break get_current_core_id' \ 37 | # -ex 'break main' \ 38 | # 39 | 40 | # For symbol loading address, refer to following console logs: 41 | # [HYP] runtime_ipa: 0x80480000 42 | # [HYP] app_ipa: 0x8048c000 43 | # 44 | # For runtime: (Adjust the address above in the cmd line args) 45 | # Symbol load address = runtime_ipa + entry point in runtime elf file 46 | # Sym load Address: 0x80481000 = 0x80480000 + 0x1000 47 | # 48 | # 49 | # For resource-manager: (Adjust the address above in the cmd line args) 50 | # Symbol load address = app_ipa + entry point in resource-manager elf file 51 | # 0x804A0000 = 0x8048c000 + 0x14000 52 | 53 | # 54 | # rm log buffer is located at $2 in the following msg on gdb terminal window using above p/a: 55 | # $1 = 0x20d6c1 56 | # $2 = 0x804e34f8 57 | # Here : p/a (uint32_t*)((0x8049D000-0x12000+0x1000)+(uint64_t)(&rm_log_area))' 58 | # 0x8049D000 : Address where symbol is loaded 59 | # 0x12000 : is Entrypoint address in resource-manager elf file 60 | # 0x1000 : accounts for elf headers overhead 61 | # 62 | # Above results into the location of rm_log_area variable, which holds pointer 63 | # of the memory where RM logs reside, that address can be used to dump logs in 64 | # monitor 65 | # 66 | # Some useful breakpoints to consider: 67 | # Hyp (EL2) : 68 | # rootvm_init : 69 | # vcpu_thread_start : 70 | # platform_cpu_on : 71 | # smccc_handle_call : 72 | # *(vcpu_exception_return+0x58) : Transition to a new VM entry point 73 | # vcpu_exception_dispatch : 74 | # 75 | # Runtime (EL1 Root VM) : 76 | # _start : Entry point of runtime 77 | # *(_start+0xb0) : Transition over to Resource manager (blr x0) 78 | # 79 | # Resource-manager (EL1 Root VM) : 80 | # main : 81 | # hlos_vm_create : 82 | # platform_vm_create : 83 | # vm_memory_init : 84 | # vm_creation_msg_handler : 85 | # 86 | -------------------------------------------------------------------------------- /scripts/utils/kern-test.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # © 2022 Qualcomm Innovation Center, Inc. All rights reserved. 4 | # 5 | # SPDX-License-Identifier: BSD-3-Clause 6 | 7 | set -e 8 | 9 | if [[ -z $KERNEL_IMAGE_DIR ]]; then 10 | KERNEL_IMAGE_DIR="./imgs" 11 | fi 12 | 13 | if [[ -z $INITRD_IMAGE_DIR ]]; then 14 | INITRD_IMAGE_DIR="./imgs" 15 | fi 16 | 17 | if [[ -z $HYP_IMAGE_DIR ]]; then 18 | HYP_IMAGE_DIR="./imgs" 19 | fi 20 | 21 | VIRTIO_BLK_DEVMAP=" " 22 | PLATFORM_DDR_SIZE=2G 23 | CPU_TYPE=max 24 | #CPU_TYPE=cortex-a72 25 | 26 | if [[ ! -z ${VIRTIO_DEVICE_FILE} ]]; then 27 | if [[ -f ${VIRTIO_DEVICE_FILE} ]]; then 28 | VIRTIO_BLK_DEVMAP=" -drive file=${VIRTIO_DEVICE_FILE},if=none,id=vd0,cache=writeback,format=raw -device virtio-blk,drive=vd0 " 29 | else 30 | echo "Virtio disk file is not found, booting without using the disk" 31 | fi 32 | 33 | fi 34 | 35 | echo -e "Starting qemu-system-aarch64... (It may take a minute to see console output logs)\n\n" 36 | 37 | qemu-system-aarch64 -machine virt,virtualization=on,gic-version=3,highmem=off \ 38 | -cpu max -m size=$PLATFORM_DDR_SIZE -smp cpus=8 -nographic \ 39 | -kernel ${KERNEL_IMAGE_DIR}/Image \ 40 | -initrd ${INITRD_IMAGE_DIR}/initrd.img \ 41 | ${VIRTIO_BLK_DEVMAP} \ 42 | -append "rw root=/dev/ram rdinit=/sbin/init earlyprintk=serial,ttyAMA0 console=ttyAMA0 nokaslr" 43 | -------------------------------------------------------------------------------- /scripts/utils/mnt-img.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # © 2023 Qualcomm Innovation Center, Inc. All rights reserved. 4 | # 5 | # SPDX-License-Identifier: BSD-3-Clause 6 | 7 | MOUNT_FOLDER="./tmpfs-mnt" 8 | VALID_ARGS=$(getopt -o f: --long folder: -- "$@") 9 | 10 | if [[ $# -lt 1 ]]; then 11 | echo -e "Usage: $(basename $0) [options] " 12 | echo -e "Options:" 13 | echo -e "\t -f|--folder path : Folder path to mount on" 14 | return 15 | fi 16 | 17 | eval set -- "$VALID_ARGS" 18 | while [ : ]; do 19 | case "$1" in 20 | -f | --folder) 21 | MOUNT_FOLDER=$2 22 | #echo "Input folder: ${IN_FOLDER}" 23 | shift 2 24 | ;; 25 | --) shift; 26 | break 27 | ;; 28 | esac 29 | done 30 | 31 | 32 | mkdir -p $MOUNT_FOLDER 33 | 34 | if [[ $# -eq 0 ]]; then 35 | >&2 echo "Need image file as argument" 36 | return 37 | fi 38 | 39 | if [[ ! -f $1 ]]; then 40 | >&2 echo "Image file $1 not found" 41 | return 42 | fi 43 | 44 | FILE_TYPE=`file $1` 45 | # output of above command should determine how we handle below 46 | #echo $FILE_TYPE 47 | 48 | GZ_COMPRESSED=`echo $FILE_TYPE | grep "gzip compressed"` 49 | 50 | FILE_TO_MOUNT=$1 51 | 52 | #echo $GZ_COMPRESSED 53 | if [[ ! -z $GZ_COMPRESSED ]]; then 54 | echo "gz Compressed image" 55 | ARG_FILE=$1 56 | UNC_OP_FILE=${ARG_FILE%%.*}-uc.${ARG_FILE##*.} 57 | cp $ARG_FILE $UNC_OP_FILE.gz 58 | gunzip -v $UNC_OP_FILE 59 | FILE_TO_MOUNT=$UNC_OP_FILE 60 | FILE_TYPE=`file $FILE_TO_MOUNT` 61 | fi 62 | 63 | EXT_FS_TYPE=`echo $FILE_TYPE | grep "ext4 filesystem"` 64 | CPIO_ARCHIVE_TYPE=`echo $FILE_TYPE | grep "cpio archive"` 65 | 66 | #echo $EXT_FS_TYPE 67 | if [[ ! -z $EXT_FS_TYPE ]]; then 68 | echo "Extfs image" 69 | # Linux rev 1.0 ext4 filesystem data 70 | sudo mount -o loop $FILE_TO_MOUNT $MOUNT_FOLDER 71 | 72 | echo "Mounted the device with extfs to $MOUNT_FOLDER" 73 | echo "exit shell to unmount after examining files " 74 | bash 75 | 76 | sudo umount $MOUNT_FOLDER 77 | rm -rf $MOUNT_FOLDER 78 | fi 79 | 80 | #echo $CPIO_ARCHIVE_TYPE 81 | if [[ ! -z $CPIO_ARCHIVE_TYPE ]]; then 82 | echo "cpio archive image" 83 | # ASCII cpio archive (SVR4 with no CRC) 84 | cd $MOUNT_FOLDER 85 | cat ../$FILE_TO_MOUNT | cpio -idmv 86 | 87 | echo "Extracted cpio files into this folder, delete the folder after done" 88 | echo "rm -rf $MOUNT_FOLDER" 89 | echo "rm -f $FILE_TO_MOUNT" 90 | fi 91 | -------------------------------------------------------------------------------- /scripts/utils/run-qemu.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # © 2022 Qualcomm Innovation Center, Inc. All rights reserved. 4 | # 5 | # SPDX-License-Identifier: BSD-3-Clause 6 | 7 | set -e 8 | 9 | if [[ "$1" == "dbg" ]]; then 10 | ARGS=" -s -S " 11 | #ARGS=" -gdb tcp::1236 -S " 12 | echo "Running with debug stop" 13 | fi 14 | 15 | IMGS_FOLDER="./imgs" 16 | 17 | if [[ -z "${HYP_IMG_FOLDER}" ]]; then 18 | HYP_IMG_FOLDER="${IMGS_FOLDER}" 19 | fi 20 | 21 | VIRTIO_BLK_DEVMAP=" " 22 | 23 | # 24 | # Qemu generates the dtb and copies to the base of DDR @ 0x40000000 25 | # Since the size of in memory dtb is 0x100000 (1MB) move other images 26 | # past that address to avoid overlapping 27 | # 28 | # So current allocation: 29 | # DDR Base : 0x4000_0000 -- 0xC000_0000 -1 (for 2GB config) 30 | # 31 | # Free : 0x4000_0000 -- 0x4010_0000 -1 (1MB) Qemu places dt here for non elf kernels 32 | # Align : 0x4010_0000 -- 0x4020_0000 -1 (1MB) Align hypervisor 33 | # 34 | # Hyp : 0x4020_0000 -- 0x4080_0000 -1 (6MB) 35 | # 36 | # Initrd : 0x4080_0000 -- 0x40D0_0000 -1 (7MB) 37 | # Dtb : 0x40F0_0000 -- 0x4100_0000 -1 (1MB) 38 | # Kernel : 0x4100_0000 -- 0x4400_0000 -1 39 | # 40 | # Linux primary VM will have 1GB of space till 0x8000_0000 41 | # 42 | 43 | INITRD_BASE=0x40800000 44 | DT_BASE=0x40F00000 45 | LINUX_BASE=0x41000000 46 | 47 | HYP_BASE=0x40200000 48 | HYP_SIZE=0x600000 49 | 50 | PLATFORM_DDR_SIZE=2G 51 | LINUX_VM_MEMORY_SIZE=1G 52 | CPU_TYPE=max 53 | #CPU_TYPE=cortex-a72 54 | 55 | if [[ "$1" == "dtb" ]]; then 56 | CMD_LINE="rw root=/dev/ram rdinit=/sbin/init earlyprintk=serial,ttyAMA0 console=ttyAMA0 " 57 | if [[ ! -z "${VIRTIO_DEVICE_FILE}" ]]; then 58 | if [[ -f "${VIRTIO_DEVICE_FILE}" ]]; then 59 | CMD_LINE="root=/dev/vda" 60 | else 61 | echo "Virtio disk file is not found, will boot without the disk" 62 | fi 63 | fi 64 | 65 | if [[ ! -f $IMGS_FOLDER/virt.dtb ]]; then 66 | qemu-system-aarch64 -machine virt,virtualization=on,gic-version=3,highmem=off \ 67 | -cpu $CPU_TYPE -m size=$LINUX_VM_MEMORY_SIZE -smp cpus=8 -nographic \ 68 | -kernel $HYP_IMG_FOLDER/hypvm.elf \ 69 | -device loader,file=$IMGS_FOLDER/Image,addr=$LINUX_BASE \ 70 | -device loader,file=$IMGS_FOLDER/initrd.img,addr=$INITRD_BASE \ 71 | -append "$CMD_LINE " \ 72 | -machine dumpdtb=$IMGS_FOLDER/virt.dtb 73 | fi 74 | 75 | # Give Linux kernel only 1GB of memory 76 | echo "Generating dtb and exiting..." 77 | 78 | qemu-system-aarch64 -machine virt,virtualization=on,gic-version=3,highmem=off \ 79 | -cpu $CPU_TYPE -m size=$LINUX_VM_MEMORY_SIZE -smp cpus=8 -nographic \ 80 | -kernel $HYP_IMG_FOLDER/hypvm.elf \ 81 | -device loader,file=$IMGS_FOLDER/Image,addr=$LINUX_BASE \ 82 | -device loader,file=$IMGS_FOLDER/virt.dtb,addr=$DT_BASE \ 83 | -device loader,file=$IMGS_FOLDER/initrd.img,addr=$INITRD_BASE \ 84 | -append "$CMD_LINE " \ 85 | -machine dumpdtb=$IMGS_FOLDER/virt_qemu.dtb 86 | 87 | #-append "rw root=/dev/ram rdinit=/sbin/init earlyprintk=serial,ttyAMA0 console=ttyAMA0 nokaslr " \ 88 | 89 | export INITRD_SIZE=$(stat -Lc %s $IMGS_FOLDER/initrd.img) 90 | export INITRD_END=$(printf "0x%x" $((${INITRD_BASE} + ${INITRD_SIZE}))) 91 | export HYP_MEM_BASE=$(printf "%x" $((${HYP_BASE}))) 92 | export HYP_MEM_SIZE=$(printf "0x%x" $((${HYP_SIZE}))) 93 | cat < $IMGS_FOLDER/overlay.dts 94 | /dts-v1/; 95 | /{ 96 | fragment@0{ 97 | target-path = "/chosen"; 98 | __overlay__{ 99 | linux,initrd-start = <${INITRD_BASE}>; 100 | linux,initrd-end = <${INITRD_END}>; 101 | }; 102 | }; 103 | 104 | }; 105 | EOF 106 | 107 | dtc -@ -I dts -O dtb $IMGS_FOLDER/overlay.dts -o $IMGS_FOLDER/overlay.dtbo 108 | fdtoverlay -v -i $IMGS_FOLDER/virt_qemu.dtb -o $IMGS_FOLDER/virt.dtb $IMGS_FOLDER/overlay.dtbo 109 | dtc -I dtb -O dts -o $IMGS_FOLDER/virt.dts $IMGS_FOLDER/virt.dtb 110 | 111 | # Remove ITS node and msi-map which doesn't work well 112 | cat $IMGS_FOLDER/virt.dts | sed '/its@/,/};/d' | grep -v "msi-map " > $IMGS_FOLDER/virt-pp.dts && mv $IMGS_FOLDER/virt-pp.dts $IMGS_FOLDER/virt.dts 113 | 114 | dtc -I dts -O dtb -o $IMGS_FOLDER/virt.dtb $IMGS_FOLDER/virt.dts 115 | 116 | echo "Generated dtb $IMGS_FOLDER/virt.dtb" 117 | 118 | else 119 | 120 | if [[ ! -z "${VIRTIO_DEVICE_FILE}" ]]; then 121 | if [[ -f "${VIRTIO_DEVICE_FILE}" ]]; then 122 | # Map the virt device 123 | VIRTIO_BLK_DEVMAP=" -drive file=${VIRTIO_DEVICE_FILE},if=none,id=vd0,cache=writeback,format=raw -device virtio-blk,drive=vd0 " 124 | # VIRTIO_BLK_DEVMAP=" -drive if=virtio,format=raw,file=${VIRTIO_DEVICE_FILE} -device virtio-scsi-pci,id=scsi0 " 125 | else 126 | echo "Virtio disk file is not found, booting without using the disk" 127 | fi 128 | fi 129 | 130 | # Update the image for every run, just in case if we built it as part of development process. 131 | # Set the env variable in rc file to udpate 132 | if [[ ! -z "${HYP_ROOT_PATH}" ]]; then 133 | cp ${HYP_ROOT_PATH}/qemu/hypvm.elf ./imgs/ 134 | fi 135 | 136 | qemu-system-aarch64 -machine virt,virtualization=on,gic-version=3,highmem=off \ 137 | -cpu max,sve128=on -m size=$PLATFORM_DDR_SIZE -smp cpus=8 -nographic \ 138 | -accel tcg,thread=multi \ 139 | -kernel $HYP_IMG_FOLDER/hypvm.elf \ 140 | -device loader,file=$IMGS_FOLDER/virt.dtb,addr=$DT_BASE \ 141 | -device loader,file=$IMGS_FOLDER/Image,addr=$LINUX_BASE \ 142 | -device loader,file=$IMGS_FOLDER/initrd.img,addr=$INITRD_BASE \ 143 | ${VIRTIO_BLK_DEVMAP} \ 144 | $ARGS 145 | 146 | # -monitor telnet::45454,server,nowait \ 147 | # -serial telnet:localhost:1235,server \ 148 | 149 | fi 150 | -------------------------------------------------------------------------------- /scripts/version.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # © 2022 Qualcomm Innovation Center, Inc. All rights reserved. 4 | # 5 | # SPDX-License-Identifier: BSD-3-Clause 6 | 7 | # This file must be sourced from the script where this variable is required 8 | # Keep the version number as x.xx for easy comparison 9 | CURRENT_VER=1.20 10 | -------------------------------------------------------------------------------- /svm_booting_linux.md: -------------------------------------------------------------------------------- 1 | # Demo - Linux Booting in a SVM 2 | 3 | To verify Linux booting in SVM, we need the following components: 4 | 5 | - SVM components (in guest VM) 6 | - Linux kernel 7 | - RAM Disk image 8 | - Device tree 9 | 10 | - HOST HLOS components 11 | - Linux kernel with Gunyah Hypervisor linux drivers 12 | - crosvm user space app as VMM 13 | - rootfs disk image containing all host related tools and SVM image files 14 | 15 | Following sections describe details on the images. These are already generated into mounted volumes as part of the docker image building process. If any of the images need to be rebuilt, the same script ```build-docker-img.sh``` can be used to create the tools, it will only re-create any image that's missing. If the sources need to be retained, then ```rm -rf``` command in the build scripts can be commented off. 16 | 17 | ## SVM components (in guest VM) 18 | ### Linux kernel 19 | The linux kernel image generated during docker build process to test qemu/aarch64 functionality should work for this purpose. 20 | ### RAM Disk image 21 | The RAM Disk image (based on busybox) generated during docker build process to test qemu/aarch64 functionality should work here as well. 22 | ### Device tree 23 | Device tree will be generated by crosvm (VMM) running in Host HLOS (PVM) based on the command line arguments passed while launching it. Resource Manager will patch the DT before its passed on to the SVM Linux kernel. 24 | 25 | ## HOST HLOS components 26 | > Note: Following details are provided only as information if anything need to be changed for experimentation purposes. All these following images have already been generated during the ```build-docker-img.sh``` step. So if you do not wish to change anything, proceed to step [Test SVM loading](#test-svm-loading) 27 | 28 | ### Linux kernel with Gunyah Hypervisor linux drivers 29 | The linux kernel image generated during docker build process already patches the linux kernel sources with Gunyah hypervisor linux drivers. These details can be examined in the provided script ```clone-linux.sh``` which is used to build linux image.enables to test qemu/aarch64 functionality should work for this 30 | 31 | ### crosvm user space app as VMM 32 | A VMM is required to interact with Gunyah hypervisor to create, configure and launch the SVM and execute any OS in it. For the demo, ```crosvm``` user space app is built to handle this task. The details can be found in the included scripts ```clone-crosvm.sh``` and ```build-crosvm.sh```. 33 | 34 | Since crosvm sources and build can be really huge, sources and build output are deleted after copying the output binary image. This output binary is included in the rootfs disk image which is built after. 35 | 36 | 37 | > NOTE: Either a newer version of the Docker or a newer ubuntu distro (22.04 and later) has been encountering the following error..!! Observation was that the build works on host machine but doesn't in docker environment or vice versa. Currently we do not have a workaround for this problem, but if an older version of ubuntu distro (20.04) is available should work fine. 38 | 39 | ``` 40 | Trying to pull gcr.io/crosvm-infra/crosvm_dev:r0040... 41 | Error: parsing image configuration: Get "https://storage.googleapis.com/artifacts.crosvm-infra.appspot.com/containers/images/sha256:6b0c28f5c282c677cc50a927e2227e5d89209ae3c6fa15db10593f7837a644fb": x509: certificate signed by unknown authority 42 | ``` 43 | 44 | 45 | Crosvm generated here will have a dependency on libgcc_s.so.1, which will be generated later during the rootfs disk image generation. 46 | 47 | ### rootfs disk image containing all host related tools and SVM image files 48 | 49 | Since the SVM linux kernel/ramdisk images and other host HLOS components need significant amount of storage space, we cannot just use ramdisk for host HLOS running in PVM. Instead we have to prepare the extfs formatted rootfs disk image that can be mounted as virtio disk. This method will provide the storage space for host HLOS without consuming the valuable RAM space. 50 | 51 | Since generation of rootfs image also consumes huge space to generate, source and build files will be deleted after the binaries are copied to reference folder. If needed to retain source/build files, comment off the ```rm -rf``` command in the build shell scripts. 52 | 53 | At a high level, this process does the following: 54 | - Downloads the stock linaro rootfs extfs disk image (tiny initramfs to keep the size small) 55 | - Downloads the Linaro OpenEmbedded Reference Platform Build (OE RPB) repo and generates the disk image 56 | - This step is taken only for the purpose of getting libgcc_s.so.1 file which is dependency for crosvm 57 | - This step takes significant amount of time 58 | - A potential optimization to avoid this huge build burden is a TODO later, but provides a potential if any useful components from this build can be utilized 59 | - Prepare ```reference``` folder for the filesystem 60 | - Prepare an empty extfs disk image of required size 61 | - Copy all the required files into the rootfs extfs disk image 62 | - File tree from stock Linaro rootfs 63 | - All kernel .ko files from the kernel build 64 | - crosvm (VMM) 65 | - ramdisk, kernel image (for SVM) 66 | 67 | After the rootfs reference folder is created at the path ```~/mnt/workspace/rootfs/reference```, any additional file changes can be performed there. To make a new rootfs image with this modified rootfs reference folder, perform the following operations. 68 | 69 | delete the generated rootfs file 70 | ```bash 71 | rm -f ~/mnt/workspace/rootfs/rootfs-extfs-disk.img 72 | ``` 73 | 74 | and then run the rootfs generating script 75 | 76 | ```bash 77 | cd ~/mnt/workspace/rootfs 78 | build-rootfs-img.sh 79 | ``` 80 | 81 | This will add the environment variable into ```~/mnt/workspace/.wsp-env``` make sure to remove the duplicates and retain only the relevant ones. 82 | 83 | Alternately ```build-docker-img.sh``` can be run on host machine again to generate this image. 84 | 85 | If any modifications to the reference tree is done that exceeds the disk size created, or if more free disk space needed in the PVM, the disk size value in the script ``build-rootfs-img.sh`` can be adjusted to any other value (say 1G) 86 | 87 | Before: 88 | ```bash 89 | . ~/utils/bldextfs.sh -f ${WORKSPACE}/rootfs/reference -o ${WORKSPACE}/rootfs/rootfs-extfs-disk.img -s 800M 90 | ``` 91 | 92 | After: 93 | ```bash 94 | . ~/utils/bldextfs.sh -f ${WORKSPACE}/rootfs/reference -o ${WORKSPACE}/rootfs/rootfs-extfs-disk.img -s 1G 95 | ``` 96 | 97 | ## Test SVM loading 98 | Now all the components needed to test linux running in SVM are ready. Run qemu with the following commands, note that dtb need to be generated again (When any size changes that can potentially move around stuff in the memory map/dt). 99 | 100 | Note that the rootfs disk image is mounted RW, so any changes made will be persistent from HLOS running in PVM. So even the command history or any changes to .rc files would be available. But then it also means if the image gets corrupted it needs to be re-generated as mentioned above from the reference folder. If reference folder is deleted to reclaim space, keep a backup copy of the ```rootfs-extfs-disk.img``` file in order to avoid download/build time and space consumption. 101 | 102 | ```bash 103 | cd ~/mnt/workspace 104 | run-qemu.sh dtb 105 | ``` 106 | After dtb is generated, run the command to launch qemu 107 | 108 | ```bash 109 | run-qemu.sh 110 | ``` 111 | This should result into booting to much complete linux environment in PVM via systemd init. Ignore the errors shown (unless those are fatal errors or crashes), if things went well, then a login prompt and a shell prompt should be seen 112 | 113 | ```bash 114 | . 115 | . 116 | [ OK ] Started User Login Management. 117 | [ OK ] Reached target Multi-User System. 118 | Starting Record Runlevel Change in UTMP... 119 | 120 | Reference-Platform-Build-X11 3.0+linaro qemuarm64 ttyAMA0 121 | 122 | qemuarm64 login: root (automatic login) 123 | . 124 | . 125 | root@qemuarm64:~# 126 | ``` 127 | 128 | Once on the prompt, verify if the hypervisor related nodes showup 129 | 130 | ```bash 131 | cd /sys/firmware/devicetree/base/hypervisor 132 | find . 133 | ``` 134 | 135 | Now launch crosvm using the following command to create a SVM and run Linux OS in it. A shell script has been provided at ```/usr/gunyah/svm.sh``` to make it easier to launch. 136 | 137 | ```bash 138 | /usr/gunyah/crosvm --no-syslog run --disable-sandbox \ 139 | --serial=type=stdout,hardware=virtio-console,console,stdin,num=1 \ 140 | --serial=type=stdout,hardware=serial,earlycon,num=1 \ 141 | --initrd /usr/gunyah/initrd.img --no-balloon --no-rng \ 142 | --params "rw root=/dev/ram rdinit=/sbin/init earlyprintk=serial panic=0" \ 143 | /usr/gunyah/Image 144 | ``` 145 | 146 | OR 147 | 148 | ```bash 149 | /usr/gunyah/svm.sh 150 | ``` 151 | 152 | The following shows the console logs of SVM. This could be pretty slow since each uart IO is going through the crosvm virtio-console (ie the control needs to switch from SVM back to PVM crosvm app to output the char to the actual console). 153 | 154 | ```bash 155 | root@qemuarm64:~# /usr/gunyah/crosvm --no-syslog run --disable-sandbox --serial=type=stdout,hardware=virtio-console,console,stdin,num=1 156 | --serial=type=stdout,hardware=serial,earlycon,num=1 --initrd /usr/gunyah/initrd.img --no-balloon --no-rng --params "rw root=/dev/ram rd 157 | init=/sbin/init earlyprintk=serial panic=0" /usr/gunyah/Image 158 | [2023-09-15T21:44:00.904189296+00:00 INFO crosvm] crosvm started. 159 | [2023-09-15T21:44:00.935025408+00:00 INFO crosvm] CLI arguments parsed. 160 | [HYP] Emulated RAZ for ID register: ISS 0x36002f 161 | [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0x000f0480] 162 | [ 0.000000] Linux version 6.5.0-00025-gd9af2a6a5a22 (yvasi@buildkitsandbox) (clang version 15.0.7 (https://github.com/llvm/llvm-project.git 8dfdcc7b7bf66834a761bd8de445840ef68e4d1a), LLD 15.0.7) #1 SMP PREEMPT Fri Sep 15 10:13:26 PDT 2023 163 | [ 0.000000] KASLR enabled 164 | [ 0.000000] random: crng init done 165 | [ 0.000000] Machine model: linux,dummy-virt 166 | [ 0.000000] earlycon: uart8250 at MMIO 0x00000000000003f8 (options '') 167 | [ 0.000000] printk: bootconsole [uart8250] enabled 168 | [ 0.000000] efi: UEFI not found. 169 | [ 0.000000] NUMA: No NUMA configuration found 170 | [ 0.000000] NUMA: Faking a node at [mem 0x0000000080000000-0x000000008fffffff] 171 | [ 0.000000] NUMA: NODE_DATA [mem 0x8ff709c0-0x8ff72fff] 172 | [ 0.000000] Zone ranges: 173 | 174 | ``` 175 | This script ```svm.sh``` can take additional arguments and pass on to ```crosvm``` app. As an example, we can provide the number of cpu's as additional argument to create SVM with more CPU cores. Note that adding more cores to SVM linux session might make console logs even slower..!! 176 | 177 | ```bash 178 | /usr/gunyah/svm.sh --cpus 4 179 | ``` 180 | 181 | This should show that the Linux kernel is running in the SVM. Once verification of this linux environment is complete, issue ```poweroff``` command in the SVM linux shell to go back into the PVM linux environment. SVM can be launched again to verify a different configuration. 182 | 183 | ```bash 184 | / # poweroff 185 | / # umount: can't unmount /: Invalid argument 186 | swapoff: can't open '/etc/fstab': No such file or directory 187 | The system is going down NOW! 188 | Sent SIGTERM to all processes 189 | Sent SIGKILL to all processes 190 | Requesting system poweroff 191 | [ 7204.608721] reboot: Power down 192 | ``` 193 | > Note: Current latset version of Gunyah freezes on second time launch after SVM power off. This is working fine in the original version of gunyah before the updates, SVM relaunch feature can be verified on that original version of Gunyah/RM sources. 194 | 195 | Press ```ctrl-a and x``` to exit Qemu 196 | 197 | Follow-up TODO's: 198 | 199 | The following areas require work, any help on these is appreciated: 200 | - Enable ```ssh_server``` in PVM Linux OS image, so that we can connect multiple ssh sessions and test multiple SVM sessions simultaneously. 201 | - Fix poor performance of the SVM Linux console, for example using the Gunayh RM Console driver. 202 | --------------------------------------------------------------------------------