├── Poster-OpenBSD-RISCV.pdf ├── env ├── 00-Create OpenBSD instance on GCP.md ├── 06-Install FreeBSD on GCP.md ├── 02-Setup RISCV-GNU tool chain on Linux | Openbsd.md ├── 01-Install QEMU on Linux | FreeBSD | OpenBSD.md ├── 08-GDB FreeBSD on qemu-system-riscv64 from Ubuntu.md ├── 09-Build LLVM for RISCV on OpenBSD.md ├── 03-Run busybear-linux on qemu-system-riscv64.md ├── 05-GDB openbsd kernel from Ubuntu.md ├── 07-Build FreeBSD for RISCV.md └── 04-GDB busybear-linux on qemu-system-riscv64.md ├── Porting_OpenBSD_to_RISCV_FinalReport.pdf ├── .gitmodules └── README.md /Poster-OpenBSD-RISCV.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MengshiLi/openbsd-riscv-notes/HEAD/Poster-OpenBSD-RISCV.pdf -------------------------------------------------------------------------------- /env/00-Create OpenBSD instance on GCP.md: -------------------------------------------------------------------------------- 1 | check submodule `openbsd-amd64-create-img` for more information. 2 | -------------------------------------------------------------------------------- /Porting_OpenBSD_to_RISCV_FinalReport.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MengshiLi/openbsd-riscv-notes/HEAD/Porting_OpenBSD_to_RISCV_FinalReport.pdf -------------------------------------------------------------------------------- /.gitmodules: -------------------------------------------------------------------------------- 1 | [submodule "openbsd-amd64-create-img"] 2 | path = openbsd-amd64-create-img 3 | url = git@github.com:MengshiLi/openbsd-amd64-create-img.git 4 | [submodule "busybear-linux"] 5 | path = busybear-linux 6 | url = git@github.com:MengshiLi/busybear-linux.git 7 | [submodule "riscv-pk"] 8 | path = riscv-pk 9 | url = git@github.com:MengshiLi/riscv-pk.git 10 | -------------------------------------------------------------------------------- /env/06-Install FreeBSD on GCP.md: -------------------------------------------------------------------------------- 1 | ### How to list freeBSD images: 2 | ``` 3 | gcloud compute images list \ 4 | --project freebsd-org-cloud-dev \ 5 | --no-standard-images | grep -i freebsd-12 6 | ``` 7 | 8 | ### How to instantiate a freeBSD OS on google cloud platform: 9 | From gdk console: 10 | ``` 11 | gcloud compute instances create "freebsd12-2" \ 12 | --zone "us-west1-b" \ 13 | --machine-type "n1-standard-8" \ 14 | --network "default" \ 15 | --maintenance-policy "MIGRATE" \ 16 | --image "freebsd-12-1-release-amd64" --image-project=freebsd-org-cloud-dev \ 17 | --boot-disk-size "200" 18 | ``` 19 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # openbsd-riscv-notes 2 | This repo provides a series of environment setup notes while we are porting openbsd to riscv. 3 | 4 | Table of Contents for env/ folder: 5 | - 00-Create OpenBSD instance on GCP.md 6 | - 01-Install QEMU on Linux | FreeBSD | OpenBSD.md 7 | - 02-Setup RISCV-GNU tool chain on Linux | Openbsd.md 8 | - 03-Run busybear-linux on qemu-system-riscv64.md 9 | - 04-GDB busybear-linux on qemu-system-riscv64.md 10 | - 05-GDB openbsd kernel from Ubuntu.md 11 | - 06-Install FreeBSD on GCP.md 12 | - 07-Build FreeBSD for RISCV.md 13 | - 08-GDB FreeBSD on qemu-system-riscv64 from Ubuntu.md 14 | - 09-Build LLVM for RISCV on OpenBSD.md 15 | -------------------------------------------------------------------------------- /env/02-Setup RISCV-GNU tool chain on Linux | Openbsd.md: -------------------------------------------------------------------------------- 1 | ## Install GNU Tool Chain on Linux (Ubuntu18) 2 | [reference](https://github.com/riscv/riscv-gnu-toolchain) 3 | 4 | ### Get repo: 5 | ``` 6 | git clone --recursive https://github.com/riscv/riscv-gnu-toolchain 7 | ``` 8 | ### Prerequisites: 9 | ``` 10 | sudo apt-get install autoconf automake autotools-dev curl libmpc-dev libmpfr-dev libgmp-dev gawk build-essential bison flex texinfo gperf libtool patchutils bc zlib1g-dev libexpat-dev 11 | ``` 12 | 13 | ### Installation 14 | ``` 15 | cd riscv-gnu-toolchain 16 | ./configure --prefix=/usr/local/riscv 17 | sudo make linux -j8 18 | sudo make install 19 | export PATH=$PATH:/usr/local/riscv/bin 20 | export RISCV=/usr/local/riscv 21 | ``` 22 | 23 | ## Install GDB alone on Linux (Ubuntu18) 24 | 25 | ### Repo 26 | ``` 27 | git clone https://sourceware.org/git/binutils-gdb.git 28 | ``` 29 | 30 | ### Installation 31 | ``` 32 | git checkout 6b9374f1e07cb250736815ff8db263199416adc6 33 | mkdir build && cd build 34 | ../configure --target=riscv64-unknown-elf --prefix=/usr/local/riscv 35 | sudo make -j4 36 | sudo make install -j4 37 | export PATH=$PATH:/usr/local/riscv/bin 38 | export RISCV=/usr/local/riscv 39 | ``` 40 | 41 | ## Install GNU Tool Chain on Openbsd 42 | - `pkg_add riscv-elf-binutils riscv-elf-gcc riscv-elf-newlib` 43 | -------------------------------------------------------------------------------- /env/01-Install QEMU on Linux | FreeBSD | OpenBSD.md: -------------------------------------------------------------------------------- 1 | Suggest install from source, so that we can get latest bits for risc-v support. 2 | 3 | ### Install on Linux: 4 | - Install build dependencies: 5 | ``` 6 | sudo apt update 7 | sudo apt install build-essential flex bison glib2.0 libpixman-1-dev 8 | ``` 9 | 10 | - note 1: specify target to install on demand 11 | ``` 12 | git clone https://git.qemu.org/git/qemu.git 13 | cd qemu 14 | git submodule init 15 | git submodule update --recursive 16 | ./configure --target-list=riscv64-softmmu 17 | make -j8 18 | sudo make install 19 | ``` 20 | 21 | ### Install on FreeBSD: 22 | ``` 23 | sudo pkg install glib pixman 24 | 25 | export PREFIX="/usr/local/riscv" 26 | 27 | git clone https://github.com/qemu/qemu 28 | cd qemu 29 | mkdir build && cd build 30 | ../configure --target-list=riscv64-softmmu --prefix=$PREFIX 31 | sudo gmake -j8 32 | sudo gmake install 33 | 34 | export PATH="$PATH:$PREFIX/bin" 35 | ``` 36 | 37 | ### Install on OpenBSD: 38 | - prerequisite 39 | - `pkg_add -r python` 40 | - `pkg_add glib2` 41 | - `pkg_add gmake` 42 | - `pkg_add bison` 43 | 44 | ``` 45 | export PREFIX="/usr/local/riscv" 46 | 47 | git clone https://github.com/qemu/qemu 48 | cd qemu 49 | git submodule init 50 | git submodule update --recursive 51 | mkdir build && cd build 52 | ../configure --target-list=riscv64-softmmu --prefix=$PREFIX --enable-debug 53 | doas gmake -j8 54 | doas gmake install 55 | export PATH="$PATH:$PREFIX/bin" 56 | ``` 57 | -------------------------------------------------------------------------------- /env/08-GDB FreeBSD on qemu-system-riscv64 from Ubuntu.md: -------------------------------------------------------------------------------- 1 | ### 1. Install GDB on FreeBSD (Failed) 2 | Therefore we GDB on Ubuntu18 instead. 3 | 4 | ### 2. Start Freebsd kernel in QEMU 5 | - install qemu first, refer [here](https://github.com/MengshiLi/openbsd-riscv-notes/blob/master/env/01-Install%20QEMU%20on%20Linux%20%7C%20FreeBSD%20%7C%20OpenBSD.md) 6 | 7 | - run freebsd (w/o network device) 8 | ``` 9 | sudo qemu-system-riscv64 -s -S\ 10 | -machine virt \ 11 | -m 2048M \ 12 | -kernel /home/mengsli/freebsd-img/bbl \ 13 | -nographic \ 14 | -drive file=/home/mengsli/freebsd-img/riscv.img,format=raw,id=hd0 \ 15 | -device virtio-blk-device,drive=hd0 16 | ``` 17 | 18 | ### 3. Debug from Ubuntu18 19 | - copy `kernel.debug` to Ubuntu18 so that GDB can refer to the symbols. 20 | - use `gcloud compute scp` 21 | 22 | - the kernel.debug is referencing freebsd source code, which is located at `/usr/home/mengsli/freebsd-riscv` on GCP machine-`freebsd12`. Therefore we need to clone same repop on ubuntu, and need to put it at the same absolute path as `freebsd12`. 23 | - `sudo mkdir /usr/home/mengsli/` 24 | - `cd /usr/home/mengsli/` 25 | - `sudo git clone https://github.com/freebsd/freebsd freebsd-riscv` 26 | 27 | - assuming riscv-xtools have been installed on ubuntu18, it will provide `riscv64-unknown-linux-gnu-gdb` 28 | - enter the folder of `kernel.debug` 29 | - `riscv64-unknown-linux-gnu-gdb kernel.debug` 30 | - inside gdb: 31 | - `target remote 10.138.0.3:1234` 32 | - `break _start` 33 | 34 | -------------------------------------------------------------------------------- /env/09-Build LLVM for RISCV on OpenBSD.md: -------------------------------------------------------------------------------- 1 | ### Prerequisites 2 | - OpenBSD 6.6 3 | - git: `pkg_add git` 4 | - cmake: `pkg_add cmake` 5 | - ninja: `pkg_add ninja` 6 | - python3: `pkt_add python` 7 | 8 | ### Clone the LLVM Project repo from github 9 | - `git clone https://github.com/llvm/llvm-project.git` 10 | - switch to a stable version 11 | - `git checkout origin/release/10.x` 12 | 13 | ### Build the LLVM Cross-compiler Toolchain 14 | - starting from the root of the cloned llvm-project repo 15 | ``` 16 | mkdir build 17 | cd build 18 | cmake \ 19 | -DLLVM_ENABLE_FFI:Bool=False \ 20 | -DLLVM_ENABLE_TERMINFO:Bool=False \ 21 | -DLLVM_ENABLE_RTTI:Bool=False \ 22 | -DLLVM_BUILD_LLVM_DYLIB:Bool=True \ 23 | -DLLVM_LINK_LLVM_DYLIB:Bool=True \ 24 | -DLLVM_INSTALL_TOOLCHAIN_ONLY:Bool=True \ 25 | -DLLVM_TARGETS_TO_BUILD="X86" \ 26 | -DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD="RISCV" \ 27 | -DLLVM_ENABLE_PROJECTS="clang" \ 28 | -DCMAKE_BUILD_TYPE=Release \ 29 | -DCMAKE_INSTALL_PREFIX=/usr/local/riscv \ 30 | -DCMAKE_CXX_FLAGS="-fno-ret-protector -mno-retpoline" \ 31 | -G Ninja \ 32 | ../llvm 33 | 34 | ninja -j8 35 | ninja install 36 | ``` 37 | 38 | ### Some comments 39 | - install prefix is `/usr/local/riscv`, which is not part of the default library search paths, therefore need to `export LD_LIBRARY_PATH=/usr/local/riscv/lib`. 40 | - it dlopen()s things that are not in a normal library path like `libLLVM-9svn.so'` 41 | - could put that in some sort of setup script, e.g. `~/.profile`, that you run before doing riscv stuff 42 | - But had we installed tools into prefix `/usr` then it may have worked without `LD_LIBRARY_PATH` 43 | - but that could clobber the system compiler 44 | - In this way, when we need to rev the cross compiler, we can all just dump /usr/local/riscv and un-tar a new version, no fuss. 45 | -------------------------------------------------------------------------------- /env/03-Run busybear-linux on qemu-system-riscv64.md: -------------------------------------------------------------------------------- 1 | ### [main ref](https://github.com/michaeljclark/busybear-linux) 2 | 3 | ### [supplementary ref](https://www.cnx-software.com/2018/03/16/how-to-run-linux-on-risc-v-with-qemu-emulator/) 4 | 5 | ### prerequisites: 6 | - riscv-gnu tool chain 7 | - qemu 8 | 9 | ### build busybear-linux 10 | ``` 11 | git clone --recursive https://github.com/michaeljclark/busybear-linux.git 12 | cd busybear-linux 13 | make 14 | ``` 15 | - might need to download riscv-pk and move it to `busybear-linux/src` manually. `git clone https://github.com/riscv/riscv-pk.git` 16 | 17 | - then will automatically build: 18 | - busybox 19 | - dropbear 20 | - linux 21 | - bbl 22 | 23 | - if build bbl fails due to `stubs-lp64.h`, check this patch to the Makefile.in.[patch](https://github.com/riscv/riscv-pk/pull/114/commits/00f0dd04cbdb670f7e81d7fe5c686cb49e7cd182) 24 | 25 | 26 | ### Or if you want to build them manually: 27 | - build riscv-linux 28 | ``` 29 | git clone https://github.com/riscv/riscv-linux.git 30 | cd riscv-linux 31 | git checkout riscv-linux-4.14 32 | cp ../busybear-linux/conf/linux.config .config 33 | make ARCH=riscv CROSS_COMPILE=riscv64-unknown-linux-gnu- olddefconfig 34 | make -j4 ARCH=riscv CROSS_COMPILE=riscv64-unknown-linux-gnu- vmlinux 35 | ``` 36 | 37 | - build bbl 38 | ``` 39 | git clone https://github.com/riscv/riscv-pk.git 40 | cd riscv-pk 41 | mkdir build && cd build 42 | ../configure \ 43 | --enable-logo \ 44 | --host=riscv64-unknown-linux-gnu \ 45 | --with-payload=../../riscv-linux/vmlinux 46 | make 47 | ``` 48 | 49 | Note: the automatically build bbl is located at: `busybear-linux/build/riscv-pk/bbl` 50 | 51 | ### run it on qemu 52 | ``` 53 | sudo qemu-system-riscv64 -s -S\ 54 | -nographic -machine virt \ 55 | -kernel build/riscv-pk/bbl \ 56 | -append "root=/dev/vda ro console=ttyS0" \ 57 | -drive file=busybear.bin,format=raw,id=hd0 \ 58 | -device virtio-blk-device,drive=hd0 \ 59 | -netdev type=tap,script=scripts/ifup.sh,downscript=scripts/ifdown.sh,id=net0 \ 60 | -device virtio-net-device,netdev=net0 61 | ``` 62 | - on openbsd 63 | ``` 64 | doas qemu-system-riscv64 -s \ 65 | -nographic -machine virt \ 66 | -kernel bbl \ 67 | -append "root=/dev/vda ro console=ttyS0" \ 68 | -drive file=busybear.bin,format=raw,id=hd0 \ 69 | -device virtio-blk-device,drive=hd0 70 | ``` 71 | - after around 1 second, booted up! 72 | - root:busybear 73 | - some basic information: 74 | ``` 75 | root@busybear:/# uname -a 76 | Linux busybear 5.0.0 #1 SMP Mon Sep 2 03:26:15 UTC 2019 riscv64 GNU/Linux 77 | root@busybear:/# cat /proc/interrupts 78 | CPU0 79 | 7: 82 SiFive PLIC 7 virtio1 80 | 8: 192 SiFive PLIC 8 virtio0 81 | 10: 515 SiFive PLIC 10 ttyS0 82 | IPI0: 0 Rescheduling interrupts 83 | IPI1: 0 Function call interrupts 84 | IPI2: 0 CPU stop interrupts 85 | root@busybear:/# cat /proc/cpuinfo 86 | processor : 0 87 | hart : 0 88 | isa : rv64imafdcu 89 | mmu : sv48 90 | ``` 91 | - don't forget to `halt` to quit linux and qemu. 92 | -------------------------------------------------------------------------------- /env/05-GDB openbsd kernel from Ubuntu.md: -------------------------------------------------------------------------------- 1 | ### 1. Environment is setup on Ubuntu: 2 | - `uname -a` output: 3 | Linux ubuntu-1804 5.0.0-1029-gcp #30~18.04.1-Ubuntu SMP Mon Jan 13 05:40:56 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux 4 | - `sudo apt update` 5 | - `sudo apt install build-essential` 6 | 7 | 8 | ### 2. Install QEMU on Linux to emulate risc-v machine: 9 | - potential prerequisites: 10 | - `sudo apt-get install glib2.0` 11 | - `sudo apt-get install libpixman-1-dev` 12 | 13 | - note: specify target-list to install riscv only 14 | ``` 15 | git clone https://git.qemu.org/git/qemu.git 16 | cd qemu 17 | git submodule init 18 | git submodule update --recursive 19 | ./configure --target-list=riscv64-softmmu,riscv32-softmmu 20 | make -j8 21 | sudo make install 22 | ``` 23 | 24 | 25 | ### 3. Install RISC-V cross-compiler tool chain: 26 | - `git clone --recursive https://github.com/riscv/riscv-gnu-toolchain` 27 | - prerequisites for ubuntu: 28 | - `sudo apt-get install autoconf automake autotools-dev curl python3 libmpc-dev libmpfr-dev libgmp-dev gawk build-essential bison flex texinfo gperf libtool patchutils bc zlib1g-dev libexpat-dev` 29 | 30 | - the following will install gcc, binutilities, gdb, etc. 31 | ``` 32 | export RISCV="/usr/local/riscv" 33 | ./configure --prefix=$RISCV 34 | make -j4 linux 35 | export PATH=$PATH:$RISCV/bin 36 | ``` 37 | 38 | 39 | ### 4. Get the OpenBSD kernel 40 | - two files: 41 | - `bsd`: kernel without debug information 42 | - `bsd.gdb`: kernel with debug stubs. 43 | - `bsd` is obtained by `llvm-strip bsd.gdb` 44 | 45 | - copy these two files to Ubuntu18 so that GDB can refer to the symbols. 46 | - `scp root@10.138.0.2:/home/mars/riscv/src/openbsd/sys/arch/riscv64/compile/GENERIC/obj/bsd.gdb .` 47 | - if failed to scp from freebsd12 to ubuntu18 directly, can use gcloud compute scp instead: on local laptop (as a relay): 48 | - `gcloud compute scp mars@openbsd65-rv64:/home/mars/riscv/img/bsd.gdb .` 49 | - `gcloud compute scp ./bsd.gdb mars@ubuntu18:/home/mars/riscv/gdb4openbsd/` 50 | 51 | - the bsd.gdb is referencing openbsd source code, prepare the source code and put it the same absolute path as it is on openbsd machine. 52 | - `mkdir /home/mars/riscv/src/` 53 | - `git clone git@github.com:MengshiLi/openbsd.git` 54 | 55 | 56 | ### 5. Build bbl with bsd as payload 57 | ``` 58 | rm -rf build && mkdir build && cd build 59 | ../configure --host=riscv64-unknown-linux-gnu --with-payload=/home/mars/riscv/openbsdGDB/bsd 60 | 61 | make 62 | ``` 63 | - the binary bbl is located at: `build/riscv-pk/bbl`, copy it to `/home/mars/riscv/openbsdGDB/` for easy manipulation 64 | 65 | - if build bbl fails due to `stubs-lp64.h`, check this patch and apply it to Makefile.in.[patch](https://github.com/riscv/riscv-pk/pull/114/commits/00f0dd04cbdb670f7e81d7fe5c686cb49e7cd182) 66 | 67 | - if normal user has no privilege, try `sudo su` first. 68 | 69 | 70 | ### 5b. Build bbl on OpenBSD (not recommended due to CC conflicts) 71 | - install riscv gnu tool chain on OpenBSD: `pkg_add riscv-elf-binutils riscv-elf-gcc riscv-elf-newlib` 72 | 73 | - build bbl: first ensure $CC is not set 74 | ``` 75 | git clone https://github.com/riscv/riscv-pk.git 76 | cd riscv-pk 77 | mkdir build && cd build 78 | ../configure --host=riscv64-unknown-elf --with-payload=../../../img/bsd 79 | gmake 80 | ``` 81 | 82 | 83 | ### 6. Run openbsd bbl in QEMU 84 | `qemu-system-riscv64 -s -S -nographic -machine virt -kernel /home/mars/riscv/riscv/openbsdGDB/bbl` 85 | 86 | 87 | ### 7. Start gdb 88 | - launch: 89 | - `cd /home/mars/riscv/openbsdGDB/` 90 | - `riscv64-unknown-linux-gnu-gdb bsd.gdb`, or after enter gdb, `file /path/to/bsd.gdb` 91 | 92 | - inside (gdb): 93 | ``` 94 | set architecture riscv:rv64 95 | target remote localhost:1234 96 | set riscv use-compressed-breakpoints no 97 | break *0x80200000 //_start_kern_bootstrap 98 | continue 99 | ``` 100 | 101 | ### Reference: 102 | - [How to debug](http://docs.keystone-enclave.org/en/latest/Getting-Started/How-to-Debug.html) 103 | - [sbi to linux](https://github.com/slavaim/riscv-notes/blob/master/bbl/sbi-to-linux.md) 104 | 105 | 106 | -------------------------------------------------------------------------------- /env/07-Build FreeBSD for RISCV.md: -------------------------------------------------------------------------------- 1 | #### Before reading: 2 | This manual shows the build process using devel toolchain. 3 | 4 | #### [main ref](https://wiki.freebsd.org/riscv) 5 | 6 | ## Install FreeBSD on GCP 7 | - see [6-Install FreeBSD on GCP](https://github.com/MengshiLi/openbsd-riscv-notes/blob/master/env/06-Install%20FreeBSD%20on%20GCP.md), note freeBSD version is 12.0 8 | 9 | ## Prerequisites after get a fresh freebsd OS. 10 | - `sudo pkg install vim git` 11 | 12 | ## Install GNU Cross Compiler Tool Chain for RISCV 13 | 14 | #### 2. Install prerequired packages 15 | - `sudo pkg install bison gmp mpfr mpc gawk gsed pkgconf texinfo gmake` 16 | 17 | #### 3. Install from port directly. [Preferred] 18 | - [riscv64-xtoolchain-gcc](https://www.freshports.org/devel/riscv64-xtoolchain-gcc) 19 | - `sudo pkg install riscv64-xtoolchain-gcc` 20 | - after which, we will get: 21 | ``` 22 | New packages to be INSTALLED: 23 | riscv64-xtoolchain-gcc: 0.4_1 24 | riscv64-gcc: 8.3.0 25 | riscv64-binutils: 2.33.1,1 26 | ``` 27 | - These package are installed into `/usr/local/bin`, with prefix `riscv64-unknown-freebsd12.0-` 28 | - the binutils is at: `/usr/local/riscv64-unknown-freebsd12.0` 29 | - They do include the same bunch of x-tools as we built above manually. 30 | 31 | 32 | ## Build FreeBSD Bundle 33 | 34 | #### 4. Get _FreeBSD Sources_ 35 | - `git clone https://github.com/freebsd/freebsd freebsd-riscv` 36 | - `cd freebsd-riscv` 37 | 38 | #### 5. Build FreeBSD World (w/o Kernel) using devel/riscv64-xtoolchain-gcc(Succeed) 39 | ``` 40 | export MAKEOBJDIRPREFIX=/home/$USER/obj/ 41 | export WITHOUT_FORMAT_EXTENSIONS=yes 42 | make CROSS_TOOLCHAIN=riscv64-gcc TARGET_ARCH=riscv64 buildworld -j8 43 | ``` 44 | - approximately takes following time on 8 skylake core: `World built in 1618 seconds, ncpu: 8, make -j8` 45 | 46 | #### 6. Build FreeBSD kernel using devel/riscv64-xtoolchain-gcc: 47 | - `make TARGET_ARCH=riscv64 buildkernel CROSS_TOOLCHAIN=riscv64-gcc KERNCONF=QEMU` 48 | - the resultant kernel can be found at `/usr/home/$USER/obj/usr/home/$USER/freebsd-riscv/riscv.riscv64/sys/QEMU/`: 49 | - kernel 50 | - kernel.debug 51 | - kernel.full 52 | 53 | 54 | #### 7. build _BBL_ to boot FreeBSD 55 | ``` 56 | git clone https://github.com/freebsd-riscv/riscv-pk 57 | cd riscv-pk 58 | mkdir build && cd build 59 | export PATH=${PATH}:${PREFIX}/bin 60 | export CPP=cpp 61 | export CFLAGS="-nostdlib" 62 | export WITH_ARCH=rv64g 63 | ../configure --enable-logo --prefix=$PREFIX --host=riscv64-unknown-freebsd12.0 --with-payload= 64 | gmake bbl 65 | unset CFLAGS 66 | unset CPP 67 | ``` 68 | 69 | #### 8. Install FreeBSD world and kernel 70 | ``` 71 | export DESTDIR=/home/$USER/riscv-world 72 | export CROSS_BINUTILS_PREFIX=/usr/local/riscv64-unknown-freebsd12.0/bin/ 73 | make TARGET_ARCH=riscv64 -DNO_ROOT DESTDIR=$DESTDIR installworld 74 | make TARGET_ARCH=riscv64 -DNO_ROOT DESTDIR=$DESTDIR distribution 75 | make TARGET_ARCH=riscv64 INSTALLKERNEL=QEMU -DNO_ROOT DESTDIR=$DESTDIR installkernel 76 | ``` 77 | 78 | #### 9. Build complete rootfs image to run in QEMU 79 | ``` 80 | cd $DESTDIR 81 | echo 'hostname="qemu"' > etc/rc.conf 82 | echo "/dev/vtbd0 / ufs ro 1 1" > etc/fstab 83 | echo "./etc/fstab type=file uname=root gname=wheel mode=0644" >> METALOG 84 | echo "./etc/rc.conf type=file uname=root gname=wheel mode=0644" >> METALOG 85 | makefs -D -f 5000000 -o version=2 -s 20g riscv.img METALOG 86 | ``` 87 | 88 | #### 10. optional, setup the network 89 | - as root: 90 | - [ref0:](http://bsdwiki.reedmedia.net/wiki/networking_qemu_virtual_bsd_systems.html) 91 | - [ref1:](https://www.freshports.org/emulators/qemu) 92 | - Needs to set net.link.tap.user_open sysctl in order to use /dev/tap* 93 | networking as non-root. Don't forget to adjust device node permissions in 94 | /etc/devfs.rules. 95 | - [ref2:](https://romain.blogreen.org/blog/2007/09/setting-up-qemu-for-networking-under-freebsd/) 96 | - [ref3:](https://wiki.freebsd.org/qemu) 97 | 98 | ``` 99 | setenv iface vtnet0 # SET iface TO YOUR MAIN INTERFACE, setenv is for csh. 100 | echo "Make configuration persistent through reboots:" 101 | echo net.link.tap.user_open=1 >> /etc/sysctl.conf 102 | echo net.link.tap.up_on_open=1 >> /etc/sysctl.conf 103 | echo chmod 0660 /dev/tap0 >> /etc/rc.local # this should be done with devfs(8) 104 | echo 'cloned_interfaces="tap0 bridge0"' >> /etc/rc.conf 105 | echo 'ifconfig_bridge0="addm '$iface' addm tap0 up"' >> /etc/rc.conf 106 | 107 | echo 'Actually configure everything (unless you want to reboot at this point):' 108 | /etc/rc.d/sysctl start 109 | ifconfig bridge0 create 110 | ifconfig tap0 create 111 | ifconfig bridge0 addm $iface addm tap0 up 112 | chmod 0660 /dev/tap0 113 | ``` 114 | 115 | #### 11. Folder Explains: 116 | - freebsd-riscv: freebsd source code which support riscv 117 | - qemu: qemu source code 118 | - obj: output position of build freebsd world and freebsd kernel 119 | - riscv-world: freebsd world with kernel, ready to make a complete rootfs. 120 | -------------------------------------------------------------------------------- /env/04-GDB busybear-linux on qemu-system-riscv64.md: -------------------------------------------------------------------------------- 1 | #### 1. Environment is setup on Ubuntu: 2 | - `uname -a` output: 3 | Linux ubuntu-1804 5.0.0-1029-gcp #30~18.04.1-Ubuntu SMP Mon Jan 13 05:40:56 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux 4 | - `sudo apt update` 5 | - `sudo apt install build-essential` 6 | 7 | 8 | #### 2. Install QEMU on Linux to support emulating risc-v: 9 | - potential prerequisites: 10 | - `sudo apt-get install glib2.0` 11 | - `sudo apt-get install libpixman-1-dev` 12 | 13 | - note: specify target-list to install riscv only 14 | ``` 15 | git clone https://git.qemu.org/git/qemu.git 16 | cd qemu 17 | git submodule init 18 | git submodule update --recursive 19 | ./configure --target-list=riscv64-softmmu,riscv32-softmmu 20 | make -j8 21 | sudo make install 22 | ``` 23 | 24 | 25 | #### 3. Install RISC-V cross-compiler tool chain: 26 | - `git clone --recursive https://github.com/riscv/riscv-gnu-toolchain` 27 | - prerequisites for ubuntu: `sudo apt-get install autoconf automake autotools-dev curl python3 libmpc-dev libmpfr-dev libgmp-dev gawk build-essential bison flex texinfo gperf libtool patchutils bc zlib1g-dev libexpat-dev` 28 | - create isolated folder as the install path: `/usr/local/riscv` 29 | - `./configure --prefix=/usr/local/riscv` 30 | - `make -j4 linux` 31 | - add it to PATH `export PATH=$PATH:/usr/local/riscv/bin`. 32 | 33 | 34 | #### 4. OR substitutely, just install GDB instead of the whole tool chain: 35 | - download the riscv GDB repo: `git clone https://github.com/riscv/riscv-binutils-gdb.git` 36 | - `cd riscv-binutils-gdb` 37 | - `git checkout fsf-gdb-8.3-with-sim` 38 | - `./configure --target=riscv64-unknown-linux-gnu` 39 | - `make` 40 | - `sudo make install` 41 | - optionally copy gdb to `/usr/local/riscv/bin` 42 | - full name is: `riscv64-unknown-linux-gnu-gdb` 43 | - verified above steps also succeed on macos 44 | 45 | 46 | #### 5. build busybear linux with debugging stub installed: 47 | - prerequisites: 48 | - riscv-gnu tool chain (as in step 4) 49 | - qemu (as in step 2) 50 | - get busybear-linux: 51 | - `git clone --recursive https://github.com/michaeljclark/busybear-linux.git` 52 | - `cd busybear-linux` 53 | - if submodule recursive download fails, download riscv-pk `git clone https://github.com/riscv/riscv-pk.git` and move it to `busybear-linux/src` manually. 54 | 55 | - add a config patch to `busybear-linux/config/`, these lines install the debug stub: 56 | ``` 57 | cat <.config-fragment 58 | CONFIG_DEBUG_INFO=y 59 | CONFIG_DEBUG_KERNEL=y 60 | CONFIG_GDB_SCRIPTS=y 61 | EOF 62 | ``` 63 | 64 | - edit `busybear-linux/scripts/build.sh`: 65 | - find line 56: `cp conf/linux.config build/linux-${LINUX_KERNEL_VERSION}/.config` 66 | - add one line below: `cp conf/.config-fragment build/linux-${LINUX_KERNEL_VERSION}/.config-fragment` 67 | - find line 73: `cd build/linux-${LINUX_KERNEL_VERSION}`; 68 | - add one line below to patch the config file: `./scripts/kconfig/merge_config.sh .config .config-fragment` 69 | 70 | - run the script, `./script/build.sh`, which will automatically build the following targets sequentially: 71 | - busybox 72 | - dropbear 73 | - linux 74 | - bbl 75 | 76 | - if build bbl fails due to `stubs-lp64.h`, check this patch and apply it to Makefile.in.[patch](https://github.com/riscv/riscv-pk/pull/114/commits/00f0dd04cbdb670f7e81d7fe5c686cb49e7cd182) 77 | 78 | - if normal user has no privilege, try `sudo su` to enter a new root-privileged shell first. 79 | 80 | #### 6. Run busybear linux in QEMU 81 | ``` 82 | sudo qemu-system-riscv64 -s -S\ 83 | -nographic -machine virt \ 84 | -kernel build/riscv-pk/bbl \ 85 | -append "root=/dev/vda ro console=ttyS0" \ 86 | -drive file=busybear.bin,format=raw,id=hd0 \ 87 | -device virtio-blk-device,drive=hd0 88 | 89 | //// skip the following two lines so far 90 | -netdev type=tap,script=scripts/ifup.sh,downscript=scripts/ifdown.sh,id=net0 \ 91 | -device virtio-net-device,netdev=net0 92 | ``` 93 | - qemu `-gdb {device}`), remote debug stub 94 | - allows you to specify QEMU to wait for a connection in the specified device. 95 | - It can accept serial, socket, udp, tcp, stdio, etc. 96 | - E.g. `-gdb tcp::9000` to listen on port 9000, then from _GDB_ you can connect to it with `target remote localhost:9000`. 97 | - The `-s` switch is a shorthand for `-gdb tcp::1234`. 98 | - qemu `-S` (stop machine at start up): 99 | - This gives time for the debugger to connect and allows to start debugging from the very beginning, even the early platform firmware. 100 | - To start execution, you must send QEMU the "continue" command, either via the debugger or the monitor console. 101 | - ?? no symbols in gdb. 102 | - if gdb is too old, report register missing error. 103 | - egdb. 104 | 105 | #### 7. Start debugging with GDB 106 | - launch: 107 | - `cd build/linux-5.0/` 108 | - `riscv64-unknown-linux-gnu-gdb vmlinux`, or after enter gdb, `file /path/to/vmlinuz` 109 | - 110 | - inside gdb: 111 | - `set architecture riscv:rv64` 112 | - `target remote localhost:1234` 113 | - `break start_kernel` 114 | - `continue` 115 | 116 | 117 | #### 8. have fun 118 | - [ref:](https://stackoverflow.com/questions/11408041/how-to-debug-the-linux-kernel-with-gdb-and-qemu/33203642#33203642) 119 | --------------------------------------------------------------------------------