├── .github └── ISSUE_TEMPLATE │ ├── bug_report.md │ ├── dead-link.md │ ├── documentation-enhancement.md │ ├── factual-error.md │ └── feature_request.md ├── LICENSE ├── demo ├── demo_qemu.tar └── readme.md ├── readme.md ├── thanks.md └── wiki ├── 1_intro ├── 1_prereq.md ├── 2_component.md ├── 3_bootProcess.md └── 4_envSetup.md ├── 2_crossToolchain ├── 1_need.md ├── 2_packages.md └── 3_build.md ├── 3_miniSystem ├── 1_dirAndFiles.md ├── 2_install.md ├── 3_miniKernel.md ├── 4_boot.md └── readme.md ├── 4_beyond ├── 1_kernel │ └── 1_qemuHardware.md ├── readme.md └── unusedDocs │ ├── 1_dir.md │ ├── 2_install.md │ └── 3_miniKernel.md └── readme.md /.github/ISSUE_TEMPLATE/bug_report.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Bug report 3 | about: Create a report about a bug 4 | title: '' 5 | labels: bug 6 | assignees: '' 7 | 8 | --- 9 | 10 | **Describe the bug** 11 | What the bug is, what should the expected behavior be, etc. 12 | 13 | **Steps to reproduce the behavior** (if applicable) 14 | 15 | **Additional context** (if applicable) 16 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/dead-link.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Dead Link 3 | about: Report a dead link in the Microdot documentation 4 | title: '' 5 | labels: dead link 6 | assignees: '' 7 | 8 | --- 9 | 10 | Where is the dead link located? (please provide file name / URL) 11 | 12 | Please also paste a short quote of the text (so I can quickly locate it). 13 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/documentation-enhancement.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Documentation Enhancement 3 | about: Suggest possible actions to improve the usefulness / clarity of the documentation 4 | title: '' 5 | labels: enhancement 6 | assignees: '' 7 | 8 | --- 9 | 10 | Please describe what specifically are you trying to do. Here are some possibilities: 11 | * rewrite existing content to make it clearer and easier to understand 12 | * add more information to supplement the docs 13 | * remove / replace some information that is redundant or misleading 14 | * etc... 15 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/factual-error.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Factual Error 3 | about: Report factual errors in the Microdot documentation 4 | title: "[Factual Error]" 5 | labels: '' 6 | assignees: '' 7 | 8 | --- 9 | 10 | What is the exact problem? (incorrect/missing instructions, wrong explanation about a concept, etc.) 11 | 12 | Where is the error located? (please provide file name / URL link) 13 | 14 | Please also paste a short quote of the erroneous content (so I can locate it quickly) 15 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/feature_request.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Feature request 3 | about: Suggest an idea for this project 4 | title: '' 5 | labels: '' 6 | assignees: '' 7 | 8 | --- 9 | 10 | **Is your feature request related to a problem? Please describe.** 11 | A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] 12 | 13 | **Describe the solution you'd like** 14 | A clear and concise description of what you want to happen. 15 | 16 | **Describe alternatives you've considered** 17 | A clear and concise description of any alternative solutions or features you've considered. 18 | 19 | **Additional context** 20 | Add any other context or screenshots about the feature request here. 21 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2019 Yusen Hu 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /demo/demo_qemu.tar: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Unturned3/Microdot/b54f99863273c2927eb44f2c357cc68e634585be/demo/demo_qemu.tar -------------------------------------------------------------------------------- /demo/readme.md: -------------------------------------------------------------------------------- 1 | 2 | # Microdot Linux 3 | 4 | * demo\_qemu.tar: Microdot running inside QEMU. Unpack the tarball and follow the 5 | instructions inside. 6 | 7 | * vbox\_demo.img: Microdot (w/ bootloader) for Oracle VirtualBox (not implemented yet) 8 | 9 | -------------------------------------------------------------------------------- /readme.md: -------------------------------------------------------------------------------- 1 | > Coming soon: deploying Microdot on Allwinner F1C100s-based systems! 2 | 3 | # Microdot Linux 4 | 5 | [![asciicast](https://asciinema.org/a/UXVIlhIViY1qOOzCBW5Aayuqa.svg)](https://asciinema.org/a/UXVIlhIViY1qOOzCBW5Aayuqa) 6 | 7 | The Microdot Linux project will guide you through the process of 8 | creating a miniature but fully functional Linux system from the 9 | ground up, with special emphasis on toolchain creation and kernel 10 | configuration. The minimal kernel occupies 672KB of space, while the root file 11 | system (busybox, with a proper init system) occupies 432KB. 12 | Under the Quick Emulator 13 | (QEMU) with a single CPU core, the system boots in 0.6 seconds, with only 14 | 32MB of RAM. You can tune the kernel & the 15 | userspace to support more hardware and extend the functionality of the base 16 | system. 17 | 18 | > You can download the demo shown above [here](/demo). 19 | 20 | # Overview 21 | 22 | - Section 1: prerequisites, background information, and build environment setup 23 | - Section 2: constructs a musl-based cross compilation toolchain 24 | - Section 3: builds a minimum Linux system composed of a tiny kernel and an 25 | initramfs archive. 26 | - Section 4: tutorials for more specific topics, such as kernel fine tuning, 27 | porting Microdot to other architectures, installing other programs, etc. 28 | This section will probably contain a lot of work contributed by other users. 29 | For more details, see the `readme.md` file inside `wiki/4_beyond`. 30 | 31 | > Note: some of the sections can be skipped. If such conditions apply, it will 32 | > be made clear in the documentation. For example, you can choose to skip the 33 | > section about manually building a cross toolchain and use a prebuilt one 34 | > instead. 35 | 36 | 37 | # Getting Started 38 | 39 | The Microdot [wiki](/wiki) is where all the tutorials and documentation are 40 | stored. Check out the Wiki to start building your own Microdot Linux! 41 | You can also visit the FAQ section to learn more about the Microdot project. 42 | 43 | 44 | # FAQ 45 | 46 | 47 | ### Why did you build the Microdot project? 48 | 49 | I started building Microdot purely out of curiosity (and building a Linux 50 | system from scratch does seem like a popular pastime for people). However, 51 | along the way, I encountered many problems, such as: 52 | 53 | * How can I replace glibc with the much smaller musl-libc? 54 | * How do I properly make a cross compilation toolchain using musl-libc? 55 | * How do I configure the Linux kernel and make it tiny? 56 | * What components are needed in a minimal root file system? 57 | 58 | There is suprisingly little documentation or tutorials on the internet 59 | about these topics. For any information that I did manage to find, they 60 | hardly agreed with each other and were either incorrect or inconsistent. 61 | And some of them just expect you to blindly follow insturctions 62 | and type commands without explaining why. 63 | 64 | "cross toolchain building" and "kernel configuration" were the nastiest 65 | out of all the problems I faced. There are literally thousands of 66 | options that you can use to configure the packages, and 67 | misusing one could cause the toolchain to fail or render the kernel 68 | unbootable. I partially used trail-and-error to figure things out (believe 69 | me, applying trail-and-error that on a process that takes hours to finish 70 | every time is not a fun task to do), but luckily I got some help from 71 | [other people](/thanks.md), which spared me from wasting more time. 72 | 73 | I learned a lot in the end, and I thought it would be good for me to 74 | share my experience, so other people wouldn't have to go through the same 75 | process and waste an awful lot of time. 76 | 77 | ### What's the difference between the Microdot and other projects? 78 | 79 | Here are some projects that has some similar aspects to Microdot: 80 | 81 | * Tiny Core Linux 82 | 83 | Does an exellent job making a complete & tiny Linux distro, 84 | but isn't primarily focused on teaching people _how_ to make one. 85 | 86 | * Linux From Scratch 87 | 88 | Teaches people how to manually assemble a working Linux system, but 89 | uses _a lot_ of packages, employs a complicated process, and makes a 90 | big system (200+ MB). Doesn't offer much details on how to configure 91 | a kernel. 92 | 93 | * Minimal Linux Live 94 | 95 | Has a concise and fully automated build process, explains a lot 96 | about the boot process and system internals, and the resulting system 97 | is quite small (around 8MB). However, it doesn't use a customized 98 | toolchain (makes the end system unnecessarily big), and it doesn't 99 | explain how to configure a kernel. 100 | 101 | ### What does "Microdot" mean? 102 | 103 | Microdots are a steganographic technique that involves 104 | photographically shrinking a large chunk of text to the 105 | size of a typographical dot, such as a period. This term 106 | is used here to figuratively show the fact that Microdot 107 | Linux compresses a lot of functionality into a tiny 108 | amount of space. 109 | 110 | 111 | ### Who is the intended audience of the Microdot Project? 112 | 113 | The Microdot Project is primarily intended for people who 114 | is interested in constructing a custom Linux system (it seems 115 | like that this is quite a popular activity for Linux users). 116 | However, besides that, the Microdot Project also provides 117 | useful information on topics such as cross compilation, kernel 118 | configuration, userspace building, etc. which could be useful 119 | outside the scope of Microdot. 120 | 121 | 122 | ### What can Microdot Linux be used for? 123 | 124 | Microdot Linux is made to be extendable and you can tailor it to your 125 | needs. Because it comes with no bloat and has a small size, it can 126 | be used as an embedded Linux system, where resources 127 | are constrained. For example, you can replace Rasbian with 128 | Microdot on a Raspberry Pi in order to save more memory 129 | and processing power for your applications. Tutorials on deploying Microdot 130 | onto actual hardware will be added soon. 131 | 132 | 133 | 134 | -------------------------------------------------------------------------------- /thanks.md: -------------------------------------------------------------------------------- 1 | 2 | 3 | Many people helped me along my way, and without them Microdot probably would 4 | never see the light of day. Here's a short list of people who I want to thank: 5 | 6 | * [Laurent Bercot](http://skarnet.org): recommended me to the #musl channel 7 | 8 | * richfelker: author of [musl-libc](http://www.musl-libc.org) and admin of the 9 | #musl irc channel 10 | 11 | * [hazel](https://www.linuxquestions.org/questions/user/hazel-1029471/), and 12 | many other members from [linuxquestions.org](https://www.linuxquestions.org) 13 | , who answered a lot of my questions 14 | 15 | * people from the 16 | [busybox mailing list](http://lists.busybox.net/pipermail/busybox) 17 | 18 | -------------------------------------------------------------------------------- /wiki/1_intro/1_prereq.md: -------------------------------------------------------------------------------- 1 | 2 | # Skill Requirements 3 | 4 | Building a Linux system from the ground up is not an easy task, and 5 | it requires familiarity with the Linux command line. As a minimum, 6 | you should be able to do basic operations such as: 7 | 8 | * copying/moving/renaming files 9 | * changing/listing directories 10 | * printing file contents 11 | * deleting files 12 | * creating files/directories 13 | * using variables 14 | * editing files using a CLI text editor 15 | 16 | A lot of jargon is used throughout the documentation. If there are terms or 17 | concepts that you don't know about, please consult Google and make sure 18 | that you understand them before moving on, or else you might get stuck later 19 | because you missed something. 20 | 21 | Many resources on the internet has already explained certain important 22 | concepts that we will be working with; So instead of reinventing the wheel 23 | and rewriting what others have wrote, I will put links to them in the 24 | documentation. 25 | 26 | If you feel like that something isn't explained clearly enough in the docs, 27 | feel free to 28 | [open up a new Issue on GitHub](github.com/Unturned3/Microdot/issues) 29 | and tell me about it there. 30 | 31 | Useful websites to learn about the Linux command line: 32 | * [linuxcommand.org](http://linuxcommand.org) 33 | * [ryanstutorials.net/linuxtutorial](https://ryanstutorials.net/linuxtutorial) 34 | * [linux-tutorial.info](http://linux-tutorial.info) 35 | 36 | # Host System Requirements 37 | 38 | You need to have a working Linux environment (Ideally a mainstream 39 | distribution such as Debian or Arch) on your machine. It can either be 40 | installed directly onto the computer's hard disk or inside a virtual 41 | machine. VMs are comparatively easier to set up, but the downside is 42 | that the guest operating system won't have access to all the CPU power 43 | on the computer, so it might take longer when compiling software. If you 44 | do not have a working Linux system yet, just visit Google and you can 45 | find plenty of good tutorials out there about how to set up a VM. 46 | 47 | I used a Debian 9 system installed in a VM to build Microdot Linux, because 48 | I didn't bother to do a hard drive install and mess with the Apple boot 49 | loader. The VM has 16GB of free disk space, 2GB of RAM, and 2 CPU cores. 50 | 51 | Here's a list of required packages for building Microdot Linux. You can 52 | use Google to find distribution-specific instructions for installing 53 | these packages onto your host Linux system. 54 | 55 | * gcc (The GNU compiler collection) 56 | * wget (a simple tool to download files) 57 | * make (a build tool that simplifies the compilation process) 58 | * bc (needed for building the Linux kernel) 59 | * qemu (Quick Emulator, used to run and test Microdot Linux) 60 | * ncurses (a text-based menu interface for certain configuration scripts) 61 | * gawk (a text-processing language) 62 | * flex (for building the kernel) 63 | * bison (for building the kernel) 64 | * rsync (for building the kernel) 65 | 66 | > Note: the list above might not be exhaustive. If you run into errors in the form 67 | > of "command not found" or similar, you might be missing a package. These errors 68 | > are usually easily fixed by "apt install ..." (or whatever package manager you're using). 69 | > Feel free to report these issues so I can update the docs accordinly. 70 | 71 | 72 | 73 | 74 | -------------------------------------------------------------------------------- /wiki/1_intro/2_component.md: -------------------------------------------------------------------------------- 1 | 2 | # Components of a Minimal Linux System 3 | 4 | Any sensible computer system (that a user can use) probably includes the 5 | following components: 6 | 7 | * Bootloader 8 | 9 | When the computer powers on, the operating system can't load 10 | itself into RAM and start to execute. A small program (the 11 | bootloader) will copy the operating system from a storage media 12 | (hard drive, CD-ROM, etc.) into RAM and pass the execution control 13 | to it. 14 | 15 | * Kernel 16 | 17 | The kernel is the core of an operating system, as it interacts 18 | the platform's hardware, manages the computer's resources 19 | (such as how much RAM should be given to an application), and 20 | provides an almost platform-independent environment for userspace 21 | applications to run in. 22 | 23 | * Userspace Applications 24 | 25 | Although vital, the kernel can't actually do anything that's useful 26 | for the user. The functionality of the operating system depends 27 | on the userspace applications that runs on top of the kernel. For 28 | example, one might have programs that initiates a network connection, 29 | or make backups to a hard drive, etc. 30 | 31 | * User Interface 32 | 33 | An operating system can't be used if the user can't interact with 34 | it (obviously). The UI of the system can be graphical, or command 35 | line, or just a few buttons (for embedded systems, like a coffee 36 | machine). 37 | 38 | 39 | We will hand-pick the vital components that we will include in Microdot 40 | Linux and tailor it to our needs. 41 | For the bootloader, we will use `syslinux`, as it is easy to configure 42 | and has a tiny footprint. For the userspace, we will use the "Swiss Army 43 | Knife of Embedded Linux", aka. `busybox`. It combines roughly 300 common 44 | Linux utilities into 1 program, and it only uses around 900KB of disk 45 | space. For the user interface, we will use the `ash` shell provided in 46 | `busybox`. Of course, for the kernel, we will use the Linux kernel (or 47 | else this project would be named something else, like Microdot Windows) 48 | 49 | In the next section, you will learn how these components work together 50 | to bring up a usable system when the computer powers on. 51 | 52 | -------------------------------------------------------------------------------- /wiki/1_intro/3_bootProcess.md: -------------------------------------------------------------------------------- 1 | # Linux Boot Process Overview 2 | 3 | Understanding how does a Linux system start and what components are 4 | loaded at which stage is vital for ensuring the proper functionality of 5 | our own mini system. Generally, for a standard `x86_64` machine, this is 6 | what happens during power on: 7 | 8 | 1. The BIOS (basic input-output system) gets loaded into RAM from the 9 | motherboard, looks for the 1st bootable device it finds (hard drive, 10 | CD-ROM), then it loads the bootloader installed on that device. In our 11 | case, it would be `syslinux`. 12 | 13 | 2. The bootloader then loads the kernel into memory and then passes the 14 | control over to the running kernel. 15 | 16 | 3. The kernel mounts the root file system, then executes a userspace program 17 | to initialize the system, which is usually `/sbin/init`. In our case, the 18 | init program is provided by `busybox`. 19 | 20 | 4. `/sbin/init` is the first userspace process that the kernel launches and 21 | its process ID is always 1. This process cannot exit during the runtime 22 | of the system or else you will end up with a kernel panic (when the kernel 23 | runs into an fatal error). All other userspace processes are descendants 24 | of init, and all orphaned processes are automatically adopted by init. 25 | Commonly, init reads its configuration file, `/etc/inittab`, and carries 26 | out tasks specified by the file, such as opening a text terminal, mounting 27 | other disks, starting a network connection, etc. 28 | 29 | 5. The user can now interact with the system. 30 | 31 | On a conventional PC, this bootup process typically takes around 20 seconds 32 | to a minute. On the other hand, Microdot Linux can boot up within a 33 | second, because its components are highly optimized for both speed and 34 | size. Since the system only has a handful of programs, it uses suprisingly 35 | little memory, and can boot with as little as 32MB of RAM. 36 | -------------------------------------------------------------------------------- /wiki/1_intro/4_envSetup.md: -------------------------------------------------------------------------------- 1 | # Setting up the build environment 2 | 3 | In this section, we will prepare our host system for building Microdot 4 | Linux. 5 | First, we need to create a few directories in `/opt` to store the all 6 | the files needed during the build. The `targetfs` directory stands for 7 | "target file system", and it will contain files that forms the bootable 8 | disk image. The `cross` directory contains our cross compilation toolchain 9 | that we will build in the next section. The `cross/src` directory contains 10 | all the source packages that we will be compiling. 11 | 12 | > Note: execute the following commands as the root user, since you (as a 13 | > normal user) wouldn't have write access to the `/opt` directory. 14 | 15 | ```bash 16 | mkdir /opt/targetfs 17 | mkdir /opt/cross 18 | mkdir /opt/cross/src 19 | ``` 20 | 21 | 22 | # Create the Builder User 23 | 24 | Instead of using your own user account to build Microdot, we will create 25 | a dedicated user named `builder` to do this. First, your default user 26 | account will include a lot of environmental variables, and some of which 27 | can interfere with our build process. Adding a new user account enables 28 | us to eliminate dangerous/unused environmental variables. Also, you 29 | probably don't want your default user's home directory to be cluttered 30 | with a pile of temporary build files. 31 | 32 | First, we add a group named `builder`, then we use `useradd` to actually 33 | create the new user. The `-s` option specifies the login shell that 34 | `builder` will be using. The `-g` option specifies the group that `builder` 35 | belongs to. The `-m` option tells `useradd` to create a home directory for 36 | the new user. The `-k` option specifies the "skeletal files" that will 37 | be copied to the new user's home directory. Here, we tell it to copy 38 | nothing by specifying `/dev/null`, otherwise it might pollute the new home 39 | directory. Lastly, we specify `builder` as the name of the new user. 40 | 41 | The `passwd` command sets a password for the new user. 42 | > Note: you cannot see the password as you type it in. Your keyboard is 43 | > working fine! 44 | 45 | Finally, the `su - builder` command allows us to change our user to 46 | `builder` without logging out and logging in again, which is quite 47 | convenient. 48 | > Note: the dash in `su - builder` is important! It will spawn a login 49 | > shell for the new user instead of just simply substituting our identity. 50 | > This will reset all the old environmental variables and provide us with 51 | > a clean environment. If you forget the dash then the point of creating 52 | > a new user is defeated. 53 | 54 | ```bash 55 | groupadd builder 56 | useradd -s /bin/bash -g builder -m -k /dev/null builder 57 | passwd builder 58 | su - builder 59 | ``` 60 | 61 | 62 | # Setup the Initialization Scripts for Builder 63 | 64 | Every time a user logs in, a set of scripts are automatically executed 65 | to initialize the users command line environment (set up variables, etc). 66 | By default, a _login shell_ is spawned when a user logs in, and this shell 67 | is initialized by the `.bash_profile` script. Notice the leading dot in the 68 | file name: it makes the file invisible if you run `ls` in a directory. This 69 | is commonly done for configuration files that the user doesn't want to see 70 | all the time. You can force `ls` to show invisible files by invoking it as 71 | `ls -a`. 72 | 73 | The commands placed in `.bash_profile` replaces the current _login shell_ 74 | with a new shell and with an empty environment except for the `HOME`, 75 | `TERM`, and `PS1` variables. This ensures that no dangerous environmental 76 | variables can leak in to this new shell. 77 | 78 | > Note: we uses the `cat` command to write text to a file instead of using 79 | > a text editor here. The syntax below tells `cat` to read input from STDIN 80 | > until reading the characters `EOF`, and redirect all the entered text 81 | > into `~/.bash_profile`. 82 | 83 | ```bash 84 | cat > ~/.bash_profile << "EOF" 85 | exec env -i HOME=$HOME TERM=$TERM PS1='\u:\w\$ ' /bin/bash 86 | EOF 87 | ``` 88 | 89 | The newly spawned shell by the `.bash_profile` script is a non-login shell, 90 | and it uses `.bashrc` to initialize it self. The `set +h` option turns off 91 | the hashing functionality of the bash shell. This prevents bash from 92 | remembering the paths to previously executed commands. We need this because 93 | we will first be using the host system's programs, then we will use the 94 | programs we built ourselves during the construction stage. If bash 95 | remembers the old paths, then the new programs will be ignored. The `umask 96 | 022` command sets the user file creation mask, and this ensures that all 97 | files we make can only be write to by ourselves, but can be read by others. 98 | 99 | > Note: a hashtag character ("#") means that the rest of the line is a 100 | > comment. It is used to insert useful information into the code snippits. 101 | > Do not type the comment as a part of the command, as it will be ignored 102 | > by the shell anyways. 103 | 104 | ```bash 105 | cat > ~/.bashrc << "EOF" 106 | 107 | set +h 108 | umask 022 109 | 110 | host=x86_64-cross-linux-gnu 111 | target=x86_64-linux-musl 112 | arch=x86 113 | 114 | export host target arch 115 | unset CFLAGS 116 | 117 | install=/opt/cross 118 | sysroot=/opt/cross/$target 119 | targetfs=/opt/targetfs 120 | LC_ALL=POSIX # turns off localization 121 | PATH=$install/bin:/bin:/usr/bin 122 | 123 | export install sysroot targetfs LC_ALL PATH 124 | 125 | EOF # tell cat that this is the end of input 126 | ``` 127 | 128 | The `host` and `target` variables contains two system triplets. Basically, 129 | a system triplet specifies the information about a platform. Its format is 130 | generally like this: `architecture-kernel-userspace`. Here, the architecture 131 | for both the host and the target is `x86_64`. For the host triplet, we 132 | added an extra word `cross` in there, to indicate that it is the platform 133 | on which we are building the cross compiler toolchain. The kernel is the 134 | same for both the host and the target, which is `linux`. Finally, the hosts 135 | userspace applications are all using the GNU C library, which is why we put 136 | `gnu` there. However, for the target, we are using a much smaller C library 137 | called musl-libc, so we put `musl` there instead. 138 | 139 | The `arch` variable contains the target hardware architecture. Here, `x86` 140 | refers to a generic 64-bit PC architecture. 141 | 142 | The `install` variable holds the location in which we would like all of our 143 | compiled programs to be installed in. This prevents the program we build 144 | from getting mixed into the hosts programs. 145 | 146 | The `sysroot` variable contains the location of the "root file system" of 147 | the target system. Of course, it is an incomplete and imagined root file 148 | system. It contains the target architecture specific stuff, such as the 149 | C library, etc. Our cross compilers will be installed in `$install` instead 150 | of `$sysroot` because it is meant to run on our host, and not intended 151 | for the target. 152 | 153 | The `targetfs` variable points to the directory containing the files that 154 | will form the bootable disk image that we will make later on. This 155 | directory is not used until we get to the stage where we assemble the final 156 | system. 157 | 158 | The `PATH` variable holds the location that `bash` will check to find 159 | programs. For example, if the program `ls` or `cat` is not located in 160 | your `PATH`, then you wouldn't be able to execute them. Notice how we 161 | put `$install/bin` before `/bin` and `/usr/bin`. We want the bash shell 162 | to use our new tools immediately when we install them and ignore the 163 | tools that came with the host system. 164 | 165 | 166 | Finally, we let the current shell re-read the `~/.bash_profile` script, 167 | so all the stuff that we just did can be applied. 168 | 169 | ```bash 170 | source ~/.bash_profile 171 | exit 172 | ``` 173 | 174 | # Change permission and ownership 175 | 176 | Now, we will make `builder` the owner of the directories that we just made. 177 | 178 | > Note: run the following commands as the root user 179 | ```bash 180 | chown builder:builder -R /opt/cross 181 | chown builder:builder -R /opt/targetfs 182 | chmod 755 -R /opt/cross 183 | ``` 184 | -------------------------------------------------------------------------------- /wiki/2_crossToolchain/1_need.md: -------------------------------------------------------------------------------- 1 | # The Need for Cross Compiling 2 | 3 | What does "_compile_" mean? Why do we need to "_cross compile_"? 4 | 5 | Excellent articles written by other people that explains these concepts: 6 | 7 | * [Introduction to cross-compiling for Linux](http://landley.net/writing/docs/cross-compiling.html) 8 | * [GCC Cross-Compiler](https://wiki.osdev.org/GCC_Cross-Compiler) 9 | 10 | Of course, you can always ask Google for more related to these topics. 11 | The more you know, the easier this will be! 12 | 13 | -------------------------------------------------------------------------------- /wiki/2_crossToolchain/2_packages.md: -------------------------------------------------------------------------------- 1 | # Required Packages 2 | 3 | The following packages will be used when we build the 4 | cross compilation toolchain. Each package's functionality will 5 | be explained later. You need to download these files and store 6 | them inside the `/opt/cross/src` directory. The download links given 7 | here are for your convenience; Alternatively, you can find mirror 8 | sites that are closer to your geographical location to speed up the 9 | downloads. 10 | 11 | > __Important__: you must use the _exact_ version specified for 12 | > each package, as it has been observed that certain incompatible 13 | > versions may break the build process in obscure ways. 14 | 15 | * [binutils-2.30.tar.gz](https://ftp.gnu.org/gnu/binutils/binutils-2.30.tar.gz) 16 | * [linux-4.18.5.tar.gz](https://mirrors.edge.kernel.org/pub/linux/kernel/v4.x/linux-4.18.5.tar.gz) 17 | * [mpfr-4.0.1.tar.xz](https://ftp.gnu.org/gnu/mpfr/mpfr-4.0.1.tar.xz) 18 | * [mpc-1.1.0.tar.gz](https://ftp.gnu.org/gnu/mpc/mpc-1.1.0.tar.gz) 19 | * [gmp-6.1.2.tar.xz](https://ftp.gnu.org/pub/gnu/gmp/gmp-6.1.2.tar.xz) 20 | * [gcc-7.3.0.tar.gz](https://ftp.gnu.org/gnu/gcc/gcc-7.3.0//gcc-7.3.0.tar.gz) 21 | * [musl-1.1.20.tar.gz](http://git.musl-libc.org/cgit/musl/snapshot/musl-1.1.20.tar.gz) 22 | * [busybox-1.30.0.tar.bz2](https://busybox.net/downloads/busybox-1.30.0.tar.bz2) 23 | 24 | > Note: previously it was thought that the ISL package is unnecessary. However, some users 25 | > have reported `gcc` build errors due to missing the ISL package. The exact reason 26 | > is unclear as of now. For more information, see [#27](https://github.com/Unturned3/Microdot/issues/27) 27 | > and [#29](https://github.com/Unturned3/Microdot/pull/29). 28 | 29 | * [isl-0.24.tar.xz](https://gcc.gnu.org/pub/gcc/infrastructure/isl-0.24.tar.bz2) 30 | 31 | -------------------------------------------------------------------------------- /wiki/2_crossToolchain/3_build.md: -------------------------------------------------------------------------------- 1 | # Constructing the Cross Compilation Toolchain 2 | 3 | In this section, we will use the source packages that we previously 4 | downloaded to make a cross toolchain that runs on `x86_64` and produces 5 | code for `x86_64`. Here, we are using the cross compiler not to generate 6 | code for different platforms, but to isolate the final binaries from 7 | the host system's environment, because by definition a cross compiler 8 | cannot rely on anything in the host environment. Doing so eliminates the 9 | risk of our host system contaminating the binaries that will be used 10 | in Microdot Linux. 11 | 12 | The process that we use to build our cross compiler is adapted from the 13 | [musl-cross-make](https://github.com/richfelker/musl-cross-make) script by 14 | richfelker. Without his help I would 15 | have wasted a lot more time doing trail-and-error by myself and Microdot 16 | probably won't even exist. For more information, check out the 17 | [thanks](/thanks.md) page. 18 | 19 | Unlike other build procedure on the internet (such as Linux from Scratch, or 20 | crosstool-ng), the one that we will use only builds `gcc` once instead of twice 21 | or even three times in some cases. The result is a drastically simpler process, 22 | and it is a lot less time consuming. 23 | 24 | ## Environment Setup 25 | 26 | First, we need to substitute our identity as the `builder` user. This 27 | allows us to work in a clean environment with access to the variables 28 | that we previously defined in the initialization scripts. 29 | 30 | Note that if you exit the shell that you're working in or shut down your 31 | system during the middle of the build process, you need to run the 32 | substitute user command again before resuming from where you left off. 33 | 34 | ```bash 35 | su - builder 36 | ``` 37 | 38 | Then, we will setup a few directories and links in the cross compiler 39 | building area for convenience purposes. The `$sysroot` folder contains 40 | an imagined root file system for our target, and it will contain target- 41 | specific files, such as its C library. The `lib` folder contains the 42 | various libraries and support files that our tools will be using. We then 43 | point `lib64` to `lib`, because sometimes the build system of a package 44 | will install its contents into the `lib64` folder when we are building 45 | on a 64 bit system. We want to keep all the files in `lib` so we link them. 46 | We also point `$sysroot/usr` to `.` because we don't want anything to be 47 | installed into a separate `usr` directory. 48 | 49 | ```bash 50 | 51 | mkdir -pv $sysroot 52 | mkdir -pv $install/lib 53 | mkdir -pv $sysroot/lib 54 | 55 | ln -sfv $install/lib $install/lib64 56 | ln -sfv $sysroot/lib $sysroot/lib64 57 | ln -sfv . $sysroot/usr 58 | 59 | cd $install/src # go to the directory containing all the packages 60 | ``` 61 | 62 | ## How to Work With Source Code Packages 63 | 64 | For every section below, you need to use the `tar` command to uncompress 65 | and untar the archives (such as `.tar.gz` or `.tar.xz`, etc). Then you 66 | need to change directory into the folder that's created by the untarred 67 | package. Instructions in these sections assume that you are starting in the 68 | untarred folder of a package. An example of how this is done is given in 69 | the _linux kernel headers_ section. 70 | 71 | ## Linux Kernel Headers 72 | 73 | First, we will install the Linux kernel headers into the `$sysroot` 74 | directory. The kernel headers can allow an application to know what 75 | features are enabled and usable in the kernel's interface. The headers 76 | are needed by `busybox` because it implements a lot of its functions 77 | by utilizing the kernel's API. We have to install the headers first, 78 | because installing it later would cause it to overwrite other files installed 79 | in the same location by `gcc`. 80 | 81 | ```bash 82 | tar -xf linux-4.18.5.tar.gz # uncompress & untar the package 83 | cd linux-4.18.5 # cd into the folder created 84 | 85 | # First, we use the "distclean" target to clean up the source folder 86 | make distclean 87 | 88 | # specify the architecture via the variable ARCH 89 | make ARCH=$arch headers_check 90 | 91 | # specify that we want the headers to be installed in $sysroot 92 | make ARCH=$arch INSTALL_HDR_PATH=$sysroot headers_install 93 | 94 | cd .. # leave the source folder 95 | ``` 96 | 97 | For every section onwards, it is assumed that you will first untar the 98 | specified package, change directory into it, and go on from there. Do 99 | not delete the source folder after finishing a section, because it might 100 | be needed later. 101 | 102 | ## binutils 103 | 104 | The binutils package contains the linker, assembler, and a few other tools 105 | to manipulate binary files. When you invoke `gcc` to compile C code, 106 | `gcc` actually uses tools from `binutils` to assemble the final binary. 107 | This is why binutils is built first, as `gcc` needs it in order to compile 108 | code, and both `gcc` and `musl-libc` performs several tests on the 109 | binutils tools and determine what features to enable/disable. 110 | 111 | The binutils built in this step will be installed in `$install/bin` and the 112 | tool's name will all be prefixed with the target architecture triplet 113 | (`x86_64-linux-musl` in this case). This set of binutils tools will be used 114 | by `gcc` in the finished cross toolchain later, as it is configured to run 115 | on our host machine, but generates code for the target architecture. Even 116 | though our host and targets are the same architecture, this is done to 117 | isolate the target against possible contaminations from the host system. 118 | 119 | > Unpack and change directory into the binutils-2.30 package 120 | 121 | ```bash 122 | mkdir build # create a folder named "build" inside the source folder 123 | cd build # build binutils in this folder instead 124 | 125 | ../configure \ # the backslashe breaks 1 line into several 126 | --prefix=$install \ 127 | --target=$target \ 128 | --with-sysroot=$sysroot \ 129 | --with-lib-path=$sysroot/lib \ 130 | --disable-werror \ # prevents warnings from stopping the build 131 | --disable-nls # disable internationalization 132 | 133 | make -jN 134 | make install # install the compiled files 135 | ``` 136 | 137 | * `--prefix` 138 | 139 | This option specifies the location of installation. In this case, we 140 | want to install all the binutils to `$install`, which is `/opt/cross` 141 | 142 | * `--target` 143 | 144 | This option specifies the target architecture that binutils will 145 | be built for. 146 | 147 | * `--with-sysroot` 148 | 149 | This option specifies the location of the system root. This is where 150 | all the header files, libraries, etc. for the target system are 151 | contained. When linking binaries for the target, binutils tools will 152 | look into this sysroot directory for header files, etc. 153 | 154 | * `--with-lib-path` 155 | 156 | This option explicitly states the directory to find libraries in when 157 | binutils tools are used. In this case, libraries will be installed 158 | in `$sysroot/lib`. This prevents the tools from accidentally linking 159 | a target binary to a library located on the host system, such as 160 | `/usr/lib`. 161 | 162 | * `make -jN` 163 | 164 | Replace N with the number of CPU cores that your system has. This 165 | enables `make` to speed up the build job by working in parallel across 166 | multipul CPU cores. For example, if you have a dual-core system, then 167 | you would type `make -j2`. 168 | 169 | 170 | ## gcc compiler 171 | 172 | The `gcc-7.3.0` package contains the C/C++ compiler, the `libgcc` compiler 173 | support library, the C++ standard library, and a few others. 174 | 175 | > Unpack and change directory into the gcc-7.3.0 package 176 | 177 | `gcc` depends on a few external libraries to implement certain functions. 178 | These external libraries doesn't depend on anything else, so we can just 179 | build them directly with gcc here. We will unpack the libraries into the 180 | source folder of `gcc`: 181 | 182 | ```bash 183 | tar -xf ../mpfr* 184 | tar -xf ../mpc* 185 | tar -xf ../gmp* 186 | tar -xf ../isl* # For more information about ISL, see Note below 187 | ``` 188 | 189 | > Note: previously it was thought that the ISL package is unnecessary. However, some users 190 | > have reported `gcc` build errors due to missing the ISL package. The exact reason 191 | > is unclear as of now. For more information, see [#27](https://github.com/Unturned3/Microdot/issues/27) 192 | > and [#29](https://github.com/Unturned3/Microdot/pull/29). 193 | 194 | Then we have to "clean up" the names by leaving just the letters and 195 | removing the version number. For example, we will rename `mpfr-4.0.1` as 196 | `mpfr`. 197 | 198 | ```bash 199 | mv mpfr* mpfr 200 | mv mpc* mpc 201 | mv gmp* gmp 202 | mv isl* isl 203 | ``` 204 | 205 | `gcc` build system will automatically detect the presence of these 206 | directories and compile them before compiling `gcc` it self. Now, we 207 | will configure and build the bootstrap compiler. Again, we do this in a 208 | separate directory: 209 | 210 | ```bash 211 | mkdir build 212 | cd build 213 | 214 | ../configure \ 215 | --build=$host \ 216 | --host=$host \ 217 | --target=$target \ 218 | --prefix=$install \ 219 | --with-sysroot=$sysroot \ 220 | --with-native-system-header-dir=/include \ 221 | --enable-shared \ 222 | --enable-tls \ 223 | --enable-languages=c,c++ \ 224 | --enable-c99 \ 225 | --enable-long-long \ 226 | --disable-nls \ 227 | --disable-libmudflap \ 228 | --disable-libmpx \ 229 | --disable-libsanitizer \ 230 | --disable-multilib 231 | ``` 232 | 233 | * `--build=xxx, --host=xxx, --target=xxx` 234 | 235 | This three flags specifies the architecture of the machine that `gcc` 236 | is being _built_ on, the machine that `gcc` will _run_ on, and the 237 | machine that `gcc` will _generate code for_, respectively. We are both 238 | building and running `gcc` on `x86_64-cross-linux-gnu`, and we want 239 | `gcc` to generate code for `x86_64-linux-musl`, as that is the 240 | architecture of Microdot Linux. 241 | 242 | * `--with-native-system-header-dir` 243 | 244 | Tells `gcc` to look for headers in this directory. Note that 245 | this directory is relative to the previously set `--with-sysroot` 246 | option, so `/include` actually refers to `$sysroot/include` 247 | 248 | * `--enable-shared` 249 | 250 | Tells `gcc` to build libraries as shared (aka. dynamic). This 251 | affects the `libgcc` and `libstdc++` libraries later, and makes 252 | it possible to dynamically link to them. 253 | 254 | * `--disable-nls` 255 | 256 | We do not need native language support 257 | 258 | * `--disable-libmudflap, --disable-libmpx, --disable-libsanitizer 259 | --disable-multilib` 260 | 261 | These features are either broken or won't work with `musl-libc` 262 | 263 | Install the compiler: 264 | 265 | ```bash 266 | make -jN all-gcc # only build the gcc compiler, nothing else 267 | make install-gcc # only install what we built 268 | ``` 269 | 270 | If you look under `$install/bin`, you would see that `gcc` and various 271 | other tools has been installed in there, all prefixed with our target 272 | architecture triplet, which is `x86_64-linux-musl`. If you were 273 | to invoke `$target-gcc` and try to compile some C code, you would be 274 | greeted with a load of error messages saying that certain files are 275 | missing. This is because our current compiler has no access to any 276 | part of the C library, which makes it impossible to compile a runnable 277 | binary. However, this compiler can compile _freestanding_ code, 278 | that is, code that does not use the C library at all. The Linux kernel is 279 | an example of _freestanding_ code. 280 | 281 | 282 | ## musl-libc headers 283 | 284 | In this step we will install the header files provided by `musl-libc`. The 285 | headers are needed for us to build our compiler support library (`libgcc`) 286 | in the next step. We will tell the build system of `musl-libc` to use our 287 | previously made bootstrap compiler instead of the one on our host system. 288 | 289 | > Unpack and change directory into the musl-1.1.10 package 290 | 291 | ```bash 292 | ./configure \ 293 | CROSS_COMPILE=$target- \ 294 | --prefix=/ \ 295 | --target=$target 296 | ``` 297 | 298 | * `CROSS_COMPILE=$target-` 299 | 300 | We use this variable to specify the `cross compiler prefix` that the 301 | build system will use. So instead of invoking just `gcc`, it will 302 | prepend the prefix that we provided, so `gcc` turns into `$target-gcc`, 303 | which is `x86_64-linux-musl-gcc` in this case. This allows the build 304 | system to use our bootstrap compiler. 305 | 306 | * `--prefix=/` 307 | 308 | The `prefix` option in `musl-libc` doesn't seem to be able to control 309 | the installation location. So we set it to `/` here, and rectify it 310 | later via the `DESTDIR` variable during the installation step. 311 | 312 | Compile and install the `musl-libc` headers: 313 | 314 | ```bash 315 | make -jN DESTDIR=$sysroot install-headers 316 | ``` 317 | 318 | Now, if you look into `$sysroot/include`, you should see all the familiar 319 | C header files like `stdio.h` and `stdlib.h`, and so on. These headers 320 | will be used when building `libgcc` and other binaries later on. 321 | 322 | 323 | ## Static libgcc 324 | 325 | `libgcc` is a low-level compiler support library and it is used by all 326 | dynamic binaries produced by `gcc`. It contains "support code" that 327 | implements certain features, such as 64-bit division, etc. `musl-libc` 328 | requires `libgcc`, so we will build a static one here in order to construct 329 | `musl-libc` in the next step. 330 | 331 | > Change directory into the previously unpacked `gcc` folder. `libgcc` is a 332 | > part of the `gcc` package. 333 | 334 | ```bash 335 | cd build 336 | make -jN MAKE="make enable_shared=no" all-target-libgcc 337 | make -jN MAKE="make enable_shared=no" install-target-libgcc 338 | ``` 339 | 340 | * `MAKE="make enable_shared=no"` 341 | 342 | The `make` command will recursively pass the `MAKE` variable to itself 343 | when it builds certain subtasks required by a target. We pass the 344 | `enable_shared=no` flag here to make sure only a static `libgcc` is 345 | built. 346 | 347 | * `all-target-libgcc` 348 | 349 | This tells `make` to only build `libgcc` and nothing else. 350 | 351 | 352 | ## Complete musl-libc 353 | 354 | Now we are ready to build the final C library. 355 | 356 | > Change directory into the previously unpacked `musl` folder. 357 | 358 | ```bash 359 | make -jN \ 360 | CC=$target-gcc \ 361 | LIBCC=$install/lib/gcc/$target/7.3.0/libgcc.a \ 362 | DESTDIR=$sysroot \ 363 | all 364 | 365 | make DESTDIR=$sysroot install 366 | ``` 367 | 368 | * `CC=$target-gcc` 369 | 370 | This tells the build system of `musl-libc` to use `$target-gcc` as the 371 | compiler when compiling the source code. 372 | 373 | * `LIBCC=...` 374 | 375 | This tells the build system of `musl-libc` where is the static libgcc 376 | located. 377 | 378 | After running `make install`, you should see files such as `libc.so` inside 379 | the `$sysroot/lib` directory. `libc.so` is our `musl` C library (.so stands 380 | for "shared object", aka. a dynamic library). Binaries built using our cross 381 | compiler toolchain will be linked to this library instead of our host's 382 | `glibc` (GNU C library). 383 | 384 | ## Other gcc components 385 | 386 | > Change directory into the previously unpacked gcc-7.3.0 folder 387 | 388 | ```bash 389 | cd build 390 | make -jN all 391 | make install 392 | ``` 393 | 394 | After installing everything, you should find `$target-gcc` and `$target-g++` 395 | and a few other tools installed inside `$install/bin`, and the C++ standard 396 | library & headers installed in `$sysroot`. Now we can use this complete 397 | toolchain to compile binaries for Microdot. 398 | 399 | ## Testing the Cross Compiler 400 | 401 | Instead of using `gcc` (which calls the gcc installed on the 402 | host system), we will invoke `$target-gcc`, which translates 403 | to `x86_64-linux-musl-gcc`. This is our cross compiler installed 404 | in the `$install/bin` directory. 405 | 406 | First, write a simple C program in builder's home directory: 407 | (we want to keep the cross toolchain directory clean) 408 | 409 | 410 | ```bash 411 | cd ~ 412 | cat > test.c << "EOF" 413 | 414 | #include 415 | int main() 416 | { 417 | printf("Hello, world!\n"); 418 | return 0; 419 | } 420 | 421 | EOF 422 | ``` 423 | 424 | Then, invoke `$target-gcc` to compile this C program. The `-o test` option 425 | tells gcc to name the output executable as `test`. The `--static` option 426 | tells gcc to statically link the executable (aka. copy the musl-libc 427 | contents into the final executable file). If we compile the binary 428 | dynamically, it will be linked to `musl-libc`. However, `musl-libc` is only 429 | present inside the `$sysroot/lib` and not installed to the host systems 430 | `/lib` directory. So if we were to run this program, then it wouldn't be 431 | able to find `musl-libc`. 432 | 433 | ```bash 434 | $target-gcc test.c -o test --static 435 | ``` 436 | 437 | Now run the program, and you should see `Hello, world!` on the screen. 438 | 439 | ```bash 440 | ./test 441 | ``` 442 | 443 | ## You Made It! 444 | 445 | Congratulations! This is what I consider to be the hardest and most 446 | agitating part of the build process. In the next section, we will utilize 447 | this toolchain to add software to our Microdot system. Then after that, 448 | we will go on to configure our first Linux kernel. 449 | -------------------------------------------------------------------------------- /wiki/3_miniSystem/1_dirAndFiles.md: -------------------------------------------------------------------------------- 1 | 2 | # Constructing the Skeleton 3 | 4 | Before proceeding onto installing software using our cross toolchain, we will 5 | first create the skeleton of our Microdot system. That is, we need to create 6 | a few important directories and files, and we'll name these 7 | folders according to the established convention. The skeleton will be built in 8 | the `$targetfs` directory. The contents of this folder will later be copied 9 | onto a bootable disk image. 10 | 11 | The directory structure that we will construct here is a greatly simplified 12 | version of the one specified by the Filesystem Hierarchy Standard, with a bunch 13 | of folders omitted. For more details regarding the Linux directory structures, 14 | check out the following links: 15 | 16 | * [PUT LINK HERE](https://www.a.com) 17 | * [PUT LINK HERE](https://www.b.com) 18 | 19 | # Important Directories 20 | 21 | ```bash 22 | # essential directories 23 | # note that we are not using the /usr directory 24 | mkdir -p $targetfs/{bin,sbin,dev,proc,sys,lib} 25 | mkdir -p $targetfs/etc/init.d 26 | mkdir -p $targetfs/var/{lock,log} 27 | 28 | # Create these directories with special permissions 29 | install -dm 0750 $targetfs/root # 0750: rwxr-x--- 30 | install -dm 1777 $targetfs/tmp # 1777: rwxrwxrwx, with sticky bit set 31 | install -dm 1777 $targetfs/var/tmp # same as above 32 | 33 | # link "lib64" to "lib" so all libraries can be installed neatly in one place 34 | ln -sfv $targetfs/lib $targetfs/lib64 35 | ``` 36 | 37 | # Configuration & Log Files 38 | 39 | `/etc/mtab` is a file that shows a list of mounted file systems, and 40 | `/proc/mounts` contains exactly that information, so we symlink these two 41 | together. 42 | ```bash 43 | ln -sfv ../proc/mounts $targetfs/etc/mtab 44 | ``` 45 | 46 | The `/etc/passwd` file is a text-based database of information about users, such 47 | as their user name, user/group ID number, login shell, and so on. For now, we 48 | only need the root user in here. For more information about `/etc/passwd`, 49 | please visit [this](www.c.com)(DEADLINK: FIXME) link. 50 | 51 | ```bash 52 | cat > $targetfs/etc/passwd << "EOF" 53 | root::0:0:root:/root:/bin/ash 54 | EOF 55 | ``` 56 | 57 | `/etc/group` is another text-based database of information about the groups 58 | present on the system. For now, we only have the root group. 59 | 60 | ```bash 61 | cat > $targetfs/etc/group << "EOF" 62 | root:x:0: 63 | EOF 64 | ``` 65 | 66 | The `/var/log/lastlog` file is used by programs like `init` or `login` to record 67 | information about users that logged into the system. These programs won't write 68 | to the `lastlog` file if it doesn't exist, so we create them here. 69 | 70 | ```bash 71 | touch $targetfs/var/log/lastlog 72 | chmod -v 664 $targetfs/var/log/lastlog 73 | ``` 74 | 75 | # System Initialization Scripts 76 | 77 | We will use `busybox init` as our init system. When the kernel passes control 78 | to `/sbin/init`, it will read `/etc/inittab` and carry out actions specified in 79 | there. The `inittab` file format for busybox init is a bit different from the 80 | traditional Sys V Init system. For more information, check out 81 | [this](https://git.busybox.net/busybox/tree/examples/inittab) page from 82 | `busybox.net`. 83 | 84 | ```bash 85 | cat > $targetfs/etc/inittab << "EOF" 86 | 87 | ::sysinit:/etc/init.d/startup # execute a script that we will write ourselves 88 | ::respawn:/sbin/getty -L ttyS0 9600 vt100 # start a terminal on ttyS0 (serial line) 89 | # ::askfirst:-/bin/sh 90 | 91 | ::ctrlaltdel:/sbin/reboot # what do do when the ctrl-alt-del keys are pressed 92 | 93 | ::shutdown:/bin/umount -a -r # what to do during shutdown 94 | ::restart:/sbin/init # what to do during restart 95 | 96 | EOF 97 | ``` 98 | 99 | This is the custom "startup" script that `busybox init` will run when the system starts. 100 | The script performs a few tasks that's vital for the operation of the system, such as mounting 101 | the virtual file systems, and printing some messages. 102 | 103 | ```bash 104 | cat > $targetfs/etc/init.d/startup << "EOF" 105 | #!/bin/sh 106 | 107 | # mount pseudo file systems 108 | mount -t proc proc /proc 109 | mount -t sysfs sysfs /sys 110 | mount -t devtmpfs devtmpfs /dev 111 | 112 | # print some messages 113 | echo Boot took $(cut -d' ' -f1 /proc/uptime) seconds. 114 | echo Welcome to Microdot Linux! 115 | 116 | EOF 117 | 118 | chmod +x $targetfs/etc/init.d/startup 119 | ``` 120 | 121 | With the skeleton finished, we can now populate it with some software. 122 | 123 | -------------------------------------------------------------------------------- /wiki/3_miniSystem/2_install.md: -------------------------------------------------------------------------------- 1 | # Installing Basic Software 2 | 3 | In this section we will install the essential system software programs into 4 | `$targetfs`. The `$targetfs` directory forms the skeleton of our system, and 5 | we will later "package" the contents into an image readable by the kernel. 6 | 7 | ## Clean Up the Source Directory 8 | 9 | We've finished constructing the cross toolchain, and now we will remove the 10 | directories extracted previously so we can work in a clean environment: 11 | 12 | ```bash 13 | cd $install/src 14 | rm -rf binutils-2.30 gcc-7.3.0 linux-4.18.5 musl-1.1.20 15 | ``` 16 | 17 | ## libgcc 18 | 19 | Install the `libgcc` support library, so other dynamic binaries compiled by 20 | `gcc` can actually run on our final system. 21 | 22 | > Note: libgcc is a part of the gcc-7.3.0 package. Unpack and change directory into gcc-7.3.0 23 | 24 | ```bash 25 | tar -xf ../mpfr* 26 | tar -xf ../mpc* 27 | tar -xf ../gmp* 28 | 29 | mv mpfr* mpfr 30 | mv gmp* gmp 31 | mv mpc* mpc 32 | 33 | mkdir build 34 | cd build 35 | 36 | ../configure \ 37 | --prefix=$targetfs \ 38 | --host=$target \ 39 | --target=$target \ 40 | --disable-multilib \ 41 | --disable-bootstrap \ 42 | --disable-nls \ 43 | --disable-libmpx \ 44 | --disable-libsanitizer \ 45 | --disable-libmudflap \ 46 | --enable-c99 \ 47 | --enable-long-long \ 48 | --enable-tls \ 49 | --enable-languages=c,c++ 50 | 51 | make -jN all-target-libgcc 52 | make install-target-libgcc 53 | 54 | $target-strip $targetfs/lib/libgcc_s.so.1 55 | rm -r $targetfs/lib/gcc 56 | ``` 57 | 58 | # static vs dynamic system 59 | 60 | You can either build a statically linked userspace or a dynamically linked one. 61 | "Dynamically linked" refers to the situation where the programs in the userspace 62 | all depend on some shared library (such as musl-libc). In this case, musl-libc needs 63 | to be present in the final Microdot system, or else programs such as Busybox won't run. 64 | "Statically linked" refers to the situation where the portion of the shared library is 65 | directly copied into the program that you are compiling. For example, if we compile 66 | Busybox as statically linked, it will copy a portion of musl-libc into the finished 67 | Busybox binary, making it sort of "self-contained" in the sense that we don't need 68 | musl-libc to be present on the final Microdot system. 69 | 70 | An obvious downside to statically linking every userspace program is that there will 71 | be many redundant copies of the same code in every program. Dynamic linking reduces 72 | the redundant space waste by packing the code that every program uses in a shared library. 73 | 74 | For more information on this topic, visit [link1](https://stackoverflow.com/questions/1993390/static-linking-vs-dynamic-linking) and 75 | [link2](https://cs-fundamentals.com/tech-interview/c/difference-between-static-and-dynamic-linking). 76 | 77 | However, for our system, we only have 1 binary, which is Busybox. Statically linking 78 | Busybox in this case will result in an overall smaller root file system, but dynamic 79 | linking (and installing musl-libc) will make expanding the system & adding more software 80 | easier. You can choose either option here. 81 | 82 | # musl-libc 83 | 84 | Install musl-libc onto the target system, so dynamically linked binaries can 85 | run. If you are building a statically-linked system with no shared libs, skip 86 | this step and continue from the busybox section below. 87 | 88 | > Note: unpack and change directory into the musl-1.1.20 directory 89 | 90 | ```bash 91 | ./configure \ 92 | CROSS_COMPILE=$target- \ 93 | --prefix=/ \ 94 | --disable-static \ 95 | --target=$target 96 | 97 | make -j4 98 | make DESTDIR=$targetfs install-libs 99 | ``` 100 | 101 | # busybox 102 | 103 | Busybox is the main component of Microdot's userspace. It contains hundreds of 104 | Linux utilities combined into one tiny executable. Linking it with `musl-libc` 105 | results in an even smaller binary compared to other libraries. 106 | 107 | > Note: unpack and change directory into the busybox-1.30.0 directory 108 | 109 | ```bash 110 | make distclean 111 | make ARCH=$arch defconfig 112 | ``` 113 | 114 | Now, run the following command to launch the "menuconfig" interface. We need to 115 | configure busybox to install everything to `/bin` and `/sbin`, and ignore `/usr`. 116 | 117 | ```bash 118 | make ARCH=$arch menuconfig 119 | ``` 120 | 121 | You should now see a text-based configuration interface. You can use the arrow keys, 122 | enter key, spacebar, and ESC key to navigate. "Y" means to select the option, and "N" 123 | means to deselect it. We aren't using the `/usr` directory for the sake of simplicity, 124 | and we are disabling `linuxrc` because it is a feature designed to provide 125 | compatibility for the `initrd` (initial ram disk) mechanism. Initramfs is preferred 126 | over initrd in our case. 127 | 128 | ``` 129 | Settings ---> 130 | Don't use /usr: Y 131 | Init Utilities ---> 132 | linuxrc: support running init from initrd (not initramfs): N 133 | 134 | # If you are building a statically-linked busybox, select the following as well: 135 | 136 | Settings ---> 137 | Build static binary (no shared libs): Y 138 | ``` 139 | 140 | Save and exit the configuration interface. Now compile & install busybox: 141 | 142 | ```bash 143 | make -jN ARCH=$arch CROSS_COMPILE=$target- CONFIG_PREFIX=$targetfs install 144 | ``` 145 | 146 | Lastly, we have to "setuid" the busybox exectuable. Normally, when a binary 147 | is run, its privileges are the same as the user who started it. However, if 148 | a binary has the setuid bit set, it will execute with the privilege 149 | of the user who owns it. In our case, `root` owns the busybox binary. We need 150 | to set the "setuid" bit of busybox or else commands like `passwd` won't work. 151 | Busybox drops the elevated privileges for most other applets, so this isn't a 152 | security risk. 153 | 154 | ```bash 155 | chmod 4755 $targetfs/bin/busybox # the leading '4' is the setuid bit 156 | ``` 157 | 158 | You have to change the ownership of the `busybox` binary to root. However, 159 | you can only do that if you have root privileges. 160 | 161 | ```bash 162 | su # login as root uesr 163 | chown root:root -R /opt/targetfs 164 | exit 165 | ``` 166 | 167 | 168 | 169 | 170 | 171 | -------------------------------------------------------------------------------- /wiki/3_miniSystem/3_miniKernel.md: -------------------------------------------------------------------------------- 1 | # Configuring a Minimal Kernel 2 | 3 | The Linux kernel is made out of many subsystems and it supports a lot of 4 | different hardware platforms. If we never manually configure it and 5 | leave it to use the defaults, then a lot of unneeded subsystems and code 6 | will be added to the final product. How much space does this waste? 7 | An unmodified default configuration will generate a 8 to 10 megabyte kernel 8 | image. On the other hand, a nicely tuned manual config will keep the image 9 | under 1MB with plenty of functionality included. 10 | 11 | Configuring the kernel basically boils down to selecting code that supports the 12 | various hardware devices that you want to use, and including features that will 13 | benefit userspace programs. For example, you might want to select the relevant 14 | drivers for your onboard ethernet card, or for a hard drive, etc. Please read 15 | [this](https://wiki.gentoo.org/wiki/Kernel/Gentoo_Kernel_Configuration_Guide) 16 | Gentoo Wiki page in order to consolidate your understanding of the various 17 | concepts involved. It might have some Gentoo-specific terminologies that 18 | we won't use here. 19 | 20 | > Unpack & change directory into the linux-4.18.5 directory 21 | 22 | ```bash 23 | # clean up the source directory 24 | make distclean 25 | 26 | # configure a kernel with minimum features enabled 27 | make ARCH=$arch CROSS_COMPILE=$target- tinyconfig 28 | 29 | # launch the configuration interface 30 | make ARCH=$arch CROSS_COMPILE=$target- nconfig 31 | ``` 32 | 33 | Now you should be seeing a text based configuration interface, with entries 34 | similar to the following (hashtags are comments) 35 | 36 | ```text 37 | 64-bit kernel 38 | 39 | # configure various kernel features 40 | General setup 41 | 42 | # should the kernel use any loadable modules? 43 | Enable loadable module support 44 | 45 | # include code to handle block devices? (hard disk, etc.) 46 | Enable the block layer 47 | 48 | # configure features related to CPUs 49 | Processor type and features 50 | 51 | # power management features 52 | Power management and ACPI options 53 | 54 | # system bus features 55 | Bus options (PCI etc.) 56 | 57 | Executable file formats / Emulations 58 | 59 | # networking subsystem 60 | Networking support 61 | 62 | # support code for different hardware devices 63 | Device Drivers 64 | 65 | Firmware Drivers 66 | 67 | # file system support (ext4, FAT, etc.) 68 | File systems 69 | 70 | Kernel hacking 71 | Security options 72 | Cryptographic 73 | Virtualization 74 | Library routines 75 | ``` 76 | 77 | 78 | 79 | You can navigate 80 | around using the arrow keys, ESC key, and return key. Every indentation in the 81 | recipe means that you need to enter a sub-menu to access the options. "Y" means 82 | that you need to select the corresponding option ("N" means deselect). Again, 83 | a hashtag ("#") indicates a comment. For each option, you can hit the "?" key to 84 | show a description/explanation (this is great for learning how things work). 85 | The descriptions sometimes will 86 | point you to more information in the Linux documentation. 87 | 88 | Configure the kernel according to the following recipe. As we said, this section 89 | focuses on constructing a bare minimum system, and you should notice that we 90 | didn't really include support for any real hardware. 91 | 92 | ```text 93 | 64-bit kernel: Y 94 | 95 | General Setup 96 | Default hostname: Microdot 97 | 98 | Initial RAM filesystem and RAM disk (initramfs/initrd) support: Y 99 | Support initial ramdisk/ramfs compressed using bzip2: N 100 | Support initial ramdisk/ramfs compressed using LZMA: N 101 | Support initial ramdisk/ramfs compressed using XZ: N 102 | Support initial ramdisk/ramfs compressed using LZO: N 103 | Support initial ramdisk/ramfs compressed using LZ4: N 104 | 105 | Configure standard kernel features (expert users) 106 | Multiple users, groups and capabilities support: Y 107 | Posix Clocks & timers: Y 108 | 109 | # printk() is what the kernel uses to print messages onto the screen 110 | Enable support for printk: Y 111 | 112 | Executable file formats / Emulations 113 | Kernel support for ELF binaries: Y 114 | Kernel support for scripts starting with #!: Y 115 | 116 | Device Drivers 117 | Generic Driver Options 118 | Maintain a devtmpfs filesystem to mount at /dev: Y 119 | Automount devtmpfs at /dev, after the kernel mounted the rootfs: Y 120 | Character devices 121 | Enable TTY: Y 122 | Virtual terminal: N 123 | Unix98 PTY support: N 124 | Legacy (BSD) PTY support: N 125 | Serial drivers 126 | 8250/16550 and compatible serial support: Y 127 | Console on 8250/16550 and compatible serial port: Y 128 | 129 | File systems 130 | Enable POSIX file locking API: Y 131 | Pseudo filesystems 132 | /proc file system support: Y 133 | sysfs file system support: Y 134 | 135 | ``` 136 | 137 | Save, exit, and compile the kernel. The finished image should be around 700kB. 138 | If we used `make defconfig` instead of `make tinyconfig` and hand-tuning the 139 | kernel, we would end up with a 8MB image, which contains a plethora of bloat 140 | that we'll never use. `make defconfig` is what most tutorials out there on the 141 | internet instructs you to do. 142 | 143 | ```bash 144 | make ARCH=$arch CROSS_COMPILE=$target- -jN 145 | 146 | # copy the image to /opt (you probably need root permission for this) 147 | cp arch/x86/boot/bzImage /opt 148 | ``` 149 | 150 | In the next section, we will pack the content of targetfs into a `initramfs`, 151 | and boot our Microdot system. 152 | 153 | 154 | 155 | 156 | 157 | 158 | 159 | 160 | 161 | 162 | -------------------------------------------------------------------------------- /wiki/3_miniSystem/4_boot.md: -------------------------------------------------------------------------------- 1 | 2 | # Assembling & Running the Final System 3 | 4 | In this section, we will assemble the parts of our mini system and boot it 5 | via QEMU. First, we need to pack the contents of `$targetfs` into an `initramfs` 6 | archive. An `initramfs` is a compressed 7 | [cpio](https://en.wikipedia.org/wiki/Cpio) archive that is loaded into RAM at 8 | boot by the bootloader, and the kernel mounts it as a temporary root file 9 | system. You should read [this](https://landley.net/writing/rootfs-intro.html) 10 | article by Rob Landley to understand what it 11 | is and why is it useful before proceeding to build Microdot. 12 | 13 | The procedure to create a `cpio` archive is not very straight forward, but its 14 | easy to understand. If we want to put the content of the `$targetfs` directory 15 | into an initramfs, we have to run the following command (as root): 16 | 17 | ```bash 18 | cd $targetfs 19 | find . | cpio -H newc -o | gzip > /opt/initramfs.gz 20 | ``` 21 | 22 | This is how the command works: 23 | 24 | The `find .` command prints a list of files present in the current directory. 25 | Then the output of that gets piped into the input of `cpio`. It will then read 26 | the list of files given, pack them using the `newc` format, and output the 27 | finished archive. The output from `cpio` is then piped to the input of `gzip`, 28 | which compresses the archive. Finally, the output from `gzip` is redirected to 29 | the file `/opt/initramfs.gz`, and that gives us a gzip compressed cpio archive 30 | of the `$targetfs` directory. 31 | 32 | # Running the minimum system 33 | 34 | Finally! We made it to the last step. Let's fire up QEMU and boot our system. 35 | You can run the following command as the builder user or a regular user on your system: 36 | 37 | ```bash 38 | qemu-system-x86_64 \ 39 | -kernel bzImage \ 40 | -append "rdinit=/sbin/init console=ttyS0 quiet" \ 41 | -initrd initramfs.gz \ 42 | -m 32M \ 43 | -nographic 44 | ``` 45 | 46 | The mini system should boot, and you should be soon presented with a login 47 | prompt. Login with root (no password), and do whatever you like. Note that 48 | this tiny system has no presistent storage. That is, if you make any changes 49 | to the files and reboot the machine, all changes would be lost. This is 50 | because our root file system is an `initramfs`, which means that the content 51 | of it resides in `RAM`. Whatever modifications we make to the data in RAM 52 | will be lost if we can't write it to a non-volatile storage device. 53 | 54 | If you are done, you can `poweroff` the system. `QEMU`, however, will 55 | not terminate and return you to a terminal prompt, because our systems kernel 56 | has no support for the ACPI subsystem yet, so the kernel has no way of telling 57 | the hardware that the system is shutting down. We can open up a new terminal 58 | and kill `QEMU` with the following command: 59 | 60 | ```bash 61 | killall qemu-system-x86_64 62 | ``` 63 | -------------------------------------------------------------------------------- /wiki/3_miniSystem/readme.md: -------------------------------------------------------------------------------- 1 | # Bare Minimum 2 | 3 | In this section, we will create a Linux system so small to the extent that it's 4 | somewhat useless. It has no support for any peripheral devices such as keyboard, 5 | screen, storage, or network, and its only means of interacting with the outside 6 | world is via a serial connection. However, it is still a "complete" system, in 7 | the sense that it has a kernel, an init system, and a decent userspace, so it 8 | suffices for the job of demonstrating various part's internal workings and functionality. 9 | 10 | This bare-minimum Linux system will be ran inside an emulated environment using 11 | the Quick Emulator (QEMU) because: 12 | 13 | 1. There really is no point to run it on physical hardware 14 | 2. All kinds of hardware setups exist, and this mini system probably won't 15 | run smoothly on all of them. Using an emulator makes the environment much 16 | more controlled and eliminates the issue of incompatible hardware. 17 | 18 | In the next section, we will expand upon this and transform the bare minimum 19 | system into a more useful one that can boot on real hardware. 20 | -------------------------------------------------------------------------------- /wiki/4_beyond/1_kernel/1_qemuHardware.md: -------------------------------------------------------------------------------- 1 | 2 | # Adapting the kernel to support various QEMU devices 3 | 4 | Previously, in the `3_miniSystem` section, we configured a minimum kernel that 5 | doesn't support any devices. As an exercise to familiarize you with the task 6 | of configuring a kernel, we will now attempt to get Microdot to support all the 7 | virtual devices provided by `QEMU`. 8 | 9 | A quick search through [wikipedia](https://en.wikipedia.org/wiki/QEMU#x86) 10 | and the man pages reveals a list of features that `qemu-system-x86_64` provides: 11 | 12 | * PCI Host Bridge: Intel 440FX 13 | * PCI IDE interface with hard disk and CD-ROM support 14 | * PCI Network Adapter E1000 15 | * PCI UHCI USB controller 16 | * Cirrus CLGD 5446 PCI VGA card 17 | * SMP, up to 255 CPUs 18 | * etc... 19 | 20 | We will now go throuh each one of these, explain what they do, and configure 21 | our kernel to support them. I am assuming that you followed the instructions 22 | in `wiki/3_miniSystem/3_miniKernel.md` and configured the kernel accordingly 23 | before reading this document. 24 | 25 | Note that very basic command and instructions, such as changing directory into 26 | the Linux kernel source folder or opening up another terminal as a normal 27 | user and launching QEMU with it, are not written. How to do such things are 28 | covered in documents in previous sections. Also, execute all commands in this 29 | section as the builder user, unless the docs specifically say to use another 30 | user, or it's launching QEMU to test the system. 31 | 32 | 33 | 34 | # Adding support for the PCI bus and the ACPI subsystem 35 | 36 | You might have noticed that when we issue the `poweroff` command in our mini 37 | Linux system, the system halts, but the QEMU process does not exit, as if it 38 | doesn't know the system has stopped running. This is the case because our Linux 39 | system has no way of telling the hardware that the system is going down. We 40 | will add support for the ACPI subsystem to allow our system to communicate 41 | with the various hardware components available. ACPI is generally present on all 42 | modern PC hardware. 43 | 44 | The PCI bus is a generic extension bus used by many PC systems, and many 45 | hardware devices are attached to the system via the PCI bus, such as hard disk 46 | controllers, network cards, USB controllers, etc. The ACPI subsystem will 47 | be automatically enabled if you enable PCI support in the kernel. 48 | 49 | ```text 50 | Bus options (PCI etc.) 51 | PCI support: Y 52 | ``` 53 | 54 | Compile the kernel, copy the `bzImage` to `/opt`, and Run QEMU. When you boot 55 | into the system, typing the `lspci` command should show you a list of PCI 56 | devices. `Busybox lspci` only shows the device ID and no human readable names, 57 | unlike the normal `lspci` that you would find in many Linux distros. You can 58 | search up the IDs online and find out the various devices detected on the PCI 59 | bus. 60 | 61 | If you type `poweroff`, the system will shutdown, QEMU will exit, and return you 62 | to the terminal prompt, during which you will see a message from `ACPI` saying 63 | that its preparing the system for a sleep state. 64 | 65 | 66 | # Adding support for the networking subsystem & specific network devices 67 | 68 | In order to use the ethernet card provided by QEMU, we will first need to add 69 | support for the networking subsystem, which contains code that support sockets, 70 | TCP/IP, 71 | 72 | ```text 73 | Networking support: Y 74 | Networking options: 75 | Unix domain sockets: Y 76 | TCP/IP networking: Y 77 | Wireless: N # we have no wireless networking device in QEMU 78 | ``` 79 | 80 | Besides that, we also need to include support for the specific network card 81 | hardware that we are provided with. If you look in the QEMU man pages, you can 82 | see that this would be the "e1000" network card. We will use the `SymSearch` 83 | function provided by `nconfig` to search for `e1000`. Launch nconfig, and press 84 | `F8`, type in `e1000`, and press enter. The displayed message should look 85 | something like this: 86 | 87 | ```text 88 | Symbol: E1000 [=n] 89 | Type : tristate 90 | Prompt: Intel(R) PRO/1000 Gigabit Ethernet support 91 | Location: 92 | -> Device Drivers 93 | -> Network device support (NETDEVICES [=n]) 94 | -> Ethernet driver support (ETHERNET [=n]) 95 | -> Intel devices (NET_VENDOR_INTEL [=n]) 96 | ``` 97 | 98 | The above search results shows us the location of the config option for 99 | `Intel(R) PRO/1000 Gigabit Ethernet support`. We can now add support for the 100 | `e1000` ethernet card, and remove the support for a few other unnecessary 101 | things. 102 | 103 | > Note: Once you are in the `Ethernet driver support` menu, you should notice 104 | > that support for a lot of hardware vendors are included. Uncheck everything, 105 | > except for the Intel devices specified below. 106 | 107 | ```text 108 | Device Drivers: 109 | Network device support: Y 110 | Ethernet driver support: Y 111 | Intel (82586/82593/82596) devices: N 112 | Intel devices: Y 113 | Intel(R) PRO/1000 Gigabit Ethernet support: Y 114 | ``` 115 | 116 | Save, exit, and compile, then launch QEMU. After the system has booted, type 117 | in the `ip link` command, and you should see the `eth0` interface present. 118 | We will cover network configuration in another section. 119 | 120 | # Symmetric Multi-Processing (SMP) 121 | 122 | This feature allows our kernel to utilize more than 1 CPU cores. Personal 123 | computers commonly ships with 4 cores or even 8 cores these days, and even 124 | embedded systems like the Raspberry Pi has a quad-core cpu, so there's really 125 | no reason not to enable this feature. Go into `nconfig` again and configure 126 | the following: 127 | 128 | ```text 129 | 130 | Processor type and features: 131 | Symmetric multi-processing support: Y 132 | 133 | # each CPU supported adds 8KB to the kernel image. 134 | # we will probably never have 64 CPUs on a system 135 | Maximum number of CPUs: 8 136 | ``` 137 | 138 | -------------------------------------------------------------------------------- /wiki/4_beyond/readme.md: -------------------------------------------------------------------------------- 1 | # Content Overview 2 | 3 | > Note: this section will expand as I and other users add more tutorials. 4 | > For now, it only has a skeleton of what could be included in the future 5 | 6 | ### Kernel 7 | * Exercise: tuning the kernel to support all hardware emulated by QEMU 8 | * Generating and using kernel modules 9 | 10 | ### Booting & Init Systems 11 | * Exercise: installing Syslinux as the bootloader 12 | * Exercise: generating a bootable CD-ROM iso image 13 | * Exercise: converting the CD-ROM iso image into a hybrid image 14 | * Using runit as the init system 15 | * How to `pivot_root` from an initramfs to a real hard drive 16 | * Overlay file systems 17 | 18 | ### Toolchain 19 | * Toolchain targeting the ARM architecture 20 | * Using pre-built musl toolchains 21 | * Using the musl-cross-make script to make toolchains 22 | 23 | ### Porting Microdot 24 | * installing Microdot on a Raspberry Pi 25 | * installing Microdot on a Allwinner H3 NanoPi 26 | 27 | ### Software 28 | * List of tiny utilities to have in Microdot 29 | * How to install more programs 30 | * Using the (experimental) `tpm` package manager 31 | * Setting up TinyX 32 | * Setting up Xorg 33 | -------------------------------------------------------------------------------- /wiki/4_beyond/unusedDocs/1_dir.md: -------------------------------------------------------------------------------- 1 | 2 | # Creating Files & Directories 3 | 4 | Before proceeding onto installing software using our cross toolchain, we will 5 | first create the skeleton of our Microdot system. That is, we need to create 6 | a few important directories that will be populated later. We will name these 7 | folders according to the established convention. The skeleton will be built in 8 | the `$targetfs` directory. The contents of this folder will later be copied 9 | onto a bootable disk image. 10 | 11 | For more details regarding the Linux directory structures, check out the 12 | following links: 13 | 14 | * [a](https://www.a.com) 15 | * [b](https://www.b.com) 16 | * [d](https://www.c.com) 17 | 18 | 19 | ```bash 20 | # essential directories 21 | mkdir -p $targetfs/{bin,sbin,dev,etc,proc,sys} 22 | mkdir -p $targetfs/{mnt,opt,home,usr/{lib,bin,sbin}} 23 | mkdir -p $targetfs/var/{lib,spool,lock,cache,log,run} 24 | 25 | # link all "lib" and "lib64" folders to "usr/lib" 26 | ln -sfv $targetfs/usr/lib $targetfs/lib 27 | ln -sfv $targetfs/usr/lib $targetfs/lib64 28 | ln -sfv $targetfs/usr/lib $targetfs/usr/lib64 29 | 30 | # Create these directories with special permissions 31 | install -dm 0750 $targetfs/root 32 | install -dm 1777 $targetfs/tmp 33 | install -dm 1777 $targetfs/var/tmp 34 | ``` 35 | 36 | Now we will create a few important files: 37 | 38 | ```bash 39 | ln -sfv ../proc/mounts $targetfs/etc/mtab 40 | 41 | cat > $targetfs/etc/passwd << "EOF" 42 | root::0:0:root:/root:/bin/ash 43 | EOF 44 | 45 | cat > $targetfs/etc/group << "EOF" 46 | root:x:0: 47 | bin:x:1: 48 | EOF 49 | 50 | cat > ${CLFS}/targetfs/etc/fstab << "EOF" 51 | # file-system mount-point type options dump fsck 52 | EOF 53 | 54 | touch $targetfs/var/log/lastlog 55 | chmod -v 664 $targetfs/var/log/lastlog 56 | ``` 57 | -------------------------------------------------------------------------------- /wiki/4_beyond/unusedDocs/2_install.md: -------------------------------------------------------------------------------- 1 | # Installing Basic Software 2 | 3 | In this section we will install the essential system software programs into 4 | `$targetfs`, so the system can actually be used later. 5 | 6 | ## Clean Up the Source Directory 7 | 8 | We've finished constructing the cross toolchain, and now we will remove the 9 | directories extracted previously so we can work in a clean environment: 10 | 11 | ```bash 12 | cd $install/src 13 | rm -rf binutils-2.30 gcc-7.3.0 linux-4.18.5 musl-1.1.20 14 | ``` 15 | 16 | ## libgcc 17 | 18 | Install the `libgcc` support library, so other dynamic binaries compiled by 19 | `gcc` can actually run on our final system. 20 | 21 | ```bash 22 | tar -xf ../mpfr* 23 | tar -xf ../mpc* 24 | tar -xf ../gmp* 25 | 26 | mv mpfr* mpfr 27 | mv gmp* gmp 28 | mv mpc* mpc 29 | 30 | mkdir build 31 | cd build 32 | 33 | ../configure \ 34 | --prefix=$targetfs/usr \ 35 | --host=$target \ 36 | --target=$target \ 37 | --disable-multilib \ 38 | --disable-bootstrap \ 39 | --disable-nls \ 40 | --disable-libmpx \ 41 | --disable-libsanitizer \ 42 | --disable-mudflap \ 43 | --enable-c99 \ 44 | --enable-long-long \ 45 | --enable-tls \ 46 | --enable-languages=c,c++ 47 | 48 | make -jN all-target-libgcc 49 | make install-target-libgcc 50 | 51 | $target-strip $targetfs/lib/libgcc_s.so.1 52 | rm -r $targetfs/lib/gcc 53 | ``` 54 | 55 | # musl-libc 56 | 57 | Install musl-libc onto the target system, so dynamically linked binaries can 58 | run. 59 | 60 | ```bash 61 | ./configure \ 62 | CROSS_COMPILE=$target- \ 63 | --prefix=/ \ 64 | --disable-static \ 65 | --target=$target 66 | 67 | make -j4 68 | make DESTDIR=$targetfs/usr install-libs 69 | ``` 70 | 71 | # busybox 72 | 73 | Busybox is the main component of Microdot's userspace. It contains hundreds of 74 | Linux utilities combined into one tiny executable. Linking it with `musl-libc` 75 | results in an even smaller binary compared to other libraries. 76 | 77 | ```bash 78 | make distclean 79 | make ARCH=$arch defconfig 80 | make -jN ARCH=$arch CROSS_COMPILE=$target- CONFIG_PREFIX=$targetfs install 81 | ``` 82 | 83 | 84 | 85 | 86 | 87 | 88 | 89 | -------------------------------------------------------------------------------- /wiki/4_beyond/unusedDocs/3_miniKernel.md: -------------------------------------------------------------------------------- 1 | # Configuring a Minimal Kernel 2 | 3 | ```bash 4 | # clean up the source directory 5 | make distclean 6 | 7 | # configure a kernel with minimum features enabled 8 | make ARCH=$arch CROSS_COMPILE=$target- tinyconfig 9 | 10 | # launch the configuration interface 11 | make ARCH=$arch CROSS_COMPILE=$target- nconfig 12 | ``` 13 | 14 | Now you should be seeing a text based configuration interface. You can navigate 15 | around using the arrow keys, ESC key, and return key. Every indentation in the 16 | recipe means that you need to enter a sub-menu to access the options. "Y" means 17 | that you need to select the corresponding option ("N" means deselect). Again, 18 | a hashtag ("#") indicates a comment. 19 | 20 | ``` 21 | 22 | 64-bit kernel: Y 23 | 24 | General Setup 25 | Default hostname: Microdot 26 | 27 | Initial RAM filesystem and RAM disk (initramfs/initrd) support: Y 28 | Support initial ramdisk/ramfs compressed using bzip2: N 29 | Support initial ramdisk/ramfs compressed using LZMA: N 30 | Support initial ramdisk/ramfs compressed using XZ: N 31 | Support initial ramdisk/ramfs compressed using LZO: N 32 | Support initial ramdisk/ramfs compressed using LZ4: N 33 | 34 | Configure standard kernel features (expert users) 35 | Multiple users, groups and capabilities support: Y 36 | Posix Clocks & timers: Y 37 | 38 | # printk() is what the kernel uses to print messages onto the screen 39 | Enable support for printk: Y 40 | 41 | Executable file formats / Emulations 42 | Kernel support for ELF binaries: Y 43 | Kernel support for scripts starting with #!: Y 44 | 45 | Device Drivers 46 | Generic Driver Options 47 | Maintain a devtmpfs filesystem to mount at /dev: Y 48 | Automount devtmpfs at /dev, after the kernel mounted the rootfs: Y 49 | Character devices 50 | Enable TTY: Y 51 | Virtual terminal: N 52 | Unix98 PTY support: N 53 | Legacy (BSD) PTY support: N 54 | Serial drivers 55 | 8250/16550 and compatible serial support: Y 56 | Console on 8250/16550 and compatible serial port: Y 57 | 58 | File systems 59 | Enable POSIX file locking API: Y 60 | Pseudo filesystems 61 | /proc file system support: Y 62 | sysfs file system support: Y 63 | 64 | ``` 65 | 66 | Save, exit, and compile the kernel. The finished image should be around 750kB. 67 | If we used `make defconfig` instead of `make tinyconfig` and hand-tuning the 68 | kernel, we would end up with a 8MB image, which contains a plethora of bloat 69 | that we'll never use. `make defconfig` is what most tutorials out there on the 70 | internet instructs you to do. 71 | 72 | ```bash 73 | make ARCH=$arch CROSS_COMPILE=$target- -jN 74 | 75 | # copy the image to /opt 76 | cp arch/x86/boot/bzImage /opt 77 | ``` 78 | 79 | In the next section, we will pack the content of targetfs into a `initramfs`, 80 | and boot our Microdot system. 81 | 82 | 83 | 84 | 85 | 86 | 87 | 88 | 89 | 90 | 91 | -------------------------------------------------------------------------------- /wiki/readme.md: -------------------------------------------------------------------------------- 1 | # Microdot Wiki Overview 2 | 3 | - Section 1: prerequisites, background information, and build environment setup 4 | - Section 2: constructs the musl-based cross compilation toolchain 5 | - Section 3: builds a minimum Linux system to demostrate how various parts 6 | of the system work together 7 | - Section 4: tutorials for more specific topics, such as kernel fine tuning, 8 | porting Microdot to other architectures, installing other programs, etc. 9 | This section will probably contain a lot of work contributed by other users. 10 | For more details, see the `readme.md` file inside `4_beyond`. 11 | --------------------------------------------------------------------------------