├── README.md └── text ├── 0000-bootstrap.md ├── 0001-architecture.md ├── 0002-asides.md ├── 0003-book-participation.md ├── 0004-supplemental-book.md └── 0005-vga-console.md /README.md: -------------------------------------------------------------------------------- 1 | # intermezzOS RFCs 2 | 3 | This is a “request for comments” repository for intermezzOS. Here’s the idea: 4 | every design choice in intermezzOS must be justified through writing an RFC, 5 | before it’s accepted that this is the path that we’ll take. 6 | 7 | “Design choice” is a bit vague, but it’s basically any high-level choice. This 8 | isn’t “every PR must be justified” but any sort of “Why is intermezzOS like 9 | $x?” question should probably have an answer here. 10 | 11 | ## Process 12 | 13 | To initially discuss a topic, open an issue. We’ll discuss things in those 14 | issues. 15 | 16 | When a proposal is ready to be made, submit a pull request adding a new file in 17 | the `text` directory. We’ll discuss things in the PR, and then either merge or 18 | not. 19 | -------------------------------------------------------------------------------- /text/0000-bootstrap.md: -------------------------------------------------------------------------------- 1 | # An RFC process for intermezzOS 2 | 3 | Every design choice in intermezzOS must be justified through writing an RFC, 4 | before it’s accepted that this is the path that we’ll take. 5 | 6 | “Design choice” is a bit vague, but it’s basically any high-level choice. This 7 | isn’t “every PR must be justified” but any sort of “Why is intermezzOS like 8 | $x?” question should probably have an answer here. 9 | 10 | ## Details 11 | 12 | To initially discuss a topic, open an issue. We’ll discuss things in those 13 | issues. 14 | 15 | When a proposal is ready to be made, submit a pull request adding a new file in 16 | the `text` directory. We’ll discuss things in the PR, and then either merge or 17 | not. 18 | 19 | ## Drawbacks 20 | 21 | This is a bit more heavyweight than just doing whatever I feel like doing. 22 | 23 | ## Alternatives 24 | 25 | Not have an RFC process. This is a pretty valid alternative, given that I’m the 26 | only one working on this project for now. However: 27 | 28 | 1. I don’t have a good place to collect research on various topics as I make my 29 | decisions. 30 | 2. It might be easier to not have one, but I’ve found RFCs so useful in Rust that 31 | I am willing to bet that I wish I had done this previously. 32 | 3. RFCs allow for newcomers to learn about what design decisions have been made, and why. 33 | -------------------------------------------------------------------------------- /text/0001-architecture.md: -------------------------------------------------------------------------------- 1 | # Architecture 2 | 3 | intermezzOS will support `x86_64` only. 4 | 5 | ## Details 6 | 7 | There’s a basic tradeoff for any hobby kernel: write it for ‘the real world’ 8 | of computers you probably own, or write it in something that’s easier. 9 | Unfortunately, `x86` as a family is full of all kinds of odd legacy bullshit, 10 | and newer architectures are much nicer. 11 | 12 | However, it’s magical to actually boot up your ‘real’ computer with an OS you 13 | made. To me, this is more important than just being ‘easier’. We can learn 14 | the hard stuff! 15 | 16 | Finally, I have significantly more experience on `x86_64` than anything else. 17 | 18 | ## Drawbacks 19 | 20 | Other architectures can be a lot easier, especially with the 64-bit variant of 21 | `x86`. 22 | 23 | I think that this drawback isn’t that big of a deal, and that the advantages 24 | are larger. 25 | 26 | ## Alternatives 27 | 28 | There are two broad categories here: pick an initial different architecture, or 29 | support multiple architectures. 30 | 31 | ### Different? 32 | 33 | I could pick something else. With the `x86` family, sticking to 32 bit would 34 | be similar, but easier. I could also choose ARM, MIPS, or something else. 35 | 36 | ### Multiple 37 | 38 | This is still an option in the future, and is easier to do then rather than 39 | now. It’s possible that I might make a choice which makes porting difficult; 40 | that should be a consideration for future RFCs. 41 | -------------------------------------------------------------------------------- /text/0002-asides.md: -------------------------------------------------------------------------------- 1 | # Asides 2 | 3 | IntermezzOS the book will visually separate asides from the rest of the text. 4 | 5 | ## Details 6 | 7 | Extra tidbits of information, "fun facts", or little asides are great for increasing 8 | the reader's attention and engagement. Some information is useful only as context or 9 | is just plain fun to learn about even if not 100% relevant. 10 | 11 | The book already includes many asides inline which can make finding more "important 12 | information" harder and may prevent some readers (especially those who are 13 | reading a section for second or third time) from finding relevant information. 14 | 15 | Therefore, asides should be visually separated from the rest of the text and 16 | marked as asides. 17 | 18 | ## Drawbacks 19 | 20 | Asides are a great way to "slow things down" when things are getting harder to 21 | understand. Without them, other devices will need to be used to keep the right pace. 22 | 23 | Also, writers must always remember to appropriately format the text they are 24 | writing. 25 | 26 | ## Alternatives 27 | 28 | Two alternatives: keep as is or remove all asides. 29 | 30 | ### Keep as is 31 | 32 | Asides are not a "problem" per say, but the advantages of keeping them separate 33 | outweigh the disadvantages. 34 | 35 | ### Remove all asides 36 | 37 | How boring! 38 | -------------------------------------------------------------------------------- /text/0003-book-participation.md: -------------------------------------------------------------------------------- 1 | # Participation in writing the book 2 | 3 | This RFC aims to lay out how participating in authoring the book works. 4 | 5 | ## Details 6 | 7 | I would like for the book to not be solely written by myself. That said, when 8 | I started this project, I wasn’t sure if anyone would even want to help out, 9 | and I had a plan. Including others means that plan may change. 10 | 11 | Specifically, here is my proposal: 12 | 13 | * There are two groups of contributors to the book: editors and authors. 14 | * All editors are authors, but not all authors are editors. 15 | * Editors have a commit bit, authors must send in PRs. 16 | * The editorial board can change over time. 17 | 18 | ### Authors 19 | 20 | Anyone may be an author of the book. Authors: 21 | 22 | * Tweak and modify the text of the book. 23 | * Collaborate with editors to produce rough drafts of new chapters. 24 | 25 | There’s two kinds of contributions, book-wise, and these two responsibilities 26 | are reflected in these two points. There’s editing existing text, and creating 27 | new text. These two cases are slightly different: 28 | 29 | #### Tweak and modify 30 | 31 | The book will need a _lot_ of refinement over time. Authors will be a big hand 32 | in this. Submitting improvments to our existing text is really, really helpful. 33 | 34 | These changes will go through the PR process and will be accepted by editors. 35 | 36 | #### New Chapters 37 | 38 | Until we have a complete first draft of the book, there will need to be initial 39 | drafts for chapters. Having someone contribute these would be great! However, 40 | they need to fit in with the overall book. It’s hard to keep consistent tone 41 | amongst a group of authors, though. As such, editors will have to help out 42 | authors a lot to make sure that the book is coherent overall. 43 | 44 | There are two forms of coherence: the tone and style of the text itself, and a 45 | chapter fitting into the overal vision of the work. New chapters need to be on 46 | the kind of material we need and want. The text of the chapter needs to be 47 | similar to the other chapters. 48 | 49 | As such, an author that wishes to write a new chapter should check with the 50 | editors by opening an issue about it, so that they don’t do a lot of work that 51 | may end up rejected. Furthermore, the editors may ask for significant revisions 52 | of the draft before it’s merged to make everything consistent overall. 53 | 54 | ### Editors 55 | 56 | Editors are primarily responsible for the overall quality of the book. 57 | 58 | The editorial team should work to produce an table of contents, so that authors 59 | can contribute chapters. If problems arise with the initial table of contents, 60 | editors should work together to revise it to solve these problems. 61 | 62 | Editors will review pull requests from authors and make sure that they’re of 63 | a high enough quality before merging. 64 | 65 | Editors are also authors. When acting as an author, they must still act in 66 | accordance with those roles: they should still submit content through the pull 67 | request process. Editorship is about being able to sign off on changes, not 68 | just push whatever changes you want. 69 | 70 | #### Editorial team 71 | 72 | The initial editorial team will be: 73 | 74 | * Steve Klabnik 75 | * Ryan Levick 76 | 77 | The team may add and remove editors at their discretion. Adding a member 78 | requires consensus, and removing a member requires consensus amongst all of the 79 | editors other than the individual being removed. 80 | 81 | ## Drawbacks 82 | 83 | Working with multiple authors in a distributed fashion is a lot more work. 84 | This drawback is mitigated by increased quality. 85 | 86 | ## Alternatives 87 | 88 | I could say “I’m the sole author of the book, sorry.” That would certainly 89 | simplify some things. However, that means that this is more of a personal 90 | project of mine, and less of a community one, and I would like to culitivate a 91 | community around intermezzOS. I also think it would lead to a significantly 92 | worse book. 93 | -------------------------------------------------------------------------------- /text/0004-supplemental-book.md: -------------------------------------------------------------------------------- 1 | # Computer Architecture Book 2 | 3 | Operating Systems Development requires knowledge of low level computer concepts 4 | such as how CPUs, RAM, and IO devices work. The intermezzOS book touches on 5 | these concepts only when learning about them is necessary to be able to build a 6 | basic OS. The book doesn't linger too long on a specific subject, and only tries 7 | to teach the reader "just enough" to help them understand the task at hand. But 8 | if the reader wants to learn more, where should they go? 9 | 10 | This is a proposal for a supplemental book on the topic of computers at the 11 | lowest level. The book will try to teach the student the wide range of topics 12 | associated with computer architecture that are essential for being able to 13 | understand computers and operating systems on a deep level but aren't 100% necessary 14 | for the task of actually building a basic OS. 15 | 16 | These topics include: 17 | * Assembly Programming 18 | * Von Neumann Architecture 19 | * CPU Architecture and ISA 20 | * Stack and Heap memory 21 | * Virtual Memory 22 | * etc. 23 | 24 | The book should not be thought of as a "learn to program assembly language" 25 | tutorial. Instead, assembly will be used as a guide introducing the reader to new 26 | topics and helping them "get their hands dirty". 27 | 28 | ## Drawbacks 29 | 30 | The intermezzOS project already has one book. Adding another book adds more 31 | responsibility for the community to handle. Because teaching readers about computers 32 | on a low level and teaching them to build a basic OS have a lot in common, 33 | material may overlap between the two books. The question of when one book 34 | ends and the other begins may be difficult to answer. 35 | 36 | ## Alternatives 37 | 38 | Of course, we can choose not write another book. Any topics that would have gone 39 | into this book are instead covered in appendix chapters, covered as asides, addressed 40 | through references to external blog posts or left out entirely. Those who are thirsty 41 | for more will have to quench their thirst elsewhere. 42 | -------------------------------------------------------------------------------- /text/0005-vga-console.md: -------------------------------------------------------------------------------- 1 | # VGA Console 2 | 3 | The VGA spec and the guaranteed presence of VGA hardware on any machine or VM 4 | intermezzOS targets provide convenient methods of printing information for us to 5 | observe the kernel in action. 6 | 7 | ## Details 8 | 9 | The VGA spec says that the magic address `0xB8000` is the start of video memory, 10 | and represents the top left cell on the screen. The screen is 80 columns wide, 11 | 25 rows high, and each column is two bytes. In total, VGA video memory is 4000 12 | bytes (just under 4K). 13 | 14 | ### Overview 15 | 16 | At present, intermezzOS' VGA implementation consists of a struct that owns a 17 | VGA-sized buffer **not** at the magic location, the VGA memory that is at the 18 | magic location, and some minor internal state. 19 | 20 | The console structure uses the buffer to collect text until `flush()` is called, 21 | at which point the buffer is copied en masse to the video memory. This provides 22 | apparently-atomic updates to the screen, although this operation need not be 23 | *actually* atomic as (a) good design and Rust checks should guarantee only one 24 | VGA structure owns video memory, and (b) moving 4k of memory is a long time to 25 | not have any other interrupts. 26 | 27 | ### Proposed Design 28 | 29 | Philipp Oppermann suggests using a Unique pointer to wrap the magic memory 30 | location so that the compiler can guarantee that there is ever only one mutable 31 | owner of the video memory. I have not yet found a working way to magically 32 | assure Rust that the magic raw memory is the type we want -- I remember building 33 | this out of `core::ptr::from_raw_parts_mut()` in the workshop but have not been 34 | able to get it to work properly with any type other than `[u8; 4000]`. 35 | Regardless of *how* a VGA-memory-owner gets its memory, though, we will have to 36 | do magic to assure Rust that the raw memory at the VGA magic address is able to 37 | be used within the type system. 38 | 39 | I propose designing a VGA structure which owns only a single VGA-memory-sized 40 | slice, rather than the current implementation which gives one VGA structure two 41 | regions of memory. Since the magic address is the kind of implementation detail 42 | which should be hidden from consumers, the VGA structure will implement multiple 43 | constructors: the standard `new()` method will take a generic buffer, which can 44 | be allocated anywhere in normal memory (but as intermezzOS does not yet have a 45 | dynamic allocator, it will probably be in BSS), and an **unsafe** `new_at_vga()` 46 | method will create a VGA structure that has taken ownership of the video memory 47 | region. 48 | 49 | In order to achive the pseudo-atomic functionality currently present in our 50 | console implementation, the VGA structure will need to implement a method that 51 | will permit one structure to borrow another and copy the borrowed buffer into 52 | its own storage. When the VGA structure at video memory does this, we achieve 53 | pseudo-atomic video update, and we will also be able to rapidly change display 54 | contexts by switching between multiple different VGA structures in main memory. 55 | 56 | Incidentally, since VGA-memory structures are 4000-byte slabs of memory with no 57 | other special considerations, we can implement Clone and Copy for the atomic 58 | units (each cell of the grid), and thus for each row and for the entire grid. 59 | This means that such a borrow-for-copy method would simply look something like 60 | this: 61 | 62 | ```rust 63 | impl VgaMemory { 64 | // ... 65 | fn copy_from_other(&mut self, other: &VgaMemory) { 66 | // do whatever magic needs be done to access the actual buffer 67 | self.buffer = other.buffer; 68 | // assignment uses Copy trait to memcpy 4000 bytes from other to self 69 | } 70 | } 71 | ``` 72 | 73 | No loops or bounds checking need be done by us; everything can be verified at 74 | compile-time thanks to Rust's intrinsic type- and bounds- checking. Valid 75 | `VgaMemory` (or however this gets implemented) instances will be guaranteed to 76 | own memory of the appropriate size and type, so this doesn't even need to use 77 | unsafe code (at this level) in any way. 78 | 79 | ### Other Considerations 80 | 81 | From the intermezzOS workshop and from personal experience, I noticed that 82 | forgetting some details of the VGA buffer's size was a common source of bugs. 83 | Mixing up rows vs columns, or forgetting that each cell was two bytes wide not 84 | one, leads to some interesting learning opportunities in debugging. 85 | 86 | Some of the problems with C that Rust aims to overcome are type safety and 87 | buffer overruns. Working with video memory provides opporunities to showcase 88 | Rust's power in both regards. 89 | 90 | While we obviously can continue to treat VGA memory as a single chunk of 4000 91 | `u8`s, I feel that doing so offers no benefit to writing intermezzOS in C. Mr. 92 | Oppermann's tutorial defines a character struct two bytes wide that internally 93 | handles ordering (little- vs big- endianness is a frequent bug) and sizing, so 94 | the screen can be represented in terms of these two-byte character structs 95 | rather than individual bytes, eliminating the need to remember to multiply the 96 | memory size by two. 97 | 98 | Furthermore, while memory itself is clearly one continuous stream of cells, 99 | representing a 2D grid on that requires some mathematics. At present, the VGA 100 | structure requires that the programmer remember to do division and modulo work 101 | themselves to map between cells on screen and memory locations. However, if the 102 | VGA structure were to represent video memory as slices of slices of cell 103 | structs, then Rust could provide firm guarantees about not overflowing 104 | individual rows or the buffer as a whole. 105 | 106 | (A benefit to this approach I noted was that my tests for wrapping long lines 107 | would fail when trying to overrun a single row, rather than letting me write 108 | into the next row without taking appropriate action). 109 | 110 | ## Advantages 111 | 112 | - Simplify the design of the VGA structure and implementation 113 | - Isolate magic numbers such as the video memory address 114 | - Use Rust type-checking to make screen cells, not bytes, the atomic unit of the 115 | console 116 | - Use Rust bounds-checking to ensure that the console is correctly manipulated, 117 | both in entirety and also for individual rows 118 | - Simplify future development plans, such as rapidly switching the display 119 | between multiple display-sized buffers (multiple consoles, full-screen contexts) 120 | - Demonstrate the power of Rust abstractions over hardware, without the cost 121 | such abstractions usually impose 122 | - More thorough unit testing 123 | 124 | ## Disadvantages 125 | 126 | - Would require a nearly full rewrite of the VGA console 127 | - Increased complexity causes confusion about different things 128 | - at present, confusion stems from the magic numbers and byte ordering 129 | - with this, confusion would stem from the complex structure and design 130 | - Performance penalty? Memcpy is memcpy, regardless of ownership, but maybe 131 | decoupling might cause some extra overhead 132 | - More complex unit/integration testing 133 | 134 | ## Potential Pain Points 135 | 136 | ### How do we point at VGA memory? 137 | 138 | Speaking entirely from personal experience, the Unique pointer mechanism is a 139 | bit of an educational ride. When I first saw it, I assumed from the name that 140 | Rust would verify at compile time that only one Unique pointer to a memory hunk 141 | existed at a time. It turns out this doesn't happen; Unique is paired with a 142 | Shared pointer type and simply means "this pointer has mutable access" while a 143 | Shared pointer type will not allow mutable access. See its [documentation][docu] 144 | and [issue][ghu]. Unique should really be named Owned, because it signifies that 145 | it has full ownership of the memory to which it points, and a Mutex would be 146 | used at runtime to guarantee synchronized access. IntermezzOS already has a 147 | Mutex in use in the current kernel source, but this is not discussed in the book 148 | yet. 149 | 150 | Constructing a pointer from raw parts is also an option, but in my personal 151 | experience and opinion much more prone to bugs. Whether we use Unique (an 152 | unstable, slightly misleading, and not-yet-standardized feature) or construction 153 | from raw parts (a manual, difficult, means of getting the compiler to whine at 154 | us) is a question worth considering regardless of how else we structure the VGA 155 | design. 156 | 157 | ### How do we handle safety and testing? 158 | 159 | Giving Rust access to the magic memory address is always going to be (a) unsafe 160 | and (b) impossible to test except in production. 161 | 162 | - unsafe: No matter what method we choose for doing so, we will wind up having 163 | to convince Rust to just give use the VGA magic memory, and damn the rules. This 164 | is, obviously, going to require `unsafe` functions. In my experimental 165 | implementations, I have elected to use unsafe functions, rather than unsafe 166 | blocks inside functions, for the instantiation at magic memory, like so: 167 | 168 | ```rust 169 | impl VgaScreen { 170 | unsafe fn new_at_vga() -> VgaScreen { 171 | VgaScreen { 172 | // however we trick Rust into giving us 4000 bytes at 0xB800; 173 | } 174 | } 175 | } 176 | impl VgaWriter { 177 | unsafe fn new_at_vga() -> VgaWriter { 178 | VgaWriter { 179 | // other fields omitted... 180 | screen: VgaScreen::new_at_vga(), 181 | } 182 | } 183 | } 184 | 185 | // Safe -- uses non-special memory allocated by the compiler and linker 186 | let some_writer: VgaWriter = VgaWriter::new(); 187 | // Unsafe -- uses the magic memory location 188 | let main_writer: VgaWriter = unsafe { VgaWriter::new_at_vga() }; 189 | ``` 190 | 191 | This provides a clear visual cue when instantiating an owner of magic memory, 192 | that they are using magic memory instead of general memory and that doing so can 193 | come back to haunt them if they're not careful, such as by making two writers on 194 | magic memory. Requiring explicit acknowledgement of unsafety by the consumer is, 195 | in my opinion, both educational and desirable. 196 | 197 | - segfaulting: A real operating system will never grant access to physical 198 | memory at the VGA address, and the test suite can't guarantee that it will own 199 | enough virtual memory for that address to function as a virtual address, or even 200 | if that address is in scope, that it won't belong to some other part. 201 | 202 | So `cargo test` can't test any code using `new_at_vga()`, because it will always 203 | segfault. The only way to test that the implementation runs properly is by 204 | ensuring `new_at_vga()` functions are the ONLY functions touching magic memory, 205 | and that the structures we design function identically regardless of where it is 206 | they own a buffer. Testing against `new()` and all high-level uses of the VGA 207 | structs must work correctly, and then using `new_at_vga()` in the compiled VM 208 | should successfully manipulate video memory. 209 | 210 | This will, I hope, wind up as an encouragement of automated testing protocols. 211 | There are only two ways to give confident assurances that literally-untestable 212 | code will function correctly, and one of them requires a lot of formal math. 213 | The other is using good design and automated tests wherever else possible. 214 | IntermezzOS already encourages and uses such a plan, so this should be somewhat 215 | painless. 216 | 217 | ## How do we clearly explain what's going on? 218 | 219 | Rust has a great tradition of documentation already; once a settled design is 220 | chosen, clean documentation and a chapter in the book should suffice just fine. 221 | 222 | [docu]: https://doc.rust-lang.org/core/ptr/struct.Unique.html 223 | [ghu]: https://github.com/rust-lang/rust/issues/27730 224 | --------------------------------------------------------------------------------