├── pictures ├── Sim.png ├── trap.png ├── mmu_trace.png ├── trap_spec.png ├── Cache_code.png ├── Cache_miss.png ├── Core_spike.png ├── Spike_main.png ├── cache_trace.png ├── hart_spike.png ├── instruction.png ├── trap_class.png ├── Bus_Add_device.png ├── Memory_system.png ├── Instruction_MMU.png ├── abstract_device_t.png ├── instruction_spec.png └── trap_define_class.png ├── LICENSE └── README.md /pictures/Sim.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/poweihuang17/Documentation_Spike/HEAD/pictures/Sim.png -------------------------------------------------------------------------------- /pictures/trap.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/poweihuang17/Documentation_Spike/HEAD/pictures/trap.png -------------------------------------------------------------------------------- /pictures/mmu_trace.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/poweihuang17/Documentation_Spike/HEAD/pictures/mmu_trace.png -------------------------------------------------------------------------------- /pictures/trap_spec.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/poweihuang17/Documentation_Spike/HEAD/pictures/trap_spec.png -------------------------------------------------------------------------------- /pictures/Cache_code.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/poweihuang17/Documentation_Spike/HEAD/pictures/Cache_code.png -------------------------------------------------------------------------------- /pictures/Cache_miss.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/poweihuang17/Documentation_Spike/HEAD/pictures/Cache_miss.png -------------------------------------------------------------------------------- /pictures/Core_spike.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/poweihuang17/Documentation_Spike/HEAD/pictures/Core_spike.png -------------------------------------------------------------------------------- /pictures/Spike_main.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/poweihuang17/Documentation_Spike/HEAD/pictures/Spike_main.png -------------------------------------------------------------------------------- /pictures/cache_trace.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/poweihuang17/Documentation_Spike/HEAD/pictures/cache_trace.png -------------------------------------------------------------------------------- /pictures/hart_spike.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/poweihuang17/Documentation_Spike/HEAD/pictures/hart_spike.png -------------------------------------------------------------------------------- /pictures/instruction.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/poweihuang17/Documentation_Spike/HEAD/pictures/instruction.png -------------------------------------------------------------------------------- /pictures/trap_class.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/poweihuang17/Documentation_Spike/HEAD/pictures/trap_class.png -------------------------------------------------------------------------------- /pictures/Bus_Add_device.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/poweihuang17/Documentation_Spike/HEAD/pictures/Bus_Add_device.png -------------------------------------------------------------------------------- /pictures/Memory_system.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/poweihuang17/Documentation_Spike/HEAD/pictures/Memory_system.png -------------------------------------------------------------------------------- /pictures/Instruction_MMU.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/poweihuang17/Documentation_Spike/HEAD/pictures/Instruction_MMU.png -------------------------------------------------------------------------------- /pictures/abstract_device_t.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/poweihuang17/Documentation_Spike/HEAD/pictures/abstract_device_t.png -------------------------------------------------------------------------------- /pictures/instruction_spec.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/poweihuang17/Documentation_Spike/HEAD/pictures/instruction_spec.png -------------------------------------------------------------------------------- /pictures/trap_define_class.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/poweihuang17/Documentation_Spike/HEAD/pictures/trap_define_class.png -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | BSD 3-Clause License 2 | 3 | Copyright (c) 2017, poweihuang17 4 | All rights reserved. 5 | 6 | Redistribution and use in source and binary forms, with or without 7 | modification, are permitted provided that the following conditions are met: 8 | 9 | * Redistributions of source code must retain the above copyright notice, this 10 | list of conditions and the following disclaimer. 11 | 12 | * Redistributions in binary form must reproduce the above copyright notice, 13 | this list of conditions and the following disclaimer in the documentation 14 | and/or other materials provided with the distribution. 15 | 16 | * Neither the name of the copyright holder nor the names of its 17 | contributors may be used to endorse or promote products derived from 18 | this software without specific prior written permission. 19 | 20 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 21 | AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 22 | IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 23 | DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE 24 | FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 25 | DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 26 | SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 27 | CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 28 | OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 29 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 30 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Tutuorial on Spike Internal 2 | 3 | 4 | Documentation editor: Po-wei Huang 5 | 6 | Acknowledgement 7 | ================== 8 | I would like to thank the following peoples for their time, feedback, and contribution:
9 | Wei Song 10 | 11 | 12 | Tutorial on Spike Internal 13 | ================== 14 | * [Goal of this document](#goal-of-this-document) 15 | * [Which branch is being tageted?](#which-branch-is-being-tageted) 16 | * [Overview of Spike](#overview-of-spike) 17 | * [Top Level Structure](#top-level-structure) 18 | * [What is modelled by Spike?](#what-is-modelled-by-spike) 19 | * [Spike's source code](#spikes-source-code) 20 | * [Memory System Overview](#Memory) 21 | * [What does Spike try to model?](#model_memory) 22 | * [TLB & MMU](#MMU_TLB) 23 | * [Spike's source code?](#source_MMU_TLB) 24 | * [Cache simulation](#Cache_sim) 25 | * [Spike's source code?](#source_cache) 26 | * [Result of cache simulation](#Result_of_cache) 27 | * [Processor Overview](#Processor) 28 | * [What does Spike try to model?](#model_processor) 29 | * [Hart modeling](#hart) 30 | * [What does Spike try to model?](#model_hart) 31 | * [Spike's implementation](#source_hart) 32 | * [Trap modeling](#trap) 33 | * [What does Spike try to model?](#model_trap) 34 | * [Spike's implementation](#source_trap) 35 | * [Interrupt modeling](#interrupt) 36 | * [Exception modeling](#interrupt) 37 | * [Bus and Miscellaneous devices](#bus) 38 | * [Device Simulation](#device_sim) 39 | * [Appendix](#appendix) 40 | * [Dealing with Instructions](#Instruction) 41 | 42 | Goal of this document 43 | ----------------- 44 | * Let people understand the implementation of Spike. 45 | * Work with Spike to help people understand RISC-V more as Spike is a golden reference 46 | * Provide information about how to use the spike, especially those features that are in the code but not well known to people. Ex. cache simulation, multi-core simulation. 47 | 48 | As Spike is a functional simulator, the simulator structure would not necessarily match the hardware structure. In order to make simulation faster, sometimes simulator optimization will be used, and these optimization will make the structure completely different. We will try to point out these difference when we meet them. 49 | 50 | Which branch is being tageted? 51 | ----------------- 52 | 53 | This tutorial is for branch master from the RISC-V ISA SIM repo and the commit is [daaf28f](https://github.com/riscv/riscv-isa-sim/tree/daaf28f7296c0a5f5c90fe6646a4f8a73a720af5). 54 | 55 | Overview of Spike 56 | ----------------- 57 | 1. Spike is an ISS (instruction set simulator), which is not cycle accurate. 58 | 2. Spike is a function simulator which omits all internal delays such as cache misses, memory transactions, IO accesses. 59 | 3. Spike does not have a full cache model, instead, the cache is a tracer or monitor (It doesn't allocate a space to cache any data). 60 | 61 | Top Level Structure 62 | ----------------- 63 | 64 | ### What is modelled by Spike? 65 | 66 | For spike, they use a multi-core framework. Each core includes a MMU for virtual memory, and all of the core have a common I$ and D$. Then, both I$ and D$ connect to a single L2$. The main memory follows. 67 | 68 | 69 | The cores and the memory hierarchy are inside a class sim, and the class could interact with outside by interactive command. Moreover, the sim includes bus, debug module, boot rom, and real time clock (RTC) . The processors, boot ROM, debug module and RTC are hooked on the bus, but the memory is not. These components together enable spike to run a simple proxy kernel pk. 70 | 71 | ![Top level overview](./pictures/Sim.png) 72 | 73 | ### Spike's source code 74 | 75 | The code below comes from `riscv-isa-sim/spike_main/spike.cc`. You could see that I$ and D$ connect to L2$ by miss handler. Moreover, for each core, it has a mmu and the mmu connect to a single ic and dc. 76 | After all the components are connected, the method run is called to start the simulation. 77 | 78 | ~~~cpp 79 | if (ic && l2) ic->set_miss_handler(&*l2); 80 | if (dc && l2) dc->set_miss_handler(&*l2); 81 | for (size_t i = 0; i < nprocs; i++) 82 | { 83 | if (ic) s.get_core(i)->get_mmu()->register_memtracer(&*ic); 84 | if (dc) s.get_core(i)->get_mmu()->register_memtracer(&*dc); 85 | if (extension) s.get_core(i)->register_extension(extension()); 86 | } 87 | 88 | s.set_debug(debug); 89 | s.set_log(log); 90 | s.set_histogram(histogram); 91 | return s.run(); 92 | ~~~ 93 | 94 | On the other hand, inside riscv-isa-sim/riscv/sim.cc, you could see many bus.add_device(), just like the following figure shows. Spike use this function to attach device on bus. After these attachments are done, spike could start to run. 95 | 96 | ![Source of Bus add](./pictures/Bus_Add_device.png) 97 | 98 |

Memory system overview

99 |

What does Spike try to model?

100 | 101 | ![Memory system overview](./pictures/Memory_system.png) 102 | 103 | The picture above is an overview of the memory system. The MMU contains a TLB, which could send back the data without invocation of cache. If the TLB fail, they will go through the table and access the cache. For cache, they model a write-back cache, and use sets/ways/line size to set the configuration. This scheme actually will make cache simulation inaccurate, but they do this in order to speed up performance of simulator. 104 | 105 |

TLB & MMU

106 |

Spike's source code

107 | When an instruction execute a load, it will call load function of MMU and use WRITE_RD to write the data back t register. Then, how to implement the MMU load?
108 |
109 | 110 | ![Instruction load](./pictures/Instruction_MMU.png) 111 | 112 | 113 | Below is an excerpt of riscv-isa-sim/riscv/mmu.h. The functions are defined in macro. The load will go through TLB first and then go to the slow path if TLB miss happens. 114 | 115 | ![MMU trace](./pictures/mmu_trace.png) 116 |
117 | Then, when TLB fail, MMU will call the slow path, and it will ask tracer to call trace. The trace will start to access the cache. Finally, when we jump to riscv-isa-sim/riscv/cachesim.h, we could see that the tracer will call access function of cache. 118 | 119 | ![cache trace](./pictures/cache_trace.png) 120 |

Cache_simulation

121 |

Spike's source code

122 | To understand how the cache is accessed, we could see riscv-isa-sim/riscv/cachesim.cc shown below. There are two functions, access and victimize. When tracer calls trace, the cache will call access.The access will check tag and then do the write or read. Moreover, it use lfsr to find the victim when a replacement happens. 123 | 124 | ![cache access](./pictures/Cache_code.png)
125 |

Result of Cache simulation

126 | The picture below is a result of cache simulation. It could show read/write miss for I$, D$ and 127 | L2. Though it’s not accurate, it could provide a basic analysis. 128 | 129 | ![result_cache](./pictures/Cache_miss.png)
130 |

Processor_Overview

131 |

What does Spike try to model?

132 | Basically, to model a processor, we need the following:
133 | * Model a RISC-V hart
134 | * Processor stepping, including fetch and execution.
135 | * Trap Handling including exception and interrupt handling.
136 | * Optional: MMU for VA->PA
137 |

Hart modeling

138 |

What does Spike try to model?

139 | * Architecture state of a hart, including CSR, pc, registers and floating point registers. 140 |

Spike’s implementation

141 | Below is an excerpt from spike/riscv/processor.c. The state_t contains pc, register_file, and CSR. Notice that Spike only implement some of the CSR inside the hart. It implements other CSR in the processor. 142 | 143 | ![Hart](./pictures/hart_spike.png)
144 |

Trap modeling

145 |

What does Spike try to model?

146 | To model a trap, the followings are needed:
147 | * Cause of the trap. The information is in mcause ( machine cause register)
148 | * For memory related trap, the faulting address needs to be saved in mbadaddr (machine bad address register).
149 | * For trap caused by exception, virtual address of the instruction that encountered the exception. It’s in mepc(machine exception pc register).
150 | * For trap caused by interrupt?
151 |

Spike's source code

152 | Inside encoding.h, the causes are defined. 153 | 154 | ![trap_code](./pictures/trap.png) 155 | 156 | ![trap_spec](./pictures/trap_spec.png)
157 | Inside trap.h , two base classes are defined. The which and badaddr are for the cause and faulting address respectively. Then, macros are used to construct classes for each kind of trap and the cause are saved into the class at the same time. 158 | 159 | ![trap_class](./pictures/trap_class.png) 160 | 161 | ![trap_define](./pictures/trap_define_class.png)
162 | Then, how about epc? (Todo)
163 |

Interrupt Modeling

164 | Todo 165 |

Exception Modeling

166 | Todo 167 |

Bus and Miscellaneous devices

168 |

Device simulation

169 | Related file:
170 | * riscv/device.h
171 | * riscv/device.cc
172 | In this section, we want to describe how to simulate or add a device. The devices inherit from a base class abstract_device_t, which has virtual functions load and store. Then, each device implements the load/store, and provides their special functions. 173 | 174 | ![device](./pictures/abstract_device_t.png)
175 | In spike, five devices are simulated, including bus, rom, real time clock (rtc), processor and debug module. 176 |

Appendix

177 |

Dealing with Instructions

178 | Related file:
179 | * riscv/decode.h
180 | The spike use a class instruction_t to represent instructions. To extract each field, it defines functions like rs1() or rm(), as the following code shows. 181 | 182 | ![instruction](./pictures/instruction.png)
183 | The number of x comes from the following encoding table from the spec. 184 | ![instruction_spec](./pictures/instruction_spec.png)
185 | --------------------------------------------------------------------------------