├── images ├── Pasted image 20211126232934.png ├── Pasted image 20211127174426.png ├── Pasted image 20211127190654.png ├── Pasted image 20211128235325.png ├── Pasted image 20211204205245.png ├── Pasted image 20211204205259.png ├── Pasted image 20211204205447.png ├── Pasted image 20211211174154.png ├── Pasted image 20211211191215.png ├── Pasted image 20211212175227.png ├── Pasted image 20211220150721.png ├── Pasted image 20211220152933.png ├── Pasted image 20211223102118.png ├── Pasted image 20211223104905.png ├── Pasted image 20211223133236.png ├── Pasted image 20211223135944.png ├── Pasted image 20211223150051.png ├── Pasted image 20211229000322.png ├── Pasted image 20211230215955.png ├── Pasted image 20211230222607.png ├── Pasted image 20220101111331.png ├── Pasted image 20220101122220.png ├── Pasted image 20220101122910.png ├── Pasted image 20220101124107.png ├── Pasted image 20220101125014.png ├── Pasted image 20220101134205.png ├── Pasted image 20220101135022.png ├── Pasted image 20220101223536.png ├── Pasted image 20220102114655.png ├── Pasted image 20220102114944.png ├── Pasted image 20220102233943.png ├── Pasted image 20220105152432.png ├── Pasted image 20220105153322.png ├── Pasted image 20220105154118.png ├── Pasted image 20220106132621.png ├── Pasted image 20220107103316.png ├── Pasted image 20220107104340.png ├── Pasted image 20220107110147.png ├── Pasted image 20220107112911.png ├── Pasted image 20220107113311.png ├── Pasted image 20220107132719.png ├── Pasted image 20220115181245.png ├── Pasted image 20220115195843.png ├── Pasted image 20220115195851.png ├── Pasted image 20220115195900.png ├── Pasted image 20220115204939.png ├── Pasted image 20220115223403.png ├── Pasted image 20220115230730.png ├── Pasted image 20220115233259.png ├── Pasted image 20220116151659.png ├── Pasted image 20220116153404.png ├── Pasted image 20220116155322.png ├── Pasted image 20220116162149.png ├── Pasted image 20220118221006.png ├── Pasted image 20220118225402.png ├── Pasted image 20220118225410.png ├── Pasted image 20220118225910.png ├── Pasted image 20220118225917.png ├── Pasted image 20220121161939.png ├── Pasted image 20220121210749.png ├── Pasted image 20220121215213.png ├── Pasted image 20220121215220.png ├── Pasted image 20220130110032.png ├── Pasted image 20220130113240.png ├── Pasted image 20220130115028.png ├── Pasted image 20220130115314.png ├── Pasted image 20220131164754.png ├── Pasted image 20220131165244.png ├── Pasted image 20220131165449.png ├── Pasted image 20220131172634.png ├── Pasted image 20220201224258.png ├── Pasted image 20220202000632.png ├── Pasted image 20220202092048.png ├── Pasted image 20220202093032.png ├── Pasted image 20220202155658.png ├── Pasted image 20220202155711.png ├── Pasted image 20220202163920.png ├── Pasted image 20220202164622.png ├── Pasted image 20220202164639.png ├── Pasted image 20220202170011.png ├── Pasted image 20220429133046.png ├── Pasted image 20220429152114.png ├── Pasted image 20220430212406.png ├── Pasted image 20220430223957.png ├── Pasted image 20220430225112.png ├── Pasted image 20220430225719.png ├── Pasted image 20220504204043.png └── Pasted image 20220510191420.png ├── README.md ├── 01 - Course Overview.md ├── 10 - Optimization.md ├── 26 - Thread Level Parallelism.md ├── 04 - Floating Point.md ├── 23 - Concurrent Programming.md ├── 02-03 - Bits, Bytes and Integers.md ├── 11 - Memory Hierarchy.md ├── 12 - Cache Memories.md ├── 16 - System Level IO.md ├── 19-20 - Dynamic Memory Allocation.md ├── 13 - Linking.md ├── 24-25 - Synchronization.md ├── 21-22 - Network Programming.md ├── 17-18 - Virtual Memory.md ├── 14-15 - Exceptional Control Flow.md └── 05-09 - Machine Level.md /images/Pasted image 20211126232934.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20211126232934.png -------------------------------------------------------------------------------- /images/Pasted image 20211127174426.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20211127174426.png -------------------------------------------------------------------------------- /images/Pasted image 20211127190654.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20211127190654.png -------------------------------------------------------------------------------- /images/Pasted image 20211128235325.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20211128235325.png -------------------------------------------------------------------------------- /images/Pasted image 20211204205245.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20211204205245.png -------------------------------------------------------------------------------- /images/Pasted image 20211204205259.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20211204205259.png -------------------------------------------------------------------------------- /images/Pasted image 20211204205447.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20211204205447.png -------------------------------------------------------------------------------- /images/Pasted image 20211211174154.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20211211174154.png -------------------------------------------------------------------------------- /images/Pasted image 20211211191215.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20211211191215.png -------------------------------------------------------------------------------- /images/Pasted image 20211212175227.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20211212175227.png -------------------------------------------------------------------------------- /images/Pasted image 20211220150721.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20211220150721.png -------------------------------------------------------------------------------- /images/Pasted image 20211220152933.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20211220152933.png -------------------------------------------------------------------------------- /images/Pasted image 20211223102118.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20211223102118.png -------------------------------------------------------------------------------- /images/Pasted image 20211223104905.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20211223104905.png -------------------------------------------------------------------------------- /images/Pasted image 20211223133236.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20211223133236.png -------------------------------------------------------------------------------- /images/Pasted image 20211223135944.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20211223135944.png -------------------------------------------------------------------------------- /images/Pasted image 20211223150051.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20211223150051.png -------------------------------------------------------------------------------- /images/Pasted image 20211229000322.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20211229000322.png -------------------------------------------------------------------------------- /images/Pasted image 20211230215955.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20211230215955.png -------------------------------------------------------------------------------- /images/Pasted image 20211230222607.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20211230222607.png -------------------------------------------------------------------------------- /images/Pasted image 20220101111331.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220101111331.png -------------------------------------------------------------------------------- /images/Pasted image 20220101122220.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220101122220.png -------------------------------------------------------------------------------- /images/Pasted image 20220101122910.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220101122910.png -------------------------------------------------------------------------------- /images/Pasted image 20220101124107.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220101124107.png -------------------------------------------------------------------------------- /images/Pasted image 20220101125014.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220101125014.png -------------------------------------------------------------------------------- /images/Pasted image 20220101134205.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220101134205.png -------------------------------------------------------------------------------- /images/Pasted image 20220101135022.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220101135022.png -------------------------------------------------------------------------------- /images/Pasted image 20220101223536.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220101223536.png -------------------------------------------------------------------------------- /images/Pasted image 20220102114655.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220102114655.png -------------------------------------------------------------------------------- /images/Pasted image 20220102114944.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220102114944.png -------------------------------------------------------------------------------- /images/Pasted image 20220102233943.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220102233943.png -------------------------------------------------------------------------------- /images/Pasted image 20220105152432.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220105152432.png -------------------------------------------------------------------------------- /images/Pasted image 20220105153322.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220105153322.png -------------------------------------------------------------------------------- /images/Pasted image 20220105154118.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220105154118.png -------------------------------------------------------------------------------- /images/Pasted image 20220106132621.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220106132621.png -------------------------------------------------------------------------------- /images/Pasted image 20220107103316.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220107103316.png -------------------------------------------------------------------------------- /images/Pasted image 20220107104340.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220107104340.png -------------------------------------------------------------------------------- /images/Pasted image 20220107110147.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220107110147.png -------------------------------------------------------------------------------- /images/Pasted image 20220107112911.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220107112911.png -------------------------------------------------------------------------------- /images/Pasted image 20220107113311.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220107113311.png -------------------------------------------------------------------------------- /images/Pasted image 20220107132719.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220107132719.png -------------------------------------------------------------------------------- /images/Pasted image 20220115181245.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220115181245.png -------------------------------------------------------------------------------- /images/Pasted image 20220115195843.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220115195843.png -------------------------------------------------------------------------------- /images/Pasted image 20220115195851.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220115195851.png -------------------------------------------------------------------------------- /images/Pasted image 20220115195900.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220115195900.png -------------------------------------------------------------------------------- /images/Pasted image 20220115204939.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220115204939.png -------------------------------------------------------------------------------- /images/Pasted image 20220115223403.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220115223403.png -------------------------------------------------------------------------------- /images/Pasted image 20220115230730.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220115230730.png -------------------------------------------------------------------------------- /images/Pasted image 20220115233259.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220115233259.png -------------------------------------------------------------------------------- /images/Pasted image 20220116151659.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220116151659.png -------------------------------------------------------------------------------- /images/Pasted image 20220116153404.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220116153404.png -------------------------------------------------------------------------------- /images/Pasted image 20220116155322.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220116155322.png -------------------------------------------------------------------------------- /images/Pasted image 20220116162149.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220116162149.png -------------------------------------------------------------------------------- /images/Pasted image 20220118221006.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220118221006.png -------------------------------------------------------------------------------- /images/Pasted image 20220118225402.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220118225402.png -------------------------------------------------------------------------------- /images/Pasted image 20220118225410.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220118225410.png -------------------------------------------------------------------------------- /images/Pasted image 20220118225910.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220118225910.png -------------------------------------------------------------------------------- /images/Pasted image 20220118225917.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220118225917.png -------------------------------------------------------------------------------- /images/Pasted image 20220121161939.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220121161939.png -------------------------------------------------------------------------------- /images/Pasted image 20220121210749.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220121210749.png -------------------------------------------------------------------------------- /images/Pasted image 20220121215213.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220121215213.png -------------------------------------------------------------------------------- /images/Pasted image 20220121215220.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220121215220.png -------------------------------------------------------------------------------- /images/Pasted image 20220130110032.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220130110032.png -------------------------------------------------------------------------------- /images/Pasted image 20220130113240.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220130113240.png -------------------------------------------------------------------------------- /images/Pasted image 20220130115028.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220130115028.png -------------------------------------------------------------------------------- /images/Pasted image 20220130115314.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220130115314.png -------------------------------------------------------------------------------- /images/Pasted image 20220131164754.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220131164754.png -------------------------------------------------------------------------------- /images/Pasted image 20220131165244.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220131165244.png -------------------------------------------------------------------------------- /images/Pasted image 20220131165449.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220131165449.png -------------------------------------------------------------------------------- /images/Pasted image 20220131172634.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220131172634.png -------------------------------------------------------------------------------- /images/Pasted image 20220201224258.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220201224258.png -------------------------------------------------------------------------------- /images/Pasted image 20220202000632.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220202000632.png -------------------------------------------------------------------------------- /images/Pasted image 20220202092048.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220202092048.png -------------------------------------------------------------------------------- /images/Pasted image 20220202093032.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220202093032.png -------------------------------------------------------------------------------- /images/Pasted image 20220202155658.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220202155658.png -------------------------------------------------------------------------------- /images/Pasted image 20220202155711.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220202155711.png -------------------------------------------------------------------------------- /images/Pasted image 20220202163920.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220202163920.png -------------------------------------------------------------------------------- /images/Pasted image 20220202164622.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220202164622.png -------------------------------------------------------------------------------- /images/Pasted image 20220202164639.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220202164639.png -------------------------------------------------------------------------------- /images/Pasted image 20220202170011.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220202170011.png -------------------------------------------------------------------------------- /images/Pasted image 20220429133046.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220429133046.png -------------------------------------------------------------------------------- /images/Pasted image 20220429152114.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220429152114.png -------------------------------------------------------------------------------- /images/Pasted image 20220430212406.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220430212406.png -------------------------------------------------------------------------------- /images/Pasted image 20220430223957.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220430223957.png -------------------------------------------------------------------------------- /images/Pasted image 20220430225112.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220430225112.png -------------------------------------------------------------------------------- /images/Pasted image 20220430225719.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220430225719.png -------------------------------------------------------------------------------- /images/Pasted image 20220504204043.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220504204043.png -------------------------------------------------------------------------------- /images/Pasted image 20220510191420.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yarkhinephyo/15-213-computer-systems-notes/HEAD/images/Pasted image 20220510191420.png -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 15-213-notes 2 | --- 3 | 4 | For my future reference. 5 | 6 | Video playlist: [CMU 15213 - Introduction to Computer Systems](https://www.youtube.com/playlist?list=PLcQU3vbfgCc9sVAiHf5761UUApjZ3ZD3x). 7 | 8 | Lecture slides: [Spring 2015 course website](http://www.cs.cmu.edu/afs/cs/academic/class/15213-f15/www/schedule.html) 9 | 10 | The ICS course provides a programmer's view of how computer systems execute programs, store information, and communicate. It enables students to become more effective programmers, especially in dealing with issues of performance, portability and robustness. It also serves as a foundation for courses on compilers, networks, operating systems, and computer architecture, where a deeper understanding of systems-level issues is required. Topics covered include: machine-level code and its generation by optimizing compilers, performance evaluation and optimization, computer arithmetic, memory organization and management, networking technology and protocols, and supporting concurrent computation. -------------------------------------------------------------------------------- /01 - Course Overview.md: -------------------------------------------------------------------------------- 1 | **Computer Arithmetic**: x ^ 2 >= 0 is true for floats but not always for integers. 2 | 3 | Even with overflow, Integer Arithmetic is commutative and associative. 4 | ``` 5 | 300 * 400 * 500 * 600 -> 1640261632 6 | 400 * 500 * 600 * 300 -> 1640261632 7 | ``` 8 | 9 | This does not apply to floats which can have some numbers disappearing. 10 | ``` 11 | (1e20 + -1e20) + 3.14 -> 3.14 12 | 1e20 + (-1e20 + 3.14) -> 1e20 13 | ``` 14 | 15 | **Assembly**: Understanding assembly is key to machine-level execution model. 16 | 17 | **Memory Systems**: Memory is not unbounded. Memory referencing bugs can be hard to find. For example in C, there is no bound checking on arrays. 18 | 19 | ```c 20 | typedef struct { 21 | int a[2]; 22 | double d; 23 | } struct_t; 24 | 25 | double fun(int i) { 26 | volatile struct_t s; 27 | s.d = 3.14; 28 | s.a[i] = 1073741824; /* May be out of bounds */ 29 | return s.d; 30 | } 31 | 32 | // fun(0) -> 3.14 33 | // fun(1) -> 3.14 34 | // fun(2) -> 3.139999866 35 | // fun(3) -> 2.000000610 36 | // fun(4) -> 3.14 37 | // fun(6) -> Segmentation Fault 38 | ``` 39 | 40 | **Performance Outside Asymptotic Complexity**: Memory access pattern can improve the performance. 41 | 42 | ```c 43 | void copyij(int src[2048][2048], int dst[2048][2048]) { 44 | int i,j; 45 | for (i=0; i<2048; i++) 46 | for (j=0; j<2048; j++) 47 | dst[i][j] = src[i][j] 48 | } 49 | 50 | // Runtime: 4 ms 51 | 52 | void copyij(int src[2048][2048], int dst[2048][2048]) { 53 | int i,j; 54 | for (j=0; j<2048; j++) 55 | for (i=0; i<2048; i++) 56 | dst[i][j] = src[i][j] 57 | } 58 | 59 | // Runtime: 81 ms 60 | ``` 61 | 62 | **Networking and Concurrency** 63 | Concepts of I/O. Programming with socket interface to communicate with any machine. -------------------------------------------------------------------------------- /10 - Optimization.md: -------------------------------------------------------------------------------- 1 | **General Optimizations** 2 | 3 | **Code motion**: Reduce computation frequency. 4 | 5 | ```c 6 | for (j=0; jMemory aliasing: In performing safet optimizations, the compiler must assumed that pointers may be aliased. Aliasing is when two different memory references specify single location. 51 | 52 | ```c 53 | /* twiddle1 CANNOT be optimized to twiddle2 */ 54 | /* Will break if yp points to xp */ 55 | 56 | void twiddle1(long *xp, long *yp) { 57 | *xp += *yp; 58 | *xp += *yp; 59 | } 60 | void twiddle2(long *xp, long *yp) { 61 | *xp += 2* *yp; 62 | } 63 | ``` 64 | 65 | Procedure calls: Compiler treats procedure call as a black box as it may have side effects. 66 | 67 | ```c 68 | /* strlen is an O(N) operation */ 69 | /* A better way is to calculate once before the loop */ 70 | 71 | void lower(char *s) { 72 | size_t i; 73 | for (i=0; i= 'A' && s[i] <= 'Z') 75 | s[i] -= ('A' - 'a'); 76 | } 77 | ``` 78 | 79 | **Instruction-level parallelism** 80 | 81 | Superscalar design: The processor can issue multiple instructions in a single clock. One way is the out-of-order execution where independent instructions are executed in parallel. 82 | 83 | Pipelining: Divides an instruction into steps where each step is executed in a different part of the processor. The example shows completion of 3 multiplications in 7 cycles, even though each individually requires 3 cycles. 84 | 85 | ``` 86 | long p1 = a * b; 87 | long p2 = a * c; 88 | long p3 = p1 * p2; 89 | ``` 90 | 91 | ![](images/Pasted%20image%2020211212175227.png) 92 | 93 | **Example of 5-stage pipeline, 2-instructions** 94 | 95 | ![](images/Pasted%20image%2020220510191420.png) 96 | 97 | **SIMD instruction**: Single instruction, multiple data. Floating point multipler can do 8 single-precision floating point multiplications in 3 cycles. (4 double-precision multiplications in 3 cycles) -------------------------------------------------------------------------------- /26 - Thread Level Parallelism.md: -------------------------------------------------------------------------------- 1 | **Out-of-order processor**: Processor executes instructions in an ordered governed by the availability of input data and execution units instead of the original order in the program. 2 | 3 | In single execution, there is one program counter and queue for operations. 4 | 5 | ![](images/Pasted%20image%2020220202155658.png) 6 | 7 | **Hyperthreading**: Converting physical cores into virtual cores. Assumes that most programs do not make use of all the functional units. Multiple threads can share the functional units in the same core while operating independently. 8 | 9 | ![](images/Pasted%20image%2020220202155711.png) 10 | 11 | **Sum program example**: Use multiple threads to sum a sequence of numbers. 12 | 13 | Simplest approach is have threads summing into a global variable protected by a semaphore mutex. (Results) Single thread is very slow and it gets slower as more cores are used. Locking and unlocking are very time-consuming tasks. 14 | 15 | ```c 16 | /* Thread routine */ 17 | void *sum_mutex(void *vargp) { 18 | long myid = *((long *) vargp); 19 | long start = myid * nelems_per_thread; 20 | long end = start + nelems_per_thread; 21 | long i; 22 | 23 | for (i=start; i 101.11 13 | 2 7/8 -> 10.111 14 | 1 7/16 -> 1.0111 15 | ``` 16 | 17 | **Floating Point Representation** 18 | 19 | **Numerical form**: `(-1)s (M) * (2^E)` where "s" determines whether the number is -ve or +ve, "M" is a fractional value in [1.0, 2.0) and "E" weights value by a power of two. 20 | 21 | 32-bit: 1-bit (s), 8-bits (exp), 23-bits (frac) 22 | 23 | 64-bit: 1-bit (s), 11-bits (exp), 52-bits (frac) 24 | 25 | **Normalized Values**: Exp is not all zeros or ones. 26 | 27 | Significand (M): Encoded with implied leading 1. 28 | 29 | Exponent (E): Requires calculation from exponent bits and the bias term. 30 | 31 | ``` 32 | E = exp - bias 33 | bias = 2^(k - 1) - 1 # where k is the number of "esp" bits. 34 | 35 | Single precision: 127 (Exp: 1...254, E: -126...127) 36 | Double precision: 1023 (Exp: 1...2046, E: -1022...1023) 37 | 38 | // Example 39 | float F = 15213.0 40 | = 11101101101101 41 | = 1.1101101101101 x 2 ^ (13) 42 | 43 | frac = 11011011011010000000000 (23-bit frac) 44 | 45 | bias = 127 46 | exp = 140 = 10001100 47 | E = exp - bias = 13 48 | ``` 49 | 50 | **Denormalized Values**: Exp is all zeros. 51 | 52 | Significand: Encoded with implied leading 0. 53 | 54 | Exponent: `E = 1 - Bias`. 55 | 56 | **Infinity**: Exp is all ones and frac is all zeros. Floating points overflow to a special value called infinity. 57 | 58 | **NaN**: Exp is all ones and frac is not all zeros. 59 | 60 | **Number Line**: Distribution gets dense when closer to 0. 61 | 62 | ``` 63 | |-inf| - Normalized | -Denorm |-0| 0| +Denorm | +Normalized |+inf| 64 | ``` 65 | 66 | **Comparison**: With the exception of special values, can be compared as Unsigned Integer. 67 | 68 | **8-Bit Floating Point Example** 69 | 70 | ``` 71 | Bias = 7 (4-bit exp) 72 | 73 | n: M = 1.xxx... 74 | d: M = 0.xxx... 75 | n: E = exp - bias 76 | d: E = 1 - bias 77 | ``` 78 | 79 | ![](images/Pasted%20image%2020211126232934.png) 80 | 81 | **Round Nearest Even**: Round up or round down to closer integers. If exactly in the middle, round to the nearest even number. Example: 1.4 -> 1, 1.5 -> 2, -1.5 -> -2. Other rounding modes are statistically biased. 82 | 83 | **Round Binary Numbers** 84 | For binary numbers, "even" occurs when the least significant bit is zero. "Halfway" occurs when bits to the right are "100....". 85 | 86 | ``` 87 | // Round to nearest 1/4 (2-bits right) 88 | 89 | Value Binary Rounded Rounded Value 90 | 2 3/2 10.00011 10.00 2 91 | 2 3/16 10.00110 10.01 2 1/4 92 | 2 7/8 10.11100 11.00 3 93 | 2 5/8 10.10100 10.10 2 1/2 94 | ``` 95 | 96 | **Properties of FP Addition**: Commutative but not associative. 97 | 98 | **Properties of FP Multiply**: Commutative but not associative. Multiplication does not distributes over addition. 99 | 100 | **Floating Point in C** 101 | 102 | Unlike between "int" and "unsigned", casts between "float", "int", "double" changes the bit representations. 103 | 104 | 1. Float cast to an Int may not need any rounding. (23-bit to 32-bit) 105 | 2. Double cast to an Int requires some rounding. (52-bit to 32-bit) 106 | 3. Int to double does not lose any bits. (32-bit to 52-bit) 107 | 4. Int to float requires some rounding. (32-bit to 23-bit) 108 | 109 | **Floating Point Puzzles** 110 | 111 | ``` 112 | int x; 113 | float f; 114 | double d; 115 | 116 | x == (int)(float) x -> False 117 | x == (int)(double) x -> True 118 | f == (float)(double) f -> True 119 | d == (double)(float) d -> False 120 | f == - (-f) -> True (Only toggling bit) 121 | 2/3 == 2/3.0 -> False (Int & float cast to float) 122 | d < 0.0 => ((d*2) < 0.0) -> True (Even overflow will be -INF) 123 | d > f => -f > -d -> True 124 | d * d >= 0.0 -> True 125 | (d+f) - d == f -> False 126 | ``` 127 | -------------------------------------------------------------------------------- /23 - Concurrent Programming.md: -------------------------------------------------------------------------------- 1 | **Races**: Outcome depends on arbitrary scheduling decisions in the system. 2 | **Deadlock**: Improper resource allocation prevents forward progress. 3 | **Livelock/starvation/fairness**: External events or scheduling decisions prevent sub-task progress. 4 | 5 | **Problem with iterative server** 6 | Client 2 returns on `connect` call even though the connection is not accepted because server-side TCP manager queues the request. Client 2 returns after `write` call because server-side TCP manager buffers the input data. Only client 2's `read` call is blocked. 7 | 8 | ![](images/Pasted%20image%2020220130110032.png) 9 | 10 | **Approaches to concurrent servers** 11 | 12 | Type | Interleaving by | Address space | Note 13 | ------ | ------- | ------ | ------- 14 | Process-based | Kernel | Separate | 15 | Event-based | Programmer | Shared | I/O multiplexing 16 | Thread-based | Kernel | Shared | 17 | 18 | **Process-based concurrent server** 19 | Fork a child request for every client connection so that a blocking client cannot influence the other clients. 20 | 21 | Both parent and child have copies of `listenfd` and `connfd` so appropriate connections must be closed. The (long running) server process must reap its children to prevent memory leak. 22 | 23 | There is an additional overhead for process control. Furthermore, it is nontrivial to share data between processes. 24 | 25 | ```c 26 | int main(int argc, char **argv) { 27 | int listenfd, connfd; 28 | socklen_t clientlen; 29 | struct sockaddr_storage clientaddr; /* Handle both IPv4 IPv6 */ 30 | 31 | Signal(SIGCHLD, sigchld_handler); 32 | listenfd = Open_listenfd(argv[1]); 33 | while(1) { 34 | clientlen = sizeof(struct sockaddr_storage); 35 | connfd = Accept(listenfd, (SA *) &clientaddr, &clientlen); 36 | if (Fork() == 0) { 37 | Close(listenfd); /* Child stops listening */ 38 | echo(connfd); /* Child services client */ 39 | Close(connfd); /* Child closes connection with client */ 40 | exit(0); 41 | } 42 | Close(connfd); /* Parent closes connected socket */ 43 | } 44 | } 45 | 46 | // Avoid memory leak by reaping children 47 | void sigchld_handler(int sig) { 48 | while(waitpid(-1, 0, WNOHANG) > 0) 49 | ; 50 | return; 51 | } 52 | ``` 53 | 54 | **Event-based concurrent server** 55 | Server maintains a set of active connections with an array of `connfd`. It determines which descriptors have pending inputs with `select` or `epoll` functions. If `listenfd` has input, the server accepts connection and add to the `connfd` array. 56 | 57 | No process or thread control overhead. Easy to debug due to one logical control flow. 58 | 59 | More complex to code then process or thread based designs. Cannot take advantage of multi-core. 60 | 61 | ![](images/Pasted%20image%2020220130113240.png) 62 | 63 | **Process overview** 64 | Program context and kernel context stored in the registers and the private virtual address space (Kernel virtual memory + Process virtual memory). 65 | 66 | ![](images/Pasted%20image%2020220130115028.png) 67 | 68 | **Thread overview** 69 | Independent stack, stack pointer and the thread context. 70 | 71 | Each thread has its own logical control flow and stack (Local variables). All thread share the same code, data and kernel context. Context switching threads has a very low overhead. 72 | 73 | Threads associated with a process form a pool of peers without hierarchy. 74 | 75 | ![](images/Pasted%20image%2020220130115314.png) 76 | 77 | **Posix threads interface**: Standard interface for manipulation threads in C programs. 78 | 79 | **Pthreads hello world** 80 | Creates a peer thread with `pthread_create` and waits for it to finish with `pthread_join`. The `thread` callback takes in one generic pointer. 81 | 82 | ```c 83 | #include "csapp.h" 84 | void *thread(void *vargp); /* Takes in one generic pointer */ 85 | 86 | int main() { 87 | pthread_t tid; 88 | /* Last argument is a pointer for thread function */ 89 | Pthread_create(&tid, NULL, thread, NULL); 90 | Pthread_join(tid, NULL); 91 | exit(0); 92 | } 93 | 94 | void *thread(void *vargp) { 95 | printf("Hello world!\n"); 96 | return NULL; 97 | } 98 | ``` 99 | 100 | **Thread-based concurrent server** 101 | It is important to run detached threads to avoid memory leak. Joinable threads (default configuration) must be reaped with `pthread_join` to free memory resources. 102 | 103 | The file descriptor pointer has to be malloc-ed instead of being passed directly. If not, we will be assuming wrongly that the peer thread would dereference the pointer before the main thread gets a new connected file descriptor. By using malloc, the pointer is provided with new memory location every loop. 104 | 105 | ```c 106 | int main(int argc, char **argv) { 107 | int listenfd, *connfd; 108 | socklen_t clientlen; 109 | struct sockaddr_storage clientaddr; /* Handle both IPv4 IPv6 */ 110 | pthread_t tid; 111 | 112 | listenfd = Open_listenfd(argv[1]); 113 | while(1) { 114 | clientlen = sizeof(struct sockaddr_storage); 115 | /* Needed to prevent a race condition */ 116 | /* Now a different pointer is allocated each loop */ 117 | connfdp = Malloc(sizeof(int)); 118 | *connfd = Accept(listenfd, (SA *) &clientaddr, &clientlen); 119 | Pthread_create(&tid, NULL, thread, connfdp); 120 | } 121 | } 122 | 123 | void *thread(void *vargp) { 124 | int connfd = *((int *) vargp); 125 | /* When detached, kernel automatically reaps terminated thread */ 126 | Pthread_detach(pthread_self()); 127 | /* Free data in heap as it has been copied onto stack */ 128 | Free(vargp); 129 | echo(connfd); 130 | Close(connfd); 131 | return NULL; 132 | } 133 | ``` -------------------------------------------------------------------------------- /02-03 - Bits, Bytes and Integers.md: -------------------------------------------------------------------------------- 1 | **Hexadecimal**: 1010 -> A, 1100 -> C, 1111 -> F 2 | 3 | **Word size**: Nominal size of pointer data. _64-bit_ machine means 8-byte addresses and _32-bit_ machine means 4-byte addresses. 4 | 5 | **Fixed-size integer types**: Using `int32_t`, `uint32_t`, `int64_t` instead of `int`, `unsigned int`, `long` respectively ensures that data sizes are independent of compilers or machines. 6 | 7 | **Shift operations** 8 | 9 | Left Shift: Fill 0 on the right. 10 | 11 | Right Shift (Logical): Fill 0 on the left. 12 | 13 | Right Shift (Arithmetic): Fill the most significant bit on the left. 14 | 15 | Almost all compilers use the arithmetic version for signed data. For unsigned data, however, rights shifts must be logical. 16 | 17 | Bit shifting operates on numbers in the register. This is independent of the byte ordering in the memory. 18 | 19 | **Unsigned values**: Bit positions multiply with the corresponding powers of 2. 20 | 21 | **Two's complement values**: The most left bit is a negative value. 22 | 23 | Type | Decimal | Hex 24 | ------ | ------- | ------ 25 | UMax | 65535 | FF FF 26 | TMax | 32767 | 7F FF 27 | TMin | -32768 | 80 00 28 | -1 | -1 | FF FF 29 | 0 | 0 | 00 00 30 | 31 | **Mappings between unsigned and two's complement**: Keep the bit representations and reinterpret. Interpretation may change but the bit representation remains the same. 32 | 33 | For nonnegative values, the encodings are the same. Negative values can be easily converted to/from unsigned format with a difference of 2w. This can be thought of as 2w-1 to negate the first bit and another 2w-1 to add the positional value. 34 | 35 | ``` 36 | Signed FF FF -> -1 37 | Unsigned FF FF -> -1 + (2 ^ 16) -> 65535 38 | ``` 39 | 40 | ![](images/Pasted%20image%2020220429152114.png) 41 | 42 | **Casting surprises**: If there is a mix of two's complement and unsigned, values are implicitly cast to unsigned. 43 | 44 | ``` 45 | -1 < 0 46 | -1 > 0U 47 | -1 > -2 48 | (unsigned) -1 > -2 49 | ``` 50 | 51 | Infinite loop bug in the example below. 52 | 53 | ```c 54 | // First example 55 | unsigned i; 56 | for (i=n-1; i>=0; i--) { 57 | func(a[i]) 58 | } 59 | 60 | // Second example 61 | int i; 62 | for (i=n-1; i - sizeof(int) >= 0; i--) { 63 | func(a[i]) 64 | } 65 | ``` 66 | 67 | **Extension**: 68 | 69 | Unsigned: Zero extension. 70 | 71 | Two's complement: Sign extension. Copy the most significant bit. 72 | 73 | ``` 74 | 1 1 1 0 -> -8 + 4 + 2 = -2 75 | 1 1 1 1 0 -> -16 + 8 + 4 + 2 = -2 76 | ``` 77 | 78 | **Truncation** 79 | 80 | Unsigned: x' = x mod 2k where truncation target is k-bits. 81 | 82 | Two's complement: Convert to unsigned for truncation. Then convert back to two's complement. x' = U2T( T2U(x) mod 2k ). 83 | 84 | **Addition** 85 | 86 | Unsigned: Add and drop the overflow. UAdd(u, v) = (u + v) mod 2w. Overflow has occurred if the result is less than either of the numbers. 87 | 88 | Two's complement: TAdd has the same bit level behaviour as UAdd. TAdd = U2T( UAdd( T2U(x), T2U(y) ) ). Positive overflow has occurred if x > 0 and y > 0 but s <= 0. Negative overflow has occurred if x < 0 and y < 0 but s >= 0. 89 | 90 | ``` 91 | // Negative overflow 92 | // -9 cannot be represented with Signed 4-bit 93 | 1 1 0 1 -> -3 94 | 1 0 1 0 -> -6 95 | ------- 96 | 0 1 1 1 -> +7 97 | 98 | // Positve overflow 99 | 0 1 1 1 -> +7 100 | 0 1 0 1 -> +5 101 | ------- 102 | 1 1 0 0 -> -4 103 | ``` 104 | 105 | **Negation** 106 | 107 | Unsigned: 2w - x if x > 0. You can get 0 if you do unsigned addition with the original number. 108 | 109 | Two's complement: -x if x != TMin else TMin. In a two's complement system, `-x == ~x + 1`. There is always the corner case of TMin that should be checked during testing. 110 | 111 | **Multiplication** 112 | 113 | Unsigned: Multiply and drop the overflow. UMult(u, v) = (u \* v) mod 2w. 114 | 115 | Two's complement: Similar to addition, bit level equivalence of two's complement and unsigned multiplications are identical. TMult(u, v) = U2T( (u \* v) mod 2w ). 116 | 117 | **Multiply Power-of-2**: Bit-shift left. 118 | 119 | **Divide Power-of-2**: Bit-shift right (Arithmetic) rounds towards zero. 120 | 121 | **Counting Down with Unsigned** 122 | 123 | ```c 124 | unsigned i; 125 | for (i = cnt-2; i < cnt; i--) { 126 | a[i] += a[i+1] 127 | } 128 | ``` 129 | 130 | **Byte ordering**: Addresses specify first byte locations. Addresses of successive words differ by 4-bytes (32-bits) and 8-bytes (64-bits). 131 | 132 | Little Endian: Least significant byte has lowest address. Common in most Intel-compatible machines. 133 | 134 | Big Endian: Least significant byte has highest address. Common in machines from IBM and Oracle. 135 | 136 | ``` 137 | // Variable x has 4-byte of 0x01234567 138 | 139 | // Big Endian 140 | 0x100|0x101|0x102|0x103| 141 | 01 |23 |45 |67 | 142 | 143 | // Little Endian 144 | 0x100|0x101|0x102|0x103| 145 | 67 |45 |23 |01 | 146 | ``` 147 | 148 | **Representing Strings**: Array of characters with a final character of 0. All machines have the same order. 149 | 150 | ``` 151 | char S[6] = "18213"; 152 | 153 | 0x31 | 0x38 | 0x32 | 0x31 | 0x33 | 0x00 154 | ``` 155 | 156 | **C Puzzles** 157 | 158 | ``` 159 | int x,y; 160 | unsigned ux, uy; 161 | ux = x; 162 | uy = y; 163 | 164 | x < 0 -> ((x*2) < 0) [False statement, consider TMin] 165 | ux >= 0 [True Statement] 166 | x & 7 == 7 -> (x << 30) < 0 [True Statement] 167 | ux > -1 [False Statement] 168 | x > y -> -x < -y [False Statement, consider TMin] 169 | x * x >= 0 [False Statement, consider overflow] 170 | x >= 0 -> -x <= 0 [True statement, all +ve have inverse] 171 | x <= 0 -> -x >= 0 [False Statement, consider TMin] 172 | (x|-x) >> 31 == -1 [False Statement, only False for 0] 173 | # Note: Signed int has arithmetic shift 174 | ``` -------------------------------------------------------------------------------- /11 - Memory Hierarchy.md: -------------------------------------------------------------------------------- 1 | 2 | **Volatile memories** 3 | 4 | | . | SRAM | DRAM | 5 | | -------------- | ----- | ----------- | 6 | | Trans. per-bit | 6 | 1 | 7 | | Access time | 1x | 10x | 8 | | Needs refresh | No | Yes | 9 | | Cost | 100x | 1x | 10 | | Applications | Cache | Main memory | 11 | | Volatile | Yes | Yes | 12 | 13 | SRAM: Each bit is stored in a bistable memory cell that can stay in either of two voltage configurations as long as it is powered. 14 | 15 | DRAM: Each bit is stored as charge on a capacitor. Memory system periodically refreshes every bit of memory by reading and rewriting it. 16 | 17 | Each DRAM chip has _d_ supercells, each consisting of _w_ DRAM cells. In the example below, the chip has 16 supercells and 8 bits per supercell. When the memory controller sends row address strobe (RAS), the DRAM copies the supercells in row 2 onto an internal buffer. When the memory controller sends column address strobe (CAS), the DRAM copies the 8 bits inside the specific supercell (now in the internal buffer) and sends them to the memory controller. 18 | 19 | ![](images/Pasted%20image%2020220430212406.png) 20 | 21 | **Nonvolatile memories** 22 | 23 | ROM: Read-only memory. 24 | 25 | PROM: Can be programmed exactly once. 26 | 27 | EPROM: Can be bulk cleared to zero and rewritten. Requires a separate device to write _ones_. 28 | 29 | EEPROM: Can be bulk cleared to zero and rewritten in-place without a separate device. 30 | 31 | Flash memory: Based on EEPROMs. Partial erase capability which wears out over time. 32 | 33 | **Firmware**: Programs stored in ROMs such as BIOS, disk controllers. 34 | 35 | **Bus Interface**: Collection of parallel wires that carry address, data and control signals. The control wires carry signals that synchronize the transaction and identify the type being performed. 36 | 37 | **CPU read transaction** 38 | 39 | 1. Command `movq A, %rax`. 40 | 2. CPU places address A on the memory bus. 41 | 3. Main memory reads address A from the memory bus and writes its data on the bus. 42 | 4. CPU reads the data and places it in `%rax`. 43 | 44 | To be precise, the CPU's bus interface places the address A on the system bus and the I/O bridge forwards it to the memory bus. 45 | 46 | ![](images/Pasted%20image%2020220430223957.png) 47 | 48 | **CPU write transaction** 49 | 50 | 1. Command `movq %rax, A`. 51 | 2. CPU places address A on the memory bus. 52 | 3. Main memory reads it and waits for data to arrive. 53 | 4. CPU places the data on the bus. 54 | 5. Main memory reads the data and stores it at address A. 55 | 56 | **Bus is expensive**: Memory reads and writes are about 50-100 ns while operations between registers are < 1 ns. 57 | 58 | **Rotating disks**: Consist of one or more platters, each with two surfaces. Each surface consists of concentric rings called tracks. Each track consists of sectors separated by gaps. 59 | 60 | ![](images/Pasted%20image%2020220430225112.png) 61 | 62 | **Disk capacity**: Number of bits for storage. 63 | 64 | ![](images/Pasted%20image%2020220430225719.png) 65 | 66 | **Disk access**: Seek -> Rotational Latency -> Data Transfer. Seek time dominates the disk access time. After the first byte, the remaining bytes are essentially free. 67 | 68 | ![](images/Pasted%20image%2020211220150721.png) 69 | 70 | ``` 71 | Given rotational rate = 7200 RPM 72 | Average seek time = 9 ms 73 | Average #sectors / track = 400 74 | 75 | Tavg rotation = 1/2 x 60 /(7200 RPM) x 1000 ms/sec = 4 ms 76 | Tavg transfer 77 | = 60 /(7200 RPM) x 1/(400 sectors/track) x 1000 ms/sec = 0.02 ms 78 | Taccess = 9ms + 4 ms + 0.02 ms 79 | ``` 80 | 81 | **Disk controller**: Keeps mapping of logical disk blocks (1 ... B-1) to physical sectors. Sets aside spare cylinders for each zone. 82 | 83 | **Read a disk sector** 84 | 85 | 1. CPU writes a command + logical block number + destination memory address to a port associated with the disk controller. 86 | 2. Firmware on the disk controller performs a fast table lookup. 87 | 3. Hardware on the disk controller reads the sector and transfer into main memory. 88 | 4. Once data transfer completes, the disk controller notifies the CPU with an interrupt. 89 | 90 | **I/O devices**: I/O bridge connects with system bus, memory bus and I/O bus. Unlike system bus and memory bus, I/O bus is designed to be CPU independent. It can accomodate a variety of third party I/O devices. 91 | 92 | **Solid state disk (SSD)**: Storage technology based on flash memory. Flash translation layer serves the same purpose as the disk controller. 93 | 94 | Data is written in units of pages but a page can be only written after its block has been erased. In other words, to modify a single page, other pages have to be copied onto a clean block. A block wears after about 100,000 erases. Consequently, reading is performed much faster than writing. 95 | 96 | Performance: Average seq read time of 50 us and average seq write time of 60 us. Random writes are slower because erasing a block takes a long time (~1ms) 97 | 98 | ![](images/Pasted%20image%2020211220152933.png) 99 | 100 | **Principle of locality**: Programs tend to use data and instructions with addresses near or equal to those used recently. 101 | 102 | Temporal locality: Recently referenced items will be referenced again. 103 | 104 | Spatial locality: Items with nearby addresses tend to be referenced close together in time. 105 | 106 | **Memory hierachy** 107 | 108 | 1. Registers 109 | 2. L1 (SRAM) 110 | 3. L2 (SRAM) 111 | 4. L3 (SRAM) 112 | 5. Main memory (DRAM) 113 | 6. Local secondary storage (Disk) 114 | 7. Remote secondary storage (Servers) 115 | 116 | **Types of cache misses** 117 | 118 | Cold miss: Cache is initially empty. 119 | 120 | Capacity miss: Occurs when active cache blocks (working set) is larger than the cache. 121 | 122 | Conflict miss: Most caches limit blocks at level-(k+1) to a small subset of positions at level k. This is because randomly placed blocks are expensive to locate. However, this means that multiple data can map to the same level-k block and evicting one another. -------------------------------------------------------------------------------- /12 - Cache Memories.md: -------------------------------------------------------------------------------- 1 | **General cache organization** 2 | 3 | ![](images/Pasted%20image%2020211223102118.png) 4 | 5 | **Cache size**: S x E x B bytes. The tag bits and the valid bit are not included in the calculation. 6 | 7 | **Cache READ**: When CPU sends address to cache, cache divides the address into a number of regions (tag, set index, block offset). 8 | 9 | Set index: Locate the set in the cache 10 | 11 | Tag: Check if any line matches the tag 12 | 13 | Block offset: Byte location in a cache line 14 | 15 | ![](images/Pasted%20image%2020211223104905.png) 16 | 17 | **Direct-mapped cache simulation** 18 | 19 | Direct-mapped cache contains 1 line per set only. 20 | B = 2 bytes/block, S = 4 sets, E = 1 block/set, 4-bit address. 21 | 22 | ``` 23 | // t=1, s=2, b=1 24 | // One-byte per read 25 | 26 | 0 [0000], miss 27 | 1 [0001], hit 28 | 7 [0111], miss 29 | 8 [1000], miss // Prev line is evicted 30 | 0 [0000], miss // Conflict miss 31 | ``` 32 | 33 | Note: Individual data can be padded to prevent thrashing via conflict miss. 34 | 35 | **E-way associative cache**: (E = 2) means 2-way associative. Searches for matching tag in all the lines. 36 | 37 | **2-way associative cache simulation** 38 | 39 | B = 2 bytes/block, S = 2 sets, E = 2 blocks/set, 4-bit address. 40 | 41 | ``` 42 | // t=2, s=1, b=1 43 | // One-byte per read 44 | 45 | 0 [0000], miss 46 | 1 [0001], hit 47 | 7 [0111], miss 48 | 8 [1000], miss 49 | 0 [0000], hit 50 | ``` 51 | 52 | Line replacement: LRU and LFU are some examples. At further away from CPU in the memory hierarchy, the higher the cost of a miss. So the overhead of replacement policies are justified. 53 | 54 | **Cache WRITE**: Multiple copies of data exist in L1, L2, L3, main memory, disk. 55 | 56 | **Write-hit** 57 | 58 | Write-through: Write immediately to memory on cache hit. Simple but causes bus traffic every write. 59 | 60 | Write-back (Common): Defer write to memory until the replacement of line in cache. Another bit is needed to indicate whether the cache block has been modified. 61 | 62 | **Write-miss** 63 | 64 | Write-allocate (Common): Loads from the lower level into cache and updates the line. 65 | 66 | No-write-allocate: Writes the word directly to the next lower level. 67 | 68 | **Cache hierarchy** 69 | 70 | ![](images/Pasted%20image%2020211223133236.png) 71 | 72 | Separating i-cache and d-cache ensures that data access do not create conflict misses with instructional access. I-caches are typically read-only (simpler). 73 | 74 | L1 i-cache and d-cache : 32 kB, 8-way associative, 4 cycles. 75 | 76 | L2 unified cache: 256 kB, 8-way associative, 10 cycles. 77 | 78 | L3 unified cache: 8 MB, 16-way associative, 40-75 cycles. 79 | 80 | **Cache metrics** 81 | 82 | Miss rate: 3-10% for L1, less than 1% for L2. 83 | 84 | Hit rate: 1 - miss rate. 85 | 86 | Hit time: Time required to transfer data to CPU. 87 | 88 | Miss penalty: 10 cycles for L1 miss, 50 cycles for L3 miss, 200 cycles for main memory miss. 89 | 90 | **Cache design considerations** 91 | 92 | Cache size: Larger cache increases hit rate but increases hit time. 93 | 94 | Block size: Larger block sizes exploit spatial locality but decreases the number of cache lines. This means temporal locality is exploited less. 95 | 96 | Associativity: Higher associativity reduces conflict misses but increases hit time and cost of implementation. 97 | 98 | Write strategy: Write-through cache means that read misses are less expensive because they do not trigger memory write (Multi-cache). Write-back caches means fewer data transfers thus allowing more bandwidth for other devices. 99 | 100 | **Memory mountain** 101 | 102 | Iterate over the first "elem" number of elements (bytes) in an array with stride of "stride", with 4x4 loop unrolling. 103 | 104 | As stride increases, spatial locality decreases. As size increases, temporal locality decreases, because there are fewer caches in the hierachy that can hold the data. 105 | 106 | ![](images/Pasted%20image%2020211223135944.png) 107 | 108 | Highest peak of L1 is at 14 GB/s while lowest point of main memory is at 900 MB/s. Even when working set is too large to fit in any caches, the highest point on main memory is 8x higher than lowest point. 109 | 110 | Prefetching: At stride 1, read throughput is relatively flat because of a hardware prefetching mechanism that identifies stride-1 reference patterns. 111 | 112 | **Matrix multiplication example** 113 | 114 | Multiple N x N matrix where each element is a double (8 bytes). Block size is 32 bytes (4 doubles) and cache is not big enough to hold multiple rows. 115 | 116 | ![](images/Pasted%20image%2020211223150051.png) 117 | 118 | ```c 119 | /* ijk */ 120 | 121 | for (i=0; i2 = (9/8) \* n3 180 | 181 | **Blocking example (After)** 182 | 183 | Any row or column contains n/B blocks. Block size is 8 doubles. (B2)/8 misses for each block. 184 | 185 | One block of C: 2n/B \* (B2)/8 = nB/4 186 | 187 | All blocks of C: nB/4 \* (n/B)2 = (n3)/4B 188 | 189 | Ensure that three blocks fit into cache: 3(B2) < C. -------------------------------------------------------------------------------- /16 - System Level IO.md: -------------------------------------------------------------------------------- 1 | **Linux file**: Sequence of bytes. 2 | Even I/O devices and sockets are represented as files in Unix. 3 | 4 | **Seek**: Changing the current file position. Indicates next offset in the file to read or write. 5 | 6 | **File types** 7 | Regular files contain arbitrary data. 8 | Directory is an index for a related group of files. 9 | Socket is for communicating with a process on another machine. 10 | (Out of scope) Pipe, Symbolic link, Character and block devices. 11 | 12 | **Text file** 13 | Kernel does not know the difference between text files and binary files. They are differentiated at the application level, which ensures all are ASCII or unicode characters. 14 | EOL in Linux: `\n` (0xa) 15 | EOL in windows: `\r\n` (0xd 0xa) 16 | 17 | **Directory** 18 | Consists of an array of links where each link maps a filename to a file. At least two entries which are `.` and `..` 19 | 20 | **Opening file** 21 | Informs the kernel that the user is ready to access the file. Takes in a flag which can be bitwise OR. Returns a small identifying integer "file descriptor" where -1 means an error occurred. 22 | 23 | ```c 24 | int fd; 25 | if ((fd = open("/etc/hosts", O_RDONLY)) < 0) { 26 | perror("open"); 27 | exit(1); 28 | } 29 | ``` 30 | 31 | **Closing file** 32 | Closing an already closed file is a recipe of disaster in threaded programs. Always check return codes for error. 33 | 34 | **Reading file** 35 | Read at least one byte but no more than the number told. Assume that the `read()` call will not reach this number. 36 | 37 | For std input, it waits till a string is typed in with `\n`. If it is network connection, it waits till something arrives. 38 | 39 | Returns `ssize_t` which is a signed long int. 0 means EOF. +ve is number of bytes. -ve means an error occurred. 40 | 41 | ```c 42 | char buf[512]; 43 | int fd; /* File descriptor */ 44 | int nbytes; /* Number of bytes read */ 45 | 46 | if ((nbytes = read(fd, buf, sizeof(buf))) < 0) { 47 | perror("read"); 48 | exit(1); 49 | } 50 | ``` 51 | 52 | **Writing file** 53 | Write at least one byte but no more than the number told. Not all the text provided may be written! 54 | 55 | **Short counts**: Reading less than bytes allocated to the buffer. 56 | 57 | Encountering EOF on reads. 58 | Reading text lines from a terminal. 59 | Reading and writing network sockets. 60 | 61 | **Unbuffered RIO** 62 | Unbuffered input and output of binary data: `rio_readn` and `rio_writen`. Especially useful for transferring data on network sockets. 63 | 64 | For reading, short count is returned on EOF, else error is thrown. 65 | For writing, the bytes written will be of the provided size. 66 | 67 | ```c 68 | /* Robustly read n bytes (unbuffered) */ 69 | ssize_t rio_readn(int fd, void *userbuf, size_t n) { 70 | size_t nleft = n; 71 | ssize_t nread; 72 | char *bufp = userbuf; 73 | while (nleft > 0) { 74 | if ((nread = read(fd, bufp, nleft)) < 0) { 75 | if (errno = EINTR) /* Interrupted by sig handler */ 76 | nread = 0; /* call read() again */ 77 | else 78 | return -1; /* errno is set by read() */ 79 | } else if (nread == 0) 80 | break; /* EOF */ 81 | nleft -= nread; 82 | bufp += nread; /* Move the pointer forward */ 83 | } 84 | return (n - nleft); 85 | } 86 | ``` 87 | 88 | **Buffered RIO** 89 | Buffered input of text lines and binary data: `rio_readlineb` and `rio_readnb`. Thread-safe. 90 | 91 | ```c 92 | typedef struct { 93 | int rio_fd; 94 | int rio_cnt; /* Unread bytes in internal buffer */ 95 | char *rio_bufptr; /* Next unread byte */ 96 | char rio_buf[RIO_BUFSIZE]; 97 | } rio_t; 98 | 99 | /* Associates a file descriptor with a read buffer */ 100 | void rio_readinitb(rio_t *rp, int fd); 101 | 102 | ssize_t rio_readlineb(rio_t *rp, void *usrbuf, size_t maxlen); 103 | 104 | ssize_t rio_readnb(rio_t *rp, void *usrbuf, size_t n); 105 | ``` 106 | 107 | File has associated buffer to hold bytes that have been read from file but not yet read by user code. 108 | 109 | ![](images/Pasted%20image%2020220102233943.png) 110 | 111 | ```c 112 | /* Robustly read n bytes (Buffered) */ 113 | /* The code for rio_readnb is similar to rio_readn */ 114 | /* Instead of read(), it uses rio_read() internally */ 115 | 116 | /* Uses rp->rio_buf as internal buffer before copying to usrbuf */ 117 | static ssize_t rio_read(rio_t *rp, char *usrbuf, size_t n) { 118 | int cnt; 119 | while (rp->rio_cnt <= 0) { /* Refill if buf empty */ 120 | rp->rio_cnt = read(rp->rio_fd, rp->rio_buf, sizeof(rp->rio_buf)); 121 | 122 | if (rp->rio_cnt < 0) { 123 | if (errno != EINTR) /* Allow sig handler interrupt */ 124 | return -1; 125 | } else if (rp->rio_cnt == 0) 126 | return 0; 127 | else 128 | rp->rio_bufptr = rp->rio_buf; /* Reset buf ptr */ 129 | } 130 | 131 | /* Copy min(n, rp->rio_cnt) bytes from internal buf to user buf */ 132 | cnt = n; 133 | if (rp->rio_cnt < n) 134 | cnt = rp->rio_cnt; 135 | memcpy(usrbuf, rp->rioptr, cnt); 136 | rp->rio_bufptr += cnt; 137 | rp->rio_cnt -= cnt; 138 | return cnt; 139 | } 140 | ``` 141 | 142 | **Copying text from std input to std output** 143 | 144 | ```c 145 | #include "csapp.h" 146 | 147 | int main(int argc, char **argv) { 148 | int n; 149 | rio_t rio; 150 | char buf[MAXLINE]; 151 | 152 | Rio_readinitb(&rio, STDIN_FILENO); 153 | while ((n = Rio_readlineb(&rio, buf, MAXLINE)) != 0) 154 | Rio_writen(STDOUT_FILENO, buf, n); 155 | exit(0); 156 | } 157 | ``` 158 | 159 | **File metadata**: Maintained by kernel. Accessed by users with `stat` or `fstat` functions. Stored in `struct stat` data structure. 160 | 161 | **Accessing file metadata** 162 | 163 | ```c 164 | int main(int argc, char **argv) { 165 | struct stat stat; 166 | char *type, *readok; 167 | Stat(argv[1], &stat); /* Filename and pointer to stat*/ 168 | if (S_ISREG(stat.st_mode)) /* Determine file type */ 169 | type = "regular"; 170 | else if (S_ISDIR(stat.st_mode)) 171 | type = "directory"; 172 | else 173 | type = "other"; 174 | if ((stat.st_mode & S_IRUSR)) /* Check read access */ 175 | readok = "yes"; 176 | else 177 | readok = "no"; 178 | printf("type: %s, read: %s\n", type, readok); 179 | exit(0); 180 | } 181 | ``` 182 | 183 | **File descriptor table**: Every process has a file descriptor table that contains pointers to the file records in the Open File table shared globally. 184 | **Open file table**: Table of file records shared globally. Each record contains the last read position and the reference count to check if it is still needed. 185 | **v-node table**: Every file has an associated v-node entry which contains information on the file. 186 | 187 | **File sharing**: Two distinct file descriptors sharing the same disk file through two distinct entries in the Open File table. 188 | 189 | ![](images/Pasted%20image%2020220105152432.png) 190 | 191 | **Processes sharing file with fork** 192 | Child process inherits its parent's open files. Each `refcnt` increments by 1. The file positions are shared, so if the parent does a read, the child will have its file position shifted too. 193 | 194 | ![](images/Pasted%20image%2020220105153322.png) 195 | 196 | **I/O redirection**: Copies descriptor table entry `oldfd` to entry `newfd`. 197 | 198 | Open file A -> Open file B -> Call `dup2(4,1)`. This copies the entry at index 4 to index 1. Writing file descriptor 1 (Original stdout) will write to disk instead. 199 | 200 | ![](images/Pasted%20image%2020220105154118.png) 201 | 202 | **Standard I/O**: Uses a buffer as Unix I/O calls are expensive around 10,000 clock cycles. 203 | 204 | ```c 205 | #include 206 | 207 | int main() { 208 | printf("h"); 209 | printf("i"); 210 | printf("!"); 211 | printf("\n"); 212 | fflush(stdout); 213 | exit(0); 214 | } 215 | 216 | // This only calls Unix I/O one time 217 | // write(1, "hi!\n", 4) 218 | ``` 219 | 220 | **When use Standard I/O**: Working with disk or terminal files 221 | **When use Unix I/O**: Inside signal handlers (Async-signal-safe) 222 | **When use RIO**: Reading and writing network sockets -------------------------------------------------------------------------------- /19-20 - Dynamic Memory Allocation.md: -------------------------------------------------------------------------------- 1 | **Dynamic memory allocation**: Acquire virtual memory at runtime for data structures whose sizes are known only at runtime. Managed in an area of process virtual memory known as "heap". 2 | 3 | **Heap**: Collection of variable sized blocks which are either allocated or free. 4 | 5 | **Allocate memory**: `void *malloc(size_t size)` If successful, returns a pointer to a memory block of `size` bytes. Boundary aligned 8-byte for x86 and 16-byte for x86-64. 6 | 7 | **Free memory**: `void free(void *p)` where p must come from a previous call to `malloc` or `realloc`. 8 | 9 | **Malloc example** 10 | 11 | ```c 12 | #include 13 | #include 14 | 15 | void foo(int n) { 16 | int i, *p; 17 | p = (int *) malloc(n * sizeof(int)); 18 | if (p == NULL) { 19 | perror("malloc"); 20 | exit(0); 21 | } 22 | for (i=0; iThroughput: Number of operations per unit time. 45 | 46 | Peak memory utilization: How efficiently the allocator uses the heap. The sum of all the payloads divided by the total size of the heap. 47 | 48 | **Memory fragmentation** 49 | 50 | Internal fragmentation: Occurs when memory block assigned to the process is larger than the memory requested. 51 | 52 | External fragmentation: When there is enough aggregate heap memory but no single free block is large enough. 53 | 54 | **Track allocated block size**: Requires additional space for every allocated block. These headers decrease the memory utilization. (In the picture, each box is 4-byte) 55 | 56 | ![](images/Pasted%20image%2020220116151659.png) 57 | 58 | **Track free blocks** 59 | 60 | - Method one: Implicit list using length to link all blocks. 61 | - Method two: Explicit list among the free blocks with pointers. 62 | - Method three: Different free lists for different size classes. 63 | - Method four: Blocks sorted by size. 64 | 65 | **Implicit list**: Besides block length, will have to mark as free or allocated. Since there is 8-byte or 16-byte alignment, the lower-order 3-bit or 4-bit are "always zero" for each block address. The lowest bit of the length can be used as a flag. 66 | 67 | For the example below, assume 4-byte words aligned on 8-byte boundary. The headers are not aligned so that the payloads are aligned. The first allocated block (16/1) is for an 8-byte payload only, as the last 4-byte is a padding. Allocated block of size zero terminates the heap. 68 | 69 | ![](images/Pasted%20image%2020220116153404.png) 70 | 71 | Allocation cost: O(n), for looking through the list. Free cost: O(1). Memory usage depends on the allocation policy. 72 | 73 | **First fit**: First free block from the beginning. 74 | 75 | **Next fit**: First free block starting from end of previous search. 76 | 77 | **Best fit**: Choose best free block with fewest bytes left over. 78 | 79 | **Allocation by splitting**: Split the block if the allocated space is smaller than free space. 80 | 81 | ![](images/Pasted%20image%2020220116155322.png) 82 | 83 | ```c 84 | addblock(p, 4); 85 | 86 | void addblock(ptr p, int len) { 87 | /* Round up to even blocks: 8-byte align */ 88 | int newsize = ((len + 1) >> 1) << 1; 89 | int oldsize = *p & -2; 90 | *p = newsize | 1; 91 | if (newsize < oldsize) 92 | *(p+newsize) = oldsize - newsize; 93 | } 94 | ``` 95 | 96 | **Freeing by coalescing**: Join the next/previous blocks if they are free. Freeing the next block is simple. To free the previous block, time complexity would be linear with the current method. 97 | 98 | **Boundary tags**: Replicate the block length header at the end of blocks (Called footer). The content is identical. The disadvantage is added internal fragmentation. 99 | 100 | ![](images/Pasted%20image%2020220116162149.png) 101 | 102 | For optimization, the allocated blocks do not need footer tags, only free blocks do (For checking previous block). Use one more bit in the length header to track the allocation status of the previous block. If it is not allocated, that previous block will have footer tag. Coalescing can then be carried out. 103 | 104 | **Explicit free list**: Maintain list of free blocks with "next" and "prev" pointers after the block length header. Note that these header, footer and pointers impose a minimum block size. 105 | 106 | ![](images/Pasted%20image%2020220118221006.png) 107 | 108 | Compared to implicit list, allocation is faster as only free blocks are checked. Extra space is needed, but only in the free blocks. 109 | 110 | **Allocating explicit free list**: Updates 6 pointers. Split as necessary. 111 | 112 | **Freeing explicit free list** 113 | 114 | LIFO policy inserts the freed block at the beginning of the free list. Constant time (to free) but may cause worse fragmentation. 115 | 116 | Address-ordered policy inserts the freed block so that free list blocks are in address order. Decreases fragmentation but requires a search algorithm. 117 | 118 | **LIFO policy illustration** 119 | 120 | Free block at the front, allocated block at the back. 121 | 122 | ![](images/Pasted%20image%2020220118225402.png) 123 | ![](images/Pasted%20image%2020220118225410.png) 124 | 125 | Free blocks at the front and back. 126 | 127 | ![](images/Pasted%20image%2020220118225910.png) 128 | ![](images/Pasted%20image%2020220118225917.png) 129 | 130 | **Seglist (Segregated list)** 131 | 132 | Given an array of free lists for each size class, allocate a new block (Size N) by searching through free lists for size M > N. If no block is found, request additional memory with `sbrk`. Allocate N bytes from the new memory and place the rest in the free list of the largest size class. 133 | 134 | Higher throughput because individual size classes are smaller than the entire list. Better memory utilization because first-fit search of segregated free list approximates a best-fit search of the entire heap. 135 | 136 | The free lists are stored at the beginning of the heap. 137 | 138 | **Implicit memory management**: Application allocates space but the system automatically frees them. Identify "garbage", area of memory that cannot be referenced anymore (No pointers), and free up. 139 | 140 | ```c 141 | void foo() { 142 | int *p = malloc(128); 143 | return; /* p block is now garbage */ 144 | } 145 | ``` 146 | 147 | Need to assume that memory manager can distinguish pointers from non-pointers, pointers point to the start of a block and pointers cannot be hidden. 148 | 149 | **Garbage collection**: Automatic reclamation of heap-allocated storage. 150 | 151 | **Classic GC algorithms** 152 | 153 | 1. Mark-and-sweep collection 154 | 2. Reference counting 155 | 3. Copying collection 156 | 4. Generational collections 157 | 158 | **Memory as a directed graph** 159 | 160 | A node (Memory block) is reachable if there is a path from a root node (Has pointers into the heap but outside the heap). Examples include registers, global variables, stack variables. Non-reachable nodes are garbage. 161 | 162 | ![](images/Pasted%20image%2020220121161939.png) 163 | 164 | **Mark and sweep collecting** 165 | 166 | Mark phase: Use one of the extra mark bits at block headers to mark reachable blocks. 167 | 168 | ``` 169 | // Pseudocode 170 | ptr mark(ptr p) { 171 | if (!is_ptr(p)) return; 172 | if (markBitSet(p)) return; 173 | setMarkBit(p); 174 | for (i=0; iSweep phase: Scan all blocks and free blocks that are unmarked. 183 | 184 | ``` 185 | // Pseudocode 186 | ptr sweep(ptr p, ptr end) { 187 | while (p < end) { 188 | if markBitSet(p) 189 | clearMarkBit(); 190 | else if (allocateBitSet(p)) 191 | free(p); 192 | p += length(p); 193 | } 194 | } 195 | ``` 196 | 197 | The function `is_ptr()` checks if a word is by pointer by checking if it points to an allocated block of memory. However, C pointers can point to the middle of an allocated block. The solution is to use a balanced tree and keep track of the allocated blocks. 198 | 199 | **C pointer declarations** 200 | 201 | ``` 202 | Precedence: () [] is higher than * & 203 | 204 | int *p # p is a ptr to int 205 | int *p[13] # p is an array of ptrs to int 206 | int *(p[13]) # 207 | int **p # p is a ptr to a ptr to int 208 | int (*p)[13] # p is a ptr to an array of 13 ints 209 | int *f() # f is a function returning a ptr to int 210 | 211 | int (*(*f())[13])() 212 | # f is a function returning ptr to an array[13] of ptrs to functions returning int 213 | ``` -------------------------------------------------------------------------------- /13 - Linking.md: -------------------------------------------------------------------------------- 1 | **Linking**: Collection and combining pieces of code and data into a single file for execution. Can be done at different times. 2 | 3 | Compile time: Source code is translated into machine code. 4 | 5 | load time: Program is loaded into memory and executed by the loader. 6 | 7 | Run time: When instructions of the program are executed. 8 | 9 | **Why linkers**: Enable separate compilation of a software. Code can be more modularized. Compilation time is reduced for individual modules. Executable size is reduced when shared libraries can be taken advantage of. 10 | 11 | **Compiler driver**: Invokes the language preprocessor, compiler, assembler, linker on behalf of the user. An example is the GCC driver. 12 | 13 | Language processor: GNU C Preprocessor includes the headers and expands the macros to convert `*.c` to `*.i`. 14 | 15 | Compiler: C compiler compiles preprocessed code to assembly code by converting `*.i` to `*.s` file. (Some compilers generate object code directly) 16 | 17 | Assembler: Assembler converts the assembly code to machine code in the relocatable object file `*.o`. 18 | 19 | Linker: Combines the object files to create a binary executable object file. 20 | 21 | **Static Linkers**: Take a collection of relocatable object files and generate a fully-linked executable object file. The file consists of continguous code and data sections. The linkers must perform two functions. 22 | 23 | Symbol resolution: Ensuring that each reference is associated with exactly one definition. 24 | 25 | ```c 26 | void swap() {...} /* definition */ 27 | swap() /* reference */ 28 | int *xp = &x; /* definition & reference */ 29 | ``` 30 | 31 | Symbol relocation: Addresses of symbol definitions from object files are associated with memory locations. 32 | 33 | **Types of object files** 34 | 35 | A relocatable object file is an `.o` file that is produced from one source `.c` file. An executable object file is an `a.out` file generated by the linker. A shared object file is a relocatable object `.so` file that can be loaded into memory and linked dynamically. 36 | 37 | The object file format used by Linux and Unix is called _Executable and Linkable Format (ELF)_. 38 | 39 | ``` 40 | ELF header - Word size, byte ordering, file type (o, so, out) 41 | Section header table - Where different sections are in memory 42 | 43 | .text - Machine code 44 | .rodata - Read only data such as jump tables 45 | .data - Initialized global variables 46 | .bss - Uninitialized global variables (No space taken by data) 47 | .symtab - Symbol entries for procedures and global variables 48 | .rel.text - Relocation info for .text section (Not needed in exe) 49 | .rel.data - Relocation info for .data section (Not needed in exe) 50 | .debug - Source code to line number in machine code 51 | .strtab - String table for symbols in .symtab .debug 52 | 53 | Section header table - Where different sections start 54 | ``` 55 | 56 | **Symbol** 57 | 58 | Global symbols: Symbols defined by module "m" that can be referenced by other modules. Nonstatic C functions and global variables. 59 | 60 | External symbols: Symbols that are referenced by module "m" but defined by other modules. 61 | 62 | Local symbols: Symbols defined and referenced exclusively by module "m", such as static C functions and global variables. These are stored in `.bss` or `.data`. Local linker symbols are not the same as local program variables which are stored in the stack. 63 | 64 | **Symbol table**: Assemblers use the symbols exported by compilers to build the table. This means `*.s` files do not have them yet. Each entry is shown below. 65 | 66 | ```c 67 | typedef struct { 68 | int name; /* .strtab offset */ 69 | char type:4, binding:4; /* Function or data, local or global */ 70 | char reserved; /* Unused */ 71 | short section; /* Section header index */ 72 | long value; /* Section offset or abs address */ 73 | long size; /* Object size in bytes */ 74 | } 75 | ``` 76 | 77 | Pseudosections: Sections not indexed in the section header table. ABS means that the symbol should not be relocated. UNDEF means symbols are defined not in the module. COMMON means uninitialized data objects that are not allocated. 78 | 79 | Uninitialized data objects may be defined in other modules so compiler defers the decision to the linker by assigning COMMON section in relocatable object files. COMMON section is not present in executable object files. 80 | 81 | Aside from uninitialized variables... zero-initialized variables are assigned it to `.bss` instead of `.data` becuase at runtime, variables in this section are assigned zero. This saves space. 82 | 83 | **Symbol resolution** 84 | 85 | Straightforward for references to local symbols which are defined in the same module. Compiler ensures that static variables have unique names and defined only once. 86 | 87 | It is more complex for global symbols. The compiler can accept undefined variables anyway and the failure will occur at the linker. 88 | 89 | Compiler exports each global symbol either as strong (Procedures and initialized globals) or weak (Uninitialized globals). The assembler encodes the information implicitly in the symbol tables. The linker uses the following rules - 90 | 91 | 1. Multiple strong symbols not allowed. 92 | 2. Strong symbol is chosen over weak symbols. 93 | 3. If multiple weak symbols, an arbitrary one is picked. 94 | 95 | For example, calling the function f() results in `x -> 0x0` and `y -> 0x80000000`. 96 | 97 | ```c 98 | /* foo.c */ 99 | #include 100 | 101 | void f(void); 102 | int y = 15213; /* Address: 0x601020 */ 103 | int x = 15213; /* Address: 0x601024 */ 104 | int main() { 105 | f(); 106 | return 0; 107 | } 108 | ``` 109 | 110 | ```c 111 | /* bar.c */ 112 | double x; /* Assigned to 8-byte memory */ 113 | void f() { 114 | x = = -0.0; 115 | } 116 | ``` 117 | 118 | **Static libraries** 119 | 120 | Related functions are compiled into separate object modules and packaged into a single static library. At link time, the linker will only resolve object modules referenced by the program. On Linux, static libraries are stored in a format called _archive_ as `*.a` files. 121 | 122 | To create a static library of multiple modules, we can use the AR tool - 123 | 124 | ```bash 125 | gcc -c addvec.c multvec.c 126 | ar rcs libvector.a addvec.o multvec.o 127 | ``` 128 | 129 | We can also create a `vector.h` header file to define the function prototypes. 130 | 131 | ```c 132 | /* main2.c */ 133 | #include 134 | #include "vector.h" 135 | 136 | int x[2] = {1, 2}; 137 | int y[2] = {3, 4}; 138 | int x[2]; 139 | 140 | int main() { 141 | addvec(x, y, z, 2); 142 | printf("z = [%d %d]\n", z[0], z[1]); 143 | return 0; 144 | } 145 | ``` 146 | 147 | ![](images/Pasted%20image%2020211230222607.png) 148 | 149 | **Symbol resolution of static libraries** 150 | 151 | Linker scans the relocatable object files and archives from left to right as given in the command line. 152 | 153 | ``` 154 | E = { relocatable object files to be merged } 155 | U = { unresolved symbols } 156 | D = { defined symbols } 157 | 158 | for each file f (left to right): 159 | if f is relocatable object file: 160 | Add f to E. 161 | Update U and D with definitions and references. 162 | if f is an archive: 163 | Try to match symbols in U and add matched members to E. 164 | Update U and D with definitions and references. 165 | 166 | if U is non-empty: 167 | Throw error 168 | ``` 169 | 170 | Rule of thumb is that static libraries should be ordered at the end of the command line. The libraries requiring dependencies should not be placed at the end. 171 | 172 | **Relocation** 173 | 174 | After symbol resolution, the linker knows the exact sizes of the code and data sections. In relocation, it merges the input modules and assign runtime addresses to each symbol. There are two steps. 175 | 176 | Relocating sections: Linker aggregates sections of the same type and assign runtime addresses to sections and symbol definitions. 177 | 178 | Relocating symbol references: Linker modifies every symbol reference so that each points to the correct runtime address. Linker relies on _relocation entries_ for while have been generated by the assembler. 179 | 180 | ![](images/Pasted%20image%2020211229000322.png) 181 | 182 | **Relocation entry** 183 | 184 | Assembler generates a _relocation entry_ whenever external variables are referenced by the module. Entries for code are in `.rel.text` and entries for data are in `.rel.data`. 185 | 186 | ```c 187 | typedef struct { 188 | long offset; /* Section offset of the reference */ 189 | long type:32, /* Relocation type */ 190 | symbol:32; /* Symbol table index */ 191 | long addend; /* Constant part of relocation expression */ 192 | } Elf64_Rela; 193 | ``` 194 | 195 | There are two relocation types. _R_X86_64_PC32_ is for PC-relative referencing. _R_X86_64_32_ is for absolute referencing. 196 | 197 | **Relocation algorithm** 198 | 199 | ``` 200 | foreach section s { 201 | foreach relocation entry r { 202 | // Ptr to the placeholder bytes 203 | // The address in the current process (not runtime) 204 | refptr = s + r.offset; 205 | 206 | if (r.type == R_X86_64_PC32) { 207 | // The address during runtime 208 | refaddr = ADDR(s) + r.offset; 209 | *refptr = ADDR(r.symbol) + r.addend - refaddr; 210 | } 211 | 212 | if (r.type == R_X86_64_32) 213 | *refptr = ADDR(r.symbol) + r.addend; 214 | } 215 | } 216 | ``` 217 | 218 | **Example of relocation** 219 | 220 | At offset 0xa, `array` will have to be referenced with PC-relative address. At offset 0xf, `sum` will have to be referenced with absolute address. 221 | 222 | ``` 223 | int array[2] = {1, 2}; 224 | int main() { 225 | int val = sum(array, 2); 226 | return val; 227 | } 228 | 229 | 0000000000000000
: 230 | 0: 48 83 ec 08 sub $0x8, %rsp 231 | 4: be 02 00 00 00 mov $0x2, %esi 232 | 9: bf 00 00 00 00 mov $0x0, %edi # %edi = &array 233 | (R_X86_64_32 array) 234 | e: e8 00 00 00 00 callq 13 # sum() 235 | (R_X86_64_PC32 sum-0x4) 236 | 13: 48 83 c4 09 add $0x8, %rsp 237 | 17: c3 retq 238 | ``` 239 | 240 | At line 0xe, there is 1-byte opcode 0xe8 and a 4-byte placeholder for the PC-relative reference to `sum()` function. The placeholder will be replaced with PC-relative reference after the relocation algorithm is run. (Note: Jumps and Calls always use PC-relative addresses) 241 | 242 | ```c 243 | r.offset = 0xf 244 | r.symbol = sum 245 | r.type = R_X86_64_PC32 246 | r.addend = -4 247 | ``` 248 | 249 | At line 0x9, there is 1-byte opcode and a 4-byte placeholder for the absolute reference to `array`. After relocation, the address will point to somewhere in the `.data` section. 250 | 251 | At load time, the loader can copy the bytes directly into memory and execute without any modifications. The `.rel.text` and `.rel.data` sections are no longer required. 252 | 253 | **Executable object file** 254 | 255 | The format is similar to relocatable object file. It includes the _entry point_ which is the first instruction to execute. There is also `.init` section which executes before the main program. 256 | 257 | **Loading executable object file into memory** 258 | 259 | The loader copies _(Map virtual memory pages)_ the code and data from the executable object file into memory and jumps to the _entry point_. The runtime _heap_ follows the `.data` section and grows upwards. The user stack starts from the largest legal user address (248 - 1) and grows down. 260 | 261 | Heap, data and code segments actually have gaps between them due to alignment requirements and the address-space layout randomization. 262 | 263 | ![](images/Pasted%20image%2020211230215955.png) 264 | 265 | **Shared libraries (Dynamic link libraries)** 266 | 267 | Provide a mechanism where one executable copy can be shared between different programs. Unlike static libraries, the linking of references is deferred until load time or even runtime. 268 | 269 | The linker does not resolve the references such as `printf()` during relocation. It only notes in the symbol table that these references will have to be resolved during load time. When executable file is created, linking is only partially complete. 270 | 271 | During load time, the loader (execve) loads the executable and runs the dynamic linker. The dynamic linker allocates text and data of shared libraries into memory and relocate the references in the executable object file. Control is returned to the _entry point_ afterwards. 272 | 273 | ![](images/Pasted%20image%2020220429133046.png) 274 | 275 | **Linking during runtime example** 276 | 277 | ```c 278 | #include 279 | #include 280 | #include 281 | 282 | int x[2] = {1, 2}; 283 | int y[2] = {3, 4}; 284 | int z[2]; 285 | 286 | int main() { 287 | void *handle; 288 | void (*addvec)(int *, int *, int *, int); 289 | char *error; 290 | 291 | /* Dynamically load the shared library that contains addvec() */ 292 | handle = dlopen("./libvector.so", RTLD_LAZY); 293 | if (!handle) { 294 | fprintf(stderr, "%s\n", dlerror()); 295 | exit(1); 296 | } 297 | 298 | ... 299 | 300 | 301 | /* Get a pointer to the addvec() function we just loaded */ 302 | addvec = dlsym(handle, "addvec"); 303 | if ((error = dlerror()) != NULL) { 304 | fprintf(stderr, "%s\n", error); 305 | exit(1); 306 | } 307 | 308 | /* Now we can call addvec() just like any other function */ 309 | addvec(x, y, z, 2); 310 | printf("z = [%d %d]\n", z[0], z[1]); 311 | 312 | /* Unload the shared library */ 313 | if (dlclose(handle) < 0) { 314 | fprintf(stderr, "%s\n", dlerror()); 315 | exit(1); 316 | } 317 | return 0; 318 | } 319 | ``` 320 | 321 | **Virtual Address Space of a Linux Process** 322 | 323 | Kernel address space starts with 1 (top). User address space starts with 0 (bottom). 324 | 325 | ![](images/Pasted%20image%2020220115223403.png) -------------------------------------------------------------------------------- /24-25 - Synchronization.md: -------------------------------------------------------------------------------- 1 | **Shared variable**: Multiple threads reference some instance of `x`. 2 | 3 | **Conceptual thread memory model**: Each thread has its own thread context and shares the remaining process context. 4 | **Operational thread memory model**: Register values are truly separate but any thread can read and write the stacks of other threads. 5 | 6 | **Global variable**: Virtual memory ensures only one instance is available. 7 | **Local variable**: Each thread stack contains one instance of the variable. 8 | **Local static variable**: Variables inside functions with `static` attribute. The scope is limited to the function but only one instance is stored. 9 | 10 | **Example of shared variables** 11 | Just for demonstration, not good practice. 12 | 13 | ```c 14 | char **ptr; /* Stored in .data - SHARED */ 15 | 16 | int main() { 17 | long i; /* Stored on stack - NOT SHARED */ 18 | pthread_t tid; 19 | char *msgs[2] = { 20 | "Hello from foo", 21 | "Hello from bar" 22 | }; 23 | 24 | ptr = msgs; /* Stored on stack - SHARED */ 25 | for (i=0; i<2; i++) 26 | Pthread_create(&tid, NULL, thread, (void *)i); 27 | Pthread_exit(NULL); 28 | } 29 | 30 | void *thread(void *vargp) { 31 | long myid = (long) vargp; /* Stored on stack - NOT SHARED */ 32 | static int cnt = 0; /* Stored in .data - SHARED */ 33 | printf("[%ld]: %s (cnt=%d)\n", myid, ptr[myid], ++cnt); 34 | return NULL; 35 | } 36 | ``` 37 | 38 | **Example of improper synchronization** 39 | Both threads increment the same global variable `cnt`. The increment step is broken down into a few operations in assembly (Load register -> Increment -> Store to memory). When these operations for the two threads are interleaved randomly by the OS, the final state of the global variable depends on the scheduling decisions. 40 | 41 | ```c 42 | /* Global shared variable */ 43 | volatile long cnt = 0; 44 | 45 | int main(int argc, char **argv) { 46 | long niters; 47 | pthread_t tid1, tid2; 48 | 49 | niters = atoi(argv[1]); 50 | Pthread_create(&tid1, NULL, thread, &niters); 51 | Pthread_create(&tid2, NULL, thread, &niters); 52 | Pthread_join(tid1, NULL); 53 | Pthread_join(tid2, NULL); 54 | 55 | if (cnt != (2 * niters)) 56 | printf("BOOM! cnt=%ld\n", cnt); 57 | else 58 | printf("OK cnt=%ld\n", cnt); 59 | } 60 | 61 | void *thread(void *vargp) { 62 | long i, niters = *((long *) vargp); 63 | for (i=0; i 107 | 108 | int sem_init(sem_t *s, unsigned int val); 109 | int sem_wait(sem_t *s); /* P(s) */ 110 | int sem_post(sem_t *s); /* V(s) */ 111 | ``` 112 | 113 | Semaphore creates a forbidden region that encloses unsafe region. 114 | 115 | ![](images/Pasted%20image%2020220131172634.png) 116 | 117 | **Producer-consumer problem** 118 | Producer waits for empty slot in buffer, inserts item and notifies consumer. Consumer waits for item, removes item and notifies producer. 119 | 120 | For example in event-driven graphical user interface, produce detects mouse movements or keys and place in buffer. Consumer retrieves events from buffer and paints the display. 121 | 122 | **Producer-consumer on an n-element buffer** 123 | Requires one mutex (Buffer) and two counting semaphores. The producers wait for `slots` semaphore with `P()`. The consumers announce the available slots with `V()`. The opposite is true for `items` semaphore. 124 | 125 | **Creating a shared buffer package** 126 | To allow a shared buffer between threads, where each item is an `int`. 127 | 128 | ```c 129 | #include "csapp.h" 130 | 131 | typedef struct { 132 | int *buf, /* Buffer array */ 133 | int n; /* Max number of slots */ 134 | int front; /* buf[(front+1)%n] is first item */ 135 | int rear; /* buf[rear%n] is last item */ 136 | sem_t mutex; /* Protects access to buf */ 137 | sem_t slots; /* Counts available slots */ 138 | sem_t items; /* Counts available items */ 139 | } 140 | 141 | /* Creates empty, bounded, shared FIFO buffer with n slots */ 142 | void sbuf_init(sbuf_t *sp, int n) { 143 | sp->buf = Calloc(n, sizeof(int)); 144 | sp->n = n; 145 | sp->front = sp->rear = 0; 146 | Sem_init(&sp->mutex, 0, 1); 147 | Sem_init(&sp->slots, 0, n); /* Initially, buf has n empty slots */ 148 | Sem_init(&sp->items, 0, 0); /* Initially, buf has 0 items */ 149 | } 150 | 151 | void sbuf_deinit(sbuf_t *sp) { 152 | Free(sp->buf); 153 | } 154 | 155 | void sbuf_insert(sbuf_t *sp, int item) { 156 | P(&sp->slots); 157 | P(&sp->mutex); 158 | sp->buf[(++sp->rear)%(sp->n)] = item; /* Insert at rear pointer */ 159 | V(&sp->mutex); 160 | V(&sp->items); /* Increment semaphore and notify consumers */ 161 | } 162 | 163 | void sbuf_remove(sbuf_t *sp) { 164 | int item; 165 | P(&sp->items); 166 | P(&sp->mutex); 167 | item = sp->buf[(++sp->front)%(sp->n)]; /* Remove item at front */ 168 | V(&sp->mutex); 169 | V(&sp->slots); 170 | return item; 171 | } 172 | ``` 173 | 174 | **Readers-writers problem** 175 | Generalization of the mutual exclusion problem. Writes must have exclusive access to the object but unlimited number of readers can access the object. 176 | 177 | The implementation below favors readers over writers. There is another variant favoring the writers. 178 | 179 | The semaphore `mutex` protects the `readcnt`. The semaphore `w` protects the critical reading/writing section. 180 | 181 | ```c 182 | int readcnt; /* Initially 0 */ 183 | sem_t mutex, w; /* Initially 1 */ 184 | 185 | void reader(void) { 186 | while (1) { 187 | P(&mutex); 188 | readcnt++; 189 | if (readcnt == 1) /* First reader in */ 190 | P(&w); 191 | V(&mutex); 192 | 193 | /* Critical reading section */ 194 | 195 | P(&mutex); 196 | readcnt--; 197 | if (readcnt == 0) /* Last reader out */ 198 | V(&w); 199 | V(&mutex); 200 | } 201 | } 202 | 203 | void writer(void) { 204 | while (1) { 205 | P(&w); 206 | /* Critical writing section*/ 207 | V(&w); 208 | } 209 | } 210 | ``` 211 | 212 | **Prethreaded concurrent echo server** 213 | Uses producer-consumer model and the shared buffer to implement a concurrent web server that makes use of a pool of worker threads. This removes the overhead of producing and terminating threads. 214 | 215 | ![](images/Pasted%20image%2020220201224258.png) 216 | 217 | The main thread accepts new connections and immediately add to the shared buffer. 218 | 219 | ```c 220 | sbuf_t sbuf; /* Shared buffer of connected descriptors */ 221 | 222 | int main(int argc, char **argv) { 223 | int i, listenfd, connfd; 224 | socklen_t clientlen; 225 | struct sockaddr_storage clientaddr; 226 | pthread_t tid; 227 | 228 | listenfd = Open_listenfd(argv[1]); 229 | sbuf_init(&sbuf, SBUFSIZE); 230 | for (i=0; iprocess and one or more client processes. Server provides service for clients by manipulating some resource. A single host/machine can run many servers and clients concurrently. 2 | 3 | **Network interface card**: It looks to the computer as an I/O device, similar to a disk controller. Sending a message is writing a message to a virtual file called the _network_. Receiving data is reading the virtual file. 4 | 5 | **Network backbone devices** 6 | 7 | Hub: Connects a collection of hosts to form an _ethernet segment_. A hub slavishly copies each bit from each port to every other port. 8 | 9 | Bridge: Connects multiple _ethernet segments_. Learns which hosts are reachable from which ports and selectively copy frames from port to port. 10 | 11 | Router: Connects multiple incompatible LANs physically to form the internet. 12 | 13 | **IP (Internet protocol)**: Provides basic naming scheme for hosts and unreliable delivery capability of packets from host-to-host. 14 | 15 | **UDP**: Uses IP to provide unreliable datagram delivery from process-to-process. 16 | 17 | **TCP**: Uses IP to provide reliable byte streams from process-to-process over connections. 18 | 19 | **Socket interface**: Provides the routines required for interprocess communication between applications. 20 | 21 | **IP address**: 32-bit IP addresses are stored in IP address struct. Always stored in memory in network byte order (Big Endian). Need to use helper functions to convert to host byte order (Little Endian). 22 | 23 | ```c 24 | struct in_address { 25 | uint32_t s_addr; /* network byte order */ 26 | } 27 | ``` 28 | 29 | **Helper functions** 30 | 31 | ```c 32 | // Returns value in network-byte-order 33 | uint32_t htonl(uint32_t hostlong); 34 | uint16_t htons(uint16_t hostshort); 35 | 36 | // Returns value in host-byte-order 37 | uint32_t ntohl(uint32_t netlong); 38 | uint16_t ntohs(unit16_t netshort); 39 | 40 | // (IP address) "84.52.184.224" 41 | // --> (NBO) 0x 54 34 B8 E0 42 | // --> (HBO) 0x E0 B8 34 54 43 | 44 | // Converts dotted-decimal string to binary IP address (NBO) 45 | int inet_pton(AF_INET, const char *src, void *dst); 46 | ``` 47 | 48 | **Internet connection**: Client and servers communicate by sending streams of bytes over connections. Point-to-point, full-duplex and reliable. Connections are uniquely identified by the socket addresses of its endpoints `(cliaddr:cliport, servaddr:servport)`. 49 | 50 | Ephemeral port: Assigned automatically by client kernel when client makes a connection request. 51 | 52 | Well-known port: Is associated with some service provided by a server. Mappings between well-known ports and service names are in `/etc/services`. 53 | 54 | **Socket**: An `IPaddress:port` pair. To the kernel, a socket is an endpoint of communication. To an application, a socket is a file descriptor for reading/writing from/to the network. 55 | 56 | **Socket address structure** 57 | 58 | 16-byte worth of information. The leading 2 bytes designate what kind of socket it is (TCP, IPv6 etc.). 59 | 60 | ```c 61 | struct sockaddr { 62 | uint16_t sa_family; /* Protocol */ 63 | char sa_data[14]; /* Address data */ 64 | }; 65 | typedef struct sockaddr SA; 66 | ``` 67 | 68 | There is a more specific socket address version for internet called `sockaddr_in`. Must cast appropriately as some functions only take in the generic version `sockaddr`. 69 | 70 | ```c 71 | struct sockaddr_in { 72 | uint16_t sin_family; /* Protocol, always AF_INET */ 73 | uint16_t sin_port; /* Port num in network byte order */ 74 | struct in_addr sin_addr; /* IP addr in network byte order */ 75 | unsighed char sin_zero[8]; /* Pad to sizeof(struct sockaddr) */ 76 | }; 77 | ``` 78 | 79 | **Sockets interface overview** 80 | 81 | Server runs a program that is ready to receive connections from clients to perform various services -> Start up a client -> Maintain a session, which is back and forth communication between client and server -> Client disconnects -> Server shuts down. 82 | 83 | ![](images/Pasted%20image%2020220121210749.png) 84 | 85 | **Socket descriptor**: Clients and servers use the socket function (System call) to create a generic socket descriptor. 86 | 87 | ```c 88 | int socket(int domain, int type, int protocol) 89 | 90 | // IPv4, stream of bytes (connection endpoint) 91 | int clientfd = Socket(AF_INET, SOCKET_STREAM, 0); 92 | ``` 93 | 94 | **Socket bind**: Asks the kernel to associate the server's socket address with a socket descriptor. The process can read bytes that arrive on the connection whose endpoint is `addr` by reading from the descriptor `sockfd`. Writes to the socket descriptor are transferred along the same connection. 95 | 96 | ```c 97 | int bind(int sockfd, SA *addr, socklen_t addrlen); 98 | ``` 99 | 100 | **Socket listen**: Converts the active socket (default) to passive socket. Tells the kernel that the descriptor will be used by a server rather than a client. Active socket means the client is responsible for the active opening of TCP connection (SYN). Passive socket means the server waits for requests (SYN) and creates active sockets for each connection accepted. 101 | 102 | ```c 103 | /* sockfd becomes listenfd */ 104 | /* backlog is the number of connection requests in queue */ 105 | int listen(int sockfd, int backlog); 106 | ``` 107 | 108 | **Socket accept**: Server blocks in the function to wait for connection requests from clients on `listenfd`. On connection, the function returns a `connfd` that can be used to communicate with the client via I/O routines. This allows the program to fork a child whenever a new request is received. 109 | 110 | ```c 111 | int accept(int listenfd, SA *addr, int *addrlen); /* Returns connfd */ 112 | ``` 113 | 114 | **Socket connect**: Client establishes a connection with a server at the socket address `addr`. If successful, the `clientfd` is ready for reading and writing. No binding with specific address is necessary as there is an internal bind to an ephemeral port. 115 | 116 | ```c 117 | int connect(int clientfd, SA *addr, socklen_t addrlen); 118 | ``` 119 | 120 | ![](images/Pasted%20image%2020220121215213.png) 121 | ![](images/Pasted%20image%2020220121215220.png) 122 | 123 | **Socket get address info** 124 | 125 | The modern way to convert string representations of host addresses, ports, service names to socket address structures. Reentrant, thus can be used in threaded programs. Convenient for the developer as the API returns in Network Byte Order. 126 | 127 | ```c 128 | int getaddrinfo(const chat *host, /* Hostname|address */ 129 | const char *service, /* Port or service */ 130 | const struct addrinfo *hints, /* Input parameters */ 131 | struct addrinfo **result); /* Output linkedlist */ 132 | 133 | void freeaddrinfo(struct addrinfo *result); /* May ret errcode */ 134 | 135 | const char *gai_strerror(int errcode); /* Error message */ 136 | ``` 137 | 138 | The result is a linkedlist of `addrinfo` as sometimes a hostname can map to multiple addresses. For example, _google.com_ has more than one IP address. 139 | 140 | Clients can walk this list to try each `ai_addr` and `ai_addrlen` until `socket` and `connect` succeed. Servers can walk the list until calls to `socket` and `bind` succeed. 141 | 142 | ```c 143 | struct addrinfo { 144 | int ai_flags; /* Hints argument flags */ 145 | int ai_family; /* First arg to socket function */ 146 | int ai_socktype; /* Second arg to socket function */ 147 | int ai_protocol; /* Third arg to socket function */ 148 | char *ai_canonname; /* Canonical host name */ 149 | size_t ai_addrlen; /* Size of ai_addr struct */ 150 | struct sockaddr *ai_addr; /* Ptr to socket address struct */ 151 | struct addrinfo *ai_next; /* Ptr to next addrinfo */ 152 | } 153 | 154 | // ai_flags 155 | // AI_ADDRCONFIG - Recommended for connections 156 | // AI_NUMERICSERV - Service argument is port number instead of name 157 | // AI_PASSIVE - Returns potential passive sockets, host = NULL 158 | ``` 159 | 160 | Example of linked list usage 161 | 162 | ```c 163 | #include "csapp.h" 164 | 165 | int main(int argc, char **argv) { 166 | struct addrinfo *p, *listp, hints; 167 | char buf[MAXLINE]; 168 | int rc, flags; 169 | 170 | memset(&hints, 0, sizeof(struct addrinfo)); 171 | hints.ai_family = AF_INET; /* IPv4 only */ 172 | hints.ai_socktype = SOCK_STREAM; /* TCP connections only */ 173 | if ((rc = getaddrinfo(argv[1], NULL, &hints, &listp)) != 0) { 174 | fprintf(stderr, "getaddrinfo error: %s\n", gai_strerror(rc)); 175 | exit(1); 176 | } 177 | 178 | flags = NI_NUMERICHOST; /* Display address instead of name */ 179 | for (p=listp, p; p= p->ai_next) { 180 | Getnameinfo(p->ai_addr, p->ai_addrlen, 181 | buf, MAXLINE, NULL, 0, flags); 182 | printf("%s\n", buf); 183 | } 184 | 185 | Freeaddrinfo(listp); 186 | exit(0); 187 | } 188 | ``` 189 | 190 | **Socket get name info**: Inverse of `getaddrinfo`, converts socket address to the corresponding host and service. Reentrant and protocol independent. 191 | 192 | ```c 193 | itn getnameinfo(const SA *sa, socklen_t salen, 194 | char *host, size_t hostlen, 195 | char *serv, size_t servlen, 196 | int flags); 197 | 198 | // AI_NUMERICHOST - Return numeric address instead of domain name 199 | // AI_NUMERICSERV - Service argument is port number instead of name 200 | ``` 201 | 202 | **Helper for client to establish connection** 203 | 204 | Establish a client connection with a server and return the socket descriptor. 205 | 206 | ```c 207 | int open_clientfd(char *hostname, char *port) { 208 | int clientfd; 209 | struct addrinfo hints, *listp, *p; 210 | 211 | /* Get a list of potential server addresses */ 212 | memset(&hints, 0, sizeof(struct addrinfo)); 213 | hints.ai_socktype = SOCK_STREAM; /* TCP connection */ 214 | hints.ai_flags = AI_NUMERICSERV; /* Port is numeric */ 215 | hints.ai_flags |= AI_ADDRCONFIG; /* Recommended for conns */ 216 | Getaddrinfo(hostname, port, &hints, &listp); 217 | 218 | /* Walk the list to connect */ 219 | for (p=listp; p; p->ai_next) { 220 | /* Create socket descriptor */ 221 | if ((clientfd = socket(p->ai_family, p->ai_socktype, 222 | p->ai_protocol)) < 0) 223 | continue; 224 | if (connect(clientfd, p->ai_addr, p->ai_addrlen) != -1) 225 | break; /* Success */ 226 | Close(clientfd); 227 | } 228 | 229 | /* Clean up */ 230 | Freeaddrinfo(listp); 231 | if (!p) 232 | return -1; 233 | else 234 | return p; 235 | } 236 | ``` 237 | 238 | **Helper for server to listen** 239 | 240 | Create a listening descriptor that can be used to accept connections. 241 | 242 | ```c 243 | int open_listenfd(char *port) { 244 | struct addrinfo hints, *listp, *p; 245 | int listenfd, optval=1; 246 | 247 | /* Get a list of potential server addresses */ 248 | memset(&hints, 0, sizeof(struct addrinfo)); 249 | hints.ai_socktype = SOCK_STREAM; 250 | hints.ai_flags = AI_PASSIVE | AI_ADDRCONFIG; 251 | hints.ai_flags |= AI_NUMERICSERV; 252 | Getaddrinfo(NULL, port, &hints, &listp); /* This machine */ 253 | 254 | /* Walk the list to bind */ 255 | for (p=listp; p; p->ai_next) { 256 | /* Create socket descriptor */ 257 | if ((listenfd = socket(p->ai_family, p->ai_socktype, 258 | p->ai_protocol)) < 0) 259 | continue; 260 | 261 | /* Eliminates "Address already in use" error from bind */ 262 | Setsockopt(listenfd, SOL_SOCKET, SO_REUSEADDR, 263 | (const void *)&optval, sizeof(int)); 264 | 265 | /* Bind the descriptor to the address */ 266 | if (listen(listenfd, p->ai_addr, p->ai_addrlen) == 0) 267 | break; 268 | Close(listenfd); 269 | } 270 | 271 | /* Clean up */ 272 | Freeaddrinfo(listp); 273 | if (!p) 274 | return -1; 275 | /* Make it a listening socket to accept connection requests */ 276 | if (listen(listenfd, LISTENQ) < 0) { 277 | Close(listenfd); 278 | return -1; 279 | } 280 | return listenfd; 281 | } 282 | ``` 283 | 284 | **Echo client** 285 | 286 | Sends the stdin to the server until EOF character. Prints out data the server sends back. Similar to CLI application `telnet`. 287 | 288 | ```c 289 | #include "csapp.h" 290 | 291 | int main(int argc, char **argv) { 292 | int clientfd; 293 | char *host, *port, buf[MAXLINE]; 294 | rio_t rio; 295 | 296 | host = argv[1]; 297 | port = argv[2]; 298 | clientfd = Open_clientfd(host, port); 299 | Rio_readinitb(&rio, clientfd); 300 | 301 | /* Return on new line, NULL on EOF */ 302 | while (Fgets(buf, MAXLINE, stdin) != NULL) { 303 | Rio_writen(clientfd, buf, strlen(buf)); 304 | 305 | /* Read server's data into buffer */ 306 | Rio_readlineb(&rio, buf, MAXLINE); 307 | Fputs(buf, stdout); 308 | } 309 | /* Client initiates the termination */ 310 | Close(clientfd); 311 | exit(0); 312 | } 313 | ``` 314 | 315 | **Iterative echo server** 316 | 317 | Reads and echo data from the client connnection until EOF (Client closing). Only supports one connection. 318 | 319 | ```c 320 | #include "csapp.h" 321 | void echo(int connfd); 322 | 323 | int main(int argc, char **argv) { 324 | int listenfd, connfd; 325 | socklen_t clientlen; 326 | struct sockaddr_storage clientaddr; /* Enough for any addrtype */ 327 | char client_hostname[MAXLINE], client_port[MAXLINE]; 328 | 329 | listenfd = Open_listenfd(argv[1]); 330 | while (1) { 331 | clientlen = sizeof(struct sockaddr_storage); 332 | connfd = Accept(listenfd, (SA *)&clientaddr, &clientlen); 333 | Getnameinfo((SA *)&clientaddr, clientlen, 334 | client_hostname, MAXLINE, client_port, MAXLINE, 0); 335 | printf("Connected to (%s, %s)\n", 336 | client_hostname, client_port); 337 | echo(connfd); 338 | Close(connfd); 339 | } 340 | exit(0); 341 | } 342 | 343 | void echo(int connfd) { 344 | size_t n; 345 | char buf[MAXLINE]; 346 | rio_t rio; 347 | 348 | Rio_readinitb(&rio, connfd); 349 | /* Read until the client sends EOF */ 350 | while((n = Rio_readlineb(&rio, buf, MAXLINE)) != 0) { 351 | printf("server received %d bytes\n", (int)n); 352 | Rio_writen(connfd, buf, n); 353 | } 354 | } 355 | ``` 356 | 357 | **Web content**: Sequence of bytes with an associated MIME (Multipurpose Internal Mail Externsion). 358 | 359 | **HTTP request**: A request line followed by zero or more request headers. 360 | 361 | ``` 362 | Request line - 363 | Headers -
:
364 | ``` 365 | 366 | **HTTP response**: A response line followed by zero or more response headers followed by content. 367 | 368 | ``` 369 | Response line - 370 | Headers -
:
371 | ``` 372 | 373 | **Tiny web server for static content** 374 | 375 | Serves static and dynamic content for `GET` requests. Full code [here](http://csapp.cs.cmu.edu/2e/ics2/code/netp/tiny/tiny.c). 376 | 377 | In the example below, the server has already received a particular filename and looked up its size. 378 | 379 | ```c 380 | void serve_static(int fd, char *filename, int filesize) { 381 | int srcfd; 382 | char *srcp, filetype[MAXLINE], buf[MAXBUF]; 383 | 384 | get_filetype(filename, filetype); 385 | sprintf(buf, "HTTP/1.0 200 OK\r\n"); 386 | sprintf(buf, "%sServer: Tiny Web Server\r\n", buf); 387 | sprintf(buf, "%sConnection: close\r\n", buf); 388 | sprintf(buf, "%sContent-length: %d\r\n", buf, filesize); 389 | sprintf(buf, "%sContent-type: %s\r\n\r\n", buf, filetype); 390 | 391 | /* Response Body */ 392 | srcfd = Open(filename, O_RDONLY, 0); 393 | srcp = Mmap(0, filesize, PROT_READ, MAP_PRIVATE, srcfd, 0); 394 | Close(srcfd); 395 | Rio_writen(fd, srcp, filesize); 396 | Munmap(srcp, filesize); 397 | } 398 | ``` 399 | 400 | **Issue in serving dynamic content**: The client has to know how to pass the program arguments to the server which then passes to its child process. The server has to capture the content produced by the child process and pass back to the client. 401 | 402 | **Common gateway interface**: Defines a standard for transferring information between the client, the server and the child process. 403 | 404 | Example interface for GET: Arguments list starts with `?` and chains with `&` in the URI. The server creates an environment variable `QUERY_STRING` with all the parameters concatenated. After forking the child process, stdout is redirected to the socket descriptor. The CGI program retrieves arguments from environment variables and outputs to stdout. 405 | 406 | **Tiny web server dynamic content** 407 | 408 | If the request line URI contains "cgi-bin", dynamic content is served. The server creates a child process and runs the program determined by the URI. 409 | 410 | ```c 411 | void serve_dynamic(int fd, char *filename, char *cgiargs) { 412 | char buf[MAXLINE], *emptylist[] = { NULL }; 413 | 414 | sprintf(buf, "HTTP/1.0 200 OK\r\n"); 415 | Rio_writen(fd, buf, strlen(buf)); 416 | sprintf(buf, "Server: Tiny Web Server\r\n"); 417 | Rio_writen(fd, buf, strlen(buf)); 418 | 419 | if (Fork() == 0) { 420 | /* Real server would set all CGI vars here */ 421 | setenv("QUERY_STRING", cgiargs, 1); 422 | Dup2(fd, STDOUT_FILENO); /* Redirect stdout to fd */ 423 | Execve(filename, emptylist, environ); /* Runs CGI program */ 424 | } 425 | Wait(NULL); /* Parent reaps child */ 426 | } 427 | ``` 428 | 429 | Note: the child process knows the content type and length. So it must generate those response headers. 430 | -------------------------------------------------------------------------------- /17-18 - Virtual Memory.md: -------------------------------------------------------------------------------- 1 | **Physical address**: Hardware address in physical memory. Used in simple systems such as embedded microcontrollers. 2 | 3 | **Linear address space**: Ordered set of contiguous non-negative integer addresses. 4 | 5 | Virtual address space: Set of `N = 2^n` virtual addresses. Typically larger than physical address space. The same for all processes running on the system. 6 | 7 | Physical address space: Set of `M = 2^m` physical addresses. Amount of DRAM present in the system. 8 | 9 | **Virtual memory advantages** 10 | 11 | 1. Efficient usage of memory by using DRAM as a cache for an address space stored on disk. 12 | 2. Simplifies memory management for each process by providing uniform address space. 13 | 3. Protects the address space of each process from corruption by other processes. 14 | 15 | **Virtual memory**: A storage allocation scheme in which secondary memory can be addressed as though it were part of the main memory. 16 | 17 | The physical memory (DRAM) is used as a cache. The cache blocks are called virtual pages. 18 | 19 | There are 3 subsets of virtual pages. _Unallocated_ means that the pages have not been created by the VM system. _Cached_ and _Uncached_ refers to the virtual pages in and not in the physical memory. 20 | 21 | ![](images/Pasted%20image%2020220106132621.png) 22 | 23 | **DRAM cache organization**: Driven by enormous miss penalty. DRAM 10x slower than SRAM. Disk 10,000x slower than DRAM. 24 | 25 | 1. Large page size (4kB - 4MB). 26 | 2. Fully associative. Only 1 set to prevent conflict misses. 27 | 3. Highly sophisticated replacement algorithms. 28 | 4. Write-back rather than write-through. 29 | 30 | **Memory management unit (MMU)**: Translates virtual address given by CPU to physical address in memory. 31 | 32 | **Page table**: Array of page table entries (PTEs) that maps virtual pages to physical pages. Per-process kernel data structure in DRAM. 33 | 34 | Page table entry: Each virtual page has a PTE. If the valid-bit is set, the virtual page is _cached_ and the address indicates the page in physical memory. If the valid-bit is not set, the address indicates the location on disk. 35 | 36 | Max virtual memory in 64-bit Windows: 256 TB. 40 bits of space to store physical page address in PTE. 12 bits of addresses per page (4096 bytes). 253 bytes = 256 TB. 37 | 38 | **Page hit**: Reference to a VM word that is in physical memory. 39 | 40 | **Page fault**: The address translation hardware infers that page is not in physical memory and triggers page fault (exception). 41 | 42 | 1. Transfer of control to the kernel's page fault handler 43 | 2. Handler evicts a victim and loads the page from disk into memory 44 | 3. Handler updates the page table with physical address 45 | 4. The faulting instruction is re-executed. 46 | 47 | Demand paging: Strategy of waiting until the last moment to swap in a page into physical memory. 48 | 49 | **Page fault to disk address**: OS maintains a table of virtual address to the data on disk. 50 | 51 | If data is uninitialized, a clean physical page may be mapped. Otherwise, the mapping may be in a page file or swap file or an executable. 52 | 53 | **Allocating page**: `malloc` implements `sbrk` which allocates a page table entry to a location in disk, but no swapping occurs yet. Accessing the page will now trigger a page fault instead of a segmentation fault. 54 | 55 | **Working set**: Set of active virtual pages that are accessed at any point in time. If `Working set < main memory set` -> no disk traffic after compulsory misses. 56 | 57 | **Thrashing**: `SUM(working sets) > main memory` -> performance meltdown when pages are swapped in and out continuously. 58 | 59 | **Sharing data among processes**: Map virtual pages in multiple processes to map to the same physical page. 60 | 61 | This is how shared libraries are implemented. For example, `libc` only has to map into one physical memory. 62 | 63 | **VM as memory management** 64 | 65 | Linking simplified: Uniform virtual address space means that code, data and heap can start at the same addresses after linking. 66 | 67 | Loading simplified: Loader `execve` only allocates virtual pages for `.text` and `.data` sections. These sections are loaded into main memory only on demand. 68 | 69 | Memory allocation simplified: A process can request for contiguous virtual pages even if there is no contiguous physical pages. 70 | 71 | **VM as memory protection** 72 | 73 | Virtual memory extends PTEs with permission bits. For each virtual memory address in one process, it records `SUP`, `READ`, `WRITE`, `EXEC`. MMU checks the bits on each access. 74 | 75 | **Address translation** 76 | 77 | `MAP(a) = a'` if data at virtual address `a` is in physical address `a'`. `MAP(a) = ∅` if data at virtual address `a` is not in physical memory. 78 | 79 | Symbols 80 | 81 | | Symbol | Meaning | 82 | | ----------------- | ------------------------------------------ | 83 | | N = 2n | No. of addresses in virtual address space | 84 | | M = 2m | No. of addresses in physical address space | 85 | | P = 2p | Page size (bytes) | 86 | | TLBI | TLB index | 87 | | TLBT | TLB tag | 88 | | VPO | Virtual page offset | 89 | | VPN | Virtual page number | 90 | | PPO | Physical page offset | 91 | | PPN | Physical page number | 92 | 93 | Address translation with page table 94 | 95 | Page table base register contains the physical address of page table for current process. Virtual page number is used as an index of the page table to retrieve physical page number. The virtual page offset is concatenated afterwards to produce the actual physical address. 96 | 97 | Note: Page table is in memory too. 98 | 99 | ![](images/Pasted%20image%2020220107103316.png) 100 | 101 | Page hit 102 | 103 | Even for a hit, there is still memory reference to retrieve PTE with PTE address (VPN added to page table starting address). 104 | 105 | ![](images/Pasted%20image%2020220107104340.png) 106 | 107 | Page fault 108 | 109 | 1. When valid bit is 0 (uncached), MMU throws page fault exception 110 | 2. Control is transferred to page fault handler 111 | 3. A victim is evicted (Paged to disk if dirty) 112 | 4. The desired page is fetched into memory and PTE is updated 113 | 5. Handler returns control to the original process which restarts the faulting instruction 114 | 115 | ![](images/Pasted%20image%2020220107110147.png) 116 | 117 | **Translation lookaside buffer (TLB)** 118 | 119 | Small set-associative hardware cache in MMU. Maps virtual page numbers to page table entries. If PTEs have been cached in L1 instead, they may be evicted by other data references. 120 | 121 | ![](images/Pasted%20image%2020220107112911.png) 122 | 123 | TLB hit 124 | 125 | Eliminates a memory access as VPN can be used to retrieve PTE inside the on-chip MMU. 126 | 127 | ![](images/Pasted%20image%2020220107113311.png) 128 | 129 | **Multi-level page table** 130 | 131 | Level 1 table resides in memory and each PTE points to a page table. 132 | 133 | ``` 134 | Two-level page table with 32-bit address 135 | 136 | For each virtual address - 137 | 10-bit: 0-1023 idx of level 1 page table 138 | 10-bit: 0-1023 idx of level 2 page table 139 | 12-bit: 4KB virtual page offset 140 | ``` 141 | 142 | ![](images/Pasted%20image%2020220107132719.png) 143 | 144 | Number of page tables depends on the architecture. Level 1 and 2 tables cover a huge swath of address space. They should always be in TLB. If the program has reasonable locality, most of the TLB lookups will generally hit in the TLB. 145 | 146 | Memory requirements reduction: Level 2 page tables do not exist if a PTE in level 1 is null. Since only level 1 page table has to be in the memory all the time, there is less pressure on main memory. 147 | 148 | ![](images/Pasted%20image%2020220115181245.png) 149 | 150 | **Simple memory system example** 151 | 152 | - 64-bytes page size - 26 addresses 153 | - 14-bit virtual address - 8-bit VPN + 6-bit VPO 154 | - 12-bit physical address - 6-bit PPN + 6-bit PPO 155 | - 4-way associative TLB, 16 entries (4 Sets) - 6-bit TLBT + 2-bit TLBI 156 | 157 | _TLB Entry - 6-bit TLBT + 6-bit PPN + 1-bit valid_ 158 | 159 | ![](images/Pasted%20image%2020220115195843.png) 160 | 161 | _Page table with 256 entries (28 virtual page numbers)_ 162 | 163 | ![](images/Pasted%20image%2020220115195851.png) 164 | 165 | _System cache with direct-mapped, 16 sets, 4-byte blocks_ 166 | 167 | 12-bit physical address - 6-bit CT + 4-bit CI + 2-bit CO. 168 | 169 | ![](images/Pasted%20image%2020220115195900.png) 170 | 171 | **Address translation examples** 172 | 173 | TLB hit and cache hit with virtual address: 0x03D4 174 | 175 | ``` 176 | 13 <--------------------- 0 177 | 178 | <- TLBT --> <-> 179 | 0 0 0 0 1 1 1 1 0 1 0 1 0 0 180 | <---- VPN ----> <-- VPO --> 181 | 182 | Check TLB: TLB hit 183 | PPN: 0x0D 184 | 185 | <-- CT ---> <- CI > <-> 186 | 0 0 1 1 0 1 0 1 0 1 0 0 187 | <-- PPN --> <-- PPO --> 188 | 189 | Check cache: Cache hit 190 | Byte: 0x36 191 | ``` 192 | 193 | TLB miss and cache miss with virtual address: 0x0020 194 | 195 | ``` 196 | 13 <--------------------- 0 197 | 198 | <- TLBT --> <-> 199 | 0 0 0 0 0 0 0 0 1 0 0 0 0 0 200 | <---- VPN ----> <-- VPO --> 201 | 202 | Check TLB: TLB miss 203 | Check page table (Valid-bit 1 -> No page fault) 204 | PPN: 0x28 205 | 206 | <-- CT ---> <- CI > <-> 207 | 1 0 1 0 0 0 1 0 0 0 0 0 208 | <-- PPN --> <-- PPO --> 209 | 210 | Check cache: Cache miss 211 | Byte: Access memory 212 | ``` 213 | 214 | **Intel core i7 memory** 215 | 216 | The unified L2 cache ensures that for programs which utilizes more data than instructions or vice versa, the miss penalty for L1 does not grow too large. The time taken to access L3 is much larger since it is located off the CPU chip. 217 | 218 | MMU is also supported by L1 and L2 TLBs. L1 i-TLB's 128 entries > L1 d-TLB's 64 entries. One possible reason that the penalty for missing instructions may be much larger. 219 | 220 | **End-to-end core i7 address translation** 221 | 222 | VPO = PPO = (L1 cache) CI + CO = 12 bits. While address translation is taking place, L1 can be indexed at the same time. Generally TLB is hit, so PPN bits can be used to check against CT bits in the set afterwards. 223 | 224 | ![](images/Pasted%20image%2020220115204939.png) 225 | 226 | Level 1-3 page table entries 227 | 228 | Intel supports 48-bit virtual address space and 52-bit physical address space. Linux uses 4KB pages and the page tables are 4KB aligned (212 addresses). So each page table takes up one virtual page. Only 40 most significant bits of the physical addresses are necessary to be stored as 12 zero-bits can be appended. 229 | 230 | ``` 231 | 63 XD: Disable or enable instructions fetched from the PTE 232 | 233 | 62 234 | .. Unused 235 | 52 236 | 237 | 51 238 | .. Page table physical base address (40 most significant bits) 239 | 12 240 | 241 | 11 242 | .. Unused 243 | 9 244 | 245 | 8 246 | 7 PS: Page size either 4KB or 4MB 247 | 6 248 | 5 A: Reference bit set by MMU on reads and writes 249 | 4 250 | 3 WT: Write-through or write-back cache policy 251 | 2 U/S: User or supervisor access permission 252 | 1 R/W: Read-only or read-write access 253 | 0 P: Child page table present in memory 254 | ``` 255 | 256 | Level 4 page table entries 257 | 258 | ``` 259 | 63 XD: Disable or enable instructions fetched from this page 260 | 261 | 62 262 | .. Unused 263 | 52 264 | 265 | 51 266 | .. Page physical base address (40 most significant bits) 267 | 12 268 | 269 | 11 270 | .. Unused 271 | 9 272 | 273 | 8 274 | 7 275 | 6 D: Dirty bit set by MMU on writes 276 | 5 A: Reference bit set by MMU on reads and writes 277 | 4 278 | 3 WT: Write-through or write-back cache policy 279 | 2 U/S: User or supervisor access permission 280 | 1 R/W: Read-only or read-write access 281 | 0 P: Child page present in memory 282 | ``` 283 | 284 | **Core i7 page tables**: 29 entries per table. Most programs require only one L1 page table entry. 285 | 286 | 1. L1 page table - 512GB region per entry. 287 | 2. L2 page table - 1GB region per entry. 288 | 3. L3 page table - 2MB region per entry. 289 | 4. L4 page table - 4KB region per entry. 290 | 291 | **Virtual address space of a linux process** 292 | 293 | User address space starts with 0 (bottom), kernel address space starts at 1 (top). Kernel exists in virtual address space of each process. 294 | 295 | The "physical memory" region can be used to access physical memory directly. For example, an offset of 0x1 in the region can access physical memory 0x1. 296 | 297 | ![](images/Pasted%20image%2020220115223403.png) 298 | 299 | **Organization of process virtual memory as collection of "areas"** 300 | 301 | An _area_ is a contiguous chunk of allocated virtual memory whose pages are related in some way (Code, heap, data etc). The notion allows the virtual address space to have gaps. 302 | 303 | Each process has a data structure called `task_struct` which contains a pointer to `mm_struct`. 304 | 305 | `mm_struct` contains `pgd` field, the level 1 page table address. When the process is scheduled, the kernel copies `pgd` entry into CR3 register. 306 | 307 | `mm_struct` also contains a pointer to a linked list of `vm_area_struct` which identifies the start/end of each area, their read/write permissions and other flags. 308 | 309 | **Page fault exception handling** 310 | 311 | MMU triggers a page fault if it does not exist in physical memory. The page fault handler carrys out - 312 | 313 | 1. Is the virtual address legal? Is it within an area defined by some `vm_area_struct`? 314 | 2. Is the access legal? Is a store instruction writing to a read-only segment? 315 | 3. If not, the handler evicts a victim and swaps in the page. 316 | 317 | **Memory mapping** 318 | 319 | Linux initializes the contents of a virtual memory area by associating with disk objects. Areas can be backed by regular file on disk (e.g - executable object file) or anonymous file (e.g - nothing). For anonymous file, first fault allocates a physical page full of 0s. 320 | 321 | When the page becomes dirty, it is like any other page. Dirty pages are copied back and forth between memory and a special _swap file_. 322 | 323 | **Shared object** 324 | 325 | Imagine that a process has already mapped a shared disk object into an area of its virtual memory. If another process tries to map the same object, the kernel can point its page table entry to the physical page that is already cached in DRAM. 326 | 327 | **Private object (COW)** 328 | 329 | Similarly, two virtual address spaces map to the same region in physical memory. However, the area struct is flagged as private copy-on-write. 330 | 331 | 1. If an instruction writes to one of the pages, a protection fault is triggered 332 | 2. Fault handler creates a new copy in the physical memory and update the page table entry 333 | 3. The instruction restarts upon handler returning control 334 | 335 | ![](images/Pasted%20image%2020220115230730.png) 336 | 337 | **Fork function virtual address space** 338 | 339 | Creates exact copies of `mm_struct`, `vm_area_struct` and page tables. Then the kernal flags each `vm_area_struct` in both processes as private copy-on-write. New pages are created with the COW mechanism only during subsequent writes. 340 | 341 | **Execve function virtual address space** 342 | 343 | 1. Deletes the existing `vm_area_struct`. 344 | 2. Maps private (COW) areas by creating new area structs for code, data, bss and stack. The code and data are mapped to the executable object file on disk. The bss area is mapped to an anonymous file whose size is contained the executable object file. The stack and heap are also demand-zero, initially zero length. 345 | 3. Maps shared areas such as the standard C library during dynamic linking. 346 | 4. Set the program counter to the entry point. 347 | 348 | ![](images/Pasted%20image%2020220115233259.png) 349 | 350 | **User-level memory mapping** 351 | 352 | Map `len` bytes starting at `offset` from the file descriptor `fd`, preferably at the address `start`. 353 | 354 | ```c 355 | void *mmap(void *start, int len, int prot, int flags, int fd, int offset); 356 | 357 | // start: Choose 0 for no preferred address. 358 | // prot: Protection flags such as PROT_READ, PROT_WRITE... 359 | // flags: MAP_ANON, MAP_PRIVATE, MAP_SHARED... 360 | ``` 361 | 362 | If a virtual address is read after running mmap, the kernel will trigger page fault and the swapped pages will be the contents of the file. 363 | 364 | **Using mmap to copy files example**: Copy a file to stdout without transferring data to user space. The write call reads the bytes which are faulted in by the kernel and writes them to stdout. Avoids a step of buffering before the write command. 365 | 366 | ```c 367 | #include "csapp.h" 368 | 369 | void mmapcopy(int fd, int size) { 370 | char *bufp; 371 | bufp = Mmap(NULL, size, PROT_READ, MAP_PRIVATE, fd, 0); 372 | Write(1, bufp, size); /* STDOUT file descriptor */ 373 | return; 374 | } 375 | 376 | int main(int argc, char **argv) { 377 | struct stat stat; 378 | int fd; 379 | if (argc != 2) { 380 | printf("usage: %s \n", argv[0]); 381 | exit(0); 382 | } 383 | 384 | /* Get file-descriptor */ 385 | fd = Open(argv[1], O_RDONLY, 0); 386 | Fstat(fd, &stat); 387 | mmapcopy(fd, stat.st_size); 388 | exit(0); 389 | } 390 | ``` 391 | 392 | Theoretical difference from read() 393 | 394 | In traditional file I/O involving read, data is copied from the disk to a kernel buffer, then the kernel buffer is copied into the process's heap space for use. In memory-mapped file I/O, data is copied from the disk straight into the process's address space, into the segment where the file is mapped. -------------------------------------------------------------------------------- /14-15 - Exceptional Control Flow.md: -------------------------------------------------------------------------------- 1 | **Control flow**: Sequence of instructions for CPU. 2 | 3 | **Altering control flow**: Jumps and branches, Procedure call and return. Such instructions allow programs to reacts to changes in _program state_ such as internal variables. However, programs may also need to react to changes _system state_ which cannot be captured in internal variables. 4 | 5 | Exceptional control flow: A program reacting to changes in the system state that are not captured in the internal variables. 6 | 7 | | Level | Example | 8 | | ----------- | ------------------------------------------------------------------------------------------------ | 9 | | Hardware | Events detected by the hardware triggers control transfer to exception handlers | 10 | | OS | Kernel transfer control between processes via context switches | 11 | | Application | A process can send a signal to another process to transfer control | 12 | | Application | Program can react to errors by sidestepping the usual track discipline and making nonlocal jumps | 13 | 14 | **Exception**: Abrupt change in the control flow in response to some change in the processor’s state. 15 | 16 | ![](images/Pasted%20image%2020220101223536.png) 17 | 18 | **Exception table**: Every type of exception has a unique number which is indexed in a jump table with the exception handling code. 19 | 20 | Exception number: A number that indexes into the exception table. They can be assigned by processor designers (Example: Page faults, Divide-by-zero) or by kernel developers (Example: System calls, I/O signals). 21 | 22 | ![](images/Pasted%20image%2020220504204043.png) 23 | 24 | **Difference from procedure call** 25 | 26 | 1. The return address pushed onto the stack can be the current instruction or the next. 27 | 2. Processor states such as condition codes may also be pushed onto the stack. 28 | 3. The items are pushed onto the kernel's stack not the user's stack. 29 | 30 | **Types of exception**: Asynchronous exceptions are caused by events external to the processor. Synchronous exception are caused by events that occur as a result of executing an instruction. 31 | 32 | Interrupt (async): I/O devices trigger interrupts by signaling the interrupt pin. Always returns to the next instruction. 33 | 34 | Traps (sync): Intentional exceptions. For example, `syscall` instruction causes a trap and the exception handler calls the appropriate kernel routine. Always returns to the next instruction. 35 | 36 | Faults: Unintentional but possibly recoverable. For example, page fault handler loads the page from disk and returns control to the instruction. May return to the current instruction or `abort`. 37 | 38 | Aborts: Unintentional and unrecoverable. The handler returns control to an `abort` routine. 39 | 40 | **System calls**: Each call has a unique ID number. It looks like a function call but it is actually transferring control to the kernel. 41 | 42 | **System-level function**: C wrapper functions for system calls. For example, when the user calls `open(filename, options)`, the unique ID is placed in %rax before the `syscall` instruction is executed. 43 | 44 | ``` 45 | <__open>: 46 | ... 47 | mov $0x2, %eax # open is syscall #2 48 | syscall # Kernel returns value in %rax. -ve means error 49 | cmp $0xfffffffffffff001, %rax 50 | ... 51 | retq 52 | ``` 53 | 54 | **Process**: Instance of a running program. It provides each program with two key abstractions. 55 | 56 | Logical control flow: Each program seems to have exclusive use of CPU. 57 | 58 | Private address space: Each program seems to have exclusive use of main memory. 59 | 60 | Each program runs within the _context_ of some process. The context consists of states such as program counter, stack, registers, open file descriptors and code/data in memory. 61 | 62 | **Multiprocessing (Single Core)**: Single processor executes multiple processes concurrently. 63 | 64 | Context switch: Saves the context of the current process -> Restores the context of some previous process -> Pass control to the newly restored process. 65 | 66 | ![](images/Pasted%20image%2020220101111331.png) 67 | 68 | **Concurrent process**: Two processes run concurrently if their logical flows overlap in time. 69 | 70 | ``` 71 | ---- time ----> 72 | A -- -- 73 | B -- 74 | C -- -- 75 | 76 | Concurrent: A&B, A&C 77 | Sequential: B&C 78 | ``` 79 | 80 | **System call error handling**: On error, Linux system-level functions return -1 and set global variable `errno` to indicate cause. Must always check return status of every system-level function. 81 | 82 | ```c 83 | if ((pid = fork()) < 0) { 84 | fprintf(stderr, "fork error: %s\n", strerror(errno)); 85 | exit(0); 86 | } 87 | ``` 88 | 89 | **Error-handling wrapper**: Identical interface as the original function with first letter uppercase. 90 | 91 | ```c 92 | pid_t Fork(void) { 93 | pid_t pid; 94 | if ((pid = fork()) < 0) { 95 | fprintf(stderr, "fork error: %s\n", strerror(errno)); 96 | exit(0); 97 | } 98 | return pid; 99 | } 100 | 101 | pid = Fork(); 102 | ``` 103 | 104 | **Process states** 105 | 106 | Running: Either executing or waiting to be executed. 107 | 108 | Stopped: Execution is suspended until further notice. Runs again when the process receives a `SIGCONT` signal. 109 | 110 | Terminated: Process is stopped permanently. Three reasons - Receiving a signal whose default action is to terminate, returning from the `main` routine, calling the `exit` function. 111 | 112 | **Fork**: Parent creates a new running child process with `fork()`. Returns 0 to the child process, returns child's PID to parent process. Cannot predict the execution order of parent and child. 113 | 114 | ```c 115 | int main() { 116 | pid_t pid; 117 | int x = 1; 118 | 119 | pid = Fork(); 120 | if (pid == 0) { 121 | printf("child: x=%d\n", ++x); 122 | exit(0); 123 | } 124 | 125 | printf("parent: x=%d\n", --x); 126 | exit(0); 127 | } 128 | ``` 129 | 130 | Duplicate but separate address space: The address space of each process is identical. Any changes made to a variable is not reflected in another process. 131 | 132 | Shared files: The child process inherits the parent's open files. Since `stdout` file is open in the parent, the child's output is also directed to the screen. 133 | 134 | **Modeling fork with process graphs**: Each vertex is the execution of a statement. Each edge can be labeled with the current value of variables. 135 | 136 | ![](images/Pasted%20image%2020220101122220.png) 137 | 138 | **Nested forks in parent** 139 | 140 | ```c 141 | void fork4() { 142 | printf("LO\n"); 143 | if (fork() != 0) { 144 | printf("l1\n"); 145 | if (fork() != 0) { 146 | printf("L2\n"); 147 | } 148 | } 149 | printf("Bye\n"); 150 | } 151 | ``` 152 | 153 | ![](images/Pasted%20image%2020220101122910.png) 154 | 155 | **Reaping child process** 156 | 157 | When a process terminates, it still consumes system resources in case parent wants to read its states. ITs called a zombie process. 158 | 159 | Parent uses `wait` or `waitpid` to reap the zombie process. If parent does not reap, the orphaned child will be reaped by the `init` process (PID 1) when the parent terminates. 160 | 161 | Reaping is important in long-running processes such as shells and servers. 162 | 163 | **Zombie example** 164 | 165 | ```c 166 | void fork7() { 167 | if (fork() == 0) { 168 | printf("Terminating child, pid=%d\n", getpid()); 169 | exit(0); 170 | } else { 171 | printf("Running parent, pid=%d\n", getpid()); 172 | while (1) 173 | ; /* Infinite loop */ 174 | } 175 | } 176 | ``` 177 | 178 | ![](images/Pasted%20image%2020220101124107.png) 179 | 180 | **Wait**: Parent reaps a child by calling the `wait` function. Parameter is a pointer for the child's status and the return value is the `pid` of the child process that terminated. 181 | 182 | ```c 183 | void fork9() { 184 | int child_status; 185 | 186 | if (fork() == 0) { 187 | printf("HC: hello from child\n"); 188 | exit(0); 189 | } else { 190 | printf("HP: hello from parent\n"); 191 | wait(&child_status); 192 | printf("CT: child has terminated\n"); 193 | } 194 | printf("Bye\n"); 195 | } 196 | ``` 197 | 198 | ![](images/Pasted%20image%2020220101125014.png) 199 | 200 | **Waitpid**: If `pid` argument is -1 the wait set includes all the child processes. This behaviour is the same as `wait`. 201 | 202 | Default behaviour: Suspends the current process until one child in the wait set terminates. 203 | 204 | WNOHANG: Returns immediately (with 0) if none of the child processes have terminated yet. 205 | 206 | **Multiple wait**: If multiple children has terminated, they will be reaped in arbitrary order. Can use macros such as `WIFEXITED` and `WEXITSTATUS` to get more information about the child process. 207 | 208 | ```c 209 | void fork10() { 210 | pid_t pid[N]; 211 | int i, child_status; 212 | 213 | for (i=0; iExample: Run `/bin/ls -lt /usr/include` in a child process with the current environment. 246 | 247 | ```c 248 | /* After creating myargv[] */ 249 | 250 | if ((pid = Fork()) == 0) { 251 | if (execve(myargv[0], myargv, environ) < 0) { 252 | printf("%s: Command not found.\n", myargv[0]); 253 | exit(1); 254 | } 255 | } 256 | ``` 257 | 258 | ![](images/Pasted%20image%2020220101135022.png) 259 | 260 | **Linux process hierachy** 261 | 262 | ``` 263 | [0] 264 | | 265 | init [1] 266 | / \ \ 267 | / \ Login shell 268 | Daemon \ 269 | Login shell 270 | / \ 271 | child child 272 | ``` 273 | 274 | **Shell**: Application program that runs programs on behalf of the user. It takes in an input string and parse into arguments for `execve` in a child process. If it is a foreground job, a `waitpid` is run to reap the child process. 275 | 276 | **Signal**: A small message that notifies a process that an event of some type has occurred in the system. Akin to exceptions and interrupts. Only information in a signal is its ID and the fact that it arrived. 277 | 278 | ID | Name | Action | Event 279 | ------ | ------ | ------ | ------ 280 | 2 | SIGINT | Terminate | User typed ctrl-C 281 | 9 | SIGKILL | Terminate | Kill program (Can't override/ignore) 282 | 11 | SIGSEGV | Terminate | Segmentation violation 283 | 14 | SIGALRM | Terminate | Timer signal 284 | 17 | SIGCHLD | Ignore | Child terminated, stopped or resumed 285 | 286 | Sending a signal: Kernel updates state in the context of the destination process. 287 | 288 | Receiving a signal: Destination process is forced by the kernel to react in some way. 289 | 290 | **Ways to react a signal** 291 | 292 | 1. Ignore the signal. 293 | 2. Terminate the process. 294 | 3. Stop the process until SIGCONT signal. 295 | 4. Catch the signal by executing a user-level function called signal handler. 296 | 297 | Signal handler is similar to a hardware exception handler responding to an asynchronous interrupt. However, signal handlers are in C code and exception handlers are in the kernel. 298 | 299 | **Pending signal**: Only one pending signal of each type is allowed. The subsequent signals of the same type sent to the process are discarded. 300 | 301 | Pending bits: Maintains a 32-bit int where each bit tracks the delivery and receipt of signals. 302 | 303 | **Blocked signal**: Process can block the receipt of certain signals. Can be delivered but will not be received till unblocked. 304 | 305 | Blocked bits: Maintains a 32-bit int where each bit represent blocked signals. 306 | 307 | **Process group**: Each process belongs to one process group. 308 | 309 | **Sending signals with /bin/kill**: Program that sends signal to a process or process group. 310 | 311 | ``` 312 | # Send SIGKILL to process 24818 313 | /bin/kill -9 24818 314 | 315 | # Send SIGKILL to process group 24817 316 | /bin/kill -9 -24817 317 | ``` 318 | 319 | **Sending signals with keyboard** 320 | 321 | Ctrl-C: Sends SIGINT (terminate) to every job in foreground process group. 322 | 323 | Ctrl-Z: Sends SIGTSTP (suspend) to every job in the foreground process group. 324 | 325 | **Sending signals with kill()** 326 | 327 | If PID is zero, the signal is sent to every process in the process group. If PID is less than zero, the signal is sent to the process group |PID|. 328 | 329 | ```c 330 | void fork12() { 331 | pid_t pid[N]; 332 | int i; 333 | int child_status; 334 | 335 | for (i=0; i 371 | typedef void (*sighandler_t)(int); 372 | sighandler_t signal(int _signum_, sighandler_t handler); 373 | 374 | // Values for sighandler_t 375 | // SIG_IGN: Ignore 376 | // SIG_DFL: Default action for type signum 377 | // Otherwise, address of the user-level signal handler 378 | 379 | // Example 380 | 381 | void sigint_handler(int sig) { 382 | printf("Signal received is %d\n", sig); 383 | } 384 | 385 | int main() { 386 | /* Install signal handler */ 387 | if (signal(SIGINT, sigint_handler) == SIG_ERR) 388 | unix_error("signal error"); 389 | 390 | /* Wait for receipt of a signal */ 391 | pause(); 392 | 393 | return 0; 394 | } 395 | ``` 396 | 397 | Handler for a signal of type T can be interrupted by a signal of another type. The same type of signal is implicitly blocked. 398 | 399 | ![](images/Pasted%20image%2020220102114944.png) 400 | 401 | **Signal handler as concurrent flow** 402 | 403 | A signal handler is a separate logical flow (not process) that runs concurrently with the main program. Shared global address space. 404 | 405 | ``` 406 | -- time --> 407 | process A: while(1); |-- -- 408 | process A: handler(){...}; | -- 409 | process B: | -- -- 410 | ``` 411 | 412 | ![](images/Pasted%20image%2020220102114655.png) 413 | 414 | **Explicitly blocking signals** 415 | 416 | ```c 417 | sigset_t mask, prev_mask; 418 | 419 | Sigemptyset(&mask); 420 | Sigaddset(&mask, SIGINT); 421 | 422 | /* Block SIGINT and save prev blocked set */ 423 | Sigprocmask(SIG_BLOCK, &mask, &prev_mask); 424 | ... 425 | (Will not be interrupted by SIGINT) 426 | ... 427 | /* Restore prev blocked set */ 428 | Sigprocmask(SIG_SETMASK, &prev_mask, NULL); 429 | ``` 430 | 431 | **Writing safe handlers** 432 | 433 | 1. Keep handlers as simple as possible. 434 | 2. Call only async-signal-safe functions in the handlers. 435 | 3. Save and restore `errno` on entry and exit. 436 | 4. Protect accesses to shared data structures by blocking all signals temporarity. 437 | 5. Declare global variables as `volatile` to prevent unexpected optimizations. (Read/write from memory directly) 438 | 6. Declare global flags as `volatile sig_atomic_t`. (Read/write in one uninterruptible step) 439 | 440 | **Async-signal-safety**: A function is either reentrant (All variables on stack frame) or non-interruptible by signals. 441 | 442 | For example, suppose a program is in the middle of a call to `printf` and a signal occurs whose handler itself calls `printf`. In this case, the output of the two `printf` statements would be intertwined. 443 | 444 | This problem cannot be solved by using synchronization primitives because any attempt would produce immediate deadlock. 445 | 446 | **Signals are not queued example** 447 | 448 | Signals are not queued. So the receipt of one signal means may be more than one signal of the same type has been sent. 449 | 450 | ```c 451 | // SIGCHLD means at least one children has terminated 452 | 453 | int ccount = 0 454 | void child_handler(int sig) { 455 | int olderrno = errno; 456 | pid_t pid; 457 | while ((pid = wait(NULL)) > 0) { 458 | ccount--; 459 | Sio_puts("Handler reaped a child\n"); 460 | } 461 | if (errno != ECHILD) 462 | Sio_error("wait error"); 463 | errno = olderrno; 464 | } 465 | 466 | void fork14() { 467 | pid_t pid[N]; 468 | int i; 469 | ccount = N; 470 | Signal(SIGCHLD, child_handler); 471 | 472 | for (i=0; i 0) /* Parent spins */ 480 | ; 481 | } 482 | ``` 483 | 484 | **Portable signal handling with sigaction** 485 | 486 | Different versions of Unix can have different signal handling semantics. This wrapper function can be used to unify the behaviour. 487 | 488 | ```c 489 | sighandler_t *Signal(int signum, sighandler_t handler) { 490 | struct sigaction action, old_action; 491 | 492 | action.sa_handler = handler; 493 | sigemptyset(&action.sa_mask); /* Block sigs of type being handled */ 494 | action.sa_flags = SA_RESTART; /* Restart syscalls if possible */ 495 | 496 | if (sigaction(signum, &action, &old_action) < 0) 497 | unix_error("Signal error"); 498 | 499 | return (old_action.sa_handler); 500 | } 501 | ``` 502 | 503 | **Synchronizing flows to avoid races** 504 | 505 | If SIGCHLD is not blocked before forking, deletejob() in the child process may run before addjob() in the parent process. 506 | 507 | ```c 508 | void handler(int sig) { 509 | int olderrno = errno; 510 | sigset_t mask_all, prev_all; 511 | pid_t pid; 512 | 513 | Sigfillset(&mask_all); 514 | while((pid = waitpid(-1, NULL, 0)) > 0) { /* Reap child */ 515 | Sigprocmask(SIG_BLOCK, &mask_all, &prev_all); 516 | deletejob(pid); /* Delete child from job list */ 517 | Sigprocmask(SIG_SETMASK, &prev_all, NULL); 518 | } 519 | if (errno != ECHILD) 520 | Sio_error("waitpid error"); 521 | errno = olderrno; 522 | } 523 | 524 | int main(int argc, char **argv) { 525 | int pid; 526 | sigset_t mask_all, mask_one, prev_one; 527 | 528 | Sigfillset(&mask_all); 529 | Sigemptyset(&mask_one); 530 | Sigaddset(&mask_one, SIGCHLD); 531 | Signal(SIGCHLD, handler); 532 | initjobs(); /* Initialize job list */ 533 | 534 | while(1) { 535 | Sigprocmask(SIG_BLOCK, &mask_one, &prev_one); // Blk SIGCHLD 536 | if ((pid = Fork()) == 0) { // Child 537 | Sigprocmask(SIG_SETMASK, &prev_one, NULL); // Unblock 538 | Execve("/bin/date", argv, NULL); 539 | } 540 | Sigprocmask(SIG_BLOCK, &mask_all, NULL); 541 | addjob(pid); // Add child to list 542 | Sigprocmask(SIG_SETMASK, &prev_one, NULL); // Unblock 543 | } 544 | exit(0); 545 | } 546 | ``` 547 | 548 | **Waiting for signals with sigsuspend** 549 | 550 | Uninterruptible version of the commands below. Atomicity eliminates the possibility that the signal is received after the call to `sigprocmask` and before `pause`. 551 | 552 | ```c 553 | sigprocmask(SIG_BLOCK, &mask, &prev); 554 | pause(); 555 | sigprocmask(SIG_SETMASK, &prev, NULL); 556 | ``` 557 | 558 | While condition is needed to ensure that the execution is suspended if other signals, such as SIGINT, are handled by `pause`. 559 | 560 | ```c 561 | volatile sig_atomic_t pid; 562 | 563 | void sigchld_handler(int s) { 564 | int olderrno = errno; 565 | pid = Waitpid(-1, NULL, 0); 566 | errno = olderrno; 567 | } 568 | void sigint_handler(int s){} 569 | 570 | int main() { 571 | sigset_t mask, prev; 572 | Signal(SIGCHLD, sigchld_handler); 573 | Signal(SIGINT, sigint_handler); 574 | Sigemptyset(&mask); 575 | Sigaddset(&mask, SIGCHLD); 576 | 577 | while (1) { 578 | Sigprocmask(SIG_BLOCK, &mask, &prev); // Block SIGCHLD 579 | if (Fork() == 0) 580 | exit(0); 581 | 582 | /* Wait for SIGCHLD to be received */ 583 | pid = 0; 584 | while (!pid) 585 | Sigsuspend(&prev); // Pause for any signal 586 | 587 | /* Optionally unblock SIGCHLD */ 588 | Sigprocmask(SIG_SETMASK, &prev, NULL); 589 | 590 | /* Do more work */ 591 | } 592 | exit(0); 593 | } 594 | ``` -------------------------------------------------------------------------------- /05-09 - Machine Level.md: -------------------------------------------------------------------------------- 1 | **x86: Intel processor line** 2 | 3 | | Name | Description | 4 | | ---------- | ---------------------------------------------------- | 5 | | 8086 | First 16-bit Intel processor with 1 MB address space | 6 | | i386 | First 32-bit Intel processor (IA32), Can run Unix | 7 | | Pentium 4E | First 64-bit processor (x86-64), Hyperthreading | 8 | | Core 2 | First multi-core Intel processor, No hyperthreading | 9 | | Core i7 | Four cores, Hyperthreading | 10 | 11 | **Assembly Code View** 12 | 13 | Program Counter: Address of next instruction. 14 | 15 | Register File: Heavily used program data. 16 | 17 | Condition Code: Status information about the most recently executed instruction. 18 | 19 | Memory: Virtual address. 20 | 21 | Assembly code makes no distinction between signed, unsigned integers or even pointers. 22 | 23 | **C to Assembly**: `gcc -Og -S sum.c` 24 | 25 | ![](images/Pasted%20image%2020211127174426.png) 26 | 27 | The program executed by the machine is simply a sequence of bytes encoding a series of instructions. 28 | 29 | **Assembly code suffix** 30 | 31 | | Data type | Suffix | Bytes | 32 | | ---------------- | ------ | ----- | 33 | | Byte | b | 1 | 34 | | Word | w | 2 | 35 | | Double word | l | 4 | 36 | | Quad word | q | 8 | 37 | | Single precision | s | 4 | 38 | | Double precision | l | 8 | 39 | 40 | Note that `l` is used for double word and double precision, but there is no ambiguation since floating-point operations use different set of instructions. 41 | 42 | Note that 32-bit instructions will zero the top 32 bits of 64-bit registers while 8-bit and 16-bit instructions do not. 43 | 44 | ``` 45 | movabsq $0xffffffffffffffff, %rax 46 | 47 | movb $0, %al # rax = 0xffffffffffffff00 48 | movw $0, %ax # rax = 0xffffffffffff0000 49 | movl $0, %eax # rax = 0x0000000000000000 50 | ``` 51 | 52 | **x86-64 Integer Registers** 53 | 54 | | Register | Meaning | 55 | | -------- | ------------------------ | 56 | | %exx | 32-bit | 57 | | %rxx | 64-bit | 58 | | %rsp | Stack register | 59 | | %rdi | First argument register | 60 | | %rsi | Second argument register | 61 | | %rdx | Third argument register | 62 | | %rip | Instruction pointer | 63 | 64 | **Memory Addressing Mode** 65 | 66 | Normal: (R) 67 | 68 | Displacement: D (R) # Mem[Reg[R] + D] 69 | 70 | Indexed: D (Rb,Ri) # Mem[Reg[Rb] + Reg[Ri] + D] 71 | 72 | Scaled indexed: D (Rb,Ri,S) # Mem[Reg[Rb] + S\*Reg[Ri] + D] 73 | 74 | ``` 75 | %rdx 0xf000 76 | %rcx 0x0100 77 | 78 | 0x8 (%rds) -> 0xf000 + 0x8 -> 0xf008 79 | (%rdx, %rcx) -> 0xf000 + 0x100 -> 0xf100 80 | (%rdx, %rcx, 4) -> 0xf000 + 4*0x100 -> 0xf400 81 | 0x80(, %rdx, 2) -> 2*0xf000 + 0x80 -> 0x1e080 82 | ``` 83 | 84 | **Moving Data**: `movq Source, Dest` 85 | 86 | Immediate: $0x400, Register: %rax, Memory: (%rax). Memory-memory transfer is not allowed. 87 | 88 | Immediate source operands are limited to 32-bit two's complement numbers. The `movabsq` can have 64-bit immediate value as the source operand and can only have a register as the destination. 89 | 90 | Instructions in `movz` class fill out the remaining bytes of the destination with zeros. Instructions in `movs` class fills out with sign extension. 91 | 92 | Casting example 93 | 94 | ``` 95 | *dp = (dest_t) *sp; 96 | 97 | // unsigned char to long 98 | movzbl (%rdi), %eax // Zero-extension because unsigned 99 | movq %rax, (%rsi) // Store 8 bytes 100 | 101 | // int to char 102 | movl (%rdi), %eax // Read 4 bytes 103 | movb %al, (%rsi) // Store low-order byte 104 | ``` 105 | 106 | 107 | **Load Effective Address**: `leaq Src, Dest` 108 | 109 | 1. It can compute the address without a memory reference. 110 | 2. It can compute arithmetic expressions of the form x + k * y. 111 | 112 | In `movq` (and all other instruction except `leaq`), these parentheses denote a genuine dereference, whereas in `leaq` they do not and are purely convenient syntax. 113 | 114 | ``` 115 | x * 12 = ? 116 | 117 | leaq (%rdi, %rdi, 2), %rax # 3 * x 118 | salq $2, %rax # (3 * x) << 2 119 | ``` 120 | 121 | **Arithmetic Operations** 122 | 123 | | Format | Computation | 124 | | ------ | ---------------------------- | 125 | | addq | Dest = Dest + Src | 126 | | subq | Dest = Dest - Src | 127 | | imulq | Dest = Dest * Src | 128 | | salq | Dest = Dest << Src | 129 | | sarq | Dest = Dest >> Src | 130 | | shrq | Dest = Dest >> Src (Logical) | 131 | | xorq | Dest = Dest ^ Src | 132 | | andq | Dest = Dest & Src | 133 | | orq | Dest = Dest \| Src | 134 | 135 | **One Operand Instructions** 136 | 137 | | Format | Computation | 138 | | ------ | --------------- | 139 | | incq | Dest = Dest + 1 | 140 | | decq | Dest = Dest - 1 | 141 | | negq | Dest = - Dest | 142 | | notq | Dest = ~Dest | 143 | 144 | **Example Arithmetic Expressions** 145 | 146 | ![](images/Pasted%20image%2020211127190654.png) 147 | 148 | **Condition Code registers** 149 | 150 | CF: Carry flag. Overflow for unsigned. 151 | 152 | ZF: Zero flag. Operation yielded zero. 153 | 154 | SF: Sign flag. Operation yielded negative value. 155 | 156 | OF: Overflow flag. Overflow for two's complement. 157 | 158 | **Explicit set with cmp**: `cmpq src, dest` 159 | 160 | ``` 161 | cmpq b, a is like a - b without a destination 162 | 163 | CF set if unsigned overflow 164 | ZF set if a - b == 0 165 | SF set if a - b < 0 166 | OF set if two complement overflow (a > 0 && b < 0 && (a-b) < 0) || ... 167 | ``` 168 | 169 | **Explicit set with test**: `testq src1, src2` 170 | 171 | ``` 172 | testq b, a is like a & b without a destination 173 | 174 | ZF set if a & b == 0 175 | SF set if a & b < 0 176 | ``` 177 | 178 | **SetX**: Set low-order byte destination to 0 or 1 based on combinations of condition codes. Does not affect other 7 bytes. 179 | 180 | Note how (SF ^ OF) is manipulated for signed comparisons. 181 | 182 | | Format | Computation | Description | 183 | | ------ | ---------------- | ------------------------- | 184 | | sete | ZF | Equal / Zero | 185 | | setne | ~ZF | Not equal / Not Zero | 186 | | sets | SF | Negative | 187 | | setns | ~SF | Nonnegative | 188 | | setg | ~(SF ^ OF) & ~ZF | Greater (Signed) | 189 | | setge | ~(SF ^ OF) | Greater or equal (Signed) | 190 | | setl | (SF ^ OF) | Less | 191 | | setle | (SF ^ OF) \| ZF | Less or equal (Signed) | 192 | | seta | ~CF & ~ZF | Above (Unsigned) | 193 | | setb | CF | Below (Unsigned) | 194 | 195 | Condition example 196 | 197 | ``` 198 | int gt (long x, long y) { 199 | return x > y; 200 | } 201 | 202 | # rdi -> x 203 | # rsi -> y 204 | # rax -> return val 205 | 206 | gt: 207 | cmpq %rsi, %rdi # Compare x:y 208 | setg %al # Set when > (Can only one byte) 209 | movzbl %al, %eax # Zero other bytes 210 | ret 211 | 212 | # When result is 32-bit result, zeros are added for the other 32-bits 213 | ``` 214 | 215 | **jX**: Jump instructions 216 | 217 | | Format | Computation | Description | 218 | | ------ | ---------------- | ------------------------- | 219 | | jmp | 1 | Unconditional | 220 | | je | ZF | Equal / Zero | 221 | | jne | ~ZF | Not equal / Not Zero | 222 | | js | SF | Negative | 223 | | jns | ~SF | Nonnegative | 224 | | jg | ~(SF ^ OF) & ~ZF | Greater (Signed) | 225 | | jge | ~(SF ^ OF) | Greater or equal (Signed) | 226 | | jl | (SF ^ OF) | Less | 227 | | jle | (SF ^ OF) \| ZF | Less or equal (Signed) | 228 | | ja | ~CF & ~ZF | Above (Unsigned) | 229 | | jb | CF | Below (Unsigned) | 230 | 231 | PC-relative addressing: Jump instruction commonly use this addressing, which allows encodings to take less bytes and be functional even when the code is shifted around memory. 232 | 233 | For example, the jump instruction at 0x3 jumps to 0x8. The relative address 0x3 is added to %rip at 0x5. 234 | 235 | ``` 236 | 0x0: 48 89 f8 mov %rdi,%rax 237 | 0x3: eb 03 jmp 0x8 238 | 0x5: 48 d1 f8 sar %rax 239 | 0x8: 48 85 c0 test %rax,%rax 240 | ``` 241 | 242 | Condition Branch Example 243 | 244 | ``` 245 | int absdiff (long x, long y) { 246 | long result; 247 | if (x > y) 248 | result = x - y; 249 | else 250 | result = y - x; 251 | return result 252 | } 253 | 254 | absdiff: 255 | cmpq %rsi, %rdi # Compare x:y 256 | jle .L4 # Jump when <= 257 | movq %rdi, %rax 258 | subq %rsi, %rax 259 | ret 260 | .L4: 261 | movq %rsi, %rax 262 | subq %rdi, %rax 263 | ret 264 | ``` 265 | 266 | **Conditional Moves**: Compute both if and else statements first. Only after that the choosing is done with the condition. Because branches are very disruptive to instruction flow (Interferes with pipelining). 267 | 268 | Not good for complex computations, risky computations, computations with side effects. 269 | 270 | ``` 271 | val = Test ? Then_exp : Else_exp; 272 | 273 | # Branching Logic 274 | 275 | ntest = !Test; 276 | if (nTest) goto Else; 277 | val = Then_exp; 278 | goto Done; 279 | Else: 280 | val = Else_exp; 281 | Done: 282 | ... 283 | 284 | # Conditonal Move Logic 285 | 286 | result = Then_exp; 287 | eval = Else_exp; 288 | nt = !Test; 289 | 290 | if (nt) result = eval; 291 | return result; 292 | ``` 293 | 294 | Example 295 | 296 | ``` 297 | absdiff: 298 | movq %rdi, %rax # x 299 | subq %rsi, %rax # result = x - y 300 | movq %rsi, %rdx 301 | subq %rdi, %rdx # eval = y - x 302 | cmpq %rsi, %rdi # Compare x:y 303 | cmovle %rdx, %rax # if <=, result = eval 304 | ret 305 | ``` 306 | 307 | **While loop** 308 | 309 | Jump-to-middle" translation 310 | 311 | ``` 312 | while (Test) 313 | Body 314 | 315 | # Goto version logic -Og 316 | 317 | goto Test; 318 | loop: 319 | Body 320 | test: 321 | if (Test) 322 | goto loop; 323 | done: 324 | ``` 325 | 326 | Guarded-do translation 327 | 328 | ``` 329 | # Goto version logic -O1 330 | 331 | if (!Test) 332 | goto done; 333 | loop: 334 | Body 335 | if (Test) 336 | goto loop; 337 | done: 338 | ``` 339 | 340 | **For loop**: Translate for-loop into while-loop. 341 | 342 | ``` 343 | for (Init; Test; Update) 344 | Body 345 | 346 | Init; 347 | while (Test) { 348 | Body; 349 | Update; 350 | } 351 | ``` 352 | 353 | **Switch statement**: 354 | 355 | The asterisk `*` means an indirect jump. Jump to the value at the address, and not the direct address. 356 | 357 | ![](images/Pasted%20image%2020211128235325.png) 358 | 359 | ``` 360 | long switch_eg(long x, long y, long z) { 361 | long w = 1; 362 | switch (x) { 363 | ... 364 | case 2: 365 | w = y/z; 366 | case 3: 367 | w += z; 368 | break; 369 | ... 370 | case 6: 371 | w -= x; 372 | break; 373 | default; 374 | w = 2; 375 | } 376 | return w; 377 | } 378 | 379 | 380 | switch_eg: 381 | movq %rdx, %rcx 382 | cmpq $6, %rdi # Compare x to 6 383 | ja .L8 # Jump default if x < 0 or x > 6 384 | jmp *.L4(,%rdi,8) # goto *JTab[x] 385 | 386 | 387 | # Jump Table 388 | 389 | .section .rodata 390 | .align 8 # Spaced by 8 bytes 391 | .L4 392 | .quad .L8 393 | .quad .L3 394 | .quad .L5 # Case 2 395 | .quad .L9 # Case 3 396 | .quad .L8 397 | .quad .L7 398 | .quad .L7 399 | 400 | # Code Blocks (x == 2, x == 3) 401 | 402 | .L5 # Case 2 403 | movq %rsi, %rax 404 | cqto 405 | idivq %rcx # y/z 406 | jmp .L6 407 | .L9 # Case 3 408 | movl $1, %eax # w = 1 409 | .L6 410 | addq %rcx, %rax # w += z 411 | ret 412 | ``` 413 | 414 | **Mechanisms in procedures** 415 | 416 | 1. Passing Control: Program counter must change accordingly. 417 | 2. Passing Data: One or more parameters and one return value. 418 | 3. Memory Management: Allocate and free local variables. 419 | 420 | **x86-64 stack**: Register %rsp contains the lowest stack address. 421 | 422 | **Push onto stack**: `pushq src` 423 | 424 | Fetch operand at src -> decrement %rsp by 8 -> write operand at the new %rsp address. 425 | 426 | **Pop from stack**: `popq dest` 427 | 428 | Read value from %rsp address -> increment %rsp by 8 -> store the value at dest. 429 | 430 | **Call procedure**: `callq