├── EngineeringTerms.md ├── README.md ├── distributed-systems └── distributed-systems.md ├── infrastructure-engineering ├── CloudComputing.md └── Containers.md ├── languages ├── Clojure.md ├── Elixir.md ├── Go.md ├── Javascript.md ├── Lisp.md ├── Lua.md ├── Nim.md ├── Pony.md ├── Ruby.md └── Rust.md ├── operating-systems └── ostep.md └── protocols └── HTTP.md /EngineeringTerms.md: -------------------------------------------------------------------------------- 1 | ## Important Software Engineering Terms and their Definitions 2 | 3 | As a newbie developer you must have come across certain terms while reading 4 | the documentation or a blog post or any kind of technical writing related to 5 | software development that you didn't understand. Ever read something like "low latency and high throughput" and scratched your head then this is for you. 6 | 7 | - API 8 | - Latency 9 | - throughput 10 | - Middleware 11 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ## Notebooks 2 | 3 | My collection of notes :notebook: on various subjects related to software :computer: engineering. 4 | 5 | ### Table of Contents 6 | - [Programming Languages](https://github.com/palash25/Notes/tree/master/languages) 7 | - [Clojure](https://github.com/palash25/Notes/blob/master/languages/Clojure.md) 8 | - [Elixir](https://github.com/palash25/Notes/blob/master/languages/Elixir.md) 9 | - [Go](https://github.com/palash25/Notes/blob/master/languages/Go.md) 10 | - [Lisp](https://github.com/palash25/Notes/blob/master/languages/Lisp.md) 11 | - [Lua](https://github.com/palash25/Notes/blob/master/languages/Lua.md) 12 | - [Nim](https://github.com/palash25/Notes/blob/master/languages/Nim.md) 13 | - [Pony](https://github.com/palash25/Notes/blob/master/languages/Pony.md) 14 | - [Ruby](https://github.com/palash25/Notes/blob/master/languages/Ruby.md) 15 | - [Rust](https://github.com/palash25/Notes/blob/master/languages/Rust.md) 16 | - [Distributed Systems]() **(WIP)** 17 | - [Concurrent Programming]() 18 | - [Infrastructure Engineering](https://github.com/palash25/Notes/tree/master/infrastructure-engineering) 19 | - [Containers](https://github.com/palash25/Notes/blob/master/infrastructure-engineering/Containers.md) 20 | - [Protocols]() **(WIP)** 21 | - [JSON RPC]() 22 | - [Language Server]() 23 | - [HTTP]() 24 | - [SSH]() 25 | -------------------------------------------------------------------------------- /distributed-systems/distributed-systems.md: -------------------------------------------------------------------------------- 1 | # Introduction 2 | 3 | ## What is a distributed system? 4 | 5 | A group of computers working together that to its user seems as if they are 6 | a single computer. 7 | 8 | ## Characteristics 9 | 10 | - These computers work concurrently 11 | - They fail independently 12 | - They do not share a global clock (The computing that each machine does is 13 | asynchronous to the other computers in the system). 14 | 15 | Examples: Amazon.com and Cassandra DB. 16 | A personal latop cannot be considered a dist-sys since all the peripherals 17 | share the same data bus (global clock) and hence the opterations performed by 18 | them are synchronous. 19 | 20 | ## Distributed Storage 21 | 22 | Simple systems can have a single master database for their storage needs but 23 | what if the consumers of that system grow and you are to scale the system to 24 | handle the number of users. In this case we use read replication. 25 | 26 | **Read Replication:** Let's say that the number of consumers of our system increase 27 | then we can scale the system using multiple databases. 28 | 29 | The original database remains the Master DB (leader node) and the rest are Slave DBs (follower node). 30 | 31 | ``` 32 | Slave A 33 | -------- 34 | | | 35 | | | 36 | -------- 37 | ^ 38 | ---------- __________| 39 | | | | 40 | | Master | -----------| 41 | | | |__________ 42 | ---------- | 43 | | 44 | V 45 | -------- 46 | | | 47 | | | 48 | -------- 49 | Slave B 50 | ``` 51 | 52 | Whenever there is a change in an entry of the DB the change is made in the leader 53 | DB and then it is propogated to the followers. 54 | This has the following effects: 55 | - In case of large read traffic read replication seems to solve the problem. 56 | - This reduces the consistency of the DB meaning that an entry might not be properly 57 | updated in the follower nodes at the time of the query but this is something that 58 | a DistSys developer have to accept. 59 | - This is only beneficial in case of read heavy workloads. 60 | 61 | **Sharding:** In case of write traffic heavy workload we can have multiple masters 62 | or divide the master DB into *shards*. 63 | 64 | Now each master (or shard) will have its set of slave DBs and also each master DB 65 | will contain the records for a specific range. For example in a DB of records 66 | containing keys from A to Z, shard1 will cotain records from A to L, shard2 will 67 | contain records from M to S and shard3 from T to Z and their corresponding slaves will 68 | have the exact copies of the records. 69 | 70 | This can also be seen as a multiple sets of read replications 71 | 72 | This has the following demerits: 73 | - More Complexities: This model also introduces the complexities of adding a feature 74 | to know which shard to query for the records that you want. For this another layer has 75 | to be added to the application or an ORM hack that will automate the process of directing 76 | the query to its corresponding shard. 77 | - Limited Data Model: Sharding works well only with a key-value store. It has to be 78 | the case that every query we issue should have a single common key. For e.g. SaaS 79 | business like Trello every query will have the user-id as the key this kind of model 80 | is shardable. 81 | - Limited Data Access Patterns: Lets say that we include some metadata in our key-value 82 | store and makes it difficult to shard. For example if we add who are the most frequent customers and have a query to fetch the most active customers that is an analytic query. For this kind of query we can go to each read replica on the shard and scatter that query on all of those and gather the results back (scatter-gather). Scatter-Gather is possible but shouldn't be considered as an option if there are a lot of such queries that reqiure scatter-gather 83 | 84 | 85 | ### CAP Theorem 86 | **C** *onsistency*: When we do a read from a DB we should get the most recent update written. 87 | 88 | **A** *vailability*: The DB should always be ready for read and write operation. If we try to do a read or write it shouldn't tell us that "you cannot read/write at this time". 89 | 90 | **P** *artition* Tolerance: When a part of the distributed system or DB is disconnected or 91 | out of sync from the rest of the system then that is called a partition. 92 | 93 | 94 | The CAP thereom states that we can never have all three of the properties together in a 95 | distributed system. For any given system **P** will always be there since each node in a 96 | distributed system fails independently. So the possible combinations one can have are **CP** and **AP**. A distributed systems engineer has to choose between either C or A. 97 | 98 | **CP**: Lets say there is a network partition and one of the components goes out of system 99 | with the rest of the system and we choose to have consistency. If we do a read operation at 100 | this time it wouldn't be possible since the consistent system will choose not to share any 101 | data that might have gone out of sync with the other component rather than being available. 102 | 103 | **AP**: In this case a read operation will get an answer from the DB which can be out of sync and wrong but since we prioritized availability we cannot get consistency. 104 | 105 | 106 | ## Distributed Transaction 107 | **ACID** Transaction: Stands for Atomic Consistent Isolated Durable. 108 | 109 | - Atomic: The transaction (a bundle of DB statements) either runs completely 110 | (all the statetments are executed) or not at all. 111 | 112 | - Consistent: After the transaction is complete the DB is left in a valid state 113 | (meaning writing to the DB / when the transaction completes we don't break 114 | anything in the DB) 115 | 116 | - Isolated: Different transactions when trying to access the same elements in 117 | a DB can't see each other. For example there is a transaction running that is 118 | updating a block in the DB and another transaction tries to read or write to 119 | the same block then the second transaction wouldn't be able to see the changes 120 | made by the first one until after it completes. 121 | Complete isolation is hard to achieve in a distributed system. 122 | 123 | - Durable: The DB remembers everything after a transaction commits. 124 | 125 | **ACID with an example:** Lets take the example of a coffeshop and various steps 126 | involved in getting a customer their coffee 127 | 128 | *Steps* 129 | ``` 130 | - Receive order -------------\ 131 | - Process Payment ------------- >========> carried out by a separate actor (waiter) 132 | - Enqueue order -------------/ 133 | 134 | - Make coffee ----------------> Carried out by another actor 135 | - Deliver it ----------------> or two individual actors (barista(s)) 136 | ``` 137 | This work is split up to achieve *parallelization*. 138 | This kind of system can contain *uneven workloads* too (making the coffee takes longer than the payment or deliver). 139 | 140 | **Note:** Enqueuing is an example of distributed messaging. 141 | 142 | *Possible failures* 143 | - Payment failure 144 | - Insufficient resources 145 | - Equipment failure 146 | - Worker failure 147 | - Consumer failure 148 | 149 | In case of an acid (*emphasis on the atomic part of the acronym*) transaction all the steps described in a previous section are wrapped together as one operation and if any one of the steps fail then the whole operation fails and has to roll back 150 | 151 | *Possible Response to Failure* 152 | - Write-off: In case of consumer failure we throw out the drink (discard the work without advancing to the next step) 153 | - Retry: If there is an equipment or payment failure then we can always retry 154 | - Compensating action: Customer has paid and you cannot produce the coffee then you try to compensate it by executing a different transaction (pay them back) 155 | 156 | **A coffee shop doesn't follow dist transactional model since that would lead to a lower business or profit (lower throughput). This is why coffe shops don't follow atomic transaction to get a higher throughput and lower latency** 157 | 158 | ## Distribution Computation 159 | 160 | Once a distributed storage has been deployed there needs to be some analysis done on the stored date. It follows the scatter gatther paradigm. 161 | 162 | So our date is distributed amongst a network of computers (dist storage) to do some 163 | analysis different programs are sent to these distributed storage units to do some computation on the data contained in those locally (scatter) and then the analysis is returned back (gather). [The programs are moved close to the data and ran on it] 164 | 165 | ### Map Reduce 166 | 167 | This mathematical model consists of two functions `Map` and `Reduce`. This model 168 | as stated earlier works on the `Scatter-Gather` principle. Process consists of 169 | 3 steps: 170 | 1. `Mapper` function: This function accepts a bunch of key-value pairs as inputs 171 | and produces a list of key-value pairs as the output of the function. Ex- The 172 | input is a file path (key) and the text content (value) contained inside the 173 | file. When it goes through the mapper function, it tokenizes the content into 174 | words (keys) and maps them to their number of occurences in the text (values). 175 | This task can be distributed over a network of servers **(Scatter)** 176 | 177 | 2. `Shuffler` function: The output obtained from the `mapper` is sent as an input 178 | to the `Shuffler` which is internal to the framework being used and the developer 179 | has no control over it. The shuffler is responsible to move the data around. Ex- 180 | The key-value pairs obtained from the `mapper` the ones with the common keys are 181 | gathered from the network the values of all the common keys is collected as a list 182 | of values at the same locality or node **(Gather)** 183 | 184 | 3. `Reducer` function: This function is responsible for doing some kind of 185 | computation on the shuffled output. Ex- The list of values of the common keys 186 | are added together to produce the total number of occurences of the word in the 187 | text. This function often outputs either a scalar (or a record of scalars) or 188 | a list of values (but the list is smaller the any of the inputs in the previous 189 | steps) 190 | 191 | ### Hadoop 192 | 193 | Simply put it can be termed as a map reduce API that comes with map reduce job 194 | management and a file system (HDFS) to make it easier for developers to write map 195 | reduce jobs. 196 | 197 | **HDFS:** Hadoop Distributed File System. 198 | - It can store files and directories. 199 | - All the metadata management is done by a single replicated master (name node/server) 200 | all of it being stored in memory (so fast but can take a lot of space since Hadoop 201 | is written in Java). 202 | - Files are stored in large, immutable and replicated blocks. The smallest block 203 | (chunk) is a 128 MB. Files cannot be modified but only created and deleted 204 | (immutable) 205 | 206 | #### Distributed storage in HDFS 207 | ``` 208 | 209 | 210 | ____ CLIENT APP (There can be 211 | _______________| multiple apps) 212 | _______|_________ 213 | | | 214 | _________________ | NAME NODE |________________ 215 | | |_________________| | 216 | | | | 217 | | | | 218 | | | | 219 | | | | 220 | ________|________ _________|_______ ____|____________ 221 | | | | | | | 222 | | DATA NODE 1 | | DATA NODE 2 | | DATA NODE 3 | 223 | |_________________| |_________________| |_________________| 224 | 225 | ``` 226 | - The client apps query the name node to read/write data into the data nodes. 227 | - The name node tells the apps which data node they need to contact for the 228 | read/write operation requested by the app` 229 | - Since its a DistributedSystem designed to handle multiple requests from multiple 230 | apps each client app will have its own connections to the data nodes. 231 | - The app will then query the data node and request it to perform the operation. 232 | > Let's say a new file has to be created and the app is redirected to the node3 233 | for this change 234 | 235 | - The data node 3 creates the new file and writes the data sent to it. 236 | - Once the data is written in a block inside a node it is replicated in the others, 237 | this is managed by the name node. 238 | - HDFS is built for storing large chunks of data in it. That means it is not 239 | built for handling high velocity of queries from the apps. 240 | - If one data node goes down then it always calls up the name node to query which 241 | activaties it missed while offline and replicates them. 242 | - For replication the data nodes talk to each other and make a copy of the blocks 243 | from the other nodes. 244 | 245 | #### Distributed Computation in Hadoop 246 | ``` 247 | 248 | ____ CLIENT APP (There can be 249 | _______________| multiple apps) 250 | _______|_________ 251 | | | 252 | _________________ | JobTracker |________________ 253 | | |_________________| | 254 | | | | 255 | | | | 256 | | | | 257 | | | | 258 | ________|________ _________|_______ ____|____________ 259 | | | | | | | 260 | | DATA NODE 1 | | DATA NODE 2 | | DATA NODE 3 | 261 | |_________________| |_________________| |_________________| 262 | |_____REDUCER_____| |_____REDUCER_____| |_____REDUCER_____| 263 | |=================| |=================| |=================| 264 | |___TASK TRACKER__| |___TASK TRACKER__| |___TASK TRACKER__| 265 | 266 | 267 | ``` 268 | - In large clusters the Job Tracker and the Name Node are isolated entities ( 269 | reside on different servers) 270 | - The client app submits a job (a map and a reduce function) to the tracker 271 | in the form of a jar file. 272 | - The jar file is serialize and dumped onto the tracker 273 | - The job tracker splits up the job into mappers and send them to the data nodes. 274 | - The map functions are seriaziled and sent to the `TaskTracker` (resides on the 275 | data node) and that mapper is executed on the data in the node. 276 | - The mapper outputs new KV pairs which are written to the FileSystem 277 | - Then the shuffler is run which might move around data to various nodes. 278 | - The JobTracker then sends the reducer to the data nodes which works on the 279 | shuffled KV pairs and generates new pairs to be written to the FileSystem. 280 | 281 | 282 | ### Spark 283 | - Uses scatter/gather 284 | - Different Data Models: Resilient Distributed Datasets (RDDs) 285 | - A set of transform/action functions providing a more general programming model 286 | (map is one of the transform and reduce one of the action functions) 287 | - Storage agnostic (Spark+HDFS or Spark+Cassandra) unlike Hadoop where using HDFS is mandatory 288 | 289 | #### Architecture 290 | -------------------------------------------------------------------------------- /infrastructure-engineering/CloudComputing.md: -------------------------------------------------------------------------------- 1 | > Cloud Computing is a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction. 2 | 3 | Cloud computing provides different services such as: 4 | - IaaS 5 | - PaaS 6 | - SaaS 7 | 8 | **Advantages:** 9 | 10 | - Speed and Agility 11 | - It reduces the up-front cost to setup the infrastructure and allows us to focus on applications and business. 12 | - Easy access to resources 13 | - Maintenance All the maintenance work for the resources is done by the provider. 14 | - Multi-tenancy 15 | - Reliability: Resources can be hosted in different Datacenter locations, to provide increased reliability. 16 | 17 | ## Virtualization 18 | A *hypervisor* sits on top of the host machine's OS. On top of this hypervisor a 19 | VM is created. Hypervisors emulate hardware like CPU, Disk and Network. For ex: 20 | VM Ware, VirtualBox, KVM, etc. 21 | 22 | ### VMs in development 23 | 24 | Vagrant is a tool for managing multiple VMs for the same project. It can also 25 | manage docker containers. 26 | -------------------------------------------------------------------------------- /infrastructure-engineering/Containers.md: -------------------------------------------------------------------------------- 1 | # What is a container? 2 | 3 | In the simplest of terms a container is a **sandbox** that helps us **isolate**, **run** and **monitor** a specific process in an operating system. 4 | 5 | **Sandbox** implies that the process has its won process namespace, whereas in a usual operating system environment process share namespaces. 6 | 7 | Typically once container has once container process. The container process and the container lifecycle is linked. When one ends the other one is also terminated. 8 | 9 | ## Container Image 10 | The **binary** representation of the runtime environment (the one thats described above :point_up: ) of a container (so basically a binary file). This binary representation follows a **parent-child model** and **image layering** 11 | 12 | Lets say there is just a scratch image that we start with which is just an empty file system. Then we add on top of it is a basic operating system (say Debian). This is a child of scratch. Let's add something else on top of that, say Perl, Ruby environment. This will act as another layer. 13 | 14 | #### Advantages 15 | - This gives us a tree-like structure of different OS environments from which we can pull any of the images and run our applications (sharing images by pulling branches of a tree). 16 | - This tree also follows inheritance. Let's say if we discovered a security vulnerablity in Debian and we put a path in at the debian image then that patch will be inherited by all its descendents with having to fix the vulnerablity in all the child images. 17 | 18 | If we look at it from an OO perspective then an image is a sort of a class (a template for creating runtime OS envs) and the runtime representation of a container is an instance of a class. 19 | 20 | ## DockerFile 21 | It is the text file used to create images. The parent image is the one mentioned in the `FROM:` field. Every line in a dockerfile is a layer (or cache layer). From `Dockerfile -> Image -> Runtime container`. We can also generate an image from a runtime env. 22 | 23 | ## Dependencies 24 | All the dependencies that are above the kernel are packaged inside a container. So running a container doesn't install the dependecies in the OS. The container sits on top of the OS and contains its dependencies. This enables us to run different versions of different applications with different dependencies without having to pollute our OS env by installing different versions of a dependency. 25 | 26 | ## The Docker Architecture 27 | - DockerFile 28 | - Docker Host (which contains) 29 | - Daemon 30 | - Image Cache 31 | - Volume 32 | - Registry 33 | - Docker Client 34 | 35 | The client talks to the daemon, the registry to the cache 36 | -------------------------------------------------------------------------------- /languages/Clojure.md: -------------------------------------------------------------------------------- 1 | # Clojure 2 | 3 | These notes were made while going through [Clojure from the ground up series](https://aphyr.com/posts/301-clojure-from-the-ground-up-welcome) 4 | 5 | - `nil` is the most basic value in Clojure. It represents emptiness, nothing-doing, not-a-thing. The absence of information. 6 | - true, false, and nil form the three poles of the Lisp logical system. 7 | - an expression in any dialect of a Lisp is a list `(inc 0)` 8 | - the order of evaluation of lists in from L-R and innermost lists get evaluated first 9 | ```clojure 10 | (+ 1 (- 5 2) (+ 3 4)) 11 | (+ 1 3 (+ 3 4)) 12 | (+ 1 3 7) 13 | 11 14 | ``` 15 | - to return the expression itself add a quote before its name `'inc` or `(+ 2 2)` 16 | - Clojure has a strongly typed, dynamic type system. 17 | 18 | ```clojure 19 | user=> (= 3 3.0) 20 | false 21 | user=> (== 3 3.0) 22 | true 23 | ``` 24 | 25 | - `str` can be used to turn things into a string or for concatanetation 26 | ```clojure 27 | user=> (str true) 28 | "true" 29 | 30 | user=> (str "meow " 3 " times") 31 | "meow 3 times" 32 | 33 | ``` 34 | - regexes using `re-find` and `re-matches` 35 | ```clojure 36 | user=> (re-find #"cat" "mystic cat mouse") 37 | "cat" 38 | user=> (re-find #"cat" "only dogs here") 39 | nil 40 | ``` 41 | - only `nil` and `false` are false in clojure 42 | 43 | ```clojure 44 | user=> (boolean false) 45 | false 46 | user=> (boolean nil) 47 | false 48 | ``` 49 | - `and` returns the first negative(false) or the last value if all are positive value and `or` returns the first positive(true) value 50 | 51 | ```clojure 52 | user=> (and true false true) 53 | false 54 | user=> (and true true true) 55 | true 56 | user=> (and 1 2 3) 57 | 3 58 | 59 | user=> (or false 2 3) 60 | 2 61 | user=> (or false nil) 62 | nil 63 | ``` 64 | 65 | - Symbols: The job of symbols is to refer to things, to point to other values. When evaluating a program, symbols are looked up and replaced by their corresponding values 66 | ```clojure 67 | user=> (class 'str) 68 | clojure.lang.Symbol 69 | user=> (= str clojure.core/str) 70 | true 71 | user=> (name 'clojure.core/str) 72 | "str" 73 | ``` 74 | - Keywords are like strings in that they’re made up of text, but are specifically intended for use as labels or identifiers. 75 | ```clojure 76 | user=> (type :cat) 77 | clojure.lang.Keyword 78 | ``` 79 | - Keywords can also be used as verbs to look up specific values in other data types 80 | - Lists can be constructed using `list` 81 | ```clojure 82 | user=> (list 1 2 3) 83 | (1 2 3) 84 | ``` 85 | 86 | - elements can be conjoined to a list 87 | 88 | ```clojure 89 | user=> (conj '(1 2 3) 4) 90 | (4 1 2 3) 91 | ``` 92 | 93 | - and get different elements of a list 94 | 95 | ```clojure 96 | user=> (first (list 1 2 3)) 97 | 1 98 | user=> (second (list 1 2 3)) 99 | 2 100 | user=> (nth (list 1 2 3) 2) 101 | 3 102 | ``` 103 | - Vectors are an alternative to lists when you have to deal with large amount of elements and arbitrary access. 104 | ```clojure 105 | user=> [1 2 3] 106 | [1 2 3] 107 | user=> (type [1 2 3]) 108 | clojure.lang.PersistentVector 109 | user=> (vector 1 2 3) 110 | [1 2 3] 111 | user=> (vec (list 1 2 3)) 112 | [1 2 3] 113 | user=> (conj [1 2 3] 4) 114 | [1 2 3 4] 115 | user=> (rest [1 2 3]) 116 | (2 3) 117 | user=> (next [1 2 3]) 118 | (2 3) 119 | user=> (rest [1]) 120 | () 121 | user=> (next [1]) 122 | nil 123 | user=> (last [1 2 3]) 124 | 3 125 | user=> (count [1 2 3]) 126 | 3 127 | ``` 128 | - We can have direct index lookups in vectors 129 | ```clojure 130 | user=> ([:a :b :c] 1) 131 | :b 132 | ``` 133 | - Finally, note that vectors and lists containing the same elements are considered equal in Clojure 134 | - Sets 135 | ```clojure 136 | user=> #{:a :b :c} 137 | #{:a :c :b} 138 | user=> (conj #{:a :b :c} :d) 139 | #{:a :c :b :d} 140 | user=> (conj #{:a :b :c} :a) 141 | #{:a :c :b} 142 | user=> (disj #{"hornet" "hummingbird"} "hummingbird") 143 | #{"hornet"} 144 | user=> (contains? #{1 2 3} 3) 145 | true 146 | ``` 147 | - Sets themselves can be used as a verb and perform lookpups 148 | ```clojure 149 | user=> (#{1 2 3} 3) 150 | 3 151 | user=> (#{1 2 3} 4) 152 | nil 153 | ``` 154 | - Maps 155 | ```clojure 156 | user=> {:name "mittens" :weight 9 :color "black"} 157 | {:weight 9, :name "mittens", :color "black"} 158 | user=> (get {"cat" "meow" "dog" "woof"} "cat") 159 | "meow" 160 | user=> (get {:a 1 :b 2} :c) 161 | nil 162 | ``` 163 | - `get` can take a default value too and maps themselves can be used as verbs 164 | ```clojure 165 | user=> (get {:glinda :good} :wicked :not-here) 166 | :not-here 167 | user=> ({"amlodipine" 12 "ibuprofen" 50} "ibuprofen") 168 | 50 169 | ``` 170 | - in case of keyword keys the syntax for lookup is a bit different 171 | ```clojure 172 | user=> (:raccoon {:weasel "queen" :raccoon "king"}) 173 | "king" 174 | ``` 175 | - add/update in a map 176 | ```clojure 177 | user=> (assoc {:bolts 1088} :camshafts 3) 178 | {:camshafts 3 :bolts 1088} 179 | user=> (assoc {:camshafts 3} :camshafts 2) 180 | {:camshafts 2} 181 | ``` 182 | - new map creating using `assoc` 183 | ```clojure 184 | user=> (assoc nil 5 2) 185 | {5 2} 186 | ``` 187 | - remove values and merge maps 188 | ```clojure 189 | user=> (dissoc {:potatoes 5 :mushrooms 2} :mushrooms) 190 | {:potatoes 5} 191 | user=> (merge {:a 1 :b 2} {:b 3 :c 4}) 192 | {:c 4, :a 1, :b 3} 193 | ``` 194 | 195 | - for binding values to names we can use `let` expression which takes a vector of bindings alternating between symbols and their values. Bindings are also evaluated in order 196 | and bindings declared once in the vector can be reused in the next bindings. 197 | ```clojure 198 | user=> (let [cats 3 199 | legs (* 4 cats)] 200 | (str legs " legs all together")) 201 | "12 legs all together" 202 | 203 | ``` 204 | - let bindings are limited to the scope of the let expression 205 | ```clojure 206 | user=> (let [+ -] (+ 2 3)) 207 | -1 208 | ``` 209 | - let bindings are immutable for mutable bindings we use `def` to define them and they are of var type 210 | ```clojure 211 | user=> (def cats 5) 212 | #'user/cats 213 | user=> (type #'user/cats) 214 | clojure.lang.Var 215 | ``` 216 | - functions/verbs in the clojure core library have two layers of indirection, for e.g. the symbol `inc` points to a var `clojure.core/inc` which in turn points to the function defined in the core `#object[clojure.core$inc 0x12e5be11 "clojure.core$inc@12e5be11"]` 217 | ```clojure 218 | user=> 'inc 219 | inc ; the symbol 220 | user=> (resolve 'inc) 221 | #'clojure.core/inc ; the var 222 | user=> (eval 'inc) 223 | # ; the value 224 | ``` 225 | 226 | > This way you can change what inc points to 227 | ```clojure 228 | scratch.core=> (def inc (fn [x] (* x 2))) 229 | WARNING: inc already refers to: #'clojure.core/inc in namespace: scratch.core, being replaced by: #'scratch.core/inc 230 | #'scratch.core/inc 231 | scratch.core=> (inc 2) 232 | 4 233 | ``` 234 | > Good Clojurists use def to set up a program initially, and only change those definitions with careful thought. 235 | 236 | ### Conditionals 237 | clojure provides `if`, `if-not`, `when` and `when-not` (in case we only have one branch to be evaluated), and also `if-let` and `when-let` in case we want to store the value of the predicate 238 | ```Clojure 239 | user=> (when-let [x (+ 1 2 3 4)] 240 | (str x)) 241 | "10" 242 | user=> (when-let [x (first [])] 243 | (str x)) 244 | nil 245 | ``` 246 | 247 | ### Functions 248 | ```clojure 249 | user=> ((fn [x] (+ x 1)) 2) 250 | 3 251 | user=> (let [burrito #(list "beans" % "cheese")] 252 | (burrito "carnitas")) 253 | ("beans" "carnitas" "cheese") 254 | ``` 255 | 256 | - functions can also be written with a shorthand `#(+ %1 %2)` which translates to `(fn [x y] (+ x y))` where %1 and %2 refer to the first and second arg in case of a lone arg we can simply use `%` 257 | - `def` can be used to mutate a var and declare a function as seen an the last example of the previous section but there is a syntax sugar for defining functions `(defn half [number] (/ number 2))` which is equivalent to `(def half (fn [number] (/ number 2)))` 258 | - For functions that take no arg you can leave the vector empty `defn funfunfunc [] (...)` 259 | - To handle multiple arities in our functions we can define a list of args and bodies much like function clauses in Elixir but in the case of Clojure they are the part of the same function definition 260 | ```clojure 261 | user=> (defn half 262 | ([] 1/2) 263 | ([x] (/ x 2))) 264 | ``` 265 | - For variadic arguments we can use `&` to slurp the rest of the args as a list and `""` can be used to write docstrings for functions 266 | ```clojure 267 | user=> (defn vargs 268 | "Takes two mandatory arguments x and y and the rest are variadic" 269 | [x y & more-args] 270 | {:x x 271 | :y y 272 | :more more-args}) 273 | #'user/vargs 274 | user=> (vargs 1 2 3 4 5) 275 | {:x 1, :y 2, :more (3 4 5)} 276 | ``` 277 | - `doc` function can be used to look up the docstring of a function and `meta` for function's metadata `(meta #'inc)` and `source` to look at the source code of functions except for special forms like `let`, `def`. 278 | - to know whether a symbol points to a function we can use `(fn? let)` which will give us false and `(fn? inc)` which returns true. 279 | 280 | ### Recursion 281 | 282 | ```clojure 283 | (defn inc-more [nums] 284 | (if (first nums) 285 | (cons (inc (first nums)) 286 | (inc-more (rest nums))) 287 | ; base case or the recurrence relation 288 | (list))) 289 | user=> (inc-more [1 2 3 4]) 290 | (2 3 4 5) 291 | ``` 292 | 293 | - A more generalized form of this would be the `map` function that can apply any kind of transformation on a collection of values. 294 | - `map` can also be used as a zip kind of function to stitch together multiple collections 295 | ```clojure 296 | user=> (map (fn [index element] (str index ". " element)) 297 | ; since this is an infinite sequence 298 | (iterate inc 0) 299 | ; map will stop at the length of this samller collection 300 | ["erlang" "ruby" "haskell"]) 301 | ("0. erlang" "1. ruby" "2. haskell") 302 | ``` 303 | - Clojure has an `iterate` and, `repeatedly` `repeat` function for building sequences 304 | ```clojure 305 | user=> (take 10 (iterate inc 0)) 306 | (0 1 2 3 4 5 6 7 8 9) 307 | user=> (take 10 (repeat :hi)) 308 | (:hi :hi :hi :hi :hi :hi :hi :hi :hi :hi) 309 | user=> (repeat 3 :echo) 310 | (:echo :echo :echo) 311 | user=> (take 3 (repeatedly rand)) 312 | (0.44442397843046755 0.33668691162169784 0.18244875487846746) 313 | ``` 314 | - Note: `rand` is an impure function since it gives a different result each time. 315 | - `cycle` allows us to infinitely repeat a collection 316 | ```clojure 317 | user=> (take 10 (cycle [1 2 3])) 318 | (1 2 3 1 2 3 1 2 3 1) 319 | ``` 320 | - clojure also has a `range` function 321 | ```clojure 322 | ; first and third args are optional 323 | ; an reversed order of UB and LB returns an empty list instead of an error 324 | user=> (range 0 100 5) 325 | (0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95) 326 | ``` 327 | 328 | - A few more useful functions that operate on sequences. 329 | ```clojure 330 | 331 | user=> (map-indexed (fn [index element] (str index ". " element)) 332 | ["erlang" "ruby" "haskell"]) 333 | ("0. erlang" "1. ruby" "2. haskell") 334 | user=> (concat [1 2 3] [:a :b :c] [4 5 6]) 335 | (1 2 3 :a :b :c 4 5 6) 336 | user=> (interleave [:a :b :c] [1 2 3]) 337 | (:a 1 :b 2 :c 3) 338 | user=> (interpose :and [1 2 3 4]) 339 | (1 :and 2 :and 3 :and 4) 340 | user=> (reverse [1 2 3]) 341 | (3 2 1) 342 | user=> (reverse "woolf") 343 | (\f \l \o \o \w) 344 | user=> (apply str (reverse "woolf")) 345 | "floow" 346 | ; break strings into a sequence 347 | user=> (seq "sato") 348 | user=> (shuffle [1 2 3 4]) 349 | [3 1 2 4] 350 | (\s \a \t \o) 351 | ``` 352 | - functions for subsequences 353 | ```clojure 354 | user=> (take 3 (range 10)) 355 | (0 1 2) 356 | user=> (drop 3 (range 10)) 357 | (3 4 5 6 7 8 9) 358 | user=> (take-last 3 (range 10)) 359 | (7 8 9) 360 | user=> (drop-last 3 (range 10)) 361 | (0 1 2 3 4 5 6) 362 | user=> (take-while pos? [3 2 1 0 -1 -2 10]) 363 | (3 2 1) 364 | user=> (drop-while pos? [3 2 1 0 -1 -2 10]) 365 | (0 -1 -2 10) 366 | 367 | ; for cutting sequences 368 | (split-at 4 (range 10)) 369 | [(0 1 2 3) (4 5 6 7 8 9)] 370 | user=> (split-with number? [1 2 3 :mark 4 5 6 :mark 7]) 371 | [(1 2 3) (:mark 4 5 6 :mark 7)] 372 | 373 | user=> (filter pos? [1 5 -4 -7 3 0]) 374 | (1 5 3) 375 | 376 | ; `filter`'s complement 377 | user=> (remove string? [1 "turing" :apple]) 378 | (1 :apple) 379 | 380 | 381 | user=> (partition 2 [:cats 5 :bats 27 :crocodiles 0]) 382 | ((:cats 5) (:bats 27) (:crocodiles 0)) 383 | (user=> (partition-by neg? [1 2 3 2 1 -1 -2 -3 -2 -1 1 2]) 384 | ((1 2 3 2 1) (-1 -2 -3 -2 -1) (1 2)) 385 | ``` 386 | - functions to collapse sequences 387 | ```clojure 388 | user=> (frequencies [:meow :mrrrow :meow :meow]) 389 | {:meow 3, :mrrrow 1} 390 | 391 | 392 | user=> (pprint (group-by :first [{:first "Li" :last "Zhou"} 393 | {:first "Sarah" :last "Lee"} 394 | {:first "Sarah" :last "Dunn"} 395 | {:first "Li" :last "O'Toole"}])) 396 | {"Li" [{:last "Zhou", :first "Li"} {:last "O'Toole", :first "Li"}], 397 | "Sarah" [{:last "Lee", :first "Sarah"} {:last "Dunn", :first "Sarah"}]} 398 | ``` 399 | - when it comes to collapsing sequences `reduce` is the counterpart to map 400 | ```clojure 401 | 402 | user=> (reduce + [1 2 3 4]) 403 | 10 404 | 405 | ; reductions can show us intermediate steps of reduce 406 | user=> (reductions + [1 2 3 4]) 407 | (1 3 6 10) 408 | 409 | ; reduce can also be used with an initial/default value 410 | ; this example starts with an empty set 411 | user=> (reduce conj #{} [:a :b :b :b :a :a]) 412 | #{:a :b} 413 | 414 | ; Reducing elements into a collection has its own name 415 | user=> (into {} [[:a 2] [:b 3]]) 416 | {:a 2, :b 3} 417 | ; reverse order 418 | user=> (into (list) [1 2 3 4]) 419 | (4 3 2 1) 420 | 421 | ; in order 422 | user=> (reduce conj [] [1 2 3 4 5]) 423 | (reduce conj [] [1 2 3 4 5]) 424 | [1 2 3 4 5] 425 | ``` 426 | 427 | - `map` can be written in the form of `reduce` 428 | ```clojure 429 | (defn my-map [f coll] 430 | (reduce (fn [output element] 431 | (conj output (f element))) 432 | [] 433 | coll)) 434 | user=> (my-map inc [1 2 3 4]) 435 | [2 3 4 5] 436 | ``` 437 | - clojure has lazy functions (most of the sequence functions are lazy) which return unreliazed sequences 438 | ```clojure 439 | user=> (def infseq (map inc (iterate inc 0))) 440 | #'user/infseq 441 | user=> (realized? infseq) 442 | false 443 | ``` 444 | 445 | ## Quotes 446 | - A single quote can be used in Clojure to return an expression without evaluating it `'(list 1 2)` would return `(list 1 2)` 447 | - We can also use backticks (called syntax quote operator) for this along with unquote (`~`) and unquote-splice (`~@`) operators. 448 | ```Clojure 449 | ; ~ substitutes the value but the expressin isn't evaluated 450 | user=> (let [x 2] `(inc x)) 451 | (clojure.core/inc user/x) 452 | user=> (let [x 2] `(inc ~x)) 453 | (clojure.core/inc 2) 454 | 455 | ; ~@ turns lists to multiple expressions 456 | user=> `(foo ~[1 2 3]) 457 | (user/foo [1 2 3]) 458 | user=> `(foo ~@[1 2 3]) 459 | (user/foo 1 2 3) 460 | ``` 461 | ## Macros 462 | - `defmacro` can be used to define macros and macroexpand with a `'` to the macro call to see the expanded code after the macro rewrites it. 463 | - Since macros generate code we might need to rename the symbols in the generated code for that we can use `gensym` which gives us new variable names in our macros 464 | ```Clojure 465 | scratch.core=> (gensym "or") 466 | or1615 467 | ``` 468 | - gensyms can also be generated by suffixing a `#` to a particular symbol 469 | ```Clojure 470 | user=> `(let [x# 2] x#) 471 | (clojure.core/let [x__339__auto__ 2] x__339__auto__) 472 | ``` 473 | - We can also override the local variables by not generating a gensym using `~'or` but this leads to an unhygenic macro 474 | ```Clojure 475 | user=> (source or) 476 | (defmacro or 477 | "Evaluates exprs one at a time, from left to right. If a form 478 | returns a logical true value, or returns that value and doesn't 479 | evaluate any of the other expressions, otherwise it returns the 480 | value of the last expression. (or) returns nil." 481 | {:added "1.0"} 482 | ([] nil) 483 | ([x] x) 484 | ([x & next] 485 | `(let [or# ~x] 486 | (if or# or# (or ~@next))))) 487 | 488 | user=> (pprint (clojure.walk/macroexpand-all 489 | '(or (mossy? stone) (cool? stone) (wet? stone)))) 490 | (let* 491 | [or__3943__auto__ (mossy? stone)] 492 | (if 493 | or__3943__auto__ 494 | or__3943__auto__ 495 | (let* 496 | [or__3943__auto__ (cool? stone)] 497 | (if or__3943__auto__ or__3943__auto__ (wet? stone))))) 498 | ``` -------------------------------------------------------------------------------- /languages/Elixir.md: -------------------------------------------------------------------------------- 1 | # From "The Little Elixir and OTP Guidebook" 2 | 3 | ## Functions 4 | 5 | ### Function Clauses 6 | 7 | ```elixir 8 | defmodule MeterToLengthConverter do 9 | # syntax for single lined functions :point-down: 10 | def convert(:feet, m), do: m * 3.28084 11 | def convert(:inch, m), do: m * 39.3701 12 | def convert(:yard, m), do: m * 1.09361 13 | end 14 | ``` 15 | 16 | ### Optional argument 17 | 18 | Optional arguments can be specified using `\\`. For e.g. 19 | ```elixir 20 | def fetch_something(opts \\ []) do 21 | ## an empty list as default options 22 | end 23 | ``` 24 | 25 | **Note: ** function clauses in a module should be placed together like the above example. 26 | 27 | ### Nested Modules 28 | 29 | To flatten a nested module use the `.` operator 30 | 31 | ```elixir 32 | defmodule FirePokemon.Charizard do 33 | def attack do 34 | "fireball" 35 | end 36 | end 37 | 38 | def FirePokemon.Magma do 39 | def attack do 40 | "lava" 41 | end 42 | end 43 | ``` 44 | 45 | ## OTP 46 | 47 | ### GenServer 48 | 49 | Writing client-server code using GenServer is pretty straightforward in terms of code organization. The functions can be divided into 3 categories 50 | - Client API functions: as the name states they are the entrypoint of a request from the client 51 | - Server Callbacks: Genserver callbacks invoked by the client side functions 52 | - Helper functions: Code that doesn't belong in either of the two types 53 | 54 | For e.g. 55 | ```elixir 56 | defmodule Gentex.Worker do 57 | use GenServer 58 | 59 | ## Client API 60 | 61 | def start_link(opts \\ []) do 62 | GenServer.start_link(__MODULE__, :ok, opts) 63 | end 64 | 65 | def reset_stats(pid) do 66 | GenServer.cast(pid, :reset_stats) 67 | end 68 | 69 | def temperature(pid, location) do 70 | GenServer.call(pid, {:location, location}) 71 | end 72 | 73 | def stop(pid) do 74 | GenServer.cast(pid, :stop) 75 | end 76 | 77 | ## Server callbacks 78 | def init(:ok) do 79 | {:ok, %{}} 80 | end 81 | 82 | def handle_call({:location, location}, _from, stats) do 83 | case temperature_of(location) do 84 | {:ok, temp} -> 85 | new_stats = update_stats(stats, location) 86 | {:reply, "#{temp}°C", new_stats} 87 | _ -> 88 | {:reply, :error, stats} 89 | end 90 | end 91 | 92 | def handle_cast(:reset_stats, _stats) do 93 | {:noreply, %{}} 94 | end 95 | 96 | def handle_cast(:stop, stats) do 97 | {:stop, :normal, :ok, stats} 98 | end 99 | 100 | def terminate(reason, stats) do 101 | IO.puts "server stopped because of #{inspect reason}" 102 | inspect stats 103 | :ok 104 | end 105 | 106 | ## Helper functions 107 | 108 | defp update_stats(old_stats, location) do 109 | case Map.has_key?(old_stats, location) do 110 | true-> 111 | Map.update!(old_stats, location, &(&1+1)) 112 | false-> 113 | Map.put_new(old_stats, location, 1) 114 | end 115 | end 116 | 117 | defp temperature_of(location) do 118 | url_for(location) |> HTTPoison.get |> parse_response 119 | end 120 | 121 | defp url_for(location) do 122 | # do something 123 | end 124 | 125 | defp parse_response({:ok, %HTTPoison.Response{body: body, status_code: 200}}) do 126 | body |> JSON.decode! |> compute_temp 127 | end 128 | 129 | defp parse_response(_) do 130 | :error 131 | end 132 | 133 | defp compute_temp(json) do 134 | try do 135 | temp = (json["main"]["temp"] - 273.15) |> Float.round(1) 136 | {:ok, temp} 137 | rescue 138 | _ -> :error 139 | end 140 | end 141 | end 142 | 143 | ``` 144 | 145 | `GenServer.start_link`: the first argument is the module name that contains the `init` function and invokes it. Our client `start_link` is just a wrapper around the function provided by the GenServer to provide a way for clients to invoke GenServer functions instead of manually calling Genserver functions and pass the API module to it. 146 | 147 | `handle_call`: The first argument declares the expected request to be handled. The second argument returns a tuple in the form of {pid, tag}, where the pid is the pid of the client 148 | and tag is a unique reference to the message. 149 | 150 | `handle_call`(s) & `handle_cast`(s) should be grouped together. 151 | 152 | good use case for GenServer.cast/2? A 153 | fine example is a command that’s issued to a server and that causes a side effect in the 154 | server’s state. In that case, the client issuing the command shouldn’t care about a 155 | reply. 156 | 157 | Messages may arrive from processes that aren’t defined in handle_call/3/handle 158 | _cast/2. That’s where handle_info/2 comes in. It’s invoked to handle any other 159 | messages that are received by the process, sometimes referred to as out-of-band messages. You don’t need to supply a client API counterpart for handle_info/2. 160 | 161 | 162 | In case of distributed nodes you can use a name attribute for your GenServers 163 | ```elixir 164 | defmodule Metex.Worker do 165 | use GenServer 166 | @name MW 167 | ## Client API 168 | def start_link(opts \\ []) do 169 | GenServer.start_link(__MODULE__, :ok, opts ++ [name: MW]) 170 | end 171 | 172 | def get_temperature(location) do 173 | GenServer.call(@name, {:location, location}) 174 | end 175 | end 176 | ``` 177 | 178 | Things to keep in mind while writing GenServers 179 |  The state with which you want to initialize the server 180 |  The kinds of messages the server handles 181 |  When to reply to the client 182 |  What message to use to reply to the client 183 |  What resources to clean up after termination 184 | 185 | ### Processes 186 | - There’s no shared memory. The only way a change 187 | of state can occur within a process is when a message is sent to it. This is different from 188 | threads, because threads share memory. 189 | - messages can come in the form of any valid Elixir 190 | term. This means tuples, lists, and atoms are all fair game. 191 | 192 | #### Links and Monitors 193 | - Processes can be linked together using `Process.link(pid)` which will link the process with the id stored in `pid` to `self` 194 | - `Process.info(pid)` can be used to view the process metadata (returned as a map) including links to other processes 195 | - Linked processes crash if one of the linked processes crashes. Example 196 | ```Elixir 197 | defmodule Ring do 198 | 199 | def create_processes(n) do 200 | 1..n |> Enum.map(fn _ -> spawn(fn -> loop end) end) 201 | end 202 | 203 | def loop do 204 | receive do 205 | {:link, link_to} when is_pid(link_to) -> 206 | Process.link(link_to) 207 | loop 208 | :crash -> 209 | 1/0 210 | end 211 | end 212 | 213 | def link_processes(procs) do 214 | link_processes(procs, []) 215 | end 216 | 217 | def link_processes([proc_1, proc_2|rest], linked_processes) do 218 | send(proc_1, {:link, proc_2}) 219 | link_processes([proc_2|rest], [proc_1|linked_processes]) 220 | end 221 | 222 | def link_processes([proc|[]], linked_processes) do 223 | first_process = linked_processes |> List.last 224 | send(proc, {:link, first_process}) 225 | :ok 226 | end 227 | end 228 | ``` 229 | 230 | ### Agents 231 | 232 | Agents are simple wrappers around state. If all you want from a process is to keep state, agents are a great fit. 233 | 234 | ### Different OTP behaviours and their uses 235 | 236 | GenServer => Implementing the server of a client-server relationship 237 | GenEvent => Implementing event-handling functionality 238 | Supervisor => Implementing supervision functionality 239 | Application => Working with applications and defining application callbacks -------------------------------------------------------------------------------- /languages/Go.md: -------------------------------------------------------------------------------- 1 | # My notes on the GO programming language 2 | Learning Go 3 | 4 | ## Packaging 5 | The root directory of a Go package/application is called a workspace and is divided into the following parts: 6 | 7 | - **src** – This directory contains source files organized as packages. The application is written in this directory. 8 | - **pkg** – This directory contains Go package objects. 9 | - **bin** – This directory contains executable programs. 10 | 11 | ### A few important points about packages 12 | - Source files are organized into system directories called packages, which enable code reusability across the Go applications. 13 | - The naming convention for a Go package is to use the name of the system directory where we are putting our Go source files. Within a single folder, the package name will be same for the all source files which belong to that directory. 14 | - We develop our Go programs in the $GOPATH directory, where we organize source code files into directories as packages. 15 | - All identifiers will be exported to other packages if the first letter of the identifier name starts with an uppercase letter. 16 | - The functions and types will not be exported to other packages if we start with a lowercase letter for the identifier name. 17 | 18 | #### Package main 19 | Only used when we want to write executable scripts and not a shared library/package. 20 | Include `package main` in the scripts to make them executable (kind of like shebang) 21 | ``` 22 | package main 23 | 24 | import ( 25 | "fmt" 26 | "buffio" 27 | ) 28 | 29 | func main(){ 30 | // do some stuff 31 | } 32 | ``` 33 | 34 | - Here this script is an executable and function `main` acts as the entry point for the Go compiler while executing the script. 35 | - The keyword `import` is used for importing a package into another package. 36 | - When you import packages, the Go compiler will look on the locations specified by the environment variable GOROOT (standard library package) and GOPATH (third party packages). 37 | 38 | #### Init function 39 | While writing Go packages we can provide an init function that executes some initialization logic into the package at the time of execution. 40 | 41 | In some cases where one wants to use only the `init` function of the package are not using the package identifier while importing, Go compiler will show an error. 42 | In such a case, use a blank identifier ( _ ) as the package alias name, so the compiler ignores the error of not using the package identifier, but will still invoke the init function. 43 | 44 | ``` 45 | package main 46 | 47 | import ( 48 | _ "mywebapp/libs/mongodb/db" 49 | "fmt" 50 | "log" 51 | ) 52 | 53 | func main() { 54 | //implementation here 55 | } 56 | ``` 57 | -------------------------------------------------------------------------------- /languages/Javascript.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/palash25/Notes/8b655d8df2a8bf524316857dfa6649893c377829/languages/Javascript.md -------------------------------------------------------------------------------- /languages/Lisp.md: -------------------------------------------------------------------------------- 1 | # Notes About Common Lisp (From Practical Common Lisp) 2 | 3 | ## SetUp 4 | Install SBCL and Quicklisp (package manager) 5 | 6 | ## Notes 7 | - Strings, numbers are self-evaluating objects. 8 | - LISP's version of null is `NIL` 9 | - Functions are defined using `defun` e.g. 10 | ```lisp 11 | (defun hello-world () 12 | (format t "hello, world")) 13 | ``` 14 | - The above code only defines the function to run the function it needs to be called like `(hello-world)` 15 | - Its a convention to separate words (in function names, etc) using `-` rather than camelCase or `_`. 16 | -------------------------------------------------------------------------------- /languages/Lua.md: -------------------------------------------------------------------------------- 1 | ## From the 7 in 7 series 2 | 3 | - Prototype based langauges 4 | > In prototype based languages the class and the instances are not separate entities. Once an object/instance is defined it doesn't serve as a blueprint for other objects but several copies of the same object are made that can then be mutated. 5 | 6 | - Supports procedural, object-oriented, event-driven programming 7 | 8 | ### Basic features 9 | - Concatanation `'abc' .. 'xyz'` 10 | - Variadic functions are possible 11 | ```lua 12 | function something(a, ...) 13 | otherArgs = {..} 14 | return otherArgs[1] 15 | ``` 16 | - If lesser args are passed to a funtion call the the parameters that don't receive an argument get initialized to `nil` 17 | - More arguments are passed then they are ignored 18 | - Functions are first class citizen and can be stored in variables and passsed around as arguments 19 | - `#'fdsf'` gives the length of the string 20 | - `repeat-until` loop in Lua is the same as a `do-while` 21 | 22 | - **Tail call optimization:** Lua is able to handle recursive function with large number of iterations provided that the recursive call is at the very end of the function 23 | - Multiple return values are possible 24 | - *Flow of control:* `while-do`, `for-do` and `if-then-elseif-then-else` 25 | 26 | #### Tables 27 | 28 | - Tables are like dictionaries. 29 | - They can be written as key-value pairs between curly braces (also known as table constructor) 30 | ```lua 31 | > book = { 32 | >> title = "Grail Diary", 33 | >> author = "Henry Jones", 34 | >> pages = 100 35 | >> } 36 | ``` 37 | - Access values using dot `book.title` 38 | - New keys can be added and existing values can be modified using dot 39 | - Keys that are calculated at runtime or keys with spaces and decimal points can be used with `[]` for e.g. `book[title]` 40 | - Items can be removed by setting a key to `nil` 41 | - Tables can also be used as arrays by ommitting the keys and just writing the values. The indicies start with 1 and values can be accessed using `[]`. 42 | ```lua 43 | > medals = { 44 | >> "gold", 45 | >> "silver", 46 | >> "bronze" 47 | >> } 48 | ``` 49 | - Both the array and dictionary styles can be used together and be separated using a `;` (a best practice) 50 | 51 | ```lua 52 | > ice_cream_scoops = { 53 | >> "vanilla", 54 | >> "chocolate"; 55 | >> 56 | >> sprinkles = true 57 | >> } 58 | >> 59 | =ice_cream_scoops[1] 60 | vanilla 61 | > =ice_cream_scoops.sprinkles 62 | true 63 | ``` 64 | **Metatables (Overriding table's default behaviour)** 65 | 66 | Every table in lua has a corresponding metatable which is set to `nil` by default. 67 | ```lua 68 | > 69 | =getmetatable(some_table) 70 | nil 71 | ``` 72 | Metatables can be used to override the behaviour of tables but defining our own functions and assigning them to to a particular key (kind of like dunder functions in python) in the metatable 73 | 74 | ```lua 75 | greek_numbers = { 76 | ena = "one", 77 | dyo = "two", 78 | tria = "three" 79 | } 80 | 81 | function table_to_string(t) 82 | local result = {} 83 | for k, v in pairs(t) do 84 | result[#result + 1] = k .. ": " .. v 85 | end 86 | return table.concat(result, "\n") 87 | end 88 | 89 | mt = { 90 | __tostring = table_to_string 91 | } 92 | 93 | setmetatable(greek_numbers, mt) 94 | ``` 95 | 96 | Custom read/write logic can also be implemented 97 | ```lua 98 | local mt = { 99 | __index = strict_read, 100 | __newindex = strict_write 101 | } 102 | treasure = {} 103 | setmetatable(treasure, mt) 104 | 105 | -- internal table to read the data of the actual table 106 | local _private = {} 107 | 108 | function strict_read(table, key) 109 | if _private[key] then 110 | return _private[key] 111 | else 112 | error("Invalid key: " .. key) 113 | end 114 | end 115 | 116 | function strict_write(table, key, value) 117 | if _private[key] then 118 | error("Duplicate key: " .. key) 119 | else 120 | _private[key] = value 121 | end 122 | end 123 | ``` 124 | 125 | **User Defined OO Scheme** 126 | 127 | Lua isn't OOP language but its prototypes provide a way to replicate OO patterns. 128 | A table containing a set of common characteristics and methods can be used to create several objects with different states. 129 | 130 | ```lua 131 | -- Prototype table 132 | Villain = { 133 | health = 100, 134 | 135 | new = function(self, name) 136 | local obj = { 137 | name = name, 138 | health = self.health, 139 | } 140 | return obj 141 | end, 142 | 143 | take_hit = function(self) 144 | self.health = self.health - 10 145 | end 146 | } 147 | 148 | -- A new Villian object/table can be created using the prototype Villian 149 | dietrich = Villain.new(Villain, "Dietrich") 150 | ``` 151 | But this `dietrich.take_hit(dietrich)` is not possible with the above prototype. To be able to override the lookup behaviour we can make use of metatables inside of our prototypes. 152 | 153 | So now if the field is not found in the table `dietrich` it will look it up in our prototype `Villian` object. This can be done by making a few changes to the `new` function. 154 | 155 | ```lua 156 | new = function(self, name) 157 | local obj = { 158 | name = name, 159 | health = self.health, 160 | } 161 | setmetatable(obj, self) 162 | self.__index = self 163 | return obj 164 | end 165 | ``` 166 | So essentially all objects in Lua are copies of one object/prototype 167 | 168 | **Inheritance** 169 | 170 | It can be achieved by cloning an existing prototype 171 | 172 | ```lua 173 | function SuperVillain.take_hit(self) 174 | -- Haha, armor! 175 | self.health = self.health - 5 176 | end 177 | 178 | -- Clone the newly created prototype. 179 | toht = SuperVillain.new(SuperVillain, "Toht") 180 | toht.take_hit(toht) 181 | print(toht.health) --> 95 182 | ``` 183 | 184 | **Syntactic Sugar** 185 | 186 | The self parameter can be ommitted when calling a method on a table 187 | 188 | ```lua 189 | function Villain:new(name) 190 | -- ...same implementation as before... 191 | end 192 | 193 | function Villain:take_hit() 194 | -- ...same implementation as before... 195 | end 196 | 197 | SuperVillain = Villain:new() 198 | 199 | function SuperVillain:take_hit() 200 | -- ...same implementation as before... 201 | end 202 | 203 | toht = SuperVillain:new("Toht") 204 | toht:take_hit() 205 | print(toht.health) --> 95 206 | ``` 207 | 208 | ### Links 209 | - [Lua Reference Manual](https://www.lua.org/manual/5.3/) 210 | - [Learn X in Y minutes](https://learnxinyminutes.com/docs/lua/) 211 | -------------------------------------------------------------------------------- /languages/Nim.md: -------------------------------------------------------------------------------- 1 | # Nim 2 | 3 | Notes from "Nim in Action". 4 | 5 | ## Tooling related features 6 | - Nim Supports cross-compilation 7 | - Supports different GC modes (ref counting, a custom GC, and Boehm) and also an option to disable the GC 8 | 9 | ## Language related features 10 | - `main` functions are implicit in Nim, even if you don't provide one the compiler assumes one to be there. 11 | - `procedure`s unlike methods found in mainstream langauges are not bound to an object/type in Nim but can still be called on it 12 | ```Nim 13 | 14 | type 15 | Dog = object 16 | 17 | proc bark(self: Dog) = 18 | echo("Woof!") 19 | 20 | let dog = Dog() 21 | dog.bark() 22 | ``` 23 | > Reason: Nim rewrites `dog.bark()` to `bark(dog)`, making it possible for you to call methods using the traditional OOP style without having to explicitly bind them to a class. This is called Uniform Function Call Syntax and allows one to define new procedures on existing objects and chain function calls 24 | - You can also do some FP-style coding usig closures 25 | ```Nim 26 | import sequtils, future, strutils 27 | let list = @["Dominik Picheta", "Andreas Rumpf", "Desmond Hume"] 28 | list.map( 29 | (x: string) -> (string, string) => (x.split[0], x.split[1]) 30 | ).echo 31 | ``` 32 | - Nim implements an exception-tracking mechanism that is entirely opt-in. With exception tracking, you can ensure that a procedure 33 | won’t raise any exceptions, or that it will only raise exceptions from a predefined list. 34 | This prevents unexpected crashes by ensuring that you handle exceptions. 35 | - type names start with an uppercase letter. Built-in types don’t follow 36 | this convention, 37 | - Comments and disabling code: 38 | ```Nim 39 | 40 | # Single-line comment 41 | #[ 42 | Multiline comment 43 | ]# 44 | # disables code, parsed by the compiler but not executed 45 | when false: 46 | echo("Commented-out code") 47 | ``` 48 | - `let` is used to create immutable variable bindings and `var` for mutable. 49 | - when explicitly mentioning the type of a variable it is initialized to a default value 50 | ```Nim 51 | 52 | var number: int # initialized to `0` 53 | var text: string # initialized to `nil.nil` 54 | ``` 55 | - immutable variables (let bindings) can't be declared without initialization -------------------------------------------------------------------------------- /languages/Pony.md: -------------------------------------------------------------------------------- 1 | # Pony 2 | 3 | - Pony is an object oriented, actor based language. 4 | - Pony introduces the concept of reference capabilities to make concurrency safe. Reference capabilities are just type annotations that define the number of reference and read and write permissions for a type. 5 | - Just like processes in Erlang/Elixir or goroutines in Go, Pony provides threads which are super lightweight, just 256 bytes as compared to Go's 4KB per goroutine. 6 | - The highlight of Pony is no doubt its reference capabilities which are based on its type system. 7 | 8 | ## Classes 9 | 10 | ```pony 11 | class Wombat 12 | let name: String 13 | var _hunger_level: U64 14 | var _thirst_level: U64 = 1 15 | 16 | new create(name': String) => 17 | name = name' 18 | _hunger_level = 0 19 | 20 | new hungry(name': String, hunger': U64) => 21 | name = name' 22 | _hunger_level = hunger' 23 | 24 | fun hunger(): U64 => _hunger_level 25 | 26 | fun ref set_hunger(to: U64 = 0): U64 => _hunger_level = to 27 | ``` 28 | 29 | `let` is used to define fields that are set just once, `var`s can be mutated. 30 | 31 | ### Reference capabilities of methods 32 | 33 | The method `set_hunger` has a `ref` before it which indicates that the receiver (the object of Wombat) of the method is mutable since we need to set a field. In case of the method `hunger` which has no reference capability assigned to it, `box` is considered as the default. 34 | 35 | The return value of `set_hunger` method is the old value of `_hunger_level`. Because in Pony even assignments are expressions rather than statements so they need to return a value. Unlike other languages Pony returns the old value which is known as a "destructive read". For e.g. swapping two number in Pony is a one liner `a = b = a` (b is set to the the value of a but the expression `b = a` returns the original value of b setting it to a) 36 | 37 | ## Primitives 38 | 39 | ```pony 40 | // 2 "marker values" 41 | primitive OpenedDoor 42 | primitive ClosedDoor 43 | 44 | // An "enumeration" type 45 | type DoorState is (OpenedDoor | ClosedDoor) 46 | 47 | // A collection of functions 48 | primitive BasicMath 49 | fun add(a: U64, b: U64): U64 => 50 | a + b 51 | 52 | fun multiply(a: U64, b: U64): U64 => 53 | a * b 54 | 55 | actor Main 56 | new create(env: Env) => 57 | let doorState : DoorState = ClosedDoor 58 | let isDoorOpen : Bool = match doorState 59 | | OpenedDoor => true 60 | | ClosedDoor => false 61 | end 62 | env.out.print("Is door open? " + isDoorOpen.string()) 63 | env.out.print("2 + 3 = " + BasicMath.add(2,3).string()) 64 | ``` 65 | 66 | Primitives can have two special functions, _init and _final. _init is called before any actor starts. _final is called after all actors have terminated. 67 | 68 | ## Actors 69 | 70 | They are just like classes but instead of methods/functions they have behaviours. Behaviours are pices of code that when called upon are executed asynchronously unlike functions. 71 | 72 | ```pony 73 | actor Main 74 | new create(env: Env) => 75 | call_me_later(env) 76 | env.out.print("This is printed first") 77 | 78 | be call_me_later(env: Env) => 79 | env.out.print("This is printed last") 80 | ``` 81 | 82 | Since behaviours are async they return `None`. Actors are akin to Processes. When you call upon a behaviour on an actor you are actually passing a message to it. 83 | 84 | Actors by themselves are synchronous, at a time we can call only one behaviour on an actor. 85 | 86 | Actors can have finalisers, after executing a finaliser, actors can't receive messages. 87 | 88 | ## Subtyping 89 | ### Nominal Subtyping using Traits 90 | 91 | ```pony 92 | 93 | trait Named 94 | fun name(): String => "Bob" 95 | 96 | trait Bald 97 | fun hair(): Bool => false 98 | 99 | class Bob is (Named & Bald) 100 | 101 | class Larry 102 | fun name(): String => "Larry" 103 | 104 | ``` 105 | 106 | ### Structural Subtyping using Interfaces 107 | 108 | ```pony 109 | interface HasName 110 | fun name(): String 111 | ``` 112 | 113 | A class doesnt have to explicitly mention in this case that it belongs to the interface `HasName` both Bob and Harry *provide* HasName. Its the same as subtyping in Go and might lead to accidental subtyping. 114 | 115 | ## Structs 116 | 117 | These are just classes used for FFI. 118 | 119 | ## Type Aliases 120 | 121 | - [TODO](https://tutorial.ponylang.io/types/type-aliases.html) 122 | 123 | -------------------------------------------------------------------------------- /languages/Ruby.md: -------------------------------------------------------------------------------- 1 | ## Learning Ruby from 7 languages in 7 weeks 2 | 3 | ### Key features 4 | - Interpreted 5 | - OOP 6 | - Strongly Typed 7 | - Duck Typing 8 | - Conditionals can be written in both one line or in blocks. Ex- 9 | 10 | ```ruby 11 | if x == 4 12 | puts 'This appears to be true.' 13 | end 14 | 15 | puts 'This appears to be true.' if x == 4 16 | ``` 17 | 18 | - Instead of `not` or `!` `until` and `unless` can be used for negation and looks much cleaner than the former. 19 | 20 | - Arrays (like python list) and Hash (like python dicts) are 2 major data structures in Ruby 21 | 22 | - Hashes have a `label-object` pair. Ex 23 | 24 | ```ruby 25 | numbers = {1 => 'one', 2 => 'two'} 26 | 27 | stuff = {:array => [1, 2, 3], :string => 'Hi, mom!'} 28 | ``` 29 | 30 | > The labels with colon are called symbols and are unique objects so the symbols with the same name are the same physical object 31 | Symbols are lightweight strings used when we don't want the strings to be printed to the screen 32 | 33 | - Capitalized variables in Ruby are constants e.g `EmpireStateBuilding = "350 5th Avenue, NYC, NY"` 34 | 35 | - There are no named parameters in Ruby but this concept can be emulated using a hash 36 | 37 | ```ruby 38 | def truth(option={}) 39 | if option[:profession] == :lawyer 40 | puts 'could be false' 41 | else 42 | puts 'true' 43 | end 44 | end 45 | 46 | truth 47 | 48 | truth :profession => :lawyer 49 | ``` 50 | 51 | - A code block is a function without a name `3.times {puts 'hiya there, kiddo'}` will execute the block/function 3 times. 52 | 53 | - For 1 line blocks braces are used and for code blocks exceeding that limit we use `do/end` 54 | 55 | - Passing executable code around in Ruby 56 | ```ruby 57 | >> def call_block(&block) 58 | >> block.call 59 | >> end 60 | => nil 61 | >> def pass_block(&block) 62 | >> call_block(&block) 63 | >> end 64 | => nil 65 | >> pass_block {puts 'Hello, block'} 66 | Hello, block 67 | ``` 68 | 69 | - Ruby equivalent for pythons *args and *kwargs are called splat(*) and double splat operator(**). Example: 70 | ```ruby 71 | def test_method 72 | yield({hash_one: "argument one", hash_two: "argument two", hash_three: "argument three"}) 73 | end 74 | 75 | test_method do |**double_splat_parameter| 76 | puts double_splat_parameter 77 | end 78 | 79 | ``` 80 | **Classes** 81 | - In ruby classes can only inherit from one parent known as a "superclass". Example 82 | ``` 83 | >> 4.class 84 | => Fixnum 85 | >> 4.class.superclass 86 | => Integer 87 | >> 4.class.superclass.superclass 88 | => Numeric 89 | >> 4.class.superclass.superclass.superclass 90 | => Object 91 | >> 4.class.superclass.superclass.superclass.superclass 92 | => nil 93 | ``` 94 | 95 | - Anything used after a dot is a method. Methods can be chanined together. For e.g. `" some String ".trim.capitalize` 96 | 97 | - *Class Methods:* Unlike usual methods class methods in Ruby are attached to variables/constants using (`::`). E.g. `Network::Connect` 98 | 99 | - *Global variables:* Represented using `$variable_name` 100 | - *Instance variables:* Using @ symbol `@var`. Used to define attributes of an object 101 | - *Class Variables*: Represented by double ats. E.g. `@@class_var` 102 | 103 | - *Concatanation:* The `<<` method is used for this. 104 | 105 | 106 | Classes start 107 | with capital letters and typically use CamelCase to denote capitalization. You must prepend instance variables (one value per object) with 108 | @ and class variables (one value per class) with @@. Instance variables 109 | and method names begin with lowercase letters in the underscore_style. 110 | Constants are in ALL_CAPS. Functions and 111 | methods that test typically use a question mark (if test?). 112 | 113 | The attr keyword defines an instance variable and a method of the same name to access it 114 | 115 | Modules are ruby's equivalent to Java's interfaces and provide a way to replace multiple inheritance. 116 | 117 | Module: a bunch of functions and constants that when included into classes becomes a part of it. 118 | 119 | > This is called implementing a Mixin 120 | 121 | Ex: 122 | 123 | ```ruby 124 | 125 | module TestModule 126 | def func_1 127 | ... 128 | end 129 | 130 | def func_2 131 | ... 132 | end 133 | end 134 | 135 | class Something 136 | include TestModule 137 | def initialize() 138 | ... 139 | end 140 | end 141 | 142 | Something.new(...).func_1 143 | ``` 144 | 145 | Functions implemented in the class can also be used in a module. 146 | 147 | **Note:** When you're creating libraries with Ruby, it is a good practice to namespace your code under the name of your library or project. 148 | 149 | 150 | ### Self 151 | 152 | - Can be used to define class methods 153 | 154 | ```ruby 155 | class Post 156 | def self.print_author 157 | puts "The author of all posts is Jimmy" 158 | end 159 | end 160 | 161 | Post.print_author 162 | ``` 163 | -------------------------------------------------------------------------------- /languages/Rust.md: -------------------------------------------------------------------------------- 1 | ## Installation 2 | 3 | Rust can be installed through `rusetup` a CLI to manage Rust versions and 4 | associated tools. 5 | 6 | To install rust download and run the following shell script: 7 | 8 | ``` 9 | $ curl https://sh.rustup.rs -sSf | sh 10 | ``` 11 | 12 | ## Update 13 | 14 | ``` 15 | $ rustup update 16 | ``` 17 | 18 | ## Uninstall 19 | 20 | ``` 21 | $ rustup self uninstall 22 | ``` 23 | ### Hello World 24 | 25 | Write a file `main.rs` for word separators in filenames use underscores. 26 | 27 | ```rust 28 | fn main() { 29 | println!("Hello world"); 30 | } 31 | ``` 32 | 33 | **Compile and Run** 34 | 35 | ``` 36 | $ rustc main.rs 37 | $ ./main 38 | ``` 39 | Rust is an ahead-of-time compiled language, which means that one can compile a 40 | program, give the executable to someone else, and they can run it even without 41 | having Rust installed. 42 | 43 | ## Rust Program: Anatomy 44 | 45 | - The main function is the first one to run for every executable rust program. 46 | - Indentation uses 4 spaces (not tabs). 47 | - Function calls with `!` at the end like the `println!` in the above example 48 | means its calling a Rust macro. Normal function calls don't include `!`. 49 | - Semicolons are used to indicate end of expressions. 50 | 51 | 52 | ## Cargo 53 | 54 | Cargo is used for the following tasks in rust projects: 55 | - as a package manager 56 | - as a build system 57 | - downloads and builds the project dependencies 58 | 59 | ### Creating a project with Cargo 60 | 61 | ``` 62 | $ cargo new hello_cargo --bin 63 | $ cd hello_cargo 64 | ``` 65 | 66 | This creates a new executable in the current directory from which the command 67 | was run from named `hello_cargo`. 68 | 69 | The `--bin` argument creates a new binary by the name `hello_cargo`. A project 70 | directory with the same name is also with an src directory and a config file. 71 | 72 | The config file `Cargo.toml` contains the project metadata. 73 | 74 | A code package in rust is called a *crate*. All the logic of a crate should 75 | reside in the `src` directory. 76 | 77 | To build the project run `cargo build` from the project directory. This will 78 | create an executable in the folder `target/debug/hello_cargo`. This command also 79 | creates a `Cargo.lock` at the project root to keep a track of the versions of 80 | all the dependencies used just like a `pip.lock` file. 81 | 82 | Then run `./target/debug/hello_cargo` to execute the code. 83 | 84 | To compile and run the binary in one go use `cargo run`. 85 | 86 | `cargo check` to only compile the code without producing any binaries. 87 | 88 | `cargo build --release` when the project is ready for realease. This will 89 | generate the executable in the `target/release` directory. This binary will be 90 | created with optimizations to make the rust code run faster. 91 | 92 | ## Rust Basics 93 | 94 | - To include packages from the stdlib use `use std::io;`. This will include the 95 | io package from the standard library to be used in your program. 96 | - `let` keyword can be used to create variables. For e.g. `let foo = bar;` 97 | creates a variable `foo` (immutable by default) and binds it to a value `bar`. 98 | - `mut` keyword can be used before variable names to make them mutable. E.g. 99 | `let mut a = b;` 100 | - For comments we can use `//` 101 | - Variables can be assigned to different types. Like `let mut v = String::new()` 102 | This calls the `new` function of type `String` to return a new instance of 103 | string. `::new` means `new` is a function associated with the type `String` 104 | - Example of using stdlib methods provided we have declared `io` in the scope of 105 | the program using `use std::io`: 106 | ```rust 107 | io::stdin().read_line(&mut v) 108 | .expect("Failed to read line"); 109 | ``` 110 | `stdin` function returns an instance of `std::io::Stdin` which is a handle for 111 | user input. `read_line` is a method on the standard input handle to get the 112 | user input and store it as a string. `&mut` is a reference to a particular 113 | value so that it can be used throughout the code without the need to copy that 114 | value into the memory everytime it is used. The argument is mutable so that 115 | the `read_line` method can change the value of the string value and store the 116 | user input in it. 117 | `read_line` in this case returns `io:Result` which is an enum with two 118 | *variants* (fixed set of values that enums contain) `Ok` and `Err`. 119 | `io::Result` has an `expect` method that displays the message provided if it 120 | has the variant `Err`. 121 | - *Placeholders:* 122 | ```rust 123 | let x = 5; 124 | let y = 10; 125 | 126 | println!("x = {} and y = {}", x, y); 127 | ``` 128 | 129 | - All the crates in a project can be updated using `cargo update`. 130 | - To use external dependencies write this line at the top of the file. 131 | `extern crate ` 132 | - To generate the documentation `cargo doc --open`. Opens up autogenerated docs 133 | in the browser. 134 | 135 | ## Better Error Handling 136 | ```rust 137 | let guess: u32 = match guess.trim().parse() { 138 | Ok(num) => num, 139 | Err(_) => continue, 140 | }; 141 | ``` 142 | The underscore, `_`, is a catchall value; in this example, we’re saying we want to match all Err values, no matter what information they have inside them. 143 | 144 | ## Variable and Mutability 145 | 146 | - Variables in rust are *immutable* by default. The compiler will complain if 147 | you try to assign values to a variable mutilple times 148 | ``` 149 | error[E0384]: cannot assign twice to immutable variable `x` 150 | --> src/main.rs:4:5 151 | | 152 | 2 | let x = 5; 153 | | - first assignment to `x` 154 | 3 | println!("The value of x is: {}", x); 155 | 4 | x = 6; 156 | | ^^^^^ cannot assign twice to immutable variable 157 | ``` 158 | - Variables can be made mutable by adding `mut` in front of the variable name. 159 | 160 | > 161 | **Trade-offs:** In cases where you’re using large data structures, mutating an instance in place may be faster than copying and returning newly allocated instances. With smaller data structures, creating new instances and writing in a more functional programming style may be easier to think through, so lower performance might be a worthwhile penalty for gaining that clarity. 162 | 163 | - **Constants:** Are immutable forever and cannot be used with `mut`. They are 164 | declared using `const` instead of `let` and must be type annotated. 165 | `const MAX_POINTS: u32 = 100_000;` 166 | Constants can be declared in any scope, including the global scope, which makes them useful for values that many parts of code need to know about. The last difference is that constants may be set only to a constant expression, not the result of a function call or any other value that could only be computed at runtime. 167 | 168 | > Constants are valid for the entire time a program runs, within the scope they were declared in, making them a useful choice for values in your application domain that multiple parts of the program might need to know about 169 | 170 | ### Shadowing 171 | 172 | A new variable can be declared with the same name as that of a previous one. The 173 | new variable then shadows the previous one and the value displayed is that of 174 | most recent variable that shadowed its previous ones. 175 | ```rust 176 | fn main() { 177 | let x = 5; 178 | 179 | let x = x + 1; 180 | 181 | let x = x * 2; 182 | 183 | println!("The value of x is: {}", x); 184 | } 185 | ``` 186 | ``` 187 | The value of x is: 12 188 | ``` 189 | 190 | > Shadowing is different than marking a variable as mut, because we’ll get a compile-time error if we accidentally try to reassign to this variable without using the let keyword. 191 | By using let, we can perform a few transformations on a value but have the variable be immutable after those transformations have been completed. 192 | 193 | The other difference between mut and shadowing is that because we’re effectively creating a new variable when we use the let keyword again, we can change the type of the value but reuse the same name. For e.g: 194 | ```rust 195 | let spaces = " "; 196 | let spaces = spaces.len(); 197 | ``` 198 | The second spaces is a new variable so can be assigned a different types 199 | but the same thing with `mut` raises errors 200 | ```rust 201 | let mut spaces = " "; 202 | spaces = spaces.len(); 203 | ``` 204 | 205 | ## Data Types 206 | 207 | Rust has two data types `scalar` and `compound`. Since Rust is statically typed 208 | in cases where many types are possible we need to add type annotations or else 209 | the compiler might complain. 210 | 211 | ### Scalar 212 | 1. Integer: There are 10 kinds: i8, i16, i32, i64 & isize and their unsigned 213 | equivalents. The primary situation in which you’d use isize or usize is when indexing some sort of collection. 214 | 2. Floats: f32, f64 215 | 3. Bools: true and false 216 | 4. Chars 217 | 218 | ## Compounds 219 | 220 | 1. Tuples (multiple types of elements): 221 | ```rust 222 | fn main() { 223 | // declares a tuple `tup` 224 | let tup: (i32, f64, u8) = (500, 6.4, 1); 225 | 226 | // pattern matching to get individual tuple values (destructuring) 227 | let (x, y, z) = tup; 228 | 229 | let a = tup.2 // a = z 230 | 231 | println!("The value of y is: {} and z is: {}", y, z); 232 | } 233 | ``` 234 | This :point_up: can also be done without the type annotations 235 | 236 | 2. Array (same types of element): The following gives an index out of bounds 237 | ```rust 238 | fn main() { 239 | let a = [1, 2, 3, 4, 5]; 240 | let index = 10; 241 | 242 | let element = a[index]; 243 | 244 | println!("The value of element is: {}", element); 245 | } 246 | 247 | ``` 248 | 249 | ## Functions 250 | - Functions are pervasive. 251 | - Function names use snake case for naming. 252 | - Functions can be defined before and after the `main` function. 253 | - Type annotations are a must for functions parameters. 254 | - Functions can return values and if there is no `return` statement the last 255 | expression's value is implicitly returned as the return value for that 256 | function. 257 | 258 | ```rust 259 | fn main() { 260 | let x = plus_one(5); 261 | 262 | println!("The value of x is: {}", x); 263 | } 264 | 265 | fn plus_one(x: i32) -> i32 { 266 | x + 1 267 | } 268 | ``` 269 | 270 | Note: In rust the assignment of a value doesn't return the assigned value. So 271 | something like `x = y = 5` wouldn't be possible in Rust since statements don't 272 | have return values. 273 | 274 | ## Expressions 275 | ```rust 276 | fn main() { 277 | let x = 5; 278 | 279 | let y = { 280 | let x = 3; 281 | x + 1 282 | }; 283 | 284 | println!("The value of y is: {}", y); 285 | } 286 | 287 | ``` 288 | 289 | `The value of y is 4` (since the {} start a new scope with a different `x`) 290 | 291 | An expression does not contain a `;` at the end which would make it a statement 292 | Anything that returns a value is without a `;` 293 | 294 | 295 | ## Control Flow 296 | 297 | - The condition in an if statement must result into a bool. This gives an error 298 | ```rust 299 | fn main() { 300 | let number = 3; 301 | 302 | if number { 303 | println!("number was three"); 304 | } 305 | } 306 | ``` 307 | - Since `if`s are expressions they can be used on the RHS of `let`s 308 | ```rust 309 | let number = if condition { 310 | 5 311 | } else { 312 | 6 313 | }; 314 | ``` 315 | This on the other hand won't work. The type of number needs to be know at 316 | compile time. 317 | ```rust 318 | let number = if condition { 319 | 5 320 | } else { 321 | "six" 322 | }; 323 | ``` 324 | - There are 3 kinds of loops: `loop`, `while`, `for`. A `for` (more safe and 325 | concise than `while`s) loop looks like: 326 | ```rust 327 | fn main() { 328 | let a = [10, 20, 30, 40, 50]; 329 | 330 | for element in a.iter() { 331 | println!("the value is: {}", element); 332 | } 333 | } 334 | ``` 335 | 336 | ## Ownership 337 | 338 | Instead of an automatic garbage collector or manually allocating and 339 | de-allocating memory Rust uses a system of Ownership to assign memory to various 340 | 341 | **Stack and Heap**: Data with unknown size that might change later is stored 342 | on the heap and the rest (fixed size) on the stack. 343 | 344 | **Note:** When your code calls a function, the values passed into the function (including, potentially, pointers to data on the heap) and the function’s local variables get pushed onto the stack. When the function is over, those values get popped off the stack. 345 | 346 | **Rules:** 347 | -Each value in Rust has a variable that’s called its owner. 348 | -There can only be one owner at a time. 349 | -When the owner goes out of scope, the value will be dropped. 350 | 351 | 352 | The data types of the variables and arguments and the pointers to the heap 353 | addresses are all stored on the stack but the actual data values are stored on 354 | the heap. 355 | 356 | 357 | ```rust 358 | let ims = "immutable string stored on the stack" 359 | 360 | let mut s = String::from("hello"); 361 | s.push_str(", world!"); // push_str() appends a literal to a String 362 | println!("{}", s); // This will print `hello, world!` 363 | ``` 364 | 365 | As soon as the variable (owner of a data value) goes out of scope the memory 366 | allocated for that data is returned to the OS. At the `}` Rust calls a special 367 | funcions `drop` to do this. 368 | 369 | ```rust 370 | let x = 5; // bind the value 5 to x 371 | let y = x; // make a copy of the value in x and bind it to y 372 | 373 | let s1 = String::from("hello"); 374 | let s2 = s1; 375 | ``` 376 | 377 | The string however is made up of three parts: the ptr (that point to the 378 | contents stored on the heap), the length (how many bytes the string is acquiring 379 | right now) and the capacity (total memory that string receives from the OS) 380 | (all 3 stored on the stack). 381 | 382 | When we do `let s2 = s1;` we copy the length, capacity and ptr from the stack 383 | and bind it to `s2` but not the actual string data on the heap. 384 | 385 | *Besides this the original varialble is also invalidated before reaching its end 386 | of scope as soon as the copy of the three data elements from s1 to s2 occurs.* 387 | This is done because if both the variables point to the same scope once the 388 | scope ends and drop function is called both the variables will try to delete the 389 | same data on the heap leading to memory corruption. [This is known as the move 390 | where s1 is moved to s2 and is no longer valid] 391 | 392 | This way Rust automatically prevents making deep copies of your data which might 393 | give us a performance gain. 394 | 395 | **Making deliberate Deep Copy:** 396 | ```rust 397 | let s1 = String::from("hello"); 398 | let s2 = s1.clone(); // the heap data is copied and neither of the variable is 399 | // invalidated 400 | 401 | println!("s1 = {}, s2 = {}", s1, s2); 402 | ``` 403 | 404 | #### Contradicting Example (Stack-only data copy) 405 | ```rust 406 | let x = 5; 407 | let y = x; 408 | 409 | println!("x = {}, y = {}", x, y); 410 | ``` 411 | 412 | Without using `clone` `x` is still valid and hasn't moved to `y` because types 413 | like integers are stored on the stack so making their copies is much quicker 414 | as compared to making copies on the heap. 415 | 416 | Types like integers have a special trait called `Copy` that prevent them from 417 | getting moved but if the drop trait has been implemented for this type Rust won't 418 | be able to annotate this type with `Copy` 419 | 420 | ```rust 421 | fn main() { 422 | let s = String::from("hello"); // s comes into scope 423 | 424 | takes_ownership(s); // s's value moves into the function... 425 | // ... and so is no longer valid here 426 | 427 | let x = 5; // x comes into scope 428 | 429 | makes_copy(x); // x would move into the function, 430 | // but i32 is Copy, so it’s okay to still 431 | // use x afterward 432 | 433 | } // Here, x goes out of scope, then s. But because s's value was moved, nothing 434 | // special happens. 435 | 436 | fn takes_ownership(some_string: String) { // some_string comes into scope 437 | println!("{}", some_string); 438 | } // Here, some_string goes out of scope and `drop` is called. The backing 439 | // memory is freed. 440 | 441 | fn makes_copy(some_integer: i32) { // some_integer comes into scope 442 | println!("{}", some_integer); 443 | } // Here, some_integer goes out of scope. Nothing special happens. 444 | 445 | ``` 446 | 447 | Transferring Ownerships 448 | 449 | ```rust 450 | fn main() { 451 | let s1 = gives_ownership(); // gives_ownership moves its return 452 | // value into s1 453 | 454 | let s2 = String::from("hello"); // s2 comes into scope 455 | 456 | let s3 = takes_and_gives_back(s2); // s2 is moved into 457 | // takes_and_gives_back, which also 458 | // moves its return value into s3 459 | } // Here, s3 goes out of scope and is dropped. s2 goes out of scope but was 460 | // moved, so nothing happens. s1 goes out of scope and is dropped. 461 | 462 | fn gives_ownership() -> String { // gives_ownership will move its 463 | // return value into the function 464 | // that calls it 465 | 466 | let some_string = String::from("hello"); // some_string comes into scope 467 | 468 | some_string // some_string is returned and 469 | // moves out to the calling 470 | // function. 471 | } 472 | 473 | // takes_and_gives_back will take a String and return one. 474 | fn takes_and_gives_back(a_string: String) -> String { // a_string comes into 475 | // scope 476 | 477 | a_string // a_string is returned and moves out to the calling function 478 | } 479 | ``` 480 | 481 | The above approach is a bit tedious because we have to transfer ownership and 482 | then return it to the same function. 483 | 484 | *To just pass the value without the ownership we can use references* 485 | 486 | ### References and Borrowing 487 | 488 | ```rust 489 | fn main() { 490 | let s1 = String::from("hello"); 491 | 492 | let len = calculate_length(&s1); // refers to the string but does not own it 493 | 494 | println!("The length of '{}' is {}.", s1, len); 495 | } 496 | 497 | fn calculate_length(s: &String) -> usize { // Borrowing 498 | s.len() 499 | } // s goes out of scope but since it does not own s1 nothing happens 500 | ``` 501 | 502 | If references are used as function parameters it is called as *Borrowing* 503 | **References are also immutable by default** This can be changed by doing 504 | something like 505 | 506 | ```rust 507 | fn main() { 508 | let mut s = String::from("hello"); 509 | 510 | change(&mut s); 511 | } 512 | 513 | fn change(some_string: &mut String) { 514 | some_string.push_str(", world"); 515 | } 516 | ``` 517 | 518 | **Restriction:** Only one mutable reference can exist in a particular scope. 519 | 520 | Not valid 521 | 522 | ```rust 523 | 524 | let r1 = &mut s; 525 | let r2 = &mut s; 526 | 527 | ``` 528 | 529 | This prevents data races which occur in one of the following situations: 530 | - Two or more pointers access the same data at the same time. 531 | - At least one of the pointers is being used to write to the data. 532 | - There’s no mechanism being used to synchronize access to the data. 533 | 534 | A workaround for this could be creating new scopes using `{}` to allow 535 | multiple mutatable references. 536 | 537 | ```rust 538 | // mutable and immutable references cannot be combined 539 | 540 | let mut s = String::from("hello"); 541 | 542 | let r1 = &s; // no problem 543 | let r2 = &s; // no problem 544 | let r3 = &mut s; // BIG PROBLEM 545 | ``` 546 | 547 | It gives the following error. 548 | 549 | `error[E0502]: cannot borrow `s` as mutable because it is also borrowed as 550 | immutable` 551 | 552 | ### Dangling References 553 | 554 | ```rust 555 | fn main() { 556 | let reference_to_nothing = dangle(); 557 | } 558 | 559 | fn dangle() -> &String { // dangle returns a reference to a String 560 | 561 | let s = String::from("hello"); // s is a new String 562 | 563 | &s // we return a reference to the String, s 564 | } // Here, s goes out of scope, and is dropped. Its memory goes away. Danger! 565 | 566 | 567 | fn no_dangle() -> String { 568 | let s = String::from("hello"); 569 | 570 | s 571 | } 572 | ``` 573 | 574 | **Update:** I abandoned Rust after a while due to lack of time. These are the notes from my second attempt at learning Rust. 575 | 576 | - Rust has auto referencing/de-referencing 577 | 578 | LIFETIMES 579 | 580 | ```rust 581 | fn longest<'a>(x: &'a str, y: &'a str) -> &'a str { 582 | if x.len() > y.len() { 583 | x 584 | } else { 585 | y 586 | } 587 | } 588 | 589 | ``` 590 | 591 | When we pass concrete references to longest, the concrete lifetime that is substituted for 'a is the part of the scope of x that overlaps with the scope of y. In other words, the generic lifetime 'a will get the concrete lifetime that is equal to the smaller of the lifetimes of x and y. Because we’ve annotated the returned reference with the same lifetime parameter 'a, the returned reference will also be valid for the length of the smaller of the lifetimes of x and y. 592 | 593 | elision rules 594 | 595 | 596 | The first rule is that each parameter that is a reference gets its own lifetime parameter. In other words, a function with one parameter gets one lifetime parameter: fn foo<'a>(x: &'a i32); a function with two parameters gets two separate lifetime parameters: fn foo<'a, 'b>(x: &'a i32, y: &'b i32); and so on. 597 | 598 | The second rule is if there is exactly one input lifetime parameter, that lifetime is assigned to all output lifetime parameters: fn foo<'a>(x: &'a i32) -> &'a i32. 599 | 600 | The third rule is if there are multiple input lifetime parameters, but one of them is &self or &mut self because this is a method, the lifetime of self is assigned to all output lifetime parameters. This third rule makes methods much nicer to read and write because fewer symbols are necessary. 601 | 602 | 603 | - When invoking trait methods, the receiver is borrowed implicitly: 604 | 605 | ```rust 606 | fn main() { 607 | let n = Number { odd: true, value: 51 }; 608 | let mut m = n.clone(); 609 | m.value += 100; 610 | 611 | print_number(&n); 612 | print_number(&m); 613 | } 614 | ``` 615 | To highlight this: these are equivalent: 616 | 617 | ```rust 618 | let m = n.clone(); 619 | 620 | let m = std::clone::Clone::clone(&n); 621 | ```` 622 | 623 | - Marker traits like Copy have no methods `impl std::marker::Copy for Number {}` 624 | 625 | Some traits are so common, they can be implemented automatically by using the derive attribute: 626 | 627 | ```rust 628 | #[derive(Clone, Copy)] 629 | struct Number { 630 | odd: bool, 631 | value: i32, 632 | } 633 | 634 | // this expands to `impl Clone for Number` and `impl Copy for Number` blocks. 635 | ``` 636 | 637 | - `Box` means the function will return a type that implements the Error trait, but we don’t have to specify what particular type the return value will be. This gives us flexibility to return error values that may be of different types in different error cases. The dyn keyword is short for “dynamic.” 638 | 639 | - For many types in Rust, there are owned and non-owned variants: 640 | - Strings: String is owned, &str is a reference 641 | - Paths: PathBuf is owned, &Path is a reference 642 | - Collections: Vec is owned, &[T] is a reference 643 | > Rust has slices - they're a reference to multiple contiguous elements. 644 | 645 | - To make the rightmost bound of a range inclusive use `=` like this `println!("{:?}", (..=20).contains(&20))` 646 | 647 | - We need to use trait bounds when using closures (which implement at least one of the following `Fn`, `FnMut`, `FnOnce` trait) with structs 648 | ```Rust 649 | // a simple example of memoization 650 | 651 | struct Cacher 652 | where 653 | T: Fn(u32) -> u32, 654 | { 655 | calculation: T, 656 | value: Option, 657 | } 658 | 659 | impl Cacher 660 | where 661 | T: Fn(u32) -> u32, 662 | { 663 | fn new(calculation: T) -> Cacher { 664 | Cacher { 665 | calculation, 666 | value: None, 667 | } 668 | } 669 | 670 | fn value(&mut self, arg: u32) -> u32 { 671 | match self.value { 672 | Some(v) => v, 673 | None => { 674 | let v = (self.calculation)(arg); 675 | self.value = Some(v); 676 | v 677 | } 678 | } 679 | } 680 | } 681 | ``` 682 | 683 | - Unlike functions, closures can capture their enviroment or use the variables defined in the parent scope (**Note** this has a memory overhead) 684 | ```Rust 685 | fn main() { 686 | let x = 4; 687 | let equal_to_x = |z| z == x; 688 | let y = 4; 689 | assert!(equal_to_x(y)); 690 | } 691 | ``` 692 | - Closures are able to capture the enviroment because of the following traits: 693 | - `Fn` borrows values from its environment immutably 694 | - `FnMut` borrows them mutably and hence we can change the values borrowed inside closures implementting this trait 695 | - `FnOnce` the closure consumes the values by taking ownership of them only "once", hence the name. 696 | 697 | - Rust can infer closure traits. All closures implement FnOnce because they can all be called at least once. Closures that don’t move the captured variables also implement FnMut. 698 | - A closure can be forced to take ownership of a value using the `move` keyword, used when passing data between two threads to avoid race conditions `let equal_to_x = move |z| z == x;` 699 | - Any type in Rust that can be iterated implements the `Iterator` trait with the `next` method and returns an item wrapped in a `Some` and a `None` at the end. We can either get the items by calling `next` on an iterator (returned using the `iter` method) or simply using a for loop 700 | ```Rust 701 | // The iterator trait 702 | 703 | 704 | pub trait Iterator { 705 | type Item; 706 | 707 | fn next(&mut self) -> Option; 708 | 709 | // methods with default implementations elided 710 | } 711 | 712 | // instantiating an iterator 713 | 714 | fn main() { 715 | let v1 = vec![1, 2, 3]; 716 | // the iterator is lazy 717 | let v1_iter = v1.iter(); 718 | for val in v1_iter { 719 | println!("Got: {}", val); 720 | } 721 | } 722 | 723 | ``` 724 | - The `iter` method returns an iterator over immutable references and the `next` method returns immutable references to the values, but we can also use `iter_mut` to return mutable references and `into_iter` to make a iterator that takes ownership of the values instead of `iter()` 725 | - *consumer adaptor* are methods that consume elements from an iterator, for example `sum` method takes the ownership of an iterator and returns the sum of elements. 726 | - *iterator adaptors* are methods defined over `Iterator` trait, they allow you to change the iterators into new iterators `v1.iter().map(|x| x + 1);` These are only executed when they are consumed, all adaptors are lazy for example adding a `collect` call to the above snippet `let v2: Vec<_> = v1.iter().map(|x| x + 1).collect();`. `filter` is another iterator adaptor and works the same way as Clojure's `filter`. 727 | - Implemet `Iterator` for custom types: 728 | ```Rust 729 | struct Counter { 730 | count: u32, 731 | } 732 | 733 | impl Counter { 734 | fn new() -> Counter { 735 | Counter { count: 0 } 736 | } 737 | } 738 | 739 | impl Iterator for Counter { 740 | type Item = u32; 741 | 742 | fn next(&mut self) -> Option { 743 | if self.count < 5 { 744 | self.count += 1; 745 | Some(self.count) 746 | } else { 747 | None 748 | } 749 | } 750 | } 751 | ``` 752 | - Once the trait is implemented for the custome type we can use all the other Iterator trait methods defined in standard library like `sum`, `filter`, `map`, etc, the only method that we need to define is next the rest are reusable. 753 | - We can use `///` for documentation comments and it supports markdown. Commonly used docs sections are Examples (which are also doc tests), Errors (if the function returns a `Result`), Panics, Safety (if the function is `unsafe to call`) 754 | - `cargo doc --open` can be used to build the docs and open them in your browser 755 | - `//!` are used for crate/module level docs 756 | - `pub use` statements can be used to re-expport crate modules to make it useful for library use to use different modules exposed by our crates. This is useful in case of nested modules and makes our crate user friendly 757 | ```Rust 758 | // CRATE SOURCE 759 | 760 | pub use self::kinds::PrimaryColor; 761 | pub use self::kinds::SecondaryColor; 762 | pub use self::utils::mix; 763 | 764 | pub mod kinds { 765 | // --snip-- 766 | } 767 | 768 | pub mod utils { 769 | // --snip-- 770 | } 771 | 772 | // USAGE CODE 773 | 774 | use art::mix; 775 | use art::PrimaryColor; 776 | 777 | fn main() { 778 | // --snip-- 779 | } 780 | ``` 781 | - Smart Pointers are usually implemented using structs and store some metadata along with references/pointers to values. They implement the `Deref` and `Drop` traits 782 | - `Box` is a smart pointer and can be used in the following situations: 783 | - When you have a type whose size can’t be known at compile time and you want to use a value of that type in a context that requires an exact size 784 | - When you have a large amount of data and you want to transfer ownership but ensure the data won’t be copied when you do so 785 | - When you want to own a value and you care only that it’s a type that implements a particular trait rather than being of a specific type 786 | - Box helps us in storing recursive types, by storing them on the heap and storing a pointer to that data on the stack (a kind of indirection). Example 787 | ```Rust 788 | enum List { 789 | Cons(i32, Box), 790 | Nil, 791 | } 792 | 793 | use crate::List::{Cons, Nil}; 794 | 795 | fn main() { 796 | let list = Cons(1, Box::new(Cons(2, Box::new(Cons(3, Box::new(Nil)))))); 797 | } 798 | ``` 799 | - Rust treats `Box` like a normal reference because it implements the `Deref` trait essentially has a `deref` method that returns a "reference" to the value so that there is not transfer of ownership and still allow the dereference operator `*` to get the value of the `&` reference that was returned 800 | 801 | ```Rust 802 | use std::ops::Deref; 803 | 804 | impl Deref for MyBox { 805 | type Target = T; 806 | 807 | fn deref(&self) -> &T { 808 | &self.0 809 | } 810 | } 811 | ``` -------------------------------------------------------------------------------- /operating-systems/ostep.md: -------------------------------------------------------------------------------- 1 | # Operating Systems 2 | ## These notes were made while studying operating systems from the OSTEP book. 3 | 4 | ## Part1: Virtualization 5 | 6 | ### Process API 7 | 8 | - The OS uses a technique to run multiple concurrent process called time-sharing which enables virtualization. 9 | - A process is just an API or abstraction for a running program and the system resources like memory that it can use. 10 | - The process API has methods such as create, destroy, wait, status and others. 11 | - Program code (or instructions) are permanently saved on the disk along with static data (if any). It is loaded into the memory (RAM) when a process needs to be created. (these programs are loaded lazily) 12 | - After the process is loaded into the memory some amount of the memory is allocated for storing the program, static data and for the runtime stack, running it. The OS might also initialize some heap memory and I/O based initialization such as giving the process the 3 file descriptors (stdin, stdout, stderr). 13 | - There are 3 basic process states: running, ready and blocked. 14 | - There are special states like the zombie states which means that a process has exited but has not yet been cleaned up. 15 | - A process keeps a track of several differnt details. Here is a basic process struct: 16 | 17 | ```C 18 | // the registers xv6 will save and restore 19 | // to stop and subsequently restart a process 20 | struct context { 21 | int eip; 22 | int esp; 23 | int ebx; 24 | int ecx; 25 | int edx; 26 | int esi; 27 | int edi; 28 | int ebp; 29 | }; 30 | 31 | // the different states a process can be in 32 | enum proc_state { UNUSED, EMBRYO, SLEEPING, RUNNABLE, RUNNING, ZOMBIE }; 33 | 34 | // the information xv6 tracks about each process 35 | // including its register context and state 36 | struct proc { 37 | char *mem; // Start of process memory 38 | uint sz; // Size of process memory 39 | char *kstack; // Bottom of kernel stack for this process 40 | enum proc_state state; // Process state 41 | int pid; // Process ID 42 | struct proc *parent; // Parent process 43 | void *chan; // If non-zero, sleeping on chan 44 | int killed; // If non-zero, have been killed 45 | struct file *ofile[NOFILE]; // Open files 46 | struct inode *cwd; // Current directory 47 | struct context context; // Switch here to run process 48 | struct trapframe *tf; // Trap frame for the current interrupt 49 | }; 50 | ``` 51 | 52 | ### Mechanism 53 | 54 | - OS aims to virtualize CPU resources while making sure that performance and control (over procs) is achieved. 55 | > AISDE: System calls look similar to procedure calls because they are infact procedures defined in the c library. A call to e.g. `open()` calls the C code which implements a trap instruction (hand-coded assembly instructions that are provided by the hardware to trap into the kernel and return-from-trap) which transfers control to the kernel. After the kernel is done completing the task the result and the control both are transferred back the procedure (user program). 56 | 57 | - Control over processes is achieved by having two process modes. The code which runs in `user mode` cannot do priveleged operations such as I/O whereas `kernel mode` allows the code (the OS) to do all sorts of priveledged 58 | - Whenever a user process wants to execute priveledged operations it calls the relevant syscall to do it. Syscalls act as a bridge b/w user and kernel mode. Syscalls are essentially an API for priveledged operations that the kernel exposes to the user programs. 59 | - When toggling b/w proc modes the hardware must make sure to save all the necessary registers of the caller to be able to return after completion. For e.g. 60 | > On x86, for example, the processor will push the program counter, flags, and a few other registers onto a per-process kernel stack; the return-fromtrap will pop these values off the stack and resume execution of the usermode program 61 | - A trap table is used by the kernel to know which code to run in kernel mode. The trap table is initialized during system boot (which runs in kernel mode). For example, what code should run when a when a keyboard interrupt occurs. So during boot up the hardware is made to remember address of syscall handler. [Ref: Fig 6.2 Pg 71] 62 | - This approach offers a certain protection to system resources as instead of referring to a certain piece of code to jump to the user programs request a certain service through syscalls. This is called Limited Direct Execution (LDE protocol). 63 | - Since the two proc modes solve the priveledge the OS also needs to find a way to switch b/w processes and more importantly switch b/w a running user proc and itself. 64 | - There are two ways of doing this. A co-operative approach expects the control to be transferred back to OS if (1) the user program executes a syscall to transfer control back & (2) if it performs an illegal operation and the OS stops and terminates it. But this approach fails in cases in which neither of this happens (say an infinite loop) 65 | - A non-cooperative approach uses a special timer interrupt (started by the OS during boot time) which is programmed to transfer the control back to the OS at specified intervals of time. 66 | - After transferring the control the previous process can either be resumed or the OS can switch to a new process (decided by the scheduler). In case of a switch a piece of code known as context switch is executed (which saves a few registers etc) 67 | 68 | ##### Context Switch 69 | 70 | - The first is when the timer interrupt occurs; in this 71 | case, the user registers of the running process are implicitly saved by the hardware, using the kernel stack of that process. 72 | - The second is when the OS decides to switch from A to B; in this case, the kernel registers are explicitly saved by the software (i.e., the OS), but this time into memory in 73 | the process structure of the process. The latter action moves the system from running as if it just trapped into the kernel from A to as if it just trapped into the kernel from B 74 | 75 | ### Scheduling Policies 76 | 77 | - Turnaround time of job `Tturnaround = Tcompletion − Tarrival` 78 | - In FCFS if the longer (resource heavy) jobs are run prior to the shorter ones it can increase the TAT by a lot. This is known as the convoy effect whereas in SJF this effect is minimized to quite a large extent (if all procs arrive at the same time). 79 | - Both these policies are non-preemptive and not used in modern schedulers 80 | - SCTF a preemptive SJF can be used instead to run the jobs which have the least remaining time to complete. This leads to lesser avg TAT 81 | - Another metric Response Time `Tresponse = Tfirstrun − Tarrival` when taken into account highlights the area where SCTF or SJF might not be suitable policies, the area being interactivity. If all the procs arrive at the same time according to the aforementioned algorithms the last proc will have a large enough response time to hamper the interactivity of the system. 82 | - Round Robin greatly decreases response time for each process by running each proc for a specified time slice/quantum, although RR dissappoints when we take TAT into account. The length of the time slice is usually a multiple of the **timer interrupt**. 83 | - Shorter time slices lead to better responsiveness but if they are too short they can lead to a much slower peformance due to an increased overhead of execessive context switching. 84 | - To reduce the cost of context switching the duration of the time slice can be adjusted in such a way that a very small %age of the time slice is spent in context switching. -------------------------------------------------------------------------------- /protocols/HTTP.md: -------------------------------------------------------------------------------- 1 | ## HTTP 2 | 3 | *TCP* is a connection-oriented protocol while *UDP* is connectionless. 4 | 5 | **IP datagram:** The IP layer of the network stack provides connectionless service by transfering datagrams. All of which are independent of each other and any information about datagram associations must be provided by a higher network layer 6 | 7 | > What is a datagram? 8 | > Basic unit that is transfered over the network. In short `headers + payload = datagram`. 9 | 10 | IP also supplies a checksum including its own header (which in turn includes the source and destination of the datagram) 11 | 12 | IP also handles routing and breaking up large datagrams into smaller ones and reassembling them at the other end. 13 | 14 | **UDP:** 15 | - Built on top of the IP layer. 16 | - Connectionless and unreliable packet delivery just as IP 17 | - It adds a checksum for contents of the datagram and the port numbers to the IP layer. 18 | 19 | **TCP** 20 | - Another layer built on top of IP 21 | - Focussed on accuracy and provides ordered delivery of stream of packets 22 | - Provides logic to give a reliable connection oriented protocol on top of IP 23 | - It provides a virtual circuit for two processes to communicate with each other 24 | 25 | ### Adressing Scheme 26 | Schemes are used to locate servies and devices over a network. 27 | 28 | #### IPV4 29 | - 32 bit int written on the network interface card of a device 30 | - Format: 4 bytes in decimal separated by a "." (like 127.0.0.1) 31 | --------------------------------------------------------------------------------