├── README.md
└── readings
├── decrease_and_conquer.md
├── frequency_count.md
└── multiple_pointers.md
/README.md:
--------------------------------------------------------------------------------
1 | # Intro to Algorithms
2 |
3 | ## Purpose
4 | The main goal of this guide is to get developers started with algorithms. This guide will serve as a roadmap for developers who do not have not taken an introductory algorithms course or need a refresher on basic algorithm design.
5 |
6 | *In addition, this guide will also prepare developers for the [Outco program](http://outco.io) and their admissions fundamentals check.*
7 |
8 | ## Commitment and Requirements
9 | Please allow for ample time practice; up to **2-4 weeks** to allow the concepts to be internalized. It is best to follow through with the recommended structure in the order given.
10 |
11 | **Minimum Requirements**:
12 |
13 | * Fluency in a programming language (e.g., Java, JavaScript, Python, Ruby, PHP, Objective-C, C++, C)
14 | * Strong knowledge of looping constructs like `for` and `while` loops.
15 | * Strong knowledge of control flow `if`, `else`, `and`, `or`
16 | * Array: lookup, insertion, looping
17 | * Object (Hashtables): lookup, looping, insertion
18 | * Language specific built-in methods
19 | * Mathematics: Algebra (e.g., logarithms, exponentials, primes, powers)
20 |
21 | If you are missing some of the minimum requirements, then I recommend taking a refresher course. Here are some suggestions that are pleasant to digest:
22 |
23 | * **JavaScript** [Codecademy](https://www.codecademy.com/learn/introduction-to-javascript) Sections 1-8, no need for PRO material
24 | * **JavaScript** [Eloquent JavaScript](http://eloquentjavascript.net/): Chapters 1-4
25 | * **Ruby** [Codecademy](https://www.codecademy.com/learn/learn-ruby) Sections 1-6
26 | * **Python** [Learn Python the Hard Way](https://learnpythonthehardway.org/book/)
27 | * **Java** [Java Tutorial for Complete Beginners](https://www.udemy.com/java-tutorial/#curriculum)
28 |
29 |
30 | ## Strategy
31 |
32 | The strategy to develop your algorithm skills revolves around a few extremely important concepts that need to be repeated consistently. They are: 1) acquiring **knowledge**, 2) **applying** concepts, and 3) **testing** one's understand of the concepts.
33 |
34 | But there is a structured approach to algorithms that will make your life easier to learn faster. In math, you have prerequisites that you should take before taking a more advanced class. These prerequisites are laid out in an organized fashion so you do not get overwhelmed skipping over foundational concepts before moving on.
35 |
36 | Same with algorithms and data structures. There is a structure to the topics that you can learn that will greatly improve your understanding if you progress through the algorithms. Here are the steps I suggest you follow:
37 |
38 | 1. Learn to evaluate efficiency (complexity) of algorithms
39 | 2. Develop a process for problem solving
40 | 3. Learn basic algorithm patterns
41 | 4. Learn basic data structures
42 | 5. Put into practice
43 | 6. Start easy and progress harder
44 | 7. Track and measure
45 |
46 | Lets cover each one of those areas step by step.
47 |
48 | ## 1. Evaluating Efficiency
49 |
50 | Evaluating the efficiency of algorithms is the first step to really developing the ability solve challenging algorithms. If you cannot measure the efficiency of an algorithm, how can one compare the scalability of one approach to another.
51 |
52 | So lets start with time and space complexity.
53 |
54 | **Time complexity** - measures the amount of computations
55 | **Space complexity** - measures the amount of memory
56 |
57 | The amount of computations or memory required to solve the algorithm is measured relative to the input size as the input scales up. Remember, for both time and space complexity, we are measuring against the **size of the input**.
58 |
59 | We use **Big-O notation** to measure time and space complexity. Big-O is a mathematical way to gauge the rate in which something grows. Big-O refers to the worse case scenario. That means, if we chose the worse possible input for the algorithm, how long would it take? There is a bunch more technical details on Big-O, how it is accessed in academia vs industry, but for now, we should wait for later to explore this topic more.
60 |
61 | Before we dive further into how to analyze for Big-O, lets focus on the input first.
62 |
63 | ### 1a. Determining What is Scaling
64 | When you are trying to access the time or space complexity of an algorithm, always ask yourself: *what about my input is scaling?*
65 |
66 | ---
67 |
68 | **Q:** For the following code below, what is scaling?
69 |
70 | ```javascript
71 |
72 | // Take in an integer and print its value
73 | function printInteger(num) {
74 | console.log(num);
75 | }
76 |
77 | ```
78 |
79 | **A:** If your answer was the **value** of ```num``` then you are correct. We would be scaling the value of ```num``` itself.
80 |
81 | ---
82 |
83 | Lets try another problem:
84 | **Q:** For the following code below, what is scaling?
85 |
86 | ```javascript
87 |
88 | // Print the first item in the array
89 | function printFirst(arr) {
90 | console.log(arr[0]);
91 | }
92 |
93 | ```
94 |
95 | A: In this case, its the **length of the array**. If you are given an array as input, it usually is the length of the array.
96 |
97 | ---
98 |
99 | It can get tricky with different types of inputs, here are some of the input types and common factors that are scaling:
100 |
101 | | Input | Common Factor that is Scaling |
102 | |--- |---|
103 | | Integer | Magnitude of number|
104 | | String | Length of String|
105 | | Array | Length of Array|
106 | | LinkedList | Number of nodes|
107 | | Tree | Number of nodes|
108 | | Hashtable | Number of key-value pairs|
109 | | Matrix | Width and height of matrix|
110 | | Graph | Number of vertices and edges|
111 |
112 | *Disclaimer: These are 'common' factors that scale. Sometimes with particular problems there might be less obvious factors that scale.*
113 |
114 |
115 | ### 1b. Units of Time and Space
116 | To help you begin to analyze for complexity you need to get a sense of what a unit of time and a unit of space is.
117 |
118 | **One Unit of Time**
119 |
120 | * A unit of time can be one arithmetic operation: ```5 + 7```
121 | * Or printing something: ```console.log('hello')```
122 | * Or instantiating a new variable: ```let name = 'foo';```
123 | * Or accessing an item in an Array: ```arr[5]```
124 | * Or returning something: ```return 'foobar';```
125 |
126 | **One Unit of Space**
127 |
128 | * A unit of space can be creating a single new variable: ```var i = 1;```
129 | * Or creating an empty array: ```var list = [];```
130 | * Or adding a new item to your list: ```list.push('foo');```
131 | * Or adding a key-value pair into a hashtable: ```obj[foo] = 'bar';```
132 |
133 |
134 | ### 1c. Analyze Line by Line
135 |
136 | When you are starting out, try evaluating the complexity line by line and determine the complexity for each line. Then sum up the complexity of each line to get the total.
137 |
138 | Lets start with time complexity for now.
139 |
140 | ---
141 | **Q:** Answer these two questions for the code below:
142 |
143 | 1. What is scaling?
144 | 2. What is the amount of operations for each line?
145 |
146 | ```javascript
147 | // print the first and last item in the array
148 | function printFirstLast(arr) {
149 | console.log(arr[0]);
150 | console.log(arr[arr.length-1]);
151 | }
152 | ```
153 | ---
154 | **A:**
155 |
156 | 1. The length of the input array is scaling
157 | 2. The first ```print``` statement has 2 operations and the second ```print``` statement has has 4 operations. Do you know why?
158 |
159 | Here is how we could mark the computations line by line:
160 |
161 | ```javascript
162 | // print the first and last item in the array
163 | function printFirstLast(arr) {
164 | console.log(arr[0]); // 2
165 | console.log(arr[arr.length-1]); // 4
166 | }
167 |
168 | // total = 2 + 4 = 6
169 | ```
170 |
171 | The first print statement has an array access and then a printing to the console, so thats 2 operations. The second print statement involves: accessing the length, subtracting from the length, accessing the index, and printing to the console. Thats a total of 4 for the second print statement.
172 |
173 | In total there are 6 computations, we would say this is O(1) time complexity. Read the next section to find out why.
174 |
175 | ### 1d. Drop Lower Order Terms and Coefficients
176 |
177 | With complexity analysis, we need to do two things after finding the total amount of operations or memory:
178 |
179 | 1. Drop lower order terms
180 | 2. Drop coefficients in front of the leading term.
181 |
182 | This is because for Big-O analysis we only care about the largest order of magnitude. As the input size scales extremely large, the lower terms make less of an impact. Furthermore, since the magnitude is what is used for Big-O, we don't include the coefficients in front of the leading term.
183 |
184 | ---
185 | **Q:** For the previous example, we determined the algorithm has 6 operations, reduce it using the two conditions above.
186 |
187 | **A:**
188 | 1. 6 is the lowest (and only term)
189 | 2. 6 itself is a coefficient of the constant term, thus it gets reduced to 1.
190 |
191 | So the time complexity for the function above is **O(1) or constant time**.
192 |
193 | ---
194 |
195 | What does O(1) time mean? Well it means the amount of operations the algorithm takes to execute as the input scales remains constant. This makes sense, because no matter how large the input array gets, the function ```printFirstLast``` will still take roughly the same time to execute.
196 |
197 | Lets get some more practice looking at the following 10 totals and reduce it to the magnitude order for Big-O. You may have to look up some power laws. Also check out [Big-O Cheetsheet](http://bigocheatsheet.com/) if you are unsure how logarithmic terms compare to other terms.
198 |
199 | ```
200 | PROBLEM SET 1:
201 | Reduce the following to it Big-O magnitude:
202 | 1) 5 + N
203 | 2) N + N^2
204 | 3) 15N + 13N
205 | 4) 10000
206 | 5) log(N) + 1
207 | 6) log(N) * 3 + 14N + 3
208 | 7) Nlog(N) + 3N^2
209 | 8) N^3 + log(N^4)
210 | 9) N! + 180000N^2
211 | 10) 300N^5 + 15002^N + 3N
212 | ```
213 |
214 | (Scroll to the bottom to get the answers to this problem set.)
215 |
216 | ### 1e. Looping are Sometimes Linear but not Always
217 |
218 | **For Loops:** Lets explore looping more. Most of the time, when we loop through a collection that also happens to be an input, then we would consider the loop to be O(N).
219 |
220 | **Q:** Determine the time complexity of this algorithm below:
221 |
222 | ```javascript
223 | // print each item in the array.
224 | function printAll(arr) {
225 | for(let i = 0; i < arr.length; i++) {
226 | console.log(arr[i]);
227 | }
228 | }
229 | ```
230 |
231 | **A:**
232 | You may have guessed that it is **linear** time or **O(N)**. And yes, that would be correct.
233 |
234 | ---
235 |
236 | To determine why the operations for a loop has a linear order magnitude. Lets explore it in more detail line by line. When there is a loop we have to multiply the operations inside the loop by the total number of iteration the loop runs.
237 |
238 | ```javascript
239 | // print each item in the array.
240 | function printAll(arr) {
241 | // 1 (initiating i to 0 is an operation)
242 | // the loop runs N iterations
243 | for(let i = 0; i < arr.length; i++) { // 2 operations per break condition check
244 | console.log(arr[i]); // 2 operations to print
245 | }
246 | }
247 |
248 | // total = 1 + N * (2 + 2) = 4N + 1
249 | // reduce down to O(N) or linear time complexity
250 | ```
251 |
252 | **While Loops** Sometimes loops can be tricky, especially with ```while``` loops. You have to really evaluate the code to see how many times the loops runs and if it is proportional to the input.
253 |
254 | ---
255 |
256 | **Q:** Evaluate the following code for time complexity:
257 |
258 | ```javascript
259 | // print the first ten items in the array
260 | function firstTen(arr) {
261 | let i = 0;
262 | while(i < 10 && i < arr.length) {
263 | console.log(arr[10]);
264 | i++;
265 | }
266 | }
267 | ```
268 |
269 | **A:** Lets explore the code line by line, keep in mind we have
270 |
271 |
272 | ```javascript
273 | // print the first ten items in the array
274 | function firstTen(arr) {
275 | let i = 0; // 1
276 | // maximum of 10 iterations
277 | while(i < 10 && i < arr.length){ // 4 operations for each break condition check
278 | console.log(arr[10]); // 2 operations to print
279 | i++; // 1 to increment
280 | }
281 | }
282 |
283 | // total = 1 + 10 * (4 + 2 + 1) = 71
284 | // reduce down to O(1) or constant time complexity
285 | ```
286 | ---
287 | **While Loop continued:** Lets evaluate a slight variation of the previous problem.
288 |
289 | **Q:** Evaluate the following code for time complexity:
290 |
291 | ```javascript
292 | // print from the 11th to the last items in the array.
293 | function afterTen(arr) {
294 | let i = 11;
295 | while(i < arr.length) {
296 | console.log(arr[i]);
297 | i++;
298 | }
299 | }
300 | ```
301 | **A:** Lets explore the code line by line, keep in mind we have
302 |
303 |
304 | ```javascript
305 | // print from the 11th to the last items in the array.
306 | function afterTen(arr) {
307 | let i = 11; // 1
308 | // maximum of N-10 iterations
309 | while(i < arr.length){ // 2 operations for each break condition check
310 | console.log(arr[i]); // 2 operations to print
311 | i++; // 1 to increment
312 | }
313 | }
314 |
315 | // total = 1 + (N - 10) * (2 + 2 + 1) = 5N + 51
316 | // reduce down to O(N) or Linear time complexity
317 | ```
318 |
319 | Notice how when the problem prints from index 11 to the end of the array, when the input gets larger, so does the amount of iterations in the while loop.
320 |
321 | ---
322 | ### 1f. Analyze by Chunks
323 |
324 | Analyzing line by line is good for when you are starting out. But you may quickly learn that it isn't always necessary. We did a lot of work to compute the exact number of operations only to reduce it back down.
325 |
326 | Knowing that lower order terms and coefficients will get dropped anyway, you can speed up your analysis by looking at **chunks** of code at a time, and evaluating based on **magnitude**.
327 |
328 | Lets try this out on a few problems.
329 |
330 | ---
331 | **Q:** Evaluate the following problem for time complexity:
332 |
333 | ```javascript
334 | // given an array of integers, return all the even items.
335 | function evens(arr){
336 | let results = [];
337 | for(let i = 0; i < arr.length; i++) {
338 | if(arr[i] % 2 === 0) {
339 | results.push(arr[i]);
340 | }
341 | }
342 | return result;
343 | }
344 | ```
345 | **A:** Lets separate this into chunks and use magnitudes.
346 |
347 | ```javascript
348 | // given an array of integers, return all the even items.
349 | function evens(arr){
350 | let results = []; // constant
351 | for(let i = 0; i < arr.length; i++) { // linear
352 | if(arr[i] % 2 === 0) { // constant
353 | results.push(arr[i]);
354 | }
355 | return result; // constant
356 | }
357 |
358 | ```
359 | The loop is a linear chunk of operations because there are a lineat amount of iterations and everything inside of it is constant. A linear chunk is the largest out of the entire algorithm and therefore we have **O(N) time**.
360 |
361 | ---
362 |
363 | ### 1g. Nested vs Un-nested Loops
364 |
365 | Be careful not to mistakenly assume that just because there are multiple ```for-loops``` in an algorithm that that it is automatically quadratic O(N^2) complexity or more. The simple rule you should remember here is:
366 |
367 | * **nested loops multiply**
368 | * **un-nested loops add**
369 |
370 | Lets go over a few different examples and analyze problems to see how nested versus un-nested loops affect the overall complexity of the algorithm.
371 |
372 | **Nested Loop** Lets take a look at a problem with a nested loop:
373 |
374 | ```javascript
375 | // print all values that have a matching pairs
376 | function findPairs(arr) {
377 | for (let i = 0; i < arr.length; i++) {
378 | for (let j = i; j < arr.length; j++) {
379 | if(arr[i] === arr[j]) {
380 | console.log(i, 'is a pair');
381 | }
382 | }
383 | }
384 | }
385 | ```
386 | The problem here has a nested loop, lets evaluate this algorithm for time complexity:
387 |
388 | ```javascript
389 | // print all values that have a matching pairs
390 | function findPairs(arr) {
391 | for (let i = 0; i < arr.length; i++) { // linear
392 | for (let j = i; j < arr.length; j++) { // linear
393 | if(arr[i] === arr[j]) { // constant
394 | console.log(i, 'is a pair');
395 | }
396 | }
397 | }
398 | }
399 | ```
400 | Since there are two nested loops, we multiply linear, linear, and constant together to make quadratic O(N^2 ) time complexity. Note how the second loop starts at index ```i``` and moves towards the end of the array. The second loop is getting smaller and smaller each time. We still consider the inner loop linear because on average the number of iterations will be about 0.5N. Since we drop coefficients anyway, the inner loop is simplified to linear.
401 |
402 |
403 | ### 1h. Understand Complexity of Native Methods
404 |
405 | If your language has a native sorting method, do you know what happening behind the scenes? Take sure you take the time to research and look up what sorting method your language is using. Then loop up the time and space complexity requirements of that sorting function.
406 |
407 | The native methods of your language have inherent time and space complexity that you need to understand. One good way you can learn more about the time and space complexity of these functions is to code some of them out yourself.
408 |
409 | Here is an example list for JavaScript arrays:
410 |
411 | * Array - indexOf
412 | * Array - map
413 | * Array - forEach
414 | * Array - reverse
415 | * Array - sort
416 |
417 | In other languages you may look up methods for:
418 |
419 | * Java - ArrayList, HashMap
420 | * Python - List, Dictionary
421 | * Ruby - Array, Object
422 |
423 | Many of the methods associated with these data structures are going to be more costly than time O(1) and some create a new copy which can lead to O(N) space.
424 |
425 | Try rewritting some of the native methods yourself with these problems:
426 |
427 | ```
428 | PROBLEM SET 2:
429 | You are given an collection based on your language, write a few functions that perform operations on this collection. Determine with the time complexity is for each solution.
430 |
431 | Based on your language, the collection will be in this format:
432 | JavaScript: Array
433 | Java: ArrayList
434 | Python: List
435 | Ruby: Array
436 | Other: Dynamic Array
437 |
438 | 1) indexOf: given a collection and a target value, return the index in
439 | which the value at the index matches the target value. If there is no
440 | match, return -1.
441 |
442 | 2) evens: given a collection of integers, return only the even
443 | values in a new copy of the collection
444 |
445 | 3) split: given a string of characters, return a collection with
446 | each character separated into its own separate item.
447 |
448 | 4) sum: given a collection of integers, return the sum of them
449 |
450 | ```
451 | ***Solutions are Under construction***
452 |
453 | ### 1i. Understand Complexity of Data Structure Methods
454 |
455 | For now go to this website: [Big-O Cheatsheet](http://bigocheatsheet.com/) and take a look through it. For sorting and data structures, it is important to take note of the *average* time complexity. The *average* case will be the commonly used if you are apply these sorting methods and data structures in a larger algorithm.
456 |
457 | Why suddenly focus *average* if we have been focused on Big-O which is looking at the worse case senario? Its because when working on an algorithm that uses sorting/data structures methods, the worse case input for the entire algorithm typically does not coincide with the worse case input for the sorting/data structure methods.
458 |
459 | We will expand more on this in the future, but for now when we evaluate algorithms, if using sorts and data structure methods as part of your solution, it is more common to use *average* case when evaluating for worse case. Otherwise everytime we lookup a key in a hashtable, it would be O(N) which only happens in ultra-rare senarios.
460 |
461 | ### 1h. Recognize Common Orders of Magnitudes
462 |
463 | Below are some common examples of algorithms and patterns that lead to different magnitudes of time complexity. When problems have an associated constraint, use the constraints as clues to what the algorithm may end up being.
464 |
465 | *Disclaimer: these examples highly depend on the actual input that is scaling.*
466 |
467 | | Input | Common Examples |
468 | |---|---|
469 | |Constant: O(1)| Array access, arithmetic operators|
470 | |Logarithmic: O(log(N))| Binary search, binary search tree retrieval/insertion (balanced)|
471 | |Linear: O(N)| Looping through an array or hashtable|
472 | |Quasilinear: O(Nlog(N))| Quicksort, mergesort, heapsort|
473 | |Quadratic: O(N^2)| Nested loops |
474 | |Polynomial: O(N^C)| Deeply nested loops|
475 | |Exponential: O(C^N)| Multiple-recursion |
476 | |Factorial: O(!N)| Permutations |
477 |
478 |
479 | ### 1j. Leveling Up on Complexity Analysis
480 |
481 | Learn more by doing. Work to analyze problems and evaluate the code you have written out. Go through these two tutorials to get more practice on some problems. Furthermore, from now on, every algorithm you write should be analyzed for complexity.
482 |
483 | 1. [Asymptotic Notation](https://www.khanacademy.org/computing/computer-science/algorithms/asymptotic-notation/a/asymptotic-notation) by Khan Academy
484 |
485 | 2. [Intro to Big-O Notation](https://www.rithmschool.com/courses/javascript-computer-science-fundamentals/introduction-to-big-o-notation) by Rithm School
486 |
487 | 3. [How to Calculate Big-O](https://justin.abrah.ms/computer-science/how-to-calculate-big-o.html) by Justin Abrah
488 |
489 |
490 | ## 2. Learn Basic Algorithm Patterns
491 |
492 | The basic list of algorithm patterns will help you progress throughout the first series of algorithms. The basic algorithms will require understanding how to use arrays and hashtables, nest loops, and a strong logic controls (```if```, ```if-else```, ```and```, ```or```).
493 |
494 |
495 | * **Linear Search:** The most basic search there is, this is a starting point to understand searching.
496 | * [Linear Search - GeeksforGeeks](https://www.geeksforgeeks.org/linear-search/)
497 |
498 | * **Multiple Pointers:** Looping and using multiple pointers that start at different positions and sometimes travel at different speeds are important to ensure you can strengthen your looping game.
499 | * [Outco - Reading on Multiple Pointers](https://github.com/OutcoSF/prep_material/blob/master/readings/multiple_pointers.md)
500 |
501 | * **Frequency Count:** Learn to use hashtables to shave down time complexity on a lot of problems. One class of problems involve knowing/tracking counts.
502 | * [Outco - Reading on Frequency Count](https://github.com/OutcoSF/prep_material/blob/master/readings/frequency_count.md)
503 |
504 | * **Decrease and Conquer:** Its cousin to the more popular Divide and Conquer algorithm, these class of problems are important because they can be solved easily without recursion. It uses a single subproblem which can be solved with a while loop. As we begin on recursion later, converting these decrease and conquer problems to single recursion problems will help strengthen your understanding of recursion.
505 | * [Outco - Reading on Decrease & Conquer](https://github.com/OutcoSF/prep_material/blob/master/readings/decrease_and_conquer.md)
506 |
507 | * **Basic Sorting:** Insertion, Selection, and Bubble sort are basic sorts tha have O(N^2 ) time complexity. Understanding that there are often many ways to solve the same problem, and looking at the advantages and disadvantages of them allows us to start making better algorithm design decisions.
508 | * [Youtube - Visualize Bubble and Insertion Sort](https://www.youtube.com/watch?v=WaNLJf8xzC4&t=114s)
509 | * [Khan Academy - Selection, Insertion](https://www.khanacademy.org/computing/computer-science/algorithms#sorting-algorithms)
510 | * [Visualgo - Visualize Sorting Algorithms](https://visualgo.net/sorting)
511 |
512 | * **Recursion** Finally, getting to recursion which can be a tough hump to get over for thosejust starting out. We recommend using the Helper Method Recursion pattern which uses a helper method to actually do the recursion. One way to start is to replace simple loops with recursion. We will be posting simpler problems for you to start off soon.
513 | * [Khan Academy - Recursive Algorithms](https://www.khanacademy.org/computing/computer-science/algorithms/recursive-algorithms/p/challenge-recursive-powers)
514 | * [Eloquent JavaScript - Recursion Section](http://eloquentjavascript.net/03_functions.html)
515 | * [Rithm School - Intro to Recursion](https://www.rithmschool.com/courses/javascript-computer-science-fundamentals/introduction-to-recursion)
516 |
517 | ***Under Construction***
518 |
519 | ## 3. Learn Linear Data Structures
520 |
521 | * **Arrays and Dynamic Arrays:** Start here to learn more about arrays and dynamic arrays. If you began your development journey learning Python, Ruby, or JavaScript, you may not have been exposed to primitive array data structures (continuous blocks of memory) because your language has abstracted the concept of arrays into a more powerful, more functional data structures. Lets jump back to the old times and find out what arrays really are under the hood.
522 | * [Fixed and Dynamic Arrays - Rebus Community](https://press.rebus.community/programmingfundamentals/chapter/fixed-and-dynamic-arrays/)
523 |
524 | * **Linked Lists, Stacks, and Queues** These linear data structures are often neglected as data structures in Python, Ruby, and JavaScript because Lists and Arrays in these languages has combined their functionality. It is still important to understand how they are constructed, and how to nagivate through these more limited functionality data structures. Often we are given constraints, learning to work with these constraints make us more aware ways to tackle a problem. That starts with really understanding these linear data structues.
525 | * [MyCodeSchool - LinkedList](https://www.youtube.com/watch?v=NobHlGUjV3g&t)
526 | * [Visualgo - LinkedList, Stacks, Queues](https://visualgo.net/list)
527 | * [Gayle McDowell - Stacks and Queues](https://www.youtube.com/watch?v=wjI1WNcIntg)
528 |
529 | More problem sets to come for these problems.
530 |
531 | *** Under Construction ***
532 |
533 | ## 4. Develop a Process for Problem Solving
534 |
535 | Read the following 4 points and ask yourself if you ascribe to some or all of these challenges:
536 |
537 | * I often find it difficult to understand what the problem is even asking.
538 | * I can understand the problem, but am just not sure where to begin or I just blank out.
539 | * I can jump into coding quickly, but when I find errors in my logic my algorithm seems to break down and I become stuck.
540 | * I can come up with an algorithm in my head that I think might work, but I cannot seem to write it out correctly.
541 |
542 | These are challenges that having a more structured process for problem solving would help. What you want to establish is a framework for solving problems. This framework will provide you with a list of steps and line items to check off to ensure:
543 |
544 | 1. you properly understand the problem
545 | 2. you are able to vocalize and visualize your code and talk over solutions at a high level
546 | 3. you can analyze your solution to determine the validity of your algorithm
547 | 4. you are able to translate your algorithm into code you can verify
548 |
549 | Here is an example framework which you can adopt and modify as needed to fit your style of problem solving:
550 |
551 | 1. **Understanding the problem**
552 | * Ask clarifying questions
553 | * Get the inputs, outputs, constraints (time/space)
554 | * Get example case, edge conditions
555 | * Clarify what functions/methods you have to work with
556 | 2. **Exploration of Solutions**
557 | * Communicate aloud (or with interviewer)
558 | * Diagramming (pencil/paper or on the whiteboard)
559 | * Explain the naive solution
560 | * Brainstorm
561 | 3. **Solution Analysis**
562 | * Complexity analysis to verify solutions
563 | * Verifying solution edge conditions
564 | 4. **Coding**
565 | * Pseudocoding
566 | * Coding out solution from pseudocode
567 | * Run example cases
568 | * Checking edge conditions
569 |
570 | ### Roadmap to develop a problem solving process
571 |
572 | 1. Adopt a framework and print out the steps for reference.
573 | 2. Select easier problems to practice your framework (two each day minimum). Focus on the process rather than the algorithm.
574 | 3. Follow the steps of your framework. Focus on applying each step before moving to the next.
575 | 4. Practice different formats: whiteboard, syntax highlighted text editor, and plain text editor.
576 | 5. Practice your framework daily until you can execute without the printed reference.
577 |
578 |
579 | ## 5. Put in Practice
580 |
581 | Practice easy level algorithms to get into the groove of things. Remember the steps from the previous section. Grab a notepad and pencil and prepare to diagram things out if necessary. Collectively, you should work on at least 15 problems from these resources:
582 |
583 | * [Leetcode](https://leetcode.com/problemset/algorithms/) - Do Easy Problems with Over 55% Acceptance Rate
584 | * [Codewars](https://www.codewars.com/) - Kata 7-8 problems
585 | * [Coderbyte](https://coderbyte.com/challenges#easyChals) - 10 Easy problems
586 |
587 | *After completing these problems, you should have a good shot at passing Outco's fundamentals check. If you are looking to apply/re-apply please reach out to the [Outco Outreach team](https://outco.io).*
588 |
589 | ## 6. Progress to Intermediate Concepts
590 |
591 | If you want to progress more, then these concepts are what you should tackle next. They are common enough on interviews that having a solid understanding of these concepts is critical to progressing past technical screens involving algorithms.
592 |
593 | ### Algorithm Patterns
594 |
595 | * Pure Recursion
596 | * ***under construction***
597 | * Divide and Conquer, Quicksort, Mergesort
598 | * [Chand John - What's the fastest way to alphabetize your bookshelf?](https://www.youtube.com/watch?v=WaNLJf8xzC4&t=114s)
599 | * [Khan Academy - Mergesort](https://www.khanacademy.org/computing/computer-science/algorithms/merge-sort/a/divide-and-conquer-algorithms)
600 | * [Khan Academy - Quicksort](https://www.khanacademy.org/computing/computer-science/algorithms/quick-sort/a/overview-of-quicksort)
601 | * [Visualgo - Quicksort, MergeSort](https://visualgo.net/sorting)
602 | * Tree and Graph Traversals
603 | * [Geeks for Geeks - Tree Traversals](http://www.geeksforgeeks.org/tree-traversals-inorder-preorder-and-postorder/)
604 | * [Khan Academy - Breadth First Search](https://www.khanacademy.org/computing/computer-science/algorithms/breadth-first-search/a/breadth-first-search-and-its-uses)
605 | * [Geeks for Geeks - Depth First Search of Graphs](http://www.geeksforgeeks.org/depth-first-traversal-for-a-graph/)
606 |
607 | ### Data Structures
608 |
609 | * Binary Search Trees
610 | * [Hacker Earth - Binary Search Trees](https://www.hackerearth.com/practice/data-structures/trees/binary-search-tree/tutorial/)
611 | * [MyCodeSchool - Binary Search Trees](https://www.youtube.com/watch?v=pYT9F8_LFTM)
612 | * Graphs
613 | * [Khan Academy - Graphs](https://www.khanacademy.org/computing/computer-science/algorithms#graph-representation)
614 |
615 | ### Practice
616 | Do at the minimum 20 problems of the level of difficulty listed here:
617 |
618 | * [Leetcode - Do Problems 45-55% Acceptance Rate](https://leetcode.com/problemset/algorithms/)
619 | * [Codewars Kata 6-7 problems](https://www.codewars.com/)
620 |
621 |
622 | ## 7. Track and Measure
623 |
624 | If you are interviewing, you need to evaluate your ability to solve algorithms under a set amount of time and possibly with someone on the other end. Get yourself into to pressure setting that interviews by doing mock interviews.
625 |
626 | Here are a few resources that you can use to test yourself:
627 |
628 | * [Leetcode Mock Interview](https://leetcode.com/mockinterview/)
629 | * [Pramp - practice with others live](https://pramp.com/)
630 |
631 | ***Under Construction***
632 |
633 | ## 8. Final Notes
634 | Algorithms is not something you learn overnight, it takes practice and consistency. I hope you found this as a useful starting point. I will be expanding on this resource in the weeks to come. Please reach out to me if you have any questions.
635 |
636 | **If you found this guide helpful, please feel free to star and share with anyone that would get value from this guide.**
637 |
638 |
639 | ## Solutions
640 | ```
641 | SOLUTIONS TO PROBLEM SET 1:
642 | Reduce the following to it Big-O magnitude:
643 |
644 | 1) 5 + n // O(N)
645 | 2) n + n^2 // O(n^2)
646 | 3) 15n + 13n // O(n)
647 | 4) 10000 // O(1)
648 | 5) log(n) + 1 // O(log(n))
649 | 6) log(n) * 3 + 14n + 3 // O(n)
650 | 7) nlog(n) + 3n^2 // O(n^2)
651 | 8) n^3 + log(n^4) // O(n^3)
652 | 9) n! + 180000n^2 // O(n!)
653 | 10) 300n^5 + 15002^n + 3n // O(15002^n)
654 | ```
655 |
--------------------------------------------------------------------------------
/readings/decrease_and_conquer.md:
--------------------------------------------------------------------------------
1 | You may have heard of a foundational algorithm technique called [Divide and Conquer](https://en.wikipedia.org/wiki/Divide_and_conquer_algorithm). In this section, we will focus on its lesser known cousin Decrease and Conquer.
2 |
3 | Lets clarify the difference now:
4 |
5 | **Divide and Conquer**:
6 |
7 | * divides a problem into **two or more** subproblems.
8 | * the subproblems are solved independently
9 | * (optionally) the results are combined back together.
10 | * implemented with multiple recursion or a `stack` and `while` loop
11 | * e.g., Quicksort, Mergesort, [Strassen matrix multiplication](https://en.wikipedia.org/wiki/Strassen_algorithm)
12 |
13 | **Decrease and Conquer**:
14 |
15 | * reduces a problem to a **single** smaller subproblem.
16 | * the reduction can be by a constant value or a constant factor.
17 | * implemented with a `while` loop (stack often is not necessary with a single subproblem) or single recursion
18 | * e.g., Binary Search, Quickselect, [Russian Peasant Multiplication](http://www.cut-the-knot.org/Curriculum/Algebra/PeasantMultiplication.shtml)
19 |
20 | We will focus on **decrease and conquer** for now and introduce divide and conquer in a later section. Lets cover few algorithms using decrease and conquer to reduce the problem by a constant or variable factor.
21 |
22 | Note: Decrease and conquer includes reduction by a constant value as well (subtract by a value), however we will focus on reduction by a factor (division by a factor).
23 |
24 | ### Example 1: Binary Search
25 |
26 | Given a sorted array of unique integers, and a target value determine the index of a matching value within the array. If there is not match, return `-1`.
27 |
28 | Input: `[1,3,4,5,6,7,8,10,11,13,15,17,20,22]`, `17`
29 |
30 | Output: `11`
31 |
32 | Binary search is a foundational search algorithm used to find the index of a value within a sorted array. It converges at a solution in logarithmic time O(log(N)) which is quicker than performing a linear search. Binary search uses a process of elimination to discard of the potential locations where a value could be.
33 |
34 | #### Dictionary Analogy
35 |
36 | 
37 | Think about how one searches for the definition of a word within an English dictionary. We can begin a picking a page in the middle of the book. Then check whether the word is on the page. If it isn’t there, the word has to be either before or after this page. We deduce from alphabetical order which half to continue the search.
38 |
39 | Next we split the appropriate section in half again and look for the word in the new page. We repeat until either the current page has the target word, or we find that the word does not exist in the dictionary.
40 |
41 | #### Pseudocode
42 |
43 | 1\. Start with the full range of the array (0 to array length - 1).
44 | 2\. Check the value at the middle of that range.
45 | 2\. If mid matches target we return the mid index.
46 | 3\. If the mid is larger than target we can eliminate the right half.
47 | 4\. If the mid is less than target, we can eliminate the left half.
48 | 5\. Adjust the range depending on which half if still active.
49 | 6\. Repeat steps 2-5 until a match is found or the range is empty
50 | 7\. If the range is empty, return -1.
51 |
52 |
53 |
54 | #### Hint 1
55 |
56 | Consider using a while loop (instead of recursion) to simplify the logic. What step in the pseudo code above would be the while condition?
57 |
58 | #### Hint 2
59 |
60 | When reducing the active range, remember to also eliminate the mid index each time as well.
61 |
62 | #### Solution
63 |
64 | function binarySearch(arr, target) {
65 | let start = 0;
66 | let end = arr.length - 1;
67 | let mid;
68 | while(start <= end) {
69 | mid = Math.floor((start + end) / 2);
70 | if(arr[mid] === target) { return mid; }
71 | if(target < arr[mid]) {
72 | end = mid - 1;
73 | } else {
74 | start = mid + 1;
75 | }
76 | }
77 | return -1;
78 | }
79 |
80 |
81 |
82 | ### Example 2: Greatest Common Divisor (GCD)
83 |
84 | Given two integers, find the greatest common divisor (GCD).
85 |
86 | Input: `52, 78`
87 |
88 | Output: `26`
89 |
90 | In mathematics, the GCD of two integers is the largest positive integer that is a factor of both integers. In the case both `52` and `78` are divisible by `26`. Which also happens to be the largest common factor as well.
91 |
92 | #### Prime Factorization
93 |
94 | One potential approach is to find all the prime factors of each number. In the example case above, we get:
95 |
96 | `52` has prime factors `[2, 2, 13]`
97 | `78` has prime factors `[2, 3, 13]`
98 |
99 | Multiply all the prime factors common to both integers. In this case, `2` and `13` are common therefore `2 x 13 = 26` is the GCD.
100 |
101 | Finding all the prime factors for both input integers and then computing the product of all the common factors can take more time than necessary. Lets explore a faster approach.
102 |
103 | #### Euclid’s Algorithm (Decrease and Conquer)
104 |
105 | [Euclids’s algorithm](https://en.wikipedia.org/wiki/Greatest_common_divisor#Using_Euclid.27s_algorithm) uses decrease an conquer to converge at the GCD faster than the prime factorization approach. The basis for Euclid’s algorithm is that the GCD of two numbers must be a factor of its _difference_ as well.
106 |
107 | In the example above: `78 - 52 = 26`, the GCD must be a factor of `26` as well. Another way to put it is the `GCD(78, 52)` must also equal to the `GCD(52, 26)` and/or the `GCD(78, 26)`.
108 |
109 | Solving the smaller subproblem `GCD(52, 26)` we can apply the difference again `52 - 26 = 26` and reduce the problem further to `GCD(26,26)`. The GCD of two equal numbers that the number itself so we get `GCD(26,26) = 26` (since the GCD must be positive, if the equal pair is negative, take the absolute value).
110 |
111 | So instead of finding all the divisors or prime factors of two really large values, lets find the _difference_ between them to reduce the problem to smaller inputs. If we keep finding the difference the problem reduces to the point where the inputs are much smaller.
112 |
113 | #### Use Modulo instead of Subtract
114 |
115 | Finding the difference is fine, but finding the modulo which gives the remainder is even faster. This is because remainder is the result performing multiple subtractions in one shot. Lets see how to approach the algorithm.
116 |
117 | #### Pseudocode
118 |
119 | Given integers A and B, where A > B, return its GCD.
120 |
121 | 1\. Save B as a temp variable.
122 | 2\. Set B equal to A % B.
123 | 3\. Set A equal to the temp.
124 | 4\. Repeat steps 1-3 until B equals 0.
125 | 5\. Return A.
126 |
127 |
128 |
129 | #### Resources
130 |
131 | To get a deeper understanding, check out a proof on Euclid’s Algorithm [here](https://www.khanacademy.org/computing/computer-science/cryptography/modarithmetic/a/the-euclidean-algorithm) by Khan Academy.
132 |
133 | #### Hint 1
134 |
135 | Try using a while loop. Which step would be the while condition?
136 |
137 | #### Hint 2
138 |
139 | To make the pseudocode work for the general case where the first integer can be smaller _or_ larger than the second integer by swapping them if A < B. What if one or more of the values are negative?
140 |
141 | #### Solution
142 |
143 | function gcd(a, b){
144 |
145 | if(a < 0) { a *= -1; }
146 | if(b < 0) { b *= -1; }
147 | let temp;
148 |
149 | if(a < b) {
150 | temp = a;
151 | a = b;
152 | b = temp;
153 | }
154 |
155 | while(b > 0) {
156 | temp = b;
157 | b = a % b;
158 | a = temp;
159 | }
160 |
161 | return a;
162 | }
163 |
164 |
165 |
166 | #### Refactoring
167 |
168 | In some languages (Python, Ruby, Javascript ES6) swapping can be done a bit more clean, lets refactor using an ES6 feature of JavaScript called [destructuring assignment](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Destructuring_assignment).
169 |
170 |
171 | function gcd(a, b){
172 |
173 | if(a < 0) { a *= -1; }
174 | if(b < 0) { b *= -1; }
175 |
176 | if(a < b) {
177 | [a, b] = [b, a];
178 | }
179 |
180 | while(b > 0) {
181 |
182 | [a, b] = [b, a % b];
183 | }
184 |
185 | return a;
186 | }
187 |
188 |
189 |
190 | #### More Refactoring
191 |
192 | Notice if we take the modulo A % B where A < B, we actually just get A back. For example `5 % 10 = 5`. This is because `10` goes into `5` zero times and we get a remainder of `5`.
193 |
194 | A further optimization is that we can just remove the initial swap out completely. Because the logic inside the `while` loop takes care of the swap for us if A < B initially. Lets remove all the comments out for clarity.
195 |
196 | #### Final Solution
197 |
198 | function gcd(a, b){
199 | if(a < 0) { a *= -1; }
200 | if(b < 0) { b *= -1; }
201 | while(b > 0) { [a, b] = [b, a % b]; }
202 | return a;
203 | }
204 |
205 |
206 |
207 | ### Challenge 1 : Number of Ones in a Sorted Bit Array
208 |
209 | Given a sorted bit array (values of either 0 or 1), determine the number of 1’s in the array.
210 |
211 | Perform this in `O(log(N))` time complexity.
212 |
213 | Input: `[0,0,0,1,1,1,1,1,1,1,1]`
214 |
215 | Output: `8`
216 |
217 | #### Hint 1
218 |
219 | If we know the index of the first 1 in the array we can compute the the number of 1’s using the length of the array.
220 |
221 | #### Hint 2
222 |
223 | Try using a binary search. But what target value would we even be searching for?
224 |
--------------------------------------------------------------------------------
/readings/frequency_count.md:
--------------------------------------------------------------------------------
1 | Frequency counting is a technique that can be used to solve problems what requires tracking, counting, or looking up values quickly. These type of problems often involve a collection of some sort (i.e., array, hashtable, or string). Often the problem will involve matching, counting, or confirming values in the collection.
2 |
3 | #### Implementation
4 |
5 | To implement a frequency count, we typically uses a hashtable, However for specific cases, we may opt to use a set, or an array.
6 |
7 | **Hashtable**: General all purpose use
8 |
9 | **Array**: Values in the collection that are of a small range of integer values.
10 |
11 | **Set**: If only needed to track if something exists.
12 |
13 | To populate our count we will have to loop through our input collection. This leads to a O(N) time and potentially O(N) auxiliary space (if all values are unique). However future lookups can be performed in O(1) time as a result.
14 |
15 | Lets look at examples to see its implementation.
16 |
17 | ### Example 1: Two Sum
18 |
19 | Given an array of integers, and a target value determine if there are two integers that add to the sum.
20 |
21 | Input: `[4,2,6,5,7,9,10]`, `13`
22 |
23 | Output: `true`
24 |
25 | #### Brute Force
26 |
27 | This problem looks quite similar to the [_sorted two sum_](http://class.outco.io/courses/technical/lectures/1950839) problem, but the input array is not sorted. If we use a brute force approach we could try every single unique pair in the array. This would solve the problem in O(N^2) time.
28 |
29 | Try to solve this problem without using hints first. If you get stuck then use the minimum hints to get yourself unstuck. Goodluck!
30 |
31 | #### Hint 1
32 |
33 | Try using a hash table (or a set) to store values we have come across to speed up lookup time.
34 |
35 | #### Hint 2
36 |
37 | 1. As we look through each value in the array, what do we need to check in the hash to know whether there is a sum that matches the target?
38 | 2. If the hash does not contain a matching value, then what should we add to the hash table?
39 | 3. If the hash _does_ contains a matching value, what should we do?
40 | 4. If we finish the loop but have not found a match, what should we return?
41 |
42 | #### Solution
43 |
44 | function twoSum(numbers, target){
45 | let hash = {};
46 | let current;
47 | for(let i = 0; i < numbers.length; i++) {
48 | current = numbers[i];
49 | if(hash[current]) { return true; }
50 | hash[target - current] = true;
51 | }
52 | return false;
53 | }
54 |
55 |
56 |
57 | ### Challenge 1: Sort a Bit Array
58 |
59 | Given a bit array, return it sorted in-place (a bit array is simply an array that contains only bits, either 0 or 1).
60 |
61 | See if you can solve this in O(N) time and O(1) auxiliary space.
62 |
63 | Try to solve this using a frequency count rather than using multiple pointers, or using a comparison sort function.
64 |
65 | Input : `[0, 1, 1, 0, 1, 1, 1, 0]`
66 |
67 | Output : `[0, 0, 0, 1, 1, 1, 1, 1]`
68 |
69 | ### Hint 1:
70 |
71 | Since there are only two values we could use a two item `array` to keep a count of zeros and ones.
72 |
73 | ### Hint 2:
74 |
75 | After creating and populating a frequency count, how do we use the number of zeros and number of ones to populate the original input `array`.
76 |
--------------------------------------------------------------------------------
/readings/multiple_pointers.md:
--------------------------------------------------------------------------------
1 | A number of problems involving arrays, lists, and strings require us to iterate with more than one pointer. Sometimes pointers may travel at different speeds and may start from a different initial position.
2 |
3 | For example, an algorithm that requires one to swap a bunch of values may require multiple pointers.
4 |
5 | ### Language Specific
6 |
7 | * Languages like Java, C, C++, and Javascript, have `for` loops that are more flexible and can be easily customized to handle two or more pointers. But universally, a `while` loop will give the same flexibility for all languages.
8 | * Languages like Ruby or Python can use `while` loop to handle looping with multiple pointers.
9 | * When there are multiple pointers to keep track of, avoid using built in iterators that simply loop from left to right.
10 | * Lets demonstrate the use of while loops with a few examples.
11 |
12 | ### Example 1 : Sort a Bit Array
13 |
14 | Given a bit array, return it sorted in-place (a bit array is simply an array that contains only bits, either a 1 or a 0).
15 |
16 | See if you can solve this in O(N) time and O(1) auxiliary space.
17 |
18 | #### Hint 1
19 |
20 | Since we want to sort it in-place we should be modifying the values change changing its position rather than creating a new array. Additionally because there are only two possible values, and we know that the 0’s will have to be on the left, and the ones have to be on the right. How can we identify elements to swap?
21 |
22 | #### Hint 2
23 |
24 | 1. If we have two pointers: one that starts on the very **left**, and one on the very **right**, then we can iterate inward.
25 | 2. We need to iterate the **left** pointer until it hits a 1
26 | 3. Then decrement the **right** pointer to the left until it reaches a 0
27 | 4. Once we find them we will do a swap.
28 |
29 | #### Solution
30 |
31 |
32 |
33 |
function bitArraySort(arr) {
34 | let left = 0;
35 | let right = arr.length - 1;
36 | while(left < right) {
37 | while(arr[left] === 0) { left++; }
38 | while(arr[right] === 1) { right--; }
39 | if(left < right) {
40 | // swap the left and right values
41 | [arr[left], arr[right]] = [arr[right], arr[left]];
42 | }
43 | }
44 | return arr;
45 | }
46 |
47 |
48 |