├── README.md ├── count_path ├── bottomup.py ├── recursion.py └── topdown.py ├── fibonacci ├── bottomup.py ├── recursion.py └── topdown.py ├── increasing_sequence ├── bottomup.py └── recursion.py └── partition_string ├── bottomup.py ├── recursion.py └── topdown.py /README.md: -------------------------------------------------------------------------------- 1 | ## A Collection of Dynamic Programming Problems 2 | This is a collection of interesting algorithm problems written first recursively, then using memoization and finally a bottom-up approach.This allows to well capture the logic of dynamic programming. 3 | 4 | ### Dynamic Programming 5 | 6 | The idea behind dynamic programming as describe in The Algorithm Design Manual (S. Skiena): 7 | - Find the recursion in the problem 8 | - Build a table of possible values 9 | - Find the right order to evaluate the results so that partial results are available when needed. 10 | 11 | There are two main approaches in the implementation of dynamic programming : 12 | 13 | #### Top Down - Memoization 14 | When the recursion does a lot of unecessary calculation, an easy way to solve this is to cache the results and to check before executing the call if the result is already in the cache. 15 | 16 | #### Bottom-Up 17 | A better way to do this is to get rid of the recursion all-together by evaluating the results in the right order and building the array as we iterate. The partial results are available when needed if the iteration is done in the right order. 18 | 19 | We have to identify and initialize the boundary conditions such as when we start the iteration, those are available. 20 | 21 | We can most of the time optimize the space and avoid storing all the partial results along the way by storing only the stricly necessary partials. 22 | 23 | 24 | ### Collection 25 | 26 | For each problem, a simple recursion is presented together with a top down and a bottom-up approach. This allows to have a clear view of the logic of dynamic programming. 27 | 28 | The simple recursion is very inefficient obviously and very limited in use, it is there for illustration only. 29 | 30 | #### [Fibonnaci](https://github.com/tristanguigue/dynamic-programming/tree/master/fibonacci) 31 | Finding the nth fibonacci number 32 | 33 | Input: `4` 34 | 35 | Output: `3` 36 | 37 | #### [String Partition](https://github.com/tristanguigue/dynamic-programming/tree/master/partition_string) 38 | The partition string problem retrieving the original string whose spaces were removed. 39 | We are given a list of words belonging to a dictionary 40 | 41 | Input: "carlosthinksthattheweatherisnice" 42 | 43 | Output: "carlos thinks that the weather is nice" 44 | 45 | #### [Counting Paths](https://github.com/tristanguigue/dynamic-programming/tree/master/count_path) 46 | Count the number of possible paths from `(x1, y1)` to `(x2, y2)` by moving right or down 47 | 48 | Input: `(0, 0, 1, 1)` 49 | 50 | Output : `2` 51 | 52 | #### [Largest Increasing Sequence](https://github.com/tristanguigue/dynamic-programming/tree/master/increasing_sequence) 53 | Find the largest non-continuous increasing sequence from an array 54 | 55 | Input: `[1, 6, 7, 4, 6, 1, 3, 4, 6, 19, 12, 14, 35, 66]` 56 | 57 | Output: `[1, 3, 4, 6, 12, 14, 35, 66]` 58 | 59 | Note that for this problem, the memoization solution is not available since the recursion access each partial solution once. 60 | 61 | 62 | 63 | 64 | -------------------------------------------------------------------------------- /count_path/bottomup.py: -------------------------------------------------------------------------------- 1 | # number of possible paths from (x1, y1) to (x2, y2) moving right or down 2 | # Input: (0, 0, 1, 1) 3 | # Output : 2 4 | 5 | cache = {} 6 | 7 | 8 | def count_paths(x1, y1, x2, y2): 9 | 10 | # boundary conditions 11 | # at the destination, there is no more paths to consider 12 | cache[x2, y2] = 0 13 | 14 | # if we are on the border, the only path left is all the way right/down 15 | for i in range(0, x2): 16 | cache[i, y2] = 1 17 | 18 | for j in range(0, y2): 19 | cache[x2, j] = 1 20 | 21 | # move left / up and use the partial solutions at i + 1, j and i, j + 1 22 | for i in range(x2 - 1, -1, -1): 23 | for j in range(y2 - 1, -1, -1): 24 | cache[i, j] = cache[i + 1, j] + cache[i, j + 1] 25 | 26 | # the number of path from point 0, 0 to the destination 27 | return cache[0, 0] 28 | 29 | print count_paths(0, 0, 1, 1) 30 | -------------------------------------------------------------------------------- /count_path/recursion.py: -------------------------------------------------------------------------------- 1 | # number of possible paths from (x1, y1) to (x2, y2) moving right or down 2 | # Input: (0, 0, 1, 1) 3 | # Output : 2 4 | 5 | 6 | def count_paths(x1, y1, x2, y2): 7 | # invalid path 8 | if x1 > x2 or y1 > y2: 9 | return 0 10 | 11 | # we've arrived to destination, add 1 for each leaf 12 | if x1 == x2 and y1 == y2: 13 | return 1 14 | 15 | # move right and down 16 | return count_paths(x1 + 1, y1, x2, y2) + count_paths(x1, y1 + 1, x2, y2) 17 | 18 | print count_paths(0, 0, 1, 1) 19 | -------------------------------------------------------------------------------- /count_path/topdown.py: -------------------------------------------------------------------------------- 1 | # number of possible paths from (x1, y1) to (x2, y2) moving right or down 2 | # Input: (0, 0, 1, 1) 3 | # Output : 2 4 | 5 | cache = {} 6 | 7 | 8 | def count_paths(x1, y1, x2, y2): 9 | # invalid path 10 | if x1 > x2 or y1 > y2: 11 | return 0 12 | 13 | # we've arrived to destination, add 1 for each leaf 14 | if x1 == x2 and y1 == y2: 15 | return 1 16 | 17 | if (x1, y1) in cache: 18 | return cache[x1, y1] 19 | 20 | # move right and down 21 | count = count_paths(x1 + 1, y1, x2, y2) + count_paths(x1, y1 + 1, x2, y2) 22 | 23 | cache[x1, y1] = count 24 | 25 | return count 26 | 27 | 28 | print count_paths(0, 0, 1, 1) 29 | -------------------------------------------------------------------------------- /fibonacci/bottomup.py: -------------------------------------------------------------------------------- 1 | # Find the nth fibonacci number 2 | # 3 | # Input: 4 4 | # Output: 3 5 | 6 | 7 | def fibonacci(n): 8 | 9 | fn_2 = 0 10 | fn_1 = 1 11 | 12 | for i in range(2, n + 1): 13 | fi = fn_1 + fn_2 14 | fn_1, fn_2 = fi, fn_1 15 | 16 | return fi 17 | 18 | print fibonacci(10) 19 | # print 55 20 | -------------------------------------------------------------------------------- /fibonacci/recursion.py: -------------------------------------------------------------------------------- 1 | # Find the nth fibonacci number 2 | # 3 | # Input: 4 4 | # Output: 3 5 | 6 | 7 | def fibonacci(n): 8 | # boundary conditions: 9 | if n == 0: 10 | return 0 11 | elif n == 1: 12 | return 1 13 | 14 | return fibonacci(n - 1) + fibonacci(n - 2) 15 | 16 | 17 | print fibonacci(4) 18 | # print 3 19 | -------------------------------------------------------------------------------- /fibonacci/topdown.py: -------------------------------------------------------------------------------- 1 | # Find the nth fibonacci number 2 | # 3 | # Input: 4 4 | # Output: 3 5 | 6 | cache = {} 7 | 8 | 9 | def fibonacci(n): 10 | 11 | if n in cache: 12 | return cache[n] 13 | 14 | # boundary conditions: 15 | if n == 0: 16 | return 0 17 | elif n == 1: 18 | return 1 19 | 20 | fn = fibonacci(n - 1) + fibonacci(n - 2) 21 | cache[n] = fn 22 | return fn 23 | 24 | 25 | print fibonacci(45) 26 | # print 1134903170 27 | -------------------------------------------------------------------------------- /increasing_sequence/bottomup.py: -------------------------------------------------------------------------------- 1 | # Find the longest increasing sequence from an array 2 | # 3 | # Input: [1, 6, 7, 4, 6, 1, 3, 4, 6, 19, 12, 14, 35, 66] 4 | # Output: [1, 3, 4, 6, 12, 14, 35, 66] 5 | 6 | 7 | def longest_increasing_sequence(input_array): 8 | """Finds the longest increasing sequence for the array 9 | Args: 10 | input_array (array): the given array 11 | Returns: 12 | The longest increasing sequence of this array 13 | """ 14 | if not input_array: 15 | return [] 16 | 17 | # Initialize the max_sequence with first element 18 | max_seq = [input_array[0]] 19 | sequences = [max_seq] 20 | 21 | for i in range(1, len(input_array)): 22 | # Loop through all the previous sequences and add current element if 23 | # possible 24 | for i in range(len(sequences) - 1, -1, -1): 25 | seq = sequences[i] 26 | 27 | if seq[- 1] < input_array[i]: 28 | sequences.append(seq + [input_array[i]]) 29 | 30 | # Update the longest sequence found so far 31 | if len(sequences[-1]) > len(max_seq): 32 | max_seq = sequences[-1] 33 | break 34 | else: 35 | # Add a new sequence starting at i if we didn't append the element 36 | sequences.append([input_array[i]]) 37 | 38 | return max_seq 39 | 40 | 41 | print longest_increasing_sequence( 42 | [1, 6, 7, 4, 6, 1, 3, 4, 6, 19, 12, 14, 35, 66]) 43 | # [1, 3, 4, 6, 12, 14, 35, 66] 44 | -------------------------------------------------------------------------------- /increasing_sequence/recursion.py: -------------------------------------------------------------------------------- 1 | # Find the longest increasing sequence from an array 2 | # 3 | # Input: [1, 6, 7, 4, 6, 1, 3, 4, 6, 19, 12, 14, 35, 66] 4 | # Output: [1, 3, 4, 6, 12, 14, 35, 66] 5 | 6 | 7 | def longest_increasing_sequence(input_array, sequences, i): 8 | """Finds the longest increasing sequence ending at `i` 9 | Args: 10 | sequences (array): the list of sequences to be appended 11 | input_array (array): the initial sequence 12 | i (int): the position of the cursor 13 | Returns: 14 | The longest increasing sequence ending at `i` 15 | """ 16 | 17 | # Once we reach the bottom of the array, append the first element 18 | if i == 0: 19 | sequences.append([input_array[0]]) 20 | return sequences[0] 21 | 22 | # Recurse to find the longest increasing sequence ending at `i - 1` 23 | max_seq = longest_increasing_sequence(input_array, sequences, i - 1) 24 | 25 | # Loop through all the previous sequences and add current element if 26 | # possible 27 | for seq in sequences: 28 | 29 | if seq[- 1] < input_array[i]: 30 | sequences.append(seq + [input_array[i]]) 31 | 32 | # Update the longest sequence found so far 33 | if len(sequences[-1]) > len(max_seq): 34 | max_seq = sequences[-1] 35 | 36 | sequences.append([input_array[i]]) 37 | 38 | return max_seq 39 | 40 | 41 | a = [1, 6, 7, 4, 6, 3, 4, 6, 19, 12, 14, 35, 66] 42 | 43 | print longest_increasing_sequence(a, [], len(a) - 1) 44 | # [1, 3, 4, 6, 12, 14, 35, 66] 45 | -------------------------------------------------------------------------------- /partition_string/bottomup.py: -------------------------------------------------------------------------------- 1 | # The partition string problem retrieving the original string whose spaces 2 | # were removed. 3 | # We are given a list of words belonging to a dictionary 4 | # 5 | # Input: carlosthinksthattheweatherisnice 6 | # Output: carlos thinks that the weather is nice 7 | 8 | # The dictionary of available words 9 | dictionary = { 10 | "thinks": True, 11 | "that": True, 12 | "the": True, 13 | "weather": True, 14 | "is": True, 15 | "nice": True 16 | } 17 | 18 | 19 | def cost(word): 20 | """Evaluate the cost of a given word. 0 if the word is in the dictionary, 21 | the number of characters otherwise 22 | 23 | Args: 24 | word (string): a string whose cost need to be evaluated 25 | 26 | Returns: 27 | The cost of the word (int) 28 | """ 29 | if word in dictionary: 30 | return 0 31 | else: 32 | return len(word) 33 | 34 | 35 | def split(s): 36 | # this is the array of partial solutions 37 | m = {} 38 | 39 | # we initialize our boundary condition 40 | m[len(s)] = ('', 0) 41 | 42 | # we build our partial solution backward 43 | for i in range(len(s) - 1, -1, -1): 44 | 45 | min_cost = None 46 | min_string = None 47 | 48 | for k in range(i, len(s)): 49 | substring = s[i:k + 1] 50 | # we use the already calculated cost of the substring starting at 51 | # k + 1 52 | current_cost = cost(substring) + m[k + 1][1] 53 | 54 | if min_cost is None or current_cost < min_cost: 55 | # if the two parts are not empty join them with space 56 | if substring and m[k + 1][0]: 57 | min_string = substring + ' ' + m[k + 1][0] 58 | min_cost = current_cost + 1 59 | else: 60 | min_string = substring + m[k + 1][0] 61 | min_cost = current_cost 62 | 63 | m[i] = min_string, min_cost 64 | 65 | return m[0] 66 | 67 | 68 | print split("carlosthinksthattheweatherisnice") 69 | # print ('carlos thinks that the weather is nice', 12) 70 | -------------------------------------------------------------------------------- /partition_string/recursion.py: -------------------------------------------------------------------------------- 1 | # The partition string problem retrieving the original string whose spaces 2 | # were removed. 3 | # We are given a list of words belonging to a dictionary 4 | # 5 | # Input: carlosthinksthattheweatherisnice 6 | # Output: carlos thinks that the weather is nice 7 | 8 | # The dictionary of available words 9 | dictionary = { 10 | "thinks": True, 11 | "that": True, 12 | "the": True, 13 | "weather": True, 14 | "is": True, 15 | "nice": True 16 | } 17 | 18 | 19 | def cost(word): 20 | """Evaluate the cost of a given word. 0 if the word is in the dictionary, 21 | the number of characters otherwise 22 | 23 | Args: 24 | word (string): a string whose cost need to be evaluated 25 | 26 | Returns: 27 | The cost of the word (int) 28 | """ 29 | if word in dictionary: 30 | return 0 31 | else: 32 | return len(word) 33 | 34 | 35 | def split(input_string, start): 36 | """The recrusive function, it tries to split the substring starting at 37 | `start` into `number_of_partitions` words 38 | 39 | Args: 40 | input_string (string): the initial string that needs to be split 41 | start (int): we want to split the substring of input_string 42 | starting at that index 43 | 44 | Returns: 45 | A tupple form of the partial solution and its cost 46 | """ 47 | 48 | # the substring to split 49 | substring = input_string[start:] 50 | 51 | # This is the boundary conditions 52 | # if the substring is empty return an empty string with no cost 53 | if not len(substring): 54 | return '', 0 55 | 56 | min_cost = None 57 | min_string = None 58 | 59 | # we place our next partition somewhere between start + 1 and the end of 60 | # the input_string 61 | for i in range(1, len(substring) + 1): 62 | # we split the rest of the string recursively 63 | rest_string, rest_cost = split(input_string, start + i) 64 | 65 | current_string = substring[:i] 66 | current_cost = cost(current_string) + rest_cost 67 | 68 | # update minum cost and string if it's the best so far 69 | if min_cost is None or current_cost < min_cost: 70 | 71 | # if the two parts are not empty join them with space 72 | if current_string and rest_string: 73 | min_string = current_string + ' ' + rest_string 74 | # We add a cost for white space to avoid spliting unkown words 75 | # into small pieces 76 | current_cost += 1 77 | else: 78 | min_string = current_string + rest_string 79 | 80 | min_cost = current_cost 81 | 82 | return min_string, min_cost 83 | 84 | 85 | print split("carlosisnice", 0) 86 | # print ('carlos is nice', 8) 87 | -------------------------------------------------------------------------------- /partition_string/topdown.py: -------------------------------------------------------------------------------- 1 | # The partition string problem retrieving the original string whose spaces 2 | # were removed. 3 | # We are given a list of words belonging to a dictionary 4 | # 5 | # Input: carlosthinksthattheweatherisnice 6 | # Output: carlos thinks that the weather is nice 7 | 8 | # The dictionary of available words 9 | dictionary = { 10 | "thinks": True, 11 | "that": True, 12 | "the": True, 13 | "weather": True, 14 | "is": True, 15 | "nice": True 16 | } 17 | 18 | 19 | def cost(word): 20 | """Evaluate the cost of a given word. 0 if the word is in the dictionary, 21 | the number of characters otherwise 22 | 23 | Args: 24 | word (string): a string whose cost need to be evaluated 25 | 26 | Returns: 27 | The cost of the word (int) 28 | """ 29 | if word in dictionary: 30 | return 0 31 | else: 32 | return len(word) 33 | 34 | # The cache to memorize partial solutions 35 | cache = {} 36 | 37 | 38 | def split(input_string, start): 39 | """The recrusive function, it tries to split the substring starting at 40 | `start` into `number_of_partitions` words 41 | 42 | Args: 43 | input_string (string): the initial string that needs to be split 44 | start (int): we want to split the substring of input_string 45 | starting at that index 46 | 47 | Returns: 48 | A tupple form of the partial solution and its cost 49 | """ 50 | # we have already calculated the optimal solution from the point 51 | if start in cache: 52 | return cache[start] 53 | 54 | # the substring to split 55 | substring = input_string[start:] 56 | 57 | # This is the boundary conditions 58 | # if the substring is empty return an empty string with no cost 59 | if not len(substring): 60 | return '', 0 61 | 62 | min_cost = None 63 | min_string = None 64 | 65 | # we place our next partition somewhere between start + 1 and the end of 66 | # the input_string 67 | for i in range(1, len(substring) + 1): 68 | # we split the rest of the string recursively 69 | rest_string, rest_cost = split(input_string, start + i) 70 | 71 | current_string = substring[:i] 72 | current_cost = cost(current_string) + rest_cost 73 | 74 | # update minum cost and string if it's the best so far 75 | if min_cost is None or current_cost < min_cost: 76 | 77 | # if the two parts are not empty join them with space 78 | if current_string and rest_string: 79 | min_string = current_string + ' ' + rest_string 80 | # We add a cost for white space to avoid spliting unkown words 81 | # into small pieces 82 | current_cost += 1 83 | else: 84 | min_string = current_string + rest_string 85 | 86 | min_cost = current_cost 87 | 88 | cache[start] = min_string, min_cost 89 | return min_string, min_cost 90 | 91 | 92 | print split("carlosthinksthattheweatherisnice", 0) 93 | # print ('carlos thinks that the weather is nice', 12) 94 | --------------------------------------------------------------------------------