39 |41 |Find a minimum cardinality subset of edges to disconnect a graph.
40 |
43 |45 |Contract a random edge.
44 |
While there are more than two vertices in the graph, choose a uniformly random edge uv and contract the two endpoints. Output the edges between the remaining two vertices.
46 |Theorem: Success probability of returning a min-cut is at least \frac{2}{n(n-1)}.
47 |48 |52 |Let F \le E be a minimum cut, say |F| = k. The probability that we don’t choose an edge in F in the i-th iteration is \frac{|F|}{|E|}. In the i-th iteration, we have (n - i + 1) vertices. The minimum cut is assumed to be at least k, so the minimum degree is also at least k, so the number of edges is at least \frac{(n - i + 1) k}{2}. So probability for the i-th iteration is upper-bounded by \frac{2}{n - i + 1}
49 |Probability of success in all iteration.
50 |\Pi_{i=0}^{n-3}\left(1 - \frac{2}{n-i}\right) = \frac{n-2}{n} \cdot \frac{n-3}{n-1} \cdot \frac{n-4}{n-2} ... = \frac{2}{n(n-1)}
51 |
Let us repeat T many times, taking the best result. The failure probability is now \left(1 - \frac{2}{n(n-1)}\right)^T \le e^{\frac{-2T}{n(n-1)}}.
53 | 54 | 55 | -------------------------------------------------------------------------------- /cs370/cs370help.md: -------------------------------------------------------------------------------- 1 | CS 370 Help Session 2 | = 3 | 4 | > Question and answer format. 5 | 6 | 1. Page 77, Question 5. 7 | - Did not cover in class. 8 | 2. A2, Question 1. Difference between standard form and dynamics function. 9 | - Same thing. 10 | 3. Page 40, Question 1a. 11 | - Set up $p(x) = a + bx + cx^2 + dx^3$ and the four givens. 12 | - $$\begin{bmatrix} 13 | 1 & x_0 & x_0^2 & x_0^3 \\ 14 | 1 & x_2 & x_2^2 & x_2^3 \\ 15 | 0 & 1 & 2x_1 & 3x_1^2 \\ 16 | 0 & 0 & 2 & 6x_1 \\ 17 | \end{bmatrix}$$ 18 | - For 1b, set rows to 1 and others to 0. 19 | 4. Local Truncation Error. A2. 20 | - Everything on the right-hand side is exactly correct. 21 | - Taylor's and then simplify. 22 | - $$\begin{aligned} 23 | y_{n+1} &= 4y_n - 3y_{n-1} - 2hf(t_{n-1}, y_{n-1}) \\ 24 | &= 4y(t_n) - 3y(t_{n-1}) - 2hy^\prime(t_{n-1}) \\ 25 | &= y(t_n) + y^\prime(t_n)h + y^{\prime\prime}(t_n)\frac{h^2}{2} - y^{\prime\prime\prime}(t_n)\frac{h^3}{2} + ... 26 | \end{aligned}$$ 27 | - So $y_{n+1} - y(t_{n+1}) = O(h^3)$. 28 | - If the question asked for the global error of this method, do local truncation and remember global is one less. But that would be a dirty question. 29 | 30 | > Note: Exams do not try to fool anybody. 31 | 32 | 5. Fair game for midterm. 33 | - Anything from the course notes that was taught, up to and including last friday. 34 | 35 | 6. Page 23, Question 1a. 36 | - $F(2, 5, -1, 10)$. 37 | - Largest number looks like $0.11111 \cdot 2^{10}$ in binary, $992$ in decimal. Brute force factorials to find $6!$ is the largest. 38 | - Turns out $720$ requires 6 bits. So it is not in our floating point number system. This was an oversight, $5!$ or $6!$ would be accepted on an examination. 39 | 7. Page 23, Question 2a. 40 | - Could we be asked to plot elements on the real axis in an exam. 41 | - Sure. Kind of a pain. 42 | - OFL, UFL are overflow, underflow. 43 | - More than one formula in the notes for calculating epsilon. 44 | - Just remember $fl(x) = x(1 + \delta)$, $|\delta| < \epsilon$. $\beta^{1-t}$. 45 | 8. Calculators? 46 | - Wont need them. 47 | 9. Page 24, Question 5c. 48 | - Exactly the assignment question. Errors follow the same recursion. 49 | 10. Page 23, Question 3. 50 | - We are adding $1 + \frac{1}{2} + \frac{1}{3} + ... + \frac{1}{M}$. 51 | - For large $M$, when we go from back-to-front, it does not give the same value as front-to-back. 52 | - Remember that $(a + b) + c \neq a + (b + c)$. 53 | - It should happen that going from back-to-front is better, since we add small to large. We can only keep a fixed number of digits. We are not losing anything. 54 | - *Follow up*: catastrophic cancellation? No, because $n, M$ are both integers. 55 | 11. Page 18, Exercise. 56 | - $0.111, 0.011, 0.001$. 57 | 12. From first or second lecture, doing the integral backwards was better. 58 | - $e_n = \alpha e_{n-1} = \alpha^n e_0$. If $\alpha > 1$, then lets do $e_{n-1} = \frac{1}{\alpha}e_n$. 59 | - Issue because we dont have $I_{n}$ in the first place. 60 | 13. Page 40, Question 4. 61 | - Seems like $S(0) = 0$, but we have to compute the spline. 62 | 14. Not-a-knot boundary conditions. 63 | - Use third derivatives. 64 | 15. Page 41, Question 9. 65 | - We have three cubic polynomials. Not-a-knot says that $S_1(x) = S_2(x) = S_3(x)$. So it is just $P(x) = 2x^3 + 5x - 7$. 66 | 16. Matlab. 67 | - No Matlab questions, but pseudocode is fair. 68 | 17. Notes. 69 | - Must be your own. Can have printed notes from iPad, but you must show professor before. 70 | 18. Page 40, Question 5. 71 | - $S^\prime(x_1) = -S^{\prime\prime}(x_n)$, $S^{\prime\prime}(x_1) = -S^{\prime}(x_n)$. 72 | - $S_1(x) = a_1 + b_1(x - x_1) + c_1(x-x_1)^2 + d(x-x_1)^3$. Similar for $S_{n-1}(x)$. 73 | - The coefficient depend on the last two points, and the last two derivatives. 74 | i. $$S_1^\prime(x_1) = b_1 = s_1 = -S^{\prime\prime}_{n-1}(x_n) = -2c_{n-1} - 6d_{n-1}(x_n - x_{n-1})$$ 75 | ii. $$\begin{aligned}S^{\prime\prime}(x_1) &= 2c_1 \\ &= -S^\prime(x_n) \\ &= -(b_{n-1} + 2c_{n-1}(x_n - x_{n-1}) + 3d_{n-1}(x_n - x_{n-1})^2)\end{aligned}$$ 76 | 19. Page 41, Question 7. 77 | - A little bit of a pain. 78 | - Plug in $x_i, x_{i+1}$ in order to solve for some terms. 79 | - $S_i(x_i) = y_i = a_i \Delta x_i$. $S_i(x_{i+1}) = y_{i+1} = c_i \Delta x_i$. 80 | - To get $b_i$, we take derivatives. 81 | -------------------------------------------------------------------------------- /cs341/cs341.md: -------------------------------------------------------------------------------- 1 | CS 341 2 | = 3 | 4 | An **algorithm** is a well-defined procedure to solve a computational problem. 5 | 6 | ## Review of Asymptotic Notation 7 | 8 | $T(n) \in O(f(n))$ if and only if $\exists$ two positive constants $c, n_0$ such that $\forall n \ge n_0$, $T(n) \le c\cdot f(n)$. 9 | 10 | Example: $T(n) = \sum_{i=0}^k a_i n^i \in O(n^k)$, $a_k > 0$. 11 | 12 | > $$\begin{aligned}T(n) &\le \sum_{i=0}^k|a_i|n^i \\ &= n^k\sum_{i=0}^k|a_i|\end{aligned}$$ 13 | > So for $c = \sum_{i=0}^k|a_i|, n_0 = 1$, $T(n) \in O(n^k)$. 14 | 15 | $T(n) \in \Omega(f(n))$ if and only if $\exists c, n_0 > 0$ such that $\forall n \ge n_0, T(n) \ge c\cdot f(n)$. 16 | 17 | Example: $T(n) = \sum_{i=0}^k a_i n^i \in \Omega(n^k)$. 18 | 19 | > Let us guess $c = \frac{a_k}{2}$. 20 | > $$\begin{aligned}\sum_{i=0}^ka_in^i &\ge \frac{a_k}{2}n^k \\ n &\ge \sum_{i=0}^{k-1} \frac{-2a_in^i}{a_k n^{k-1}}\end{aligned}$$ 21 | > The problem is that the sum is not a constant so it cannot be $n_0$. 22 | > $$n = \sum_{i=0}^k\left|\frac{2a_i}{a_k}\right|$$ 23 | 24 | $T(n) \in \Theta(f(n))$ if and only if $T(n) \in O(f(n))$ and $T(n) \in \Omega(f(n))$. 25 | 26 | $T(n) \in o(f(n))$ if and only if $\forall c > 0, \exists n_0$ such that $\forall n \ge n_0 T(n) \le c \cdot f(n)$. 27 | 28 | $T(n) \in \omega(f(n))$ if and only if $\forall c > 0, \exists n_0$ such that $\forall n \ge n_0, T(n) \ge c\cdot f(n)$. 29 | 30 | ### Limits 31 | 32 | $T(n) \in o(f(n))$ iff $\lim_{n\to \infty}\frac{T(n)}{f(n)} = 0$. 33 | 34 | $T(n) \in \omega(f(n))$ iff $\lim_{n\to \infty}\frac{T(n)}{f(n)} = \infty$. 35 | 36 | $T(n) \in \Theta(f(n))$ iff $\lim_{n\to\infty}\frac{T(n)}{f(n)} = c > 0$. 37 | 38 | ## Important Summations 39 | 40 | 1. $\sum_{i=1}^n i = \frac{n}{n-1}{2} \in \Theta(n^2)$. 41 | 2. $\sum_{i=1}^n i^2 = \frac{n}{n+1}{2n + 1}{6} \in \Theta(n^3)$. 42 | 3. $\sum_{i=1}^ni^d \in \Theta(n^{d + 1})$. 43 | 4. $\sum_{i=1}^nc^i = \frac{c^{k-1} -1}{c-1} \in \begin{cases}\Theta(c^k), c > 1\\ \Theta(1), c < 1\\ \Theta(k), c = 1\end{cases}$ 44 | 5. $\sum_{i=1}^n \frac{1}{i} \in \Theta(\log n)$. 45 | 6. $\log(n!) = n\log n - \Theta(n)$, by Stirling's formula. 46 | 47 | # Reductions 48 | 49 | > Solving a computational problem $C_1$ by using an algorithm that solves a problem $C_2$. 50 | 51 | An equation or an inequality that describes a function $T(n)$ in terms of $T$'s value on inputs smaller than $n$ and a base case. 52 | 53 | Example: MergeSort. 54 | 55 | > $T(n) \le 2T(\frac{n}{2}) + 7n$, $T(2) = 2$. 56 | 57 | ## Solving Reccurences 58 | 59 | 1. Proof by induction. 60 | 2. Recursion tree method. 61 | 3. Master theorem. 62 | - If the input is of the form $T(n) \le aT(\frac{n}{b}) + f(n)$. 63 | 64 | ### Induction 65 | 66 | Exampe: Median-of-Medians. 67 | 68 | $$T(n) \le T(\frac{n}{5}) + T(\frac{7n}{10}) + n, T(1) = 1$$ 69 | 70 | > **Claim**: $T(n) \le 10n$. 71 | > 72 | > **Base Case**: $T(1) = 1 < 10$. 73 | > 74 | > **Induction Hypothesis**: $\forall k < n, T(k) \le 10k$. 75 | > 76 | > $$\begin{aligned}T(n) &\le \frac{10n}{5} + \frac{70n}{10} + n\\ &= 2n + 7n + n\\ &= 10\end{aligned}$$ 77 | 78 | ### Master Theorem 79 | 80 | We observed using the recursion tree method, that there are three possibilities in the overall runtime analysis. 81 | 82 | 1. Work at each level stays the same. 83 | 2. Work increases per level and work at the leaves dominate. 84 | 3. Work keeps decreasing per level and the work at the root dominates. 85 | 86 | Assume all subproblems are of equal size. 87 | 88 | $T(n) \le aT(\frac{n}{b}) + O(n^d), T(1) \le 1$, where $a$ is the number of recursive calls, $b$ is the input size shrinkage order, and $d$ is the exponent in the work done at the combine step. 89 | 90 | $$T(n) \in \begin{cases} 91 | O(n^d \log(n)),\ &a = b^d \\ 92 | O(n^{\log_b(a)}),\ &a > b^d \\ 93 | O(n^d),\ &a < b^d \\ 94 | \end{cases}$$ 95 | 96 | **Proof**. Assume for simplicity that $n$ is a power of $b$. We have $\log_b(n)$ levels, with $a^{\log_b(n)}$ leaves. The total work done at each level $j$ is $a^j(\frac{n}{b})^d$ so the total work is. 97 | 98 | $$\begin{aligned}\sum_{j=0}^{\log_b(n)}a^j\frac{n^d}{b^{jd}} &= cn^d\sum_{j=0}^{\log_b(n)} \left(\frac{a}{b^d}\right)^j\end{aligned}$$ 99 | 100 | We can obtain the runtimes by checking the convergence of the above sum. 101 | 102 | ## 2D Maxima 103 | 104 | **Input**: Set $P$ of $n$ 2D points. 105 | 106 | **Output**: All maximal points. $p$ is maximal if there is no such $p^\prime$ that dominates $p$ ($p^\prime.x > p.x \cap p^\prime.y > p.y)$. 107 | 108 | ## Integer Multiplication 109 | 110 | **Input**: 2 $n$-digit integers $X, Y$. 111 | 112 | **Output**: $Z = XY$. 113 | 114 | Let $X = a10^{\frac{n}{2}} + b$, $Y = c10^\frac{n}{2} + d$. So $XY = ac10^n + (ad + bc)10^\frac{n}{2} + bd$. We have a divide and conquer algorithm. 115 | 116 | $$T(n) \le 4T\left(\frac{n}{2}\right) + O(n)$$ 117 | 118 | $a = 4, b = 2, d = 1$, so $O(n^{\log_2(4)}) = O(n^2)$ by Master Theorem. Which is not faster than the naive approach. 119 | -------------------------------------------------------------------------------- /co250/co250.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 |Given a set A \subseteq \mathbb{R}^n and a function f: A \to \mathbb{R}, our goal is to find x \in A that minimizes or maximizes f.
31 |We consider optimization problems of the form \min\{f(x): g_i(x)\le b_i, 1 \le i \le m, x \in \mathbb{R}^n\} where n, m \in \mathbb{N}, b_i \in \mathbb{R}, and f, g_i are functions with form \mathbb{R}^n \to \mathbb{R}.
44 |45 |47 |Problems are very hard to solve in general, so we focus on special cases. We consider affine functions.
46 |
49 |51 |f: \mathbb{R}^n \to \mathbb{R} is affine if f(x) = a^Tx + \beta for a \in \mathbb{R}^n, \beta \in \mathbb{R}. It is linear if \beta = 0.
50 |
52 |54 |An optimization problem \min\{f(x): g_i(x) \le b_i, \forall 1 \le i \le m, x \in \mathbb{R}^n\} is called a linear program if f is affine and g_i is finite number of linear functions.
53 |
In practice, we make a series of decisions that influence each other. An example of this is multiperiod models.
59 |Fractional solutions are often not desirable. We can force solutions to take only integer values by formulating it was an integer program.
66 |In order to express binary decisions, we introduce a new variable y, such that y = 0 implies the first decision, and y = 1 implies the second.
70 |Example: x_1 + x_2 \ge 5 or x_3 + x_4 \ge 5.
71 |72 |79 |73 |
78 |- x_1 + x_2 \ge 5y.
74 |- x_3 + x_4 \ge 5(1-y).
75 |- 0 \le y \le 1.
76 |- y \in \mathbb{Z}.
77 |
We can solve similar situations by using a set of binary variables which sum to a specific value.
80 | 81 | 82 | -------------------------------------------------------------------------------- /pandoc.css: -------------------------------------------------------------------------------- 1 | /* 2 | * https://gist.github.com/killercup/5917178 3 | */ 4 | 5 | html { 6 | font-size: 100%; 7 | overflow-y: scroll; 8 | -webkit-text-size-adjust: 100%; 9 | -ms-text-size-adjust: 100%; 10 | } 11 | 12 | body { 13 | color: #444; 14 | font-family: Georgia, Palatino, 'Palatino Linotype', Times, 'Times New Roman', serif; 15 | font-size: 12px; 16 | line-height: 1.7; 17 | padding: 1em; 18 | margin: auto; 19 | max-width: 42em; 20 | background: #fefefe; 21 | } 22 | 23 | a { 24 | color: #0645ad; 25 | text-decoration: none; 26 | } 27 | 28 | a:visited { 29 | color: #0b0080; 30 | } 31 | 32 | a:hover { 33 | color: #06e; 34 | } 35 | 36 | a:active { 37 | color: #faa700; 38 | } 39 | 40 | a:focus { 41 | outline: thin dotted; 42 | } 43 | 44 | *::-moz-selection { 45 | background: rgba(255, 255, 0, 0.3); 46 | color: #000; 47 | } 48 | 49 | *::selection { 50 | background: rgba(255, 255, 0, 0.3); 51 | color: #000; 52 | } 53 | 54 | a::-moz-selection { 55 | background: rgba(255, 255, 0, 0.3); 56 | color: #0645ad; 57 | } 58 | 59 | a::selection { 60 | background: rgba(255, 255, 0, 0.3); 61 | color: #0645ad; 62 | } 63 | 64 | p { 65 | margin: 1em 0; 66 | } 67 | 68 | img { 69 | max-width: 100%; 70 | } 71 | 72 | h1, h2, h3, h4, h5, h6 { 73 | color: #111; 74 | line-height: 125%; 75 | margin-top: 2em; 76 | font-weight: normal; 77 | } 78 | 79 | h4, h5, h6 { 80 | font-weight: bold; 81 | } 82 | 83 | h1 { 84 | font-size: 2.5em; 85 | } 86 | 87 | h2 { 88 | font-size: 2em; 89 | } 90 | 91 | h3 { 92 | font-size: 1.5em; 93 | } 94 | 95 | h4 { 96 | font-size: 1.2em; 97 | } 98 | 99 | h5 { 100 | font-size: 1em; 101 | } 102 | 103 | h6 { 104 | font-size: 0.9em; 105 | } 106 | 107 | blockquote { 108 | color: #666666; 109 | margin: 0; 110 | padding-left: 3em; 111 | border-left: 0.5em #EEE solid; 112 | } 113 | 114 | hr { 115 | display: block; 116 | height: 2px; 117 | border: 0; 118 | border-top: 1px solid #aaa; 119 | border-bottom: 1px solid #eee; 120 | margin: 1em 0; 121 | padding: 0; 122 | } 123 | 124 | pre, code, kbd, samp { 125 | color: #000; 126 | font-family: monospace, monospace; 127 | _font-family: 'courier new', monospace; 128 | font-size: 0.98em; 129 | } 130 | 131 | pre { 132 | white-space: pre; 133 | white-space: pre-wrap; 134 | word-wrap: break-word; 135 | } 136 | 137 | b, strong { 138 | font-weight: bold; 139 | } 140 | 141 | dfn { 142 | font-style: italic; 143 | } 144 | 145 | ins { 146 | background: #ff9; 147 | color: #000; 148 | text-decoration: none; 149 | } 150 | 151 | mark { 152 | background: #ff0; 153 | color: #000; 154 | font-style: italic; 155 | font-weight: bold; 156 | } 157 | 158 | sub, sup { 159 | font-size: 75%; 160 | line-height: 0; 161 | position: relative; 162 | vertical-align: baseline; 163 | } 164 | 165 | sup { 166 | top: -0.5em; 167 | } 168 | 169 | sub { 170 | bottom: -0.25em; 171 | } 172 | 173 | ul, ol { 174 | margin: 1em 0; 175 | padding: 0 0 0 2em; 176 | } 177 | 178 | li p:last-child { 179 | margin-bottom: 0; 180 | } 181 | 182 | ul ul, ol ol { 183 | margin: .3em 0; 184 | } 185 | 186 | dl { 187 | margin-bottom: 1em; 188 | } 189 | 190 | dt { 191 | font-weight: bold; 192 | margin-bottom: .8em; 193 | } 194 | 195 | dd { 196 | margin: 0 0 .8em 2em; 197 | } 198 | 199 | dd:last-child { 200 | margin-bottom: 0; 201 | } 202 | 203 | img { 204 | border: 0; 205 | -ms-interpolation-mode: bicubic; 206 | vertical-align: middle; 207 | } 208 | 209 | figure { 210 | display: block; 211 | text-align: center; 212 | margin: 1em 0; 213 | } 214 | 215 | figure img { 216 | border: none; 217 | margin: 0 auto; 218 | } 219 | 220 | figcaption { 221 | font-size: 0.8em; 222 | font-style: italic; 223 | margin: 0 0 .8em; 224 | } 225 | 226 | table { 227 | margin-bottom: 2em; 228 | border-bottom: 1px solid #ddd; 229 | border-right: 1px solid #ddd; 230 | border-spacing: 0; 231 | border-collapse: collapse; 232 | } 233 | 234 | table th { 235 | padding: .2em 1em; 236 | background-color: #eee; 237 | border-top: 1px solid #ddd; 238 | border-left: 1px solid #ddd; 239 | } 240 | 241 | table td { 242 | padding: .2em 1em; 243 | border-top: 1px solid #ddd; 244 | border-left: 1px solid #ddd; 245 | vertical-align: top; 246 | } 247 | 248 | .author { 249 | font-size: 1.2em; 250 | text-align: center; 251 | } 252 | 253 | @media only screen and (min-width: 480px) { 254 | body { 255 | font-size: 14px; 256 | } 257 | } 258 | @media only screen and (min-width: 768px) { 259 | body { 260 | font-size: 16px; 261 | } 262 | } 263 | @media print { 264 | * { 265 | background: transparent !important; 266 | color: black !important; 267 | filter: none !important; 268 | -ms-filter: none !important; 269 | } 270 | 271 | body { 272 | font-size: 12pt; 273 | max-width: 100%; 274 | } 275 | 276 | a, a:visited { 277 | text-decoration: underline; 278 | } 279 | 280 | hr { 281 | height: 1px; 282 | border: 0; 283 | border-bottom: 1px solid black; 284 | } 285 | 286 | a[href]:after { 287 | content: " (" attr(href) ")"; 288 | } 289 | 290 | abbr[title]:after { 291 | content: " (" attr(title) ")"; 292 | } 293 | 294 | .ir a:after, a[href^="javascript:"]:after, a[href^="#"]:after { 295 | content: ""; 296 | } 297 | 298 | pre, blockquote { 299 | border: 1px solid #999; 300 | padding-right: 1em; 301 | page-break-inside: avoid; 302 | } 303 | 304 | tr, img { 305 | page-break-inside: avoid; 306 | } 307 | 308 | img { 309 | max-width: 100% !important; 310 | } 311 | 312 | @page :left { 313 | margin: 15mm 20mm 15mm 10mm; 314 | } 315 | 316 | @page :right { 317 | margin: 15mm 10mm 15mm 20mm; 318 | } 319 | 320 | p, h2, h3 { 321 | orphans: 3; 322 | widows: 3; 323 | } 324 | 325 | h2, h3 { 326 | page-break-after: avoid; 327 | } 328 | } 329 | -------------------------------------------------------------------------------- /cs370/cs370help.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 |30 |32 |Question and answer format.
31 |
66 |68 |Note: Exams do not try to fool anybody.
67 |
An algorithm is a well-defined procedure to solve a computational problem.
30 |T(n) \in O(f(n)) if and only if \exists two positive constants c, n_0 such that \forall n \ge n_0, T(n) \le c\cdot f(n).
32 |Example: T(n) = \sum_{i=0}^k a_i n^i \in O(n^k), a_k > 0.
33 |34 |36 |\begin{aligned}T(n) &\le \sum_{i=0}^k|a_i|n^i \\ &= n^k\sum_{i=0}^k|a_i|\end{aligned} So for c = \sum_{i=0}^k|a_i|, n_0 = 1, T(n) \in O(n^k).
35 |
T(n) \in \Omega(f(n)) if and only if \exists c, n_0 > 0 such that \forall n \ge n_0, T(n) \ge c\cdot f(n).
37 |Example: T(n) = \sum_{i=0}^k a_i n^i \in \Omega(n^k).
38 |39 |41 |Let us guess c = \frac{a_k}{2}. \begin{aligned}\sum_{i=0}^ka_in^i &\ge \frac{a_k}{2}n^k \\ n &\ge \sum_{i=0}^{k-1} \frac{-2a_in^i}{a_k n^{k-1}}\end{aligned} The problem is that the sum is not a constant so it cannot be n_0. n = \sum_{i=0}^k\left|\frac{2a_i}{a_k}\right|
40 |
T(n) \in \Theta(f(n)) if and only if T(n) \in O(f(n)) and T(n) \in \Omega(f(n)).
42 |T(n) \in o(f(n)) if and only if \forall c > 0, \exists n_0 such that \forall n \ge n_0 T(n) \le c \cdot f(n).
43 |T(n) \in \omega(f(n)) if and only if \forall c > 0, \exists n_0 such that \forall n \ge n_0, T(n) \ge c\cdot f(n).
44 |T(n) \in o(f(n)) iff \lim_{n\to \infty}\frac{T(n)}{f(n)} = 0.
46 |T(n) \in \omega(f(n)) iff \lim_{n\to \infty}\frac{T(n)}{f(n)} = \infty.
47 |T(n) \in \Theta(f(n)) iff \lim_{n\to\infty}\frac{T(n)}{f(n)} = c > 0.
48 |59 |61 |Solving a computational problem C_1 by using an algorithm that solves a problem C_2.
60 |
An equation or an inequality that describes a function T(n) in terms of T’s value on inputs smaller than n and a base case.
62 |Example: MergeSort.
63 |64 |66 |T(n) \le 2T(\frac{n}{2}) + 7n, T(2) = 2.
65 |
Exampe: Median-of-Medians.
77 |T(n) \le T(\frac{n}{5}) + T(\frac{7n}{10}) + n, T(1) = 1
78 |79 |84 |Claim: T(n) \le 10n.
80 |Base Case: T(1) = 1 < 10.
81 |Induction Hypothesis: \forall k < n, T(k) \le 10k.
82 |\begin{aligned}T(n) &\le \frac{10n}{5} + \frac{70n}{10} + n\\ &= 2n + 7n + n\\ &= 10\end{aligned}
83 |
We observed using the recursion tree method, that there are three possibilities in the overall runtime analysis.
86 |Assume all subproblems are of equal size.
92 |T(n) \le aT(\frac{n}{b}) + O(n^d), T(1) \le 1, where a is the number of recursive calls, b is the input size shrinkage order, and d is the exponent in the work done at the combine step.
93 |T(n) \in \begin{cases} 94 | O(n^d \log(n)),\ &a = b^d \\ 95 | O(n^{\log_b(a)}),\ &a > b^d \\ 96 | O(n^d),\ &a < b^d \\ 97 | \end{cases}
98 |Proof. Assume for simplicity that n is a power of b. We have \log_b(n) levels, with a^{\log_b(n)} leaves. The total work done at each level j is a^j(\frac{n}{b})^d so the total work is.
99 |\begin{aligned}\sum_{j=0}^{\log_b(n)}a^j\frac{n^d}{b^{jd}} &= cn^d\sum_{j=0}^{\log_b(n)} \left(\frac{a}{b^d}\right)^j\end{aligned}
100 |We can obtain the runtimes by checking the convergence of the above sum.
101 |Input: Set P of n 2D points.
103 |Output: All maximal points. p is maximal if there is no such p^\prime that dominates p (p^\prime.x > p.x \cap p^\prime.y > p.y).
104 |Input: 2 n-digit integers X, Y.
106 |Output: Z = XY.
107 |Let X = a10^{\frac{n}{2}} + b, Y = c10^\frac{n}{2} + d. So XY = ac10^n + (ad + bc)10^\frac{n}{2} + bd. We have a divide and conquer algorithm.
108 |T(n) \le 4T\left(\frac{n}{2}\right) + O(n)
109 |a = 4, b = 2, d = 1, so O(n^{\log_2(4)}) = O(n^2) by Master Theorem. Which is not faster than the naive approach.
110 | 111 | 112 | -------------------------------------------------------------------------------- /cs370/cs370.md: -------------------------------------------------------------------------------- 1 | CS 370 2 | = 3 | 4 | # Floating Point 5 | 6 | Computers have a limited amount of memory, they cannot represent all numbers. 7 | 8 | $$S \subset \mathbb{R}$$ 9 | 10 | Assume we want to compute $e^{-5.5}$. Using the Taylor expansion, we know that $e^x = 1 + x + \frac{x^2}{2!} + \frac{x^3}{3!} + ...$. We cannot include infinite terms, so we need to make an approximation. This will lead to a result of 0.0026363. 11 | 12 | However, if we use $e^{-x} = \frac{1}{1 + x + \frac{x^2}{2!} + ...}$, we will get a result of 0.0040865 which is much closer to the true value. 13 | 14 | Let us consider the number 12.25. This is equal to $1\cdot 10 + 2\cdot 10^0 + 2\cdot 10^{-1} + 5\cdot10^{-5}$. We can generalize this as $d_k \in \{0, 1, ..., 9\}$ with $d_0d_1...d_{j-1}.d_jd_{j+1}...d_{j+i-1}$and is represented as $d_0\cdot 10^{j-1} + d_1\cdot 10^{j-2} + ... + d_{j+i-1} \cdot 10^{-i}$. Computers only understand bits, so $d_k \in \{0, 1\}$. So instead we have $(\sum_{k=0}^{j+i-1}d_k\cdot 2^{-k})\cdot 2^{j-1}$. We have some limitations because $j, i$ cannot be arbitrarily large. 15 | 16 | We need to truncate, we decide an upper bound on the exponent and the number of significant digits we can represent. 17 | 18 | ### What numbers can we represent? 19 | 20 | Example: $\beta = 2, m = 3$. $d_k \in \{0, 1\}, 0 \le j \le 3$. 21 | 22 | $$\overline{x} = \left(\sum_{k=0}^2 d_k 2^{-k}\right)\cdot 2^{j-1}$$ 23 | 24 | Assume $d_0 > 0$, so $d_0 = 1$ (normalized floating point convention). We can simplify the above expression. 25 | 26 | $$\overline{x} = 2^{j-1} + \left(\sum_{k=1}^2 d_k 2^{-k}\right)\cdot 2^{j-1}$$ 27 | 28 | We minimize $\overline{x}$ with $j=0, d_k = 0$. We maximize with $j=3, d_k = 1$. 29 | 30 | In general, given $\beta, m, j_{max}, j_{min}$, the minimum is $\beta^{j_{min} - 1}$ and the maximum is $(\beta - \beta^{1 - m}) \cdot \beta^{j_{max}}$. So all numbers are in the range of $[\beta^{j_{min} - 1}, (\beta - \beta^{1 - m})\cdot \beta^{j_{max}}]$. 31 | 32 | The minimum significant is 1, and the maximum significant is $\beta$. Fixing $j$, then $\Delta = \beta^{j-m}$. 33 | 34 | The number of floating point numbers within the interval are $\frac{\beta^j - \beta^{j-1}}{\Delta} = (\beta - 1) \beta^{m-1}$. So the error is bounded by $|f(x) - x| \le \frac{1}{2} \beta^{-(m-1)}\beta^{j-1}$, with relative error $\frac{|f(x) - x|}{|x|} \le \frac{1}{2}\beta^{-(m-1)}$. 35 | 36 | ### Errors 37 | 38 | We can have error when the arithmetic operation results in a number with $m^\prime > m$. 39 | 40 | Example: $\overline{x} = 1.235, \overline{y} = 1.234$. $\overline{x} - \overline{y} = 1.509\mathbf{1}$. 41 | 42 | Example: $I_n = \int_{0}^1 \frac{x^n}{x+\alpha}dx$ 43 | 44 | $$I_0 = \int_0^1 \frac{1}{x+\alpha}dx = \log(1+\alpha) - \log(\alpha) = \log(\frac{1 + \alpha}{\alpha})$$ 45 | We can compute recursively, $I_n = \frac{1}{n} - \alpha I_{n-1}$. 46 | 47 | # Polynomial Interpolation 48 | 49 | Given $n$ points $(x_i, y_i)$ where $x_i$ are distinct. Find a polynomial $p(x)$ of degree at most $n-1$ such that $p(x_i) = y_i$. 50 | 51 | We want $p(x) = c_1 + c_2x + ... + c_nx^{n-1}$. To find $p(x)$, we set up $n$ linear equations in $n$ unknowns. 52 | 53 | 1. Does the interpolating polynoials always exist? 54 | 2. Are interpolating polynomials **unique**? 55 | 56 | We can either show that $y_i = 0 \Rightarrow c_i = 0$ *or* that we can **always** find an answer *or* that $\det V = \Pi_{iComputers have a limited amount of memory, they cannot represent all numbers.
31 |S \subset \mathbb{R}
32 |Assume we want to compute e^{-5.5}. Using the Taylor expansion, we know that e^x = 1 + x + \frac{x^2}{2!} + \frac{x^3}{3!} + .... We cannot include infinite terms, so we need to make an approximation. This will lead to a result of 0.0026363.
33 |However, if we use e^{-x} = \frac{1}{1 + x + \frac{x^2}{2!} + ...}, we will get a result of 0.0040865 which is much closer to the true value.
34 |Let us consider the number 12.25. This is equal to 1\cdot 10 + 2\cdot 10^0 + 2\cdot 10^{-1} + 5\cdot10^{-5}. We can generalize this as d_k \in \{0, 1, ..., 9\} with d_0d_1...d_{j-1}.d_jd_{j+1}...d_{j+i-1}and is represented as d_0\cdot 10^{j-1} + d_1\cdot 10^{j-2} + ... + d_{j+i-1} \cdot 10^{-i}. Computers only understand bits, so d_k \in \{0, 1\}. So instead we have (\sum_{k=0}^{j+i-1}d_k\cdot 2^{-k})\cdot 2^{j-1}. We have some limitations because j, i cannot be arbitrarily large.
35 |We need to truncate, we decide an upper bound on the exponent and the number of significant digits we can represent.
36 |Example: \beta = 2, m = 3. d_k \in \{0, 1\}, 0 \le j \le 3.
38 |\overline{x} = \left(\sum_{k=0}^2 d_k 2^{-k}\right)\cdot 2^{j-1}
39 |Assume d_0 > 0, so d_0 = 1 (normalized floating point convention). We can simplify the above expression.
40 |\overline{x} = 2^{j-1} + \left(\sum_{k=1}^2 d_k 2^{-k}\right)\cdot 2^{j-1}
41 |We minimize \overline{x} with j=0, d_k = 0. We maximize with j=3, d_k = 1.
42 |In general, given \beta, m, j_{max}, j_{min}, the minimum is \beta^{j_{min} - 1} and the maximum is (\beta - \beta^{1 - m}) \cdot \beta^{j_{max}}. So all numbers are in the range of [\beta^{j_{min} - 1}, (\beta - \beta^{1 - m})\cdot \beta^{j_{max}}].
43 |The minimum significant is 1, and the maximum significant is \beta. Fixing j, then \Delta = \beta^{j-m}.
44 |The number of floating point numbers within the interval are \frac{\beta^j - \beta^{j-1}}{\Delta} = (\beta - 1) \beta^{m-1}. So the error is bounded by |f(x) - x| \le \frac{1}{2} \beta^{-(m-1)}\beta^{j-1}, with relative error \frac{|f(x) - x|}{|x|} \le \frac{1}{2}\beta^{-(m-1)}.
45 |We can have error when the arithmetic operation results in a number with m^\prime > m.
47 |Example: \overline{x} = 1.235, \overline{y} = 1.234. \overline{x} - \overline{y} = 1.509\mathbf{1}.
48 |Example: I_n = \int_{0}^1 \frac{x^n}{x+\alpha}dx
49 |I_0 = \int_0^1 \frac{1}{x+\alpha}dx = \log(1+\alpha) - \log(\alpha) = \log(\frac{1 + \alpha}{\alpha}) We can compute recursively, I_n = \frac{1}{n} - \alpha I_{n-1}.
50 |Given n points (x_i, y_i) where x_i are distinct. Find a polynomial p(x) of degree at most n-1 such that p(x_i) = y_i.
52 |We want p(x) = c_1 + c_2x + ... + c_nx^{n-1}. To find p(x), we set up n linear equations in n unknowns.
53 |We can either show that y_i = 0 \Rightarrow c_i = 0 or that we can always find an answer or that \det V = \Pi_{i<j}(x_i - x_j) \neq 0 when x_i distinct.
58 |We can always construct an interpolating polynomial as follows. p(x) = \sum_{i=1}^n y_i L_i(x). Where each L_i(x) is a polynomial of degree at most n-1 which also satisfies L_i(x_i) = 1, L_i(x_j) = 0, \forall i \neq j.
60 |61 |63 |L_i(x) = \frac{(x - x_1)(x - x_2)...(x - x_{i-1})(x - x_{i+1})...(x_n)}{(x_i - x_1)(x_i - x_2)...(x_i - x_{i-1})(x_i - x_{i+1})...(x_i - x_n)}
62 |
I_0, I_1, ..., I_n = \frac{1}{n} - \alpha I_{n-1}.
65 |66 |68 |If our starting point is incorrect, we want to look at the errors in every step. How does the error grow? We have a recurrence for the error.
67 |
Given n points (x_i, y_i) with dervative values s_i, we attempt to find S(x) of degree n+1 satisfying S(x_i) = y_i, S(x_i)^\prime = s_i.
73 |75 |77 |Polynomial interpolation does not tend give answers which keep the shape of the given points, it flows between them. We want a function which looks good.
76 |
We want a function S(x) which satisfies the following conditions.
78 |S(x) = \begin{cases} 85 | S_1(x),\ &x_1 \le x \le x_2\\ 86 | S_2(x),\ &x_2 \le x \le x_2 \\ 87 | ... \\ 88 | S_{n-1}(x),\ &x_{n-1} \le x \le x_n\\ 89 | \end{cases}
90 |Where s_i, s_i^\prime are unknowns, well-defined.
91 |How many unknowns? 4(n-1). How many conditions? 2(n-1) interpolation, 2(n-2) derivatives, 2 boundary conditions.
92 |S^{\prime\prime}(x_1) = S^{\prime\prime}(x_n) = 0.
95 |S^{\prime}(x_1) = s_1, S^{\prime}(x_n) = s_n, where s_1, s_n are constants.
97 |S^\prime(x_1) = S^\prime(x_n), S^{\prime\prime}(x_1) = S^{\prime\prime}(x_n).
99 |S^{\prime\prime\prime} is continuous at x_2 and x_{n-1}.
101 |103 |105 |We dont want to sovle a system of 4(n-1) unknowns.
104 |
We know that in general.
114 |121 |123 |For natural splines.
122 |
Determined by first boundary condition.
125 |126 |S^{\prime\prime} = 0 \Rightarrow 2c_1 = 0. So then 2s_1 + s_2 = 3y_1^\prime.
127 |
Determined by second boundary condition.
129 |130 |S^{\prime\prime} = 0 \Rightarrow S_{n-1}^{\prime\prime} = 0. So 2c_{n-1} + 6d_{n-1}(x_n - x_{n-1}) = 0. \begin{aligned}\frac{3y_{n-1}^\prime - 2s_{n-1} - s_n}{\Delta x_{n-1}} + \frac{3(s_{n-1} + s_n - 2y_{n-1}^\prime)}{\Delta x_{n-1}^2} \Delta x_{n-1} &= 0 \\ s_{n-1} + 2s_n &= 3y_{n-1}^\prime\end{aligned}
131 |
Based on the interior points.
133 |134 |\begin{aligned} 135 | S^{\prime\prime}_{i-1}(x_i) &= S^{\prime\prime}_i(x_i) \\ 136 | 2c_{i-1}+6d_{i-1}(x_i - x_{i+1}) &= 2c_i - 6d_i(x_i - x_i) \\ 137 | c_{i-1} + 3d_{i-1}\Delta x_{i-1} &= c_i \\ 138 | \frac{3y_{i-1}^\prime - 2s_{i-1} - s_i}{\Delta x_{i-1}} + \frac{3(s_{i-1} + s_i - 2y_{i-1}^\prime)}{\Delta x_{i-1}^2}\Delta x_{i-1} &= \frac{3y_{i}^\prime - 2s_i - 2s_{i+1}}{\Delta x_i} \\ 139 | \Delta x_i S_{i-1} + 2(\Delta x_{i-1} + \Delta x_i)S_i + \Delta x_{i-1} S_{i+1} &= 3y_i^\prime \Delta x_{i-1} + 3y_{i-1}^\prime \Delta x_i \\ 140 | \end{aligned}
141 |
\begin{bmatrix} 144 | 2 & 1 & 0 & 0 & 0 & ... & 0 & 0 & 0 \\ 145 | \star & \star & \star & 0 & 0 & ... & 0 & 0 & 0\\ 146 | 0 & \star & \star & \star & 0 & ... & 0 & 0 & 0\\ 147 | 0 & 0 & \star & \star & \star & ... & 0 & 0 & 0\\ 148 | & & & & & ... & & & \\ 149 | 0 & 0 & 0 & 0 & 0 & ... & \star & \star & \star \\ 150 | 0 & 0 & 0 & 0 & 0 & ... & 0 & 1 & 2\\ 151 | \end{bmatrix}\begin{bmatrix} 152 | s_1 \\ s_2 \\ s_3 \\ s_4 \\ \ \\ ... \\ s_n \end{bmatrix} = \begin{bmatrix} 153 | 3y_1^\prime \\\ \\\ \\\\ \\\ \\ 3y_n^\prime 154 | \end{bmatrix}
155 |Example: (1, 1), (2, 2), (3, 1), (4, 0), (5, 3), (6, 3).
156 |157 |167 |\Delta x_{i} = 1. y_1^\prime = 1, y_2^\prime = -1, y_3^\prime = -1, y_4^\prime = 3, y_5^\prime = 0.
158 |\left[\begin{array}{cccccc|c} 159 | 2 & 1 & 0 & 0 & 0 & 0 & 3\\ 160 | 1 & 4 & 1 & 0 & 0 & 0 & 0\\ 161 | 0 & 1 & 4 & 1 & 0 & 0 & -6\\ 162 | 0 & 0 & 1 & 4 & 1 & 0 & 6\\ 163 | 0 & 0 & 0 & 1 & 4 & 1 & 9\\ 164 | 0 & 0 & 0 & 0 & 1 & 2 & 0\\ 165 | \end{array}\right]
166 |
Given (x_i, y_i) as points on a curve, we want to find (x(t), y(t)), t_1 \le t \le t_n.
169 |Examples.
195 |Solving means to numerically solve. We want t_i, y_i such that y_k \approx y(t_k).
205 |Three methods are provided.
206 |In each we will assume we are given a time increment h = \Delta t and all time increments will be constant t_k = t_k + k \cdot h.
212 | 213 | 214 | -------------------------------------------------------------------------------- /cs350/cs350.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 |An OS is a system that manages resources, creates execution environments, loads programs, and provides common services and utilities.
30 |Provides an execution environment for running programs.
41 |The OS itself also uses resources.
53 |Kernel: Part of OS that responds to system calls, interrupts, and exceptions.
59 |Operating System: OS as a while includes the kernel, and may include other related programs to provide services for applications.
60 |The execution environment provides by the OS includes a variety of abstractions.
66 |74 |76 |Better utilization of the CPU.
75 |
What is a tread? It is a sequence of instructions.
77 |OS/161 Thread Interface.
83 |84 |86 |kern/include/thread.h
85 |
int thread_fork(
87 | const char* name,
88 | struct proc* proc,
89 | void (*func)(void*, unsigned long),
90 | void* data1,
91 | unsigned long data2);
92 |
93 | // Terminate the calling thread.
94 | void thread_exit(void);
95 |
96 | // Voluntarily yield execution.
97 | void thread_yield(void);100 |102 |What options exist?
101 |
109 |111 |Various Stack Frames \to thread_yield \to thread_switch \to switchframe.
110 |
112 |114 |kern/arch/mips/thread/switch.S
113 |
switchframe_switch:
115 | /* a0: switchframe pointer of old thread. */
116 | /* a1: switchframe pointer of new thread. */
117 |
118 | /* Allocate space for 10 registers. */
119 | addi sp, sp, -40
120 | /* Return address. */
121 | sw ra, 36(sp)
122 | /* Globals pointer. */
123 | sw gp, 32(sp)
124 | sw s8, 28(sp)
125 | sw s6, 24(sp)
126 | sw s5, 20(sp)
127 | sw s4, 16(sp)
128 | sw s3, 12(sp)
129 | sw s2, 8(sp)
130 | sw s1, 4(sp)
131 | sw s0, 0(sp)
132 |
133 | /* Store old stack pointer in old thread. */
134 | sw sp, 0(a0)136 |138 |Loading data from the new stack pointer is similar. Uses nop operation in order to ensure loads have completed before usage.
137 |
182 |184 |OS/161 threads use preemptive round-robin scheduling.
183 |
Every time an interrupt is called, a trap frame is the first. Then the interrupt handler stack frame is called, followed by the normal context switching procedures.
185 |When returning from the trap_frame, the original state is restored.
186 |Before entering a critical section, acquire a lock. Upon leaving, release the lock.
188 |Implemented in hardware because we need an atomic test-and-set.
189 |Acquire(bool* lock) {
190 | while (Xchg(true, lock) == true);
191 | }
192 | Release(bool* lock) {
193 | *lock = false;
194 | }196 |198 |printf forces the execution of threads to be a certain way, and can hide race conditions in your code!
197 |
ARMTestAndSet(addr, value) {
200 | // Load value.
201 | tmp = LDREX addr
202 | // Store new value.
203 | result = STREX value, addr
204 | if (result == SUCCEED) return tmp
205 | return TRUE
206 | }
207 |
208 | Acquire(bool *lock) {
209 | while(ARMTestAndSet(lock, true) == true) {};
210 | }“Spins” repeatedly until the lock is available.
214 |struct spinlock { volatile spinlock_data_t lk_lock; struct cpu* lk_holder; };
215 |void spinlock_init(struct spinlock* lk); void spinlock_acquire(struct spinlock* lk); void spinlock_release(struct spinlock* lk);
218 |220 |spinlock_acquire calls spinlock_data_testandset in a loop until the lock is acquired. It turns off interrupts on the cpu.
219 |
Spinlocks are not efficient, if we want large chunks of code protected, we should use a lock (mutex). The difference is that spinlocks spin, locks block.
221 |232 |234 |Wait channels are used to implement thread blocking.
233 |
void wchan_sleep(struct wchan* wc);
235 | void wchan_wakeall(struct wchan* wc);
236 | void wchan_wakeone(struct wchan* wc);
237 | void wchan_lock(struct wchan* wc);253 |255 |P and V are atomic.
254 |
Example.
260 |volatile int total = 0;
261 | struct semaphore *total_sem;
262 | total_sem = sem_create("total_mutex", 1);
263 |
264 | void add() {
265 | int i;
266 | for (i = 0; i < N; ++i) {
267 | P(sem);
268 | total++;
269 | V(sem);
270 | }
271 | }Multiple ways to to keep track of where cars are in the intersection.
311 |320 |322 |sy2, uw1, sy3 are the tests to run.
321 |
lock lock_a, lock_b
341 |
342 | int FuncA() {
343 | lock_acquire(lock_a);
344 | lock_acquire(lock_b);
345 | ...
346 | lock_release(lock_a);
347 | lock_relase(lock_b);
348 | }
349 |
350 | int FuncB() {
351 | lock_acquire(lock_b);
352 | lock_acquire(lock_a);
353 | ...
354 | lock_relase(lock_b);
355 | lock_release(lock_a);
356 | }Example.
365 |lock A, B
366 |
367 | try_acquire() {
368 | spin_acquire(lk->spin);
369 | if (lk->held) {
370 | release(lk->spin);
371 | return false;
372 | }
373 | lk->held = true;
374 | lk->owner = me;
375 | release(lk->spin);
376 | return true;
377 | }
378 |
379 | FuncA() {
380 | acquire(A)
381 | while(!try_acquire(B)) {
382 | release(A);
383 | acquire(A);
384 | }
385 | }Network Flow Theory: Class of optimization problems on graphs (directed graphs), usually involving the movement of stuff.
39 |Example: Transshipment problem.
40 |41 |44 |Directed graph. Nodes weights are the supply or demand, arc weights are the cost to move one good across the arc. We want to move all goods from supply to demand while minimizing total cost.
42 |Solutions are representing using another digraph with arcs representing the amount of goods transfered between nodes.
43 |
Trees.
61 |80 |82 |For S \subseteq N, the cut induced by S is the set of all arcs whose tails are in S and heads are not in S. \delta(S) = \{uv \in A: u \in S, v \not\in S\}. \delta(\overline{S}) = \{uv \in A, u \not\in S, v \in S\}.
81 |
Proposition: There exists an s,t-dipath if and only if every s,t-cut is non-empty.
86 |87 |90 |(\Rightarrow) Let s = v_1,v_2,...,v_k=t be a s,t-dipath. Let \delta(S) be an s,t-cut. Since v_1 \in S, v_k \not\in S, there exists a largest index i < k such that v_i \in S, v_{i+1} \not\in S. So v_i v_{i+1} \in \delta(S), so \delta(S) is non-empty.
88 |(\Leftarrow) Assume every s,t-cut is non-empty. Let S be the set of all nodes v such that an s,v-dipath exists. If t \in S, then an s,t-dipath exists, and we are done. If t \not\in S, then \delta(S) \neq \emptyset, so there must exist some arc uv \in \delta(S). Since u \in S, there exists a u,v-dipath P. Then P + uv is an s,v-dipath so v \in S, contradiction.
89 |
92 |94 |Remove the directions on the arcs to get edges.
93 |
109 |111 |A solution is called a flow. The goal is to minimize the total cost of the flow.
110 |
113 |115 |Notation: If x \in \mathbb{R}^T and S \subseteq T, then x(S) = \sum_{i \in S}x_i.
114 |
\begin{aligned} 121 | \min\ &w^Tx \\ 122 | \text{s.t } &x(\delta(\overline{v})) - x(\delta(v)) = b_v,\ \forall v \in N \\ 123 | &x \ge 0 124 | \end{aligned}
125 |Matrix form is called an incidence matrix where the rows represent nodes and the columns represent arcs.
126 |\begin{aligned} 128 | \min\ &w^T x \\ 129 | \text{s.t. } &x(\delta(\overline{v})) - x(\delta(v)) = b_v,\ &\forall v \in N \\ 130 | x \ge \mathbf{0} 131 | \end{aligned}
132 |\begin{aligned} 136 | \max\ &b^T y \\ 137 | \text{s.t. } &y_v - y_u \le w_{uv}\ \forall uv \in A 138 | \end{aligned}
139 |141 |143 |Recall: Variable is zero or the corresponding constraint is tight.
142 |
x_{uv} = 0 or y_v - y_u = w_{uv}, \forall uv \in A. So x_{uv} > 0 implies y_v - y_u = w_{uv}.
147 |148 |150 |Recall: Let (P), (D) be a primal-dual pair. Then x, y are optimal solutions to (P), (D) if and only if they are feasible and satisfy CS conditions.
149 |
Given a feasible solution to (P), how do we generate a feasible solution to (D)?
151 |152 |155 |Arbitrarily assign the value of a node to be 0, then use the arc constraints to fill in the other nodes. So we have a feasible x and x,y satisfy CS conditions. It remains to show that y is feasible.
153 |We need to check arc conditions not included in the solution.
154 |
Alternatively, given that x, y are feasible, use strong duality.
156 |158 |160 |Recall: Basis is a set of linearly independent column that form a square invertible matrix. Every basis corresponds to a unique basic solution.
159 |
Consider the incidence matrix of a digraph. The rows are linearly independent, because every column will sum to zero. So the rank of the matrix is at most |N|-1.
161 |We assume that the coefficient matrix has full row rank in simplex. So we row reduce to get an equivalent system with full row rank.
162 |What is the rank of the incidence matrix of a digraph D(N, A)?
163 |164 |167 |There are |N| rows, rows are dependent, so the rank is at most |N| - 1. Every column represents an arc, so we can ask which arcs can we pick so the corresponding columns form a basis.
165 |Consider any undirected cycle in the digraph. This corresponds to a square matrix with dependent columns.
166 |
Proposition: If a set of arcs does not contain a cycle, then the columns of the incidence matrix corresponding to the arcs are linearly independent.
168 |169 |171 |Suppose these columns are not independent. Let us consider the set of columns with non-zero coefficients. So for any row, we have either all entries zero or at least two non-zero. Let us collect all these arcs and the nodes with at least 2 non-zero entries. We have a digraph where every node is incident with at least 2 arcs. So this digraph must have a cycle.
170 |
So to find independent columns, we find arcs that don’t have cycles. To find a basis, we need as many arcs as possible. For any connected digraph, a spanning tree exists with |N| - 1 arcs, corresponding to |N| - 1 independent columns. So the rank is at least |N| - 1, but the rank is at most |N| - 1. So the rank is equal to |N| - 1 and we have a basis.
172 |Theorem: Let M be the incidence matrix of a connected digraph D = (N, A). Then the rank of M is |N| - 1. Moreover, a set of |N| - 1 columns of M is a basis if and only if the |N| - 1 arcs corresponding to these columns is a spanning tree of D.
173 |Exercise: Determine the rank for a disconnected digraph.
174 |Consider a state of the simplex algorithm.
183 |Suppose we have some coefficients in the objective vector equal to zero (canonical form). What does this mean for the corresponding dual?
185 |186 |188 |The dual constraint is tight for the basic variables. Non-basic variables have value 0.
187 |
So the CS conditions are satisfied.
Suppose a coefficient is positive, what does this mean for the corresponding dual?
190 |191 |The dual constraint is not satisfied. We are “trying to fix” that constraint in the step of the simplex algorithm.
192 |
We stop when all constriants are non-positive, what does this mean for the corresponding dual?
194 |195 |All dual constraints are satisfied, y is feasible.
196 |
So we see that the simplex keeps x feasible, with CS conditions satisfied, and works towards the feasibility of y. When y becomes feasible, we have optimal solutions.
199 |For the dual, we add a slack variable so that it is an equality constraint. -y_u + y_v + \overline{w_{uv}} = w_{uv}, \forall uv \in A, \overline{w} \ge \mathbf{0}.
202 |Definition: A vector y \in \mathbb{R}^N is called a node potential. Given y, the reduced cost of arc uv is \overline{w_{uv}} = y_u + w_{uv} - y_v. A node potential is feasible if \overline{w_{uv}} \ge 0, \forall uv \in A.
208 |Exercise: How do we find a “leaving arc”? Add flow to the entering arc, adjust flow on basic arcs while maintaining feasibility.
209 | 210 | 211 | -------------------------------------------------------------------------------- /co487/co487.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 |Symmetric-key cryptography: The client and server share some secret information k, called a key.
37 |Public-key cryptography: Share an authenticated (but not secret) information.
38 |Is SSL really secure?
50 |There are many potential security vulnerabilities.
51 |Crypto might be strong, but information security is more difficult (cybersecurity).
64 |76 |78 |Such that D_k(E_k(m)) = m.
77 |
Why not always use the secure channel? It is commonly slow, inconvenient.
84 |Security model: Defines the computational abilities of the adversary, and how she interacts.
91 |Basic assumption: Adversary knows everything about the SKES, only missing the key k.
92 |Attacks which are not considered in this course.
105 |122 |124 |We want to guard against the strongest attacker going for the weakest goal.
123 |
A SKES is said to be secure if it is semantically secure against a chosen-plaintext attack by a computational bounded adversary.
129 |To break a SKES, the attacker needs to do the following.
130 |144 |146 |The Bitcoin network is presently performing hash operations at the rate of 2^{65} per second, 2^{90} per year.
145 |
147 |149 |The Laundeur limit from thermodynamics proposes that exhaustively trying 2^{128} symmetric keys would require more power for a year than the world produces.
148 |
A cryptographic scheme is said to have a security level of l bits if the fastest known attack on the scheme takes approximately 2^l operations. As of 2019, a security level of 128 bits is desirable in practice.
151 |153 |155 |Basic Idea. Use several permutations, plaintext encrypted to one of several possible cipher letters.
154 |
Example: Vigenere Cipher.
156 |If the key is shorter than the ciphertext, the key should not be re-used.
166 |So c_1 - c_2 depends only on the plaintext and hence can leak information about the plaintext.
It is not useful unless the key is at least as long as the plaintext, so we should use a stream cipher.
Use a pseudorandom generator PRBG (Pseudo-Random Bit Generator).
175 |188 |190 |K[i], \overline{K}[i], S[i] are bytes.
189 |
Input: Secret key K[0], K[1], ..., K[d-1].
191 |Output: 256-long array: S[0], S[1], ..., S[255].
192 |for i in range(256):
193 | S[i] = i
194 | K2[i] = K[i % d]
195 | j = 0
196 | for i in range(256):
197 | j = (K2[i] + S[i] + j) % 256
198 | S[i], S[j] = S[j], S[i] Idea: S is a “random-looking” permutation of \{0, 1, ..., 255\} which is generated from the secret key.
200 |Input: 256-long byte array S[0], S[1], ..., S[255].
202 |Output: Keystream.
203 |i = 0, j = 0
204 | while keystream_bytes_are_required:
205 | i = (i + 1) % 256
206 | j = (S[i] + j) % 256
207 | S[i], S[j] = S[j], S[i]
208 | t = (S[i] + S[j]) % 256
209 | output(S[t])Encryption: Keystream bytes are xored with the plaintext bytes.
211 |Fluhrer-Mantin-Shamir attack exploits known biases in the first few bytes of the keystream. Defeated by discarding the first few bytes.
212 |Because of several new weaknesses, RC4 is no longer used in applications such as SSL. Instead, use AES-CTR.
213 |215 |217 |More attack opportunities, no physical access. Attackers can be at a distance and leave no physical evidence of attack.
216 |
Mobile stations share a key k with access points. Messages are divided into packets of some fixed length. WEP uses a per-packet 24-bit initialization vector (IV) v to process each packet. v is prepended to the key to generate the keystream.
238 |Question: Are confidentiality, data integrity, and access control achieved?
239 |Since there are only 2^{24} choices for IV, collisions are guaranteed after enough time. If they are selected randomly, then one can expect a collision after 2^{12} packets. This allows attackers to obtain multiple messages with the same v, so they can obtain m_1 \oplus m_2.
240 |It is also easy to make controlled changes to encrypted packets without invalidating the checksum.
241 |Knowledge of one plaintext m corresponding to (v, c) allows an attacker to compute a valid encrypted packet for any plaintext with the same keystream.
248 |284 |286 |Take something fairly simple and repeat.
285 |
No restrictions are needed to be placed on f for the encryption to work.
299 |Component function f: \{0, 1\}^{64} \to \{0, 1\}^{64} is complicated.
316 |327 |329 |Complicated, but is it secure?
328 |
338 |340 |Given an encryption oracle with respect to Alice’s secret key, we want to find k.
339 |
Observation: Let T denote one round of encryption. Let F represent the full encryption function.
341 |T((m_{i-1}, m_i)) = (m_i, m_{i-1}\oplus f(m_i)) F((m_0, m_1)) = (m_16, m_17)
342 |So F = T^{16}. So T(F(m)) = T(T^{16}(m)) = T^{16}(m)T(m) = F(T(m)). \mathbf{F, T} commute.
343 |Then T(F(u)) = T((a, b)) = (b, ?). Since F, T commute, we have F(T(u)) = (b, ?).
358 |Let’s make a guess for k(r) = t \in \{0, 1\}^8. We apply one round of encryption to u assuming that k(r) = t. Call the result T_t(u). Obtain the encryption of T_t(u) from Alice, F(T_t(u)) = (c, d).
359 |Now, if indeed k(r) = t, then T_t(u) = T(u), so F(T_t(u)) = F(T(u)). So b = c. On the other hand, if k(r) \neq t, then we claim T_t(u) \neq T(u) by the second constraint on the input.
360 |361 |363 |Assuming now that F behaves like a random permutation (heuristic assumption). Then we can assume that F(T_t(u)) is a random bit string of length 128. In particular, the probability that c = b is roughly \frac{1}{2^{64}} which is negligible. So if b \neq c then k(r) \neq t.
362 |
Expected Chosen-Plaintexts: 256(1 + 128) \approx 32000.
364 |Component function f_i: \{0, 1\}^{32} \to \{0, 1\}^{32}.
366 |Question: How can one construct a more secure block cipher from DES?
370 |Multiple Encryption. Re-encrypt the ciphertext one or more times using independent keys. Hope that this increases effective key length.
371 |k = (k_1, k_2) \in_R\{0, 1\}^{56}.
373 |Encryption. c = E_{k_2}(E_{k_1}(m)).
374 |Double-DES key length is l=112, so exhaustive key search is infeasible.
375 |E^{-1}_{k_2}(c) = E_{k_1}(m) which leads to a meet-in-the-middle attack.
376 |378 |380 |How should we encrypt blocks?
379 |
389 |391 |Widely used today.
390 |
Requirements.
392 |400 |402 |Takes in a message of arbitrary length and outputs fixed length. H is fixed and public.
401 |
To show that a hash property is violated, you need to design efficient algorithms for.
419 |442 |444 |Decentralized storage system.
443 |
Given D_1, D_2, ..., D_t, find a 256-bit string s such that \text{SHA256}(D_1, D_2, ..., D_t, s) begins with 74 zeros. So selecting arbitrary s is expected 2^{74}. Then s is the proof of work.
451 | 452 | 453 | --------------------------------------------------------------------------------