├── .gitignore ├── CS271 ├── CS271.pdf ├── CS271.tex └── complexity.jpg ├── CS70 ├── calc_review │ ├── boundary.png │ ├── calc_review.pdf │ ├── calc_review.tex │ ├── calc_review.toc │ ├── def_integral.png │ ├── playground.nb │ ├── x2+y2.png │ ├── x2y2.jpeg │ ├── ye-x.jpeg │ └── ye-x.png ├── clt │ ├── clt.pdf │ └── clt.tex ├── hard_counting │ ├── hard_counting.pdf │ ├── hard_counting.tex │ ├── hard_counting_sols.pdf │ └── hard_counting_sols.tex ├── recap0a │ ├── recap0.tex │ ├── recap0a.pdf │ └── recap0a.tex ├── recap0b │ ├── recap0b.pdf │ ├── recap0b.tex │ └── recap1a.tex ├── recap1a │ ├── recap1a.pdf │ ├── recap1a.tex │ ├── recap1b.pdf │ └── recap1b.tex ├── recap1b │ ├── recap1b.tex │ └── recap2a.tex ├── recap2b │ └── recap2b.tex ├── recap3a │ ├── recap3a.pdf │ └── recap3a.tex ├── recap3b │ ├── function-mapping.png │ ├── injsurj.svg │ ├── recap3b.pdf │ └── recap3b.tex ├── recap4a │ ├── recap4a.pdf │ └── recap4a.tex └── recap4b │ ├── recap4b.pdf │ └── recap4b.tex ├── EECS126 ├── BEC.png ├── EECS126.pdf ├── EECS126.tex ├── LLN.png ├── MMSE.png ├── conv_u2.png ├── hmm.png ├── recurrent_transient.png ├── shannon_paper_fig.png ├── tourn.png ├── trellis.jpg └── trellis_complete.jpg ├── EECS127 ├── EECS127.pdf ├── EECS127.tex ├── README.md ├── box_uncertainty.png ├── convexnonconvex.png ├── hp0.png ├── hpb.png ├── kernel.png ├── lowrankappx.png ├── pca.jpg ├── pw_constant.png ├── qfconvex.jpeg ├── qfconvex.png ├── robust_lp_box.png ├── robust_lp_spherical.png ├── scenario_uncertainty.png ├── spherical_uncertainty.png └── toyLP.png ├── LICENSE ├── Math250A ├── math250a.pdf └── math250a.tex ├── README.md ├── coursenotes.code-workspace ├── templates ├── hw-template.pdf ├── hw-template.synctex.gz ├── hw-template.tex ├── notes-template.pdf ├── notes-template.synctex.gz └── notes-template.tex └── tyler.sty /.gitignore: -------------------------------------------------------------------------------- 1 | *.aux 2 | *.fdb_latexmk 3 | *.fls 4 | *.log 5 | *.out 6 | *.toc 7 | **/.aux 8 | **/.bbl 9 | **/.bcf 10 | **/.blg 11 | **/.fdb_latexmk 12 | **/.fls 13 | **/.log 14 | **/.out 15 | **/.run.xml 16 | **/.toc 17 | **/.swp 18 | -------------------------------------------------------------------------------- /CS271/CS271.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tyleryzhu/course-notes/4a08721614b5d4ef2b749a028083514093b7a82c/CS271/CS271.pdf -------------------------------------------------------------------------------- /CS271/complexity.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tyleryzhu/course-notes/4a08721614b5d4ef2b749a028083514093b7a82c/CS271/complexity.jpg -------------------------------------------------------------------------------- /CS70/calc_review/boundary.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tyleryzhu/course-notes/4a08721614b5d4ef2b749a028083514093b7a82c/CS70/calc_review/boundary.png -------------------------------------------------------------------------------- /CS70/calc_review/calc_review.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tyleryzhu/course-notes/4a08721614b5d4ef2b749a028083514093b7a82c/CS70/calc_review/calc_review.pdf -------------------------------------------------------------------------------- /CS70/calc_review/calc_review.tex: -------------------------------------------------------------------------------- 1 | \documentclass[11 pt]{scrartcl} 2 | \usepackage[header, margin, koma]{tyler} 3 | 4 | \newcommand{\hwtitle}{Calculus and Probability Review} 5 | 6 | \pagestyle{fancy} 7 | \fancyhf{} 8 | \fancyhead[l]{\hwtitle{}} 9 | \fancyhead[r]{CS 70 Staff} 10 | \cfoot{\thepage} 11 | 12 | % TODO: add a joint pdf example to the double int. section 13 | 14 | \begin{document} 15 | \title{\Large \hwtitle{}} 16 | \author{\large CS 70 Staff} 17 | \date{\large\today} 18 | 19 | \maketitle 20 | 21 | This is a quick refresher of single and multi variable calculus in the context of probability. If any material needs refreshing, \cite{stewart} is the canonical calculus textbook, while Khan Academy has fantastic video explanations. Finally, this is a reminder that \textbf{calculus will not be over-emphasized on the final}; after all, this is a probability course, not a calculus course. 22 | %\tableofcontents 23 | 24 | \section{Derivatives} 25 | Single variable calculus starts with the notion of a derivative, which represents the \emph{instantaneous} rate of change of a function $f(x)$. Formally, 26 | 27 | \begin{definition}[Derivative] 28 | The \textbf{derivative} of a function $f(x)$ is defined to be 29 | \[ f'(x) = \lim_{h\to 0} \dfrac{f(x+h)-f(x)}{h}.\] 30 | We sometimes also denote it by $\frac{df}{dx}$ or $\frac{d}{dx} f(x)$ to emphasize that the derivative is with respect to $x$. 31 | \end{definition} 32 | 33 | The idea is that this value gives us the slope of the secant line passing through $(x, f(x))$ and $(x+h, f(x+h))$, and as $h$ becomes smaller and smaller, the secant line becomes the tangent line at the point $x$. 34 | 35 | This definition is seldomly used once we know the basics of dealing with derivatives though. That comes in the form of the following common forms and properties. Assume $c\in \RR$ is a constant. 36 | 37 | \begin{itemize} 38 | \ii $\frac{d}{dx} c = 0$. 39 | \ii $\frac{d}{dx} x^n = nx^{n-1}$. 40 | \ii $\frac{d}{dx} \sin x = \cos x$. 41 | \ii $\frac{d}{dx} \cos x = -\sin x$. 42 | \ii $\frac{d}{dx} e^x = e^x$ (the only such function!). 43 | \ii $\frac{d}{dx} c^x = c^x\ln c$. 44 | \ii $\frac{d}{dx} \ln x = \frac{1}{x}$. 45 | \end{itemize} 46 | 47 | Of course, derivatives add and subtract, and with the next two rules, you're ready to differentiate nearly any function you come across. 48 | 49 | \begin{lemma}[Product Rule] 50 | For any two functions $f(x)$ and $g(x)$, 51 | \[ \dfrac{d}{dx} (f\cdot g)(x) = f'(x) g(x) + f(x)g'(x) \] 52 | or in other words, 53 | \[ (fg)' = f'g + fg'.\] 54 | \end{lemma} 55 | 56 | \begin{lemma}[Chain Rule] 57 | Suppose I have a function $f(x) = g(h(x))$. Then 58 | \[ f'(x) = g'(h(x))\cdot h'(x).\] 59 | Equivalently, 60 | \[ \dfrac{df}{dx} = \dfrac{df}{dh} \dfrac{dh}{dx}.\] 61 | \end{lemma} 62 | 63 | In other words, I keep unrolling my derivatives from outside in when I use the chain rule. 64 | 65 | \begin{example} 66 | Compute the derivative of $f(x) = x^2e^{2x}$. 67 | \end{example} 68 | \begin{proof}[Solution] 69 | Applying the product rule followed by the chain rule gives 70 | \[ f'(x) = (x^2)'e^{2x} + x^2 (e^{2x})' = 2xe^{2x} + x^2 e^{2x} (2x)' = 2xe^{2x} + 2x^2e^{2x}.\] 71 | \end{proof} 72 | 73 | \begin{exercise} 74 | Find the derivatives of the following functions: 75 | \alphanum 76 | \ii $F(x) = 5(5-x)^4$ 77 | \ii $F(x) = e^{-3x}$ 78 | \ii $F(x) = \sqrt{x^3}$ 79 | \enumend 80 | \end{exercise} 81 | 82 | \section{Integrals} 83 | \subsection{Definitions} 84 | Complementing the derivative is first the \emph{definite} integral, which represents the signed\footnote{Meaning positive if we're above the $x$-axis and negative otherwise.} area under the curve. The nice interpretations and motivations for this come from physics, so I'll move on and give an informal definition. 85 | 86 | \begin{definition}[Definite Integral] 87 | The \textbf{definite integral} $\int_a^b f(x) dx$ is the signed area of the region of the $xy$-plane bounded by the graph of $f$, the $x$-axis, and the vertical lines $x = a$, $x = b$. 88 | \end{definition} 89 | 90 | \begin{figure}[!htb] 91 | \centering 92 | \includegraphics[scale=0.5]{def_integral.png} 93 | \caption{An example of such an integral.} 94 | \end{figure} 95 | 96 | It turns out that integration and differentiation are inverse operations. So we have the notion of an \emph{antiderivative}, a function $F$ whose derivative is a given function $f$, which is also sometimes called an \emph{indefinite} integral. 97 | 98 | \begin{definition}[Indefinite Integral] 99 | An antiderivative $F(x)$ of $f(x)$ is called an \textbf{indefinite integral}, and written as 100 | \[ F(x) = \int f(x) dx.\] 101 | \end{definition} 102 | 103 | One important point to note is that since constants vanish under differentiation, given an antiderivative $F(x)$, any other function $F(x) + c$ where $c$ is a constant is also an antiderivative. Never forget to plus $c$! 104 | 105 | In the midst of all this back and forth, it's still not clear at all how we'd compute the area under the curve in order to actually find the definite integral. This is where the Fundamental Theorem of Calculus comes into play; it creates the connection between the definite and indefinite integrals (and therefore between definite integrals and derivatives). 106 | 107 | \begin{theorem}[Fundamental Theorem of Calculus] 108 | If $f$ is a real-valued, integrable function on $[a,b]$, and $F$ is an antiderivative of $f$ on $[a,b]$ (so that $F' = f$), then 109 | \[ \int_a^b f(x) dx = F(b) - F(a).\] 110 | \end{theorem} 111 | 112 | This theorem is paramount for going back and forth between integrals and derivatives. If you think about it, this theorem is what lets us use CDFs to evaluate our pdfs. So under the surface, problems like Example~\ref{example:ft_rv} are really using the fundamental theorem somewhere. 113 | 114 | 115 | Here are some common integrals you should know: 116 | \begin{itemize} 117 | \ii $\int x^n\; dx = \frac{x^{n+1}}{n+1} + C$ for $n \not= -1$. 118 | \ii $\int \frac{1}{x}\; dx = \ln |x| + C$ 119 | \ii $\int e^x\; dx = e^x + C$ 120 | \ii $\int \sin x\; dx = - \cos x + C$ 121 | \ii $\int \cos x\; dx = \sin x + C$ 122 | \end{itemize} 123 | 124 | \begin{example} 125 | Compute $\int_0^4 x^2 dx$. 126 | \end{example} 127 | \begin{proof}[Solution] 128 | By inspection (or by looking at the list above) we find that an antiderivative is $F(x) = \frac{x^3}{3}$, and so by the fundamental theorem, our answer is just $F(4) - F(0)$. Written more concisely, 129 | \[ \int_0^4 x^2 dx = \dfrac{x^3}{3}\Big\rvert^4_0 = \dfrac{4^3}{3} - \dfrac{0}{3} = \dfrac{64}{3}.\] 130 | \end{proof} 131 | 132 | \subsection{Integrating techniques: $u$-sub, integration by parts, and others} 133 | There's two common techniques you'll use when solving integrals. The first is pretty simple. It's utilizing a change of variables, otherwise known as $u$-substitution. 134 | 135 | \begin{example} 136 | Compute $\int_0^5 e^{-6x}dx$. 137 | \end{example} 138 | \begin{proof}[Solution] 139 | This isn't an integral of one of the above forms since we have extra coefficients in the exponent, so we'll perform a change of variables to get it to look like it. Let $u = -6x$, so $du = -6 dx$ or $-\frac 16 du = dx$. Now if we change variables, we will get an antiderivative of 140 | \[\int e^{-6x} dx = \int -\frac 16 e^{u} du = -\frac 16 e^u + C = -\frac 16 e^{-6x} + C.\] 141 | 142 | Plugging this in gives 143 | \[ \int_0^5 e^{-6x} dx = -\frac 16 e^{-6x} \Big\rvert^5_0 = -\frac 16 e^{-30} + \frac 16 .\] 144 | \end{proof} 145 | 146 | The second technique is \emph{integration by parts}, which is not as straightforward. The idea is that we want to be able to integrate products of functions that aren't as nice. This reminds us of the product rule for derivatives, which states that if I had two functions $u(x)$ and $v(x)$, then 147 | \[ (uv)' = u'v + uv'.\] 148 | 149 | Integrating both sides with respect to $x$ gives 150 | \[ u(x) v(x) = \int (u(x) v(x))' dx = \int u'(x) v(x) dx + \int u(x) v'(x) dx \] 151 | 152 | If we use differentials and let $du = u'(x) dx$ and $dv = v'(x) dx$,\footnote{A differential is change in the function based on the dependent variable (how far up we move based on horizontal movement and the slope).} we can write this in a format thats more recognizable, namely 153 | \[ u(x) v(x) = \int v(x) du + \int u(x) dv \implies \boxed{\int u\; dv = uv - \int v\; du}.\] 154 | 155 | So now we can take an integral that's hard to compute, and flip around what we're integrating to make it potentially easier to simplify. It's easiest to understand through an example. 156 | 157 | \begin{example} 158 | Compute $\int xe^{3x}\; dx$. 159 | \end{example} 160 | \begin{proof}[Solution] 161 | If I'm applying integration by parts, I need to find one expression to take the derivative of (which will be $u$) and another to take the integral of (which will be $dv$) to simplify. Here, I'll let $u = x$ and $dv = e^{3x} dx$, so that $du = dx$ and $v = \frac 13 e^{3x}$. Then applying integration by parts gives 162 | \[ \int xe^{3x} = \int u\; dv = uv - \int v\; du = \frac 13 xe^{3x} - \int \frac 13 e^{3x} dx = \frac 13 xe^{3x} - \frac 19 e^{3x} + C.\] 163 | \end{proof} 164 | 165 | It's not always obvious what I should be setting $u$ and $dv$ to typically. A good rule of thumb is to set $u$ to the first term you see in this list: 166 | \begin{itemize} 167 | \ii Logarithm 168 | \ii Inverse trig function 169 | \ii Algebraic function (e.x. polynomials) 170 | \ii Trig function 171 | \ii Exponential function 172 | \end{itemize} 173 | 174 | Here's another example. 175 | \begin{example} 176 | Compute $\int \ln x \; dx$. 177 | \end{example} 178 | \begin{proof}[Solution] 179 | You might think that we can't apply integration by parts here since we only have one term. But in fact, let $u = \ln x$ and $dv = dx$ so that $du = \frac{1}{x} dx$ and $v = x$. Then 180 | \[ \int \ln x\; dx = \int u \; dv = uv - \int v\; du = x\ln x - \int x\cdot \frac{1}{x} \; dx = x\ln x - x + C.\] 181 | \end{proof} 182 | 183 | You should also search for the tabular method if you want to speed things up when doing multiple integrations by parts (Khan academy is a good place to watch). 184 | 185 | \begin{exercise} 186 | Compute $\int x^2e^{3x}\; dx$. 187 | \end{exercise} 188 | 189 | \section{Series} 190 | A quick informal definition so everyone's on the same page here. 191 | 192 | \begin{definition}[Series] 193 | Given some sequence of terms $(a_1, a_2, \dots, a_n)$, the \textbf{series} corresponding to this sequence is the sum 194 | \[ a_1 + a_2 + \dots + a_n.\] 195 | If the sequence is infinite, then the series will contain infinitely many summands. 196 | \end{definition} 197 | 198 | Series come up often in discrete probability when we ask what the probability of some range of events is. So it's useful to know how to sum the two main types of series and some tricks we can do with them. 199 | 200 | There's two main series to know: the geometric series, and the Taylor series. 201 | 202 | In a geometric series, we have an initial term $a$, and every term after that is some common ratio, $r$, times the previous terms. So a typical finite geometric series looks something like 203 | 204 | \[ S = a + ar + ar^2 + \dots + ar^k\] 205 | 206 | where $S$ is the final value of the series. To calculate this, notice that 207 | 208 | \[ rS = ar + ar^2 + \dots + ar^k + ar^{k+1},\] 209 | 210 | so if we subtract one from the other, all the middle terms cancel out, leaving us with 211 | 212 | \[ S - rS = a - ar^{k+1} \implies \boxed{S = \dfrac{a(1-r^{k+1})}{1-r}}.\] 213 | 214 | 215 | If we take $k \to \infty$, then we get an infinite geometric series, for which 216 | \[ \sum_{i=0}^\infty ar^i = \dfrac{a}{1-r}\] 217 | 218 | but only when $|r| < 1$ (otherwise the series won't converge). Here's an example that utilizes geometric series for a very fitting distribution. 219 | \begin{example} 220 | Show that the sum of the probabilities of a random variable $X \sim \Geo(p)$ is 1. 221 | \end{example} 222 | \begin{proof}[Solution] 223 | Recall that $\PP(X = k) = (1-p)^{k-1} p$. Then 224 | \[ \sum_{k=1}^\infty \PP(X=k) = \sum_{k=1}^\infty (1-p)^{k-1} p.\] 225 | But this is a \emph{geometric} series with initial term $p$ and ratio $1-p$, so the sum is $\frac{p}{1-(1-p)} = 1$. 226 | \end{proof} 227 | 228 | The idea of a Taylor series is to approximate a function $f(x)$ by its derivatives. This looks like 229 | 230 | \[ f(x) = f(0) + f'(0)x + \dfrac{1}{2!} f''(0) x^2 + \dfrac{1}{3!} f'''(0)x^3 + \dots = \sum_{k=0}^\infty \dfrac{1}{k!}f^{(k)}(0)x^k\] 231 | 232 | where $f^{(k)}$ is the $k$th derivative of $f$. The only one you'll need for this class is $e^x$: 233 | 234 | \[ e^x = 1 + x + \dfrac{x^2}{2!} + \dfrac{x^3}{3!} + \dots = \sum_{k=0}^\infty \dfrac{x^k}{k!}.\] 235 | 236 | One last interesting note is that geometric series are \emph{also} Taylor series. If we let $f(r) = \frac{a}{1-r}$, then the derivatives are 237 | 238 | \[ f(r) = \frac{a}{1-r}, \quad\quad f'(r) = \frac{a}{(1-r)^2}, \quad\quad f''(r) = 2!\cdot\frac{a}{(1-r)^3}, \quad\dots,\quad f^{(k)}(r) = k!\cdot \dfrac{a}{(1-r)^{k+1}} \] 239 | 240 | so 241 | 242 | \[ f(0) = a, \quad\quad f'(0) = 1!\cdot a, \quad\quad f''(0) = 2!\cdot a, \quad\dots,\quad f^{(k)}(0) = k!\cdot a.\] 243 | 244 | Plugging this into the Taylor series expansion gives 245 | 246 | \[ f(r) = \dfrac{a}{1-r} = a + ar + ar^2 + \dots\] 247 | 248 | which is the geometric distribution as expected. 249 | 250 | \begin{exercise} 251 | Show that the sum of the probabilities of a Poisson random variable $X\sim \text{Poisson}(\lambda)$ is 1. 252 | \end{exercise} 253 | 254 | \section{Intermission: Calculus and Series in Probability} 255 | Now that we've covered some calculus, let's look at how the different results we've seen so far are used in probability. 256 | 257 | Here's two problems that make use of the Taylor series for $e^x$. 258 | \begin{example}[Dis 12 \#3] 259 | Let $X \sim \Geo(p)$ and $Y \sim \Poisson(\lambda)$ be independent random variables. Compute $\PP(X > Y)$. 260 | \end{example} 261 | \begin{proof}[Solution] 262 | Let's condition on $Y$ so that we can use $\PP(X > k) = (1-p)^k$. This gives 263 | \begin{align*} 264 | \PP(X > Y) &= \sum_{y=0}^\infty \PP(Y=y) \PP(X > y | Y = y) \\ 265 | &= \sum_{y=0}^\infty \dfrac{e^{-\la}\la^y}{y!}\cdot (1-p)^y \\ 266 | &= e^{-\la}\sum_{y=0}^\infty \dfrac{(\la(1-p))^y}{y!} 267 | \end{align*} 268 | But this sum is the Taylor series for $e^{\la(1-p)}$, so the probability is just $e^{-\la}e^{\la-\la p} = e^{-\la p}$. 269 | \end{proof} 270 | 271 | This somewhat surprising result was derived in Lecture 21. 272 | \begin{example} 273 | Let $X_n$ be a random variable counting the number of fixed points of a permutation on $n$ variables. What is the distribution of $X_n$ as $n\to\infty$? 274 | \end{example} 275 | \begin{proof}[Solution] 276 | We wish to compute $\PP(X_n = j)$, so we count the number of permutations with $j$ fixed points. We need to choose $j$ points to be fixed and the rest to not have any fixed points, which means they form a derangment, so there are $\binom{n}{j}D_{n-j}$ such permutations. Hence 277 | \[ \PP(X_n = j) = \dfrac{\binom{n}{j} D_{n-j}}{n!}.\] 278 | 279 | Recalling that $D_j = j!\sum_{k=0}^j \frac{(-1)^k}{k!}$, we find 280 | \begin{align*} 281 | \PP(X_n = j) &= \dfrac{1}{n!}\left[ \dfrac{n!}{j!(n-j)!} (n-j)! \sum_{k=0}^{n-j}\dfrac{(-1)^k}{k!}\right] \\ 282 | &= \dfrac{1}{j!} \sum_{k=0}^{n-j}\dfrac{(-1)^k}{k!}. 283 | \end{align*} 284 | But when $n\to\infty$, this is the Taylor Series for $e^{-1}$, so 285 | \[ \lim_{n\to\infty} \PP(X_n = j) = \frac{e^{-1} 1^j }{j!} \] 286 | which is the pdf of a $\Poisson(1)$ distribution. 287 | \end{proof} 288 | This result also agrees with the fact that $\EE[X_n] = 1$, which is true by linearity of expectation. 289 | 290 | Here's an idea similar to Discussion 13, Problem 2 but in a different context. Under the hood, you should realize that we can do this cdf manipulation thanks to the Fundamental Theorem of Calculus. 291 | \begin{example} 292 | Let $X, Y$ be a random variables where $Y = X^2$. Express the pdf of $Y$, $f_Y$, in terms of the pdf of $X$, $f_X$. 293 | \label{example:ft_rv} 294 | \end{example} 295 | \begin{proof}[Solution] 296 | The first step should usually be to compute the cdf of $Y$ in terms of $X$'s cdf. This gives 297 | \[ F_Y(x) = \PP(Y \leq x) = \PP(X^2 \leq x) = \PP(-\sqrt{x} \leq X \leq \sqrt{x}) = F_X(\sqrt{x}) - F_X(-\sqrt{x}).\] 298 | From here, we differentiate with respect to $X$ (being careful to use the chain rule) and get 299 | \[ f_Y(x) = \dfrac{dF_Y}{dx} = \dfrac{1}{2\sqrt{x}}F_X'(\sqrt{x}) - \left(-\dfrac{1}{2\sqrt{x}}\right)F_X'(-\sqrt{x}) 300 | = \dfrac{1}{2\sqrt{x}} (f_X(\sqrt{x})+f_X(-\sqrt{x})) \] 301 | 302 | \end{proof} 303 | 304 | \section{Double Integrals} 305 | Now we're finally talking about multivariable calculus. Before we start, we should ask the question: what even is a double integral? 306 | 307 | We should first think about what a single integral represented. In summary, we defined it as 308 | \[ \boxed{\text{Finding an \textcolor{blue}{area} under } f(x) \text{ over a \textcolor{blue}{one-dim. interval} } {\color{blue} [a,b]} \text{ on the } {\color{blue} x \text{-axis: }} A = \int_a^b f(x) \; dx.}\] 309 | 310 | The double integral will be the natural generalization of this. Instead of integrating an area over an interval, we will be integrating a volume over a region in 2D space. 311 | \[ \boxed{\text{Finding a \textcolor{red}{volume} under } f(x,y) \text{ over a \textcolor{red}{two-dim. region} } {\color{red} \mathcal{R}} \text{ on the } {\color{red} xy \text{-plane: }} V = \iint_\mathcal{R} f(x,y) \; dA.}\] 312 | 313 | The hardest part of solving double integrals is often figuring out the region of integration and setting up the limits. After that, it's just single-variable calculus. Let's look at an example. 314 | 315 | \begin{example} 316 | Integrate $f(x,y) = x^2 + y^2$ over the $4\times 4$ square centered at $(0,0)$. 317 | \end{example} 318 | \begin{proof}[Solution] 319 | Before we begin, take a look at this visualization of our integral. 320 | 321 | \begin{figure}[!htb] 322 | \centering 323 | \includegraphics[scale=0.5]{x2y2.jpeg} 324 | \caption{A visualization of the integral of $f(x,y) = x^2 + y^2$ over the square $[-2,2]\times [-2,2]$.} 325 | \end{figure} 326 | 327 | As you can tell, we're looking for the volume under the curve. Our region of integration is the region bounded by the lines $x = -2, x = 2, y = -2, y = 2$. 328 | 329 | There's two ways in which we can integrate over this region: we can either integrate with respect to $x$ first, or with respect to $y$ first. Either way is the same in this case since everything is symmetric, so I'll set it up as the former. 330 | This means our integral is 331 | \[ \int_{-2}^2 \int_{-2}^2 x^2+y^2 \; dx \; dy.\] 332 | 333 | In order to integrate this, we treat the inside integral like a single integral and treat everything that's not $x$ like a constant. This gives 334 | 335 | \[ \int_{-2}^2 \int_{-2}^2 x^2+y^2 \; dx \; dy = \int_{-2}^2 \left(\frac{x^3}{3}+xy^2\right)\Big\rvert^2_{x=-2} \; dy = \int_{-2}^2 \frac{16}{3} + 4y^2 \; dy\] 336 | 337 | Now that we've gotten rid of our $x$ variable (as expected), all that's left is to finish the single integral, which gives us 338 | 339 | \[ \int_{-2}^2 \frac{16}{3} + 4y^2 \; dy = \left(\frac{16}{3}y + \frac{4}{3} y^3\right)\Big\rvert^{2}_{-2} = \frac{128}{3}.\] 340 | \end{proof} 341 | 342 | Here's one where the region of integration is trickier. 343 | 344 | \begin{example} 345 | Integrate $ye^{-x}$ over the \emph{triangle} enclosed by $y = x$, $y = 3$, and $x = 0$. 346 | \label{ex:ye-x} 347 | \end{example} 348 | \begin{proof}[Solution] 349 | \begin{figure}[!htb] 350 | \centering 351 | \begin{minipage}{0.45\linewidth} 352 | \includegraphics[scale=0.35]{boundary.png} 353 | \end{minipage}% 354 | \begin{minipage}{0.45\linewidth} 355 | \includegraphics[scale=0.45]{ye-x.png} 356 | \end{minipage}% 357 | \caption{The boundary and the integral of $f(x,y) = ye^{-x}$ over the region in Example~\ref{ex:ye-x}.} 358 | \end{figure} 359 | 360 | We already know the region is a triangle, but we need to figure out how to integrate over it. If we first integrate over $y$ then $x$, then the limits of our outer integral should go from $0$ to 3. In this case, we then have the variable $x$ to work with to set up our limits for $y$. Using the equations, we find that we should integrate from $x$ to $3$. Setting up our integral and simplifying gives 361 | 362 | \begin{align*} 363 | \int_0^3 \int_x^3 ye^{-x} \; dy \; dx &= \int_0^3 \frac 12 (y^2 e^{-x})\Big\rvert^3_{y = x} \; dx \\ 364 | &= \frac 12 \int_0^3 9e^{-x} - x^2e^{-x}\; dx 365 | \end{align*} 366 | 367 | which would require the use of \emph{two} integrations by parts to evaluate; yuck! Let's see what happens if we set up our integral the other way. The outer limits are still from 0 to 3, but the inner ones are now from 0 to $y$ (convince yourself of this), so the integral is 368 | \begin{align*} 369 | \int_0^3 \int_0^y ye^{-x} \; dx \; dy &= \int_0^3 (-y e^{-x})\Big\rvert^y_{x = 0} \; dy = \int_0^3 -y e^{-y} + y \; dy \\ 370 | &= \int_0^3 y \; dy - \left(-ye^{-y}\Big\rvert^3_0 + \int_0^3 e^{-y}\; dy \right) \\ 371 | &= \frac{9}{2} - (-3e^{-3} + 1 - e^{-3}) = \boxed{\frac{7}{2} + \frac{4}{e^3}}. 372 | \end{align*} 373 | \end{proof} 374 | 375 | We don't always have to integrate using $x$ and $y$ though. Sometimes it's more natural for us to use \emph{polar} coordinates where we label every point as $(r, \theta)$, $r$ being its distance from the origin and $\theta$ being its angle with the $x$-axis. 376 | 377 | \begin{example} 378 | Integrate $f(x,y) = x^2 + y^2$ over the circle of radius $2$ centered at $(0,0)$. 379 | \end{example} 380 | \begin{proof}[Solution] 381 | The first step is to translate our function in terms of $r$ and $\theta$. We know that $x^2 + y^2 = r^2$, so $f(r, \theta) = r^2$. Our region is $0\leq r \leq 2$, so our integral is 382 | 383 | \[ \int_{0}^{2\pi} \int_{0}^2 r^2 \; dr\; d\theta = \int_0^{2\pi} \frac{r^3}{3} \Big\rvert^2_{r=0} \; d\theta = \int_0^{2\pi} \frac{8}{3} \; d\theta = \dfrac{16}{3}\pi.\] 384 | \end{proof} 385 | We did something similar with Buffon's needle, where we integrated with respect to $y$ and $\theta$. All that matters is that we find the appropriate regions and limits to integrate over. 386 | 387 | \begin{exercise} 388 | Integrate $e^{x+y}$ over the rectangular region determined by the lines $y = 0$, $x = 0$, $y = 3$, $x = 2$. 389 | \end{exercise} 390 | \begin{exercise} 391 | Integrate $-\frac{xe^x}{y^2}$ over the region defined by the lines $x = 5$, $y = -x$, $y = x$. 392 | \end{exercise} 393 | \begin{exercise} 394 | Suppose $X \sim \Exp(\la_1), Y \sim \Exp(\la_2)$ are independent exponential distributions so that $f(x,y) = \la_1\la_2 e^{-(\la_1+\la_2)x}$ for $x, y \geq 0$ is the density of their joint distribution. 395 | Show that the integral of this density is 1. % over its support.\footnote{The \emph{support} a function $f$ are all inputs where its nonzero. For example, the support of a uniform random variable on $[a,b]$ is just $[a,b]$.} 396 | \end{exercise} 397 | 398 | %\section{Joint Densities of Continuous Random Variables} 399 | 400 | 401 | \begin{thebibliography}{12} 402 | \bibitem{stewart} 403 | James Stewart, \textit{Calculus 7 ed}. Cengage Learning, 2012. 404 | \end{thebibliography} 405 | 406 | 407 | 408 | 409 | 410 | \end{document} 411 | -------------------------------------------------------------------------------- /CS70/calc_review/calc_review.toc: -------------------------------------------------------------------------------- 1 | \contentsline {section}{\numberline {1}Derivatives}{1}{section.1} 2 | \contentsline {section}{\numberline {2}Integrals}{1}{section.2} 3 | \contentsline {section}{\numberline {3}Series}{1}{section.3} 4 | \contentsline {section}{\numberline {4}Intermission: Continuous Probability}{1}{section.4} 5 | \contentsline {section}{\numberline {5}Multivariate Derivatives}{1}{section.5} 6 | \contentsline {section}{\numberline {6}Double Integrals}{1}{section.6} 7 | \contentsline {section}{\numberline {7}Joint Densities of Continuous Random Variables}{1}{section.7} 8 | -------------------------------------------------------------------------------- /CS70/calc_review/def_integral.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tyleryzhu/course-notes/4a08721614b5d4ef2b749a028083514093b7a82c/CS70/calc_review/def_integral.png -------------------------------------------------------------------------------- /CS70/calc_review/x2+y2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tyleryzhu/course-notes/4a08721614b5d4ef2b749a028083514093b7a82c/CS70/calc_review/x2+y2.png -------------------------------------------------------------------------------- /CS70/calc_review/x2y2.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tyleryzhu/course-notes/4a08721614b5d4ef2b749a028083514093b7a82c/CS70/calc_review/x2y2.jpeg -------------------------------------------------------------------------------- /CS70/calc_review/ye-x.jpeg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tyleryzhu/course-notes/4a08721614b5d4ef2b749a028083514093b7a82c/CS70/calc_review/ye-x.jpeg -------------------------------------------------------------------------------- /CS70/calc_review/ye-x.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tyleryzhu/course-notes/4a08721614b5d4ef2b749a028083514093b7a82c/CS70/calc_review/ye-x.png -------------------------------------------------------------------------------- /CS70/clt/clt.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tyleryzhu/course-notes/4a08721614b5d4ef2b749a028083514093b7a82c/CS70/clt/clt.pdf -------------------------------------------------------------------------------- /CS70/clt/clt.tex: -------------------------------------------------------------------------------- 1 | \documentclass[11 pt]{scrartcl} 2 | \usepackage[header, margin, koma]{tyler} 3 | 4 | \newcommand{\hwtitle}{LLN and the Central Limit Theorem} 5 | 6 | \pagestyle{fancy} 7 | \fancyhf{} 8 | \fancyhead[l]{\hwtitle{}} 9 | \fancyhead[r]{Tyler Zhu} 10 | \cfoot{\thepage} 11 | 12 | \begin{document} 13 | \title{\Large \hwtitle{}} 14 | \author{\large Tyler Zhu} 15 | \date{\large\today} 16 | 17 | \maketitle 18 | 19 | The idea of this worksheet is to get you to the point where applying the Central Limit Theorem becomes natural and doesn't require memorizing a formula to apply it. If you actually try solving each problem on its own, this should do its job. 20 | 21 | \section{Warm-Up: Law of Large Numbers} 22 | 23 | \begin{problem} 24 | This problem will walk you through a proof of the Law of Large Numbers. The setup is as follows: we have $n$ i.i.d. random variables $X_1, \dots, X_n$ where $\EE[X_i] = \mu$ and $\Var(X_i) = \si^2$. Our intuition tells us that in the long run, we'd expect the average of these random variables to approach the true underlying mean. 25 | \alphanum 26 | \ii Create an unbiased estimator $M_n$ for the mean $\mu$. 27 | \ii What are $\EE[M_n]$ and $\Var(M_n)$? 28 | \ii Suppose I used Chebyshev's inequality to create a confidence interval $[\mu-\eps, \mu+\eps]$ for $M_n$. As $n\to\infty$, how confident am I that $M_n$ will be in this interval? How does this relate to the law of large numbers? 29 | \enumend 30 | \end{problem} 31 | 32 | So LLN tells us that their average will eventually be very close to the mean, but it doesn't tell me how confident I can be for a particular $n$. That's where CLT comes in. 33 | 34 | \section{Central Limit Theorem} 35 | First we'll see how we can derive the Central Limit Theorem to gain intuition for why it works, then I'll present it. 36 | 37 | \begin{problem} 38 | Your friend comes up to you and tells you about the new BLT theorem, which says that the sum of $n$ i.i.d. random variables is approximately Gaussian for large $n$, but forgot what the parameters of it looks like. It's your job to figure out exactly what they should be. Similar to the last problem, suppose that $X_1, \dots, X_n$ are i.i.d. where $\EE[X_i] = \mu$ and $\Var(X_i) = \si^2$. 39 | \alphanum 40 | \ii Let $S_n = X_1 + \dots + X_n$. What are $\EE[S_n]$ and $\Var(S_n)$? 41 | \ii Define a new r.v. $Z_n$ in terms of $S_n$ which has mean 0 and standard deviation 1. 42 | \ii The \emph{z-score} of a data point is how many standard deviations (+/-) it's away from the mean. Given some observation $x$ in the distribution $S_n$, what would it's z-score be? 43 | \ii Your friend's BLT theorem tells you that $Z_n\sim \Nor(0,1)$, i.e. it's approximately a standard normal Gaussian, in the following sense: 44 | \[ \lim_{n\to\infty} \PP[Z_n \leq z] = \Phi(z)\] 45 | where $\Phi(z)$ is the CDF of the standard normal Gaussian. Given this, what is $\PP[S_n \leq x]$ for some $x$? 46 | \enumend 47 | \end{problem} 48 | 49 | Here's the general theorem for reference. 50 | \begin{theorem}[Central Limit Theorem] 51 | Let $X_1, \dots, X_n$ be i.i.d. r.v.'s where $\EE[X_i] = \mu$ and $\Var(X_i) = \si^2$, and define $S_n = X_1 + \dots + X_n$. Then as $n\to\infty$, the CDF of $S_n$ approaches that of the CDF of a $\Nor(n\mu, n\si^2)$ distribution. In terms of z-scores, if $Z_n = \frac{S_n-n\mu}{\si\sqrt{n}}$, then 52 | \[ \lim_{n\to\infty} \PP[Z_n \leq z] = \Phi(z)\] 53 | where $\Phi(z)$ is the CDF of a standard normal Gaussian. 54 | \end{theorem} 55 | 56 | 57 | \section{Past Practice Problems} 58 | 59 | \begin{problem}[Sp19 Final \#7] 60 | Suppose the number of people that walk into Jonathan's favorite McDonald's in an hour is $\sim\Poisson(\la)$, where $\la$ is unknown but is definitely at most 10. How many hours does Jonathan need to be at McDonalds to be able to construct a 95\% confidence interval for $\la$ that is of width 2? 61 | \end{problem} 62 | 63 | \begin{problem}[Fa19 Final \#8] 64 | Let $X_i\sim \Poisson(1)$ be independent r.v.'s and $S_n = X_1 + \dots + X_n$ be their sum, and let $c,\eps$ be some constants. For $\eps < \frac 12$, what is $\lim_{n\to\infty}\PP[S_n < cn^\eps+n]$? 65 | \end{problem} 66 | 67 | \end{document} 68 | -------------------------------------------------------------------------------- /CS70/hard_counting/hard_counting.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tyleryzhu/course-notes/4a08721614b5d4ef2b749a028083514093b7a82c/CS70/hard_counting/hard_counting.pdf -------------------------------------------------------------------------------- /CS70/hard_counting/hard_counting.tex: -------------------------------------------------------------------------------- 1 | \documentclass[11 pt]{scrartcl} 2 | \usepackage[header, margin, koma]{tyler} 3 | 4 | \newcommand{\hwtitle}{Hard Counting Problems} 5 | 6 | \newif\ifproblemsol 7 | \problemsolfalse 8 | 9 | \pagestyle{fancy} 10 | \fancyhf{} 11 | \fancyhead[l]{\hwtitle{}} 12 | \fancyhead[r]{Tyler Zhu} 13 | \cfoot{\thepage} 14 | 15 | \begin{document} 16 | \title{\Large \hwtitle{}} 17 | \author{\large Tyler Zhu} 18 | \date{\large\today} 19 | 20 | \maketitle 21 | 22 | These are some pretty arbitrary, hard counting problems. They are more so for testing your general ability to see connections in counting well than how well you bookkeep. Safe travels. 23 | 24 | \section{Warm up} 25 | \begin{problem} 26 | How many cubic polynomials $f(x)$ with positive integer coefficients are there such that $f(1) = 9$? 27 | \end{problem} 28 | \ifproblemsol 29 | \begin{proof}[Solution] 30 | Let $f(x) = ax^3+bx^2+cx+d$. The only condition on $f(x)$ is that $f(1) = 9$, which means $a + b + c + d = 9$. Since $a,b,c,d$ are positive integers, there are $\binom{8}{3} = 56$ such polynomials. 31 | \end{proof} 32 | \fi 33 | 34 | \section{Assorted Candies} 35 | \begin{problem} 36 | Prove $\sum_{k=0}^n \binom{n}{k} 2^k = 3^n$ with a combinatorial argument. 37 | \end{problem} 38 | \ifproblemsol 39 | \begin{proof}[Solution] 40 | Suppose we're trying to create a $n$-bead string with 3 colors; red, blue and green. We can either have 3 chocies for each spot for a total of $3^n$, or pick the $n-k$ spots which will be red, and then fill in the remaining $k$ spots with either blue or green for a total of $\binom{n}{k}2^k$ for each choice of $k$. 41 | \end{proof} 42 | \fi 43 | 44 | \begin{problem} 45 | Let $a_1, a_2, \dots, a_n$ be a sequence of arbitrary natural numbers. Define $b_k$ to be the number of elements $a_i$ for which $a_i \geq k$. Prove that $a_1 + a_2 + \dots + a_n = b_1 + b_2 + \cdots$. 46 | \end{problem} 47 | \ifproblemsol 48 | \begin{proof}[Solution] 49 | The idea is to double count. Drawing a picture is the best way to see this: for each $a_i$, draw $a_i$ circles vertically. Then the LHS is counting the number of circles going vertically, while the RHS is counting them horizontally. 50 | 51 | Formally, one way to count is simply to sum the $a_i$'s. Another way to count uses the fact that they are natural numbers. All the $b_i$ start at 0. Then, for any given $a_i$, $b_1$ through $b_{a_i}$ will have all their values increased by $1$, increasing the total on the RHS by $a_i$. Hence the total contributions of all $a_i$ to the RHS is just $a_1 + \dots + a_n$. 52 | \end{proof} 53 | \fi 54 | 55 | \begin{problem} 56 | How many ways are there to insert $+$’s between the digits of 111111111111111 (fifteen 1’s) so that the result will be a multiple of 30? 57 | \end{problem} 58 | \ifproblemsol 59 | \begin{proof}[Solution] 60 | No matter how many $+$'s we insert, the result will always be a multiple of $3$ since there are fifteen 1's. For it to be a multiple of 10, we need exactly 10 numbers, which means we're adding 9 $+$'s. There are 14 gaps, so our answer is $\binom{14}{9}$. 61 | \end{proof} 62 | \fi 63 | 64 | 65 | \begin{problem} 66 | Compute 67 | \[ \sum_{n_{60}=0}^2\sum_{n_{59}=0}^{n_{60}}\dots \sum_{n_2=0}^{n_3} \sum_{n_1=0}^{n_2}\sum_{n_0=0}^{n_1} 1.\] 68 | \end{problem} 69 | \ifproblemsol 70 | \begin{proof}[Solution] 71 | Another way of phrasing the problem is to find the number of solutions to $0\leq n_0 \leq n_1 \leq n_2 \leq \dots \leq n_{60} \leq 2$. This corresponds to the number of right-up walks on a $61\times 2$ grid from the bottom left to the top right, which we all know to be $\binom{63}{2}$. 72 | 73 | Alternatively, notice that every solution is of the form $(0,\dots, 0, 1, \dots, 1, 2, \dots, 2)$, so we only need to specify the number of 0s, 1s, and 2s. This is equivalent to solving the equation $x+y+z = 61$ for nonnegative $x,y,z$, which (by stars and bars) has $\binom{63}{2}$ solutions. 74 | \end{proof} 75 | \fi 76 | 77 | \begin{problem} 78 | There’s a new (virtual) game show featuring $N$ people where a few lucky contestants get to compete for the ultimate prize: a roll of toilet paper. The game works as follows: everyone lines up in front of a jar of ping pong balls numbered 1 through 100 and one-by-one randomly select a ball until everyone has one. Then the winning number is announced, and anyone with the winning number wins the prize. 79 | \alphanum 80 | \ii Suppose the winning number is 42. What’s the probability that if $N=3$, then the third person wins the game? 81 | \ii If $N = 100$, what’s the probability that the third person wins the game now? 82 | \ii You and a friend are watching the game (for $N = 100$) and after all of the balls have been drawn, you both decide to bet on the results. Your friend picks contestant 42, thinking they must have the winning number, but before you can pick a contestant, 98 of them groan in realization that they have losing numbers, which leaves you with no choice but to bet on contestant 20. What’s the probability that you win the bet? 83 | \enumend 84 | \end{problem} 85 | \ifproblemsol 86 | \begin{proof}[Solution] 87 | \alphanum 88 | \ii By symmetry, $\frac{1}{100}$. One can also compute it out to be $\frac{99}{100}\cdot \frac{98}{99}\frac{1}{98} = \frac{1}{100}$. 89 | \ii As above, by symmetry, $\frac{1}{100}$. Computation also works. 90 | \ii This is just the Monty Hall problem in disguise, where you choose to switch! Once the other 98 doors (contestant's numbers) were revealed, all of the $\frac{99}{100}$ probability went into contest 20's chances, so you have a $\frac{99}{100}$ probability of winning the bet. 91 | \enumend 92 | \end{proof} 93 | \fi 94 | 95 | 96 | \section{Dessert} 97 | These aren't actually relevant to course material; please don't do them unless you really want to! 98 | 99 | \begin{exercise} 100 | Let $(a_1,a_2, \dots, a_{12})$ be a permutation of $(1,2,\dots,12)$ for which 101 | \[ a_1>a_2>a_3>a_4>a_5>a_6 \text{ and } a_6a_2>a_3>a_4>a_5>a_6 \text{ and } a_6=stealth', auto, semithick, node distance=2cm] 97 | \tikzstyle{every state}=[fill=white,draw=black,thick,text=black,scale=0.8] 98 | \node[state] (A) {$A$}; 99 | \node[state] (B)[below of=A] {$B$}; 100 | \node[state] (C)[right of=A] {$C$}; 101 | \node[state] (D)[right of=B] {$D$}; 102 | \path 103 | (A) edge (C) 104 | edge (D) 105 | (B) edge (C) 106 | edge (D); 107 | \end{tikzpicture} 108 | \end{center} 109 | \caption{Casting the problem in terms of graphs.} 110 | \end{figure} 111 | 112 | We also see that sitting people around a table so that they sit next to their friends is just asking to find a cycle of length at least $m+1$. Using the above graph, it's easy to find one such cycle, $ACBD$, by starting at $A$ and walking on the graph until you come back to $A$. 113 | 114 | So that's the easy part of the problem. Now let's see how we would create such a cycle. Naively, we might start at an arbitrary vertex, say $v_0$, and try to walk along this graph. We know $v_0$ has at least $m$ neighbors, so let's visit one of them, $v_1$. 115 | 116 | Now $v_1$ also has at least $m$ neighbors, but we've already been to one of them, $v_0$, so it really only has $m-1$ unvisited neighbors, so let's visit one of them, $v_2$. We can keep repeating this process of visiting unvisited neighbors until we're out of new neighbors to get a path 117 | 118 | \[ P = v_0v_1\dots v_l.\] 119 | 120 | One thing we should note is that in the worst case, the number of unvisited neighbors for each $v_i$ is at least $m-i$, which is if all of the $i$ vertices already in the path are neighbors of $v_i$. 121 | So $l \geq m$, or else we would still have neighbors we could visit. 122 | 123 | All that's left for us now is to get a cycle from this path. Since we're stuck when we reach $v_l$, all of its neighbors must be in $P$. So let's make a new path starting at $v_l$ by backpedaling from $v_l$ until we reach the last neighbor of $v_l$ in $P$, say $v_k$. Now we have a path 124 | \[ P' = v_l v_{l-1} \dots v_k\] 125 | where $P'$ is also at least $m+1$ since $v_l$ and all of its neighbors are in the path. But $v_k$ is neighbors with $v_l$, so this is also a cycle, and we're done. 126 | \end{proof} 127 | 128 | We used the principle of trying things until we get stuck, and trying to fix things until we're free. You'll find that this often works surprisingly well. 129 | 130 | The official solution is quite slick as well, but less motivated. 131 | It considers the \emph{longest} path in the graph, and constructs the cycle from that. 132 | This is an example of the \textbf{Extremal Principle}, where one looks at either the largest or the smallest object satisfying some property. 133 | 134 | \begin{proof}[Official Solution] 135 | Let $P=v_0v_1\dots v_l$ be a longest path in the graph. 136 | Such a path exists because the length of paths is bounded above by $n$. 137 | All neighbors of $v_0$ must be in $P$, since otherwise $P$ can be extended to be even longer by appending this edge at the beginning of path $P$. 138 | Let $k$ be the maximum index of neighbors of $v_0$ along $P$. 139 | Since $v_0$ has at least $m$ neighbors, we must have $k\geq m$. 140 | Then $v_0v_1\dots v_kv_0$ gives us the desired cycle. 141 | \end{proof} 142 | 143 | 144 | Here's a cute example of the Extremal Principle applied in another setting. Before you read on, think about it for a few minutes. 145 | 146 | \begin{problem} 147 | There are $n$ students standing in a field such that the distance between each pair is distinct. Each student is holding a ball, and when the teacher blows a whistle, each student throws their ball to the nearest student. Prove that there is a pair of students that throw their balls to each other. 148 | \end{problem} 149 | \begin{proof} 150 | Consider the two students closest to each other. Since they are each other's nearest student, they must throw their balls to each other. 151 | \end{proof} 152 | 153 | Try this one for another extremal flavored graph problem. How would you do it without looking at the extreme cases? 154 | \begin{exercise} 155 | Show that every tree $G$ contians at least two leaves, or vertices of degree 1. 156 | \end{exercise} 157 | 158 | 159 | \iffalse 160 | 161 | %%%%%%%%%%%%% SECTION %%%%%%%%%%%%5 162 | \section{Planarity Bounds} 163 | A graph $G$ is called a \textbf{planar graph} if $G$ can be drawn in the plane so that no two of its edges cross each other. If $G$ is planar, then it divides the plane into pieces called \textbf{regions}. Recall that for any planar, connected graph $G$, if $G$ has $V$ vertices, $E$ edges, and $F$ faces, Euler's formula tells us that 164 | \[ V-E+F = 2. \] 165 | 166 | You can imagine how hard it is to prove a graph to be non-planar: you can't possibly check every way of drawing the graph. Euler's Identity leads to some simple conditions on planarity. 167 | 168 | We can derive many useful bounds using this formula that involve only two of the three quantities, which is helpful especially for showing that certain graphs are nonplanar. Usually we omit faces since those are tricky to quantify, so let's try to compare the number of faces to the number of edges. 169 | 170 | \begin{figure}[!htb] 171 | \begin{center} 172 | \begin{tikzpicture}[>=stealth', auto, semithick, node distance=2cm] 173 | \tikzstyle{every state}=[fill=white,draw=black,thick,text=black,scale=0.8] 174 | \node[state] (A) {}; 175 | \node[state] (B)[right of=A] {}; 176 | \node[state] (C)[right of=B] {}; 177 | \node[state] (D)[below of=A] {}; 178 | \node[state] (E)[right of=D] {}; 179 | \path 180 | (A) edge node[below = 0.5cm] {$F_1$} (B) 181 | edge (D) 182 | (B) edge node[below = 0.6cm, left] {$F_2$} (C) 183 | edge (E) 184 | (C) edge node[below right = 0.5cm] {$F_3$} (E) 185 | (D) edge (E); 186 | \end{tikzpicture} 187 | \end{center} 188 | \caption{An example planar graph $G_1$ with its three faces marked.} 189 | \label{fig:graph} 190 | \end{figure} 191 | 192 | Let's look at the graph $G_1$ shown in Figure~\ref{fig:graph}. Notice that every edge is a part of exactly two faces. So we can count the number of edges also by looking at how many edges border each face. Let $|F_i|$ denote the number of edges that border the face $F_i$. Then, 193 | \begin{align*} 194 | 2E &= |F_1| + |F_2| + |F_3| \\ 195 | &= 4 + 3 + 5 \\ 196 | &\geq 3 + 3 + 3 \\ 197 | &= 3F 198 | \end{align*} 199 | where we used the very deep fact that every face is bordered by at least 3 edges (how else do you have a face?). Hence, we've arrived at the inequality $2E \geq 3F$. Now if we solve our formula for $F$ and substitute, we get the all important bound 200 | 201 | \[ 2 - V + E = F \leq \frac{2}{3} E \implies \boxed{E \leq 3V-6}.\] 202 | 203 | This is useful now as a method for showing a graph is nonplanar, as otherwise we'd have to draw every possible configuration of a graph and show all of them have crossings, which is neither feasible nor convincing. We can use this to show that $K_5$, the complete graph on 5 vertices, is nonplanar. Since it has $5$ vertices and $10$ edges, $10 \not\leq 3\cdot 5 - 6 = 9$. 204 | 205 | One extension of the above bound is to find a bound when $G$ is triangle-free, i.e. no face is bounded by 3 edges. I'll leave that as an exercise for you. Recall that the graph with six vertices, where three vertices are connected to all three other vertices, is the complete bipartite graph $K_{3,3}$. We can use this new bound to show $K_{3,3}$ is nonplanar. 206 | 207 | 208 | \begin{exercise} 209 | Show that if $G$ is a connected, planar, triangle-free graph, then $E \leq 2V - 4$. Use this to show that $K_{3,3}$ is nonplanar. 210 | \end{exercise} 211 | 212 | As a hint, we proved that all bipartite graphs have no odd tours. So what do we know about the presence of triangles in such a graph? 213 | 214 | \begin{figure}[!htb] 215 | \centering 216 | \begin{tikzpicture}[scale = 0.7] 217 | 218 | \begin{scope}[xshift=-6cm,yshift=-1cm] 219 | \foreach \x in {4,2,0} { 220 | \foreach \y in {0,2,4} { 221 | \drawLinewithBG{\x,0}{\y,2}; 222 | } 223 | } 224 | 225 | \foreach \x in {0,2,4} { 226 | \foreach \y in {0,2} { 227 | \node at (\x,\y) [circle,fill=black] {}; 228 | } 229 | } 230 | 231 | \node at (2,-1.5) {\Large$K_{3,3}$}; 232 | \end{scope} 233 | 234 | \begin{scope}[xshift=3cm] 235 | \foreach \a in { 18, 90, 162, 234, 306 } { 236 | \foreach \b in { 18, 90, 162, 234, 306 } { 237 | \drawPolarLinewithBG{\a:2}{\b:2}; 238 | } 239 | } 240 | 241 | \foreach \a in {18,90, 162, 234, 306 } { 242 | \node at (\a:2cm) [circle,fill=black] {}; 243 | } 244 | \node at (0,-2.5) {\Large$K_5$}; 245 | \end{scope} 246 | \end{tikzpicture} 247 | \caption{The Kuratowski graphs} 248 | \end{figure} 249 | 250 | We can already see how much trouble it can be to determine planarity for arbitrary graphs, let alone simple ones like $K_{3,3}$ and $K_5$. Surprisingly, the main enemies to planarity are precisely these two graphs, which leads to a simple check to see if a graph is planar, discovered in 1930. Before I can state Kuratowski's remarkable theorem, I need to state two notions. 251 | 252 | A \textbf{subdivision} of a graph is constructed by replacing one edge by two edges with a vertex in between, and a \textbf{subgraph} is constructed by removing vertices or edges. 253 | 254 | \begin{theorem}[Kuratowski] 255 | A graph $G$ is planar if and only if $G$ does not contain a subdivision of $K_5$ or $K_{3,3}$ as a subgraph. 256 | \end{theorem} 257 | 258 | In other words, $G$ is planar if it is not possible to subdivide the edges of $K_5$ or $K_{3,3}$, and then possibly add edges or vertices, to get $G$. 259 | 260 | \subsection{To Take Home} 261 | \begin{exercise}[Dis. Problem 2b] 262 | Consider graphs with the property $T$: For every three distinct vertices $v_1,v_2,v_3$ of graph $G$ , there are at least two edges among them. 263 | Prove that if $G$ is a graph on $\geq$ 7 vertices, and $G$ has property $T$, then $G$ is nonplanar. 264 | \end{exercise} 265 | Hint: Proof by contradiction when $v = 7$. What do we know about groups of five vertices in a planar graph? 266 | 267 | \section{Induction on Edges and Vertices} 268 | Problem 3 from the discussion sometimes gives people trouble, so I'll try to explain it some more in depth. Here is the problem, paraphrased. 269 | 270 | \begin{problem} 271 | An edge coloring of a graph is an assignment of colors to edges in a graph where any two edges incident to the same vertex have different colors. 272 | \alphanum 273 | \ii Prove that any graph with maximum degree $d\geq 1$ can be edge colored with $2d-1$ colors. 274 | \ii Show that any tree with maximum degree $d\geq 1$ can be edge colored with $d$ colors. 275 | \enumend 276 | \end{problem} 277 | 278 | The solution is to use induction on the number of edges in the first question, and induction on the number of vertices in the second. I'm not going to re-explain the solution (you can find it online) but I'll try to explain why it's fine to induct this way. 279 | 280 | Imagine if I had the following problem: 281 | \begin{question} 282 | Prove that any graph with maximum degree $4$ can be edge colored with $7$ colors. 283 | \end{question} 284 | 285 | Now induction, especially on the number of edges, seems like a viable approach. We're proving some statement about all graphs, so let's assume its true for a graph with $m$ edges and prove it for one with $m+1$ edges (both with max degree 4). What about this problem: 286 | 287 | \begin{question} 288 | Prove that any graph with maximum degree $10$ can be edge colored with $19$ colors. 289 | \end{question} 290 | 291 | See what I'm getting at? Just because we have a variable $d$ in our original statement doesn't mean we need to induct over $d$. We can treat $d$ as a constant and show the statement is true for all graphs given a specific $d$, which then shows the statement is true for all $d$. 292 | 293 | Another analogy: we don't need to induct over our variable $d$ much like how the number of vertices and edges are also variables, but we don't need to induct over both. 294 | 295 | \fi 296 | 297 | \end{document} 298 | -------------------------------------------------------------------------------- /CS70/recap3a/recap3a.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tyleryzhu/course-notes/4a08721614b5d4ef2b749a028083514093b7a82c/CS70/recap3a/recap3a.pdf -------------------------------------------------------------------------------- /CS70/recap3a/recap3a.tex: -------------------------------------------------------------------------------- 1 | \documentclass[11 pt]{scrartcl} 2 | \usepackage[header, margin, koma]{tyler} 3 | 4 | \newcommand{\hwtitle}{Discussion 3A Recap} 5 | 6 | \pagestyle{fancy} 7 | \fancyhf{} 8 | \fancyhead[l]{\hwtitle{}} 9 | \fancyhead[r]{Tyler Zhu} 10 | \cfoot{\thepage} 11 | 12 | \begin{document} 13 | \title{\Large \hwtitle{}} 14 | \author{\large Tyler Zhu} 15 | \date{\large\today} 16 | 17 | \maketitle 18 | 19 | \section{Modular Arithmetic} 20 | \begin{definition} 21 | We say that $a$ is \emph{congruent} to $b \pmod{m}$ if 22 | \[ a\equiv b \pmod{m} \iff m | a-b \iff a-b = m\cdot k, \quad k\in\ZZ .\] 23 | \end{definition} 24 | 25 | One interpretation is that mod $m$ gets the remainder when we divide by $m$, but the mod operator is more powerful than just that. For example, we have that 26 | \begin{align*} 27 | \dots \equiv -9 \equiv -4 \equiv \boxed{1} \equiv 6 \equiv 11 \equiv \dots \pmod{5} \\ 28 | \dots \equiv -8 \equiv -3 \equiv \boxed{2} \equiv 7 \equiv 12 \equiv \dots \pmod{5} \\ 29 | \dots \equiv -7 \equiv -2 \equiv \boxed{3} \equiv 8 \equiv 13 \equiv \dots \pmod{5} 30 | \end{align*} 31 | 32 | Given that so many numbers are equivalent to each other when working over a certain modulus, it helps us to agree upon a set of \emph{representatives} for each equivalence class of numbers. In the above example, the representatives for each class has been boxed. In general, the representatives are $\{0, 1, \dots, m-1\}$. 33 | 34 | To drive this point home, compare to how we say that all of the following fractions are the same, but we use the boxed one as their representative (namely $\frac 13$): 35 | \[ \dots = \dfrac{-3}{-9} = \dfrac{-2}{-6} = \dfrac{-1}{-3} = \boxed{\dfrac{1}{3}} = \dfrac{2}{6} = \dfrac{3}{9} = \dots \] 36 | 37 | Doing math over a modulus is similar to normal arithmetic; addition, subtraction, multiplication, and exponentiation all hold. 38 | 39 | Division is tricky however. Being able to divide by $a$ is equivalent to having an \emph{inverse}, i.e. a number $x$ which makes $ax \equiv 1 \pmod{m}$. The existence of an inverse is equivalent to having a solution $(x,k)$ over integers to the equation $ax = 1 + m\cdot k$. We saw in the notes (and you can reason why) that this happens only when $\gcd(a,m) = 1$, and is unique mod $m$. 40 | 41 | In general, it's a good question to ask when we have solutions to the equation 42 | \[ ax + by = c\] 43 | for $a,b,d \in \ZZ$. If we let $d = \gcd(a,b)$, then solutions exist only when $d | c$ (take this equation mod $d$ if you don't believe me). Additionally, mod $a$ or $b$ these solutions are unique. 44 | 45 | We even have an algorithm called the \emph{Extended Euclidean Algorithm} which helps us find solutions to $ax+by = d$, from which we can get solutions to any equation in the above form (the Euclidean Algorithm lets us compute $\gcd(a,b)$ efficiently; you can imagine why extending it lets us recover the above solutons). 46 | 47 | One common followup is finding the number of solutions to an equation like $10x \equiv 25 \pmod{30}$. Think about this (you may find the above context helpful). 48 | 49 | \section{Tips} 50 | \itemnum 51 | \ii ``Find the last digit'' $\rar$ Take mod $10$. ``Last two digits'' $\rar$ Take mod $100$, and so on. 52 | \ii It's useful to write a number $n$ as $n = \sum_{i=0}^k d_i 10^i = \overline{d_kd_{k-1}\dots d_1d_0}$ when taking mods. 53 | \ii Recall that $a\equiv b \implies a^n \equiv b^n \pmod{n}$; in other words, reduce the bases of your powers. 54 | \ii It can be helpful to work with different definitions of modular arithmetic. For example, showing that $3x\equiv 10 \pmod{21}$ has no solutions is easiest by demonstrating that $3x = 10 + 21k$ reduces to $0\equiv 1 \pmod{3}$. 55 | \ii While the EEA is great, finding inverses will often be faster/easier by writing out multiples of $m$. For example, if I'm finding the inverse of $9$ mod $11$, it's easier to look for a multiple of 9 in $1, 12, 23, 34, 45, \dots$ and then divide than to do the EEA. 56 | \itemend 57 | 58 | \section{Extra Practice} 59 | \begin{exercise} 60 | Prove that for any number $n$, the alternating sum of the digits of $n$, i.e. $d_0 - d_1 + d_2 - \dots$, is divisible by 11 if and only if $n$ is divisible by 11. 61 | \end{exercise} 62 | 63 | \end{document} 64 | -------------------------------------------------------------------------------- /CS70/recap3b/function-mapping.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tyleryzhu/course-notes/4a08721614b5d4ef2b749a028083514093b7a82c/CS70/recap3b/function-mapping.png -------------------------------------------------------------------------------- /CS70/recap3b/injsurj.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | A 41 | B 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | 61 | A 62 | B 63 | 64 | 65 | 66 | 67 | 68 | 69 | 70 | 71 | 72 | 73 | 74 | 75 | 76 | 77 | 78 | 79 | 80 | 81 | 82 | 83 | 84 | 85 | 86 | 87 | A 88 | B 89 | 90 | 91 | 92 | 93 | 94 | 95 | 96 | 97 | 98 | 99 | 100 | 101 | 102 | 103 | 104 | 105 | 106 | 107 | 108 | 109 | 110 | 111 | 112 | 113 | 114 | 115 | 116 | 117 | 118 | A 119 | B 120 | 121 | General 122 | Injective 123 | Surjective 124 | Bijective 125 | NOT a 126 | 127 | 128 | 129 | 130 | 131 | 132 | 133 | 134 | 135 | 136 | 137 | A 138 | B 139 | 140 | 141 | Function 142 | (not surjective) 143 | (not injective) 144 | (injective, surjective) 145 | Function 146 | A has many B 147 | B can't have many A 148 | Every B has some A 149 | A to B, perfectly 150 | B can have many A 151 | 152 | 153 | -------------------------------------------------------------------------------- /CS70/recap3b/recap3b.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tyleryzhu/course-notes/4a08721614b5d4ef2b749a028083514093b7a82c/CS70/recap3b/recap3b.pdf -------------------------------------------------------------------------------- /CS70/recap3b/recap3b.tex: -------------------------------------------------------------------------------- 1 | \documentclass[11 pt]{scrartcl} 2 | \usepackage[header, margin, koma, stylish]{tyler} 3 | 4 | \newcommand{\hwtitle}{Discussion 3B Recap} 5 | 6 | \pagestyle{fancy} 7 | \fancyhf{} 8 | \fancyhead[l]{\hwtitle{}} 9 | \fancyhead[r]{Tyler Zhu} 10 | \cfoot{\thepage} 11 | 12 | \begin{document} 13 | \title{\Large \hwtitle{}} 14 | \author{\large Tyler Zhu} 15 | \date{\large\today} 16 | 17 | \maketitle 18 | 19 | \section{Quiz Review} 20 | We went over the following problem in discussion. 21 | \begin{problem} 22 | If all vertices of an undirected graph have degree 4, the graph must be the complete graph on five vertices, $K_5$. 23 | \end{problem} 24 | The answer is no, and there are \emph{many} examples (in fact, a whole class of graphs called the \href{https://en.wikipedia.org/wiki/Regular_graph}{4-regular graphs}). These counter-examples are all quite instructive: 25 | \itemnum 26 | \ii $K_{4,4}$, the complete bipartite graph on two sets of 4 vertices 27 | \ii $H_4$, the 4-dimensional hypercube (in scope) 28 | \ii $K_5\cup K_5$, the union of two $K_5$'s (albeit disconnected) 29 | \ii The double of any 3-regular graph (see Homework 3). 30 | \ii \dots and many more 31 | \itemend 32 | 33 | In our proof that all trees with at least 2 vertices are bipartite, we used the following fact: 34 | \begin{fact} 35 | Every tree has at least 2 leaves, or vertices of degree 1. 36 | \end{fact} 37 | I'll present two proofs of this fact, one constructive and one not (but only for the existence of one leaf). 38 | \begin{proof}[Extremal Proof] 39 | Take any longest path in the tree. The two endpoints of this path must be leaves, as otherwise I could further traverse them and create a longer path, contradiction. 40 | \end{proof} 41 | 42 | Here's the second proof, which I actually got from one of my students. It only shows that there is one leaf, but that's all we need for our proof actually. 43 | 44 | \begin{proof}[Non-constructive Proof] 45 | We use the following fact: 46 | \begin{fact} 47 | A graph with $k$ edges and $n$ vertices has a vertex of degree at most $2k/n$. 48 | \end{fact} 49 | The proof is exactly the same as Problem 2(a), since some number must be $\leq$ the average. But a tree has $n-1$ edges and $n$ vertices, so there is a vertex of degree $2(n-1)/n < 2$, which means there exists a vertex of degree 1. 50 | \end{proof} 51 | 52 | 53 | In our first proof, we made use of the \emph{Extremal Principle}, which is just a fancy way of saying to look at the extreme cases. Here's one cute example, and one slightly harder exercise. 54 | 55 | \begin{exercise} 56 | There are $n$ students standing in a field such that the distance between each pair is distinct. Each student is holding a ball, and when the teacher blows a whistle, each student throws their ball to the nearest student. Prove that there is a pair of students that throw their balls to each other. 57 | \end{exercise} 58 | \begin{exercise} 59 | Suppose $n$ people are attending a banquet, and each of them has at least $m$ 60 | friends $(2 \leq m \leq n)$, where friendship is mutual. Prove that we can put at least $m + 1$ of the attendants on the same round table, so that each person sits next to his or her friends on both sides. 61 | \end{exercise} 62 | 63 | \section{Fermat's Little Theorem} 64 | \begin{theorem}[Fermat's Little Theorem] 65 | For all prime $p$ with $\gcd(a,p) = 1$, $a^{p-1} \equiv 1 \pmod{p}$. 66 | \end{theorem} 67 | 68 | Use this theorem to help you reduce arbitrary powers more easily. For example, the theorem makes $23^{36}\equiv 1 \pmod{37}$ obvious immediately. Combined with CRT, powers become a piece of cake. 69 | 70 | \section{Bijections} 71 | A \emph{function} maps inputs from a set $A$ to elements in a set $B$. We denote them as $f: A\to B$. You've already seen many examples of functions. In fact, any mod operator like $f(x) = x\pmod{m}$ is also a function (what are $A$ and $B$?). 72 | 73 | There are two main types of functions that we're interested in (fix a function $f:A\to B$): 74 | \itemnum 75 | \ii $f$ is \textbf{onto} (surjective) if $\forall b\in B, \exists a\in A$ s.t. $f(a) = b$ 76 | \itemnum 77 | \ii i.e. every $b\in B$ has a pre-image. 78 | \itemend 79 | \ii $f$ is \textbf{one-to-one} (injective) if $\forall a,a' \in A, f(a) = f(a') \implies a = a'$. 80 | \itemnum 81 | \ii i.e. different inputs map to different outputs, 82 | \itemend 83 | \itemend 84 | 85 | If $f$ is both onto and one-to-one, we say that $f$ is \textbf{bijective}. Bijections are useful because they have formalize what it means for a function to be invertible. 86 | 87 | \begin{theorem} 88 | A function $f:A\to B$ is a bijection if and only if it has an inverse, i.e. there exists a bijection $g: B\to A$ for which $\forall a\in A, g(f(a)) = a$ and $\forall b\in B, f(g(b)) = b$. 89 | \end{theorem} 90 | 91 | Here's a helpful graphic illustrating the differences between all of these functions. 92 | \begin{figure}[!htb] 93 | \centering 94 | \includegraphics[scale=0.75]{function-mapping.png} 95 | \caption{Examples of the four types of functions, and a non-function. Source: Math is Fun.} 96 | \end{figure} 97 | 98 | One interesting observation you might make is that for $f:A\to B$ to be injective, $|A| \leq |B|$ must hold. Similarly, if $f:A\to B$ is surjective, $|A| \geq |B|$ must hold. Therefore if $f$ is bijective, $|A| = |B|$ must be true. For finite sets this isn't so remarkable, but later on in the course we will see how this gives us a method of comparing sets that are infinite in size! 99 | 100 | \section{Chinese Remainder Theorem} 101 | 102 | I will first begin with the formal statement of the theorem. 103 | \begin{theorem}[Chinese Remainder Theorem] 104 | Let $m_1, \dots, m_k$ be pairwise\footnote{Every pair of integers is relatively prime, as opposed to being relatively prime as a whole.} relatively prime positive integers, and let 105 | \[ M = m_1 \dots m_k.\] 106 | Then for every $k$-tuple $(x_1, \dots, x_k)$ of integers, there is exactly one residue class $x \pmod{M}$ such that 107 | \begin{align*} 108 | x &\equiv x_1 \pmod{m_1} \\ 109 | x &\equiv x_2 \pmod{m_2} \\ 110 | &\vdots \\ 111 | x &\equiv x_k \pmod{m_k}. 112 | \end{align*} 113 | \end{theorem} 114 | 115 | We call this a theorem, but it's really just a formalization of an intuition you should have. In other words, every integer mod $M$ can be uniquely identified by its values mod $m_1$, mod $m_2$, and so on. 116 | 117 | For example, knowing that $x\equiv 10 \pmod{15}$ is equivalent to knowing that $x\equiv 1\pmod{3}$ and $x \equiv 0 \pmod{5}$. The first condition restricts the possibilities to 5 different values (1, 4, 7, 10, 13), and the second condition tells you exactly which of these 5 the number is (10, being the only multiple of 5). You should expect this to happen in general. 118 | 119 | \begin{example} 120 | Suppose we're trying to determine $8^{39} \pmod{15}$. Without using CRT, we're pretty much stuck with hand computation. But let's break this down mod 3 and mod 5. We have that 121 | \[ 8^{39} \equiv (-1)^{39} \equiv -1 \equiv 2 \pmod{3} \] 122 | and 123 | \[ 8^{39} \equiv 3^{39} \equiv 3\cdot 9^{19} \equiv 3\cdot (-1)^{19} \equiv 2\pmod{5} \] 124 | so by CRT, $8^{39}\equiv 2 \pmod{15}$. 125 | \end{example} 126 | 127 | Extending the previous analogy, we can interpret this equivalence as understanding $x$ mod $M$ in terms of its basis vectors (1 mod $m_1$), (1 mod $m_2$), $\cdots$, (1 mod $m_k$). Each one specifies $x$'s value along that dimension (or that mod space). Then all we need to do is find out the correct coefficients to multiply each of the vectors by to create a linear combination that produces $x$ mod $M$. 128 | 129 | Let's create a general method for finding $x\mod{M}$ based on its values mod $m_1, \dots,$ mod $m_k$. Throughout this section, we will be using a fact that you have seen already, so we will refer to it by its common name. 130 | 131 | \begin{theorem}[Bezout's Lemma] 132 | Let $a,b$ be integers with $\gcd(a,b) = d$. Then there exists integers $x,y$ such that $ax+by = d$. Furthermore, all integers of the form $ax+by$ are multiples of $d$. 133 | \end{theorem} 134 | 135 | First we will do a short, concrete example. 136 | 137 | \begin{example}[Dis 3B \#3] 138 | Let's find the $x$ for which 139 | \begin{align*} 140 | x&\equiv 3 \pmod{11} \\ 141 | x&\equiv 7 \pmod{13}. 142 | \end{align*} 143 | Recall that since $\gcd(11,13) = 1$, by Bezout's Lemma there are integers $a,b$ so that 144 | \[ x = 11a + 13b.\] 145 | We just need to pick them so that $x$ satisfies our above constraints (which will be unique mod 143 by CRT). First, let's take our equation mod 11 and use our constraint; this gives 146 | \[ 3\equiv x \equiv 13b \pmod{11}.\] 147 | So $b = 13^{-1}\times 3 \equiv 6\times 3 \equiv 7 \pmod{11}$. Now let's take our equation mod 13 and use the other constraint to get 148 | \[ 7\equiv x \equiv 11a \pmod{13}.\] 149 | So $a = 11^{-1}\times 7 \equiv 6\times 7 \equiv 3\pmod{13}$. Putting this all together, we find that 150 | \[ x = 11a + 13b = 11\times 3 + 13\times 7 = 33 + 91 \implies \boxed{x \equiv 124 \pmod{143}}.\] 151 | \end{example} 152 | 153 | With this example in mind, let's see how we would carry out these steps in general. Suppose we have just two equations 154 | \begin{align*} 155 | x &\equiv x_1 \pmod{m_1} \\ 156 | x &\equiv x_2 \pmod{m_2}. 157 | \end{align*} 158 | 159 | Since $\gcd(m_1, m_2) = 1$, by Bezout's Lemma there are integers $c_1, c_2$ for which 160 | \[ x = c_1m_1 + c_2m_2\] 161 | unique up to mod $m_1m_2$. Taking this mod $m_1$, we find 162 | \[ x_1 \equiv x \equiv c_2m_2 \pmod{m_1} \implies c_2 \equiv m_2^{-1} x_1 \pmod{m_1}.\] 163 | 164 | Taking the equation mod $m_2$, we also get 165 | \[ x_2 \equiv x \equiv c_1m_1 \pmod{m_2} \implies c_1 \equiv m_1^{-1} x_2 \pmod{m_2}.\] 166 | 167 | Putting this together, we find that 168 | 169 | \[ \boxed{x \equiv (m_1)^{-1}_{m_2}m_1 x_2 + (m_2)^{-1}_{m_1}m_2 x_1 \pmod{m_1m_2}}\] 170 | 171 | where $(a)^{-1}_m$ denotes $a^{-1}$ mod $m$. 172 | 173 | This looks complicated, but remember that what we did was pretty natural. We simply had a system of equations with some constraints, and using those constraints we found out what each of the required coefficients are. 174 | 175 | Having seen the case of two moduli, we can proceed with the general case. Let $M = m_1\dots m_k$ be a product of pairwise coprime integers, and suppose we have the system 176 | \begin{align*} 177 | x &\equiv x_1 \pmod{m_1} \\ 178 | x &\equiv x_2 \pmod{m_2} \\ 179 | &\vdots \\ 180 | x &\equiv x_k \pmod{m_k}. 181 | \end{align*} 182 | 183 | Fix an $m_i$. Since $\gcd(M/m_i, m_i) = 1$, by Bezout's Lemma, there are integers $C_i, c_i$ for which 184 | \[ x = C_i \left(\dfrac{M}{m_i}\right) + c_i m_i.\] 185 | 186 | Taking this mod $m_i$, we find that 187 | 188 | \[ x_i \equiv x \equiv C_i \left(\dfrac{M}{m_i}\right) \implies C_i \equiv \left(\dfrac{M}{m_i}\right)^{-1} x_i \pmod{m_i}.\] 189 | 190 | I claim that these coefficients create a solution for $x$, i.e. that 191 | \[ x = \sum_{i = 1}^k C_i \left(\dfrac{M}{m_i}\right) \equiv \sum_{i=1}^k \left(\dfrac{M}{m_i}\right)^{-1}_{m_i} \left(\dfrac{M}{m_i}\right) x_i \pmod{M}.\] 192 | 193 | If I took this expression mod $m_i$, every term that's not the $x_i$ term would be 0 as $\frac{M}{m_j}$ would contain $m_i$. The $i$th term would cancel out to just $x_i$, which is precisely what we wanted. 194 | 195 | In a sense, these coefficients are our ``basis coefficients'' which are always 1 mod $m_i$ and 0 otherwise, so that $x$ mod $m_i$ is always $x_i$. When we study Lagrange Interpolation, you will see a deep connection between these two concepts, as they are essentially the same idea applied in two different contexts. 196 | 197 | 198 | \subsection{Quick Remark} 199 | Here's a quick and dirty way to solve Question 3. 200 | \begin{example}[Very, Very Fast] 201 | Let's find the $x$ for which 202 | \begin{align*} 203 | x&\equiv 3 \pmod{11} \\ 204 | x&\equiv 7 \pmod{13}. 205 | \end{align*} 206 | The second equation tells us that $x = 7 + 13k$ for some $k\in \ZZ$. If we take this equation mod 11, we know it must be equivalent to 3, so 207 | \[ 3 \equiv x \equiv 7 + 2k \implies 2k \equiv 7\pmod{11}\] 208 | from which it's easy to tell that $k = 9$. Thus, $x = 7 + 13\cdot 9 \equiv 128 \pmod{143}$ is our answer. 209 | \end{example} 210 | 211 | \end{document} 212 | -------------------------------------------------------------------------------- /CS70/recap4a/recap4a.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tyleryzhu/course-notes/4a08721614b5d4ef2b749a028083514093b7a82c/CS70/recap4a/recap4a.pdf -------------------------------------------------------------------------------- /CS70/recap4a/recap4a.tex: -------------------------------------------------------------------------------- 1 | \documentclass[11 pt]{scrartcl} 2 | \usepackage[header, margin, koma]{tyler} 3 | 4 | \newcommand{\hwtitle}{Discussion 4A Recap} 5 | 6 | \pagestyle{fancy} 7 | \fancyhf{} 8 | \fancyhead[l]{\hwtitle{}} 9 | \fancyhead[r]{Tyler Zhu} 10 | \cfoot{\thepage} 11 | 12 | \begin{document} 13 | \title{\Large \hwtitle{}} 14 | \author{\large Tyler Zhu} 15 | \date{\large\today} 16 | 17 | \maketitle 18 | 19 | \section{RSA} 20 | The setup is that Alice is trying to send a message to Bob securely so eavesdroppers can't figure out the message. RSA is a public-key cryptography scheme, meaning that anyone can encrypt a message and send it to Bob with his public key, but only he has the private key to decrypt the message. 21 | 22 | \textbf{Setup:} Bob picks two large primes $p,q$ and another (small) number $e$ which is relatively prime to $(p-1)(q-1)$. He releases $(N=pq, e)$ to the world as his public key. He also computes $d = e^{-1} \pmod{(p-1)(q-1)}$ but keeps it private. 23 | 24 | \textbf{Encryption:} If Alice wants to send a message $x$, she transmits $E(x) = x^e \pmod{N}$ to Bob. 25 | 26 | \textbf{Decryption:} Bob receives $y = E(x)$ and decrypts it by doing $D(y) = y^d \pmod{N} = x^{ed} \equiv x \pmod{(p-1)(q-1)}$. 27 | 28 | The proof of correctness relies on Fermat's Little Theorem (really Euler's Totient Theorem) and the proof of security relies on the hardness of the discrete logarithm problem, i.e. there is no efficient way to undo exponents. 29 | 30 | \section{Remarks} 31 | \itemnum 32 | \ii The message $x$ needs to be $0$ and $g^n = (g^{-1})^n$ when $n < 0$, and $g^0 = e$ by definition. The element $g$ is called the \emph{generator} of the group. 62 | 63 | If we wrote this additively, we would write this as $G = \{n\cdot g | n\in \ZZ\}$. 64 | 65 | \begin{example}[Examples of Cyclic Groups] 66 | Some examples of cyclic groups. 67 | \itemnum 68 | \ii The set of integers, $\ZZ$, itself. $1\in \ZZ$ is one of the two generators (-1 being the other). 69 | \ii The set of integers mod $n$ is also a cyclic group under addition, noted as $\ZZ/n\ZZ$. 70 | \ii The multiplicative group $(\ZZ/n\ZZ)^\times$, defined as 71 | \[ \{a\in \ZZ/n\ZZ | \gcd(a,n) = 1\},\] 72 | when $n$ is a prime number, which we will prove later in this course. 73 | \itemend 74 | \end{example} 75 | 76 | The last example above lends itself to a broader generalization. Here's a taste of what's to come... 77 | \begin{theorem} 78 | Let $F^\times$ be the group of nonzero elements of a field $F$ (under multiplication). If $G\subseteq F^\times$ is a finite subgroup, then $G$ is cyclic. 79 | \end{theorem} 80 | For example, the $n$th roots of unity are finite subgroups of $\CC^\times$, and hence are also cyclic. We knew that already though, since they're generated by $e^{2\pi i/n}$. 81 | 82 | \begin{example} 83 | If $V$ is a vector space over $K$, the set of invertible linear maps $T: V\to V$ forms a group under composition, called the \emph{general linear} group $\GL(V)$. 84 | 85 | If $V = K^n$, then $\GL(V) = \GL(n, K)$, the group of invertible $n\times n$ matrices with coefficients in $K$. The \emph{special linear} group $\SL(n, K)$ is the subgroup with determinant 1. 86 | \end{example} 87 | 88 | Suppose we wanted to construct an element of $\GL(n, K)$ using this matrix interpretation. We can approach it by building it column by column, keeping in mind to make the columns linearly independent so that the determinant is not 0. In other words, if this matrix is $[C_1\;C_2\dots C_n]$ where the $C_i$ are the columns, then we want $C_i \not\in \Span(C_1, \dots, C_{{i-1}})$ for $i > 1$ and that $C_1$ is not the zero vector. 89 | 90 | Let $K$ be a finite field now, with order $q = |K|$, like $\ZZ/q\ZZ$. What is $|\GL(n, K)|$? 91 | 92 | We can count by counting the choices we have for each column. For $C_1 \in K^n$, it just needs to be non-empty, so we have $q^n - 1$ choices. For $C_2$, we need $C_2 \not\in \Span(C_1) = \{\text{multiples of }C_1\}$. There are $q$ multiples, so we have $q^n - q$ choices for $C_2$. 93 | 94 | In general, the number of possibilities for $C_i$ is $q^n - q^{i-1}$ since the span of $i-1$ linearly independent vectors in $K$ is isomorphic to $K^{i-1}$, which has $q^{i-1}$ elements. Multiplying everything up gives the total order of the group as 95 | \[ \GL(n, K) = (q^n-1)(q^n-q)\dots (q^n - q^{n-1}).\] 96 | 97 | If $\sum$ is a set, the set of invertible maps $f:\sum\to \sum$ forms a group under composition called the \emph{symmetric} group on the set $\sum$. Notation is $S_\Sigma$, but Lang calls it $\Perm(\sum)$. Borrowing notation from combinatorics, we let $[n] = \{1, 2, \dots, n\}$, so that the symmetric group on $n$ elements is $\Perm([n]) = S_{[n]}$. 98 | 99 | \begin{remark} 100 | The examples $\GL(V)$ and $\Perm(\Sigma)$ are special examples of a construction where the set of automorphisms is a group (category theory makes this precise). 101 | \end{remark} 102 | 103 | \begin{remark} 104 | When Lang was alive, he'd yell at Berkeley undergrads who'd tell him that a morphism $X\to Y$ is invertible iff it is 1-1 and onto. Mostly true, but false in simple example. For example, the map $x^3$ from $(-1, 1)\mapsto (-1, 1)$ in the category of differentiable maps has a set-theoretic inverse $x^{1/3}$, which is nondifferentiable at $x = 0$. 105 | \end{remark} 106 | 107 | We can form subgroups of groups. Some special ones are the cosets of $G$. If $H < G$, the set $G/H$ is the set of subsets of $G$ of the form $gH$ (for $g$ in $G$). These are called the \emph{left cosets} of $H$ in $G$. 108 | 109 | Similarly, $H\setminus G$ is the set of \emph{right} cosets $Hg$ with $g\in G$. 110 | 111 | Left cosets are the equivalence classes for the equivalence relation $x\sim y \iff x^{-1}y \in H$, while right cosets are equivialence classes for $x\sim y \iff xy^{-1} \in H$. 112 | 113 | The cosets $gH$ all have the same cardinality, i.e. the map $h\mapsto gh$ is a bijection. The group $G$ is the disjoint union of the distinct cosets $gH$ since they're all equivalence classes. Thus, 114 | \[ |G| = \sum_{gH \in G/H} |gH| = \sum_{gH\in G/H} |H| = [G:H]\cdot |H|.\] 115 | In particular, when $G$ is a finite group, the order of every subgroup $H < G$ divides the order of $G$, which is Lagrange's theorem. 116 | 117 | \subsection{Homomorphisms} 118 | Note: homomorphisms are the morphisms in the category of groups! 119 | 120 | If $X$ and $Y$ are groups, the \emph{trivial} homomorphism $X\to Y$ is the map that takes everything to the identity of $Y$. 121 | 122 | The floor (or greatest integer map) 123 | \[ \floor{\cdot}: \QQ\to \ZZ, x\mapsto \floor{x}\] 124 | is \emph{not} a homomorphism since $\floor{\frac 34} + \floor{\frac 34} \not= \floor{\frac 32} = 1$. 125 | 126 | The identity map on $X$ is a homomorphism $X\to X$. 127 | 128 | If $A$ is abelian and $n$ is an integer, the $n$th power map $a\mapsto a^n$ is an endomorphism of $A$ (i.e. a homomorphism from a group to itself, especially when the group is abelian). 129 | 130 | The determinant map is a homomorphism $\GL(n, K) \to K^\times$ when $n > 0$ and $K$ is a field. 131 | 132 | A subgroup $N$ of $G$ is normal if for all $g\in G$, $gNg^{-1} \subseteq N$. As we've learned before, the kernel of a homomorphism is normal. In this case, the ``canonical'' quotient map 133 | \[ \pi : G\to G/N, g\mapsto gN\] 134 | is a homomorphism with kernel $N$. 135 | 136 | \end{document} 137 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # course-notes 2 | Source code for most of my course notes 3 | 4 | If you do end up using parts of my notes, attribution would be appreciated. 5 | -------------------------------------------------------------------------------- /coursenotes.code-workspace: -------------------------------------------------------------------------------- 1 | { 2 | "folders": [ 3 | { 4 | "path": "." 5 | } 6 | ], 7 | "settings": {} 8 | } -------------------------------------------------------------------------------- /templates/hw-template.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tyleryzhu/course-notes/4a08721614b5d4ef2b749a028083514093b7a82c/templates/hw-template.pdf -------------------------------------------------------------------------------- /templates/hw-template.synctex.gz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tyleryzhu/course-notes/4a08721614b5d4ef2b749a028083514093b7a82c/templates/hw-template.synctex.gz -------------------------------------------------------------------------------- /templates/hw-template.tex: -------------------------------------------------------------------------------- 1 | \documentclass[11 pt]{scrartcl} 2 | \usepackage[header, margin, koma]{tyler} 3 | 4 | \newcommand{\hwtitle}{CS271 Homework 1} 5 | 6 | \pagestyle{fancy} 7 | \fancyhf{} 8 | \fancyhead[l]{\hwtitle{}} 9 | \fancyhead[r]{Tyler Zhu} 10 | \cfoot{\thepage} 11 | 12 | \begin{document} 13 | \title{\Large \hwtitle{}} 14 | \author{\large Tyler Zhu} 15 | \date{\large\today} 16 | 17 | \maketitle 18 | 19 | \section{Problem 1} 20 | 21 | \end{document} 22 | -------------------------------------------------------------------------------- /templates/notes-template.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tyleryzhu/course-notes/4a08721614b5d4ef2b749a028083514093b7a82c/templates/notes-template.pdf -------------------------------------------------------------------------------- /templates/notes-template.synctex.gz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tyleryzhu/course-notes/4a08721614b5d4ef2b749a028083514093b7a82c/templates/notes-template.synctex.gz -------------------------------------------------------------------------------- /templates/notes-template.tex: -------------------------------------------------------------------------------- 1 | \documentclass[11 pt]{scrartcl} 2 | \usepackage[header, margin, koma]{tyler} 3 | %\usetikzlibrary{automata,arrows,positioning,calc} 4 | \usepackage{csquotes} 5 | 6 | \pagestyle{fancy} 7 | \fancyhf{} 8 | \fancyhead[l]{CS 294-165 Notes} 9 | \fancyhead[r]{Tyler Zhu} 10 | \cfoot{\thepage} 11 | 12 | \begin{document} 13 | \title{\Large CS 294-165: Sketching Algorithms} 14 | \author{\large Tyler Zhu} 15 | \date{\large\today} 16 | 17 | \maketitle 18 | 19 | \begin{center} 20 | \begin{displayquote} 21 | \emph{"A good stock of examples, as large as possible, is indispensable for a thorough understanding of any concept, and when I want to learn something new, I make it my first job to build one."} \\ \begin{flushright} \emph{– Paul Halmos}. \end{flushright} 22 | \end{displayquote} 23 | \end{center} 24 | 25 | 26 | These are course notes for the Fall 2020 rendition of CS 294-165, Sketching Algorithms, taught by Professor Jelani Nelson. 27 | 28 | \tableofcontents 29 | 30 | \newpage 31 | 32 | \section{Wednesday, August 26} 33 | 34 | 35 | 36 | \end{document} 37 | -------------------------------------------------------------------------------- /tyler.sty: -------------------------------------------------------------------------------- 1 | %%%%%%%%%%%% PACKAGES %%%%%%%%%%%%%%%% 2 | \usepackage{amsmath,amssymb,amsthm} 3 | \usepackage[usenames,svgnames,dvipsnames]{xcolor} 4 | \usepackage{mathtools} 5 | \usepackage{graphicx} 6 | \usepackage{geometry} 7 | \usepackage{caption} 8 | \usepackage[framemethod=TikZ]{mdframed} 9 | \usepackage[shortlabels]{enumitem} 10 | \usepackage{algorithm} 11 | \usepackage[noend]{algpseudocode} 12 | \usepackage{array} 13 | \usepackage{thmtools} 14 | \usepackage{hyperref} % for clickable links in toc 15 | \usepackage{bbm} 16 | \usepackage{bm} 17 | \usepackage{easybmat} 18 | 19 | %%%%%%%%% If/Else 20 | \newif\iftylerheader\tylerheaderfalse 21 | \newif\iftylerhints\tylerhintsfalse 22 | \newif\iftyleranswers\tyleranswersfalse 23 | \newif\iftylermargin\tylermarginfalse 24 | \newif\iftylerkoma\tylerkomafalse 25 | \newif\iftylerstylish\tylerstylishfalse 26 | \newif\iftylerbib\tylerbibfalse 27 | 28 | %%%%%%% Receive Arguments 29 | \DeclareOption{header}{\tylerheadertrue} 30 | \DeclareOption{noheader}{\tylerheaderfalse} 31 | \DeclareOption{hints}{\tylerhintstrue} 32 | \DeclareOption{nohints}{\tylerhintsfalse} 33 | \DeclareOption{answers}{\tyleranswerstrue} 34 | \DeclareOption{noanswers}{\tyleranswersfalse} 35 | \DeclareOption{margin}{\tylermargintrue} 36 | \DeclareOption{nomargin}{\tylermarginfalse} 37 | \DeclareOption{koma}{\tylerkomatrue} 38 | \DeclareOption{nokoma}{\tylerkomafalse} 39 | \DeclareOption{stylish}{\tylerstylishtrue} 40 | \DeclareOption{nostylish}{\tylerstylishfalse} 41 | \DeclareOption{bib}{\tylerbibtrue} 42 | \DeclareOption{nobib}{\tylerbibfalse} 43 | \ProcessOptions 44 | 45 | %%%%%%%%%%%%%% PAGE SETUP %%%%%%%%%%%%%%%%% 46 | \iftylerkoma 47 | \thispagestyle{empty} 48 | \usepackage{sectsty} 49 | \allsectionsfont{\normalfont\sffamily\bfseries} 50 | \setkomafont{section}{\large} %sets section headers to a reasonable size 51 | \setkomafont{author}{\large\scshape} 52 | \setkomafont{title}{\Large\sffamily\bfseries} 53 | \setkomafont{date}{\Large\normalsize} 54 | \fi 55 | 56 | \iftylermargin 57 | \geometry{ 58 | left=1in, 59 | right=1in, 60 | top=1in, 61 | bottom=1in 62 | } 63 | \fi 64 | 65 | \iftylerheader 66 | \usepackage{fancyhdr} 67 | \pagestyle{fancy} 68 | \fancyhf{} 69 | \fancyhead[l]{\tiny\@title} 70 | \fancyhead[r]{Tyler Zhu} 71 | \cfoot{\thepage} 72 | \fi 73 | 74 | \iftylerhints 75 | \usepackage{answers} 76 | \Newassociation{hint}{hintitem}{hints} 77 | \renewcommand{\solutionextension}{out} 78 | \Opensolutionfile{hints} 79 | \newcommand{\makehints}{\Closesolutionfile{hints}\input{hints.out}} 80 | \fi 81 | % Define a Sans-Serif Environment for KOMA Type Documents 82 | \newtheoremstyle{ss-definition}% 83 | {5pt}% 84 | {3pt}% 85 | {}% 86 | {}% 87 | {\sffamily\bfseries}% 88 | {.}% 89 | {.5em}% 90 | {}% 91 | 92 | 93 | %%%%%%% Nice Theorem Style %%%%% 94 | \iftylerstylish 95 | \mdfdefinestyle{mdbluebox}{% 96 | % roundcorner = 10pt, 97 | linewidth=1pt, 98 | skipabove=8pt, 99 | innertopmargin=7pt, 100 | innerbottommargin=6pt, 101 | skipbelow=2pt, 102 | % linecolor=blue, 103 | nobreak=true, 104 | backgroundcolor=TealBlue!5, 105 | } 106 | \declaretheoremstyle[ 107 | headfont=\sffamily\bfseries,%\color{MidnightBlue}, 108 | mdframed={style=mdbluebox}, 109 | headpunct={.}, 110 | postheadspace={4pt} 111 | ]{thmbluebox} 112 | \declaretheorem[% 113 | style=thmbluebox,name=Theorem]{theorem} 114 | 115 | \mdfdefinestyle{mdredbox}{% 116 | linewidth=0.5pt, 117 | skipabove=8pt, 118 | frametitleaboveskip=7pt, 119 | frametitlebelowskip=6pt, 120 | skipbelow=2pt, 121 | frametitlefont=\bfseries, 122 | innertopmargin=6pt, 123 | innerbottommargin=10pt, 124 | backgroundcolor=Salmon!5, 125 | linecolor=RawSienna, 126 | } 127 | \declaretheoremstyle[ 128 | headfont=\sffamily\bfseries\color{RawSienna}, 129 | mdframed={style=mdredbox}, 130 | headpunct={.}, 131 | postheadspace={4pt}, 132 | ]{thmredbox} 133 | \declaretheorem[% 134 | style=thmredbox,name=Example,numberwithin=section]{example} 135 | \declaretheorem[% 136 | style=thmredbox,name=Problem]{problem} 137 | 138 | %%%% left red border 139 | \mdfdefinestyle{mdredleftborder}{% 140 | linewidth=4pt, 141 | skipabove=8pt, 142 | innertopmargin=7pt, 143 | innerbottommargin=6pt, 144 | skipbelow=2pt, 145 | % linecolor=blue, 146 | nobreak=true, % add back no breaks for defs + theorems 147 | linecolor=FireBrick, 148 | hidealllines=true, 149 | leftline=true 150 | } 151 | \declaretheoremstyle[ 152 | headfont=\sffamily\bfseries, 153 | mdframed={style=mdredleftborder}, 154 | headpunct={.}, 155 | postheadspace={4pt} 156 | ]{redleftborder} 157 | \declaretheorem[% 158 | style=redleftborder,name=Definition, sibling=theorem]{definition} 159 | 160 | \else 161 | \iftylerkoma 162 | \theoremstyle{ss-definition} 163 | \else 164 | \theoremstyle{definition} 165 | \fi 166 | \newtheorem{theorem}{Theorem} 167 | \newtheorem{example}{Example}[section] 168 | \newtheorem*{example*}{Example} 169 | \newtheorem{problem}{Problem} 170 | \newtheorem{definition}[theorem]{Definition} 171 | \fi 172 | 173 | \iftylerkoma 174 | \theoremstyle{ss-definition} 175 | \else 176 | \theoremstyle{definition}e.x. take 177 | \fi 178 | \newtheorem{corollary}[theorem]{Corollary} 179 | \newtheorem{proposition}[theorem]{Proposition} 180 | \newtheorem{lemma}[theorem]{Lemma} 181 | \newtheorem{fact}[theorem]{Fact} 182 | 183 | \newtheorem{exercise}[example]{Exercise} 184 | \newtheorem*{question}{Question} 185 | \newtheorem*{remark}{Remark} 186 | \newtheorem*{claim}{Claim} 187 | 188 | %\newtheorem{problem}{Problem} 189 | %\newtheorem{example}{Example} 190 | 191 | %%% bibliography 192 | \iftylerbib 193 | \usepackage{biblatex} 194 | \fi 195 | 196 | % Blackboard Math Fonts 197 | \newcommand{\one}{\mathbbm 1} 198 | \newcommand{\CC}{\mathbb C} 199 | \newcommand{\EE}{\mathbb E} 200 | \newcommand{\FF}{\mathbb F} 201 | \newcommand{\II}{\mathbf I} 202 | \newcommand{\NN}{\mathbb N} 203 | \newcommand{\PP}{\mathbb P} 204 | \newcommand{\QQ}{\mathbb Q} 205 | \newcommand{\RR}{\mathbb R} 206 | \renewcommand{\SS}{\mathbb S} 207 | \newcommand{\ZZ}{\mathbb Z} 208 | \newcommand{\vx}{{\bm{x}}}% 209 | 210 | % Mathcal Fonts 211 | \newcommand{\Acal}{\mathcal A} 212 | \newcommand{\Bcal}{\mathcal B} 213 | \newcommand{\Ccal}{\mathcal C} 214 | \newcommand{\Dcal}{\mathcal D} 215 | \newcommand{\Ecal}{\mathcal E} 216 | \newcommand{\Fcal}{\mathcal F} 217 | \newcommand{\Gcal}{\mathcal G} 218 | \newcommand{\Hcal}{\mathcal H} 219 | \newcommand{\Ical}{\mathcal I} 220 | \newcommand{\Jcal}{\mathcal J} 221 | \newcommand{\Kcal}{\mathcal K} 222 | \newcommand{\Lcal}{\mathcal L} 223 | \newcommand{\Mcal}{\mathcal M} 224 | \newcommand{\Ncal}{\mathcal N} 225 | \newcommand{\Ocal}{\mathcal O} 226 | \newcommand{\Pcal}{\mathcal P} 227 | \newcommand{\Qcal}{\mathcal Q} 228 | \newcommand{\Rcal}{\mathcal R} 229 | \newcommand{\Scal}{\mathcal S} 230 | \newcommand{\Tcal}{\mathcal T} 231 | \newcommand{\Ucal}{\mathcal U} 232 | \newcommand{\Vcal}{\mathcal V} 233 | \newcommand{\Wcal}{\mathcal W} 234 | \newcommand{\Xcal}{\mathcal X} 235 | \newcommand{\Ycal}{\mathcal Y} 236 | \newcommand{\Zcal}{\mathcal Z} 237 | 238 | \DeclarePairedDelimiter\ceil{\lceil}{\rceil} 239 | \DeclarePairedDelimiter\floor{\lfloor}{\rfloor} 240 | 241 | % Shorthands 242 | \newcommand{\ii}{\item} 243 | \newcommand{\alphanum}{\begin{enumerate}[(a)]} 244 | \newcommand{\enumend}{\end{enumerate}} 245 | \newcommand{\itemnum}{\begin{itemize}} 246 | \newcommand{\itemend}{\end{itemize}} 247 | \newcommand{\eps}{\epsilon} 248 | \newcommand{\la}{\lambda} 249 | \newcommand{\om}{\omega} 250 | \newcommand{\si}{\sigma} 251 | \newcommand{\vv}{\vec{v}} 252 | \newcommand{\x}{\times} 253 | \newcommand{\Col}[1]{$\text{Col}$($#1$)} 254 | \newcommand{\Nul}[1]{$\text{Nul}$($#1$)} 255 | \newcommand{\Null}[1]{\text{Null} #1 } 256 | \newcommand{\tr}[1]{\text{tr} #1 } 257 | \newcommand{\Tr}{\text{Tr}} 258 | \newcommand{\dom}{\text{dom}\,} 259 | \newcommand{\rank}[1]{\text{rank} #1 } 260 | \newcommand{\range}{\text{range}} 261 | \newcommand{\spn}{\text{span}} % don't renew \span! fucks up align environments :-( 262 | \newcommand{\bbm}{\begin{bmatrix}} 263 | \newcommand{\ebm}{\end{bmatrix}} 264 | \newcommand{\bpm}{\begin{pmatrix}} 265 | \newcommand{\epm}{\end{pmatrix}} 266 | \newcommand{\T}{\top} 267 | \newcommand{\lar}{\Leftarrow} 268 | \newcommand{\rar}{\Rightarrow} 269 | \newcommand{\diam}{\text{diam }} 270 | \newcommand{\diag}{\text{diag}} 271 | \newcommand{\se}{\subseteq} 272 | \newcommand{\sgn}{\text{sgn}} 273 | \newcommand{\card}{\text{card}\,} 274 | \newcommand{\ord}{\text{ord}} 275 | \newcommand{\Syl}{\text{Syl}} 276 | \newcommand{\Res}{\text{Res}} 277 | \newcommand{\per}{\text{per}} 278 | \newcommand{\tsig}{\tilde{\Sigma}} 279 | 280 | % Typesetting 281 | \newcommand{\styl}[1]{{\sffamily\bfseries #1}} 282 | \def\upint{\mathchoice% 283 | {\mkern13mu\overline{\vphantom{\intop}\mkern7mu}\mkern-20mu}% 284 | {\mkern7mu\overline{\vphantom{\intop}\mkern7mu}\mkern-14mu}% 285 | {\mkern7mu\overline{\vphantom{\intop}\mkern7mu}\mkern-14mu}% 286 | {\mkern7mu\overline{\vphantom{\intop}\mkern7mu}\mkern-14mu}% 287 | \int} 288 | \def\lowint{\mkern3mu\underline{\vphantom{\intop}\mkern7mu}\mkern-10mu\int} 289 | 290 | % Probability Theory Commands 291 | \newcommand{\Bin}{\text{Bin}} 292 | \newcommand{\Ber}{\text{Ber}} 293 | \newcommand{\Exp}{\text{Exp}} 294 | \newcommand{\Var}{\text{Var}} 295 | \newcommand{\var}{\,\text{var}} 296 | \newcommand{\Cov}{\text{Cov}} 297 | \newcommand{\cov}{\,\text{cov}} 298 | \newcommand{\Corr}{\,\text{Corr}} 299 | \newcommand{\Expo}{\text{Expo}} 300 | \newcommand{\Geo}{\text{Geo}} 301 | \newcommand{\Poisson}{\text{Poisson}} 302 | \newcommand{\PoisP}{\text{PP}} 303 | \newcommand{\Uniform}{\text{Uniform}} 304 | \newcommand{\Nor}{\mathcal{N}} 305 | \newcommand{\asure}{\xrightarrow{\text{a.s.}}} 306 | \newcommand{\cinprob}{\xrightarrow[n\to\infty]{\PP}} 307 | \newcommand{\BEC}[1]{\text{BEC}(#1)} 308 | \newcommand{\grle}[2]{\underset{{#2}}{\overset{#1}{\gtrless}}} 309 | \newcommand{\legr}[2]{\underset{{#2}}{\overset{#1}{\lessgtr}}} 310 | \newcommand{\cyc}[1]{\left\langle #1 \right\rangle} 311 | \newcommand{\proj}{\text{proj}} 312 | \newcommand{\MMSE}{\text{MMSE}} 313 | \newcommand{\MAP}{\text{MAP}} 314 | \newcommand{\MLE}{\text{MLE}} 315 | 316 | %%%%%%%%%% Math (255) Things %%%%%%%%%%%%%% 317 | \newcommand{\pd}{\partial} 318 | \newcommand{\parx}[1]{\dfrac{\partial #1}{\partial x}} 319 | \newcommand{\pary}[1]{\dfrac{\partial #1}{\partial y}} 320 | \newcommand{\parz}[1]{\dfrac{\partial #1}{\partial z}} 321 | \newcommand{\SL}{\text{SL}} 322 | \newcommand{\ASL}{\text{ASL}} 323 | \newcommand{\AGL}{\text{AGL}} 324 | \newcommand{\Gal}{\text{Gal}} 325 | 326 | %%%%%%%%%% Analaysis (H104) command %%%%%%%% 327 | 328 | % Augmented Matrices 329 | \newenvironment{amatrix}[1]{% 330 | \left[\begin{array}{@{}*{#1}{c}|c@{}} 331 | }{% 332 | \end{array}\right] 333 | } 334 | 335 | % For ease in using arrays 336 | \newcolumntype{C}{>$c<$} 337 | 338 | % Column Vectors 339 | \newcount\colveccount 340 | \newcommand*\colvec[1]{ 341 | \global\colveccount#1 342 | \begin{bmatrix} 343 | \colvecnext 344 | } 345 | \def\colvecnext#1{ 346 | #1 347 | \global\advance\colveccount-1 348 | \ifnum\colveccount>0 349 | \\ 350 | \expandafter\colvecnext 351 | \else 352 | \end{bmatrix} 353 | \fi 354 | } 355 | 356 | % Row Vectors 357 | \newtoks\rowvectoks 358 | \newcommand{\rowvec}[2]{% 359 | \rowvectoks={#2}\count255=#1\relax 360 | \advance\count255 by -1 361 | \rowvecnexta} 362 | \newcommand{\rowvecnexta}{% 363 | \ifnum\count255>0 364 | \expandafter\rowvecnextb 365 | \else 366 | \begin{bmatrix}\the\rowvectoks\end{bmatrix} 367 | \fi} 368 | \newcommand\rowvecnextb[1]{% 369 | \rowvectoks=\expandafter{\the\rowvectoks}% 370 | \advance\count255 by -1 371 | \rowvecnexta} 372 | 373 | 374 | %%%%% ALGORITHMS %%%%% 375 | \makeatletter 376 | \def\BState{\State\hskip-\ALG@thistlm} 377 | \makeatother 378 | 379 | %%%%%%%%% Drawing Markov Chains / Graphs %%%%%%%%%%% 380 | \usetikzlibrary{automata,arrows,positioning,calc} 381 | --------------------------------------------------------------------------------