├── guide.pdf ├── .latexmkrc ├── images ├── sliding-puzzle.png └── paper_clip_chain.jpg ├── .gitignore ├── chapters ├── nonsense.tex ├── acknowledgments.tex ├── title.tex ├── preface.tex ├── dp.tex ├── computational_geometry.tex ├── implementation.tex ├── strings.tex ├── math.tex ├── problems.tex ├── graphs.tex ├── trees.tex ├── big_ideas.tex └── fundamentals.tex ├── README.md ├── main.tex ├── main.sty └── evan.sty /guide.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alwayswimmin/cs_guide/HEAD/guide.pdf -------------------------------------------------------------------------------- /.latexmkrc: -------------------------------------------------------------------------------- 1 | $clean_ext = "loa pre"; 2 | $pdf_mode = 1; 3 | $jobname = "guide"; 4 | -------------------------------------------------------------------------------- /images/sliding-puzzle.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alwayswimmin/cs_guide/HEAD/images/sliding-puzzle.png -------------------------------------------------------------------------------- /images/paper_clip_chain.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alwayswimmin/cs_guide/HEAD/images/paper_clip_chain.jpg -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | *.aux 2 | *.fdb_latexmk 3 | *.fls 4 | *.loa 5 | *.log 6 | *.out 7 | *.pre 8 | *.toc 9 | main.pdf 10 | .DS_STORE 11 | -------------------------------------------------------------------------------- /chapters/nonsense.tex: -------------------------------------------------------------------------------- 1 | \chapter{Nonsense} 2 | 3 | Have fun. 4 | 5 | \section{Segment Tree Extensions} 6 | 7 | \subsection{Fractional Cascading} 8 | 9 | \subsection{Persistence} 10 | 11 | \subsection{Higher Dimensions} 12 | 13 | \section{DP Optimizations} 14 | 15 | \section{Top Tree} 16 | 17 | Tree representation of a graph. Scary. 18 | 19 | \section{Link-Cut Cactus} 20 | 21 | A \textit{cactus} is a graph such that any two distinct cycles in the graph share at most one vertex. Graphs that are not trees are somehow hard to characterize; a cactus is a more general kind of graph that still has some nice properties. In particular, certain tree algorithms still run in polynomial time when modified on a cactus where no such generalization to all graphs exists. 22 | 23 | Cacti are scary. 24 | -------------------------------------------------------------------------------- /chapters/acknowledgments.tex: -------------------------------------------------------------------------------- 1 | \chapter{Acknowledgments} 2 | 3 | %I am truly thankful for all my school teachers and USACO instructors, who gave me the knowledge which I wish to pass on through this text. I am grateful for Shankar and Ross, Evan, and Dr. Osborne, who separately made me believe such a task was possible. I must credit Valerie, for she was the one who drove me to really dive into and come to love computer science. Shwetark were extremely helpful with writing this text, as they provided input and helped me find problems to supplement my work. Lastly, huge shoutouts to Alex, who has been tremendous as a partner in writing this! Finally, I'd like to thank all my friends and avid programmers in the TJ community. This is for all of you, and I hope you find it useful in your studies. 4 | % 5 | %\begin{flushright} 6 | %-- Samuel 7 | %\end{flushright} 8 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # cs_guide 2 | Collection of Useful CS Algorithms 3 | 4 | Usage Notes 5 | -------------- 6 | Learn something [: 7 | 8 | For Learners 9 | -------------- 10 | A Dropbox link with the latest version can be found here: 11 | 12 | https://www.dropbox.com/s/z2lur71042pjaet/guide.pdf?dl=0 13 | 14 | For Collaborators 15 | -------------- 16 | The file evan.sty contains Evan Chen's formatting package. I've changed it to match my name, 17 | but any other changes can be found my preamble is simply at the top of main.tex. Please leave 18 | evan.sty untouched if possible. 19 | 20 | Compile in Terminal using the command 21 | 22 | latexmk -pdf -jobname=guide main.tex 23 | 24 | Contact Us 25 | -------------- 26 | Samuel Hsiang 27 | 28 | Thomas Jefferson High School for Science and Technology 29 | 30 | samuel.c.hsiang@gmail.com 31 | 32 | Alexander Wei 33 | 34 | Yang Liu 35 | -------------------------------------------------------------------------------- /main.tex: -------------------------------------------------------------------------------- 1 | \documentclass[12pt]{book} 2 | 3 | \usepackage{main} 4 | 5 | % \includeonly{chapters/fundamentals} 6 | 7 | \begin{document} 8 | 9 | \frontmatter 10 | 11 | \include{chapters/title} 12 | 13 | \include{chapters/acknowledgments} 14 | 15 | \tableofcontents 16 | 17 | \listofalgorithms 18 | 19 | \include{chapters/preface} 20 | 21 | \mainmatter 22 | 23 | \include{chapters/fundamentals} 24 | 25 | \include{chapters/big_ideas} 26 | 27 | \include{chapters/standard_ds} 28 | 29 | \include{chapters/graphs} 30 | 31 | \include{chapters/complex} 32 | 33 | \include{chapters/computational_geometry} 34 | 35 | \include{chapters/trees} 36 | 37 | \include{chapters/strings} 38 | 39 | \include{chapters/more_graphs} 40 | 41 | \include{chapters/math} 42 | 43 | \include{chapters/nonsense} 44 | 45 | \include{chapters/problems} 46 | 47 | \appendix 48 | 49 | \backmatter 50 | 51 | \end{document} 52 | -------------------------------------------------------------------------------- /chapters/title.tex: -------------------------------------------------------------------------------- 1 | \title{Crash Course Coding Companion} 2 | \author{ 3 | Samuel Hsiang \\ 4 | Thomas Jefferson High School for Science and Technology \\ 5 | \href{mailto:samuel.c.hsiang@gmail.com}{\texttt{\textup{samuel.c.hsiang@gmail.com}}} \\ 6 | \vspace{.7em} 7 | Alexander Wei \\ 8 | Phillips Exeter Academy \\ 9 | \vspace{.7em} 10 | Yang Liu \\ 11 | Massachusetts Institute of Technology 12 | } 13 | \date{\today} 14 | 15 | \maketitle 16 | 17 | \newpage 18 | ~\vfill 19 | \thispagestyle{empty} 20 | 21 | [copyright and license] 22 | 23 | % \noindent Copyright \copyright\ 2013 John Smith\\ % Copyright notice 24 | 25 | % \noindent Copyright, publishing, and licensing information should go here. I don't have anything fancy like that. It is of my (na\"{i}ve, privileged teenager) belief that knowledge should be free and easily accessible to ambitious high schoolers, but it is paramount to respect intellectual property and someone else's work. I ask that you use this material to its fullest potential, however you like, provided it doesn't completely butcher the philosophy with which I embarked on this project. 26 | 27 | % \begin{flushright} 28 | % -- Samuel 29 | % \end{flushright} 30 | 31 | % \noindent \textsc{Published by Publisher}\\ % Publisher 32 | 33 | % \noindent \textsc{book-website.com}\\ % URL 34 | 35 | \noindent \url{https://www.dropbox.com/s/z2lur71042pjaet/guide.pdf?dl=0} 36 | 37 | \noindent \url{https://github.com/alwayswimmin/cs_guide} 38 | 39 | % \noindent Licensed under the Creative Commons Attribution-NonCommercial 3.0 Unported License (the ``License''). You may not use this file except in compliance with the License. You may obtain a copy of the License at \url{http://creativecommons.org/licenses/by-nc/3.0}. Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an \textsc{``as is'' basis, without warranties or conditions of any kind}, either express or implied. See the License for the specific language governing permissions and limitations under the License.\\ % License information 40 | 41 | % \noindent \textit{First printing, March 2013} % Printing/edition date 42 | 43 | \newpage 44 | \thispagestyle{empty} 45 | % \par\vspace*{.35\textheight}{\centering For Rachel \par} 46 | -------------------------------------------------------------------------------- /chapters/preface.tex: -------------------------------------------------------------------------------- 1 | \chapter{Preface} 2 | 3 | You might have heard of \href{https://www.dropbox.com/s/z8qdndxmrmxqsam/Napkin.pdf?oref=e&n=97419869}{Evan Chen's Napkin}, a resource for olympiad math people that serves as a jumping point into higher mathematics.\footnote{In fact, I'm using Evan's template right now. Thanks Evan!} The Wikipedia articles on higher mathematics are just so dense in vocabulary and deter many smart young students from learning them before they are formally taught in a course in college. Evan's Napkin aims to provide that background necessary to leap right in. 4 | 5 | I feel the same way about computer science. For most, the ease of the AP Computer Science test means that the AP coursework is often inadequate in teaching the simplest data structures, algorithms, and big ideas necessary to approach even silver USACO problems. On the other hand, even the best reference books, like Sedgewick, are too dense and unapproachable for someone who just wants to sit down and learn something interesting.\footnote{Sedgewick, notably, is getting better. \href{http://algs4.cs.princeton.edu/home/}{Check out his online companion} to \textit{Algorithms, 4th Edition}.} The road, for many, stalls here until college. Everyone should be able to learn the simplest data structures in Java or C++ standard libraries, and someone with problem-solving experience can easily jump right into understanding algorithms and more advanced data structures. 6 | 7 | A few important notes, before we begin. 8 | 9 | \begin{itemize} 10 | 11 | \item 12 | 13 | I'm assuming some fluency in C-style syntax. If this is your first time seeing code, please look somewhere else for now. 14 | 15 | \item 16 | 17 | It is essential that you understand the motivations and the complexities behind everything we cover. I feel that this is not stressed at all in AP Computer Science and lost under the heavy details of rigorous published works. I'm avoiding what I call the heavy details because they don't focus on the math behind the computer science and lose the bigger picture. My goal is for every mathematician or programmer, after working through this, to be able to code short scripts to solve problems. Once you understand how things work, you can then move on to those details which are necessary for building larger projects. The heavy details become meaningless as languages develop or become phased out. The math and ideas behind the data structures and algorithms will last a lifetime. 18 | 19 | \item 20 | 21 | It is recommended actually code up each data structure with its most important functions or algorithm as you learn them. I truly believe the only way to build a solid foundation is to code. Do not become reliant on using the standard library (\texttt{java.util}, for instance) without understanding how the tool you are using works. 22 | 23 | \end{itemize} 24 | -------------------------------------------------------------------------------- /chapters/dp.tex: -------------------------------------------------------------------------------- 1 | \chapter{Dynamic Programming} 2 | 3 | The idea behind dynamic programming is to avoid doing the same thing twice. Two nodes in the search tree described in the complete search techniques might very well represent the same state. For example, two different sets of initial moves could result in the same chessboard configuration, and if we already calculated who has the winning or losing position in that configuration, we ought to remember that fact somehow so we don't need to calculate it again. Of course, however, actually keeping track of every possible game state is intractable by both time and memory constraints. 4 | 5 | Here's a much more reasonable problem. Given a sequence of $N \le 10,000$ integers, what is the maximum decreasing subsequence? A subsequence does not have to consist of consecutive terms in the original sequence. 6 | 7 | \begin{center} 8 | { 9 | \begin{tikzpicture}[ 10 | thick, 11 | myrect/.style={ 12 | draw, 13 | fill=myseagreen, 14 | rectangle split, 15 | rectangle split horizontal, 16 | rectangle split parts=#1, 17 | rectangle split part align=left, 18 | text width=5ex, 19 | text centered 20 | }, 21 | mycallout/.style={ 22 | shape=rectangle callout, 23 | rounded corners, 24 | fill=mysalmon, 25 | callout absolute pointer={#1}, 26 | callout pointer width=1cm 27 | } 28 | ] 29 | 30 | \node[myrect=6] 31 | (array) 32 | { 33 | \strut 6 34 | \nodepart{two} \strut 9 35 | \nodepart{three} \strut 8 36 | \nodepart{four} \strut 4 37 | \nodepart{five} \strut 7 38 | \nodepart{six} \strut 5 39 | }; 40 | \foreach \Valor [count=\Valori from 1] in {one ,two ,three , four , five , six } 41 | \node[below] at (array.\Valor south) {\Valori}; 42 | 43 | \end{tikzpicture} 44 | } 45 | \end{center} 46 | 47 | The natural complete search approach would be to use recursion, or DFS. When we process an element in the list, we recursively process all elements that come after it and choose the one that gives the maximum subsequence. 48 | 49 | \noindent \begin{minipage}{\textwidth} 50 | \begin{algorithmic} 51 | \Function{Process}{$i$} 52 | \State $max \gets 0$ 53 | \For{$j\equiv i+1,N$} 54 | \If{$value(i) > value(j)$} 55 | \State $x \gets \Call{Process}{$j$}$ 56 | \If{$x > max$} 57 | \State $max \gets x$ 58 | \EndIf 59 | \EndIf 60 | \EndFor 61 | \State \Return $max + 1$ 62 | \EndFunction 63 | \end{algorithmic} 64 | \end{minipage} 65 | 66 | However, this algorithm is exponential. In the worst case, it is $O(2^N)$. We notice a lot of repetition: processing the 9 in the list above, for example, requires finding the longest subsequences beginning with 8, 4, 7, and 5, while processing 8 requires finding subsequences for 4, 7, and 5. It seems silly to do the same task twice, so we'll keep track of the length of the longest subsequence in a separate array. 67 | 68 | \noindent \begin{minipage}{\textwidth} 69 | \begin{algorithmic} 70 | \Function{Process}{$i$} 71 | \If{$i$ has already been processed} 72 | \State \Return $dp(i)$ 73 | \EndIf 74 | \State $max \gets 0$ 75 | \For{$j\equiv i+1,N$} 76 | \If{$value(i) > value(j)$} 77 | \State $x \gets \Call{Process}{j}$ 78 | \If{$x > max$} 79 | \State $max \gets x$ 80 | \EndIf 81 | \EndIf 82 | \EndFor 83 | \State $dp(i) \gets max + 1$ 84 | \State \Return $max + 1$ 85 | \EndFunction 86 | \end{algorithmic} 87 | \end{minipage} 88 | 89 | This reduces the complexity of the algorithm to $O(n^2)$. Note that to process an index, we must process first all later indices. This imposes a natural ordering in which to process the indices: in reverse. This idea lends itself to a nice iterative solution. 90 | 91 | \noindent \begin{minipage}{\textwidth} 92 | \begin{algorithmic} 93 | \For{$i \equiv N,1$} 94 | \Comment $i$ goes in reverse 95 | \State $max \gets 0$ 96 | \For{$j\equiv i+1,N$} 97 | \If{$value(i) > value(j)$} 98 | \If{$dp(j) > max$} 99 | \State $max \gets dp(j)$ 100 | \EndIf 101 | \EndIf 102 | \EndFor 103 | \State $dp(i) \gets max + 1$ 104 | \EndFor 105 | \end{algorithmic} 106 | \end{minipage} 107 | 108 | The answer to the original problem is then the maximum value of $dp(i)$ for all $i$. For this specific problem, it's relatively easy to speed up the algorithm to $O(\log{n})$ by replacing the linear search with something else. 109 | 110 | The integer knapsack problem is another example where dynamic programming may be useful. \textit{Knapsack problems} are a family of problems with the following form: 111 | 112 | We are given a list of $K$ objects each assigned an availability, a size, and a value. We have a total amount of ``space'' available in our knapsack and need to find the set of objects from our list that maximizes the total the value of objects in the set such that the total size does not exceed the space and the number of times we take one particular object does not exceed its availability. 113 | 114 | Dynamic programming yields a straightforward $O(NK)$ solution. See if you can find it. Note, however, if $N$ is very large, this solution is no longer practical. In general, the knapsack problem is NP-complete, so don't think dynamic programming works on everything! 115 | 116 | Brian Dean compiled some standard dynamic programming problems with animations and analyses. Practice dynamic programming here: \url{http://people.cs.clemson.edu/~bcdean/dp_practice/} 117 | -------------------------------------------------------------------------------- /main.sty: -------------------------------------------------------------------------------- 1 | \usepackage{emptypage} 2 | 3 | \usepackage[T1]{fontenc} 4 | \usepackage[utf8x]{inputenc} 5 | \usepackage{lmodern} 6 | 7 | \usepackage{pdflscape} 8 | 9 | \usepackage{amsmath,amssymb,amsthm} 10 | 11 | \usepackage{verbatim} 12 | 13 | % \newcommand{\nothtml}{\comment} 14 | 15 | \usepackage{listings} 16 | \usepackage[hyperfootnotes=false]{hyperref} 17 | \usepackage{exercise} 18 | 19 | \usepackage{algorithm} 20 | \usepackage{algorithmicx} 21 | \usepackage[noend]{algpseudocode} 22 | 23 | \usepackage[toc,page]{appendix} 24 | 25 | \usepackage{answers} 26 | 27 | \usepackage[secthm,mdthm]{evan} 28 | 29 | \lhead{Hsiang, Wei, Liu} 30 | 31 | \usepackage{graphicx} 32 | 33 | \usepackage{color} 34 | 35 | \definecolor{myseagreen}{RGB}{88,197,191} 36 | \definecolor{mysalmon}{RGB}{255,160,122} 37 | \definecolor{myred}{RGB}{255,102,102} 38 | \definecolor{mypurple}{RGB}{225,145,255} 39 | \definecolor{myblack}{RGB}{0,0,0} 40 | \definecolor{mywhite}{RGB}{255,255,255} 41 | 42 | \usepackage{tikz} 43 | 44 | \usetikzlibrary{calc,shapes.multipart,chains,arrows,positioning} 45 | 46 | \tikzstyle{vertex}=[draw,fill=myseagreen,circle,minimum size=24pt,inner sep=0pt] 47 | 48 | \tikzstyle{splitvertex}=[draw,fill=myseagreen,circle split,minimum size=24pt] 49 | 50 | \usetikzlibrary{ 51 | shapes.multipart, 52 | matrix, 53 | positioning, 54 | shapes.callouts, 55 | shapes.arrows, 56 | shapes.geometric, 57 | decorations.shapes, 58 | shapes, 59 | fit, 60 | arrows, 61 | positioning, 62 | trees, 63 | mindmap, 64 | calc} 65 | 66 | \tikzset{ 67 | squarecross/.style={ 68 | draw, rectangle,minimum size=18pt, fill=myseagreen, 69 | inner sep=0pt, text=black, 70 | path picture = { 71 | \draw[black] 72 | (path picture bounding box.north west) -- 73 | (path picture bounding box.south east) 74 | (path picture bounding box.south west) -- 75 | (path picture bounding box.north east); 76 | } 77 | } 78 | } 79 | 80 | \lstset{language=Java} 81 | 82 | \definecolor{codegreen}{rgb}{0,0.6,0} 83 | \definecolor{codegray}{rgb}{0.5,0.5,0.5} 84 | \definecolor{codepurple}{rgb}{0.58,0,0.82} 85 | \definecolor{backcolour}{rgb}{0.95,0.95,0.92} 86 | 87 | \makeatletter 88 | \@ifpackageloaded{tex4ht} 89 | { 90 | \newcommand{\mylstlisting}{\verbatim} 91 | } 92 | { 93 | \lstnewenvironment{mylstlisting}[1][]% 94 | {\noindent\minipage{\linewidth}\medskip 95 | \lstset{ 96 | backgroundcolor=\color{backcolour}, 97 | commentstyle=\color{codegreen}, 98 | keywordstyle=\color{magenta}, 99 | numberstyle=\tiny\color{codegray}, 100 | stringstyle=\color{codepurple}, 101 | basicstyle=\footnotesize, 102 | breakatwhitespace=false, 103 | breaklines=true, 104 | captionpos=t, 105 | keepspaces=true, 106 | numbers=left, 107 | numbersep=5pt, 108 | showspaces=false, 109 | showstringspaces=false, 110 | showtabs=false, 111 | tabsize=4, 112 | basicstyle=\ttfamily\footnotesize, 113 | frame=single,#1}} 114 | {\endminipage} 115 | } 116 | \makeatother 117 | 118 | \raggedbottom 119 | 120 | \usepackage{footnote} 121 | \makesavenoteenv{tabular} 122 | \makesavenoteenv{table} 123 | 124 | \renewcommand{\arraystretch}{1.5} 125 | 126 | % Alex's stuff. 127 | 128 | % Make \emph bold. 129 | \makeatletter 130 | \DeclareRobustCommand{\em}{% 131 | \@nomath\em \if b\expandafter\@car\f@series\@nil 132 | \normalfont \else \bfseries \fi} 133 | \makeatother 134 | 135 | \usepackage{tabularx} % For more sophisticated tables. 136 | 137 | \newenvironment{typewriter}{ 138 | \ttfamily 139 | \fontdimen2\font=0.4em% interword space 140 | \fontdimen3\font=0.2em% interword stretch 141 | \fontdimen4\font=0.1em% interword shrink 142 | \fontdimen7\font=0.1em% extra space 143 | \hyphenchar\font=`\-% allowing hyphenation 144 | }{\par} 145 | 146 | % Commands for problem statements and sample cases. 147 | 148 | % Command for generating problem statements. Optional argument is title. Others 149 | % are problem statement, input specification, output specification, and source. 150 | \newcommand{\statement}[7][]{ 151 | {\sffamily 152 | \if\relax\detokenize{#1}\relax\else 153 | \noindent{\large \textbf{#1}} 154 | \fi 155 | \hfill{\small\emph{Time limit: #2. Memory limit: #3.}}\par\nopagebreak 156 | \noindent #4 157 | \par\vspace{0.2em} 158 | \if\relax\detokenize{#5}\relax\else 159 | \noindent\textbf{Input}\par 160 | \noindent #5 161 | \fi 162 | \par\vspace{0.2em} 163 | \if\relax\detokenize{#6}\relax\else 164 | \noindent\textbf{Output}\par 165 | \noindent #6 166 | \fi 167 | \par\vspace{0.2em} 168 | \if\relax\detokenize{#7}\relax\else 169 | \noindent[Adapted from #7.] 170 | \fi 171 | \par\vspace{0.2em} 172 | } 173 | } 174 | 175 | % Dummy command for \sample. 176 | \newcommand{\sampel}[2]{ 177 | {\sffamily 178 | \noindent\begin{tabularx}{\textwidth}{X|X} 179 | \textbf{Sample Input} & \textbf{Sample Output} \\ 180 | {\ttfamily #1} & {\ttfamily #2}% 181 | \end{tabularx} 182 | \vspace{0.2em} 183 | }\endgroup\par 184 | } 185 | 186 | % Make \sample respect newlines. 187 | \newcommand{\sample}{ 188 | \begingroup\obeylines\sampel 189 | } 190 | -------------------------------------------------------------------------------- /chapters/computational_geometry.tex: -------------------------------------------------------------------------------- 1 | \chapter{Computational Geometry} 2 | 3 | For actual geometry problems, and not graph theory problems hiding in the plane. 4 | 5 | I'm too lazy to actually write this section right now so here are some useful links from the USACO Training Pages. 6 | 7 | \url{https://www.dropbox.com/s/nqzk63bjby1iaq9/Computational\%20Geometry.pdf?dl=0} 8 | 9 | \url{https://www.dropbox.com/s/ykf65dk6sefb6zk/2-D\%20Convex\%20Hull.pdf?dl=0} 10 | 11 | These essentially cover anything I would want to say in this chapter anyway, so I'll likely fill this chapter out last. 12 | 13 | \section{Basic Tools} 14 | 15 | \subsubsection{Cross Product} 16 | 17 | \subsubsection{Dot Product} 18 | 19 | \subsubsection{$\tan^{-1}$, \texttt{atan2}} 20 | 21 | \section{Formulas} 22 | 23 | \subsection{Area} 24 | 25 | \subsection{Distance} 26 | 27 | \subsection{Configuration} 28 | 29 | \subsection{Intersection} 30 | 31 | \section{Convex Hull} 32 | 33 | \section{Sweep Line} 34 | 35 | Oftentimes when trying to solve geometry problems, or combinatorial problems that we give a geometric interpretation, we have something that involves one or two dimensions. If this is difficult to deal with directly, we can use a sweep line to reduce the dimension of the problem. 36 | 37 | Sweep lines are best explained with an example. Suppose we're given several intervals on the number line, and we want to find the length of their union. If we wanted to, we could use a segment tree with lazy propagation to insert the intervals and calculate the total length. However, a sweep line gives us a much easier approach. Imagine a vertical beam of light sweeping from $-\infty$ to $\infty$ on the number line. At any given time, we keep track of the number of intervals that are hit by this beam. This takes the problem from one dimension to zero dimensions. Our answer is then the length over which our beam hit a positive number of intervals. 38 | 39 | Our implementation of a sweep line simulates this process. If we sort the endpoints of our intervals by their positions on the number line, we can simply increment our counter each time we encounter a ``start'' endpoint and decrement our counter each time we encounter an ``end'' endpoint. We can look at the regions between consecutive endpoints, and add to our result if our counter is positive in that region. 40 | 41 | \begin{center}\begin{tikzpicture} 42 | \coordinate (A) at (2,1); 43 | \coordinate (B) at (6,1); 44 | \coordinate (C) at (0,2); 45 | \coordinate (D) at (3,2); 46 | \coordinate (E) at (5,2); 47 | \coordinate (F) at (7,2); 48 | \coordinate (G) at (1,3); 49 | \coordinate (H) at (5,3); 50 | \draw[fill] (A) circle [radius=2pt]; 51 | \draw[fill] (B) circle [radius=2pt]; 52 | \draw[fill] (C) circle [radius=2pt]; 53 | \draw[fill] (D) circle [radius=2pt]; 54 | \draw[fill] (E) circle [radius=2pt]; 55 | \draw[fill] (F) circle [radius=2pt]; 56 | \draw[fill] (G) circle [radius=2pt]; 57 | \draw[fill] (H) circle [radius=2pt]; 58 | \draw[ultra thick] (A) -- (B); 59 | \draw[very thick] (C) -- (D); 60 | \draw[very thick] (E) -- (F); 61 | \draw[ultra thick] (G) -- (H); 62 | 63 | \draw[<->] (-1,0) -- (8,0); 64 | \foreach \x in {0, 1, 2, 3, 4, 5, 6, 7} 65 | \draw[shift={(\x,0)}] (0,3pt) -- (0,-3pt); 66 | 67 | \draw[<->, very thick, dashed, color=red] (3.75,-1) -- (3.75,4); 68 | \end{tikzpicture}\end{center} 69 | 70 | One other way to interpret sweep lines is to consider time as the dimension we sweep along. For the example above, this would mean that each interval appears and then disappears on our beam, existing only when the time is between its endpoints. Although time may not seem useful for our one dimensional problem, this type of thinking helps in higher dimensions. 71 | 72 | Most sweep lines that you use won't be as simple as this. Sweep line problems usually involve reducing a two dimensional problem to a one dimensional problem and require maintaining data structures such as BBSTs or segment trees along that dimension. This technique also generalizes to higher dimensions---to solve a three dimensional problem, we can sweep along one dimension and use two dimensional data structures to maintain the other two dimensions. 73 | 74 | To finish, let's go over another example, Cow Rectangles from USACO 2015 January: 75 | 76 | \texttt{ 77 | \fontdimen2\font=0.4em% interword space 78 | \fontdimen3\font=0.2em% interword stretch 79 | \fontdimen4\font=0.1em% interword shrink 80 | \fontdimen7\font=0.1em% extra space 81 | \hyphenchar\font=`\-% allowing hyphenation 82 | The locations of Farmer John's $N$ cows ($1 \le N \le 500$) are described by distinct points in the 2D plane. The cows belong to two different breeds: Holsteins and Guernseys. Farmer John wants to build a rectangular fence with sides parallel to the coordinate axes enclosing only Holsteins, with no Guernseys (a cow counts as enclosed even if it is on the boundary of the fence). Among all such fences, Farmer John wants to build a fence enclosing the maximum number of Holsteins. And among all these fences, Farmer John wants to build a fence of minimum possible area. Please determine this area. A fence of zero width or height is allowable.} 83 | 84 | The first observation we should make is that we want a Holstein on every side of the fence. Otherwise, we would be able to decrease the area of our fence without decreasing the number of Holsteins. Even with our observation, this problem still seems hard to deal with. We can make it easier by adding another constraint: Suppose we knew which Holstein was on the leftmost boundary of our fence. Since we added the constraint to the $x$-dimension, we naturally want to figure out where our rightmost Holstein is next. We can do this with a sweep line moving to the right. 85 | 86 | The reason we want a sweep line here is because for any given rightmost cow, we want to know information about the Guernseys and Holsteins in between. With a sweep line, we can collect and maintain all this data by processing the cows from left to right. For example, whenever we see a Guernsey, we have to limit the $y$-coordinates that our Holsteins can take. And whenever we see a Holstein, we have to consider this Holstein as our rightmost cow and also store it in case we include it in a fence later on. 87 | 88 | What remains is to find an appropriate data structure to track all this. A set data structure (STL set or Java TreeMap) turns out to be enough. We can insert the Holsteins, sorted by $y$-coordinate, and delete one whenever any rectangle bounded by that Holstein, the leftmost Holstein, and our sweep line includes a Guernsey. Thus we take $O(n \log n)$ time to sweep, giving us an $O(n^2 \log n)$ solution overall. 89 | 90 | This type of analysis is pretty representative of sweep line problems. Whether you're given rectangles, line segments or even polygons, you want to think about how you can reduce the dimension and obtain a tractable problem. Note that sweep lines don't always have to move along some axis. Radial sweep lines (sweeping by rotating) and sweeping at an angle (rotating the plane by $45^\circ$) also work. 91 | -------------------------------------------------------------------------------- /chapters/implementation.tex: -------------------------------------------------------------------------------- 1 | \chapter{Implementation Tricks} 2 | 3 | \subsection{Comparison} 4 | 5 | We'll finish our section on sorting by taking a look at how Java and C++ sorts objects. We also provide a brief glimpse into using standard data structures that require a natural ordering on the objects within the container. 6 | 7 | In Java, there is a built-in sort method, \texttt{Arrays.sort()}, in \texttt{java.util.Arrays}, which takes an array and sorts it. To sort an array \texttt{int array[]}, we call \texttt{Arrays.sort(array)}. Java always sorts least to greatest. 8 | 9 | If we want to sort an array of our own objects, we have to first implement the \texttt{Comparable} interface. This is how Java orders a set of objects. Implementing \texttt{Comparable} requires support of the method \texttt{int compareTo(Object o)}. If \texttt{a.compareTo(b) < 0}, then \texttt{a} comes before \texttt{b}; if \texttt{a.compareTo(b) > 0}, then \texttt{a} comes after \texttt{b}; if \texttt{a.compareTo(b) == 0}, then \texttt{a} is considered equal to \texttt{b}. (\texttt{a.compareTo(b) == 0} should be equivalent to \texttt{a.equals(b)}.) For example, consider the class \texttt{MyPair}: 10 | 11 | \begin{mylstlisting} 12 | class MyPair implements Comparable { 13 | int x; 14 | int y; 15 | // sort by x-coordinate first 16 | public int compareTo(Object o) { 17 | MyPair c = (MyPair) o; 18 | if(x < c.x) return -1; 19 | if(x > c.x) return 1; 20 | // if x-coordinates equal, compare y 21 | if(y < c.x) return -1; 22 | if(y > c.x) return 1; 23 | return 0; // equal 24 | } 25 | } 26 | \end{mylstlisting} 27 | 28 | If you know generics (Section \ref{sec:generics}), we can clean this up a bit. Note that no casting is required anymore. 29 | 30 | \begin{mylstlisting} 31 | class MyPair implements Comparable { 32 | int x; 33 | int y; 34 | public int compareTo(MyPair c) { 35 | if(x < c.x) return -1; 36 | if(x > c.x) return 1; 37 | if(y < c.x) return -1; 38 | if(y > c.x) return 1; 39 | return 0; 40 | } 41 | } 42 | \end{mylstlisting} 43 | 44 | We can sort much more than just arrays. Java provides us many data structures we can use, much more than just arrays. In this section, I'll mention briefly the \texttt{ArrayList} for Java and the \texttt{vector} for C++. If you don't know what either of these are, you can just pretend like they are arrays for now; they are very similar and will be covered in detail in a later chapter. To sort these more complex data structures in Java, we use \texttt{Collections.sort()}. 45 | 46 | Suppose we wanted to code a custom comparison operation for something that already exists. For example, suppose we wanted to sort an array of \texttt{ArrayList}s based on the sum of the elements in the \texttt{ArrayList}. We clearly don't want to code an entire \texttt{ArrayList} just because we want to sort it in a special way. One way we can easily resolve this is by using \texttt{extends}. 47 | 48 | \begin{mylstlisting} 49 | class MyArrayList extends ArrayList implements Comparable { 50 | public int compareTo(MyArrayList c) { 51 | int sum = 0; 52 | for(int k = 0; k < size(); k++) { 53 | sum += get(k); 54 | } 55 | int csum = 0; 56 | for(int k = 0; k < c.size(); k++) { 57 | csum += c.get(k); 58 | } 59 | if(sum < csum) return -1; 60 | if(sum == csum) return 0; 61 | return 1; 62 | } 63 | } 64 | \end{mylstlisting} 65 | 66 | Alternatively, we could use a \texttt{Comparator}. This is considered much better practice than the hack I presented earlier by coding a new class with \texttt{extends}. \texttt{Comparator}s are especially useful if we are to compare the same object different ways in the same program. A \texttt{Comparator} is an entirely separate class that takes two objects and determines which ought to come before the other in the ordering. Here is the \texttt{Comparator} implementation of the example I showed above. 67 | 68 | \begin{mylstlisting} 69 | class Cmp implements Comparator> { 70 | public int compare(ArrayList o1, ArrayList o2) { 71 | int sum1 = 0; 72 | for(int k = 0; k < o1.size(); k++) { 73 | sum1 += o1.get(k); 74 | } 75 | int sum2 = 0; 76 | for(int k = 0; k < o2.size(); k++) { 77 | sum2 += o2.get(k); 78 | } 79 | if(sum1 < sum2) return -1; 80 | if(sum1 == sum2) return 0; 81 | return 1; 82 | } 83 | } 84 | \end{mylstlisting} 85 | 86 | Now, when we sort, we'll need to pass our \texttt{Comparator} in addition to the data structure to be sorted. For example, we use the code \texttt{Arrays.sort(array, new Cmp())}. When we use a data structure that requires a custom comparison, we use a different constructor: \texttt{new TreeSet>(new Cmp())}. 87 | 88 | In C++, comparing objects is slightly different. We have three methods, and all can be used with \texttt{sort}, which is in \texttt{}, or with any other standard library function or data structure which requires an ordering. As in Java, sorting is done least to greatest. 89 | 90 | The first method is to implement the method \texttt{operator<} in a \texttt{struct} or \texttt{class} we coded ourselves. Here's an example \texttt{struct my\_pair} which is equivalent to the above Java \texttt{class MyPair}. 91 | 92 | \begin{mylstlisting}[language=C++] 93 | struct my_pair { 94 | int x; 95 | int y; 96 | friend bool operator<(const my_pair& lhs, const my_pair& rhs) { 97 | return lhs.x < rhs.x || lhs.x == rhs.x && lhs.y < rhs.y; 98 | } 99 | }; 100 | \end{mylstlisting} 101 | 102 | Suppose we have an array \texttt{my\_pair array[]}. We can then call \texttt{sort(array, array + n)} to sort the first \texttt{n} elements of the array. If we have a different data structure, like \texttt{vector v}, we can sort it using \texttt{sort(v.begin(), v.end())}. 103 | 104 | The second method is to use a custom comparison function. This is very similar to what we would put in function contained in the Java \texttt{Comparator}, but is distinct from the \texttt{Comparator} itself as it is only a function and not a class. Suppose we wanted to sort pairs primarily increasing in the first element and secondarily decreasing in the second. It seems inefficient to code a new pair class when C++ provides one, so we can simply use a custom comparison function to get around this. 105 | 106 | \begin{mylstlisting}[language=C++] 107 | typedef std::pair pii; 108 | bool cmp(const pii& lhs, const pii& rhs) { 109 | return lhs.x < rhs.x || lhs.x == rhs.x && lhs.y > rhs.y; 110 | } 111 | \end{mylstlisting} 112 | 113 | \texttt{cmp}, like \texttt{operator<}, should return \texttt{true} when \texttt{lhs} is considered to come before \texttt{rhs} in the ordering. To use \texttt{sort} here, we pass a third argument containing our custom comparison: \texttt{sort(array, array + n, cmp)}. Passing a custom comparison function to a standard library data structure is possible but unfortunately quite confusing and will not be covered here. This can be done more easily using the final method. 114 | 115 | The final method uses a functor, which is an object that can also work as a function. This is done by implementing the function \texttt{operator()()}. In the case of sorting, the functor works somewhat like a Java \texttt{Comparator}. 116 | 117 | \begin{mylstlisting}[language=C++] 118 | typedef std::pair pii; 119 | struct cmp { 120 | bool operator()(const pii& lhs, const pii& rhs) { 121 | return lhs.x < rhs.x || lhs.x == rhs.x && lhs.y > rhs.y; 122 | } 123 | }; 124 | \end{mylstlisting} 125 | 126 | The functor method works just as the custom comparison function for \texttt{sort}: \texttt{sort(array, array + n, cmp)}. However, since it is now an object, it can be passed as a template argument to standard library containers much more easily than the comparison function can. For example, for the binary search tree implemented by C++, we can pass a custom comparison using \texttt{set s}. 127 | 128 | -------------------------------------------------------------------------------- /evan.sty: -------------------------------------------------------------------------------- 1 | \ProvidesPackage{evan} 2 | % Last updated August 7, 2015 3 | \usepackage{amsmath,amssymb,amsthm} 4 | 5 | % Default Arguments 6 | % We include "Evan" in all of these to make sure 7 | % that they don't collide with anything in external packages 8 | \newif\ifevanfancy\evanfancytrue 9 | \newif\ifevanhdr\evanhdrtrue 10 | \newif\ifevanhref\evanhreftrue 11 | \newif\ifevansetup\evansetuptrue 12 | \newif\ifevanthm\evanthmtrue 13 | \newif\ifevansecthm\evansecthmfalse 14 | \newif\ifevanht\evanhtfalse 15 | \newif\ifevanpkg\evanpkgtrue 16 | \newif\ifevanpdf\evanpdftrue 17 | \newif\ifevantitling\evantitlingtrue 18 | \newif\ifevanauthor\evanauthortrue 19 | \newif\ifevanchinese\evanchinesefalse 20 | \newif\ifevanmdthm\evanmdthmfalse 21 | \newif\ifevandiagrams\evandiagramsfalse 22 | 23 | %Receive Arguments 24 | \DeclareOption{chinese}{\evanhreffalse\evanchinesetrue} % Chinese support 25 | % allow href to override this one 26 | 27 | \DeclareOption{fancy}{\evanfancytrue} 28 | \DeclareOption{nofancy}{\evanfancyfalse} 29 | \DeclareOption{hdr}{\evanhdrtrue} 30 | \DeclareOption{nohdr}{\evanhdrfalse} 31 | \DeclareOption{href}{\evanhreftrue} 32 | \DeclareOption{nohref}{\evanhreffalse} 33 | 34 | \DeclareOption{nosetup}{\evansetupfalse} 35 | \DeclareOption{thm}{\evanthmtrue} 36 | \DeclareOption{nothm}{\evanthmfalse} 37 | \DeclareOption{secthm}{\evansecthmtrue} 38 | \DeclareOption{ht}{\evanhttrue} 39 | \DeclareOption{nopdf}{\evanpdffalse} 40 | \DeclareOption{nopkg}{\evanpkgfalse} 41 | \DeclareOption{oldtitle}{\evantitlingfalse} 42 | \DeclareOption{newtitle}{\evantitlingtrue} 43 | \DeclareOption{noauthor}{\evanauthorfalse} 44 | 45 | \DeclareOption{mdthm}{\evanmdthmtrue} 46 | \DeclareOption{nomdthm}{\evanmdthmfalse} 47 | \DeclareOption{diagrams}{\evandiagramstrue} 48 | \DeclareOption{nodiagrams}{\evandiagramsfalse} 49 | 50 | \ProcessOptions\relax 51 | 52 | % if packages not loaded, turn off mdthm 53 | \ifevanpkg\else\evanmdthmfalse\fi 54 | 55 | %Small commands 56 | \newcommand{\myqed}{\textsc{Q.e.d.}} 57 | \newcommand{\cbrt}[1]{\sqrt[3]{#1}} 58 | \newcommand{\floor}[1]{\left\lfloor #1 \right\rfloor} 59 | \newcommand{\ceiling}[1]{\left\lceil #1 \right\rceil} 60 | \newcommand{\mailto}[1]{\href{mailto:#1}{\texttt{#1}}} 61 | \newcommand{\ol}[1]{\overline{#1}} 62 | \newcommand{\ul}[1]{\underline{#1}} 63 | \newcommand{\eps}{\varepsilon} 64 | %\renewcommand{\iff}{\Leftrightarrow} 65 | %\renewcommand{\implies}{\Rightarrow} 66 | \newcommand{\hrulebar}{ 67 | \par\hspace{\fill}\rule{0.95\linewidth}{.7pt}\hspace{\fill} 68 | \par\nointerlineskip \vspace{\baselineskip} 69 | } 70 | \def\half{\frac{1}{2}} 71 | 72 | %For use in author command 73 | \newcommand{\plusemail}[1]{\\ \normalfont \texttt{\mailto{#1}}} 74 | 75 | %More commands and math operators 76 | \DeclareMathOperator{\cis}{cis} 77 | \DeclareMathOperator{\lcm}{lcm} 78 | 79 | %Convenient Environments 80 | \newenvironment{soln}{\begin{proof}[Solution]}{\end{proof}} 81 | \newenvironment{parlist}{\begin{inparaenum}[(i)]}{\end{inparaenum}} 82 | \newenvironment{gobble}{\setbox\z@\vbox\bgroup}{\egroup} 83 | 84 | %Inequalities 85 | \newcommand{\cycsum}{\sum_{\mathrm{cyc}}} 86 | \newcommand{\symsum}{\sum_{\mathrm{sym}}} 87 | \newcommand{\cycprod}{\prod_{\mathrm{cyc}}} 88 | \newcommand{\symprod}{\prod_{\mathrm{sym}}} 89 | 90 | %From H113 "Introduction to Abstract Algebra" at UC Berkeley 91 | \def\CC{\mathbb C} 92 | \def\FF{\mathbb F} 93 | \def\NN{\mathbb N} 94 | \def\QQ{\mathbb Q} 95 | \def\RR{\mathbb R} 96 | \def\ZZ{\mathbb Z} 97 | \newcommand{\charin}{\text{ char }} 98 | \DeclareMathOperator{\sign}{sign} 99 | \DeclareMathOperator{\Aut}{Aut} 100 | \DeclareMathOperator{\Inn}{Inn} 101 | \DeclareMathOperator{\Syl}{Syl} 102 | 103 | %From Kiran Kedlaya's "Geometry Unbound" 104 | \def\abs#1{\left\lvert #1 \right\rvert} 105 | \def\norm#1{\left\lVert #1 \right\rVert} 106 | \def\dang{\measuredangle} %% Directed angle 107 | \def\line#1{\overleftrightarrow{#1}} 108 | \def\ray#1{\overrightarrow{#1}} 109 | \def\seg#1{\overline{#1}} 110 | \def\arc#1{\wideparen{#1}} 111 | 112 | %From M275 "Topology" at SJSU 113 | \newcommand{\id}{\mathrm{id}} 114 | \newcommand{\taking}[1]{\xrightarrow{#1}} 115 | \newcommand{\inv}{^{-1}} 116 | 117 | %From M170 "Introduction to Graph Theory" at SJSU 118 | \DeclareMathOperator{\diam}{diam} 119 | \DeclareMathOperator{\ord}{ord} 120 | \newcommand{\defeq}{\overset{\mathrm{def}}{=}} 121 | 122 | %From the USAMO .tex filse 123 | \def\st{^{\text{st}}} 124 | \def\nd{^{\text{nd}}} 125 | \def\rd{^{\text{rd}}} 126 | \def\th{^{\text{th}}} 127 | \def\dg{^\circ} 128 | \def\ii{\item} 129 | 130 | % From Math 55 and Math 145 at Harvard 131 | \newenvironment{subproof}[1][Proof]{% 132 | \begin{proof}[#1] \renewcommand{\qedsymbol}{$\blacksquare$}}% 133 | {\end{proof}} 134 | 135 | \newcommand{\PP}{\mathcal P} % power set 136 | \newcommand{\liff}{\leftrightarrow} 137 | \newcommand{\lthen}{\rightarrow} 138 | \newcommand{\opname}{\operatorname} 139 | \newcommand{\surjto}{\twoheadrightarrow} 140 | \newcommand{\injto}{\hookrightarrow} 141 | \newcommand{\On}{\mathrm{On}} % ordinals 142 | \DeclareMathOperator{\img}{Im} % Image 143 | \DeclareMathOperator{\rank}{rank} 144 | \DeclareMathOperator{\Spec}{Spec} % spectrum 145 | \DeclareMathOperator{\Tr}{Tr} % trace 146 | \DeclareMathOperator{\pr}{pr} % projection 147 | \DeclareMathOperator{\ext}{ext} % extension 148 | \DeclareMathOperator{\pred}{pred} % predecessor 149 | \DeclareMathOperator{\dom}{dom} % domain 150 | \DeclareMathOperator{\ran}{ran} % range 151 | \DeclareMathOperator{\Hom}{Hom} % homomorphism 152 | \DeclareMathOperator{\End}{End} % endomorphism 153 | 154 | 155 | %Asy commands 156 | \usepackage{asymptote} 157 | \begin{asydef} 158 | import olympiad; 159 | import cse5; 160 | pointpen = black; 161 | pathpen = black; 162 | pathfontpen = black; 163 | anglepen = black; 164 | anglefontpen = black; 165 | pointfontsize = 9; 166 | defaultpen(fontsize(9pt)); 167 | size(6cm); // set a reasonable default 168 | usepackage("amsmath"); 169 | usepackage("amssymb"); 170 | \end{asydef} 171 | 172 | 173 | %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 174 | %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 175 | %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 176 | 177 | \ifevansetup 178 | %Setup Title/Auth/Date 179 | \ifevanauthor 180 | \AtBeginDocument{% 181 | % \author{Samuel Hsiang} 182 | \date{\today} 183 | } 184 | \fi 185 | %Href 186 | \ifevanpkg 187 | \ifevanhref 188 | \usepackage{hyperref} 189 | \fi 190 | \usepackage[nameinlink]{cleveref} 191 | \fi 192 | 193 | \ifevanthm 194 | %New Theorem Styles 195 | \theoremstyle{plain} 196 | 197 | %Braching here: the option secthm changes theorems to be labelled by section 198 | \ifevanmdthm 199 | \usepackage{thmtools} 200 | \usepackage[framemethod=TikZ]{mdframed} 201 | 202 | \mdfdefinestyle{mdbluebox}{% 203 | roundcorner = 10pt, 204 | linewidth=1pt, 205 | skipabove=12pt, 206 | innerbottommargin=9pt, 207 | skipbelow=2pt, 208 | linecolor=blue, 209 | nobreak=true 210 | } 211 | \declaretheoremstyle[ 212 | headfont=\sffamily\bfseries, 213 | mdframed={style=mdbluebox}, 214 | headpunct={\\[3pt]}, 215 | postheadspace={0pt} 216 | ]{thmbluebox} 217 | 218 | \mdfdefinestyle{mdrecbox}{% 219 | linewidth=0.5pt, 220 | skipabove=12pt, 221 | frametitleaboveskip=5pt, 222 | frametitlebelowskip=0pt, 223 | skipbelow=2pt, 224 | frametitlefont=\bfseries, 225 | innertopmargin=4pt, 226 | innerbottommargin=8pt, 227 | nobreak=true, 228 | } 229 | \declaretheoremstyle[ 230 | headfont=\bfseries, 231 | mdframed={style=mdrecbox}, 232 | headpunct={\\[3pt]}, 233 | postheadspace={0pt}, 234 | ]{thmrecbox} 235 | 236 | \ifevansecthm 237 | \declaretheorem[% 238 | style=thmbluebox,name=Theorem,numberwithin=section]{theorem} 239 | \else 240 | \declaretheorem[% 241 | style=thmbluebox,name=Theorem]{theorem} 242 | \fi 243 | \declaretheorem[style=thmbluebox,name=Lemma,sibling=theorem]{lemma} 244 | \declaretheorem[style=thmbluebox,name=Proposition,sibling=theorem]{proposition} 245 | \declaretheorem[style=thmbluebox,name=Corollary,sibling=theorem]{corollary} 246 | \else 247 | \ifevansecthm 248 | \newtheorem{theorem}{Theorem}[section] 249 | \else 250 | \newtheorem{theorem}{Theorem} 251 | \fi 252 | \newtheorem{lemma}[theorem]{Lemma} 253 | \newtheorem{proposition}[theorem]{Proposition} 254 | \newtheorem{corollary}[theorem]{Corollary} 255 | \newtheorem*{theorem*}{Theorem} 256 | \newtheorem*{lemma*}{Lemma} 257 | \newtheorem*{proposition*}{Proposition} 258 | \newtheorem*{corollary*}{Corollary} 259 | \fi 260 | 261 | 262 | %Def-style theorems 263 | \theoremstyle{definition} 264 | 265 | % This ought to be a real theorem, but would be too much italics 266 | \ifevanmdthm 267 | %\declaretheorem[style=thmbluebox,name=Algorithm,sibling=theorem]{algorithm} 268 | % \else 269 | % \newtheorem{algorithm}[theorem]{Algorithm} 270 | % \newtheorem*{algorithm*}{Algorithm} 271 | % \fi 272 | 273 | \newtheorem{claim}[theorem]{Claim} 274 | \newtheorem{definition}[theorem]{Definition} 275 | \newtheorem{fact}[theorem]{Fact} 276 | 277 | \newtheorem{answer}[theorem]{Answer} 278 | \newtheorem{case}[theorem]{Case} 279 | \newtheorem{ques}[theorem]{Question} 280 | \newtheorem{exercise}[theorem]{Exercise} 281 | \newtheorem{problem}[theorem]{Problem} 282 | 283 | \newtheorem*{answer*}{Answer} 284 | \newtheorem*{case*}{Case} 285 | \newtheorem*{claim*}{Claim} 286 | \newtheorem*{definition*}{Definition} 287 | \newtheorem*{fact*}{Fact} 288 | \newtheorem*{joke*}{Joke} 289 | \newtheorem*{ques*}{Question} 290 | \newtheorem*{exercise*}{Exercise} 291 | \newtheorem*{problem*}{Problem} 292 | 293 | 294 | \ifevanmdthm 295 | \declaretheorem[style=thmrecbox,name=Example,sibling=theorem]{example} 296 | \declaretheorem[style=thmrecbox,name=Example,numbered=no]{example*} 297 | \else 298 | \newtheorem{example}[theorem]{Example} 299 | \newtheorem*{example*}{Example} 300 | \fi 301 | 302 | % Remark-style theorems 303 | %\theoremstyle{remark} 304 | \newtheorem{note}[theorem]{Note} 305 | \newtheorem{remark}[theorem]{Remark} 306 | \newtheorem*{note*}{Note} 307 | \newtheorem*{remark*}{Remark} 308 | \fi 309 | 310 | %Loads a bunch of useful packages 311 | \ifevanpkg 312 | \usepackage{listings} 313 | \lstset{basicstyle=\footnotesize\ttfamily, 314 | numbers=left, 315 | numbersep=5pt, 316 | numberstyle=\tiny, 317 | keywordstyle=\bfseries, 318 | % title=\lstname, 319 | showstringspaces=false, 320 | tabsize=4, 321 | frame=single, 322 | keywordstyle=\bfseries\color{blue}, 323 | commentstyle=\color{green!70!black}, 324 | identifierstyle=\color{green!20!black}, 325 | stringstyle=\color{orange}, 326 | breaklines=true, 327 | } 328 | \usepackage{paralist} 329 | \usepackage{todonotes} 330 | % Tiny tiny optimizations: 331 | \usepackage{mathtools} 332 | \usepackage{microtype} 333 | 334 | \ifevantitling 335 | %If we have titling... 336 | \usepackage{titling} 337 | \@ifundefined{KOMAClassName}{}{ 338 | \newcommand{\thesubtitle}{} 339 | \renewcommand{\subtitle}[1]{% 340 | \renewcommand{\thesubtitle}{#1} 341 | \posttitle{\\ {\normalsize \thesubtitle} \end{center}\vspace{\posttitledrop}} 342 | } 343 | } 344 | \newlength{\posttitledrop} 345 | \setlength{\posttitledrop}{-0.7em} 346 | \newlength{\postauthordrop} 347 | \setlength{\postauthordrop}{-1.2em} 348 | \pretitle{\begin{center}\bfseries \sffamily \Large} 349 | \posttitle{\end{center}\vspace{\posttitledrop}} 350 | \preauthor{\begin{center} \scshape} 351 | \postauthor{\end{center}\vspace{\postauthordrop}} 352 | \predate{\begin{center}\small} 353 | \postdate{\end{center}} 354 | \fi 355 | \fi 356 | 357 | % Commutative diagrams support 358 | \ifevandiagrams 359 | \usepackage{diagrams} 360 | \newarrow{Inj}C---> 361 | \newarrow{Surj}----{>>} 362 | \newarrow{Proj}----{>>} 363 | \newarrow{Embed}>---> 364 | \newarrow{Incl}C---> 365 | \newarrow{Mapsto}|---> 366 | \newarrow{Isom}<---> 367 | \newarrow{Dotted}....> 368 | \newarrow{Dashed}{}{dash}{}{dash}> 369 | \newarrow{Line}----- 370 | \fi 371 | 372 | %Page Setup 373 | \ifevanfancy 374 | \usepackage{fancyhdr} 375 | \@ifundefined{KOMAClassName} 376 | { 377 | \setlength{\headheight}{0.75in} 378 | \setlength{\oddsidemargin}{0in} 379 | \setlength{\evensidemargin}{0in} 380 | \setlength{\voffset}{-1.0in} 381 | \setlength{\headsep}{10pt} 382 | \setlength{\textwidth}{6.5in} 383 | \setlength{\headwidth}{6.5in} 384 | \setlength{\textheight}{8.75in} 385 | \setlength{\parskip}{1ex plus 0.5ex minus 0.2ex} 386 | \setlength{\footskip}{0.3in} 387 | \ifevantitling 388 | \addtolength{\posttitledrop}{0.4em} 389 | \addtolength{\postauthordrop}{0.2em} 390 | \fi 391 | } 392 | { 393 | \addtolength{\textheight}{3.14cm} 394 | \setlength{\footskip}{0.5in} 395 | \setlength{\headsep}{10pt} 396 | } 397 | 398 | 399 | %Setup FancyHeaders 400 | \ifevanhdr 401 | \renewcommand{\headrulewidth}{0.5pt} 402 | \renewcommand{\footrulewidth}{0.0pt} 403 | \pagestyle{fancy} 404 | \ifevantitling 405 | \lhead{\theauthor} 406 | \else 407 | \lhead{Samuel Hsiang} % this is overridden in main 408 | \fi 409 | \chead{} 410 | \rhead{\nouppercase{\leftmark}} 411 | \lfoot{} 412 | \cfoot{\thepage} 413 | \rfoot{} 414 | \fi 415 | \fi 416 | 417 | \ifevanchinese 418 | \usepackage[encapsulated]{CJK} 419 | \usepackage{ucs} 420 | \usepackage[utf8x]{inputenc} 421 | \newenvironment{bsmi}{\begin{CJK}{UTF8}{bsmi}}{\end{CJK}} 422 | \newcommand{\cn}[1]{\begin{bsmi}#1\end{bsmi}} 423 | \AtBeginDocument{\begin{CJK}{UTF8}{bsmi}} 424 | \AtEndDocument{\end{CJK}} 425 | \fi 426 | \fi 427 | -------------------------------------------------------------------------------- /chapters/strings.tex: -------------------------------------------------------------------------------- 1 | \chapter{Strings} 2 | 3 | \section{String Hashing} 4 | 5 | String hashing is a probabilistic technique for dealing with strings. The most common hashing algorithm used in programming contests is the \emph{polynomial hash}, also known as the \emph{Rabin-Karp hash}. This method allows for the fast comparison of strings and their substrings---$O(1)$ after linear time precomputation. Thus we can often use conceptually simpler hashing to replace trickier string algorithms, such as Knuth-Morris-Pratt and the Z-algorithm. In addition, adding binary search allows for the computation of the longest common prefix in $\log n$ time, useful in constructing suffix arrays. 6 | 7 | Jonathan Paulson gives a great introduction to polynomial hashing: \url{https://www.reddit.com/r/usaco/comments/39qrta/string_hashing/} 8 | 9 | \section{Knuth-Morris-Pratt} 10 | 11 | Knuth-Morris-Pratt is an easy-to-code linear time string matching algorithm. Using the ``needle and haystack'' analogy, for a given needle of length $n$ and a haystack of length $m$, KMP takes $O(n)$ to preprocess the needle and $O(m)$ to search. Hariank and Corwin explain the algorithm well: \url{https://activities.tjhsst.edu/sct/lectures/1415/stringmatching_10_3_14.pdf} 12 | 13 | Beyond string matching, KMP is also useful for problems involving periodic strings. This is because a string with an equal prefix and suffix of length $l$ has a period of $n - l$, which is exactly what KMP computes. 14 | 15 | A similar (and equivalent) algorithm is the Z-algorithm, explained here: \url{http://codeforces.com/blog/entry/3107} 16 | 17 | \section{Trie} 18 | 19 | A trie (from re\emph{trie}val) is a data structure for storing strings that supports insertion and look-up in linear time. The trie maintains the strings in a rooted tree, where each vertex represents a prefix and each edge is labeled with a character. The prefix of a node $n$ is the string of characters on the path from the root to $n$. (In particular, the prefix of the root is the empty string.) Every string that is stored in the tree is represented by a path starting from the root. Below is a picture of a trie storing ``COW,'' ``MO,'' ``MOM,'' ``MOO,'' and ``MOP.'' 20 | 21 | \begin{center} 22 | \begin{tikzpicture}[ 23 | very thick, 24 | level 1/.style={sibling distance=42mm}, 25 | level 2/.style={sibling distance=30mm}, 26 | level 3/.style={sibling distance=24mm}, 27 | level 4/.style={sibling distance=18mm}, 28 | level distance=16mm, 29 | myrect/.style={ 30 | draw, 31 | fill=myseagreen, 32 | text width=8ex, 33 | text centered 34 | } 35 | ] 36 | \node [myrect] {\vphantom{A}} 37 | child { 38 | node [myrect] {C} 39 | child { 40 | node [myrect] {CO} 41 | child { 42 | node [myrect] {COW} 43 | child { 44 | node [myrect] {COW\$} 45 | edge from parent [->] node [left] {\$} 46 | } 47 | edge from parent [->] node [left] {W} 48 | } 49 | child [missing] 50 | edge from parent [->] node [left, xshift=-0.5mm] {O} 51 | } 52 | child [missing] 53 | edge from parent [->] node [left, xshift=-1mm] {C} 54 | } 55 | child { 56 | node [myrect] {M} 57 | child { 58 | node [myrect] {MO} 59 | child { 60 | node [myrect] {MO\$} 61 | edge from parent [->] node [left, xshift=-3mm] {\$} 62 | } 63 | child { 64 | node [myrect] {MOM} 65 | child { 66 | node [myrect] {MOM\$} 67 | edge from parent [->] node [left, xshift=-1mm] {\$} 68 | } 69 | edge from parent [->] node [left, xshift=-1mm] {M} 70 | } 71 | child { 72 | node [myrect] {MOO} 73 | child { 74 | node [myrect] {MOO\$} 75 | edge from parent [->] node [left] {\$} 76 | } 77 | edge from parent [->] node [left] {O} 78 | } 79 | child { 80 | node [myrect] {MOP} 81 | child { 82 | node [myrect] {MOP\$} 83 | edge from parent [->] node [left] {\$} 84 | } 85 | edge from parent [->] node [right, xshift=2mm] {P} 86 | } 87 | edge from parent [->] node [left] {O} 88 | } 89 | edge from parent [->] node [right, xshift=1mm] {M} 90 | } 91 | ; 92 | 93 | \end{tikzpicture} 94 | \end{center} 95 | 96 | Insertion into a trie is straightforward. We start at the root, and create edges and vertices as necessary until a path representing the string is formed. If a vertex already has an edge of the correct character leading out from it, we just travel down that edge. In order to identify the end of a string, we can append a ``\$'' to every string before we insert. Searching is the same as insertion, except we terminate our search when we can't go any further instead of adding a new edge. 97 | 98 | What we've seen so far doesn't make the trie seem particularly useful---string insertion and look up can be easily done in linear time with Rabin-Karp and a hash table as well. The advantage of the trie comes from its tree structure. Having common prefixes bundled together on the same path means we can compute compute information relating to prefixes easily. This allows for tree DPs that we wouldn't be able to do with hashing. (However, hashing does handle certain forms of prefix queries well, such as longest common prefix.) 99 | 100 | Tries are also a building block for other string data structures, such as the Aho-Corasick automaton and the suffix tree. (In fact, tries can be viewed as a very simple form of string automaton.) 101 | 102 | \section{Suffix Array} 103 | 104 | For a string of length $n$, each index $i$ ($0 \le i < n$) can represent the suffix that starts at that index. A \emph{suffix array} is a list of these indices, sorted in lexicographically increasing order by suffix. Thus a suffix array is essentially a sorted list of the suffixes of a string. Below is the suffix array of ``mississippi\$.'' 105 | 106 | \begin{center} 107 | \begin{tabular}{r | l} 108 | Suffix Array & Suffix \phantom{moooooooooo} \\ \hline 109 | 11 & \$ \\ 110 | 10 & i\$ \\ 111 | 7 & ippi\$ \\ 112 | 4 & issippi\$ \\ 113 | 1 & ississippi\$ \\ 114 | 0 & mississippi\$ \\ 115 | 9 & pi\$ \\ 116 | 8 & ppi\$ \\ 117 | 6 & sippi\$ \\ 118 | 3 & sissippi\$ \\ 119 | 5 & ssippi\$ \\ 120 | 2 & ssissippi\$ \\ 121 | \end{tabular} 122 | \end{center} 123 | 124 | At first, it might seem surprising that such a structure is useful---why do we care about suffixes at all? The crucial observation is that every substring of a string is a prefix of a suffix. Thus if we have something that does well with prefixes, such as hashing or a trie, we use this to compute information about substrings. A trie built from suffixes is known as a \emph{suffix tree}, which we'll cover later. In this section, we'll go over what we can do with hashing and a suffix array. 125 | 126 | First, let's figure out how we can construct a suffix array. The naive solution is to sort all $n$ suffixes using $O(n)$ string comparison. Since sorting itself takes $O(n \log n)$ comparisons, we have an $O(n^2 \log n)$ algorithm. However, with hashing and binary search, we can lexicographically compare two strings in $O(\log n)$ time. We do this by binary searching for the longest common prefix and then comparing the next character. We can can compute the hash of any substring with a polynomial hash, so it's easy to compare if the prefixes of two suffixes (a.k.a two substrings) are equal. Now that we have $O(\log n)$ comparison, our algorithm runs in $O(n \log^2 n)$ overall. 127 | 128 | Note that it is also possible to compute suffix arrays in $O(n \log n)$ and even $O(n)$, but these algorithms are much more involved. For the purposes of contests, an $O(n \log^2 n)$ suffix array algorithm should almost always be enough. 129 | 130 | With a suffix array, we can check if any queried ``needle'' string exists using a binary search with hashing. We can even count the number of occurrences by binary searching for the first and last match. This works in $O(m + \log n \log m)$, where $m$ is the length of the needle, because we need $O(\log m)$ operations for string comparison and $O(\log n)$ iterations of the binary search. Suffix arrays differ from KMP because KMP preprocesses the ``needle,'' while suffix arrays preprocess the ``haystack.'' 131 | 132 | From the suffix array, we can also obtain another useful data structure called the \emph{LCP array}. This array stores the longest common prefix between adjacent suffixes in the suffix array. We can use it to speed up pattern matching to $O(m + \log n)$. In addition, we can build a segment tree over the LCP array to answer queries, such as the LCP between two arbitrary suffixes. 133 | 134 | Suffix arrays can also be used to solve many string problems beyond matching. This data structure does very well with most problems involving substrings. For example, suffix arrays/LCP arrays can count the number of distinct substrings in a string or return the $k$th lexicographically largest substring. The minimum string rotation problem can is another problem solvable with suffix arrays, since each rotation of a string $S$ is a substring of $S + S$. 135 | 136 | To handle multiple strings with suffix arrays, we can either concatenate them with separator characters in between or separately compute their hashes. 137 | 138 | Below is a short C++ code for computing suffix arrays. Look up lambda functions in C++ if you're not familiar with the syntax. 139 | 140 | \begin{mylstlisting}[language=C++] 141 | typedef unsigned long long ull; 142 | // We use A as the exponent and M as the mod for our hash. 143 | const ull M = 1000000007; 144 | const ull A = 127; 145 | 146 | void build(string S, int *sa){ 147 | // Builds a suffix array for a string S in array sa. 148 | S.push_back('$'); 149 | int N = S.size(); 150 | ull P[N + 1], H[N + 1]; 151 | P[0] = 1; // P stores the precomputed powers of A. 152 | H[0] = 0; // H stores the prefix hashes of S. 153 | for(int i = 0; i < N; i++){ 154 | // Precomputing powers and hashes. 155 | P[i + 1] = A * P[i] % M; 156 | H[i + 1] = (A * H[i] + S[i]) % M; 157 | sa[i] = i; 158 | } 159 | auto f = [&](int a, int l){ 160 | // Returns the hash of the substring starting at index a with length l. 161 | return (H[a + l] - H[a] * P[l] % M + M) % M; 162 | }; 163 | auto comp = [&](int a, int b){ 164 | // Compares the suffixes starting at indices a and b. 165 | int lo = 0, hi = min(N - a, N - b); 166 | while(lo + 1 < hi){ 167 | int m = (lo + hi) / 2; 168 | (f(a, m) == f(b, m) ? lo : hi) = m; 169 | } 170 | return S[a + lo] < S[b + lo]; 171 | }; 172 | sort(sa, sa + N, comp); 173 | } 174 | \end{mylstlisting} 175 | 176 | \section{Aho-Corasick} 177 | 178 | One way to think about the Aho-Corasick algorithm is KMP/Z-algorithm on a trie. This algorithm is used for matching multiple needles in a single haystack and runs in time linear in the length of the haystack, with preprocessing linear in the total length of the needles. To do this, we first add all strings to a trie that stores some extra information at its nodes. Upon inserting each string, we mark the last node visited as a endpoint node. Each node, in addition to storing its children and its status as an endpoint node, stores a \emph{suffix link}. This link points to the longest proper suffix that is also a node on the trie. If necessary, each node can also store a \emph{dictionary suffix link}, which points to the first endpoint node reachable by only following suffix links. 179 | 180 | To build an Aho-Corasick automaton, we start with a trie of the needle strings. What we want to do is compute the suffix links. We simplify this by using an $O(nk)$ approach, where $k$ is the alphabet size. For each node $n$, we compute an additional failure function, with one value corresponding to each letter of the alphabet. Suppose $p$ is the prefix represented by $n$. Then the failure function of $n$ for the letter $\alpha$ points to the node representing longest suffix of $p + \alpha$ in the trie. 181 | 182 | We can compute all of this with a BFS. When we are at a node, we can find the suffix links of its children using the failure function of its own suffix link. We also have to calculate the node's own failure function. For every letter that has a corresponding child, its failure function is equal to that child. Otherwise, its failure function is equal to the corresponding failure function of the node's suffix link. Take a moment to think through why this works. 183 | 184 | To query, we can iterate through our haystack string character by character and make the corresponding moves in the automaton. We follow the failure function when no child exists. However, note that there isn't really a distinction between a normal trie edge and a failed edge anymore. To check for matches, we can look at the values we store at each node and/or follow dictionary suffix links. To find all matches with dictionary suffix links, we have to follow the pointers until we reach a node that doesn't have a suffix in the dictionary. Note that if we follow dictionary suffix links, the complexity of our algorithm will also be linear in the number of matches. Here's an implementation of Aho-Corasick in C++ without dictionary suffix links: 185 | 186 | \begin{mylstlisting}[language=C++] 187 | const int SIGMA = 26; 188 | const int MAXN = 100005; 189 | // Each node is assigned an index where its data is stored. 190 | // ch first stores the children of each node and later the failure function. 191 | // val counts the number of strings ending at a given node. 192 | // link stores suffix links. 193 | int ch[MAXN][SIGMA], val[MAXN], link[MAXN], sz = 1; 194 | // Array q is a hand-implemented queue. 195 | // h and t point to the head and tail, respectively. 196 | int q[MAXN], h, t; 197 | 198 | void add(string s){ 199 | // Adds a string to the trie. 200 | int p = 0; // p tracks our current position and starts at 0, the root. 201 | for(int i = 0; i < s.size(); i++){ 202 | int c = s[i] - 'A'; 203 | // ch[p][c] is 0 if null, so we have to allocate a new node/index. 204 | if(!ch[p][c]) ch[p][c] = sz++; 205 | p = ch[p][c]; 206 | } 207 | val[p]++; // Updates endpoint marker. 208 | } 209 | 210 | void build(){ 211 | // Computes all suffix links with a BFS, starting from the root. 212 | h = t = 0; 213 | q[t++] = 0; 214 | while(h < t){ 215 | int v = q[h++], u = link[v]; 216 | val[v] += val[u]; // Propagates endpoint marker. 217 | for(int c = 0; c < SIGMA; c++){ 218 | if(ch[v][c]){ 219 | // If child exists, we create a suffix link. 220 | // The node's failure function here is the child. 221 | // Also, note that we have a special case if v is the root. 222 | link[ch[v][c]] = v ? ch[u][c] : 0; 223 | q[t++] = ch[v][c]; 224 | } else { 225 | // Otherwise, we need to figure out its failure function. 226 | // We do this by using the failure function of the node's suffix link. 227 | ch[v][c] = ch[u][c]; 228 | } 229 | } 230 | } 231 | } 232 | \end{mylstlisting} 233 | 234 | \section{Advanced Suffix Data Structures} 235 | 236 | I've yet to see these used in a contest, but they do exist. 237 | 238 | \subsection{Suffix Tree} 239 | 240 | \subsection{Suffix Automaton} 241 | 242 | -------------------------------------------------------------------------------- /chapters/math.tex: -------------------------------------------------------------------------------- 1 | \chapter{Math} 2 | 3 | Algorithms here exhibit a different flavor than the graph theory, string, or geometry algorithms from earlier. This chapter is placed towards the end because material here doesn't really fit in any other section or the USACO canon, \textit{not} because this material is particularly difficult. 4 | 5 | \section{Number Theory} 6 | 7 | This section will contain much information about primes in many different ways. Indeed, prime numbers are the building blocks of number theory! 8 | 9 | \subsection{Random Prime Numbers Bounds} 10 | 11 | The point of this section is mostly to make the complexity analysis of the later sections easier. Therefore, I will list some useful theorems and bounds without proof. The proofs can be found in any standard number theory textbook or online. They are extremely insignificant in the context of the goals of this section. 12 | 13 | \textbf{Prime Number Theorem}: The number of prime numbers less than $n$ is $O(\frac{n}{\log n})$ for any positive integer $n.$ 14 | 15 | \emph{Corollary}: The $n$-th prime number, $p_n$, is $O(n \log n).$ 16 | 17 | \textbf{Prime Number Harmonic Sum}: $\sum_{p \le n} \frac{1}{p} = O(\log \log n).$ 18 | 19 | \subsection{Prime Number Testing} 20 | 21 | Let's say if you want to test is a number $n$ is prime. The most obvious way is to test all integers from $2$ to $n-1$, and see if they divide $n$. This takes $O(n)$ time. We can do better though! Note that if $n$ has a nontrivial factor, then it has one that is less than $\sqrt{n}.$ Indeed, if $n = pq,$ (as a nontrivial factorization) if both $p, q > \sqrt{n}$, then $pq > \sqrt{n}^2 = n$, a contradiction. So we only need to check whether $n$ has factors between $2$ and $\sqrt{n}.$ This takes $O(\sqrt{n})$ time. If you have precomputed a list of prime numbers (up to say $\sqrt{n}$), ten you only need to check whether $n$ has any prime factors in the same $2$ to $\sqrt{n}.$ This would give runtime $O(\frac{\sqrt{n}}{\log n})$ from the \emph{Prime Number Theorem} in section 10.1.1, a small but nontrivial improvement. 22 | 23 | \subsection{Sieve of Eratosthenes} 24 | 25 | If you want to compute all primes in the range $1$ to $n$. Runs in $O(n \log \log n)$. In other words, super fast. 26 | 27 | \subsection{Prime Factorization} 28 | 29 | How can we prime factorize a number? That is, given a positive integer $n$, how can we write it in the form $p_1 \cdot p_2 \cdot \dots \cdot p_k$, where the $p_i$ are all primes? Form example, $30 = 2 \cdot 3 \cdot 5$ and $24 = 2 \cdot 2 \cdot 2 \cdot 3$. The fact that every number can be uniquely factorized is called the \emph{Fundamental Theorem of Arithmetic}. Let's now figure out how to prime factorize every number efficiently. Using our intuition from above, we should expect that checking up to around $\sqrt{n}$ suffices. This is indeed true. The code below demonstrates: 30 | 31 | \subsection{GCD and the Euclidean Algorithm} 32 | 33 | I'll start once again with a theorem in number theory: if positive integers $a, b$ satisfy $\gcd(a, b) = 1$, then there exist integers $x, y$ such that $ax+by=1.$ This is called \emph{Bezout's Lemma.} 34 | 35 | Similarly, if $\gcd(a, b) = g$, then there are integers $x, y$ such that $ax+by = g.$ 36 | 37 | %example gcd code here... and also some description of Euclidean Algorithm 38 | 39 | \subsection{Fermat's Little Theorem} 40 | 41 | Fermat's Little Theorem states that $a^p \equiv a \pmod{p}$ for any positive integer $a.$ Equivalently, we can say that $a^{p-1} \equiv 1 \pmod{p}.$ 42 | 43 | \subsection{Modular Inverses} 44 | 45 | If $p$ is a prime, and $x$ satisfies $\gcd(x, p) = 1$, then there exists a positive integer $k$ such that $kx \equiv 1 \pmod{p}.$ $k$ is called the \emph{modular inverse} of $x \pmod{p},$ often just denoted as $1/x$ or $x^{-1}.$ Indeed, $1/x \cdot x = x^{-1} \cdot x = 1,$ so this notation is natural. The cool thing about modular inverses is that you can see them as division modulo $p.$ Since $x$ is nonzero $\pmod{p},$ it makes sense that you should be able to divide by $x.$ This is the main application of modular inverses. But how to find one quickly? There are two common algorithms, both of which run in $O(\log p)$ time. You can do this by running the Euclidean Algorithm described above: find positive integers $j, k$ such that $pj + kx = 1.$ Then obviously $kx \equiv 1 \pmod{p}.$ The other way is to notice that $x^{p-1} \equiv 1 \pmod{p}$ by Fermat's Little Theorem, so $x^{p-2} \equiv x^{-1} \pmod{p}.$ Now compute the left hand side of the previous expression using binary exponentiation. 46 | 47 | Exercises related: Do $O(n \log p)$ precomputation to compute any binomial coefficient of the form $\binom{m}{k} \pmod{p}$ where $0 \le m, k \le n$ in $O(1)$ time. 48 | 49 | \section{Combinatorial Games} 50 | 51 | \url{https://activities.tjhsst.edu/sct/lectures/1314/impartial_games_12_06_13.pdf} 52 | 53 | \section{Karatsuba} 54 | 55 | Consider two polynomials $P(x)$ and $Q(x)$ with $n$ coefficients. 56 | The standard, long multiplication method of calculating $R(x) = P(x) \cdot Q(x)$ 57 | involves $O(n^2)$ operations. We'll see if there's a way to speed this up. 58 | 59 | We first write 60 | \begin{align*} 61 | P(x) &= A(x) \cdot x^{n/2} + B(x) \\ 62 | Q(x) &= C(x) \cdot x^{n/2} + D(x). 63 | \end{align*} 64 | 65 | where $A$, $B$, $C$, and $D$ are all polynomials with $n/2$ coefficients. 66 | Then, $R(x)$ can be written as 67 | \[P(x) \cdot Q(x) = (A(x) \cdot C(x)) x^n + (A(x) \cdot D(x) + B(x) \cdot C(x)) x^{n/2} + (B(x) \cdot D(x)).\] 68 | 69 | Addition of polynomials is linear, so assuming we have each of these parts computed, 70 | added execution time is minimal. At first glance, it seems we have reduced the multiplication 71 | of two degree $n$ polynomials to summing three expressions that involve multiplication 72 | of degree $n/2$ polynomials, but unfortunately the middle expression involves two such 73 | multiplications, so there is no speedup. 74 | 75 | However, we observe that removing the various $x^k$ from the multiplication actually 76 | allows us to calculate that middle expression from the other two and one new 77 | degree $n/2$ multiplication: 78 | \[ 79 | (A(x) + B(x))(C(x) + D(x)) = A(x) \cdot C(x) + (A(x) \cdot D(x) + B(x) \cdot C(x)) + B(x) \cdot D(x). 80 | \] 81 | 82 | Then we can extract all the necessary pieces to $R(x)$ using only three smaller 83 | multiplications, and proceed recursively. The complexity of this process satisfies 84 | \[T(n) = 3 T\left(\frac{n}{2}\right) + O(n),\] 85 | which solves to $O(n^{\log_2{3}}) \approx O(n^{1.585})$ by the Master theorem. 86 | 87 | \begin{algorithm}[H] 88 | \caption{Karatsuba} 89 | \begin{algorithmic} 90 | \Function{Multiply}{$P(x)$, $Q(x)$, $n$} 91 | \If {$n \le 1$} 92 | \State \Return $P(x) \cdot Q(x)$ \Comment{using normal multiplication} 93 | \EndIf 94 | \State Let $A$, $B$ satisfy $P(x) = A(x) \cdot x^{\floor{n/2}} + B(x)$ 95 | \State Let $C$, $D$ satisfy $Q(x) = C(x) \cdot x^{\floor{n/2}} + D(x)$ 96 | \State $F \gets \Call{Multiply}{A(x), C(x), \ceiling{n/2}}$ 97 | \State $G \gets \Call{Multiply}{B(x), D(x), \floor{n/2}}$ 98 | \State $H \gets \Call{Multiply}{A(x) + B(x), C(x) + D(x), \ceiling{n/2}}$ 99 | \State \Return $F(x) \cdot x^{2 \floor{n/2}} + H(x) \cdot x^{\floor{n/2}} + G(x)$ 100 | \EndFunction 101 | \end{algorithmic} 102 | \end{algorithm} 103 | 104 | \section{Matrices} 105 | 106 | \section{Fast Fourier Transform} 107 | 108 | \subsubsection{Introduction} 109 | 110 | The \emph{Fast Fourier Transform} (FFT) is a technique used to multiply two polynomials of degree $n$ in $O(n \cdot \log n)$ time. At a high level, what this algorithm is doing is something called \emph{polynomial interpolation}. This refers to the fact that if we know the value of a polynomial $P$ of degree $n$ at $n+1$ points, then we can uniquely determine the polynomial $P.$ The proof of this statement is simple, and involves the Lagrange Interpolation Formula for one direction and a simple root counting argument to prove uniqueness. I won't go into detail here because the proof isn't important, but it is useful to know some motivation behind the algorithm. 111 | 112 | \subsubsection{Algorithm Outline} 113 | 114 | Let's say we want to multiply 2 polynomials $A(x), B(x)$, both of degree $n$. Let $C(x) = A(x)B(x).$ The algorithm will proceed in 3 steps. Choose an integer $m > 2n,$ and choose $m$ numbers $x_0, x_1, \dots, x_{m-1}.$ I'll clarify what to choose $m$ and the $x_i$ as later. Just keep in mind that we can choose these values to be anything we want. 115 | 116 | \begin{enumerate} 117 | 118 | \item Evaluate $A(x_0), A(x_1), \dots, A(x_{m-1}).$ Similarly, evaluate $B(x_0), B(x_1), \dots, B(x_{m-1}).$ 119 | 120 | \item Evaluate $C(x_i) = A(x_i)B(x_i)$ for $0 \le i \le m-1.$ 121 | 122 | \item Interpolate the coefficients of $C(x)$ given the values $C(x_0), C(x_1), \dots, C(x_{m-1}).$ 123 | 124 | \end{enumerate} 125 | 126 | The last step explains why we need $m > 2n.$ The degree of $C(x)$ is $2n$, so we need at least $2n+1$ points to determine $C(x)$ uniquely. 127 | 128 | You should be skeptical that the above approach does any better than $O(n^2).$ In particular, step 1 seems like it should take $O(n^2).$ It turns out that if we choose the values $x_0, x_1, \dots, x_{m-1}$ properly, then we can do much better. Let me now describe what to choose $m$ to be, and what to choose $x_0, x_1, \dots, x_{m-1}$ to be. 129 | 130 | \subsubsection{Roots of Unity} 131 | 132 | Before telling you precisely, here's some food for thought. Let's take then example polynomial $A(x) = 1+2x+3x^2+4x^3+5x^4+6x^5+7x^6+8x^7.$ Let's split this into 2 groups: the even degree coefficients and odd degree coefficients. Let's call these two groups $A_{even}$ and $A_{odd}.$ Define $A_{even}(x) = 1+3x+5x^2+7x^3, A_{odd}(x) = 2+4x+6x^2+8x^7.$ Then clearly \[ A(x) = A_{even}(x^2) + x \cdot A_{odd}(x^2) \] 133 | 134 | Notice that the $x^2$ in the formula makes it extremely easy to compute $A(-x)$ given $A(x).$ It's only one sign flip off! Therefore, it would be cool if our $x_i$ above were symmetric with respect to 0, i.e. if we want to compute $x$, we also should want to compute $-x$. An example set like this is $\{1, 2, 3, 4, -1, -2, -3, -4 \}.$ So if we wanted to compute $A(x)$ at these values, we would need to compute the values of $A_{even}, A_{odd}$ at their squares, that is $\{1, 4, 9, 16 \}.$ But this set is no longer symmetric! So this is not what we want exactly, since that means that we can't just recursively compute $A_{even}$ and $A_{odd}.$ So let's say that the set we want to compute $A_{even}, A_{odd}$ on is something like $\{1, 4, -1, -4 \}.$ Then the values that we get for $A(x)$ are $\{1, -1, 2, -2, i, -i, 2i, -2i \}.$ Complex numbers! This is closer to what we want, but still not precisely. The selection of the $x_i$ explained below makes everything work out perfectly. 135 | 136 | So what should we choose $m$ and the $x_i$ to be? I'll tell you now and then explain why this selection works so well: choose $m$ to be a power of $2$ that is larger than $2n$, and choose $x_0, x_1, \dots, x_{m-1}$ to be the $m$\emph{-th roots of unity}. The $m$-th roots of unity are complex numbers that satisfy the equation $x^m = 1.$ They are of the form $\cos\left(\frac{2k\pi i}{m}\right) + i \cdot \sin\left(\frac{2k\pi i}{m} \right)$ for any integer $k$ from $0$ to $m-1.$ 137 | 138 | Let $\omega$ be a \emph{primitive} root of unity, i.e. the smallest $m'$ such that $\omega^{m'} = 1$ is $m' = m.$ We can without loss of generality set $\omega = \cos\left(\frac{2\pi i}{m}\right) + i \cdot \sin\left(\frac{2\pi i}{m} \right).$ One can easily check the the remaining roots of unity are $\omega^2, \omega^3, \dots, \omega^m.$ From now on, let's just set $x_i = \omega^i$ for all $0 \le i \le m-1.$ Note that $x_0 = 1.$ Also set $m = 2^k > 2n.$ Now I can proceed to describing why this algorithm works. 139 | 140 | \subsubsection{Algorithm Specifics: Step 1} 141 | 142 | In this section I'll implement (using pseudocode) a function 143 | 144 | vector fft(vector $A$, k, $\omega$). This means that this will return a vector of length $2^k$ containing the values $A(\omega^0), A(\omega^1), A(\omega^2), \dots, A(\omega^{2^k-1}).$ Remember that $\omega^{2^k} = 1.$ The vector $A$ stores the coefficients of $A(x)$. The $x^i$ coefficient of $A$ would be stored as $A[i].$ 145 | 146 | Here's an implementation. The brackets $\{ \}$ will be shorthand for containing a vector. 147 | 148 | \begin{algorithm}[H] 149 | \caption{Fast Fourier Transform} 150 | %\label{} 151 | \begin{algorithmic} 152 | \Function{FFT}{vector $A, k, \omega$} \Comment $\omega^{2^k} = 1$ 153 | \State $A$.resize($2^k$) 154 | \State vector $f, f_{even}, f_{odd}$ 155 | \State $f$.resize($2^k$) 156 | 157 | \If {$k = 0$} 158 | \State \Return $ \{A[0] \}$ \Comment a vector of length 1 159 | \EndIf 160 | \State $A_{even}$ = $ \{A[0], A[2], \dots, A[2^k-2] \}$ 161 | \State $A_{odd}$ = $ \{A[1], A[3], \dots, A[2^k-1] \}$ 162 | \State $f_{even} = $ \Call{FFT}{$A_{even}, k-1, \omega^2$} 163 | \State $f_{odd} = $ \Call{FFT}{$A_{odd}, k-1, \omega^2$} 164 | 165 | \ForAll {$i = 0, i < 2^{k-1}, i++$} 166 | \State $f[i] = f_{even}[i] + \omega^i \cdot f_{odd}[i]$ 167 | \State $f[i+2^{k-1}] = f_{even}[i] - \omega^i \cdot f_{odd}[i]$ 168 | 169 | \EndFor 170 | 171 | \Return f 172 | 173 | \EndFunction 174 | 175 | \end{algorithmic} 176 | \end{algorithm} 177 | 178 | So why does this algorithm work? Note that the algorithm proceeds recursively, calling itself twice (once for even, once for odd). $A_{even}(x)$ is the polynomial $A[0] + A[2]x + A[4]x^2 + \dots + A[2^k-2]x^{2^{k-1}-1}.$ Therefore, what the recursive call is doing is computing the values of $A_{even}$ at the values $x = \omega^0, \omega^2, \omega^4, \dots, \omega^{2^k-2}.$ This is equivalent to computing the values of the polynomial $B_{even}(x) = A[0] + A[2]x^2 + A[4]x^4 + \dots + A[2^k-2]x^{2^{k}-2}$ at the values $x = \omega^0, \omega^1, \omega^2, \dots, \omega^{2^{k-1}-1}.$ The recursive call for $A_{odd}(x) = A[1] + A[3]x + A[5]x^2 + \dots + A[2^k-1]x^{2^{k-1}-1}$ behaves in a similar way. Similarly, define $B_{odd}(x) = A[1] + A[3]x^2 + A[5]x^4 + \dots + A[2^k-1]x^{2^{k}-2}.$ 179 | 180 | The key is to note that $A(x) = B_{even}(x) + x \cdot B_{odd}(x).$ Since we know the values of $B_{even}(x), B_{odd}(x)$ for $x = \omega^0, \omega^1, \dots, \omega^{2^{k-1}-1},$ we also can compute the values of $A(x)$ for these values of $x$. But what about the remaining values? This requires the observation that $\omega^{i + 2^{k-1}} = -\omega^i.$ Since $B_{even}(x), B_{odd}(x)$ only have even powers, $B_{even}(\omega^i) = B_{even}(-\omega^i) = B_{even}(\omega^{i + 2^{k-1}}).$ The same equality also holds for $B_{odd}(x).$ Using this observation along with the equation $A(x) = B_{even}(x) + x \cdot B_{odd}(x),$ we can see now why the two equations in the for loop of the code above hold. And that's how we do Step 1! 181 | 182 | Let's analyze the runtime of this function FFT. Let $T(2^k)$ denote the time needed to run FFT on an array of length $2^k.$ Then $T(2^k) = 2T(2^{k-1}) + O(2^k) \implies T(2^k) = k \cdot 2^k,$ by the Master Theorem. Since $2^k$ is $O(n)$ by the above discussion, this steps runs in $O(n \cdot \log n)$ time. 183 | 184 | \subsubsection{Algorithm Specifics: Step 2} 185 | 186 | I'll give you a rest after a hard theoretical previous section. You do this step by looping through the values $A(x_0), A(x_1), \dots, A(x_{2^k-1}), B(x_0), B(x_1), \dots, B(x_{2^k-1})$ and directly multiplying $C(x_i) = A(x_i)B(x_i).$ This step runs in $O(n)$ time. 187 | 188 | \subsubsection{Algorithm Specifics: Step 3} 189 | 190 | This may seem like something completely new, but actually most of the work is already done. I'll explain. In fact I claim that writing the code $C = \text{fft}(C, k, \omega^{-1}),$ and then dividing all elements of $C$ by $2^k$ finishes. Step 3 is very easy once you have Step 1! 191 | 192 | To see this, let's take a look at how step 1 works. We'll consider the coefficients of $A$ as a vector (not a C++ vector, but an actual math vector -- matrices!). 193 | 194 | \begin{align*} 195 | \begin{pmatrix} 196 | A(\omega^0) \\ 197 | A(\omega^1) \\ 198 | A(\omega^2) \\ 199 | A(\omega^3) \\ 200 | \vdots \\ 201 | A(\omega^{2^k - 2}) \\ 202 | A(\omega^{2^k - 1}) 203 | \end{pmatrix} 204 | &= 205 | \begin{pmatrix} 206 | \omega^0 & \omega^0 & \omega^0 & \omega^0 & \cdots & \omega^0 & \omega^0 \\ 207 | \omega^0 & \omega^1 & \omega^2 & \omega^3 & \cdots & \omega^{2^k-2} & \omega^{2^k-1} \\ 208 | \omega^0 & \omega^2 & \omega^4 & \omega^6 & \cdots & \omega^{2(2^k-2)} & \omega^{2(2^k-1)} \\ 209 | \omega^0 & \omega^3 & \omega^6 & \omega^9 & \cdots & \omega^{3(2^k-2)} & \omega^{3(2^k-1)} \\ 210 | \vdots & \vdots & \vdots & \vdots & \ddots & \vdots & \vdots \\ 211 | \omega^0 & \omega^{2^k - 2} & \omega^{2(2^k - 2)} & \omega^{3(2^k - 2)} & \cdots & \omega^{(2^k - 2)(2^k-2)} & \omega^{(2^k-1)(2^k - 2)} \\ 212 | \omega^0 & \omega^{2^k - 1} & \omega^{2(2^k - 1)} & \omega^{3(2^k - 1)} & \cdots & \omega^{(2^k - 2)(2^k-1)} & \omega^{(2^k-1)(2^k - 1)} 213 | \end{pmatrix} 214 | \begin{pmatrix} 215 | A[0] \\ 216 | A[1] \\ 217 | A[2] \\ 218 | A[3] \\ 219 | \vdots \\ 220 | A[2^k - 2] \\ 221 | A[2^k - 1] 222 | \end{pmatrix} 223 | \end{align*} 224 | 225 | Now suppose we perform the FFT with $\omega^{-1}$. This is equivalent by multiplying by a similar matrix. 226 | 227 | \eject \pdfpagewidth=13in \pdfpageheight=11in 228 | 229 | \begin{align*} 230 | &\phantom{= } 231 | \begin{pmatrix} 232 | \omega^0 & \omega^0 & \omega^0 & \omega^0 & \cdots & \omega^0 & \omega^0 \\ 233 | \omega^0 & \omega^{-1} & \omega^{-2} & \omega^{-3} & \cdots & \omega^{-(2^k-2)} & \omega^{-(2^k-1)} \\ 234 | \omega^0 & \omega^{-2} & \omega^{-4} & \omega^{-6} & \cdots & \omega^{-2(2^k-2)} & \omega^{-2(2^k-1)} \\ 235 | \omega^0 & \omega^{-3} & \omega^{-6} & \omega^{-9} & \cdots & \omega^{-3(2^k-2)} & \omega^{-3(2^k-1)} \\ 236 | \vdots & \vdots & \vdots & \vdots & \ddots & \vdots & \vdots \\ 237 | \omega^0 & \omega^{-(2^k - 2)} & \omega^{-2(2^k - 2)} & \omega^{-3(2^k - 2)} & \cdots & \omega^{-(2^k - 2)(2^k-2)} & \omega^{-(2^k-1)(2^k - 2)} \\ 238 | \omega^0 & \omega^{-(2^k - 1)} & \omega^{-2(2^k - 1)} & \omega^{-3(2^k - 1)} & \cdots & \omega^{-(2^k - 2)(2^k-1)} & \omega^{-(2^k-1)(2^k - 1)} 239 | \end{pmatrix} 240 | \begin{pmatrix} 241 | A(\omega^0) \\ 242 | A(\omega^1) \\ 243 | A(\omega^2) \\ 244 | A(\omega^3) \\ 245 | \vdots \\ 246 | A(\omega^{2^k - 2}) \\ 247 | A(\omega^{2^k - 1}) 248 | \end{pmatrix} 249 | \\ 250 | &= 251 | \begin{pmatrix} 252 | \omega^0 & \omega^0 & \omega^0 & \omega^0 & \cdots & \omega^0 & \omega^0 \\ 253 | \omega^0 & \omega^{-1} & \omega^{-2} & \omega^{-3} & \cdots & \omega^{-(2^k-2)} & \omega^{-(2^k-1)} \\ 254 | \omega^0 & \omega^{-2} & \omega^{-4} & \omega^{-6} & \cdots & \omega^{-2(2^k-2)} & \omega^{-2(2^k-1)} \\ 255 | \omega^0 & \omega^{-3} & \omega^{-6} & \omega^{-9} & \cdots & \omega^{-3(2^k-2)} & \omega^{-3(2^k-1)} \\ 256 | \vdots & \vdots & \vdots & \vdots & \ddots & \vdots & \vdots \\ 257 | \omega^0 & \omega^{-(2^k - 2)} & \omega^{-2(2^k - 2)} & \omega^{-3(2^k - 2)} & \cdots & \omega^{-(2^k - 2)(2^k-2)} & \omega^{-(2^k-1)(2^k - 2)} \\ 258 | \omega^0 & \omega^{-(2^k - 1)} & \omega^{-2(2^k - 1)} & \omega^{-3(2^k - 1)} & \cdots & \omega^{-(2^k - 2)(2^k-1)} & \omega^{-(2^k-1)(2^k - 1)} 259 | \end{pmatrix} 260 | \begin{pmatrix} 261 | \omega^0 & \omega^0 & \omega^0 & \omega^0 & \cdots & \omega^0 & \omega^0 \\ 262 | \omega^0 & \omega^1 & \omega^2 & \omega^3 & \cdots & \omega^{2^k-2} & \omega^{2^k-1} \\ 263 | \omega^0 & \omega^2 & \omega^4 & \omega^6 & \cdots & \omega^{2(2^k-2)} & \omega^{2(2^k-1)} \\ 264 | \omega^0 & \omega^3 & \omega^6 & \omega^9 & \cdots & \omega^{3(2^k-2)} & \omega^{3(2^k-1)} \\ 265 | \vdots & \vdots & \vdots & \vdots & \ddots & \vdots & \vdots \\ 266 | \omega^0 & \omega^{2^k - 2} & \omega^{2(2^k - 2)} & \omega^{3(2^k - 2)} & \cdots & \omega^{(2^k - 2)(2^k-2)} & \omega^{(2^k-1)(2^k - 2)} \\ 267 | \omega^0 & \omega^{2^k - 1} & \omega^{2(2^k - 1)} & \omega^{3(2^k - 1)} & \cdots & \omega^{(2^k - 2)(2^k-1)} & \omega^{(2^k-1)(2^k - 1)} 268 | \end{pmatrix} 269 | \begin{pmatrix} 270 | A[0] \\ 271 | A[1] \\ 272 | A[2] \\ 273 | A[3] \\ 274 | \vdots \\ 275 | A[2^k - 2] \\ 276 | A[2^k - 1] 277 | \end{pmatrix} 278 | \\ 279 | &= 280 | \begin{pmatrix} 281 | 2^k & 0 & 0 & 0 & \cdots & 0 & 0 \\ 282 | 0 & 2^k & 0 & 0 & \cdots & 0 & 0 \\ 283 | 0 & 0 & 2^k & 0 & \cdots & 0 & 0 \\ 284 | 0 & 0 & 0 & 2^k & \cdots & 0 & 0 \\ 285 | \vdots & \vdots & \vdots & \vdots & \ddots & \vdots & \vdots \\ 286 | 0 & 0 & 0 & 0 & \cdots & 2^k & 0 \\ 287 | 0 & 0 & 0 & 0 & \cdots & 0 & 2^k \\ 288 | \end{pmatrix} 289 | \begin{pmatrix} 290 | A[0] \\ 291 | A[1] \\ 292 | A[2] \\ 293 | A[3] \\ 294 | \vdots \\ 295 | A[2^k - 2] \\ 296 | A[2^k - 1] 297 | \end{pmatrix} 298 | \\ 299 | &= 300 | 2^k 301 | \begin{pmatrix} 302 | A[0] \\ 303 | A[1] \\ 304 | A[2] \\ 305 | A[3] \\ 306 | \vdots \\ 307 | A[2^k - 2] \\ 308 | A[2^k - 1] 309 | \end{pmatrix}. 310 | \end{align*} 311 | 312 | \eject \pdfpagewidth=8.5in \pdfpageheight=11in 313 | 314 | It follows that the \textit{inverse FFT} is simply the same function, but run with $\omega^{-1}$ instead of $\omega$, and dividing out $m=2^k$. 315 | 316 | \subsubsection{A Brief Note on Number Theory} 317 | 318 | There is a slight variation on FTT that avoids using complex numbers completely. Complex numbers are often bad at precision and run slowly. There is a way to make FFT work over only the integers, which is pretty cool. Here's a brief description how. 319 | 320 | We work $\pmod{p}$ for a suitable prime $p$. The reason we need $p$ to be prime is so that division works as we want. As explained in the above section on number theory, every nonzero number modulo $p$ has a unique inverse, so division works as expected. There are also integers $g$ such that $g^0, g^1, \dots, g^{p-2}$ run through all nonzero numbers $\pmod{p}.$ These are called \emph{primitive roots}. This means that modulo $p$ behaves a whole lot like the roots of unity that we want. More specifically, we want an integer $\omega$ such that the smallest integer $m$ that satisfies $\omega^m = 1$ is $m = 2^k,$ for some power of $2.$ So when can we find such an $\omega?$ By the discussion of primitive roots, this can happen precisely when $2^k | p-1.$ Then all we do is find a primitive root $g$, and take $\omega = g^{\frac{p-1}{2^k}}.$ Finding primitive roots is fast because math proves that there is always a small primitive root. So everything works just as you expect! The same exact code works, just put a $\pmod{p}$ at the end of every line and you're set. 321 | -------------------------------------------------------------------------------- /chapters/problems.tex: -------------------------------------------------------------------------------- 1 | \chapter{Problems} 2 | 3 | Problems, to be worked through as you progress through the text. 4 | 5 | \section{Bronze} 6 | 7 | USACO bronze is quite ad hoc. Knowing basic data structures in the standard library helps. 8 | 9 | \section{Silver} 10 | 11 | USACO silver tends to have the most standard problems. Silver tests knowledge of basic algorithms and coding ideas. Silver questions also sometimes require knowledge of the language, like familiarity with standard library data structures. 12 | 13 | \subsection{Complete Search} 14 | 15 | \subsection{Greedy} 16 | 17 | \subsection{Standard Dynamic Programming} 18 | 19 | \begin{enumerate} 20 | 21 | \item 22 | (USACO Training) 23 | There is a long list of stalls, some of which need to be covered with boards. You can use up to $N$ ($1 \le N \le 50$) boards, each of which may cover any number of consecutive stalls. Cover all the necessary stalls, while covering as few total stalls as possible. 24 | 25 | \item 26 | (IOI 1996) 27 | You are given a three-valued (1, 2, or 3) sequence of length up to 1000. Find a minimum set of exchanges to put the sequence in sorted order. 28 | 29 | \item 30 | (Samir Khuller) 31 | We are given $N$ jobs, each of which requires one unit of time to complete. The $i$th job opens at some time $t_i$ and must be completed by some deadline $d_i$, where $t_i,d_i\in \mathbb{Z}$. Given that only one job can be completed at a time, determine if all $N$ can be completed by their deadlines. 32 | 33 | \end{enumerate} 34 | 35 | \subsection{Standard Graph Theory} 36 | 37 | \subsubsection{Flood Fill} 38 | 39 | \subsubsection{Shortest Path} 40 | 41 | \begin{enumerate} 42 | 43 | \item 44 | (USACO Training Pages, butter) 45 | Farmer John owns a collection of pastures with weighted edges between some 46 | pairs of locations. Each pasture is inhabited by a cow, and the cows wish to all congregate at one of the 47 | pastures. Find the pasture at which the cows should meet in order to minimize combined travel distance. 48 | 49 | \item 50 | (USACO February 2012, relocate) 51 | FJ is moving! He is trying to find the best place to build a new farm so as to 52 | minimize his daily travel time. 53 | 54 | The region to which FJ plans to move has $N$ towns ($1 \le N \le 10, 000$). There are $M$ bi-directional roads 55 | ($1 \le M \le 50, 000$) connecting certain pairs of towns. All towns are reachable from each-other via some 56 | combination of roads. FJ needs your help selecting the best town as the home for his new farm. 57 | 58 | There are markets in $K$ of the towns ($1 \le K \le 5$) that FJ wants to visit every day. In particular, every day 59 | he plans to leave his new farm, visit the $K$ towns with markets, and then return to his farm. FJ can visit the 60 | markets in any order he wishes. When selecting a town in which to build his new farm, FJ wants to choose 61 | only from the $N - K$ towns that do not have markets, since housing prices are lower in those towns. 62 | 63 | Please help FJ compute the minimum distance he will need to travel during his daily schedule, if he builds his 64 | farm in an optimal location and chooses his travel schedule to the markets as smartly as possible. 65 | 66 | \item 67 | (USACO December 2012, mroute) 68 | Farmer John's farm has an outdated network of $M$ pipes ($1 \le M \le 500$) for 69 | pumping milk from the barn to his milk storage tank. He wants to remove 70 | and update most of these over the next year, but he wants to leave exactly 71 | one path worth of pipes intact, so that he can still pump milk from the 72 | barn to the storage tank. 73 | 74 | The pipe network is described by $N$ junction points ($1 \le N \le 500$), each of 75 | which can serve as the endpoint of a set of pipes. Junction point 1 is the 76 | barn, and junction point $N$ is the storage tank. Each of the $M$ 77 | bi-directional pipes runs between a pair of junction points, and has an 78 | associated latency (the amount of time it takes milk to reach one end of 79 | the pipe from the other) and capacity (the amount of milk per unit time 80 | that can be pumped through the pipe in steady state). Multiple pipes 81 | can connect between the same pair of junction points. 82 | 83 | For a path of pipes connecting from the barn to the tank, the latency 84 | of the path is the sum of the latencies of the pipes along the path, 85 | and the capacity of the path is the minimum of the capacities of the 86 | pipes along the path (since this is the ``bottleneck'' constraining the 87 | overall rate at which milk can be pumped through the path). If FJ 88 | wants to send a total of $X$ units of milk through a path of pipes with 89 | latency $L$ and capacity $C$, the time this takes is therefore $L + \frac{X}{C}$. 90 | 91 | Given the structure of FJ's pipe network, please help him select a single 92 | path from the barn to the storage tank that will allow him to pump $X$ units 93 | of milk in a minimum amount of total time. 94 | 95 | \item 96 | (IOI 1999, Traffic Lights) 97 | In the city of Dingilville the traffic is arranged in an unusual way. There are junctions 98 | and roads connecting the junctions. There is at most one road between any two different junctions. There is no 99 | road connecting a junction to itself. Travel time for a road is the same for both directions. At every junction 100 | there is a single traffic light that is either blue or purple at any moment. The color of each light alternates 101 | periodically: blue for certain duration and then purple for another duration. Traffic is permitted to travel 102 | down the road between any two junctions, if and only if the lights at both junctions are the same color at the 103 | moment of departing from one junction for the other. If a vehicle arrives at a junction just at the moment the 104 | lights switch it must consider the new colors of lights. Vehicles are allowed to wait at the junctions. You are 105 | given the city map which shows 106 | \begin{itemize} 107 | \item 108 | the travel times for all roads (integers), 109 | \item 110 | the durations of the two colors at each junction (integers) 111 | \item 112 | the initial color of the light and the remaining time (integer) for this color to change at each junction. 113 | \end{itemize} 114 | Your task is to find a path which takes the minimum time from a given source junction to a given destination 115 | junction for a vehicle when the traffic starts. In case more than one such path exists you are required to report 116 | only one of them. 117 | \end{enumerate} 118 | 119 | \subsubsection{Minimal Spanning Tree} 120 | 121 | \begin{enumerate} 122 | 123 | \item 124 | (USACO March 2014, irrigation) 125 | Due to a lack of rain, Farmer John wants to build an irrigation system to 126 | send water between his $N$ fields ($1 \le N \le 2000$). 127 | 128 | Each field $i$ is described by a distinct point $(x_i, y_i)$ in the 2D plane, 129 | with 0 <= xi, yi <= 1000. The cost of building a water pipe between two 130 | fields $i$ and $j$ is equal to the squared Euclidean distance between them: 131 | 132 | \[(x_i - x_j)^2 + (y_i - y_j)^2\] 133 | 134 | FJ would like to build a minimum-cost system of pipes so that all of his 135 | fields are linked together -- so that water in any field can follow a 136 | sequence of pipes to reach any other field. 137 | 138 | Unfortunately, the contractor who is helping FJ install his irrigation 139 | system refuses to install any pipe unless its cost (squared Euclidean 140 | length) is at least $C$ ($1 \le C \le 1,000,000$). 141 | 142 | Please help FJ compute the minimum amount he will need pay to connect all 143 | his fields with a network of pipes. 144 | 145 | \item 146 | (USACO February 2015, superbull) 147 | Bessie and her friends are playing hoofball in the annual Superbull championship, and Farmer John is in charge of making the tournament as exciting as possible. A total of $N$ ($1 \le N \le 2000$) teams are playing in the Superbull. Each team is assigned a distinct integer team ID in the range $[1,2^{30}-1]$ to distinguish it from the other teams. The Superbull is an elimination tournament -- after every game, Farmer John chooses which team to eliminate from the Superbull, and the eliminated team can no longer play in any more games. The Superbull ends when only one team remains. 148 | 149 | Farmer John notices a very unusual property about the scores in matches! In any game, the combined score of the two teams always ends up being the bitwise exclusive OR ($XOR$) of the two team IDs. For example, if teams 12 and 20 were to play, then 24 points would be scored in that game, since $01100 XOR 10100 = 11000$. 150 | 151 | Farmer John believes that the more points are scored in a game, the more exciting the game is. Because of this, he wants to choose a series of games to be played such that the total number of points scored in the Superbull is maximized. Please help Farmer John organize the matches. 152 | 153 | \item 154 | (SPOJ, INVENT) 155 | Given tree with $N$ ($1 \le N \le 15,000$) vertices, find the minimum possible weight of a complete 156 | graph (a graph where every pair of vertices is connected) such that the given tree is its unique minimum spanning 157 | tree. 158 | \end{enumerate} 159 | 160 | \subsubsection{Union-Find} 161 | 162 | \begin{enumerate} 163 | 164 | \item 165 | (USACO February 2013, tractor) 166 | One of Farmer John's fields is particularly hilly, and he wants to purchase 167 | a new tractor to drive around on it. The field is described by an $N \times N$ 168 | grid of non-negative integer elevations ($1 \le N \le 500$). A tractor capable 169 | of moving from one grid cell to an adjacent cell (one step north, east, 170 | south, or west) of height difference $D$ costs exactly $D$ units of money. 171 | 172 | FJ would like to pay enough for his tractor so that, starting from some 173 | grid cell in his field, he can successfully drive the tractor around to 174 | visit at least half the grid cells in the field (if the number of total 175 | cells in the field is odd, he wants to visit at least half the cells 176 | rounded up). Please help him compute the minimum cost necessary for buying 177 | a tractor capable of this task. 178 | 179 | \item 180 | (CF 266E) 181 | There are $n$ ($1 \le n \le 10^5$) employees working in company ``X'' (let's number them from 1 to $n$ for convenience). Initially the employees didn't have any relationships among each other. On each of $m$ ($1 \le m \le 10^5$) next days one of the following events took place: 182 | 183 | \begin{itemize} 184 | \item 185 | either employee $y$ became the boss of employee $x$ (at that, employee $x$ didn't have a boss before); 186 | \item 187 | or employee $x$ gets a packet of documents and signs them; then he gives the packet to his boss. The boss signs the documents and gives them to his boss and so on (the last person to sign the documents sends them to the archive); 188 | \item 189 | or comes a request of type ``determine whether employee $x$ signs certain documents''. 190 | \end{itemize} 191 | 192 | Your task is to write a program that will, given the events, answer the queries of the described type. At that, it is guaranteed that throughout the whole working time the company didn't have cyclic dependencies. 193 | 194 | \end{enumerate} 195 | 196 | \subsubsection{Euler Tour} 197 | 198 | \begin{enumerate} 199 | 200 | \item 201 | (USACO Training, Airplane Hopping) 202 | Given a collection of cities, along with the flights between those cities, determine if there is a sequence of flights such that you take every flight exactly once, and end up at the place you started. (In other words, find an Eulerian circuit on a directed graph.) 203 | 204 | \item 205 | (USACO Training, Cows on Parade) 206 | Farmer John has two types of cows: black Angus and white Jerseys. While marching 19 of their cows to market the other day, John's wife Farmeress Joanne, noticed that all 16 possibilities of four successive black and white cows (e.g., $bbbb$, $bbbw$, $bbwb$, $bbww$, \ldots, $wwww$) were present. Of course, some of the combinations overlapped others. 207 | 208 | Given $N$ ($2 \le N \le 15$), find the minimum length sequence of cows such that every combination of $N$ successive black and white cows occurs in that sequence. 209 | 210 | (The answer is not hard to guess, but use Eulerian circuits to prove that it is correct.) 211 | 212 | \end{enumerate} 213 | 214 | \subsection{Easy Computational Geometry} 215 | 216 | \section{Gold} 217 | 218 | USACO gold problems generally fall into two families; the first tests knowledge of more complex data structures not in the standard library, and the second tests cleverness with more nonstandard greedy or dynamic programming strategies. 219 | 220 | \subsection{More Dynamic Programming} 221 | 222 | \subsection{Binary Search} 223 | 224 | \begin{enumerate} 225 | 226 | \item 227 | (USACO March 2014, sabotage) 228 | Farmer John's arch-nemesis, Farmer Paul, has decided to sabotage Farmer 229 | John's milking equipment! 230 | 231 | The milking equipment consists of a row of $N$ ($3 \le N \le 100,000$) 232 | milking machines, where the $i$th machine produces $M_i$ units of milk ($1 \le M_i \le 10,000$). Farmer Paul plans to disconnect a contiguous block 233 | of these machines -- from the $i$th machine up to the $j$th machine ($2 \le i \le j \le N-1$); note that Farmer Paul does not want to disconnect 234 | either the first or the last machine, since this will make his plot 235 | too easy to discover. Farmer Paul's goal is to minimize the average 236 | milk production of the remaining machines. Farmer Paul plans to 237 | remove at least 1 cow, even if it would be better for him to avoid 238 | sabotage entirely. 239 | 240 | Fortunately, Farmer John has learned of Farmer Paul's evil plot, and 241 | he is wondering how bad his milk production will suffer if the plot 242 | succeeds. Please help Farmer John figure out the minimum average milk 243 | production of the remaining machines if Farmer Paul does succeed. 244 | 245 | \item 246 | (Matthew Savage) 247 | Ashley's journey through Unova is just beginning, and she has just picked her first Pok\'{e}mon! Unfortunately, not knowing much about them, she picked a Snivy, and a particularly stubborn and unfriendly one at that. 248 | 249 | Being Ashley, she decides to try to win over the Snivy in the only way she knows how -- baked goods. 250 | 251 | Ashley knows $r$ $(0 \le r \le 1000$) recipes for Pok\'{e}puffs. Each recipe $r_i$ has a deliciousness rating $d_i$ ($0 \le d_i \le 1000$) and requires some combination of the $I$ ($0 \le I \le 1000$) available ingredients. (More specifically, the recipe $r_i$ uses the quantity $I_{ij}$ ($0 \le I_{ij} \le 1000$) of ingredient $I_j$.) 252 | 253 | Ashley has some amount of each ingredient $I_j$ on hand $A_j$ ($0 \le A_j \le 10^9$) and can buy more from the nearby store for a price of $c_j$ ($1 \le c_j \le 10^9$) dollars per unit using the $M$ ($0 \le M \le 10^{12}$) dollars she currently has. 254 | 255 | Of course, Ashley also has limited supplies and therefore can only produce Pok\'{e}puffs from a single recipe. However, she can make as many as she wants, in integer increments. 256 | 257 | We define ``total deliciousness'' ($D$) to be the sum of the deliciousnesses of the individual Poképuffs that Ashley has baked. 258 | 259 | Ashley wants to have the best chance possible with Snivy, and therefore would like to know - what is the maximum possible deliciousness ($\max(D)$) that she can produce? 260 | 261 | Note: there is a ``just do it'' solution that is faster than the binary search by a log factor. It is also \textit{much} more annoying to code; so annoying that I was unable to debug my ``just do it'' solution in the actual contest environment. I included this problem as an exercise to demonstrate how easy binary searching on the answer can be. 262 | 263 | \item 264 | (CF 287B) 265 | Vova, the Ultimate Thule new shaman, wants to build a pipeline. As there are exactly $n$ ($1 \le n \le 10^{18}$) houses in Ultimate Thule, Vova wants the city to have exactly $n$ pipes, each such pipe should be connected to the water supply. A pipe can be connected to the water supply if there's water flowing out of it. Initially Vova has only one pipe with flowing water. Besides, Vova has several splitters. 266 | 267 | A splitter is a construction that consists of one input (it can be connected to a water pipe) and $x$ output pipes. When a splitter is connected to a water pipe, water flows from each output pipe. You can assume that the output pipes are ordinary pipes. For example, you can connect water supply to such pipe if there's water flowing out from it. At most one splitter can be connected to any water pipe. 268 | 269 | Vova has one splitter of each kind: with $2, 3, 4, \ldots, k$ ($2 \le k \le 10^9$) outputs. Help Vova use the minimum number of splitters to build the required pipeline or otherwise state that it's impossible. 270 | 271 | Vova needs the pipeline to have exactly n pipes with flowing out water. Note that some of those pipes can be the output pipes of the splitters. 272 | 273 | \item 274 | (IOI 2009) 275 | Mecho the bear has found a little treasure - the bees' secret honeypot, which is full of honey! He was happily eating his newfound treasure until suddenly one bee saw him and sounded the bee alarm. He knows that at this very moment hordes of bees will emerge from their hives and start spreading around trying to catch him. He knows he has to leave the honeypot and go home quickly, but the honey is so sweet that Mecho doesn't want to leave too soon. Help Mecho determine the latest possible moment when he can leave. 276 | 277 | Mecho's forest is represented by a square grid of $N$ by $N$ ($1 \le N \le 800$) unit cells, whose sides are parallel to the north-south and east-west directions. Each cell is occupied by a tree, by a patch of grass, by a hive or by Mecho's home. Two cells are considered adjacent if one of them is immediately to the north, south, east or west of the other (but not on a diagonal). Mecho is a clumsy bear, so every time he makes a step, it has to be to an adjacent cell. Mecho can only walk on grass and cannot go through trees or hives, and he can make at most $S$ ($1 \le S \le 1000$) steps per minute. At the moment when the bee alarm is sounded, Mecho is in the grassy cell containing the honeypot, and the bees are in every cell containing a hive (there may be more than one hive in the forest). During each minute from this time onwards, the following events happen in the following order: 278 | \begin{itemize} 279 | \item 280 | If Mecho is still eating honey, he decides whether to keep eating or to leave. If he continues eating, he does not move for the whole minute. Otherwise, he leaves immediately and takes up to $S$ steps through the forest as described above. Mecho cannot take any of the honey with him, so once he has moved he cannot eat honey again. 281 | \item 282 | After Mecho is done eating or moving for the whole minute, the bees spread one unit further across the grid, moving only into the grassy cells. Specifically, the swarm of bees spreads into every grassy cell that is adjacent to any cell already containing bees. Furthermore, once a cell contains bees it will always contain bees (that is, the swarm does not move, but it grows). 283 | \end{itemize} 284 | In other words, the bees spread as follows: When the bee alarm is sounded, the bees only occupy the cells where the hives are located. At the end of the first minute, they occupy all grassy cells adjacent to hives (and still the hives themselves). At the end of the second minute, they additionally occupy all grassy cells adjacent to grassy cells adjacent to hives, and so on. Given enough time, the bees will end up simultaneously occupying all grassy cells in the forest that are within their reach. 285 | 286 | Neither Mecho nor the bees can go outside the forest. Also, note that according to the rules above, Mecho will always eat honey for an integer number of minutes. 287 | 288 | The bees catch Mecho if at any point in time Mecho finds himself in a cell occupied by bees. 289 | 290 | Write a program that, given a map of the forest, determines the largest number of minutes that Mecho can continue eating honey at his initial location, while still being able to get to his home before any of the bees catch him. 291 | 292 | \end{enumerate} 293 | 294 | \subsection{Segment Tree} 295 | 296 | \begin{enumerate} 297 | 298 | \item 299 | (SPOJ, CPAIR) 300 | Given a sequence $A_k$ of $N$ ($1 \le N \le 100, 000$) non-negative integers ($\forall k, A_k < 1000$), Answer $Q$ ($1 \le Q \le 301 | 100, 000$) queries, where each query consists of three integers, $v$, $a$, $b$. The answer is the number of pairs of 302 | integers $i$ and $j$ such that $1 \le i \le j \le N$, $a \le j - i + 1 \le b$, and $A_k \ge v$ for every integer $k$ between $i$, and 303 | $j$, inclusive. 304 | 305 | \end{enumerate} 306 | 307 | \subsection{More Standard Graph Theory} 308 | 309 | \subsubsection{Maximum Flow and Variants} 310 | 311 | \subsubsection{Strongly Connected Components} 312 | 313 | \subsubsection{Other Graph Theory} 314 | 315 | \begin{enumerate} 316 | \item 317 | (USACO Open 2008, nabor) 318 | Farmer John has $N$ ($1 \le N 319 | \le 100,000$) cows who group themselves into ``Cow 320 | Neighborhoods''. Each cow is at a unique rectilinear coordinate, on a pasture whose $x$ and $y$ coordinates are 321 | in the range $1\ldots1,000,000,000$. Two cows are neighbors if at least one of two criteria is met: (1) If the cows are 322 | no further than some integer Manhattan distance $C$ ($1 \le C \le 1,000,000,000$) apart. (2) If cow $A$ and $B$ are 323 | both neighbors of cow $Z$, then cow $A$ is a neighbor of cow $B$. Given the locations of the cows and the distance 324 | $C$, determine the number of neighborhoods and the number of cows in the largest neighborhood. 325 | 326 | \item 327 | (CF 260C) Andrew plays a game called ``Civilization''. Dima helps him. 328 | 329 | The game has $n$ ($1 \le n \le 3 \cdot 10^5$) cities and $m$ ($0 \le m < n$) bidirectional roads. The cities are numbered from 1 to $n$. Between any pair of cities there either is a single (unique) path, or there is no path at all. A path is such a sequence of distinct cities $v_1, v_2, \ldots, v_k$, that there is a road between any contiguous cities $v_i$ and $v_{i + 1}$ ($1 \le i < k$). The length of the described path equals to $k - 1$. We assume that two cities lie in the same region if and only if, there is a path connecting these two cities. 330 | 331 | During the game events of two types take place: 332 | 333 | \begin{itemize} 334 | \item 335 | Andrew asks Dima about the length of the longest path in the region where city $x$ lies. 336 | \item 337 | Andrew asks Dima to merge the region where city $x$ lies with the region where city $y$ lies. If the cities lie in the same region, then no merging is needed. Otherwise, you need to merge the regions as follows: choose a city from the first region, a city from the second region and connect them by a road so as to minimize the length of the longest path in the resulting region. If there are multiple ways to do so, you are allowed to choose any of them. 338 | \end{itemize} 339 | 340 | Dima finds it hard to execute Andrew's $q$ ($1 \le q \le 3 \cdot 10^5$) queries, so he asks you to help him. Help Dima. 341 | 342 | \end{enumerate} 343 | 344 | \subsection{Standard Computational Geometry} 345 | 346 | \subsection{Less Standard Problems} 347 | 348 | \section{Beyond} 349 | 350 | \subsection{Data Structure Nonsense} 351 | 352 | \subsection{Other Nonsense} 353 | -------------------------------------------------------------------------------- /chapters/graphs.tex: -------------------------------------------------------------------------------- 1 | \chapter{Graph Algorithms} 2 | 3 | In this chapter we explore some of the most famous results in graph theory. 4 | 5 | \section{Connected Components} 6 | 7 | A \textit{connected component} of an undirected graph is a subgraph such that, for any two vertices in the component, there exists a path from one to the other. The diagram illustrates three connected components of a graph, where each vertex is colored together with its associated component. 8 | 9 | \begin{center} 10 | \begin{tikzpicture}[very thick,level/.style={sibling distance=70mm/#1}] 11 | \draw (0, 0) node [vertex] (n1) {1}; 12 | \draw (2, 1) node [vertex] (n2) {2}; 13 | \draw (2, -1) node [vertex] (n3) {3}; 14 | \draw (4, 0) node [vertex] (n4) {4}; 15 | \draw (n1) -- (n2); 16 | \draw (n2) -- (n3); 17 | \draw (n3) -- (n4); 18 | \draw (n2) -- (n4); 19 | \draw (n1) -- (n3); 20 | \draw (6, -1) node [vertex, fill=mysalmon] (n5) {5}; 21 | \draw (6, 1) node [vertex, fill=mysalmon] (n6) {6}; 22 | \draw (8, 1) node [vertex, fill=mysalmon] (n7) {7}; 23 | \draw (8, -1) node [vertex, fill=mysalmon] (n8) {8}; 24 | \draw (n5) -- (n6) -- (n7) -- (n8) -- (n6); 25 | \draw (10, 0) node[vertex, fill=mypurple] (n9) {9}; 26 | \draw (12, 0) node[vertex, fill=mypurple] (n10) {10}; 27 | \draw (n9) -- (n10); 28 | \end{tikzpicture} 29 | \end{center} 30 | 31 | A \textit{strongly connected component} of a directed graph is a subgraph such that every vertex in the component can be reached from any other vertex in the component. 32 | 33 | \begin{center} 34 | \begin{tikzpicture}[very thick,level/.style={sibling distance=70mm/#1}] 35 | \draw (0, 0) node [vertex] (n1) {5}; 36 | \draw (2, 0) node [vertex] (n2) {6}; 37 | \draw (4, 0) node [vertex, fill=mysalmon] (n3) {7}; 38 | \draw (6, 0) node [vertex, fill=myred] (n4) {8}; 39 | \draw (0, 2) node [vertex] (m1) {1}; 40 | \draw (2, 2) node [vertex] (m2) {2}; 41 | \draw (4, 2) node [vertex, fill=mypurple] (m3) {3}; 42 | \draw (6, 2) node [vertex, fill=mysalmon] (m4) {4}; 43 | \draw[->] (m1) -- (m2); 44 | \draw[->] (m2) -- (n1); 45 | \draw[->] (n1) -- (m1); 46 | \draw[->] (n2) edge [bend left] (m2); 47 | \draw[->] (m2) edge [bend left] (n2); 48 | \draw[->] (n2) -- (n3); 49 | \draw[->] (m2) -- (m3); 50 | \draw[->] (m2) -- (n3); 51 | \draw[->] (n3) -- (m3); 52 | \draw[->] (n3) edge [bend left] (m4); 53 | \draw[->] (m4) edge [bend left] (n3); 54 | \draw[->] (n4) -- (m4); 55 | \end{tikzpicture} 56 | \end{center} 57 | 58 | Finding the connected components of an undirected graph is a straightforward problem, while finding the strongly connected components of a directed graph is more complicated and won't be covered in this chapter. 59 | 60 | \subsection{Flood Fill} 61 | 62 | Really any kind of search method solves the undirected graph connected components problem. We could use recursion with a depth-first search. To avoid using the run-time stack, we could use a queue to perform a breadth-first search. Both of these run in $O(E+V)$ time. I would recommend in general to use the BFS. 63 | 64 | \subsection{Union-Find (Disjoint Set Union)} 65 | 66 | The union-find data structure is another way for us to solve the connected components problem. Union-find is unique from the other search techniques in that it can process input as it is presented, edge by edge. This also means it is possible to add more edges at the end, therefore changing the graph, while still running quickly. An algorithm that works like this is an \textit{online algorithm}, while an algorithm that requires all the input data presented at the beginning is an \textit{offline algorithm}. 67 | 68 | A natural idea for solving the connected components problem is for each vertex to maintain a pointer to another vertex it's connected to, forming a \textit{forest}, or collection of trees. To check whether two elements are in the same component, simply trace the tree up to the root by jumping up each pointer. 69 | 70 | \begin{center} 71 | \begin{tikzpicture}[very thick,edge from parent/.style={draw,<-},level/.style={sibling distance=50mm/#1}] 72 | \node [vertex, fill = mysalmon] (r2) {5} 73 | child { 74 | node [vertex, fill = mysalmon] {6} 75 | child { node [vertex, fill=mysalmon] {7} } 76 | child { node [vertex, fill=mysalmon] {8} } 77 | }; 78 | 79 | \node [vertex] [left=6cm of r2] (r1) {1} 80 | child { 81 | node [vertex] {2} 82 | child { 83 | node [vertex] {4} 84 | } 85 | } 86 | child {node [vertex] {3} }; 87 | 88 | \node [vertex, fill=mypurple] [right=3cm of r2] (r3) {9} 89 | child { node [vertex, fill=mypurple] {10} }; 90 | \end{tikzpicture} 91 | \end{center} 92 | 93 | The idea of a pointer can easily be stored within an array. 94 | 95 | \begin{center} 96 | { 97 | \begin{tikzpicture}[ 98 | thick, 99 | myrect/.style={ 100 | draw, 101 | rectangle split, 102 | rectangle split horizontal, 103 | rectangle split parts=#1, 104 | rectangle split part align=left, 105 | text width=5ex, 106 | text centered 107 | }, 108 | mycallout/.style={ 109 | shape=rectangle callout, 110 | rounded corners, 111 | fill=mysalmon, 112 | callout absolute pointer={#1}, 113 | callout pointer width=1cm 114 | } 115 | ] 116 | 117 | \node[myrect=10, rectangle split part fill={myseagreen, myseagreen, myseagreen, myseagreen, mysalmon, mysalmon, mysalmon, mysalmon, mypurple, mypurple}] 118 | (array1) 119 | { 120 | \strut -1 121 | \nodepart{two} \strut 1 122 | \nodepart{three} \strut 1 123 | \nodepart{four} \strut 2 124 | \nodepart{five} \strut -1 125 | \nodepart{six} \strut 5 126 | \nodepart{seven} \strut 6 127 | \nodepart{eight} \strut 6 128 | \nodepart{nine} \strut -1 129 | \nodepart{ten} \strut 9 130 | }; 131 | \foreach \Valor [count=\Valori from 1] in {one ,two ,three ,four ,five ,six ,seven ,eight ,nine ,ten } 132 | \node[below] at (array1.\Valor south) {\Valori}; 133 | 134 | \end{tikzpicture} 135 | } 136 | \end{center} 137 | 138 | We want to support two operations: $find(v)$, which returns the root of the tree containing $v$, and $union(u,v)$, which merges the components containing $u$ and $v$. This second operation is easy given the first; simply set the pointer of $find(u)$ to be $find(v)$. 139 | 140 | $union(4, 6)$, unoptimized: 141 | 142 | \begin{center} 143 | \begin{tikzpicture}[very thick,edge from parent/.style={draw,<-},level/.style={sibling distance=50mm/#1}] 144 | \node [vertex] (r2) {5} 145 | child { 146 | node [vertex] (r1) {1} 147 | child { 148 | node [vertex] {2} 149 | child { 150 | node [vertex] {4} 151 | } 152 | } 153 | child {node [vertex] {3} } 154 | } 155 | child { 156 | node [vertex] {6} 157 | child { node [vertex] {7} } 158 | child { node [vertex] {8} } 159 | }; 160 | \node [vertex] [right=6cm of r2] (r3) {9} 161 | child { node [vertex] {10} }; 162 | \end{tikzpicture} 163 | \end{center} 164 | 165 | A problem quickly arises -- the $find$ operation threatens to become linear. There are two simple things we can do to optimize this. 166 | 167 | The first is to always add the shorter tree to the taller tree, as we want to minimize the maximum height. An easy heuristic for the height of the tree is simply the number of elements in that tree. We can keep track of the size of the tree with a second array. This heuristic is obviously not perfect, as a larger tree can be shorter than a smaller tree, but it turns out with our second optimization that this problem doesn't matter. 168 | 169 | The second fix is to simply assign the pointer associated with $v$ to be $find(v)$ at the end of the $find$ operation. We can design $find(v)$ to recursively call $find$ on the pointer associated with $v$, so this fix sets pointers associated with nodes along the entire chain from $v$ to $find(v)$ to be $find(v)$. These two optimizations combined make the $union$ and $find$ operations $O(\alpha (V))$, where $\alpha(n)$ is the inverse Ackermann function, and for all practical values of $n$, $\alpha(n) < 5$. 170 | 171 | $find(4)$, optimized: 172 | 173 | \begin{center} 174 | \begin{tikzpicture}[very thick,edge from parent/.style={draw,<-},level/.style={sibling distance=30mm/#1}] 175 | \node [vertex] (r2) {5} 176 | child { 177 | node [vertex] (r1) {1} 178 | child {node [vertex] {3} } 179 | } 180 | child { 181 | node [vertex] {6} 182 | child { node [vertex] {7} } 183 | child { node [vertex] {8} } 184 | } 185 | child {node[vertex] {2}} 186 | child {node[vertex] {4}}; 187 | \node [vertex] [right=6cm of r2] (r3) {9} 188 | child { node [vertex] {10} }; 189 | \end{tikzpicture} 190 | \end{center} 191 | 192 | \begin{algorithm}[H] 193 | \caption{Union-Find} 194 | %\label{} 195 | \begin{algorithmic} 196 | \Function{Find}{$v$} 197 | \If {$v$ is the root} 198 | \State \Return $v$ 199 | \EndIf 200 | \State $parent(v) \gets \Call{Find}{parent(v)}$ 201 | \State \Return $parent(v)$ 202 | \EndFunction 203 | \Function{Union}{$u$, $v$} 204 | \State $uRoot \gets \Call{Find}{u}$ 205 | \State $vRoot \gets \Call{Find}{v}$ 206 | \If {$uRoot = vRoot$} 207 | \State \Return 208 | \EndIf 209 | \If {$size(uRoot)] (v1) -- (v2) node[midway, left] {4}; 267 | \draw[->] (v2) -- (v3) node[midway, above right] {2}; 268 | \draw[->] (v1) -- (v3) node[midway, below] {7}; 269 | \draw[->] (v2) -- (v4) node[midway, above] {6}; 270 | \draw[->] (v3) -- (v4) node[midway, right] {3}; 271 | \draw[->] (v3) -- (v5) node[midway, below] {5}; 272 | \draw[->] (v4) -- (v5) node[midway, above] {4}; 273 | \end{tikzpicture} 274 | \end{center} 275 | 276 | Let's run Dijkstra's algorithm on the above graph with vertex 1 as the source. We first set all the distances besides the source to be $\infty$. 277 | 278 | \begin{center} 279 | \begin{tikzpicture}[very thick,edge from parent/.style={draw,<-},level/.style={sibling distance=30mm/#1}] 280 | \draw (0, 0) node [splitvertex, fill=mysalmon] (v1) {1\nodepart{lower}0}; 281 | \draw (1, 4) node [splitvertex] (v2) {2\nodepart{lower}$\infty$}; 282 | \draw (4, 1) node [splitvertex] (v3) {3\nodepart{lower}$\infty$}; 283 | \draw (5, 5) node [splitvertex] (v4) {4\nodepart{lower}$\infty$}; 284 | \draw (8, 3) node [splitvertex] (v5) {5\nodepart{lower}$\infty$}; 285 | \draw[->] (v1) -- (v2) node[midway, left] {4}; 286 | \draw[->] (v2) -- (v3) node[midway, above right] {2}; 287 | \draw[->] (v1) -- (v3) node[midway, below] {7}; 288 | \draw[->] (v2) -- (v4) node[midway, above] {6}; 289 | \draw[->] (v3) -- (v4) node[midway, right] {3}; 290 | \draw[->] (v3) -- (v5) node[midway, below] {5}; 291 | \draw[->] (v4) -- (v5) node[midway, above] {4}; 292 | \end{tikzpicture} 293 | \end{center} 294 | 295 | Now, we continue choosing the closest unvisited node, mark it as visited, and and update its neighbors. 296 | 297 | \begin{center} 298 | \begin{tikzpicture}[very thick,edge from parent/.style={draw,<-},level/.style={sibling distance=30mm/#1}] 299 | \draw (0, 0) node [splitvertex, fill=myred] (v1) {1\nodepart{lower}0}; 300 | \draw (1, 4) node [splitvertex, fill=mysalmon] (v2) {2\nodepart{lower}4}; 301 | \draw (4, 1) node [splitvertex, fill=mysalmon] (v3) {3\nodepart{lower}7}; 302 | \draw (5, 5) node [splitvertex] (v4) {4\nodepart{lower}$\infty$}; 303 | \draw (8, 3) node [splitvertex] (v5) {5\nodepart{lower}$\infty$}; 304 | \draw[->] (v1) -- (v2) node[midway, left] {4}; 305 | \draw[->] (v2) -- (v3) node[midway, above right] {2}; 306 | \draw[->] (v1) -- (v3) node[midway, below] {7}; 307 | \draw[->] (v2) -- (v4) node[midway, above] {6}; 308 | \draw[->] (v3) -- (v4) node[midway, right] {3}; 309 | \draw[->] (v3) -- (v5) node[midway, below] {5}; 310 | \draw[->] (v4) -- (v5) node[midway, above] {4}; 311 | \end{tikzpicture} 312 | \end{center} 313 | 314 | \begin{center} 315 | \begin{tikzpicture}[very thick,edge from parent/.style={draw,<-},level/.style={sibling distance=30mm/#1}] 316 | \draw (0, 0) node [splitvertex, fill=myred] (v1) {1\nodepart{lower}0}; 317 | \draw (1, 4) node [splitvertex, fill=myred] (v2) {2\nodepart{lower}4}; 318 | \draw (4, 1) node [splitvertex, fill=mysalmon] (v3) {3\nodepart{lower}6}; 319 | \draw (5, 5) node [splitvertex, fill=mysalmon] (v4) {4\nodepart{lower}10}; 320 | \draw (8, 3) node [splitvertex] (v5) {5\nodepart{lower}$\infty$}; 321 | \draw[->] (v1) -- (v2) node[midway, left] {4}; 322 | \draw[->] (v2) -- (v3) node[midway, above right] {2}; 323 | \draw[->] (v1) -- (v3) node[midway, below] {7}; 324 | \draw[->] (v2) -- (v4) node[midway, above] {6}; 325 | \draw[->] (v3) -- (v4) node[midway, right] {3}; 326 | \draw[->] (v3) -- (v5) node[midway, below] {5}; 327 | \draw[->] (v4) -- (v5) node[midway, above] {4}; 328 | \end{tikzpicture} 329 | \end{center} 330 | 331 | \begin{center} 332 | \begin{tikzpicture}[very thick,edge from parent/.style={draw,<-},level/.style={sibling distance=30mm/#1}] 333 | \draw (0, 0) node [splitvertex, fill=myred] (v1) {1\nodepart{lower}0}; 334 | \draw (1, 4) node [splitvertex, fill=myred] (v2) {2\nodepart{lower}4}; 335 | \draw (4, 1) node [splitvertex, fill=myred] (v3) {3\nodepart{lower}6}; 336 | \draw (5, 5) node [splitvertex, fill=mysalmon] (v4) {4\nodepart{lower}9}; 337 | \draw (8, 3) node [splitvertex, fill=mysalmon] (v5) {5\nodepart{lower}14}; 338 | \draw[->] (v1) -- (v2) node[midway, left] {4}; 339 | \draw[->] (v2) -- (v3) node[midway, above right] {2}; 340 | \draw[->] (v1) -- (v3) node[midway, below] {7}; 341 | \draw[->] (v2) -- (v4) node[midway, above] {6}; 342 | \draw[->] (v3) -- (v4) node[midway, right] {3}; 343 | \draw[->] (v3) -- (v5) node[midway, below] {5}; 344 | \draw[->] (v4) -- (v5) node[midway, above] {4}; 345 | \end{tikzpicture} 346 | \end{center} 347 | 348 | \begin{center} 349 | \begin{tikzpicture}[very thick,edge from parent/.style={draw,<-},level/.style={sibling distance=30mm/#1}] 350 | \draw (0, 0) node [splitvertex, fill=myred] (v1) {1\nodepart{lower}0}; 351 | \draw (1, 4) node [splitvertex, fill=myred] (v2) {2\nodepart{lower}4}; 352 | \draw (4, 1) node [splitvertex, fill=myred] (v3) {3\nodepart{lower}6}; 353 | \draw (5, 5) node [splitvertex, fill=myred] (v4) {4\nodepart{lower}9}; 354 | \draw (8, 3) node [splitvertex, fill=mysalmon] (v5) {5\nodepart{lower}14}; 355 | \draw[->] (v1) -- (v2) node[midway, left] {4}; 356 | \draw[->] (v2) -- (v3) node[midway, above right] {2}; 357 | \draw[->] (v1) -- (v3) node[midway, below] {7}; 358 | \draw[->] (v2) -- (v4) node[midway, above] {6}; 359 | \draw[->] (v3) -- (v4) node[midway, right] {3}; 360 | \draw[->] (v3) -- (v5) node[midway, below] {5}; 361 | \draw[->] (v4) -- (v5) node[midway, above] {4}; 362 | \end{tikzpicture} 363 | \end{center} 364 | 365 | \begin{center} 366 | \begin{tikzpicture}[very thick,edge from parent/.style={draw,<-},level/.style={sibling distance=30mm/#1}] 367 | \draw (0, 0) node [splitvertex, fill=myred] (v1) {1\nodepart{lower}0}; 368 | \draw (1, 4) node [splitvertex, fill=myred] (v2) {2\nodepart{lower}4}; 369 | \draw (4, 1) node [splitvertex, fill=myred] (v3) {3\nodepart{lower}6}; 370 | \draw (5, 5) node [splitvertex, fill=myred] (v4) {4\nodepart{lower}9}; 371 | \draw (8, 3) node [splitvertex, fill=myred] (v5) {5\nodepart{lower}14}; 372 | \draw[line width=2mm] (v1) -- (v2) node[midway, left] {4}; 373 | \draw[line width=2mm] (v2) -- (v3) node[midway, above right] {2}; 374 | \draw[->] (v1) -- (v3) node[midway, below] {7}; 375 | \draw[->] (v2) -- (v4) node[midway, above] {6}; 376 | \draw[line width=2mm] (v3) -- (v4) node[midway, right] {3}; 377 | \draw[line width=2mm] (v3) -- (v5) node[midway, below] {5}; 378 | \draw[->] (v4) -- (v5) node[midway, above] {4}; 379 | \end{tikzpicture} 380 | \end{center} 381 | 382 | The slow part of the $O(V^2)$ formulation is the linear search for the vertex $v$ with the minimum $dist(v)$. We happen to have a data structure that resolves this problem -- a binary heap. The main problem with using the standard library heap is having repeated vertices in the heap. We could just ignore this problem and discard visited vertices as they come out of the heap. Alternatively, we could choose never to have repeated vertices in the heap. To do this, we need to be able to change the value of the distances once they are already in the heap, or \textit{decrease-key}. This is a pretty simple function to add, however, if you have a heap already coded. Either way, we achieve $O(E \log{V})$, as we do $E+V$ updates to our heap, each costing $O(V)$. 383 | 384 | \subsection{Floyd-Warshall} 385 | 386 | Dijkstra is nice when we are dealing with edges with nonnegative weights and are looking for the distances from one vertex to all the others. Floyd-Warshall solves the shortest path problem for all pairs of vertices in $O(V^3)$ time, which is faster than $V$ single-source Dijkstra runs on a dense graph. Floyd-Warshall works even if some edge weights are negative but not if the graph has a negative cycle. 387 | 388 | \begin{algorithm}[H] 389 | \caption{Floyd-Warshall} 390 | \begin{algorithmic} 391 | \ForAll{vertices $v$} 392 | \State $dist(v,v)=0$ 393 | \EndFor 394 | \ForAll{edges $(u,v)$} 395 | \State $dist(u,v)=weight(u,v)$ 396 | \EndFor 397 | \ForAll{vertices $k$} 398 | \ForAll{vertices $i$} 399 | \ForAll{vertices $j$} 400 | \If{$dist(i,j) > dist(i,k)+dist(k,j)$} 401 | \State $dist(i,j) \gets dist(i,k)+dist(k,j)$ 402 | \EndIf 403 | \EndFor 404 | \EndFor 405 | \EndFor 406 | \end{algorithmic} 407 | \end{algorithm} 408 | 409 | \subsection{Bellman-Ford} 410 | 411 | Bellman-Ford is a single-source $O(VE)$ shortest path algorithm that works when edge weights can be negative. It is preferable to Floyd-Warshall when the graph is sparse and we only need the answer for one source. Like Floyd-Warshall, the algorithm fails if the graph contains a negative cycle, but the algorithm is still useful for detecting negative cycles. 412 | 413 | The idea here is the shortest path, assuming no negative cycles, has length at most $V-1$. 414 | 415 | \begin{algorithm}[H] 416 | \caption{Bellman-Ford} 417 | \begin{algorithmic} 418 | \ForAll{vertices $v$} 419 | \State $dist(v)\gets\infty$ 420 | \State $prev(v)=\gets -1$ 421 | \EndFor 422 | \State $dist(src) \gets 0$ 423 | \For{$i\equiv 1,V-1$} 424 | \ForAll{edges $(u,v)$} 425 | \If{$dist(u)+weight(u,v) < dist(v)$} 426 | \State $dist(v) \gets dist(u)+weight(u,v)$ 427 | \State $prev(v) \gets u$ 428 | \EndIf 429 | \EndFor 430 | \EndFor 431 | \ForAll{edges $(u,v)$} 432 | \Comment{check for negative cycles} 433 | \If{$dist(u)+weight(u,v) < dist(v)$} 434 | \State{negative cycle detected} 435 | \EndIf 436 | \EndFor 437 | \end{algorithmic} 438 | \end{algorithm} 439 | 440 | \section{Minimum Spanning Tree} 441 | 442 | Consider a connected, undirected graph. A \textit{spanning tree} is a subgraph that is a tree and contains every vertex in the original graph. A \textit{minimum spanning tree} is a spanning tree such that the sum of the edge weights of the tree is minimized. Finding the minimum spanning tree uses many of the same ideas discussed earlier. 443 | 444 | \begin{center} 445 | \begin{tikzpicture}[very thick,edge from parent/.style={draw,<-},level/.style={sibling distance=30mm/#1}] 446 | \draw (0, 0) node [vertex] (v1) {1}; 447 | \draw (1, 4) node [vertex] (v2) {2}; 448 | \draw (4, 1) node [vertex] (v3) {3}; 449 | \draw (5, 5) node [vertex] (v4) {4}; 450 | \draw (8, 3) node [vertex] (v5) {5}; 451 | \draw[line width=2mm] (v1) -- (v2) node[midway, left] {4}; 452 | \draw[line width=2mm] (v2) -- (v3) node[midway, above right] {2}; 453 | \draw (v1) -- (v3) node[midway, below] {7}; 454 | \draw (v2) -- (v4) node[midway, above] {6}; 455 | \draw[line width=2mm] (v3) -- (v4) node[midway, right] {3}; 456 | \draw (v3) -- (v5) node[midway, below] {5}; 457 | \draw[line width=2mm] (v4) -- (v5) node[midway, above] {4}; 458 | \end{tikzpicture} 459 | \end{center} 460 | 461 | \subsection{Prim} 462 | 463 | Prim's algorithm for finding the minimum spanning tree is very similar to Dijkstra's algorithm for finding the shortest path. Like Dijkstra, it iteratively adds a new vertex at a time to build a tree. The only difference is $dist(v)$ stores the shortest distance from \textit{any} visited node instead of the source. 464 | 465 | \begin{algorithm}[H] 466 | \caption{Prim} 467 | \begin{algorithmic} 468 | \ForAll{vertices $v$} 469 | \State $dist(v) \gets \infty$ 470 | \State $visited(v) \gets 0$ 471 | \State $prev(v) \gets -1$ 472 | \EndFor 473 | \State $dist(src) \gets 0$ 474 | \While{$\exists v$ s.t. $visited(v)=0$} 475 | \State $v \equiv v$ s.t. $visited(v)=0$ with min $dist(v)$ 476 | \State $visited(v) \gets 1$ 477 | \ForAll{neighbors $u$ of $v$} 478 | \If{$visited(u) = 0$} 479 | \If{$weight(v, u) < dist(u)$} 480 | \State $dist(u) \gets weight(v, u)$ 481 | \State $prev(u) \gets v$ 482 | \EndIf 483 | \EndIf 484 | \EndFor 485 | \EndWhile 486 | \end{algorithmic} 487 | \end{algorithm} 488 | 489 | The proof of correctness is left as an exercise. The complexity of this algorithm depends on how the minimum unvisited vertex is calculated. Using the same approaches as Dijkstra, we can achieve $O(V^2)$ or $O(E \log{V})$. 490 | 491 | \subsection{Kruskal} 492 | 493 | While Prim greedily adds vertices to the tree, Kruskal's algorithm greedily adds edges. It iterates over all the edges, sorted by weight. We need to watch out for adding a cycle, breaking the tree structure, which means we need to keep track of each vertex's connected component. If an edge connects two vertices from the same connected component, we don't want to add it to our tree. However, we have a union-find algorithm that works perfectly for this. 494 | 495 | \begin{algorithm}[H] 496 | \caption{Kruskal} 497 | \begin{algorithmic} 498 | \ForAll{edges $(u,v)$ in sorted order} 499 | \If{$\Call{Find}{u} \not= \Call{Find}{v}$} 500 | \State add $(u,v)$ to spanning tree 501 | \State $\Call{Union}{u,v}$ 502 | \EndIf 503 | \EndFor 504 | \end{algorithmic} 505 | \end{algorithm} 506 | 507 | This algorithm requires a sort of the edges and thus has complexity $O(E \log{E}) = O(E \log{V})$. 508 | 509 | \section{Eulerian Tour} 510 | 511 | An \textit{Eulerian tour} of a graph is a path that traverses every edge exactly once. If the tour ends exactly where it started, it is called an \textit{Eulerian circuit}. A graph has an Eulerian circuit if it is connected and every vertex has even degree. A graph has an Eulerian path if it is connected and all vertices but exactly two have even degrees. The mathematical proofs for these graph properties hinge on the idea that removing a cycle from the graph maintains the Eulerian property. We construct an Eulerian tour by appealing to this idea. 512 | 513 | \begin{algorithm}[H] 514 | \caption{Eulerian Tour} 515 | \begin{algorithmic} 516 | \Function{FindTour}{$v$} 517 | \While{$v$ has a neighbor $u$} 518 | \State delete edge $(v,u)$ 519 | \State \Call{FindTour}{$u$} 520 | \Comment{\Call{FindTour}{$u$} must trace a circuit back to $v$} 521 | \EndWhile 522 | \State add $v$ to tour 523 | \EndFunction 524 | \end{algorithmic} 525 | \end{algorithm} 526 | 527 | It is not preferable to use the run-time stack; we can use our own stack if necessary. 528 | 529 | If the graph contains an Eulerian circuit, we call this function on any vertex we like. If it contains an Eulerian path, we call this function on one of the vertices with odd degree. 530 | -------------------------------------------------------------------------------- /chapters/trees.tex: -------------------------------------------------------------------------------- 1 | \chapter{Tree Algorithms} 2 | 3 | Up until now, we have only looked at algorithms that deal with general graphs. However, there is also much to be said about graphs with additional structure. In this section, we'll explore some problems and their solutions on trees. First, some definitions. 4 | 5 | An undirected graph $G$ is a \emph{tree} if one of the following equivalent conditions holds: 6 | \begin{itemize} 7 | \item 8 | $G$ is connected and has no cycles. 9 | \item 10 | $G$ is connected with $n$ vertices and $n - 1$ edges. 11 | \item 12 | There exists exactly one simple path between any two vertices of $G$. 13 | \end{itemize} 14 | A \emph{rooted tree} is a tree where one vertex has been designated as the \emph{root}. This gives each edge a natural direction -- whether it leads towards or away from the root. In many problems, it is useful to arbitrarily designate a root. For example, one interpretation of a DFS on a tree is that we start from the root and traverse the tree downwards towards the leaves. (On trees, a \emph{leaf} is a vertex with degree 1.) 15 | 16 | We'll define a few more terms for rooted trees. An \emph{ancestor} of a vertex $v$ is another vertex $a$ that lies on the path between $v$ and the root. The \emph{parent} of a vertex is its closest ancestor. The \emph{depth} of a vertex is its distance from the root. We will use $depth(v)$ to denote the depth of a vertex $v$. 17 | 18 | \section{DFS on Trees} 19 | 20 | DFS is a very useful technique for collecting information about the structure of a tree. For example, we can compute in linear time the depth of each node, the size of each subtree or the diameter of the entire tree. This can be done by recursively computing results for each subtree, and then combining this data to obtain the final result. Sometimes, it takes more than one DFS to collect all the necessary information. Calculating node depth and subtree size via DFS is relatively straightforward. Below is a snippet of pseudocode computing these two sets of values. 21 | 22 | \noindent \begin{minipage}{\textwidth} 23 | \begin{algorithmic} 24 | \Function{DFS}{$v$, $p$} 25 | \Comment{$v$ is the current vertex, $p$ is its parent} 26 | \State $sum(v) \gets 1$ 27 | \Comment $sum(v)$ is size of the subtree rooted at vertex $v$ 28 | \State $depth(v) \gets depth(p) + 1$ 29 | \Comment $depth(v)$ is the depth of vertex $v$ 30 | \ForAll{vertices $n$ adjacent to $v$} 31 | \If{$n \neq p$} 32 | \State \Call{DFS}{$n$, $v$} 33 | \State $sum(v) \gets sum(v) + sum(n)$ 34 | \EndIf 35 | \EndFor 36 | \EndFunction 37 | \end{algorithmic} 38 | \end{minipage} 39 | 40 | Computing the diameter of a tree is a bit trickier. For a given tree, let $r$ be its root, and let $a$ be a vertex of maximum depth. It turns out that the diameter of the tree is the maximum distance between $a$ and any other vertex in the tree. (Try to prove this!) Thus we can run two DFSes to compute the diameter, calculating the depth of each vertex with each pass. 41 | 42 | Depth, subtree size and diameter are only a few of the values we can compute for trees. This style of DFS shows up often in problems and is also fundamental to many more complex tree algorithms. 43 | 44 | \section{Jump Pointers} 45 | 46 | \emph{Jump pointers} are a technique for decomposing paths on rooted trees. The central idea is that we can always take a path and break it into $O(\log n)$ chunks of size $2^i$. By augmenting these smaller chunks with data, we can use jump pointers to quickly answer a variety of queries, including level ancestor and lowest common ancestor. 47 | 48 | Let's start with an example. The most fundamental application of jump pointers is to level ancestor queries---for any vertex $v$, we want to be able to find its $k$th ancestor in $O(\log n)$ time.To solve this problem, we precompute for each vertex a pointer to its $2^i$th ancestor for all possible values of $i$. These are the ``jump pointers'' for which this technique is named. If our tree has $n$ vertices, then each vertex has at most $\lfloor \log_2 n \rfloor$ pointers leading out from it. Thus we have at most $O(n \log n)$ jump pointers in total. We can compute these with a DP in $O(n \log n)$. 49 | 50 | Using jump pointers, we can finish the level ancestor problem. Let $2^i$ be the largest power of 2 that is at most $k$. We use our jump pointers to obtain $u$, the $2^i$th ancestor of $v$. If $2^i$ is not equal to $k$, we can jump up again from $u$. Continuing recursively, we'll finish in $O(\log n)$ iterations, since we reduce $k$ by at least a factor of 2 each time. 51 | 52 | Now we'll tackle lowest common ancestor queries. The lowest common ancestor (LCA), $l$, of two vertices $u$ and $v$ is the deepest vertex that is an ancestor of both $u$ and $v$. LCA queries are especially useful because any path on a rooted tree can be broken into two shorter paths joined at the LCA of its endpoints. 53 | 54 | The solution of the LCA problem with jump pointers consists of two steps. The first step involves bringing the two queried vertices to the same depth. If we are looking for the LCA of $u$ and $v$, we can assume that $depth(u) > depth(v)$ without loss of generality. Let $u'$ be the ancestor of $u$ satisfying $depth(u') = depth(v)$. We can compute $u'$ with a single level ancestor query in $O(\log n)$ time. If $u' = v$, then we are done. 55 | 56 | Otherwise, if $u' \neq v$, we can find the LCA of $u'$ and $v$ by advancing them towards the root in increments of $2^i$ in a binary-search-like manner. If the $2^i$th ancestors of $u'$ and $v$ are distinct, then the LCA of $u'$ and $v$ is equal to the LCA of the $2^i$th ancestors of $u'$ and $v$. Thus, iterating down from the largest power of 2 less than $n$, we can move $u'$ and $v$ up the tree until they share a parent. This common parent is the LCA of $u$ and $v$. Since jump pointers allow us to access the $2^i$th ancestor of a vertex in $O(1)$, our algorithm runs in $O(\log n)$ time. 57 | 58 | \begin{algorithm}[H] 59 | \caption{Jump Pointers, Level Ancestor and LCA} 60 | \begin{algorithmic} 61 | \Function{BuildJumpPointers}{$V$, $par$} 62 | \Comment{Initially, $par(0, v)$ holds the parent of vertex $v$.} 63 | \State $par(i, v)$ denotes the $2^i$th ancestor of vertex $v$ 64 | \ForAll{i from 1 to $\lfloor \log_2 n \rfloor$} 65 | \ForAll{vertices $v$} 66 | \State $par(i, v) \gets par(i - 1, par(i - 1, v))$ 67 | \EndFor 68 | \EndFor 69 | \EndFunction 70 | \Function{LevelAncestor}{$u$, $k$} 71 | \ForAll{i from $\lfloor \log_2 n \rfloor$ to 0} 72 | \If{$2^i \le k$} 73 | \State $u \gets par(i, u)$ 74 | \State $k \gets k - 2^i$ 75 | \EndIf 76 | \EndFor 77 | \State \Return $u$ 78 | \EndFunction 79 | \Function{LCA}{$u$, $v$} 80 | \If{$depth(u) < depth(v)$} 81 | \State \Call{swap}{$u$, $v$} 82 | \EndIf 83 | \State $u \gets \Call{LevelAncestor}{u, depth(v) - depth(u)}$ 84 | \If{$u = v$} 85 | \State \Return $u$ 86 | \EndIf 87 | \ForAll{i from $\lfloor \log_2 n \rfloor$ to 0} 88 | \If{$par(i, u) \neq par(i, v)$} 89 | \State $u \gets par(i, u)$ 90 | \State $v \gets par(i, v)$ 91 | \EndIf 92 | \EndFor 93 | \State \Return $par(0, u)$ 94 | \EndFunction 95 | \end{algorithmic} 96 | \end{algorithm} 97 | 98 | Level ancestor and LCA are the two basic tools to jump pointers. Applying these, we can quickly answer queries about paths, such as the length of the path between two vertices $u$ and $v$. If we augment the jump pointers by storing additional information, we can also compute maximum weight or distance queries on weighted trees. Another important observation that we can make is that we can compute jump pointers on the fly, adding edges and answering queries online. 99 | 100 | We now take some time to acknowledge a few limitations of jump pointers. Because each vertex of the tree is covered by $O(n)$ jump pointers on average (including the ones jumping over it), this structure cannot handle updates efficiently. Thus we usually apply jump pointers only if we know that the weights/values stored with the jump pointers will not change after being computed. For example, if we want to both answer maximum weight queries for paths and update edge weights, we should use heavy-light decomposition or link-cut trees to do so. 101 | 102 | Overall, however, jump pointers are still a very flexible technique. With some creativity, these ideas can be used to provide elegant and easy-to-code solutions for problems that would otherwise require more complex data structures. 103 | 104 | \section{Euler Tour Technique} 105 | 106 | The \textit{Euler tour technique} is a method of representing a tree (not necessarily binary) as a list. The idea is that a tree is uniquely determined by its DFS traversal order, going both down and up the tree. We'll keep track of the order in which we traverse the edges. Since each edge is traversed twice, each edge will appear in our list twice. We'll also name each edge by the child vertex in the parent-child pair. 107 | 108 | \begin{center} 109 | \begin{tikzpicture}[very thick,level 1/.style={sibling distance=50mm}, level 2/.style={sibling distance=50mm}, level 3/.style={sibling distance=40mm}, level distance=3cm, auto] 110 | 111 | \node[vertex] (a) {$A$} 112 | child { 113 | node[vertex] (b) {$B$} 114 | child { 115 | node[vertex] (e) {$E$} 116 | child { 117 | node[vertex] (h) {$H$} 118 | edge from parent node [swap] {$h$} 119 | } 120 | child { 121 | node[vertex] (i) {$I$} 122 | edge from parent node {$i$} 123 | } 124 | edge from parent node {$e$} 125 | } 126 | edge from parent node [swap] {$b$} 127 | } 128 | child { 129 | node[vertex] (c) {$C$} 130 | edge from parent node {$c$} 131 | } 132 | child { 133 | node[vertex] (d) {$D$} 134 | child { 135 | node[vertex] (f) {$F$} 136 | edge from parent node [swap] {$f$} 137 | } 138 | child { 139 | node[vertex] (g) {$G$} 140 | edge from parent node {$g$} 141 | } 142 | edge from parent node {$d$} 143 | }; 144 | 145 | \path[->] (a) edge [bend right=20] node [swap] {1} (b); 146 | \path[->] (b) edge [bend right] node [swap] {2} (e) ; 147 | \path[->] (e) edge [bend right] node [swap] {3} (h) ; 148 | \path[->] (h) edge [bend right] node [swap] {4} (e) ; 149 | \path[->] (e) edge [bend right] node [swap] {5} (i) ; 150 | \path[->] (i) edge [bend right] node [swap] {6} (e) ; 151 | \path[->] (e) edge [bend right] node [swap] {7} (b) ; 152 | \path[->] (b) edge [bend right=20] node [swap] {8} (a) ; 153 | \path[->] (a) edge [bend right] node [swap] {9} (c) ; 154 | \path[->] (c) edge [bend right] node [swap] {10} (a) ; 155 | \path[->] (a) edge [bend right=20] node [swap] {11} (d) ; 156 | \path[->] (d) edge [bend right] node [swap] {12} (f) ; 157 | \path[->] (f) edge [bend right] node [swap] {13} (d) ; 158 | \path[->] (d) edge [bend right] node [swap] {14} (g) ; 159 | \path[->] (g) edge [bend right] node [swap] {15} (d) ; 160 | \path[->] (d) edge [bend right=20] node [swap] {16} (a); 161 | 162 | \end{tikzpicture} 163 | \end{center} 164 | 165 | The edge traversal order is then described by the ordered list below. 166 | 167 | \begin{center} 168 | { 169 | \begin{tikzpicture}[ 170 | thick, 171 | myrect/.style={ 172 | draw, 173 | fill=myseagreen, 174 | rectangle split, 175 | rectangle split horizontal, 176 | rectangle split parts=#1, 177 | rectangle split part align=left, 178 | text width=4ex, 179 | text centered 180 | }, 181 | mycallout/.style={ 182 | shape=rectangle callout, 183 | rounded corners, 184 | fill=mysalmon, 185 | callout absolute pointer={#1}, 186 | callout pointer width=1cm 187 | } 188 | ] 189 | 190 | \node[myrect=16] 191 | (array1) 192 | { 193 | \strut $b_1$ 194 | \nodepart{two} \strut $e_1$ 195 | \nodepart{three} \strut $h_1$ 196 | \nodepart{four} \strut $h_2$ 197 | \nodepart{five} \strut $i_1$ 198 | \nodepart{six} \strut $i_2$ 199 | \nodepart{seven} \strut $e_2$ 200 | \nodepart{eight} \strut $b_2$ 201 | \nodepart{nine} \strut $c_1$ 202 | \nodepart{ten} \strut $c_2$ 203 | \nodepart{eleven} \strut $d_1$ 204 | \nodepart{twelve} \strut $f_1$ 205 | \nodepart{thirteen} \strut $f_2$ 206 | \nodepart{fourteen} \strut $g_1$ 207 | \nodepart{fifteen} \strut $g_2$ 208 | \nodepart{sixteen} \strut $d_2$ 209 | }; 210 | \foreach \Valor [count=\Valori from 1] in {one ,two ,three , four , five , six , seven , eight , nine , ten , eleven , twelve , thirteen , fourteen , fifteen , sixteen } 211 | \node[below] at (array1.\Valor south) {\Valori}; 212 | 213 | \end{tikzpicture} 214 | } 215 | \end{center} 216 | 217 | We see a pattern in this list -- the subtree of a node is contained between the two edges representing that node in the list, inclusive. For example, from the first $e$ to the second $e$, the entire subtree of $E$ is contained within the range $[2,7]$. 218 | 219 | \subsection{Euler Tour Tree} 220 | 221 | The \textit{Euler tour tree} uses this idea. Because we see the relationship between the list and the subtrees of the tree, we'll just have the list store the vertex names. To complete this pattern for the subtree of $A$, which is the entire tree, we'll simply add that letter to the beginning and end of the list. 222 | 223 | \begin{center} 224 | { 225 | \begin{tikzpicture}[ 226 | thick, 227 | myrect/.style={ 228 | draw, 229 | fill=myseagreen, 230 | rectangle split, 231 | rectangle split horizontal, 232 | rectangle split parts=#1, 233 | rectangle split part align=left, 234 | text width=3.5ex, 235 | text centered 236 | }, 237 | mycallout/.style={ 238 | shape=rectangle callout, 239 | rounded corners, 240 | fill=mysalmon, 241 | callout absolute pointer={#1}, 242 | callout pointer width=1cm 243 | } 244 | ] 245 | 246 | \node[myrect=18] 247 | (array1) 248 | { 249 | \strut $A_1$ 250 | \nodepart{two} \strut $B_1$ 251 | \nodepart{three} \strut $E_1$ 252 | \nodepart{four} \strut $H_1$ 253 | \nodepart{five} \strut $H_2$ 254 | \nodepart{six} \strut $I_1$ 255 | \nodepart{seven} \strut $I_2$ 256 | \nodepart{eight} \strut $E_2$ 257 | \nodepart{nine} \strut $B_2$ 258 | \nodepart{ten} \strut $C_1$ 259 | \nodepart{eleven} \strut $C_2$ 260 | \nodepart{twelve} \strut $D_1$ 261 | \nodepart{thirteen} \strut $F_1$ 262 | \nodepart{fourteen} \strut $F_2$ 263 | \nodepart{fifteen} \strut $G_1$ 264 | \nodepart{sixteen} \strut $G_2$ 265 | \nodepart{seventeen} \strut $D_2$ 266 | \nodepart{eighteen} \strut $A_2$ 267 | }; 268 | \foreach \Valor [count=\Valori from 1] in {one ,two ,three , four , five , six , seven , eight , nine , ten , eleven , twelve , thirteen , fourteen , fifteen , sixteen , seventeen , eighteen } 269 | \node[below] at (array1.\Valor south) {\Valori}; 270 | 271 | \end{tikzpicture} 272 | } 273 | \end{center} 274 | 275 | Now suppose we wanted to perform some operations on a forest, or collection of trees. The first operation is to \textit{find} the root of the tree containing a vertex $v$. The second is to \textit{link} the trees of two vertices $u$, $v$ together by making $v$ the parent of $u$. The third is to \textit{cut} the subtree of $v$ away from the rest of the tree by deleting the edge from $v$ to its parent. 276 | 277 | Note that, without the cut operation, this is simply union-find. However, because of the cut operation, we have to maintain the structure of the original forest, as otherwise we lose track of the actual parent of any given node. 278 | 279 | We see that all operations are not simply on a single vertex but on an entire subtree. However, if we look at what each of the operations does to the Euler tour lists representing our forest, find returns the first element in the list, link places one ordered list within another, and cut removes a large contiguous section of one ordered list. 280 | 281 | To cut $E$ from the tree, we need to split the list in two: 282 | 283 | \begin{center} 284 | { 285 | \begin{tikzpicture}[ 286 | thick, 287 | myrect/.style={ 288 | draw, 289 | fill=myseagreen, 290 | rectangle split, 291 | rectangle split horizontal, 292 | rectangle split parts=#1, 293 | rectangle split part align=left, 294 | text width=3ex, 295 | text centered 296 | }, 297 | mycallout/.style={ 298 | shape=rectangle callout, 299 | rounded corners, 300 | fill=mysalmon, 301 | callout absolute pointer={#1}, 302 | callout pointer width=1cm 303 | } 304 | ] 305 | 306 | \node[myrect=18] 307 | (array1) 308 | { 309 | \strut $A_1$ 310 | \nodepart{two} \strut $B_1$ 311 | \nodepart{three} \strut $E_1$ 312 | \nodepart{four} \strut $H_1$ 313 | \nodepart{five} \strut $H_2$ 314 | \nodepart{six} \strut $I_1$ 315 | \nodepart{seven} \strut $I_2$ 316 | \nodepart{eight} \strut $E_2$ 317 | \nodepart{nine} \strut $B_2$ 318 | \nodepart{ten} \strut $C_1$ 319 | \nodepart{eleven} \strut $C_2$ 320 | \nodepart{twelve} \strut $D_1$ 321 | \nodepart{thirteen} \strut $F_1$ 322 | \nodepart{fourteen} \strut $F_2$ 323 | \nodepart{fifteen} \strut $G_1$ 324 | \nodepart{sixteen} \strut $G_2$ 325 | \nodepart{seventeen} \strut $D_2$ 326 | \nodepart{eighteen} \strut $A_2$ 327 | }; 328 | \foreach \Valor [count=\Valori from 1] in {one ,two ,three , four , five , six , seven , eight , nine , ten , eleven , twelve , thirteen , fourteen , fifteen , sixteen , seventeen , eighteen } 329 | \node[below] at (array1.\Valor south) {\Valori}; 330 | 331 | \end{tikzpicture} 332 | } 333 | 334 | \begin{tikzpicture}[ 335 | thick, 336 | myrect/.style={ 337 | draw, 338 | fill=myseagreen, 339 | rectangle split, 340 | rectangle split horizontal, 341 | rectangle split parts=#1, 342 | rectangle split part align=left, 343 | text width=3ex, 344 | text centered 345 | }, 346 | mycallout/.style={ 347 | shape=rectangle callout, 348 | rounded corners, 349 | fill=mysalmon, 350 | callout absolute pointer={#1}, 351 | callout pointer width=1cm 352 | } 353 | ] 354 | \node[myrect=2] 355 | (array2) 356 | { 357 | \strut $A_1$ 358 | \nodepart{two} \strut $B_1$ 359 | }; 360 | \foreach \Valor [count=\Valori from 1] in {one ,two } 361 | \node[below] at (array2.\Valor south) {\Valori}; 362 | 363 | \node[myrect=6, fill=mysalmon] [right=of array2] 364 | (array3) 365 | { 366 | \strut $E_1$ 367 | \nodepart{two} \strut $H_1$ 368 | \nodepart{three} \strut $H_2$ 369 | \nodepart{four} \strut $I_1$ 370 | \nodepart{five} \strut $I_2$ 371 | \nodepart{six} \strut $E_2$ 372 | }; 373 | \foreach \Valor [count=\Valori from 3] in {one ,two ,three , four , five , six } 374 | \node[below] at (array3.\Valor south) {\Valori}; 375 | 376 | \node[myrect=10] [right=of array3] 377 | (array4) 378 | { 379 | \strut $B_2$ 380 | \nodepart{two} \strut $C_1$ 381 | \nodepart{three} \strut $C_2$ 382 | \nodepart{four} \strut $D_1$ 383 | \nodepart{five} \strut $F_1$ 384 | \nodepart{six} \strut $F_2$ 385 | \nodepart{seven} \strut $G_1$ 386 | \nodepart{eight} \strut $G_2$ 387 | \nodepart{nine} \strut $D_2$ 388 | \nodepart{ten} \strut $A_2$ 389 | }; 390 | \foreach \Valor [count=\Valori from 9] in {one ,two ,three , four , five , six , seven , eight , nine , ten } 391 | \node[below] at (array4.\Valor south) {\Valori}; 392 | \end{tikzpicture} 393 | 394 | 395 | \begin{tikzpicture}[ 396 | thick, 397 | myrect/.style={ 398 | draw, 399 | fill=myseagreen, 400 | rectangle split, 401 | rectangle split horizontal, 402 | rectangle split parts=#1, 403 | rectangle split part align=left, 404 | text width=3ex, 405 | text centered 406 | }, 407 | mycallout/.style={ 408 | shape=rectangle callout, 409 | rounded corners, 410 | fill=mysalmon, 411 | callout absolute pointer={#1}, 412 | callout pointer width=1cm 413 | } 414 | ] 415 | \node[myrect=12] 416 | (array5) 417 | { 418 | \strut $A_1$ 419 | \nodepart{two} \strut $B_1$ 420 | \nodepart{three} \strut $B_2$ 421 | \nodepart{four} \strut $C_1$ 422 | \nodepart{five} \strut $C_2$ 423 | \nodepart{six} \strut $D_1$ 424 | \nodepart{seven} \strut $F_1$ 425 | \nodepart{eight} \strut $F_2$ 426 | \nodepart{nine} \strut $G_1$ 427 | \nodepart{ten} \strut $G_2$ 428 | \nodepart{eleven} \strut $D_2$ 429 | \nodepart{twelve} \strut $A_2$ 430 | }; 431 | \foreach \Valor [count=\Valori from 1] in {one ,two ,three , four , five , six , seven , eight , nine , ten , eleven , twelve } 432 | \node[below] at (array5.\Valor south) {\Valori}; 433 | 434 | \node[myrect=6, fill=mysalmon] [right=of array5] 435 | (array6) 436 | { 437 | \strut $E_1$ 438 | \nodepart{two} \strut $H_1$ 439 | \nodepart{three} \strut $H_2$ 440 | \nodepart{four} \strut $I_1$ 441 | \nodepart{five} \strut $I_2$ 442 | \nodepart{six} \strut $E_2$ 443 | }; 444 | \foreach \Valor [count=\Valori from 1] in {one ,two ,three , four , five , six } 445 | \node[below] at (array6.\Valor south) {\Valori}; 446 | \end{tikzpicture} 447 | \end{center} 448 | 449 | Let's see what happens when we link $E$ to $D$. We need to split the first list immediately after $D_1$. 450 | 451 | \begin{center} 452 | \begin{tikzpicture}[ 453 | thick, 454 | myrect/.style={ 455 | draw, 456 | fill=myseagreen, 457 | rectangle split, 458 | rectangle split horizontal, 459 | rectangle split parts=#1, 460 | rectangle split part align=left, 461 | text width=3ex, 462 | text centered 463 | }, 464 | mycallout/.style={ 465 | shape=rectangle callout, 466 | rounded corners, 467 | fill=mysalmon, 468 | callout absolute pointer={#1}, 469 | callout pointer width=1cm 470 | } 471 | ] 472 | \node[myrect=12] 473 | (array5) 474 | { 475 | \strut $A_1$ 476 | \nodepart{two} \strut $B_1$ 477 | \nodepart{three} \strut $B_2$ 478 | \nodepart{four} \strut $C_1$ 479 | \nodepart{five} \strut $C_2$ 480 | \nodepart{six} \strut $D_1$ 481 | \nodepart{seven} \strut $F_1$ 482 | \nodepart{eight} \strut $F_2$ 483 | \nodepart{nine} \strut $G_1$ 484 | \nodepart{ten} \strut $G_2$ 485 | \nodepart{eleven} \strut $D_2$ 486 | \nodepart{twelve} \strut $A_2$ 487 | }; 488 | \foreach \Valor [count=\Valori from 1] in {one ,two ,three , four , five , six , seven , eight , nine , ten , eleven , twelve } 489 | \node[below] at (array5.\Valor south) {\Valori}; 490 | 491 | \node[myrect=6, fill=mysalmon] [right=of array5] 492 | (array6) 493 | { 494 | \strut $E_1$ 495 | \nodepart{two} \strut $H_1$ 496 | \nodepart{three} \strut $H_2$ 497 | \nodepart{four} \strut $I_1$ 498 | \nodepart{five} \strut $I_2$ 499 | \nodepart{six} \strut $E_2$ 500 | }; 501 | \foreach \Valor [count=\Valori from 1] in {one ,two ,three , four , five , six } 502 | \node[below] at (array6.\Valor south) {\Valori}; 503 | \end{tikzpicture} 504 | 505 | \begin{tikzpicture}[ 506 | thick, 507 | myrect/.style={ 508 | draw, 509 | fill=myseagreen, 510 | rectangle split, 511 | rectangle split horizontal, 512 | rectangle split parts=#1, 513 | rectangle split part align=left, 514 | text width=3ex, 515 | text centered 516 | }, 517 | mycallout/.style={ 518 | shape=rectangle callout, 519 | rounded corners, 520 | fill=mysalmon, 521 | callout absolute pointer={#1}, 522 | callout pointer width=1cm 523 | } 524 | ] 525 | \node[myrect=6] 526 | (array2) 527 | { 528 | \strut $A_1$ 529 | \nodepart{two} \strut $B_1$ 530 | \nodepart{three} \strut $B_2$ 531 | \nodepart{four} \strut $C_1$ 532 | \nodepart{five} \strut $C_2$ 533 | \nodepart{six} \strut $D_1$ 534 | }; 535 | \foreach \Valor [count=\Valori from 1] in {one ,two ,three , four , five , six } 536 | \node[below] at (array2.\Valor south) {\Valori}; 537 | 538 | \node[myrect=6] [right=of array2] 539 | (array4) 540 | { 541 | \strut $F_1$ 542 | \nodepart{two} \strut $F_2$ 543 | \nodepart{three} \strut $G_1$ 544 | \nodepart{four} \strut $G_2$ 545 | \nodepart{five} \strut $D_2$ 546 | \nodepart{six} \strut $A_2$ 547 | }; 548 | 549 | \node[myrect=6, fill=mysalmon] [right=of array4] 550 | (array3) 551 | { 552 | \strut $E_1$ 553 | \nodepart{two} \strut $H_1$ 554 | \nodepart{three} \strut $H_2$ 555 | \nodepart{four} \strut $I_1$ 556 | \nodepart{five} \strut $I_2$ 557 | \nodepart{six} \strut $E_2$ 558 | }; 559 | \foreach \Valor [count=\Valori from 1] in {one ,two ,three , four , five , six } 560 | \node[below] at (array3.\Valor south) {\Valori}; 561 | \foreach \Valor [count=\Valori from 7] in {one ,two ,three , four , five , six } 562 | \node[below] at (array4.\Valor south) {\Valori}; 563 | \end{tikzpicture} 564 | 565 | { 566 | \begin{tikzpicture}[ 567 | thick, 568 | myrect/.style={ 569 | draw, 570 | fill=myseagreen, 571 | rectangle split, 572 | rectangle split horizontal, 573 | rectangle split parts=#1, 574 | rectangle split part align=left, 575 | text width=3ex, 576 | text centered 577 | }, 578 | mycallout/.style={ 579 | shape=rectangle callout, 580 | rounded corners, 581 | fill=mysalmon, 582 | callout absolute pointer={#1}, 583 | callout pointer width=1cm 584 | } 585 | ] 586 | 587 | \node[myrect=18] 588 | (array1) 589 | { 590 | \strut $A_1$ 591 | \nodepart{two} \strut $B_1$ 592 | \nodepart{three} \strut $B_2$ 593 | \nodepart{four} \strut $C_1$ 594 | \nodepart{five} \strut $C_2$ 595 | \nodepart{six} \strut $D_1$ 596 | \nodepart{seven} \strut $E_1$ 597 | \nodepart{eight} \strut $H_1$ 598 | \nodepart{nine} \strut $H_2$ 599 | \nodepart{ten} \strut $I_1$ 600 | \nodepart{eleven} \strut $I_2$ 601 | \nodepart{twelve} \strut $E_2$ 602 | \nodepart{thirteen} \strut $F_1$ 603 | \nodepart{fourteen} \strut $F_2$ 604 | \nodepart{fifteen} \strut $G_1$ 605 | \nodepart{sixteen} \strut $G_2$ 606 | \nodepart{seventeen} \strut $D_2$ 607 | \nodepart{eighteen} \strut $A_2$ 608 | }; 609 | \foreach \Valor [count=\Valori from 1] in {one ,two ,three , four , five , six , seven , eight , nine , ten , eleven , twelve , thirteen , fourteen , fifteen , sixteen , seventeen , eighteen } 610 | \node[below] at (array1.\Valor south) {\Valori}; 611 | 612 | \end{tikzpicture} 613 | } 614 | 615 | \end{center} 616 | 617 | We have a data structure that maintains a set of ordered elements that can split and merge quickly. Splay trees can maintain an ordered list, and the split and join operations on splay trees can easily implement link and cut. Each tree of our forest is then represented by a splay tree maintaining that tree's Euler tour list, and we can support each of the three necessary operations in $O(\log{n})$ amortized time. 618 | 619 | 620 | \section{Heavy-Light Decomposition} 621 | 622 | Brute-force algorithms on trees are often fast when the tree is nearly complete, or has small depth. Unfortunately, brute force is far too slow when the tree becomes more and more unbalanced and approximates a linked list. 623 | 624 | Certain problems can be solved very nicely in a list with segment trees and similar data structures. Heavy-light decomposition provides us with a way to exploit the fact that long ``chains," which are present in unbalanced trees, might be slow in brute force but can be sped up with other data structures. 625 | 626 | \textit{Heavy-light decomposition} provides us a way to dynamically find the lowest common ancestor as well as a way to prove time complexities about tree data structures, like link-cut trees. 627 | 628 | Heavy-light decomposition is a coloring of all the edges in a binary tree either heavy or light. For each vertex $v$, let $s(v)$ denote the number of nodes in the subtree with $v$ as its head. Then, if $u,w$ are the children of $v$, $s(v)=s(u)+s(w)+1$. 629 | 630 | We see that the smaller child $u$ or $w$ must have less than half the subtree size as $v$, and so that child exhibits qualities similar to those of a completely balanced tree. We color the edge connecting $v$ to its lesser child \textit{light}. We color the other edge \textit{heavy}. 631 | 632 | We see that from any node, the number of light edges needed to reach the head of the tree is at most $\log{n}$, as each light edge doubles the subtree size. Then, the number of ``heavy chains" along the upwards path from any node is also of the order $\log{n}$. 633 | 634 | \begin{center} 635 | \begin{tikzpicture}[very thick,level/.style={sibling distance=70mm/#1}] 636 | \node [vertex] (a) {17} 637 | child { 638 | node [vertex] (b) {12} 639 | child { 640 | node [vertex] (c) {6} 641 | child { 642 | node [vertex] (d) {3} 643 | child { 644 | node [vertex] (e) {1} 645 | } 646 | child { 647 | node [vertex] (f) {1} 648 | } 649 | } 650 | child {node [vertex] (g) {2} 651 | child { 652 | node[vertex] (h) {1} 653 | } 654 | } 655 | child[missing] 656 | } 657 | child { 658 | node [vertex] (i) {5} 659 | child {node [vertex] (j) {1}} 660 | child {node [vertex] (k) {1}} 661 | child { 662 | node [vertex] (l) {2} 663 | child {node[vertex] (m) {1}} 664 | } 665 | } 666 | } 667 | child { 668 | node [vertex] (n) {4} 669 | child { 670 | node[vertex] (o) {3} 671 | child { 672 | node[vertex] (p) {1} 673 | } 674 | child { 675 | node[vertex] (q) {1} 676 | } 677 | } 678 | }; 679 | \draw[line width=6pt] (a) -- (b); 680 | \draw[line width=6pt] (b) -- (c); 681 | \draw[line width=6pt] (c) -- (d); 682 | \draw[line width=6pt] (d) -- (e); 683 | \draw[line width=6pt] (g) -- (h); 684 | \draw[line width=6pt] (i) -- (l); 685 | \draw[line width=6pt] (l) -- (m); 686 | \draw[line width=6pt] (n) -- (o); 687 | \draw[line width=6pt] (o) -- (p); 688 | \end{tikzpicture} 689 | \end{center} 690 | 691 | \section{Link-Cut Tree} 692 | 693 | -------------------------------------------------------------------------------- /chapters/big_ideas.tex: -------------------------------------------------------------------------------- 1 | \chapter{Big Ideas} 2 | 3 | In this chapter, we'll discuss some general problem solving ideas: brute force, depth-first search, and the greedy algorithm. We can think of these as the building blocks to more complex methods---each provides a very general approach to simplifying problems. In programming contests, they also appear frequently by themselves as the core ideas to solutions. Since the concepts we cover are independent of language, we will no longer present algorithms in concrete Java or C++ code, but rather in more abstract pseudocode. 4 | 5 | \section{Brute Force} 6 | 7 | Sometimes, the best way to approach a problem is to try everything. This idea of exhaustively searching all possibilities is called \emph{brute force}. For example, if we want to unlock a friend's iPhone, we could try all of the $10^4$ possible passcodes. As the name and this example suggest, brute force is often crude and inefficient. Usually we want to make some clever observations to make the problem more tractable. However, if the input size is small (check the number of operations against $10^8$) or if we want to squeeze a few points out of a problem by solving only the small cases, brute force could be the way to go. And if you're stuck on a problem, thinking about a brute force is not a bad way to start. Simpler, slower algorithms can often inspire faster ones. Through the following problems, we'll show you how to brutally apply the idea of brute force. 8 | 9 | \subsection{Square Root} 10 | 11 | \begin{typewriter} 12 | Given an integer $n$, $1 \le n \le 10^{12}$, find the greatest integer less than or equal to $\sqrt{n}$ without using any library functions. (This means you can't call functions like Math.sqrt or Math.log.) 13 | \end{typewriter} 14 | 15 | At first, it's not obvious how we can compute square roots. However, we can always go simple. Set $i = 1$, and while $(i+1)^2\le n$, increment $i$. That is, we increment $i$ until increasing it further will cause $i$ to exceed $\sqrt n$. Since our answer $i$ is at most $\sqrt n \le 10^6$, our program runs in time. This is about the silliest approach we can use to calculate square roots, but hey, it works! 16 | 17 | When implementing this algorithm, be careful about the size of $n$. The 32-bit \texttt{int} type in Java and C++ only holds values up to $2^{31} - 1 = 2,147,483,647$, which is exceeded by the maximum possible value of $n$. Thus we need to use a 64-bit integer type for our calculations: \texttt{long} in Java and \texttt{long long} in C++. 18 | 19 | \subsection{Combination Lock} 20 | 21 | \begin{typewriter} 22 | Farmer John purchases a combination lock to stop his cows from escaping their pasture and causing mischief! His lock has three circular dials, each with tick marks numbered $1$ through $N$ ($1\le N\le 100$), with $1$ and $N$ adjacent. There are two combinations that open the lock: one combination set by Farmer John and one ``master'' combination set by the locksmith. The lock has a small tolerance for error, however, so it will open if the numbers on each of the dials are at most two positions away from that of a valid combination. Given Farmer John's combination and the master combination, determine the number of distinct settings for the dials that will open the lock. 23 | 24 | (For example, if Farmer John's combination is $(1,2,3)$ and the master combination is $(4,5,6)$, the lock will open if its dials are set to $(1,N,5)$ (since this is close to Farmer John's combination) or to $(2,4,8)$ (since this is close to the master combination). Note that $(1,5,6)$ would not open the lock, since it is not close enough to any single combination. Furthermore, order matters, so $(1,2,3)$ is distinct from $(3,2,1)$.) [Adapted from \href{http://usaco.org/index.php?page=viewproblem2&cpid=340}{USACO 2013, Combination Lock}.] 25 | \end{typewriter} 26 | 27 | Again, the simplest idea works. We can iterate over all possible settings of the lock, and for each setting, check if it matches either Farmer John's combination or the master combination. To do this, we can use three nested \texttt{for} loops. The first loop goes through the values for the first dial, the second loop through the values for the second dial, and the third loop through the values for the third dial. Since there are three dials, the lock has at most $N^3 \le 10^6$ possible settings. We can check if each dial matches in $O(1)$ time, hence our algorithm runs in less than a second. 28 | 29 | In terms of implementation, \texttt{Combination Lock} is a great example of how a problem can decompose into two easier components that we can think about separately. The first component is to use nested loops to iterate through the possible settings, which we've described above. (Nested \texttt{for} loops like this show up often!) The second component is to check if a given setting is close to either of the given combinations. If we implement a function \texttt{is\_valid(a, b, c)} to do this, then the code becomes quite clean. 30 | 31 | \subsection{Ski Course Design} 32 | 33 | \begin{typewriter} 34 | Farmer John has $N$ hills on his farm ($1\le N\le 1000$), each with an integer elevation in the range $0$ to $100$. In the winter, since there is abundant snow on these hills, he routinely operates a ski training camp. In order to evade taxes, Farmer John wants to add or subtract height from each of his hills so that the difference between the heights of his shortest and tallest hills is at most $17$ before this year's camp. 35 | 36 | Suppose it costs $x^2$ dollars for Farmer John to change the height of a hill by $x$ units. Given the current heights of his hills, what is the minimum amount that Farmer John will need to pay? (Farmer John is only willing to change the height of each hill by an integer amount.) [Adapted from \href{http://usaco.org/index.php?page=viewproblem2&cpid=376}{USACO 2014, Ski Course Design}.] 37 | \end{typewriter} 38 | 39 | For \texttt{Ski Course Design}, we need to be a bit clever about how to implement our brute force. There are infinitely many ways we could change the heights of each hill, so it seems intractable to iterate over the possible heights for each hill separately. Instead, we look at the final range of the ski slope heights, which has length at most $17$. This final range has to fall within the interval $[0,100]$, hence there are less than $100$ possibilities. (The possible ranges are $[0, 17]$, $[1, 18]$, $\cdots$, $[83, 100]$.) Once we fix a range, we can calculate in $O(N)$ the minimum cost to make the height of each hill fall within that range. Thus if we let $M$ be the number of possible ranges ($M < 100$), we have an $O(MN)$ algorithm. 40 | 41 | This problem shows that even when we brute force, we still have to think. Some approaches are better than others. In particular, we don't want to deal with cases that are irrelevant---for example, when the heights are not within a range of width $17$ or when Farmer John has not used the cheapest set of changes. We also don't want our possibilities to explode out of control, which would have happened had we adjusted the height of each hill separately with nested \texttt{for} loops or recursion. By iterating over a more restricted set of possibilities, we have created a brute force solution that runs significantly faster. 42 | 43 | \subsection{Contest Practice} 44 | 45 | \href{http://codeforces.com/group/5tN48zOVvQ/contest/204642}{Here} is a collection of problems solvable through brute force. Try working through them on your own and applying the ideas you've seen so far. (May the brute force be with you.) 46 | 47 | \section{Depth-First Search (DFS)} 48 | 49 | Depth-first search is a recursive search technique that provides us with another way to brute force. Instead of using nested loops to iterate through all possibilities, we can generate the possibilities using recursion, checking all possible choices in each level. Here's an example: All of your friends, upon realizing that it only takes four nested \texttt{for} loops to iterate through all possible 4-digit iPhone passcodes, have decided to make their passcodes $n$ digits long. Since you don't know $n$, nested \texttt{for} loops will no longer do the trick. Instead, we can use a DFS to recursively generate all $n$-digit passcodes. 50 | 51 | Depth-first search works as follows: We check all passcodes starting with ``0'', then all passcodes starting with ``1'', then all passcodes starting with ``2'', and so on. To check all passcodes starting with ``0'', we check all passcodes starting with ``00'', then all passcodes starting with ``01'', then all passcodes starting with ``02'', and so on. To check all passcodes starting with ``00'', we have to check all passcodes starting with ``000'', then all passcodes starting with ``001'' and so on... (Think about why DFS is \emph{depth-first}.) 52 | 53 | In this way, we recursively generate all possible passcodes by extending the prefix character by character. We keep recursing until we have to check a passcode starting with a string of length $n$, in which case that string is the passcode itself. Thus the first passcode we check is ``00$\cdots$0'' and the last passcode we check is ``99$\cdots$9''. We implement this algorithm by writing a function that generates all passcodes given a prefix. Below is some pseudocode describing the algorithm. Make sure you understand how the function calls itself! 54 | 55 | \noindent \begin{minipage}{\textwidth} 56 | \begin{algorithmic}[1] 57 | \Function{generatePasscodes}{$depth$, $prefix$} 58 | \If{$depth = n$} 59 | \Comment {If we've reached maximum depth, then print and return.} 60 | \State \Call{print}{$prefix$} 61 | \State \Return 62 | \EndIf 63 | \For{$c$ from `0' to `9'} 64 | \Comment{Iterates over all possible next digits.} 65 | \State \Call{generatePasscodes}{$depth + 1$, $prefix + c$} 66 | \Comment{Recurses with a longer prefix.} 67 | \EndFor 68 | \EndFunction 69 | \end{algorithmic} 70 | \end{minipage} 71 | 72 | \subsection{Permutations} 73 | 74 | \begin{typewriter} 75 | Given $n$ ($n \le 8$), print all permutations of the sequence $\{1, 2, \cdots, n\}$ in lexicographic (alphabetical) order. (For $n=3$, this would be $(1, 2, 3)$, $(1, 3, 2)$, $(2, 1, 3)$, $(2, 3, 1)$, $(3, 1, 2)$, and $(3, 2, 1)$.) 76 | \end{typewriter} 77 | 78 | Like the passcode problem, we use DFS instead of nested \texttt{for} loops, since we don't know $n$. However, we have to be careful with implementation---we can use each number only once. Along with our current prefix, we have to keep track of the set of numbers that we've already used. This is best done with a Boolean ``used'' array outside of the recursive function. Here's the pseudocode: 79 | 80 | \noindent \begin{minipage}{\textwidth} 81 | \begin{algorithmic}[1] 82 | \State $used \gets \{false, false, \cdots, false\}$ 83 | \Comment{Initialize $used$ as an array of $false$ values.} 84 | \Function{generatePermutations}{$depth$, $prefix$} 85 | \If{$depth = n$} 86 | \State \Call{print}{$prefix$} 87 | \State \Return 88 | \EndIf 89 | \For{$i=1$ to $n$} 90 | \If{not $used[i]$} 91 | \State $used[i] \gets true$ 92 | \State \Call{generatePermutations}{$depth + 1$, $prefix + i$} 93 | \State $used[i] \gets false$ 94 | \Comment We have to reset the $used[i]$ variable once we're done. 95 | \EndIf 96 | \EndFor 97 | \EndFunction 98 | \end{algorithmic} 99 | \end{minipage} 100 | 101 | To understand the order in which we visit the permutations, we can visualize this algorithm as traversing a tree-like structure. An animation of this algorithm for $n = 5$ is \href{http://dabbler0.github.io/ecc-animations/dfs.html}{here}. 102 | 103 | \subsection{Basketball} 104 | 105 | \begin{typewriter} 106 | Two teams are competing in a game of basketball: the Exonians and the Smurfs. There are $n$ players on the Exonian team and $m$ players on the Smurf team, with $n + m \le 17$. Each player has an integer skill level $s$ between $1$ and $10^8$. Define the strength of a set of players as the sum of their individual skill levels. In order to ensure a fair game, the Exonians and Smurfs plan on choosing two equally strong starting lineups. In how many ways can the two teams choose their lineups? (Two lineups are considered different if there exists a player who starts in one game, but not in the other.) 107 | \end{typewriter} 108 | 109 | We use a DFS to recursively generate all possible starting lineups. Each starting lineup can be represented by a sequence of $n + m$ 0's and 1's, where a player starts if and only if he/she is assigned a 1. We do this the same way we generate all passcodes of length $n + m$. Once we have a starting lineup, it is straightforward to check for fairness. (Is it also possible to keep track of the strength of each team as we DFS? Hint: Keep an extra variable similar to ``used'' in \texttt{Permutations}.) 110 | 111 | \subsection{Problem Break} 112 | 113 | Before moving on, try to implement the DFS problems described above. You can test your code on the problem set \href{http://codeforces.com/group/5tN48zOVvQ/contest/205012}{here}. Try to do some complexity analysis too. How does the runtime of these algorithms grow with respect to $n$? 114 | 115 | \subsection{Generalizing DFS} 116 | 117 | Thus far, all of the DFS solutions that we've seen have involved sequences. However, we can also use DFS in a much more general setting. In the same spirit as brute force, if we want to enumerate or construct something and we have a number of options at each step, we can recurse on each of the possible options and thereby check all possibilities. The examples below show the flexibility of depth-first search---a huge class of problems can be solved in a brute force manner like this. 118 | 119 | \subsection{Dungeon} 120 | 121 | \begin{typewriter} 122 | Bessie is trying to escape from the dungeon of the meat packing plant! The dungeon is represented by an $n$-by-$n$ grid ($2 \le n \le 6$) where each of the grid cells is trapped and can only be stepped on once. Some cells of the grid also contain obstacles that block her way. Bessie is currently located in the upper-left corner and wants to make her way to the exit in the lower-right corner. How many paths can Bessie take to escape, assuming that she avoids all obstacles and steps on no cell twice? 123 | \end{typewriter} 124 | 125 | We write a function \texttt{DFS(x, y)} thats runs a DFS from cell $(x, y)$ and counts the number of paths to the lower-right corner given obstacles and previously visited squares. Upon arriving at $(x, y)$, we mark that cell as visited. If $(x, y)$ is the destination, we increment our answer by one. Otherwise, we try recursing in each direction---up, down, left, and right. Before recursing, we check that we don't go out of bounds and that we don't step on an obstacle or previously visited square. Once we're done counting paths from $(x, y)$, we have to remember to mark this cell as unvisited again. 126 | 127 | In terms of implementation, it is easiest to store the obstacles and visited cells in Boolean arrays outside of the recursive function. We can also define a function \texttt{ok(x, y)} to check if a cell $(x, y)$ is safe to step on---if $x$ and $y$ are in bounds, $(x, y)$ is not an obstacle, and $(x, y)$ is unvisited. Finally, to avoid doing four cases, one for each direction, we can define two arrays \texttt{dx} and \texttt{dy} which contain \texttt{[1, 0, -1, 0]} and \texttt{[0, 1, 0, -1]}, respectively. Then \texttt{dx[i]} and \texttt{dy[i]} for $0 \le i < 4$ represent the changes in $x$- and $y$-coordinates for each of Bessie's possible moves. 128 | \subsection{$n$ Queens Puzzle} 129 | 130 | \begin{typewriter} 131 | Given $n$ ($4 \le n \le 8$), find an arrangement of $n$ queens on an $n$-by-$n$ chessboard so that no two queens attack each other. 132 | \end{typewriter} 133 | 134 | First, observe that each row must contain exactly one queen. Thus we can assume that the $i$-th queen is placed in the $i$-th row. Like before, we try putting each queen on each possible square in its row and recurse, keeping track of used columns and diagonals. When we add the $i$-th queen, we mark its column and diagonals (both of them) as attacked. As usual, once we're done checking all possibilities for the $i$-th queen, we unmark its column and diagonals. We terminate when we find an arrangement using all $n$ queens. 135 | 136 | \section{Greedy Algorithms} 137 | 138 | A greedy algorithm is an algorithm that makes the (locally) most desirable choice at each step. Instead of checking all possibilities as we do in a depth-first search, we choose the option that ``seems'' best at the time. This usually results in a fast and easy to implement algorithm. While this may not be a good way to live life---doing homework would always be relegated in favor of solving programming contest problems---there exist many computer science problems where being greedy produces optimal or almost optimal results. Keep in mind, however, that greedy algorithms usually don't work. Before coding any greedy solution, you should be able to convince yourself with a proof of correctness. Let's go through a few examples to get a better sense for what greedy algorithms are like. 139 | 140 | \subsection{Bessie the Polyglot} 141 | 142 | \begin{typewriter} 143 | Bessie would like to learn how to code! There are $n$ ($1\le n\le 100$) programming languages suitable for cows and the $i$-th of them takes $a_i$ ($1\le a_i\le 100$) days to learn. Help Bessie determine the maximum number of languages she could know after $k$ ($1\le k\le 10^4$) days. [Adapted from \href{http://codeforces.com/problemset/problem/507/A}{Codeforces 507A}.] 144 | \end{typewriter} 145 | 146 | It's pretty easy to see that we want to learn programming languages in order of increasing $a_i$, starting from the smallest. Thus Bessie can obtain an optimal solution with a greedy approach. She first learns the language that requires the fewest days to learn, then the language that takes the next fewest days, and so on, until she can't learn another language without exceeding $k$ days. To implement this, we sort $a_i$ and find the longest prefix whose sum is at most $k$. Due to sorting, the complexity is $O(n\log n)$. 147 | 148 | With greedy algorithms, we usually want to have an ordering or heuristic by which we decide what is the best choice at a given instance. In this case, our ordering was the most straightforward possible, just choosing languages in order of increasing learning time. However, figuring out how we want to greedily make decisions is not always obvious. Oftentimes, coming up with the correct heuristic is what makes greedy algorithms hard to find. 149 | 150 | \subsection{More Cowbell} 151 | 152 | \begin{typewriter} 153 | Kevin Sun wants to move his precious collection of $n$ ($1\le n\le 10^5$) cowbells from Naperthrill to Exeter, where there is actually grass instead of corn. Before moving, he must pack his cowbells into $k$ boxes of a fixed size. In order to keep his collection safe during transportation, he will not place more than two cowbells into a single box. Since Kevin wishes to minimize expenses, he is curious about the smallest size box he can use to pack his entire collection. 154 | 155 | Kevin is a meticulous cowbell collector and knows that the size of his $i$-th ($1\le i\le n$) cowbell is an integer $s_i$. In fact, he keeps his cowbells sorted by size, so $s_{i - 1}\le s_i$ for any $i > 1$. Also an expert packer, Kevin can fit one or two cowbells into a box of size $s$ if and only if the sum of their sizes does not exceed $s$. Given this information, help Kevin determine the smallest $s$ for which it is possible to put all of his cowbells into $k$ ($n\le 2k\le 10^5$) boxes of size $s$. [Adapted from \href{http://codeforces.com/problemset/problem/604/B}{Codeforces 604B}.] 156 | \end{typewriter} 157 | 158 | Intuitively, we want to use as many boxes as we can and put the largest cowbells by themselves. Then, we want to pair the leftover cowbells so that the largest sum of a pair is minimized. This leads to the following greedy algorithm: 159 | 160 | First, if $k \ge n$, then each cowbell can go into its own box, so our answer is $\max(s_1, s_2, \cdots, s_n)$. Otherwise, we can have at most $2k-n$ boxes that contain one cowbell. Thus we put the $2k-n$ largest cowbells into their own boxes. For the remaining $n-(2k-n) = 2(n-k)$ cowbells, we pair the $i$th largest cowbell with the $(2(n-k) - i + 1)$-th largest. In other words, we match the smallest remaining cowbell with the largest, the second smallest with the second largest, and so on. Given these pairings, we can loop through them to find the largest box we'll need. The complexity of this algorithm is $O(n)$ since we need to iterate through all of the cowbells. 161 | 162 | To prove that this greedy algorithm works, we first prove the case $n = 2k$, where each box must contain exactly two cowbells. Consider any optimal pairing of cowbells. If $s_{2k}$ is not paired with $s_1$, then we can perform a swap so that they are paired without increasing the size of the largest box: Suppose we initially have the pairs $(s_1, s_i)$ and $(s_{2k}, s_j)$. If we rearrange them so that we have the pairs $(s_1, s_{2k})$ and $(s_i, s_j)$, then $s_1 + s_{2k}, s_i + s_j \le s_{2k} + s_j$. After we've paired the largest cowbell with the smallest, we can apply the same logic to the second largest, third largest, and so on until we're done. Therefore, our construction is optimal if $n = 2k$. For $n < 2k$, we can imagine that we have $2k - n$ cowbells of size $0$ and use the same argument. 163 | 164 | This method of proving correctness for a greedy algorithm is rather common. We want to show that our greedily constructed solution is as good as an arbitrarily chosen optimal solution, so we compare where the optimal solution and our greedy solution differ. Once we find a difference, we try to transform one to the other without changing the value we're trying to optimize. In this case, since transforming the optimal solution to our greedily constructed solution doesn't make it worse, our solution must be optimal as well. 165 | 166 | \subsection{Farmer John and Boxes} 167 | 168 | \begin{typewriter} 169 | Farmer John has $n$ ($1 \le n \le 100$) boxes in his barn. His boxes all have the same size and weight but have varying strengths---the $i$-th box is able to support at most $x_i$ other boxes on top of it. If Farmer John wants to stack his boxes so that each box is directly on top of at most one other box, what is the minimum number of stacks that he'll need? [Adapted from \href{http://codeforces.com/problemset/problem/388/A}{Codeforces 388A}.] 170 | \end{typewriter} 171 | 172 | We first observe that if a stronger box is on top of a weaker box, then we can swap the two boxes and still have a valid stacking. Therefore, an optimal solution exists where no box is stronger than any box below it. This allows us to sort the boxes in increasing order of strength and construct an optimal solution by inserting stronger boxes below piles of weaker boxes. 173 | 174 | Initially, we start with a single empty stack. We then add boxes one-by-one as follows: If we can, we insert our new box below some stack that it can successfully support. Otherwise, if no such stack exists, we create a new stack containing only the new box. It's possible to check all existing stacks in $O(n)$; therefore this algorithm runs in $O(n^2)$. (In fact, binary searching allows us to do the checking step in $O(\log n)$.) To prove correctness, we can again compare our construction to an arbitrary optimal construction. Finishing the argument is left as an exercise for the reader. 175 | 176 | The subtle part about this problem is the order in which we process the boxes. Processing boxes from weakest to strongest offers an elegant greedy solution, while processing from strongest to weakest offers no simple approach. (At least I haven't found one yet.) Again, we see that ordering is important. Stepping back a bit, this problem also isn't one that immediately suggests a greedy approach. Since greedy solutions can often be unexpected, it's always worthwhile to take a moment to consider various orderings and ``na\"ive'' approaches to see if any work. Telltale hints of a greedy approach are observations about order and monotonicity---for example ``we can always have stronger boxes below weaker ones.'' 177 | 178 | \subsection{Snack Time} 179 | 180 | Here's the \href{http://codeforces.com}{link} to the greedy problem set. Feast away! 181 | 182 | \chapter*{Interlude B} 183 | \addcontentsline{toc}{chapter}{Interlude B} 184 | % \renewcommand\thesection{\Alph{chapter}.\arabic{section}} 185 | % \setcounter{section}{0} 186 | 187 | \section{Prefix Sums} 188 | 189 | Suppose we were given an array of numbers of length $N$ and were asked to compute a series of $Q$ queries each asking for the sum of the numbers in the contiguous 190 | subsequence of numbers indexed from $i$ to $j$. 191 | 192 | \begin{center} 193 | { 194 | \begin{tikzpicture}[ 195 | thick, 196 | myrect/.style={ 197 | draw, 198 | fill=myseagreen, 199 | rectangle split, 200 | rectangle split horizontal, 201 | rectangle split parts=#1, 202 | rectangle split part align=left, 203 | text width=4ex, 204 | text centered 205 | } 206 | ] 207 | 208 | \node[myrect=16] 209 | (array) 210 | { 211 | \strut 2 212 | \nodepart{two} \strut 4 213 | \nodepart{three} \strut 7 214 | \nodepart{four} \strut $-5$ 215 | \nodepart{five} \strut 3 216 | \nodepart{six} \strut 6 217 | \nodepart{seven} \strut $-3$ 218 | \nodepart{eight} \strut 1 219 | \nodepart{nine} \strut $-2$ 220 | \nodepart{ten} \strut $-4$ 221 | \nodepart{eleven} \strut $-6$ 222 | \nodepart{twelve} \strut 2 223 | \nodepart{thirteen} \strut 8 224 | \nodepart{fourteen} \strut 6 225 | \nodepart{fifteen} \strut 0 226 | \nodepart{sixteen} \strut $-7$ 227 | }; 228 | \foreach \Valor [count=\Valori from 1] in {one ,two ,three , four , five , six , seven , eight , nine , ten , eleven , twelve , thirteen , fourteen , fifteen , sixteen } 229 | \node[below] at (array.\Valor south) {\Valori}; 230 | 231 | \end{tikzpicture} 232 | } 233 | \end{center} 234 | 235 | For example, the query on $[4,14]$ would sum 11 numbers and return the answer 6. 236 | The na\"{i}ve solution would simply resum the numbers from position $i$ to position $j$ each time: 237 | 238 | \begin{mylstlisting} 239 | int arr[MAXN]; 240 | int query(int i, int j) { 241 | int sum = 0; 242 | for(int k = i; k <= j; ++k) 243 | sum += arr[k]; 244 | return sum; 245 | } 246 | \end{mylstlisting} 247 | 248 | In the worst case, this for loop is $O(N)$, since each query could loop over the majority of the size $N$ array. 249 | This solution is fine if we only have a constant number of queries. But what if $Q$ is large? Notice that in case our na\"{i}ve solution is doing a lot of repeated work. 250 | If we were to query $[3,12]$, for example, we would sum $[4,12]$ all over again, three fourths of the whole array! 251 | 252 | How can we eliminate this excess work? Our intuition from this specific example 253 | would tell us to take our answer for $[4,14]$, subtract out anything that is not in the overlap ($[13,14]$), 254 | and add any part of the new query that we haven't yet included ($[3,3]$). In this way we avoid summing over $[4,12]$ twice. But we quickly see that this solution is 255 | unnecessarily complicated and does not always speed up our solution: the ranges $[1,8]$ and $[9,16]$ are both half the array size but have no overlap whatsoever, so each 256 | query could still be $O(N)$. 257 | 258 | We find our answer using something called \textbf{prefix sums}. Prefix sums are an example of precomputation: performing some computation on our data before we read in any 259 | queries to make the subsequent series of queries faster. Another example of precomputation you may have encountered before is sorting prior to a binary search, where we might 260 | be queried how many numbers in a list are greater than $x$ for some number $x$. Na\"{i}vely checking each element in the list would result in a linear $O(N)$ time solution 261 | per query, while sorting beforehand costs $O(N \log N)$ but results in each query costing $O(\log N)$. 262 | 263 | In this case, we notice that if we knew the answers to the queries $[1,j]$ and $[1,i-1]$, we could simply subtract the two to get the answer to the query for $[i,j]$. 264 | Consider the following array, where the number stored in index $i$ is the answer to the query $[1,i]$. 265 | 266 | \begin{center} 267 | \begin{tikzpicture}[ 268 | thick, 269 | myrect/.style={ 270 | draw, 271 | fill=myseagreen, 272 | rectangle split, 273 | rectangle split horizontal, 274 | rectangle split parts=#1, 275 | rectangle split part align=left, 276 | text width=4ex, 277 | text centered 278 | } 279 | ] 280 | 281 | \node[myrect=17] 282 | (array) 283 | { 284 | \strut 0 285 | \nodepart{two} \strut 2 286 | \nodepart{three} \strut 6 287 | \nodepart{four} \strut 13 288 | \nodepart{five} \strut 8 289 | \nodepart{six} \strut 11 290 | \nodepart{seven} \strut 17 291 | \nodepart{eight} \strut 14 292 | \nodepart{nine} \strut 15 293 | \nodepart{ten} \strut 13 294 | \nodepart{eleven} \strut 9 295 | \nodepart{twelve} \strut 3 296 | \nodepart{thirteen} \strut 5 297 | \nodepart{fourteen} \strut 13 298 | \nodepart{fifteen} \strut 19 299 | \nodepart{sixteen} \strut 19 300 | \nodepart{seventeen} \strut 12 301 | }; 302 | \foreach \Valor [count=\Valori from 0] in {one ,two ,three , four , five , six , seven , eight , nine , ten , eleven , twelve , thirteen , fourteen , fifteen , sixteen , seventeen } 303 | \node[below] at (array.\Valor south) {\Valori}; 304 | 305 | \end{tikzpicture} 306 | 307 | \end{center} 308 | 309 | Then the query for $[4,14]$ would simply subtract 13 from 19 to get 6. In general, once we have constructed this array, each query takes $O(1)$, since we simply look up 310 | two numbers in the array and subtract them. 311 | 312 | How might we construct this array in an efficient manner, though? We could use our na\"{i}ve solution $N$ times to get the $N$ sums that we need. This would then result 313 | in an $O(N^2 + Q)$ overall solution. But the $N^2$ factor is very offputting. Here, we can make use of our observation that we are doing a lot of repeated work. If we know the 314 | sum for $[1,i]$, then calculating the sum for $[1,i+1]$ is a simple matter of taking our previous answer and adding the $(i+1)$th term in the array: 315 | 316 | \begin{mylstlisting} 317 | int N; 318 | int arr[MAXN]; 319 | int prefix[MAXN]; 320 | void preprocess() { 321 | prefix[0] = 0; 322 | for(int k = 1; k <= N; ++k) 323 | prefix[k] = prefix[k - 1] + arr[k]; 324 | } 325 | int query(int i, int j) { 326 | return prefix[j] - prefix[i - 1]; 327 | } 328 | \end{mylstlisting} 329 | 330 | This solution is now $O(N + Q)$, much faster than the original $O(NQ)$ that we originally conceived! 331 | 332 | So, at a high level, what happened here? We first noticed there was a lot of repetition in the work we were doing. We found a way to cut out the repetition by first 333 | finding the answer to a small (in this case, linear) number of nice cases. These nice cases have a certain structure that allow us to solve individual queries very quickly. 334 | Finally, we exploited the same structure to compute those nice cases in a quick, neat fashion. 335 | 336 | \section{Two Pointers} 337 | 338 | -------------------------------------------------------------------------------- /chapters/fundamentals.tex: -------------------------------------------------------------------------------- 1 | \chapter{Fundamentals} 2 | 3 | \section{Introduction} 4 | 5 | Welcome to competitive programming! If you know a bit about coding and you're curious about programming contests, then you're in the right place. We'll start these notes with some basic background: what happens during a programming contest, which skills they train, and how to practice and become better. In the second part of this section, we'll walk through an example contest problem together. 6 | 7 | As you move forward, these notes will both guide you through key algorithmic and data structural ideas and provide interesting problems for you to think about. We'll try to challenge you, make you a better problem solver and programmer, and give you a sense for how the foundational concepts in computer science are intricately connected. The skills you'll pick up will be useful in not just other areas of computer science, but will also make you a stronger and more creative thinker. If you have questions or feedback about anything as you go along, feel free to contact us at \mailto{@gmail.com}. We hope you'll enjoy the world of competitive programming as much as we have. Have fun with it! 8 | 9 | \subsection{What is Competitive Programming?} 10 | 11 | A programming contest typically consists of a collection of problems and a time limit within which they must be solved. To solve a problem, you'll have to code a program that reads some parameters as input and produces an output of the desired format. You'll then submit your code to a \emph{judging server}, which is a server that compiles your code and executes it on a set of \emph{test cases}. To pass, your program must solve each test case using only predetermined amounts of \emph{time} and \emph{memory}. That is, your program must be \textit{efficient}. Furthermore, your program must be \textit{correct}. To check correctness, the output for each test case is either compared to the output of a correct program written by the contest organizers or plugged into a verifier that checks that your output satisfies the desired conditions. So as a contestant, your job will be to find an efficient algorithm, write correct code, and submit. 12 | 13 | This is just a very general description of programming contests. Each contest has its particularities in terms of scoring and the feedback you get on your submissions. Some contests will only mark your submission as correct if it correctly solves every test case, while others will give you partial credit for every test case you get correct. Some contests will also execute your submissions in real time, so you'll know if your code is correct within seconds, while others will only judge your final submissions after the contest is over. But the commonalities of these contests are in the skills they select for and train. 14 | 15 | Problem solving is possibly the most important skill you can learn. This skill, and not whether you can crank out code quickly, is what interesting programming contests are about. You'll be given problems you'll have no idea how to solve, and you'll want to creatively reason about these problems and discover efficient solutions. Of course, being able to write clean, accurate code is also of high importance, since it is your code, not the solution in your head, that gets judged. For programming language, we recommend (and our notes will cover) coding in \emph{Java} or \emph{C++}. Finally, a solid understanding of algorithms and data structures will give you the tools you'll need to crack these problems. This combination of problem solving, coding, and algorithmic thinking will get you a long way in programming contests, and you'll definitely be able to apply them elsewhere too. 16 | 17 | We'll do our best to teach you these skills by providing you with notes that show you the important algorithmic and data structural ideas, giving you pointers on how to write code, and presenting you with problems that will solidify your knowledge. I cannot emphasize how important solving the problems and writing the code is for your growth. Try to avoid reading solutions until you've given the problem a genuine shot and feel like you've stopped making progress. And if you do read a solution, think about how you'll be able to solve a similar problem the next time it appears. After you solve a problem, code it up, and don't give up trying to fix and optimize your code until you get all test cases accepted. Think of each bug you find as another bug you'll never see again. 18 | 19 | But there is only so much we can do. The challenging part---persevering with difficult problems, spending long hours spent debugging, and taking time from your busy day to code---is all on you. 20 | 21 | \subsection{First Problem: Sliding Puzzles} 22 | 23 | We'll now work through an example problem, adapted from a recent Codeforces contest. On the next page is a typical problem statement, with the problem description followed by input and output specifications and sample cases. Try to come up with a solution (and possibly write some code) before reading our analysis. As in the rest of these notes, we'll follow our analysis with some discussion of how to implement the solution and provide links to C++ and/or Java code. 24 | 25 | \clearpage 26 | 27 | \statement[Sliding Puzzles]{2s}{256MB}{% 28 | Bessie and her best friend Elsie each own a sliding puzzle. Each of their sliding puzzles consists of a $2\times 2$ grid and three tiles labeled $\mathcal A$, $\mathcal B$, and $\mathcal C$. The three tiles sit on top of the grid, with one grid cell left empty. To make a move, one slides a tile adjacent to the empty cell into the empty cell, as shown below: 29 | \begin{center} 30 | \includegraphics[scale=0.6]{images/sliding-puzzle.png} 31 | \end{center} 32 | Bessie and Elsie would like to know if there exists a sequence of moves that takes their puzzles to the same configuration. (Moves can be performed on both puzzles.) Two puzzles are considered to be in the same configuration if each tile is on top of the same grid cell in both puzzles. Since the tiles are labeled with letters, rotations and reflections are not allowed. 33 | }{% 34 | The first two lines of the input consist of a $2\times 2$ grid describing the initial configuration of Bessie's puzzle. The next two lines contain a $2\times 2$ grid describing the initial configuration of Elsie's puzzle. The positions of the tiles are labeled $\mathtt A$, $\mathtt B$, and $\mathtt C$, while the empty cell is labeled $\mathtt X$. It is guaranteed that the input contains two valid descriptions of puzzles. 35 | }{% 36 | Print \texttt{YES} if the puzzles can reach the same configuration. Otherwise, print \texttt{NO}. 37 | }{\href{http://codeforces.com/contest/645/problem/A}{Codeforces 645A}} 38 | \sample{% 39 | AB 40 | XC 41 | XB 42 | AC% 43 | }{YES} 44 | \sample{% 45 | AB 46 | XC 47 | AC 48 | BX% 49 | }{NO} 50 | 51 | \clearpage 52 | 53 | We present two solutions to this problem. The first is the more ``obvious'' one, while the second shows how solutions can often be simplified through some thinking and some clever observations. 54 | 55 | \begin{proof}[Solution 1] 56 | One straightforward approach to this problem is to notice that there are only $4! = 24$ possible configurations that a puzzle could be in. Thus, for each puzzle in the input, it shouldn't be hard to find the list of configurations that the puzzle can reach. Once we have the lists of configurations, we can compare the two lists and check if they have an element in common. If they do, we output $\mathtt{YES}$; otherwise, we output $\mathtt{NO}$. 57 | 58 | To find the list of possible configurations, we maintain a list containing all of the possible configurations we have found so far. (This list starts off as the puzzle itself.) For every configuration in our list, we check if a single move can take us to a configuration we haven't seen before. Once we find a new configuration, we append it to the list and repeat. If there exist no such new configurations, then our list contains all possible configurations for that puzzle. 59 | \end{proof} 60 | 61 | However, this solution may be somewhat difficult to implement---we have to figure out how to nicely represent each configuration as a string and make moves to reach new configurations. Writing the code to find the list of possible configurations is also a bit complex. Instead, a simple observation can reduce the trickiness of the code significantly: 62 | 63 | \begin{proof}[Solution 2] 64 | Notice that two puzzles can reach the same configuration if and only if the $\mathcal A$, $\mathcal B$, and $\mathcal C$ tiles appear in the same orientation---clockwise or counterclockwise---in the two puzzles. Thus, it suffices to check if the two puzzles have the same orientation. We can do so by writing down the tiles of each puzzle in clockwise order and checking if one string is a cyclic shift of the other. 65 | \end{proof} 66 | 67 | To implement the first solution, you might want to make your list of possible configurations a \emph{dynamic array}---that is, a Java \texttt{ArrayList} or a C++ \texttt{vector}. This will allow you to easily append elements to the list. 68 | 69 | For the second solution, once we have the tiles in clockwise order, we'll want to check if one ordering is a cyclic shift of the other. Given two strings $s$ and $t$ of the same length, a clever way to do this is to check if $s$ is a substring of $t+t$. (Convince yourself that this is correct!) Take a look at a \href{http://codeforces.com/contest/645/submission/16801722}{C++} implementation of Solution 2 using this trick. 70 | 71 | \section{More Problems!} 72 | \label{sec:moreproblems} 73 | 74 | Because problems are the most important thing in the world, below are a couple more for you to chew on. We'll cover solutions to these problems later in the chapter, but if you feel ready, try submitting your code on \href{http://codeforces.com}{Codeforces} first. 75 | 76 | \statement[Vasily and Candles]{1s}{256MB}{% 77 | Vasily the programmer loves romance, so he plans to write code while bathed in the warm glow of candlelight. He has $a$ new candles, each of which burns for exactly one hour. Vasily is smart, so he can make a new candle from $b$ burnt out candles. Vasily wonders, for how many hours can his candles light up his room if he acts optimally? 78 | }{% 79 | The input contains two integers, $a$ and $b$ ($1\le a\le 1000$ and $2\le b\le 1000$). 80 | }{% 81 | Print a single integer---the maximum number of hours for which Vasily's candles can keep his room lit. 82 | }{\href{http://codeforces.com/problemset/problem/379/A}{Codeforces 379A}} 83 | \sample{4 2}{7} 84 | \sample{6 3}{8} 85 | 86 | \statement[Kefa and First Steps]{2s}{256MB}{% 87 | Kefa has started an Internet business $n$ days ago. On the $i$-th day $(1\le i\le n)$, he made a profit of $a_i$ dollars. Kefa loves progress, so he wants to know the length of the longest non-decreasing subsegment in her sequence of profits. (Here, a subsegment of a sequence denotes a contiguous subsequence $a_i,a_{i+1}\ldots,a_j$ ($i < j$).) 88 | }{% 89 | The first line contains an integer $n$ ($1\le 10^5$). The second line contains $n$ integers $a_1,a_2,\ldots,a_n$ ($1\le a_i\le 10^9$). 90 | }{% 91 | Print a single integer--the length of the maximum non-decreasing sequence in Kefa's profits. 92 | }{\href{http://codeforces.com/problemset/problem/580/A}{Codeforces 580A}} 93 | \sample{% 94 | 6 95 | 2 2 1 3 4 1% 96 | }{3} 97 | \sample{% 98 | 3 99 | 2 2 9% 100 | }{3} 101 | 102 | \section{Input and Output} 103 | 104 | Now that you've familiarized yourself with what programming contest problems are like, let's get down to the details. The first part of solving any problem is reading the input correctly. As you may expect, doing so is very dependent on programming language. In this section, we'll cover input and output (I/O) in both Java and C++. Read our discussion for whichever one you plan to use. 105 | 106 | \subsection{Java} 107 | Here, we'll focus on Java I/O using \texttt{java.util.Scanner} and \texttt{java.io.PrintWriter}. There are two scenarios that you should be familiar with: standard I/O and file I/O. That is, interacting with \texttt{System.in/System.out} and files like \texttt{in.txt/out.txt}, respectively. You may have encountered standard I/O when you enter input and see output while running a program in the command line. 108 | 109 | When using standard I/O, we can read from \texttt{System.in} using \texttt{java.util.Scanner} and output using \texttt{System.out.println}. To declare a new Scanner, simply call the constructor with \texttt{new Scanner(System.in)}. Here's a quick outline of Scanner methods: 110 | \begin{center} 111 | \begin{tabularx}{0.75\textwidth}{|l|X|} 112 | \hline 113 | Method & Description \\ \hline 114 | \texttt{Scanner.next()} & Reads the next token in the input (i.e. up to a whitespace) and returns the token as a \texttt{string}. \\ \hline 115 | \texttt{Scanner.nextLine()} & Reads the input up to a line break and returns the contents read as a \texttt{string}. \\ \hline 116 | \texttt{Scanner.nextInt()} & Reads the next token in the input (i.e. up to a whitespace) and returns the token as an \texttt{int}. \\ \hline 117 | \texttt{Scanner.nextLong()} & Reads the next token in the input (i.e. up to a whitespace) and returns the token as an \texttt{long}. \\ \hline 118 | \texttt{Scanner.nextDouble()} & Reads the next token in the input (i.e. up to a whitespace) and returns the token as an \texttt{double}. \\ \hline 119 | \end{tabularx} 120 | \end{center} 121 | \texttt{System.out.println()} prints its argument and adds a newline at the end. (If you don't want the newline, you can use \texttt{System.out.print()}.) Here's an example of a \texttt{main} method that takes two integers and outputs their sum: 122 | 123 | \begin{mylstlisting} 124 | public static void main(String args[]) { 125 | // hint: you should have "import java.util.*;" at the top of your code. 126 | Scanner sc = new Scanner(System.in); 127 | int x = sc.nextInt(); 128 | int y = sc.nextInt(); 129 | System.out.println(x + y); 130 | } 131 | \end{mylstlisting} 132 | 133 | File I/O is a touch more complicated. For our Scanner, we now have to call the constructor with a File object (e.g. with \texttt{new File("in.txt")}). We do the same with output for our PrintWriter (e.g. with \texttt{new File("out.txt")}). We can then use PrintWriter like we use \texttt{System.out}, by calling \texttt{pw.println()} and \texttt{pw.print()} for a PrintWriter \texttt{pw}. 134 | 135 | However, PrintWriter also comes with a couple more usage notes. First, we should include \texttt{throws IOException} after our \texttt{main} method, since Java requires that we acknowledge the possibility of an \texttt{IOException} in the case that something goes wrong. After we finish printing, we must also close the PrintWriter in order to ensure that everything gets written to the file. Here's a snippet showing how Scanner and PrintWriter work together with files: 136 | 137 | \begin{mylstlisting} 138 | public static void main(String args[]) throws IOException { 139 | // hint: for file I/O, you should also have "import java.io.*;" 140 | Scanner sc = new Scanner(new File("in.txt")); 141 | int x = sc.nextInt(); 142 | int y = sc.nextInt(); 143 | PrintWriter pw = new PrintWriter(new File("out.txt")); 144 | pw.println(x + y); 145 | pw.close(); 146 | } 147 | \end{mylstlisting} 148 | 149 | Although more efficient methods of I/O exist, such as BufferedReader and BufferedWriter, what we've covered here should be sufficient for now. For example, it is possible to read $10^5$ integers with Scanner in a fraction of a second. 150 | 151 | \subsection{C++} 152 | 153 | Here, we discuss I/O in C++ using the Standard Template Library's (STL) \texttt{iostream} and \texttt{fstream}. There are two scenarios that you should be familiar with: standard I/O and file I/O. You may have encountered standard I/O when you enter input and see output while running a program in the command line, whereas file I/O involves reading from and writing to files like \texttt{in.txt} or \texttt{out.txt}. In C++, standard I/O is done with the \texttt{cin} and \texttt{cout} objects in \texttt{iostream}, and file I/O is done with the \texttt{ofstream} and \texttt{ifstream} classes in \texttt{fstream}. We'll go through each of these below. 154 | 155 | Using \texttt{cin} and \texttt{cout} is pretty straightforward. If you have a variable \texttt{x} that you want to read input into, you can do so by writing \verb+cin >> x+. If \texttt{x} is an \texttt{int}, \texttt{double}, or \texttt{long long}, this will read the next such number in the input (up to a whitespace) into \texttt{x}. If \texttt{x} is a string, then \texttt{cin} will read similarly the input up to a whitespace into \texttt{x}. To output a variable \texttt{x} that is of type \texttt{int}, \texttt{double}, \texttt{long long}, \texttt{string}, or \texttt{bool}, we simply write \verb+cout << x+. To output a newline, you can write \verb+cout << endl+. And that's all! 156 | 157 | Here's an example with \texttt{cin} and \texttt{cout} that outputs the sum of two integers: 158 | 159 | \begin{mylstlisting} 160 | int main() { 161 | // hint: you should have "#include " at the top of your code. 162 | int x, y; 163 | cin >> x >> y; 164 | cout << x + y << endl; 165 | } 166 | \end{mylstlisting} 167 | 168 | Moving on to file I/O, suppose we want to read from \texttt{in.txt} and write to \texttt{out.txt}. We construct an \texttt{ifstream} and an \texttt{ofstream} on \texttt{in.txt} and \texttt{out.txt}, respectively. We can do so by writing \texttt{ifstream fin("in.txt")} and \texttt{ofstream fout("out.txt")}. Then \texttt{fin} and \texttt{fout} behave just like \texttt{cin} and \texttt{cout}. Here's an example: 169 | 170 | \begin{mylstlisting} 171 | int main() { 172 | // hint: you should have "#include " at the top of your code. 173 | ifstream fin("in.txt"); 174 | ofstream fout("in.txt"); 175 | int x, y; 176 | fin >> x >> y; 177 | fout << x + y << endl; 178 | } 179 | \end{mylstlisting} 180 | 181 | Although more efficient methods of I/O exist, such as \texttt{scanf} and \texttt{printf}, what we've covered here should be sufficient for now. For example, it is possible to read $10^5$ integers with \texttt{cin} and \texttt{cout} in a fraction of a second. 182 | 183 | \section{Complexity} 184 | 185 | Before, we mentioned that contest problems test your ability to come up with efficient algorithms and to write accurate code. \emph{Implementation problems} are problems that for the most part, assess the latter---that is, your ability to write code quickly and accurately. However, these problems are usually only common in easier contests, since they don't involve too much thinking or creativity; you just have to carefully implement what's written in the problem statement. Instead, most competitive programming problems ask you to come up with clever algorithms that are both fast and space-efficient. 186 | 187 | To formally analyze the efficiency of algorithms, computer scientists use the notion of \emph{complexity}. Complexity is roughly the number of steps an algorithm takes as a function of the input size. You can imagine algorithms that require $3n$, $n^4/3$ or even $2^n + n^2$ steps to halt for an input of size $n$. We categorize algorithms of different complexities using \emph{big-O notation}: If an algorithm takes $f(n)$ steps to halt on an input of size $n$, we say that the algorithm is $O(f(n))$. However, this notation ignores any constant factors and lower-order terms in the expression. For example, an algorithm that requires $100n^2 + 5$ steps is still $O(n^2)$.\footnote{Actually, this isn't entirely accurate. Saying an algorithm is $O(f(n))$ really means that there exists a $c > 0$ such that the algorithm takes \emph{at most} $c\cdot f(n)$ steps to halt.} I'll explain why in a moment---let's look at some examples for now. 188 | 189 | Suppose we have three programs $A$, $B$, and $C$ that require $3n$, $n^4/3+10$ and $2^n + n^2$ steps to finish, respectively. The complexity of the program A$A$ is $O(n)$ because we ignore the constant factor $3$ on the $3n$. The complexity of the program $B$ is $O(n^4)$. Here, we drop the constant $1/3$ and the lower-order term $10$. For program $C$, we write its complexity as $O(2^n)$ because $n^2$ is a lower-order term relative to $2^n$. 190 | 191 | As to why we drop the constants and the lower order terms, consider programs $A$ and $B$ from above. When $n=300$, the first program takes $900$ steps, while the second program takes $2,700,000,010$ steps. The second program is much slower, despite a smaller constant factor. Meanwhile, if we had another program that runs in $5n$ steps, it would still only take $1,500$ steps to finish. Notice how the $10$ after $n^4/3$ also looks irrelevant here. The takeaway is that constant factors and lower-order terms get dwarfed when comparing functions that grow at different rates. 192 | 193 | Thus in programming contests, we usually want a program to have a sufficiently good complexity, without worrying about too much constant factors. Complexity will be the difference between whether a program gets \emph{\color{green}accepted} or \emph{\color{red}time limit exceeded}. As a rule of thumb, a modern processor can do around $10^8$ computations each second. When you plug the maximum possible input into the complexity of your algorithm, it should never be much more than that. 194 | 195 | We've focused on time and haven't talked much about memory so far, but memory can also be tested. However, in contests, memory limits are usually much more generous than time limits. The amount of memory a program uses as a function of $n$ is called its \emph{space complexity}, as opposed to the \emph{time complexity} that we discussed earlier. If a program uses $2n^2$ bytes of memory on an input of length $n$, then it has a space complexity of $O(n^2)$. 196 | 197 | \section{More Solutions} 198 | 199 | % Now that you know the basics, let's get started by solving a few problems. Try to figure out a solution to each problem before reading our analysis. To test your code for the problems in this chapter, you should create an account on Codeforces. After you login, there will be an ``Introduction'' problem set available \href{http://codeforces.com/group/5tN48zOVvQ/contests}{here} that includes these problems and a few extras. 200 | 201 | Have you given the problems from \ref{sec:moreproblems} a shot yet and submitted some code? If you haven't, do so before reading the solutions below. 202 | 203 | \subsection{Vasily and Candles} 204 | 205 | \begin{proof}[Solution] 206 | Vasily and Candles is a straightforward implementation problem. Since $a$ and $b$ are small, we can simulate Vasily's candle burning process. We track the number of candles we have left, the number of burnt out candles, and the number of hours that have passed. Whenever we have more than $b$ burnt out candles, we make a new candle. Once we run out of candles, we print the answer. In terms of implementation, a \texttt{while} loop is handy here. 207 | \end{proof} 208 | 209 | \subsection{Kefa and First Steps} 210 | 211 | \begin{proof}[Solution] 212 | Our first thought upon reading this problem might be to check each subsegment of the sequence $a_i$ and see if that subsegment is non-decreasing. Unfortunately, this is too slow: There are approximately $n^2/2$ pairs of endpoints that we could choose, far too many when $n$ can be up to $10^5$. (Recall that the maximum number of computations we can do is \textasciitilde$10^8$.) Instead, we can solve this problem in $O(n)$ by executing a single \texttt{for} loop over the sequence. We maintain a counter representing Kefa's current ``streak''---the number of days since her profit last decreased. If her profit decreases from day $i$ to day $i+1$, then we reset the counter to zero. The answer we report is the longest streak that we ever see. 213 | \end{proof} 214 | 215 | This problem is a clear example of how getting the right complexity is essential. Our initial idea, which could have been implemented in $O(n^3)$ or $O(n^2)$, was too slow. To make our program finish within time limit, we had to come up with a more efficient approach that runs in $O(n)$. 216 | 217 | \section{Even More Problems} 218 | 219 | For some more fun, check out the rest of the \href{http://codeforces.com/group/5tN48zOVvQ/contest/204614}{Introduction} problem set that we put together on Codeforces. These problems will familiarize you with the essentials of contest programming---algorithmic thinking and writing accurate code. Enjoy! 220 | 221 | \begingroup 222 | \chapter*{Interlude A} 223 | \addcontentsline{toc}{chapter}{Interlude A} 224 | % \renewcommand\thesection{\Alph{chapter}.\arabic{section}} 225 | % \setcounter{section}{0} 226 | 227 | \section{Sorting} 228 | 229 | To further explore the concept of complexity, we will use sorting algorithms as a case study. Sorting is just as it sounds---given a collection of objects, we want to sort them according to some ordering. For example, suppose we have a list of scores from a programming contest. In order to generate the final standings, we'll need to sort the contestants in descending order by score. Below, we present three classic sorting algorithms of varying complexity: insertion sort, merge sort, and quicksort. Insertion sort runs in $O(n^2)$, while merge sort and quicksort both run in $O(n\log n)$. 230 | 231 | Don't worry too much about the details of these algorithms for now. You'll rarely need to implement them from scratch, since almost all modern programming languages come with built-in sorting algorithms. In our last subsection, we'll provide an example using these library functions in Java and C++ by working through a problem for which sorting is a subtask. 232 | 233 | % To supplement our descriptions of the algorithms, you can check out the animations at \url{http://visualgo.net/sorting.html}. 234 | 235 | \subsection{Insertion Sort} 236 | 237 | Insertion sort builds up a sorted list by inserting new elements one at a time. Inserting an element into a sorted list can be done in time proportional to the length of the list, so the runtime of this algorithm is $1 + 2 + 3 + \cdots + n = (n^2 + n) / 2$, which is $O(n^2)$. Here's some pseudo code for insertion sort: 238 | 239 | Notice that after the $i$-th iteration, we have a sorted list in the first $i$ entries of the array. Insertion sort, despite being slower than merge sort and quicksort, is still useful because of its efficiency on small inputs. Many implementations of merge sort and quicksort actually use insertion sort once the problem size gets small. 240 | 241 | \begin{exercise} 242 | Around how long is the longest list that you can sort with insertion sort in less than a second? 243 | \end{exercise} 244 | 245 | \subsection{Merge Sort} 246 | 247 | The idea behind merge sort is the observation that given two sorted lists of length $n/2$, we only need $n$ comparisons to merge them into a single sorted list of length $n$. We merge the two lists as follows: First, it is easy to find the smallest element among the two lists, since this element has to be the smallest element in one of the lists. To find the second-smallest element, we can delete the smallest element and do the same comparison again. 248 | 249 | This method of merging lists allows us to sort using a divide and conquer approach. We split the array in half, sort each half recursively with merge sort, and then merge the two halves back together. Because our recursion goes $\log_2 n$ levels deep and requires $O(n)$ operations per level, this algorithm runs in $O(n \log n)$. (In fact, it is possible to prove that $O(n \log n)$ is optimal for comparison-based sorting algorithms, one of the few problems in computer science that has a non-trivial lower bound.) 250 | 251 | \begin{exercise} 252 | Around how long is the longest list that you can sort with merge sort in less than a second? 253 | \end{exercise} 254 | 255 | \subsection{Quicksort} 256 | 257 | Quicksort also uses a divide and conquer strategy to run in $O(n \log n)$ on average. We first choose a random element from the array and call it the \emph{pivot}. We rearrange the array so that anything less than the pivot to the left of the pivot and anything greater than the pivot to the right of the pivot. This rearranging can be done in $O(n)$. Like merge sort, we can then recursively quicksort the two ``halves'' of the array defined by the pivot. Since we chose the pivot randomly, our problem size gets reduced down by a factor of $3/4$ most of the time, giving us $O(\log n)$ levels of recursion with $O(n)$ operations at each level. Thus quicksort runs in $O(n \log n)$ on average. We say ``on average'' because there do exist cases that make quicksort run in $O(n^2)$. 258 | 259 | \begin{exercise} 260 | What would happen if we chose the smallest element of the array as the pivot each time? 261 | \end{exercise} 262 | 263 | \subsection{Sorting Applied} 264 | 265 | \statement[Ms. Manana's Puzzles]{1s}{256MB}{% 266 | The end of the school year is near and Ms. Manana, the teacher, will soon have to say goodbye to yet another class. As a farewell present, she decides to give each of here $n$ students a jigsaw puzzle. The shop asssistant tells Ms. Manana that there are $m$ puzzles in the shop, but they differ in difficulty and size. Specifically, the $i$-th jigsaw puzzle consists of $f_i$ pieces. Ms. Manana doesn't want to upset the children, so she wants the difference between the numbers of pieces in the largest puzzle and the smallest puzzle that she buys to be as small as possible. Help Ms. Manana compute this minimum difference. 267 | }{% 268 | The first line contains space-separated integers $n$ and $m$ ($2\le n\le m\le 50$). The next line contains $m$ space-separated integers $f_1,f_2,\ldots,f_m$ ($4\le f_i\le 1000$). 269 | }{% 270 | Print a single integer---the least possible difference that Ms. Manana can obtain. 271 | }{\href{http://codeforces.com/problemset/problem/337/A}{Codeforces 337A}} 272 | \sample{% 273 | 4 6 274 | 10 12 10 7 5 22 275 | }{5} 276 | 277 | \begin{proof}[Solution] 278 | We solve this problem by first sorting the sequence $f_i$. After sorting, Ms. Manana will want to buy puzzles from a contiguous block of the sequence. (If she doesn't, then the difference between the largest and smallest puzzles will be greater than necessary.) Thus we can iterate through the sorted sequence to find the minimum difference between the endpoints of each subsegment of length $n$. 279 | \end{proof} 280 | 281 | Usually, when solving a sorting problem, we don't need to implement our own sorting function. Java users have \texttt{Arrays.sort} in \texttt{java.utils.Arrays} that does the magic for you. For those who code in C++, if you \texttt{\#include }, you can then use \texttt{std::sort}. While coding up Ms. Manana's Puzzles, try to use the library sort function in your language. 282 | 283 | Here are code snippets for sorting an array \texttt{arr} of length \texttt{n} in Java and C++, respectively: 284 | 285 | \begin{mylstlisting} 286 | // hint: you should have "import java.util.*;" at the top of your code. 287 | int[] arr = new int[n]; 288 | // do something to fill up the array. 289 | Arrays.sort(arr); 290 | \end{mylstlisting} 291 | 292 | \begin{mylstlisting} 293 | // hint: you should have "#include " at the top of your code. 294 | int arr[n]; 295 | // do something to fill up the array. 296 | std::sort(arr, arr + n); 297 | \end{mylstlisting} 298 | \endgroup 299 | --------------------------------------------------------------------------------