├── .DS_Store ├── Fall18 ├── README.md ├── _config.yml ├── _layouts │ └── default.html ├── hw │ ├── CS7545hw_template.tex │ ├── hw1.pdf │ ├── hw2.pdf │ ├── hw3.pdf │ ├── hw4.pdf │ └── hw5.pdf └── scribe │ ├── .DS_Store │ ├── CS7545scribe_template.tex │ ├── lec1.pdf │ ├── lec10.pdf │ ├── lec11.pdf │ ├── lec12.pdf │ ├── lec13.pdf │ ├── lec14.pdf │ ├── lec15.pdf │ ├── lec16.pdf │ ├── lec17.pdf │ ├── lec18.pdf │ ├── lec19.pdf │ ├── lec2.pdf │ ├── lec20.pdf │ ├── lec21.pdf │ ├── lec22.pdf │ ├── lec23.pdf │ ├── lec24.pdf │ ├── lec25.pdf │ ├── lec26.pdf │ ├── lec3.pdf │ ├── lec4.pdf │ ├── lec5.pdf │ ├── lec6.pdf │ ├── lec7.pdf │ ├── lec8.pdf │ └── lec9.pdf ├── Fall19 └── README.md ├── Fall2023 └── README.md ├── README.md ├── _config.yml ├── _layouts ├── default.html └── default.html.old ├── exam ├── CS7545_Spring_2023_PracticeExam.pdf └── CS7545_practice_exam_sol.pdf ├── hw ├── .DS_Store ├── CS7545_HW1.pdf ├── CS7545_HW1_sol.pdf ├── CS7545_HW2.pdf ├── CS7545_HW2_sol.pdf ├── CS7545hw_template.tex ├── HW3-Problem.pdf └── HW3-Solution.pdf ├── robot_blackboard.png ├── robot_blackboard_2024.png └── scribe ├── .DS_Store ├── CS7545scribe_template.tex ├── hu.pdf ├── lec1.pdf ├── lec10.pdf ├── lec11.pdf ├── lec12.pdf ├── lec13.pdf ├── lec14.pdf ├── lec15.pdf ├── lec16.pdf ├── lec17.pdf ├── lec18.pdf ├── lec19.pdf ├── lec2.pdf ├── lec20.pdf ├── lec21.pdf ├── lec22.pdf ├── lec23.pdf ├── lec24.pdf ├── lec25.pdf ├── lec26.pdf ├── lec3.pdf ├── lec4.pdf ├── lec5.pdf ├── lec6.pdf ├── lec7.pdf ├── lec8.pdf ├── lec9.pdf └── wlec21.pdf /.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/.DS_Store -------------------------------------------------------------------------------- /Fall18/README.md: -------------------------------------------------------------------------------- 1 | 2 | ### Course Information 3 | 4 | * **Course Info:** CS7545, Fall 2018 5 | * **Instructor:** Jacob Abernethy 6 | - **Office:** Klaus 2134 7 | - **Email:** prof_at_gatech_dot_edu 8 | - **Office Hours:** 10-11am Wednesdays in Klaus 2134 9 | * **Course Time&Place:** MW 4:30-5:45pm, Klaus 2443 10 | * **Teaching Assistants**: 11 | - *Bhuvesh Kumar* 12 | - **Email:** bhuvesh_at_gatech.edu 13 | - **Office Hours**: Mondays 2:30-3:30pm, alcove of Klaus 2116/2124 14 | - *Jun-Kun Wang* 15 | - **Email:** jimwang_at_gatech.edu 16 | - **Office Hours**: Wednesdays 11am-12pm, alcove of Klaus 2116/2124 17 | 18 | 19 | ### Course Description 20 | 21 | This course will study theoretical aspects of prediction and decision-making problems, where our goal is to understand the mathematical underpinnings of machine learning. A primary objective of the class is to bring students to the frontiers of research and to prepare students to publish in this area. The course will cover, among other things, concentration inqualities, uniform deviation bounds, Vapnik-Chervonenkis Theory, Rademacher Complexity, margin bounds, boosting, some theoretical aspects of deep learning, online learning theory, regret minimization, multi-armed bandit algorithms, and connections to convex optimization. Along the way, we may dive into several related topics, including minimax equilibrium in games, calibration, sequential portfolio selection, option pricing, and differential privacy. 22 | 23 | **Prerequisites:** Familiarity with the analysis of algorithms, probabilistic analysis, and several similar topics. CS7641 (Machine Learning) will be quite helpful but not strictly necessary. The material is going to be about 90% "theory" and thus potential students must have a strong mathematical background. We shall rely heavily on techniques from calculus, probability, and convex analysis, but many tools will be presented in lecture. 24 | 25 | **Coursework:** There will be 5 problem sets throughout the semester. 26 | 27 | **Grade Breakdown:** 28 | * 50% - *Homeworks* 29 | * 40% - *Final Exam* 30 | * 10% - *Participation* 31 | 32 | **Note**: The final exam will be held on Friday, December 7, from 2:40-5:30pm. 33 | 34 | 35 | ### References: 36 | 37 | Roughly half of the course will follow material from the following text: 38 | 39 | * "[Foundations of Machine Learning](https://www.amazon.com/Foundations-Machine-Learning-Adaptive-Computation/dp/026201825X)" by Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar 40 | 41 | Much of the material in online learning (aka regret minimization) is of my own taste, and I will present these topics how I enjoy. But for students that want reading material on this topic, there are several surveys released in the last several years that explore several many that we shall cover. I will link to them here, and will mention them in various lectures when appropriate: 42 | 43 | * [The Multiplicative Weights Update Method](http://www.cs.princeton.edu/~arora/pubs/MWsurvey.pdf) by Sanjeev Arora, Elad Hazan, and Satyen Kale. 44 | * [Online Learning and Online Convex Optimization survey](http://www.cs.huji.ac.il/~shais/papers/OLsurvey.pdf) by Shai Shalev-Shwartz. 45 | * [The convex optimization approach to regret minimization survey](http://www.cs.princeton.edu/~ehazan/papers/OCO-survey.pdf) by Elad Hazan. 46 | * [Sasha Rakhlin's Lecture Notes](http://www-stat.wharton.upenn.edu/~rakhlin/courses/stat928/stat928_notes.pdf). 47 | 48 | 49 | ### Scribe Notes 50 | 51 | | Lecture | Date | Topic | 52 | | :------------: |:-------------: |:-------------: | 53 | | [1](./scribe/lec1.pdf) | 20 Aug 2018 | Introduction and norms | 54 | | [2](./scribe/lec2.pdf) | 22 Aug 2018 | Convex analysis | 55 | | [3](./scribe/lec3.pdf) | 27 Aug 2018 | Convex Analysis + Deviation Bounds | 56 | | [4](./scribe/lec4.pdf) | 29 Aug 2018 | Concentration Inequalities | 57 | | [5](./scribe/lec5.pdf) | 05 Sep 2018 | Martingales + Online Learning | 58 | | [6](./scribe/lec6.pdf) | 10 Sep 2018 | Multiplicative Weights | 59 | | [7](./scribe/lec7.pdf) | 12 Sep 2018 | EWA + Perceptron | 60 | | [8](./scribe/lec8.pdf) | 17 Sep 2018 | Perceptron + Game Theory | 61 | | [9](./scribe/lec9.pdf) | 19 Sep 2018 | Zero-sum Games + Boosting | 62 | | [10](./scribe/lec10.pdf) | 26 Sep 2018 | Boosting + Online Convex Optimization| 63 | | [11](./scribe/lec11.pdf) | 01 Oct 2018 | Online Convex Optimization | 64 | | [12](./scribe/lec12.pdf) | 03 Oct 2018 | Online Convex Optimization (Cont.)| 65 | | [13](./scribe/lec13.pdf) | 10 Oct 2018 | FTRL | 66 | | [14](./scribe/lec14.pdf) | 15 Oct 2018 | Multi-armed Bandits | 67 | | [15](./scribe/lec15.pdf) | 17 Oct 2018 | Stochastic Multi-Armed Bandits | 68 | | [16](./scribe/lec16.pdf) | 22 Oct 2018 | UCB algorithm | 69 | | [17](./scribe/lec17.pdf) | 24 Oct 2018 | Statistical Learning Theory | 70 | | [18](./scribe/lec18.pdf) | 29 Oct 2018 | Contextual Bandits | 71 | | [19](./scribe/lec19.pdf) | 31 Oct 2018 | Reinforcement Learning | 72 | | [20](./scribe/lec20.pdf) | 05 Nov 2018 | VC Dimension + Rademacher Complexity | 73 | | [21](./scribe/lec21.pdf) | 07 Nov 2018 | Growth Function and Massart’s Lemma | 74 | | [22](./scribe/lec22.pdf) | 12 Nov 2018 | Massart’s Lemma and Sauer’s Lemma | 75 | | [23](./scribe/lec23.pdf) | 14 Nov 2018 | VC-Dimension Upper & Lower Bounds | 76 | | [24](./scribe/lec24.pdf) | 19 Nov 2018 | Generalization Error + Margin Theory| 77 | | [25](./scribe/lec25.pdf) | 26 Nov 2018 | VC Dimension of Neural Networks | 78 | | [26](./scribe/lec26.pdf) | 28 Nov 2018 | Variational inference | 79 | 80 | [The Latex template for scribes is available here.](./scribe/CS7545scribe_template.tex) 81 | 82 | ### Homeworks 83 | 84 | | Homework | Due Date | 85 | | :------------: |:-------------: | 86 | | [1](./hw/hw1.pdf) | Sep 10 2018, 2:00 pm | 87 | | [2](./hw/hw2.pdf) | Sep 24 2018, 2:00 pm | 88 | | [3](./hw/hw3.pdf) | Oct 17 2018, 2:00 pm | 89 | | [4](./hw/hw4.pdf) | Nov 16 2018, 4:00 pm | 90 | | [5](./hw/hw5.pdf) | Dec 2 2018, 11:59 pm | 91 | 92 | 93 | [The Latex template for HW submissions is available here.](./hw/CS7545hw_template.tex) 94 | 95 | -------------------------------------------------------------------------------- /Fall18/_config.yml: -------------------------------------------------------------------------------- 1 | theme: jekyll-theme-slate 2 | 3 | title: CS7545-Machine Learning Theory, Fall 2018 4 | description: Georgia Tech 5 | -------------------------------------------------------------------------------- /Fall18/_layouts/default.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | {% seo %} 13 | 14 | 15 | 16 | 17 | 18 |
19 |
20 | 21 | 22 |

{{ site.title | default: site.github.repository_name }}

23 |

{{ site.description | default: site.github.project_tagline }}

24 | 25 | {% if site.show_downloads %} 26 |
27 | Download this project as a .zip file 28 | Download this project as a tar.gz file 29 |
30 | {% endif %} 31 |
32 |
33 | 34 | 35 |
36 |
37 | {{ content }} 38 |
39 |
40 | 41 | 42 | 49 | 50 | {% if site.google_analytics %} 51 | 59 | {% endif %} 60 | 61 | 62 | -------------------------------------------------------------------------------- /Fall18/hw/CS7545hw_template.tex: -------------------------------------------------------------------------------- 1 | \documentclass{article} 2 | \usepackage[T1]{fontenc} 3 | \usepackage{amssymb, amsmath, graphicx, subfigure, enumerate} 4 | \usepackage{amsthm,alltt} 5 | \usepackage[margin=1.25in]{geometry} %geometry (sets margin) and other useful packages 6 | \usepackage{graphicx,ctable,booktabs} 7 | \usepackage{mathtools} 8 | \usepackage[boxed]{algorithm2e} 9 | \usepackage{fancyhdr} %Fancy-header package to modify header/page numbering 10 | \usepackage{cleveref} 11 | 12 | \setlength{\oddsidemargin}{.25in} 13 | \setlength{\evensidemargin}{.25in} 14 | \setlength{\textwidth}{6in} 15 | \setlength{\topmargin}{-0.4in} 16 | \setlength{\textheight}{8.5in} 17 | 18 | \newcommand{\heading}[6]{ 19 | \renewcommand{\thepage}{\arabic{page}} % used to be #1-\arabic{page} 20 | \noindent 21 | \begin{center} 22 | \framebox{ 23 | \vbox{ 24 | \hbox to 5.78in { \textbf{#2} \hfill #3 } 25 | \vspace{4mm} 26 | \hbox to 5.78in { {\Large \hfill #6 \hfill} } 27 | \vspace{2mm} 28 | \hbox to 5.78in { \textit{Instructor: #4 \hfill #5} } 29 | } 30 | } 31 | \end{center} 32 | \vspace*{4mm} 33 | } 34 | 35 | %Redefining sections as problems 36 | \makeatletter 37 | \newenvironment{problem}{\@startsection 38 | {section} 39 | {2} 40 | {-.2em} 41 | {-3.5ex plus -1ex minus -.2ex} 42 | {2.3ex plus .2ex} 43 | {\pagebreak[3]%forces pagebreak when space is small; use \eject for better results 44 | \large\bf\noindent{Problem } 45 | } 46 | } 47 | %{%\vspace{1ex}\begin{center} \rule{0.3\linewidth}{.3pt}\end{center}} 48 | %\begin{center}\large\bf \ldots\ldots\ldots\end{center}} 49 | \makeatother 50 | 51 | 52 | \newtheorem{theorem}{Theorem}[section] 53 | \newtheorem{definition}[theorem]{Definition} 54 | \newtheorem{remark}[theorem]{Remark} 55 | \newtheorem{lemma}[theorem]{Lemma} 56 | \newtheorem{corollary}[theorem]{Corollary} 57 | \newtheorem{proposition}[theorem]{Proposition} 58 | \newtheorem{claim}[theorem]{Claim} 59 | \newtheorem{observation}[theorem]{Observation} 60 | \newtheorem{fact}[theorem]{Fact} 61 | \newtheorem{assumption}[theorem]{Assumption} 62 | 63 | % don't modify this unless you know what you're doing 64 | \newcommand{\problemset}[3]{\heading{#1}{\classname: Machine Learning Theory}{#2}{Jake Abernethy}{#3}{Problem Set #1}} 65 | 66 | 67 | %%%%%%%% ENTER YOUR INFORMATION HERE %%%%%%%% 68 | \newcommand{\problemsetnum}{1} % problem set number 69 | \newcommand{\duedate}{September 10, 2018} % problem set deadline 70 | \newcommand{\studentname}{STUDENT NAME} % name of student (i.e., you) 71 | \newcommand{\classname}{CS 7545} 72 | %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 73 | 74 | \pagestyle{fancy} 75 | %\addtolength{\headwidth}{\marginparsep} %these change header-rule width 76 | %\addtolength{\headwidth}{\marginparwidth} 77 | \lhead{\classname} %Problem \thesection} 78 | \chead{} 79 | \rhead{\thepage} 80 | \lfoot{\small\scshape \classname} 81 | \cfoot{} 82 | \rfoot{\footnotesize PS \#\problemsetnum} 83 | \renewcommand{\headrulewidth}{.3pt} 84 | \renewcommand{\footrulewidth}{.3pt} 85 | \setlength\voffset{-0.25in} 86 | \setlength\textheight{648pt} 87 | 88 | 89 | \newcommand{\sit}{\shortintertext} 90 | \newcommand\deq{\mathrel{\overset{\makebox[0pt]{\mbox{\normalfont\tiny\sffamily def}}}{=}}} 91 | \newcommand{\ones}{\mathbbm{1}} 92 | \newcommand{\e}{\vec{e}} 93 | \newcommand{\tr}{\text{tr}} 94 | \newcommand{\bs}{\boldsymbol} 95 | \mathchardef\mhyphen="2D 96 | \newcommand{\C}{\mathbb{C}} 97 | \newcommand{\R}{\mathbb{R}} 98 | 99 | \newcommand{\vol}{\text{vol}} 100 | 101 | \renewcommand{\thesubsection}{\thesection.\alph{subsection}} 102 | 103 | 104 | % auto sized delimiters 105 | 106 | \newcommand{\br}[1]{\left[#1\right]} 107 | \newcommand{\pr}[1]{\left(#1\right)} 108 | \newcommand{\ceil}[1]{\left\lceil#1\right\rceil} 109 | \newcommand{\floor}[1]{\left\lfloor#1\right\rfloor} 110 | \newcommand{\abs}[1]{\left|#1\right|} 111 | 112 | %default delimiter for Pr and E 113 | \DeclarePairedDelimiter{\defaultDelim}{[}{]} 114 | 115 | \DeclareMathOperator{\capPr}{Pr} 116 | \renewcommand{\Pr}[2][]{\capPr_{#1}\defaultDelim*{#2}} 117 | \DeclareMathOperator{\capE}{E} 118 | \newcommand{\E}[2][]{\capE_{#1}\defaultDelim*{#2}} 119 | \DeclareMathOperator{\capVar}{Var} 120 | \newcommand{\Var}[2][]{\capVar_{#1}\defaultDelim*{#2}} 121 | 122 | %\DeclareMathOperator*{}{} puts subscript below 123 | 124 | 125 | %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 126 | \begin{document} 127 | \problemset{\problemsetnum}{\duedate}{\studentname} 128 | 129 | \begin{problem}{} 130 | 131 | \end{problem} 132 | 133 | \begin{problem}{} 134 | \end{problem} 135 | 136 | \end{document} -------------------------------------------------------------------------------- /Fall18/hw/hw1.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/Fall18/hw/hw1.pdf -------------------------------------------------------------------------------- /Fall18/hw/hw2.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/Fall18/hw/hw2.pdf -------------------------------------------------------------------------------- /Fall18/hw/hw3.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/Fall18/hw/hw3.pdf -------------------------------------------------------------------------------- /Fall18/hw/hw4.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/Fall18/hw/hw4.pdf -------------------------------------------------------------------------------- /Fall18/hw/hw5.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/Fall18/hw/hw5.pdf -------------------------------------------------------------------------------- /Fall18/scribe/.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/Fall18/scribe/.DS_Store -------------------------------------------------------------------------------- /Fall18/scribe/CS7545scribe_template.tex: -------------------------------------------------------------------------------- 1 | %%%%%PLEASE CONSIDER CORRECTIONS AT PLACES INDICATED%%%%%%%% 2 | \documentclass{article} 3 | %%%%%Packages Used, add more if necessary%%%% 4 | \usepackage{amsmath,amsfonts,amssymb,graphicx,fullpage} 5 | \setlength{\topmargin}{-0.6 in} 6 | \setlength{\textheight}{8.5 in} 7 | \setlength{\headsep}{0.75 in} 8 | 9 | 10 | 11 | %%%%% NO NEED TO EDIT THIS PREAMBLE %%%%%%%% 12 | %%% PREAMBLE %%% 13 | \newcounter{lecnum} 14 | \renewcommand{\thepage}{\thelecnum-\arabic{page}} 15 | \renewcommand{\thesection}{\thelecnum.\arabic{section}} 16 | \renewcommand{\theequation}{\thelecnum.\arabic{equation}} 17 | \renewcommand{\thefigure}{\thelecnum.\arabic{figure}} 18 | \renewcommand{\thetable}{\thelecnum.\arabic{table}} 19 | \newcommand{\lecture}[4]{ 20 | \pagestyle{myheadings} 21 | \thispagestyle{plain} 22 | \newpage 23 | \setcounter{lecnum}{#1} 24 | \setcounter{page}{1} 25 | \noindent 26 | \begin{center} 27 | \framebox{ 28 | \vbox{\vspace{2mm} 29 | \hbox to 6.28in { {\bf CS 7545: Machine Learning Theory 30 | \hfill Fall 2018} } 31 | \vspace{4mm} 32 | \hbox to 6.28in { {\Large \hfill Lecture #1: #2 \hfill} } 33 | \vspace{2mm} 34 | \hbox to 6.28in { {\it Lecturer: #3 \hfill Scribes: #4} } 35 | \vspace{2mm}} 36 | } 37 | \end{center} 38 | \markboth{Lecture #1: #2}{Lecture #1: #2} 39 | 40 | 41 | {\bf Disclaimer}: {\it These notes have not been subjected to the 42 | usual scrutiny reserved for formal publications.} 43 | \vspace*{4mm} 44 | } 45 | \renewcommand{\cite}[1]{[#1]} 46 | \def\beginrefs{\begin{list} 47 | {[\arabic{equation}]}{\usecounter{equation} 48 | \setlength{\leftmargin}{2.0truecm}\setlength{\labelsep}{0.4truecm}% 49 | \setlength{\labelwidth}{1.6truecm}}} 50 | \def\endrefs{\end{list}} 51 | \def\bibentry#1{\item[\hbox{[#1]}]} 52 | 53 | \newcommand{\challenge}[2]{\noindent \textbf{(Challenge Problem)} \emph{#1}: #2 } 54 | 55 | \newcommand{\exercise}[1]{\noindent \textbf{(Exercise)} #1 } 56 | 57 | \newcommand{\fig}[3]{ 58 | \vspace{#2} 59 | \begin{center} 60 | Figure \thelecnum.#1:~#3 61 | \end{center} 62 | } 63 | %%% END_OF_PREAMBLE %%% 64 | 65 | 66 | 67 | %%%%You may add more \newtheorem if necessary%%%%%% 68 | \newtheorem{theorem}{Theorem}[lecnum] 69 | \newtheorem{lemma}[theorem]{Lemma} 70 | \newtheorem{proposition}[theorem]{Proposition} 71 | \newtheorem{claim}[theorem]{Claim} 72 | \newtheorem{corollary}[theorem]{Corollary} 73 | \newtheorem{definition}[theorem]{Definition} 74 | \newenvironment{proof}{{\bf Proof:}}{\hfill\rule{2mm}{2mm}} 75 | 76 | 77 | \newcommand{\dom}{\mathrm{dom}} 78 | \begin{document} 79 | 80 | %%%%%CHANGE HERE%%%%%%% 81 | %%%%%\section{title of the section} similarly with the rest \section{} or \subsection{} or \subsubsection{} etc 82 | 83 | 84 | %%%%%CHANGE HERE%%%%%%%%% 85 | %%%%%\lecture{the ordinal number of the lecture}{lecture title}{Jacob Abernethy}{scriber's name}%%%%%%%% 86 | \lecture{1}{TITLE OF LECTURE}{Jacob Abernethy}{NAME OF SCRIBE(S)} 87 | 88 | \section{Style Guide} 89 | \paragraph{Notations} 90 | 91 | \begin{itemize} 92 | \item Use \verb|\vec{x}| to denote an $n$-dimensional vector $\vec{x}$, and $x_i$ (not boldfaced) to denote its $i$-th coordinate of $\vec{x}$. 93 | \item Use an uppercase letter to denote a matrix: $M$. 94 | \item Use \verb|\top| for transposition: $\vec{x}^\top M \vec{x}$. 95 | \item Define new math operators in the preamble: \verb|\newcommand{\dom}{\mathrm{dom}}| and then use $\dom(f)$. 96 | \end{itemize} 97 | 98 | \paragraph{Styles} 99 | 100 | \begin{itemize} 101 | \item Please avoid using logic symbols (such as $\forall$, $\iff$, and $\implies$) as a part of your English sentence when you present a definition, statement (claim/proposition/theorem/lemma), or proof. Feel free to use them in the less ``formal'' settings. 102 | \item In a definition block, boldface the concept being defined. \end{itemize} 103 | Example: 104 | 105 | \begin{definition}[strongly convex] 106 | A differentiable function $f$ is \textbf{$c$-strongly convex} if for all $\vec{x},\vec{y} \in \dom(f)$, 107 | \[f(\vec{y}) \ge f(\vec{x}) + \nabla f(\vec{x})^T (\vec{y} - \vec{x}) + \frac{c}{2} \|\vec{y} - \vec{x}\|^2.\] 108 | \end{definition} 109 | %%%%%CHANGE HERE%%%%%%% 110 | %%%%%\section{title of the section} similarly with the rest \section{} or \subsection{} or \subsubsection{} etc 111 | \section{Title of the first section of the lecture} 112 | %%%%%Use itemize to layout bullet points that are not numbered%%%%% 113 | 114 | The professor said a few things. Then he wrote this on the board. This was followed by a list of things. 115 | \begin{itemize} 116 | \item This is the first thing in the list 117 | \item The following is a true statment: $1 + 1 = 2$ 118 | \item This sentence is false. (Or is it?) 119 | \item Now I'm going to use new notation. Let's let $n$ be an integer. Then notice that 120 | \[ 121 | \sum_{i=1}^\infty 1[n \geq i] = n 122 | \] 123 | where $1[]$ is the indicator function\footnote{I just used some new notation so I should define it here: 124 | $ 125 | 1[\textnormal{statement}] = \begin{cases} 126 | 127 | 1 & \mbox{if statement true;} \\ 128 | 129 | 0 & \mbox{if statement false} 130 | \end{cases} 131 | $.} 132 | \end{itemize} 133 | 134 | \subsection{Some special commands} 135 | 136 | \exercise{The professor mentioned an exercise in class that would be useful to work out. You can use the \texttt{$\backslash$exercise} command in these cases.} 137 | 138 | \challenge{Name of Challenge Problem}{If a challenge problem is given out, give it a name and put it in the previous field and then write down the description in this field.} 139 | 140 | \section{Theorems etc.} 141 | 142 | \begin{lemma} \label{lem:biglemma} 143 | We are going to need this result in a moment. 144 | \end{lemma} 145 | \begin{proof} 146 | I'm now proving the lemma. 147 | \end{proof} 148 | 149 | \begin{theorem} 150 | Here's a big statement 151 | \end{theorem} 152 | \begin{proof} 153 | Here's a proof of this big statement. It basically follows from Lemma~\ref{lem:biglemma}. 154 | \end{proof} 155 | 156 | \begin{corollary} 157 | This follows from the theorem, I swear. 158 | \end{corollary} 159 | 160 | 161 | \end{document} 162 | -------------------------------------------------------------------------------- /Fall18/scribe/lec1.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/Fall18/scribe/lec1.pdf -------------------------------------------------------------------------------- /Fall18/scribe/lec10.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/Fall18/scribe/lec10.pdf -------------------------------------------------------------------------------- /Fall18/scribe/lec11.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/Fall18/scribe/lec11.pdf -------------------------------------------------------------------------------- /Fall18/scribe/lec12.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/Fall18/scribe/lec12.pdf -------------------------------------------------------------------------------- /Fall18/scribe/lec13.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/Fall18/scribe/lec13.pdf -------------------------------------------------------------------------------- /Fall18/scribe/lec14.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/Fall18/scribe/lec14.pdf -------------------------------------------------------------------------------- /Fall18/scribe/lec15.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/Fall18/scribe/lec15.pdf -------------------------------------------------------------------------------- /Fall18/scribe/lec16.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/Fall18/scribe/lec16.pdf -------------------------------------------------------------------------------- /Fall18/scribe/lec17.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/Fall18/scribe/lec17.pdf -------------------------------------------------------------------------------- /Fall18/scribe/lec18.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/Fall18/scribe/lec18.pdf -------------------------------------------------------------------------------- /Fall18/scribe/lec19.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/Fall18/scribe/lec19.pdf -------------------------------------------------------------------------------- /Fall18/scribe/lec2.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/Fall18/scribe/lec2.pdf -------------------------------------------------------------------------------- /Fall18/scribe/lec20.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/Fall18/scribe/lec20.pdf -------------------------------------------------------------------------------- /Fall18/scribe/lec21.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/Fall18/scribe/lec21.pdf -------------------------------------------------------------------------------- /Fall18/scribe/lec22.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/Fall18/scribe/lec22.pdf -------------------------------------------------------------------------------- /Fall18/scribe/lec23.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/Fall18/scribe/lec23.pdf -------------------------------------------------------------------------------- /Fall18/scribe/lec24.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/Fall18/scribe/lec24.pdf -------------------------------------------------------------------------------- /Fall18/scribe/lec25.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/Fall18/scribe/lec25.pdf -------------------------------------------------------------------------------- /Fall18/scribe/lec26.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/Fall18/scribe/lec26.pdf -------------------------------------------------------------------------------- /Fall18/scribe/lec3.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/Fall18/scribe/lec3.pdf -------------------------------------------------------------------------------- /Fall18/scribe/lec4.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/Fall18/scribe/lec4.pdf -------------------------------------------------------------------------------- /Fall18/scribe/lec5.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/Fall18/scribe/lec5.pdf -------------------------------------------------------------------------------- /Fall18/scribe/lec6.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/Fall18/scribe/lec6.pdf -------------------------------------------------------------------------------- /Fall18/scribe/lec7.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/Fall18/scribe/lec7.pdf -------------------------------------------------------------------------------- /Fall18/scribe/lec8.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/Fall18/scribe/lec8.pdf -------------------------------------------------------------------------------- /Fall18/scribe/lec9.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/Fall18/scribe/lec9.pdf -------------------------------------------------------------------------------- /Fall19/README.md: -------------------------------------------------------------------------------- 1 | 2 | ### Course Information 3 | 4 | * **Course Info:** CS7545, Fall 2019 5 | * **Instructor:** Jacob Abernethy 6 | - **Office:** Coda S1221 7 | - **Email:** prof_at_gatech_dot_edu 8 | - **Office Hours:** Wednesdays, 10-11am, in Coda S1221 9 | * **Course Time&Place:** MW 4:30-5:45pm, Weber SST III (Lecture Hall 2) 10 | * **Teaching Assistants**: 11 | - *Bhuvesh Kumar* 12 | - **Email:** bhuvesh_at_gatech.edu 13 | - **Office Hours**: Thursdays 3:00-4:00pm, alcove of Klaus 2116/2124 14 | - *Zihao Hu* 15 | - **Email:** zihaohu_at_gatech.edu 16 | - **Office Hours**: Mondays 3:00-4:00pm, alcove of Klaus 2116/2124 17 | 18 | 19 | ### Course Description 20 | 21 | This course will study theoretical aspects of prediction and decision-making probelms, where our goal is to understand the mathematical underpinnings of machine learning. A primary objective of the class is to bring students to the frontiers of research and to prepare students to publish in this area. The course will cover, among other things, concentration inequalities, uniform deviation bounds, Vapnik-Chervonenkis Theory, Rademacher Complexity, margin bounds, boosting, some theoretical aspects of deep learning, online learning theory, regret minimization, multi-armed bandit algorithms, and connections to convex optimization. Along the way, we may dive into several related topics, including minimax equilibrium in games, calibration, sequential portfolio selection, option pricing, and differential privacy. 22 | 23 | **Prerequisites:** Familiarity with the analysis of algorithms, probabilistic analysis, and several similar topics. CS7641 (Machine Learning) will be quite helpful but not strictly necessary. The material is going to be about 90% "theory" and thus potential students must have a strong mathematical background. We shall rely heavily on techniques from calculus, probability, and convex analysis, but many tools will be presented in lecture. 24 | 25 | **Coursework:** There will be 5 problem sets throughout the semester. 26 | 27 | **Grade Breakdown:** 28 | * 50% - *Homeworks* 29 | * 40% - *Final Exam* 30 | * 10% - *Participation* 31 | 32 | **Note**: The final exam will be held on Wednesday, December 11, from 2:40-5:30pm. 33 | 34 | 35 | ### References: 36 | 37 | Roughly half of the course will follow material from the following text: 38 | 39 | * "[Foundations of Machine Learning](https://www.amazon.com/Foundations-Machine-Learning-Adaptive-Computation/dp/026201825X)" by Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar 40 | 41 | Much of the material in online learning (aka regret minimization) is of my own taste, and I will present these topics how I enjoy. But for students that want reading material on this topic, there are several surveys released in the last several years that explore several many that we shall cover. I will link to them here, and will mention them in various lectures when appropriate: 42 | 43 | * [The Multiplicative Weights Update Method](http://www.cs.princeton.edu/~arora/pubs/MWsurvey.pdf) by Sanjeev Arora, Elad Hazan, and Satyen Kale. 44 | * [Online Learning and Online Convex Optimization survey](http://www.cs.huji.ac.il/~shais/papers/OLsurvey.pdf) by Shai Shalev-Shwartz. 45 | * [The convex optimization approach to regret minimization survey](http://www.cs.princeton.edu/~ehazan/papers/OCO-survey.pdf) by Elad Hazan. 46 | * [Sasha Rakhlin's Lecture Notes](http://www-stat.wharton.upenn.edu/~rakhlin/courses/stat928/stat928_notes.pdf). 47 | 48 | 49 | ### Scribe Notes 50 | 51 | | Lecture | Date | Topic | 52 | | :------------: |:-------------: |:-------------: | 53 | | [1](./scribe/lec1.pdf) | 19 Aug 2019 | Introduction and and Linear Algebra Review | 54 | | [2](./scribe/lec2.pdf) | 21 Aug 2019 | Convex Analysis | 55 | | [3](./scribe/lec3.pdf) | 26 Aug 2019 | Convex Analysis and Deviation Bounds | 56 | | [4](./scribe/lec4.pdf) | 28 Aug 2019 | Chernoff Bounds | 57 | | [5](./scribe/lec5.pdf) | 04 Sep 2019 | Martingale and Online Learning Intro | 58 | | [6](./scribe/lec6.pdf) | 09 Sep 2019 | Weighted Majority Algorithm | 59 | | [7](./scribe/lec7.pdf) | 11 Sep 2019 | Exponential Weights Algorithm | 60 | | [8](./scribe/lec8.pdf) | 16 Sep 2019 | Perceptron and Game Theory Intro | 61 | | [9](./scribe/lec9.pdf) | 18 Sep 2019 | Game Theory and Boosting | 62 | | [10](./scribe/lec10.pdf) | 23 Sep 2019 | Boosting | 63 | | [11](./scribe/lec11.pdf) | 25 Sep 2019 | Online Convex Optimization | 64 | | [12](./scribe/lec12.pdf) | 30 Sep 2019 | Online Convex Optimization | 65 | | [13](./scribe/lec13.pdf) | 2 Oct 2019 | SGD and Mirror Descent | 66 | | [14](./scribe/lec14.pdf) | 7 Oct 2019 | Mirror Descent Continued | 67 | | [15](./scribe/lec15.pdf) | 9 Oct 2019 | FTRL, Multi-Armed Bandits, & EXP3 | 68 | | [16](./scribe/lec16.pdf) | 21 Oct 2019 | EXP3 and Stochastic Bandits | 69 | | [17](./scribe/lec17.pdf) | 23 Oct 2019 | Stochastic Bandits & UCB | 70 | | [18](./scribe/lec18.pdf) | 28 Oct 2019 | UCB Algorithm| 71 | | [19](./scribe/lec19.pdf) | 30 Oct 2019 | Stochastic Learning Theory| 72 | | [20](./scribe/lec20.pdf) | 04 Nov 2019 | VC Dimension & Rademacher complexity| 73 | | [21](./scribe/lec21.pdf) | 06 Nov 2019 | Rademacher complexity & Massart's Lemma| 74 | | [22](./scribe/lec22.pdf) | 11 Nov 2019 | Massart's Lemma & Sauer's Lemma| 75 | | [23](./scribe/lec23.pdf) | 13 Nov 2019 | Sauers's Lemma & VC Dim bounds| 76 | | [24](./scribe/lec24.pdf) | 18 Nov 2019 | Generalization Bounds & Neural Network| 77 | | [25](./scribe/lec25.pdf) | 20 Nov 2019 | Reinforcement Learning| 78 | | [26](./scribe/lec26.pdf) | 25 Nov 2019 | Margin Theory| 79 | 80 | [The Latex template for scribes is available here.](./scribe/CS7545scribe_template.tex) 81 | 82 | ### Homeworks 83 | 84 | | Homework | Due Date | 85 | | :------------: |:-------------: | 86 | | [1](./hw/hw1.pdf) | Sep 8 2019, 11:59 pm | 87 | | [2](./hw/hw2.pdf) | Sep 30 2019, 2:00 pm | 88 | | [3](./hw/hw3.pdf) | Oct 29 2019,11:59 pm | 89 | | [4](./hw/hw4.pdf) | Nov 14 2019,11:59 pm | 90 | | [5](./hw/hw5.pdf) | Dec 3 2019,11:59 pm | 91 | 92 | [The Latex template for HW submissions is available here.](./hw/CS7545hw_template.tex) 93 | 94 | Previous offerings of the course: [Fall 2018](./Fall18) 95 | 96 | -------------------------------------------------------------------------------- /Fall2023/README.md: -------------------------------------------------------------------------------- 1 | 3 | 4 | ### Course Information 5 | 6 | * **Course Info:** CS7545, Spring 2023 7 | * **Instructor:** 8 | * **Course Time&Place:** TuTh 3:30-4:45pm, Weber SST III (Lecture Hall 1) 9 | * **TA Office Hours:** We have office hours every Monday. Normally, we have one hour per week for the office hours (virtual office hours). In the homework submission weeks, we have three hours per week for the office hours (in-person office hours). 10 | - *Date: 1/23, 2/6, 2/13, 2/20, 3/6, 4/3, 4/10, 4/17* 11 | - Monday 7pm-8pm (virtual office hours) 12 | - *Date: 1/30, 2/27, 3/13, 3/27, 4/24* 13 | - Monday 2pm-5pm (in-person office hours), CODA 2nd floor 14 | - *This week's zoom link* 15 | - [https://gatech.zoom.us/j/97392962480?pwd=R3pWNVd4a2hTVFFkOElkMTR5blpOdz09](https://gatech.zoom.us/j/97392962480?pwd=R3pWNVd4a2hTVFFkOElkMTR5blpOdz09) 16 | * **Course Staff**: 17 | - *Jacob Abernethy* 18 | - **Office:** Coda S1221 19 | - **Email:** prof_at_gatech_dot_edu 20 | - **Office Hours:** TBD 21 | - *Zihao Hu* 22 | - **Email:** zihaohu_at_gatech_dot_edu 23 | - **Office Hours**: Monday 7pm-8pm(1/23, 4/3), Monday 2pm-5pm (1/30, 3/27) 24 | - *Yeojoon Youn* 25 | - **Email:** yjyoun92_at_gatech_dot_edu 26 | - **Office Hours**: Monday 7pm-8pm(2/6, 2/13, 2/20), Monday 2pm-5pm (2/27, 3/27) 27 | - *Guanghui Wang* 28 | - **Email:** gwang369_at_gatech_dot_edu 29 | - **Office Hours:** Monday 7pm-8pm(3/6, 4/10, 4/24), Monday 2pm-5pm (3/13, 3/27) 30 | - *Tyler Labonte* 31 | - **Email:** tlabonte_gatech_dot_edu 32 | 33 | ### Course Description 34 | 35 | This course will study theoretical aspects of prediction and decision-making probelms, and to explore the mathematical underpinnings of machine learning. We hope to bring students to the frontiers of research and to develop tools that can be used to contribute to emerging literature. The course will cover, among other things, concentration inequalities, uniform deviation bounds, Vapnik-Chervonenkis Theory, Rademacher Complexity, margin bounds, boosting, some theoretical aspects of deep learning, online learning theory, regret minimization, multi-armed bandit algorithms, and connections to convex optimization. Additional topics may be covered if time permits, including reinforcement learning theory, differential privacy, sampling, and and other areas of interest. 36 | 37 | **Prerequisites:** Familiarity with the analysis of algorithms, probabilistic analysis, and several similar topics. CS7641 (Machine Learning) may be helpful but not strictly necessary. The material will be about 90% "theory" and thus students must have a strong mathematical background. We shall rely heavily on techniques from calculus, probability, and convex analysis, but many tools will be reviewed in lecture. 38 | 39 | **Coursework:** There will be 4 problem sets throughout the semester. 40 | 41 | **Grade Breakdown:** 42 | * 40% - *Homeworks* 43 | * 30% - *Exam* 44 | * 20% - *Final project* 45 | * 10% - *Scribing/participation* 46 | 47 | **Note**: The exam will be held on Thursday March 30 during classtime. 48 | 49 | ### Lecture notes 50 | 51 | Every lecture will have two student scribes, and this pair of students will be asked to take detailed notes in class on what was covered. The quality of these notes will be graded! 52 | 53 | **Grade Rubric:** 54 | * 20% - *punctuality* 55 | * 40% - *correctness* 56 | * 40% - *readability* 57 | 58 | [Here is the course wiki](https://github.com/mltheory/CS7545/wiki). 59 | 60 | Note: all students are allowed and encouraged to contribute to this wiki! Once the scribes have submitted their notes, other students are welcome to add any additional comments, results, material, etc. to the lecture notes. It would be great if the wiki provides a broad set of resources for students, beyond what was covered in lecture. 61 | 62 | 63 | ### References: 64 | 65 | Roughly half of the course will follow material from the following text: 66 | 67 | * "[Foundations of Machine Learning](https://www.amazon.com/Foundations-Machine-Learning-Adaptive-Computation/dp/026201825X)" by Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar 68 | * "[Understanding Machine Learning: From Theory to Algorithms](https://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning/understanding-machine-learning-theory-algorithms.pdf)" by Shai Shalev-Shwartz and Shai Ben-David 69 | 70 | Much of the material in online learning is specific to this course. For students that want more in-depth reading material on this topic, however, there are several surveys released in the last several years that explore several many areas we shall cover. These include: 71 | 72 | * [The Multiplicative Weights Update Method](http://www.cs.princeton.edu/~arora/pubs/MWsurvey.pdf) by Sanjeev Arora, Elad Hazan, and Satyen Kale. 73 | * [Online Learning and Online Convex Optimization survey](http://www.cs.huji.ac.il/~shais/papers/OLsurvey.pdf) by Shai Shalev-Shwartz. 74 | * [The convex optimization approach to regret minimization survey](http://www.cs.princeton.edu/~ehazan/papers/OCO-survey.pdf) by Elad Hazan. 75 | * [Sasha Rakhlin's Lecture Notes](http://www-stat.wharton.upenn.edu/~rakhlin/courses/stat928/stat928_notes.pdf). 76 | 77 | [The Latex template for HW submissions is available here.](./hw/CS7545hw_template.tex) 78 | 79 | Previous offerings of the course: 80 | - [Fall 2018](./Fall18) 81 | - [Fall 2019](./Fall19) 82 | 83 | ### Homeworks 84 | 85 | | Homework | Due Date | Solution | 86 | | :------------: |:-------------: |:-------------: | 87 | | [Homework 1](./hw/CS7545_HW1.pdf) | January 31, 2023, 11:59 pm | [HW1 solution](./hw/CS7545_HW1_sol.pdf) | 88 | | [Homework 2](./hw/CS7545_HW2.pdf) | February 28, 2023, 11:59 pm | [HW2 solution](./hw/CS7545_HW2_sol.pdf) | 89 | | [Homework 3](./hw/CS7545_HW3.pdf) | March 19, 2023, 11:59 pm | [HW3 solution](./hw/CS7545_HW3_sol.pdf) | 90 | | [Homework 4](./hw/CS7545_HW4.pdf) | April 25, 2023, 11:59 pm | [HW4 solution](./hw/CS7545_HW4_sol.pdf) | 91 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 3 | 4 | ### Course Information 5 | 6 | * **Course Info:** CS7545, Spring 2024 7 | * **Course Time&Place:** TuTh 3:30-4:45pm, Weber SST III (Lecture Hall 1) 8 | * **Course Staff**: 9 | - *Jacob Abernethy* 10 | - **Office:** Coda S1221 11 | - **Email:** prof_at_gatech_dot_edu 12 | - **Office Hours:** Friday 2pm-3pm(1/19, 1/26, 2/2) 13 | - *Tyler Labonte* 14 | - **Email:** tlabonte_gatech_dot_edu 15 | - **Office Hours:** Friday 3pm-5pm(2/16, 2/23, 3/1) 16 | - *Guanghui Wang* 17 | - **Email:** gwang369_at_gatech_dot_edu 18 | - **Office Hours:** Friday 2pm-4pm(3/8, 3/15, 3/29, 4/5) 19 | - *Yeojoon Youn* 20 | - **Email:** yjyoun92_at_gatech_dot_edu 21 | - **Office Hours**: Friday 3pm-4pm(1/19, 1/26), Friday 9am-11am(2/2), Friday 3pm-5pm(2/9) 22 | 23 | ### Course Description 24 | 25 | This course will study theoretical aspects of prediction and decision-making probelms, and to explore the mathematical underpinnings of machine learning. We hope to bring students to the frontiers of research and to develop tools that can be used to contribute to emerging literature. The course will cover, among other things, concentration inequalities, uniform deviation bounds, Vapnik-Chervonenkis Theory, Rademacher Complexity, boosting, some theoretical aspects of deep learning, online learning theory, regret minimization, multi-armed bandit algorithms, and connections to convex optimization. Additional topics may be covered if time permits, including reinforcement learning theory, federated learning, etc. 26 | 27 | **Prerequisites:** Familiarity with the analysis of algorithms, probabilistic analysis, and several similar topics. CS7641 (Machine Learning) may be helpful but not strictly necessary. The material will be about 90% "theory" and thus students must have a strong mathematical background. We shall rely heavily on techniques from calculus, probability, and convex analysis, but many tools will be reviewed in lecture. 28 | 29 | **Grade Breakdown:** 30 | * 60% *Exams* (in-class!) 31 | * 30% *Homeworks* 32 | * 10% *Scribing/participation* 33 | 34 | ### Three Topic Segments, Three Exams 35 | 36 | The course instruction will be divided into three segments, with a particular area of focus, a different lecturer, and will end with an exam. The agenda will go as follows. 37 | 38 | - **Segment 1** 39 | - *Topic.* Basic tools for learning: linear algebra, convex analysis, probability, deviation bounds, martingales. 40 | - *Instructor.* Prof. Jake Abernethy 41 | - *Exam.* In class on Tuesday ~~1/30/2024~~ 2/6/2024 42 | - **Segment 2** 43 | - *Topic.* Statistical learning theory: generalization bounds, uniform convergence, VC theory, Rademacher complexity. 44 | - *Instructor.* Tyler Labonte 45 | - *Exam.* In class on Tuesday ~~2/27/2024~~ 3/5/2024 46 | - **Segment 3** 47 | - *Topic.* Online learning and optimization: sequential learning framework, regret minimization, online convex optimization, multi-armed bandits. 48 | - *Instructor.* Guanghui Wang 49 | - *Exam.* In class on Tuesday ~~4/2/2024~~ 4/9/2024 50 | 51 | 52 | Note: this will leave around 5-6 lectures at the end of the course. We will use this as "extra-curriculur" time and we may have a set of guest lectures on special topics. 53 | 54 | ### Homework 55 | 56 | We're going to try something different this semester: one homework problem per lecture! 57 | 58 | During lectures, the presenting instructor will state one or more problems, with some of these clearly marked **HOMEWORK**. They will also be added to the lecture notes on the course wiki. The goal of these problems is to get students to engage with the material, and to make sure they understand the course material. Our homework policy is as follows: 59 | - We **encourage**, but do not require, students to complete the problems within the week they are given out. Doing small bits of work to keep up with the course material is the best way to learn the material gradually. 60 | - We **require** all of the homework problems for each segment of the course to be completed *four days before the exam*. We will release solutions at this time, and homework will not be accepted later than this date. 61 | - While exams will be graded in detail, homework will be **lightly graded** according to this rubric: 62 | - 0 points for no submission 63 | - 1 point for any attempt at the problem 64 | - 2 points for a correct, or even mostly-correct, answer 65 | - We **strongly encourage** students to use LaTeX to typeset solutions. Handwritten solutions are acceptable but discouraged. 66 | - We will soon post details on Piazza for how to submit homework solutions. 67 | 68 | ### Lecture notes 69 | 70 | Every lecture will have two student scribes, and this pair of students will be asked to take detailed notes in class on what was covered. The quality of these notes will be graded! 71 | 72 | **Grade Rubric:** 73 | * 20% - *punctuality* 74 | * 40% - *correctness* 75 | * 40% - *readability* 76 | 77 | [Here is the course wiki](https://github.com/mltheory/CS7545/wiki). 78 | 79 | Note: all students are allowed and encouraged to contribute to this wiki! Once the scribes have submitted their notes, other students are welcome to add any additional comments, results, material, etc. to the lecture notes. It would be great if the wiki provides a broad set of resources for students, beyond what was covered in lecture. 80 | 81 | 82 | ### References: 83 | 84 | The bulk of material from the course will follow material from the following texts: 85 | 86 | * "[Foundations of Machine Learning](https://www.amazon.com/Foundations-Machine-Learning-Adaptive-Computation/dp/026201825X)" by Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar 87 | * [The convex optimization approach to regret minimization survey](https://arxiv.org/abs/1909.05207) by Elad Hazan. 88 | 89 | Much of the material in online learning is specific to this course. For students that want more in-depth reading material on this topic, however, there are several surveys released in the last several years that explore several many areas we shall cover. These include: 90 | 91 | * [Understanding Machine Learning: From Theory to Algorithms](https://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning/understanding-machine-learning-theory-algorithms.pdf) by Shai Shalev-Shwartz and Shai Ben-David 92 | * [A Modern Introduction to Online Learning](https://arxiv.org/abs/1912.13213) by Francesco Orabona 93 | * [Online Learning and Online Convex Optimization survey](http://www.cs.huji.ac.il/~shais/papers/OLsurvey.pdf) by Shai Shalev-Shwartz. 94 | * [Sasha Rakhlin's Lecture Notes](http://www-stat.wharton.upenn.edu/~rakhlin/courses/stat928/stat928_notes.pdf). 95 | 96 | [The Latex template for HW submissions is available here.](./hw/CS7545hw_template.tex) 97 | 98 | Previous offerings of the course: 99 | - [Fall 2018](./Fall18) 100 | - [Fall 2019](./Fall19) 101 | - [Fall 2023](./Fall23) 102 | 103 | ### Homeworks 104 | 105 | | Homework | Due Date | Solution | 106 | | :------------: |:-------------: |:-------------: | 107 | | [Homework 1](./hw/CS7545_HW1.pdf) | February 2, 2024, 11:59 pm | [HW1 solution](./hw/CS7545_HW1_sol.pdf) | 108 | | [Homework 2](./hw/CS7545_HW2.pdf) | March 1, 2024, 11:59 pm | [HW2 solution](./hw/CS7545_HW2_sol.pdf) | 109 | | [Homework 3](./hw/HW3-Problem.pdf) | April 5, 2024, 11:59 pm | [HW3 solution](./hw/HW3-Solution.pdf) | 110 | -------------------------------------------------------------------------------- /_config.yml: -------------------------------------------------------------------------------- 1 | remote_theme: pages-themes/hacker@v0.2.0 2 | plugins: 3 | - jekyll-remote-theme 4 | 5 | title: Machine Learning Theory 6 | description: CS 7545 -- Georgia Tech 7 | -------------------------------------------------------------------------------- /_layouts/default.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | {% include head-custom.html %} 9 | 10 | {% seo %} 11 | 12 | 13 | 14 | 15 |
16 |
17 | 18 |

{{ site.title | default: site.github.repository_name }}

19 | 20 |

{{ site.description | default: site.github.project_tagline }}

21 | 22 |
23 | {% if site.show_downloads %} 24 | Download as .zip 25 | Download as .tar.gz 26 | {% endif %} 27 | View on GitHub 28 |
29 |
30 |
31 | 32 |
33 |
34 | {{ content }} 35 |
36 |
37 | 38 | -------------------------------------------------------------------------------- /_layouts/default.html.old: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | {% seo %} 13 | 14 | 15 | 16 | 17 | 18 |
19 |
20 | 21 | 22 |

{{ site.title | default: site.github.repository_name }}

23 |

{{ site.description | default: site.github.project_tagline }}

24 | 25 | {% if site.show_downloads %} 26 |
27 | Download this project as a .zip file 28 | Download this project as a tar.gz file 29 |
30 | {% endif %} 31 |
32 |
33 | 34 | 35 |
36 |
37 | {{ content }} 38 |
39 |
40 | 41 | 42 | 49 | 50 | {% if site.google_analytics %} 51 | 59 | {% endif %} 60 | 61 | 62 | -------------------------------------------------------------------------------- /exam/CS7545_Spring_2023_PracticeExam.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/exam/CS7545_Spring_2023_PracticeExam.pdf -------------------------------------------------------------------------------- /exam/CS7545_practice_exam_sol.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/exam/CS7545_practice_exam_sol.pdf -------------------------------------------------------------------------------- /hw/.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/hw/.DS_Store -------------------------------------------------------------------------------- /hw/CS7545_HW1.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/hw/CS7545_HW1.pdf -------------------------------------------------------------------------------- /hw/CS7545_HW1_sol.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/hw/CS7545_HW1_sol.pdf -------------------------------------------------------------------------------- /hw/CS7545_HW2.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/hw/CS7545_HW2.pdf -------------------------------------------------------------------------------- /hw/CS7545_HW2_sol.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/hw/CS7545_HW2_sol.pdf -------------------------------------------------------------------------------- /hw/CS7545hw_template.tex: -------------------------------------------------------------------------------- 1 | \documentclass{article} 2 | \usepackage[T1]{fontenc} 3 | \usepackage{amssymb, amsmath, graphicx, subfigure, enumerate} 4 | \usepackage{amsthm,alltt} 5 | \usepackage[margin=1.25in]{geometry} %geometry (sets margin) and other useful packages 6 | \usepackage{graphicx,ctable,booktabs} 7 | \usepackage{mathtools} 8 | \usepackage[boxed]{algorithm2e} 9 | \usepackage{fancyhdr} %Fancy-header package to modify header/page numbering 10 | \usepackage{cleveref} 11 | 12 | \setlength{\oddsidemargin}{.25in} 13 | \setlength{\evensidemargin}{.25in} 14 | \setlength{\textwidth}{6in} 15 | \setlength{\topmargin}{-0.4in} 16 | \setlength{\textheight}{8.5in} 17 | 18 | \newcommand{\heading}[6]{ 19 | \renewcommand{\thepage}{\arabic{page}} % used to be #1-\arabic{page} 20 | \noindent 21 | \begin{center} 22 | \framebox{ 23 | \vbox{ 24 | \hbox to 5.78in { \textbf{#2} \hfill #3 } 25 | \vspace{4mm} 26 | \hbox to 5.78in { {\Large \hfill #6 \hfill} } 27 | \vspace{2mm} 28 | \hbox to 5.78in { \textit{Instructor: #4 \hfill #5} } 29 | } 30 | } 31 | \end{center} 32 | \vspace*{4mm} 33 | } 34 | 35 | %Redefining sections as problems 36 | \makeatletter 37 | \newenvironment{problem}{\@startsection 38 | {section} 39 | {2} 40 | {-.2em} 41 | {-3.5ex plus -1ex minus -.2ex} 42 | {2.3ex plus .2ex} 43 | {\pagebreak[3]%forces pagebreak when space is small; use \eject for better results 44 | \large\bf\noindent{Problem } 45 | } 46 | } 47 | %{%\vspace{1ex}\begin{center} \rule{0.3\linewidth}{.3pt}\end{center}} 48 | %\begin{center}\large\bf \ldots\ldots\ldots\end{center}} 49 | \makeatother 50 | 51 | 52 | \newtheorem{theorem}{Theorem}[section] 53 | \newtheorem{definition}[theorem]{Definition} 54 | \newtheorem{remark}[theorem]{Remark} 55 | \newtheorem{lemma}[theorem]{Lemma} 56 | \newtheorem{corollary}[theorem]{Corollary} 57 | \newtheorem{proposition}[theorem]{Proposition} 58 | \newtheorem{claim}[theorem]{Claim} 59 | \newtheorem{observation}[theorem]{Observation} 60 | \newtheorem{fact}[theorem]{Fact} 61 | \newtheorem{assumption}[theorem]{Assumption} 62 | 63 | % don't modify this unless you know what you're doing 64 | \newcommand{\problemset}[3]{\heading{#1}{\classname: Machine Learning Theory}{#2}{Jake Abernethy}{#3}{Problem Set #1}} 65 | 66 | 67 | %%%%%%%% ENTER YOUR INFORMATION HERE %%%%%%%% 68 | \newcommand{\problemsetnum}{1} % problem set number 69 | \newcommand{\duedate}{September 10, 2018} % problem set deadline 70 | \newcommand{\studentname}{STUDENT NAME} % name of student (i.e., you) 71 | \newcommand{\classname}{CS 7545} 72 | %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 73 | 74 | \pagestyle{fancy} 75 | %\addtolength{\headwidth}{\marginparsep} %these change header-rule width 76 | %\addtolength{\headwidth}{\marginparwidth} 77 | \lhead{\classname} %Problem \thesection} 78 | \chead{} 79 | \rhead{\thepage} 80 | \lfoot{\small\scshape \classname} 81 | \cfoot{} 82 | \rfoot{\footnotesize PS \#\problemsetnum} 83 | \renewcommand{\headrulewidth}{.3pt} 84 | \renewcommand{\footrulewidth}{.3pt} 85 | \setlength\voffset{-0.25in} 86 | \setlength\textheight{648pt} 87 | 88 | 89 | \newcommand{\sit}{\shortintertext} 90 | \newcommand\deq{\mathrel{\overset{\makebox[0pt]{\mbox{\normalfont\tiny\sffamily def}}}{=}}} 91 | \newcommand{\ones}{\mathbbm{1}} 92 | \newcommand{\e}{\vec{e}} 93 | \newcommand{\tr}{\text{tr}} 94 | \newcommand{\bs}{\boldsymbol} 95 | \mathchardef\mhyphen="2D 96 | \newcommand{\C}{\mathbb{C}} 97 | \newcommand{\R}{\mathbb{R}} 98 | 99 | \newcommand{\vol}{\text{vol}} 100 | 101 | \renewcommand{\thesubsection}{\thesection.\alph{subsection}} 102 | 103 | 104 | % auto sized delimiters 105 | 106 | \newcommand{\br}[1]{\left[#1\right]} 107 | \newcommand{\pr}[1]{\left(#1\right)} 108 | \newcommand{\ceil}[1]{\left\lceil#1\right\rceil} 109 | \newcommand{\floor}[1]{\left\lfloor#1\right\rfloor} 110 | \newcommand{\abs}[1]{\left|#1\right|} 111 | 112 | %default delimiter for Pr and E 113 | \DeclarePairedDelimiter{\defaultDelim}{[}{]} 114 | 115 | \DeclareMathOperator{\capPr}{Pr} 116 | \renewcommand{\Pr}[2][]{\capPr_{#1}\defaultDelim*{#2}} 117 | \DeclareMathOperator{\capE}{E} 118 | \newcommand{\E}[2][]{\capE_{#1}\defaultDelim*{#2}} 119 | \DeclareMathOperator{\capVar}{Var} 120 | \newcommand{\Var}[2][]{\capVar_{#1}\defaultDelim*{#2}} 121 | 122 | %\DeclareMathOperator*{}{} puts subscript below 123 | 124 | 125 | %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 126 | \begin{document} 127 | \problemset{\problemsetnum}{\duedate}{\studentname} 128 | 129 | \begin{problem}{} 130 | 131 | \end{problem} 132 | 133 | \begin{problem}{} 134 | \end{problem} 135 | 136 | \end{document} -------------------------------------------------------------------------------- /hw/HW3-Problem.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/hw/HW3-Problem.pdf -------------------------------------------------------------------------------- /hw/HW3-Solution.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/hw/HW3-Solution.pdf -------------------------------------------------------------------------------- /robot_blackboard.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/robot_blackboard.png -------------------------------------------------------------------------------- /robot_blackboard_2024.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/robot_blackboard_2024.png -------------------------------------------------------------------------------- /scribe/.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/scribe/.DS_Store -------------------------------------------------------------------------------- /scribe/CS7545scribe_template.tex: -------------------------------------------------------------------------------- 1 | %%%%%PLEASE CONSIDER CORRECTIONS AT PLACES INDICATED%%%%%%%% 2 | \documentclass{article} 3 | %%%%%Packages Used, add more if necessary%%%% 4 | \usepackage{amsmath,amsfonts,amssymb,graphicx,fullpage} 5 | \setlength{\topmargin}{-0.6 in} 6 | \setlength{\textheight}{8.5 in} 7 | \setlength{\headsep}{0.75 in} 8 | 9 | 10 | 11 | %%%%% NO NEED TO EDIT THIS PREAMBLE %%%%%%%% 12 | %%% PREAMBLE %%% 13 | \newcounter{lecnum} 14 | \renewcommand{\thepage}{\thelecnum-\arabic{page}} 15 | \renewcommand{\thesection}{\thelecnum.\arabic{section}} 16 | \renewcommand{\theequation}{\thelecnum.\arabic{equation}} 17 | \renewcommand{\thefigure}{\thelecnum.\arabic{figure}} 18 | \renewcommand{\thetable}{\thelecnum.\arabic{table}} 19 | \newcommand{\lecture}[4]{ 20 | \pagestyle{myheadings} 21 | \thispagestyle{plain} 22 | \newpage 23 | \setcounter{lecnum}{#1} 24 | \setcounter{page}{1} 25 | \noindent 26 | \begin{center} 27 | \framebox{ 28 | \vbox{\vspace{2mm} 29 | \hbox to 6.28in { {\bf CS 7545: Machine Learning Theory 30 | \hfill Fall 2019} } 31 | \vspace{4mm} 32 | \hbox to 6.28in { {\Large \hfill Lecture #1: #2 \hfill} } 33 | \vspace{2mm} 34 | \hbox to 6.28in { {\it Lecturer: #3 \hfill Scribes: #4} } 35 | \vspace{2mm}} 36 | } 37 | \end{center} 38 | \markboth{Lecture #1: #2}{Lecture #1: #2} 39 | 40 | 41 | {\bf Disclaimer}: {\it These notes have not been subjected to the 42 | usual scrutiny reserved for formal publications.} 43 | \vspace*{4mm} 44 | } 45 | \renewcommand{\cite}[1]{[#1]} 46 | \def\beginrefs{\begin{list} 47 | {[\arabic{equation}]}{\usecounter{equation} 48 | \setlength{\leftmargin}{2.0truecm}\setlength{\labelsep}{0.4truecm}% 49 | \setlength{\labelwidth}{1.6truecm}}} 50 | \def\endrefs{\end{list}} 51 | \def\bibentry#1{\item[\hbox{[#1]}]} 52 | 53 | \newcommand{\challenge}[2]{\noindent \textbf{(Challenge Problem)} \emph{#1}: #2 } 54 | 55 | \newcommand{\exercise}[1]{\noindent \textbf{(Exercise)} #1 } 56 | 57 | \newcommand{\fig}[3]{ 58 | \vspace{#2} 59 | \begin{center} 60 | Figure \thelecnum.#1:~#3 61 | \end{center} 62 | } 63 | %%% END_OF_PREAMBLE %%% 64 | 65 | 66 | 67 | %%%%You may add more \newtheorem if necessary%%%%%% 68 | \newtheorem{theorem}{Theorem}[lecnum] 69 | \newtheorem{lemma}[theorem]{Lemma} 70 | \newtheorem{proposition}[theorem]{Proposition} 71 | \newtheorem{claim}[theorem]{Claim} 72 | \newtheorem{corollary}[theorem]{Corollary} 73 | \newtheorem{definition}[theorem]{Definition} 74 | \newenvironment{proof}{{\bf Proof:}}{\hfill\rule{2mm}{2mm}} 75 | 76 | 77 | \newcommand{\dom}{\mathrm{dom}} 78 | \begin{document} 79 | 80 | %%%%%CHANGE HERE%%%%%%% 81 | %%%%%\section{title of the section} similarly with the rest \section{} or \subsection{} or \subsubsection{} etc 82 | 83 | 84 | %%%%%CHANGE HERE%%%%%%%%% 85 | %%%%%\lecture{the ordinal number of the lecture}{lecture title}{Jacob Abernethy}{scriber's name}%%%%%%%% 86 | \lecture{1}{TITLE OF LECTURE}{Jacob Abernethy}{NAME OF SCRIBE(S)} 87 | 88 | \section{Style Guide} 89 | \paragraph{Notations} 90 | 91 | \begin{itemize} 92 | \item Use \verb|\vec{x}| to denote an $n$-dimensional vector $\vec{x}$, and $x_i$ (not boldfaced) to denote its $i$-th coordinate of $\vec{x}$. 93 | \item Use an uppercase letter to denote a matrix: $M$. 94 | \item Use \verb|\top| for transposition: $\vec{x}^\top M \vec{x}$. 95 | \item Define new math operators in the preamble: \verb|\newcommand{\dom}{\mathrm{dom}}| and then use $\dom(f)$. 96 | \end{itemize} 97 | 98 | \paragraph{Styles} 99 | 100 | \begin{itemize} 101 | \item Please avoid using logic symbols (such as $\forall$, $\iff$, and $\implies$) as a part of your English sentence when you present a definition, statement (claim/proposition/theorem/lemma), or proof. Feel free to use them in the less ``formal'' settings. 102 | \item In a definition block, boldface the concept being defined. \end{itemize} 103 | Example: 104 | 105 | \begin{definition}[strongly convex] 106 | A differentiable function $f$ is \textbf{$c$-strongly convex} if for all $\vec{x},\vec{y} \in \dom(f)$, 107 | \[f(\vec{y}) \ge f(\vec{x}) + \nabla f(\vec{x})^T (\vec{y} - \vec{x}) + \frac{c}{2} \|\vec{y} - \vec{x}\|^2.\] 108 | \end{definition} 109 | %%%%%CHANGE HERE%%%%%%% 110 | %%%%%\section{title of the section} similarly with the rest \section{} or \subsection{} or \subsubsection{} etc 111 | \section{Title of the first section of the lecture} 112 | %%%%%Use itemize to layout bullet points that are not numbered%%%%% 113 | 114 | The professor said a few things. Then he wrote this on the board. This was followed by a list of things. 115 | \begin{itemize} 116 | \item This is the first thing in the list 117 | \item The following is a true statment: $1 + 1 = 2$ 118 | \item This sentence is false. (Or is it?) 119 | \item Now I'm going to use new notation. Let's let $n$ be an integer. Then notice that 120 | \[ 121 | \sum_{i=1}^\infty 1[n \geq i] = n 122 | \] 123 | where $1[]$ is the indicator function\footnote{I just used some new notation so I should define it here: 124 | $ 125 | 1[\textnormal{statement}] = \begin{cases} 126 | 127 | 1 & \mbox{if statement true;} \\ 128 | 129 | 0 & \mbox{if statement false} 130 | \end{cases} 131 | $.} 132 | \end{itemize} 133 | 134 | \subsection{Some special commands} 135 | 136 | \exercise{The professor mentioned an exercise in class that would be useful to work out. You can use the \texttt{$\backslash$exercise} command in these cases.} 137 | 138 | \challenge{Name of Challenge Problem}{If a challenge problem is given out, give it a name and put it in the previous field and then write down the description in this field.} 139 | 140 | \section{Theorems etc.} 141 | 142 | \begin{lemma} \label{lem:biglemma} 143 | We are going to need this result in a moment. 144 | \end{lemma} 145 | \begin{proof} 146 | I'm now proving the lemma. 147 | \end{proof} 148 | 149 | \begin{theorem} 150 | Here's a big statement 151 | \end{theorem} 152 | \begin{proof} 153 | Here's a proof of this big statement. It basically follows from Lemma~\ref{lem:biglemma}. 154 | \end{proof} 155 | 156 | \begin{corollary} 157 | This follows from the theorem, I swear. 158 | \end{corollary} 159 | 160 | 161 | \end{document} 162 | -------------------------------------------------------------------------------- /scribe/hu.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/scribe/hu.pdf -------------------------------------------------------------------------------- /scribe/lec1.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/scribe/lec1.pdf -------------------------------------------------------------------------------- /scribe/lec10.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/scribe/lec10.pdf -------------------------------------------------------------------------------- /scribe/lec11.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/scribe/lec11.pdf -------------------------------------------------------------------------------- /scribe/lec12.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/scribe/lec12.pdf -------------------------------------------------------------------------------- /scribe/lec13.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/scribe/lec13.pdf -------------------------------------------------------------------------------- /scribe/lec14.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/scribe/lec14.pdf -------------------------------------------------------------------------------- /scribe/lec15.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/scribe/lec15.pdf -------------------------------------------------------------------------------- /scribe/lec16.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/scribe/lec16.pdf -------------------------------------------------------------------------------- /scribe/lec17.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/scribe/lec17.pdf -------------------------------------------------------------------------------- /scribe/lec18.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/scribe/lec18.pdf -------------------------------------------------------------------------------- /scribe/lec19.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/scribe/lec19.pdf -------------------------------------------------------------------------------- /scribe/lec2.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/scribe/lec2.pdf -------------------------------------------------------------------------------- /scribe/lec20.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/scribe/lec20.pdf -------------------------------------------------------------------------------- /scribe/lec21.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/scribe/lec21.pdf -------------------------------------------------------------------------------- /scribe/lec22.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/scribe/lec22.pdf -------------------------------------------------------------------------------- /scribe/lec23.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/scribe/lec23.pdf -------------------------------------------------------------------------------- /scribe/lec24.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/scribe/lec24.pdf -------------------------------------------------------------------------------- /scribe/lec25.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/scribe/lec25.pdf -------------------------------------------------------------------------------- /scribe/lec26.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/scribe/lec26.pdf -------------------------------------------------------------------------------- /scribe/lec3.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/scribe/lec3.pdf -------------------------------------------------------------------------------- /scribe/lec4.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/scribe/lec4.pdf -------------------------------------------------------------------------------- /scribe/lec5.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/scribe/lec5.pdf -------------------------------------------------------------------------------- /scribe/lec6.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/scribe/lec6.pdf -------------------------------------------------------------------------------- /scribe/lec7.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/scribe/lec7.pdf -------------------------------------------------------------------------------- /scribe/lec8.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/scribe/lec8.pdf -------------------------------------------------------------------------------- /scribe/lec9.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/scribe/lec9.pdf -------------------------------------------------------------------------------- /scribe/wlec21.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mltheory/CS7545/3958607a119a7e603cad0a264fab37ad195ac37d/scribe/wlec21.pdf --------------------------------------------------------------------------------