├── .gitattributes ├── Lecture_4 ├── PFI.pdf └── PFI.tex ├── Lecture_1 ├── Basics.pdf ├── Nelder_Mead.png └── Basics.tex ├── Lecture_5 ├── Distribution.pdf └── Distribution.tex ├── Lecture_6 ├── Perturbation.pdf └── Perturbation.tex ├── Lecture_2 ├── VFI_discretization.pdf └── VFI_discretization.tex ├── Lecture_7 ├── HeterogeneousAgents.pdf └── HeterogeneousAgents.tex ├── Lecture_3 ├── Global_Approximation.pdf ├── Global_Approximation.tex └── linear_spline.eps ├── .gitignore ├── README.md └── LICENSE /.gitattributes: -------------------------------------------------------------------------------- 1 | # Auto detect text files and perform LF normalization 2 | * text=auto 3 | -------------------------------------------------------------------------------- /Lecture_4/PFI.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/wupeifan/NumericalMethod_Notes/HEAD/Lecture_4/PFI.pdf -------------------------------------------------------------------------------- /Lecture_1/Basics.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/wupeifan/NumericalMethod_Notes/HEAD/Lecture_1/Basics.pdf -------------------------------------------------------------------------------- /Lecture_1/Nelder_Mead.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/wupeifan/NumericalMethod_Notes/HEAD/Lecture_1/Nelder_Mead.png -------------------------------------------------------------------------------- /Lecture_5/Distribution.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/wupeifan/NumericalMethod_Notes/HEAD/Lecture_5/Distribution.pdf -------------------------------------------------------------------------------- /Lecture_6/Perturbation.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/wupeifan/NumericalMethod_Notes/HEAD/Lecture_6/Perturbation.pdf -------------------------------------------------------------------------------- /Lecture_2/VFI_discretization.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/wupeifan/NumericalMethod_Notes/HEAD/Lecture_2/VFI_discretization.pdf -------------------------------------------------------------------------------- /Lecture_7/HeterogeneousAgents.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/wupeifan/NumericalMethod_Notes/HEAD/Lecture_7/HeterogeneousAgents.pdf -------------------------------------------------------------------------------- /Lecture_3/Global_Approximation.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/wupeifan/NumericalMethod_Notes/HEAD/Lecture_3/Global_Approximation.pdf -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # LaTeX temporary files 2 | *.aux 3 | *.log 4 | *.toc 5 | 6 | # Latexmk 7 | *.fdb_latexmk 8 | 9 | # SyncTeX 10 | *.synctex.gz 11 | 12 | # LaTeX Beamer 13 | *.snm 14 | *.vrb 15 | *.nav 16 | *.out 17 | *.fls 18 | 19 | # BibTeX 20 | *.bbl 21 | *.blg 22 | 23 | # Lyx 24 | *.lyx 25 | *.lyx~ 26 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Notes for Numerical Methods in Quantitative Macro 2 | Slides prepared for teaching numerical methods in quantitative macroeconomics, especially for heterogeneous agent models in general equilibrium with aggregate shocks. 3 | **WARNING: IN PROGRESS** 4 | 5 | There seems to be an ongoing interest in this set of lecture notes, but please be aware that they are incomplete. I'll have them updated in a few months. 6 | 7 | ## Contents 8 | 1. Root-finding, Optimization Methods, Numerical Integration 9 | 1. Value Function Iteration with Discretization 10 | 1. Global Approximation Methods 11 | 1. Policy Function Iteration, Endogenous Grid Method, and Envelope Condition Method 12 | 1. Approximating the Distribution 13 | 1. Perturbation Methods 14 | 1. Solving Models with Aggregate and Idiosyncratic Risks 15 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2019 wupeifan 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /Lecture_7/HeterogeneousAgents.tex: -------------------------------------------------------------------------------- 1 | % \documentclass[handout]{beamer} % Suppress the pauses to print as handout 2 | \documentclass[aspectratio=169, 11pt]{beamer} 3 | % \documentclass[11pt]{beamer} % Comment the line above and use this line for 4:3 aspect ratio screens 4 | \usepackage{advdate} 5 | \usepackage{amsmath} 6 | \usepackage{amsthm} 7 | \usepackage{amsfonts} 8 | \usepackage{amssymb} 9 | \usepackage{array} 10 | \usepackage{color} 11 | \usepackage{helvet} 12 | \usepackage{hyperref} 13 | \usepackage{mathpazo} 14 | \usepackage{multirow} 15 | 16 | \usetheme{CambridgeUS} 17 | \usecolortheme{dolphin} 18 | \usefonttheme[onlymath]{serif} 19 | 20 | % Define colors considering color blindness 21 | % Use MATLAB default color palette 22 | \definecolor{blue}{rgb}{0, 0.447, 0.741} 23 | \definecolor{red}{rgb}{0.85, 0.325, 0.098} 24 | \definecolor{yellow}{rgb}{0.929, 0.694, 0.125} 25 | \definecolor{purple}{rgb}{0.494, 0.184, 0.556} 26 | \definecolor{green}{rgb}{0.466, 0.674, 0.188} 27 | \definecolor{cyan}{rgb}{0.301, 0.745, 0.933} 28 | \definecolor{magenta}{rgb}{0.635, 0.078, 0.184} 29 | \definecolor{amber}{rgb}{1.0, 0.75, 0.0} 30 | 31 | % Self-defined background and frametitle colors 32 | \definecolor{MyBackground}{RGB}{255, 255, 230} 33 | % Uncomment the following line for plain white background 34 | % \definecolor{MyBackground}{RGB}{255, 255, 255} 35 | \setbeamercolor{background canvas}{bg=MyBackground!40} 36 | \setbeamercolor{frametitle}{bg=MyBackground!80} 37 | % Customize word colors 38 | \setbeamercolor{frametitle}{fg=blue} 39 | \setbeamercolor{title}{fg=blue} 40 | \setbeamercolor{itemize item}{fg=blue} 41 | \setbeamercolor{itemize subitem}{fg=blue} 42 | \setbeamercolor{enumerate item}{fg=blue} 43 | \setbeamercolor{enumerate subitem}{fg=blue} 44 | \setbeamercolor{button}{bg=MyBackground,fg=blue} 45 | 46 | % Page number on the bottom 47 | \setbeamertemplate{footline}[frame number] 48 | \setbeamerfont{footline}{size=\fontsize{10}{12}\selectfont} 49 | \setbeamercolor{page number in head/foot}{fg=blue} 50 | % No navigation symbols 51 | \setbeamertemplate{navigation symbols}{} 52 | % No headlines 53 | \setbeamertemplate{headline}{} 54 | % Appendix slides don't count 55 | \newcommand{\backupbegin}{ 56 | \newcounter{framenumberappendix} 57 | \setcounter{framenumberappendix}{\value{framenumber}} 58 | } 59 | \newcommand{\backupend}{ 60 | \addtocounter{framenumberappendix}{-\value{framenumber}} 61 | \addtocounter{framenumber}{\value{framenumberappendix}} 62 | } 63 | % Add section title at the beginning of each part 64 | \AtBeginSection[]{ 65 | \begin{frame}[noframenumbering, plain] 66 | \vfill 67 | \centering 68 | \begin{beamercolorbox}[sep=8pt, center, shadow=true, rounded=true]{title} 69 | \usebeamerfont{frametitle}\insertsectionhead\par% 70 | \end{beamercolorbox} 71 | \vfill 72 | \end{frame} 73 | } 74 | 75 | % New theorem environments 76 | \newtheorem{thm}{Theorem} 77 | \newtheorem{lem}[thm]{Lemma} 78 | \newtheorem{prop}[thm]{Proposition} 79 | 80 | \beamersetuncovermixins{\opaqueness<1>{25}}{\opaqueness<2->{15}} 81 | 82 | % Set pause options to be invisible instead of transparent 83 | \setbeamercovered{invisible} 84 | 85 | \begin{document} 86 | \title{Lecture 6 \\ Solving Models with Idiosyncratic and Aggregate Risks} 87 | \author[Wu]{Peifan Wu} 88 | \date{UBC} 89 | 90 | \begin{frame} 91 | \titlepage 92 | \end{frame} 93 | 94 | \begin{frame} 95 | \frametitle{Projection-Perturbation Approach} 96 | \begin{itemize} 97 | \item[1.] Provide a finite representation of the economy 98 | \begin{itemize} 99 | \medskip 100 | \item[--] Polynomial interpolations of value functions / Euler equations, and policy functions (lec 3) 101 | \medskip 102 | \item[--] Finite approximate the distribution (lec 5) 103 | \end{itemize} 104 | \bigskip 105 | \item[2.] Compute the deterministic steady state of the economy. This step yields a finite representation of the steady-state value functions, the steady-state policy functions, and the invariant distribution. (lec 3-4) 106 | \bigskip 107 | \item[3.] Compute a first-order perturbation of all these variables around the steady state. (lec 6) 108 | \end{itemize} 109 | \end{frame} 110 | 111 | \end{document} 112 | -------------------------------------------------------------------------------- /Lecture_6/Perturbation.tex: -------------------------------------------------------------------------------- 1 | % \documentclass[handout]{beamer} % Suppress the pauses to print as handout 2 | \documentclass[aspectratio=169, 11pt]{beamer} 3 | % \documentclass[11pt]{beamer} % Comment the line above and use this line for 4:3 aspect ratio screens 4 | \usepackage{advdate} 5 | \usepackage{amsmath} 6 | \usepackage{amsthm} 7 | \usepackage{amsfonts} 8 | \usepackage{amssymb} 9 | \usepackage{array} 10 | \usepackage{color} 11 | \usepackage{helvet} 12 | \usepackage{hyperref} 13 | \usepackage{mathpazo} 14 | \usepackage{multirow} 15 | 16 | \usetheme{CambridgeUS} 17 | \usecolortheme{dolphin} 18 | \usefonttheme[onlymath]{serif} 19 | 20 | % Define colors considering color blindness 21 | % Use MATLAB default color palette 22 | \definecolor{blue}{rgb}{0, 0.447, 0.741} 23 | \definecolor{red}{rgb}{0.85, 0.325, 0.098} 24 | \definecolor{yellow}{rgb}{0.929, 0.694, 0.125} 25 | \definecolor{purple}{rgb}{0.494, 0.184, 0.556} 26 | \definecolor{green}{rgb}{0.466, 0.674, 0.188} 27 | \definecolor{cyan}{rgb}{0.301, 0.745, 0.933} 28 | \definecolor{magenta}{rgb}{0.635, 0.078, 0.184} 29 | \definecolor{amber}{rgb}{1.0, 0.75, 0.0} 30 | 31 | % Self-defined background and frametitle colors 32 | \definecolor{MyBackground}{RGB}{255, 255, 230} 33 | % Uncomment the following line for plain white background 34 | % \definecolor{MyBackground}{RGB}{255, 255, 255} 35 | \setbeamercolor{background canvas}{bg=MyBackground!40} 36 | \setbeamercolor{frametitle}{bg=MyBackground!80} 37 | % Customize word colors 38 | \setbeamercolor{frametitle}{fg=blue} 39 | \setbeamercolor{title}{fg=blue} 40 | \setbeamercolor{itemize item}{fg=blue} 41 | \setbeamercolor{itemize subitem}{fg=blue} 42 | \setbeamercolor{enumerate item}{fg=blue} 43 | \setbeamercolor{enumerate subitem}{fg=blue} 44 | \setbeamercolor{button}{bg=MyBackground,fg=blue} 45 | 46 | % Page number on the bottom 47 | \setbeamertemplate{footline}[frame number] 48 | \setbeamerfont{footline}{size=\fontsize{10}{12}\selectfont} 49 | \setbeamercolor{page number in head/foot}{fg=blue} 50 | % No navigation symbols 51 | \setbeamertemplate{navigation symbols}{} 52 | % No headlines 53 | \setbeamertemplate{headline}{} 54 | % Appendix slides don't count 55 | \newcommand{\backupbegin}{ 56 | \newcounter{framenumberappendix} 57 | \setcounter{framenumberappendix}{\value{framenumber}} 58 | } 59 | \newcommand{\backupend}{ 60 | \addtocounter{framenumberappendix}{-\value{framenumber}} 61 | \addtocounter{framenumber}{\value{framenumberappendix}} 62 | } 63 | % Add section title at the beginning of each part 64 | \AtBeginSection[]{ 65 | \begin{frame}[noframenumbering, plain] 66 | \vfill 67 | \centering 68 | \begin{beamercolorbox}[sep=8pt, center, shadow=true, rounded=true]{title} 69 | \usebeamerfont{frametitle}\insertsectionhead\par% 70 | \end{beamercolorbox} 71 | \vfill 72 | \end{frame} 73 | } 74 | 75 | % New theorem environments 76 | \newtheorem{thm}{Theorem} 77 | \newtheorem{lem}[thm]{Lemma} 78 | \newtheorem{prop}[thm]{Proposition} 79 | 80 | \beamersetuncovermixins{\opaqueness<1>{25}}{\opaqueness<2->{15}} 81 | 82 | % Set pause options to be invisible instead of transparent 83 | \setbeamercovered{invisible} 84 | 85 | \begin{document} 86 | \title{Lecture 6 \\ Perturbation Methods} 87 | \author[Wu]{Peifan Wu} 88 | \date{UBC} 89 | 90 | \begin{frame} 91 | \titlepage 92 | \end{frame} 93 | 94 | \begin{frame} 95 | \frametitle{Local Approximations} 96 | \begin{itemize} 97 | \item[--] DSGE models are characterized by \textcolor{blue}{a set of nonlinear equations} 98 | \bigskip 99 | \item[--] Use local approximation to transform the nonlinear system into a system of linear equations 100 | \bigskip 101 | \item[--] Two requirements: small shocks, no occasionally binding constraints 102 | \end{itemize} 103 | \end{frame} 104 | 105 | \begin{frame} 106 | \frametitle{Perturbing Equations} 107 | \begin{itemize} 108 | \item[--] Want: find a solution $y=g\left(x\right)$ for equation system $f\left(x,y\right)=0$ 109 | \item[--] Steady state $f\left(\bar{x},\bar{y}\right)=0$ 110 | \bigskip 111 | \item[--] (First-order) Taylor expansion of $f$ 112 | \[ 113 | f\left(x,g\left(x\right)\right)\approx f\left(\bar{x},\bar{y}\right)+\frac{\partial f}{\partial x}\left(\bar{x}\right)\left(x-\bar{x}\right)+\frac{\partial f}{\partial y}\left(\bar{y}\right)g'\left(\bar{x}\right)\left(x-\bar{x}\right) 114 | \] 115 | \bigskip 116 | \item[--] Back out $g'\left(\bar{x}\right)$ and approximate the policy function $g\left(x\right)\approx g\left(\bar{x}\right)+\left(x-\bar{x}\right)g'\left(\bar{x}\right)$ 117 | \end{itemize} 118 | \end{frame} 119 | 120 | \begin{frame} 121 | \frametitle{Perturbing DSGE} 122 | \begin{itemize} 123 | \item[--] Equation system: $\mathbb{E}_{t}\mathcal{H}\left(y',y,x',x;\sigma\right)=0$ 124 | \item[--] Want: \textcolor{blue}{policy function} $y=g\left(x;\sigma\right)$ and \textcolor{blue}{law of the motions} of the states $x'=h\left(x;\sigma\right)$ 125 | \bigskip 126 | \item[--] Perturbation parameter is the variance scalar of the shocks, \textcolor{red}{$\sigma$} 127 | \item[--] There can be multiple uncertainties, but \textcolor{red}{only one perturbation variable} 128 | \bigskip 129 | \item[--] Find approximate solution of $g$ and $h$ around $x=\bar{x},\sigma=0$ 130 | \end{itemize} 131 | \end{frame} 132 | 133 | \begin{frame} 134 | \frametitle{Perturbing DSGE} 135 | \begin{itemize} 136 | \item[--] Plug in the proposed solution 137 | \[ 138 | F\left(x;\sigma\right)\equiv\mathbb{E}_{t}\mathcal{H}\left(g\left(h\left(x;\sigma\right)\right),g\left(x;\sigma\right),h\left(x;\sigma\right),x;\sigma\right)=0 139 | \] 140 | \bigskip 141 | \item[--] $F=0$ for \textcolor{red}{any} values of $x$ and $\sigma$, so the directives at any direction should be $0$ 142 | \bigskip 143 | \item[--] In particular, we take the derivatives around $x=\bar{x}, \sigma=0$ 144 | \[ 145 | F_{x^{i}\sigma^{j}}\left(\bar{x};0\right)=0,\forall i,j 146 | \] 147 | \end{itemize} 148 | \end{frame} 149 | 150 | \begin{frame} 151 | \frametitle{Perturbing DSGE} 152 | \begin{itemize} 153 | \item[--] We propose solution of $g,h$ 154 | \begin{align*} 155 | g\left(x;\sigma\right) & =g\left(\bar{x};0\right)+g_{x}\left(x-\bar{x}\right)+g_{\sigma}\sigma\\ 156 | h\left(x;\sigma\right) & =h\left(\bar{x};0\right)+h_{x}\left(x-\bar{x}\right)+h_{\sigma}\sigma 157 | \end{align*} 158 | \bigskip 159 | \item[--] $g_x,h_x,g_\sigma,h_\sigma$ should solve $F_x=0,F_\sigma=0$ 160 | \begin{align*} 161 | F_{x} & =\mathcal{H}_{y'}g_{x}h_{x}+\mathcal{H}_{y}g_{x}+\mathcal{H}_{x'}h_{x}+\mathcal{H}_{x}=0\\ 162 | F_{\sigma} & =\mathcal{H}_{y'}g_{x}h_{\sigma}+\mathcal{H}_{y}g_{\sigma}+\mathcal{H}_{y}g_{\sigma}+\mathcal{H}_{x'}h_{\sigma}=0 163 | \end{align*} 164 | \bigskip 165 | \item[--] Observations: 166 | \begin{itemize} 167 | \item[1.] The first equation is a linear-quadratic system. Typical solution method applies (Blanchard-Kahn, Klein, Sims \textit{gensys}) 168 | \item[2.] The second equation is homogeneous in $g_{\sigma}$ and $h_{\sigma}$ -- certainty equivalence 169 | \end{itemize} 170 | \end{itemize} 171 | \end{frame} 172 | 173 | \end{document} 174 | -------------------------------------------------------------------------------- /Lecture_5/Distribution.tex: -------------------------------------------------------------------------------- 1 | % \documentclass[handout]{beamer} % Suppress the pauses to print as handout 2 | \documentclass[aspectratio=169, 11pt]{beamer} 3 | % \documentclass[11pt]{beamer} % Comment the line above and use this line for 4:3 aspect ratio screens 4 | \usepackage{advdate} 5 | \usepackage{amsmath} 6 | \usepackage{amsthm} 7 | \usepackage{amsfonts} 8 | \usepackage{amssymb} 9 | \usepackage{array} 10 | \usepackage{color} 11 | \usepackage{helvet} 12 | \usepackage{hyperref} 13 | \usepackage{mathpazo} 14 | \usepackage{multirow} 15 | 16 | \usetheme{CambridgeUS} 17 | \usecolortheme{dolphin} 18 | \usefonttheme[onlymath]{serif} 19 | 20 | % Define colors considering color blindness 21 | % Use MATLAB default color palette 22 | \definecolor{blue}{rgb}{0, 0.447, 0.741} 23 | \definecolor{red}{rgb}{0.85, 0.325, 0.098} 24 | \definecolor{yellow}{rgb}{0.929, 0.694, 0.125} 25 | \definecolor{purple}{rgb}{0.494, 0.184, 0.556} 26 | \definecolor{green}{rgb}{0.466, 0.674, 0.188} 27 | \definecolor{cyan}{rgb}{0.301, 0.745, 0.933} 28 | \definecolor{magenta}{rgb}{0.635, 0.078, 0.184} 29 | \definecolor{amber}{rgb}{1.0, 0.75, 0.0} 30 | 31 | % Self-defined background and frametitle colors 32 | \definecolor{MyBackground}{RGB}{255, 255, 230} 33 | % Uncomment the following line for plain white background 34 | % \definecolor{MyBackground}{RGB}{255, 255, 255} 35 | \setbeamercolor{background canvas}{bg=MyBackground!40} 36 | \setbeamercolor{frametitle}{bg=MyBackground!80} 37 | % Customize word colors 38 | \setbeamercolor{frametitle}{fg=blue} 39 | \setbeamercolor{title}{fg=blue} 40 | \setbeamercolor{itemize item}{fg=blue} 41 | \setbeamercolor{itemize subitem}{fg=blue} 42 | \setbeamercolor{enumerate item}{fg=blue} 43 | \setbeamercolor{enumerate subitem}{fg=blue} 44 | \setbeamercolor{button}{bg=MyBackground,fg=blue} 45 | 46 | % Page number on the bottom 47 | \setbeamertemplate{footline}[frame number] 48 | \setbeamerfont{footline}{size=\fontsize{10}{12}\selectfont} 49 | \setbeamercolor{page number in head/foot}{fg=blue} 50 | % No navigation symbols 51 | \setbeamertemplate{navigation symbols}{} 52 | % No headlines 53 | \setbeamertemplate{headline}{} 54 | % Appendix slides don't count 55 | \newcommand{\backupbegin}{ 56 | \newcounter{framenumberappendix} 57 | \setcounter{framenumberappendix}{\value{framenumber}} 58 | } 59 | \newcommand{\backupend}{ 60 | \addtocounter{framenumberappendix}{-\value{framenumber}} 61 | \addtocounter{framenumber}{\value{framenumberappendix}} 62 | } 63 | % Add section title at the beginning of each part 64 | \AtBeginSection[]{ 65 | \begin{frame}[noframenumbering, plain] 66 | \vfill 67 | \centering 68 | \begin{beamercolorbox}[sep=8pt, center, shadow=true, rounded=true]{title} 69 | \usebeamerfont{frametitle}\insertsectionhead\par% 70 | \end{beamercolorbox} 71 | \vfill 72 | \end{frame} 73 | } 74 | 75 | % New theorem environments 76 | \newtheorem{thm}{Theorem} 77 | \newtheorem{lem}[thm]{Lemma} 78 | \newtheorem{prop}[thm]{Proposition} 79 | 80 | \beamersetuncovermixins{\opaqueness<1>{25}}{\opaqueness<2->{15}} 81 | 82 | % Set pause options to be invisible instead of transparent 83 | \setbeamercovered{invisible} 84 | 85 | \begin{document} 86 | \title{Lecture 5 \\ Approximating Distribution} 87 | \author[Wu]{Peifan Wu} 88 | \date{UBC} 89 | 90 | \begin{frame} 91 | \titlepage 92 | \end{frame} 93 | 94 | \begin{frame} 95 | \frametitle{Approaches} 96 | \begin{itemize} 97 | \item[--] Continuum of households makes the wealth distribution an \textcolor{red}{infinite dimensional object} in the state space 98 | \item[--] Need approximation 99 | \bigskip 100 | \item[--] Several approaches beyond Monte-Carlo simulation 101 | \begin{itemize} 102 | \item[1.] Discretization of the density function 103 | \item[2.] Eigenvector method 104 | \item[3.] Parametrizing in exponential family 105 | \end{itemize} 106 | \end{itemize} 107 | \end{frame} 108 | 109 | \begin{frame} 110 | \frametitle{Discretization of the Invariant Density Function} 111 | \begin{itemize} 112 | \item[--] The grid should be finer than the one we used to compute the optimal savings rule 113 | \bigskip 114 | \item[--] Choose some initial density function $\lambda^{0}\left(a_{k},y_{i}\right)$ 115 | \bigskip 116 | \item[--] For every $\left(a_{k},y_{j}\right)$ on the grid: 117 | \begin{align*} 118 | \lambda^{1}\left(a_{k},y_{j}\right) & =\sum_{y_{i}\in Y}\pi_{ij}\sum_{m\in M_{i}}\frac{a_{k+1}-g\left(a_{m},y_{i}\right)}{a_{k+1}-a_{k}}\lambda^{0}\left(a_{m},y_{i}\right)\\ 119 | \lambda^{1}\left(a_{k+1},y_{j}\right) & =\sum_{y_{i}\in Y}\pi_{ij}\sum_{m\in M_{i}}\frac{g\left(a_{m}.y_{i}\right)-a_{k}}{a_{k+1}-a_{k}}\lambda^{0}\left(a_{m},y_{i}\right) 120 | \end{align*} 121 | where $M_{i}=\left\{ m=1,\ldots,N|a_{k}\leqslant g\left(a_{m},y_{i}\right)\leqslant a_{k+1}\right\}$ 122 | \bigskip 123 | \end{itemize} 124 | \end{frame} 125 | 126 | \begin{frame} 127 | \frametitle{Lottery Rule} 128 | \begin{itemize} 129 | \item[--] We can think of this way of handling the discrete approximation to the density function as \textcolor{red}{forcing the agents in the economy to play a lottery}. 130 | \bigskip 131 | \item[--] If the optimal policy is to save $a'\in\left[a_{k},a_{k+1}\right]$, then with probability $\frac{a_{k+1}-a'}{a_{k+1}-a_{k}}$ you go to $a_{k}$, and with probability $\frac{a'-a_{k}}{a_{k+1}-a_{k}}$ you go to $a_{k+1}$. 132 | \end{itemize} 133 | \end{frame} 134 | 135 | \begin{frame} 136 | \frametitle{Eigenvector Method} 137 | \begin{itemize} 138 | \item[--] The invariant pdf for a Markov transition matrix $Q$ is $\lambda^{*}$ that satisfies $\lambda^{*}Q=\lambda^{*}$ 139 | \bigskip 140 | \item[--] \textcolor{red}{Perron-Frobenius Theorem}: $Q$ has a unique dominant eigenvalue $\epsilon=1$ such that 141 | \begin{itemize} 142 | \medskip 143 | \item[--] its associated eigenvector has all positive entries 144 | \medskip 145 | \item[--] all other eigenvalues are smaller than $\epsilon$ in absolute value 146 | \medskip 147 | \item[--] $Q$ has no other eigenvector with all non-negative entries 148 | \end{itemize} 149 | \bigskip 150 | \item[--] This eigenvector (renormalized so that it sums to one) is the unique invariant distribution 151 | \end{itemize} 152 | \end{frame} 153 | 154 | \begin{frame} 155 | \frametitle{Parameterized Distribution} 156 | \begin{itemize} 157 | \item[--] For smooth and unimodal distributions we can approximate with parameterized functions in the \textcolor{red}{exponential families} 158 | \bigskip 159 | \item[--] Central moments are sufficient statistics of the distribution 160 | \end{itemize} 161 | \end{frame} 162 | 163 | \begin{frame} 164 | \frametitle{Parameterized Distribution} 165 | \begin{itemize} 166 | \item[--] To approximate a one-dimensional distribution on $k$, assume the density function form 167 | \[ 168 | P\left(k,\rho\right)=\rho_{0}\exp\left(\begin{array}{c} 169 | \rho_{1}\left(k-m_{1}\right)\\ 170 | +\rho_{2}\left[\left(k-m_{1}\right)^{2}-m_{2}\right]\\ 171 | +\rho_{3}\left[\left(k-m_{1}\right)^{3}-m_{3}\right]\\ 172 | \cdots\\ 173 | +\rho_{N}\left[\left(k-m_{1}\right)^{N}-m_{N}\right] 174 | \end{array}\right) 175 | \] 176 | \item[--] $\rho$ -- parameters to pin down, $m$ -- central moments 177 | \end{itemize} 178 | \end{frame} 179 | 180 | \begin{frame} 181 | \frametitle{Parameterized Distribution} 182 | \begin{itemize} 183 | \item[--] We choose $\rho$ such that moment condition holds 184 | \begin{align*} 185 | \int\left(k-m_{1}\right)P\left(k,\rho\right)\mathrm{d}k & =0\\ 186 | \int\left[\left(k-m_{1}\right)^{2}-m_{2}\right]P\left(k,\rho\right)\mathrm{d}k & =0\\ 187 | & \cdots\\ 188 | \int\left[\left(k-m_{1}\right)^{N}-m_{N}\right]P\left(k,\rho\right)\mathrm{d}k & =0 189 | \end{align*} 190 | \item[--] These condition happen to be the optimal condition of 191 | \[ 192 | \min_{\rho_{1},\ldots,\rho_{N}}\int P\left(k,\rho\right)\mathrm{d}k 193 | \] 194 | \item[--] And the distribution has to be normalized to unity 195 | \[ 196 | \frac{1}{\rho_{0}}=\int P\left(k,\rho\right)\mathrm{d}k 197 | \] 198 | \end{itemize} 199 | \end{frame} 200 | 201 | \begin{frame} 202 | \frametitle{Parameterized Distribution} 203 | Steps to compute invariant distribution: 204 | \begin{itemize} 205 | \item[1.] Choose capital grid $\left\{ k_{i}\right\}$ and guess initial central moments $m_{j,0}$ 206 | \item[2.] Solve the minimization problem for $\rho$ with $m_{j,0}$ 207 | \[ 208 | \frac{1}{\rho_{0}}=\min_{\rho_{1},\ldots,\rho_{N}}\int P\left(k,\rho\right)\mathrm{d}k 209 | \] 210 | \item[3.] With current distribution $P\left(k,\rho\right)$, compute moments using policy function $g(k)$ 211 | \begin{align*} 212 | m_{1,1} & =\int g\left(k\right)P\left(k,\rho\right)\mathrm{d}k\\ 213 | m_{2,1} & =\int\left(g\left(k\right)-m_{1,1}\right)^{2}P\left(k,\rho\right)\mathrm{d}k\\ 214 | & \cdots\\ 215 | m_{N,1} & =\int\left(g\left(k\right)-m_{1,1}\right)^{N}P\left(k,\rho\right)\mathrm{d}k 216 | \end{align*} 217 | \item[4.] Check convergence of $m$ 218 | \end{itemize} 219 | \end{frame} 220 | 221 | \end{document} 222 | -------------------------------------------------------------------------------- /Lecture_4/PFI.tex: -------------------------------------------------------------------------------- 1 | % \documentclass[handout]{beamer} % Suppress the pauses to print as handout 2 | \documentclass[aspectratio=169, 11pt]{beamer} 3 | % \documentclass[11pt]{beamer} % Comment the line above and use this line for 4:3 aspect ratio screens 4 | \usepackage{advdate} 5 | \usepackage{amsmath} 6 | \usepackage{amsthm} 7 | \usepackage{amsfonts} 8 | \usepackage{amssymb} 9 | \usepackage{array} 10 | \usepackage{color} 11 | \usepackage{helvet} 12 | \usepackage{hyperref} 13 | \usepackage{mathpazo} 14 | \usepackage{multirow} 15 | 16 | \usetheme{CambridgeUS} 17 | \usecolortheme{dolphin} 18 | \usefonttheme[onlymath]{serif} 19 | 20 | % Define colors considering color blindness 21 | % Use MATLAB default color palette 22 | \definecolor{blue}{rgb}{0, 0.447, 0.741} 23 | \definecolor{red}{rgb}{0.85, 0.325, 0.098} 24 | \definecolor{yellow}{rgb}{0.929, 0.694, 0.125} 25 | \definecolor{purple}{rgb}{0.494, 0.184, 0.556} 26 | \definecolor{green}{rgb}{0.466, 0.674, 0.188} 27 | \definecolor{cyan}{rgb}{0.301, 0.745, 0.933} 28 | \definecolor{magenta}{rgb}{0.635, 0.078, 0.184} 29 | \definecolor{amber}{rgb}{1.0, 0.75, 0.0} 30 | 31 | % Self-defined background and frametitle colors 32 | \definecolor{MyBackground}{RGB}{255, 255, 230} 33 | % Uncomment the following line for plain white background 34 | % \definecolor{MyBackground}{RGB}{255, 255, 255} 35 | \setbeamercolor{background canvas}{bg=MyBackground!40} 36 | \setbeamercolor{frametitle}{bg=MyBackground!80} 37 | % Customize word colors 38 | \setbeamercolor{frametitle}{fg=blue} 39 | \setbeamercolor{title}{fg=blue} 40 | \setbeamercolor{itemize item}{fg=blue} 41 | \setbeamercolor{itemize subitem}{fg=blue} 42 | \setbeamercolor{enumerate item}{fg=blue} 43 | \setbeamercolor{enumerate subitem}{fg=blue} 44 | \setbeamercolor{button}{bg=MyBackground,fg=blue} 45 | 46 | % Page number on the bottom 47 | \setbeamertemplate{footline}[frame number] 48 | \setbeamerfont{footline}{size=\fontsize{10}{12}\selectfont} 49 | \setbeamercolor{page number in head/foot}{fg=blue} 50 | % No navigation symbols 51 | \setbeamertemplate{navigation symbols}{} 52 | % No headlines 53 | \setbeamertemplate{headline}{} 54 | % Appendix slides don't count 55 | \newcommand{\backupbegin}{ 56 | \newcounter{framenumberappendix} 57 | \setcounter{framenumberappendix}{\value{framenumber}} 58 | } 59 | \newcommand{\backupend}{ 60 | \addtocounter{framenumberappendix}{-\value{framenumber}} 61 | \addtocounter{framenumber}{\value{framenumberappendix}} 62 | } 63 | % Add section title at the beginning of each part 64 | \AtBeginSection[]{ 65 | \begin{frame}[noframenumbering, plain] 66 | \vfill 67 | \centering 68 | \begin{beamercolorbox}[sep=8pt, center, shadow=true, rounded=true]{title} 69 | \usebeamerfont{frametitle}\insertsectionhead\par% 70 | \end{beamercolorbox} 71 | \vfill 72 | \end{frame} 73 | } 74 | 75 | % New theorem environments 76 | \newtheorem{thm}{Theorem} 77 | \newtheorem{lem}[thm]{Lemma} 78 | \newtheorem{prop}[thm]{Proposition} 79 | 80 | \beamersetuncovermixins{\opaqueness<1>{25}}{\opaqueness<2->{15}} 81 | 82 | % Set pause options to be invisible instead of transparent 83 | \setbeamercovered{invisible} 84 | 85 | \begin{document} 86 | \title{Lecture 4 \\ Policy Function Iteration} 87 | \author[Wu]{Peifan Wu} 88 | \date{UBC} 89 | 90 | \begin{frame} 91 | \titlepage 92 | \end{frame} 93 | 94 | \begin{frame} 95 | \frametitle{Income Fluctuation Problem} 96 | \begin{align*} 97 | V\left(a,y\right) & =\max_{c,a'} u\left(c\right)+\beta\sum_{y'\in Y}\pi\left(y'|y\right)V\left(a',y'\right)\\ 98 | c+a' & \leqslant Ra+y\\ 99 | a' & \geqslant\underline{a} 100 | \end{align*} 101 | \bigskip 102 | \textcolor{red}{Euler equation} reads 103 | \[ 104 | u_{c}\left(Ra+y-a'\right)-\beta R\sum_{y'\in Y}\pi\left(y'|y\right)u_{c}\left(Ra'+y'-a''\right)\geqslant0 105 | \] 106 | where the strict inequality holds when the constraint is binding 107 | \bigskip 108 | 109 | Policy function iteration (and improvements): 110 | \begin{itemize} 111 | \item[--] PFI with linear interpolation 112 | \item[--] \textcolor{red}{Endogenous grid method} 113 | \item[--] \textcolor{red}{Envelope condition method} 114 | \end{itemize} 115 | \end{frame} 116 | 117 | \begin{frame} 118 | \frametitle{Policy Function Iteration with Linear Interpolation} 119 | \begin{itemize} 120 | \item[1.] Construct a grid on the asset space $\left\{ a_{0},a_{1},\ldots,a_{m}\right\}$ with $a_{0}=\underline{a}$ 121 | \item[2.] Guess an initial vector of decision rules for $a'$ on the grid points, call it $\hat{a}_{0}\left(a_{i},y\right)$ 122 | \item[3.] For each point $\left(a_{i},y\right)$ on the grid, check whether the borrowing constraint binds, i.e. 123 | \[ 124 | u_{c}\left(Ra_{i}+y-\textcolor{red}{\underline{a}}\right)-\beta R\sum_{y'\in Y}\pi\left(y'|y\right)u_{c}\left(R\textcolor{red}{\underline{a}}+y'-\hat{a}_{0}\left(\textcolor{red}{\underline{a}},y'\right)\right)>0 125 | \] 126 | \item[4.] If the inequality holds then the borrowing constraint binds. Set $\hat{a}_{0}\left(a_{i},y\right)=\underline{a}$ in this case. Otherwise we have an interior solution and we proceed to the next step. 127 | \end{itemize} 128 | \end{frame} 129 | 130 | \begin{frame} 131 | \frametitle{Policy Function Iteration with Linear Interpolation} 132 | \begin{itemize} 133 | \item[5.] For each point $\left(a_{i},y\right)$ on the grid, use a \textcolor{blue}{nonlinear equation solver} to find the solution $a^{*}$ of the nonlinear equation 134 | \[ 135 | u_{c}\left(Ra_{i}+y-\textcolor{red}{a^{*}}\right)-\beta R\sum_{y'\in Y}\pi\left(y'|y\right)u_{c}\left(R\textcolor{red}{a^{*}}+y'-\hat{a}_{0}\left(\textcolor{red}{a^{*}},y'\right)\right)=0 136 | \] 137 | We need to evaluate the function $\hat{a}_{0}$ outside grid points: assume it is \textcolor{blue}{piecewise linear}. Find the pair of adjacent grid points such that $a_{i}{25}}{\opaqueness<2->{15}} 81 | 82 | % Set pause options to be invisible instead of transparent 83 | \setbeamercovered{invisible} 84 | 85 | \begin{document} 86 | \title{Lecture 2 \\ Value Function Iteration with Discretization} 87 | \author[Wu]{Peifan Wu} 88 | \date{UBC} 89 | 90 | \begin{frame} 91 | \titlepage 92 | \end{frame} 93 | 94 | \begin{frame} 95 | \frametitle{Stochastic Growth Model} 96 | Consider the following problem: 97 | \begin{align*} 98 | V\left(z,k\right) & =\max_{c,k,l}u\left(c\right)+v\left(1-l\right)+\beta\mathbb{E}V\left(z',k'\right)\\ 99 | & \text{s.t.}\\ 100 | c+k' & =\exp\left(z\right)f\left(k.l\right)+\left(1-\delta\right)k\\ 101 | z' & =\rho z+\epsilon'\\ 102 | c,k' & \geqslant0\\ 103 | l & \in\left(0,1\right)\\ 104 | \epsilon & \sim\mathcal{N}\left(0,\sigma^{2}\right) 105 | \end{align*} 106 | We want to solve policy functions: $c\left(z,k\right),k\left(z,k\right),l\left(z,k\right)$ \\ 107 | \medskip 108 | We solve them by 109 | \begin{itemize} 110 | \item[--] Discretization of the state space $\left(Z, K\right)$ 111 | \item[--] Value Function Iteration 112 | \end{itemize} 113 | \end{frame} 114 | 115 | \begin{frame} 116 | \frametitle{Discretize AR(1) Process} 117 | Two dominant methodologies: 118 | \bigskip 119 | 120 | \begin{itemize} 121 | \item[1.] \textcolor{red}{Tauchen method} (Tauchen, EL 1986) 122 | \begin{itemize} 123 | \item[--] Not restricted to normal innovations 124 | \end{itemize} 125 | \bigskip 126 | \item[2.] \textcolor{red}{Rouwenhorst method} (Kopecky-Suen, RED 2010) 127 | \begin{itemize} 128 | \item[--] Works better as $\rho\to1$ 129 | \item[--] Replicate exactly unconditional mean, variance, and first-order autocorrelation 130 | \item[--] \textcolor{blue}{Multivariate version}: Terry-Knotek (EL, 2011), Galindev-Lkhagvasuren (JEDC, 2010) 131 | \end{itemize} 132 | \end{itemize} 133 | \end{frame} 134 | 135 | \begin{frame} 136 | \frametitle{Tauchen Method} 137 | AR(1) process $y'=\rho y+\epsilon $, $\epsilon \sim \mathcal{N}\left(0,\sigma^2_{\epsilon}\right)$ \\ 138 | \bigskip 139 | 140 | \textcolor{red}{Choice of Points}: The maximum value $y_{N}$ should be multiple $m$ (e.g. $m=3$) of the unconditional standard deviation, 141 | \[ 142 | y_{N}=m\sqrt{\frac{\sigma_{\epsilon}^{2}}{1-\rho^{2}}} 143 | \] 144 | Let $y_{1}=-y_{N}$ and $\left\{ y_{2},y_{3},\ldots,y_{N-1}\right\}$ equally spaced. \\ 145 | \bigskip 146 | \textcolor{red}{Transition Probabilities}: $d=y_{i}-y_{i-1}$, $\pi_{ij}$ is the transition probability from $i$ to $j$, 147 | \begin{align*} 148 | \pi_{i1} & =F\left(\frac{y_{1}+d/2-\rho y_{i}}{\sigma_{\epsilon}}\right)\\ 149 | \pi_{ij} & =F\left(\frac{y_{j}+d/2-\rho y_{i}}{\sigma_{\epsilon}}\right)-F\left(\frac{y_{j}-d/2-\rho y_{i}}{\sigma_{\epsilon}}\right)\\ 150 | \pi_{iN} & =1-F\left(\frac{y_{N}-d/2-\rho y_{i}}{\sigma_{\epsilon}}\right) 151 | \end{align*} 152 | 153 | \end{frame} 154 | 155 | \begin{frame} 156 | \frametitle{Rouwenhorst Method} 157 | AR(1) process $y'=\rho y+\epsilon $, $\epsilon \sim \mathcal{N}\left(0,\sigma^2_{\epsilon}\right)$ \\ 158 | \bigskip 159 | 160 | \textcolor{red}{Choice of Points}: Let $y_{1}=-y_{N}$ and $\left\{ y_{2},y_{3},\ldots,y_{N-1}\right\}$ equally spaced, 161 | \[ 162 | y_{N}=\sqrt{\frac{\sigma_{\epsilon}^{2}}{1-\rho^{2}}}\sqrt{N-1} 163 | \] 164 | \bigskip 165 | \textcolor{red}{Transition Matrix}: Let $p=\frac{1+\rho}{2}$ 166 | \begin{itemize} 167 | \item[--] For $N=2$, 168 | \[ 169 | \Pi_{2}=\left[\begin{array}{cc} 170 | p & 1-p\\ 171 | 1-p & p 172 | \end{array}\right] 173 | \] 174 | \item[--] For $N\geqslant3$, 175 | \[ 176 | \Pi_{N}=p\left[\begin{array}{cc} 177 | \Pi_{N-1} & \mathbf{0}\\ 178 | \mathbf{0}' & 0 179 | \end{array}\right]+p\left[\begin{array}{cc} 180 | 0 & \mathbf{0}'\\ 181 | \mathbf{0} & \Pi_{N-1} 182 | \end{array}\right]+\left(1-p\right)\left[\begin{array}{cc} 183 | \mathbf{0}' & 0\\ 184 | \Pi_{N-1} & \mathbf{0} 185 | \end{array}\right]+\left(1-p\right)\left[\begin{array}{cc} 186 | \mathbf{0} & \Pi_{N-1}\\ 187 | 0 & \mathbf{0}' 188 | \end{array}\right] 189 | \] 190 | \end{itemize} 191 | \end{frame} 192 | 193 | \begin{frame} 194 | \frametitle{Discretize Capital Grid} 195 | \begin{itemize} 196 | \item[--] Evenly spaced grid of $N$ points: 197 | \[ 198 | k_{i}=k_{1}+\left(i-1\right)\eta\text{ where }\eta=\frac{k_{N}-k_{1}}{N-1} 199 | \] 200 | \item[--] In general, \textcolor{red}{more points where policy functions have curvature}. 201 | \item[--] A grid denser close to $k_{1}$: 202 | \[ 203 | k_{i}=k_{1}+\exp\left[\left(i-1\right)\eta\right]\text{ where }\eta=\frac{\log\left(k_{N}-k_{1}\right)}{N-1} 204 | \] 205 | \item[--] How do we choose $k_{1},k_{N}$? Solve the steady state $k^{*}$, choose $k_{1}{25}}{\opaqueness<2->{15}} 81 | 82 | % Set pause options to be invisible instead of transparent 83 | \setbeamercovered{invisible} 84 | 85 | \begin{document} 86 | \title{Lecture 3 \\ Global Approximation Methods} 87 | \author[Wu]{Peifan Wu} 88 | \date{UBC} 89 | 90 | \begin{frame} 91 | \titlepage 92 | \end{frame} 93 | 94 | \begin{frame} 95 | \frametitle{Global Function Approximation} 96 | \begin{itemize} 97 | \item[--] \textcolor{red}{Global Function}: function interpolation over the entire domain 98 | \bigskip 99 | \item[--] Types of interpolations: 100 | \begin{itemize} 101 | \medskip 102 | \item[--] \textcolor{blue}{Spectral methods} -- orthogonal polynomials (e.g. Chebyshev) that use \textcolor{red}{global basis} 103 | \medskip 104 | \item[--] \textcolor{blue}{Finite element methods} -- splines (e.g. B-splines, cubic splines, Schumaker splines) that use \textcolor{red}{local basis} 105 | \end{itemize} 106 | \bigskip 107 | \item[--] To approximate a known function $f\left(x\right)$ by a linear combination of \textcolor{red}{basis functions} $\phi$ 108 | \[ 109 | f\left(x\right)\equiv\sum_{j=0}^{n}w_{j}\phi_{j}\left(x\right) 110 | \] 111 | \begin{itemize} 112 | \item[--] Basis functions: linearly independent functions that span the family of functions chosen for the interpolation 113 | \medskip 114 | \item[--] In general we are interested in the space of $\mathcal{C}^{0}$ and $\mathcal{C}^{1}$ functions 115 | \end{itemize} 116 | \end{itemize} 117 | \end{frame} 118 | 119 | \begin{frame} 120 | \frametitle{Interpolation} 121 | \begin{itemize} 122 | \item[--] The problem boils down to determining the weights $w_{i}$ 123 | \item[--] Several choices: 124 | \begin{itemize} 125 | \item[--] \textcolor{red}{Collocation}: use the same number of interpolant nodes as the number of basis functions. $w$ comes from the solution to $\Phi w=y$ 126 | \item[--] If we have more interpolation nodes than basis functions, then we have a "curve fitting" problem. One way is to minimize SSR by applying \textcolor{red}{Least Square}, 127 | \[ 128 | w=\left(\Phi'\Phi\right)^{-1}\Phi'y 129 | \] 130 | \item[--] Or apply \textcolor{red}{Galerkin method}: define residuals 131 | \[ 132 | r(x)=f(x)-\sum_{j=0}^{n}w_{j}\phi_{j}\left(x\right) 133 | \] 134 | and solve the equations 135 | \[ 136 | \int_{a}^{b}r\left(x\right)\phi_{j}\left(x\right)\mathrm{d}x=0,j=0,1,\ldots,n 137 | \] 138 | \end{itemize} 139 | \bigskip 140 | \item[--] We apply collocation method for the following exercises 141 | \end{itemize} 142 | \end{frame} 143 | 144 | \begin{frame} 145 | \frametitle{Spectral Methods} 146 | \begin{itemize} 147 | \item[--] We use polynomial basis 148 | \bigskip 149 | \item[--] Why polynomials? \textcolor{red}{Weierstrass Theorem}: if $\mathcal{C}\left[a,b\right]$ is the set of all continuous function on $\left[a,b\right]$ then for all $f\in\mathcal{C}\left[a,b\right]$ and $\epsilon>0$ there exists a polynomial $q$ for which 150 | \[ 151 | \sup_{x\in\left[a,b\right]}\left|f\left(x\right)-q\left(x\right)\right|<\epsilon 152 | \] 153 | \bigskip 154 | \item[--] Polynomials can approximate any continuous function over a compact domain arbitrarily well 155 | \end{itemize} 156 | \end{frame} 157 | 158 | \begin{frame} 159 | \frametitle{Monomial Basis} 160 | \begin{itemize} 161 | \item[--] The most naive procedure: use monomials as basis functions 162 | \[ 163 | \phi_{j}\left(x\right)=x^{j},j=0,1,\ldots,n 164 | \] 165 | with interpolating polynomials of the form 166 | \[ 167 | \bar{f}\left(x\right)\equiv p_{n}\left(x\right)=w_{0}+w_{1}x+w_{2}x^{2}+\ldots+w_{n}x^{n} 168 | \] 169 | \item[--] Use a linear system of $n+1$ equations to solve $w_{i}$ 170 | \[ 171 | \left[\begin{array}{cccc} 172 | 1 & x_{0} & \cdots & x_{0}^{n}\\ 173 | 1 & x_{1} & \cdots & x_{1}^{n}\\ 174 | \vdots & \vdots & \ddots & \vdots\\ 175 | 1 & x_{n} & \cdots & x_{n}^{n} 176 | \end{array}\right]\left[\begin{array}{c} 177 | w_{0}\\ 178 | w_{1}\\ 179 | \vdots\\ 180 | w_{n} 181 | \end{array}\right]=\left[\begin{array}{c} 182 | y_{0}\\ 183 | y_{1}\\ 184 | \vdots\\ 185 | y_{n} 186 | \end{array}\right] 187 | \] 188 | \end{itemize} 189 | \end{frame} 190 | 191 | \begin{frame} 192 | \frametitle{Monomials in $\left[0,1\right]$} 193 | \begin{figure} 194 | \centering 195 | \includegraphics[width = 4in]{monomial.eps} 196 | \end{figure} 197 | \end{frame} 198 | 199 | \begin{frame} 200 | \frametitle{Chebyshev Polynomials} 201 | \begin{itemize} 202 | \item[--] Chebyshev polynomials are trigonometric and can be constructed recursively, 203 | \begin{align*} 204 | T_{0}\left(x\right) & =1\\ 205 | T_{1}\left(x\right) & =x\\ 206 | T_{k+1}\left(x\right) & =2xT_{k}\left(x\right)-T_{k-1}\left(x\right) 207 | \end{align*} 208 | \bigskip 209 | \item[--] $T_{n}\left(x\right)$ has $n$ distinct roots in $\left[-1, 1\right]$ with expression 210 | \[ 211 | x_{k}=\cos\left(\frac{2k-1}{2n}\pi\right),k=1,\ldots,n 212 | \] 213 | \bigskip 214 | \item[--] The interpolation nodes that minimize the error of Chebyshev interpolation are the zeros of the Chebyshev polynomials 215 | \end{itemize} 216 | \end{frame} 217 | 218 | \begin{frame} 219 | \frametitle{Chebyshev Basis Functions in $\left[0,1\right]$} 220 | \begin{figure} 221 | \centering 222 | \includegraphics[width = 4in]{chebyshev.eps} 223 | \end{figure} 224 | \end{frame} 225 | 226 | \begin{frame} 227 | \frametitle{Interpolation using Chebyshev Polynomials} 228 | \begin{itemize} 229 | \item[--] Choose $x_{i}$ to be the roots of $T_{n}$ 230 | \bigskip 231 | \item[--] The vector of $w_{i}$ can be obtained as 232 | \[ 233 | \left[\begin{array}{cccc} 234 | T_{0}\left(x_{0}\right) & T_{1}\left(x_{0}\right) & \cdots & T_{n}\left(x_{0}\right)\\ 235 | T_{0}\left(x_{1}\right) & T_{1}\left(x_{1}\right) & \cdots & T_{n}\left(x_{1}\right)\\ 236 | \vdots & \vdots & \ddots & \vdots\\ 237 | T_{0}\left(x_{n}\right) & T_{1}\left(x_{n}\right) & \cdots & T_{n}\left(x_{n}\right) 238 | \end{array}\right]\left[\begin{array}{c} 239 | w_{0}\\ 240 | w_{1}\\ 241 | \vdots\\ 242 | w_{n} 243 | \end{array}\right]=\left[\begin{array}{c} 244 | y_{0}\\ 245 | y_{1}\\ 246 | \vdots\\ 247 | y_{n} 248 | \end{array}\right] 249 | \] 250 | \bigskip 251 | \item[--] Finally the polynomial approximation of $f$ is 252 | \[ 253 | \bar{f}\left(x\right)=\sum_{j=0}^{n}w_{j}T_{j}\left(x\right) 254 | \] 255 | \end{itemize} 256 | \end{frame} 257 | 258 | \begin{frame} 259 | \frametitle{Linear B-splines} 260 | \begin{itemize} 261 | \item[--] $B^{1}$ splines implement piecewise linear interpolation 262 | \[ 263 | B_{k}^{1}\left(x\right)=\begin{cases} 264 | \frac{x-x_{k-1}}{x_{k}-x_{k-1}} & \text{if }x_{k-1}\leqslant x{25}}{\opaqueness<2->{15}} 82 | 83 | % Set pause options to be invisible instead of transparent 84 | \setbeamercovered{invisible} 85 | 86 | \begin{document} 87 | \title{Lecture 1 \\ Root-finding, Optimization, Numerical Integration} 88 | \author[Wu]{Peifan Wu} 89 | \date{UBC} 90 | 91 | \begin{frame} 92 | \titlepage 93 | \end{frame} 94 | 95 | \section{Root-finding} 96 | \begin{frame} 97 | \frametitle{Root-finding Problem} 98 | \begin{itemize} 99 | \item[--] We are interested in finding $x^{*}$ that satisfies 100 | \[ 101 | \begin{array}{cc} 102 | f\left(x\right)=0 & f\text{ }:\text{}\mathbb{R}\to\mathbb{R}\end{array} 103 | \] 104 | Or the fixed point problem 105 | \[ 106 | f(x)=x 107 | \] 108 | \bigskip 109 | \item[--] Methods: 110 | \begin{itemize} 111 | \item[--] Bisection method 112 | \item[--] Newton's method 113 | \item[--] Quasi-Newton and Broyden's algorithm 114 | \end{itemize} 115 | \end{itemize} 116 | \end{frame} 117 | 118 | \begin{frame} 119 | \frametitle{Bisection Method} 120 | \begin{itemize} 121 | \item[--] Simple and robust for \textcolor{red}{one-dimensional} continuous function on a closed interval 122 | \bigskip 123 | \item[--] Suppose $f(x)$ is defined on $[a,b]$ where $f(a)$ and $f(b)$ have opposite signs 124 | \item[--] e.g., $f(a)<0$ and $f(b)>0$ 125 | \bigskip 126 | \item[--] By the mean value theorem there exists at least a zero 127 | \begin{itemize} 128 | \medskip 129 | \item[1.] Set $n=1$, $a^{n}=a$, and $b^{n}=b$ 130 | \medskip 131 | \item[2.] Compute $c^{n}=\frac{a^{n}+b^{n}}{2}$ 132 | \medskip 133 | \item[3.] If $f(c^{n})<0$ set $a^{n+1}=c^{n}$ and $b^{n+1}=b^{n}$. Otherwise set $b^{n+1}=c^{n}$ and $a^{n+1}=a^{n}$ 134 | \medskip 135 | \item[4.] Go back to step \textcolor{blue}{2.} until convergence criterion is satisfied 136 | \end{itemize} 137 | \end{itemize} 138 | \end{frame} 139 | 140 | \begin{frame} 141 | \frametitle{Advantages and Disadvantages of Bisection Method} 142 | \begin{itemize} 143 | \item[--] \textcolor{red}{Robust and Stable}: guaranteed to find an approximate solution 144 | \bigskip 145 | \item[--] Does not require computation of derivatives: needs continuity but not differentiability 146 | \bigskip 147 | \item[--] \textcolor{red}{Slow} because it does not exploit information on the slope of the function 148 | \bigskip 149 | \item[--] \textcolor{red}{Cannot be generalized to the multi-dimensional case} 150 | \end{itemize} 151 | \end{frame} 152 | 153 | \begin{frame} 154 | \frametitle{Newton's Method} 155 | \begin{itemize} 156 | \item[--] Approximate the function $f$ at point $x^{n}$ in iteration $n$ as 157 | \[ 158 | f\left(x\right)\approx f\left(x^{n}\right)+f'\left(x^{n}\right)\left(x-x^{n}\right) 159 | \] 160 | and update $x$ \textcolor{blue}{as if you were solving for a zero of the approximation} 161 | \[ 162 | x^{n+1}=x^{n}-\frac{f\left(x^{n}\right)}{f'\left(x^{n}\right)} 163 | \] 164 | \bigskip 165 | \item[--] $f$ must be differentiable 166 | \bigskip 167 | \item[--] fast, but \textcolor{red}{unstable} if function changes derivative quickly 168 | \end{itemize} 169 | \end{frame} 170 | 171 | \begin{frame} 172 | \frametitle{Quasi-Newton Method} 173 | \begin{itemize} 174 | \item[--] Use the slope of the secant function going through $f\left(x^{n}\right)$ and $f\left(x^{n+1}\right)$ 175 | \bigskip 176 | \item[--] Iteration scheme becomes 177 | \[ 178 | x^{n+1}=x^{n}-\left[\frac{x^{n}-x^{n-1}}{f\left(x^{n}\right)-f\left(x^{n-1}\right)}\right]f\left(x^{n}\right) 179 | \] 180 | \bigskip 181 | \item[--] convergence is slower than Newton method 182 | \end{itemize} 183 | \end{frame} 184 | 185 | \begin{frame} 186 | \frametitle{Broyden's Algorithm} 187 | \begin{itemize} 188 | \item[--] \textcolor{red}{Multidimensional version} of the univariate quasi-Newton 189 | \bigskip 190 | \item[--] Let $\mathbf{f}:\mathbb{R}^{k}\to\mathbb{R}^{k}$, a system of $k$ equations and $k$ unknowns 191 | \bigskip 192 | \item[--] Iteration rule is 193 | \[ 194 | \mathbf{x}^{n+1}=\mathbf{x}^{n}-\left(\mathbf{J}^{n}\right)^{-1}\mathbf{f}\left(\mathbf{x}^{n}\right) 195 | \] 196 | where $\mathbf{J}^{n}$ is the Jacobian evaluated at $\mathbf{x}^{n}$ 197 | \bigskip 198 | \item[--] To avoid computing $\mathbf{J}^{n}$, use $\mathbf{A}^{n}$ given by secant method: 199 | \[ 200 | \mathbf{A}^{n+1}=\mathbf{A}^{n}+\left[\mathbf{f}\left(\mathbf{x}^{n+1}\right)-\mathbf{f}\left(\mathbf{x}^{n}\right)-\mathbf{A}^{n}\mathbf{d}^{n}\right]\frac{\left(\mathbf{d}^{n}\right)'}{\left(\mathbf{d}^{n}\right)'\mathbf{d}^{n}} 201 | \] 202 | where $\mathbf{d}^{n}=\mathbf{x}^{n+1}-\mathbf{x}^{n}$ 203 | \item[--] Guess of $\mathbf{A}^0$: scaled identity matrix 204 | \end{itemize} 205 | \end{frame} 206 | 207 | \section{Optimization} 208 | \begin{frame} 209 | \frametitle{Minimization} 210 | \begin{itemize} 211 | \item[--] We describe \textcolor{red}{minimization} problems: to maximize $f$ is to minimize $-f$ 212 | \item[--] Isomorphisms: 213 | \begin{itemize} 214 | \item[--] If $f$ is concave then the minimization problem 215 | \[ 216 | \min_{x\in\left[a,b\right]}f\left(x\right) 217 | \] 218 | is equivalent to root-finding on the FOC $f'\left(x\right)=0$ 219 | \medskip 220 | \item[--] The multivariate root-finding problem 221 | \begin{align*} 222 | f^{1}\left(x_{1},x_{2},\ldots,x_{n}\right) & =0\\ 223 | f^{2}\left(x_{1},x_{2},\ldots,x_{n}\right) & =0\\ 224 | \vdots\\ 225 | f^{n}\left(x_{1},x_{2},\ldots,x_{n}\right) & =0 226 | \end{align*} 227 | can be restated as a nonlinear least square problem 228 | \[ 229 | \min_{\left\{ x_{1},x_{2},\ldots,x_{n}\right\} }\sum_{i=1}^{n}f^{i}\left(x_{1},x_{2},\ldots,x_{n}\right)^{2} 230 | \] 231 | \end{itemize} 232 | \end{itemize} 233 | \end{frame} 234 | 235 | \begin{frame} 236 | \frametitle{Bracketing Method} 237 | \begin{itemize} 238 | \item[--] Most reliable for \textcolor{red}{one dimensional} problems 239 | \bigskip 240 | \item[--] Initialization: find $a f(b)$ 241 | \begin{itemize} 242 | \item[1.] Choose $d\in\left(a,b\right)$ and compute $f(d)$ 243 | \medskip 244 | \item[2.] Choose new $(a,b,c)$ triplet: 245 | 246 | If $f(d)>f(b)$ then the minimum is in $\left[d,c\right]$. Update the triple $(a,b,c)$ with $(d,b,c)$. 247 | 248 | If $f(d)