├── Makefile ├── README.md ├── argobast.png ├── log.png ├── operators.bib ├── operators.pdf ├── operators.tex └── shift2.png /Makefile: -------------------------------------------------------------------------------- 1 | operators.pdf: operators.tex 2 | /usr/local/texlive/2017/bin/x86_64-darwin/pdflatex -shell-escape operators 3 | /usr/local/texlive/2017/bin/x86_64-darwin/bibtex operators 4 | /usr/local/texlive/2017/bin/x86_64-darwin/pdflatex -shell-escape operators 5 | /usr/local/texlive/2017/bin/x86_64-darwin/pdflatex -shell-escape operators 6 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | Some unpolished notes on abusing the differential operator $D=\frac{d}{dx}$ for both symbolic and numeric integration and differential equation solving. 2 | Starts with some methods we were taught at high school(!) for solving differential equations which appear not to be well known among students today. 3 | Apologies for some inconsistencies in notation. 4 | -------------------------------------------------------------------------------- /argobast.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dpiponi/Operators/3e4dd125fd67145baa1fed0cd85bc3f9f448f340/argobast.png -------------------------------------------------------------------------------- /log.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dpiponi/Operators/3e4dd125fd67145baa1fed0cd85bc3f9f448f340/log.png -------------------------------------------------------------------------------- /operators.bib: -------------------------------------------------------------------------------- 1 | @article{Fornberg1988, 2 | abstract = {Simple recursions are derived for calculating the weights in compact finite difference formulas for any order of derivative and to any order of accuracy on one- dimensional grids with arbitrary spacing. Tables are included for some special cases (of equispaced grids).}, 3 | author = {Fornberg, Bengt}, 4 | doi = {10.1090/S0025-5718-1988-0935077-0}, 5 | file = {:Users/jess/Documents/Mendeley Desktop/Fornberg{\_}1988{\_}Generation of finite difference formulas on arbitrarily spaced grids.pdf:pdf}, 6 | issn = {0025-5718}, 7 | journal = {Mathematics of Computation}, 8 | keywords = {Algorithm,Finite differences,High-order accuracy,Interpolation,Math,Numerical Analysis}, 9 | mendeley-tags = {Algorithm,Finite differences,High-order accuracy,Interpolation,Math,Numerical Analysis}, 10 | number = {184}, 11 | pages = {699--699}, 12 | title = {{Generation of finite difference formulas on arbitrarily spaced grids}}, 13 | url = {http://www.ams.org/journals/mcom/1988-51-184/S0025-5718-1988-0935077-0/ http://amath.colorado.edu/faculty/fornberg/Docs/MathComp{\_}88{\_}FD{\_}formulas.pdf}, 14 | volume = {51}, 15 | year = {1988} 16 | } 17 | @inproceedings {SCA:SCA08:009-018, 18 | booktitle = {Eurographics/SIGGRAPH Symposium on Computer Animation}, 19 | editor = {Markus Gross and Doug James}, 20 | title = {{Low Viscosity Flow Simulations for Animation}}, 21 | author = {Molemaker, Jeroen and Cohen, Jonathan M. and Patel, Sanjit and Noh, Jonyong}, 22 | year = {2008}, 23 | publisher = {The Eurographics Association}, 24 | ISSN = {1727-5288}, 25 | ISBN = {978-3-905674-10-1}, 26 | DOI = {10.2312/SCA/SCA08/009-018} 27 | } 28 | -------------------------------------------------------------------------------- /operators.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dpiponi/Operators/3e4dd125fd67145baa1fed0cd85bc3f9f448f340/operators.pdf -------------------------------------------------------------------------------- /operators.tex: -------------------------------------------------------------------------------- 1 | \documentclass[a4paper]{article} 2 | 3 | %% Language and font encodings 4 | \usepackage[english]{babel} 5 | \usepackage[utf8x]{inputenc} 6 | \usepackage[T1]{fontenc} 7 | \usepackage{epigraph} 8 | \usepackage{float} 9 | \usepackage{graphicx} 10 | \usepackage{amsmath} 11 | \usepackage{amsthm} 12 | \usepackage{amstext} % for \text macro 13 | \usepackage{array} % for \newcolumntype macro 14 | \usepackage{tabu} 15 | \newcolumntype{R}{>{$}r<{$}} % math-mode version of ``l" column type 16 | 17 | %% Sets page size and margins 18 | \usepackage[a4paper,top=3cm,bottom=2cm,left=3cm,right=3cm,marginparwidth=1.75cm]{geometry} 19 | 20 | %% Useful packages 21 | \usepackage{amsmath} 22 | \usepackage{graphicx} 23 | %\usepackage{url} 24 | \usepackage{hyperref} 25 | \usepackage[colorinlistoftodos]{todonotes} 26 | \usepackage{hyperref} 27 | \usepackage{pgfplots} 28 | \pgfplotsset{compat=1.9} 29 | 30 | \usepackage{tikz} 31 | \usetikzlibrary{decorations.pathmorphing,patterns} 32 | \usetikzlibrary{external} 33 | \tikzexternalize 34 | \tikzset{mystyle/.style={shape=circle,fill=black,scale=0.3}} 35 | \tikzset{withtext/.style={fill=white}} 36 | 37 | \theoremstyle{definition} 38 | \newtheorem{example}{Example}[section] 39 | 40 | \title{Operator methods for integrals and differential equations} 41 | \author{Dan Piponi} 42 | 43 | \begin{document} 44 | \maketitle 45 | 46 | \epigraph{In memory of Tom Mower who started me on this journey more than three decades ago} 47 | 48 | \begin{abstract} 49 | Some unpolished notes on abusing the differential operator $D=\frac{d}{dx}$ for both symbolic and numeric integration and differential equation solving. 50 | Starts with some methods we were taught at high school(!) for solving differential equations which appear not to be well known among students today. 51 | Apologies for some inconsistencies in notation. 52 | \end{abstract} 53 | 54 | \section{Warm up} 55 | \subsection{A differential equation} 56 | Suppose we are give the differential equation: 57 | \[ 58 | \frac{df}{dx}+f = x^2 59 | \] 60 | Let's use the shorthand $D=\frac{d}{dx}$ and rewrite this as 61 | \[ 62 | (D+1)f = x^2 63 | \] 64 | The standard approach is first to solve the homogeneous form of the equation 65 | \[ 66 | (D+1)f = 0 67 | \] 68 | giving $f(x) = A\exp(-x)$ for some constant $A$. 69 | We then have to find a \emph{particular integral}, i.e. any solution to the original equation. 70 | The full solution to the original equation is the particular integral plus $A\exp(-x)$. 71 | 72 | So how do we find a particular integral? 73 | In my experience students are encouraged to use educated guesswork. 74 | Anything goes as long as you prove that what you have found is indeed a solution. 75 | So in this case a popular approach might be to assume $f(x)=Ax^2+Bx+C$, substitute into the equation, and solve for $A$, $B$ and $C$. 76 | 77 | Here's a more direct way: 78 | \begin{eqnarray*} 79 | (D+1)f & = & x^2 \\ 80 | f & = & \frac{1}{1+D}x^2 \\ 81 | & = & (1-D+D^2-D^3-\ldots)x^2 \\ 82 | & = & x^2-2x+2 \\ 83 | \end{eqnarray*} 84 | We have performed two unusual operations: 85 | \begin{enumerate} 86 | \item we've ``divided'' by $1+D$ and 87 | \item we've expanded $(1+D)^{-1}$ as a power series. 88 | \end{enumerate} 89 | 90 | \subsection{An integral} 91 | Another example. Suppose we wish to find the integral: 92 | \[ 93 | \int x^3\exp(2x) dx 94 | \] 95 | Integration is the ``inverse'' of differentiation so let's rewrite this as 96 | \[ 97 | \frac{1}{D}x^3\exp(2x) 98 | \] 99 | Now we use a trick known as the shift rule. It states 100 | \[ 101 | f(D)\exp(ax) = \exp(ax)f(D+a) 102 | \] 103 | So we can rewrite our integral as 104 | \begin{eqnarray} 105 | & \exp(2x)\frac{1}{D+2}x^3 \\ 106 | = & \exp(2x)\frac{1}{2}\frac{1}{1+D/2}x^3 \\ 107 | = & \exp(2x)\frac{1}{2}(1-\frac{D}{2}+(\frac{D}{2})^2-(\frac{D}{2})^3+\ldots)x^3 \\ 108 | %= & \frac{1}{2}\exp(2x)(x^3-\frac{3}{2}x^2+\frac{3}{2}x-\frac{3}{4}) \\ 109 | = & \frac{1}{8}\exp(2x)(4x^3-6x^2+6x-3) \\ 110 | \end{eqnarray} 111 | We have used the same division and power series operations as above. 112 | And of course for the full answer we need to add a constant $C$. 113 | 114 | My goal here is to sketch (without full rigour) why you might expect such methods to work and give many more examples. 115 | I also want to show how you can stretch these methods to give some hand-wavey arguments for some well-known theorems. 116 | 117 | \section{Some justification} 118 | \subsection{Truncated power series} 119 | Let $P(x)$ be a polynomial in $x$. 120 | Suppose we can expand $P(x)^{-1}$ as a power series with some non-empty circle of convergence: 121 | \[ 122 | \frac{1}{P(x)} = \sum_{i=0}^\infty a_ix^i 123 | \] 124 | We have that 125 | \[ 126 | P(x)\sum_{i=0}^\infty a_ix^i = 1 127 | \] 128 | so 129 | \[ 130 | P(x)(\sum_{i=0}^{n-1} a_ix^i+\sum_{i=n}^\infty a_ix^i) = 1 131 | \] 132 | and 133 | \[ 134 | P(x)\sum_{i=0}^{n-1} a_ix^i = 1-P(x)\sum_{i=n}^\infty a_ix^i 135 | \] 136 | The left hand side 137 | \[ 138 | P(x)\sum_{i=0}^{n-1} a_ix^i 139 | \] 140 | is a polynomial. 141 | It must be $1$ followed by terms of degree $n$ or higher. 142 | 143 | For example 144 | \begin{eqnarray*} 145 | (1+x)(1-x+x^2-x^3+x^4) & = & 1-x+x^2-x^3+x^4-x^2+x^3-x^4+x^5 \\ 146 | & = & 1+x^5 147 | \end{eqnarray*} 148 | 149 | This tells us that we can use truncated power series to compute reciprocals of polynomials as long as we don't mind some higher order terms. 150 | The nice thing about this is that we know that for any polynomial $Q(x)$, $D^nQ(x) = 0$ for $n$ large enough. 151 | So we can use truncated power series in $D$ to obtain exact results when applied to polynomials in $x$. 152 | 153 | Let me rewrite the first argument above in a more rigorous way: 154 | \begin{eqnarray*} 155 | (D+1)f & = & x^2 \\ 156 | (1-D+D^2)(D+1)f & = & (1-D+D^2)x^2 \\ 157 | 1f & = & x^2-2x+2 \mbox{ (assuming that $f$ has no terms beyond $x^2$)} \\ 158 | \end{eqnarray*} 159 | 160 | We can see this as justifying the use of the operator $(D+1)^{-1}$ as long as we remember that the original argument is just a kind of shorthand for the rigorous one. 161 | 162 | \subsection{The shift rule} 163 | By the Leibniz rule we have 164 | \begin{eqnarray*} 165 | \frac{d}{dx}(\exp(ax)f(x)) & = & \exp(ax)\frac{df(x)}{dx}+a\exp(ax)f(x) 166 | \end{eqnarray*} 167 | We can rewrite this as 168 | \begin{eqnarray*} 169 | D(\exp(ax)f(x)) & = & \exp(ax)(Df(x)+af(x)) \\ 170 | & = & \exp(ax)(D+a)f(x) 171 | \end{eqnarray*} 172 | 173 | We can write this even more compactly as 174 | \[ 175 | D\exp(ax) = \exp(ax)(D+a) 176 | \] 177 | as long as we remember that in this version there is an implication that the $\exp(ax)$ on the left hand side is intended to be multiplied by some further expresion to its right. 178 | 179 | We can prove more. 180 | \begin{eqnarray*} 181 | D^n\exp(ax) & = & D^{n-1}\exp(ax)(D+a) \\ 182 | & = & D^{n-2}\exp(ax)(D+a)^2 \\ 183 | & \vdots & \\ 184 | & = & \exp(ax)(D+a)^n \\ 185 | \end{eqnarray*} 186 | More generally we have The Shift Rule 187 | \[ 188 | \boxed{f(D)\exp(ax) = \exp(ax)f(D+a)} 189 | \] 190 | for any polynomial $f$. 191 | We also expect this to hold in situations where $f$ is a power series but we know it's being applied to a polynomial so terms beyond a certain point contribute zero. 192 | 193 | \subsection{Integration again} 194 | Now consider our integral above. 195 | We can consider this as a solution to the differential equation 196 | \[ 197 | Df = \exp(2x)x^3 198 | \] 199 | Using the shift rule we have 200 | \begin{eqnarray*} 201 | Df & = & \exp(2x)x^3 \\ 202 | \exp(2x)(D+2)\exp(-2x)f(x) & = & \exp(2x)x^2 \\ 203 | (D+2)\exp(-2x)f(x) & = & x^2 \\ 204 | \end{eqnarray*} 205 | Writing $g(x) = \exp(-2x)f(x)$ we can now use the differential equation solving methods above to solve $(D+2)g = x^2$ and our final integral is given by $\exp(2x)g(x)$. 206 | Again, we can see the original argument as being shorthand for this more rigorous argument. 207 | 208 | \section{Diagonalisation} 209 | The functions $\sin$, $\cos$ and $\exp$ ``diagonalise'' various differential operators. 210 | This means that differential operators act on these functions just like multiplication by some number. 211 | (That's what \emph{diagonalisation} means - finding elements that operators act on like real numbers.) 212 | For example $D\exp(ax) = a\exp(ax)$. 213 | But we also have $D^n\exp(ax) = a^n\exp(ax)$ and so 214 | \[ 215 | f(D)\exp(ax) = f(a)\exp(ax) 216 | \] 217 | You can see this is a special case of the shift rule applied to $f(D)(\exp(ax)\times1)$. 218 | 219 | We also have $D^2\sin(ax) = -a^2\sin(ax)$ and $D^2\cos(ax) = -a^2\cos(x)$. 220 | So for any polynomial $f$ 221 | \begin{eqnarray*} 222 | f(D^2)\sin(ax) & = & f(-a^2)\sin(ax) \mbox { and} \\ 223 | f(D^2)\cos(ax) & = & f(-a^2)\cos(ax). 224 | \end{eqnarray*} 225 | 226 | \begin{example} 227 | Find a solution to 228 | \[ 229 | \frac{d^2f}{dx^2}-3\frac{df}{dx}+2f = \exp(3x) 230 | \] 231 | Rewrite as 232 | \[ 233 | (D^2-3D+2)f = \exp(3x) 234 | \] 235 | so 236 | \begin{eqnarray*} 237 | f(x) & = & (D^2-3D+2)^{-1}\exp(3x) \\ 238 | & = & (3^2-3\times3+2)^{-1}\exp(3x) \\ 239 | & = & \frac{1}{2}\exp(3x) \\ 240 | \end{eqnarray*} 241 | \end{example} 242 | 243 | \begin{example} 244 | Find a solution to 245 | \[ 246 | (D^2+1)f = \cos(2x) 247 | \] 248 | We have 249 | \begin{eqnarray*} 250 | f(x) & = & (D^2+1)^{-1}\cos(2x) \\ 251 | & = & (-2^2+1)^{-1}\cos(2x) \\ 252 | & = & -\frac{1}{3}\cos(2x) \\ 253 | \end{eqnarray*} 254 | \end{example} 255 | 256 | \section{Lots of examples} 257 | \begin{example} 258 | Find a solution to 259 | \[ 260 | \frac{df}{dx}+f = \sin x 261 | \] 262 | Unfortunately we have an odd power of $D$ applied to the $\sin$ function so we can't directly use the diagonalisation technique. 263 | Instead we write $\sin x$ using Euler's formula: 264 | \begin{eqnarray*} 265 | \frac{1}{1+D}\sin x & = & \frac{1}{1+D}\Im(\exp{ix}) \\ 266 | & = & \Im\big(\frac{1}{1+D}\exp{ix}\big) \mbox{ (using fact that $\Im(D(f)) = D(\Im(f))$)} \\ 267 | & = & \Im\big(\frac{1}{1+i}\exp{ix}\big) \mbox{ (using diagonalisation for $\exp$)} \\ 268 | & = & \Im\big(\frac{1-i}{2}\exp{ix}\big) \\ 269 | & = & \frac{1}{2}(\sin x-\cos x) \\ 270 | \end{eqnarray*} 271 | \end{example} 272 | 273 | \begin{example} 274 | Find a solution to 275 | \[ 276 | \frac{d^3f}{dx^3}-f = \sin x 277 | \] 278 | We want 279 | \[ 280 | \frac{-1}{1-D^3}\sin x 281 | \] 282 | We can use the fact that $D^2\sin x = -\sin x$ to reduce this to: 283 | \[ 284 | \frac{-1}{1+D}\sin x 285 | \] 286 | The solution is just minus the previous example 287 | \[ 288 | f(x) = \frac{1}{2}(\cos x-\sin x) 289 | \] 290 | \end{example} 291 | 292 | \begin{example} 293 | Find a solution to 294 | \[ 295 | \frac{d^2f}{dx^2}-2\frac{df}{dx}+f=e^x 296 | \] 297 | The only slight subtlety here is noticing that when we use the shift rule, we slide $f(D)$ past $\exp x$ leaving behind a 1 that needs to be integrated twice. 298 | \begin{eqnarray*} 299 | (D^2-2D+1)f & = & e^x \\ 300 | f & = & \frac{1}{(D-1)^2} e^x \\ 301 | & = & e^x\frac{1}{D^2}1 \\ 302 | & = & \frac{x^2}{2}e^x 303 | \end{eqnarray*} 304 | \end{example} 305 | 306 | \begin{example} 307 | Find a solution to 308 | \[ 309 | 4\frac{d^2f}{dx^2}+f = x\exp(-x) 310 | \] 311 | Solution: 312 | \begin{eqnarray*} 313 | \frac{1}{1+4D^2}\exp(-x)x & = & \exp(-x)\frac{1}{1+4(D-1)^2}x \\ 314 | & = & \exp(-x)\frac{1}{4D^2-8D+5}x \\ 315 | & = & \frac{1}{5}\exp(-x)\frac{1}{1-8D/5}x \mbox{ (using $D^2x=0$)}\\ 316 | & = & \frac{1}{5}\exp(-x)(1+\frac{8}{5}D)x \\ 317 | & = & \frac{1}{25}\exp(-x)(5x+8) \\ 318 | \end{eqnarray*} 319 | \end{example} 320 | 321 | \begin{example} 322 | Find a solution to 323 | \[ 324 | \frac{d^2f}{dx^2}+f = x e^{-x}\sin(2x) 325 | \] 326 | \end{example} 327 | No avoiding some messy complex number arithmetic here. We want 328 | \begin{eqnarray*} 329 | \frac{1}{D^2+1}e^{-x}\sin(2x)x 330 | & = & \Im \Big[ \frac{1}{D^2+1}e^{(2i-1)x}x \Big] \\ 331 | & = & e^{-x}\Im \Big[ e^{2ix}\frac{1}{(D+2i-1)^2+1}x \Big] \\ 332 | & = & e^{-x}\Im \Big[ e^{2ix}\frac{-1}{2+4i}\frac{1}{1-\frac{2(2i-1)}{2+4i}D}x \Big] \\ 333 | & = & e^{-x}\Im \Big[ e^{2ix}\frac{-1}{2+4i}(x+\frac{2(2i-1)}{2+4i}) \Big] \\ 334 | & = & \frac{1}{50}e^{-x}\Im \Big[ (\cos(2x)+i\sin(2x))\big((10i-5)x-(11-2i)\big) \Big] \\ 335 | & = & \frac{1}{50}e^{-x}\big((2+10x)\cos(2x)-(11+5x)\sin(2x)\big) \\ 336 | \end{eqnarray*} 337 | 338 | \begin{example} 339 | Find 340 | \[ 341 | \int_0^\infty x^n\exp(-x)dx 342 | \] 343 | First the indefinite integral 344 | \begin{eqnarray*} 345 | & & \frac{1}{D}x^n\exp(-x) \\ 346 | & = & \exp(-x)\frac{1}{D-1}x^n \\ 347 | & = & -\exp(-x)(1+D+D^2+\ldots+D^n)x^n \\ 348 | \end{eqnarray*} 349 | We're going to be evaluating this at zero and in the limit as $x$ goes to infinity. 350 | All of these terms vanish at infinity. 351 | All of the non-constant derivatives of $x^n$ vanish at zero. 352 | So we're left with 353 | \[ 354 | \Big[-\exp(-x)D^nx^n\Big]_0^\infty 355 | \] 356 | This is $n!$. 357 | \end{example} 358 | 359 | \begin{example} 360 | Find a solution to 361 | \[ 362 | \frac{df}{dx}+af(x) = g(x) 363 | \] 364 | \end{example} 365 | This becomes 366 | \begin{eqnarray*} 367 | f(x) & = & \frac{1}{a+D}g(x) \\ 368 | & = & e^{-ax}\frac{1}{D}e^{ax} g(x) \\ 369 | & = & e^{-ax}\int^x e^{ay} g(y) dy \\ 370 | & = & \int_{-\infty}^x e^{a(y-x)} g(y) dy \mbox{ (choosing a particular integral)} \\ 371 | \end{eqnarray*} 372 | The value at $x$ is essentially a weighted sum of the history of $g$ before $x$. 373 | You can think of $f$ as a ``leaky'' integral of $g$ - it would be the integral but $f$ ``leaks'' at a rate proportional to $a$ and $f$. 374 | 375 | \begin{figure} 376 | \centering 377 | \frame{\includegraphics[width=5in]{shift2.png}} 378 | \caption{These techniques are old. This snippet is from Boole's \emph{A Treatise on Differential Equations} from 1859.} 379 | \end{figure} 380 | 381 | \section{Exponentials of $D$} 382 | Taylor's theorem tells us that 383 | \[ 384 | f(x+a) = f(x)+af'(x)+\frac{a^2}{2!}f''(x)+\frac{a^3}{3!}f^{(3)}(x)+\ldots+\frac{a^n}{n!}f^{(n)}(x)+\mbox{ a remainder term} 385 | \] 386 | The form of the remainder depends on the class of function $f$. 387 | For polynomials the remainder is precisely zero for $n$ large enough. 388 | For analytic functions the remainder goes to zero as $n$ goes to infinity so we can write 389 | \[ 390 | f(x+a) = \sum_{n=0}^\infty a^n\frac{D^n}{n!}(f)(x) 391 | \] 392 | We can now write this as 393 | \[ 394 | f(x+a) = (\sum_{n=0}^\infty a^n\frac{D^n}{n!})(f)(x) 395 | \] 396 | and therefore as 397 | \[ 398 | f(x+a) = \exp(aD)(f)(x) 399 | \] 400 | In other words, $\exp(aD)$ is the operator that shifts a function by $a$. 401 | 402 | \section{Sums and recurrence relations} 403 | Armed with exponentials of $D$ we can extend our methods for integrals and differential equations to sums and recurrence relations. 404 | \begin{example} 405 | Find 406 | \[ 407 | \sum_{x=0}^{n-1}x^3 408 | \] 409 | Write $f(x) = \sum_{y=0}^{x-1}y^3$. 410 | Then we want to solve 411 | \[ 412 | f(x+1)-f(x) = x^3 413 | \] 414 | \end{example} 415 | We can now use the methods above: 416 | \begin{eqnarray*} 417 | \exp(D)f-f & = & x^3 \\ 418 | (\exp(D)-1)f & = & x^3 \\ 419 | f & = & \frac{1}{\exp(D)-1}x^3 \\ 420 | f & = & \frac{1}{D+\frac{1}{2}D^2+\frac{1}{3!}D^3+\frac{1}{4!}D^4+\frac{1}{5!}D^5}x^3 \\ 421 | f & = & \frac{1}{1+\frac{1}{2}D+\frac{1}{3!}D^2+\frac{1}{4!}D^3+\frac{1}{5!}D^4}\int x^3dx \\ 422 | f & = & \frac{1}{1+\frac{1}{2}D+\frac{1}{3!}D^2+\frac{1}{4!}D^3+\frac{1}{5!}D^4}\frac{1}{4}x^4 \\ 423 | \end{eqnarray*} 424 | There are tricks we can use to minimise the work here although I'm going to be a bit more explicit than needed so everything is clear. 425 | Analogously to solving differential equations, we are going to end up with a ``particular sum''. 426 | The full solution is going to require determining a constant term. 427 | But it's clear that the sum needs to be zero for $x=0$ so the constant term should be zero. 428 | Applying $D^4$ to $x^4$ is going to give us a constant term. 429 | So we only need to keep terms up to $D^3$. 430 | We get 431 | \begin{eqnarray*} 432 | f & = & \frac{1}{4}(1-(\frac{1}{2}D+\frac{1}{3!}D^2+\frac{1}{4!}D^3)+ 433 | (\frac{1}{2}D+\frac{1}{3!}D^2)^2-(\frac{1}{2}D)^3)x^4 \\ 434 | & = & \frac{1}{4}(1-\frac{1}{2}D-\frac{1}{6}D^2-\frac{1}{24}D^3+ 435 | \frac{1}{4}D^2+\frac{1}{6}D^3-\frac{1}{8}D^3)x^4 \\ 436 | & = & \frac{1}{4}(1-\frac{1}{2}D+\frac{1}{12}D^2)x^4 \mbox{ (nice for us, the cubic term vanishes)} \\ 437 | & = & \frac{x^4}{4}-\frac{x^3}{2}+\frac{x^2}{4} \\ 438 | \end{eqnarray*} 439 | This may seem borderline magical. 440 | One way to think about it is that if we know $f$ is a degree 4 polynomial, then $f(x+1) = f(x)+f'(x)+\frac{1}{2}f''(x)+\frac{1}{3!}f^{(3)}(x)+\frac{1}{4!}f^{(4)}(x)$ exactly. 441 | So solving the summation is equivalent to solving the differential equation 442 | \[ 443 | \frac{1}{4!}\frac{d^4f}{dx^4}+\frac{1}{3!}\frac{d^3f}{dx^3}+\frac{1}{2}\frac{d^2f}{dx^2}+\frac{df}{dx} = x^3 444 | \] 445 | It is entirely reasonable to solve this using differential equation methods. 446 | If you extend this approach beyond polynomials you rediscover the Euler-Maclaurin summation formula. 447 | 448 | \begin{example} 449 | Solve 450 | \[ 451 | a_{n+2}=a_{n+1}+a_n+n^2 \mbox{ with } a_0=0, a_1=0 452 | \] 453 | In this case the problem requires finding a spolution with specific initial conditions. 454 | So we're going to need both a ``particular sum'' and a solution to the homogeneous equation. 455 | It's well known that the solutions to the homogeneous equation are the Fibonacci numbers $F_n$ and the Lucas numbers $L_n$. 456 | Any other solution is a linear combination of these. 457 | The Fibonacci numbers start with $F_0=0$ and $F_1=1$ and the Lucas numbers start with $L_0=2$ and $L_1=1$. 458 | 459 | Let's write $a_x=f(x)+AF_x+BL_x$ so our notation matches what used earlier. 460 | We have 461 | \begin{eqnarray*} 462 | (e^{2D}-e^D-1)f & = & x^2 \\ 463 | f & = & \frac{1}{e^{2D}-e^D-1}x^2 \\ 464 | & = & (-1-D-\frac{5}{2}D^2)x^2 \\ 465 | & = & -x^2-2x-5 466 | \end{eqnarray*} 467 | Because $F_0=0$, the initial condition at $x=0$ immediately implies that $B$ is $\frac{5}{L_0}=\frac{5}{2}$. 468 | The initial condition at $x=1$ now gives $A=\frac{11}{2}$. 469 | The complete solution is 470 | \[ 471 | a_n = \frac{5}{2}L_n+\frac{11}{2}F_n-n^2-2n-5 472 | \] 473 | By the way, Mathematica makes a real mess of this. 474 | I've seen similar methods in a work at least a century old. 475 | Maybe Boole. 476 | \end{example} 477 | 478 | \section{Numerical methods} 479 | \subsection{Derivatives} 480 | The relation $(e^{hD}-1)f(x) = f(x+h)-f(x)$ expresses a finite difference in terms of the differential operator. 481 | We can do this in reverse and derive formulae for derivatives in terms of finite differences. 482 | A good application of this is to derive approximations of derivatives that we can use in numerical methods on a grid. 483 | Define $E_h = e^{hD}$. We'd like to write $D$ in terms of $E_h$. 484 | That seems straightforward. We expect 485 | \[ 486 | D=\frac{1}{h}\log E_h 487 | \] 488 | 489 | The problem is, we can't simply apply a Taylor expansion to $\log E_h$ about $0$. 490 | Suppose we'd like our numerical methods to work well on polynomials - typically giving exact results on low order polynomials. 491 | The operator $E_h-1$ corresponds to finite differencing and we know that repeated finite differences applied to polynomials eventually give zero. 492 | So if we seek a power series in $E_h-1$ we can guarantee convergence on polynomials. 493 | Let's define the finite difference operator $\Delta_h = e^{hD}-1 = E_h-1$. 494 | So we should expand $\log E_h$ around $1$ and use 495 | \[ 496 | D = \log(1+(E_h-1)) 497 | \] 498 | 499 | \begin{figure} 500 | \centering 501 | \frame{\includegraphics[width=5in]{argobast.png}} 502 | \caption{A snippet from Arbogast's \emph{Du calcul des d\'{e}rivations} published in 1800} 503 | \end{figure} 504 | 505 | Let's try a first order expansion. 506 | We get 507 | \[ 508 | \log(1+(E_h-1)) = E_h-1 + \mbox{higher order terms} 509 | \] 510 | In other words we can approximate the derivative of $f$ with 511 | \[ 512 | f'(x) \approx \frac{f(x+h)-f(x)}{h} 513 | \] 514 | As we might expect, this is exact if $f$ is a linear function. 515 | What if we try second order. 516 | \begin{eqnarray*} 517 | \log(1+(E_h-1)) & \approx & E_h-1 - \frac{1}{2}(E_h-1)^2 \\ 518 | & = & -\frac{1}{2}E_h^2+2E_h-\frac{3}{2} 519 | \end{eqnarray*} 520 | So we get 521 | \[ 522 | f'(x) \approx \frac{-f(x+2h)+4f(x+h)-3f(x)}{2h} 523 | \] 524 | This is known as a second-order upwind scheme. 525 | It's ``upwind'' because it's using values of $f$ on one side of $x$ to estimate the derivative. 526 | This is good in numerical methods where you only want to look at one side - for example in simulations of convection where we're trying to estimate what happens in the future and so shouldn't be using information corresponding to fluid that has already come and gone. 527 | But often we want a balanced estimate of the derivative. 528 | But a power series in $e^{hD}$ can't ever refer to points left of $x$. 529 | On the other hand we could try to estimate $f'(x+h)$. 530 | In other words, we should compute $E_hD=e^{hD}D$ in terms of $E_h$. 531 | Clearly 532 | \[ 533 | E_hD = E_h\log E_h 534 | \] 535 | Expanding around $1$ we get 536 | \begin{eqnarray*} 537 | E_hD & = & (1+\Delta_h)\log(1+\Delta_h) \\ 538 | & \approx & (1+\Delta_h)(\Delta_h-\frac{1}{2}\Delta_h^2) \\ 539 | & \approx & \Delta_h+\frac{1}{2}\Delta_h^2 \\ 540 | & = & E_h-1+\frac{1}{2}(E_h-1)^2 \\ 541 | & = & -\frac{1}{2}+\frac{1}{2}E_h^2\\ 542 | \end{eqnarray*} 543 | So $f'(x+h) \approx \frac{1}{h}(f(x+2h)-x(f))$ or 544 | \[ 545 | f'(x) \approx \frac{f(x+h)-f(x-h)}{2h}. 546 | \] 547 | which is the usual central difference formula. 548 | 549 | More generally, we can use the power series of $\Delta_h^m\log\Delta_h$ up to terms in $\Delta_h^k$ to generate an $k$th order estimate for $D$. 550 | Even more generally, we can use the power series of $\Delta_h^m(\log\Delta_h)^n$ to estimate $D^n$. 551 | 552 | \begin{example} 553 | Derive a 3rd order upwind estimate for $D$ using $f(x-h),\ldots,f(x+3h)$. 554 | Solution: 555 | \begin{eqnarray*} 556 | (1+\Delta_h)\log(1+\Delta_h) & \approx & (1+\Delta_h)(\Delta_h-\frac{1}{2}\Delta_h^2+\frac{1}{3}\Delta_h^3) \\ 557 | & = & \Delta_h-\frac{1}{2}\Delta_h^2+\frac{1}{3}\Delta_h^3+\Delta_h^2-\frac{1}{2}\Delta_h^3 \\ 558 | & = & \Delta_h+\frac{1}{2}\Delta_h^2-\frac{1}{6}\Delta_h^3 \\ 559 | & = & E_h-1+\frac{1}{2}(E_h-1)^2-\frac{1}{6}(E_h-1)^3 \\ 560 | & = & E_h-1+\frac{1}{2}(E_h^2-2E_h+1)-\frac{1}{6}(E_h^3-3E_h^2+3E_h-1) \\ 561 | & = & -\frac{1}{3}-\frac{1}{2}E_h+E_h^2-\frac{1}{6}E_h^3 \\ 562 | \end{eqnarray*} 563 | So 564 | \[ 565 | f'(x) = \frac{-2f(x-h)-3f(x)+6f(x+h)-f(x+2h)}{6} 566 | \] 567 | \end{example} 568 | 569 | \begin{example} 570 | Derive a symmetric 4th order order estimate for the third drivative. 571 | Solution: 572 | \begin{eqnarray*} 573 | (1+\Delta_h)^2(\log(1+\Delta_h))^3 & \approx & 574 | (1+\Delta_h)^2(\Delta_h-\frac{1}{2}\Delta_h^2+\frac{1}{3}\Delta_h^3-\frac{1}{4}\Delta_h^4)^2 \\ 575 | & \approx & \Delta_h^3+\frac{1}{2}\Delta_h^4 \\ 576 | & = & \frac{1}{2}(-1+2\Delta_h^2-2\Delta_h^2+\Delta_h^3) \\ 577 | \end{eqnarray*} 578 | So 579 | \[ 580 | f^{(3)}(x) \approx \frac{-f(x-2h)+2f(x-h)-2f(x+h)+f(x+2h)}{2} 581 | \] 582 | \end{example} 583 | 584 | \subsection{A generating function for all one-sided derivative schemes} 585 | Suppose we have a power series $f(x) = a_0+a_1x+a_2x^2+\ldots$. 586 | We can form a generating function for the $n$th partial sum $a_0+a_1x+\ldots+a_{n-1}x^{n-1}$ as 587 | \[ 588 | a_0+k(a_0+a_1x)+k^2(a_0+a_1x+a_2x^2)+\ldots 589 | \] 590 | Assuming absolute convergence can rearrange this as 591 | \begin{eqnarray*} 592 | & & (1+k+k^2+\ldots)a_0 593 | +k(1+k+k^2+\ldots)a_1x 594 | +k^2(1+k+k^2+\ldots)a_2x^2+\ldots \\ 595 | & = & \frac{1}{1-k}(a_0+a_1(kx)+a_2(kx)^2+\ldots) \\ 596 | & = & f(kx)/(1-k) 597 | \end{eqnarray*} 598 | The $n$th order one-sided derivative is derived using $n$ terms from $\log E_h = \log(1+\Delta_h)$. 599 | So the coefficients from the $n$th order scheme are the coefficients of the polynomial in $x$ that forms the coefficient of $k^n$ in $\frac{\log(1+(x-1)k)}{1-k}$. 600 | We can tabulate this as 601 | \begin{center} 602 | \tabulinesep=1.2mm 603 | \begin{tabu}{|c| R R R R R R R R|} 604 | \hline 605 | \text{Order} & \multicolumn{8}{| l |}{Coefficients} \\ 606 | \hline 607 | 0 & 0 & & & & & & & \\ 608 | 1 & -1 & 1 & & & & & & \\ 609 | 2 & \frac{-3}{2} & 2 & \frac{-1}{2} & & & & & \\ 610 | 3 & \frac{-11}{6} & 3 & \frac{-3}{2} & \frac{1}{3} & & & & \\ 611 | 4 & \frac{-25}{12} & 4 & -3 & \frac{4}{3} & \frac{-1}{4} & & & \\ 612 | 5 & \frac{-137}{60} & 5 & -5 & \frac{10}{3} & \frac{-5}{4} & \frac{1}{5} & & \\ 613 | \hline 614 | \end{tabu} 615 | \end{center} 616 | This reproduces the table in \cite{Fornberg1988}. 617 | All of the results in that paper can be reproduced by forming power series from expressions of the form $(1+x)^m(\log(1+x))^n$, with $m$ a half-integer in some cases. 618 | 619 | Incidentally, I used the second order scheme from this table in ILM's GPU based fluid simulator basing my work on \cite{SCA:SCA08:009-018}. 620 | I derived it using the techniques described above but it disagreed with the paper. 621 | It turned out that the paper had an error. 622 | The moral is: it's worth knowing how to derive these schemes even if they appear to be available already in publications. 623 | (That paper is otherwise excellent and contains many powerful methods.) 624 | 625 | \subsection{Caveat} 626 | Before using any of these methods in any kind of differential equation solver please consider doing a von Neumann stability analysis. 627 | It's sometimes hard to guess which methods are and aren't stable. 628 | 629 | \subsection{Numerical Integration} 630 | We can write 631 | \[ 632 | \int_a^b f(x)dx = \Big(\frac{e^{bD}-e^{aD}}{D}f\Big)(0) 633 | \] 634 | There are two ways to see this. 635 | One is to view $\frac{1}{D}f$ as the indefinite integral with $e^{bD}$ and $e^{aD}$ picking out the value of this indefinite integral at $x=b$ and $x=a$. 636 | Another is the following derivation: 637 | \begin{eqnarray*} 638 | \int_a^b f(x)dx & = & \Big(\int_a^b\exp(yD)f\Big)(0) dy \\ 639 | & = & \Big(\int_a^b\exp(yD)dy f\Big)(0) \\ 640 | & = & \Big(\frac{e^{bD}-e^{aD}}{D}f\Big)(0) \\ 641 | \end{eqnarray*} 642 | if you can make yourself to believe that $D$ can behave like an ordinary number in an integration. 643 | Again we use the technique of writing $D=\frac{1}{h}\log E_h$. 644 | 645 | \subsection{Simpson's rule} 646 | Let $E=E_1$, $\Delta=\Delta_1$. 647 | We start with 648 | \[ 649 | \int_0^2f(x)dx = \frac{E^{2D}-1}{D} = \frac{E^2-1}{\log E} 650 | \] 651 | We'll expand up to terms in $\Delta^2$: 652 | 653 | \begin{eqnarray*} 654 | \frac{E^2-1}{\log E} 655 | & = & \frac{(\Delta+1)^2-1}{\Delta-\frac{1}{2}\Delta^2+\frac{1}{3}\Delta^3} \\ 656 | & = & \frac{\Delta^2+2\Delta}{\Delta-\frac{1}{2}\Delta^2+\frac{1}{3}\Delta^3} \\ 657 | & = & \frac{\Delta+2}{1-\frac{1}{2}\Delta+\frac{1}{3}\Delta^2} \\ 658 | & = & (\Delta+2)(1+\frac{1}{2}\Delta-\frac{1}{3}\Delta^2+(\frac{1}{2}\Delta)^2) \\ 659 | & = & (\Delta+2)(1+\frac{1}{2}\Delta-\frac{1}{12}\Delta^2) \\ 660 | & = & \Delta+\frac{1}{2}\Delta^2+2+\Delta-\frac{1}{6}\Delta^2 \\ 661 | & = & 2+2\Delta+\frac{1}{3}\Delta^2 \\ 662 | \end{eqnarray*} 663 | Now substitute $\Delta=E-1$ to get 664 | \begin{eqnarray*} 665 | 2+2(E-1)+\frac{1}{3}(E^2-2E+1) & = &2+2E-2+\frac{1}{3}E^2-\frac{2}{3}E+\frac{1}{3} \\ 666 | & = & \frac{1}{3}+\frac{4}{3}E+\frac{1}{3}E^2 \\ 667 | \end{eqnarray*} 668 | So 669 | \[ 670 | \int_0^2f(x)dx \approx \frac{1}{3}(f(0)+4f(1)+f(2)) 671 | \] 672 | Rescaling the $x$-axis gives: 673 | \[ 674 | \int_a^bf(x)dx \approx \frac{b-a}{6}(f(x_0)+4f(x_1)+f(x_2)) 675 | \] 676 | where $x_0=a$, $x_1=\frac{a+b}{2}$, $x_2=b$. 677 | This is the usual Simpson rule. 678 | Note that we could have kept terms up to $\Delta^3$ in the derivation above and we would have had the same result. 679 | So Simpson's rule is good for cubics as well as quadratics. 680 | 681 | \subsection{Newton-Cotes rules} 682 | The higher order integration rules known as the Newton-Cotes rules can be derived from Taylor expansions of 683 | \[ 684 | \frac{(1+\Delta)^n-1}{n\log(1+\Delta)} 685 | \] 686 | taken to terms in $\Delta^n$ and then substituting $\Delta=E-1$. 687 | For example expanding the $n=4$ case gives 688 | \[ 689 | \frac{1}{90}(7+32E+12E^2+32E^3+7E^4) 690 | \] 691 | These are the coefficients from Boole's integration rule 692 | \[ 693 | \int_a^bf(x)dx \approx \frac{b-a}{90}(7f(x_0)+32f(x_1)+12f(x_2)+32f(x_3)+7f(x_4)) 694 | \] 695 | with the $x_i$ equally spaced from $a$ to $b$. 696 | This is probably the method used by Boole to derive these coefficients. 697 | 698 | \begin{figure} 699 | \centering 700 | \frame{\includegraphics[width=5in]{log.png}} 701 | \caption{A snippet from Boole's \emph{Calculus of Finite Differences}} 702 | \end{figure} 703 | 704 | \subsection{Numerical integration over arbitrary regions} 705 | Suppose we wish to find a numerical method for the integral 706 | \[ 707 | I(f) = \int_D f(x,y)dxdy 708 | \] 709 | where the integral is over some domain $D$ in two dimensions, say. 710 | We'd like to compute this in terms of samples of $f$ at points on some grid. 711 | How can we derive a suitable method? 712 | As we're working in two dimensions, let's define $X=\frac{\partial}{\partial x}$ and $Y=\frac{\partial}{\partial y}$. 713 | Write 714 | \begin{eqnarray*} 715 | I(f) & = & \int_D f(x,y)dxdy \\ 716 | & = & \int_D f(x',y')dx'dy' \\ 717 | & = & \Big(\int_D e^{x'X}e^{y'Y}dx'dy' f\Big)(0) \\ 718 | \end{eqnarray*} 719 | So we need to expand the operator 720 | \[ 721 | \int_D e^{x'X}e^{y'Y}dx'dy' \\ 722 | \] 723 | in terms of $\Delta_x = E_x-1$ and $\Delta_y = E_y-1$ with $E_x=e^X$ and $E_y=e^Y$. 724 | (Note change of notation from $\Delta_h$ I used earlier.) 725 | 726 | As a practical example considering integrating a function over the unit disk using a grid of 25 points. 727 | It's convenient to start with the disk of radius 2 with the $5\times 5$ grid from $(-2,-2)$ to $(2,2)$ illustrated below: 728 | 729 | \begin{figure}[H] 730 | \centering 731 | \begin{tikzpicture}[scale=1] 732 | 733 | \draw[fill=gray!50] (2,2) circle (2cm); 734 | 735 | \foreach \x in {0,...,4} 736 | \foreach \y in {0,...,4} 737 | { 738 | \node[mystyle] (\x-\y) at (\x,\y){}; 739 | } 740 | 741 | \end{tikzpicture} 742 | \end{figure} 743 | 744 | First we compute the integral 745 | \begin{eqnarray*} 746 | U(a,b) & = & \frac{1}{\pi}\int_D \exp(ax+by)dxdy \\ 747 | & = & \frac{1}{\pi}\int_0^{2\pi}\int_0^2 \exp(a\cos\theta+b\cos\theta)rdrd\theta \\ 748 | \end{eqnarray*} 749 | This is in fact a slight variation on the classic integration that gives us the Airy disk in optics. 750 | The result is 751 | \[ 752 | U(a,b) = \frac{4I_1(2\sqrt{a^2+b^2})}{\sqrt{a^2+b^2}} 753 | \] 754 | where $I_1$ is the first order modified Bessel function of the first kind. 755 | 756 | Scaling by $\frac{1}{4}$ so as to get coefficients appropriate for the unit circle, we compute the Taylor series up to 4th order in $\Delta_x$ and $\Delta_y$ in 757 | \[ 758 | \frac{1}{4}(1+\Delta_x)^2(1+\Delta_y)^2U(\log(1+\Delta_x),\log(1+\Delta_y)) 759 | \] 760 | 761 | When this expansion is written in terms of $E_x$ and $E_y$ we get the grid of coefficients: 762 | 763 | \tabulinesep=1.2mm 764 | \begin{center} 765 | $\frac{1}{4320}$ 766 | \begin{tabu}{|c|c|c|c|c|} 767 | \hline 768 | -1 & 34 & 114 & 34 & -1 \\ 769 | \hline 770 | 34 & 464 & 444 & 464 & 34 \\ 771 | \hline 772 | 114 & 444 & -36 & 444 & 114 \\ 773 | \hline 774 | 34 & 464 & 444 & 464 & 34 \\ 775 | \hline 776 | -1 & 34 & 114 & 34 & -1 \\ 777 | \hline 778 | \end{tabu} 779 | \end{center} 780 | 781 | We can try using these coefficents to integrate an example function. 782 | First an approximation computed by another 783 | \[ 784 | \int_D \cos(2x)\log(2+y^2) dx dy \approx 0.477976. 785 | \] 786 | Using the coefficients above we get $0.477383$. 787 | 788 | Note that this example is sub-optimal. 789 | For example, it uses values outside of the integration region. 790 | But for smooth enough functions it gives good results and the method can be adapted to many other problems. 791 | 792 | \section{Quadratic exponentials $\exp(aD^2/2)$} 793 | Consider 794 | \[ 795 | e^{aD^2/2}f 796 | \] 797 | To get some intution, pick a large $N$ and write this as 798 | \[ 799 | (e^{aD^2/2N})^Nf 800 | \] 801 | So this is multiple applications of $\exp(aD^2/2N) \approx 1+\frac{aD^2}{2N}$. 802 | The operator $D^2$ measures convexity. 803 | In areas of convexity, $D^2f$ is positive and so applying $1+\frac{aD^2}{2N}$ will ``push up'' the convexity, smoothing things out. 804 | Conversely, concave areas get pushed down. 805 | So we have a sequence of $N$ steps, each of which reduces concavity or convexity. 806 | In other words, this is a smoothing operation. 807 | 808 | Here is an informal argument. 809 | We start by convolving $f$ with $\exp(-\frac{y^2}{2a})$: 810 | \begin{eqnarray*} 811 | \int_{-\infty}^\infty\exp(-\frac{y^2}{2a}) f(x-y) dy 812 | & = & \int_{-\infty}^\infty\exp(-\frac{y^2}{2a})\exp(-yD) dy f(x) \\ 813 | & = & \int_{-\infty}^\infty\exp(\frac{-(y+aD)^2}{2a})\exp(\frac{aD^2}{2}) dy f(x) \\ 814 | & = & \sqrt{2\pi}\exp(\frac{aD^2}{2}) f(x) \\ 815 | \end{eqnarray*} 816 | We assumed that $\int_{-\infty}^\infty\exp(-(x-a)^2)dx = \sqrt{2\pi}$ even when $a$ is a differential operator. 817 | 818 | So 819 | \[ 820 | \exp{(\frac{aD^2}{2})}f(x) = \frac{1}{\sqrt{2\pi}}\int_{-\infty}^\infty\exp(-\frac{y^2}{2a}) f(x-y) dy 821 | \] 822 | This operation is sometimes called the Weierstrass transform, but better known in the graphics and image processing world as Gaussian blur. 823 | 824 | \section{Dirac deltas of $D$ and the Fourier transform} 825 | We now throw all caution to the wind. 826 | A standard representation of the Dirac delta function is 827 | \[ 828 | 2\pi\delta(x) = \int_{-\infty}^\infty \exp(i\omega x)d\omega 829 | \] 830 | So we have 831 | \begin{eqnarray*} 832 | 2\pi\delta(iD-\omega)f(x) & = & \int_{-\infty}^\infty\exp(iy(iD-\omega))dyf(x) \\ 833 | & = & \int_{-\infty}^\infty\exp(iy(iD-\omega))dyf(x) \\ 834 | & = & \int_{-\infty}^\infty\exp(-iy\omega)f(x-y)dy \\ 835 | & = & \int_{-\infty}^\infty\exp(i(x-y)\omega)f(y)dy \\ 836 | & = & \exp(ix\omega)\int_{-\infty}^\infty\exp(-iy\omega)f(y)dy \\ 837 | & = & \sqrt{2\pi}\exp(ix\omega)\tilde{F}(\omega) 838 | \end{eqnarray*} 839 | where I'm using the definition 840 | \[ 841 | \tilde{F}(\omega) = \frac{1}{\sqrt{2\pi}}\int_{-\infty}^\infty\exp(-ix\omega)f(x)dx \\ 842 | \] 843 | Therefore 844 | \[ 845 | \delta(iD-\omega)f(x) = \frac{1}{\sqrt{2\pi}}\exp(ix\omega)\tilde{F}(\omega) 846 | \] 847 | Notice what it's doing. 848 | It's projecting the original $f$ to a plane wave scaled by the Fourier transform at $\omega$. 849 | It's the projection of $f$ onto the Fourier component corresponding to $\omega$. 850 | We expect to be able to reassemble the original function $f$ by summing these projections back up again. 851 | First on the right hand side: 852 | \begin{eqnarray*} 853 | \int_{-\infty}^\infty\frac{1}{\sqrt{2\pi}}\exp(ix\omega)\tilde{F}(w)d\omega & = & f(x) \\ 854 | \end{eqnarray*} 855 | This is the usual statement of how to invert the Fourier transform. 856 | Now on the left hand side: 857 | \begin{eqnarray*} 858 | \int_{-\infty}^\infty\delta(iD-\omega)d\omega f(x) & = & 859 | \frac{1}{\sqrt{2\pi}}\int_{-\infty}^\infty\delta(iD-\omega)\int_{-\infty}^\infty\tilde{F}(\nu)\exp(i\nu x)d\nu d\omega \\ 860 | & = & \frac{1}{\sqrt{2\pi}}\int_{-\infty}^\infty\int_{-\infty}^\infty\tilde{F}(\nu)\delta(iD-\omega)\exp(i\nu x)d\omega d\nu \\ 861 | & = & \frac{1}{\sqrt{2\pi}}\int_{-\infty}^\infty\int_{-\infty}^\infty\tilde{F}(\nu)\delta(i(i\nu)-\omega)\exp(i\nu x)d\omega d\nu \mbox{ (diagonalisation)}\\ 862 | & = & \frac{1}{\sqrt{2\pi}}\int_{-\infty}^\infty\int_{-\infty}^\infty\tilde{F}(\nu)\delta(-\nu-\omega)\exp(i\nu x)d\omega d\nu \\ 863 | & = & \frac{1}{\sqrt{2\pi}}\int_{-\infty}^\infty\tilde{F}(-\omega)\exp(-i\omega x)d\omega \mbox{ (standard property of $\delta$)} \\ 864 | & = & \frac{1}{\sqrt{2\pi}}\int_{-\infty}^\infty\tilde{F}(\omega)\exp(i\omega x)d\omega \\ 865 | & = & f(x) \\ 866 | \end{eqnarray*} 867 | This illustrates how we're able to work with a Dirac delta of a differential operator much like how you can work with a conventional Dirac delta. 868 | 869 | On its own the Dirac delta of $D$ isn't so useful. 870 | It gets more interesting when you consider $\delta(iD+ax-\omega)$ but that's for another time. 871 | 872 | %(Two mistakes I often make: dropping $2\pi$ and minus signs. 873 | %I may have made both here. 874 | %Must check.) 875 | 876 | %(Also, I think that if $P$ is some quantum mechanical observable, then $\delta(P-p)$ projects a state onto the space of eigenvectors corresponding to eigenvalue $p$ for $P$.) 877 | % 878 | %\section{The Wigner transformation, the Heisenberg group 'n' all that} 879 | %Maybe I'll get onto this\dots 880 | 881 | \section{A short note on integration and differentiation schemes} 882 | The technique for both integration and differentiation schemes is to write your differential operator in terms of finite difference operators, expand as a power series, truncate, and only then rewrite using shift operators. 883 | It's important to first expand in terms of finite differences because high powers of finite differences eventually go to zero when applied to polynomials. 884 | So we have convergence on vector spaces polynomials. 885 | Generic linear combinations of powers of shift operators don't have this property. 886 | 887 | \section{Summary} 888 | \begin{center} 889 | \tabulinesep=1.2mm 890 | \begin{tabu}{|c|c|c|} 891 | \hline 892 | $1$ & identity & $f(x)$ \\ 893 | $D^n$ & $n$th derivative & $f^{(n)}(x)$ \\ 894 | $\exp(aD)$ & shift by $a$ & $f(x+a)$ \\ 895 | $\frac{1}{D}$ & integration & $\int f(x)dx$ \\ 896 | $\frac{1}{D+a}$ & leaky integration & $\int_{-\infty}^x e^{a(y-x)} g(y) dy$ \\ 897 | $\sinh(aD)$ & finite difference & $\frac{1}{2}(f(x+a)-f(x-a))$ \\ 898 | $\exp(aD^2/2)$ & Weierstrass transform & $\int_{-\infty}^\infty \exp(-y^2/2a) f(x+y)dy$ \\ 899 | $\frac{1}{\exp D-1}$ & Euler-Maclaurin sum & $\sum_x f(x)$ \\ 900 | $\delta(iD-\omega)$ & Fourier transform & $\frac{1}{\sqrt{2\pi}}\exp(ix\omega)\tilde{F}(\omega)$ \\ 901 | \hline 902 | \end{tabu} 903 | \end{center} 904 | 905 | \section{More} 906 | So far I've looked at functions of $D$ but it's also possible look at functions of both $x$ and $D$. 907 | For example $\exp(xD)$ is an operator that appears in many places. 908 | However, I have to draw the line somewhere. 909 | So in these notes I've chosen to stick with functions of just $D$ and leave the larger class of operators to a sequel. 910 | 911 | I've left out any mention of fractional powers of $D$. 912 | Whenever I try to read about this subject I mostly find fractional powers of $D$ being used to solve problems about fractional powers of $D$. 913 | But there is a large literature on the subject out there. 914 | 915 | \section{Final thoughts} 916 | There are a couple of other uniform approaches to much of what I've said above. 917 | 918 | One is to use the shift rule to turn your problem into some kind of constraint on a polynomial. 919 | Write the polynomial in general form as $a_0+a_1x+\ldots+a_nx^n$ and write the constraint as a linear system in the vector $(a_0,\ldots,a_n)$. 920 | This is the traditional method taught to students when solving differential equations. 921 | Guess something general enough and solve for the coefficients. 922 | 923 | Another approach, relevant to the numerical methods, is to again assume your problem is about polynomials, and use Lagrange interpolation to fit a polynomial to your data. 924 | You now integrate or differentiate the interpolating polynomial and relate this back as a linear operation on your original data. 925 | 926 | Both of these miss out on the hidden structure given by the rational and transcendental functions of $D$ I've written about here. 927 | 928 | \bibliographystyle{unsrt} 929 | \bibliography{operators} 930 | 931 | \end{document} 932 | -------------------------------------------------------------------------------- /shift2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/dpiponi/Operators/3e4dd125fd67145baa1fed0cd85bc3f9f448f340/shift2.png --------------------------------------------------------------------------------