├── README.md ├── 1. Functions.ipynb ├── 1. Root finding.ipynb ├── 3. Type-based dispatch.ipynb ├── live2 └── 3. Type-based dispatch.ipynb └── solutions ├── 3. Type-based dispatch.ipynb └── 1. Root finding.ipynb /README.md: -------------------------------------------------------------------------------- 1 | # Julia workshop 2 | 3 | ## Lindner College of Business, University of Cincinnati 4 | 5 | ### August 20, 2019 6 | 7 | These are Jupyter notebooks for the Julia workshop at the University of Cincinnati. 8 | They consist of a series of exercises that the attendees should implement. 9 | 10 | The notebooks may be viewed (but not edited) online at [nbviewer](https://nbviewer.jupyter.org/github/dpsanders/cincinnati_2019/tree/master). 11 | 12 | 13 | 14 | 15 | ### Author 16 | - [David P. Sanders](http://sistemas.fciencias.unam.mx/~dsanders), Departamento de Física, Facultad de Ciencias, Universidad Nacional Autónoma de México (UNAM) 17 | 18 | 19 | ### License 20 | Code in this repository is licensed under the MIT license, and text under the This work is licensed under the Creative Commons Attribution-ShareAlike 4.0 International License. 21 | -------------------------------------------------------------------------------- /1. Functions.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Functions in Julia" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "Functions play a crucial role in any programming language, since they allow us to turn a piece of special code into a general, reusable tool.\n", 15 | "\n", 16 | "We will have a short hands-on workshop on functions to cover some of the following tricky topics:" 17 | ] 18 | }, 19 | { 20 | "cell_type": "markdown", 21 | "metadata": {}, 22 | "source": [ 23 | "1. Global vs local variables\n", 24 | "\n", 25 | "\n", 26 | "2. Anonymous functions\n", 27 | "\n", 28 | "\n", 29 | "3. Keyword arguments and overwriting of methods" 30 | ] 31 | } 32 | ], 33 | "metadata": { 34 | "kernelspec": { 35 | "display_name": "Julia 1.1.0", 36 | "language": "julia", 37 | "name": "julia-1.1" 38 | }, 39 | "language_info": { 40 | "file_extension": ".jl", 41 | "mimetype": "application/julia", 42 | "name": "julia", 43 | "version": "1.1.0" 44 | } 45 | }, 46 | "nbformat": 4, 47 | "nbformat_minor": 2 48 | } 49 | -------------------------------------------------------------------------------- /1. Root finding.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Intermediate Julia for scientific computing" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "This workshop is designed to introduce two fundamental concepts in Julia: **types** and **metaprogramming**.\n", 15 | "\n", 16 | "In order to cover various key uses of types in Julia, we have chosen to frame the discussion around a concrete topic in scientific computing, namely **root-finding**. \n", 17 | "The goal is *not* to learn algorithms for root finding *per se*, but rather to have a (pseudo-)real context in which to explore various concepts centered around types and how they arise naturally in real applications of Julia, in particular applications of **multiple dispatch**, which is one of the core choices in Julia that differentiate it from other common languages.\n", 18 | "\n", 19 | "We will implement a couple of root-finding algorithms just to have something to work with. These will just be toy implementations that are far away from the best implementations. \n", 20 | "\n", 21 | "Instead we should use one of the high-quality packages that are available in Julia for this purpose. The large number of them shows the importance of root finding. The ones that I am aware of are the following (in alphabetical order):" 22 | ] 23 | }, 24 | { 25 | "cell_type": "markdown", 26 | "metadata": {}, 27 | "source": [ 28 | "- Single root of a nonlinear function:\n", 29 | " - [`NLsolve.jl`](https://github.com/JuliaNLSolvers/NLsolve.jl)\n", 30 | " - [`Roots.jl`](https://github.com/JuliaMath/Roots.jl)\n", 31 | "\n", 32 | "- All roots of polynomial:\n", 33 | " - [`HomotopyContinuation.jl`](https://www.juliahomotopycontinuation.org)\n", 34 | " - [`PolynomialRoots.jl`](https://github.com/giordano/PolynomialRoots.jl)\n", 35 | " - [`Polynomials.jl`](https://github.com/JuliaMath/Polynomials.jl)\n", 36 | " \n", 37 | "- All roots of a nonlinear function:\n", 38 | " - [`ApproxFun.jl`](https://github.com/JuliaApproximation/ApproxFun.jl)\n", 39 | " - [`IntervalRootFinding.jl`](https://github.com/JuliaIntervals/IntervalRootFinding.jl)\n", 40 | " - [`MDBM.jl`](https://github.com/bachrathyd/MDBM.jl)\n", 41 | " - [`Roots.jl`](https://github.com/JuliaMath/Roots.jl)" 42 | ] 43 | }, 44 | { 45 | "cell_type": "markdown", 46 | "metadata": {}, 47 | "source": [ 48 | "Each of these uses different techniques, with different advantages and disadvantages." 49 | ] 50 | }, 51 | { 52 | "cell_type": "markdown", 53 | "metadata": {}, 54 | "source": [ 55 | "The challenge exercise for the workshop is: develop a package which integrates all of these disparate packages into a coherent whole!" 56 | ] 57 | }, 58 | { 59 | "cell_type": "markdown", 60 | "metadata": {}, 61 | "source": [ 62 | "### Logistics of the workshop" 63 | ] 64 | }, 65 | { 66 | "cell_type": "markdown", 67 | "metadata": {}, 68 | "source": [ 69 | "The workshop is based around a series of exercises to be done during the workshop. We will pause to work on the exercises and then I will discuss possible solutions during the workshop." 70 | ] 71 | }, 72 | { 73 | "cell_type": "markdown", 74 | "metadata": {}, 75 | "source": [ 76 | "These techniques are useful for both users and developers; indeed, in Julia the distinction between users and developers is not useful, since it's much easier than in other languages to join the two categories together." 77 | ] 78 | }, 79 | { 80 | "cell_type": "markdown", 81 | "metadata": {}, 82 | "source": [ 83 | "### Outline" 84 | ] 85 | }, 86 | { 87 | "cell_type": "markdown", 88 | "metadata": {}, 89 | "source": [ 90 | "We will start by quickly reviewing roots of functions and quickly reviewing one of the standard algorithms, **Newton's algorithm**. We will restrict to finding roots of 1D functions for simplicity.\n", 91 | "\n", 92 | "Newton's algorithm requires the calculation of derivatives, for which several choices of algorithm are available. We will see how to encode the choice of algorithm using dispatch.\n", 93 | "\n", 94 | "Then we will define types which will contain all information about a root-finding problem." 95 | ] 96 | }, 97 | { 98 | "cell_type": "markdown", 99 | "metadata": {}, 100 | "source": [ 101 | "## Roots" 102 | ] 103 | }, 104 | { 105 | "cell_type": "markdown", 106 | "metadata": {}, 107 | "source": [ 108 | "Given a function $f: \\mathbb{R} \\to \\mathbb{R}$ (i.e. that accepts a single real number as argument and returns another real number), recall that a **root** or **zero** of the function is a number $x^*$ such that\n", 109 | "\n", 110 | "$$ f(x^*) = 0, $$\n", 111 | "\n", 112 | "i.e. it is a solution of the equation $f(x) = 0$.\n", 113 | "\n", 114 | "In general it is impossible to solve this equation exactly for $x^*$, so we use iterative numerical algorithms instead." 115 | ] 116 | }, 117 | { 118 | "cell_type": "markdown", 119 | "metadata": {}, 120 | "source": [ 121 | "#### Example" 122 | ] 123 | }, 124 | { 125 | "cell_type": "markdown", 126 | "metadata": {}, 127 | "source": [ 128 | "Recall that the function $f$ given by $f(x) := x^2 - 2$ has exactly two roots, at $x^*_1 = +\\sqrt{2}$ and $x^*_2 = -\\sqrt{2}$. Note that it is impossible to represent these values exactly using floating-point arithmetic." 129 | ] 130 | }, 131 | { 132 | "cell_type": "markdown", 133 | "metadata": {}, 134 | "source": [ 135 | "## Newton algorithm" 136 | ] 137 | }, 138 | { 139 | "cell_type": "markdown", 140 | "metadata": {}, 141 | "source": [ 142 | "The Newton algorithm for (possibly) finding a root of a nonlinear function $f(x)$ in 1D is the following iteration:" 143 | ] 144 | }, 145 | { 146 | "cell_type": "markdown", 147 | "metadata": {}, 148 | "source": [ 149 | "$$x_{n+1} = x_n - \\frac{f(x_n)}{f'(x_n)},$$\n", 150 | "\n", 151 | "where $f'$ is the derivative of $f$. We start from an initial guess $x_0$ that can be almost anything (except points for which $f'(x_0) = 0$)." 152 | ] 153 | }, 154 | { 155 | "cell_type": "markdown", 156 | "metadata": {}, 157 | "source": [ 158 | "#### Exercise 1" 159 | ] 160 | }, 161 | { 162 | "cell_type": "markdown", 163 | "metadata": {}, 164 | "source": [ 165 | "1. Implement the Newton algorithm for a fixed number $n$ of steps in a function `newton`, starting from a given starting point $x_0$. \n", 166 | "\n", 167 | " Hint: Which information does the function require?\n", 168 | "\n", 169 | "\n", 170 | "2. Does your function work with other number types, such as `BigFloat`? What do you need in order to run it with those types? Use it to calculate $\\sqrt{2}$. How many decimal places are correct with the standard precision of `BigFloat`?" 171 | ] 172 | }, 173 | { 174 | "cell_type": "code", 175 | "execution_count": 15, 176 | "metadata": {}, 177 | "outputs": [ 178 | { 179 | "data": { 180 | "text/plain": [ 181 | "(1.414213562373095, 10)" 182 | ] 183 | }, 184 | "execution_count": 15, 185 | "metadata": {}, 186 | "output_type": "execute_result" 187 | } 188 | ], 189 | "source": [ 190 | "newton(f, df, 3.0)" 191 | ] 192 | }, 193 | { 194 | "cell_type": "code", 195 | "execution_count": 5, 196 | "metadata": {}, 197 | "outputs": [ 198 | { 199 | "data": { 200 | "text/plain": [ 201 | "1.4142135623730951" 202 | ] 203 | }, 204 | "execution_count": 5, 205 | "metadata": {}, 206 | "output_type": "execute_result" 207 | } 208 | ], 209 | "source": [ 210 | "sqrt(2)" 211 | ] 212 | }, 213 | { 214 | "cell_type": "markdown", 215 | "metadata": {}, 216 | "source": [ 217 | "## Calculating derivatives" 218 | ] 219 | }, 220 | { 221 | "cell_type": "markdown", 222 | "metadata": {}, 223 | "source": [ 224 | "The Newton algorithm requires us to specify the derivative of a function. If $f$ is a complicated function, we certainly don't want to do that by hand.\n", 225 | "\n", 226 | "One standard solution is to use a *finite-difference approximation*:\n", 227 | "\n", 228 | "$$f'(a) \\simeq \\frac{f(a + h) - f(a - h)}{2h}.$$" 229 | ] 230 | }, 231 | { 232 | "cell_type": "markdown", 233 | "metadata": {}, 234 | "source": [ 235 | "#### Exercise 2" 236 | ] 237 | }, 238 | { 239 | "cell_type": "markdown", 240 | "metadata": {}, 241 | "source": [ 242 | "1. Implement a function `finite_difference` with a default value $h = 0.001$ to calculate $f'(a)$ at a given point $a$.\n", 243 | "\n", 244 | "\n", 245 | "2. Use an anonymous function to make a method of `finite_difference` that calculates the *function* $f'$.\n", 246 | "\n", 247 | "\n", 248 | "3. Implement a version of `newton` that does not take the derivative as argument and uses `finite_difference` to calculate the derivative. This version of `newton` should **re-use** the previous version by defining the function `fp` and calling that version." 249 | ] 250 | }, 251 | { 252 | "cell_type": "markdown", 253 | "metadata": {}, 254 | "source": [ 255 | "`newton` is a **generic function**" 256 | ] 257 | }, 258 | { 259 | "cell_type": "code", 260 | "execution_count": 27, 261 | "metadata": {}, 262 | "outputs": [ 263 | { 264 | "data": { 265 | "text/html": [ 266 | "3 methods for generic function newton:" 267 | ], 268 | "text/plain": [ 269 | "# 3 methods for generic function \"newton\":\n", 270 | "[1] newton(f, x0) in Main at In[24]:3\n", 271 | "[2] newton(f, df, x0) in Main at In[12]:5\n", 272 | "[3] newton(f, df, x0, n) in Main at In[12]:5" 273 | ] 274 | }, 275 | "execution_count": 27, 276 | "metadata": {}, 277 | "output_type": "execute_result" 278 | } 279 | ], 280 | "source": [ 281 | "methods(newton)" 282 | ] 283 | }, 284 | { 285 | "cell_type": "markdown", 286 | "metadata": {}, 287 | "source": [ 288 | "### Algorithmic differentiation" 289 | ] 290 | }, 291 | { 292 | "cell_type": "markdown", 293 | "metadata": {}, 294 | "source": [ 295 | "An alternative way to calculate derivatives is by using [**algorithmic differentiation**](https://en.wikipedia.org/wiki/Automatic_differentiation) (also called **automatic differentiation** or **computational differentiation**). This gives exact results (up to rounding error).\n", 296 | "\n", 297 | "\n", 298 | "We will implement this algorithm in the next notebook, but for now let's just use the implementation in the excellent [`ForwardDiff.jl` package](https://github.com/JuliaDiff/ForwardDiff.jl).\n" 299 | ] 300 | }, 301 | { 302 | "cell_type": "markdown", 303 | "metadata": {}, 304 | "source": [ 305 | "#### Exercise 3" 306 | ] 307 | }, 308 | { 309 | "cell_type": "markdown", 310 | "metadata": {}, 311 | "source": [ 312 | "1. Install `ForwardDiff.jl` if necessary.\n", 313 | "\n", 314 | "\n", 315 | "2. Import it.\n", 316 | "\n", 317 | "\n", 318 | "3. Define a function `forwarddiff` that uses the `ForwardDiff.derivative` function to calculate a derivative." 319 | ] 320 | }, 321 | { 322 | "cell_type": "markdown", 323 | "metadata": {}, 324 | "source": [ 325 | "### Choosing between algorithms" 326 | ] 327 | }, 328 | { 329 | "cell_type": "markdown", 330 | "metadata": {}, 331 | "source": [ 332 | "We now have two different algorithms available to calculate derivatives. This kind of situation is common in scientific computing; for example, the [`DifferentialEquations.jl`](http://docs.juliadiffeq.org/latest/) ecosystem has some 300 algorithms for solving differential equations. One of the techniques we will learn is how to easily be able to specify different algorithms." 333 | ] 334 | }, 335 | { 336 | "cell_type": "markdown", 337 | "metadata": {}, 338 | "source": [ 339 | "#### Exercise 4" 340 | ] 341 | }, 342 | { 343 | "cell_type": "markdown", 344 | "metadata": {}, 345 | "source": [ 346 | "1. Make a version of the Newton algorithm that takes an argument which is the method (algorithm) to use to calculate the derivative, given as a function. \n", 347 | "The new method should have the signature `newton(f, df, x0, n, derivative)`.\n", 348 | "(The **signature** of a function means the collection of arguments that it takes.)" 349 | ] 350 | }, 351 | { 352 | "cell_type": "markdown", 353 | "metadata": {}, 354 | "source": [ 355 | "## A first taste of multiple dispatch" 356 | ] 357 | }, 358 | { 359 | "cell_type": "markdown", 360 | "metadata": {}, 361 | "source": [ 362 | "In the above, we ended up with a complicated signature. Maybe we would like something simpler, such as \n", 363 | "```\n", 364 | "newton(f, x0, derivative)\n", 365 | "```\n", 366 | "where `derivative` is the derivative method to use (finite differencing or `forwarddiff`).\n", 367 | "\n", 368 | "The problem is that we already have a method for `newton` that takes three arguments, namely `newton(f, df, x0)`. If we define this new method, we will *overwrite* (destroy) that method, since Julia cannot distinguish between the signature `(f, x0, derivative)` and `(f, df, x0)` -- they are both simply three arguments with different names." 369 | ] 370 | }, 371 | { 372 | "cell_type": "markdown", 373 | "metadata": {}, 374 | "source": [ 375 | "There is a way to distinguish them, however: we (as humans) know intuitively that the first method, `(f, df, x0)`, should take arguments of types `(function, function, number)`, whereas `(f, x0, derivative)` should take `(function, number, function)`. So far, however, we have not told Julia this, since although we recognise by eye that `f` is a function and `x0` is a number, for Julia it is quite possible for `f` to be a number and `x0` to be a function!" 376 | ] 377 | }, 378 | { 379 | "cell_type": "markdown", 380 | "metadata": {}, 381 | "source": [ 382 | "So what we need is a mechanism to *specify* to Julia which *type* of arguments each version of `newton` takes, in which order." 383 | ] 384 | }, 385 | { 386 | "cell_type": "markdown", 387 | "metadata": {}, 388 | "source": [ 389 | "### Type annotations" 390 | ] 391 | }, 392 | { 393 | "cell_type": "markdown", 394 | "metadata": {}, 395 | "source": [ 396 | "When we need to specify types in Julia, we use a *type annotation*, written `::T`, where `T` is the type.\n", 397 | "\n", 398 | "For example, let's define a function `rounded_square` of one argument, `x`, that calculates a rounded-down square. If `x` is an integer then it should just return `x^2`; but if `x` is a float, it should do a more complicated operation. We can write two *methods* for `rounded_square` as follows:" 399 | ] 400 | }, 401 | { 402 | "cell_type": "code", 403 | "execution_count": 5, 404 | "metadata": {}, 405 | "outputs": [ 406 | { 407 | "data": { 408 | "text/plain": [ 409 | "rounded_square (generic function with 2 methods)" 410 | ] 411 | }, 412 | "execution_count": 5, 413 | "metadata": {}, 414 | "output_type": "execute_result" 415 | } 416 | ], 417 | "source": [ 418 | "rounded_square(x::Int) = x^2\n", 419 | "\n", 420 | "rounded_square(x::Float64) = floor(x^2)" 421 | ] 422 | }, 423 | { 424 | "cell_type": "code", 425 | "execution_count": 4, 426 | "metadata": {}, 427 | "outputs": [ 428 | { 429 | "data": { 430 | "text/plain": [ 431 | "9" 432 | ] 433 | }, 434 | "execution_count": 4, 435 | "metadata": {}, 436 | "output_type": "execute_result" 437 | } 438 | ], 439 | "source": [ 440 | "rounded_square(3)" 441 | ] 442 | }, 443 | { 444 | "cell_type": "code", 445 | "execution_count": 6, 446 | "metadata": {}, 447 | "outputs": [ 448 | { 449 | "data": { 450 | "text/plain": [ 451 | "9.0" 452 | ] 453 | }, 454 | "execution_count": 6, 455 | "metadata": {}, 456 | "output_type": "execute_result" 457 | } 458 | ], 459 | "source": [ 460 | "rounded_square(3.1)" 461 | ] 462 | }, 463 | { 464 | "cell_type": "markdown", 465 | "metadata": {}, 466 | "source": [ 467 | "But:" 468 | ] 469 | }, 470 | { 471 | "cell_type": "code", 472 | "execution_count": 9, 473 | "metadata": {}, 474 | "outputs": [ 475 | { 476 | "data": { 477 | "text/plain": [ 478 | "9.0" 479 | ] 480 | }, 481 | "execution_count": 9, 482 | "metadata": {}, 483 | "output_type": "execute_result" 484 | } 485 | ], 486 | "source": [ 487 | "rounded_square(big\"3.1\")" 488 | ] 489 | }, 490 | { 491 | "cell_type": "markdown", 492 | "metadata": {}, 493 | "source": [ 494 | "We see that we restricted the second method too much: really we would like to allow any real number:" 495 | ] 496 | }, 497 | { 498 | "cell_type": "code", 499 | "execution_count": 8, 500 | "metadata": {}, 501 | "outputs": [ 502 | { 503 | "data": { 504 | "text/plain": [ 505 | "rounded_square (generic function with 3 methods)" 506 | ] 507 | }, 508 | "execution_count": 8, 509 | "metadata": {}, 510 | "output_type": "execute_result" 511 | } 512 | ], 513 | "source": [ 514 | "rounded_square(x::Real) = floor(x^2)" 515 | ] 516 | }, 517 | { 518 | "cell_type": "code", 519 | "execution_count": 10, 520 | "metadata": {}, 521 | "outputs": [ 522 | { 523 | "data": { 524 | "text/plain": [ 525 | "9.0" 526 | ] 527 | }, 528 | "execution_count": 10, 529 | "metadata": {}, 530 | "output_type": "execute_result" 531 | } 532 | ], 533 | "source": [ 534 | "rounded_square(big\"3.1\")" 535 | ] 536 | }, 537 | { 538 | "cell_type": "code", 539 | "execution_count": 11, 540 | "metadata": {}, 541 | "outputs": [ 542 | { 543 | "data": { 544 | "text/plain": [ 545 | "9.0" 546 | ] 547 | }, 548 | "execution_count": 11, 549 | "metadata": {}, 550 | "output_type": "execute_result" 551 | } 552 | ], 553 | "source": [ 554 | "rounded_square(π)" 555 | ] 556 | }, 557 | { 558 | "cell_type": "markdown", 559 | "metadata": {}, 560 | "source": [ 561 | "If we later discover other cases that we would like to be covered, we can *add new methods* to the function, even for new kinds of types that we define (although if they are subtypes of `Real` then they *are already covered*!)" 562 | ] 563 | }, 564 | { 565 | "cell_type": "markdown", 566 | "metadata": {}, 567 | "source": [ 568 | "### Multiple dispatch" 569 | ] 570 | }, 571 | { 572 | "cell_type": "markdown", 573 | "metadata": {}, 574 | "source": [ 575 | "Julia checks the types of *all arguments of a function* and chooses a method that matches all of them simultaneously. This is known as **multiple dispatch**. (\"Dispatch\" is the act of choosing which version of a function to call.)\n", 576 | "\n", 577 | "Although this does not necessarily sound like a complicated idea, it is one of the key things that differentiates Julia from most other programming languages, and it has led to many interesting developments; I highly recommend [Stefan Karpinski's talk from JuliaCon 2019](https://www.youtube.com/watch?v=kc9HwsxE1OY)." 578 | ] 579 | }, 580 | { 581 | "cell_type": "markdown", 582 | "metadata": {}, 583 | "source": [ 584 | "Let's return to the `newton` example. " 585 | ] 586 | }, 587 | { 588 | "cell_type": "markdown", 589 | "metadata": {}, 590 | "source": [ 591 | "#### Exercise 5\n", 592 | "\n", 593 | "1. Write a method for `newton` that takes as arguments `f`, `x0` and a `derivative` method by annotating the `derivative` method as being of type `Function`. \n", 594 | "\n", 595 | "\n", 596 | "2. Alternatively, annotate the method which takes `f`, `df` and `x0` by annotating `x0` as being `Real`." 597 | ] 598 | } 599 | ], 600 | "metadata": { 601 | "kernelspec": { 602 | "display_name": "Julia 1.1.0", 603 | "language": "julia", 604 | "name": "julia-1.1" 605 | }, 606 | "language_info": { 607 | "file_extension": ".jl", 608 | "mimetype": "application/julia", 609 | "name": "julia", 610 | "version": "1.1.0" 611 | } 612 | }, 613 | "nbformat": 4, 614 | "nbformat_minor": 2 615 | } 616 | -------------------------------------------------------------------------------- /3. Type-based dispatch.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Structuring programs using types and dispatch" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "Previously we specified an algorithm directly using a function. In many Julia packages it is common to use *dispatch* to do this. \n", 15 | "\n", 16 | "In Julia, **dispatch** refers to choosing which **method** (version) of a function to use, according to the type of the arguments. (**Multiple dispatch** is when the types of several different arguments are involved.)" 17 | ] 18 | }, 19 | { 20 | "cell_type": "markdown", 21 | "metadata": {}, 22 | "source": [ 23 | "Let's define some types to represent different differentiation methods." 24 | ] 25 | }, 26 | { 27 | "cell_type": "code", 28 | "execution_count": 5, 29 | "metadata": {}, 30 | "outputs": [], 31 | "source": [ 32 | "struct Dual \n", 33 | " v::Float64\n", 34 | " d::Float64\n", 35 | "end" 36 | ] 37 | }, 38 | { 39 | "cell_type": "code", 40 | "execution_count": 6, 41 | "metadata": {}, 42 | "outputs": [ 43 | { 44 | "data": { 45 | "text/html": [ 46 | "2 methods for generic function Type:" 47 | ], 48 | "text/plain": [ 49 | "# 2 methods for generic function \"(::Type)\":\n", 50 | "[1] Dual(v::Float64, d::Float64) in Main at In[5]:2\n", 51 | "[2] Dual(v, d) in Main at In[5]:2" 52 | ] 53 | }, 54 | "execution_count": 6, 55 | "metadata": {}, 56 | "output_type": "execute_result" 57 | } 58 | ], 59 | "source": [ 60 | "methods(Dual)" 61 | ] 62 | }, 63 | { 64 | "cell_type": "markdown", 65 | "metadata": {}, 66 | "source": [ 67 | "#### Exercise 1\n", 68 | "\n", 69 | "1. Define an abstract type `DifferentiationAlgorithm`.\n", 70 | "\n", 71 | "\n", 72 | "2. Define subtypes `FiniteDifference`, `MyAutoDiff` (for our implementation) and `AutoDiff` (for the `ForwardDiff` implementation).\n", 73 | "\n", 74 | "\n", 75 | "3. Implement the function `derivative(f, x, algorithm)` using **dispatch**: for each of the three types, define a version of this function in which `algorithm` is specified to be of that type by using the type annotation operator `::`.\n", 76 | "\n", 77 | "\n", 78 | "4. Verify that these work by writing tests for them." 79 | ] 80 | }, 81 | { 82 | "cell_type": "markdown", 83 | "metadata": {}, 84 | "source": [ 85 | "You could do:" 86 | ] 87 | }, 88 | { 89 | "cell_type": "code", 90 | "execution_count": 4, 91 | "metadata": {}, 92 | "outputs": [ 93 | { 94 | "ename": "LoadError", 95 | "evalue": "syntax: invalid identifier name \"...\"", 96 | "output_type": "error", 97 | "traceback": [ 98 | "syntax: invalid identifier name \"...\"", 99 | "" 100 | ] 101 | } 102 | ], 103 | "source": [ 104 | "function derivative(f, x, algorithm)\n", 105 | " if algorithm == :newton\n", 106 | " ...\n", 107 | " elseif algorithm == :bisection\n", 108 | " ...\n", 109 | " elseif\n", 110 | " ...\n", 111 | " end\n", 112 | "end" 113 | ] 114 | }, 115 | { 116 | "cell_type": "code", 117 | "execution_count": 1, 118 | "metadata": {}, 119 | "outputs": [], 120 | "source": [ 121 | "abstract type DifferentiationAlgorithm end" 122 | ] 123 | }, 124 | { 125 | "cell_type": "code", 126 | "execution_count": 2, 127 | "metadata": {}, 128 | "outputs": [], 129 | "source": [ 130 | "struct FiniteDifference <: DifferentiationAlgorithm \n", 131 | " h::Float64\n", 132 | "end" 133 | ] 134 | }, 135 | { 136 | "cell_type": "code", 137 | "execution_count": 9, 138 | "metadata": {}, 139 | "outputs": [], 140 | "source": [ 141 | "struct AutoDiff <: DifferentiationAlgorithm end" 142 | ] 143 | }, 144 | { 145 | "cell_type": "code", 146 | "execution_count": 7, 147 | "metadata": {}, 148 | "outputs": [ 149 | { 150 | "data": { 151 | "text/plain": [ 152 | "finite_difference (generic function with 1 method)" 153 | ] 154 | }, 155 | "execution_count": 7, 156 | "metadata": {}, 157 | "output_type": "execute_result" 158 | } 159 | ], 160 | "source": [ 161 | "finite_difference(f, a, h) = (f(a + h) - f(a - h)) / (2h)" 162 | ] 163 | }, 164 | { 165 | "cell_type": "code", 166 | "execution_count": 8, 167 | "metadata": {}, 168 | "outputs": [ 169 | { 170 | "data": { 171 | "text/plain": [ 172 | "forwarddiff (generic function with 1 method)" 173 | ] 174 | }, 175 | "execution_count": 8, 176 | "metadata": {}, 177 | "output_type": "execute_result" 178 | } 179 | ], 180 | "source": [ 181 | "using ForwardDiff\n", 182 | "forwarddiff(f, x) = ForwardDiff.derivative(f, x)" 183 | ] 184 | }, 185 | { 186 | "cell_type": "code", 187 | "execution_count": 10, 188 | "metadata": {}, 189 | "outputs": [ 190 | { 191 | "data": { 192 | "text/plain": [ 193 | "derivative (generic function with 1 method)" 194 | ] 195 | }, 196 | "execution_count": 10, 197 | "metadata": {}, 198 | "output_type": "execute_result" 199 | } 200 | ], 201 | "source": [ 202 | "function derivative(f, x, algorithm::AutoDiff)\n", 203 | " return forwarddiff(f, x)\n", 204 | "end" 205 | ] 206 | }, 207 | { 208 | "cell_type": "code", 209 | "execution_count": 12, 210 | "metadata": {}, 211 | "outputs": [ 212 | { 213 | "data": { 214 | "text/plain": [ 215 | "3.0" 216 | ] 217 | }, 218 | "execution_count": 12, 219 | "metadata": {}, 220 | "output_type": "execute_result" 221 | } 222 | ], 223 | "source": [ 224 | "g(x) = x^2 - 2\n", 225 | "a = 3.0" 226 | ] 227 | }, 228 | { 229 | "cell_type": "code", 230 | "execution_count": 13, 231 | "metadata": {}, 232 | "outputs": [ 233 | { 234 | "ename": "MethodError", 235 | "evalue": "MethodError: no method matching derivative(::typeof(g), ::Float64, ::Type{AutoDiff})\nClosest candidates are:\n derivative(::Any, ::Any, !Matched::AutoDiff) at In[10]:2", 236 | "output_type": "error", 237 | "traceback": [ 238 | "MethodError: no method matching derivative(::typeof(g), ::Float64, ::Type{AutoDiff})\nClosest candidates are:\n derivative(::Any, ::Any, !Matched::AutoDiff) at In[10]:2", 239 | "", 240 | "Stacktrace:", 241 | " [1] top-level scope at In[13]:1" 242 | ] 243 | } 244 | ], 245 | "source": [ 246 | "derivative(g, a, AutoDiff)" 247 | ] 248 | }, 249 | { 250 | "cell_type": "code", 251 | "execution_count": 14, 252 | "metadata": {}, 253 | "outputs": [ 254 | { 255 | "data": { 256 | "text/plain": [ 257 | "DataType" 258 | ] 259 | }, 260 | "execution_count": 14, 261 | "metadata": {}, 262 | "output_type": "execute_result" 263 | } 264 | ], 265 | "source": [ 266 | "typeof(AutoDiff)" 267 | ] 268 | }, 269 | { 270 | "cell_type": "code", 271 | "execution_count": 15, 272 | "metadata": {}, 273 | "outputs": [ 274 | { 275 | "data": { 276 | "text/plain": [ 277 | "AutoDiff()" 278 | ] 279 | }, 280 | "execution_count": 15, 281 | "metadata": {}, 282 | "output_type": "execute_result" 283 | } 284 | ], 285 | "source": [ 286 | "autodiff = AutoDiff()" 287 | ] 288 | }, 289 | { 290 | "cell_type": "code", 291 | "execution_count": 16, 292 | "metadata": {}, 293 | "outputs": [ 294 | { 295 | "data": { 296 | "text/plain": [ 297 | "AutoDiff" 298 | ] 299 | }, 300 | "execution_count": 16, 301 | "metadata": {}, 302 | "output_type": "execute_result" 303 | } 304 | ], 305 | "source": [ 306 | "typeof(autodiff)" 307 | ] 308 | }, 309 | { 310 | "cell_type": "code", 311 | "execution_count": 17, 312 | "metadata": {}, 313 | "outputs": [ 314 | { 315 | "data": { 316 | "text/plain": [ 317 | "6.0" 318 | ] 319 | }, 320 | "execution_count": 17, 321 | "metadata": {}, 322 | "output_type": "execute_result" 323 | } 324 | ], 325 | "source": [ 326 | "derivative(g, a, autodiff)" 327 | ] 328 | }, 329 | { 330 | "cell_type": "code", 331 | "execution_count": 18, 332 | "metadata": {}, 333 | "outputs": [ 334 | { 335 | "data": { 336 | "text/plain": [ 337 | "6.0" 338 | ] 339 | }, 340 | "execution_count": 18, 341 | "metadata": {}, 342 | "output_type": "execute_result" 343 | } 344 | ], 345 | "source": [ 346 | "derivative(g, a, AutoDiff())" 347 | ] 348 | }, 349 | { 350 | "cell_type": "code", 351 | "execution_count": 19, 352 | "metadata": {}, 353 | "outputs": [ 354 | { 355 | "data": { 356 | "text/plain": [ 357 | "derivative (generic function with 2 methods)" 358 | ] 359 | }, 360 | "execution_count": 19, 361 | "metadata": {}, 362 | "output_type": "execute_result" 363 | } 364 | ], 365 | "source": [ 366 | "function derivative(f, x, algorithm::FiniteDifference)\n", 367 | " h = algorithm.h\n", 368 | " return finite_difference(f, x, h)\n", 369 | "end" 370 | ] 371 | }, 372 | { 373 | "cell_type": "code", 374 | "execution_count": 22, 375 | "metadata": {}, 376 | "outputs": [ 377 | { 378 | "ename": "MethodError", 379 | "evalue": "MethodError: no method matching FiniteDifference()\nClosest candidates are:\n FiniteDifference(!Matched::Float64) at In[2]:2\n FiniteDifference(!Matched::Any) at In[2]:2", 380 | "output_type": "error", 381 | "traceback": [ 382 | "MethodError: no method matching FiniteDifference()\nClosest candidates are:\n FiniteDifference(!Matched::Float64) at In[2]:2\n FiniteDifference(!Matched::Any) at In[2]:2", 383 | "", 384 | "Stacktrace:", 385 | " [1] top-level scope at In[22]:1" 386 | ] 387 | } 388 | ], 389 | "source": [ 390 | "derivative(g, a, FiniteDifference())" 391 | ] 392 | }, 393 | { 394 | "cell_type": "code", 395 | "execution_count": 23, 396 | "metadata": {}, 397 | "outputs": [ 398 | { 399 | "data": { 400 | "text/plain": [ 401 | "5.999999999999872" 402 | ] 403 | }, 404 | "execution_count": 23, 405 | "metadata": {}, 406 | "output_type": "execute_result" 407 | } 408 | ], 409 | "source": [ 410 | "derivative(g, a, FiniteDifference(0.01))" 411 | ] 412 | }, 413 | { 414 | "cell_type": "code", 415 | "execution_count": 25, 416 | "metadata": {}, 417 | "outputs": [ 418 | { 419 | "data": { 420 | "text/plain": [ 421 | "FiniteDifference(0.01)" 422 | ] 423 | }, 424 | "execution_count": 25, 425 | "metadata": {}, 426 | "output_type": "execute_result" 427 | } 428 | ], 429 | "source": [ 430 | "algorithm = FiniteDifference(0.01)" 431 | ] 432 | }, 433 | { 434 | "cell_type": "code", 435 | "execution_count": 26, 436 | "metadata": {}, 437 | "outputs": [ 438 | { 439 | "data": { 440 | "text/plain": [ 441 | "FiniteDifference" 442 | ] 443 | }, 444 | "execution_count": 26, 445 | "metadata": {}, 446 | "output_type": "execute_result" 447 | } 448 | ], 449 | "source": [ 450 | "typeof(algorithm)" 451 | ] 452 | }, 453 | { 454 | "cell_type": "code", 455 | "execution_count": 27, 456 | "metadata": {}, 457 | "outputs": [ 458 | { 459 | "data": { 460 | "text/plain": [ 461 | "0.01" 462 | ] 463 | }, 464 | "execution_count": 27, 465 | "metadata": {}, 466 | "output_type": "execute_result" 467 | } 468 | ], 469 | "source": [ 470 | "algorithm.h" 471 | ] 472 | }, 473 | { 474 | "cell_type": "code", 475 | "execution_count": 30, 476 | "metadata": {}, 477 | "outputs": [ 478 | { 479 | "data": { 480 | "text/plain": [ 481 | "(:h,)" 482 | ] 483 | }, 484 | "execution_count": 30, 485 | "metadata": {}, 486 | "output_type": "execute_result" 487 | } 488 | ], 489 | "source": [ 490 | "fieldnames(typeof(algorithm))" 491 | ] 492 | }, 493 | { 494 | "cell_type": "code", 495 | "execution_count": 31, 496 | "metadata": {}, 497 | "outputs": [ 498 | { 499 | "ename": "MethodError", 500 | "evalue": "MethodError: no method matching FiniteDifference()\nClosest candidates are:\n FiniteDifference(!Matched::Float64) at In[2]:2\n FiniteDifference(!Matched::Any) at In[2]:2", 501 | "output_type": "error", 502 | "traceback": [ 503 | "MethodError: no method matching FiniteDifference()\nClosest candidates are:\n FiniteDifference(!Matched::Float64) at In[2]:2\n FiniteDifference(!Matched::Any) at In[2]:2", 504 | "", 505 | "Stacktrace:", 506 | " [1] top-level scope at In[31]:1" 507 | ] 508 | } 509 | ], 510 | "source": [ 511 | "FiniteDifference()" 512 | ] 513 | }, 514 | { 515 | "cell_type": "code", 516 | "execution_count": 32, 517 | "metadata": {}, 518 | "outputs": [ 519 | { 520 | "data": { 521 | "text/html": [ 522 | "2 methods for generic function Type:" 523 | ], 524 | "text/plain": [ 525 | "# 2 methods for generic function \"(::Type)\":\n", 526 | "[1] FiniteDifference(h::Float64) in Main at In[2]:2\n", 527 | "[2] FiniteDifference(h) in Main at In[2]:2" 528 | ] 529 | }, 530 | "execution_count": 32, 531 | "metadata": {}, 532 | "output_type": "execute_result" 533 | } 534 | ], 535 | "source": [ 536 | "methods(FiniteDifference)" 537 | ] 538 | }, 539 | { 540 | "cell_type": "code", 541 | "execution_count": 33, 542 | "metadata": {}, 543 | "outputs": [ 544 | { 545 | "data": { 546 | "text/plain": [ 547 | "FiniteDifference" 548 | ] 549 | }, 550 | "execution_count": 33, 551 | "metadata": {}, 552 | "output_type": "execute_result" 553 | } 554 | ], 555 | "source": [ 556 | "FiniteDifference() = FiniteDifference(0.001)" 557 | ] 558 | }, 559 | { 560 | "cell_type": "code", 561 | "execution_count": 34, 562 | "metadata": {}, 563 | "outputs": [ 564 | { 565 | "data": { 566 | "text/html": [ 567 | "3 methods for generic function Type:" 568 | ], 569 | "text/plain": [ 570 | "# 3 methods for generic function \"(::Type)\":\n", 571 | "[1] FiniteDifference() in Main at In[33]:1\n", 572 | "[2] FiniteDifference(h::Float64) in Main at In[2]:2\n", 573 | "[3] FiniteDifference(h) in Main at In[2]:2" 574 | ] 575 | }, 576 | "execution_count": 34, 577 | "metadata": {}, 578 | "output_type": "execute_result" 579 | } 580 | ], 581 | "source": [ 582 | "methods(FiniteDifference)" 583 | ] 584 | }, 585 | { 586 | "cell_type": "code", 587 | "execution_count": 35, 588 | "metadata": {}, 589 | "outputs": [ 590 | { 591 | "data": { 592 | "text/plain": [ 593 | "FiniteDifference(0.001)" 594 | ] 595 | }, 596 | "execution_count": 35, 597 | "metadata": {}, 598 | "output_type": "execute_result" 599 | } 600 | ], 601 | "source": [ 602 | "FiniteDifference()" 603 | ] 604 | }, 605 | { 606 | "cell_type": "code", 607 | "execution_count": 36, 608 | "metadata": {}, 609 | "outputs": [ 610 | { 611 | "data": { 612 | "text/plain": [ 613 | "5.999999999999339" 614 | ] 615 | }, 616 | "execution_count": 36, 617 | "metadata": {}, 618 | "output_type": "execute_result" 619 | } 620 | ], 621 | "source": [ 622 | "derivative(g, a, FiniteDifference() )" 623 | ] 624 | }, 625 | { 626 | "cell_type": "code", 627 | "execution_count": 37, 628 | "metadata": {}, 629 | "outputs": [ 630 | { 631 | "data": { 632 | "text/html": [ 633 | "2 methods for generic function derivative:" 634 | ], 635 | "text/plain": [ 636 | "# 2 methods for generic function \"derivative\":\n", 637 | "[1] derivative(f, x, algorithm::AutoDiff) in Main at In[10]:2\n", 638 | "[2] derivative(f, x, algorithm::FiniteDifference) in Main at In[19]:2" 639 | ] 640 | }, 641 | "execution_count": 37, 642 | "metadata": {}, 643 | "output_type": "execute_result" 644 | } 645 | ], 646 | "source": [ 647 | "methods(derivative)" 648 | ] 649 | }, 650 | { 651 | "cell_type": "markdown", 652 | "metadata": {}, 653 | "source": [ 654 | "#### Exercise 2" 655 | ] 656 | }, 657 | { 658 | "cell_type": "markdown", 659 | "metadata": {}, 660 | "source": [ 661 | "1. Write a version of the Newton algorithm that takes an optional keyword argument `algorithm` specifying the differentiation algorithm to use, and which has a default value of `AutoDiff`." 662 | ] 663 | }, 664 | { 665 | "cell_type": "code", 666 | "execution_count": 42, 667 | "metadata": {}, 668 | "outputs": [ 669 | { 670 | "data": { 671 | "text/plain": [ 672 | "newton (generic function with 2 methods)" 673 | ] 674 | }, 675 | "execution_count": 42, 676 | "metadata": {}, 677 | "output_type": "execute_result" 678 | } 679 | ], 680 | "source": [ 681 | "function newton(f, x0, n=10; \n", 682 | " algorithm::DifferentiationAlgorithm=AutoDiff())\n", 683 | " \n", 684 | " df = x -> derivative(f, x, algorithm)\n", 685 | " \n", 686 | "# newton(f, df, x0)\n", 687 | " \n", 688 | "end" 689 | ] 690 | }, 691 | { 692 | "cell_type": "code", 693 | "execution_count": 43, 694 | "metadata": {}, 695 | "outputs": [ 696 | { 697 | "data": { 698 | "text/plain": [ 699 | "6.0" 700 | ] 701 | }, 702 | "execution_count": 43, 703 | "metadata": {}, 704 | "output_type": "execute_result" 705 | } 706 | ], 707 | "source": [ 708 | "newton(g, a)" 709 | ] 710 | }, 711 | { 712 | "cell_type": "code", 713 | "execution_count": null, 714 | "metadata": {}, 715 | "outputs": [], 716 | "source": [] 717 | }, 718 | { 719 | "cell_type": "markdown", 720 | "metadata": {}, 721 | "source": [ 722 | "## Functions as types" 723 | ] 724 | }, 725 | { 726 | "cell_type": "markdown", 727 | "metadata": {}, 728 | "source": [ 729 | "How could we differentiate the `sin` function? Of course, we can do it using e.g. finite differences, but in cases like this we actually know the exact, analytical derivative, in this case `cos`, i.e $\\sin'(x) = \\cos(x)$.\n", 730 | "Is there are a way to tell Julia to use this directly? I.e. if `f` is equal to the `sin` function, then we should make a special version of our `derivative` function.\n", 731 | "\n", 732 | "It turns out that there is, using dispatch, by checking what *type* the `sin` function has:" 733 | ] 734 | }, 735 | { 736 | "cell_type": "markdown", 737 | "metadata": {}, 738 | "source": [ 739 | "#### Exercise 3\n", 740 | "\n", 741 | "1. Use `typeof` to find the type of the `sin` function.\n", 742 | "\n", 743 | "\n", 744 | "2. Use this to make a special dispatch for `derivative(sin, x)`." 745 | ] 746 | }, 747 | { 748 | "cell_type": "code", 749 | "execution_count": 44, 750 | "metadata": {}, 751 | "outputs": [ 752 | { 753 | "data": { 754 | "text/plain": [ 755 | "typeof(sin)" 756 | ] 757 | }, 758 | "execution_count": 44, 759 | "metadata": {}, 760 | "output_type": "execute_result" 761 | } 762 | ], 763 | "source": [ 764 | "typeof(sin)" 765 | ] 766 | }, 767 | { 768 | "cell_type": "code", 769 | "execution_count": 45, 770 | "metadata": {}, 771 | "outputs": [ 772 | { 773 | "data": { 774 | "text/plain": [ 775 | "derivative (generic function with 3 methods)" 776 | ] 777 | }, 778 | "execution_count": 45, 779 | "metadata": {}, 780 | "output_type": "execute_result" 781 | } 782 | ], 783 | "source": [ 784 | "derivative(::typeof(sin), x) = cos(x)" 785 | ] 786 | }, 787 | { 788 | "cell_type": "code", 789 | "execution_count": null, 790 | "metadata": {}, 791 | "outputs": [], 792 | "source": [] 793 | }, 794 | { 795 | "cell_type": "markdown", 796 | "metadata": {}, 797 | "source": [ 798 | "The package [`ChainRules.jl`](https://github.com/JuliaDiff/ChainRules.jl) contains definitions like this and is used inside `ForwardDiff.jl` and other packages that need to know the derivatives of functions." 799 | ] 800 | }, 801 | { 802 | "cell_type": "markdown", 803 | "metadata": {}, 804 | "source": [ 805 | "## Representing a problem using a type" 806 | ] 807 | }, 808 | { 809 | "cell_type": "markdown", 810 | "metadata": {}, 811 | "source": [ 812 | "A root-finding problem requires several pieces of information: a function, a starting point, a root-finding algorithm to use, possibly a derivative, etc. We could just pass all of these as arguments to a function.\n", 813 | "An alternative is to wrap up the various pieces of information into a new composite type." 814 | ] 815 | }, 816 | { 817 | "cell_type": "markdown", 818 | "metadata": {}, 819 | "source": [ 820 | "#### Exercise 4\n", 821 | "\n", 822 | "1. Make a `RootAlgorithm` abstract type.\n", 823 | "\n", 824 | "\n", 825 | "2. Make a `Newton` subtype.\n", 826 | "\n", 827 | "\n", 828 | "3. Make `Newton` a *callable* type using the syntax\n", 829 | "\n", 830 | " ```\n", 831 | " function (algorithm::Newton)(x)\n", 832 | " ...\n", 833 | " end\n", 834 | " \n", 835 | " ```\n", 836 | " \n", 837 | " This means that you will be able to call an object of type `Newton` as if it were a standard function, by passing in an argument. (You can add further arguments as necessary.)\n", 838 | "\n", 839 | "\n", 840 | "4. Make a `RootProblem` type that contains the necessary information for a root problem. Do not specify types for the fields in this type yet. One of the fields should be called `algorithm`. \n", 841 | "\n", 842 | "\n", 843 | "5. Make a function `solve` that calls the field `algorithm` as a function." 844 | ] 845 | }, 846 | { 847 | "cell_type": "code", 848 | "execution_count": 46, 849 | "metadata": {}, 850 | "outputs": [], 851 | "source": [ 852 | "struct MyNewton\n", 853 | "end" 854 | ] 855 | }, 856 | { 857 | "cell_type": "code", 858 | "execution_count": 48, 859 | "metadata": {}, 860 | "outputs": [ 861 | { 862 | "data": { 863 | "text/plain": [ 864 | "MyNewton()" 865 | ] 866 | }, 867 | "execution_count": 48, 868 | "metadata": {}, 869 | "output_type": "execute_result" 870 | } 871 | ], 872 | "source": [ 873 | "mynewton = MyNewton()" 874 | ] 875 | }, 876 | { 877 | "cell_type": "code", 878 | "execution_count": 52, 879 | "metadata": {}, 880 | "outputs": [], 881 | "source": [ 882 | "function (f::MyNewton)(x)\n", 883 | " return 3x\n", 884 | "end" 885 | ] 886 | }, 887 | { 888 | "cell_type": "code", 889 | "execution_count": 53, 890 | "metadata": {}, 891 | "outputs": [ 892 | { 893 | "data": { 894 | "text/plain": [ 895 | "30" 896 | ] 897 | }, 898 | "execution_count": 53, 899 | "metadata": {}, 900 | "output_type": "execute_result" 901 | } 902 | ], 903 | "source": [ 904 | "mynewton(10)" 905 | ] 906 | }, 907 | { 908 | "cell_type": "markdown", 909 | "metadata": {}, 910 | "source": [ 911 | "Make FiniteDifference into a callable thing:" 912 | ] 913 | }, 914 | { 915 | "cell_type": "code", 916 | "execution_count": 54, 917 | "metadata": {}, 918 | "outputs": [ 919 | { 920 | "data": { 921 | "text/plain": [ 922 | "FiniteDifference(0.001)" 923 | ] 924 | }, 925 | "execution_count": 54, 926 | "metadata": {}, 927 | "output_type": "execute_result" 928 | } 929 | ], 930 | "source": [ 931 | "fd = FiniteDifference()" 932 | ] 933 | }, 934 | { 935 | "cell_type": "markdown", 936 | "metadata": {}, 937 | "source": [ 938 | "If I try to treat `fd` as a function, what happens?" 939 | ] 940 | }, 941 | { 942 | "cell_type": "code", 943 | "execution_count": 55, 944 | "metadata": {}, 945 | "outputs": [ 946 | { 947 | "ename": "MethodError", 948 | "evalue": "MethodError: objects of type FiniteDifference are not callable", 949 | "output_type": "error", 950 | "traceback": [ 951 | "MethodError: objects of type FiniteDifference are not callable", 952 | "", 953 | "Stacktrace:", 954 | " [1] top-level scope at In[55]:1" 955 | ] 956 | } 957 | ], 958 | "source": [ 959 | "fd(10)" 960 | ] 961 | }, 962 | { 963 | "cell_type": "markdown", 964 | "metadata": {}, 965 | "source": [ 966 | "Tell Julia how to treat it as a function:" 967 | ] 968 | }, 969 | { 970 | "cell_type": "code", 971 | "execution_count": 56, 972 | "metadata": {}, 973 | "outputs": [], 974 | "source": [ 975 | "function (fd::FiniteDifference)(f, x)\n", 976 | " h = fd.h\n", 977 | " finite_difference(f, x, h)\n", 978 | "end" 979 | ] 980 | }, 981 | { 982 | "cell_type": "code", 983 | "execution_count": 57, 984 | "metadata": {}, 985 | "outputs": [ 986 | { 987 | "data": { 988 | "text/plain": [ 989 | "5.999999999999339" 990 | ] 991 | }, 992 | "execution_count": 57, 993 | "metadata": {}, 994 | "output_type": "execute_result" 995 | } 996 | ], 997 | "source": [ 998 | "fd(g, a)" 999 | ] 1000 | }, 1001 | { 1002 | "cell_type": "code", 1003 | "execution_count": 58, 1004 | "metadata": {}, 1005 | "outputs": [], 1006 | "source": [ 1007 | "abstract type RootAlgorithm end" 1008 | ] 1009 | }, 1010 | { 1011 | "cell_type": "code", 1012 | "execution_count": 59, 1013 | "metadata": {}, 1014 | "outputs": [], 1015 | "source": [ 1016 | "struct Newton <: RootAlgorithm end" 1017 | ] 1018 | }, 1019 | { 1020 | "cell_type": "code", 1021 | "execution_count": 71, 1022 | "metadata": {}, 1023 | "outputs": [], 1024 | "source": [ 1025 | "(n::Newton)(args...) = ( @show args; newton(args...) )" 1026 | ] 1027 | }, 1028 | { 1029 | "cell_type": "code", 1030 | "execution_count": 67, 1031 | "metadata": {}, 1032 | "outputs": [ 1033 | { 1034 | "data": { 1035 | "text/html": [ 1036 | "1 method for generic function Type:" 1037 | ], 1038 | "text/plain": [ 1039 | "# 1 method for generic function \"(::Type)\":\n", 1040 | "[1] Newton() in Main at In[59]:1" 1041 | ] 1042 | }, 1043 | "execution_count": 67, 1044 | "metadata": {}, 1045 | "output_type": "execute_result" 1046 | } 1047 | ], 1048 | "source": [ 1049 | "n = Newton()\n", 1050 | "methods(Newton)" 1051 | ] 1052 | }, 1053 | { 1054 | "cell_type": "code", 1055 | "execution_count": 72, 1056 | "metadata": {}, 1057 | "outputs": [ 1058 | { 1059 | "name": "stdout", 1060 | "output_type": "stream", 1061 | "text": [ 1062 | "args = (g, 3.0)\n" 1063 | ] 1064 | }, 1065 | { 1066 | "data": { 1067 | "text/plain": [ 1068 | "6.0" 1069 | ] 1070 | }, 1071 | "execution_count": 72, 1072 | "metadata": {}, 1073 | "output_type": "execute_result" 1074 | } 1075 | ], 1076 | "source": [ 1077 | "newt = Newton()\n", 1078 | "\n", 1079 | "newt(g, a)" 1080 | ] 1081 | }, 1082 | { 1083 | "cell_type": "code", 1084 | "execution_count": 73, 1085 | "metadata": {}, 1086 | "outputs": [], 1087 | "source": [ 1088 | "struct RootProblem\n", 1089 | " f\n", 1090 | " x0\n", 1091 | " algorithm::RootAlgorithm\n", 1092 | "end" 1093 | ] 1094 | }, 1095 | { 1096 | "cell_type": "code", 1097 | "execution_count": 81, 1098 | "metadata": {}, 1099 | "outputs": [ 1100 | { 1101 | "data": { 1102 | "text/plain": [ 1103 | "solve (generic function with 1 method)" 1104 | ] 1105 | }, 1106 | "execution_count": 81, 1107 | "metadata": {}, 1108 | "output_type": "execute_result" 1109 | } 1110 | ], 1111 | "source": [ 1112 | "function solve(prob::RootProblem)\n", 1113 | " prob.algorithm(prob.f, prob.x0)\n", 1114 | "end" 1115 | ] 1116 | }, 1117 | { 1118 | "cell_type": "code", 1119 | "execution_count": 82, 1120 | "metadata": {}, 1121 | "outputs": [ 1122 | { 1123 | "data": { 1124 | "text/plain": [ 1125 | "RootProblem(g, 3.0, Newton())" 1126 | ] 1127 | }, 1128 | "execution_count": 82, 1129 | "metadata": {}, 1130 | "output_type": "execute_result" 1131 | } 1132 | ], 1133 | "source": [ 1134 | "prob = RootProblem(g, a, Newton())" 1135 | ] 1136 | }, 1137 | { 1138 | "cell_type": "code", 1139 | "execution_count": 83, 1140 | "metadata": {}, 1141 | "outputs": [ 1142 | { 1143 | "name": "stdout", 1144 | "output_type": "stream", 1145 | "text": [ 1146 | "args = (g, 3.0)\n" 1147 | ] 1148 | }, 1149 | { 1150 | "data": { 1151 | "text/plain": [ 1152 | "6.0" 1153 | ] 1154 | }, 1155 | "execution_count": 83, 1156 | "metadata": {}, 1157 | "output_type": "execute_result" 1158 | } 1159 | ], 1160 | "source": [ 1161 | "solve(prob)" 1162 | ] 1163 | }, 1164 | { 1165 | "cell_type": "markdown", 1166 | "metadata": {}, 1167 | "source": [ 1168 | "### Type-based dispatch" 1169 | ] 1170 | }, 1171 | { 1172 | "cell_type": "markdown", 1173 | "metadata": {}, 1174 | "source": [ 1175 | "So far we are not using the types to their full advantage: we wish to *dispatch* on the *type* of `algorithm`. We can do so by **parametrising the type**:" 1176 | ] 1177 | }, 1178 | { 1179 | "cell_type": "code", 1180 | "execution_count": 1, 1181 | "metadata": {}, 1182 | "outputs": [], 1183 | "source": [ 1184 | "struct RootProblem2{T<:RootAlgorithm}\n", 1185 | " ...\n", 1186 | " algorithm::T\n", 1187 | "end" 1188 | ] 1189 | }, 1190 | { 1191 | "cell_type": "markdown", 1192 | "metadata": {}, 1193 | "source": [ 1194 | "When we create an object of type `RootProblem2`, we will get a specialised version with the correct type. We can now use that in dispatch:" 1195 | ] 1196 | }, 1197 | { 1198 | "cell_type": "code", 1199 | "execution_count": null, 1200 | "metadata": {}, 1201 | "outputs": [], 1202 | "source": [ 1203 | "solve(prob::RootProblem2{Newton}) = ..." 1204 | ] 1205 | }, 1206 | { 1207 | "cell_type": "code", 1208 | "execution_count": null, 1209 | "metadata": {}, 1210 | "outputs": [], 1211 | "source": [ 1212 | "solve(prob::RootProblem2{Bisection}) = ..." 1213 | ] 1214 | }, 1215 | { 1216 | "cell_type": "code", 1217 | "execution_count": null, 1218 | "metadata": {}, 1219 | "outputs": [], 1220 | "source": [ 1221 | "struct RootProblem2{F, X<:Real, T<:RootAlgorithm}\n", 1222 | " f::F\n", 1223 | " x0::X\n", 1224 | " algorithm::T\n", 1225 | "end" 1226 | ] 1227 | }, 1228 | { 1229 | "cell_type": "code", 1230 | "execution_count": 84, 1231 | "metadata": {}, 1232 | "outputs": [], 1233 | "source": [] 1234 | }, 1235 | { 1236 | "cell_type": "markdown", 1237 | "metadata": {}, 1238 | "source": [ 1239 | "#### Exercise 5\n", 1240 | "\n", 1241 | "1. Implement this.\n", 1242 | "\n", 1243 | "\n", 1244 | "2. Put everything together to be able to solve a root problem using a particular derivative algorithm." 1245 | ] 1246 | }, 1247 | { 1248 | "cell_type": "code", 1249 | "execution_count": null, 1250 | "metadata": {}, 1251 | "outputs": [], 1252 | "source": [] 1253 | }, 1254 | { 1255 | "cell_type": "markdown", 1256 | "metadata": {}, 1257 | "source": [ 1258 | "#### Exercise 6\n", 1259 | "\n", 1260 | "1. Implement a `MultipleRootProblem` type that specifies an interval over which we would like to find all roots.\n", 1261 | "\n", 1262 | "\n", 1263 | "2. Write a simple implementation of the algorithm using multiple starting points in the interval and making a list of unique roots found by that procedure.\n", 1264 | "\n", 1265 | "\n", 1266 | "3. Load the `Polynomials.jl` package and write a dispatch that specialises on polynomials and calls the root finder in that package." 1267 | ] 1268 | }, 1269 | { 1270 | "cell_type": "code", 1271 | "execution_count": 85, 1272 | "metadata": {}, 1273 | "outputs": [], 1274 | "source": [ 1275 | "using Polynomials" 1276 | ] 1277 | }, 1278 | { 1279 | "cell_type": "code", 1280 | "execution_count": 86, 1281 | "metadata": {}, 1282 | "outputs": [ 1283 | { 1284 | "data": { 1285 | "text/html": [ 1286 | "1 + 2∙x + 3∙x^2" 1287 | ], 1288 | "text/latex": [ 1289 | "$1 + 2\\cdot x + 3\\cdot x^{2}$" 1290 | ], 1291 | "text/plain": [ 1292 | "Poly(1 + 2*x + 3*x^2)" 1293 | ] 1294 | }, 1295 | "execution_count": 86, 1296 | "metadata": {}, 1297 | "output_type": "execute_result" 1298 | } 1299 | ], 1300 | "source": [ 1301 | "p = Poly([1, 2, 3])" 1302 | ] 1303 | }, 1304 | { 1305 | "cell_type": "code", 1306 | "execution_count": 87, 1307 | "metadata": {}, 1308 | "outputs": [ 1309 | { 1310 | "data": { 1311 | "text/plain": [ 1312 | "321" 1313 | ] 1314 | }, 1315 | "execution_count": 87, 1316 | "metadata": {}, 1317 | "output_type": "execute_result" 1318 | } 1319 | ], 1320 | "source": [ 1321 | "p(10)" 1322 | ] 1323 | }, 1324 | { 1325 | "cell_type": "code", 1326 | "execution_count": 89, 1327 | "metadata": {}, 1328 | "outputs": [ 1329 | { 1330 | "data": { 1331 | "text/plain": [ 1332 | "false" 1333 | ] 1334 | }, 1335 | "execution_count": 89, 1336 | "metadata": {}, 1337 | "output_type": "execute_result" 1338 | } 1339 | ], 1340 | "source": [ 1341 | "p isa Function" 1342 | ] 1343 | }, 1344 | { 1345 | "cell_type": "markdown", 1346 | "metadata": {}, 1347 | "source": [ 1348 | "## Other uses of types" 1349 | ] 1350 | }, 1351 | { 1352 | "cell_type": "markdown", 1353 | "metadata": {}, 1354 | "source": [ 1355 | "Other examples of different usages of types include:\n", 1356 | "\n", 1357 | "- [`ModelingToolkit.jl`](https://github.com/JuliaDiffEq/ModelingToolkit.jl)\n", 1358 | "\n", 1359 | " Types are introduced to represent variables and operations. In this way it is relatively simple to build up a way to output symbolic expressions from standard Julia functions.\n", 1360 | " \n", 1361 | " \n", 1362 | "- https://github.com/MikeInnes/diff-zoo defines types to represent \"tapes\" recording sequences of operations. This is a precursor to tools such as [`Zygote.jl`](https://github.com/FluxML/Zygote.jl), which performs advanced automatic differentiation on code at a lower level." 1363 | ] 1364 | }, 1365 | { 1366 | "cell_type": "markdown", 1367 | "metadata": {}, 1368 | "source": [ 1369 | "### Traits" 1370 | ] 1371 | }, 1372 | { 1373 | "cell_type": "markdown", 1374 | "metadata": {}, 1375 | "source": [ 1376 | "An important use of types that we have not addressed here is to define **traits**. These are labels that can be assigned to different types that may then be dispatched on, even if those types are not in a Julia type hierarchy.\n", 1377 | "\n", 1378 | "See e.g. the implementation in [`SimpleTraits.jl`](https://github.com/mauro3/SimpleTraits.jl)." 1379 | ] 1380 | } 1381 | ], 1382 | "metadata": { 1383 | "kernelspec": { 1384 | "display_name": "Julia 1.1.0", 1385 | "language": "julia", 1386 | "name": "julia-1.1" 1387 | }, 1388 | "language_info": { 1389 | "file_extension": ".jl", 1390 | "mimetype": "application/julia", 1391 | "name": "julia", 1392 | "version": "1.1.0" 1393 | } 1394 | }, 1395 | "nbformat": 4, 1396 | "nbformat_minor": 2 1397 | } 1398 | -------------------------------------------------------------------------------- /live2/3. Type-based dispatch.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Structuring programs using types and dispatch" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "Previously we specified an algorithm directly using a function. In many Julia packages it is common to use *dispatch* to do this. \n", 15 | "\n", 16 | "In Julia, **dispatch** refers to choosing which **method** (version) of a function to use, according to the type of the arguments. (**Multiple dispatch** is when the types of several different arguments are involved.)" 17 | ] 18 | }, 19 | { 20 | "cell_type": "markdown", 21 | "metadata": {}, 22 | "source": [ 23 | "Let's define some types to represent different differentiation methods." 24 | ] 25 | }, 26 | { 27 | "cell_type": "code", 28 | "execution_count": 5, 29 | "metadata": {}, 30 | "outputs": [], 31 | "source": [ 32 | "struct Dual \n", 33 | " v::Float64\n", 34 | " d::Float64\n", 35 | "end" 36 | ] 37 | }, 38 | { 39 | "cell_type": "code", 40 | "execution_count": 6, 41 | "metadata": {}, 42 | "outputs": [ 43 | { 44 | "data": { 45 | "text/html": [ 46 | "2 methods for generic function Type:" 47 | ], 48 | "text/plain": [ 49 | "# 2 methods for generic function \"(::Type)\":\n", 50 | "[1] Dual(v::Float64, d::Float64) in Main at In[5]:2\n", 51 | "[2] Dual(v, d) in Main at In[5]:2" 52 | ] 53 | }, 54 | "execution_count": 6, 55 | "metadata": {}, 56 | "output_type": "execute_result" 57 | } 58 | ], 59 | "source": [ 60 | "methods(Dual)" 61 | ] 62 | }, 63 | { 64 | "cell_type": "markdown", 65 | "metadata": {}, 66 | "source": [ 67 | "#### Exercise 1\n", 68 | "\n", 69 | "1. Define an abstract type `DifferentiationAlgorithm`.\n", 70 | "\n", 71 | "\n", 72 | "2. Define subtypes `FiniteDifference`, `MyAutoDiff` (for our implementation) and `AutoDiff` (for the `ForwardDiff` implementation).\n", 73 | "\n", 74 | "\n", 75 | "3. Implement the function `derivative(f, x, algorithm)` using **dispatch**: for each of the three types, define a version of this function in which `algorithm` is specified to be of that type by using the type annotation operator `::`.\n", 76 | "\n", 77 | "\n", 78 | "4. Verify that these work by writing tests for them." 79 | ] 80 | }, 81 | { 82 | "cell_type": "code", 83 | "execution_count": 1, 84 | "metadata": {}, 85 | "outputs": [], 86 | "source": [ 87 | "abstract type DifferentiationAlgorithm end" 88 | ] 89 | }, 90 | { 91 | "cell_type": "code", 92 | "execution_count": 2, 93 | "metadata": {}, 94 | "outputs": [ 95 | { 96 | "data": { 97 | "text/plain": [ 98 | "DifferentiationAlgorithm" 99 | ] 100 | }, 101 | "execution_count": 2, 102 | "metadata": {}, 103 | "output_type": "execute_result" 104 | } 105 | ], 106 | "source": [ 107 | "DifferentiationAlgorithm" 108 | ] 109 | }, 110 | { 111 | "cell_type": "code", 112 | "execution_count": 23, 113 | "metadata": {}, 114 | "outputs": [], 115 | "source": [ 116 | "struct FiniteDifference <: DifferentiationAlgorithm\n", 117 | " h::Float64\n", 118 | "end" 119 | ] 120 | }, 121 | { 122 | "cell_type": "code", 123 | "execution_count": 4, 124 | "metadata": {}, 125 | "outputs": [], 126 | "source": [ 127 | "struct AutoDiff <: DifferentiationAlgorithm\n", 128 | "end" 129 | ] 130 | }, 131 | { 132 | "cell_type": "markdown", 133 | "metadata": {}, 134 | "source": [ 135 | "You could do:" 136 | ] 137 | }, 138 | { 139 | "cell_type": "code", 140 | "execution_count": 4, 141 | "metadata": {}, 142 | "outputs": [ 143 | { 144 | "ename": "LoadError", 145 | "evalue": "syntax: invalid identifier name \"...\"", 146 | "output_type": "error", 147 | "traceback": [ 148 | "syntax: invalid identifier name \"...\"", 149 | "" 150 | ] 151 | } 152 | ], 153 | "source": [ 154 | "function derivative(f, x, algorithm)\n", 155 | " if algorithm == :newton\n", 156 | " ...\n", 157 | " elseif algorithm == :bisection\n", 158 | " ...\n", 159 | " elseif\n", 160 | " ...\n", 161 | " end\n", 162 | "end" 163 | ] 164 | }, 165 | { 166 | "cell_type": "code", 167 | "execution_count": 20, 168 | "metadata": {}, 169 | "outputs": [ 170 | { 171 | "data": { 172 | "text/plain": [ 173 | "finite_difference (generic function with 1 method)" 174 | ] 175 | }, 176 | "execution_count": 20, 177 | "metadata": {}, 178 | "output_type": "execute_result" 179 | } 180 | ], 181 | "source": [ 182 | "finite_difference(f, a, h) = (f(a + h) - f(a - h)) / (2h)" 183 | ] 184 | }, 185 | { 186 | "cell_type": "code", 187 | "execution_count": 21, 188 | "metadata": {}, 189 | "outputs": [ 190 | { 191 | "data": { 192 | "text/plain": [ 193 | "forwarddiff (generic function with 1 method)" 194 | ] 195 | }, 196 | "execution_count": 21, 197 | "metadata": {}, 198 | "output_type": "execute_result" 199 | } 200 | ], 201 | "source": [ 202 | "using ForwardDiff\n", 203 | "forwarddiff(f, x) = ForwardDiff.derivative(f, x)" 204 | ] 205 | }, 206 | { 207 | "cell_type": "code", 208 | "execution_count": 22, 209 | "metadata": {}, 210 | "outputs": [ 211 | { 212 | "data": { 213 | "text/plain": [ 214 | "derivative (generic function with 1 method)" 215 | ] 216 | }, 217 | "execution_count": 22, 218 | "metadata": {}, 219 | "output_type": "execute_result" 220 | } 221 | ], 222 | "source": [ 223 | "function derivative(f, x, algorithm::AutoDiff)\n", 224 | " return forwarddiff(f, x)\n", 225 | "end" 226 | ] 227 | }, 228 | { 229 | "cell_type": "code", 230 | "execution_count": null, 231 | "metadata": {}, 232 | "outputs": [], 233 | "source": [] 234 | }, 235 | { 236 | "cell_type": "code", 237 | "execution_count": 13, 238 | "metadata": {}, 239 | "outputs": [ 240 | { 241 | "data": { 242 | "text/plain": [ 243 | "3.0" 244 | ] 245 | }, 246 | "execution_count": 13, 247 | "metadata": {}, 248 | "output_type": "execute_result" 249 | } 250 | ], 251 | "source": [ 252 | "g(x) = x^2 - 2\n", 253 | "a = 3.0" 254 | ] 255 | }, 256 | { 257 | "cell_type": "code", 258 | "execution_count": 14, 259 | "metadata": {}, 260 | "outputs": [ 261 | { 262 | "ename": "MethodError", 263 | "evalue": "MethodError: no method matching derivative(::typeof(g), ::Float64, ::Type{AutoDiff})\nClosest candidates are:\n derivative(::Any, ::Any, !Matched::AutoDiff) at In[10]:2", 264 | "output_type": "error", 265 | "traceback": [ 266 | "MethodError: no method matching derivative(::typeof(g), ::Float64, ::Type{AutoDiff})\nClosest candidates are:\n derivative(::Any, ::Any, !Matched::AutoDiff) at In[10]:2", 267 | "", 268 | "Stacktrace:", 269 | " [1] top-level scope at In[14]:1" 270 | ] 271 | } 272 | ], 273 | "source": [ 274 | "derivative(g, a, AutoDiff)" 275 | ] 276 | }, 277 | { 278 | "cell_type": "code", 279 | "execution_count": 14, 280 | "metadata": {}, 281 | "outputs": [ 282 | { 283 | "data": { 284 | "text/plain": [ 285 | "DataType" 286 | ] 287 | }, 288 | "execution_count": 14, 289 | "metadata": {}, 290 | "output_type": "execute_result" 291 | } 292 | ], 293 | "source": [ 294 | "typeof(AutoDiff)" 295 | ] 296 | }, 297 | { 298 | "cell_type": "code", 299 | "execution_count": 15, 300 | "metadata": {}, 301 | "outputs": [ 302 | { 303 | "data": { 304 | "text/plain": [ 305 | "AutoDiff()" 306 | ] 307 | }, 308 | "execution_count": 15, 309 | "metadata": {}, 310 | "output_type": "execute_result" 311 | } 312 | ], 313 | "source": [ 314 | "autodiff = AutoDiff()" 315 | ] 316 | }, 317 | { 318 | "cell_type": "code", 319 | "execution_count": 16, 320 | "metadata": {}, 321 | "outputs": [ 322 | { 323 | "data": { 324 | "text/plain": [ 325 | "AutoDiff" 326 | ] 327 | }, 328 | "execution_count": 16, 329 | "metadata": {}, 330 | "output_type": "execute_result" 331 | } 332 | ], 333 | "source": [ 334 | "typeof(autodiff)" 335 | ] 336 | }, 337 | { 338 | "cell_type": "code", 339 | "execution_count": 16, 340 | "metadata": {}, 341 | "outputs": [ 342 | { 343 | "data": { 344 | "text/plain": [ 345 | "6.0" 346 | ] 347 | }, 348 | "execution_count": 16, 349 | "metadata": {}, 350 | "output_type": "execute_result" 351 | } 352 | ], 353 | "source": [ 354 | "derivative(g, a, autodiff)" 355 | ] 356 | }, 357 | { 358 | "cell_type": "code", 359 | "execution_count": 17, 360 | "metadata": {}, 361 | "outputs": [ 362 | { 363 | "data": { 364 | "text/plain": [ 365 | "6.0" 366 | ] 367 | }, 368 | "execution_count": 17, 369 | "metadata": {}, 370 | "output_type": "execute_result" 371 | } 372 | ], 373 | "source": [ 374 | "derivative(g, a, AutoDiff())" 375 | ] 376 | }, 377 | { 378 | "cell_type": "code", 379 | "execution_count": 24, 380 | "metadata": {}, 381 | "outputs": [ 382 | { 383 | "data": { 384 | "text/plain": [ 385 | "derivative (generic function with 2 methods)" 386 | ] 387 | }, 388 | "execution_count": 24, 389 | "metadata": {}, 390 | "output_type": "execute_result" 391 | } 392 | ], 393 | "source": [ 394 | "function derivative(f, x, algorithm::FiniteDifference)\n", 395 | " h = algorithm.h\n", 396 | " return finite_difference(f, x, h)\n", 397 | "end" 398 | ] 399 | }, 400 | { 401 | "cell_type": "code", 402 | "execution_count": 25, 403 | "metadata": {}, 404 | "outputs": [ 405 | { 406 | "ename": "MethodError", 407 | "evalue": "MethodError: no method matching FiniteDifference()\nClosest candidates are:\n FiniteDifference(!Matched::Float64) at In[23]:2\n FiniteDifference(!Matched::Any) at In[23]:2", 408 | "output_type": "error", 409 | "traceback": [ 410 | "MethodError: no method matching FiniteDifference()\nClosest candidates are:\n FiniteDifference(!Matched::Float64) at In[23]:2\n FiniteDifference(!Matched::Any) at In[23]:2", 411 | "", 412 | "Stacktrace:", 413 | " [1] top-level scope at In[25]:1" 414 | ] 415 | } 416 | ], 417 | "source": [ 418 | "derivative(g, a, FiniteDifference())" 419 | ] 420 | }, 421 | { 422 | "cell_type": "code", 423 | "execution_count": 26, 424 | "metadata": {}, 425 | "outputs": [ 426 | { 427 | "data": { 428 | "text/plain": [ 429 | "5.999999999999872" 430 | ] 431 | }, 432 | "execution_count": 26, 433 | "metadata": {}, 434 | "output_type": "execute_result" 435 | } 436 | ], 437 | "source": [ 438 | "derivative(g, a, FiniteDifference(0.01))" 439 | ] 440 | }, 441 | { 442 | "cell_type": "code", 443 | "execution_count": 25, 444 | "metadata": {}, 445 | "outputs": [ 446 | { 447 | "data": { 448 | "text/plain": [ 449 | "FiniteDifference(0.01)" 450 | ] 451 | }, 452 | "execution_count": 25, 453 | "metadata": {}, 454 | "output_type": "execute_result" 455 | } 456 | ], 457 | "source": [ 458 | "algorithm = FiniteDifference(0.01)" 459 | ] 460 | }, 461 | { 462 | "cell_type": "code", 463 | "execution_count": 26, 464 | "metadata": {}, 465 | "outputs": [ 466 | { 467 | "data": { 468 | "text/plain": [ 469 | "FiniteDifference" 470 | ] 471 | }, 472 | "execution_count": 26, 473 | "metadata": {}, 474 | "output_type": "execute_result" 475 | } 476 | ], 477 | "source": [ 478 | "typeof(algorithm)" 479 | ] 480 | }, 481 | { 482 | "cell_type": "code", 483 | "execution_count": 27, 484 | "metadata": {}, 485 | "outputs": [ 486 | { 487 | "data": { 488 | "text/plain": [ 489 | "0.01" 490 | ] 491 | }, 492 | "execution_count": 27, 493 | "metadata": {}, 494 | "output_type": "execute_result" 495 | } 496 | ], 497 | "source": [ 498 | "algorithm.h" 499 | ] 500 | }, 501 | { 502 | "cell_type": "code", 503 | "execution_count": 30, 504 | "metadata": {}, 505 | "outputs": [ 506 | { 507 | "data": { 508 | "text/plain": [ 509 | "(:h,)" 510 | ] 511 | }, 512 | "execution_count": 30, 513 | "metadata": {}, 514 | "output_type": "execute_result" 515 | } 516 | ], 517 | "source": [ 518 | "fieldnames(typeof(algorithm))" 519 | ] 520 | }, 521 | { 522 | "cell_type": "code", 523 | "execution_count": 31, 524 | "metadata": {}, 525 | "outputs": [ 526 | { 527 | "ename": "MethodError", 528 | "evalue": "MethodError: no method matching FiniteDifference()\nClosest candidates are:\n FiniteDifference(!Matched::Float64) at In[2]:2\n FiniteDifference(!Matched::Any) at In[2]:2", 529 | "output_type": "error", 530 | "traceback": [ 531 | "MethodError: no method matching FiniteDifference()\nClosest candidates are:\n FiniteDifference(!Matched::Float64) at In[2]:2\n FiniteDifference(!Matched::Any) at In[2]:2", 532 | "", 533 | "Stacktrace:", 534 | " [1] top-level scope at In[31]:1" 535 | ] 536 | } 537 | ], 538 | "source": [ 539 | "FiniteDifference()" 540 | ] 541 | }, 542 | { 543 | "cell_type": "code", 544 | "execution_count": 32, 545 | "metadata": {}, 546 | "outputs": [ 547 | { 548 | "data": { 549 | "text/html": [ 550 | "2 methods for generic function Type:" 551 | ], 552 | "text/plain": [ 553 | "# 2 methods for generic function \"(::Type)\":\n", 554 | "[1] FiniteDifference(h::Float64) in Main at In[2]:2\n", 555 | "[2] FiniteDifference(h) in Main at In[2]:2" 556 | ] 557 | }, 558 | "execution_count": 32, 559 | "metadata": {}, 560 | "output_type": "execute_result" 561 | } 562 | ], 563 | "source": [ 564 | "methods(FiniteDifference)" 565 | ] 566 | }, 567 | { 568 | "cell_type": "code", 569 | "execution_count": 27, 570 | "metadata": {}, 571 | "outputs": [ 572 | { 573 | "data": { 574 | "text/plain": [ 575 | "FiniteDifference" 576 | ] 577 | }, 578 | "execution_count": 27, 579 | "metadata": {}, 580 | "output_type": "execute_result" 581 | } 582 | ], 583 | "source": [ 584 | "FiniteDifference() = FiniteDifference(0.001)" 585 | ] 586 | }, 587 | { 588 | "cell_type": "code", 589 | "execution_count": 28, 590 | "metadata": {}, 591 | "outputs": [ 592 | { 593 | "data": { 594 | "text/html": [ 595 | "3 methods for generic function Type:" 596 | ], 597 | "text/plain": [ 598 | "# 3 methods for generic function \"(::Type)\":\n", 599 | "[1] FiniteDifference() in Main at In[27]:1\n", 600 | "[2] FiniteDifference(h::Float64) in Main at In[23]:2\n", 601 | "[3] FiniteDifference(h) in Main at In[23]:2" 602 | ] 603 | }, 604 | "execution_count": 28, 605 | "metadata": {}, 606 | "output_type": "execute_result" 607 | } 608 | ], 609 | "source": [ 610 | "methods(FiniteDifference)" 611 | ] 612 | }, 613 | { 614 | "cell_type": "code", 615 | "execution_count": 35, 616 | "metadata": {}, 617 | "outputs": [ 618 | { 619 | "data": { 620 | "text/plain": [ 621 | "FiniteDifference(0.001)" 622 | ] 623 | }, 624 | "execution_count": 35, 625 | "metadata": {}, 626 | "output_type": "execute_result" 627 | } 628 | ], 629 | "source": [ 630 | "FiniteDifference()" 631 | ] 632 | }, 633 | { 634 | "cell_type": "code", 635 | "execution_count": 29, 636 | "metadata": {}, 637 | "outputs": [ 638 | { 639 | "data": { 640 | "text/plain": [ 641 | "5.999999999999339" 642 | ] 643 | }, 644 | "execution_count": 29, 645 | "metadata": {}, 646 | "output_type": "execute_result" 647 | } 648 | ], 649 | "source": [ 650 | "derivative(g, a, FiniteDifference() )" 651 | ] 652 | }, 653 | { 654 | "cell_type": "code", 655 | "execution_count": 30, 656 | "metadata": {}, 657 | "outputs": [ 658 | { 659 | "data": { 660 | "text/html": [ 661 | "2 methods for generic function derivative:" 662 | ], 663 | "text/plain": [ 664 | "# 2 methods for generic function \"derivative\":\n", 665 | "[1] derivative(f, x, algorithm::AutoDiff) in Main at In[22]:2\n", 666 | "[2] derivative(f, x, algorithm::FiniteDifference) in Main at In[24]:2" 667 | ] 668 | }, 669 | "execution_count": 30, 670 | "metadata": {}, 671 | "output_type": "execute_result" 672 | } 673 | ], 674 | "source": [ 675 | "methods(derivative)" 676 | ] 677 | }, 678 | { 679 | "cell_type": "code", 680 | "execution_count": 31, 681 | "metadata": {}, 682 | "outputs": [ 683 | { 684 | "data": { 685 | "text/plain": [ 686 | "derivative (generic function with 3 methods)" 687 | ] 688 | }, 689 | "execution_count": 31, 690 | "metadata": {}, 691 | "output_type": "execute_result" 692 | } 693 | ], 694 | "source": [ 695 | "derivative(f, x) = derivative(f, x, AutoDiff())" 696 | ] 697 | }, 698 | { 699 | "cell_type": "code", 700 | "execution_count": 35, 701 | "metadata": {}, 702 | "outputs": [ 703 | { 704 | "data": { 705 | "text/plain": [ 706 | "6.0" 707 | ] 708 | }, 709 | "execution_count": 35, 710 | "metadata": {}, 711 | "output_type": "execute_result" 712 | } 713 | ], 714 | "source": [ 715 | "f(x) = x^2 - 2\n", 716 | "derivative(f, 3.0)" 717 | ] 718 | }, 719 | { 720 | "cell_type": "code", 721 | "execution_count": 36, 722 | "metadata": {}, 723 | "outputs": [ 724 | { 725 | "data": { 726 | "text/plain": [ 727 | "6.000000000000005" 728 | ] 729 | }, 730 | "execution_count": 36, 731 | "metadata": {}, 732 | "output_type": "execute_result" 733 | } 734 | ], 735 | "source": [ 736 | "derivative(f, 3.0, FiniteDifference(0.1))" 737 | ] 738 | }, 739 | { 740 | "cell_type": "code", 741 | "execution_count": 37, 742 | "metadata": {}, 743 | "outputs": [ 744 | { 745 | "data": { 746 | "text/plain": [ 747 | "derivative (generic function with 4 methods)" 748 | ] 749 | }, 750 | "execution_count": 37, 751 | "metadata": {}, 752 | "output_type": "execute_result" 753 | } 754 | ], 755 | "source": [ 756 | "derivative(f, x, something) = error(\"Can't use $something in derivative function\")" 757 | ] 758 | }, 759 | { 760 | "cell_type": "code", 761 | "execution_count": 39, 762 | "metadata": {}, 763 | "outputs": [ 764 | { 765 | "ename": "ErrorException", 766 | "evalue": "Can't use getfield(Main, Symbol(\"##7#8\"))() in derivative function", 767 | "output_type": "error", 768 | "traceback": [ 769 | "Can't use getfield(Main, Symbol(\"##7#8\"))() in derivative function", 770 | "", 771 | "Stacktrace:", 772 | " [1] error(::String) at ./error.jl:33", 773 | " [2] derivative(::Function, ::Float64, ::Function) at ./In[37]:1", 774 | " [3] top-level scope at In[39]:1" 775 | ] 776 | } 777 | ], 778 | "source": [ 779 | "derivative(f, a, x->x^2)" 780 | ] 781 | }, 782 | { 783 | "cell_type": "markdown", 784 | "metadata": {}, 785 | "source": [ 786 | "#### Exercise 2" 787 | ] 788 | }, 789 | { 790 | "cell_type": "markdown", 791 | "metadata": {}, 792 | "source": [ 793 | "1. Write a version of the Newton algorithm that takes an optional keyword argument `algorithm` specifying the differentiation algorithm to use, and which has a default value of `AutoDiff`." 794 | ] 795 | }, 796 | { 797 | "cell_type": "code", 798 | "execution_count": 42, 799 | "metadata": {}, 800 | "outputs": [ 801 | { 802 | "data": { 803 | "text/plain": [ 804 | "newton (generic function with 2 methods)" 805 | ] 806 | }, 807 | "execution_count": 42, 808 | "metadata": {}, 809 | "output_type": "execute_result" 810 | } 811 | ], 812 | "source": [ 813 | "function newton(f, x0, n=10; \n", 814 | " algorithm::DifferentiationAlgorithm=AutoDiff())\n", 815 | " \n", 816 | " df = x -> derivative(f, x, algorithm)\n", 817 | " \n", 818 | "# newton(f, df, x0)\n", 819 | " \n", 820 | "end" 821 | ] 822 | }, 823 | { 824 | "cell_type": "code", 825 | "execution_count": 43, 826 | "metadata": {}, 827 | "outputs": [ 828 | { 829 | "data": { 830 | "text/plain": [ 831 | "6.0" 832 | ] 833 | }, 834 | "execution_count": 43, 835 | "metadata": {}, 836 | "output_type": "execute_result" 837 | } 838 | ], 839 | "source": [ 840 | "newton(g, a)" 841 | ] 842 | }, 843 | { 844 | "cell_type": "code", 845 | "execution_count": null, 846 | "metadata": {}, 847 | "outputs": [], 848 | "source": [] 849 | }, 850 | { 851 | "cell_type": "markdown", 852 | "metadata": {}, 853 | "source": [ 854 | "## Functions as types" 855 | ] 856 | }, 857 | { 858 | "cell_type": "markdown", 859 | "metadata": {}, 860 | "source": [ 861 | "How could we differentiate the `sin` function? Of course, we can do it using e.g. finite differences, but in cases like this we actually know the exact, analytical derivative, in this case `cos`, i.e $\\sin'(x) = \\cos(x)$.\n", 862 | "Is there are a way to tell Julia to use this directly? I.e. if `f` is equal to the `sin` function, then we should make a special version of our `derivative` function.\n", 863 | "\n", 864 | "It turns out that there is, using dispatch, by checking what *type* the `sin` function has:" 865 | ] 866 | }, 867 | { 868 | "cell_type": "markdown", 869 | "metadata": {}, 870 | "source": [ 871 | "#### Exercise 3\n", 872 | "\n", 873 | "1. Use `typeof` to find the type of the `sin` function.\n", 874 | "\n", 875 | "\n", 876 | "2. Use this to make a special dispatch for `derivative(sin, x)`." 877 | ] 878 | }, 879 | { 880 | "cell_type": "code", 881 | "execution_count": 44, 882 | "metadata": {}, 883 | "outputs": [ 884 | { 885 | "data": { 886 | "text/plain": [ 887 | "typeof(sin)" 888 | ] 889 | }, 890 | "execution_count": 44, 891 | "metadata": {}, 892 | "output_type": "execute_result" 893 | } 894 | ], 895 | "source": [ 896 | "typeof(sin)" 897 | ] 898 | }, 899 | { 900 | "cell_type": "code", 901 | "execution_count": 45, 902 | "metadata": {}, 903 | "outputs": [ 904 | { 905 | "data": { 906 | "text/plain": [ 907 | "derivative (generic function with 3 methods)" 908 | ] 909 | }, 910 | "execution_count": 45, 911 | "metadata": {}, 912 | "output_type": "execute_result" 913 | } 914 | ], 915 | "source": [ 916 | "derivative(::typeof(sin), x) = cos(x)" 917 | ] 918 | }, 919 | { 920 | "cell_type": "code", 921 | "execution_count": null, 922 | "metadata": {}, 923 | "outputs": [], 924 | "source": [] 925 | }, 926 | { 927 | "cell_type": "markdown", 928 | "metadata": {}, 929 | "source": [ 930 | "The package [`ChainRules.jl`](https://github.com/JuliaDiff/ChainRules.jl) contains definitions like this and is used inside `ForwardDiff.jl` and other packages that need to know the derivatives of functions." 931 | ] 932 | }, 933 | { 934 | "cell_type": "markdown", 935 | "metadata": {}, 936 | "source": [ 937 | "## Representing a problem using a type" 938 | ] 939 | }, 940 | { 941 | "cell_type": "markdown", 942 | "metadata": {}, 943 | "source": [ 944 | "A root-finding problem requires several pieces of information: a function, a starting point, a root-finding algorithm to use, possibly a derivative, etc. We could just pass all of these as arguments to a function.\n", 945 | "An alternative is to wrap up the various pieces of information into a new composite type." 946 | ] 947 | }, 948 | { 949 | "cell_type": "markdown", 950 | "metadata": {}, 951 | "source": [ 952 | "#### Exercise 4\n", 953 | "\n", 954 | "1. Make a `RootAlgorithm` abstract type.\n", 955 | "\n", 956 | "\n", 957 | "2. Make a `Newton` subtype.\n", 958 | "\n", 959 | "\n", 960 | "3. Make `Newton` a *callable* type using the syntax\n", 961 | "\n", 962 | " ```\n", 963 | " function (algorithm::Newton)(x)\n", 964 | " ...\n", 965 | " end\n", 966 | " \n", 967 | " ```\n", 968 | " \n", 969 | " This means that you will be able to call an object of type `Newton` as if it were a standard function, by passing in an argument. (You can add further arguments as necessary.)\n", 970 | "\n", 971 | "\n", 972 | "4. Make a `RootProblem` type that contains the necessary information for a root problem. Do not specify types for the fields in this type yet. One of the fields should be called `algorithm`. \n", 973 | "\n", 974 | "\n", 975 | "5. Make a function `solve` that calls the field `algorithm` as a function." 976 | ] 977 | }, 978 | { 979 | "cell_type": "code", 980 | "execution_count": 46, 981 | "metadata": {}, 982 | "outputs": [], 983 | "source": [ 984 | "struct MyNewton\n", 985 | "end" 986 | ] 987 | }, 988 | { 989 | "cell_type": "code", 990 | "execution_count": 48, 991 | "metadata": {}, 992 | "outputs": [ 993 | { 994 | "data": { 995 | "text/plain": [ 996 | "MyNewton()" 997 | ] 998 | }, 999 | "execution_count": 48, 1000 | "metadata": {}, 1001 | "output_type": "execute_result" 1002 | } 1003 | ], 1004 | "source": [ 1005 | "mynewton = MyNewton()" 1006 | ] 1007 | }, 1008 | { 1009 | "cell_type": "code", 1010 | "execution_count": 52, 1011 | "metadata": {}, 1012 | "outputs": [], 1013 | "source": [ 1014 | "function (f::MyNewton)(x)\n", 1015 | " return 3x\n", 1016 | "end" 1017 | ] 1018 | }, 1019 | { 1020 | "cell_type": "code", 1021 | "execution_count": 53, 1022 | "metadata": {}, 1023 | "outputs": [ 1024 | { 1025 | "data": { 1026 | "text/plain": [ 1027 | "30" 1028 | ] 1029 | }, 1030 | "execution_count": 53, 1031 | "metadata": {}, 1032 | "output_type": "execute_result" 1033 | } 1034 | ], 1035 | "source": [ 1036 | "mynewton(10)" 1037 | ] 1038 | }, 1039 | { 1040 | "cell_type": "markdown", 1041 | "metadata": {}, 1042 | "source": [ 1043 | "Make FiniteDifference into a callable thing:" 1044 | ] 1045 | }, 1046 | { 1047 | "cell_type": "code", 1048 | "execution_count": 54, 1049 | "metadata": {}, 1050 | "outputs": [ 1051 | { 1052 | "data": { 1053 | "text/plain": [ 1054 | "FiniteDifference(0.001)" 1055 | ] 1056 | }, 1057 | "execution_count": 54, 1058 | "metadata": {}, 1059 | "output_type": "execute_result" 1060 | } 1061 | ], 1062 | "source": [ 1063 | "fd = FiniteDifference()" 1064 | ] 1065 | }, 1066 | { 1067 | "cell_type": "markdown", 1068 | "metadata": {}, 1069 | "source": [ 1070 | "If I try to treat `fd` as a function, what happens?" 1071 | ] 1072 | }, 1073 | { 1074 | "cell_type": "code", 1075 | "execution_count": 55, 1076 | "metadata": {}, 1077 | "outputs": [ 1078 | { 1079 | "ename": "MethodError", 1080 | "evalue": "MethodError: objects of type FiniteDifference are not callable", 1081 | "output_type": "error", 1082 | "traceback": [ 1083 | "MethodError: objects of type FiniteDifference are not callable", 1084 | "", 1085 | "Stacktrace:", 1086 | " [1] top-level scope at In[55]:1" 1087 | ] 1088 | } 1089 | ], 1090 | "source": [ 1091 | "fd(10)" 1092 | ] 1093 | }, 1094 | { 1095 | "cell_type": "markdown", 1096 | "metadata": {}, 1097 | "source": [ 1098 | "Tell Julia how to treat it as a function:" 1099 | ] 1100 | }, 1101 | { 1102 | "cell_type": "code", 1103 | "execution_count": 56, 1104 | "metadata": {}, 1105 | "outputs": [], 1106 | "source": [ 1107 | "function (fd::FiniteDifference)(f, x)\n", 1108 | " h = fd.h\n", 1109 | " finite_difference(f, x, h)\n", 1110 | "end" 1111 | ] 1112 | }, 1113 | { 1114 | "cell_type": "code", 1115 | "execution_count": 57, 1116 | "metadata": {}, 1117 | "outputs": [ 1118 | { 1119 | "data": { 1120 | "text/plain": [ 1121 | "5.999999999999339" 1122 | ] 1123 | }, 1124 | "execution_count": 57, 1125 | "metadata": {}, 1126 | "output_type": "execute_result" 1127 | } 1128 | ], 1129 | "source": [ 1130 | "fd(g, a)" 1131 | ] 1132 | }, 1133 | { 1134 | "cell_type": "code", 1135 | "execution_count": 58, 1136 | "metadata": {}, 1137 | "outputs": [], 1138 | "source": [ 1139 | "abstract type RootAlgorithm end" 1140 | ] 1141 | }, 1142 | { 1143 | "cell_type": "code", 1144 | "execution_count": 59, 1145 | "metadata": {}, 1146 | "outputs": [], 1147 | "source": [ 1148 | "struct Newton <: RootAlgorithm end" 1149 | ] 1150 | }, 1151 | { 1152 | "cell_type": "code", 1153 | "execution_count": 71, 1154 | "metadata": {}, 1155 | "outputs": [], 1156 | "source": [ 1157 | "(n::Newton)(args...) = ( @show args; newton(args...) )" 1158 | ] 1159 | }, 1160 | { 1161 | "cell_type": "code", 1162 | "execution_count": 67, 1163 | "metadata": {}, 1164 | "outputs": [ 1165 | { 1166 | "data": { 1167 | "text/html": [ 1168 | "1 method for generic function Type:" 1169 | ], 1170 | "text/plain": [ 1171 | "# 1 method for generic function \"(::Type)\":\n", 1172 | "[1] Newton() in Main at In[59]:1" 1173 | ] 1174 | }, 1175 | "execution_count": 67, 1176 | "metadata": {}, 1177 | "output_type": "execute_result" 1178 | } 1179 | ], 1180 | "source": [ 1181 | "n = Newton()\n", 1182 | "methods(Newton)" 1183 | ] 1184 | }, 1185 | { 1186 | "cell_type": "code", 1187 | "execution_count": 72, 1188 | "metadata": {}, 1189 | "outputs": [ 1190 | { 1191 | "name": "stdout", 1192 | "output_type": "stream", 1193 | "text": [ 1194 | "args = (g, 3.0)\n" 1195 | ] 1196 | }, 1197 | { 1198 | "data": { 1199 | "text/plain": [ 1200 | "6.0" 1201 | ] 1202 | }, 1203 | "execution_count": 72, 1204 | "metadata": {}, 1205 | "output_type": "execute_result" 1206 | } 1207 | ], 1208 | "source": [ 1209 | "newt = Newton()\n", 1210 | "\n", 1211 | "newt(g, a)" 1212 | ] 1213 | }, 1214 | { 1215 | "cell_type": "code", 1216 | "execution_count": 73, 1217 | "metadata": {}, 1218 | "outputs": [], 1219 | "source": [ 1220 | "struct RootProblem\n", 1221 | " f\n", 1222 | " x0\n", 1223 | " algorithm::RootAlgorithm\n", 1224 | "end" 1225 | ] 1226 | }, 1227 | { 1228 | "cell_type": "code", 1229 | "execution_count": 81, 1230 | "metadata": {}, 1231 | "outputs": [ 1232 | { 1233 | "data": { 1234 | "text/plain": [ 1235 | "solve (generic function with 1 method)" 1236 | ] 1237 | }, 1238 | "execution_count": 81, 1239 | "metadata": {}, 1240 | "output_type": "execute_result" 1241 | } 1242 | ], 1243 | "source": [ 1244 | "function solve(prob::RootProblem)\n", 1245 | " prob.algorithm(prob.f, prob.x0)\n", 1246 | "end" 1247 | ] 1248 | }, 1249 | { 1250 | "cell_type": "code", 1251 | "execution_count": 82, 1252 | "metadata": {}, 1253 | "outputs": [ 1254 | { 1255 | "data": { 1256 | "text/plain": [ 1257 | "RootProblem(g, 3.0, Newton())" 1258 | ] 1259 | }, 1260 | "execution_count": 82, 1261 | "metadata": {}, 1262 | "output_type": "execute_result" 1263 | } 1264 | ], 1265 | "source": [ 1266 | "prob = RootProblem(g, a, Newton())" 1267 | ] 1268 | }, 1269 | { 1270 | "cell_type": "code", 1271 | "execution_count": 83, 1272 | "metadata": {}, 1273 | "outputs": [ 1274 | { 1275 | "name": "stdout", 1276 | "output_type": "stream", 1277 | "text": [ 1278 | "args = (g, 3.0)\n" 1279 | ] 1280 | }, 1281 | { 1282 | "data": { 1283 | "text/plain": [ 1284 | "6.0" 1285 | ] 1286 | }, 1287 | "execution_count": 83, 1288 | "metadata": {}, 1289 | "output_type": "execute_result" 1290 | } 1291 | ], 1292 | "source": [ 1293 | "solve(prob)" 1294 | ] 1295 | }, 1296 | { 1297 | "cell_type": "markdown", 1298 | "metadata": {}, 1299 | "source": [ 1300 | "### Type-based dispatch" 1301 | ] 1302 | }, 1303 | { 1304 | "cell_type": "markdown", 1305 | "metadata": {}, 1306 | "source": [ 1307 | "So far we are not using the types to their full advantage: we wish to *dispatch* on the *type* of `algorithm`. We can do so by **parametrising the type**:" 1308 | ] 1309 | }, 1310 | { 1311 | "cell_type": "code", 1312 | "execution_count": 1, 1313 | "metadata": {}, 1314 | "outputs": [], 1315 | "source": [ 1316 | "struct RootProblem2{T<:RootAlgorithm}\n", 1317 | " ...\n", 1318 | " algorithm::T\n", 1319 | "end" 1320 | ] 1321 | }, 1322 | { 1323 | "cell_type": "markdown", 1324 | "metadata": {}, 1325 | "source": [ 1326 | "When we create an object of type `RootProblem2`, we will get a specialised version with the correct type. We can now use that in dispatch:" 1327 | ] 1328 | }, 1329 | { 1330 | "cell_type": "code", 1331 | "execution_count": null, 1332 | "metadata": {}, 1333 | "outputs": [], 1334 | "source": [ 1335 | "solve(prob::RootProblem2{Newton}) = ..." 1336 | ] 1337 | }, 1338 | { 1339 | "cell_type": "code", 1340 | "execution_count": null, 1341 | "metadata": {}, 1342 | "outputs": [], 1343 | "source": [ 1344 | "solve(prob::RootProblem2{Bisection}) = ..." 1345 | ] 1346 | }, 1347 | { 1348 | "cell_type": "code", 1349 | "execution_count": null, 1350 | "metadata": {}, 1351 | "outputs": [], 1352 | "source": [ 1353 | "struct RootProblem2{F, X<:Real, T<:RootAlgorithm}\n", 1354 | " f::F\n", 1355 | " x0::X\n", 1356 | " algorithm::T\n", 1357 | "end" 1358 | ] 1359 | }, 1360 | { 1361 | "cell_type": "code", 1362 | "execution_count": 84, 1363 | "metadata": {}, 1364 | "outputs": [], 1365 | "source": [] 1366 | }, 1367 | { 1368 | "cell_type": "markdown", 1369 | "metadata": {}, 1370 | "source": [ 1371 | "#### Exercise 5\n", 1372 | "\n", 1373 | "1. Implement this.\n", 1374 | "\n", 1375 | "\n", 1376 | "2. Put everything together to be able to solve a root problem using a particular derivative algorithm." 1377 | ] 1378 | }, 1379 | { 1380 | "cell_type": "code", 1381 | "execution_count": null, 1382 | "metadata": {}, 1383 | "outputs": [], 1384 | "source": [] 1385 | }, 1386 | { 1387 | "cell_type": "markdown", 1388 | "metadata": {}, 1389 | "source": [ 1390 | "#### Exercise 6\n", 1391 | "\n", 1392 | "1. Implement a `MultipleRootProblem` type that specifies an interval over which we would like to find all roots.\n", 1393 | "\n", 1394 | "\n", 1395 | "2. Write a simple implementation of the algorithm using multiple starting points in the interval and making a list of unique roots found by that procedure.\n", 1396 | "\n", 1397 | "\n", 1398 | "3. Load the `Polynomials.jl` package and write a dispatch that specialises on polynomials and calls the root finder in that package." 1399 | ] 1400 | }, 1401 | { 1402 | "cell_type": "code", 1403 | "execution_count": 85, 1404 | "metadata": {}, 1405 | "outputs": [], 1406 | "source": [ 1407 | "using Polynomials" 1408 | ] 1409 | }, 1410 | { 1411 | "cell_type": "code", 1412 | "execution_count": 86, 1413 | "metadata": {}, 1414 | "outputs": [ 1415 | { 1416 | "data": { 1417 | "text/html": [ 1418 | "1 + 2∙x + 3∙x^2" 1419 | ], 1420 | "text/latex": [ 1421 | "$1 + 2\\cdot x + 3\\cdot x^{2}$" 1422 | ], 1423 | "text/plain": [ 1424 | "Poly(1 + 2*x + 3*x^2)" 1425 | ] 1426 | }, 1427 | "execution_count": 86, 1428 | "metadata": {}, 1429 | "output_type": "execute_result" 1430 | } 1431 | ], 1432 | "source": [ 1433 | "p = Poly([1, 2, 3])" 1434 | ] 1435 | }, 1436 | { 1437 | "cell_type": "code", 1438 | "execution_count": 87, 1439 | "metadata": {}, 1440 | "outputs": [ 1441 | { 1442 | "data": { 1443 | "text/plain": [ 1444 | "321" 1445 | ] 1446 | }, 1447 | "execution_count": 87, 1448 | "metadata": {}, 1449 | "output_type": "execute_result" 1450 | } 1451 | ], 1452 | "source": [ 1453 | "p(10)" 1454 | ] 1455 | }, 1456 | { 1457 | "cell_type": "code", 1458 | "execution_count": 89, 1459 | "metadata": {}, 1460 | "outputs": [ 1461 | { 1462 | "data": { 1463 | "text/plain": [ 1464 | "false" 1465 | ] 1466 | }, 1467 | "execution_count": 89, 1468 | "metadata": {}, 1469 | "output_type": "execute_result" 1470 | } 1471 | ], 1472 | "source": [ 1473 | "p isa Function" 1474 | ] 1475 | }, 1476 | { 1477 | "cell_type": "markdown", 1478 | "metadata": {}, 1479 | "source": [ 1480 | "## Other uses of types" 1481 | ] 1482 | }, 1483 | { 1484 | "cell_type": "markdown", 1485 | "metadata": {}, 1486 | "source": [ 1487 | "Other examples of different usages of types include:\n", 1488 | "\n", 1489 | "- [`ModelingToolkit.jl`](https://github.com/JuliaDiffEq/ModelingToolkit.jl)\n", 1490 | "\n", 1491 | " Types are introduced to represent variables and operations. In this way it is relatively simple to build up a way to output symbolic expressions from standard Julia functions.\n", 1492 | " \n", 1493 | " \n", 1494 | "- https://github.com/MikeInnes/diff-zoo defines types to represent \"tapes\" recording sequences of operations. This is a precursor to tools such as [`Zygote.jl`](https://github.com/FluxML/Zygote.jl), which performs advanced automatic differentiation on code at a lower level." 1495 | ] 1496 | }, 1497 | { 1498 | "cell_type": "markdown", 1499 | "metadata": {}, 1500 | "source": [ 1501 | "### Traits" 1502 | ] 1503 | }, 1504 | { 1505 | "cell_type": "markdown", 1506 | "metadata": {}, 1507 | "source": [ 1508 | "An important use of types that we have not addressed here is to define **traits**. These are labels that can be assigned to different types that may then be dispatched on, even if those types are not in a Julia type hierarchy.\n", 1509 | "\n", 1510 | "See e.g. the implementation in [`SimpleTraits.jl`](https://github.com/mauro3/SimpleTraits.jl)." 1511 | ] 1512 | } 1513 | ], 1514 | "metadata": { 1515 | "kernelspec": { 1516 | "display_name": "Julia 1.1.0", 1517 | "language": "julia", 1518 | "name": "julia-1.1" 1519 | }, 1520 | "language_info": { 1521 | "file_extension": ".jl", 1522 | "mimetype": "application/julia", 1523 | "name": "julia", 1524 | "version": "1.1.0" 1525 | } 1526 | }, 1527 | "nbformat": 4, 1528 | "nbformat_minor": 2 1529 | } 1530 | -------------------------------------------------------------------------------- /solutions/3. Type-based dispatch.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Structuring programs using types and dispatch" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "Previously we specified an algorithm directly using a function, but saw that it could be tricky to get it right. In many Julia packages it is common to use *dispatch* to do this. \n", 15 | "\n", 16 | "In Julia, **dispatch** refers to choosing which **method** (version) of a function to use, according to the type of the arguments. (**Multiple dispatch** is when the types of several different arguments are involved.)" 17 | ] 18 | }, 19 | { 20 | "cell_type": "markdown", 21 | "metadata": {}, 22 | "source": [ 23 | "Let's define some types to represent different differentiation methods." 24 | ] 25 | }, 26 | { 27 | "cell_type": "code", 28 | "execution_count": 5, 29 | "metadata": {}, 30 | "outputs": [], 31 | "source": [ 32 | "struct Dual \n", 33 | " v::Float64\n", 34 | " d::Float64\n", 35 | "end" 36 | ] 37 | }, 38 | { 39 | "cell_type": "code", 40 | "execution_count": 6, 41 | "metadata": {}, 42 | "outputs": [ 43 | { 44 | "data": { 45 | "text/html": [ 46 | "2 methods for generic function Type:" 47 | ], 48 | "text/plain": [ 49 | "# 2 methods for generic function \"(::Type)\":\n", 50 | "[1] Dual(v::Float64, d::Float64) in Main at In[5]:2\n", 51 | "[2] Dual(v, d) in Main at In[5]:2" 52 | ] 53 | }, 54 | "execution_count": 6, 55 | "metadata": {}, 56 | "output_type": "execute_result" 57 | } 58 | ], 59 | "source": [ 60 | "methods(Dual)" 61 | ] 62 | }, 63 | { 64 | "cell_type": "markdown", 65 | "metadata": {}, 66 | "source": [ 67 | "#### Exercise 1\n", 68 | "\n", 69 | "1. Define an abstract type `DifferentiationAlgorithm`.\n", 70 | "\n", 71 | "\n", 72 | "2. Define subtypes `FiniteDifference`, `MyAutoDiff` (for our implementation) and `AutoDiff` (for the `ForwardDiff` implementation).\n", 73 | "\n", 74 | "\n", 75 | "3. Implement the function `derivative(f, x, algorithm)` using **dispatch**: for each of the three types, define a version of this function in which `algorithm` is specified to be of that type by using the type annotation operator `::`.\n", 76 | "\n", 77 | "\n", 78 | "4. Verify that these work by writing tests for them." 79 | ] 80 | }, 81 | { 82 | "cell_type": "markdown", 83 | "metadata": {}, 84 | "source": [ 85 | "#### Solution 1" 86 | ] 87 | }, 88 | { 89 | "cell_type": "markdown", 90 | "metadata": {}, 91 | "source": [ 92 | "Note that we could do something like the following:" 93 | ] 94 | }, 95 | { 96 | "cell_type": "code", 97 | "execution_count": 4, 98 | "metadata": {}, 99 | "outputs": [ 100 | { 101 | "ename": "LoadError", 102 | "evalue": "syntax: invalid identifier name \"...\"", 103 | "output_type": "error", 104 | "traceback": [ 105 | "syntax: invalid identifier name \"...\"", 106 | "" 107 | ] 108 | } 109 | ], 110 | "source": [ 111 | "function derivative(f, x, algorithm)\n", 112 | " if algorithm == :newton\n", 113 | " ...\n", 114 | " elseif algorithm == :bisection\n", 115 | " ...\n", 116 | " elseif\n", 117 | " ...\n", 118 | " end\n", 119 | "end" 120 | ] 121 | }, 122 | { 123 | "cell_type": "markdown", 124 | "metadata": {}, 125 | "source": [ 126 | "But if there are many functions then this gets messy; it is also difficult to specify behaviour which is common to only some of the options.\n", 127 | "\n", 128 | "A better alternative in Julia is to use **type-based dispatch**: since Julia already has the multiple dispatch mechanism, which chooses a method of a function based on the types of all its arguments, we can exploit this to direct the flow of code by creating new types.\n", 129 | "\n", 130 | "In many cases it may be that the types will not actually need to have any fields; the types are a method to control the flow of logic through the program." 131 | ] 132 | }, 133 | { 134 | "cell_type": "markdown", 135 | "metadata": {}, 136 | "source": [ 137 | "For example, let's create an abstract type for any kind of differentiation algorithm, together with two concrete subtypes: a finite difference type that has a field for the step size, and an automatic differentiation type that has no fields:" 138 | ] 139 | }, 140 | { 141 | "cell_type": "code", 142 | "execution_count": 17, 143 | "metadata": {}, 144 | "outputs": [], 145 | "source": [ 146 | "abstract type DifferentiationAlgorithm end" 147 | ] 148 | }, 149 | { 150 | "cell_type": "code", 151 | "execution_count": 18, 152 | "metadata": {}, 153 | "outputs": [], 154 | "source": [ 155 | "struct FiniteDifference <: DifferentiationAlgorithm \n", 156 | " h::Float64\n", 157 | "end" 158 | ] 159 | }, 160 | { 161 | "cell_type": "code", 162 | "execution_count": 19, 163 | "metadata": {}, 164 | "outputs": [], 165 | "source": [ 166 | "struct AutoDiff <: DifferentiationAlgorithm end" 167 | ] 168 | }, 169 | { 170 | "cell_type": "markdown", 171 | "metadata": {}, 172 | "source": [ 173 | "Recall the differentiation functions that we defined in a previous notebook:" 174 | ] 175 | }, 176 | { 177 | "cell_type": "code", 178 | "execution_count": 20, 179 | "metadata": {}, 180 | "outputs": [ 181 | { 182 | "data": { 183 | "text/plain": [ 184 | "finite_difference (generic function with 1 method)" 185 | ] 186 | }, 187 | "execution_count": 20, 188 | "metadata": {}, 189 | "output_type": "execute_result" 190 | } 191 | ], 192 | "source": [ 193 | "finite_difference(f, a, h) = (f(a + h) - f(a - h)) / (2h)" 194 | ] 195 | }, 196 | { 197 | "cell_type": "code", 198 | "execution_count": 21, 199 | "metadata": {}, 200 | "outputs": [ 201 | { 202 | "data": { 203 | "text/plain": [ 204 | "forwarddiff (generic function with 1 method)" 205 | ] 206 | }, 207 | "execution_count": 21, 208 | "metadata": {}, 209 | "output_type": "execute_result" 210 | } 211 | ], 212 | "source": [ 213 | "using ForwardDiff\n", 214 | "forwarddiff(f, x) = ForwardDiff.derivative(f, x)" 215 | ] 216 | }, 217 | { 218 | "cell_type": "markdown", 219 | "metadata": {}, 220 | "source": [ 221 | "Now we can make a *uniform interface* by dispatching on the type of algorithm:" 222 | ] 223 | }, 224 | { 225 | "cell_type": "code", 226 | "execution_count": 22, 227 | "metadata": {}, 228 | "outputs": [ 229 | { 230 | "data": { 231 | "text/plain": [ 232 | "derivative (generic function with 2 methods)" 233 | ] 234 | }, 235 | "execution_count": 22, 236 | "metadata": {}, 237 | "output_type": "execute_result" 238 | } 239 | ], 240 | "source": [ 241 | "function derivative(f, x, algorithm::AutoDiff)\n", 242 | " return forwarddiff(f, x)\n", 243 | "end" 244 | ] 245 | }, 246 | { 247 | "cell_type": "code", 248 | "execution_count": 23, 249 | "metadata": {}, 250 | "outputs": [ 251 | { 252 | "data": { 253 | "text/plain": [ 254 | "3.0" 255 | ] 256 | }, 257 | "execution_count": 23, 258 | "metadata": {}, 259 | "output_type": "execute_result" 260 | } 261 | ], 262 | "source": [ 263 | "g(x) = x^2 - 2\n", 264 | "a = 3.0" 265 | ] 266 | }, 267 | { 268 | "cell_type": "code", 269 | "execution_count": 24, 270 | "metadata": {}, 271 | "outputs": [ 272 | { 273 | "ename": "MethodError", 274 | "evalue": "MethodError: no method matching derivative(::typeof(g), ::Float64, ::Type{AutoDiff})\nClosest candidates are:\n derivative(::Any, ::Any, !Matched::FiniteDifference) at In[15]:2\n derivative(::Any, ::Any, !Matched::AutoDiff) at In[22]:2", 275 | "output_type": "error", 276 | "traceback": [ 277 | "MethodError: no method matching derivative(::typeof(g), ::Float64, ::Type{AutoDiff})\nClosest candidates are:\n derivative(::Any, ::Any, !Matched::FiniteDifference) at In[15]:2\n derivative(::Any, ::Any, !Matched::AutoDiff) at In[22]:2", 278 | "", 279 | "Stacktrace:", 280 | " [1] top-level scope at In[24]:1" 281 | ] 282 | } 283 | ], 284 | "source": [ 285 | "derivative(g, a, AutoDiff)" 286 | ] 287 | }, 288 | { 289 | "cell_type": "markdown", 290 | "metadata": {}, 291 | "source": [ 292 | "What happened here?" 293 | ] 294 | }, 295 | { 296 | "cell_type": "code", 297 | "execution_count": 25, 298 | "metadata": {}, 299 | "outputs": [ 300 | { 301 | "data": { 302 | "text/plain": [ 303 | "DataType" 304 | ] 305 | }, 306 | "execution_count": 25, 307 | "metadata": {}, 308 | "output_type": "execute_result" 309 | } 310 | ], 311 | "source": [ 312 | "typeof(AutoDiff)" 313 | ] 314 | }, 315 | { 316 | "cell_type": "code", 317 | "execution_count": 26, 318 | "metadata": {}, 319 | "outputs": [ 320 | { 321 | "data": { 322 | "text/plain": [ 323 | "AutoDiff()" 324 | ] 325 | }, 326 | "execution_count": 26, 327 | "metadata": {}, 328 | "output_type": "execute_result" 329 | } 330 | ], 331 | "source": [ 332 | "autodiff = AutoDiff()" 333 | ] 334 | }, 335 | { 336 | "cell_type": "code", 337 | "execution_count": 27, 338 | "metadata": {}, 339 | "outputs": [ 340 | { 341 | "data": { 342 | "text/plain": [ 343 | "AutoDiff" 344 | ] 345 | }, 346 | "execution_count": 27, 347 | "metadata": {}, 348 | "output_type": "execute_result" 349 | } 350 | ], 351 | "source": [ 352 | "typeof(autodiff)" 353 | ] 354 | }, 355 | { 356 | "cell_type": "code", 357 | "execution_count": 28, 358 | "metadata": {}, 359 | "outputs": [ 360 | { 361 | "data": { 362 | "text/plain": [ 363 | "6.0" 364 | ] 365 | }, 366 | "execution_count": 28, 367 | "metadata": {}, 368 | "output_type": "execute_result" 369 | } 370 | ], 371 | "source": [ 372 | "derivative(g, a, autodiff)" 373 | ] 374 | }, 375 | { 376 | "cell_type": "code", 377 | "execution_count": 29, 378 | "metadata": {}, 379 | "outputs": [ 380 | { 381 | "data": { 382 | "text/plain": [ 383 | "6.0" 384 | ] 385 | }, 386 | "execution_count": 29, 387 | "metadata": {}, 388 | "output_type": "execute_result" 389 | } 390 | ], 391 | "source": [ 392 | "derivative(g, a, AutoDiff())" 393 | ] 394 | }, 395 | { 396 | "cell_type": "markdown", 397 | "metadata": {}, 398 | "source": [ 399 | "We needed an *instance* of the type, i.e. an object of that type, rather than the type itself." 400 | ] 401 | }, 402 | { 403 | "cell_type": "code", 404 | "execution_count": 30, 405 | "metadata": {}, 406 | "outputs": [ 407 | { 408 | "data": { 409 | "text/plain": [ 410 | "derivative (generic function with 2 methods)" 411 | ] 412 | }, 413 | "execution_count": 30, 414 | "metadata": {}, 415 | "output_type": "execute_result" 416 | } 417 | ], 418 | "source": [ 419 | "function derivative(f, x, algorithm::FiniteDifference)\n", 420 | " h = algorithm.h # extract step size\n", 421 | " return finite_difference(f, x, h)\n", 422 | "end" 423 | ] 424 | }, 425 | { 426 | "cell_type": "code", 427 | "execution_count": 31, 428 | "metadata": {}, 429 | "outputs": [ 430 | { 431 | "ename": "MethodError", 432 | "evalue": "MethodError: no method matching FiniteDifference()\nClosest candidates are:\n FiniteDifference(!Matched::Float64) at In[18]:2\n FiniteDifference(!Matched::Any) at In[18]:2", 433 | "output_type": "error", 434 | "traceback": [ 435 | "MethodError: no method matching FiniteDifference()\nClosest candidates are:\n FiniteDifference(!Matched::Float64) at In[18]:2\n FiniteDifference(!Matched::Any) at In[18]:2", 436 | "", 437 | "Stacktrace:", 438 | " [1] top-level scope at In[31]:1" 439 | ] 440 | } 441 | ], 442 | "source": [ 443 | "derivative(g, a, FiniteDifference())" 444 | ] 445 | }, 446 | { 447 | "cell_type": "code", 448 | "execution_count": 32, 449 | "metadata": {}, 450 | "outputs": [ 451 | { 452 | "data": { 453 | "text/plain": [ 454 | "5.999999999999872" 455 | ] 456 | }, 457 | "execution_count": 32, 458 | "metadata": {}, 459 | "output_type": "execute_result" 460 | } 461 | ], 462 | "source": [ 463 | "derivative(g, a, FiniteDifference(0.01))" 464 | ] 465 | }, 466 | { 467 | "cell_type": "code", 468 | "execution_count": 34, 469 | "metadata": {}, 470 | "outputs": [ 471 | { 472 | "data": { 473 | "text/plain": [ 474 | "FiniteDifference(0.01)" 475 | ] 476 | }, 477 | "execution_count": 34, 478 | "metadata": {}, 479 | "output_type": "execute_result" 480 | } 481 | ], 482 | "source": [ 483 | "algorithm = FiniteDifference(0.01)" 484 | ] 485 | }, 486 | { 487 | "cell_type": "code", 488 | "execution_count": 35, 489 | "metadata": {}, 490 | "outputs": [ 491 | { 492 | "data": { 493 | "text/plain": [ 494 | "FiniteDifference" 495 | ] 496 | }, 497 | "execution_count": 35, 498 | "metadata": {}, 499 | "output_type": "execute_result" 500 | } 501 | ], 502 | "source": [ 503 | "typeof(algorithm)" 504 | ] 505 | }, 506 | { 507 | "cell_type": "code", 508 | "execution_count": 36, 509 | "metadata": {}, 510 | "outputs": [ 511 | { 512 | "data": { 513 | "text/plain": [ 514 | "0.01" 515 | ] 516 | }, 517 | "execution_count": 36, 518 | "metadata": {}, 519 | "output_type": "execute_result" 520 | } 521 | ], 522 | "source": [ 523 | "algorithm.h" 524 | ] 525 | }, 526 | { 527 | "cell_type": "code", 528 | "execution_count": 37, 529 | "metadata": {}, 530 | "outputs": [ 531 | { 532 | "data": { 533 | "text/plain": [ 534 | "(:h,)" 535 | ] 536 | }, 537 | "execution_count": 37, 538 | "metadata": {}, 539 | "output_type": "execute_result" 540 | } 541 | ], 542 | "source": [ 543 | "fieldnames(typeof(algorithm))" 544 | ] 545 | }, 546 | { 547 | "cell_type": "code", 548 | "execution_count": 32, 549 | "metadata": {}, 550 | "outputs": [ 551 | { 552 | "data": { 553 | "text/html": [ 554 | "2 methods for generic function Type:" 555 | ], 556 | "text/plain": [ 557 | "# 2 methods for generic function \"(::Type)\":\n", 558 | "[1] FiniteDifference(h::Float64) in Main at In[2]:2\n", 559 | "[2] FiniteDifference(h) in Main at In[2]:2" 560 | ] 561 | }, 562 | "execution_count": 32, 563 | "metadata": {}, 564 | "output_type": "execute_result" 565 | } 566 | ], 567 | "source": [ 568 | "methods(FiniteDifference)" 569 | ] 570 | }, 571 | { 572 | "cell_type": "markdown", 573 | "metadata": {}, 574 | "source": [ 575 | "We can specify a default step size by adding a new constructor:" 576 | ] 577 | }, 578 | { 579 | "cell_type": "code", 580 | "execution_count": 39, 581 | "metadata": {}, 582 | "outputs": [ 583 | { 584 | "data": { 585 | "text/plain": [ 586 | "FiniteDifference" 587 | ] 588 | }, 589 | "execution_count": 39, 590 | "metadata": {}, 591 | "output_type": "execute_result" 592 | } 593 | ], 594 | "source": [ 595 | "FiniteDifference() = FiniteDifference(0.001)" 596 | ] 597 | }, 598 | { 599 | "cell_type": "markdown", 600 | "metadata": {}, 601 | "source": [ 602 | "There is currently no syntax in Base Julia for default values of fields, although this is possible using the [Paramters.jl package](https://github.com/mauro3/Parameters.jl)." 603 | ] 604 | }, 605 | { 606 | "cell_type": "code", 607 | "execution_count": 40, 608 | "metadata": {}, 609 | "outputs": [ 610 | { 611 | "data": { 612 | "text/html": [ 613 | "3 methods for generic function Type:" 614 | ], 615 | "text/plain": [ 616 | "# 3 methods for generic function \"(::Type)\":\n", 617 | "[1] FiniteDifference() in Main at In[39]:1\n", 618 | "[2] FiniteDifference(h::Float64) in Main at In[18]:2\n", 619 | "[3] FiniteDifference(h) in Main at In[18]:2" 620 | ] 621 | }, 622 | "execution_count": 40, 623 | "metadata": {}, 624 | "output_type": "execute_result" 625 | } 626 | ], 627 | "source": [ 628 | "methods(FiniteDifference)" 629 | ] 630 | }, 631 | { 632 | "cell_type": "code", 633 | "execution_count": 41, 634 | "metadata": {}, 635 | "outputs": [ 636 | { 637 | "data": { 638 | "text/plain": [ 639 | "FiniteDifference(0.001)" 640 | ] 641 | }, 642 | "execution_count": 41, 643 | "metadata": {}, 644 | "output_type": "execute_result" 645 | } 646 | ], 647 | "source": [ 648 | "FiniteDifference()" 649 | ] 650 | }, 651 | { 652 | "cell_type": "code", 653 | "execution_count": 42, 654 | "metadata": {}, 655 | "outputs": [ 656 | { 657 | "data": { 658 | "text/plain": [ 659 | "5.999999999999339" 660 | ] 661 | }, 662 | "execution_count": 42, 663 | "metadata": {}, 664 | "output_type": "execute_result" 665 | } 666 | ], 667 | "source": [ 668 | "derivative(g, a, FiniteDifference() )" 669 | ] 670 | }, 671 | { 672 | "cell_type": "code", 673 | "execution_count": 43, 674 | "metadata": {}, 675 | "outputs": [ 676 | { 677 | "data": { 678 | "text/html": [ 679 | "2 methods for generic function derivative:" 680 | ], 681 | "text/plain": [ 682 | "# 2 methods for generic function \"derivative\":\n", 683 | "[1] derivative(f, x, algorithm::AutoDiff) in Main at In[22]:2\n", 684 | "[2] derivative(f, x, algorithm::FiniteDifference) in Main at In[30]:2" 685 | ] 686 | }, 687 | "execution_count": 43, 688 | "metadata": {}, 689 | "output_type": "execute_result" 690 | } 691 | ], 692 | "source": [ 693 | "methods(derivative)" 694 | ] 695 | }, 696 | { 697 | "cell_type": "markdown", 698 | "metadata": {}, 699 | "source": [ 700 | "#### Exercise 2" 701 | ] 702 | }, 703 | { 704 | "cell_type": "markdown", 705 | "metadata": {}, 706 | "source": [ 707 | "1. Write a version of the Newton algorithm that takes an optional argument `algorithm` specifying the differentiation algorithm to use, and which has a default value of `AutoDiff`." 708 | ] 709 | }, 710 | { 711 | "cell_type": "markdown", 712 | "metadata": {}, 713 | "source": [ 714 | "#### Solution 2" 715 | ] 716 | }, 717 | { 718 | "cell_type": "code", 719 | "execution_count": 45, 720 | "metadata": {}, 721 | "outputs": [ 722 | { 723 | "data": { 724 | "text/plain": [ 725 | "newton (generic function with 3 methods)" 726 | ] 727 | }, 728 | "execution_count": 45, 729 | "metadata": {}, 730 | "output_type": "execute_result" 731 | } 732 | ], 733 | "source": [ 734 | "function newton(f, df, x0, n=10) # n=10 specifies a default value\n", 735 | " x = x0 # initialise\n", 736 | " \n", 737 | " for i in 1:n\n", 738 | " x_new = x - f(x) / df(x)\n", 739 | " \n", 740 | " x = x_new # update for next step\n", 741 | " end\n", 742 | " \n", 743 | " return x\n", 744 | "end" 745 | ] 746 | }, 747 | { 748 | "cell_type": "code", 749 | "execution_count": 54, 750 | "metadata": {}, 751 | "outputs": [ 752 | { 753 | "data": { 754 | "text/plain": [ 755 | "newton (generic function with 4 methods)" 756 | ] 757 | }, 758 | "execution_count": 54, 759 | "metadata": {}, 760 | "output_type": "execute_result" 761 | } 762 | ], 763 | "source": [ 764 | "function newton(f, x0, n=10, algorithm::DifferentiationAlgorithm=AutoDiff())\n", 765 | " \n", 766 | " df = x -> derivative(f, x, algorithm)\n", 767 | " \n", 768 | " newton(f, df, x0, n) # requires the code for Newton from the previous notebook\n", 769 | " \n", 770 | "end" 771 | ] 772 | }, 773 | { 774 | "cell_type": "code", 775 | "execution_count": 55, 776 | "metadata": {}, 777 | "outputs": [ 778 | { 779 | "data": { 780 | "text/html": [ 781 | "4 methods for generic function newton:" 782 | ], 783 | "text/plain": [ 784 | "# 4 methods for generic function \"newton\":\n", 785 | "[1] newton(f, x0) in Main at In[54]:3\n", 786 | "[2] newton(f, x0, n; algorithm) in Main at In[54]:3\n", 787 | "[3] newton(f, x0, n, algorithm::DifferentiationAlgorithm) in Main at In[54]:3\n", 788 | "[4] newton(f, df, x0, n) in Main at In[45]:2" 789 | ] 790 | }, 791 | "execution_count": 55, 792 | "metadata": {}, 793 | "output_type": "execute_result" 794 | } 795 | ], 796 | "source": [ 797 | "methods(newton)" 798 | ] 799 | }, 800 | { 801 | "cell_type": "code", 802 | "execution_count": 52, 803 | "metadata": {}, 804 | "outputs": [ 805 | { 806 | "data": { 807 | "text/plain": [ 808 | "1.2599210498948732" 809 | ] 810 | }, 811 | "execution_count": 52, 812 | "metadata": {}, 813 | "output_type": "execute_result" 814 | } 815 | ], 816 | "source": [ 817 | "g(x) = x^3 - 2\n", 818 | "x0 = 3.0\n", 819 | "\n", 820 | "newton(g, x0)" 821 | ] 822 | }, 823 | { 824 | "cell_type": "markdown", 825 | "metadata": {}, 826 | "source": [ 827 | "Note that Base Julia does not dispatch on keyword arguments, although this is possible using the [KeywordDispatch.jl package](https://github.com/simonbyrne/KeywordDispatch.jl).\n", 828 | "\n", 829 | "It's important to make sure that you don't overwrite other methods that you need." 830 | ] 831 | }, 832 | { 833 | "cell_type": "markdown", 834 | "metadata": {}, 835 | "source": [ 836 | "## Functions as types" 837 | ] 838 | }, 839 | { 840 | "cell_type": "markdown", 841 | "metadata": {}, 842 | "source": [ 843 | "How could we differentiate the `sin` function? Of course, we can do it using e.g. finite differences, but in cases like this we actually know the exact, analytical derivative, in this case `cos`, i.e $\\sin'(x) = \\cos(x)$.\n", 844 | "Is there are a way to tell Julia to use this directly? I.e. if `f` is equal to the `sin` function, then we should make a special version of our `derivative` function.\n", 845 | "\n", 846 | "It turns out that there is, using dispatch, by checking what *type* the `sin` function has:" 847 | ] 848 | }, 849 | { 850 | "cell_type": "markdown", 851 | "metadata": {}, 852 | "source": [ 853 | "#### Exercise 3\n", 854 | "\n", 855 | "1. Use `typeof` to find the type of the `sin` function.\n", 856 | "\n", 857 | "\n", 858 | "2. Use this to make a special dispatch for `derivative(sin, x)`." 859 | ] 860 | }, 861 | { 862 | "cell_type": "markdown", 863 | "metadata": {}, 864 | "source": [ 865 | "#### Solution 3" 866 | ] 867 | }, 868 | { 869 | "cell_type": "code", 870 | "execution_count": 44, 871 | "metadata": {}, 872 | "outputs": [ 873 | { 874 | "data": { 875 | "text/plain": [ 876 | "typeof(sin)" 877 | ] 878 | }, 879 | "execution_count": 44, 880 | "metadata": {}, 881 | "output_type": "execute_result" 882 | } 883 | ], 884 | "source": [ 885 | "typeof(sin)" 886 | ] 887 | }, 888 | { 889 | "cell_type": "code", 890 | "execution_count": 45, 891 | "metadata": {}, 892 | "outputs": [ 893 | { 894 | "data": { 895 | "text/plain": [ 896 | "derivative (generic function with 3 methods)" 897 | ] 898 | }, 899 | "execution_count": 45, 900 | "metadata": {}, 901 | "output_type": "execute_result" 902 | } 903 | ], 904 | "source": [ 905 | "derivative(::typeof(sin), x) = cos(x)" 906 | ] 907 | }, 908 | { 909 | "cell_type": "markdown", 910 | "metadata": {}, 911 | "source": [ 912 | "The package [`ChainRules.jl`](https://github.com/JuliaDiff/ChainRules.jl) contains definitions like this and is used inside `ForwardDiff.jl` and other packages that need to know the derivatives of functions." 913 | ] 914 | }, 915 | { 916 | "cell_type": "markdown", 917 | "metadata": {}, 918 | "source": [ 919 | "## Representing a problem using a type" 920 | ] 921 | }, 922 | { 923 | "cell_type": "markdown", 924 | "metadata": {}, 925 | "source": [ 926 | "A root-finding problem requires several pieces of information: a function, a starting point, a root-finding algorithm to use, possibly a derivative, etc. We could just pass all of these as arguments to a function.\n", 927 | "An alternative is to wrap up the various pieces of information into a new composite type." 928 | ] 929 | }, 930 | { 931 | "cell_type": "markdown", 932 | "metadata": {}, 933 | "source": [ 934 | "#### Exercise 4\n", 935 | "\n", 936 | "1. Make a `RootAlgorithm` abstract type.\n", 937 | "\n", 938 | "\n", 939 | "2. Make a `Newton` subtype.\n", 940 | "\n", 941 | "\n", 942 | "3. Make `Newton` a *callable* type using the syntax\n", 943 | "\n", 944 | " ```\n", 945 | " function (algorithm::Newton)(x)\n", 946 | " ...\n", 947 | " end\n", 948 | " \n", 949 | " ```\n", 950 | " \n", 951 | " This means that you will be able to call an object of type `Newton` as if it were a standard function, by passing in an argument. (You can add further arguments as necessary.)\n", 952 | "\n", 953 | "\n", 954 | "4. Make a `RootProblem` type that contains the necessary information for a root problem. Do not specify types for the fields in this type yet. One of the fields should be called `algorithm`. \n", 955 | "\n", 956 | "\n", 957 | "5. Make a function `solve` that calls the field `algorithm` as a function." 958 | ] 959 | }, 960 | { 961 | "cell_type": "code", 962 | "execution_count": 46, 963 | "metadata": {}, 964 | "outputs": [], 965 | "source": [ 966 | "struct MyNewton\n", 967 | "end" 968 | ] 969 | }, 970 | { 971 | "cell_type": "code", 972 | "execution_count": 48, 973 | "metadata": {}, 974 | "outputs": [ 975 | { 976 | "data": { 977 | "text/plain": [ 978 | "MyNewton()" 979 | ] 980 | }, 981 | "execution_count": 48, 982 | "metadata": {}, 983 | "output_type": "execute_result" 984 | } 985 | ], 986 | "source": [ 987 | "mynewton = MyNewton()" 988 | ] 989 | }, 990 | { 991 | "cell_type": "code", 992 | "execution_count": 52, 993 | "metadata": {}, 994 | "outputs": [], 995 | "source": [ 996 | "function (f::MyNewton)(x)\n", 997 | " return 3x\n", 998 | "end" 999 | ] 1000 | }, 1001 | { 1002 | "cell_type": "code", 1003 | "execution_count": 53, 1004 | "metadata": {}, 1005 | "outputs": [ 1006 | { 1007 | "data": { 1008 | "text/plain": [ 1009 | "30" 1010 | ] 1011 | }, 1012 | "execution_count": 53, 1013 | "metadata": {}, 1014 | "output_type": "execute_result" 1015 | } 1016 | ], 1017 | "source": [ 1018 | "mynewton(10)" 1019 | ] 1020 | }, 1021 | { 1022 | "cell_type": "markdown", 1023 | "metadata": {}, 1024 | "source": [ 1025 | "Make FiniteDifference into a callable object:" 1026 | ] 1027 | }, 1028 | { 1029 | "cell_type": "code", 1030 | "execution_count": 54, 1031 | "metadata": {}, 1032 | "outputs": [ 1033 | { 1034 | "data": { 1035 | "text/plain": [ 1036 | "FiniteDifference(0.001)" 1037 | ] 1038 | }, 1039 | "execution_count": 54, 1040 | "metadata": {}, 1041 | "output_type": "execute_result" 1042 | } 1043 | ], 1044 | "source": [ 1045 | "fd = FiniteDifference()" 1046 | ] 1047 | }, 1048 | { 1049 | "cell_type": "markdown", 1050 | "metadata": {}, 1051 | "source": [ 1052 | "If I try to treat `fd` as a function, what happens?" 1053 | ] 1054 | }, 1055 | { 1056 | "cell_type": "code", 1057 | "execution_count": 55, 1058 | "metadata": {}, 1059 | "outputs": [ 1060 | { 1061 | "ename": "MethodError", 1062 | "evalue": "MethodError: objects of type FiniteDifference are not callable", 1063 | "output_type": "error", 1064 | "traceback": [ 1065 | "MethodError: objects of type FiniteDifference are not callable", 1066 | "", 1067 | "Stacktrace:", 1068 | " [1] top-level scope at In[55]:1" 1069 | ] 1070 | } 1071 | ], 1072 | "source": [ 1073 | "fd(10)" 1074 | ] 1075 | }, 1076 | { 1077 | "cell_type": "markdown", 1078 | "metadata": {}, 1079 | "source": [ 1080 | "Tell Julia how to treat it as a function:" 1081 | ] 1082 | }, 1083 | { 1084 | "cell_type": "code", 1085 | "execution_count": 56, 1086 | "metadata": {}, 1087 | "outputs": [], 1088 | "source": [ 1089 | "function (fd::FiniteDifference)(f, x)\n", 1090 | " h = fd.h\n", 1091 | " finite_difference(f, x, h)\n", 1092 | "end" 1093 | ] 1094 | }, 1095 | { 1096 | "cell_type": "code", 1097 | "execution_count": 57, 1098 | "metadata": {}, 1099 | "outputs": [ 1100 | { 1101 | "data": { 1102 | "text/plain": [ 1103 | "5.999999999999339" 1104 | ] 1105 | }, 1106 | "execution_count": 57, 1107 | "metadata": {}, 1108 | "output_type": "execute_result" 1109 | } 1110 | ], 1111 | "source": [ 1112 | "fd(g, a)" 1113 | ] 1114 | }, 1115 | { 1116 | "cell_type": "code", 1117 | "execution_count": 58, 1118 | "metadata": {}, 1119 | "outputs": [], 1120 | "source": [ 1121 | "abstract type RootAlgorithm end" 1122 | ] 1123 | }, 1124 | { 1125 | "cell_type": "code", 1126 | "execution_count": 59, 1127 | "metadata": {}, 1128 | "outputs": [], 1129 | "source": [ 1130 | "struct Newton <: RootAlgorithm end" 1131 | ] 1132 | }, 1133 | { 1134 | "cell_type": "code", 1135 | "execution_count": 71, 1136 | "metadata": {}, 1137 | "outputs": [], 1138 | "source": [ 1139 | "(n::Newton)(args...) = ( @show args; newton(args...) )" 1140 | ] 1141 | }, 1142 | { 1143 | "cell_type": "code", 1144 | "execution_count": 67, 1145 | "metadata": {}, 1146 | "outputs": [ 1147 | { 1148 | "data": { 1149 | "text/html": [ 1150 | "1 method for generic function Type:" 1151 | ], 1152 | "text/plain": [ 1153 | "# 1 method for generic function \"(::Type)\":\n", 1154 | "[1] Newton() in Main at In[59]:1" 1155 | ] 1156 | }, 1157 | "execution_count": 67, 1158 | "metadata": {}, 1159 | "output_type": "execute_result" 1160 | } 1161 | ], 1162 | "source": [ 1163 | "n = Newton()\n", 1164 | "methods(Newton)" 1165 | ] 1166 | }, 1167 | { 1168 | "cell_type": "code", 1169 | "execution_count": 72, 1170 | "metadata": {}, 1171 | "outputs": [ 1172 | { 1173 | "name": "stdout", 1174 | "output_type": "stream", 1175 | "text": [ 1176 | "args = (g, 3.0)\n" 1177 | ] 1178 | }, 1179 | { 1180 | "data": { 1181 | "text/plain": [ 1182 | "6.0" 1183 | ] 1184 | }, 1185 | "execution_count": 72, 1186 | "metadata": {}, 1187 | "output_type": "execute_result" 1188 | } 1189 | ], 1190 | "source": [ 1191 | "newt = Newton()\n", 1192 | "\n", 1193 | "newt(g, a)" 1194 | ] 1195 | }, 1196 | { 1197 | "cell_type": "code", 1198 | "execution_count": 73, 1199 | "metadata": {}, 1200 | "outputs": [], 1201 | "source": [ 1202 | "struct RootProblem\n", 1203 | " f\n", 1204 | " x0\n", 1205 | " algorithm::RootAlgorithm\n", 1206 | "end" 1207 | ] 1208 | }, 1209 | { 1210 | "cell_type": "code", 1211 | "execution_count": 81, 1212 | "metadata": {}, 1213 | "outputs": [ 1214 | { 1215 | "data": { 1216 | "text/plain": [ 1217 | "solve (generic function with 1 method)" 1218 | ] 1219 | }, 1220 | "execution_count": 81, 1221 | "metadata": {}, 1222 | "output_type": "execute_result" 1223 | } 1224 | ], 1225 | "source": [ 1226 | "function solve(prob::RootProblem)\n", 1227 | " prob.algorithm(prob.f, prob.x0)\n", 1228 | "end" 1229 | ] 1230 | }, 1231 | { 1232 | "cell_type": "code", 1233 | "execution_count": 82, 1234 | "metadata": {}, 1235 | "outputs": [ 1236 | { 1237 | "data": { 1238 | "text/plain": [ 1239 | "RootProblem(g, 3.0, Newton())" 1240 | ] 1241 | }, 1242 | "execution_count": 82, 1243 | "metadata": {}, 1244 | "output_type": "execute_result" 1245 | } 1246 | ], 1247 | "source": [ 1248 | "prob = RootProblem(g, a, Newton())" 1249 | ] 1250 | }, 1251 | { 1252 | "cell_type": "code", 1253 | "execution_count": 83, 1254 | "metadata": {}, 1255 | "outputs": [ 1256 | { 1257 | "name": "stdout", 1258 | "output_type": "stream", 1259 | "text": [ 1260 | "args = (g, 3.0)\n" 1261 | ] 1262 | }, 1263 | { 1264 | "data": { 1265 | "text/plain": [ 1266 | "6.0" 1267 | ] 1268 | }, 1269 | "execution_count": 83, 1270 | "metadata": {}, 1271 | "output_type": "execute_result" 1272 | } 1273 | ], 1274 | "source": [ 1275 | "solve(prob)" 1276 | ] 1277 | }, 1278 | { 1279 | "cell_type": "markdown", 1280 | "metadata": {}, 1281 | "source": [ 1282 | "### Type-based dispatch" 1283 | ] 1284 | }, 1285 | { 1286 | "cell_type": "markdown", 1287 | "metadata": {}, 1288 | "source": [ 1289 | "So far we are not using the types to their full advantage: we wish to *dispatch* on the *type* of `algorithm`. We can do so by **parametrising the type**:" 1290 | ] 1291 | }, 1292 | { 1293 | "cell_type": "code", 1294 | "execution_count": 1, 1295 | "metadata": {}, 1296 | "outputs": [], 1297 | "source": [ 1298 | "struct RootProblem2{T<:RootAlgorithm}\n", 1299 | " ...\n", 1300 | " algorithm::T\n", 1301 | "end" 1302 | ] 1303 | }, 1304 | { 1305 | "cell_type": "markdown", 1306 | "metadata": {}, 1307 | "source": [ 1308 | "When we create an object of type `RootProblem2`, we will get a specialised version with the correct type. We can now use that in dispatch:" 1309 | ] 1310 | }, 1311 | { 1312 | "cell_type": "code", 1313 | "execution_count": null, 1314 | "metadata": {}, 1315 | "outputs": [], 1316 | "source": [ 1317 | "solve(prob::RootProblem2{Newton}) = ..." 1318 | ] 1319 | }, 1320 | { 1321 | "cell_type": "code", 1322 | "execution_count": null, 1323 | "metadata": {}, 1324 | "outputs": [], 1325 | "source": [ 1326 | "solve(prob::RootProblem2{Bisection}) = ..." 1327 | ] 1328 | }, 1329 | { 1330 | "cell_type": "code", 1331 | "execution_count": null, 1332 | "metadata": {}, 1333 | "outputs": [], 1334 | "source": [ 1335 | "struct RootProblem2{F, X<:Real, T<:RootAlgorithm}\n", 1336 | " f::F\n", 1337 | " x0::X\n", 1338 | " algorithm::T\n", 1339 | "end" 1340 | ] 1341 | }, 1342 | { 1343 | "cell_type": "code", 1344 | "execution_count": 84, 1345 | "metadata": {}, 1346 | "outputs": [], 1347 | "source": [] 1348 | }, 1349 | { 1350 | "cell_type": "markdown", 1351 | "metadata": {}, 1352 | "source": [ 1353 | "#### Exercise 5\n", 1354 | "\n", 1355 | "1. Implement this.\n", 1356 | "\n", 1357 | "\n", 1358 | "2. Put everything together to be able to solve a root problem using a particular derivative algorithm." 1359 | ] 1360 | }, 1361 | { 1362 | "cell_type": "code", 1363 | "execution_count": null, 1364 | "metadata": {}, 1365 | "outputs": [], 1366 | "source": [] 1367 | }, 1368 | { 1369 | "cell_type": "markdown", 1370 | "metadata": {}, 1371 | "source": [ 1372 | "#### Exercise 6\n", 1373 | "\n", 1374 | "1. Implement a `MultipleRootProblem` type that specifies an interval over which we would like to find all roots.\n", 1375 | "\n", 1376 | "\n", 1377 | "2. Write a simple implementation of the algorithm using multiple starting points in the interval and making a list of unique roots found by that procedure.\n", 1378 | "\n", 1379 | "\n", 1380 | "3. Load the `Polynomials.jl` package and write a dispatch that specialises on polynomials and calls the root finder in that package." 1381 | ] 1382 | }, 1383 | { 1384 | "cell_type": "code", 1385 | "execution_count": 85, 1386 | "metadata": {}, 1387 | "outputs": [], 1388 | "source": [ 1389 | "using Polynomials" 1390 | ] 1391 | }, 1392 | { 1393 | "cell_type": "code", 1394 | "execution_count": 86, 1395 | "metadata": {}, 1396 | "outputs": [ 1397 | { 1398 | "data": { 1399 | "text/html": [ 1400 | "1 + 2∙x + 3∙x^2" 1401 | ], 1402 | "text/latex": [ 1403 | "$1 + 2\\cdot x + 3\\cdot x^{2}$" 1404 | ], 1405 | "text/plain": [ 1406 | "Poly(1 + 2*x + 3*x^2)" 1407 | ] 1408 | }, 1409 | "execution_count": 86, 1410 | "metadata": {}, 1411 | "output_type": "execute_result" 1412 | } 1413 | ], 1414 | "source": [ 1415 | "p = Poly([1, 2, 3])" 1416 | ] 1417 | }, 1418 | { 1419 | "cell_type": "code", 1420 | "execution_count": 87, 1421 | "metadata": {}, 1422 | "outputs": [ 1423 | { 1424 | "data": { 1425 | "text/plain": [ 1426 | "321" 1427 | ] 1428 | }, 1429 | "execution_count": 87, 1430 | "metadata": {}, 1431 | "output_type": "execute_result" 1432 | } 1433 | ], 1434 | "source": [ 1435 | "p(10)" 1436 | ] 1437 | }, 1438 | { 1439 | "cell_type": "code", 1440 | "execution_count": 89, 1441 | "metadata": {}, 1442 | "outputs": [ 1443 | { 1444 | "data": { 1445 | "text/plain": [ 1446 | "false" 1447 | ] 1448 | }, 1449 | "execution_count": 89, 1450 | "metadata": {}, 1451 | "output_type": "execute_result" 1452 | } 1453 | ], 1454 | "source": [ 1455 | "p isa Function" 1456 | ] 1457 | }, 1458 | { 1459 | "cell_type": "markdown", 1460 | "metadata": {}, 1461 | "source": [ 1462 | "## Other uses of types" 1463 | ] 1464 | }, 1465 | { 1466 | "cell_type": "markdown", 1467 | "metadata": {}, 1468 | "source": [ 1469 | "Other examples of different usages of types include:\n", 1470 | "\n", 1471 | "- [`ModelingToolkit.jl`](https://github.com/JuliaDiffEq/ModelingToolkit.jl)\n", 1472 | "\n", 1473 | " Types are introduced to represent variables and operations. In this way it is relatively simple to build up a way to output symbolic expressions from standard Julia functions.\n", 1474 | " \n", 1475 | " \n", 1476 | "- https://github.com/MikeInnes/diff-zoo defines types to represent \"tapes\" recording sequences of operations. This is a precursor to tools such as [`Zygote.jl`](https://github.com/FluxML/Zygote.jl), which performs advanced automatic differentiation on code at a lower level." 1477 | ] 1478 | }, 1479 | { 1480 | "cell_type": "markdown", 1481 | "metadata": {}, 1482 | "source": [ 1483 | "### Traits" 1484 | ] 1485 | }, 1486 | { 1487 | "cell_type": "markdown", 1488 | "metadata": {}, 1489 | "source": [ 1490 | "An important use of types that we have not addressed here is to define **traits**. These are labels that can be assigned to different types that may then be dispatched on, even if those types are not in a Julia type hierarchy.\n", 1491 | "\n", 1492 | "See e.g. the implementation in [`SimpleTraits.jl`](https://github.com/mauro3/SimpleTraits.jl)." 1493 | ] 1494 | } 1495 | ], 1496 | "metadata": { 1497 | "kernelspec": { 1498 | "display_name": "Julia 1.1.0", 1499 | "language": "julia", 1500 | "name": "julia-1.1" 1501 | }, 1502 | "language_info": { 1503 | "file_extension": ".jl", 1504 | "mimetype": "application/julia", 1505 | "name": "julia", 1506 | "version": "1.1.0" 1507 | } 1508 | }, 1509 | "nbformat": 4, 1510 | "nbformat_minor": 2 1511 | } 1512 | -------------------------------------------------------------------------------- /solutions/1. Root finding.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Intermediate Julia for scientific computing" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "This workshop is designed to introduce two fundamental concepts in Julia: **types** and **metaprogramming**.\n", 15 | "\n", 16 | "In order to cover various key uses of types in Julia, we have chosen to frame the discussion around a concrete topic in scientific computing, namely **root-finding**. \n", 17 | "The goal is *not* to learn algorithms for root finding *per se*, but rather to have a (pseudo-)real context in which to explore various concepts centered around types and how they arise naturally in real applications of Julia, in particular applications of **multiple dispatch**, which is one of the core choices in Julia that differentiate it from other common languages.\n", 18 | "\n", 19 | "We will implement a couple of root-finding algorithms just to have something to work with. These will just be toy implementations that are far away from the best implementations. \n", 20 | "\n", 21 | "Instead we should use one of the high-quality packages that are available in Julia for this purpose; the large number of them shows the importance of root finding. I am aware of the following (in alphabetical order):" 22 | ] 23 | }, 24 | { 25 | "cell_type": "markdown", 26 | "metadata": {}, 27 | "source": [ 28 | "- Single root of a nonlinear function:\n", 29 | " - [`NLsolve.jl`](https://github.com/JuliaNLSolvers/NLsolve.jl)\n", 30 | " - [`Roots.jl`](https://github.com/JuliaMath/Roots.jl)\n", 31 | "\n", 32 | "- All roots of a polynomial:\n", 33 | " - [`HomotopyContinuation.jl`](https://www.juliahomotopycontinuation.org)\n", 34 | " - [`PolynomialRoots.jl`](https://github.com/giordano/PolynomialRoots.jl)\n", 35 | " - [`Polynomials.jl`](https://github.com/JuliaMath/Polynomials.jl)\n", 36 | " \n", 37 | "- All roots of a nonlinear function:\n", 38 | " - [`ApproxFun.jl`](https://github.com/JuliaApproximation/ApproxFun.jl)\n", 39 | " - [`IntervalRootFinding.jl`](https://github.com/JuliaIntervals/IntervalRootFinding.jl)\n", 40 | " - [`MDBM.jl`](https://github.com/bachrathyd/MDBM.jl)\n", 41 | " - [`Roots.jl`](https://github.com/JuliaMath/Roots.jl)" 42 | ] 43 | }, 44 | { 45 | "cell_type": "markdown", 46 | "metadata": {}, 47 | "source": [ 48 | "Each of these uses different techniques, with different advantages and disadvantages." 49 | ] 50 | }, 51 | { 52 | "cell_type": "markdown", 53 | "metadata": {}, 54 | "source": [ 55 | "The challenge exercise for the workshop is: develop a package which integrates all of these disparate packages into a coherent whole!" 56 | ] 57 | }, 58 | { 59 | "cell_type": "markdown", 60 | "metadata": {}, 61 | "source": [ 62 | "### Logistics of the workshop" 63 | ] 64 | }, 65 | { 66 | "cell_type": "markdown", 67 | "metadata": {}, 68 | "source": [ 69 | "The workshop is based around a series of exercises to be done during the workshop. We will pause to work on the exercises and then I will discuss possible solutions during the workshop." 70 | ] 71 | }, 72 | { 73 | "cell_type": "markdown", 74 | "metadata": {}, 75 | "source": [ 76 | "These techniques are useful for both users and developers; indeed, in Julia the distinction between users and developers is not useful, since it's much easier than in other languages to join the two categories together." 77 | ] 78 | }, 79 | { 80 | "cell_type": "markdown", 81 | "metadata": {}, 82 | "source": [ 83 | "### Outline" 84 | ] 85 | }, 86 | { 87 | "cell_type": "markdown", 88 | "metadata": {}, 89 | "source": [ 90 | "We will start by quickly reviewing roots of functions and quickly reviewing one of the standard algorithms, **Newton's algorithm**. We will restrict to finding roots of 1D functions for simplicity.\n", 91 | "\n", 92 | "Newton's algorithm requires the calculation of derivatives, for which several choices of algorithm are available. We will see how to encode the choice of algorithm using dispatch.\n", 93 | "\n", 94 | "Then we will define types which will contain all information about a root-finding problem." 95 | ] 96 | }, 97 | { 98 | "cell_type": "markdown", 99 | "metadata": {}, 100 | "source": [ 101 | "## Roots" 102 | ] 103 | }, 104 | { 105 | "cell_type": "markdown", 106 | "metadata": {}, 107 | "source": [ 108 | "Given a function $f: \\mathbb{R} \\to \\mathbb{R}$ (i.e. that accepts a single real number as argument and returns another real number), recall that a **root** or **zero** of the function is a number $x^*$ such that\n", 109 | "\n", 110 | "$$ f(x^*) = 0, $$\n", 111 | "\n", 112 | "i.e. it is a solution of the equation $f(x) = 0$.\n", 113 | "\n", 114 | "In general it is impossible to solve this equation exactly for $x^*$, so we use iterative numerical algorithms instead." 115 | ] 116 | }, 117 | { 118 | "cell_type": "markdown", 119 | "metadata": {}, 120 | "source": [ 121 | "#### Example" 122 | ] 123 | }, 124 | { 125 | "cell_type": "markdown", 126 | "metadata": {}, 127 | "source": [ 128 | "Recall that the function $f$ given by $f(x) := x^2 - 2$ has exactly two roots, at $x^*_1 = +\\sqrt{2}$ and $x^*_2 = -\\sqrt{2}$. Note that it is impossible to represent these values exactly using floating-point arithmetic." 129 | ] 130 | }, 131 | { 132 | "cell_type": "markdown", 133 | "metadata": {}, 134 | "source": [ 135 | "## Newton algorithm" 136 | ] 137 | }, 138 | { 139 | "cell_type": "markdown", 140 | "metadata": {}, 141 | "source": [ 142 | "The Newton algorithm for (possibly) finding a root of a nonlinear function $f(x)$ in 1D is the following iteration:" 143 | ] 144 | }, 145 | { 146 | "cell_type": "markdown", 147 | "metadata": {}, 148 | "source": [ 149 | "$$x_{n+1} = x_n - \\frac{f(x_n)}{f'(x_n)},$$\n", 150 | "\n", 151 | "where $f'$ is the derivative of $f$. We start from an initial guess $x_0$ that can be almost anything (except points for which $f'(x_0) = 0$)." 152 | ] 153 | }, 154 | { 155 | "cell_type": "markdown", 156 | "metadata": {}, 157 | "source": [ 158 | "#### Exercise 1" 159 | ] 160 | }, 161 | { 162 | "cell_type": "markdown", 163 | "metadata": {}, 164 | "source": [ 165 | "1. Implement the Newton algorithm for a fixed number $n$ of steps in a function `newton`, starting from a given starting point $x_0$. \n", 166 | "\n", 167 | " Hint: Which information does the function require?\n", 168 | "\n", 169 | "\n", 170 | "2. Does your function work with other number types, such as `BigFloat`? What do you need in order to run it with those types? Use it to calculate $\\sqrt{2}$. How many decimal places are correct with the standard precision of `BigFloat`?" 171 | ] 172 | }, 173 | { 174 | "cell_type": "markdown", 175 | "metadata": {}, 176 | "source": [ 177 | "#### Solution 1" 178 | ] 179 | }, 180 | { 181 | "cell_type": "markdown", 182 | "metadata": {}, 183 | "source": [ 184 | "1. The `newton` function needs the function `f` whose root we wish to find, its derivative `df`, an initial condition `x0`, and a number of iterations `n`:" 185 | ] 186 | }, 187 | { 188 | "cell_type": "code", 189 | "execution_count": 1, 190 | "metadata": {}, 191 | "outputs": [ 192 | { 193 | "data": { 194 | "text/plain": [ 195 | "newton (generic function with 2 methods)" 196 | ] 197 | }, 198 | "execution_count": 1, 199 | "metadata": {}, 200 | "output_type": "execute_result" 201 | } 202 | ], 203 | "source": [ 204 | "function newton(f, df, x0, n=10) # n=10 specifies a default value\n", 205 | " x = x0 # initialise\n", 206 | " \n", 207 | " for i in 1:n\n", 208 | " x_new = x - f(x) / df(x)\n", 209 | " \n", 210 | " x = x_new # update for next step\n", 211 | " end\n", 212 | " \n", 213 | " return x\n", 214 | "end" 215 | ] 216 | }, 217 | { 218 | "cell_type": "markdown", 219 | "metadata": {}, 220 | "source": [ 221 | "Define a function `g` whose root we wish to find, together with its derivative `dg` specified \"by hand\":" 222 | ] 223 | }, 224 | { 225 | "cell_type": "code", 226 | "execution_count": 2, 227 | "metadata": {}, 228 | "outputs": [ 229 | { 230 | "data": { 231 | "text/plain": [ 232 | "dg (generic function with 1 method)" 233 | ] 234 | }, 235 | "execution_count": 2, 236 | "metadata": {}, 237 | "output_type": "execute_result" 238 | } 239 | ], 240 | "source": [ 241 | "g(x) = x^2 - 2\n", 242 | "dg(x) = 2x" 243 | ] 244 | }, 245 | { 246 | "cell_type": "markdown", 247 | "metadata": {}, 248 | "source": [ 249 | "Run `newton` with these functions and a starting point `y0`:" 250 | ] 251 | }, 252 | { 253 | "cell_type": "code", 254 | "execution_count": 3, 255 | "metadata": {}, 256 | "outputs": [ 257 | { 258 | "data": { 259 | "text/plain": [ 260 | "1.414213562373095" 261 | ] 262 | }, 263 | "execution_count": 3, 264 | "metadata": {}, 265 | "output_type": "execute_result" 266 | } 267 | ], 268 | "source": [ 269 | "y0 = 3.0\n", 270 | "\n", 271 | "newton(g, dg, y0) # we did not need to specify a number of iterations; the default value is used" 272 | ] 273 | }, 274 | { 275 | "cell_type": "markdown", 276 | "metadata": {}, 277 | "source": [ 278 | "Note that it is very easy to pass functions as arguments to other functions, by specifying the name of the function. This is also efficient." 279 | ] 280 | }, 281 | { 282 | "cell_type": "code", 283 | "execution_count": 4, 284 | "metadata": {}, 285 | "outputs": [ 286 | { 287 | "name": "stdout", 288 | "output_type": "stream", 289 | "text": [ 290 | "Body\u001b[36m::Float64\u001b[39m\n", 291 | "\u001b[90m1 ──\u001b[39m %1 = (Base.sle_int)(1, n)\u001b[36m::Bool\u001b[39m\n", 292 | "\u001b[90m│ \u001b[39m %2 = (Base.ifelse)(%1, n, 0)\u001b[36m::Int64\u001b[39m\n", 293 | "\u001b[90m│ \u001b[39m %3 = (Base.slt_int)(%2, 1)\u001b[36m::Bool\u001b[39m\n", 294 | "\u001b[90m└───\u001b[39m goto #3 if not %3\n", 295 | "\u001b[90m2 ──\u001b[39m goto #4\n", 296 | "\u001b[90m3 ──\u001b[39m goto #4\n", 297 | "\u001b[90m4 ┄─\u001b[39m %7 = φ (#2 => true, #3 => false)\u001b[36m::Bool\u001b[39m\n", 298 | "\u001b[90m│ \u001b[39m %8 = φ (#3 => 1)\u001b[36m::Int64\u001b[39m\n", 299 | "\u001b[90m│ \u001b[39m %9 = (Base.not_int)(%7)\u001b[36m::Bool\u001b[39m\n", 300 | "\u001b[90m└───\u001b[39m goto #10 if not %9\n", 301 | "\u001b[90m5 ┄─\u001b[39m %11 = φ (#4 => _4, #9 => %17)\u001b[36m::Float64\u001b[39m\n", 302 | "\u001b[90m│ \u001b[39m %12 = φ (#4 => %8, #9 => %23)\u001b[36m::Int64\u001b[39m\n", 303 | "\u001b[90m│ \u001b[39m %13 = (Base.mul_float)(%11, %11)\u001b[36m::Float64\u001b[39m\n", 304 | "\u001b[90m│ \u001b[39m %14 = (Base.sub_float)(%13, 2.0)\u001b[36m::Float64\u001b[39m\n", 305 | "\u001b[90m│ \u001b[39m %15 = (Base.mul_float)(2.0, %11)\u001b[36m::Float64\u001b[39m\n", 306 | "\u001b[90m│ \u001b[39m %16 = (Base.div_float)(%14, %15)\u001b[36m::Float64\u001b[39m\n", 307 | "\u001b[90m│ \u001b[39m %17 = (Base.sub_float)(%11, %16)\u001b[36m::Float64\u001b[39m\n", 308 | "\u001b[90m│ \u001b[39m %18 = (%12 === %2)\u001b[36m::Bool\u001b[39m\n", 309 | "\u001b[90m└───\u001b[39m goto #7 if not %18\n", 310 | "\u001b[90m6 ──\u001b[39m goto #8\n", 311 | "\u001b[90m7 ──\u001b[39m %21 = (Base.add_int)(%12, 1)\u001b[36m::Int64\u001b[39m\n", 312 | "\u001b[90m└───\u001b[39m goto #8\n", 313 | "\u001b[90m8 ┄─\u001b[39m %23 = φ (#7 => %21)\u001b[36m::Int64\u001b[39m\n", 314 | "\u001b[90m│ \u001b[39m %24 = φ (#6 => true, #7 => false)\u001b[36m::Bool\u001b[39m\n", 315 | "\u001b[90m│ \u001b[39m %25 = (Base.not_int)(%24)\u001b[36m::Bool\u001b[39m\n", 316 | "\u001b[90m└───\u001b[39m goto #10 if not %25\n", 317 | "\u001b[90m9 ──\u001b[39m goto #5\n", 318 | "\u001b[90m10 ┄\u001b[39m %28 = φ (#8 => %17, #4 => _4)\u001b[36m::Float64\u001b[39m\n", 319 | "\u001b[90m└───\u001b[39m return %28\n" 320 | ] 321 | } 322 | ], 323 | "source": [ 324 | "@code_warntype newton(g, dg, 0.1, 10)" 325 | ] 326 | }, 327 | { 328 | "cell_type": "markdown", 329 | "metadata": {}, 330 | "source": [ 331 | "Note that the functions `f` and `df` were *inlined*: there is no longer a call to the functions, but rather the code for those functions is inserted into the code for the Newton method. Whether this inlining is performed or not is governed by a heuristic based on the code complexity." 332 | ] 333 | }, 334 | { 335 | "cell_type": "markdown", 336 | "metadata": {}, 337 | "source": [ 338 | "Compare with the analytical solution:" 339 | ] 340 | }, 341 | { 342 | "cell_type": "code", 343 | "execution_count": 5, 344 | "metadata": {}, 345 | "outputs": [ 346 | { 347 | "data": { 348 | "text/plain": [ 349 | "1.4142135623730951" 350 | ] 351 | }, 352 | "execution_count": 5, 353 | "metadata": {}, 354 | "output_type": "execute_result" 355 | } 356 | ], 357 | "source": [ 358 | "sqrt(2)" 359 | ] 360 | }, 361 | { 362 | "cell_type": "markdown", 363 | "metadata": {}, 364 | "source": [ 365 | "2. Run the function starting from a `BigFloat` initial condition:" 366 | ] 367 | }, 368 | { 369 | "cell_type": "code", 370 | "execution_count": 6, 371 | "metadata": {}, 372 | "outputs": [ 373 | { 374 | "data": { 375 | "text/plain": [ 376 | "2.0" 377 | ] 378 | }, 379 | "execution_count": 6, 380 | "metadata": {}, 381 | "output_type": "execute_result" 382 | } 383 | ], 384 | "source": [ 385 | "y0 = big(2.0)" 386 | ] 387 | }, 388 | { 389 | "cell_type": "code", 390 | "execution_count": 7, 391 | "metadata": {}, 392 | "outputs": [ 393 | { 394 | "data": { 395 | "text/plain": [ 396 | "1.414213562373095048801689623502530243614981925776197428498289498623195824228933" 397 | ] 398 | }, 399 | "execution_count": 7, 400 | "metadata": {}, 401 | "output_type": "execute_result" 402 | } 403 | ], 404 | "source": [ 405 | "newton(g, dg, y0, 5)" 406 | ] 407 | }, 408 | { 409 | "cell_type": "code", 410 | "execution_count": 8, 411 | "metadata": {}, 412 | "outputs": [ 413 | { 414 | "data": { 415 | "text/plain": [ 416 | "1.414213562373095048801689623502530243614981925776197428498289498623195824228933" 417 | ] 418 | }, 419 | "execution_count": 8, 420 | "metadata": {}, 421 | "output_type": "execute_result" 422 | } 423 | ], 424 | "source": [ 425 | "result = newton(g, dg, y0, 5)" 426 | ] 427 | }, 428 | { 429 | "cell_type": "markdown", 430 | "metadata": {}, 431 | "source": [ 432 | "How far is it from the true square root?" 433 | ] 434 | }, 435 | { 436 | "cell_type": "code", 437 | "execution_count": 9, 438 | "metadata": {}, 439 | "outputs": [ 440 | { 441 | "data": { 442 | "text/plain": [ 443 | "8.992928321650453100503992493553216097606324633457668310222718673825519970228542e-25" 444 | ] 445 | }, 446 | "execution_count": 9, 447 | "metadata": {}, 448 | "output_type": "execute_result" 449 | } 450 | ], 451 | "source": [ 452 | "result - sqrt(y0)" 453 | ] 454 | }, 455 | { 456 | "cell_type": "code", 457 | "execution_count": 10, 458 | "metadata": {}, 459 | "outputs": [ 460 | { 461 | "name": "stdout", 462 | "output_type": "stream", 463 | "text": [ 464 | "0.08578643762690495119831127579030192143032812462305192682332026200926752153789802\n", 465 | "0.00245310429357161786497794245696858809699479128971859348998692867593418820455893\n", 466 | "2.123901414755119879903241282313587190869721091142509594771813189090165348662204e-06\n", 467 | "1.594861824606854680436831546887746738795971408225281209321893373282554461870123e-12\n", 468 | "8.992928321650453100503992493553216097606324633457668310222718673825519970228542e-25\n", 469 | "2.859283843333951225327771682478558595433666338812917695204101993287842530821706e-49\n", 470 | "0.0\n", 471 | "0.0\n", 472 | "0.0\n", 473 | "0.0\n" 474 | ] 475 | } 476 | ], 477 | "source": [ 478 | "for n in 1:10\n", 479 | " result = newton(g, dg, y0, n)\n", 480 | " println(abs(result - sqrt(y0)))\n", 481 | "end" 482 | ] 483 | }, 484 | { 485 | "cell_type": "markdown", 486 | "metadata": {}, 487 | "source": [ 488 | "## Calculating derivatives" 489 | ] 490 | }, 491 | { 492 | "cell_type": "markdown", 493 | "metadata": {}, 494 | "source": [ 495 | "The Newton algorithm requires us to specify the derivative of a function. If $f$ is a complicated function, we certainly don't want to do that by hand.\n", 496 | "\n", 497 | "One standard solution is to use a *finite-difference approximation*:\n", 498 | "\n", 499 | "$$f'(a) \\simeq \\frac{f(a + h) - f(a - h)}{2h}.$$" 500 | ] 501 | }, 502 | { 503 | "cell_type": "markdown", 504 | "metadata": {}, 505 | "source": [ 506 | "#### Exercise 2" 507 | ] 508 | }, 509 | { 510 | "cell_type": "code", 511 | "execution_count": 11, 512 | "metadata": {}, 513 | "outputs": [ 514 | { 515 | "ename": "LoadError", 516 | "evalue": "syntax: extra token \"found\" after end of expression", 517 | "output_type": "error", 518 | "traceback": [ 519 | "syntax: extra token \"found\" after end of expression", 520 | "" 521 | ] 522 | } 523 | ], 524 | "source": [ 525 | "We found the exact" 526 | ] 527 | }, 528 | { 529 | "cell_type": "markdown", 530 | "metadata": {}, 531 | "source": [ 532 | "1. Implement a function `finite_difference` with a default value $h = 0.001$.\n", 533 | "\n", 534 | "\n", 535 | "2. Use an anonymous function to make a method of `finite_difference` that calculates the *function* $f'$.\n", 536 | "\n", 537 | "\n", 538 | "3. Implement a version of `newton` that does not take the derivative as argument and uses `finite_difference` to calculate the derivative. This version of `newton` should **re-use** the previous version by defining the function `fp` and calling that version." 539 | ] 540 | }, 541 | { 542 | "cell_type": "markdown", 543 | "metadata": {}, 544 | "source": [ 545 | "### Solution 2" 546 | ] 547 | }, 548 | { 549 | "cell_type": "markdown", 550 | "metadata": {}, 551 | "source": [ 552 | "1. We copy the mathematical definition. We need the function `f`, the value `a` and the step size `h`, which has a default value:" 553 | ] 554 | }, 555 | { 556 | "cell_type": "code", 557 | "execution_count": 12, 558 | "metadata": {}, 559 | "outputs": [ 560 | { 561 | "data": { 562 | "text/plain": [ 563 | "finite_difference (generic function with 2 methods)" 564 | ] 565 | }, 566 | "execution_count": 12, 567 | "metadata": {}, 568 | "output_type": "execute_result" 569 | } 570 | ], 571 | "source": [ 572 | "function finite_difference(f, a, h=0.001)\n", 573 | " return ( f(a + h) - f(a - h) ) / (2h)\n", 574 | "end" 575 | ] 576 | }, 577 | { 578 | "cell_type": "markdown", 579 | "metadata": {}, 580 | "source": [ 581 | "We specify the function and the exact derivatie:" 582 | ] 583 | }, 584 | { 585 | "cell_type": "code", 586 | "execution_count": 13, 587 | "metadata": {}, 588 | "outputs": [ 589 | { 590 | "data": { 591 | "text/plain": [ 592 | "dg (generic function with 1 method)" 593 | ] 594 | }, 595 | "execution_count": 13, 596 | "metadata": {}, 597 | "output_type": "execute_result" 598 | } 599 | ], 600 | "source": [ 601 | "g(x) = x^3 - 2\n", 602 | "dg(x) = 3x^2" 603 | ] 604 | }, 605 | { 606 | "cell_type": "code", 607 | "execution_count": 14, 608 | "metadata": {}, 609 | "outputs": [ 610 | { 611 | "data": { 612 | "text/plain": [ 613 | "27.000000999995777" 614 | ] 615 | }, 616 | "execution_count": 14, 617 | "metadata": {}, 618 | "output_type": "execute_result" 619 | } 620 | ], 621 | "source": [ 622 | "a = 3.0\n", 623 | "\n", 624 | "finite_difference(g, a)" 625 | ] 626 | }, 627 | { 628 | "cell_type": "code", 629 | "execution_count": 15, 630 | "metadata": {}, 631 | "outputs": [ 632 | { 633 | "data": { 634 | "text/plain": [ 635 | "27.0" 636 | ] 637 | }, 638 | "execution_count": 15, 639 | "metadata": {}, 640 | "output_type": "execute_result" 641 | } 642 | ], 643 | "source": [ 644 | "dg(a)" 645 | ] 646 | }, 647 | { 648 | "cell_type": "markdown", 649 | "metadata": {}, 650 | "source": [ 651 | "How good is the approximation?" 652 | ] 653 | }, 654 | { 655 | "cell_type": "code", 656 | "execution_count": 16, 657 | "metadata": {}, 658 | "outputs": [ 659 | { 660 | "data": { 661 | "text/plain": [ 662 | "-9.999957768513923e-7" 663 | ] 664 | }, 665 | "execution_count": 16, 666 | "metadata": {}, 667 | "output_type": "execute_result" 668 | } 669 | ], 670 | "source": [ 671 | "dg(a) - finite_difference(g, a)" 672 | ] 673 | }, 674 | { 675 | "cell_type": "code", 676 | "execution_count": 17, 677 | "metadata": {}, 678 | "outputs": [ 679 | { 680 | "data": { 681 | "text/plain": [ 682 | "-0.01000000000002288" 683 | ] 684 | }, 685 | "execution_count": 17, 686 | "metadata": {}, 687 | "output_type": "execute_result" 688 | } 689 | ], 690 | "source": [ 691 | "dg(a) - finite_difference(g, a, 0.1)" 692 | ] 693 | }, 694 | { 695 | "cell_type": "markdown", 696 | "metadata": {}, 697 | "source": [ 698 | "2. Now we want to make a new method of `finite_difference` that takes just the function `f` and returns its derivative. Note that its derivative is *itself a function*, so this will be a function that accepts a function and returns another function:" 699 | ] 700 | }, 701 | { 702 | "cell_type": "code", 703 | "execution_count": 18, 704 | "metadata": {}, 705 | "outputs": [ 706 | { 707 | "data": { 708 | "text/plain": [ 709 | "finite_difference (generic function with 3 methods)" 710 | ] 711 | }, 712 | "execution_count": 18, 713 | "metadata": {}, 714 | "output_type": "execute_result" 715 | } 716 | ], 717 | "source": [ 718 | "finite_difference(f) = x -> finite_difference(f, x)" 719 | ] 720 | }, 721 | { 722 | "cell_type": "markdown", 723 | "metadata": {}, 724 | "source": [ 725 | "Mathematically we can write this as $x \\mapsto f\\prime(x)$, which we read as \"the function that maps any value $x$ to the result of $f'(x)$\"." 726 | ] 727 | }, 728 | { 729 | "cell_type": "markdown", 730 | "metadata": {}, 731 | "source": [ 732 | "Let's try it out:" 733 | ] 734 | }, 735 | { 736 | "cell_type": "code", 737 | "execution_count": 19, 738 | "metadata": {}, 739 | "outputs": [ 740 | { 741 | "data": { 742 | "text/plain": [ 743 | "5.999999999999339" 744 | ] 745 | }, 746 | "execution_count": 19, 747 | "metadata": {}, 748 | "output_type": "execute_result" 749 | } 750 | ], 751 | "source": [ 752 | "dg2 = finite_difference(x->x^2)\n", 753 | "dg2(3.0)" 754 | ] 755 | }, 756 | { 757 | "cell_type": "markdown", 758 | "metadata": {}, 759 | "source": [ 760 | "2. We want to write a \"version\" (*method*) of the function `newton`, in which we no longer need to specify the derivative by hand, but rather use a finite difference.\n", 761 | "\n", 762 | "Since we already have a version of the `newton` function written which contains the main code for the method, we should try to *re-use* it. (This is the \"Don't Repeat Yourself\", or DRY, principle.)" 763 | ] 764 | }, 765 | { 766 | "cell_type": "code", 767 | "execution_count": 20, 768 | "metadata": {}, 769 | "outputs": [ 770 | { 771 | "data": { 772 | "text/plain": [ 773 | "newton (generic function with 3 methods)" 774 | ] 775 | }, 776 | "execution_count": 20, 777 | "metadata": {}, 778 | "output_type": "execute_result" 779 | } 780 | ], 781 | "source": [ 782 | "function newton(f, x0, n=10)\n", 783 | "\n", 784 | " # calculate the derivative *function*\n", 785 | " df = finite_difference(f)\n", 786 | " \n", 787 | " # pass it to the previously-defined method of `newton`:\n", 788 | " result = newton(f, df, x0, n) \n", 789 | " \n", 790 | " return result\n", 791 | "end" 792 | ] 793 | }, 794 | { 795 | "cell_type": "code", 796 | "execution_count": 21, 797 | "metadata": {}, 798 | "outputs": [ 799 | { 800 | "data": { 801 | "text/plain": [ 802 | "1.2599210498948732" 803 | ] 804 | }, 805 | "execution_count": 21, 806 | "metadata": {}, 807 | "output_type": "execute_result" 808 | } 809 | ], 810 | "source": [ 811 | "newton(g, 3.0)" 812 | ] 813 | }, 814 | { 815 | "cell_type": "code", 816 | "execution_count": 22, 817 | "metadata": {}, 818 | "outputs": [ 819 | { 820 | "data": { 821 | "text/plain": [ 822 | "1.2599210498948732" 823 | ] 824 | }, 825 | "execution_count": 22, 826 | "metadata": {}, 827 | "output_type": "execute_result" 828 | } 829 | ], 830 | "source": [ 831 | "cbrt(2)" 832 | ] 833 | }, 834 | { 835 | "cell_type": "markdown", 836 | "metadata": {}, 837 | "source": [ 838 | "`newton` is a **generic function**:" 839 | ] 840 | }, 841 | { 842 | "cell_type": "code", 843 | "execution_count": 23, 844 | "metadata": {}, 845 | "outputs": [ 846 | { 847 | "data": { 848 | "text/html": [ 849 | "newton(f, x0) in Main at In[20]:4" 850 | ], 851 | "text/plain": [ 852 | "newton(f, x0) in Main at In[20]:4" 853 | ] 854 | }, 855 | "execution_count": 23, 856 | "metadata": {}, 857 | "output_type": "execute_result" 858 | } 859 | ], 860 | "source": [ 861 | "@which newton(g, 3.0)" 862 | ] 863 | }, 864 | { 865 | "cell_type": "code", 866 | "execution_count": 24, 867 | "metadata": {}, 868 | "outputs": [ 869 | { 870 | "data": { 871 | "text/html": [ 872 | "3 methods for generic function newton:" 873 | ], 874 | "text/plain": [ 875 | "# 3 methods for generic function \"newton\":\n", 876 | "[1] newton(f, x0) in Main at In[20]:4\n", 877 | "[2] newton(f, x0, n) in Main at In[20]:4\n", 878 | "[3] newton(f, df, x0, n) in Main at In[1]:2" 879 | ] 880 | }, 881 | "execution_count": 24, 882 | "metadata": {}, 883 | "output_type": "execute_result" 884 | } 885 | ], 886 | "source": [ 887 | "methods(newton)" 888 | ] 889 | }, 890 | { 891 | "cell_type": "markdown", 892 | "metadata": {}, 893 | "source": [ 894 | "Note that default arguments have the effect of creating additional methods." 895 | ] 896 | }, 897 | { 898 | "cell_type": "markdown", 899 | "metadata": {}, 900 | "source": [ 901 | "Things already seem to be getting a bit complicated, since we are building up layers of functions on top of each other. Is this a good idea in terms of performance?" 902 | ] 903 | }, 904 | { 905 | "cell_type": "code", 906 | "execution_count": 25, 907 | "metadata": {}, 908 | "outputs": [ 909 | { 910 | "name": "stdout", 911 | "output_type": "stream", 912 | "text": [ 913 | "Body\u001b[36m::Float64\u001b[39m\n", 914 | "\u001b[90m1 ──\u001b[39m %1 = (Base.sle_int)(1, n)\u001b[36m::Bool\u001b[39m\n", 915 | "\u001b[90m│ \u001b[39m %2 = (Base.ifelse)(%1, n, 0)\u001b[36m::Int64\u001b[39m\n", 916 | "\u001b[90m│ \u001b[39m %3 = (Base.slt_int)(%2, 1)\u001b[36m::Bool\u001b[39m\n", 917 | "\u001b[90m└───\u001b[39m goto #3 if not %3\n", 918 | "\u001b[90m2 ──\u001b[39m goto #4\n", 919 | "\u001b[90m3 ──\u001b[39m goto #4\n", 920 | "\u001b[90m4 ┄─\u001b[39m %7 = φ (#2 => true, #3 => false)\u001b[36m::Bool\u001b[39m\n", 921 | "\u001b[90m│ \u001b[39m %8 = φ (#3 => 1)\u001b[36m::Int64\u001b[39m\n", 922 | "\u001b[90m│ \u001b[39m %9 = (Base.not_int)(%7)\u001b[36m::Bool\u001b[39m\n", 923 | "\u001b[90m└───\u001b[39m goto #10 if not %9\n", 924 | "\u001b[90m5 ┄─\u001b[39m %11 = φ (#4 => _4, #9 => %27)\u001b[36m::Float64\u001b[39m\n", 925 | "\u001b[90m│ \u001b[39m %12 = φ (#4 => %8, #9 => %33)\u001b[36m::Int64\u001b[39m\n", 926 | "\u001b[90m│ \u001b[39m %13 = (Base.mul_float)(%11, %11)\u001b[36m::Float64\u001b[39m\n", 927 | "\u001b[90m│ \u001b[39m %14 = (Base.mul_float)(%13, %11)\u001b[36m::Float64\u001b[39m\n", 928 | "\u001b[90m│ \u001b[39m %15 = (Base.sub_float)(%14, 2.0)\u001b[36m::Float64\u001b[39m\n", 929 | "\u001b[90m│ \u001b[39m %16 = (Base.add_float)(%11, 0.001)\u001b[36m::Float64\u001b[39m\n", 930 | "\u001b[90m│ \u001b[39m %17 = (Base.mul_float)(%16, %16)\u001b[36m::Float64\u001b[39m\n", 931 | "\u001b[90m│ \u001b[39m %18 = (Base.mul_float)(%17, %16)\u001b[36m::Float64\u001b[39m\n", 932 | "\u001b[90m│ \u001b[39m %19 = (Base.sub_float)(%18, 2.0)\u001b[36m::Float64\u001b[39m\n", 933 | "\u001b[90m│ \u001b[39m %20 = (Base.sub_float)(%11, 0.001)\u001b[36m::Float64\u001b[39m\n", 934 | "\u001b[90m│ \u001b[39m %21 = (Base.mul_float)(%20, %20)\u001b[36m::Float64\u001b[39m\n", 935 | "\u001b[90m│ \u001b[39m %22 = (Base.mul_float)(%21, %20)\u001b[36m::Float64\u001b[39m\n", 936 | "\u001b[90m│ \u001b[39m %23 = (Base.sub_float)(%22, 2.0)\u001b[36m::Float64\u001b[39m\n", 937 | "\u001b[90m│ \u001b[39m %24 = (Base.sub_float)(%19, %23)\u001b[36m::Float64\u001b[39m\n", 938 | "\u001b[90m│ \u001b[39m %25 = (Base.div_float)(%24, 0.002)\u001b[36m::Float64\u001b[39m\n", 939 | "\u001b[90m│ \u001b[39m %26 = (Base.div_float)(%15, %25)\u001b[36m::Float64\u001b[39m\n", 940 | "\u001b[90m│ \u001b[39m %27 = (Base.sub_float)(%11, %26)\u001b[36m::Float64\u001b[39m\n", 941 | "\u001b[90m│ \u001b[39m %28 = (%12 === %2)\u001b[36m::Bool\u001b[39m\n", 942 | "\u001b[90m└───\u001b[39m goto #7 if not %28\n", 943 | "\u001b[90m6 ──\u001b[39m goto #8\n", 944 | "\u001b[90m7 ──\u001b[39m %31 = (Base.add_int)(%12, 1)\u001b[36m::Int64\u001b[39m\n", 945 | "\u001b[90m└───\u001b[39m goto #8\n", 946 | "\u001b[90m8 ┄─\u001b[39m %33 = φ (#7 => %31)\u001b[36m::Int64\u001b[39m\n", 947 | "\u001b[90m│ \u001b[39m %34 = φ (#6 => true, #7 => false)\u001b[36m::Bool\u001b[39m\n", 948 | "\u001b[90m│ \u001b[39m %35 = (Base.not_int)(%34)\u001b[36m::Bool\u001b[39m\n", 949 | "\u001b[90m└───\u001b[39m goto #10 if not %35\n", 950 | "\u001b[90m9 ──\u001b[39m goto #5\n", 951 | "\u001b[90m10 ┄\u001b[39m %38 = φ (#8 => %27, #4 => _4)\u001b[36m::Float64\u001b[39m\n", 952 | "\u001b[90m└───\u001b[39m return %38\n" 953 | ] 954 | } 955 | ], 956 | "source": [ 957 | "@code_warntype newton(g, finite_difference(g), 3.0, 10)" 958 | ] 959 | }, 960 | { 961 | "cell_type": "markdown", 962 | "metadata": {}, 963 | "source": [ 964 | "Again everything has got inlined and will be performant!\n", 965 | "\n", 966 | "However, note that a specialised version of `newton` will be compiled for each function that you pass in as we have written it." 967 | ] 968 | }, 969 | { 970 | "cell_type": "code", 971 | "execution_count": 26, 972 | "metadata": {}, 973 | "outputs": [ 974 | { 975 | "data": { 976 | "text/plain": [ 977 | "0.0" 978 | ] 979 | }, 980 | "execution_count": 26, 981 | "metadata": {}, 982 | "output_type": "execute_result" 983 | } 984 | ], 985 | "source": [ 986 | "newton(x->2x, 3.0)" 987 | ] 988 | }, 989 | { 990 | "cell_type": "markdown", 991 | "metadata": {}, 992 | "source": [] 993 | }, 994 | { 995 | "cell_type": "markdown", 996 | "metadata": {}, 997 | "source": [ 998 | "### Algorithmic differentiation" 999 | ] 1000 | }, 1001 | { 1002 | "cell_type": "markdown", 1003 | "metadata": {}, 1004 | "source": [ 1005 | "An alternative way to calculate derivatives is by using [**algorithmic differentiation**](https://en.wikipedia.org/wiki/Automatic_differentiation) (also called **automatic differentiation** or **computational differentiation**). This gives exact results (up to rounding error).\n", 1006 | "\n", 1007 | "\n", 1008 | "We will implement this algorithm in the next notebook, but for now let's just use the implementation in the excellent [`ForwardDiff.jl` package](https://github.com/JuliaDiff/ForwardDiff.jl). A Julia **package** contains Julia code, written as functions, that provides additional functionality that you may use in your code by calling those functions.\n" 1009 | ] 1010 | }, 1011 | { 1012 | "cell_type": "markdown", 1013 | "metadata": {}, 1014 | "source": [ 1015 | "#### Exercise 3" 1016 | ] 1017 | }, 1018 | { 1019 | "cell_type": "markdown", 1020 | "metadata": {}, 1021 | "source": [ 1022 | "1. Install `ForwardDiff.jl` if necessary.\n", 1023 | "\n", 1024 | "\n", 1025 | "2. Import it.\n", 1026 | "\n", 1027 | "\n", 1028 | "3. Define a function `forwarddiff` that uses the `ForwardDiff.derivative` function to calculate a derivative." 1029 | ] 1030 | }, 1031 | { 1032 | "cell_type": "markdown", 1033 | "metadata": {}, 1034 | "source": [ 1035 | "#### Solution 3" 1036 | ] 1037 | }, 1038 | { 1039 | "cell_type": "markdown", 1040 | "metadata": {}, 1041 | "source": [ 1042 | "We install the package using the excellent built-in package manager in Julia. If the package is a registered package then Julia automatically knows where to find it and downloads it into a standard location in your local system. Installing a package is only necessary once in a given installation of Julia." 1043 | ] 1044 | }, 1045 | { 1046 | "cell_type": "code", 1047 | "execution_count": 27, 1048 | "metadata": {}, 1049 | "outputs": [], 1050 | "source": [ 1051 | "# using Pkg\n", 1052 | "\n", 1053 | "# Pkg.add(\"ForwardDiff\")" 1054 | ] 1055 | }, 1056 | { 1057 | "cell_type": "markdown", 1058 | "metadata": {}, 1059 | "source": [ 1060 | "Alternative:" 1061 | ] 1062 | }, 1063 | { 1064 | "cell_type": "code", 1065 | "execution_count": 28, 1066 | "metadata": {}, 1067 | "outputs": [], 1068 | "source": [ 1069 | "# ]add ForwardDiff" 1070 | ] 1071 | }, 1072 | { 1073 | "cell_type": "markdown", 1074 | "metadata": {}, 1075 | "source": [ 1076 | "In order to use the package we must load it. There are two different ways to load a package: `using`, which makes available the functions that the package author has chosen to export by pulling them into your global namespace, and `import`, which does not do so.\n", 1077 | "\n", 1078 | "In the case of `ForwardDiff`, the functions provided in the package are in any case not exported:" 1079 | ] 1080 | }, 1081 | { 1082 | "cell_type": "code", 1083 | "execution_count": 29, 1084 | "metadata": {}, 1085 | "outputs": [], 1086 | "source": [ 1087 | "using ForwardDiff # load the package" 1088 | ] 1089 | }, 1090 | { 1091 | "cell_type": "markdown", 1092 | "metadata": {}, 1093 | "source": [ 1094 | "We may choose to define our own function to access the relevant function of the package for simplicity:" 1095 | ] 1096 | }, 1097 | { 1098 | "cell_type": "code", 1099 | "execution_count": 30, 1100 | "metadata": {}, 1101 | "outputs": [ 1102 | { 1103 | "data": { 1104 | "text/plain": [ 1105 | "forwarddiff (generic function with 1 method)" 1106 | ] 1107 | }, 1108 | "execution_count": 30, 1109 | "metadata": {}, 1110 | "output_type": "execute_result" 1111 | } 1112 | ], 1113 | "source": [ 1114 | "forwarddiff(f, x) = ForwardDiff.derivative(f, x)" 1115 | ] 1116 | }, 1117 | { 1118 | "cell_type": "markdown", 1119 | "metadata": {}, 1120 | "source": [ 1121 | "Note that the syntax `ForwardDiff.derivative` refers to the function called `derivative` that comes from the `ForwardDiff` package." 1122 | ] 1123 | }, 1124 | { 1125 | "cell_type": "code", 1126 | "execution_count": 31, 1127 | "metadata": {}, 1128 | "outputs": [ 1129 | { 1130 | "data": { 1131 | "text/plain": [ 1132 | "derivative (generic function with 4 methods)" 1133 | ] 1134 | }, 1135 | "execution_count": 31, 1136 | "metadata": {}, 1137 | "output_type": "execute_result" 1138 | } 1139 | ], 1140 | "source": [ 1141 | "ForwardDiff.derivative" 1142 | ] 1143 | }, 1144 | { 1145 | "cell_type": "code", 1146 | "execution_count": 32, 1147 | "metadata": {}, 1148 | "outputs": [ 1149 | { 1150 | "data": { 1151 | "text/plain": [ 1152 | "27.0" 1153 | ] 1154 | }, 1155 | "execution_count": 32, 1156 | "metadata": {}, 1157 | "output_type": "execute_result" 1158 | } 1159 | ], 1160 | "source": [ 1161 | "forwarddiff(g, a)" 1162 | ] 1163 | }, 1164 | { 1165 | "cell_type": "markdown", 1166 | "metadata": {}, 1167 | "source": [ 1168 | "### Choosing between algorithms" 1169 | ] 1170 | }, 1171 | { 1172 | "cell_type": "markdown", 1173 | "metadata": {}, 1174 | "source": [ 1175 | "We now have two different algorithms available to calculate derivatives. This kind of situation is common in scientific computing; for example, the [`DifferentialEquations.jl`](http://docs.juliadiffeq.org/latest/) ecosystem has some 300 algorithms for solving differential equations. One of the techniques we need to learn is how to specify which algorithm to use." 1176 | ] 1177 | }, 1178 | { 1179 | "cell_type": "markdown", 1180 | "metadata": {}, 1181 | "source": [ 1182 | "One possible solution is just by specifying the *function* to use as an argument to another function:" 1183 | ] 1184 | }, 1185 | { 1186 | "cell_type": "markdown", 1187 | "metadata": {}, 1188 | "source": [ 1189 | "#### Exercise 4" 1190 | ] 1191 | }, 1192 | { 1193 | "cell_type": "markdown", 1194 | "metadata": {}, 1195 | "source": [ 1196 | "1. Make a version of the Newton algorithm that takes an argument which is the method (algorithm) to use to calculate the derivative, given as a function. \n", 1197 | "The new method should have the signature `newton(f, df, x0, n, derivative)`." 1198 | ] 1199 | }, 1200 | { 1201 | "cell_type": "markdown", 1202 | "metadata": {}, 1203 | "source": [ 1204 | "#### Solution 4" 1205 | ] 1206 | }, 1207 | { 1208 | "cell_type": "code", 1209 | "execution_count": 33, 1210 | "metadata": {}, 1211 | "outputs": [ 1212 | { 1213 | "data": { 1214 | "text/html": [ 1215 | "3 methods for generic function newton:" 1216 | ], 1217 | "text/plain": [ 1218 | "# 3 methods for generic function \"newton\":\n", 1219 | "[1] newton(f, x0) in Main at In[20]:4\n", 1220 | "[2] newton(f, x0, n) in Main at In[20]:4\n", 1221 | "[3] newton(f, df, x0, n) in Main at In[1]:2" 1222 | ] 1223 | }, 1224 | "execution_count": 33, 1225 | "metadata": {}, 1226 | "output_type": "execute_result" 1227 | } 1228 | ], 1229 | "source": [ 1230 | "methods(newton)" 1231 | ] 1232 | }, 1233 | { 1234 | "cell_type": "code", 1235 | "execution_count": 34, 1236 | "metadata": {}, 1237 | "outputs": [ 1238 | { 1239 | "data": { 1240 | "text/plain": [ 1241 | "newton (generic function with 4 methods)" 1242 | ] 1243 | }, 1244 | "execution_count": 34, 1245 | "metadata": {}, 1246 | "output_type": "execute_result" 1247 | } 1248 | ], 1249 | "source": [ 1250 | "function newton(f, df, x0, n, derivative)\n", 1251 | " df = x -> derivative(f, x)\n", 1252 | " \n", 1253 | " return newton(f, df, x0, n)\n", 1254 | "end" 1255 | ] 1256 | }, 1257 | { 1258 | "cell_type": "code", 1259 | "execution_count": 35, 1260 | "metadata": {}, 1261 | "outputs": [ 1262 | { 1263 | "data": { 1264 | "text/plain": [ 1265 | "1.2599210498948732" 1266 | ] 1267 | }, 1268 | "execution_count": 35, 1269 | "metadata": {}, 1270 | "output_type": "execute_result" 1271 | } 1272 | ], 1273 | "source": [ 1274 | "newton(g, dg, a, 10, finite_difference)" 1275 | ] 1276 | }, 1277 | { 1278 | "cell_type": "code", 1279 | "execution_count": 36, 1280 | "metadata": {}, 1281 | "outputs": [ 1282 | { 1283 | "data": { 1284 | "text/plain": [ 1285 | "1.2599210498948732" 1286 | ] 1287 | }, 1288 | "execution_count": 36, 1289 | "metadata": {}, 1290 | "output_type": "execute_result" 1291 | } 1292 | ], 1293 | "source": [ 1294 | "newton(g, dg, a, 10, forwarddiff)" 1295 | ] 1296 | }, 1297 | { 1298 | "cell_type": "code", 1299 | "execution_count": 37, 1300 | "metadata": {}, 1301 | "outputs": [ 1302 | { 1303 | "data": { 1304 | "text/html": [ 1305 | "4 methods for generic function newton:" 1306 | ], 1307 | "text/plain": [ 1308 | "# 4 methods for generic function \"newton\":\n", 1309 | "[1] newton(f, x0) in Main at In[20]:4\n", 1310 | "[2] newton(f, x0, n) in Main at In[20]:4\n", 1311 | "[3] newton(f, df, x0, n) in Main at In[1]:2\n", 1312 | "[4] newton(f, df, x0, n, derivative) in Main at In[34]:2" 1313 | ] 1314 | }, 1315 | "execution_count": 37, 1316 | "metadata": {}, 1317 | "output_type": "execute_result" 1318 | } 1319 | ], 1320 | "source": [ 1321 | "methods(newton)" 1322 | ] 1323 | }, 1324 | { 1325 | "cell_type": "markdown", 1326 | "metadata": {}, 1327 | "source": [ 1328 | "We see that the `newton` function API (interface) is getting clumsy: it is difficult to remember which order and what each of the arguments represents. In future notebooks we will see how to improve this." 1329 | ] 1330 | } 1331 | ], 1332 | "metadata": { 1333 | "@webio": { 1334 | "lastCommId": null, 1335 | "lastKernelId": null 1336 | }, 1337 | "kernelspec": { 1338 | "display_name": "Julia 1.1.0", 1339 | "language": "julia", 1340 | "name": "julia-1.1" 1341 | }, 1342 | "language_info": { 1343 | "file_extension": ".jl", 1344 | "mimetype": "application/julia", 1345 | "name": "julia", 1346 | "version": "1.1.0" 1347 | } 1348 | }, 1349 | "nbformat": 4, 1350 | "nbformat_minor": 2 1351 | } 1352 | --------------------------------------------------------------------------------