├── requirements.txt ├── README.md ├── tutorial_notebooks ├── Part 8.ipynb ├── Part 5.ipynb ├── Part 1.ipynb ├── Part 6.ipynb ├── Part 4.ipynb ├── Part 2.ipynb ├── Part 3.ipynb ├── Part 7.ipynb └── churn_data.csv ├── full_notebooks ├── Part 8.ipynb ├── Part 5.ipynb ├── Part 4.ipynb ├── churn_data.csv ├── Part 1.ipynb ├── Part 2.ipynb ├── Part 6.ipynb └── Part 3.ipynb └── churn_data.csv /requirements.txt: -------------------------------------------------------------------------------- 1 | # 2 | # To install these dependencies, use 3 | # 4 | # `pip install -r requirements.txt` 5 | # 6 | # on the command line. 7 | # 8 | matplotlib 9 | numpy 10 | scipy 11 | autograd 12 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ### PyData NY 2019 Tutorial - New Trends in Estimation and Inference 2 | 3 | 4 | #### Installation 5 | 6 | ``` 7 | git clone https://github.com/CamDavidsonPilon/PyDataNY_2019_tutorial.git 8 | 9 | ``` 10 | 11 | 12 | To pip install the requirements: 13 | 14 | ``` 15 | pip install -r requirements.txt 16 | 17 | ``` 18 | 19 | You may need to update the local repo too (I've added recent commits!) 20 | 21 | ``` 22 | git pull origin master 23 | ``` 24 | 25 | #### Usage 26 | 27 | For people attending the tutorial, we will use the `tutorial_notebooks/` folder. Complete notebooks (for reference and checking) are in `full_notebooks/`. 28 | -------------------------------------------------------------------------------- /tutorial_notebooks/Part 8.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "### Optimizing in a unbounded space\n", 8 | "\n", 9 | "Sometimes we have the inclusion bounds, like $[0, \\inf)$, or $[0, 1]$. An option is to lean on `optimize.minimize` to enforce these bounds, but alternatively we can express the problem a domain that has no constraint, and then project back to the original domain. We'll use an example to make this concrete.\n", 10 | "\n", 11 | "1. To make a parameter cover $[0, \\inf)$\n", 12 | " 1. Use `bounds = (0, None)`. \n", 13 | " 2. Use a parameter over $(-\\inf, \\inf)$, and put it into the exponential function, which has range $(0, \\inf)$. \n", 14 | "2. To make a parameter cover $[0, 1]$:\n", 15 | " 1. Use `bounds = (0, None)`. \n", 16 | " 2. Use a parameter over $(-\\inf, \\inf)$, and put it into the expit function, which has range $(0, 1)$. \n", 17 | " \n", 18 | "$$ \\text{expit}(x) = \\frac{1}{1+\\exp(-x)} $$\n", 19 | "\n", 20 | "The B. cases typically have better convergence, and can take advantage of a larger family of optimization algorithms. " 21 | ] 22 | }, 23 | { 24 | "cell_type": "code", 25 | "execution_count": 1, 26 | "metadata": {}, 27 | "outputs": [], 28 | "source": [ 29 | "from autograd import numpy as np\n", 30 | "from autograd import elementwise_grad, value_and_grad, hessian\n", 31 | "from scipy.optimize import minimize\n", 32 | "\n", 33 | "df = pd.read_csv(\"../churn_data.csv\")\n", 34 | "T = df['T'].values\n", 35 | "E = df['E'].values\n", 36 | "\n", 37 | "breakpoints = np.array([28, 33, 58, 63, 88, 93, 117, 122, 148, 153])\n" 38 | ] 39 | }, 40 | { 41 | "cell_type": "code", 42 | "execution_count": 2, 43 | "metadata": {}, 44 | "outputs": [ 45 | { 46 | "name": "stdout", 47 | "output_type": "stream", 48 | "text": [ 49 | "[0.00103402 0.0332657 0.00104655 0.01516591 0.0005534 0.01317368\n", 50 | " 0.00099481 0.01216889 0.00125312 0.0068996 ]\n" 51 | ] 52 | } 53 | ], 54 | "source": [ 55 | "# same model as last time, and note we have the `bounds` argument inactive.\n", 56 | "\n", 57 | "def cumulative_hazard(log_params, times):\n", 58 | " # this is NumPy black magic to get piecewise hazards, let's chat after. \n", 59 | " times = np.atleast_1d(times)\n", 60 | " n = times.shape[0]\n", 61 | " times = times.reshape((n, 1))\n", 62 | " M = np.minimum(np.tile(breakpoints, (n, 1)), times)\n", 63 | " M = np.hstack([M[:, tuple([0])], np.diff(M, axis=1)])\n", 64 | " return np.dot(M, np.exp(log_params)) # diff here, use to be np.dot(M, params)\n", 65 | "\n", 66 | "hazard = elementwise_grad(cumulative_hazard, argnum=1)\n", 67 | "\n", 68 | "def survival_function(params, t):\n", 69 | " return np.exp(-cumulative_hazard(params, t))\n", 70 | "\n", 71 | "def log_hazard(params, t):\n", 72 | " return np.log(np.clip(hazard(params, t), 1e-25, np.inf))\n", 73 | "\n", 74 | "def log_likelihood(params, t, e):\n", 75 | " return np.sum(e * log_hazard(params, t)) - np.sum(cumulative_hazard(params, t))\n", 76 | "\n", 77 | "def negative_log_likelihood(params, t, e):\n", 78 | " return -log_likelihood(params, t, e)\n", 79 | "\n", 80 | "from autograd import value_and_grad\n", 81 | "\n", 82 | "results = minimize(\n", 83 | " value_and_grad(negative_log_likelihood), \n", 84 | " x0 = np.zeros(len(breakpoints)),\n", 85 | " method=None, \n", 86 | " args=(T, E),\n", 87 | " jac=True,\n", 88 | " bounds=None\n", 89 | ")\n", 90 | "\n", 91 | "log_estimates_ = results.x\n", 92 | "estimates_ = # what goes here?\n", 93 | "print(estimates_)\n", 94 | "# see previous Part 7 to confirm these are \"really close enough\"" 95 | ] 96 | }, 97 | { 98 | "cell_type": "markdown", 99 | "metadata": {}, 100 | "source": [ 101 | "Congratulations! You've sucessfully implemented a pretty good approximations to the Python package [lifelines](https://lifelines.readthedocs.io/)! \n", 102 | "\n", 103 | "Let's move onto Part 9 (slides), which is off this track: interpreting output. " 104 | ] 105 | } 106 | ], 107 | "metadata": { 108 | "kernelspec": { 109 | "display_name": "Python 3", 110 | "language": "python", 111 | "name": "python3" 112 | }, 113 | "language_info": { 114 | "codemirror_mode": { 115 | "name": "ipython", 116 | "version": 3 117 | }, 118 | "file_extension": ".py", 119 | "mimetype": "text/x-python", 120 | "name": "python", 121 | "nbconvert_exporter": "python", 122 | "pygments_lexer": "ipython3", 123 | "version": "3.7.3" 124 | } 125 | }, 126 | "nbformat": 4, 127 | "nbformat_minor": 2 128 | } 129 | -------------------------------------------------------------------------------- /tutorial_notebooks/Part 5.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "### Hessians to compute variances\n", 8 | "\n", 9 | " - We have our point estimates, but now we want variances of the estimates\n", 10 | " - We can approximate them (under some asymptotic conditions) using the _Hessian at the (negative) MLE minimum_\n", 11 | " - Think of the the Hessian as a matrix representing the curvature at the minimum point. If we move slightly away from that minimum, how much change do we see in the likelihood?\n", 12 | " - high curvature (think of a tight peak) => low variance of estimate.\n", 13 | " - low curvature (think of a broad hill) => high variance of estimate.\n", 14 | "\n", 15 | "To extract the parameter estimates, we invert the Hessian at the MLE, $\\hat{\\theta}$, and take the diagonal:\n", 16 | "\n", 17 | "$$\\text{Var}(\\hat{\\theta}_i) = \\text{diag}(H(\\hat{\\theta})^{-1})_i $$\n", 18 | "\n", 19 | "The Hessian is the matrix of all second derivatives. `autograd` has this built in:" 20 | ] 21 | }, 22 | { 23 | "cell_type": "code", 24 | "execution_count": null, 25 | "metadata": {}, 26 | "outputs": [], 27 | "source": [ 28 | "from autograd import numpy as np\n", 29 | "from autograd import elementwise_grad, value_and_grad, hessian\n", 30 | "from scipy.optimize import minimize" 31 | ] 32 | }, 33 | { 34 | "cell_type": "code", 35 | "execution_count": null, 36 | "metadata": {}, 37 | "outputs": [], 38 | "source": [ 39 | "T = (np.random.exponential(size=1000)/1.5) ** 2.3\n", 40 | "E = np.random.binomial(1, 0.95, size=1000)" 41 | ] 42 | }, 43 | { 44 | "cell_type": "code", 45 | "execution_count": null, 46 | "metadata": {}, 47 | "outputs": [], 48 | "source": [ 49 | "# seen all this...\n", 50 | "def cumulative_hazard(params, t):\n", 51 | " lambda_, rho_ = params\n", 52 | " return (t / lambda_) ** rho_\n", 53 | "\n", 54 | "hazard = elementwise_grad(cumulative_hazard, argnum=1)\n", 55 | "\n", 56 | "def log_hazard(params, t):\n", 57 | " return np.log(hazard(params, t))\n", 58 | "\n", 59 | "def log_likelihood(params, t, e):\n", 60 | " return np.sum(e * log_hazard(params, t)) - np.sum(cumulative_hazard(params, t))\n", 61 | "\n", 62 | "def negative_log_likelihood(params, t, e):\n", 63 | " return -log_likelihood(params, t, e)\n", 64 | "\n", 65 | "from autograd import value_and_grad\n", 66 | "\n", 67 | "results = minimize(\n", 68 | " value_and_grad(negative_log_likelihood), \n", 69 | " x0 = np.array([1.0, 1.0]),\n", 70 | " method=None, \n", 71 | " args=(T, E),\n", 72 | " jac=True,\n", 73 | " bounds=((0.00001, None), (0.00001, None)))\n", 74 | "\n", 75 | "print(results)" 76 | ] 77 | }, 78 | { 79 | "cell_type": "code", 80 | "execution_count": null, 81 | "metadata": {}, 82 | "outputs": [], 83 | "source": [ 84 | "estimates_ = results.x\n", 85 | "\n", 86 | "# Note: this will produce a matrix\n", 87 | "hessian(negative_log_likelihood)(estimates_, T, E)" 88 | ] 89 | }, 90 | { 91 | "cell_type": "code", 92 | "execution_count": null, 93 | "metadata": {}, 94 | "outputs": [], 95 | "source": [ 96 | "H = hessian(negative_log_likelihood)(estimates_, T, E)\n", 97 | "variance_ = np.diag(np.linalg.inv(H))\n", 98 | "print(variance_)" 99 | ] 100 | }, 101 | { 102 | "cell_type": "code", 103 | "execution_count": null, 104 | "metadata": {}, 105 | "outputs": [], 106 | "source": [ 107 | "std_error = np.sqrt(variance_)\n", 108 | "print(std_error)" 109 | ] 110 | }, 111 | { 112 | "cell_type": "code", 113 | "execution_count": null, 114 | "metadata": {}, 115 | "outputs": [], 116 | "source": [ 117 | "print(\"lambda_ %.3f (±%.2f)\" % (estimates_[0], std_error[0]))\n", 118 | "print(\"rho_ %.3f (±%.2f)\" % (estimates_[1], std_error[1]))" 119 | ] 120 | }, 121 | { 122 | "cell_type": "markdown", 123 | "metadata": {}, 124 | "source": [ 125 | "From here you can construct confidence intervals (CIs), and such. Let's move to Part 6, which is where you connect these abstract parameters to business logic. " 126 | ] 127 | } 128 | ], 129 | "metadata": { 130 | "kernelspec": { 131 | "display_name": "Python 3", 132 | "language": "python", 133 | "name": "python3" 134 | }, 135 | "language_info": { 136 | "codemirror_mode": { 137 | "name": "ipython", 138 | "version": 3 139 | }, 140 | "file_extension": ".py", 141 | "mimetype": "text/x-python", 142 | "name": "python", 143 | "nbconvert_exporter": "python", 144 | "pygments_lexer": "ipython3", 145 | "version": "3.7.3" 146 | } 147 | }, 148 | "nbformat": 4, 149 | "nbformat_minor": 2 150 | } 151 | -------------------------------------------------------------------------------- /tutorial_notebooks/Part 1.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Part 1 \n", 8 | "\n", 9 | " - We'll start with fitting data to a parametric model\n", 10 | " \n", 11 | " - So we need to create the log-likelihood, which we will optimize over. \n", 12 | " - Let's create some fake data first." 13 | ] 14 | }, 15 | { 16 | "cell_type": "code", 17 | "execution_count": null, 18 | "metadata": {}, 19 | "outputs": [], 20 | "source": [ 21 | "%matplotlib inline\n", 22 | "import numpy as np\n", 23 | "from matplotlib import pyplot as plt\n", 24 | "\n", 25 | "\n", 26 | "T = (np.random.exponential(size=1000)/1.5) ** 2.3\n", 27 | "\n", 28 | "plt.hist(T, bins=50);" 29 | ] 30 | }, 31 | { 32 | "cell_type": "markdown", 33 | "metadata": {}, 34 | "source": [ 35 | "Let's use the Weibull parametric model: https://en.wikipedia.org/wiki/Weibull_distribution" 36 | ] 37 | }, 38 | { 39 | "cell_type": "code", 40 | "execution_count": null, 41 | "metadata": {}, 42 | "outputs": [], 43 | "source": [ 44 | "def pdf(params, t):\n", 45 | " lambda_, rho_ = params\n", 46 | " # I'm not going to make you type this out - it's annoying and error prone. \n", 47 | " return rho_ / lambda_ * (t / lambda_) ** (rho_ - 1) * np.exp(-(t/lambda_) ** rho_)\n", 48 | "\n", 49 | "# okay, but we actually need the _log_ of the pdf\n", 50 | "def log_pdf(params, t):\n", 51 | " lambda_, rho_ = params\n", 52 | " # I'm not going to make you type this out - it's annoying and error prone. \n", 53 | " return np.log(rho_) - np.log(lambda_) + (rho_ - 1) * (np.log(t) - np.log(lambda_)) - (t/lambda_) ** rho_\n", 54 | "\n", 55 | "# now we can define the log likehood\n", 56 | "\n", 57 | "def log_likelihood(params, t):\n", 58 | " return np.sum(log_pdf(params, t)) \n" 59 | ] 60 | }, 61 | { 62 | "cell_type": "markdown", 63 | "metadata": {}, 64 | "source": [ 65 | "## `scipy.optimize.minimize`" 66 | ] 67 | }, 68 | { 69 | "cell_type": "code", 70 | "execution_count": null, 71 | "metadata": { 72 | "slideshow": { 73 | "slide_type": "-" 74 | } 75 | }, 76 | "outputs": [], 77 | "source": [ 78 | "from scipy.optimize import minimize\n", 79 | "\n", 80 | "results = minimize(log_likelihood, \n", 81 | " x0 = # some initial guess of the parameters. \n", 82 | " method=None, \n", 83 | " args=(T, ))\n", 84 | "\n", 85 | "print(results)" 86 | ] 87 | }, 88 | { 89 | "cell_type": "code", 90 | "execution_count": null, 91 | "metadata": {}, 92 | "outputs": [], 93 | "source": [ 94 | "def log_likelihood(params, t):\n", 95 | " return # this should change\n" 96 | ] 97 | }, 98 | { 99 | "cell_type": "code", 100 | "execution_count": null, 101 | "metadata": {}, 102 | "outputs": [], 103 | "source": [ 104 | "results = minimize(log_likelihood, \n", 105 | " x0 = np.array([1.0, 1.0]), # some initial guess of the parameters. \n", 106 | " method=None, \n", 107 | " args=(T, ))\n", 108 | "\n", 109 | "print(results)" 110 | ] 111 | }, 112 | { 113 | "cell_type": "code", 114 | "execution_count": null, 115 | "metadata": {}, 116 | "outputs": [], 117 | "source": [ 118 | "# weibull parameters must be greater than 0!\n", 119 | "# we can \"nudge\" the minimizer to understand this using the bounds argument\n", 120 | "results = minimize(log_likelihood, \n", 121 | " x0 = np.array([1.0, 1.0]),\n", 122 | " method=None, \n", 123 | " args=(T, ),\n", 124 | " bounds=#fill this in)\n", 125 | "\n", 126 | "print(results)" 127 | ] 128 | }, 129 | { 130 | "cell_type": "markdown", 131 | "metadata": {}, 132 | "source": [ 133 | "## Takeaway: `minimize` is a very flexible function for small-mid size parameter optimization" 134 | ] 135 | }, 136 | { 137 | "cell_type": "markdown", 138 | "metadata": {}, 139 | "source": [ 140 | "### A few problems though:\n", 141 | "\n", 142 | "1. You are stuck with only using known parametric models that are easy to implement. \n", 143 | "2. Not very fast minimization routines\n", 144 | "\n", 145 | "Let's move to Part 2. " 146 | ] 147 | } 148 | ], 149 | "metadata": { 150 | "kernelspec": { 151 | "display_name": "Python 3", 152 | "language": "python", 153 | "name": "python3" 154 | }, 155 | "language_info": { 156 | "codemirror_mode": { 157 | "name": "ipython", 158 | "version": 3 159 | }, 160 | "file_extension": ".py", 161 | "mimetype": "text/x-python", 162 | "name": "python", 163 | "nbconvert_exporter": "python", 164 | "pygments_lexer": "ipython3", 165 | "version": "3.7.3" 166 | } 167 | }, 168 | "nbformat": 4, 169 | "nbformat_minor": 2 170 | } 171 | -------------------------------------------------------------------------------- /full_notebooks/Part 8.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "### Optimizing in a unbounded space\n", 8 | "\n", 9 | "Sometimes we have the inclusion bounds, like $[0, \\inf)$, or $[0, 1]$. An option is to lean on `optimize.minimize` to enforce these bounds, but alternatively we can express the problem a domain that has no constraint, and then project back to the original domain. We'll use an example to make this concrete.\n", 10 | "\n", 11 | "1. To make a parameter cover $[0, \\inf)$\n", 12 | " 1. Use `bounds = (0, None)`. \n", 13 | " 2. Use a parameter over $(-\\inf, \\inf)$, and put it into the exponential function, which has range $(0, \\inf)$. \n", 14 | "2. To make a parameter cover $[0, 1]$:\n", 15 | " 1. Use `bounds = (0, None)`. \n", 16 | " 2. Use a parameter over $(-\\inf, \\inf)$, and put it into the expit function, which has range $(0, 1)$. \n", 17 | " \n", 18 | "$$ \\text{expit}(x) = \\frac{1}{1+\\exp(-x)} $$\n", 19 | "\n", 20 | "The B. cases typically have better convergence, and can take advantage of a larger family of optimization algorithms. " 21 | ] 22 | }, 23 | { 24 | "cell_type": "code", 25 | "execution_count": 1, 26 | "metadata": {}, 27 | "outputs": [], 28 | "source": [ 29 | "from autograd import numpy as np\n", 30 | "from autograd import elementwise_grad, value_and_grad, hessian\n", 31 | "from scipy.optimize import minimize\n", 32 | "\n", 33 | "df = pd.read_csv(\"../churn_data.csv\")\n", 34 | "T = df['T'].values\n", 35 | "E = df['E'].values\n", 36 | "\n", 37 | "breakpoints = np.array([28, 33, 58, 63, 88, 93, 117, 122, 148, 153])\n" 38 | ] 39 | }, 40 | { 41 | "cell_type": "code", 42 | "execution_count": 2, 43 | "metadata": {}, 44 | "outputs": [ 45 | { 46 | "name": "stdout", 47 | "output_type": "stream", 48 | "text": [ 49 | "[0.00103402 0.0332657 0.00104655 0.01516591 0.0005534 0.01317368\n", 50 | " 0.00099481 0.01216889 0.00125312 0.0068996 ]\n" 51 | ] 52 | } 53 | ], 54 | "source": [ 55 | "# same model as last time, and note we have the `bounds` argument inactive.\n", 56 | "\n", 57 | "def cumulative_hazard(log_params, times):\n", 58 | " # this is NumPy black magic to get piecewise hazards, let's chat after. \n", 59 | " times = np.atleast_1d(times)\n", 60 | " n = times.shape[0]\n", 61 | " times = times.reshape((n, 1))\n", 62 | " M = np.minimum(np.tile(breakpoints, (n, 1)), times)\n", 63 | " M = np.hstack([M[:, tuple([0])], np.diff(M, axis=1)])\n", 64 | " return np.dot(M, np.exp(log_params)) # diff here, use to be np.dot(M, params)\n", 65 | "\n", 66 | "hazard = elementwise_grad(cumulative_hazard, argnum=1)\n", 67 | "\n", 68 | "def survival_function(params, t):\n", 69 | " return np.exp(-cumulative_hazard(params, t))\n", 70 | "\n", 71 | "def log_hazard(params, t):\n", 72 | " return np.log(np.clip(hazard(params, t), 1e-25, np.inf))\n", 73 | "\n", 74 | "def log_likelihood(params, t, e):\n", 75 | " return np.sum(e * log_hazard(params, t)) - np.sum(cumulative_hazard(params, t))\n", 76 | "\n", 77 | "def negative_log_likelihood(params, t, e):\n", 78 | " return -log_likelihood(params, t, e)\n", 79 | "\n", 80 | "from autograd import value_and_grad\n", 81 | "\n", 82 | "results = minimize(\n", 83 | " value_and_grad(negative_log_likelihood), \n", 84 | " x0 = np.zeros(len(breakpoints)),\n", 85 | " method=None, \n", 86 | " args=(T, E),\n", 87 | " jac=True,\n", 88 | " bounds=None\n", 89 | ")\n", 90 | "\n", 91 | "log_estimates_ = results.x\n", 92 | "estimates_ = np.exp(log_estimates_)\n", 93 | "print(estimates_)\n", 94 | "# see previous Part 7 to confirm these are \"really close enough\"" 95 | ] 96 | }, 97 | { 98 | "cell_type": "markdown", 99 | "metadata": {}, 100 | "source": [ 101 | "Congratulations! You've sucessfully implemented a pretty good approximations to the Python package [lifelines](https://lifelines.readthedocs.io/)! \n", 102 | "\n", 103 | "Let's move onto Part 9 (slides), which is off this track: interpreting output. " 104 | ] 105 | }, 106 | { 107 | "cell_type": "code", 108 | "execution_count": null, 109 | "metadata": {}, 110 | "outputs": [], 111 | "source": [] 112 | } 113 | ], 114 | "metadata": { 115 | "kernelspec": { 116 | "display_name": "Python 3", 117 | "language": "python", 118 | "name": "python3" 119 | }, 120 | "language_info": { 121 | "codemirror_mode": { 122 | "name": "ipython", 123 | "version": 3 124 | }, 125 | "file_extension": ".py", 126 | "mimetype": "text/x-python", 127 | "name": "python", 128 | "nbconvert_exporter": "python", 129 | "pygments_lexer": "ipython3", 130 | "version": "3.7.3" 131 | } 132 | }, 133 | "nbformat": 4, 134 | "nbformat_minor": 2 135 | } 136 | -------------------------------------------------------------------------------- /tutorial_notebooks/Part 6.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "### The Delta Method\n", 8 | "\n", 9 | "> CFO: What's our variation in churn this year?\n", 10 | "\n", 11 | "> Data scientist: Our $\\lambda$ value has been increasing, but $\\rho$ is staying the same, so we should see-\n", 12 | "\n", 13 | "> CFO: Our banana value is increasing?\n", 14 | "\n", 15 | "We want to connect our parameters to business logic _and_ carry over variance in estimates. \n", 16 | "\n", 17 | "Example: It's silly to present a point estimate without confidence intervals (CIs), since arguably the CIs contains more useful information than a point estimate. \n", 18 | "\n", 19 | "We'll start with asking: \n", 20 | "\n", 21 | "> what is the CI for the survival function, $S(t; \\hat{\\theta})$? \n", 22 | "\n", 23 | "\n", 24 | "We will use the **Delta method** to do this (bolded because it's awesome):\n", 25 | "\n", 26 | "$$\\text{Var}(f(\\hat{\\theta})) \\approx \\text{grad}(f)(\\hat{\\theta}) \\cdot \\text{Var}(\\hat{\\theta}) \\cdot \\text{grad}(f)(\\hat{\\theta}) ^ T $$\n", 27 | "\n", 28 | "1. $f$ in our case is the survival function, $S$\n", 29 | "2. We know $\\text{Var}(\\hat{\\theta})$ (inverse of the Hessian)\n", 30 | "3. Do we need to compute $\\text{grad}(f)$ by hand? Heck no, use `autograd`" 31 | ] 32 | }, 33 | { 34 | "cell_type": "code", 35 | "execution_count": null, 36 | "metadata": {}, 37 | "outputs": [], 38 | "source": [ 39 | "# seen all this...\n", 40 | "%matplotlib inline\n", 41 | "from autograd import numpy as np\n", 42 | "from autograd import elementwise_grad, value_and_grad, hessian\n", 43 | "from scipy.optimize import minimize\n", 44 | "\n", 45 | "# N = 50 for this example\n", 46 | "T = (np.random.exponential(size=50)/1.5) ** 2.3\n", 47 | "E = np.random.binomial(1, 0.95, size=50)\n", 48 | "\n", 49 | "def cumulative_hazard(params, t):\n", 50 | " lambda_, rho_ = params\n", 51 | " return (t / lambda_) ** rho_\n", 52 | "\n", 53 | "hazard = elementwise_grad(cumulative_hazard, argnum=1)\n", 54 | "\n", 55 | "def log_hazard(params, t):\n", 56 | " return np.log(hazard(params, t))\n", 57 | "\n", 58 | "def log_likelihood(params, t, e):\n", 59 | " return np.sum(e * log_hazard(params, t)) - np.sum(cumulative_hazard(params, t))\n", 60 | "\n", 61 | "def negative_log_likelihood(params, t, e):\n", 62 | " return -log_likelihood(params, t, e)\n", 63 | "\n", 64 | "from autograd import value_and_grad\n", 65 | "\n", 66 | "results = minimize(\n", 67 | " value_and_grad(negative_log_likelihood), \n", 68 | " x0 = np.array([1.0, 1.0]),\n", 69 | " method=None, \n", 70 | " args=(T, E),\n", 71 | " jac=True,\n", 72 | " bounds=((0.00001, None), (0.00001, None)))\n", 73 | "\n", 74 | "estimates_ = results.x\n", 75 | "H = hessian(negative_log_likelihood)(estimates_, T, E)\n", 76 | "variance_matrix_ = np.linalg.inv(H)" 77 | ] 78 | }, 79 | { 80 | "cell_type": "code", 81 | "execution_count": null, 82 | "metadata": {}, 83 | "outputs": [], 84 | "source": [ 85 | "from autograd import grad\n", 86 | "\n", 87 | "def survival_function(params, t):\n", 88 | " return np.exp(-cumulative_hazard(params, t))\n", 89 | "\n", 90 | "grad_sf = # what goes here?\n", 91 | "grad_sf(estimates_, 5.0)" 92 | ] 93 | }, 94 | { 95 | "cell_type": "code", 96 | "execution_count": null, 97 | "metadata": {}, 98 | "outputs": [], 99 | "source": [ 100 | "def variance_at_t(t):\n", 101 | " return # what goes here?" 102 | ] 103 | }, 104 | { 105 | "cell_type": "code", 106 | "execution_count": null, 107 | "metadata": {}, 108 | "outputs": [], 109 | "source": [ 110 | "variance_at_t(5)" 111 | ] 112 | }, 113 | { 114 | "cell_type": "code", 115 | "execution_count": null, 116 | "metadata": {}, 117 | "outputs": [], 118 | "source": [ 119 | "t = np.linspace(.001, 10, 100)\n", 120 | "\n", 121 | "std_sf = np.sqrt(np.array([variance_at_t(_) for _ in t]))" 122 | ] 123 | }, 124 | { 125 | "cell_type": "code", 126 | "execution_count": null, 127 | "metadata": {}, 128 | "outputs": [], 129 | "source": [ 130 | "plt.plot(t, survival_function(estimates_, t))\n", 131 | "plt.fill_between(t, \n", 132 | " y1=survival_function(estimates_, t) + 1.65 * std_sf, \n", 133 | " y2=survival_function(estimates_, t) - 1.65 * std_sf,\n", 134 | " alpha=0.3\n", 135 | " )\n", 136 | "plt.ylim(0, 1)\n", 137 | "plt.title(\"Estimated survival function with CIs (Delta method)\")" 138 | ] 139 | }, 140 | { 141 | "cell_type": "markdown", 142 | "metadata": {}, 143 | "source": [ 144 | "Next, we will explore a subscription service LTV example. Move to Part 7! " 145 | ] 146 | } 147 | ], 148 | "metadata": { 149 | "kernelspec": { 150 | "display_name": "Python 3", 151 | "language": "python", 152 | "name": "python3" 153 | }, 154 | "language_info": { 155 | "codemirror_mode": { 156 | "name": "ipython", 157 | "version": 3 158 | }, 159 | "file_extension": ".py", 160 | "mimetype": "text/x-python", 161 | "name": "python", 162 | "nbconvert_exporter": "python", 163 | "pygments_lexer": "ipython3", 164 | "version": "3.7.3" 165 | } 166 | }, 167 | "nbformat": 4, 168 | "nbformat_minor": 2 169 | } 170 | -------------------------------------------------------------------------------- /tutorial_notebooks/Part 4.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "Let's try generalizing our model now. For survival analysis, it is very useful to think about cumulative hazards - that is, modelling the cumulative hazard is typically easiest. " 8 | ] 9 | }, 10 | { 11 | "cell_type": "code", 12 | "execution_count": null, 13 | "metadata": {}, 14 | "outputs": [], 15 | "source": [ 16 | "# Note: we are shifting the data 4 units to the right.\n", 17 | "T = (np.random.exponential(size=1000)/1.5) ** 2.3 + 4\n", 18 | "E = np.random.binomial(1, 0.95, size=1000)" 19 | ] 20 | }, 21 | { 22 | "cell_type": "code", 23 | "execution_count": null, 24 | "metadata": {}, 25 | "outputs": [], 26 | "source": [ 27 | "from autograd import numpy as np\n", 28 | "\n", 29 | "def cumulative_hazard(params, t):\n", 30 | " lambda_, rho_, mu = params\n", 31 | " return ((t - mu) / lambda_) ** rho_\n", 32 | "\n", 33 | "def log_hazard(params, t):\n", 34 | " lambda_, rho_, mu = params\n", 35 | " return ... # this could get arbitrarly complicated." 36 | ] 37 | }, 38 | { 39 | "cell_type": "code", 40 | "execution_count": null, 41 | "metadata": {}, 42 | "outputs": [], 43 | "source": [ 44 | "from autograd import elementwise_grad\n", 45 | "hazard = elementwise_grad(cumulative_hazard, argnum=# what should this be?)\n", 46 | "\n", 47 | "def log_hazard(params, t):\n", 48 | " return np.log(hazard(params, t))" 49 | ] 50 | }, 51 | { 52 | "cell_type": "code", 53 | "execution_count": null, 54 | "metadata": {}, 55 | "outputs": [], 56 | "source": [ 57 | "log_hazard(np.array([2., 2. , 0.]), np.array([1,2,3]))" 58 | ] 59 | }, 60 | { 61 | "cell_type": "code", 62 | "execution_count": null, 63 | "metadata": {}, 64 | "outputs": [], 65 | "source": [ 66 | "# same as previous\n", 67 | "\n", 68 | "from autograd import value_and_grad\n", 69 | "from scipy.optimize import minimize\n", 70 | "\n", 71 | "def log_likelihood(params, t, e):\n", 72 | " return np.sum(e * log_hazard(params, t)) - np.sum(cumulative_hazard(params, t))\n", 73 | "\n", 74 | "def negative_log_likelihood(params, t, e):\n", 75 | " return -log_likelihood(params, t, e)\n", 76 | "\n", 77 | "results = minimize(\n", 78 | " value_and_grad(negative_log_likelihood), \n", 79 | " x0 = np.array([1.0, 1.0, 0.0]),\n", 80 | " method=None, \n", 81 | " args=(T, E),\n", 82 | " jac=True,\n", 83 | " bounds=((0.00001, None), (0.00001, None), (None, np.min(T)-0.001)))\n", 84 | "\n", 85 | "print(results)" 86 | ] 87 | }, 88 | { 89 | "cell_type": "markdown", 90 | "metadata": {}, 91 | "source": [ 92 | "So long as you are good at \"parameter accounting\", then you could create arbitrarly complicated survival models simply by specifying the hazard. \n", 93 | "\n", 94 | "Extending to including covariates is straightforward too, with some modifications to the code. Here's a simple model of the cumulative hazard:\n", 95 | "\n", 96 | "$$H(t; x) = \\left(\\frac{t}{\\lambda(x)}\\right)^\\rho $$\n", 97 | "$$ \\lambda(x) = \\beta_1 x_1 + \\beta_2 x_2 $$" 98 | ] 99 | }, 100 | { 101 | "cell_type": "code", 102 | "execution_count": null, 103 | "metadata": {}, 104 | "outputs": [], 105 | "source": [ 106 | "from scipy.stats import weibull_min\n", 107 | "\n", 108 | "# create some regression data. \n", 109 | "N = 10000\n", 110 | "X = 0.1 * np.random.normal(size=(N, 2))\n", 111 | "T = np.exp(2 * X[:, 0] + -2 * X[:, 1]) * weibull_min.rvs(1, scale=1, loc=0, size=N)\n", 112 | "E = np.random.binomial(1, 0.99, size=N)" 113 | ] 114 | }, 115 | { 116 | "cell_type": "code", 117 | "execution_count": null, 118 | "metadata": {}, 119 | "outputs": [], 120 | "source": [ 121 | "def cumulative_hazard(params, t, X):\n", 122 | " log_lambda_coefs_, rho_ = # what goes here - need to unpack the parameters.\n", 123 | " lambda_ = np.exp(np.dot(X, log_lambda_coefs_))\n", 124 | " return (t / lambda_) ** rho_\n", 125 | "\n", 126 | "# these functions are almost identical to above, \n", 127 | "# expect they have an additional argument, X\n", 128 | "hazard = elementwise_grad(cumulative_hazard, argnum=1)\n", 129 | "\n", 130 | "def log_hazard(params, t, X):\n", 131 | " return np.log(hazard(params, t, X))\n", 132 | "\n", 133 | "def log_likelihood(params, t, e, X):\n", 134 | " return np.sum(e * log_hazard(params, t, X)) - np.sum(cumulative_hazard(params, t, X))\n", 135 | "\n", 136 | "def negative_log_likelihood(params, t, e, X):\n", 137 | " return -log_likelihood(params, t, e, X)\n", 138 | "\n", 139 | "results = minimize(\n", 140 | " value_and_grad(negative_log_likelihood), \n", 141 | " x0 = np.array([0.0, 0.0, 1.0]),\n", 142 | " method=None, \n", 143 | " args=(T, E, X),\n", 144 | " jac=True,\n", 145 | " bounds=((None, None), (None, None), (0.00001, None)))\n", 146 | "\n", 147 | "print(results)" 148 | ] 149 | }, 150 | { 151 | "cell_type": "markdown", 152 | "metadata": {}, 153 | "source": [ 154 | "Great! Let's move onto part 5, how to estimate the _variances_ of the parameters." 155 | ] 156 | } 157 | ], 158 | "metadata": { 159 | "kernelspec": { 160 | "display_name": "Python 3", 161 | "language": "python", 162 | "name": "python3" 163 | }, 164 | "language_info": { 165 | "codemirror_mode": { 166 | "name": "ipython", 167 | "version": 3 168 | }, 169 | "file_extension": ".py", 170 | "mimetype": "text/x-python", 171 | "name": "python", 172 | "nbconvert_exporter": "python", 173 | "pygments_lexer": "ipython3", 174 | "version": "3.7.3" 175 | } 176 | }, 177 | "nbformat": 4, 178 | "nbformat_minor": 2 179 | } 180 | -------------------------------------------------------------------------------- /tutorial_notebooks/Part 2.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "### Introducing censoring\n", 8 | "\n", 9 | "![img](https://lifelines.readthedocs.io/en/latest/_images/survival_analysis_intro_censoring.png)\n", 10 | "\n", 11 | "\n", 12 | "![img2](https://lifelines.readthedocs.io/en/latest/_images/survival_analysis_intro_censoring_revealed.png)" 13 | ] 14 | }, 15 | { 16 | "cell_type": "markdown", 17 | "metadata": {}, 18 | "source": [ 19 | "All we know is that actual lifetime is greater than some threshold. Mathematically, we know $P(T \\ge t) = 1 - F(t) := S(t)$. We can use this in our log likelihood:\n", 20 | "\n", 21 | "No censoring cases:\n", 22 | "\n", 23 | "$$l(\\theta, t) = \\sum_{\\text{observed}} \\log{\\text{pdf}(\\theta, t)} $$\n", 24 | "\n", 25 | "With censoring cases:\n", 26 | "$$ \n", 27 | "\\begin{align}\n", 28 | "l(\\theta, t) & = \\sum_{\\text{observed}} \\log{\\text{pdf}(t, \\theta)} + \\sum_{\\text{censored}} \\log{\\text{S}(t, \\theta)} \\\\\n", 29 | "& = \\sum_{\\text{observed}} \\log{\\text{pdf}(t, \\theta)} \\frac{S(t)}{S(t)} + \\sum_{\\text{censored}} \\log{\\text{S}(t, \\theta)} \\\\\n", 30 | "& = \\sum_{\\text{observed}} (\\log{\\frac{\\text{pdf}(t, \\theta)}{S(t)}} + \\log{S(t)}) + \\sum_{\\text{censored}} \\log{\\text{S}(t, \\theta)} \\\\\n", 31 | "& = \\sum_{\\text{observed}} \\log{\\frac{\\text{pdf}(t, \\theta)}{S(t)}} + \\sum_{\\text{observed}} \\log{S(t)} + \\sum_{\\text{censored}} \\log{\\text{S}(t, \\theta)} \\\\\n", 32 | "& = \\sum_{\\text{observed}} \\log{\\frac{\\text{pdf}(t, \\theta)}{S(t)}} + \\sum \\log{S(t)} \n", 33 | "\\end{align}\n", 34 | "$$\n", 35 | "\n", 36 | "\n", 37 | "\n", 38 | "The $-\\log{S(t)}$ is known as the _cumulative hazard_, denoted $H(t)$. \n", 39 | "\n", 40 | "$$l(\\theta, t) = \\sum_{\\text{observed}} \\log{\\frac{\\text{pdf}(t, \\theta)}{S(t)}} - \\sum H(t, \\theta) $$\n", 41 | "\n", 42 | "Also, $\\frac{dH}{dt} = \\frac{\\text{pdf}(t, \\theta)}{S(t)}$. Denote that $h(t)$. \n", 43 | "\n", 44 | "$$l(\\theta, t) = \\sum_{\\text{observed}} \\log{h(t, \\theta}) - \\sum H(t, \\theta) $$" 45 | ] 46 | }, 47 | { 48 | "cell_type": "markdown", 49 | "metadata": {}, 50 | "source": [ 51 | "Phew! Now, instead of working in probability space, we will work in hazard space! Here's a link to all the relatioships: https://lifelines.readthedocs.io/en/latest/Survival%20Analysis%20intro.html#hazard-function \n", 52 | "\n", 53 | "\n", 54 | "## Take away: the likelihood function can be used to \"add\" information about the system (think about how penalizers are used...)" 55 | ] 56 | }, 57 | { 58 | "cell_type": "code", 59 | "execution_count": null, 60 | "metadata": {}, 61 | "outputs": [], 62 | "source": [ 63 | "# the hazard and cumulative hazard for Weibull are much simplier to implement 😗👌\n", 64 | "\n", 65 | "def cumulative_hazard(params, t):\n", 66 | " lambda_, rho_ = params\n", 67 | " return (t / lambda_) ** rho_\n", 68 | "\n", 69 | "def hazard(params, t):\n", 70 | " # diff of cumulative hazard w.r.t. t\n", 71 | " lambda_, rho_ = params\n", 72 | " return rho_ / lambda_ * (t / lambda_) ** (rho_ - 1)\n", 73 | "\n", 74 | "def log_hazard(params, t):\n", 75 | " return np.log(hazard(params, t))\n", 76 | "\n", 77 | "def log_likelihood(params, t, e):\n", 78 | " return np.sum(e * log_hazard(params, t)) - np.sum(cumulative_hazard(params, t))\n" 79 | ] 80 | }, 81 | { 82 | "cell_type": "code", 83 | "execution_count": null, 84 | "metadata": {}, 85 | "outputs": [], 86 | "source": [ 87 | "T = (np.random.exponential(size=1000)/1.5) ** 2.3\n", 88 | "E = np.random.binomial(1, 0.95, size=1000)\n", 89 | "\n", 90 | "from scipy.optimize import minimize\n", 91 | "\n", 92 | "results = minimize(log_likelihood, \n", 93 | " x0 = np.array([1.0, 1.0]),\n", 94 | " method=None, \n", 95 | " args=(T, E),\n", 96 | " bounds=((0.00001, None), (0.00001, None)))\n", 97 | "\n", 98 | "print(results)" 99 | ] 100 | }, 101 | { 102 | "cell_type": "code", 103 | "execution_count": null, 104 | "metadata": {}, 105 | "outputs": [], 106 | "source": [ 107 | "# why did this fail? Note the -inf." 108 | ] 109 | }, 110 | { 111 | "cell_type": "code", 112 | "execution_count": null, 113 | "metadata": {}, 114 | "outputs": [], 115 | "source": [ 116 | "def negative_log_likelihood(params, t, e):\n", 117 | " return -log_likelihood(params, t, e)\n", 118 | "\n", 119 | "results = minimize( # what goes here?, \n", 120 | " x0 = np.array([1.0, 1.0]),\n", 121 | " method=None, \n", 122 | " args=(T, E),\n", 123 | " bounds=((0.00001, None), (0.00001, None)))\n", 124 | "\n", 125 | "print(results)" 126 | ] 127 | }, 128 | { 129 | "cell_type": "code", 130 | "execution_count": null, 131 | "metadata": {}, 132 | "outputs": [], 133 | "source": [ 134 | "t = np.linspace(0, 10, 100)\n", 135 | "plt.plot(t, np.exp(-cumulative_hazard(results.x, t)))\n", 136 | "plt.ylim(0, 1)\n", 137 | "plt.title(\"Estimated survival function\")" 138 | ] 139 | }, 140 | { 141 | "cell_type": "markdown", 142 | "metadata": {}, 143 | "source": [ 144 | "Let's move to part 3 - automatic differentiation! " 145 | ] 146 | } 147 | ], 148 | "metadata": { 149 | "kernelspec": { 150 | "display_name": "Python 3", 151 | "language": "python", 152 | "name": "python3" 153 | }, 154 | "language_info": { 155 | "codemirror_mode": { 156 | "name": "ipython", 157 | "version": 3 158 | }, 159 | "file_extension": ".py", 160 | "mimetype": "text/x-python", 161 | "name": "python", 162 | "nbconvert_exporter": "python", 163 | "pygments_lexer": "ipython3", 164 | "version": "3.7.3" 165 | } 166 | }, 167 | "nbformat": 4, 168 | "nbformat_minor": 2 169 | } 170 | -------------------------------------------------------------------------------- /full_notebooks/Part 5.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "### Hessians to compute variances\n", 8 | "\n", 9 | " - We have our point estimates, but now we want variances of the estimates\n", 10 | " - We can approximate them (under some asymptotic conditions) using the _Hessian at the (negative) MLE minimum_\n", 11 | " - Think of the the Hessian as a matrix representing the curvature at the minimum point. If we move slightly away from that minimum, how much change do we see in the likelihood?\n", 12 | " - high curvature (think of a tight peak) => low variance of estimate.\n", 13 | " - low curvature (think of a broad hill) => high variance of estimate.\n", 14 | "\n", 15 | "To extract the parameter estimates, we invert the Hessian at the MLE, $\\hat{\\theta}$, and take the diagonal:\n", 16 | "\n", 17 | "$$\\text{Var}(\\hat{\\theta}_i) = \\text{diag}(H(\\hat{\\theta})^{-1})_i $$\n", 18 | "\n", 19 | "The Hessian is the matrix of all second derivatives. `autograd` has this built in:" 20 | ] 21 | }, 22 | { 23 | "cell_type": "code", 24 | "execution_count": 1, 25 | "metadata": {}, 26 | "outputs": [], 27 | "source": [ 28 | "from autograd import numpy as np\n", 29 | "from autograd import elementwise_grad, value_and_grad, hessian\n", 30 | "from scipy.optimize import minimize" 31 | ] 32 | }, 33 | { 34 | "cell_type": "code", 35 | "execution_count": 2, 36 | "metadata": {}, 37 | "outputs": [], 38 | "source": [ 39 | "T = (np.random.exponential(size=1000)/1.5) ** 2.3\n", 40 | "E = np.random.binomial(1, 0.95, size=1000)" 41 | ] 42 | }, 43 | { 44 | "cell_type": "code", 45 | "execution_count": 3, 46 | "metadata": {}, 47 | "outputs": [ 48 | { 49 | "name": "stdout", 50 | "output_type": "stream", 51 | "text": [ 52 | " fun: 249.99413038558657\n", 53 | " hess_inv: <2x2 LbfgsInvHessProduct with dtype=float64>\n", 54 | " jac: array([-0.00046244, 0.00022406])\n", 55 | " message: b'CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH'\n", 56 | " nfev: 11\n", 57 | " nit: 7\n", 58 | " status: 0\n", 59 | " success: True\n", 60 | " x: array([0.46459167, 0.44953271])\n" 61 | ] 62 | } 63 | ], 64 | "source": [ 65 | "# seen all this...\n", 66 | "def cumulative_hazard(params, t):\n", 67 | " lambda_, rho_ = params\n", 68 | " return (t / lambda_) ** rho_\n", 69 | "\n", 70 | "hazard = elementwise_grad(cumulative_hazard, argnum=1)\n", 71 | "\n", 72 | "def log_hazard(params, t):\n", 73 | " return np.log(hazard(params, t))\n", 74 | "\n", 75 | "def log_likelihood(params, t, e):\n", 76 | " return np.sum(e * log_hazard(params, t)) - np.sum(cumulative_hazard(params, t))\n", 77 | "\n", 78 | "def negative_log_likelihood(params, t, e):\n", 79 | " return -log_likelihood(params, t, e)\n", 80 | "\n", 81 | "from autograd import value_and_grad\n", 82 | "\n", 83 | "results = minimize(\n", 84 | " value_and_grad(negative_log_likelihood), \n", 85 | " x0 = np.array([1.0, 1.0]),\n", 86 | " method=None, \n", 87 | " args=(T, E),\n", 88 | " jac=True,\n", 89 | " bounds=((0.00001, None), (0.00001, None)))\n", 90 | "\n", 91 | "print(results)" 92 | ] 93 | }, 94 | { 95 | "cell_type": "code", 96 | "execution_count": 4, 97 | "metadata": {}, 98 | "outputs": [ 99 | { 100 | "data": { 101 | "text/plain": [ 102 | "array([[ 887.54177234, -758.67183105],\n", 103 | " [-758.67183105, 8295.5360413 ]])" 104 | ] 105 | }, 106 | "execution_count": 4, 107 | "metadata": {}, 108 | "output_type": "execute_result" 109 | } 110 | ], 111 | "source": [ 112 | "estimates_ = results.x\n", 113 | "\n", 114 | "# Note: this will produce a matrix\n", 115 | "hessian(negative_log_likelihood)(estimates_, T, E)" 116 | ] 117 | }, 118 | { 119 | "cell_type": "code", 120 | "execution_count": 5, 121 | "metadata": {}, 122 | "outputs": [ 123 | { 124 | "name": "stdout", 125 | "output_type": "stream", 126 | "text": [ 127 | "[0.00122226 0.00013077]\n" 128 | ] 129 | } 130 | ], 131 | "source": [ 132 | "H = hessian(negative_log_likelihood)(estimates_, T, E)\n", 133 | "variance_ = np.diag(np.linalg.inv(H))\n", 134 | "print(variance_)" 135 | ] 136 | }, 137 | { 138 | "cell_type": "code", 139 | "execution_count": 6, 140 | "metadata": {}, 141 | "outputs": [ 142 | { 143 | "name": "stdout", 144 | "output_type": "stream", 145 | "text": [ 146 | "[0.03496082 0.01143546]\n" 147 | ] 148 | } 149 | ], 150 | "source": [ 151 | "std_error = np.sqrt(variance_)\n", 152 | "print(std_error)" 153 | ] 154 | }, 155 | { 156 | "cell_type": "code", 157 | "execution_count": 7, 158 | "metadata": {}, 159 | "outputs": [ 160 | { 161 | "name": "stdout", 162 | "output_type": "stream", 163 | "text": [ 164 | "lambda_ 0.465 (±0.03)\n", 165 | "rho_ 0.450 (±0.01)\n" 166 | ] 167 | } 168 | ], 169 | "source": [ 170 | "print(\"lambda_ %.3f (±%.2f)\" % (estimates_[0], std_error[0]))\n", 171 | "print(\"rho_ %.3f (±%.2f)\" % (estimates_[1], std_error[1]))" 172 | ] 173 | }, 174 | { 175 | "cell_type": "markdown", 176 | "metadata": {}, 177 | "source": [ 178 | "From here you can construct confidence intervals (CIs), and such. Let's move to Part 6, which is where you connect these abstract parameters to business logic. " 179 | ] 180 | } 181 | ], 182 | "metadata": { 183 | "kernelspec": { 184 | "display_name": "Python 3", 185 | "language": "python", 186 | "name": "python3" 187 | }, 188 | "language_info": { 189 | "codemirror_mode": { 190 | "name": "ipython", 191 | "version": 3 192 | }, 193 | "file_extension": ".py", 194 | "mimetype": "text/x-python", 195 | "name": "python", 196 | "nbconvert_exporter": "python", 197 | "pygments_lexer": "ipython3", 198 | "version": "3.7.3" 199 | } 200 | }, 201 | "nbformat": 4, 202 | "nbformat_minor": 2 203 | } 204 | -------------------------------------------------------------------------------- /tutorial_notebooks/Part 3.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "- Currently we are using a naive minimization algorithm. It will approximate the gradient and move in that direction. This requires lots of function calls. \n", 8 | "- We have no way to get the Hessian - which is needed for the variance estimates of our parameters. " 9 | ] 10 | }, 11 | { 12 | "cell_type": "markdown", 13 | "metadata": {}, 14 | "source": [ 15 | "### Automatic differentiation - how to differentiate an algorithm\n", 16 | "\n", 17 | "Symbolic differentiation is what we learned in school, and software like SymPy, Wolfram and Mathematica do this well. But I want to differentiate the following:" 18 | ] 19 | }, 20 | { 21 | "cell_type": "code", 22 | "execution_count": null, 23 | "metadata": {}, 24 | "outputs": [], 25 | "source": [ 26 | "%matplotlib inline\n", 27 | "import numpy as np\n", 28 | "from matplotlib import pyplot as plt\n", 29 | "\n", 30 | "def f(x):\n", 31 | " y = 0.\n", 32 | " for i in range(100):\n", 33 | " y = np.sin(y + x)\n", 34 | " return y\n", 35 | "\n", 36 | "x = np.linspace(-2, 2, 250)\n", 37 | "plt.plot(x, [f(_) for _ in x]);" 38 | ] 39 | }, 40 | { 41 | "cell_type": "markdown", 42 | "metadata": {}, 43 | "source": [ 44 | "Mathematically, it's something like:\n", 45 | "\n", 46 | "$$f(x) = \\sin(x + \\sin(x + \\sin(x + ...\\sin(x)))...)$$\n", 47 | "\n", 48 | "Good luck differentiating that and getting a nice closed form. If this is not complicated enough for you, feel free to add some `if` statements. \n", 49 | "\n", 50 | "We can use `autograd`, an automatical diff package, to compute _exact, pointwise_ derivatives." 51 | ] 52 | }, 53 | { 54 | "cell_type": "code", 55 | "execution_count": null, 56 | "metadata": {}, 57 | "outputs": [], 58 | "source": [ 59 | "from autograd import grad\n", 60 | "from autograd import numpy as np\n", 61 | "\n", 62 | "def f(x):\n", 63 | " y = 0.\n", 64 | " for i in range(100):\n", 65 | " # this np. is now from autograd - important! \n", 66 | " y = np.sin(y + x)\n", 67 | " return y\n", 68 | "\n", 69 | "grad_f = grad(f)\n", 70 | "grad_f(1.)\n", 71 | "# magic! " 72 | ] 73 | }, 74 | { 75 | "cell_type": "markdown", 76 | "metadata": {}, 77 | "source": [ 78 | "### We just differentiated an algorithm. Pick your brain pieces off the walls please. \n", 79 | "\n", 80 | "At a high level, autograd has a lookup of simple functions and their derivatives, and uses repeated use of the chain rule + calculus rules \n", 81 | "\n", 82 | "Of course, you can string together these pointwise values into a function:" 83 | ] 84 | }, 85 | { 86 | "cell_type": "code", 87 | "execution_count": null, 88 | "metadata": {}, 89 | "outputs": [], 90 | "source": [ 91 | "x = np.linspace(-2, 2, 250)\n", 92 | "plt.plot(x, [grad_f(x_) for x_ in x])" 93 | ] 94 | }, 95 | { 96 | "cell_type": "markdown", 97 | "metadata": {}, 98 | "source": [ 99 | "To use this in our optimization routines:\n", 100 | "\n", 101 | " - we can automatically compute gradients that the optimizer can use. \n", 102 | " - Hessians can be exactly calculated (we will do this later)\n" 103 | ] 104 | }, 105 | { 106 | "cell_type": "code", 107 | "execution_count": null, 108 | "metadata": {}, 109 | "outputs": [], 110 | "source": [ 111 | "T = (np.random.exponential(size=1000)/1.5) ** 2.3\n", 112 | "E = np.random.binomial(1, 0.95, size=1000)" 113 | ] 114 | }, 115 | { 116 | "cell_type": "code", 117 | "execution_count": null, 118 | "metadata": {}, 119 | "outputs": [], 120 | "source": [ 121 | "def cumulative_hazard(params, t):\n", 122 | " lambda_, rho_ = params\n", 123 | " return (t / lambda_) ** rho_\n", 124 | "\n", 125 | "def log_hazard(params, t):\n", 126 | " lambda_, rho_ = params\n", 127 | " return np.log(rho_) - np.log(lambda_) + (rho_ - 1) * (np.log(t) - np.log(lambda_))\n", 128 | "\n", 129 | "def log_likelihood(params, t, e):\n", 130 | " return np.sum(e * log_hazard(params, t)) - np.sum(cumulative_hazard(params, t))\n", 131 | "\n", 132 | "def negative_log_likelihood(params, t, e):\n", 133 | " return -log_likelihood(params, t, e)\n" 134 | ] 135 | }, 136 | { 137 | "cell_type": "markdown", 138 | "metadata": {}, 139 | "source": [ 140 | "So `grad(negative_log_likelihood)` will find the gradient of `negative_log_likelihood` with respect to the first parameter, `params`. " 141 | ] 142 | }, 143 | { 144 | "cell_type": "code", 145 | "execution_count": null, 146 | "metadata": {}, 147 | "outputs": [], 148 | "source": [ 149 | "grad_negative_log_likelihood = # what goes here?\n", 150 | "\n", 151 | "print(grad_negative_log_likelihood(np.array([1., 1.]), T, E))" 152 | ] 153 | }, 154 | { 155 | "cell_type": "code", 156 | "execution_count": null, 157 | "metadata": {}, 158 | "outputs": [], 159 | "source": [ 160 | "from scipy.optimize import minimize\n", 161 | "\n", 162 | "results = minimize(negative_log_likelihood, \n", 163 | " x0 = np.array([1.0, 1.0]),\n", 164 | " method=None, \n", 165 | " args=(T, E),\n", 166 | " jac=grad_negative_log_likelihood,\n", 167 | " bounds=((0.00001, None), (0.00001, None)))\n", 168 | "\n", 169 | "print(results)" 170 | ] 171 | }, 172 | { 173 | "cell_type": "code", 174 | "execution_count": null, 175 | "metadata": {}, 176 | "outputs": [], 177 | "source": [ 178 | "from autograd import value_and_grad\n", 179 | "\n", 180 | "results = minimize(\n", 181 | " # fill this in. \n", 182 | " x0 = np.array([1.0, 1.0]),\n", 183 | " method=None, \n", 184 | " args=(T, E),\n", 185 | " jac=True, # notice this set to True now.\n", 186 | " bounds=((0.00001, None), (0.00001, None)))\n" 187 | ] 188 | }, 189 | { 190 | "cell_type": "code", 191 | "execution_count": null, 192 | "metadata": {}, 193 | "outputs": [], 194 | "source": [ 195 | "results" 196 | ] 197 | }, 198 | { 199 | "cell_type": "markdown", 200 | "metadata": {}, 201 | "source": [ 202 | "Let's continue this analytical-train 🚂 to Part 4! " 203 | ] 204 | } 205 | ], 206 | "metadata": { 207 | "kernelspec": { 208 | "display_name": "Python 3", 209 | "language": "python", 210 | "name": "python3" 211 | }, 212 | "language_info": { 213 | "codemirror_mode": { 214 | "name": "ipython", 215 | "version": 3 216 | }, 217 | "file_extension": ".py", 218 | "mimetype": "text/x-python", 219 | "name": "python", 220 | "nbconvert_exporter": "python", 221 | "pygments_lexer": "ipython3", 222 | "version": "3.7.3" 223 | } 224 | }, 225 | "nbformat": 4, 226 | "nbformat_minor": 2 227 | } 228 | -------------------------------------------------------------------------------- /full_notebooks/Part 4.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "Let's try generalizing our model now. For survival analysis, it is very useful to think about cumulative hazards - that is, modelling the cumulative hazard is typically easiest. " 8 | ] 9 | }, 10 | { 11 | "cell_type": "code", 12 | "execution_count": 1, 13 | "metadata": {}, 14 | "outputs": [], 15 | "source": [ 16 | "# Note: we are shifting the data 4 units to the right.\n", 17 | "T = (np.random.exponential(size=1000)/1.5) ** 2.3 + 4\n", 18 | "E = np.random.binomial(1, 0.95, size=1000)" 19 | ] 20 | }, 21 | { 22 | "cell_type": "code", 23 | "execution_count": 2, 24 | "metadata": {}, 25 | "outputs": [], 26 | "source": [ 27 | "from autograd import numpy as np\n", 28 | "\n", 29 | "def cumulative_hazard(params, t):\n", 30 | " lambda_, rho_, mu = params\n", 31 | " return ((t - mu) / lambda_) ** rho_\n", 32 | "\n", 33 | "def log_hazard(params, t):\n", 34 | " lambda_, rho_, mu = params\n", 35 | " return ... # this could get arbitrarly complicated." 36 | ] 37 | }, 38 | { 39 | "cell_type": "code", 40 | "execution_count": 3, 41 | "metadata": {}, 42 | "outputs": [], 43 | "source": [ 44 | "from autograd import elementwise_grad\n", 45 | "hazard = elementwise_grad(cumulative_hazard, argnum=1)\n", 46 | "\n", 47 | "def log_hazard(params, t):\n", 48 | " return np.log(hazard(params, t))" 49 | ] 50 | }, 51 | { 52 | "cell_type": "code", 53 | "execution_count": 4, 54 | "metadata": {}, 55 | "outputs": [ 56 | { 57 | "data": { 58 | "text/plain": [ 59 | "array([-0.69314718, 0. , 0.40546511])" 60 | ] 61 | }, 62 | "execution_count": 4, 63 | "metadata": {}, 64 | "output_type": "execute_result" 65 | } 66 | ], 67 | "source": [ 68 | "log_hazard(np.array([2., 2. , 0.]), np.array([1,2,3]))" 69 | ] 70 | }, 71 | { 72 | "cell_type": "code", 73 | "execution_count": 5, 74 | "metadata": {}, 75 | "outputs": [ 76 | { 77 | "name": "stdout", 78 | "output_type": "stream", 79 | "text": [ 80 | " fun: 341.1505998865307\n", 81 | " hess_inv: <3x3 LbfgsInvHessProduct with dtype=float64>\n", 82 | " jac: array([ 8.74288685e-06, -7.51244260e-04, -4.29427489e+04])\n", 83 | " message: b'CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH'\n", 84 | " nfev: 30\n", 85 | " nit: 20\n", 86 | " status: 0\n", 87 | " success: True\n", 88 | " x: array([0.49171791, 0.48311375, 3.99900001])\n" 89 | ] 90 | } 91 | ], 92 | "source": [ 93 | "# same as previous\n", 94 | "\n", 95 | "from autograd import value_and_grad\n", 96 | "from scipy.optimize import minimize\n", 97 | "\n", 98 | "def log_likelihood(params, t, e):\n", 99 | " return np.sum(e * log_hazard(params, t)) - np.sum(cumulative_hazard(params, t))\n", 100 | "\n", 101 | "def negative_log_likelihood(params, t, e):\n", 102 | " return -log_likelihood(params, t, e)\n", 103 | "\n", 104 | "results = minimize(\n", 105 | " value_and_grad(negative_log_likelihood), \n", 106 | " x0 = np.array([1.0, 1.0, 0.0]),\n", 107 | " method=None, \n", 108 | " args=(T, E),\n", 109 | " jac=True,\n", 110 | " bounds=((0.00001, None), (0.00001, None), (None, np.min(T)-0.001)))\n", 111 | "\n", 112 | "print(results)" 113 | ] 114 | }, 115 | { 116 | "cell_type": "markdown", 117 | "metadata": {}, 118 | "source": [ 119 | "So long as you are good at \"parameter accounting\", then you could create arbitrarly complicated survival models simply by specifying the hazard. \n", 120 | "\n", 121 | "Extending to including covariates is straightforward too, with some modifications to the code. Here's a simple model of the cumulative hazard:\n", 122 | "\n", 123 | "$$H(t; x) = \\left(\\frac{t}{\\lambda(x)}\\right)^\\rho $$\n", 124 | "$$ \\lambda(x) = \\beta_1 x_1 + \\beta_2 x_2 $$" 125 | ] 126 | }, 127 | { 128 | "cell_type": "code", 129 | "execution_count": 6, 130 | "metadata": {}, 131 | "outputs": [], 132 | "source": [ 133 | "from scipy.stats import weibull_min\n", 134 | "\n", 135 | "# create some regression data. \n", 136 | "N = 10000\n", 137 | "X = 0.1 * np.random.normal(size=(N, 2))\n", 138 | "T = np.exp(2 * X[:, 0] + -2 * X[:, 1]) * weibull_min.rvs(1, scale=1, loc=0, size=N)\n", 139 | "E = np.random.binomial(1, 0.99, size=N)" 140 | ] 141 | }, 142 | { 143 | "cell_type": "code", 144 | "execution_count": 7, 145 | "metadata": {}, 146 | "outputs": [ 147 | { 148 | "name": "stdout", 149 | "output_type": "stream", 150 | "text": [ 151 | " fun: 10012.505418738723\n", 152 | " hess_inv: <3x3 LbfgsInvHessProduct with dtype=float64>\n", 153 | " jac: array([ 5.64816887e-06, -4.95439586e-05, 2.98904267e-03])\n", 154 | " message: b'CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH'\n", 155 | " nfev: 10\n", 156 | " nit: 8\n", 157 | " status: 0\n", 158 | " success: True\n", 159 | " x: array([ 2.0380495 , -2.14856494, 0.99908182])\n" 160 | ] 161 | } 162 | ], 163 | "source": [ 164 | "def cumulative_hazard(params, t, X):\n", 165 | " log_lambda_coefs_, rho_ = params[:2], params[-1]\n", 166 | " lambda_ = np.exp(np.dot(X, log_lambda_coefs_))\n", 167 | " return (t / lambda_) ** rho_\n", 168 | "\n", 169 | "# these functions are almost identical to above, \n", 170 | "# expect they have an additional argument, X\n", 171 | "hazard = elementwise_grad(cumulative_hazard, argnum=1)\n", 172 | "\n", 173 | "def log_hazard(params, t, X):\n", 174 | " return np.log(hazard(params, t, X))\n", 175 | "\n", 176 | "def log_likelihood(params, t, e, X):\n", 177 | " return np.sum(e * log_hazard(params, t, X)) - np.sum(cumulative_hazard(params, t, X))\n", 178 | "\n", 179 | "def negative_log_likelihood(params, t, e, X):\n", 180 | " return -log_likelihood(params, t, e, X)\n", 181 | "\n", 182 | "results = minimize(\n", 183 | " value_and_grad(negative_log_likelihood), \n", 184 | " x0 = np.array([0.0, 0.0, 1.0]),\n", 185 | " method=None, \n", 186 | " args=(T, E, X),\n", 187 | " jac=True,\n", 188 | " bounds=((None, None), (None, None), (0.00001, None)))\n", 189 | "\n", 190 | "print(results)" 191 | ] 192 | }, 193 | { 194 | "cell_type": "markdown", 195 | "metadata": {}, 196 | "source": [ 197 | "Great! Let's move onto part 5, how to estimate the _variances_ of the parameters." 198 | ] 199 | }, 200 | { 201 | "cell_type": "code", 202 | "execution_count": null, 203 | "metadata": {}, 204 | "outputs": [], 205 | "source": [] 206 | } 207 | ], 208 | "metadata": { 209 | "kernelspec": { 210 | "display_name": "Python 3", 211 | "language": "python", 212 | "name": "python3" 213 | }, 214 | "language_info": { 215 | "codemirror_mode": { 216 | "name": "ipython", 217 | "version": 3 218 | }, 219 | "file_extension": ".py", 220 | "mimetype": "text/x-python", 221 | "name": "python", 222 | "nbconvert_exporter": "python", 223 | "pygments_lexer": "ipython3", 224 | "version": "3.7.3" 225 | } 226 | }, 227 | "nbformat": 4, 228 | "nbformat_minor": 2 229 | } 230 | -------------------------------------------------------------------------------- /tutorial_notebooks/Part 7.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "### Lifetime value example \n", 8 | "\n", 9 | "\n", 10 | "Suppose we have a subscription business that has monthly churn, and we'd like to know an estimate of LTV (lifetime value) and build confidence intervals for it. \n", 11 | "\n", 12 | "\n", 13 | "Subscription businesses have a very predictable churn profile (it looks piecewise) but the rates are unknown. \n", 14 | "\n", 15 | "\n", 16 | "\n", 17 | "\n", 18 | "\n", 19 | "We'll use a piecewise-constant hazard model with known breakpoints, $\\tau$.\n", 20 | "$$ \n", 21 | "h(t) = \\begin{cases}\n", 22 | " \\lambda_0 & \\text{if $t \\le \\tau_0$} \\\\\n", 23 | " \\lambda_1 & \\text{if $\\tau_0 < t \\le \\tau_1$} \\\\\n", 24 | " \\lambda_2 & \\text{if $\\tau_1 < t \\le \\tau_2$} \\\\\n", 25 | " ...\n", 26 | " \\end{cases}\n", 27 | "$$\n", 28 | "\n" 29 | ] 30 | }, 31 | { 32 | "cell_type": "code", 33 | "execution_count": null, 34 | "metadata": {}, 35 | "outputs": [], 36 | "source": [ 37 | "%matplotlib inline\n", 38 | "from autograd import numpy as np\n", 39 | "from autograd import elementwise_grad, value_and_grad, hessian\n", 40 | "from scipy.optimize import minimize\n", 41 | "\n", 42 | "df = pd.read_csv(\"../churn_data.csv\")\n", 43 | "T = df['T'].values\n", 44 | "E = df['E'].values" 45 | ] 46 | }, 47 | { 48 | "cell_type": "code", 49 | "execution_count": null, 50 | "metadata": {}, 51 | "outputs": [], 52 | "source": [ 53 | "breakpoints = np.array([28, 33, 58, 63, 88, 93, 117, 122, 148, 153])\n", 54 | "\n", 55 | "def cumulative_hazard(params, times):\n", 56 | " # this is NumPy black magic to get piecewise hazards, let's chat after. \n", 57 | " times = np.atleast_1d(times)\n", 58 | " n = times.shape[0]\n", 59 | " times = times.reshape((n, 1))\n", 60 | " M = np.minimum(np.tile(breakpoints, (n, 1)), times)\n", 61 | " M = np.hstack([M[:, tuple([0])], np.diff(M, axis=1)])\n", 62 | " return np.dot(M, params)\n", 63 | "\n", 64 | "hazard = elementwise_grad(cumulative_hazard, argnum=1)\n", 65 | "\n", 66 | "def survival_function(params, t):\n", 67 | " return np.exp(-cumulative_hazard(params, t))\n", 68 | "\n", 69 | "def log_hazard(params, t):\n", 70 | " return np.log(np.clip(hazard(params, t), 1e-25, np.inf))\n", 71 | "\n", 72 | "def log_likelihood(params, t, e):\n", 73 | " return np.sum(e * log_hazard(params, t)) - np.sum(cumulative_hazard(params, t))\n", 74 | "\n", 75 | "def negative_log_likelihood(params, t, e):\n", 76 | " return -log_likelihood(params, t, e)\n", 77 | "\n", 78 | "from autograd import value_and_grad\n", 79 | "\n", 80 | "results = minimize(\n", 81 | " value_and_grad(negative_log_likelihood), \n", 82 | " x0 = np.ones(len(breakpoints)),\n", 83 | " method=None, \n", 84 | " args=(T, E),\n", 85 | " jac=True,\n", 86 | " bounds=[(0.0001, None)] * len(breakpoints)\n", 87 | ")\n", 88 | "\n", 89 | "print(results)\n", 90 | "estimates_ = results.x\n", 91 | "H = hessian(negative_log_likelihood)(estimates_, T, E)\n", 92 | "variance_matrix_ = np.linalg.inv(H)" 93 | ] 94 | }, 95 | { 96 | "cell_type": "code", 97 | "execution_count": null, 98 | "metadata": {}, 99 | "outputs": [], 100 | "source": [ 101 | "t = np.linspace(.001, 150, 100)\n", 102 | "plt.plot(t, survival_function(estimates_, t))\n", 103 | "plt.ylim(0.5, 1)\n", 104 | "plt.title(\"\"\"Estimated survival function using \\npiecewise hazards\"\"\");" 105 | ] 106 | }, 107 | { 108 | "cell_type": "markdown", 109 | "metadata": {}, 110 | "source": [ 111 | "On day 30, we charge users \\\\$10, and on every 30 days after that, we charge \\\\$20. What's the LTV, and CIs, at the end of day 120?" 112 | ] 113 | }, 114 | { 115 | "cell_type": "code", 116 | "execution_count": null, 117 | "metadata": {}, 118 | "outputs": [], 119 | "source": [ 120 | "def LTV_120(params):\n", 121 | " # think about how complicated the gradient of this function is. Now imagine an even more\n", 122 | " # complicated function.\n", 123 | " # how do we implement this function?\n", 124 | " return ..." 125 | ] 126 | }, 127 | { 128 | "cell_type": "code", 129 | "execution_count": null, 130 | "metadata": {}, 131 | "outputs": [], 132 | "source": [ 133 | "ltv_ = LTV_120(estimates_)\n", 134 | "print(\"LTV estimate: \", ltv_)" 135 | ] 136 | }, 137 | { 138 | "cell_type": "code", 139 | "execution_count": null, 140 | "metadata": {}, 141 | "outputs": [], 142 | "source": [ 143 | "from autograd import grad\n", 144 | "var_ltv_ = # what goes here?\n", 145 | "print(\"Variance LTV estimate:\", var_ltv_)" 146 | ] 147 | }, 148 | { 149 | "cell_type": "code", 150 | "execution_count": null, 151 | "metadata": {}, 152 | "outputs": [], 153 | "source": [ 154 | "std_ltv = np.sqrt(var_ltv_)\n", 155 | "print(\"Estimated LTV at day 120: ($%.2f, $%.2f)\" % (ltv_ - 1.96 * std_ltv, ltv_ + 1.96 * std_ltv))" 156 | ] 157 | }, 158 | { 159 | "cell_type": "markdown", 160 | "metadata": {}, 161 | "source": [ 162 | "From here, we can compute p-values, scenario analysis, sensitvity analysis, etc. \n", 163 | "\n", 164 | "Let's continue this analytical train to Part 8. " 165 | ] 166 | }, 167 | { 168 | "cell_type": "markdown", 169 | "metadata": {}, 170 | "source": [ 171 | "### Bonus, if time permits and Cameron is talking too fast. \n", 172 | "\n", 173 | "In the above model, we are not suggesting to the model much apriori information about this \"predictable\" process. For example, suppose we want a model that gives \"inbetween\" period rates to be close to each other, and likewise for the \"jump\" rates. " 174 | ] 175 | }, 176 | { 177 | "cell_type": "code", 178 | "execution_count": null, 179 | "metadata": {}, 180 | "outputs": [], 181 | "source": [ 182 | "\n", 183 | "def negative_log_likelihood(params, t, e):\n", 184 | " return -log_likelihood(params, t, e) + 1e12 * (np.var(params[::2]) + np.var(params[1::2]))\n", 185 | "\n", 186 | "from autograd import value_and_grad\n", 187 | "\n", 188 | "results = minimize(\n", 189 | " value_and_grad(negative_log_likelihood), \n", 190 | " x0 = np.ones(len(breakpoints)),\n", 191 | " method=None, \n", 192 | " args=(T, E),\n", 193 | " jac=True,\n", 194 | " bounds=[(0.0001, None)] * len(breakpoints)\n", 195 | ")\n", 196 | "\n", 197 | "print(results)\n", 198 | "estimates_ = results.x" 199 | ] 200 | }, 201 | { 202 | "cell_type": "code", 203 | "execution_count": null, 204 | "metadata": {}, 205 | "outputs": [], 206 | "source": [ 207 | "t = np.linspace(.001, 150, 100)\n", 208 | "plt.plot(t, hazard(estimates_, t))\n", 209 | "plt.title(\"\"\"Estimated hazard function using \\npiecewise hazards \\nand penalizing variance\"\"\");" 210 | ] 211 | }, 212 | { 213 | "cell_type": "markdown", 214 | "metadata": {}, 215 | "source": [ 216 | "### Why do this?\n", 217 | "\n", 218 | "1) From a Bayesian perspective, this is a way to add prior information into a model. Note that if we have _lots_ of observations, the prior becomes less relevant (just like a traditional Bayesian model). That's good. *We are again using the likelihood to \"add\" information to our system.*\n", 219 | "\n", 220 | "2) When we have low data sizes, we can \"borrow\" information between periods. That is, deaths in the earlier \"inbetween\" periods can inform the rate in the later \"inbetween\" periods - thus we can do better inference. \n" 221 | ] 222 | }, 223 | { 224 | "cell_type": "code", 225 | "execution_count": null, 226 | "metadata": {}, 227 | "outputs": [], 228 | "source": [] 229 | } 230 | ], 231 | "metadata": { 232 | "kernelspec": { 233 | "display_name": "Python 3", 234 | "language": "python", 235 | "name": "python3" 236 | }, 237 | "language_info": { 238 | "codemirror_mode": { 239 | "name": "ipython", 240 | "version": 3 241 | }, 242 | "file_extension": ".py", 243 | "mimetype": "text/x-python", 244 | "name": "python", 245 | "nbconvert_exporter": "python", 246 | "pygments_lexer": "ipython3", 247 | "version": "3.7.3" 248 | } 249 | }, 250 | "nbformat": 4, 251 | "nbformat_minor": 2 252 | } 253 | -------------------------------------------------------------------------------- /churn_data.csv: -------------------------------------------------------------------------------- 1 | T,E 2 | 29.841029048536946,True 3 | 19.235401904818797,False 4 | 59.770381823177594,False 5 | 11.614955147622757,False 6 | 90.25117783086436,True 7 | 749.6275796691885,True 8 | 60.08754635652852,False 9 | 28.90093653801583,False 10 | 940.3119571600598,True 11 | 7.562715491070635,True 12 | 118.43077982868809,True 13 | 3332.728063243511,True 14 | 189.96405909716992,True 15 | 28.748819980693927,True 16 | 968.6265818546981,True 17 | 132.36990299355205,True 18 | 1724.1595593213137,False 19 | 863.6613559607827,False 20 | 90.27796976654776,True 21 | 1255.085346464945,False 22 | 31.49191408433294,True 23 | 395.97404501920346,False 24 | 31.553875226891016,False 25 | 23.49417857416158,True 26 | 2835.580085892919,False 27 | 32.44335789042166,False 28 | 68.30811672729322,False 29 | 784.4015939784244,True 30 | 563.4818707270822,True 31 | 9.446233244734277,False 32 | 1922.7808182783579,True 33 | 59.91405885500417,True 34 | 62.65252302254179,True 35 | 2873.564656452592,True 36 | 92.27109338068449,False 37 | 32.24084331897224,True 38 | 108.01963297929811,True 39 | 29.254330110587077,True 40 | 29.11795376102093,False 41 | 29.451671455076276,False 42 | 90.26881947451027,True 43 | 28.004874458768644,False 44 | 30.604829625414897,True 45 | 32.3277934123205,True 46 | 29.356775835380176,True 47 | 32.03583045556982,True 48 | 89.78256964723387,True 49 | 939.168113042314,False 50 | 28.176531862656383,True 51 | 779.7374927049406,False 52 | 1790.532566599261,True 53 | 62.9116808868676,True 54 | 124.69023672658149,True 55 | 31.083656652844866,True 56 | 321.81875983654834,True 57 | 630.9572160877894,True 58 | 546.6190209562415,True 59 | 1448.4932757561905,False 60 | 62.51850599512492,False 61 | 33.26108179770319,False 62 | 28.66521511656203,True 63 | 26.272571595435924,False 64 | 985.173969633019,False 65 | 28.203739924596107,True 66 | 59.44707290949923,True 67 | 28.779335379771336,True 68 | 28.476531912143205,False 69 | 1578.1807820425254,True 70 | 134.87705657574662,True 71 | 31.791180860200363,True 72 | 6.148491397051444,False 73 | 713.252244360728,False 74 | 32.448526681068124,True 75 | 292.18566047515384,False 76 | 28.447860580933813,True 77 | 31.083449679637425,False 78 | 60.392531627946596,False 79 | 34.63291506948891,False 80 | 1133.6848814282503,True 81 | 150.85439283775074,False 82 | 1165.802146751959,False 83 | 149.33432336684953,False 84 | 137.61665572925799,False 85 | 12.360765836853446,False 86 | 62.56919594899421,True 87 | 28.599563525000455,False 88 | 30.878614720804382,True 89 | 30.256164058955605,True 90 | 32.58487398134688,True 91 | 92.41886070807126,True 92 | 90.32316649505057,True 93 | 117.58145284743394,True 94 | 1505.272104330573,True 95 | 117.95063812205166,True 96 | 297.2437226653309,True 97 | 411.0172179139138,False 98 | 92.98261840927553,True 99 | 28.61012322745767,False 100 | 976.8910944289486,True 101 | 444.09767881525437,True 102 | 435.1334568028524,False 103 | 2185.538716775944,True 104 | 372.93793412356393,True 105 | 809.0525994475074,True 106 | 52.4805684653793,False 107 | 800.5115018621061,True 108 | 1626.9182950270283,True 109 | 334.5397209108936,True 110 | 191.72313322161602,False 111 | 98.41733242800848,False 112 | 170.71687083039504,True 113 | 347.24774266805576,True 114 | 613.5903439941773,True 115 | 148.7450018496149,True 116 | 751.9941409779703,False 117 | 986.0587681768271,False 118 | 92.46900018182961,True 119 | 899.8886517260548,False 120 | 438.58489665719594,True 121 | 1116.851336057323,False 122 | 600.1230069922237,False 123 | 31.53305894584185,True 124 | 247.64864373987712,True 125 | 63.73124272268049,False 126 | 200.66930366822132,True 127 | 91.60840476518082,True 128 | 398.4399528344566,False 129 | 353.4874568524641,True 130 | 69.41325701154008,True 131 | 28.604879206858385,True 132 | 57.74457683977354,False 133 | 30.179114649562056,True 134 | 612.2629088420713,True 135 | 29.598545259676143,False 136 | 1716.3073628600605,False 137 | 553.2975424063993,False 138 | 149.74021056541386,True 139 | 32.995768058233864,False 140 | 1531.8749250403612,False 141 | 30.850209468589245,True 142 | 408.59544859893833,False 143 | 26.342719300027817,True 144 | 31.407225170072337,True 145 | 1475.104985532452,False 146 | 324.6811205957641,True 147 | 3439.2539275618897,False 148 | 299.41022594033996,False 149 | 492.37939307282824,False 150 | 68.90948474992942,False 151 | 119.21576326706419,False 152 | 6.973700888175082,True 153 | 2620.950555436224,True 154 | 91.53103890845695,True 155 | 107.1048385373958,True 156 | 28.810902324699335,False 157 | 2314.031840246363,False 158 | 310.92247499987053,False 159 | 31.479650900809762,False 160 | 55.51699371762213,True 161 | 31.80984893634224,True 162 | 184.6382429008999,False 163 | 62.73723025264546,False 164 | 148.99696447484754,True 165 | 887.1754712264874,False 166 | 196.83584772530287,False 167 | 427.3433658323352,True 168 | 2753.458893722906,False 169 | 136.8188291980203,False 170 | 28.914079234806046,True 171 | 698.9645986024577,False 172 | 31.27141394777689,True 173 | 203.65377347085428,True 174 | 4.3327531995502895,False 175 | 30.945513210097637,False 176 | 5.48598042955233,False 177 | 374.8720133788329,False 178 | 540.0857518113667,False 179 | 123.99368163008103,True 180 | 89.01356025997872,True 181 | 394.9495191633853,False 182 | 19.691647524455064,True 183 | 406.3154132069972,True 184 | 117.8643087504223,True 185 | 98.8964857892037,True 186 | 807.249143085234,True 187 | 62.91264939120062,False 188 | 62.15176496678456,True 189 | 1010.0544599395618,True 190 | 29.27456309505731,True 191 | 149.7806931036858,True 192 | 58.39244204449046,True 193 | 91.26084295206968,False 194 | 215.37902953868826,True 195 | 117.61039116455274,False 196 | 30.963267622746113,True 197 | 720.6964302218174,True 198 | 820.7373314284807,False 199 | 32.61440736362276,True 200 | 32.63141230087092,False 201 | 30.33259470392027,False 202 | 39.19473321798825,True 203 | 110.99148026040444,False 204 | 58.06040004870006,False 205 | 118.16394218698878,False 206 | 31.842571486053703,False 207 | 9.81985328633289,True 208 | 961.8136831461086,True 209 | 1266.906464552719,False 210 | 104.6645963515516,False 211 | 31.56097185290679,False 212 | 22.891483083489213,True 213 | 62.654741730532855,True 214 | 473.20561583972545,True 215 | 214.0201444103429,False 216 | 89.14198374251086,False 217 | 227.0120987897212,True 218 | 28.394427870489288,True 219 | 90.18054211586873,True 220 | 28.125216814654294,True 221 | 148.45968716703987,True 222 | 90.31923660257821,False 223 | 2255.8268923255496,True 224 | 1034.5646510893637,False 225 | 807.3097356075709,False 226 | 1603.7547845528545,True 227 | 221.07153302891612,True 228 | 2579.439437559638,False 229 | 2120.36741794898,True 230 | 31.741587158659435,True 231 | 954.4627824750396,False 232 | 164.69950412509678,True 233 | 80.5857391221066,True 234 | 21.10083461777976,True 235 | 1443.6234220514696,False 236 | 29.35193579570192,False 237 | 28.057468171915048,True 238 | 475.608396204843,True 239 | 546.0978176792827,False 240 | 325.02953557491423,True 241 | 1523.756442095222,False 242 | 83.25369404969273,True 243 | 578.009245369848,False 244 | 117.20207542263891,False 245 | 148.07961345815087,False 246 | 1442.365278811541,False 247 | 30.04194179371582,True 248 | 62.058705255388396,True 249 | 152.1760722280506,False 250 | 92.7553241468895,True 251 | 708.0482318892845,False 252 | 1589.5230520829907,True 253 | 28.30004886483892,False 254 | 4.0334080221294455,True 255 | 31.29405377561044,True 256 | 69.69065182112232,False 257 | 318.05648668862693,False 258 | 30.863486847323664,False 259 | 564.3355035425941,False 260 | 461.4719596585432,False 261 | 206.9450092564257,False 262 | 577.1772100859705,False 263 | 29.58126542033427,True 264 | 411.36520324609796,False 265 | 30.61422366319469,False 266 | 30.739898103811264,True 267 | 567.4935756621824,True 268 | 552.8358610187217,True 269 | 1279.4962673555715,True 270 | 136.53941076654567,False 271 | 3004.237457384779,True 272 | 1368.1628130176025,False 273 | 888.9815815512031,True 274 | 263.0927298567659,True 275 | 29.705600321910495,False 276 | 30.35605178208393,False 277 | 1172.5931676975956,False 278 | 722.3793437608534,True 279 | 31.06223145545288,False 280 | 137.16435632122347,True 281 | 618.9273476964263,False 282 | 32.483821236366126,False 283 | 92.97109909422593,False 284 | 999.3681457682009,False 285 | 1064.974913558552,True 286 | 2024.1524949005654,True 287 | 30.23847631335697,False 288 | 120.3732926595238,False 289 | 918.9182128620814,True 290 | 21.67766105026386,True 291 | 1142.7172242016482,True 292 | 28.10585581514121,True 293 | 28.82897217198856,True 294 | 224.83813125328362,False 295 | 1536.610029958752,True 296 | 40.19400438633373,True 297 | 137.49558447697635,False 298 | 119.45279116390394,True 299 | 89.65293313475654,False 300 | 22.80547689325,False 301 | 30.225926612870442,False 302 | 29.64766826576005,True 303 | 2012.3934733233766,False 304 | 691.9190935881635,True 305 | 2641.8490781192495,True 306 | 120.93508211429646,True 307 | 471.79022870163607,True 308 | 1878.9695654948487,True 309 | 841.4385065807377,False 310 | 89.98306323748685,True 311 | 1324.9297400828257,False 312 | 30.549343764260627,False 313 | 118.55800666900193,True 314 | 364.66792957445966,True 315 | 16.028133868158253,False 316 | 92.08787766755677,True 317 | 60.81710712791432,True 318 | 1107.9693656975833,True 319 | 91.26869648777769,True 320 | 151.67408840766043,True 321 | 99.97909899040584,True 322 | 130.39806533010642,False 323 | 632.7292347458686,True 324 | 59.30757384344846,False 325 | 1146.0653830081133,False 326 | 123.21193144570329,True 327 | 2440.873673052241,True 328 | 134.88843933216654,True 329 | 89.00594326137373,True 330 | 30.49414205531244,False 331 | 777.9221651731206,False 332 | 32.75220921140697,True 333 | 1573.8246629917016,True 334 | 224.1613955325626,True 335 | 2212.5401949274897,False 336 | 82.22304885844079,False 337 | 60.803213249014576,True 338 | 61.76239675895975,True 339 | 151.9396293649489,True 340 | 371.7817178852579,True 341 | 28.66973437826407,True 342 | 673.5701368830452,True 343 | 29.32597176797941,False 344 | 117.68361132347992,True 345 | 62.74323617326399,False 346 | 29.46178317327272,True 347 | 32.99882083217404,False 348 | 29.892012840414687,True 349 | 19.944767935656046,False 350 | 196.18937426434042,True 351 | 118.23669659679354,False 352 | 28.935682041541657,True 353 | 59.66881966129494,False 354 | 58.68122167985428,False 355 | 15.8390892904974,True 356 | 31.30092240174752,True 357 | 577.906820807727,False 358 | 28.818857058314034,True 359 | 31.955096429871283,True 360 | 28.97844136620908,True 361 | 243.35511591410196,False 362 | 26.851689753428396,False 363 | 525.9797495490236,True 364 | 28.4471811942955,True 365 | 114.13881367401355,False 366 | 28.241445531662013,False 367 | 572.24756757932,True 368 | 1424.8077314904915,False 369 | 120.14160851696685,True 370 | 59.256476556582484,True 371 | 90.68972192978461,False 372 | 61.3252838845291,True 373 | 4.838887222005952,True 374 | 30.81479330679898,True 375 | 719.9172763459198,True 376 | 28.084540871302668,False 377 | 162.95076478471904,True 378 | 702.2811957843027,False 379 | 28.72950638900236,False 380 | 59.53593473758157,True 381 | 150.7715656973741,False 382 | 117.7306047351688,True 383 | 315.16014401213215,False 384 | 281.7273857299291,True 385 | 31.836397249681724,False 386 | 118.2649748426501,True 387 | 5.691191042718932,False 388 | 28.846650534905713,True 389 | 120.69162565766668,True 390 | 4170.897282127189,True 391 | 97.02521812885931,True 392 | 89.8265148822239,False 393 | 59.32693539186718,True 394 | 29.61096518616431,False 395 | 344.63551743301497,True 396 | 58.14259414422845,False 397 | 98.20005818653888,True 398 | 1972.031709360136,False 399 | 43.24247130262344,False 400 | 58.647566886556476,True 401 | 351.5113158685246,True 402 | 32.88596637499176,False 403 | 28.242030492671127,True 404 | 28.9219286350609,False 405 | 543.7000112789897,True 406 | 28.051816000964294,True 407 | 599.9601621194172,False 408 | 68.07142025136505,False 409 | 31.969472418115362,False 410 | 49.25939110205156,True 411 | 228.2158546120038,False 412 | 81.36621199908447,False 413 | 1751.4157821884858,False 414 | 337.0690978946528,True 415 | 118.68044033970799,True 416 | 38.73827923959679,False 417 | 109.80160663928098,False 418 | 308.6182893917726,True 419 | 3611.4515210696854,False 420 | 389.3866130830857,True 421 | 31.31768076552471,False 422 | 413.22469701645366,True 423 | 92.01871389020319,True 424 | 47.8339720771014,True 425 | 96.06092530209062,False 426 | 61.31549067915408,True 427 | 3791.8267904325594,True 428 | 8.709596570957293,False 429 | 60.537525988503056,False 430 | 1617.9385528253256,False 431 | 296.4221815252353,False 432 | 60.666727864895684,True 433 | 74.42659186990136,False 434 | 32.627433413606234,True 435 | 31.471563405179968,True 436 | 1983.9934316828385,True 437 | 60.21878389190786,True 438 | 29.94530279252027,True 439 | 29.318426058774577,True 440 | 117.1545727474523,False 441 | 587.1825334607802,False 442 | 60.178663301306635,True 443 | 250.8645256148131,False 444 | 561.1471250071182,True 445 | 762.577059285565,True 446 | 17.957897343591224,False 447 | 6.728785195358067,False 448 | 32.77384682885445,True 449 | 766.2901877053558,True 450 | 353.5694827602467,False 451 | 54.55139060367998,True 452 | 18.82914476162354,False 453 | 8.944536049832113,False 454 | 29.511540343725088,True 455 | 1017.3462657463184,False 456 | 117.87984447366455,False 457 | 44.26129627721977,True 458 | 55.9583159602013,True 459 | 28.976656418092414,False 460 | 11.328183265798332,False 461 | 876.2667042431026,True 462 | 32.491094349836146,True 463 | 1024.108816353904,True 464 | 30.92860278910428,False 465 | 46.31934517656243,False 466 | 31.397693618807427,True 467 | 973.7210988232857,True 468 | 1409.678884347155,True 469 | 1135.4198001777113,False 470 | 412.328032894075,False 471 | 30.642877173560162,True 472 | 118.09236377087237,False 473 | 62.315858335145585,True 474 | 58.4251236758209,True 475 | 28.084818958634898,False 476 | 62.099072565182944,False 477 | 1848.79521371316,False 478 | 118.83339499129077,True 479 | 74.03415084041526,True 480 | 28.903294982765026,True 481 | 121.74639536021999,False 482 | 1312.068642385296,False 483 | 0.9421719144598579,True 484 | 59.077649812339764,False 485 | 1248.318129664553,False 486 | 29.37883405877664,False 487 | 58.81012647494916,False 488 | 29.082394632452235,False 489 | 1498.4558416611105,True 490 | 3588.9674786956766,False 491 | 59.60180548914225,True 492 | 189.7660391089659,False 493 | 266.0329739065698,False 494 | 1035.5454542735492,False 495 | 29.14045466507694,False 496 | 413.42441801169105,False 497 | 243.33058679650773,True 498 | 30.437873335061326,True 499 | 34.499688565972164,True 500 | 19.502975948772832,True 501 | 265.0130238821961,False 502 | -------------------------------------------------------------------------------- /full_notebooks/churn_data.csv: -------------------------------------------------------------------------------- 1 | T,E 2 | 1918.8444334189646,True 3 | 60.58709569858339,False 4 | 235.85783356743804,False 5 | 5.769406867985342,False 6 | 58.888729395523626,False 7 | 59.55210047985336,False 8 | 1199.2959953859872,True 9 | 1662.937394487151,False 10 | 61.47712837947204,True 11 | 29.53359180984878,False 12 | 30.347768896515717,True 13 | 739.069812044534,False 14 | 59.07555932308129,True 15 | 31.402037914336695,False 16 | 62.06673215741615,False 17 | 1113.1019697919096,False 18 | 40.87179188075572,False 19 | 59.73628056481591,True 20 | 25.374348003740494,False 21 | 324.4288687454613,False 22 | 120.34605752294291,False 23 | 30.35090004833161,True 24 | 30.92787733204829,True 25 | 851.9547926360306,True 26 | 126.2657797900945,True 27 | 122.12235315246735,True 28 | 32.555168797561976,False 29 | 31.219016337721865,True 30 | 32.47869466439353,True 31 | 59.20652124308241,False 32 | 296.58994854803706,False 33 | 26.402067591858838,True 34 | 1250.4678112609135,True 35 | 30.17278633110269,True 36 | 61.30707996464904,True 37 | 28.217691936937356,True 38 | 89.78683030642097,False 39 | 287.24263472284446,True 40 | 29.697243433357656,False 41 | 492.9566828815502,True 42 | 1611.6148732043434,False 43 | 58.83654075345078,False 44 | 61.654187693099374,True 45 | 117.90605292971095,True 46 | 28.87503274105026,False 47 | 61.84132233626951,False 48 | 325.45405865127685,True 49 | 0.7519377397520679,False 50 | 94.95142854355235,False 51 | 925.208049022244,True 52 | 2027.1982302543663,False 53 | 1164.494835463643,False 54 | 2312.452505791591,True 55 | 608.8724473978314,False 56 | 92.54222070098558,True 57 | 28.12592868802813,False 58 | 1157.9296379225195,True 59 | 62.30815717036579,True 60 | 2912.611457461077,False 61 | 121.58835553922992,True 62 | 930.1698890575404,False 63 | 1529.75297801831,False 64 | 826.8411732231668,False 65 | 817.8678917149413,False 66 | 30.443957573730717,True 67 | 121.23084533462364,False 68 | 90.73069218106288,True 69 | 466.17431777910576,True 70 | 29.075597277793197,True 71 | 852.9808407834448,False 72 | 89.02185081344733,True 73 | 28.856424017805516,False 74 | 152.82123037263594,False 75 | 92.53143599650774,True 76 | 26.59601378192261,False 77 | 32.82152829669721,False 78 | 32.106806876336925,True 79 | 32.335983218509575,False 80 | 61.310887804687084,True 81 | 29.5699344638767,True 82 | 91.71313184351109,False 83 | 30.631998551680955,False 84 | 31.811819947385047,True 85 | 8.72650465909297,True 86 | 612.4424150017494,True 87 | 58.29092872054026,False 88 | 966.3917581569004,False 89 | 1654.0891896029284,True 90 | 913.4239976968645,False 91 | 60.780990076171186,True 92 | 186.9866850494446,False 93 | 28.819132946425853,False 94 | 92.24886202505344,True 95 | 2276.350297445255,True 96 | 2004.803649020715,False 97 | 28.564568067340968,True 98 | 1037.315708842767,False 99 | 257.0319036539696,False 100 | 152.26498628304913,True 101 | 48.601419226930915,True 102 | 62.341527827984265,False 103 | 62.73031066136864,False 104 | 1034.7600234854333,False 105 | 14.77967612353376,True 106 | 28.917053425047396,False 107 | 30.813119913187247,False 108 | 529.9660682467472,True 109 | 32.6717257535947,True 110 | 88.21450942805357,False 111 | 120.7509491194449,False 112 | 1322.9150708208326,True 113 | 2088.1127815122118,False 114 | 3711.4748197367717,False 115 | 994.5252678556161,True 116 | 150.1620244024024,True 117 | 29.78744343077413,False 118 | 831.5400280592003,True 119 | 118.21698138959938,True 120 | 60.65229652196831,True 121 | 310.7325247386722,True 122 | 406.57838193447594,True 123 | 89.87053872974116,True 124 | 78.51007923947273,False 125 | 597.4648804418093,True 126 | 229.9557473398346,True 127 | 117.4467243530542,True 128 | 1299.0643322190238,True 129 | 59.19402611170842,False 130 | 718.4474671474558,True 131 | 28.41946664005578,True 132 | 50.82913701474391,True 133 | 152.71262931754265,False 134 | 88.33563210098966,False 135 | 401.47954010834286,True 136 | 24.098726228533,False 137 | 61.30559738967629,False 138 | 82.60848016084975,False 139 | 89.12275157944123,True 140 | 118.65801493219864,True 141 | 29.056047728546055,True 142 | 886.6122178889367,False 143 | 71.85291138030507,False 144 | 31.287081026331187,True 145 | 30.754132710504873,False 146 | 62.81444015838644,False 147 | 89.12143473095507,False 148 | 88.64415866196615,False 149 | 547.7653075140499,False 150 | 1176.7097377227624,True 151 | 62.33400396044171,False 152 | 143.7744265448816,True 153 | 1201.4544898011527,True 154 | 1893.8543498935096,False 155 | 507.6969496482953,False 156 | 58.570298247797254,True 157 | 30.69393547976163,True 158 | 2053.804983416033,False 159 | 60.892646323606684,False 160 | 19.22783335254502,True 161 | 32.34079475085986,True 162 | 200.60395771288663,False 163 | 28.829807163527406,False 164 | 746.1924431899437,True 165 | 3806.357457468599,True 166 | 756.1880304647035,False 167 | 421.1718165654018,True 168 | 409.27934562186414,True 169 | 119.36526570035555,False 170 | 77.21530419089683,False 171 | 92.71526911308673,True 172 | 29.72500732025589,False 173 | 119.29939712147427,True 174 | 30.991188016501095,True 175 | 363.8692789322257,False 176 | 151.82682314962875,True 177 | 157.7620675681842,True 178 | 571.7737770815355,True 179 | 91.48071906544784,False 180 | 148.83216091860857,True 181 | 29.44672037276354,True 182 | 2489.388272000309,True 183 | 29.551349375741587,True 184 | 31.964306154747234,True 185 | 2070.920181709684,True 186 | 32.624270721992566,True 187 | 31.754539890250665,True 188 | 32.48121422381833,False 189 | 213.73225709350103,False 190 | 1378.1785992454832,False 191 | 754.5779023688779,True 192 | 28.790507073748504,False 193 | 32.327325166715084,True 194 | 89.84386677986329,True 195 | 31.10916160650975,True 196 | 191.44558699657395,False 197 | 314.02181461711683,False 198 | 391.9633719623474,True 199 | 422.93722780189006,False 200 | 62.86213954654554,True 201 | 60.153277536160275,False 202 | 838.3856396918914,True 203 | 583.438782423287,False 204 | 68.35753933280675,False 205 | 1115.4888389234834,False 206 | 28.590548026576094,False 207 | 1392.9549030126561,False 208 | 59.83775346706255,True 209 | 32.63457216218257,False 210 | 31.69724978836863,False 211 | 688.8279457019694,True 212 | 60.45535588981685,True 213 | 906.5747776673177,False 214 | 395.1901508108459,False 215 | 118.62170278094324,False 216 | 1617.3448630034316,False 217 | 376.3996818184131,False 218 | 134.51129324514284,True 219 | 179.30060210502003,True 220 | 61.31425100594675,False 221 | 32.86472298461616,True 222 | 31.688750627568172,True 223 | 354.72452350881963,True 224 | 441.2620020002763,False 225 | 61.71079636631,False 226 | 183.07254714974002,False 227 | 31.775732311387838,False 228 | 90.38060816320188,True 229 | 25.111657127139264,False 230 | 409.3552998162392,False 231 | 31.924689597458215,True 232 | 548.6553971413332,False 233 | 149.4468043115037,False 234 | 685.8859587488788,False 235 | 59.85013186191726,True 236 | 118.69715138994641,False 237 | 28.543017327745325,True 238 | 862.7581442788589,True 239 | 121.4707632457652,True 240 | 1148.019568906269,False 241 | 152.0121728279416,True 242 | 29.637877965508288,True 243 | 28.130749123812127,False 244 | 62.13324641275469,True 245 | 75.73809765068884,True 246 | 31.51727912683262,True 247 | 41.05006291393443,False 248 | 227.94667633552785,False 249 | 3.058919737420856,False 250 | 58.39432539336273,True 251 | 30.78675104661579,False 252 | 149.27277350148873,True 253 | 62.51150783464718,False 254 | 170.72540650748343,True 255 | 150.1088527965297,True 256 | 106.31105166977042,False 257 | 30.377824139866657,False 258 | 131.69306005897886,False 259 | 2024.394961587824,True 260 | 561.2272372407239,True 261 | 29.811439015573885,True 262 | 199.91119914322908,True 263 | 96.82322764751888,True 264 | 80.07827911277673,False 265 | 1623.5338597549658,True 266 | 1632.4744540716943,True 267 | 118.59967709511635,True 268 | 150.7709368341028,False 269 | 148.0170429612313,True 270 | 491.16731172663077,False 271 | 732.3012866390409,False 272 | 370.753216846552,False 273 | 907.4146324278167,False 274 | 32.25413248631612,True 275 | 28.065180835795083,True 276 | 154.03998875952342,False 277 | 526.3977457634726,True 278 | 1911.7278176554034,True 279 | 864.0277955626417,True 280 | 17.845108709950047,True 281 | 681.6973379003855,False 282 | 121.84326347964706,True 283 | 61.09507235284781,False 284 | 631.4090035419022,True 285 | 400.1792983150306,False 286 | 1215.5700957071895,True 287 | 722.9777253609994,False 288 | 970.9329265218747,False 289 | 28.403901941439347,True 290 | 1577.7239379062978,False 291 | 28.75054283134122,False 292 | 151.2490401536432,False 293 | 529.6854006292965,True 294 | 135.13348271173058,False 295 | 1780.2514745833366,True 296 | 47.96318457906845,False 297 | 762.5602271286647,False 298 | 150.1387140254899,True 299 | 1625.1318141083068,True 300 | 62.421792668902214,False 301 | 792.3099062648015,False 302 | 246.0493311290191,True 303 | 103.1702788793879,True 304 | 31.817911849953305,True 305 | 296.3028888173469,False 306 | 89.17613034533403,True 307 | 210.03038803304713,False 308 | 529.1765500263801,False 309 | 88.26803105402293,True 310 | 662.9086542101448,False 311 | 367.8040932462375,True 312 | 742.4538586567774,True 313 | 374.4129561177297,False 314 | 29.6494671922694,True 315 | 1527.7583464750282,False 316 | 2453.09223195441,False 317 | 357.40891732789737,True 318 | 30.73317172474882,False 319 | 31.85230212370816,False 320 | 210.39712357550812,True 321 | 244.81699776312743,False 322 | 90.44073431278457,True 323 | 1891.4275052536593,True 324 | 4.907421648763451,True 325 | 558.1778070093968,False 326 | 59.18175078711195,True 327 | 1297.180328335456,False 328 | 58.797230096619494,True 329 | 29.05541177358451,True 330 | 392.8390247709013,False 331 | 1166.3786695552096,True 332 | 82.89574721853467,False 333 | 29.217282193488828,True 334 | 491.55555306284924,False 335 | 92.2913959530088,True 336 | 301.3094774236432,True 337 | 59.06648188815554,False 338 | 174.46829894289797,True 339 | 31.73450731170866,False 340 | 29.115649154734673,False 341 | 385.93672265890723,True 342 | 61.80518290464266,True 343 | 32.95167342580057,False 344 | 361.9272869136181,False 345 | 149.16099324561122,True 346 | 2375.896774829766,True 347 | 73.82620333481805,False 348 | 35.822413313505685,True 349 | 88.93328872973142,True 350 | 1319.1944909284634,True 351 | 96.25775824813032,True 352 | 66.6306395003955,True 353 | 92.23792197211867,False 354 | 1384.2290970844904,True 355 | 151.7952505457692,True 356 | 498.4785707382946,False 357 | 28.231616498050954,False 358 | 61.38272715979188,False 359 | 29.725475113479952,False 360 | 164.36744456075562,True 361 | 776.3874853900305,True 362 | 14.586805834498481,True 363 | 292.01044374267224,False 364 | 1092.6709609758054,False 365 | 29.858164531271818,False 366 | 565.5583815703344,True 367 | 237.3450853895253,False 368 | 30.657181191133134,False 369 | 9.374326522091975,True 370 | 60.03233949109139,True 371 | 30.007259123772634,False 372 | 30.281399353661058,True 373 | 28.459061244003635,False 374 | 148.60384068030507,False 375 | 3.023642623625279,False 376 | 353.7001518483322,False 377 | 59.94359286683783,False 378 | 61.27890260545947,False 379 | 722.6329459309281,True 380 | 339.65494141780596,False 381 | 68.92046872298398,True 382 | 62.218455219133986,True 383 | 496.5314265130781,True 384 | 282.7264728126524,False 385 | 60.73611541425321,True 386 | 30.223676848763372,False 387 | 119.75517421228751,False 388 | 32.27355801992015,True 389 | 132.65226515640904,False 390 | 560.0208370835843,True 391 | 386.50089341546516,True 392 | 306.3657850616188,False 393 | 121.80172656586242,True 394 | 30.885719272519513,True 395 | 121.63282585013677,False 396 | 525.4534040316942,False 397 | 417.80879304378374,True 398 | 578.6032328652717,False 399 | 521.3654038440606,True 400 | 45.37116162430878,True 401 | 91.17800868566145,True 402 | 45.45393563865068,False 403 | 5.916012457248374,True 404 | 1170.8406831523043,True 405 | 31.02046255491225,False 406 | 120.07722480413295,True 407 | 90.44559894282997,False 408 | 879.7412712243565,True 409 | 1130.7583078179123,True 410 | 20.763180064832138,False 411 | 39.079290312186615,False 412 | 117.21649757568903,True 413 | 567.8034290232428,False 414 | 587.4600495928695,False 415 | 333.63121832312606,False 416 | 843.7871545203496,True 417 | 21.874339230054357,True 418 | 119.63522229340563,False 419 | 13.61725605618718,False 420 | 28.056621041144393,False 421 | 1588.1418439010213,False 422 | 1528.0644410505133,True 423 | 96.85702639804879,True 424 | 825.143766728794,False 425 | 457.4745337793762,False 426 | 1980.7286793884527,False 427 | 30.912409674687396,False 428 | 77.08103657928947,False 429 | 62.6721005298554,False 430 | 28.912496650183797,True 431 | 60.46889516373128,False 432 | 30.247129365118436,False 433 | 2280.22717524111,False 434 | 89.20216736987088,False 435 | 25.02280120671306,True 436 | 59.967676640843166,False 437 | 89.19126763079139,False 438 | 169.50201631314434,True 439 | 1476.008589726766,False 440 | 1576.1778187278064,False 441 | 458.01509139977,False 442 | 65.22465984840144,False 443 | 32.47697785990427,True 444 | 238.09750362403636,True 445 | 59.239935016594345,True 446 | 1907.8543147117803,True 447 | 29.589106598282687,False 448 | 1393.5128196930484,True 449 | 815.1330174594034,False 450 | 28.201382023002022,False 451 | 15.448507130962687,True 452 | 5785.553314582964,True 453 | 2271.820607296963,False 454 | 2529.686936063965,False 455 | 29.3272824119191,False 456 | 30.220751104055875,False 457 | 337.6967807342231,False 458 | 252.48788110074705,False 459 | 795.1892175185887,False 460 | 278.88987869033474,False 461 | 75.22389641620303,False 462 | 116.46859420972886,False 463 | 91.9427982639135,True 464 | 31.77056519527369,True 465 | 67.4578099864663,False 466 | 1997.2513314018518,True 467 | 227.98333441387865,True 468 | 65.7025043336093,False 469 | 29.262617910758,True 470 | 30.906246610614506,False 471 | 58.20120113753568,False 472 | 16.67159436995523,False 473 | 148.71702398683001,True 474 | 227.87679945845687,False 475 | 33.60520429694742,True 476 | 31.315654398088387,True 477 | 29.090020638256124,False 478 | 383.1146337652536,False 479 | 953.0308817423811,False 480 | 534.7935780578857,False 481 | 136.01269944278056,True 482 | 2201.7721462798295,True 483 | 116.72761310621203,True 484 | 91.07383575790782,False 485 | 80.45702392460052,True 486 | 150.0001779059884,False 487 | 8.917986727940868,True 488 | 1304.45563269596,True 489 | 61.311806475795876,False 490 | 30.39391208905676,True 491 | 285.8362523251053,False 492 | 89.7759900505312,True 493 | 52.613914050715294,False 494 | 940.1055190760954,False 495 | 29.76325523553747,True 496 | 207.87518928791374,False 497 | 418.14474774997433,True 498 | 2346.0008749733893,True 499 | 387.66311418127736,False 500 | 54.32352962945931,True 501 | 275.88936047447595,True 502 | -------------------------------------------------------------------------------- /tutorial_notebooks/churn_data.csv: -------------------------------------------------------------------------------- 1 | T,E 2 | 61.11966889512959,True 3 | 28.29329131957169,False 4 | 1055.4294587444056,False 5 | 18.969402299134416,True 6 | 734.9852146798564,False 7 | 44.26506154972145,True 8 | 919.5456822703437,True 9 | 3440.925290396631,False 10 | 428.2962736896216,False 11 | 121.89920238809441,True 12 | 148.46262644188414,True 13 | 30.413815992327375,False 14 | 28.24137339053798,False 15 | 130.05101503483996,True 16 | 111.53800161826473,True 17 | 117.13663798118831,True 18 | 327.73436911404724,False 19 | 764.4913033092399,True 20 | 1571.0736082743972,False 21 | 58.8603584298655,True 22 | 31.204285972912732,True 23 | 243.485146280934,False 24 | 657.7485148942856,False 25 | 117.00315087851209,True 26 | 62.893549568610005,False 27 | 28.489108578033807,True 28 | 1297.7507837326834,True 29 | 1119.527415558363,True 30 | 37.3210915412213,False 31 | 118.11581420262728,False 32 | 1045.7593514205741,True 33 | 1498.6271717617792,True 34 | 954.3250211214369,False 35 | 60.31065246514634,False 36 | 29.716172491131115,True 37 | 2434.1346942788055,False 38 | 2476.8086244536184,False 39 | 24.658993373469247,True 40 | 148.2962451139875,False 41 | 30.92719506173756,False 42 | 90.174395739397,True 43 | 1444.8652472936105,False 44 | 29.647461408122563,False 45 | 1836.4653803310448,True 46 | 80.68055192086933,False 47 | 898.3186532188165,True 48 | 59.605455881326044,False 49 | 169.22163131928303,True 50 | 155.16043809382626,True 51 | 46.92797669890706,False 52 | 303.08633181153664,False 53 | 29.17983130116863,False 54 | 507.4390777580862,False 55 | 976.2834150880803,False 56 | 92.37173022557359,True 57 | 370.5876080948349,False 58 | 416.1115572690018,False 59 | 87.77911739916968,False 60 | 62.71743482104752,True 61 | 29.665870232954926,False 62 | 1711.6625026224694,True 63 | 85.5262468389674,True 64 | 542.8457977270848,False 65 | 2949.634432553643,True 66 | 59.418322882519654,True 67 | 44.61096328343211,True 68 | 289.7455347695958,True 69 | 1511.314249463554,False 70 | 108.1154716498452,False 71 | 28.867658734728714,True 72 | 1430.260935047219,False 73 | 32.669749967965416,True 74 | 279.25598464609425,False 75 | 58.608210436434945,True 76 | 29.338885138543116,False 77 | 58.96264000839855,True 78 | 101.22754723188585,True 79 | 467.98766400031354,True 80 | 36.52260340732223,False 81 | 59.686200311664244,False 82 | 60.286351473089496,True 83 | 416.58336138373744,True 84 | 725.8156836463281,False 85 | 399.32367075830905,True 86 | 71.59672383127632,True 87 | 61.70804131902698,False 88 | 1044.3038064115094,False 89 | 58.59350819088008,True 90 | 148.10937267372168,True 91 | 216.5759596972818,True 92 | 599.832298961752,False 93 | 570.1460144895366,False 94 | 138.69524095964084,True 95 | 29.787231529712752,False 96 | 89.73310383086215,False 97 | 29.554489999111265,True 98 | 29.885240373485974,False 99 | 1255.309126156514,False 100 | 62.819605291946026,False 101 | 392.6263249369914,True 102 | 430.95006395476145,True 103 | 89.70230621900212,True 104 | 151.91749426669588,True 105 | 118.86843358899235,True 106 | 653.4232710276237,False 107 | 148.66168855206868,True 108 | 420.3146425950929,True 109 | 990.0178744068562,False 110 | 150.20185192674887,False 111 | 14.920712106839686,True 112 | 32.68014503113899,True 113 | 662.9662314333631,False 114 | 109.69746783359562,False 115 | 396.3596252607747,False 116 | 3.941616149862312,True 117 | 1529.0755965091482,False 118 | 30.968805547249655,False 119 | 823.9982956979987,True 120 | 152.36040880838982,True 121 | 1027.514392352339,False 122 | 84.41768835473033,False 123 | 1769.3235086469174,False 124 | 54.15079694647002,False 125 | 117.0275071220785,True 126 | 58.63761236337845,False 127 | 1215.0623789368929,False 128 | 1342.8077709491602,False 129 | 847.9053073567708,True 130 | 151.63490737751766,False 131 | 28.442674106099155,False 132 | 62.96905373169369,False 133 | 832.2697316928865,True 134 | 158.59767082446584,False 135 | 32.01605361636607,True 136 | 12.528921508896378,True 137 | 1208.2107645503231,False 138 | 30.198025300569043,True 139 | 32.367256759847564,False 140 | 120.41669256947975,True 141 | 883.5795744661547,True 142 | 1510.2134823652348,False 143 | 28.527834546473557,False 144 | 96.76504895706152,True 145 | 32.53788398223247,True 146 | 234.7037377275002,True 147 | 818.6046952905523,False 148 | 37.585076175187645,True 149 | 28.50956149612961,False 150 | 88.25778342901312,False 151 | 30.142249100054542,False 152 | 31.9932661480591,True 153 | 78.8661049088623,False 154 | 24.66880306990277,True 155 | 31.92310079771387,True 156 | 446.67446132318884,True 157 | 1041.4740395315898,False 158 | 607.3614795540494,True 159 | 18.797160460515517,False 160 | 29.102806548834746,False 161 | 62.38048649489862,True 162 | 748.5816993516154,True 163 | 19.18908837467661,False 164 | 59.359475485280115,True 165 | 32.634681689502834,True 166 | 2328.560162658755,True 167 | 61.61119127629763,True 168 | 251.2933642747838,False 169 | 768.4751465161995,True 170 | 559.2679693370118,True 171 | 478.4524296528603,False 172 | 932.5276159685712,False 173 | 28.259624474306207,True 174 | 1094.0336762163138,True 175 | 89.94443900625802,True 176 | 89.99119169179922,True 177 | 118.8174579047762,False 178 | 3142.7117242532872,False 179 | 279.2419109391353,True 180 | 29.631012484859028,True 181 | 58.360934886899095,False 182 | 744.8747267225775,True 183 | 91.56365527531739,False 184 | 58.706788994725585,False 185 | 58.11500626943964,False 186 | 1096.711539503755,True 187 | 303.382344200779,False 188 | 152.150484017376,True 189 | 30.205270073817935,True 190 | 395.2834667350893,True 191 | 25.17730686439135,True 192 | 199.30746408965473,True 193 | 32.48079118452889,True 194 | 1783.4807896664172,False 195 | 550.75243776232,False 196 | 1036.850309701936,True 197 | 28.173725433171327,False 198 | 30.35108922428749,True 199 | 341.58814380206184,False 200 | 30.4204689802693,True 201 | 42.76135494936754,False 202 | 30.746685093154973,False 203 | 62.31299956563369,False 204 | 929.48969837861,False 205 | 30.24226616564645,True 206 | 92.57196170786402,False 207 | 149.72649235205225,True 208 | 1216.7341006411941,True 209 | 32.168929116639596,True 210 | 150.22559993479405,False 211 | 28.1546936086836,True 212 | 121.95601690482053,False 213 | 359.607185995672,True 214 | 101.88908192658506,False 215 | 48.19736441240593,False 216 | 92.99338426560819,False 217 | 28.60157871238954,False 218 | 851.7521401012998,True 219 | 25.837959750287872,False 220 | 28.574893907710713,False 221 | 339.01543408915404,False 222 | 118.01968594778633,True 223 | 29.046087606815526,True 224 | 56.52391653837993,True 225 | 62.35159258072066,True 226 | 509.28964511077686,False 227 | 60.2908393940133,False 228 | 149.51087982791537,True 229 | 30.966027898674902,True 230 | 148.78015783235907,False 231 | 92.52477338982864,True 232 | 90.38235727620852,True 233 | 28.866854933329517,True 234 | 29.15082432710313,False 235 | 29.470151372377803,True 236 | 30.49502097552584,False 237 | 1174.083678821613,True 238 | 30.037986282980032,True 239 | 60.401526660904786,True 240 | 31.337578321117117,True 241 | 120.19757593627766,True 242 | 873.4600547384599,True 243 | 73.81100905417263,True 244 | 134.4033374290313,True 245 | 2600.21539839032,False 246 | 155.34485952410648,False 247 | 117.08588986036598,True 248 | 32.078951855407055,False 249 | 149.26526238469984,False 250 | 28.22443793157215,False 251 | 1189.6250082580955,True 252 | 73.37263228710353,False 253 | 30.195402276296765,True 254 | 395.25653848334196,True 255 | 2032.5850902469433,False 256 | 539.5483743968858,True 257 | 14.261933349096706,False 258 | 30.673966448400428,True 259 | 18.50699171909481,False 260 | 1201.6657844083209,False 261 | 47.87044460342523,True 262 | 58.59626670420383,False 263 | 29.955622074398967,False 264 | 31.63159996903291,False 265 | 414.78692359643804,False 266 | 529.7641560754917,True 267 | 28.07484533958263,False 268 | 1035.9533158719732,True 269 | 479.41075244202165,True 270 | 1823.0301063915706,True 271 | 200.91012102779354,True 272 | 317.6227581640427,True 273 | 90.02454798684833,True 274 | 24.423624161567133,False 275 | 692.5750417873672,True 276 | 382.4160555613045,True 277 | 29.213234631813332,False 278 | 1481.9126045154087,False 279 | 704.7075637900365,True 280 | 28.961486209138975,True 281 | 194.88164430717262,False 282 | 288.6355830916604,False 283 | 31.959776823645385,True 284 | 58.05165549627123,True 285 | 76.3689116083565,True 286 | 28.562314173361155,True 287 | 119.5938413977045,False 288 | 62.00576208523608,False 289 | 28.28474665909703,False 290 | 59.3534255833145,True 291 | 645.6861969830597,False 292 | 150.7617535518349,True 293 | 90.37539851100226,True 294 | 16.364640864396456,False 295 | 958.9313748683071,False 296 | 313.2145000385791,True 297 | 115.41779692525259,True 298 | 158.43817447047715,True 299 | 151.84461785118435,True 300 | 29.034126402912957,True 301 | 62.46472451981519,True 302 | 118.69005281446428,False 303 | 60.95193793208848,False 304 | 32.073712739071794,False 305 | 1262.6001456821143,True 306 | 91.3750126395549,True 307 | 334.126278168936,True 308 | 163.9942257856624,False 309 | 60.6390403278174,True 310 | 749.5046465776333,True 311 | 88.5018505713718,False 312 | 117.32515120996212,True 313 | 2188.1758479178825,True 314 | 61.48726927211163,False 315 | 420.4247760696587,True 316 | 553.2047950173247,False 317 | 95.43155525026334,False 318 | 32.287109179048535,False 319 | 29.265856966058223,True 320 | 798.812152100487,False 321 | 88.68216457079365,True 322 | 0.1537358491765172,True 323 | 2053.2469690087464,False 324 | 148.17383785524345,True 325 | 92.47386070617,True 326 | 32.40126047425528,False 327 | 355.2619108246928,True 328 | 14.247583888553892,False 329 | 260.7182009270666,False 330 | 24.88661360190424,False 331 | 59.39917401886674,False 332 | 30.372116688798766,True 333 | 30.65965698067487,True 334 | 680.2622725340158,False 335 | 32.277328851217966,False 336 | 29.481184440600032,False 337 | 28.191956791844433,False 338 | 102.4053041639592,True 339 | 235.9083765727364,False 340 | 31.42430525552857,False 341 | 1121.3499735838873,True 342 | 32.33787987730689,True 343 | 30.522078929281008,False 344 | 148.5069655408873,True 345 | 31.638611710784637,True 346 | 901.0171393954065,False 347 | 30.27079787159191,True 348 | 945.9196247372258,False 349 | 10.575904582279785,True 350 | 640.9133015432244,False 351 | 635.922422567605,True 352 | 7.254647614600136,True 353 | 146.77932785268754,False 354 | 3409.166311941784,True 355 | 136.03024708872482,False 356 | 31.142722979540682,True 357 | 138.27726106121395,False 358 | 431.5595848637359,False 359 | 59.129966153947116,False 360 | 902.2247792339687,False 361 | 151.63695087357573,True 362 | 651.9587566676353,True 363 | 3277.6527901243844,True 364 | 247.29377608563541,True 365 | 1455.5234207976316,True 366 | 1518.6860620285674,False 367 | 1411.7437038917167,True 368 | 961.0412389490714,True 369 | 30.857436856636365,True 370 | 28.377527131424536,True 371 | 31.189457045670494,True 372 | 32.3448561306321,False 373 | 60.79539427349783,False 374 | 728.0032575149924,False 375 | 717.2657871014911,False 376 | 31.42648600161855,False 377 | 30.868718305515376,False 378 | 312.8103449170618,True 379 | 62.692528661438914,True 380 | 32.814221994676465,False 381 | 160.66628606407684,False 382 | 117.96302188532493,False 383 | 60.62729199974702,False 384 | 88.85616472941261,False 385 | 549.2968217997941,False 386 | 425.3007482247608,False 387 | 13.969643858989343,False 388 | 59.86204256403313,True 389 | 22.644661299882834,True 390 | 61.849620084823286,True 391 | 29.076303442734467,False 392 | 19.145795760328216,True 393 | 62.193076123522935,False 394 | 39.05337204561768,True 395 | 32.34730961934042,False 396 | 60.57045143281781,False 397 | 1031.4275861035312,False 398 | 3.3719551118858053,False 399 | 45.79492673114906,True 400 | 44.6189119880803,False 401 | 62.182168096411985,True 402 | 213.32017840718976,True 403 | 1754.739149120707,True 404 | 463.0754194966577,False 405 | 89.67208338131476,True 406 | 91.1258738133434,True 407 | 616.5561854291193,False 408 | 91.74850006341556,True 409 | 45.97431374038945,True 410 | 1004.8667930536408,False 411 | 312.2484297089851,False 412 | 1605.221151442954,False 413 | 796.9374508695325,True 414 | 107.24474748310081,False 415 | 284.63536847322484,True 416 | 2019.255355036492,False 417 | 440.6577539713162,False 418 | 361.54809464068796,False 419 | 122.86571809159737,False 420 | 29.88847100027103,False 421 | 58.806956624155006,True 422 | 515.3938638555213,False 423 | 29.4238702233274,False 424 | 30.35689651164651,True 425 | 1569.270637152898,True 426 | 433.33675149745716,True 427 | 32.66884087433735,False 428 | 1990.3617115113163,True 429 | 30.597318892596615,True 430 | 28.135803045213017,True 431 | 751.6681487272759,False 432 | 118.27599286235659,False 433 | 521.0225992517599,True 434 | 479.987275961792,True 435 | 1262.4165960172863,False 436 | 368.4339279850394,True 437 | 28.724690165115486,True 438 | 516.6348293564506,True 439 | 245.96979187902963,True 440 | 84.64390145673502,False 441 | 101.4195122477787,True 442 | 463.4770345564719,True 443 | 367.4858491974274,True 444 | 90.42121550753447,True 445 | 30.505581876729874,False 446 | 2085.3134137774337,False 447 | 500.22920858027624,False 448 | 1365.8972244147953,False 449 | 642.879175664619,True 450 | 89.75310357252421,True 451 | 287.3113337131189,True 452 | 1869.3682917061765,False 453 | 596.715670988975,False 454 | 270.82076475764273,True 455 | 1033.3808155140816,True 456 | 32.72296388170144,False 457 | 31.938532946151888,True 458 | 58.35527006846759,False 459 | 97.88918934927854,False 460 | 163.99453543532655,True 461 | 309.7241365899543,False 462 | 117.9804420613358,False 463 | 28.212321429403467,True 464 | 1.3342441439607822,False 465 | 29.1774403244467,False 466 | 347.15264955730726,False 467 | 245.22168040729719,True 468 | 719.6838650733515,True 469 | 149.26069540142143,False 470 | 992.1650570905002,False 471 | 63.869301902798256,True 472 | 32.91943546746089,False 473 | 7.738687413750223,False 474 | 376.0193932861698,True 475 | 32.5500434388038,False 476 | 3.677635569434773,False 477 | 29.73396622740457,False 478 | 26.313693458611997,False 479 | 92.25992665982373,False 480 | 32.416023821060776,False 481 | 61.275408926000914,False 482 | 36.14150822719322,True 483 | 306.82454973011795,True 484 | 819.5403029742129,True 485 | 7.878116503370133,True 486 | 152.57569706026723,True 487 | 554.578755617533,True 488 | 89.6257224696395,True 489 | 152.28052982878282,False 490 | 56.05244351920008,True 491 | 11.483775921684149,False 492 | 505.2289458933011,True 493 | 146.96835158631936,True 494 | 62.1834000215891,True 495 | 1468.2348720307652,True 496 | 150.69973112869093,True 497 | 30.395760626916275,False 498 | 2900.0389168931974,False 499 | 2162.822749210184,True 500 | 88.82161551760666,False 501 | 59.65656793109161,False 502 | -------------------------------------------------------------------------------- /full_notebooks/Part 1.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Part 1 \n", 8 | "\n", 9 | " - start with fitting data to a parametric model\n", 10 | " \n", 11 | " - so we need to create the log-likehood\n", 12 | " - let's create some fake data first." 13 | ] 14 | }, 15 | { 16 | "cell_type": "code", 17 | "execution_count": 1, 18 | "metadata": {}, 19 | "outputs": [ 20 | { 21 | "data": { 22 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD7CAYAAACRxdTpAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAARq0lEQVR4nO3df6zddX3H8edrVMS5zfLjrmFtXVloNGSZyG5YjWZxdC6AxvKHIzg3GtLk7g+26XSZnf84ly2BZBElW0ga61YWhzLUtVGy2RSM2x+gF0EUquHKxLYp9IqAOuYP9L0/zqfh0N72ntv7i37u85GcnM/3/fl8z/mcb3pf99vP/Z5zUlVIkvryc8s9AUnSwjPcJalDhrskdchwl6QOGe6S1CHDXZI6NFK4J/nzJA8l+VqS25KcleSCJPcmmUryiSRntrEvbdtTrX/DYr4ASdLxZg33JGuBPwPGq+rXgTOAa4AbgZuq6kLgKWBb22Ub8FSr39TGSZKW0Ko5jHtZkp8APw8cBi4D/qD17wL+GrgF2NLaAHcA/5AkdZJ3S5133nm1YcOGuc5dkla0++677ztVNTZT36zhXlWHkvw98G3g/4DPAfcBT1fVc23YQWBta68FDrR9n0vyDHAu8J3hx00yAUwAvPKVr2RycnKur0uSVrQkj52ob5RlmbMZnI1fAPwK8HLg8vlOqqp2VNV4VY2Pjc34i0eSdIpG+YPq7wL/U1XTVfUT4FPA64HVSY6e+a8DDrX2IWA9QOt/BfDkgs5aknRSo4T7t4FNSX4+SYDNwMPA3cDb2pitwO7W3tO2af13nWy9XZK08GYN96q6l8EfRr8MfLXtswN4L/DuJFMM1tR3tl12Aue2+ruB7Yswb0nSSeTFcFI9Pj5e/kFVkuYmyX1VNT5Tn+9QlaQOGe6S1CHDXZI6ZLhLUodG/fiBF60N2z87Y/1bN7x5iWciSS8enrlLUocMd0nqkOEuSR0y3CWpQ4a7JHXIcJekDhnuktQhw12SOmS4S1KHDHdJ6pDhLkkdMtwlqUOGuyR1aNZwT/KqJA8M3b6X5F1JzkmyN8kj7f7sNj5Jbk4yleTBJJcs/suQJA0b5Quyv1FVF1fVxcBvAs8Cn2bwxdf7qmojsI/nvwj7CmBju00AtyzGxCVJJzbXZZnNwDer6jFgC7Cr1XcBV7X2FuDWGrgHWJ3k/AWZrSRpJHMN92uA21p7TVUdbu3HgTWtvRY4MLTPwVZ7gSQTSSaTTE5PT89xGpKkkxk53JOcCbwV+Ldj+6qqgJrLE1fVjqoar6rxsbGxuewqSZrFXM7crwC+XFVPtO0nji63tPsjrX4IWD+037pWkyQtkbmE+9t5fkkGYA+wtbW3AruH6te2q2Y2Ac8MLd9IkpbASF+QneTlwJuAPx4q3wDcnmQb8BhwdavfCVwJTDG4sua6BZutJGkkI4V7Vf0vcO4xtScZXD1z7NgCrl+Q2UmSTonvUJWkDhnuktQhw12SOmS4S1KHDHdJ6pDhLkkdMtwlqUOGuyR1yHCXpA4Z7pLUIcNdkjpkuEtShwx3SeqQ4S5JHTLcJalDhrskdchwl6QOGe6S1KGRwj3J6iR3JPl6kv1JXpfknCR7kzzS7s9uY5Pk5iRTSR5McsnivgRJ0rFGPXP/MPAfVfVq4DXAfmA7sK+qNgL72jbAFcDGdpsAblnQGUuSZjVruCd5BfDbwE6AqvpxVT0NbAF2tWG7gKtaewtwaw3cA6xOcv6Cz1ySdEKjnLlfAEwD/5Tk/iQfSfJyYE1VHW5jHgfWtPZa4MDQ/gdb7QWSTCSZTDI5PT196q9AknScUcJ9FXAJcEtVvRb4X55fggGgqgqouTxxVe2oqvGqGh8bG5vLrpKkWYwS7geBg1V1b9u+g0HYP3F0uaXdH2n9h4D1Q/uvazVJ0hKZNdyr6nHgQJJXtdJm4GFgD7C11bYCu1t7D3Btu2pmE/DM0PKNJGkJrBpx3J8CH0tyJvAocB2DXwy3J9kGPAZc3cbeCVwJTAHPtrGSpCU0UrhX1QPA+Axdm2cYW8D185yXJGkefIeqJHXIcJekDhnuktQhw12SOmS4S1KHDHdJ6pDhLkkdMtwlqUOGuyR1yHCXpA4Z7pLUIcNdkjpkuEtShwx3SeqQ4S5JHTLcJalDhrskdchwl6QOjRTuSb6V5KtJHkgy2WrnJNmb5JF2f3arJ8nNSaaSPJjkksV8AZKk483lzP13quriqjr6XarbgX1VtRHY17YBrgA2ttsEcMtCTVaSNJr5LMtsAXa19i7gqqH6rTVwD7A6yfnzeB5J0hyNGu4FfC7JfUkmWm1NVR1u7ceBNa29FjgwtO/BVnuBJBNJJpNMTk9Pn8LUJUknsmrEcW+oqkNJfhnYm+Trw51VVUlqLk9cVTuAHQDj4+Nz2leSdHIjnblX1aF2fwT4NHAp8MTR5ZZ2f6QNPwSsH9p9XatJkpbIrOGe5OVJfvFoG/g94GvAHmBrG7YV2N3ae4Br21Uzm4BnhpZvJElLYJRlmTXAp5McHf+vVfUfSb4E3J5kG/AYcHUbfydwJTAFPAtct+CzliSd1KzhXlWPAq+Zof4ksHmGegHXL8jsJEmnxHeoSlKHDHdJ6pDhLkkdMtwlqUOGuyR1yHCXpA4Z7pLUIcNdkjpkuEtShwx3SeqQ4S5JHTLcJalDhrskdchwl6QOGe6S1CHDXZI6ZLhLUocMd0nq0MjhnuSMJPcn+UzbviDJvUmmknwiyZmt/tK2PdX6NyzO1CVJJzKXM/d3AvuHtm8EbqqqC4GngG2tvg14qtVvauMkSUtopHBPsg54M/CRth3gMuCONmQXcFVrb2nbtP7NbbwkaYmMeub+IeAvgZ+17XOBp6vqubZ9EFjb2muBAwCt/5k2XpK0RGYN9yRvAY5U1X0L+cRJJpJMJpmcnp5eyIeWpBVvlDP31wNvTfIt4OMMlmM+DKxOsqqNWQccau1DwHqA1v8K4MljH7SqdlTVeFWNj42NzetFSJJeaNZwr6q/qqp1VbUBuAa4q6reAdwNvK0N2wrsbu09bZvWf1dV1YLOWpJ0UvO5zv29wLuTTDFYU9/Z6juBc1v93cD2+U1RkjRXq2Yf8ryq+jzw+dZ+FLh0hjE/BH5/AeYmSTpFvkNVkjpkuEtShwx3SeqQ4S5JHTLcJalDhrskdchwl6QOGe6S1CHDXZI6ZLhLUocMd0nqkOEuSR0y3CWpQ4a7JHXIcJekDhnuktQhw12SOmS4S1KHDHdJ6tCs4Z7krCRfTPKVJA8l+UCrX5Dk3iRTST6R5MxWf2nbnmr9Gxb3JUiSjjXKmfuPgMuq6jXAxcDlSTYBNwI3VdWFwFPAtjZ+G/BUq9/UxkmSltCs4V4DP2ibL2m3Ai4D7mj1XcBVrb2lbdP6NyfJgs1YkjSrkdbck5yR5AHgCLAX+CbwdFU914YcBNa29lrgAEDrfwY4d4bHnEgymWRyenp6fq9CkvQCI4V7Vf20qi4G1gGXAq+e7xNX1Y6qGq+q8bGxsfk+nCRpyJyulqmqp4G7gdcBq5Osal3rgEOtfQhYD9D6XwE8uSCzlSSNZJSrZcaSrG7tlwFvAvYzCPm3tWFbgd2tvadt0/rvqqpayElLkk5u1exDOB/YleQMBr8Mbq+qzyR5GPh4kr8F7gd2tvE7gX9JMgV8F7hmEeYtSTqJWcO9qh4EXjtD/VEG6+/H1n8I/P6CzE6SdEp8h6okdchwl6QOGe6S1CHDXZI6ZLhLUocMd0nqkOEuSR0y3CWpQ4a7JHXIcJekDhnuktQhw12SOmS4S1KHDHdJ6pDhLkkdMtwlqUOGuyR1yHCXpA6N8gXZ65PcneThJA8leWern5Nkb5JH2v3ZrZ4kNyeZSvJgkksW+0VIkl5olDP354D3VNVFwCbg+iQXAduBfVW1EdjXtgGuADa22wRwy4LPWpJ0UrOGe1Udrqovt/b3gf3AWmALsKsN2wVc1dpbgFtr4B5gdZLzF3zmkqQTmtOae5INwGuBe4E1VXW4dT0OrGnttcCBod0OttqxjzWRZDLJ5PT09BynLUk6mZHDPckvAJ8E3lVV3xvuq6oCai5PXFU7qmq8qsbHxsbmsqskaRYjhXuSlzAI9o9V1ada+Ymjyy3t/kirHwLWD+2+rtUkSUtklKtlAuwE9lfVB4e69gBbW3srsHuofm27amYT8MzQ8o0kaQmsGmHM64E/Ar6a5IFWex9wA3B7km3AY8DVre9O4EpgCngWuG5BZyxJmtWs4V5V/w3kBN2bZxhfwPXznJckaR58h6okdchwl6QOGe6S1CHDXZI6ZLhLUocMd0nqkOEuSR0y3CWpQ4a7JHXIcJekDhnuktQhw12SOmS4S1KHDHdJ6pDhLkkdMtwlqUOGuyR1yHCXpA6N8gXZH01yJMnXhmrnJNmb5JF2f3arJ8nNSaaSPJjkksWcvCRpZqOcuf8zcPkxte3AvqraCOxr2wBXABvbbQK4ZWGmKUmai1nDvaq+AHz3mPIWYFdr7wKuGqrfWgP3AKuTnL9Qk5UkjeZU19zXVNXh1n4cWNPaa4EDQ+MOttpxkkwkmUwyOT09fYrTkCTNZN5/UK2qAuoU9ttRVeNVNT42NjbfaUiShpxquD9xdLml3R9p9UPA+qFx61pNkrSETjXc9wBbW3srsHuofm27amYT8MzQ8o0kaYmsmm1AktuANwLnJTkIvB+4Abg9yTbgMeDqNvxO4EpgCngWuG4R5ixJmsWs4V5Vbz9B1+YZxhZw/XwntRA2bP/sjPVv3fDmJZ6JJC0936EqSR0y3CWpQ4a7JHXIcJekDhnuktQhw12SOjTrpZC9OdElkuBlkpL64Zm7JHXIcJekDhnuktQhw12SOmS4S1KHDHdJ6tCKuxTyZPwkSUm98MxdkjpkuEtSh1yWGYHLNZJON565S1KHFuXMPcnlwIeBM4CPVNUNi/E8y+1kn1MzE8/0JS2VBQ/3JGcA/wi8CTgIfCnJnqp6eKGf63Tj8o6kpbIYZ+6XAlNV9ShAko8DW4AVH+4nMtf/ASynuf4i8heatDwWI9zXAgeGtg8Cv3XsoCQTwETb/EGSb5zi850HfOcU9+3Voh2T3Pjiepw58N/J8TwmMzudjsuvnqhj2a6WqaodwI75Pk6SyaoaX4ApdcNjcjyPyfE8JjPr5bgsxtUyh4D1Q9vrWk2StEQWI9y/BGxMckGSM4FrgD2L8DySpBNY8GWZqnouyZ8A/8ngUsiPVtVDC/08Q+a9tNMhj8nxPCbH85jMrIvjkqpa7jlIkhaY71CVpA4Z7pLUodM63JNcnuQbSaaSbF/u+SyHJB9NciTJ14Zq5yTZm+SRdn/2cs5xqSVZn+TuJA8neSjJO1t9xR6XJGcl+WKSr7Rj8oFWvyDJve1n6BPtIogVJckZSe5P8pm23cUxOW3DfehjDq4ALgLenuSi5Z3Vsvhn4PJjatuBfVW1EdjXtleS54D3VNVFwCbg+vZvYyUflx8Bl1XVa4CLgcuTbAJuBG6qqguBp4BtyzjH5fJOYP/QdhfH5LQNd4Y+5qCqfgwc/ZiDFaWqvgB895jyFmBXa+8CrlrSSS2zqjpcVV9u7e8z+MFdywo+LjXwg7b5knYr4DLgjlZfUccEIMk64M3AR9p26OSYnM7hPtPHHKxdprm82KypqsOt/TiwZjkns5ySbABeC9zLCj8ubfnhAeAIsBf4JvB0VT3XhqzEn6EPAX8J/Kxtn0snx+R0DneNoAbXuq7I612T/ALwSeBdVfW94b6VeFyq6qdVdTGDd41fCrx6mae0rJK8BThSVfct91wWw+n8TUx+zMGJPZHk/Ko6nOR8BmdqK0qSlzAI9o9V1adaecUfF4CqejrJ3cDrgNVJVrUz1ZX2M/R64K1JrgTOAn6JwfdQdHFMTuczdz/m4MT2AFtbeyuwexnnsuTauulOYH9VfXCoa8UelyRjSVa39ssYfN/CfuBu4G1t2Io6JlX1V1W1rqo2MMiPu6rqHXRyTE7rd6i237gf4vmPOfi7ZZ7SkktyG/BGBh9T+gTwfuDfgduBVwKPAVdX1bF/dO1WkjcA/wV8lefXUt/HYN19RR6XJL/B4I+DZzA4qbu9qv4mya8xuBjhHOB+4A+r6kfLN9PlkeSNwF9U1Vt6OSandbhLkmZ2Oi/LSJJOwHCXpA4Z7pLUIcNdkjpkuEtShwx3SeqQ4S5JHfp/paL2i7eBPXAAAAAASUVORK5CYII=\n", 23 | "text/plain": [ 24 | "
" 25 | ] 26 | }, 27 | "metadata": { 28 | "needs_background": "light" 29 | }, 30 | "output_type": "display_data" 31 | } 32 | ], 33 | "source": [ 34 | "%matplotlib inline\n", 35 | "import numpy as np\n", 36 | "from matplotlib import pyplot as plt\n", 37 | "\n", 38 | "\n", 39 | "T = (np.random.exponential(size=1000)/1.5) ** 2.3\n", 40 | "\n", 41 | "plt.hist(T, bins=50);" 42 | ] 43 | }, 44 | { 45 | "cell_type": "markdown", 46 | "metadata": {}, 47 | "source": [ 48 | "Let's use the Weibull parametric model: https://en.wikipedia.org/wiki/Weibull_distribution" 49 | ] 50 | }, 51 | { 52 | "cell_type": "code", 53 | "execution_count": 2, 54 | "metadata": {}, 55 | "outputs": [], 56 | "source": [ 57 | "def pdf(params, t):\n", 58 | " lambda_, rho_ = params\n", 59 | " # I'm not going to make you type this out - it's annoying and error prone. \n", 60 | " return rho_ / lambda_ * (t / lambda_) ** (rho_ - 1) * np.exp(-(t/lambda_) ** rho_)\n", 61 | "\n", 62 | "# okay, but we actually need the _log_ of the pdf\n", 63 | "def log_pdf(params, t):\n", 64 | " lambda_, rho_ = params\n", 65 | " # I'm not going to make you type this out - it's annoying and error prone. \n", 66 | " return np.log(rho_) - np.log(lambda_) + (rho_ - 1) * (np.log(t) - np.log(lambda_)) - (t/lambda_) ** rho_\n", 67 | "\n", 68 | "# now we can define the log likehood\n", 69 | "\n", 70 | "def log_likelihood(params, t):\n", 71 | " return np.sum(log_pdf(params, t)) \n" 72 | ] 73 | }, 74 | { 75 | "cell_type": "markdown", 76 | "metadata": {}, 77 | "source": [ 78 | "## `scipy.optimize.minimize`" 79 | ] 80 | }, 81 | { 82 | "cell_type": "code", 83 | "execution_count": 3, 84 | "metadata": { 85 | "slideshow": { 86 | "slide_type": "-" 87 | } 88 | }, 89 | "outputs": [ 90 | { 91 | "name": "stdout", 92 | "output_type": "stream", 93 | "text": [ 94 | " fun: nan\n", 95 | " hess_inv: array([[1, 0],\n", 96 | " [0, 1]])\n", 97 | " jac: array([nan, nan])\n", 98 | " message: 'Desired error not necessarily achieved due to precision loss.'\n", 99 | " nfev: 448\n", 100 | " nit: 1\n", 101 | " njev: 112\n", 102 | " status: 2\n", 103 | " success: False\n", 104 | " x: array([ -29.10248229, 1034.80182732])\n" 105 | ] 106 | }, 107 | { 108 | "name": "stderr", 109 | "output_type": "stream", 110 | "text": [ 111 | "/Users/camerondavidson-pilon/venvs/data/lib/python3.7/site-packages/ipykernel_launcher.py:10: RuntimeWarning: invalid value encountered in log\n", 112 | " # Remove the CWD from sys.path while we load stuff.\n", 113 | "/Users/camerondavidson-pilon/venvs/data/lib/python3.7/site-packages/ipykernel_launcher.py:10: RuntimeWarning: invalid value encountered in power\n", 114 | " # Remove the CWD from sys.path while we load stuff.\n", 115 | "/Users/camerondavidson-pilon/venvs/data/lib/python3.7/site-packages/ipykernel_launcher.py:10: RuntimeWarning: invalid value encountered in log\n", 116 | " # Remove the CWD from sys.path while we load stuff.\n", 117 | "/Users/camerondavidson-pilon/venvs/data/lib/python3.7/site-packages/ipykernel_launcher.py:10: RuntimeWarning: invalid value encountered in power\n", 118 | " # Remove the CWD from sys.path while we load stuff.\n", 119 | "/Users/camerondavidson-pilon/venvs/data/lib/python3.7/site-packages/ipykernel_launcher.py:10: RuntimeWarning: invalid value encountered in log\n", 120 | " # Remove the CWD from sys.path while we load stuff.\n", 121 | "/Users/camerondavidson-pilon/venvs/data/lib/python3.7/site-packages/ipykernel_launcher.py:10: RuntimeWarning: invalid value encountered in power\n", 122 | " # Remove the CWD from sys.path while we load stuff.\n" 123 | ] 124 | } 125 | ], 126 | "source": [ 127 | "from scipy.optimize import minimize\n", 128 | "\n", 129 | "results = minimize(log_likelihood, \n", 130 | " x0 = np.array([1.0, 1.0]), # some initial guess of the parameters. \n", 131 | " method=None, \n", 132 | " args=(T, ))\n", 133 | "\n", 134 | "print(results)" 135 | ] 136 | }, 137 | { 138 | "cell_type": "code", 139 | "execution_count": 4, 140 | "metadata": {}, 141 | "outputs": [], 142 | "source": [ 143 | "def log_likelihood(params, t):\n", 144 | " return -np.sum(log_pdf(params, t)) \n" 145 | ] 146 | }, 147 | { 148 | "cell_type": "code", 149 | "execution_count": 5, 150 | "metadata": {}, 151 | "outputs": [ 152 | { 153 | "name": "stdout", 154 | "output_type": "stream", 155 | "text": [ 156 | " fun: nan\n", 157 | " hess_inv: array([[1, 0],\n", 158 | " [0, 1]])\n", 159 | " jac: array([nan, nan])\n", 160 | " message: 'Desired error not necessarily achieved due to precision loss.'\n", 161 | " nfev: 448\n", 162 | " nit: 1\n", 163 | " njev: 112\n", 164 | " status: 2\n", 165 | " success: False\n", 166 | " x: array([ 31.10248229, -1032.80182732])\n" 167 | ] 168 | }, 169 | { 170 | "name": "stderr", 171 | "output_type": "stream", 172 | "text": [ 173 | "/Users/camerondavidson-pilon/venvs/data/lib/python3.7/site-packages/ipykernel_launcher.py:10: RuntimeWarning: invalid value encountered in log\n", 174 | " # Remove the CWD from sys.path while we load stuff.\n", 175 | "/Users/camerondavidson-pilon/venvs/data/lib/python3.7/site-packages/ipykernel_launcher.py:10: RuntimeWarning: overflow encountered in power\n", 176 | " # Remove the CWD from sys.path while we load stuff.\n", 177 | "/Users/camerondavidson-pilon/venvs/data/lib/python3.7/site-packages/ipykernel_launcher.py:10: RuntimeWarning: invalid value encountered in log\n", 178 | " # Remove the CWD from sys.path while we load stuff.\n", 179 | "/Users/camerondavidson-pilon/venvs/data/lib/python3.7/site-packages/ipykernel_launcher.py:10: RuntimeWarning: overflow encountered in power\n", 180 | " # Remove the CWD from sys.path while we load stuff.\n", 181 | "/Users/camerondavidson-pilon/venvs/data/lib/python3.7/site-packages/ipykernel_launcher.py:10: RuntimeWarning: invalid value encountered in log\n", 182 | " # Remove the CWD from sys.path while we load stuff.\n", 183 | "/Users/camerondavidson-pilon/venvs/data/lib/python3.7/site-packages/ipykernel_launcher.py:10: RuntimeWarning: overflow encountered in power\n", 184 | " # Remove the CWD from sys.path while we load stuff.\n" 185 | ] 186 | } 187 | ], 188 | "source": [ 189 | "results = minimize(log_likelihood, \n", 190 | " x0 = np.array([1.0, 1.0]), # some initial guess of the parameters. \n", 191 | " method=None, \n", 192 | " args=(T, ))\n", 193 | "\n", 194 | "print(results)" 195 | ] 196 | }, 197 | { 198 | "cell_type": "code", 199 | "execution_count": 6, 200 | "metadata": {}, 201 | "outputs": [ 202 | { 203 | "name": "stdout", 204 | "output_type": "stream", 205 | "text": [ 206 | " fun: nan\n", 207 | " hess_inv: <2x2 LbfgsInvHessProduct with dtype=float64>\n", 208 | " jac: array([ 59.76914963, 2616.4018891 ])\n", 209 | " message: b'ABNORMAL_TERMINATION_IN_LNSRCH'\n", 210 | " nfev: 129\n", 211 | " nit: 1\n", 212 | " status: 2\n", 213 | " success: False\n", 214 | " x: array([1.16054112, 0.99805959])\n" 215 | ] 216 | }, 217 | { 218 | "name": "stderr", 219 | "output_type": "stream", 220 | "text": [ 221 | "/Users/camerondavidson-pilon/venvs/data/lib/python3.7/site-packages/ipykernel_launcher.py:10: RuntimeWarning: divide by zero encountered in log\n", 222 | " # Remove the CWD from sys.path while we load stuff.\n", 223 | "/Users/camerondavidson-pilon/venvs/data/lib/python3.7/site-packages/ipykernel_launcher.py:10: RuntimeWarning: invalid value encountered in add\n", 224 | " # Remove the CWD from sys.path while we load stuff.\n", 225 | "/Users/camerondavidson-pilon/venvs/data/lib/python3.7/site-packages/ipykernel_launcher.py:10: RuntimeWarning: divide by zero encountered in true_divide\n", 226 | " # Remove the CWD from sys.path while we load stuff.\n", 227 | "/Users/camerondavidson-pilon/venvs/data/lib/python3.7/site-packages/ipykernel_launcher.py:10: RuntimeWarning: invalid value encountered in double_scalars\n", 228 | " # Remove the CWD from sys.path while we load stuff.\n" 229 | ] 230 | } 231 | ], 232 | "source": [ 233 | "# weibull parameters must be greater than 0!\n", 234 | "# we can \"nudge\" the minimizer to understand this using the bounds argument\n", 235 | "results = minimize(log_likelihood, \n", 236 | " x0 = np.array([1.0, 1.0]),\n", 237 | " method=None, \n", 238 | " args=(T, ),\n", 239 | " bounds=((0, None), (0, None)))\n", 240 | "\n", 241 | "print(results)" 242 | ] 243 | }, 244 | { 245 | "cell_type": "markdown", 246 | "metadata": {}, 247 | "source": [ 248 | "## Takeaway: `minimize` is a very flexible function for small-mid size parameter optimization" 249 | ] 250 | }, 251 | { 252 | "cell_type": "markdown", 253 | "metadata": {}, 254 | "source": [ 255 | "### A few problems though:\n", 256 | "\n", 257 | "1. You are stuck with only using known parametric models that are easy to implement. \n", 258 | "2. Not very fast minimization routines\n", 259 | "\n", 260 | "Let's move to Part 2. " 261 | ] 262 | } 263 | ], 264 | "metadata": { 265 | "kernelspec": { 266 | "display_name": "Python 3", 267 | "language": "python", 268 | "name": "python3" 269 | }, 270 | "language_info": { 271 | "codemirror_mode": { 272 | "name": "ipython", 273 | "version": 3 274 | }, 275 | "file_extension": ".py", 276 | "mimetype": "text/x-python", 277 | "name": "python", 278 | "nbconvert_exporter": "python", 279 | "pygments_lexer": "ipython3", 280 | "version": "3.7.3" 281 | } 282 | }, 283 | "nbformat": 4, 284 | "nbformat_minor": 2 285 | } 286 | -------------------------------------------------------------------------------- /full_notebooks/Part 2.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "### Introducing censoring\n", 8 | "\n", 9 | "![img](https://lifelines.readthedocs.io/en/latest/_images/survival_analysis_intro_censoring.png)\n", 10 | "\n", 11 | "\n", 12 | "![img2](https://lifelines.readthedocs.io/en/latest/_images/survival_analysis_intro_censoring_revealed.png)" 13 | ] 14 | }, 15 | { 16 | "cell_type": "markdown", 17 | "metadata": {}, 18 | "source": [ 19 | "All we know is that actual lifetime is greater than some threshold. Mathematically, we know $P(T \\ge t) = 1 - F(t) := S(t)$. We can use this in our log likelihood:\n", 20 | "\n", 21 | "No censoring cases:\n", 22 | "\n", 23 | "$$l(\\theta, t) = \\sum_{\\text{observed}} \\log{\\text{pdf}(\\theta, t)} $$\n", 24 | "\n", 25 | "With censoring cases:\n", 26 | "$$ \n", 27 | "\\begin{align}\n", 28 | "l(\\theta, t) & = \\sum_{\\text{observed}} \\log{\\text{pdf}(t, \\theta)} + \\sum_{\\text{censored}} \\log{\\text{S}(t, \\theta)} \\\\\n", 29 | "& = \\sum_{\\text{observed}} \\log{\\text{pdf}(t, \\theta)} \\frac{S(t)}{S(t)} + \\sum_{\\text{censored}} \\log{\\text{S}(t, \\theta)} \\\\\n", 30 | "& = \\sum_{\\text{observed}} (\\log{\\frac{\\text{pdf}(t, \\theta)}{S(t)}} + \\log{S(t)}) + \\sum_{\\text{censored}} \\log{\\text{S}(t, \\theta)} \\\\\n", 31 | "& = \\sum_{\\text{observed}} \\log{\\frac{\\text{pdf}(t, \\theta)}{S(t)}} + \\sum_{\\text{observed}} \\log{S(t)} + \\sum_{\\text{censored}} \\log{\\text{S}(t, \\theta)} \\\\\n", 32 | "& = \\sum_{\\text{observed}} \\log{\\frac{\\text{pdf}(t, \\theta)}{S(t)}} + \\sum \\log{S(t)} \n", 33 | "\\end{align}\n", 34 | "$$\n", 35 | "\n", 36 | "\n", 37 | "\n", 38 | "The $-\\log{S(t)}$ is known as the _cumulative hazard_, denoted $H(t)$. \n", 39 | "\n", 40 | "$$l(\\theta, t) = \\sum_{\\text{observed}} \\log{\\frac{\\text{pdf}(t, \\theta)}{S(t)}} - \\sum H(t, \\theta) $$\n", 41 | "\n", 42 | "Also, $\\frac{dH}{dt} = \\frac{\\text{pdf}(t, \\theta)}{S(t)}$. Denote that $h(t)$. \n", 43 | "\n", 44 | "$$l(\\theta, t) = \\sum_{\\text{observed}} \\log{h(t, \\theta}) - \\sum H(t, \\theta) $$" 45 | ] 46 | }, 47 | { 48 | "cell_type": "markdown", 49 | "metadata": {}, 50 | "source": [ 51 | "Phew! Now, instead of working in probability space, we will work in hazard space! Here's a link to all the relatioships: https://lifelines.readthedocs.io/en/latest/Survival%20Analysis%20intro.html#hazard-function \n", 52 | "\n", 53 | "\n", 54 | "## Take away: the likelihood function can be used to \"add\" information about the system (think about how penalizers are used...)" 55 | ] 56 | }, 57 | { 58 | "cell_type": "code", 59 | "execution_count": 1, 60 | "metadata": {}, 61 | "outputs": [], 62 | "source": [ 63 | "# the hazard and cumulative hazard for Weibull are much simplier to implement 😗👌\n", 64 | "\n", 65 | "def cumulative_hazard(params, t):\n", 66 | " lambda_, rho_ = params\n", 67 | " return (t / lambda_) ** rho_\n", 68 | "\n", 69 | "def hazard(params, t):\n", 70 | " # diff of cumulative hazard w.r.t. t\n", 71 | " lambda_, rho_ = params\n", 72 | " return rho_ / lambda_ * (t / lambda_) ** (rho_ - 1)\n", 73 | "\n", 74 | "def log_hazard(params, t):\n", 75 | " lambda_, rho_ = params\n", 76 | " return np.log(rho_) - np.log(lambda_) + (rho_ - 1) * (np.log(t) - np.log(lambda_))\n", 77 | "\n", 78 | "\n", 79 | "\n", 80 | "def log_likelihood(params, t, e):\n", 81 | " return np.sum(e * log_hazard(params, t)) - np.sum(cumulative_hazard(params, t))\n" 82 | ] 83 | }, 84 | { 85 | "cell_type": "code", 86 | "execution_count": 2, 87 | "metadata": {}, 88 | "outputs": [ 89 | { 90 | "name": "stdout", 91 | "output_type": "stream", 92 | "text": [ 93 | " fun: -inf\n", 94 | " hess_inv: <2x2 LbfgsInvHessProduct with dtype=float64>\n", 95 | " jac: array([ 39.23116765, -2483.29052965])\n", 96 | " message: b'ABNORMAL_TERMINATION_IN_LNSRCH'\n", 97 | " nfev: 63\n", 98 | " nit: 0\n", 99 | " status: 2\n", 100 | " success: False\n", 101 | " x: array([1., 1.])\n" 102 | ] 103 | }, 104 | { 105 | "name": "stderr", 106 | "output_type": "stream", 107 | "text": [ 108 | "/Users/camerondavidson-pilon/venvs/data/lib/python3.7/site-packages/ipykernel_launcher.py:5: RuntimeWarning: overflow encountered in power\n", 109 | " \"\"\"\n", 110 | "/Users/camerondavidson-pilon/venvs/data/lib/python3.7/site-packages/scipy/optimize/optimize.py:670: RuntimeWarning: invalid value encountered in double_scalars\n", 111 | " grad[k] = (f(*((xk + d,) + args)) - f0) / d[k]\n" 112 | ] 113 | } 114 | ], 115 | "source": [ 116 | "T = (np.random.exponential(size=1000)/1.5) ** 2.3\n", 117 | "E = np.random.binomial(1, 0.95, size=1000)\n", 118 | "\n", 119 | "from scipy.optimize import minimize\n", 120 | "\n", 121 | "results = minimize(log_likelihood, \n", 122 | " x0 = np.array([1.0, 1.0]),\n", 123 | " method=None, \n", 124 | " args=(T, E),\n", 125 | " bounds=((0.00001, None), (0.00001, None)))\n", 126 | "\n", 127 | "print(results)" 128 | ] 129 | }, 130 | { 131 | "cell_type": "code", 132 | "execution_count": 3, 133 | "metadata": {}, 134 | "outputs": [ 135 | { 136 | "name": "stdout", 137 | "output_type": "stream", 138 | "text": [ 139 | " fun: nan\n", 140 | " hess_inv: <2x2 LbfgsInvHessProduct with dtype=float64>\n", 141 | " jac: array([ 85.23691122, 2313.87556369])\n", 142 | " message: b'ABNORMAL_TERMINATION_IN_LNSRCH'\n", 143 | " nfev: 129\n", 144 | " nit: 1\n", 145 | " status: 2\n", 146 | " success: False\n", 147 | " x: array([1.1570978 , 0.99599559])\n" 148 | ] 149 | }, 150 | { 151 | "name": "stderr", 152 | "output_type": "stream", 153 | "text": [ 154 | "/Users/camerondavidson-pilon/venvs/data/lib/python3.7/site-packages/ipykernel_launcher.py:14: RuntimeWarning: divide by zero encountered in log\n", 155 | " \n", 156 | "/Users/camerondavidson-pilon/venvs/data/lib/python3.7/site-packages/ipykernel_launcher.py:14: RuntimeWarning: invalid value encountered in double_scalars\n", 157 | " \n", 158 | "/Users/camerondavidson-pilon/venvs/data/lib/python3.7/site-packages/ipykernel_launcher.py:5: RuntimeWarning: divide by zero encountered in true_divide\n", 159 | " \"\"\"\n", 160 | "/Users/camerondavidson-pilon/venvs/data/lib/python3.7/site-packages/ipykernel_launcher.py:19: RuntimeWarning: invalid value encountered in multiply\n", 161 | "/Users/camerondavidson-pilon/venvs/data/lib/python3.7/site-packages/ipykernel_launcher.py:14: RuntimeWarning: invalid value encountered in add\n", 162 | " \n" 163 | ] 164 | } 165 | ], 166 | "source": [ 167 | "def negative_log_likelihood(params, t, e):\n", 168 | " return -log_likelihood(params, t, e)\n", 169 | "\n", 170 | "results = minimize(negative_log_likelihood, \n", 171 | " x0 = np.array([1.0, 1.0]),\n", 172 | " method=None, \n", 173 | " args=(T, E),\n", 174 | " bounds=((0., None), (0., None)))\n", 175 | "\n", 176 | "print(results)" 177 | ] 178 | }, 179 | { 180 | "cell_type": "code", 181 | "execution_count": 4, 182 | "metadata": {}, 183 | "outputs": [ 184 | { 185 | "data": { 186 | "text/plain": [ 187 | "Text(0.5, 1.0, 'Estimated survival function')" 188 | ] 189 | }, 190 | "execution_count": 4, 191 | "metadata": {}, 192 | "output_type": "execute_result" 193 | }, 194 | { 195 | "data": { 196 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEICAYAAABPgw/pAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deZwcdZ3/8ddneq5M5koyR5KZXJBzchEYCBJFVMDAAmFXRFARlYX9reJ6/XQ99geK/lx1XZVV1hVBQLlFWYMbBEEuIUBCEghJSDI5J/fknMwkc3/2j6qEzjjJTJKeqenu9/Px6Ed3VX276lM9yburv3WZuyMiIskvI+oCREQkMRToIiIpQoEuIpIiFOgiIilCgS4ikiIU6CIiKUKBLl0ys3eZ2cqo6+iKmZ1nZpv6QR3LzOy8BMxnvZmdf5RpA8zsMTPbZ2a/OdllHWddCVk/6TuZURcgiWVm64FyoD1u9N3ufmM373NgnLvXALj7C8CEXqrxbmCTu/9Lb8y/r7j75D5YzBUEf88h7t7WWwvp6m/SR+snCaRAT02XuvtTUReRzMwsszcD9DiMAlb1k1qkn1OXSxoxs7Fm9lz4832nmT0Ujn8+bPK6mTWY2Yc6d2uE3QJfMrM3zKzRzO40s3Ize9zM9pvZU2Y2KK79b8xsW7is581scjj+BuAjwJfDZT0Wjh9uZr81szozW2dm/xQ3rwFmdreZ7TGz5cCZx1hHM7MfmdkOM6s3s6VmNiWc9qyZ/X1c24+b2V/iht3MPm1mq4HVZvYzM/tBp/n/3sy+EPeZnB/WftDMBse1mxF+xllmdqqZ/dnMdoXj7jOz4h78vb4J3AR8KPysrjOzb5jZvXFtRod1Z8at47fM7MXw7/KkmZXEtX+nmb1kZnvNrDb8DI72NzncFWRmOWb2YzPbEj5+bGY54bTzzGyTmX0x/Ny3mtknuls/STwFenr5FvAkMAioBH4C4O7nhtOnu3u+uz90lPd/ALgAGA9cCjwOfA0oJfi39E9xbR8HxgFlwCLgvnBZt4evvx8u61IzywAeA14HKoD3AZ8zs/eH87oZODV8vB+49hjreCFwblhjEXAlsOuYn8qRLgdmAlXAAwRhagDhF9aFwIPxb3D3LcB8gs/nkA8Dj7h7K2DAvwLDgUnACOAb3RXi7jcD3wEeCj+rO3u4Dh8GPkHw2WcD/zesfxTB3+UnBH+z04AlXf1Nupjn14Gzw/dMB84C4rvMhhJ83hXAdcBt8V/w0jcU6Knpv8MtsEOP68PxrQQ/4Ye7e5O7/+UY8+jKT9x9u7tvBl4AXnH3xe7eBDwKzDjU0N1/6e773b2ZILymm1nRUeZ7JlDq7re4e4u7rwV+AVwVTr8S+P/uvtvda4H/OEaNrUABMBEwd1/h7luPYx3/NVzOwXAdHXhXOO0KYH4Y4J3dD1wNwa+EsPb7Ady9xt3/5O7N7l4H/BB493HUdLzucvdV4To8TBDCEAT9U+7+gLu3uvsud1/Sw3l+BLjF3XeE6/BN4Jq46a3h9FZ3nwc00Ev7YOToFOip6XJ3L457/CIc/2WCrcVXLTiC4ZPHOd/tca8PdjGcD2BmMTP7rpmtMbN6YH3YpoSujQKGx38JEWz5l4fThwO1ce03HK1Ad/8z8FPgNmCHmd1uZoU9WrvA4eV4cOW6BwmDmiAQ7zvK+34LvMPMhhH8Qugg+EIg7Jp60Mw2h5/HvRz9s0iEbXGvDxD+XQh+Gaw5wXkO58jPfUM47pBdnfr545crfUSBnkbcfZu7X+/uw4F/AP7TzMb2wqI+DMwBzif4GT46HG+HSunUvhZY1+lLqMDdLw6nbyUIo0NGHmvh7v4f7n4GQbfJeOBL4aRGIC+u6dCu3t5p+AHgirC7YiZBcHe1zD0E3VkfIlj/B/3tS5l+J5zvVHcvBD7K25/F8erJOhxNLUG3VVe6u+zqFoIv3kNGhuOkH1GgpxEz+6CZVYaDewj+E3eEw9uBUxK0qAKgmaDvOo8g0OJ1XtarwH4z++dwB2jMzKaY2aGdnw8DXzWzQWH9nznags3sTDObaWZZBOHXxNvruAT4OzPLC7/IrutuRdx9MbATuAN4wt33HqP5/cDHCLpm7o8bX0DQBbHPzCp4+wvmRCwBzjWzkWEX1leP4733Aeeb2ZVmlmlmQ8zsUHdMd3//B4B/MbPScCfrTQS/NKQfUaCnpsfCoxUOPR4Nx58JvGJmDcBc4LNhfzUE/dz3hF0eV57k8n9F8JN8M7AceLnT9DuBqnBZ/+3u7cAlBH2963g7QA/1uX8znN86gq3gXx9j2YUE/e97wvfsAv4tnPYjoIUgvO7h6N0nnd1P8Gvj/m7azSXYEbzN3V+PG/9N4HRgH/A/wO96uNy/4u5/Ah4C3gBeA/5wHO/dCFwMfBHYTfDlMD2cfMTfpIu3fxtYGC53KcGO7m+f4GpILzHd4EJEJDVoC11EJEV0G+hm9svwZIE3jzLdzOw/zKzGgpNOTk98mSIi0p2ebKHfDcw+xvSLCPoNxwE3AD87+bJEROR4dRvo7v48wQ6Uo5kD/MoDLwPF4bG4IiLShxJxca4KjjzpY1M47q/OzguvGXEDwMCBA8+YOHHicS9sZ0MzW/c1MWlYIZkZJ3oor4hIcnrttdd2untpV9P69GqL4TUjbgeorq72hQsXHvc8nl25g4/ftYCf33A2M08ZkugSRUT6NTM76pnSiTjKZTNHnsVXGY7rFePLCwBYtaOhtxYhIpKUEhHoc4GPhUe7nA3sO86LIR2XYUW5FORksnr7/t5ahIhIUuq2y8XMHgDOA0osuD72zUAWgLv/FzCP4OyzGoIL8vTqdZDNjLHl+axSoIuIHKHbQHf3q7uZ7sCnE1ZRD4wvK+CpFdu7bygikkaS8kzRceX57GpsYVdDc9SliIj0G0kZ6Id3jG7XjlERkUOSOtBX71A/uojIIUkZ6OWFORTkZrJaW+giIoclZaCbGePKdKSLiEi8pAx0CLpdVuvkIhGRw5I20MeVF7C7sYWdOtJFRARI4kAfXx7cUFzdLiIigSQO9PBIF+0YFREBkjjQywpyKMzN1Ba6iEgoaQPdzIIdo9pCFxEBkjjQIdgxumrHfoLLyYiIpLekDvTx5fnsPdBKnY50ERFJ7kCfMDTYMfrWVvWji4gkdaBXDSsEYPnW+ogrERGJXlIHenFeNsOLclmhQBcRSe5AB5g0rFCBLiJCigT6mrpGmlrboy5FRCRSKRHo7R1OjS7UJSJpLgUCPTjSRTtGRSTdJX2gjxoykAFZMfWji0jaS/pAj2UYE4YWKNBFJO0lfaBD0I++fEu9LgEgImktJQK9angh9U1tbNnXFHUpIiKRSY1AD3eMrtiibhcRSV8pEegThgaXAFA/uoiks5QI9PycTEYNyWPFNgW6iKSvlAh0gElDC1mhqy6KSBpLnUAfVsj6XY0caGmLuhQRkUikTKBXDS/EXf3oIpK+UibQp1YUAbB0076IKxERiUbKBHp5YQ4l+Tks3awtdBFJTykT6GbGtMoilm7eG3UpIiKRSJlAB5hSUUTNjgbtGBWRtJRSgT6toogOh+U6Y1RE0lCPAt3MZpvZSjOrMbOvdDF9pJk9Y2aLzewNM7s48aV2b2pluGN0s3aMikj66TbQzSwG3AZcBFQBV5tZVadm/wI87O4zgKuA/0x0oT1RXphLWUGOjnQRkbTUky30s4Aad1/r7i3Ag8CcTm0cKAxfFwFbElfi8ZlaUaQtdBFJSz0J9AqgNm54Uzgu3jeAj5rZJmAe8JmuZmRmN5jZQjNbWFdXdwLldm9KRRE1dQ00NmvHqIikl0TtFL0auNvdK4GLgV+b2V/N291vd/dqd68uLS1N0KKPNK2yCHfdY1RE0k9PAn0zMCJuuDIcF+864GEAd58P5AIliSjweB06Y/QN9aOLSJrpSaAvAMaZ2RgzyybY6Tm3U5uNwPsAzGwSQaD3Tp9KN8oKcykvzOFN9aOLSJrpNtDdvQ24EXgCWEFwNMsyM7vFzC4Lm30RuN7MXgceAD7uEd7gc2pFEW9s0hmjIpJeMnvSyN3nEezsjB93U9zr5cCsxJZ24qZWFPP0WztoaG4jP6dHqygikvRS6kzRQ6ZWBpfSVbeLiKSTlAz06ZXFACypVbeLiKSPlAz0Ifk5jBqSx+KNe6IuRUSkz6RkoAOcPnIQizbuJcJ9syIifSplA33GyGLq9jezee/BqEsREekTqRvoIwYBsHij+tFFJD2kbKBPHFZAblaGAl1E0kbKBnpWLINpFcUsrtWOURFJDykb6BD0oy/bXE9zW3vUpYiI9LoUD/RBtLR3sEy3pBORNJDigR6cYKR+dBFJBykd6OWFuVQUD2CRTjASkTSQ0oEOwVb6Em2hi0gaSINAH8TmvQfZXt8UdSkiIr0qDQI96EdftEHdLiKS2lI+0KcMLyInM4MF6xXoIpLaUj7QszMzmDGymAXrd0ddiohIr0r5QAc4a8wQlm3Zx/6m1qhLERHpNekR6KMH0+GwSEe7iEgKS4tAP31UMZkZxqvrdkVdiohIr0mLQM/LzmRyRREL1mnHqIikrrQIdICZYwazpHYvTa26UJeIpKa0CfQzRw+mpb2DNzbti7oUEZFekUaBHtzBSP3oIpKq0ibQi/OymVBewKs6wUhEUlTaBDrAWWMG89r63bS1d0RdiohIwqVVoJ85ZjCNLe2s2Lo/6lJERBIurQJ95pjBALy8Vv3oIpJ60irQywtzOaV0IC+t2Rl1KSIiCZdWgQ4w69QSXlm3m1b1o4tIikm/QB87hAMt7Syp1XVdRCS1pF2gn33KEMzgxRp1u4hIakm7QC/Oy2ZqRREv1WjHqIiklrQLdIBzTi1hce0eDrS0RV2KiEjC9CjQzWy2ma00sxoz+8pR2lxpZsvNbJmZ3Z/YMhNr1tghtLY7r67TXYxEJHV0G+hmFgNuAy4CqoCrzayqU5txwFeBWe4+GfhcL9SaMNWjBpMdy+ClNep2EZHU0ZMt9LOAGndf6+4twIPAnE5trgduc/c9AO6+I7FlJtaA7BinjyrmL6u1Y1REUkdPAr0CqI0b3hSOizceGG9mL5rZy2Y2u6sZmdkNZrbQzBbW1dWdWMUJMuvUEpZvrWd3Y0ukdYiIJEqidopmAuOA84CrgV+YWXHnRu5+u7tXu3t1aWlpghZ9YmaNKwFgvrpdRCRF9CTQNwMj4oYrw3HxNgFz3b3V3dcBqwgCvt+aVlFEQW4mL6yO9peCiEii9CTQFwDjzGyMmWUDVwFzO7X5b4Ktc8yshKALZm0C60y4zFgG7xpXwrMr63D3qMsRETlp3Qa6u7cBNwJPACuAh919mZndYmaXhc2eAHaZ2XLgGeBL7t7v+zLOG1/Gtvom3tqmy+mKSPLL7Ekjd58HzOs07qa41w58IXwkjXdPCPrxn11Zx6RhhRFXIyJyctLyTNFDygtzqRpWyLMr+/VRliIiPZLWgQ5w3oRSFm7YQ31Ta9SliIicFAX6hDLaO5wXdZKRiCS5tA/000cWU5CbyTPqdhGRJJf2gZ4Zy+DccaU8t0qHL4pIckv7QIfgaJft9c2s2KrDF0UkeSnQgfPGB4cvqttFRJKZAh0oK8xlWmURT63YHnUpIiInTIEeurCqnMUb97KjvinqUkRETogCPXTh5KEA/Elb6SKSpBTooXFl+YwekseTyxToIpKcFOghM+PCyUN5ac1O9uusURFJQgr0OBdWldPa7jy7UtdIF5Hko0CPM2PkIErys3lyubpdRCT5KNDjxDKM8yeV88xbO2hua4+6HBGR46JA7+TCyeU0NLfpXqMiknQU6J2cc2oJA7NjPKGjXUQkySjQO8nNivHeSeU8sWwbbe0dUZcjItJjCvQuXDJtGLsbW3hJ3S4ikkQU6F149/hSCnIyeez1LVGXIiLSYwr0LuRmxbhgctDt0tKmbhcRSQ4K9KO4dPpw6pvaeGG1TjISkeSgQD+Kd44toTgvS90uIpI0FOhHkRXLYPbkofxp+XaaWnWSkYj0fwr0Y7h0+nAaW9p5VncyEpEkoEA/hpljBlOSn81cdbuISBJQoB9DZiyDS6YN56nlO9h3QJfUFZH+TYHejSvOqKSlvYPH3tBWuoj0bwr0bkweXsjEoQU88tqmqEsRETkmBXo3zIwPnF7Jktq91OxoiLocEZGjUqD3wJwZw4llGL9dpK10Eem/FOg9UFaQy7vHl/Loos20d3jU5YiIdEmB3kNXnFHJtvomXlqzM+pSRES6pEDvofdNKqNoQBa/WahuFxHpnxToPZSTGWPOacP547Jt7GlsibocEZG/0qNAN7PZZrbSzGrM7CvHaPcBM3Mzq05cif3HR2aOoqWtQztHRaRf6jbQzSwG3AZcBFQBV5tZVRftCoDPAq8kusj+YsLQAs4cPYj7XtlIh3aOikg/05Mt9LOAGndf6+4twIPAnC7afQv4HtCUwPr6nY/MHMW6nY3MX6vb04lI/9KTQK8AauOGN4XjDjOz04ER7v4/x5qRmd1gZgvNbGFdXXLeOGL2lKEMysvivlc2RF2KiMgRTnqnqJllAD8EvthdW3e/3d2r3b26tLT0ZBcdidysGB+sHsGTy7azoz6lf4yISJLpSaBvBkbEDVeG4w4pAKYAz5rZeuBsYG6q7hgFuPqskbR1OA8vrO2+sYhIH+lJoC8AxpnZGDPLBq4C5h6a6O773L3E3Ue7+2jgZeAyd1/YKxX3A2NKBvLOsSXc/8pG2tp1E2kR6R+6DXR3bwNuBJ4AVgAPu/syM7vFzC7r7QL7q4+fM5ot+5p4/M1tUZciIgJAZk8aufs8YF6ncTcdpe15J19W//feiWWMKRnIHX9ZxyXThmFmUZckImlOZ4qeoIwM45OzRvN67V4WbdwTdTkiIgr0k/GBMyopGpDFHS+si7oUEREF+snIy87k6rNG8sSybdTuPhB1OSKS5hToJ+nac0aRYcZdL66PuhQRSXMK9JM0rGgAl0wbxkMLNrL3gK7CKCLRUaAnwD+8+1QaW9q1lS4ikVKgJ8CkYYVcUFXOXS+uY39Ta9TliEiaUqAnyI3vGUt9Uxv3vrwx6lJEJE0p0BNk+ohizh1fyh0vrOVgS3vU5YhIGlKgJ9Bn3juWXY0tPPCqttJFpO8p0BPozNGDmTlmMD9/fg1NrdpKF5G+pUBPsM+eP47t9c3c+7JugCEifUuBnmDnnFrCu8aVcNszNTriRUT6lAK9F3zp/RPYc6CVX+gaLyLShxTovWBaZTEXTx3KnS+sZVdDc9TliEiaUKD3ki9cMIGDre3c9syaqEsRkTShQO8lY8vy+eAZI7j35Q26EqOI9AkFei/6/AXjiWUY35m3IupSRCQNKNB70dCiXD513qk8/uY25q/ZFXU5IpLiFOi97PpzT6GieAC3/GE57R0edTkiksIU6L0sNyvG1y6exIqt9Ty0oDbqckQkhSnQ+8DFU4dy1ujB/ODJlew7oJONRKR3KND7gJlx82VV7DvYynf/+FbU5YhIilKg95HJw4v45KzRPPDqRhas3x11OSKSghTofejzF4ynongAX/vdUlraOqIuR0RSjAK9D+VlZ/Lty6ewekcDP39OZ5CKSGIp0PvYeyaW8TfThvGTZ2qo2dEQdTkikkIU6BG4+dIq8rJjfPE3r9PWrq4XEUkMBXoEygpy+fblU3i9di8/e1ZdLyKSGAr0iFwybTiXTR/OrU+v5s3N+6IuR0RSgAI9QrfMmcyQ/Gw+/9AS3YNURE6aAj1CxXnZfP+K6aze0aArMorISVOgR+zd40u5/l1j+NX8DcxbujXqckQkiSnQ+4Evz57IaSOK+edH3mDDrsaoyxGRJNWjQDez2Wa20sxqzOwrXUz/gpktN7M3zOxpMxuV+FJTV1Ysg59+eAZm8On7F9Hcpv50ETl+3Qa6mcWA24CLgCrgajOr6tRsMVDt7tOAR4DvJ7rQVFc5KI9/v/I03txcz82/X4a7rp0uIsenJ1voZwE17r7W3VuAB4E58Q3c/Rl3P3TjzJeBysSWmR4uqCrnxveM5cEFtfxq/oaoyxGRJNOTQK8A4u/MsCkcdzTXAY93NcHMbjCzhWa2sK6urudVppEvXDCe8yeVc8sflvNSzc6oyxGRJJLQnaJm9lGgGvi3rqa7++3uXu3u1aWlpYlcdMrIyDB+9KHpnFIykE/dv4iNuw50/yYREXoW6JuBEXHDleG4I5jZ+cDXgcvcvTkx5aWngtws7ri2GoCP3/UquxtbIq5IRJJBTwJ9ATDOzMaYWTZwFTA3voGZzQB+ThDmOxJfZvoZNWQgd3ysms17D3LdPQs42KIjX0Tk2LoNdHdvA24EngBWAA+7+zIzu8XMLgub/RuQD/zGzJaY2dyjzE6OQ/Xowdx61QyW1O7lMw8s1pUZReSYLKrD46qrq33hwoWRLDvZ/Gr+em76/TKurK7ku383jYwMi7okEYmImb3m7tVdTcvs62Lk+H3sHaPZ1dDCrU+vJjszg2/NmYKZQl1EjqRATxKfO38czW0d/Ndza8iKZXDTJVUKdRE5ggI9SZgZ/zx7Ai1tHfzyxXXEzPj630xSqIvIYQr0JGJm/L9LJtHhzh1/WUdjSxvfvnwqMfWpiwgK9KRjZtx8aRUDc2Lc9swaGprb+eGV08mK6cKZIulOgZ6EzIwvvX8i+TlZfO+Pb7G/qZWffvh08nP05xRJZ9qsS2L/eN6pfOdvp/LC6p1c+V/z2bavKeqSRCRCCvQk9+GZI7nj2mo27Grk8tteZPmW+qhLEpGIKNBTwHsmlPGb/3MOAB/42Uv84Y0tEVckIlFQoKeIquGFzL1xFlXDC7nx/sV8Z94KXSpAJM0o0FNIWWEuD1x/NtecPYrbn1/LNXe+yvZ69auLpAsFeorJzszgW5dP4QcfnM6S2r3M/vHzPL1ie9RliUgfUKCnqCvOqOSxz7yToUUDuO6ehXxj7jJdglckxSnQU9jYsnwe/dQ5fGLWaO5+aT0X3fo8C9bvjrosEeklCvQUl5sV4+ZLJ3P/9TNpd+fKn8/nG3OX0dDcFnVpIpJgCvQ0cc6pJfzxs+dyzdmjuGf+es7/9+eYt3QrUV0PX0QST4GeRgbmZHLLnCn89h/PYfDAbD513yKuvWsBq7fvj7o0EUkABXoaOn3kIObeOIubL61i8cY9zL71Bb7+6FJ2Nuje3iLJTLegS3O7G1u49alV3PvKRgZkxfjkO8fw9+8aQ2FuVtSliUgXjnULOgW6AFCzo4EfPLGSPy7bRtGALG449xQ+9o5RFCjYRfoVBbr02Jub9/GjP63i6bd2UJCbycfeMYpPzBpDSX5O1KWJCAp0OQFLN+3jZ8/V8Pib28iOZfC3Myr4+KzRTBxaGHVpImlNgS4nbE1dA3e8sI5HF2+iqbWDd5wyhI+ePYoLqsrJztQ+dZG+pkCXk7ansYWHFtby6/kb2Lz3IEMGZvOBMyr54BmVjCsviLo8kbShQJeEae9wXlhdxwOvbuSpFTto73CmVBTytzMquXTaMMoKc6MuUSSlKdClV9Ttb2bu61t4dPEm3txcjxmcOWowF08dyuwpwxhapHAXSTQFuvS61dv38z9LtzJv6VZWbW8AYGpFEedPKud9k8qoGlZIRoZFXKVI8lOgS5+q2bGfJ5dv56nl21lcuxd3KMnP5txxpbxzXAnnnFqirXeRE6RAl8jU7W/mhdV1PLeqjudX1bHnQCsAp5QMZOYpQzhz9CDOHD2YykEDMNMWvEh3FOjSL3R0OCu21TN/zS5eWrOLBet2sz+8jG9ZQQ6njSjmtJHFnDaimCkVRbr8gEgXFOjSL7V3OKu272fB+t0s2rCHJbV7Wb/rwOHpo4fkMbmiiKphhUwcWsDEYYUML8rVlrykNQW6JI09jS28sXkfb27ex9JN+1i2dR+1uw8enp6fk8nYsnzGl+dzSmk+Y0oGcmrpQEYMziMnMxZh5SJ9Q4EuSW1/Uysrt+3nrW37Wb19P6u2N7B6R8MRl/s1g+FFAxg1JI+Rg/MYMTiPykEDqBw0gOHFAygryCWmo2wkBRwr0DP7uhiR41WQm0X16MFUjx58xPj6plbW1jWybmcD63ceYMOuRtbvOsBTK7azs6HliLaZGUZZQQ5Di3IZWpRLWUEu5YW5lBXkUFaYQ0l+8Bg8MFvBL0lLgS5JqzA3K9iROqL4r6YdaGlj056DbNl7kM17g+et+5rYXt/EW9v28/yqnV3eV9UMBuVlM3hg8BiUl8XggdkU52VTPCCL4rwsigZkUTggi8Lc4HVBbib5OZlkxnRtG4lWjwLdzGYDtwIx4A53/26n6TnAr4AzgF3Ah9x9fWJLFem5vOxMxpcXMP4Y15lpbG5je30TOxta2NnQHD5a2NXQzO7GFnY3trBuZyOLNu5lT2MLbR3H7p7My44dDvf8nEwGHnpkx8jLySQvK0ZedowB2Znhc4wBWcEjNytGblbG4eeczBg54XB2LIOczAztDJZudRvoZhYDbgMuADYBC8xsrrsvj2t2HbDH3cea2VXA94AP9UbBIokyMCeTU0rzOaW0+7buzoGWdvYebGXvgRbqD7ZR39TKvoOt7G9qY39T8NzY3Mb+5jYamtoO/0pobG7jQEs7B1qC5xOVHcsgOzN45ITPWbHgkR2zw68zY0Z2+JwZyyArI3yOGZkZGcQyjMwMIxYLnjMzMg4Px8yIZbz9yIgfNiMjg7fHmWGHpxO8tuA9GQYZGYYRPGdYMP3wtPDLKcMMC4eDNkE7i5tmhM9x044YD9BpuHM7DrXpNJ1Obbr6zuw8/e33vd3YOrWnizZ9oSdb6GcBNe6+FsDMHgTmAPGBPgf4Rvj6EeCnZmauW8pLijCzw1vcFcUDTng+HR1OU1s7B1vaOdDSTlNrO02tHRxoaaOprSMcbqe5rSN4hK9bwuGWtg5a2ttpaeugtd3D4WB8W0cHrW3OwdZ22jo6aGt3WtqD57b2Dlo7nPaO4HVbh9MWDrd388tDEu/bl0/ho2ePSvh8exLoFUBt3PAmYObR2rh7m5ntA4YAO+MbmdkNwA3hYIOZrTyRooGSzvNOA1rn9KB1TgPXfI+Sa058nY/6TdCnO0Xd/Xbg9pOdj5ktPH1G+jsAAANZSURBVNphO6lK65wetM7pobfWuSe75TcDI+KGK8NxXbYxs0ygiGDnqIiI9JGeBPoCYJyZjTGzbOAqYG6nNnOBa8PXVwB/Vv+5iEjf6rbLJewTvxF4guCwxV+6+zIzuwVY6O5zgTuBX5tZDbCbIPR700l32yQhrXN60Dqnh15Z58hO/RcRkcTSqW0iIilCgS4ikiKSLtDNbLaZrTSzGjP7StT19DYzG2Fmz5jZcjNbZmafjbqmvmBmMTNbbGZ/iLqWvmBmxWb2iJm9ZWYrzOwdUdfU28zs8+G/6TfN7AEzS7n7EprZL81sh5m9GTdusJn9ycxWh8+DErW8pAr0uMsQXARUAVebWVW0VfW6NuCL7l4FnA18Og3WGeCzwIqoi+hDtwJ/dPeJwHRSfN3NrAL4J6Da3acQHHDR2wdTROFuYHancV8Bnnb3ccDT4XBCJFWgE3cZAndvAQ5dhiBluftWd18Uvt5P8B+9ItqqepeZVQJ/A9wRdS19wcyKgHMJjhbD3VvcfW+0VfWJTGBAeO5KHrAl4noSzt2fJzjyL94c4J7w9T3A5YlaXrIFeleXIUjpcItnZqOBGcAr0VbS634MfBnoiLqQPjIGqAPuCruZ7jCzgVEX1ZvcfTPwA2AjsBXY5+5PRltVnyl3963h621AeaJmnGyBnrbMLB/4LfA5d6+Pup7eYmaXADvc/bWoa+lDmcDpwM/cfQbQSAJ/hvdHYb/xHIIvs+HAQDP7aLRV9b3wBMyEHTuebIHek8sQpBwzyyII8/vc/XdR19PLZgGXmdl6gi6195rZvdGW1Os2AZvc/dAvr0cIAj6VnQ+sc/c6d28FfgecE3FNfWW7mQ0DCJ93JGrGyRboPbkMQUqx4ILKdwIr3P2HUdfT29z9q+5e6e6jCf6+f3b3lN5yc/dtQK2ZTQhHvY8jL0+dijYCZ5tZXvhv/H2k+I7gOPGXSrkW+H2iZpxUt6A72mUIIi6rt80CrgGWmtmScNzX3H1ehDVJ4n0GuC/cUFkLfCLienqVu79iZo8AiwiO5FpMCl4CwMweAM4DSsxsE3Az8F3gYTO7DtgAXJmw5enUfxGR1JBsXS4iInIUCnQRkRShQBcRSREKdBGRFKFAFxFJEQp0EZEUoUAXEUkR/wt6Z8im4m4+kgAAAABJRU5ErkJggg==\n", 197 | "text/plain": [ 198 | "
" 199 | ] 200 | }, 201 | "metadata": { 202 | "needs_background": "light" 203 | }, 204 | "output_type": "display_data" 205 | } 206 | ], 207 | "source": [ 208 | "t = np.linspace(0, 10, 100)\n", 209 | "plt.plot(t, np.exp(-cumulative_hazard(results.x, t)))\n", 210 | "plt.ylim(0, 1)\n", 211 | "plt.title(\"Estimated survival function\")" 212 | ] 213 | }, 214 | { 215 | "cell_type": "markdown", 216 | "metadata": {}, 217 | "source": [ 218 | "Let's move to part 3. " 219 | ] 220 | } 221 | ], 222 | "metadata": { 223 | "kernelspec": { 224 | "display_name": "Python 3", 225 | "language": "python", 226 | "name": "python3" 227 | }, 228 | "language_info": { 229 | "codemirror_mode": { 230 | "name": "ipython", 231 | "version": 3 232 | }, 233 | "file_extension": ".py", 234 | "mimetype": "text/x-python", 235 | "name": "python", 236 | "nbconvert_exporter": "python", 237 | "pygments_lexer": "ipython3", 238 | "version": "3.7.3" 239 | } 240 | }, 241 | "nbformat": 4, 242 | "nbformat_minor": 2 243 | } 244 | -------------------------------------------------------------------------------- /full_notebooks/Part 6.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "### The Delta Method\n", 8 | "\n", 9 | "> CFO: What's our variation in churn this year?\n", 10 | "\n", 11 | "> Data scientist: Our $\\lambda$ value has been increasing, but $\\rho$ is staying the same, so we should see-\n", 12 | "\n", 13 | "> CFO: Our banana value is increasing?\n", 14 | "\n", 15 | "We want to connect our parameters to business logic _and_ carry over variance in estimates. \n", 16 | "\n", 17 | "Example: It's silly to present a point estimate without confidence intervals (CIs), since arguably the CIs contains more useful information than a point estimate. \n", 18 | "\n", 19 | "We'll start with asking: what is the CI for the survival function, $S(t; \\hat{\\theta})$? \n", 20 | "\n", 21 | "\n", 22 | "We will use the **DELTA METHOD** to do this (bolded because it's awesome):\n", 23 | "\n", 24 | "$$\\text{Var}(f(\\hat{\\theta})) \\approx \\text{grad}(f)(\\hat{\\theta}) \\cdot \\text{Var}(\\hat{\\theta}) \\cdot \\text{grad}(f)(\\hat{\\theta}) ^ T $$\n", 25 | "\n", 26 | "1. $f$ in our case is the survival function, $S$\n", 27 | "2. We know $\\text{Var}(\\hat{\\theta})$ (inverse of the Hessian)\n", 28 | "3. Do we need to compute $\\text{grad}(f)$ by hand? Heck no, use `autograd`" 29 | ] 30 | }, 31 | { 32 | "cell_type": "code", 33 | "execution_count": 7, 34 | "metadata": {}, 35 | "outputs": [], 36 | "source": [ 37 | "# seen all this...\n", 38 | "%matplotlib inline\n", 39 | "from autograd import numpy as np\n", 40 | "from autograd import elementwise_grad, value_and_grad, hessian\n", 41 | "from scipy.optimize import minimize\n", 42 | "\n", 43 | "# N = 50 for this example\n", 44 | "T = (np.random.exponential(size=50)/1.5) ** 2.3\n", 45 | "E = np.random.binomial(1, 0.95, size=50)\n", 46 | "\n", 47 | "def cumulative_hazard(params, t):\n", 48 | " lambda_, rho_ = params\n", 49 | " return (t / lambda_) ** rho_\n", 50 | "\n", 51 | "hazard = elementwise_grad(cumulative_hazard, argnum=1)\n", 52 | "\n", 53 | "def log_hazard(params, t):\n", 54 | " return np.log(hazard(params, t))\n", 55 | "\n", 56 | "def log_likelihood(params, t, e):\n", 57 | " return np.sum(e * log_hazard(params, t)) - np.sum(cumulative_hazard(params, t))\n", 58 | "\n", 59 | "def negative_log_likelihood(params, t, e):\n", 60 | " return -log_likelihood(params, t, e)\n", 61 | "\n", 62 | "from autograd import value_and_grad\n", 63 | "\n", 64 | "results = minimize(\n", 65 | " value_and_grad(negative_log_likelihood), \n", 66 | " x0 = np.array([1.0, 1.0]),\n", 67 | " method=None, \n", 68 | " args=(T, E),\n", 69 | " jac=True,\n", 70 | " bounds=((0.00001, None), (0.00001, None)))\n", 71 | "\n", 72 | "estimates_ = results.x\n", 73 | "H = hessian(negative_log_likelihood)(estimates_, T, E)\n", 74 | "variance_matrix_ = np.linalg.inv(H)" 75 | ] 76 | }, 77 | { 78 | "cell_type": "code", 79 | "execution_count": 2, 80 | "metadata": {}, 81 | "outputs": [ 82 | { 83 | "data": { 84 | "text/plain": [ 85 | "array([ 0.11163842, -0.24322003])" 86 | ] 87 | }, 88 | "execution_count": 2, 89 | "metadata": {}, 90 | "output_type": "execute_result" 91 | } 92 | ], 93 | "source": [ 94 | "from autograd import grad\n", 95 | "\n", 96 | "def survival_function(params, t):\n", 97 | " return np.exp(-cumulative_hazard(params, t))\n", 98 | "\n", 99 | "grad_sf = grad(survival_function)\n", 100 | "grad_sf(estimates_, 5.0)" 101 | ] 102 | }, 103 | { 104 | "cell_type": "code", 105 | "execution_count": 3, 106 | "metadata": {}, 107 | "outputs": [], 108 | "source": [ 109 | "def variance_at_t(t):\n", 110 | " return grad_sf(estimates_, t) @ variance_matrix_ @ grad_sf(estimates_, t)" 111 | ] 112 | }, 113 | { 114 | "cell_type": "code", 115 | "execution_count": 4, 116 | "metadata": {}, 117 | "outputs": [ 118 | { 119 | "data": { 120 | "text/plain": [ 121 | "0.00036047601896770017" 122 | ] 123 | }, 124 | "execution_count": 4, 125 | "metadata": {}, 126 | "output_type": "execute_result" 127 | } 128 | ], 129 | "source": [ 130 | "variance_at_t(5)" 131 | ] 132 | }, 133 | { 134 | "cell_type": "code", 135 | "execution_count": 5, 136 | "metadata": {}, 137 | "outputs": [], 138 | "source": [ 139 | "t = np.linspace(.001, 10, 100)\n", 140 | "\n", 141 | "std_sf = np.sqrt(np.array([variance_at_t(_) for _ in t]))" 142 | ] 143 | }, 144 | { 145 | "cell_type": "code", 146 | "execution_count": 6, 147 | "metadata": {}, 148 | "outputs": [ 149 | { 150 | "data": { 151 | "text/plain": [ 152 | "Text(0.5, 1.0, 'Estimated survival function with CIs (Delta method)')" 153 | ] 154 | }, 155 | "execution_count": 6, 156 | "metadata": {}, 157 | "output_type": "execute_result" 158 | }, 159 | { 160 | "data": { 161 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEICAYAAABPgw/pAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deZRc51nn8e9Tt9beu9Xa1ZY3eZFN4kVxQowTZwMnJHHYkpgJJJkQwzkYGMgBAjMnOIYZMgMHMgxmcUJWYhuTQBCJIbuzObYlx0ssybJkWVa3pG71vtf+zB/vre7b1dXd1a3qpaqezzl9uureW7feW8vvvvXe975XVBVjjDHVL7TeBTDGGFMZFujGGFMjLNCNMaZGWKAbY0yNsEA3xpgaYYFujDE1ou4CXURuEpGj612OUkTkZhHp2QDlOCQiN1dgPSdF5PULzEuIyL+LyKiI/PP5Ptcyy1WR7Vut5xeRh0TkV9awSAsSke+LyLUVXN+Cn4mNRkTuFJF/rNC63iMi3/Nvx0TkWRHZXIl1B1VNoPsfhGkRmQj8/XUZj1MRubRwX1W/q6qXr1IZPyUif7Ia615LqnqVqj60yk/z88BWYJOq/sJqPUmp92SNtm9BweevRGiISIuIfFRETvnfi+f9+53+/BWFqIi8BRhX1ScCZc2IyLj/95yI/LWIbF9huSsWmOdrLStTqpoCPgF8sNLrrppA971FVZsCf3esd4GqjYiE17sMvt3Ac6qaXe+CVDMRiQLfAK4CbgFagB8HBoEbznP1vwZ8tmjaP6lqM9AB/AywDXh8paFex+4F3i0isYquVVWr4g84Cbx+gXmXAt8GRoEB3IcO4DuAApPABPAO4Gagp2i9vws87S/3D7ia438A48DXgfbA8v8M9PrP9R3gKn/67UAGSPvP9e/+9B3AF4B+4AXgNwPrSgCfAoaBw345ehbYRgH+EjgHjAE/Aq725z0E/Epg2fcA3wvcV+DXgWN+Gf4W+POi9f8b8DvB19ov+zTQEVjuWv81jgCXAN/EhccA8Dmgban3DPiw/zpl/NfqfcCdwD8GlrnQL3c4sI1/DHzff1++CnQGlv8J4GFgBOj2X4OF3pOZcgEx4KPAGf/vo0DMn3cz0AN8wH/dzwLvXeD9eQ3wo8D9rwEHAve/C7yt6PW9peh1eKqcbS163l8B+oCmcr47LPBdKfGYqP/e7wpMm/Me+dM84CkCnyfgzcCT/nvxMPCS4rIssu3vBY74230C+NVFtus9/mv0l/5znQBe6U/v9t+zdweWjwF/DpzyX7O/w30HG/1tzftlmcB99u8EHgA+45fnELAvsL4r/fdqxJ/31sC8TcB+3Hf1Mf/9/F5R+Y8Br65oTlZyZav5x+KBfh/w33G/OOLATwTmKXBp4P7NzA/0R3AhvtP/EPwQF1xxXGD9UWD5/wo0MxsETwbmfQr4k8D9EPA48CH/C3Kx/6H7KX/+R3Bf9A6gC3iGhQP9p/x1teHC/Upgu84GwFKB/jX/eRLAq/wPvPjz2/0P9I4SAfBN4P2Bdf0Z8HeBcHiD/1psxu3gPlrme3YncwO8+P6FzA/054HL/G14CPiIP2837gt3G25Hswm4ptR7UmL77vLf/y3+NjwM/HHgs5L1l4kAbwKmCOzgA+tMAEmg01+2DziN+6wk/Nd3U4nnn7PdS21riee9H/h0ud8dFvmuFD3mKmBysfcsMP0u4FH/9rW479DLcWH/bv/5Y2Vu+0/jKgoCvNp/va9boIzv8d+f9/rP9Se4sL4b95n8Sf9z0eQv/5e4kO3w35d/B/60VC4Eypf033cP+FPgEX9eBDgO/CHuu/1a/7kuD7wvD+B2Flf7n4XiQN9PoIJXib9qa3L5ooiMBP7e70/P4L7UO1Q1qarfW+Z6/5+q9qnqaVzAPqqqT6hqEvhX3IcUAFX9hKqOq2sHuxN4qYi0LrDelwGbVfUuVU2r6gngY8A7/flvB/6nqg6pajfwV4uUMYP7EF6BC+Ijqnp2Gdv4p/7zTPvbqMBN/ryfB36gqmdKPO5eXFAiIuKX/V4AVT2uql9T1ZSq9gN/gfsSrpZPqupz/jY8AFzjT/9F4Ouqep+qZlR1UFWfLHOd/wW4S1XP+dvwYeCXAvMz/vyMqj6Iq73NOwbjl+kAbmd5Pa7W+n3gRuAVwDFVHazAthbbhPvlUK5yvyttuIAqxxlcSIL7VfT3qvqoquZU9dNACvcaLElVv6yqz6vzbdyvk5sWecgLqvpJVc0B/4SrGN3lfya/ivsVcKn/2b0d+G3/ezAO/C9mv4sL+Z6qPuiv/7PAS/3prwCacDvatKp+E/gScJuIeMDPAR9S1UlVfQb4dIl1j+Ne54qptkB/m6q2Bf4+5k//Pdwe/TG/B8F/XeZ6+wK3p0vcbwIQEU9EPuIfdBrD1TbA1cpK2Q3sCO6EcHv0rf78HbiacsGLCxXQ/8D8Na72cU5E7hGRlrK2zpl5HnXVg/vxgxoXiJ9b4HFfAH7cbyN9Fe5n6XcBRGSriNwvIqf91+MfWfi1qITewO0p/PcF9yV+foXr3MHc1/1Ff1rBoM5t5w8+b7Fv42p6r/JvP4Tbwb3av78cC21rsUFgOe3X5X5XhnEViHLsBIb827uBDxR95ruY+5ouSETeKCKPiMiQ/9g3sfhnqvi7iqqW+v5uBhpw7f2Fcv2nP30xxe9D3D8OtQPoVtV8YP6LuNdiMxBm6e92M665pmKqLdBLUtVeVX2/qu4AfhX4m2DPlgr6ReBWXBtgK65ZANwXBFytN6gbV4MI7oSaVfVN/vyzuA97wQWLPbmq/pWqXg/sxf0c/11/1iTuw1qwrdTDi+7fB/y8iOzG/Tz+wgLPOYyrJb0Dt/33+zsEcDUcBX5MVVuAdzH7WixXOduwkG7cz/RSlhpO9AwuhAou8KetRHGgf5ulA/18hzv9OvBTItJYzsLL+K4cx/0o27nY+kQkBLwFfyePey/+Z9FnvkFV7ytVnKJ1xXCfwz8HtqpqG/AgK/9MBQ3gwv2qQLlaVbWwo1zu+3AG6PK3v+ACXNNKP64paKnv9pW4X3IVUxOBLiK/ICK7/LvDuDensOfsw7VdV0Iz7ufjIC58/lfR/OLnegwYF5Hf9/tdeyJytYi8zJ//APAHItLul/83FnpiEXmZiLxcRCK48Esyu41PAj8rIg3+l/N9S22Iuq5oA8DHga+o6mI1hXuBX8Y1zdwbmN6Ma4IY9b/4v1viseV6EniViFzgN2H9wTIe+zng9SLydhEJi8gmESk0USz1/t8H/A8R2ex38/sQ7pfGSjyMa465AXhMVQ/hdhYvxx1fKKUPuLAoGJbjs7gQ/YKIXCEiIX/7/1BE3lS88BLflRmqmsbtLEo2ofmv85W4128brrkNXJPir/mfVRGRRhH5aREpVdsv3vYoru27H8iKyBtx7eDnza9Jfwz4SxHZ4m/DThH5qUBZNi3SfFrsUVyN/fdEJCLuvIK34Co8OeBfgDv97+Re3LGEGf73pQN3/KZiqi3Q/13m9kP/V3/6y4BHRWQCd6Dht/z2anDt3J/2f2a9/Tyf/zO4n06ncb1Sit+MfwD2+s/1Rf+NfTOu/fMFZgO08KH5sL++F3C14OIuYkEtuA/ksP+YQdwBSnAHe9K4D+WnWbj5pNi9uF8b9y6x3H5gD9CrqsEaxYeB63A9Jr6M+xCviKp+DdcG+jTu4O+XlvHYU7if5h/A/fR/ktm2zjnvSYmH/wlw0H/eH+EOiK/oXAJVnfQff8gPRIAfAC+q6rkFHlY4qWpQRH64gudM4d7DZ3EHvgu9KjpxoVNsse9Ksb9n7vEEgHf4jx31Hz8IXF84/qKqB4H345oHh3E1/fcssP452+63a/8mrqIzjPtFuH/BjV++3/fL84jfRPh1/OMhqvosbud0wv+sLNpE5L+/bwHeiPte/w3wy/56AO7ANfX04g7Mf7JoFb+IO5idqsB2zSj0cjDGmHlE5PvAHf4vOlMBftPSU8CrFtnRr2zdFujGGFMblmxyEZFPiMg5EXlmgfkiIn8lIsdF5GkRua7yxTTGGLOUctrQP4U7q2shb8S1r+7B9fP82/MvljHGmOVaMtBV9TvM9jEt5VbgM/6JAI8AbWLjOhhjzJqrxEBNO5nbgb7Hnzbv7DURuR1Xi6exsfH6K664ogJPb4wx9ePxxx8fUNWSJ0St6ch7qnoPcA/Avn379ODBg2v59MYYU/VEZMEzyivRD/00c8+I2uVPWzW5vPXMMcaYYpUI9P3AL/u9XV4BjC5z0Khly+TmndhmjDF1b8kmFxG5Dzc+Rae4K3r8EW7oSFT173BjLbwJdwbWFG4oy1WVtRq6McbMs2Sgq+ptS8wvXDxhTZzon+DAySF+4fouQqFKjNljjDG1odrGcuFrh/v4/S/8iOlMbr2LYowxG0rVBXpjzP2omEzbpSiNMSaoCgPdA2AyZTV0Y4wJqr5Aj/o19JTV0I0xJqj6At1vcplKWw3dGGOCqjbQrYZujDFzVV+gR/02dDsoaowxc1RdoDf4NfTxpAW6McYEVV2gN0ULgZ5Z55IYY8zGUnWB3uB3W7QaujHGzFV1gR7xQkQ8YcIC3Rhj5qi6QAdoiHqMWy8XY4yZo0oDPWzdFo0xpkhVBnoi6lmgG2NMkaoM9MZomEk7U9QYY+aoykBviHpM2YlFxhgzR1UGemMsbGO5GGNMkeoM9KjHtAW6McbMUZ2BHgvbFYuMMaZIVQZ6UyxsNXRjjClStYGezSvpbH69i2KMMRtGVQZ6Y7xwkQvr6WKMMQVVGejN/hC6E3ZykTHGzKjKQG/ya+hj0zaErjHGFFRloBcuQzc2bTV0Y4wpqM5A9y9yMWYXuTDGmBnVGeh2kQtjjJmnOgO9cBm6lNXQjTGmoDoDvdDLxWroxhgzo0oD3TW5WKAbY8ysqgz0RMRDgAk7scgYY2ZUZaCLCPGIx2TKxnMxxpiCqgx0sMvQGWNMseoN9IhnF7kwxpiA6g10uwydMcbMUVagi8gtInJURI6LyAdLzL9ARL4lIk+IyNMi8qbKF3Uud11Rq6EbY0zBkoEuIh5wN/BGYC9wm4jsLVrsfwAPqOq1wDuBv6l0QYs1RD2SdtUiY4yZUU4N/QbguKqeUNU0cD9wa9EyCrT4t1uBM5UrYmmNUbsMnTHGBJUT6DuB7sD9Hn9a0J3Au0SkB3gQ+I1SKxKR20XkoIgc7O/vX0FxZzXEPJKZPPm8ntd6jDGmVlTqoOhtwKdUdRfwJuCzIjJv3ap6j6ruU9V9mzdvPq8nbIyGSWZyZPJ2GTpjjIHyAv000BW4v8ufFvQ+4AEAVf0BEAc6K1HAhTTGPNLZPBm7rqgxxgDlBfoBYI+IXCQiUdxBz/1Fy5wCXgcgIlfiAv382lSW0BQLo8C4nVxkjDFAGYGuqlngDuArwBFcb5ZDInKXiLzVX+wDwPtF5CngPuA9qrqqjduzVy2yIXSNMQYgXM5Cqvog7mBncNqHArcPAzdWtmiLa/IDfdQuQ2eMMUAVnynaZDV0Y4yZo2oDvTkeAWB4Kr3OJTHGmI2hagN9tsnFaujGGANVHOjNcQt0Y4wJqtpAb/IDfTyZYZU71BhjTFWo2kBvjrk29Ol0npSdXGSMMdUb6A3+haJT2ZyNumiMMVRxoEe8EBFPSGbyNuqiMcZQxYEOEAt7pLI5pu1CF8YYU92Bnoh6VkM3xhhfdQd6JETS2tCNMQao8kCPR9xl6OzaosYYU+WB3hANk8rkSdmVi4wxproDPRHxZvqgJ7NWSzfG1LeqDvSGqDfTfm49XYwx9a7qA71QQ7d2dGNMvavuQI+FZ2ro1tPFGFPvqjrQG6Me2bySzVlfdGOMqe5A98dET2bz1oZujKl7VR3ohYtcTKdzVkM3xtS9qg70ba1xAIYm02RzSiZnw+gaY+pXVQf6rrYEAAOTKQCrpRtj6lpVB/r2tgQCDE64C0UnrR3dGFPHqjrQExGPtoYIAxOuhm590Y0x9ayqAz0UEjY1xmZq6BOp7DqXyBhj1k9VB7oXEjY1RRn029DHkpl1LpExxqyf6g50ETqbYgxNpsnllel0jpyNumiMqVNVHeihEGxqipJXGJlKo2rNLsaY+lXVge6J0NkYA2DA2tGNMXWuugPdb0OH2b7o49aOboypU1Ud6CJCZ3N0Tl/0iaTV0I0x9amqAx0gFp7bF308lUXVDowaY+pP1Qe6V9QXPZdTkhkb08UYU3+qP9Blbl90gPGUtaMbY+pPWYEuIreIyFEROS4iH1xgmbeLyGEROSQi91a2mAsLheb2RQcYt3Z0Y0wdCi+1gIh4wN3AG4Ae4ICI7FfVw4Fl9gB/ANyoqsMismW1Clys0NOl0Bd9U1PMDowaY+pSOTX0G4DjqnpCVdPA/cCtRcu8H7hbVYcBVPVcZYu5sFCJvuhWQzfG1KNyAn0n0B243+NPC7oMuExEvi8ij4jILaVWJCK3i8hBETnY39+/shIXKdUXPZnJ2cUujDF1p1IHRcPAHuBm4DbgYyLSVryQqt6jqvtUdd/mzZsr8sSeCB2Nc/uig/VHN8bUn3IC/TTQFbi/y58W1APsV9WMqr4APIcL+FUX9oSIF5rTFx2s2cUYU3/KCfQDwB4RuUhEosA7gf1Fy3wRVztHRDpxTTAnKljOBcUjHsCcvuhgQ+kaY+rPkoGuqlngDuArwBHgAVU9JCJ3ichb/cW+AgyKyGHgW8DvqurgahU6KFEI9KK+6MNT6YUeYowxNWnJbosAqvog8GDRtA8FbivwO/7fmopH3D6psynGgZND5PKKFxJSmTzT6RyJqLfWRTLGmHVR9WeKxgM19LzOrZmPTFst3RhTP6o+0GPhECKwpdn1Re8dTc7MG560dnRjTP2o+kAXEeIRj672BgC6h6dm5lkN3RhTT6o+0MG1ozfGwnQ0Rukemp6ZPpXKkcrm1rFkxhizdmoi0GNh147e1Z6YU0MHGJ2yZhdjTH2oiUAvHBjt6migdyxJOjt72v+wBboxpk7URKAXuiZ2tTegCqdHZptdRqw/ujGmTtREoMfDbjO6OhLA3AOjE6msDdRljKkLtRHofpNLZ1OMWDhE99BsoKvC6LQ1uxhjal9NBXpIhF3tCXqGp+fMt2YXY0w9qIlA90JCxG92uaCjge7hKfKqM/ODg3YZY0ytqolAh9l29F3tDSQz+TkhPp7MksxYf3RjTG2rmUCf6elS4sAoMGesdGOMqUU1E+iFdvSdbQlEmHNgFGavN2qMMbWqdgLdP1s0FvbY2hynu+jA6PBkmlxeSz3UGGNqQu0EenR2U7o6EvQUNbnk8moXvTDG1LTaCfTI7IUsutobGJhIM5Wee11Ra0c3xtSy2gn0cCDQO9xQuqeK29HHrYZujKldNRPo0XAILyQAXNTZiADHzk3MWSaZyTFuF482xtSomgl0gJh/fdGmWJid7Qme6x2ft4z1djHG1KqaCvRgO/plW5t5vn+SbNHAXP3j1o5ujKlNNRXoiUCgX761mXQuz8nBue3oY9OZeQdLjTGmFtRUoDdGwzO3L9vaBMBzffObXc4GLiRtjDG1oqYCvTkeDtyOsKM1XjLQ+yzQjTE1qKYCvSkQ6ODa0Y+dm5h3huhUOmfXGjXG1JyaCvSIF5p3YDSVzc/rjw7QO2a1dGNMbampQIe5tfTLtzUDpdvR+8aSqNrYLsaY2lFzgR5sR29NRNjaEuNoiUBPZ/MMTlqfdGNM7ai9QI/NbUe/fGszx/omyJcYabHXDo4aY2pIzQV68YHRPVubmc7k5l1nFODceJJ0Nj9vujHGVKOaC/RExMPzZOb+FX47+jNnRuctm8/D2dH5QW+MMdWo5gJdROY0u7Q3RNm9qYEnu0dKLt8zPG0HR40xNaHmAh3mN7tc09XGCwOTjE7P73s+nc7ZwVFjTE2ozUCPzQ90BZ5apJZujDHVriYDvTkWmXN/V1uCzqbogs0uA+MpptO5tSiaMcasmrICXURuEZGjInJcRD64yHI/JyIqIvsqV8Tla4qHkdnjoogI13S1cfjsGMlM6eAuvgapMcZUmyUDXUQ84G7gjcBe4DYR2VtiuWbgt4BHK13I5fJCQiLqzZl2TVcb2bxy6MxYycecHpmeN3a6McZUk3Jq6DcAx1X1hKqmgfuBW0ss98fA/wY2xNk6xc0ue7Y00xD1Fmx2yebU2tKNMVWtnEDfCXQH7vf402aIyHVAl6p+ebEVicjtInJQRA729/cvu7DL0VzU08ULCS/Z1crTPSPzRl8sODU0teA8Y4zZ6M77oKiIhIC/AD6w1LKqeo+q7lPVfZs3bz7fp15Ue0N03rRrutqYTOc4XnTx6IJ0Ns+ZEaulG2OqUzmBfhroCtzf5U8raAauBh4SkZPAK4D9631gtDkennPGKMDVO1qJeiEeOzm04ONODk6WHPfFGGM2unIC/QCwR0QuEpEo8E5gf2Gmqo6qaqeqXqiqFwKPAG9V1YOrUuIyhUJCW2JuO3o84nHd7jYee2FowTFcUpk8Z22sdGNMFVoy0FU1C9wBfAU4AjygqodE5C4ReetqF/B8lGp2ufGSTqYzuQUPjgKcHLBaujGm+oSXXgRU9UHgwaJpH1pg2ZvPv1iV0d44P9Av39ZMR2OU7z8/wA0XdZR83HQ6x+mRabo6Gla7iMYYUzE1eaZoQUuJdvSQCK+8ZBOHz44xPLXwGC4nBiatX7oxpqrUdKCLSMlml1desglV+MHzgws+NpPNc3LQzh41xlSPmg50gPaGyLxpW5rj7NnSxMPPDy46dO6pockFhwowxpiNpvYDvUQ7OriDo71jSU4MTC742Hwenu8v3WfdGGM2mpoP9ObY/HZ0gH0XthMLh3jo6OJnrJ4dSZYcR90YYzaamg/0hdrR4xGPGy/t5LGTQ4wscnAU4MjZMbuqkTFmw6v5QAfoKBHoAK+/cgv5vPKtJWrpE8ks3UM2JIAxZmOri0Df3BwrOX1Lc5xrLmjjoaPnSGUXP/j5fP+EHSA1xmxodRHoiag37zqjBW+4ciuT6RyPnFh4fBeAXF452ju+GsUzxpiKqItAB9iyQC19z5YmLtzUwNcO95Ffop28fzxFn43zYozZoOon0FviJaeLCG/Yu5XesSQ/Oj265HqOLHIZO2OMWU91E+hNsTANMa/kvOt3t9PRGOXLT59dsjdLNqcctl4vxpgNqG4CHdxB0FLCoRBvfsl2TgxM8lTP0rX0oYm0Xa7OGLPh1Fegt5RuRwc3vsuW5hhffPL0km3pAMfOjTORylayeMYYc17qKtBb4hHikdLNLuFQiFtfuoOe4WkOnhxecl35PDzdPWIjMhpjNoy6CnRYvJb+sos62NmW4N+eOl3WxaKn0jkOnx2rZPGMMWbF6i7Qty7Q2wXcWOlvu2YHfWMpHn5+oKz1nRtLccqG2TXGbAB1F+iticiCJxkBXNPVxsWdjXzxyTNMp8vrnnjs3DjDk4uPB2OMMaut7gIdYGdbYsF5IsIv3nABY9MZ9j91pqz1qcJTPSNMpe0gqTFm/dRloG9rjeOF5g+pW3BhZyM37enkG8/2cbrM7onZnPLkqRHSWTtIaoxZH3UZ6BEvtOjBUYCfvXYXiYjH5x57seyTiKbSOZ7uGSFfxgFVY4yptLoMdIBdbQ2Lzm+Kh/nZ63bxXN8Ej51cfOCuoJGpDIfO2Jmkxpi1V7eB3toQoTG28MFRgJsu7eTCTQ3cf6CbsWVctahvLMmRszYyozFmbdVtoAPsal/44ChAKCS898aLmE7n+Oyj5Te9AJwZmea5Pgt1Y8zaqetA394aJ1zieqNBO9sSvO2anTxxamTJMdOLnRqc4vg5u8i0MWZt1HWgh70QXR2Lt6UD/OTerVy6uYl7HzvF0DL7m58cmOT4OaupG2NWX10HOkBXe8OiXRih0PRyITlVPvn9F5bdi+XkwJQ1vxhjVl3dB3o0HGLnEm3p4IYMuO1lXRzpHWf/0+WdcBR0anCKIzaOujFmFdV9oANc0NFAqIxX4icu7eTGSzbxpafPlnV1o2Knh6d5ume0rIG/jDFmuSzQgXjEY1vL0rV0EeG/vHw3Xe0JPv7dEwxOpJb9XP3jKZ44NUzGht01xlSYBbrvos7Gsmrp0XCIX3v1JeQV7n7oeVIruL7oyFSGAy8MMWkXyDDGVJAFui8R9djVvnSPF3Dt6e+/6SK6h6f42HeXf5AU3DABB04OraiWb4wxpVigB1zU2UgkXN5L8pJdbdz2sgt4smeEBx7vXtHzZXPKk90jNp66MaYiLNADIl6Iizsby17+tVds4Q1XbuXrR87x9SN9K3pOVXiub5yne0asXd0Yc17KCnQRuUVEjorIcRH5YIn5vyMih0XkaRH5hojsrnxR18bOtgQNsdLXHS3lF67fxbUXtPFPB7r5wfODK37ec2MpDrwwxHiy/DFjjDEmaMlAFxEPuBt4I7AXuE1E9hYt9gSwT1VfAnwe+D+VLuhaCYWEPVual7X87TddzBXbmvnEwy9wYBkjMxYrtKufGpyy/urGmGUrp4Z+A3BcVU+oahq4H7g1uICqfktVCw3BjwC7KlvMtbW5ObbkeOlBES/EHa+5lEs3N/Hx777AE6eGV/zc+bxrgnmie4TkCnrQGGPqVzmBvhMIHvXr8act5H3Af5SaISK3i8hBETnY399ffinXweXbmpccuCsoFvH4zdfu4YJNDfzdd05w8MWV19QBhibSPHJikN7R5HmtxxhTPyp6UFRE3gXsA/6s1HxVvUdV96nqvs2bN1fyqSsuFva4fFv5TS/guj7+9uv3cOGmBv7+Oyf43rGB8ypDNqc8c3qUJ622bowpQzmBfhroCtzf5U+bQ0ReD/x34K2qWhOdq7e3JuhsLr/pBaAhGuZ3Xn8Ze7e18KkfnOSrh3vPuxwD4ykeOTFI95C1rRtjFlZOoB8A9ojIRSISBd4J7A8uICLXAn+PC/NzlS/m+rlimU0v4Jpf7njtpVy/u50HDvZw32Onzvs6o9mccrR3nAMnhxmznjDGmBKWDHRVzQJ3AF8BjgAPqOohEblLRN7qL/ZnQBPwzyLypIjsX2B1VSce8bhqR+uyHxfxQvzqTRfzhiu38o1nz/H/vnWc6fT5N5uMTYusyasAABDUSURBVGd47MQQh8+MkcpaM4wxZpas10/4ffv26cGDB9fluVfi+LlxTg6s7IzOh46e497HTrG9NcGvv+YStjTHK1ImzxMu7mykq72B0BJjuhtjaoOIPK6q+0rNszNFy3RxZxNtDZEVPfbmy7fw3153GcNTaf74S0fOq1tjUC6nHOub4OHnBzkzMm3t68bUOQv0MoVCwtU7W4mWOdZLsb07WvjQm/eypSXG3Q89z+cf7yGbr8yp/slMjsNnxnjkxBDnxq2bozH1ygJ9GeIRj5d2tS15ybqFdDbF+OAtV/Dqyzbzn4d6+T//eZS+scoF8GQqy9PdozxyYtCC3Zg6ZIG+TK2JCFftbFnx4yNeiF96xW5+9VUX0zuW5MNfOsx3nuuvaHPJRHI22HtHk9YUY0ydsEBfgS3NcS7buryTjoq97MIO7nzLVVzS2chnHnmRj379GAMVHht9IpnlmdOjPPy868Nul74zprZZL5fzcD49Xwryqjx0tJ8v/LAHBX7mmp287ootq9JrJewJu9ob2NWeIB4pf0RJY8zGsVgvFwv08/Rc33hFLlAxOJHis4++yDOnx7igo4Hbbuha1qiPyyHifmXsak/Q3hhdlecwxqwOC/RVdrR3nO6h8w91VeXAyWH++fFuhqcy/PjFm/i563bS1rB6odsYC7OzLcG21viKe/AYY9aOBfoaqFRNHSCVyfHlZ87y1UN9hER4w96t3HLVNhLR1WsmCYVcrX17a5yOxigidqKSMRuRBfoaeXFwkmN9ExVbX/94in994jSPnRyiKRbmTT+2jZsv27LqNelYJMT21jjbWhM0xcKr+lzGmOWxQF9DZ0enOXxmjEq+rCcHJvnCD3s40jtOayLCLVdt49WXbV6TJpKmeJhtLXG2tsRX9ReCMaY8FuhrbGgyzY9Oj5LJVvaiz8/1jbP/qTM82ztOczzM667Ywmsu30LjGtWiWxIRtrbE2NJs4W7MerFAXwfT6RxP9YwwkcxWfN1He8f5j2fO8syZMWLhEDft6eR1V2xl8zLHbj8fTfEwW5pjdDbHaImvbIwbY8zyWaCvk1xeOXJ2bNUuI3dqaIqvHOrl4Mlh8qq8dFcbr7liM1dubyG0hgc14xGPzuYomxpjdDRGVzw0gjFmaRbo6+zs6DTP9o6Ty63Oaz08leaho/18+7l+JlJZNjfHeNWeTl55SSetibWtPYdC0NYQpbMxRkdT1A6qGlNhFugbwFQ6y6EzY4xOrd7VhjK5PD98cZhvH+vnub4JPBGu3tnCKy/p5KW7Wgl7a9/PPBYJ0d4QZVNTlPaGqJ2hasx5skDfIFSV7qFpnu+fWPVxVc6OTvP944P84MQgo9MZGqIe+3a3c8NFHVy2pXndLojREPVoa4jS0RilrSFiAW/MMlmgbzDT6RxHescYmkiv+nMV2vF/cGKQJ7tHSGXztCYiXHdBG9dd0M5lW5vXtc07HvFoa4jQ1hChNRGhKRa2k5qMWYQF+gZ1bizJsXMTFbnWaDlS2RxPdY/y+IvD/Oj0KOlcnqZYmB/b2co1XW1ctaNl3WvMnie0JiK0xF3AtyTCxMJWizemwAJ9A8vllVNDU5wcnFy1g6alpDI5njkzxg9PuXCfSucIh4Q9W5u4ekcrP7azle2t8Q1RW45HPFoSYVriEZrjYVoSESLrcDzAmI3AAr0KpLN5XhycpHt4igpdma5subxy/NwET/WM8MyZUc6MuG6W7Q0RrtzewpXbW7hiWzPtqzhI2HIloh5NsTDN8TBNcRf26/3rwpi1YIFeRZKZHKeGpjg9PL1uF6QYnEhx6MwYh8+O8WzvOBMpd3LU1uYYl29rZs+WZi7d0kRn08YaxCvsCU0xF/CNURf2jbGw1eZNTbFAr0LpbJ7u4Sl6hqcrPoTAcuRV6R6a4mjfOM/1TnC0b5zpjGvzb0tEuGRLExd3NnLJ5iZ2b2rYkOEZDYdojIVpjHk0Rl3IN0Q9q9GbqmSBXsXyeaV3LMmpoalVGUZg2eVR5fTINMf7Jjh2boITAxMM+L11PBF2tie4qLOR3Zsa2N3RwM62xLr0fy+H5wmNURfu7i9Mwr+9EXdMxoAFes0YncpwemSavrHkhro+6MhUmhMDk5wcmOSFgUlODk7N1OLDIWFHW4Ku9gRdHe7ydzvbEjRv8PFfIuEQiYg3U5NviHokIh6JqEcsHNpQTU2mvlig15hsLs+58RS9Y8k16cu+XHlV+sdTvDg4xYtDk/QMTXNqeIrxwC+M1kSEHW1xdrYl2NGaYHtbnO0tCZriG3+ogFAI4mGPeNRz/yMhEjO3XeCv14lbpvYtFugb/9tj5gl7IXa0JdjRliCZydE/nqJvLMnIKg4rsBwhEbb6Y6jfcFEH4M6SHZ3O0DM8zemR2b/vHBsgHThG0BRz469va42ztSXG1pY4W5pjbG6ObZj+6Pk8TKVzTC1y/kAsEiLmh30h5Av/Y2ELfbM6rIZeQ1JZF+4DE2mGJlNr3v1xJfKqDE6kOTs6Te9Ykt7RJGdHk5wbTzE6PXcH1ZaIsNkP986mGJ1NUTqbYmxqdOPEVFtARsIhYuEQUf9/IegL06LhEFEvtGGPQZj1YTX0OhELe+xqb2BXewO5vDI8lWZoMs3ARIqp1NqcjbpcIZGZkH5J0bzpdI6+MRfu58bd/4GJFEfOjjEylUHnrIeZQcA6GqN0+OPFtPth396w8YYVyGTzZfVg8kIyE/oRb27YRwPTIp4Q9ax9v55ZoNcoLyR+LTbGZVubSWZyjExlGJpMMzKVXrS5YKNIRD0u7Gzkws7GefMyufzMzmpgIs3gZIrBCbcDO9Y3wchUhlzRr89wSGbGjGlriLr/CXffDTMQoSUepjke2VBjuufyynQ6V/YQEWE/2Ath7/7E/feDPxJyt8Mht2y1/boxpVmg14l4xGNbq8e21jjgmmdGpzOMTmUYS2YYm85uqJ4zS4l4oZl2+lLyeWUsmWFoKs3wZIbhqTQjUxlGpt3/08Pu2q+F3jhBAjTGwrQkwjTH3HADzX7QNwfOTm2Kzf5tpGaRbE7J5hZv4y/mhYSwJ4RDs+EfLvwPzd73Qm5nUJjnhYRwSOxXwQZhgV6nYmGPLc0eW5pdIKoqk+kc48kM48ksY9MZJlJZsms4vkwlhUJCW0OUtoYodC68XCqbY2w663Zu04WdW4axZJaxZIaJZJZuv4fOogdBwyGaYu6kpcaoN3Py0sz/mf7u7n+hv3si6hEOrf/OIJdXcnklxcoOvHieC/bCDsHzb3shmXvfX66wIwguV/gzK2eBbgAQkZna5vbW2enJTI7xZJbJVJbJdJbJVI7JdHZNBxJbTbGwx+Zmr6zrsWbzeSZTbqc3kcq6v6T7P5nKuf9p91oND08zmXY7gaV++UQ91+0xEZntAulu+33fIx4xv7dMPOIRL/SYiYTmdJWMRULrtnPI5ZRcbuU7hAIR5oR7MOzDISEkMme+J4Lnuf+hkDu5LRwKudvB5UXqolnJAt0sqhAixYGXzLg23alMjmk/uKbSOZKZXNXW6pcSDoVoTYSWdVk/VSWVzTPlt4EXQn46nWMqnWW68Dqmc0xn3Os3nXG/Ggrzktkc5XZGCwcOoMbCXuD27P3CtGjRAdbInAOtMmdaoR2+MG21rlmrWmgyKmxw5Y71iLhfbp4f8rM7B1ehCU4v7BxCheX8aYXHiPjzZ5aZ3YEUz1tLFuhmRQpB315iXiaXnwmnZDpPKpsjmZn9n87lqqJLZSWIyMxrxfxju2VRVdLZPMls3r2mmbmvZ/B/KpsnFbydzZPO5plIZRmaTJPOzU5LZ/OsdNcbbFcvhP289vdQaOZ2OCSEA8uFA000wfnhQFv+nKaZUtOK/sKhECFhwfZ8Vf+XxIq3emUKO4LC35YW11FhNZQV6CJyC/B/AQ/4uKp+pGh+DPgMcD0wCLxDVU9WtqimWhR6VrQscnp/OutCJ53Nu5DJ5MkUwibnuvNlclpX4b8QESEW8YhFvIpe9FtVyebdL4hMbjbk07nZ++49KLwfhXlKxl+mcDvrL5fN5cnm3Q5oMpcjm8uTyevM9Gxh+VU8AB9sjgl7s80tc5ppQoXaOPOml6rFB2vghfnFtfiQEFjevx+s4fs7m47GKPGwxwWbGiq+7UsGuoh4wN3AG4Ae4ICI7FfVw4HF3gcMq+qlIvJO4H8D76h4aU3NKPzUL0cuPzdACiGRybpgyOTyM8u4/+4AXzbv7q/TuXMbnojM1KjXmqqSU51pXsnm/cAvhH/OzQ++rzPvrbr2+sL7W1hPYfngX9b/nw/OC9zO+/fT2by/HDPL5Es8Nq/M3FdlXtfYcjXFw7xr0+4Kv6rl1dBvAI6r6gkAEbkfuBUIBvqtwJ3+7c8Dfy0iout1GqqpKa4GtfLhbgvhns8zGwLzvthuXvALW/gyF77kqrNf6kIgFZZX1br/JbEcIkJYhHAI2NjjtC0pXwj94s9HiR1AXpX2hig/sWeRrlfnoZxA3wl0B+73AC9faBlVzYrIKLAJGAguJCK3A7f7dydE5OhKCo3riDaw5FK1xba5Ptg214fz2eYFq/ZrelBUVe8B7jnf9YjIwYXGMqhVts31wba5PqzWNpfTeHYa6Arc3+VPK7mMiISBVtzBUWOMMWuknEA/AOwRkYtEJAq8E9hftMx+4N3+7Z8Hvmnt58YYs7aWbHLx28TvAL6C67b4CVU9JCJ3AQdVdT/wD8BnReQ4MIQL/dV03s02Vci2uT7YNteHVdnmdRsP3RhjTGWt/6hAxhhjKsIC3RhjakTVBbqI3CIiR0XkuIh8cL3Ls9pEpEtEviUih0XkkIj81nqXaS2IiCciT4jIl9a7LGtBRNpE5PMi8qyIHBGRH1/vMq02Eflt/zP9jIjcJyKlB7evYiLyCRE5JyLPBKZ1iMjXROSY/7/UkEgrUlWBHhiG4I3AXuA2Edm7vqVadVngA6q6F3gF8Ot1sM0AvwUcWe9CrKH/C/ynql4BvJQa33YR2Qn8JrBPVa/GdbhY7c4U6+FTwC1F0z4IfENV9wDf8O9XRFUFOoFhCFQ1DRSGIahZqnpWVX/o3x7HfdF3rm+pVpeI7AJ+Gvj4epdlLYhIK/AqXG8xVDWtqiPrW6o1EQYS/rkrDcCZdS5Pxanqd3A9/4JuBT7t3/408LZKPV+1BXqpYQhqOtyCRORC4Frg0fUtyar7KPB7cJ5XS6geFwH9wCf9ZqaPi8gKB9utDqp6Gvhz4BRwFhhV1a+ub6nWzFZVPevf7gW2VmrF1RbodUtEmoAvAP9NVcfWuzyrRUTeDJxT1cfXuyxrKAxcB/ytql4LTFLBn+Ebkd9ufCtuZ7YDaBSRd61vqdaefwJmxfqOV1uglzMMQc0RkQguzD+nqv+y3uVZZTcCbxWRk7gmtdeKyD+ub5FWXQ/Qo6qFX16fxwV8LXs98IKq9qtqBvgX4JXrXKa10ici2wH8/+cqteJqC/RyhiGoKeIuv/IPwBFV/Yv1Ls9qU9U/UNVdqnoh7v39pqrWdM1NVXuBbhG53J/0OuYOT12LTgGvEJEG/zP+Omr8QHBAcKiUdwP/VqkVV9Ul6BYahmCdi7XabgR+CfiRiDzpT/tDVX1wHctkKu83gM/5FZUTwHvXuTyrSlUfFZHPAz/E9eR6ghocAkBE7gNuBjpFpAf4I+AjwAMi8j7gReDtFXs+O/XfGGNqQ7U1uRhjjFmABboxxtQIC3RjjKkRFujGGFMjLNCNMaZGWKAbY0yNsEA3xpga8f8B8FPmG1MTLfoAAAAASUVORK5CYII=\n", 162 | "text/plain": [ 163 | "
" 164 | ] 165 | }, 166 | "metadata": { 167 | "needs_background": "light" 168 | }, 169 | "output_type": "display_data" 170 | } 171 | ], 172 | "source": [ 173 | "plt.plot(t, survival_function(estimates_, t))\n", 174 | "plt.fill_between(t, \n", 175 | " y1=survival_function(estimates_, t) + 1.65 * std_sf, \n", 176 | " y2=survival_function(estimates_, t) - 1.65 * std_sf,\n", 177 | " alpha=0.3\n", 178 | " )\n", 179 | "plt.ylim(0, 1)\n", 180 | "plt.title(\"Estimated survival function with CIs (Delta method)\")" 181 | ] 182 | }, 183 | { 184 | "cell_type": "markdown", 185 | "metadata": {}, 186 | "source": [ 187 | "Next, we will explore a subscription service LTV example. Move to Part 7! " 188 | ] 189 | } 190 | ], 191 | "metadata": { 192 | "kernelspec": { 193 | "display_name": "Python 3", 194 | "language": "python", 195 | "name": "python3" 196 | }, 197 | "language_info": { 198 | "codemirror_mode": { 199 | "name": "ipython", 200 | "version": 3 201 | }, 202 | "file_extension": ".py", 203 | "mimetype": "text/x-python", 204 | "name": "python", 205 | "nbconvert_exporter": "python", 206 | "pygments_lexer": "ipython3", 207 | "version": "3.7.3" 208 | } 209 | }, 210 | "nbformat": 4, 211 | "nbformat_minor": 2 212 | } 213 | -------------------------------------------------------------------------------- /full_notebooks/Part 3.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "- Currently using a pretty naive minimization algorithm. It will approximate the gradient and move in that direction. This requires lots of function calls. \n", 8 | "- We have no way to get the Hessian - which is needed for the variance estimates of our parameters. " 9 | ] 10 | }, 11 | { 12 | "cell_type": "markdown", 13 | "metadata": {}, 14 | "source": [ 15 | "### Automatic differentiation - how to differentiate an algorithm\n", 16 | "\n", 17 | "Symbolic differentiation is what we learned in school, and software like SymPy, Wolfram and Mathematica do this well. But I want to differentiate the following:" 18 | ] 19 | }, 20 | { 21 | "cell_type": "code", 22 | "execution_count": 1, 23 | "metadata": {}, 24 | "outputs": [ 25 | { 26 | "data": { 27 | "text/plain": [ 28 | "[]" 29 | ] 30 | }, 31 | "execution_count": 1, 32 | "metadata": {}, 33 | "output_type": "execute_result" 34 | }, 35 | { 36 | "data": { 37 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD4CAYAAADhNOGaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXxU5b348c83O4EACQkBSSBhE1EUMIDsCGix14p70S7gUtqrttpWW3vtdv3VXrdetdWqVBHqAi6tilYEZJUiS5B9D2ENSQh7IOsk398fc/COMQHinMnkJN/36zWvnDnnOTPfnDyZ7zznec55RFUxxhjTfEWEOwBjjDHhZYnAGGOaOUsExhjTzFkiMMaYZs4SgTHGNHNR4Q7g60hOTtaMjIxwh2GMMZ6yevXqQ6qaUnO9JxNBRkYG2dnZ4Q7DGGM8RUT21LbeTg0ZY0wzZ4nAGGOaOUsExhjTzFkiMMaYZs4SgTHGNHOuJAIRmSoiB0VkYx3bRUT+LCI5IrJeRPoHbJsoIjucx0Q34jHGGHPu3GoRTAPGnWH7VUAP5zEZeB5ARJKA3wGDgIHA70Qk0aWYjDHGnANXriNQ1SUiknGGIuOBv6v/ntfLRaStiHQERgHzVPUIgIjMw59QZrgRlzHNXbmviryjpeQdK+XIqQpOlPk4UVqJr0qpVkVViYmKoEVMFPExkbRtEU1yQiztWsbQoU0c8TGevNTI1FND/ZU7AfsCnu931tW1/itEZDL+1gSdO3cOTZTGeFhJhY+Vu46wfv9x1u8/zuYDx8k/UUYwU460axlDWlI8me3i6ZGaQPf2rejVIYHOSfGIiHvBm7DyTLpX1SnAFICsrCybTccYoKi4nH+tP8CCbUUszz1Mha8aEeia3JIBmUlkJrckPTGetMQWtGsVS+sWUbSOiyYmMgIREBEqfNWUVlRxqsLHsZJKDp0s59DJcvKPl7H/aAn7jpSyavdR3lt74Iv3TYiL4sLzWnNJWlv6d0nk0i6JJLeKDeORMMFoqESQB6QHPE9z1uXhPz0UuH5RA8VkjCdVVyuf5hxi5sq9zNtciK9a6ZrSku9d1oVR56fQN70tCXHR5/x6MVERxERF0CY+mvPatqiz3MlyHzkHT7I1/wQb8o6zMe84r/x7Ny8uyQUgM7kll3VNYnC3ZAZ3bUdKgiUGr2ioRDALuEdEZuLvGD6uqvkiMgf4Y0AH8ZXArxooJmM8RVVZsPUgT87dzpb8EyTGRzNpSAbfHpBOj9SEkL9/q9go+qa3pW96WyY468oqq9h04Dir9xxl5a4jfLgunxkr/Wd7e3dszcjzUxjZM4WsLolERdpo9cZK3JizWERm4P9mnwwU4h8JFA2gqi+I/2Tis/g7gkuA21Q129n3duC/nJd6RFVfOdv7ZWVlqd10zjQna/cd478/2MSavcfonBTPvWN6cPUlHYmNigx3aF/iq6pm04ETLM05xJLtRazecxRftdKmRTSjzk9h7AWpXN6rPa1iPXNWukkRkdWqmvWV9V6cvN4SgWkuyn1VPPPJDl5YvJOUhFjuHdOTm7LSiPbIt+viskqW7jjEJ1sOsmBrIUdLKomJimBEjxS+2acDY3un0roep7FMcOpKBJaWjWmkthac4N4Za9lWWMzNWWn8+urenvvQTIiL5qo+HbmqT0eqqpXVe44ye2M+H28s4JMthcRERXD5+Slcc0knxlzQnrjoxtXCaS6sRWBMI7R4exF3v/45LWIieeyGPozulRrukFxVXa2s2XeMD9cf4MP1+RQVl5MQG8VVfTpwff80BmYkERFhw1PdZqeGjPGIN1ft5b/e3UjP1ARemTSADm3iwh1SSFVVK8tzD/Pumjxmb8jnVEUV6UktuLF/Ojdc2om0xPhwh9hkWCIwxgOeW5jDE3O2MaJnCs/d2q9ew0CbgpIKH3M3FfL26n38O+cwIjCsezK3DuzM2N6pnukbaawsERjTyL2VvY9fvLOe8X3P48mbLmn2H3r7jpTwzur9vJW9j/zjZSS3iuXbA9K4dVAXOp3hegdTN0sExjRii7cXcfu0VQzu2o6pkwYQE9W8k0Cgqmpl8faDvLFiHwu2FgIwuld7vjc4g+Hdk60voR5s1JAxjdTGvOPc9dpqeqYm8Px3+1sSqCEyQhjdK5XRvVLJO1bKjBV7mblqL59sWUnX5JZMHJLBDZem2bUJQbAWgTFhVFLhY9zTn+Krqubdu4eS2rppdwy7pdxXxewNBUxbtpu1+46REBvFtwekM3FIBulJ1rlcF2sRGNMIPf7xNvYeKeHNyZdZEqiH2KhIru3XiWv7dWLtvmNMXbqLact2M/Xfuxh3UQd+MLwr/Trb1CbnyloExoTJ8tzDTJiynElDMvj9NReGOxzPyz9eyvRle3hjxR5OlPkYmJHED0Z0ZUyv9taP4LDOYmMakdOnhERg9r3DbQIYF50s9/HWqn28vHQXecdK6d6+FT8c0ZXxfTs1+/6XuhJB8z4qxoTJk3O2s/dICY/fcLElAZe1io3i9mGZLH5gFM9M6EtUhPDAO+sZ+cRCpi7dRUmFL9whNjqWCIxpYPnHS3l1+W5uGZjOoK7twh1OkxUVGcH4vp2Yfe9wpt02gM5J8Tz84WaGPbaQv8zfwfHSynCH2GjYVxFjGtiUJbmowl2juoc7lGZBRBh1fntGnd+e7N1HeG5hDn+at50pn+YyaUgGtw/NJLFlTLjDDCtrERjTgIqKy5mxci/X9utkwxzDICsjiVduG8iHPx7G0G7J/GVBDsMeW8Cjs7dy+GR5uMMLG0sExjSgl5fuotxXzV2juoU7lGbtok5teOF7lzL3pyO4vFd7Xlyyk+GPL+R/Zm/hyKmKcIfX4FxJBCIyTkS2iUiOiDxYy/anRGSt89guIscCtlUFbJvlRjzGNEbHSip49bPdXH3xeXRNaRXucAzQMzWBZ2/tz9z7RjD2glSmLMll+GMLeGLOVo6VNJ+EEPTwURGJBLYDVwD7gVXALaq6uY7yPwb6qertzvOTqlqv/wobPmq86OlPtvP0Jzv4+L7h9OrQOtzhmFrsKCzmmfk7+NeGfFrGRHHHsEzuGJ7puQmB6hLK4aMDgRxVzVXVCmAmMP4M5W8BZrjwvsZ4RlW18tryPYzp1d6SQCPWw2khfHzvCIZ1T+aZ+TsY/thCnluYw6nypjvs1I1E0AnYF/B8v7PuK0SkC5AJLAhYHSci2SKyXESuretNRGSyUy67qKjIhbCNaTjZu49w6GQF1/ar9V/DNDLnd0jghe9dyoc/HsalXRJ5Ys42Rj6xkGn/3kW5ryrc4bmuoTuLJwDvqGrgkeziNFVuBZ4WkVp70VR1iqpmqWpWSkpKQ8RqjGtmbyzwz8/bq324QzH1cFGnNkydNIB//OcQurdvxe8/2MzoJxfzdvY+qqq9d1eGuriRCPKA9IDnac662kygxmkhVc1zfuYCi4B+LsRkTKOhqszZVMCIHil2q2SPurRLIjN+cBmv3TGI5FYxPPDOesY9vYQ5mwrw4m16anIjEawCeohIpojE4P+w/8roHxHpBSQCnwWsSxSRWGc5GRgK1NrJbIxXrdt/nPzjZYy7qEO4QzFBEBGG9UjmvbuH8vx3+lOlyg9fXc31zy9j5a4j4Q4vKEEnAlX1AfcAc4AtwFuquklEHhaRawKKTgBm6pfT5wVAtoisAxYCj9Y12sgYr5q9MZ+oCOGKC1LDHYpxgYhwVZ+OzL1vBI9e34cDx0q5+cXPuGPaKrYXFoc7vK/F7j5qTAipKpc/uYj0pHhevWNQuMMxIVBaUcW0Zbv56yL/yKKbLk3np1f0pEObxje/hN191Jgw2FpQzO7DJVx1Ucdwh2JCpEVMJP85qhtLHric24dm8u6aPEY9uZA/zd3GSY8MObVEYEwIzd5YgAhc0dtOCzV1iS1j+PXVvZn/85Fc2bsDf1mQw6gnFvLa8j34qqrDHd4ZWSIwJoTmbipgQEYSKQmx4Q7FNJD0pHj+fEs/3r97KF1TWvHr9zbyjaeXsGBrYaMdYWSJwJgQKa2oYlthMUO62ZwDzdEl6W15c/JlTPnepajC7dOy+e7LK9h84ES4Q/sKSwTGhEjOwZOowvmpCeEOxYSJiHDlhR2Y89MR/P5bvdl04AT/8ZdP+eU76zlYXBbu8L5gicCYEDk9lLBnB0sEzV10ZASThmay+H5/h/I/1+zn8icW8dzCHMoqw3/LCksExoTI9sJiYiIj6GIT0BhHm/hofnN1b+b+dCRDuifzxJxtjPnTYv61Pj+s/QeWCIwJkW2FxXRr34qoSPs3M1+WmdySv30/izfuHERCXBR3v/E5335xORvzjoclHquhxoTIjsKT9Ey1CWhM3YZ0T+ZfPxnOH6/rQ07RSb717FIe/Md6DjXwtJmWCIwJgeKySvKOldLTOorNWURGCLcO6szC+0dxx9BM3lnt7z/425JcKnwNc/2BJQJjQmDHwZMAlgjMOWvTIppfX92bOT8dwaUZiTzy0RbGPbOERdsOhvy9LREYEwLbC/wjhmzoqKmvbimtmHbbQKZOykIVJr2yijunr2LP4VMhe09LBMaEwLbCYlpER5KW2CLcoRiPGt0rlY/vG86DV/Xis52HueJ/l/DEnK2UVLh//yJLBMaEwI7Ck/RIbUVEhIQ7FONhsVGR/GhkNxbcP4r/uLgjLy7OZd+RUtffxxKBMSGwrbDY+geMa1Jbx/HUt/uy6IFRnB+CCxRdSQQiMk5EtolIjog8WMv2SSJSJCJrncedAdsmisgO5zHRjXiMCaejpyooKi63oaPGdWmJobk4MegJVEUkEngOuALYD6wSkVm1zDT2pqreU2PfJOB3QBagwGpn36PBxmVMuHxxawlrERiPcKNFMBDIUdVcVa0AZgLjz3HfbwDzVPWI8+E/DxjnQkzGhM12GzpqPMaNRNAJ2BfwfL+zrqYbRGS9iLwjIun13NcYz9heUExCbBQdG+FUhcbUpqE6iz8AMlT1Yvzf+qfX9wVEZLKIZItIdlFRkesBGuOWbYXF9OyQgIiNGDLe4EYiyAPSA56nOeu+oKqHVfX0zTNeAi49130DXmOKqmapalZKSooLYRsTGjsKi62j2HiKG4lgFdBDRDJFJAaYAMwKLCAigTN3XwNscZbnAFeKSKKIJAJXOuuM8aSqauVoSSWpre20kPGOoEcNqapPRO7B/wEeCUxV1U0i8jCQraqzgJ+IyDWADzgCTHL2PSIi/w9/MgF4WFWPBBuTMeFS7vNPMhIXHRnmSIw5d0EnAgBV/Qj4qMa63wYs/wr4VR37TgWmuhGHMeFWXum/W2RslF2rabzDaqsxLir3nU4E1iIw3mGJwBgXnZ5/Ni7a/rWMd1htNcZF1iIwXmSJwBgXne4stj4C4yVWW41xUZnTWWyjhoyXWCIwxkVftAisj8B4iNVWY1xkw0eNF1ltNcZFZXZBmfEgSwTGuMhaBMaLrLYa46KyL0YNWYvAeIclAmNcVP7FqCH71zLeYbXVGBfZBWXGiywRGOOi07eYsD4C4yVWW41xUbmvmpjICCIibHYy4x2WCIxxUbmvyloDxnOsxhrjorLKamLtGgLjMa4kAhEZJyLbRCRHRB6sZfvPRGSziKwXkfki0iVgW5WIrHUes2rua4yXWIvAeFHQM5SJSCTwHHAFsB9YJSKzVHVzQLE1QJaqlojIfwKPA992tpWqat9g4zCmMSj3Vdt9hoznuFFjBwI5qpqrqhXATGB8YAFVXaiqJc7T5UCaC+9rTKNTXllFnA0dNR7jRiLoBOwLeL7fWVeXO4DZAc/jRCRbRJaLyLV17SQik51y2UVFRcFFbEyIWIvAeJErk9efKxH5LpAFjAxY3UVV80SkK7BARDao6s6a+6rqFGAKQFZWljZIwMbUU3lltfURGM9xo8bmAekBz9OcdV8iImOBh4BrVLX89HpVzXN+5gKLgH4uxGRMWJT5quzOo8Zz3EgEq4AeIpIpIjHABOBLo39EpB/wIv4kcDBgfaKIxDrLycBQILCT2RhPsRaB8aKgTw2pqk9E7gHmAJHAVFXdJCIPA9mqOgt4AmgFvC0iAHtV9RrgAuBFEanGn5QerTHayBhP8Q8ftRaB8RZX+ghU9SPgoxrrfhuwPLaO/ZYBfdyIwZjGoKyy2u48ajzHaqwxLrIWgfEiSwTGuKjcZy0C4z1WY41xiapSVmktAuM9lgiMcYmvWqlWm4vAeI/VWGNccnp2MruOwHiNJQJjXPLF7GTWR2A8xmqsMS75v/mK7d/KeIvVWGNcUu60COzUkPEaSwTGuKSs0loExpusxhrjknKf00dgw0eNx1giMMYlX/QRWGex8Rirsca45ItRQ9YiMB5jicAYl9ioIeNVVmONcUmZjRoyHmWJwBiXWIvAeJXVWGNcYreYMF7lSiIQkXEisk1EckTkwVq2x4rIm872FSKSEbDtV876bSLyDTfiMSYcyu0WE8ajgq6xIhIJPAdcBfQGbhGR3jWK3QEcVdXuwFPAY86+vfHPcXwhMA74q/N6xniOnRoyXuVGjR0I5KhqrqpWADOB8TXKjAemO8vvAGPEP3nxeGCmqpar6i4gx3k9YzynvLIKEYiJtERgvMWNGtsJ2BfwfL+zrtYyquoDjgPtznFfAERksohki0h2UVGRC2Eb464yXzWxURH4v+MY4x2e+eqiqlNUNUtVs1JSUsIdjjFfUW6zkxmPciMR5AHpAc/TnHW1lhGRKKANcPgc9zXGE2y+YuNVbtTaVUAPEckUkRj8nb+zapSZBUx0lm8EFqiqOusnOKOKMoEewEoXYjKmwdl8xcarooJ9AVX1icg9wBwgEpiqqptE5GEgW1VnAS8Dr4pIDnAEf7LAKfcWsBnwAXeralWwMRkTDuVOH4ExXhN0IgBQ1Y+Aj2qs+23AchlwUx37PgI84kYcxoST/9SQtQiM99jXF2Nc4j81ZP9Sxnus1hrjknJftV1VbDzJaq0xLin3VRFnncXGgywRGOOSskprERhvslprjEvKfTZ81HiTJQJjXFJeaReUGW+yWmuMS+yCMuNVlgiMcYmNGjJeZbXWGBeoqnNlsbUIjPdYIjDGBTYpjfEyq7XGuMDmKzZeZonAGBeU+5z5iq1FYDzIaq0xLiivtFNDxrus1hrjgtMtAjs1ZLzIEoExLiizFoHxMKu1xrjgiz4CaxEYDwoqEYhIkojME5Edzs/EWsr0FZHPRGSTiKwXkW8HbJsmIrtEZK3z6BtMPMaEy+k+gjhrERgPCrbWPgjMV9UewHzneU0lwPdV9UJgHPC0iLQN2P6AqvZ1HmuDjMeYsCizFoHxsGATwXhgurM8Hbi2ZgFV3a6qO5zlA8BBICXI9zWmUbFRQ8bLgq21qaqa7ywXAKlnKiwiA4EYYGfA6kecU0ZPiUjsGfadLCLZIpJdVFQUZNjGuMsuKDNedtZEICKfiMjGWh7jA8upqgJ6htfpCLwK3Kaq1c7qXwG9gAFAEvDLuvZX1SmqmqWqWSkp1qAwjUtZpV1QZrwr6mwFVHVsXdtEpFBEOqpqvvNBf7COcq2BfwEPqerygNc+3ZooF5FXgPvrFb0xjYS1CIyXBfv1ZRYw0VmeCLxfs4CIxADvAn9X1XdqbOvo/BT8/Qsbg4zHmLCwW0wYLwu21j4KXCEiO4CxznNEJEtEXnLK3AyMACbVMkz0dRHZAGwAkoE/BBmPMWFhF5QZLzvrqaEzUdXDwJha1mcDdzrLrwGv1bH/6GDe35jGotxXRVSEEBVpicB4j9VaY1xQXlltrQHjWVZzjXFBma/KLiYznmWJwBgXlFdW2+0ljGdZzTXGBf6J661FYLzJEoExLiirrLI+AuNZVnONcYG1CIyXWSIwxgXlPmsRGO+ymmuMC8ps+KjxMKu5xrig3Fdt9xkynmWJwBgX2Kkh42VWc41xgf/KYmsRGG+yRGCMC8p9VcRF27+T8Saruca4wFoExsssERjjgjJrERgPs5prTJCqqpXKKrUWgfGsoBKBiCSJyDwR2eH8TKyjXFXApDSzAtZnisgKEckRkTed2cyM8ZQKZ5rKWGsRGI8KtuY+CMxX1R7AfOd5bUpVta/zuCZg/WPAU6raHTgK3BFkPMY0uEMnywFo2yI6zJEY8/UEmwjGA9Od5en45x0+J848xaOB0/MY12t/YxqLHQeLAeiR2irMkRjz9QSbCFJVNd9ZLgBS6ygXJyLZIrJcRE5/2LcDjqmqz3m+H+gUZDzGNLhtBScB6JGaEOZIjPl6zjpnsYh8AnSoZdNDgU9UVUVE63iZLqqaJyJdgQXOhPXH6xOoiEwGJgN07ty5PrsaE1LbC4s5r00crePs1JDxprMmAlUdW9c2ESkUkY6qmi8iHYGDdbxGnvMzV0QWAf2AfwBtRSTKaRWkAXlniGMKMAUgKyurroRjTIPbXlhsrQHjacGeGpoFTHSWJwLv1ywgIokiEussJwNDgc2qqsBC4MYz7W9MY1ZVrew4eJLzO1giMN4VbCJ4FLhCRHYAY53niEiWiLzklLkAyBaRdfg/+B9V1c3Otl8CPxORHPx9Bi8HGc8ZlVVW4c8/xrhjz+FTVPiq6dHeOoqNd5311NCZqOphYEwt67OBO53lZUCfOvbPBQYGE0N9/Oa9jeQdK+V337rQvsEZV2wv9HcUW30yXtasroC5OL0tmw6c4Jt//pTfz9rE8ZLKcIdkPG57YTEi0N1aBMbDmlUi+N5lXVh0/ygmDEhn+me7ufxPi3hjxV6qqu10kfl6thcWk54YT3xMUI1rY8KqWSUCgMSWMTxyXR8+uGcY3VJa8l/vbuCaZ5eyaveRcIdmPGh7YTE9bcSQ8bhmlwhOu6hTG9764WD+fEs/jpyq4KYXPuPHM9aQd6w03KEZj6jwVZNbdIqedkWx8bhmmwgARIRrLjmP+T8fyU/G9GDupgLG/GkRT83bTmlFVbjDM43c7sOn8FWrdRQbz2vWieC0+JgofnZFT+b/fCRjLkjlmfk7GP2nRby/Ns+Gm5o6bStw7jHU3hKB8TZLBAHSEuN57tb+vPXDwbRrFcO9M9dy/fPL+Hzv0XCHZhqh7YXFREYIXVNahjsUY4JiiaAWAzOTmHX3MB6/8WL2Hy3l+r8u4yfWf2Bq2F5YTEa7eOKibUIa422WCOoQESHcnJXOovtH8ePR3ZmzqYDRTy7i8Y+3Ulxm1x8Y/8VkNmLINAWWCM6iZWwUP7/yfBbcP4qrLurAXxft5PInF/H6ij34qqrDHZ4Jk7LKKvYcPmWJwDQJlgjOUae2LXh6Qj/ev3somckteejdjVz1zKcs3HrQOpSboc/3HKVa4cLzWoc7FGOCZomgni5Jb8tbPxzMC9+9FF+1ctu0VXznpRVszKvX9ArG4z7eVEBcdATDeiSHOxRjgmaJ4GsQEcZd1IG5Px3Bf19zIVvyT3D1X5Zy38w17D9aEu7wTIhVVysfbyxgVM/2dmsJ0yRYIghCdGQEE4dksPgXl3PXqG7M3ljA6CcX84cPN3P0VEW4wzMhsmbfUQ4WlzPuotom7jPGeywRuKB1XDS/GNeLRQ+M4tp+5zH137sY8fhCnluYY1coN0GzNxQQHSmMvqB9uEMxxhWWCFzUsU0LHr/xEj6+bwSDurbjiTnbGPnEQl5fsYdKG2HUJKgqH28qYFj3ZJuj2DQZQSUCEUkSkXkissP5mVhLmctFZG3Ao0xErnW2TRORXQHb+gYTT2PRMzWBlyZm8faPBtM5KZ6H3t3IFf+7mFnrDlBtt7z2tE0HTrD/aKmdFjJNSrAtggeB+araA5jvPP8SVV2oqn1VtS8wGigB5gYUeeD0dlVdG2Q8jcqAjCTe/tFgpk7KIi46kp/MWMM3//wp87cU2pBTj5q9MZ/ICOGK3pYITNMRbCIYD0x3lqcD156l/I3AbFVtNkNrRITRvVL56CfDeWZCX0orq7hjejbXP7+Mf+ccCnd4pp4+3ljAoMwkklrGhDsUY1wTbCJIVdV8Z7kASD1L+QnAjBrrHhGR9SLylIjE1rWjiEwWkWwRyS4qKgoi5PCIiBDG9+3EJz8byR+v60PB8TK+89IKJkz5zCbF8YjNB06ws+iUnRYyTY6c7RSFiHwC1FbzHwKmq2rbgLJHVfUr/QTOto7AeuA8Va0MWFcAxABTgJ2q+vDZgs7KytLs7OyzFWvUyiqrmLlyL88u3Mmhk+UM75HMfWN7cmmXWg+faQR+MmMN87cUsvSXo0m0FoHxIBFZrapZNdef9WoYVR17hhctFJGOqprvfKgfPMNL3Qy8ezoJOK99ujVRLiKvAPefLZ6mIi46kklDM7l5QDqvLd/DC4tzueH5ZYzsmcK9Y3vQv7MlhMZk16FTfLj+AD8Y3tWSgGlygj01NAuY6CxPBN4/Q9lbqHFayEkeiIjg71/YGGQ8nhMfE8XkEd349BeX84tx57N+/zGu/+syvj91Jav32DwIjcXzi3KIjozgjuGZ4Q7FGNcFmwgeBa4QkR3AWOc5IpIlIi+dLiQiGUA6sLjG/q+LyAZgA5AM/CHIeDyrZWwUd43qztJfjubBq3qxMe84Nzy/jO+8tJzluYfDHV6ztv9oCf/8PI9bBnamfUJcuMMxxnVn7SNojJpCH8HZlFT4eGPFXl5YnMuhk+UMyEjkrsu7M6pnCv4GlGkov3lvIzNX7WXxA5dzXtsW4Q7HmK+trj4Cu7K4kYqPieLO4V1Z+svL+f23epN3tJTbXlnF1X9ZyofrD1BlF6Y1iMITZbyZvY8b+qdZEjBNliWCRu50p/KiBy7n8RsvprSiinveWMOYP/knxymrtHsZhYqq8pv3/N1Wd43qHuZojAkdSwQeERMVwc1Z6cz72Uie/05/2rSI5qF3NzLssYU8u2AHx0rsbqdum7XuAHM3F/LzK3rSuV18uMMxJmSsj8CjVJXPcg8zZUkui7YV0SI6kpuz0rh9WCZd2rUMd3ied7C4jCufWkJmckve+dEQIiOsX8Z439e+jsA0TiLCkG7JDOmWzLaCYv72aS5vrNzL35fvYewFqdw+NJPLuiZZx/LXoKr8+t2NlFRU8cSNl1gSME2eJYIm4PwOCTx50yX84hvn8+ryPby2fA/zNhfSq0MCk4ZkMNEQuJsAAApYSURBVL5vJ1rERIY7TM94fcVe5m4u5FdX9aJ7+1bhDseYkLNTQ01QWWUV763JY9qy3WwtKKZtfDQ3Z6XznUGd7bTRWczfUsgP/p7N8B4pTJ00wFoDpkmp69SQJYImTFVZuesI0z/bzdxNhfiqlRE9U/jOoM6M7tWe6EgbKxBo3b5jTJiynG7tW/Lm5MG0jLUGs2larI+gGRIRBnVtx6Cu7Sg8UcbMlfuYsXIvP3x1Ne0TYrkpK42bs9KtlQDkFp3k9mmraNcqhqmTBlgSMM2KtQiaGV9VNYu2FTFj5V4WbjtItcKgzCRuzkrnqj4diI9pfh+Aq/cc4c7p2YgIb/9oMN1SrF/ANE12ash8Rf7xUv75eR5vZe9jz+ES4mMiGXdRB67vl8bgbu2axfnxjzbkc9+bazmvTRzTbhtIRrK1jkzTZYnA1Ol0X8K7a/L41/p8ist9tE+I5T8u7sg1l5xH3/S2TW4YaoWvmmcX7OAvC3Po3zmRv30/y2YdM02eJQJzTsoqq/hkSyEfrDvAwq1FVFRV06ltC666qANX9elAv/REIjzeUth84AT3v72OzfknuKF/Go9cdxFx0Ta81jR9lghMvZ0oq2TOxgJmbyxg6Y5DVFRVk5IQy5he7Rl7QSpDuyd76vqEYyUVvLA4l5eX5tKmRTR/vK4PV15o006a5sMSgQnKibJKFmw5yLzNhSzeXsTJch+xUREM6tqOET2SGdkzhe7tWzXKU0jFZZVMXbqblz7N5WSFj+v6duLXV/e2U0Gm2bFEYFxT4atmee5hFm0rYvH2g+wsOgVAcqtYBndrx2Vdk8jqkkSP9q3CdhpJVVm77xgzV+7jg/UHKKmo4sreqfzsyp706tA6LDEZE24huY5ARG4Cfg9cAAxU1Vo/nUVkHPAMEAm8pKqnZzLLBGYC7YDVwPdU1W6j2cjFREUwomcKI3qmAL3Zd6SEZTsP8dnOwyzbeZgP1h0AICEuir7pbenTqQ19OrXhok5t6NS2RciSQ7mvilW7jrJg60EWbjvIrkOniI+J5FsXn8d3L+tCn7Q2IXlfY7wuqBaBiFwAVAMvAvfXlghEJBLYDlwB7AdWAbeo6mYReQv4p6rOFJEXgHWq+vzZ3tdaBI2XqrLncAmr9xwle89R1uw9Ss7Bk/iciXRaREfSvX0rurdvRXpSPOmJLUhLjCclIZaUVrG0bhF1xtNLqkpZZTUHjpey70gJ+46WsiX/BBv2H2drwQkqq5SYqAgGd23HuIs6cPXFHUmIi26oX9+YRi0kLQJV3eK8+JmKDQRyVDXXKTsTGC8iW4DRwK1Ouen4WxdnTQSm8RIRMpJbkpHckhsuTQP8I5G2FRSzOf8EOwpPsuNgMStyD/Pe2jxqfg+JihBaxkYRHxNJi5hIBFCFKlVOlfs4Ueqjoqr6S/skxEVxcVob7hjWlQEZiQzp5q1ObGPCrSEuI+0E7At4vh8YhP900DFV9QWs71TXi4jIZGAyQOfOnUMTqQmJuOhILklvyyXpbb+0vsJXzYFjpeQdK+XQyXKKiss5fKqCknIfJRVVlDizr0WIECHQMjaK1nHRtG4RRYfWcU6LIp72CbGeH9JqTDidNRGIyCdAbWPsHlLV990PqXaqOgWYAv5TQw31viZ0YqIivmg9GGPC56yJQFXHBvkeeUB6wPM0Z91hoK2IRDmtgtPrjTHGNKCGuA/xKqCHiGSKSAwwAZil/l7qhcCNTrmJQIO1MIwxxvgFlQhE5DoR2Q8MBv4lInOc9eeJyEcAzrf9e4A5wBbgLVXd5LzEL4GfiUgO/j6Dl4OJxxhjTP3ZBWXGGNNM1DV81KaoMsaYZs4SgTHGNHOWCIwxppmzRGCMMc2cJzuLRaQI2PM1d08GDrkYjlssrvqxuOrH4qqfphpXF1VNqbnSk4kgGCKSXVuvebhZXPVjcdWPxVU/zS0uOzVkjDHNnCUCY4xp5ppjIpgS7gDqYHHVj8VVPxZX/TSruJpdH4Exxpgva44tAmOMMQEsERhjTDPX5BOBiDwhIltFZL2IvCsibesoN05EtolIjog82ABx3SQim0SkWkTqHA4mIrtFZIOIrBWRkN9prx5xNfTxShKReSKyw/mZWEe5KudYrRWRWSGM54y/v4jEisibzvYVIpIRqljqGdckESkKOEZ3NlBcU0XkoIhsrGO7iMifnbjXi0j/RhDTKBE5HnCsfhvqmJz3TReRhSKy2flfvLeWMu4eL1Vt0g/gSiDKWX4MeKyWMpHATqArEAOsA3qHOK4LgPOBRUDWGcrtBpIb8HidNa4wHa/HgQed5Qdr+zs62042wDE66+8P3AW84CxPAN5sJHFNAp5tqPoU8L4jgP7Axjq2fxOYDQhwGbCiEcQ0CvgwDMeqI9DfWU4Attfyd3T1eDX5FoGqztX/mxd5Of6Z0GoaCOSoaq6qVgAzgfEhjmuLqm4L5Xt8HecYV4MfL+f1pzvL04FrQ/x+Z3Iuv39gvO8AY0Qk1BMrh+Pvck5UdQlw5AxFxgN/V7/l+Gcv7BjmmMJCVfNV9XNnuRj/PC4153N39Xg1+URQw+34s2hNnYB9Ac/389UDHy4KzBWR1SIyOdzBOMJxvFJVNd9ZLgBS6ygXJyLZIrJcREKVLM7l9/+ijPNF5Dj+yZdC6Vz/Ljc4pxPeEZH0WraHQ2P9HxwsIutEZLaIXNjQb+6cUuwHrKixydXjddY5i71ARD4BOtSy6SFVfd8p8xDgA15vTHGdg2Gqmici7YF5IrLV+SYT7rhcd6a4Ap+oqopIXeOeuzjHqyuwQEQ2qOpOt2P1sA+AGapaLiI/xN9qGR3mmBqrz/HXp5Mi8k3gPaBHQ725iLQC/gHcp6onQvleTSIRqOrYM20XkUnA1cAYdU6w1ZAHBH4zSnPWhTSuc3yNPOfnQRF5F3/zP6hE4EJcDX68RKRQRDqqar7TBD5Yx2ucPl65IrII/7cptxPBufz+p8vsF5EooA1w2OU46h2XqgbG8BL+vpfGICR1KhiBH76q+pGI/FVEklU15DejE5Fo/EngdVX9Zy1FXD1eTf7UkIiMA34BXKOqJXUUWwX0EJFMEYnB37kXshEn50pEWopIwull/B3ftY5waGDhOF6zgInO8kTgKy0XEUkUkVhnORkYCmwOQSzn8vsHxnsjsKCOLyENGleN88jX4D//3BjMAr7vjIa5DDgecCowLESkw+l+HREZiP/zMtTJHOc9Xwa2qOr/1lHM3ePV0D3iDf0AcvCfS1vrPE6P5DgP+Cig3Dfx987vxH+KJNRxXYf/vF45UAjMqRkX/tEf65zHpsYSV5iOVztgPrAD+ARIctZnAS85y0OADc7x2gDcEcJ4vvL7Aw/j/8IBEAe87dS/lUDXUB+jc4zrf5y6tA5YCPRqoLhmAPlApVO/7gB+BPzI2S7Ac07cGzjDSLoGjOmegGO1HBjSQMdqGP6+wfUBn1vfDOXxsltMGGNMM9fkTw0ZY4w5M0sExhjTzFkiMMaYZs4SgTHGNHOWCIwxppmzRGCMMc2cJQJjjGnm/j+wSaPC1yIHwQAAAABJRU5ErkJggg==\n", 38 | "text/plain": [ 39 | "
" 40 | ] 41 | }, 42 | "metadata": { 43 | "needs_background": "light" 44 | }, 45 | "output_type": "display_data" 46 | } 47 | ], 48 | "source": [ 49 | "%matplotlib inline\n", 50 | "import numpy as np\n", 51 | "from matplotlib import pyplot as plt\n", 52 | "\n", 53 | "\n", 54 | "def f(x):\n", 55 | " y = 0.\n", 56 | " for i in range(100):\n", 57 | " y = np.sin(y + x)\n", 58 | " return y\n", 59 | "\n", 60 | "x = np.linspace(-2, 2, 100)\n", 61 | "plt.plot(x, [f(x_) for x_ in x])" 62 | ] 63 | }, 64 | { 65 | "cell_type": "markdown", 66 | "metadata": {}, 67 | "source": [ 68 | "Mathematically, it's something like:\n", 69 | "\n", 70 | "$$f(x) = \\sin(x + \\sin(x + \\sin(x + ...\\sin(x)))...)$$\n", 71 | "\n", 72 | "Good luck differentiating that and getting a nice closed form. If this is not complicated enough for you, feel free to add some `if` statements. \n", 73 | "\n", 74 | "We can use `autograd`, an automatical diff package, to compute _exact, pointwise_ derivatives." 75 | ] 76 | }, 77 | { 78 | "cell_type": "code", 79 | "execution_count": 2, 80 | "metadata": {}, 81 | "outputs": [ 82 | { 83 | "data": { 84 | "text/plain": [ 85 | "-0.26242653107144726" 86 | ] 87 | }, 88 | "execution_count": 2, 89 | "metadata": {}, 90 | "output_type": "execute_result" 91 | } 92 | ], 93 | "source": [ 94 | "from autograd import grad\n", 95 | "from autograd import numpy as np\n", 96 | "\n", 97 | "def f(x):\n", 98 | " y = 0.\n", 99 | " for i in range(100):\n", 100 | " y = np.sin(y + x)\n", 101 | " return y\n", 102 | "\n", 103 | "\n", 104 | "grad(f)(1.)\n", 105 | "# magic! " 106 | ] 107 | }, 108 | { 109 | "cell_type": "markdown", 110 | "metadata": {}, 111 | "source": [ 112 | "Of course, you can string together these pointwise derivatives into a function:" 113 | ] 114 | }, 115 | { 116 | "cell_type": "code", 117 | "execution_count": 3, 118 | "metadata": {}, 119 | "outputs": [ 120 | { 121 | "data": { 122 | "text/plain": [ 123 | "[]" 124 | ] 125 | }, 126 | "execution_count": 3, 127 | "metadata": {}, 128 | "output_type": "execute_result" 129 | }, 130 | { 131 | "data": { 132 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de3Bc5Zkm8Oftu64tWWpLsixjG5uLMdgwwtxhwiUFhODNJjskmbAhhDFkk2wyla1MZrOzNUnVzO5saphkJpNMGGCADSGZcEkMFRIgmIAXbJDBsjHGxpaNLevWlq27+v7uH31aluRuqSX16T7n+PlVqXSkPup+1ZIeff197zlHVBVERGRdrlIXQEREM2NQExFZHIOaiMjiGNRERBbHoCYisjiPGXdaX1+vy5cvN+OuiYgcaceOHcdVNZTtNlOCevny5WhrazPjromIHElEPsx1G6c+iIgsjkFNRGRxDGoiIotjUBMRWRyDmojI4hjUREQWx6AmIrI4U/qoiaxkNJrAI68fRjSeBACU+z2468rlCHjdJa6MKD8ManK8rQeO43u/2zflcxc2B3HVqvoSVUQ0N5z6IMeLGCPp33/jOvz6y1dN+RyRHTCoyfGiiRQAwOd2wedJ/8rHjM8R2QGnPsjxMkHt97gQS7qmfI7IDmYdUYvIuSKyc9LbkIh8vRjFERVCbCKo3fBzRE02NOuIWlX3AVgPACLiBnAMwDMm10VUMJlQ9nlc8GVG1EkGNdnHXOeobwBwUFVzno6PyGomB7Xf7Z7yOSI7mGtQfxrAE9luEJFNItImIm3hcHjhlREVSDSRhNslcLtkYjExmmDXB9lH3kEtIj4AtwP4ZbbbVfUBVW1V1dZQKOtFCohKIpZITcxNs+uD7GguI+pbALytqr1mFUNkhlgyNRHQbpfA4xIGNdnKXIL6M8gx7UFkZbFECj73qV91n8fFoCZbySuoRaQCwE0Anja3HKLCiyZOjaiBdFCzj5rsJK8DXlR1FECdybUQmWLyHDVgHPjCoCYb4SHk5HjpEfWpM+X5PC7E2EdNNsKgJsebvJgIpM/5wRE12QmDmhwvGk/CP2Ux0c0+arIVBjU5XiyZgt87dY6ai4lkJwxqcjy255HdMajJ8WLT2vP8XEwkm2FQk+Od1kftdiEaZ1CTfTCoyfFO66P2ckRN9sKgJsdjex7ZHYOaHC+9mDjtgBcGNdkIg5ocL5pIZjnXB/uoyT4Y1ORoqZQintRp5/pwc0RNtsKgJkfLLBpOH1FzMZHshEFNjpYJZP+0xcR4UpFKaanKIpoTBjU5WqZfevqIGgBH1WQbDGpytGwjav/EBW4Z1GQPDGpytMyi4fRDyCffRmR1+V6Kq0ZEnhSR90Vkr4hcYXZhRIWQacOb3kc9+TYiq8vrUlwAfgDgt6r6KRHxASg3sSaigsk2ovZxRE02M2tQi0gQwLUA7gIAVY0BiJlbFlFhZMJ4eh81wMVEso98pj5WAAgD+DcReUdEHjSuSj6FiGwSkTYRaQuHwwUvlGg+so6o3RxRk73kE9QeAJcA+LGqXgxgFMC3pu+kqg+oaquqtoZCoQKXSTQ/0RmmPtj1QXaRT1B3AuhU1e3Gx08iHdxEljcR1G7OUZN9zRrUqtoD4KiInGt86gYA75laFVGBZOahA16255F95dv18VUAjxsdHx0AvmBeSUSFMzFHnbU9j0FN9pBXUKvqTgCtJtdCVHATfdRZj0xkHzXZA49MJEfL3vXhnnIbkdUxqMnRsvZRe3lSJrIXBjU5GvuoyQkY1ORo0UQKIoDHJROf42Ii2Q2DmhwtlkzB53ZB5PSg5oia7IJBTY4WS6SmzE8D6dG1SxjUZB8ManK0aCIFn8c95XMiwusmkq0wqMnRoonkaSNqIL2gGI2zj5rsgUFNjhZLpKZ0fGT4PG6OqMk2GNTkaNnmqIF0XzW7PsguGNTkaLFk9hG13+PiYiLZBoOaHC0aT005xWmGjyNqshEGNTlarhG1jyNqshEGNTnaTHPUDGqyCwY1OVrurg/2UZN9MKjJ0aKJ5GkHvABGHzXPR002kdeFA0TkMIBhAEkACVXlRQTIFmKJ3IuJnPogu8j3UlwA8BFVPW5aJUQmiCVTE+efnszvcTOoyTY49UGOFuWImhwg36BWAC+IyA4R2ZRtBxHZJCJtItIWDocLVyHRAkRzdH2wj5rsJN+gvlpVLwFwC4Avi8i103dQ1QdUtVVVW0OhUEGLJJoPVc3d9eHmiJrsI6+gVtVjxvs+AM8A2GBmUUSFEE8qAGTvo/a6EGV7HtnErEEtIhUiUpXZBvBRAO+aXRjRQmX6pLOe68MYUatqscsimrN8uj4aADxjXMrIA+BnqvpbU6siKoDM+aZzLSYCRldIlj5rIiuZNahVtQPAuiLUQlRQp0bUWQ54mXTdRAY1WR3b88ixMouF2c/14Z6yD5GVMajJsTIhnOtcHwB4vg+yBQY1OVZ0pqA25q2jcQY1WR+DmhxrxqDmiJpshEFNjjXzHLVryj5EVsagJsfKjJZzHUIOgIeRky0wqMmxTvVR527P4zmpyQ4Y1ORYEyPqHKc5BTj1QfbAoCbHmmjPy3JkIueoyU4Y1ORY7KMmp2BQk2Oxj5qcgkFNjjVje56XI2qyDwY1OdZMpznNjKg5R012wKAmx4rOsJjo42Ii2QiDmhwrmkjC53bBOJf6FOyjJjthUJNjxXJc2Bbg1AfZC4OaHCvXhW0BQETSVyLnYiLZQN5BLSJuEXlHRJ4zsyCiQpkpqIFT100ksrq5jKi/BmCvWYUQFVp0lqD2eVw8KRPZQl5BLSJLAXwMwIPmlkNUODPNUQPp/mqOqMkO8h1Rfx/ANwHk/K0WkU0i0iYibeFwuCDFES1ELDn7iJpBTXYwa1CLyG0A+lR1x0z7qeoDqtqqqq2hUKhgBRLNVyyRytpDncGgJrvIZ0R9FYDbReQwgJ8DuF5EfmpqVUQFEE0k85ijZh81Wd+sQa2qf6mqS1V1OYBPA3hZVT9nemVEC5Seoz79ogEZfo+b5/ogW2AfNTnWrF0fbM8jm/DMZWdVfQXAK6ZUQlRgs/VR+zwujI4lilgR0fxwRE2OFU2k4OdiIjkAg5ocK5ZMZb1eYgb7qMkuGNTkWPm05/HIRLIDBjU51mzteX4GNdkEg5oca9bFRLcLMfZRkw0wqMmREskUUoqZ+6i97KMme2BQkyPNdL3EjEwftaoWqyyieWFQkyNF47mvl5jh87iQUiCRYlCTtTGoyZHyGlHzArdkEwxqcqRM+M52PurJ+xJZFYOaHCnTdpfXiJoLimRxDGpypMzpS2caUWfmrzPz2URWxaAmR4rNaUTNXmqyNgY1OdKpOeqZz0cNgEcnkuUxqMmR8un64GIi2QWDmhwp3z5qgCNqsr58Lm4bEJE3RaRdRPaIyHeKURjRQrCPmpwknyu8RAFcr6ojIuIFsFVEnlfVbSbXRjRv7KMmJ5k1qDV9IoQR40Ov8cZjbsnSMu15M89RczGR7CGvOWoRcYvITgB9AF5U1e3mlkW0MMdHYgCAugp/zn1qK7wAgP7RaFFqIpqvvIJaVZOquh7AUgAbRGTt9H1EZJOItIlIWzgcLnSdRHPSMxhBdcCDMl/u9rz6Cj88LkH3YKSIlRHN3Zy6PlR1AMAWADdnue0BVW1V1dZQKFSo+ojmpWcogsZgYMZ9XC7B4io/ehnUZHH5dH2ERKTG2C4DcBOA980ujGgheociaKieOagBoCEYQM8Qg5qsLZ8RdROALSKyC8BbSM9RP2duWUQL0zMYQdMsI2oAaGJQkw3k0/WxC8DFRaiFqCASyRSOj0TRmM+IujqAP+zjmgpZG49MJMcJj0SR0vS0xmwaqwMYjSUxHIkXoTKi+WFQk+NkujjyGVFnFhx7uKBIFsagJsfJdHHktZho7MN5arIyBjU5TiZ0Z2vPA06NujmiJitjUJPj9AxF4HO7sKjcN+u+mTDv5YiaLIxBTY7TOxjB4mo/XC6Zdd+A142aci+nPsjSGNTkOD1DkbwWEjMaqwPoGeT5Psi6GNTkOD2Dkbxa8zIaqgPoGRo3sSKihWFQk6OoKkfU5DgManKUofEEIvFUXoePZzQGA+gfjSKe5HmpyZoY1OQomUXBfHqoMxqDAagCfcMcVZM1MajJUebSQ53BXmqyOgY1OUrPYHpRcC5z1A0MarI4BjU5SmZRcHF17ktwTTdxvg/2UpNFMajJUXqGIqir8E1cuDYfteVe+DwuHp1IlsWgJkfJ98ouk4mI0aLHoCZrYlCTo/QMzn6txGwaq3mlF7KufK6Z2CIiW0TkPRHZIyJfK0ZhRPMxnxE1kL7IAKc+yKryGVEnAHxDVdcAuBzAl0VkjbllEc1dNJFE/2hsTh0fGY3VfnQPRqCqJlRGtDCzBrWqdqvq28b2MIC9AJrNLoxorvqG0h0fjcH8Oz4yGqoDiCVSGBjjJbnIeuY0Ry0iy5G+0O32LLdtEpE2EWkLh3mxUCq+Uwe7lM35a5uMr+E8NVlR3kEtIpUAngLwdVUdmn67qj6gqq2q2hoKhQpZI1FeeuZwrcTpMqNwBjVZUV5BLSJepEP6cVV92tySiOanex5HJWZkRuFdAzzdKVlPPl0fAuAhAHtV9X7zSyKan93HhtBYHUCw3Dvnr22qDqAq4MG7x057sUhUcvmMqK8CcCeA60Vkp/F2q8l1Ec1Z+9EBrG+pmdfXulyCdUtr0H50oMBVES2cZ7YdVHUrgNkvPkdUQv0jURw5MYbPXrZs3vexvqUGP/7DQYzHkijz5X8IOpHZeGQiOUJ7Z3okPN8RdeZrkynF7mODhSqLqCAY1OQIO48MwCXAhc3Bed/H+mXpkN959GShyiIqCAY1OcI7RwdwTkMVKvyzzublVF/pR8uiMuzkPDVZDIOabC+VUrQfHcDFy+Y/7ZGxvqUWO48wqMlaGNRke4f6RzEUSSxofjpjfUsNugYj6OOBL2QhDGqyvcwIeH1L7YLvKxP273D6gyyEQU22t/PoACp8bqxaXLng+7pgSTW8buE8NVkKg5psb+fRAVy0tAZu18Lb/QNeN85vquY8NVkKg5psLRJPYm/30ERrXSGsb6nBrs4BJFM8NzVZA4OabG1P1yASKS3IQmLG+pYajMaSONA3UrD7JFoIBjXZ2jvGFMXFBQ5qgAe+kHUwqMnWfr+3DytDFVg8j1Ob5rKivgIN1X68tLevYPdJtBAMarKtnsEIth3qx+3rlhT0fkUEH79oCV7Z14dBXpqLLIBBTbb13K4uqKLgQQ0AG9c3I55UPP9ud8Hvm2iuGNRkW5vbu3BhcxArQwvvn55ubXM1VtRXYHN7V8Hvm2iuGNRkSx3hEezqHMTG9YUfTQPp6Y/b1y3BGx396OXh5FRiDGqypc3tXRABbrvInKAGgNvXL4Eq8CxH1VRi+Vwz8WER6RORd4tRENFsVBWb27tw2YpFaAwWrttjurNDlVjbXM2gppLLZ0T9CICbTa6DKG97uobQER7FxvXNpj/WxnXNaO8cxKHjo6Y/FlEuswa1qr4K4EQRaiHKy5M7OuF1C25Z22j6Y922rgkiwNNvd5r+WES5FGyOWkQ2iUibiLSFw+FC3S3RFH3DEfz8rSP4+EVLUFPuM/3xmoJluOn8Bjzy+mH2VFPJFCyoVfUBVW1V1dZQKFSouyWa4kdbDiKeVPzXG1YX7TH//KZzMBxJ4MGtHUV7TKLJ2PVBttE9OI6fbT+CT12yFMvrK4r2uOc3VeNjFzbh4a2HcGI0VrTHJcpgUJNt/PDlA1AovnrDqqI/9tdvXI2xeBI/efVg0R+bKJ/2vCcAvAHgXBHpFJEvml8W0VRHT4zh39uO4o5LW7C0trzoj7+6oQob1y3BY69/iPBwtOiPT2e2fLo+PqOqTarqVdWlqvpQMQojmuzvX9gHEcFXPlK8uenpvnbjOYglU/jB7/eXrAY6M3Hqgyzvt+9241c7u3DvtStNPcBlNivqK/D5K5bjp9uO4LUP2NlExcOgJkvrG47gL5/ejQubg0Xt9Mjlmzefi1WLK/HfftmOgTEuLFJxMKjJslQVf/HkLozFkviHO9bB6y79r2vA68b371iP/pEY/urXe0pdDp0hSv+bT5TDT7cfwZZ9Yfz3W8/HqsVVpS5nwtrmIP78pnPwbHsXfvXOsVKXQ2cABjVZ0qv7w/jO5j247pwQ7rz8rFKXc5p7r12JS5fX4i+e2oW2wzzDApmLQU2Ws6tzAPf9dAdWN1Thnz57MVwuKXVJp/G4XfjJna1orinD3Y+8hf29w6UuiRyMQU2Wcuj4KL7wb29hUYUPj37hUlQHvKUuKadFFT48evcGBLxufP7hN9E1MF7qksihGNRkGR/0DuNP/3UbFMBjd28o6JXFzdKyqByP3r0BI5EEPvuv2/BhP0+HSoXHoCZL2N7Rj0/++HXEU4rH7t5gynUQzXJ+UzUeuXsDBsbj+I8/eh3tRwdKXRI5DIOaSu7Z9i7c+dCbCFX58fSXrsTa5mCpS5qzPzqrFk996UqU+9349APb8NJ7vaUuiRyEQU0lE4kn8Ve/ehdffeIdXLQ0iKe+dCVaFhX/PB6FcnaoEk996UqsWlyJex5rw9/+Zi9iiVSpyyIHYFBTSezvHcbGH/4//N9tH+LPrlmBx//ssqJcCMBsi6sC+Pd7r8CfXrYMD7zagU/++HVexosWjEFNRRWJJ3H/C/tw2z9uRf9oFI984VJ8+2Nr4Pe4S11awZT53PibT1yIf/ncH+HIiTHc8oNX8c9bDnB0TfPmKXUBdGZQVbz8fh/++tk9OHpiHBvXL8H/+NgahKr8pS7NNDevbcS6liC+s/k9fO93+/DU25347u1rcfXq+lKXRjYjqlrwO21tbdW2traC3y/Z0/aOfvz9C/vx5uETWLW4Et/deAGuPPvMCqst+/rw15v34MP+MVy1qg7f+Oi5uGRZbanLIgsRkR2q2pr1NgY1mUFVsfXAcfzkDx3YeuA4Flf58dXrV+GOS5fB5zkzZ9wi8SQe334EP9pyAP2jMXzk3BDuve5sXLZiEUSsd/QlFdeCg1pEbgbwAwBuAA+q6v+eaX8G9ZlrPJbEs+1deGjrIezrHUZ9pR/3XbcSn7v8LAS8zpmHXojRaAKPvnEYD76Wvgbj2uZqfPHqFbhlbROfozPYgoJaRNwA9gO4CUAngLcAfEZV38v1NQzqM4uqYvexQfziraPYvLMLw9EEzmuswj3XrMTH1zU5aqGwkCLxJJ555xgefK0DB8OjCJZ58YmLm/EnrS1Ys6S61OVRkc0U1PksJm4AcEBVO4w7+zmAjQByBjWdGT7oHcazu7rxXHsXOo6PIuB14da1TfiTS1v4cj4PAa8bn9mwDHe0tuD1g/34RdtR/Gz7ETzy+mGc01CJ2y5agtsuarLVUZpkjnyCuhnA0UkfdwK4bPpOIrIJwCYAWLZsWUGKI2tJJFN4+8gAXtrbi5fe60XH8VGIAFesrMM916zEbeuaLH0SJatyuQRXr67H1avrMTAWw7PtXXi2vRv3v7gf97+4H6sXV+LGNQ248fwGrG+pgduCZxMkc+Uz9fEpADer6j3Gx3cCuExVv5Lrazj14QyqioPhUbzR0Y/X9ofxxsF+DEcT8LoFl6+sw01rGnDzBY22OHmSHXUPjuP53T14aW8vth86gWRKESzz4qpVdbh6VQhXnF2H5XXlfOXiEAud+jgGoGXSx0uNz5HDxBIpvNc9hHeOnETb4ZPYfugEjo9EAQDNNWW4bV0TrlkdwjWr61HFkbPpmoJluPvqFbj76hUYHIvjDx+E8dr+MLYeOI7f7O4BADRU+7FhRR0uXV6Li1tqcV5TlSUuWUaFlc+I2oP0YuINSAf0WwA+q6o5LxjHEbX1xZMpHOgbwbvHBvHusUHsOjaIPV1DE0fPLQkGcNnKOmxYsQiXr+TIzUoyr3S2dfTjzUMnsP1QP3qH0v9QA14X1i4JYm1zEBctTb9fWV8BD8Pb8grRnncrgO8j3Z73sKr+zUz7M6itI5VSHBsYxwd9w9jfO4L9vcN4v3sYB/pGEEumQ7nC58YFS9J/2JecVYuLl9WgKVhW4sopX6rpn/E7Rwbw9pGT2N2Z/qc7Hk8CAHweF85pqMR5jdU4p6ESqxuqsHpxJZYEyyx59ZwzFQ94cThVRXgkiiP9YzjcP4bDx0dxqH8UHeFRdIRHEJ10jomGaj/ObazG+U1VWNNUjQuWVGNFfSUXqBwmmVIcDI9gT9cg9nYP472uIezrHUZ4ODqxT5nXjZWhCqwMVWJFXTmW11fgrLoKLFtUjvpKH19BFRmD2uYSyRT6hqPoHhzHsYEIugfG0XlyHJ0nx9B5chxHT44hEj8Vxm6XoKW2DCtDlTjb+ENcvbgSqxdXIVjOueUz2cnRGPb3DuNgeBQHwyM4GB5BR3gUnSfHkJoUBWVeN1oWlaGlthxLa8vQXFuGJTVlaAqWYUlNAKFKP6dTCmyhi4lkkmgiif6RGI6PRBEePvXWOxxB71AUfUMR9AxFEB6OTvkjAoBgmRdLa8uwor4C150TwrK6crQsKsfyugosrS3jghJlVVvhw2Ur63DZyropn48lUjh6cgwf9o/iSP8YjpwYx5ETYzg2MI43D53AcDQxZX+XpE/p2hAMoLHan9423oeq/AhV+VFX6UNdhf+MPWVAITGoCySRTGFwPI7B8ThOjsUxMBbDwFgcJ8diODEaw8mxGPpH0tsnRtPhPBRJZL2v2nIvFlcFsLjaj3Mbq9BYHUBjsAxNNQE015ShKRhg1wUVlM/jwtmhSpyd4+CawfE4ugfH0T0QwbGBcfQORdA9GEHPYASHjo9i+6ETGBiLZ/3aYJnXCG0fFlX4sKjCj0UVXtSW+9JvFV7UGNvBMi+qAx6O1qdhUCMdsqOxJEajCYxk3iLp98OROIYjCQxFJm2PxzEUiWNwPL09OB7HSDR76AKAxyWorfChttyLugo/1iypRl2FD/WVftRX+VFf6Z8YhdRX+njINVlOsMyLYJkX5zXmPrQ9Ek9OvDrsG45OvFo8PhJF/2gM/SNRdIRHsePDAZwciyE5/WXiJFV+D6qNxwyWeVFd5kF1wIuqgBdVAQ+qAumPK43tSn/6rcLvQWXAgwqfx1HrLpYPalVFIqWIxJOIxFOIxJOIJk5tR+IpjMeTGI8nEYklJ7bHYkmMxxLG+yRGJ22PRNPbo9EERmOJKfO7M6n0exAsO/WL0lwTwPlNVQiWeVFT5kOwzIOach+C5ZnRQnqkUB3wcGGGHC/gdWNpbTmW1s5+OTVVxdB4AifH0q82B8biGBg33o+lBz9DkfjEQOjD/jFjgJSYcVA0tR4XKv0elPvSAV7hc6PM50aFz4NyYzv93vjYm34LGNvlPjcCXhf8HjcC3vR2wOuG35N+X8zpRUsF9W3/9BqGIwlE4ylEE0lEE+kwnuEf74wyT/bED8ef/ri2ohwVPjfKM/+FfR5U+N3p/8qB9A+1yu9B1eT/2D4PW5mICkREECz3IljuxXJUzOlrkymdeLWbefU7HJn6ang0ljBeIacHbJlXzMORBHqHIhiNJhHJDOiMNsa5crtkIrT9Hhf8Hlf6Umz3XTGv+5uJpYJ69eIqqCr8Hjd8xjc++T9Y5r+b3+tK/+cz3tL/+dK3l/s8xm0ujmKJHMjtkokpkUJIpRTRRApjxqvuaCKJ8Vj6lXok82o9nkQ0nkIkMXU78z6WSCGaSKHMpNPUWiqo/+GO9aUugYjOMC6XoMx45V03++4lwaVVIiKLY1ATEVkcg5qIyOIY1EREFsegJiKyOAY1EZHFMaiJiCyOQU1EZHGmnI9aRMIAPpznl9cDOF7AcgqFdc0N65ob1jU3TqzrLFUNZbvBlKBeCBFpy3Xy7FJiXXPDuuaGdc3NmVYXpz6IiCyOQU1EZHFWDOoHSl1ADqxrbljX3LCuuTmj6rLcHDUREU1lxRE1ERFNwqAmIrK4kge1iHxPRN4XkV0i8oyI1OTY72YR2SciB0TkW0Wo6z+JyB4RSYlIznYbETksIrtFZKeItFmormI/X4tE5EUR+cB4X5tjv6TxXO0Ukc0m1jPj9y8ifhH5hXH7dhFZblYtc6zrLhEJT3qO7ilCTQ+LSJ+IvJvjdhGRfzRq3iUil5hdU551/bGIDE56rv5nkepqEZEtIvKe8bf4tSz7FPY5U9WSvgH4KACPsf13AP4uyz5uAAcBrATgA9AOYI3JdZ0P4FwArwBonWG/wwDqi/h8zVpXiZ6v/wPgW8b2t7L9HI3bRorwHM36/QP4LwD+xdj+NIBfWKSuuwD8sFi/T8ZjXgvgEgDv5rj9VgDPAxAAlwPYbpG6/hjAc8V8rozHbQJwibFdBWB/lp9jQZ+zko+oVfUFVc1cVngbgKVZdtsA4ICqdqhqDMDPAWw0ua69qrrPzMeYjzzrKvrzZdz/o8b2owD+g8mPN5N8vv/J9T4J4AYx/yKbpfi5zEpVXwVwYoZdNgJ4TNO2AagRkSYL1FUSqtqtqm8b28MA9gJonrZbQZ+zkgf1NHcj/V9oumYARyd93InTn5hSUQAviMgOEdlU6mIMpXi+GlS129juAdCQY7+AiLSJyDYRMSvM8/n+J/YxBgqDgOmXzMv35/JJ4+XykyLSYnJN+bDy398VItIuIs+LyAXFfnBjyuxiANun3VTQ56woF7cVkZcANGa56duq+mtjn28DSAB4vBg15VtXHq5W1WMishjAiyLyvjESKHVdBTdTXZM/UFUVkVx9n2cZz9dKAC+LyG5VPVjoWm3sWQBPqGpURO5FetR/fYlrsqq3kf59GhGRWwH8CsDqYj24iFQCeArA11V1yMzHKkpQq+qNM90uIncBuA3ADWpM8ExzDMDkkcVS43Om1pXnfRwz3veJyDNIv7xdUFAXoK6iP18i0isiTarabbzE68txH5nnq0NEXkF6NFLooM7n+8/s0ykiHgBBAP0FrmPOdanq5BoeRHruv9RM+X1aqMnhqKq/EZEfiUi9qpp+siYR8SId0o+r6tNZdinoc1byqQ8RuRnANwHcrqpjOXZ7C8BqEVkhIj6kF39M6xjIl4hUiEhVZhvphdGsK9RFVornazOAzxvbnx27fEwAAAE7SURBVAdw2shfRGpFxG9s1wO4CsB7JtSSz/c/ud5PAXg5xyChqHVNm8e8Hen5z1LbDOA/G50MlwMYnDTNVTIi0phZVxCRDUjnmdn/bGE85kMA9qrq/Tl2K+xzVuwV0ywrqAeQnsvZabxlVuKXAPjNtFXU/UiPvr5dhLo+gfS8UhRAL4DfTa8L6dX7duNtj1XqKtHzVQfg9wA+APASgEXG51sBPGhsXwlgt/F87QbwRRPrOe37B/BdpAcEABAA8Evj9+9NACvNfo7yrOt/Gb9L7QC2ADivCDU9AaAbQNz43foigPsA3GfcLgD+2ah5N2bogipyXV+Z9FxtA3Blkeq6Gum1qV2TcutWM58zHkJORGRxJZ/6ICKimTGoiYgsjkFNRGRxDGoiIotjUBMRWRyDmojI4hjUREQW9/8BONFa/NgvuysAAAAASUVORK5CYII=\n", 133 | "text/plain": [ 134 | "
" 135 | ] 136 | }, 137 | "metadata": { 138 | "needs_background": "light" 139 | }, 140 | "output_type": "display_data" 141 | } 142 | ], 143 | "source": [ 144 | "grad_f = grad(f)\n", 145 | "\n", 146 | "x = np.linspace(-2, 2, 100)\n", 147 | "plt.plot(x, [grad_f(x_) for x_ in x])" 148 | ] 149 | }, 150 | { 151 | "cell_type": "markdown", 152 | "metadata": {}, 153 | "source": [ 154 | "To use this in our optimization routines:\n", 155 | "\n", 156 | " - we can automatically compute gradients that the optimizer can use. \n", 157 | " - Hessians can be exactly calculated (we will do this later)\n" 158 | ] 159 | }, 160 | { 161 | "cell_type": "code", 162 | "execution_count": 4, 163 | "metadata": {}, 164 | "outputs": [], 165 | "source": [ 166 | "T = (np.random.exponential(size=1000)/1.5) ** 2.3\n", 167 | "E = np.random.binomial(1, 0.95, size=1000)" 168 | ] 169 | }, 170 | { 171 | "cell_type": "code", 172 | "execution_count": 5, 173 | "metadata": {}, 174 | "outputs": [], 175 | "source": [ 176 | "def cumulative_hazard(params, t):\n", 177 | " lambda_, rho_ = params\n", 178 | " return (t / lambda_) ** rho_\n", 179 | "\n", 180 | "def log_hazard(params, t):\n", 181 | " lambda_, rho_ = params\n", 182 | " return np.log(rho_) - np.log(lambda_) + (rho_ - 1) * (np.log(t) - np.log(lambda_))\n", 183 | "\n", 184 | "def log_likelihood(params, t, e):\n", 185 | " return np.sum(e * log_hazard(params, t)) - np.sum(cumulative_hazard(params, t))\n", 186 | "\n", 187 | "def negative_log_likelihood(params, t, e):\n", 188 | " return -log_likelihood(params, t, e)\n" 189 | ] 190 | }, 191 | { 192 | "cell_type": "markdown", 193 | "metadata": {}, 194 | "source": [ 195 | "So `grad(negative_log_likelihood)` will find the gradient of `negative_log_likelihood` with respect to the first parameter, `params`. " 196 | ] 197 | }, 198 | { 199 | "cell_type": "code", 200 | "execution_count": 6, 201 | "metadata": {}, 202 | "outputs": [ 203 | { 204 | "name": "stdout", 205 | "output_type": "stream", 206 | "text": [ 207 | "[-199.44295882 2906.3619735 ]\n" 208 | ] 209 | } 210 | ], 211 | "source": [ 212 | "grad_negative_log_likelihood = grad(negative_log_likelihood)\n", 213 | "\n", 214 | "print(grad_negative_log_likelihood(np.array([1., 1.]), T, E))" 215 | ] 216 | }, 217 | { 218 | "cell_type": "code", 219 | "execution_count": 7, 220 | "metadata": {}, 221 | "outputs": [ 222 | { 223 | "name": "stdout", 224 | "output_type": "stream", 225 | "text": [ 226 | " fun: 243.1558229286153\n", 227 | " hess_inv: <2x2 LbfgsInvHessProduct with dtype=float64>\n", 228 | " jac: array([ 0.00101051, -0.00293405])\n", 229 | " message: b'CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH'\n", 230 | " nfev: 11\n", 231 | " nit: 7\n", 232 | " status: 0\n", 233 | " success: True\n", 234 | " x: array([0.4693 , 0.4273751])\n" 235 | ] 236 | } 237 | ], 238 | "source": [ 239 | "from scipy.optimize import minimize\n", 240 | "\n", 241 | "results = minimize(negative_log_likelihood, \n", 242 | " x0 = np.array([1.0, 1.0]),\n", 243 | " method=None, \n", 244 | " args=(T, E),\n", 245 | " jac=grad_negative_log_likelihood,\n", 246 | " bounds=((0.00001, None), (0.00001, None)))\n", 247 | "\n", 248 | "print(results)" 249 | ] 250 | }, 251 | { 252 | "cell_type": "code", 253 | "execution_count": 8, 254 | "metadata": {}, 255 | "outputs": [], 256 | "source": [ 257 | "from autograd import value_and_grad\n", 258 | "\n", 259 | "results = minimize(\n", 260 | " value_and_grad(negative_log_likelihood), \n", 261 | " x0 = np.array([1.0, 1.0]),\n", 262 | " method=None, \n", 263 | " args=(T, E),\n", 264 | " jac=True, # notice this set to True now.\n", 265 | " bounds=((0.00001, None), (0.00001, None)))\n" 266 | ] 267 | }, 268 | { 269 | "cell_type": "code", 270 | "execution_count": 9, 271 | "metadata": {}, 272 | "outputs": [ 273 | { 274 | "data": { 275 | "text/plain": [ 276 | " fun: 243.1558229286153\n", 277 | " hess_inv: <2x2 LbfgsInvHessProduct with dtype=float64>\n", 278 | " jac: array([ 0.00101051, -0.00293405])\n", 279 | " message: b'CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH'\n", 280 | " nfev: 11\n", 281 | " nit: 7\n", 282 | " status: 0\n", 283 | " success: True\n", 284 | " x: array([0.4693 , 0.4273751])" 285 | ] 286 | }, 287 | "execution_count": 9, 288 | "metadata": {}, 289 | "output_type": "execute_result" 290 | } 291 | ], 292 | "source": [ 293 | "results" 294 | ] 295 | }, 296 | { 297 | "cell_type": "markdown", 298 | "metadata": {}, 299 | "source": [ 300 | "Let's continue this analytical-train 🚂 to Part 4! " 301 | ] 302 | } 303 | ], 304 | "metadata": { 305 | "kernelspec": { 306 | "display_name": "Python 3", 307 | "language": "python", 308 | "name": "python3" 309 | }, 310 | "language_info": { 311 | "codemirror_mode": { 312 | "name": "ipython", 313 | "version": 3 314 | }, 315 | "file_extension": ".py", 316 | "mimetype": "text/x-python", 317 | "name": "python", 318 | "nbconvert_exporter": "python", 319 | "pygments_lexer": "ipython3", 320 | "version": "3.7.3" 321 | } 322 | }, 323 | "nbformat": 4, 324 | "nbformat_minor": 2 325 | } 326 | --------------------------------------------------------------------------------