├── .gitignore ├── LICENSE.txt ├── README.md ├── cheatcodes.py ├── environment.yml ├── gravity-inversion-solution.ipynb ├── gravity-inversion.ipynb ├── images ├── Half-graben_sedimentation.png ├── README.md ├── basin-example1.png ├── basin-example2.png ├── gravity-disturbance.png ├── gravity-earth.png └── gravity-normal-earth.png ├── notes.pdf └── notes.xopp /.gitignore: -------------------------------------------------------------------------------- 1 | .ipynb_checkpoints 2 | *~ 3 | -------------------------------------------------------------------------------- /LICENSE.txt: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2020 Leonardo Uieda 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Practical non-linear gravity inversion 2 | 3 | **Author:** [Leonardo Uieda](https://www.leouieda.com/)1 4 | 5 | > 1 Department of Earth, Ocean and Ecological Sciences, 6 | > School of Environmental Sciences, University of Liverpool, UK 7 | 8 | ## About 9 | 10 | Inverse problems abound in geophysics. 11 | It is the primary way in which we investigate the subsurface of the Earth, 12 | which is largely inaccessible to us beyond the first dozen or so kilometers. 13 | From measurements acquired on land, sea, air, and from space, geophysicists 14 | tease out the inner structure of the Earth - from a few meters to thousands of 15 | kilometers deep in the inner core. 16 | Observations of disturbances in the Earth's gravity field are one of the key 17 | elements used by geophysicists to investigate the crust-mantle interface, the 18 | large-scale structure of sedimentary basins (which are reservoirs for water and 19 | hydrocarbons), and even the mass balance of the world's ice sheets. 20 | However, the gravity inverse problem is particularly challenging due to the 21 | physics of potential fields. 22 | Unique solutions are difficult to come by and only exist under strict 23 | assumptions, which often don't hold for real world scenarios. 24 | For these problems, regularization plays a critical role and has been the focus 25 | of much research in the past 20 years. 26 | 27 | In this tutorial, we will work together to solve a 2D gravity inverse problem 28 | in Python. 29 | Our code will estimate the shape of a sedimentary basin from gravity 30 | observations. 31 | This non-linear inverse problem will allow us to visually explore the effects 32 | of different types of regularization from a geometric perspective (smoothness, 33 | equality constraints, and more). 34 | We will discuss the challenges involved in real world applications and the 35 | difficulties of quantifying the uncertainty in the solutions. 36 | The main goal of this tutorial is to impart theoretical and practical 37 | skills that can be easily transferred to other domains. 38 | 39 | ## Learning objectives 40 | 41 | This course is designed to empower you to: 42 | 43 | * Learn/revise the mathematics of non-linear inverse problems 44 | * Translate mathematical knowledge into code 45 | * Apply non-linear inversion theory to a real geophysical problem 46 | * Analyze the effects of regularization on geophysical models 47 | 48 | ## Prerequisites 49 | 50 | I will assume that you: 51 | 52 | * Are comfortable with linear algebra (matrix and vector operations, 53 | norms, inverses, linear systems, etc) 54 | * Have an understanding of basic calculus (partial derivatives, gradients, 55 | Taylor series expansions) 56 | * Are able to program a computer to build and manipulate matrices and 57 | vectors, solve linear systems, and make graphs/plots (in any language 58 | but Python or Matlab would be best) 59 | 60 | ## Running the code 61 | 62 | You can run and experiment with the code for this tutorial 63 | on Binder (click on the badge): 64 | [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/GeophysicsLibrary/non-linear-gravity-inversion/HEAD) 65 | 66 | **WARNING**: Binder sessions won't save your progress and may 67 | disconnect at any point. Download and backup your notebook 68 | frequently if you want to keep your changes. 69 | Alternatively, see below to setup your own computer and run 70 | the code locally. 71 | 72 | ## Computer setup 73 | 74 | Since there is large component of live coding, participants will 75 | have to set up their computers **before the workshop**. It's 76 | extremely important that everyone has a working Python environment 77 | ahead of time as there will not be enough time to sort out 78 | individual problems during the workshop. 79 | 80 | 1. **Download and install the Anaconda Python Distribution**. 81 | Please follow the instructions here: 82 | https://carpentries.github.io/workshop-template/#python 83 | 1. Make sure your installation works by opening JupyterLab through 84 | the Anaconda Navigator app (on Windows) or by running 85 | `jupyter lab` in a terminal (Mac/Linux). You browser should 86 | open with JupyterLab. 87 | 1. Download a [zip archive of this repository](https://github.com/GeophysicsLibrary/non-linear-gravity-inversion/archive/refs/heads/main.zip) 88 | and unzip it. 89 | 1. In JupyterLab, navigate to the place where you unzipped the archive 90 | and open the `gravity-inversion.ipynb` notebook. 91 | 92 | We will be using [Jupyter Notebooks](https://jupyter.org/) to run our 93 | Python code and the libraries [numpy](https://numpy.org/), 94 | [scipy](https://www.scipy.org/), and [matplotlib](https://matplotlib.org/). 95 | Anaconda already comes with all of these installed. 96 | 97 | The forward modelling and some plotting utilities are in the 98 | `cheatcodes.py` file. It's **very important that this file is in the 99 | same folder as the notebooks!** 100 | 101 | ## Workshop material 102 | 103 | All of the code and notes for this workshop are (or will be) uploaded 104 | to this repository. In here, you'll find: 105 | 106 | * `cheatcodes.py`: The ready-made Python functions for forward modelling and 107 | plotting. 108 | * `gravity-inversion.ipynb`: Jupyter notebook with the code that I wrote live 109 | in the workshop (not including solutions to exercises). 110 | * `gravity-inversion-solution.ipynb`: Same as the above but with the exercise 111 | solutions. 112 | * `notes.pdf`: Notes and mathematical derivations. 113 | 114 | You may also want to brush up on your coding skills with 115 | [Software Carpentry's Introduction to Python](https://swcarpentry.github.io/python-novice-inflammation/) 116 | lesson. 117 | 118 | ## License 119 | 120 | All Python source code in this repository is free software: you can 121 | redistribute it and/or modify it under the terms of the MIT License. A copy of 122 | this license is provided in [LICENSE.txt](LICENSE.txt). 123 | 124 | All other materials, including text and images, are distributed under the 125 | [CC-BY 4.0](https://creativecommons.org/licenses/by/4.0/) 126 | license (except where otherwise noted). 127 | 128 | Originally developed for a 129 | [short course on geophysical inversion at RWTH Aachen University graduate school IRTG-2379 Modern Inverse Problems](https://github.com/compgeolab/2020-aachen-inverse-problems). 130 | -------------------------------------------------------------------------------- /cheatcodes.py: -------------------------------------------------------------------------------- 1 | """ 2 | Code for cheating in the inversion programming. 3 | 4 | Includes functions for: 5 | 6 | * forward modelling with prisms 7 | * generating a synthetic model 8 | * plotting the models and solutions 9 | 10 | """ 11 | import numpy as np 12 | import matplotlib.pyplot as plt 13 | 14 | 15 | def plot_prisms( 16 | depths, 17 | basin_boundaries, 18 | ax=None, 19 | color="#00000000", 20 | edgecolor="black", 21 | linewidth=1, 22 | label=None, 23 | figsize=(9, 3), 24 | ): 25 | """ 26 | Plot the prism model using matplotlib. 27 | """ 28 | # Create lines with the outline of the prisms 29 | boundaries = np.linspace(*basin_boundaries, depths.size + 1) 30 | x = [boundaries[0]] 31 | y = [0] 32 | for i in range(depths.size): 33 | x.extend([boundaries[i], boundaries[i + 1]]) 34 | y.extend([depths[i], depths[i]]) 35 | x.append(boundaries[-1]) 36 | y.append(0) 37 | x = np.array(x) / 1000 38 | y = np.array(y) / 1000 39 | # Plot the outline with optional filling 40 | if ax is None: 41 | fig = plt.figure(figsize=figsize) 42 | ax = plt.subplot(111) 43 | ax.set_xlabel("x [km]") 44 | ax.set_ylabel("depth [km]") 45 | ax.fill_between( 46 | x, y, color=color, edgecolor=edgecolor, linewidth=linewidth, label=label 47 | ) 48 | ax.set_ylim(max(y) * 1.05, 0) 49 | return ax 50 | 51 | 52 | def gaussian(x, shift, std, amplitude): 53 | """ 54 | A simple Gaussian function we'll use to make a model 55 | """ 56 | return amplitude * np.exp(-(((x - shift) / std) ** 2)) 57 | 58 | 59 | def synthetic_model(): 60 | """ 61 | Generate a synthetic model using Gaussian functions 62 | """ 63 | size = 100 64 | basin_boundaries = (0, 100e3) 65 | boundaries = np.linspace(*basin_boundaries, size + 1) 66 | x = boundaries[:-1] + 0.5 * (boundaries[1] - boundaries[0]) 67 | depths = gaussian(x, shift=45e3, std=20e3, amplitude=5e3) + gaussian( 68 | x, shift=80e3, std=10e3, amplitude=1e3 69 | ) 70 | # Make sure the boundaries are at zero to avoid steps 71 | depths -= depths.min() 72 | return depths, basin_boundaries 73 | 74 | 75 | def prism_boundaries(depths, basin_boundaries): 76 | """ 77 | Calculate the x coordinate of the boundaries of all prisms. 78 | Will have depths.size + 1 elements. 79 | """ 80 | boundaries = np.linspace(*basin_boundaries, depths.size + 1) 81 | return boundaries 82 | 83 | 84 | def forward_model(depths, basin_boundaries, density, x): 85 | """ 86 | Calculate the predicted gravity for a given basin at x locations 87 | """ 88 | easting = x 89 | boundaries = prism_boundaries(depths, basin_boundaries) 90 | result = np.zeros_like(x) 91 | for m in range(depths.size): 92 | prism_gravity( 93 | x, boundaries[m], boundaries[m + 1], 0, depths[m], density, output=result 94 | ) 95 | return result 96 | 97 | 98 | def prism_gravity(x, east, west, top, bottom, density, output=None): 99 | """ 100 | Calculate the gravity of a single prism. 101 | Append the result to output if it's given. 102 | """ 103 | prism = [east, west, -200e3, 200e3, -bottom, -top] 104 | si2mgal = 1e5 105 | # The gravitational constant in SI units 106 | GRAVITATIONAL_CONST = 0.00000000006673 107 | scale = GRAVITATIONAL_CONST * density * si2mgal 108 | northing = 0 109 | upward = 10 110 | if output is None: 111 | output = np.zeros_like(x) 112 | # Iterate over the prism boundaries to compute the result of the 113 | # integration (see Nagy et al., 2000) 114 | for i in range(2): 115 | for j in range(2): 116 | for k in range(2): 117 | shift_east = prism[1 - i] 118 | shift_north = prism[3 - j] 119 | shift_upward = prism[5 - k] 120 | output += ( 121 | (-1) ** (i + j + k) 122 | * scale 123 | * kernel( 124 | shift_east - x, 125 | shift_north - northing, 126 | shift_upward - upward, 127 | ) 128 | ) 129 | return output 130 | 131 | 132 | def kernel(easting, northing, upward): 133 | """ 134 | The kernel function for calculating the vertical component of gravity 135 | """ 136 | radius = np.sqrt(easting ** 2 + northing ** 2 + upward ** 2) 137 | result = ( 138 | easting * np.log(northing + radius) 139 | + northing * np.log(easting + radius) 140 | + upward * np.arctan2(easting * northing, -upward * radius) 141 | ) 142 | return result 143 | -------------------------------------------------------------------------------- /environment.yml: -------------------------------------------------------------------------------- 1 | name: non-linear-gravity-inversion 2 | channels: 3 | - conda-forge 4 | - defaults 5 | dependencies: 6 | - python==3.9 7 | - numpy 8 | - matplotlib 9 | - jupyter 10 | - jupyterlab 11 | -------------------------------------------------------------------------------- /gravity-inversion.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Building a non-linear gravity inversion from scratch (almost)\n", 8 | "\n", 9 | "In this notebook, we'll build a non-linear gravity inversion to estimate the relief of a sedimentary basin. We'll implement smoothness regularization and see its effects on the solution. We'll also see how we can break the inversion by adding random noise, abusing regularization, and breaking the underlying assumptions.\n", 10 | "\n", 11 | "This notebook is part of the Geophysics Library lesson: [GeophysicsLibrary/non-linear-gravity-inversion](https://github.com/GeophysicsLibrary/non-linear-gravity-inversion) \n", 12 | "\n", 13 | "See the lesson material for extra code and setup instructions." 14 | ] 15 | }, 16 | { 17 | "cell_type": "markdown", 18 | "metadata": {}, 19 | "source": [ 20 | "## Imports\n", 21 | "\n", 22 | "We'll use the basic scientific Python stack for this tutorial plus a custom module with the forward modelling function (based on the code from the [Harmonica](https://github.com/fatiando/harmonica) library)." 23 | ] 24 | }, 25 | { 26 | "cell_type": "code", 27 | "execution_count": null, 28 | "metadata": {}, 29 | "outputs": [], 30 | "source": [ 31 | "import numpy as np\n", 32 | "import matplotlib.pyplot as plt\n", 33 | "# Our custom code (cheatcodes.py) with forward modelling and some utilities.\n", 34 | "import cheatcodes as cc " 35 | ] 36 | }, 37 | { 38 | "cell_type": "markdown", 39 | "metadata": {}, 40 | "source": [ 41 | "This is a little trick to make the resolution of the matplotlib figures better for larger screens." 42 | ] 43 | }, 44 | { 45 | "cell_type": "code", 46 | "execution_count": null, 47 | "metadata": {}, 48 | "outputs": [], 49 | "source": [ 50 | "plt.rc(\"figure\", dpi=120)" 51 | ] 52 | }, 53 | { 54 | "cell_type": "markdown", 55 | "metadata": {}, 56 | "source": [ 57 | "## Assumptions\n", 58 | "\n", 59 | "Here are some assumptions we'll work with:\n", 60 | "\n", 61 | "1. The basin is much larger in the y-dimension so we'll assume it's infinite (reducing the problem to 2D)\n", 62 | "1. The gravity disturbance is entirely due to the sedimentary basin\n", 63 | "1. The top of the basin is a flat surface at $z=0$\n", 64 | "1. The data are measured at a constant height of $z=1\\ m$" 65 | ] 66 | }, 67 | { 68 | "cell_type": "markdown", 69 | "metadata": {}, 70 | "source": [ 71 | "## Making synthetic data\n", 72 | "\n", 73 | "First, we'll explore the forward modelling function and create some synthetic data." 74 | ] 75 | }, 76 | { 77 | "cell_type": "code", 78 | "execution_count": null, 79 | "metadata": {}, 80 | "outputs": [], 81 | "source": [ 82 | "depths, basin_boundaries = cc.synthetic_model()\n", 83 | "\n", 84 | "print(basin_boundaries)\n", 85 | "print(depths)" 86 | ] 87 | }, 88 | { 89 | "cell_type": "markdown", 90 | "metadata": {}, 91 | "source": [ 92 | "Plot the model." 93 | ] 94 | }, 95 | { 96 | "cell_type": "code", 97 | "execution_count": null, 98 | "metadata": {}, 99 | "outputs": [], 100 | "source": [ 101 | "cc.plot_prisms(depths, basin_boundaries)" 102 | ] 103 | }, 104 | { 105 | "cell_type": "markdown", 106 | "metadata": {}, 107 | "source": [ 108 | "Forward model some gravity data at a set of locations." 109 | ] 110 | }, 111 | { 112 | "cell_type": "code", 113 | "execution_count": null, 114 | "metadata": {}, 115 | "outputs": [], 116 | "source": [ 117 | "x = np.linspace(-5e3, 105e3, 60)\n", 118 | "density = -300 # kg/m³\n", 119 | "data = cc.forward_model(depths, basin_boundaries, density, x)\n", 120 | "\n", 121 | "plt.figure(figsize=(9, 3))\n", 122 | "plt.plot(x / 1000, data, \".k\")\n", 123 | "plt.xlabel(\"x [km]\")\n", 124 | "plt.ylabel(\"gravity disturbance [mGal]\")\n", 125 | "plt.show()" 126 | ] 127 | }, 128 | { 129 | "cell_type": "markdown", 130 | "metadata": {}, 131 | "source": [ 132 | "## Calculating the Jacobian matrix\n", 133 | "\n", 134 | "The first step to most inverse problems is being able to calculate the Jacobian matrix. We'll do this for our problem by a first-order finite differences approximation. If you can get analytical derivatives, that's usually a lot better." 135 | ] 136 | }, 137 | { 138 | "cell_type": "code", 139 | "execution_count": null, 140 | "metadata": {}, 141 | "outputs": [], 142 | "source": [ 143 | "def make_jacobian(parameters, basin_boundaries, density, x):\n", 144 | " \"\"\"\n", 145 | " Calculate the Jacobian matrix by finite differences.\n", 146 | " \"\"\"\n", 147 | " jacobian = np.empty((x.size, parameters.size))\n", 148 | " step = np.zeros_like(parameters)\n", 149 | " delta = 10\n", 150 | " for j in range(jacobian.shape[1]):\n", 151 | " step[j] += delta\n", 152 | " jacobian[:, j] = (\n", 153 | " (\n", 154 | " cc.forward_model(parameters + step, basin_boundaries, density, x)\n", 155 | " - cc.forward_model(parameters, basin_boundaries, density, x)\n", 156 | " ) \n", 157 | " / delta\n", 158 | " )\n", 159 | " step[j] = 0\n", 160 | " return jacobian" 161 | ] 162 | }, 163 | { 164 | "cell_type": "markdown", 165 | "metadata": {}, 166 | "source": [ 167 | "Calculate and plot an example so we can see what this matrix looks like. We'll use a parameter vector with constant depths at first." 168 | ] 169 | }, 170 | { 171 | "cell_type": "code", 172 | "execution_count": null, 173 | "metadata": {}, 174 | "outputs": [], 175 | "source": [ 176 | "parameters = np.zeros(30) + 5000\n", 177 | "\n", 178 | "jacobian = make_jacobian(parameters, basin_boundaries, density, x)\n", 179 | "\n", 180 | "plt.figure()\n", 181 | "plt.imshow(jacobian)\n", 182 | "plt.colorbar(label=\"mGal/m\")\n", 183 | "plt.xlabel(\"columns\")\n", 184 | "plt.ylabel(\"rows\")\n", 185 | "plt.show()" 186 | ] 187 | }, 188 | { 189 | "cell_type": "markdown", 190 | "metadata": {}, 191 | "source": [ 192 | "## Solve the inverse problem \n", 193 | "\n", 194 | "Now that we have a way of forward modelling and calculating the Jacobian matrix, we can implement the Gauss-Newton method for solving the non-linear inverse problem. The function below takes the input data, model configuration, and an initial estimate and outputs the estimated parameters and a list with the goal function value per iteration. " 195 | ] 196 | }, 197 | { 198 | "cell_type": "code", 199 | "execution_count": null, 200 | "metadata": {}, 201 | "outputs": [], 202 | "source": [ 203 | "def basin2d_inversion(x, data, basin_boundaries, density, initial, max_iterations=10):\n", 204 | " \"\"\"\n", 205 | " Solve the inverse problem using the Gauss-Newton method.\n", 206 | " \"\"\"\n", 207 | " parameters = initial.astype(np.float64).copy() \n", 208 | " predicted = cc.forward_model(parameters, basin_boundaries, density, x)\n", 209 | " residuals = data - predicted\n", 210 | " goal_function = [np.linalg.norm(residuals)**2]\n", 211 | " for i in range(max_iterations): \n", 212 | " jacobian = make_jacobian(parameters, basin_boundaries, density, x)\n", 213 | " hessian = jacobian.T @ jacobian\n", 214 | " gradient = jacobian.T @ residuals\n", 215 | " deltap = np.linalg.solve(hessian, gradient)\n", 216 | " new_parameters = parameters + deltap\n", 217 | " predicted = cc.forward_model(new_parameters, basin_boundaries, density, x)\n", 218 | " residuals = data - predicted\n", 219 | " current_goal = np.linalg.norm(residuals)**2\n", 220 | " if current_goal > goal_function[-1]:\n", 221 | " break\n", 222 | " parameters = new_parameters\n", 223 | " goal_function.append(current_goal)\n", 224 | " return parameters, goal_function" 225 | ] 226 | }, 227 | { 228 | "cell_type": "markdown", 229 | "metadata": {}, 230 | "source": [ 231 | "Now we can use this function to invert our synthetic data." 232 | ] 233 | }, 234 | { 235 | "cell_type": "code", 236 | "execution_count": null, 237 | "metadata": {}, 238 | "outputs": [], 239 | "source": [ 240 | "estimated, goal_function = basin2d_inversion(\n", 241 | " x, data, basin_boundaries, density, initial=np.full(30, 1000),\n", 242 | ")\n", 243 | "predicted = cc.forward_model(estimated, basin_boundaries, density, x)" 244 | ] 245 | }, 246 | { 247 | "cell_type": "markdown", 248 | "metadata": {}, 249 | "source": [ 250 | "Plot the observed vs predicted data so we can inspect the fit." 251 | ] 252 | }, 253 | { 254 | "cell_type": "code", 255 | "execution_count": null, 256 | "metadata": {}, 257 | "outputs": [], 258 | "source": [ 259 | "plt.figure(figsize=(9, 3))\n", 260 | "plt.plot(x / 1e3, data, \".k\", label=\"observed\")\n", 261 | "plt.plot(x / 1e3, predicted, \"-r\", label='predicted')\n", 262 | "plt.legend()\n", 263 | "plt.xlabel(\"x [km]\")\n", 264 | "plt.ylabel(\"gravity disturbance [mGal]\")\n", 265 | "plt.show()" 266 | ] 267 | }, 268 | { 269 | "cell_type": "markdown", 270 | "metadata": {}, 271 | "source": [ 272 | "Look at the convergence of the method." 273 | ] 274 | }, 275 | { 276 | "cell_type": "code", 277 | "execution_count": null, 278 | "metadata": {}, 279 | "outputs": [], 280 | "source": [ 281 | "plt.figure()\n", 282 | "plt.plot(goal_function)\n", 283 | "plt.yscale(\"log\")\n", 284 | "plt.xlabel(\"iteration\")\n", 285 | "plt.ylabel(\"goal function (mGal²)\")\n", 286 | "plt.show()" 287 | ] 288 | }, 289 | { 290 | "cell_type": "markdown", 291 | "metadata": {}, 292 | "source": [ 293 | "And finally see if our estimate is close to the true model." 294 | ] 295 | }, 296 | { 297 | "cell_type": "code", 298 | "execution_count": null, 299 | "metadata": {}, 300 | "outputs": [], 301 | "source": [ 302 | "ax = cc.plot_prisms(depths, basin_boundaries)\n", 303 | "cc.plot_prisms(estimated, basin_boundaries, edgecolor=\"blue\", ax=ax)" 304 | ] 305 | }, 306 | { 307 | "cell_type": "markdown", 308 | "metadata": {}, 309 | "source": [ 310 | "Perfect! It seems that our inversion works well under these conditions (this initial estimate and no noise in the data). **Now let's break it!**" 311 | ] 312 | }, 313 | { 314 | "cell_type": "markdown", 315 | "metadata": {}, 316 | "source": [ 317 | "## **Your turn**\n", 318 | "\n", 319 | "**Add pseudo-random noise to the data using `np.random.normal` function and investigate the effect this has on the inversion results.** A typical gravity survey has accuracy in between 0.5-1 mGal. " 320 | ] 321 | }, 322 | { 323 | "cell_type": "code", 324 | "execution_count": null, 325 | "metadata": {}, 326 | "outputs": [], 327 | "source": [] 328 | }, 329 | { 330 | "cell_type": "code", 331 | "execution_count": null, 332 | "metadata": {}, 333 | "outputs": [], 334 | "source": [] 335 | }, 336 | { 337 | "cell_type": "code", 338 | "execution_count": null, 339 | "metadata": {}, 340 | "outputs": [], 341 | "source": [] 342 | }, 343 | { 344 | "cell_type": "markdown", 345 | "metadata": {}, 346 | "source": [ 347 | "## Stability test\n", 348 | "\n", 349 | "We can go one step further and run several of these inversion in a loop for different random noise realizations. This will give us an idea of how stable the inversion is." 350 | ] 351 | }, 352 | { 353 | "cell_type": "code", 354 | "execution_count": null, 355 | "metadata": {}, 356 | "outputs": [], 357 | "source": [ 358 | "estimates = []\n", 359 | "for i in range(5):\n", 360 | " noise = np.random.normal(loc=0, scale=1, size=data.size)\n", 361 | " noisy_data = data + noise\n", 362 | " estimated, goal_function = basin2d_inversion(\n", 363 | " x, noisy_data, basin_boundaries, density, initial=np.full(30, 1000),\n", 364 | " )\n", 365 | " estimates.append(estimated)" 366 | ] 367 | }, 368 | { 369 | "cell_type": "code", 370 | "execution_count": null, 371 | "metadata": {}, 372 | "outputs": [], 373 | "source": [ 374 | "ax = cc.plot_prisms(depths, basin_boundaries)\n", 375 | "for estimated in estimates:\n", 376 | " cc.plot_prisms(estimated, basin_boundaries, edgecolor=\"#0000ff66\", ax=ax)" 377 | ] 378 | }, 379 | { 380 | "cell_type": "markdown", 381 | "metadata": {}, 382 | "source": [ 383 | "## **Question time!**\n", 384 | "\n", 385 | "**Why does the inversion become more unstable for the deeper portions of the model?**\n", 386 | "\n", 387 | "Hint: It's related to the physics of the forward modelling and the Jacobian matrix." 388 | ] 389 | }, 390 | { 391 | "cell_type": "markdown", 392 | "metadata": {}, 393 | "source": [ 394 | "## Regularization to the rescue\n", 395 | "\n", 396 | "To deal with the instability issues we encountered, we will apply **first-order Tikhonov regularization** (aka \"smoothness\"). \n", 397 | "\n", 398 | "First thing we need to do is create the finite difference matrix $\\bar{\\bar{R}}$." 399 | ] 400 | }, 401 | { 402 | "cell_type": "code", 403 | "execution_count": null, 404 | "metadata": {}, 405 | "outputs": [], 406 | "source": [ 407 | "def finite_difference_matrix(nparams):\n", 408 | " \"\"\"\n", 409 | " Create the finite difference matrix for regularization.\n", 410 | " \"\"\"\n", 411 | " fdmatrix = np.zeros((nparams - 1, nparams))\n", 412 | " for i in range(fdmatrix.shape[0]):\n", 413 | " fdmatrix[i, i] = -1\n", 414 | " fdmatrix[i, i + 1] = 1\n", 415 | " return fdmatrix" 416 | ] 417 | }, 418 | { 419 | "cell_type": "code", 420 | "execution_count": null, 421 | "metadata": {}, 422 | "outputs": [], 423 | "source": [ 424 | "finite_difference_matrix(10)" 425 | ] 426 | }, 427 | { 428 | "cell_type": "markdown", 429 | "metadata": {}, 430 | "source": [ 431 | "Now we can use this to make a new inversion function with smoothness." 432 | ] 433 | }, 434 | { 435 | "cell_type": "code", 436 | "execution_count": null, 437 | "metadata": {}, 438 | "outputs": [], 439 | "source": [ 440 | "def basin2d_smooth_inversion(x, data, basin_boundaries, density, initial, smoothness, max_iterations=10):\n", 441 | " \"\"\"\n", 442 | " Solve the regularized inverse problem using the Gauss-Newton method.\n", 443 | " \"\"\"\n", 444 | " parameters = initial.astype(np.float64).copy() \n", 445 | " predicted = cc.forward_model(parameters, basin_boundaries, density, x)\n", 446 | " residuals = data - predicted\n", 447 | " goal_function = [np.linalg.norm(residuals)**2]\n", 448 | " fdmatrix = finite_difference_matrix(parameters.size)\n", 449 | " for i in range(max_iterations): \n", 450 | " jacobian = make_jacobian(parameters, basin_boundaries, density, x)\n", 451 | " hessian = jacobian.T @ jacobian + smoothness * fdmatrix.T @ fdmatrix\n", 452 | " gradient = jacobian.T @ residuals - smoothness * fdmatrix.T @ fdmatrix @ parameters\n", 453 | " deltap = np.linalg.solve(hessian, gradient)\n", 454 | " new_parameters = parameters + deltap\n", 455 | " predicted = cc.forward_model(new_parameters, basin_boundaries, density, x)\n", 456 | " residuals = data - predicted\n", 457 | " current_goal = np.linalg.norm(residuals)**2\n", 458 | " if current_goal > goal_function[-1]:\n", 459 | " break\n", 460 | " parameters = new_parameters\n", 461 | " goal_function.append(current_goal)\n", 462 | " return parameters, goal_function" 463 | ] 464 | }, 465 | { 466 | "cell_type": "markdown", 467 | "metadata": {}, 468 | "source": [ 469 | "Now we check if it works on our noisy data." 470 | ] 471 | }, 472 | { 473 | "cell_type": "code", 474 | "execution_count": null, 475 | "metadata": {}, 476 | "outputs": [], 477 | "source": [ 478 | "estimates = []\n", 479 | "for i in range(5):\n", 480 | " noise = np.random.normal(loc=0, scale=1, size=data.size)\n", 481 | " noisy_data = data + noise\n", 482 | " estimated, goal_function = basin2d_smooth_inversion(\n", 483 | " x, noisy_data, basin_boundaries, density, initial=np.full(30, 1000), smoothness=1e-5\n", 484 | " )\n", 485 | " estimates.append(estimated)" 486 | ] 487 | }, 488 | { 489 | "cell_type": "code", 490 | "execution_count": null, 491 | "metadata": {}, 492 | "outputs": [], 493 | "source": [ 494 | "ax = cc.plot_prisms(depths, basin_boundaries)\n", 495 | "for estimated in estimates:\n", 496 | " cc.plot_prisms(estimated, basin_boundaries, edgecolor=\"#0000ff66\", ax=ax)" 497 | ] 498 | }, 499 | { 500 | "cell_type": "markdown", 501 | "metadata": {}, 502 | "source": [ 503 | "## **Question time!**\n", 504 | "\n", 505 | "**What happens when the regularization paramater is extremely high?** Try to predict what the answer would be and then execute the code to check your reasoning.\n", 506 | "\n", 507 | "Hint: what is the smoothest possible model?" 508 | ] 509 | }, 510 | { 511 | "cell_type": "markdown", 512 | "metadata": {}, 513 | "source": [ 514 | "## **Your turn**\n", 515 | "\n", 516 | "**Can our regularized model recover a non-smooth geometry?** For example, real sedimentary basins often have [faults](https://en.wikipedia.org/wiki/Fault_(geology)) running through them, causing sharp jumps in the sediment thickness (up or down). \n", 517 | "\n", 518 | "To answer this question:\n", 519 | "\n", 520 | "1. Use the modified model depths below (the `depths` array) that introduce a shift up or down by 1-2 km in a section of the model of about 5-10 km.\n", 521 | "2. Generate new noisy data with this new model\n", 522 | "3. Invert the noisy data and try to find a model that:\n", 523 | " 1. Fits the data\n", 524 | " 2. Is stable (doesn't vary much if we change the noise)\n", 525 | " 3. Recovers the sharp boundary" 526 | ] 527 | }, 528 | { 529 | "cell_type": "code", 530 | "execution_count": null, 531 | "metadata": {}, 532 | "outputs": [], 533 | "source": [ 534 | "fault_model = np.copy(depths)\n", 535 | "fault_model[45:55] -= 2000" 536 | ] 537 | }, 538 | { 539 | "cell_type": "code", 540 | "execution_count": null, 541 | "metadata": {}, 542 | "outputs": [], 543 | "source": [ 544 | "cc.plot_prisms(fault_model, basin_boundaries)" 545 | ] 546 | }, 547 | { 548 | "cell_type": "markdown", 549 | "metadata": {}, 550 | "source": [ 551 | "Now fill out the rest of the code below!" 552 | ] 553 | }, 554 | { 555 | "cell_type": "code", 556 | "execution_count": null, 557 | "metadata": {}, 558 | "outputs": [], 559 | "source": [] 560 | }, 561 | { 562 | "cell_type": "code", 563 | "execution_count": null, 564 | "metadata": {}, 565 | "outputs": [], 566 | "source": [] 567 | }, 568 | { 569 | "cell_type": "code", 570 | "execution_count": null, 571 | "metadata": {}, 572 | "outputs": [], 573 | "source": [] 574 | }, 575 | { 576 | "cell_type": "code", 577 | "execution_count": null, 578 | "metadata": {}, 579 | "outputs": [], 580 | "source": [] 581 | }, 582 | { 583 | "cell_type": "markdown", 584 | "metadata": {}, 585 | "source": [ 586 | "## **Question time!**\n", 587 | "\n", 588 | "**What would happen if we used a \"sharpness\" regularization?** Would we be able to recover the faults? What about the smoother parts of the model? \n", 589 | "\n", 590 | "One type of sharpness regularization is called \"total-variation regularization\" and it [has been used for this problem in the past](https://doi.org/10.1190/1.3524286)." 591 | ] 592 | }, 593 | { 594 | "cell_type": "markdown", 595 | "metadata": {}, 596 | "source": [ 597 | "## Extra thinking points\n", 598 | "\n", 599 | "* What happens if we get the density wrong?\n", 600 | "* What are the sources of uncertainty in our final solution? Is it just the noise in the data?\n", 601 | "* How much does the solution depend on the inital estimate?" 602 | ] 603 | }, 604 | { 605 | "cell_type": "markdown", 606 | "metadata": {}, 607 | "source": [ 608 | "## **Bonus:** Optimizing code\n", 609 | "\n", 610 | "The code we wrote is not the greatest and it does take a while to run even for these really small 2D problems. There are ways in which we can make the code fast. But before we do any of that, **we need to know where our code spends most of its time**. Otherwise, we could spend hours optimizing a part of the code that is already really fast.\n", 611 | "\n", 612 | "This can be done with tools called **profilers**, which measure the time spent in each function of your code. This is also why its very important to **break up your code into functions**. In a Jupyter notebook, you can run the standard Python profiler by using the `%%prun` cell magic:" 613 | ] 614 | }, 615 | { 616 | "cell_type": "code", 617 | "execution_count": null, 618 | "metadata": {}, 619 | "outputs": [], 620 | "source": [ 621 | "%%prun \n", 622 | "basin2d_smooth_inversion(\n", 623 | " x, noisy_data, basin_boundaries, density, initial=np.full(30, 1000), smoothness=1e-5\n", 624 | ")" 625 | ] 626 | }, 627 | { 628 | "cell_type": "markdown", 629 | "metadata": {}, 630 | "source": [ 631 | "The `tottime` column is the amount of time spent on the function itself (not counting functions called inside it) and `cumtime` is the total time spent in the function, including function calls inside it. \n", 632 | "\n", 633 | "We can see from the profiling that the majority of the computation is spend in forward modelling, in particular for building the Jacobian. So if we can optimize `make_jacobian` that will have the biggest impact on performance of all.\n", 634 | "\n", 635 | "To start let's measure the computation time of `make_jacobian` with the `%%timeit` magic:" 636 | ] 637 | }, 638 | { 639 | "cell_type": "code", 640 | "execution_count": null, 641 | "metadata": {}, 642 | "outputs": [], 643 | "source": [ 644 | "%%timeit\n", 645 | "make_jacobian(np.full(30, 1000), basin_boundaries, density, x)" 646 | ] 647 | }, 648 | { 649 | "cell_type": "markdown", 650 | "metadata": {}, 651 | "source": [ 652 | "Alright, now we can try to do better.\n", 653 | "\n", 654 | "For many of these problems, the biggest return on investment is **not** parallelization or going to C/Fortran. **The largest improvements come from better maths/physics**. Here, we can take advantage of potential-field theory to cut down on the computation time of the Jacobian. \n", 655 | "\n", 656 | "We'll use the fact that the difference in gravity values produced by two models is the same as the gravity value produced by the difference in the models. Meaning that $\\delta g = g(m_1) - g(m_2) = g(m_1 - m_2)$. This way, we can reduce by more than half the number of forward modelling operations we do in the finite-difference computations.\n", 657 | "\n", 658 | "So instead of calculating the entire basin model with and without a small step in a single parameter, we can only calculate the effect of that small step." 659 | ] 660 | }, 661 | { 662 | "cell_type": "code", 663 | "execution_count": null, 664 | "metadata": {}, 665 | "outputs": [], 666 | "source": [ 667 | "def make_jacobian_fast(parameters, basin_boundaries, density, x):\n", 668 | " \"\"\"\n", 669 | " Calculate the Jacobian matrix by finite differences.\n", 670 | " \"\"\"\n", 671 | " jacobian = np.empty((x.size, parameters.size))\n", 672 | " delta = 10\n", 673 | " boundaries = cc.prism_boundaries(parameters, basin_boundaries)\n", 674 | " for j in range(jacobian.shape[1]):\n", 675 | " jacobian[:, j] = (\n", 676 | " (\n", 677 | " # Replace with a single forward modelling of a single prism\n", 678 | " cc.prism_gravity(x, boundaries[j], boundaries[j + 1], parameters[j], parameters[j] + delta, density)\n", 679 | " ) \n", 680 | " / delta\n", 681 | " )\n", 682 | " return jacobian" 683 | ] 684 | }, 685 | { 686 | "cell_type": "markdown", 687 | "metadata": {}, 688 | "source": [ 689 | "First, we check if the results are still correct." 690 | ] 691 | }, 692 | { 693 | "cell_type": "code", 694 | "execution_count": null, 695 | "metadata": {}, 696 | "outputs": [], 697 | "source": [ 698 | "np.allclose(\n", 699 | " make_jacobian(np.full(30, 1000), basin_boundaries, density, x),\n", 700 | " make_jacobian_fast(np.full(30, 1000), basin_boundaries, density, x)\n", 701 | ")" 702 | ] 703 | }, 704 | { 705 | "cell_type": "markdown", 706 | "metadata": {}, 707 | "source": [ 708 | "Now we can measure the time again:" 709 | ] 710 | }, 711 | { 712 | "cell_type": "code", 713 | "execution_count": null, 714 | "metadata": {}, 715 | "outputs": [], 716 | "source": [ 717 | "%%timeit\n", 718 | "make_jacobian_fast(np.full(30, 1000), basin_boundaries, density, x)" 719 | ] 720 | }, 721 | { 722 | "cell_type": "markdown", 723 | "metadata": {}, 724 | "source": [ 725 | "This one change gave use 2 orders of magnitude improvement in the function that makes up most of the computation time. **Now that is time well spent!**\n", 726 | "\n", 727 | "We can measure how much of a difference this makes for the inversion as a whole by making a new function with our fast Jacobian matrix calculation." 728 | ] 729 | }, 730 | { 731 | "cell_type": "code", 732 | "execution_count": null, 733 | "metadata": {}, 734 | "outputs": [], 735 | "source": [ 736 | "def fast_basin2d_smooth_inversion(x, data, basin_boundaries, density, initial, smoothness, max_iterations=10):\n", 737 | " \"\"\"\n", 738 | " Solve the regularized inverse problem using the Gauss-Newton method.\n", 739 | " \"\"\"\n", 740 | " parameters = initial.astype(np.float64).copy() \n", 741 | " predicted = cc.forward_model(parameters, basin_boundaries, density, x)\n", 742 | " residuals = data - predicted\n", 743 | " goal_function = [np.linalg.norm(residuals)**2]\n", 744 | " fdmatrix = finite_difference_matrix(parameters.size)\n", 745 | " for i in range(max_iterations): \n", 746 | " # Swap out the slow jacobian for the fast one\n", 747 | " jacobian = make_jacobian_fast(parameters, basin_boundaries, density, x)\n", 748 | " hessian = jacobian.T @ jacobian + smoothness * fdmatrix.T @ fdmatrix\n", 749 | " gradient = jacobian.T @ residuals - smoothness * fdmatrix.T @ fdmatrix @ parameters\n", 750 | " deltap = np.linalg.solve(hessian, gradient)\n", 751 | " new_parameters = parameters + deltap\n", 752 | " predicted = cc.forward_model(new_parameters, basin_boundaries, density, x)\n", 753 | " residuals = data - predicted\n", 754 | " current_goal = np.linalg.norm(residuals)**2\n", 755 | " if current_goal > goal_function[-1]:\n", 756 | " break\n", 757 | " parameters = new_parameters\n", 758 | " goal_function.append(current_goal)\n", 759 | " return parameters, goal_function" 760 | ] 761 | }, 762 | { 763 | "cell_type": "markdown", 764 | "metadata": {}, 765 | "source": [ 766 | "And now we can measure the computation time for both." 767 | ] 768 | }, 769 | { 770 | "cell_type": "code", 771 | "execution_count": null, 772 | "metadata": {}, 773 | "outputs": [], 774 | "source": [ 775 | "%%timeit \n", 776 | "basin2d_smooth_inversion(\n", 777 | " x, noisy_data, basin_boundaries, density, initial=np.full(30, 1000), smoothness=1e-5\n", 778 | ")" 779 | ] 780 | }, 781 | { 782 | "cell_type": "code", 783 | "execution_count": null, 784 | "metadata": {}, 785 | "outputs": [], 786 | "source": [ 787 | "%%timeit \n", 788 | "fast_basin2d_smooth_inversion(\n", 789 | " x, noisy_data, basin_boundaries, density, initial=np.full(30, 1000), smoothness=1e-5\n", 790 | ")" 791 | ] 792 | }, 793 | { 794 | "cell_type": "markdown", 795 | "metadata": {}, 796 | "source": [ 797 | "**We changed 3 lines of code and achived a factor of 10 speedup.** Again, this could only be done because we first profiled the code and then focused on finding a fundamentally better way of calculating. " 798 | ] 799 | } 800 | ], 801 | "metadata": { 802 | "kernelspec": { 803 | "display_name": "Python [conda env:non-linear-gravity-inversion]", 804 | "language": "python", 805 | "name": "conda-env-non-linear-gravity-inversion-py" 806 | }, 807 | "language_info": { 808 | "codemirror_mode": { 809 | "name": "ipython", 810 | "version": 3 811 | }, 812 | "file_extension": ".py", 813 | "mimetype": "text/x-python", 814 | "name": "python", 815 | "nbconvert_exporter": "python", 816 | "pygments_lexer": "ipython3", 817 | "version": "3.9.0" 818 | } 819 | }, 820 | "nbformat": 4, 821 | "nbformat_minor": 4 822 | } 823 | -------------------------------------------------------------------------------- /images/Half-graben_sedimentation.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeophysicsLibrary/non-linear-gravity-inversion/df8d5f629b2993af317b22addc8ad289217bc9c7/images/Half-graben_sedimentation.png -------------------------------------------------------------------------------- /images/README.md: -------------------------------------------------------------------------------- 1 | # Images used in notebook, slides, and notes 2 | 3 | Except where otherwise noted, all images are my own work (Leonardo Uieda) and 4 | distributed under the [CC-BY 4.0](https://creativecommons.org/licenses/by/4.0/) 5 | license. 6 | 7 | The exceptions are: 8 | 9 | * [`Half-graben_sedimentation.png`](https://commons.wikimedia.org/wiki/File:Half-graben_sedimentation.png) 10 | by Mikenorton (CC-BY-SA) 11 | -------------------------------------------------------------------------------- /images/basin-example1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeophysicsLibrary/non-linear-gravity-inversion/df8d5f629b2993af317b22addc8ad289217bc9c7/images/basin-example1.png -------------------------------------------------------------------------------- /images/basin-example2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeophysicsLibrary/non-linear-gravity-inversion/df8d5f629b2993af317b22addc8ad289217bc9c7/images/basin-example2.png -------------------------------------------------------------------------------- /images/gravity-disturbance.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeophysicsLibrary/non-linear-gravity-inversion/df8d5f629b2993af317b22addc8ad289217bc9c7/images/gravity-disturbance.png -------------------------------------------------------------------------------- /images/gravity-earth.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeophysicsLibrary/non-linear-gravity-inversion/df8d5f629b2993af317b22addc8ad289217bc9c7/images/gravity-earth.png -------------------------------------------------------------------------------- /images/gravity-normal-earth.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeophysicsLibrary/non-linear-gravity-inversion/df8d5f629b2993af317b22addc8ad289217bc9c7/images/gravity-normal-earth.png -------------------------------------------------------------------------------- /notes.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeophysicsLibrary/non-linear-gravity-inversion/df8d5f629b2993af317b22addc8ad289217bc9c7/notes.pdf -------------------------------------------------------------------------------- /notes.xopp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/GeophysicsLibrary/non-linear-gravity-inversion/df8d5f629b2993af317b22addc8ad289217bc9c7/notes.xopp --------------------------------------------------------------------------------