├── .gitignore ├── 1-intro-and-data.ipynb ├── 2-temperature.ipynb ├── 3-injectivity.ipynb ├── 4-feedzones.ipynb ├── Data-FlowRate.xlsx ├── Data-PTS.xlsx ├── Data-Temp-Heating37days.csv ├── Figures ├── Figure1.png ├── Figure2.jpg ├── Figure3.png ├── Figure4.png ├── Figure5.png └── Table1.png ├── LICENSE ├── README.md ├── bonus-combine-data.ipynb ├── bonus-filter-by-toolspeed.ipynb └── environment.yml /.gitignore: -------------------------------------------------------------------------------- 1 | # Don't track content of these folders 2 | 3 | #node_modules/ 4 | .ipynb_checkpoints 5 | .pyc 6 | .vscode/ 7 | *__pycache__/ 8 | 9 | #someOtherfoler/ 10 | 11 | # graphics 12 | ########## 13 | *.ai 14 | 15 | # Compiled source # 16 | ################### 17 | *.com 18 | *.class 19 | *.dll 20 | *.exe 21 | *.o 22 | *.so 23 | 24 | # Packages # 25 | ############ 26 | # it's better to unpack these files and commit the raw source 27 | # git has its own built in compression methods 28 | *.7z 29 | *.dmg 30 | *.gz 31 | *.iso 32 | *.jar 33 | *.rar 34 | *.tar 35 | *.zip -------------------------------------------------------------------------------- /1-intro-and-data.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": { 6 | "iooxa": { 7 | "id": { 8 | "block": "m7G866LtzKJf514citC5", 9 | "project": "mmReuqVTAa9JzPpNr22I", 10 | "version": 1 11 | } 12 | }, 13 | "tags": [] 14 | }, 15 | "source": [ 16 | "***\n", 17 | "\n", 18 | "# Geothermal Well Test Analysis with Python\n", 19 | "### Notebook 1: Introduction and Data Preparation \n", 20 | "#### Irene Wallis and Katie McLean \n", 21 | "#### Software Underground, Transform 2021\n", 22 | "\n", 23 | "***" 24 | ] 25 | }, 26 | { 27 | "cell_type": "markdown", 28 | "metadata": { 29 | "iooxa": { 30 | "id": { 31 | "block": "H5L6ePY3Avvd5cyrJzUY", 32 | "project": "mmReuqVTAa9JzPpNr22I", 33 | "version": 1 34 | } 35 | }, 36 | "tags": [] 37 | }, 38 | "source": [ 39 | "### Google Colab Setup\n", 40 | "\n", 41 | "If you are using Google Colab to run this notebook, we assume you have already followed the Google Colab setup steps outlined [here](https://github.com/ICWallis/T21-Tutorial-WellTestAnalysis).\n", 42 | "\n", 43 | "Because we are importing data, we need to \"mount your Google Drive\", which is where we tell this notebook to look for the data files. You will need to mount the Google Drive into each notebook. \n", 44 | "\n", 45 | "1. Run the cell below if you are in Google Colab. If you are not in Google Colab, running the cell below will just return an error that says \"No module named 'google'\". If you get a Google Colab error that says \"Unrecognised runtime 'geothrm'; defaulting to 'python3' Notebook settings\", just ignore it. \n", 46 | "\n", 47 | "2. Follow the link generated by running this code. That link will ask you to sign in to your google account (use the one where you have saved these tutorial materials in) and to allow this notebook access to your google drive. \n", 48 | "\n", 49 | "3. Completing step 2 above will generate a code. Copy this code, paste below where it says \"Enter your authorization code:\", and press ENTER. \n", 50 | "\n", 51 | "Congratulations, this notebook can now import data!" 52 | ] 53 | }, 54 | { 55 | "cell_type": "code", 56 | "execution_count": null, 57 | "metadata": { 58 | "iooxa": { 59 | "id": { 60 | "block": "dY52tZz5qE2efiMqESlU", 61 | "project": "mmReuqVTAa9JzPpNr22I", 62 | "version": 1 63 | }, 64 | "outputId": null 65 | } 66 | }, 67 | "outputs": [], 68 | "source": [ 69 | "from google.colab import drive\n", 70 | "drive.mount('/content/drive')" 71 | ] 72 | }, 73 | { 74 | "cell_type": "markdown", 75 | "metadata": { 76 | "iooxa": { 77 | "id": { 78 | "block": "fILAjTKQtyjpgkJpCFDl", 79 | "project": "mmReuqVTAa9JzPpNr22I", 80 | "version": 1 81 | } 82 | } 83 | }, 84 | "source": [ 85 | "\n", 86 | "***\n", 87 | "\n", 88 | "# 1. Introduction\n", 89 | "\n", 90 | "#### Tutorial overview\n", 91 | "\n", 92 | "In this tutorial, we provide an overview of geothermal wells, well testing in general, and the types of data that are captured during completion testing. We then step though a completion test analysis workflow including:\n", 93 | "- Import, formatting and cleaning of pressure, temperature and spinner log (PTS) data and the associated pump data\n", 94 | "- Extraction and plotting of selected temperature logs\n", 95 | "- Interpolation of injectivity index from pressure and pump flowrate data\n", 96 | "- Detection of feed zones through spinner analysis\n", 97 | "\n", 98 | "Completion testing involves the capture of a large volume of data, which can be cumbersome to analyse in Excel. During this tutorial, we demonstrate efficient and agile processing of these data using Python. Working inside the Jupyter Notebook environment also enables us to generate an annotated record of our work that can be easily repeated or audited, which is worthwhile considering the high-value decisions that are based on completion test analysis. [Jupyter Lab](https://jupyterlab.readthedocs.io/en/stable/user/export.html) and [VS Studio Code](https://code.visualstudio.com/docs/python/jupyter-support) both enable exporting these Jupyter Notebooks to formats more typically used for reports, such as HTML or pdf. \n", 99 | "\n", 100 | "#### Reuse of the material in this tutorial\n", 101 | "\n", 102 | "We have made this workflow open-source under the Apache 2.0 licence, which is a permissive licence that allows reuse in your own work with attribution to the original authors. If you alter the code or text, then you need to add a prominent notification that indicates where you have made changes.\n", 103 | "\n", 104 | "This code, markdown text, and sample dataset can be cited using the following details:\n", 105 | "\n", 106 | "_Wallis, I.C. and McLean, K. (2021) Geothermal Well Test Analysis with Python, Software Underground Transform 2021 Tutorial, https://github.com/ICWallis/T21-Tutorial-WellTestAnalysis_\n", 107 | "\n", 108 | "***\n", 109 | "\n", 110 | "## 1.1 A definition for geothermal\n", 111 | "\n", 112 | "In this tutorial, we discuss ‘conventional’ geothermal systems where a localised heat source (e.g., shallow crustal magma) or high background heat flow produces a plume of heated fluid that rises buoyantly towards the surface. The hot fluids that are close enough to the surface to be intersected by drilling (a few km) are the conventional geothermal reservoir (Figure 1). At a large scale, we think of these geothermal systems as convection cells that are recharged by regional groundwater. However, their geometry at the reservoir-scale more resembles a thermal conveyor belt where hot fluids are transported by buoyancy process from depth to the near subsurface: They enter the reservoir through one or more deep upflow zones and then exit though one or more shallow outflow zones. We do not discuss engineered (hot dry rock) geothermal systems or low-temperature sedimentary aquifers. However, the testing methods used in those settings are similar to those described in this tutorial. " 113 | ] 114 | }, 115 | { 116 | "cell_type": "code", 117 | "execution_count": null, 118 | "metadata": { 119 | "iooxa": { 120 | "id": { 121 | "block": "lKUcYib19aaBUKYdLyBv", 122 | "project": "mmReuqVTAa9JzPpNr22I", 123 | "version": 2 124 | }, 125 | "outputId": { 126 | "block": "YFZh1TiCP2nqOieoPxSe", 127 | "project": "mmReuqVTAa9JzPpNr22I", 128 | "version": 1 129 | } 130 | } 131 | }, 132 | "outputs": [], 133 | "source": [ 134 | "from IPython.display import Image\n", 135 | "Image('https://raw.githubusercontent.com/ICWallis/T21-Tutorial-WellTestAnalysis/main/Figures/Figure1.png',width = 800,)" 136 | ] 137 | }, 138 | { 139 | "cell_type": "markdown", 140 | "metadata": { 141 | "iooxa": { 142 | "id": { 143 | "block": "RJkI01K0WMBw46ZuzVJO", 144 | "project": "mmReuqVTAa9JzPpNr22I", 145 | "version": 1 146 | } 147 | } 148 | }, 149 | "source": [ 150 | "_Figure 1: Cross-section of conventional geothermal reservoir, showing rising plume of buoyant fluid (from Zarrouk and McLean, 2019 and reproduced from Brian Lovelock, Jacobs Ltd. with kind permission)._\n", 151 | "\n", 152 | "For geothermal reservoir engineering, it is useful to classify conventional geothermal systems by enthalpy because it informs on the dominant heat transfer mechanism at work in the reservoir: conduction, convection, or advection (Table 1). \n", 153 | "\n", 154 | "_Table 1: Classification for geothermal systems by enthalpy (from Kaya et al., 2011)._" 155 | ] 156 | }, 157 | { 158 | "cell_type": "code", 159 | "execution_count": null, 160 | "metadata": { 161 | "iooxa": { 162 | "id": { 163 | "block": "4VNnCugFW134yribDHjU", 164 | "project": "mmReuqVTAa9JzPpNr22I", 165 | "version": 2 166 | }, 167 | "outputId": { 168 | "block": "K8srrIulfjkPJ0x3oIjQ", 169 | "project": "mmReuqVTAa9JzPpNr22I", 170 | "version": 1 171 | } 172 | } 173 | }, 174 | "outputs": [], 175 | "source": [ 176 | "Image('https://raw.githubusercontent.com/ICWallis/T21-Tutorial-WellTestAnalysis/main/Figures/Table1.png',width = 650,)" 177 | ] 178 | }, 179 | { 180 | "cell_type": "markdown", 181 | "metadata": { 182 | "iooxa": { 183 | "id": { 184 | "block": "bMN55dWeUYLGiw0IVqpM", 185 | "project": "mmReuqVTAa9JzPpNr22I", 186 | "version": 1 187 | } 188 | } 189 | }, 190 | "source": [ 191 | "***\n", 192 | "\n", 193 | "## 1.2 Introduction to geothermal wells\n", 194 | "\n", 195 | "Geothermal wells completions differ significantly from petroleum wells that are typically only perforated over very short intervals (i.e., have limited access into the reservoir). Geothermal wells have very long perforated intervals with access to the reservoir, often in excess of 1000 metres in length (Figure 2). " 196 | ] 197 | }, 198 | { 199 | "cell_type": "code", 200 | "execution_count": null, 201 | "metadata": { 202 | "iooxa": { 203 | "id": { 204 | "block": "05stv76zHnFhHsKcLhGJ", 205 | "project": "mmReuqVTAa9JzPpNr22I", 206 | "version": 2 207 | }, 208 | "outputId": { 209 | "block": "u4kdGOs6KFYrIWYgz3yv", 210 | "project": "mmReuqVTAa9JzPpNr22I", 211 | "version": 1 212 | } 213 | } 214 | }, 215 | "outputs": [], 216 | "source": [ 217 | "Image('https://raw.githubusercontent.com/ICWallis/T21-Tutorial-WellTestAnalysis/main/Figures/Figure2.jpg',width = 500,)" 218 | ] 219 | }, 220 | { 221 | "cell_type": "markdown", 222 | "metadata": { 223 | "iooxa": { 224 | "id": { 225 | "block": "NSDN3DyHgWrH9MLduwLu", 226 | "project": "mmReuqVTAa9JzPpNr22I", 227 | "version": 1 228 | } 229 | } 230 | }, 231 | "source": [ 232 | "_Figure 2: Schematic of geothermal well showing feed zones within the production zone accessing the wellbore via the perforated liner (Zarrouk and McLean, 2019)._\n", 233 | "\n", 234 | "Geothermal wells will self-discharge if there is sufficient temperature to generate boiling. In these conditions, flow is driven by the volume expansion of water during boiling rather than high pressures as is the case in groundwater wells. Geothermal wells that are too cool to boil will still flow if there is sufficient hydrostatic head, such as in mountainous areas. Otherwise, these wells are pumped. Pressure gradients in liquid-dominated reservoirs are typically hot hydrostatic (i.e., at a lower pressure than the regional cold hydrostatic gradient). In other words, the pressure is lower inside the reservoir than outside – counter-intuitive but true. \n", 235 | "\n", 236 | "***\n", 237 | "\n", 238 | "## 1.3 Introduction to well testing\n", 239 | "\n", 240 | "A geothermal well is typically drilled over a period of a few weeks. Immediately after drilling is completed, a program of testing commences. The typical testing sequence is: completion testing, then heating runs, and finally output testing. \n", 241 | "\n", 242 | "The **completion test** captures pressure, temperature and spinner (PTS) data using a downhole PTS tool (Figure 3). This tool is suspended in the wellbore on a wireline whilst cold water is injected into the well at different rates through the side valve (Figure 4). \n", 243 | "\n", 244 | "The main objectives of completion testing are to:\n", 245 | "1. Identify permeable feed zones in the well, including whether there is a dominant feed zone.\n", 246 | "2. Gain an indication of the future potential of the well once in service. \n", 247 | "\n", 248 | "After completion testing, a set of **heating runs** are conducted in order to observe the pressure and temperature response of the well after cold water injection stops (hence heating commences). Typically at least three heating runs are acquired at increasing time intervals (i.e., immediately after the completion test, then days later, then weeks or months later depending on the reservoir conditions). These heatup data contribute to objectives 1 and 2 of completion testing. \n" 249 | ] 250 | }, 251 | { 252 | "cell_type": "code", 253 | "execution_count": null, 254 | "metadata": { 255 | "iooxa": { 256 | "id": { 257 | "block": "QVkIpL5oetP2PldnmGvS", 258 | "project": "mmReuqVTAa9JzPpNr22I", 259 | "version": 2 260 | }, 261 | "outputId": { 262 | "block": "BFUerDJDIDtezG2PiyTy", 263 | "project": "mmReuqVTAa9JzPpNr22I", 264 | "version": 1 265 | } 266 | } 267 | }, 268 | "outputs": [], 269 | "source": [ 270 | "Image('https://raw.githubusercontent.com/ICWallis/T21-Tutorial-WellTestAnalysis/main/Figures/Figure3.png',width = 500,)" 271 | ] 272 | }, 273 | { 274 | "cell_type": "markdown", 275 | "metadata": { 276 | "iooxa": { 277 | "id": { 278 | "block": "qcYXe8KwyxBvUZVHiVcc", 279 | "project": "mmReuqVTAa9JzPpNr22I", 280 | "version": 1 281 | } 282 | } 283 | }, 284 | "source": [ 285 | "_Figure 3: Pressure-temperature-spinner (PTS) tool, 2.6 metres in length (Zarrouk and McLean, 2019)._" 286 | ] 287 | }, 288 | { 289 | "cell_type": "code", 290 | "execution_count": null, 291 | "metadata": { 292 | "iooxa": { 293 | "id": { 294 | "block": "z2U1DRjE9lHzwWUBFqhw", 295 | "project": "mmReuqVTAa9JzPpNr22I", 296 | "version": 2 297 | }, 298 | "outputId": { 299 | "block": "5cX6NaJmMtTKgxpIxiRy", 300 | "project": "mmReuqVTAa9JzPpNr22I", 301 | "version": 1 302 | } 303 | } 304 | }, 305 | "outputs": [], 306 | "source": [ 307 | "Image('https://raw.githubusercontent.com/ICWallis/T21-Tutorial-WellTestAnalysis/main/Figures/Figure4.png',width = 800,)" 308 | ] 309 | }, 310 | { 311 | "cell_type": "markdown", 312 | "metadata": { 313 | "iooxa": { 314 | "id": { 315 | "block": "aRexeJ7PrPo1DaNROA3r", 316 | "project": "mmReuqVTAa9JzPpNr22I", 317 | "version": 1 318 | } 319 | } 320 | }, 321 | "source": [ 322 | "_Figure 4: Schematic of downhole tool suspended on wireline, including surface equipment setup (Zarrouk and McLean, 2019)._\n", 323 | "\n", 324 | "If the well is destined to be a production well, then it is subjected to output testing (production testing) after heating. Some injection wells are not output tested, if it is certain that they will never be used for production. Typically the well is discharged at multiple flow rates during the output test, while the mass flow and well head pressure recorded for each. From this a single output curve is generated that shows the behaviour/potential of the well under various operating conditions. PTS data are also captured during discharge in order to see which of the feed zones are contributing to the flow, and how much.\n", 325 | "\n", 326 | "If you want to know more about geothermal well testing, check out a book co-authored by Katie with Sadiq Zarrouk that covers the theory and practice of geothermal well test analysis https://www.elsevier.com/books/geothermal-well-test-analysis/zarrouk/978-0-12-814946-1" 327 | ] 328 | }, 329 | { 330 | "cell_type": "markdown", 331 | "metadata": { 332 | "iooxa": { 333 | "id": { 334 | "block": "bYLMoJy2BbmveRDjqckj", 335 | "project": "mmReuqVTAa9JzPpNr22I", 336 | "version": 1 337 | } 338 | } 339 | }, 340 | "source": [ 341 | "***\n", 342 | "\n", 343 | "# 2. Import and munge test data\n", 344 | "\n", 345 | "We are working with two sources of data:\n", 346 | "\n", 347 | "1. Pumps operating at the surface (datetime and flow rate)\n", 348 | "2. PTS tool in the well (datetime, pressure, temperature, spinner, other tool information)\n", 349 | "\n", 350 | "The following steps take the raw data that has been provided as excel files, imports them as a Pandas dataframe (one called flowrate and the other called pts), and then do some manipulation to generate a nice clean dataset that we can work with.\n", 351 | "\n", 352 | "We will see that data format and data quality are common challenges, especially when datetime is involved. " 353 | ] 354 | }, 355 | { 356 | "cell_type": "code", 357 | "execution_count": null, 358 | "metadata": { 359 | "iooxa": { 360 | "id": { 361 | "block": "fE07fvbOZXnG23CTJ1J4", 362 | "project": "mmReuqVTAa9JzPpNr22I", 363 | "version": 1 364 | }, 365 | "outputId": null 366 | } 367 | }, 368 | "outputs": [], 369 | "source": [ 370 | "import openpyxl\n", 371 | "import pandas as pd\n", 372 | "from datetime import datetime" 373 | ] 374 | }, 375 | { 376 | "cell_type": "markdown", 377 | "metadata": { 378 | "iooxa": { 379 | "id": { 380 | "block": "7eW2AaBHOpzXX5kCYokV", 381 | "project": "mmReuqVTAa9JzPpNr22I", 382 | "version": 1 383 | } 384 | } 385 | }, 386 | "source": [ 387 | "***\n", 388 | "\n", 389 | "## 2.1. Pump (flowrate) data\n", 390 | "\n", 391 | "We will turn the excel data into a Pandas dataframe that includes:\n", 392 | "- A datetime field we can use with the tool data\n", 393 | "- Flowrate in the units we need\n", 394 | "\n", 395 | "### 2.1.1 Import flowrate data using Pandas\n", 396 | "\n", 397 | "We read the data in with the header location defined as row 1. This automatically skips row 0 which just contains some text that we don’t want." 398 | ] 399 | }, 400 | { 401 | "cell_type": "code", 402 | "execution_count": null, 403 | "metadata": { 404 | "iooxa": { 405 | "id": { 406 | "block": "XAwQM8ffvMIGuJgSJGr9", 407 | "project": "mmReuqVTAa9JzPpNr22I", 408 | "version": 2 409 | }, 410 | "outputId": null 411 | } 412 | }, 413 | "outputs": [], 414 | "source": [ 415 | "# Uncomment (remove the hash symbol) from the method that you want to use\n", 416 | "\n", 417 | "# Use this method if you are running this notebook in Google Colab\n", 418 | "# flowrate = pd.read_excel(r'/content/drive/My Drive/T21-Tutorial-WellTestAnalysis-main/Data-FlowRate.xlsx', header=1) \n", 419 | "\n", 420 | "# Use this method if you are running this notebook locally (Anaconda)\n", 421 | "flowrate = pd.read_excel(r'Data-FlowRate.xlsx', header=1) " 422 | ] 423 | }, 424 | { 425 | "cell_type": "markdown", 426 | "metadata": { 427 | "iooxa": { 428 | "id": { 429 | "block": "qZxM1TzbxumyXC6lEYWW", 430 | "project": "mmReuqVTAa9JzPpNr22I", 431 | "version": 1 432 | } 433 | } 434 | }, 435 | "source": [ 436 | "### Some useful Pandas methods\n", 437 | "\n", 438 | "We will regularly use a set of Pandas methods to check our data as we munge it\n", 439 | "\n", 440 | "\n", 441 | "Display the first few rows of the dataframe\n", 442 | " \n", 443 | " df.head() _or_ df.head(n) _to show n rows_\n", 444 | "\n", 445 | "Display the last few rows of the dataframe\n", 446 | " \n", 447 | " df.tail() _or_ df.tail(n) _to show n rows_\n", 448 | "\n", 449 | "Useful information like the dataframe shape and the data types\n", 450 | "\n", 451 | " df.info() \n", 452 | "\n", 453 | "A list of the column headers\n", 454 | "\n", 455 | " df.columns \n", 456 | "\n", 457 | "Data types in each column\n", 458 | "\n", 459 | " df.dtypes\n", 460 | "\n", 461 | "Shape of the dataframe\n", 462 | " \n", 463 | " df.shape" 464 | ] 465 | }, 466 | { 467 | "cell_type": "markdown", 468 | "metadata": { 469 | "iooxa": { 470 | "id": { 471 | "block": "pRs67jvk0MAcBD3AvVbL", 472 | "project": "mmReuqVTAa9JzPpNr22I", 473 | "version": 1 474 | } 475 | } 476 | }, 477 | "source": [ 478 | "### 2.1.2 Rename columns\n", 479 | "\n", 480 | "We do this to remove spaces from the column headers. \n", 481 | "\n", 482 | "This step is not compulsory, but the presence of spaces in the column names dictates the code format. Once a column has been created, we can interchangeably use the flowrate.raw_datetime or flowrate\\['raw_datetime'\\] notation. If there are spaces in the column names, the df.column_name notation will not work and we must use the df\\['column_name'\\] format.\n" 483 | ] 484 | }, 485 | { 486 | "cell_type": "code", 487 | "execution_count": null, 488 | "metadata": { 489 | "iooxa": { 490 | "id": { 491 | "block": "Ck6YMVamlovJ0JAVaDbr", 492 | "project": "mmReuqVTAa9JzPpNr22I", 493 | "version": 2 494 | }, 495 | "outputId": { 496 | "block": "OHZeaGR0WZdLclWhdc4O", 497 | "project": "mmReuqVTAa9JzPpNr22I", 498 | "version": 1 499 | } 500 | } 501 | }, 502 | "outputs": [], 503 | "source": [ 504 | "flowrate.columns = ['raw_datetime','flow_Lpm'] \n", 505 | "\n", 506 | "flowrate.head(2)" 507 | ] 508 | }, 509 | { 510 | "cell_type": "markdown", 511 | "metadata": { 512 | "iooxa": { 513 | "id": { 514 | "block": "Mj04KyYGfEwQ2rZNKAm2", 515 | "project": "mmReuqVTAa9JzPpNr22I", 516 | "version": 1 517 | } 518 | } 519 | }, 520 | "source": [ 521 | "### 2.1.3 Convert datetime string into Python datetime \n", 522 | "\n", 523 | "The datetime is in [ISO format](https://en.wikipedia.org/wiki/ISO_8601) and, because it includes the timezone, this is the most globally useful datetime format. \n", 524 | "\n", 525 | "> 2020-12-11T06:00:00+13:00 \n", 526 | "\n", 527 | "where T indicates the time data and +13:00 is the time shift between UTC and New Zealand where the pumps were recording data. " 528 | ] 529 | }, 530 | { 531 | "cell_type": "code", 532 | "execution_count": null, 533 | "metadata": { 534 | "iooxa": { 535 | "id": { 536 | "block": "hnDxNSzTzX6ZCuTyvz1h", 537 | "project": "mmReuqVTAa9JzPpNr22I", 538 | "version": 2 539 | }, 540 | "outputId": { 541 | "block": "wyjsp4o9p6ICo3S4GOpo", 542 | "project": "mmReuqVTAa9JzPpNr22I", 543 | "version": 1 544 | } 545 | } 546 | }, 547 | "outputs": [], 548 | "source": [ 549 | "flowrate.info()" 550 | ] 551 | }, 552 | { 553 | "cell_type": "markdown", 554 | "metadata": { 555 | "iooxa": { 556 | "id": { 557 | "block": "HytLUJyBpSF8it24TH8E", 558 | "project": "mmReuqVTAa9JzPpNr22I", 559 | "version": 1 560 | } 561 | } 562 | }, 563 | "source": [ 564 | "Look at the output above and note that the _raw_datetime_ data type is _object_ and not _datetime_\n", 565 | "\n", 566 | "We will use a two-step process to parse this date:\n", 567 | "\n", 568 | "1. Convert it to a datetime object with the default UTC timezone\n", 569 | "2. Convert the datetime object to local time in New Zealand so it can be combined with the pts tool data" 570 | ] 571 | }, 572 | { 573 | "cell_type": "code", 574 | "execution_count": null, 575 | "metadata": { 576 | "iooxa": { 577 | "id": { 578 | "block": "MyHm58bTP0oSeh1EbL0v", 579 | "project": "mmReuqVTAa9JzPpNr22I", 580 | "version": 2 581 | }, 582 | "outputId": { 583 | "block": "NTMqZWwSzDNjcxDOF54j", 584 | "project": "mmReuqVTAa9JzPpNr22I", 585 | "version": 1 586 | } 587 | } 588 | }, 589 | "outputs": [], 590 | "source": [ 591 | "# use a method that recognises the ISO format to parse the datetime\n", 592 | "\n", 593 | "list = []\n", 594 | "for date in flowrate['raw_datetime']:\n", 595 | " newdate = datetime.fromisoformat(date)\n", 596 | " list.append(newdate)\n", 597 | "flowrate['ISO_datetime'] = list\n", 598 | "\n", 599 | "flowrate.tail(2)" 600 | ] 601 | }, 602 | { 603 | "cell_type": "markdown", 604 | "metadata": { 605 | "iooxa": { 606 | "id": { 607 | "block": "n1FGWrr9U5cTFkTzFauT", 608 | "project": "mmReuqVTAa9JzPpNr22I", 609 | "version": 1 610 | } 611 | } 612 | }, 613 | "source": [ 614 | "The method fromisoformate() is new in version Python 3.7\n", 615 | "\n", 616 | "More information can be found [here](https://docs.python.org/3/library/datetime.html) and [here](https://pythontic.com/datetime/date/fromisoformat)" 617 | ] 618 | }, 619 | { 620 | "cell_type": "code", 621 | "execution_count": null, 622 | "metadata": { 623 | "iooxa": { 624 | "id": { 625 | "block": "HVm6Xfee9rCcySq8FrcM", 626 | "project": "mmReuqVTAa9JzPpNr22I", 627 | "version": 2 628 | }, 629 | "outputId": { 630 | "block": "7ZZ614JH8uV2QNWG0qyC", 631 | "project": "mmReuqVTAa9JzPpNr22I", 632 | "version": 1 633 | } 634 | } 635 | }, 636 | "outputs": [], 637 | "source": [ 638 | "flowrate.info()" 639 | ] 640 | }, 641 | { 642 | "cell_type": "markdown", 643 | "metadata": { 644 | "iooxa": { 645 | "id": { 646 | "block": "W3EiSOZJG930qqEwBN3C", 647 | "project": "mmReuqVTAa9JzPpNr22I", 648 | "version": 1 649 | } 650 | } 651 | }, 652 | "source": [ 653 | "Now we can see that the data type for the ISO_datetime column is a UTC datetime: datetime64\\[ns, UTC+13:00\\]\n", 654 | "\n", 655 | "But we need it in local NZ time to combine with the PTS tool data. Luckily, the datetime.strftime method recognises the timezone in the datetime object and does the conversion for us. " 656 | ] 657 | }, 658 | { 659 | "cell_type": "code", 660 | "execution_count": null, 661 | "metadata": { 662 | "iooxa": { 663 | "id": { 664 | "block": "YpzJ4SDmAJKvvKrq7W9a", 665 | "project": "mmReuqVTAa9JzPpNr22I", 666 | "version": 2 667 | }, 668 | "outputId": { 669 | "block": "L0WoQMprc81JUoXjWc2x", 670 | "project": "mmReuqVTAa9JzPpNr22I", 671 | "version": 1 672 | } 673 | } 674 | }, 675 | "outputs": [], 676 | "source": [ 677 | "list = []\n", 678 | "for date in flowrate.ISO_datetime:\n", 679 | " newdate = pd.to_datetime(datetime.strftime(date,'%Y-%m-%d %H:%M:%S'))\n", 680 | " list.append(newdate)\n", 681 | "flowrate['datetime'] = list\n", 682 | "\n", 683 | "flowrate.head()" 684 | ] 685 | }, 686 | { 687 | "cell_type": "code", 688 | "execution_count": null, 689 | "metadata": { 690 | "iooxa": { 691 | "id": { 692 | "block": "WvDArWfvoM5ImPYB2YW0", 693 | "project": "mmReuqVTAa9JzPpNr22I", 694 | "version": 2 695 | }, 696 | "outputId": { 697 | "block": "lLo98YRejPuSHAM7WjSR", 698 | "project": "mmReuqVTAa9JzPpNr22I", 699 | "version": 1 700 | } 701 | } 702 | }, 703 | "outputs": [], 704 | "source": [ 705 | "flowrate.info()" 706 | ] 707 | }, 708 | { 709 | "cell_type": "markdown", 710 | "metadata": { 711 | "iooxa": { 712 | "id": { 713 | "block": "6BFTl8xFoDEaZNGUic1c", 714 | "project": "mmReuqVTAa9JzPpNr22I", 715 | "version": 1 716 | } 717 | } 718 | }, 719 | "source": [ 720 | "Now we have a Python datetime object that we can combine with the PTS data which does not have timezone information. " 721 | ] 722 | }, 723 | { 724 | "cell_type": "markdown", 725 | "metadata": { 726 | "iooxa": { 727 | "id": { 728 | "block": "soaYNI5DXzIquCDLvcXP", 729 | "project": "mmReuqVTAa9JzPpNr22I", 730 | "version": 1 731 | } 732 | } 733 | }, 734 | "source": [ 735 | "### 2.1.4 Add a flowrate field in our desired units\n", 736 | "\n", 737 | "Conversions for typical geothermal flowrate units:\n", 738 | "\n", 739 | "- L/sec to t/hr 3.6\n", 740 | "- L/min to t/hr 0.060" 741 | ] 742 | }, 743 | { 744 | "cell_type": "code", 745 | "execution_count": null, 746 | "metadata": { 747 | "iooxa": { 748 | "id": { 749 | "block": "MaZLlmJ5MsVfCyoSxsRE", 750 | "project": "mmReuqVTAa9JzPpNr22I", 751 | "version": 2 752 | }, 753 | "outputId": { 754 | "block": "zTLrvR0D11VlqNTG67S5", 755 | "project": "mmReuqVTAa9JzPpNr22I", 756 | "version": 1 757 | } 758 | } 759 | }, 760 | "outputs": [], 761 | "source": [ 762 | "flowrate['flow_tph'] = flowrate.flow_Lpm * 0.060\n", 763 | "\n", 764 | "flowrate.head(2) # 23 to see the non-zero values" 765 | ] 766 | }, 767 | { 768 | "cell_type": "markdown", 769 | "metadata": { 770 | "iooxa": { 771 | "id": { 772 | "block": "EI77qWidXjIrwTdTxzeq", 773 | "project": "mmReuqVTAa9JzPpNr22I", 774 | "version": 1 775 | } 776 | } 777 | }, 778 | "source": [ 779 | "### 2.1.5 Trim excess columns from the dataframe\n", 780 | "\n", 781 | "Now we just drop the columns that we don't want from the dataframe" 782 | ] 783 | }, 784 | { 785 | "cell_type": "code", 786 | "execution_count": null, 787 | "metadata": { 788 | "iooxa": { 789 | "id": { 790 | "block": "Vaaev33GaFciqtS3yEu5", 791 | "project": "mmReuqVTAa9JzPpNr22I", 792 | "version": 2 793 | }, 794 | "outputId": { 795 | "block": "M5MWzVSwPhEXTXioMhMm", 796 | "project": "mmReuqVTAa9JzPpNr22I", 797 | "version": 1 798 | } 799 | } 800 | }, 801 | "outputs": [], 802 | "source": [ 803 | "flowrate.drop(columns = ['raw_datetime', 'flow_Lpm', 'ISO_datetime'], inplace = True)\n", 804 | "\n", 805 | "flowrate.head(2)" 806 | ] 807 | }, 808 | { 809 | "cell_type": "markdown", 810 | "metadata": { 811 | "iooxa": { 812 | "id": { 813 | "block": "tFRyApCAjKo0AU6whGoY", 814 | "project": "mmReuqVTAa9JzPpNr22I", 815 | "version": 1 816 | } 817 | } 818 | }, 819 | "source": [ 820 | "### inplace = True\n", 821 | "\n", 822 | "Each time we do something something like drop columns, we generate a new Pandas dataframe. \n", 823 | "\n", 824 | "The argument inplace=True overwrites the old dataframe with the new one but we could have achieved the same thing with assignment:\n", 825 | "\n", 826 | " flowrate = flowrate.drop(columns = \\['raw_datetime', 'flow_Lpm', 'ISO_datetime'\\])" 827 | ] 828 | }, 829 | { 830 | "cell_type": "markdown", 831 | "metadata": { 832 | "iooxa": { 833 | "id": { 834 | "block": "63SWx9trQANyQYYhYzWM", 835 | "project": "mmReuqVTAa9JzPpNr22I", 836 | "version": 1 837 | } 838 | } 839 | }, 840 | "source": [ 841 | "***\n", 842 | "## 2.2 Pressure, temperature, spinner tool data\n", 843 | "\n", 844 | "We will use a similar process as above to import and munge the pressure, temperature, and spinner (PTS) tool data. However, the PTS dataset includes an addional challenge generated by bad data.\n" 845 | ] 846 | }, 847 | { 848 | "cell_type": "markdown", 849 | "metadata": { 850 | "iooxa": { 851 | "id": { 852 | "block": "WMAJ1mmTOtXFrfSQGFkn", 853 | "project": "mmReuqVTAa9JzPpNr22I", 854 | "version": 1 855 | } 856 | } 857 | }, 858 | "source": [ 859 | "### 2.1.1 Import pts data using Pandas" 860 | ] 861 | }, 862 | { 863 | "cell_type": "code", 864 | "execution_count": null, 865 | "metadata": { 866 | "iooxa": { 867 | "id": { 868 | "block": "hf7FUr31yjWOIPj7fVVa", 869 | "project": "mmReuqVTAa9JzPpNr22I", 870 | "version": 2 871 | }, 872 | "outputId": null 873 | } 874 | }, 875 | "outputs": [], 876 | "source": [ 877 | "# Use this method if you are running this notebook in Google Colab\n", 878 | "# pts = pd.read_excel(r'/content/drive/My Drive/T21-Tutorial-WellTestAnalysis-main/Data-PTS.xlsx') \n", 879 | "\n", 880 | "# Use this method if you are running this notebook locally (Anaconda)\n", 881 | "pts = pd.read_excel(r'Data-PTS.xlsx') " 882 | ] 883 | }, 884 | { 885 | "cell_type": "markdown", 886 | "metadata": { 887 | "iooxa": { 888 | "id": { 889 | "block": "MIqmcq07JC6Ah0BR5cUw", 890 | "project": "mmReuqVTAa9JzPpNr22I", 891 | "version": 1 892 | } 893 | } 894 | }, 895 | "source": [ 896 | "If you run the cell below, you will see that we have a lot of data in the pts dataframe (101294 rows and 26 columns). \n", 897 | "\n", 898 | "Efficient handling of large datasets is one of the reasons why Python is better for these kinds of analysis than excel. " 899 | ] 900 | }, 901 | { 902 | "cell_type": "code", 903 | "execution_count": null, 904 | "metadata": { 905 | "iooxa": { 906 | "id": { 907 | "block": "fJXX8wbHbsydE9x2RLXE", 908 | "project": "mmReuqVTAa9JzPpNr22I", 909 | "version": 2 910 | }, 911 | "outputId": { 912 | "block": "bFwYj2XhS1scgduCBN9t", 913 | "project": "mmReuqVTAa9JzPpNr22I", 914 | "version": 1 915 | } 916 | } 917 | }, 918 | "outputs": [], 919 | "source": [ 920 | "pts.shape" 921 | ] 922 | }, 923 | { 924 | "cell_type": "code", 925 | "execution_count": null, 926 | "metadata": { 927 | "iooxa": { 928 | "id": { 929 | "block": "OdgCM1r85MBVNnJYmqlW", 930 | "project": "mmReuqVTAa9JzPpNr22I", 931 | "version": 2 932 | }, 933 | "outputId": { 934 | "block": "eNdu1lweTTSSL0DPKCiy", 935 | "project": "mmReuqVTAa9JzPpNr22I", 936 | "version": 1 937 | } 938 | } 939 | }, 940 | "outputs": [], 941 | "source": [ 942 | "pts.head(3)" 943 | ] 944 | }, 945 | { 946 | "cell_type": "markdown", 947 | "metadata": { 948 | "iooxa": { 949 | "id": { 950 | "block": "nkYBNZI590igho1o3vyk", 951 | "project": "mmReuqVTAa9JzPpNr22I", 952 | "version": 1 953 | } 954 | } 955 | }, 956 | "source": [ 957 | "### 2.2.2 Rename selected column headders\n", 958 | "\n", 959 | "The dataframe has a row of units below the header row that we need to remove. But because we want to know the units of our data, we will rename key columns to remove spaces and include units before dropping the row containing the units." 960 | ] 961 | }, 962 | { 963 | "cell_type": "code", 964 | "execution_count": null, 965 | "metadata": { 966 | "iooxa": { 967 | "id": { 968 | "block": "3TBoCJrLMKA7WHfBZeCw", 969 | "project": "mmReuqVTAa9JzPpNr22I", 970 | "version": 2 971 | }, 972 | "outputId": { 973 | "block": "fnFVhFuFev20Ibcv5SjU", 974 | "project": "mmReuqVTAa9JzPpNr22I", 975 | "version": 1 976 | } 977 | } 978 | }, 979 | "outputs": [], 980 | "source": [ 981 | "# dictionary method for renaming some columns\n", 982 | "dict = {\n", 983 | " 'DEPTH':'depth_m',\n", 984 | " 'SPEED': 'speed_mps',\n", 985 | " 'Cable Weight': 'cweight_kg',\n", 986 | " 'WHP': 'whp_barg',\n", 987 | " 'Temperature': 'temp_degC',\n", 988 | " 'Pressure': 'pressure_bara',\n", 989 | " 'Frequency': 'frequency_hz'\n", 990 | "}\n", 991 | "\n", 992 | "pts.rename(columns=dict, inplace=True)\n", 993 | "\n", 994 | "pts.head(3)" 995 | ] 996 | }, 997 | { 998 | "cell_type": "markdown", 999 | "metadata": { 1000 | "iooxa": { 1001 | "id": { 1002 | "block": "NVriHzL7wjFkFtjeke5c", 1003 | "project": "mmReuqVTAa9JzPpNr22I", 1004 | "version": 1 1005 | } 1006 | } 1007 | }, 1008 | "source": [ 1009 | "### 2.2.3 Drop units row\n", 1010 | "\n", 1011 | "Note that we are using the index value to drop the row of units and then we remake the index. Therefore, if we run this cell over and over, it will keep dropping the first row of data. \n", 1012 | "\n", 1013 | "If you are unsure what state your dataframe is in or how many times you have run the cell below, it is useful to run the process from the start again (i.e., go back to step _2.1.1 Import pts data using Pandas_ and work down from there). " 1014 | ] 1015 | }, 1016 | { 1017 | "cell_type": "code", 1018 | "execution_count": null, 1019 | "metadata": { 1020 | "iooxa": { 1021 | "id": { 1022 | "block": "y0gzm2QBEHr5yGDvnNO0", 1023 | "project": "mmReuqVTAa9JzPpNr22I", 1024 | "version": 1 1025 | }, 1026 | "outputId": null 1027 | } 1028 | }, 1029 | "outputs": [], 1030 | "source": [ 1031 | "# drop units row using the index value of that row\n", 1032 | "pts.drop(0, inplace=True)\n", 1033 | "\n", 1034 | "# re-index the dataframe\n", 1035 | "pts.reset_index(drop=True, inplace=True)\n", 1036 | "\n", 1037 | "# WARNING if you run this cell again, it will drop the first line of data" 1038 | ] 1039 | }, 1040 | { 1041 | "cell_type": "code", 1042 | "execution_count": null, 1043 | "metadata": { 1044 | "iooxa": { 1045 | "id": { 1046 | "block": "8sp0DJpYqDxS3LcIJZW6", 1047 | "project": "mmReuqVTAa9JzPpNr22I", 1048 | "version": 2 1049 | }, 1050 | "outputId": { 1051 | "block": "fbtwJRhlyDRYz3z1OwD9", 1052 | "project": "mmReuqVTAa9JzPpNr22I", 1053 | "version": 1 1054 | } 1055 | } 1056 | }, 1057 | "outputs": [], 1058 | "source": [ 1059 | "pts.head(2)" 1060 | ] 1061 | }, 1062 | { 1063 | "cell_type": "markdown", 1064 | "metadata": { 1065 | "iooxa": { 1066 | "id": { 1067 | "block": "5HI5j81mzcGDbjCty830", 1068 | "project": "mmReuqVTAa9JzPpNr22I", 1069 | "version": 1 1070 | } 1071 | } 1072 | }, 1073 | "source": [ 1074 | "### 2.2.4 Recognise bad datetime data & how to handle excel datetime floats\n", 1075 | "\n", 1076 | "If you investigate the _Time_ column in our pts dataframe, you'll find that the hours are all set to 00 despite the test running for nearly 12 hours and starting at around 9 in the morning. Unfortunately, these kinds of issues with bad data are common and they are why it's important to QA/QC data when it is acquired. \n", 1077 | "\n", 1078 | "We will (reluctantly) use the excel timestamp to generate a new datetime.\n", 1079 | "\n", 1080 | "An excel timestamp is a floating-point number that reflects the number of days (or part thereof) since a specified start day. There are two systems that start counting from different dates (1900 and 1904) and some fundamental bugs in the way they are implemented. The start date is stored in the excel workbook and we need to know which system is used to correctly generate a datetime. Check out [this documentation](https://openpyxl.readthedocs.io/en/stable/datetime.html) for more information on the challenges of excel timestamps. **The short version is that we should avoid using excel float time whenever possible.** Unfortunately, it is not possible here because of the bad data.\n", 1081 | "\n", 1082 | "Because we have an xlsx file, will use the openpyxl package to determine the start date in the workbook and parse the excel float to datetime. If you have an xls file, then the xlrd module may be used to achieve the same result." 1083 | ] 1084 | }, 1085 | { 1086 | "cell_type": "code", 1087 | "execution_count": null, 1088 | "metadata": { 1089 | "iooxa": { 1090 | "id": { 1091 | "block": "nW1vhrhloQP4Yoi0ArA4", 1092 | "project": "mmReuqVTAa9JzPpNr22I", 1093 | "version": 2 1094 | }, 1095 | "outputId": { 1096 | "block": "omH3UpRjD6TndRrUDrk3", 1097 | "project": "mmReuqVTAa9JzPpNr22I", 1098 | "version": 1 1099 | } 1100 | } 1101 | }, 1102 | "outputs": [], 1103 | "source": [ 1104 | "# first check the start day system in the workbook\n", 1105 | "\n", 1106 | "# Use this method if you are in Google Colab\n", 1107 | "# wb = openpyxl.load_workbook(filename=r'/content/drive/My Drive/T21-Tutorial-WellTestAnalysis-main/Data-PTS.xlsx')\n", 1108 | "\n", 1109 | "# Use this method if you are running the notebook locally in Anaconda\n", 1110 | "wb = openpyxl.load_workbook(filename=r'Data-PTS.xlsx')\n", 1111 | "\n", 1112 | "if wb.epoch == openpyxl.utils.datetime.CALENDAR_WINDOWS_1900: \n", 1113 | " print(\"This workbook is using the 1900 date system.\")\n", 1114 | "if wb.epoch == openpyxl.utils.datetime.CALENDAR_MAC_1904: \n", 1115 | " print(\"This workbook is using the 1904 date system.\")\n" 1116 | ] 1117 | }, 1118 | { 1119 | "cell_type": "markdown", 1120 | "metadata": { 1121 | "iooxa": { 1122 | "id": { 1123 | "block": "t07xNz7W9KTVzxk9sSrb", 1124 | "project": "mmReuqVTAa9JzPpNr22I", 1125 | "version": 1 1126 | } 1127 | } 1128 | }, 1129 | "source": [ 1130 | "We can see in the printed statement above that we have the 1900 date system. Therefore, we can use the default arguments in the from_excel method in the openpyxl utilities. Refer to [the docs](https://openpyxl.readthedocs.io/en/stable/api/openpyxl.utils.datetime.html) for more information." 1131 | ] 1132 | }, 1133 | { 1134 | "cell_type": "code", 1135 | "execution_count": null, 1136 | "metadata": { 1137 | "iooxa": { 1138 | "id": { 1139 | "block": "YClw7gmigjTCS9W0DmDF", 1140 | "project": "mmReuqVTAa9JzPpNr22I", 1141 | "version": 1 1142 | }, 1143 | "outputId": null 1144 | } 1145 | }, 1146 | "outputs": [], 1147 | "source": [ 1148 | "list = []\n", 1149 | "for date in pts.Timestamp:\n", 1150 | " newdate = openpyxl.utils.datetime.from_excel(date)\n", 1151 | " list.append(newdate)\n", 1152 | "pts['datetime'] = list" 1153 | ] 1154 | }, 1155 | { 1156 | "cell_type": "code", 1157 | "execution_count": null, 1158 | "metadata": { 1159 | "iooxa": { 1160 | "id": { 1161 | "block": "rIKpo6SSbQw1qs8n1QBa", 1162 | "project": "mmReuqVTAa9JzPpNr22I", 1163 | "version": 2 1164 | }, 1165 | "outputId": { 1166 | "block": "z0ESi4Y3Ixl714DzupuK", 1167 | "project": "mmReuqVTAa9JzPpNr22I", 1168 | "version": 1 1169 | } 1170 | } 1171 | }, 1172 | "outputs": [], 1173 | "source": [ 1174 | "pts.head(3)" 1175 | ] 1176 | }, 1177 | { 1178 | "cell_type": "markdown", 1179 | "metadata": { 1180 | "iooxa": { 1181 | "id": { 1182 | "block": "fbu06pwlVpXJElJoEVYN", 1183 | "project": "mmReuqVTAa9JzPpNr22I", 1184 | "version": 1 1185 | } 1186 | } 1187 | }, 1188 | "source": [ 1189 | "### 2.2.5 Drop unwanted columns from dataframe\n", 1190 | "\n", 1191 | "There are a lot of columns we do not need for our analysis, so we will drop these\n" 1192 | ] 1193 | }, 1194 | { 1195 | "cell_type": "code", 1196 | "execution_count": null, 1197 | "metadata": { 1198 | "iooxa": { 1199 | "id": { 1200 | "block": "OpPeMqBmsoBWPIfUwl6C", 1201 | "project": "mmReuqVTAa9JzPpNr22I", 1202 | "version": 2 1203 | }, 1204 | "outputId": { 1205 | "block": "1Yc5PQ9b6p4bzJPy9op7", 1206 | "project": "mmReuqVTAa9JzPpNr22I", 1207 | "version": 1 1208 | } 1209 | } 1210 | }, 1211 | "outputs": [], 1212 | "source": [ 1213 | "pts.columns" 1214 | ] 1215 | }, 1216 | { 1217 | "cell_type": "code", 1218 | "execution_count": null, 1219 | "metadata": { 1220 | "iooxa": { 1221 | "id": { 1222 | "block": "WE10NayG3VA9MsIXcf1n", 1223 | "project": "mmReuqVTAa9JzPpNr22I", 1224 | "version": 1 1225 | }, 1226 | "outputId": null 1227 | } 1228 | }, 1229 | "outputs": [], 1230 | "source": [ 1231 | "pts.drop(columns = ['Date', 'Time', 'Timestamp','Reed 0',\n", 1232 | " 'Reed 1', 'Reed 2', 'Reed 3', 'Battery Voltage', \n", 1233 | " 'PRT Ref Voltage','SGS Voltage', 'Internal Temp 1', \n", 1234 | " 'Internal Temp 2', 'Internal Temp 3','Cal Temp', \n", 1235 | " 'Error Code 1', 'Error Code 2', 'Error Code 3',\n", 1236 | " 'Records Saved', 'Bad Pages',], inplace = True)" 1237 | ] 1238 | }, 1239 | { 1240 | "cell_type": "code", 1241 | "execution_count": null, 1242 | "metadata": { 1243 | "iooxa": { 1244 | "id": { 1245 | "block": "KEHcfVKDdWIgeF75xFSy", 1246 | "project": "mmReuqVTAa9JzPpNr22I", 1247 | "version": 2 1248 | }, 1249 | "outputId": { 1250 | "block": "8Ji3lJ15kErZLF1sQ1HM", 1251 | "project": "mmReuqVTAa9JzPpNr22I", 1252 | "version": 1 1253 | } 1254 | } 1255 | }, 1256 | "outputs": [], 1257 | "source": [ 1258 | "pts.head(3)" 1259 | ] 1260 | }, 1261 | { 1262 | "cell_type": "markdown", 1263 | "metadata": { 1264 | "iooxa": { 1265 | "id": { 1266 | "block": "TZOv6NiHLadfDdo05J7M", 1267 | "project": "mmReuqVTAa9JzPpNr22I", 1268 | "version": 1 1269 | } 1270 | } 1271 | }, 1272 | "source": [ 1273 | "### 2.2.6 Correct the data types\n", 1274 | "\n", 1275 | "Run the cell below and note that the data types are 'object'. This is because we had a mix of text and numeric when importing these data (remember the units row). We convert the dtype to numeric using the pd.to_numeric method." 1276 | ] 1277 | }, 1278 | { 1279 | "cell_type": "code", 1280 | "execution_count": null, 1281 | "metadata": { 1282 | "iooxa": { 1283 | "id": { 1284 | "block": "VrTqGcBLQkTRAtyEJBNE", 1285 | "project": "mmReuqVTAa9JzPpNr22I", 1286 | "version": 2 1287 | }, 1288 | "outputId": { 1289 | "block": "3I55AzB3dtQDECrcD0dr", 1290 | "project": "mmReuqVTAa9JzPpNr22I", 1291 | "version": 1 1292 | } 1293 | } 1294 | }, 1295 | "outputs": [], 1296 | "source": [ 1297 | "pts.dtypes" 1298 | ] 1299 | }, 1300 | { 1301 | "cell_type": "code", 1302 | "execution_count": null, 1303 | "metadata": { 1304 | "iooxa": { 1305 | "id": { 1306 | "block": "ast0rBbFWN2hY4a5VVJp", 1307 | "project": "mmReuqVTAa9JzPpNr22I", 1308 | "version": 2 1309 | }, 1310 | "outputId": { 1311 | "block": "YZeDCTXqqut22C8b6b6Z", 1312 | "project": "mmReuqVTAa9JzPpNr22I", 1313 | "version": 1 1314 | } 1315 | } 1316 | }, 1317 | "outputs": [], 1318 | "source": [ 1319 | "pts[\n", 1320 | " ['depth_m', 'speed_mps','cweight_kg','whp_barg','temp_degC','pressure_bara','frequency_hz']\n", 1321 | "] = pts[\n", 1322 | " ['depth_m','speed_mps','cweight_kg','whp_barg','temp_degC','pressure_bara','frequency_hz']\n", 1323 | " ].apply(pd.to_numeric)\n", 1324 | "\n", 1325 | "pts.dtypes" 1326 | ] 1327 | }, 1328 | { 1329 | "cell_type": "code", 1330 | "execution_count": null, 1331 | "metadata": { 1332 | "iooxa": { 1333 | "id": { 1334 | "block": "a1xftRDFvOnbT1c865Lw", 1335 | "project": "mmReuqVTAa9JzPpNr22I", 1336 | "version": 2 1337 | }, 1338 | "outputId": { 1339 | "block": "pEEK0fMlPNYlQoNCp5Ck", 1340 | "project": "mmReuqVTAa9JzPpNr22I", 1341 | "version": 1 1342 | } 1343 | } 1344 | }, 1345 | "outputs": [], 1346 | "source": [ 1347 | "pts.columns" 1348 | ] 1349 | }, 1350 | { 1351 | "cell_type": "markdown", 1352 | "metadata": { 1353 | "iooxa": { 1354 | "id": { 1355 | "block": "bMcZOT4UQCY4tvH8BS6p", 1356 | "project": "mmReuqVTAa9JzPpNr22I", 1357 | "version": 1 1358 | } 1359 | } 1360 | }, 1361 | "source": [ 1362 | "Because methods like ipywidget interactive plots only work with floats, we make and append the time in seconds that has elapsed since the start of the test." 1363 | ] 1364 | }, 1365 | { 1366 | "cell_type": "code", 1367 | "execution_count": null, 1368 | "metadata": { 1369 | "iooxa": { 1370 | "id": { 1371 | "block": "nCM6b2FwsVWFFKSfJyRT", 1372 | "project": "mmReuqVTAa9JzPpNr22I", 1373 | "version": 1 1374 | }, 1375 | "outputId": null 1376 | } 1377 | }, 1378 | "outputs": [], 1379 | "source": [ 1380 | "def timedelta_seconds(dataframe_col, test_start):\n", 1381 | " '''\n", 1382 | " Make a float in seconds since the start of the test\n", 1383 | "\n", 1384 | " args: dataframe_col: dataframe column containing datetime objects\n", 1385 | " test_start: test start time formatted '2020-12-11 09:00:00'\n", 1386 | "\n", 1387 | " returns: float in seconds since the start of the test\n", 1388 | " '''\n", 1389 | " test_start_datetime = pd.to_datetime(test_start)\n", 1390 | " list = []\n", 1391 | " for datetime in dataframe_col:\n", 1392 | " time_delta = datetime - test_start_datetime\n", 1393 | " seconds = time_delta.total_seconds()\n", 1394 | " list.append(seconds)\n", 1395 | " return list\n", 1396 | "\n", 1397 | "test_start_datetime = '2020-12-11 09:26:44.448'\n", 1398 | "\n", 1399 | "flowrate['timedelta_sec'] = timedelta_seconds(flowrate.datetime, test_start_datetime)\n", 1400 | "pts['timedelta_sec'] = timedelta_seconds(pts.datetime, test_start_datetime)\n" 1401 | ] 1402 | }, 1403 | { 1404 | "cell_type": "markdown", 1405 | "metadata": { 1406 | "iooxa": { 1407 | "id": { 1408 | "block": "i5Od3lyWxLQr5loKRHB1", 1409 | "project": "mmReuqVTAa9JzPpNr22I", 1410 | "version": 1 1411 | } 1412 | } 1413 | }, 1414 | "source": [ 1415 | "***\n", 1416 | "\n", 1417 | "## We are finished importing and munging the data!\n", 1418 | "\n", 1419 | "We now have flowrate and pts dataframes that we can work with for the rest of our analysis. \n", 1420 | "\n", 1421 | "The steps taken above to import and munge the data are wrapped into custom functions that will be at the top of each notebook.\n", 1422 | "\n", 1423 | "In the rest of this tutorial, we will call these functions rather than stepping through the method again. " 1424 | ] 1425 | }, 1426 | { 1427 | "cell_type": "markdown", 1428 | "metadata": { 1429 | "iooxa": { 1430 | "id": { 1431 | "block": "inbEDirfTM61mqUL8Dbl", 1432 | "project": "mmReuqVTAa9JzPpNr22I", 1433 | "version": 1 1434 | } 1435 | } 1436 | }, 1437 | "source": [ 1438 | "***\n", 1439 | "\n", 1440 | "# 3. Overview of the completion test data\n" 1441 | ] 1442 | }, 1443 | { 1444 | "cell_type": "code", 1445 | "execution_count": null, 1446 | "metadata": { 1447 | "iooxa": { 1448 | "id": { 1449 | "block": "oIjYzYwSCbZ6mGWyH4fE", 1450 | "project": "mmReuqVTAa9JzPpNr22I", 1451 | "version": 2 1452 | }, 1453 | "outputId": { 1454 | "block": "Uw3t0K484Z068mzpJzrw", 1455 | "project": "mmReuqVTAa9JzPpNr22I", 1456 | "version": 1 1457 | } 1458 | } 1459 | }, 1460 | "outputs": [], 1461 | "source": [ 1462 | "import matplotlib.pyplot as plt \n", 1463 | "import matplotlib.dates as mdates\n", 1464 | "\n", 1465 | "fig, (ax1, ax2, ax3, ax4) = plt.subplots(4, 1,figsize=(18,12),sharex=True)\n", 1466 | "\n", 1467 | "ax1.plot(flowrate.datetime, flowrate.flow_tph, label='Surface pump flowrate', \n", 1468 | " c='k', linewidth=0.8, marker='.')\n", 1469 | "ax1.set_ylabel('Surface flowrate [t/hr]')\n", 1470 | "ax1.set_ylim(0,150)\n", 1471 | "\n", 1472 | "for ax in [ax2, ax3, ax4]:\n", 1473 | " ax.plot(pts.datetime, pts.depth_m, label='PTS tool depth', \n", 1474 | " c='k', linewidth=1.5)\n", 1475 | " ax.set_ylabel('PTS tool depth [m]')\n", 1476 | " ax.set_ylim(1000,0)\n", 1477 | "\n", 1478 | "ax4.set_xlabel('Time [hh:mm]')\n", 1479 | "ax4.xaxis.set_major_formatter(mdates.DateFormatter('%H:%M'))\n", 1480 | " \n", 1481 | "injectivity_index = [ # purple\n", 1482 | " (pd.to_datetime('2020-12-11 10:28:00'),pd.to_datetime('2020-12-11 10:30:00')),\n", 1483 | " (pd.to_datetime('2020-12-11 12:51:00'),pd.to_datetime('2020-12-11 12:53:00')),\n", 1484 | " (pd.to_datetime('2020-12-11 15:20:00'),pd.to_datetime('2020-12-11 15:23:00')), # pumps shut off too soon\n", 1485 | " ]\n", 1486 | "injectivity_index_alt = [ # light purple \n", 1487 | " (pd.to_datetime('2020-12-11 14:38:00'),pd.to_datetime('2020-12-11 14:40:00')) # alternate used because of pump issue\n", 1488 | " ]\n", 1489 | "pressure_transient = [ # green\n", 1490 | " (pd.to_datetime('2020-12-11 10:36:00'),pd.to_datetime('2020-12-11 12:10:00')),\n", 1491 | " (pd.to_datetime('2020-12-11 12:55:15'),pd.to_datetime('2020-12-11 14:40:00')), \n", 1492 | " ]\n", 1493 | "temperature_profile = [ # red\n", 1494 | " (pd.to_datetime('2020-12-11 10:03:00'),pd.to_datetime('2020-12-11 10:11:30')),\n", 1495 | " (pd.to_datetime('2020-12-11 12:28:30'),pd.to_datetime('2020-12-11 12:36:00')),\n", 1496 | " (pd.to_datetime('2020-12-11 14:57:30'),pd.to_datetime('2020-12-11 15:05:00')),\n", 1497 | " (pd.to_datetime('2020-12-11 16:09:00'),pd.to_datetime('2020-12-11 16:15:30'))\n", 1498 | " ]\n", 1499 | "fluid_velocity = [ # yellow\n", 1500 | " (pd.to_datetime('2020-12-11 09:46:00'),pd.to_datetime('2020-12-11 10:25:00')),\n", 1501 | " (pd.to_datetime('2020-12-11 12:10:30'),pd.to_datetime('2020-12-11 12:51:00')),\n", 1502 | " (pd.to_datetime('2020-12-11 14:40:30'),pd.to_datetime('2020-12-11 15:19:00')),\n", 1503 | " ]\n", 1504 | "\n", 1505 | "ax1.set_title('Pressure transient analysis (not included in this tutorial)')\n", 1506 | "for top, bottom in pressure_transient:\n", 1507 | " ax1.axvspan(top, bottom, color='#369a33', alpha=0.3) # green\n", 1508 | "\n", 1509 | "ax2.set_title('Temperature profile analysis - Notebook 2')\n", 1510 | "for top, bottom in temperature_profile:\n", 1511 | " ax2.axvspan(top, bottom, color='#9a3336', alpha = 0.5) # red \n", 1512 | "\n", 1513 | "ax3.set_title('Determine well capacity (injectivity index) - Notebook 3')\n", 1514 | "for top, bottom in injectivity_index:\n", 1515 | " ax3.axvspan(top, bottom, color='#33369a', alpha=.6) # purple\n", 1516 | "\n", 1517 | "for top, bottom in injectivity_index_alt:\n", 1518 | " ax3.axvspan(top, bottom, color='#33369a', alpha=.3) # light purple\n", 1519 | " \n", 1520 | "ax4.set_title('Feed zones from fluid velocity analysis - Notebook 4')\n", 1521 | "for top, bottom in fluid_velocity:\n", 1522 | " ax4.axvspan(top, bottom, color='#ffc000', alpha=.3) # yellow \n", 1523 | "\n", 1524 | "for ax in [ax1, ax2, ax3, ax4]:\n", 1525 | " ax.grid()\n", 1526 | " \n", 1527 | "plt.show();" 1528 | ] 1529 | }, 1530 | { 1531 | "cell_type": "markdown", 1532 | "metadata": { 1533 | "iooxa": { 1534 | "id": { 1535 | "block": "Edapbj09achzt97OCl4D", 1536 | "project": "mmReuqVTAa9JzPpNr22I", 1537 | "version": 1 1538 | } 1539 | } 1540 | }, 1541 | "source": [ 1542 | "#### Overview of the completion test (plot above)\n", 1543 | "\n", 1544 | "The highlighted areas on the plot above are the data we will extract for analysis during this tutorial. \n", 1545 | "\n", 1546 | "The zig-zags in the PTS tool depth show the tool going up and down inside the well. Injectivity index (well capacity) uses pressure and flow rate just after these tool passes are complete and before the pump rate is changed (dark purple). However, after the third set of passes (i.e., during the third flowrate) the pumps were shut off too soon, so we will use the pressure and flow rate just prior to the passes (light purple).\n", 1547 | "\n", 1548 | "#### Use the plot below to explore the entire dataset" 1549 | ] 1550 | }, 1551 | { 1552 | "cell_type": "code", 1553 | "execution_count": null, 1554 | "metadata": { 1555 | "iooxa": { 1556 | "id": { 1557 | "block": "uoigj1oiuRgDYiaeXzDj", 1558 | "project": "mmReuqVTAa9JzPpNr22I", 1559 | "version": 1 1560 | }, 1561 | "outputId": null 1562 | } 1563 | }, 1564 | "outputs": [], 1565 | "source": [ 1566 | "def overview_fig(pts_df,flowrate_df,title=''):\n", 1567 | " fig, (ax1, ax2, ax3, ax4, ax5, ax6) = plt.subplots(6, 1,figsize=(10,15),sharex=True)\n", 1568 | " ax1.set_title(title,y=1.1,fontsize=15)\n", 1569 | "\n", 1570 | " ax1.plot(flowrate_df.datetime, flowrate_df.flow_tph, label='Surface pump flowrate', \n", 1571 | " c='k', linewidth=0.8, marker='.')\n", 1572 | " ax1.set_ylabel('Surface flowrate [t/hr]')\n", 1573 | " ax1.set_ylim(0,150)\n", 1574 | " \n", 1575 | " ax2.plot(pts_df.datetime, pts_df.depth_m, label='PTS tool depth', \n", 1576 | " c='k', linewidth=0.8)\n", 1577 | " ax2.set_ylabel('PTS tool depth [m]')\n", 1578 | " ax2.set_ylim(1000,0)\n", 1579 | " \n", 1580 | " ax3.plot(pts_df.datetime, pts_df.pressure_bara, label='PTS pressure', \n", 1581 | " c='tab:blue', linewidth=0.8)\n", 1582 | " ax3.set_ylabel('PTS pressure [bara]')\n", 1583 | " \n", 1584 | " ax4.plot(pts_df.datetime, pts_df.temp_degC, label='PTS temperature', \n", 1585 | " c='tab:red', linewidth=0.8)\n", 1586 | " ax4.set_ylabel('PTS temperature')\n", 1587 | " \n", 1588 | " ax5.plot(pts_df.datetime, pts_df.frequency_hz, label='PTS impeller frequency', \n", 1589 | " c='tab:green', linewidth=0.8)\n", 1590 | " ax5.set_ylim(-30,30)\n", 1591 | " ax5.set_ylabel('PTS impeller frequency [hz]')\n", 1592 | " # 1 hz = 60 rpm\n", 1593 | "\n", 1594 | " ax6.plot(pts_df.datetime, pts_df.speed_mps, label='PTS tool speed', \n", 1595 | " c='tab:orange', linewidth=0.8)\n", 1596 | " ax6.set_ylim(-2,2)\n", 1597 | " ax6.set_ylabel('PTS tool speed [mps]')\n", 1598 | " \n", 1599 | " ax6.set_xlabel('Time [hh:mm]')\n", 1600 | " plt.gca().xaxis.set_major_formatter(mdates.DateFormatter('%H:%M'))\n", 1601 | " \n", 1602 | " for ax in [ax1,ax2,ax3,ax4,ax5,ax6]:\n", 1603 | " ax.grid()\n", 1604 | " \n", 1605 | " return plt" 1606 | ] 1607 | }, 1608 | { 1609 | "cell_type": "code", 1610 | "execution_count": null, 1611 | "metadata": { 1612 | "iooxa": { 1613 | "id": { 1614 | "block": "A4zY8RFp07RNhFRXswcP", 1615 | "project": "mmReuqVTAa9JzPpNr22I", 1616 | "version": 2 1617 | }, 1618 | "outputId": { 1619 | "block": "YteUPsYCdASYQu8NdNvo", 1620 | "project": "mmReuqVTAa9JzPpNr22I", 1621 | "version": 1 1622 | } 1623 | } 1624 | }, 1625 | "outputs": [], 1626 | "source": [ 1627 | "overview_fig(pts,flowrate);" 1628 | ] 1629 | }, 1630 | { 1631 | "cell_type": "markdown", 1632 | "metadata": { 1633 | "iooxa": { 1634 | "id": { 1635 | "block": "xad220gtnyM9R5vYGPFG", 1636 | "project": "mmReuqVTAa9JzPpNr22I", 1637 | "version": 1 1638 | } 1639 | } 1640 | }, 1641 | "source": [ 1642 | "Note that I sometimes use a ; at the end of the code in the cell to suppress extra output, as is has been done above." 1643 | ] 1644 | }, 1645 | { 1646 | "cell_type": "markdown", 1647 | "metadata": { 1648 | "iooxa": { 1649 | "id": { 1650 | "block": "F3gJD6r2sfnWV5gdoJJF", 1651 | "project": "mmReuqVTAa9JzPpNr22I", 1652 | "version": 1 1653 | } 1654 | } 1655 | }, 1656 | "source": [ 1657 | "***\n", 1658 | "\n", 1659 | "### Cited references\n", 1660 | "\n", 1661 | "Kaya, E., Zarrouk, S.J., and O’Sullivan, M.J. (2011): Reinjection in geothermal fields: a review of worldwide experience. Renew. Sustain. Energy Rev. 15 (1), 47-68. \n", 1662 | "\n", 1663 | "Zarrouk, S.J. and McLean, K. (2019): Geothermal well test analysis: fundamentals, applications, and advanced techniques. 1st edition, Elsevier. \n", 1664 | "\n", 1665 | "***\n", 1666 | "\n", 1667 | "© 2021 [Irene Wallis](https://www.cubicearth.nz/) and [Katie McLean](https://www.linkedin.com/in/katie-mclean-25994315/) \n", 1668 | "\n", 1669 | "Licensed under the Apache License, Version 2.0\n", 1670 | "\n", 1671 | "\n", 1672 | "***" 1673 | ] 1674 | }, 1675 | { 1676 | "cell_type": "code", 1677 | "execution_count": null, 1678 | "metadata": { 1679 | "iooxa": null 1680 | }, 1681 | "outputs": [], 1682 | "source": [] 1683 | } 1684 | ], 1685 | "metadata": { 1686 | "iooxa": { 1687 | "id": { 1688 | "block": "xsmu9oKVnHvU3tpyB9XG", 1689 | "project": "mmReuqVTAa9JzPpNr22I", 1690 | "version": 2 1691 | } 1692 | }, 1693 | "kernelspec": { 1694 | "display_name": "Python 3", 1695 | "language": "python", 1696 | "name": "python3" 1697 | }, 1698 | "language_info": { 1699 | "codemirror_mode": { 1700 | "name": "ipython", 1701 | "version": 3 1702 | }, 1703 | "file_extension": ".py", 1704 | "mimetype": "text/x-python", 1705 | "name": "python", 1706 | "nbconvert_exporter": "python", 1707 | "pygments_lexer": "ipython3", 1708 | "version": "3.7.9" 1709 | } 1710 | }, 1711 | "nbformat": 4, 1712 | "nbformat_minor": 4 1713 | } 1714 | -------------------------------------------------------------------------------- /2-temperature.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": { 6 | "iooxa": { 7 | "id": { 8 | "block": "u4FWgTc5O538ZFPdjtDW", 9 | "project": "mmReuqVTAa9JzPpNr22I", 10 | "version": 1 11 | } 12 | } 13 | }, 14 | "source": [ 15 | "***\n", 16 | "\n", 17 | "# Geothermal Well Test Analysis with Python\n", 18 | "### Notebook 2: Temperature log data extraction and interpretation\n", 19 | "#### Irene Wallis and Katie McLean \n", 20 | "#### Software Underground, Transform 2021\n", 21 | "***" 22 | ] 23 | }, 24 | { 25 | "cell_type": "markdown", 26 | "metadata": { 27 | "iooxa": { 28 | "id": { 29 | "block": "hFBY2GRSBmjOgVYlPkdF", 30 | "project": "mmReuqVTAa9JzPpNr22I", 31 | "version": 1 32 | } 33 | } 34 | }, 35 | "source": [ 36 | "### Google Colab Setup\n", 37 | "\n", 38 | "If you are using Google Colab to run this notebook, we assume you have already followed the Google Colab setup steps outlined [here](https://github.com/ICWallis/T21-Tutorial-WellTestAnalysis).\n", 39 | "\n", 40 | "Because we are importing data, we need to \"mount your Google Drive\", which is where we tell this notebook to look for the data files. You will need to mount the Google Drive into each notebook. \n", 41 | "\n", 42 | "1. Run the cell below if you are in Google Colab. If you are not in Google Colab, running the cell below will just return an error that says \"No module named 'google'\". If you get a Google Colab error that says \"Unrecognised runtime 'geothrm'; defaulting to 'python3' Notebook settings\", just ignore it. \n", 43 | "\n", 44 | "2. Follow the link generated by running this code. That link will ask you to sign in to your google account (use the one where you have saved these tutorial materials in) and to allow this notebook access to your google drive. \n", 45 | "\n", 46 | "3. Completing step 2 above will generate a code. Copy this code, paste below where it says \"Enter your authorization code:\", and press ENTER. \n", 47 | "\n", 48 | "Congratulations, this notebook can now import data!" 49 | ] 50 | }, 51 | { 52 | "cell_type": "code", 53 | "execution_count": null, 54 | "metadata": { 55 | "iooxa": { 56 | "id": { 57 | "block": "T6OaduxDfOxZPNsFYVsk", 58 | "project": "mmReuqVTAa9JzPpNr22I", 59 | "version": 1 60 | }, 61 | "outputId": null 62 | } 63 | }, 64 | "outputs": [], 65 | "source": [ 66 | "from google.colab import drive\n", 67 | "drive.mount('/content/drive')" 68 | ] 69 | }, 70 | { 71 | "cell_type": "markdown", 72 | "metadata": { 73 | "iooxa": { 74 | "id": { 75 | "block": "VICvvm0wcV1LqPQ2hGJI", 76 | "project": "mmReuqVTAa9JzPpNr22I", 77 | "version": 1 78 | } 79 | } 80 | }, 81 | "source": [ 82 | "***\n", 83 | "\n", 84 | "# 4. Introduction to temperature logs\n", 85 | "\n", 86 | "Given that geothermal development seeks to utilise the heat energy of the Earth, temperature logs are arguably the most important data we acquire. In this tutorial, we will give a short introduction to handling these data and how they are interpreted.\n", 87 | "\n", 88 | "## 4.1 Uses of temperature logs in geothermal exploration and development\n", 89 | "\n", 90 | "- Feed zone identification\n", 91 | "- Interpreting well/reservoir events (e.g., heating after drilling or the initiation of a cool reservoir down flow)\n", 92 | "- Constraining conceptual models (natural state temperature profiles)\n", 93 | "- Calibrating reservoir models\n", 94 | "\n", 95 | "## 4.2 Key considerations when evaluating temperature data\n", 96 | "\n", 97 | "- The well status (shut, injection, flowing) while the log was acquired \n", 98 | "- When the temperature log was acquired (e.g., the time that has elapsed since drilling or since the well was flowed) \n", 99 | "- The thermodynamic conditions inside the well (liquid, two-phase, steam, cold gas)" 100 | ] 101 | }, 102 | { 103 | "cell_type": "markdown", 104 | "metadata": { 105 | "iooxa": { 106 | "id": { 107 | "block": "I8nhAd04Fs6pKXqZJ8rW", 108 | "project": "mmReuqVTAa9JzPpNr22I", 109 | "version": 1 110 | } 111 | } 112 | }, 113 | "source": [ 114 | "***\n", 115 | "\n", 116 | "# 5. Extract a completion test temperature log\n", 117 | "\n", 118 | "## 5.1 Use functions to import and munge the PTS and pump data\n", 119 | "\n", 120 | "Install all packages required for this notebook. \n", 121 | "\n", 122 | "If you do not already have iapws in your environment, then you will need to pip install it. This will need to be done in Google Colab. If you followed the Anaconda setup instructions to make an environment with the environment.yml, then you will not need to do this." 123 | ] 124 | }, 125 | { 126 | "cell_type": "code", 127 | "execution_count": null, 128 | "metadata": { 129 | "iooxa": { 130 | "id": { 131 | "block": "tZ97VsCHE3kcGy1vUBpB", 132 | "project": "mmReuqVTAa9JzPpNr22I", 133 | "version": 1 134 | }, 135 | "outputId": null 136 | } 137 | }, 138 | "outputs": [], 139 | "source": [ 140 | "!pip install iapws" 141 | ] 142 | }, 143 | { 144 | "cell_type": "code", 145 | "execution_count": null, 146 | "metadata": { 147 | "iooxa": { 148 | "id": { 149 | "block": "2tkmoDOXGStNgafWBiX3", 150 | "project": "mmReuqVTAa9JzPpNr22I", 151 | "version": 1 152 | }, 153 | "outputId": null 154 | } 155 | }, 156 | "outputs": [], 157 | "source": [ 158 | "import iapws # steam tables\n", 159 | "import openpyxl\n", 160 | "import pandas as pd\n", 161 | "from datetime import datetime\n", 162 | "import matplotlib.pyplot as plt\n", 163 | "import matplotlib.dates as mdates\n", 164 | "from ipywidgets import interactive, Layout, FloatSlider" 165 | ] 166 | }, 167 | { 168 | "cell_type": "markdown", 169 | "metadata": { 170 | "iooxa": { 171 | "id": { 172 | "block": "XhNYcLcmUzmPkELlMyJZ", 173 | "project": "mmReuqVTAa9JzPpNr22I", 174 | "version": 1 175 | } 176 | } 177 | }, 178 | "source": [ 179 | "The data munging process in 1-overview.ipynb has been turned into specialised functions. The cell below contains a set of helper functions" 180 | ] 181 | }, 182 | { 183 | "cell_type": "code", 184 | "execution_count": null, 185 | "metadata": { 186 | "iooxa": { 187 | "id": { 188 | "block": "1BZXPJ7h3XWsGzQDPwkh", 189 | "project": "mmReuqVTAa9JzPpNr22I", 190 | "version": 1 191 | }, 192 | "outputId": null 193 | } 194 | }, 195 | "outputs": [], 196 | "source": [ 197 | "\n", 198 | "\n", 199 | "def timedelta_seconds(dataframe_col, test_start):\n", 200 | " '''\n", 201 | " Make a float in seconds since the start of the test\n", 202 | "\n", 203 | " args: dataframe_col: dataframe column containing datetime objects\n", 204 | " test_start: test start time formatted '2020-12-11 09:00:00'\n", 205 | "\n", 206 | " returns: float in seconds since the start of the test\n", 207 | " '''\n", 208 | " test_start_datetime = pd.to_datetime(test_start)\n", 209 | " list = []\n", 210 | " for datetime in dataframe_col:\n", 211 | " time_delta = datetime - test_start_datetime\n", 212 | " seconds = time_delta.total_seconds()\n", 213 | " list.append(seconds)\n", 214 | " return list\n", 215 | "\n", 216 | "\n", 217 | "\n", 218 | "def read_flowrate(filename):\n", 219 | " ''' \n", 220 | " Read PTS-2-injection-rate.xlsx in as a pandas dataframe and munge for analysis\n", 221 | "\n", 222 | " args: filename is r'PTS-2-injection-rate.xlsx'\n", 223 | "\n", 224 | " returns: pandas dataframe with local NZ datetime and flowrate in t/hr\n", 225 | " '''\n", 226 | " df = pd.read_excel(filename, header=1) \n", 227 | " df.columns = ['raw_datetime','flow_Lpm']\n", 228 | "\n", 229 | " list = []\n", 230 | " for date in df['raw_datetime']:\n", 231 | " newdate = datetime.fromisoformat(date)\n", 232 | " list.append(newdate)\n", 233 | " df['ISO_datetime'] = list \n", 234 | "\n", 235 | " list = []\n", 236 | " for date in df.ISO_datetime:\n", 237 | " newdate = pd.to_datetime(datetime.strftime(date,'%Y-%m-%d %H:%M:%S'))\n", 238 | " list.append(newdate)\n", 239 | " df['datetime'] = list\n", 240 | "\n", 241 | " df['flow_tph'] = df.flow_Lpm * 0.060\n", 242 | "\n", 243 | " df['timedelta_sec'] = timedelta_seconds(df.datetime, '2020-12-11 09:26:44.448')\n", 244 | "\n", 245 | " df.drop(columns = ['raw_datetime', 'flow_Lpm', 'ISO_datetime'], inplace = True)\n", 246 | "\n", 247 | " return df\n", 248 | "\n", 249 | "\n", 250 | "\n", 251 | "def read_pts(filename):\n", 252 | " '''\n", 253 | " Read PTS-2.xlsx in as a Pandas dataframe and munge for analysis\n", 254 | "\n", 255 | " args: filename is r'PTS-2.xlsx'\n", 256 | "\n", 257 | " returns: Pandas dataframe with datetime (local) and key coloumns of PTS data with the correct dtype\n", 258 | " '''\n", 259 | " df = pd.read_excel(filename)\n", 260 | "\n", 261 | " dict = {\n", 262 | " 'DEPTH':'depth_m',\n", 263 | " 'SPEED': 'speed_mps',\n", 264 | " 'Cable Weight': 'cweight_kg',\n", 265 | " 'WHP': 'whp_barg',\n", 266 | " 'Temperature': 'temp_degC',\n", 267 | " 'Pressure': 'pressure_bara',\n", 268 | " 'Frequency': 'frequency_hz'\n", 269 | " }\n", 270 | " df.rename(columns=dict, inplace=True)\n", 271 | "\n", 272 | " df.drop(0, inplace=True)\n", 273 | " df.reset_index(drop=True, inplace=True)\n", 274 | "\n", 275 | " list = []\n", 276 | " for date in df.Timestamp:\n", 277 | " newdate = openpyxl.utils.datetime.from_excel(date)\n", 278 | " list.append(newdate)\n", 279 | " df['datetime'] = list\n", 280 | "\n", 281 | " df.drop(columns = ['Date', 'Time', 'Timestamp','Reed 0',\n", 282 | " 'Reed 1', 'Reed 2', 'Reed 3', 'Battery Voltage', \n", 283 | " 'PRT Ref Voltage','SGS Voltage', 'Internal Temp 1', \n", 284 | " 'Internal Temp 2', 'Internal Temp 3','Cal Temp', \n", 285 | " 'Error Code 1', 'Error Code 2', 'Error Code 3',\n", 286 | " 'Records Saved', 'Bad Pages',], inplace = True)\n", 287 | " \n", 288 | " df[\n", 289 | " ['depth_m', 'speed_mps','cweight_kg','whp_barg','temp_degC','pressure_bara','frequency_hz']\n", 290 | " ] = df[\n", 291 | " ['depth_m','speed_mps','cweight_kg','whp_barg','temp_degC','pressure_bara','frequency_hz']\n", 292 | " ].apply(pd.to_numeric)\n", 293 | " \n", 294 | " df['timedelta_sec'] = timedelta_seconds(df.datetime, '2020-12-11 09:26:44.448')\n", 295 | "\n", 296 | " return df\n", 297 | "\n", 298 | "\n", 299 | "\n", 300 | "def append_flowrate_to_pts(flowrate_df, pts_df):\n", 301 | " '''\n", 302 | " Add surface flowrate to pts data\n", 303 | "\n", 304 | " Note that the flowrate data is recorded at a courser time resolution than the pts data\n", 305 | " The function makes a linear interpolation to fill the data gaps\n", 306 | " Refer to bonus-combine-data.ipynb to review this method and adapt it for your own data\n", 307 | "\n", 308 | " Args: flowrate and pts dataframes generated by the read_flowrate and read_pts functions\n", 309 | "\n", 310 | " Returns: pts dataframe with flowrate tph added\n", 311 | " \n", 312 | " '''\n", 313 | " flowrate_df = flowrate_df.set_index('timedelta_sec')\n", 314 | " pts_df = pts_df.set_index('timedelta_sec')\n", 315 | " combined_df = pts_df.join(flowrate_df, how = 'outer', lsuffix = '_pts', rsuffix = '_fr')\n", 316 | " combined_df.drop(columns = ['datetime_fr'], inplace = True)\n", 317 | " combined_df.columns = ['depth_m', 'speed_mps', 'cweight_kg', 'whp_barg', 'temp_degC',\n", 318 | " 'pressure_bara', 'frequency_hz', 'datetime', 'flow_tph']\n", 319 | " combined_df['interpolated_flow_tph'] = combined_df['flow_tph'].interpolate(method='linear')\n", 320 | " trimmed_df = combined_df[combined_df['depth_m'].notna()]\n", 321 | " trimmed_df.reset_index(inplace=True)\n", 322 | " return trimmed_df\n", 323 | "\n", 324 | "\n", 325 | "\n", 326 | "def find_index(value, df, colname):\n", 327 | " '''\n", 328 | " Find the dataframe index for the exact matching value or nearest two values\n", 329 | "\n", 330 | " args: value: (float or int) the search term\n", 331 | " df: (obj) the name of the dataframe that is searched\n", 332 | " colname: (str) the name of the coloum this is searched\n", 333 | "\n", 334 | " returns: dataframe index(s) for the matching value or the two adjacent values\n", 335 | " rows can be called from a df using df.iloc[[index_number,index_number]]\n", 336 | " '''\n", 337 | " exactmatch = df[df[colname] == value]\n", 338 | " if not exactmatch.empty:\n", 339 | " return exactmatch.index\n", 340 | " else:\n", 341 | " lowerneighbour_index = df[df[colname] < value][colname].idxmax()\n", 342 | " upperneighbour_index = df[df[colname] > value][colname].idxmin()\n", 343 | " return [lowerneighbour_index, upperneighbour_index] \n", 344 | "\n", 345 | " \n", 346 | "\n", 347 | "def overview_fig(pts_df,flowrate_df,title=''):\n", 348 | " fig, (ax1, ax2, ax3, ax4, ax5, ax6) = plt.subplots(6, 1,figsize=(10,15),sharex=True)\n", 349 | " ax1.set_title(title,y=1.1,fontsize=15)\n", 350 | "\n", 351 | " ax1.plot(flowrate_df.datetime, flowrate_df.flow_tph, label='Surface pump flowrate', \n", 352 | " c='k', linewidth=0.8, marker='.')\n", 353 | " ax1.set_ylabel('Surface flowrate [t/hr]')\n", 354 | " ax1.set_ylim(0,150)\n", 355 | " \n", 356 | " ax2.plot(pts_df.datetime, pts_df.depth_m, label='PTS tool depth', \n", 357 | " c='k', linewidth=0.8)\n", 358 | " ax2.set_ylabel('PTS tool depth [m]')\n", 359 | " ax2.set_ylim(1000,0)\n", 360 | " \n", 361 | " ax3.plot(pts_df.datetime, pts_df.pressure_bara, label='PTS pressure', \n", 362 | " c='tab:blue', linewidth=0.8)\n", 363 | " ax3.set_ylabel('PTS pressure [bara]')\n", 364 | " \n", 365 | " ax4.plot(pts_df.datetime, pts_df.temp_degC, label='PTS temperature', \n", 366 | " c='tab:red', linewidth=0.8)\n", 367 | " ax4.set_ylabel('PTS temperature')\n", 368 | " \n", 369 | " ax5.plot(pts_df.datetime, pts_df.frequency_hz, label='PTS impeller frequency', \n", 370 | " c='tab:green', linewidth=0.8)\n", 371 | " ax5.set_ylim(-30,30)\n", 372 | " ax5.set_ylabel('PTS impeller frequency [hz]')\n", 373 | " # 1 hz = 60 rpm\n", 374 | "\n", 375 | " ax6.plot(pts_df.datetime, pts_df.speed_mps, label='PTS tool speed', \n", 376 | " c='tab:orange', linewidth=0.8)\n", 377 | " ax6.set_ylim(-2,2)\n", 378 | " ax6.set_ylabel('PTS tool speed [mps]')\n", 379 | " \n", 380 | " ax6.set_xlabel('Time [hh:mm]')\n", 381 | " plt.gca().xaxis.set_major_formatter(mdates.DateFormatter('%H:%M'))\n", 382 | " \n", 383 | " for ax in [ax1,ax2,ax3,ax4,ax5,ax6]:\n", 384 | " ax.grid()\n", 385 | " \n", 386 | " return plt" 387 | ] 388 | }, 389 | { 390 | "cell_type": "markdown", 391 | "metadata": { 392 | "iooxa": { 393 | "id": { 394 | "block": "GOdxLOgcGiCXmgQlkaL6", 395 | "project": "mmReuqVTAa9JzPpNr22I", 396 | "version": 1 397 | } 398 | } 399 | }, 400 | "source": [ 401 | "The cells below will take a little while to run because it includes all steps required to import and munge the data (i.e., everything we did in notebook 1)." 402 | ] 403 | }, 404 | { 405 | "cell_type": "code", 406 | "execution_count": null, 407 | "metadata": { 408 | "iooxa": { 409 | "id": { 410 | "block": "Z6qdxhjRNww1xjoHHByz", 411 | "project": "mmReuqVTAa9JzPpNr22I", 412 | "version": 1 413 | }, 414 | "outputId": null 415 | } 416 | }, 417 | "outputs": [], 418 | "source": [ 419 | "# Use this method if you are running this notebook in Google Colab\n", 420 | "flowrate = read_flowrate(r'/content/drive/My Drive/T21-Tutorial-WellTestAnalysis-main/Data-FlowRate.xlsx')\n", 421 | "\n", 422 | "# Use this method if you are running this notebook locally (Anaconda)\n", 423 | "#flowrate = read_flowrate(r'Data-FlowRate.xlsx')" 424 | ] 425 | }, 426 | { 427 | "cell_type": "code", 428 | "execution_count": null, 429 | "metadata": { 430 | "iooxa": { 431 | "id": { 432 | "block": "SP1NxIbyMDFvJqjFeyPQ", 433 | "project": "mmReuqVTAa9JzPpNr22I", 434 | "version": 1 435 | }, 436 | "outputId": null 437 | } 438 | }, 439 | "outputs": [], 440 | "source": [ 441 | "# Use this method if you are running this notebook in Google Colab\n", 442 | "pts = read_pts(r'/content/drive/My Drive/T21-Tutorial-WellTestAnalysis-main/Data-PTS.xlsx')\n", 443 | "\n", 444 | "# Use this method if you are running this notebook locally (Anaconda)\n", 445 | "#pts = read_pts(r'Data-PTS.xlsx')" 446 | ] 447 | }, 448 | { 449 | "cell_type": "code", 450 | "execution_count": null, 451 | "metadata": { 452 | "iooxa": { 453 | "id": { 454 | "block": "mcYUuBoCIRzGOrPFKHsh", 455 | "project": "mmReuqVTAa9JzPpNr22I", 456 | "version": 1 457 | }, 458 | "outputId": null 459 | } 460 | }, 461 | "outputs": [], 462 | "source": [ 463 | "print(pts.shape)\n", 464 | "pts.head(2)" 465 | ] 466 | }, 467 | { 468 | "cell_type": "markdown", 469 | "metadata": { 470 | "iooxa": { 471 | "id": { 472 | "block": "kOAPYG6LmBNz6DZqJLre", 473 | "project": "mmReuqVTAa9JzPpNr22I", 474 | "version": 1 475 | } 476 | } 477 | }, 478 | "source": [ 479 | "## 5.2 Look at the data\n", 480 | "\n", 481 | "It is good practice to plot data after import so any issues are revealed. \n", 482 | "\n", 483 | "Note that the temperature logs look a little odd because they include the data acquired while the tool was held stationary for pressure monitoring. " 484 | ] 485 | }, 486 | { 487 | "cell_type": "code", 488 | "execution_count": null, 489 | "metadata": { 490 | "iooxa": { 491 | "id": { 492 | "block": "vgGlTxzdFHFlzKKnRIyD", 493 | "project": "mmReuqVTAa9JzPpNr22I", 494 | "version": 1 495 | }, 496 | "outputId": null 497 | } 498 | }, 499 | "outputs": [], 500 | "source": [ 501 | "fig, (ax1, ax2) = plt.subplots(1, 2,figsize=(24,8),sharey=True)\n", 502 | "\n", 503 | "spinner_scatter = ax1.scatter(pts.temp_degC, pts.depth_m, c = pts.timedelta_sec, s = 5, linewidths = 0)\n", 504 | "datetime_scatter = ax2.scatter(pts.datetime, pts.depth_m, c = pts.timedelta_sec, s = 5, linewidths = 0)\n", 505 | "\n", 506 | "ax3 = ax2.twinx()\n", 507 | "ax3.plot(flowrate.datetime, flowrate.flow_tph, \n", 508 | " c='k', linestyle = '-', linewidth = 3, alpha = 0.3, \n", 509 | " label='Surface pump flowrate')\n", 510 | "\n", 511 | "ax1.set_ylabel('Depth [m]')\n", 512 | "ax1.set_xlabel('Temp [degC]')\n", 513 | "\n", 514 | "ax2.set_xlabel('Time [hh:mm]')\n", 515 | "ax2.xaxis.set_major_formatter(mdates.DateFormatter('%H:%M'))\n", 516 | "\n", 517 | "ax3.set_ylabel('Flowrate [t/hr]')\n", 518 | "\n", 519 | "ax1.set_ylim(1000,0)\n", 520 | "ax1.set_xlim(0,200)\n", 521 | "\n", 522 | "for ax in [ax1, ax2]:\n", 523 | " ax.grid()\n" 524 | ] 525 | }, 526 | { 527 | "cell_type": "markdown", 528 | "metadata": { 529 | "iooxa": { 530 | "id": { 531 | "block": "X9dxZiuGwU5A27RcOsSi", 532 | "project": "mmReuqVTAa9JzPpNr22I", 533 | "version": 1 534 | } 535 | } 536 | }, 537 | "source": [ 538 | "## 5.3 Remove data acquired when the tool is stationary or slowing\n", 539 | "\n", 540 | "We decided that, in this case, data acquired when the tool is moving down the well at > 0.02 m/s is fast enough to be included. The reasoning behind this method is described in the notebook Bonus-filter-by-toolspeed.ipynb\n", 541 | " \n", 542 | "In the cell below, we use a Boolean expression to filter the pts dataframe. Our new working dataframe is called moving_pts " 543 | ] 544 | }, 545 | { 546 | "cell_type": "code", 547 | "execution_count": null, 548 | "metadata": { 549 | "iooxa": { 550 | "id": { 551 | "block": "8gXy69Wl8xH43Q5to1qB", 552 | "project": "mmReuqVTAa9JzPpNr22I", 553 | "version": 1 554 | }, 555 | "outputId": null 556 | } 557 | }, 558 | "outputs": [], 559 | "source": [ 560 | "moving_pts = pts[\n", 561 | " (pts.speed_mps > 0.9 ) & (pts.speed_mps < pts.speed_mps.max()) | \n", 562 | " (pts.speed_mps > pts.speed_mps.min() ) & (pts.speed_mps < -0.9)\n", 563 | " ]" 564 | ] 565 | }, 566 | { 567 | "cell_type": "code", 568 | "execution_count": null, 569 | "metadata": { 570 | "iooxa": { 571 | "id": { 572 | "block": "YE2Eq17BF5gX2uxxw7jI", 573 | "project": "mmReuqVTAa9JzPpNr22I", 574 | "version": 1 575 | }, 576 | "outputId": null 577 | } 578 | }, 579 | "outputs": [], 580 | "source": [ 581 | "moving_pts.describe()" 582 | ] 583 | }, 584 | { 585 | "cell_type": "markdown", 586 | "metadata": { 587 | "iooxa": { 588 | "id": { 589 | "block": "Jlx9vruFdA9bJgdvnrM6", 590 | "project": "mmReuqVTAa9JzPpNr22I", 591 | "version": 1 592 | } 593 | } 594 | }, 595 | "source": [ 596 | "## 5.4 Extract the first heating run from the data\n", 597 | "\n", 598 | "We often want to select a single temperature log for analysis. In this example, we will extract the first heating run (i.e., the temperature log acquired after we stopped pumping into the well)\n", 599 | "\n", 600 | "We will use an interactive tool called ipywidgets to select the start and end time of our desired data range. " 601 | ] 602 | }, 603 | { 604 | "cell_type": "code", 605 | "execution_count": null, 606 | "metadata": { 607 | "iooxa": { 608 | "id": { 609 | "block": "uKBBylPT2mHQMM7lJcIs", 610 | "project": "mmReuqVTAa9JzPpNr22I", 611 | "version": 1 612 | }, 613 | "outputId": null 614 | } 615 | }, 616 | "outputs": [], 617 | "source": [ 618 | "min_timestamp = pts.timedelta_sec.iloc[0]\n", 619 | "max_timestamp = pts.timedelta_sec.iloc[-1]\n", 620 | "\n", 621 | "def subselect_plot(start_value, stop_value):\n", 622 | " f,ax = plt.subplots(1,1, figsize = (20,6))\n", 623 | " ax.scatter(moving_pts.timedelta_sec, moving_pts.depth_m,\n", 624 | " c = 'k', s = 1, linewidths = 0, label = 'Tool depth')\n", 625 | " ax1 = ax.twinx()\n", 626 | " ax1.plot(flowrate.timedelta_sec, flowrate.flow_tph, \n", 627 | " ':', c='k', label='Surface pump flowrate')\n", 628 | " ymin = pts.depth_m.min()\n", 629 | " ymax = pts.depth_m.max() + 100\n", 630 | " ax.vlines(start_value, ymin, ymax, color='tab:green')\n", 631 | " ax.vlines(stop_value, ymin, ymax, color='tab:red')\n", 632 | " ax.set_ylim(pts.depth_m.max() + 100, 0)\n", 633 | " ax.set_xlabel('Timestamp [sec]')\n", 634 | " ax.set_ylabel('Tool depth [m]')\n", 635 | " ax1.set_ylabel('Flowrate [t/hr]')\n", 636 | " ax.set_xlabel('Time elapsed since the test started [sec]')\n", 637 | " ax.set_ylabel('Tool depth [m]')\n", 638 | " ax1.set_ylabel('Flowrate [t/hr]')\n", 639 | "\n", 640 | "result = interactive(subselect_plot,\n", 641 | " \n", 642 | " start_value = FloatSlider\n", 643 | " (\n", 644 | " value = (max_timestamp - min_timestamp)/3 + min_timestamp,\n", 645 | " description = 'start',\n", 646 | " min = min_timestamp, \n", 647 | " max = max_timestamp, \n", 648 | " step = 10, \n", 649 | " continuous_update=False,\n", 650 | " layout = Layout(width='80%'),\n", 651 | " ),\n", 652 | " \n", 653 | " stop_value = FloatSlider\n", 654 | " (\n", 655 | " value = (max_timestamp - min_timestamp)/2 + min_timestamp, \n", 656 | " description = 'stop',\n", 657 | " min = min_timestamp, \n", 658 | " max = max_timestamp, \n", 659 | " step = 10, \n", 660 | " continuous_update=False,\n", 661 | " layout = Layout(width='80%')\n", 662 | " )\n", 663 | ")\n", 664 | "\n", 665 | "display(result);\n" 666 | ] 667 | }, 668 | { 669 | "cell_type": "markdown", 670 | "metadata": { 671 | "iooxa": { 672 | "id": { 673 | "block": "9AawjQBc0g9PFLr1hrNu", 674 | "project": "mmReuqVTAa9JzPpNr22I", 675 | "version": 1 676 | } 677 | } 678 | }, 679 | "source": [ 680 | "### Make your selection\n", 681 | "\n", 682 | "Move the sliders above and then run the cell below to return the timestamp at the location of the slider. \n", 683 | "\n", 684 | "Each time you change this plot, the underlying array of data that we call with the result.children\\[x\\] statement is changed." 685 | ] 686 | }, 687 | { 688 | "cell_type": "code", 689 | "execution_count": null, 690 | "metadata": { 691 | "iooxa": { 692 | "id": { 693 | "block": "Fa9swJKtzO5H2RbFnWkU", 694 | "project": "mmReuqVTAa9JzPpNr22I", 695 | "version": 1 696 | }, 697 | "outputId": null 698 | } 699 | }, 700 | "outputs": [], 701 | "source": [ 702 | "print(\n", 703 | " 'start =',result.children[0].value, \n", 704 | " '\\nstop =', result.children[1].value,\n", 705 | " )" 706 | ] 707 | }, 708 | { 709 | "cell_type": "markdown", 710 | "metadata": { 711 | "iooxa": { 712 | "id": { 713 | "block": "e9WaRLK5eDnq1rBJATuA", 714 | "project": "mmReuqVTAa9JzPpNr22I", 715 | "version": 1 716 | } 717 | } 718 | }, 719 | "source": [ 720 | "### Record your analysis\n", 721 | "\n", 722 | "As part of making this process repeatable and so that it is easy to come back later to check the analysis, you need to use the range you have selected to manually define objects in the next step.\n", 723 | "\n", 724 | "Copy-paste the timestamps printed above into the cell below. They are now the objects that define the start and stop points for splicing the moving_pts dataframe. \n", 725 | "\n", 726 | "We could have defined the start and stop objects as the result.children\\[0\\].value and result.children\\[1\\].value objects. But if we did this, you would lose your work because these ipywidget results change every time the sliders are moved or the notebook is re-run. " 727 | ] 728 | }, 729 | { 730 | "cell_type": "code", 731 | "execution_count": null, 732 | "metadata": { 733 | "iooxa": { 734 | "id": { 735 | "block": "ddUJkZPEfladTtElr35d", 736 | "project": "mmReuqVTAa9JzPpNr22I", 737 | "version": 1 738 | }, 739 | "outputId": null 740 | } 741 | }, 742 | "outputs": [], 743 | "source": [ 744 | "# paste your values here\n", 745 | "start = 24150.0 \n", 746 | "stop = 24470.0\n", 747 | "\n", 748 | "# create an informative title for the data\n", 749 | "name = 'First heating run' \n", 750 | "\n", 751 | "pts_first_heating_run = moving_pts[\n", 752 | " (moving_pts.timedelta_sec > start) \n", 753 | " & (moving_pts.timedelta_sec < stop)\n", 754 | "]\n", 755 | "\n", 756 | "flowrate_first_heating_run = flowrate[\n", 757 | " (flowrate.timedelta_sec > start) \n", 758 | " & (flowrate.timedelta_sec < stop)\n", 759 | "]" 760 | ] 761 | }, 762 | { 763 | "cell_type": "code", 764 | "execution_count": null, 765 | "metadata": { 766 | "iooxa": { 767 | "id": { 768 | "block": "1bFY2XGY4UYDwCY9RoKg", 769 | "project": "mmReuqVTAa9JzPpNr22I", 770 | "version": 1 771 | }, 772 | "outputId": null 773 | } 774 | }, 775 | "outputs": [], 776 | "source": [ 777 | "pts_first_heating_run.shape" 778 | ] 779 | }, 780 | { 781 | "cell_type": "markdown", 782 | "metadata": { 783 | "iooxa": { 784 | "id": { 785 | "block": "NCh2ZbRNdeYWKUOVkz5q", 786 | "project": "mmReuqVTAa9JzPpNr22I", 787 | "version": 1 788 | } 789 | } 790 | }, 791 | "source": [ 792 | "The overview_fig function generates a figure that displays our data selection. \n", 793 | "\n", 794 | "Note that surface flow rate = 0 because this log is acquired after the pumps were shut off. " 795 | ] 796 | }, 797 | { 798 | "cell_type": "code", 799 | "execution_count": null, 800 | "metadata": { 801 | "iooxa": { 802 | "id": { 803 | "block": "JxKnPmMvMFtMzGuDgsOS", 804 | "project": "mmReuqVTAa9JzPpNr22I", 805 | "version": 1 806 | }, 807 | "outputId": null 808 | } 809 | }, 810 | "outputs": [], 811 | "source": [ 812 | "overview_fig(pts_first_heating_run, flowrate_first_heating_run, title = name).show()" 813 | ] 814 | }, 815 | { 816 | "cell_type": "markdown", 817 | "metadata": { 818 | "iooxa": { 819 | "id": { 820 | "block": "PUrFEi6qICX8Q6SxDVib", 821 | "project": "mmReuqVTAa9JzPpNr22I", 822 | "version": 1 823 | } 824 | } 825 | }, 826 | "source": [ 827 | "### Export data as a csv \n", 828 | "\n", 829 | "Use the following method to export your selected data to use elsewhere. " 830 | ] 831 | }, 832 | { 833 | "cell_type": "code", 834 | "execution_count": null, 835 | "metadata": { 836 | "iooxa": { 837 | "id": { 838 | "block": "xcuyvGjBVlyjXsA9v6Wp", 839 | "project": "mmReuqVTAa9JzPpNr22I", 840 | "version": 1 841 | }, 842 | "outputId": null 843 | } 844 | }, 845 | "outputs": [], 846 | "source": [ 847 | "#pts_first_heating_run.to_csv('Data-Temp-Heating0days.csv', index=False)" 848 | ] 849 | }, 850 | { 851 | "cell_type": "markdown", 852 | "metadata": { 853 | "iooxa": { 854 | "id": { 855 | "block": "IB4SrgBWP2iWHSkkbduh", 856 | "project": "mmReuqVTAa9JzPpNr22I", 857 | "version": 1 858 | } 859 | } 860 | }, 861 | "source": [ 862 | "## 5.5 Import and calculate contextualising data\n", 863 | "\n", 864 | "We have provided some contextualising data that:\n", 865 | "1. Illustrates how geothermal wells heat up over time \n", 866 | "2. Helps us interpret the temperature log we just extracted \n", 867 | "\n", 868 | "### 5.5.1 Stable temperature\n", 869 | "\n", 870 | "A temperature log was acquired in this well 37 days after it was drilled and tested. In this resource, it reflects the stable temperature of this well. In low permeability settings, wells may take months to fully heat after drilling and testing." 871 | ] 872 | }, 873 | { 874 | "cell_type": "code", 875 | "execution_count": null, 876 | "metadata": { 877 | "iooxa": { 878 | "id": { 879 | "block": "Ug2Rz2Keuz8YeXMvEzCG", 880 | "project": "mmReuqVTAa9JzPpNr22I", 881 | "version": 1 882 | }, 883 | "outputId": null 884 | } 885 | }, 886 | "outputs": [], 887 | "source": [ 888 | "# Use this method if you are running this notebook in Google Colab\n", 889 | "heating_37days = pd.read_csv(r'/content/drive/My Drive/T21-Tutorial-WellTestAnalysis-main/Data-Temp-Heating37days.csv') \n", 890 | "\n", 891 | "# Use this method if you are running this notebook locally (Anaconda)\n", 892 | "#heating_37days = pd.read_csv('Data-Temp-Heating37days.csv')" 893 | ] 894 | }, 895 | { 896 | "cell_type": "code", 897 | "execution_count": null, 898 | "metadata": { 899 | "iooxa": { 900 | "id": { 901 | "block": "rKQmseCPmXULCBib1hqm", 902 | "project": "mmReuqVTAa9JzPpNr22I", 903 | "version": 1 904 | }, 905 | "outputId": null 906 | } 907 | }, 908 | "outputs": [], 909 | "source": [ 910 | "# Convert bar gauge to bar atmosphere\n", 911 | "\n", 912 | "heating_37days['pressure_bara'] = heating_37days.pres_barg - 1\n", 913 | "\n", 914 | "heating_37days.head(2)" 915 | ] 916 | }, 917 | { 918 | "cell_type": "markdown", 919 | "metadata": { 920 | "iooxa": { 921 | "id": { 922 | "block": "k0JO1ezHXJFUrDT0nQxI", 923 | "project": "mmReuqVTAa9JzPpNr22I", 924 | "version": 1 925 | } 926 | } 927 | }, 928 | "source": [ 929 | "### 5.5.2 Saturation temperature aka the boiling point for depth (BPD) curve\n", 930 | "\n", 931 | "Water will boil at a given pressure - temperature condition. We use steam tables to calculate the temperature boiling will occur given the measured pressure. This curve helps us to understand the thermodynamic conditions inside the well. \n", 932 | "\n", 933 | "The module we use to calculate the BPD is iapws and does not come standard with Anaconda. If you get a 'module not found' error, then return to the top of this notebook and run the iapws installation or pip install it into your Conda environment. " 934 | ] 935 | }, 936 | { 937 | "cell_type": "code", 938 | "execution_count": null, 939 | "metadata": { 940 | "iooxa": { 941 | "id": { 942 | "block": "zJkPIJyFHwSMieDyrGq1", 943 | "project": "mmReuqVTAa9JzPpNr22I", 944 | "version": 1 945 | }, 946 | "outputId": null 947 | } 948 | }, 949 | "outputs": [], 950 | "source": [ 951 | "# note that iapws uses SI units so some unit conversion is required\n", 952 | "\n", 953 | "heating_37days['pressure_mpa'] = heating_37days.pressure_bara * 0.1 # convert pressure to MPa for ipaws\n", 954 | "\n", 955 | "pressure = heating_37days['pressure_mpa'].tolist()\n", 956 | "tsat = []\n", 957 | "for p in pressure:\n", 958 | " saturation_temp = iapws.iapws97._TSat_P(p) - 273.15 # calculate saturation temp in Kelvin & convert to degC\n", 959 | " tsat.append(saturation_temp)\n", 960 | "heating_37days['tsat_degC'] = tsat\n", 961 | "\n", 962 | "heating_37days.head(2)" 963 | ] 964 | }, 965 | { 966 | "cell_type": "markdown", 967 | "metadata": { 968 | "iooxa": { 969 | "id": { 970 | "block": "J3UTsEvhAhkp8V7oeXLK", 971 | "project": "mmReuqVTAa9JzPpNr22I", 972 | "version": 1 973 | } 974 | } 975 | }, 976 | "source": [ 977 | "### 5.5.3 Well completion information\n", 978 | "\n", 979 | "Understanding where we are in the well helps us to interpret the data. This is particularly important when we seek to understand what is a process occurring inside the well and what is related to the reservoir. " 980 | ] 981 | }, 982 | { 983 | "cell_type": "code", 984 | "execution_count": null, 985 | "metadata": { 986 | "iooxa": { 987 | "id": { 988 | "block": "r4sEUsCiXskd53hjfkIj", 989 | "project": "mmReuqVTAa9JzPpNr22I", 990 | "version": 1 991 | }, 992 | "outputId": null 993 | } 994 | }, 995 | "outputs": [], 996 | "source": [ 997 | "production_shoe = 462.5 # 13 3/8 production casing shoe in meters measured depth (mMD) from the casing head flange (CHF)\n", 998 | "top_of_liner = 425 # top of perferated 10 3/4 liner in meters measured depth (mMD) from CHF\n", 999 | "terminal_depth = 946 # deepest drilled depth \n", 1000 | "# the perferated liner is squatted on bottom but didn't quite make it all the way down (bottom of liner is 931 mMD)" 1001 | ] 1002 | }, 1003 | { 1004 | "cell_type": "markdown", 1005 | "metadata": { 1006 | "iooxa": { 1007 | "id": { 1008 | "block": "eup2bYGezZL50fbjxUEa", 1009 | "project": "mmReuqVTAa9JzPpNr22I", 1010 | "version": 1 1011 | } 1012 | } 1013 | }, 1014 | "source": [ 1015 | "### Datums and depth measurements for well test data\n", 1016 | "\n", 1017 | "The depth of downhole data comes in two forms: \n", 1018 | "- Meters measured depth (mMD), which is the number of meters along the well path\n", 1019 | "- Vertical meters (mVD), which is the number of meters in the vertical dimension \n", 1020 | "\n", 1021 | "It is typical for completion test data to be acquired and analysed in measured depth. However, when we seek to understand features of the reservoir, we need to also consider data in vertical depth. \n", 1022 | "\n", 1023 | "There are a range of datums that are used in geothermal: \n", 1024 | "- During drilling, most data is acquired relative to the rig floor or rotary table\n", 1025 | "- Well test data acquired after the rig has been demobilised is typically acquired relative to the casing head flange (CHF) or the cellar top (CT), where the latter typically has a survey marker\n", 1026 | "- Data may also be presented relative to sea level (RSL) or above sea level (ASL)\n", 1027 | "\n", 1028 | "It is ambiguous to define data relative to ground level (GL) because of the degree of civil construction that occurs on well pads. " 1029 | ] 1030 | }, 1031 | { 1032 | "cell_type": "markdown", 1033 | "metadata": { 1034 | "iooxa": { 1035 | "id": { 1036 | "block": "0JDUmcAv8ckUdl36HiOJ", 1037 | "project": "mmReuqVTAa9JzPpNr22I", 1038 | "version": 1 1039 | } 1040 | } 1041 | }, 1042 | "source": [ 1043 | "## 5.6 Visualise the data for interpretation\n", 1044 | "\n" 1045 | ] 1046 | }, 1047 | { 1048 | "cell_type": "code", 1049 | "execution_count": null, 1050 | "metadata": { 1051 | "iooxa": { 1052 | "id": { 1053 | "block": "6h9So26dfMzYzdLVkVUn", 1054 | "project": "mmReuqVTAa9JzPpNr22I", 1055 | "version": 1 1056 | }, 1057 | "outputId": null 1058 | } 1059 | }, 1060 | "outputs": [], 1061 | "source": [ 1062 | "fig, (ax1, ax2) = plt.subplots(1, 2,figsize=(10,10),sharey=True)\n", 1063 | "\n", 1064 | "\n", 1065 | "ax1.set_title('All completion test temperature logs')\n", 1066 | "\n", 1067 | "# all temp data colored by time \n", 1068 | "ax1.scatter(moving_pts.temp_degC, moving_pts.depth_m, \n", 1069 | " c = moving_pts.timedelta_sec, s = 5, linewidths = 0)\n", 1070 | "\n", 1071 | "# our sub-selected data\n", 1072 | "ax1.plot(pts_first_heating_run.temp_degC, pts_first_heating_run.depth_m,\n", 1073 | " color = 'k', linestyle = ':', label = 'Our selected data')\n", 1074 | "\n", 1075 | "# blank well casing\n", 1076 | "ax1.plot([1, 1],[0, production_shoe],\n", 1077 | " color = 'k', linewidth = 3, linestyle = '-')\n", 1078 | "\n", 1079 | "# perforated well casing\n", 1080 | "ax1.plot([5, 5],[top_of_liner, terminal_depth], \n", 1081 | " color = 'k', linewidth = 1.5, linestyle = '--')\n", 1082 | "\n", 1083 | "\n", 1084 | "ax2.set_title('Heating (shut) temperature logs')\n", 1085 | "\n", 1086 | "# our sub-selected pressure + temp data\n", 1087 | "ax2.plot(pts_first_heating_run.temp_degC, pts_first_heating_run.depth_m, \n", 1088 | " color ='#EBE85B', linewidth = 3, label = 'Day 0 - temp')\n", 1089 | "\n", 1090 | "ax2.plot(pts_first_heating_run.pressure_bara, pts_first_heating_run.depth_m, \n", 1091 | " linestyle = '--', linewidth = 3, color = '#EBE85B', label = 'Day 0 - pres')\n", 1092 | "\n", 1093 | "# stable pressure + temp data\n", 1094 | "ax2.plot(heating_37days.temp_degC, heating_37days.depth_m, \n", 1095 | " color = 'k', label = 'Day 37 - temp')\n", 1096 | "\n", 1097 | "ax2.plot(heating_37days.pressure_bara, heating_37days.depth_m, \n", 1098 | " linestyle = '--', color = 'k', label = 'Day 37 - pres')\n", 1099 | "\n", 1100 | "# saturation temp for the stable pressure\n", 1101 | "ax2.plot(heating_37days.tsat_degC, heating_37days.depth_m, \n", 1102 | " linestyle = ':', color = 'k', label = 'Day 37 - BPD')\n", 1103 | "\n", 1104 | "\n", 1105 | "ax1.set_ylim(1000,0) \n", 1106 | "ax1.set_ylabel('Depth [m]')\n", 1107 | "ax1.set_xlim(0,200)\n", 1108 | "ax1.legend()\n", 1109 | "\n", 1110 | "ax2.legend()\n", 1111 | "ax2.set_xlim(0,300)\n", 1112 | "\n", 1113 | "for ax in [ax1,ax2]:\n", 1114 | " ax.set_xlabel('Temperature [degC] or Pressure [bara]')\n", 1115 | " ax.grid()\n" 1116 | ] 1117 | }, 1118 | { 1119 | "cell_type": "markdown", 1120 | "metadata": { 1121 | "iooxa": { 1122 | "id": { 1123 | "block": "3Dl7PDQvyFY6gXHvZJ6c", 1124 | "project": "mmReuqVTAa9JzPpNr22I", 1125 | "version": 1 1126 | } 1127 | } 1128 | }, 1129 | "source": [ 1130 | "## 6. Temperature in our case study well\n", 1131 | "\n", 1132 | "This is not a course on how to read temperature logs. However, we can make a couple of quick observations of these data. \n", 1133 | "\n", 1134 | "- The injecting temperature profiles are a completely different shape from the heating (shut) profiles. This illustrates the importance of knowing what the well condition was when the temperature log was acquired (i.e., shut, injection, flowing, bleed).\n", 1135 | "- The two heating (shut) profiles have a completely different shape. This illustrates how important it is to know _when_ the temperature log was acquired relative to operations that impact well temperature (i.e., drilling, injection testing or quenching, flowing, etc). \n", 1136 | "- It is heating up the most rapidly just below 700 mMD where two-phase fluid is entering the well. \n", 1137 | "- The 37-day heating profile is on the BPD profile from 450-600 mMD, so we would expect two-phase conditions there. The BPD used is for pure water and may not be exact. Below 600 mMD we expect that the well is compressed liquid.\n" 1138 | ] 1139 | }, 1140 | { 1141 | "cell_type": "markdown", 1142 | "metadata": { 1143 | "iooxa": { 1144 | "id": { 1145 | "block": "QTQpMFaGQsJetZRbKNS9", 1146 | "project": "mmReuqVTAa9JzPpNr22I", 1147 | "version": 1 1148 | } 1149 | } 1150 | }, 1151 | "source": [ 1152 | "***\n", 1153 | "\n", 1154 | "© 2021 [Irene Wallis](https://www.cubicearth.nz/) and [Katie McLean](https://www.linkedin.com/in/katie-mclean-25994315/) \n", 1155 | "\n", 1156 | "Licensed under the Apache License, Version 2.0\n", 1157 | "\n", 1158 | "***" 1159 | ] 1160 | } 1161 | ], 1162 | "metadata": { 1163 | "iooxa": { 1164 | "id": { 1165 | "block": "6J2TDk2GvAAOQxzyYQfL", 1166 | "project": "mmReuqVTAa9JzPpNr22I", 1167 | "version": 1 1168 | } 1169 | }, 1170 | "kernelspec": { 1171 | "display_name": "geothrm", 1172 | "language": "python", 1173 | "name": "geothrm" 1174 | }, 1175 | "language_info": { 1176 | "codemirror_mode": { 1177 | "name": "ipython", 1178 | "version": 3 1179 | }, 1180 | "file_extension": ".py", 1181 | "mimetype": "text/x-python", 1182 | "name": "python", 1183 | "nbconvert_exporter": "python", 1184 | "pygments_lexer": "ipython3", 1185 | "version": "3.7.9" 1186 | } 1187 | }, 1188 | "nbformat": 4, 1189 | "nbformat_minor": 4 1190 | } 1191 | -------------------------------------------------------------------------------- /3-injectivity.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": { 6 | "iooxa": { 7 | "id": { 8 | "block": "6iJGW9Ee1v0UFTmVNXBU", 9 | "project": "mmReuqVTAa9JzPpNr22I", 10 | "version": 1 11 | } 12 | } 13 | }, 14 | "source": [ 15 | "***\n", 16 | "\n", 17 | "# Geothermal Well Test Analysis with Python\n", 18 | "### Notebook 3: Determining well capacity as injectivity index\n", 19 | "#### Irene Wallis and Katie McLean \n", 20 | "#### Software Underground, Transform 2021\n", 21 | "\n", 22 | "***" 23 | ] 24 | }, 25 | { 26 | "cell_type": "markdown", 27 | "metadata": { 28 | "iooxa": { 29 | "id": { 30 | "block": "5uEy8MtapeDwl2BFVfq5", 31 | "project": "mmReuqVTAa9JzPpNr22I", 32 | "version": 1 33 | } 34 | } 35 | }, 36 | "source": [ 37 | "### Google Colab Setup\n", 38 | "\n", 39 | "If you are using Google Colab to run this notebook, we assume you have already followed the Google Colab setup steps outlined [here](https://github.com/ICWallis/T21-Tutorial-WellTestAnalysis).\n", 40 | "\n", 41 | "Because we are importing data, we need to \"mount your Google Drive\", which is where we tell this notebook to look for the data files. You will need to mount the Google Drive into each notebook. \n", 42 | "\n", 43 | "1. Run the cell below if you are in Google Colab. If you are not in Google Colab, running the cell below will just return an error that says \"No module named 'google'\". If you get a Google Colab error that says \"Unrecognised runtime 'geothrm'; defaulting to 'python3' Notebook settings\", just ignore it. \n", 44 | "\n", 45 | "2. Follow the link generated by running this code. That link will ask you to sign in to your google account (use the one where you have saved these tutorial materials in) and to allow this notebook access to your google drive. \n", 46 | "\n", 47 | "3. Completing step 2 above will generate a code. Copy this code, paste below where it says \"Enter your authorization code:\", and press ENTER. \n", 48 | "\n", 49 | "Congratulations, this notebook can now import data!" 50 | ] 51 | }, 52 | { 53 | "cell_type": "code", 54 | "execution_count": null, 55 | "metadata": { 56 | "iooxa": { 57 | "id": { 58 | "block": "2yc0A14MocmIUBJeyxJS", 59 | "project": "mmReuqVTAa9JzPpNr22I", 60 | "version": 1 61 | }, 62 | "outputId": null 63 | } 64 | }, 65 | "outputs": [], 66 | "source": [ 67 | "from google.colab import drive\n", 68 | "drive.mount('/content/drive')" 69 | ] 70 | }, 71 | { 72 | "cell_type": "markdown", 73 | "metadata": { 74 | "iooxa": { 75 | "id": { 76 | "block": "yUDqmVG46PXRPDkJOroN", 77 | "project": "mmReuqVTAa9JzPpNr22I", 78 | "version": 1 79 | } 80 | } 81 | }, 82 | "source": [ 83 | "***\n", 84 | "\n", 85 | "# 7. Injectivity index\n", 86 | "\n", 87 | "The injectivity index is a measure of the amount of fluid the well can accept during injection. Specifically, it is the change in mass rate (t/hr) per change in pressure (bar), hence it has the units t/hr/bar. \n", 88 | "\n", 89 | "If the well is destined to be used as an injection well, then the injectivity is a direct measure of the future well performance (though it must be corrected for different injectate temperatures). \n", 90 | "\n", 91 | "If the well is destined to be used as a production well then the injectivity is used to give an indication of future productivity. Productivity index also has the units t/h/bar though it refers to the flow rate out of the well during production, divided by the pressure drop (pressure drawdown) downhole during production. \n", 92 | " \n", 93 | "This notebook gives a workflow to: \n", 94 | "- Import and check data \n", 95 | "- Select the stable pressure values to use for each flow rate\n", 96 | "- Create the flow rate vs stable pressure plot (t/h vs bar)\n", 97 | "- Use linear regression to find the slope (t/h/bar)" 98 | ] 99 | }, 100 | { 101 | "cell_type": "markdown", 102 | "metadata": { 103 | "iooxa": { 104 | "id": { 105 | "block": "mKBqf6Z7RNrIFeXo8UvP", 106 | "project": "mmReuqVTAa9JzPpNr22I", 107 | "version": 1 108 | } 109 | } 110 | }, 111 | "source": [ 112 | "***\n", 113 | "\n", 114 | "# 8. Import, munge and check data\n", 115 | "\n", 116 | "## 8.1 Use bespoke functions to import and munge data\n", 117 | "\n", 118 | "Install all packages required for this notebook. \n", 119 | "\n", 120 | "If you do not already have iapws in your environment, then you will need to pip install it. This will need to be done in Google Colab. If you followed the Anaconda setup instructions to make an environment with the environment.yml, then you will not need to do this. " 121 | ] 122 | }, 123 | { 124 | "cell_type": "code", 125 | "execution_count": null, 126 | "metadata": { 127 | "iooxa": { 128 | "id": { 129 | "block": "bHC0oDQlQoHOzqf3ALqI", 130 | "project": "mmReuqVTAa9JzPpNr22I", 131 | "version": 1 132 | }, 133 | "outputId": null 134 | } 135 | }, 136 | "outputs": [], 137 | "source": [ 138 | "!pip install iapws" 139 | ] 140 | }, 141 | { 142 | "cell_type": "code", 143 | "execution_count": null, 144 | "metadata": { 145 | "iooxa": { 146 | "id": { 147 | "block": "80RxZjPjjfjCCP2THrOd", 148 | "project": "mmReuqVTAa9JzPpNr22I", 149 | "version": 1 150 | }, 151 | "outputId": null 152 | } 153 | }, 154 | "outputs": [], 155 | "source": [ 156 | "import iapws # steam tables\n", 157 | "import openpyxl\n", 158 | "import numpy as np\n", 159 | "import pandas as pd\n", 160 | "from scipy import stats\n", 161 | "from datetime import datetime\n", 162 | "import matplotlib.pyplot as plt\n", 163 | "import matplotlib.dates as mdates\n", 164 | "from IPython.display import Image\n", 165 | "from ipywidgets import interactive, Layout, FloatSlider" 166 | ] 167 | }, 168 | { 169 | "cell_type": "code", 170 | "execution_count": null, 171 | "metadata": { 172 | "iooxa": { 173 | "id": { 174 | "block": "sdYoDOiIv6MK0laKujbZ", 175 | "project": "mmReuqVTAa9JzPpNr22I", 176 | "version": 1 177 | }, 178 | "outputId": null 179 | } 180 | }, 181 | "outputs": [], 182 | "source": [ 183 | "\n", 184 | "\n", 185 | "def timedelta_seconds(dataframe_col, test_start):\n", 186 | " '''\n", 187 | " Make a float in seconds since the start of the test\n", 188 | "\n", 189 | " args: dataframe_col: dataframe column containing datetime objects\n", 190 | " test_start: test start time formatted '2020-12-11 09:00:00'\n", 191 | "\n", 192 | " returns: float in seconds since the start of the test\n", 193 | " '''\n", 194 | " test_start_datetime = pd.to_datetime(test_start)\n", 195 | " list = []\n", 196 | " for datetime in dataframe_col:\n", 197 | " time_delta = datetime - test_start_datetime\n", 198 | " seconds = time_delta.total_seconds()\n", 199 | " list.append(seconds)\n", 200 | " return list\n", 201 | "\n", 202 | "\n", 203 | "\n", 204 | "def read_flowrate(filename):\n", 205 | " ''' \n", 206 | " Read PTS-2-injection-rate.xlsx in as a pandas dataframe and munge for analysis\n", 207 | "\n", 208 | " args: filename is r'PTS-2-injection-rate.xlsx'\n", 209 | "\n", 210 | " returns: pandas dataframe with local NZ datetime and flowrate in t/hr\n", 211 | " '''\n", 212 | " df = pd.read_excel(filename, header=1) \n", 213 | " df.columns = ['raw_datetime','flow_Lpm']\n", 214 | "\n", 215 | " list = []\n", 216 | " for date in df['raw_datetime']:\n", 217 | " newdate = datetime.fromisoformat(date)\n", 218 | " list.append(newdate)\n", 219 | " df['ISO_datetime'] = list \n", 220 | "\n", 221 | " list = []\n", 222 | " for date in df.ISO_datetime:\n", 223 | " newdate = pd.to_datetime(datetime.strftime(date,'%Y-%m-%d %H:%M:%S'))\n", 224 | " list.append(newdate)\n", 225 | " df['datetime'] = list\n", 226 | "\n", 227 | " df['flow_tph'] = df.flow_Lpm * 0.060\n", 228 | "\n", 229 | " df['timedelta_sec'] = timedelta_seconds(df.datetime, '2020-12-11 09:26:44.448')\n", 230 | "\n", 231 | " df.drop(columns = ['raw_datetime', 'flow_Lpm', 'ISO_datetime'], inplace = True)\n", 232 | "\n", 233 | " return df\n", 234 | "\n", 235 | "\n", 236 | "\n", 237 | "def read_pts(filename):\n", 238 | " '''\n", 239 | " Read PTS-2.xlsx in as a Pandas dataframe and munge for analysis\n", 240 | "\n", 241 | " args: filename is r'PTS-2.xlsx'\n", 242 | "\n", 243 | " returns: Pandas dataframe with datetime (local) and key coloumns of PTS data with the correct dtype\n", 244 | " '''\n", 245 | " df = pd.read_excel(filename)\n", 246 | "\n", 247 | " dict = {\n", 248 | " 'DEPTH':'depth_m',\n", 249 | " 'SPEED': 'speed_mps',\n", 250 | " 'Cable Weight': 'cweight_kg',\n", 251 | " 'WHP': 'whp_barg',\n", 252 | " 'Temperature': 'temp_degC',\n", 253 | " 'Pressure': 'pressure_bara',\n", 254 | " 'Frequency': 'frequency_hz'\n", 255 | " }\n", 256 | " df.rename(columns=dict, inplace=True)\n", 257 | "\n", 258 | " df.drop(0, inplace=True)\n", 259 | " df.reset_index(drop=True, inplace=True)\n", 260 | "\n", 261 | " list = []\n", 262 | " for date in df.Timestamp:\n", 263 | " newdate = openpyxl.utils.datetime.from_excel(date)\n", 264 | " list.append(newdate)\n", 265 | " df['datetime'] = list\n", 266 | "\n", 267 | " df.drop(columns = ['Date', 'Time', 'Timestamp','Reed 0',\n", 268 | " 'Reed 1', 'Reed 2', 'Reed 3', 'Battery Voltage', \n", 269 | " 'PRT Ref Voltage','SGS Voltage', 'Internal Temp 1', \n", 270 | " 'Internal Temp 2', 'Internal Temp 3','Cal Temp', \n", 271 | " 'Error Code 1', 'Error Code 2', 'Error Code 3',\n", 272 | " 'Records Saved', 'Bad Pages',], inplace = True)\n", 273 | " \n", 274 | " df[\n", 275 | " ['depth_m', 'speed_mps','cweight_kg','whp_barg','temp_degC','pressure_bara','frequency_hz']\n", 276 | " ] = df[\n", 277 | " ['depth_m','speed_mps','cweight_kg','whp_barg','temp_degC','pressure_bara','frequency_hz']\n", 278 | " ].apply(pd.to_numeric)\n", 279 | " \n", 280 | " df['timedelta_sec'] = timedelta_seconds(df.datetime, '2020-12-11 09:26:44.448')\n", 281 | "\n", 282 | " return df\n", 283 | "\n", 284 | "\n", 285 | "\n", 286 | "def append_flowrate_to_pts(flowrate_df, pts_df):\n", 287 | " '''\n", 288 | " Add surface flowrate to pts data\n", 289 | "\n", 290 | " Note that the flowrate data is recorded at a courser time resolution than the pts data\n", 291 | " The function makes a linear interpolation to fill the data gaps\n", 292 | " Refer to bonus-combine-data.ipynb to review this method and adapt it for your own data\n", 293 | "\n", 294 | " Args: flowrate and pts dataframes generated by the read_flowrate and read_pts functions\n", 295 | "\n", 296 | " Returns: pts dataframe with flowrate tph added\n", 297 | " \n", 298 | " '''\n", 299 | " flowrate_df = flowrate_df.set_index('timedelta_sec')\n", 300 | " pts_df = pts_df.set_index('timedelta_sec')\n", 301 | " combined_df = pts_df.join(flowrate_df, how = 'outer', lsuffix = '_pts', rsuffix = '_fr')\n", 302 | " combined_df.drop(columns = ['datetime_fr'], inplace = True)\n", 303 | " combined_df.columns = ['depth_m', 'speed_mps', 'cweight_kg', 'whp_barg', 'temp_degC',\n", 304 | " 'pressure_bara', 'frequency_hz', 'datetime', 'flow_tph']\n", 305 | " combined_df['interpolated_flow_tph'] = combined_df['flow_tph'].interpolate(method='linear')\n", 306 | " trimmed_df = combined_df[combined_df['depth_m'].notna()]\n", 307 | " trimmed_df.reset_index(inplace=True)\n", 308 | " return trimmed_df\n", 309 | "\n", 310 | "\n", 311 | "\n", 312 | "def find_index(value, df, colname):\n", 313 | " '''\n", 314 | " Find the dataframe index for the exact matching value or nearest two values\n", 315 | "\n", 316 | " args: value: (float or int) the search term\n", 317 | " df: (obj) the name of the dataframe that is searched\n", 318 | " colname: (str) the name of the coloum this is searched\n", 319 | "\n", 320 | " returns: dataframe index(s) for the matching value or the two adjacent values\n", 321 | " rows can be called from a df using df.iloc[[index_number,index_number]]\n", 322 | " '''\n", 323 | " exactmatch = df[df[colname] == value]\n", 324 | " if not exactmatch.empty:\n", 325 | " return exactmatch.index\n", 326 | " else:\n", 327 | " lowerneighbour_index = df[df[colname] < value][colname].idxmax()\n", 328 | " upperneighbour_index = df[df[colname] > value][colname].idxmin()\n", 329 | " return [lowerneighbour_index, upperneighbour_index] \n", 330 | "\n", 331 | " \n", 332 | "\n", 333 | "def overview_fig(pts_df,flowrate_df,title=''):\n", 334 | " fig, (ax1, ax2, ax3, ax4, ax5, ax6) = plt.subplots(6, 1,figsize=(10,15),sharex=True)\n", 335 | " ax1.set_title(title,y=1.1,fontsize=15)\n", 336 | "\n", 337 | " ax1.plot(flowrate_df.datetime, flowrate_df.flow_tph, label='Surface pump flowrate', \n", 338 | " c='k', linewidth=0.8, marker='.')\n", 339 | " ax1.set_ylabel('Surface flowrate [t/hr]')\n", 340 | " ax1.set_ylim(0,150)\n", 341 | " \n", 342 | " ax2.plot(pts_df.datetime, pts_df.depth_m, label='PTS tool depth', \n", 343 | " c='k', linewidth=0.8)\n", 344 | " ax2.set_ylabel('PTS tool depth [m]')\n", 345 | " ax2.set_ylim(1000,0)\n", 346 | " \n", 347 | " ax3.plot(pts_df.datetime, pts_df.pressure_bara, label='PTS pressure', \n", 348 | " c='tab:blue', linewidth=0.8)\n", 349 | " ax3.set_ylabel('PTS pressure [bara]')\n", 350 | " \n", 351 | " ax4.plot(pts_df.datetime, pts_df.temp_degC, label='PTS temperature', \n", 352 | " c='tab:red', linewidth=0.8)\n", 353 | " ax4.set_ylabel('PTS temperature')\n", 354 | " \n", 355 | " ax5.plot(pts_df.datetime, pts_df.frequency_hz, label='PTS impeller frequency', \n", 356 | " c='tab:green', linewidth=0.8)\n", 357 | " ax5.set_ylim(-30,30)\n", 358 | " ax5.set_ylabel('PTS impeller frequency [hz]')\n", 359 | " # 1 hz = 60 rpm\n", 360 | "\n", 361 | " ax6.plot(pts_df.datetime, pts_df.speed_mps, label='PTS tool speed', \n", 362 | " c='tab:orange', linewidth=0.8)\n", 363 | " ax6.set_ylim(-2,2)\n", 364 | " ax6.set_ylabel('PTS tool speed [mps]')\n", 365 | " \n", 366 | " ax6.set_xlabel('Time [hh:mm]')\n", 367 | " plt.gca().xaxis.set_major_formatter(mdates.DateFormatter('%H:%M'))\n", 368 | " \n", 369 | " for ax in [ax1,ax2,ax3,ax4,ax5,ax6]:\n", 370 | " ax.grid()\n", 371 | " \n", 372 | " return plt" 373 | ] 374 | }, 375 | { 376 | "cell_type": "markdown", 377 | "metadata": { 378 | "iooxa": { 379 | "id": { 380 | "block": "Qm0lajQWUB998JCUjf3F", 381 | "project": "mmReuqVTAa9JzPpNr22I", 382 | "version": 1 383 | } 384 | } 385 | }, 386 | "source": [ 387 | "The cells below will take a little while to run because it includes all steps required to import and munge the data (i.e., everything we did in notebook 1)." 388 | ] 389 | }, 390 | { 391 | "cell_type": "code", 392 | "execution_count": null, 393 | "metadata": { 394 | "iooxa": { 395 | "id": { 396 | "block": "A4W82lnuPbxjRnkih13j", 397 | "project": "mmReuqVTAa9JzPpNr22I", 398 | "version": 1 399 | }, 400 | "outputId": null 401 | } 402 | }, 403 | "outputs": [], 404 | "source": [ 405 | "# Use this method if you are running this notebook in Google Colab\n", 406 | "flowrate = read_flowrate(r'/content/drive/My Drive/T21-Tutorial-WellTestAnalysis-main/Data-FlowRate.xlsx')\n", 407 | "\n", 408 | "# Use this method if you are running this notebook locally (Anaconda)\n", 409 | "#flowrate = read_flowrate(r'Data-FlowRate.xlsx')" 410 | ] 411 | }, 412 | { 413 | "cell_type": "code", 414 | "execution_count": null, 415 | "metadata": { 416 | "iooxa": { 417 | "id": { 418 | "block": "2xKu79NCPc3Uc98xcYVG", 419 | "project": "mmReuqVTAa9JzPpNr22I", 420 | "version": 1 421 | }, 422 | "outputId": null 423 | } 424 | }, 425 | "outputs": [], 426 | "source": [ 427 | "# Use this method if you are running this notebook in Google Colab\n", 428 | "pts = read_pts(r'/content/drive/My Drive/T21-Tutorial-WellTestAnalysis-main/Data-PTS.xlsx')\n", 429 | "\n", 430 | "# Use this method if you are running this notebook locally (Anaconda)\n", 431 | "#pts = read_pts(r'Data-PTS.xlsx')" 432 | ] 433 | }, 434 | { 435 | "cell_type": "markdown", 436 | "metadata": { 437 | "iooxa": { 438 | "id": { 439 | "block": "2On4PwPSZAeAnXmcFrKJ", 440 | "project": "mmReuqVTAa9JzPpNr22I", 441 | "version": 1 442 | } 443 | } 444 | }, 445 | "source": [ 446 | "We use surface pump flow rate and downhole pressure together for our analysis, so we append flow rate to our pts dataframe to make the method simpler. This is done using the append_flowrate_to_pts function from the collection of helper functions above. " 447 | ] 448 | }, 449 | { 450 | "cell_type": "code", 451 | "execution_count": null, 452 | "metadata": { 453 | "iooxa": { 454 | "id": { 455 | "block": "rb9a3TbH9IZNzofj8cax", 456 | "project": "mmReuqVTAa9JzPpNr22I", 457 | "version": 1 458 | }, 459 | "outputId": null 460 | } 461 | }, 462 | "outputs": [], 463 | "source": [ 464 | "pts = append_flowrate_to_pts(flowrate, pts)" 465 | ] 466 | }, 467 | { 468 | "cell_type": "markdown", 469 | "metadata": { 470 | "iooxa": { 471 | "id": { 472 | "block": "rXJCeaRg7QyrUjPaOzhS", 473 | "project": "mmReuqVTAa9JzPpNr22I", 474 | "version": 1 475 | } 476 | } 477 | }, 478 | "source": [ 479 | "## 8.2 Import stable temperature/pressure data for comparison\n", 480 | "\n", 481 | "After drilling and injection testing, geothermal wells heat up over time. \n", 482 | "So for comparison, we have provided a temperature log acquired when the well had reached its stable condition.\n" 483 | ] 484 | }, 485 | { 486 | "cell_type": "code", 487 | "execution_count": null, 488 | "metadata": { 489 | "iooxa": { 490 | "id": { 491 | "block": "MO0wNqo4xUJIbFco80QX", 492 | "project": "mmReuqVTAa9JzPpNr22I", 493 | "version": 1 494 | }, 495 | "outputId": null 496 | } 497 | }, 498 | "outputs": [], 499 | "source": [ 500 | "# Use this method if you are running this notebook in Google Colab\n", 501 | "heating_37days = pd.read_csv(r'/content/drive/My Drive/T21-Tutorial-WellTestAnalysis-main/Data-Temp-Heating37days.csv') \n", 502 | "\n", 503 | "# Use this method if you are running this notebook locally (Anaconda)\n", 504 | "#heating_37days = pd.read_csv('Data-Temp-Heating37days.csv')\n" 505 | ] 506 | }, 507 | { 508 | "cell_type": "code", 509 | "execution_count": null, 510 | "metadata": { 511 | "iooxa": { 512 | "id": { 513 | "block": "5pbUb1lrKx07UgA9wFUm", 514 | "project": "mmReuqVTAa9JzPpNr22I", 515 | "version": 1 516 | }, 517 | "outputId": null 518 | } 519 | }, 520 | "outputs": [], 521 | "source": [ 522 | "# Convert bar gauge to bar atmosphere\n", 523 | "\n", 524 | "heating_37days['pressure_bara'] = heating_37days.pres_barg - 1\n", 525 | "\n", 526 | "heating_37days.head(2)" 527 | ] 528 | }, 529 | { 530 | "cell_type": "markdown", 531 | "metadata": { 532 | "iooxa": { 533 | "id": { 534 | "block": "HfECgu82Qhka9Q1QKiAc", 535 | "project": "mmReuqVTAa9JzPpNr22I", 536 | "version": 1 537 | } 538 | } 539 | }, 540 | "source": [ 541 | "## 8.3 Check the data\n", 542 | "\n", 543 | "It is good practice to check your data after import. \n", 544 | "\n", 545 | "You can use the Pandas methods listed in Section 2.1.1 (1-intro-and-data.ipynb) to check your data. " 546 | ] 547 | }, 548 | { 549 | "cell_type": "code", 550 | "execution_count": null, 551 | "metadata": { 552 | "iooxa": { 553 | "id": { 554 | "block": "DBoGLVWSGnjUmCgeF6IN", 555 | "project": "mmReuqVTAa9JzPpNr22I", 556 | "version": 1 557 | }, 558 | "outputId": null 559 | } 560 | }, 561 | "outputs": [], 562 | "source": [ 563 | "pts.head(2)" 564 | ] 565 | }, 566 | { 567 | "cell_type": "code", 568 | "execution_count": null, 569 | "metadata": { 570 | "iooxa": { 571 | "id": { 572 | "block": "0oyWggUcUEwCyKnGzHG7", 573 | "project": "mmReuqVTAa9JzPpNr22I", 574 | "version": 1 575 | }, 576 | "outputId": null 577 | } 578 | }, 579 | "outputs": [], 580 | "source": [ 581 | "flowrate.head(2)" 582 | ] 583 | }, 584 | { 585 | "cell_type": "code", 586 | "execution_count": null, 587 | "metadata": { 588 | "iooxa": { 589 | "id": { 590 | "block": "pGpKRfLajjDeN7BQxMHN", 591 | "project": "mmReuqVTAa9JzPpNr22I", 592 | "version": 1 593 | }, 594 | "outputId": null 595 | } 596 | }, 597 | "outputs": [], 598 | "source": [ 599 | "heating_37days.head(2)" 600 | ] 601 | }, 602 | { 603 | "cell_type": "markdown", 604 | "metadata": { 605 | "iooxa": { 606 | "id": { 607 | "block": "qOruxRo8LVnLUi69LTzi", 608 | "project": "mmReuqVTAa9JzPpNr22I", 609 | "version": 1 610 | } 611 | } 612 | }, 613 | "source": [ 614 | "We made the plot below to check that we have imported what we expected to import. " 615 | ] 616 | }, 617 | { 618 | "cell_type": "code", 619 | "execution_count": null, 620 | "metadata": { 621 | "iooxa": { 622 | "id": { 623 | "block": "qA75D5hJTEMpEaEyrpa1", 624 | "project": "mmReuqVTAa9JzPpNr22I", 625 | "version": 1 626 | }, 627 | "outputId": null 628 | } 629 | }, 630 | "outputs": [], 631 | "source": [ 632 | "fig, (ax1, ax2) = plt.subplots(1, 2,figsize=(24,8),sharey=True)\n", 633 | "\n", 634 | "ax1.scatter(pts.pressure_bara, pts.depth_m, c = pts.timedelta_sec, s = 5, linewidths = 0)\n", 635 | "\n", 636 | "ax1.plot(heating_37days.pressure_bara, heating_37days.depth_m, c = 'k', label = 'Heating 37 days')\n", 637 | "\n", 638 | "ax1.legend()\n", 639 | "\n", 640 | "ax2.scatter(pts.datetime, pts.depth_m, c = pts.timedelta_sec, s = 5, linewidths = 0)\n", 641 | "\n", 642 | "ax3 = ax2.twinx()\n", 643 | "ax3.plot(flowrate.datetime, flowrate.flow_tph, \n", 644 | " c='k', linestyle = '-', linewidth = 3, alpha = 0.3, \n", 645 | " label='Surface pump flowrate')\n", 646 | "\n", 647 | "ax1.set_ylim(1000,0) # 940,400 \n", 648 | "ax1.set_xlim(0,55) # 20,500\n", 649 | "\n", 650 | "ax1.set_xlabel('Pressure [bara]')\n", 651 | "ax1.set_ylabel('Depth [m]')\n", 652 | "\n", 653 | "ax2.set_xlabel('Time [hh:mm]')\n", 654 | "ax2.xaxis.set_major_formatter(mdates.DateFormatter('%H:%M'))\n", 655 | "\n", 656 | "ax3.set_ylabel('Flowrate [t/hr]')\n", 657 | "\n", 658 | "for ax in [ax1, ax2]:\n", 659 | " ax.grid()" 660 | ] 661 | }, 662 | { 663 | "cell_type": "markdown", 664 | "metadata": { 665 | "iooxa": { 666 | "id": { 667 | "block": "HpdPHqZesnUuvImI5aiX", 668 | "project": "mmReuqVTAa9JzPpNr22I", 669 | "version": 1 670 | } 671 | } 672 | }, 673 | "source": [ 674 | "***\n", 675 | "\n", 676 | "# 9. Pressure inside geothermal wells\n", 677 | "\n", 678 | "Note that the pressure measured inside the well does not equate to the pressure in the reservoir. It varies depending on the liquid level and the density (temperature & phase) of the fluid inside the well. As the well heats up after drilling, pressure profiles will appear to pivot around a single point. That pressure of the pivot point equates to the reservoir pressure at the depth of pivoting. " 679 | ] 680 | }, 681 | { 682 | "cell_type": "markdown", 683 | "metadata": { 684 | "iooxa": { 685 | "id": { 686 | "block": "wYFX7D7BaS997ED83Wh4", 687 | "project": "mmReuqVTAa9JzPpNr22I", 688 | "version": 1 689 | } 690 | } 691 | }, 692 | "source": [ 693 | "***\n", 694 | "\n", 695 | "# 10. Select the most stable pressure values for each flow rate \n", 696 | "\n", 697 | "## 10.1 Interactive plot with ipywidgets\n", 698 | "\n", 699 | "Use the interactive plot to select the times within the completion test program when the pressure down-hole is most likely to be stable for a given flow rate. \n", 700 | "\n", 701 | "The most stable pressure is usually just after the pts tool returned to the programmed hanging depth that was for pressure transient after the well passes are complete and before the pump rate is changed. " 702 | ] 703 | }, 704 | { 705 | "cell_type": "code", 706 | "execution_count": null, 707 | "metadata": { 708 | "iooxa": { 709 | "id": { 710 | "block": "xuSK8ON5isw9Wony7y3n", 711 | "project": "mmReuqVTAa9JzPpNr22I", 712 | "version": 1 713 | }, 714 | "outputId": null 715 | } 716 | }, 717 | "outputs": [], 718 | "source": [ 719 | "\n", 720 | "min_timestamp = pts.timedelta_sec.iloc[0]\n", 721 | "max_timestamp = pts.timedelta_sec.iloc[-1]\n", 722 | "\n", 723 | "def subselect_plot(first_value, second_value, third_value):\n", 724 | " f,ax1 = plt.subplots(1,1, figsize = (20,6))\n", 725 | " ax1.plot(pts.timedelta_sec, pts.depth_m, c = 'k', label = 'PTS tool depth')\n", 726 | " ax2 = ax1.twinx()\n", 727 | " ax2.plot(flowrate.timedelta_sec, flowrate.flow_tph, c='k', linestyle = ':', label='Surface pump flowrate')\n", 728 | " ymin = pts.depth_m.min()\n", 729 | " ymax = pts.depth_m.max() + 100\n", 730 | " ax1.vlines(first_value, ymin, ymax, color='tab:green')\n", 731 | " ax1.vlines(second_value, ymin, ymax, color='tab:orange')\n", 732 | " ax1.vlines(third_value, ymin, ymax, color='tab:red')\n", 733 | " ax1.set_ylim(pts.depth_m.max() + 100, 0)\n", 734 | " ax1.set_xlabel('Time elapsed since the test started [sec]')\n", 735 | " ax2.set_ylabel('Surface pump flowrate [t/hr]')\n", 736 | " ax1.set_ylabel('PTS tool depth [mMD]')\n", 737 | "\n", 738 | "result = interactive(subselect_plot,\n", 739 | " \n", 740 | " first_value = FloatSlider\n", 741 | " (\n", 742 | " value = (max_timestamp - min_timestamp)/6 + min_timestamp,\n", 743 | " description = '1st value',\n", 744 | " min = min_timestamp, \n", 745 | " max = max_timestamp, \n", 746 | " step = 10, \n", 747 | " continuous_update=False,\n", 748 | " layout = Layout(width='80%'),\n", 749 | " ),\n", 750 | " \n", 751 | " second_value = FloatSlider\n", 752 | " (\n", 753 | " value = (max_timestamp - min_timestamp)/4 + min_timestamp, \n", 754 | " description = '2nd value',\n", 755 | " min = min_timestamp, \n", 756 | " max = max_timestamp, \n", 757 | " step = 10, \n", 758 | " continuous_update=False,\n", 759 | " layout = Layout(width='80%')\n", 760 | " ),\n", 761 | "\n", 762 | " third_value = FloatSlider\n", 763 | " (\n", 764 | " value = (max_timestamp - min_timestamp)/2 + min_timestamp, \n", 765 | " description = '3rd value',\n", 766 | " min = min_timestamp, \n", 767 | " max = max_timestamp, \n", 768 | " step = 10, \n", 769 | " continuous_update=False,\n", 770 | " layout = Layout(width='80%')\n", 771 | " )\n", 772 | ")\n", 773 | "\n", 774 | "display(result);\n" 775 | ] 776 | }, 777 | { 778 | "cell_type": "markdown", 779 | "metadata": { 780 | "iooxa": { 781 | "id": { 782 | "block": "xx06lhQWwgjPhYs7ixo7", 783 | "project": "mmReuqVTAa9JzPpNr22I", 784 | "version": 1 785 | } 786 | } 787 | }, 788 | "source": [ 789 | "## 10.2 Call results from the interactive plot\n", 790 | "\n", 791 | "After you place the 1st (green), 2nd (orange), and 3rd (red) line locations, run the cell below to call the results. " 792 | ] 793 | }, 794 | { 795 | "cell_type": "code", 796 | "execution_count": null, 797 | "metadata": { 798 | "iooxa": { 799 | "id": { 800 | "block": "95nWlAGj9C9g5kIdjSKl", 801 | "project": "mmReuqVTAa9JzPpNr22I", 802 | "version": 1 803 | }, 804 | "outputId": null 805 | } 806 | }, 807 | "outputs": [], 808 | "source": [ 809 | "# extract pressure and flow rate at the marked points\n", 810 | "print(\n", 811 | " 'first_timestamp =',result.children[0].value, \n", 812 | " '\\nsecond_timestamp =', result.children[1].value, \n", 813 | " '\\nthird_timestamp =', result.children[2].value, \n", 814 | " )\n" 815 | ] 816 | }, 817 | { 818 | "cell_type": "markdown", 819 | "metadata": { 820 | "iooxa": { 821 | "id": { 822 | "block": "ayifs1FmLCEbbmaOo6hX", 823 | "project": "mmReuqVTAa9JzPpNr22I", 824 | "version": 1 825 | } 826 | } 827 | }, 828 | "source": [ 829 | "## 10.3 Store analysis\n", 830 | "\n", 831 | "Because result.children will change each time you move the sliders in the plot above or re-run this Jupyter Notebook, we copy-paste our selection below. This records your choice and will be the values you do the rest of the interpretation with. " 832 | ] 833 | }, 834 | { 835 | "cell_type": "markdown", 836 | "metadata": { 837 | "iooxa": { 838 | "id": { 839 | "block": "ZulLwvtPRngBKEad39pc", 840 | "project": "mmReuqVTAa9JzPpNr22I", 841 | "version": 1 842 | } 843 | } 844 | }, 845 | "source": [ 846 | "#### Our selected data and some metadata\n", 847 | " \n", 848 | "The third value was selected before the tool made passes because the pumps were shut off so quickly \n", 849 | "that it is difficult to reliably pick the value after the tool passes at this plot scale. \n", 850 | "\n", 851 | "If we were concerned that the pressure was not stable before the logging passes, \n", 852 | "we could take the time to adjust the scale of our plot and pick the correct location." 853 | ] 854 | }, 855 | { 856 | "cell_type": "code", 857 | "execution_count": null, 858 | "metadata": { 859 | "iooxa": { 860 | "id": { 861 | "block": "HGkb9vgz9BX90XXGbBnU", 862 | "project": "mmReuqVTAa9JzPpNr22I", 863 | "version": 1 864 | }, 865 | "outputId": null 866 | } 867 | }, 868 | "outputs": [], 869 | "source": [ 870 | "# define a set of objects using our elapsed time\n", 871 | "# we will use these to extract the data\n", 872 | "\n", 873 | "first_timestamp = 3620.0 \n", 874 | "second_timestamp = 12370.0 \n", 875 | "third_timestamp = 18740.0" 876 | ] 877 | }, 878 | { 879 | "cell_type": "markdown", 880 | "metadata": { 881 | "iooxa": { 882 | "id": { 883 | "block": "uvY08GQfo4IZfjUx4Jkp", 884 | "project": "mmReuqVTAa9JzPpNr22I", 885 | "version": 1 886 | } 887 | } 888 | }, 889 | "source": [ 890 | "***\n", 891 | "\n", 892 | "# 11. Generate an array of stable pressure and flow rate\n", 893 | "\n", 894 | "Recall that we used the append_flowrate_to_pts function from our utilities.py in Section 8.1 above to append surface pump flow rate to our pts dataframe.\n", 895 | "\n", 896 | "## 11.1 Make new dataframes containing the pts log values at our timestamps\n", 897 | "\n", 898 | "Making our new dataframes is a two-step process:\n", 899 | "1. Find the index value where the timestamp is either an exact match to the value we pass in or is the nearest neighbour above and below using the helper function at the start of this notebook\n", 900 | "2. Make new dataframes using the .iloc method and the timestamp returned by step 1 (i.e., the find_index function)" 901 | ] 902 | }, 903 | { 904 | "cell_type": "code", 905 | "execution_count": null, 906 | "metadata": { 907 | "iooxa": { 908 | "id": { 909 | "block": "Pswur9Zh6qQrdCNQVPzz", 910 | "project": "mmReuqVTAa9JzPpNr22I", 911 | "version": 1 912 | }, 913 | "outputId": null 914 | } 915 | }, 916 | "outputs": [], 917 | "source": [ 918 | "first_pts = pts.iloc[find_index(first_timestamp, pts, 'timedelta_sec')]\n", 919 | "second_pts = pts.iloc[find_index(second_timestamp, pts, 'timedelta_sec')]\n", 920 | "third_pts = pts.iloc[find_index(third_timestamp, pts, 'timedelta_sec')]" 921 | ] 922 | }, 923 | { 924 | "cell_type": "code", 925 | "execution_count": null, 926 | "metadata": { 927 | "iooxa": { 928 | "id": { 929 | "block": "EHHyQZ5QjxaabJ2lXVcH", 930 | "project": "mmReuqVTAa9JzPpNr22I", 931 | "version": 1 932 | }, 933 | "outputId": null 934 | } 935 | }, 936 | "outputs": [], 937 | "source": [ 938 | "first_pts" 939 | ] 940 | }, 941 | { 942 | "cell_type": "code", 943 | "execution_count": null, 944 | "metadata": { 945 | "iooxa": { 946 | "id": { 947 | "block": "1q3rqBwNw0evW28Xgg9j", 948 | "project": "mmReuqVTAa9JzPpNr22I", 949 | "version": 1 950 | }, 951 | "outputId": null 952 | } 953 | }, 954 | "outputs": [], 955 | "source": [ 956 | "second_pts" 957 | ] 958 | }, 959 | { 960 | "cell_type": "code", 961 | "execution_count": null, 962 | "metadata": { 963 | "iooxa": { 964 | "id": { 965 | "block": "DeBRHC89PNL9y8lWb8x5", 966 | "project": "mmReuqVTAa9JzPpNr22I", 967 | "version": 1 968 | }, 969 | "outputId": null 970 | } 971 | }, 972 | "outputs": [], 973 | "source": [ 974 | "third_pts" 975 | ] 976 | }, 977 | { 978 | "cell_type": "markdown", 979 | "metadata": { 980 | "iooxa": { 981 | "id": { 982 | "block": "gBVdIH8NzTVhQElaKtHM", 983 | "project": "mmReuqVTAa9JzPpNr22I", 984 | "version": 1 985 | } 986 | } 987 | }, 988 | "source": [ 989 | "## 11.2 Make pressure and flow rate arrays\n", 990 | "\n", 991 | "Now we either use the exact match value or the mean of the two neighbouring values to make an array of pressure and flow rate that we can use in our injectivity index analysis." 992 | ] 993 | }, 994 | { 995 | "cell_type": "code", 996 | "execution_count": null, 997 | "metadata": { 998 | "iooxa": { 999 | "id": { 1000 | "block": "ORTWVJoN5HXjvrsTQFnB", 1001 | "project": "mmReuqVTAa9JzPpNr22I", 1002 | "version": 1 1003 | }, 1004 | "outputId": null 1005 | } 1006 | }, 1007 | "outputs": [], 1008 | "source": [ 1009 | "# make array of mean pressure values\n", 1010 | "\n", 1011 | "list = []\n", 1012 | "for df in [first_pts, second_pts, third_pts]:\n", 1013 | " mean_pressure = df['pressure_bara'].mean()\n", 1014 | " list.append(mean_pressure)\n", 1015 | "pressure_array = np.array(list)\n", 1016 | "\n", 1017 | "print('pressure data =', pressure_array)\n", 1018 | "print('object type =', type(pressure_array))\n" 1019 | ] 1020 | }, 1021 | { 1022 | "cell_type": "code", 1023 | "execution_count": null, 1024 | "metadata": { 1025 | "iooxa": { 1026 | "id": { 1027 | "block": "wuo1cKaHbAt5jUIFQgcM", 1028 | "project": "mmReuqVTAa9JzPpNr22I", 1029 | "version": 1 1030 | }, 1031 | "outputId": null 1032 | } 1033 | }, 1034 | "outputs": [], 1035 | "source": [ 1036 | "# make array of mean flowrate values\n", 1037 | "\n", 1038 | "list = []\n", 1039 | "for df in [first_pts, second_pts, third_pts]:\n", 1040 | " mean_flowrate = df['interpolated_flow_tph'].mean()\n", 1041 | " list.append(mean_flowrate)\n", 1042 | "flowrate_array = np.array(list)\n", 1043 | "\n", 1044 | "print('flowrate data =', flowrate_array)\n", 1045 | "print('object type =', type(flowrate_array))" 1046 | ] 1047 | }, 1048 | { 1049 | "cell_type": "markdown", 1050 | "metadata": { 1051 | "iooxa": { 1052 | "id": { 1053 | "block": "lJ2DIRsF7VGCz8VUAn5f", 1054 | "project": "mmReuqVTAa9JzPpNr22I", 1055 | "version": 1 1056 | } 1057 | } 1058 | }, 1059 | "source": [ 1060 | "***\n", 1061 | "\n", 1062 | "# 12. Generate a linear regression to find injectivity index\n", 1063 | "\n", 1064 | "The injectivity is a measure of total well capacity during injection. It is calculated from the change in pressure (bar) that occurs in response to changing the mass rate (t/hr) that is injected into the well. \n", 1065 | "\n", 1066 | "If there is nothing unusual going on downhole, then we can find this with simple linear regression. There are many cases where a single linear model is not the best approach, such as where changes in pressure or thermal conditions in the well change the permeability. In these cases, a two-slope approach may be better. \n", 1067 | "\n", 1068 | "## 12.1 Make the linear model\n", 1069 | "\n", 1070 | "There are many ways to do a simple linear regression. We selected the stats.linregress method from Scipy because it is relatively simple to use. It's also fast because it is tooled specifically for our two-variable use case. The stats.linregress method returns the slope, intersect and R value that we need for our analysis. " 1071 | ] 1072 | }, 1073 | { 1074 | "cell_type": "code", 1075 | "execution_count": null, 1076 | "metadata": { 1077 | "iooxa": { 1078 | "id": { 1079 | "block": "cJNE7hdVvIE8oYMXf1dR", 1080 | "project": "mmReuqVTAa9JzPpNr22I", 1081 | "version": 1 1082 | }, 1083 | "outputId": null 1084 | } 1085 | }, 1086 | "outputs": [], 1087 | "source": [ 1088 | "linear_model = stats.linregress(pressure_array, flowrate_array) \n", 1089 | "\n", 1090 | "linear_model" 1091 | ] 1092 | }, 1093 | { 1094 | "cell_type": "code", 1095 | "execution_count": null, 1096 | "metadata": { 1097 | "iooxa": { 1098 | "id": { 1099 | "block": "TVla75NTuyUX1qfVTE9u", 1100 | "project": "mmReuqVTAa9JzPpNr22I", 1101 | "version": 1 1102 | }, 1103 | "outputId": null 1104 | } 1105 | }, 1106 | "outputs": [], 1107 | "source": [ 1108 | "# Define some sensibly named objects for reuse below\n", 1109 | "\n", 1110 | "slope = linear_model[0]\n", 1111 | "intercept = linear_model[1]\n", 1112 | "rvalue = linear_model[2]" 1113 | ] 1114 | }, 1115 | { 1116 | "cell_type": "code", 1117 | "execution_count": null, 1118 | "metadata": { 1119 | "iooxa": { 1120 | "id": { 1121 | "block": "ifTNzsjj2SxcccY9W18M", 1122 | "project": "mmReuqVTAa9JzPpNr22I", 1123 | "version": 1 1124 | }, 1125 | "outputId": null 1126 | } 1127 | }, 1128 | "outputs": [], 1129 | "source": [ 1130 | "# Print a nicely formatted string describing of our model\n", 1131 | "\n", 1132 | "print(\"The linear model for our data is y = {:.5} + {:.5} * pressure\"\n", 1133 | " .format(intercept, slope))" 1134 | ] 1135 | }, 1136 | { 1137 | "cell_type": "markdown", 1138 | "metadata": { 1139 | "iooxa": { 1140 | "id": { 1141 | "block": "xcnGi9tggYnhfUgXieyc", 1142 | "project": "mmReuqVTAa9JzPpNr22I", 1143 | "version": 1 1144 | } 1145 | } 1146 | }, 1147 | "source": [ 1148 | "## 12.2 Plot the results" 1149 | ] 1150 | }, 1151 | { 1152 | "cell_type": "code", 1153 | "execution_count": null, 1154 | "metadata": { 1155 | "iooxa": { 1156 | "id": { 1157 | "block": "WODF6RAAHji9MWZYXUhN", 1158 | "project": "mmReuqVTAa9JzPpNr22I", 1159 | "version": 1 1160 | }, 1161 | "outputId": null 1162 | } 1163 | }, 1164 | "outputs": [], 1165 | "source": [ 1166 | "f,ax = plt.subplots(1,1,figsize=(6,4))\n", 1167 | "\n", 1168 | "ax.scatter(pressure_array, flowrate_array, \n", 1169 | " s=100, c='k', label = 'Data')\n", 1170 | "\n", 1171 | "ax.plot(pressure_array, # x values\n", 1172 | " slope * pressure_array + intercept, # use the model to generate y values\n", 1173 | " color='tab:orange', linestyle='-', linewidth=4, alpha=0.5, label='Linear fit')\n", 1174 | "\n", 1175 | "\n", 1176 | "ax.set_title(\"Sample well injectivity index = {:} t/hr.bar\".format(round(slope)))\n", 1177 | "\n", 1178 | "#ax.set_ylim(0,130)\n", 1179 | "#ax.set_xlim(0,40)\n", 1180 | "\n", 1181 | "ax.legend()\n", 1182 | "ax.set_ylabel('Rate [t/hr]')\n", 1183 | "ax.set_xlabel('Pressure [bara]') " 1184 | ] 1185 | }, 1186 | { 1187 | "cell_type": "markdown", 1188 | "metadata": { 1189 | "iooxa": { 1190 | "id": { 1191 | "block": "4NMvebhNSQxhD4xSzwoD", 1192 | "project": "mmReuqVTAa9JzPpNr22I", 1193 | "version": 1 1194 | } 1195 | } 1196 | }, 1197 | "source": [ 1198 | "***\n", 1199 | "\n", 1200 | "# 13. Interpreting injectivity index\n", 1201 | "\n", 1202 | "### 13.1 Understanding the scale of injectivity index\n", 1203 | "\n", 1204 | "Based on data from New Zealand geothermal fields, pre-eminent reservoir engineer Malcolm Grant determined that injectivity index forms a lognormal distribution with a median value around 20 t/hr.bar (Figure 5)." 1205 | ] 1206 | }, 1207 | { 1208 | "cell_type": "code", 1209 | "execution_count": null, 1210 | "metadata": { 1211 | "iooxa": { 1212 | "id": { 1213 | "block": "1Tl17lWE3qtSjy5Xlpjg", 1214 | "project": "mmReuqVTAa9JzPpNr22I", 1215 | "version": 1 1216 | }, 1217 | "outputId": null 1218 | } 1219 | }, 1220 | "outputs": [], 1221 | "source": [ 1222 | "Image('https://raw.githubusercontent.com/ICWallis/T21-Tutorial-WellTestAnalysis/main/Figures/Figure5.png',width = 500,)" 1223 | ] 1224 | }, 1225 | { 1226 | "cell_type": "markdown", 1227 | "metadata": { 1228 | "iooxa": { 1229 | "id": { 1230 | "block": "RSLHZ5ynZRX9pz8DiQOK", 1231 | "project": "mmReuqVTAa9JzPpNr22I", 1232 | "version": 1 1233 | } 1234 | } 1235 | }, 1236 | "source": [ 1237 | "_Figure 5: Like many natural phenomena, geothermal well injectivity has a lognormal distribution. Figure adapted from [Grant (2008)](https://pangea.stanford.edu/ERE/pdf/IGAstandard/SGW/2008/grant.pdf)._\n", 1238 | "\n", 1239 | "There is no 1:1 relationship between the injectivity index (II) and productivity index (PI) of a geothermal well. Each reservoir is subject to a unique set of conditions that influence the relationship between these indices, such as depth to the liquid level, and enthalpy. \n", 1240 | "\n", 1241 | "The table below reflects a general rule of thumb to be used for a reservoir where the local relationship between II and PI has not already been established.\n", 1242 | "\n", 1243 | "|Permeability magnitude | II \\[t/hr.bar\\] |\n", 1244 | "| --- | --- |\n", 1245 | "| Very low permeability (near-conductive) | < 1 |\n", 1246 | "| Poor permeability, usable in special cases | 1 - 5 |\n", 1247 | "| Likely productive | 5 - 20 |\n", 1248 | "| Median production well | 20 |\n", 1249 | "| Reliably economic production for well if T > 250$^{\\circ}$C | 20-50 |\n", 1250 | "| High permeability | 50 - 100 |\n", 1251 | "| Very high permeability (unusual) | > 100 |\n", 1252 | "\n", 1253 | "\n", 1254 | "### 13.2 Injectivity index and temperature\n", 1255 | "\n", 1256 | "The injectivity index is only truly valid for the injectate temperature at which it was measured (i.e., ambient temperature for completion testing). If you inject at a higher temperature (i.e., injection of separated geothermal water or condensate in operational injection wells), then the injectivity index will be less. Empirical corrections are available to adjust injectivity index for temperature - for more information refer to [Siega et al. (2014)](https://www.geothermal-energy.org/pdf/IGAstandard/NZGW/2014/109.Siega.pdf)\n", 1257 | "\n", 1258 | "\n", 1259 | "# 14. Injectivity in our case study well\n", 1260 | "\n", 1261 | "With an II of 252 t/hr.bar, our case study well is extremely permeable! Yah! \n", 1262 | "\n", 1263 | "It is worth noting that as II increases, so does the level of uncertainty in the actual value. This is because the pressure changes resulting from the flow changes become so small. This is a good \"problem\" to have. " 1264 | ] 1265 | }, 1266 | { 1267 | "cell_type": "markdown", 1268 | "metadata": { 1269 | "iooxa": { 1270 | "id": { 1271 | "block": "A6n7OYBhdxS1DUb0sHgV", 1272 | "project": "mmReuqVTAa9JzPpNr22I", 1273 | "version": 1 1274 | } 1275 | } 1276 | }, 1277 | "source": [ 1278 | "***\n", 1279 | "\n", 1280 | "### Cited references\n", 1281 | "\n", 1282 | "Grant, M.A. (2008) Decision tree analysis of possible drilling outcomes to optimise drilling decisions: Proceedings, Thrity-Third Workshop of Geothermal Reservoir Engineering. Stanford University, CA. \n", 1283 | "\n", 1284 | "Siega, C., Grant, M.A., Bixley, P. Mannington, W. (2014) Quantifying the effect of temperature on well injectivity: Proceedings 36th New Zealand Geothermal Workshop. Auckland, NZ.\n", 1285 | "\n", 1286 | "Zarrouk, S.J. and McLean, K. (2019): Geothermal well test analysis: fundamentals, applications, and advanced techniques. 1st edition, Elsevier. \n", 1287 | "\n", 1288 | "***\n", 1289 | "\n", 1290 | "© 2021 [Irene Wallis](https://www.cubicearth.nz/) and [Katie McLean](https://www.linkedin.com/in/katie-mclean-25994315/) \n", 1291 | "\n", 1292 | "Licensed under the Apache License, Version 2.0\n", 1293 | "\n", 1294 | "***" 1295 | ] 1296 | } 1297 | ], 1298 | "metadata": { 1299 | "iooxa": { 1300 | "id": { 1301 | "block": "CZKZK8UMST12MbM5OpK9", 1302 | "project": "mmReuqVTAa9JzPpNr22I", 1303 | "version": 1 1304 | } 1305 | }, 1306 | "kernelspec": { 1307 | "display_name": "geothrm", 1308 | "language": "python", 1309 | "name": "geothrm" 1310 | }, 1311 | "language_info": { 1312 | "codemirror_mode": { 1313 | "name": "ipython", 1314 | "version": 3 1315 | }, 1316 | "file_extension": ".py", 1317 | "mimetype": "text/x-python", 1318 | "name": "python", 1319 | "nbconvert_exporter": "python", 1320 | "pygments_lexer": "ipython3", 1321 | "version": "3.7.9" 1322 | } 1323 | }, 1324 | "nbformat": 4, 1325 | "nbformat_minor": 4 1326 | } 1327 | -------------------------------------------------------------------------------- /Data-FlowRate.xlsx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ICWallis/T21-Tutorial-WellTestAnalysis/18d1b23d753982386318f93b6992396bbd3b5b41/Data-FlowRate.xlsx -------------------------------------------------------------------------------- /Data-PTS.xlsx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ICWallis/T21-Tutorial-WellTestAnalysis/18d1b23d753982386318f93b6992396bbd3b5b41/Data-PTS.xlsx -------------------------------------------------------------------------------- /Data-Temp-Heating37days.csv: -------------------------------------------------------------------------------- 1 | depth_m,whp_barg,pres_barg,temp_degC 2 | 0.0,4.019267731307571,3.929267731307572,10.48 3 | 10.5,4.019267731307571,3.929267731307572,10.86 4 | 20.7,4.019267731307571,3.929267731307572,11.63 5 | 30.4,4.019267731307571,3.939267731307571,12.25 6 | 40.3,4.019267731307571,3.949267731307571,12.77 7 | 50.1,4.019267731307571,3.959267731307571,13.09 8 | 61.1,4.079267731307571,3.959267731307571,13.55 9 | 70.8,4.019267731307571,3.979267731307571,14.26 10 | 80.5,4.019267731307571,3.979267731307571,15.19 11 | 90.3,4.019267731307571,3.979267731307571,16.51 12 | 100.2,4.019267731307571,3.989267731307571,18.12 13 | 110.7,4.019267731307571,3.999267731307571,20.04 14 | 120.4,4.019267731307571,3.999267731307571,22.73 15 | 130.5,4.019267731307571,4.009267731307571,47.2 16 | 140.5,4.019267731307571,4.009267731307571,107.78 17 | 150.7,4.079267731307571,4.009267731307571,131.08 18 | 160.7,4.019267731307571,4.009267731307571,143.74 19 | 170.3,4.019267731307571,4.009267731307571,148.07 20 | 181.1,4.019267731307571,4.019267731307571,150.09 21 | 190.7,4.019267731307571,4.019267731307571,150.96 22 | 200.1,4.019267731307571,4.019267731307571,151.42 23 | 210.4,4.019267731307571,4.019267731307571,151.72 24 | 220.2,4.019267731307571,4.029267731307571,151.9 25 | 230.2,4.019267731307571,4.029267731307571,152.02 26 | 240.2,4.019267731307571,4.029267731307571,152.11 27 | 250.2,4.019267731307571,4.029267731307571,152.18 28 | 260.3,4.019267731307571,4.039267731307571,152.23 29 | 270.2,4.079267731307571,4.039267731307571,152.28 30 | 280.1,4.019267731307571,4.039267731307571,152.33 31 | 291.2,4.019267731307571,4.039267731307571,152.35 32 | 301.1,4.019267731307571,4.039267731307571,152.38 33 | 311.1,4.019267731307571,4.049267731307571,152.39 34 | 320.9,4.019267731307571,4.049267731307571,152.42 35 | 330.4,4.079267731307571,4.049267731307571,152.46 36 | 340.1,4.079267731307571,4.049267731307571,152.46 37 | 350.9,4.019267731307571,4.379267731307571,153.31 38 | 360.2,4.019267731307571,5.089267731307571,157.02 39 | 370.8,4.079267731307571,5.939267731307571,161.87 40 | 380.4,4.019267731307571,6.699267731307571,166.23 41 | 390.1,4.079267731307571,7.459267731307571,170.3 42 | 400.1,4.019267731307571,8.259267731307572,174.27 43 | 411.0,4.019267731307571,9.119267731347573,178.33 44 | 420.7,4.019267731307571,9.869267731347572,181.62 45 | 430.2,4.019267731307571,10.659267731347573,183.91 46 | 441.0,4.019267731307571,11.599267731347572,184.75 47 | 450.4,4.019267731307571,12.399267731347573,185.22 48 | 460.9,4.019267731307571,13.309267731347571,186.87 49 | 470.8,4.019267731307571,14.169267731347572,190.46 50 | 480.9,4.019267731307571,15.029267731347572,194.01 51 | 490.8,4.019267731307571,15.869267731347573,197.26 52 | 500.9,4.019267731307571,16.71926773134757,200.76 53 | 510.8,3.989267731307571,17.55926773134757,203.4 54 | 520.4,4.019267731307571,18.369267731347573,205.59 55 | 530.1,4.019267731307571,19.189267731347574,207.37 56 | 541.0,4.019267731307571,20.09926773134757,209.61 57 | 550.5,4.019267731307571,20.889267731347577,211.68 58 | 560.8,4.019267731307571,21.75926773134757,214.0 59 | 570.3,4.019267731307571,22.549267731347577,216.69 60 | 580.2,4.019267731307571,23.369267731347573,218.8 61 | 590.0,4.019267731307571,24.16926773134757,220.0 62 | 600.0,4.019267731307571,24.98926773134757,221.18 63 | 610.2,4.019267731307571,25.839267731347572,222.16 64 | 620.1,4.019267731307571,26.64926773134757,222.65 65 | 631.0,4.019267731307571,27.549267731347577,222.98 66 | 640.7,4.019267731307571,28.339267731347572,223.16 67 | 651.1,4.019267731307571,29.169267731347567,223.33 68 | 660.8,4.019267731307571,29.979267731347576,223.5 69 | 670.7,4.019267731307571,30.799267731347577,223.62 70 | 680.6,3.989267731307571,31.609267731347575,223.77 71 | 690.6,4.019267731307571,32.42926773134758,223.88 72 | 700.3,3.989267731307571,33.229267731347576,224.14 73 | 711.1,4.019267731307571,34.11926773134758,224.91 74 | 720.7,3.989267731307571,34.91926773134757,226.46 75 | 730.3,4.019267731307571,35.68926773134758,228.46 76 | 741.1,4.019267731307571,36.569267731347566,230.04 77 | 750.5,4.019267731307571,37.339267731347576,230.43 78 | 760.8,4.019267731307571,38.16926773134757,230.51 79 | 770.3,3.989267731307571,38.93926773134758,230.52 80 | 780.1,4.019267731307571,39.729267731347576,230.54 81 | 791.2,3.989267731307571,40.629267731347575,230.54 82 | 801.2,4.019267731307571,41.459267731347566,230.56 83 | 811.2,3.989267731307571,42.25926773134758,230.51 84 | 820.1,3.989267731307571,42.98926773134757,230.53 85 | 831.0,4.019267731307571,43.879267731347575,230.56 86 | 840.7,4.019267731307571,44.66926773134757,230.58 87 | 850.4,3.989267731307571,45.459267731347566,230.62 88 | 860.1,4.019267731307571,46.24926773134758,230.66 89 | 870.8,4.019267731307571,47.11926773134758,230.68 90 | 880.0,4.019267731307571,47.879267731347575,230.73 91 | 890.8,4.019267731307571,48.75926773134758,230.79 92 | 900.7,4.019267731307571,49.54926773134758,230.82 93 | 910.5,4.019267731307571,50.35926773134758,230.81 94 | 920.4,4.079267731307571,51.16926773134757,230.56 95 | 926.4,4.019267731307571,51.66926773134757,230.4 96 | -------------------------------------------------------------------------------- /Figures/Figure1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ICWallis/T21-Tutorial-WellTestAnalysis/18d1b23d753982386318f93b6992396bbd3b5b41/Figures/Figure1.png -------------------------------------------------------------------------------- /Figures/Figure2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ICWallis/T21-Tutorial-WellTestAnalysis/18d1b23d753982386318f93b6992396bbd3b5b41/Figures/Figure2.jpg -------------------------------------------------------------------------------- /Figures/Figure3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ICWallis/T21-Tutorial-WellTestAnalysis/18d1b23d753982386318f93b6992396bbd3b5b41/Figures/Figure3.png -------------------------------------------------------------------------------- /Figures/Figure4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ICWallis/T21-Tutorial-WellTestAnalysis/18d1b23d753982386318f93b6992396bbd3b5b41/Figures/Figure4.png -------------------------------------------------------------------------------- /Figures/Figure5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ICWallis/T21-Tutorial-WellTestAnalysis/18d1b23d753982386318f93b6992396bbd3b5b41/Figures/Figure5.png -------------------------------------------------------------------------------- /Figures/Table1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ICWallis/T21-Tutorial-WellTestAnalysis/18d1b23d753982386318f93b6992396bbd3b5b41/Figures/Table1.png -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "[]" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright [yyyy] [name of copyright owner] 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Transform 2021 Tutorial
Geothermal Well Test Analysis with Python 2 | 3 | This tutorial was developed by Irene Wallis and Katie McLean, industry practitioners with extensive experience in geothermal well logging and testing. It will live-stream to YouTube during the SWUG Transform 2021 conference and will be available through the SWUG YouTube channel for viewing later. 4 | 5 | **The tutorial will cover:** 6 | - General introduction to geothermal wells 7 | - Overview of the completion test process 8 | - Introduction to temperature log analysis 9 | - Injectivitly index determination (i.e., well capacity in t/hr.bar) 10 | - Spinner data analysis (i.e., feedzone identification) 11 | 12 | Production tests and pressure transient analysis are not covered. 13 | 14 | Python offers us the opportunity to generate a robust and repeatable well test analysis that is easy to audit. This is important given the high-value decisions based on these analyses. 15 | We will demo how the Jupyter Notebook environment enables us to store the steps taken to process and interpret data along with key metadata, things that are difficult to do reliability in Excel. 16 | 17 | **You should attend this tutorial if you are:** 18 | - Interested in geothermal wells and how they are characterised using injection testing and temperature logs 19 | - A practising geothermal reservoir engineer who is interested in using python for well test analysis 20 | 21 | The tutorial uses methods at an intermediate python level but, because all code is provided, a novice python user can follow along. 22 | 23 | **For more information,** check out a book co-authored by Katie with Sadiq Zarrouk that covers the theory and practice of geothermal well test analysis https://www.elsevier.com/books/geothermal-well-test-analysis/zarrouk/978-0-12-814946-1 24 | 25 | *** 26 | 27 | **Here's how you participate in this event:** 28 | 29 | 1. Follow this link and register for the Transform 2021 unconference https://softwareunderground.org/events/transform-2021. 30 | 31 | 2. If you're not already a member, join the [Software Underground slack community](https://softwareunderground.org/slack). 32 | 33 | 3. Join the tutorial slack channel #t21-thurs-geothermal because this is where you can ask questions during the tutorial and meet others who are attending. 34 | 35 | 4. Decide how you want to interact with the tutorial. You will be able to read along in Curvenote, try out the method yourself without installing anything by using Google Colaboratory, or instal Anaconda to run the tutorial notebooks locally on your computer. More detailed setup instructions are provided below. 36 | 37 | 5. Watch the tutorial live stream to YouTube (https://www.youtube.com/watch?v=VEKrTV89Ln8) **The tutorial livestream will be UTC Thursday 22 April 22:00** but will also be freely available after the event. 38 | 39 | 6. After the livestream, come meet Irene and Katie, and other tutorial participants, in the Transform 2021 social and hackathon space that's been built using GatherTown (https://gather.town/). A link to this space is included in the topic of the tutorial slack channel (hover over the bar at the top to reveal the gather.town link). 40 | 41 | *** 42 | 43 | # Setup instructions 44 | 45 | As well as the livestream to YouTube, you will be able to interact with the tutorial in three ways: 46 | 47 | - Read the notebooks at Curvenote by following [this link](https://curvenote.com/@swung/geothermal-well-test-analysis-transform-2021). Big thanks to [Steve Purves](https://github.com/stevejpurves) for setting this up for us. 48 | 49 | - Follow the Google Colaboratory instructions below to interact with the notebooks without installing anything. Big thanks to [Thomas Martin](https://github.com/ThomasMGeo) for the Google Colab advise. 50 | 51 | - Follow the Anaconda Setup instructions below to run the notebooks locally on your computer. Big thanks to [Martin Bentley](https://github.com/mtb-za) for testing the local instal method and helping with datetime debugging. 52 | 53 | 54 | *** 55 | ## Google Colaboratory Setup Instructions 56 | 57 | Google Colab is a way to run the tutorial Jupyter Notebooks (.ipynb) in the cloud. It is free to use but, as is typical of Google products, you have to sign in. We will use the free Google Colab (there is no need to sign up for Google Colab Pro). 58 | 59 | 1. If you do not already have one, you will need a Google account. If you have a gmail address, you already have a google account. 60 | 2. Download the content of this repository by clicking on the green "Code" button above and selecting "Download zip" 61 | 3. Unzip the T21-Tutorial-WellTestAnalysis-main.zip 62 | 4. Sign into your [Google Drive](http://www.drive.google.com/) using your Google account details and upload the unzipped T21-Tutorial-WellTestAnalysis-main folder to google drive. Note that you need to upload into the browser google drive, not the local Google Drive on your hard drive (if you happen to have one). 63 | 5. Open the T21-Tutorial-WellTestAnalysis-main on Google Drive, right click any file that ends with .ipynb and select "open with". If Google Colaboratory appears in your list, then select this. If Google Colaboratory is not already an option, you need to install this into your Google Drive. To install Google Colaboratory: 1) Right click an .ipynb -> open with -> connect more apps, 2) search the Google Workspace Marketplace for 'Colaboratory', and 3) install Google Colaboratory. Now you will be able to go back to your .ipynb and open with Google Colaboratory. 64 | 65 | After you have opened the Jupyter Notebooks (.ipynb) in Colab, you will need to mount your Google Drive (i.e., make Google Drive visible to the notebook) so you can import data. Instructions for how to do this is included in the top of each notebook after the introductory text. 66 | 67 | *** 68 | ## Anaconda Setup Instructions 69 | 70 | To run this tutorial, you will need an environment that contains all of the required packages. If you are familiar with setting up your own environment, then go ahead with your usual approach. Otherwise, use the following steps. 71 | 72 | Download this tutorial from GitHub using the green 'code' button and unzip to somewhere that is easy to find, such as your Documents folder. 73 | 74 | Download and instal [Anaconda individual edition](https://www.anaconda.com/products/individual) if you don't already have it. When prompted, accept the default installation settings. 75 | 76 | In Windows open the anaconda prompt or in macOS open a terminal. Use the prompt/terminal to navigate to where you have saved this tutorial (hint: use _cd \_ to change directory) 77 | 78 | In the tutorial folder, you will find an environment.yml file (hint: use _ls_ in MacOS or _dir_ in Windows to list files in your current directory). We will use this file to create an environment that will run the tutorial. Execute the following command in the prompt/terminal to create the environment: 79 | 80 | $ conda env create -f environment.yml 81 | 82 | You will see a lot of text scroll past in the the prompt/terminal and may need to respond y + ENTER at some point. The environment is automatically named geothrm. Once it has built, we need to activate the environment by executing: 83 | 84 | $ conda activate geothrm 85 | 86 | \(geotherm\) should now appear on the far left of your current line in the prompt/terminal window. Now you are inside the right environment, you can execute the following command to launch a browser window containing Juypter notebook: 87 | 88 | $ jupyter notebook 89 | 90 | Now you can open the tutorial notebooks and use them. 91 | 92 | When you are finished with Juypter Notebook, you can close the browser window and go back to the prompt/terminal to kill the process with CTRL + C. 93 | 94 | When you come back to the tutorial at a later date, you will probably have to activate the geothrm environment again before launching Juypter Notebook. 95 | 96 | **Other useful commands** 97 | 98 | Print a list of your conda environments 99 | 100 | $ conda env list 101 | 102 | Print a list of what is inside your active environment 103 | 104 | $ conda list 105 | 106 | To install an additional package into the active environment 107 | 108 | $ pip install -------------------------------------------------------------------------------- /bonus-combine-data.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": { 6 | "iooxa": { 7 | "id": { 8 | "block": "wUFcCPb85Y5NTvEDOreS", 9 | "project": "mmReuqVTAa9JzPpNr22I", 10 | "version": 1 11 | } 12 | } 13 | }, 14 | "source": [ 15 | "***\n", 16 | "\n", 17 | "# Geothermal Well Test Analysis with Python\n", 18 | "### Bonus material: Combine PTS and pump flow rate data\n", 19 | "#### Irene Wallis and Katie McLean \n", 20 | "#### Software Underground, Transform 2021\n", 21 | "\n", 22 | "***" 23 | ] 24 | }, 25 | { 26 | "cell_type": "code", 27 | "execution_count": 13, 28 | "metadata": { 29 | "iooxa": { 30 | "id": { 31 | "block": "H7awF9Z0d62cuLrMu6dr", 32 | "project": "mmReuqVTAa9JzPpNr22I", 33 | "version": 1 34 | }, 35 | "outputId": null 36 | } 37 | }, 38 | "outputs": [], 39 | "source": [ 40 | "import pandas as pd\n", 41 | "import matplotlib.pyplot as plt\n", 42 | "import matplotlib.dates as mdates\n", 43 | "from datetime import datetime\n", 44 | "import openpyxl\n", 45 | "\n", 46 | "\n", 47 | "#from utilities import* # functions in the utilities.py file" 48 | ] 49 | }, 50 | { 51 | "cell_type": "code", 52 | "execution_count": 14, 53 | "metadata": {}, 54 | "outputs": [], 55 | "source": [ 56 | "def timedelta_seconds(dataframe_col, test_start):\n", 57 | " '''\n", 58 | " Make a float in seconds since the start of the test\n", 59 | "\n", 60 | " args: dataframe_col: dataframe column containing datetime objects\n", 61 | " test_start: test start time formatted '2020-12-11 09:00:00'\n", 62 | "\n", 63 | " returns: float in seconds since the start of the test\n", 64 | " '''\n", 65 | " test_start_datetime = pd.to_datetime(test_start)\n", 66 | " list = []\n", 67 | " for datetime in dataframe_col:\n", 68 | " time_delta = datetime - test_start_datetime\n", 69 | " seconds = time_delta.total_seconds()\n", 70 | " list.append(seconds)\n", 71 | " return list\n", 72 | "\n", 73 | "\n", 74 | "\n", 75 | "def read_pts(filename):\n", 76 | " '''\n", 77 | " Read PTS-2.xlsx in as a Pandas dataframe and munge for analysis\n", 78 | "\n", 79 | " args: filename is r'PTS-2.xlsx'\n", 80 | "\n", 81 | " returns: Pandas dataframe with datetime (local) and key coloumns of PTS data with the correct dtype\n", 82 | " '''\n", 83 | " df = pd.read_excel(filename)\n", 84 | "\n", 85 | " dict = {\n", 86 | " 'DEPTH':'depth_m',\n", 87 | " 'SPEED': 'speed_mps',\n", 88 | " 'Cable Weight': 'cweight_kg',\n", 89 | " 'WHP': 'whp_barg',\n", 90 | " 'Temperature': 'temp_degC',\n", 91 | " 'Pressure': 'pressure_bara',\n", 92 | " 'Frequency': 'frequency_hz'\n", 93 | " }\n", 94 | " df.rename(columns=dict, inplace=True)\n", 95 | "\n", 96 | " df.drop(0, inplace=True)\n", 97 | " df.reset_index(drop=True, inplace=True)\n", 98 | "\n", 99 | " list = []\n", 100 | " for date in df.Timestamp:\n", 101 | " newdate = openpyxl.utils.datetime.from_excel(date)\n", 102 | " list.append(newdate)\n", 103 | " df['datetime'] = list\n", 104 | "\n", 105 | " df.drop(columns = ['Date', 'Time', 'Timestamp','Reed 0',\n", 106 | " 'Reed 1', 'Reed 2', 'Reed 3', 'Battery Voltage', \n", 107 | " 'PRT Ref Voltage','SGS Voltage', 'Internal Temp 1', \n", 108 | " 'Internal Temp 2', 'Internal Temp 3','Cal Temp', \n", 109 | " 'Error Code 1', 'Error Code 2', 'Error Code 3',\n", 110 | " 'Records Saved', 'Bad Pages',], inplace = True)\n", 111 | " \n", 112 | " df[\n", 113 | " ['depth_m', 'speed_mps','cweight_kg','whp_barg','temp_degC','pressure_bara','frequency_hz']\n", 114 | " ] = df[\n", 115 | " ['depth_m','speed_mps','cweight_kg','whp_barg','temp_degC','pressure_bara','frequency_hz']\n", 116 | " ].apply(pd.to_numeric)\n", 117 | " \n", 118 | " df['timedelta_sec'] = timedelta_seconds(df.datetime, '2020-12-11 09:26:44.448')\n", 119 | "\n", 120 | " return df\n", 121 | "\n", 122 | "\n", 123 | "\n", 124 | "def read_flowrate(filename):\n", 125 | " ''' \n", 126 | " Read PTS-2-injection-rate.xlsx in as a pandas dataframe and munge for analysis\n", 127 | "\n", 128 | " args: filename is r'PTS-2-injection-rate.xlsx'\n", 129 | "\n", 130 | " returns: pandas dataframe with local NZ datetime and flowrate in t/hr\n", 131 | " '''\n", 132 | " df = pd.read_excel(filename, header=1) \n", 133 | " df.columns = ['raw_datetime','flow_Lpm']\n", 134 | "\n", 135 | " list = []\n", 136 | " for date in df['raw_datetime']:\n", 137 | " newdate = datetime.fromisoformat(date)\n", 138 | " list.append(newdate)\n", 139 | " df['ISO_datetime'] = list \n", 140 | "\n", 141 | " list = []\n", 142 | " for date in df.ISO_datetime:\n", 143 | " newdate = pd.to_datetime(datetime.strftime(date,'%Y-%m-%d %H:%M:%S'))\n", 144 | " list.append(newdate)\n", 145 | " df['datetime'] = list\n", 146 | "\n", 147 | " df['flow_tph'] = df.flow_Lpm * 0.060\n", 148 | "\n", 149 | " df['timedelta_sec'] = timedelta_seconds(df.datetime, '2020-12-11 09:26:44.448')\n", 150 | "\n", 151 | " df.drop(columns = ['raw_datetime', 'flow_Lpm', 'ISO_datetime'], inplace = True)\n", 152 | "\n", 153 | " return df" 154 | ] 155 | }, 156 | { 157 | "cell_type": "code", 158 | "execution_count": 15, 159 | "metadata": { 160 | "iooxa": { 161 | "id": { 162 | "block": "RVwCyhNC72F630z843N1", 163 | "project": "mmReuqVTAa9JzPpNr22I", 164 | "version": 1 165 | }, 166 | "outputId": null 167 | }, 168 | "tags": [] 169 | }, 170 | "outputs": [], 171 | "source": [ 172 | "flowrate = read_flowrate(r'Data-FlowRate.xlsx')\n", 173 | "pts = read_pts(r'Data-PTS.xlsx')" 174 | ] 175 | }, 176 | { 177 | "cell_type": "code", 178 | "execution_count": 16, 179 | "metadata": { 180 | "iooxa": { 181 | "id": { 182 | "block": "WDajMKHXZ57KYRTDtnnw", 183 | "project": "mmReuqVTAa9JzPpNr22I", 184 | "version": 1 185 | }, 186 | "outputId": null 187 | } 188 | }, 189 | "outputs": [ 190 | { 191 | "data": { 192 | "text/plain": [ 193 | "Index(['datetime', 'flow_tph', 'timedelta_sec'], dtype='object')" 194 | ] 195 | }, 196 | "execution_count": 16, 197 | "metadata": {}, 198 | "output_type": "execute_result" 199 | } 200 | ], 201 | "source": [ 202 | "flowrate.columns" 203 | ] 204 | }, 205 | { 206 | "cell_type": "code", 207 | "execution_count": 17, 208 | "metadata": { 209 | "iooxa": { 210 | "id": { 211 | "block": "6Lem3ULLT74NWhsTbW2D", 212 | "project": "mmReuqVTAa9JzPpNr22I", 213 | "version": 1 214 | }, 215 | "outputId": null 216 | } 217 | }, 218 | "outputs": [ 219 | { 220 | "ename": "KeyError", 221 | "evalue": "\"None of ['timestamp'] are in the columns\"", 222 | "output_type": "error", 223 | "traceback": [ 224 | "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", 225 | "\u001b[1;31mKeyError\u001b[0m Traceback (most recent call last)", 226 | "\u001b[1;32m\u001b[0m in \u001b[0;36m\u001b[1;34m\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mflowrate\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mflowrate\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mset_index\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m'timestamp'\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 2\u001b[0m \u001b[0mflowrate\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mhead\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;36m3\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", 227 | "\u001b[1;32m~\\anaconda3\\lib\\site-packages\\pandas\\core\\frame.py\u001b[0m in \u001b[0;36mset_index\u001b[1;34m(self, keys, drop, append, inplace, verify_integrity)\u001b[0m\n\u001b[0;32m 4725\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 4726\u001b[0m \u001b[1;32mif\u001b[0m \u001b[0mmissing\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 4727\u001b[1;33m \u001b[1;32mraise\u001b[0m \u001b[0mKeyError\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34mf\"None of {missing} are in the columns\"\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 4728\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 4729\u001b[0m \u001b[1;32mif\u001b[0m \u001b[0minplace\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", 228 | "\u001b[1;31mKeyError\u001b[0m: \"None of ['timestamp'] are in the columns\"" 229 | ] 230 | } 231 | ], 232 | "source": [ 233 | "flowrate = flowrate.set_index('timestamp')\n", 234 | "flowrate.head(3)" 235 | ] 236 | }, 237 | { 238 | "cell_type": "code", 239 | "execution_count": null, 240 | "metadata": { 241 | "iooxa": { 242 | "id": { 243 | "block": "pRY08VPzKQ4uHERkL9es", 244 | "project": "mmReuqVTAa9JzPpNr22I", 245 | "version": 1 246 | }, 247 | "outputId": null 248 | } 249 | }, 250 | "outputs": [], 251 | "source": [ 252 | "pts = pts.set_index('timestamp')\n", 253 | "pts.head(3)" 254 | ] 255 | }, 256 | { 257 | "cell_type": "code", 258 | "execution_count": null, 259 | "metadata": { 260 | "iooxa": { 261 | "id": { 262 | "block": "3VMCxPrHADDYNbiu2zKR", 263 | "project": "mmReuqVTAa9JzPpNr22I", 264 | "version": 1 265 | }, 266 | "outputId": null 267 | } 268 | }, 269 | "outputs": [], 270 | "source": [ 271 | "combined = pts.join(flowrate, how = 'outer', lsuffix = '_pts', rsuffix = '_fr', )" 272 | ] 273 | }, 274 | { 275 | "cell_type": "code", 276 | "execution_count": null, 277 | "metadata": { 278 | "iooxa": { 279 | "id": { 280 | "block": "1wal7NRqmZm3pSihXbHk", 281 | "project": "mmReuqVTAa9JzPpNr22I", 282 | "version": 1 283 | }, 284 | "outputId": null 285 | } 286 | }, 287 | "outputs": [], 288 | "source": [ 289 | "combined.columns" 290 | ] 291 | }, 292 | { 293 | "cell_type": "code", 294 | "execution_count": null, 295 | "metadata": { 296 | "iooxa": { 297 | "id": { 298 | "block": "cG9tfyOR33pM4IKIq41v", 299 | "project": "mmReuqVTAa9JzPpNr22I", 300 | "version": 1 301 | }, 302 | "outputId": null 303 | } 304 | }, 305 | "outputs": [], 306 | "source": [ 307 | "combined.head(3)" 308 | ] 309 | }, 310 | { 311 | "cell_type": "code", 312 | "execution_count": null, 313 | "metadata": { 314 | "iooxa": { 315 | "id": { 316 | "block": "fiaeF2o06lNDtplRlFCG", 317 | "project": "mmReuqVTAa9JzPpNr22I", 318 | "version": 1 319 | }, 320 | "outputId": null 321 | } 322 | }, 323 | "outputs": [], 324 | "source": [ 325 | "combined.drop(columns = ['datetime_fr'], inplace = True)" 326 | ] 327 | }, 328 | { 329 | "cell_type": "code", 330 | "execution_count": null, 331 | "metadata": { 332 | "iooxa": { 333 | "id": { 334 | "block": "9YYMdBYV3tT0YKbddzTY", 335 | "project": "mmReuqVTAa9JzPpNr22I", 336 | "version": 1 337 | }, 338 | "outputId": null 339 | } 340 | }, 341 | "outputs": [], 342 | "source": [ 343 | "combined.columns" 344 | ] 345 | }, 346 | { 347 | "cell_type": "code", 348 | "execution_count": null, 349 | "metadata": { 350 | "iooxa": { 351 | "id": { 352 | "block": "0W2UP8ceIf5LjDbDHHR1", 353 | "project": "mmReuqVTAa9JzPpNr22I", 354 | "version": 1 355 | }, 356 | "outputId": null 357 | } 358 | }, 359 | "outputs": [], 360 | "source": [ 361 | "combined.columns = ['depth_m', 'speed_mps', 'cweight_kg', 'whp_barg', 'temp_degC',\n", 362 | " 'pressure_bara', 'frequency_hz', 'datetime', 'flow_tph']" 363 | ] 364 | }, 365 | { 366 | "cell_type": "code", 367 | "execution_count": null, 368 | "metadata": { 369 | "iooxa": { 370 | "id": { 371 | "block": "XcTnMkWP7RlshjC5g4TP", 372 | "project": "mmReuqVTAa9JzPpNr22I", 373 | "version": 1 374 | }, 375 | "outputId": null 376 | } 377 | }, 378 | "outputs": [], 379 | "source": [ 380 | "combined.columns" 381 | ] 382 | }, 383 | { 384 | "cell_type": "code", 385 | "execution_count": null, 386 | "metadata": { 387 | "iooxa": { 388 | "id": { 389 | "block": "BOaXuiVjf8IVfv7ZhMPj", 390 | "project": "mmReuqVTAa9JzPpNr22I", 391 | "version": 1 392 | }, 393 | "outputId": null 394 | } 395 | }, 396 | "outputs": [], 397 | "source": [ 398 | "combined.describe()" 399 | ] 400 | }, 401 | { 402 | "cell_type": "code", 403 | "execution_count": null, 404 | "metadata": { 405 | "iooxa": { 406 | "id": { 407 | "block": "ujWpWcyMhdMKiPoJKSZ5", 408 | "project": "mmReuqVTAa9JzPpNr22I", 409 | "version": 1 410 | }, 411 | "outputId": null 412 | } 413 | }, 414 | "outputs": [], 415 | "source": [ 416 | "combined['flow_tph'].isna().sum()" 417 | ] 418 | }, 419 | { 420 | "cell_type": "code", 421 | "execution_count": null, 422 | "metadata": { 423 | "iooxa": { 424 | "id": { 425 | "block": "bTbUnJF8AqzEmB67G8gi", 426 | "project": "mmReuqVTAa9JzPpNr22I", 427 | "version": 1 428 | }, 429 | "outputId": null 430 | } 431 | }, 432 | "outputs": [], 433 | "source": [ 434 | "combined['new_flow_tph'] = combined['flow_tph'].interpolate(method='linear')" 435 | ] 436 | }, 437 | { 438 | "cell_type": "code", 439 | "execution_count": null, 440 | "metadata": { 441 | "iooxa": { 442 | "id": { 443 | "block": "CxoYfuWE73HYQYE9Spgp", 444 | "project": "mmReuqVTAa9JzPpNr22I", 445 | "version": 1 446 | }, 447 | "outputId": null 448 | } 449 | }, 450 | "outputs": [], 451 | "source": [ 452 | "combined['new_flow_tph'].isna().sum()" 453 | ] 454 | }, 455 | { 456 | "cell_type": "code", 457 | "execution_count": null, 458 | "metadata": { 459 | "iooxa": { 460 | "id": { 461 | "block": "aHqKKTZBprw7DlZju790", 462 | "project": "mmReuqVTAa9JzPpNr22I", 463 | "version": 1 464 | }, 465 | "outputId": null 466 | } 467 | }, 468 | "outputs": [], 469 | "source": [ 470 | "combined.tail()" 471 | ] 472 | }, 473 | { 474 | "cell_type": "code", 475 | "execution_count": null, 476 | "metadata": { 477 | "iooxa": { 478 | "id": { 479 | "block": "vbp1plx87vU2dGk26vZy", 480 | "project": "mmReuqVTAa9JzPpNr22I", 481 | "version": 1 482 | }, 483 | "outputId": null 484 | } 485 | }, 486 | "outputs": [], 487 | "source": [ 488 | "fig, (ax) = plt.subplots(1, 1, figsize=(24,8))\n", 489 | "\n", 490 | "ax.scatter(combined.datetime, combined.new_flow_tph, \n", 491 | " c='k', s = 5, linewidths = 0)\n", 492 | "\n", 493 | "ax.scatter(combined.datetime, combined.flow_tph, \n", 494 | " c='r', s = 30, linewidths = 0)\n", 495 | "\n", 496 | "ax.scatter(pts.datetime, pts.cweight_kg, \n", 497 | " c='b', s = 5, linewidths = 0, label = 'Raw cabel weight')\n", 498 | "\n", 499 | "ax.scatter(remaining_nan.datetime, remaining_nan.cweight_kg, \n", 500 | " c='g', s = 30, linewidths = 0, label = 'NAN cabel weight')\n", 501 | "\n", 502 | "ax.set_ylim(0,150)\n", 503 | "\n", 504 | "ax.set_xlabel('Time [hh:mm]')\n", 505 | "ax.xaxis.set_major_formatter(mdates.DateFormatter('%H:%M'))\n", 506 | "\n", 507 | "\n", 508 | "start_time = pd.to_datetime('2020-12-11 09:25:00')\n", 509 | "end_time = pd.to_datetime('2020-12-11 09:30:00')\n", 510 | "\n", 511 | "#ax.set_xlim(start_time,end_time);" 512 | ] 513 | }, 514 | { 515 | "cell_type": "markdown", 516 | "metadata": { 517 | "iooxa": { 518 | "id": { 519 | "block": "jfNGHUo2E9esn0S3c9C1", 520 | "project": "mmReuqVTAa9JzPpNr22I", 521 | "version": 1 522 | } 523 | } 524 | }, 525 | "source": [ 526 | "As the plot and the remaining_nan.describe() show that I have filled all the PTS values with flow rates, I will now drop the flow values that have no pts value" 527 | ] 528 | }, 529 | { 530 | "cell_type": "code", 531 | "execution_count": null, 532 | "metadata": { 533 | "iooxa": { 534 | "id": { 535 | "block": "IKSL8aysVGD2z7R9GAL0", 536 | "project": "mmReuqVTAa9JzPpNr22I", 537 | "version": 1 538 | }, 539 | "outputId": null 540 | } 541 | }, 542 | "outputs": [], 543 | "source": [ 544 | "nan_trimmed = combined.dropna()\n", 545 | "\n", 546 | "nan_trimmed = combined[combined['depth_m'].notna()]\n" 547 | ] 548 | }, 549 | { 550 | "cell_type": "code", 551 | "execution_count": null, 552 | "metadata": { 553 | "iooxa": { 554 | "id": { 555 | "block": "j0XRar4L3uj15MEJlHvT", 556 | "project": "mmReuqVTAa9JzPpNr22I", 557 | "version": 1 558 | }, 559 | "outputId": null 560 | } 561 | }, 562 | "outputs": [], 563 | "source": [ 564 | "nan_trimmed.shape " 565 | ] 566 | }, 567 | { 568 | "cell_type": "code", 569 | "execution_count": null, 570 | "metadata": { 571 | "iooxa": { 572 | "id": { 573 | "block": "XguCBTRhojITwWoFqX9Y", 574 | "project": "mmReuqVTAa9JzPpNr22I", 575 | "version": 1 576 | }, 577 | "outputId": null 578 | } 579 | }, 580 | "outputs": [], 581 | "source": [ 582 | "combined.shape" 583 | ] 584 | }, 585 | { 586 | "cell_type": "code", 587 | "execution_count": null, 588 | "metadata": { 589 | "iooxa": { 590 | "id": { 591 | "block": "eIKtqMZ4rvehUDzEXxeU", 592 | "project": "mmReuqVTAa9JzPpNr22I", 593 | "version": 1 594 | }, 595 | "outputId": null 596 | } 597 | }, 598 | "outputs": [], 599 | "source": [ 600 | "nan_trimmed.reset_index(inplace=True)" 601 | ] 602 | }, 603 | { 604 | "cell_type": "code", 605 | "execution_count": null, 606 | "metadata": { 607 | "iooxa": { 608 | "id": { 609 | "block": "88qnZ53E6M9yll14TSBS", 610 | "project": "mmReuqVTAa9JzPpNr22I", 611 | "version": 1 612 | }, 613 | "outputId": null 614 | } 615 | }, 616 | "outputs": [], 617 | "source": [ 618 | "nan_trimmed.head(2)" 619 | ] 620 | }, 621 | { 622 | "cell_type": "code", 623 | "execution_count": null, 624 | "metadata": {}, 625 | "outputs": [], 626 | "source": [] 627 | }, 628 | { 629 | "cell_type": "code", 630 | "execution_count": null, 631 | "metadata": { 632 | "iooxa": { 633 | "id": { 634 | "block": "V38e6lu4e2iRktb25bd3", 635 | "project": "mmReuqVTAa9JzPpNr22I", 636 | "version": 1 637 | }, 638 | "outputId": null 639 | } 640 | }, 641 | "outputs": [], 642 | "source": [ 643 | "fig, (ax) = plt.subplots(1, 1, figsize=(24,8))\n", 644 | "\n", 645 | "ax.scatter(combined.datetime, combined.new_flow_tph, \n", 646 | " c='k', s = 5, linewidths = 0)\n", 647 | "\n", 648 | "ax.scatter(combined.datetime, combined.flow_tph, \n", 649 | " c='r', s = 30, linewidths = 0)\n", 650 | "\n", 651 | "ax.scatter(pts.datetime, pts.cweight_kg, \n", 652 | " c='b', s = 5, linewidths = 0, label = 'Raw cabel weight')\n", 653 | "\n", 654 | "ax.scatter(nan_trimmed.datetime, nan_trimmed.cweight_kg, \n", 655 | " c='g', s = 30, linewidths = 0, label = 'NAN cabel weight')\n", 656 | "\n", 657 | "ax.set_ylim(0,150)\n", 658 | "\n", 659 | "ax.set_xlabel('Time [hh:mm]')\n", 660 | "ax.xaxis.set_major_formatter(mdates.DateFormatter('%H:%M'))\n", 661 | "\n", 662 | "\n", 663 | "start_time = pd.to_datetime('2020-12-11 09:25:00')\n", 664 | "end_time = pd.to_datetime('2020-12-11 09:30:00')\n", 665 | "\n", 666 | "#ax.set_xlim(start_time,end_time);" 667 | ] 668 | }, 669 | { 670 | "cell_type": "code", 671 | "execution_count": null, 672 | "metadata": { 673 | "iooxa": { 674 | "id": { 675 | "block": "YFwBIE34H74W0yqBwWen", 676 | "project": "mmReuqVTAa9JzPpNr22I", 677 | "version": 1 678 | }, 679 | "outputId": null 680 | } 681 | }, 682 | "outputs": [], 683 | "source": [ 684 | "# turning the method into a function\n", 685 | "\n", 686 | "flowrate_f = read_flowrate(r'Data-FlowRate.xlsx')\n", 687 | "pts_f = read_pts(r'Data-PTS.xlsx')" 688 | ] 689 | }, 690 | { 691 | "cell_type": "code", 692 | "execution_count": null, 693 | "metadata": { 694 | "iooxa": { 695 | "id": { 696 | "block": "koL2FPouhpabsvyK34XJ", 697 | "project": "mmReuqVTAa9JzPpNr22I", 698 | "version": 1 699 | }, 700 | "outputId": null 701 | } 702 | }, 703 | "outputs": [], 704 | "source": [ 705 | "def append_flowrate_to_pts(flowrate_df, pts_df):\n", 706 | " '''\n", 707 | " Add surface flowrate to pts data\n", 708 | "\n", 709 | " Note that the flowrate data is recorded at a courser time resolution than the pts data\n", 710 | " The function makes a linear interpolation to fill the data gaps\n", 711 | " Refer to bonus-combine-data.ipynb to review this method and adapt it for your own data\n", 712 | "\n", 713 | " Args: flowrate and pts dataframes generated by the read_flowrate and read_pts functions\n", 714 | "\n", 715 | " Returns: pts dataframe with flowrate tph added\n", 716 | " '''\n", 717 | " flowrate_df = flowrate_df.set_index('timestamp')\n", 718 | " pts_df = pts_df.set_index('timestamp')\n", 719 | " combined_df = pts_df.join(flowrate_df, how = 'outer', lsuffix = '_pts', rsuffix = '_fr')\n", 720 | " combined_df.drop(columns = ['datetime_fr'], inplace = True)\n", 721 | " combined_df.columns = ['depth_m', 'speed_mps', 'cweight_kg', 'whp_barg', 'temp_degC',\n", 722 | " 'pressure_bara', 'frequency_hz', 'datetime', 'flow_tph']\n", 723 | " combined_df['interpolated_flow_tph'] = combined_df['flow_tph'].interpolate(method='linear')\n", 724 | " trimmed_df = combined_df[combined_df['depth_m'].notna()]\n", 725 | " trimmed_df.reset_index(inplace=True)\n", 726 | " return trimmed_df\n", 727 | "\n", 728 | "mudged_df = append_flowrate_to_pts(flowrate_f, pts_f)" 729 | ] 730 | }, 731 | { 732 | "cell_type": "code", 733 | "execution_count": null, 734 | "metadata": { 735 | "iooxa": { 736 | "id": { 737 | "block": "iH3CuxDafoGsyheGJPd7", 738 | "project": "mmReuqVTAa9JzPpNr22I", 739 | "version": 1 740 | }, 741 | "outputId": null 742 | } 743 | }, 744 | "outputs": [], 745 | "source": [ 746 | "mudged_df.shape" 747 | ] 748 | }, 749 | { 750 | "cell_type": "code", 751 | "execution_count": null, 752 | "metadata": { 753 | "iooxa": { 754 | "id": { 755 | "block": "H5CbXVTkXJbLcJUjOG6Q", 756 | "project": "mmReuqVTAa9JzPpNr22I", 757 | "version": 1 758 | }, 759 | "outputId": null 760 | } 761 | }, 762 | "outputs": [], 763 | "source": [ 764 | "mudged_df.head()" 765 | ] 766 | }, 767 | { 768 | "cell_type": "markdown", 769 | "metadata": { 770 | "iooxa": { 771 | "id": { 772 | "block": "6TqbXoJJOfXv1oeGbJIQ", 773 | "project": "mmReuqVTAa9JzPpNr22I", 774 | "version": 1 775 | } 776 | } 777 | }, 778 | "source": [ 779 | "***\n", 780 | "\n", 781 | "

© 2021 Irene Wallis and Katie McLean

\n", 782 | "\n", 783 | "

Licensed under the Apache License, Version 2.0

\n", 784 | "\n", 785 | "***" 786 | ] 787 | }, 788 | { 789 | "cell_type": "code", 790 | "execution_count": null, 791 | "metadata": {}, 792 | "outputs": [], 793 | "source": [] 794 | } 795 | ], 796 | "metadata": { 797 | "iooxa": { 798 | "id": { 799 | "block": "CuSEFk7WdWBVriyCxLD2", 800 | "project": "mmReuqVTAa9JzPpNr22I", 801 | "version": 1 802 | } 803 | }, 804 | "kernelspec": { 805 | "display_name": "Python 3", 806 | "language": "python", 807 | "name": "python3" 808 | }, 809 | "language_info": { 810 | "codemirror_mode": { 811 | "name": "ipython", 812 | "version": 3 813 | }, 814 | "file_extension": ".py", 815 | "mimetype": "text/x-python", 816 | "name": "python", 817 | "nbconvert_exporter": "python", 818 | "pygments_lexer": "ipython3", 819 | "version": "3.8.8" 820 | } 821 | }, 822 | "nbformat": 4, 823 | "nbformat_minor": 4 824 | } 825 | -------------------------------------------------------------------------------- /bonus-filter-by-toolspeed.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": { 6 | "iooxa": null, 7 | "tags": [] 8 | }, 9 | "source": [ 10 | "***\n", 11 | "\n", 12 | "# Geothermal Well Test Analysis with Python\n", 13 | "### Bonus material: Remove data aquired when the PTS tool is stationary or slowing\n", 14 | "#### Irene Wallis and Katie McLean \n", 15 | "#### Software Underground, Transform 2021\n", 16 | "\n", 17 | "***\n", 18 | "\n", 19 | "This method is used in 2-temperature.ipynb and 4-feedzones.ipynb" 20 | ] 21 | }, 22 | { 23 | "cell_type": "code", 24 | "execution_count": null, 25 | "metadata": { 26 | "iooxa": null 27 | }, 28 | "outputs": [], 29 | "source": [ 30 | "import pandas as pd\n", 31 | "from utilities import* # functions in the utilities.py file" 32 | ] 33 | }, 34 | { 35 | "cell_type": "code", 36 | "execution_count": null, 37 | "metadata": {}, 38 | "outputs": [], 39 | "source": [ 40 | "flowrate = read_flowrate(r'Data-FlowRate.xlsx')\n", 41 | "pts = read_pts(r'Data-PTS.xlsx')" 42 | ] 43 | }, 44 | { 45 | "cell_type": "markdown", 46 | "metadata": {}, 47 | "source": [ 48 | "Look at the spinner data plotted below and note where there is noise at the top, bottom and ~750 mMD because the tool is not moving or is moving at a slower pace. \n", 49 | "\n", 50 | "We need to clean these from the data to do the spinner analysis." 51 | ] 52 | }, 53 | { 54 | "cell_type": "code", 55 | "execution_count": null, 56 | "metadata": {}, 57 | "outputs": [], 58 | "source": [ 59 | "fig, (ax1, ax2) = plt.subplots(1, 2,figsize=(24,8),sharey=True)\n", 60 | "\n", 61 | "ax1.scatter(pts.frequency_hz, pts.depth_m, c = pts.timestamp, s = 5, linewidths = 0)\n", 62 | "ax2.scatter(pts.datetime, pts.depth_m, c = pts.timestamp, s = 5, linewidths = 0)\n", 63 | "\n", 64 | "ax3 = ax2.twinx()\n", 65 | "ax3.plot(flowrate.datetime, flowrate.flow_tph, \n", 66 | " c='k', linestyle = '-', linewidth = 3, alpha = 0.3, \n", 67 | " label='Surface pump flowrate')\n", 68 | "\n", 69 | "ax1.set_ylim(1000,0)\n", 70 | "ax1.set_xlim(-30,30)\n", 71 | "\n", 72 | "ax1.set_ylabel('Depth [m]')\n", 73 | "ax1.set_xlabel('Spinner frequency [hz]')\n", 74 | "\n", 75 | "ax2.set_xlabel('Time [hh:mm]')\n", 76 | "ax2.xaxis.set_major_formatter(mdates.DateFormatter('%H:%M'))\n", 77 | "\n", 78 | "ax3.set_ylabel('Flowrate [t/hr]')\n", 79 | "\n", 80 | "for ax in [ax1, ax2]:\n", 81 | " ax.grid()" 82 | ] 83 | }, 84 | { 85 | "cell_type": "markdown", 86 | "metadata": { 87 | "iooxa": { 88 | "id": { 89 | "block": "4vXVN8gvYimi0eoRYMt6", 90 | "project": "mmReuqVTAa9JzPpNr22I", 91 | "version": 1 92 | } 93 | } 94 | }, 95 | "source": [ 96 | "\n", 97 | "\n", 98 | "## Decide on a speed limit for the filer\n", 99 | "\n", 100 | "We can use descriptive and plotting tools that are built into Pandas to understand the data" 101 | ] 102 | }, 103 | { 104 | "cell_type": "code", 105 | "execution_count": null, 106 | "metadata": {}, 107 | "outputs": [], 108 | "source": [ 109 | "pts.describe()" 110 | ] 111 | }, 112 | { 113 | "cell_type": "code", 114 | "execution_count": null, 115 | "metadata": {}, 116 | "outputs": [], 117 | "source": [ 118 | "pts['speed_mps'].hist(bins=30)" 119 | ] 120 | }, 121 | { 122 | "cell_type": "markdown", 123 | "metadata": {}, 124 | "source": [ 125 | "## Explore the data at various scales using a plot\n", 126 | "\n", 127 | "Because we will use this filter on the temperature and spinner data, we want to remove both the stationary data and the data acquired as the tool slows down before it stops.\n", 128 | "\n", 129 | "We use the matplotlib plot below to zoom into the data and decide on a reasonable cut-off value, which looks to be 0.05 m/s.\n", 130 | "\n", 131 | "Hint: Explore the data by changing the y axis limits: ax1.set_ylim(-0.05,0.05)" 132 | ] 133 | }, 134 | { 135 | "cell_type": "code", 136 | "execution_count": null, 137 | "metadata": {}, 138 | "outputs": [], 139 | "source": [ 140 | "import matplotlib.pyplot as plt\n", 141 | "import matplotlib.dates as mdates\n", 142 | "\n", 143 | "fig, (ax) = plt.subplots(1, 1,figsize=(12,6))\n", 144 | "\n", 145 | "ax.set_title('All Data')\n", 146 | "\n", 147 | "ax.scatter(pts.datetime, pts.speed_mps, s = 3, c='k', linewidths = 0)\n", 148 | "\n", 149 | "ax.xaxis.set_major_formatter(mdates.DateFormatter('%H:%M'))\n", 150 | "ax.set_xlabel('Time [hh:mm]')\n", 151 | "\n", 152 | "ax.set_ylabel('Tool speed [m/sec]')\n", 153 | "\n", 154 | "#ax.set_ylim(-0.2,0.2)\n", 155 | "\n", 156 | ";" 157 | ] 158 | }, 159 | { 160 | "cell_type": "code", 161 | "execution_count": null, 162 | "metadata": {}, 163 | "outputs": [], 164 | "source": [ 165 | "fig, (ax1, ax2) = plt.subplots(1, 2,figsize=(28,8))\n", 166 | "\n", 167 | "#ax1.plot(pts.datetime, pts.frequency_hz, c='k', linestyle = '-', linewidth = 1, alpha = 0.3)\n", 168 | "ax1.scatter(pts.datetime, pts.frequency_hz, c = pts.timestamp, s = 5, linewidths = 0)\n", 169 | "ax2.scatter(pts.datetime, pts.depth_m, c = pts.timestamp, s = 5, linewidths = 0)\n", 170 | "\n", 171 | "ax3 = ax2.twinx()\n", 172 | "ax3.plot(flowrate.datetime, flowrate.flow_tph, \n", 173 | " c='k', linestyle = '-', linewidth = 3, alpha = 0.3, \n", 174 | " label='Surface pump flowrate')\n", 175 | "\n", 176 | "ax4 = ax1.twinx()\n", 177 | "ax4.plot(pts.datetime, pts.depth_m, \n", 178 | " c='k', linestyle = '-', linewidth = 2, alpha = 0.3, \n", 179 | " label='Tool depth [m]')\n", 180 | "ax4.set_ylim(1000,-1000)\n", 181 | "ax4.set_ylabel('Tool depth [m]')\n", 182 | "\n", 183 | "ax1.set_ylim(-30,30)\n", 184 | "ax1.set_ylabel('Spinner frequency [hz]')\n", 185 | "\n", 186 | "ax2.set_ylim(1000,0)\n", 187 | "ax2.set_ylabel('Tool depth [m/s]')\n", 188 | "\n", 189 | "for ax in [ax1,ax2]:\n", 190 | " ax.set_xlabel('Time [hh:mm]')\n", 191 | " ax.xaxis.set_major_formatter(mdates.DateFormatter('%H:%M'))\n", 192 | "\n", 193 | "ax3.set_ylabel('Flowrate [t/hr]')\n", 194 | "\n", 195 | "for ax in [ax1, ax2]:\n", 196 | " ax.grid()\n", 197 | "\n", 198 | "#\n", 199 | "# Limit to a time range\n", 200 | "#\n", 201 | "\n", 202 | "start_time = pd.to_datetime('2020-12-11 09:30:00')\n", 203 | "end_time = pd.to_datetime('2020-12-11 10:30:00')\n", 204 | "\n", 205 | "ax1.set_xlim(start_time,end_time)\n", 206 | "ax4.set_ylim(1000,400);" 207 | ] 208 | }, 209 | { 210 | "cell_type": "markdown", 211 | "metadata": {}, 212 | "source": [ 213 | "## Use a boolean expression to select only the desired data\n", 214 | "\n", 215 | "We decided that, in this case, data acquired when the tool is moving down the well at > 0.9 m/s is fast enough to be included. Remember that the sign indicates the direction that a tool is moving inside the well and is not an actual -ve speed. \n", 216 | "\n", 217 | "Our new working dataframe is called moving_pts" 218 | ] 219 | }, 220 | { 221 | "cell_type": "code", 222 | "execution_count": null, 223 | "metadata": { 224 | "iooxa": { 225 | "id": { 226 | "block": "D2mZcbWRMpMIzKwInZv6", 227 | "project": "mmReuqVTAa9JzPpNr22I", 228 | "version": 1 229 | }, 230 | "outputId": null 231 | } 232 | }, 233 | "outputs": [], 234 | "source": [ 235 | "moving_pts = pts[\n", 236 | " (pts.speed_mps > 0.9 ) & (pts.speed_mps < pts.speed_mps.max()) | \n", 237 | " (pts.speed_mps > pts.speed_mps.min() ) & (pts.speed_mps < -0.9)\n", 238 | " ]" 239 | ] 240 | }, 241 | { 242 | "cell_type": "code", 243 | "execution_count": null, 244 | "metadata": {}, 245 | "outputs": [], 246 | "source": [ 247 | "moving_pts.shape" 248 | ] 249 | }, 250 | { 251 | "cell_type": "code", 252 | "execution_count": null, 253 | "metadata": {}, 254 | "outputs": [], 255 | "source": [ 256 | "fig, (ax) = plt.subplots(1, 1,figsize=(12,6))\n", 257 | "\n", 258 | "ax.set_title('Data Acquired while the Tool is Moving')\n", 259 | "\n", 260 | "ax.scatter(moving_pts.datetime, moving_pts.speed_mps, s = 3, c='k', linewidths = 0)\n", 261 | "\n", 262 | "ax.xaxis.set_major_formatter(mdates.DateFormatter('%H:%M'))\n", 263 | "ax.set_xlabel('Time [hh:mm]')\n", 264 | "\n", 265 | "ax.set_ylabel('Tool speed [m/sec]')\n", 266 | "\n", 267 | "#ax.set_ylim(-0.5,0.5)\n", 268 | "\n", 269 | ";" 270 | ] 271 | }, 272 | { 273 | "cell_type": "code", 274 | "execution_count": null, 275 | "metadata": {}, 276 | "outputs": [], 277 | "source": [ 278 | "fig, (ax1, ax2) = plt.subplots(1, 2,figsize=(28,8))\n", 279 | "\n", 280 | "ax1.scatter(moving_pts.datetime, moving_pts.frequency_hz, c = moving_pts.timestamp, s = 5, linewidths = 0)\n", 281 | "ax2.scatter(moving_pts.datetime, moving_pts.depth_m, c = moving_pts.timestamp, s = 5, linewidths = 0)\n", 282 | "\n", 283 | "ax3 = ax2.twinx()\n", 284 | "ax3.plot(flowrate.datetime, flowrate.flow_tph, \n", 285 | " c='k', linestyle = '-', linewidth = 3, alpha = 0.3, \n", 286 | " label='Surface pump flowrate')\n", 287 | "\n", 288 | "ax4 = ax1.twinx()\n", 289 | "ax4.plot(pts.datetime, pts.depth_m, \n", 290 | " c='k', linestyle = '-', linewidth = 2, alpha = 0.3, \n", 291 | " label='Tool depth [m]')\n", 292 | "ax4.set_ylim(1000,400)\n", 293 | "ax4.set_ylabel('Tool depth [m]')\n", 294 | "\n", 295 | "ax1.set_ylim(-30,30)\n", 296 | "ax1.set_ylabel('Spinner frequency [hz]')\n", 297 | "\n", 298 | "ax2.set_ylim(1000,0)\n", 299 | "ax2.set_ylabel('Tool depth [m]')\n", 300 | "\n", 301 | "for ax in [ax1,ax2]:\n", 302 | " ax.set_xlabel('Time [hh:mm]')\n", 303 | " ax.xaxis.set_major_formatter(mdates.DateFormatter('%H:%M:%S'))\n", 304 | "\n", 305 | "ax3.set_ylabel('Flowrate [t/hr]')\n", 306 | "\n", 307 | "for ax in [ax1, ax2]:\n", 308 | " ax.grid()\n", 309 | "\n", 310 | "#\n", 311 | "# Limit to a time range\n", 312 | "#\n", 313 | "\n", 314 | "start_time = pd.to_datetime('2020-12-11 14:30:00')\n", 315 | "end_time = pd.to_datetime('2020-12-11 15:30:00')\n", 316 | "\n", 317 | "ax1.set_xlim(start_time,end_time);" 318 | ] 319 | }, 320 | { 321 | "cell_type": "markdown", 322 | "metadata": { 323 | "iooxa": { 324 | "id": { 325 | "block": "4ca0ewMvunDQ3eLy3Oqt", 326 | "project": "mmReuqVTAa9JzPpNr22I", 327 | "version": 1 328 | } 329 | } 330 | }, 331 | "source": [ 332 | "# Remove data from inside the casing\n", 333 | "\n" 334 | ] 335 | }, 336 | { 337 | "cell_type": "code", 338 | "execution_count": null, 339 | "metadata": {}, 340 | "outputs": [], 341 | "source": [ 342 | "production_shoe = 462.5 \n", 343 | "\n", 344 | "depth_filtered = moving_pts[(moving_pts.depth_m < moving_pts.depth_m.max()) & (moving_pts.depth_m > production_shoe)]\n", 345 | "\n", 346 | "depth_filtered.describe()" 347 | ] 348 | }, 349 | { 350 | "cell_type": "markdown", 351 | "metadata": {}, 352 | "source": [ 353 | "## Plot the cleaned data\n", 354 | "\n", 355 | "Now we have two Boolean statements that we can use to filter this PTS dataset down to only the by-depth data that we want for the spinner analysis. \n", 356 | "\n", 357 | "There are still a few stray data points that missed by this simple approach. We plan to develop something a little more elegant to clean these data fully, but this approach is near enough for most purposes. " 358 | ] 359 | }, 360 | { 361 | "cell_type": "code", 362 | "execution_count": null, 363 | "metadata": {}, 364 | "outputs": [], 365 | "source": [ 366 | "fig, (ax1, ax2) = plt.subplots(1, 2,figsize=(24,8),sharey=True)\n", 367 | "\n", 368 | "ax1.set_title('All data')\n", 369 | "ax1.scatter(pts.frequency_hz, pts.depth_m, c = pts.timestamp, s = 5, linewidths = 0)\n", 370 | "\n", 371 | "ax2.set_title('Filtered data')\n", 372 | "ax2.scatter(depth_filtered.frequency_hz, depth_filtered.depth_m, c = depth_filtered.timestamp, s = 5, linewidths = 0)\n", 373 | "\n", 374 | "ax1.set_ylabel('Depth [m]')\n", 375 | "\n", 376 | "for ax in [ax1,ax2]:\n", 377 | " ax.set_ylim(1000,0)\n", 378 | " ax.set_xlim(-30,30)\n", 379 | " ax.set_xlabel('Spinner frequency [hz]')\n", 380 | " ax.grid();" 381 | ] 382 | }, 383 | { 384 | "cell_type": "markdown", 385 | "metadata": {}, 386 | "source": [ 387 | "***\n", 388 | "\n", 389 | "

© 2021 Irene Wallis and Katie McLean

\n", 390 | "\n", 391 | "

Licensed under the Apache License, Version 2.0

\n", 392 | "\n", 393 | "\n", 394 | "***" 395 | ] 396 | }, 397 | { 398 | "cell_type": "code", 399 | "execution_count": null, 400 | "metadata": {}, 401 | "outputs": [], 402 | "source": [] 403 | } 404 | ], 405 | "metadata": { 406 | "kernelspec": { 407 | "display_name": "Python 3", 408 | "language": "python", 409 | "name": "python3" 410 | }, 411 | "language_info": { 412 | "codemirror_mode": { 413 | "name": "ipython", 414 | "version": 3 415 | }, 416 | "file_extension": ".py", 417 | "mimetype": "text/x-python", 418 | "name": "python", 419 | "nbconvert_exporter": "python", 420 | "pygments_lexer": "ipython3", 421 | "version": "3.7.9" 422 | } 423 | }, 424 | "nbformat": 4, 425 | "nbformat_minor": 4 426 | } 427 | -------------------------------------------------------------------------------- /environment.yml: -------------------------------------------------------------------------------- 1 | name: geothrm 2 | channels: 3 | - conda-forge 4 | - defaults 5 | dependencies: 6 | - python=3.7.9 7 | - jupyter 8 | - jupyterlab 9 | - ipykernel 10 | - matplotlib 11 | - numpy 12 | - pandas 13 | - scipy 14 | - pip 15 | - pip: 16 | - ipywidgets==7.6.3 17 | - jupyterlab-widgets==1.0.0 18 | - widgetsnbextension==3.5.1 19 | - jdcal==1.4.1 20 | - pytz==2021.1 21 | - openpyxl==3.0.6 22 | - xlrd==2.0.1 23 | - iapws==1.5.2 24 | 25 | --------------------------------------------------------------------------------