├── 00_WhatToExpect.ipynb ├── 01_GettingStarted.ipynb ├── 02_DownloadDatafromECMWF.ipynb ├── 03_NetCDFFiles.ipynb ├── 04_Plotting.ipynb ├── 05_PlotWindData.ipynb ├── 06_VerticallyIntegratedMoistureFlux.ipynb ├── 07_Correlation Maps.ipynb ├── 08_AverageoverSphere.ipynb ├── 09_ToGoFurther.ipynb ├── LICENSE ├── README.md └── img ├── 3D_netCDF.gif ├── dataset-diagram.png ├── hierarchy1.JPG ├── netCDF.gif └── xarraylogo.png /01_GettingStarted.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# 01_Getting Started\n" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "In this chapter, we will get to know Jupyter-Notebook, a tool of the python library and python in general! If you never coded with Python or if you are a beginner and want to learn the basics, go for this section!" 15 | ] 16 | }, 17 | { 18 | "cell_type": "markdown", 19 | "metadata": {}, 20 | "source": [ 21 | "## 01_01 Jupyter-Notebook" 22 | ] 23 | }, 24 | { 25 | "cell_type": "markdown", 26 | "metadata": {}, 27 | "source": [ 28 | "Since all the text, you can now read here was produced with Jupyter-Notbook, we will first have a closer look at this tool. Jupyter-Notebook originated from the python language and comes with one big advantage: it can include text, python code and plots all in the same document! That means you can have the whole computation process in just one application: developing, documenting, and executing code, as well as communicating the results." 29 | ] 30 | }, 31 | { 32 | "cell_type": "markdown", 33 | "metadata": {}, 34 | "source": [ 35 | "### Start Jupyter - Notebook" 36 | ] 37 | }, 38 | { 39 | "cell_type": "markdown", 40 | "metadata": {}, 41 | "source": [ 42 | "To start the interface, type the following command in a terminal window:\n", 43 | " \n", 44 | " $ jupyter-notebook\n", 45 | "\n", 46 | "A new tab in your default web browser will be opened. This is the notebook \"**explorer**\". From this window you can navigate through your folders, open existing notebooks like this one or create new ones (\"New\" button on the upper right panel).\n", 47 | "\n", 48 | "**Note**: do not close the terminal where you started the notebook. This is where python is \"actually\" running." 49 | ] 50 | }, 51 | { 52 | "cell_type": "markdown", 53 | "metadata": {}, 54 | "source": [ 55 | "Once, you created a new notebook, you will be presented with several things: The **notebook name** appears right next to the jupyter sign at the very top of the page. The new notebook will be named \"untitled\", but you can change this at any time by clicking on the name. Below the toolbar, you can see a **code cell**. This is the default type of cell in a jupyter-notbook and you can write and execute python code with it. There is another cell type called **markdown**, in which you can write simple text like the one you actually read right now. " 56 | ] 57 | }, 58 | { 59 | "cell_type": "markdown", 60 | "metadata": {}, 61 | "source": [ 62 | "### Keyboard Shortcuts" 63 | ] 64 | }, 65 | { 66 | "cell_type": "markdown", 67 | "metadata": {}, 68 | "source": [ 69 | "Before we learn more about markdown and code cells, we will get to know some nice keyboard shortcuts, that make life much easier in a jupyter-notebook!\n", 70 | "\n", 71 | "To be able to use those shortcuts, you will first need to get into the so called **command mode** by pressing `esc`. You will also enter this mode, if you single click on a cell. The color of the cells left margin will turn from green (edit mode) to blue.\n", 72 | "Now you can \n", 73 | "* **Switch your cell between code and markdown**: press `[m]` to markdown and `[y]` to code. \n", 74 | "* **Add a cell:** press `[b]` to add a cell below, `[a]` to add one above.\n", 75 | "* **Delete a cell:** double-press `[d]`.\n", 76 | "* **Move up/down:** `[k]` / `[j]`\n", 77 | "* **Cut/Copy/paste cells:** `[x]` /`[c]` / `[v]`\n", 78 | "\n", 79 | "If you are currently in command mode and want to change back to the **edit mode**, in which you can edit the text or code of your cells, just press `enter` or double click on the cell you want to edit. \n", 80 | "\n", 81 | "If you want to **execute/run your cell** of code or text, press `shift + enter`. If it was a cell of python code, the output will appear underneath. \n", 82 | "\n", 83 | "The `Help->Keyboard Shortcuts` dialog lists all the available shortcuts." 84 | ] 85 | }, 86 | { 87 | "cell_type": "markdown", 88 | "metadata": {}, 89 | "source": [ 90 | "### Markdown Cells " 91 | ] 92 | }, 93 | { 94 | "cell_type": "markdown", 95 | "metadata": {}, 96 | "source": [ 97 | "In the following cells, I will show you how to work with markdown cells in jupyter notebook. Double click on my markdown cells, to see how the different styles of text are achieved!" 98 | ] 99 | }, 100 | { 101 | "cell_type": "markdown", 102 | "metadata": {}, 103 | "source": [ 104 | "You can make text *italic* or **bold** by surrounding a block of text with a single or double * respectively." 105 | ] 106 | }, 107 | { 108 | "cell_type": "markdown", 109 | "metadata": {}, 110 | "source": [ 111 | "You can build nested itemized or enumerated lists:\n", 112 | "\n", 113 | "* One\n", 114 | " - Sublist\n", 115 | " - This\n", 116 | " - Sublist\n", 117 | " - That\n", 118 | " - The other thing\n", 119 | "* Two\n", 120 | " - Sublist\n", 121 | "* Three\n", 122 | " - Sublist\n", 123 | "\n", 124 | "Now another list:\n", 125 | "\n", 126 | "1. Here we go\n", 127 | " 1. Sublist\n", 128 | " 2. Sublist\n", 129 | "2. There we go\n", 130 | "3. Now this\n", 131 | "\n" 132 | ] 133 | }, 134 | { 135 | "cell_type": "markdown", 136 | "metadata": {}, 137 | "source": [ 138 | "You can add shorthand for links:\n", 139 | "[Jupyter's website](https://jupyter.org).\n", 140 | "\n", 141 | "You can use backslash \\ to generate literal characters which would otherwise have special meaning in the Markdown syntax: \\*\\*Hello\\*\\*. Without the backslash, \\** would stand for bold: **Hello**.\n" 142 | ] 143 | }, 144 | { 145 | "cell_type": "markdown", 146 | "metadata": {}, 147 | "source": [ 148 | "### Heading" 149 | ] 150 | }, 151 | { 152 | "cell_type": "markdown", 153 | "metadata": {}, 154 | "source": [ 155 | "You can add headings by starting a line with one (or multiple) `#` followed by a space, as in the following example:\n", 156 | "\n", 157 | "```\n", 158 | "# Heading 1\n", 159 | "# Heading 2\n", 160 | "## Heading 2.1\n", 161 | "## Heading 2.2\n", 162 | "```" 163 | ] 164 | }, 165 | { 166 | "cell_type": "markdown", 167 | "metadata": {}, 168 | "source": [ 169 | "You can embed code meant for illustration instead of execution in Python:\n", 170 | "\n", 171 | " def f(x):\n", 172 | " \"\"\"a docstring\"\"\"\n", 173 | " return x**2" 174 | ] 175 | }, 176 | { 177 | "cell_type": "markdown", 178 | "metadata": {}, 179 | "source": [ 180 | "If you have local files in your Notebook directory, you can refer to these files in Markdown cells directly:\n", 181 | "\n", 182 | " [subdirectory/]\n", 183 | " \n", 184 | "Like this you can for example show an image or a video.\n", 185 | "These do not embed the data into the notebook file, and require that the files exist when you are viewing the notebook." 186 | ] 187 | }, 188 | { 189 | "cell_type": "markdown", 190 | "metadata": {}, 191 | "source": [ 192 | "### Code Cells" 193 | ] 194 | }, 195 | { 196 | "cell_type": "markdown", 197 | "metadata": {}, 198 | "source": [ 199 | "In code cells, you can write and execute code. The output will appear underneath the cell, once you execute it." 200 | ] 201 | }, 202 | { 203 | "cell_type": "code", 204 | "execution_count": null, 205 | "metadata": {}, 206 | "outputs": [], 207 | "source": [ 208 | "print('Sunshine')" 209 | ] 210 | }, 211 | { 212 | "cell_type": "markdown", 213 | "metadata": {}, 214 | "source": [ 215 | "You can execute your code, as already mentioned before, with the keyboard shortcut `[Shift+Enter]` or press the `Run` button in the toolbar. Afterwards, the next cell underneath will be selected automatically." 216 | ] 217 | }, 218 | { 219 | "cell_type": "markdown", 220 | "metadata": {}, 221 | "source": [ 222 | "The `Cell` menu has a number of menu items for running code in different ways. These includes:\n", 223 | "\n", 224 | "* **Run and Select Below**: Runs the currently selected cell and afterwards selects the cell below.\n", 225 | " That's what you get by pressing `[Shift+Enter]`\n", 226 | "* **Run and Insert Below**: Runs the currently selected cell and inserts a new cell below. Press `[Alt+Enter]`!\n", 227 | "* **Run All**: Runs all the code cells included in your jupyter-notebook\n", 228 | "* **Run All Above**: Runs all the code cells above the cell you currently selected, excluding this one\n", 229 | "* **Run All Below**: All below " 230 | ] 231 | }, 232 | { 233 | "cell_type": "markdown", 234 | "metadata": {}, 235 | "source": [ 236 | "The normal workflow in a notebook is, then, quite similar to a standard IPython session, with the difference that you can edit cells in-place multiple times until you obtain the desired results, rather than having to rerun separate scripts with the `%run` command." 237 | ] 238 | }, 239 | { 240 | "cell_type": "markdown", 241 | "metadata": {}, 242 | "source": [ 243 | "Typically, you will work on a computational problem in pieces, organizing related ideas into cells and moving forward once previous parts work correctly. This is much more convenient for interactive exploration than breaking up a computation into scripts that must be executed together, as was previously necessary, especially if parts of them take a long time to run. " 244 | ] 245 | }, 246 | { 247 | "cell_type": "markdown", 248 | "metadata": {}, 249 | "source": [ 250 | "If you want to **interrupt** a computation, because it either takes to long or you found an error and so the code won't work anyway, there is a keybord shortcut: double-press `[i]`.\n", 251 | "\n", 252 | "The code cells of this book are not executed yet. Just work with the `[Shift+Enter]` command to execute them on your own and see the ouput." 253 | ] 254 | }, 255 | { 256 | "cell_type": "markdown", 257 | "metadata": {}, 258 | "source": [ 259 | "## 01_02 A Python Introduction" 260 | ] 261 | }, 262 | { 263 | "cell_type": "markdown", 264 | "metadata": {}, 265 | "source": [ 266 | "A python interpreter is a program, that reads the code you wrote and executes the instructions you gave in your code. In this jupyter-notebook, the interpreter is called iPython. It works as a simple calculator: You write an expression and the output will return the result." 267 | ] 268 | }, 269 | { 270 | "cell_type": "code", 271 | "execution_count": null, 272 | "metadata": {}, 273 | "outputs": [], 274 | "source": [ 275 | "(2 + 3) / 5" 276 | ] 277 | }, 278 | { 279 | "cell_type": "code", 280 | "execution_count": null, 281 | "metadata": {}, 282 | "outputs": [], 283 | "source": [ 284 | "3**2\n", 285 | "# ** is used to calculate powers\n", 286 | "# with a # sign in front of the line, you can write comments, that are not executed!" 287 | ] 288 | }, 289 | { 290 | "cell_type": "markdown", 291 | "metadata": {}, 292 | "source": [ 293 | "The equal sign `=` is used to assign a value to a variable! " 294 | ] 295 | }, 296 | { 297 | "cell_type": "code", 298 | "execution_count": null, 299 | "metadata": {}, 300 | "outputs": [], 301 | "source": [ 302 | "a = 10 #cases matter in python!\n", 303 | "A = 11\n", 304 | "print(a+A)" 305 | ] 306 | }, 307 | { 308 | "cell_type": "markdown", 309 | "metadata": {}, 310 | "source": [ 311 | "A variable in Python is very similar to a variable in Matlab or in other languages. A variable can be initialised, used and re-initialised:\n" 312 | ] 313 | }, 314 | { 315 | "cell_type": "code", 316 | "execution_count": null, 317 | "metadata": {}, 318 | "outputs": [], 319 | "source": [ 320 | "x = 10\n", 321 | "y = x**2\n", 322 | "print(y)\n", 323 | "y = 'Hi!'\n", 324 | "print(y)" 325 | ] 326 | }, 327 | { 328 | "cell_type": "markdown", 329 | "metadata": {}, 330 | "source": [ 331 | "If you do not assign any value to a variable, trying to use it will result in an error..." 332 | ] 333 | }, 334 | { 335 | "cell_type": "code", 336 | "execution_count": null, 337 | "metadata": {}, 338 | "outputs": [], 339 | "source": [ 340 | "print(c)" 341 | ] 342 | }, 343 | { 344 | "cell_type": "markdown", 345 | "metadata": {}, 346 | "source": [ 347 | "**Error messages** in python are most of the time very useful! They tell you, what kind of mistake you have in your code (In this case: `NameError`, since you forgot to define c before using it). They will furthermore give you a traceback to where in your code the mistake happened. Often, you can google the error and you will find tons of solutions for your problem. " 348 | ] 349 | }, 350 | { 351 | "cell_type": "markdown", 352 | "metadata": {}, 353 | "source": [ 354 | "### 01_02_01 Importing Modules in Python" 355 | ] 356 | }, 357 | { 358 | "cell_type": "markdown", 359 | "metadata": {}, 360 | "source": [ 361 | "There are only a couple of functions in Python (like for example `print`) that are always and everywhere available: These are called **built-in** functions, since they are \"built-in\" the basic python language. However, if you create functions on your own or want to use more than the built-in functions, you will have to `import` those. \n", 362 | "For example you can `import` the module `numpy`. A module is a storage for a certain collection of functions and variables. Numpy is a module, in which functions and variables are stored that help you do arithmetics or vector computations. Well, that's exactly what we want to do here! So numpy will be one of tools we will use later on. " 363 | ] 364 | }, 365 | { 366 | "cell_type": "code", 367 | "execution_count": null, 368 | "metadata": {}, 369 | "outputs": [], 370 | "source": [ 371 | "import numpy" 372 | ] 373 | }, 374 | { 375 | "cell_type": "markdown", 376 | "metadata": {}, 377 | "source": [ 378 | "With this simple command, we just \"imported\" the entire `Numpy` library! This means we have now acces on all its variables and functions. Since you will use numpy a lot and it is more convenient to not always type its full name, you can give it a nickname (alias): " 379 | ] 380 | }, 381 | { 382 | "cell_type": "code", 383 | "execution_count": null, 384 | "metadata": {}, 385 | "outputs": [], 386 | "source": [ 387 | "import numpy as np # np is the common convention used for numpy" 388 | ] 389 | }, 390 | { 391 | "cell_type": "markdown", 392 | "metadata": {}, 393 | "source": [ 394 | "We imported the numpy module and names it \"np\". So from now on, you can use all the methods that numpy provides by calling `np.`!" 395 | ] 396 | }, 397 | { 398 | "cell_type": "code", 399 | "execution_count": null, 400 | "metadata": {}, 401 | "outputs": [], 402 | "source": [ 403 | "a = np.arange(5)\n", 404 | "print(a)" 405 | ] 406 | }, 407 | { 408 | "cell_type": "markdown", 409 | "metadata": {}, 410 | "source": [ 411 | "To get an idea of all the new functions available to us, you can write \"`np.`\" (\"np\" followed by a dot) in a free cell, then type `tab` (`tab` is the **autocompletion** shortcut of ipython, it is very helpful when writing code). A list will appear, that shows all the new possibilities available to you because of importing numpy." 412 | ] 413 | }, 414 | { 415 | "cell_type": "markdown", 416 | "metadata": {}, 417 | "source": [ 418 | "If you want to know, how all of those functions work, you can type `?` plus the function (e.g. `? np.full()`). This gives you help and explains the function. \n", 419 | "Or you simply google the function and there will be tons of good explanations.\n" 420 | ] 421 | }, 422 | { 423 | "cell_type": "markdown", 424 | "metadata": {}, 425 | "source": [ 426 | "There are mainly **three modules** that we will keep on using when we work with climate data:\n", 427 | "\n", 428 | "* [numpy](http://docs.scipy.org/doc/numpy/reference/): this is the base on which any scientific python project is built.\n", 429 | "* [xarray](http://xarray.pydata.org/en/stable/): working with our netCDF multidimensional data\n", 430 | "* [matplotlib](http://matplotlib.org/index.html): gives us the tools we need to plot our data\n" 431 | ] 432 | }, 433 | { 434 | "cell_type": "markdown", 435 | "metadata": {}, 436 | "source": [ 437 | "### 01_02_02 Data Types" 438 | ] 439 | }, 440 | { 441 | "cell_type": "markdown", 442 | "metadata": {}, 443 | "source": [ 444 | "Python objects have a type (synonym: data type). There are built-in data types and others that are not built in. Built-in means, that those data types are directly available in the interpreter, as opposed to other data types which maybe obtained either by importing them or by creating new data types yourselves. The following ones are all built-in!" 445 | ] 446 | }, 447 | { 448 | "cell_type": "markdown", 449 | "metadata": {}, 450 | "source": [ 451 | "**Numbers**: Integers and Floats" 452 | ] 453 | }, 454 | { 455 | "cell_type": "code", 456 | "execution_count": null, 457 | "metadata": {}, 458 | "outputs": [], 459 | "source": [ 460 | "a = 5\n", 461 | "type(a) # what is the data type of this?" 462 | ] 463 | }, 464 | { 465 | "cell_type": "code", 466 | "execution_count": null, 467 | "metadata": {}, 468 | "outputs": [], 469 | "source": [ 470 | "a = 5.5\n", 471 | "type(a)" 472 | ] 473 | }, 474 | { 475 | "cell_type": "markdown", 476 | "metadata": {}, 477 | "source": [ 478 | "**Booleans**: Has the values `True` and `False`. It is very useful, to test the truth of an expression!" 479 | ] 480 | }, 481 | { 482 | "cell_type": "markdown", 483 | "metadata": {}, 484 | "source": [ 485 | "The mathematical \"equal\" is written with a double equal sign `==` in python, since the single equal is already used to define variables." 486 | ] 487 | }, 488 | { 489 | "cell_type": "code", 490 | "execution_count": null, 491 | "metadata": {}, 492 | "outputs": [], 493 | "source": [ 494 | "a = 10\n", 495 | "b = 10\n", 496 | "a == b" 497 | ] 498 | }, 499 | { 500 | "cell_type": "code", 501 | "execution_count": null, 502 | "metadata": {}, 503 | "outputs": [], 504 | "source": [ 505 | "type(a == b)" 506 | ] 507 | }, 508 | { 509 | "cell_type": "code", 510 | "execution_count": null, 511 | "metadata": {}, 512 | "outputs": [], 513 | "source": [ 514 | "5 < 3" 515 | ] 516 | }, 517 | { 518 | "cell_type": "markdown", 519 | "metadata": {}, 520 | "source": [ 521 | "**String**: This is simply text. " 522 | ] 523 | }, 524 | { 525 | "cell_type": "code", 526 | "execution_count": null, 527 | "metadata": {}, 528 | "outputs": [], 529 | "source": [ 530 | "a = 'Rainbow'\n", 531 | "b = \"Thunderstorm\" # you can define a string either with single or double quotes. It doesn't matter!\n", 532 | "print(a)" 533 | ] 534 | }, 535 | { 536 | "cell_type": "markdown", 537 | "metadata": {}, 538 | "source": [ 539 | "Strings can be concatenated:" 540 | ] 541 | }, 542 | { 543 | "cell_type": "code", 544 | "execution_count": null, 545 | "metadata": {}, 546 | "outputs": [], 547 | "source": [ 548 | "a = 'Happy Birthday'\n", 549 | "b = ' Hans! '\n", 550 | "\n", 551 | "c = a + b\n", 552 | "print(c)" 553 | ] 554 | }, 555 | { 556 | "cell_type": "markdown", 557 | "metadata": {}, 558 | "source": [ 559 | "But if you want to concatenate a string and a number (`int` / `float`), you will first have to turn the number into a string. This works the following:" 560 | ] 561 | }, 562 | { 563 | "cell_type": "code", 564 | "execution_count": null, 565 | "metadata": {}, 566 | "outputs": [], 567 | "source": [ 568 | "d = str(50) # turn my number into data type 'string'!\n", 569 | "e = ' years is a great age!'\n", 570 | "f = c + d + e\n", 571 | "print(f)" 572 | ] 573 | }, 574 | { 575 | "cell_type": "markdown", 576 | "metadata": {}, 577 | "source": [ 578 | "Strings have a length, that you can ask for with the built-in function `len()` and they can be indexed." 579 | ] 580 | }, 581 | { 582 | "cell_type": "code", 583 | "execution_count": null, 584 | "metadata": {}, 585 | "outputs": [], 586 | "source": [ 587 | "len(a)" 588 | ] 589 | }, 590 | { 591 | "cell_type": "code", 592 | "execution_count": null, 593 | "metadata": {}, 594 | "outputs": [], 595 | "source": [ 596 | "a[0:3] # when indexing, the first index is always included, while the last index is always excluded. \n", 597 | " # Unlike in some other programming languages, indexing in python starts with 0!" 598 | ] 599 | }, 600 | { 601 | "cell_type": "markdown", 602 | "metadata": {}, 603 | "source": [ 604 | "Strings are immutable, which means that once you created a string object, you cannot change a part of its content. You can only change the whole variable. Trying to change e.g. one character of the whole word will result in an error:" 605 | ] 606 | }, 607 | { 608 | "cell_type": "code", 609 | "execution_count": null, 610 | "metadata": {}, 611 | "outputs": [], 612 | "source": [ 613 | "a[0] = 'J'" 614 | ] 615 | }, 616 | { 617 | "cell_type": "markdown", 618 | "metadata": {}, 619 | "source": [ 620 | "Strings in python are an object. This means that, like everything else in python, they have methods and attributes attached to them. You can for example transform every character of an object into uppercase letters, format your string and much more. Those are methods, which will do something to your object and give you an output. Methods (you might know them as \"functions\") are called with parentheses, and sometimes they require arguments in those brackets. \n", 621 | "\n", 622 | "Have a look at the [Python Documentation](https://docs.python.org/2/library/string.html) of Strings to get to know all the methods available!\n", 623 | "You can use them the following way:" 624 | ] 625 | }, 626 | { 627 | "cell_type": "code", 628 | "execution_count": null, 629 | "metadata": {}, 630 | "outputs": [], 631 | "source": [ 632 | "a.upper() #append your object with a dot `.` and the method you want to use e.g. upper()" 633 | ] 634 | }, 635 | { 636 | "cell_type": "markdown", 637 | "metadata": {}, 638 | "source": [ 639 | "You can also split your string and rejoin it afterwards:" 640 | ] 641 | }, 642 | { 643 | "cell_type": "code", 644 | "execution_count": null, 645 | "metadata": { 646 | "scrolled": true 647 | }, 648 | "outputs": [], 649 | "source": [ 650 | "a_splitted = a.split(' ') #split, where the space is!\n", 651 | "print(a_splitted)" 652 | ] 653 | }, 654 | { 655 | "cell_type": "code", 656 | "execution_count": null, 657 | "metadata": {}, 658 | "outputs": [], 659 | "source": [ 660 | "' '.join(a_splitted) #join with a space in between the two variables" 661 | ] 662 | }, 663 | { 664 | "cell_type": "markdown", 665 | "metadata": {}, 666 | "source": [ 667 | "Some more examples:" 668 | ] 669 | }, 670 | { 671 | "cell_type": "code", 672 | "execution_count": null, 673 | "metadata": {}, 674 | "outputs": [], 675 | "source": [ 676 | "a.find('a') # gives the position of the first 'a' in my string!" 677 | ] 678 | }, 679 | { 680 | "cell_type": "code", 681 | "execution_count": null, 682 | "metadata": {}, 683 | "outputs": [], 684 | "source": [ 685 | "a.swapcase() # swaps cases!" 686 | ] 687 | }, 688 | { 689 | "cell_type": "markdown", 690 | "metadata": {}, 691 | "source": [ 692 | "Use the `a.` + `tab` autocompletion if you want to find more examples!" 693 | ] 694 | }, 695 | { 696 | "cell_type": "markdown", 697 | "metadata": {}, 698 | "source": [ 699 | "**Lists**: This is a sequence of \"things\". Those things can have any type you want them to have. Things are called objects in python." 700 | ] 701 | }, 702 | { 703 | "cell_type": "code", 704 | "execution_count": null, 705 | "metadata": {}, 706 | "outputs": [], 707 | "source": [ 708 | "x = [a , 50, 'laughing', 1, 2, 3, True]" 709 | ] 710 | }, 711 | { 712 | "cell_type": "markdown", 713 | "metadata": {}, 714 | "source": [ 715 | "Unlike strings, lists are mutable. This means you can change their content elemenwise, delete elements or add new ones. " 716 | ] 717 | }, 718 | { 719 | "cell_type": "code", 720 | "execution_count": null, 721 | "metadata": {}, 722 | "outputs": [], 723 | "source": [ 724 | "h = x + [1,1,1] #add elements. You can also use the append method: x.append([1,1,1])\n", 725 | "h[2] = 'crying' #assign a new value\n", 726 | "h[0:1] = [] #remove elements\n", 727 | "print(h)" 728 | ] 729 | }, 730 | { 731 | "cell_type": "markdown", 732 | "metadata": {}, 733 | "source": [ 734 | "As for strings, there are methods attached to lists. [Here](https://docs.python.org/2/tutorial/datastructures.html), you will find all of them! " 735 | ] 736 | }, 737 | { 738 | "cell_type": "code", 739 | "execution_count": null, 740 | "metadata": {}, 741 | "outputs": [], 742 | "source": [ 743 | "l = [5 , 7, 3, -1, 2.44]\n", 744 | "l.sort() #sort by value\n", 745 | "l" 746 | ] 747 | }, 748 | { 749 | "cell_type": "markdown", 750 | "metadata": {}, 751 | "source": [ 752 | "Similar to strings, lists have a length and can be indexed (see above). Lists are not like arrays! Adding two lists will concatenate them and not add the respective first, second or third entries of both lists elementwise." 753 | ] 754 | }, 755 | { 756 | "cell_type": "code", 757 | "execution_count": null, 758 | "metadata": {}, 759 | "outputs": [], 760 | "source": [ 761 | "y = [1 , 1, 1]\n", 762 | "z = [2, 2, 2]\n", 763 | "print(y + z)" 764 | ] 765 | }, 766 | { 767 | "cell_type": "markdown", 768 | "metadata": {}, 769 | "source": [ 770 | "So it looks like there is no built-in tool in python, that would allow us to work with multidimensional data or vectors and matrices! Therefore, python comes with several modules, which you can easily import and which will serve very well for this kind of data crunching." 771 | ] 772 | }, 773 | { 774 | "cell_type": "markdown", 775 | "metadata": {}, 776 | "source": [ 777 | "**Arrays**: Arrays are THE data type to work with, if you want to do arithmetics and vector computations. Arrays can be generated by the module `numpy`, that we already got to know earlier. " 778 | ] 779 | }, 780 | { 781 | "cell_type": "markdown", 782 | "metadata": {}, 783 | "source": [ 784 | "### 01_02_03 Numpy " 785 | ] 786 | }, 787 | { 788 | "cell_type": "markdown", 789 | "metadata": {}, 790 | "source": [ 791 | "N-dimensional ndarray's are the core structure of the numpy library. Those are multidimensional container of items of the same type and size. The number of dimensions and items in an array is defined by its shape, which is a tuple of N positive integers that specify the number of items in each dimension. The type of items in the array is specified by a separate data-type object (dtype), one of which is associated with each ndarray.\n", 792 | "\n", 793 | "All ndarrays are homogenous: every item takes up the same size block of memory, and all blocks are interpreted in exactly the same way." 794 | ] 795 | }, 796 | { 797 | "cell_type": "markdown", 798 | "metadata": {}, 799 | "source": [ 800 | "A numpy ndarray look like this one, for example:" 801 | ] 802 | }, 803 | { 804 | "cell_type": "code", 805 | "execution_count": null, 806 | "metadata": {}, 807 | "outputs": [], 808 | "source": [ 809 | "np.arange(10) # one dimensional array" 810 | ] 811 | }, 812 | { 813 | "cell_type": "markdown", 814 | "metadata": {}, 815 | "source": [ 816 | "Or like this:" 817 | ] 818 | }, 819 | { 820 | "cell_type": "code", 821 | "execution_count": null, 822 | "metadata": {}, 823 | "outputs": [], 824 | "source": [ 825 | "np.ones((5,4,3)) # 3-dimensional array with 5 elements in first dimension, 4 elements in 2nd dim, 3 elements in 3rd dim\n", 826 | " # the command np.ones creates an array with all emelents of it having the value 1" 827 | ] 828 | }, 829 | { 830 | "cell_type": "markdown", 831 | "metadata": {}, 832 | "source": [ 833 | "Enough for now about python in general! Lets jump to the next section and have a look at downloading data from the ECMWF server!" 834 | ] 835 | } 836 | ], 837 | "metadata": { 838 | "kernelspec": { 839 | "display_name": "Python 3", 840 | "language": "python", 841 | "name": "python3" 842 | }, 843 | "language_info": { 844 | "codemirror_mode": { 845 | "name": "ipython", 846 | "version": 3 847 | }, 848 | "file_extension": ".py", 849 | "mimetype": "text/x-python", 850 | "name": "python", 851 | "nbconvert_exporter": "python", 852 | "pygments_lexer": "ipython3", 853 | "version": "3.6.6" 854 | } 855 | }, 856 | "nbformat": 4, 857 | "nbformat_minor": 2 858 | } 859 | -------------------------------------------------------------------------------- /02_DownloadDatafromECMWF.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# 02 How to Download Data from ECMWF " 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "For the rest of the book, we will use climate data of ECMWF. Therefore, this chapter deals with downloading the data from the ECMWF server." 15 | ] 16 | }, 17 | { 18 | "cell_type": "markdown", 19 | "metadata": {}, 20 | "source": [ 21 | "# 02_01 Era-Interim" 22 | ] 23 | }, 24 | { 25 | "cell_type": "markdown", 26 | "metadata": {}, 27 | "source": [ 28 | "This section is about downloading Era-Interim data from the ECMWF webpage. \n", 29 | "\n", 30 | "ERA-Interim is a global atmospheric reanalysis from 1979 until 31 August 2019. See chapter 00, to again check what reanalysis data is. The spatial resolution of the data set is approximately 80 km on 60 vertical levels from the surface up to 0.1 hPa. For more information, there is an open-access [journal article](https://rmets.onlinelibrary.wiley.com/doi/full/10.1002/qj.828) describing the ERA-Interim reanalysis from the Quarterly Journal of the Royal Meteorological Society. Era-Interim is now being replaced by ERA5, which produces the same type of data but has a higher spatial resolution and a more recent model. " 31 | ] 32 | }, 33 | { 34 | "cell_type": "markdown", 35 | "metadata": {}, 36 | "source": [ 37 | "### 02_01_01 Registration" 38 | ] 39 | }, 40 | { 41 | "cell_type": "markdown", 42 | "metadata": {}, 43 | "source": [ 44 | "To be able to download any data from the ECMWF page, you will first have to register. This can easily be done [here](https://apps.ecmwf.int/registration/?back=https://www.ecmwf.int/user/login/sso?destination=en/computing/access-computing-facilities/registration-forms). Log in and accept their conditions for the use of data." 45 | ] 46 | }, 47 | { 48 | "cell_type": "markdown", 49 | "metadata": {}, 50 | "source": [ 51 | "### 02_01_02 How to get the Data" 52 | ] 53 | }, 54 | { 55 | "cell_type": "markdown", 56 | "metadata": {}, 57 | "source": [ 58 | "There are two options to get your data: You can either retrieve it directly from the ECMWF website by navigating through their [selection tool](https://apps.ecmwf.int/datasets/data/interim-full-daily/levtype=ml/) and simply clicking on the `retrieve grib` / `retrieve netcdf` button in the end. For this you will not need any python code at all. \n", 59 | "\n", 60 | "Anyway, the ECMWF selection tool might not always fit your needs and you might not be able to download the data that you would like to. E.g. you cannot select more than one year of data there. So if your needs are more specific and you would like to be able to download any kind of data selection on your own in a very dynamic and easy way, some short python code will be the prefered way to go.\n", 61 | "\n", 62 | "The general procedure for this second and more flexible option is very simple: the user writes a request in Python, submits it and retrieves a file (**grib** or **netcdf** format) including the requested data.\n", 63 | "\n", 64 | "If your decided for the second option, follow the inctructions below." 65 | ] 66 | }, 67 | { 68 | "cell_type": "markdown", 69 | "metadata": {}, 70 | "source": [ 71 | "### Preparations" 72 | ] 73 | }, 74 | { 75 | "cell_type": "markdown", 76 | "metadata": {}, 77 | "source": [ 78 | "Before you can write your request in Python code, there are some easy steps to be done. \n", 79 | "\n", 80 | "**First**, you will need to install the client library. Therefore, open a terminal window and tipe in \n", 81 | "\n", 82 | " $ pip install ecmwf-api-client. \n", 83 | "\n", 84 | "**Second**, you have to save your API key for the ECMWF webpage in a file, so that ECMWF will know who is actually requesting the data. This works the following: \n", 85 | "\n", 86 | "Make sure that you are currently logged in and then find your key [here](https://pypi.org/project/ecmwf-api-client/), by clicking on \"retrieve your key at\". Copy paste the content of `$HOME/.ecmwfapirc` into a blank textfile. Save this file to your `$HOME` directory as `.ecmwfapirc`.\n", 87 | "If you work on **Windows**, put the file in `C:\\Users\\\\.ecmwfapirc`. Therefore first save it as `ecmwfapirc.txt` and then rename it into `.ecmwfapirc.`, which will create a file called `.ecmwfapirc`. If you later on run your python script for retrieving the data, it will access this file in order to know your identity. \n" 88 | ] 89 | }, 90 | { 91 | "cell_type": "markdown", 92 | "metadata": {}, 93 | "source": [ 94 | "### The Python Code request\n", 95 | "\n" 96 | ] 97 | }, 98 | { 99 | "cell_type": "markdown", 100 | "metadata": {}, 101 | "source": [ 102 | "Now, we will develope an easy way how to write a data request in Python and adjust it to your specific needs.\n", 103 | "\n", 104 | "Before you write your python code request, have a look at the ECMWF [archive catalogue](https://apps.ecmwf.int/archive-catalogue/) to check, if the data you want to download is available. For any kind of selections the system will update the attributes in a dynamic way to reflect the current availability. So if you, for example, change the steps some of the available parameters will be added or removed.\n", 105 | "\n" 106 | ] 107 | }, 108 | { 109 | "cell_type": "markdown", 110 | "metadata": {}, 111 | "source": [ 112 | "If your data is available, you can now retrieve it! You will first copy paste some general Python code request and later on adjust this code to your own needs. Therefore go [here](https://apps.ecmwf.int/datasets/data/interim-full-daily/levtype=ml/) and select the data you need. You will not be able to select more than one year at a time here, which is maybe not satisfiying. You can adjust this later. Click on `view data retrieval request` and the python code for your selection will appear. Copy paste that one to your python script.\n", 113 | "\n", 114 | "This will look similar to the following code: " 115 | ] 116 | }, 117 | { 118 | "cell_type": "markdown", 119 | "metadata": {}, 120 | "source": [ 121 | " #!/usr/bin/env python\n", 122 | " from ecmwfapi import ECMWFDataServer\n", 123 | " server = ECMWFDataServer()\n", 124 | " server.retrieve({\n", 125 | " \"class\": \"ei\",\n", 126 | " \"dataset\": \"interim\",\n", 127 | " \"date\": \"19790101/19790201/19790301/19790401/19790501/19790601/19790701/19790801/19790901/19791001/19791101/19791201\",\n", 128 | " \"expver\": \"1\",\n", 129 | " \"grid\": \"0.75/0.75\",\n", 130 | " \"levtype\": \"sfc\",\n", 131 | " \"param\": \"167.128\",\n", 132 | " \"stream\": \"moda\",\n", 133 | " \"type\": \"an\",\n", 134 | " \"target\": \"output\",\n", 135 | " })" 136 | ] 137 | }, 138 | { 139 | "cell_type": "markdown", 140 | "metadata": {}, 141 | "source": [ 142 | "Now, let's go through all of the keywords that are included in the code, see what they mean and how we can adjust them. \n", 143 | "You should keep the keywords that are not specified here as they were when copy pasted them from the ECMWF website. \n", 144 | "\n", 145 | "**date**: Those are the dates that will be downloaded. There are two ways to adjust them. The first one is to use the format `\"date\": \"1979-01-01/to/2018-12-31\"` to specify the date range you want to download. This is the simplest solution. However, this might not always work as expected. E.g. for monthly means of daily means, this format will produce an output that looks the following: 19790101/19790102/19790103/... although monthly means of daily means only needs the first of every month (wanted: 19790101/19790201/19790301/...). Therefore, the second option might be the saver one. To get a list of the dates you want to download, go to the [Archive Catalogue](https://apps.ecmwf.int/archive-catalogue/) and select the data you want. You can select various months/years there plus the parameter of interest. Finally click on `View the MARS request` and copy paste the list of dates from the output.\n", 146 | "\n", 147 | "**stream**: This identifies the forecasting system used to generated the data when the same meteorological types are archived. In our case, `stream` is set to `moda`, which means monthly means of daily means. [Here](https://apps.ecmwf.int/codes/grib/format/mars/stream/) is a full list of all the different ECMWF streams.\n", 148 | "\n", 149 | "\n", 150 | "**type**: Determines the type of fields to be retrieved. This keyword makes the selection between observations, images or fields. Examples of fields are: Analysis `an`, Forecast `fc`, Perturbed Forecast `pf`, Control Forecast `cf`. In our case, we will take `an` as `type` since we want to download an analysis product.\n", 151 | "\n", 152 | "**levtype**: This is the type of your level. Common values are: model level `ml`, pressure level `pl`, surface `sfc`, potential vorticity `pv`, potential temperature `pt` and depth `dp`. \n", 153 | "\n", 154 | "**levelist**: This parameter specifies the required levels. Though, you will only have to define `levelist´, if there are any levels required. If you choose the `levtype` to be `sfc` you will therefore not have to specify any `levelist`! Valid values for the `levelist` have to correspond to the selected levtype. For example, model levels can range from 1 to 91. Pressure levels are specified in hPa, e.g. 1000 or 500. \n", 155 | "\n", 156 | "**param**: This is the Parameter you want to download. The best solution to get your parameter code number, is to go to https://apps.ecmwf.int/datasets/data/interim-full-daily/levtype=ml/, select the desired parameter and retrieve the python code for this parameter. You can now copy paste the `param` code line. There is also a [list](http://apps.ecmwf.int/codes/grib/param-db) for the codes of each parameter, but it is fairly hard to find the right parameter there.\n", 157 | "\n", 158 | "**target**: This is the name of the file that you will create by downloading the data. If you want to download the data in netcdf format, you should name the `target`something similar to `mydata.nc`.\n", 159 | "\n", 160 | "**format**: This is the format of your downloaded file. Take `netcdf`if you want the data to be in netcdf format. This is the format we will use for the next chapters. This keyword is not included in the python code that you will get via copy paste from the `view data retrieval request` button of the ECMWF website. So do not forget to add it yourself if you would like to retrieve the data in netcdf format! \n", 161 | "\n", 162 | "Klick [here](https://confluence.ecmwf.int/display/UDOC/MARS+keywords) for more information about the keywords and how to adjust them. \n", 163 | "\n", 164 | "\n", 165 | "\n" 166 | ] 167 | }, 168 | { 169 | "cell_type": "markdown", 170 | "metadata": {}, 171 | "source": [ 172 | "A simple example of code for the **monthly means of daily means** of the **2m temperature** and the period **1999-2000** in format `netcdf` would look like the following: " 173 | ] 174 | }, 175 | { 176 | "cell_type": "markdown", 177 | "metadata": {}, 178 | "source": [ 179 | " from ecmwfapi import ECMWFDataServer\n", 180 | " server = ECMWFDataServer()\n", 181 | " server.retrieve({\n", 182 | " \"class\": \"ei\",\n", 183 | " \"dataset\": \"interim\",\n", 184 | " \"date\": \"19990101/19990201/19990301/19990401/19990501/19990601/19990701/19990801/19990901/19991001/19991101/19991201/20000101/20000201/20000301/20000401/20000501/20000601/20000701/20000801/20000901/20001001/20001101/20001201\",\n", 185 | " \"expver\": \"1\",\n", 186 | " \"grid\": \"0.75/0.75\",\n", 187 | " \"levtype\": \"sfc\",\n", 188 | " \"levelist\": \"200/500/850\",\n", 189 | " \"param\": \"167.128\",\n", 190 | " \"stream\": \"moda\",\n", 191 | " \"type\": \"an\",\n", 192 | " \"format\": \"netcdf\",\n", 193 | " \"target\": \"2mt_1999to2000.nc\",\n", 194 | " })" 195 | ] 196 | }, 197 | { 198 | "cell_type": "markdown", 199 | "metadata": {}, 200 | "source": [ 201 | "Or the code for the **monthly means of daily means** of the **U wind** on pressure levels **850/500/200 hPa** and the period **1999-2000** in format `netcdf`:" 202 | ] 203 | }, 204 | { 205 | "cell_type": "markdown", 206 | "metadata": {}, 207 | "source": [ 208 | " from ecmwfapi import ECMWFDataServer\n", 209 | " server = ECMWFDataServer()\n", 210 | " server.retrieve({\n", 211 | " \"class\": \"ei\",\n", 212 | " \"dataset\": \"interim\",\n", 213 | " \"date\": \"19990101/19990201/19990301/19990401/19990501/19990601/19990701/19990801/19990901/19991001/19991101/19991201/20000101/20000201/20000301/20000401/20000501/20000601/20000701/20000801/20000901/20001001/20001101/20001201\",\n", 214 | " \"expver\": \"1\",\n", 215 | " \"grid\": \"0.75/0.75\",\n", 216 | " \"levtype\": \"pl\",\n", 217 | " \"levelist\": \"200/500/850\",\n", 218 | " \"param\": \"131.128\",\n", 219 | " \"stream\": \"moda\",\n", 220 | " \"type\": \"an\",\n", 221 | " \"format\": \"netcdf\",\n", 222 | " \"target\": \"U_200500850_1999to2000.nc\",\n", 223 | " })" 224 | ] 225 | }, 226 | { 227 | "cell_type": "markdown", 228 | "metadata": {}, 229 | "source": [ 230 | "If you get the following Error-Message: \n", 231 | "\n", 232 | "`self._sslobj.do_handshake()\n", 233 | " SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)`\n", 234 | "\n", 235 | "Simply copy paste the text below to the start of your notebook. This should make the code run properly." 236 | ] 237 | }, 238 | { 239 | "cell_type": "markdown", 240 | "metadata": {}, 241 | "source": [ 242 | " import ssl\n", 243 | " import os\n", 244 | "\n", 245 | " if (not os.environ.get('PYTHONHTTPSVERIFY', '') and\n", 246 | " getattr(ssl, '_create_unverified_context', None)): \n", 247 | " ssl._create_default_https_context = ssl._create_unverified_context" 248 | ] 249 | }, 250 | { 251 | "cell_type": "markdown", 252 | "metadata": {}, 253 | "source": [ 254 | "By now you should have been able to retrieve the data you need! In the next chapter, you will lern how to work with this data and generate plots." 255 | ] 256 | }, 257 | { 258 | "cell_type": "markdown", 259 | "metadata": {}, 260 | "source": [ 261 | "## 02_02 ERA5" 262 | ] 263 | }, 264 | { 265 | "cell_type": "markdown", 266 | "metadata": {}, 267 | "source": [ 268 | "Why better use ERA5 instead of ERA-Interim? There are several reasons: \n", 269 | "\n", 270 | "The first and most obvious one: Era-Interim stopped producing data the 31st August 2019. So if you need current data, you will have to use ERA5. Furthermore, ERA-Interim was based on a data assimilation system and a model, which was operational at ECMWF in 2006. ERA5 thus benefits from a decade of developments in model physics, core dynamics and data assimilation relative to ERA-Interim. ERA5 also has a significantly enhanced horizontal resolution (31 km grid spacing compared to 79 km for ERA-Interim) and provides an enhanced number of output parameters, including for example a 100 m wind product. The move from ERA-Interim to ERA5 represents a **step change in overall quality and level of detail**.\n", 271 | "\n", 272 | "Downloading this newly generated data is quite similar to the process we learned for ERA-Interim:" 273 | ] 274 | }, 275 | { 276 | "cell_type": "markdown", 277 | "metadata": {}, 278 | "source": [ 279 | "### 02_02_01 Registration" 280 | ] 281 | }, 282 | { 283 | "cell_type": "markdown", 284 | "metadata": {}, 285 | "source": [ 286 | "To be able to download ERA5 data, you will first have to register at the Climate Data Store (CDS) infrastructure. This can easily be done [here](https://cds.climate.copernicus.eu/user/register). Log in and accept their conditions for the use of data." 287 | ] 288 | }, 289 | { 290 | "cell_type": "markdown", 291 | "metadata": {}, 292 | "source": [ 293 | "### 02_02_02 How to get the data" 294 | ] 295 | }, 296 | { 297 | "cell_type": "markdown", 298 | "metadata": {}, 299 | "source": [ 300 | "Again, there are two ways to get the data: either follow the selection tool of the webpage, or do your own python code. \n", 301 | "\n", 302 | "#### For the selection tool:\n", 303 | "\n", 304 | "* Go to the [datasets](https://cds.climate.copernicus.eu/cdsapp#!/search?type=dataset) of the C3S Climate Data Store. On the left-hand side menu, you can expand 'Product type' and select the product type of interest. In our case, this is 'Reanalysis' for the ERA5 datasets. Select the dataset you are interested in, e.g. [\"ERA5 monthly averaged data on single levels from 1979 to present\"](https://cds.climate.copernicus.eu/cdsapp#!/dataset/reanalysis-era5-single-levels-monthly-means?tab=overview), if you are interested in monthly averages for 2m temperature, surface winds or total precipitation, or [\"ERA5 monthly averaged data on pressure levels from 1979 to present\"](https://cds.climate.copernicus.eu/cdsapp#!/dataset/reanalysis-era5-pressure-levels-monthly-means?tab=overview) for data at different heights. \n", 305 | "* Click on the tab \"download data\" and you will get to a similar selection tool as on the ECMWF page. This one is even more flexible, so you will e.g. be able to select more than one year and also more than one variable! \n", 306 | "* Once you are done with your selections, accept the Copernicus licence and click on \"submit form\". A \"your requests\" page will be displayed showing the status of your submitted request.\n" 307 | ] 308 | }, 309 | { 310 | "cell_type": "markdown", 311 | "metadata": {}, 312 | "source": [ 313 | "A basic request for 2m temperature and total precipitation for monthly means of the years 1997 and 1998 would look like this:" 314 | ] 315 | }, 316 | { 317 | "cell_type": "markdown", 318 | "metadata": {}, 319 | "source": [ 320 | " import cdsapi\n", 321 | "\n", 322 | " c = cdsapi.Client()\n", 323 | "\n", 324 | " c.retrieve(\n", 325 | " 'reanalysis-era5-single-levels-monthly-means',\n", 326 | " {\n", 327 | " 'product_type':'monthly_averaged_reanalysis',\n", 328 | " 'variable':['2m_temperature','total_precipitation'],\n", 329 | " 'year':['1997','1998'],\n", 330 | " 'month':['01','02','03','04','05','06','07','08','09','10','11','12'],\n", 331 | " 'time':'00:00',\n", 332 | " 'format':'netcdf'\n", 333 | " },\n", 334 | " 'download.nc')" 335 | ] 336 | }, 337 | { 338 | "cell_type": "markdown", 339 | "metadata": {}, 340 | "source": [ 341 | "The file downloaded would be calles 'download.nc' in this case. The default resolution is 0.25° x 0.25° !" 342 | ] 343 | }, 344 | { 345 | "cell_type": "markdown", 346 | "metadata": {}, 347 | "source": [ 348 | "#### For the python code request:" 349 | ] 350 | }, 351 | { 352 | "cell_type": "markdown", 353 | "metadata": {}, 354 | "source": [ 355 | "* #### Install CDS API on your machine\n", 356 | "\n", 357 | " If you have Anaconda installed you can install the CDS API by\n", 358 | " conda config --add channels conda-forge\n", 359 | " conda install cdsapi\n", 360 | " \n", 361 | "* #### Save your API-Key\n", 362 | " \n", 363 | " As before, you will again have to save [your API-Key](https://cds.climate.copernicus.eu/api-how-to). You must be logged in to make this work. Copy paste the two line code into a textfile and save it as '%USERPROFILE%\\.cdsapirc'. In your windows environment, %USERPROFILE% is usually located at C:\\Users\\Username folder. In Linux, save it to '$HOME/.cdsapirc.' Follow the instructions above from \"ERA-Interim\" if you are not familiar with how to create such a file, but this time name it '.cdsapirc' instead of '.ecmwfapirc'!\n", 364 | " \n", 365 | " \n", 366 | "* It is **necessary to agree to the Terms of Use of every dataset that you intend to download**.\n", 367 | "\n", 368 | "\n", 369 | "* Attached to each dataset download form, the \"Show API request\" button displays the python code to be used. Therefore, you can simply follow the instruction from the \"selection tool\" section above, select your data, agree the terms of Use and click on \"Show API request\"!\n", 370 | "\n", 371 | "* #### Refining your CDS API script\n", 372 | "\n", 373 | " For a **geographical area subset**, use key `area` in the form `'area: [northern Boundary(B), western B, southern B, eastern B]`.\n", 374 | "\n", 375 | " For a **different grid resolution**, use key `grid`.\n", 376 | " \n", 377 | " An example for a geographical subset reaching from 50 to 60° N and from -10 to 10° E and a grid resolution of 1°x1°:" 378 | ] 379 | }, 380 | { 381 | "cell_type": "markdown", 382 | "metadata": {}, 383 | "source": [ 384 | " import cdsapi\n", 385 | "\n", 386 | " c = cdsapi.Client()\n", 387 | "\n", 388 | " c.retrieve(\n", 389 | " 'reanalysis-era5-single-levels-monthly-means',\n", 390 | " {\n", 391 | " 'product_type':'monthly_averaged_reanalysis',\n", 392 | " 'variable':['2m_temperature','total_precipitation'],\n", 393 | " 'year':['1997','1998'],\n", 394 | " 'month':['01','02','03','04','05','06','07','08','09','10','11','12'],\n", 395 | " 'time':'00:00',\n", 396 | " 'area': [60, -10, 50, 10], # North, West, South, East\n", 397 | " 'grid': [1.0, 1.0], # east-west (longitude) and north-south resolution (latitude). Default: 0.25 x 0.25\n", 398 | " 'format':'netcdf'\n", 399 | " },\n", 400 | " 'ERA5_2mt_precip.nc')" 401 | ] 402 | }, 403 | { 404 | "cell_type": "markdown", 405 | "metadata": {}, 406 | "source": [ 407 | "This time, we changed the name from 'donwoad.nc' to something more meaningful." 408 | ] 409 | }, 410 | { 411 | "cell_type": "markdown", 412 | "metadata": {}, 413 | "source": [ 414 | "If you download data like in the above example, you will notive that your data goes from 0° to 360° East in terms of longitude. Since a normal world map goes from -180° to 180° E, a more useful dataset would have the same coordinate system. \n", 415 | "\n", 416 | "We can change the longitudinal axis from 0°-360° to -180°-180° by adding the following to our python program:\n", 417 | "\n", 418 | "`'area': [90, -180, -90, 180],`" 419 | ] 420 | }, 421 | { 422 | "cell_type": "markdown", 423 | "metadata": {}, 424 | "source": [ 425 | "As you see, the whole process of requesting ERA5 data works pretty similar to ERA-Interim. It is even a bit easier: the selection tool works better and you do not have those annoying codes for your parameters anymore! \n", 426 | "\n", 427 | "In the following sections, we will work with ERA5 data with a grid resolution of 0.75 x 0.75, since a higher resolution would result in a huge amount of data and therefore longer processing times. A resolution of 0.75 x 0.75 is enough for us to have a qualitative look at the data!" 428 | ] 429 | } 430 | ], 431 | "metadata": { 432 | "kernelspec": { 433 | "display_name": "Python 3", 434 | "language": "python", 435 | "name": "python3" 436 | }, 437 | "language_info": { 438 | "codemirror_mode": { 439 | "name": "ipython", 440 | "version": 3 441 | }, 442 | "file_extension": ".py", 443 | "mimetype": "text/x-python", 444 | "name": "python", 445 | "nbconvert_exporter": "python", 446 | "pygments_lexer": "ipython3", 447 | "version": "3.6.6" 448 | } 449 | }, 450 | "nbformat": 4, 451 | "nbformat_minor": 2 452 | } 453 | -------------------------------------------------------------------------------- /03_NetCDFFiles.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# 02 Working with NetCDF Files" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "We already had a very quick introduction on NetCDF files in the very first chapter of this book. Now we will get to know some more details!" 15 | ] 16 | }, 17 | { 18 | "cell_type": "markdown", 19 | "metadata": {}, 20 | "source": [ 21 | "## 02_01 What is a NetCDF File?" 22 | ] 23 | }, 24 | { 25 | "cell_type": "markdown", 26 | "metadata": {}, 27 | "source": [ 28 | "Most of today's meteorological data are stored in the NetCDF format (``*.nc``), a format that supports the creation, access, and sharing of array-oriented scientific data. We will, from now on, work with this type of data storage.\n", 29 | "\n", 30 | "In the python basics chapter, we introduced numpy arrays as a new tool for arithmetics and vector computation. Since the climate data we work with is always multi-dimensional and can be stored in such an array, you might think that numpy arrays were the perfect tool to store our data. So why do we still use NetCDF Files for storing climatological and meteorological data? What is the main difference to a normal numpy array?\n", 31 | "\n", 32 | "NetCDF Files can be read by the python module `xarray`! And those xarrays are pretty cool:\n", 33 | "\n", 34 | "Xarray introduces labels in the form of dimensions, coordinates and attributes on top of raw NumPy-like multidimensional arrays. This allows for a more intuitive, more concise, and less error-prone developer experience. \n", 35 | "That means: In a numpy array you could only store e.g. the temperature data of all latitudes and longitudes for a certain time in an array. However, real-world datasets are usually more than just raw numbers: how would you know, where each of the elements in your array is located in a spatial grid? Here comes xarray: you can store a temperature field **and** a label which encodes information about how the array values map to locations in the spatial grid! \n", 36 | "\n", 37 | "So know all of a sudden, an axis that would have been called 0,1,2,... in a numpy array gets a meaningful name such as \"longitude\", \"latitude\" or \"time\"! That helps us a lot when dealing with this huge amount of data stored in NetCDF files. Still, they behave like normal arrays, which means you can do normal arithmetics and vector computations with them!\n", 38 | "\n", 39 | "Another advantage of an xarray in comparison to numpy arrays is, that you can store metadata on your object, e.g. what unit your data has or who provided the dataset,...\n", 40 | "Furthermore, a NumPy array can only have one data type, while NetCDF can hold heterogeneous data in one array.\n", 41 | "\n", 42 | "We will be able to read the NetCDF files in a python interpreter which will transform them into a xarray and do computations and plots with them. \n", 43 | "Let's get started and see how that works!" 44 | ] 45 | }, 46 | { 47 | "cell_type": "markdown", 48 | "metadata": {}, 49 | "source": [ 50 | "## 02_02 NetCDF Data - How to get it, read it and understand it" 51 | ] 52 | }, 53 | { 54 | "cell_type": "markdown", 55 | "metadata": {}, 56 | "source": [ 57 | "### 02_02_01 Importing the Modules and some Options" 58 | ] 59 | }, 60 | { 61 | "cell_type": "markdown", 62 | "metadata": {}, 63 | "source": [ 64 | "Before we can start to look at our data, we will always need to import our three most important modules: `matplotlib`, `numpy` and `xarray`. We will do some more presettings, so everything will look well sized and nice in our notebook." 65 | ] 66 | }, 67 | { 68 | "cell_type": "code", 69 | "execution_count": null, 70 | "metadata": {}, 71 | "outputs": [], 72 | "source": [ 73 | "# Display the plots in the notebook:\n", 74 | "%matplotlib inline" 75 | ] 76 | }, 77 | { 78 | "cell_type": "code", 79 | "execution_count": null, 80 | "metadata": {}, 81 | "outputs": [], 82 | "source": [ 83 | "# Import the tools we are going to need today:\n", 84 | "import matplotlib.pyplot as plt # plotting library\n", 85 | "import numpy as np # numerical library\n", 86 | "import xarray as xr # netCDF library\n", 87 | "import cartopy # Map projections libary, needed to display world maps\n", 88 | "import cartopy.crs as ccrs # Projections list\n", 89 | "# Some defaults:\n", 90 | "plt.rcParams['figure.figsize'] = (12, 5) # Default plot size\n", 91 | "np.set_printoptions(threshold=20) # avoid to print very large arrays on screen\n", 92 | "# The commands below are to ignore certain warnings.\n", 93 | "import warnings\n", 94 | "warnings.filterwarnings('ignore')" 95 | ] 96 | }, 97 | { 98 | "cell_type": "markdown", 99 | "metadata": {}, 100 | "source": [ 101 | "### 02_02_01 Get NetCDF Data" 102 | ] 103 | }, 104 | { 105 | "cell_type": "markdown", 106 | "metadata": {}, 107 | "source": [ 108 | "The data we are going to use in this chapter is from ECMWF. We will use a dataset, downloaded with the methods explained in the chapter 02. \n", 109 | "\n", 110 | "The dataset contains the monthly means of daily means of the 2 meter temperature as well as the total precipitation. You can either get the data [here]() or download it on your own, following the instructions of the previous chapter!" 111 | ] 112 | }, 113 | { 114 | "cell_type": "markdown", 115 | "metadata": {}, 116 | "source": [ 117 | "### 02_02_02 Read NetCDF Data" 118 | ] 119 | }, 120 | { 121 | "cell_type": "markdown", 122 | "metadata": {}, 123 | "source": [ 124 | "NetCDF files are binary files, which means that you can't just open them in a text editor. You need a special reader for it. \n", 125 | "\n", 126 | "To handle NetCDF data, python comes with the module [xarray](http://xarray.pydata.org/en/stable/). We already imported it before as `xr`! Xarray provides a lot of useful methods. The one to read NetCDF files is `xr.open_dataset(datadir)`. See the example underneath to understand how it works!" 127 | ] 128 | }, 129 | { 130 | "cell_type": "code", 131 | "execution_count": null, 132 | "metadata": {}, 133 | "outputs": [], 134 | "source": [ 135 | "dataDIR = './data/ERA5-LowRes-MonthlyAvg-t2m_tp.nc' \n", 136 | "# the path to which you saved the netCDF file in the step before\n", 137 | "# Here I downloaded the file in the \"data\" folder which I placed in the same folder as the notebook --> the dot \".\" \n", 138 | "# in the beginning means \"look in the current foler\"\n", 139 | "\n", 140 | "ds = xr.open_dataset(dataDIR) # the data of the netCDF File will be stored in \"ds\" (dataset)" 141 | ] 142 | }, 143 | { 144 | "cell_type": "markdown", 145 | "metadata": {}, 146 | "source": [ 147 | "**Note**: you'll have to give an absolute or relative path to the file for this to work. For example ``'C:\\PATH\\TO\\FILE\\ERA5-MonthlyAvg-2tm_tp-75-rolled.nc'`` in windows." 148 | ] 149 | }, 150 | { 151 | "cell_type": "markdown", 152 | "metadata": {}, 153 | "source": [ 154 | "Let's see, what our **ds** object looks like!" 155 | ] 156 | }, 157 | { 158 | "cell_type": "code", 159 | "execution_count": null, 160 | "metadata": {}, 161 | "outputs": [], 162 | "source": [ 163 | "ds" 164 | ] 165 | }, 166 | { 167 | "cell_type": "markdown", 168 | "metadata": {}, 169 | "source": [ 170 | "Alright. So let's go through this step by step. \n", 171 | "\n", 172 | "Our `ds`is of the type `xarray.DataSet` (convince yourself above). A`xarray.Dataset` generally consists of the following key properties: \n", 173 | "\n", 174 | "* **Dimensions**: dimension names for each axis (in our case:'latitude', 'longitude', 'month'), specifies the number of elements for each Dimension, e.g. `time: 488`.\n", 175 | "* **Coordinates**: a container of arrays (coordinates) that label each point. E.g.: 1-dim arrays of numbers (like a coordinate vector) , DateTime objects (for time labeling), or strings. On the right hand side you can see the actual values that the coordinates have.\n", 176 | "* **Variables**: a numpy.ndarray holding the array’s values, this is where the actual data is stored! In our case, we can expect three arrays of size [241, 480, 488].\n", 177 | "* **Attributes**: does not contain any data, is a container that holds arbitrary metadata (attributes), like the title of the data, additional information about the dataset,...\n", 178 | "\n", 179 | "A `xarray.DataSet` is a collection of `xarray.DataArray`s. Each NetCDF file contains such a DataSet.\n", 180 | "\n", 181 | "So what is a `xarray.DataArray`?\n", 182 | "It is a multi-dimensional array with labeled or named dimensions. DataArray objects add metadata such as dimension names, coordinates, and attributes (defined below) to underlying “unlabeled” data structures such as our normal numpy arrays.\n", 183 | "\n", 184 | "In our example above, each `xarray.DataArray` would consist of one of the listed Data Variables, e.g. `t2m` the 2 meter temperature. Together those three DataArrays form a `xarray.DataSet`, which in turn is stored in our downloaded netCDF file. \n" 185 | ] 186 | }, 187 | { 188 | "cell_type": "markdown", 189 | "metadata": {}, 190 | "source": [ 191 | "The **xarray logo** gives us a visual understanding of how a xarray Dataset looks like:" 192 | ] 193 | }, 194 | { 195 | "cell_type": "markdown", 196 | "metadata": {}, 197 | "source": [ 198 | "![4D-Data: Data of a specific area dependent on height z and time](xarraylogo.png \"\")" 199 | ] 200 | }, 201 | { 202 | "cell_type": "markdown", 203 | "metadata": {}, 204 | "source": [ 205 | "![](dataset-diagram.png \"\")" 206 | ] 207 | }, 208 | { 209 | "cell_type": "markdown", 210 | "metadata": {}, 211 | "source": [ 212 | "For us, the most interesting two properties will be the coordinates and the variables. Let's have a closer look at them! " 213 | ] 214 | }, 215 | { 216 | "cell_type": "markdown", 217 | "metadata": {}, 218 | "source": [ 219 | "#### Coordinates" 220 | ] 221 | }, 222 | { 223 | "cell_type": "markdown", 224 | "metadata": {}, 225 | "source": [ 226 | "You can adress all the different properties of the `xarray.DataArray` via the dot `.` expression:" 227 | ] 228 | }, 229 | { 230 | "cell_type": "code", 231 | "execution_count": null, 232 | "metadata": {}, 233 | "outputs": [], 234 | "source": [ 235 | "ds.time # adress the coordinate 'time'" 236 | ] 237 | }, 238 | { 239 | "cell_type": "markdown", 240 | "metadata": {}, 241 | "source": [ 242 | "**Time** goes from 1979 to 2019 and has a resolution of one month. You can read this out of the data listed to the right of `time` at the coordinates. The type of the given values is datetime64." 243 | ] 244 | }, 245 | { 246 | "cell_type": "markdown", 247 | "metadata": {}, 248 | "source": [ 249 | "The **spatial coordinates** are as easy to understand:" 250 | ] 251 | }, 252 | { 253 | "cell_type": "code", 254 | "execution_count": null, 255 | "metadata": {}, 256 | "outputs": [], 257 | "source": [ 258 | "ds.latitude" 259 | ] 260 | }, 261 | { 262 | "cell_type": "markdown", 263 | "metadata": {}, 264 | "source": [ 265 | "Latitude goes from 90 to -90 and has the unit 'degrees north'. The spatial resolution of this dataset is 0.75°, as you can easily see from the given values. " 266 | ] 267 | }, 268 | { 269 | "cell_type": "code", 270 | "execution_count": null, 271 | "metadata": {}, 272 | "outputs": [], 273 | "source": [ 274 | "ds.longitude" 275 | ] 276 | }, 277 | { 278 | "cell_type": "markdown", 279 | "metadata": {}, 280 | "source": [ 281 | "#### Variables" 282 | ] 283 | }, 284 | { 285 | "cell_type": "markdown", 286 | "metadata": {}, 287 | "source": [ 288 | "As for coordinates, variables can also be accessed directly from the dataset via the `.` syntax! By doing this, you will actually extract one DataArray from the whole DataSet. We can try it now for e.g. the ` t2m` variable." 289 | ] 290 | }, 291 | { 292 | "cell_type": "code", 293 | "execution_count": null, 294 | "metadata": {}, 295 | "outputs": [], 296 | "source": [ 297 | "ds.t2m" 298 | ] 299 | }, 300 | { 301 | "cell_type": "markdown", 302 | "metadata": {}, 303 | "source": [ 304 | "The **attributes** of a variable are extremely important, they cary the *metadata* and must be specified by the data provider. Here we can read in which units the variable is defined (K for Kelvin), as well as a description of the variable (the \"long_name\" attribute), and sometimes also what the valid value range is (not here). \n", 305 | "\n", 306 | "From the upper description \"... values with dtype=float32\", we can also see the data type of the values of our data: in our case this is \"float32\", a floating point number with 32 bits. " 307 | ] 308 | }, 309 | { 310 | "cell_type": "code", 311 | "execution_count": null, 312 | "metadata": { 313 | "scrolled": true 314 | }, 315 | "outputs": [], 316 | "source": [ 317 | "ds.tp" 318 | ] 319 | }, 320 | { 321 | "cell_type": "markdown", 322 | "metadata": {}, 323 | "source": [ 324 | "The total precipitation is given in m! Since mm is the common unit for precipitation data, we will need to calculate that in order to get expected values of precipitation." 325 | ] 326 | }, 327 | { 328 | "cell_type": "code", 329 | "execution_count": null, 330 | "metadata": {}, 331 | "outputs": [], 332 | "source": [ 333 | "tp_mm = ds.tp / 1000" 334 | ] 335 | }, 336 | { 337 | "cell_type": "code", 338 | "execution_count": null, 339 | "metadata": {}, 340 | "outputs": [], 341 | "source": [ 342 | "ds.tp.long_name # adress the attributes of a variable via a second '.'!" 343 | ] 344 | }, 345 | { 346 | "cell_type": "markdown", 347 | "metadata": {}, 348 | "source": [ 349 | "## 02_03 NetCDF Data - How to work with it" 350 | ] 351 | }, 352 | { 353 | "cell_type": "markdown", 354 | "metadata": {}, 355 | "source": [ 356 | "Let's do some first, simple analyses with our XArray-Dataset!" 357 | ] 358 | }, 359 | { 360 | "cell_type": "markdown", 361 | "metadata": {}, 362 | "source": [ 363 | "### Mean of a variable along a certain dimension " 364 | ] 365 | }, 366 | { 367 | "cell_type": "markdown", 368 | "metadata": {}, 369 | "source": [ 370 | "A lot of times, you will want to average your data along a certain dimension axis: Right now the data has three dimensions (latitude, longitude, month). To be able to plot the data (2- or 1-dimensional!), you will need to average over at least one of those three dimensions. This is very easy: there is a method `.mean(dim = 'the_dimension_you_want_to_average_over')` attached to your DataSet variables. So you can just type: `ds.variable_name.mean(dim = '...')`! \n", 371 | "\n", 372 | "Let's first try to **average over one dimension**, e.g. time. This means, we will get a DataSet of mean sealevel pressure for all the lons and lats, but averaged over the whole period from 1979 to 2019!" 373 | ] 374 | }, 375 | { 376 | "cell_type": "code", 377 | "execution_count": null, 378 | "metadata": {}, 379 | "outputs": [], 380 | "source": [ 381 | "t2m_avg = ds.t2m.mean(dim = 'time')\n", 382 | "t2m_avg" 383 | ] 384 | }, 385 | { 386 | "cell_type": "markdown", 387 | "metadata": {}, 388 | "source": [ 389 | "As expected, the time coordinate disappears! Instead, we get a new, two-dimensional xarray DataArray of dimensions [latitude, longitude]. We can plot this, to make sure we understood, what the new data looks like:" 390 | ] 391 | }, 392 | { 393 | "cell_type": "code", 394 | "execution_count": null, 395 | "metadata": {}, 396 | "outputs": [], 397 | "source": [ 398 | "t2_tavg = ds.t2m.mean(dim='time')\n", 399 | "t2c_tavg = t2_tavg - 273.15\n", 400 | "ax = plt.axes(projection=ccrs.Robinson())\n", 401 | "t2c_tavg.plot(ax=ax, transform=ccrs.PlateCarree(), cmap='inferno', center=False, \n", 402 | " vmin=-40, vmax=20, levels=7, cbar_kwargs={'label':'°C'}) \n", 403 | "ax.set_title('Average annual 2m air temperature, ERA5 1979-2019')\n", 404 | "ax.coastlines(); ax.gridlines(); " 405 | ] 406 | }, 407 | { 408 | "cell_type": "markdown", 409 | "metadata": {}, 410 | "source": [ 411 | "We are looking at the mean 2m temperature, expressed in ° Celcius. Such time averages are often written with a bar on top of them:\n", 412 | "\n", 413 | "$\\overline{msl} = temporal\\_mean(msl)$\n", 414 | "\n", 415 | "Don't worry about plotting right now, we will deal with this later. The plots are just here, to give you a visual understanding of our data.\n", 416 | "\n", 417 | "\n", 418 | "Let's try the same thing for the longitudinal dimension:" 419 | ] 420 | }, 421 | { 422 | "cell_type": "code", 423 | "execution_count": null, 424 | "metadata": {}, 425 | "outputs": [], 426 | "source": [ 427 | "ds.t2m.mean(dim ='longitude') # we are left with two dimensions: the latitudinal coordinate axis and how the TOA\n", 428 | " # Shortwave Flux for All-Sky Conditions evolves with time for each latitude" 429 | ] 430 | }, 431 | { 432 | "cell_type": "markdown", 433 | "metadata": {}, 434 | "source": [ 435 | "You can also build the **average over two dimensions:**" 436 | ] 437 | }, 438 | { 439 | "cell_type": "code", 440 | "execution_count": null, 441 | "metadata": {}, 442 | "outputs": [], 443 | "source": [ 444 | "ds.t2m.mean(dim = ['time', 'longitude'])" 445 | ] 446 | }, 447 | { 448 | "cell_type": "code", 449 | "execution_count": null, 450 | "metadata": {}, 451 | "outputs": [], 452 | "source": [ 453 | "zonal_t2m_avg = ds.t2m.mean(dim = 'time').mean(dim = 'longitude') # this gives the same output as the command above!" 454 | ] 455 | }, 456 | { 457 | "cell_type": "markdown", 458 | "metadata": {}, 459 | "source": [ 460 | "This leaves us with only the meridional component of our dataset, which means we calculated the average over time and all longitudes for each latitude. This is also called the \"zonal average\".\n", 461 | "\n", 462 | "**\"Zonal average\"** means \"along a latitude circle\". It is often writen with ``[]`` or ``<>`` in formulas:\n", 463 | "\n", 464 | "$\\left[ \\overline{msl} \\right] = zonal\\_mean(temporal\\_mean(msl))$\n", 465 | "\n", 466 | "Note that the two operators are commutative, i.e.:\n", 467 | "\n", 468 | "$\\left[ \\overline{msl} \\right] = \\overline{\\left[ msl \\right]}$\n" 469 | ] 470 | }, 471 | { 472 | "cell_type": "code", 473 | "execution_count": null, 474 | "metadata": {}, 475 | "outputs": [], 476 | "source": [ 477 | "ds_zavg = ds.t2m.mean(dim=['time', 'longitude'])\n", 478 | "t2m_zavg = ds_zavg - 273.15\n", 479 | "t2m_zavg.plot();\n", 480 | "plt.xlim([-90, 90]);\n", 481 | "plt.title('2 meter temperature');\n", 482 | "plt.ylabel('°C');" 483 | ] 484 | }, 485 | { 486 | "cell_type": "markdown", 487 | "metadata": {}, 488 | "source": [ 489 | "If you do not specify, over which dimension you want to average, you will get the **average over all your dimensions**. The result of this will be a scalar: The mean sealevel pressure over all longitudes, latitudes and over the whole year." 490 | ] 491 | }, 492 | { 493 | "cell_type": "code", 494 | "execution_count": null, 495 | "metadata": {}, 496 | "outputs": [], 497 | "source": [ 498 | "ds.t2m.mean()" 499 | ] 500 | }, 501 | { 502 | "cell_type": "markdown", 503 | "metadata": {}, 504 | "source": [ 505 | "So the msl over all longitudes, latitudes and over the whole year is 278.36!\n", 506 | "Well, this means we have an average temperature of ca. 5° Celcius on our planet earth. That is obviously way to cold: the global average air temperature at the Earth surface is about 14°C . \n", 507 | "The error is due to the fact that we averaged over latitudes!" 508 | ] 509 | }, 510 | { 511 | "cell_type": "markdown", 512 | "metadata": {}, 513 | "source": [ 514 | "### Averaging over Latitudes" 515 | ] 516 | }, 517 | { 518 | "cell_type": "markdown", 519 | "metadata": {}, 520 | "source": [ 521 | "Whenever you **average over latitudes**, you will need to **take into account that our planet is a sphere** and do arithmetics before you calculate the average. **Otherwise** you will get **wrong results**! Go to chapter 08_AveragingoverSphere, to see how this works. " 522 | ] 523 | }, 524 | { 525 | "cell_type": "markdown", 526 | "metadata": {}, 527 | "source": [ 528 | "### Maxima and Minima" 529 | ] 530 | }, 531 | { 532 | "cell_type": "markdown", 533 | "metadata": {}, 534 | "source": [ 535 | "You might not only want to calculate an average of your dataset. You can for example also calculate the **minimum** or **maximum** value of your dataset! Let's try it: " 536 | ] 537 | }, 538 | { 539 | "cell_type": "code", 540 | "execution_count": null, 541 | "metadata": {}, 542 | "outputs": [], 543 | "source": [ 544 | "ds_min = ds.t2m.min()\n", 545 | "ds_max = ds.t2m.max()\n", 546 | "print(ds_min-273.15)\n", 547 | "print(ds_max-273.15)" 548 | ] 549 | }, 550 | { 551 | "cell_type": "markdown", 552 | "metadata": {}, 553 | "source": [ 554 | "As for the average, you can also search for a **minimum or maximum value along a certain axis**:" 555 | ] 556 | }, 557 | { 558 | "cell_type": "code", 559 | "execution_count": null, 560 | "metadata": {}, 561 | "outputs": [], 562 | "source": [ 563 | "ds.t2m.min(dim = 'longitude') - 273.15" 564 | ] 565 | }, 566 | { 567 | "cell_type": "markdown", 568 | "metadata": {}, 569 | "source": [ 570 | "This gives me the minimum values of all longitudes for each latitude and each month. \n", 571 | "\n", 572 | "Let's also have a look at the minimum values of all longitudes and all year, for each latitude:" 573 | ] 574 | }, 575 | { 576 | "cell_type": "code", 577 | "execution_count": null, 578 | "metadata": {}, 579 | "outputs": [], 580 | "source": [ 581 | "ds.t2m.min(dim = ['longitude', 'time']) - 273.15" 582 | ] 583 | }, 584 | { 585 | "cell_type": "markdown", 586 | "metadata": {}, 587 | "source": [ 588 | "There are more methods attached to the DataSet variables: e.g. [standard deviation](http://xarray.pydata.org/en/stable/generated/xarray.Dataset.std.html),..." 589 | ] 590 | }, 591 | { 592 | "cell_type": "markdown", 593 | "metadata": {}, 594 | "source": [ 595 | "### Data Selection based on the Coordinate Axes" 596 | ] 597 | }, 598 | { 599 | "cell_type": "markdown", 600 | "metadata": {}, 601 | "source": [ 602 | "We cannot only average over dimensions, but also select some fix value of a specific axis, e.g. a certain longitude and latitude!\n", 603 | "\n", 604 | "One of the things we'd also like to do is **select** certain years, which is an easy task with xarray and the method `.sel()`:" 605 | ] 606 | }, 607 | { 608 | "cell_type": "code", 609 | "execution_count": null, 610 | "metadata": {}, 611 | "outputs": [], 612 | "source": [ 613 | "ds_avg_lon = ds.t2m.mean(dim ='longitude')\n", 614 | "ds_avg_lon.sel(time = '2008') # select the year 2008" 615 | ] 616 | }, 617 | { 618 | "cell_type": "code", 619 | "execution_count": null, 620 | "metadata": {}, 621 | "outputs": [], 622 | "source": [ 623 | "ds_avg_lon_08 = ds.t2m.mean(dim = 'longitude').sel(time = '2008')\n", 624 | "# You can also do both, the averaging and the selection in just one step! The result is the same.\n", 625 | "# this is possible for all kinds of methods attached to our dataset ds. If you want to plot in the same step, just \n", 626 | "# attach .plot()!\n", 627 | "# Anyway, to keep everything ordered, it is sometimes better to do it in single steps than to create one veeeeery long\n", 628 | "# command that nobody understands lateron." 629 | ] 630 | }, 631 | { 632 | "cell_type": "markdown", 633 | "metadata": {}, 634 | "source": [ 635 | "Let's see what we have created:" 636 | ] 637 | }, 638 | { 639 | "cell_type": "code", 640 | "execution_count": null, 641 | "metadata": { 642 | "scrolled": true 643 | }, 644 | "outputs": [], 645 | "source": [ 646 | "ds_avg_lon_08 = ds_avg_lon_08 - 273.15;\n", 647 | "ds_avg_lon_08.T.plot()\n", 648 | "plt.ylim([-90, 90]);\n" 649 | ] 650 | }, 651 | { 652 | "cell_type": "code", 653 | "execution_count": null, 654 | "metadata": { 655 | "scrolled": true 656 | }, 657 | "outputs": [], 658 | "source": [ 659 | "ds.t2m.sel(time = \"2006-12\") # we can even select a special month of a year, this makes the time coordinate vanish" 660 | ] 661 | }, 662 | { 663 | "cell_type": "code", 664 | "execution_count": null, 665 | "metadata": { 666 | "scrolled": false 667 | }, 668 | "outputs": [], 669 | "source": [ 670 | "ds.t2m.sel(time = slice('1999','2006')) # or select a series of years, with a slice object!" 671 | ] 672 | }, 673 | { 674 | "cell_type": "markdown", 675 | "metadata": {}, 676 | "source": [ 677 | "**Note**: in the code above, we used new python syntax! The **slice** object. A slice object is used for indexing in python and always conists of three parameters: start, stop, step. The start value gives python the index, where it should start selecting the data, stop where it should stop and step, in which steps the data should be indexed. Step is 1 by default. We can also set it to another value, e.g. 2: " 678 | ] 679 | }, 680 | { 681 | "cell_type": "code", 682 | "execution_count": null, 683 | "metadata": {}, 684 | "outputs": [], 685 | "source": [ 686 | "t2m_selection = ds.t2m.sel(time = slice('1999', '2006', 2)) # select the values of the years 1999-2006 in steps of 2 \n", 687 | " # --> only every second month!\n", 688 | "t2m_selection" 689 | ] 690 | }, 691 | { 692 | "cell_type": "code", 693 | "execution_count": null, 694 | "metadata": {}, 695 | "outputs": [], 696 | "source": [ 697 | "t2m_selection.time" 698 | ] 699 | }, 700 | { 701 | "cell_type": "markdown", 702 | "metadata": {}, 703 | "source": [ 704 | "As for time, you could also select a certain area of the globe via a slice object:" 705 | ] 706 | }, 707 | { 708 | "cell_type": "code", 709 | "execution_count": null, 710 | "metadata": { 711 | "scrolled": true 712 | }, 713 | "outputs": [], 714 | "source": [ 715 | "region = ds.t2m.mean(dim = 'time').sel(longitude = slice(-20,50)) #select all longitudes from -20° to 50°E\n", 716 | "\n", 717 | "ax = plt.axes(projection=ccrs.PlateCarree()) \n", 718 | "region = region - 273.15\n", 719 | "region.plot(ax=ax, transform=ccrs.PlateCarree()) \n", 720 | "ax.add_feature(cartopy.feature.BORDERS); \n", 721 | "ax.coastlines();\n", 722 | "plt.xlim([-20,50]);" 723 | ] 724 | }, 725 | { 726 | "cell_type": "code", 727 | "execution_count": null, 728 | "metadata": { 729 | "scrolled": true 730 | }, 731 | "outputs": [], 732 | "source": [ 733 | "ds.t2m.latitude\n" 734 | ] 735 | }, 736 | { 737 | "cell_type": "markdown", 738 | "metadata": {}, 739 | "source": [ 740 | "Be careful, to slice the data in the correct way: e.g. slicing the longitude via `slice(40,-20)` will not work, since the data is listed from -180 to 180°E and not from 180 to -180°E. \n", 741 | "\n", 742 | "However, if we have a closer look at the latitudinal coordinate axis, we see: latitude goes from 90°N to -90°S! Therefore slicing over latitudes works the following: `latitude = slice(40,-40)`!\n", 743 | "\n", 744 | "There is another **problem** that you will encounter **when selecting**: If you try to select values across the 180°E meridian, this will not work! Let's try it:" 745 | ] 746 | }, 747 | { 748 | "cell_type": "code", 749 | "execution_count": null, 750 | "metadata": {}, 751 | "outputs": [], 752 | "source": [ 753 | "ds.t2m.sel(longitude = slice(160.5,-160.5)) # we try to select from 160.5 to -160.5 --> across the 180° meridian!" 754 | ] 755 | }, 756 | { 757 | "cell_type": "markdown", 758 | "metadata": {}, 759 | "source": [ 760 | "We get `longitude: 0`! Obviously this did not work. That is, because we tried to select longitude values across the borders of the coordinate axis array. This does not work.\n", 761 | "\n", 762 | "Let's try something else:\n" 763 | ] 764 | }, 765 | { 766 | "cell_type": "code", 767 | "execution_count": null, 768 | "metadata": {}, 769 | "outputs": [], 770 | "source": [ 771 | "ds.roll(longitude = 1)" 772 | ] 773 | }, 774 | { 775 | "cell_type": "markdown", 776 | "metadata": {}, 777 | "source": [ 778 | "If we now look at the longitude coordinate axis, we notice that it changed from starting at -180°E to starting from 179.25°E! We \"rolled\" the axes, which means we rotated all variables. This is done by the `ds.roll(dim = offset)` syntax. In our case, the dimension is the longitude axis and the offset was 1. \n", 779 | "\n", 780 | "We can also roll with an offset of e.g. 50:" 781 | ] 782 | }, 783 | { 784 | "cell_type": "code", 785 | "execution_count": null, 786 | "metadata": { 787 | "scrolled": true 788 | }, 789 | "outputs": [], 790 | "source": [ 791 | "ds_rolled = ds.roll(longitude = 50)\n", 792 | "ds_rolled" 793 | ] 794 | }, 795 | { 796 | "cell_type": "markdown", 797 | "metadata": {}, 798 | "source": [ 799 | "And now try again to select longitudes from 160 to -160°E (from 160.5 to , since 160°E is not an existing coordinate):" 800 | ] 801 | }, 802 | { 803 | "cell_type": "code", 804 | "execution_count": null, 805 | "metadata": { 806 | "scrolled": true 807 | }, 808 | "outputs": [], 809 | "source": [ 810 | "ds_rolled.sel(longitude = slice(160.5,-160.5))" 811 | ] 812 | }, 813 | { 814 | "cell_type": "markdown", 815 | "metadata": {}, 816 | "source": [ 817 | "Now it works! We solved the problem of selecting across the borders of the coordinate axis array." 818 | ] 819 | }, 820 | { 821 | "cell_type": "markdown", 822 | "metadata": {}, 823 | "source": [ 824 | "One last example of selection based on the coordinate axes!\n", 825 | "Lets plot temperature for the coordinates of Innsbruck( more or less since we have only 0.75° resolution) for the year 2018.\n", 826 | "\n", 827 | "**Note:** If you do not want to type in the correct coordinates (e.g. you don't know if a 160.0° exist or if it is 160.5°), you can instead specify the keyword argument `method` as `nearest`. This will automatically select the values of coordinates, that are nearest to the ones you specified:\n" 828 | ] 829 | }, 830 | { 831 | "cell_type": "code", 832 | "execution_count": null, 833 | "metadata": {}, 834 | "outputs": [], 835 | "source": [ 836 | "ibk_18 = ds.t2m.sel(time = '2018', latitude = 47, longitude = 11, method = 'nearest')-273.15 \n", 837 | "# instead of typing 47.25 and 11.25, just type the integer value and \"method = 'nearest'\"\n", 838 | "ibk_18" 839 | ] 840 | }, 841 | { 842 | "cell_type": "code", 843 | "execution_count": null, 844 | "metadata": {}, 845 | "outputs": [], 846 | "source": [ 847 | "#This is Innsbruck! We selected three dimensions here: lat and lon of ibk and the year 2018\n", 848 | "ibk_18.plot()\n", 849 | "plt.title('mean t2m Innsbruck 2018(47.25°, 11.25°)');\n", 850 | "plt.xlim(['2018-01', '2018-12']);" 851 | ] 852 | }, 853 | { 854 | "cell_type": "markdown", 855 | "metadata": {}, 856 | "source": [ 857 | "**Notice**, that selecting data in xarray is way easier than in numpy! When we index with numpy, we need stuff like `a[1:3:2]`,...! Now we can simply tell the array that we want all the values of a certain month or a certain latitude. If we were in a numpy environment, we would have to know, which axis is associated with e.g. the time and then tell numpy somthing similar to: `a[:,1]`. This is one of the things that make xarray so very convenient, without us even noticing it!\n" 858 | ] 859 | }, 860 | { 861 | "cell_type": "markdown", 862 | "metadata": {}, 863 | "source": [ 864 | "### Selection based on a condition" 865 | ] 866 | }, 867 | { 868 | "cell_type": "markdown", 869 | "metadata": {}, 870 | "source": [ 871 | "What if we are interested into air temperature on land only, and want to remove the oceans from our analyses? For this we are going to have to \"mask out\" the oceans grid points. First, we will need to open the \"invariant\" file: " 872 | ] 873 | }, 874 | { 875 | "cell_type": "code", 876 | "execution_count": null, 877 | "metadata": {}, 878 | "outputs": [], 879 | "source": [ 880 | "nc_inv = xr.open_dataset('./data/ERA5_75_invariant.nc')\n", 881 | "nc_inv" 882 | ] 883 | }, 884 | { 885 | "cell_type": "code", 886 | "execution_count": null, 887 | "metadata": {}, 888 | "outputs": [], 889 | "source": [ 890 | "nc_inv.lsm" 891 | ] 892 | }, 893 | { 894 | "cell_type": "markdown", 895 | "metadata": {}, 896 | "source": [ 897 | "In this file, all the ocean gridpoints have the value `0` and all the land gridpoints the value `1`. We will use this information, to \"mask out\" all the ocean values!\n" 898 | ] 899 | }, 900 | { 901 | "cell_type": "code", 902 | "execution_count": null, 903 | "metadata": {}, 904 | "outputs": [], 905 | "source": [ 906 | "masked_t2m = ds.t2m.mean(dim = 'time').where(nc_inv.lsm == 1)" 907 | ] 908 | }, 909 | { 910 | "cell_type": "markdown", 911 | "metadata": {}, 912 | "source": [ 913 | "If we want to select based on a condition, we take the `.where(data == value)` syntax! Let's see what we have created:" 914 | ] 915 | }, 916 | { 917 | "cell_type": "code", 918 | "execution_count": null, 919 | "metadata": {}, 920 | "outputs": [], 921 | "source": [ 922 | "masked_t2m = masked_t2m - 273.15 # convert into celcius\n", 923 | "ax = plt.axes(projection=ccrs.Robinson())\n", 924 | "masked_t2m.plot(ax=ax, transform=ccrs.PlateCarree()) \n", 925 | "ax.add_feature(cartopy.feature.BORDERS); \n", 926 | "ax.coastlines();" 927 | ] 928 | }, 929 | { 930 | "cell_type": "markdown", 931 | "metadata": {}, 932 | "source": [ 933 | "### Resampling Data" 934 | ] 935 | }, 936 | { 937 | "cell_type": "markdown", 938 | "metadata": {}, 939 | "source": [ 940 | "Another important tool is resampling data. Resampling is the operation of changing the sampling of the data, i.e. the frequency at which it is sampled. For example you could change the sampling frequency from monthly to yearly. One of the most meaningful way to resample is to do an average, for example over the year:" 941 | ] 942 | }, 943 | { 944 | "cell_type": "code", 945 | "execution_count": null, 946 | "metadata": { 947 | "scrolled": true 948 | }, 949 | "outputs": [], 950 | "source": [ 951 | "res = ds.t2m.resample(time = 'Y').mean()\n", 952 | "res" 953 | ] 954 | }, 955 | { 956 | "cell_type": "markdown", 957 | "metadata": {}, 958 | "source": [ 959 | "What we get from the resampling operation above is a new dataset, that hast only 36 elements in the time dimension: the 36 years from 1979 to 2014. We averaged the data for each year: `data.resample(time = 'y')` means change the frequency to yearly and `.mean()` says take the mean to get the new values! We can now either change the frequency or the operation by which we resample:" 960 | ] 961 | }, 962 | { 963 | "cell_type": "code", 964 | "execution_count": null, 965 | "metadata": { 966 | "scrolled": false 967 | }, 968 | "outputs": [], 969 | "source": [ 970 | "ds.t2m.resample(time = '5Y').mean() #new frequency: every 5 years" 971 | ] 972 | }, 973 | { 974 | "cell_type": "markdown", 975 | "metadata": {}, 976 | "source": [ 977 | "Ways to change the sampling frequency: either add a number in front of the `Y` (as above), or change the `Y` e.g. to `M` for monthly sampling or `D` for daily sampling. " 978 | ] 979 | }, 980 | { 981 | "cell_type": "code", 982 | "execution_count": null, 983 | "metadata": {}, 984 | "outputs": [], 985 | "source": [ 986 | "ds.t2m.resample(time = 'Y').std() # `.std()` gives us the standard deviation instead of the mean!" 987 | ] 988 | }, 989 | { 990 | "cell_type": "markdown", 991 | "metadata": {}, 992 | "source": [ 993 | "Other possible operations for resampling would be: `.max()` bzw. `.min()` if you want all the maximas/minimas or `.sum()` if you want to sum up all the single values. This would make sense for precipitation in mm. " 994 | ] 995 | }, 996 | { 997 | "cell_type": "markdown", 998 | "metadata": {}, 999 | "source": [ 1000 | "### GroupBy: Compute monthly/seasonal Climatographies" 1001 | ] 1002 | }, 1003 | { 1004 | "cell_type": "markdown", 1005 | "metadata": {}, 1006 | "source": [ 1007 | "Another way to look at time series data is to average them according to the time of year to study the annual cycle. This is done with the `.groupby()` method:" 1008 | ] 1009 | }, 1010 | { 1011 | "cell_type": "code", 1012 | "execution_count": null, 1013 | "metadata": {}, 1014 | "outputs": [], 1015 | "source": [ 1016 | "grouped = ds.t2m.groupby('time.month')\n", 1017 | "grouped" 1018 | ] 1019 | }, 1020 | { 1021 | "cell_type": "markdown", 1022 | "metadata": {}, 1023 | "source": [ 1024 | "We get a DataArray, grouped into the different months. Now we can apply an average and what we will get is the average t2m of each month over the period 1979-2014!" 1025 | ] 1026 | }, 1027 | { 1028 | "cell_type": "code", 1029 | "execution_count": null, 1030 | "metadata": { 1031 | "scrolled": true 1032 | }, 1033 | "outputs": [], 1034 | "source": [ 1035 | "grouped.mean()" 1036 | ] 1037 | }, 1038 | { 1039 | "cell_type": "markdown", 1040 | "metadata": {}, 1041 | "source": [ 1042 | "Note that with groupby, you can also compute seasonal averages:" 1043 | ] 1044 | }, 1045 | { 1046 | "cell_type": "code", 1047 | "execution_count": null, 1048 | "metadata": {}, 1049 | "outputs": [], 1050 | "source": [ 1051 | "ds.t2m.groupby('time.season').mean(dim='time')" 1052 | ] 1053 | }, 1054 | { 1055 | "cell_type": "markdown", 1056 | "metadata": {}, 1057 | "source": [ 1058 | "We learned a lot about working with datasets and doing analysis. Now, let's finally move on to plotting!" 1059 | ] 1060 | } 1061 | ], 1062 | "metadata": { 1063 | "kernelspec": { 1064 | "display_name": "Python 3", 1065 | "language": "python", 1066 | "name": "python3" 1067 | }, 1068 | "language_info": { 1069 | "codemirror_mode": { 1070 | "name": "ipython", 1071 | "version": 3 1072 | }, 1073 | "file_extension": ".py", 1074 | "mimetype": "text/x-python", 1075 | "name": "python", 1076 | "nbconvert_exporter": "python", 1077 | "pygments_lexer": "ipython3", 1078 | "version": "3.6.6" 1079 | } 1080 | }, 1081 | "nbformat": 4, 1082 | "nbformat_minor": 2 1083 | } 1084 | -------------------------------------------------------------------------------- /05_PlotWindData.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# 05 How to Plot Winddata" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "This chapter is all about plotting wind data. First we will deal with 10m winds in x- and y-direction and second we will have a look at 4D-data: wind at different heights!\n", 15 | "\n", 16 | "The very first, we import our modules and have some presettings again:" 17 | ] 18 | }, 19 | { 20 | "cell_type": "code", 21 | "execution_count": null, 22 | "metadata": {}, 23 | "outputs": [], 24 | "source": [ 25 | "# Display the plots in the notebook:\n", 26 | "%matplotlib inline" 27 | ] 28 | }, 29 | { 30 | "cell_type": "code", 31 | "execution_count": null, 32 | "metadata": {}, 33 | "outputs": [], 34 | "source": [ 35 | "# Import the tools we are going to need today:\n", 36 | "import matplotlib.pyplot as plt # plotting library\n", 37 | "import numpy as np # numerical library\n", 38 | "import xarray as xr # netCDF library\n", 39 | "import cartopy # Map projections libary\n", 40 | "import cartopy.crs as ccrs # Projections list\n", 41 | "# Some defaults:\n", 42 | "plt.rcParams['figure.figsize'] = (12, 5) # Default plot size\n", 43 | "np.set_printoptions(threshold=20) # avoid to print very large arrays on screen\n", 44 | "# The commands below are to ignore certain warnings.\n", 45 | "import warnings\n", 46 | "warnings.filterwarnings('ignore')" 47 | ] 48 | }, 49 | { 50 | "cell_type": "markdown", 51 | "metadata": {}, 52 | "source": [ 53 | "## 05_01 3D-Data: Surface Winds" 54 | ] 55 | }, 56 | { 57 | "cell_type": "markdown", 58 | "metadata": {}, 59 | "source": [ 60 | "Get the data:" 61 | ] 62 | }, 63 | { 64 | "cell_type": "code", 65 | "execution_count": null, 66 | "metadata": { 67 | "scrolled": false 68 | }, 69 | "outputs": [], 70 | "source": [ 71 | "ds = xr.open_dataset('./data/ERA5_LowRes_Monthly_uvslp_Fabien.nc')\n", 72 | "print(ds)" 73 | ] 74 | }, 75 | { 76 | "cell_type": "markdown", 77 | "metadata": {}, 78 | "source": [ 79 | "Let's select a year out of our data and print our variable:" 80 | ] 81 | }, 82 | { 83 | "cell_type": "code", 84 | "execution_count": null, 85 | "metadata": { 86 | "scrolled": true 87 | }, 88 | "outputs": [], 89 | "source": [ 90 | "u = ds.u10.sel(time = '2018')\n", 91 | "u" 92 | ] 93 | }, 94 | { 95 | "cell_type": "markdown", 96 | "metadata": {}, 97 | "source": [ 98 | "On the second line, you should see something similar to:\n", 99 | "\n", 100 | " [1156800 values with dtype=float32]\n", 101 | " \n", 102 | "As we are going to see, this is a \"trick\" that xarray uses to hide an important information from us: xarray didn't actually read the data! However, as we have seen, xarray is very capable to plot this data or do mathematical operations on them. \n", 103 | "\n", 104 | "So why didn't xarray read the data yet? \n", 105 | "\n", 106 | "Because it *knows* that we don't need it right away. This feature is called [lazy evaluation](https://en.wikipedia.org/wiki/Lazy_evaluation) in computer sciences. It is extremely useful, as it spares time and memory by loading the data only when really needed." 107 | ] 108 | }, 109 | { 110 | "cell_type": "markdown", 111 | "metadata": {}, 112 | "source": [ 113 | "In some cases this feature can be undesirable. For example: when reading the data is quite slow. In theses exercises, we are working with data on remote directories, and the data we are using is highly compressed (the size that the data takes on disk is *very* small in comparison to the space it takes in memory). Reading compressed data takes time, and the operations we are going to make below make decompression even slowier.\n", 114 | "\n", 115 | "This is why I'll sometimes recommend to do the following:" 116 | ] 117 | }, 118 | { 119 | "cell_type": "code", 120 | "execution_count": null, 121 | "metadata": { 122 | "scrolled": true 123 | }, 124 | "outputs": [], 125 | "source": [ 126 | "u = ds.u10.sel(time = '2019').load()\n", 127 | "u" 128 | ] 129 | }, 130 | { 131 | "cell_type": "markdown", 132 | "metadata": {}, 133 | "source": [ 134 | "Now we can actually see the data in the form of an array!\n", 135 | "\n", 136 | "We are now calculating the **averaged u and v windfields at the 10 m level for the month of January**:" 137 | ] 138 | }, 139 | { 140 | "cell_type": "code", 141 | "execution_count": null, 142 | "metadata": {}, 143 | "outputs": [], 144 | "source": [ 145 | "u = ds.u10.groupby('time.month').mean(dim = 'time').sel(month=1).load()\n", 146 | "v = ds.v10.groupby('time.month').mean(dim = 'time').sel(month=1).load()" 147 | ] 148 | }, 149 | { 150 | "cell_type": "markdown", 151 | "metadata": {}, 152 | "source": [ 153 | "### 05_01_01 Quiver Plots" 154 | ] 155 | }, 156 | { 157 | "cell_type": "markdown", 158 | "metadata": {}, 159 | "source": [ 160 | "One traditional way to plot the wind data is as vector arrows. We can use a function called [quiver(X, Y, U, V)](http://matplotlib.org/api/pyplot_api.html?highlight=quiver#matplotlib.pyplot.quiver) for that. \n", 161 | "It takes the following arguments: U and V are the arrow data, X and Y set the location of the arrows. You can also add a C argument, that will set the color of the arrows. \n", 162 | "\n", 163 | "For example:" 164 | ] 165 | }, 166 | { 167 | "cell_type": "code", 168 | "execution_count": null, 169 | "metadata": { 170 | "scrolled": true 171 | }, 172 | "outputs": [], 173 | "source": [ 174 | "ax = plt.axes(projection=ccrs.PlateCarree()) # note that I changed the map projection\n", 175 | "qv = ax.quiver(u.longitude, u.latitude, u, v, transform=ccrs.PlateCarree())\n", 176 | "ax.coastlines(color='grey');" 177 | ] 178 | }, 179 | { 180 | "cell_type": "markdown", 181 | "metadata": {}, 182 | "source": [ 183 | "Well, this does not really look useful. The problem is, that there are too many grid points for the plot to be able to illustrate the data as single arrows. So the solution is, to not use every grid point but e.g. only every 10th!\n", 184 | "\n", 185 | "Let's try it:" 186 | ] 187 | }, 188 | { 189 | "cell_type": "code", 190 | "execution_count": null, 191 | "metadata": { 192 | "scrolled": false 193 | }, 194 | "outputs": [], 195 | "source": [ 196 | "ax = plt.axes(projection=ccrs.PlateCarree()) \n", 197 | "pu, pv = u[::9,::9], v[::9,::9] # we will discuss what this line does in the cells below\n", 198 | "qv = ax.quiver(pu.longitude, pu.latitude, pu, pv, transform=ccrs.PlateCarree())\n", 199 | "ax.coastlines(color='grey');\n", 200 | "pu" 201 | ] 202 | }, 203 | { 204 | "cell_type": "markdown", 205 | "metadata": {}, 206 | "source": [ 207 | "Note the new syntax in the line ``pu, pv = u[::10,::10], v[::10,::10]``. Let's explain it in detail. the ``::9`` is the Numpy way to index each 9th element of an array. It is called [slicing](http://docs.scipy.org/doc/numpy-1.10.0/reference/arrays.indexing.html#basic-slicing-and-indexing) in the numpy jargon. We already learned how to slice with a slice object!\n", 208 | "\n", 209 | "The `start:stop:step` syntax is actually creating such a python slice object too. So it is just a more convenient way, to do slicing in python.\n", 210 | "Above, `start` and `stop` were not defined. That means: take the whole array, from the start until the end!\n", 211 | "\n", 212 | "\n", 213 | "Let's try this new form of slicing on a simpler array:" 214 | ] 215 | }, 216 | { 217 | "cell_type": "code", 218 | "execution_count": null, 219 | "metadata": { 220 | "scrolled": false 221 | }, 222 | "outputs": [], 223 | "source": [ 224 | "a = np.arange(11)\n", 225 | "print(a)" 226 | ] 227 | }, 228 | { 229 | "cell_type": "markdown", 230 | "metadata": {}, 231 | "source": [ 232 | "Underneath there will be some different slicings, each explained right next to the command." 233 | ] 234 | }, 235 | { 236 | "cell_type": "code", 237 | "execution_count": null, 238 | "metadata": {}, 239 | "outputs": [], 240 | "source": [ 241 | "a[:] #takes all elements of the array" 242 | ] 243 | }, 244 | { 245 | "cell_type": "code", 246 | "execution_count": null, 247 | "metadata": {}, 248 | "outputs": [], 249 | "source": [ 250 | "a[2:] #takes the elements from the second one until the end. Neither the stop nor the step value are defined, \n", 251 | " # so python will assume you want the data until the end of the array in steps of one (default)" 252 | ] 253 | }, 254 | { 255 | "cell_type": "code", 256 | "execution_count": null, 257 | "metadata": {}, 258 | "outputs": [], 259 | "source": [ 260 | "a[:-1] #takes all the elements from the start until the second last one: -1 means the end of the array -1" 261 | ] 262 | }, 263 | { 264 | "cell_type": "code", 265 | "execution_count": null, 266 | "metadata": {}, 267 | "outputs": [], 268 | "source": [ 269 | "a[2:5] #start from the third element (remember: in python the firs element is number 0!) and take every element until \n", 270 | " #the fourth (remember: in python the stop index is excluded!)" 271 | ] 272 | }, 273 | { 274 | "cell_type": "code", 275 | "execution_count": null, 276 | "metadata": { 277 | "scrolled": true 278 | }, 279 | "outputs": [], 280 | "source": [ 281 | "a[::-1] #take all elements of the array, the step value is minus one which means: step trough them backwards!\n", 282 | " # --> we get the reversed array" 283 | ] 284 | }, 285 | { 286 | "cell_type": "code", 287 | "execution_count": null, 288 | "metadata": {}, 289 | "outputs": [], 290 | "source": [ 291 | "a[::-2] #same as above but only take every second element. --> reversed array with every second element" 292 | ] 293 | }, 294 | { 295 | "cell_type": "markdown", 296 | "metadata": {}, 297 | "source": [ 298 | "OK, so that was slincing in one dimensions. Slicing also works in N-dimensions. For example:" 299 | ] 300 | }, 301 | { 302 | "cell_type": "code", 303 | "execution_count": null, 304 | "metadata": { 305 | "scrolled": true 306 | }, 307 | "outputs": [], 308 | "source": [ 309 | "a = np.arange(4*5).reshape((4, 5)) #don't worry about how we created this array, it does not matter for us now\n", 310 | "print(a)" 311 | ] 312 | }, 313 | { 314 | "cell_type": "code", 315 | "execution_count": null, 316 | "metadata": {}, 317 | "outputs": [], 318 | "source": [ 319 | "a[:,::-1] # the first slice corresponds to the columns, the second one to the rows!" 320 | ] 321 | }, 322 | { 323 | "cell_type": "code", 324 | "execution_count": null, 325 | "metadata": {}, 326 | "outputs": [], 327 | "source": [ 328 | "a[::2,::2]" 329 | ] 330 | }, 331 | { 332 | "cell_type": "markdown", 333 | "metadata": {}, 334 | "source": [ 335 | "OK, that was slicing in two dimensions, so we now better understand what ``u[::9,::9]`` means. But what was about that comma in between? This is simply what people call \"syntaxic sugar\", a nice and easy way to write one line in two: " 336 | ] 337 | }, 338 | { 339 | "cell_type": "code", 340 | "execution_count": null, 341 | "metadata": {}, 342 | "outputs": [], 343 | "source": [ 344 | "a, b = 12, 'Hello'\n", 345 | "print(a)\n", 346 | "print(b)" 347 | ] 348 | }, 349 | { 350 | "cell_type": "markdown", 351 | "metadata": {}, 352 | "source": [ 353 | "This kind of shortcuts should be used only if:\n", 354 | "- the lines of code are easy\n", 355 | "- the two variables are somehow related (for example u and v winds)\n", 356 | "\n", 357 | "We can now try some different slicing for our quiver plot:" 358 | ] 359 | }, 360 | { 361 | "cell_type": "code", 362 | "execution_count": null, 363 | "metadata": {}, 364 | "outputs": [], 365 | "source": [ 366 | "ax = plt.axes(projection=ccrs.PlateCarree()) \n", 367 | "pu, pv = u[::5,::5], v[::5,::5]\n", 368 | "qv = ax.quiver(pu.longitude, pu.latitude, pu, pv, transform=ccrs.PlateCarree())\n", 369 | "ax.coastlines(color='grey');" 370 | ] 371 | }, 372 | { 373 | "cell_type": "code", 374 | "execution_count": null, 375 | "metadata": { 376 | "scrolled": true 377 | }, 378 | "outputs": [], 379 | "source": [ 380 | "ax = plt.axes(projection=ccrs.PlateCarree()) \n", 381 | "pu, pv = u[::15,::15], v[::15,::15]\n", 382 | "qv = ax.quiver(pu.longitude, pu.latitude, pu, pv, transform=ccrs.PlateCarree())\n", 383 | "ax.coastlines(color='grey');" 384 | ] 385 | }, 386 | { 387 | "cell_type": "markdown", 388 | "metadata": {}, 389 | "source": [ 390 | "Or change the color of our arrows:" 391 | ] 392 | }, 393 | { 394 | "cell_type": "code", 395 | "execution_count": null, 396 | "metadata": { 397 | "scrolled": true 398 | }, 399 | "outputs": [], 400 | "source": [ 401 | "ax = plt.axes(projection=ccrs.PlateCarree()) \n", 402 | "pu, pv = u[::10,::10], v[::10,::10]\n", 403 | "qv = ax.quiver(pu.longitude, pu.latitude, pu, pv, color = 'r', transform=ccrs.PlateCarree())\n", 404 | "ax.coastlines(color='grey');" 405 | ] 406 | }, 407 | { 408 | "cell_type": "markdown", 409 | "metadata": {}, 410 | "source": [ 411 | "It seems reasonable, to somehow better visualize the windspeed than just by the length of the arrows. We can for example plot the wind speed as a shaded color plot (as we did for temperature and precipitation) and plot the wind arrows on top of it.! \n", 412 | "\n", 413 | "Therefore, we must first determine the windspeed:" 414 | ] 415 | }, 416 | { 417 | "cell_type": "code", 418 | "execution_count": null, 419 | "metadata": {}, 420 | "outputs": [], 421 | "source": [ 422 | "ws = (u**2 + v**2)**0.5" 423 | ] 424 | }, 425 | { 426 | "cell_type": "markdown", 427 | "metadata": {}, 428 | "source": [ 429 | "Now plot the windspeed first and on top of it the arrows:" 430 | ] 431 | }, 432 | { 433 | "cell_type": "code", 434 | "execution_count": null, 435 | "metadata": { 436 | "scrolled": true 437 | }, 438 | "outputs": [], 439 | "source": [ 440 | "ax = plt.axes(projection=ccrs.PlateCarree()) \n", 441 | "ws.plot.contourf(ax=ax, transform=ccrs.PlateCarree(), cmap='cool', levels=np.arange(0, 15), cbar_kwargs={'label':'Windspeed [m/s]'});\n", 442 | "pu, pv = u[::10,::10], v[::10,::10]\n", 443 | "qv = ax.quiver(pu.longitude, pu.latitude, pu, pv, transform=ccrs.PlateCarree())\n", 444 | "ax.coastlines();\n", 445 | "ax.set_title('Average Wind and Windspeed January');" 446 | ] 447 | }, 448 | { 449 | "cell_type": "markdown", 450 | "metadata": {}, 451 | "source": [ 452 | "### 05_01_02 Streamline Plots" 453 | ] 454 | }, 455 | { 456 | "cell_type": "markdown", 457 | "metadata": {}, 458 | "source": [ 459 | "Another way to plot wind data are the so called [streamlines](http://matplotlib.org/api/pyplot_api.html?highlight=streamplot#matplotlib.pyplot.streamplot):" 460 | ] 461 | }, 462 | { 463 | "cell_type": "code", 464 | "execution_count": null, 465 | "metadata": {}, 466 | "outputs": [], 467 | "source": [ 468 | "fig = plt.figure('figsize', [12,5])\n", 469 | "ax = plt.axes(projection=ccrs.PlateCarree())\n", 470 | "ax.streamplot(u.longitude, u.latitude, u.values, v.values, transform=ccrs.PlateCarree(), density=4)\n", 471 | "ax.coastlines(color='grey');" 472 | ] 473 | }, 474 | { 475 | "cell_type": "markdown", 476 | "metadata": {}, 477 | "source": [ 478 | "The `u.values` syntax extracts the actual u values as a Numpy NdArray from the XArray DataArray! That is what we need as an argument for the streamplot function!" 479 | ] 480 | }, 481 | { 482 | "cell_type": "code", 483 | "execution_count": null, 484 | "metadata": {}, 485 | "outputs": [], 486 | "source": [ 487 | "u.values" 488 | ] 489 | }, 490 | { 491 | "cell_type": "code", 492 | "execution_count": null, 493 | "metadata": {}, 494 | "outputs": [], 495 | "source": [ 496 | "type(u.values)" 497 | ] 498 | }, 499 | { 500 | "cell_type": "markdown", 501 | "metadata": {}, 502 | "source": [ 503 | "**Note**: if the above plot was too slow for your taste, you can slice your data like we did for the quiver plot, this will produce the plot faster. Anyway, this does not have any impact on the denseness of the streamlines!\n", 504 | "\n", 505 | "For example:" 506 | ] 507 | }, 508 | { 509 | "cell_type": "code", 510 | "execution_count": null, 511 | "metadata": {}, 512 | "outputs": [], 513 | "source": [ 514 | "pu, pv = u[::10,::10], v[::10,::10]\n", 515 | "ax = plt.axes(projection=ccrs.PlateCarree())\n", 516 | "ax.streamplot(pu.longitude, pu.latitude, pu.values, pv.values, transform=ccrs.PlateCarree(), density=4)\n", 517 | "ax.coastlines(color='grey');" 518 | ] 519 | }, 520 | { 521 | "cell_type": "markdown", 522 | "metadata": {}, 523 | "source": [ 524 | "By changing the `density` argument, you can control the closeness of streamlines. The deault value is density = 1.\n", 525 | "Let's try it:" 526 | ] 527 | }, 528 | { 529 | "cell_type": "code", 530 | "execution_count": null, 531 | "metadata": { 532 | "scrolled": true 533 | }, 534 | "outputs": [], 535 | "source": [ 536 | "ax = plt.axes(projection=ccrs.PlateCarree())\n", 537 | "ax.streamplot(u.longitude, u.latitude, u.values, v.values, transform=ccrs.PlateCarree(), density=2)\n", 538 | "ax.coastlines(color='grey');" 539 | ] 540 | }, 541 | { 542 | "cell_type": "markdown", 543 | "metadata": {}, 544 | "source": [ 545 | "You can also vary the linewidth of the streamlines by specifying the argument `linewidth`. You can either set it to a numeric or a 2D-array. " 546 | ] 547 | }, 548 | { 549 | "cell_type": "code", 550 | "execution_count": null, 551 | "metadata": {}, 552 | "outputs": [], 553 | "source": [ 554 | "ax = plt.axes(projection=ccrs.PlateCarree())\n", 555 | "strm = ax.streamplot(u.longitude, u.latitude, u.values, v.values, transform=ccrs.PlateCarree(),\n", 556 | " density=2, linewidth = 3)\n", 557 | "ax.coastlines(color='grey'); ax.gridlines(draw_labels=True);" 558 | ] 559 | }, 560 | { 561 | "cell_type": "code", 562 | "execution_count": null, 563 | "metadata": { 564 | "scrolled": true 565 | }, 566 | "outputs": [], 567 | "source": [ 568 | "ax = plt.axes(projection=ccrs.PlateCarree())\n", 569 | "strm = ax.streamplot(u.longitude, u.latitude, u.values, v.values, transform=ccrs.PlateCarree(),\n", 570 | " density=5, linewidth = 0.5)\n", 571 | "ax.coastlines(color='grey'); ax.gridlines(draw_labels=True);" 572 | ] 573 | }, 574 | { 575 | "cell_type": "markdown", 576 | "metadata": {}, 577 | "source": [ 578 | "A problem with streamlines is that they provide no information about the strength of the flow. Again, it is possible to display this information with colors. Just define the `color` argument as the windspeed array `ws` we already created further up! In order to make this work, we again have to use `ws.values`, since color wants a Numpy Array and not a DataArray as input." 579 | ] 580 | }, 581 | { 582 | "cell_type": "code", 583 | "execution_count": null, 584 | "metadata": {}, 585 | "outputs": [], 586 | "source": [ 587 | "ax = plt.axes(projection=ccrs.PlateCarree())\n", 588 | "strm = ax.streamplot(u.longitude, u.latitude, u.values, v.values, transform=ccrs.PlateCarree(),\n", 589 | " density=4, color=ws.values, cmap='cool')\n", 590 | "bar = plt.colorbar(strm.lines) # add a colorbar defined by the lines of the streamplot (strm.lines)\n", 591 | "bar.set_label('Windspeed [m/s]') #give a label to the colorbar\n", 592 | "ax.coastlines(color='grey'); ax.gridlines(draw_labels=True);" 593 | ] 594 | }, 595 | { 596 | "cell_type": "markdown", 597 | "metadata": {}, 598 | "source": [ 599 | "### 05_01_03 1D-Plots" 600 | ] 601 | }, 602 | { 603 | "cell_type": "markdown", 604 | "metadata": {}, 605 | "source": [ 606 | "We can also display wind data in a 1D-Plot rather than in a 2D one as the above examples. A possible and useful form of representation would be the zonal average of the meridional and the zonal 10m winds:" 607 | ] 608 | }, 609 | { 610 | "cell_type": "code", 611 | "execution_count": null, 612 | "metadata": { 613 | "scrolled": false 614 | }, 615 | "outputs": [], 616 | "source": [ 617 | "ds.u10.mean(dim=['time', 'longitude']).plot();\n", 618 | "plt.axhline(0, color='k');\n", 619 | "plt.xlim([-90, 90]);\n", 620 | "plt.title('Zonal Wind at 10m');\n", 621 | "plt.ylabel('m s$^{-1}$');\n", 622 | "plt.xlabel('° North');\n", 623 | "plt.figure();\n", 624 | "ds.v10.mean(dim=['time', 'longitude']).plot();\n", 625 | "plt.axhline(0, color='k');\n", 626 | "plt.xlim([-90, 90]);\n", 627 | "plt.title('Meridional Wind at 10m');\n", 628 | "plt.ylabel('m s$^-1$');\n", 629 | "plt.xlabel('° North');" 630 | ] 631 | }, 632 | { 633 | "cell_type": "markdown", 634 | "metadata": {}, 635 | "source": [ 636 | "We encounter a new syntax in the above commands: `plt.ylabel('m s$^{-1}$')`. What do those dollar signs mean? \n", 637 | "\n", 638 | "In matplotlib, one can write [**mathematical expressions**](https://matplotlib.org/3.1.1/tutorials/text/mathtext.html) by using dollar signs as a wrapper of the expressions. Everything that is written in between of two $-signs is recognised as a mathematical expression by matplotlib.\n", 639 | "\n", 640 | "* To make subscripts and superscripts, you can use the `_` and `^` symbols. In the above example, we used `^` to create a superscript of -1. \n", 641 | "* Radicals can be produced with the `\\sqrt[]{}` command. For example: `\\sqrt{2}`. Any base can optionally be provided inside the square brackets. \n", 642 | "* Look at the webpage above, to get to know more. E.g. how to write mathematical symbols or greek letters." 643 | ] 644 | }, 645 | { 646 | "cell_type": "markdown", 647 | "metadata": {}, 648 | "source": [ 649 | "As another example, we can plot the monthly means of the windspeed for the year 2018 at the location of Innsbruck:" 650 | ] 651 | }, 652 | { 653 | "cell_type": "code", 654 | "execution_count": null, 655 | "metadata": { 656 | "scrolled": true 657 | }, 658 | "outputs": [], 659 | "source": [ 660 | "u10_ibk_18 = ds.u10.sel(latitude = 47.25, longitude = 11.25, time = '2018') #select Innsbruck and year 2018\n", 661 | "v10_ibk_18 = ds.v10.sel(latitude = 47.25, longitude = 11.25, time = '2018') #select Innsbruck and year 2018\n", 662 | "ws_ibk_18 = ws = (u10_ibk_18**2 + v10_ibk_18**2)**0.5 #compute windspeed for Innsbruck 2018\n", 663 | "\n", 664 | "ax = ws_ibk_18.plot();\n", 665 | "plt.title('Windspeed 2018 @ 47.25°North, 11.25°East ');\n", 666 | "plt.ylabel('m s$^{-1}$');\n", 667 | "plt.xlabel('');\n", 668 | "plt.xlim(['2018-01','2018-12']);" 669 | ] 670 | }, 671 | { 672 | "cell_type": "markdown", 673 | "metadata": {}, 674 | "source": [ 675 | "## 05_02 4D-Data: Wind at different pressure levels" 676 | ] 677 | }, 678 | { 679 | "cell_type": "markdown", 680 | "metadata": {}, 681 | "source": [ 682 | "We will now deal with a new kind of dataset: 4D-Data. Let's open such a dataset and see what it looks like:" 683 | ] 684 | }, 685 | { 686 | "cell_type": "code", 687 | "execution_count": null, 688 | "metadata": { 689 | "scrolled": true 690 | }, 691 | "outputs": [], 692 | "source": [ 693 | "ds = xr.open_dataset('./data/ERA5_LowRes_MonthlyAvg_4D_zuvw_Fabien.nc')\n", 694 | "ds" 695 | ] 696 | }, 697 | { 698 | "cell_type": "markdown", 699 | "metadata": {}, 700 | "source": [ 701 | "We have a new coordinate axis: level! Those are the different pressure levels, at which wind data (u,v,w) and height data (z) is available." 702 | ] 703 | }, 704 | { 705 | "cell_type": "code", 706 | "execution_count": null, 707 | "metadata": {}, 708 | "outputs": [], 709 | "source": [ 710 | "ds.level" 711 | ] 712 | }, 713 | { 714 | "cell_type": "markdown", 715 | "metadata": {}, 716 | "source": [ 717 | "The data is available at the pressure levels 50, 100, 200, 300, 400, 500, 700, 850 and 1000 millibars. Millibars are equivalent to hectopascal. \n", 718 | "\n", 719 | "We can use this new data, to plot the height- (= pressure-) dependency of the horizontal as well as the vertical windfields! Before we do this, let's have a quick look at our variables: u, v, w, and z." 720 | ] 721 | }, 722 | { 723 | "cell_type": "code", 724 | "execution_count": null, 725 | "metadata": { 726 | "scrolled": true 727 | }, 728 | "outputs": [], 729 | "source": [ 730 | "ds.u" 731 | ] 732 | }, 733 | { 734 | "cell_type": "code", 735 | "execution_count": null, 736 | "metadata": { 737 | "scrolled": false 738 | }, 739 | "outputs": [], 740 | "source": [ 741 | "ds.z # this is the geopotential! get the geopotential height by dividing by g" 742 | ] 743 | }, 744 | { 745 | "cell_type": "markdown", 746 | "metadata": {}, 747 | "source": [ 748 | "We notice furthermore, that the time coordinate changed to a **month coordinate**! The data is already grouped by months and averaged." 749 | ] 750 | }, 751 | { 752 | "cell_type": "markdown", 753 | "metadata": {}, 754 | "source": [ 755 | "### 05_02_01 Vertical Veloctiy" 756 | ] 757 | }, 758 | { 759 | "cell_type": "code", 760 | "execution_count": null, 761 | "metadata": { 762 | "scrolled": true 763 | }, 764 | "outputs": [], 765 | "source": [ 766 | "ds.w" 767 | ] 768 | }, 769 | { 770 | "cell_type": "markdown", 771 | "metadata": {}, 772 | "source": [ 773 | "Let's select southamerica @500 hPa, average over time and plot the vertical veloctiy w:" 774 | ] 775 | }, 776 | { 777 | "cell_type": "code", 778 | "execution_count": null, 779 | "metadata": {}, 780 | "outputs": [], 781 | "source": [ 782 | "ax = plt.axes(projection=ccrs.PlateCarree())\n", 783 | "w = ds.w.mean(dim='month').sel(level=500, longitude = slice(-80,-20), latitude = slice(20,-60))\n", 784 | "w.plot(transform = ccrs.PlateCarree() )\n", 785 | "ax.set_title('500 hPa Mean Vertical Winds')\n", 786 | "ax.coastlines(color='grey'); ax.gridlines();" 787 | ] 788 | }, 789 | { 790 | "cell_type": "markdown", 791 | "metadata": {}, 792 | "source": [ 793 | "Sinking air at the upstream side of the andes and rising air at the downstream side? This does not seem very reasonable. We need to change the sign of w!\n" 794 | ] 795 | }, 796 | { 797 | "cell_type": "code", 798 | "execution_count": null, 799 | "metadata": {}, 800 | "outputs": [], 801 | "source": [ 802 | "ax = plt.axes(projection=ccrs.PlateCarree())\n", 803 | "w = ds.w.mean(dim='month').sel(level=500, longitude = slice(-80,-20), latitude = slice(20,-60))*-1\n", 804 | "w.plot(transform = ccrs.PlateCarree() )\n", 805 | "ax.set_title('500 hPa Mean Vertical Winds')\n", 806 | "ax.coastlines(color='grey'); ax.gridlines();" 807 | ] 808 | }, 809 | { 810 | "cell_type": "markdown", 811 | "metadata": {}, 812 | "source": [ 813 | "Better.\n", 814 | "\n", 815 | "We can now plot the vertical windfield in a different way. One option would be, to take a certain area, average the zonal winds over that area and plot them as a function of the month!\n", 816 | "\n", 817 | "E.g. for the African Horn:" 818 | ] 819 | }, 820 | { 821 | "cell_type": "code", 822 | "execution_count": null, 823 | "metadata": {}, 824 | "outputs": [], 825 | "source": [ 826 | "w_horn=ds.w.sel(longitude=slice(40, 52), latitude=slice(15, -5)).mean(dim= ['longitude', 'latitude'])\n", 827 | "w_horn *= -1\n", 828 | "plt.figure(figsize=(20,7))\n", 829 | "w_horn.T.plot.contourf(levels=30, cbar_kwargs={'label':'m s$^{-1}$'})\n", 830 | "plt.ylim([1000,50])\n", 831 | "plt.xlim([1,12])\n", 832 | "plt.yscale('log')\n", 833 | "plt.title('Vertical Wind distibution over the North- and South-Eastern Horn (40° to 55°E and 5°S to 15°N)');" 834 | ] 835 | }, 836 | { 837 | "cell_type": "markdown", 838 | "metadata": {}, 839 | "source": [ 840 | "Another option: average over longitude and time and plot the vertical wind as a function of latitude!" 841 | ] 842 | }, 843 | { 844 | "cell_type": "code", 845 | "execution_count": null, 846 | "metadata": { 847 | "scrolled": true 848 | }, 849 | "outputs": [], 850 | "source": [ 851 | "w_horn=ds.w.mean(dim= ['longitude', 'month']).sel(latitude = slice(30,-30))\n", 852 | "w_horn *= -1\n", 853 | "plt.figure(figsize=(20,7))\n", 854 | "w_horn.plot.contourf(levels=30, cbar_kwargs={'label':'m s$^{-1}$'})\n", 855 | "plt.ylim([1000,50])\n", 856 | "plt.ylabel('Pressure [hPa]')\n", 857 | "plt.xlabel('Latitude [°N]')\n", 858 | "plt.yscale('log')\n", 859 | "plt.title('Mean Vertical Wind Distribution from -30 to 30°N');" 860 | ] 861 | }, 862 | { 863 | "cell_type": "markdown", 864 | "metadata": {}, 865 | "source": [ 866 | "### 05_02_02 Zonal Average Plots" 867 | ] 868 | }, 869 | { 870 | "cell_type": "markdown", 871 | "metadata": {}, 872 | "source": [ 873 | "It's pretty easy, to produce a zonal plot of the vertical structure of the atmosphere:" 874 | ] 875 | }, 876 | { 877 | "cell_type": "code", 878 | "execution_count": null, 879 | "metadata": {}, 880 | "outputs": [], 881 | "source": [ 882 | "u_allz = ds.u.sel(month = 1).load() #select average of january\n", 883 | "u_allz = u_allz.mean(dim='longitude')\n", 884 | "u_allz" 885 | ] 886 | }, 887 | { 888 | "cell_type": "code", 889 | "execution_count": null, 890 | "metadata": { 891 | "scrolled": true 892 | }, 893 | "outputs": [], 894 | "source": [ 895 | "u_allz.plot();" 896 | ] 897 | }, 898 | { 899 | "cell_type": "markdown", 900 | "metadata": {}, 901 | "source": [ 902 | "We notice, that the y-axis needs to be reversed in order to go from the ground up to the higher atmosphere! Furthermore, the imshow plot does not really look nice. Let's make a contourf plot instead:" 903 | ] 904 | }, 905 | { 906 | "cell_type": "code", 907 | "execution_count": null, 908 | "metadata": { 909 | "scrolled": false 910 | }, 911 | "outputs": [], 912 | "source": [ 913 | "u_allz.plot.contourf(levels = np.arange(-45, 45, 5), cbar_kwargs={'label':'U-wind (m s$^{-1}$)'});\n", 914 | "plt.ylim([1000, 50]); #reverse the y-axis\n", 915 | "plt.xlim([-90,90]);\n", 916 | "plt.ylabel('Pressure [hPa]');\n", 917 | "plt.title('Average Zonal Winds January');" 918 | ] 919 | }, 920 | { 921 | "cell_type": "markdown", 922 | "metadata": {}, 923 | "source": [ 924 | "To represent more acurately the real altitude of the winds, a logarithmic scale might be better:" 925 | ] 926 | }, 927 | { 928 | "cell_type": "code", 929 | "execution_count": null, 930 | "metadata": {}, 931 | "outputs": [], 932 | "source": [ 933 | "u_allz.plot.contourf(levels = np.arange(-45, 45, 5), cbar_kwargs={'label':'U-wind (m s$^{-1}$)'});\n", 934 | "plt.ylim([50,1000]);\n", 935 | "plt.xlim([-90,90]);\n", 936 | "plt.ylabel('Pressure [hPa]');\n", 937 | "plt.title('Average Zonal Winds January')\n", 938 | "plt.yscale('log') #change yscale from linear to logarithmic!" 939 | ] 940 | }, 941 | { 942 | "cell_type": "markdown", 943 | "metadata": {}, 944 | "source": [ 945 | "One can also take a certain area, average the zonal winds over that area and plot them as a function of the month!\n", 946 | "\n", 947 | "We try this for the African Horn:" 948 | ] 949 | }, 950 | { 951 | "cell_type": "code", 952 | "execution_count": null, 953 | "metadata": {}, 954 | "outputs": [], 955 | "source": [ 956 | "u_horn=ds.u.sel(longitude=slice(50,70), latitude=slice(12, -3)).mean(dim= ['longitude', 'latitude'])\n", 957 | "plt.figure(figsize=(20,7))\n", 958 | "u_horn.T.plot.contourf(levels=30, cbar_kwargs={'label':'m s$^{-1}$'}, cmap = 'PuOr')\n", 959 | "plt.ylim([1000,50])\n", 960 | "plt.yscale('log')\n", 961 | "plt.title('Zonal Wind distibution over the North- and South-Eastern Horn (40° to 55°E and 5°S to 15°N)');" 962 | ] 963 | } 964 | ], 965 | "metadata": { 966 | "kernelspec": { 967 | "display_name": "Python 3", 968 | "language": "python", 969 | "name": "python3" 970 | }, 971 | "language_info": { 972 | "codemirror_mode": { 973 | "name": "ipython", 974 | "version": 3 975 | }, 976 | "file_extension": ".py", 977 | "mimetype": "text/x-python", 978 | "name": "python", 979 | "nbconvert_exporter": "python", 980 | "pygments_lexer": "ipython3", 981 | "version": "3.6.6" 982 | } 983 | }, 984 | "nbformat": 4, 985 | "nbformat_minor": 2 986 | } 987 | -------------------------------------------------------------------------------- /06_VerticallyIntegratedMoistureFlux.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# 06_Vertical Integrated Moisture Transport" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "In this chapter, we aim to diplay moisture transports over the globe. As we will see, we can plot the integrated moisture transport in a similar way as a windfield!" 15 | ] 16 | }, 17 | { 18 | "cell_type": "markdown", 19 | "metadata": {}, 20 | "source": [ 21 | "## 06_01 Background\n" 22 | ] 23 | }, 24 | { 25 | "cell_type": "markdown", 26 | "metadata": {}, 27 | "source": [ 28 | "The essence of the overall hydrological cycle is the evaporation of moisture ($E$) in one place and the precipitation in other places ($P$). In particular, evaporation exceeds precipitation over the oceans, which allows moisture to be transported by the atmosphere onto land where precipitation exceeds evapotranspiration, and the runoff flows into streams and rivers and discharges into the ocean, completing the cycle (Trenberth et al., 2011).\n" 29 | ] 30 | }, 31 | { 32 | "cell_type": "markdown", 33 | "metadata": {}, 34 | "source": [ 35 | "This cycle can be expressed in a equation. The atmospheric conservation of moisture equation when vertically integrated in flux form is" 36 | ] 37 | }, 38 | { 39 | "cell_type": "markdown", 40 | "metadata": {}, 41 | "source": [ 42 | "$$\\frac{\\partial w}{\\partial t}+ \\nabla\\cdot\\frac{1}{g}\\int_{0}^{p_s} vq dp = E - P$$ " 43 | ] 44 | }, 45 | { 46 | "cell_type": "markdown", 47 | "metadata": {}, 48 | "source": [ 49 | "Where $q$ is the specific humidity, $v$ the horizontal wind vector, $w = g^{-1}\\int_{0}^{p_s}qdp$ is the precipitable water (total column water vapor), $E$ is the surface evaporation, and $P$ is the net surface precipitation rate.\n", 50 | "\n", 51 | "The second term on the left hand side of the equation is the divergence of the vertical integrated moisture flux. \n", 52 | "Areas for which there is an excess of evaporation over precipitation ($E − P > 0$) are moisture source regions over which the atmospheric moisture fluxes diverge (divergence term >0), and other areas, where moisture fluxes converge, are moisture sink regions.\n", 53 | "\n", 54 | "The local change of the total column water vapor is therefore dependent on the local precipitation, evaporation and the moisture flux which transports moisture from neigbouring columns into the respective column." 55 | ] 56 | }, 57 | { 58 | "cell_type": "markdown", 59 | "metadata": {}, 60 | "source": [ 61 | "To illustrate, how moisture is tranported on our globe and where it origins, we can either plot the integrated moisture flux or its divergence. The divergence will show us the sink and source regions of moisture as explained above.Let's have a quick look at the integrated moisture flux itself:\n", 62 | "\n", 63 | "$\\frac{1}{g}\\int_{0}^{p_s} vq dp$\n", 64 | "\n", 65 | "A flux generally consists of the transported quantity, in this case the specific humidity $q$ in [kg/kg] and a velocity, in this case $v$. Integrated from the TOA @ 0 Pa to the surface @ $p_s$ and multplied with $g$, we get the unit $Pa \\cdot m s^{-1}\\cdot s^2 m^{-1}$, which gives $kg m^{-1}s^{-1}$ on the whole. " 66 | ] 67 | }, 68 | { 69 | "cell_type": "markdown", 70 | "metadata": {}, 71 | "source": [ 72 | "## 06_02 The Data" 73 | ] 74 | }, 75 | { 76 | "cell_type": "markdown", 77 | "metadata": {}, 78 | "source": [ 79 | "Let's load the data of vertical integrated moisture fluxes and have a closer look at it:" 80 | ] 81 | }, 82 | { 83 | "cell_type": "code", 84 | "execution_count": null, 85 | "metadata": {}, 86 | "outputs": [], 87 | "source": [ 88 | "# Display the plots in the notebook:\n", 89 | "%matplotlib inline" 90 | ] 91 | }, 92 | { 93 | "cell_type": "code", 94 | "execution_count": null, 95 | "metadata": {}, 96 | "outputs": [], 97 | "source": [ 98 | "# Import the tools we are going to need today:\n", 99 | "import matplotlib.pyplot as plt # plotting library\n", 100 | "import numpy as np # numerical library\n", 101 | "import xarray as xr # netCDF library\n", 102 | "import cartopy # Map projections libary\n", 103 | "import cartopy.crs as ccrs # Projections list\n", 104 | "# Some defaults:\n", 105 | "plt.rcParams['figure.figsize'] = (12, 5) # Default plot size\n", 106 | "np.set_printoptions(threshold=20) # avoid to print very large arrays on screen\n", 107 | "# The commands below are to ignore certain warnings.\n", 108 | "import warnings\n", 109 | "warnings.filterwarnings('ignore')" 110 | ] 111 | }, 112 | { 113 | "cell_type": "code", 114 | "execution_count": null, 115 | "metadata": {}, 116 | "outputs": [], 117 | "source": [ 118 | "ds = xr.open_dataset('./data/ERA5-LowRes-MonthlyAvg-VertIntegratedMoistureFlux.nc')" 119 | ] 120 | }, 121 | { 122 | "cell_type": "code", 123 | "execution_count": null, 124 | "metadata": {}, 125 | "outputs": [], 126 | "source": [ 127 | "ds = ds.rename({\"p71.162_0001\" : \"EMT\", \"p72.162_0001\":\"NMT\"})" 128 | ] 129 | }, 130 | { 131 | "cell_type": "code", 132 | "execution_count": null, 133 | "metadata": {}, 134 | "outputs": [], 135 | "source": [ 136 | "ds.EMT" 137 | ] 138 | }, 139 | { 140 | "cell_type": "markdown", 141 | "metadata": {}, 142 | "source": [ 143 | "As expected, the unit of the integrated water vapour flux is $kg m^{-1}s^{-1}$! \n", 144 | "We have a variable for the eastward transport and one for the northward transport of water vapour with $u$, the zonal wind or $v$, the meridional wind respectively. This is pretty similar to having $u$ and $v$ as the vector components of the horizontal windfield. Hence, we can take the eastward and northward vertical integral of water vapour flux as the vector components of the integrated water vapour flux field! \n", 145 | "\n", 146 | "We can plot this field as a vectorfield; this is shown below." 147 | ] 148 | }, 149 | { 150 | "cell_type": "markdown", 151 | "metadata": {}, 152 | "source": [ 153 | "## 06_03 How to Plot it" 154 | ] 155 | }, 156 | { 157 | "cell_type": "markdown", 158 | "metadata": {}, 159 | "source": [ 160 | "### 06_03_01 Vertical Integral of Water Vapour Flux" 161 | ] 162 | }, 163 | { 164 | "cell_type": "markdown", 165 | "metadata": {}, 166 | "source": [ 167 | "Similar to when we plotted the windfield, we start by averaging over the months of the year and selecting a specific month (january)." 168 | ] 169 | }, 170 | { 171 | "cell_type": "code", 172 | "execution_count": null, 173 | "metadata": {}, 174 | "outputs": [], 175 | "source": [ 176 | "emt = ds.EMT.groupby('time.month').mean(dim = 'time').sel(month=1).load()\n", 177 | "nmt = ds.NMT.groupby('time.month').mean(dim = 'time').sel(month=1).load()" 178 | ] 179 | }, 180 | { 181 | "cell_type": "markdown", 182 | "metadata": {}, 183 | "source": [ 184 | "Now plot the vectorfield with `quiver`:" 185 | ] 186 | }, 187 | { 188 | "cell_type": "code", 189 | "execution_count": null, 190 | "metadata": {}, 191 | "outputs": [], 192 | "source": [ 193 | "ax = plt.axes(projection=ccrs.PlateCarree()) \n", 194 | "e, n = emt[::10,::10], nmt[::10,::10] #read 05_PlotWindData for explanations on this!\n", 195 | "qv = ax.quiver(e.longitude, e.latitude, e, n, transform=ccrs.PlateCarree())\n", 196 | "ax.coastlines(color='grey');\n", 197 | "ax.set_title('Average Integrated Water Vapour Flux January [$kg m^{-1} s^{-1}$]');" 198 | ] 199 | }, 200 | { 201 | "cell_type": "markdown", 202 | "metadata": {}, 203 | "source": [ 204 | "As expected and mentioned above, the oceans are a source of water vapour and therfore a region where the water vapour flux diverges!" 205 | ] 206 | }, 207 | { 208 | "cell_type": "markdown", 209 | "metadata": {}, 210 | "source": [ 211 | "Again, we lack a discriptive information about the values of the depicted arrows. We can plot this as a shaded colour plot in the background.\n", 212 | "\n", 213 | "Therefore, we must first determine the absolute values of the vectors:" 214 | ] 215 | }, 216 | { 217 | "cell_type": "code", 218 | "execution_count": null, 219 | "metadata": {}, 220 | "outputs": [], 221 | "source": [ 222 | "av = np.sqrt(emt**2+nmt**2)\n", 223 | "av" 224 | ] 225 | }, 226 | { 227 | "cell_type": "markdown", 228 | "metadata": {}, 229 | "source": [ 230 | "And now plot it:" 231 | ] 232 | }, 233 | { 234 | "cell_type": "code", 235 | "execution_count": null, 236 | "metadata": { 237 | "scrolled": true 238 | }, 239 | "outputs": [], 240 | "source": [ 241 | "ax = plt.axes(projection=ccrs.PlateCarree()) \n", 242 | "av.plot.contourf(ax=ax, transform=ccrs.PlateCarree(), levels = 10, cmap='cool', cbar_kwargs={'label':'Water Vapour Flux [$kg m^{-1} s^{-1}$]'});\n", 243 | "e, n = emt[::10,::10], nmt[::10,::10]\n", 244 | "qv = ax.quiver(e.longitude, e.latitude, e, n, transform=ccrs.PlateCarree())\n", 245 | "ax.coastlines(color='grey');\n", 246 | "ax.set_title('Average Integrated Water Vapour Flux January');" 247 | ] 248 | }, 249 | { 250 | "cell_type": "markdown", 251 | "metadata": {}, 252 | "source": [ 253 | "We can also plot one of the vector components of the flux field in a 1D-plot! E.g. the northward water vapour flux, which is proportional to the energy transported by the latent heat flux. Values greater than zero indicate a northward transport, values smaller than zero a southward transport of water vapour and therefore latent heat." 254 | ] 255 | }, 256 | { 257 | "cell_type": "code", 258 | "execution_count": null, 259 | "metadata": { 260 | "scrolled": true 261 | }, 262 | "outputs": [], 263 | "source": [ 264 | "nmt_zonal = ds.NMT.mean(dim = ['time', 'longitude'])\n", 265 | "nmt_zonal.plot()\n", 266 | "plt.xlim(-90, 90) # defines the limits on the x-axis\n", 267 | "plt.title('Vertical integral of northward water vapour flux') # plots your title\n", 268 | "plt.ylabel('[kg $m^{-1}s^{-1}$]'); # changes the y-label from the default label to the one you want\n", 269 | "plt.xlabel('Latitude [° North]');\n", 270 | "plt.xticks((-90, -60, -30, 0, 30, 60, 90));\n", 271 | "plt.axhline(0, color = 'k');\n", 272 | "plt.grid(linestyle = '-.')" 273 | ] 274 | }, 275 | { 276 | "cell_type": "markdown", 277 | "metadata": {}, 278 | "source": [ 279 | "### 06_03_02 Vertical integral of divergence of water vapour flux" 280 | ] 281 | }, 282 | { 283 | "cell_type": "markdown", 284 | "metadata": {}, 285 | "source": [ 286 | "We can also plot the divergence of the flux we plotted above. Let's load the data first:" 287 | ] 288 | }, 289 | { 290 | "cell_type": "code", 291 | "execution_count": null, 292 | "metadata": { 293 | "scrolled": true 294 | }, 295 | "outputs": [], 296 | "source": [ 297 | "ds = xr.open_dataset('./data/ERA5-LowRes-MonthlyAvg-VertIntDivMoistFlux.nc')\n", 298 | "ds = ds.rename({\"p84.162\" : \"div\"})\n", 299 | "ds.div" 300 | ] 301 | }, 302 | { 303 | "cell_type": "markdown", 304 | "metadata": {}, 305 | "source": [ 306 | "Since it is a divergence of the moisture flux field, the unit changes from $kg m^{-1}s^{-1}$ to $kg m^{-2}s^{-1}$. We can transform this unit into $mm/day$ by multiplying it with $24\\cdot60\\cdot60$, the seconds of a day! This is a more intuitiv unit.\n", 307 | "\n", 308 | "Let's do this and then plot it:" 309 | ] 310 | }, 311 | { 312 | "cell_type": "code", 313 | "execution_count": null, 314 | "metadata": {}, 315 | "outputs": [], 316 | "source": [ 317 | "div_jan = ds.div.groupby('time.month').mean(dim = 'time').sel(month = 1)\n", 318 | "div_jan = div_jan *24*60*60\n", 319 | "ax = plt.axes(projection=ccrs.Robinson())\n", 320 | "div_jan.plot(ax=ax, transform=ccrs.PlateCarree(), levels = np.arange(-8, 8, 2),vmin = -8, vmax= 8, cmap = 'cool', \n", 321 | " cbar_kwargs={'label': '$mm/day$'}) \n", 322 | "ax.set_title('Averaged Vertical integral of divergence of moisture flux January')\n", 323 | "ax.coastlines(); ax.gridlines();" 324 | ] 325 | }, 326 | { 327 | "cell_type": "markdown", 328 | "metadata": {}, 329 | "source": [ 330 | "The major source regions for atmospheric moisture can be identified in the subtropical anticyclones, and major sink regions in the convergence zones and summer monsoon rains (Trenberth et al., 2011). Trenberth furhtermore states, that the tendency term $\\frac{\\partial w}{\\partial t}$ is very small for time averages. Therefore, one can take the divergence of the moisture flux as proportional to $E-P$. " 331 | ] 332 | } 333 | ], 334 | "metadata": { 335 | "kernelspec": { 336 | "display_name": "Python 3", 337 | "language": "python", 338 | "name": "python3" 339 | }, 340 | "language_info": { 341 | "codemirror_mode": { 342 | "name": "ipython", 343 | "version": 3 344 | }, 345 | "file_extension": ".py", 346 | "mimetype": "text/x-python", 347 | "name": "python", 348 | "nbconvert_exporter": "python", 349 | "pygments_lexer": "ipython3", 350 | "version": "3.6.6" 351 | } 352 | }, 353 | "nbformat": 4, 354 | "nbformat_minor": 2 355 | } 356 | -------------------------------------------------------------------------------- /07_Correlation Maps.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# 07_Correlation Maps" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "In this chapter, we will learn how to produce correlation plots in 1D (scatter plots) and 2D (maps). As an example we will take the prominent feature of the North Atlantic Oscillation (NAO).\n", 15 | "\n", 16 | "Some word about this phenomena:\n", 17 | "\n", 18 | "Over the middle and high latitudes of the Northern Hemisphere, especially during the cold season months(November-April), the most prominent and recurrent pattern of atmospheric variability is the NAO (James Hurrel et al., 2003). It is a weather phenomenon of fluctuations in the difference of atmospheric pressure at sea level (SLP) between the Icelandic Low and the Azores High. Through fluctuations in the strength of the Icelandic low and the Azores high, it controls the strength and direction of westerly winds and location of storm tracks across the North Atlantic (James Hurrel et al., 2003). It is a perfect example for correlation: in phases of positive NAO+, both the Azores High as well as the Icelandic Low are well pronounced compared to the mean over time. In contrast for phases of NAO-, both are rather weak.\n", 19 | "\n", 20 | "See the plots below from Wanner, H., Brönnimann, S., Casty, C., et al. (2001):" 21 | ] 22 | }, 23 | { 24 | "cell_type": "markdown", 25 | "metadata": {}, 26 | "source": [ 27 | "**NAO+**\n", 28 | "\n", 29 | "" 30 | ] 31 | }, 32 | { 33 | "cell_type": "markdown", 34 | "metadata": {}, 35 | "source": [ 36 | "**NAO-**\n", 37 | "\n", 38 | "" 39 | ] 40 | }, 41 | { 42 | "cell_type": "code", 43 | "execution_count": null, 44 | "metadata": {}, 45 | "outputs": [], 46 | "source": [ 47 | "# Display the plots in the notebook:\n", 48 | "%matplotlib inline" 49 | ] 50 | }, 51 | { 52 | "cell_type": "code", 53 | "execution_count": null, 54 | "metadata": {}, 55 | "outputs": [], 56 | "source": [ 57 | "# Import the tools we are going to need today:\n", 58 | "import matplotlib.pyplot as plt # plotting library\n", 59 | "import numpy as np # numerical library\n", 60 | "import xarray as xr # netCDF library\n", 61 | "import cartopy # Map projections libary\n", 62 | "import cartopy.crs as ccrs # Projections list\n", 63 | "import pandas as pd # new package! this is the package at the base of xarray\n", 64 | "#from eofs.xarray import Eof # new package! http://ajdawson.github.io/eofs/index.html\n", 65 | "# Some defaults:\n", 66 | "plt.rcParams['figure.figsize'] = (12, 5) # Default plot size\n", 67 | "np.set_printoptions(threshold=20) # avoid to print very large arrays on screen\n", 68 | "# The commands below are to ignore certain warnings.\n", 69 | "import warnings\n", 70 | "warnings.filterwarnings('ignore')\n" 71 | ] 72 | }, 73 | { 74 | "cell_type": "markdown", 75 | "metadata": {}, 76 | "source": [ 77 | "## 07_01 One Point Correlation Maps" 78 | ] 79 | }, 80 | { 81 | "cell_type": "markdown", 82 | "metadata": {}, 83 | "source": [ 84 | "A one point correlation map correlates a variable all over the globe with the same variable at a specific site.\n", 85 | "\n", 86 | "Our aim is now, to correlate the northern hemisphere winter season geopotential @500 hPa to the winter season geopotential on Iceland. We expect a highly negative correlation between Iceland and the Azores! We will first need to load the data:" 87 | ] 88 | }, 89 | { 90 | "cell_type": "code", 91 | "execution_count": null, 92 | "metadata": {}, 93 | "outputs": [], 94 | "source": [ 95 | "geop = xr.open_dataset('./data/ERA5-LowRes-MonthlyAvg-500hPa-UVZ.nc').sel(latitude=slice(90, 20)).z" 96 | ] 97 | }, 98 | { 99 | "cell_type": "markdown", 100 | "metadata": {}, 101 | "source": [ 102 | "We loaded the dataset, selected the northern hemisphere and the variable z all in on step. See what we got:" 103 | ] 104 | }, 105 | { 106 | "cell_type": "code", 107 | "execution_count": null, 108 | "metadata": { 109 | "scrolled": true 110 | }, 111 | "outputs": [], 112 | "source": [ 113 | "geop = xr.open_dataset('./data/ERA5-LowRes-MonthlyAvg-500hPa-UVZ.nc').sel(latitude=slice(90, 20)).z\n", 114 | "geop" 115 | ] 116 | }, 117 | { 118 | "cell_type": "markdown", 119 | "metadata": {}, 120 | "source": [ 121 | "Divide this by $9.8$ to get the geopotential height out of the geopotential!" 122 | ] 123 | }, 124 | { 125 | "cell_type": "code", 126 | "execution_count": null, 127 | "metadata": {}, 128 | "outputs": [], 129 | "source": [ 130 | "geop = geop/9.8" 131 | ] 132 | }, 133 | { 134 | "cell_type": "markdown", 135 | "metadata": {}, 136 | "source": [ 137 | "That worked out well. Next, we need to get the two variables we need for the correlation: we need to select the winter season DJF plus take an average over each season and we need to create another variable, the geopotential of iceland (select lon and lat)." 138 | ] 139 | }, 140 | { 141 | "cell_type": "code", 142 | "execution_count": null, 143 | "metadata": {}, 144 | "outputs": [], 145 | "source": [ 146 | "# create the northern hemisphere variable\n", 147 | "geop_djf = geop.where(geop['time.season'] == 'DJF') #select winter season since correlation is strongest here\n", 148 | "geop_djf = geop_djf.groupby('time.year').mean('time') #average over the year (over each season)" 149 | ] 150 | }, 151 | { 152 | "cell_type": "code", 153 | "execution_count": null, 154 | "metadata": { 155 | "scrolled": false 156 | }, 157 | "outputs": [], 158 | "source": [ 159 | "# create the iceland variable\n", 160 | "iceland = geop_djf.sel(latitude=65, longitude=-30, method='nearest') #select the lat & lon coords of iceland\n", 161 | "iceland" 162 | ] 163 | }, 164 | { 165 | "cell_type": "markdown", 166 | "metadata": {}, 167 | "source": [ 168 | "What we aim to compute next is the correlation coefficient $r_{X,Y}$ of those two variables: \n", 169 | "\n", 170 | "$r_{X,Y} = \\frac{cov(X,Y)}{\\sigma_X\\sigma_Y}$\n", 171 | "\n", 172 | "\n", 173 | "$cov(X,Y) = \\overline{(X-\\overline{X})(Y-\\overline{Y})}$\n", 174 | "\n", 175 | "$r_{X,Y}$ can take values between $-1$ and $+1$, where $-1$ means perfect negative correlation and $+1$ perfect positive correlation of the two variables $X$ and $Y$." 176 | ] 177 | }, 178 | { 179 | "cell_type": "markdown", 180 | "metadata": {}, 181 | "source": [ 182 | "To compute $r_{X,Y}$ of two specific sites is quite easy, since only the time dimension remains and the other coordinates (lat and lon) vanish. There is a special numpy method, that does exactly this: `np.corrcoef(X,Y)`.\n", 183 | "Let's try this first:" 184 | ] 185 | }, 186 | { 187 | "cell_type": "code", 188 | "execution_count": null, 189 | "metadata": {}, 190 | "outputs": [], 191 | "source": [ 192 | "azores = geop_djf.sel(latitude = 38, longitude = -25.5, method = 'nearest') #select lat,lon of azores\n", 193 | "r_xy = np.corrcoef(iceland,azores)\n", 194 | "r_xy" 195 | ] 196 | }, 197 | { 198 | "cell_type": "markdown", 199 | "metadata": {}, 200 | "source": [ 201 | "The numpy function returns a matrix of the following form:\n", 202 | "\n", 203 | "$\\left(\\begin{array}{rr} \n", 204 | "r_{X,X} & r_{X,Y}\\\\ \n", 205 | "r_{Y,X} & r_{Y,Y} \\\\ \n", 206 | "\\end{array}\\right)$\n", 207 | "\n", 208 | "so what we need is the upper right entry of this matrix. We can select it with `[0, 1]`:" 209 | ] 210 | }, 211 | { 212 | "cell_type": "code", 213 | "execution_count": null, 214 | "metadata": {}, 215 | "outputs": [], 216 | "source": [ 217 | "r_xy [0,1] #select first row, second column" 218 | ] 219 | }, 220 | { 221 | "cell_type": "markdown", 222 | "metadata": {}, 223 | "source": [ 224 | "We can also plot a scatter plot of the geopotential of the azores vs. iceland with `plt.scatter(X,Y)`. We can clearly see the negative correlation in there too:" 225 | ] 226 | }, 227 | { 228 | "cell_type": "code", 229 | "execution_count": null, 230 | "metadata": {}, 231 | "outputs": [], 232 | "source": [ 233 | "plt.scatter(azores, iceland)\n", 234 | "plt.xlabel('Geopotential Height Azores [m]');\n", 235 | "plt.ylabel('Geopotential Height Iceland [m]');\n", 236 | "plt.title('Negative Correlation of Geopotential Height');" 237 | ] 238 | }, 239 | { 240 | "cell_type": "markdown", 241 | "metadata": {}, 242 | "source": [ 243 | "From this, we can already conclude that the icelandic low and the azores high are correlated. Still, a map would be more appealing to see which regions correlate and which do not. Therefore, we must compute the correlation coefficient for each grid point on the northern hemisphere. We can do this by two so called **for loops**!\n", 244 | "\n", 245 | "What is a for loop?\n", 246 | "\n", 247 | "A loop is a tool in programming, to repeat the same command for different variables with only one time writing the command itself. A for loop iterates over those variables. Let's try it on an example:\n" 248 | ] 249 | }, 250 | { 251 | "cell_type": "code", 252 | "execution_count": null, 253 | "metadata": {}, 254 | "outputs": [], 255 | "source": [ 256 | "for i in list([1,2,3,4,5]): \n", 257 | " print(i)" 258 | ] 259 | }, 260 | { 261 | "cell_type": "markdown", 262 | "metadata": {}, 263 | "source": [ 264 | "The for loop picks the i'th value out of the list, starting at $i=0$. Next, the command inside the for loop is executed: the ith list value (in this case 1) is printed. Next the for loop iteration selects $i=1$ and the execution of the print command prints 2, and so on until the end of the list is reached.\n", 265 | "We can now also generate a nested set of two for loops:\n" 266 | ] 267 | }, 268 | { 269 | "cell_type": "code", 270 | "execution_count": null, 271 | "metadata": {}, 272 | "outputs": [], 273 | "source": [ 274 | "for i in list([1,2,3,4,5]):\n", 275 | " for j in list([6,7,8,9,10]):\n", 276 | " print(i,j)\n", 277 | " " 278 | ] 279 | }, 280 | { 281 | "cell_type": "markdown", 282 | "metadata": {}, 283 | "source": [ 284 | "We start with $i=0$, so $1$ is printed. The $i$ value stays constant, while we loop through all the $j$ values! Once the j-loop is completed, $i$ is set to $1$ (the second value of the list is printed: $2$).\n", 285 | "\n", 286 | "That's exactly what we need for our correlation coefficient map, that persists of the correlation coefficient at each grid point! We simply need to iterate over all latitudes and longitudes in a nested for loop and compute the correlation coeffiecient with `np.corrcoef(X,Y)` for each grid point. See the explanations underneath each line:" 287 | ] 288 | }, 289 | { 290 | "cell_type": "code", 291 | "execution_count": null, 292 | "metadata": {}, 293 | "outputs": [], 294 | "source": [ 295 | "# first, we make an empty array that we will fill with the correlation coefficients of each grid point\n", 296 | "cor_map = geop_djf[0,:,:] * 0.\n", 297 | "# for loops over lats and lons\n", 298 | "for j in np.arange(len(geop_djf.latitude)):\n", 299 | "#len(geop_djf.latitude) returns the lengts of the latitude coordinate. np.arange(-\"-) therefore creates an array \n", 300 | "#starting from 0 to len(geop_djf.latitude) --> those are the indexes we iterate with!\n", 301 | " for i in np.arange(len(geop_djf.longitude)):\n", 302 | " cor_map.values[j, i] = np.corrcoef(geop_djf.values[:, j, i], iceland.values)[0, 1]\n", 303 | " # for every lat, lon combination (j,i) we compute the np.corrcoef and safe it to the respective j,i matrix entry\n", 304 | " # of the predefined cor_map\n", 305 | " # geop_djf.values[:,j,i] selects all values in time, but only the latitude and longitude that we are dealing\n", 306 | " # with in this iteration (j,i)\n", 307 | " # iceland only has one lat and lon, so nothing needs to be selected\n", 308 | " # we use the .values attribute because this is much faster" 309 | ] 310 | }, 311 | { 312 | "cell_type": "markdown", 313 | "metadata": {}, 314 | "source": [ 315 | "Let's have a look if the just created `cor_map` looks as expected:\n" 316 | ] 317 | }, 318 | { 319 | "cell_type": "code", 320 | "execution_count": null, 321 | "metadata": {}, 322 | "outputs": [], 323 | "source": [ 324 | "cor_map" 325 | ] 326 | }, 327 | { 328 | "cell_type": "markdown", 329 | "metadata": {}, 330 | "source": [ 331 | "Now, we can plot the result of the correlation: the correlation coefficient! To get a nicer view on the northern hemisphere, we select a different projection, the North Polar Stereo Projection. " 332 | ] 333 | }, 334 | { 335 | "cell_type": "code", 336 | "execution_count": null, 337 | "metadata": {}, 338 | "outputs": [], 339 | "source": [ 340 | "fig = plt.figure(figsize=(9, 7))\n", 341 | "ax = plt.axes(projection=ccrs.NorthPolarStereo()) \n", 342 | "ax.set_extent([-180, 180, 20, 90], ccrs.PlateCarree())\n", 343 | "ax.coastlines(); ax.gridlines();\n", 344 | "cs = cor_map.plot.contourf(ax=ax, transform=ccrs.PlateCarree(), cbar_kwargs={'label':'Correlation Coefficient'}, \n", 345 | " levels=np.linspace(-0.8, 0.8, 9), extend='both')\n", 346 | "plt.title('One point correlation map (ref 65°N, 30°W)');" 347 | ] 348 | }, 349 | { 350 | "cell_type": "markdown", 351 | "metadata": {}, 352 | "source": [ 353 | "If you want, you can also make a circular plot instead of the quadratic one. First, you'll have to run these few lines (only once for the notebook!):" 354 | ] 355 | }, 356 | { 357 | "cell_type": "code", 358 | "execution_count": null, 359 | "metadata": {}, 360 | "outputs": [], 361 | "source": [ 362 | "import matplotlib.path as mpath\n", 363 | "theta = np.linspace(0, 2*np.pi, 100)\n", 364 | "map_circle = mpath.Path(np.vstack([np.sin(theta), np.cos(theta)]).T * 0.5 + [0.5, 0.5])" 365 | ] 366 | }, 367 | { 368 | "cell_type": "markdown", 369 | "metadata": {}, 370 | "source": [ 371 | "And then add one line to the plot commands:" 372 | ] 373 | }, 374 | { 375 | "cell_type": "code", 376 | "execution_count": null, 377 | "metadata": { 378 | "scrolled": true 379 | }, 380 | "outputs": [], 381 | "source": [ 382 | "fig = plt.figure(figsize=(9, 7))\n", 383 | "ax = plt.axes(projection=ccrs.NorthPolarStereo()) \n", 384 | "ax.set_boundary(map_circle, transform=ax.transAxes) #this is the only new line, the rest did not change\n", 385 | "ax.set_extent([-180, 180, 20, 90], ccrs.PlateCarree())\n", 386 | "ax.coastlines(); ax.gridlines();\n", 387 | "cs = cor_map.plot.contourf(ax=ax, transform=ccrs.PlateCarree(), cbar_kwargs={'label':'Correlation Coefficient'}, \n", 388 | " levels=np.linspace(-0.8, 0.8, 9), extend='both')\n", 389 | "plt.title('One point correlation map (ref 65°N, 30°W)');" 390 | ] 391 | }, 392 | { 393 | "cell_type": "markdown", 394 | "metadata": {}, 395 | "source": [ 396 | "## 07_02 Correlation Maps" 397 | ] 398 | }, 399 | { 400 | "cell_type": "markdown", 401 | "metadata": {}, 402 | "source": [ 403 | "Another example of correlation maps would be to correlate to variables all over a certain area instead of one variable all over the area with the same variable at a certain location (one point correlationn maps).\n", 404 | "\n", 405 | "We try this with the NAO index and the temperatures over europe in the winter season. Therefore, we must first get the NAO index. ython makes it very easy to read data directly from a given url:" 406 | ] 407 | }, 408 | { 409 | "cell_type": "code", 410 | "execution_count": null, 411 | "metadata": {}, 412 | "outputs": [], 413 | "source": [ 414 | "# import the modules we need\n", 415 | "import io, requests" 416 | ] 417 | }, 418 | { 419 | "cell_type": "code", 420 | "execution_count": null, 421 | "metadata": { 422 | "scrolled": false 423 | }, 424 | "outputs": [], 425 | "source": [ 426 | "# This just reads the data from an url\n", 427 | "# data from http://www.cpc.ncep.noaa.gov/data/teledoc/nao.shtml\n", 428 | "url = 'http://ftp.cpc.ncep.noaa.gov/wd52dg/data/indices/nao_index.tim'\n", 429 | "s = requests.get(url).content\n", 430 | "df = pd.read_csv(io.StringIO(s.decode('utf-8')), delim_whitespace=True, skiprows=7)\n", 431 | "# Parse the time and convert to xarray\n", 432 | "time = pd.to_datetime(df.YEAR.astype(str) + '-' + df.MONTH.astype(str))\n", 433 | "nao = xr.DataArray(df.INDEX, dims='time', coords={'time':time})\n", 434 | "# Select the ERA period\n", 435 | "nao = nao.sel(time=slice('1979', '2014'))" 436 | ] 437 | }, 438 | { 439 | "cell_type": "markdown", 440 | "metadata": {}, 441 | "source": [ 442 | "See what we got:\n" 443 | ] 444 | }, 445 | { 446 | "cell_type": "code", 447 | "execution_count": null, 448 | "metadata": {}, 449 | "outputs": [], 450 | "source": [ 451 | "nao" 452 | ] 453 | }, 454 | { 455 | "cell_type": "code", 456 | "execution_count": null, 457 | "metadata": { 458 | "scrolled": true 459 | }, 460 | "outputs": [], 461 | "source": [ 462 | "nao.plot()\n", 463 | "plt.axhline(0, color = 'k');\n", 464 | "plt.ylabel('NAO Index');\n", 465 | "plt.xlim(['1979-01-01', '2014-12-01']);" 466 | ] 467 | }, 468 | { 469 | "cell_type": "markdown", 470 | "metadata": {}, 471 | "source": [ 472 | "To be able to create a scatter plot, we can now have a look the NAO index versus the temperatures over norway!\n", 473 | "Load the temperature dataset, select and average over a part norway and take the winter season again:" 474 | ] 475 | }, 476 | { 477 | "cell_type": "code", 478 | "execution_count": null, 479 | "metadata": { 480 | "scrolled": true 481 | }, 482 | "outputs": [], 483 | "source": [ 484 | "t2m = xr.open_dataset('./data/ERA5--LowRes-MonthlyAvg-t2m_tp.nc').t2m.sel(time = slice('1979','2014'))\n", 485 | "t2m_nor = t2m.sel(latitude = slice(63,60), longitude = slice(8.5,10.5)).mean(dim = ['latitude', 'longitude']) - 273.15\n", 486 | "t2m_nor" 487 | ] 488 | }, 489 | { 490 | "cell_type": "code", 491 | "execution_count": null, 492 | "metadata": {}, 493 | "outputs": [], 494 | "source": [ 495 | "t2m_nor = t2m_nor.where(t2m_nor['time.season'] == 'DJF') #select winter season since correlation is strongest here\n", 496 | "t2m_nor = t2m_nor.groupby('time.year').mean('time') #average over the year (over each season)\n", 497 | "\n", 498 | "nao = nao.where(nao['time.season'] == 'DJF') \n", 499 | "nao = nao.groupby('time.year').mean('time') " 500 | ] 501 | }, 502 | { 503 | "cell_type": "markdown", 504 | "metadata": {}, 505 | "source": [ 506 | "Let's have a look at the scatter plot to get a first idea of the correlation:" 507 | ] 508 | }, 509 | { 510 | "cell_type": "code", 511 | "execution_count": null, 512 | "metadata": {}, 513 | "outputs": [], 514 | "source": [ 515 | "plt.scatter(nao, t2m_nor)\n", 516 | "plt.xlabel('NAO Index');\n", 517 | "plt.ylabel('Temperature over Norway [°C]');\n", 518 | "plt.title('Positive Correlation')\n", 519 | "plt.xlim([-1.2,1.0]);" 520 | ] 521 | }, 522 | { 523 | "cell_type": "markdown", 524 | "metadata": {}, 525 | "source": [ 526 | "Now, we want to create a map of europe with the correlation values of NAO Index with temperatures.\n", 527 | "In order to do that, we need a for loop again. " 528 | ] 529 | }, 530 | { 531 | "cell_type": "code", 532 | "execution_count": null, 533 | "metadata": {}, 534 | "outputs": [], 535 | "source": [ 536 | "t2m = t2m.where(t2m['time.season'] == 'DJF') #select winter season since correlation is strongest here\n", 537 | "t2m = t2m.groupby('time.year').mean('time') #average over the year (over each season)" 538 | ] 539 | }, 540 | { 541 | "cell_type": "code", 542 | "execution_count": null, 543 | "metadata": {}, 544 | "outputs": [], 545 | "source": [ 546 | "# first, we make an empty array that we will fill with the correlation coefficients of each grid point\n", 547 | "cor_map = t2m[0,:,:] * 0.\n", 548 | "# for loops over lats and lons\n", 549 | "for j in np.arange(len(t2m.latitude)):\n", 550 | " for i in np.arange(len(t2m.longitude)):\n", 551 | " cor_map.values[j, i] = np.corrcoef(t2m.values[:, j, i], nao.values)[0, 1]" 552 | ] 553 | }, 554 | { 555 | "cell_type": "code", 556 | "execution_count": null, 557 | "metadata": {}, 558 | "outputs": [], 559 | "source": [ 560 | "fig = plt.figure(figsize=(9, 7))\n", 561 | "ax = plt.axes(projection=ccrs.EuroPP()) # take the projection for europe \n", 562 | "ax.coastlines(); ax.gridlines();\n", 563 | "cs = cor_map.plot.contourf(ax=ax, transform=ccrs.PlateCarree(), cbar_kwargs={'label':'Correlation Coefficient'}, \n", 564 | " levels=np.linspace(-0.8, 0.8, 9), extend='both')\n", 565 | "plt.title('Correlation map DJF NAO Index and Temperature');" 566 | ] 567 | }, 568 | { 569 | "cell_type": "markdown", 570 | "metadata": {}, 571 | "source": [ 572 | "As already expected when trying to understand the plots from Wanner, H., Brönnimann, S., Casty, C., et al. (2001) at the very beginning of this chapter, NAO and temperatures over northern europe are highly positive correlated. This means, NAO+ causes the temperatures over northern europe to be higher than the mean." 573 | ] 574 | } 575 | ], 576 | "metadata": { 577 | "kernelspec": { 578 | "display_name": "Python 3", 579 | "language": "python", 580 | "name": "python3" 581 | }, 582 | "language_info": { 583 | "codemirror_mode": { 584 | "name": "ipython", 585 | "version": 3 586 | }, 587 | "file_extension": ".py", 588 | "mimetype": "text/x-python", 589 | "name": "python", 590 | "nbconvert_exporter": "python", 591 | "pygments_lexer": "ipython3", 592 | "version": "3.6.6" 593 | } 594 | }, 595 | "nbformat": 4, 596 | "nbformat_minor": 2 597 | } 598 | -------------------------------------------------------------------------------- /08_AverageoverSphere.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# 08_How to average over a sphere" 8 | ] 9 | }, 10 | { 11 | "cell_type": "code", 12 | "execution_count": null, 13 | "metadata": {}, 14 | "outputs": [], 15 | "source": [ 16 | "# Display the plots in the notebook:\n", 17 | "%matplotlib inline" 18 | ] 19 | }, 20 | { 21 | "cell_type": "code", 22 | "execution_count": null, 23 | "metadata": {}, 24 | "outputs": [], 25 | "source": [ 26 | "# Import the tools we are going to need today:\n", 27 | "import matplotlib.pyplot as plt # plotting library\n", 28 | "import numpy as np # numerical library\n", 29 | "import xarray as xr # netCDF library\n", 30 | "import cartopy # Map projections libary\n", 31 | "import cartopy.crs as ccrs # Projections list\n", 32 | "# Some defaults:\n", 33 | "plt.rcParams['figure.figsize'] = (12, 5) # Default plot size\n", 34 | "np.set_printoptions(threshold=20) # avoid to print very large arrays on screen\n", 35 | "# The commands below are to ignore certain warnings.\n", 36 | "import warnings\n", 37 | "warnings.filterwarnings('ignore')" 38 | ] 39 | }, 40 | { 41 | "cell_type": "markdown", 42 | "metadata": {}, 43 | "source": [ 44 | "### Averaging over a Sphere" 45 | ] 46 | }, 47 | { 48 | "cell_type": "markdown", 49 | "metadata": {}, 50 | "source": [ 51 | "As our planet earth is a sphere (more or less), if we average over latitude and longitude we will always take the average over a sphere! You will have to pay attention with that, and here comes why: \n", 52 | "When averaging `[lon, lat]` data, one gives too much weight to high latitudes.\n", 53 | "We can see that, when comparing a regular 2D plot to a spheric one. \n", 54 | "\n", 55 | "We do this for the 2 meter temperature:" 56 | ] 57 | }, 58 | { 59 | "cell_type": "code", 60 | "execution_count": null, 61 | "metadata": { 62 | "scrolled": true 63 | }, 64 | "outputs": [], 65 | "source": [ 66 | "ds = xr.open_dataset('./data/ERA5-LowRes-MonthlyAvg-t2m_tp.nc')\n", 67 | "ds" 68 | ] 69 | }, 70 | { 71 | "cell_type": "code", 72 | "execution_count": null, 73 | "metadata": {}, 74 | "outputs": [], 75 | "source": [ 76 | "t2m_avg = ds.t2m.mean(dim='time')\n", 77 | "t2m_avg.mean() -273.15 # this is way too cold!" 78 | ] 79 | }, 80 | { 81 | "cell_type": "code", 82 | "execution_count": null, 83 | "metadata": {}, 84 | "outputs": [], 85 | "source": [ 86 | "ax = plt.axes(projection=ccrs.Robinson())\n", 87 | "t2m_avg.plot(ax=ax, transform=ccrs.PlateCarree()) \n", 88 | "ax.coastlines(); ax.gridlines(); " 89 | ] 90 | }, 91 | { 92 | "cell_type": "code", 93 | "execution_count": null, 94 | "metadata": {}, 95 | "outputs": [], 96 | "source": [ 97 | "t2m_avg.plot();" 98 | ] 99 | }, 100 | { 101 | "cell_type": "markdown", 102 | "metadata": {}, 103 | "source": [ 104 | "If we compare those two plots, we recognise that in a grid space as we have it in our dataset (PlateCarre, second plot), high latitudes are given too much weight in comparison to the spheric plot! \n", 105 | "\n", 106 | "Fortunately, this can be solved by noting that we have to weight each latitudinal band by the cosinus of the latitude, i.e. $\\cos \\varphi$. This results in low latitudes having a lot of weight ($\\cos ({0}) = 1$) and vice versa. We are going to compute a new average, but [weighted](https://en.wikipedia.org/wiki/Weighted_arithmetic_mean) this time. First, let's make a weight array:" 107 | ] 108 | }, 109 | { 110 | "cell_type": "code", 111 | "execution_count": null, 112 | "metadata": {}, 113 | "outputs": [], 114 | "source": [ 115 | "weight = np.cos(np.deg2rad(ds.latitude)) #deg2rad converts the latitude from degrees to radians\n", 116 | "weight = weight / weight.sum() #weigh each latitudinal band by dividing it by the summed weights" 117 | ] 118 | }, 119 | { 120 | "cell_type": "code", 121 | "execution_count": null, 122 | "metadata": {}, 123 | "outputs": [], 124 | "source": [ 125 | "weight.plot();\n", 126 | "plt.ylabel('Weight');" 127 | ] 128 | }, 129 | { 130 | "cell_type": "markdown", 131 | "metadata": {}, 132 | "source": [ 133 | "Weight is an array of 180 elements, which is normalised so that it's sum is 1. This is exactly what we need to compute a weighted average! First, we have to average over the longitudes (this is fine, because along a latitude circle all points have the same weight), and then compute the weighted average." 134 | ] 135 | }, 136 | { 137 | "cell_type": "code", 138 | "execution_count": null, 139 | "metadata": {}, 140 | "outputs": [], 141 | "source": [ 142 | "zonal_t2m_avg = t2m_avg.mean(dim='longitude') # important! Always average over longitudes first\n", 143 | "#this averaging is needed so that the arithmetic below makes sense (multiply two arrays of 180 elements with each other)\n", 144 | "weighted_t2m_avg = np.sum(zonal_t2m_avg * weight) #sum over all weighted latitudinal t2m values\n", 145 | "weighted_t2m_avg - 273.15 #physically resonable 2 meter temperature average" 146 | ] 147 | }, 148 | { 149 | "cell_type": "markdown", 150 | "metadata": {}, 151 | "source": [ 152 | "The new result for the average 2 meter temperature on our globe is 14°C. This seems much more reasonable than the 5°C we had before doing those calculations." 153 | ] 154 | }, 155 | { 156 | "cell_type": "markdown", 157 | "metadata": {}, 158 | "source": [ 159 | "To sum it up once more: whenever you **average over latitudes**, you will need to **take into account that our planet is a sphere** and do the arithmetics above in order to get the correct result!" 160 | ] 161 | } 162 | ], 163 | "metadata": { 164 | "kernelspec": { 165 | "display_name": "Python 3", 166 | "language": "python", 167 | "name": "python3" 168 | }, 169 | "language_info": { 170 | "codemirror_mode": { 171 | "name": "ipython", 172 | "version": 3 173 | }, 174 | "file_extension": ".py", 175 | "mimetype": "text/x-python", 176 | "name": "python", 177 | "nbconvert_exporter": "python", 178 | "pygments_lexer": "ipython3", 179 | "version": "3.6.6" 180 | } 181 | }, 182 | "nbformat": 4, 183 | "nbformat_minor": 2 184 | } 185 | -------------------------------------------------------------------------------- /09_ToGoFurther.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "## 09 More About Numpy" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "#### 01_02_03_02 Numpy ndarrays" 15 | ] 16 | }, 17 | { 18 | "cell_type": "markdown", 19 | "metadata": {}, 20 | "source": [ 21 | "N-dimensional ndarray's are the core structure of the numpy library. Those are multidimensional container of items of the same type and size. The number of dimensions and items in an array is defined by its shape, which is a tuple of N positive integers that specify the number of items in each dimension. The type of items in the array is specified by a separate data-type object (dtype), one of which is associated with each ndarray.\n", 22 | "\n", 23 | "All ndarrays are homogenous: every item takes up the same size block of memory, and all blocks are interpreted in exactly the same way." 24 | ] 25 | }, 26 | { 27 | "cell_type": "markdown", 28 | "metadata": {}, 29 | "source": [ 30 | "#### 01_02_03_02 Creating ndarrays" 31 | ] 32 | }, 33 | { 34 | "cell_type": "markdown", 35 | "metadata": {}, 36 | "source": [ 37 | "You can create ndarrays in several different [ways](https://docs.scipy.org/doc/numpy/reference/routines.array-creation.html#array-creation-routines) with the functions, that come with importing the numpy library:" 38 | ] 39 | }, 40 | { 41 | "cell_type": "code", 42 | "execution_count": null, 43 | "metadata": {}, 44 | "outputs": [], 45 | "source": [ 46 | "import numpy as np" 47 | ] 48 | }, 49 | { 50 | "cell_type": "markdown", 51 | "metadata": {}, 52 | "source": [ 53 | "**`np.emtpy`, `np.zeros`, `np.ones`, `np.full`**: Creates a ndarray with empty elements, zeros, ones or a given fill value. You will need to indicate the desired shape of your array in the form: `np.ones((2,3)`, which means two rows, three columns. Furthermore you can indicate the type of data, you want to create within your array: `np.ones(2, dtype = np.bool`. If you use np.full, you must also specify the fill value in the form `np.full((2,3), 5)`." 54 | ] 55 | }, 56 | { 57 | "cell_type": "code", 58 | "execution_count": null, 59 | "metadata": {}, 60 | "outputs": [], 61 | "source": [ 62 | "np.ones((2,3))" 63 | ] 64 | }, 65 | { 66 | "cell_type": "code", 67 | "execution_count": null, 68 | "metadata": {}, 69 | "outputs": [], 70 | "source": [ 71 | "np.ones(2, dtype = np.bool)" 72 | ] 73 | }, 74 | { 75 | "cell_type": "code", 76 | "execution_count": null, 77 | "metadata": {}, 78 | "outputs": [], 79 | "source": [ 80 | "np.full((2,3), 5)" 81 | ] 82 | }, 83 | { 84 | "cell_type": "markdown", 85 | "metadata": {}, 86 | "source": [ 87 | "**`np.array`**: converts existing data into a numpy ndarray. " 88 | ] 89 | }, 90 | { 91 | "cell_type": "code", 92 | "execution_count": null, 93 | "metadata": {}, 94 | "outputs": [], 95 | "source": [ 96 | "a = [1, 2, 3] #this is a list in python\n", 97 | "b = [4, 5, 6]\n", 98 | "\n", 99 | "np.array([a,b]) #this is a numpy ndarray --> does not behave like a list!" 100 | ] 101 | }, 102 | { 103 | "cell_type": "markdown", 104 | "metadata": {}, 105 | "source": [ 106 | "**`np.arange`**: creates a ndarray startin from the start argument, stepping by the step interval to the stop argument. You will define it in the form `np.arange(start, stop, step)`.\n", 107 | "Be careful! the start and stop arguments define the half open interval `[start, stop[`:\n" 108 | ] 109 | }, 110 | { 111 | "cell_type": "code", 112 | "execution_count": null, 113 | "metadata": {}, 114 | "outputs": [], 115 | "source": [ 116 | "np.arange(5)" 117 | ] 118 | }, 119 | { 120 | "cell_type": "code", 121 | "execution_count": null, 122 | "metadata": {}, 123 | "outputs": [], 124 | "source": [ 125 | "np.arange(0, 10, 2) #10 is left out!!" 126 | ] 127 | }, 128 | { 129 | "cell_type": "markdown", 130 | "metadata": {}, 131 | "source": [ 132 | "#### 01_02_03_03 Indexing" 133 | ] 134 | }, 135 | { 136 | "cell_type": "markdown", 137 | "metadata": {}, 138 | "source": [ 139 | "Indexing gets quite important with huge datasets where you want to pick out specific data for your analysis. Before we start with indexing in general, it is very important to notice that there a two ways of indexing: basic indexing and advanced indexing. \n", 140 | "\n", 141 | "* **Basic indexing**: if you select a part of your array by basic indexing, this will only create a **view** on the original array! The new array uses the same memory slot as the old array, but the data pointer only picks out the selected pieces of this old array. That means, if you change your newly created array, the original array will be changed as well. " 142 | ] 143 | }, 144 | { 145 | "cell_type": "code", 146 | "execution_count": null, 147 | "metadata": {}, 148 | "outputs": [], 149 | "source": [ 150 | "a = np.full((2,3),10)\n", 151 | "a" 152 | ] 153 | }, 154 | { 155 | "cell_type": "code", 156 | "execution_count": null, 157 | "metadata": {}, 158 | "outputs": [], 159 | "source": [ 160 | "b = a[0:1,::]\n", 161 | "b" 162 | ] 163 | }, 164 | { 165 | "cell_type": "code", 166 | "execution_count": null, 167 | "metadata": { 168 | "scrolled": true 169 | }, 170 | "outputs": [], 171 | "source": [ 172 | "b[0,2] = 0\n", 173 | "b" 174 | ] 175 | }, 176 | { 177 | "cell_type": "code", 178 | "execution_count": null, 179 | "metadata": {}, 180 | "outputs": [], 181 | "source": [ 182 | "print(a) # the element of a changed as well!! --> basic indexing as we did for b only returns a view!" 183 | ] 184 | }, 185 | { 186 | "cell_type": "markdown", 187 | "metadata": {}, 188 | "source": [ 189 | "The following types of indexing are **basic**:" 190 | ] 191 | }, 192 | { 193 | "cell_type": "markdown", 194 | "metadata": {}, 195 | "source": [ 196 | "**Slicing**: \n" 197 | ] 198 | }, 199 | { 200 | "cell_type": "markdown", 201 | "metadata": {}, 202 | "source": [ 203 | "The basic slice syntax is `i:j:k` where i is the starting index, j is the stopping index, and k is the step." 204 | ] 205 | }, 206 | { 207 | "cell_type": "code", 208 | "execution_count": null, 209 | "metadata": {}, 210 | "outputs": [], 211 | "source": [ 212 | "a = np.arange(10)\n", 213 | "a" 214 | ] 215 | }, 216 | { 217 | "cell_type": "code", 218 | "execution_count": null, 219 | "metadata": {}, 220 | "outputs": [], 221 | "source": [ 222 | "a[0:8:2]" 223 | ] 224 | }, 225 | { 226 | "cell_type": "markdown", 227 | "metadata": {}, 228 | "source": [ 229 | "If you do not indicate any starting or stopping index, this means \"take all the elements along this axis\"!" 230 | ] 231 | }, 232 | { 233 | "cell_type": "code", 234 | "execution_count": null, 235 | "metadata": {}, 236 | "outputs": [], 237 | "source": [ 238 | "a[::2]" 239 | ] 240 | }, 241 | { 242 | "cell_type": "markdown", 243 | "metadata": {}, 244 | "source": [ 245 | "A negative step means start from the end of the array. A step of`-1` starts to select the entries of the array from the very end and takes each element (step = -1). So if you take all elements of one axis (`::`) and a step of `-1`, you will have your array flipped!" 246 | ] 247 | }, 248 | { 249 | "cell_type": "code", 250 | "execution_count": null, 251 | "metadata": {}, 252 | "outputs": [], 253 | "source": [ 254 | "a[::-1]" 255 | ] 256 | }, 257 | { 258 | "cell_type": "code", 259 | "execution_count": null, 260 | "metadata": { 261 | "scrolled": true 262 | }, 263 | "outputs": [], 264 | "source": [ 265 | "a[::-4]" 266 | ] 267 | }, 268 | { 269 | "cell_type": "markdown", 270 | "metadata": {}, 271 | "source": [ 272 | "**Indexing with an Integer**:" 273 | ] 274 | }, 275 | { 276 | "cell_type": "code", 277 | "execution_count": null, 278 | "metadata": {}, 279 | "outputs": [], 280 | "source": [ 281 | "x = np.array([[10, 20, 30],\n", 282 | " [40, 50, 60]])\n", 283 | "y = x[:,2]\n", 284 | "print(y)" 285 | ] 286 | }, 287 | { 288 | "cell_type": "code", 289 | "execution_count": null, 290 | "metadata": {}, 291 | "outputs": [], 292 | "source": [ 293 | "y[1] = 3\n", 294 | "x" 295 | ] 296 | }, 297 | { 298 | "cell_type": "markdown", 299 | "metadata": {}, 300 | "source": [ 301 | "* **Advanced indexing**: if you select a part of your array by advanced indexing, this will create an actual **copy** of the original array! That means there is an extra memory slot used for your new array. You can now change your new array without changing the old one too. Advanced indexing is triggered when the selection occurs over an ndarray. \n", 302 | "\n", 303 | " There are two ways of advaned indexing:" 304 | ] 305 | }, 306 | { 307 | "cell_type": "markdown", 308 | "metadata": {}, 309 | "source": [ 310 | "**Integer Indexing**: Means indexing the elements of an arrays by their coordinates inside the array. " 311 | ] 312 | }, 313 | { 314 | "cell_type": "code", 315 | "execution_count": null, 316 | "metadata": {}, 317 | "outputs": [], 318 | "source": [ 319 | "x = np.array([[ 0, 1, 2],\n", 320 | " [ 3, 4, 5],\n", 321 | " [ 6, 7, 8],\n", 322 | " [ 9, 10, 11]]) # Lets get the corner elements of this 4x3 array:\n", 323 | "\n", 324 | "x_coords = [0, 0, 2, 2]\n", 325 | "y_coords = [0,3, 0, 3]\n", 326 | "\n", 327 | "# Read the location of each element we want to select: [0,0] [0,3] [2,0] [2,3]!\n", 328 | "\n", 329 | "y = x[y_coords, x_coords] # first the column-coords and second the row-coords!\n", 330 | "print(y)\n" 331 | ] 332 | }, 333 | { 334 | "cell_type": "markdown", 335 | "metadata": {}, 336 | "source": [ 337 | "**Boolean Indexing**: Means indexing based on a conditions. Instead of the coordinate array, we use a boolean array!" 338 | ] 339 | }, 340 | { 341 | "cell_type": "code", 342 | "execution_count": null, 343 | "metadata": {}, 344 | "outputs": [], 345 | "source": [ 346 | "a = np.array([1,2,3,4])\n", 347 | "b = [True, True, False, True]\n", 348 | "\n", 349 | "a[b] # only selects the elements of a, where b is true!\n" 350 | ] 351 | }, 352 | { 353 | "cell_type": "code", 354 | "execution_count": null, 355 | "metadata": {}, 356 | "outputs": [], 357 | "source": [ 358 | "a[a<3] # This internally creates a boolean array like the b array above!" 359 | ] 360 | }, 361 | { 362 | "cell_type": "code", 363 | "execution_count": null, 364 | "metadata": {}, 365 | "outputs": [], 366 | "source": [] 367 | } 368 | ], 369 | "metadata": { 370 | "kernelspec": { 371 | "display_name": "Python 3", 372 | "language": "python", 373 | "name": "python3" 374 | }, 375 | "language_info": { 376 | "codemirror_mode": { 377 | "name": "ipython", 378 | "version": 3 379 | }, 380 | "file_extension": ".py", 381 | "mimetype": "text/x-python", 382 | "name": "python", 383 | "nbconvert_exporter": "python", 384 | "pygments_lexer": "ipython3", 385 | "version": "3.6.6" 386 | } 387 | }, 388 | "nbformat": 4, 389 | "nbformat_minor": 2 390 | } 391 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Attribution-ShareAlike 4.0 International 2 | 3 | ======================================================================= 4 | 5 | Creative Commons Corporation ("Creative Commons") is not a law firm and 6 | does not provide legal services or legal advice. Distribution of 7 | Creative Commons public licenses does not create a lawyer-client or 8 | other relationship. Creative Commons makes its licenses and related 9 | information available on an "as-is" basis. Creative Commons gives no 10 | warranties regarding its licenses, any material licensed under their 11 | terms and conditions, or any related information. Creative Commons 12 | disclaims all liability for damages resulting from their use to the 13 | fullest extent possible. 14 | 15 | Using Creative Commons Public Licenses 16 | 17 | Creative Commons public licenses provide a standard set of terms and 18 | conditions that creators and other rights holders may use to share 19 | original works of authorship and other material subject to copyright 20 | and certain other rights specified in the public license below. The 21 | following considerations are for informational purposes only, are not 22 | exhaustive, and do not form part of our licenses. 23 | 24 | Considerations for licensors: Our public licenses are 25 | intended for use by those authorized to give the public 26 | permission to use material in ways otherwise restricted by 27 | copyright and certain other rights. Our licenses are 28 | irrevocable. Licensors should read and understand the terms 29 | and conditions of the license they choose before applying it. 30 | Licensors should also secure all rights necessary before 31 | applying our licenses so that the public can reuse the 32 | material as expected. Licensors should clearly mark any 33 | material not subject to the license. This includes other CC- 34 | licensed material, or material used under an exception or 35 | limitation to copyright. More considerations for licensors: 36 | wiki.creativecommons.org/Considerations_for_licensors 37 | 38 | Considerations for the public: By using one of our public 39 | licenses, a licensor grants the public permission to use the 40 | licensed material under specified terms and conditions. If 41 | the licensor's permission is not necessary for any reason--for 42 | example, because of any applicable exception or limitation to 43 | copyright--then that use is not regulated by the license. Our 44 | licenses grant only permissions under copyright and certain 45 | other rights that a licensor has authority to grant. Use of 46 | the licensed material may still be restricted for other 47 | reasons, including because others have copyright or other 48 | rights in the material. A licensor may make special requests, 49 | such as asking that all changes be marked or described. 50 | Although not required by our licenses, you are encouraged to 51 | respect those requests where reasonable. More_considerations 52 | for the public: 53 | wiki.creativecommons.org/Considerations_for_licensees 54 | 55 | ======================================================================= 56 | 57 | Creative Commons Attribution-ShareAlike 4.0 International Public 58 | License 59 | 60 | By exercising the Licensed Rights (defined below), You accept and agree 61 | to be bound by the terms and conditions of this Creative Commons 62 | Attribution-ShareAlike 4.0 International Public License ("Public 63 | License"). To the extent this Public License may be interpreted as a 64 | contract, You are granted the Licensed Rights in consideration of Your 65 | acceptance of these terms and conditions, and the Licensor grants You 66 | such rights in consideration of benefits the Licensor receives from 67 | making the Licensed Material available under these terms and 68 | conditions. 69 | 70 | 71 | Section 1 -- Definitions. 72 | 73 | a. Adapted Material means material subject to Copyright and Similar 74 | Rights that is derived from or based upon the Licensed Material 75 | and in which the Licensed Material is translated, altered, 76 | arranged, transformed, or otherwise modified in a manner requiring 77 | permission under the Copyright and Similar Rights held by the 78 | Licensor. For purposes of this Public License, where the Licensed 79 | Material is a musical work, performance, or sound recording, 80 | Adapted Material is always produced where the Licensed Material is 81 | synched in timed relation with a moving image. 82 | 83 | b. Adapter's License means the license You apply to Your Copyright 84 | and Similar Rights in Your contributions to Adapted Material in 85 | accordance with the terms and conditions of this Public License. 86 | 87 | c. BY-SA Compatible License means a license listed at 88 | creativecommons.org/compatiblelicenses, approved by Creative 89 | Commons as essentially the equivalent of this Public License. 90 | 91 | d. Copyright and Similar Rights means copyright and/or similar rights 92 | closely related to copyright including, without limitation, 93 | performance, broadcast, sound recording, and Sui Generis Database 94 | Rights, without regard to how the rights are labeled or 95 | categorized. For purposes of this Public License, the rights 96 | specified in Section 2(b)(1)-(2) are not Copyright and Similar 97 | Rights. 98 | 99 | e. Effective Technological Measures means those measures that, in the 100 | absence of proper authority, may not be circumvented under laws 101 | fulfilling obligations under Article 11 of the WIPO Copyright 102 | Treaty adopted on December 20, 1996, and/or similar international 103 | agreements. 104 | 105 | f. Exceptions and Limitations means fair use, fair dealing, and/or 106 | any other exception or limitation to Copyright and Similar Rights 107 | that applies to Your use of the Licensed Material. 108 | 109 | g. License Elements means the license attributes listed in the name 110 | of a Creative Commons Public License. The License Elements of this 111 | Public License are Attribution and ShareAlike. 112 | 113 | h. Licensed Material means the artistic or literary work, database, 114 | or other material to which the Licensor applied this Public 115 | License. 116 | 117 | i. Licensed Rights means the rights granted to You subject to the 118 | terms and conditions of this Public License, which are limited to 119 | all Copyright and Similar Rights that apply to Your use of the 120 | Licensed Material and that the Licensor has authority to license. 121 | 122 | j. Licensor means the individual(s) or entity(ies) granting rights 123 | under this Public License. 124 | 125 | k. Share means to provide material to the public by any means or 126 | process that requires permission under the Licensed Rights, such 127 | as reproduction, public display, public performance, distribution, 128 | dissemination, communication, or importation, and to make material 129 | available to the public including in ways that members of the 130 | public may access the material from a place and at a time 131 | individually chosen by them. 132 | 133 | l. Sui Generis Database Rights means rights other than copyright 134 | resulting from Directive 96/9/EC of the European Parliament and of 135 | the Council of 11 March 1996 on the legal protection of databases, 136 | as amended and/or succeeded, as well as other essentially 137 | equivalent rights anywhere in the world. 138 | 139 | m. You means the individual or entity exercising the Licensed Rights 140 | under this Public License. Your has a corresponding meaning. 141 | 142 | 143 | Section 2 -- Scope. 144 | 145 | a. License grant. 146 | 147 | 1. Subject to the terms and conditions of this Public License, 148 | the Licensor hereby grants You a worldwide, royalty-free, 149 | non-sublicensable, non-exclusive, irrevocable license to 150 | exercise the Licensed Rights in the Licensed Material to: 151 | 152 | a. reproduce and Share the Licensed Material, in whole or 153 | in part; and 154 | 155 | b. produce, reproduce, and Share Adapted Material. 156 | 157 | 2. Exceptions and Limitations. For the avoidance of doubt, where 158 | Exceptions and Limitations apply to Your use, this Public 159 | License does not apply, and You do not need to comply with 160 | its terms and conditions. 161 | 162 | 3. Term. The term of this Public License is specified in Section 163 | 6(a). 164 | 165 | 4. Media and formats; technical modifications allowed. The 166 | Licensor authorizes You to exercise the Licensed Rights in 167 | all media and formats whether now known or hereafter created, 168 | and to make technical modifications necessary to do so. The 169 | Licensor waives and/or agrees not to assert any right or 170 | authority to forbid You from making technical modifications 171 | necessary to exercise the Licensed Rights, including 172 | technical modifications necessary to circumvent Effective 173 | Technological Measures. For purposes of this Public License, 174 | simply making modifications authorized by this Section 2(a) 175 | (4) never produces Adapted Material. 176 | 177 | 5. Downstream recipients. 178 | 179 | a. Offer from the Licensor -- Licensed Material. Every 180 | recipient of the Licensed Material automatically 181 | receives an offer from the Licensor to exercise the 182 | Licensed Rights under the terms and conditions of this 183 | Public License. 184 | 185 | b. Additional offer from the Licensor -- Adapted Material. 186 | Every recipient of Adapted Material from You 187 | automatically receives an offer from the Licensor to 188 | exercise the Licensed Rights in the Adapted Material 189 | under the conditions of the Adapter's License You apply. 190 | 191 | c. No downstream restrictions. You may not offer or impose 192 | any additional or different terms or conditions on, or 193 | apply any Effective Technological Measures to, the 194 | Licensed Material if doing so restricts exercise of the 195 | Licensed Rights by any recipient of the Licensed 196 | Material. 197 | 198 | 6. No endorsement. Nothing in this Public License constitutes or 199 | may be construed as permission to assert or imply that You 200 | are, or that Your use of the Licensed Material is, connected 201 | with, or sponsored, endorsed, or granted official status by, 202 | the Licensor or others designated to receive attribution as 203 | provided in Section 3(a)(1)(A)(i). 204 | 205 | b. Other rights. 206 | 207 | 1. Moral rights, such as the right of integrity, are not 208 | licensed under this Public License, nor are publicity, 209 | privacy, and/or other similar personality rights; however, to 210 | the extent possible, the Licensor waives and/or agrees not to 211 | assert any such rights held by the Licensor to the limited 212 | extent necessary to allow You to exercise the Licensed 213 | Rights, but not otherwise. 214 | 215 | 2. Patent and trademark rights are not licensed under this 216 | Public License. 217 | 218 | 3. To the extent possible, the Licensor waives any right to 219 | collect royalties from You for the exercise of the Licensed 220 | Rights, whether directly or through a collecting society 221 | under any voluntary or waivable statutory or compulsory 222 | licensing scheme. In all other cases the Licensor expressly 223 | reserves any right to collect such royalties. 224 | 225 | 226 | Section 3 -- License Conditions. 227 | 228 | Your exercise of the Licensed Rights is expressly made subject to the 229 | following conditions. 230 | 231 | a. Attribution. 232 | 233 | 1. If You Share the Licensed Material (including in modified 234 | form), You must: 235 | 236 | a. retain the following if it is supplied by the Licensor 237 | with the Licensed Material: 238 | 239 | i. identification of the creator(s) of the Licensed 240 | Material and any others designated to receive 241 | attribution, in any reasonable manner requested by 242 | the Licensor (including by pseudonym if 243 | designated); 244 | 245 | ii. a copyright notice; 246 | 247 | iii. a notice that refers to this Public License; 248 | 249 | iv. a notice that refers to the disclaimer of 250 | warranties; 251 | 252 | v. a URI or hyperlink to the Licensed Material to the 253 | extent reasonably practicable; 254 | 255 | b. indicate if You modified the Licensed Material and 256 | retain an indication of any previous modifications; and 257 | 258 | c. indicate the Licensed Material is licensed under this 259 | Public License, and include the text of, or the URI or 260 | hyperlink to, this Public License. 261 | 262 | 2. You may satisfy the conditions in Section 3(a)(1) in any 263 | reasonable manner based on the medium, means, and context in 264 | which You Share the Licensed Material. For example, it may be 265 | reasonable to satisfy the conditions by providing a URI or 266 | hyperlink to a resource that includes the required 267 | information. 268 | 269 | 3. If requested by the Licensor, You must remove any of the 270 | information required by Section 3(a)(1)(A) to the extent 271 | reasonably practicable. 272 | 273 | b. ShareAlike. 274 | 275 | In addition to the conditions in Section 3(a), if You Share 276 | Adapted Material You produce, the following conditions also apply. 277 | 278 | 1. The Adapter's License You apply must be a Creative Commons 279 | license with the same License Elements, this version or 280 | later, or a BY-SA Compatible License. 281 | 282 | 2. You must include the text of, or the URI or hyperlink to, the 283 | Adapter's License You apply. You may satisfy this condition 284 | in any reasonable manner based on the medium, means, and 285 | context in which You Share Adapted Material. 286 | 287 | 3. You may not offer or impose any additional or different terms 288 | or conditions on, or apply any Effective Technological 289 | Measures to, Adapted Material that restrict exercise of the 290 | rights granted under the Adapter's License You apply. 291 | 292 | 293 | Section 4 -- Sui Generis Database Rights. 294 | 295 | Where the Licensed Rights include Sui Generis Database Rights that 296 | apply to Your use of the Licensed Material: 297 | 298 | a. for the avoidance of doubt, Section 2(a)(1) grants You the right 299 | to extract, reuse, reproduce, and Share all or a substantial 300 | portion of the contents of the database; 301 | 302 | b. if You include all or a substantial portion of the database 303 | contents in a database in which You have Sui Generis Database 304 | Rights, then the database in which You have Sui Generis Database 305 | Rights (but not its individual contents) is Adapted Material, 306 | 307 | including for purposes of Section 3(b); and 308 | c. You must comply with the conditions in Section 3(a) if You Share 309 | all or a substantial portion of the contents of the database. 310 | 311 | For the avoidance of doubt, this Section 4 supplements and does not 312 | replace Your obligations under this Public License where the Licensed 313 | Rights include other Copyright and Similar Rights. 314 | 315 | 316 | Section 5 -- Disclaimer of Warranties and Limitation of Liability. 317 | 318 | a. UNLESS OTHERWISE SEPARATELY UNDERTAKEN BY THE LICENSOR, TO THE 319 | EXTENT POSSIBLE, THE LICENSOR OFFERS THE LICENSED MATERIAL AS-IS 320 | AND AS-AVAILABLE, AND MAKES NO REPRESENTATIONS OR WARRANTIES OF 321 | ANY KIND CONCERNING THE LICENSED MATERIAL, WHETHER EXPRESS, 322 | IMPLIED, STATUTORY, OR OTHER. THIS INCLUDES, WITHOUT LIMITATION, 323 | WARRANTIES OF TITLE, MERCHANTABILITY, FITNESS FOR A PARTICULAR 324 | PURPOSE, NON-INFRINGEMENT, ABSENCE OF LATENT OR OTHER DEFECTS, 325 | ACCURACY, OR THE PRESENCE OR ABSENCE OF ERRORS, WHETHER OR NOT 326 | KNOWN OR DISCOVERABLE. WHERE DISCLAIMERS OF WARRANTIES ARE NOT 327 | ALLOWED IN FULL OR IN PART, THIS DISCLAIMER MAY NOT APPLY TO YOU. 328 | 329 | b. TO THE EXTENT POSSIBLE, IN NO EVENT WILL THE LICENSOR BE LIABLE 330 | TO YOU ON ANY LEGAL THEORY (INCLUDING, WITHOUT LIMITATION, 331 | NEGLIGENCE) OR OTHERWISE FOR ANY DIRECT, SPECIAL, INDIRECT, 332 | INCIDENTAL, CONSEQUENTIAL, PUNITIVE, EXEMPLARY, OR OTHER LOSSES, 333 | COSTS, EXPENSES, OR DAMAGES ARISING OUT OF THIS PUBLIC LICENSE OR 334 | USE OF THE LICENSED MATERIAL, EVEN IF THE LICENSOR HAS BEEN 335 | ADVISED OF THE POSSIBILITY OF SUCH LOSSES, COSTS, EXPENSES, OR 336 | DAMAGES. WHERE A LIMITATION OF LIABILITY IS NOT ALLOWED IN FULL OR 337 | IN PART, THIS LIMITATION MAY NOT APPLY TO YOU. 338 | 339 | c. The disclaimer of warranties and limitation of liability provided 340 | above shall be interpreted in a manner that, to the extent 341 | possible, most closely approximates an absolute disclaimer and 342 | waiver of all liability. 343 | 344 | 345 | Section 6 -- Term and Termination. 346 | 347 | a. This Public License applies for the term of the Copyright and 348 | Similar Rights licensed here. However, if You fail to comply with 349 | this Public License, then Your rights under this Public License 350 | terminate automatically. 351 | 352 | b. Where Your right to use the Licensed Material has terminated under 353 | Section 6(a), it reinstates: 354 | 355 | 1. automatically as of the date the violation is cured, provided 356 | it is cured within 30 days of Your discovery of the 357 | violation; or 358 | 359 | 2. upon express reinstatement by the Licensor. 360 | 361 | For the avoidance of doubt, this Section 6(b) does not affect any 362 | right the Licensor may have to seek remedies for Your violations 363 | of this Public License. 364 | 365 | c. For the avoidance of doubt, the Licensor may also offer the 366 | Licensed Material under separate terms or conditions or stop 367 | distributing the Licensed Material at any time; however, doing so 368 | will not terminate this Public License. 369 | 370 | d. Sections 1, 5, 6, 7, and 8 survive termination of this Public 371 | License. 372 | 373 | 374 | Section 7 -- Other Terms and Conditions. 375 | 376 | a. The Licensor shall not be bound by any additional or different 377 | terms or conditions communicated by You unless expressly agreed. 378 | 379 | b. Any arrangements, understandings, or agreements regarding the 380 | Licensed Material not stated herein are separate from and 381 | independent of the terms and conditions of this Public License. 382 | 383 | 384 | Section 8 -- Interpretation. 385 | 386 | a. For the avoidance of doubt, this Public License does not, and 387 | shall not be interpreted to, reduce, limit, restrict, or impose 388 | conditions on any use of the Licensed Material that could lawfully 389 | be made without permission under this Public License. 390 | 391 | b. To the extent possible, if any provision of this Public License is 392 | deemed unenforceable, it shall be automatically reformed to the 393 | minimum extent necessary to make it enforceable. If the provision 394 | cannot be reformed, it shall be severed from this Public License 395 | without affecting the enforceability of the remaining terms and 396 | conditions. 397 | 398 | c. No term or condition of this Public License will be waived and no 399 | failure to comply consented to unless expressly agreed to by the 400 | Licensor. 401 | 402 | d. Nothing in this Public License constitutes or may be interpreted 403 | as a limitation upon, or waiver of, any privileges and immunities 404 | that apply to the Licensor or You, including from the legal 405 | processes of any jurisdiction or authority. 406 | 407 | 408 | ======================================================================= 409 | 410 | Creative Commons is not a party to its public 411 | licenses. Notwithstanding, Creative Commons may elect to apply one of 412 | its public licenses to material it publishes and in those instances 413 | will be considered the “Licensor.” The text of the Creative Commons 414 | public licenses is dedicated to the public domain under the CC0 Public 415 | Domain Dedication. Except for the limited purpose of indicating that 416 | material is shared under a Creative Commons public license or as 417 | otherwise permitted by the Creative Commons policies published at 418 | creativecommons.org/policies, Creative Commons does not authorize the 419 | use of the trademark "Creative Commons" or any other trademark or logo 420 | of Creative Commons without its prior written consent including, 421 | without limitation, in connection with any unauthorized modifications 422 | to any of its public licenses or any other arrangements, 423 | understandings, or agreements concerning use of licensed material. For 424 | the avoidance of doubt, this paragraph does not form part of the 425 | public licenses. 426 | 427 | Creative Commons may be contacted at creativecommons.org. 428 | 429 | © 2020 GitHub, Inc. 430 | Terms 431 | Privacy 432 | Security 433 | Status 434 | Help 435 | 436 | Contact GitHub 437 | Pricing 438 | API 439 | Training 440 | Blog 441 | About 442 | 443 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Working with Climate Data in Python 2 | A recipe book on how to deal with climate data in a progammatical way: how to work with python, the climatological datasets of ECMWF and how to generate different plots out of it. 3 | 4 | Shield: [![CC BY-SA 4.0][cc-by-sa-shield]][cc-by-sa] 5 | 6 | This work is licensed under a [Creative Commons Attribution-ShareAlike 4.0 7 | International License][cc-by-sa]. 8 | 9 | [![CC BY-SA 4.0][cc-by-sa-image]][cc-by-sa] 10 | 11 | [cc-by-sa]: http://creativecommons.org/licenses/by-sa/4.0/ 12 | [cc-by-sa-image]: https://licensebuttons.net/l/by-sa/4.0/88x31.png 13 | [cc-by-sa-shield]: https://img.shields.io/badge/License-CC%20BY--SA%204.0-lightgrey.svg 14 | -------------------------------------------------------------------------------- /img/3D_netCDF.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MarieHofmann/Working-with-Climate-Data-in-Python/50d966beb8086c09a3133b643dfd6169f637e222/img/3D_netCDF.gif -------------------------------------------------------------------------------- /img/dataset-diagram.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MarieHofmann/Working-with-Climate-Data-in-Python/50d966beb8086c09a3133b643dfd6169f637e222/img/dataset-diagram.png -------------------------------------------------------------------------------- /img/hierarchy1.JPG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MarieHofmann/Working-with-Climate-Data-in-Python/50d966beb8086c09a3133b643dfd6169f637e222/img/hierarchy1.JPG -------------------------------------------------------------------------------- /img/netCDF.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MarieHofmann/Working-with-Climate-Data-in-Python/50d966beb8086c09a3133b643dfd6169f637e222/img/netCDF.gif -------------------------------------------------------------------------------- /img/xarraylogo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MarieHofmann/Working-with-Climate-Data-in-Python/50d966beb8086c09a3133b643dfd6169f637e222/img/xarraylogo.png --------------------------------------------------------------------------------