├── README.md ├── _config.yml ├── images ├── jupyter_dashboard.png └── navigator.png └── notebooks ├── 01_test.ipynb ├── 02_resources.ipynb ├── 03_Jupyter_notebook.ipynb ├── 04_Introduction_Python.ipynb ├── 05_Numpy.ipynb ├── 06_Scipy.ipynb ├── 07_Pandas.ipynb ├── 08_xarray.ipynb ├── 09_Plotting.ipynb ├── 10_Mapping.ipynb ├── data ├── AKL_aero_rain_monthly.xlsx ├── Daily_clim_data_Auckland.csv ├── ISO_datetime.xls ├── NICO_NIWA_SOI_1941_2010.csv ├── OULOUP_PMER.csv ├── Wellington_rain.xlsx ├── rainfall_calibration.xlsx ├── soi_nino.csv └── year_month_day.xlsx ├── images ├── Vanderplas_Python_Stack.png ├── fig_map.png ├── ipython_console.png ├── pandas_freqs.png ├── presentation.png ├── python_console.png ├── qtconsole.png └── split-apply-combine.png └── recipes ├── MJO_composite_plot.ipynb └── MJO_phase_plot.ipynb /README.md: -------------------------------------------------------------------------------- 1 | # Python for Climate and Meteorological Data Analysis and Visualisation 2 | 3 | [NIWA Auckland](http://www.niwa.co.nz/) / Climate and Weather Applications, Forecasting Services 4 | Auckland, New Zealand, 31 August and 1 September 2017 5 | 6 | Contact: 7 | 8 | Nicolas Fauchereau 9 | 10 | + [@gmail](mailto:Nicolas.Fauchereau@gmail.com) 11 | + [@niwa](mailto:Nicolas.Fauchereau@niwa.co.nz) 12 | 13 | 14 | 15 | 16 | ### Table of contents 17 | 18 | - [The Anaconda python distribution](#the-anaconda-python-distribution) 19 | - [Installation of Some additional libraries](#installation-of-additional-libraries) 20 | - [Running the Jupyter notebooks](#running-the-jupyter-notebooks) 21 | - [Content of the workshop](#contents) 22 | 23 | ## The Anaconda python distribution 24 | 25 | For this tutorial, I **strongly** recommend installing the **Anaconda Python distribution**. It is a completely free *enterprise-ready* Python distribution for large-scale data processing, predictive analytics, and scientific computing. It includes the python interpreter itself, the python standard library as well as a set of packages exposing data structures and methods for data manipulation and scientific computing and visualization. In particular it provides [Numpy](http://www.numpy.org/), [Scipy](http://www.scipy.org/), [Pandas](http://pandas.pydata.org/), [Matplotlib](http://matplotlib.org/), etc ... i.e. all the main packages we will be using during the tutorial. The full list of packages is available at: 26 | 27 | [http://docs.continuum.io/anaconda/pkgs.html](http://docs.continuum.io/anaconda/pkgs.html) 28 | 29 | The Anaconda python distribution (**NOTE**: select the version shipping with Python 3.6) must be downloaded from: 30 | 31 | [http://continuum.io/downloads](http://continuum.io/downloads) 32 | 33 | For your platform. 34 | 35 | You should not need administrator rights, as Anaconda is completely self-contained and can be installed in your `HOME` directory. I suggest installing `Anaconda` in `/Users/USERNAME/anaconda` on Macs, and `C:\Users\USERNAME\anaconda` on Windows 36 | 37 | Once you have installed Anaconda, you can update to the latest compatible versions of all the pre-installed packages by running, at the command line, i.e. in Windows select `Cmd.exe`, on Mac the `Terminal` application in `Utilities`, the `$` sign signifies the command prompt, so do not enter it !, also when prompted `do you want to proceed?` just enter yes (`y`) : 38 | 39 | ``` 40 | $ conda update conda 41 | ``` 42 | 43 | Then 44 | 45 | ``` 46 | $ conda update anaconda 47 | ``` 48 | 49 | You also might want to install [pip](https://github.com/pypa/pip) to install packages from the [Python Package Index](http://pypi.python.org/pypi). 50 | 51 | ``` 52 | $ conda install pip 53 | ``` 54 | 55 | ## Installation of additional libraries 56 | 57 | ### netcdf4 58 | 59 | [netcdf4](https://github.com/Unidata/netcdf4-python) allows you to read and write netcdf files (version 3 and 4 supported), install it by: 60 | 61 | ``` 62 | $ conda install netcdf4 63 | ``` 64 | 65 | ### Basemap 66 | 67 | **Basemap** is a graphic library for plotting (static, publication quality) geographical maps (see [http://matplotlib.org/basemap/](http://matplotlib.org/basemap/)). **Basemap** is available directly in **Anaconda** for Mac and Linux using the conda package manager, install with : 68 | 69 | ``` 70 | $ conda install basemap 71 | ``` 72 | 73 | but on Windows, you'll have to use an alternative channel (conda-forge), the command becomes: 74 | 75 | ``` 76 | $ conda install -c conda-forge basemap 77 | ``` 78 | 79 | ### Cartopy 80 | 81 | [**Cartopy**](http://scitools.org.uk/cartopy/) is a the second library for making geographical maps that we are going to see. It has been developed by the UKMO, and will eventually replace **Basemap**. However to this date Cartopy does not have all the features present in Basemap. 82 | 83 | Cartopy is not available through the anaconda standard channel, to install you need to use the community maintained *conda-forge* channel, with this syntax: 84 | 85 | ``` 86 | $ conda install -c conda-forge cartopy 87 | ``` 88 | 89 | ### Seaborn 90 | 91 | [seaborn](http://web.stanford.edu/~mwaskom/software/seaborn/) is a Python visualization library based on matplotlib. It provides a high-level interface for drawing attractive statistical graphics. You should be able to install it with ```conda``` as well: 92 | 93 | ``` 94 | $ conda install seaborn 95 | ``` 96 | ### xarray 97 | 98 | [xarray](https://github.com/xarray/xarray) (previously *xray*) is a library aimed at bringing the power of Pandas to multidimensional labelled arrays, such as the ones usually associated with geophysical quantities varying along time and space dimensions (e.g. [time, latitudes, longitudes], [time, level, latitudes, longitudes], etc) and supports reading and writing netcdf files. It can be installed via `conda`: 99 | 100 | ``` 101 | $ conda install xarray 102 | ``` 103 | 104 | All of this can also be done using the Anaconda Navigator GUI (Graphical User Interface) which looks like below 105 | 106 | ![](https://github.com/nicolasfauchereau/Auckland_Python_Workshop/raw/master/images/navigator.png) 107 | 108 | If you go the `Environments` tab on the left, and select the `root` environment (we will talk about the concept of Python environments later in the workshop) you can then search, select and install available packages, including the ones mentionned above. 109 | 110 | 111 | ## Running the Jupyter notebook 112 | 113 | The material of the tutorial is in the form of [Jupyter notebooks](http://jupyter-notebook-beginner-guide.readthedocs.io/en/latest/what_is_jupyter.html). In a nutshell a Jupyter notebook is a web-based (i.e. running in the browser) interactive computational environment where you can combine Python code execution, text, rendered mathematic expression, plots, images, and rich media into a **single document**. 114 | 115 | It is in my opinion the best environment when you are doing **exploratory data analysis**, and it allows you to weave comments, background information, interpretations and references along with the code itself, and it can be exported to a variety of formats (HTML, pdf) for sharing with colleagues. I **strongly** recommend spending some time learning about the notebook and its many features. 116 | 117 | After uncompressing the archive of the repo (or after cloning it with ```git```), navigate to the `notebooks` directory (containing the ```*.ipynb``` files) and type: 118 | 119 | ``` 120 | $ jupyter notebook 121 | ``` 122 | 123 | That should bring up the Jupyter notebook dashboard (looking as below) in your default Browser, you should be ready to go ! 124 | 125 | ![](https://github.com/nicolasfauchereau/Auckland_Python_Workshop/raw/master/images/jupyter_dashboard.png) 126 | 127 | ## Contents 128 | 129 | The following is a brief description of the workshop material content: 130 | 131 | 132 | 133 | + `01_test.ipynb`: 134 | 135 | A simple test Jupyter notebook to test the installation of the main libraries and their version 136 | 137 | + `02_resources.ipynb`: 138 | 139 | Some links to interesting resources and topics not covered during the workshop 140 | 141 | + `03_Jupyter_notebook.ipynb` 142 | 143 | Introduction to the main features of the **Jupyter notebook** 144 | 145 | + `04_introduction_Python.ipynb`: Introduction the **basics** of the python language: 146 | 147 | + what's an *interpreted programming language*, what is the python *interpreter* 148 | 149 | + how to write a python script 150 | 151 | + basic Python native data types: 152 | + dealing with numbers in native Python 153 | + dealing with `strings` and their formatting 154 | + `lists`, `tuples`, `dictionnaries`, ... 155 | 156 | + arithmetic operators (`+. -, *, =`) 157 | 158 | + logical operators (`==, >=, <=, !=`) and how they relate to booleans (`True / False`) 159 | 160 | + control flow structures (`for, if / elif / else, while` etc.) 161 | 162 | + reusing your code: writing *functions*, *modules* and *packages* 163 | 164 | + `05_Numpy.ipynb`: 165 | 166 | A introduction to Numpy: Numpy introduces in particular a data structure called the `ndarray` (*N-dimensional* array) which can store numerical values. It is the foundational library of the Python *scientific ecosystem*. In this notebook we'll learn the main principles of manipulating numpy *ndarrays*. While you might actually not spend much time working with numpy itself, it is necessary to understand its basic principles, as (almost) everything else in scientific Python is built on top of `numpy`. We'll see: 167 | 168 | + How to create numpy arrays 169 | 170 | + indexing, slicing, reshaping, transposing, etc. 171 | 172 | + The main methods and functions in numpy operating on numpy `ndarrays` 173 | 174 | + `06_Scipy.ipynb`: 175 | 176 | [Scipy](http://www.scipy.org) is the second pilar of the python scientific ecosystem. Where Numpy introduces a *data structure* (the ndarray), Scipy provides a collection of efficient *scientific algorithms*. These are organized in *submodules*, and cover topics ranging from linear algebra to signal and image processing and optimisation (see [here](https://docs.scipy.org/doc/scipy/reference/) for list of the submodules and their dedicated tutorial). In this notebook we will focus on **interpolation**, and on some of the **statistical** algorithms and methods that scipy makes available. 177 | 178 | + `07_Pandas.ipynb` 179 | 180 | This is where we're gonna spend quite a bit of time ! [Pandas](http://pandas.pydata.org/) is **THE** library that you need to use when dealing with *tabular* data, *i.e.* "spreadsheet-like" data, where values are stored in 2D arrays with rows and column labels. It is typically the type of data you find in csv, tab / space delimited or Excel files. In this notebook we'll see first: 181 | 182 | + The basics of the main data structures in Pandas: the **Series** and the **Dataframe** 183 | + How to read from / write to common data files (csv, excel, space delimited, tab delimited etc). It will include how to read from a list of e.g. csv files, and concatenating their data in a Dataframe 184 | + How to manipulate tabular data in Pandas, from selecting rows / columns to more sophisticated conditional indexing 185 | + How to perform complex queries and assignements on Pandas Dataframes 186 | + How to perform **groupby** operations (*split / apply / combine*) 187 | + How to deal with Missing Values 188 | 189 | One of the strenths of Pandas is its ability to store, and perform sophisticated operations on, **time-series**: i.e. data that is indexed by dates, dates and times, timestamps, etc. In the second part of the Pandas tutorial we will focus on **time-series manipulation in Pandas**. In particular we'll see: 190 | 191 | + how to read in and correctly parse files containing date / time information 192 | 193 | + how to resample time-series 194 | 195 | + rolling window operations (e.g. moving averages) 196 | 197 | + how to deal with missing values and missing index values 198 | 199 | + `08_xarray.ipynb`: 200 | 201 | [xarray](http://xarray.pydata.org/en/stable/) is a library to read / write netcdf files and for the manipulation of multi-dimensional labelled arrays, it is especially handy for *gridded data* varying along *latitudes, longitudes, time, depth, height*, etc. Its design follows closely that of Pandas, meaning that familiarity with Pandas allows to quickly pick up xarray. We'll see how to read, write netcdf files in xarray, and perform a series of common analyses such as: 202 | 203 | + calculating a climatology 204 | + calculating anomalies 205 | + going from and to Pandas Dataframes 206 | + calculating composite anomalies 207 | + resampling, aggregation, groupby operations 208 | 209 | + `09_plotting.ipynb`: 210 | 211 | In this notebook we'll go over the basics of [**Matplotlib**](https://matplotlib.org/), the main plotting library in python. This is more or less the equivalent of *numpy* but for plotting: *i.e.* a foundational library of the Python scientific ecosystem, on wich a number of — more specialised — plotting libraries have been built. One of these we will briefly go over in particular is [**seaborn**](https://seaborn.pydata.org/), a plotting library handy for statistical plots. A short and non-exhaustive list of plotting libraries in Python is provided covering both static plots (i.e. *plotnine*, *ggplot2*, etc) and interactive plots (*bokeh*, *plotly*, *holoviews*, etc) 212 | 213 | + `10_Mapping.ipynb`: 214 | 215 | Making maps in Python: a brief overview of [**basemap**](https://matplotlib.org/basemap/), [**cartopy**](http://scitools.org.uk/cartopy/), and — very briefly, for interactive maps — [**folium**](https://github.com/python-visualization/folium). We'll see also briefly how to read shapefiles (with [geopandas](http://geopandas.org/)), transform into a [geojson](http://geojson.org/) and edit in the browser. 216 | 217 | ​ 218 | -------------------------------------------------------------------------------- /_config.yml: -------------------------------------------------------------------------------- 1 | theme: jekyll-theme-cayman -------------------------------------------------------------------------------- /images/jupyter_dashboard.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nicolasfauchereau/Auckland_Python_Workshop/33365e02449818dd7feadc3b20aeb487ed0624cd/images/jupyter_dashboard.png -------------------------------------------------------------------------------- /images/navigator.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nicolasfauchereau/Auckland_Python_Workshop/33365e02449818dd7feadc3b20aeb487ed0624cd/images/navigator.png -------------------------------------------------------------------------------- /notebooks/01_test.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# This is a test Jupyter Notebook\n", 8 | "\n", 9 | "It imports and display the version number of the foundational libraries we will be using \n", 10 | "\n", 11 | "to run a ```IPython notebook cell``` just click on it (should be highligthed in gray or green) and either enter ```shift-enter``` or use the ```play``` button in the menu\n", 12 | "\n", 13 | "If one of the import fails please come back to me at [Nicolas.Fauchereau@gmail.com](mailto:nicolas.fauchereau@gmail.com), the last library (basemap) is only \n", 14 | "required for making geographical maps, so it might not be essential to all of you. \n", 15 | "\n", 16 | "if you have version number significantly lower than the following, you might \n", 17 | "also run into minor problems during the lectures: \n", 18 | "\n", 19 | "+ **Numpy**: version 1.11\n", 20 | "+ **scipy**: version 0.19\n", 21 | "+ **pandas**: version 0.20\n", 22 | "+ **xarray**: version 0.9\n", 23 | "+ **matplotlib**: version 2.0\n", 24 | "+ **basemap**: version 0.8\n", 25 | "+ **cartopy**: version 0.14\n" 26 | ] 27 | }, 28 | { 29 | "cell_type": "code", 30 | "execution_count": null, 31 | "metadata": {}, 32 | "outputs": [], 33 | "source": [ 34 | "import numpy as np; print(np.__version__)" 35 | ] 36 | }, 37 | { 38 | "cell_type": "code", 39 | "execution_count": null, 40 | "metadata": {}, 41 | "outputs": [], 42 | "source": [ 43 | "import scipy as sp; print(sp.__version__)" 44 | ] 45 | }, 46 | { 47 | "cell_type": "code", 48 | "execution_count": null, 49 | "metadata": {}, 50 | "outputs": [], 51 | "source": [ 52 | "import pandas as pd; print(pd.__version__)" 53 | ] 54 | }, 55 | { 56 | "cell_type": "code", 57 | "execution_count": null, 58 | "metadata": {}, 59 | "outputs": [], 60 | "source": [ 61 | "import xarray as xr; print(xr.__version__)" 62 | ] 63 | }, 64 | { 65 | "cell_type": "code", 66 | "execution_count": null, 67 | "metadata": {}, 68 | "outputs": [], 69 | "source": [ 70 | "import matplotlib as mpl; print(mpl.__version__)" 71 | ] 72 | }, 73 | { 74 | "cell_type": "code", 75 | "execution_count": null, 76 | "metadata": {}, 77 | "outputs": [], 78 | "source": [ 79 | "from mpl_toolkits import basemap; print(basemap.__version__)" 80 | ] 81 | }, 82 | { 83 | "cell_type": "code", 84 | "execution_count": null, 85 | "metadata": {}, 86 | "outputs": [], 87 | "source": [ 88 | "import cartopy; print(cartopy.__version__)" 89 | ] 90 | }, 91 | { 92 | "cell_type": "markdown", 93 | "metadata": {}, 94 | "source": [ 95 | "The following command calls matplotlib with the `inline` backend, which allows to display plots inline inside the notebook, if this command fails or a plot does not appear, please come back to me " 96 | ] 97 | }, 98 | { 99 | "cell_type": "code", 100 | "execution_count": null, 101 | "metadata": { 102 | "collapsed": true 103 | }, 104 | "outputs": [], 105 | "source": [ 106 | "%matplotlib inline" 107 | ] 108 | }, 109 | { 110 | "cell_type": "code", 111 | "execution_count": null, 112 | "metadata": { 113 | "collapsed": true 114 | }, 115 | "outputs": [], 116 | "source": [ 117 | "import matplotlib.pyplot as plt" 118 | ] 119 | }, 120 | { 121 | "cell_type": "code", 122 | "execution_count": null, 123 | "metadata": {}, 124 | "outputs": [], 125 | "source": [ 126 | "plt.plot(np.sin(np.linspace(-np.pi, np.pi, 100)))" 127 | ] 128 | }, 129 | { 130 | "cell_type": "code", 131 | "execution_count": null, 132 | "metadata": { 133 | "collapsed": true 134 | }, 135 | "outputs": [], 136 | "source": [] 137 | } 138 | ], 139 | "metadata": { 140 | "anaconda-cloud": {}, 141 | "kernelspec": { 142 | "display_name": "Python [conda env:IOOS]", 143 | "language": "python", 144 | "name": "conda-env-IOOS-py" 145 | }, 146 | "language_info": { 147 | "codemirror_mode": { 148 | "name": "ipython", 149 | "version": 3 150 | }, 151 | "file_extension": ".py", 152 | "mimetype": "text/x-python", 153 | "name": "python", 154 | "nbconvert_exporter": "python", 155 | "pygments_lexer": "ipython3", 156 | "version": "3.5.2" 157 | }, 158 | "latex_envs": { 159 | "bibliofile": "biblio.bib", 160 | "cite_by": "apalike", 161 | "current_citInitial": 1, 162 | "eqLabelWithNumbers": true, 163 | "eqNumInitial": 0 164 | }, 165 | "nav_menu": {}, 166 | "toc": { 167 | "colors": { 168 | "hover_highlight": "#DAA520", 169 | "running_highlight": "#FF0000", 170 | "selected_highlight": "#FFD700" 171 | }, 172 | "moveMenuLeft": true, 173 | "nav_menu": { 174 | "height": "30px", 175 | "width": "252px" 176 | }, 177 | "navigate_menu": true, 178 | "number_sections": true, 179 | "sideBar": true, 180 | "threshold": 4, 181 | "toc_cell": false, 182 | "toc_section_display": "block", 183 | "toc_window_display": false, 184 | "widenNotebook": false 185 | } 186 | }, 187 | "nbformat": 4, 188 | "nbformat_minor": 1 189 | } 190 | -------------------------------------------------------------------------------- /notebooks/02_resources.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# resources" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "### Object Oriented Programming in Python " 15 | ] 16 | }, 17 | { 18 | "cell_type": "markdown", 19 | "metadata": {}, 20 | "source": [ 21 | "+ [www.google.com](http://www.google.com)\n", 22 | "+ [A good tutorial by Alan Gauld](http://www.freenetpages.co.uk/hp/alan.gauld/tutclass.htm)\n", 23 | "+ The [classes](http://docs.python.org/2/tutorial/classes.html) entry from the official Python doc \n", 24 | "+ A short [pdf](http://www1.gly.bris.ac.uk/~walker/PythonEarthSci/PfES_3.pdf) from Andrew Walker's [Python for Earth Scientists](http://www1.gly.bris.ac.uk/~walker/PythonEarthSci/) course\n" 25 | ] 26 | }, 27 | { 28 | "cell_type": "markdown", 29 | "metadata": {}, 30 | "source": [ 31 | "### Python as a glue: wrapping of C, C++, Fortran or the Cython module " 32 | ] 33 | }, 34 | { 35 | "cell_type": "markdown", 36 | "metadata": {}, 37 | "source": [ 38 | "+ [using python as a *glue*](http://docs.scipy.org/doc/numpy/user/c-info.python-as-glue.html)\n", 39 | "+ [SWIG (Simplified Wrapper and Interface Generator)](http://www.swig.org/Doc1.3/Python.html)\n", 40 | "+ [Cython](http://cython.org)\n", 41 | "+ [f2py](http://www.f2py.com/)\n", 42 | "+ [A notebook on using C and Fortran with Python](http://nbviewer.ipython.org/github/jrjohansson/scientific-python-lectures/blob/master/Lecture-6A-Fortran-and-C.ipynb)" 43 | ] 44 | }, 45 | { 46 | "cell_type": "markdown", 47 | "metadata": {}, 48 | "source": [ 49 | "### Real-time acquisition, physical programming" 50 | ] 51 | }, 52 | { 53 | "cell_type": "markdown", 54 | "metadata": {}, 55 | "source": [ 56 | "+ [pyserial](http://pyserial.sourceforge.net): Module that encapsulates the access for the serial port\n", 57 | "+ [Real World Instrumentation, Automated Data Acquisition and Control Systems](http://www.amazon.com/Real-World-Instrumentation-Python-Acquisition/dp/0596809565): A book available on Amazon.\n", 58 | "+ [Programming Raspberry Pi: Getting Started with Python](http://www.monkmakes.com/?page_id=63): book (not free) with code (free)\n", 59 | "+ [PyVISA](https://pyvisa.readthedocs.org/en/latest/): The PyVISA package enables you to control all kinds of measurement equipment through various busses (GPIB, RS232, USB) with Python programs" 60 | ] 61 | }, 62 | { 63 | "cell_type": "markdown", 64 | "metadata": {}, 65 | "source": [ 66 | "### Image processing" 67 | ] 68 | }, 69 | { 70 | "cell_type": "markdown", 71 | "metadata": {}, 72 | "source": [ 73 | "+ PIL [(Python Image Library)](http://www.pythonware.com/products/pil/): Image processing capabilities to your Python interpreter.\n", 74 | "+ [pillow](http://python-pillow.github.io/): A more user-friendly `fork` of PIL\n", 75 | "+ [scikit-image](http://scikit-image.org): more Image processing, built on top of Numpy / Scipy." 76 | ] 77 | }, 78 | { 79 | "cell_type": "markdown", 80 | "metadata": {}, 81 | "source": [ 82 | "### Symbolic maths" 83 | ] 84 | }, 85 | { 86 | "cell_type": "markdown", 87 | "metadata": {}, 88 | "source": [ 89 | "+ The [SymPy](www.sympy.org) library is a Python library for symbolic mathematics. It supports polynomials, calculus, solving equations, etc\n", 90 | "+ The [sage](http://www.sagemath.org/) software: `Mission: Creating a viable free open source alternative to Magma, Maple, Mathematica and Matlab`" 91 | ] 92 | }, 93 | { 94 | "cell_type": "markdown", 95 | "metadata": {}, 96 | "source": [ 97 | "### Geospatial statistics" 98 | ] 99 | }, 100 | { 101 | "cell_type": "markdown", 102 | "metadata": {}, 103 | "source": [ 104 | "+ [PySAL](http://pysal.readthedocs.org/en/v1.6/): PySAL is a cross-platform library of spatial analysis functions written in Python. It is intended to support the development of high level applications for spatial analysis.\n", 105 | "+ [GeoPandas](http://geopandas.org/): A project based on [Pandas](http://pandas.pydata.org/) to make working with geospatial data in python easier\n", 106 | "+ [Rasterio](https://github.com/mapbox/rasterio): Clean and fast and geospatial raster I/O with Numpy support, developed by the team at [https://www.mapbox.com/](https://www.mapbox.com/)\n", 107 | "+ [Pyproj](https://code.google.com/p/pyproj/): Performs cartographic transformations and geodetic computations. Wrapper around the Proj version 4 library\n", 108 | "+ [Python GDAL/OGR](https://pypi.python.org/pypi/GDAL/): Python bindings + tools around the Geospatial Data Abstraction Library\n", 109 | "+ [Python GIS resources](http://pythongisresources.wordpress.com/interpolation): a blog on geospatial python\n", 110 | "+ [High Performance Geostatistics Library](http://hpgl.mit-ufa.com/): A library written in C++ / Python implementing geostatistical algorithms (*e.g.* kriging, correlograms, etc)" 111 | ] 112 | }, 113 | { 114 | "cell_type": "markdown", 115 | "metadata": {}, 116 | "source": [ 117 | "### computational statistics in Python" 118 | ] 119 | }, 120 | { 121 | "cell_type": "markdown", 122 | "metadata": {}, 123 | "source": [ 124 | "+ [Computational Statistics in Python](https://people.duke.edu/~ccc14/sta-663/): impressive resource on computational stats with Python, from the basics to more advanced topics" 125 | ] 126 | }, 127 | { 128 | "cell_type": "markdown", 129 | "metadata": {}, 130 | "source": [ 131 | "### Bayesian statistics with Python" 132 | ] 133 | }, 134 | { 135 | "cell_type": "markdown", 136 | "metadata": {}, 137 | "source": [ 138 | "+ [Think Bayes](https://github.com/AllenDowney/ThinkBayes2): The code examples and Jupyter notebooks for the (free) book **Think Bayes** by Allen Downey, available [here](http://greenteapress.com/wp/think-bayes/)\n", 139 | "+ [Bayesian Methods for Hackers](https://github.com/CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers): Code examples and Jupyter notebook accompanying the book [Bayesian Methods for Hackers: Probabilistic Programming and Bayesian Inference](https://www.amazon.com/Bayesian-Methods-Hackers-Probabilistic-Addison-Wesley/dp/0133902838)\n", 140 | "+ [pyMC3](http://pymc-devs.github.io/pymc3/): Bayesian statistical modeling and Probabilistic Machine Learning\n", 141 | "+ [Statistical Rethinking in Python](https://github.com/aloctavodia/Statistical-Rethinking-with-Python-and-PyMC3): Porting the example in the book [Statistical Rethinking, by R. MacElreath](http://xcelab.net/rm/statistical-rethinking/) from R/Stan to Python" 142 | ] 143 | }, 144 | { 145 | "cell_type": "markdown", 146 | "metadata": {}, 147 | "source": [ 148 | "### Climate and Meteorology" 149 | ] 150 | }, 151 | { 152 | "cell_type": "markdown", 153 | "metadata": { 154 | "collapsed": true 155 | }, 156 | "source": [ 157 | "+ [The UNIDATA gallery](http://unidata.github.io/python-gallery/index.html)\n", 158 | "+ [MetPy](https://github.com/Unidata/MetPy): a Python package for Meteorological data, can do stuff like plot sounding data, meteograms, etc" 159 | ] 160 | }, 161 | { 162 | "cell_type": "markdown", 163 | "metadata": {}, 164 | "source": [ 165 | "### Biology, ecology" 166 | ] 167 | }, 168 | { 169 | "cell_type": "markdown", 170 | "metadata": {}, 171 | "source": [ 172 | "+ [Biopython](http://biopython.org/wiki/Main_Page): Biopython is a set of freely available tools for biological computation written in Python by an international team of developers. \n", 173 | "+ [GeoEco](https://pypi.python.org/pypi/GeoEco): Open source geoprocessing toolbox designed for coastal and marine researchers and GIS analysts who work with spatially-explicit ecological and oceanographic data. For Windows (> XP) only.\n", 174 | "+ [Galaxy](http://galaxyproject.org/): Galaxy is a scientific workflow, data integration and data and analysis persistence and publishing platform that aims to make computational biology accessible to research scientists that do not have computer programming experience." 175 | ] 176 | }, 177 | { 178 | "cell_type": "markdown", 179 | "metadata": {}, 180 | "source": [ 181 | "### Computational Fluid Dynamics and PDE solvers" 182 | ] 183 | }, 184 | { 185 | "cell_type": "markdown", 186 | "metadata": {}, 187 | "source": [ 188 | "+ [CFD Python: 12 steps to Navier-Stokes](http://lorenabarba.com/blog/cfd-python-12-steps-to-navier-stokes/)\n", 189 | "+ [github repository for the above](https://github.com/barbagroup/CFDPython)\n", 190 | "+ [Python bindings to OpenFOAM](http://openfoamwiki.net/index.php/Contrib_PyFoam)\n", 191 | "+ [FiPy: A Finite Volume PDE Solver Using Python](http://matforge.org/fipy/)\n", 192 | "+ [\"Practical Numerical Methods with Python\" MOOC](http://openedx.seas.gwu.edu/courses/GW/MAE6286/2014_fall/about)" 193 | ] 194 | }, 195 | { 196 | "cell_type": "markdown", 197 | "metadata": {}, 198 | "source": [ 199 | "### Python on the GPU " 200 | ] 201 | }, 202 | { 203 | "cell_type": "markdown", 204 | "metadata": {}, 205 | "source": [ 206 | "+ [PyCUDA](http://mathema.tician.de/software/pycuda/): PyCUDA lets you access Nvidia‘s CUDA parallel computation API from Python" 207 | ] 208 | }, 209 | { 210 | "cell_type": "markdown", 211 | "metadata": {}, 212 | "source": [ 213 | "### Parallel computing with Python / IPython" 214 | ] 215 | }, 216 | { 217 | "cell_type": "markdown", 218 | "metadata": {}, 219 | "source": [ 220 | "+ [IPython parallel introduction](http://ipython.org/ipython-doc/2/parallel/parallel_intro.html)\n", 221 | "+ [Interactive Parallel computing with IPython](https://www.youtube.com/watch?v=y4hgalfhc1Y&index=49&list=PLYx7XA2nY5GfuhCvStxgbynFNrxr3VFog): tutorials by Min. Raglan-Kelley at Scipy2014 (3 parts in the youtube list)\n", 222 | "+ [IPython parallel for distributed computing](https://www.youtube.com/watch?v=eyUeYnT-18s&index=32&list=PLs4CJRBY5F1Jm7H1dlesRvEgr-28QGXpR): A talk at PyCON Australia by Nathan Faggian\n", 223 | "+ [IPython & Jupyter in depth: high productivity interactive and parallel python](https://www.youtube.com/watch?v=05fA_DXgW-Y): A talk at PyCON 2015 by Thomas Kluyver and Kyle Kelley" 224 | ] 225 | }, 226 | { 227 | "cell_type": "markdown", 228 | "metadata": {}, 229 | "source": [ 230 | "### Signal processing in Python" 231 | ] 232 | }, 233 | { 234 | "cell_type": "markdown", 235 | "metadata": {}, 236 | "source": [ 237 | "+ [scipy.signal](http://docs.scipy.org/doc/scipy/reference/tutorial/signal.html)\n", 238 | "+ [Python for signal processing blog](http://python-for-signal-processing.blogspot.co.nz/)\n", 239 | "+ [Python for signal processing github repository](https://github.com/unpingco/Python-for-Signal-Processing)\n", 240 | "+ [Python for signal processing book](http://www.springer.com/engineering/signals/book/978-3-319-01341-1)" 241 | ] 242 | }, 243 | { 244 | "cell_type": "markdown", 245 | "metadata": {}, 246 | "source": [ 247 | "### Python for Matlab and R users " 248 | ] 249 | }, 250 | { 251 | "cell_type": "markdown", 252 | "metadata": {}, 253 | "source": [ 254 | "+ [Moving from MATLAB matrices to NumPy arrays - A Matrix Cheatsheet](http://sebastianraschka.com/Articles/2014_matlab_vs_numpy.html)\n", 255 | "\n", 256 | "+ [NumPy for Matlab Users](http://wiki.scipy.org/NumPy_for_Matlab_Users)\n" 257 | ] 258 | }, 259 | { 260 | "cell_type": "markdown", 261 | "metadata": {}, 262 | "source": [ 263 | "### Some URLs and blogs, notebooks repositories for climatology / meteorology / oceanography" 264 | ] 265 | }, 266 | { 267 | "cell_type": "markdown", 268 | "metadata": {}, 269 | "source": [ 270 | "+ [Python Scientific Lectures Notes](http://scipy-lectures.github.io): Tutorial material on the scientific Python ecosystem, a quick introduction to central tools and techniques.\n", 271 | "\n", 272 | "+ [Python for earth scientists](http://www1.gly.bris.ac.uk/~walker/PythonEarthSci/): A two afternoons course by Andrew Walker (University of Bristol) on Python in the earth sciences. \n", 273 | "\n", 274 | "+ [oceanpython](http://oceanpython.org/): Python for oceanography \n", 275 | "\n", 276 | "+ [PyAOS](http://pyaos.johnny-lin.com/): Python for the Atmospheric and Oceanic Sciences \n", 277 | "\n", 278 | "+ [python4oceanographers](http://ocefpaf.github.io/): Learn python with examples applied to marine sciences.\n", 279 | "\n", 280 | "+ [http://earthpy.org/](http://earthpy.org/): EarthPy is a collection of IPython notebooks with examples of Earth Science related Python code\n", 281 | "\n", 282 | "+ [Jupyter notebook gallery from Unidata](http://unidata.github.io/notebook-gallery/): Exploring using Python for Meteorology" 283 | ] 284 | }, 285 | { 286 | "cell_type": "markdown", 287 | "metadata": {}, 288 | "source": [ 289 | "### Github repositories: notebooks and accompanying material" 290 | ] 291 | }, 292 | { 293 | "cell_type": "markdown", 294 | "metadata": {}, 295 | "source": [ 296 | "+ [A gallery of interesting IPython Notebooks](https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks)\n", 297 | "\"...a curated collection of IPython notebooks that are notable for some reason.\"\n", 298 | "\n", 299 | "+ [Scikit-learn tutorial](https://github.com/jakevdp/sklearn_scipy2013): Files and other info associated with the Scipy 2013 scikit-learn tutorial developed by Gaël Varoquaux, Olivier Grisel and Jake VanderPlas.\n", 300 | "\n", 301 | "+ [Statistical Analysis tutorial](https://github.com/fonnesbeck/statistical-analysis-python-tutorial) from Chris. Fonnesbeck. \n", 302 | "\n", 303 | "+ [Bayesian Statistical Analysis in Python](https://github.com/fonnesbeck/scipy2014_tutorial): Ipython notebooks for the Scipy 2014 tutorial on Bayesian data analysis with Python, by Chris. Fonnesbeck. \n", 304 | "\n", 305 | "+ [AstroML: Machine Learning and Data Mining for Astronomy](http://www.astroml.org): A library and tutorial by Jake VanderPlas and co-authors, accompanying the book [Statistics, Data Mining, and Machine Learning in Astronomy](http://www.amazon.com/Statistics-Mining-Machine-Learning-Astronomy/dp/0691151687/)\n", 306 | "\n", 307 | "+ [Scientific Python lectures from Robert Johansson](https://github.com/jrjohansson/scientific-python-lectures)\n", 308 | "\n", 309 | "+ [Data Science in Python](http://blog.yhathq.com/posts/data-science-in-python-tutorial.html): A series of annotated notebooks on data science (i.e. geared towards machine learning) in python\n", 310 | "\n", 311 | "+ [Pythonic perambulations](http://jakevdp.github.io): A blog by Jake VanderPlas\n" 312 | ] 313 | }, 314 | { 315 | "cell_type": "markdown", 316 | "metadata": {}, 317 | "source": [ 318 | "### Some books " 319 | ] 320 | }, 321 | { 322 | "cell_type": "markdown", 323 | "metadata": {}, 324 | "source": [ 325 | "+ [Python for Data Analysis](http://shop.oreilly.com/product/0636920023784.do): From Wes McKinney (Developer of [Pandas](www.pandas.pydata.org))\n", 326 | "\n", 327 | "+ [Think stats](http://www.greenteapress.com/thinkstats/): Probability and statistics for programmers, from Allen Downey, pdf available for free. \n", 328 | "\n", 329 | "+ [Think complexity](http://www.greenteapress.com/compmod/): Complexity science (graphs, cellular automata, agent-based models), from Allen Downey, pdf available for free.\n", 330 | "\n", 331 | "+ [Python in hydrology](http://www.greenteapress.com/pythonhydro/pythonhydro.html): A book freely available in pdf, from Sat Kumar Tomer.\n", 332 | "\n", 333 | "+ [Programming collective intelligence](http://shop.oreilly.com/product/9780596529321.do): By Toby Segaran, Good intro on (general) Machine Learning algorithms.\n", 334 | "\n", 335 | "+ [Machine Learning in action](http://www.manning.com/pharrington/): By Peter Harrington.\n", 336 | "\n", 337 | "+ [Introduction to Python for Econometrics, Statistics and Numerical Analysis: Second Edition](http://www.kevinsheppard.com/images/0/09/Python_introduction.pdf): By Kevin Sheppard, Oxford Uni.\n", 338 | "\n", 339 | "+ [A Hands-On Introduction to Using Python in the Atmospheric and Oceanic Sciences](http://www.johnny-lin.com/pyintro/): By [Johnny Lin](http://www.johnny-lin.com), Professor of physics and head of the Climate Research Group at North Park University.\n", 340 | "\n", 341 | "+ [principles of planetary climate](http://geosci.uchicago.edu/~rtp1/PrinciplesPlanetaryClimate/index.html): by Ray Pierrehumbert. Excellent book on the physics of planetary climates, with freely downloadable python code to follow the examples given in the book. " 342 | ] 343 | }, 344 | { 345 | "cell_type": "markdown", 346 | "metadata": {}, 347 | "source": [ 348 | "### Some interesting libraries, built on top of the main Scientific stack" 349 | ] 350 | }, 351 | { 352 | "cell_type": "markdown", 353 | "metadata": {}, 354 | "source": [ 355 | "+ [PYMC](http://pymc-devs.github.io/pymc/): By Chris Fonnesbeck, Bayesian statistical models and fitting algorithms, including Markov chain Monte Carlo. \n", 356 | "\n", 357 | "+ [Seaborn](http://www.stanford.edu/~mwaskom/software/seaborn/): Statistical data visualization, by Michael Waskom. Its [graphical representation of linear models](http://www.stanford.edu/~mwaskom/software/seaborn/linear_models.html) is particularly interesting. \n", 358 | "\n", 359 | "+ [ggplot](https://github.com/yhat/ggplot/): For [R](http://cran.r-project.org) users, a 'port' of the [ggplot2](http://ggplot2.org) package to Python, see [here](http://blog.yhathq.com/posts/ggplot-0.4-released.html) for what's new in the latest release. \n", 360 | "\n", 361 | "+ [coards](https://pypi.python.org/pypi/coards): A COARDS compliant time parser. See also [netcdftime](http://netcdf4-python.googlecode.com/svn/trunk/docs/netcdftime.netcdftime-module.html) which is part of the [NetCDF4 module](https://code.google.com/p/netcdf4-python/)\n", 362 | "\n", 363 | "+ [seawater](http://www.imr.no/~bjorn/python/seawater/index.html): Similar to the MATLAB toolboxes SEAWATER from CSIRO and parts of OCEANS from Woods Hole Institute.\n", 364 | "\n", 365 | "+ [fluid](https://pypi.python.org/pypi/fluid): Procedures to study fluids on Python, focused for oceanography, meteorology and related sciences.\n", 366 | "\n", 367 | "+ [kyPyWavelet](https://github.com/regeirk/kPyWavelet): Continuous wavelet transform module for Python *ala* Torrence and Compo. Some manual edits were necessary to make it work for me ... \n", 368 | "\n", 369 | "+ [pyresample](https://code.google.com/p/pyresample/): Resampling (reprojection) of geospatial image data in Python\n", 370 | "\n", 371 | "+ [Rpy2](http://rpy.sourceforge.net/rpy2.html): calling R from Python" 372 | ] 373 | }, 374 | { 375 | "cell_type": "markdown", 376 | "metadata": {}, 377 | "source": [ 378 | "### Some articles on open and reproducible research" 379 | ] 380 | }, 381 | { 382 | "cell_type": "markdown", 383 | "metadata": {}, 384 | "source": [ 385 | "+ [Reproducible Research in Computational Science](http://dx.doi.org/10.1126/science.1213847), Roger D. Peng, Science 334, 1226 (2011).\n", 386 | "\n", 387 | "+ [Shining Light into Black Boxes](http://dx.doi.org/10.1126/science.1218263), A. Morin et al., Science 336, 159-160 (2012).\n", 388 | " \n", 389 | "+ [The case for open computer programs](http://dx.doi.org/doi:10.1038/nature10836), D.C. Ince, Nature 482, 485 (2012).\n", 390 | "\n", 391 | "+ [Best practices for scientific computing](http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1001745): Paper in PLOS Biology exposing some of the tools and methods to build better Scientific software.\n" 392 | ] 393 | }, 394 | { 395 | "cell_type": "markdown", 396 | "metadata": {}, 397 | "source": [ 398 | "## Acknowledgments" 399 | ] 400 | }, 401 | { 402 | "cell_type": "markdown", 403 | "metadata": {}, 404 | "source": [ 405 | "In building this material, I have liberally '*borrowed*' from lecture notes, online notebooks, video recording of talks, code examples, articles, etc, that are freely available online, I would like to acknowledge in particular (list not exhaustive and in no particular order): \n", 406 | "\n", 407 | "+ [Fernando Perez](http://blog.fperez.org): Lead developer of [IPython](www.ipython.org), great advocate of [open science](http://en.wikipedia.org/wiki/Open_science) and reproducible research\n", 408 | "+ Brian Granger: Lead developer of the [IPython notebook](http://ipython.org/notebook.html)\n", 409 | "+ [Wes McKinney](http://blog.wesmckinney.com): Lead developer of Pandas, author of [Python for Data Analysis](http://shop.oreilly.com/product/0636920023784.do)\n", 410 | "+ [Christopher Fonnesbeck](http://biostat.mc.vanderbilt.edu/wiki/Main/ChrisFonnesbeck): developer of [PYMC](http://pymc-devs.github.io/pymc/) (Bayesian statistical models and fitting algorithms, including Markov chain Monte Carlo)\n", 411 | "+ [Jake VanderPlas](http://www.astro.washington.edu/users/vanderplas/): Astronomer, contributor to [scikit-learn](http://scikit-learn.org)\n", 412 | "+ [Gaël Varoquaux](http://gael-varoquaux.info): One of the main contributors to [scikit-learn](http://scikit-learn.org)\n", 413 | "+ [Olivier Grisel](http://ogrisel.com): Also main contributor to [scikit-learn](http://scikit-learn.org)\n", 414 | "+ [Travis Oliphant](http://technicaldiscovery.blogspot.co.nz): One of the lead developers of Numpy / Scipy, and founder of [Continuum Analytics](http://continuum.io).\n", 415 | "+ J.R. Johansson http://dml.riken.jp/~rob/\n", 416 | "+ [Thomas Wiecki](http://twiecki.github.io/): developer of [PYMC3](https://github.com/pymc-devs/pymc), a fork of PYMC. \n", 417 | "+ [Skipper Seabold](http://jseabold.net/): The main developer of [statsmodels](http://statsmodels.sourceforge.net/)\n", 418 | "+ [Kevin Sheppard](http://www.kevinsheppard.com/wiki/Main_Page) from University of Oxford: author or the book [Introduction to Python for Econometrics, Statistics and Numerical Analysis: Second Edition](http://www.kevinsheppard.com/images/0/09/Python_introduction.pdf). \n", 419 | "\n", 420 | "And finally [John Hunter](http://blog.fperez.org/2013/07/in-memoriam-john-d-hunter-iii-1968-2012.html), He was the founder and lead developer of [Matplotlib](http://matplotlib.org), a pivotal library to make Python a viable free and open-source alternative to commercial scientific software, and very sadly passed away in 2012.\n", 421 | "\n", 422 | "I encourage you - if you carry on using Python for science - to look up these people on google, have a look at their github repositories and the projects they contribute to, and follow them on twitter, some of them (*e.g.* Jake VanderPlas) have also very informative blogs. You can also donate to the [John Hunter memorial fund](http://numfocus.org/johnhunter.html) as a way to give back to an important contributor to the Python scientific community.\n", 423 | "\n", 424 | "---" 425 | ] 426 | }, 427 | { 428 | "cell_type": "markdown", 429 | "metadata": {}, 430 | "source": [ 431 | "
\n", 432 | "![](files/images/python.png)\n", 433 | "
" 434 | ] 435 | } 436 | ], 437 | "metadata": { 438 | "anaconda-cloud": {}, 439 | "kernelspec": { 440 | "display_name": "Python [conda env:IOOS]", 441 | "language": "python", 442 | "name": "conda-env-IOOS-py" 443 | }, 444 | "language_info": { 445 | "codemirror_mode": { 446 | "name": "ipython", 447 | "version": 3 448 | }, 449 | "file_extension": ".py", 450 | "mimetype": "text/x-python", 451 | "name": "python", 452 | "nbconvert_exporter": "python", 453 | "pygments_lexer": "ipython3", 454 | "version": "3.5.2" 455 | }, 456 | "latex_envs": { 457 | "bibliofile": "biblio.bib", 458 | "cite_by": "apalike", 459 | "current_citInitial": 1, 460 | "eqLabelWithNumbers": true, 461 | "eqNumInitial": 0 462 | }, 463 | "nav_menu": {}, 464 | "toc": { 465 | "colors": { 466 | "hover_highlight": "#DAA520", 467 | "running_highlight": "#FF0000", 468 | "selected_highlight": "#FFD700" 469 | }, 470 | "moveMenuLeft": true, 471 | "nav_menu": { 472 | "height": "407px", 473 | "width": "252px" 474 | }, 475 | "navigate_menu": true, 476 | "number_sections": true, 477 | "sideBar": true, 478 | "threshold": 4, 479 | "toc_cell": false, 480 | "toc_section_display": "block", 481 | "toc_window_display": false, 482 | "widenNotebook": false 483 | } 484 | }, 485 | "nbformat": 4, 486 | "nbformat_minor": 1 487 | } 488 | -------------------------------------------------------------------------------- /notebooks/03_Jupyter_notebook.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "## Overview of IPython interpreter and the Jupyter notebook " 8 | ] 9 | }, 10 | { 11 | "cell_type": "code", 12 | "execution_count": null, 13 | "metadata": { 14 | "collapsed": true 15 | }, 16 | "outputs": [], 17 | "source": [ 18 | "from IPython.display import display, HTML" 19 | ] 20 | }, 21 | { 22 | "cell_type": "markdown", 23 | "metadata": {}, 24 | "source": [ 25 | "Python ships with a basic interactive **interpreter** ... not much fun here\n", 26 | "\n", 27 | "\n", 28 | "
\n", 29 | "\n", 30 | "
\n", 31 | "\n", 32 | "[IPython](www.ipython.org) is an enhanced interactive interpreter, developped for scientific development, adding e.g. history, tab-completion, interactive help, and lots and lots of goodies: \n", 33 | "\n", 34 | "
\n", 35 | "\n", 36 | "
\n", 37 | "\n", 38 | "The [jupyter qtconsole](http://jupyter.org/qtconsole/stable/) provides a number of enhancements only possible in a GUI, such as inline figures, proper multiline editing with syntax highlighting, graphical calltips, and much more. To launch it: \n", 39 | "\n", 40 | "```\n", 41 | "$ jupyter qtconsole\n", 42 | "```\n", 43 | "\n", 44 | "
\n", 45 | "\n", 46 | "
\n", 47 | "\n", 48 | "Since 2005, the IPython development team has introduced the **IPython (now Jupyter) notebook**. This is a web application, running in the browser, that is connected to a Python kernel running in the background. " 49 | ] 50 | }, 51 | { 52 | "cell_type": "code", 53 | "execution_count": null, 54 | "metadata": { 55 | "collapsed": true 56 | }, 57 | "outputs": [], 58 | "source": [ 59 | "from IPython.display import HTML\n", 60 | "HTML('')" 61 | ] 62 | }, 63 | { 64 | "cell_type": "markdown", 65 | "metadata": {}, 66 | "source": [ 67 | "
" 68 | ] 69 | }, 70 | { 71 | "cell_type": "markdown", 72 | "metadata": {}, 73 | "source": [ 74 | "It behaves as an **interactive notebook**, in which you can weave Python code and outputs, figures generated from Python / matplotlib, images (either local and remote), websites, videos and richly formatted comments using [markdown](https://en.wikipedia.org/wiki/Markdown), which is a superset of HTML with a very simple syntax (see [here](www.markdown.com) for more) \n", 75 | "\n", 76 | "It is structured into executable cells, into which by default you can run arbitrary python code\n", 77 | "\n", 78 | "It has a sophisticated tab completion and help system \n", 79 | "\n", 80 | "The power of IPython / Jupyter comes in part because of its numerous extensions and *magic* functions\n", 81 | "\n", 82 | "And finally you can *export* a notebook in different formats, including HTML and Latex (and PDF). " 83 | ] 84 | }, 85 | { 86 | "cell_type": "markdown", 87 | "metadata": {}, 88 | "source": [ 89 | "Recently the Jupyter / Ipython notebook as been featured in the **toolbox** section of [Nature](http://www.nature.com)" 90 | ] 91 | }, 92 | { 93 | "cell_type": "code", 94 | "execution_count": null, 95 | "metadata": { 96 | "collapsed": true 97 | }, 98 | "outputs": [], 99 | "source": [ 100 | "from IPython.display import HTML\n", 101 | "HTML('')" 102 | ] 103 | }, 104 | { 105 | "cell_type": "markdown", 106 | "metadata": {}, 107 | "source": [ 108 | "## Using the Jupyter notebook" 109 | ] 110 | }, 111 | { 112 | "cell_type": "markdown", 113 | "metadata": {}, 114 | "source": [ 115 | "To start a Jupyter notebook session open a command prompt, navigate to a desired working directory then issue the following command:\n", 116 | "\n", 117 | "`$ jupyter notebook`\n", 118 | "\n", 119 | "A new window open in your browser where you can open an existing notebook or start a new one.\n", 120 | "\n", 121 | "it should look like \n", 122 | "\n", 123 | "
\n", 124 | "\n", 125 | "
\n", 126 | "\n", 127 | "Notebooks are organized with **cells**. You may have a **code cell** for inputing commands followed by its result cell which contains the output of the code. You may also have a text cell, such as this one you're reading right now. Cell type can be changed in the above dropdown menu.\n", 128 | "\n", 129 | "There is the menubar above for navigating a notebook but you will find the following shortcuts helpful:\n", 130 | "\n", 131 | "- `Enter` : Enter the edit mode of a cell\n", 132 | "- `Esc` : Leave the edit mode and get into the *control* mode (see icons on the right of the menu bar)\n", 133 | "- `Shift + Enter` : Execute cell and advance to next cell (or use the play button in the toolbar)\n", 134 | "- `Alt + Enter` : Execute cell and insert an empty cell below\n", 135 | "- `Ctrl + Enter` : Execute cell in place (do not advance to the next cell)\n", 136 | "- Press `esc` (command mode) then `h` to display keyboard shortcuts\n", 137 | "\n", 138 | "At times you might run code that gets stuck in an infinite loop or simply you want to clear all your workspace variables and start over. To do that you can do:\n", 139 | "\n", 140 | "`Kernel -> Interrupt`\n", 141 | "\n", 142 | "`Kernel -> Restart`\n" 143 | ] 144 | }, 145 | { 146 | "cell_type": "code", 147 | "execution_count": null, 148 | "metadata": { 149 | "collapsed": true 150 | }, 151 | "outputs": [], 152 | "source": [ 153 | "# This is a code cell, containing python code, it is executed by - or -\n", 154 | "\n", 155 | "def square_that(x):\n", 156 | " \"\"\"\n", 157 | " this is a function that squares a number\n", 158 | " \"\"\"\n", 159 | " z = x**2\n", 160 | " return z" 161 | ] 162 | }, 163 | { 164 | "cell_type": "code", 165 | "execution_count": null, 166 | "metadata": { 167 | "collapsed": true 168 | }, 169 | "outputs": [], 170 | "source": [ 171 | "square_that(2)" 172 | ] 173 | }, 174 | { 175 | "cell_type": "markdown", 176 | "metadata": {}, 177 | "source": [ 178 | "### Getting help" 179 | ] 180 | }, 181 | { 182 | "cell_type": "markdown", 183 | "metadata": {}, 184 | "source": [ 185 | "Jupyter has a sophisticated help and tab completion system, which allows **introspection**: i.e.\n", 186 | "If you want details regarding the properties and functionality of any Python objects currently loaded into your name space, you can use the `?` to reveal any details that are available:" 187 | ] 188 | }, 189 | { 190 | "cell_type": "code", 191 | "execution_count": null, 192 | "metadata": { 193 | "collapsed": true 194 | }, 195 | "outputs": [], 196 | "source": [ 197 | "square_that?" 198 | ] 199 | }, 200 | { 201 | "cell_type": "code", 202 | "execution_count": null, 203 | "metadata": { 204 | "collapsed": true 205 | }, 206 | "outputs": [], 207 | "source": [ 208 | "from numpy import random \n", 209 | "random.randn?" 210 | ] 211 | }, 212 | { 213 | "cell_type": "markdown", 214 | "metadata": {}, 215 | "source": [ 216 | "### Tab completion" 217 | ] 218 | }, 219 | { 220 | "cell_type": "markdown", 221 | "metadata": {}, 222 | "source": [ 223 | "Because Jupyter allows for introspection, it is able to afford the user the ability to tab-complete commands or objects that have been partially typed. This is done by pressing the `` key at any point during the process of typing a command:" 224 | ] 225 | }, 226 | { 227 | "cell_type": "code", 228 | "execution_count": null, 229 | "metadata": { 230 | "collapsed": true 231 | }, 232 | "outputs": [], 233 | "source": [ 234 | "listA = [1, 2., 'sentence', 1, (1,2), {'answer': 42}] # here I construct a LIST containing different items" 235 | ] 236 | }, 237 | { 238 | "cell_type": "code", 239 | "execution_count": null, 240 | "metadata": { 241 | "collapsed": true 242 | }, 243 | "outputs": [], 244 | "source": [ 245 | "listA" 246 | ] 247 | }, 248 | { 249 | "cell_type": "code", 250 | "execution_count": null, 251 | "metadata": { 252 | "collapsed": true 253 | }, 254 | "outputs": [], 255 | "source": [ 256 | "listA.index(1)" 257 | ] 258 | }, 259 | { 260 | "cell_type": "markdown", 261 | "metadata": {}, 262 | "source": [ 263 | "### including markdown comments " 264 | ] 265 | }, 266 | { 267 | "cell_type": "markdown", 268 | "metadata": {}, 269 | "source": [ 270 | "**see [markdown](http://daringfireball.net/projects/markdown/)**\n", 271 | "\n", 272 | "Using markdown cells, you can insert formatted comments, observations ... directly in between executable code and outputs ... \n", 273 | "\n", 274 | "To toggle from *code cell* (the default) to *markdown cell*, you can use the toolbar, or ```- + ``` (```- + ``` on Macs)\n", 275 | "\n", 276 | "You can *italicize*, **boldface**\n", 277 | "\n", 278 | "\n", 279 | "\n", 280 | "+ build\n", 281 | "\n", 282 | "+ lists\n", 283 | "\n", 284 | "\n", 285 | "1. enumerate \n", 286 | "\n", 287 | "2. stuff\n", 288 | "\n", 289 | "and embed syntax highlighted code meant for illustration instead of execution in Python:\n", 290 | "\n", 291 | "```python\n", 292 | "print('hello')\n", 293 | "```\n", 294 | "\n", 295 | "```python\n", 296 | "def f(x):\n", 297 | " \"\"\"a docstring\"\"\"\n", 298 | " return x**2\n", 299 | "```\n", 300 | "\n", 301 | "or other languages:\n", 302 | "\n", 303 | "```C\n", 304 | "/* Fibonacci Series c language */\n", 305 | "#include\n", 306 | " \n", 307 | "int main()\n", 308 | "{\n", 309 | " int n, first = 0, second = 1, next, c;\n", 310 | " \n", 311 | " printf(\"Enter the number of terms\\n\");\n", 312 | " scanf(\"%d\",&n);\n", 313 | " \n", 314 | " printf(\"First %d terms of Fibonacci series are :-\\n\",n);\n", 315 | " \n", 316 | " for ( c = 0 ; c < n ; c++ )\n", 317 | " {\n", 318 | " if ( c <= 1 )\n", 319 | " next = c;\n", 320 | " else\n", 321 | " {\n", 322 | " next = first + second;\n", 323 | " first = second;\n", 324 | " second = next;\n", 325 | " }\n", 326 | " printf(\"%d\\n\",next);\n", 327 | " }\n", 328 | " \n", 329 | " return 0;\n", 330 | "}\n", 331 | "```" 332 | ] 333 | }, 334 | { 335 | "cell_type": "markdown", 336 | "metadata": {}, 337 | "source": [ 338 | "You can insert images seemlessly into a Markdown cell, but if it is **local** instead of on the web, it needs to reside in an **images** folder in the **same directory as your notebook**. \n", 339 | "\n", 340 | "The syntax is then \n", 341 | "\n", 342 | " ![](files/images/image.png)\n", 343 | " \n", 344 | "e.g.\n", 345 | "\n", 346 | "
\n", 347 | "![tag](files/images/presentation.png)\n", 348 | "
\n" 349 | ] 350 | }, 351 | { 352 | "cell_type": "markdown", 353 | "metadata": {}, 354 | "source": [ 355 | "Thanks to [MathJax](www.wikipedia/mathjax), you can include mathematical expressions in a markdown cell using Latex syntax both inline and displayed: \n", 356 | "\n", 357 | "+ inline: `$ Latex expression here $`\n", 358 | "\n", 359 | "+ displayed: `$$ Latex expression here $$`\n", 360 | "\n", 361 | "\n", 362 | "For example ```$e^{i\\pi} + 1 = 0$``` becomes $e^{i\\pi} + 1 = 0$ \n", 363 | "\n", 364 | "and \n", 365 | "\n", 366 | "```$$e^x=\\sum_{i=0}^\\infty \\frac{1}{i!}x^i$$```\n", 367 | "\n", 368 | "becomes: \n", 369 | "\n", 370 | "$$e^x=\\sum_{i=0}^\\infty \\frac{1}{i!}x^i$$\n", 371 | "\n", 372 | "Mathjax is a JavaScript library, and by default IPython uses the online version, from the [Mathjax Content Distribution Network](http://cdn.mathjax.org/) (CDN), if you want to be able to use it **offline**, you need to install MathJax locally by running that (once and for all) in a code cell: \n", 373 | "\n", 374 | "```python\n", 375 | "from IPython.external.mathjax import install_mathjax\n", 376 | "install_mathjax()\n", 377 | "```" 378 | ] 379 | }, 380 | { 381 | "cell_type": "markdown", 382 | "metadata": {}, 383 | "source": [ 384 | "### The IPython notebook rich display system" 385 | ] 386 | }, 387 | { 388 | "cell_type": "code", 389 | "execution_count": null, 390 | "metadata": { 391 | "collapsed": true 392 | }, 393 | "outputs": [], 394 | "source": [ 395 | "from IPython.display import Image, YouTubeVideo" 396 | ] 397 | }, 398 | { 399 | "cell_type": "code", 400 | "execution_count": null, 401 | "metadata": { 402 | "collapsed": true 403 | }, 404 | "outputs": [], 405 | "source": [ 406 | "### that will display a frame with specified width and height ... \n", 407 | "\n", 408 | "from IPython.display import HTML\n", 409 | "HTML('')" 411 | ] 412 | }, 413 | { 414 | "cell_type": "code", 415 | "execution_count": null, 416 | "metadata": { 417 | "collapsed": true 418 | }, 419 | "outputs": [], 420 | "source": [ 421 | "### This inserts an image in the output cell\n", 422 | "\n", 423 | "Image(url='https://niwa.co.nz/sites/niwa.co.nz/files/styles/large/public/outlook-august-october-2017.png')" 424 | ] 425 | }, 426 | { 427 | "cell_type": "code", 428 | "execution_count": null, 429 | "metadata": { 430 | "collapsed": true 431 | }, 432 | "outputs": [], 433 | "source": [ 434 | "### This embeds a video from YouTube\n", 435 | "### You can embed Vimeo videos by calling from IPython.display import VimeoVideo\n", 436 | "\n", 437 | "# Fernando Pérez at PyConCA, here I start at 30 minutes \n", 438 | "YouTubeVideo('F4rFuIb1Ie4', start = 30*60) " 439 | ] 440 | }, 441 | { 442 | "cell_type": "markdown", 443 | "metadata": {}, 444 | "source": [ 445 | "### Inline plots with matplotlib " 446 | ] 447 | }, 448 | { 449 | "cell_type": "markdown", 450 | "metadata": {}, 451 | "source": [ 452 | "That is one of the most appealing aspect of the IPython notebook: the graphic outputs of some Python code can be displayed directly in the notebook, there is in particular great integration with the \n", 453 | "Python plotting library [Matplotlib](http://www.matplotlib.org) via the `%matplotlib inline` magic command" 454 | ] 455 | }, 456 | { 457 | "cell_type": "code", 458 | "execution_count": null, 459 | "metadata": { 460 | "collapsed": true 461 | }, 462 | "outputs": [], 463 | "source": [ 464 | "%matplotlib inline" 465 | ] 466 | }, 467 | { 468 | "cell_type": "code", 469 | "execution_count": null, 470 | "metadata": { 471 | "collapsed": true 472 | }, 473 | "outputs": [], 474 | "source": [ 475 | "import numpy as np\n", 476 | "import matplotlib.pyplot as plt" 477 | ] 478 | }, 479 | { 480 | "cell_type": "code", 481 | "execution_count": null, 482 | "metadata": { 483 | "collapsed": true 484 | }, 485 | "outputs": [], 486 | "source": [ 487 | "t = np.linspace(0, 2*np.pi, 100)\n", 488 | "plt.plot(np.sin(t))" 489 | ] 490 | }, 491 | { 492 | "cell_type": "markdown", 493 | "metadata": {}, 494 | "source": [ 495 | "### Some useful magic commands in IPython" 496 | ] 497 | }, 498 | { 499 | "cell_type": "code", 500 | "execution_count": null, 501 | "metadata": { 502 | "collapsed": true 503 | }, 504 | "outputs": [], 505 | "source": [ 506 | "# for a list of available magic command \n", 507 | "\n", 508 | "%lsmagic " 509 | ] 510 | }, 511 | { 512 | "cell_type": "markdown", 513 | "metadata": {}, 514 | "source": [ 515 | "For example \n", 516 | "\n", 517 | " %whos\n", 518 | "\n", 519 | "list all variables (incl. classes, functions, etc) in the **namespace**" 520 | ] 521 | }, 522 | { 523 | "cell_type": "code", 524 | "execution_count": null, 525 | "metadata": { 526 | "collapsed": true 527 | }, 528 | "outputs": [], 529 | "source": [ 530 | "%whos " 531 | ] 532 | }, 533 | { 534 | "cell_type": "markdown", 535 | "metadata": {}, 536 | "source": [ 537 | "### Writing the content of a cell to file " 538 | ] 539 | }, 540 | { 541 | "cell_type": "code", 542 | "execution_count": null, 543 | "metadata": { 544 | "collapsed": true 545 | }, 546 | "outputs": [], 547 | "source": [ 548 | "%%writefile /Users/nicolasf/Desktop/tmp.py\n", 549 | "#!/Users/nicolasf/anaconda/bin/python \n", 550 | "import numpy as np\n", 551 | "\n", 552 | "print('Hello Auckland')\n", 553 | "\n", 554 | "b = np.arange(5) ### will be available in the namespace " 555 | ] 556 | }, 557 | { 558 | "cell_type": "markdown", 559 | "metadata": {}, 560 | "source": [ 561 | "### Running some external python script (local or online)" 562 | ] 563 | }, 564 | { 565 | "cell_type": "code", 566 | "execution_count": null, 567 | "metadata": { 568 | "collapsed": true 569 | }, 570 | "outputs": [], 571 | "source": [ 572 | "%run /Users/nicolasf/Desktop/tmp.py" 573 | ] 574 | }, 575 | { 576 | "cell_type": "code", 577 | "execution_count": null, 578 | "metadata": { 579 | "collapsed": true 580 | }, 581 | "outputs": [], 582 | "source": [ 583 | "b" 584 | ] 585 | }, 586 | { 587 | "cell_type": "markdown", 588 | "metadata": {}, 589 | "source": [ 590 | "### Loading the content of some local python script " 591 | ] 592 | }, 593 | { 594 | "cell_type": "markdown", 595 | "metadata": {}, 596 | "source": [ 597 | "you can load actually anything, e.g. a Markdown file" 598 | ] 599 | }, 600 | { 601 | "cell_type": "code", 602 | "execution_count": null, 603 | "metadata": { 604 | "collapsed": true 605 | }, 606 | "outputs": [], 607 | "source": [ 608 | "# %load /Users/nicolasf/Desktop/tmp.py\n", 609 | "#!/Users/nicolasf/anaconda/bin/python \n", 610 | "import numpy as np\n", 611 | "\n", 612 | "print('Hello Auckland')\n", 613 | "\n", 614 | "b = np.arange(5) ### will be available in the namespace " 615 | ] 616 | }, 617 | { 618 | "cell_type": "markdown", 619 | "metadata": {}, 620 | "source": [ 621 | "works as well for scripts available online" 622 | ] 623 | }, 624 | { 625 | "cell_type": "code", 626 | "execution_count": null, 627 | "metadata": { 628 | "collapsed": true 629 | }, 630 | "outputs": [], 631 | "source": [ 632 | "# %load http://matplotlib.org/mpl_examples/statistics/histogram_demo_features.py\n", 633 | "\"\"\"\n", 634 | "=========================================================\n", 635 | "Demo of the histogram (hist) function with a few features\n", 636 | "=========================================================\n", 637 | "\n", 638 | "In addition to the basic histogram, this demo shows a few optional\n", 639 | "features:\n", 640 | "\n", 641 | " * Setting the number of data bins\n", 642 | " * The ``normed`` flag, which normalizes bin heights so that the\n", 643 | " integral of the histogram is 1. The resulting histogram is an\n", 644 | " approximation of the probability density function.\n", 645 | " * Setting the face color of the bars\n", 646 | " * Setting the opacity (alpha value).\n", 647 | "\n", 648 | "Selecting different bin counts and sizes can significantly affect the\n", 649 | "shape of a histogram. The Astropy docs have a great section on how to\n", 650 | "select these parameters:\n", 651 | "http://docs.astropy.org/en/stable/visualization/histogram.html\n", 652 | "\"\"\"\n", 653 | "\n", 654 | "import numpy as np\n", 655 | "import matplotlib.mlab as mlab\n", 656 | "import matplotlib.pyplot as plt\n", 657 | "\n", 658 | "np.random.seed(0)\n", 659 | "\n", 660 | "# example data\n", 661 | "mu = 100 # mean of distribution\n", 662 | "sigma = 15 # standard deviation of distribution\n", 663 | "x = mu + sigma * np.random.randn(437)\n", 664 | "\n", 665 | "num_bins = 50\n", 666 | "\n", 667 | "fig, ax = plt.subplots()\n", 668 | "\n", 669 | "# the histogram of the data\n", 670 | "n, bins, patches = ax.hist(x, num_bins, normed=1)\n", 671 | "\n", 672 | "# add a 'best fit' line\n", 673 | "y = mlab.normpdf(bins, mu, sigma)\n", 674 | "ax.plot(bins, y, '--')\n", 675 | "ax.set_xlabel('Smarts')\n", 676 | "ax.set_ylabel('Probability density')\n", 677 | "ax.set_title(r'Histogram of IQ: $\\mu=100$, $\\sigma=15$')\n", 678 | "\n", 679 | "# Tweak spacing to prevent clipping of ylabel\n", 680 | "fig.tight_layout()\n", 681 | "plt.show()\n" 682 | ] 683 | }, 684 | { 685 | "cell_type": "markdown", 686 | "metadata": {}, 687 | "source": [ 688 | "### Interacting with the OS: the IPython notebook as an enhanced shell" 689 | ] 690 | }, 691 | { 692 | "cell_type": "code", 693 | "execution_count": null, 694 | "metadata": { 695 | "collapsed": true 696 | }, 697 | "outputs": [], 698 | "source": [ 699 | "!ls *.py" 700 | ] 701 | }, 702 | { 703 | "cell_type": "code", 704 | "execution_count": null, 705 | "metadata": { 706 | "collapsed": true 707 | }, 708 | "outputs": [], 709 | "source": [ 710 | "notebooks = !ls *.ipynb ### notebooks is a python list ... not sure how to use wildcards on Windows" 711 | ] 712 | }, 713 | { 714 | "cell_type": "code", 715 | "execution_count": null, 716 | "metadata": { 717 | "collapsed": true 718 | }, 719 | "outputs": [], 720 | "source": [ 721 | "notebooks" 722 | ] 723 | }, 724 | { 725 | "cell_type": "markdown", 726 | "metadata": {}, 727 | "source": [ 728 | "And if you need to actually quickly put together and run a bash script (linux / Mac), you don't need to escape the IPython notebook thanks to the \n", 729 | "\n", 730 | " %%bash \n", 731 | " \n", 732 | "cell magic ..." 733 | ] 734 | }, 735 | { 736 | "cell_type": "code", 737 | "execution_count": null, 738 | "metadata": { 739 | "collapsed": true 740 | }, 741 | "outputs": [], 742 | "source": [ 743 | "%%bash\n", 744 | "\n", 745 | "# Substitutes underscores for blanks in all the filenames in a directory.\n", 746 | "\n", 747 | "ONE=1 # For getting singular/plural right (see below).\n", 748 | "number=0 # Keeps track of how many files actually renamed.\n", 749 | "FOUND=0 # Successful return value.\n", 750 | "\n", 751 | "for filename in * # Traverses all files in directory.\n", 752 | "do\n", 753 | " echo \"$filename\" | grep -q \" \" # Checks whether filename\n", 754 | " if [ $? -eq $FOUND ] # contains space(s).\n", 755 | " then\n", 756 | " fname=$filename # Strips off path.\n", 757 | " n=`echo $fname | sed -e \"s/ /_/g\"` # Substitutes underscore for blank.\n", 758 | " mv \"$fname\" \"$n\" # Do the actual renaming.\n", 759 | " let \"number += 1\"\n", 760 | " fi\n", 761 | "done \n", 762 | "\n", 763 | "if [ \"$number\" -eq \"$ONE\" ] # For correct grammar.\n", 764 | "then\n", 765 | " echo \"$number file renamed.\"\n", 766 | "else \n", 767 | " echo \"$number files renamed.\"\n", 768 | "fi \n", 769 | "\n", 770 | "exit 0" 771 | ] 772 | }, 773 | { 774 | "cell_type": "markdown", 775 | "metadata": {}, 776 | "source": [ 777 | "### Exporting your notebook in other formats " 778 | ] 779 | }, 780 | { 781 | "cell_type": "markdown", 782 | "metadata": {}, 783 | "source": [ 784 | "A notebook (extension .ipynb) is actually just a [JSON](http://en.wikipedia.org/wiki/JSON) file, using built-in converters you can \n", 785 | "convert a notebook into a variety of formats for sharing, illustration, publishing, etc. " 786 | ] 787 | }, 788 | { 789 | "cell_type": "code", 790 | "execution_count": null, 791 | "metadata": { 792 | "collapsed": true 793 | }, 794 | "outputs": [], 795 | "source": [ 796 | "!jupyter nbconvert --help" 797 | ] 798 | }, 799 | { 800 | "cell_type": "code", 801 | "execution_count": null, 802 | "metadata": { 803 | "collapsed": true 804 | }, 805 | "outputs": [], 806 | "source": [ 807 | "name = '03_Jupyter_notebook'" 808 | ] 809 | }, 810 | { 811 | "cell_type": "code", 812 | "execution_count": null, 813 | "metadata": { 814 | "collapsed": true 815 | }, 816 | "outputs": [], 817 | "source": [ 818 | "!jupyter nbconvert {name}.ipynb --to html" 819 | ] 820 | }, 821 | { 822 | "cell_type": "code", 823 | "execution_count": null, 824 | "metadata": { 825 | "collapsed": true 826 | }, 827 | "outputs": [], 828 | "source": [ 829 | "from IPython.display import HTML\n", 830 | "HTML(\" {name}.html \".format(name=name))" 831 | ] 832 | }, 833 | { 834 | "cell_type": "markdown", 835 | "metadata": {}, 836 | "source": [ 837 | "## Sharing your Jupyter notebook" 838 | ] 839 | }, 840 | { 841 | "cell_type": "markdown", 842 | "metadata": {}, 843 | "source": [ 844 | "1. host your jupyter notebook on the public internet (e.g. [github](http://github.com) repo, github gist)\n", 845 | "2. go to [http://nbviewer.ipython.org/](http://nbviewer.ipython.org/) and copy the URL\n", 846 | "3. Note that now github automatically renders IPython / Jupyter notebooks" 847 | ] 848 | } 849 | ], 850 | "metadata": { 851 | "anaconda-cloud": {}, 852 | "kernelspec": { 853 | "display_name": "Python [default]", 854 | "language": "python", 855 | "name": "python3" 856 | }, 857 | "language_info": { 858 | "codemirror_mode": { 859 | "name": "ipython", 860 | "version": 3 861 | }, 862 | "file_extension": ".py", 863 | "mimetype": "text/x-python", 864 | "name": "python", 865 | "nbconvert_exporter": "python", 866 | "pygments_lexer": "ipython3", 867 | "version": "3.5.2" 868 | }, 869 | "latex_envs": { 870 | "bibliofile": "biblio.bib", 871 | "cite_by": "apalike", 872 | "current_citInitial": 1, 873 | "eqLabelWithNumbers": true, 874 | "eqNumInitial": 0 875 | }, 876 | "nav_menu": {}, 877 | "toc": { 878 | "colors": { 879 | "hover_highlight": "#DAA520", 880 | "running_highlight": "#FF0000", 881 | "selected_highlight": "#FFD700" 882 | }, 883 | "moveMenuLeft": true, 884 | "nav_menu": { 885 | "height": "280px", 886 | "width": "252px" 887 | }, 888 | "navigate_menu": true, 889 | "number_sections": true, 890 | "sideBar": true, 891 | "threshold": 4, 892 | "toc_cell": false, 893 | "toc_section_display": "block", 894 | "toc_window_display": false, 895 | "widenNotebook": false 896 | } 897 | }, 898 | "nbformat": 4, 899 | "nbformat_minor": 1 900 | } 901 | -------------------------------------------------------------------------------- /notebooks/06_Scipy.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Scipy" 8 | ] 9 | }, 10 | { 11 | "cell_type": "code", 12 | "execution_count": null, 13 | "metadata": { 14 | "collapsed": true 15 | }, 16 | "outputs": [], 17 | "source": [ 18 | "from IPython.display import Image, HTML\n", 19 | "import numpy as np\n", 20 | "from numpy import ma\n", 21 | "import matplotlib.pyplot as plt\n", 22 | "%matplotlib inline" 23 | ] 24 | }, 25 | { 26 | "cell_type": "markdown", 27 | "metadata": {}, 28 | "source": [ 29 | "**Scipy** can is a library (but can be thought more as a scientific python distribution) that is built on top of Numpy and\n", 30 | "and provides a large set of standard scientific computing algorithms, in particular: \n", 31 | "\n", 32 | "* Special functions ([scipy.special](http://docs.scipy.org/doc/scipy/reference/special.html))\n", 33 | "* Integration ([scipy.integrate](http://docs.scipy.org/doc/scipy/reference/integrate.html))\n", 34 | "* Optimization ([scipy.optimize](http://docs.scipy.org/doc/scipy/reference/optimize.html))\n", 35 | "* Interpolation ([scipy.interpolate](http://docs.scipy.org/doc/scipy/reference/interpolate.html))\n", 36 | "* Fourier Transforms ([scipy.fftpack](http://docs.scipy.org/doc/scipy/reference/fftpack.html))\n", 37 | "* Signal Processing ([scipy.signal](http://docs.scipy.org/doc/scipy/reference/signal.html))\n", 38 | "* Linear Algebra ([scipy.linalg](http://docs.scipy.org/doc/scipy/reference/linalg.html))\n", 39 | "* Sparse Eigenvalue Problems ([scipy.sparse](http://docs.scipy.org/doc/scipy/reference/sparse.html))\n", 40 | "* Statistics ([scipy.stats](http://docs.scipy.org/doc/scipy/reference/stats.html))\n", 41 | "* Multi-dimensional image processing ([scipy.ndimage](http://docs.scipy.org/doc/scipy/reference/ndimage.html))\n", 42 | "* File IO ([scipy.io](http://docs.scipy.org/doc/scipy/reference/io.html))\n", 43 | "\n", 44 | "We'll only have a look at some of these sub-packages, more directly relevant to **analyzing** data\n", 45 | " \n", 46 | " \n", 47 | "+ **scipy.stats**: standard continuous and discrete probability distributions (density functions, samplers, ...), various statistical tests, and more descriptive statistics. \n", 48 | "+ **scipy.interpolate**: 1D and 2D interpolation\n" 49 | ] 50 | }, 51 | { 52 | "cell_type": "code", 53 | "execution_count": null, 54 | "metadata": {}, 55 | "outputs": [], 56 | "source": [ 57 | "HTML(\"\")" 58 | ] 59 | }, 60 | { 61 | "cell_type": "code", 62 | "execution_count": null, 63 | "metadata": { 64 | "collapsed": true 65 | }, 66 | "outputs": [], 67 | "source": [ 68 | "import scipy" 69 | ] 70 | }, 71 | { 72 | "cell_type": "markdown", 73 | "metadata": {}, 74 | "source": [ 75 | "## Statistics: Scipy.stats" 76 | ] 77 | }, 78 | { 79 | "cell_type": "code", 80 | "execution_count": null, 81 | "metadata": { 82 | "collapsed": true 83 | }, 84 | "outputs": [], 85 | "source": [ 86 | "from scipy import stats" 87 | ] 88 | }, 89 | { 90 | "cell_type": "code", 91 | "execution_count": null, 92 | "metadata": {}, 93 | "outputs": [], 94 | "source": [ 95 | "dir(stats)" 96 | ] 97 | }, 98 | { 99 | "cell_type": "markdown", 100 | "metadata": {}, 101 | "source": [ 102 | "### Distributions and fitting distributions with Scipy stats" 103 | ] 104 | }, 105 | { 106 | "cell_type": "markdown", 107 | "metadata": {}, 108 | "source": [ 109 | "#### distributions in scipy " 110 | ] 111 | }, 112 | { 113 | "cell_type": "markdown", 114 | "metadata": {}, 115 | "source": [ 116 | "see some references: \n", 117 | "+ [distributions in scipy by John D. Cook](http://www.johndcook.com/distributions_scipy.html)\n", 118 | "+ [Probability distribution parameterizations in SciPy](http://www.johndcook.com/blog/2010/02/03/statistical-distributions-in-scipy/)" 119 | ] 120 | }, 121 | { 122 | "cell_type": "code", 123 | "execution_count": null, 124 | "metadata": {}, 125 | "outputs": [], 126 | "source": [ 127 | "stats.distributions." 128 | ] 129 | }, 130 | { 131 | "cell_type": "code", 132 | "execution_count": null, 133 | "metadata": { 134 | "collapsed": true 135 | }, 136 | "outputs": [], 137 | "source": [ 138 | "# create a (continous) random variable with normal distribution\n", 139 | "Y = stats.distributions.norm()" 140 | ] 141 | }, 142 | { 143 | "cell_type": "code", 144 | "execution_count": null, 145 | "metadata": {}, 146 | "outputs": [], 147 | "source": [ 148 | "Y" 149 | ] 150 | }, 151 | { 152 | "cell_type": "code", 153 | "execution_count": null, 154 | "metadata": {}, 155 | "outputs": [], 156 | "source": [ 157 | "x = np.linspace(-5,5,100)\n", 158 | "\n", 159 | "fig, axes = plt.subplots(3,1, sharex=True, figsize=(10,10))\n", 160 | "\n", 161 | "# plot the probability distribution function (PDF)\n", 162 | "axes[0].plot(x, Y.pdf(x))\n", 163 | "axes[0].set_title('PDF')\n", 164 | "\n", 165 | "# plot the commulative distributin function (CDF)\n", 166 | "axes[1].plot(x, Y.cdf(x));\n", 167 | "axes[1].set_title('CDF')\n", 168 | "\n", 169 | "# plot histogram of 1000 random realizations of the variable Y ~ N(0,1)\n", 170 | "axes[2].hist(Y.rvs(size=1000), bins=20);\n", 171 | "axes[2].set_title('Random Variable');" 172 | ] 173 | }, 174 | { 175 | "cell_type": "markdown", 176 | "metadata": {}, 177 | "source": [ 178 | "#### Fitting data to a particular distribution" 179 | ] 180 | }, 181 | { 182 | "cell_type": "markdown", 183 | "metadata": {}, 184 | "source": [ 185 | "An recurring statistical problem is finding estimates of the relevant parameters that correspond to the distribution that best represents our data. In **parametric** inference, we specify *a priori* a suitable distribution, then choose the parameters that best fit the data.\n", 186 | "\n", 187 | "We're going to see how to do that in Python using two methods:\n", 188 | "\n", 189 | "* The **Method of moments** chooses the parameters so that the sample moments (typically the sample mean and variance) match the theoretical moments of our chosen distribution.\n", 190 | "* The **Maximum likelihood** chooses the parameters to maximize the likelihood, which measures how likely it is to observe our given sample given the parameters. This is effectively done by **optimization**\n", 191 | "\n", 192 | "A real life example: Monthly rainfall amounts at Auckland Airport station" 193 | ] 194 | }, 195 | { 196 | "cell_type": "code", 197 | "execution_count": null, 198 | "metadata": {}, 199 | "outputs": [], 200 | "source": [ 201 | "import pandas as pd; print(pd.__version__)" 202 | ] 203 | }, 204 | { 205 | "cell_type": "code", 206 | "execution_count": null, 207 | "metadata": {}, 208 | "outputs": [], 209 | "source": [ 210 | "data = pd.read_excel('./data/AKL_aero_rain_monthly.xlsx', sheetname='AKL',index_col=1)" 211 | ] 212 | }, 213 | { 214 | "cell_type": "code", 215 | "execution_count": null, 216 | "metadata": {}, 217 | "outputs": [], 218 | "source": [ 219 | "data.head()" 220 | ] 221 | }, 222 | { 223 | "cell_type": "markdown", 224 | "metadata": {}, 225 | "source": [ 226 | "let's look at the empirical distribution (thanks to Pandas)" 227 | ] 228 | }, 229 | { 230 | "cell_type": "code", 231 | "execution_count": null, 232 | "metadata": {}, 233 | "outputs": [], 234 | "source": [ 235 | "data['rain'].hist(normed=True, bins=20)" 236 | ] 237 | }, 238 | { 239 | "cell_type": "markdown", 240 | "metadata": {}, 241 | "source": [ 242 | "There are a few possible choices, but one suitable alternative is the [**gamma distribution**](http://en.wikipedia.org/wiki/Gamma_distribution):\n", 243 | "\n", 244 | "There are three different parametrizations in common use, we're gonna use the the one below, with shape parameter $\\alpha$ and an inverse scale parameter $\\beta$ called a *rate parameter*.\n", 245 | "\n", 246 | "
\n", 247 | "$$x \\sim \\text{Gamma}(\\alpha, \\beta) = \\frac{\\beta^{\\alpha}x^{\\alpha-1}e^{-\\beta x}}{\\Gamma(\\alpha)}$$\n", 248 | "
" 249 | ] 250 | }, 251 | { 252 | "cell_type": "markdown", 253 | "metadata": {}, 254 | "source": [ 255 | "##### Method of moments" 256 | ] 257 | }, 258 | { 259 | "cell_type": "markdown", 260 | "metadata": {}, 261 | "source": [ 262 | "The ***method of moments*** simply assigns the empirical mean and variance to their theoretical counterparts, so that we can solve for the parameters.\n", 263 | "\n", 264 | "So, for the gamma distribution, it turns out (see the relevant section in [the wikipedia article](http://en.wikipedia.org/wiki/Gamma_distribution)) that the mean and variance are:\n", 265 | "\n", 266 | "
\n", 267 | "$$ \\hat{\\mu} = \\alpha \\beta $$\n", 268 | "$$ \\hat{\\sigma}^2 = \\alpha \\beta^2 $$\n", 269 | "
" 270 | ] 271 | }, 272 | { 273 | "cell_type": "markdown", 274 | "metadata": {}, 275 | "source": [ 276 | "So, if we solve for $\\alpha$ and $\\beta$, using the **sample** mean ($\\bar{X}$) and variance ($S^2$), we can use a gamma distribution to describe our data:\n", 277 | "\n", 278 | "
\n", 279 | "$$ \\alpha = \\frac{\\bar{X}^2}{S^2}, \\, \\beta = \\frac{S^2}{\\bar{X}} $$\n", 280 | "
" 281 | ] 282 | }, 283 | { 284 | "cell_type": "markdown", 285 | "metadata": {}, 286 | "source": [ 287 | "first step: we calculate the **sample mean and variance**, using pandas convenience methods " 288 | ] 289 | }, 290 | { 291 | "cell_type": "code", 292 | "execution_count": null, 293 | "metadata": { 294 | "collapsed": true 295 | }, 296 | "outputs": [], 297 | "source": [ 298 | "precip_mean = data['rain'].mean() # sample mean \n", 299 | "precip_var = data['rain'].var() # sample variance" 300 | ] 301 | }, 302 | { 303 | "cell_type": "code", 304 | "execution_count": null, 305 | "metadata": {}, 306 | "outputs": [], 307 | "source": [ 308 | "print(\"mean: {:4.2f}\\nvariance: {:4.2f}\".format(precip_mean, precip_var))" 309 | ] 310 | }, 311 | { 312 | "cell_type": "markdown", 313 | "metadata": {}, 314 | "source": [ 315 | "second step: we calculate the parameters $\\alpha$ and $\\beta$ using the relations above" 316 | ] 317 | }, 318 | { 319 | "cell_type": "code", 320 | "execution_count": null, 321 | "metadata": { 322 | "collapsed": true 323 | }, 324 | "outputs": [], 325 | "source": [ 326 | "alpha_mom = precip_mean ** 2 / precip_var\n", 327 | "beta_mom = precip_var / precip_mean" 328 | ] 329 | }, 330 | { 331 | "cell_type": "markdown", 332 | "metadata": {}, 333 | "source": [ 334 | "We import the ```gamma``` function from the scipy.stats.distributions sub-package \n", 335 | "The implementation of the Gamma function in scipy requires a *location* parameter, left to zero" 336 | ] 337 | }, 338 | { 339 | "cell_type": "code", 340 | "execution_count": null, 341 | "metadata": { 342 | "collapsed": true 343 | }, 344 | "outputs": [], 345 | "source": [ 346 | "from scipy.stats.distributions import gamma " 347 | ] 348 | }, 349 | { 350 | "cell_type": "code", 351 | "execution_count": null, 352 | "metadata": { 353 | "collapsed": true 354 | }, 355 | "outputs": [], 356 | "source": [ 357 | "gmom = gamma(alpha_mom, 0, beta_mom)\n", 358 | "\n", 359 | "#or gmom = gamma(alpha_mom, scale=beta_mom)" 360 | ] 361 | }, 362 | { 363 | "cell_type": "code", 364 | "execution_count": null, 365 | "metadata": {}, 366 | "outputs": [], 367 | "source": [ 368 | "plt.hist(data['rain'], normed=True, bins=20)\n", 369 | "plt.plot(np.linspace(0, 300, 100), gmom.pdf(np.linspace(0, 300, 100)), lw=3, c='r')" 370 | ] 371 | }, 372 | { 373 | "cell_type": "markdown", 374 | "metadata": {}, 375 | "source": [ 376 | "##### Maximum Likelihood Estimates using the .fit() method" 377 | ] 378 | }, 379 | { 380 | "cell_type": "code", 381 | "execution_count": null, 382 | "metadata": { 383 | "collapsed": true 384 | }, 385 | "outputs": [], 386 | "source": [ 387 | "shape,loc,scale = gamma.fit(data['rain'])" 388 | ] 389 | }, 390 | { 391 | "cell_type": "code", 392 | "execution_count": null, 393 | "metadata": {}, 394 | "outputs": [], 395 | "source": [ 396 | "shape, loc, scale" 397 | ] 398 | }, 399 | { 400 | "cell_type": "code", 401 | "execution_count": null, 402 | "metadata": {}, 403 | "outputs": [], 404 | "source": [ 405 | "alpha_mom, beta_mom" 406 | ] 407 | }, 408 | { 409 | "cell_type": "code", 410 | "execution_count": null, 411 | "metadata": { 412 | "collapsed": true 413 | }, 414 | "outputs": [], 415 | "source": [ 416 | "gmle = gamma(shape,loc,scale)" 417 | ] 418 | }, 419 | { 420 | "cell_type": "code", 421 | "execution_count": null, 422 | "metadata": {}, 423 | "outputs": [], 424 | "source": [ 425 | "f, (ax0, ax1) = plt.subplots(1,2,figsize=(13,5))\n", 426 | "\n", 427 | "rmax = 300; N=100\n", 428 | "\n", 429 | "ax0.hist(data['rain'], normed=True, bins=20, color='0.6')\n", 430 | "ax0.plot(np.linspace(0, rmax, N), gmle.pdf(np.linspace(0, rmax, N)), 'b-', lw=2, label='MLE')\n", 431 | "ax0.plot(np.linspace(0, rmax, N), gmom.pdf(np.linspace(0, rmax, N)), 'r-', lw=2, label='Moments')\n", 432 | "ax0.legend()\n", 433 | "ax0.set_title('Histogram and fitted PDFs')\n", 434 | "ax1.plot(np.linspace(0, rmax, N), gmle.cdf(np.linspace(0, rmax, N)), 'b-', lw=2, label='MLE')\n", 435 | "ax1.plot(np.linspace(0, rmax, N), gmom.cdf(np.linspace(0, rmax, N)), 'r-', lw=2, label='Moments')\n", 436 | "ax1.legend()\n", 437 | "ax1.set_title('CDFs');" 438 | ] 439 | }, 440 | { 441 | "cell_type": "markdown", 442 | "metadata": {}, 443 | "source": [ 444 | "**Goodness of fit** tests are available in scipy.stats, through e.g. \n", 445 | "\n", 446 | "+ the Kolmogorov-Smirnoff test (`scipy.stats.kstest`)\n", 447 | "+ the Anderson-Darling test (`scipy.stats.anderson`)" 448 | ] 449 | }, 450 | { 451 | "cell_type": "code", 452 | "execution_count": null, 453 | "metadata": { 454 | "collapsed": true 455 | }, 456 | "outputs": [], 457 | "source": [ 458 | "from scipy.stats import kstest" 459 | ] 460 | }, 461 | { 462 | "cell_type": "code", 463 | "execution_count": null, 464 | "metadata": {}, 465 | "outputs": [], 466 | "source": [ 467 | "kstest(data['rain'], 'gamma', (shape, loc, scale))" 468 | ] 469 | }, 470 | { 471 | "cell_type": "code", 472 | "execution_count": null, 473 | "metadata": {}, 474 | "outputs": [], 475 | "source": [ 476 | "kstest(data['rain'], 'gamma', (alpha_mom, 0, beta_mom))" 477 | ] 478 | }, 479 | { 480 | "cell_type": "markdown", 481 | "metadata": {}, 482 | "source": [ 483 | "### Non Parametric Density estimation: Kernel Density Estimation" 484 | ] 485 | }, 486 | { 487 | "cell_type": "markdown", 488 | "metadata": {}, 489 | "source": [ 490 | "In some instances, we may not be interested in the parameters of a particular distribution of data, but just a smoothed representation of the data. In this case, we can estimate the distribution *non-parametrically* (i.e. making no assumptions about the form of the underlying distribution) using [Kernel Density Estimation](http://en.wikipedia.org/wiki/Kernel_density_estimation).\n", 491 | "\n", 492 | "If you are interested into an excellent discussion of the various options in Python to perform Kernel Density Estimation, have a look at this [post](http://jakevdp.github.io/blog/2013/12/01/kernel-density-estimation/) on Jake VanderPlas blog [Pythonic Perambulations](http://jakevdp.github.io/). Some more stuff on KDE is available [here](http://www.mglerner.com/blog/?p=28) from Michael Lerner. " 493 | ] 494 | }, 495 | { 496 | "cell_type": "markdown", 497 | "metadata": {}, 498 | "source": [ 499 | "In a nutschell, (Gaussian) Kernel density estimation works 'simply' by summing up Gaussian functions (PDFs) centered on each data point. Each Gaussian function is characterized by a location parameter (the value of the data point), and a scale parameter ($\\sigma$) which is related to the 'bandwith' of your (Gaussian) kernel." 500 | ] 501 | }, 502 | { 503 | "cell_type": "code", 504 | "execution_count": null, 505 | "metadata": {}, 506 | "outputs": [], 507 | "source": [ 508 | "# Some random data\n", 509 | "y = np.random.random(15) * 10\n", 510 | "y" 511 | ] 512 | }, 513 | { 514 | "cell_type": "code", 515 | "execution_count": null, 516 | "metadata": {}, 517 | "outputs": [], 518 | "source": [ 519 | "x = np.linspace(0, 10, 100)\n", 520 | "# Smoothing parameter\n", 521 | "s = 0.4\n", 522 | "# Calculate the kernels\n", 523 | "kernels = np.transpose([stats.distributions.norm.pdf(x, yi, s) for yi in y])\n", 524 | "plt.plot(x, kernels, 'k:', lw=1)\n", 525 | "plt.plot(x, kernels.sum(1), lw=1.5)\n", 526 | "plt.plot(y, np.zeros(len(y)), 'ro', ms=10)" 527 | ] 528 | }, 529 | { 530 | "cell_type": "markdown", 531 | "metadata": {}, 532 | "source": [ 533 | "#### Univariate KDE" 534 | ] 535 | }, 536 | { 537 | "cell_type": "markdown", 538 | "metadata": {}, 539 | "source": [ 540 | "Kernel Density Estimation is implemented in scipy.stats.kde by [scipy.stats.kde.gaussian_kde](http://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.gaussian_kde.html)" 541 | ] 542 | }, 543 | { 544 | "cell_type": "code", 545 | "execution_count": null, 546 | "metadata": { 547 | "collapsed": true 548 | }, 549 | "outputs": [], 550 | "source": [ 551 | "from scipy.stats.kde import gaussian_kde" 552 | ] 553 | }, 554 | { 555 | "cell_type": "code", 556 | "execution_count": null, 557 | "metadata": {}, 558 | "outputs": [], 559 | "source": [ 560 | "f, ax = plt.subplots(figsize=(7,7))\n", 561 | "\n", 562 | "xgrid = np.linspace(data['rain'].min(), data['rain'].max(), 100)\n", 563 | "density = gaussian_kde(data['rain']).evaluate(xgrid)\n", 564 | "\n", 565 | "ax.hist(data['rain'], bins=20, normed=True, label='data', alpha=.6)\n", 566 | "ax.plot(xgrid, density, 'r-', lw=2, label='KDE')\n", 567 | "ax.legend()" 568 | ] 569 | }, 570 | { 571 | "cell_type": "markdown", 572 | "metadata": {}, 573 | "source": [ 574 | "#### Multivariate Kernel Density Estimation" 575 | ] 576 | }, 577 | { 578 | "cell_type": "markdown", 579 | "metadata": {}, 580 | "source": [ 581 | "An example in 2 dimensions (from [https://gist.github.com/endolith/1035069](https://gist.github.com/endolith/1035069))" 582 | ] 583 | }, 584 | { 585 | "cell_type": "code", 586 | "execution_count": null, 587 | "metadata": {}, 588 | "outputs": [], 589 | "source": [ 590 | "# Create some dummy data\n", 591 | "rvs = np.append(stats.norm.rvs(loc=2,scale=1,size=(200,1)),\n", 592 | " stats.norm.rvs(loc=1,scale=3,size=(200,1)),\n", 593 | " axis=1)\n", 594 | " \n", 595 | "kde = stats.kde.gaussian_kde(rvs.T)\n", 596 | " \n", 597 | "# Regular grid to evaluate kde upon\n", 598 | "x_flat = np.r_[rvs[:,0].min():rvs[:,0].max():128j]\n", 599 | "y_flat = np.r_[rvs[:,1].min():rvs[:,1].max():128j]\n", 600 | "\n", 601 | "x,y = np.meshgrid(x_flat,y_flat)\n", 602 | "\n", 603 | "grid_coords = np.append(x.reshape(-1,1),y.reshape(-1,1),axis=1)\n", 604 | " \n", 605 | "z = kde(grid_coords.T)\n", 606 | "z = z.reshape(128,128)\n", 607 | " \n", 608 | "# Plot\n", 609 | "f, ax = plt.subplots(figsize=(8,8))\n", 610 | "ax.scatter(rvs[:,0],rvs[:,1],alpha=0.8,color='white')\n", 611 | "im = ax.imshow(z,aspect=x_flat.ptp()/y_flat.ptp(),origin='lower',\\\n", 612 | " extent=(rvs[:,0].min(),rvs[:,0].max(),rvs[:,1].min(),rvs[:,1].max()),\\\n", 613 | " cmap=plt.get_cmap('spectral'))\n", 614 | "plt.colorbar(im, shrink=0.8);" 615 | ] 616 | }, 617 | { 618 | "cell_type": "markdown", 619 | "metadata": {}, 620 | "source": [ 621 | "### Statistical tests " 622 | ] 623 | }, 624 | { 625 | "cell_type": "markdown", 626 | "metadata": {}, 627 | "source": [ 628 | "One commonly used parametric test is the *Student t-test*, which determines whether two independant samples have - statistically - different means, under the assumption that the two samples are normally distributed. The Null hypothesis is that the two samples have **identical** means." 629 | ] 630 | }, 631 | { 632 | "cell_type": "code", 633 | "execution_count": null, 634 | "metadata": { 635 | "collapsed": true 636 | }, 637 | "outputs": [], 638 | "source": [ 639 | "stats.ttest_ind?" 640 | ] 641 | }, 642 | { 643 | "cell_type": "code", 644 | "execution_count": null, 645 | "metadata": { 646 | "collapsed": true 647 | }, 648 | "outputs": [], 649 | "source": [ 650 | "X = stats.distributions.norm(loc=10, scale=10)\n", 651 | "Y = stats.distributions.norm(loc=1, scale=10)\n", 652 | "\n", 653 | "Xrvs = X.rvs(size=(1000))\n", 654 | "Yrvs = Y.rvs(size=(1000))" 655 | ] 656 | }, 657 | { 658 | "cell_type": "code", 659 | "execution_count": null, 660 | "metadata": {}, 661 | "outputs": [], 662 | "source": [ 663 | "f, ax = plt.subplots(figsize=(6,5))\n", 664 | "ax.hist(Xrvs,histtype='stepfilled', bins=20, color='coral', alpha=.6, label=r'$\\mu = 5$')\n", 665 | "ax.hist(Yrvs,histtype='stepfilled', bins=20, color='steelblue', alpha=.6, label=r'$\\mu = 3$')\n", 666 | "ax.legend();" 667 | ] 668 | }, 669 | { 670 | "cell_type": "code", 671 | "execution_count": null, 672 | "metadata": {}, 673 | "outputs": [], 674 | "source": [ 675 | "t, p = stats.ttest_ind(Xrvs, Yrvs)\n", 676 | "print(\"\"\"\\n\n", 677 | "T statistics = {0} ~= {0:4.2f}\n", 678 | "P-value = {1} ~= {1:4.2f}\n", 679 | "\"\"\".format(t, p))" 680 | ] 681 | }, 682 | { 683 | "cell_type": "markdown", 684 | "metadata": {}, 685 | "source": [ 686 | "### Correlation (Pearson's R)" 687 | ] 688 | }, 689 | { 690 | "cell_type": "code", 691 | "execution_count": null, 692 | "metadata": { 693 | "collapsed": true 694 | }, 695 | "outputs": [], 696 | "source": [ 697 | "from scipy.stats import pearsonr" 698 | ] 699 | }, 700 | { 701 | "cell_type": "markdown", 702 | "metadata": {}, 703 | "source": [ 704 | "We're gonna apply that to Time-Series of NINO3.4 and the SOI \n", 705 | "\n", 706 | "The NINO indices come from [http://www.cpc.ncep.noaa.gov/data/indices/ersst3b.nino.mth.81-10.ascii](http://www.cpc.ncep.noaa.gov/data/indices/ersst3b.nino.mth.81-10.ascii)\n", 707 | "\n", 708 | "The SOI is in the `data` directory" 709 | ] 710 | }, 711 | { 712 | "cell_type": "code", 713 | "execution_count": null, 714 | "metadata": { 715 | "collapsed": true 716 | }, 717 | "outputs": [], 718 | "source": [ 719 | "import os\n", 720 | "from datetime import datetime\n", 721 | "from dateutil import parser\n", 722 | "import pandas as pd" 723 | ] 724 | }, 725 | { 726 | "cell_type": "code", 727 | "execution_count": null, 728 | "metadata": { 729 | "collapsed": true 730 | }, 731 | "outputs": [], 732 | "source": [ 733 | "nino_url = \"http://www.cpc.ncep.noaa.gov/data/indices/ersst3b.nino.mth.81-10.ascii\"" 734 | ] 735 | }, 736 | { 737 | "cell_type": "code", 738 | "execution_count": null, 739 | "metadata": { 740 | "collapsed": true 741 | }, 742 | "outputs": [], 743 | "source": [ 744 | "nino = pd.read_table(nino_url, sep='\\s+', engine='python')\n", 745 | "# if not working try: \n", 746 | "# nino = pd.read_table('../data/ersst3b.nino.mth.81-10.ascii', sep='\\s+', engine='python')\n", 747 | "dates = [datetime(x[0], x[1], 1) for x in zip(nino['YR'].values, nino['MON'].values)]\n", 748 | "nino.index = dates" 749 | ] 750 | }, 751 | { 752 | "cell_type": "code", 753 | "execution_count": null, 754 | "metadata": {}, 755 | "outputs": [], 756 | "source": [ 757 | "nino.tail()" 758 | ] 759 | }, 760 | { 761 | "cell_type": "code", 762 | "execution_count": null, 763 | "metadata": { 764 | "collapsed": true 765 | }, 766 | "outputs": [], 767 | "source": [ 768 | "soipath = './data'\n", 769 | "def get_SOI(soipath, start_date='1950-01-01'):\n", 770 | " soi = pd.read_csv(os.path.join(soipath,'NICO_NIWA_SOI_1941_2010.csv'), index_col=0)\n", 771 | " soi = soi.stack()\n", 772 | " soi = soi.dropna()\n", 773 | " dates = [parser.parse(\"%s-%s-%s\" % (str(int(x[0])), x[1], \"1\")) for x in soi.index]\n", 774 | " soidf = pd.DataFrame(soi.values, index=dates, columns=['SOI'])\n", 775 | " soidf = soidf.truncate(before=start_date)\n", 776 | " return soidf" 777 | ] 778 | }, 779 | { 780 | "cell_type": "markdown", 781 | "metadata": {}, 782 | "source": [ 783 | "creates another DataFrame containing the SOI" 784 | ] 785 | }, 786 | { 787 | "cell_type": "code", 788 | "execution_count": null, 789 | "metadata": {}, 790 | "outputs": [], 791 | "source": [ 792 | "df = get_SOI(soipath)" 793 | ] 794 | }, 795 | { 796 | "cell_type": "code", 797 | "execution_count": null, 798 | "metadata": {}, 799 | "outputs": [], 800 | "source": [ 801 | "df.tail()" 802 | ] 803 | }, 804 | { 805 | "cell_type": "markdown", 806 | "metadata": {}, 807 | "source": [ 808 | "Will automaticall **ALIGN** the DataFrames, see [Pandas.ipynb](./Pandas.ipynb)" 809 | ] 810 | }, 811 | { 812 | "cell_type": "code", 813 | "execution_count": null, 814 | "metadata": { 815 | "collapsed": true 816 | }, 817 | "outputs": [], 818 | "source": [ 819 | "df['nino'] = nino['ANOM.3']" 820 | ] 821 | }, 822 | { 823 | "cell_type": "code", 824 | "execution_count": null, 825 | "metadata": { 826 | "collapsed": true 827 | }, 828 | "outputs": [], 829 | "source": [ 830 | "df.dropna(inplace=True)" 831 | ] 832 | }, 833 | { 834 | "cell_type": "code", 835 | "execution_count": null, 836 | "metadata": {}, 837 | "outputs": [], 838 | "source": [ 839 | "r, p = pearsonr(df['SOI'], df['nino'])\n", 840 | "\n", 841 | "print(\"R = {0:<4.2f}, p-value = {1:<4.2f}\".format(r,p))" 842 | ] 843 | }, 844 | { 845 | "cell_type": "code", 846 | "execution_count": null, 847 | "metadata": { 848 | "collapsed": true 849 | }, 850 | "outputs": [], 851 | "source": [ 852 | "df = pd.DataFrame({'SOI':soi['SOI'],'nino':nino['ANOM.3']}, index=nino.index)" 853 | ] 854 | }, 855 | { 856 | "cell_type": "code", 857 | "execution_count": null, 858 | "metadata": {}, 859 | "outputs": [], 860 | "source": [ 861 | "df.head()" 862 | ] 863 | }, 864 | { 865 | "cell_type": "code", 866 | "execution_count": null, 867 | "metadata": { 868 | "collapsed": true 869 | }, 870 | "outputs": [], 871 | "source": [ 872 | "df.to_csv('./data/soi_nino.csv')" 873 | ] 874 | }, 875 | { 876 | "cell_type": "markdown", 877 | "metadata": {}, 878 | "source": [ 879 | "## Interpolation: Scipy.interpolate" 880 | ] 881 | }, 882 | { 883 | "cell_type": "markdown", 884 | "metadata": {}, 885 | "source": [ 886 | "There are several interfaces for performing 1D, 2D or N Dimensional interpolation using scipy.\n", 887 | "\n", 888 | "For more on that I refer you to \n", 889 | "\n", 890 | "+ [http://docs.scipy.org/doc/scipy/reference/interpolate.html](http://docs.scipy.org/doc/scipy/reference/interpolate.html)\n", 891 | "+ [http://wiki.scipy.org/Cookbook/Interpolation](http://wiki.scipy.org/Cookbook/Interpolation)\n", 892 | "\n", 893 | "Here we're going to see one simple example in 1D and 2D using [Radial Basis Functions](http://en.wikipedia.org/wiki/Radial_basis_function) using the default (Multiquadric)" 894 | ] 895 | }, 896 | { 897 | "cell_type": "markdown", 898 | "metadata": {}, 899 | "source": [ 900 | "### 1D interpolation" 901 | ] 902 | }, 903 | { 904 | "cell_type": "code", 905 | "execution_count": null, 906 | "metadata": { 907 | "collapsed": true 908 | }, 909 | "outputs": [], 910 | "source": [ 911 | "from scipy.interpolate import interp1d" 912 | ] 913 | }, 914 | { 915 | "cell_type": "code", 916 | "execution_count": null, 917 | "metadata": { 918 | "collapsed": true 919 | }, 920 | "outputs": [], 921 | "source": [ 922 | "x = np.linspace(0, 10, 10)\n", 923 | "y = np.sin(x)" 924 | ] 925 | }, 926 | { 927 | "cell_type": "code", 928 | "execution_count": null, 929 | "metadata": {}, 930 | "outputs": [], 931 | "source": [ 932 | "x" 933 | ] 934 | }, 935 | { 936 | "cell_type": "code", 937 | "execution_count": null, 938 | "metadata": {}, 939 | "outputs": [], 940 | "source": [ 941 | "plt.plot(x,y,'ro')" 942 | ] 943 | }, 944 | { 945 | "cell_type": "code", 946 | "execution_count": null, 947 | "metadata": { 948 | "collapsed": true 949 | }, 950 | "outputs": [], 951 | "source": [ 952 | "f1 = interp1d(x, y) # default is linear" 953 | ] 954 | }, 955 | { 956 | "cell_type": "code", 957 | "execution_count": null, 958 | "metadata": { 959 | "collapsed": true 960 | }, 961 | "outputs": [], 962 | "source": [ 963 | "f2 = interp1d(x, y, kind='cubic') # and we're gonna try cubic interpolation for comparison" 964 | ] 965 | }, 966 | { 967 | "cell_type": "code", 968 | "execution_count": null, 969 | "metadata": { 970 | "collapsed": true 971 | }, 972 | "outputs": [], 973 | "source": [ 974 | "xnew = np.linspace(0, 10, 50)" 975 | ] 976 | }, 977 | { 978 | "cell_type": "code", 979 | "execution_count": null, 980 | "metadata": { 981 | "collapsed": true 982 | }, 983 | "outputs": [], 984 | "source": [ 985 | "ylin = f1(xnew)\n", 986 | "ycub = f2(xnew)" 987 | ] 988 | }, 989 | { 990 | "cell_type": "code", 991 | "execution_count": null, 992 | "metadata": {}, 993 | "outputs": [], 994 | "source": [ 995 | "f, ax = plt.subplots(figsize=(10,6))\n", 996 | "ax.plot(xnew, ylin, 'b.', ms=8, label='linear')\n", 997 | "ax.plot(xnew, ycub, 'gx-', ms=10, label='cubic')\n", 998 | "ax.plot(x,y,'ro-', label='data')\n", 999 | "ax.legend(loc='lower left')" 1000 | ] 1001 | }, 1002 | { 1003 | "cell_type": "markdown", 1004 | "metadata": {}, 1005 | "source": [ 1006 | "### 2D interpolation using Radial Basis Functions" 1007 | ] 1008 | }, 1009 | { 1010 | "cell_type": "code", 1011 | "execution_count": null, 1012 | "metadata": { 1013 | "collapsed": true 1014 | }, 1015 | "outputs": [], 1016 | "source": [ 1017 | "from scipy.interpolate import Rbf" 1018 | ] 1019 | }, 1020 | { 1021 | "cell_type": "code", 1022 | "execution_count": null, 1023 | "metadata": { 1024 | "collapsed": true 1025 | }, 1026 | "outputs": [], 1027 | "source": [ 1028 | "# defines a function of X and Y\n", 1029 | "def func(x,y):\n", 1030 | " return x*np.exp(-x**2-y**2)" 1031 | ] 1032 | }, 1033 | { 1034 | "cell_type": "code", 1035 | "execution_count": null, 1036 | "metadata": { 1037 | "collapsed": true 1038 | }, 1039 | "outputs": [], 1040 | "source": [ 1041 | "np.random.seed(1) # we 'fix' the generation of random numbers so that we've got consistent results\n", 1042 | "\n", 1043 | "x = np.random.uniform(-2., 2., 100)\n", 1044 | "y = np.random.uniform(-2., 2., 100)\n", 1045 | "\n", 1046 | "z = func(x,y)\n", 1047 | "\n", 1048 | "ti = np.linspace(-2.0, 2.0, 100)\n", 1049 | "\n", 1050 | "XI, YI = np.meshgrid(ti, ti) # meshgrid creates uniform 2D grids from 1D vectors\n", 1051 | "\n", 1052 | "# use RBF\n", 1053 | "rbf = Rbf(x, y, z, epsilon=2) # instantiates the interpolator\n", 1054 | "# you might want to play with the epsilon optional parameter \n", 1055 | "\n", 1056 | "ZI = rbf(XI, YI) # interpolate on grid" 1057 | ] 1058 | }, 1059 | { 1060 | "cell_type": "code", 1061 | "execution_count": null, 1062 | "metadata": { 1063 | "collapsed": true 1064 | }, 1065 | "outputs": [], 1066 | "source": [ 1067 | "# this is the 'True' field, the result of the function evaluated on a regular grid\n", 1068 | "true_Z = func(XI, YI)" 1069 | ] 1070 | }, 1071 | { 1072 | "cell_type": "code", 1073 | "execution_count": null, 1074 | "metadata": {}, 1075 | "outputs": [], 1076 | "source": [ 1077 | "# plot the result\n", 1078 | "f, ax = plt.subplots(figsize=(8,6))\n", 1079 | "im = ax.pcolor(XI, YI, ZI, cmap=plt.get_cmap('RdBu_r'))\n", 1080 | "ax.scatter(x, y, 50, z, cmap=plt.get_cmap('RdBu_r'), edgecolor='.5')\n", 1081 | "ax.set_title('RBF interpolation - multiquadrics')\n", 1082 | "ax.set_xlim(-2, 2)\n", 1083 | "ax.set_ylim(-2, 2)\n", 1084 | "plt.colorbar(im, orientation='vertical', pad=0.06); " 1085 | ] 1086 | }, 1087 | { 1088 | "cell_type": "code", 1089 | "execution_count": null, 1090 | "metadata": {}, 1091 | "outputs": [], 1092 | "source": [ 1093 | "# plot the difference between the 'true' field and the interpolated \n", 1094 | "f, ax = plt.subplots(figsize=(8,6))\n", 1095 | "im = ax.pcolor(XI, YI, ZI - true_Z, cmap=plt.get_cmap('RdBu_r'), vmin=-1e-3, vmax=1e-3)\n", 1096 | "ax.set_xlim(-2, 2)\n", 1097 | "ax.set_ylim(-2, 2)\n", 1098 | "plt.colorbar(im, orientation='vertical', pad=0.06); " 1099 | ] 1100 | }, 1101 | { 1102 | "cell_type": "markdown", 1103 | "metadata": {}, 1104 | "source": [ 1105 | "## Curve fitting with scipy.optimize" 1106 | ] 1107 | }, 1108 | { 1109 | "cell_type": "markdown", 1110 | "metadata": {}, 1111 | "source": [ 1112 | "In this case you want to fit noisy, e.g. experimental, data to **a specific function**, with *unknown parameters*" 1113 | ] 1114 | }, 1115 | { 1116 | "cell_type": "code", 1117 | "execution_count": null, 1118 | "metadata": { 1119 | "collapsed": true 1120 | }, 1121 | "outputs": [], 1122 | "source": [ 1123 | "from scipy.optimize import curve_fit" 1124 | ] 1125 | }, 1126 | { 1127 | "cell_type": "markdown", 1128 | "metadata": {}, 1129 | "source": [ 1130 | "The algorithm uses the [Levenberg-Marquardt](http://en.wikipedia.org/wiki/Levenberg%E2%80%93Marquardt_algorithm) algorithm to perform non-linear least-square optimization" 1131 | ] 1132 | }, 1133 | { 1134 | "cell_type": "code", 1135 | "execution_count": null, 1136 | "metadata": { 1137 | "collapsed": true 1138 | }, 1139 | "outputs": [], 1140 | "source": [ 1141 | "def fitFunc(t, a, b, c):\n", 1142 | " \"\"\" \n", 1143 | " defines the function\n", 1144 | " takes 3 parameters: a, b and c\n", 1145 | " \"\"\"\n", 1146 | " return a*np.exp(-b*t) + c" 1147 | ] 1148 | }, 1149 | { 1150 | "cell_type": "code", 1151 | "execution_count": null, 1152 | "metadata": {}, 1153 | "outputs": [], 1154 | "source": [ 1155 | "### defines the evaluation domain\n", 1156 | "t = np.linspace(0,4,50)\n", 1157 | "\n", 1158 | "### defines the paramaters \n", 1159 | "a = 5.0\n", 1160 | "b = 1.5\n", 1161 | "c = 0.5\n", 1162 | "\n", 1163 | "### create the response\n", 1164 | "temp = fitFunc(t, a, b, c)\n", 1165 | "\n", 1166 | "### add some noise to simulate \"real world observations\" such as experimental data\n", 1167 | "noisy = temp + 0.4 * np.random.normal(size=len(temp)) \n", 1168 | "\n", 1169 | "### use curve_fit to estimate the parameters and the covariance matrices\n", 1170 | "fitParams, fitCovariances = curve_fit(fitFunc, t, noisy)\n", 1171 | "\n", 1172 | "afit, bfit, cfit = tuple(fitParams)\n", 1173 | "\n", 1174 | "print(\"\\nEstimated parameters\\na: {0:<4.2f}, b: {1:<4.2f}, c: {2:<4.2f}\\n\\n\".format(afit, bfit, cfit))" 1175 | ] 1176 | }, 1177 | { 1178 | "cell_type": "code", 1179 | "execution_count": null, 1180 | "metadata": {}, 1181 | "outputs": [], 1182 | "source": [ 1183 | "f, ax = plt.subplots(figsize=(8,8))\n", 1184 | "ax.set_ylabel(u'Temperature (\\xb0C)', fontsize = 16)\n", 1185 | "ax.set_xlabel('time (s)', fontsize = 16)\n", 1186 | "ax.set_xlim(0,4.1)\n", 1187 | "# plot the data as red circles with vertical errorbars\n", 1188 | "ax.errorbar(t, noisy, fmt = 'ro', yerr = 0.2)\n", 1189 | "# now plot the best fit curve \n", 1190 | "\n", 1191 | "ax.plot(t, fitFunc(t, afit, bfit, cfit),'k-', lw=2)\n", 1192 | "\n", 1193 | "# and plot the +- 1 sigma curves\n", 1194 | "# (the square root of the diagonal covariance matrix \n", 1195 | "# element is the uncertainty on the fit parameter.)\n", 1196 | "\n", 1197 | "sigma_a, sigma_b, sigma_c = np.sqrt(fitCovariances[0,0]), \\\n", 1198 | "np.sqrt(fitCovariances[1,1]), \\\n", 1199 | "np.sqrt(fitCovariances[2,2])\n", 1200 | "\n", 1201 | "ax.plot(t, fitFunc(t, afit + sigma_a, bfit - sigma_b, cfit + sigma_c), 'b-')\n", 1202 | "ax.plot(t, fitFunc(t, afit - sigma_a, bfit + sigma_b, cfit - sigma_c), 'b-');" 1203 | ] 1204 | }, 1205 | { 1206 | "cell_type": "markdown", 1207 | "metadata": {}, 1208 | "source": [ 1209 | "### A little exercise of curve fitting " 1210 | ] 1211 | }, 1212 | { 1213 | "cell_type": "markdown", 1214 | "metadata": {}, 1215 | "source": [ 1216 | "solution available at [http://nbviewer.ipython.org/gist/nicolasfauchereau/79131703a0340e1ed82a](http://nbviewer.ipython.org/gist/nicolasfauchereau/79131703a0340e1ed82a)" 1217 | ] 1218 | }, 1219 | { 1220 | "cell_type": "code", 1221 | "execution_count": null, 1222 | "metadata": { 1223 | "collapsed": true 1224 | }, 1225 | "outputs": [], 1226 | "source": [ 1227 | "import pandas as pd" 1228 | ] 1229 | }, 1230 | { 1231 | "cell_type": "code", 1232 | "execution_count": null, 1233 | "metadata": {}, 1234 | "outputs": [], 1235 | "source": [ 1236 | "data= pd.read_excel('./data/rainfall_calibration.xlsx', sheetname='Sheet1')" 1237 | ] 1238 | }, 1239 | { 1240 | "cell_type": "code", 1241 | "execution_count": null, 1242 | "metadata": {}, 1243 | "outputs": [], 1244 | "source": [ 1245 | "data.head()" 1246 | ] 1247 | }, 1248 | { 1249 | "cell_type": "code", 1250 | "execution_count": null, 1251 | "metadata": {}, 1252 | "outputs": [], 1253 | "source": [ 1254 | "f, ax = plt.subplots()\n", 1255 | "ax.plot(data['Rainfall'], data['Level'], 'b', lw=1.5)\n", 1256 | "ax.plot(data['Rainfall'], data['Level'], 'ro')\n", 1257 | "ax.set_xlabel('Rainfall')\n", 1258 | "ax.set_ylabel('Level')" 1259 | ] 1260 | }, 1261 | { 1262 | "cell_type": "markdown", 1263 | "metadata": {}, 1264 | "source": [ 1265 | "..." 1266 | ] 1267 | } 1268 | ], 1269 | "metadata": { 1270 | "anaconda-cloud": {}, 1271 | "kernelspec": { 1272 | "display_name": "Python [default]", 1273 | "language": "python", 1274 | "name": "python3" 1275 | }, 1276 | "language_info": { 1277 | "codemirror_mode": { 1278 | "name": "ipython", 1279 | "version": 3 1280 | }, 1281 | "file_extension": ".py", 1282 | "mimetype": "text/x-python", 1283 | "name": "python", 1284 | "nbconvert_exporter": "python", 1285 | "pygments_lexer": "ipython3", 1286 | "version": "3.5.2" 1287 | }, 1288 | "latex_envs": { 1289 | "bibliofile": "biblio.bib", 1290 | "cite_by": "apalike", 1291 | "current_citInitial": 1, 1292 | "eqLabelWithNumbers": true, 1293 | "eqNumInitial": 0 1294 | }, 1295 | "nav_menu": {}, 1296 | "toc": { 1297 | "colors": { 1298 | "hover_highlight": "#DAA520", 1299 | "running_highlight": "#FF0000", 1300 | "selected_highlight": "#FFD700" 1301 | }, 1302 | "moveMenuLeft": true, 1303 | "nav_menu": { 1304 | "height": "407px", 1305 | "width": "252px" 1306 | }, 1307 | "navigate_menu": true, 1308 | "number_sections": true, 1309 | "sideBar": true, 1310 | "threshold": 4, 1311 | "toc_cell": false, 1312 | "toc_section_display": "block", 1313 | "toc_window_display": false, 1314 | "widenNotebook": false 1315 | } 1316 | }, 1317 | "nbformat": 4, 1318 | "nbformat_minor": 1 1319 | } 1320 | -------------------------------------------------------------------------------- /notebooks/09_Plotting.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Plotting in Python: Matplotlib and others ... " 8 | ] 9 | }, 10 | { 11 | "cell_type": "code", 12 | "execution_count": null, 13 | "metadata": { 14 | "collapsed": true 15 | }, 16 | "outputs": [], 17 | "source": [ 18 | "from IPython.display import Image, HTML" 19 | ] 20 | }, 21 | { 22 | "cell_type": "markdown", 23 | "metadata": {}, 24 | "source": [ 25 | "## MATPLOTLIB" 26 | ] 27 | }, 28 | { 29 | "cell_type": "markdown", 30 | "metadata": {}, 31 | "source": [ 32 | "[Matplotlib](http://matplotlib.org) is the main (static) graphic library for Scientific Python, it allows to create complex and publication-ready plots in a variety of formats (png, ps, pdf ...). It has a *Matlab - like* plotting as well as object-oriented API, you have *very* fine-grained control on everything on your graph (at the expense of being a bit *verbose*), relatively smooth learning curve if you come from *Matlab*, not so much from *R*.\n", 33 | "\n", 34 | "
\n", 35 | "\n", 36 | "[basemap](http://matplotlib.org/basemap/) is a Matplotlib **toolkit** that allows to draw geographical maps and overlay data on these maps, it supports a large number of projections (see [http://matplotlib.org/basemap/users/mapsetup.html](http://matplotlib.org/basemap/users/mapsetup.html) for a list and later on in the present notebook)" 37 | ] 38 | }, 39 | { 40 | "cell_type": "code", 41 | "execution_count": null, 42 | "metadata": { 43 | "collapsed": true 44 | }, 45 | "outputs": [], 46 | "source": [ 47 | "HTML('')" 48 | ] 49 | }, 50 | { 51 | "cell_type": "markdown", 52 | "metadata": {}, 53 | "source": [ 54 | "The [Gallery](http://matplotlib.org/gallery.html) is your goto place when you are looking at creating a brand new \n", 55 | "graph using Matplotlib" 56 | ] 57 | }, 58 | { 59 | "cell_type": "code", 60 | "execution_count": null, 61 | "metadata": { 62 | "collapsed": true 63 | }, 64 | "outputs": [], 65 | "source": [ 66 | "HTML('')" 67 | ] 68 | }, 69 | { 70 | "cell_type": "markdown", 71 | "metadata": {}, 72 | "source": [ 73 | "Colormaps can be tricky, here are some examples and advice for using colormaps with Matplotlib" 74 | ] 75 | }, 76 | { 77 | "cell_type": "code", 78 | "execution_count": null, 79 | "metadata": { 80 | "collapsed": true 81 | }, 82 | "outputs": [], 83 | "source": [ 84 | "HTML('')" 85 | ] 86 | }, 87 | { 88 | "cell_type": "code", 89 | "execution_count": null, 90 | "metadata": { 91 | "collapsed": true 92 | }, 93 | "outputs": [], 94 | "source": [ 95 | "import matplotlib\n", 96 | "from matplotlib import pyplot as plt # conventional way to import the plotting interface of matplotlib\n", 97 | "import numpy as np" 98 | ] 99 | }, 100 | { 101 | "cell_type": "markdown", 102 | "metadata": {}, 103 | "source": [ 104 | "### backends in matplotlib" 105 | ] 106 | }, 107 | { 108 | "cell_type": "markdown", 109 | "metadata": {}, 110 | "source": [ 111 | "Matplotlib can use different `backends`, in the first part of this notebook, we use the IPython inline backend, which is actually part of IPython" 112 | ] 113 | }, 114 | { 115 | "cell_type": "code", 116 | "execution_count": null, 117 | "metadata": { 118 | "collapsed": true 119 | }, 120 | "outputs": [], 121 | "source": [ 122 | "%matplotlib inline" 123 | ] 124 | }, 125 | { 126 | "cell_type": "code", 127 | "execution_count": null, 128 | "metadata": { 129 | "collapsed": true 130 | }, 131 | "outputs": [], 132 | "source": [ 133 | "print(matplotlib.rcParams['backend'])" 134 | ] 135 | }, 136 | { 137 | "cell_type": "code", 138 | "execution_count": null, 139 | "metadata": { 140 | "collapsed": true 141 | }, 142 | "outputs": [], 143 | "source": [ 144 | "matplotlib.rcsetup.all_backends" 145 | ] 146 | }, 147 | { 148 | "cell_type": "code", 149 | "execution_count": null, 150 | "metadata": { 151 | "collapsed": true 152 | }, 153 | "outputs": [], 154 | "source": [ 155 | "x = np.linspace(0, 10, 1000)\n", 156 | "plt.plot(x, np.sin(x))" 157 | ] 158 | }, 159 | { 160 | "cell_type": "markdown", 161 | "metadata": {}, 162 | "source": [ 163 | "we can switch backends, but need to **restart the kernel** \n", 164 | "\n", 165 | "In the following we use the *notebook* backend, which provides some degree of control on the figure \n", 166 | "in the notebook itself" 167 | ] 168 | }, 169 | { 170 | "cell_type": "code", 171 | "execution_count": null, 172 | "metadata": { 173 | "collapsed": true 174 | }, 175 | "outputs": [], 176 | "source": [ 177 | "import matplotlib\n", 178 | "matplotlib.use('nbAgg')" 179 | ] 180 | }, 181 | { 182 | "cell_type": "code", 183 | "execution_count": null, 184 | "metadata": { 185 | "collapsed": true 186 | }, 187 | "outputs": [], 188 | "source": [ 189 | "from matplotlib import pyplot as plt\n", 190 | "import numpy as np\n", 191 | "x = np.linspace(0, 10, 1000)\n", 192 | "plt.plot(x, np.sin(x))\n", 193 | "plt.show()" 194 | ] 195 | }, 196 | { 197 | "cell_type": "markdown", 198 | "metadata": {}, 199 | "source": [ 200 | "we go back to inline plotting via the `%matplotlib inline` magic command" 201 | ] 202 | }, 203 | { 204 | "cell_type": "code", 205 | "execution_count": null, 206 | "metadata": { 207 | "collapsed": true 208 | }, 209 | "outputs": [], 210 | "source": [ 211 | "%matplotlib inline" 212 | ] 213 | }, 214 | { 215 | "cell_type": "code", 216 | "execution_count": null, 217 | "metadata": { 218 | "collapsed": true 219 | }, 220 | "outputs": [], 221 | "source": [ 222 | "import matplotlib\n", 223 | "from matplotlib import pyplot as plt # conventional way to import the plotting interface of matplotlib\n", 224 | "import numpy as np" 225 | ] 226 | }, 227 | { 228 | "cell_type": "markdown", 229 | "metadata": {}, 230 | "source": [ 231 | "### Matlab-like interactive plotting" 232 | ] 233 | }, 234 | { 235 | "cell_type": "markdown", 236 | "metadata": {}, 237 | "source": [ 238 | "Matplotlib was originally designed to have an interface similar to that of matlab.\n", 239 | "For this reason, the library maintains pointers to the currently active figure and\n", 240 | "axes, and makes simple plots straightforward to create. For example:" 241 | ] 242 | }, 243 | { 244 | "cell_type": "code", 245 | "execution_count": null, 246 | "metadata": { 247 | "collapsed": true 248 | }, 249 | "outputs": [], 250 | "source": [ 251 | "x = np.linspace(0, 10, 1000)" 252 | ] 253 | }, 254 | { 255 | "cell_type": "code", 256 | "execution_count": null, 257 | "metadata": { 258 | "collapsed": true 259 | }, 260 | "outputs": [], 261 | "source": [ 262 | "plt.figure() # this is optional: if you just call plt.plot(), a figure will be created\n", 263 | "plt.plot(x, np.sin(x))\n", 264 | "plt.title('plot 1: sine')\n", 265 | "\n", 266 | "plt.figure()\n", 267 | "plt.plot(x, np.cos(x))\n", 268 | "plt.title('plot 2: cosine')" 269 | ] 270 | }, 271 | { 272 | "cell_type": "markdown", 273 | "metadata": {}, 274 | "source": [ 275 | "### Object-oriented Matplotlib" 276 | ] 277 | }, 278 | { 279 | "cell_type": "markdown", 280 | "metadata": {}, 281 | "source": [ 282 | "This is fine for simple plots, but when you start wanting to do more powerful\n", 283 | "things, it can get a little bit tricky. A better way to approach it is to use the **Object-Oriented API** of Matplotlib" 284 | ] 285 | }, 286 | { 287 | "cell_type": "code", 288 | "execution_count": null, 289 | "metadata": { 290 | "collapsed": true 291 | }, 292 | "outputs": [], 293 | "source": [ 294 | "fig1 = plt.figure() # creates a figure object\n", 295 | "ax1 = fig1.add_subplot(1, 1, 1) # creates an 'axes' object in this figure\n", 296 | "ax1.plot(x, np.sin(x)) # create plot in ax\n", 297 | "ax1.set_title('sine') # set properties of the axes\n", 298 | "\n", 299 | "fig2 = plt.figure()\n", 300 | "ax2 = fig2.add_subplot(1, 1, 1)\n", 301 | "ax2.plot(x, np.cos(x))\n", 302 | "ax2.set_title('cosine')" 303 | ] 304 | }, 305 | { 306 | "cell_type": "code", 307 | "execution_count": null, 308 | "metadata": { 309 | "collapsed": true 310 | }, 311 | "outputs": [], 312 | "source": [ 313 | "Image('./images/fig_map.png')" 314 | ] 315 | }, 316 | { 317 | "cell_type": "markdown", 318 | "metadata": {}, 319 | "source": [ 320 | "The big advantage of using the Object-Oriented API is more fined grained control, and also the fact that you can organise your code more (IMHO) logically, making for more readable code" 321 | ] 322 | }, 323 | { 324 | "cell_type": "code", 325 | "execution_count": null, 326 | "metadata": { 327 | "collapsed": true 328 | }, 329 | "outputs": [], 330 | "source": [ 331 | "# creates the figure objects\n", 332 | "fig1 = plt.figure()\n", 333 | "fig2 = plt.figure()\n", 334 | "\n", 335 | "# axes\n", 336 | "ax1 = fig1.add_subplot(1, 1, 1)\n", 337 | "ax2 = fig2.add_subplot(1, 1, 1)\n", 338 | "\n", 339 | "# plot\n", 340 | "ax1.plot(x, np.sin(x))\n", 341 | "ax2.plot(x, np.cos(x))\n", 342 | "\n", 343 | "# set attributes\n", 344 | "ax1.set_title('sine')\n", 345 | "ax2.set_title('cosine')" 346 | ] 347 | }, 348 | { 349 | "cell_type": "code", 350 | "execution_count": null, 351 | "metadata": { 352 | "collapsed": true 353 | }, 354 | "outputs": [], 355 | "source": [ 356 | "x = np.linspace(-5, 5, 10)\n", 357 | "\n", 358 | "y = x ** 2" 359 | ] 360 | }, 361 | { 362 | "cell_type": "code", 363 | "execution_count": null, 364 | "metadata": { 365 | "collapsed": true 366 | }, 367 | "outputs": [], 368 | "source": [ 369 | "fig = plt.figure(figsize=[10,6])\n", 370 | "ax = fig.add_subplot(111)\n", 371 | "\n", 372 | "ax.plot(x, x**2, 'g--', label=\"y = x**2\") \n", 373 | "ax.plot(x, x**3, 'bo', label=\"$y = x^3$\") # can use LaTex notation \n", 374 | "ax.plot(x, x**2 - x**3, \n", 375 | " color='red', # can also abbrev to c='g'\n", 376 | " linestyle='-', # also ls=\n", 377 | " marker='o', # can also be specified in the linespec\n", 378 | " markersize=8, # note this is different from the scatter sizes\n", 379 | " markeredgecolor='#01A9DB', # can use html color codes (see http://html-color-codes.info/)\n", 380 | " markeredgewidth=1,\n", 381 | " alpha=0.5, # sets the transparency\n", 382 | " zorder=-1, # sets its position obove or below other items layering\n", 383 | " label=\"$y = x^3 - x^2$\") # gives the legend name. When using legend() you don't have to give names\n", 384 | "\n", 385 | "ax.set_xlabel('x')\n", 386 | "ax.set_ylabel('y')\n", 387 | "ax.set_title('title', fontsize=16)\n", 388 | "ax.grid()\n", 389 | "ax.legend(loc=0); # upper left corner" 390 | ] 391 | }, 392 | { 393 | "cell_type": "markdown", 394 | "metadata": {}, 395 | "source": [ 396 | "### Other Object Types" 397 | ] 398 | }, 399 | { 400 | "cell_type": "markdown", 401 | "metadata": {}, 402 | "source": [ 403 | "all plot elements in a matplotlib figure is associated to an object that can be passed around, manipulated, updated ... \n", 404 | "\n", 405 | "To illustrate this , we will go back to the `MacOSX` backend (`qt` for Windows): the plots should be displayed in a separate window" 406 | ] 407 | }, 408 | { 409 | "cell_type": "code", 410 | "execution_count": null, 411 | "metadata": { 412 | "collapsed": true 413 | }, 414 | "outputs": [], 415 | "source": [ 416 | "%matplotlib osx " 417 | ] 418 | }, 419 | { 420 | "cell_type": "code", 421 | "execution_count": null, 422 | "metadata": { 423 | "collapsed": true 424 | }, 425 | "outputs": [], 426 | "source": [ 427 | "import matplotlib\n", 428 | "from matplotlib import pyplot as plt\n", 429 | "import numpy as np" 430 | ] 431 | }, 432 | { 433 | "cell_type": "code", 434 | "execution_count": null, 435 | "metadata": { 436 | "collapsed": true 437 | }, 438 | "outputs": [], 439 | "source": [ 440 | "print(matplotlib.rcParams['backend'])" 441 | ] 442 | }, 443 | { 444 | "cell_type": "markdown", 445 | "metadata": {}, 446 | "source": [ 447 | "In a **script** you generally don't need to specify the backend, if you need to you can call: \n", 448 | "```\n", 449 | "import matplotlib \n", 450 | "matplotlib.use('MacOSX ')\n", 451 | "```\n", 452 | "\n", 453 | "At the beginning of the script, **before** you import the pyplot interface with \n", 454 | "\n", 455 | "```\n", 456 | "from matplotlib import pyplot as plt\n", 457 | "```\n", 458 | "\n", 459 | "See the list of available backends (interactive and non-interactive) at: \n", 460 | "\n", 461 | "[http://matplotlib.org/faq/usage_faq.html#what-is-a-backend](http://matplotlib.org/faq/usage_faq.html#what-is-a-backend)\n", 462 | "\n", 463 | "**TIP**: if running a python + matplotlib script using *cron* on a Linux / Mac machine, you my need to use the 'Agg' backend" 464 | ] 465 | }, 466 | { 467 | "cell_type": "code", 468 | "execution_count": null, 469 | "metadata": { 470 | "collapsed": true 471 | }, 472 | "outputs": [], 473 | "source": [ 474 | "x = np.linspace(-np.pi, np.pi, 100)\n", 475 | "\n", 476 | "fig, ax = plt.subplots()\n", 477 | "lines = ax.plot(x, np.sin(x))\n", 478 | "fig.show()" 479 | ] 480 | }, 481 | { 482 | "cell_type": "markdown", 483 | "metadata": {}, 484 | "source": [ 485 | "Now that we've created the plot, we can adjust the attributes of the `Line2D`\n", 486 | "instance. In general, any keyword in the ``plot`` command has an associated\n", 487 | "``set_`` command.\n", 488 | "\n", 489 | "After setting the different parameters, we can call the ``draw()`` method of\n", 490 | "the figure canvas to make the plot reflect the changes." 491 | ] 492 | }, 493 | { 494 | "cell_type": "code", 495 | "execution_count": null, 496 | "metadata": { 497 | "collapsed": true 498 | }, 499 | "outputs": [], 500 | "source": [ 501 | "lines[0].set_color('red')\n", 502 | "fig.canvas.draw()" 503 | ] 504 | }, 505 | { 506 | "cell_type": "code", 507 | "execution_count": null, 508 | "metadata": { 509 | "collapsed": true 510 | }, 511 | "outputs": [], 512 | "source": [ 513 | "lines[0].set_linewidth(6)\n", 514 | "fig.canvas.draw()" 515 | ] 516 | }, 517 | { 518 | "cell_type": "markdown", 519 | "metadata": {}, 520 | "source": [ 521 | "Other plot elements can be manipulated as well. For example, text:" 522 | ] 523 | }, 524 | { 525 | "cell_type": "code", 526 | "execution_count": null, 527 | "metadata": { 528 | "collapsed": true 529 | }, 530 | "outputs": [], 531 | "source": [ 532 | "txt = plt.text(3.5, 0.5, \"Hello Auckland\")" 533 | ] 534 | }, 535 | { 536 | "cell_type": "code", 537 | "execution_count": null, 538 | "metadata": { 539 | "collapsed": true 540 | }, 541 | "outputs": [], 542 | "source": [ 543 | "txt.set_size(20)\n", 544 | "txt.set_color('blue')\n", 545 | "fig.canvas.draw()" 546 | ] 547 | }, 548 | { 549 | "cell_type": "markdown", 550 | "metadata": {}, 551 | "source": [ 552 | "You can also dynamically change the location of the text:" 553 | ] 554 | }, 555 | { 556 | "cell_type": "code", 557 | "execution_count": null, 558 | "metadata": { 559 | "collapsed": true 560 | }, 561 | "outputs": [], 562 | "source": [ 563 | "txt.set_x(6)\n", 564 | "txt.set_y(-0.5)\n", 565 | "fig.canvas.draw()" 566 | ] 567 | }, 568 | { 569 | "cell_type": "markdown", 570 | "metadata": {}, 571 | "source": [ 572 | "Similarly, you can even dynamically change the locations of each point\n", 573 | "in the line! For example, changing the y data looks like this:" 574 | ] 575 | }, 576 | { 577 | "cell_type": "code", 578 | "execution_count": null, 579 | "metadata": { 580 | "collapsed": true 581 | }, 582 | "outputs": [], 583 | "source": [ 584 | "lines[0].set_ydata(np.sin(2 * x))\n", 585 | "fig.canvas.draw()" 586 | ] 587 | }, 588 | { 589 | "cell_type": "code", 590 | "execution_count": null, 591 | "metadata": { 592 | "collapsed": true 593 | }, 594 | "outputs": [], 595 | "source": [ 596 | "# back to the inline backend \n", 597 | "%matplotlib inline " 598 | ] 599 | }, 600 | { 601 | "cell_type": "code", 602 | "execution_count": null, 603 | "metadata": { 604 | "collapsed": true 605 | }, 606 | "outputs": [], 607 | "source": [ 608 | "from IPython.display import Image, HTML" 609 | ] 610 | }, 611 | { 612 | "cell_type": "code", 613 | "execution_count": null, 614 | "metadata": { 615 | "collapsed": true 616 | }, 617 | "outputs": [], 618 | "source": [ 619 | "import matplotlib\n", 620 | "from matplotlib import pyplot as plt\n", 621 | "import numpy as np" 622 | ] 623 | }, 624 | { 625 | "cell_type": "markdown", 626 | "metadata": {}, 627 | "source": [ 628 | "### styles in matplotlib (available from version 1.4)" 629 | ] 630 | }, 631 | { 632 | "cell_type": "code", 633 | "execution_count": null, 634 | "metadata": { 635 | "collapsed": true 636 | }, 637 | "outputs": [], 638 | "source": [ 639 | "from matplotlib import style" 640 | ] 641 | }, 642 | { 643 | "cell_type": "code", 644 | "execution_count": null, 645 | "metadata": { 646 | "collapsed": true 647 | }, 648 | "outputs": [], 649 | "source": [ 650 | "style.available" 651 | ] 652 | }, 653 | { 654 | "cell_type": "code", 655 | "execution_count": null, 656 | "metadata": { 657 | "collapsed": true 658 | }, 659 | "outputs": [], 660 | "source": [ 661 | "with style.context('ggplot'): # use context rather than style.use(), and use with as a context manager\n", 662 | " plt.figure(figsize=(6,6))\n", 663 | " plt.hist(np.random.randn(1000), bins=20, normed=True)\n", 664 | " plt.title('histogram')\n", 665 | " plt.grid()" 666 | ] 667 | }, 668 | { 669 | "cell_type": "markdown", 670 | "metadata": {}, 671 | "source": [ 672 | "[xkcd](http://xkcd.com/)-style plots in Matplotlib" 673 | ] 674 | }, 675 | { 676 | "cell_type": "code", 677 | "execution_count": null, 678 | "metadata": { 679 | "collapsed": true 680 | }, 681 | "outputs": [], 682 | "source": [ 683 | "Image('http://imgs.xkcd.com/comics/front_door.png')" 684 | ] 685 | }, 686 | { 687 | "cell_type": "code", 688 | "execution_count": null, 689 | "metadata": { 690 | "collapsed": true 691 | }, 692 | "outputs": [], 693 | "source": [ 694 | "x = np.linspace(-np.pi, np.pi, 100)" 695 | ] 696 | }, 697 | { 698 | "cell_type": "code", 699 | "execution_count": null, 700 | "metadata": { 701 | "collapsed": true 702 | }, 703 | "outputs": [], 704 | "source": [ 705 | "with plt.xkcd():\n", 706 | " fig = plt.figure()\n", 707 | " ax = fig.add_subplot(1, 1, 1)\n", 708 | " ax.plot(x, np.sin(x))\n", 709 | " ax.set_title('sine')" 710 | ] 711 | }, 712 | { 713 | "cell_type": "code", 714 | "execution_count": null, 715 | "metadata": { 716 | "collapsed": true 717 | }, 718 | "outputs": [], 719 | "source": [ 720 | "with plt.xkcd():\n", 721 | " plt.figure(figsize=(6,6))\n", 722 | " plt.hist(np.random.randn(1000), bins=20, normed=True, color='b', alpha=0.5)\n", 723 | " plt.title('histogram')\n", 724 | " plt.grid()" 725 | ] 726 | }, 727 | { 728 | "cell_type": "markdown", 729 | "metadata": {}, 730 | "source": [ 731 | "you can combine xkcd plots and styles" 732 | ] 733 | }, 734 | { 735 | "cell_type": "code", 736 | "execution_count": null, 737 | "metadata": { 738 | "collapsed": true 739 | }, 740 | "outputs": [], 741 | "source": [ 742 | "with plt.xkcd():\n", 743 | " with style.context('ggplot'): \n", 744 | " plt.figure(figsize=(6,6))\n", 745 | " plt.hist(np.random.randn(1000), bins=20, normed=True, color='b', alpha=0.5)\n", 746 | " plt.title('histogram')\n", 747 | " plt.grid()" 748 | ] 749 | }, 750 | { 751 | "cell_type": "markdown", 752 | "metadata": {}, 753 | "source": [ 754 | "### subplots and layouts" 755 | ] 756 | }, 757 | { 758 | "cell_type": "markdown", 759 | "metadata": {}, 760 | "source": [ 761 | "#### simple subplots: `plt.subplots()`" 762 | ] 763 | }, 764 | { 765 | "cell_type": "code", 766 | "execution_count": null, 767 | "metadata": { 768 | "collapsed": true 769 | }, 770 | "outputs": [], 771 | "source": [ 772 | "f, axes = plt.subplots(nrows=2, ncols=1)" 773 | ] 774 | }, 775 | { 776 | "cell_type": "code", 777 | "execution_count": null, 778 | "metadata": { 779 | "collapsed": true 780 | }, 781 | "outputs": [], 782 | "source": [ 783 | "type(axes)" 784 | ] 785 | }, 786 | { 787 | "cell_type": "code", 788 | "execution_count": null, 789 | "metadata": { 790 | "collapsed": true 791 | }, 792 | "outputs": [], 793 | "source": [ 794 | "axes.shape" 795 | ] 796 | }, 797 | { 798 | "cell_type": "code", 799 | "execution_count": null, 800 | "metadata": { 801 | "collapsed": true 802 | }, 803 | "outputs": [], 804 | "source": [ 805 | "f, axes = plt.subplots(nrows=2, ncols=1)\n", 806 | "axes = axes.flatten()\n", 807 | "axes[0].plot(x, np.sin(x))\n", 808 | "axes[1].plot(x, np.cos(x))" 809 | ] 810 | }, 811 | { 812 | "cell_type": "markdown", 813 | "metadata": {}, 814 | "source": [ 815 | "#### more complex layouts: `plt.subplot2grid() `" 816 | ] 817 | }, 818 | { 819 | "cell_type": "code", 820 | "execution_count": null, 821 | "metadata": { 822 | "collapsed": true 823 | }, 824 | "outputs": [], 825 | "source": [ 826 | "f = plt.figure(figsize=[10,10])\n", 827 | "ax1 = plt.subplot2grid((3,3), (0,0), colspan=1)\n", 828 | "ax2 = plt.subplot2grid((3,3), (1,0), colspan=2)\n", 829 | "ax3 = plt.subplot2grid((3,3), (0,2), rowspan=3)\n", 830 | "ax4 = plt.subplot2grid((3,3), (2,0))\n", 831 | "ax5 = plt.subplot2grid((3,3), (2,1))\n", 832 | "\n", 833 | "f.subplots_adjust(wspace=0.8)\n", 834 | "f.tight_layout() # some nice spacing!\n", 835 | "\n", 836 | "# adding data to each plot\n", 837 | "ax1.plot(np.random.rand(10))\n", 838 | "ax2.plot(np.random.rand(100), np.random.rand(100), '.r')\n", 839 | "ax3.hist(np.random.normal(size=1000), bins=20)\n", 840 | "ax4.fill_between(x, x**2, x**3, color=\"orange\", alpha=0.5)\n", 841 | "ax5.plot(np.random.rand(10), c='g', lw=4, marker='o', ms=10, mec='none')\n", 842 | "ax5.set_ylabel('x axis')" 843 | ] 844 | }, 845 | { 846 | "cell_type": "markdown", 847 | "metadata": {}, 848 | "source": [ 849 | "# seaborn" 850 | ] 851 | }, 852 | { 853 | "cell_type": "markdown", 854 | "metadata": {}, 855 | "source": [ 856 | "[Seaborn](https://seaborn.pydata.org/) is a visualization library based on matplotlib. It provides a high-level interface for drawing attractive statistical graphics." 857 | ] 858 | }, 859 | { 860 | "cell_type": "code", 861 | "execution_count": null, 862 | "metadata": { 863 | "collapsed": true 864 | }, 865 | "outputs": [], 866 | "source": [ 867 | "HTML('')" 868 | ] 869 | }, 870 | { 871 | "cell_type": "markdown", 872 | "metadata": {}, 873 | "source": [ 874 | "The usual way to import seaborn is: \n", 875 | "```python\n", 876 | "import seaborn as sns\n", 877 | "```" 878 | ] 879 | }, 880 | { 881 | "cell_type": "code", 882 | "execution_count": null, 883 | "metadata": { 884 | "collapsed": true 885 | }, 886 | "outputs": [], 887 | "source": [ 888 | "import seaborn as sns" 889 | ] 890 | }, 891 | { 892 | "cell_type": "code", 893 | "execution_count": null, 894 | "metadata": { 895 | "collapsed": true 896 | }, 897 | "outputs": [], 898 | "source": [ 899 | "%matplotlib inline" 900 | ] 901 | }, 902 | { 903 | "cell_type": "code", 904 | "execution_count": null, 905 | "metadata": { 906 | "collapsed": true 907 | }, 908 | "outputs": [], 909 | "source": [ 910 | "y = np.random.randn(1000)" 911 | ] 912 | }, 913 | { 914 | "cell_type": "code", 915 | "execution_count": null, 916 | "metadata": { 917 | "collapsed": true 918 | }, 919 | "outputs": [], 920 | "source": [ 921 | "len(y)" 922 | ] 923 | }, 924 | { 925 | "cell_type": "code", 926 | "execution_count": null, 927 | "metadata": { 928 | "collapsed": true 929 | }, 930 | "outputs": [], 931 | "source": [ 932 | "sns.distplot(y)" 933 | ] 934 | }, 935 | { 936 | "cell_type": "markdown", 937 | "metadata": {}, 938 | "source": [ 939 | "seaborn plays well with Pandas Dataframes" 940 | ] 941 | }, 942 | { 943 | "cell_type": "code", 944 | "execution_count": null, 945 | "metadata": { 946 | "collapsed": true 947 | }, 948 | "outputs": [], 949 | "source": [ 950 | "import pandas as pd" 951 | ] 952 | }, 953 | { 954 | "cell_type": "code", 955 | "execution_count": null, 956 | "metadata": { 957 | "collapsed": true 958 | }, 959 | "outputs": [], 960 | "source": [ 961 | "data = pd.read_csv('./data/Daily_clim_data_Auckland.csv', index_col=0, parse_dates=True)" 962 | ] 963 | }, 964 | { 965 | "cell_type": "code", 966 | "execution_count": null, 967 | "metadata": { 968 | "collapsed": true 969 | }, 970 | "outputs": [], 971 | "source": [ 972 | "data.head()" 973 | ] 974 | }, 975 | { 976 | "cell_type": "code", 977 | "execution_count": null, 978 | "metadata": { 979 | "collapsed": true 980 | }, 981 | "outputs": [], 982 | "source": [ 983 | "sns.distplot(data.loc[:,'tmax'])" 984 | ] 985 | }, 986 | { 987 | "cell_type": "code", 988 | "execution_count": null, 989 | "metadata": { 990 | "collapsed": true 991 | }, 992 | "outputs": [], 993 | "source": [ 994 | "data.loc[:,'month'] = data.index.month" 995 | ] 996 | }, 997 | { 998 | "cell_type": "code", 999 | "execution_count": null, 1000 | "metadata": { 1001 | "collapsed": true 1002 | }, 1003 | "outputs": [], 1004 | "source": [ 1005 | "data.head()" 1006 | ] 1007 | }, 1008 | { 1009 | "cell_type": "code", 1010 | "execution_count": null, 1011 | "metadata": { 1012 | "collapsed": true 1013 | }, 1014 | "outputs": [], 1015 | "source": [ 1016 | "sns.boxplot(x=\"month\", y=\"tmin\", data=data, color=\"steelblue\")" 1017 | ] 1018 | }, 1019 | { 1020 | "cell_type": "code", 1021 | "execution_count": null, 1022 | "metadata": { 1023 | "collapsed": true 1024 | }, 1025 | "outputs": [], 1026 | "source": [ 1027 | "sns.boxplot(x=\"month\", y=\"tmin\", data=data, palette=\"PRGn\")" 1028 | ] 1029 | }, 1030 | { 1031 | "cell_type": "code", 1032 | "execution_count": null, 1033 | "metadata": { 1034 | "collapsed": true 1035 | }, 1036 | "outputs": [], 1037 | "source": [ 1038 | "f, ax = plt.subplots(figsize=(10,8))\n", 1039 | "sns.boxplot(x=\"month\", y=\"tmin\", data=data, palette=\"PRGn\", ax=ax)" 1040 | ] 1041 | }, 1042 | { 1043 | "cell_type": "code", 1044 | "execution_count": null, 1045 | "metadata": { 1046 | "collapsed": true 1047 | }, 1048 | "outputs": [], 1049 | "source": [ 1050 | "data = pd.read_csv('./data/soi_nino.csv', index_col=0, parse_dates=True)" 1051 | ] 1052 | }, 1053 | { 1054 | "cell_type": "code", 1055 | "execution_count": null, 1056 | "metadata": { 1057 | "collapsed": true 1058 | }, 1059 | "outputs": [], 1060 | "source": [ 1061 | "data.head()" 1062 | ] 1063 | }, 1064 | { 1065 | "cell_type": "markdown", 1066 | "metadata": {}, 1067 | "source": [ 1068 | "linear regression plot" 1069 | ] 1070 | }, 1071 | { 1072 | "cell_type": "code", 1073 | "execution_count": null, 1074 | "metadata": { 1075 | "collapsed": true 1076 | }, 1077 | "outputs": [], 1078 | "source": [ 1079 | "sns.lmplot(x=\"nino\", y=\"SOI\", data=data)" 1080 | ] 1081 | }, 1082 | { 1083 | "cell_type": "code", 1084 | "execution_count": null, 1085 | "metadata": { 1086 | "collapsed": true 1087 | }, 1088 | "outputs": [], 1089 | "source": [ 1090 | "sns.jointplot(\"nino\", \"SOI\", data=data, kind=\"reg\", color=\"steelblue\")" 1091 | ] 1092 | }, 1093 | { 1094 | "cell_type": "markdown", 1095 | "metadata": {}, 1096 | "source": [ 1097 | "## Other libraries of interest" 1098 | ] 1099 | }, 1100 | { 1101 | "cell_type": "markdown", 1102 | "metadata": {}, 1103 | "source": [ 1104 | "### [plotnine](https://plotnine.readthedocs.io/en/stable/): R's `GGPLOT2` library, but in Python" 1105 | ] 1106 | }, 1107 | { 1108 | "cell_type": "code", 1109 | "execution_count": null, 1110 | "metadata": { 1111 | "collapsed": true 1112 | }, 1113 | "outputs": [], 1114 | "source": [ 1115 | "HTML('')" 1116 | ] 1117 | }, 1118 | { 1119 | "cell_type": "markdown", 1120 | "metadata": {}, 1121 | "source": [ 1122 | "### [bokeh](https://bokeh.pydata.org/en/latest/): interactive visualisation in the browser with Python" 1123 | ] 1124 | }, 1125 | { 1126 | "cell_type": "code", 1127 | "execution_count": null, 1128 | "metadata": { 1129 | "collapsed": true 1130 | }, 1131 | "outputs": [], 1132 | "source": [ 1133 | "HTML('')" 1134 | ] 1135 | }, 1136 | { 1137 | "cell_type": "markdown", 1138 | "metadata": {}, 1139 | "source": [ 1140 | "### [plotly](https://plot.ly/python/) for Python, also for creating interactive visualisations" 1141 | ] 1142 | }, 1143 | { 1144 | "cell_type": "code", 1145 | "execution_count": null, 1146 | "metadata": { 1147 | "collapsed": true 1148 | }, 1149 | "outputs": [], 1150 | "source": [ 1151 | "HTML('')" 1152 | ] 1153 | }, 1154 | { 1155 | "cell_type": "code", 1156 | "execution_count": null, 1157 | "metadata": { 1158 | "collapsed": true 1159 | }, 1160 | "outputs": [], 1161 | "source": [] 1162 | }, 1163 | { 1164 | "cell_type": "markdown", 1165 | "metadata": {}, 1166 | "source": [ 1167 | "### Jake VanderPlas's talk at scipy 2017 on the Python Visualisation Landscape" 1168 | ] 1169 | }, 1170 | { 1171 | "cell_type": "code", 1172 | "execution_count": null, 1173 | "metadata": { 1174 | "collapsed": true 1175 | }, 1176 | "outputs": [], 1177 | "source": [ 1178 | "from IPython.display import YouTubeVideo" 1179 | ] 1180 | }, 1181 | { 1182 | "cell_type": "code", 1183 | "execution_count": null, 1184 | "metadata": { 1185 | "collapsed": true 1186 | }, 1187 | "outputs": [], 1188 | "source": [ 1189 | "YouTubeVideo('https://www.youtube.com/watch?v=FytuB8nFHPQ')" 1190 | ] 1191 | }, 1192 | { 1193 | "cell_type": "markdown", 1194 | "metadata": {}, 1195 | "source": [ 1196 | "his slides are at [https://speakerdeck.com/jakevdp/pythons-visualization-landscape-pycon-2017](https://speakerdeck.com/jakevdp/pythons-visualization-landscape-pycon-2017)" 1197 | ] 1198 | }, 1199 | { 1200 | "cell_type": "markdown", 1201 | "metadata": {}, 1202 | "source": [ 1203 | "see also [https://rougier.github.io/python-visualization-landscape/landscape-colors.html](https://rougier.github.io/python-visualization-landscape/landscape-colors.html)" 1204 | ] 1205 | }, 1206 | { 1207 | "cell_type": "code", 1208 | "execution_count": null, 1209 | "metadata": { 1210 | "collapsed": true 1211 | }, 1212 | "outputs": [], 1213 | "source": [] 1214 | } 1215 | ], 1216 | "metadata": { 1217 | "kernelspec": { 1218 | "display_name": "Python [conda env:IOOS]", 1219 | "language": "python", 1220 | "name": "conda-env-IOOS-py" 1221 | }, 1222 | "language_info": { 1223 | "codemirror_mode": { 1224 | "name": "ipython", 1225 | "version": 3 1226 | }, 1227 | "file_extension": ".py", 1228 | "mimetype": "text/x-python", 1229 | "name": "python", 1230 | "nbconvert_exporter": "python", 1231 | "pygments_lexer": "ipython3", 1232 | "version": "3.5.2" 1233 | }, 1234 | "toc": { 1235 | "colors": { 1236 | "hover_highlight": "#DAA520", 1237 | "running_highlight": "#FF0000", 1238 | "selected_highlight": "#FFD700" 1239 | }, 1240 | "moveMenuLeft": true, 1241 | "nav_menu": { 1242 | "height": "244px", 1243 | "width": "252px" 1244 | }, 1245 | "navigate_menu": true, 1246 | "number_sections": true, 1247 | "sideBar": true, 1248 | "threshold": 4, 1249 | "toc_cell": false, 1250 | "toc_section_display": "block", 1251 | "toc_window_display": false, 1252 | "widenNotebook": false 1253 | } 1254 | }, 1255 | "nbformat": 4, 1256 | "nbformat_minor": 1 1257 | } 1258 | -------------------------------------------------------------------------------- /notebooks/data/AKL_aero_rain_monthly.xlsx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nicolasfauchereau/Auckland_Python_Workshop/33365e02449818dd7feadc3b20aeb487ed0624cd/notebooks/data/AKL_aero_rain_monthly.xlsx -------------------------------------------------------------------------------- /notebooks/data/ISO_datetime.xls: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nicolasfauchereau/Auckland_Python_Workshop/33365e02449818dd7feadc3b20aeb487ed0624cd/notebooks/data/ISO_datetime.xls -------------------------------------------------------------------------------- /notebooks/data/NICO_NIWA_SOI_1941_2010.csv: -------------------------------------------------------------------------------- 1 | Year,Jan,Feb,Mar,Apr,May,Jun,Jul,Aug,Sep,Oct,Nov,Dec 2 | 1876,1.1397361251378086,0.9632608595046365,0.07224233583061418,0.9576192337672369,0.7930559665177136,1.8308283536948842,-0.5263670810327393,1.264818488148623,0.9858299808907625,-0.7761115040039729,-0.34136503675674623,-0.38278421770953625 3 | 1877,-0.9264129422106144,-0.5950437164583634,-0.40184799305697544,-0.7329147205806765,0.48030150084895523,-1.6019748094826054,-0.9902517387771135,-0.7793326038087578,-1.7530591808597316,-1.548826458756377,-1.3074924992760963,-1.3562125342556495 4 | 1878,-0.8324970755129903,-1.9006502530759175,-1.4553820572516456,-0.6594132443047457,0.32392426801453156,-0.2125068624821333,1.6605177340480646,1.3286982097722322,1.700322805695195,1.06651646502084,1.3976643957780401,1.7177716232582454 5 | 1879,1.2806099251842713,1.2580752387408383,1.3364832128642063,1.2516251388712945,0.32392426801453156,1.749094945047874,2.2569408654337315,2.286894034127241,1.8194049431626342,1.4825937483490628,0.8823964157677493,-0.6389495641690275 6 | 1880,1.0927781917889698,0.6684464802683867,1.4418366192836376,0.5901118523872485,1.3403762814381075,1.013494267224133,0.20259452399414504,1.4564576530195956,0.7476657059559516,0.47212034598056035,0.6247624257626039,-0.28031807912583295 7 | 1881,-0.6916232754665276,-0.5108110366765163,0.23027244545979075,0.14910299473132918,-0.3015846633229853,-0.3759736797763393,-0.5263670810327393,-1.098731211927094,-1.3958127684574815,-2.3215414135087133,0.6247624257626039,0.8980425145878617 8 | 1882,-0.6446653421176889,-0.13176397765851514,0.5463326647182037,0.22260447100726002,0.7930559665177136,-1.1115743576001733,-2.1168287647279516,-2.5040850876478022,-1.5148949059248529,-0.2411549968677347,0.17390294325354452,0.9492755838797716 9 | 1883,0.6231988583006894,0.7947954999410378,-2.403562715026885,1.3986280914232398,1.4967535142724424,0.4413604066945049,-0.9902517387771889,0.17886322054632348,-0.8599431498541401,0.47212034598056035,0.4315369332586899,-1.6123778807151408 10 | 1884,-1.2081605423035933,-0.4686946967856645,0.9677462903961078,-1.2474250545126937,0.24573565159731972,1.01349426722404,-0.2612901337502291,-0.45993399569042154,-0.7408610123867685,0.41268073407658645,-0.21254804175421016,-1.3562125342556495 11 | 1885,-1.5838240090942497,0.12093406168678703,0.5463326647182037,0.0756015184552312,-0.3015846633229853,-1.3567745835413894,-0.4600978442121683,-0.9070920470561212,-0.443155668718238,-1.7271452944684338,-1.6295349867825464,0.43694489096078915 12 | 1886,-0.03421220858289237,0.12093406168678703,0.33562585187928173,0.5166103761112341,0.7148673501005018,0.604827223988618,0.7990176553798873,1.392577931395914,1.2835353245592929,1.304274912637006,0.9468049132690539,1.3591401382149926 13 | 1887,1.2336519918354325,0.9632608595045886,1.0204229936057934,0.9576192337672369,-0.3015846633229853,0.604827223988618,0.533940708097377,0.498261828664587,0.4499603622873534,0.47212034598056035,-0.5989990267618917,0.43694489096078915 14 | 1888,-0.26900187532703257,-0.21599665744031443,-1.0866451347834871,-1.9824398172726707,-0.8489049782433792,-1.5202414008355953,-1.6529441069835022,-0.8432123254324395,-0.9790252873215793,-1.429947234948294,-1.3074924992760963,-0.33155114841768457 15 | 1889,-2.5229826760708103,-0.1738803175494148,-2.614269527865807,0.0756015184552312,-0.0670188140714387,2.3212288055774093,0.20259452399422037,0.2427429421699326,1.0453710496244821,0.41268073407658645,2.1705663657935492,2.127636177593466 16 | 1890,2.078894792114316,0.9632608595045886,1.4418366192836973,0.7371148049392773,0.48030150084895523,0.686560632635721,-0.1950208969295827,-0.2682948308194489,0.8667478434233231,0.35324112217254494,0.17390294325354452,-0.024152732666283448 17 | 1891,1.5623575252772501,-0.34234567711296554,-0.875938321944625,0.5166103761112341,0.08935841876289606,-0.04904004518802021,-0.5926363178533858,-0.8432123254323669,-1.0981074247889508,0.056043062652405135,-0.5345905292606602,-0.536483425585266 18 | 1892,0.29449332485887186,-0.9319744355854649,1.1257764000252843,0.7371148049392773,1.105810432186561,2.0760285796361932,0.799017655379812,0.626021271911878,0.5690424997547927,0.8287580174047416,-0.14813954425290551,0.28324568308505943 19 | 1893,1.1397361251377551,0.6684464802683389,-0.0857877737985624,0.22260447100726002,-0.22339604690586237,1.176961084518246,1.4617100235861253,0.8176604367829232,0.509501431021073,0.7693184055007678,0.17390294325361774,0.07831340591753634 20 | 1894,1.7501892586725516,0.8790281797227892,0.5990093679279492,-0.14490291037272843,-0.37977327974019714,-0.04904004518802021,-0.1950208969295827,-0.5238137173141033,-0.20499139378342718,0.1749222864604881,0.6247624257626039,-0.07538580195819335 21 | 1895,0.5762409249518508,0.2472830813594381,0.019565632620868693,-0.5124102917527169,-0.6925277454090445,-0.3759736797763393,0.0037868135322811775,-0.5876934389377124,-0.4431556687181703,-0.5383530563878746,-0.9210415142683417,-0.43401728700144615 22 | 1896,0.15361952481235572,0.4157484409230367,-0.559878102686152,-0.6594132443047457,-4.054638251348265,-2.9914427564830777,-2.0505595279073052,-2.1846864795293937,-1.9316823870608226,-1.8460245182764492,-1.2430840017747917,-1.509911742131321 23 | 1897,-1.2081605423035933,-0.6792763962401148,-1.5607354636710766,-1.4679294833406533,-1.552602525998108,0.11442677210618579,-0.1950208969295827,0.11498349892256908,-0.02636818758220058,0.1749222864604881,-0.8566330167671102,0.9492755838797716 24 | 1898,0.7171147249983135,0.5420974605956878,1.915926948171287,1.104622186319182,-0.0670188140714387,-0.1307734538351232,0.6664791817385946,0.2427429421699326,0.2713371560863299,-0.06283616115567782,-0.34136503675674623,-0.12661887125010324 25 | 1899,1.32756785853311,0.7947954999410378,1.3891599160739518,0.5166103761112341,-0.6143391289917437,-0.9481075403059673,-0.5263670810327393,-0.9709717686797305,-0.20499139378335948,0.5909995697886433,1.462072893279418,-0.3827842177095945 26 | 1900,-0.6916232754664742,-0.5950437164583634,-2.403562715026825,-1.5414309596166678,-0.6143391289917437,2.729895848812831,1.0640946026623976,0.8176604367829232,-1.6935181121260119,-1.6677056825643923,-0.6634075242631963,-0.6389495641690857 27 | 1901,0.012745724765892962,0.24728308135948598,0.9677462903961078,0.5166103761113177,0.08935841876298495,2.0760285796361004,1.5279792604067717,1.009299601653896,-1.6339770433922922,-2.1432225777966565,-0.9210415142683417,-0.28031807912577467 28 | 1902,1.703231325323713,-0.21599665744031443,1.1784531032350298,0.8106162812152081,0.8712445829349255,0.3596269980474019,0.20259452399422037,-0.8432123254324395,-1.8126002495934512,-0.7166718920999989,-0.4057735342579777,-0.3827842177095945 29 | 1903,-0.879455008861829,-0.9319744355854649,1.7578968385421103,1.6926339965272137,0.8712445829348365,0.03269336345908279,0.6664791817385946,0.051103777298959936,0.8072067746896036,0.4126807340765189,0.04508594825100845,1.512839346090664 30 | 1904,1.4214837252307875,1.426540598304389,0.9677462903960479,2.942159093219208,1.027621815769349,-0.6211739057176483,-0.8577132651359713,0.11498349892256908,-0.026368187582268273,0.11548267455644662,-1.7583519817851556,0.1807795445012979 31 | 1905,-0.879455008861829,-1.5216031940579164,-2.8776530439144143,-3.672973771620584,-3.585506552845171,-3.073176165130088,-2.1168287647278765,-0.715452882185076,-0.7408610123867008,-0.5383530563878746,-1.8227604792863872,-1.4074456035475593 32 | 1906,-0.31595980867587126,-0.6792763962401148,-0.45452469626672093,-0.6594132443046621,0.24573565159731972,-0.2942402711293292,0.7327484185592409,1.584217096266959,1.7598638744289146,0.8881976293088507,2.04174937079094,0.38571182166893747 33 | 1907,0.5292829916030121,0.12093406168678703,0.019565632620868693,0.5166103761113177,1.1058104321864721,0.93176085857703,-0.3938286073915219,-0.7793326038087578,-0.026368187582268273,0.056043062652405135,-0.2769565392554416,0.7955763760041001 34 | 1908,-1.0203288089082918,0.6684464802683867,0.07224233583061418,1.6191325202511995,0.011169802345773125,-0.1307734538351232,0.2688637608148668,0.5621415502882688,1.7003228056952626,0.7693184055007002,0.17390294325354452,-0.6389495641690857 35 | 1909,-0.22204394197819383,-0.30022933722206585,0.01956563262092858,-1.1739235782366793,0.32392426801453156,2.4029622142245124,1.1303638394829687,1.0092996016539686,0.03317288115145136,0.41268073407658645,0.8179879182664446,0.38571182166893747 36 | 1910,0.5762409249518508,1.3423079185225897,1.2838065096545208,0.5901118523872485,0.16754703518010788,2.3212288055774093,2.124402391792514,1.0092996016539686,1.4621585307603842,1.0070768531168661,1.8485238782870994,1.512839346090664 37 | 1911,0.34145125820771055,0.1209340616868349,0.3883025550890272,0.29610594728327444,-0.6925277454089556,-1.1115743576000805,-1.255328686059699,-1.1626109335507757,-0.9194842185878597,-1.1327491754281542,-0.7922245192657323,-0.2290850098338648 38 | 1912,-0.9264129422106678,-1.5637195339488161,-0.8232616187348795,-1.761935388444711,-1.1616594439121377,-0.5394404970705453,0.0037868135322811775,-0.715452882185076,-0.443155668718238,-0.7761115040040405,0.17390294325361774,-0.895114910628577 39 | 1913,-0.31595980867592466,-0.4686946967856645,0.17759574225010516,-0.43890881547670246,-0.6925277454089556,-0.2942402711293292,-0.1287516601089363,-0.715452882185076,-0.9790252873215793,-0.8949907278120558,-1.243084001774865,-0.7926487720447571 40 | 1914,-0.5037915420712261,0.1630504015776867,0.9677462903961078,-1.1739235782366793,0.08935841876289606,-1.6019748094826054,-1.785482580624795,-1.6736487065400847,-1.2767306309901099,-0.8355511159080143,-1.2430840017747917,-0.2290850098338648 41 | 1915,-2.100361275931369,-0.21599665744031443,-1.929472386139235,-1.467929483340737,-1.0834708274949258,0.768294041282824,1.4617100235862006,0.7537807151592415,0.6881246372222319,0.2343618983645296,-1.5007179917799371,0.8980425145878617 42 | 1916,0.5762409249518508,-0.3423456771130134,-0.5598781026862119,0.07560151845531478,0.7930559665177136,1.01349426722404,2.6545562863575345,1.6480968178905684,0.3904192935537015,0.5909995697886433,0.8823964157678225,1.461606276798754 43 | 1917,0.5292829916030121,0.8790281797228372,1.8105735417518558,2.060141377907202,2.278639678444383,2.239495396930306,2.9196332336401203,3.5006087449769763,2.891144180369317,1.4825937483491303,1.9773408732897086,2.1788692468853177 44 | 1918,1.4684416585795728,1.4686569381952888,-0.13846447700830788,1.6191325202511995,1.105810432186561,-0.3759736797763393,-1.3878671597009167,-0.39605427406673976,-0.8599431498541401,-0.4789134444839006,0.04508594825100845,-0.895114910628577 45 | 1919,-1.4429502090477868,-1.0162071153672163,-1.191998541202978,-0.14490291037272843,-0.6143391289917437,-0.9481075403059673,-0.8577132651359713,-0.6515731605613942,-0.6217788749193293,-1.0138699516201388,-1.178675504273487,-0.9975810492123968 46 | 1920,0.20057745816119443,-0.1738803175494148,-0.34917128984722995,0.14910299473132918,-0.14520743048856163,0.768294041282824,0.9978253658417512,0.5621415502882688,0.4499603622873534,-0.41947383257985915,-0.08373104675160085,0.8980425145878617 47 | 1921,1.0927781917889166,0.5842138004865874,0.9150695871863623,-0.5124102917527169,0.32392426801453156,2.3212288055774093,0.33513299763543786,-0.6515731605614669,0.4499603622874211,0.9476372412128247,0.7535794207652132,0.7443433067121903 48 | 1922,0.8110305916959909,0.7947954999410378,0.5990093679279492,-0.3654073392006881,-0.37977327974019714,0.686560632635721,0.26886376081479146,-0.07665566594840359,0.4499603622874211,0.5909995697886433,0.7535794207652132,1.102974791755443 49 | 1923,0.5762409249518508,0.37363210103213707,0.9150695871863623,0.8841177574912225,0.32392426801453156,0.19616018075319586,-1.1227902124184064,-1.8014081497874481,-1.5148949059249206,-0.597792668291916,-1.3074924992760963,0.129546475209388 50 | 1924,-0.5037915420712261,0.07881772179593526,0.28294914866953624,-1.2474250545126937,1.2621876650208956,0.931760858576937,0.799017655379812,1.0731793232776503,0.7476657059559516,0.7693184055007678,1.07562190827159,0.43694489096078915 51 | 1925,0.5762409249518508,1.2159588988499386,1.494513322493383,1.3986280914232398,0.011169802345773125,-0.3759736797764322,-1.3215979228803456,-1.0348514903034123,-0.6813199436529812,-1.2516283992362371,-0.9854500117696463,-0.7926487720447571 52 | 1926,-0.5037915420711727,-1.311021494603466,-1.2446752444127236,-0.5124102917527169,-0.14520743048865054,-0.6211739057175554,-0.06248242328828989,-0.715452882185076,0.092713949885171,0.41268073407658645,0.04508594825100845,0.539411029544609 53 | 1927,0.5292829916030121,0.07881772179593526,1.8105735417518558,0.7371148049392773,0.7148673501005907,0.93176085857703,0.6664791817385946,-0.45993399569042154,-0.0859092563159879,-0.41947383257985915,-0.8566330167670371,0.6931102374202803 54 | 1928,-0.9733708755595065,0.9211445196137368,1.3891599160739518,1.1781236625951965,-0.14520743048856163,-0.7029073143646584,0.0037868135322811775,1.009299601653896,0.7476657059559516,0.8881976293087831,0.17390294325354452,1.1029747917555013 55 | 1929,1.6093154586260356,1.5950059578679396,0.5463326647182037,0.5166103761112341,-1.0834708274949258,0.19616018075319586,0.20259452399422037,0.051103777298887314,-0.0859092563159879,0.7693184055007678,1.0112134107703585,0.48817796025269905 56 | 1930,1.280609925184218,0.6684464802683867,0.23027244545979075,-0.21840438664874284,0.32392426801453156,-0.45770708842344227,-0.3938286073915219,-0.14053538757208536,-0.7408610123867685,0.35324112217254494,0.1094944457523131,-0.2290850098338648 57 | 1931,0.7171147249983135,-1.3531378344943656,0.5990093679279492,0.8841177574912225,1.4185648978552305,1.9942951709890901,0.9978253658417512,0.051103777298959936,0.4499603622873534,-1.2516283992362371,-0.534590529260587,0.38571182166893747 58 | 1932,0.2005774581612478,-0.3423456771130134,-0.19114118021805337,-0.07140143409679758,0.40211288443174337,-0.3759736797763393,-0.46009784421209293,-0.6515731605613942,-0.9194842185878597,-0.41947383257985915,-0.5345905292606602,0.23201261379320778 59 | 1933,-1.0672867422571306,0.4157484409230367,-0.13846447700830788,0.44310889983521967,0.7148673501005018,-0.2942402711292363,0.40140223445608425,-0.012775944324794451,0.15225501861889063,0.35324112217254494,0.6247624257626039,0.7443433067121903 60 | 1934,0.6701567916495282,-0.005414957985864055,0.07224233583061418,0.6636133286632628,-0.6143391289917437,1.176961084518246,0.33513299763543786,-2.1846864795293937,-0.6813199436530488,0.41268073407658645,1.2044389032741993,-0.33155114841768457 61 | 1935,0.6701567916495282,-0.4265783568947648,1.2311298064447154,0.36960742355928883,-0.5361505125746208,-0.1307734538351232,0.0037868135322811775,0.2427429421699326,0.5690424997548603,0.7098787935967262,0.30271993825615384,-0.4852503562934143 62 | 1936,-0.1750860086294085,0.0367013819050356,0.23027244545979075,2.1336428541832166,0.5584901172660781,-0.04904004518802021,0.46767147127673064,-0.8432123254324395,0.21179608735254257,-0.0033965492515687697,-1.4363094942787056,-0.024152732666283448 63 | 1937,0.9519043917424537,-0.4686946967856645,0.6516860711376947,0.29610594728327444,0.08935841876298495,0.44136040669441196,-0.5263670810327393,0.37050238541729613,0.03317288115145136,-0.2411549968677347,-0.2769565392554416,0.5906440988365188 64 | 1938,0.7640726583471522,0.28939942125038565,-0.29649458663754436,0.44310889983521967,1.4185648978553194,1.912561762341987,1.9255946813306501,1.3286982097723048,0.6881246372222319,1.2448353007329644,0.1094944457523131,1.3079070689230827 65 | 1939,1.703231325323713,0.6684464802683867,1.1784531032350298,0.9576192337672369,0.011169802345773125,-0.04904004518802021,0.8652868922005337,-0.012775944324721828,-0.9790252873215793,-1.429947234948294,-0.8566330167671102,-0.9463479799204868 66 | 1940,0.012745724765946346,-0.3844620170038652,-0.981291728364056,-0.7329147205806765,-1.3180366767465612,-1.8471750354238217,-1.5204056333422846,-1.8014081497874481,-1.99122345579461,-1.7865849063724752,-0.727816021764501,-3.0469038208882684 67 | 1941,-0.9264129422106144,-1.3952541743852653,-0.981291728364056,-0.8799176731327053,-0.5361505125746208,-1.3567745835413894,-2.0505595279072297,-1.8652878714110572,-0.8599431498541401,-1.9649037420845321,-0.9854500117696463,-0.9463479799204286 68 | 1942,-1.255118475652432,-0.34234567711296554,-0.5072013994764664,-0.3654073392006881,0.6366787336833789,0.93176085857703,-0.06248242328836522,0.4343821070409053,0.8072067746896036,0.8287580174047416,-0.4701820317592823,1.3079070689230243 69 | 1943,0.9519043917425071,0.9211445196137368,0.4409792582987727,1.3251266151472254,0.40211288443174337,-0.7029073143647513,0.33513299763543786,0.8176604367829232,0.509501431021073,0.8881976293087831,0.30271993825615384,-0.9463479799204868 70 | 1944,-0.7855391421642051,0.33151576114123743,0.5990093679279492,-0.3654073392007716,0.011169802345773125,-0.2942402711293292,-0.8577132651358961,0.37050238541729613,0.21179608735261027,-0.8355511159080143,-0.727816021764501,0.33447875237702757 71 | 1945,0.5292829916030121,0.5420974605956878,1.3364832128642663,-0.5124102917527169,0.08935841876298495,0.93176085857703,0.40140223445608425,1.2009387665249411,0.8072067746896712,0.234361898364462,-0.4057735342579777,0.5906440988364605 72 | 1946,-0.22204394197819383,0.37363210103213707,-0.13846447700830788,-0.7329147205806765,-1.0052822110778028,-0.8663741316588643,-0.9902517387771135,-0.39605427406673976,-1.63397704339236,-1.1921887873321282,-0.21254804175421016,-0.6389495641690275 73 | 1947,-0.45683360872238743,-0.3844620170038652,1.1784531032350298,-0.29190586292475723,-1.2398480603293496,0.3596269980474019,0.9978253658417512,0.7537807151592415,1.104912118358134,-0.18171538496369322,0.8179879182664446,0.43694489096078915 74 | 1948,-0.26900187532703257,-0.2581129973312141,-0.34917128984728985,0.36960742355928883,0.48030150084895523,-0.3759736797764322,0.13632528717357398,-0.39605427406673976,-0.8004020811204882,0.5909995697886433,0.3671284357574585,-0.6389495641690857 75 | 1949,-0.6916232754664742,0.1630504015776867,0.5990093679279492,0.22260447100726002,-0.45796189615740895,-1.1115743576000805,-0.1287516601089363,-0.39605427406673976,0.15225501861889063,0.5315599578846694,-0.6634075242631963,0.6931102374202803 76 | 1950,0.5292829916030121,1.552889617977088,1.7578968385421103,1.6191325202511995,0.8712445829349255,2.811629257459934,2.190671628613085,1.264818488148623,0.6285835684885123,1.6609125840611871,1.1400304057728947,2.230102316177228 77 | 1951,1.6562733919749277,0.8369118398319375,-0.0857877737985624,0.002100042179300372,-0.536150512574532,0.604827223988618,-0.7914440283152496,-0.012775944324794451,-0.7408610123867008,-0.7761115040039729,-0.4057735342580509,-0.3827842177095945 78 | 1952,-0.879455008861829,-0.7213927361310145,0.07224233583061418,-0.6594132443046621,0.7148673501005907,0.850027449929927,0.40140223445608425,-0.33217455244305805,-0.38361459998451836,0.1749222864604881,-0.14813954425290551,-1.3562125342556495 79 | 1953,0.24753539151003315,-0.5529273765674159,-0.5072013994764664,0.07560151845531478,-3.0381862379247773,-0.1307734538351232,-0.06248242328828989,-1.6736487065400847,-1.3362716997238295,-0.0033965492515687697,-0.2769565392554416,-0.4852503562934143 80 | 1954,0.6231988583006894,-0.34234567711296554,-0.033111070588876794,0.7371148049391937,0.5584901172660781,-0.04904004518802021,0.46767147127673064,1.0731793232776503,0.3904192935537015,0.17492228646042052,0.30271993825615384,1.2054409303392628 81 | 1955,-0.5037915420711727,1.3423079185225897,0.33562585187928173,-0.14490291037272843,1.4185648978553194,1.749094945047874,1.9918639181512212,1.5203373746432773,1.3430763932930125,1.4825937483491303,1.3976643957780401,0.8468094452959518 82 | 1956,1.1397361251378086,1.0896098791772397,0.9677462903961078,1.1046221863192658,1.8876965963584125,1.340427901812452,1.329171549944908,1.1370590449012594,-0.026368187582268273,1.7797918078692025,0.10949444575223988,0.9492755838797716 83 | 1957,0.5762409249518508,-0.21599665744031443,-0.033111070588816904,0.22260447100726002,-1.0834708274949258,-0.1307734538351232,0.13632528717357398,-0.9070920470560486,-1.0981074247889508,-0.12227577305965173,-1.2430840017747917,-0.4340172870015044 84 | 1958,-1.6307819424430883,-0.6371600563492152,-0.08578777379862228,0.22260447100726002,-0.6925277454089556,0.11442677210609287,0.2688637608148668,0.8176604367829232,-0.38361459998451836,-0.18171538496369322,-0.5345905292606602,-0.7414157027528473 85 | 1959,-0.8324970755129903,-1.2689051547126142,0.8623928839766168,0.4431088998353032,0.40211288443174337,-0.5394404970704524,-0.46009784421209293,-0.45993399569042154,-0.026368187582268273,0.41268073407658645,1.0112134107702853,0.7443433067121903 86 | 1960,0.059703658114731675,-0.21599665744031443,0.5990093679279492,0.8106162812152081,0.63667873368329,-0.1307734538351232,0.533940708097377,0.6899009935356323,0.6285835684885123,-0.06283616115567782,0.6247624257626039,0.5906440988364605 87 | 1961,-0.22204394197824723,0.5420974605956878,-1.9821490893489806,0.9576192337672369,0.24573565159731972,-0.2125068624821333,0.26886376081479146,0.051103777298887314,0.03317288115145136,-0.47891344448383305,0.6247624257626039,1.3079070689230243 88 | 1962,1.703231325323713,0.4578647808138885,-0.0857877737985624,0.22260447100726002,1.3403762814380187,0.604827223988618,0.003786813532356517,0.498261828664587,0.4499603622873534,1.0070768531168661,0.4315369332586899,-0.024152732666283448 89 | 1963,0.9519043917425071,0.24728308135948598,0.7570394775571259,0.6636133286632628,0.40211288443174337,-0.8663741316588643,-0.06248242328836522,-0.2044151091956945,-0.5622378061856096,-1.2516283992362371,-0.9854500117696463,-1.253746395671888 90 | 1964,-0.36291774202471,-0.04753129787676371,0.8623928839766168,1.3251266151472254,0.40211288443174337,0.850027449929927,0.7327484185592409,1.4564576530195956,1.3430763932930125,1.2448353007329644,0.17390294325361774,-0.3827842177095945 91 | 1965,-0.36291774202471,0.12093406168678703,0.33562585187928173,-1.0269206256846506,0.08935841876298495,-1.1933077662471834,-2.249367238369169,-1.098731211927094,-1.455353837191201,-1.0733095635241128,-1.8227604792863872,0.0783134059174781 92 | 1966,-1.161202608954808,-0.3844620170039131,-1.297351947622469,-0.5124102917527169,-0.7707163618261674,0.19616018075328878,-0.06248242328828989,0.4343821070409053,-0.2645324625170791,-0.2411549968677347,-0.08373104675160085,-0.4852503562934143 93 | 1967,1.4684416585795728,1.1317262190681392,0.8097161807668714,-0.14490291037272843,-0.22339604690586237,0.768294041282824,0.20259452399422037,0.6260212719119506,0.4499603622874211,-0.0033965492515687697,-0.4701820317592823,-0.6389495641690275 94 | 1968,0.43536712490533463,0.8369118398319375,-0.24381788342779884,-0.14490291037272843,1.574942130689654,1.340427901812359,0.799017655379812,0.051103777298887314,-0.32407353125079874,-0.18171538496369322,-0.4057735342579777,0.129546475209388 95 | 1969,-1.3020764090012706,-0.6371600563492152,0.23027244545985065,-0.6594132443046621,-0.5361505125746208,0.03269336345898987,-0.6589055546740321,-0.39605427406673976,-1.0981074247889508,-1.1327491754281542,-0.08373104675160085,0.28324568308511766 96 | 1970,-0.9733708755595065,-0.9740907754763646,0.23027244545979075,-0.29190586292475723,0.32392426801453156,1.095227675871143,-0.5263670810327393,0.4343821070409053,1.2239942558255732,1.0070768531167986,1.8485238782870994,1.6665385539663937 97 | 1971,0.29449332485887186,1.3844242584134894,1.915926948171287,2.1336428541832166,1.027621815769349,0.3596269980474019,0.20259452399422037,1.5203373746432773,1.5216995994941038,1.720352195965161,0.6247624257626039,0.12954647520944623 98 | 1972,0.3884091915565493,0.7105628201592386,0.28294914866953624,-0.3654073392007716,-1.4744139095808961,-1.1115743576000805,-1.851751817445366,-0.8432123254323669,-1.5148949059249206,-1.0733095635241128,-0.4057735342579777,-1.3049794649637396 99 | 1973,-0.26900187532703257,-1.2267888148216668,0.12491903904035967,-0.07140143409671403,0.40211288443174337,1.340427901812452,0.6664791817385946,1.264818488148623,1.2835353245592929,0.9476372412128247,3.0078768333103634,1.6153054846744255 100 | 1974,2.078894792114316,1.426540598304389,2.021280354590778,1.1046221863192658,1.183999048603684,0.3596269980474019,1.2629023131242614,0.6899009935355597,1.1644531870918535,0.8287580174047416,-0.21254804175413694,-0.17785194054201314 101 | 1975,-0.45683360872238743,0.4578647808139364,1.1784531032350298,1.3986280914232398,0.7148673501005018,1.667361536400771,2.1906716286131602,2.095254869256341,2.1766513555648843,1.7203521959652288,1.268847400775504,1.8714708311339168 102 | 1976,1.186694058486594,1.1317262190681872,1.3364832128642063,0.22260447100726002,0.32392426801453156,0.11442677210618579,-1.255328686059699,-1.1626109335507757,-1.3362716997237618,0.2938015102685035,0.8823964157677493,-0.3827842177095945 103 | 1977,-0.36291774202471,0.6684464802683389,-0.875938321944625,-0.7329147205806765,-1.005282211077714,-1.6837082181297085,-1.4541363965216383,-1.1626109335507757,-0.9790252873215793,-1.2516283992361696,-1.5007179917799371,-1.1512802570880682 104 | 1978,-0.2690018753270859,-2.1954646323121194,-0.5072013994764664,-0.5859117680287312,1.7313193635240778,0.686560632635721,0.6664791817385946,0.17886322054625084,0.03317288115145136,-0.5977926682918484,-0.2769565392554416,-0.17785194054195488 105 | 1979,-0.36291774202471,0.5842138004865874,-0.24381788342779884,-0.3654073392007716,0.4803015008488663,0.686560632635721,-0.791444028315325,-0.45993399569034893,0.0927139498851033,-0.2411549968677347,-0.5345905292606602,-0.8438818413366671 106 | 1980,0.34145125820771055,0.07881772179593526,-0.770584915525134,-1.026920625684734,-0.22339604690577347,-0.3759736797763393,-0.12875166010901162,0.17886322054625084,-0.5622378061856096,-0.1817153849637608,-0.4057735342580509,-0.17785194054195488 107 | 1981,0.29449332485887186,-0.30022933722211376,-1.5607354636711366,-0.3654073392006881,0.8712445829349255,1.258694493165349,0.9978253658417512,0.6260212719119506,0.6881246372221642,-0.47891344448383305,0.17390294325361774,0.38571182166887924 108 | 1982,0.9519043917424537,0.0367013819050356,0.28294914866953624,-0.21840438664874284,-0.6925277454089556,-1.9289084440709245,-1.9180210542660125,-2.312445922776757,-2.1698466619956336,-1.9649037420845321,-3.1109304293122606,-2.227174712217885 109 | 1983,-2.992562009559091,-2.995675090239069,-2.6669462310755523,-1.3944280070647226,0.7148673501005018,-0.2125068624821333,-0.7251747914946032,0.051103777298887314,0.9262889121570428,0.41268073407658645,-0.14813954425290551,-0.07538580195819335 110 | 1984,0.15361952481235572,0.499981120704836,-0.5072013994764664,0.29610594728327444,0.08935841876298495,-0.7846407230118543,0.26886376081479146,0.30662266379361436,0.15225501861882293,-0.47891344448383305,0.30271993825615384,-0.22908500983392302 111 | 1985,-0.31595980867592466,0.5842138004865874,-0.13846447700830788,1.3986280914232398,0.40211288443174337,-0.8663741316588643,-0.1950208969295827,0.881540158406605,-0.026368187582268273,-0.5383530563878746,-0.21254804175413694,0.129546475209388 112 | 1986,0.8110305916959376,-0.9740907754763167,0.12491903904035967,0.22260447100726002,-0.5361505125746208,1.176961084518246,0.2688637608148668,-0.715452882185076,-0.5622378061856096,0.5909995697886433,-1.4363094942786325,-1.4586786728394694 113 | 1987,-0.5977074087688502,-1.1425561350399154,-1.5607354636711366,-2.055941293548685,-2.02173422450129,-1.9289084440710176,-1.851751817445366,-1.3542500984217485,-1.1576484935226705,-0.5383530563878746,-0.21254804175413694,-0.536483425585266 114 | 1988,-0.08117014193173108,-0.4686946967856645,0.28294914866959614,0.002100042179216811,1.105810432186561,-0.2942402711292363,1.1966330763036903,1.5203373746432773,1.9384870806300059,1.4231541364450213,1.9773408732897086,1.0005086531716816 115 | 1989,1.32756785853311,0.7947954999410378,0.7043627743474402,1.9866399016311878,1.574942130689654,0.850027449929927,0.9978253658417512,-0.5876934389377124,0.509501431021073,0.7098787935967262,-0.27695653925551483,-0.5877164948771758 116 | 1990,-0.08117014193173108,-1.5637195339488161,-0.770584915525074,0.0756015184552312,1.4185648978553194,0.19616018075328878,0.6002099449180235,-0.45993399569042154,-0.8004020811204204,0.1749222864604881,-0.5989990267618917,-0.33155114841768457 117 | 1991,0.5292829916030121,0.0367013819050356,-0.981291728364056,-1.0269206256846506,-1.7871683752497434,-0.45770708842344227,-0.1287516601089363,-0.715452882185076,-1.6935181121260119,-1.2516283992362371,-0.7922245192658056,-1.7660770885908121 118 | 1992,-2.4760247427219717,-0.8477417558037135,-2.298209308607394,-1.5414309596166678,0.16754703518010788,-1.1933077662471834,-0.6589055546740321,0.17886322054632348,0.03317288115145136,-1.6677056825644598,-0.7922245192658056,-0.6389495641690275 119 | 1993,-0.7855391421641517,-0.7213927361310145,-0.770584915525134,-1.7619353884446274,-0.6925277454089556,-1.5202414008355025,-1.0565209755978353,-1.4820095416690393,-0.8004020811204204,-1.311068011140211,-0.019322549250296205,0.0783134059174781 120 | 1994,-0.12812807528062317,0.0367013819050356,-0.981291728364056,-1.9089383409966563,-1.1616594439121377,-0.9481075403059673,-1.785482580624795,-1.6736487065400847,-1.7530591808597316,-1.3705076230442526,-0.7922245192657323,-1.2537463956718298 121 | 1995,-0.36291774202471,-0.30022933722211376,0.33562585187928173,-1.173923578236763,-0.7707163618261674,-0.04904004518802021,0.4676714712766553,0.11498349892256908,0.21179608735254257,-0.1222757730597193,0.04508594825100845,-0.5877164948771758 122 | 1996,0.8579885250447763,0.036701381904987715,0.6516860711376947,0.8106162812152081,0.24573565159731972,1.422161310459555,0.7327484185592409,0.498261828664587,0.6285835684885123,0.41268073407658645,-0.08373104675160085,0.795576376004042 123 | 1997,0.4823250582542267,1.1738425589590868,-0.7179082123153885,-1.3944280070647226,-2.021734224501201,-2.3375754873064394,-0.9902517387771135,-1.929167593034739,-1.5148949059249206,-1.6677056825643923,-1.4363094942787056,-0.9975810492123968 124 | 1998,-2.335150942675509,-1.7321848935123667,-2.7722996374949833,-2.1294427698246996,0.16754703518019678,1.095227675871143,1.5279792604068472,1.0092996016539686,1.0453710496244144,1.06651646502084,1.140030405772968,1.2566739996311727 125 | 1999,1.5623575252772501,0.7105628201592386,0.9150695871863623,1.6926339965272137,0.16754703518010788,0.11442677210618579,0.6002099449179481,0.2427429421699326,-0.08590925631592021,0.8881976293087831,1.1400304057728947,1.2054409303392628 126 | 2000,0.5292829916030121,1.1317262190681392,0.9677462903961078,1.619132520251283,0.48030150084895523,-0.45770708842344227,-0.3275593705708755,0.5621415502882688,0.9262889121570428,0.9476372412128247,2.106157868292245,0.6931102374202803 127 | 2001,0.8579885250447763,1.0474935392863878,0.7043627743473804,0.1491029947312456,-0.7707163618261674,0.27789358940029885,-0.3275593705708755,-0.8432123254323669,0.092713949885171,-0.18171538496369322,0.6247624257626039,-0.9975810492123385 128 | 2002,0.29449332485887186,0.6684464802683867,-0.45452469626672093,-0.21840438664874284,-1.3180366767464724,-0.5394404970705453,-0.7251747914946786,-1.4181298200453576,-0.8004020811204882,-0.7166718920999314,-0.6634075242631963,-1.1512802570880682 129 | 2003,-0.1750860086294085,-0.6792763962401148,-0.6125548058958975,-0.3654073392006881,-0.6143391289917437,-1.1933077662471834,0.3351329976355132,-0.14053538757208536,-0.2645324625170791,-0.18171538496369322,-0.4057735342580509,0.8468094452960101 130 | 2004,-1.1142446756059692,0.7947954999410378,0.07224233583061418,-1.2474250545126937,1.4185648978553194,-1.4385079921883994,-0.6589055546740321,-0.715452882185076,-0.32407353125079874,-0.36003422067581764,-0.9210415142683417,-0.895114910628577 131 | 2005,0.20057745816119443,-2.5745116913301684,0.07224233583061418,-0.8799176731327053,-1.3180366767465612,0.3596269980474019,0.13632528717349865,-0.6515731605613942,0.33087822481998186,1.06651646502084,-0.2769565392554416,-0.07538580195819335 132 | 2006,1.2806099251842713,-0.005414957985864055,1.3891599160739518,1.3986280914232398,-0.8489049782433792,-0.5394404970704524,-0.7251747914946032,-1.5458892632927936,-0.6217788749193293,-1.548826458756377,-0.21254804175421016,-0.43401728700144615 133 | 2007,-0.6916232754665276,-0.2581129973312141,-0.08578777379862228,-0.14490291037272843,-0.14520743048865054,0.6048272239887108,-0.4600978442121683,0.30662266379361436,0.092713949885171,0.5315599578846019,0.8179879182665178,1.3591401382149342 134 | 2008,1.421483725230734,1.8898203371041415,1.2311298064447154,0.5166103761113177,-0.22339604690586237,0.5230938153415149,0.2688637608148668,0.9454198800302868,1.2835353245592929,1.304274912637006,1.590889888281954,1.2054409303392628 135 | 2009,0.9519043917424537,1.30019157863169,0.07224233583061418,0.884117757491306,-0.5361505125746208,-0.1307734538351232,0.26886376081479146,-0.39605427406673976,0.33087822481998186,-1.429947234948294,-0.727816021764501,-0.7926487720447571 136 | 2010,-0.9733708755594531,-1.311021494603466,-1.0339684315738016,1.4721295676991706,1.1058104321864721,0.27789358940029885,2.124402391792514,1.9036157043852227,2.414815630499695,1.7797918078692703,1.5264813907806494,2.6399668705124486 137 | 2011,1.9849789254166919,1.974053016885941,2.126633761010269,2.4276487592871905,0.32392426801453156,0.11442677210618579,1.1303638394829687,0.2427429421699326,1.104912118358134,0.7098787935967262,1.2044389032741993,2.230102316177228 138 | 2012,0.9519043917424537,0.20516674146853844,0.33562585187928173,-0.5124102917527169,-0.14520743048856163,-0.9481075403058744,-0.1287516601089363,-0.45993399569042154,0.21179608735254257,0.2343618983645296,0.3671284357574585,-0.6901826334609956 139 | 2013,-0.08117014193173108,-0.3423456771130134,1.0730996968155389,0.1491029947312456,0.9494331993521373,1.503894719106565,0.8652868922005337,-0.012775944324721828,0.33087822481998186,-0.1817153849637608,0.8179879182664446,-0.024152732666341693 140 | 2014,1.2336519918354325,-0.13176397765856301,-1.2446752444127236,0.8841177574912225,0.5584901172660781,-0.04904004518802021,-0.2612901337502291,-1.098731211927094,-0.8004020811204204,-0.7761115040039729,-1.049858509270951,-0.6389495641690857 141 | 2015,-0.7385812088153663,0.0367013819050356,-1.0339684315738016,-0.21840438664874284,-1.2398480603293496,-1.1115743576000805,-1.454136396521563,-1.929167593034739,-1.8126002495934512,-1.9649037420845321,-0.5989990267618917,-0.9975810492123968 142 | 2016,-1.9125295425360138,-1.7743012334032664,-0.40184799305703534,-1.8354368647206418,0.40211288443174337,0.686560632635721,0.46767147127673064,0.5621415502882688,1.2835353245592929,-0.41947383257979154,-0.1481395442528323,0.1807795445012979 143 | 2017,0.1536195248124091,-0.21599665744031443,0.5463326647182037,-0.438908815476786,0.16754703518010788,-0.9481075403059673,0.8652868922004584,0.42685324305032124,,,, 144 | -------------------------------------------------------------------------------- /notebooks/data/Wellington_rain.xlsx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nicolasfauchereau/Auckland_Python_Workshop/33365e02449818dd7feadc3b20aeb487ed0624cd/notebooks/data/Wellington_rain.xlsx -------------------------------------------------------------------------------- /notebooks/data/rainfall_calibration.xlsx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nicolasfauchereau/Auckland_Python_Workshop/33365e02449818dd7feadc3b20aeb487ed0624cd/notebooks/data/rainfall_calibration.xlsx -------------------------------------------------------------------------------- /notebooks/data/soi_nino.csv: -------------------------------------------------------------------------------- 1 | ,SOI,nino 2 | 1950-01-01,0.5292829916030121,-1.84 3 | 1950-02-01,1.5528896179770881,-1.63 4 | 1950-03-01,1.7578968385421103,-1.3 5 | 1950-04-01,1.6191325202511997,-1.43 6 | 1950-05-01,0.8712445829349255,-1.72 7 | 1950-06-01,2.811629257459934,-1.17 8 | 1950-07-01,2.1906716286130847,-0.86 9 | 1950-08-01,1.264818488148623,-0.95 10 | 1950-09-01,0.6285835684885123,-1.04 11 | 1950-10-01,1.6609125840611871,-0.83 12 | 1950-11-01,1.1400304057728947,-1.1 13 | 1950-12-01,2.230102316177228,-1.19 14 | 1951-01-01,1.6562733919749275,-1.21 15 | 1951-02-01,0.8369118398319375,-1.05 16 | 1951-03-01,-0.0857877737985624,-0.61 17 | 1951-04-01,0.0021000421793003717,-0.56 18 | 1951-05-01,-0.5361505125745321,-0.23 19 | 1951-06-01,0.604827223988618,-0.23 20 | 1951-07-01,-0.7914440283152496,0.44 21 | 1951-08-01,-0.012775944324794451,0.43 22 | 1951-09-01,-0.7408610123867008,0.69 23 | 1951-10-01,-0.7761115040039729,0.64 24 | 1951-11-01,-0.4057735342580509,0.74 25 | 1951-12-01,-0.3827842177095945,0.43 26 | 1952-01-01,-0.879455008861829,0.18 27 | 1952-02-01,-0.7213927361310145,-0.04 28 | 1952-03-01,0.07224233583061418,-0.01 29 | 1952-04-01,-0.6594132443046621,0.08 30 | 1952-05-01,0.7148673501005907,0.07 31 | 1952-06-01,0.850027449929927,-0.36 32 | 1952-07-01,0.40140223445608425,-0.56 33 | 1952-08-01,-0.33217455244305805,-0.47 34 | 1952-09-01,-0.3836145999845184,-0.29 35 | 1952-10-01,0.1749222864604881,-0.25 36 | 1952-11-01,-0.14813954425290551,-0.37 37 | 1952-12-01,-1.3562125342556497,-0.15 38 | 1953-01-01,0.24753539151003315,0.18 39 | 1953-02-01,-0.5529273765674159,0.36 40 | 1953-03-01,-0.5072013994764664,0.35 41 | 1953-04-01,0.07560151845531478,0.4 42 | 1953-05-01,-3.0381862379247773,0.38 43 | 1953-06-01,-0.1307734538351232,0.33 44 | 1953-07-01,-0.06248242328828989,0.24 45 | 1953-08-01,-1.673648706540085,0.23 46 | 1953-09-01,-1.3362716997238295,0.31 47 | 1953-10-01,-0.0033965492515687697,0.23 48 | 1953-11-01,-0.2769565392554416,0.23 49 | 1953-12-01,-0.4852503562934143,0.3 50 | 1954-01-01,0.6231988583006894,0.36 51 | 1954-02-01,-0.34234567711296554,0.39 52 | 1954-03-01,-0.033111070588876794,-0.12 53 | 1954-04-01,0.7371148049391937,-0.93 54 | 1954-05-01,0.5584901172660781,-0.85 55 | 1954-06-01,-0.049040045188020215,-0.76 56 | 1954-07-01,0.4676714712767306,-0.91 57 | 1954-08-01,1.0731793232776503,-1.2 58 | 1954-09-01,0.3904192935537015,-1.45 59 | 1954-10-01,0.17492228646042052,-1.28 60 | 1954-11-01,0.30271993825615384,-1.07 61 | 1954-12-01,1.2054409303392628,-1.32 62 | 1955-01-01,-0.5037915420711727,-1.13 63 | 1955-02-01,1.3423079185225897,-0.93 64 | 1955-03-01,0.3356258518792817,-0.95 65 | 1955-04-01,-0.14490291037272846,-1.14 66 | 1955-05-01,1.4185648978553194,-1.25 67 | 1955-06-01,1.749094945047874,-1.13 68 | 1955-07-01,1.9918639181512208,-0.98 69 | 1955-08-01,1.5203373746432771,-1.33 70 | 1955-09-01,1.3430763932930123,-1.19 71 | 1955-10-01,1.4825937483491305,-2.13 72 | 1955-11-01,1.39766439577804,-2.45 73 | 1955-12-01,0.8468094452959518,-2.1 74 | 1956-01-01,1.1397361251378086,-1.39 75 | 1956-02-01,1.0896098791772395,-0.83 76 | 1956-03-01,0.9677462903961078,-0.71 77 | 1956-04-01,1.1046221863192658,-0.83 78 | 1956-05-01,1.8876965963584125,-0.68 79 | 1956-06-01,1.340427901812452,-0.7 80 | 1956-07-01,1.329171549944908,-0.91 81 | 1956-08-01,1.1370590449012594,-1.02 82 | 1956-09-01,-0.026368187582268277,-0.92 83 | 1956-10-01,1.7797918078692023,-0.95 84 | 1956-11-01,0.10949444575223988,-1.1 85 | 1956-12-01,0.9492755838797716,-0.98 86 | 1957-01-01,0.5762409249518508,-0.58 87 | 1957-02-01,-0.21599665744031446,-0.25 88 | 1957-03-01,-0.033111070588816904,0.32 89 | 1957-04-01,0.22260447100726,0.55 90 | 1957-05-01,-1.0834708274949258,0.68 91 | 1957-06-01,-0.1307734538351232,0.59 92 | 1957-07-01,0.13632528717357398,0.83 93 | 1957-08-01,-0.9070920470560486,0.82 94 | 1957-09-01,-1.0981074247889508,0.71 95 | 1957-10-01,-0.12227577305965172,0.7 96 | 1957-11-01,-1.2430840017747915,1.06 97 | 1957-12-01,-0.4340172870015044,1.4 98 | 1958-01-01,-1.6307819424430885,1.7 99 | 1958-02-01,-0.6371600563492152,1.59 100 | 1958-03-01,-0.08578777379862228,1.01 101 | 1958-04-01,0.22260447100726,0.55 102 | 1958-05-01,-0.6925277454089556,0.44 103 | 1958-06-01,0.11442677210609288,0.36 104 | 1958-07-01,0.2688637608148668,0.2 105 | 1958-08-01,0.8176604367829232,-0.03 106 | 1958-09-01,-0.3836145999845184,-0.32 107 | 1958-10-01,-0.1817153849636932,-0.18 108 | 1958-11-01,-0.5345905292606602,0.16 109 | 1958-12-01,-0.7414157027528473,0.09 110 | 1959-01-01,-0.8324970755129903,0.47 111 | 1959-02-01,-1.2689051547126142,0.42 112 | 1959-03-01,0.8623928839766168,0.22 113 | 1959-04-01,0.4431088998353032,0.14 114 | 1959-05-01,0.40211288443174337,-0.11 115 | 1959-06-01,-0.5394404970704524,-0.35 116 | 1959-07-01,-0.4600978442120929,-0.72 117 | 1959-08-01,-0.4599339956904216,-0.61 118 | 1959-09-01,-0.026368187582268277,-0.65 119 | 1959-10-01,0.4126807340765865,-0.36 120 | 1959-11-01,1.0112134107702853,-0.41 121 | 1959-12-01,0.7443433067121903,-0.25 122 | 1960-01-01,0.059703658114731675,-0.42 123 | 1960-02-01,-0.21599665744031446,-0.56 124 | 1960-03-01,0.5990093679279492,-0.36 125 | 1960-04-01,0.8106162812152081,-0.29 126 | 1960-05-01,0.6366787336832901,-0.29 127 | 1960-06-01,-0.1307734538351232,-0.45 128 | 1960-07-01,0.533940708097377,-0.21 129 | 1960-08-01,0.6899009935356323,-0.08 130 | 1960-09-01,0.6285835684885123,-0.23 131 | 1960-10-01,-0.06283616115567782,-0.43 132 | 1960-11-01,0.6247624257626039,-0.39 133 | 1960-12-01,0.5906440988364605,-0.28 134 | 1961-01-01,-0.22204394197824726,-0.29 135 | 1961-02-01,0.5420974605956878,-0.22 136 | 1961-03-01,-1.9821490893489808,-0.33 137 | 1961-04-01,0.9576192337672368,-0.29 138 | 1961-05-01,0.2457356515973197,-0.06 139 | 1961-06-01,-0.2125068624821333,0.17 140 | 1961-07-01,0.26886376081479146,-0.12 141 | 1961-08-01,0.051103777298887314,-0.5 142 | 1961-09-01,0.03317288115145136,-0.85 143 | 1961-10-01,-0.478913444483833,-0.85 144 | 1961-11-01,0.6247624257626039,-0.53 145 | 1961-12-01,1.3079070689230243,-0.49 146 | 1962-01-01,1.703231325323713,-0.55 147 | 1962-02-01,0.4578647808138885,-0.55 148 | 1962-03-01,-0.0857877737985624,-0.53 149 | 1962-04-01,0.22260447100726,-0.57 150 | 1962-05-01,1.340376281438019,-0.68 151 | 1962-06-01,0.604827223988618,-0.51 152 | 1962-07-01,0.0037868135323565174,-0.35 153 | 1962-08-01,0.49826182866458707,-0.41 154 | 1962-09-01,0.4499603622873534,-0.75 155 | 1962-10-01,1.007076853116866,-0.73 156 | 1962-11-01,0.4315369332586899,-0.85 157 | 1962-12-01,-0.024152732666283448,-0.94 158 | 1963-01-01,0.9519043917425072,-0.81 159 | 1963-02-01,0.247283081359486,-0.44 160 | 1963-03-01,0.7570394775571259,-0.01 161 | 1963-04-01,0.6636133286632628,0.07 162 | 1963-05-01,0.40211288443174337,-0.2 163 | 1963-06-01,-0.8663741316588643,-0.0 164 | 1963-07-01,-0.06248242328836522,0.62 165 | 1963-08-01,-0.2044151091956945,0.7 166 | 1963-09-01,-0.5622378061856096,0.64 167 | 1963-10-01,-1.2516283992362371,0.77 168 | 1963-11-01,-0.9854500117696464,0.87 169 | 1963-12-01,-1.253746395671888,1.02 170 | 1964-01-01,-0.36291774202471005,0.76 171 | 1964-02-01,-0.04753129787676371,0.44 172 | 1964-03-01,0.8623928839766168,-0.22 173 | 1964-04-01,1.3251266151472254,-0.73 174 | 1964-05-01,0.40211288443174337,-0.97 175 | 1964-06-01,0.850027449929927,-1.03 176 | 1964-07-01,0.7327484185592409,-0.86 177 | 1964-08-01,1.4564576530195956,-1.19 178 | 1964-09-01,1.3430763932930123,-1.34 179 | 1964-10-01,1.2448353007329644,-1.25 180 | 1964-11-01,0.17390294325361774,-1.34 181 | 1964-12-01,-0.3827842177095945,-1.27 182 | 1965-01-01,-0.36291774202471005,-0.89 183 | 1965-02-01,0.12093406168678705,-0.46 184 | 1965-03-01,0.3356258518792817,-0.21 185 | 1965-04-01,-1.0269206256846506,-0.18 186 | 1965-05-01,0.08935841876298495,0.15 187 | 1965-06-01,-1.1933077662471834,0.5 188 | 1965-07-01,-2.2493672383691687,0.74 189 | 1965-08-01,-1.0987312119270942,1.14 190 | 1965-09-01,-1.455353837191201,1.2 191 | 1965-10-01,-1.0733095635241128,1.47 192 | 1965-11-01,-1.8227604792863872,1.56 193 | 1965-12-01,0.0783134059174781,1.38 194 | 1966-01-01,-1.1612026089548078,1.08 195 | 1966-02-01,-0.3844620170039131,0.83 196 | 1966-03-01,-1.297351947622469,0.89 197 | 1966-04-01,-0.5124102917527169,0.45 198 | 1966-05-01,-0.7707163618261674,-0.24 199 | 1966-06-01,0.1961601807532888,0.08 200 | 1966-07-01,-0.06248242328828989,0.14 201 | 1966-08-01,0.4343821070409053,-0.28 202 | 1966-09-01,-0.2645324625170791,-0.34 203 | 1966-10-01,-0.2411549968677347,-0.39 204 | 1966-11-01,-0.08373104675160085,-0.32 205 | 1966-12-01,-0.4852503562934143,-0.48 206 | 1967-01-01,1.4684416585795728,-0.49 207 | 1967-02-01,1.1317262190681392,-0.48 208 | 1967-03-01,0.8097161807668714,-0.66 209 | 1967-04-01,-0.14490291037272846,-0.89 210 | 1967-05-01,-0.2233960469058624,-0.42 211 | 1967-06-01,0.7682940412828241,-0.07 212 | 1967-07-01,0.2025945239942204,-0.11 213 | 1967-08-01,0.6260212719119506,-0.31 214 | 1967-09-01,0.4499603622874211,-0.72 215 | 1967-10-01,-0.0033965492515687697,-0.6 216 | 1967-11-01,-0.4701820317592823,-0.47 217 | 1967-12-01,-0.6389495641690275,-0.59 218 | 1968-01-01,0.4353671249053345,-0.89 219 | 1968-02-01,0.8369118398319375,-1.04 220 | 1968-03-01,-0.24381788342779884,-0.95 221 | 1968-04-01,-0.14490291037272846,-0.71 222 | 1968-05-01,1.574942130689654,-0.76 223 | 1968-06-01,1.340427901812359,-0.0 224 | 1968-07-01,0.799017655379812,0.33 225 | 1968-08-01,0.051103777298887314,0.19 226 | 1968-09-01,-0.32407353125079874,0.07 227 | 1968-10-01,-0.1817153849636932,0.2 228 | 1968-11-01,-0.4057735342579777,0.65 229 | 1968-12-01,0.129546475209388,0.73 230 | 1969-01-01,-1.3020764090012706,0.88 231 | 1969-02-01,-0.6371600563492152,1.12 232 | 1969-03-01,0.23027244545985065,0.69 233 | 1969-04-01,-0.6594132443046621,0.6 234 | 1969-05-01,-0.5361505125746208,0.59 235 | 1969-06-01,0.03269336345898987,0.3 236 | 1969-07-01,-0.6589055546740321,0.02 237 | 1969-08-01,-0.39605427406673976,0.23 238 | 1969-09-01,-1.0981074247889508,0.51 239 | 1969-10-01,-1.1327491754281542,0.72 240 | 1969-11-01,-0.08373104675160085,0.61 241 | 1969-12-01,0.2832456830851177,0.61 242 | 1970-01-01,-0.9733708755595064,0.38 243 | 1970-02-01,-0.9740907754763646,0.2 244 | 1970-03-01,0.23027244545979075,0.02 245 | 1970-04-01,-0.29190586292475723,0.16 246 | 1970-05-01,0.3239242680145316,-0.14 247 | 1970-06-01,1.095227675871143,-0.55 248 | 1970-07-01,-0.5263670810327393,-0.8 249 | 1970-08-01,0.4343821070409053,-1.07 250 | 1970-09-01,1.2239942558255732,-0.96 251 | 1970-10-01,1.0070768531167986,-1.0 252 | 1970-11-01,1.8485238782870996,-0.9 253 | 1970-12-01,1.6665385539663935,-1.22 254 | 1971-01-01,0.29449332485887186,-1.58 255 | 1971-02-01,1.3844242584134894,-1.43 256 | 1971-03-01,1.9159269481712868,-1.23 257 | 1971-04-01,2.1336428541832166,-1.02 258 | 1971-05-01,1.027621815769349,-0.85 259 | 1971-06-01,0.3596269980474019,-0.99 260 | 1971-07-01,0.2025945239942204,-0.99 261 | 1971-08-01,1.5203373746432771,-0.94 262 | 1971-09-01,1.5216995994941038,-0.88 263 | 1971-10-01,1.720352195965161,-1.13 264 | 1971-11-01,0.6247624257626039,-1.04 265 | 1971-12-01,0.12954647520944626,-1.14 266 | 1972-01-01,0.3884091915565493,-0.83 267 | 1972-02-01,0.7105628201592386,-0.41 268 | 1972-03-01,0.28294914866953624,-0.12 269 | 1972-04-01,-0.3654073392007716,0.13 270 | 1972-05-01,-1.474413909580896,0.35 271 | 1972-06-01,-1.1115743576000805,0.54 272 | 1972-07-01,-1.851751817445366,0.9 273 | 1972-08-01,-0.8432123254323669,1.17 274 | 1972-09-01,-1.5148949059249206,1.27 275 | 1972-10-01,-1.0733095635241128,1.68 276 | 1972-11-01,-0.4057735342579777,1.96 277 | 1972-12-01,-1.3049794649637396,2.12 278 | 1973-01-01,-0.26900187532703257,1.77 279 | 1973-02-01,-1.2267888148216668,1.17 280 | 1973-03-01,0.12491903904035967,0.39 281 | 1973-04-01,-0.07140143409671403,-0.41 282 | 1973-05-01,0.40211288443174337,-0.79 283 | 1973-06-01,1.340427901812452,-0.95 284 | 1973-07-01,0.6664791817385946,-1.33 285 | 1973-08-01,1.264818488148623,-1.47 286 | 1973-09-01,1.2835353245592929,-1.51 287 | 1973-10-01,0.9476372412128248,-1.76 288 | 1973-11-01,3.007876833310364,-2.24 289 | 1973-12-01,1.6153054846744257,-2.25 290 | 1974-01-01,2.078894792114316,-2.15 291 | 1974-02-01,1.426540598304389,-1.71 292 | 1974-03-01,2.021280354590778,-1.42 293 | 1974-04-01,1.1046221863192658,-1.17 294 | 1974-05-01,1.183999048603684,-1.07 295 | 1974-06-01,0.3596269980474019,-0.98 296 | 1974-07-01,1.2629023131242614,-0.8 297 | 1974-08-01,0.6899009935355597,-0.56 298 | 1974-09-01,1.1644531870918535,-0.59 299 | 1974-10-01,0.8287580174047416,-0.87 300 | 1974-11-01,-0.2125480417541369,-1.11 301 | 1974-12-01,-0.17785194054201314,-0.99 302 | 1975-01-01,-0.4568336087223874,-0.49 303 | 1975-02-01,0.4578647808139364,-0.61 304 | 1975-03-01,1.1784531032350298,-0.89 305 | 1975-04-01,1.3986280914232398,-0.81 306 | 1975-05-01,0.7148673501005018,-0.94 307 | 1975-06-01,1.6673615364007712,-1.4 308 | 1975-07-01,2.19067162861316,-1.3 309 | 1975-08-01,2.095254869256341,-1.46 310 | 1975-09-01,2.1766513555648843,-1.62 311 | 1975-10-01,1.7203521959652288,-1.92 312 | 1975-11-01,1.2688474007755042,-1.64 313 | 1975-12-01,1.8714708311339168,-1.93 314 | 1976-01-01,1.186694058486594,-1.95 315 | 1976-02-01,1.1317262190681872,-1.25 316 | 1976-03-01,1.3364832128642063,-0.72 317 | 1976-04-01,0.22260447100726,-0.8 318 | 1976-05-01,0.3239242680145316,-0.64 319 | 1976-06-01,0.1144267721061858,-0.35 320 | 1976-07-01,-1.2553286860596988,-0.02 321 | 1976-08-01,-1.1626109335507755,0.16 322 | 1976-09-01,-1.3362716997237618,0.3 323 | 1976-10-01,0.2938015102685035,0.7 324 | 1976-11-01,0.8823964157677493,0.68 325 | 1976-12-01,-0.3827842177095945,0.48 326 | 1977-01-01,-0.36291774202471005,0.64 327 | 1977-02-01,0.6684464802683389,0.32 328 | 1977-03-01,-0.875938321944625,0.24 329 | 1977-04-01,-0.7329147205806765,-0.18 330 | 1977-05-01,-1.0052822110777142,0.06 331 | 1977-06-01,-1.6837082181297085,0.27 332 | 1977-07-01,-1.4541363965216385,0.28 333 | 1977-08-01,-1.1626109335507755,-0.0 334 | 1977-09-01,-0.9790252873215792,0.38 335 | 1977-10-01,-1.2516283992361696,0.59 336 | 1977-11-01,-1.5007179917799371,0.54 337 | 1977-12-01,-1.1512802570880682,0.68 338 | 1978-01-01,-0.2690018753270859,0.65 339 | 1978-02-01,-2.1954646323121194,0.34 340 | 1978-03-01,-0.5072013994764664,-0.09 341 | 1978-04-01,-0.5859117680287312,-0.54 342 | 1978-05-01,1.7313193635240778,-0.55 343 | 1978-06-01,0.6865606326357211,-0.53 344 | 1978-07-01,0.6664791817385946,-0.47 345 | 1978-08-01,0.17886322054625084,-0.7 346 | 1978-09-01,0.03317288115145136,-0.62 347 | 1978-10-01,-0.5977926682918484,-0.38 348 | 1978-11-01,-0.2769565392554416,-0.3 349 | 1978-12-01,-0.17785194054195488,-0.2 350 | 1979-01-01,-0.36291774202471005,-0.22 351 | 1979-02-01,0.5842138004865874,-0.22 352 | 1979-03-01,-0.24381788342779884,0.15 353 | 1979-04-01,-0.3654073392007716,0.07 354 | 1979-05-01,0.4803015008488663,-0.17 355 | 1979-06-01,0.6865606326357211,-0.14 356 | 1979-07-01,-0.7914440283153249,-0.33 357 | 1979-08-01,-0.4599339956903489,-0.16 358 | 1979-09-01,0.0927139498851033,0.33 359 | 1979-10-01,-0.2411549968677347,0.22 360 | 1979-11-01,-0.5345905292606602,0.35 361 | 1979-12-01,-0.8438818413366671,0.4 362 | 1980-01-01,0.3414512582077105,0.43 363 | 1980-02-01,0.07881772179593527,0.2 364 | 1980-03-01,-0.7705849155251341,0.05 365 | 1980-04-01,-1.026920625684734,-0.01 366 | 1980-05-01,-0.22339604690577347,0.07 367 | 1980-06-01,-0.3759736797763393,0.3 368 | 1980-07-01,-0.12875166010901162,0.07 369 | 1980-08-01,0.17886322054625084,-0.31 370 | 1980-09-01,-0.5622378061856096,-0.26 371 | 1980-10-01,-0.1817153849637608,-0.27 372 | 1980-11-01,-0.4057735342580509,-0.11 373 | 1980-12-01,-0.17785194054195488,-0.1 374 | 1981-01-01,0.29449332485887186,-0.5 375 | 1981-02-01,-0.30022933722211376,-0.7 376 | 1981-03-01,-1.5607354636711366,-0.58 377 | 1981-04-01,-0.3654073392006881,-0.48 378 | 1981-05-01,0.8712445829349255,-0.5 379 | 1981-06-01,1.258694493165349,-0.34 380 | 1981-07-01,0.9978253658417512,-0.62 381 | 1981-08-01,0.6260212719119506,-0.68 382 | 1981-09-01,0.6881246372221642,-0.29 383 | 1981-10-01,-0.478913444483833,-0.26 384 | 1981-11-01,0.17390294325361774,-0.33 385 | 1981-12-01,0.3857118216688792,-0.22 386 | 1982-01-01,0.9519043917424536,0.02 387 | 1982-02-01,0.0367013819050356,-0.12 388 | 1982-03-01,0.28294914866953624,0.03 389 | 1982-04-01,-0.21840438664874284,0.11 390 | 1982-05-01,-0.6925277454089556,0.51 391 | 1982-06-01,-1.9289084440709243,0.63 392 | 1982-07-01,-1.9180210542660123,0.37 393 | 1982-08-01,-2.312445922776757,0.71 394 | 1982-09-01,-2.1698466619956336,1.38 395 | 1982-10-01,-1.964903742084532,1.89 396 | 1982-11-01,-3.1109304293122606,1.98 397 | 1982-12-01,-2.227174712217885,2.25 398 | 1983-01-01,-2.9925620095590912,2.26 399 | 1983-02-01,-2.995675090239069,1.92 400 | 1983-03-01,-2.6669462310755523,1.43 401 | 1983-04-01,-1.3944280070647226,0.94 402 | 1983-05-01,0.7148673501005018,0.91 403 | 1983-06-01,-0.2125068624821333,0.58 404 | 1983-07-01,-0.7251747914946032,-0.1 405 | 1983-08-01,0.051103777298887314,-0.42 406 | 1983-09-01,0.9262889121570428,-0.64 407 | 1983-10-01,0.4126807340765865,-1.03 408 | 1983-11-01,-0.14813954425290551,-1.19 409 | 1983-12-01,-0.07538580195819335,-0.87 410 | 1984-01-01,0.15361952481235572,-0.53 411 | 1984-02-01,0.499981120704836,-0.11 412 | 1984-03-01,-0.5072013994764664,-0.28 413 | 1984-04-01,0.29610594728327444,-0.5 414 | 1984-05-01,0.08935841876298495,-0.6 415 | 1984-06-01,-0.7846407230118543,-0.74 416 | 1984-07-01,0.26886376081479146,-0.44 417 | 1984-08-01,0.3066226637936144,-0.26 418 | 1984-09-01,0.15225501861882293,-0.39 419 | 1984-10-01,-0.478913444483833,-0.64 420 | 1984-11-01,0.30271993825615384,-1.13 421 | 1984-12-01,-0.22908500983392305,-1.31 422 | 1985-01-01,-0.31595980867592466,-1.1 423 | 1985-02-01,0.5842138004865874,-0.72 424 | 1985-03-01,-0.13846447700830788,-0.8 425 | 1985-04-01,1.3986280914232398,-0.87 426 | 1985-05-01,0.40211288443174337,-0.82 427 | 1985-06-01,-0.8663741316588643,-0.77 428 | 1985-07-01,-0.1950208969295827,-0.64 429 | 1985-08-01,0.8815401584066049,-0.65 430 | 1985-09-01,-0.026368187582268277,-0.74 431 | 1985-10-01,-0.5383530563878746,-0.56 432 | 1985-11-01,-0.2125480417541369,-0.35 433 | 1985-12-01,0.129546475209388,-0.5 434 | 1986-01-01,0.8110305916959376,-0.63 435 | 1986-02-01,-0.9740907754763168,-0.57 436 | 1986-03-01,0.12491903904035967,-0.34 437 | 1986-04-01,0.22260447100726,-0.15 438 | 1986-05-01,-0.5361505125746208,-0.34 439 | 1986-06-01,1.1769610845182459,-0.13 440 | 1986-07-01,0.2688637608148668,0.1 441 | 1986-08-01,-0.7154528821850759,0.28 442 | 1986-09-01,-0.5622378061856096,0.53 443 | 1986-10-01,0.5909995697886433,0.81 444 | 1986-11-01,-1.4363094942786323,0.94 445 | 1986-12-01,-1.4586786728394694,1.06 446 | 1987-01-01,-0.5977074087688502,1.15 447 | 1987-02-01,-1.1425561350399154,1.19 448 | 1987-03-01,-1.5607354636711366,1.16 449 | 1987-04-01,-2.055941293548685,0.89 450 | 1987-05-01,-2.02173422450129,0.79 451 | 1987-06-01,-1.9289084440710176,0.98 452 | 1987-07-01,-1.851751817445366,1.24 453 | 1987-08-01,-1.3542500984217485,1.55 454 | 1987-09-01,-1.1576484935226703,1.59 455 | 1987-10-01,-0.5383530563878746,1.29 456 | 1987-11-01,-0.2125480417541369,1.2 457 | 1987-12-01,-0.536483425585266,0.91 458 | 1988-01-01,-0.08117014193173107,0.82 459 | 1988-02-01,-0.4686946967856645,0.19 460 | 1988-03-01,0.2829491486695961,0.12 461 | 1988-04-01,0.002100042179216811,-0.23 462 | 1988-05-01,1.1058104321865612,-0.94 463 | 1988-06-01,-0.2942402711292363,-1.48 464 | 1988-07-01,1.1966330763036903,-1.62 465 | 1988-08-01,1.5203373746432771,-1.22 466 | 1988-09-01,1.9384870806300056,-1.11 467 | 1988-10-01,1.423154136445021,-1.97 468 | 1988-11-01,1.9773408732897089,-2.01 469 | 1988-12-01,1.0005086531716816,-2.01 470 | 1989-01-01,1.32756785853311,-1.98 471 | 1989-02-01,0.7947954999410378,-1.49 472 | 1989-03-01,0.7043627743474402,-1.21 473 | 1989-04-01,1.986639901631188,-0.96 474 | 1989-05-01,1.574942130689654,-0.74 475 | 1989-06-01,0.850027449929927,-0.46 476 | 1989-07-01,0.9978253658417512,-0.48 477 | 1989-08-01,-0.5876934389377124,-0.5 478 | 1989-09-01,0.509501431021073,-0.36 479 | 1989-10-01,0.7098787935967262,-0.48 480 | 1989-11-01,-0.27695653925551483,-0.44 481 | 1989-12-01,-0.5877164948771758,-0.11 482 | 1990-01-01,-0.08117014193173107,-0.05 483 | 1990-02-01,-1.563719533948816,0.2 484 | 1990-03-01,-0.7705849155250741,0.06 485 | 1990-04-01,0.0756015184552312,0.14 486 | 1990-05-01,1.4185648978553194,0.17 487 | 1990-06-01,0.1961601807532888,-0.06 488 | 1990-07-01,0.6002099449180235,0.14 489 | 1990-08-01,-0.4599339956904216,0.23 490 | 1990-09-01,-0.8004020811204204,0.13 491 | 1990-10-01,0.1749222864604881,0.22 492 | 1990-11-01,-0.5989990267618917,0.15 493 | 1990-12-01,-0.33155114841768457,0.28 494 | 1991-01-01,0.5292829916030121,0.39 495 | 1991-02-01,0.0367013819050356,0.29 496 | 1991-03-01,-0.9812917283640559,0.06 497 | 1991-04-01,-1.0269206256846506,0.31 498 | 1991-05-01,-1.7871683752497434,0.45 499 | 1991-06-01,-0.4577070884234423,0.72 500 | 1991-07-01,-0.1287516601089363,0.89 501 | 1991-08-01,-0.7154528821850759,0.8 502 | 1991-09-01,-1.693518112126012,0.59 503 | 1991-10-01,-1.2516283992362371,0.83 504 | 1991-11-01,-0.7922245192658056,1.2 505 | 1991-12-01,-1.766077088590812,1.63 506 | 1992-01-01,-2.476024742721972,1.69 507 | 1992-02-01,-0.8477417558037135,1.6 508 | 1992-03-01,-2.2982093086073943,1.33 509 | 1992-04-01,-1.5414309596166678,1.26 510 | 1992-05-01,0.16754703518010788,1.1 511 | 1992-06-01,-1.1933077662471834,0.66 512 | 1992-07-01,-0.6589055546740321,0.28 513 | 1992-08-01,0.17886322054632348,0.03 514 | 1992-09-01,0.03317288115145136,-0.16 515 | 1992-10-01,-1.6677056825644598,-0.39 516 | 1992-11-01,-0.7922245192658056,-0.13 517 | 1992-12-01,-0.6389495641690275,0.03 518 | 1993-01-01,-0.7855391421641517,0.2 519 | 1993-02-01,-0.7213927361310145,0.37 520 | 1993-03-01,-0.7705849155251341,0.38 521 | 1993-04-01,-1.7619353884446274,0.7 522 | 1993-05-01,-0.6925277454089556,0.79 523 | 1993-06-01,-1.5202414008355023,0.39 524 | 1993-07-01,-1.0565209755978353,0.31 525 | 1993-08-01,-1.4820095416690393,0.08 526 | 1993-09-01,-0.8004020811204204,0.31 527 | 1993-10-01,-1.3110680111402109,0.22 528 | 1993-11-01,-0.019322549250296202,0.19 529 | 1993-12-01,0.0783134059174781,0.1 530 | 1994-01-01,-0.12812807528062314,0.07 531 | 1994-02-01,0.0367013819050356,0.14 532 | 1994-03-01,-0.9812917283640559,0.2 533 | 1994-04-01,-1.9089383409966565,0.33 534 | 1994-05-01,-1.1616594439121375,0.36 535 | 1994-06-01,-0.9481075403059672,0.4 536 | 1994-07-01,-1.785482580624795,0.39 537 | 1994-08-01,-1.673648706540085,0.47 538 | 1994-09-01,-1.7530591808597316,0.42 539 | 1994-10-01,-1.3705076230442526,0.75 540 | 1994-11-01,-0.7922245192657323,1.22 541 | 1994-12-01,-1.2537463956718298,1.33 542 | 1995-01-01,-0.36291774202471005,1.07 543 | 1995-02-01,-0.30022933722211376,0.84 544 | 1995-03-01,0.3356258518792817,0.58 545 | 1995-04-01,-1.173923578236763,0.37 546 | 1995-05-01,-0.7707163618261674,-0.04 547 | 1995-06-01,-0.049040045188020215,0.11 548 | 1995-07-01,0.4676714712766553,-0.04 549 | 1995-08-01,0.11498349892256908,-0.51 550 | 1995-09-01,0.21179608735254254,-0.62 551 | 1995-10-01,-0.1222757730597193,-0.71 552 | 1995-11-01,0.04508594825100845,-0.91 553 | 1995-12-01,-0.5877164948771758,-0.82 554 | 1996-01-01,0.8579885250447763,-0.84 555 | 1996-02-01,0.03670138190498772,-0.85 556 | 1996-03-01,0.6516860711376947,-0.6 557 | 1996-04-01,0.8106162812152081,-0.38 558 | 1996-05-01,0.2457356515973197,-0.26 559 | 1996-06-01,1.4221613104595547,-0.2 560 | 1996-07-01,0.7327484185592409,-0.26 561 | 1996-08-01,0.49826182866458707,-0.18 562 | 1996-09-01,0.6285835684885123,-0.39 563 | 1996-10-01,0.4126807340765865,-0.3 564 | 1996-11-01,-0.08373104675160085,-0.3 565 | 1996-12-01,0.7955763760040421,-0.56 566 | 1997-01-01,0.4823250582542267,-0.58 567 | 1997-02-01,1.1738425589590868,-0.37 568 | 1997-03-01,-0.7179082123153885,-0.2 569 | 1997-04-01,-1.3944280070647226,0.18 570 | 1997-05-01,-2.021734224501201,0.72 571 | 1997-06-01,-2.3375754873064394,1.17 572 | 1997-07-01,-0.9902517387771136,1.58 573 | 1997-08-01,-1.929167593034739,1.76 574 | 1997-09-01,-1.5148949059249206,2.06 575 | 1997-10-01,-1.6677056825643923,2.35 576 | 1997-11-01,-1.4363094942787056,2.39 577 | 1997-12-01,-0.9975810492123968,2.36 578 | 1998-01-01,-2.3351509426755093,2.31 579 | 1998-02-01,-1.7321848935123667,1.88 580 | 1998-03-01,-2.7722996374949838,1.33 581 | 1998-04-01,-2.1294427698247,0.85 582 | 1998-05-01,0.16754703518019678,0.56 583 | 1998-06-01,1.095227675871143,-0.28 584 | 1998-07-01,1.5279792604068472,-0.76 585 | 1998-08-01,1.0092996016539686,-1.01 586 | 1998-09-01,1.0453710496244144,-1.17 587 | 1998-10-01,1.06651646502084,-1.33 588 | 1998-11-01,1.140030405772968,-1.25 589 | 1998-12-01,1.2566739996311729,-1.57 590 | 1999-01-01,1.56235752527725,-1.71 591 | 1999-02-01,0.7105628201592386,-1.25 592 | 1999-03-01,0.9150695871863624,-0.9 593 | 1999-04-01,1.6926339965272137,-0.88 594 | 1999-05-01,0.16754703518010788,-0.92 595 | 1999-06-01,0.1144267721061858,-1.03 596 | 1999-07-01,0.6002099449179481,-1.0 597 | 1999-08-01,0.2427429421699326,-1.11 598 | 1999-09-01,-0.08590925631592021,-1.06 599 | 1999-10-01,0.8881976293087831,-1.17 600 | 1999-11-01,1.1400304057728947,-1.54 601 | 1999-12-01,1.2054409303392628,-1.72 602 | 2000-01-01,0.5292829916030121,-1.79 603 | 2000-02-01,1.1317262190681392,-1.6 604 | 2000-03-01,0.9677462903961078,-1.06 605 | 2000-04-01,1.6191325202512834,-0.81 606 | 2000-05-01,0.4803015008489552,-0.81 607 | 2000-06-01,-0.4577070884234423,-0.72 608 | 2000-07-01,-0.3275593705708755,-0.59 609 | 2000-08-01,0.5621415502882688,-0.47 610 | 2000-09-01,0.9262889121570428,-0.51 611 | 2000-10-01,0.9476372412128248,-0.67 612 | 2000-11-01,2.106157868292245,-0.73 613 | 2000-12-01,0.6931102374202803,-0.86 614 | 2001-01-01,0.8579885250447763,-0.74 615 | 2001-02-01,1.0474935392863878,-0.58 616 | 2001-03-01,0.7043627743473804,-0.44 617 | 2001-04-01,0.1491029947312456,-0.41 618 | 2001-05-01,-0.7707163618261674,-0.21 619 | 2001-06-01,0.27789358940029885,-0.08 620 | 2001-07-01,-0.3275593705708755,0.1 621 | 2001-08-01,-0.8432123254323669,0.03 622 | 2001-09-01,0.09271394988517098,-0.08 623 | 2001-10-01,-0.1817153849636932,-0.12 624 | 2001-11-01,0.6247624257626039,-0.26 625 | 2001-12-01,-0.9975810492123384,-0.36 626 | 2002-01-01,0.29449332485887186,-0.18 627 | 2002-02-01,0.6684464802683867,-0.0 628 | 2002-03-01,-0.4545246962667209,0.15 629 | 2002-04-01,-0.21840438664874284,0.2 630 | 2002-05-01,-1.3180366767464724,0.52 631 | 2002-06-01,-0.5394404970705453,0.79 632 | 2002-07-01,-0.7251747914946786,0.75 633 | 2002-08-01,-1.4181298200453576,0.73 634 | 2002-09-01,-0.8004020811204882,0.92 635 | 2002-10-01,-0.7166718920999314,1.16 636 | 2002-11-01,-0.6634075242631963,1.4 637 | 2002-12-01,-1.1512802570880682,1.4 638 | 2003-01-01,-0.1750860086294085,1.01 639 | 2003-02-01,-0.6792763962401148,0.8 640 | 2003-03-01,-0.6125548058958975,0.52 641 | 2003-04-01,-0.3654073392006881,-0.05 642 | 2003-05-01,-0.6143391289917437,-0.49 643 | 2003-06-01,-1.1933077662471834,-0.12 644 | 2003-07-01,0.3351329976355132,0.38 645 | 2003-08-01,-0.14053538757208536,0.4 646 | 2003-09-01,-0.2645324625170791,0.28 647 | 2003-10-01,-0.1817153849636932,0.55 648 | 2003-11-01,-0.4057735342580509,0.4 649 | 2003-12-01,0.8468094452960101,0.37 650 | 2004-01-01,-1.1142446756059692,0.24 651 | 2004-02-01,0.7947954999410378,0.23 652 | 2004-03-01,0.07224233583061418,0.07 653 | 2004-04-01,-1.2474250545126937,0.12 654 | 2004-05-01,1.4185648978553194,0.12 655 | 2004-06-01,-1.4385079921883994,0.26 656 | 2004-07-01,-0.6589055546740321,0.59 657 | 2004-08-01,-0.7154528821850759,0.75 658 | 2004-09-01,-0.32407353125079874,0.77 659 | 2004-10-01,-0.36003422067581764,0.77 660 | 2004-11-01,-0.9210415142683416,0.65 661 | 2004-12-01,-0.8951149106285771,0.71 662 | 2005-01-01,0.20057745816119446,0.64 663 | 2005-02-01,-2.5745116913301684,0.34 664 | 2005-03-01,0.07224233583061418,0.31 665 | 2005-04-01,-0.8799176731327053,0.23 666 | 2005-05-01,-1.3180366767465612,0.39 667 | 2005-06-01,0.3596269980474019,0.3 668 | 2005-07-01,0.13632528717349865,0.17 669 | 2005-08-01,-0.6515731605613942,0.19 670 | 2005-09-01,0.33087822481998186,-0.02 671 | 2005-10-01,1.06651646502084,-0.07 672 | 2005-11-01,-0.2769565392554416,-0.6 673 | 2005-12-01,-0.07538580195819335,-0.88 674 | 2006-01-01,1.2806099251842713,-1.01 675 | 2006-02-01,-0.005414957985864055,-0.68 676 | 2006-03-01,1.3891599160739518,-0.52 677 | 2006-04-01,1.3986280914232398,-0.25 678 | 2006-05-01,-0.8489049782433792,0.0 679 | 2006-06-01,-0.5394404970704524,0.14 680 | 2006-07-01,-0.7251747914946032,0.09 681 | 2006-08-01,-1.5458892632927936,0.33 682 | 2006-09-01,-0.6217788749193293,0.49 683 | 2006-10-01,-1.5488264587563771,0.69 684 | 2006-11-01,-0.21254804175421016,1.08 685 | 2006-12-01,-0.43401728700144615,1.18 686 | 2007-01-01,-0.6916232754665276,0.71 687 | 2007-02-01,-0.2581129973312141,0.16 688 | 2007-03-01,-0.08578777379862228,-0.07 689 | 2007-04-01,-0.14490291037272846,-0.25 690 | 2007-05-01,-0.14520743048865056,-0.36 691 | 2007-06-01,0.6048272239887108,-0.15 692 | 2007-07-01,-0.4600978442121683,-0.34 693 | 2007-08-01,0.3066226637936144,-0.63 694 | 2007-09-01,0.09271394988517098,-0.85 695 | 2007-10-01,0.5315599578846019,-1.03 696 | 2007-11-01,0.8179879182665178,-1.26 697 | 2007-12-01,1.3591401382149342,-1.3 698 | 2008-01-01,1.421483725230734,-1.59 699 | 2008-02-01,1.889820337104141,-1.64 700 | 2008-03-01,1.2311298064447154,-1.16 701 | 2008-04-01,0.5166103761113177,-0.83 702 | 2008-05-01,-0.2233960469058624,-0.72 703 | 2008-06-01,0.5230938153415149,-0.55 704 | 2008-07-01,0.2688637608148668,-0.28 705 | 2008-08-01,0.9454198800302868,-0.12 706 | 2008-09-01,1.2835353245592929,-0.12 707 | 2008-10-01,1.304274912637006,-0.12 708 | 2008-11-01,1.5908898882819542,-0.41 709 | 2008-12-01,1.2054409303392628,-0.81 710 | 2009-01-01,0.9519043917424536,-0.95 711 | 2009-02-01,1.30019157863169,-0.74 712 | 2009-03-01,0.07224233583061418,-0.54 713 | 2009-04-01,0.8841177574913059,-0.16 714 | 2009-05-01,-0.5361505125746208,0.24 715 | 2009-06-01,-0.1307734538351232,0.43 716 | 2009-07-01,0.26886376081479146,0.53 717 | 2009-08-01,-0.39605427406673976,0.6 718 | 2009-09-01,0.33087822481998186,0.71 719 | 2009-10-01,-1.4299472349482938,0.99 720 | 2009-11-01,-0.727816021764501,1.54 721 | 2009-12-01,-0.7926487720447571,1.79 722 | 2010-01-01,-0.9733708755594532,1.58 723 | 2010-02-01,-1.3110214946034657,1.32 724 | 2010-03-01,-1.0339684315738016,1.0 725 | 2010-04-01,1.4721295676991706,0.64 726 | 2010-05-01,1.105810432186472,0.11 727 | 2010-06-01,0.27789358940029885,-0.46 728 | 2010-07-01,2.124402391792514,-0.9 729 | 2010-08-01,1.9036157043852229,-1.3 730 | 2010-09-01,2.414815630499695,-1.52 731 | 2010-10-01,1.7797918078692705,-1.52 732 | 2010-11-01,1.5264813907806494,-1.42 733 | 2010-12-01,2.6399668705124486,-1.41 734 | 2011-01-01,1.984978925416692,-1.59 735 | 2011-02-01,1.974053016885941,-1.15 736 | 2011-03-01,2.126633761010269,-0.96 737 | 2011-04-01,2.4276487592871905,-0.6 738 | 2011-05-01,0.3239242680145316,-0.25 739 | 2011-06-01,0.1144267721061858,-0.05 740 | 2011-07-01,1.1303638394829687,-0.19 741 | 2011-08-01,0.2427429421699326,-0.39 742 | 2011-09-01,1.104912118358134,-0.65 743 | 2011-10-01,0.7098787935967262,-0.8 744 | 2011-11-01,1.2044389032741991,-1.03 745 | 2011-12-01,2.230102316177228,-1.08 746 | 2012-01-01,0.9519043917424536,-0.83 747 | 2012-02-01,0.20516674146853844,-0.67 748 | 2012-03-01,0.3356258518792817,-0.41 749 | 2012-04-01,-0.5124102917527169,-0.27 750 | 2012-05-01,-0.14520743048856166,-0.2 751 | 2012-06-01,-0.9481075403058744,-0.0 752 | 2012-07-01,-0.1287516601089363,0.09 753 | 2012-08-01,-0.4599339956904216,0.28 754 | 2012-09-01,0.21179608735254254,0.78 755 | 2012-10-01,0.2343618983645296,0.58 756 | 2012-11-01,0.3671284357574585,0.47 757 | 2012-12-01,-0.6901826334609956,-0.46 758 | 2013-01-01,-0.08117014193173107,-0.79 759 | 2013-02-01,-0.3423456771130134,-0.63 760 | 2013-03-01,1.0730996968155389,-0.45 761 | 2013-04-01,0.1491029947312456,0.01 762 | 2013-05-01,0.9494331993521372,-0.12 763 | 2013-06-01,1.5038947191065648,-0.3 764 | 2013-07-01,0.8652868922005337,-0.33 765 | 2013-08-01,-0.012775944324721828,-0.37 766 | 2013-09-01,0.33087822481998186,-0.27 767 | 2013-10-01,-0.1817153849637608,-0.13 768 | 2013-11-01,0.8179879182664446,-0.19 769 | 2013-12-01,-0.02415273266634169,-0.49 770 | 2014-01-01,1.2336519918354325,-0.64 771 | 2014-02-01,-0.131763977658563,-0.77 772 | 2014-03-01,-1.2446752444127236,-0.48 773 | 2014-04-01,0.8841177574912225,-0.13 774 | 2014-05-01,0.5584901172660781,0.24 775 | 2014-06-01,-0.049040045188020215,0.11 776 | 2014-07-01,-0.2612901337502291,0.01 777 | 2014-08-01,-1.0987312119270942,-0.11 778 | 2014-09-01,-0.8004020811204204,0.17 779 | 2014-10-01,-0.7761115040039729,0.46 780 | 2014-11-01,-1.049858509270951,0.83 781 | 2014-12-01,-0.6389495641690857,0.66 782 | 2015-01-01,-0.7385812088153663,0.52 783 | 2015-02-01,0.0367013819050356,0.47 784 | 2015-03-01,-1.0339684315738016,0.5 785 | 2015-04-01,-0.21840438664874284,0.81 786 | 2015-05-01,-1.2398480603293496,0.91 787 | 2015-06-01,-1.1115743576000805,1.07 788 | 2015-07-01,-1.4541363965215632,1.36 789 | 2015-08-01,-1.929167593034739,1.59 790 | 2015-09-01,-1.8126002495934512,1.85 791 | 2015-10-01,-1.964903742084532,2.0 792 | -------------------------------------------------------------------------------- /notebooks/data/year_month_day.xlsx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nicolasfauchereau/Auckland_Python_Workshop/33365e02449818dd7feadc3b20aeb487ed0624cd/notebooks/data/year_month_day.xlsx -------------------------------------------------------------------------------- /notebooks/images/Vanderplas_Python_Stack.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nicolasfauchereau/Auckland_Python_Workshop/33365e02449818dd7feadc3b20aeb487ed0624cd/notebooks/images/Vanderplas_Python_Stack.png -------------------------------------------------------------------------------- /notebooks/images/fig_map.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nicolasfauchereau/Auckland_Python_Workshop/33365e02449818dd7feadc3b20aeb487ed0624cd/notebooks/images/fig_map.png -------------------------------------------------------------------------------- /notebooks/images/ipython_console.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nicolasfauchereau/Auckland_Python_Workshop/33365e02449818dd7feadc3b20aeb487ed0624cd/notebooks/images/ipython_console.png -------------------------------------------------------------------------------- /notebooks/images/pandas_freqs.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nicolasfauchereau/Auckland_Python_Workshop/33365e02449818dd7feadc3b20aeb487ed0624cd/notebooks/images/pandas_freqs.png -------------------------------------------------------------------------------- /notebooks/images/presentation.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nicolasfauchereau/Auckland_Python_Workshop/33365e02449818dd7feadc3b20aeb487ed0624cd/notebooks/images/presentation.png -------------------------------------------------------------------------------- /notebooks/images/python_console.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nicolasfauchereau/Auckland_Python_Workshop/33365e02449818dd7feadc3b20aeb487ed0624cd/notebooks/images/python_console.png -------------------------------------------------------------------------------- /notebooks/images/qtconsole.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nicolasfauchereau/Auckland_Python_Workshop/33365e02449818dd7feadc3b20aeb487ed0624cd/notebooks/images/qtconsole.png -------------------------------------------------------------------------------- /notebooks/images/split-apply-combine.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nicolasfauchereau/Auckland_Python_Workshop/33365e02449818dd7feadc3b20aeb487ed0624cd/notebooks/images/split-apply-combine.png --------------------------------------------------------------------------------