├── README.md
├── _config.yml
├── images
├── jupyter_dashboard.png
└── navigator.png
└── notebooks
├── 01_test.ipynb
├── 02_resources.ipynb
├── 03_Jupyter_notebook.ipynb
├── 04_Introduction_Python.ipynb
├── 05_Numpy.ipynb
├── 06_Scipy.ipynb
├── 07_Pandas.ipynb
├── 08_xarray.ipynb
├── 09_Plotting.ipynb
├── 10_Mapping.ipynb
├── data
├── AKL_aero_rain_monthly.xlsx
├── Daily_clim_data_Auckland.csv
├── ISO_datetime.xls
├── NICO_NIWA_SOI_1941_2010.csv
├── OULOUP_PMER.csv
├── Wellington_rain.xlsx
├── rainfall_calibration.xlsx
├── soi_nino.csv
└── year_month_day.xlsx
├── images
├── Vanderplas_Python_Stack.png
├── fig_map.png
├── ipython_console.png
├── pandas_freqs.png
├── presentation.png
├── python_console.png
├── qtconsole.png
└── split-apply-combine.png
└── recipes
├── MJO_composite_plot.ipynb
└── MJO_phase_plot.ipynb
/README.md:
--------------------------------------------------------------------------------
1 | # Python for Climate and Meteorological Data Analysis and Visualisation
2 |
3 | [NIWA Auckland](http://www.niwa.co.nz/) / Climate and Weather Applications, Forecasting Services
4 | Auckland, New Zealand, 31 August and 1 September 2017
5 |
6 | Contact:
7 |
8 | Nicolas Fauchereau
9 |
10 | + [@gmail](mailto:Nicolas.Fauchereau@gmail.com)
11 | + [@niwa](mailto:Nicolas.Fauchereau@niwa.co.nz)
12 |
13 |
14 |
15 |
16 | ### Table of contents
17 |
18 | - [The Anaconda python distribution](#the-anaconda-python-distribution)
19 | - [Installation of Some additional libraries](#installation-of-additional-libraries)
20 | - [Running the Jupyter notebooks](#running-the-jupyter-notebooks)
21 | - [Content of the workshop](#contents)
22 |
23 | ## The Anaconda python distribution
24 |
25 | For this tutorial, I **strongly** recommend installing the **Anaconda Python distribution**. It is a completely free *enterprise-ready* Python distribution for large-scale data processing, predictive analytics, and scientific computing. It includes the python interpreter itself, the python standard library as well as a set of packages exposing data structures and methods for data manipulation and scientific computing and visualization. In particular it provides [Numpy](http://www.numpy.org/), [Scipy](http://www.scipy.org/), [Pandas](http://pandas.pydata.org/), [Matplotlib](http://matplotlib.org/), etc ... i.e. all the main packages we will be using during the tutorial. The full list of packages is available at:
26 |
27 | [http://docs.continuum.io/anaconda/pkgs.html](http://docs.continuum.io/anaconda/pkgs.html)
28 |
29 | The Anaconda python distribution (**NOTE**: select the version shipping with Python 3.6) must be downloaded from:
30 |
31 | [http://continuum.io/downloads](http://continuum.io/downloads)
32 |
33 | For your platform.
34 |
35 | You should not need administrator rights, as Anaconda is completely self-contained and can be installed in your `HOME` directory. I suggest installing `Anaconda` in `/Users/USERNAME/anaconda` on Macs, and `C:\Users\USERNAME\anaconda` on Windows
36 |
37 | Once you have installed Anaconda, you can update to the latest compatible versions of all the pre-installed packages by running, at the command line, i.e. in Windows select `Cmd.exe`, on Mac the `Terminal` application in `Utilities`, the `$` sign signifies the command prompt, so do not enter it !, also when prompted `do you want to proceed?` just enter yes (`y`) :
38 |
39 | ```
40 | $ conda update conda
41 | ```
42 |
43 | Then
44 |
45 | ```
46 | $ conda update anaconda
47 | ```
48 |
49 | You also might want to install [pip](https://github.com/pypa/pip) to install packages from the [Python Package Index](http://pypi.python.org/pypi).
50 |
51 | ```
52 | $ conda install pip
53 | ```
54 |
55 | ## Installation of additional libraries
56 |
57 | ### netcdf4
58 |
59 | [netcdf4](https://github.com/Unidata/netcdf4-python) allows you to read and write netcdf files (version 3 and 4 supported), install it by:
60 |
61 | ```
62 | $ conda install netcdf4
63 | ```
64 |
65 | ### Basemap
66 |
67 | **Basemap** is a graphic library for plotting (static, publication quality) geographical maps (see [http://matplotlib.org/basemap/](http://matplotlib.org/basemap/)). **Basemap** is available directly in **Anaconda** for Mac and Linux using the conda package manager, install with :
68 |
69 | ```
70 | $ conda install basemap
71 | ```
72 |
73 | but on Windows, you'll have to use an alternative channel (conda-forge), the command becomes:
74 |
75 | ```
76 | $ conda install -c conda-forge basemap
77 | ```
78 |
79 | ### Cartopy
80 |
81 | [**Cartopy**](http://scitools.org.uk/cartopy/) is a the second library for making geographical maps that we are going to see. It has been developed by the UKMO, and will eventually replace **Basemap**. However to this date Cartopy does not have all the features present in Basemap.
82 |
83 | Cartopy is not available through the anaconda standard channel, to install you need to use the community maintained *conda-forge* channel, with this syntax:
84 |
85 | ```
86 | $ conda install -c conda-forge cartopy
87 | ```
88 |
89 | ### Seaborn
90 |
91 | [seaborn](http://web.stanford.edu/~mwaskom/software/seaborn/) is a Python visualization library based on matplotlib. It provides a high-level interface for drawing attractive statistical graphics. You should be able to install it with ```conda``` as well:
92 |
93 | ```
94 | $ conda install seaborn
95 | ```
96 | ### xarray
97 |
98 | [xarray](https://github.com/xarray/xarray) (previously *xray*) is a library aimed at bringing the power of Pandas to multidimensional labelled arrays, such as the ones usually associated with geophysical quantities varying along time and space dimensions (e.g. [time, latitudes, longitudes], [time, level, latitudes, longitudes], etc) and supports reading and writing netcdf files. It can be installed via `conda`:
99 |
100 | ```
101 | $ conda install xarray
102 | ```
103 |
104 | All of this can also be done using the Anaconda Navigator GUI (Graphical User Interface) which looks like below
105 |
106 | 
107 |
108 | If you go the `Environments` tab on the left, and select the `root` environment (we will talk about the concept of Python environments later in the workshop) you can then search, select and install available packages, including the ones mentionned above.
109 |
110 |
111 | ## Running the Jupyter notebook
112 |
113 | The material of the tutorial is in the form of [Jupyter notebooks](http://jupyter-notebook-beginner-guide.readthedocs.io/en/latest/what_is_jupyter.html). In a nutshell a Jupyter notebook is a web-based (i.e. running in the browser) interactive computational environment where you can combine Python code execution, text, rendered mathematic expression, plots, images, and rich media into a **single document**.
114 |
115 | It is in my opinion the best environment when you are doing **exploratory data analysis**, and it allows you to weave comments, background information, interpretations and references along with the code itself, and it can be exported to a variety of formats (HTML, pdf) for sharing with colleagues. I **strongly** recommend spending some time learning about the notebook and its many features.
116 |
117 | After uncompressing the archive of the repo (or after cloning it with ```git```), navigate to the `notebooks` directory (containing the ```*.ipynb``` files) and type:
118 |
119 | ```
120 | $ jupyter notebook
121 | ```
122 |
123 | That should bring up the Jupyter notebook dashboard (looking as below) in your default Browser, you should be ready to go !
124 |
125 | 
126 |
127 | ## Contents
128 |
129 | The following is a brief description of the workshop material content:
130 |
131 |
132 |
133 | + `01_test.ipynb`:
134 |
135 | A simple test Jupyter notebook to test the installation of the main libraries and their version
136 |
137 | + `02_resources.ipynb`:
138 |
139 | Some links to interesting resources and topics not covered during the workshop
140 |
141 | + `03_Jupyter_notebook.ipynb`
142 |
143 | Introduction to the main features of the **Jupyter notebook**
144 |
145 | + `04_introduction_Python.ipynb`: Introduction the **basics** of the python language:
146 |
147 | + what's an *interpreted programming language*, what is the python *interpreter*
148 |
149 | + how to write a python script
150 |
151 | + basic Python native data types:
152 | + dealing with numbers in native Python
153 | + dealing with `strings` and their formatting
154 | + `lists`, `tuples`, `dictionnaries`, ...
155 |
156 | + arithmetic operators (`+. -, *, =`)
157 |
158 | + logical operators (`==, >=, <=, !=`) and how they relate to booleans (`True / False`)
159 |
160 | + control flow structures (`for, if / elif / else, while` etc.)
161 |
162 | + reusing your code: writing *functions*, *modules* and *packages*
163 |
164 | + `05_Numpy.ipynb`:
165 |
166 | A introduction to Numpy: Numpy introduces in particular a data structure called the `ndarray` (*N-dimensional* array) which can store numerical values. It is the foundational library of the Python *scientific ecosystem*. In this notebook we'll learn the main principles of manipulating numpy *ndarrays*. While you might actually not spend much time working with numpy itself, it is necessary to understand its basic principles, as (almost) everything else in scientific Python is built on top of `numpy`. We'll see:
167 |
168 | + How to create numpy arrays
169 |
170 | + indexing, slicing, reshaping, transposing, etc.
171 |
172 | + The main methods and functions in numpy operating on numpy `ndarrays`
173 |
174 | + `06_Scipy.ipynb`:
175 |
176 | [Scipy](http://www.scipy.org) is the second pilar of the python scientific ecosystem. Where Numpy introduces a *data structure* (the ndarray), Scipy provides a collection of efficient *scientific algorithms*. These are organized in *submodules*, and cover topics ranging from linear algebra to signal and image processing and optimisation (see [here](https://docs.scipy.org/doc/scipy/reference/) for list of the submodules and their dedicated tutorial). In this notebook we will focus on **interpolation**, and on some of the **statistical** algorithms and methods that scipy makes available.
177 |
178 | + `07_Pandas.ipynb`
179 |
180 | This is where we're gonna spend quite a bit of time ! [Pandas](http://pandas.pydata.org/) is **THE** library that you need to use when dealing with *tabular* data, *i.e.* "spreadsheet-like" data, where values are stored in 2D arrays with rows and column labels. It is typically the type of data you find in csv, tab / space delimited or Excel files. In this notebook we'll see first:
181 |
182 | + The basics of the main data structures in Pandas: the **Series** and the **Dataframe**
183 | + How to read from / write to common data files (csv, excel, space delimited, tab delimited etc). It will include how to read from a list of e.g. csv files, and concatenating their data in a Dataframe
184 | + How to manipulate tabular data in Pandas, from selecting rows / columns to more sophisticated conditional indexing
185 | + How to perform complex queries and assignements on Pandas Dataframes
186 | + How to perform **groupby** operations (*split / apply / combine*)
187 | + How to deal with Missing Values
188 |
189 | One of the strenths of Pandas is its ability to store, and perform sophisticated operations on, **time-series**: i.e. data that is indexed by dates, dates and times, timestamps, etc. In the second part of the Pandas tutorial we will focus on **time-series manipulation in Pandas**. In particular we'll see:
190 |
191 | + how to read in and correctly parse files containing date / time information
192 |
193 | + how to resample time-series
194 |
195 | + rolling window operations (e.g. moving averages)
196 |
197 | + how to deal with missing values and missing index values
198 |
199 | + `08_xarray.ipynb`:
200 |
201 | [xarray](http://xarray.pydata.org/en/stable/) is a library to read / write netcdf files and for the manipulation of multi-dimensional labelled arrays, it is especially handy for *gridded data* varying along *latitudes, longitudes, time, depth, height*, etc. Its design follows closely that of Pandas, meaning that familiarity with Pandas allows to quickly pick up xarray. We'll see how to read, write netcdf files in xarray, and perform a series of common analyses such as:
202 |
203 | + calculating a climatology
204 | + calculating anomalies
205 | + going from and to Pandas Dataframes
206 | + calculating composite anomalies
207 | + resampling, aggregation, groupby operations
208 |
209 | + `09_plotting.ipynb`:
210 |
211 | In this notebook we'll go over the basics of [**Matplotlib**](https://matplotlib.org/), the main plotting library in python. This is more or less the equivalent of *numpy* but for plotting: *i.e.* a foundational library of the Python scientific ecosystem, on wich a number of — more specialised — plotting libraries have been built. One of these we will briefly go over in particular is [**seaborn**](https://seaborn.pydata.org/), a plotting library handy for statistical plots. A short and non-exhaustive list of plotting libraries in Python is provided covering both static plots (i.e. *plotnine*, *ggplot2*, etc) and interactive plots (*bokeh*, *plotly*, *holoviews*, etc)
212 |
213 | + `10_Mapping.ipynb`:
214 |
215 | Making maps in Python: a brief overview of [**basemap**](https://matplotlib.org/basemap/), [**cartopy**](http://scitools.org.uk/cartopy/), and — very briefly, for interactive maps — [**folium**](https://github.com/python-visualization/folium). We'll see also briefly how to read shapefiles (with [geopandas](http://geopandas.org/)), transform into a [geojson](http://geojson.org/) and edit in the browser.
216 |
217 |
218 |
--------------------------------------------------------------------------------
/_config.yml:
--------------------------------------------------------------------------------
1 | theme: jekyll-theme-cayman
--------------------------------------------------------------------------------
/images/jupyter_dashboard.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/nicolasfauchereau/Auckland_Python_Workshop/33365e02449818dd7feadc3b20aeb487ed0624cd/images/jupyter_dashboard.png
--------------------------------------------------------------------------------
/images/navigator.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/nicolasfauchereau/Auckland_Python_Workshop/33365e02449818dd7feadc3b20aeb487ed0624cd/images/navigator.png
--------------------------------------------------------------------------------
/notebooks/01_test.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "markdown",
5 | "metadata": {},
6 | "source": [
7 | "# This is a test Jupyter Notebook\n",
8 | "\n",
9 | "It imports and display the version number of the foundational libraries we will be using \n",
10 | "\n",
11 | "to run a ```IPython notebook cell``` just click on it (should be highligthed in gray or green) and either enter ```shift-enter``` or use the ```play``` button in the menu\n",
12 | "\n",
13 | "If one of the import fails please come back to me at [Nicolas.Fauchereau@gmail.com](mailto:nicolas.fauchereau@gmail.com), the last library (basemap) is only \n",
14 | "required for making geographical maps, so it might not be essential to all of you. \n",
15 | "\n",
16 | "if you have version number significantly lower than the following, you might \n",
17 | "also run into minor problems during the lectures: \n",
18 | "\n",
19 | "+ **Numpy**: version 1.11\n",
20 | "+ **scipy**: version 0.19\n",
21 | "+ **pandas**: version 0.20\n",
22 | "+ **xarray**: version 0.9\n",
23 | "+ **matplotlib**: version 2.0\n",
24 | "+ **basemap**: version 0.8\n",
25 | "+ **cartopy**: version 0.14\n"
26 | ]
27 | },
28 | {
29 | "cell_type": "code",
30 | "execution_count": null,
31 | "metadata": {},
32 | "outputs": [],
33 | "source": [
34 | "import numpy as np; print(np.__version__)"
35 | ]
36 | },
37 | {
38 | "cell_type": "code",
39 | "execution_count": null,
40 | "metadata": {},
41 | "outputs": [],
42 | "source": [
43 | "import scipy as sp; print(sp.__version__)"
44 | ]
45 | },
46 | {
47 | "cell_type": "code",
48 | "execution_count": null,
49 | "metadata": {},
50 | "outputs": [],
51 | "source": [
52 | "import pandas as pd; print(pd.__version__)"
53 | ]
54 | },
55 | {
56 | "cell_type": "code",
57 | "execution_count": null,
58 | "metadata": {},
59 | "outputs": [],
60 | "source": [
61 | "import xarray as xr; print(xr.__version__)"
62 | ]
63 | },
64 | {
65 | "cell_type": "code",
66 | "execution_count": null,
67 | "metadata": {},
68 | "outputs": [],
69 | "source": [
70 | "import matplotlib as mpl; print(mpl.__version__)"
71 | ]
72 | },
73 | {
74 | "cell_type": "code",
75 | "execution_count": null,
76 | "metadata": {},
77 | "outputs": [],
78 | "source": [
79 | "from mpl_toolkits import basemap; print(basemap.__version__)"
80 | ]
81 | },
82 | {
83 | "cell_type": "code",
84 | "execution_count": null,
85 | "metadata": {},
86 | "outputs": [],
87 | "source": [
88 | "import cartopy; print(cartopy.__version__)"
89 | ]
90 | },
91 | {
92 | "cell_type": "markdown",
93 | "metadata": {},
94 | "source": [
95 | "The following command calls matplotlib with the `inline` backend, which allows to display plots inline inside the notebook, if this command fails or a plot does not appear, please come back to me "
96 | ]
97 | },
98 | {
99 | "cell_type": "code",
100 | "execution_count": null,
101 | "metadata": {
102 | "collapsed": true
103 | },
104 | "outputs": [],
105 | "source": [
106 | "%matplotlib inline"
107 | ]
108 | },
109 | {
110 | "cell_type": "code",
111 | "execution_count": null,
112 | "metadata": {
113 | "collapsed": true
114 | },
115 | "outputs": [],
116 | "source": [
117 | "import matplotlib.pyplot as plt"
118 | ]
119 | },
120 | {
121 | "cell_type": "code",
122 | "execution_count": null,
123 | "metadata": {},
124 | "outputs": [],
125 | "source": [
126 | "plt.plot(np.sin(np.linspace(-np.pi, np.pi, 100)))"
127 | ]
128 | },
129 | {
130 | "cell_type": "code",
131 | "execution_count": null,
132 | "metadata": {
133 | "collapsed": true
134 | },
135 | "outputs": [],
136 | "source": []
137 | }
138 | ],
139 | "metadata": {
140 | "anaconda-cloud": {},
141 | "kernelspec": {
142 | "display_name": "Python [conda env:IOOS]",
143 | "language": "python",
144 | "name": "conda-env-IOOS-py"
145 | },
146 | "language_info": {
147 | "codemirror_mode": {
148 | "name": "ipython",
149 | "version": 3
150 | },
151 | "file_extension": ".py",
152 | "mimetype": "text/x-python",
153 | "name": "python",
154 | "nbconvert_exporter": "python",
155 | "pygments_lexer": "ipython3",
156 | "version": "3.5.2"
157 | },
158 | "latex_envs": {
159 | "bibliofile": "biblio.bib",
160 | "cite_by": "apalike",
161 | "current_citInitial": 1,
162 | "eqLabelWithNumbers": true,
163 | "eqNumInitial": 0
164 | },
165 | "nav_menu": {},
166 | "toc": {
167 | "colors": {
168 | "hover_highlight": "#DAA520",
169 | "running_highlight": "#FF0000",
170 | "selected_highlight": "#FFD700"
171 | },
172 | "moveMenuLeft": true,
173 | "nav_menu": {
174 | "height": "30px",
175 | "width": "252px"
176 | },
177 | "navigate_menu": true,
178 | "number_sections": true,
179 | "sideBar": true,
180 | "threshold": 4,
181 | "toc_cell": false,
182 | "toc_section_display": "block",
183 | "toc_window_display": false,
184 | "widenNotebook": false
185 | }
186 | },
187 | "nbformat": 4,
188 | "nbformat_minor": 1
189 | }
190 |
--------------------------------------------------------------------------------
/notebooks/02_resources.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "markdown",
5 | "metadata": {},
6 | "source": [
7 | "# resources"
8 | ]
9 | },
10 | {
11 | "cell_type": "markdown",
12 | "metadata": {},
13 | "source": [
14 | "### Object Oriented Programming in Python "
15 | ]
16 | },
17 | {
18 | "cell_type": "markdown",
19 | "metadata": {},
20 | "source": [
21 | "+ [www.google.com](http://www.google.com)\n",
22 | "+ [A good tutorial by Alan Gauld](http://www.freenetpages.co.uk/hp/alan.gauld/tutclass.htm)\n",
23 | "+ The [classes](http://docs.python.org/2/tutorial/classes.html) entry from the official Python doc \n",
24 | "+ A short [pdf](http://www1.gly.bris.ac.uk/~walker/PythonEarthSci/PfES_3.pdf) from Andrew Walker's [Python for Earth Scientists](http://www1.gly.bris.ac.uk/~walker/PythonEarthSci/) course\n"
25 | ]
26 | },
27 | {
28 | "cell_type": "markdown",
29 | "metadata": {},
30 | "source": [
31 | "### Python as a glue: wrapping of C, C++, Fortran or the Cython module "
32 | ]
33 | },
34 | {
35 | "cell_type": "markdown",
36 | "metadata": {},
37 | "source": [
38 | "+ [using python as a *glue*](http://docs.scipy.org/doc/numpy/user/c-info.python-as-glue.html)\n",
39 | "+ [SWIG (Simplified Wrapper and Interface Generator)](http://www.swig.org/Doc1.3/Python.html)\n",
40 | "+ [Cython](http://cython.org)\n",
41 | "+ [f2py](http://www.f2py.com/)\n",
42 | "+ [A notebook on using C and Fortran with Python](http://nbviewer.ipython.org/github/jrjohansson/scientific-python-lectures/blob/master/Lecture-6A-Fortran-and-C.ipynb)"
43 | ]
44 | },
45 | {
46 | "cell_type": "markdown",
47 | "metadata": {},
48 | "source": [
49 | "### Real-time acquisition, physical programming"
50 | ]
51 | },
52 | {
53 | "cell_type": "markdown",
54 | "metadata": {},
55 | "source": [
56 | "+ [pyserial](http://pyserial.sourceforge.net): Module that encapsulates the access for the serial port\n",
57 | "+ [Real World Instrumentation, Automated Data Acquisition and Control Systems](http://www.amazon.com/Real-World-Instrumentation-Python-Acquisition/dp/0596809565): A book available on Amazon.\n",
58 | "+ [Programming Raspberry Pi: Getting Started with Python](http://www.monkmakes.com/?page_id=63): book (not free) with code (free)\n",
59 | "+ [PyVISA](https://pyvisa.readthedocs.org/en/latest/): The PyVISA package enables you to control all kinds of measurement equipment through various busses (GPIB, RS232, USB) with Python programs"
60 | ]
61 | },
62 | {
63 | "cell_type": "markdown",
64 | "metadata": {},
65 | "source": [
66 | "### Image processing"
67 | ]
68 | },
69 | {
70 | "cell_type": "markdown",
71 | "metadata": {},
72 | "source": [
73 | "+ PIL [(Python Image Library)](http://www.pythonware.com/products/pil/): Image processing capabilities to your Python interpreter.\n",
74 | "+ [pillow](http://python-pillow.github.io/): A more user-friendly `fork` of PIL\n",
75 | "+ [scikit-image](http://scikit-image.org): more Image processing, built on top of Numpy / Scipy."
76 | ]
77 | },
78 | {
79 | "cell_type": "markdown",
80 | "metadata": {},
81 | "source": [
82 | "### Symbolic maths"
83 | ]
84 | },
85 | {
86 | "cell_type": "markdown",
87 | "metadata": {},
88 | "source": [
89 | "+ The [SymPy](www.sympy.org) library is a Python library for symbolic mathematics. It supports polynomials, calculus, solving equations, etc\n",
90 | "+ The [sage](http://www.sagemath.org/) software: `Mission: Creating a viable free open source alternative to Magma, Maple, Mathematica and Matlab`"
91 | ]
92 | },
93 | {
94 | "cell_type": "markdown",
95 | "metadata": {},
96 | "source": [
97 | "### Geospatial statistics"
98 | ]
99 | },
100 | {
101 | "cell_type": "markdown",
102 | "metadata": {},
103 | "source": [
104 | "+ [PySAL](http://pysal.readthedocs.org/en/v1.6/): PySAL is a cross-platform library of spatial analysis functions written in Python. It is intended to support the development of high level applications for spatial analysis.\n",
105 | "+ [GeoPandas](http://geopandas.org/): A project based on [Pandas](http://pandas.pydata.org/) to make working with geospatial data in python easier\n",
106 | "+ [Rasterio](https://github.com/mapbox/rasterio): Clean and fast and geospatial raster I/O with Numpy support, developed by the team at [https://www.mapbox.com/](https://www.mapbox.com/)\n",
107 | "+ [Pyproj](https://code.google.com/p/pyproj/): Performs cartographic transformations and geodetic computations. Wrapper around the Proj version 4 library\n",
108 | "+ [Python GDAL/OGR](https://pypi.python.org/pypi/GDAL/): Python bindings + tools around the Geospatial Data Abstraction Library\n",
109 | "+ [Python GIS resources](http://pythongisresources.wordpress.com/interpolation): a blog on geospatial python\n",
110 | "+ [High Performance Geostatistics Library](http://hpgl.mit-ufa.com/): A library written in C++ / Python implementing geostatistical algorithms (*e.g.* kriging, correlograms, etc)"
111 | ]
112 | },
113 | {
114 | "cell_type": "markdown",
115 | "metadata": {},
116 | "source": [
117 | "### computational statistics in Python"
118 | ]
119 | },
120 | {
121 | "cell_type": "markdown",
122 | "metadata": {},
123 | "source": [
124 | "+ [Computational Statistics in Python](https://people.duke.edu/~ccc14/sta-663/): impressive resource on computational stats with Python, from the basics to more advanced topics"
125 | ]
126 | },
127 | {
128 | "cell_type": "markdown",
129 | "metadata": {},
130 | "source": [
131 | "### Bayesian statistics with Python"
132 | ]
133 | },
134 | {
135 | "cell_type": "markdown",
136 | "metadata": {},
137 | "source": [
138 | "+ [Think Bayes](https://github.com/AllenDowney/ThinkBayes2): The code examples and Jupyter notebooks for the (free) book **Think Bayes** by Allen Downey, available [here](http://greenteapress.com/wp/think-bayes/)\n",
139 | "+ [Bayesian Methods for Hackers](https://github.com/CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers): Code examples and Jupyter notebook accompanying the book [Bayesian Methods for Hackers: Probabilistic Programming and Bayesian Inference](https://www.amazon.com/Bayesian-Methods-Hackers-Probabilistic-Addison-Wesley/dp/0133902838)\n",
140 | "+ [pyMC3](http://pymc-devs.github.io/pymc3/): Bayesian statistical modeling and Probabilistic Machine Learning\n",
141 | "+ [Statistical Rethinking in Python](https://github.com/aloctavodia/Statistical-Rethinking-with-Python-and-PyMC3): Porting the example in the book [Statistical Rethinking, by R. MacElreath](http://xcelab.net/rm/statistical-rethinking/) from R/Stan to Python"
142 | ]
143 | },
144 | {
145 | "cell_type": "markdown",
146 | "metadata": {},
147 | "source": [
148 | "### Climate and Meteorology"
149 | ]
150 | },
151 | {
152 | "cell_type": "markdown",
153 | "metadata": {
154 | "collapsed": true
155 | },
156 | "source": [
157 | "+ [The UNIDATA gallery](http://unidata.github.io/python-gallery/index.html)\n",
158 | "+ [MetPy](https://github.com/Unidata/MetPy): a Python package for Meteorological data, can do stuff like plot sounding data, meteograms, etc"
159 | ]
160 | },
161 | {
162 | "cell_type": "markdown",
163 | "metadata": {},
164 | "source": [
165 | "### Biology, ecology"
166 | ]
167 | },
168 | {
169 | "cell_type": "markdown",
170 | "metadata": {},
171 | "source": [
172 | "+ [Biopython](http://biopython.org/wiki/Main_Page): Biopython is a set of freely available tools for biological computation written in Python by an international team of developers. \n",
173 | "+ [GeoEco](https://pypi.python.org/pypi/GeoEco): Open source geoprocessing toolbox designed for coastal and marine researchers and GIS analysts who work with spatially-explicit ecological and oceanographic data. For Windows (> XP) only.\n",
174 | "+ [Galaxy](http://galaxyproject.org/): Galaxy is a scientific workflow, data integration and data and analysis persistence and publishing platform that aims to make computational biology accessible to research scientists that do not have computer programming experience."
175 | ]
176 | },
177 | {
178 | "cell_type": "markdown",
179 | "metadata": {},
180 | "source": [
181 | "### Computational Fluid Dynamics and PDE solvers"
182 | ]
183 | },
184 | {
185 | "cell_type": "markdown",
186 | "metadata": {},
187 | "source": [
188 | "+ [CFD Python: 12 steps to Navier-Stokes](http://lorenabarba.com/blog/cfd-python-12-steps-to-navier-stokes/)\n",
189 | "+ [github repository for the above](https://github.com/barbagroup/CFDPython)\n",
190 | "+ [Python bindings to OpenFOAM](http://openfoamwiki.net/index.php/Contrib_PyFoam)\n",
191 | "+ [FiPy: A Finite Volume PDE Solver Using Python](http://matforge.org/fipy/)\n",
192 | "+ [\"Practical Numerical Methods with Python\" MOOC](http://openedx.seas.gwu.edu/courses/GW/MAE6286/2014_fall/about)"
193 | ]
194 | },
195 | {
196 | "cell_type": "markdown",
197 | "metadata": {},
198 | "source": [
199 | "### Python on the GPU "
200 | ]
201 | },
202 | {
203 | "cell_type": "markdown",
204 | "metadata": {},
205 | "source": [
206 | "+ [PyCUDA](http://mathema.tician.de/software/pycuda/): PyCUDA lets you access Nvidia‘s CUDA parallel computation API from Python"
207 | ]
208 | },
209 | {
210 | "cell_type": "markdown",
211 | "metadata": {},
212 | "source": [
213 | "### Parallel computing with Python / IPython"
214 | ]
215 | },
216 | {
217 | "cell_type": "markdown",
218 | "metadata": {},
219 | "source": [
220 | "+ [IPython parallel introduction](http://ipython.org/ipython-doc/2/parallel/parallel_intro.html)\n",
221 | "+ [Interactive Parallel computing with IPython](https://www.youtube.com/watch?v=y4hgalfhc1Y&index=49&list=PLYx7XA2nY5GfuhCvStxgbynFNrxr3VFog): tutorials by Min. Raglan-Kelley at Scipy2014 (3 parts in the youtube list)\n",
222 | "+ [IPython parallel for distributed computing](https://www.youtube.com/watch?v=eyUeYnT-18s&index=32&list=PLs4CJRBY5F1Jm7H1dlesRvEgr-28QGXpR): A talk at PyCON Australia by Nathan Faggian\n",
223 | "+ [IPython & Jupyter in depth: high productivity interactive and parallel python](https://www.youtube.com/watch?v=05fA_DXgW-Y): A talk at PyCON 2015 by Thomas Kluyver and Kyle Kelley"
224 | ]
225 | },
226 | {
227 | "cell_type": "markdown",
228 | "metadata": {},
229 | "source": [
230 | "### Signal processing in Python"
231 | ]
232 | },
233 | {
234 | "cell_type": "markdown",
235 | "metadata": {},
236 | "source": [
237 | "+ [scipy.signal](http://docs.scipy.org/doc/scipy/reference/tutorial/signal.html)\n",
238 | "+ [Python for signal processing blog](http://python-for-signal-processing.blogspot.co.nz/)\n",
239 | "+ [Python for signal processing github repository](https://github.com/unpingco/Python-for-Signal-Processing)\n",
240 | "+ [Python for signal processing book](http://www.springer.com/engineering/signals/book/978-3-319-01341-1)"
241 | ]
242 | },
243 | {
244 | "cell_type": "markdown",
245 | "metadata": {},
246 | "source": [
247 | "### Python for Matlab and R users "
248 | ]
249 | },
250 | {
251 | "cell_type": "markdown",
252 | "metadata": {},
253 | "source": [
254 | "+ [Moving from MATLAB matrices to NumPy arrays - A Matrix Cheatsheet](http://sebastianraschka.com/Articles/2014_matlab_vs_numpy.html)\n",
255 | "\n",
256 | "+ [NumPy for Matlab Users](http://wiki.scipy.org/NumPy_for_Matlab_Users)\n"
257 | ]
258 | },
259 | {
260 | "cell_type": "markdown",
261 | "metadata": {},
262 | "source": [
263 | "### Some URLs and blogs, notebooks repositories for climatology / meteorology / oceanography"
264 | ]
265 | },
266 | {
267 | "cell_type": "markdown",
268 | "metadata": {},
269 | "source": [
270 | "+ [Python Scientific Lectures Notes](http://scipy-lectures.github.io): Tutorial material on the scientific Python ecosystem, a quick introduction to central tools and techniques.\n",
271 | "\n",
272 | "+ [Python for earth scientists](http://www1.gly.bris.ac.uk/~walker/PythonEarthSci/): A two afternoons course by Andrew Walker (University of Bristol) on Python in the earth sciences. \n",
273 | "\n",
274 | "+ [oceanpython](http://oceanpython.org/): Python for oceanography \n",
275 | "\n",
276 | "+ [PyAOS](http://pyaos.johnny-lin.com/): Python for the Atmospheric and Oceanic Sciences \n",
277 | "\n",
278 | "+ [python4oceanographers](http://ocefpaf.github.io/): Learn python with examples applied to marine sciences.\n",
279 | "\n",
280 | "+ [http://earthpy.org/](http://earthpy.org/): EarthPy is a collection of IPython notebooks with examples of Earth Science related Python code\n",
281 | "\n",
282 | "+ [Jupyter notebook gallery from Unidata](http://unidata.github.io/notebook-gallery/): Exploring using Python for Meteorology"
283 | ]
284 | },
285 | {
286 | "cell_type": "markdown",
287 | "metadata": {},
288 | "source": [
289 | "### Github repositories: notebooks and accompanying material"
290 | ]
291 | },
292 | {
293 | "cell_type": "markdown",
294 | "metadata": {},
295 | "source": [
296 | "+ [A gallery of interesting IPython Notebooks](https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks)\n",
297 | "\"...a curated collection of IPython notebooks that are notable for some reason.\"\n",
298 | "\n",
299 | "+ [Scikit-learn tutorial](https://github.com/jakevdp/sklearn_scipy2013): Files and other info associated with the Scipy 2013 scikit-learn tutorial developed by Gaël Varoquaux, Olivier Grisel and Jake VanderPlas.\n",
300 | "\n",
301 | "+ [Statistical Analysis tutorial](https://github.com/fonnesbeck/statistical-analysis-python-tutorial) from Chris. Fonnesbeck. \n",
302 | "\n",
303 | "+ [Bayesian Statistical Analysis in Python](https://github.com/fonnesbeck/scipy2014_tutorial): Ipython notebooks for the Scipy 2014 tutorial on Bayesian data analysis with Python, by Chris. Fonnesbeck. \n",
304 | "\n",
305 | "+ [AstroML: Machine Learning and Data Mining for Astronomy](http://www.astroml.org): A library and tutorial by Jake VanderPlas and co-authors, accompanying the book [Statistics, Data Mining, and Machine Learning in Astronomy](http://www.amazon.com/Statistics-Mining-Machine-Learning-Astronomy/dp/0691151687/)\n",
306 | "\n",
307 | "+ [Scientific Python lectures from Robert Johansson](https://github.com/jrjohansson/scientific-python-lectures)\n",
308 | "\n",
309 | "+ [Data Science in Python](http://blog.yhathq.com/posts/data-science-in-python-tutorial.html): A series of annotated notebooks on data science (i.e. geared towards machine learning) in python\n",
310 | "\n",
311 | "+ [Pythonic perambulations](http://jakevdp.github.io): A blog by Jake VanderPlas\n"
312 | ]
313 | },
314 | {
315 | "cell_type": "markdown",
316 | "metadata": {},
317 | "source": [
318 | "### Some books "
319 | ]
320 | },
321 | {
322 | "cell_type": "markdown",
323 | "metadata": {},
324 | "source": [
325 | "+ [Python for Data Analysis](http://shop.oreilly.com/product/0636920023784.do): From Wes McKinney (Developer of [Pandas](www.pandas.pydata.org))\n",
326 | "\n",
327 | "+ [Think stats](http://www.greenteapress.com/thinkstats/): Probability and statistics for programmers, from Allen Downey, pdf available for free. \n",
328 | "\n",
329 | "+ [Think complexity](http://www.greenteapress.com/compmod/): Complexity science (graphs, cellular automata, agent-based models), from Allen Downey, pdf available for free.\n",
330 | "\n",
331 | "+ [Python in hydrology](http://www.greenteapress.com/pythonhydro/pythonhydro.html): A book freely available in pdf, from Sat Kumar Tomer.\n",
332 | "\n",
333 | "+ [Programming collective intelligence](http://shop.oreilly.com/product/9780596529321.do): By Toby Segaran, Good intro on (general) Machine Learning algorithms.\n",
334 | "\n",
335 | "+ [Machine Learning in action](http://www.manning.com/pharrington/): By Peter Harrington.\n",
336 | "\n",
337 | "+ [Introduction to Python for Econometrics, Statistics and Numerical Analysis: Second Edition](http://www.kevinsheppard.com/images/0/09/Python_introduction.pdf): By Kevin Sheppard, Oxford Uni.\n",
338 | "\n",
339 | "+ [A Hands-On Introduction to Using Python in the Atmospheric and Oceanic Sciences](http://www.johnny-lin.com/pyintro/): By [Johnny Lin](http://www.johnny-lin.com), Professor of physics and head of the Climate Research Group at North Park University.\n",
340 | "\n",
341 | "+ [principles of planetary climate](http://geosci.uchicago.edu/~rtp1/PrinciplesPlanetaryClimate/index.html): by Ray Pierrehumbert. Excellent book on the physics of planetary climates, with freely downloadable python code to follow the examples given in the book. "
342 | ]
343 | },
344 | {
345 | "cell_type": "markdown",
346 | "metadata": {},
347 | "source": [
348 | "### Some interesting libraries, built on top of the main Scientific stack"
349 | ]
350 | },
351 | {
352 | "cell_type": "markdown",
353 | "metadata": {},
354 | "source": [
355 | "+ [PYMC](http://pymc-devs.github.io/pymc/): By Chris Fonnesbeck, Bayesian statistical models and fitting algorithms, including Markov chain Monte Carlo. \n",
356 | "\n",
357 | "+ [Seaborn](http://www.stanford.edu/~mwaskom/software/seaborn/): Statistical data visualization, by Michael Waskom. Its [graphical representation of linear models](http://www.stanford.edu/~mwaskom/software/seaborn/linear_models.html) is particularly interesting. \n",
358 | "\n",
359 | "+ [ggplot](https://github.com/yhat/ggplot/): For [R](http://cran.r-project.org) users, a 'port' of the [ggplot2](http://ggplot2.org) package to Python, see [here](http://blog.yhathq.com/posts/ggplot-0.4-released.html) for what's new in the latest release. \n",
360 | "\n",
361 | "+ [coards](https://pypi.python.org/pypi/coards): A COARDS compliant time parser. See also [netcdftime](http://netcdf4-python.googlecode.com/svn/trunk/docs/netcdftime.netcdftime-module.html) which is part of the [NetCDF4 module](https://code.google.com/p/netcdf4-python/)\n",
362 | "\n",
363 | "+ [seawater](http://www.imr.no/~bjorn/python/seawater/index.html): Similar to the MATLAB toolboxes SEAWATER from CSIRO and parts of OCEANS from Woods Hole Institute.\n",
364 | "\n",
365 | "+ [fluid](https://pypi.python.org/pypi/fluid): Procedures to study fluids on Python, focused for oceanography, meteorology and related sciences.\n",
366 | "\n",
367 | "+ [kyPyWavelet](https://github.com/regeirk/kPyWavelet): Continuous wavelet transform module for Python *ala* Torrence and Compo. Some manual edits were necessary to make it work for me ... \n",
368 | "\n",
369 | "+ [pyresample](https://code.google.com/p/pyresample/): Resampling (reprojection) of geospatial image data in Python\n",
370 | "\n",
371 | "+ [Rpy2](http://rpy.sourceforge.net/rpy2.html): calling R from Python"
372 | ]
373 | },
374 | {
375 | "cell_type": "markdown",
376 | "metadata": {},
377 | "source": [
378 | "### Some articles on open and reproducible research"
379 | ]
380 | },
381 | {
382 | "cell_type": "markdown",
383 | "metadata": {},
384 | "source": [
385 | "+ [Reproducible Research in Computational Science](http://dx.doi.org/10.1126/science.1213847), Roger D. Peng, Science 334, 1226 (2011).\n",
386 | "\n",
387 | "+ [Shining Light into Black Boxes](http://dx.doi.org/10.1126/science.1218263), A. Morin et al., Science 336, 159-160 (2012).\n",
388 | " \n",
389 | "+ [The case for open computer programs](http://dx.doi.org/doi:10.1038/nature10836), D.C. Ince, Nature 482, 485 (2012).\n",
390 | "\n",
391 | "+ [Best practices for scientific computing](http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1001745): Paper in PLOS Biology exposing some of the tools and methods to build better Scientific software.\n"
392 | ]
393 | },
394 | {
395 | "cell_type": "markdown",
396 | "metadata": {},
397 | "source": [
398 | "## Acknowledgments"
399 | ]
400 | },
401 | {
402 | "cell_type": "markdown",
403 | "metadata": {},
404 | "source": [
405 | "In building this material, I have liberally '*borrowed*' from lecture notes, online notebooks, video recording of talks, code examples, articles, etc, that are freely available online, I would like to acknowledge in particular (list not exhaustive and in no particular order): \n",
406 | "\n",
407 | "+ [Fernando Perez](http://blog.fperez.org): Lead developer of [IPython](www.ipython.org), great advocate of [open science](http://en.wikipedia.org/wiki/Open_science) and reproducible research\n",
408 | "+ Brian Granger: Lead developer of the [IPython notebook](http://ipython.org/notebook.html)\n",
409 | "+ [Wes McKinney](http://blog.wesmckinney.com): Lead developer of Pandas, author of [Python for Data Analysis](http://shop.oreilly.com/product/0636920023784.do)\n",
410 | "+ [Christopher Fonnesbeck](http://biostat.mc.vanderbilt.edu/wiki/Main/ChrisFonnesbeck): developer of [PYMC](http://pymc-devs.github.io/pymc/) (Bayesian statistical models and fitting algorithms, including Markov chain Monte Carlo)\n",
411 | "+ [Jake VanderPlas](http://www.astro.washington.edu/users/vanderplas/): Astronomer, contributor to [scikit-learn](http://scikit-learn.org)\n",
412 | "+ [Gaël Varoquaux](http://gael-varoquaux.info): One of the main contributors to [scikit-learn](http://scikit-learn.org)\n",
413 | "+ [Olivier Grisel](http://ogrisel.com): Also main contributor to [scikit-learn](http://scikit-learn.org)\n",
414 | "+ [Travis Oliphant](http://technicaldiscovery.blogspot.co.nz): One of the lead developers of Numpy / Scipy, and founder of [Continuum Analytics](http://continuum.io).\n",
415 | "+ J.R. Johansson http://dml.riken.jp/~rob/\n",
416 | "+ [Thomas Wiecki](http://twiecki.github.io/): developer of [PYMC3](https://github.com/pymc-devs/pymc), a fork of PYMC. \n",
417 | "+ [Skipper Seabold](http://jseabold.net/): The main developer of [statsmodels](http://statsmodels.sourceforge.net/)\n",
418 | "+ [Kevin Sheppard](http://www.kevinsheppard.com/wiki/Main_Page) from University of Oxford: author or the book [Introduction to Python for Econometrics, Statistics and Numerical Analysis: Second Edition](http://www.kevinsheppard.com/images/0/09/Python_introduction.pdf). \n",
419 | "\n",
420 | "And finally [John Hunter](http://blog.fperez.org/2013/07/in-memoriam-john-d-hunter-iii-1968-2012.html), He was the founder and lead developer of [Matplotlib](http://matplotlib.org), a pivotal library to make Python a viable free and open-source alternative to commercial scientific software, and very sadly passed away in 2012.\n",
421 | "\n",
422 | "I encourage you - if you carry on using Python for science - to look up these people on google, have a look at their github repositories and the projects they contribute to, and follow them on twitter, some of them (*e.g.* Jake VanderPlas) have also very informative blogs. You can also donate to the [John Hunter memorial fund](http://numfocus.org/johnhunter.html) as a way to give back to an important contributor to the Python scientific community.\n",
423 | "\n",
424 | "---"
425 | ]
426 | },
427 | {
428 | "cell_type": "markdown",
429 | "metadata": {},
430 | "source": [
431 | "
\n",
31 | "\n",
32 | "[IPython](www.ipython.org) is an enhanced interactive interpreter, developped for scientific development, adding e.g. history, tab-completion, interactive help, and lots and lots of goodies: \n",
33 | "\n",
34 | "
\n",
35 | "\n",
36 | "
\n",
37 | "\n",
38 | "The [jupyter qtconsole](http://jupyter.org/qtconsole/stable/) provides a number of enhancements only possible in a GUI, such as inline figures, proper multiline editing with syntax highlighting, graphical calltips, and much more. To launch it: \n",
39 | "\n",
40 | "```\n",
41 | "$ jupyter qtconsole\n",
42 | "```\n",
43 | "\n",
44 | "
\n",
45 | "\n",
46 | "
\n",
47 | "\n",
48 | "Since 2005, the IPython development team has introduced the **IPython (now Jupyter) notebook**. This is a web application, running in the browser, that is connected to a Python kernel running in the background. "
49 | ]
50 | },
51 | {
52 | "cell_type": "code",
53 | "execution_count": null,
54 | "metadata": {
55 | "collapsed": true
56 | },
57 | "outputs": [],
58 | "source": [
59 | "from IPython.display import HTML\n",
60 | "HTML('')"
61 | ]
62 | },
63 | {
64 | "cell_type": "markdown",
65 | "metadata": {},
66 | "source": [
67 | ""
68 | ]
69 | },
70 | {
71 | "cell_type": "markdown",
72 | "metadata": {},
73 | "source": [
74 | "It behaves as an **interactive notebook**, in which you can weave Python code and outputs, figures generated from Python / matplotlib, images (either local and remote), websites, videos and richly formatted comments using [markdown](https://en.wikipedia.org/wiki/Markdown), which is a superset of HTML with a very simple syntax (see [here](www.markdown.com) for more) \n",
75 | "\n",
76 | "It is structured into executable cells, into which by default you can run arbitrary python code\n",
77 | "\n",
78 | "It has a sophisticated tab completion and help system \n",
79 | "\n",
80 | "The power of IPython / Jupyter comes in part because of its numerous extensions and *magic* functions\n",
81 | "\n",
82 | "And finally you can *export* a notebook in different formats, including HTML and Latex (and PDF). "
83 | ]
84 | },
85 | {
86 | "cell_type": "markdown",
87 | "metadata": {},
88 | "source": [
89 | "Recently the Jupyter / Ipython notebook as been featured in the **toolbox** section of [Nature](http://www.nature.com)"
90 | ]
91 | },
92 | {
93 | "cell_type": "code",
94 | "execution_count": null,
95 | "metadata": {
96 | "collapsed": true
97 | },
98 | "outputs": [],
99 | "source": [
100 | "from IPython.display import HTML\n",
101 | "HTML('')"
102 | ]
103 | },
104 | {
105 | "cell_type": "markdown",
106 | "metadata": {},
107 | "source": [
108 | "## Using the Jupyter notebook"
109 | ]
110 | },
111 | {
112 | "cell_type": "markdown",
113 | "metadata": {},
114 | "source": [
115 | "To start a Jupyter notebook session open a command prompt, navigate to a desired working directory then issue the following command:\n",
116 | "\n",
117 | "`$ jupyter notebook`\n",
118 | "\n",
119 | "A new window open in your browser where you can open an existing notebook or start a new one.\n",
120 | "\n",
121 | "it should look like \n",
122 | "\n",
123 | "
\n",
124 | "\n",
125 | "
\n",
126 | "\n",
127 | "Notebooks are organized with **cells**. You may have a **code cell** for inputing commands followed by its result cell which contains the output of the code. You may also have a text cell, such as this one you're reading right now. Cell type can be changed in the above dropdown menu.\n",
128 | "\n",
129 | "There is the menubar above for navigating a notebook but you will find the following shortcuts helpful:\n",
130 | "\n",
131 | "- `Enter` : Enter the edit mode of a cell\n",
132 | "- `Esc` : Leave the edit mode and get into the *control* mode (see icons on the right of the menu bar)\n",
133 | "- `Shift + Enter` : Execute cell and advance to next cell (or use the play button in the toolbar)\n",
134 | "- `Alt + Enter` : Execute cell and insert an empty cell below\n",
135 | "- `Ctrl + Enter` : Execute cell in place (do not advance to the next cell)\n",
136 | "- Press `esc` (command mode) then `h` to display keyboard shortcuts\n",
137 | "\n",
138 | "At times you might run code that gets stuck in an infinite loop or simply you want to clear all your workspace variables and start over. To do that you can do:\n",
139 | "\n",
140 | "`Kernel -> Interrupt`\n",
141 | "\n",
142 | "`Kernel -> Restart`\n"
143 | ]
144 | },
145 | {
146 | "cell_type": "code",
147 | "execution_count": null,
148 | "metadata": {
149 | "collapsed": true
150 | },
151 | "outputs": [],
152 | "source": [
153 | "# This is a code cell, containing python code, it is executed by - or -\n",
154 | "\n",
155 | "def square_that(x):\n",
156 | " \"\"\"\n",
157 | " this is a function that squares a number\n",
158 | " \"\"\"\n",
159 | " z = x**2\n",
160 | " return z"
161 | ]
162 | },
163 | {
164 | "cell_type": "code",
165 | "execution_count": null,
166 | "metadata": {
167 | "collapsed": true
168 | },
169 | "outputs": [],
170 | "source": [
171 | "square_that(2)"
172 | ]
173 | },
174 | {
175 | "cell_type": "markdown",
176 | "metadata": {},
177 | "source": [
178 | "### Getting help"
179 | ]
180 | },
181 | {
182 | "cell_type": "markdown",
183 | "metadata": {},
184 | "source": [
185 | "Jupyter has a sophisticated help and tab completion system, which allows **introspection**: i.e.\n",
186 | "If you want details regarding the properties and functionality of any Python objects currently loaded into your name space, you can use the `?` to reveal any details that are available:"
187 | ]
188 | },
189 | {
190 | "cell_type": "code",
191 | "execution_count": null,
192 | "metadata": {
193 | "collapsed": true
194 | },
195 | "outputs": [],
196 | "source": [
197 | "square_that?"
198 | ]
199 | },
200 | {
201 | "cell_type": "code",
202 | "execution_count": null,
203 | "metadata": {
204 | "collapsed": true
205 | },
206 | "outputs": [],
207 | "source": [
208 | "from numpy import random \n",
209 | "random.randn?"
210 | ]
211 | },
212 | {
213 | "cell_type": "markdown",
214 | "metadata": {},
215 | "source": [
216 | "### Tab completion"
217 | ]
218 | },
219 | {
220 | "cell_type": "markdown",
221 | "metadata": {},
222 | "source": [
223 | "Because Jupyter allows for introspection, it is able to afford the user the ability to tab-complete commands or objects that have been partially typed. This is done by pressing the `` key at any point during the process of typing a command:"
224 | ]
225 | },
226 | {
227 | "cell_type": "code",
228 | "execution_count": null,
229 | "metadata": {
230 | "collapsed": true
231 | },
232 | "outputs": [],
233 | "source": [
234 | "listA = [1, 2., 'sentence', 1, (1,2), {'answer': 42}] # here I construct a LIST containing different items"
235 | ]
236 | },
237 | {
238 | "cell_type": "code",
239 | "execution_count": null,
240 | "metadata": {
241 | "collapsed": true
242 | },
243 | "outputs": [],
244 | "source": [
245 | "listA"
246 | ]
247 | },
248 | {
249 | "cell_type": "code",
250 | "execution_count": null,
251 | "metadata": {
252 | "collapsed": true
253 | },
254 | "outputs": [],
255 | "source": [
256 | "listA.index(1)"
257 | ]
258 | },
259 | {
260 | "cell_type": "markdown",
261 | "metadata": {},
262 | "source": [
263 | "### including markdown comments "
264 | ]
265 | },
266 | {
267 | "cell_type": "markdown",
268 | "metadata": {},
269 | "source": [
270 | "**see [markdown](http://daringfireball.net/projects/markdown/)**\n",
271 | "\n",
272 | "Using markdown cells, you can insert formatted comments, observations ... directly in between executable code and outputs ... \n",
273 | "\n",
274 | "To toggle from *code cell* (the default) to *markdown cell*, you can use the toolbar, or ```- + ``` (```- + ``` on Macs)\n",
275 | "\n",
276 | "You can *italicize*, **boldface**\n",
277 | "\n",
278 | "\n",
279 | "\n",
280 | "+ build\n",
281 | "\n",
282 | "+ lists\n",
283 | "\n",
284 | "\n",
285 | "1. enumerate \n",
286 | "\n",
287 | "2. stuff\n",
288 | "\n",
289 | "and embed syntax highlighted code meant for illustration instead of execution in Python:\n",
290 | "\n",
291 | "```python\n",
292 | "print('hello')\n",
293 | "```\n",
294 | "\n",
295 | "```python\n",
296 | "def f(x):\n",
297 | " \"\"\"a docstring\"\"\"\n",
298 | " return x**2\n",
299 | "```\n",
300 | "\n",
301 | "or other languages:\n",
302 | "\n",
303 | "```C\n",
304 | "/* Fibonacci Series c language */\n",
305 | "#include\n",
306 | " \n",
307 | "int main()\n",
308 | "{\n",
309 | " int n, first = 0, second = 1, next, c;\n",
310 | " \n",
311 | " printf(\"Enter the number of terms\\n\");\n",
312 | " scanf(\"%d\",&n);\n",
313 | " \n",
314 | " printf(\"First %d terms of Fibonacci series are :-\\n\",n);\n",
315 | " \n",
316 | " for ( c = 0 ; c < n ; c++ )\n",
317 | " {\n",
318 | " if ( c <= 1 )\n",
319 | " next = c;\n",
320 | " else\n",
321 | " {\n",
322 | " next = first + second;\n",
323 | " first = second;\n",
324 | " second = next;\n",
325 | " }\n",
326 | " printf(\"%d\\n\",next);\n",
327 | " }\n",
328 | " \n",
329 | " return 0;\n",
330 | "}\n",
331 | "```"
332 | ]
333 | },
334 | {
335 | "cell_type": "markdown",
336 | "metadata": {},
337 | "source": [
338 | "You can insert images seemlessly into a Markdown cell, but if it is **local** instead of on the web, it needs to reside in an **images** folder in the **same directory as your notebook**. \n",
339 | "\n",
340 | "The syntax is then \n",
341 | "\n",
342 | " \n",
343 | " \n",
344 | "e.g.\n",
345 | "\n",
346 | "
"
249 | ]
250 | },
251 | {
252 | "cell_type": "markdown",
253 | "metadata": {},
254 | "source": [
255 | "##### Method of moments"
256 | ]
257 | },
258 | {
259 | "cell_type": "markdown",
260 | "metadata": {},
261 | "source": [
262 | "The ***method of moments*** simply assigns the empirical mean and variance to their theoretical counterparts, so that we can solve for the parameters.\n",
263 | "\n",
264 | "So, for the gamma distribution, it turns out (see the relevant section in [the wikipedia article](http://en.wikipedia.org/wiki/Gamma_distribution)) that the mean and variance are:\n",
265 | "\n",
266 | "
"
270 | ]
271 | },
272 | {
273 | "cell_type": "markdown",
274 | "metadata": {},
275 | "source": [
276 | "So, if we solve for $\\alpha$ and $\\beta$, using the **sample** mean ($\\bar{X}$) and variance ($S^2$), we can use a gamma distribution to describe our data:\n",
277 | "\n",
278 | "