| \n", 133 | " | spcs | \n", 134 | "name | \n", 135 | "epsg | \n", 136 | "
|---|---|---|---|
| 66 | \n", 141 | "3101 | \n", 142 | "NAD83 / New York East | \n", 143 | "32115 | \n", 144 | "
| 67 | \n", 147 | "3102 | \n", 148 | "NAD83 / New York Central | \n", 149 | "32116 | \n", 150 | "
| 68 | \n", 153 | "3103 | \n", 154 | "NAD83 / New York West | \n", 155 | "32117 | \n", 156 | "
| 69 | \n", 159 | "3104 | \n", 160 | "NAD83 / New York Long Island | \n", 161 | "32118 | \n", 162 | "
| 195 | \n", 165 | "13101 | \n", 166 | "NAD27 / New York East | \n", 167 | "32015 | \n", 168 | "
| 196 | \n", 171 | "13102 | \n", 172 | "NAD27 / New York Central | \n", 173 | "32016 | \n", 174 | "
| 197 | \n", 177 | "13103 | \n", 178 | "NAD27 / New York West | \n", 179 | "32017 | \n", 180 | "
| 198 | \n", 183 | "13104 | \n", 184 | "NAD27 / New York Long Island | \n", 185 | "32018 | \n", 186 | "
x=np.sqrt(x) can be confusing to people who are used to algebraic notation. In algebra, the equal sign describes a relationship between the left and right sides. So, $x = \\sqrt{x}$ tells us about how the quantity $x$ and the quantity $\\sqrt{x}$ are related. Students are usually trained to 'solve' such a relationship, going through a series of algebraic steps to find values for $x$ that are consistent with the mathematical statement (for $x = \\sqrt{x}$, the solutions are $x = 0$ and $x = 1$). In contrast, the assignment command x = np.sqrt(x) is a way of replacing the previous values stored in x with new values that are the square-root of the old ones.\n",
335 | ""
336 | ]
337 | },
338 | {
339 | "cell_type": "markdown",
340 | "metadata": {},
341 | "source": [
342 | "If you want to change the value of `x`, you need to use the assignment operator:"
343 | ]
344 | },
345 | {
346 | "cell_type": "code",
347 | "collapsed": false,
348 | "input": [
349 | "x = np.sqrt(x)"
350 | ],
351 | "language": "python",
352 | "metadata": {},
353 | "outputs": []
354 | },
355 | {
356 | "cell_type": "markdown",
357 | "metadata": {},
358 | "source": [
359 | "### Connecting Computations\n",
360 | "\n",
361 | "The brilliant thing about organizing operators in terms of unput arguments and output values is that the output of one operator can be used as an input to another. This lets complicated computations be built out of simpler ones.\n",
362 | "\n",
363 | "For example, suppose you have a list of 10000 voters in a precinct and you want to select a random sample of 20 of them for a survey. The `np.arange` operator can be used to generate a set of 10000 choices. The `np.random.choice` operator can then be used to select a subset of these values at random.\n",
364 | "\n",
365 | "One way to connect the computations is by using objects to store the intermediate outputs:"
366 | ]
367 | },
368 | {
369 | "cell_type": "code",
370 | "collapsed": false,
371 | "input": [
372 | "choices = np.arange(1, 10000)\n",
373 | "np.random.choice(choices, 20, replace=False) # sample _without_ replacement"
374 | ],
375 | "language": "python",
376 | "metadata": {},
377 | "outputs": []
378 | },
379 | {
380 | "cell_type": "markdown",
381 | "metadata": {},
382 | "source": [
383 | "You can also pass the output of an operator *directly* as an argument to another operator. Here's another way to accomplish exactly the same thing as the above (note that the values will differ because we are performing a *random* sample):"
384 | ]
385 | },
386 | {
387 | "cell_type": "code",
388 | "collapsed": false,
389 | "input": [
390 | "np.random.choice(np.arange(1, 10000), 20, replace=False)"
391 | ],
392 | "language": "python",
393 | "metadata": {},
394 | "outputs": []
395 | },
396 | {
397 | "cell_type": "markdown",
398 | "metadata": {},
399 | "source": [
400 | "### Numbers and Arithmetic\n",
401 | "\n",
402 | "The `Python` language has a concise notation for arithmetic that looks very much like the traditional one:"
403 | ]
404 | },
405 | {
406 | "cell_type": "code",
407 | "collapsed": false,
408 | "input": [
409 | "7. + 2."
410 | ],
411 | "language": "python",
412 | "metadata": {},
413 | "outputs": []
414 | },
415 | {
416 | "cell_type": "code",
417 | "collapsed": false,
418 | "input": [
419 | "3. * 4."
420 | ],
421 | "language": "python",
422 | "metadata": {},
423 | "outputs": []
424 | },
425 | {
426 | "cell_type": "code",
427 | "collapsed": false,
428 | "input": [
429 | "5. / 2."
430 | ],
431 | "language": "python",
432 | "metadata": {},
433 | "outputs": []
434 | },
435 | {
436 | "cell_type": "code",
437 | "collapsed": false,
438 | "input": [
439 | "3. - 8."
440 | ],
441 | "language": "python",
442 | "metadata": {},
443 | "outputs": []
444 | },
445 | {
446 | "cell_type": "code",
447 | "collapsed": false,
448 | "input": [
449 | "-3."
450 | ],
451 | "language": "python",
452 | "metadata": {},
453 | "outputs": []
454 | },
455 | {
456 | "cell_type": "code",
457 | "collapsed": false,
458 | "input": [
459 | "5.**2. # same as 5^2 (or 5 to the power of 2)"
460 | ],
461 | "language": "python",
462 | "metadata": {},
463 | "outputs": []
464 | },
465 | {
466 | "cell_type": "markdown",
467 | "metadata": {},
468 | "source": [
469 | "Arithmetic operators, like any other operators, can be connected to form more complicated computations. For instance:"
470 | ]
471 | },
472 | {
473 | "cell_type": "code",
474 | "collapsed": false,
475 | "input": [
476 | "8. + 4. / 2."
477 | ],
478 | "language": "python",
479 | "metadata": {},
480 | "outputs": []
481 | },
482 | {
483 | "cell_type": "markdown",
484 | "metadata": {},
485 | "source": [
486 | "The a human reader, the command `8+4/2` might seem ambiguous. Is it intended to be `(8+4)/2` or `8+(4/2)`? The computer uses unambiguous rules to interpret the expression, but it's a good idea for you to use parentheses so that you can make sure that what you *intend* is what the computer carries out:"
487 | ]
488 | },
489 | {
490 | "cell_type": "code",
491 | "collapsed": false,
492 | "input": [
493 | "(8. + 4.) / 2."
494 | ],
495 | "language": "python",
496 | "metadata": {},
497 | "outputs": []
498 | },
499 | {
500 | "cell_type": "markdown",
501 | "metadata": {},
502 | "source": [
503 | "Traditional mathematical notations uses superscripts and radicals to indicate exponentials and roots, e.g. $3^2$ or $\\sqrt{3}$ or $\\sqrt[3]{8}$. This special typography doesn't work well with an ordinary keyboard, so `Python` and most other computer languages uses a different notation:"
504 | ]
505 | },
506 | {
507 | "cell_type": "code",
508 | "collapsed": false,
509 | "input": [
510 | "3.**2."
511 | ],
512 | "language": "python",
513 | "metadata": {},
514 | "outputs": []
515 | },
516 | {
517 | "cell_type": "code",
518 | "collapsed": false,
519 | "input": [
520 | "np.sqrt(3.) # or 3.**0.5"
521 | ],
522 | "language": "python",
523 | "metadata": {},
524 | "outputs": []
525 | },
526 | {
527 | "cell_type": "code",
528 | "collapsed": false,
529 | "input": [
530 | "8.**(1./3.)"
531 | ],
532 | "language": "python",
533 | "metadata": {},
534 | "outputs": []
535 | },
536 | {
537 | "cell_type": "markdown",
538 | "metadata": {},
539 | "source": [
540 | "There is a large set of mathematical functions: exponentials, logs, trigonometric and inverse trigonometric functions, etc. Some examples:\n",
541 | "\n",
542 | "| Traditional | \n", 545 | "Python | \n", 546 | "
|---|---|
| $e^2$ | \n", 549 | "np.exp(2) | \n",
550 | "
| $\\log_{e}(100)$ | \n", 553 | "np.log(100) | \n",
554 | "
| $\\log_{10}(100)$ | \n", 557 | "np.log10(100) | \n",
558 | "
| $\\log_{2}(100)$ | \n", 561 | "np.log2(100) | \n",
562 | "
| $\\cos(\\frac{\\pi}{2})$ | \n", 565 | "np.cos(np.pi/2) | \n",
566 | "
| $\\sin(\\frac{\\pi}{2})$ | \n", 569 | "np.sin(np.pi/2) | \n",
570 | "
| $\\tan(\\frac{\\pi}{2})$ | \n", 573 | "np.tan(np.pi/2) | \n",
574 | "
| $\\cos^{-1}(-1)$ | \n", 577 | "np.acos(-1) | \n",
578 | "
Python should have any trouble with a computation like $\\sqrt{-9}$; the result is the imaginary number $3\\jmath$ (imaginary numbers may be represented by a $\\jmath$ or a $\\imath$, depending on the field). Python works with complex numbers, but you have to explicitly tell the system that this is what you want to do. To calculate $\\sqrt{-9}$ for example, simply use np.sqrt(-9+0j).\n",
623 | ""
624 | ]
625 | },
626 | {
627 | "cell_type": "code",
628 | "collapsed": false,
629 | "input": [
630 | "np.float64(1.) / 0."
631 | ],
632 | "language": "python",
633 | "metadata": {},
634 | "outputs": []
635 | },
636 | {
637 | "cell_type": "code",
638 | "collapsed": false,
639 | "input": [
640 | "np.float64(0.) / 0."
641 | ],
642 | "language": "python",
643 | "metadata": {},
644 | "outputs": []
645 | },
646 | {
647 | "cell_type": "markdown",
648 | "metadata": {},
649 | "source": [
650 | "### Types of Objects\n",
651 | "\n",
652 | "Most of the examples used so far have dealt with numbers. But computers work with other kinds of information as well: text, photographs, sounds, sets of data, and so on. The word **type** is used to refer to the *kind* of information. Modern computer languages support a great variety of types. It's important to know about the types of data because operators expect their input arguments to be of specific types. When you use the wrong type of input, the computer might not be able to process your command."
653 | ]
654 | },
655 | {
656 | "cell_type": "markdown",
657 | "metadata": {},
658 | "source": [
659 | "\n",
660 | "In Python, data frames are not 'built in' as part of the basic language, but the excellent ['pandas'][pandas] library provides data frames and a whole slew of other functionality for researchers doing data analysis with Python. We will be learning more about 'pandas' comming up.\n",
661 | "\n",
662 | "\n",
663 | "[pandas]: http://pandas.pydata.org/"
664 | ]
665 | },
666 | {
667 | "cell_type": "markdown",
668 | "metadata": {},
669 | "source": [
670 | "#### A Note on Strings\n",
671 | "\n",
672 | "Whenever you refer to an object name, make sure that you don't use quotes. For example, in the following, we are first assigning the string `\"python\"` to the `name` object, and then returning (and printing automatically) the `name` object."
673 | ]
674 | },
675 | {
676 | "cell_type": "code",
677 | "collapsed": false,
678 | "input": [
679 | "name = \"python\"\n",
680 | "name"
681 | ],
682 | "language": "python",
683 | "metadata": {},
684 | "outputs": []
685 | },
686 | {
687 | "cell_type": "markdown",
688 | "metadata": {},
689 | "source": [
690 | "If you make a command with the object name in quotes, it won't be treated as referring to an object. Instead, it will merely mean the text itself:"
691 | ]
692 | },
693 | {
694 | "cell_type": "code",
695 | "collapsed": false,
696 | "input": [
697 | "\"name\""
698 | ],
699 | "language": "python",
700 | "metadata": {},
701 | "outputs": []
702 | },
703 | {
704 | "cell_type": "markdown",
705 | "metadata": {},
706 | "source": [
707 | "Similarly, if you omit the quotation marks from around the text, the computer will treat it as if it were an object name and will look for the object of that name. For instance, the following command directs the computer to look up the value contained in an object named `python` and insert that value into the object `name`:"
708 | ]
709 | },
710 | {
711 | "cell_type": "code",
712 | "collapsed": false,
713 | "input": [
714 | "name = python"
715 | ],
716 | "language": "python",
717 | "metadata": {},
718 | "outputs": []
719 | },
720 | {
721 | "cell_type": "markdown",
722 | "metadata": {},
723 | "source": [
724 | "As it happens, there was no object named `python` because it had not been defined by any previous assignment command. So, the computer generated an error."
725 | ]
726 | }
727 | ],
728 | "metadata": {}
729 | }
730 | ]
731 | }
--------------------------------------------------------------------------------
/notebooks/Notebook0b.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "metadata": {
3 | "name": "",
4 | "signature": "sha256:af2901089fe431c3c9794d6878e7eeff568c2bc204f2f019c92180eac7e09600"
5 | },
6 | "nbformat": 3,
7 | "nbformat_minor": 0,
8 | "worksheets": [
9 | {
10 | "cells": [
11 | {
12 | "cell_type": "markdown",
13 | "metadata": {},
14 | "source": [
15 | "# Geospatial Data in Python: Database, Desktop, and the Web\n",
16 | "## Tutorial (Part 0b)"
17 | ]
18 | },
19 | {
20 | "cell_type": "markdown",
21 | "metadata": {},
22 | "source": [
23 | "# Working with Data in Python\n",
24 | "\n",
25 | "Data is central to GIS, and the tabular arrangement of data is very common. Accordingly, Python provides a large number of ways to read in tabular data. These vary depending on how the data are stored, where they are located, etc. To help keep things as simple as possible, the 'pandas' Python library provides an operator, `read_csv()` that allows you to access data \ufb01les in tabular format on your computer as well as data stored in online repositories, or one that a course instructor might set up for his or her students.\n",
26 | "\n",
27 | "If you successfully ran the install script, then you already have 'pandas' installed. Now, you simply need to `import pandas` in order to to use `read_csv()`, as well as a variety of other 'pandas' operators that you will encounter later (it is also usually a good idea to `import numpy as np` at the same time that we `import pandas as pd`)."
28 | ]
29 | },
30 | {
31 | "cell_type": "code",
32 | "collapsed": false,
33 | "input": [
34 | "import pandas as pd\n",
35 | "import numpy as np"
36 | ],
37 | "language": "python",
38 | "metadata": {},
39 | "outputs": []
40 | },
41 | {
42 | "cell_type": "markdown",
43 | "metadata": {},
44 | "source": [
45 | "You need do this only once in each session of Python, and on systems such as IPython, the library will sometimes be reloaded automatically (if you get an error message, it\u2019s likely that the 'pandas' library has not been installed on your system.)\n",
46 | "\n",
47 | "Reading in a data table with `read_csv()` is simply a matter of knowing the name (and location) of the data set. For instance, one data table I use in my statistics classes is `\"swim100m.csv\"`. To read in this data table and create an object in Python that contains the data, use a command like this:"
48 | ]
49 | },
50 | {
51 | "cell_type": "code",
52 | "collapsed": false,
53 | "input": [
54 | "swim = pd.read_csv(\"http://www.mosaic-web.org/go/datasets/swim100m.csv\")"
55 | ],
56 | "language": "python",
57 | "metadata": {},
58 | "outputs": []
59 | },
60 | {
61 | "cell_type": "markdown",
62 | "metadata": {},
63 | "source": [
64 | "The csv part of the name in `\"swim100m.csv\"` indicates that the \ufb01le has been stored in a particular data format, comma-separated values that is handled by spreadsheet software as well as many other kinds of software. The part of this command that requires creativity is choosing a name for the Python object that will hold the data. In the above command it is called `swim`, but you might prefer another name (e.g., `s` or `sdata` or even `ralph`). Of course, it's sensible to choose names that are short, easy to type and remember, and remind you what the contents of the object are about."
65 | ]
66 | },
67 | {
68 | "cell_type": "markdown",
69 | "metadata": {},
70 | "source": [
71 | "### Data Frames\n",
72 | "\n",
73 | "The type of Python object created by `read_csv()` is called a data frame and is essentially a tabular layout. To illustrate, here are the \ufb01rst several cases of the `swim` data frame created by the previous use of `read_csv()`:\n"
74 | ]
75 | },
76 | {
77 | "cell_type": "code",
78 | "collapsed": false,
79 | "input": [
80 | "swim.head()"
81 | ],
82 | "language": "python",
83 | "metadata": {},
84 | "outputs": []
85 | },
86 | {
87 | "cell_type": "markdown",
88 | "metadata": {},
89 | "source": [
90 | "Note that the `head()` function, one of several functions built-into 'pandas' data frames, is a function of the Python object (data frame) itself; not from the main 'pandas' library.\n",
91 | "\n",
92 | "Data frames, like tabular data generally, involve variables and cases. In 'pandas' data frames, each of the variables is given a name. You can refer to the variable by name in a couple of di\ufb00erent ways. To see the variable names in a data frame, something you might want to do to remind yourself of how names a spelled and capitalized, use the `columns` attribute of the data frame object:"
93 | ]
94 | },
95 | {
96 | "cell_type": "code",
97 | "collapsed": false,
98 | "input": [
99 | "swim.columns"
100 | ],
101 | "language": "python",
102 | "metadata": {},
103 | "outputs": []
104 | },
105 | {
106 | "cell_type": "markdown",
107 | "metadata": {},
108 | "source": [
109 | "Note that we have **not** used brackets `()` in the above command. This is because `columns` is not a function; it is an *attribute* of the data frame. Attributes add 'extra' information (or metadata) to objects in the form of additional Python objects. In this case, the attributes describe the names (and data types) of the columns. Another way to get quick information about the variables in a data frame is with `describe()`:"
110 | ]
111 | },
112 | {
113 | "cell_type": "code",
114 | "collapsed": false,
115 | "input": [
116 | "swim.describe()"
117 | ],
118 | "language": "python",
119 | "metadata": {},
120 | "outputs": []
121 | },
122 | {
123 | "cell_type": "markdown",
124 | "metadata": {},
125 | "source": [
126 | "This provides a numerical summary of each of the variables contained in the data frame. To keep things simple, the output from `describe()` is itself a data frame.\n",
127 | "\n",
128 | "There are lots of different functions and attributes available for data frames (and any other Python objects). For instance, to see how many cases and variables there are in a data frame, you can use the `shape` attribute:"
129 | ]
130 | },
131 | {
132 | "cell_type": "code",
133 | "collapsed": false,
134 | "input": [
135 | "swim.shape"
136 | ],
137 | "language": "python",
138 | "metadata": {},
139 | "outputs": []
140 | },
141 | {
142 | "cell_type": "markdown",
143 | "metadata": {},
144 | "source": [
145 | "### Variables in Data Frames\n",
146 | "\n",
147 | "Perhaps the most common operation on a data frame is to refer to the values in a single variable. The two ways you will most commonly use involve referring to a variable by string-quoted name (`swim[\"year\"]`) and as an attribute of a data frame without quotes (`swim.year`)."
148 | ]
149 | },
150 | {
151 | "cell_type": "markdown",
152 | "metadata": {},
153 | "source": [
154 | "\n",
155 | "Each column or variable in a 'pandas' data frame is called a 'series', and each series can contain one of many different data types. For more information on series', data frames, and other objects in 'pandas', [have a look here][intro].\n",
156 | "\n",
157 | "\n",
158 | "[intro]: http://pandas.pydata.org/pandas-docs/dev/dsintro.html"
159 | ]
160 | },
161 | {
162 | "cell_type": "markdown",
163 | "metadata": {},
164 | "source": [
165 | "Most of the statistical/mathematical functions you will encounter in this tutorial are designed to work with data frames and allow you to refer directly to variables within a data frame. For instance:"
166 | ]
167 | },
168 | {
169 | "cell_type": "code",
170 | "collapsed": false,
171 | "input": [
172 | "swim.year.mean()"
173 | ],
174 | "language": "python",
175 | "metadata": {},
176 | "outputs": []
177 | },
178 | {
179 | "cell_type": "code",
180 | "collapsed": false,
181 | "input": [
182 | "swim[\"year\"].min()"
183 | ],
184 | "language": "python",
185 | "metadata": {},
186 | "outputs": []
187 | },
188 | {
189 | "cell_type": "markdown",
190 | "metadata": {},
191 | "source": [
192 | "It is also possible to combine 'numpy' operators with 'pandas' variables:"
193 | ]
194 | },
195 | {
196 | "cell_type": "code",
197 | "collapsed": false,
198 | "input": [
199 | "np.min(swim[\"year\"])"
200 | ],
201 | "language": "python",
202 | "metadata": {},
203 | "outputs": []
204 | },
205 | {
206 | "cell_type": "code",
207 | "collapsed": false,
208 | "input": [
209 | "np.min(swim.year)"
210 | ],
211 | "language": "python",
212 | "metadata": {},
213 | "outputs": []
214 | },
215 | {
216 | "cell_type": "markdown",
217 | "metadata": {},
218 | "source": [
219 | "The `swim` portion of the above commands tells Python which data frame we want to operate on. Leaving o\ufb00 that argument leads to an error:"
220 | ]
221 | },
222 | {
223 | "cell_type": "code",
224 | "collapsed": false,
225 | "input": [
226 | "year.min()"
227 | ],
228 | "language": "python",
229 | "metadata": {},
230 | "outputs": []
231 | },
232 | {
233 | "cell_type": "markdown",
234 | "metadata": {},
235 | "source": [
236 | "The advantage of referring to variables by name becomes evident when you construct statements that involve more than one variable within a data frame. For instance, here's a calculation of the mean year, separately for (grouping by) the different sexes:"
237 | ]
238 | },
239 | {
240 | "cell_type": "code",
241 | "collapsed": false,
242 | "input": [
243 | "swim.groupby('sex')['year'].mean()"
244 | ],
245 | "language": "python",
246 | "metadata": {},
247 | "outputs": []
248 | },
249 | {
250 | "cell_type": "markdown",
251 | "metadata": {},
252 | "source": [
253 | "Both the `mean()` and `min()` functions have been arranged by the 'pandas' library to look in the data frame when interpreting variables, but not all Python functions are designed this way. For instance:"
254 | ]
255 | },
256 | {
257 | "cell_type": "code",
258 | "collapsed": false,
259 | "input": [
260 | "swim.year.sqrt()"
261 | ],
262 | "language": "python",
263 | "metadata": {},
264 | "outputs": []
265 | },
266 | {
267 | "cell_type": "markdown",
268 | "metadata": {},
269 | "source": [
270 | "When you encounter a function that isn't supported by data frames, you can use 'numpy' functions and the special `apply` function built-into data frames (note that the `func` argument is optional):"
271 | ]
272 | },
273 | {
274 | "cell_type": "code",
275 | "collapsed": false,
276 | "input": [
277 | "swim.year.apply(func=np.sqrt).head() # There are 62 cases in total"
278 | ],
279 | "language": "python",
280 | "metadata": {},
281 | "outputs": []
282 | },
283 | {
284 | "cell_type": "markdown",
285 | "metadata": {},
286 | "source": [
287 | "Alternatively, since columns are basically just arrays, we can use built-in numpy functions directly on the columns:"
288 | ]
289 | },
290 | {
291 | "cell_type": "code",
292 | "collapsed": false,
293 | "input": [
294 | "np.sqrt(swim.year).head() # Again, there are 62 cases in total"
295 | ],
296 | "language": "python",
297 | "metadata": {},
298 | "outputs": []
299 | },
300 | {
301 | "cell_type": "markdown",
302 | "metadata": {},
303 | "source": [
304 | "### Adding a New Variable\n",
305 | "\n",
306 | "Sometimes you will compute a new quantity from the existing variables and want to treat this as a new variable. Adding a new variable to a data frame can be done similarly to *accessing* a variable. For instance, here is how to create a new variable in `swim` that holds the `time` converted from seconds to units of minutes:"
307 | ]
308 | },
309 | {
310 | "cell_type": "code",
311 | "collapsed": false,
312 | "input": [
313 | "swim['minutes'] = swim.time/60. # or swim['time']/60."
314 | ],
315 | "language": "python",
316 | "metadata": {},
317 | "outputs": []
318 | },
319 | {
320 | "cell_type": "markdown",
321 | "metadata": {},
322 | "source": [
323 | "By default, columns get inserted at the end. The `insert` function is available to insert at a particular location in the columns. "
324 | ]
325 | },
326 | {
327 | "cell_type": "code",
328 | "collapsed": false,
329 | "input": [
330 | "swim.insert(1, 'mins', swim.time/60.)"
331 | ],
332 | "language": "python",
333 | "metadata": {},
334 | "outputs": []
335 | },
336 | {
337 | "cell_type": "markdown",
338 | "metadata": {},
339 | "source": [
340 | "You could also, if you want, rede\ufb01ne an existing variable, for instance:"
341 | ]
342 | },
343 | {
344 | "cell_type": "code",
345 | "collapsed": false,
346 | "input": [
347 | "swim['time'] = swim.time/60."
348 | ],
349 | "language": "python",
350 | "metadata": {},
351 | "outputs": []
352 | },
353 | {
354 | "cell_type": "markdown",
355 | "metadata": {},
356 | "source": [
357 | "As always, we can take a quick look at the results of our operations by using the `head()` fuction of our data frame:"
358 | ]
359 | },
360 | {
361 | "cell_type": "code",
362 | "collapsed": false,
363 | "input": [
364 | "swim.head()"
365 | ],
366 | "language": "python",
367 | "metadata": {},
368 | "outputs": []
369 | },
370 | {
371 | "cell_type": "markdown",
372 | "metadata": {},
373 | "source": [
374 | "Such assignment operations do not change the original file from which the data were read, only the data frame in the current session of Python. This is an advantage, since it means that your data in the data file stay in their original state and therefore won\u2019t be corrupted by operations made during analysis."
375 | ]
376 | }
377 | ],
378 | "metadata": {}
379 | }
380 | ]
381 | }
--------------------------------------------------------------------------------
/notebooks/Notebook1.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "metadata": {
3 | "name": "",
4 | "signature": "sha256:e997bcc65d4ed8d4e17e808f973e7759a9bfabc34f78eeedbdd31c81671f79ac"
5 | },
6 | "nbformat": 3,
7 | "nbformat_minor": 0,
8 | "worksheets": [
9 | {
10 | "cells": [
11 | {
12 | "cell_type": "markdown",
13 | "metadata": {
14 | "slideshow": {
15 | "slide_type": "slide"
16 | }
17 | },
18 | "source": [
19 | "# Geospatial Data in Python: Database, Desktop, and the Web\n",
20 | "## Tutorial (Part 1)"
21 | ]
22 | },
23 | {
24 | "cell_type": "markdown",
25 | "metadata": {
26 | "slideshow": {
27 | "slide_type": "subslide"
28 | }
29 | },
30 | "source": [
31 | "# Converting coordinates with PyProj"
32 | ]
33 | },
34 | {
35 | "cell_type": "code",
36 | "collapsed": false,
37 | "input": [
38 | "from pyproj import Proj\n",
39 | "\n",
40 | "# Create projection transformation object\n",
41 | "p = Proj(init='epsg:3857') # EPSG code for Web Mercator\n",
42 | "\n",
43 | "# Convert from long/lat to Mercator and back\n",
44 | "print(p(-97.740372, 30.282642))\n",
45 | "print(p(-10880408.440985134, 3539932.8204972977, inverse=True))"
46 | ],
47 | "language": "python",
48 | "metadata": {
49 | "slideshow": {
50 | "slide_type": "-"
51 | }
52 | },
53 | "outputs": []
54 | },
55 | {
56 | "cell_type": "code",
57 | "collapsed": false,
58 | "input": [
59 | "# Fiona (which we will be using shortly) has several \n",
60 | "# helper functions for working with proj4 strings\n",
61 | "from fiona.crs import to_string, from_epsg, from_string\n",
62 | "\n",
63 | "# Create a crs dict from a proj4 string\n",
64 | "crs = from_string('+proj=lcc +lat_1=41.03333333333333 +lat_2=40.66666666666666 '\n",
65 | " '+lat_0=40.16666666666666 +lon_0=-74 +x_0=300000.0000000001 '\n",
66 | " '+y_0=0 +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 +units=us-ft +no_defs')\n",
67 | "\n",
68 | "# Using a proj4 string\n",
69 | "nyc_proj = Proj(crs, preserve_units=True)\n",
70 | "\n",
71 | "# Using an EPSG code\n",
72 | "nyc_epsg = Proj(init='epsg:2263', preserve_units=True)"
73 | ],
74 | "language": "python",
75 | "metadata": {
76 | "slideshow": {
77 | "slide_type": "subslide"
78 | }
79 | },
80 | "outputs": []
81 | },
82 | {
83 | "cell_type": "code",
84 | "collapsed": false,
85 | "input": [
86 | "# Here's about where my office in NYC is located (in long/lat)\n",
87 | "office = (-73.9637, 40.7684)\n",
88 | "\n",
89 | "# Are they close?\n",
90 | "print(nyc_proj(*office))\n",
91 | "print(nyc_epsg(*office))"
92 | ],
93 | "language": "python",
94 | "metadata": {
95 | "slideshow": {
96 | "slide_type": "fragment"
97 | }
98 | },
99 | "outputs": []
100 | },
101 | {
102 | "cell_type": "markdown",
103 | "metadata": {
104 | "slideshow": {
105 | "slide_type": "subslide"
106 | }
107 | },
108 | "source": [
109 | "## Plotting Eyjafjallaj\u00f6kull volcano with Cartopy"
110 | ]
111 | },
112 | {
113 | "cell_type": "code",
114 | "collapsed": false,
115 | "input": [
116 | "import matplotlib.pyplot as plt\n",
117 | "%matplotlib inline\n",
118 | "import cartopy.crs as ccrs\n",
119 | "import matplotlib.pyplot as plt\n",
120 | "import cartopy.io.img_tiles as cimgt\n",
121 | "\n",
122 | "# Create a MapQuest open aerial instance\n",
123 | "map_quest_aerial = cimgt.MapQuestOpenAerial()\n",
124 | "\n",
125 | "# What is the projection?\n",
126 | "print(map_quest_aerial.crs.proj4_init)"
127 | ],
128 | "language": "python",
129 | "metadata": {
130 | "slideshow": {
131 | "slide_type": "-"
132 | }
133 | },
134 | "outputs": []
135 | },
136 | {
137 | "cell_type": "code",
138 | "collapsed": false,
139 | "input": [
140 | "# Specify the lon/lat for the volcano\n",
141 | "volcano = (-19.613333, 63.62)\n",
142 | "\n",
143 | "# Define the plotting extent of the map\n",
144 | "extent = [-22, -15, 63, 65]\n",
145 | "\n",
146 | "# Specify the transform to use when plotting\n",
147 | "transform=ccrs.Geodetic()"
148 | ],
149 | "language": "python",
150 | "metadata": {
151 | "slideshow": {
152 | "slide_type": "-"
153 | }
154 | },
155 | "outputs": []
156 | },
157 | {
158 | "cell_type": "code",
159 | "collapsed": false,
160 | "input": [
161 | "fig = plt.figure(figsize=(10,8))\n",
162 | "# Create a GeoAxes in the tile's projection\n",
163 | "ax = plt.axes(projection=map_quest_aerial.crs)\n",
164 | "ax.set_extent(extent)\n",
165 | "# Add the MapQuest data at zoom level 8\n",
166 | "ax.add_image(map_quest_aerial, 8)\n",
167 | "ax.plot(*volcano, marker='o', color='yellow', markersize=12,\n",
168 | " alpha=0.7, transform=transform)\n",
169 | "ax.set_title(u'Eyjafjallaj\u00f6kull Volcano')\n",
170 | "plt.show()"
171 | ],
172 | "language": "python",
173 | "metadata": {
174 | "slideshow": {
175 | "slide_type": "subslide"
176 | }
177 | },
178 | "outputs": []
179 | },
180 | {
181 | "cell_type": "markdown",
182 | "metadata": {
183 | "slideshow": {
184 | "slide_type": "slide"
185 | }
186 | },
187 | "source": [
188 | "### Time to work on Notebook:\n",
189 | "\n",
190 | "[`Working with Projections in Python`](../exercises/Working with Projections in Python.ipynb)"
191 | ]
192 | }
193 | ],
194 | "metadata": {}
195 | }
196 | ]
197 | }
--------------------------------------------------------------------------------
/notebooks/Notebook2.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "metadata": {
3 | "name": "",
4 | "signature": "sha256:4d754bcea3d58808b0848f3e45708522b0bf3b135b99e028b2f67fdc390578aa"
5 | },
6 | "nbformat": 3,
7 | "nbformat_minor": 0,
8 | "worksheets": [
9 | {
10 | "cells": [
11 | {
12 | "cell_type": "markdown",
13 | "metadata": {
14 | "slideshow": {
15 | "slide_type": "slide"
16 | }
17 | },
18 | "source": [
19 | "# Geospatial Data in Python: Database, Desktop, and the Web\n",
20 | "## Tutorial (Part 2)"
21 | ]
22 | },
23 | {
24 | "cell_type": "code",
25 | "collapsed": false,
26 | "input": [
27 | "from shapely.geometry import LineString\n",
28 | "\n",
29 | "# Dilating a line\n",
30 | "line = LineString([(0, 0), (1, 1), (0, 2), (2, 2), (3, 1), (1, 0)])\n",
31 | "dilated = line.buffer(0.5)\n",
32 | "eroded = dilated.buffer(-0.3)\n",
33 | "\n",
34 | "# Demonstate Python Geo Interface\n",
35 | "print(line.__geo_interface__)"
36 | ],
37 | "language": "python",
38 | "metadata": {
39 | "slideshow": {
40 | "slide_type": "subslide"
41 | }
42 | },
43 | "outputs": []
44 | },
45 | {
46 | "cell_type": "markdown",
47 | "metadata": {
48 | "slideshow": {
49 | "slide_type": "subslide"
50 | }
51 | },
52 | "source": [
53 | "# Exploring the path of Hurican Katrina\n",
54 | "\n",
55 | "The data was originally sourced from the HURDAT2 dataset from [AOML/NOAA](http://www.aoml.noaa.gov/hrd/hurdat/newhurdat-all.html), and the Python lists are from the [cartopy documentation](http://scitools.org.uk/cartopy/docs/latest/examples/hurricane_katrina.html)."
56 | ]
57 | },
58 | {
59 | "cell_type": "code",
60 | "collapsed": false,
61 | "input": [
62 | "from shapely.geometry import LineString\n",
63 | "\n",
64 | "lons = [-75.1, -75.7, -76.2, -76.5, -76.9, -77.7, -78.4, -79.0,\n",
65 | " -79.6, -80.1, -80.3, -81.3, -82.0, -82.6, -83.3, -84.0,\n",
66 | " -84.7, -85.3, -85.9, -86.7, -87.7, -88.6, -89.2, -89.6,\n",
67 | " -89.6, -89.6, -89.6, -89.6, -89.1, -88.6, -88.0, -87.0,\n",
68 | " -85.3, -82.9]\n",
69 | "lats = [23.1, 23.4, 23.8, 24.5, 25.4, 26.0, 26.1, 26.2, 26.2, 26.0,\n",
70 | " 25.9, 25.4, 25.1, 24.9, 24.6, 24.4, 24.4, 24.5, 24.8, 25.2,\n",
71 | " 25.7, 26.3, 27.2, 28.2, 29.3, 29.5, 30.2, 31.1, 32.6, 34.1,\n",
72 | " 35.6, 37.0, 38.6, 40.1]\n",
73 | "\n",
74 | "# Turn the lons and lats into a shapely LineString\n",
75 | "katrina_track = LineString(zip(lons, lats))"
76 | ],
77 | "language": "python",
78 | "metadata": {
79 | "slideshow": {
80 | "slide_type": "-"
81 | }
82 | },
83 | "outputs": []
84 | },
85 | {
86 | "cell_type": "markdown",
87 | "metadata": {
88 | "slideshow": {
89 | "slide_type": "-"
90 | }
91 | },
92 | "source": [
93 | "Buffer the linestring by two degrees (doesn't really make sense!). This *should* be about 200kms, but as we'll see, it's not quite accurate... **Why not**?"
94 | ]
95 | },
96 | {
97 | "cell_type": "code",
98 | "collapsed": false,
99 | "input": [
100 | "katrina_buffer = katrina_track.buffer(2)"
101 | ],
102 | "language": "python",
103 | "metadata": {},
104 | "outputs": []
105 | },
106 | {
107 | "cell_type": "markdown",
108 | "metadata": {
109 | "slideshow": {
110 | "slide_type": "subslide"
111 | }
112 | },
113 | "source": [
114 | "What if we reproject the lon/lats to a projection that preserves distances better?\n",
115 | "\n",
116 | "We *could* use `EPSG:32616` (UTM Zone 16), which covers where Katrina meets New Orleans, but we're probably better off using a custom `proj4` string based on a Lambert Conformal Conic projection. **Why**?"
117 | ]
118 | },
119 | {
120 | "cell_type": "code",
121 | "collapsed": false,
122 | "input": [
123 | "from pyproj import Proj, transform\n",
124 | "from fiona.crs import from_string\n",
125 | "\n",
126 | "# Create custom proj4 string\n",
127 | "proj = from_string('+ellps=WGS84 +proj=lcc +lon_0=-96.0 +lat_0=39.0 '\n",
128 | " '+x_0=0.0 +y_0=0.0 +lat_1=33 +lat_2=45 +no_defs')\n",
129 | "\n",
130 | "# Create source and destination Proj objects (source is WGS84 lons/lats)\n",
131 | "src = Proj(init='epsg:4326')\n",
132 | "dst = Proj(proj)\n",
133 | "\n",
134 | "# Create a LineString from the transformed points\n",
135 | "proj_track = LineString(zip(*transform(src, dst, lons, lats)))\n",
136 | "# Buffer the LineString by 200 km (multiply by 1000 to work in meters)\n",
137 | "proj_buffer = proj_track.buffer(200*1000)"
138 | ],
139 | "language": "python",
140 | "metadata": {
141 | "slideshow": {
142 | "slide_type": "fragment"
143 | }
144 | },
145 | "outputs": []
146 | },
147 | {
148 | "cell_type": "markdown",
149 | "metadata": {
150 | "slideshow": {
151 | "slide_type": "subslide"
152 | }
153 | },
154 | "source": [
155 | "## Aside: Coordinate tuples and x, y sequences\n",
156 | "\n",
157 | "`zip` is your friend! Use it with tuple unpacking to change between sequences of `(x, y)` pairs and seperate `x` and `y` sequences."
158 | ]
159 | },
160 | {
161 | "cell_type": "code",
162 | "collapsed": false,
163 | "input": [
164 | "pts = [(0, 0), (1, 0), (1, 1), (2, 1), (2, 2)]\n",
165 | "x, y = zip(*pts)\n",
166 | "print x, y"
167 | ],
168 | "language": "python",
169 | "metadata": {
170 | "slideshow": {
171 | "slide_type": "-"
172 | }
173 | },
174 | "outputs": []
175 | },
176 | {
177 | "cell_type": "markdown",
178 | "metadata": {
179 | "slideshow": {
180 | "slide_type": "-"
181 | }
182 | },
183 | "source": [
184 | "Also, instead of calling `f(x, y)`, you can just use"
185 | ]
186 | },
187 | {
188 | "cell_type": "code",
189 | "collapsed": false,
190 | "input": [
191 | "# Skip this slide, not really needed for demo, just need Python function\n",
192 | "# Simple function to add 0.5 to each coordinate\n",
193 | "def f(x, y):\n",
194 | " new_x = [i + 0.5 for i in x]\n",
195 | " new_y = [j + 0.5 for j in y]\n",
196 | " return new_x, new_y"
197 | ],
198 | "language": "python",
199 | "metadata": {
200 | "slideshow": {
201 | "slide_type": "skip"
202 | }
203 | },
204 | "outputs": []
205 | },
206 | {
207 | "cell_type": "code",
208 | "collapsed": false,
209 | "input": [
210 | "f(*zip(*pts)) # Function f adds 0.5 to each coordinate"
211 | ],
212 | "language": "python",
213 | "metadata": {},
214 | "outputs": []
215 | },
216 | {
217 | "cell_type": "markdown",
218 | "metadata": {
219 | "slideshow": {
220 | "slide_type": "slide"
221 | }
222 | },
223 | "source": [
224 | "## Plotting the path of Hurican Katrina"
225 | ]
226 | },
227 | {
228 | "cell_type": "code",
229 | "collapsed": false,
230 | "input": [
231 | "fig = plt.figure(figsize=(8, 8))\n",
232 | "ax = plt.axes(projection=ccrs.LambertConformal())\n",
233 | "ax.stock_img() # Add background image (slow)\n",
234 | "ax.coastlines(resolution='110m')\n",
235 | "ax.add_geometries([katrina_buffer], ccrs.PlateCarree(), facecolor='blue', alpha=0.5)\n",
236 | "ax.add_geometries([katrina_track], ccrs.PlateCarree(), facecolor='none')\n",
237 | "\n",
238 | "# Let's add the projected buffer for comparison\n",
239 | "ax.add_geometries([proj_buffer], ccrs.LambertConformal(), facecolor='green', alpha=0.5)\n",
240 | "\n",
241 | "ax.set_extent([-125, -66.5, 20, 50], ccrs.PlateCarree())\n",
242 | "ax.gridlines()\n",
243 | "plt.show()"
244 | ],
245 | "language": "python",
246 | "metadata": {
247 | "slideshow": {
248 | "slide_type": "subslide"
249 | }
250 | },
251 | "outputs": []
252 | },
253 | {
254 | "cell_type": "markdown",
255 | "metadata": {},
256 | "source": [
257 | "Which `shapely` geometry method could we use to find where the tracks *differ*?"
258 | ]
259 | },
260 | {
261 | "cell_type": "markdown",
262 | "metadata": {
263 | "slideshow": {
264 | "slide_type": "subslide"
265 | }
266 | },
267 | "source": [
268 | "# Simple Transformation for Georeferencing a Raster\n",
269 | "\n",
270 | "What makes geospatial raster datasets different from other raster files is that their pixels map to regions of the Earth. In this case, we have a raster image which maps to Midtown Manhattan."
271 | ]
272 | },
273 | {
274 | "cell_type": "code",
275 | "collapsed": false,
276 | "input": [
277 | "import matplotlib.image as mpimg\n",
278 | "import os\n",
279 | "\n",
280 | "# Read in a regular PNG image of Manhattan\n",
281 | "png_file = os.path.join('..', 'data', 'manhattan.png')\n",
282 | "img = mpimg.imread(png_file)\n",
283 | "\n",
284 | "# Take a quick look at the shape\n",
285 | "print(img.shape)"
286 | ],
287 | "language": "python",
288 | "metadata": {
289 | "slideshow": {
290 | "slide_type": "-"
291 | }
292 | },
293 | "outputs": []
294 | },
295 | {
296 | "cell_type": "code",
297 | "collapsed": false,
298 | "input": [
299 | "# Specify the affine transformation\n",
300 | "from affine import Affine # You might not have this library installed\n",
301 | "A = Affine(0.999948245999997, 0.0, 583057.357, 0.0, -0.999948245999997, 4516255.36)\n",
302 | "\n",
303 | "# Compute the upper left and lower right corners\n",
304 | "ul = (0, 0)\n",
305 | "lr = img.shape[:2][::-1]\n",
306 | "\n",
307 | "print(\"Upper left: \" + str(A * ul))\n",
308 | "print(\"Lower right: \" + str(A * lr))"
309 | ],
310 | "language": "python",
311 | "metadata": {},
312 | "outputs": []
313 | },
314 | {
315 | "cell_type": "markdown",
316 | "metadata": {},
317 | "source": [
318 | "Here, the coordinate reference system is `EPSG:26918` (NAD83 / UTM Zone 18N), and the affine transformation matrix is given (in later examples, we'll get this information directly from the input raster datasets)."
319 | ]
320 | },
321 | {
322 | "cell_type": "code",
323 | "collapsed": false,
324 | "input": [
325 | "# Upper left and bottom right corners in UTM coords\n",
326 | "left, top = A * ul\n",
327 | "right, bottom = A * lr\n",
328 | "\n",
329 | "# Plot showing original PNG image (axes correspond to rows and cols) on left\n",
330 | "# and 'transformed' PNG (axes correspond to UTM coords) on the right\n",
331 | "f, (ax1, ax2) = plt.subplots(1, 2, figsize=(16, 8))\n",
332 | "ax1.imshow(img)\n",
333 | "ax1.set_title(\"PNG with row, col bounds\")\n",
334 | "ax2.imshow(img, extent=(left, right, bottom, top), aspect=\"equal\")\n",
335 | "ax2.set_title(\"PNG with correct bounds\")\n",
336 | "plt.show()"
337 | ],
338 | "language": "python",
339 | "metadata": {
340 | "slideshow": {
341 | "slide_type": "subslide"
342 | }
343 | },
344 | "outputs": []
345 | },
346 | {
347 | "cell_type": "markdown",
348 | "metadata": {},
349 | "source": [
350 | "### Time to work on Notebook:\n",
351 | "\n",
352 | "[`Plotting Great Circles in Python`](../exercises/Plotting Great Circles in Python.ipynb)"
353 | ]
354 | }
355 | ],
356 | "metadata": {}
357 | }
358 | ]
359 | }
--------------------------------------------------------------------------------
/notebooks/Notebook3.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "metadata": {
3 | "name": "",
4 | "signature": "sha256:34721fb75d1a69f20f368d7384d1c7b3d5b2f5874790b4eb673aed55e0b47177"
5 | },
6 | "nbformat": 3,
7 | "nbformat_minor": 0,
8 | "worksheets": [
9 | {
10 | "cells": [
11 | {
12 | "cell_type": "markdown",
13 | "metadata": {
14 | "slideshow": {
15 | "slide_type": "slide"
16 | }
17 | },
18 | "source": [
19 | "# Geospatial Data in Python: Database, Desktop, and the Web\n",
20 | "## Tutorial (Part 3)"
21 | ]
22 | },
23 | {
24 | "cell_type": "markdown",
25 | "metadata": {
26 | "slideshow": {
27 | "slide_type": "subslide"
28 | }
29 | },
30 | "source": [
31 | "## Convert vector data formats (Shapefile -> GeoJSON)"
32 | ]
33 | },
34 | {
35 | "cell_type": "code",
36 | "collapsed": false,
37 | "input": [
38 | "import fiona\n",
39 | "from fiona.crs import to_string\n",
40 | "import os\n",
41 | "\n",
42 | "boro_file = os.path.join(\"..\", \"data\", \"nybb\", \"nybb.shp\")\n",
43 | "out_file = os.path.join(\"..\", \"data\", \"nybb\", \"nybb.geojson\")\n",
44 | "\n",
45 | "# Register format drivers with a context manager\n",
46 | "with fiona.drivers():\n",
47 | " # Open the shapefile (can also open directly from zip files with vfs!)\n",
48 | " with fiona.open(boro_file) as source:\n",
49 | " print(\"Feature Count: %s\" % len(source))\n",
50 | " print(\"Input Driver: %s\" % source.driver)\n",
51 | " \n",
52 | " meta = source.meta\n",
53 | " meta.update(driver=\"GeoJSON\")\n",
54 | " \n",
55 | " if os.path.exists(out_file):\n",
56 | " os.remove(out_file)\n",
57 | " with fiona.open(out_file, 'w', **meta) as sink:\n",
58 | " print(\"Output Driver: %s\" % sink.driver)\n",
59 | " for rec in source:\n",
60 | " sink.write(rec)\n",
61 | "# Did it work?\n",
62 | "print(\"File Exists: %s\" % os.path.exists(out_file))"
63 | ],
64 | "language": "python",
65 | "metadata": {},
66 | "outputs": []
67 | },
68 | {
69 | "cell_type": "markdown",
70 | "metadata": {
71 | "slideshow": {
72 | "slide_type": "subslide"
73 | }
74 | },
75 | "source": [
76 | "## Read geospatial raster data\n",
77 | "\n",
78 | "Note: You can download the GeoTIFF of Manhattan used in this example [from here](https://www.dropbox.com/s/mba7obrfh2b2ucb/manhattan.tif). Make sure you put it in your `data` folder."
79 | ]
80 | },
81 | {
82 | "cell_type": "code",
83 | "collapsed": false,
84 | "input": [
85 | "import rasterio\n",
86 | "from fiona.crs import to_string\n",
87 | "import os\n",
88 | "import numpy as np\n",
89 | "\n",
90 | "image_file = os.path.join('..', 'data', 'manhattan.tif')\n",
91 | "\n",
92 | "# Register format drivers with a context manager\n",
93 | "with rasterio.drivers():\n",
94 | " with rasterio.open(image_file, 'r') as source:\n",
95 | " print(source.count, source.shape)\n",
96 | " print(source.driver)\n",
97 | " print(to_string(source.crs))\n",
98 | " \n",
99 | " # Get data from each band (newer versions of rasterio use source.read())\n",
100 | " r, g, b = map(source.read_band, (1, 2, 3))\n",
101 | " data = np.dstack((r, g, b)) # Each band is just an ndarray!\n",
102 | " print(type(data))\n",
103 | " \n",
104 | " # Get the bounds of the raster (for plotting later)\n",
105 | " bounds = source.bounds[::2] + source.bounds[1::2]"
106 | ],
107 | "language": "python",
108 | "metadata": {},
109 | "outputs": []
110 | },
111 | {
112 | "cell_type": "markdown",
113 | "metadata": {},
114 | "source": [
115 | "An alternative way (outside of `with` context manager as above):"
116 | ]
117 | },
118 | {
119 | "cell_type": "code",
120 | "collapsed": false,
121 | "input": [
122 | "source = rasterio.open(image_file, 'r')"
123 | ],
124 | "language": "python",
125 | "metadata": {},
126 | "outputs": []
127 | },
128 | {
129 | "cell_type": "markdown",
130 | "metadata": {
131 | "slideshow": {
132 | "slide_type": "subslide"
133 | }
134 | },
135 | "source": [
136 | "## Simple plot of geospatial raster"
137 | ]
138 | },
139 | {
140 | "cell_type": "code",
141 | "collapsed": false,
142 | "input": [
143 | "import matplotlib.pyplot as plt\n",
144 | "%matplotlib inline\n",
145 | "fig = plt.figure(figsize=(8, 8))\n",
146 | "ax = plt.imshow(data, extent=bounds)\n",
147 | "plt.show()"
148 | ],
149 | "language": "python",
150 | "metadata": {},
151 | "outputs": []
152 | },
153 | {
154 | "cell_type": "markdown",
155 | "metadata": {
156 | "slideshow": {
157 | "slide_type": "slide"
158 | }
159 | },
160 | "source": [
161 | "### Time to work on Notebook:\n",
162 | "\n",
163 | "[`Working with Rasters in Python`](../exercises/Working with Rasters in Python.ipynb)"
164 | ]
165 | }
166 | ],
167 | "metadata": {}
168 | }
169 | ]
170 | }
--------------------------------------------------------------------------------
/notebooks/Notebook4.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "metadata": {
3 | "name": "",
4 | "signature": "sha256:86728bd1f81b030b0152580821dbc6554431bd42b99cb9368907a6506fff70f1"
5 | },
6 | "nbformat": 3,
7 | "nbformat_minor": 0,
8 | "worksheets": [
9 | {
10 | "cells": [
11 | {
12 | "cell_type": "markdown",
13 | "metadata": {
14 | "slideshow": {
15 | "slide_type": "slide"
16 | }
17 | },
18 | "source": [
19 | "# Geospatial Data in Python: Database, Desktop, and the Web\n",
20 | "## Tutorial (Part 4)"
21 | ]
22 | },
23 | {
24 | "cell_type": "markdown",
25 | "metadata": {
26 | "slideshow": {
27 | "slide_type": "subslide"
28 | }
29 | },
30 | "source": [
31 | "## Creating a simple GeoSeries"
32 | ]
33 | },
34 | {
35 | "cell_type": "code",
36 | "collapsed": false,
37 | "input": [
38 | "from shapely.geometry import Polygon\n",
39 | "from geopandas import GeoSeries, GeoDataFrame\n",
40 | "\n",
41 | "# Create three simple polygons\n",
42 | "p1 = Polygon([(0, 0), (1, 0), (1, 1)])\n",
43 | "p2 = Polygon([(0, 0), (1, 0), (1, 1), (0, 1)])\n",
44 | "p3 = Polygon([(2, 0), (3, 0), (3, 1), (2, 1)])\n",
45 | "\n",
46 | "s = GeoSeries([p1, p2, p3])\n",
47 | "s"
48 | ],
49 | "language": "python",
50 | "metadata": {},
51 | "outputs": []
52 | },
53 | {
54 | "cell_type": "markdown",
55 | "metadata": {},
56 | "source": [
57 | "Some geographic operations return a normal pandas object. The `area` property of a `GeoSeries` will return a pandas `Series` containing the area of each item in the `GeoSeries`:"
58 | ]
59 | },
60 | {
61 | "cell_type": "code",
62 | "collapsed": false,
63 | "input": [
64 | "print(s.area)"
65 | ],
66 | "language": "python",
67 | "metadata": {},
68 | "outputs": []
69 | },
70 | {
71 | "cell_type": "markdown",
72 | "metadata": {
73 | "slideshow": {
74 | "slide_type": "subslide"
75 | }
76 | },
77 | "source": [
78 | "## Simple file I/O"
79 | ]
80 | },
81 | {
82 | "cell_type": "code",
83 | "collapsed": false,
84 | "input": [
85 | "# Specify file name\n",
86 | "import os\n",
87 | "boro_file = os.path.join(\"..\", \"data\", \"nybb\", \"nybb.shp\")\n",
88 | "\n",
89 | "# Create from file (one line)\n",
90 | "boros = GeoDataFrame.from_file(boro_file)\n",
91 | "\n",
92 | "# Do some pandas stuff\n",
93 | "boros.set_index('BoroCode', inplace=True)\n",
94 | "boros.sort()\n",
95 | "boros"
96 | ],
97 | "language": "python",
98 | "metadata": {},
99 | "outputs": []
100 | },
101 | {
102 | "cell_type": "markdown",
103 | "metadata": {
104 | "slideshow": {
105 | "slide_type": "subslide"
106 | }
107 | },
108 | "source": [
109 | "## Plotting a simple GeoSeries"
110 | ]
111 | },
112 | {
113 | "cell_type": "code",
114 | "collapsed": false,
115 | "input": [
116 | "import matplotlib.pyplot as plt\n",
117 | "%matplotlib inline\n",
118 | "\n",
119 | "f, (ax1, ax2) = plt.subplots(1, 2, figsize=(15, 5), sharey=True)\n",
120 | "s.plot(axes=ax1)\n",
121 | "ax1.set_title(\"Original Polygons\")\n",
122 | "ax1.set_xlim(-0.5, 3.5)\n",
123 | "s.buffer(0.4).plot(axes=ax2)\n",
124 | "ax2.set_title(\"Buffered Polygons\")\n",
125 | "ax2.set_ylim(-0.5, 1.5)\n",
126 | "plt.show()"
127 | ],
128 | "language": "python",
129 | "metadata": {},
130 | "outputs": []
131 | },
132 | {
133 | "cell_type": "markdown",
134 | "metadata": {
135 | "slideshow": {
136 | "slide_type": "subslide"
137 | }
138 | },
139 | "source": [
140 | "## Plotting a GeoDataFrame"
141 | ]
142 | },
143 | {
144 | "cell_type": "code",
145 | "collapsed": false,
146 | "input": [
147 | "fig = plt.figure(figsize=(8, 8))\n",
148 | "ax = boros.plot() # Regular plot\n",
149 | "# We can access and plot the geometries directly as well\n",
150 | "ax = boros.geometry.convex_hull.plot(axes=ax)\n",
151 | "plt.show()"
152 | ],
153 | "language": "python",
154 | "metadata": {},
155 | "outputs": []
156 | },
157 | {
158 | "cell_type": "markdown",
159 | "metadata": {
160 | "slideshow": {
161 | "slide_type": "subslide"
162 | }
163 | },
164 | "source": [
165 | "## GeoPandas makes databases easy\n",
166 | "\n",
167 | "You can access `PostGIS` and other spatially aware databases with a similar `API` to Pandas. The added bonus here is that GeoPandas *understands* geospatial data (such as `WKB`):\n",
168 | "\n",
169 | "It automatically converts `WKB` to the appropriate Shapely `geometry` type.\n",
170 | "\n",
171 | "*Note: If you're following along, you can use the included shapefile instead of a database*"
172 | ]
173 | },
174 | {
175 | "cell_type": "code",
176 | "collapsed": false,
177 | "input": [
178 | "import geopandas as gpd\n",
179 | "\n",
180 | "wifi_file = os.path.join(\"..\", \"data\", \"wifi\", \"wifi.shp\")\n",
181 | "\n",
182 | "df_geo = gpd.read_file(wifi_file)\n",
183 | "df_geo.head()"
184 | ],
185 | "language": "python",
186 | "metadata": {},
187 | "outputs": []
188 | },
189 | {
190 | "cell_type": "markdown",
191 | "metadata": {
192 | "slideshow": {
193 | "slide_type": "subslide"
194 | }
195 | },
196 | "source": [
197 | "## Now that we have a GeoDataFrame...\n",
198 | "\n",
199 | "### We can do all sorts of fun things!\n",
200 | "\n",
201 | "Like compute **summaries** of the geometries:"
202 | ]
203 | },
204 | {
205 | "cell_type": "code",
206 | "collapsed": false,
207 | "input": [
208 | "# What is the total area of all 5 NYC boroughs?\n",
209 | "print(boros.area.sum())"
210 | ],
211 | "language": "python",
212 | "metadata": {},
213 | "outputs": []
214 | },
215 | {
216 | "cell_type": "markdown",
217 | "metadata": {},
218 | "source": [
219 | "...change the **CRS**:"
220 | ]
221 | },
222 | {
223 | "cell_type": "code",
224 | "collapsed": false,
225 | "input": [
226 | "# Convert from NY State Plane to WGS84 (long/lat)\n",
227 | "df_wgs84 = df_geo.to_crs(epsg=4326) # 4326 is the EPSG code for WGS84\n",
228 | "print(df_geo.crs)\n",
229 | "print(df_wgs84.crs)"
230 | ],
231 | "language": "python",
232 | "metadata": {},
233 | "outputs": []
234 | },
235 | {
236 | "cell_type": "markdown",
237 | "metadata": {},
238 | "source": [
239 | "...and **much** more... plus anything else that Pandas can do"
240 | ]
241 | },
242 | {
243 | "cell_type": "markdown",
244 | "metadata": {
245 | "slideshow": {
246 | "slide_type": "subslide"
247 | }
248 | },
249 | "source": [
250 | "## Super quick plotting for the web\n",
251 | "\n",
252 | "Here's a super simple map, which shows `Free` and `Fee-based` WiFi hotspots in NYC:"
253 | ]
254 | },
255 | {
256 | "cell_type": "code",
257 | "collapsed": false,
258 | "input": [
259 | "ax = df_wgs84.plot() # This is all that's needed, but...\n",
260 | "# ...here's a hack to plot things a bit more nicely (GeoPandas is still new!)\n",
261 | "free = df_wgs84[\"type\"] == \"Free\" # Is it free?\n",
262 | "col = {True: \"green\", False: \"blue\"} # If free, green, otherwise blue\n",
263 | "for i, l in enumerate(ax.lines):\n",
264 | " l.set_markersize(6)\n",
265 | " l.set_color(col[free[i]])\n",
266 | "ax.axis('off') # Turn off axes\n",
267 | "plt.show()"
268 | ],
269 | "language": "python",
270 | "metadata": {
271 | "slideshow": {
272 | "slide_type": "-"
273 | }
274 | },
275 | "outputs": []
276 | },
277 | {
278 | "cell_type": "markdown",
279 | "metadata": {
280 | "slideshow": {
281 | "slide_type": "subslide"
282 | }
283 | },
284 | "source": [
285 | "### Let's turn this map into a 'slippy' web-map"
286 | ]
287 | },
288 | {
289 | "cell_type": "code",
290 | "collapsed": false,
291 | "input": [
292 | "import mplleaflet\n",
293 | "mplleaflet.display(fig=ax.figure, crs=df_wgs8.crs)"
294 | ],
295 | "language": "python",
296 | "metadata": {},
297 | "outputs": []
298 | },
299 | {
300 | "cell_type": "markdown",
301 | "metadata": {
302 | "slideshow": {
303 | "slide_type": "subslide"
304 | }
305 | },
306 | "source": [
307 | "#### Want that as a standalone HTML page?"
308 | ]
309 | },
310 | {
311 | "cell_type": "code",
312 | "collapsed": false,
313 | "input": [
314 | "mplleaflet.show(fig=ax.figure, crs=df_wgs84.crs)"
315 | ],
316 | "language": "python",
317 | "metadata": {},
318 | "outputs": []
319 | },
320 | {
321 | "cell_type": "markdown",
322 | "metadata": {
323 | "slideshow": {
324 | "slide_type": "-"
325 | }
326 | },
327 | "source": [
328 | "#### Or perhaps you want to share something with others online? Use `geojson.io`.\n",
329 | "\n",
330 | "Note, your map should really be reprojected to EPSG:4326 (WGS84 lon/lat) for this to align properly."
331 | ]
332 | },
333 | {
334 | "cell_type": "code",
335 | "collapsed": false,
336 | "input": [
337 | "import geojsonio\n",
338 | "res = geojsonio.display(boros.to_crs(epsg=4326).to_json(), force_gist=True)\n",
339 | "print(res)"
340 | ],
341 | "language": "python",
342 | "metadata": {},
343 | "outputs": []
344 | },
345 | {
346 | "cell_type": "markdown",
347 | "metadata": {},
348 | "source": [
349 | "Don't forget to pop on over to geojson.io and play around/edit your GeoJSON layer..."
350 | ]
351 | },
352 | {
353 | "cell_type": "markdown",
354 | "metadata": {
355 | "slideshow": {
356 | "slide_type": "slide"
357 | }
358 | },
359 | "source": [
360 | "### Time to work on Notebook:\n",
361 | "\n",
362 | "[`Working with Vectors in Python`](../exercises/Working with Vectors in Python.ipynb)"
363 | ]
364 | }
365 | ],
366 | "metadata": {}
367 | }
368 | ]
369 | }
--------------------------------------------------------------------------------