├── .gitignore ├── LICENSE ├── Makefile ├── README.md ├── damselfish.png ├── data ├── Corine2012_Uusimaa.cpg ├── Corine2012_Uusimaa.dbf ├── Corine2012_Uusimaa.prj ├── Corine2012_Uusimaa.shp ├── Corine2012_Uusimaa.shp.xml ├── Corine2012_Uusimaa.shx ├── DAMSELFISH_distributions.cpg ├── DAMSELFISH_distributions.dbf ├── DAMSELFISH_distributions.prj ├── DAMSELFISH_distributions.shp ├── DAMSELFISH_distributions.shx ├── Europe_borders.cpg ├── Europe_borders.dbf ├── Europe_borders.prj ├── Europe_borders.sbn ├── Europe_borders.sbx ├── Europe_borders.shp ├── Europe_borders.shx ├── Europe_borders.zip ├── Europe_borders │ ├── Europe_borders.cpg │ ├── Europe_borders.dbf │ ├── Europe_borders.prj │ ├── Europe_borders.sbn │ ├── Europe_borders.sbx │ ├── Europe_borders.shp │ └── Europe_borders.shx ├── Helsinki_borders.cpg ├── Helsinki_borders.dbf ├── Helsinki_borders.prj ├── Helsinki_borders.shp ├── Helsinki_borders.shx ├── Kamppi_Helsinki_Finland_buildings.cpg ├── Kamppi_Helsinki_Finland_buildings.dbf ├── Kamppi_Helsinki_Finland_buildings.prj ├── Kamppi_Helsinki_Finland_buildings.shp ├── Kamppi_Helsinki_Finland_buildings.shx ├── Kamppi_Helsinki_Finland_nodes.cpg ├── Kamppi_Helsinki_Finland_nodes.dbf ├── Kamppi_Helsinki_Finland_nodes.prj ├── Kamppi_Helsinki_Finland_nodes.shp ├── Kamppi_Helsinki_Finland_nodes.shx ├── Kamppi_Helsinki_Finland_route_OD_points.cpg ├── Kamppi_Helsinki_Finland_route_OD_points.dbf ├── Kamppi_Helsinki_Finland_route_OD_points.prj ├── Kamppi_Helsinki_Finland_route_OD_points.shp ├── Kamppi_Helsinki_Finland_route_OD_points.shx ├── Kamppi_Helsinki_Finland_streets.cpg ├── Kamppi_Helsinki_Finland_streets.dbf ├── Kamppi_Helsinki_Finland_streets.prj ├── Kamppi_Helsinki_Finland_streets.shp ├── Kamppi_Helsinki_Finland_streets.shx ├── PKS_suuralue.kml ├── Route_from_a_to_b_at_Kamppi_Helsinki_Finland.cpg ├── Route_from_a_to_b_at_Kamppi_Helsinki_Finland.dbf ├── Route_from_a_to_b_at_Kamppi_Helsinki_Finland.prj ├── Route_from_a_to_b_at_Kamppi_Helsinki_Finland.shp ├── Route_from_a_to_b_at_Kamppi_Helsinki_Finland.shx ├── TravelTimes_to_5975375_RailwayStation.cpg ├── TravelTimes_to_5975375_RailwayStation.dbf ├── TravelTimes_to_5975375_RailwayStation.prj ├── TravelTimes_to_5975375_RailwayStation.shp ├── TravelTimes_to_5975375_RailwayStation.shx ├── TravelTimes_to_5975375_RailwayStation_Helsinki.geojson ├── TravelTimes_to_5975375_RailwayStation_Helsinki.json ├── Vaestotietoruudukko_2015.dbf ├── Vaestotietoruudukko_2015.prj ├── Vaestotietoruudukko_2015.shp ├── Vaestotietoruudukko_2015.shx ├── addresses.cpg ├── addresses.dbf ├── addresses.prj ├── addresses.shp ├── addresses.shx ├── addresses.txt ├── addresses_epsg3879.cpg ├── addresses_epsg3879.dbf ├── addresses_epsg3879.prj ├── addresses_epsg3879.shp ├── addresses_epsg3879.shx ├── addresses_pop15_epsg3979.cpg ├── addresses_pop15_epsg3979.dbf ├── addresses_pop15_epsg3979.prj ├── addresses_pop15_epsg3979.shp ├── addresses_pop15_epsg3979.shx ├── lakes.cpg ├── lakes.dbf ├── lakes.shp ├── lakes.shx ├── metro.dbf ├── metro.prj ├── metro.sbn ├── metro.sbx ├── metro.shp ├── metro.shx ├── roads.dbf ├── roads.prj ├── roads.sbn ├── roads.sbx ├── roads.shp ├── roads.shx └── travel_times_to_ 5976964.txt ├── europe_euqdist.png ├── img └── logo_hy_geo_135.png ├── make.bat ├── population_points.png ├── projected.png ├── source ├── _static │ ├── data │ │ ├── L2 │ │ │ └── Europe_borders.zip │ │ └── L3 │ │ │ ├── PKS_suuralue.kml │ │ │ └── addresses.txt │ └── geo-python.css ├── _templates │ ├── footer.html │ └── layout.html ├── codes │ ├── Arcpy_1_SimplePoly2Raster.py │ ├── Arcpy_2_SpeciesPoly2Raster.py │ ├── Arcpy_3_SpeciesPoly2Raster_Iterate.py │ ├── AutoGIS15_FinalAssignment.ipynb │ ├── AutoGIS15_Lecture7_PythonGIS.ipynb │ ├── Interactive_Kruger_photos_2013-2015_Oct.ipynb │ ├── L2 │ │ └── Preparation_codes_for_using_functions_with_apply.py │ ├── L7 │ │ ├── osmnx_L7_intro.py │ │ └── osmnx_L7_routing.py │ ├── Lesson3-geocoding.ipynb │ ├── Lesson3-point-in-polygon.ipynb │ ├── Lesson3-projections.ipynb │ ├── Lesson3-spatial-index.ipynb │ ├── Lesson3-spatial-join.ipynb │ ├── Lesson4-reclassify.ipynb │ ├── Lesson4_codes.py │ ├── Lesson5-World-3D.ipynb │ ├── Lesson5-interactive-map-Bokeh-advanced-plotting-published.ipynb │ ├── Lesson5-interactive-map-Bokeh-advanced-plotting.html │ ├── Lesson5-interactive-map-Bokeh-advanced-plotting.ipynb │ ├── Lesson5-interactive-map-Bokeh-advanced.ipynb │ ├── Lesson5-interactive-map-Bokeh.ipynb │ ├── Lesson5-interactive-map-folium.ipynb │ ├── Lesson_5_update_bokeh_GeoJSON.py │ ├── Poly2Raster.py │ ├── SpeciesDataManagementArcpy.tbx │ ├── prepare_Helsinki_layer.py │ ├── prepare_TTM_layers.py │ └── species2raster.py ├── conf.py ├── course-info │ ├── Installing_Anacondas_GIS.rst │ ├── License-terms.rst │ └── course-info.rst ├── img │ ├── AutoGIS_Youtube_template.svg │ ├── AutoGIS_Youtube_template_Images │ │ └── AutoGIS_Youtube_template_ImgID1.png │ ├── AutoGIS_banner_2017.png │ ├── Backup_of_GeoPython_banner_X7.cdr │ ├── BoxSelect.png │ ├── BoxZoom.png │ ├── Earth_Eastern_Hemisphere_text2.jpg │ ├── GPLv3_Logo.jpg │ ├── GPLv3_Logo.svg │ ├── GeoPython_banner_X7.cdr │ ├── Github-pages-activated.PNG │ ├── Github-pages-repo-root.PNG │ ├── Github-pages-settings.PNG │ ├── Inspector.png │ ├── LassoSelect.png │ ├── Pan.png │ ├── Reset.png │ ├── Save.png │ ├── Some_Global_text.png │ ├── TapSelect.png │ ├── Texas_map.PNG │ ├── WheelPan.png │ ├── WheelZoom.png │ ├── Youtube_title_template.png │ ├── ZoomIn.png │ ├── ZoomOut.png │ ├── arcgis-create-toolbox.png │ ├── arcgis-idle-location.png │ ├── arcgis-idle-new.png │ ├── arcgis-idle.PNG │ ├── arcgis-my-script.PNG │ ├── arcgis-parameter-tab-lower.png │ ├── arcgis-parameter-tab-upper.png │ ├── arcgis-parameter-tab-whole-clean.png │ ├── arcgis-parameter-tab-whole.png │ ├── arcgis-properties.PNG │ ├── arcgis-python-console.PNG │ ├── arcgis-raster-result.png │ ├── arcgis-script-description.png │ ├── arcgis-script-tool.png │ ├── arcgis-source.png │ ├── arcpy-logo.png │ ├── conda_shapely_windows.PNG │ ├── damselfish-simple-map.PNG │ ├── docs-contents.PNG │ ├── github-pages-docs-folder.PNG │ ├── github-settings.PNG │ ├── google_query_conda.PNG │ ├── helsinki-population-grid.PNG │ ├── index-md.PNG │ ├── index_HTML.PNG │ ├── install_shapely.PNG │ ├── logo_hy_geo_135.png │ └── texas_unemployment.html ├── index.rst └── lessons │ ├── FA │ ├── fa-hints.rst │ └── final-assignment.rst │ ├── L1 │ ├── Geometric-Objects.rst │ ├── Intro-Python-GIS.rst │ ├── ex-1.rst │ ├── exercise-1-hints.rst │ ├── img │ │ ├── Youtube_title_L1.png │ │ ├── Youtube_title_L1_1.png │ │ └── Youtube_title_L1_2.png │ ├── lecture.rst │ └── overview.rst │ ├── L2 │ ├── ex-2.rst │ ├── exercise-2-hints.rst │ ├── geopandas-basics.rst │ ├── img │ │ ├── Youtube_title_L2_1.png │ │ ├── Youtube_title_L2_1a.png │ │ ├── Youtube_title_L2_1b.png │ │ └── Youtube_title_L2_2.png │ ├── lecture.rst │ ├── overview.rst │ ├── planning.rst │ └── projections.rst │ ├── L3 │ ├── ex-3.rst │ ├── exercise-3-hints.rst │ ├── geocoding.rst │ ├── img │ │ ├── Youtube_title_L3_1.png │ │ └── Youtube_title_L3_2.png │ ├── lecture.rst │ ├── nearest-neighbour.rst │ ├── overview.rst │ ├── point-in-polygon.rst │ └── spatial-join.rst │ ├── L4 │ ├── ex-4.rst │ ├── exercise-4-hints.rst │ ├── geometric-operations.rst │ ├── img │ │ ├── Youtube_title_L4_1.png │ │ └── Youtube_title_L4_2.png │ ├── lecture.rst │ ├── overview.rst │ └── reclassify.rst │ ├── L5 │ ├── World-3D.rst │ ├── advanced-bokeh.rst │ ├── download-data.rst │ ├── ex-5.rst │ ├── exercise-5-hints.rst │ ├── img │ │ ├── BoxSelect.png │ │ ├── BoxZoom.png │ │ ├── Github-pages-activated.PNG │ │ ├── Github-pages-repo-root.PNG │ │ ├── Github-pages-settings.PNG │ │ ├── Inspector.png │ │ ├── LassoSelect.png │ │ ├── Pan.png │ │ ├── Reset.png │ │ ├── Save.png │ │ ├── TapSelect.png │ │ ├── WheelPan.png │ │ ├── WheelZoom.png │ │ ├── Youtube_title_L5_1.png │ │ ├── Youtube_title_L5_2.png │ │ ├── ZoomIn.png │ │ ├── ZoomOut.png │ │ ├── docs-contents.PNG │ │ ├── github-pages-docs-folder.PNG │ │ ├── github-settings.PNG │ │ ├── index-md.PNG │ │ └── index_HTML.PNG │ ├── interactive-map-bokeh.rst │ ├── interactive-map-folium.rst │ ├── lecture.rst │ ├── overview.rst │ ├── share-on-github.rst │ └── static-maps.rst │ ├── L6 │ ├── 01-overview.rst │ ├── 02-pyqgis.rst │ ├── 03-processing-toolbox.rst │ ├── 04-processing-script.rst │ ├── ex-6.rst │ ├── img │ │ ├── L6-01-overview-01-pyqgis.svg │ │ ├── L6-01-overview-02-damselfish-simplemap.png │ │ ├── L6-02-pyqgis-00-builtin-python-console.png │ │ ├── L6-02-pyqgis-00-manageplugins.png │ │ ├── L6-02-pyqgis-01-install-ipyconsole.png │ │ ├── L6-02-pyqgis-02-ipyconsole.png │ │ ├── L6-03-00-processing-toolbox.png │ │ ├── L6-03-01-buffer.png │ │ ├── L6-03-02-select-by-attribute.png │ │ ├── L6-03-03-graphical-model.png │ │ ├── L6-04-create-new-script.png │ │ ├── L6-04-run-script.png │ │ ├── L6-04-save-script-as.png │ │ ├── Youtube_title_L6_1.png │ │ └── Youtube_title_L6_2.png │ └── lecture.rst │ └── L7 │ ├── ex-7.rst │ ├── exercise-7-hints.rst │ ├── img │ ├── OSM_logo.png │ ├── Youtube_title_L7_1.png │ ├── Youtube_title_L7_2.png │ └── Youtube_title_L7_3.png │ ├── lecture.rst │ ├── network-analysis.rst │ ├── overview.rst │ └── retrieve-osm-data.rst └── wgs84.png /.gitignore: -------------------------------------------------------------------------------- 1 | # Created by .ignore support plugin (hsz.mobi) 2 | .idea 3 | .ipynb_checkpoints 4 | docs 5 | 6 | 7 | 8 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2016 Python for geo-people 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | # Makefile for Sphinx documentation 2 | # 3 | 4 | # You can set these variables from the command line. 5 | SPHINXOPTS = 6 | SPHINXBUILD = sphinx-build 7 | PAPER = 8 | BUILDDIR = docs 9 | 10 | # Internal variables. 11 | PAPEROPT_a4 = -D latex_paper_size=a4 12 | PAPEROPT_letter = -D latex_paper_size=letter 13 | ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) source 14 | # the i18n builder cannot share the environment and doctrees with the others 15 | I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) source 16 | 17 | .PHONY: help 18 | help: 19 | @echo "Please use \`make ' where is one of" 20 | @echo " html to make standalone HTML files" 21 | @echo " dirhtml to make HTML files named index.html in directories" 22 | @echo " singlehtml to make a single large HTML file" 23 | @echo " pickle to make pickle files" 24 | @echo " json to make JSON files" 25 | @echo " htmlhelp to make HTML files and a HTML help project" 26 | @echo " qthelp to make HTML files and a qthelp project" 27 | @echo " applehelp to make an Apple Help Book" 28 | @echo " devhelp to make HTML files and a Devhelp project" 29 | @echo " epub to make an epub" 30 | @echo " epub3 to make an epub3" 31 | @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" 32 | @echo " latexpdf to make LaTeX files and run them through pdflatex" 33 | @echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx" 34 | @echo " text to make text files" 35 | @echo " man to make manual pages" 36 | @echo " texinfo to make Texinfo files" 37 | @echo " info to make Texinfo files and run them through makeinfo" 38 | @echo " gettext to make PO message catalogs" 39 | @echo " changes to make an overview of all changed/added/deprecated items" 40 | @echo " xml to make Docutils-native XML files" 41 | @echo " pseudoxml to make pseudoxml-XML files for display purposes" 42 | @echo " linkcheck to check all external links for integrity" 43 | @echo " doctest to run all doctests embedded in the documentation (if enabled)" 44 | @echo " coverage to run coverage check of the documentation (if enabled)" 45 | @echo " dummy to check syntax errors of document sources" 46 | 47 | .PHONY: clean 48 | clean: 49 | rm -rf $(BUILDDIR)/* 50 | 51 | .PHONY: html 52 | html: 53 | $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html 54 | @echo 55 | @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." 56 | 57 | .PHONY: dirhtml 58 | dirhtml: 59 | $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml 60 | @echo 61 | @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." 62 | 63 | .PHONY: singlehtml 64 | singlehtml: 65 | $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml 66 | @echo 67 | @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." 68 | 69 | .PHONY: pickle 70 | pickle: 71 | $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle 72 | @echo 73 | @echo "Build finished; now you can process the pickle files." 74 | 75 | .PHONY: json 76 | json: 77 | $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json 78 | @echo 79 | @echo "Build finished; now you can process the JSON files." 80 | 81 | .PHONY: htmlhelp 82 | htmlhelp: 83 | $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp 84 | @echo 85 | @echo "Build finished; now you can run HTML Help Workshop with the" \ 86 | ".hhp project file in $(BUILDDIR)/htmlhelp." 87 | 88 | .PHONY: qthelp 89 | qthelp: 90 | $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp 91 | @echo 92 | @echo "Build finished; now you can run "qcollectiongenerator" with the" \ 93 | ".qhcp project file in $(BUILDDIR)/qthelp, like this:" 94 | @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/PyGeoEdu.qhcp" 95 | @echo "To view the help file:" 96 | @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/PyGeoEdu.qhc" 97 | 98 | .PHONY: applehelp 99 | applehelp: 100 | $(SPHINXBUILD) -b applehelp $(ALLSPHINXOPTS) $(BUILDDIR)/applehelp 101 | @echo 102 | @echo "Build finished. The help book is in $(BUILDDIR)/applehelp." 103 | @echo "N.B. You won't be able to view it unless you put it in" \ 104 | "~/Library/Documentation/Help or install it in your application" \ 105 | "bundle." 106 | 107 | .PHONY: devhelp 108 | devhelp: 109 | $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp 110 | @echo 111 | @echo "Build finished." 112 | @echo "To view the help file:" 113 | @echo "# mkdir -p $$HOME/.local/share/devhelp/PyGeoEdu" 114 | @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/PyGeoEdu" 115 | @echo "# devhelp" 116 | 117 | .PHONY: epub 118 | epub: 119 | $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub 120 | @echo 121 | @echo "Build finished. The epub file is in $(BUILDDIR)/epub." 122 | 123 | .PHONY: epub3 124 | epub3: 125 | $(SPHINXBUILD) -b epub3 $(ALLSPHINXOPTS) $(BUILDDIR)/epub3 126 | @echo 127 | @echo "Build finished. The epub3 file is in $(BUILDDIR)/epub3." 128 | 129 | .PHONY: latex 130 | latex: 131 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 132 | @echo 133 | @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." 134 | @echo "Run \`make' in that directory to run these through (pdf)latex" \ 135 | "(use \`make latexpdf' here to do that automatically)." 136 | 137 | .PHONY: latexpdf 138 | latexpdf: 139 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 140 | @echo "Running LaTeX files through pdflatex..." 141 | $(MAKE) -C $(BUILDDIR)/latex all-pdf 142 | @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." 143 | 144 | .PHONY: latexpdfja 145 | latexpdfja: 146 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 147 | @echo "Running LaTeX files through platex and dvipdfmx..." 148 | $(MAKE) -C $(BUILDDIR)/latex all-pdf-ja 149 | @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." 150 | 151 | .PHONY: text 152 | text: 153 | $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text 154 | @echo 155 | @echo "Build finished. The text files are in $(BUILDDIR)/text." 156 | 157 | .PHONY: man 158 | man: 159 | $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man 160 | @echo 161 | @echo "Build finished. The manual pages are in $(BUILDDIR)/man." 162 | 163 | .PHONY: texinfo 164 | texinfo: 165 | $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo 166 | @echo 167 | @echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo." 168 | @echo "Run \`make' in that directory to run these through makeinfo" \ 169 | "(use \`make info' here to do that automatically)." 170 | 171 | .PHONY: info 172 | info: 173 | $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo 174 | @echo "Running Texinfo files through makeinfo..." 175 | make -C $(BUILDDIR)/texinfo info 176 | @echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo." 177 | 178 | .PHONY: gettext 179 | gettext: 180 | $(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale 181 | @echo 182 | @echo "Build finished. The message catalogs are in $(BUILDDIR)/locale." 183 | 184 | .PHONY: changes 185 | changes: 186 | $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes 187 | @echo 188 | @echo "The overview file is in $(BUILDDIR)/changes." 189 | 190 | .PHONY: linkcheck 191 | linkcheck: 192 | $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck 193 | @echo 194 | @echo "Link check complete; look for any errors in the above output " \ 195 | "or in $(BUILDDIR)/linkcheck/output.txt." 196 | 197 | .PHONY: doctest 198 | doctest: 199 | $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest 200 | @echo "Testing of doctests in the sources finished, look at the " \ 201 | "results in $(BUILDDIR)/doctest/output.txt." 202 | 203 | .PHONY: coverage 204 | coverage: 205 | $(SPHINXBUILD) -b coverage $(ALLSPHINXOPTS) $(BUILDDIR)/coverage 206 | @echo "Testing of coverage in the sources finished, look at the " \ 207 | "results in $(BUILDDIR)/coverage/python.txt." 208 | 209 | .PHONY: xml 210 | xml: 211 | $(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml 212 | @echo 213 | @echo "Build finished. The XML files are in $(BUILDDIR)/xml." 214 | 215 | .PHONY: pseudoxml 216 | pseudoxml: 217 | $(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml 218 | @echo 219 | @echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml." 220 | 221 | .PHONY: dummy 222 | dummy: 223 | $(SPHINXBUILD) -b dummy $(ALLSPHINXOPTS) $(BUILDDIR)/dummy 224 | @echo 225 | @echo "Build finished. Dummy builder generates no files." 226 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Automating GIS-processes - Sphinx + GitHub Pages 2 | 3 | Source documents for maintaining the [Automating GIS-processes course pages, year 2017](https://automating-gis-processes.github.io/2017/). 4 | 5 | The docs are written in [Sphinx](http://www.sphinx-doc.org/en/1.4.9/) and all the rst files for the lesson contents are located in the [source](source/) -folder. 6 | Build html pages are located in the [docs](docs/) -folder. 7 | 8 | ## License and terms of usage 9 | 10 | Creative Commons License 11 | 12 | 13 | We hope that the materials provided here would be helpful for others. Thus, we share all the lesson materials openly, and also our source codes and lesson materials are openly available **from these pages**. 14 | 15 | **Our materials and code snippets are licensed** with **Creative Commons Attribution-ShareAlike 4.0 International licence** and **GNU GPLv3 license**. 16 | 17 | **Read more about the license and terms of usage from [here](https://github.com/Automating-GIS-processes/2016/blob/master/source/License-terms.rst)**. 18 | 19 | ## Requirements 20 | 21 | Docs are written using [Sphinx](http://www.sphinx-doc.org/en/1.4.9/) with modified version of the [Read The Docs theme](http://docs.readthedocs.io/en/latest/theme.html). [Google Analytics](https://analytics.google.com/) 22 | is used for tracking the usage of the site. Thus for building these pages with Sphinx you need to install following (we recommend 23 | installing [conda](http://conda.pydata.org/docs/using/pkgs.html#install-a-package) from [Anaconda Python distribution package](https://www.continuum.io/downloads)): 24 | 25 | - Sphinx 26 | 27 | ``` 28 | conda install -c anaconda sphinx=1.5.1 29 | ``` 30 | 31 | - Read The Docs Theme 32 | 33 | ``` 34 | conda install -c anaconda sphinx_rtd_theme=0.1.9 35 | ``` 36 | 37 | - Google Analytics Sphinx plugin (exceptionally install with Pip that comes with Anaconda!) 38 | 39 | ``` 40 | pip install https://pypi.python.org/packages/48/7e/1b383d54276a743ee195f6f97a2a77054fa1f976913923e1e64fe500d975/sphinxcontrib-googleanalytics-0.1.tar.gz#md5=f9da59a753b8a045945c5e35ed1e2481 41 | ``` 42 | 43 | ## Writing .rst files 44 | 45 | Sphinx uses .rst -files ([reStucturedText](https://en.wikipedia.org/wiki/ReStructuredText)). Thus all the documentation needs to be written into .rst files. It is easy, intuitive and quite similar 46 | to write as Markdown but rst makes it possible to include many things that are impossible to do with Markdown (such as including raw html code, embedding videos or interactive visualizations, having nice 47 | colored notes or hints etc.). All the .rst -files should be placed into the [/source](/source) -folder which is the directory where Sphinx tries to find the documentation by default. **Those .rst files are also 48 | the ones that you want to modify if you desire to make changes to the documents**. 49 | 50 | ## Building the pages 51 | 52 | Unlike Markdown pages (such as this page), Sphinx pages need to be build before you can see the final result. This is because Sphinx produces html pages (into [/docs](/docs) -folder) 53 | that can have many nice features such as the navigation bar on the left, efficient search functionality etc. 54 | 55 | Build the pages by navigating to the root of the repository (i.e. to a folder where this README.md -file is located) and executing following command: 56 | 57 | ``` 58 | make html 59 | ``` 60 | 61 | Sphinx will then start building the pages and the final html pages will be located in [/docs](/docs) -folder. This is a custom location (by default the docs would go to /build -folder) that matches how GitHub 62 | wants them so that GitHub Pages works. I have edited the [make.bat](make.bat) for achieving this. 63 | 64 | ### Sphinx actually runs the codes! 65 | 66 | One of the most powerful features that Sphinx has (in my opinion), is that it will actually run all the Python codes that are written under the `.. ipython:: python` code block. This makes 67 | it possible that you can e.g. plot images dynamically to the pages without doing any manual work (adding images with links), see and show the contents of a datafile on the pages without needing to 68 | add them manually (which is how you would do it on Markdown pages). Hence, doing the documentation reminds a bit how you can write documents with [Jupyter Notebooks](https://jupyter.org/) but with a nicer 69 | looking pages. 70 | 71 | ### Data needs to be in the repository 72 | 73 | What this kind of dynamic Python interpreter of Sphinx means though, is that you need to also keep the data that you use in the documentation together with the docs. I keep all the datasets 74 | used for building these pages in the [/data](/data) -folder and then read the files from there in the background (hidden from the user). See an example of how to hide (with `:suppress:` command) 75 | the data-reading-procedure from the user, from [here](https://raw.githubusercontent.com/Automating-GIS-processes/2016/master/source/Lesson3-table-join.rst). 76 | 77 | ## Contact & Developers 78 | 79 | Main developer and maintainer of these materials: **Henrikki Tenkanen** 80 | 81 | Co-developers: **Vuokko Heikinheimo**. 82 | 83 | See contact info from [here](https://github.com/Automating-GIS-processes/2016/blob/master/source/course-info.rst#instructors). 84 | 85 | 86 | -------------------------------------------------------------------------------- /damselfish.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/damselfish.png -------------------------------------------------------------------------------- /data/Corine2012_Uusimaa.cpg: -------------------------------------------------------------------------------- 1 | ISO-8859-1 -------------------------------------------------------------------------------- /data/Corine2012_Uusimaa.dbf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/Corine2012_Uusimaa.dbf -------------------------------------------------------------------------------- /data/Corine2012_Uusimaa.prj: -------------------------------------------------------------------------------- 1 | PROJCS["UTM_Zone_35_Northern_Hemisphere",GEOGCS["GCS_GRS 1980(IUGG, 1980)",DATUM["D_unknown",SPHEROID["GRS80",6378137,298.257222101]],PRIMEM["Greenwich",0],UNIT["Degree",0.017453292519943295]],PROJECTION["Transverse_Mercator"],PARAMETER["latitude_of_origin",0],PARAMETER["central_meridian",27],PARAMETER["scale_factor",0.9996],PARAMETER["false_easting",500000],PARAMETER["false_northing",0],UNIT["Meter",1]] -------------------------------------------------------------------------------- /data/Corine2012_Uusimaa.shp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/Corine2012_Uusimaa.shp -------------------------------------------------------------------------------- /data/Corine2012_Uusimaa.shx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/Corine2012_Uusimaa.shx -------------------------------------------------------------------------------- /data/DAMSELFISH_distributions.cpg: -------------------------------------------------------------------------------- 1 | UTF-8 -------------------------------------------------------------------------------- /data/DAMSELFISH_distributions.dbf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/DAMSELFISH_distributions.dbf -------------------------------------------------------------------------------- /data/DAMSELFISH_distributions.prj: -------------------------------------------------------------------------------- 1 | GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4326"]] -------------------------------------------------------------------------------- /data/DAMSELFISH_distributions.shp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/DAMSELFISH_distributions.shp -------------------------------------------------------------------------------- /data/DAMSELFISH_distributions.shx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/DAMSELFISH_distributions.shx -------------------------------------------------------------------------------- /data/Europe_borders.cpg: -------------------------------------------------------------------------------- 1 | UTF-8 -------------------------------------------------------------------------------- /data/Europe_borders.prj: -------------------------------------------------------------------------------- 1 | GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137.0,298.257223563]],PRIMEM["Greenwich",0.0],UNIT["degree",0.0174532925199433]] -------------------------------------------------------------------------------- /data/Europe_borders.sbn: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/Europe_borders.sbn -------------------------------------------------------------------------------- /data/Europe_borders.sbx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/Europe_borders.sbx -------------------------------------------------------------------------------- /data/Europe_borders.shp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/Europe_borders.shp -------------------------------------------------------------------------------- /data/Europe_borders.shx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/Europe_borders.shx -------------------------------------------------------------------------------- /data/Europe_borders.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/Europe_borders.zip -------------------------------------------------------------------------------- /data/Europe_borders/Europe_borders.cpg: -------------------------------------------------------------------------------- 1 | UTF-8 -------------------------------------------------------------------------------- /data/Europe_borders/Europe_borders.prj: -------------------------------------------------------------------------------- 1 | GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137.0,298.257223563]],PRIMEM["Greenwich",0.0],UNIT["degree",0.0174532925199433]] -------------------------------------------------------------------------------- /data/Europe_borders/Europe_borders.sbn: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/Europe_borders/Europe_borders.sbn -------------------------------------------------------------------------------- /data/Europe_borders/Europe_borders.sbx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/Europe_borders/Europe_borders.sbx -------------------------------------------------------------------------------- /data/Europe_borders/Europe_borders.shp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/Europe_borders/Europe_borders.shp -------------------------------------------------------------------------------- /data/Europe_borders/Europe_borders.shx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/Europe_borders/Europe_borders.shx -------------------------------------------------------------------------------- /data/Helsinki_borders.cpg: -------------------------------------------------------------------------------- 1 | ISO-8859-1 -------------------------------------------------------------------------------- /data/Helsinki_borders.dbf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/Helsinki_borders.dbf -------------------------------------------------------------------------------- /data/Helsinki_borders.prj: -------------------------------------------------------------------------------- 1 | PROJCS["EUREF_FIN_TM35FIN",GEOGCS["GCS_EUREF_FIN",DATUM["D_ETRS_1989",SPHEROID["GRS_1980",6378137.0,298.257222101]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]],PROJECTION["Transverse_Mercator"],PARAMETER["False_Easting",500000.0],PARAMETER["False_Northing",0.0],PARAMETER["Central_Meridian",27.0],PARAMETER["Scale_Factor",0.9996],PARAMETER["Latitude_Of_Origin",0.0],UNIT["Meter",1.0],AUTHORITY["EPSG",3067]] -------------------------------------------------------------------------------- /data/Helsinki_borders.shp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/Helsinki_borders.shp -------------------------------------------------------------------------------- /data/Helsinki_borders.shx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/Helsinki_borders.shx -------------------------------------------------------------------------------- /data/Kamppi_Helsinki_Finland_buildings.cpg: -------------------------------------------------------------------------------- 1 | ISO-8859-1 -------------------------------------------------------------------------------- /data/Kamppi_Helsinki_Finland_buildings.dbf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/Kamppi_Helsinki_Finland_buildings.dbf -------------------------------------------------------------------------------- /data/Kamppi_Helsinki_Finland_buildings.prj: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/Kamppi_Helsinki_Finland_buildings.prj -------------------------------------------------------------------------------- /data/Kamppi_Helsinki_Finland_buildings.shp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/Kamppi_Helsinki_Finland_buildings.shp -------------------------------------------------------------------------------- /data/Kamppi_Helsinki_Finland_buildings.shx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/Kamppi_Helsinki_Finland_buildings.shx -------------------------------------------------------------------------------- /data/Kamppi_Helsinki_Finland_nodes.cpg: -------------------------------------------------------------------------------- 1 | ISO-8859-1 -------------------------------------------------------------------------------- /data/Kamppi_Helsinki_Finland_nodes.dbf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/Kamppi_Helsinki_Finland_nodes.dbf -------------------------------------------------------------------------------- /data/Kamppi_Helsinki_Finland_nodes.prj: -------------------------------------------------------------------------------- 1 | PROJCS["NAD_1983_UTM_Zone_35N",GEOGCS["GCS_North_American_1983",DATUM["D_North_American_1983",SPHEROID["GRS_1980",6378137,298.257222101]],PRIMEM["Greenwich",0],UNIT["Degree",0.017453292519943295]],PROJECTION["Transverse_Mercator"],PARAMETER["latitude_of_origin",0],PARAMETER["central_meridian",27],PARAMETER["scale_factor",0.9996],PARAMETER["false_easting",500000],PARAMETER["false_northing",0],UNIT["Meter",1]] -------------------------------------------------------------------------------- /data/Kamppi_Helsinki_Finland_nodes.shp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/Kamppi_Helsinki_Finland_nodes.shp -------------------------------------------------------------------------------- /data/Kamppi_Helsinki_Finland_nodes.shx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/Kamppi_Helsinki_Finland_nodes.shx -------------------------------------------------------------------------------- /data/Kamppi_Helsinki_Finland_route_OD_points.cpg: -------------------------------------------------------------------------------- 1 | ISO-8859-1 -------------------------------------------------------------------------------- /data/Kamppi_Helsinki_Finland_route_OD_points.dbf: -------------------------------------------------------------------------------- 1 | u 2 | AQtypeCP Origin Target -------------------------------------------------------------------------------- /data/Kamppi_Helsinki_Finland_route_OD_points.prj: -------------------------------------------------------------------------------- 1 | PROJCS["NAD_1983_UTM_Zone_35N",GEOGCS["GCS_North_American_1983",DATUM["D_North_American_1983",SPHEROID["GRS_1980",6378137,298.257222101]],PRIMEM["Greenwich",0],UNIT["Degree",0.017453292519943295]],PROJECTION["Transverse_Mercator"],PARAMETER["latitude_of_origin",0],PARAMETER["central_meridian",27],PARAMETER["scale_factor",0.9996],PARAMETER["false_easting",500000],PARAMETER["false_northing",0],UNIT["Meter",1]] -------------------------------------------------------------------------------- /data/Kamppi_Helsinki_Finland_route_OD_points.shp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/Kamppi_Helsinki_Finland_route_OD_points.shp -------------------------------------------------------------------------------- /data/Kamppi_Helsinki_Finland_route_OD_points.shx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/Kamppi_Helsinki_Finland_route_OD_points.shx -------------------------------------------------------------------------------- /data/Kamppi_Helsinki_Finland_streets.cpg: -------------------------------------------------------------------------------- 1 | ISO-8859-1 -------------------------------------------------------------------------------- /data/Kamppi_Helsinki_Finland_streets.dbf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/Kamppi_Helsinki_Finland_streets.dbf -------------------------------------------------------------------------------- /data/Kamppi_Helsinki_Finland_streets.prj: -------------------------------------------------------------------------------- 1 | PROJCS["NAD_1983_UTM_Zone_35N",GEOGCS["GCS_North_American_1983",DATUM["D_North_American_1983",SPHEROID["GRS_1980",6378137,298.257222101]],PRIMEM["Greenwich",0],UNIT["Degree",0.017453292519943295]],PROJECTION["Transverse_Mercator"],PARAMETER["latitude_of_origin",0],PARAMETER["central_meridian",27],PARAMETER["scale_factor",0.9996],PARAMETER["false_easting",500000],PARAMETER["false_northing",0],UNIT["Meter",1]] -------------------------------------------------------------------------------- /data/Kamppi_Helsinki_Finland_streets.shp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/Kamppi_Helsinki_Finland_streets.shp -------------------------------------------------------------------------------- /data/Kamppi_Helsinki_Finland_streets.shx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/Kamppi_Helsinki_Finland_streets.shx -------------------------------------------------------------------------------- /data/Route_from_a_to_b_at_Kamppi_Helsinki_Finland.cpg: -------------------------------------------------------------------------------- 1 | ISO-8859-1 -------------------------------------------------------------------------------- /data/Route_from_a_to_b_at_Kamppi_Helsinki_Finland.dbf: -------------------------------------------------------------------------------- 1 | u 2 | auosmidsC\length_mN ['1372441183', '1372441170', '60170471', '1377211668', '60170470', '1377211669', '25291564'] 803.411969199612258 -------------------------------------------------------------------------------- /data/Route_from_a_to_b_at_Kamppi_Helsinki_Finland.prj: -------------------------------------------------------------------------------- 1 | PROJCS["NAD_1983_UTM_Zone_35N",GEOGCS["GCS_North_American_1983",DATUM["D_North_American_1983",SPHEROID["GRS_1980",6378137,298.257222101]],PRIMEM["Greenwich",0],UNIT["Degree",0.017453292519943295]],PROJECTION["Transverse_Mercator"],PARAMETER["latitude_of_origin",0],PARAMETER["central_meridian",27],PARAMETER["scale_factor",0.9996],PARAMETER["false_easting",500000],PARAMETER["false_northing",0],UNIT["Meter",1]] -------------------------------------------------------------------------------- /data/Route_from_a_to_b_at_Kamppi_Helsinki_Finland.shp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/Route_from_a_to_b_at_Kamppi_Helsinki_Finland.shp -------------------------------------------------------------------------------- /data/Route_from_a_to_b_at_Kamppi_Helsinki_Finland.shx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/Route_from_a_to_b_at_Kamppi_Helsinki_Finland.shx -------------------------------------------------------------------------------- /data/TravelTimes_to_5975375_RailwayStation.cpg: -------------------------------------------------------------------------------- 1 | ISO-8859-1 -------------------------------------------------------------------------------- /data/TravelTimes_to_5975375_RailwayStation.dbf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/TravelTimes_to_5975375_RailwayStation.dbf -------------------------------------------------------------------------------- /data/TravelTimes_to_5975375_RailwayStation.prj: -------------------------------------------------------------------------------- 1 | PROJCS["EUREF_FIN_TM35FIN",GEOGCS["GCS_EUREF_FIN",DATUM["D_ETRS_1989",SPHEROID["GRS_1980",6378137.0,298.257222101]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]],PROJECTION["Transverse_Mercator"],PARAMETER["False_Easting",500000.0],PARAMETER["False_Northing",0.0],PARAMETER["Central_Meridian",27.0],PARAMETER["Scale_Factor",0.9996],PARAMETER["Latitude_Of_Origin",0.0],UNIT["Meter",1.0],AUTHORITY["EPSG",3067]] -------------------------------------------------------------------------------- /data/TravelTimes_to_5975375_RailwayStation.shp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/TravelTimes_to_5975375_RailwayStation.shp -------------------------------------------------------------------------------- /data/TravelTimes_to_5975375_RailwayStation.shx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/TravelTimes_to_5975375_RailwayStation.shx -------------------------------------------------------------------------------- /data/Vaestotietoruudukko_2015.dbf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/Vaestotietoruudukko_2015.dbf -------------------------------------------------------------------------------- /data/Vaestotietoruudukko_2015.prj: -------------------------------------------------------------------------------- 1 | PROJCS["ETRS89_ETRS_GK25FIN_2010",GEOGCS["GCS_ETRS_1989",DATUM["D_ETRS_1989",SPHEROID["Geodetic_Reference_System_of_1980",6378137,298.2572221008916]],PRIMEM["Greenwich",0],UNIT["Degree",0.017453292519943295]],PROJECTION["Transverse_Mercator"],PARAMETER["latitude_of_origin",0],PARAMETER["central_meridian",25],PARAMETER["scale_factor",1],PARAMETER["false_easting",25500000],PARAMETER["false_northing",0],UNIT["Meter",1]] -------------------------------------------------------------------------------- /data/Vaestotietoruudukko_2015.shp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/Vaestotietoruudukko_2015.shp -------------------------------------------------------------------------------- /data/Vaestotietoruudukko_2015.shx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/Vaestotietoruudukko_2015.shx -------------------------------------------------------------------------------- /data/addresses.cpg: -------------------------------------------------------------------------------- 1 | ISO-8859-1 -------------------------------------------------------------------------------- /data/addresses.dbf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/addresses.dbf -------------------------------------------------------------------------------- /data/addresses.prj: -------------------------------------------------------------------------------- 1 | GEOGCS["GCS_WGS_1984",DATUM["D_WGS_1984",SPHEROID["WGS_1984",6378137,298.257223563]],PRIMEM["Greenwich",0],UNIT["Degree",0.017453292519943295]] -------------------------------------------------------------------------------- /data/addresses.shp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/addresses.shp -------------------------------------------------------------------------------- /data/addresses.shx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/addresses.shx -------------------------------------------------------------------------------- /data/addresses.txt: -------------------------------------------------------------------------------- 1 | id;addr 2 | 1000;Itämerenkatu 14, 00101 Helsinki, Finland 3 | 1001;Kampinkuja 1, 00100 Helsinki, Finland 4 | 1002;Kaivokatu 8, 00101 Helsinki, Finland 5 | 1003;Hermannin rantatie 1, 00580 Helsinki, Finland 6 | 1005;Tyynenmerenkatu 9, 00220 Helsinki, Finland 7 | 1006;Kontulantie 18, 00940 Helsinki, Finland 8 | 1007;Itäväylä 3, 00950 Helsinki, Finland 9 | 1008;Tapulikaupungintie 3, 00750 Helsinki, Finland 10 | 1009;Sompionpolku 2, 00730 Helsinki, Finland 11 | 1010;Atomitie 5, 00370 Helsinki, Finland 12 | 1011;Rautatientori 1, 00100 Helsinki, Finland 13 | 1012;Kuparitie 8, 00440 Helsinki, Finland 14 | 1013;Rumpupolku 8, 00420 Helsinki, Finland 15 | 1014;Mäkitorpantie 1, 00620 Helsinki, Finland 16 | 1015;Malminkaari 15, 00700 Helsinki, Finland 17 | 1016;Kylätie 23, 00320 Helsinki, Finland 18 | 1017;Malminkartanontie 17, 00410 Helsinki, Finland 19 | 1018;Oulunkylän tori 2b, 00640 Helsinki, Finland 20 | 1019;Ratapihantie 6, 00101 Helsinki, Finland 21 | 1020;Pitäjänmäentie 15, 00370 Helsinki, Finland 22 | 1021;Eskolantie 2, 00720 Helsinki, Finland 23 | 1022;Tattariharjuntie, 00700 Helsinki, Finland 24 | 1023;Tallinnanaukio 1, 00930 Helsinki, Finland 25 | 1024;Tyynylaavantie 7, 00980 Helsinki, Finland 26 | 1025;Myllypurontie 5, 00920 Helsinki, Finland 27 | 1026;Mellunmäenraitio 6, 00970 Helsinki, Finland 28 | 1027;Vaasanpolku 2, 00101 Helsinki, Finland 29 | 1028;Hiihtäjäntie 2, 00810 Helsinki, Finland 30 | 1029;Ukko-Pekan porras 2, 00570 Helsinki, Finland 31 | 1030;Siltasaarenkatu 16, 00530 Helsinki, Finland 32 | 1031;Urho Kekkosen katu 1, 00100 Helsinki, Finland 33 | 1032;Ruoholahdenkatu 17, 00101 Helsinki, Finland 34 | 1033;Tyynenmerenkatu 3, 00220 Helsinki, Finland 35 | 1034;Vilhonkatu 4, 00101 Helsinki, Finland 36 | 37 | -------------------------------------------------------------------------------- /data/addresses_epsg3879.cpg: -------------------------------------------------------------------------------- 1 | ISO-8859-1 -------------------------------------------------------------------------------- /data/addresses_epsg3879.dbf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/addresses_epsg3879.dbf -------------------------------------------------------------------------------- /data/addresses_epsg3879.prj: -------------------------------------------------------------------------------- 1 | PROJCS["Transverse_Mercator",GEOGCS["GCS_GRS 1980(IUGG, 1980)",DATUM["D_unknown",SPHEROID["GRS80",6378137,298.257222101]],PRIMEM["Greenwich",0],UNIT["Degree",0.017453292519943295]],PROJECTION["Transverse_Mercator"],PARAMETER["latitude_of_origin",0],PARAMETER["central_meridian",25],PARAMETER["scale_factor",1],PARAMETER["false_easting",25500000],PARAMETER["false_northing",0],UNIT["Meter",1]] -------------------------------------------------------------------------------- /data/addresses_epsg3879.shp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/addresses_epsg3879.shp -------------------------------------------------------------------------------- /data/addresses_epsg3879.shx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/addresses_epsg3879.shx -------------------------------------------------------------------------------- /data/addresses_pop15_epsg3979.cpg: -------------------------------------------------------------------------------- 1 | ISO-8859-1 -------------------------------------------------------------------------------- /data/addresses_pop15_epsg3979.dbf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/addresses_pop15_epsg3979.dbf -------------------------------------------------------------------------------- /data/addresses_pop15_epsg3979.prj: -------------------------------------------------------------------------------- 1 | PROJCS["Transverse_Mercator",GEOGCS["GCS_GRS 1980(IUGG, 1980)",DATUM["D_unknown",SPHEROID["GRS80",6378137,298.257222101]],PRIMEM["Greenwich",0],UNIT["Degree",0.017453292519943295]],PROJECTION["Transverse_Mercator"],PARAMETER["latitude_of_origin",0],PARAMETER["central_meridian",25],PARAMETER["scale_factor",1],PARAMETER["false_easting",25500000],PARAMETER["false_northing",0],UNIT["Meter",1]] -------------------------------------------------------------------------------- /data/addresses_pop15_epsg3979.shp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/addresses_pop15_epsg3979.shp -------------------------------------------------------------------------------- /data/addresses_pop15_epsg3979.shx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/addresses_pop15_epsg3979.shx -------------------------------------------------------------------------------- /data/lakes.cpg: -------------------------------------------------------------------------------- 1 | ISO-8859-1 -------------------------------------------------------------------------------- /data/lakes.dbf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/lakes.dbf -------------------------------------------------------------------------------- /data/lakes.shp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/lakes.shp -------------------------------------------------------------------------------- /data/lakes.shx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/lakes.shx -------------------------------------------------------------------------------- /data/metro.dbf: -------------------------------------------------------------------------------- 1 | q 2 | aWNUMEROCSUUNTAN 1300M 1 1300M 2 1300M1 1 1300M1 2 1300M2 1 1300M2 2 1300M3 1 1300M3 2 1300V 1 1300V 2 1300V1 1 1300V1 2 1300V2 1 1300V2 2 1300V3 1 1300V3 2 -------------------------------------------------------------------------------- /data/metro.prj: -------------------------------------------------------------------------------- 1 | PROJCS["Finland_Zone_2",GEOGCS["GCS_KKJ",DATUM["D_KKJ",SPHEROID["International_1924",6378388.0,297.0]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]],PROJECTION["Gauss_Kruger"],PARAMETER["False_Easting",2500000.0],PARAMETER["False_Northing",0.0],PARAMETER["Central_Meridian",24.0],PARAMETER["Scale_Factor",1.0],PARAMETER["Latitude_Of_Origin",0.0],UNIT["Meter",1.0]] -------------------------------------------------------------------------------- /data/metro.sbn: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/metro.sbn -------------------------------------------------------------------------------- /data/metro.sbx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/metro.sbx -------------------------------------------------------------------------------- /data/metro.shp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/metro.shp -------------------------------------------------------------------------------- /data/metro.shx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/metro.shx -------------------------------------------------------------------------------- /data/roads.dbf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/roads.dbf -------------------------------------------------------------------------------- /data/roads.prj: -------------------------------------------------------------------------------- 1 | PROJCS["Finland_Zone_2",GEOGCS["GCS_KKJ",DATUM["D_KKJ",SPHEROID["International_1924",6378388.0,297.0]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]],PROJECTION["Gauss_Kruger"],PARAMETER["False_Easting",2500000.0],PARAMETER["False_Northing",0.0],PARAMETER["Central_Meridian",24.0],PARAMETER["Scale_Factor",1.0],PARAMETER["Latitude_Of_Origin",0.0],UNIT["Meter",1.0]] -------------------------------------------------------------------------------- /data/roads.sbn: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/roads.sbn -------------------------------------------------------------------------------- /data/roads.sbx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/roads.sbx -------------------------------------------------------------------------------- /data/roads.shp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/roads.shp -------------------------------------------------------------------------------- /data/roads.shx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/data/roads.shx -------------------------------------------------------------------------------- /europe_euqdist.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/europe_euqdist.png -------------------------------------------------------------------------------- /img/logo_hy_geo_135.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/img/logo_hy_geo_135.png -------------------------------------------------------------------------------- /population_points.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/population_points.png -------------------------------------------------------------------------------- /projected.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/projected.png -------------------------------------------------------------------------------- /source/_static/data/L2/Europe_borders.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/_static/data/L2/Europe_borders.zip -------------------------------------------------------------------------------- /source/_static/data/L3/addresses.txt: -------------------------------------------------------------------------------- 1 | id;addr 2 | 1000;Itämerenkatu 14, 00101 Helsinki, Finland 3 | 1001;Kampinkuja 1, 00100 Helsinki, Finland 4 | 1002;Kaivokatu 8, 00101 Helsinki, Finland 5 | 1003;Hermannin rantatie 1, 00580 Helsinki, Finland 6 | 1005;Tyynenmerenkatu 9, 00220 Helsinki, Finland 7 | 1006;Kontulantie 18, 00940 Helsinki, Finland 8 | 1007;Itäväylä 3, 00950 Helsinki, Finland 9 | 1008;Tapulikaupungintie 3, 00750 Helsinki, Finland 10 | 1009;Sompionpolku 2, 00730 Helsinki, Finland 11 | 1010;Atomitie 5, 00370 Helsinki, Finland 12 | 1011;Rautatientori 1, 00100 Helsinki, Finland 13 | 1012;Kuparitie 8, 00440 Helsinki, Finland 14 | 1013;Rumpupolku 8, 00420 Helsinki, Finland 15 | 1014;Mäkitorpantie 1, 00620 Helsinki, Finland 16 | 1015;Malminkaari 15, 00700 Helsinki, Finland 17 | 1016;Kylätie 23, 00320 Helsinki, Finland 18 | 1017;Malminkartanontie 17, 00410 Helsinki, Finland 19 | 1018;Oulunkylän tori 2b, 00640 Helsinki, Finland 20 | 1019;Ratapihantie 6, 00101 Helsinki, Finland 21 | 1020;Pitäjänmäentie 15, 00370 Helsinki, Finland 22 | 1021;Eskolantie 2, 00720 Helsinki, Finland 23 | 1022;Tattariharjuntie, 00700 Helsinki, Finland 24 | 1023;Tallinnanaukio 1, 00930 Helsinki, Finland 25 | 1024;Tyynylaavantie 7, 00980 Helsinki, Finland 26 | 1025;Myllypurontie 5, 00920 Helsinki, Finland 27 | 1026;Mellunmäenraitio 6, 00970 Helsinki, Finland 28 | 1027;Vaasanpolku 2, 00101 Helsinki, Finland 29 | 1028;Hiihtäjäntie 2, 00810 Helsinki, Finland 30 | 1029;Ukko-Pekan porras 2, 00570 Helsinki, Finland 31 | 1030;Siltasaarenkatu 16, 00530 Helsinki, Finland 32 | 1031;Urho Kekkosen katu 1, 00100 Helsinki, Finland 33 | 1032;Ruoholahdenkatu 17, 00101 Helsinki, Finland 34 | 1033;Tyynenmerenkatu 3, 00220 Helsinki, Finland 35 | 1034;Vilhonkatu 4, 00101 Helsinki, Finland 36 | 37 | -------------------------------------------------------------------------------- /source/_static/geo-python.css: -------------------------------------------------------------------------------- 1 | .wy-nav-content { 2 | max-width: 900px !important; 3 | } 4 | 5 | .wy-side-nav-search{ 6 | background-color: #4682B4; 7 | } 8 | 9 | .wy-nav-top{ 10 | background-color: #4682B4; 11 | } -------------------------------------------------------------------------------- /source/_templates/footer.html: -------------------------------------------------------------------------------- 1 | {% extends "!footer.html" %} 2 | {% block extrafooter %} 3 |


Creative Commons License

4 | {{ super() }} 5 | {% endblock %} -------------------------------------------------------------------------------- /source/_templates/layout.html: -------------------------------------------------------------------------------- 1 | {% extends "!layout.html" %} 2 | 3 | {% block extrahead %} 4 | 5 | {{ super() }} 6 | {% endblock %} 7 | 8 | -------------------------------------------------------------------------------- /source/codes/Arcpy_1_SimplePoly2Raster.py: -------------------------------------------------------------------------------- 1 | # Import arcpy module so we can use ArcGIS geoprocessing tools 2 | import arcpy 3 | 4 | # Enable Arcpy to overwrite existing files 5 | arcpy.env.overwriteOutput = True 6 | 7 | #--------------------------------------------------------------------------------------------- 8 | # 1. Get parameters from the toolbox using 'GetParametersAsText' method 9 | # --> check ArcGIS help for info how to use methods 10 | # Method info: http://resources.arcgis.com/en/help/main/10.2/index.html#//018v00000047000000 11 | #--------------------------------------------------------------------------------------------- 12 | 13 | input_shp = arcpy.GetParameterAsText(0) 14 | output_raster = arcpy.GetParameterAsText(1) 15 | value_attribute = arcpy.GetParameterAsText(2) 16 | 17 | #-------------------------------------------------------------------------------------------- 18 | # 2. Convert input Shapefile into a Raster Dataset using 'PolygonToRaster_conversion' method 19 | # Method info: http://resources.arcgis.com/en/help/main/10.2/index.html#//001200000030000000 20 | #-------------------------------------------------------------------------------------------- 21 | 22 | arcpy.PolygonToRaster_conversion(in_features=input_shp, value_field=value_attribute, out_rasterdataset=output_raster) 23 | -------------------------------------------------------------------------------- /source/codes/Arcpy_2_SpeciesPoly2Raster.py: -------------------------------------------------------------------------------- 1 | # Import arcpy module so we can use ArcGIS geoprocessing tools 2 | import arcpy 3 | 4 | """ This script adds a field into the input shapefile and updates the value 5 | of that field based (range: 1-5) and finally rasterizes the shapefile """ 6 | 7 | # 1. Get parameters from the toolbox using 'GetParametersAsText' method 8 | #---------------------------------------------------------------------- 9 | # --> check ArcGIS help for info how to use methods 10 | # Method info: http://resources.arcgis.com/en/help/main/10.2/index.html#//018v00000047000000 11 | 12 | input_species_shp = arcpy.GetParameterAsText(0) 13 | output_path = arcpy.GetParameterAsText(1) 14 | attribute_name = arcpy.GetParameterAsText(2) 15 | presence_value = arcpy.GetParameterAsText(3) 16 | 17 | 18 | # 2. Add a new field into the input shapefile with 'AddField_management' method 19 | #------------------------------------------------------------------------------ 20 | # Method info: http://resources.arcgis.com/en/help/main/10.2/index.html#//001700000047000000 21 | 22 | arcpy.AddField_management(in_table=input_species_shp, field_name=attribute_name, field_type="SHORT") # Other possible parameters can be left as default 23 | 24 | 25 | # 3. Update the presence value for our newly created attribute with 'CalculateField_management' method 26 | #----------------------------------------------------------------------------------------------------- 27 | # Method info: http://resources.arcgis.com/en/help/main/10.2/index.html#//00170000004m000000 28 | 29 | arcpy.CalculateField_management(in_table=input_species_shp, field=attribute_name, expression=presence_value) 30 | 31 | 32 | # 4. Convert polygon to raster using 'PolygonToRaster_conversion' method 33 | #----------------------------------------------------------------------- 34 | # Method info: http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#//001200000030000000 35 | arcpy.PolygonToRaster_conversion(in_features=input_species_shp, value_field=attribute_name, out_rasterdataset=output_path) 36 | 37 | 38 | # 5. Print info for the user that tool has finished succesfully using 'AddMessage' method 39 | #---------------------------------------------------------------------------------------- 40 | # Method info: http://resources.arcgis.com/en/help/main/10.2/index.html#//018v00000007000000 41 | 42 | my_message = "Tool finished successfully! Rock on!" 43 | arcpy.AddMessage(my_message) 44 | -------------------------------------------------------------------------------- /source/codes/Arcpy_3_SpeciesPoly2Raster_Iterate.py: -------------------------------------------------------------------------------- 1 | # Import arcpy module so we can use ArcGIS geoprocessing tools 2 | import arcpy 3 | import sys, os 4 | 5 | #--------------------------------------------------------------------------------------------- 6 | # 1. Get parameters from the toolbox using 'GetParametersAsText' method 7 | # --> check ArcGIS help for info how to use methods 8 | # Method info: http://resources.arcgis.com/en/help/main/10.2/index.html#//018v00000047000000 9 | #--------------------------------------------------------------------------------------------- 10 | 11 | # Enable Arcpy to overwrite existing files 12 | #arcpy.env.overwriteOutput = True 13 | 14 | input_species_shp = arcpy.GetParameterAsText(0) 15 | output_folder = arcpy.GetParameterAsText(1) 16 | species_attribute = arcpy.GetParameterAsText(2) 17 | attribute_name = arcpy.GetParameterAsText(3) 18 | presence_value = arcpy.GetParameterAsText(4) 19 | 20 | 21 | """ 22 | species_attribute = "binomial" 23 | input_species_shp = r"F:\Opetus\2015_GIS_Prosessiautomatisointi\Data\DAMSELFISH\DAMSELFISH_distributions.shp" 24 | attribute_name = "PresenceV" 25 | presence_value = 2 26 | output_folder = r"F:\Opetus\2015_GIS_Prosessiautomatisointi\Data\DAMSELFISH\Output" 27 | """ 28 | 29 | #--------------------------------------------------------------------------------------------- 30 | # 2. Add a new field into the table using 'AddField_management' method 31 | # Method info: http://resources.arcgis.com/en/help/main/10.2/index.html#//001700000047000000 32 | #--------------------------------------------------------------------------------------------- 33 | 34 | arcpy.AddField_management(in_table=input_species_shp, field_name=attribute_name, field_type="SHORT") 35 | 36 | #----------------------------------------------------------------------------------------------------- 37 | # 3. Update the presence value for our newly created attribute with 'CalculateField_management' method 38 | # Method info: http://resources.arcgis.com/en/help/main/10.2/index.html#//00170000004m000000 39 | #----------------------------------------------------------------------------------------------------- 40 | 41 | arcpy.CalculateField_management(in_table=input_species_shp, field=attribute_name, expression=presence_value) 42 | 43 | #----------------------------------------------------------------------------------------------------------------------------------- 44 | # 4. Get a list of unique species in the table using 'SearchCursor' method 45 | # Method info: http://resources.arcgis.com/en/help/main/10.1/index.html#//018v00000050000000 46 | # More elegant version of the function in ArcPy Cafe: https://arcpy.wordpress.com/2012/02/01/create-a-list-of-unique-field-values/ 47 | # ---------------------------------------------------------------------------------------------------------------------------------- 48 | 49 | # 4.1 CREATE a function that returns unique values of a 'field' within the 'table' 50 | def unique_values(table, field): 51 | 52 | # Create a cursor object for reading the table 53 | cursor = arcpy.da.SearchCursor(table, [field]) # A cursor iterates over rows in table 54 | 55 | # Create an empty list for unique values 56 | unique_values = [] 57 | 58 | # Iterate over rows and append value into the list if it does not exist already 59 | for row in cursor: 60 | if not row[0] in unique_values: # Append only if value does not exist 61 | unique_values.append(row[0]) 62 | return sorted(unique_values) # Return a sorted list of unique values 63 | 64 | # 4.2 USE the function to get a list of unique values 65 | unique_species = unique_values(table=input_species_shp, field=species_attribute) 66 | 67 | #-------------------------------------------------------------------------------------------------------------------------------- 68 | # 5. Create a feature layer from the shapefile with 'MakeFeatureLayer_management' method that enables us to select specific rows 69 | # Method info: http://resources.arcgis.com/en/help/main/10.2/index.html#//00170000006p000000 70 | #-------------------------------------------------------------------------------------------------------------------------------- 71 | species_lyr = arcpy.MakeFeatureLayer_management(in_features=input_species_shp, out_layer="species_lyr") 72 | 73 | #--------------------------------------------------- 74 | # 6. Iterate over unique_species list and: 75 | # 6.1) export individual species as Shapefiles and 76 | # 6.2) convert those shapefiles into Raster Datasets 77 | #--------------------------------------------------- 78 | 79 | for individual in unique_species: 80 | # 6.1): 81 | # Create an expression for selection using Python String manipulation 82 | expression = "%s = '%s'" % (species_attribute, individual) 83 | 84 | # Select rows based on individual breed using 'SelectLayerByAttribute_management' method 85 | # Method info: http://resources.arcgis.com/en/help/main/10.2/index.html#//001700000071000000 86 | selection = arcpy.SelectLayerByAttribute_management(species_lyr, "NEW_SELECTION", where_clause=expression) 87 | 88 | # Create an output path for Shapefile 89 | shape_name = individual + ".shp" 90 | individual_shp = os.path.join(output_folder, shape_name) 91 | 92 | # Export the selection as a Shapefile into the output folder using 'CopyFeatures_management' method 93 | # Method info: http://resources.arcgis.com/en/help/main/10.2/index.html#//001700000035000000 94 | arcpy.CopyFeatures_management(in_features=selection, out_feature_class=individual_shp) 95 | 96 | # 6.2): 97 | # Create an output path for the Raster Dataset (*.tif) 98 | tif_name = individual + ".tif" 99 | individual_tif = os.path.join(output_folder, tif_name) 100 | 101 | # Convert the newly created Shapefile into a Raster Dataset using 'PolygonToRaster_conversion' method 102 | # Method info: http://resources.arcgis.com/en/help/main/10.2/index.html#//001200000030000000 103 | arcpy.PolygonToRaster_conversion(in_features=individual_shp, value_field=attribute_name, out_rasterdataset=individual_tif) 104 | 105 | # Print progress info for the user 106 | info = "Processed: " + individual 107 | arcpy.AddMessage(info) 108 | 109 | # 7. Print that the process was finished successfully 110 | info = "Process was a great success! Wuhuu!" 111 | arcpy.AddMessage(info) 112 | -------------------------------------------------------------------------------- /source/codes/AutoGIS15_FinalAssignment.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Automating GIS-processes - Final work" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "**Aim of the work:**\n", 15 | "\n", 16 | "Aim of the final assignment is to apply the programming techniques and skills that we have learned during the course and create a GIS tool called *AccessHandler* (see below instructions). You can choose yourself what tools / techniques / modules you want to use. You can either do the task by applying pure Python coding, arcpy or even ArcGIS ModelBuilder (not recommended though). \n", 17 | "\n", 18 | "Write your codes into a single Python file and return the code to the Moodle (due date 2.12.2015 at 12:00). In the evaluation of the final work different functionalities of the code are evaluated individually. Thus, if you do not get all different parts / functionalities of the tool working, it is not the *\"the end of the world\"*. It's good if you get at least some parts of the code working. Good documentation of the code will be highly regarded and will affect positively in the grading of the final work. \n", 19 | "\n", 20 | "**What the tool should do?**\n", 21 | "\n", 22 | "***AccessHandler*** is a tool that is used for managing and helping to analyze MetropAccess-Travel Time Matrix (MTTM) data that can be downloaded from here. Read also the description of the dataset from the web-pages so that you get familiar with the data. \n", 23 | "\n", 24 | "AccessHandler has two main functionalities:\n", 25 | "\n", 26 | "1) It finds from the data folder all the matrices that user has specified by assigning a list of integer values that should correspond to YKR-IDs found from the attribute table of [a Shapefile called MetropAccess_YKR_grid.shp](http://www.helsinki.fi/science/accessibility/data/MetropAccess-matka-aikamatriisi/MetropAccess_YKR_grid.zip). AccessHandler will create Shapefiles from the chosen Matrix text tables (e.g. *travel_times\\_to\\_5797076.txt*) by joining the Matrix file with MetropAccess_YKR_grid Shapefile (*from_id* in Matrix file corresponds to *YKR_ID* in the Shapefile) and saves the result in the output-folder that user has defined. You should name the files in a way that it is possible to identify the ID from the name (e.g. 5797076). \n", 27 | "\n", 28 | "2) AccessHandler can also compare *travel times* or *travel distances* between two different travel modes (more than two travel modes are not allowed). Thus IF the user has specified two travel modes (passed in as a list) for the AccessHandler, the tool will calculate the time/distance difference of those travel modes into a new data column that should be created in the Shapefile. The logic of the calculation is following the order of the items passed on the list where first travel mode is always subtracted by the last one: ***travelmode1 - travelmode2***. Notice that there are NoData values present in the data (with integer value -1). In such cases the result should always be integer value -1. The tool should ensure that distances are not compared to travel times and vice versa. If the user chooses to compare travel modes to each other, you should add the travel modes to the filename (e.g. \"Accessibility\\_5797076\\_pt\\_vs\\_car.shp\") If the user has not specified any travel modes, the tool should only create the Shapefile but not execute any calculations. \n", 29 | "\n", 30 | "AccessHandler asks from the user three parameters:\n", 31 | " 1. MatrixID (type: a list of integers)\n", 32 | " 2. TravelModes (type: a list of strings with max length of 2)\n", 33 | " 3. OutputFolder (type: a string containing a folder path)\n", 34 | " \n", 35 | "With ***MatrixID*** parameter the user can pass a list of Travel Time Matrix ID-numbers (should be integers) for the program. If the ID-number that the user has specified does not exist in the data folders, the tools should warn about this to the user but still continue running. The tool should also inform the user about the execution process: tell the user what file is currently under process and how many files there are left (e.g. \"Processing file travel_times\\_to\\_5797076.txt.. Progress: 3/25\"). \n", 36 | "\n", 37 | "With ***TravelModes*** parameter the user can pass a list of travel modes that will be compared to each other (as described earlier). If this parameter is used, the length of the list should be exactly 2, otherwise, stop the program and give advice for the user how the parameter is used (\"Parameter 'TravelModes' takes exactly two items\"). Travel modes should be the same as are found in the actual TravelTimeMatrix file. Thus only five following values are accepted: 'Walk_time', 'Walk_dist', 'PT_total_time', 'PT_time', 'PT_dist'. If the user specifies something else, stop the program, and give advice what are the acceptable values. \n", 38 | "\n", 39 | "With ***OutputFolder*** parameter the user defines the directory where the Shapefiles will be created. \n" 40 | ] 41 | } 42 | ], 43 | "metadata": { 44 | "anaconda-cloud": {}, 45 | "kernelspec": { 46 | "display_name": "Python [default]", 47 | "language": "python", 48 | "name": "python3" 49 | }, 50 | "language_info": { 51 | "codemirror_mode": { 52 | "name": "ipython", 53 | "version": 3 54 | }, 55 | "file_extension": ".py", 56 | "mimetype": "text/x-python", 57 | "name": "python", 58 | "nbconvert_exporter": "python", 59 | "pygments_lexer": "ipython3", 60 | "version": "3.5.2" 61 | } 62 | }, 63 | "nbformat": 4, 64 | "nbformat_minor": 0 65 | } 66 | -------------------------------------------------------------------------------- /source/codes/L2/Preparation_codes_for_using_functions_with_apply.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ 3 | Preparation of AutoGIS Lesson 2. Using functions. 4 | 5 | Created on Mon Nov 6 07:55:16 2017 6 | 7 | @author: Henrikki Tenkanen 8 | """ 9 | import geopandas as gpd 10 | import pandas as pd 11 | from shapely.geometry import Point, LineString 12 | from fiona.crs import from_epsg 13 | 14 | fp = r"C:\HY-DATA\HENTENKA\KOODIT\Opetus\Automating-GIS-processes\2017\data\Europe_borders.shp" 15 | 16 | data = gpd.read_file(fp) 17 | 18 | # Project to metric using World Equidistant Cylindrical where distances are represented correctly from the central longitude and latitude (target point) 19 | # Here we specify our target location to be the coordinates of Helsinki (lon=24.9417 and lat=60.1666) 20 | hki_lon = 24.9417 21 | hki_lat = 60.1666 22 | 23 | # Next we need to specify that we want to center our projection to Helsinki. We need to specify the +lat_0 and +lon_0 parameters in Proj4 string to do this. 24 | equidistant_proj = "+proj=eqc +lat_ts=60 +lat_0={0} +lon_0={1} +x_0=0 +y_0=0 +ellps=WGS84 +datum=WGS84 +units=m +no_defs".format(hki_lat, hki_lon) 25 | data = data.to_crs(equidistant_proj) 26 | 27 | # Let's plot our map and see how it looks like 28 | #data.plot(facecolor='white') 29 | 30 | # Okey, from here we can see that indeed our map is now centered to Helsinki as the 0-position in both x and y is on top of Helsinki. 31 | 32 | # Let's start our analysis by creating a Point object from Helsinki and insert it into a GeoPandas GeoSeries. 33 | hki = gpd.GeoSeries([Point(hki_lon, hki_lat)], crs=from_epsg(4326)) 34 | # Let's convert this point to the same projection as our Europe data is. 35 | hki = hki.to_crs(equidistant_proj) 36 | print(hki) 37 | 38 | # Aha! So the Point coordinates of Helsinki are 0. This confirms us that the center point of our projection is indeed Helsinki. 39 | 40 | # Next we need to calculate the centroids for all the Polygons of the Europen countries. 41 | data['centroid'] = data.centroid 42 | 43 | # Now we can calculate the distances between the centroids and Helsinki 44 | def calculateDistance(row, dest_geom, src_col='geometry', target_col='distance'): 45 | """ 46 | Calculates the distance between a single Shapely Point geometry and a GeoDataFrame with Point geometries. 47 | 48 | Parameters 49 | ---------- 50 | dest_geom : shapely.Point 51 | A single Shapely Point geometry to which the distances will be calculated to. 52 | src_col : str 53 | A name of the column that has the Shapely Point objects from where the distances will be calculated from. 54 | target_col : str 55 | A name of the target column where the result will be stored. 56 | """ 57 | # Calculate the distances 58 | dist = row[src_col].distance(dest_geom) 59 | # Tranform into kilometers 60 | dist_km = dist/1000 61 | # Assign the distance to the original data 62 | row[target_col] = dist_km 63 | return row 64 | 65 | 66 | # Before using our function, we need to get the Shapely point geometry from the re-projected Helsinki center point. We can use the ``get()`` function to retrieve a value from specific index (here index 0). 67 | hki_geom = hki.get(0) 68 | 69 | # Now we have a nice function that we can use with ``apply()`` function. 70 | data = data.apply(calculateDistance, dest_geom=hki_geom, src_col='centroid', target_col='dist_to_Hki', axis=1) 71 | 72 | 73 | 74 | -------------------------------------------------------------------------------- /source/codes/L7/osmnx_L7_intro.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ 3 | L7 Introducing Osmnx for working with OpenStreetMap and routing. 4 | 5 | Created on Tue Dec 5 15:48:04 2017 6 | 7 | @author: Henrikki Tenkanen 8 | """ 9 | 10 | # Working with OpenStreetMap data 11 | # =============================== 12 | 13 | # OpenStreetMap (OSM) is a global crowd-sourced dataset that contains a lot of information about the our built environment. It contains data about streets, buildings, and services to mention a few. 14 | # OSM 15 | 16 | # This week we will explore a new and exciting Python module called `osmnx `__ 17 | # that can be used to retrieve, construct, analyze, and visualize street networks from OpenStreetMap. 18 | # It also offers a simple interface to do network routing based on walking and driving by car. 19 | 20 | # Download and visualize OpenStreetMap data with osmnx 21 | # ---------------------------------------------------- 22 | 23 | # One useful feature that osmnx provides is an easy-to-use way of retrieving `OpenStreetMap `__ data 24 | # (using `OverPass API `__ ). 25 | 26 | # Let's see how we can download and visualize street network data from a district of Kamppi in Helsinki, Finland. 27 | # Osmnx makes it really easy to do that as it allows you to specify an address to retrieve the OpenStreetMap data around that area. 28 | # In fact, osmnx uses the same Nominatim Geocoding API to achieve this which we tested during the Lesson 2. 29 | 30 | # - Let's retrieve OpenStreetMap (OSM) data by specifying ``"Kamppi, Helsinki, Finland"`` as the address where the data should be downloaded. 31 | import osmnx as ox 32 | import matplotlib.pyplot as plt 33 | 34 | place_name = "Kamppi, Helsinki, Finland" 35 | graph = ox.graph_from_place(place_name) 36 | type(graph) 37 | 38 | # Okey, as we can see the data that we retrieved is a special data object called ``networkx.classes.multidigraph.MultiDiGraph``. A DiGraph is a data type that stores nodes and edges with optional data, or attributes. 39 | # What we can see here is that this data type belongs to a Python module called `networkx `__ 40 | # that can be used to create, manipulate, and study the structure, dynamics, and functions of complex networks. 41 | # Networkx module contains algorithms that can be used to calculate `shortest paths `__ 42 | # along networks using e.g. `Dijkstra's `__ or `A* algorithm `__. 43 | 44 | # - Let's see how our street network looks like. It is easy to visualize the graph with osmnx with ``plot_graph()`` function. The function utilizes Matplotlib for visualizing the data, 45 | # hence as a result it returns a matplotlib figure and axis objects. 46 | fig, ax = ox.plot_graph(graph) 47 | 48 | # Great! Now we can see that our graph contains the nodes (blue circles) and the edges (gray lines) that connects those nodes to each other. 49 | 50 | # It is also possible to retrieve other types of OSM data features with osmnx. 51 | 52 | # - Let's download the buildings with ``buildings_from_place()`` function and plot them on top of our street network in Kamppi. Let's also plot the Polygon that represents the area of Kamppi, 53 | # Helsinki that can be retrieved with ``gdf_from_place`` function. 54 | 55 | area = ox.gdf_from_place(place_name) 56 | buildings = ox.buildings_from_place(place_name) 57 | type(area) 58 | type(buildings) 59 | 60 | # As a result we got the data as GeoDataFrames. Hence, we can plot them using the familiar ``plot()`` function of Geopandas. 61 | # As you might remember the street network data was not in GeoDataFrame. Luckily, osmnx provides a convenient function ``graph_to_gdfs()`` 62 | # that can convert the graph into two separate GeoDataFrames where the first one contains the information about the nodes and the second one 63 | # about the edges. 64 | 65 | # - Let's extract the nodes and edges from the graph as GeoDataFrames. 66 | nodes, edges = ox.graph_to_gdfs(graph) 67 | nodes.head() 68 | edges.head() 69 | type(edges) 70 | 71 | # Nice! Now, as we can see, we have our graph as GeoDataFrames and we can plot them using the same functions and tools as we have used before. 72 | # - Let's create a map out of the streets, buildings, and the area Polygon but let's exclude the nodes (to keep the figure clearer). 73 | 74 | fig, ax = plt.subplots() 75 | area.plot(ax=ax, facecolor='black') 76 | edges.plot(ax=ax, linewidth=1, edgecolor='white') 77 | buildings.plot(ax=ax, facecolor='khaki') 78 | 79 | # Cool! Now we have a nice map where we have plotted the buildings, streets and the boundaries of the selected region of 'Kamppi' in Helsinki. Pretty easy, isn't it. 80 | 81 | # .. todo:: 82 | # 83 | # **Task**: Column ``highway`` in our ``edges`` GeoDataFrame contains information about the type of the street (such as ``primacy, cycleway or footway``). 84 | # Select the streets that are walkable or that can be used with cycle and visualize only them with the buildings and the area polygon. Use different colors and line widths for the cycleways and footways. 85 | 86 | 87 | # .. hint:: 88 | # 89 | # There are a few nice and convenient high-level functions in osmnx that can be used to produce nice maps directly just by using a single function that might be useful. 90 | # If you are interested take a look of `this tutorial `__. 91 | # In the lesson we won't cover these, because we wanted to keep as much control to ourselves as possible, hence using lower-level functions. 92 | 93 | 94 | 95 | 96 | -------------------------------------------------------------------------------- /source/codes/Lesson3-spatial-index.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "## Spatial index - improving performance \n", 8 | "\n", 9 | "Doing Point in Polygon with large datasets can be a rather slow operation without optimizing the speed of a look-up query by using [spatial indexing](https://en.wikipedia.org/wiki/Spatial_database#Spatial_index). The most commonly used spatial index for boosting up the performance of PIP is [R-tree](https://en.wikipedia.org/wiki/R-tree). " 10 | ] 11 | } 12 | ], 13 | "metadata": { 14 | "anaconda-cloud": {}, 15 | "kernelspec": { 16 | "display_name": "Python [conda root]", 17 | "language": "python", 18 | "name": "conda-root-py" 19 | }, 20 | "language_info": { 21 | "codemirror_mode": { 22 | "name": "ipython", 23 | "version": 3 24 | }, 25 | "file_extension": ".py", 26 | "mimetype": "text/x-python", 27 | "name": "python", 28 | "nbconvert_exporter": "python", 29 | "pygments_lexer": "ipython3", 30 | "version": "3.5.2" 31 | } 32 | }, 33 | "nbformat": 4, 34 | "nbformat_minor": 1 35 | } 36 | -------------------------------------------------------------------------------- /source/codes/Lesson4-reclassify.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Data reclassification\n", 8 | "\n", 9 | "- Pysal classifiers\n", 10 | "- Create your own custom classifier\n", 11 | "- Classify based on multiple criteria in different columns\n" 12 | ] 13 | } 14 | ], 15 | "metadata": { 16 | "anaconda-cloud": {}, 17 | "kernelspec": { 18 | "display_name": "Python [conda root]", 19 | "language": "python", 20 | "name": "conda-root-py" 21 | }, 22 | "language_info": { 23 | "codemirror_mode": { 24 | "name": "ipython", 25 | "version": 3.0 26 | }, 27 | "file_extension": ".py", 28 | "mimetype": "text/x-python", 29 | "name": "python", 30 | "nbconvert_exporter": "python", 31 | "pygments_lexer": "ipython3", 32 | "version": "3.5.2" 33 | } 34 | }, 35 | "nbformat": 4, 36 | "nbformat_minor": 0 37 | } -------------------------------------------------------------------------------- /source/codes/Lesson_5_update_bokeh_GeoJSON.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ 3 | Created on Sun Dec 3 14:50:49 2017 4 | 5 | @author: Henrikki Tenkanen 6 | """ 7 | 8 | import geopandas as gpd 9 | from bokeh.plotting import save, figure 10 | from bokeh.models import GeoJSONDataSource 11 | 12 | addresses_fp = r'C:\HY-DATA\HENTENKA\KOODIT\Opetus\Automating-GIS-processes\2017\data\addresses.shp' 13 | roads_fp = r'C:\HY-DATA\HENTENKA\KOODIT\Opetus\Automating-GIS-processes\2017\data\roads.shp' 14 | 15 | # Read the data 16 | addresses = gpd.read_file(addresses_fp) 17 | roads = gpd.read_file(roads_fp) 18 | 19 | # Reproject to the same projection 20 | CRS = roads.crs 21 | addresses = addresses.to_crs(crs=CRS) 22 | 23 | # Convert GeoDataFrames into GeoJSONDataSource objects (similar to ColumnDataSource) 24 | point_source = GeoJSONDataSource(geojson=addresses.to_json()) 25 | roads_source = GeoJSONDataSource(geojson=roads.to_json()) 26 | 27 | # Initialize our plot figure 28 | p = figure(title="A test map") 29 | 30 | # Add the lines to the map from our GeoJSONDataSource -object (it is important to specify the columns as 'xs' and 'ys') 31 | p.multi_line('xs', 'ys', source=roads_source, color='gray', line_width=3) 32 | 33 | # Add the lines to the map from our 'msource' ColumnDataSource -object 34 | p.circle('x', 'y', source=point_source, color='black', size=6) 35 | 36 | # Output filepath 37 | outfp = r"C:\HY-DATA\HENTENKA\KOODIT\Opetus\Automating-GIS-processes\Test\Martta_Ex5\test_map.html" 38 | 39 | # Save the map 40 | save(p, outfp) 41 | 42 | -------------------------------------------------------------------------------- /source/codes/Poly2Raster.py: -------------------------------------------------------------------------------- 1 | # Import arcpy module so we can use ArcGIS geoprocessing tools 2 | import arcpy 3 | import sys, os 4 | 5 | ##input_species_shp = arcpy.GetParameterAsText(0) 6 | ##output_folder = arcpy.GetParameterAsText(1) 7 | ##species_attribute = arcpy.GetParameterAsText(2) 8 | ##attribute_name = arcpy.GetParameterAsText(3) 9 | ##presence_value = arcpy.GetParameterAsText(4) 10 | 11 | input_species_shp = r"C:\HY-Data\HENTENKA\KOODIT\Opetus\Automating-GIS-processes\AutoGIS-Sphinx\data\DAMSELFISH_distributions.shp" 12 | output_folder = r"C:\HY-Data\HENTENKA\KOODIT\Opetus\Automating-GIS-processes\test" 13 | species_attribute = "binomial" 14 | attribute_name = "testCol" 15 | presence_value = 1 16 | 17 | 18 | # 2. Add a new field into the table using 'AddField_management' method 19 | arcpy.AddField_management(in_table=input_species_shp, field_name=attribute_name, field_type="SHORT") 20 | 21 | # 3. Update the presence value for our newly created attribute with 'CalculateField_management' method 22 | arcpy.CalculateField_management(in_table=input_species_shp, field=attribute_name, expression=presence_value) 23 | 24 | # 4. Get a list of unique species in the table using 'SearchCursor' method 25 | 26 | # 4.1 CREATE a function that returns unique values of a 'field' within the 'table' 27 | def unique_values(table, field): 28 | 29 | # Create a cursor object for reading the table 30 | cursor = arcpy.da.SearchCursor(table, [field]) # A cursor iterates over rows in table 31 | 32 | # Create an empty list for unique values 33 | unique_values = [] 34 | 35 | # Iterate over rows and append value into the list if it does not exist already 36 | for row in cursor: 37 | if not row[0] in unique_values: # Append only if value does not exist 38 | unique_values.append(row[0]) 39 | return sorted(unique_values) # Return a sorted list of unique values 40 | 41 | # 4.2 USE the function to get a list of unique values 42 | unique_species = unique_values(table=input_species_shp, field=species_attribute) 43 | 44 | 45 | # 5. Create a feature layer from the shapefile with 'MakeFeatureLayer_management' method that enables us to select specific rows 46 | species_lyr = arcpy.MakeFeatureLayer_management(in_features=input_species_shp, out_layer="species_lyr") 47 | 48 | 49 | # 6. Iterate over unique_species list and: 50 | # 6.1) export individual species as Shapefiles and 51 | # 6.2) convert those shapefiles into Raster Datasets 52 | 53 | for individual in unique_species: 54 | print(individual) 55 | # 6.1): 56 | # Create an expression for selection using Python String manipulation 57 | expression = "%s = '%s'" % (species_attribute, individual) 58 | 59 | # Select rows based on individual breed using 'SelectLayerByAttribute_management' method 60 | arcpy.SelectLayerByAttribute_management(species_lyr, "NEW_SELECTION", where_clause=expression) 61 | 62 | # Create an output path for Shapefile 63 | shape_name = individual + ".shp" 64 | individual_shp = os.path.join(output_folder, shape_name) 65 | 66 | # Export the selection as a Shapefile into the output folder using 'CopyFeatures_management' method 67 | arcpy.CopyFeatures_management(in_features=species_lyr, out_feature_class=individual_shp) 68 | 69 | # 6.2): 70 | # Create an output path for the Raster Dataset (*.tif) 71 | tif_name = individual + ".tif" 72 | individual_tif = os.path.join(output_folder, tif_name) 73 | 74 | # Convert the newly created Shapefile into a Raster Dataset using 'PolygonToRaster_conversion' method 75 | arcpy.PolygonToRaster_conversion(in_features=individual_shp, value_field=attribute_name, out_rasterdataset=individual_tif) 76 | 77 | # Print progress info for the user 78 | info = "Processed: " + individual 79 | arcpy.AddMessage(info) 80 | 81 | # 7. Print that the process was finished successfully 82 | info = "Process was a great success! Wuhuu!" 83 | arcpy.AddMessage(info) 84 | -------------------------------------------------------------------------------- /source/codes/SpeciesDataManagementArcpy.tbx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/codes/SpeciesDataManagementArcpy.tbx -------------------------------------------------------------------------------- /source/codes/prepare_Helsinki_layer.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ 3 | Prepare Helsinki municipality borders for the lesson 4 | 5 | Created on Sun Nov 20 12:17:04 2016 6 | 7 | @author: tenkahen 8 | """ 9 | import geopandas as gpd 10 | from fiona.crs import from_epsg 11 | # For this classification we will be using a different dataset that represents the accessibility 12 | # (measured as travel times / distances) in Helsinki Region to city center 13 | fp = r"D:\KOODIT\Opetus\Automating-GIS-processes\AutoGIS-Sphinx\data\paituli_89425282\mml\hallintorajat_10k\2016\SuomenKuntajako_2016_10k.shp" 14 | 15 | # Read the Shapefile 16 | data = gpd.read_file(fp) 17 | 18 | # Select Helsinki 19 | hel = data.ix[data['NAMEFIN'] == 'Helsinki'] 20 | 21 | # Save to disk 22 | outfp = r"D:\KOODIT\Opetus\Automating-GIS-processes\AutoGIS-Sphinx\data\Helsinki_borders.shp" 23 | hel.to_file(outfp) 24 | 25 | -------------------------------------------------------------------------------- /source/codes/prepare_TTM_layers.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ 3 | Prepare travel time matrix layers for the lessons 4 | 5 | Created on Sun Nov 20 12:17:04 2016 6 | 7 | @author: tenkahen 8 | """ 9 | import gdal 10 | import geopandas as gpd 11 | from fiona.crs import from_epsg 12 | import glob 13 | import os 14 | import pandas as pd 15 | 16 | # For this classification we will be using a different dataset that represents the accessibility 17 | # (measured as travel times / distances) in Helsinki Region to city center 18 | folder = r"D:\KOODIT\Opetus\Automating-GIS-processes\AutoGIS-Sphinx\data\Shopping_centers_accessibility_2013_2015" 19 | fps = glob.glob(os.path.join(folder, "*Comparisons*.shp")) 20 | ykr_fp = r"D:\KOODIT\Opetus\Automating-GIS-processes\AutoGIS-Sphinx\data\Shopping_centers_accessibility_2013_2015\YKD_ID_locations.txt" 21 | ykr = pd.read_csv(ykr_fp, sep=';') 22 | 23 | #fp = r"D:\KOODIT\Opetus\Automating-GIS-processes\AutoGIS-Sphinx\data\Shopping_centers_accessibility_2013_2015\TTM_Comparisons_5975375.shp" 24 | 25 | for fp in fps: 26 | # Get Basename 27 | basename = os.path.basename(fp) 28 | 29 | # Parse the YKD_ID number 30 | ykr_id = int(basename.split('_')[-1][:-4]) 31 | 32 | try: 33 | # Get the corresponding location name 34 | loc_name = ykr.loc[ykr['id'] == ykr_id, 'location'].values[0] 35 | except: 36 | loc_name = "NA" 37 | 38 | # Read the accessibility Shapefile 39 | acc = gpd.read_file(fp) 40 | 41 | # Select only data for year 2015 42 | cols = ['car_m_d_y', 'car_m_t_y', 'car_r_d', 'car_r_t', 'from_id', 'geometry', 'pt_m_d_y', 'pt_m_t_y', 43 | 'pt_m_tt_y', 'pt_r_d', 'pt_r_t', 'pt_r_tt', 'to_id_y', 'walk_d_y', 'walk_t_y'] 44 | acc = acc[cols] 45 | 46 | # Rename columns 47 | acc = acc.rename(columns={'car_m_d_y':'car_m_d', 'car_m_t_y': 'car_m_t', 'pt_m_d_y': 'pt_m_d', 'pt_m_t_y': 'pt_m_t', 'pt_m_tt_y': 'pt_m_tt', 'to_id_y': 'to_id', 'walk_d_y': 'walk_d', 'walk_t_y': 'walk_t'}) 48 | 49 | # Reorder them 50 | order = ['from_id', 'to_id', 'walk_t','walk_d', 51 | 'car_r_t', 'car_r_d', 'car_m_t', 'car_m_d', 52 | 'pt_r_t', 'pt_r_tt', 'pt_r_d', 53 | 'pt_m_t', 'pt_m_tt', 'pt_m_d', 54 | 'geometry'] 55 | 56 | acc = acc[order] 57 | 58 | # Change the EPSG to 3067 59 | acc['geometry'] = acc['geometry'].to_crs(epsg=3067) 60 | acc.crs = from_epsg(3067) 61 | 62 | # Save to disk 63 | outfp = r"D:\KOODIT\Opetus\Automating-GIS-processes\AutoGIS-Sphinx\data\dataE4\TravelTimes_to_%s_%s.shp" % (ykr_id, loc_name) 64 | print(outfp) 65 | acc.to_file(outfp) 66 | 67 | # Save as csv file 68 | csvpath = outfp.replace(".shp", ".txt") 69 | 70 | # Drop geometry 71 | acc = acc.drop('geometry', axis=1) 72 | 73 | # Save 74 | acc.to_csv(csvpath, sep=';', index=False) 75 | 76 | 77 | # GDAL EXE files are at C:\Program Files\Anaconda3\Library\bin 78 | 79 | #import os 80 | #import stat 81 | #gdal_data = os.environ['GDAL_DATA'] 82 | # 83 | #gcs_csv = os.path.join(gdal_data, 'gcs.csv') 84 | -------------------------------------------------------------------------------- /source/codes/species2raster.py: -------------------------------------------------------------------------------- 1 | ##Rasterize Species Range Maps=name 2 | ##Conservation Geography=group 3 | ##Species_Range_Polygons=vector polygon 4 | ##Species_Attribute=field Species_Range_Polygons 5 | ##Presence_Field_Name=string presence 6 | ##Presence_Field_Value=expression 1 7 | ##Output_Directory=folder /tmp/ 8 | 9 | 10 | import os.path 11 | 12 | 13 | algorithmOutput = processing.run( 14 | "qgis:fieldcalculator", 15 | { 16 | "INPUT": Species_Range_Polygons, 17 | "FIELD_NAME": Presence_Field_Name, 18 | "FIELD_TYPE": 1, 19 | "FIELD_LENGTH": 5, 20 | "FIELD_PRECISION": 0, 21 | "NEW_FIELD": True, 22 | "FORMULA": Presence_Field_Value, 23 | "OUTPUT": "memory:speciesRangePolygonsWithPresenceValue" 24 | } 25 | ) 26 | speciesRangePolygonsWithPresenceValue = algorithmOutput["OUTPUT"] 27 | 28 | # get the field index for the column "Species_Attribute" 29 | fields = speciesRangePolygonsWithPresenceValue.fields() 30 | fieldIndex = fields.indexFromName(Species_Attribute) 31 | 32 | # get unique values for this columns 33 | uniqueSpecies = Species_Range_Polygons.uniqueValues(fieldIndex) 34 | 35 | # loop over unique species 36 | for species in uniqueSpecies: 37 | 38 | # define output file name: 39 | outputFile = os.path.join( 40 | Output_Directory, 41 | species.replace(" ","_") 42 | ) 43 | 44 | # select only feature with the current species 45 | algorithmOutput = processing.run( 46 | "qgis:selectbyattribute", 47 | { 48 | "INPUT": speciesRangePolygonsWithPresenceValue, 49 | "FIELD": Species_Attribute, 50 | "OPERATOR": 0, 51 | "VALUE": species 52 | } 53 | ) 54 | oneSpeciesRangePolygons = algorithmOutput["OUTPUT"] 55 | 56 | # save intermediate vector file 57 | algorithmOutput = processing.run( 58 | "native:saveselectedfeatures", 59 | { 60 | "INPUT": oneSpeciesRangePolygons, 61 | "OUTPUT": outputFile + ".shp" 62 | } 63 | ) 64 | oneSpeciesRangePolygons = algorithmOutput["OUTPUT"] 65 | 66 | # rasterise the vector layer 67 | algorithmOutput = processing.run( 68 | "gdal:rasterize", 69 | { 70 | "INPUT": oneSpeciesRangePolygons, 71 | "FIELD": Presence_Field_Name, 72 | "DIMENSIONS": 0, 73 | "WIDTH": 2000, 74 | "HEIGHT": 1000, 75 | "RAST_EXT": "", 76 | "RTYPE": 0, 77 | "OUTPUT": outputFile + ".tif" 78 | } 79 | ) 80 | -------------------------------------------------------------------------------- /source/course-info/License-terms.rst: -------------------------------------------------------------------------------- 1 | License and terms of usage 2 | ========================== 3 | 4 | We hope that the materials provided here would be helpful for others. Thus, we share all the lesson materials openly, and also our source codes and lesson materials are openly available `from GitHub `_. 5 | There, you can also find information how to use the materials and modify them to your own purposes. **However**, if you modify the lesson materials we wish that you would also share your materials openly e.g. on GitHub where 6 | everyone could benefit from you work, in a similar manner as we provide these materials for you. We also welcome collaboration and ideas of how to improve the materials on these pages. 7 | Contact us via `email `_ or fork the docs on **GitHub**. 8 | 9 | **Our materials and code snippets are licensed** as explained below: 10 | 11 | Instructional materials 12 | ----------------------- 13 | 14 | .. raw:: html 15 | 16 | Creative Commons License
17 | 18 | All the instructional material is made available under the **Creative Commons Attribution-ShareAlike 4.0 International licence** . See the `full licence `_. 19 | 20 | In short as a human-readable version of the license: 21 | 22 | you are free to 23 | ~~~~~~~~~~~~~~~ 24 | 25 | - **Share** - copy and redistribute the material in any medium or format 26 | - **Adapt** - remix, transform, and build upon the material 27 | 28 | for any purpose, even commercially. The licensor cannot revoke these freedoms as long as you follow the license terms. 29 | 30 | Under the following terms 31 | ~~~~~~~~~~~~~~~~~~~~~~~~~ 32 | 33 | - **Attribution** - You must give appropriate-credit_, provide a link to the license, and indicate_ if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use. 34 | - **ShareAlike** - If you remix, transform, or build upon the material, you must distribute your contributions under the same-license_ as the original. 35 | 36 | **No additional restrictions** — You may not apply legal terms or technological-measures_ that legally restrict others from doing anything the license permits. 37 | 38 | Notices 39 | ~~~~~~~ 40 | 41 | You do not have to comply with the license for elements of the material in the public domain or where your use is permitted by an applicable exception_ or **limitation**. 42 | 43 | No warranties are given. The license may not give you all of the permissions necessary for your intended use. For example, other rights such as publicity_, privacy, 44 | or moral rights may limit how you use the material. 45 | 46 | .. admonition:: Footnotes 47 | 48 | .. _appropriate-credit: 49 | 50 | **Appropriate credit**: If supplied, you must provide the name of the creator and attribution parties, a copyright notice, a license notice, a disclaimer notice, and a link to the material. CC licenses prior to Version 4.0 also require you to provide the title of the material if supplied, and may have other slight differences. 51 | 52 | .. _indicate: 53 | 54 | **Indicate changes**: In 4.0, you must indicate if you modified the material and retain an indication of previous modifications. In 3.0 and earlier license versions, the indication of changes is only required if you create a derivative. 55 | 56 | .. _same-license: 57 | 58 | **Same license**: You may also use a license listed as compatible at `https://creativecommons.org/compatiblelicenses `_. 59 | 60 | .. _technological-measures: 61 | 62 | **Technological measures**: The license prohibits application of effective technological measures, defined with reference to Article 11 of the WIPO Copyright Treaty. 63 | 64 | .. _exception: 65 | 66 | **Exception or limitation**: The rights of users under exceptions and limitations, such as fair use and fair dealing, are not affected by the CC licenses.** 67 | 68 | .. _publicity: 69 | 70 | **Publicity, privacy, or moral rights**: You may need to get additional permissions before using the material as you intend. 71 | 72 | 73 | Code snippets / software 74 | ------------------------ 75 | 76 | Except where otherwise noted, the example programs, code snippets and other software provided by the Automating GIS processes -course are made 77 | available under the **GNU GPLv3 license** (read licence `here `_. 78 | 79 | 80 | -------------------------------------------------------------------------------- /source/course-info/course-info.rst: -------------------------------------------------------------------------------- 1 | General info 2 | ============ 3 | 4 | Course meetings in Period II 5 | ---------------------------- 6 | 7 | - Mondays 12:15-16:00, C122, Exactum (30.10 - 11.12) 8 | - Work sessions on Thursdays 12:15-16:00, A113-114, Physicum (02.11 - 14.12) 9 | 10 | Instructors 11 | ----------- 12 | 13 | **Lecturer**: 14 | 15 | * Henrikki Tenkanen 16 | 17 | * Office: A120, Physicum 18 | * Email: *firstname.lastname@helsinki.fi* 19 | * Phone: +358 50 4484436 20 | 21 | **Course assistants**: 22 | 23 | * Vuokko Heikinheimo 24 | 25 | * Office: A120, Physicum 26 | * Email: *firstname.lastname@helsinki.fi* 27 | * Phone: +358 2941 50760 28 | 29 | * Samuli Massinen 30 | 31 | * Email: *firstname.lastname@helsinki.fi* 32 | 33 | * Antti Kallanranta 34 | 35 | * Email: *firstname.lastname@helsinki.fi* 36 | 37 | Course websites 38 | --------------- 39 | 40 | - Course sites for Period I (Introduction to Python programming): 41 | 42 | - Main course site: ``_ 43 | - Pebbles (cloud computing) site: ``_ 44 | - Slack (Q&A + chat): ``_ 45 | - Polling / voting (active during lectures only): ``_ 46 | 47 | Textbooks 48 | --------- 49 | 50 | There are **no required textbooks** for this course. This course uses a wide range of sources for course information and the main textbooks are given below. 51 | 52 | - Recommended textbooks (in order of relevance): 53 | 54 | - Zelle, J. (2010) `Python Programming: An Introduction to Computer Science `_, Second edition. Franklin, Beedle & Associates. 55 | - Lawhead, J. (2015) `Learning Geospatial Analysis with Python: An effective guide to geographic information systems and remote sensing analysis using Python 3 `_, Second edition. Packt Publishing. 56 | - McKinney, W. (2012) `Python for Data Analysis: Data wrangling with Pandas, NumPy and iPython `_, First edition. O´Reilly Media. 57 | 58 | - Optional textbooks: 59 | 60 | - Westra, E. (2016) `Python Geospatial Development: Develop sophisticated mapping applications from scratch using Python 3 tools for geospatial development `_, Third edition. Packt Publishing. 61 | - Zandbergen, P. (2013) `Python Scripting for ArcGIS `_, Alternate edition. ESRI press. (`Available from the library `_ 62 | - Diener, M. (2015) `Python Geospatial Analysis Cookbook: Over 60 recipes to work with topology, overlays, indoor routing, and web application analysis with Python `_. Packt Publishing. 63 | -------------------------------------------------------------------------------- /source/img/AutoGIS_Youtube_template_Images/AutoGIS_Youtube_template_ImgID1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/AutoGIS_Youtube_template_Images/AutoGIS_Youtube_template_ImgID1.png -------------------------------------------------------------------------------- /source/img/AutoGIS_banner_2017.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/AutoGIS_banner_2017.png -------------------------------------------------------------------------------- /source/img/Backup_of_GeoPython_banner_X7.cdr: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/Backup_of_GeoPython_banner_X7.cdr -------------------------------------------------------------------------------- /source/img/BoxSelect.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/BoxSelect.png -------------------------------------------------------------------------------- /source/img/BoxZoom.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/BoxZoom.png -------------------------------------------------------------------------------- /source/img/Earth_Eastern_Hemisphere_text2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/Earth_Eastern_Hemisphere_text2.jpg -------------------------------------------------------------------------------- /source/img/GPLv3_Logo.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/GPLv3_Logo.jpg -------------------------------------------------------------------------------- /source/img/GeoPython_banner_X7.cdr: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/GeoPython_banner_X7.cdr -------------------------------------------------------------------------------- /source/img/Github-pages-activated.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/Github-pages-activated.PNG -------------------------------------------------------------------------------- /source/img/Github-pages-repo-root.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/Github-pages-repo-root.PNG -------------------------------------------------------------------------------- /source/img/Github-pages-settings.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/Github-pages-settings.PNG -------------------------------------------------------------------------------- /source/img/Inspector.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/Inspector.png -------------------------------------------------------------------------------- /source/img/LassoSelect.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/LassoSelect.png -------------------------------------------------------------------------------- /source/img/Pan.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/Pan.png -------------------------------------------------------------------------------- /source/img/Reset.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/Reset.png -------------------------------------------------------------------------------- /source/img/Save.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/Save.png -------------------------------------------------------------------------------- /source/img/Some_Global_text.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/Some_Global_text.png -------------------------------------------------------------------------------- /source/img/TapSelect.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/TapSelect.png -------------------------------------------------------------------------------- /source/img/Texas_map.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/Texas_map.PNG -------------------------------------------------------------------------------- /source/img/WheelPan.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/WheelPan.png -------------------------------------------------------------------------------- /source/img/WheelZoom.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/WheelZoom.png -------------------------------------------------------------------------------- /source/img/Youtube_title_template.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/Youtube_title_template.png -------------------------------------------------------------------------------- /source/img/ZoomIn.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/ZoomIn.png -------------------------------------------------------------------------------- /source/img/ZoomOut.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/ZoomOut.png -------------------------------------------------------------------------------- /source/img/arcgis-create-toolbox.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/arcgis-create-toolbox.png -------------------------------------------------------------------------------- /source/img/arcgis-idle-location.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/arcgis-idle-location.png -------------------------------------------------------------------------------- /source/img/arcgis-idle-new.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/arcgis-idle-new.png -------------------------------------------------------------------------------- /source/img/arcgis-idle.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/arcgis-idle.PNG -------------------------------------------------------------------------------- /source/img/arcgis-my-script.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/arcgis-my-script.PNG -------------------------------------------------------------------------------- /source/img/arcgis-parameter-tab-lower.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/arcgis-parameter-tab-lower.png -------------------------------------------------------------------------------- /source/img/arcgis-parameter-tab-upper.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/arcgis-parameter-tab-upper.png -------------------------------------------------------------------------------- /source/img/arcgis-parameter-tab-whole-clean.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/arcgis-parameter-tab-whole-clean.png -------------------------------------------------------------------------------- /source/img/arcgis-parameter-tab-whole.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/arcgis-parameter-tab-whole.png -------------------------------------------------------------------------------- /source/img/arcgis-properties.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/arcgis-properties.PNG -------------------------------------------------------------------------------- /source/img/arcgis-python-console.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/arcgis-python-console.PNG -------------------------------------------------------------------------------- /source/img/arcgis-raster-result.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/arcgis-raster-result.png -------------------------------------------------------------------------------- /source/img/arcgis-script-description.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/arcgis-script-description.png -------------------------------------------------------------------------------- /source/img/arcgis-script-tool.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/arcgis-script-tool.png -------------------------------------------------------------------------------- /source/img/arcgis-source.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/arcgis-source.png -------------------------------------------------------------------------------- /source/img/arcpy-logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/arcpy-logo.png -------------------------------------------------------------------------------- /source/img/conda_shapely_windows.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/conda_shapely_windows.PNG -------------------------------------------------------------------------------- /source/img/damselfish-simple-map.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/damselfish-simple-map.PNG -------------------------------------------------------------------------------- /source/img/docs-contents.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/docs-contents.PNG -------------------------------------------------------------------------------- /source/img/github-pages-docs-folder.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/github-pages-docs-folder.PNG -------------------------------------------------------------------------------- /source/img/github-settings.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/github-settings.PNG -------------------------------------------------------------------------------- /source/img/google_query_conda.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/google_query_conda.PNG -------------------------------------------------------------------------------- /source/img/helsinki-population-grid.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/helsinki-population-grid.PNG -------------------------------------------------------------------------------- /source/img/index-md.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/index-md.PNG -------------------------------------------------------------------------------- /source/img/index_HTML.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/index_HTML.PNG -------------------------------------------------------------------------------- /source/img/install_shapely.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/install_shapely.PNG -------------------------------------------------------------------------------- /source/img/logo_hy_geo_135.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/img/logo_hy_geo_135.png -------------------------------------------------------------------------------- /source/index.rst: -------------------------------------------------------------------------------- 1 | 2 | .. figure:: img/AutoGIS_banner_2017.png 3 | 4 | Welcome to Automating GIS-processes 2017! 5 | ========================================= 6 | 7 | **Automating GIS-processes** -course teaches you how to do different GIS-related tasks in Python programming language. Each lesson is a tutorial with specific topic(s) where the aim is to learn 8 | how to solve common GIS-related problems and tasks using Python tools. In the lessons we use only publicly available data which can be used and downloaded by anyone anywhere. In this course, 9 | we assume that you know the basics of Python programming. If Python is not familiar to you, we recommend to start with our earlier course that focuses on learning the basics of Python 10 | from `geo-python.github.io `_. 11 | 12 | Course format 13 | ------------- 14 | 15 | The majority of this course will be spent in front of a computer learning to program in the Python language and working on exercises. 16 | During Teaching Period I, the Automating GIS-processes and `Introduction to Quantitative Geology `_ courses met together and focused on 17 | `learning to program in Python `_. Previously, both these courses lacked sufficient time for students to properly learn the basic concepts of programming in Python. 18 | 19 | The computer exercises will focus on developing basic programming skills using the Python language and applying those skills to various GIS related problems. 20 | Typical exercises will involve a brief introduction followed by topical computer-based tasks. At the end of the exercises, you may be asked to submit answers 21 | to relevant questions, some related plots, and/or Python codes you have written or used. You are encouraged to discuss and work together with other students 22 | on the laboratory exercises, however the laboratory summary write-ups that you submit must be completed individually and must clearly reflect your own work. 23 | 24 | .. admonition:: Earlier versions 25 | 26 | Older course materials are available at: 27 | 28 | - `2016 `__ 29 | 30 | Note, that the contents of the course are updated for each year. Hence, the older course materials may not be used, and they 31 | might be outdated. 32 | 33 | .. admonition:: Open Access! 34 | 35 | The course is **open for everyone**. The aim of this course is to share the knowledge and help people to get started with their journey for doing GIS more efficiently and in a reproducible manner 36 | using Python programming. 37 | 38 | .. admonition:: Step by step instructions with cloud computing! 39 | 40 | The materials are written in a way that you can follow them step by step exactly as they are written, as long as you use the cloud computing resources that 41 | we provide for you (unfortunately **for Uni. Helsinki people** only). Read more about our cloud computing environment from `here `_. 42 | If you work from your own computer, **you need to adjust the file paths to the data** accordingly. 43 | 44 | .. admonition:: For teachers 45 | 46 | If you would like to use these materials for your own teaching or develop them further, we highly support that. 47 | Please read more about how to do it from `here `_. 48 | 49 | 50 | Course topics by week 51 | --------------------- 52 | See earlier materials from Period I `here `_. 53 | 54 | The materials are divided into weekly sections that are published at these pages every Monday morning. Lessons are held with following schedule: 55 | 56 | +----------------+---------------------------------+ 57 | | Time | Theme | 58 | +================+=================================+ 59 | | **Class 1** | GIS in Python; | 60 | | 30.10.2017 | Spatial data model; | 61 | | | Geometric Objects; Shapely | 62 | | | | 63 | +----------------+---------------------------------+ 64 | | **Class 2** | Working with GeoDataFrames; | 65 | | 6.11.2017 | Managing projections; | 66 | | | Table join; | 67 | | | | 68 | +----------------+---------------------------------+ 69 | | **Class 3** | Geocoding and making spatial | 70 | | 13.11.2017 | queries | 71 | | | | 72 | | | | 73 | +----------------+---------------------------------+ 74 | | **Class 4** | Geometric operations; | 75 | | 20.11.2017 | Reclassifying data with Pysal | 76 | | | | 77 | +----------------+---------------------------------+ 78 | | **Class 5** | Visualization, making static | 79 | | 27.11.2017 | and interactive maps | 80 | | | | 81 | +----------------+---------------------------------+ 82 | | **Class 6** | Using Python programming in | 83 | | 4.12.2017 | Quantum GIS | 84 | | | | 85 | +----------------+---------------------------------+ 86 | | **Class 7** | Network analysis and routing | 87 | | 11.12.2017 | in Python (transport modelling) | 88 | | | | 89 | +----------------+---------------------------------+ 90 | 91 | | 92 | 93 | Contents 94 | -------- 95 | *Lesson content, readings and due dates are subject to change* 96 | 97 | .. toctree:: 98 | :maxdepth: 2 99 | :caption: Course information 100 | 101 | course-info/course-info 102 | course-info/Installing_Anacondas_GIS 103 | course-info/License-terms 104 | 105 | .. toctree:: 106 | :maxdepth: 2 107 | :caption: Lesson 1 108 | 109 | lessons/L1/Intro-Python-GIS 110 | lessons/L1/overview 111 | lessons/L1/lecture 112 | lessons/L1/Geometric-Objects 113 | lessons/L1/ex-1 114 | lessons/L1/exercise-1-hints 115 | 116 | .. toctree:: 117 | :maxdepth: 2 118 | :caption: Lesson 2 119 | 120 | lessons/L2/overview 121 | lessons/L2/lecture 122 | lessons/L2/geopandas-basics 123 | lessons/L2/projections 124 | lessons/L2/using-functions 125 | lessons/L2/ex-2 126 | lessons/L2/exercise-2-hints 127 | 128 | .. toctree:: 129 | :maxdepth: 2 130 | :caption: Lesson 3 131 | 132 | lessons/L3/overview 133 | lessons/L3/lecture 134 | lessons/L3/geocoding 135 | lessons/L3/point-in-polygon 136 | lessons/L3/spatial-join 137 | lessons/L3/nearest-neighbour 138 | lessons/L3/ex-3 139 | lessons/L3/exercise-3-hints 140 | 141 | .. toctree:: 142 | :maxdepth: 2 143 | :caption: Lesson 4 144 | 145 | lessons/L4/overview 146 | lessons/L4/lecture 147 | lessons/L4/reclassify 148 | lessons/L4/geometric-operations 149 | lessons/L4/ex-4 150 | lessons/L4/exercise-4-hints 151 | 152 | .. toctree:: 153 | :maxdepth: 2 154 | :caption: Lesson 5 155 | 156 | lessons/L5/overview 157 | lessons/L5/lecture 158 | lessons/L5/static-maps 159 | lessons/L5/interactive-map-bokeh 160 | lessons/L5/advanced-bokeh 161 | lessons/L5/interactive-map-folium 162 | lessons/L5/share-on-github 163 | lessons/L5/ex-5 164 | lessons/L5/exercise-5-hints 165 | 166 | .. toctree:: 167 | :maxdepth: 2 168 | :caption: Lesson 6 169 | 170 | lessons/L6/01-overview 171 | lessons/L6/lecture 172 | lessons/L6/02-pyqgis 173 | lessons/L6/03-processing-toolbox 174 | lessons/L6/04-processing-script 175 | lessons/L6/05-run-the-tool 176 | lessons/L6/ex-6 177 | 178 | .. toctree:: 179 | :maxdepth: 2 180 | :caption: Lesson 7 181 | 182 | lessons/L7/overview 183 | lessons/L7/lecture 184 | lessons/L7/retrieve-osm-data 185 | lessons/L7/network-analysis 186 | lessons/L7/ex-7 187 | lessons/L7/exercise-7-hints 188 | 189 | .. toctree:: 190 | :maxdepth: 2 191 | :caption: Final Assignment 192 | 193 | lessons/FA/final-assignment 194 | lessons/FA/fa-hints 195 | -------------------------------------------------------------------------------- /source/lessons/FA/fa-hints.rst: -------------------------------------------------------------------------------- 1 | Final Assignment hints 2 | ====================== 3 | 4 | Add hints here if needed. -------------------------------------------------------------------------------- /source/lessons/L1/ex-1.rst: -------------------------------------------------------------------------------- 1 | Exercise 1 2 | ========== 3 | 4 | .. admonition:: Start your assignment 5 | 6 | You can start working on your copy of Exercise 1 by `accepting the GitHub Classroom assignment `__. 7 | 8 | **Exercise 1 is due by the start of lecture on 6.11**. 9 | 10 | You can also take a look at the open course copy of `Exercise 1 in the course GitHub repository `__ (does not require logging in). 11 | Note that you should not try to make changes to this copy of the exercise, but rather only to the copy available via GitHub Classroom. 12 | -------------------------------------------------------------------------------- /source/lessons/L1/exercise-1-hints.rst: -------------------------------------------------------------------------------- 1 | Exercise 1 hints 2 | ================ 3 | 4 | Add below some tips for working on Exercise 1 if needed. 5 | 6 | -------------------------------------------------------------------------------- /source/lessons/L1/img/Youtube_title_L1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L1/img/Youtube_title_L1.png -------------------------------------------------------------------------------- /source/lessons/L1/img/Youtube_title_L1_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L1/img/Youtube_title_L1_1.png -------------------------------------------------------------------------------- /source/lessons/L1/img/Youtube_title_L1_2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L1/img/Youtube_title_L1_2.png -------------------------------------------------------------------------------- /source/lessons/L1/lecture.rst: -------------------------------------------------------------------------------- 1 | Lecture 2 | ======= 3 | 4 | .. admonition:: Lesson 1.1 - Introduction to Python GIS; Installing modules with conda 5 | 6 | .. raw:: html 7 | 8 | 9 |

Henrikki Tenkanen, University of Helsinki @ AutoGIS channel on Youtube.

10 | 11 | 12 | .. admonition:: Lesson 1.2 - Geometric objects; Spatial data model; shapely -module 13 | 14 | .. raw:: html 15 | 16 | 17 |

Henrikki Tenkanen, University of Helsinki @ AutoGIS channel on Youtube.

18 | 19 | .. admonition:: Lesson 1.3 - Exercise 1 overview 20 | 21 | Coming .. 22 | 23 | -------------------------------------------------------------------------------- /source/lessons/L1/overview.rst: -------------------------------------------------------------------------------- 1 | Lesson overview 2 | =============== 3 | 4 | In this week's lesson we will introduce you a spatial data model (geometric objects) and some basics of how spatial data 5 | can be manipulated in pure Python. 6 | 7 | 1. :doc:`Geometric Objects - Spatial Data Model ` 8 | 9 | - `Overview of Geometric Objects and Shapely -module `__ 10 | - `Point -object `__ 11 | - `LineString -object `__ 12 | - `Polygon -object `__ 13 | - `Geometry collections (optional) `__ 14 | 15 | 2. :doc:`Exercise 1: Working with geometric objects ` 16 | 3. :doc:`Exercise 1 hints ` 17 | 18 | Learning goals 19 | -------------- 20 | 21 | After this week you should: 22 | 23 | - have an idea of what kind of tools are available for doing GIS in Python 24 | - know how to install Python packages using conda 25 | - know what kind of geometric objects are available 26 | - know how to create different kind of geometries using Shapely 27 | - know how to read coordinates from a file and create Points based on those -------------------------------------------------------------------------------- /source/lessons/L2/ex-2.rst: -------------------------------------------------------------------------------- 1 | Exercise 2 2 | ========== 3 | 4 | .. admonition:: Start your assignment 5 | 6 | You can start working on your copy of Exercise 2 by `accepting the GitHub Classroom assignment `__. 7 | 8 | **Exercise 2 is due by the start of lecture on 13.11**. 9 | 10 | You can also take a look at the open course copy of `Exercise 2 in the course GitHub repository `__ (does not require logging in). 11 | Note that you should not try to make changes to this copy of the exercise, but rather only to the copy available via GitHub Classroom. 12 | -------------------------------------------------------------------------------- /source/lessons/L2/exercise-2-hints.rst: -------------------------------------------------------------------------------- 1 | Exercise 2 hints 2 | ================ 3 | 4 | Converting Pandas DataFrame into a GeoDataFrame 5 | ----------------------------------------------- 6 | 7 | Quite often you are in a situation where you have read data e.g. from text file into a Pandas DataFrame where you have latitude and longitude columns representing the location of a record. 8 | 9 | - Let's continue with the previous example and consider that we have a column where we have stored the shapely geometries: 10 | 11 | .. code:: python 12 | 13 | >>> print(data) 14 | value lat lon geometry 15 | 0 0 2 4 POINT (4 2) 16 | 1 5 1 6 POINT (6 1) 17 | 2 2 6 1 POINT (1 6) 18 | 3 6 6 3 POINT (3 6) 19 | 4 5 5 1 POINT (1 5) 20 | 21 | 22 | - Notice that now our data is still a Pandas **DataFrame**, not a GeoDataFrame: 23 | 24 | .. code:: python 25 | 26 | >>> type(data) 27 | pandas.core.frame.DataFrame 28 | 29 | 30 | - We need to convert the DataFrame into a GeoDataFrame, so that we can e.g. save it into a Shapefile. It is easily done by passing the DataFrame into a GeoDataFrame object. We need to determine 31 | which column contains the geometry information (needs to be always a column called 'geometry'), and optionally we can also determine the coordinate reference system when creating the GeoDataFrame: 32 | 33 | .. code:: python 34 | 35 | # Convert DataFrame into a GeoDataFrame 36 | geo = gpd.GeoDataFrame(data, geometry='geometry', crs=from_epsg(4326)) 37 | 38 | >>> type(geo) 39 | geopandas.geodataframe.GeoDataFrame 40 | 41 | >>> geo.crs 42 | {'init': 'epsg:4326', 'no_defs': True} 43 | 44 | Now we have converted Pandas DataFrame into a proper GeoDataFrame that we can export into a Shapefile for instance. -------------------------------------------------------------------------------- /source/lessons/L2/img/Youtube_title_L2_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L2/img/Youtube_title_L2_1.png -------------------------------------------------------------------------------- /source/lessons/L2/img/Youtube_title_L2_1a.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L2/img/Youtube_title_L2_1a.png -------------------------------------------------------------------------------- /source/lessons/L2/img/Youtube_title_L2_1b.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L2/img/Youtube_title_L2_1b.png -------------------------------------------------------------------------------- /source/lessons/L2/img/Youtube_title_L2_2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L2/img/Youtube_title_L2_2.png -------------------------------------------------------------------------------- /source/lessons/L2/lecture.rst: -------------------------------------------------------------------------------- 1 | Lecture 2 | ======= 3 | 4 | .. admonition:: Lesson 2.1a - Introduction to Geopandas 5 | 6 | .. raw:: html 7 | 8 | 9 |

Henrikki Tenkanen, University of Helsinki @ AutoGIS channel on Youtube.

10 | 11 | 12 | .. admonition:: Lesson 2.1b - Practical example: Export value groups to separate Shapefiles automatically 13 | 14 | .. raw:: html 15 | 16 | 17 |

Henrikki Tenkanen, University of Helsinki @ AutoGIS channel on Youtube.

18 | 19 | 20 | .. admonition:: Lesson 2.2 - Managing map projections 21 | 22 | .. raw:: html 23 | 24 | 25 |

Henrikki Tenkanen, University of Helsinki @ AutoGIS channel on Youtube.

26 | 27 | .. admonition:: Lesson 2.3 - Exercise 2 overview 28 | 29 | Coming .. 30 | 31 | .. admonition:: Lesson 2.4 - Exercise 1 review 32 | 33 | Coming .. 34 | -------------------------------------------------------------------------------- /source/lessons/L2/overview.rst: -------------------------------------------------------------------------------- 1 | Lesson 2 Overview 2 | ================= 3 | 4 | In this week, we will learn how to read and write spatial data, and how to deal with projections in Python using `Geopandas `_ -module 5 | and some useful data management functionalities. 6 | 7 | 1. :doc:`Introduction to Geopandas ` 8 | 2. :doc:`Managing map projections ` 9 | 3. :doc:`Exercise 2: Working with geometric objects ` 10 | 4. :doc:`Exercise 2 hints ` 11 | 12 | Learning goals 13 | -------------- 14 | 15 | After this week, you should know how to: 16 | 17 | - Read and write data from/to Shapefile 18 | - Create geometries into GeoDataFrame 19 | - Change the coordinate reference system of the data 20 | 21 | Sources 22 | ------- 23 | 24 | These materials are partly based on `Geopandas documentation `_. 25 | -------------------------------------------------------------------------------- /source/lessons/L2/planning.rst: -------------------------------------------------------------------------------- 1 | Planning 2 | -------- 3 | 4 | Last year there was only Pandas as Geopandas stuff. Now we have more time to focus on Geopandas. 5 | 6 | **Things to cover in Lesson 2:** 7 | 8 | - Reading / writing Shapefile 9 | - Basic structure / attributes of a GeoDataFrame 10 | - Working with geometries in GeoDataFrame (calculating areas etc.) 11 | - Iterating over rows and creating a LineString from centroid of Polygons to specific destination point (to demonstrate the usefulness of apply function) 12 | - Projections --> Reprojecting data 13 | - We could create a function that returns a GeoDataFrame with identical crs for each input dataset 14 | 15 | **Things to cover in Lesson 3:** 16 | 17 | - Geocoding, using Nominatim (http://nominatim.openstreetmap.org/) --> provider='nominatim' --> does not require API key 18 | - Recap of a table join (with geocoding) 19 | - Point in Polygon / Intersect 20 | - Spatial Join 21 | - Nearest point (scipy.spatial) 22 | 23 | **Things to cover in Lesson 4:** 24 | 25 | - Overlay analysis 26 | - Aggregating data 27 | - Simplifying geometries 28 | - Data reclassification (own made + pysal) 29 | 30 | **Things to cover in Lesson 5:** 31 | 32 | - Visualizing static maps (how to put layers on top of each other) 33 | - Create a subplot with different maps and a graph 34 | - Visualizing interactive maps with Bokeh 35 | - Use map tiles e.g. from OpenStreetMap 36 | - Create functions that handles MultiPolygon and MultiLineStrings (show the problem) 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /source/lessons/L3/ex-3.rst: -------------------------------------------------------------------------------- 1 | Exercise 3 2 | ========== 3 | 4 | .. admonition:: Start your assignment 5 | 6 | You can start working on your copy of Exercise 3 by `accepting the GitHub Classroom assignment `__. 7 | 8 | **Exercise 3 is due by the start of lecture on 20.11**. 9 | 10 | You can also take a look at the open course copy of `Exercise 3 in the course GitHub repository `__ (does not require logging in). 11 | Note that you should not try to make changes to this copy of the exercise, but rather only to the copy available via GitHub Classroom. 12 | -------------------------------------------------------------------------------- /source/lessons/L3/exercise-3-hints.rst: -------------------------------------------------------------------------------- 1 | Exercise 3 hints 2 | ================ 3 | 4 | Add hints for Exercise 3 here if needed. -------------------------------------------------------------------------------- /source/lessons/L3/geocoding.rst: -------------------------------------------------------------------------------- 1 | Geocoding 2 | ========= 3 | 4 | Overview of Geocoders 5 | --------------------- 6 | 7 | Geocoding, i.e. converting addresses into coordinates or vice versa, is 8 | a really common GIS task. Luckily, in Python there are nice libraries 9 | that makes the geocoding really easy. One of the libraries that can do 10 | the geocoding for us is 11 | `geopy `__ that makes it easy to 12 | locate the coordinates of addresses, cities, countries, and landmarks 13 | across the globe using third-party geocoders and other data sources. 14 | 15 | As said, **Geopy** uses third-party geocoders - i.e. services that does 16 | the geocoding - to locate the addresses and it works with multiple 17 | different service providers such as: 18 | 19 | - `ESRI 20 | ArcGIS `__ 21 | - `Baidu 22 | Maps `__ 23 | - `Bing `__ 24 | - `geocoder.us `__ 25 | - `GeocodeFarm `__ 26 | - `GeoNames `__ 27 | - `Google Geocoding API 28 | (V3) `__ 29 | - `IGN 30 | France `__ 31 | - `Mapquest `__ 32 | - `Mapzen Search `__ 33 | - `NaviData `__ 34 | - `OpenCage `__ 35 | - `OpenMapQuest `__ 36 | - `Open Street Map Nominatim `__ 37 | - `SmartyStreets `__ 38 | - `What3words `__ 39 | - `Yandex `__ 40 | 41 | Thus, there are plenty of geocoders where to choose from! However, for most of these services you might need to 42 | request so called API access-keys from the service provider to be able to use the service. 43 | 44 | Luckily, Nominatim, which is a geocoder based on OpenStreetMap data does not require a API key to use their service 45 | if it is used for small scale geocoding jobs as the service is rate-limited to 1 request per second (3600 / hour). 46 | As we are only making a small set of queries, we can do the geocoding by using Nominatim. 47 | 48 | .. note:: 49 | 50 | - **Note 1:** If you need to do larger scale geocoding jobs, use and request an API key to some of the geocoders listed above. 51 | 52 | - **Note 2:** There are also other Python modules in addition to geopy that can do geocoding such as `Geocoder `__. 53 | 54 | .. hint:: 55 | 56 | You can get your access keys to e.g. Google Geocoding API from `Google APIs console `__ by creating a Project 57 | and enabling a that API from `Library `__. Read a 58 | short introduction about using Google API Console from `here `__. 59 | 60 | Geocoding in Geopandas 61 | ---------------------- 62 | 63 | It is possible to do geocoding in Geopandas using its integrated 64 | functionalities of geopy. Geopandas has a function called ``geocode()`` 65 | that can geocode a list of addresses (strings) and return a GeoDataFrame 66 | containing the resulting point objects in ``geometry`` column. Nice, 67 | isn't it! Let's try this out. 68 | 69 | Download a text file called `addresses.txt <../../_static/data/L3/addresses.txt>`__ that 70 | contains few addresses around Helsinki Region. The first rows of the 71 | data looks like following: 72 | 73 | .. parsed-literal:: 74 | 75 | id;addr 76 | 1000;Itämerenkatu 14, 00101 Helsinki, Finland 77 | 1001;Kampinkuja 1, 00100 Helsinki, Finland 78 | 1002;Kaivokatu 8, 00101 Helsinki, Finland 79 | 1003;Hermannin rantatie 1, 00580 Helsinki, Finland 80 | 81 | We have an ``id`` for each row and an address on column ``addr``. 82 | 83 | - Let's first read the data into a Pandas DataFrame using 84 | ``read_csv()`` -function: 85 | 86 | .. code:: python 87 | 88 | # Import necessary modules 89 | import pandas as pd 90 | import geopandas as gpd 91 | from shapely.geometry import Point 92 | 93 | # Filepath 94 | fp = r"addresses.txt" 95 | 96 | # Read the data 97 | data = pd.read_csv(fp, sep=';') 98 | 99 | .. ipython:: python 100 | :suppress: 101 | 102 | # THIS CODE WILL BE RUNNING IN BACKGROUND 103 | # --------------------------------------- 104 | import gdal 105 | import pandas as pd 106 | import geopandas as gpd 107 | from shapely.geometry import Point 108 | from geopandas.tools import geocode 109 | import os 110 | fp = os.path.join(os.path.abspath('data'), "addresses.txt") 111 | data = pd.read_csv(fp, sep=';') 112 | 113 | .. ipython:: python 114 | 115 | # Let's take a look of the data 116 | data.head() 117 | 118 | Now we have our data in a Pandas DataFrame and we can geocode our addresses. 119 | 120 | - Let's 121 | .. ipython:: python 122 | 123 | # Import the geocoding tool 124 | from geopandas.tools import geocode 125 | 126 | # Geocode addresses with Nominatim backend 127 | geo = geocode(data['addr'], provider='nominatim') 128 | geo.head(2) 129 | 130 | And Voilà! As a result we have a GeoDataFrame that contains our original 131 | address and a 'geometry' column containing Shapely Point -objects that 132 | we can use for exporting the addresses to a Shapefile for example. 133 | However, the ``id`` column is not there. Thus, we need to join the 134 | information from ``data`` into our new GeoDataFrame ``geo``, thus making 135 | a **Table Join**. 136 | 137 | Table join 138 | ---------- 139 | 140 | Table joins are really common procedures when 141 | doing GIS analyses. As you might remember from our earlier lessons, combining data from different tables based on common 142 | ``key`` attribute can be done easily in Pandas/Geopandas using `.merge() `__ 143 | -function. 144 | 145 | However, sometimes it is useful to join two tables together based on the **index** of those DataFrames. In such case, we assume 146 | that there is **same number of records** in our DataFrames and that the **order of the records should be the same** in both DataFrames. 147 | In fact, now we have such a situation as we are geocoding our addresses where the order of the geocoded addresses in ``geo`` DataFrame is the same 148 | as in our original ``data`` DataFrame. 149 | 150 | Hence, we can join those tables together with ``join()`` -function which merges the two DataFrames together 151 | based on index by default. 152 | 153 | .. ipython:: python 154 | 155 | join = geo.join(data) 156 | join.head() 157 | 158 | - Let's also check the data type of our new ``join`` table. 159 | 160 | .. ipython:: python 161 | 162 | type(join) 163 | 164 | As a result we have a new GeoDataFrame called ``join`` where we now have 165 | all original columns plus a new column for ``geometry``. 166 | 167 | - Now it is easy to save our address points into a Shapefile 168 | 169 | .. code:: python 170 | 171 | # Output file path 172 | outfp = r"/home/geo/addresses.shp" 173 | 174 | # Save to Shapefile 175 | join.to_file(outfp) 176 | 177 | That's it. Now we have successfully geocoded those addresses into Points 178 | and made a Shapefile out of them. Easy isn't it! 179 | 180 | .. hint:: 181 | 182 | Nominatim works relatively nicely if you have well defined and well-known addresses such as the ones that we used in this tutorial. 183 | However, in some cases, you might not have such well-defined addresses, and you might have e.g. only the name of a museum available. 184 | In such cases, Nominatim might not provide such good results, and in such cases you might want to use e.g. `Google Geocoding API (V3) `__. 185 | Take a look from last year, `where we show how to use Google Geocoding API `__ in a similar manner as we used Nominatim here. -------------------------------------------------------------------------------- /source/lessons/L3/img/Youtube_title_L3_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L3/img/Youtube_title_L3_1.png -------------------------------------------------------------------------------- /source/lessons/L3/img/Youtube_title_L3_2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L3/img/Youtube_title_L3_2.png -------------------------------------------------------------------------------- /source/lessons/L3/lecture.rst: -------------------------------------------------------------------------------- 1 | Lecture 2 | ======= 3 | 4 | .. admonition:: Lesson 3.1 - Geocoding addresses; Point in Polygon queries 5 | 6 | .. raw:: html 7 | 8 | 9 |

Henrikki Tenkanen, University of Helsinki @ AutoGIS channel on Youtube.

10 | 11 | 12 | .. admonition:: Lesson 3.2 - Spatial join; Nearest neighbor analysis; Exercise 3 overview 13 | 14 | .. raw:: html 15 | 16 | 17 |

Henrikki Tenkanen, University of Helsinki @ AutoGIS channel on Youtube.

18 | -------------------------------------------------------------------------------- /source/lessons/L3/nearest-neighbour.rst: -------------------------------------------------------------------------------- 1 | Nearest Neighbour Analysis 2 | ========================== 3 | 4 | One commonly used GIS task is to be able to find the nearest neighbour. For instance, you might have a single Point object 5 | representing your home location, and then another set of locations representing e.g. public transport stops. Then, quite typical question is *"which of the stops is closest one to my home?"* 6 | This is a typical nearest neighbour analysis, where the aim is to find the closest geometry to another geometry. 7 | 8 | In Python this kind of analysis can be done with shapely function called ``nearest_points()`` that `returns a tuple of the nearest points in the input geometrie `__. 9 | 10 | Nearest point using Shapely 11 | --------------------------- 12 | 13 | Let's start by testing how we can find the nearest Point using the ``nearest_points()`` function of Shapely. 14 | 15 | Let's create an origin Point and a few destination Points and find out the closest destination. 16 | 17 | .. ipython:: python 18 | 19 | from shapely.geometry import Point, MultiPoint 20 | from shapely.ops import nearest_points 21 | 22 | orig = Point(1, 1.67) 23 | dest1, dest2, dest3 = Point(0, 1.45), Point(2, 2), Point(0, 2.5) 24 | 25 | To be able to find out the closest destination point from the origin, we need to create a MultiPoint object from the destination points. 26 | 27 | .. ipython:: python 28 | 29 | destinations = MultiPoint([dest1, dest2, dest3]) 30 | print(destinations) 31 | 32 | Okey, now we can see that all the destination points are represented as a single MultiPoint object. 33 | 34 | - Now we can find out the nearest destination point by using ``nearest_points()`` function. 35 | 36 | .. ipython:: python 37 | 38 | nearest_geoms = nearest_points(orig, destinations) 39 | near_idx0 = nearest_geoms[0] 40 | near_idx1 = nearest_geoms[1] 41 | print(nearest_geoms) 42 | print(near_idx0) 43 | print(near_idx1) 44 | 45 | As you can see the ``nearest_points()`` function returns a tuple of geometries where the first item is the geometry 46 | of our origin point and the second item (at index 1) is the actual nearest geometry from the destination points. 47 | Hence, the closest destination point seems to be the one located at coordinates (0, 1.45). 48 | 49 | This is the basic logic how we can find the nearest point from a set of points. 50 | 51 | Nearest points using Geopandas 52 | ------------------------------ 53 | 54 | Of course, the previous example is not really useful yet. Hence, next I show, how it is possible to find nearest points 55 | from a set of origin points to a set of destination points using GeoDataFrames. If you don't already have the addresses and PKS_suuralueet.kml datasets, 56 | you can find and download them from :doc:`geocoding ` and :doc:`Point in Polygon ` tutorials. 57 | 58 | - First we need to create a function that takes advantage of the previous function but is tailored to work with two GeoDataFrames. 59 | 60 | .. code:: python 61 | 62 | def nearest(row, geom_union, df1, df2, geom1_col='geometry', geom2_col='geometry', src_column=None): 63 | """Find the nearest point and return the corresponding value from specified column.""" 64 | # Find the geometry that is closest 65 | nearest = df2[geom2_col] == nearest_points(row[geom1_col], geom_union)[1] 66 | # Get the corresponding value from df2 (matching is based on the geometry) 67 | value = df2[nearest][src_column].get_values()[0] 68 | return value 69 | 70 | .. ipython:: python 71 | :suppress: 72 | 73 | def nearest(row, geom_union, df1, df2, geom1_col='geometry', geom2_col='geometry', src_column=None): 74 | nearest = df2[geom2_col] == nearest_points(row[geom1_col], geom_union)[1] 75 | value = df2[nearest][src_column].get_values()[0] 76 | return value 77 | 78 | Next we read the address data and the Helsinki districts data and find out the closest address to the centroid of each district. 79 | 80 | .. ipython:: python 81 | :suppress: 82 | 83 | import geopandas as gpd 84 | fp1 = r"C:\HY-DATA\HENTENKA\KOODIT\Opetus\Automating-GIS-processes\2017\data\PKS_suuralue.kml" 85 | fp2 = r"C:\HY-DATA\HENTENKA\KOODIT\Opetus\Automating-GIS-processes\2017\data\addresses.shp" 86 | gpd.io.file.fiona.drvsupport.supported_drivers['KML'] = 'rw' 87 | 88 | df1 = gpd.read_file(fp1, driver='KML') 89 | df2 = gpd.read_file(fp2) 90 | 91 | .. code:: python 92 | 93 | In [7]: import geopandas as gpd 94 | 95 | In [8]: fp1 = "/home/geo/PKS_suuralue.kml" 96 | In [9]: fp2 = "/home/geo/addresses.shp" 97 | In [10]: gpd.io.file.fiona.drvsupport.supported_drivers['KML'] = 'rw' 98 | 99 | In [11]: df1 = gpd.read_file(fp1, driver='KML') 100 | In [12]: df2 = gpd.read_file(fp2) 101 | 102 | Create unary union from Points, which basically creates a MultiPoint object from the Point geometries. 103 | 104 | .. ipython:: python 105 | 106 | unary_union = df2.unary_union 107 | print(unary_union) 108 | 109 | Calculate the centroids for each district area. 110 | 111 | .. ipython:: python 112 | 113 | df1['centroid'] = df1.centroid 114 | df1.head() 115 | 116 | Okey now we are ready to use our function and find closest Points (taking the value from id column) from df2 to df1 centroids 117 | 118 | .. ipython:: python 119 | 120 | df1['nearest_id'] = df1.apply(nearest, geom_union=unary_union, df1=df1, df2=df2, geom1_col='centroid', src_column='id', axis=1) 121 | df1.head(20) 122 | 123 | That's it! Now we found the closest point for each centroid and got the ``id`` value from our addresses into the ``df1`` GeoDataFrame. 124 | 125 | 126 | -------------------------------------------------------------------------------- /source/lessons/L3/overview.rst: -------------------------------------------------------------------------------- 1 | Overview 2 | ======== 3 | 4 | During this week we will learn a few really useful and commonly used GIS functionalities using Geopandas, namely: 5 | 6 | 1. :doc:`Geocoding addresses to Points (and vice versa) ` 7 | 2. :doc:`Conducting Point in Polygon queries ` 8 | 3. :doc:`Making Spatial joins ` 9 | 4. :doc:`Finding Nearest neighbour of a geometry ` 10 | 11 | Learning goals 12 | -------------- 13 | 14 | After this weeks' lesson you should be able to: 15 | 16 | - Do geocoding, i.e. converting addresses into Points (and vice versa) 17 | - Conduct a Point in Polygon queries 18 | - Read data from KML file 19 | - Make spatial and table joins between layers 20 | - Find the nearest neighbour from Point -objects. 21 | 22 | Sources 23 | ------- 24 | 25 | Following materials are partly based on documentation of `Geopandas `__, `geopy `__, `Google `_, `Pandas `__, 26 | `Shapely `_, and `Lawhead, J. (2013), Chapters I and V `_. 27 | -------------------------------------------------------------------------------- /source/lessons/L3/spatial-join.rst: -------------------------------------------------------------------------------- 1 | 2 | Spatial join 3 | ============ 4 | 5 | `Spatial join `__ is 6 | yet another classic GIS problem. Getting attributes from one layer and 7 | transferring them into another layer based on their spatial relationship 8 | is something you most likely need to do on a regular basis. 9 | 10 | The previous materials focused on learning how to perform a `Point in Polygon query `__. 11 | We could now apply those techniques and create our 12 | own function to perform a spatial join between two layers based on their 13 | spatial relationship. We could for example join the attributes of a 14 | polygon layer into a point layer where each point would get the 15 | attributes of a polygon that ``contains`` the point. 16 | 17 | Luckily, `spatial join `__ 18 | (``gpd.sjoin()`` -function) is already implemented in Geopandas, thus we 19 | do not need to create it ourselves. There are three possible types of 20 | join that can be applied in spatial join that are determined with ``op`` 21 | -parameter: 22 | 23 | - ``"intersects"`` 24 | - ``"within"`` 25 | - ``"contains"`` 26 | 27 | Sounds familiar? Yep, all of those spatial relationships were discussed 28 | in the `previous materials `__, thus you should know how they work. 29 | 30 | Let's perform a spatial join between the address-point Shapefile that we 31 | `created `__ and then `reprojected `__ 32 | and a Polygon layer that is a 33 | 250m x 250m grid showing the amount of people living in Helsinki Region. 34 | 35 | Download and clean the data 36 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~ 37 | 38 | For this lesson we will be using publicly available population data from 39 | Helsinki that can be downloaded from `Helsinki Region Infroshare 40 | (HRI) `__ which is an 41 | excellent source that provides all sorts of open data from Helsinki, 42 | Finland. 43 | 44 | From HRI **download a** `Population grid for year 45 | 2015 `__ 46 | that is a dataset (.shp) produced by Helsinki Region Environmental 47 | Services Authority (HSY) (see `this 48 | page `__ 49 | to access data from different years). 50 | 51 | - Unzip the file in Terminal into a folder called Pop15 (using -d flag) 52 | 53 | .. code:: bash 54 | 55 | $ cd 56 | $ unzip Vaestotietoruudukko_2015.zip -d Pop15 57 | $ ls Pop15 58 | Vaestotietoruudukko_2015.dbf Vaestotietoruudukko_2015.shp 59 | Vaestotietoruudukko_2015.prj Vaestotietoruudukko_2015.shx 60 | 61 | You should now have a folder ``/home/geo/Pop15`` with files listed 62 | above. 63 | 64 | - Let's read the data into memory and see what we have. 65 | 66 | .. ipython:: python 67 | :suppress: 68 | 69 | import os 70 | import gdal 71 | fp = os.path.join(os.path.abspath('data'), "Vaestotietoruudukko_2015.shp") 72 | pop = gpd.read_file(fp) 73 | 74 | .. code:: python 75 | 76 | import geopandas as gpd 77 | 78 | # Filepath 79 | fp = "/home/geo/Pop15/Vaestotietoruudukko_2015.shp" 80 | 81 | # Read the data 82 | pop = gpd.read_file(fp) 83 | 84 | .. ipython:: python 85 | 86 | # See the first rows 87 | pop.head() 88 | 89 | Okey so we have multiple columns in the dataset but the most important 90 | one here is the column ``ASUKKAITA`` (*population in Finnish*) that 91 | tells the amount of inhabitants living under that polygon. 92 | 93 | - Let's change the name of that columns into ``pop15`` so that it is 94 | more intuitive. Changing column names is easy in Pandas / Geopandas 95 | using a function called ``rename()`` where we pass a dictionary to a 96 | parameter ``columns={'oldname': 'newname'}``. 97 | 98 | .. ipython:: python 99 | 100 | # Change the name of a column 101 | pop = pop.rename(columns={'ASUKKAITA': 'pop15'}) 102 | 103 | # See the column names and confirm that we now have a column called 'pop15' 104 | pop.columns 105 | 106 | - Let's also get rid of all unnecessary columns by selecting only 107 | columns that we need i.e. ``pop15`` and ``geometry`` 108 | 109 | .. ipython:: python 110 | 111 | # Columns that will be sected 112 | selected_cols = ['pop15', 'geometry'] 113 | 114 | # Select those columns 115 | pop = pop[selected_cols] 116 | 117 | # Let's see the last 2 rows 118 | pop.tail(2) 119 | 120 | Now we have cleaned the data and have only those columns that we need 121 | for our analysis. 122 | 123 | Join the layers 124 | ~~~~~~~~~~~~~~~ 125 | 126 | Now we are ready to perform the spatial join between the two layers that 127 | we have. The aim here is to get information about **how many people live 128 | in a polygon that contains an individual address-point** . Thus, we want 129 | to join attributes from the population layer we just modified into the 130 | addresses point layer ``addresses_epsg3879.shp``. 131 | 132 | - Read the addresses layer into memory 133 | 134 | .. ipython:: python 135 | 136 | # Addresses filpath 137 | addr_fp = r"/home/geo/addresses_epsg3879.shp" 138 | 139 | @suppress 140 | import os 141 | 142 | @suppress 143 | "NOTICE: Following is the real path to the data, the one above is for online documentation to reflect the situation at computing instance" 144 | 145 | @suppress 146 | addr_fp = os.path.join(os.path.abspath('data'), "addresses_epsg3879.shp") 147 | 148 | # Read data 149 | addresses = gpd.read_file(addr_fp) 150 | 151 | # Check the head of the file 152 | addresses.head(2) 153 | 154 | - Let's make sure that the coordinate reference system of the layers 155 | are identical 156 | 157 | .. ipython:: python 158 | 159 | # Check the crs of address points 160 | addresses.crs 161 | 162 | # Check the crs of population layer 163 | pop.crs 164 | 165 | # Do they match? - We can test that 166 | addresses.crs == pop.crs 167 | 168 | Indeed they are identical. Thus, we can be sure that when doing spatial 169 | queries between layers the locations match and we get the right results 170 | e.g. from the spatial join that we are conducting here. 171 | 172 | - Let's now join the attributes from ``pop`` GeoDataFrame into 173 | ``addresses`` GeoDataFrame by using ``gpd.sjoin()`` -function 174 | 175 | .. ipython:: python 176 | 177 | # Make a spatial join 178 | join = gpd.sjoin(addresses, pop, how="inner", op="within") 179 | 180 | # Let's check the result 181 | join.head() 182 | 183 | Awesome! Now we have performed a successful spatial join where we got 184 | two new columns into our ``join`` GeoDataFrame, i.e. ``index_right`` 185 | that tells the index of the matching polygon in the ``pop`` layer and 186 | ``pop15`` which is the population in the cell where the address-point is 187 | located. 188 | 189 | - Let's save this layer into a new Shapefile 190 | 191 | .. code:: python 192 | 193 | # Output path 194 | outfp = r"/home/geo/addresses_pop15_epsg3979.shp" 195 | 196 | # Save to disk 197 | join.to_file(outfp) 198 | 199 | Do the results make sense? Let's evaluate this a bit by plotting the 200 | points where color intensity indicates the population numbers. 201 | 202 | - Plot the points and use the ``pop15`` column to indicate the color. 203 | ``cmap`` -parameter tells to use a sequential colormap for the 204 | values, ``markersize`` adjusts the size of a point, ``scheme`` parameter can be used to adjust the classification method based on `pysal `_, and ``legend`` tells that we want to have a legend. 205 | 206 | .. ipython:: python 207 | 208 | import matplotlib.pyplot as plt 209 | 210 | # Plot the points with population info 211 | join.plot(column='pop15', cmap="Reds", markersize=7, scheme='natural_breaks', legend=True); 212 | 213 | # Add title 214 | plt.title("Amount of inhabitants living close the the point"); 215 | 216 | # Remove white space around the figure 217 | @savefig population_points.png width=7in 218 | plt.tight_layout() 219 | 220 | By knowing approximately how population is distributed in Helsinki, it 221 | seems that the results do make sense as the points with highest 222 | population are located in the south where the city center of Helsinki 223 | is. 224 | -------------------------------------------------------------------------------- /source/lessons/L4/ex-4.rst: -------------------------------------------------------------------------------- 1 | Exercise 4 2 | ========== 3 | 4 | .. admonition:: Start your assignment 5 | 6 | You can start working on your copy of Exercise 4 by `accepting the GitHub Classroom assignment `__. 7 | 8 | **Exercise 4 is due by the start of lecture on 27.11**. 9 | 10 | You can also take a look at the open course copy of `Exercise 4 in the course GitHub repository `__ (does not require logging in). 11 | Note that you should not try to make changes to this copy of the exercise, but rather only to the copy available via GitHub Classroom. 12 | -------------------------------------------------------------------------------- /source/lessons/L4/exercise-4-hints.rst: -------------------------------------------------------------------------------- 1 | Exercise 4 hints 2 | ================ 3 | 4 | General 5 | ------- 6 | 7 | - Documentation of the Travel Time Matrix dataset and explanation for different column names can be found at the Accessibility Research Group website: `Helsinki Region Travel Time Matrix 2015 `__ 8 | 9 | Problem 1 10 | --------- 11 | 12 | - Note that the input travel time data is stored in text files when reading in the data. 13 | - Keep columns `'from_id'`,`'to_id'`,`'pt_r_tt'` and `'car_r_t'` in the travel time data files 14 | - Join the data using columns `'from_id'` from the travel time data, and `'YKR_ID'` in the grid-shapefile 15 | - See hints for joining the travel time data to the grid shapefile from our earlier materials from first period (Geo-Python course): `Table join `__ 16 | - Plotting the data takes a while (be patient!) 17 | 18 | Problem 2 19 | --------- 20 | 21 | **General steps**: 22 | 23 | 1. Read the files and prepare a single DataFrame where you have travel times for all shopping centers 24 | 2. Find out for each row what is the minimum travel time from those shopping centers 25 | 3. Find out for each row what is the column name of that shopping center that had the minimum travel time 26 | 4. Make maps from the results 27 | 28 | Reading multiple files efficiently 29 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 30 | 31 | Here we are reading multiple files from a folder. We could write the filepaths to all of those files but **it is not efficient!** 32 | Instead, you should use `glob()` -function from module glob to get a filtered list of those files that you want to read and then read the files by iterating over the list. 33 | 34 | Listing and searching for file path names from file system can be done using a specific module called `glob `__. 35 | 36 | The glob library contains a function, also called glob, that finds files and directories whose names match a pattern. 37 | We provide those patterns as strings: the character * matches zero or more characters, while ? matches any one character. 38 | 39 | - We can use this to get the names of all files in the data directory ('/home/geo/data'): 40 | 41 | .. code:: python 42 | 43 | In [0]: import glob 44 | In [1]: my_files = glob.glob('/home/geo/data/*') 45 | In [2]: print(my_files) 46 | ['/home/geo/data/inflammation-08.csv', 47 | '/home/geo/data/inflammation-10.csv', 48 | '/home/geo/data/inflammation-11.csv', 49 | '/home/geo/data/inflammation-06.csv', 50 | '/home/geo/data/inflammation-12.csv', 51 | '/home/geo/data/small-03.csv', 52 | '/home/geo/data/small-02.csv', 53 | '/home/geo/data/inflammation-07.csv', 54 | '/home/geo/data/inflammation-05.csv', 55 | '/home/geo/data/small-01.csv', 56 | '/home/geo/data/inflammation-03.csv', 57 | '/home/geo/data/inflammation-04.csv', 58 | '/home/geo/data/inflammation-02.csv', 59 | '/home/geo/data/inflammation-01.csv', 60 | '/home/geo/data/inflammation-09.csv'] 61 | 62 | - We can also search for only specific files and file formats. Here, we search for files that starts with the word 'small' and ends with file format '.csv': 63 | 64 | .. code:: python 65 | 66 | In [3]: csv_files = glob.glob('/home/geo/data/small*.csv') 67 | In [4]: print(csv_files) 68 | ['/home/geo/data/small-03.csv', '/home/geo/data/small-02.csv', '/home/geo/data/small-01.csv'] 69 | 70 | Now we have successfully filtered only certain types of files and as a result we have a list of files that we 71 | can loop over and process. 72 | 73 | Renaming column based on 'to_id' value 74 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 75 | 76 | We want to rename our column from `pt_r_tt` into `pt_r_tt_XXXXXXX` where XXXXXX is a `to_id` of our datafile which is identical for every row. How you should proceed this problem is following: 77 | 78 | - Extract a **single** value from `to_id` column into a variable called `destination` ==> you might want to use `.loc[]` functionality of Pandas 79 | - Convert the integer value of `destination` variable as **string** 80 | - Rename the `pt_r_tt` value: 81 | 82 | - Parse the new column name where you combine `pt_r_tt` text and text from `destination` variable into a new variable called `new_name`. 83 | - Rename the `pt_r_tt` column using value from `new_name` variable. Use `data.rename()` -function. See `lesson 6 materials in GeoPython `__. 84 | 85 | Finding out which shopping center is the closest 86 | 87 | We can find out the minimum value from multiple columns simply by applying a `.min()` function to those columns of a row that we are interessted in: 88 | 89 | .. code:: python 90 | 91 | # Define the columns that are used in the query 92 | value_columns = ['center1', 'center2', 'center3'] 93 | 94 | # Find out the minimum value of those column of a given row in the DataFrame 95 | minimum_values = row[value_columns].min() 96 | 97 | It is also possible to find out which column contains that value by applying ``.idxmin()`` -function (`see Pandas docs `__). 98 | 99 | .. code:: python 100 | 101 | # Find out which column contains the minimum value 102 | closest_center = row[value_columns].idxmin() 103 | 104 | In order to calculate the results for each row, you can take advantage of the `.iterrows()` and `.loc()` -functions in (geo)pandas. 105 | See example from Geo-Python course: `Lesson 5: Selecting data `__ 106 | -------------------------------------------------------------------------------- /source/lessons/L4/img/Youtube_title_L4_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L4/img/Youtube_title_L4_1.png -------------------------------------------------------------------------------- /source/lessons/L4/img/Youtube_title_L4_2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L4/img/Youtube_title_L4_2.png -------------------------------------------------------------------------------- /source/lessons/L4/lecture.rst: -------------------------------------------------------------------------------- 1 | Lecture 2 | ======= 3 | 4 | .. admonition:: Lesson 4.1 - Data classification; Pysal map classification schemes 5 | 6 | .. raw:: html 7 | 8 | 9 |

Henrikki Tenkanen, University of Helsinki @ AutoGIS channel on Youtube.

10 | 11 | 12 | .. admonition:: Lesson 4.2 - Overlay analysis; Aggregate spatial data; Simplify geometries; Exercise 4 overview 13 | 14 | .. raw:: html 15 | 16 | 17 |

Henrikki Tenkanen, University of Helsinki @ AutoGIS channel on Youtube.

18 | 19 | -------------------------------------------------------------------------------- /source/lessons/L4/overview.rst: -------------------------------------------------------------------------------- 1 | Overview 2 | ======== 3 | 4 | During this week we will learn a few really useful and commonly used GIS functionalities using Geopandas, namely: 5 | 6 | 1. :doc:`Data classification ` 7 | 2. :doc:`Geometric operations: Overlay analysis, spatial aggregation ` 8 | 3. :doc:`Exercise 4 ` 9 | 4. :doc:`Exercise 4 hints ` 10 | 11 | Learning goals 12 | -------------- 13 | 14 | After this week's lesson you should be able to: 15 | 16 | - Reclassify data based on different criteria (custom or common classifiers). 17 | 18 | - Do a simple overlay analysis such as selecting data from one layer based on boundaries of another layer. 19 | 20 | - Aggregate data, i.e. merge geometric objects together, based on some common identifier. 21 | 22 | - Simplify geometries 23 | 24 | Sources 25 | ------- 26 | 27 | Following materials are partly based on documentation of `Geopandas `__, `Pandas `__, `Shapely 28 | `__, and `pysal `_. 29 | 30 | -------------------------------------------------------------------------------- /source/lessons/L5/download-data.rst: -------------------------------------------------------------------------------- 1 | Download datasets 2 | ================= 3 | 4 | Before we start you need to download (and then extract) the dataset zip-package used during this lesson `from this link `_. 5 | 6 | You should have following Shapefiles in the ``dataE5`` folder: 7 | 8 | - addresses.shp 9 | - metro.shp 10 | - roads.shp 11 | - some.geojson 12 | - TravelTimes_to_5975375_RailwayStation.shp 13 | - Vaestotietoruudukko_2015.shp 14 | 15 | Extract the files into a folder called ``data``: 16 | 17 | .. code:: bash 18 | 19 | $ cd 20 | $ unzip dataE5.zip -d data -------------------------------------------------------------------------------- /source/lessons/L5/ex-5.rst: -------------------------------------------------------------------------------- 1 | Exercise 5 2 | ========== 3 | 4 | .. admonition:: Start your assignment 5 | 6 | You can start working on your copy of Exercise 5 by `accepting the GitHub Classroom assignment `__. 7 | 8 | **Exercise 5 is due by the start of lecture on 4.12**. 9 | 10 | You can also take a look at the open course copy of `Exercise 5 in the course GitHub repository `__ (does not require logging in). 11 | Note that you should not try to make changes to this copy of the exercise, but rather only to the copy available via GitHub Classroom. 12 | -------------------------------------------------------------------------------- /source/lessons/L5/exercise-5-hints.rst: -------------------------------------------------------------------------------- 1 | Exercise 5 hints 2 | ================ 3 | 4 | Add hints here if needed. -------------------------------------------------------------------------------- /source/lessons/L5/img/BoxSelect.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L5/img/BoxSelect.png -------------------------------------------------------------------------------- /source/lessons/L5/img/BoxZoom.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L5/img/BoxZoom.png -------------------------------------------------------------------------------- /source/lessons/L5/img/Github-pages-activated.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L5/img/Github-pages-activated.PNG -------------------------------------------------------------------------------- /source/lessons/L5/img/Github-pages-repo-root.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L5/img/Github-pages-repo-root.PNG -------------------------------------------------------------------------------- /source/lessons/L5/img/Github-pages-settings.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L5/img/Github-pages-settings.PNG -------------------------------------------------------------------------------- /source/lessons/L5/img/Inspector.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L5/img/Inspector.png -------------------------------------------------------------------------------- /source/lessons/L5/img/LassoSelect.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L5/img/LassoSelect.png -------------------------------------------------------------------------------- /source/lessons/L5/img/Pan.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L5/img/Pan.png -------------------------------------------------------------------------------- /source/lessons/L5/img/Reset.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L5/img/Reset.png -------------------------------------------------------------------------------- /source/lessons/L5/img/Save.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L5/img/Save.png -------------------------------------------------------------------------------- /source/lessons/L5/img/TapSelect.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L5/img/TapSelect.png -------------------------------------------------------------------------------- /source/lessons/L5/img/WheelPan.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L5/img/WheelPan.png -------------------------------------------------------------------------------- /source/lessons/L5/img/WheelZoom.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L5/img/WheelZoom.png -------------------------------------------------------------------------------- /source/lessons/L5/img/Youtube_title_L5_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L5/img/Youtube_title_L5_1.png -------------------------------------------------------------------------------- /source/lessons/L5/img/Youtube_title_L5_2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L5/img/Youtube_title_L5_2.png -------------------------------------------------------------------------------- /source/lessons/L5/img/ZoomIn.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L5/img/ZoomIn.png -------------------------------------------------------------------------------- /source/lessons/L5/img/ZoomOut.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L5/img/ZoomOut.png -------------------------------------------------------------------------------- /source/lessons/L5/img/docs-contents.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L5/img/docs-contents.PNG -------------------------------------------------------------------------------- /source/lessons/L5/img/github-pages-docs-folder.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L5/img/github-pages-docs-folder.PNG -------------------------------------------------------------------------------- /source/lessons/L5/img/github-settings.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L5/img/github-settings.PNG -------------------------------------------------------------------------------- /source/lessons/L5/img/index-md.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L5/img/index-md.PNG -------------------------------------------------------------------------------- /source/lessons/L5/img/index_HTML.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L5/img/index_HTML.PNG -------------------------------------------------------------------------------- /source/lessons/L5/lecture.rst: -------------------------------------------------------------------------------- 1 | Lecture 2 | ======= 3 | 4 | .. admonition:: Lesson 5.1 - Visualization; Static maps; Interactive maps with Bokeh part I 5 | 6 | .. raw:: html 7 | 8 | 9 |

Henrikki Tenkanen, University of Helsinki @ AutoGIS channel on Youtube.

10 | 11 | 12 | .. admonition:: Lesson 5.2 - Interactive maps with Bokeh part II 13 | 14 | .. raw:: html 15 | 16 | 17 |

Henrikki Tenkanen, University of Helsinki @ AutoGIS channel on Youtube.

18 | -------------------------------------------------------------------------------- /source/lessons/L5/overview.rst: -------------------------------------------------------------------------------- 1 | Lesson 5 Overview 2 | ================= 3 | 4 | This week we will focus on learning how to create beautiful maps in Python and how to share them on GitHub using `GitHub Pages `_. 5 | 6 | 1. :doc:`Static maps ` 7 | 2. :doc:`Interactive maps with bokeh ` 8 | 2.1 :doc:`Advanced map features in bokeh ` 9 | 3. :doc:`Interactive Leaflet maps with Folium ` 10 | 4. :doc:`Sharing interactive maps on GitHub ` 11 | 5. :doc:`Exercise 5 ` 12 | 6. :doc:`Exercise 5 hints ` 13 | 14 | As you might remember from our earlier `plotting lesson `_ 15 | during the Period I, there are many different Python modules that can be used for making visualizations. 16 | 17 | Quite many of them allows you to create also different kinds of maps. During this lesson we will focus on few of them, namely: 18 | 19 | - `Matplotlib `_ (static maps, integrated into `Geopandas `_) 20 | - `Bokeh `_ (interactive plots) 21 | - `Folium `_ (interactive Web maps on Leaflet) 22 | 23 | In addition to these modules, there are also at least a few good and interesting modules for making maps that we cannot cover today: 24 | 25 | - `mplleaflet `_ (converts Matplotlib plots easily to interactive Leaflet maps) 26 | - `Basemap `_ (Matplotlib's own mapping module) 27 | - `GeoViews `_ 28 | 29 | Learning goals 30 | -------------- 31 | 32 | After this weeks lesson you should be able to (at least): 33 | 34 | - Create a static map using Geopandas 35 | 36 | - Create a simple interactive map using either Bokeh or Folium (or both). 37 | 38 | - Share your maps (static / interactive) on GitHub using GitHub pages. 39 | -------------------------------------------------------------------------------- /source/lessons/L5/share-on-github.rst: -------------------------------------------------------------------------------- 1 | Sharing interactive plots on GitHub 2 | =================================== 3 | 4 | Interactive plots are really nice but of course it would be nice to be able to show them to your friend also, wouldn't it? 5 | 6 | Luckily, there is a really easy way of doing this using GitHub and their specific feature called `GitHub Pages `_ which you can easily combine into your 7 | GitHub repositories. 8 | 9 | Here is a short video demonstrating what all you can do with GitHub pages: 10 | 11 | .. raw:: html 12 | 13 | 14 | 15 | | 16 | 17 | How the repository should be organized? 18 | --------------------------------------- 19 | 20 | For the purpose of sharing our maps on GitHub, we will use the Exercise 5 repository where we will upload our maps. Thus, let's already now start with our **Exercise 5** by 21 | `accepting the GitHub classroom assignment `_. 22 | 23 | In the following examples I have accepted the Exercise 5 and what you will see are based on my personal Exercise-5 repository. You will see yours in a similar manner but under your 24 | GitHub account. 25 | 26 | When you want to publish something on GitHub Pages, it is generally a good idea to store your html files (such as the ones we made earlier) under a single folder called ``docs`` 27 | which should be located on the root of your repository. In the Exercise 5 I have created such a folder for you, thus my Exercise 5 repository looks like following where I have 28 | the folder ``docs``: 29 | 30 | .. image:: img/Github-pages-repo-root.PNG 31 | :width: 1500 px 32 | :scale: 50 % 33 | 34 | | 35 | 36 | Thus, **all your html maps and plots should be stored there!** Why? Because we can set up GitHub to search our html files from there by default and storing those files under a separate folder 37 | keeps things nicely organized. 38 | 39 | What is currently located there are following files: 40 | 41 | .. image:: img/docs-contents.PNG 42 | :width: 1500 px 43 | :scale: 50 % 44 | 45 | | 46 | 47 | Thus, there is already a single map that I have made called *test_map.html*, a html file called *index.html* and the most important file for us called ``index.md``. 48 | 49 | Let's see how does the ``index.md`` file look like (here is a `link `_ to the actual page on our public repository): 50 | 51 | .. image:: img/index-md.PNG 52 | :width: 1500 px 53 | :scale: 50 % 54 | 55 | | 56 | 57 | Okey, so we can see that this is a familiar looking page with some Markdown text and links. What is the purpose of this page is to serve as an **entrance page** for you interactive plots. 58 | Thus as the page say, you should add links to your personal Exercise 5 page such as mine which is located at ``http://automating-gis-processes.github.io/exercise-5-HTenkanen``. 59 | 60 | **Okey but how does that work, how do I get a page with such an address?** 61 | 62 | The magic here is something where **GitHub Pages** comes into a picture. That address is going to be created when I 63 | **activate the GitHub Pages** for my Exercise 5 repository. 64 | 65 | Let's see how that can be done. Luckily **it's easy**. 66 | 67 | Activating GitHub Pages 68 | ----------------------- 69 | 70 | First, you should navigate into the **Settings** of your Exercise 5 repository by clicking following tab on the right side of the upper panel: 71 | 72 | .. image:: img/github-settings.PNG 73 | :width: 1500 px 74 | :scale: 50 % 75 | 76 | | 77 | 78 | In there you can adjust different Setting regarding your repository but what we are interested in is a section called **GitHub Pages** which can be found when you scroll down a little bit. 79 | It looks like following: 80 | 81 | .. image:: img/Github-pages-settings.PNG 82 | :width: 1500 px 83 | :scale: 50 % 84 | 85 | | 86 | 87 | From there you should choose the **source for your pages**, i.e. the place where those html files are searched from. You should choose the ``master branch /docs folder`` which is the location 88 | that we just visited earlier and where we should store our html files: 89 | 90 | .. image:: img/github-pages-docs-folder.PNG 91 | :width: 1500 px 92 | :scale: 50 % 93 | 94 | | 95 | 96 | Finally, you should press the **Save** button and after that GitHub Pages are activated for you! 97 | 98 | .. image:: img/Github-pages-activated.PNG 99 | :width: 1500 px 100 | :scale: 50 % 101 | 102 | | 103 | 104 | Accessing GitHub Pages 105 | ---------------------- 106 | 107 | Now if I press the link that is shown above I will arrive into a following page: 108 | 109 | .. image:: img/index_HTML.PNG 110 | :width: 1500 px 111 | :scale: 50 % 112 | 113 | | 114 | 115 | This page is showing the contents of the ``index.html`` file that was also located in our ``docs`` folder. 116 | 117 | We can now **navigate to whatever file that is located under that folder.** Thus, we can for example access the test_map.html by typing: 118 | `https://automating-gis-processes.github.io/exercise-5-HTenkanen/test_map.html `_ 119 | 120 | **Ahaa!** So this means that if you upload e.g. an interactive map that you have prepared into that folder you are able to access it also from anywhere. And also share it with your 121 | friends! You just need to change the name of the html file according the filename that you have uploaded. So this is how we can easily share interactive maps on GitHub for anyone. 122 | 123 | .. note:: 124 | 125 | Notice that even though your personal Exercise 5 repository in GitHub is private and no-one else can access it, **all the files that you store under the docs folder CAN BE ACCESSED by anyone**. 126 | So keep this in mind when using GitHub Pages. 127 | 128 | .. hint:: 129 | 130 | Notice that now you can also do your personal GitHub Pages where you can store and share all sorts of cool stuff that you have created yourself! You just need to 131 | `create your own repository `_ and follow exactly the same steps that were mentioned here. 132 | -------------------------------------------------------------------------------- /source/lessons/L5/static-maps.rst: -------------------------------------------------------------------------------- 1 | Static maps 2 | =========== 3 | 4 | Download datasets 5 | ----------------- 6 | 7 | Before we start you need to download (and then extract) the dataset zip-package used during this lesson `from this link `_. 8 | 9 | You should have following Shapefiles in the ``dataE5`` folder: 10 | 11 | - addresses.shp 12 | - metro.shp 13 | - roads.shp 14 | - some.geojson 15 | - TravelTimes_to_5975375_RailwayStation.shp 16 | - Vaestotietoruudukko_2015.shp 17 | 18 | Extract the files into a folder called ``data``: 19 | 20 | .. code:: bash 21 | 22 | $ cd 23 | $ unzip dataE5.zip -d data 24 | 25 | Static maps in Geopandas 26 | ------------------------ 27 | 28 | We have already seen during the previous lessons quite many examples how to create static maps using Geopandas. 29 | 30 | Thus, we won't spend too much time repeating making such maps but let's create a one with more layers on it than just one 31 | which kind we have mostly done this far. 32 | 33 | Let's create a static accessibility map with roads and metro line on it. 34 | 35 | First, we need to read the data. 36 | 37 | .. code:: python 38 | 39 | import geopandas as gpd 40 | import matplotlib.pyplot as plt 41 | 42 | # Filepaths 43 | grid_fp = r"/home/geo/data/TravelTimes_to_5975375_RailwayStation.shp" 44 | roads_fp = r"/home/geo/data/roads.shp" 45 | metro_fp = r"/home/geo/data/metro.shp" 46 | 47 | # Read files 48 | grid = gpd.read_file(grid_fp) 49 | roads = gpd.read_file(roads_fp) 50 | metro = gpd.read_file(metro_fp) 51 | 52 | .. ipython:: python 53 | :suppress: 54 | 55 | import os 56 | import gdal 57 | import geopandas as gpd 58 | import maptlotlib.pyplot as plt 59 | 60 | # Filepaths 61 | grid_fp = os.path.join(os.path.abspath('data'), "TravelTimes_to_5975375_RailwayStation.shp") 62 | roads_fp = os.path.join(os.path.abspath('data'), "roads.shp") 63 | metro_fp = os.path.join(os.path.abspath('data'), "metro.shp") 64 | 65 | # Read files 66 | grid = gpd.read_file(grid_fp) 67 | roads = gpd.read_file(roads_fp) 68 | metro = gpd.read_file(metro_fp) 69 | 70 | Then, we need to be sure that the files are in the same coordinate system. Let's use the crs of our travel time grid. 71 | 72 | .. code:: python 73 | 74 | # Get the CRS of the grid 75 | gridCRS = grid.crs 76 | 77 | # Reproject geometries using the crs of travel time grid 78 | roads['geometry'] = roads['geometry'].to_crs(crs=gridCRS) 79 | metro['geometry'] = metro['geometry'].to_crs(crs=gridCRS) 80 | 81 | .. ipython:: python 82 | :suppress: 83 | 84 | gridCRS = grid.crs 85 | roads['geometry'] = roads['geometry'].to_crs(crs=gridCRS) 86 | metro['geometry'] = metro['geometry'].to_crs(crs=gridCRS) 87 | 88 | Finally we can make a visualization using the ``.plot()`` -function in Geopandas. 89 | 90 | .. code:: python 91 | 92 | # Visualize the travel times into 9 classes using "Quantiles" classification scheme 93 | # Add also a little bit of transparency with `alpha` parameter 94 | # (ranges from 0 to 1 where 0 is fully transparent and 1 has no transparency) 95 | my_map = grid.plot(column="car_r_t", linewidth=0.03, cmap="Reds", scheme="quantiles", k=9, alpha=0.9) 96 | 97 | # Add roads on top of the grid 98 | # (use ax parameter to define the map on top of which the second items are plotted) 99 | roads.plot(ax=my_map, color="grey", linewidth=1.5) 100 | 101 | # Add metro on top of the previous map 102 | metro.plot(ax=my_map, color="red", linewidth=2.5) 103 | 104 | # Remove the empty white-space around the axes 105 | plt.tight_layout() 106 | 107 | # Save the figure as png file with resolution of 300 dpi 108 | outfp = r"/home/geo/data/static_map.png" 109 | plt.savefig(outfp, dpi=300) 110 | 111 | And this is how our map should look like: 112 | 113 | .. ipython:: python 114 | :suppress: 115 | 116 | my_map = grid.plot(column="car_r_t", linewidth=0.03, cmap="Spectral", scheme="quantiles", k=9, alpha=0.9); 117 | roads.plot(ax=my_map, color="grey", linewidth=1.5); 118 | @savefig static_map.png width=7in 119 | metro.plot(ax=my_map, color="red", linewidth=2.5); 120 | 121 | This kind of approach can be used really effectively to produce large quantities of nice looking maps 122 | (*though this example of ours isn't that pretty yet, but it could be*) which is one of the most useful aspects 123 | of coding and what makes it so important to learn how to code. 124 | -------------------------------------------------------------------------------- /source/lessons/L6/01-overview.rst: -------------------------------------------------------------------------------- 1 | Lesson 6 Overview 2 | ================= 3 | 4 | .. figure:: img/L6-01-overview-01-pyqgis.svg 5 | :width: 500 px 6 | 7 | 8 | This week we will take a quick tour on **how to use Python in QGIS** using the `processing `_ framework. 9 | 10 | Until last year, this course taught the use of the `arcpy `_ module within ArcGIS. Since ArcGIS is a Desktop GIS widely used throughout the private and public sector, feel free to revisit `last year’s Lesson 6 `_. 11 | 12 | Learning objectives 13 | ------------------- 14 | 15 | We **don't** have any exercise for this week. Thus, the aim this week is to only learn the basics how the QGIS tools and algorithms can be run using processing functions, and how we can create a graphical user interface for custom scripts. The aim of this weeks' lesson is to incrementally create an QGIS Python tool that **converts the individual Damselfish species distribution Polygons into raster layers** (``.tif``). 16 | 17 | After this week's lesson you should be able to: 18 | 19 | - Create a simple user interface for a script in the QGIS Processing Toolbox. 20 | 21 | - Run different QGIS algorithms from Python. 22 | 23 | - Find and know how to read the PyQGIS and QGIS Processing documentation online. 24 | 25 | The data we use this week is the same that we used already during the `second week <../L2/geopandas-basics.html>`_: 26 | 27 | .. figure:: img/L6-01-overview-02-damselfish-simplemap.png 28 | 29 | 30 | 31 | 32 | 33 | -------------------------------------------------------------------------------- /source/lessons/L6/02-pyqgis.rst: -------------------------------------------------------------------------------- 1 | Python in QGIS 2 | ============== 3 | 4 | The core application and libraries of QGIS are programmed in C++. Nonwithstanding, Python plays an important role in its ecosystem: Most of the pre-installed plugins and even data provider modules are written in Python, and virtually all functions of the interface and the libraries are exported to a Python API (Advanced Programming Interface). It is fairly easy to author extensions to QGIS which integrate seamlessly into its user interface, create stand-alone applications using components of QGIS, such as a map window or a data backend, or run custom scripts within QGIS. To really dive into it, see the `PyQGIS Developer Cookbook `_ which walks you from easy *Hello World* examples to writing your own applications. 5 | 6 | Today, we want to concentrate on how to **use Python to combine simple algorithms in scripts within QGIS to carry out more complex operations**. 7 | 8 | Using a Python console 9 | ---------------------- 10 | 11 | There is (at least) two different Python consoles available within QGIS: 12 | 13 | 1. Access the **built-in Python console** from the menu *Plugins → Python Console*. It offers basic functionality, and allows to load and save scripts from and to files. 14 | 15 | .. figure:: img/L6-02-pyqgis-00-builtin-python-console.png 16 | 17 | 2. The more advanced **iPython console** has to be installed as a Plugin before first use: 18 | - Go to *Plugins → Manage and Install Plugins*, 19 | - Select to search in *All* plugins, and type `ipython` into the search field. 20 | - Select the **IPyConsole** plugin, and click *Install plugin* 21 | 22 | .. figure:: img/L6-02-pyqgis-01-install-ipyconsole.png 23 | 24 | The console is now available from the menu *Plugins → IPython QGIS Console → Windowed* 25 | 26 | .. figure:: img/L6-02-pyqgis-02-ipyconsole.png 27 | 28 | By default an ``iface`` object is imported, which allows the access to the currently active QGIS instance’s user interface. For example, we can easily retrieve the active (selected) layer, list its name and count its features: 29 | 30 | .. code:: python 31 | 32 | # Get active layer: 33 | In [1]: layer = iface.activeLayer() 34 | 35 | # Print its name, and its number of features: 36 | In [2]: print(layer.name()) 37 | Out[2]: 'DAMSELFISH Distributions' 38 | 39 | In [3]: print(layer.featureCount()) 40 | Out[3]: 231 41 | 42 | You can access a help text on objects using ``help()``: 43 | 44 | .. code:: python 45 | 46 | In [1]: help(layer) 47 | Out[1]: Help on QgsVectorLayer in module qgis._core object: 48 | 49 | class QgsVectorLayer(QgsMapLayer, QgsExpressionContextGenerator, QgsFeatureSink, QgsFeatureSource) 50 | | Represents a vector layer which manages a vector based data sets. 51 | | 52 | | The QgsVectorLayer is instantiated by specifying the name of a data provider, 53 | | such as postgres or wfs, and url defining the specific data set to connect to. 54 | | The vector layer constructor in turn instantiates a QgsVectorDataProvider subclass 55 | | corresponding to the provider type, and passes it the url. The data provider 56 | | connects to the data source. 57 | | 58 | | The QgsVectorLayer provides a common interface to the different data types. It also 59 | | manages editing transactions. 60 | | 61 | | Sample usage of the QgsVectorLayer class: 62 | | 63 | | \code 64 | | QString uri = "point?crs=epsg:4326&field=id:integer"; 65 | | QgsVectorLayer *scratchLayer = new QgsVectorLayer(uri, "Scratch point layer", "memory"); 66 | | \endcode 67 | | 68 | | The main data providers supported by QGIS are listed below. 69 | | 70 | | \section providers Vector data providers 71 | | 72 | | \subsection memory Memory data providerType (memory) 73 | | 74 | | The memory data provider is used to construct in memory data, for example scratch 75 | | data or data generated from spatial operations such as contouring. There is no 76 | | inherent persistent storage of the data. The data source uri is constructed. The 77 | | url specifies the geometry type ("point", "linestring", "polygon", 78 | | "multipoint","multilinestring","multipolygon"), optionally followed by url parameters 79 | | as follows: 80 | … 81 | … 82 | 83 | This help text is the same information listed in QGIS’ API documentation at `qgis.org/api `_. 84 | -------------------------------------------------------------------------------- /source/lessons/L6/03-processing-toolbox.rst: -------------------------------------------------------------------------------- 1 | The QGIS Processing Toolbox 2 | =========================== 3 | 4 | The QGIS Processing Toolbox is a set of geo-spatial algorithms installed with QGIS. Some of them are in fact calling external applications, such as SAGA or GRASS GIS, which is why the algorithms are organised by *provider*. 5 | 6 | .. figure:: img/L6-03-00-processing-toolbox.png 7 | 8 | Type in keyword into the toolbox’ search bar to see what it offers. Searching for “buffer”, for instance, you will notice the different algorithm providers have in fact different implementations of the same algorithm. Often the choice is one of your personal liking, in rare cases the implementations have subtle differences which actually make a difference for your outcomes. E.g. many of the tools from GRASS treat all data as rasters, internally, granted to GRASS’ legacy as one of the earliest GIS systems still in use and the computational limitations at its time of development. 9 | 10 | .. figure:: img/L6-03-01-buffer.png 11 | 12 | These tools can be run one at a time, or – and that’s where we touch upon our course’s topic – be chained together to form more complex models. Apart from that all the individual algorithms and also the user-defined models can be used in batch-operations, i.e. running the same operation on a (large) number of files. 13 | 14 | Let’s see how we can run two operations at once: We want to save a Shapefile with the species ranges of all damselfish of the *Stegates* genus, with simplified geometries. First we select our subset of features, then we simplify the geometries. Let’s use the *Graphical Modeler* of the processing framework. Open it from the *Processing → Graphical Modeler* menu. 15 | 16 | - As a first step, add two input variables to the model, by drag-and-dropping them into the empty window: a *Vector Layer* named ``Input Layer`` and a *String* called ``Genus``. 17 | - Then, add a *Select by attribute* algorithm, and set its input layer to refer to out ``Input Layer`` variable. Enter ``GENUS_NAME`` for the *selection attribute* (that’s the column name in the example dataset which stores the genus information). Select ``=`` as a comparison *operator*, and choose the ``Genus`` variable as comparison *value*. 18 | 19 | .. figure:: img/L6-03-02-select-by-attribute.png 20 | 21 | - In the same fashion, add a *Simplify geometries* algorithm. As input, choose the output of the *Select by attribute* algorithm, and specify a name for the output layer (e.g. ``Output Layer``). 22 | - Enter a name and a group for the model (the models and scripts are grouped in categories, which can be chosen according to the user’s need). 23 | 24 | .. figure:: img/L6-03-03-graphical-model.png 25 | 26 | Run the model by finding it from the toolbox and specifying an input layer and genus name. 27 | 28 | -------------------------------------------------------------------------------- /source/lessons/L6/ex-6.rst: -------------------------------------------------------------------------------- 1 | Exercise 6 2 | ========== 3 | 4 | **For this week there is no exercise that you should return.** 5 | 6 | However, you should go through the :doc:`lesson 6 materials <01-overview>` and test in practice how to use Python 7 | from QGIS. You need to do this by using our cloud computing environment, since there are all necessary 8 | packages installed for going through the lesson materials. 9 | 10 | In addition, you use some time to think if you would have a project of your own for the final assignment of the 11 | course that will be released next week. The project can be for example a dedicated GIS tool for some purpose (e.g. helping your MSc thesis) 12 | or a broader GIS analysis with a short written report about the results (+ documentation in GitHub). 13 | If you would like to do an own project as the final assignment: 14 | 15 | - Send me an email (or private message in Slack) to say that you are interested 16 | - Send me a proposal (by next week's lecture on 11.12) of your project to my email (word-doc), where you should explain: 17 | 18 | - What are the aims of your project? 19 | - What will be the final product of the project? 20 | - What kind of data will you need for the project? 21 | - Do you think all the necessary GIS techniques have been covered in the course to finish the project? 22 | 23 | Map Challenge 2017 24 | ------------------ 25 | 26 | AutoGIS 2017 Map Challenge is a friendly competition for the course participants where the winner is the one 27 | who makes the most interesting or beautiful map. The winner will be selected by a poll where each student of the 28 | course can give their vote for the map that they think is the best. Participation to the challenge is optional but 29 | we encourage everybody to take part, it is not a serious competition and the purpose of this is only to have fun! 30 | 31 | The winner of the AutoGIS Map Challenge will win a gift card to `GitHub WebStore `__ 32 | for buying an AWESOME T-shirt. 33 | 34 | You should upload your map to a dedicated `Map-Challenge-2017 GitHub repository `__ before Monday 19th of December. 35 | Repository is private but all of you should have access to update it (if you don't, send me a message on Slack). We would like to publish your maps on our course site afterwards 36 | but we will ask your permission before doing so (it is not obligatory). 37 | 38 | The voting will happen during the week 51 (19. - 22.12) and the winner will be announced on 23th of December. 39 | 40 | Good luck, and amaze us with some super-beautiful maps!! -------------------------------------------------------------------------------- /source/lessons/L6/img/L6-01-overview-02-damselfish-simplemap.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L6/img/L6-01-overview-02-damselfish-simplemap.png -------------------------------------------------------------------------------- /source/lessons/L6/img/L6-02-pyqgis-00-builtin-python-console.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L6/img/L6-02-pyqgis-00-builtin-python-console.png -------------------------------------------------------------------------------- /source/lessons/L6/img/L6-02-pyqgis-00-manageplugins.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L6/img/L6-02-pyqgis-00-manageplugins.png -------------------------------------------------------------------------------- /source/lessons/L6/img/L6-02-pyqgis-01-install-ipyconsole.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L6/img/L6-02-pyqgis-01-install-ipyconsole.png -------------------------------------------------------------------------------- /source/lessons/L6/img/L6-02-pyqgis-02-ipyconsole.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L6/img/L6-02-pyqgis-02-ipyconsole.png -------------------------------------------------------------------------------- /source/lessons/L6/img/L6-03-00-processing-toolbox.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L6/img/L6-03-00-processing-toolbox.png -------------------------------------------------------------------------------- /source/lessons/L6/img/L6-03-01-buffer.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L6/img/L6-03-01-buffer.png -------------------------------------------------------------------------------- /source/lessons/L6/img/L6-03-02-select-by-attribute.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L6/img/L6-03-02-select-by-attribute.png -------------------------------------------------------------------------------- /source/lessons/L6/img/L6-03-03-graphical-model.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L6/img/L6-03-03-graphical-model.png -------------------------------------------------------------------------------- /source/lessons/L6/img/L6-04-create-new-script.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L6/img/L6-04-create-new-script.png -------------------------------------------------------------------------------- /source/lessons/L6/img/L6-04-run-script.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L6/img/L6-04-run-script.png -------------------------------------------------------------------------------- /source/lessons/L6/img/L6-04-save-script-as.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L6/img/L6-04-save-script-as.png -------------------------------------------------------------------------------- /source/lessons/L6/img/Youtube_title_L6_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L6/img/Youtube_title_L6_1.png -------------------------------------------------------------------------------- /source/lessons/L6/img/Youtube_title_L6_2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L6/img/Youtube_title_L6_2.png -------------------------------------------------------------------------------- /source/lessons/L6/lecture.rst: -------------------------------------------------------------------------------- 1 | Lecture 2 | ======= 3 | 4 | .. admonition:: Lesson 6.1 - Introduction to Python in QGIS; Processing toolbox; Graphical modeller 5 | 6 | .. raw:: html 7 | 8 | 9 |

Christoph Fink / Henrikki Tenkanen, University of Helsinki @ AutoGIS channel on Youtube.

10 | 11 | 12 | .. admonition:: Lesson 6.2 - Python scripting in QGIS 3.0; Creating a processing toolbox with custom Python scripts; 13 | 14 | .. raw:: html 15 | 16 | 17 |

Christoph Fink / Henrikki Tenkanen, University of Helsinki @ AutoGIS channel on Youtube.

18 | -------------------------------------------------------------------------------- /source/lessons/L7/ex-7.rst: -------------------------------------------------------------------------------- 1 | Exercise 7 2 | ========== 3 | 4 | .. admonition:: Start your assignment 5 | 6 | You can start working on your copy of Exercise 7 by `accepting the GitHub Classroom assignment `__. 7 | 8 | **Exercise 7 is due by Monday 18.12**. 9 | 10 | You can also take a look at the open course copy of `Exercise 7 in the course GitHub repository `__ (does not require logging in). 11 | Note that you should not try to make changes to this copy of the exercise, but rather only to the copy available via GitHub Classroom. 12 | -------------------------------------------------------------------------------- /source/lessons/L7/exercise-7-hints.rst: -------------------------------------------------------------------------------- 1 | Exercise 7 hints 2 | ================ 3 | 4 | Add hints here if needed. -------------------------------------------------------------------------------- /source/lessons/L7/img/OSM_logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L7/img/OSM_logo.png -------------------------------------------------------------------------------- /source/lessons/L7/img/Youtube_title_L7_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L7/img/Youtube_title_L7_1.png -------------------------------------------------------------------------------- /source/lessons/L7/img/Youtube_title_L7_2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L7/img/Youtube_title_L7_2.png -------------------------------------------------------------------------------- /source/lessons/L7/img/Youtube_title_L7_3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/source/lessons/L7/img/Youtube_title_L7_3.png -------------------------------------------------------------------------------- /source/lessons/L7/lecture.rst: -------------------------------------------------------------------------------- 1 | Lecture 2 | ======= 3 | 4 | .. admonition:: Lesson 7.1 - Downloading and working with OpenStreetMap data in Python; Osmnx -package 5 | 6 | .. raw:: html 7 | 8 | 9 |

Henrikki Tenkanen, University of Helsinki @ AutoGIS channel on Youtube.

10 | 11 | 12 | .. admonition:: Lesson 7.2 - Network analysis and routing in Python; Osmnx and Networkx -packages 13 | 14 | .. raw:: html 15 | 16 | 17 |

Henrikki Tenkanen, University of Helsinki @ AutoGIS channel on Youtube.

18 | 19 | .. admonition:: Lesson 7.3 - Final assignment instructions 20 | 21 | .. raw:: html 22 | 23 | 24 |

Henrikki Tenkanen, University of Helsinki @ AutoGIS channel on Youtube.

25 | -------------------------------------------------------------------------------- /source/lessons/L7/overview.rst: -------------------------------------------------------------------------------- 1 | Overview 2 | ======== 3 | 4 | In this week we will see how to easily retrieve OpenStreetMap data and how to do simple 5 | network analysis such analyze different network statistics and calculate shortest paths along street network in Python. 6 | 7 | 1. :doc:`Retrieving OpenStreetMap data ` 8 | 2. :doc:`Network analysis in Python ` 9 | 3. :doc:`Exercise 7 ` 10 | 4. :doc:`Exercise 7 hints ` 11 | 12 | Learning goals 13 | -------------- 14 | 15 | After this week's lesson you should be able to: 16 | 17 | - Retrieve and save data from OpenStreetMap using Python 18 | - Extract simple street network properties and statistics (centrality measures etc.) 19 | - Do simple route optimization using shortest path algorithm in osmnx / networkx 20 | 21 | Sources 22 | ------- 23 | 24 | This week's lecture materials are based on documentation of `osmnx `__, `networkx `__ 25 | and `OpenStreetMap `__. -------------------------------------------------------------------------------- /source/lessons/L7/retrieve-osm-data.rst: -------------------------------------------------------------------------------- 1 | Retrieving OpenStreetMap data 2 | ============================= 3 | 4 | .. figure:: img/OSM_logo.png 5 | 6 | What is OpenStreetMap? 7 | ---------------------- 8 | 9 | OpenStreetMap (OSM) is a global collaborative (crowd-sourced) dataset and project that aims at creating a free editable map of the world 10 | containing a lot of information about our environment. 11 | It contains data for example about streets, buildings, different services, and landuse to mention a few. 12 | 13 | OSM has a large userbase with more than 4 million users that contribute actively on OSM by updating the OSM database 14 | with 3 million changesets per day. In total OSM contains more than 4 billion nodes that form the basis of the digitally 15 | mapped world that OSM provides (`stats from November 2017 `__. 16 | 17 | OpenStreetMap is used not only for integrating the **OSM maps** as background maps to visualizations or online maps, 18 | but also for many other purposes such as **routing**, **geocoding**, **education**, and **research**. OSM is also widely used for 19 | humanitarian response e.g. in crisis areas (e.g. after natural disasters) and for fostering economic development 20 | (see more from `Humanitarian OpenStreetMap Team (HOTOSM) website `__. 21 | 22 | Osmnx 23 | ----- 24 | 25 | This week we will explore a new and exciting Python module called `osmnx `__ 26 | that can be used to retrieve, construct, analyze, and visualize street networks from OpenStreetMap. 27 | In short it offers really handy functions to download data from OpenStreet map, analyze the properties of the OSM street networks, 28 | and conduct network routing based on walking, cycling or driving. 29 | 30 | There is also a scientific article available describing the package: 31 | 32 | - Boeing, G. 2017. `"OSMnx: New Methods for Acquiring, Constructing, Analyzing, and Visualizing Complex Street Networks." `__ Computers, Environment and Urban Systems 65, 126-139. doi:10.1016/j.compenvurbsys.2017.05.004 33 | 34 | Download and visualize OpenStreetMap data with osmnx 35 | ---------------------------------------------------- 36 | 37 | As said, one the most useful features that osmnx provides is an easy-to-use way of retrieving `OpenStreetMap `__ data 38 | (using `OverPass API `__ ). 39 | 40 | Let's see how we can download and visualize street network data from a district of Kamppi in Helsinki, Finland. 41 | Osmnx makes it really easy to do that as it allows you to specify an address to retrieve the OpenStreetMap data around that area. 42 | In fact, osmnx uses the same Nominatim Geocoding API to achieve this which we tested during the Lesson 2. 43 | 44 | - Let's retrieve OpenStreetMap (OSM) data by specifying ``"Kamppi, Helsinki, Finland"`` as the address where the data should be downloaded. 45 | 46 | .. ipython:: python 47 | 48 | import osmnx as ox 49 | import matplotlib.pyplot as plt 50 | place_name = "Kamppi, Helsinki, Finland" 51 | graph = ox.graph_from_place(place_name) 52 | type(graph) 53 | 54 | Okey, as we can see the data that we retrieved is a special data object called ``networkx.classes.multidigraph.MultiDiGraph``. A DiGraph is a data type that stores nodes and edges with optional data, or attributes. 55 | What we can see here is that this data type belongs to a Python module called `networkx `__ 56 | that can be used to create, manipulate, and study the structure, dynamics, and functions of complex networks. 57 | Networkx module contains algorithms that can be used to calculate `shortest paths `__ 58 | along networks using e.g. `Dijkstra's `__ or `A* algorithm `__. 59 | 60 | - Let's see how our street network looks like. It is easy to visualize the graph with osmnx with ``plot_graph()`` function. The function utilizes Matplotlib for visualizing the data, 61 | hence as a result it returns a matplotlib figure and axis objects. 62 | 63 | .. ipython:: python 64 | 65 | fig, ax = ox.plot_graph(graph) 66 | @savefig graph.png width=5in 67 | plt.tight_layout() 68 | 69 | Great! Now we can see that our graph contains the nodes (blue circles) and the edges (gray lines) that connects those nodes to each other. 70 | 71 | It is also possible to retrieve other types of OSM data features with osmnx. 72 | 73 | - Let's download the buildings with ``buildings_from_place()`` function and plot them on top of our street network in Kamppi. Let's also plot the Polygon that represents the area of Kamppi, 74 | Helsinki that can be retrieved with ``gdf_from_place`` function. 75 | 76 | .. ipython:: python 77 | 78 | area = ox.gdf_from_place(place_name) 79 | buildings = ox.buildings_from_place(place_name) 80 | type(area) 81 | type(buildings) 82 | 83 | As a result we got the data as GeoDataFrames. Hence, we can plot them using the familiar ``plot()`` function of Geopandas. 84 | As you might remember the street network data was not in GeoDataFrame. Luckily, osmnx provides a convenient function ``graph_to_gdfs()`` 85 | that can convert the graph into two separate GeoDataFrames where the first one contains the information about the nodes and the second one 86 | about the edges. 87 | 88 | - Let's extract the nodes and edges from the graph as GeoDataFrames. 89 | 90 | .. ipython:: python 91 | 92 | nodes, edges = ox.graph_to_gdfs(graph) 93 | nodes.head() 94 | edges.head() 95 | type(edges) 96 | 97 | Nice! Now, as we can see, we have our graph as GeoDataFrames and we can plot them using the same functions and tools as we have used before. 98 | 99 | .. note:: 100 | 101 | There are also other ways of retrieving the data from OpenStreetMap with osmnx such as passing a Polygon to extract the data from that area, 102 | or passing a Point coordinates and retrieving data around that location with specific radius. 103 | Take a look of this `tutorial to find out how to use those features of osmnx `__. 104 | 105 | - Let's create a map out of the streets, buildings, and the area Polygon but let's exclude the nodes (to keep the figure clearer). 106 | 107 | .. ipython:: python 108 | 109 | fig, ax = plt.subplots() 110 | area.plot(ax=ax, facecolor='black') 111 | edges.plot(ax=ax, linewidth=1, edgecolor='#BC8F8F') 112 | buildings.plot(ax=ax, facecolor='khaki', alpha=0.7) 113 | @savefig city_plot.png width=5in 114 | plt.tight_layout() 115 | 116 | Cool! Now we have a map where we have plotted the buildings, streets and the boundaries of the selected region of 'Kamppi' in Helsinki. 117 | And all of this required only a few lines of code. Pretty neat! Next, we will start exploring how we can use OSM 118 | data to do network analysis. 119 | 120 | .. todo:: 121 | 122 | **Task** 123 | Column ``highway`` in our ``edges`` GeoDataFrame contains information about the type of the street (such as ``primacy, cycleway or footway``). 124 | Select the streets that are walkable or that can be used with cycle and visualize only them with the buildings and the area polygon. Use different colors and line widths for the cycleways and footways. 125 | 126 | .. hint:: 127 | 128 | There are a few nice and convenient high-level functions in `osmnx `__ that can be used to produce nice maps directly just by using a single function that might be useful. 129 | If you are interested take a look of `this tutorial `__. 130 | In the lesson we won't cover these, because we wanted to keep as much control to ourselves as possible, hence using lower-level functions. 131 | 132 | 133 | 134 | 135 | -------------------------------------------------------------------------------- /wgs84.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Automating-GIS-processes/2017/8e4efd34edd79ece4db98e11af1d6a070edf1cb5/wgs84.png --------------------------------------------------------------------------------