├── .gitignore ├── AUTHORS ├── CITATION ├── CONDUCT.md ├── CONTRIBUTING.md ├── LICENSE.md ├── Makefile ├── README.md ├── _config.yml ├── _episodes ├── .gitkeep ├── 01-basics.md ├── 02-flow.md ├── 03-lib.md ├── 04-coffee.md ├── 05-functions.md ├── 06-dict.md ├── 07-lunch.md ├── 08-numpy.md ├── 09-pandas.md ├── 10-coffee.md ├── 11-plotting.md ├── 12-file-io.md ├── 13-style.md ├── 14-testing.md └── 15-wrap.md ├── _episodes_rmd ├── .gitkeep └── data │ └── .gitkeep ├── _extras ├── .gitkeep ├── about.md ├── cmdline.md ├── design.md ├── discuss.md ├── figures.md ├── guide.md ├── profiling.md └── web-data.md ├── _includes ├── all_figures.html ├── all_keypoints.html ├── carpentries.html ├── episode_break.html ├── episode_keypoints.html ├── episode_navbar.html ├── episode_overview.html ├── episode_title.html ├── javascript.html ├── lesson_footer.html ├── main_title.html ├── navbar.html ├── syllabus.html ├── workshop_ad.html └── workshop_footer.html ├── _layouts ├── base.html ├── break.html ├── episode.html ├── lesson.html ├── page.html ├── reference.html └── workshop.html ├── assets ├── css │ ├── bootstrap-theme.css │ ├── bootstrap-theme.css.map │ ├── bootstrap.css │ ├── bootstrap.css.map │ └── lesson.scss ├── fonts │ ├── glyphicons-halflings-regular.eot │ ├── glyphicons-halflings-regular.svg │ ├── glyphicons-halflings-regular.ttf │ ├── glyphicons-halflings-regular.woff │ └── glyphicons-halflings-regular.woff2 ├── img │ ├── dc-icon-black.svg │ ├── swc-icon-blue.svg │ ├── swc-logo-blue.png │ ├── swc-logo-blue.svg │ ├── swc-logo-white.png │ └── swc-logo-white.svg └── js │ ├── bootstrap.min.js │ ├── jquery.min.js │ └── lesson.js ├── bin ├── chunk-options.R ├── extract_figures.py ├── generate_md_episodes.R ├── knit_lessons.sh ├── lesson_check.py ├── lesson_initialize.py ├── markdown_ast.rb ├── repo_check.py ├── test_lesson_check.py ├── util.py └── workshop_check.py ├── code ├── .gitkeep ├── blocks.ipynb ├── dictionaries.ipynb ├── images.ipynb ├── morning.ipynb ├── numpy.ipynb ├── pandas.ipynb ├── percolation.ipynb ├── performance.ipynb ├── profiling-and-optimizing.ipynb ├── profiling.ipynb ├── profiling_pi.py └── testing.ipynb ├── data ├── .gitkeep ├── atmo_CH4.csv ├── atmo_CO2.csv ├── atmo_N2O.csv ├── halogas.csv └── temperature.csv ├── favicon-dc.ico ├── favicon-swc.ico ├── fig ├── .gitkeep ├── images_12_1.png ├── images_14_1.png ├── images_3_1.png ├── images_4_1.png ├── images_6_1.png ├── images_7_1.png ├── numpy_11_1.png ├── numpy_16_1.png ├── numpy_17_1.png ├── numpy_18_1.png ├── numpy_19_1.png ├── numpy_20_1.png ├── numpy_23_1.png ├── numpy_24_1.png ├── numpy_26_1.png ├── numpy_9_0.png ├── pandas_10_1.png ├── pandas_14_1.png ├── pandas_19_1.png ├── pandas_21_2.png ├── pandas_6_1.png └── pandas_8_1.png ├── files ├── .gitkeep └── python-second-language-data.zip ├── index.md ├── reference.md └── setup.md /.gitignore: -------------------------------------------------------------------------------- 1 | *.pyc 2 | *~ 3 | .DS_Store 4 | .ipynb_checkpoints 5 | .sass-cache 6 | __pycache__ 7 | _site 8 | -------------------------------------------------------------------------------- /AUTHORS: -------------------------------------------------------------------------------- 1 | Joseph Montoya 2 | Greg Wilson 3 | -------------------------------------------------------------------------------- /CITATION: -------------------------------------------------------------------------------- 1 | FIXME: describe how to cite this lesson. 2 | -------------------------------------------------------------------------------- /CONDUCT.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: page 3 | title: "Contributor Code of Conduct" 4 | permalink: /conduct/ 5 | --- 6 | As contributors and maintainers of this project, 7 | we pledge to respect all people who contribute through reporting issues, 8 | posting feature requests, 9 | updating documentation, 10 | submitting pull requests or patches, 11 | and other activities. 12 | 13 | We are committed to making participation in this project a harassment-free experience for everyone, 14 | regardless of level of experience, 15 | gender, 16 | gender identity and expression, 17 | sexual orientation, 18 | disability, 19 | personal appearance, 20 | body size, 21 | race, 22 | ethnicity, 23 | age, 24 | or religion. 25 | 26 | Examples of unacceptable behavior by participants include the use of sexual language or imagery, 27 | derogatory comments or personal attacks, 28 | trolling, 29 | public or private harassment, 30 | insults, 31 | or other unprofessional conduct. 32 | 33 | Project maintainers have the right and responsibility to remove, edit, or reject 34 | comments, commits, code, wiki edits, issues, and other contributions 35 | that are not aligned to this Code of Conduct. 36 | Project maintainers who do not follow the Code of Conduct may be removed from the project team. 37 | 38 | Instances of abusive, harassing, or otherwise unacceptable behavior 39 | may be reported by opening an issue or contacting one or more of the project maintainers. 40 | 41 | This Code of Conduct is adapted from 42 | the [Contributor Covenant][contrib-covenant] Version 1.0.0. 43 | 44 | [contrib-covenant]: http://contributor-covenant.org/ 45 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing 2 | 3 | [Software Carpentry][swc-site] and [Data Carpentry][dc-site] are open source projects, 4 | and we welcome contributions of all kinds: 5 | new lessons, 6 | fixes to existing material, 7 | bug reports, 8 | and reviews of proposed changes are all welcome. 9 | 10 | ## Contributor Agreement 11 | 12 | By contributing, 13 | you agree that we may redistribute your work under [our license](LICENSE.md). 14 | In exchange, 15 | we will address your issues and/or assess your change proposal as promptly as we can, 16 | and help you become a member of our community. 17 | Everyone involved in [Software Carpentry][swc-site] and [Data Carpentry][dc-site] 18 | agrees to abide by our [code of conduct](CONDUCT.md). 19 | 20 | ## How to Contribute 21 | 22 | The easiest way to get started is to file an issue 23 | to tell us about a spelling mistake, 24 | some awkward wording, 25 | or a factual error. 26 | This is a good way to introduce yourself 27 | and to meet some of our community members. 28 | 29 | 1. If you do not have a [GitHub][github] account, 30 | you can [send us comments by email][contact]. 31 | However, 32 | we will be able to respond more quickly if you use one of the other methods described below. 33 | 34 | 2. If you have a [GitHub][github] account, 35 | or are willing to [create one][github-join], 36 | but do not know how to use Git, 37 | you can report problems or suggest improvements by [creating an issue][issues]. 38 | This allows us to assign the item to someone 39 | and to respond to it in a threaded discussion. 40 | 41 | 3. If you are comfortable with Git, 42 | and would like to add or change material, 43 | you can submit a pull request (PR). 44 | Instructions for doing this are [included below](#using-github). 45 | 46 | ## Where to Contribute 47 | 48 | 1. If you wish to change this lesson, 49 | please work in , 50 | which can be viewed at . 51 | 52 | 2. If you wish to change the example lesson, 53 | please work in , 54 | which documents the format of our lessons 55 | and can be viewed at . 56 | 57 | 3. If you wish to change the template used for workshop websites, 58 | please work in . 59 | The home page of that repository explains how to set up workshop websites, 60 | while the extra pages in 61 | provide more background on our design choices. 62 | 63 | 4. If you wish to change CSS style files, tools, 64 | or HTML boilerplate for lessons or workshops stored in `_includes` or `_layouts`, 65 | please work in . 66 | 67 | ## What to Contribute 68 | 69 | There are many ways to contribute, 70 | from writing new exercises and improving existing ones 71 | to updating or filling in the documentation 72 | and and submitting [bug reports][issues] 73 | about things that don't work, aren't clear, or are missing. 74 | If you are looking for ideas, 75 | please see [the list of issues for this repository][issues], 76 | or the issues for [Data Carpentry][dc-issues] 77 | and [Software Carpentry][swc-issues] projects. 78 | 79 | Comments on issues and reviews of pull requests are just as welcome: 80 | we are smarter together than we are on our own. 81 | Reviews from novices and newcomers are particularly valuable: 82 | it's easy for people who have been using these lessons for a while 83 | to forget how impenetrable some of this material can be, 84 | so fresh eyes are always welcome. 85 | 86 | ## What *Not* to Contribute 87 | 88 | Our lessons already contain more material than we can cover in a typical workshop, 89 | so we are usually *not* looking for more concepts or tools to add to them. 90 | As a rule, 91 | if you want to introduce a new idea, 92 | you must (a) estimate how long it will take to teach 93 | and (b) explain what you would take out to make room for it. 94 | The first encourages contributors to be honest about requirements; 95 | the second, to think hard about priorities. 96 | 97 | We are also not looking for exercises or other material that only run on one platform. 98 | Our workshops typically contain a mixture of Windows, Mac OS X, and Linux users; 99 | in order to be usable, 100 | our lessons must run equally well on all three. 101 | 102 | ## Using GitHub 103 | 104 | If you choose to contribute via GitHub, 105 | you may want to look at 106 | [How to Contribute to an Open Source Project on GitHub][how-contribute]. 107 | In brief: 108 | 109 | 1. The published copy of the lesson is in the `gh-pages` branch of the repository 110 | (so that GitHub will regenerate it automatically). 111 | Please create all branches from that, 112 | and merge the [master repository][repo]'s `gh-pages` branch into your `gh-pages` branch 113 | before starting work. 114 | Please do *not* work directly in your `gh-pages` branch, 115 | since that will make it difficult for you to work on other contributions. 116 | 117 | 2. We use [GitHub flow][github-flow] to manage changes: 118 | 1. Create a new branch in your desktop copy of this repository for each significant change. 119 | 2. Commit the change in that branch. 120 | 3. Push that branch to your fork of this repository on GitHub. 121 | 4. Submit a pull request from that branch to the [master repository][repo]. 122 | 5. If you receive feedback, 123 | make changes on your desktop and push to your branch on GitHub: 124 | the pull request will update automatically. 125 | 126 | Each lesson has two maintainers who review issues and pull requests 127 | or encourage others to do so. 128 | The maintainers are community volunteers, 129 | and have final say over what gets merged into the lesson. 130 | 131 | ## Other Resources 132 | 133 | General discussion of [Software Carpentry][swc-site] and [Data Carpentry][dc-site] 134 | happens on the [discussion mailing list][discuss-list], 135 | which everyone is welcome to join. 136 | You can also [reach us by email][contact]. 137 | 138 | [contact]: mailto:admin@software-carpentry.org 139 | [dc-issues]: https://github.com/issues?q=user%3Adatacarpentry 140 | [dc-lessons]: http://datacarpentry.org/lessons/ 141 | [dc-site]: http://datacarpentry.org/ 142 | [discuss-list]: http://lists.software-carpentry.org/listinfo/discuss 143 | [example-site]: https://swcarpentry.github.io/lesson-example/ 144 | [github]: http://github.com 145 | [github-flow]: https://guides.github.com/introduction/flow/ 146 | [github-join]: https://github.com/join 147 | [how-contribute]: https://egghead.io/series/how-to-contribute-to-an-open-source-project-on-github 148 | [issues]: https://github.com/swcarpentry/python-second-language/issues/ 149 | [repo]: https://github.com/swcarpentry/python-second-language/ 150 | [swc-issues]: https://github.com/issues?q=user%3Aswcarpentry 151 | [swc-lessons]: http://software-carpentry.org/lessons/ 152 | [swc-site]: http://software-carpentry.org/ 153 | -------------------------------------------------------------------------------- /LICENSE.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: page 3 | title: "Licenses" 4 | permalink: /license/ 5 | --- 6 | ## Instructional Material 7 | 8 | All Software Carpentry and Data Carpentry instructional material is 9 | made available under the [Creative Commons Attribution 10 | license][cc-by-human]. The following is a human-readable summary of 11 | (and not a substitute for) the [full legal text of the CC BY 4.0 12 | license][cc-by-legal]. 13 | 14 | You are free: 15 | 16 | * to **Share**---copy and redistribute the material in any medium or format 17 | * to **Adapt**---remix, transform, and build upon the material 18 | 19 | for any purpose, even commercially. 20 | 21 | The licensor cannot revoke these freedoms as long as you follow the 22 | license terms. 23 | 24 | Under the following terms: 25 | 26 | * **Attribution**---You must give appropriate credit (mentioning that 27 | your work is derived from work that is Copyright © Software 28 | Carpentry and, where practical, linking to 29 | http://software-carpentry.org/), provide a [link to the 30 | license][cc-by-human], and indicate if changes were made. You may do 31 | so in any reasonable manner, but not in any way that suggests the 32 | licensor endorses you or your use. 33 | 34 | **No additional restrictions**---You may not apply legal terms or 35 | technological measures that legally restrict others from doing 36 | anything the license permits. With the understanding that: 37 | 38 | Notices: 39 | 40 | * You do not have to comply with the license for elements of the 41 | material in the public domain or where your use is permitted by an 42 | applicable exception or limitation. 43 | * No warranties are given. The license may not give you all of the 44 | permissions necessary for your intended use. For example, other 45 | rights such as publicity, privacy, or moral rights may limit how you 46 | use the material. 47 | 48 | ## Software 49 | 50 | Except where otherwise noted, the example programs and other software 51 | provided by Software Carpentry and Data Carpentry are made available under the 52 | [OSI][osi]-approved 53 | [MIT license][mit-license]. 54 | 55 | Permission is hereby granted, free of charge, to any person obtaining 56 | a copy of this software and associated documentation files (the 57 | "Software"), to deal in the Software without restriction, including 58 | without limitation the rights to use, copy, modify, merge, publish, 59 | distribute, sublicense, and/or sell copies of the Software, and to 60 | permit persons to whom the Software is furnished to do so, subject to 61 | the following conditions: 62 | 63 | The above copyright notice and this permission notice shall be 64 | included in all copies or substantial portions of the Software. 65 | 66 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 67 | EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 68 | MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 69 | NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE 70 | LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION 71 | OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION 72 | WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 73 | 74 | ## Trademark 75 | 76 | "Software Carpentry" an "Data Carpentry" and their respective logos 77 | are registered trademarks of [NumFOCUS][numfocus]. 78 | 79 | [cc-by-human]: https://creativecommons.org/licenses/by/4.0/ 80 | [cc-by-legal]: https://creativecommons.org/licenses/by/4.0/legalcode 81 | [mit-license]: http://opensource.org/licenses/mit-license.html 82 | [numfocus]: http://numfocus.org/ 83 | [osi]: http://opensource.org 84 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | ## ======================================== 2 | ## Commands for both workshop and lesson websites. 3 | 4 | # Settings 5 | MAKEFILES=Makefile $(wildcard *.mk) 6 | JEKYLL=jekyll 7 | PARSER=bin/markdown_ast.rb 8 | DST=_site 9 | 10 | # Controls 11 | .PHONY : commands clean files 12 | all : commands 13 | 14 | ## commands : show all commands. 15 | commands : 16 | @grep -h -E '^##' ${MAKEFILES} | sed -e 's/## //g' 17 | 18 | ## serve : run a local server. 19 | serve : lesson-rmd 20 | ${JEKYLL} serve 21 | 22 | ## site : build files but do not run a server. 23 | site : lesson-rmd 24 | ${JEKYLL} build 25 | 26 | # repo-check : check repository settings. 27 | repo-check : 28 | @bin/repo_check.py -s . 29 | 30 | ## clean : clean up junk files. 31 | clean : 32 | @rm -rf ${DST} 33 | @rm -rf .sass-cache 34 | @rm -rf bin/__pycache__ 35 | @find . -name .DS_Store -exec rm {} \; 36 | @find . -name '*~' -exec rm {} \; 37 | @find . -name '*.pyc' -exec rm {} \; 38 | 39 | ## clean-rmd : clean intermediate R files (that need to be committed to the repo). 40 | clear-rmd : 41 | @rm -rf ${RMD_DST} 42 | @rm -rf fig/rmd-* 43 | 44 | ## ---------------------------------------- 45 | ## Commands specific to workshop websites. 46 | 47 | .PHONY : workshop-check 48 | 49 | ## workshop-check : check workshop homepage. 50 | workshop-check : 51 | @bin/workshop_check.py . 52 | 53 | ## ---------------------------------------- 54 | ## Commands specific to lesson websites. 55 | 56 | .PHONY : lesson-check lesson-rmd lesson-files lesson-fixme 57 | 58 | # RMarkdown files 59 | RMD_SRC = $(wildcard _episodes_rmd/??-*.Rmd) 60 | RMD_DST = $(patsubst _episodes_rmd/%.Rmd,_episodes/%.md,$(RMD_SRC)) 61 | 62 | # Lesson source files in the order they appear in the navigation menu. 63 | MARKDOWN_SRC = \ 64 | index.md \ 65 | CONDUCT.md \ 66 | setup.md \ 67 | $(wildcard _episodes/*.md) \ 68 | reference.md \ 69 | $(wildcard _extras/*.md) \ 70 | LICENSE.md 71 | 72 | # Generated lesson files in the order they appear in the navigation menu. 73 | HTML_DST = \ 74 | ${DST}/index.html \ 75 | ${DST}/conduct/index.html \ 76 | ${DST}/setup/index.html \ 77 | $(patsubst _episodes/%.md,${DST}/%/index.html,$(wildcard _episodes/*.md)) \ 78 | ${DST}/reference/index.html \ 79 | $(patsubst _extras/%.md,${DST}/%/index.html,$(wildcard _extras/*.md)) \ 80 | ${DST}/license/index.html 81 | 82 | ## lesson-rmd : convert Rmarkdown files to markdown 83 | lesson-rmd: $(RMD_SRC) 84 | @bin/knit_lessons.sh $(RMD_SRC) 85 | 86 | ## lesson-check : validate lesson Markdown. 87 | lesson-check : 88 | @bin/lesson_check.py -s . -p ${PARSER} 89 | 90 | ## lesson-check-all : validate lesson Markdown, checking line lengths and trailing whitespace. 91 | lesson-check-all : 92 | @bin/lesson_check.py -s . -p ${PARSER} -l -w 93 | 94 | ## lesson-figures : re-generate inclusion displaying all figures. 95 | lesson-figures : 96 | @bin/extract_figures.py -p ${PARSER} ${MARKDOWN_SRC} > _includes/all_figures.html 97 | 98 | ## unittest : run unit tests on checking tools. 99 | unittest : 100 | python bin/test_lesson_check.py 101 | 102 | ## lesson-files : show expected names of generated files for debugging. 103 | lesson-files : 104 | @echo 'RMD_SRC:' ${RMD_SRC} 105 | @echo 'RMD_DST:' ${RMD_DST} 106 | @echo 'MARKDOWN_SRC:' ${MARKDOWN_SRC} 107 | @echo 'HTML_DST:' ${HTML_DST} 108 | 109 | ## lesson-fixme : show FIXME markers embedded in source files. 110 | lesson-fixme : 111 | @fgrep -i -n FIXME ${MARKDOWN_SRC} || true 112 | 113 | #------------------------------------------------------------------------------- 114 | # Include extra commands if available. 115 | #------------------------------------------------------------------------------- 116 | 117 | -include commands.mk 118 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | python-second-language 2 | ====================== 3 | 4 | An introduction to Python as a second language. 5 | Please see 6 | for a rendered version of this material, 7 | [the lesson template documentation][lesson-example] 8 | for instructions on formatting, building, and submitting material, 9 | or run `make` in this directory for a list of helpful commands. 10 | 11 | Maintainer(s): 12 | 13 | * [Greg Wilson][wilson-greg] 14 | 15 | [lesson-example]: https://swcarpentry.github.com/lesson-example/ 16 | [wilson-greg]: http://software-carpentry.org/team/#wilson_g 17 | -------------------------------------------------------------------------------- /_config.yml: -------------------------------------------------------------------------------- 1 | #------------------------------------------------------------ 2 | # Values for this lesson. 3 | #------------------------------------------------------------ 4 | 5 | # Which carpentry is this ("swc" or "dc")? 6 | carpentry: "swc" 7 | 8 | # Overall title for pages. 9 | title: "Python as a Second Language" 10 | 11 | # Contact email address. 12 | email: lessons@software-carpentry.org 13 | 14 | #------------------------------------------------------------ 15 | # Generic settings (should not need to change). 16 | #------------------------------------------------------------ 17 | 18 | # What kind of thing is this ("workshop" or "lesson")? 19 | kind: "lesson" 20 | 21 | # Magic to make URLs resolve both locally and on GitHub. 22 | # See https://help.github.com/articles/repository-metadata-on-github-pages/. 23 | repository: / 24 | 25 | # Sites. 26 | amy_site: "https://amy.software-carpentry.org/workshops" 27 | dc_site: "http://datacarpentry.org" 28 | swc_github: "https://github.com/swcarpentry" 29 | swc_site: "https://software-carpentry.org" 30 | swc_pages: "https://swcarpentry.github.io" 31 | template_repo: "https://github.com/swcarpentry/styles" 32 | example_repo: "https://github.com/swcarpentry/lesson-example" 33 | example_site: "https://swcarpentry.github.com/lesson-example" 34 | workshop_repo: "https://github.com/swcarpentry/workshop-template" 35 | workshop_site: "https://swcarpentry.github.io/workshop-template" 36 | training_site: "https://swcarpentry.github.io/instructor-training" 37 | 38 | # Surveys. 39 | pre_survey: "https://www.surveymonkey.com/r/swc_pre_workshop_v1?workshop_id=" 40 | post_survey: "https://www.surveymonkey.com/r/swc_post_workshop_v1?workshop_id=" 41 | 42 | # Start time in minutes (0 to be clock-independent, 540 to show a start at 09:00 am) 43 | start_time: 540 44 | 45 | # Specify that things in the episodes collection should be output. 46 | collections: 47 | episodes: 48 | output: true 49 | permalink: /:path/ 50 | extras: 51 | output: true 52 | 53 | # Set the default layout for things in the episodes collection. 54 | defaults: 55 | - values: 56 | root: .. 57 | - scope: 58 | path: "" 59 | type: episodes 60 | values: 61 | layout: episode 62 | 63 | # Files and directories that are not to be copied. 64 | exclude: 65 | - Makefile 66 | - bin 67 | 68 | # Turn off built-in syntax highlighting. 69 | highlighter: false 70 | -------------------------------------------------------------------------------- /_episodes/.gitkeep: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/swcarpentry/python-second-language/074c1d9aa7424104e8a5006623e91eeb6545bbf5/_episodes/.gitkeep -------------------------------------------------------------------------------- /_episodes/02-flow.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Control Flow" 3 | teaching: 10 4 | exercises: 20 5 | questions: 6 | - "How do I repeat operations?" 7 | - "How do I make decisions?" 8 | - "How do I call built-in functions?" 9 | objectives: 10 | - "Write scripts that use `for` loops to iterate over lists and character strings." 11 | - "Write scripts that use `if`/`elif`/`else` to perform conditional operations." 12 | - "Call built-in functions." 13 | - "Call methods on strings and lists." 14 | - "Use online help to inspect functions' documentation." 15 | - "Use `range` and `for` to iterate over a sequence of numbers." 16 | - "Correctly write programs that use if and else statements and simple Boolean expressions (without logical operators)." 17 | - "Trace the execution of unnested conditionals and conditionals inside loops." 18 | keypoints: 19 | - "Repeat actions for each element in a collection with `for` loops." 20 | - "Use `range` to generate a list of numbers." 21 | - "Use `if`/`elif`/`else` to make choices." 22 | - "Use built-in functions like `len` and `max` to do calculations." 23 | - "Objects like strings and lists have methods that operate on them." 24 | - "Use `if` statements to control whether or not a block of code is executed." 25 | - "Conditionals are often used inside loops." 26 | - "Use `else` to execute a block of code when an `if` condition is *not* true." 27 | - "Use `elif` to specify additional tests." 28 | - "Conditions are tested once, in order." 29 | - "Create a table showing variables' values to trace a program's execution." 30 | --- 31 | ## A *for loop* executes commands once for each value in a collection. 32 | 33 | * `for` loops give items rather than indices. 34 | * Because that's what people usually want. 35 | * "for each thing in this group, do these operations" 36 | 37 | ~~~ 38 | for number in [2, 3, 5]: 39 | print(number) 40 | ~~~ 41 | {: .python} 42 | ~~~ 43 | 2 44 | 3 45 | 5 46 | ~~~ 47 | {: output} 48 | 49 | ## The first line of the `for` loop must end with a colon, and the body must be indented. 50 | 51 | * The colon at the end of the first line signals the start of a block. 52 | * Python uses indentation rather than `{}` or `begin`/`end` to show *nesting*. 53 | * Any consistent indentation is legal, but almost everyone uses four spaces. 54 | 55 | ~~~ 56 | for number in [2, 3, 5]: 57 | print(number) 58 | ~~~ 59 | {: .python} 60 | ~~~ 61 | IndentationError: expected an indented block 62 | ~~~ 63 | {: .error} 64 | 65 | * Indentation is *always* meaningful in Python. 66 | 67 | ## Use `range` to iterate over a sequence of numbers. 68 | 69 | * The built-in function `range` produces a sequence of numbers. 70 | * *Not* a list: the numbers are produced on demand 71 | to make looping over large ranges more efficient. 72 | * `range(N)` is the numbers 0..N-1 73 | 74 | ~~~ 75 | print('a range is not a list: range(0, 3)') 76 | for number in range(0,3): 77 | print(number) 78 | ~~~ 79 | {: .python} 80 | ~~~ 81 | a range is not a list: range(0, 3) 82 | 0 83 | 1 84 | 2 85 | ~~~ 86 | {: .output} 87 | 88 | ## Use `if` statements to control whether or not a block of code is executed. 89 | 90 | * Structure is similar to a `for` statement. 91 | 92 | ~~~ 93 | masses = [3.54, 2.07, 9.22, 1.86, 1.71] 94 | for m in masses: 95 | if mass > 3.0: 96 | print(mass, 'is large') 97 | ~~~ 98 | {: .python} 99 | ~~~ 100 | 3.54 is large 101 | 9.22 is large 102 | ~~~ 103 | {: .output} 104 | 105 | ## Use `else` to execute a block of code when an `if` condition is *not* true. 106 | 107 | ~~~ 108 | masses = [3.54, 2.07, 9.22, 1.86, 1.71] 109 | for m in masses: 110 | if mass > 3.0: 111 | print(mass, 'is large') 112 | else: 113 | print(mass, 'is small') 114 | ~~~ 115 | {: .python} 116 | ~~~ 117 | 3.54 is large 118 | 2.07 is small 119 | 9.22 is large 120 | 1.86 is small 121 | 1.71 is small 122 | ~~~ 123 | {: .output} 124 | 125 | ## Use `elif` to specify additional tests. 126 | 127 | ~~~ 128 | masses = [3.54, 2.07, 9.22, 1.86, 1.71] 129 | for m in masses: 130 | if mass > 9.0: 131 | print(mass, 'is HUGE') 132 | elif mass > 3.0: 133 | print(mass, 'is large') 134 | else: 135 | print(mass, 'is small') 136 | ~~~ 137 | {: .python} 138 | ~~~ 139 | 3.54 is large 140 | 2.07 is small 141 | 9.22 is HUGE 142 | 1.86 is small 143 | 1.71 is small 144 | ~~~ 145 | {: .output} 146 | 147 | > ## Compound Relations Using `and`, `or`, and Parentheses 148 | > 149 | > Often, you want some combination of things to be true. You can combine 150 | > relations within a conditional using `and` and `or`. Continuing the example 151 | > above, suppose you have 152 | > 153 | > ~~~ 154 | > mass = [ 3.54, 2.07, 9.22, 1.86, 1.71] 155 | > velocity = [10.00, 20.00, 30.00, 25.00, 20.00] 156 | > 157 | > i = 0 158 | > for i in range(5): 159 | > if mass[i] > 5 and velocity[i] > 20: 160 | > print "Fast heavy object. Duck!" 161 | > elif mass[i] > 2 and mass[i] <= 5 and velocity[i] <= 20: 162 | > print "Normal traffic" 163 | > elif mass[i] <= 2 and velocity <= 20: 164 | > print "Slow light object. Ignore it" 165 | > else: 166 | > print "Whoa! Something is up with the data. Check it" 167 | > ~~~ 168 | > {: .python} 169 | > 170 | > Just like with arithmetic, you can and should use parentheses whenever there 171 | > is possible ambiguity. A good general rule is to *always* use parentheses 172 | > when mixing `and` and `or` in the same condition. That is, instead of: 173 | > 174 | > ~~~ 175 | > if mass[i] <= 2 or mass[i] >= 5 and velocity[i] > 20: 176 | > ~~~ 177 | > {: .python} 178 | > 179 | > write one of these: 180 | > 181 | > ~~~ 182 | > if (mass[i] <= 2 or mass[i] >= 5) and velocity[i] > 20: 183 | > if mass[i] <= 2 or (mass[i] >= 5 and velocity[i] > 20): 184 | > ~~~ 185 | > {: .python} 186 | > 187 | > so it is perfectly clear to a reader (and to Python) what you really mean. 188 | > 189 | > One more thing: in Python, comparisons can be chained; for example, 190 | > 191 | > ~~~ 192 | > 2 < mass[i] <= 5 193 | > ~~~ 194 | > {: .python} 195 | > 196 | > is equivalent to 197 | > 198 | > ~~~ 199 | > 2 < mass[i] and mass[i] <= 5 200 | > ~~~ 201 | > {: .python} 202 | {: .callout} 203 | 204 | > ## Reversing a String 205 | > 206 | > Fill in the blanks in the program below so that it prints "nit" 207 | > (the reverse of the original character string "tin"). 208 | > 209 | > ~~~ 210 | > original = "tin" 211 | > result = ____ 212 | > for char in original: 213 | > result = ____ 214 | > print(result) 215 | > ~~~ 216 | > {: .python} 217 | {: .challenge} 218 | 219 | > ## Accumulating 220 | > 221 | > Fill in the blanks in each of the programs below 222 | > to produce the indicated result. 223 | > 224 | > ~~~ 225 | > # Total length of the strings in the list: ["red", "green", "blue"] => 12 226 | > total = 0 227 | > for word in ["red", "green", "blue"]: 228 | > ____ = ____ + len(word) 229 | > print(total) 230 | > ~~~ 231 | > {: .python} 232 | > 233 | > ~~~ 234 | > # List of word lengths: ["red", "green", "blue"] => [3, 5, 4] 235 | > lengths = ____ 236 | > for word in ["red", "green", "blue"]: 237 | > lengths = lengths.____(____) 238 | > print(lengths) 239 | > ~~~ 240 | > {: .python} 241 | > 242 | > ~~~ 243 | > # Concatenate all words: ["red", "green", "blue"] => "redgreenblue" 244 | > words = ["red", "green", "blue"] 245 | > result = ____ 246 | > for ____ in ____: 247 | > ____ 248 | > print(result) 249 | > ~~~~ 250 | > {: .python} 251 | > 252 | > ~~~ 253 | > # Create acronym: ["red", "green", "blue"] => "RGB" 254 | > # write the whole thing 255 | > ~~~ 256 | > {: .python} 257 | {: .challenge} 258 | 259 | > ## Cumulative Sum 260 | > 261 | > Reorder and properly indent the lines of code below 262 | > so that they print an array with the cumulative sum of data. 263 | > The result should be `[1, 3, 5, 10]`. 264 | > 265 | > ~~~ 266 | > cumulative += [sum] 267 | > for number in data: 268 | > cumulative = [] 269 | > sum += number 270 | > print(cumulative) 271 | > data = [1,2,2,5] 272 | > ~~~ 273 | > {: .python} 274 | {: .challenge} 275 | 276 | > ## Identifying Variable Name Errors 277 | > 278 | > 1. Read the code below and try to identify what the errors are 279 | > *without* running it. 280 | > 2. Run the code and read the error message. 281 | > What type of `NameError` do you think this is? 282 | > Is it a string with no quotes, a misspelled variable, or a 283 | > variable that should have been defined but was not? 284 | > 3. Fix the error. 285 | > 4. Repeat steps 2 and 3, until you have fixed all the errors. 286 | > 287 | > ~~~ 288 | > for number in range(10): 289 | > # use a if the number is a multiple of 3, otherwise use b 290 | > if (Number % 3) == 0: 291 | > message = message + a 292 | > else: 293 | > message = message + "b" 294 | > print(message) 295 | > ~~~ 296 | > {: .source} 297 | {: .challenge} 298 | 299 | > ## While Loops 300 | > 301 | > Python also has a `while` loop 302 | > that keeps going as long as some condition is true: 303 | > 304 | > ~~~ 305 | > x = 15 306 | > while x > 0: 307 | > print(x) 308 | > x = x - 5 309 | > ~~~ 310 | > {: .python} 311 | > ~~~ 312 | > 15 313 | > 10 314 | > 5 315 | > ~~~ 316 | > {: .output} 317 | > 318 | > Use a `while` loop to print every second character in the string 'fluorine'. 319 | > {: .challenge} 320 | 321 | > ## Trimming Values 322 | > 323 | > Fill in the blanks so that this program creates a new list 324 | > containing zeroes where the original list's values were negative 325 | > and ones where the origina list's values were positive. 326 | > 327 | > ~~~ 328 | > original = [-1.5, 0.2, 0.4, 0.0, -1.3, 0.4] 329 | > result = ____ 330 | > for value in original: 331 | > if ____: 332 | > result.append(0) 333 | > else: 334 | > ____ 335 | > print(result) 336 | > ~~~ 337 | > {: .source} 338 | > 339 | > ~~~ 340 | > [0, 1, 1, 1, 0, 1] 341 | > ~~~ 342 | > {: .output} 343 | {: .challenge} 344 | 345 | > ## Initializing 346 | > 347 | > Modify this program so that it finds the largest and smallest values in the list 348 | > no matter what the range of values originally is. 349 | > 350 | > ~~~ 351 | > values = [...some test data...] 352 | > smallest, largest = None, None 353 | > for v in values: 354 | > if ____: 355 | > smallest, largest = v, v 356 | > ____: 357 | > smallest = min(____, v) 358 | > largest = max(____, v) 359 | > print(smallest, largest) 360 | > ~~~ 361 | > {: .source} 362 | > 363 | > What are the advantages and disadvantages of using this method 364 | > to find the range of the data? 365 | {: .challenge} 366 | -------------------------------------------------------------------------------- /_episodes/03-lib.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Libraries" 3 | teaching: 10 4 | exercises: 10 5 | questions: 6 | - "How can I use Python's standard libraries?" 7 | - "Where do I find documentation on Python's standard libraries?" 8 | objectives: 9 | - "Use `import` to load entire libraries and elements of libraries." 10 | - "Use `import` to load libraries under aliases." 11 | - "Use elements of libraries via dot notation." 12 | - "Use the `math` and `random` libraries." 13 | - "Use the `csv` library to read CSV files." 14 | keypoints: 15 | - "Use `import` to load a library." 16 | - "Use dot notation to get library's contents." 17 | - "The `math` library has common mathematical functions." 18 | - "The `random` library produces pseudo-random numbers." 19 | - "The `csv` library can read CSV files correctly." 20 | --- 21 | ## Most of the power of a programming language is in its libraries. 22 | 23 | * Python's [standard library][stdlib] is installed with it. 24 | * Many additional libraries are available from [PyPI][pypi] (the Python Package Index). 25 | 26 | ## A program must import a library in order to use it. 27 | 28 | * Use `import` to load a library into a program's memory. 29 | * Then refer to things from the library as `library_name.thing_name`. 30 | 31 | ~~~ 32 | import math 33 | 34 | print('pi is', math.pi) 35 | print('cos(pi) is', math.cos(math.pi)) 36 | ~~~ 37 | {: .python} 38 | ~~~ 39 | pi is 3.141592653589793 40 | cos(pi) is -1.0 41 | ~~~ 42 | {: .output} 43 | 44 | ## Use `help` to find out more about a library's contents. 45 | 46 | ~~~ 47 | help(math) 48 | ~~~ 49 | {: .python} 50 | ~~~ 51 | Help on module math: 52 | 53 | NAME 54 | math 55 | 56 | MODULE REFERENCE 57 | http://docs.python.org/3.5/library/math 58 | 59 | The following documentation is automatically generated from the Python 60 | source files. It may be incomplete, incorrect or include features that 61 | are considered implementation detail and may vary between Python 62 | implementations. When in doubt, consult the module reference at the 63 | location listed above. 64 | 65 | DESCRIPTION 66 | This module is always available. It provides access to the 67 | mathematical functions defined by the C standard. 68 | 69 | FUNCTIONS 70 | acos(...) 71 | acos(x) 72 | 73 | Return the arc cosine (measured in radians) of x. 74 | ⋮ ⋮ ⋮ 75 | ~~~ 76 | {: .output} 77 | 78 | ## Import specific items from a library to shorten programs. 79 | 80 | * Use `from...import...` to load only specific items from a library. 81 | * Then refer to them directly without library name as prefix. 82 | 83 | ~~~ 84 | from math import cos, pi 85 | 86 | print('cos(pi) is', cos(pi)) 87 | ~~~ 88 | {: .python} 89 | ~~~ 90 | cos(pi) is -1.0 91 | ~~~ 92 | {: .output} 93 | 94 | ## Create an alias for a library when importing it to shorten programs. 95 | 96 | * Use `import...as...` to give a library a short *alias* while importing it. 97 | * Then refer to items in the library using that shortened name. 98 | 99 | ~~~ 100 | import math as m 101 | 102 | print('cos(pi) is', m.cos(m.pi)) 103 | ~~~ 104 | {: .python} 105 | ~~~ 106 | cos(pi) is -1.0 107 | ~~~ 108 | {: .output} 109 | 110 | * Commonly used for libraries that are frequently used or have long names. 111 | * E.g., `matplotlib` plotting library is often aliased as `mpl`. 112 | * But can make programs harder to understand, 113 | since readers must learn your program's aliases. 114 | 115 | > ## Exploring the Math Library 116 | > 117 | > 1. What function from the `math` library can you use to calculate a square root 118 | > *without* using `sqrt`? 119 | > 2. Since the library contains this function, why does `sqrt` exist? 120 | {: .challenge} 121 | 122 | > ## Locating the Right Library 123 | > 124 | > You want to select a random character from a string: 125 | > ~~~ 126 | > bases = 'ACTTGCTTGAC' 127 | > ~~~ 128 | > 129 | > 1. What [standard library][stdlib] would you most expect to help? 130 | > 2. Which function would you select from that library? Are there alternatives? 131 | {: .challenge} 132 | 133 | > ## When Is Help Available? 134 | > 135 | > When a colleague of yours types `help(math)`, 136 | > Python reports an error: 137 | > 138 | > ~~~ 139 | > NameError: name 'math' is not defined 140 | > ~~~ 141 | > {: .error} 142 | > 143 | > What has your colleague forgotten to do? 144 | {: .challenge} 145 | 146 | > ## Importing With Aliases 147 | > 148 | > 1. Fill in the blanks so that the program below prints `90.0`. 149 | > 2. Rewrite the program so that it uses `import` *without* `as`. 150 | > 3. Which form do you find easier to read? 151 | > 152 | > ~~~ 153 | > import math as m 154 | > angle = ____.degrees(____.pi / 2) 155 | > print(____) 156 | > ~~~ 157 | > {: .source} 158 | {: .challenge} 159 | 160 | > ## Importing Specific Items 161 | > 162 | > 1. Fill in the blanks so that the program below prints `90.0`. 163 | > 2. Do you find this easier to read than preceding versions? 164 | > 3. Why *would't* programmers always use this form of `import`? 165 | > 166 | > ~~~ 167 | > ____ math import ____, ____ 168 | > angle = degrees(pi / 2) 169 | > print(angle) 170 | > ~~~ 171 | > {: .source} 172 | {: .challenge} 173 | 174 | > ## Checking Random Numbers 175 | > 176 | > Look up the documentation for the `random` library, 177 | > then write a short program that generates 178 | > a large number of samples from the normal distribution 179 | > with mean 0.0 and standard deviation 1.0 180 | > and see how close the sample average comes to 0.0. 181 | {: .challenge} 182 | 183 | > ## Reading Comma-Separated Values 184 | > 185 | > Look up the documentation for the `csv` library 186 | > and use it to read a file containing tabular data 187 | > in comma-separated values (CSV) format. 188 | > Why would you use a library like this 189 | > rather than just reading lines and splitting on the commas? 190 | {: .challenge} 191 | 192 | [pypi]: https://pypi.python.org/pypi/ 193 | [stdlib]: https://docs.python.org/3/library/ 194 | -------------------------------------------------------------------------------- /_episodes/04-coffee.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: break 3 | title: "Morning Coffee" 4 | teaching: 0 5 | exercises: 0 6 | break: 15 7 | --- 8 | 9 | 1. What are the pros and cons of using the Jupyter Notebook for programming 10 | compared to using a plain text editor? 11 | 12 | 2. What are the pros and cons of using indentation to show blocks of code 13 | instead of `begin`...`end` or `{}`? 14 | -------------------------------------------------------------------------------- /_episodes/05-functions.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Writing Functions" 3 | teaching: 15 4 | exercises: 20 5 | questions: 6 | - "How can I make code more readable?" 7 | - "How can I make code reusable?" 8 | objectives: 9 | - "Write a function taking a fixed number of parameters and having a single point of return." 10 | - "Write functions whose parameters have default values." 11 | - "Write functions with multiple points of return." 12 | - "Write functions that return multiple values." 13 | - "Extract functions from longer programs." 14 | - "Trace the execution of nested non-recursive function calls." 15 | - "Correctly identify the scope of variables." 16 | keypoints: 17 | - "Use `def` to define a new function." 18 | - "Give parameters default values to make use easier and intent clearer." 19 | - "Use `*args` to handle variable-length parameter lists." 20 | - "Use `return` at any point to return values." 21 | - "Turn repeated or deeply-nested pieces of code into functions." 22 | - "Functions temporarily store values on a call stack." 23 | --- 24 | ## Break programs down into functions. 25 | 26 | * Readability: human beings can only keep a few items in working memory at a time. 27 | * *Encapsulate* complexity so that we can treat it as a single "thing". 28 | * Reuse: write one time, use many times. 29 | * Testing: components with well-defined boundaries are easier to test. 30 | 31 | ## Define a function using `def` with a name, parameters, and a block of code. 32 | 33 | * Function name must obey the same rules as variable names. 34 | * Put *parameters* in parentheses. 35 | * Empty parentheses if the function doesn't take any inputs. 36 | * Then a colon and an indented block of code. 37 | 38 | ~~~ 39 | def print_greeting(): 40 | print('Hello!') 41 | ~~~ 42 | {: .python} 43 | 44 | ## Arguments in call are matched to parameters in definition. 45 | 46 | ~~~ 47 | def print_date(year, month, day): 48 | joined = str(year) + '/' + str(month) + '/' + str(day) 49 | print(joined) 50 | 51 | print_date(1871, 3, 19) 52 | ~~~ 53 | {: .python} 54 | ~~~ 55 | 1871/3/19 56 | ~~~ 57 | {: .output} 58 | 59 | ## Functions may return a result to their caller using `return`. 60 | 61 | * May occur anywhere in the function. 62 | * But functions are easier to understand if `return` occurs: 63 | * At the start to handle special cases. 64 | * At the very end, with a final result. 65 | * Functions without explicit `return` produce `None`. 66 | 67 | ~~~ 68 | def average(values): 69 | if len(values) == 0: 70 | return None 71 | return sum(values) / len(values) 72 | ~~~ 73 | {: .python} 74 | 75 | ~~~ 76 | a = average([1, 3, 4]) 77 | print('average of actual values:', a) 78 | ~~~ 79 | {: .python} 80 | ~~~ 81 | 2.6666666666666665 82 | ~~~ 83 | {: .output} 84 | 85 | ~~~ 86 | print('average of empty list:', average([])) 87 | ~~~ 88 | {: .python} 89 | ~~~ 90 | None 91 | ~~~ 92 | {: .output} 93 | 94 | ~~~ 95 | result = print_date(1871, 3, 19) 96 | print('result of call is:', result) 97 | ~~~ 98 | {: .python} 99 | ~~~ 100 | 1871/3/19 101 | result of call is: None 102 | ~~~ 103 | {: .output} 104 | 105 | ## Can specify default values for parameters. 106 | 107 | * All parameters with defaults must come *after* all parameters without. 108 | * Argument-to-parameter matching would be ambiguous otherwise. 109 | * Makes common cases simpler, and signals intent. 110 | 111 | ~~~ 112 | def sum(values, scale=1.0): 113 | result = 0.0 114 | for v in values: 115 | result += v * scale 116 | return result 117 | 118 | print('sum with default:', sum([1, 2, 3])) 119 | print('sum with factor:', sum([1, 2, 3], 0.5)) 120 | ~~~ 121 | ~~~ 122 | sum with default: 6.0 123 | sum with factor: 3.0 124 | ~~~ 125 | {: .output} 126 | 127 | ## Can pass parameters by name. 128 | 129 | * Helpful when functions have lots of options. 130 | 131 | ~~~ 132 | print('out of order:', sum(scale=0.25, values=[1, 2, 3])) 133 | ~~~ 134 | {: .python} 135 | ~~~ 136 | out of order: 1.5 137 | ~~~ 138 | {: .output} 139 | 140 | ## Functions can take a variable number of arguments. 141 | 142 | * Prefix at most one parameter's name with `*`. 143 | * By convention, everyone calls the parameters `args`. 144 | * All "extra" parameters are put in a list-like structure assigned to that parameter. 145 | * We'll explain what "list-like" means in more detail later. 146 | 147 | ~~~ 148 | def total(scale, *args): 149 | result = 0.0 150 | for a in args: 151 | result += a * scale 152 | return result 153 | 154 | print('with one value:', total(0.5, 1)) 155 | print('with two values:', total(0.5, 1, 3)) 156 | ~~~ 157 | {: .python} 158 | ~~~ 159 | with one value: 0.5 160 | with two values: 2.0 161 | ~~~ 162 | {: .output} 163 | 164 | ## Functions can return multiple values. 165 | 166 | * This is just a special case of many-to-many assignment. 167 | 168 | ~~~ 169 | red, green, blue = 10, 50, 180 170 | 171 | def order(a, b): 172 | if a < b: 173 | return a, b 174 | else: 175 | return b, a 176 | 177 | low, high = order(10, 5) 178 | print('order(10, 5):', low, high) 179 | ~~~ 180 | {: .python} 181 | ~~~ 182 | order(10, 5): 5 10 183 | ~~~ 184 | {: .output} 185 | 186 | > ## Encapsulation 187 | > 188 | > Fill in the blanks to create a function that takes a single filename as an argument, 189 | > loads the data in the file named by the argument, 190 | > and returns the minimum value in that data. 191 | > 192 | > ~~~ 193 | > import pandas 194 | > 195 | > def min_in_data(____): 196 | > data = ____ 197 | > return ____ 198 | > ~~~ 199 | > {: .source} 200 | {: .challenge} 201 | 202 | > ## Find the First 203 | > 204 | > Fill in the blanks to create a function that takes a list of numbers as an argument 205 | > and returns the first negative value in the list. 206 | > What does your function do if the list is empty? 207 | > 208 | > ~~~ 209 | > def first_negative(values): 210 | > for v in ____: 211 | > if ____: 212 | > return ____ 213 | > ~~~ 214 | > {: .python} 215 | {: .challenge} 216 | 217 | > ## Running Sum 218 | > 219 | > Write a function that calculates the running sum of any number of input arguments, 220 | > returning the result as a list. 221 | > For example: 222 | > 223 | > * `running(1, 2)` => `[1, 3]` 224 | > * `running(-5, 2, 7)` => `[-5, -3, 4]` 225 | > 226 | > What should `running()` return, and why? 227 | {: .challenge} 228 | 229 | > ## Extracting Functions 230 | > 231 | > People often create functions 232 | > by extracting pieces of code from scripts they've written. 233 | > Re-write the short program below to have one or more functions 234 | > so that no single piece of code is more than ten lines long. 235 | > Do you find the result easier to understand? 236 | > 237 | > ~~~ 238 | > # Report the number of lines in each chapter in a text file. 239 | > # A chapter heading has '##' in the first two columns. 240 | > # Blank lines are *not* counted in totals. 241 | > 242 | > reader = open('thesis.txt', 'r') 243 | > current_title = None 244 | > current_count = None 245 | > for line in reader: 246 | > line = line.strip() 247 | > if not line: 248 | > pass 249 | > elif line.startswith('##'): 250 | > if current_title is None: 251 | > current_title = line 252 | > current_count = 0 253 | > else: 254 | > print(current_title, current_count) 255 | > current_title = line 256 | > current_count = 0 257 | > else: 258 | > current_count = current_count + 1 259 | > 260 | > if current_title is not None: 261 | > print(current_title, current_count) 262 | > ~~~ 263 | > {: .python} 264 | {: .challenge} 265 | -------------------------------------------------------------------------------- /_episodes/06-dict.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Dictionaries" 3 | teaching: 15 4 | exercises: 20 5 | questions: 6 | - "How can I store and manipulate non-rectangular data?" 7 | objectives: 8 | - "Write programs that use sets to store unique values." 9 | - "Write programs that use dictionaries to store key/value pairs." 10 | - "Identify values that can and cannot be used as dictionary keys." 11 | keypoints: 12 | - "Use sets to store unique unordered values." 13 | - "Use dictionaries to store extra information with those values." 14 | --- 15 | 16 | ## Use a set to store unique values. 17 | 18 | * Create with `{...}` 19 | * But must use `set()` to create an empty set. 20 | 21 | ~~~ 22 | primes = {2, 3, 5, 7} 23 | print('is 3 prime?', 3 in primes) 24 | print('is 9 prime?', 9 in primes) 25 | ~~~ 26 | {: .python} 27 | ~~~ 28 | is 3 prime? True 29 | is 9 prime? False 30 | ~~~ 31 | {: .output} 32 | 33 | * Intersection, union, etc. 34 | 35 | ~~~ 36 | odds = {3, 5, 7, 9} 37 | print('intersection', odds & primes) 38 | print('union', odds | primes) 39 | ~~~ 40 | {: .python} 41 | ~~~ 42 | intersection {3, 5, 7} 43 | union {3, 5, 7, 9, 11} 44 | ~~~ 45 | {: .output} 46 | 47 | ## Sets are mutable. 48 | 49 | * But only store *unique* values. 50 | 51 | ~~~ 52 | primes.add(11) 53 | print('primes becomes', primes) 54 | primes.discard(7) 55 | print('after removal', primes) 56 | primes.add(11) 57 | print('after adding 11 again', primes) 58 | ~~~ 59 | {: .python} 60 | ~~~ 61 | primes becomes {11, 2, 3, 5, 7} 62 | after removal {11, 2, 3, 5} 63 | after adding 11 again {11, 2, 3, 5} 64 | ~~~ 65 | {: .output} 66 | 67 | ## Sets are unordered. 68 | 69 | * Values are stored by *hashing*. 70 | * Which is intentionally as random as possible. 71 | 72 | ~~~ 73 | names = {'Darwin', 'Newton', 'Turing'} 74 | for n in names: 75 | print(n) 76 | ~~~ 77 | {: .python} 78 | ~~~ 79 | Newton 80 | Darwin 81 | Turing 82 | ~~~ 83 | {: .output} 84 | 85 | ## Use a dictionary to store key/value pairs. 86 | 87 | * Equivalently, to store extra information with elements of a set. 88 | 89 | ~~~ 90 | birthdays = {'Newton' : 1602, 'Darwin' : 1809} 91 | print(birthdays['Newton']) 92 | birthdays['Turing'] = 1612 # oops 93 | birthdays['Turing'] = 1912 # that's better 94 | print(birthdays) 95 | ~~~ 96 | {: .python} 97 | ~~~ 98 | 1602 99 | {'Darwin': 1809, 'Newton': 1602, 'Turing': 1912} 100 | ~~~ 101 | {: .output} 102 | 103 | * Just an accident that keys are alphabetical in this case. 104 | * Like sets, dictionaries store keys by hashing, which is as random as possible. 105 | 106 | ## Set values and dictionary keys must be immutable. 107 | 108 | * Changing them after insertion would leave data in the wrong place. 109 | * Use a *tuple* for multi-valued keys. 110 | 111 | ~~~ 112 | people = {('Isaac', 'Newton'): 1602, ('Charles', 'Darwin'): 1809} 113 | ~~~ 114 | {: .python} 115 | 116 | ## Example: create a histogram. 117 | 118 | ~~~ 119 | numbers = [1, 0, 1, 2, 0, 0, 1, 2, 1, 3, 1, 0, 2] 120 | count = {} 121 | for n in numbers: 122 | if n not in count: 123 | count[n] = 1 124 | else: 125 | count[n] = count[n] + 1 126 | print(count) 127 | ~~~ 128 | {: .python} 129 | ~~~ 130 | {0: 4, 1: 5, 2: 3, 3: 1} 131 | ~~~ 132 | {: .output} 133 | 134 | ## Keys are often strings. 135 | 136 | ~~~ 137 | elements = {'H' : 1, 'He' : 2, 'Li' : 3, 'Be' : 4, 'B' : 5} 138 | print('atomic number of lithium:', elements['Li']) 139 | ~~~ 140 | {: .python} 141 | ~~~ 142 | atomic number of lithium: 3 143 | ~~~ 144 | {: .output} 145 | 146 | > ## How Heavy Is This Molecule? 147 | > 148 | > Write a function that takes two parameters: 149 | > a dictionary mapping atomic symbols to atomic weights, 150 | > and a list of (atom, count) pairs for a molecule, 151 | > and returns that molecule's molecular weight. 152 | > 153 | > ~~~ 154 | > weights = {'H' : 1.0079, 'C' : 12.0107, 'N' : 14.0067, 155 | > 'O' : 15.9994, 'P' : 30.9738, 'S' : 32.065} 156 | > 157 | > methane = [('C', 1), ('H', 4)] 158 | > print('methane', mol_weight(weights, methane)) 159 | > 160 | > aminothiazole = [('C', 3), ('H', 4), ('N', 2), ('S', 1)] 161 | > print('aminothiazole', mol_weight(weights, aminothiazole)) 162 | > ~~~ 163 | > {: .python} 164 | > ~~~ 165 | > methane 16.0423 166 | > aminothiazole 100.1421 167 | > ~~~ 168 | > {: .output} 169 | > 170 | > How did you test your function? 171 | {: .challenge} 172 | -------------------------------------------------------------------------------- /_episodes/07-lunch.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: break 3 | title: "Lunch" 4 | teaching: 0 5 | exercises: 0 6 | break: 60 7 | --- 8 | FIXME: describe what to reflect on. 9 | -------------------------------------------------------------------------------- /_episodes/08-numpy.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "NumPy Arrays" 3 | teaching: 20 4 | exercises: 20 5 | questions: 6 | - "How can I store and manipulate arrays?" 7 | - "How can I do linear algebra?" 8 | objectives: 9 | - "Write programs that create and load arrays using NumPy." 10 | - "Replace `for` loops with vectorized indexing operations." 11 | - "Perform linear algebra operations on NumPy arrays." 12 | keypoints: 13 | - "Use NumPy arrays to store multi-dimensional arrays." 14 | - "Algebraic matrices are a special case of arrays." 15 | - "Array operations make (most) loops unnecessary." 16 | --- 17 | FIXME 18 | 19 | ## Exercises 20 | 21 | * Fit polynomial curves to data using vectorized linear algebra operations. 22 | -------------------------------------------------------------------------------- /_episodes/10-coffee.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: break 3 | title: "Afternoon Coffee" 4 | teaching: 0 5 | exercises: 0 6 | break: 15 7 | --- 8 | FIXME: describe what to reflect on. 9 | -------------------------------------------------------------------------------- /_episodes/11-plotting.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Plotting" 3 | teaching: 15 4 | exercises: 15 5 | questions: 6 | - "How can I plot my results?" 7 | objectives: 8 | - "Create a time series plot of statistical data." 9 | - "Create a scatter plot of statistical data." 10 | keypoints: 11 | - "Use matplotlib with arrays or data frames to visualize data." 12 | - "Decide what kind of plot to create based on what questions you want to answer." 13 | --- 14 | FIXME 15 | 16 | ## Exercises 17 | 18 | * Modify plots used in teaching. 19 | -------------------------------------------------------------------------------- /_episodes/12-file-io.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "File I/O" 3 | teaching: 10 4 | exercises: 5 5 | questions: 6 | - "How can I read data from a file?" 7 | - "How can I write data to a file?" 8 | objectives: 9 | - "Use `open`, `read`, and `readline` to read data from a file." 10 | - "Use a file in `for` loop." 11 | - "Use `write` to save data to a file." 12 | - "Use basic string operations to process text data." 13 | keypoints: 14 | - "Open a file for reading or writing with `open`." 15 | - "Use `read` or `readline` to read directly." 16 | - "Use file in `for` loop to process lines." 17 | - "Use `write` to add data to a file." 18 | --- 19 | ## Use `open` to open files for reading or writing. 20 | 21 | * Arguments are a path and: 22 | * 'r' for reading 23 | * 'w' for writing (immediately erases existing contents) 24 | * 'a' for appending 25 | * Result is an object with methods for reading and writing. 26 | * `file.read()` reads the entire file. 27 | * `file.read(N)` reads up to that many bytes. 28 | * Close files with `file.close`. 29 | 30 | ~~~ 31 | reader = open('myfile.txt', 'r') 32 | data = reader.read() 33 | reader.close() 34 | print('file contains', len(data), 'bytes') 35 | ~~~ 36 | {: .python} 37 | ~~~ 38 | file contains 47189 bytes 39 | ~~~ 40 | {: .output} 41 | 42 | ## Usually read text files with `for` loops. 43 | 44 | * Automatically calls `file.readline`. 45 | 46 | ~~~ 47 | reader = open('myfile.txt', 'r') 48 | count = 0 49 | for line in reader: 50 | count = count + 1 51 | reader.close() 52 | print('file contains', count, 'lines') 53 | ~~~ 54 | {: .python} 55 | ~~~ 56 | file contains 261 lines 57 | ~~~ 58 | {: .output} 59 | 60 | ## Python preserves end-of-line newlines. 61 | 62 | * By default, converts Windows '\r\n' to Unix '\n'. 63 | 64 | ## Strip whitespace using string methods. 65 | 66 | * Use `str.strip` to strip leading and trailing whitespace. 67 | * `' abc '.strip()` is `'abc'`. 68 | * Use `str.rstrip` or `str.lstrip` to strip space from right or left end only. 69 | * All of these methods return new strings. 70 | * Because strings cannot be modified in place. 71 | 72 | ~~~ 73 | reader = open('myfile.txt', 'r') 74 | count = 0 75 | for line in reader: 76 | line = line.strip() 77 | if len(line) > 0: 78 | count = count + 1 79 | reader.close() 80 | print('file contains', count, 'non-blank lines') 81 | ~~~ 82 | {: .python} 83 | ~~~ 84 | file contains 225 non-blank lines 85 | ~~~ 86 | {: .output} 87 | 88 | > ## Using `with` to Guarantee a File is Closed 89 | > 90 | > It is good practice to `close` a file after you have opened it. 91 | > You can use the `with` keyword in Python to ensure this: 92 | > 93 | > ~~~ 94 | > with open('myfile.txt', 'r') as reader: 95 | > data = reader.read() 96 | > print('file contains', len(data), 'bytes') 97 | > ~~~ 98 | > {: .python} 99 | > 100 | > The `with` statement has two parts: 101 | > the expression to execute (such as opening a file) 102 | > and the variable that stores its result (in this case, `reader`). 103 | > At the end of the `with` block, 104 | > the file that was assigned to `reader` will automatically be closed. 105 | > `with` statements can be used with other kinds of objects 106 | > to achieve similar effects. 107 | {: .callout} 108 | 109 | > ## Squeezing a File 110 | > 111 | > 1. Write a small program that reads lines of text from a file called `input.dat` 112 | > and writes those lines to a file called `output.dat`. 113 | > 114 | > 2. Modify your program so that it only copies non-blank lines. 115 | > 116 | > 3. Modify your program again so that a line is not copied 117 | > if it is a duplicate of the line above it. 118 | > 119 | > 4. Compare your implementation to your neighbor's. 120 | > Did you interpret the third requirement (copying non-duplicated lines) 121 | > the same way? 122 | {: .challenge} 123 | -------------------------------------------------------------------------------- /_episodes/13-style.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Programming Style" 3 | teaching: 5 4 | exercises: 10 5 | questions: 6 | - "How can I make programs more robust?" 7 | - "How can I make programs easier to understand?" 8 | objectives: 9 | - "Add `assert` statements to programs to check pre-conditions, post-conditions, and invariants." 10 | keypoints: 11 | - "Fail early, often, and meaningfully." 12 | - "Use `assert` to check that programs are working correctly." 13 | - "Give `assert` statements meaningful error messages to help users." 14 | --- 15 | ## Follow standard Python style in your code. 16 | 17 | * [PEP8](https://www.python.org/dev/peps/pep-0008) is a style guide for Python. 18 | * The [PEP8 application and Python library](https://pypi.python.org/pypi/pep8) 19 | checks code for compliance. 20 | 21 | ## Use assertions to check for internal errors. 22 | 23 | * `assert condition, 'message'` halts with an error message if the condition isn't true. 24 | * Pre-condition: must be true in order for this function to run. 25 | * Post-condition: guaranteed true at the end of this function. 26 | * Invariant: guaranteed true at this point in the program (typically in a loop). 27 | * Proportions vary widely, but 10-20% of production code is there to check the other 80-90%. 28 | * "Fail early, fail often, fail loudly." 29 | 30 | ~~~ 31 | def bounds(values): 32 | assert len(values) > 0, 'Cannot get bounds of empty list.' 33 | low = min(values) 34 | high = max(values) 35 | assert low <= high, 'Low bound should not be greater than high bound' 36 | return low, high 37 | ~~~ 38 | {: .python} 39 | 40 | ## Use docstrings to provide online help. 41 | 42 | * If the first thing in a function is a character string 43 | that is not assigned to a variable, 44 | Python attaches it to the function as the online help. 45 | 46 | ~~~ 47 | def average(values): 48 | "Return average of values, or None if no values are supplied." 49 | 50 | if len(values) == 0: 51 | return None 52 | return sum(values) / average(values) 53 | 54 | help(average) 55 | ~~~ 56 | {: .python} 57 | ~~~ 58 | Help on function average in module __main__: 59 | 60 | average(values) 61 | Return average of values, or None if no values are supplied. 62 | ~~~ 63 | {: .output} 64 | 65 | > ## Multiline Strings 66 | > 67 | > Often use *multiline strings* for documentation. 68 | > These start and end with three quote characters (either single or double) 69 | > and end with three matching characters. 70 | > 71 | > ~~~ 72 | > """This string spans multiple lines. 73 | > 74 | > Blank lines are allowed. For documentation, typically a first line 75 | > summarizes the functionality, and a blank line separates that summary 76 | > from the remainder. The ending three quote characters, in the case of a 77 | > long documentation string, are typically on their own line. 78 | > """ 79 | > ~~~ 80 | > {: .python} 81 | {: .callout} 82 | 83 | > ## Document This 84 | > 85 | > Turn the comment on the following function into a docstring 86 | > and check that `help` displays it properly. 87 | > 88 | > ~~~ 89 | > def middle(a, b, c): 90 | > # Return the middle value of three. 91 | > # Assumes the values can actually be compared. 92 | > values = [a, b, c] 93 | > values.sort() 94 | > return values[1] 95 | > ~~~ 96 | > {: .source} 97 | {: .challenge} 98 | 99 | > ## Clean Up This Code 100 | > 101 | > 1. Read this short program and try to predict what it does. 102 | > 2. Run it: how accurate was your prediction? 103 | > 3. Refactor the program to make it more readable. 104 | > Add assertions as you go to make sure you're not breaking things, 105 | > and run it after each change to test it. 106 | > 4. Compare your rewrite with your neighbor's. 107 | > What did you do the same? 108 | > What did you do differently, and why? 109 | > 110 | > ~~~ 111 | > n = 10 112 | > s = 'et cetera' 113 | > print(s) 114 | > i = 0 115 | > while i < n: 116 | > # print('at', j) 117 | > new = '' 118 | > for j in range(len(s)): 119 | > left = j-1 120 | > right = (j+1)%len(s) 121 | > if s[left]==s[right]: new += '-' 122 | > else: new += '*' 123 | > s=''.join(new) 124 | > print(s) 125 | > i += 1 126 | > ~~~ 127 | > {: .source} 128 | {: .challenge} 129 | -------------------------------------------------------------------------------- /_episodes/14-testing.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Testing" 3 | teaching: 15 4 | exercises: 10 5 | questions: 6 | - "How can I tell if my program is working?" 7 | - "How can I tell if my program is *still* working?" 8 | objectives: 9 | - "Write and run unit tests using `py.test`." 10 | - "Turn a bug into a test case that verifies the fix to the bug." 11 | keypoints: 12 | - "Use unit testing framework to run tests repeatedly." 13 | - "Write unit tests to clarify design and make future development faster." 14 | - "Turn bugs into tests." 15 | --- 16 | FIXME 17 | 18 | ## Exercises 19 | 20 | * Add more tests to overlap finder. 21 | -------------------------------------------------------------------------------- /_episodes/15-wrap.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Wrap-Up" 3 | teaching: 0 4 | exercises: 10 5 | questions: 6 | - "What have we learned?" 7 | objectives: 8 | - "Provide feedback on the day." 9 | keypoints: 10 | - "Summarize the day's learnings." 11 | --- 12 | FIXME 13 | -------------------------------------------------------------------------------- /_episodes_rmd/.gitkeep: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/swcarpentry/python-second-language/074c1d9aa7424104e8a5006623e91eeb6545bbf5/_episodes_rmd/.gitkeep -------------------------------------------------------------------------------- /_episodes_rmd/data/.gitkeep: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/swcarpentry/python-second-language/074c1d9aa7424104e8a5006623e91eeb6545bbf5/_episodes_rmd/data/.gitkeep -------------------------------------------------------------------------------- /_extras/.gitkeep: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/swcarpentry/python-second-language/074c1d9aa7424104e8a5006623e91eeb6545bbf5/_extras/.gitkeep -------------------------------------------------------------------------------- /_extras/about.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: page 3 | title: About 4 | permalink: /about/ 5 | --- 6 | {% include carpentries.html %} 7 | -------------------------------------------------------------------------------- /_extras/cmdline.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: episode 3 | title: "Command-Line Programming" 4 | teaching: 15 5 | exercises: 15 6 | questions: 7 | - "How can I write command-line utilities in Python?" 8 | objectives: 9 | - "Write programs that process command-line arguments in `sys.argv` directly." 10 | - "Write programs that use the `argparse` library to process command-line arguments." 11 | - "Write programs that read from standard input and write to standard output." 12 | keypoints: 13 | - "Get command-line arguments from `sys.argv`." 14 | - "Read from `sys.stdin` and write to `sys.stdout`." 15 | - "Provide a meaningful usage message." 16 | --- 17 | FIXME 18 | 19 | ## Exercises 20 | 21 | * Add extra options to an existing program. 22 | -------------------------------------------------------------------------------- /_extras/design.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: page 3 | title: "Lesson Design" 4 | permalink: /design/ 5 | --- 6 | 7 | > ## Help Wanted 8 | > {:.no_toc} 9 | > 10 | > **We are filling in the exercises [below](#stage-3---learning-plan) 11 | > in order to make the lesson plan more concrete. 12 | > Contributions (both in the form of pull requests with filled-in exercises, 13 | > and comments on specific exercises, ordering, and timings) are greatly appreciated.** 14 | {: .callout} 15 | 16 | ## Process Used 17 | 18 | This lesson was developed using a slimmed-down variant of the "Understanding by Design" process. 19 | The main sections are: 20 | 21 | 1. Assumptions about audience, time, etc. 22 | 23 | 2. Desired results: 24 | * Overall goals 25 | * Summative assessments at half-day granularity 26 | * What learners will be able to do, what they will know, etc. 27 | 28 | 3. Learning plan 29 | * Each episode has a heading that summarizes what will be covered, 30 | then estimates time that will be spent on teaching and on exercises. 31 | * The exercises are outlined to make expectations concrete. 32 | 33 | ## Stage 1 - Assumptions 34 | 35 | * Audience 36 | * Graduate students and research software engineers in numerate disciplines from cosmology to economics 37 | * Who can write procedural code in a dynamic language such as Perl or MATLAB 38 | * Largely self-taught 39 | * Variables and assignment, loops, lists or arrays, conditionals, simple functions 40 | * May or may not have done object-oriented programming 41 | * Familiar with basic Unix shell commands (cd, ls, rm) and concepts (home directory, path) 42 | * But have not programmed in Python before (or if they have, they've only dabbled) 43 | * Constraints 44 | * One full day 09:00-16:00 45 | * 06:30 teaching time 46 | * 1:00 for lunch 47 | * 0:30 total for two coffee breaks 48 | * Learners use native installs on their own machines 49 | * May connect to a cloud resource at their own discretion, but they have to set it up 50 | * Assume knowledge of the Unix shell but *not* of version control 51 | * Use the Jupyter Notebook 52 | * Authentic tool 53 | * There isn't really an alternative 54 | * And means that even people who have seen a bit of Python before will probably learn something 55 | * Exercises will mostly *not* be "write this code from scratch" 56 | * Want lots of short exercises that can reliably be finished in allotted time 57 | * So use MCQs, fill-in-the-blanks, Parsons Problems, "tweak this code", etc. 58 | * Running Examples 59 | * Morning: invasion percolation 60 | * Afternoon: data analysis 61 | 62 | ## Stage 2 - Desired Results 63 | 64 | ### Essential Questions 65 | 66 | How do I... 67 | 68 | * ...express operations I'm already familiar with (loops, conditionals, lists) in Python? 69 | * ...break a program into functions? 70 | * ...write unit tests for Python programs? 71 | * ...use data structures make programs simpler and more efficient? 72 | * ...work with matrices? 73 | * ...analyze tabular data? 74 | * ...get data from the web? 75 | 76 | ### Concepts 77 | 78 | Learners will know that... 79 | 80 | * ...Python is a dynamic imperative language. 81 | * ...dictionaries can make programs simpler and more efficient at the same time. 82 | * ...unit tests are easy to express and run using supporting libraries. 83 | * ...matrices can be manipulated using MATLAB-style methods. 84 | * ...tabular data can be manipulated using structures like R's data frames. 85 | * ...fetching data from the web is not much more complicated than reading from local files. 86 | 87 | ### Summative Assessment 88 | 89 | * Mid-point: sort files in a directory into groups by size 90 | * Final: Download and process data set from the web 91 | 92 | ### Skills 93 | 94 | Learners can: 95 | 96 | 1. ...run code interactively in the Jupyter Notebook. 97 | 2. ...run code saved in a file from the Unix shell. 98 | 3. ...create, index, and slice lists. 99 | 4. ...create and index dictionaries. 100 | 5. ...call built-in functions. 101 | 6. ...use `help` and online documentation. 102 | 7. ...import code from libraries. 103 | 8. ...read tabular data into arrays and data frames. 104 | 9. ...do collective operations on arrays and data frames. 105 | 10. ...create simple plots of data in arrays and data frames. 106 | 11. ...interpret common error messages. 107 | 12. ...create and run unit tests. 108 | 13. ...write functions with default parameter values. 109 | 14. ...download data from the web programmatically. 110 | 111 | ## Stage 3 - Learning Plan 112 | 113 | ### [Basics]({{page.root}}/01-basics/) (09:00) 114 | 115 | * Teaching: 15 min 116 | * Running the Jupyter Notebook 117 | * `int`, `float`, `str`, `bool`, `list` 118 | * Exercises: 10 min 119 | * Immutable strings vs. mutable lists 120 | * Subscript games 121 | 122 | ### [Control Flow]({{page.root}}/02-flow/) (09:25) 123 | 124 | * Teaching: 10 min 125 | * `for` 126 | * `if`/`else` 127 | * Basic list and string methods 128 | * `range` 129 | * Exercises: 10 min 130 | * Acronymize 131 | 132 | ### [File I/O]({{page.root}}/03-file-io/) (09:45) 133 | 134 | * Teaching: 10 min 135 | * `open` and `close` 136 | * `for line in file` 137 | * Exercises: 5 min 138 | * Count non-blank lines 139 | 140 | ### [Libraries]({{page.root}}/04-lib/) (10:00) 141 | 142 | * Teaching: 10 min 143 | * `import` 144 | * dot notation 145 | * `math` and `random` 146 | * Exercises: 10 min 147 | * Calculate average of sequence of random values 148 | 149 | ### [Coffee]({{page.root}}/05-coffee/) (10:20): 15 min 150 | 151 | ### [Writing Functions]({{page.root}}/06-functions/) (10:35) 152 | 153 | * Teaching: 10 min 154 | * Basic definition 155 | * Default values for parameters 156 | * Scope rules 157 | * Exercises: 10 min 158 | * Extract functions from half page of code. 159 | 160 | ### [Defensive Programming]({{page.root}}/07-defensive/) (10:55) 161 | 162 | * Teaching: 5 min 163 | * The idea of `assert` 164 | * Exercises: 10 min 165 | * Add a few assertions to some functions 166 | 167 | ### [Dictionaries]({{page.root}}/08-dict/) (11:10) 168 | 169 | * Teaching: 15 min 170 | * Basic operations 171 | * Need for immutable keys 172 | * Tuples 173 | * Exercises: 15 min 174 | * Sort lines in a file into groups by length. 175 | 176 | ### [Profiling]({{page.root}}/09-profile/) (11:40) 177 | 178 | * Teaching: 10 min 179 | * Understanding a profile (cumulative time vs. per-call time) 180 | * Sampling vs. instrumenting 181 | * Exercises: 10 min 182 | * Compare performance of unique word finders. 183 | 184 | ### [Lunch]({{page.root}}/10-lunch/) (12:00): 60 min 185 | 186 | ### [NumPy Arrays]({{page.root}}/11-numpy/) (13:00) 187 | 188 | * Teaching: 15 min 189 | * Reading CSV files 190 | * Indexing 191 | * Aggregate operations 192 | * Converting arrays to images 193 | * Exercises: 15 min 194 | * Construct images with blocked regions. 195 | 196 | ### [Pandas]({{page.root}}/12-pandas/) (13:30) 197 | 198 | * Teaching: 15 min 199 | * Data frames 200 | * Reading CSV files 201 | * Aggregate operations 202 | * Indexing 203 | * Exercises: 15 min 204 | * Analyze temperature statistics. 205 | 206 | ### [Plotting]({{page.root}}/13-plotting/) (14:00) 207 | 208 | * Teaching: 15 min 209 | * Basic line plots 210 | * Basic scatter plots 211 | * Exercises: 15 min 212 | * Modify plots used in teaching. 213 | 214 | ### [Coffee]({{page.root}}/14-coffee/) (14:30): 15 min 215 | 216 | ### [Command-Line Programming]({{page.root}}/15-cmdline/) (14:45) 217 | 218 | * Teaching: 10 min 219 | * `sys.argv` 220 | * `sys.stdin` and `sys.stdout` 221 | * Exercises: 15 min 222 | * Add extra options to an existing program. 223 | 224 | ### [Testing]({{page.root}}/16-testing/) (15:10) 225 | 226 | * Teaching: 15 min 227 | * `py.test` 228 | * Exercises: 10 min 229 | * Add more tests to overlap finder. 230 | 231 | ### [Getting Data From the Web]({{page.root}}/17-web/) (15:35) 232 | 233 | * Teaching: 15 min 234 | * The `requests` library 235 | * The `json` library 236 | * Exercises: 10 min 237 | * Read and process another temperature data set. 238 | 239 | ### [Summary]({{page.root}}/18-wrap/) (16:00): 10 min 240 | -------------------------------------------------------------------------------- /_extras/discuss.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: page 3 | title: Discussion 4 | permalink: /discuss/ 5 | --- 6 | 7 | FIXME: general discussion. 8 | -------------------------------------------------------------------------------- /_extras/figures.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: page 3 | title: Figures 4 | permalink: /figures/ 5 | --- 6 | {% include all_figures.html %} 7 | -------------------------------------------------------------------------------- /_extras/guide.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: page 3 | title: "Instructors' Guide" 4 | permalink: /guide/ 5 | --- 6 | 7 | FIXME: instructors' guide. 8 | -------------------------------------------------------------------------------- /_extras/profiling.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: episode 3 | title: "Profiling" 4 | teaching: 15 5 | exercises: 15 6 | questions: 7 | - "Why is my program so slow?" 8 | objectives: 9 | - "Use a profiler to determine where a program is spending its time." 10 | - "Explain the difference between sampling and instrumenting profilers." 11 | keypoints: 12 | - "Get it right, then make it fast." 13 | - "Profile a program's execution by instrumenting or sampling." 14 | - "Look at per-call and cumulative runtime to understand what to tune." 15 | --- 16 | 17 | ## We use computers because they're faster than we are. 18 | 19 | * No matter how fast your computer is, some programs take too long to run. 20 | * Eventually you become tired of taking tea breaks to fill the time, 21 | or you need to tackle a problem so large, you cannot imagine it finishing at all. 22 | * But making a program run more quickly can be daunting: 23 | the program might be large, 24 | or might have been written by someone else. 25 | * Modern computing systems are too complex for us to predict performance reliably. 26 | 1. Paging (i.e., overflowing physical memory). 27 | 2. Searching instead of looking up directly. 28 | 3. Putting low-probability if-statements early in a conditional chain. 29 | 4. Calling procedures or using complex if-statements inside very large data loops. 30 | 5. Re-creating variables inside loops. 31 | 6. Reading and writing from disk or the network. 32 | 7. Using inefficient objects when an array might be better 33 | * This is where *profiling* can help. 34 | * Shows what part of a program is occupying most of its execution time, 35 | so you know where to focus your effort. 36 | * Shows if changes are actually improvements. 37 | 38 | ## Get it right, then make it fast. 39 | 40 | * No point getting the wrong answer faster or more often. 41 | 42 | ## Distinguish per-call time, total time, and cumulative time. 43 | 44 | * Per-call time: how long does each execution take (on average)? 45 | * Total time: how much time is spent in this function/block in total? 46 | * I.e., number of calls times time per call. 47 | * Cumulative time: how much time is spent in this function/block and the ones it calls? 48 | * Top-level program should show 100%. 49 | 50 | ## Profilers necessarily distort execution (a little). 51 | 52 | * *Sampling*: interrupt every few hundred microseconds and see where we are. 53 | * *Instrumenting*: record entry/exit times for functions, blocks, etc. 54 | * Both change the execution characteristics of the program. 55 | * But so does measuring voltage or temperature... 56 | 57 | > ## Installing a Profiler 58 | > 59 | > The `line_profiler` module is not installed as part of the base Anaconda Python installation. 60 | > You will need to use the `conda` package manager to install it: 61 | > 62 | > ~~~ 63 | > $ conda install line_profiler 64 | > ~~~ 65 | > {: .bash} 66 | > 67 | > If you are using a different Python distribution, 68 | > the `line_profiler` package can be installed through `pip`: 69 | > 70 | > ~~~ 71 | > pip install line_profiler 72 | > ~~~ 73 | > {: .bash} 74 | {: .challenge} 75 | 76 | ## Use a decorator to show which parts of the code you want to profile. 77 | 78 | * Add a *decorator* before each function you want the profiler to measure. 79 | * Tells Python to modify the function after it's defined. 80 | * Example: finding prime numbers. 81 | 82 | ~~~ 83 | def primes(n): 84 | if n==2: 85 | return [2] 86 | elif n<2: 87 | return [] 88 | 89 | s=list(range(3,n+1,2)) 90 | mroot = n ** 0.5 91 | half=(n+1)//2-1 92 | i=0 93 | m=3 94 | 95 | while m <= mroot: 96 | if s[i]: 97 | j=(m*m-3)//2 98 | s[j]=0 99 | while j ## Take a Guess 120 | > 121 | > Before going to the next step, 122 | > try to guess where this code is going to spend most of its time. 123 | > Which lines take longest to run individually and cumulatively? 124 | {: .callout} 125 | 126 | ## Use `kernprof` to run from the command line and collect profile data. 127 | 128 | * `kernprof -l -v primes.py` runs Python with extra options to collect and record data. 129 | * `kernprof` is the profiler script 130 | * `-l`: recognise the `@profile` decorator and profile your code. 131 | * `-v`: display timing information when the script has finished running. 132 | 133 | ~~~ 134 | Wrote profile results to primes.py.lprof 135 | Timer unit: 1e-06 s 136 | 137 | Total time: 0.000245 s 138 | File: primes.py 139 | Function: primes at line 1 140 | 141 | Line # Hits Time Per Hit % Time Line Contents 142 | ============================================================== 143 | 1 @profile 144 | 2 def primes(n): 145 | 3 1 8 8.0 3.3 if n==2: 146 | 4 return [2] 147 | 5 1 2 2.0 0.8 elif n<2: 148 | 6 return [] 149 | 7 150 | 8 1 11 11.0 4.5 s=range(3,n+1,2) 151 | 9 1 36 36.0 14.7 mroot = n ** 0.5 152 | 10 1 2 2.0 0.8 half=(n+1)/2-1 153 | 11 1 1 1.0 0.4 i=0 154 | 12 1 1 1.0 0.4 m=3 155 | 13 156 | 14 5 8 1.6 3.3 while m <= mroot: 157 | 15 4 4 1.0 1.6 if s[i]: 158 | 16 3 4 1.3 1.6 j=(m*m-3)/2 159 | 17 3 4 1.3 1.6 s[j]=0 160 | 18 31 33 1.1 13.5 while j ## Try It 177 | > 178 | > Replace the square root computation with `math.sqrt` 179 | > and compare the results from the previous profiler run. 180 | {: .challenge} 181 | 182 | > ## Calculating Pi 183 | > 184 | > 1. Write a program that calculates π using some approximation. 185 | > 2. Profile it. 186 | > 3. Try to make it faster. 187 | > 188 | > An example python script is given in `profiling_pi.py`. 189 | {: .challenge} 190 | 191 | ## Profiling within IPython 192 | > ## Loading line_profiler 193 | > `line_profiler` has to be loaded into the IPython/Jupyter notebook by running: 194 | > 195 | > ~~~ 196 | > `%load_ext line_profiler` 197 | > ~~~ 198 | > {: .source} 199 | {: .prereq} 200 | 201 | 202 | There are multiple ways to profile code within the IPython terminal depending how detailed you want to be 203 | These magic commands also work in Jupyter Notebooks 204 | 205 | * `%time` - time how long a statement takes to run 206 | ~~~ 207 | %time primes(100) 208 | ~~~ 209 | {: .python} 210 | ~~~ 211 | CPU times: user 21 µs, sys: 6 µs, total: 27 µs 212 | Wall time: 23.8 µs 213 | ~~~ 214 | {: .output} 215 | 216 | * `%timeit` - average time how long a cell takes to run over multiple runs 217 | ~~~ 218 | %timeit primes(100) 219 | ~~~ 220 | {: .python} 221 | ~~~ 222 | 100000 loops, best of 3: 8.11 µs per loop 223 | ~~~ 224 | {: .output} 225 | * `%prun` - time each function in a script 226 | ~~~ 227 | %prun primes(100) 228 | ~~~ 229 | {: .python} 230 | ~~~ 231 | 4 function calls in 0.000 seconds 232 | 233 | Ordered by: internal time 234 | 235 | ncalls tottime percall cumtime percall filename:lineno(function) 236 | 1 0.000 0.000 0.000 0.000 :1(primes) 237 | 1 0.000 0.000 0.000 0.000 :1() 238 | 1 0.000 0.000 0.000 0.000 {range} 239 | 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects} 240 | ~~~ 241 | {: .output} 242 | * `%lprun` - time each line in a function to run 243 | ~~~ 244 | %lprun -f primes primes(100) 245 | ~~~ 246 | {: .python} 247 | * `%lprun -f func1 -f func2 ....... ` to profile multiple functions 248 | ~~~ 249 | Timer unit: 1e-06 s 250 | 251 | Total time: 0.000122 s 252 | File: 253 | Function: primes at line 1 254 | 255 | Line # Hits Time Per Hit % Time Line Contents 256 | ============================================================== 257 | 1 def primes(n): 258 | 2 1 2 2.0 1.6 if n==2: 259 | 3 return [2] 260 | 4 1 1 1.0 0.8 elif n<2: 261 | 5 return [] 262 | 6 263 | 7 1 4 4.0 3.3 s=range(3,n+1,2) 264 | 8 1 2 2.0 1.6 mroot = n ** 0.5 265 | 9 1 1 1.0 0.8 half=(n+1)//2-1 266 | 10 1 0 0.0 0.0 i=0 267 | 11 1 0 0.0 0.0 m=3 268 | 12 269 | 13 5 4 0.8 3.3 while m <= mroot: 270 | 14 4 2 0.5 1.6 if s[i]: 271 | 15 3 3 1.0 2.5 j=(m*m-3)/2 272 | 16 3 4 1.3 3.3 s[j]=0 273 | 17 31 20 0.6 16.4 while jpng

2 |
3 |

png

4 |
5 |

png

6 |
7 |

png

8 |
9 |

png

10 |
11 |

png

12 |
13 |

png

14 |
15 |

png

16 |
17 |

png

18 |
19 |

png

20 |
21 |

png

22 |
23 |

png

24 |
25 |

png

26 |
27 |

png

28 |
29 |

png

30 |
31 |

png

32 |
33 |

png

34 |
35 |

png

36 |
37 |

png

38 |
39 |

png

40 |
41 |

png

42 |
43 |

png

44 | -------------------------------------------------------------------------------- /_includes/all_keypoints.html: -------------------------------------------------------------------------------- 1 | {% comment %} 2 | Display key points of all episodes for reference. 3 | {% endcomment %} 4 |

Key Points

5 | 6 | {% for episode in site.episodes %} 7 | {% unless episode.break %} 8 | 9 | 12 | 19 | 20 | {% endunless %} 21 | {% endfor %} 22 |
10 | {{ episode.title }} 11 | 13 |
    14 | {% for keypoint in episode.keypoints %} 15 |
  • {{ keypoint|markdownify }}
    • 16 | {% endfor %} 17 |
18 |
23 | -------------------------------------------------------------------------------- /_includes/carpentries.html: -------------------------------------------------------------------------------- 1 |
2 |
3 | Software Carpentry logo 4 |
5 |
6 | Since 1998, 7 | Software Carpentry 8 | has been teaching researchers in science, engineering, medicine, and related disciplines 9 | the computing skills they need to get more done in less time and with less pain. 10 | Its volunteer instructors have run hundreds of events 11 | for thousands of learners in the past two and a half years. 12 |
13 |
14 |
15 |
16 |
17 | Data Carpentry logo 18 |
19 |
20 | Data Carpentry develops and teaches workshops on the fundamental data skills needed to conduct research. 21 | Its target audience is researchers who have little to no prior computational experience, 22 | and its lessons are domain specific, 23 | building on learners' existing knowledge to enable them to quickly apply skills learned to their own research. 24 |
25 |
26 | 27 | 28 | -------------------------------------------------------------------------------- /_includes/episode_break.html: -------------------------------------------------------------------------------- 1 | {% comment %} 2 | Display a break's timings in a box similar to a learning episode's. 3 | {% endcomment %} 4 |
5 |

Overview

6 | 7 |
8 |
9 | Break: {{ page.break }} min 10 |
11 |
12 |
13 |
14 | 15 |
16 | -------------------------------------------------------------------------------- /_includes/episode_keypoints.html: -------------------------------------------------------------------------------- 1 |
2 |

Key Points

3 |
    4 | {% for keypoint in page.keypoints %} 5 |
  • {{ keypoint|markdownify }}
    • 6 | {% endfor %} 7 |
8 |
9 | -------------------------------------------------------------------------------- /_includes/episode_navbar.html: -------------------------------------------------------------------------------- 1 | {% comment %} 2 | Find previous and next episodes (if any). 3 | Including file must pass episode_navbar_title=true or ...=false to display episode title. 4 | {% endcomment %} 5 | {% for episode in site.episodes %} 6 | {% if episode.url == page.url %} 7 | {% unless forloop.first %} 8 | {% assign prev_episode = prev %} 9 | {% endunless %} 10 | {% unless forloop.last %} 11 | {% assign next_episode = site.episodes[forloop.index] %} 12 | {% endunless %} 13 | {% endif %} 14 | {% assign prev = episode %} 15 | {% endfor %} 16 | 17 | {% comment %} 18 | Display title and prev/next links. 19 | {% endcomment %} 20 |
21 |
22 |

23 | {% if prev_episode %} 24 | 25 | {% else %} 26 | 27 | {% endif %} 28 |

29 |
30 |
31 | {% if include.episode_navbar_title %} 32 |

{{ site.title }}

33 |

{{ page.title }}

34 | {% endif %} 35 |
36 |
37 |

38 | {% if next_episode %} 39 | 40 | {% else %} 41 | 42 | {% endif %} 43 |

44 |
45 |
46 | -------------------------------------------------------------------------------- /_includes/episode_overview.html: -------------------------------------------------------------------------------- 1 | {% comment %} 2 | Display an episode's timings and learning objectives. 3 | {% endcomment %} 4 |
5 |

Overview

6 | 7 |
8 |
9 | Teaching: {{ page.teaching }} min 10 |
11 | Exercises: {{ page.exercises }} min 12 |
13 |
14 | Questions 15 |
    16 | {% for question in page.questions %} 17 |
  • {{ question|markdownify }}
    • 18 | {% endfor %} 19 |
20 |
21 |
22 | 23 |
24 |
25 |
26 |
27 | Objectives 28 |
    29 | {% for objective in page.objectives %} 30 |
  • {{ objective|markdownify }}
    • 31 | {% endfor %} 32 |
33 |
34 |
35 | 36 |
37 | -------------------------------------------------------------------------------- /_includes/episode_title.html: -------------------------------------------------------------------------------- 1 | {% comment %} 2 | Find previous and next episodes (if any). 3 | {% endcomment %} 4 | {% for episode in site.episodes %} 5 | {% if episode.url == page.url %} 6 | {% unless forloop.first %} 7 | {% assign prev_episode = prev %} 8 | {% endunless %} 9 | {% unless forloop.last %} 10 | {% assign next_episode = site.episodes[forloop.index] %} 11 | {% endunless %} 12 | {% endif %} 13 | {% assign prev = episode %} 14 | {% endfor %} 15 | 16 | {% comment %} 17 | Display title and prev/next links. 18 | {% endcomment %} 19 |
20 |
21 |

22 | {% if prev_episode %} 23 | 24 | {% else %} 25 | 26 | {% endif %} 27 |

28 |
29 |
30 |

{{ site.title }}

31 |

{{ page.title }}

32 |
33 |
34 |

35 | {% if next_episode %} 36 | 37 | {% else %} 38 | 39 | {% endif %} 40 |

41 |
42 |
43 | -------------------------------------------------------------------------------- /_includes/javascript.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | -------------------------------------------------------------------------------- /_includes/lesson_footer.html: -------------------------------------------------------------------------------- 1 | 21 | -------------------------------------------------------------------------------- /_includes/main_title.html: -------------------------------------------------------------------------------- 1 |

{{ site.title }}{% if page.title %}: {{ page.title }}{% endif %}

2 | -------------------------------------------------------------------------------- /_includes/navbar.html: -------------------------------------------------------------------------------- 1 | 77 | -------------------------------------------------------------------------------- /_includes/syllabus.html: -------------------------------------------------------------------------------- 1 | {% comment %} 2 | Display syllabus in tabular form. 3 | Days are displayed if at least one episode has 'start = true'. 4 | {% endcomment %} 5 |
6 |

Schedule

7 | 8 | {% assign day = 0 %} 9 | {% assign multiday = false %} 10 | {% for episode in site.episodes %} 11 | {% if episode.start %}{% assign multiday = true %}{% break %}{% endif %} 12 | {% endfor %} 13 | {% assign current = site.start_time %} 14 | 15 | 16 | {% for episode in site.episodes %} 17 | {% if episode.start %} {% comment %} Starting a new day? {% endcomment %} 18 | {% assign day = day | plus: 1 %} 19 | {% if day > 1 %} {% comment %} If about to start day 2 or later, show finishing time for previous day {% endcomment %} 20 | {% assign hours = current | divided_by: 60 %} 21 | {% assign minutes = current | modulo: 60 %} 22 | 23 | {% if multiday %}{% endif %} 24 | 25 | 26 | 27 | 28 | {% endif %} 29 | {% assign current = site.start_time %} {% comment %}Re-set start time of this episode to general daily start time {% endcomment %} 30 | {% endif %} 31 | {% assign hours = current | divided_by: 60 %} 32 | {% assign minutes = current | modulo: 60 %} 33 | 34 | {% if multiday %}{% endif %} 35 | 36 | 39 | 53 | 54 | {% assign current = current | plus: episode.teaching | plus: episode.exercises | plus: episode.break %} 55 | {% endfor %} 56 | {% assign hours = current | divided_by: 60 %} 57 | {% assign minutes = current | modulo: 60 %} 58 | 59 | {% if multiday %}{% endif %} 60 | 61 | 62 | 63 | 64 |
{% if hours < 10 %}0{% endif %}{{ hours }}:{% if minutes < 10 %}0{% endif %}{{ minutes }}Finish
{% if episode.start %}Day {{ day }}{% endif %}{% if hours < 10 %}0{% endif %}{{ hours }}:{% if minutes < 10 %}0{% endif %}{{ minutes }} 37 | {{ episode.title }} 38 | 40 | {% if episode.break %} 41 | Break 42 | {% else %} 43 | {% if episode.questions %} 44 | {% for question in episode.questions %} 45 | {{question|markdownify|strip_html}} 46 | {% unless forloop.last %} 47 |
48 | {% endunless %} 49 | {% endfor %} 50 | {% endif %} 51 | {% endif %} 52 |
{% if hours < 10 %}0{% endif %}{{ hours }}:{% if minutes < 10 %}0{% endif %}{{ minutes }}Finish
65 | 66 |

67 | The actual schedule may vary slightly depending on the topics and exercises chosen by the instructor. 68 |

69 | 70 |
71 | -------------------------------------------------------------------------------- /_includes/workshop_ad.html: -------------------------------------------------------------------------------- 1 |
2 |
3 |
4 |

{{page.venue}}

5 |
6 |
7 |

{{page.humandate}}

8 |

{% if page.humantime %}{{page.humantime}}{% endif %}

9 |
10 |
11 |

12 | Instructors: 13 | {% if page.instructor %} 14 | {{page.instructor | join: ', ' %}} 15 | {% else %} 16 | to be announced. 17 | {% endif %} 18 |

19 | {% if page.helper %} 20 |

21 | Helpers: 22 | {{page.helper | join: ', ' %}} 23 |

24 | {% endif %} 25 |
26 |
27 |
28 |
29 |
30 | -------------------------------------------------------------------------------- /_includes/workshop_footer.html: -------------------------------------------------------------------------------- 1 | 11 | -------------------------------------------------------------------------------- /_layouts/base.html: -------------------------------------------------------------------------------- 1 | --- 2 | --- 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | {% if site.carpentry == "swc" %} 15 | 16 | {% endif %} 17 | {% if site.carpentry == "dc" %} 18 | 19 | {% endif %} 20 | 21 | 22 | 26 | {{ site.title }}{% if page.title %}: {{ page.title }}{% endif %} 27 | 28 | 29 |
30 | {% include navbar.html %} 31 | {{ content }} 32 | {% if site.kind == "workshop" %} 33 | {% include workshop_footer.html %} 34 | {% else %} 35 | {% include lesson_footer.html %} 36 | {% endif %} 37 |
38 | {% include javascript.html %} 39 | 40 | 41 | -------------------------------------------------------------------------------- /_layouts/break.html: -------------------------------------------------------------------------------- 1 | --- 2 | layout: base 3 | --- 4 | {% include episode_navbar.html episode_navbar_title=true %} 5 | {% include episode_break.html %} 6 | {{content}} 7 | {% include episode_navbar.html episode_navbar_title=false %} 8 | -------------------------------------------------------------------------------- /_layouts/episode.html: -------------------------------------------------------------------------------- 1 | --- 2 | layout: base 3 | --- 4 | {% include episode_navbar.html episode_navbar_title=true %} 5 | {% include episode_overview.html %} 6 | {{content}} 7 | {% include episode_keypoints.html %} 8 | {% include episode_navbar.html episode_navbar_title=false %} 9 | -------------------------------------------------------------------------------- /_layouts/lesson.html: -------------------------------------------------------------------------------- 1 | --- 2 | layout: base 3 | --- 4 | {% include main_title.html %} 5 | {{ content }} 6 | {% include syllabus.html %} 7 | -------------------------------------------------------------------------------- /_layouts/page.html: -------------------------------------------------------------------------------- 1 | --- 2 | layout: base 3 | --- 4 | {% include main_title.html %} 5 | {{content}} 6 | -------------------------------------------------------------------------------- /_layouts/reference.html: -------------------------------------------------------------------------------- 1 | --- 2 | layout: page 3 | title: "Reference" 4 | --- 5 | {% include all_keypoints.html %} 6 | {{content}} 7 | -------------------------------------------------------------------------------- /_layouts/workshop.html: -------------------------------------------------------------------------------- 1 | --- 2 | --- 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | {% if page.redirect %} 21 | 22 | {% endif %} 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | {% if site.carpentry == "swc" %} 32 | 33 | {% endif %} 34 | {% if site.carpentry == "dc" %} 35 | 36 | {% endif %} 37 | 38 | 39 | 43 | {{ page.venue }}: {{ page.humandate }} 44 | 45 | 46 |
47 | {% include navbar.html %} 48 | {% include workshop_ad.html %} 49 | {{ content }} 50 | {% include workshop_footer.html %} 51 |
52 | {% include javascript.html %} 53 | 54 | 55 | -------------------------------------------------------------------------------- /assets/css/lesson.scss: -------------------------------------------------------------------------------- 1 | --- 2 | --- 3 | 4 | //---------------------------------------- 5 | // Colors. 6 | //---------------------------------------- 7 | 8 | // branding 9 | $color-brand: #2b3990 !default; 10 | 11 | // code boxes 12 | $color-error: #bd2c00 !default; 13 | $color-output: #303030 !default; 14 | $color-source: #6e5494 !default; 15 | 16 | // blockquotes 17 | $color-callout: #f4fd9c !default; 18 | $color-challenge: #eec275 !default; 19 | $color-checklist: #dfd2a0 !default; 20 | $color-discussion: #eec275 !default; 21 | $color-keypoints: #7ae78e !default; 22 | $color-objectives: #daee84 !default; 23 | $color-prereq: #9cd6dc !default; 24 | $color-solution: #ded4b9 !default; 25 | $color-testimonial: #fc8dc1 !default; 26 | 27 | //---------------------------------------- 28 | // Specialized code blocks. 29 | //---------------------------------------- 30 | 31 | @mixin cdSetup($color) { 32 | color: $color; 33 | border-left: solid 5px $color; 34 | margin-bottom: 0px; 35 | } 36 | 37 | .error { @include cdSetup($color-error); } 38 | .output { @include cdSetup($color-output); } 39 | .source { @include cdSetup($color-source); } 40 | 41 | .bash { @include cdSetup($color-source); } 42 | .make { @include cdSetup($color-source); } 43 | .matlab { @include cdSetup($color-source); } 44 | .python { @include cdSetup($color-source); } 45 | .r { @include cdSetup($color-source); } 46 | .sql { @include cdSetup($color-source); } 47 | 48 | //---------------------------------------- 49 | // Specialized blockquote environments for learning objectives, callouts, etc. 50 | //---------------------------------------- 51 | 52 | $codeblock-padding: 5px !default; 53 | 54 | @mixin bkSetup($color, $glyph) { 55 | 56 | $gradientcolor1: $color; 57 | $gradientcolor2: scale-color($color, $lightness: 10%); 58 | 59 | padding-left: $codeblock-padding; 60 | padding-top: 0; 61 | padding-bottom: 0; 62 | padding-right: 0; 63 | border: 1px solid; 64 | border-color: $color; 65 | padding-bottom: $codeblock-padding; 66 | 67 | h2 { 68 | padding-top: $codeblock-padding; 69 | padding-bottom: $codeblock-padding; 70 | font-size: 20px; 71 | background: linear-gradient(to bottom, $gradientcolor1, $gradientcolor2); 72 | border-color: $color; 73 | margin-top: 0px; 74 | margin-left: -$codeblock-padding; // to move back to the left margin of the enclosing blockquote 75 | } 76 | h2:before { 77 | font-family: 'Glyphicons Halflings'; 78 | content: $glyph; 79 | float: left; 80 | padding-left: $codeblock-padding; 81 | padding-right: $codeblock-padding; 82 | display: inline-block; 83 | -webkit-font-smoothing: antialiased; 84 | } 85 | 86 | } 87 | 88 | .callout{ @include bkSetup($color-callout, "\e146"); } 89 | .challenge{ @include bkSetup($color-challenge, "\270f"); } 90 | .checklist{ @include bkSetup($color-checklist, "\e067"); } 91 | .discussion{ @include bkSetup($color-discussion, "\e123"); } 92 | .keypoints{ @include bkSetup($color-keypoints, "\e101"); } 93 | .objectives{ @include bkSetup($color-objectives, "\e085"); } 94 | .prereq{ @include bkSetup($color-prereq, "\e124"); } 95 | .solution{ @include bkSetup($color-solution, "\e105"); } 96 | .testimonial{ @include bkSetup($color-testimonial, "\e143"); } 97 | 98 | //---------------------------------------- 99 | // Override Bootstrap settings. 100 | //---------------------------------------- 101 | 102 | code { 103 | padding: 0 0; 104 | color: inherit; 105 | background-color: inherit; 106 | } 107 | 108 | img { 109 | max-width: 100%; 110 | } 111 | 112 | //---------------------------------------- 113 | // Miscellaneous. 114 | //---------------------------------------- 115 | 116 | .maintitle { 117 | text-align: center; 118 | } 119 | 120 | .footertext { 121 | text-align: center; 122 | } 123 | 124 | img.navbar-logo { 125 | height: 40px; // synchronize with height of navbar 126 | padding-top: 5px; 127 | padding-right: 10px; 128 | } 129 | 130 | div.branding { 131 | color: $color-brand; 132 | } 133 | 134 | ul, 135 | ol { 136 | padding-left: 1em; 137 | } 138 | 139 | span.fold-unfold { 140 | margin-left: 1em; 141 | opacity: 0.5; 142 | } 143 | -------------------------------------------------------------------------------- /assets/fonts/glyphicons-halflings-regular.eot: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/swcarpentry/python-second-language/074c1d9aa7424104e8a5006623e91eeb6545bbf5/assets/fonts/glyphicons-halflings-regular.eot -------------------------------------------------------------------------------- /assets/fonts/glyphicons-halflings-regular.ttf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/swcarpentry/python-second-language/074c1d9aa7424104e8a5006623e91eeb6545bbf5/assets/fonts/glyphicons-halflings-regular.ttf -------------------------------------------------------------------------------- /assets/fonts/glyphicons-halflings-regular.woff: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/swcarpentry/python-second-language/074c1d9aa7424104e8a5006623e91eeb6545bbf5/assets/fonts/glyphicons-halflings-regular.woff -------------------------------------------------------------------------------- /assets/fonts/glyphicons-halflings-regular.woff2: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/swcarpentry/python-second-language/074c1d9aa7424104e8a5006623e91eeb6545bbf5/assets/fonts/glyphicons-halflings-regular.woff2 -------------------------------------------------------------------------------- /assets/img/dc-icon-black.svg: -------------------------------------------------------------------------------- 1 | 2 | 16 | 18 | 19 | 21 | image/svg+xml 22 | 24 | 25 | 26 | 27 | 28 | 30 | 54 | 59 | 64 | 69 | 74 | 75 | -------------------------------------------------------------------------------- /assets/img/swc-icon-blue.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | 64 | 65 | 66 | 67 | 68 | 69 | 70 | 71 | -------------------------------------------------------------------------------- /assets/img/swc-logo-blue.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/swcarpentry/python-second-language/074c1d9aa7424104e8a5006623e91eeb6545bbf5/assets/img/swc-logo-blue.png -------------------------------------------------------------------------------- /assets/img/swc-logo-white.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/swcarpentry/python-second-language/074c1d9aa7424104e8a5006623e91eeb6545bbf5/assets/img/swc-logo-white.png -------------------------------------------------------------------------------- /assets/js/lesson.js: -------------------------------------------------------------------------------- 1 | // Make all tables striped by default. 2 | $("table").addClass("table table-striped"); 3 | 4 | 5 | // Handle foldable challenges and solutions (on click and at start). 6 | $(".challenge,.discussion,.solution").click(function(event) { 7 | var trigger = $(event.target).has(".fold-unfold").size() > 0 8 | || $(event.target).filter(".fold-unfold").size() > 0; 9 | if (trigger) { 10 | $(">*:not(h2)", this).toggle(400); 11 | $(">h2>span.fold-unfold", this).toggleClass("glyphicon-collapse-down glyphicon-collapse-up"); 12 | event.stopPropagation(); 13 | } 14 | }); 15 | $(".challenge,.discussion,.solution").each(function() { 16 | $(">*:not(h2)", this).toggle(); 17 | var h2 = $("h2:first", this); 18 | h2.append(""); 19 | }); 20 | 21 | 22 | // Handle searches. 23 | // Relies on document having 'meta' element with name 'search-domain'. 24 | function google_search() { 25 | var query = document.getElementById("google-search").value; 26 | var domain = $("meta[name=search-domain]").attr("value"); 27 | window.open("https://www.google.com/search?q=" + query + "+site:" + domain); 28 | } 29 | -------------------------------------------------------------------------------- /bin/chunk-options.R: -------------------------------------------------------------------------------- 1 | # These settings control the behavior of all chunks in the novice R materials. 2 | # For example, to generate the lessons with all the output hidden, simply change 3 | # `results` from "markup" to "hide". 4 | # For more information on available chunk options, see 5 | # http://yihui.name/knitr/options#chunk_options 6 | 7 | library("knitr") 8 | 9 | fix_fig_path <- function(pth) file.path("..", pth) 10 | 11 | 12 | ## We set the path for the figures globally below, so if we want to 13 | ## customize it for individual episodes, we can append a prefix to the 14 | ## global path. For instance, if we call knitr_fig_path("01-") in the 15 | ## first episode of the lesson, it will generate the figures in 16 | ## `fig/rmd-01-` 17 | knitr_fig_path <- function(prefix) { 18 | new_path <- paste0(opts_chunk$get("fig.path"), 19 | prefix) 20 | opts_chunk$set(fig.path = new_path) 21 | } 22 | 23 | ## We use the rmd- prefix for the figures generated by the lssons so 24 | ## they can be easily identified and deleted by `make clean-rmd`. The 25 | ## working directory when the lessons are generated is the root so the 26 | ## figures need to be saved in fig/, but when the site is generated, 27 | ## the episodes will be one level down. We fix the path using the 28 | ## `fig.process` option. 29 | 30 | opts_chunk$set(tidy = FALSE, results = "markup", comment = NA, 31 | fig.align = "center", fig.path = "fig/rmd-", 32 | fig.process = fix_fig_path) 33 | 34 | # The hooks below add html tags to the code chunks and their output so that they 35 | # are properly formatted when the site is built. 36 | 37 | hook_in <- function(x, options) { 38 | stringr::str_c("\n\n~~~\n", 39 | paste0(x, collapse="\n"), 40 | "\n~~~\n{: .r}\n\n") 41 | } 42 | 43 | hook_out <- function(x, options) { 44 | x <- gsub("\n$", "", x) 45 | stringr::str_c("\n\n~~~\n", 46 | paste0(x, collapse="\n"), 47 | "\n~~~\n{: .output}\n\n") 48 | } 49 | 50 | hook_error <- function(x, options) { 51 | x <- gsub("\n$", "", x) 52 | stringr::str_c("\n\n~~~\n", 53 | paste0(x, collapse="\n"), 54 | "\n~~~\n{: .error}\n\n") 55 | } 56 | 57 | knit_hooks$set(source = hook_in, output = hook_out, warning = hook_error, 58 | error = hook_error, message = hook_out) 59 | -------------------------------------------------------------------------------- /bin/extract_figures.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | from __future__ import print_function 4 | import sys 5 | import os 6 | import glob 7 | from optparse import OptionParser 8 | 9 | from util import Reporter, read_markdown, IMAGE_FILE_SUFFIX 10 | 11 | def main(): 12 | """Main driver.""" 13 | 14 | args = parse_args() 15 | images = [] 16 | for filename in args.filenames: 17 | images += get_images(args.parser, filename) 18 | save(sys.stdout, images) 19 | 20 | 21 | def parse_args(): 22 | """Parse command-line arguments.""" 23 | 24 | parser = OptionParser() 25 | parser.add_option('-p', '--parser', 26 | default=None, 27 | dest='parser', 28 | help='path to Markdown parser') 29 | 30 | args, extras = parser.parse_args() 31 | require(args.parser is not None, 32 | 'Path to Markdown parser not provided') 33 | require(extras, 34 | 'No filenames specified') 35 | 36 | args.filenames = extras 37 | return args 38 | 39 | 40 | def get_filenames(source_dir): 41 | """Get all filenames to be searched for images.""" 42 | 43 | return glob.glob(os.path.join(source_dir, '*.md')) 44 | 45 | 46 | def get_images(parser, filename): 47 | """Extract all images from file.""" 48 | 49 | content = read_markdown(parser, filename) 50 | result = [] 51 | find_image_nodes(content['doc'], result) 52 | find_image_links(content['doc'], result) 53 | return result 54 | 55 | 56 | def find_image_nodes(doc, result): 57 | """Find all nested nodes representing images.""" 58 | 59 | if (doc['type'] == 'img') or \ 60 | ((doc['type'] == 'html_element') and (doc['value'] == 'img')): 61 | alt = doc['attr'].get('alt', '') 62 | result.append({'alt': alt, 'src': doc['attr']['src']}) 63 | else: 64 | for child in doc.get('children', []): 65 | find_image_nodes(child, result) 66 | 67 | 68 | def find_image_links(doc, result): 69 | """Find all links to files in the 'fig' directory.""" 70 | 71 | if ((doc['type'] == 'a') and ('attr' in doc) and ('href' in doc['attr'])) \ 72 | or \ 73 | ((doc['type'] == 'html_element') and (doc['value'] == 'a') and ('href' in doc['attr'])): 74 | path = doc['attr']['href'] 75 | if os.path.splitext(path)[1].lower() in IMAGE_FILE_SUFFIX: 76 | result.append({'alt':'', 'src': doc['attr']['href']}) 77 | else: 78 | for child in doc.get('children', []): 79 | find_image_links(child, result) 80 | 81 | 82 | def save(stream, images): 83 | """Save results as Markdown.""" 84 | 85 | text = '\n
\n'.join(['

{0}

'.format(img['alt'], img['src']) for img in images]) 86 | print(text, file=stream) 87 | 88 | 89 | def require(condition, message): 90 | """Fail if condition not met.""" 91 | 92 | if not condition: 93 | print(message, file=sys.stderr) 94 | sys.exit(1) 95 | 96 | 97 | if __name__ == '__main__': 98 | main() 99 | -------------------------------------------------------------------------------- /bin/generate_md_episodes.R: -------------------------------------------------------------------------------- 1 | generate_md_episodes <- function() { 2 | 3 | if (require("knitr") && packageVersion("knitr") < '1.9.19') 4 | stop("knitr must be version 1.9.20 or higher") 5 | 6 | if (!require("stringr")) 7 | stop("The package stringr is required for generating the lessons.") 8 | 9 | if (require("checkpoint")) { 10 | required_pkgs <- 11 | checkpoint:::projectScanPackages(project = "_episodes_rmd", 12 | verbose=FALSE, use.knitr = TRUE)$pkgs 13 | } else { 14 | stop("The checkpoint package is required to build the lessons.") 15 | } 16 | 17 | missing_pkgs <- required_pkgs[!(required_pkgs %in% rownames(installed.packages()))] 18 | 19 | if (length(missing_pkgs)) { 20 | message("Installing missing required packages: ", 21 | paste(missing_pkgs, collapse=", ")) 22 | install.packages(missing_pkgs) 23 | } 24 | 25 | ## find all the Rmd files, and generates the paths for their respective outputs 26 | src_rmd <- list.files(pattern = "??-*.Rmd$", path = "_episodes_rmd", full.names = TRUE) 27 | dest_md <- file.path("_episodes", gsub("Rmd$", "md", basename(src_rmd))) 28 | 29 | ## knit the Rmd into markdown 30 | mapply(function(x, y) { 31 | knitr::knit(x, output = y) 32 | }, src_rmd, dest_md) 33 | 34 | } 35 | 36 | generate_md_episodes() 37 | -------------------------------------------------------------------------------- /bin/knit_lessons.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # Only try running R to translate files if there are some files present. 4 | # The Makefile passes in the names of files. 5 | 6 | if [ $# -ne 0 ] ; then 7 | Rscript -e "source('bin/generate_md_episodes.R')" 8 | fi 9 | -------------------------------------------------------------------------------- /bin/lesson_initialize.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | """Initialize a newly-created repository.""" 4 | 5 | 6 | from __future__ import print_function 7 | import sys 8 | import os 9 | 10 | ROOT_AUTHORS = '''\ 11 | FIXME: list authors' names and email addresses. 12 | ''' 13 | 14 | ROOT_CITATION = '''\ 15 | FIXME: describe how to cite this lesson. 16 | ''' 17 | 18 | ROOT_CONTRIBUTING_MD = '''\ 19 | # Contributing 20 | 21 | [Software Carpentry][swc-site] and [Data Carpentry][dc-site] are open source projects, 22 | and we welcome contributions of all kinds: 23 | new lessons, 24 | fixes to existing material, 25 | bug reports, 26 | and reviews of proposed changes are all welcome. 27 | 28 | ## Contributor Agreement 29 | 30 | By contributing, 31 | you agree that we may redistribute your work under [our license](LICENSE.md). 32 | In exchange, 33 | we will address your issues and/or assess your change proposal as promptly as we can, 34 | and help you become a member of our community. 35 | Everyone involved in [Software Carpentry][swc-site] and [Data Carpentry][dc-site] 36 | agrees to abide by our [code of conduct](CONDUCT.md). 37 | 38 | ## How to Contribute 39 | 40 | The easiest way to get started is to file an issue 41 | to tell us about a spelling mistake, 42 | some awkward wording, 43 | or a factual error. 44 | This is a good way to introduce yourself 45 | and to meet some of our community members. 46 | 47 | 1. If you do not have a [GitHub][github] account, 48 | you can [send us comments by email][contact]. 49 | However, 50 | we will be able to respond more quickly if you use one of the other methods described below. 51 | 52 | 2. If you have a [GitHub][github] account, 53 | or are willing to [create one][github-join], 54 | but do not know how to use Git, 55 | you can report problems or suggest improvements by [creating an issue][issues]. 56 | This allows us to assign the item to someone 57 | and to respond to it in a threaded discussion. 58 | 59 | 3. If you are comfortable with Git, 60 | and would like to add or change material, 61 | you can submit a pull request (PR). 62 | Instructions for doing this are [included below](#using-github). 63 | 64 | ## Where to Contribute 65 | 66 | 1. If you wish to change this lesson, 67 | please work in , 68 | which can be viewed at . 69 | 70 | 2. If you wish to change the example lesson, 71 | please work in , 72 | which documents the format of our lessons 73 | and can be viewed at . 74 | 75 | 3. If you wish to change the template used for workshop websites, 76 | please work in . 77 | The home page of that repository explains how to set up workshop websites, 78 | while the extra pages in 79 | provide more background on our design choices. 80 | 81 | 4. If you wish to change CSS style files, tools, 82 | or HTML boilerplate for lessons or workshops stored in `_includes` or `_layouts`, 83 | please work in . 84 | 85 | ## What to Contribute 86 | 87 | There are many ways to contribute, 88 | from writing new exercises and improving existing ones 89 | to updating or filling in the documentation 90 | and and submitting [bug reports][issues] 91 | about things that don't work, aren't clear, or are missing. 92 | If you are looking for ideas, 93 | please see [the list of issues for this repository][issues], 94 | or the issues for [Data Carpentry][dc-issues] 95 | and [Software Carpentry][swc-issues] projects. 96 | 97 | Comments on issues and reviews of pull requests are just as welcome: 98 | we are smarter together than we are on our own. 99 | Reviews from novices and newcomers are particularly valuable: 100 | it's easy for people who have been using these lessons for a while 101 | to forget how impenetrable some of this material can be, 102 | so fresh eyes are always welcome. 103 | 104 | ## What *Not* to Contribute 105 | 106 | Our lessons already contain more material than we can cover in a typical workshop, 107 | so we are usually *not* looking for more concepts or tools to add to them. 108 | As a rule, 109 | if you want to introduce a new idea, 110 | you must (a) estimate how long it will take to teach 111 | and (b) explain what you would take out to make room for it. 112 | The first encourages contributors to be honest about requirements; 113 | the second, to think hard about priorities. 114 | 115 | We are also not looking for exercises or other material that only run on one platform. 116 | Our workshops typically contain a mixture of Windows, Mac OS X, and Linux users; 117 | in order to be usable, 118 | our lessons must run equally well on all three. 119 | 120 | ## Using GitHub 121 | 122 | If you choose to contribute via GitHub, 123 | you may want to look at 124 | [How to Contribute to an Open Source Project on GitHub][how-contribute]. 125 | In brief: 126 | 127 | 1. The published copy of the lesson is in the `gh-pages` branch of the repository 128 | (so that GitHub will regenerate it automatically). 129 | Please create all branches from that, 130 | and merge the [master repository][repo]'s `gh-pages` branch into your `gh-pages` branch 131 | before starting work. 132 | Please do *not* work directly in your `gh-pages` branch, 133 | since that will make it difficult for you to work on other contributions. 134 | 135 | 2. We use [GitHub flow][github-flow] to manage changes: 136 | 1. Create a new branch in your desktop copy of this repository for each significant change. 137 | 2. Commit the change in that branch. 138 | 3. Push that branch to your fork of this repository on GitHub. 139 | 4. Submit a pull request from that branch to the [master repository][repo]. 140 | 5. If you receive feedback, 141 | make changes on your desktop and push to your branch on GitHub: 142 | the pull request will update automatically. 143 | 144 | Each lesson has two maintainers who review issues and pull requests 145 | or encourage others to do so. 146 | The maintainers are community volunteers, 147 | and have final say over what gets merged into the lesson. 148 | 149 | ## Other Resources 150 | 151 | General discussion of [Software Carpentry][swc-site] and [Data Carpentry][dc-site] 152 | happens on the [discussion mailing list][discuss-list], 153 | which everyone is welcome to join. 154 | You can also [reach us by email][contact]. 155 | 156 | [contact]: mailto:admin@software-carpentry.org 157 | [dc-issues]: https://github.com/issues?q=user%3Adatacarpentry 158 | [dc-lessons]: http://datacarpentry.org/lessons/ 159 | [dc-site]: http://datacarpentry.org/ 160 | [discuss-list]: http://lists.software-carpentry.org/listinfo/discuss 161 | [example-site]: https://swcarpentry.github.io/lesson-example/ 162 | [github]: http://github.com 163 | [github-flow]: https://guides.github.com/introduction/flow/ 164 | [github-join]: https://github.com/join 165 | [how-contribute]: https://egghead.io/series/how-to-contribute-to-an-open-source-project-on-github 166 | [issues]: https://github.com/swcarpentry/FIXME/issues/ 167 | [repo]: https://github.com/swcarpentry/FIXME/ 168 | [swc-issues]: https://github.com/issues?q=user%3Aswcarpentry 169 | [swc-lessons]: http://software-carpentry.org/lessons/ 170 | [swc-site]: http://software-carpentry.org/ 171 | ''' 172 | 173 | ROOT_CONFIG_YML = '''\ 174 | #------------------------------------------------------------ 175 | # Values for this lesson. 176 | #------------------------------------------------------------ 177 | 178 | # Which carpentry is this ("swc" or "dc")? 179 | carpentry: "swc" 180 | 181 | # Overall title for pages. 182 | title: "Lesson Title" 183 | 184 | # Contact email address. 185 | email: lessons@software-carpentry.org 186 | 187 | #------------------------------------------------------------ 188 | # Generic settings (should not need to change). 189 | #------------------------------------------------------------ 190 | 191 | # What kind of thing is this ("workshop" or "lesson")? 192 | kind: "lesson" 193 | 194 | # Magic to make URLs resolve both locally and on GitHub. 195 | # See https://help.github.com/articles/repository-metadata-on-github-pages/. 196 | repository: / 197 | 198 | # Sites. 199 | amy_site: "https://amy.software-carpentry.org/workshops" 200 | dc_site: "http://datacarpentry.org" 201 | swc_github: "https://github.com/swcarpentry" 202 | swc_site: "https://software-carpentry.org" 203 | swc_pages: "https://swcarpentry.github.io" 204 | template_repo: "https://github.com/swcarpentry/styles" 205 | example_repo: "https://github.com/swcarpentry/lesson-example" 206 | example_site: "https://swcarpentry.github.com/lesson-example" 207 | workshop_repo: "https://github.com/swcarpentry/workshop-template" 208 | workshop_site: "https://swcarpentry.github.io/workshop-template" 209 | training_site: "https://swcarpentry.github.io/instructor-training" 210 | 211 | # Surveys. 212 | pre_survey: "https://www.surveymonkey.com/r/swc_pre_workshop_v1?workshop_id=" 213 | post_survey: "https://www.surveymonkey.com/r/swc_post_workshop_v1?workshop_id=" 214 | 215 | # Start time in minutes (0 to be clock-independent, 540 to show a start at 09:00 am). 216 | start_time: 0 217 | 218 | # Specify that things in the episodes collection should be output. 219 | collections: 220 | episodes: 221 | output: true 222 | permalink: /:path/ 223 | extras: 224 | output: true 225 | 226 | # Set the default layout for things in the episodes collection. 227 | defaults: 228 | - values: 229 | root: .. 230 | - scope: 231 | path: "" 232 | type: episodes 233 | values: 234 | layout: episode 235 | 236 | # Files and directories that are not to be copied. 237 | exclude: 238 | - Makefile 239 | - bin 240 | 241 | # Turn off built-in syntax highlighting. 242 | highlighter: false 243 | ''' 244 | 245 | ROOT_INDEX_MD = '''\ 246 | --- 247 | layout: lesson 248 | root: . 249 | --- 250 | FIXME: home page introduction 251 | 252 | > ## Prerequisites 253 | > 254 | > FIXME 255 | {: .prereq} 256 | ''' 257 | 258 | ROOT_REFERENCE_MD = '''\ 259 | --- 260 | layout: reference 261 | permalink: /reference/ 262 | --- 263 | 264 | ## Glossary 265 | 266 | FIXME 267 | ''' 268 | 269 | ROOT_SETUP_MD = '''\ 270 | --- 271 | layout: page 272 | title: Setup 273 | permalink: /setup/ 274 | --- 275 | FIXME 276 | ''' 277 | 278 | EPISODES_INTRODUCTION_MD = '''\ 279 | --- 280 | title: "Introduction" 281 | teaching: 0 282 | exercises: 0 283 | questions: 284 | - "Key question" 285 | objectives: 286 | - "First objective." 287 | keypoints: 288 | - "First key point." 289 | --- 290 | ''' 291 | 292 | EXTRAS_ABOUT_MD = '''\ 293 | --- 294 | layout: page 295 | title: About 296 | permalink: /about/ 297 | --- 298 | {% include carpentries.html %} 299 | ''' 300 | 301 | EXTRAS_DISCUSS_MD = '''\ 302 | --- 303 | layout: page 304 | title: Discussion 305 | permalink: /discuss/ 306 | --- 307 | FIXME 308 | ''' 309 | 310 | EXTRAS_FIGURES_MD = '''\ 311 | --- 312 | layout: page 313 | title: Figures 314 | permalink: /figures/ 315 | --- 316 | {% include all_figures.html %} 317 | ''' 318 | 319 | EXTRAS_GUIDE_MD = '''\ 320 | --- 321 | layout: page 322 | title: "Instructor Notes" 323 | permalink: /guide/ 324 | --- 325 | FIXME 326 | ''' 327 | 328 | INCLUDES_ALL_FIGURES_HTML = '''\ 329 | 330 | ''' 331 | 332 | BOILERPLATE = ( 333 | ('AUTHORS', ROOT_AUTHORS), 334 | ('CITATION', ROOT_CITATION), 335 | ('CONTRIBUTING.md', ROOT_CONTRIBUTING_MD), 336 | ('_config.yml', ROOT_CONFIG_YML), 337 | ('index.md', ROOT_INDEX_MD), 338 | ('reference.md', ROOT_REFERENCE_MD), 339 | ('setup.md', ROOT_SETUP_MD), 340 | ('_episodes/01-introduction.md', EPISODES_INTRODUCTION_MD), 341 | ('_extras/about.md', EXTRAS_ABOUT_MD), 342 | ('_extras/discuss.md', EXTRAS_DISCUSS_MD), 343 | ('_extras/figures.md', EXTRAS_FIGURES_MD), 344 | ('_extras/guide.md', EXTRAS_GUIDE_MD), 345 | ('_includes/all_figures.html', INCLUDES_ALL_FIGURES_HTML) 346 | ) 347 | 348 | 349 | def main(): 350 | """Check for collisions, then create.""" 351 | 352 | # Check. 353 | errors = False 354 | for (path, _) in BOILERPLATE: 355 | if os.path.exists(path): 356 | print('Warning: {0} already exists.'.format(path), file=sys.stderr) 357 | errors = True 358 | if errors: 359 | print('**Exiting without creating files.**', file=sys.stderr) 360 | sys.exit(1) 361 | 362 | # Create. 363 | for (path, content) in BOILERPLATE: 364 | with open(path, 'w') as writer: 365 | writer.write(content) 366 | 367 | 368 | if __name__ == '__main__': 369 | main() 370 | -------------------------------------------------------------------------------- /bin/markdown_ast.rb: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env ruby 2 | 3 | # Use Kramdown parser to produce AST for Markdown document. 4 | 5 | require "kramdown" 6 | require "json" 7 | 8 | markdown = STDIN.read() 9 | doc = Kramdown::Document.new(markdown) 10 | tree = doc.to_hash_a_s_t 11 | puts JSON.pretty_generate(tree) 12 | -------------------------------------------------------------------------------- /bin/repo_check.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | """ 4 | Check repository settings. 5 | """ 6 | 7 | from __future__ import print_function 8 | import sys 9 | import os 10 | from subprocess import Popen, PIPE 11 | import re 12 | from optparse import OptionParser 13 | 14 | from util import Reporter, load_yaml, require 15 | 16 | # Import this way to produce a more useful error message. 17 | try: 18 | import requests 19 | except ImportError: 20 | print('Unable to import requests module: please install requests', file=sys.stderr) 21 | sys.exit(1) 22 | 23 | 24 | # Pattern to match Git command-line output for remotes => (user name, project name). 25 | P_GIT_REMOTE = re.compile(r'upstream\s+[^:]+:([^/]+)/([^.]+)\.git\s+\(fetch\)') 26 | 27 | # Repository URL format string. 28 | F_REPO_URL = 'https://github.com/{0}/{1}/' 29 | 30 | # Pattern to match repository URLs => (user name, project name) 31 | P_REPO_URL = re.compile(r'https?://github\.com/([^.]+)/([^/]+)/?') 32 | 33 | # API URL format string. 34 | F_API_URL = 'https://api.github.com/repos/{0}/{1}/labels' 35 | 36 | # Expected labels and colors. 37 | EXPECTED = { 38 | 'bug' : 'bd2c00', 39 | 'discussion' : 'fc8dc1', 40 | 'enhancement' : '9cd6dc', 41 | 'help-wanted' : 'f4fd9c', 42 | 'instructor-training' : '6e5494', 43 | 'newcomer-friendly' : 'eec275', 44 | 'question' : '808040', 45 | 'template-and-tools' : '2b3990', 46 | 'work-in-progress' : '7ae78e' 47 | } 48 | 49 | 50 | def main(): 51 | """ 52 | Main driver. 53 | """ 54 | 55 | args = parse_args() 56 | reporter = Reporter() 57 | repo_url = get_repo_url(args.source_dir, args.repo_url) 58 | check_labels(reporter, repo_url) 59 | reporter.report() 60 | 61 | 62 | def parse_args(): 63 | """ 64 | Parse command-line arguments. 65 | """ 66 | 67 | parser = OptionParser() 68 | parser.add_option('-r', '--repo', 69 | default=None, 70 | dest='repo_url', 71 | help='repository URL') 72 | parser.add_option('-s', '--source', 73 | default=os.curdir, 74 | dest='source_dir', 75 | help='source directory') 76 | 77 | args, extras = parser.parse_args() 78 | require(not extras, 79 | 'Unexpected trailing command-line arguments "{0}"'.format(extras)) 80 | 81 | return args 82 | 83 | 84 | def get_repo_url(source_dir, repo_url): 85 | """ 86 | Figure out which repository to query. 87 | """ 88 | 89 | # Explicitly specified. 90 | if repo_url is not None: 91 | return repo_url 92 | 93 | # Guess. 94 | cmd = 'git remote -v' 95 | p = Popen(cmd, shell=True, stdin=PIPE, stdout=PIPE, close_fds=True, universal_newlines=True) 96 | stdout_data, stderr_data = p.communicate() 97 | stdout_data = stdout_data.split('\n') 98 | matches = [P_GIT_REMOTE.match(line) for line in stdout_data] 99 | matches = [m for m in matches if m is not None] 100 | require(len(matches) == 1, 101 | 'Unexpected output from git remote command: "{0}"'.format(matches)) 102 | 103 | username = matches[0].group(1) 104 | require(username, 'empty username in git remote output {0}'.format(matches[0])) 105 | 106 | project_name = matches[0].group(2) 107 | require(username, 'empty project name in git remote output {0}'.format(matches[0])) 108 | 109 | url = F_REPO_URL.format(username, project_name) 110 | return url 111 | 112 | 113 | def check_labels(reporter, repo_url): 114 | """ 115 | Check labels in repository. 116 | """ 117 | 118 | actual = get_labels(repo_url) 119 | extra = set(actual.keys()) - set(EXPECTED.keys()) 120 | 121 | reporter.check(not extra, 122 | None, 123 | 'Extra label(s) in repository {0}: {1}', 124 | repo_url, ', '.join(sorted(extra))) 125 | 126 | missing = set(EXPECTED.keys()) - set(actual.keys()) 127 | reporter.check(not missing, 128 | None, 129 | 'Missing label(s) in repository {0}: {1}', 130 | repo_url, ', '.join(sorted(missing))) 131 | 132 | overlap = set(EXPECTED.keys()).intersection(set(actual.keys())) 133 | for name in sorted(overlap): 134 | reporter.check(EXPECTED[name] == actual[name], 135 | None, 136 | 'Color mis-match for label {0} in {1}: expected {2}, found {3}', 137 | name, repo_url, EXPECTED[name], actual[name]) 138 | 139 | 140 | def get_labels(repo_url): 141 | """ 142 | Get actual labels from repository. 143 | """ 144 | 145 | m = P_REPO_URL.match(repo_url) 146 | require(m, 'repository URL {0} does not match expected pattern'.format(repo_url)) 147 | 148 | username = m.group(1) 149 | require(username, 'empty username in repository URL {0}'.format(repo_url)) 150 | 151 | project_name = m.group(2) 152 | require(username, 'empty project name in repository URL {0}'.format(repo_url)) 153 | 154 | url = F_API_URL.format(username, project_name) 155 | r = requests.get(url) 156 | require(r.status_code == 200, 157 | 'Request for {0} failed with {1}'.format(url, r.status_code)) 158 | 159 | result = {} 160 | for entry in r.json(): 161 | result[entry['name']] = entry['color'] 162 | return result 163 | 164 | 165 | if __name__ == '__main__': 166 | main() 167 | -------------------------------------------------------------------------------- /bin/test_lesson_check.py: -------------------------------------------------------------------------------- 1 | import unittest 2 | 3 | import lesson_check 4 | import util 5 | 6 | class TestFileList(unittest.TestCase): 7 | def setUp(self): 8 | self.reporter = util.Reporter() ## TODO: refactor reporter class. 9 | 10 | def test_file_list_has_expected_entries(self): 11 | # For first pass, simply assume that all required files are present 12 | all_filenames = [filename.replace('%', '') 13 | for filename in lesson_check.REQUIRED_FILES] 14 | 15 | lesson_check.check_fileset('', self.reporter, all_filenames) 16 | self.assertEqual(len(self.reporter.messages), 0) 17 | 18 | if __name__ == "__main__": 19 | unittest.main() 20 | -------------------------------------------------------------------------------- /bin/util.py: -------------------------------------------------------------------------------- 1 | from __future__ import print_function 2 | import sys 3 | import os 4 | import json 5 | from subprocess import Popen, PIPE 6 | 7 | # Import this way to produce a more useful error message. 8 | try: 9 | import yaml 10 | except ImportError: 11 | print('Unable to import YAML module: please install PyYAML', file=sys.stderr) 12 | sys.exit(1) 13 | 14 | 15 | # Things an image file's name can end with. 16 | IMAGE_FILE_SUFFIX = { 17 | '.gif', 18 | '.jpg', 19 | '.png', 20 | '.svg' 21 | } 22 | 23 | # Files that shouldn't be present. 24 | UNWANTED_FILES = [ 25 | '.nojekyll' 26 | ] 27 | 28 | # Marker to show that an expected value hasn't been provided. 29 | # (Can't use 'None' because that might be a legitimate value.) 30 | REPORTER_NOT_SET = [] 31 | 32 | class Reporter(object): 33 | """Collect and report errors.""" 34 | 35 | def __init__(self): 36 | """Constructor.""" 37 | 38 | super(Reporter, self).__init__() 39 | self.messages = [] 40 | 41 | 42 | def check_field(self, filename, name, values, key, expected=REPORTER_NOT_SET): 43 | """Check that a dictionary has an expected value.""" 44 | 45 | if key not in values: 46 | self.add(filename, '{0} does not contain {1}', name, key) 47 | elif expected is REPORTER_NOT_SET: 48 | pass 49 | elif type(expected) in (tuple, set, list): 50 | if values[key] not in expected: 51 | self.add(filename, '{0} {1} value {2} is not in {3}', name, key, values[key], expected) 52 | elif values[key] != expected: 53 | self.add(filename, '{0} {1} is {2} not {3}', name, key, values[key], expected) 54 | 55 | 56 | def check(self, condition, location, fmt, *args): 57 | """Append error if condition not met.""" 58 | 59 | if not condition: 60 | self.add(location, fmt, *args) 61 | 62 | 63 | def add(self, location, fmt, *args): 64 | """Append error unilaterally.""" 65 | 66 | self.messages.append((location, fmt.format(*args))) 67 | 68 | 69 | def report(self, stream=sys.stdout): 70 | """Report all messages in order.""" 71 | 72 | if not self.messages: 73 | return 74 | 75 | def pretty(item): 76 | location, message = item 77 | if isinstance(location, type(None)): 78 | return message 79 | elif isinstance(location, str): 80 | return location + ': ' + message 81 | elif isinstance(location, tuple): 82 | return '{0}:{1}: '.format(*location) + message 83 | else: 84 | assert False, 'Unknown item "{0}"'.format(item) 85 | 86 | def key(item): 87 | location, message = item 88 | if isinstance(location, type(None)): 89 | return ('', -1, message) 90 | elif isinstance(location, str): 91 | return (location, -1, message) 92 | elif isinstance(location, tuple): 93 | return (location[0], location[1], message) 94 | else: 95 | assert False, 'Unknown item "{0}"'.format(item) 96 | 97 | for m in sorted(self.messages, key=key): 98 | print(pretty(m), file=stream) 99 | 100 | 101 | def read_markdown(parser, path): 102 | """ 103 | Get YAML and AST for Markdown file, returning 104 | {'metadata':yaml, 'metadata_len':N, 'text':text, 'lines':[(i, line, len)], 'doc':doc}. 105 | """ 106 | 107 | # Split and extract YAML (if present). 108 | with open(path, 'r') as reader: 109 | body = reader.read() 110 | metadata_raw, metadata_yaml, body = split_metadata(path, body) 111 | 112 | # Split into lines. 113 | metadata_len = 0 if metadata_raw is None else metadata_raw.count('\n') 114 | lines = [(metadata_len+i+1, line, len(line)) for (i, line) in enumerate(body.split('\n'))] 115 | 116 | # Parse Markdown. 117 | cmd = 'ruby {0}'.format(parser) 118 | p = Popen(cmd, shell=True, stdin=PIPE, stdout=PIPE, close_fds=True, universal_newlines=True) 119 | stdout_data, stderr_data = p.communicate(body) 120 | doc = json.loads(stdout_data) 121 | 122 | return { 123 | 'metadata': metadata_yaml, 124 | 'metadata_len': metadata_len, 125 | 'text': body, 126 | 'lines': lines, 127 | 'doc': doc 128 | } 129 | 130 | 131 | def split_metadata(path, text): 132 | """ 133 | Get raw (text) metadata, metadata as YAML, and rest of body. 134 | If no metadata, return (None, None, body). 135 | """ 136 | 137 | metadata_raw = None 138 | metadata_yaml = None 139 | metadata_len = None 140 | 141 | pieces = text.split('---', 2) 142 | if len(pieces) == 3: 143 | metadata_raw = pieces[1] 144 | text = pieces[2] 145 | try: 146 | metadata_yaml = yaml.load(metadata_raw) 147 | except yaml.YAMLError as e: 148 | print('Unable to parse YAML header in {0}:\n{1}'.format(path, e), file=sys.stderr) 149 | sys.exit(1) 150 | 151 | return metadata_raw, metadata_yaml, text 152 | 153 | 154 | def load_yaml(filename): 155 | """ 156 | Wrapper around YAML loading so that 'import yaml' is only needed 157 | in one file. 158 | """ 159 | 160 | try: 161 | with open(filename, 'r') as reader: 162 | return yaml.load(reader) 163 | except (yaml.YAMLError, FileNotFoundError) as e: 164 | print('Unable to load YAML file {0}:\n{1}'.format(filename, e), file=sys.stderr) 165 | sys.exit(1) 166 | 167 | 168 | def check_unwanted_files(dir_path, reporter): 169 | """ 170 | Check that unwanted files are not present. 171 | """ 172 | 173 | for filename in UNWANTED_FILES: 174 | path = os.path.join(dir_path, filename) 175 | reporter.check(not os.path.exists(path), 176 | path, 177 | "Unwanted file found") 178 | 179 | 180 | def require(condition, message): 181 | """Fail if condition not met.""" 182 | 183 | if not condition: 184 | print(message, file=sys.stderr) 185 | sys.exit(1) 186 | -------------------------------------------------------------------------------- /bin/workshop_check.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | '''Check that a workshop's index.html metadata is valid. See the 4 | docstrings on the checking functions for a summary of the checks. 5 | ''' 6 | 7 | from __future__ import print_function 8 | import sys 9 | import os 10 | import re 11 | from datetime import date 12 | from util import Reporter, split_metadata, load_yaml, check_unwanted_files 13 | 14 | # Metadata field patterns. 15 | EMAIL_PATTERN = r'[^@]+@[^@]+\.[^@]+' 16 | HUMANTIME_PATTERN = r'((0?[1-9]|1[0-2]):[0-5]\d(am|pm)(-|to)(0?[1-9]|1[0-2]):[0-5]\d(am|pm))|((0?\d|1\d|2[0-3]):[0-5]\d(-|to)(0?\d|1\d|2[0-3]):[0-5]\d)' 17 | EVENTBRITE_PATTERN = r'\d{9,10}' 18 | URL_PATTERN = r'https?://.+' 19 | 20 | # Defaults. 21 | CARPENTRIES = ("dc", "swc") 22 | DEFAULT_CONTACT_EMAIL = 'admin@software-carpentry.org' 23 | 24 | USAGE = 'Usage: "check-workshop path/to/root/directory"' 25 | 26 | # Country and language codes. Note that codes mean different things: 'ar' 27 | # is 'Arabic' as a language but 'Argentina' as a country. 28 | 29 | ISO_COUNTRY = [ 30 | 'ad', 'ae', 'af', 'ag', 'ai', 'al', 'am', 'an', 'ao', 'aq', 'ar', 'as', 31 | 'at', 'au', 'aw', 'ax', 'az', 'ba', 'bb', 'bd', 'be', 'bf', 'bg', 'bh', 32 | 'bi', 'bj', 'bm', 'bn', 'bo', 'br', 'bs', 'bt', 'bv', 'bw', 'by', 'bz', 33 | 'ca', 'cc', 'cd', 'cf', 'cg', 'ch', 'ci', 'ck', 'cl', 'cm', 'cn', 'co', 34 | 'cr', 'cu', 'cv', 'cx', 'cy', 'cz', 'de', 'dj', 'dk', 'dm', 'do', 'dz', 35 | 'ec', 'ee', 'eg', 'eh', 'er', 'es', 'et', 'eu', 'fi', 'fj', 'fk', 'fm', 36 | 'fo', 'fr', 'ga', 'gb', 'gd', 'ge', 'gf', 'gg', 'gh', 'gi', 'gl', 'gm', 37 | 'gn', 'gp', 'gq', 'gr', 'gs', 'gt', 'gu', 'gw', 'gy', 'hk', 'hm', 'hn', 38 | 'hr', 'ht', 'hu', 'id', 'ie', 'il', 'im', 'in', 'io', 'iq', 'ir', 'is', 39 | 'it', 'je', 'jm', 'jo', 'jp', 'ke', 'kg', 'kh', 'ki', 'km', 'kn', 'kp', 40 | 'kr', 'kw', 'ky', 'kz', 'la', 'lb', 'lc', 'li', 'lk', 'lr', 'ls', 'lt', 41 | 'lu', 'lv', 'ly', 'ma', 'mc', 'md', 'me', 'mg', 'mh', 'mk', 'ml', 'mm', 42 | 'mn', 'mo', 'mp', 'mq', 'mr', 'ms', 'mt', 'mu', 'mv', 'mw', 'mx', 'my', 43 | 'mz', 'na', 'nc', 'ne', 'nf', 'ng', 'ni', 'nl', 'no', 'np', 'nr', 'nu', 44 | 'nz', 'om', 'pa', 'pe', 'pf', 'pg', 'ph', 'pk', 'pl', 'pm', 'pn', 'pr', 45 | 'ps', 'pt', 'pw', 'py', 'qa', 're', 'ro', 'rs', 'ru', 'rw', 'sa', 'sb', 46 | 'sc', 'sd', 'se', 'sg', 'sh', 'si', 'sj', 'sk', 'sl', 'sm', 'sn', 'so', 47 | 'sr', 'st', 'sv', 'sy', 'sz', 'tc', 'td', 'tf', 'tg', 'th', 'tj', 'tk', 48 | 'tl', 'tm', 'tn', 'to', 'tr', 'tt', 'tv', 'tw', 'tz', 'ua', 'ug', 'um', 49 | 'us', 'uy', 'uz', 'va', 'vc', 've', 'vg', 'vi', 'vn', 'vu', 'wf', 'ws', 50 | 'ye', 'yt', 'za', 'zm', 'zw' 51 | ] 52 | 53 | ISO_LANGUAGE = [ 54 | 'aa', 'ab', 'ae', 'af', 'ak', 'am', 'an', 'ar', 'as', 'av', 'ay', 'az', 55 | 'ba', 'be', 'bg', 'bh', 'bi', 'bm', 'bn', 'bo', 'br', 'bs', 'ca', 'ce', 56 | 'ch', 'co', 'cr', 'cs', 'cu', 'cv', 'cy', 'da', 'de', 'dv', 'dz', 'ee', 57 | 'el', 'en', 'eo', 'es', 'et', 'eu', 'fa', 'ff', 'fi', 'fj', 'fo', 'fr', 58 | 'fy', 'ga', 'gd', 'gl', 'gn', 'gu', 'gv', 'ha', 'he', 'hi', 'ho', 'hr', 59 | 'ht', 'hu', 'hy', 'hz', 'ia', 'id', 'ie', 'ig', 'ii', 'ik', 'io', 'is', 60 | 'it', 'iu', 'ja', 'jv', 'ka', 'kg', 'ki', 'kj', 'kk', 'kl', 'km', 'kn', 61 | 'ko', 'kr', 'ks', 'ku', 'kv', 'kw', 'ky', 'la', 'lb', 'lg', 'li', 'ln', 62 | 'lo', 'lt', 'lu', 'lv', 'mg', 'mh', 'mi', 'mk', 'ml', 'mn', 'mr', 'ms', 63 | 'mt', 'my', 'na', 'nb', 'nd', 'ne', 'ng', 'nl', 'nn', 'no', 'nr', 'nv', 64 | 'ny', 'oc', 'oj', 'om', 'or', 'os', 'pa', 'pi', 'pl', 'ps', 'pt', 'qu', 65 | 'rm', 'rn', 'ro', 'ru', 'rw', 'sa', 'sc', 'sd', 'se', 'sg', 'si', 'sk', 66 | 'sl', 'sm', 'sn', 'so', 'sq', 'sr', 'ss', 'st', 'su', 'sv', 'sw', 'ta', 67 | 'te', 'tg', 'th', 'ti', 'tk', 'tl', 'tn', 'to', 'tr', 'ts', 'tt', 'tw', 68 | 'ty', 'ug', 'uk', 'ur', 'uz', 've', 'vi', 'vo', 'wa', 'wo', 'xh', 'yi', 69 | 'yo', 'za', 'zh', 'zu' 70 | ] 71 | 72 | 73 | def look_for_fixme(func): 74 | """Decorator to fail test if text argument starts with "FIXME".""" 75 | 76 | def inner(arg): 77 | if (arg is not None) and \ 78 | isinstance(arg, str) and \ 79 | arg.lstrip().startswith('FIXME'): 80 | return False 81 | return func(arg) 82 | return inner 83 | 84 | 85 | @look_for_fixme 86 | def check_layout(layout): 87 | '''"layout" in YAML header must be "workshop".''' 88 | 89 | return layout == 'workshop' 90 | 91 | 92 | @look_for_fixme 93 | def check_carpentry(layout): 94 | '''"carpentry" in YAML header must be "dc" or "swc".''' 95 | 96 | return layout in CARPENTRIES 97 | 98 | 99 | @look_for_fixme 100 | def check_country(country): 101 | '''"country" must be a lowercase ISO-3166 two-letter code.''' 102 | 103 | return country in ISO_COUNTRY 104 | 105 | 106 | @look_for_fixme 107 | def check_language(language): 108 | '''"language" must be a lowercase ISO-639 two-letter code.''' 109 | 110 | return language in ISO_LANGUAGE 111 | 112 | 113 | @look_for_fixme 114 | def check_humandate(date): 115 | """ 116 | 'humandate' must be a human-readable date with a 3-letter month 117 | and 4-digit year. Examples include 'Feb 18-20, 2025' and 'Feb 18 118 | and 20, 2025'. It may be in languages other than English, but the 119 | month name should be kept short to aid formatting of the main 120 | Software Carpentry web site. 121 | """ 122 | 123 | if ',' not in date: 124 | return False 125 | 126 | month_dates, year = date.split(',') 127 | 128 | # The first three characters of month_dates are not empty 129 | month = month_dates[:3] 130 | if any(char == ' ' for char in month): 131 | return False 132 | 133 | # But the fourth character is empty ("February" is illegal) 134 | if month_dates[3] != ' ': 135 | return False 136 | 137 | # year contains *only* numbers 138 | try: 139 | int(year) 140 | except: 141 | return False 142 | 143 | return True 144 | 145 | 146 | @look_for_fixme 147 | def check_humantime(time): 148 | """ 149 | 'humantime' is a human-readable start and end time for the 150 | workshop, such as '09:00 - 16:00'. 151 | """ 152 | 153 | return bool(re.match(HUMANTIME_PATTERN, time.replace(' ', ''))) 154 | 155 | 156 | def check_date(this_date): 157 | """ 158 | 'startdate' and 'enddate' are machine-readable start and end dates 159 | for the workshop, and must be in YYYY-MM-DD format, e.g., 160 | '2015-07-01'. 161 | """ 162 | 163 | # YAML automatically loads valid dates as datetime.date. 164 | return isinstance(this_date, date) 165 | 166 | 167 | @look_for_fixme 168 | def check_latitude_longitude(latlng): 169 | """ 170 | 'latlng' must be a valid latitude and longitude represented as two 171 | floating-point numbers separated by a comma. 172 | """ 173 | 174 | try: 175 | lat, lng = latlng.split(',') 176 | lat = float(lat) 177 | long = float(lng) 178 | return (-90.0 <= lat <= 90.0) and (-180.0 <= long <= 180.0) 179 | except ValueError: 180 | return False 181 | 182 | 183 | def check_instructors(instructors): 184 | """ 185 | 'instructor' must be a non-empty comma-separated list of quoted 186 | names, e.g. ['First name', 'Second name', ...']. Do not use 'TBD' 187 | or other placeholders. 188 | """ 189 | 190 | # YAML automatically loads list-like strings as lists. 191 | return isinstance(instructors, list) and len(instructors) > 0 192 | 193 | 194 | def check_helpers(helpers): 195 | """ 196 | 'helper' must be a comma-separated list of quoted names, 197 | e.g. ['First name', 'Second name', ...']. The list may be empty. 198 | Do not use 'TBD' or other placeholders. 199 | """ 200 | 201 | # YAML automatically loads list-like strings as lists. 202 | return isinstance(helpers, list) and len(helpers) >= 0 203 | 204 | 205 | @look_for_fixme 206 | def check_email(email): 207 | """ 208 | 'contact' must be a valid email address consisting of characters, 209 | an '@', and more characters. It should not be the default contact 210 | email address 'admin@software-carpentry.org'. 211 | """ 212 | 213 | return bool(re.match(EMAIL_PATTERN, email)) and \ 214 | (email != DEFAULT_CONTACT_EMAIL) 215 | 216 | 217 | def check_eventbrite(eventbrite): 218 | """ 219 | 'eventbrite' (the Eventbrite registration key) must be 9 or more 220 | digits. It may appear as an integer or as a string. 221 | """ 222 | 223 | if isinstance(eventbrite, int): 224 | return True 225 | else: 226 | return bool(re.match(EVENTBRITE_PATTERN, eventbrite)) 227 | 228 | 229 | @look_for_fixme 230 | def check_etherpad(etherpad): 231 | """ 232 | 'etherpad' must be a valid URL. 233 | """ 234 | 235 | return bool(re.match(URL_PATTERN, etherpad)) 236 | 237 | 238 | @look_for_fixme 239 | def check_pass(value): 240 | """ 241 | This test always passes (it is used for 'checking' things like the 242 | workshop address, for which no sensible validation is feasible). 243 | """ 244 | 245 | return True 246 | 247 | 248 | HANDLERS = { 249 | 'layout': (True, check_layout, 'layout isn\'t "workshop"'), 250 | 251 | 'carpentry': (True, check_carpentry, 'carpentry isn\'t in ' + 252 | ', '.join(CARPENTRIES)), 253 | 254 | 'country': (True, check_country, 255 | 'country invalid: must use lowercase two-letter ISO code ' + 256 | 'from ' + ', '.join(ISO_COUNTRY)), 257 | 258 | 'language': (False, check_language, 259 | 'language invalid: must use lowercase two-letter ISO code' + 260 | ' from ' + ', '.join(ISO_LANGUAGE)), 261 | 262 | 'humandate': (True, check_humandate, 263 | 'humandate invalid. Please use three-letter months like ' + 264 | '"Jan" and four-letter years like "2025"'), 265 | 266 | 'humantime': (True, check_humantime, 267 | 'humantime doesn\'t include numbers'), 268 | 269 | 'startdate': (True, check_date, 270 | 'startdate invalid. Must be of format year-month-day, ' + 271 | 'i.e., 2014-01-31'), 272 | 273 | 'enddate': (False, check_date, 274 | 'enddate invalid. Must be of format year-month-day, i.e.,' + 275 | ' 2014-01-31'), 276 | 277 | 'latlng': (True, check_latitude_longitude, 278 | 'latlng invalid. Check that it is two floating point ' + 279 | 'numbers, separated by a comma'), 280 | 281 | 'instructor': (True, check_instructors, 282 | 'instructor list isn\'t a valid list of format ' + 283 | '["First instructor", "Second instructor",..]'), 284 | 285 | 'helper': (True, check_helpers, 286 | 'helper list isn\'t a valid list of format ' + 287 | '["First helper", "Second helper",..]'), 288 | 289 | 'contact': (True, check_email, 290 | 'contact email invalid or still set to ' + 291 | '"{0}".'.format(DEFAULT_CONTACT_EMAIL)), 292 | 293 | 'eventbrite': (False, check_eventbrite, 'Eventbrite key appears invalid'), 294 | 295 | 'etherpad': (False, check_etherpad, 'Etherpad URL appears invalid'), 296 | 297 | 'venue': (False, check_pass, 'venue name not specified'), 298 | 299 | 'address': (False, check_pass, 'address not specified') 300 | } 301 | 302 | # REQUIRED is all required categories. 303 | REQUIRED = set([k for k in HANDLERS if HANDLERS[k][0]]) 304 | 305 | # OPTIONAL is all optional categories. 306 | OPTIONAL = set([k for k in HANDLERS if not HANDLERS[k][0]]) 307 | 308 | 309 | def check_blank_lines(reporter, raw): 310 | """ 311 | Blank lines are not allowed in category headers. 312 | """ 313 | 314 | lines = [(i, x) for (i, x) in enumerate(raw.strip().split('\n')) if not x.strip()] 315 | reporter.check(not lines, 316 | None, 317 | 'Blank line(s) in header: {0}', 318 | ', '.join(["{0}: {1}".format(i, x.rstrip()) for (i, x) in lines])) 319 | 320 | 321 | def check_categories(reporter, left, right, msg): 322 | """ 323 | Report differences (if any) between two sets of categories. 324 | """ 325 | 326 | diff = left - right 327 | reporter.check(len(diff) == 0, 328 | None, 329 | '{0}: offending entries {1}', 330 | msg, sorted(list(diff))) 331 | 332 | 333 | def check_file(reporter, path, data): 334 | """ 335 | Get header from file, call all other functions, and check file for 336 | validity. 337 | """ 338 | 339 | # Get metadata as text and as YAML. 340 | raw, header, body = split_metadata(path, data) 341 | 342 | # Do we have any blank lines in the header? 343 | check_blank_lines(reporter, raw) 344 | 345 | # Look through all header entries. If the category is in the input 346 | # file and is either required or we have actual data (as opposed to 347 | # a commented-out entry), we check it. If it *isn't* in the header 348 | # but is required, report an error. 349 | for category in HANDLERS: 350 | required, handler, message = HANDLERS[category] 351 | if category in header: 352 | if required or header[category]: 353 | reporter.check(handler(header[category]), 354 | None, 355 | '{0}\n actual value "{1}"', 356 | message, header[category]) 357 | elif required: 358 | reporter.add(None, 359 | 'Missing mandatory key "{0}"', 360 | category) 361 | 362 | # Check whether we have missing or too many categories 363 | seen_categories = set(header.keys()) 364 | check_categories(reporter, REQUIRED, seen_categories, 365 | 'Missing categories') 366 | check_categories(reporter, seen_categories, REQUIRED.union(OPTIONAL), 367 | 'Superfluous categories') 368 | 369 | 370 | def check_config(reporter, filename): 371 | """ 372 | Check YAML configuration file. 373 | """ 374 | 375 | config = load_yaml(filename) 376 | 377 | kind = config.get('kind', None) 378 | reporter.check(kind == 'workshop', 379 | filename, 380 | 'Missing or unknown kind of event: {0}', 381 | kind) 382 | 383 | carpentry = config.get('carpentry', None) 384 | reporter.check(carpentry in ('swc', 'dc'), 385 | filename, 386 | 'Missing or unknown carpentry: {0}', 387 | carpentry) 388 | 389 | 390 | def main(): 391 | '''Run as the main program.''' 392 | 393 | if len(sys.argv) != 2: 394 | print(USAGE, file=sys.stderr) 395 | sys.exit(1) 396 | 397 | root_dir = sys.argv[1] 398 | index_file = os.path.join(root_dir, 'index.html') 399 | config_file = os.path.join(root_dir, '_config.yml') 400 | 401 | reporter = Reporter() 402 | check_config(reporter, config_file) 403 | check_unwanted_files(root_dir, reporter) 404 | with open(index_file) as reader: 405 | data = reader.read() 406 | check_file(reporter, index_file, data) 407 | reporter.report() 408 | 409 | 410 | if __name__ == '__main__': 411 | main() 412 | -------------------------------------------------------------------------------- /code/.gitkeep: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/swcarpentry/python-second-language/074c1d9aa7424104e8a5006623e91eeb6545bbf5/code/.gitkeep -------------------------------------------------------------------------------- /code/dictionaries.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Dictionaries" 8 | ] 9 | }, 10 | { 11 | "cell_type": "code", 12 | "execution_count": 24, 13 | "metadata": { 14 | "collapsed": false 15 | }, 16 | "outputs": [ 17 | { 18 | "data": { 19 | "text/plain": [ 20 | "[16, 25, 20, 8, 15, 14]" 21 | ] 22 | }, 23 | "execution_count": 24, 24 | "metadata": {}, 25 | "output_type": "execute_result" 26 | } 27 | ], 28 | "source": [ 29 | "# creating a dictionary\n", 30 | "this_dict = {\"Alfred\": 92, \"Beatrice\": 97, \"Carson\": 96}\n", 31 | "\n", 32 | "this_dict.keys()\n", 33 | "this_dict.values()\n", 34 | "\n", 35 | "names = [\"Alfred\", \"Beatrice\", \"Carson\"]\n", 36 | "scores = [92, 97, 96]\n", 37 | "\n", 38 | "dict([[\"Alfred\", 92], [\"Beatrice\", 97], [\"Carson\", 96]])\n", 39 | "dict(zip(names, scores))\n", 40 | "\n", 41 | "# Dict comprehension (if we covered list comprehension)\n", 42 | "{names[i]:scores[i] for i in range(len(scores))}" 43 | ] 44 | }, 45 | { 46 | "cell_type": "markdown", 47 | "metadata": {}, 48 | "source": [ 49 | "## Exercise, create a dictionary that would help you with a caesar cipher:" 50 | ] 51 | }, 52 | { 53 | "cell_type": "code", 54 | "execution_count": 25, 55 | "metadata": { 56 | "collapsed": false 57 | }, 58 | "outputs": [ 59 | { 60 | "data": { 61 | "text/plain": [ 62 | "[16, 25, 20, 8, 15, 14]" 63 | ] 64 | }, 65 | "execution_count": 25, 66 | "metadata": {}, 67 | "output_type": "execute_result" 68 | } 69 | ], 70 | "source": [ 71 | "import string\n", 72 | "alpha_dict = dict(zip([s for s in string.ascii_lowercase], range(1, 27)))\n", 73 | "\n", 74 | "def encode(string, offset=0):\n", 75 | " return [alpha_dict[s]+offset for s in string.lower()]\n", 76 | "\n", 77 | "encode(\"python\")" 78 | ] 79 | }, 80 | { 81 | "cell_type": "markdown", 82 | "metadata": {}, 83 | "source": [ 84 | "## Exercise: Think of something from your own research that might fit into a dictionary" 85 | ] 86 | }, 87 | { 88 | "cell_type": "code", 89 | "execution_count": null, 90 | "metadata": { 91 | "collapsed": true 92 | }, 93 | "outputs": [], 94 | "source": [] 95 | } 96 | ], 97 | "metadata": { 98 | "kernelspec": { 99 | "display_name": "Python 3", 100 | "language": "python", 101 | "name": "python3" 102 | }, 103 | "language_info": { 104 | "codemirror_mode": { 105 | "name": "ipython", 106 | "version": 3 107 | }, 108 | "file_extension": ".py", 109 | "mimetype": "text/x-python", 110 | "name": "python", 111 | "nbconvert_exporter": "python", 112 | "pygments_lexer": "ipython3", 113 | "version": "3.5.1" 114 | } 115 | }, 116 | "nbformat": 4, 117 | "nbformat_minor": 0 118 | } 119 | -------------------------------------------------------------------------------- /code/performance.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Improving Performance of Invasion Percolation\n", 8 | "\n", 9 | "* Use sets" 10 | ] 11 | }, 12 | { 13 | "cell_type": "code", 14 | "execution_count": 4, 15 | "metadata": { 16 | "collapsed": true 17 | }, 18 | "outputs": [], 19 | "source": [ 20 | "import numpy as np\n", 21 | "import random" 22 | ] 23 | }, 24 | { 25 | "cell_type": "code", 26 | "execution_count": 5, 27 | "metadata": { 28 | "collapsed": true 29 | }, 30 | "outputs": [], 31 | "source": [ 32 | "def percolation(size, spread):\n", 33 | " \"\"\"\n", 34 | " Simulate invasion percolation on a size x size grid with values in [1..spread],\n", 35 | " reporting density of final filled shape.\n", 36 | " \"\"\"\n", 37 | "\n", 38 | " assert (type(size) == int) and ((size > 0) and (size % 2 == 1)), 'size must be positive odd integer'\n", 39 | " assert (type(spread) == int) and (spread > 0), 'spread must be non-negative integer'\n", 40 | " grid = make_grid(size, spread)\n", 41 | " boundary = make_boundary(spread)\n", 42 | " chosen = (int(size/2), int(size/2))\n", 43 | " fill(grid, chosen)\n", 44 | " while not on_boundary(grid, chosen):\n", 45 | " extend_boundary(grid, boundary, chosen)\n", 46 | " chosen = choose_next(grid, boundary)\n", 47 | " fill(grid, chosen)\n", 48 | " return grid, calculate_density(grid)" 49 | ] 50 | }, 51 | { 52 | "cell_type": "code", 53 | "execution_count": 6, 54 | "metadata": { 55 | "collapsed": true 56 | }, 57 | "outputs": [], 58 | "source": [ 59 | "def make_grid(size, spread):\n", 60 | " \"\"\"\n", 61 | " Create size x size grid filled with values in [1..spread].\n", 62 | " \"\"\"\n", 63 | " return np.random.randint(low=1, high=spread+1, size=(size, size))" 64 | ] 65 | }, 66 | { 67 | "cell_type": "code", 68 | "execution_count": 7, 69 | "metadata": { 70 | "collapsed": false 71 | }, 72 | "outputs": [], 73 | "source": [ 74 | "def fill(grid, loc):\n", 75 | " \"\"\"\n", 76 | " Mark a cell as filled.\n", 77 | " \"\"\"\n", 78 | " grid[loc] = 0" 79 | ] 80 | }, 81 | { 82 | "cell_type": "code", 83 | "execution_count": 8, 84 | "metadata": { 85 | "collapsed": false 86 | }, 87 | "outputs": [], 88 | "source": [ 89 | "def on_boundary(grid, loc):\n", 90 | " \"\"\"\n", 91 | " Is the specified cell on the boundary of the grid?\n", 92 | " \"\"\"\n", 93 | " grid_x, grid_y = grid.shape\n", 94 | " loc_x, loc_y = loc\n", 95 | " return (loc_x == 0) or (loc_y == 0) or (loc_x == (grid_x -1)) or (loc_y == (grid_y -1))\n", 96 | "\n", 97 | "def test_on_boundary():\n", 98 | " grid = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])\n", 99 | " assert on_boundary(grid, (0, 0))\n", 100 | " assert not on_boundary(grid, (1, 1))\n", 101 | " assert on_boundary(grid, (2, 0))" 102 | ] 103 | }, 104 | { 105 | "cell_type": "code", 106 | "execution_count": 9, 107 | "metadata": { 108 | "collapsed": true 109 | }, 110 | "outputs": [], 111 | "source": [ 112 | "def calculate_density(grid):\n", 113 | " \"\"\"\n", 114 | " Return proportion of cells that are filled.\n", 115 | " \"\"\"\n", 116 | " filled = np.sum(grid == 0)\n", 117 | " return filled / grid.size" 118 | ] 119 | }, 120 | { 121 | "cell_type": "code", 122 | "execution_count": 10, 123 | "metadata": { 124 | "collapsed": false 125 | }, 126 | "outputs": [], 127 | "source": [ 128 | "def make_boundary(spread):\n", 129 | " \"\"\"\n", 130 | " Create object to keep track of boundary cells.\n", 131 | " \"\"\"\n", 132 | " result = []\n", 133 | " for i in range(spread + 1):\n", 134 | " result.append(set())\n", 135 | " return result\n", 136 | "\n", 137 | "def test_make_boundary():\n", 138 | " assert make_boundary(3) == [set(), set(), set(), set()]" 139 | ] 140 | }, 141 | { 142 | "cell_type": "code", 143 | "execution_count": 11, 144 | "metadata": { 145 | "collapsed": false 146 | }, 147 | "outputs": [], 148 | "source": [ 149 | "def extend_boundary(grid, boundary, loc):\n", 150 | " \"\"\"\n", 151 | " Extend boundary with unfilled cells next to given location.\n", 152 | " \"\"\"\n", 153 | " loc_x, loc_y = loc\n", 154 | " for (x, y) in ((loc_x-1, loc_y), (loc_x + 1, loc_y), (loc_x, loc_y-1), (loc_x, loc_y+1)):\n", 155 | " if grid[x, y] != 0:\n", 156 | " boundary[grid[x, y]].add((x, y))\n", 157 | "\n", 158 | "def test_extend_boundary():\n", 159 | " grid = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])\n", 160 | " boundary = make_boundary(9)\n", 161 | " extend_boundary(grid, boundary, (1, 1))\n", 162 | " assert boundary == [set(), set(), {(0, 1)}, set(), {(1, 0)}, set(), {(1, 2)}, set(), {(2, 1)}, set()]\n", 163 | "\n", 164 | "test_extend_boundary()" 165 | ] 166 | }, 167 | { 168 | "cell_type": "code", 169 | "execution_count": 12, 170 | "metadata": { 171 | "collapsed": true 172 | }, 173 | "outputs": [], 174 | "source": [ 175 | "def choose_next(grid, boundary):\n", 176 | " \"\"\"\n", 177 | " Find and return coordinates of next grid cell to fill.\n", 178 | " \"\"\"\n", 179 | " for val in range(len(boundary)):\n", 180 | " if boundary[val]:\n", 181 | " break\n", 182 | " loc = random.choice(list(boundary[val]))\n", 183 | " boundary[val].discard(loc)\n", 184 | " return loc" 185 | ] 186 | }, 187 | { 188 | "cell_type": "code", 189 | "execution_count": 13, 190 | "metadata": { 191 | "collapsed": true 192 | }, 193 | "outputs": [], 194 | "source": [ 195 | "def is_adjacent(grid, loc):\n", 196 | " \"\"\"\n", 197 | " Is the location (x, y) adjacent to a filled cell?\n", 198 | " \"\"\"\n", 199 | " x, y = loc\n", 200 | " max_x, max_y = grid.shape\n", 201 | " if grid[loc] == 0:\n", 202 | " return False\n", 203 | " if (x > 0) and (grid[x-1, y] == 0):\n", 204 | " return True\n", 205 | " if (y > 0) and (grid[x, y-1] == 0):\n", 206 | " return True\n", 207 | " if (x < max_x-1) and (grid[x+1, y] == 0):\n", 208 | " return True\n", 209 | " if (y < max_y-1) and (grid[x, y+1] == 0):\n", 210 | " return True\n", 211 | " return False\n", 212 | "\n", 213 | "def test_is_adjacent():\n", 214 | " grid = np.array([[0, 1, 2], [3, 4, 5], [6, 7, 8]])\n", 215 | " assert not is_adjacent(grid, (0, 0))\n", 216 | " assert is_adjacent(grid, (1, 0))\n", 217 | " assert is_adjacent(grid, (0, 1))\n", 218 | " assert not is_adjacent(grid, (1, 1))\n", 219 | " assert not is_adjacent(grid, (2, 0))\n", 220 | " assert not is_adjacent(grid, (2, 1))\n", 221 | " assert not is_adjacent(grid, (0, 2))\n", 222 | " assert not is_adjacent(grid, (1, 2))\n", 223 | " assert not is_adjacent(grid, (2, 2))" 224 | ] 225 | }, 226 | { 227 | "cell_type": "code", 228 | "execution_count": 14, 229 | "metadata": { 230 | "collapsed": false 231 | }, 232 | "outputs": [], 233 | "source": [ 234 | "def test_all():\n", 235 | " test_on_boundary()\n", 236 | " test_is_adjacent()\n", 237 | " test_make_boundary()" 238 | ] 239 | }, 240 | { 241 | "cell_type": "code", 242 | "execution_count": 15, 243 | "metadata": { 244 | "collapsed": false 245 | }, 246 | "outputs": [], 247 | "source": [ 248 | "test_all()" 249 | ] 250 | }, 251 | { 252 | "cell_type": "code", 253 | "execution_count": 30, 254 | "metadata": { 255 | "collapsed": false 256 | }, 257 | "outputs": [ 258 | { 259 | "data": { 260 | "text/plain": [ 261 | "(array([[4, 4, 3, 1, 2],\n", 262 | " [2, 5, 0, 3, 5],\n", 263 | " [2, 3, 0, 4, 1],\n", 264 | " [3, 3, 0, 1, 1],\n", 265 | " [4, 3, 0, 2, 4]]), 0.16)" 266 | ] 267 | }, 268 | "execution_count": 30, 269 | "metadata": {}, 270 | "output_type": "execute_result" 271 | } 272 | ], 273 | "source": [ 274 | "percolation(5, 5)" 275 | ] 276 | }, 277 | { 278 | "cell_type": "code", 279 | "execution_count": 37, 280 | "metadata": { 281 | "collapsed": false 282 | }, 283 | "outputs": [ 284 | { 285 | "data": { 286 | "text/plain": [ 287 | "0.09253485100634862" 288 | ] 289 | }, 290 | "execution_count": 37, 291 | "metadata": {}, 292 | "output_type": "execute_result" 293 | } 294 | ], 295 | "source": [ 296 | "import timeit\n", 297 | "timeit.timeit(stmt='percolation(21, 10)', number=200, setup='from __main__ import percolation')" 298 | ] 299 | }, 300 | { 301 | "cell_type": "code", 302 | "execution_count": null, 303 | "metadata": { 304 | "collapsed": true 305 | }, 306 | "outputs": [], 307 | "source": [] 308 | } 309 | ], 310 | "metadata": { 311 | "kernelspec": { 312 | "display_name": "Python 3", 313 | "language": "python", 314 | "name": "python3" 315 | }, 316 | "language_info": { 317 | "codemirror_mode": { 318 | "name": "ipython", 319 | "version": 3 320 | }, 321 | "file_extension": ".py", 322 | "mimetype": "text/x-python", 323 | "name": "python", 324 | "nbconvert_exporter": "python", 325 | "pygments_lexer": "ipython3", 326 | "version": "3.5.1" 327 | } 328 | }, 329 | "nbformat": 4, 330 | "nbformat_minor": 0 331 | } 332 | -------------------------------------------------------------------------------- /code/profiling.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Profiling Invasion Percolation\n", 8 | "\n", 9 | "* How long does it take to run?\n", 10 | "* Why?" 11 | ] 12 | }, 13 | { 14 | "cell_type": "code", 15 | "execution_count": 45, 16 | "metadata": { 17 | "collapsed": true 18 | }, 19 | "outputs": [], 20 | "source": [ 21 | "import numpy as np\n", 22 | "import random" 23 | ] 24 | }, 25 | { 26 | "cell_type": "code", 27 | "execution_count": 46, 28 | "metadata": { 29 | "collapsed": true 30 | }, 31 | "outputs": [], 32 | "source": [ 33 | "def percolation(size, spread):\n", 34 | " \"\"\"\n", 35 | " Simulate invasion percolation on a size x size grid with values in [1..spread],\n", 36 | " reporting density of final filled shape.\n", 37 | " \"\"\"\n", 38 | "\n", 39 | " assert (type(size) == int) and ((size > 0) and (size % 2 == 1)), 'size must be positive odd integer'\n", 40 | " assert (type(spread) == int) and (spread > 0), 'spread must be non-negative integer'\n", 41 | " grid = make_grid(size, spread)\n", 42 | " chosen = (int(size/2), int(size/2))\n", 43 | " fill(grid, chosen)\n", 44 | " while not on_boundary(grid, chosen):\n", 45 | " chosen = choose_next(grid)\n", 46 | " fill(grid, chosen)\n", 47 | " return grid, calculate_density(grid)" 48 | ] 49 | }, 50 | { 51 | "cell_type": "code", 52 | "execution_count": 47, 53 | "metadata": { 54 | "collapsed": true 55 | }, 56 | "outputs": [], 57 | "source": [ 58 | "def make_grid(size, spread):\n", 59 | " \"\"\"\n", 60 | " Create size x size grid filled with values in [1..spread].\n", 61 | " \"\"\"\n", 62 | " return np.random.randint(low=1, high=spread+1, size=(size, size))" 63 | ] 64 | }, 65 | { 66 | "cell_type": "code", 67 | "execution_count": 48, 68 | "metadata": { 69 | "collapsed": false 70 | }, 71 | "outputs": [], 72 | "source": [ 73 | "def fill(grid, loc):\n", 74 | " \"\"\"\n", 75 | " Mark a cell as filled.\n", 76 | " \"\"\"\n", 77 | " grid[loc] = 0" 78 | ] 79 | }, 80 | { 81 | "cell_type": "code", 82 | "execution_count": 49, 83 | "metadata": { 84 | "collapsed": false 85 | }, 86 | "outputs": [], 87 | "source": [ 88 | "def on_boundary(grid, loc):\n", 89 | " \"\"\"\n", 90 | " Is the specified cell on the boundary of the grid?\n", 91 | " \"\"\"\n", 92 | " grid_x, grid_y = grid.shape\n", 93 | " loc_x, loc_y = loc\n", 94 | " return (loc_x == 0) or (loc_y == 0) or (loc_x == (grid_x -1)) or (loc_y == (grid_y -1))\n", 95 | "\n", 96 | "def test_on_boundary():\n", 97 | " grid = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])\n", 98 | " assert on_boundary(grid, (0, 0))\n", 99 | " assert not on_boundary(grid, (1, 1))\n", 100 | " assert on_boundary(grid, (2, 0))" 101 | ] 102 | }, 103 | { 104 | "cell_type": "code", 105 | "execution_count": 50, 106 | "metadata": { 107 | "collapsed": true 108 | }, 109 | "outputs": [], 110 | "source": [ 111 | "def calculate_density(grid):\n", 112 | " \"\"\"\n", 113 | " Return proportion of cells that are filled.\n", 114 | " \"\"\"\n", 115 | " filled = np.sum(grid == 0)\n", 116 | " return filled / grid.size" 117 | ] 118 | }, 119 | { 120 | "cell_type": "code", 121 | "execution_count": 51, 122 | "metadata": { 123 | "collapsed": true 124 | }, 125 | "outputs": [], 126 | "source": [ 127 | "def choose_next(grid):\n", 128 | " \"\"\"\n", 129 | " Find and return coordinates of next grid cell to fill.\n", 130 | " \"\"\"\n", 131 | " candidates = []\n", 132 | " value = 1 + grid.max()\n", 133 | " dim_x, dim_y = grid.shape\n", 134 | " for x in range(dim_x):\n", 135 | " for y in range(dim_y):\n", 136 | " if grid[x, y] == 0:\n", 137 | " pass\n", 138 | " elif is_adjacent(grid, (x, y)):\n", 139 | " if grid[x, y] < value:\n", 140 | " value = grid[x, y]\n", 141 | " candidates = [(x, y)]\n", 142 | " elif grid[x, y] == value:\n", 143 | " candidates.append((x, y))\n", 144 | " return random.choice(candidates)\n", 145 | "\n", 146 | "def test_choose_next():\n", 147 | " assert choose_next(np.array([[9, 1, 9], [9, 0, 9], [9, 9, 9]])) == (0, 1)\n", 148 | " assert choose_next(np.array([[9, 2, 9], [9, 0, 9], [9, 1, 9]])) == (2, 1)" 149 | ] 150 | }, 151 | { 152 | "cell_type": "code", 153 | "execution_count": 52, 154 | "metadata": { 155 | "collapsed": true 156 | }, 157 | "outputs": [], 158 | "source": [ 159 | "def is_adjacent(grid, loc):\n", 160 | " \"\"\"\n", 161 | " Is the location (x, y) adjacent to a filled cell?\n", 162 | " \"\"\"\n", 163 | " x, y = loc\n", 164 | " max_x, max_y = grid.shape\n", 165 | " if grid[loc] == 0:\n", 166 | " return False\n", 167 | " if (x > 0) and (grid[x-1, y] == 0):\n", 168 | " return True\n", 169 | " if (y > 0) and (grid[x, y-1] == 0):\n", 170 | " return True\n", 171 | " if (x < max_x-1) and (grid[x+1, y] == 0):\n", 172 | " return True\n", 173 | " if (y < max_y-1) and (grid[x, y+1] == 0):\n", 174 | " return True\n", 175 | " return False\n", 176 | "\n", 177 | "def test_is_adjacent():\n", 178 | " grid = np.array([[0, 1, 2], [3, 4, 5], [6, 7, 8]])\n", 179 | " assert not is_adjacent(grid, (0, 0))\n", 180 | " assert is_adjacent(grid, (1, 0))\n", 181 | " assert is_adjacent(grid, (0, 1))\n", 182 | " assert not is_adjacent(grid, (1, 1))\n", 183 | " assert not is_adjacent(grid, (2, 0))\n", 184 | " assert not is_adjacent(grid, (2, 1))\n", 185 | " assert not is_adjacent(grid, (0, 2))\n", 186 | " assert not is_adjacent(grid, (1, 2))\n", 187 | " assert not is_adjacent(grid, (2, 2))" 188 | ] 189 | }, 190 | { 191 | "cell_type": "code", 192 | "execution_count": 53, 193 | "metadata": { 194 | "collapsed": false 195 | }, 196 | "outputs": [], 197 | "source": [ 198 | "def test_all():\n", 199 | " test_on_boundary()\n", 200 | " test_is_adjacent()\n", 201 | " test_choose_next()" 202 | ] 203 | }, 204 | { 205 | "cell_type": "code", 206 | "execution_count": 54, 207 | "metadata": { 208 | "collapsed": false 209 | }, 210 | "outputs": [ 211 | { 212 | "data": { 213 | "text/plain": [ 214 | "3.790200571529567e-05" 215 | ] 216 | }, 217 | "execution_count": 54, 218 | "metadata": {}, 219 | "output_type": "execute_result" 220 | } 221 | ], 222 | "source": [ 223 | "import timeit\n", 224 | "\n", 225 | "timeit.timeit(stmt='1+2', number=1000)" 226 | ] 227 | }, 228 | { 229 | "cell_type": "code", 230 | "execution_count": 55, 231 | "metadata": { 232 | "collapsed": false 233 | }, 234 | "outputs": [ 235 | { 236 | "data": { 237 | "text/plain": [ 238 | "10.64762847199745" 239 | ] 240 | }, 241 | "execution_count": 55, 242 | "metadata": {}, 243 | "output_type": "execute_result" 244 | } 245 | ], 246 | "source": [ 247 | "timeit.timeit(stmt='percolation(21, 10)', number=200, setup='from __main__ import percolation')" 248 | ] 249 | }, 250 | { 251 | "cell_type": "code", 252 | "execution_count": 56, 253 | "metadata": { 254 | "collapsed": false 255 | }, 256 | "outputs": [ 257 | { 258 | "name": "stdout", 259 | "output_type": "stream", 260 | "text": [ 261 | " 16664919 function calls in 59.556 seconds\n", 262 | "\n", 263 | " Ordered by: internal time\n", 264 | "\n", 265 | " ncalls tottime percall cumtime percall filename:lineno(function)\n", 266 | " 16632315 46.030 0.000 46.030 0.000 :1(is_adjacent)\n", 267 | " 855 13.453 0.016 59.549 0.070 :1(choose_next)\n", 268 | " 856 0.047 0.000 0.047 0.000 {method 'reduce' of 'numpy.ufunc' objects}\n", 269 | " 22474 0.006 0.000 0.006 0.000 {method 'append' of 'list' objects}\n", 270 | " 1 0.004 0.004 59.556 59.556 :1(percolation)\n", 271 | " 855 0.004 0.000 0.010 0.000 random.py:250(choice)\n", 272 | " 855 0.004 0.000 0.005 0.000 random.py:220(_randbelow)\n", 273 | " 855 0.002 0.000 0.049 0.000 {method 'max' of 'numpy.ndarray' objects}\n", 274 | " 1567 0.001 0.000 0.001 0.000 {method 'getrandbits' of '_random.Random' objects}\n", 275 | " 856 0.001 0.000 0.001 0.000 :1(fill)\n", 276 | " 856 0.001 0.000 0.001 0.000 :1(on_boundary)\n", 277 | " 855 0.001 0.000 0.048 0.000 _methods.py:25(_amax)\n", 278 | " 1 0.001 0.001 0.001 0.001 {method 'randint' of 'mtrand.RandomState' objects}\n", 279 | " 855 0.001 0.000 0.001 0.000 {built-in method builtins.len}\n", 280 | " 855 0.000 0.000 0.000 0.000 {method 'bit_length' of 'int' objects}\n", 281 | " 1 0.000 0.000 0.000 0.000 :1(calculate_density)\n", 282 | " 1 0.000 0.000 59.556 59.556 {built-in method builtins.exec}\n", 283 | " 1 0.000 0.000 0.000 0.000 fromnumeric.py:1737(sum)\n", 284 | " 1 0.000 0.000 59.556 59.556 :1()\n", 285 | " 1 0.000 0.000 0.001 0.001 :1(make_grid)\n", 286 | " 1 0.000 0.000 0.000 0.000 {built-in method builtins.isinstance}\n", 287 | " 1 0.000 0.000 0.000 0.000 _methods.py:31(_sum)\n", 288 | " 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}\n", 289 | "\n", 290 | "\n" 291 | ] 292 | } 293 | ], 294 | "source": [ 295 | "import cProfile\n", 296 | "cProfile.run('percolation(141, 10)', sort='tottime')" 297 | ] 298 | }, 299 | { 300 | "cell_type": "code", 301 | "execution_count": null, 302 | "metadata": { 303 | "collapsed": true 304 | }, 305 | "outputs": [], 306 | "source": [] 307 | } 308 | ], 309 | "metadata": { 310 | "kernelspec": { 311 | "display_name": "Python 3", 312 | "language": "python", 313 | "name": "python3" 314 | }, 315 | "language_info": { 316 | "codemirror_mode": { 317 | "name": "ipython", 318 | "version": 3 319 | }, 320 | "file_extension": ".py", 321 | "mimetype": "text/x-python", 322 | "name": "python", 323 | "nbconvert_exporter": "python", 324 | "pygments_lexer": "ipython3", 325 | "version": "3.5.1" 326 | } 327 | }, 328 | "nbformat": 4, 329 | "nbformat_minor": 0 330 | } 331 | -------------------------------------------------------------------------------- /code/profiling_pi.py: -------------------------------------------------------------------------------- 1 | # Profiling pi methods 2 | # mw 20150916 3 | 4 | import math 5 | import random 6 | import decimal 7 | 8 | @profile 9 | def montecarlo(maxIt): 10 | ctr = 0 11 | variance = 3.141 12 | for i in range(maxIt): 13 | if math.pow(random.random(), 2.0) + math.pow(random.random(), 2.0) <= 1.0: 14 | ctr += 1 15 | result = (4.0 * ctr / maxIt) 16 | if abs(math.pi - result) <= variance: 17 | variance = abs(math.pi - result) 18 | cbest=result 19 | print " Monte Carlo method: "+str(cbest) 20 | 21 | @profile 22 | def archie(prec): 23 | decimal.getcontext().prec = prec 24 | D=decimal.Decimal 25 | 26 | eps = D(1)/D(10**prec) 27 | 28 | x = D(4) 29 | y = D(2)*D(2).sqrt() 30 | 31 | ctr = D(0) 32 | while x-y > eps: 33 | xnew = 2*x*y/(x+y) 34 | y = D(xnew*y).sqrt() 35 | x = xnew 36 | ctr +=1 37 | print " Archimedes method: "+ str((x+y)/D(2)) 38 | 39 | montecarlo(1000000) # number of iterations for the monte carlo routine to run 40 | archie(8) # level of precision, eg 8 decimal places 41 | print " Python value of pi: "+str(math.pi)[:9] # the value of pi in math.pi 42 | -------------------------------------------------------------------------------- /code/testing.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Testing Invasion Percolation\n", 8 | "\n", 9 | "* Add `assert` statements to check initial parameters\n", 10 | "* Then what?" 11 | ] 12 | }, 13 | { 14 | "cell_type": "code", 15 | "execution_count": 7, 16 | "metadata": { 17 | "collapsed": true 18 | }, 19 | "outputs": [], 20 | "source": [ 21 | "import numpy as np\n", 22 | "import random" 23 | ] 24 | }, 25 | { 26 | "cell_type": "code", 27 | "execution_count": 8, 28 | "metadata": { 29 | "collapsed": true 30 | }, 31 | "outputs": [], 32 | "source": [ 33 | "def percolation(size, spread):\n", 34 | " \"\"\"\n", 35 | " Simulate invasion percolation on a size x size grid with values in [1..spread],\n", 36 | " reporting density of final filled shape.\n", 37 | " \"\"\"\n", 38 | "\n", 39 | " assert (type(size) == int) and ((size > 0) and (size % 2 == 1)), 'size must be positive odd integer'\n", 40 | " assert (type(spread) == int) and (spread > 0), 'spread must be non-negative integer'\n", 41 | " grid = make_grid(size, spread)\n", 42 | " chosen = (int(size/2), int(size/2))\n", 43 | " fill(grid, chosen)\n", 44 | " while not on_boundary(grid, chosen):\n", 45 | " chosen = choose_next(grid)\n", 46 | " fill(grid, chosen)\n", 47 | " return grid, calculate_density(grid)" 48 | ] 49 | }, 50 | { 51 | "cell_type": "code", 52 | "execution_count": 9, 53 | "metadata": { 54 | "collapsed": true 55 | }, 56 | "outputs": [], 57 | "source": [ 58 | "def make_grid(size, spread):\n", 59 | " \"\"\"\n", 60 | " Create size x size grid filled with values in [1..spread].\n", 61 | " \"\"\"\n", 62 | " return np.random.randint(low=1, high=spread+1, size=(size, size))" 63 | ] 64 | }, 65 | { 66 | "cell_type": "code", 67 | "execution_count": 10, 68 | "metadata": { 69 | "collapsed": false 70 | }, 71 | "outputs": [], 72 | "source": [ 73 | "def fill(grid, loc):\n", 74 | " \"\"\"\n", 75 | " Mark a cell as filled.\n", 76 | " \"\"\"\n", 77 | " grid[loc] = 0" 78 | ] 79 | }, 80 | { 81 | "cell_type": "code", 82 | "execution_count": 11, 83 | "metadata": { 84 | "collapsed": false 85 | }, 86 | "outputs": [], 87 | "source": [ 88 | "def on_boundary(grid, loc):\n", 89 | " \"\"\"\n", 90 | " Is the specified cell on the boundary of the grid?\n", 91 | " \"\"\"\n", 92 | " grid_x, grid_y = grid.shape\n", 93 | " loc_x, loc_y = loc\n", 94 | " return (loc_x == 0) or (loc_y == 0) or (loc_x == (grid_x -1)) or (loc_y == (grid_y -1))\n", 95 | "\n", 96 | "def test_on_boundary():\n", 97 | " grid = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])\n", 98 | " assert on_boundary(grid, (0, 0))\n", 99 | " assert not on_boundary(grid, (1, 1))\n", 100 | " assert on_boundary(grid, (2, 0))" 101 | ] 102 | }, 103 | { 104 | "cell_type": "code", 105 | "execution_count": 12, 106 | "metadata": { 107 | "collapsed": true 108 | }, 109 | "outputs": [], 110 | "source": [ 111 | "def calculate_density(grid):\n", 112 | " \"\"\"\n", 113 | " Return proportion of cells that are filled.\n", 114 | " \"\"\"\n", 115 | " filled = np.sum(grid == 0)\n", 116 | " return filled / grid.size" 117 | ] 118 | }, 119 | { 120 | "cell_type": "code", 121 | "execution_count": 13, 122 | "metadata": { 123 | "collapsed": true 124 | }, 125 | "outputs": [], 126 | "source": [ 127 | "def choose_next(grid):\n", 128 | " \"\"\"\n", 129 | " Find and return coordinates of next grid cell to fill.\n", 130 | " \"\"\"\n", 131 | " candidates = []\n", 132 | " value = 1 + grid.max()\n", 133 | " dim_x, dim_y = grid.shape\n", 134 | " for x in range(dim_x):\n", 135 | " for y in range(dim_y):\n", 136 | " if grid[x, y] == 0:\n", 137 | " pass\n", 138 | " elif is_adjacent(grid, (x, y)):\n", 139 | " if grid[x, y] < value:\n", 140 | " value = grid[x, y]\n", 141 | " candidates = [(x, y)]\n", 142 | " elif grid[x, y] == value:\n", 143 | " candidates.append((x, y))\n", 144 | " return random.choice(candidates)\n", 145 | "\n", 146 | "def test_choose_next():\n", 147 | " assert choose_next(np.array([[9, 1, 9], [9, 0, 9], [9, 9, 9]])) == (0, 1)\n", 148 | " assert choose_next(np.array([[9, 2, 9], [9, 0, 9], [9, 1, 9]])) == (2, 1)" 149 | ] 150 | }, 151 | { 152 | "cell_type": "code", 153 | "execution_count": 14, 154 | "metadata": { 155 | "collapsed": true 156 | }, 157 | "outputs": [], 158 | "source": [ 159 | "def is_adjacent(grid, loc):\n", 160 | " \"\"\"\n", 161 | " Is the location (x, y) adjacent to a filled cell?\n", 162 | " \"\"\"\n", 163 | " x, y = loc\n", 164 | " max_x, max_y = grid.shape\n", 165 | " if grid[loc] == 0:\n", 166 | " return False\n", 167 | " if (x > 0) and (grid[x-1, y] == 0):\n", 168 | " return True\n", 169 | " if (y > 0) and (grid[x, y-1] == 0):\n", 170 | " return True\n", 171 | " if (x < max_x-1) and (grid[x+1, y] == 0):\n", 172 | " return True\n", 173 | " if (y < max_y-1) and (grid[x, y+1] == 0):\n", 174 | " return True\n", 175 | " return False\n", 176 | "\n", 177 | "def test_is_adjacent():\n", 178 | " grid = np.array([[0, 1, 2], [3, 4, 5], [6, 7, 8]])\n", 179 | " assert not is_adjacent(grid, (0, 0))\n", 180 | " assert is_adjacent(grid, (1, 0))\n", 181 | " assert is_adjacent(grid, (0, 1))\n", 182 | " assert not is_adjacent(grid, (1, 1))\n", 183 | " assert not is_adjacent(grid, (2, 0))\n", 184 | " assert not is_adjacent(grid, (2, 1))\n", 185 | " assert not is_adjacent(grid, (0, 2))\n", 186 | " assert not is_adjacent(grid, (1, 2))\n", 187 | " assert not is_adjacent(grid, (2, 2))" 188 | ] 189 | }, 190 | { 191 | "cell_type": "code", 192 | "execution_count": 15, 193 | "metadata": { 194 | "collapsed": false 195 | }, 196 | "outputs": [], 197 | "source": [ 198 | "def test_all():\n", 199 | " test_on_boundary()\n", 200 | " test_is_adjacent()\n", 201 | " test_choose_next()" 202 | ] 203 | }, 204 | { 205 | "cell_type": "code", 206 | "execution_count": 16, 207 | "metadata": { 208 | "collapsed": false 209 | }, 210 | "outputs": [], 211 | "source": [ 212 | "test_all()" 213 | ] 214 | }, 215 | { 216 | "cell_type": "code", 217 | "execution_count": 17, 218 | "metadata": { 219 | "collapsed": true 220 | }, 221 | "outputs": [], 222 | "source": [ 223 | "import sys\n", 224 | "\n", 225 | "def run_all_tests(stem='test_'):\n", 226 | " \n", 227 | " # Get all global functions whose names start with stem.\n", 228 | " everything = globals()\n", 229 | " funcs = []\n", 230 | " for name in everything:\n", 231 | " if name.startswith(stem) and callable(everything[name]):\n", 232 | " funcs.append(everything[name])\n", 233 | "\n", 234 | " # Run them, counting exceptions.\n", 235 | " success = 0\n", 236 | " failure = 0\n", 237 | " error = 0\n", 238 | " for f in funcs:\n", 239 | " try:\n", 240 | " f()\n", 241 | " success += 1\n", 242 | " except AssertionError as e:\n", 243 | " print(e, file=sys.stderr)\n", 244 | " failure += 1\n", 245 | " except Exception as e:\n", 246 | " print(e, file=sys.stderr)\n", 247 | " error += 1\n", 248 | "\n", 249 | " # Report.\n", 250 | " total = success + failure + error\n", 251 | " print('{0} tests, {1} failures, {2} errors'.format(total, failure, error))" 252 | ] 253 | }, 254 | { 255 | "cell_type": "code", 256 | "execution_count": 18, 257 | "metadata": { 258 | "collapsed": false 259 | }, 260 | "outputs": [ 261 | { 262 | "name": "stdout", 263 | "output_type": "stream", 264 | "text": [ 265 | "3 tests, 1 failures, 1 errors\n" 266 | ] 267 | }, 268 | { 269 | "name": "stderr", 270 | "output_type": "stream", 271 | "text": [ 272 | "always fail\n", 273 | "not really dividing by zero\n" 274 | ] 275 | } 276 | ], 277 | "source": [ 278 | "def always_succeed():\n", 279 | " pass\n", 280 | "\n", 281 | "def always_fail():\n", 282 | " assert False, 'always fail'\n", 283 | "\n", 284 | "def always_error():\n", 285 | " raise ZeroDivisionError('not really dividing by zero')\n", 286 | "\n", 287 | "run_all_tests('always_')" 288 | ] 289 | }, 290 | { 291 | "cell_type": "code", 292 | "execution_count": 19, 293 | "metadata": { 294 | "collapsed": false 295 | }, 296 | "outputs": [ 297 | { 298 | "name": "stdout", 299 | "output_type": "stream", 300 | "text": [ 301 | "4 tests, 0 failures, 0 errors\n" 302 | ] 303 | } 304 | ], 305 | "source": [ 306 | "run_all_tests()" 307 | ] 308 | }, 309 | { 310 | "cell_type": "code", 311 | "execution_count": 20, 312 | "metadata": { 313 | "collapsed": false 314 | }, 315 | "outputs": [ 316 | { 317 | "name": "stdout", 318 | "output_type": "stream", 319 | "text": [ 320 | "5 tests, 1 failures, 0 errors\n" 321 | ] 322 | }, 323 | { 324 | "name": "stderr", 325 | "output_type": "stream", 326 | "text": [ 327 | "just to prove the point\n" 328 | ] 329 | } 330 | ], 331 | "source": [ 332 | "def test_to_prove_it_works():\n", 333 | " assert False, 'just to prove the point'\n", 334 | "\n", 335 | "run_all_tests()" 336 | ] 337 | }, 338 | { 339 | "cell_type": "code", 340 | "execution_count": 21, 341 | "metadata": { 342 | "collapsed": false 343 | }, 344 | "outputs": [ 345 | { 346 | "data": { 347 | "text/plain": [ 348 | "function" 349 | ] 350 | }, 351 | "execution_count": 21, 352 | "metadata": {}, 353 | "output_type": "execute_result" 354 | } 355 | ], 356 | "source": [ 357 | "type(run_all_tests)" 358 | ] 359 | }, 360 | { 361 | "cell_type": "code", 362 | "execution_count": 22, 363 | "metadata": { 364 | "collapsed": false 365 | }, 366 | "outputs": [ 367 | { 368 | "data": { 369 | "text/plain": [ 370 | "True" 371 | ] 372 | }, 373 | "execution_count": 22, 374 | "metadata": {}, 375 | "output_type": "execute_result" 376 | } 377 | ], 378 | "source": [ 379 | "callable(run_all_tests)" 380 | ] 381 | }, 382 | { 383 | "cell_type": "code", 384 | "execution_count": null, 385 | "metadata": { 386 | "collapsed": true 387 | }, 388 | "outputs": [], 389 | "source": [] 390 | } 391 | ], 392 | "metadata": { 393 | "kernelspec": { 394 | "display_name": "Python 3", 395 | "language": "python", 396 | "name": "python3" 397 | }, 398 | "language_info": { 399 | "codemirror_mode": { 400 | "name": "ipython", 401 | "version": 3 402 | }, 403 | "file_extension": ".py", 404 | "mimetype": "text/x-python", 405 | "name": "python", 406 | "nbconvert_exporter": "python", 407 | "pygments_lexer": "ipython3", 408 | "version": "3.5.1" 409 | } 410 | }, 411 | "nbformat": 4, 412 | "nbformat_minor": 0 413 | } 414 | -------------------------------------------------------------------------------- /data/.gitkeep: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/swcarpentry/python-second-language/074c1d9aa7424104e8a5006623e91eeb6545bbf5/data/.gitkeep -------------------------------------------------------------------------------- /data/temperature.csv: -------------------------------------------------------------------------------- 1 | Year,Annual_Mean,5-year_Mean 1880,-0.23, 1881,-0.15, 1882,-0.18,-0.21 1883,-0.21,-0.22 1884,-0.29,-0.24 1885,-0.27,-0.27 1886,-0.26,-0.27 1887,-0.33,-0.23 1888,-0.21,-0.25 1889,-0.11,-0.25 1890,-0.34,-0.25 1891,-0.27,-0.28 1892,-0.32,-0.32 1893,-0.36,-0.31 1894,-0.33,-0.29 1895,-0.26,-0.27 1896,-0.19,-0.26 1897,-0.19,-0.23 1898,-0.32,-0.21 1899,-0.21,-0.22 1900,-0.16,-0.24 1901,-0.21,-0.25 1902,-0.31,-0.3 1903,-0.37,-0.33 1904,-0.45,-0.34 1905,-0.31,-0.37 1906,-0.27,-0.38 1907,-0.43,-0.39 1908,-0.44,-0.42 1909,-0.48,-0.45 1910,-0.47,-0.45 1911,-0.45,-0.44 1912,-0.42,-0.4 1913,-0.4,-0.34 1914,-0.24,-0.32 1915,-0.17,-0.33 1916,-0.37,-0.31 1917,-0.45,-0.32 1918,-0.32,-0.34 1919,-0.3,-0.31 1920,-0.28,-0.28 1921,-0.21,-0.27 1922,-0.3,-0.26 1923,-0.27,-0.25 1924,-0.25,-0.23 1925,-0.23,-0.21 1926,-0.11,-0.19 1927,-0.2,-0.2 1928,-0.17,-0.18 1929,-0.32,-0.18 1930,-0.13,-0.16 1931,-0.08,-0.18 1932,-0.11,-0.13 1933,-0.26,-0.14 1934,-0.1,-0.15 1935,-0.16,-0.12 1936,-0.11,-0.06 1937,0.03,-0.04 1938,0.05,0 1939,0,0.04 1940,0.06,0.04 1941,0.07,0.05 1942,0.05,0.07 1943,0.05,0.06 1944,0.13,0.03 1945,0,0.01 1946,-0.08,-0.02 1947,-0.05,-0.07 1948,-0.11,-0.11 1949,-0.12,-0.1 1950,-0.19,-0.09 1951,-0.06,-0.05 1952,0.01,-0.05 1953,0.08,-0.04 1954,-0.12,-0.07 1955,-0.13,-0.06 1956,-0.18,-0.07 1957,0.03,-0.04 1958,0.05,-0.02 1959,0.03,0.02 1960,-0.04,0.03 1961,0.06,0.03 1962,0.04,-0.01 1963,0.08,-0.02 1964,-0.2,-0.04 1965,-0.1,-0.05 1966,-0.04,-0.08 1967,-0.01,-0.03 1968,-0.05,0 1969,0.06,0 1970,0.04,0 1971,-0.06,0.04 1972,0.02,0.02 1973,0.16,0.01 1974,-0.07,0 1975,-0.01,0.02 1976,-0.12,0 1977,0.15,0.04 1978,0.05,0.08 1979,0.12,0.16 1980,0.23,0.15 1981,0.28,0.2 1982,0.09,0.2 1983,0.27,0.17 1984,0.12,0.14 1985,0.08,0.18 1986,0.14,0.19 1987,0.28,0.22 1988,0.35,0.28 1989,0.24,0.33 1990,0.39,0.31 1991,0.38,0.28 1992,0.19,0.29 1993,0.21,0.3 1994,0.28,0.29 1995,0.43,0.34 1996,0.32,0.42 1997,0.45,0.44 1998,0.61,0.44 1999,0.4,0.48 2000,0.4,0.51 2001,0.52,0.51 2002,0.61,0.53 2003,0.6,0.58 2004,0.51,0.59 2005,0.65,0.59 2006,0.59,0.57 2007,0.62,0.59 2008,0.49,0.59 2009,0.59,0.58 2010,0.66,0.57 2011,0.55,0.59 2012,0.57,0.61 2013,0.6, 2014,0.67, -------------------------------------------------------------------------------- /favicon-dc.ico: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/swcarpentry/python-second-language/074c1d9aa7424104e8a5006623e91eeb6545bbf5/favicon-dc.ico -------------------------------------------------------------------------------- /favicon-swc.ico: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/swcarpentry/python-second-language/074c1d9aa7424104e8a5006623e91eeb6545bbf5/favicon-swc.ico -------------------------------------------------------------------------------- /fig/.gitkeep: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/swcarpentry/python-second-language/074c1d9aa7424104e8a5006623e91eeb6545bbf5/fig/.gitkeep -------------------------------------------------------------------------------- /fig/images_12_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/swcarpentry/python-second-language/074c1d9aa7424104e8a5006623e91eeb6545bbf5/fig/images_12_1.png -------------------------------------------------------------------------------- /fig/images_14_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/swcarpentry/python-second-language/074c1d9aa7424104e8a5006623e91eeb6545bbf5/fig/images_14_1.png -------------------------------------------------------------------------------- /fig/images_3_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/swcarpentry/python-second-language/074c1d9aa7424104e8a5006623e91eeb6545bbf5/fig/images_3_1.png -------------------------------------------------------------------------------- /fig/images_4_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/swcarpentry/python-second-language/074c1d9aa7424104e8a5006623e91eeb6545bbf5/fig/images_4_1.png -------------------------------------------------------------------------------- /fig/images_6_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/swcarpentry/python-second-language/074c1d9aa7424104e8a5006623e91eeb6545bbf5/fig/images_6_1.png -------------------------------------------------------------------------------- /fig/images_7_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/swcarpentry/python-second-language/074c1d9aa7424104e8a5006623e91eeb6545bbf5/fig/images_7_1.png -------------------------------------------------------------------------------- /fig/numpy_11_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/swcarpentry/python-second-language/074c1d9aa7424104e8a5006623e91eeb6545bbf5/fig/numpy_11_1.png -------------------------------------------------------------------------------- /fig/numpy_16_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/swcarpentry/python-second-language/074c1d9aa7424104e8a5006623e91eeb6545bbf5/fig/numpy_16_1.png -------------------------------------------------------------------------------- /fig/numpy_17_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/swcarpentry/python-second-language/074c1d9aa7424104e8a5006623e91eeb6545bbf5/fig/numpy_17_1.png -------------------------------------------------------------------------------- /fig/numpy_18_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/swcarpentry/python-second-language/074c1d9aa7424104e8a5006623e91eeb6545bbf5/fig/numpy_18_1.png -------------------------------------------------------------------------------- /fig/numpy_19_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/swcarpentry/python-second-language/074c1d9aa7424104e8a5006623e91eeb6545bbf5/fig/numpy_19_1.png -------------------------------------------------------------------------------- /fig/numpy_20_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/swcarpentry/python-second-language/074c1d9aa7424104e8a5006623e91eeb6545bbf5/fig/numpy_20_1.png -------------------------------------------------------------------------------- /fig/numpy_23_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/swcarpentry/python-second-language/074c1d9aa7424104e8a5006623e91eeb6545bbf5/fig/numpy_23_1.png -------------------------------------------------------------------------------- /fig/numpy_24_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/swcarpentry/python-second-language/074c1d9aa7424104e8a5006623e91eeb6545bbf5/fig/numpy_24_1.png -------------------------------------------------------------------------------- /fig/numpy_26_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/swcarpentry/python-second-language/074c1d9aa7424104e8a5006623e91eeb6545bbf5/fig/numpy_26_1.png -------------------------------------------------------------------------------- /fig/numpy_9_0.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/swcarpentry/python-second-language/074c1d9aa7424104e8a5006623e91eeb6545bbf5/fig/numpy_9_0.png -------------------------------------------------------------------------------- /fig/pandas_10_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/swcarpentry/python-second-language/074c1d9aa7424104e8a5006623e91eeb6545bbf5/fig/pandas_10_1.png -------------------------------------------------------------------------------- /fig/pandas_14_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/swcarpentry/python-second-language/074c1d9aa7424104e8a5006623e91eeb6545bbf5/fig/pandas_14_1.png -------------------------------------------------------------------------------- /fig/pandas_19_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/swcarpentry/python-second-language/074c1d9aa7424104e8a5006623e91eeb6545bbf5/fig/pandas_19_1.png -------------------------------------------------------------------------------- /fig/pandas_21_2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/swcarpentry/python-second-language/074c1d9aa7424104e8a5006623e91eeb6545bbf5/fig/pandas_21_2.png -------------------------------------------------------------------------------- /fig/pandas_6_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/swcarpentry/python-second-language/074c1d9aa7424104e8a5006623e91eeb6545bbf5/fig/pandas_6_1.png -------------------------------------------------------------------------------- /fig/pandas_8_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/swcarpentry/python-second-language/074c1d9aa7424104e8a5006623e91eeb6545bbf5/fig/pandas_8_1.png -------------------------------------------------------------------------------- /files/.gitkeep: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/swcarpentry/python-second-language/074c1d9aa7424104e8a5006623e91eeb6545bbf5/files/.gitkeep -------------------------------------------------------------------------------- /files/python-second-language-data.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/swcarpentry/python-second-language/074c1d9aa7424104e8a5006623e91eeb6545bbf5/files/python-second-language-data.zip -------------------------------------------------------------------------------- /index.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: lesson 3 | root: . 4 | --- 5 | 6 | This lesson is an introduction to programming in Python 7 | for people who are already comfortable in some other language 8 | such as Perl or MATLAB. 9 | This lesson references the Jupyter Notebook, 10 | but can be taught using a regular Python interpreter as well. 11 | Please note that this lesson uses Python 3 rather than Python 2. 12 | 13 | > ## Under Design 14 | > 15 | > **This lesson is currently in its early design stage; 16 | > please check [the design notes]({{ page.root }}/design/) 17 | > to see an outline of what we have so far, 18 | > or the episodes for more details. 19 | > Contributions are very welcome: 20 | > we would be particularly grateful for exercises 21 | > and for commentary on the ones already there.** 22 | {: .callout} 23 | 24 | > ## Prerequisites 25 | > 26 | > 1. Learners need to understand what files and directories are, 27 | > what a working directory is, 28 | > how to start a Python interpreter, 29 | > and what variables, loops, conditionals, indexing, and function calls are. 30 | > 31 | > 2. Learners must install Python before the class starts. 32 | > 33 | > Please see [the setup instructions]({{ page.root }}/setup/) 34 | > for details. 35 | {: .prereq} 36 | -------------------------------------------------------------------------------- /reference.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: reference 3 | permalink: /reference/ 4 | --- 5 | 6 | FIXME: reference material. 7 | 8 | ## Glossary 9 | 10 | FIXME 11 | -------------------------------------------------------------------------------- /setup.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: page 3 | title: "Setup" 4 | permalink: /setup/ 5 | --- 6 | # Requirements 7 | 8 | 1. Python 3 - Jupyter notebook 9 | 2. Text-editor/Shell 10 | 3. Download/extracted [python-second-language.zip](../files/python-second-language-data.zip) 11 | 12 | ## 1. Installing Python3 and the Jupyter Notebook using Anaconda 13 | 14 | [Python](https://python.org) is a popular language for scientific computing, and great for 15 | general-purpose programming as well. Installing all of its scientific packages 16 | individually can be a bit difficult, so we recommend 17 | [Anaconda](https://www.continuum.io/anaconda), an all-in-one 18 | installer. 19 | 20 | Regardless of how you choose to install it, please make sure you install Python 21 | version 3.x (e.g., 3.4 is fine). 22 | 23 | We will teach Python using the Jupyter notebook, a programming environment that 24 | runs in a web browser. For this to work you will need a reasonably up-to-date 25 | browser. The current versions of the Chrome, Safari and Firefox browsers are all 26 | supported (some older browsers, including Internet Explorer version 9 and below, 27 | are not). Enumerated below are setup instructions for Windows, Mac OS X, and 28 | Linux. Please setup your python environment at least a day in advance of the 29 | workshop. If you encounter problems with the installation procedure, please ask 30 | your workshop organizers via e-mail for assistance getting set up. 31 | 32 | ### Windows - [Video tutorial](https://www.youtube.com/watch?v=xxQ0mzZ8UvA) 33 | 34 | 1. Open [http://continuum.io/downloads](http://continuum.io/downloads) with your web browser. 35 | 2. Download the Python 3 installer for Windows. 36 | 3. Install Python 3 using all of the defaults for installation except make sure to 37 | check **Make Anaconda the default Python**. 38 | 39 | ### Mac OS X - [Video tutorial](https://www.youtube.com/watch?v=TcSAln46u9U) 40 | 41 | 1. Open [http://continuum.io/downloads](http://continuum.io/downloads) with your web browser. 42 | 2. Download the Python 3 installer for OS X. 43 | 3. Install Python 3 using all of the defaults for installation. 44 | 45 | ### Linux 46 | 47 | 1. Open [http://continuum.io/downloads](http://continuum.io/downloads) with your web browser. 48 | 49 | 2. Download the Python 3 installer for OS X. 50 | 51 | 3. Install Python 3 using all of the defaults for installation. Note that 52 | installation requires using the shell, if you aren't comfortable doing then 53 | installation yourself then stop here and request help before the workshop 54 | begins. 55 | 56 | 4. Open a terminal window. 57 | 58 | 5. Type 59 | 60 | ~~~ 61 | $ bash Anaconda3- 62 | ~~~ 63 | {: .bash} 64 | 65 | and press tab. The name of the file you just downloaded should appear. 66 | 67 | 6. Press enter. You will follow the text-only prompts. When there is a colon 68 | at the bottom of the screen press the down arrow to move down through the text. 69 | Type `yes` and press enter to approve the license. Press enter to approve the 70 | default location for the files. Type `yes` and press enter to prepend Anaconda to 71 | your `PATH` (this makes the Anaconda distribution the default Python). 72 | 73 | ## 2. Text Editor/Shell 74 | 75 | We'll have at least one lesson covering python scripting from the command line. **For this 76 | section, we recommend using whichever text editor and shell (e.g. emacs/vim and bash) you 77 | typically use to program.** If you're not familiar with the command line or a text editor, 78 | Anaconda (detailed above) ships with a Python IDE, Spyder, which you can use to set up a 79 | text editor and command line window side-by-side. You can do this by 80 | 81 | * Windows 82 | 1. Start Menu -> Anaconda -> Spyder 83 | 2. On the upper toolbar, select "Tools->Open Command Prompt" 84 | 3. Create a new file using the square button in the upper left 85 | 4. Your setup allows you to edit/save/create new files in the left pane and run 86 | commands on the command line in the lower right pane 87 | 88 | * Mac/Linux 89 | 1. Open "Terminal" and type "spyder" at the command prompt 90 | 2. On the upper toolbar, select "Tools->Open Terminal" 91 | 3. Create a new file using the square button in the upper left 92 | 4. Your setup allows you to edit/save/create new files in the left pane and run 93 | commands on the command line in the lower right pane 94 | 95 | ## 3. Class Dataset 96 | 97 | The data we will be using is taken from the 98 | [EPA atmospheric greenhouse gases](https://www3.epa.gov/climatechange/science/indicators/ghg/ghg-concentrations.html) dataset. 99 | To obtain it, download and unzip the file [python-second-language-data.zip](../files/python-second-language-data.zip). 100 | In order to follow the presented material, you should create the jupyter notebook in the "data" directory. 101 | --------------------------------------------------------------------------------