├── Week9 ├── images │ └── BBQ.gif ├── lesson2.md ├── lesson3.md ├── assignment │ ├── xml │ │ ├── LGA │ │ ├── DEN │ │ ├── EWR │ │ ├── LAX │ │ ├── PIT │ │ ├── SEA │ │ ├── STL │ │ ├── DTW │ │ ├── LAS │ │ ├── SFO │ │ ├── BWI │ │ ├── DFW │ │ ├── IAH │ │ ├── PHL │ │ ├── PHX │ │ ├── ATL │ │ ├── BOS │ │ ├── CLT │ │ ├── ORD │ │ └── MSP │ └── README.md ├── lesson1.md └── README.md ├── Week14 ├── images │ └── BBQ.gif ├── assignment │ ├── top20.db │ ├── xml │ │ ├── LGA │ │ ├── DEN │ │ ├── EWR │ │ ├── LAX │ │ ├── STL │ │ ├── DTW │ │ ├── LAS │ │ ├── PIT │ │ ├── SEA │ │ ├── DFW │ │ ├── IAH │ │ ├── PHL │ │ ├── PHX │ │ ├── SFO │ │ ├── ATL │ │ ├── BWI │ │ ├── CLT │ │ ├── BOS │ │ ├── ORD │ │ └── MSP │ └── README.md ├── notebook │ └── images │ │ ├── dom-tree.png │ │ ├── html-dom.png │ │ ├── html-view.png │ │ ├── dom-element.png │ │ └── html-element.png ├── lesson1.md ├── lesson3.md ├── lesson2.md └── README.md ├── Week10 ├── assignment │ ├── kde.png │ ├── hexbin.png │ ├── pairplot.png │ ├── pmf_200.png │ ├── pmf_50.png │ ├── cdf_short.png │ ├── cdf_arrival_delay.png │ └── README.md ├── images │ └── relational_databases.jpg ├── lesson1.md ├── lesson3.md ├── README.md └── lesson2.md ├── images ├── pull_request_0.png ├── pull_request_1.png ├── pull_request_2.png ├── pull_request_3.png ├── pull_request_4.png ├── python_2_or_3.png └── Draft_Version_picture.png ├── Week1 ├── images │ ├── docker-jrepo.png │ ├── docker-jterm.png │ ├── docker-pull.png │ ├── docker-server.png │ ├── docker-shared.png │ ├── docker-test.png │ ├── docker-jupyter.png │ ├── docker-pull-win.png │ ├── docker-run-repo.png │ ├── docker-terminal.png │ ├── docker-test-win.png │ ├── UNIX-Licence-Plate.JPG │ ├── docker-helloworld.png │ ├── docker-jnotebook.png │ ├── docker-server-win.png │ ├── docker-shared-win.png │ ├── docker-test-repo.png │ ├── docker-run-repo-win.png │ ├── docker-terminal-win.png │ ├── docker-test-repo-win.png │ └── docker-helloworld-win.png ├── notebooks │ └── images │ │ ├── docker-ls.png │ │ ├── docker-rm.png │ │ └── docker-root.png ├── README.md ├── lesson1.md ├── lesson3.md └── lesson2.md ├── Week2 ├── images │ └── command-line.png ├── notebooks │ └── images │ │ ├── ipynb-unix.png │ │ ├── cell-toolbar.png │ │ └── ipython-help.png ├── assignment │ └── README.md ├── lesson1.md ├── lesson3.md ├── README.md └── lesson2.md ├── Week3 ├── images │ └── python-logo.png ├── assignment │ └── README.md ├── lesson1.md ├── lesson3.md ├── lesson2.md └── README.md ├── Week6 ├── images │ ├── extrapolating.png │ └── regular_expressions.png ├── lesson2.md ├── lesson3.md ├── lesson1.md ├── README.md └── assignment │ └── README.md ├── Week8 ├── images │ └── pandas_logo.png ├── assignment │ ├── month_cancelled.png │ └── README.md ├── lesson3.md ├── lesson2.md ├── lesson1.md └── README.md ├── Week5 ├── notebooks │ └── images │ │ ├── wget.png │ │ └── docker-exec.png ├── images │ └── xkcd_sustainable.png ├── lesson3.md ├── lesson2.md ├── lesson1.md ├── README.md └── assignment │ └── README.md ├── Week15 ├── images │ └── bluewatersimage.jpg ├── lesson1.md └── README.md ├── Week7 ├── images │ └── conditional_risk.png ├── lesson3.md ├── lesson1.md ├── lesson2.md ├── README.md └── assignment │ └── README.md ├── Week11 ├── images │ └── sqlite_python_logo.png ├── lesson2.md ├── README.md ├── lesson3.md ├── lesson1.md └── assignment │ └── README.md ├── Week4 ├── notebooks │ └── images │ │ ├── shell-view.png │ │ └── shell-script.png ├── lesson1.md ├── lesson2.md ├── lesson3.md ├── README.md └── assignment │ └── README.md ├── Week12 ├── images │ └── relational_databases.jpg ├── lesson1.md ├── lesson2.md ├── lesson3.md ├── README.md └── assignment │ └── README.md ├── orientation ├── notebooks │ └── images │ │ ├── git-add.png │ │ ├── git-clone.png │ │ ├── git-init.png │ │ ├── phdcomics.gif │ │ ├── git-commit.png │ │ ├── git-config.png │ │ ├── git-website.png │ │ ├── github-app.png │ │ ├── github-button.png │ │ ├── github-website.png │ │ └── github-appclient.png ├── Pre-Class_Activity.md └── README.md ├── docker-standalone ├── notebook.sh ├── Dockerfile └── README.md ├── .gitignore ├── LICENSE.md ├── Week13 ├── lesson1.md ├── README.md ├── lesson2.md ├── lesson3.md └── assignment │ └── README.md └── README.md /Week9/images/BBQ.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week9/images/BBQ.gif -------------------------------------------------------------------------------- /Week14/images/BBQ.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week14/images/BBQ.gif -------------------------------------------------------------------------------- /Week10/assignment/kde.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week10/assignment/kde.png -------------------------------------------------------------------------------- /images/pull_request_0.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/images/pull_request_0.png -------------------------------------------------------------------------------- /images/pull_request_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/images/pull_request_1.png -------------------------------------------------------------------------------- /images/pull_request_2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/images/pull_request_2.png -------------------------------------------------------------------------------- /images/pull_request_3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/images/pull_request_3.png -------------------------------------------------------------------------------- /images/pull_request_4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/images/pull_request_4.png -------------------------------------------------------------------------------- /images/python_2_or_3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/images/python_2_or_3.png -------------------------------------------------------------------------------- /Week14/assignment/top20.db: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week14/assignment/top20.db -------------------------------------------------------------------------------- /Week1/images/docker-jrepo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week1/images/docker-jrepo.png -------------------------------------------------------------------------------- /Week1/images/docker-jterm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week1/images/docker-jterm.png -------------------------------------------------------------------------------- /Week1/images/docker-pull.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week1/images/docker-pull.png -------------------------------------------------------------------------------- /Week1/images/docker-server.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week1/images/docker-server.png -------------------------------------------------------------------------------- /Week1/images/docker-shared.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week1/images/docker-shared.png -------------------------------------------------------------------------------- /Week1/images/docker-test.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week1/images/docker-test.png -------------------------------------------------------------------------------- /Week10/assignment/hexbin.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week10/assignment/hexbin.png -------------------------------------------------------------------------------- /Week10/assignment/pairplot.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week10/assignment/pairplot.png -------------------------------------------------------------------------------- /Week10/assignment/pmf_200.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week10/assignment/pmf_200.png -------------------------------------------------------------------------------- /Week10/assignment/pmf_50.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week10/assignment/pmf_50.png -------------------------------------------------------------------------------- /Week2/images/command-line.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week2/images/command-line.png -------------------------------------------------------------------------------- /Week3/images/python-logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week3/images/python-logo.png -------------------------------------------------------------------------------- /Week6/images/extrapolating.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week6/images/extrapolating.png -------------------------------------------------------------------------------- /Week8/images/pandas_logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week8/images/pandas_logo.png -------------------------------------------------------------------------------- /Week1/images/docker-jupyter.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week1/images/docker-jupyter.png -------------------------------------------------------------------------------- /Week1/images/docker-pull-win.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week1/images/docker-pull-win.png -------------------------------------------------------------------------------- /Week1/images/docker-run-repo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week1/images/docker-run-repo.png -------------------------------------------------------------------------------- /Week1/images/docker-terminal.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week1/images/docker-terminal.png -------------------------------------------------------------------------------- /Week1/images/docker-test-win.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week1/images/docker-test-win.png -------------------------------------------------------------------------------- /Week10/assignment/cdf_short.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week10/assignment/cdf_short.png -------------------------------------------------------------------------------- /Week5/notebooks/images/wget.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week5/notebooks/images/wget.png -------------------------------------------------------------------------------- /images/Draft_Version_picture.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/images/Draft_Version_picture.png -------------------------------------------------------------------------------- /Week1/images/UNIX-Licence-Plate.JPG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week1/images/UNIX-Licence-Plate.JPG -------------------------------------------------------------------------------- /Week1/images/docker-helloworld.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week1/images/docker-helloworld.png -------------------------------------------------------------------------------- /Week1/images/docker-jnotebook.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week1/images/docker-jnotebook.png -------------------------------------------------------------------------------- /Week1/images/docker-server-win.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week1/images/docker-server-win.png -------------------------------------------------------------------------------- /Week1/images/docker-shared-win.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week1/images/docker-shared-win.png -------------------------------------------------------------------------------- /Week1/images/docker-test-repo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week1/images/docker-test-repo.png -------------------------------------------------------------------------------- /Week14/notebook/images/dom-tree.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week14/notebook/images/dom-tree.png -------------------------------------------------------------------------------- /Week14/notebook/images/html-dom.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week14/notebook/images/html-dom.png -------------------------------------------------------------------------------- /Week15/images/bluewatersimage.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week15/images/bluewatersimage.jpg -------------------------------------------------------------------------------- /Week5/images/xkcd_sustainable.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week5/images/xkcd_sustainable.png -------------------------------------------------------------------------------- /Week7/images/conditional_risk.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week7/images/conditional_risk.png -------------------------------------------------------------------------------- /Week1/images/docker-run-repo-win.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week1/images/docker-run-repo-win.png -------------------------------------------------------------------------------- /Week1/images/docker-terminal-win.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week1/images/docker-terminal-win.png -------------------------------------------------------------------------------- /Week1/images/docker-test-repo-win.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week1/images/docker-test-repo-win.png -------------------------------------------------------------------------------- /Week1/notebooks/images/docker-ls.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week1/notebooks/images/docker-ls.png -------------------------------------------------------------------------------- /Week1/notebooks/images/docker-rm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week1/notebooks/images/docker-rm.png -------------------------------------------------------------------------------- /Week11/images/sqlite_python_logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week11/images/sqlite_python_logo.png -------------------------------------------------------------------------------- /Week14/notebook/images/html-view.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week14/notebook/images/html-view.png -------------------------------------------------------------------------------- /Week2/notebooks/images/ipynb-unix.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week2/notebooks/images/ipynb-unix.png -------------------------------------------------------------------------------- /Week4/notebooks/images/shell-view.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week4/notebooks/images/shell-view.png -------------------------------------------------------------------------------- /Week6/images/regular_expressions.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week6/images/regular_expressions.png -------------------------------------------------------------------------------- /Week8/assignment/month_cancelled.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week8/assignment/month_cancelled.png -------------------------------------------------------------------------------- /Week1/images/docker-helloworld-win.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week1/images/docker-helloworld-win.png -------------------------------------------------------------------------------- /Week1/notebooks/images/docker-root.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week1/notebooks/images/docker-root.png -------------------------------------------------------------------------------- /Week10/assignment/cdf_arrival_delay.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week10/assignment/cdf_arrival_delay.png -------------------------------------------------------------------------------- /Week10/images/relational_databases.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week10/images/relational_databases.jpg -------------------------------------------------------------------------------- /Week12/images/relational_databases.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week12/images/relational_databases.jpg -------------------------------------------------------------------------------- /Week14/notebook/images/dom-element.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week14/notebook/images/dom-element.png -------------------------------------------------------------------------------- /Week14/notebook/images/html-element.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week14/notebook/images/html-element.png -------------------------------------------------------------------------------- /Week2/notebooks/images/cell-toolbar.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week2/notebooks/images/cell-toolbar.png -------------------------------------------------------------------------------- /Week2/notebooks/images/ipython-help.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week2/notebooks/images/ipython-help.png -------------------------------------------------------------------------------- /Week4/notebooks/images/shell-script.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week4/notebooks/images/shell-script.png -------------------------------------------------------------------------------- /Week5/notebooks/images/docker-exec.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/Week5/notebooks/images/docker-exec.png -------------------------------------------------------------------------------- /orientation/notebooks/images/git-add.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/orientation/notebooks/images/git-add.png -------------------------------------------------------------------------------- /orientation/notebooks/images/git-clone.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/orientation/notebooks/images/git-clone.png -------------------------------------------------------------------------------- /orientation/notebooks/images/git-init.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/orientation/notebooks/images/git-init.png -------------------------------------------------------------------------------- /orientation/notebooks/images/phdcomics.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/orientation/notebooks/images/phdcomics.gif -------------------------------------------------------------------------------- /orientation/notebooks/images/git-commit.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/orientation/notebooks/images/git-commit.png -------------------------------------------------------------------------------- /orientation/notebooks/images/git-config.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/orientation/notebooks/images/git-config.png -------------------------------------------------------------------------------- /orientation/notebooks/images/git-website.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/orientation/notebooks/images/git-website.png -------------------------------------------------------------------------------- /orientation/notebooks/images/github-app.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/orientation/notebooks/images/github-app.png -------------------------------------------------------------------------------- /orientation/notebooks/images/github-button.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/orientation/notebooks/images/github-button.png -------------------------------------------------------------------------------- /orientation/notebooks/images/github-website.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/orientation/notebooks/images/github-website.png -------------------------------------------------------------------------------- /orientation/notebooks/images/github-appclient.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lcdm-uiuc/info490-fa15/HEAD/orientation/notebooks/images/github-appclient.png -------------------------------------------------------------------------------- /docker-standalone/notebook.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # need a wrapper script 4 | # https://github.com/ipython/ipython/issues/7062 5 | 6 | ipython notebook --no-browser --port=8888 --ip=* --matplotlib=inline 7 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Max OSX 2 | .DS_Store 3 | 4 | # BASH 5 | .bash_history 6 | 7 | # Byte-compiled / optimized / DLL files 8 | __pycache__/ 9 | *.py[cod] 10 | 11 | # IPython 12 | .ipynb_checkpoints 13 | notebooks/.ipynb_checkpoints 14 | .ipython 15 | */notebooks/.ipynb_checkpoints 16 | */.ipynb_checkpoints 17 | 18 | Week7/notebooks/test.pdf 19 | -------------------------------------------------------------------------------- /docker-standalone/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM ipython/scipystack 2 | 3 | MAINTAINER Edward J Kim 4 | 5 | RUN apt-get update && \ 6 | 7 | apt-get install -y -q \ 8 | vim \ 9 | wget \ 10 | # fonts needed in seaborn 11 | ttf-liberation \ 12 | ttf-bitstream-vera \ 13 | python3-bs4 \ 14 | python3-tables 15 | 16 | ADD notebook.sh / 17 | 18 | RUN chown 1000:1000 /notebook.sh && \ 19 | chmod u+x /notebook.sh && \ 20 | useradd -m -u 1000 -s /bin/bash data_scientist 21 | 22 | USER data_scientist 23 | ENV HOME /home/data_scientist 24 | ENV SHELL /bin/bash 25 | ENV USER data_scientist 26 | 27 | WORKDIR /home/data_scientist/ 28 | 29 | EXPOSE 8888 30 | 31 | # set colors for terminal prompt 32 | RUN echo 'export PS1="\e[1;34m\u@\h:\w$ \e[0m"' >> /home/data_scientist/.bashrc 33 | 34 | CMD ["/notebook.sh"] 35 | -------------------------------------------------------------------------------- /Week9/lesson2.md: -------------------------------------------------------------------------------- 1 | # Week 9 Lesson 2 # 2 | ## JSON Data Format ## 3 | 4 | In this lesson, you will learn about creating, and reading from a JSON file. 5 | 6 | ###Objectives ### 7 | By the end of this lesson, you will be able to: 8 | 9 | - Understand how to read and write a JSON format file. 10 | 11 | 12 | ### Time Estimate ### 13 | 14 | Approximately 2 hours. 15 | 16 | ### Readings #### 17 | 18 | - Course [IPython Notebook](notebooks/json-dataformats.ipynb) on Data Formats 19 | 20 | - The [JSON](https://en.wikipedia.org/wiki/JSON) Document format 21 | 22 | #### *Optional Additional Readings*#### 23 | 24 | - The [JSON](http://json.org/) format 25 | 26 | - Storing big data in a [JSON](http://smallworldbigdata.com/tag/json/) like format. 27 | 28 | ### Assessment ### 29 | 30 | When you have completed and worked through the above readings, please take the [Week 9 Lesson 2 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095578). 31 | -------------------------------------------------------------------------------- /docker-standalone/README.md: -------------------------------------------------------------------------------- 1 | # A standalone Docker image 2 | 3 | There are two ways to use this Docker image. You can either build your own image from `Dockerfile` or `docker pull` from Dockerhub. 4 | 5 | ## Build your own 6 | 7 | To build to own image, clone the repository 8 | 9 | ```shell 10 | git clone https://github.com/UI-DataScience/info490-fa15 11 | ``` 12 | 13 | or if you have already cloned it, change into `info490-fa15/docker-standalone` directory 14 | 15 | ```shell 16 | cd info490-fa15/docker-standalone 17 | ``` 18 | 19 | Build a Docker image with 20 | 21 | ```shell 22 | docker build --rm -t lcdm/standalone-info490 . 23 | ``` 24 | 25 | ## Docker pull 26 | 27 | Pull the image from Dockerhub by doing 28 | 29 | ```shell 30 | docker pull lcdm/standalone-info490 31 | ``` 32 | 33 | ## Run IPython/Jupyter notebook server 34 | 35 | To run an Ipython/Jupyter notebook server, run 36 | 37 | ```shell 38 | docker run -d -p 8888:8888 --name standalone lcdm/standalone-info490 39 | ``` 40 | 41 | Open your web browser, and now you can access the notebook server at `127.0.0.1:8888` or `localhost:8888`. 42 | -------------------------------------------------------------------------------- /Week9/lesson3.md: -------------------------------------------------------------------------------- 1 | # Week 9 Lesson 3 # 2 | ## XML Data Format ## 3 | 4 | In this lesson, you will learn about what the XML Data format is, how to parse a document, and how to read and write an XML file. 5 | 6 | ###Objectives ### 7 | By the end of this lesson, you will be able to: 8 | 9 | - Understand how to read and write an XML formatted file. 10 | 11 | ### Time Estimate ### 12 | 13 | Approximately 2 hours. 14 | 15 | ### Readings #### 16 | 17 | - Course [IPython Notebook](notebooks/xml-dataformat.ipynb) on Data Formats 18 | 19 | - The [XML](https://en.wikipedia.org/wiki/XML) Document format 20 | 21 | 22 | #### *Optional Additional Readings*#### 23 | 24 | - The [XML](http://www.w3.org/XML/) format 25 | 26 | - The [SVG](https://en.wikipedia.org/wiki/Scalable_Vector_Graphics) 27 | format (an XML-based image specification) 28 | 29 | - The [HTML](https://en.wikipedia.org/wiki/HTML) format (an XML-based 30 | document specification). 31 | 32 | ### Assessment ### 33 | 34 | When you have completed and worked through the above readings, please take the [Week 9 Lesson 3 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095581). 35 | -------------------------------------------------------------------------------- /Week9/assignment/xml/LGA: -------------------------------------------------------------------------------- 1 | 2 | false 3 | 4 | LGA 5 | 6 | New York 7 | 8 | La Guardia 9 | 10 | 10.00 11 | 12 | Mostly Cloudy 13 | 14 | NOAA's National Weather Service 15 | 16 | 1:51 PM Local 17 | 18 | http://weather.gov/ 19 | 20 | 61.0 F (16.1 C) 21 | 22 | Northwest at 13.8mph 23 | 24 | KLGA 25 | 26 | New York 27 | 28 | No known delays for this airport. 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /Week14/assignment/xml/LGA: -------------------------------------------------------------------------------- 1 | 2 | false 3 | 4 | LGA 5 | 6 | New York 7 | 8 | La Guardia 9 | 10 | 10.00 11 | 12 | Mostly Cloudy 13 | 14 | NOAA's National Weather Service 15 | 16 | 1:51 PM Local 17 | 18 | http://weather.gov/ 19 | 20 | 61.0 F (16.1 C) 21 | 22 | Northwest at 13.8mph 23 | 24 | KLGA 25 | 26 | New York 27 | 28 | No known delays for this airport. 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /Week14/assignment/xml/DEN: -------------------------------------------------------------------------------- 1 | 2 | false 3 | 4 | DEN 5 | 6 | Colorado 7 | 8 | Denver International 9 | 10 | 10.00 11 | 12 | Partly Cloudy 13 | 14 | NOAA's National Weather Service 15 | 16 | 11:53 AM Local 17 | 18 | http://weather.gov/ 19 | 20 | 61.0 F (16.1 C) 21 | 22 | Variable at 4.6mph 23 | 24 | KDEN 25 | 26 | Denver 27 | 28 | No known delays for this airport. 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /Week9/assignment/xml/DEN: -------------------------------------------------------------------------------- 1 | 2 | false 3 | 4 | DEN 5 | 6 | Colorado 7 | 8 | Denver International 9 | 10 | 10.00 11 | 12 | Partly Cloudy 13 | 14 | NOAA's National Weather Service 15 | 16 | 11:53 AM Local 17 | 18 | http://weather.gov/ 19 | 20 | 61.0 F (16.1 C) 21 | 22 | Variable at 4.6mph 23 | 24 | KDEN 25 | 26 | Denver 27 | 28 | No known delays for this airport. 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /Week9/assignment/xml/EWR: -------------------------------------------------------------------------------- 1 | 2 | false 3 | 4 | EWR 5 | 6 | New Jersey 7 | 8 | Newark International 9 | 10 | 10.00 11 | 12 | Mostly Cloudy 13 | 14 | NOAA's National Weather Service 15 | 16 | 1:51 PM Local 17 | 18 | http://weather.gov/ 19 | 20 | 63.0 F (17.2 C) 21 | 22 | Northwest at 13.8mph 23 | 24 | KEWR 25 | 26 | Newark 27 | 28 | No known delays for this airport. 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /Week14/assignment/xml/EWR: -------------------------------------------------------------------------------- 1 | 2 | false 3 | 4 | EWR 5 | 6 | New Jersey 7 | 8 | Newark International 9 | 10 | 10.00 11 | 12 | Mostly Cloudy 13 | 14 | NOAA's National Weather Service 15 | 16 | 1:51 PM Local 17 | 18 | http://weather.gov/ 19 | 20 | 63.0 F (17.2 C) 21 | 22 | Northwest at 13.8mph 23 | 24 | KEWR 25 | 26 | Newark 27 | 28 | No known delays for this airport. 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /Week14/assignment/xml/LAX: -------------------------------------------------------------------------------- 1 | 2 | false 3 | 4 | LAX 5 | 6 | California 7 | 8 | Los Angeles International 9 | 10 | 8.00 11 | 12 | Overcast 13 | 14 | NOAA's National Weather Service 15 | 16 | 10:53 AM Local 17 | 18 | http://weather.gov/ 19 | 20 | 71.0 F (21.7 C) 21 | 22 | Southwest at 5.8mph 23 | 24 | KLAX 25 | 26 | Los Angeles 27 | 28 | No known delays for this airport. 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /Week14/assignment/xml/STL: -------------------------------------------------------------------------------- 1 | 2 | false 3 | 4 | STL 5 | 6 | Missouri 7 | 8 | Lambert-St Louis International 9 | 10 | 10.00 11 | 12 | Fair 13 | 14 | NOAA's National Weather Service 15 | 16 | 12:51 PM Local 17 | 18 | http://weather.gov/ 19 | 20 | 63.0 F (17.2 C) 21 | 22 | Northwest at 15.0mph 23 | 24 | KSTL 25 | 26 | St Louis 27 | 28 | No known delays for this airport. 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /Week9/assignment/xml/LAX: -------------------------------------------------------------------------------- 1 | 2 | false 3 | 4 | LAX 5 | 6 | California 7 | 8 | Los Angeles International 9 | 10 | 8.00 11 | 12 | Overcast 13 | 14 | NOAA's National Weather Service 15 | 16 | 10:53 AM Local 17 | 18 | http://weather.gov/ 19 | 20 | 71.0 F (21.7 C) 21 | 22 | Southwest at 5.8mph 23 | 24 | KLAX 25 | 26 | Los Angeles 27 | 28 | No known delays for this airport. 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /Week9/assignment/xml/PIT: -------------------------------------------------------------------------------- 1 | 2 | false 3 | 4 | PIT 5 | 6 | Pennsylvania 7 | 8 | Pittsburgh International 9 | 10 | 10.00 11 | 12 | Mostly Cloudy 13 | 14 | NOAA's National Weather Service 15 | 16 | 1:51 PM Local 17 | 18 | http://weather.gov/ 19 | 20 | 55.0 F (12.8 C) 21 | 22 | West at 16.1mph 23 | 24 | KPIT 25 | 26 | Pittsburgh 27 | 28 | No known delays for this airport. 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /Week9/assignment/xml/SEA: -------------------------------------------------------------------------------- 1 | 2 | false 3 | 4 | SEA 5 | 6 | Washington 7 | 8 | Seattle-Tacoma International 9 | 10 | 10.00 11 | 12 | Partly Cloudy 13 | 14 | NOAA's National Weather Service 15 | 16 | 10:53 AM Local 17 | 18 | http://weather.gov/ 19 | 20 | 58.0 F (14.4 C) 21 | 22 | North at 3.5mph 23 | 24 | KSEA 25 | 26 | Seattle 27 | 28 | No known delays for this airport. 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /Week9/assignment/xml/STL: -------------------------------------------------------------------------------- 1 | 2 | false 3 | 4 | STL 5 | 6 | Missouri 7 | 8 | Lambert-St Louis International 9 | 10 | 10.00 11 | 12 | Fair 13 | 14 | NOAA's National Weather Service 15 | 16 | 12:51 PM Local 17 | 18 | http://weather.gov/ 19 | 20 | 63.0 F (17.2 C) 21 | 22 | Northwest at 15.0mph 23 | 24 | KSTL 25 | 26 | St Louis 27 | 28 | No known delays for this airport. 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /Week14/assignment/xml/DTW: -------------------------------------------------------------------------------- 1 | 2 | false 3 | 4 | DTW 5 | 6 | Michigan 7 | 8 | Detroit Metropolitan Wayne County 9 | 10 | 10.00 11 | 12 | Mostly Cloudy 13 | 14 | NOAA's National Weather Service 15 | 16 | 1:53 PM Local 17 | 18 | http://weather.gov/ 19 | 20 | 53.0 F (11.7 C) 21 | 22 | West at 16.1mph 23 | 24 | KDTW 25 | 26 | Detroit 27 | 28 | No known delays for this airport. 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /Week14/assignment/xml/LAS: -------------------------------------------------------------------------------- 1 | 2 | false 3 | 4 | LAS 5 | 6 | Nevada 7 | 8 | Las Vegas McCarran International 9 | 10 | 10.00 11 | 12 | Mostly Cloudy 13 | 14 | NOAA's National Weather Service 15 | 16 | 10:56 AM Local 17 | 18 | http://weather.gov/ 19 | 20 | 76.0 F (24.4 C) 21 | 22 | West at 10.4mph 23 | 24 | KLAS 25 | 26 | Las Vegas 27 | 28 | No known delays for this airport. 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /Week14/assignment/xml/PIT: -------------------------------------------------------------------------------- 1 | 2 | false 3 | 4 | PIT 5 | 6 | Pennsylvania 7 | 8 | Pittsburgh International 9 | 10 | 10.00 11 | 12 | Mostly Cloudy 13 | 14 | NOAA's National Weather Service 15 | 16 | 1:51 PM Local 17 | 18 | http://weather.gov/ 19 | 20 | 55.0 F (12.8 C) 21 | 22 | West at 16.1mph 23 | 24 | KPIT 25 | 26 | Pittsburgh 27 | 28 | No known delays for this airport. 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /Week14/assignment/xml/SEA: -------------------------------------------------------------------------------- 1 | 2 | false 3 | 4 | SEA 5 | 6 | Washington 7 | 8 | Seattle-Tacoma International 9 | 10 | 10.00 11 | 12 | Partly Cloudy 13 | 14 | NOAA's National Weather Service 15 | 16 | 10:53 AM Local 17 | 18 | http://weather.gov/ 19 | 20 | 58.0 F (14.4 C) 21 | 22 | North at 3.5mph 23 | 24 | KSEA 25 | 26 | Seattle 27 | 28 | No known delays for this airport. 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /Week9/assignment/xml/DTW: -------------------------------------------------------------------------------- 1 | 2 | false 3 | 4 | DTW 5 | 6 | Michigan 7 | 8 | Detroit Metropolitan Wayne County 9 | 10 | 10.00 11 | 12 | Mostly Cloudy 13 | 14 | NOAA's National Weather Service 15 | 16 | 1:53 PM Local 17 | 18 | http://weather.gov/ 19 | 20 | 53.0 F (11.7 C) 21 | 22 | West at 16.1mph 23 | 24 | KDTW 25 | 26 | Detroit 27 | 28 | No known delays for this airport. 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /Week9/assignment/xml/LAS: -------------------------------------------------------------------------------- 1 | 2 | false 3 | 4 | LAS 5 | 6 | Nevada 7 | 8 | Las Vegas McCarran International 9 | 10 | 10.00 11 | 12 | Mostly Cloudy 13 | 14 | NOAA's National Weather Service 15 | 16 | 10:56 AM Local 17 | 18 | http://weather.gov/ 19 | 20 | 76.0 F (24.4 C) 21 | 22 | West at 10.4mph 23 | 24 | KLAS 25 | 26 | Las Vegas 27 | 28 | No known delays for this airport. 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /Week9/assignment/xml/SFO: -------------------------------------------------------------------------------- 1 | 2 | false 3 | 4 | SFO 5 | 6 | California 7 | 8 | San Francisco International 9 | 10 | 10.00 11 | 12 | Mostly Cloudy 13 | 14 | NOAA's National Weather Service 15 | 16 | 10:56 AM Local 17 | 18 | http://weather.gov/ 19 | 20 | 66.0 F (18.9 C) 21 | 22 | West at 12.7mph 23 | 24 | KSFO 25 | 26 | San Francisco 27 | 28 | No known delays for this airport. 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /Week14/assignment/xml/DFW: -------------------------------------------------------------------------------- 1 | 2 | false 3 | 4 | DFW 5 | 6 | Texas 7 | 8 | Dallas/Ft Worth International 9 | 10 | 10.00 11 | 12 | A Few Clouds 13 | 14 | NOAA's National Weather Service 15 | 16 | 12:53 PM Local 17 | 18 | http://weather.gov/ 19 | 20 | 81.0 F (27.2 C) 21 | 22 | Northeast at 11.5mph 23 | 24 | KDFW 25 | 26 | Dallas-Ft Worth 27 | 28 | No known delays for this airport. 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /Week14/assignment/xml/IAH: -------------------------------------------------------------------------------- 1 | 2 | false 3 | 4 | IAH 5 | 6 | Texas 7 | 8 | George Bush Intercontinental/Houston 9 | 10 | 10.00 11 | 12 | A Few Clouds 13 | 14 | NOAA's National Weather Service 15 | 16 | 12:53 PM Local 17 | 18 | http://weather.gov/ 19 | 20 | 86.0 F (30.0 C) 21 | 22 | Northeast at 10.4mph 23 | 24 | KIAH 25 | 26 | Houston 27 | 28 | No known delays for this airport. 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /Week14/assignment/xml/PHL: -------------------------------------------------------------------------------- 1 | 2 | false 3 | 4 | PHL 5 | 6 | Pennsylvania 7 | 8 | Philadelphia International 9 | 10 | 10.00 11 | 12 | Partly Cloudy 13 | 14 | NOAA's National Weather Service 15 | 16 | 1:54 PM Local 17 | 18 | http://weather.gov/ 19 | 20 | 65.0 F (18.3 C) 21 | 22 | Northwest at 15.0mph 23 | 24 | KPHL 25 | 26 | Philadelphia 27 | 28 | No known delays for this airport. 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /Week14/assignment/xml/PHX: -------------------------------------------------------------------------------- 1 | 2 | false 3 | 4 | PHX 5 | 6 | Arizona 7 | 8 | Phoenix Sky Harbor International 9 | 10 | 10.00 11 | 12 | Mostly Cloudy 13 | 14 | NOAA's National Weather Service 15 | 16 | 10:51 AM Local 17 | 18 | http://weather.gov/ 19 | 20 | 91.0 F (32.8 C) 21 | 22 | Southeast at 12.7mph 23 | 24 | KPHX 25 | 26 | Phoenix 27 | 28 | No known delays for this airport. 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /Week14/assignment/xml/SFO: -------------------------------------------------------------------------------- 1 | 2 | false 3 | 4 | SFO 5 | 6 | California 7 | 8 | San Francisco International 9 | 10 | 10.00 11 | 12 | Mostly Cloudy 13 | 14 | NOAA's National Weather Service 15 | 16 | 10:56 AM Local 17 | 18 | http://weather.gov/ 19 | 20 | 66.0 F (18.9 C) 21 | 22 | West at 12.7mph 23 | 24 | KSFO 25 | 26 | San Francisco 27 | 28 | No known delays for this airport. 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /Week9/assignment/xml/BWI: -------------------------------------------------------------------------------- 1 | 2 | false 3 | 4 | BWI 5 | 6 | Maryland 7 | 8 | Baltimore-Washington International 9 | 10 | 10.00 11 | 12 | Mostly Cloudy 13 | 14 | NOAA's National Weather Service 15 | 16 | 1:54 PM Local 17 | 18 | http://weather.gov/ 19 | 20 | 63.0 F (17.2 C) 21 | 22 | Northwest at 16.1mph 23 | 24 | KBWI 25 | 26 | Baltimore 27 | 28 | No known delays for this airport. 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /Week9/assignment/xml/DFW: -------------------------------------------------------------------------------- 1 | 2 | false 3 | 4 | DFW 5 | 6 | Texas 7 | 8 | Dallas/Ft Worth International 9 | 10 | 10.00 11 | 12 | A Few Clouds 13 | 14 | NOAA's National Weather Service 15 | 16 | 12:53 PM Local 17 | 18 | http://weather.gov/ 19 | 20 | 81.0 F (27.2 C) 21 | 22 | Northeast at 11.5mph 23 | 24 | KDFW 25 | 26 | Dallas-Ft Worth 27 | 28 | No known delays for this airport. 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /Week9/assignment/xml/IAH: -------------------------------------------------------------------------------- 1 | 2 | false 3 | 4 | IAH 5 | 6 | Texas 7 | 8 | George Bush Intercontinental/Houston 9 | 10 | 10.00 11 | 12 | A Few Clouds 13 | 14 | NOAA's National Weather Service 15 | 16 | 12:53 PM Local 17 | 18 | http://weather.gov/ 19 | 20 | 86.0 F (30.0 C) 21 | 22 | Northeast at 10.4mph 23 | 24 | KIAH 25 | 26 | Houston 27 | 28 | No known delays for this airport. 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /Week9/assignment/xml/PHL: -------------------------------------------------------------------------------- 1 | 2 | false 3 | 4 | PHL 5 | 6 | Pennsylvania 7 | 8 | Philadelphia International 9 | 10 | 10.00 11 | 12 | Partly Cloudy 13 | 14 | NOAA's National Weather Service 15 | 16 | 1:54 PM Local 17 | 18 | http://weather.gov/ 19 | 20 | 65.0 F (18.3 C) 21 | 22 | Northwest at 15.0mph 23 | 24 | KPHL 25 | 26 | Philadelphia 27 | 28 | No known delays for this airport. 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /Week9/assignment/xml/PHX: -------------------------------------------------------------------------------- 1 | 2 | false 3 | 4 | PHX 5 | 6 | Arizona 7 | 8 | Phoenix Sky Harbor International 9 | 10 | 10.00 11 | 12 | Mostly Cloudy 13 | 14 | NOAA's National Weather Service 15 | 16 | 10:51 AM Local 17 | 18 | http://weather.gov/ 19 | 20 | 91.0 F (32.8 C) 21 | 22 | Southeast at 12.7mph 23 | 24 | KPHX 25 | 26 | Phoenix 27 | 28 | No known delays for this airport. 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /Week14/assignment/xml/ATL: -------------------------------------------------------------------------------- 1 | 2 | false 3 | 4 | ATL 5 | 6 | Georgia 7 | 8 | Hartsfield-Jackson Atlanta International 9 | 10 | 10.00 11 | 12 | A Few Clouds 13 | 14 | NOAA's National Weather Service 15 | 16 | 1:52 PM Local 17 | 18 | http://weather.gov/ 19 | 20 | 75.0 F (23.9 C) 21 | 22 | Northwest at 13.8mph 23 | 24 | KATL 25 | 26 | Atlanta 27 | 28 | No known delays for this airport. 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /Week14/assignment/xml/BWI: -------------------------------------------------------------------------------- 1 | 2 | false 3 | 4 | BWI 5 | 6 | Maryland 7 | 8 | Baltimore-Washington International 9 | 10 | 10.00 11 | 12 | Mostly Cloudy 13 | 14 | NOAA's National Weather Service 15 | 16 | 1:54 PM Local 17 | 18 | http://weather.gov/ 19 | 20 | 63.0 F (17.2 C) 21 | 22 | Northwest at 16.1mph 23 | 24 | KBWI 25 | 26 | Baltimore 27 | 28 | No known delays for this airport. 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /Week14/assignment/xml/CLT: -------------------------------------------------------------------------------- 1 | 2 | false 3 | 4 | CLT 5 | 6 | North Carolina 7 | 8 | Charlotte Douglas International 9 | 10 | 10.00 11 | 12 | Mostly Cloudy 13 | 14 | NOAA's National Weather Service 15 | 16 | 1:52 PM Local 17 | 18 | http://weather.gov/ 19 | 20 | 74.0 F (23.3 C) 21 | 22 | Northeast at 8.1mph 23 | 24 | KCLT 25 | 26 | Charlotte 27 | 28 | No known delays for this airport. 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /Week9/assignment/xml/ATL: -------------------------------------------------------------------------------- 1 | 2 | false 3 | 4 | ATL 5 | 6 | Georgia 7 | 8 | Hartsfield-Jackson Atlanta International 9 | 10 | 10.00 11 | 12 | A Few Clouds 13 | 14 | NOAA's National Weather Service 15 | 16 | 1:52 PM Local 17 | 18 | http://weather.gov/ 19 | 20 | 75.0 F (23.9 C) 21 | 22 | Northwest at 13.8mph 23 | 24 | KATL 25 | 26 | Atlanta 27 | 28 | No known delays for this airport. 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /Week9/assignment/xml/BOS: -------------------------------------------------------------------------------- 1 | 2 | false 3 | 4 | BOS 5 | 6 | Massachusetts 7 | 8 | General Edward Lawrence Logan International 9 | 10 | 10.00 11 | 12 | A Few Clouds 13 | 14 | NOAA's National Weather Service 15 | 16 | 1:54 PM Local 17 | 18 | http://weather.gov/ 19 | 20 | 61.0 F (16.1 C) 21 | 22 | West at 18.4mph 23 | 24 | KBOS 25 | 26 | Boston 27 | 28 | No known delays for this airport. 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /Week9/assignment/xml/CLT: -------------------------------------------------------------------------------- 1 | 2 | false 3 | 4 | CLT 5 | 6 | North Carolina 7 | 8 | Charlotte Douglas International 9 | 10 | 10.00 11 | 12 | Mostly Cloudy 13 | 14 | NOAA's National Weather Service 15 | 16 | 1:52 PM Local 17 | 18 | http://weather.gov/ 19 | 20 | 74.0 F (23.3 C) 21 | 22 | Northeast at 8.1mph 23 | 24 | KCLT 25 | 26 | Charlotte 27 | 28 | No known delays for this airport. 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /Week14/assignment/xml/BOS: -------------------------------------------------------------------------------- 1 | 2 | false 3 | 4 | BOS 5 | 6 | Massachusetts 7 | 8 | General Edward Lawrence Logan International 9 | 10 | 10.00 11 | 12 | A Few Clouds 13 | 14 | NOAA's National Weather Service 15 | 16 | 1:54 PM Local 17 | 18 | http://weather.gov/ 19 | 20 | 61.0 F (16.1 C) 21 | 22 | West at 18.4mph 23 | 24 | KBOS 25 | 26 | Boston 27 | 28 | No known delays for this airport. 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /Week14/assignment/xml/ORD: -------------------------------------------------------------------------------- 1 | 2 | true 3 | 4 | ORD 5 | 6 | Illinois 7 | 8 | Chicago OHare International 9 | 10 | 10.00 11 | 12 | Partly Cloudy 13 | 14 | NOAA's National Weather Service 15 | 16 | 12:51 PM Local 17 | 18 | http://weather.gov/ 19 | 20 | 53.0 F (11.7 C) 21 | 22 | Northwest at 12.7mph 23 | 24 | KORD 25 | 26 | Chicago 27 | 28 | VOL:Multi-taxi 29 | 30 | 31 | 32 | 33 | 34 | 16 minutes 35 | 36 | 37 | 38 | 30 minutes 39 | 40 | 41 | 42 | Increasing 43 | 44 | Departure 45 | -------------------------------------------------------------------------------- /Week9/assignment/xml/ORD: -------------------------------------------------------------------------------- 1 | 2 | true 3 | 4 | ORD 5 | 6 | Illinois 7 | 8 | Chicago OHare International 9 | 10 | 10.00 11 | 12 | Partly Cloudy 13 | 14 | NOAA's National Weather Service 15 | 16 | 12:51 PM Local 17 | 18 | http://weather.gov/ 19 | 20 | 53.0 F (11.7 C) 21 | 22 | Northwest at 12.7mph 23 | 24 | KORD 25 | 26 | Chicago 27 | 28 | VOL:Multi-taxi 29 | 30 | 31 | 32 | 33 | 34 | 16 minutes 35 | 36 | 37 | 38 | 30 minutes 39 | 40 | 41 | 42 | Increasing 43 | 44 | Departure 45 | -------------------------------------------------------------------------------- /Week9/lesson1.md: -------------------------------------------------------------------------------- 1 | # Week 9 Lesson 1 # 2 | ## Text Data Formats ## 3 | 4 | In this lesson, you will learn about the two different kinds of text files by writing the airport data to both a delimiter separated file and a fixed-width format file. 5 | 6 | ###Objectives ### 7 | By the end of this lesson, you will be able to: 8 | 9 | - Understand how to read and write a delimiter separated file. 10 | - Understand how to read and write a fixed width format file. 11 | 12 | 13 | ### Time Estimate ### 14 | 15 | Approximately 2 hours. 16 | 17 | ### Readings #### 18 | 19 | - Course [IPython Notebook](notebooks/text-dataformat.ipynb) on Data Formats 20 | - The [Delimiter](https://en.wikipedia.org/wiki/Delimiter-separated_values) Separated Value format 21 | - The [Comma](https://en.wikipedia.org/wiki/Comma-separated_values) Separated Value format 22 | 23 | #### *Optional Additional Readings*#### 24 | 25 | - The [W3C Working group](http://www.w3.org/2013/csvw/wiki/Main_Page) document repository for CSV. 26 | 27 | ### Assessment ### 28 | 29 | When you have completed and worked through the above readings, please take the [Week 9 Lesson 1 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095575). 30 | -------------------------------------------------------------------------------- /Week14/assignment/xml/MSP: -------------------------------------------------------------------------------- 1 | 2 | false 3 | 4 | MSP 5 | 6 | Minnesota 7 | 8 | Minneapolis-St Paul International/Wold-Chamberlain 9 | 10 | 10.00 11 | 12 | A Few Clouds 13 | 14 | NOAA's National Weather Service 15 | 16 | 12:53 PM Local 17 | 18 | http://weather.gov/ 19 | 20 | 44.0 F (6.7 C) 21 | 22 | Northwest at 17.3mph 23 | 24 | KMSP 25 | 26 | Minneapolis 27 | 28 | No known delays for this airport. 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /Week9/assignment/xml/MSP: -------------------------------------------------------------------------------- 1 | 2 | false 3 | 4 | MSP 5 | 6 | Minnesota 7 | 8 | Minneapolis-St Paul International/Wold-Chamberlain 9 | 10 | 10.00 11 | 12 | A Few Clouds 13 | 14 | NOAA's National Weather Service 15 | 16 | 12:53 PM Local 17 | 18 | http://weather.gov/ 19 | 20 | 44.0 F (6.7 C) 21 | 22 | Northwest at 17.3mph 23 | 24 | KMSP 25 | 26 | Minneapolis 27 | 28 | No known delays for this airport. 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /Week7/lesson3.md: -------------------------------------------------------------------------------- 1 | # Week 7 Lesson 3 # 2 | ## Python: Data Plotting ## 3 | 4 | In this lesson, you will learn to make and interpret scatter plots by 5 | using Matplotlib. Scatter plots can highlight trends or correlations 6 | between two or more columns of a given data set. 7 | 8 | 9 | ###Objectives ### 10 | By the end of this lesson, you will be able to: 11 | 12 | - Understand how to use Matplotlib to make a scatter plot. 13 | - Understand the difference between positive, negative, and null correlations. 14 | - Understand how to graphically identify outliers. 15 | - Understand how to overplot and label multiple columns on a single scatter plot. 16 | 17 | ### Time Estimate ### 18 | 19 | Approximately 2 hours. 20 | 21 | ### Readings #### 22 | 23 | - IPython Notebook on [Scatter Plots](notebooks/info490w7l3.ipynb) 24 | - Wikipedia article on [Scatter Plots](https://en.wikipedia.org/wiki/Scatter_plot) 25 | 26 | #### *Optional Additional Readings*#### 27 | 28 | - Wikipedia entry on [Anscombe's quartet](https://en.wikipedia.org/wiki/Anscombe%27s_quartet) 29 | 30 | ### Assessment ### 31 | 32 | When you have completed and worked through the above readings, please take the [Week 7 Lesson 3 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095557). 33 | -------------------------------------------------------------------------------- /Week6/lesson2.md: -------------------------------------------------------------------------------- 1 | # Week 6 Lesson 2 # 2 | ## Python Regular Expressions ## 3 | 4 | In this lesson, you will learn about Python regular expressions and how to use the ```re``` library in Python. 5 | 6 | ###Objectives ### 7 | By the end of this lesson, you will be able to: 8 | 9 | - Understand how to use the ```sub, search, and compile``` functions within the re library. 10 | - Understand how to structure verbose regular expressions. 11 | - Understand how to effectively specify the position and structure of the string you are searching for by using ```\b, ^, $, \d, \D, ?, *, +, {n,m}, (a|b)```, etc. 12 | 13 | ### Time Estimate ### 14 | 15 | Approximately 2 hours. 16 | 17 | ### Readings #### 18 | 19 | - Dive into Python3 [Regular Expressions (Chapter 5)](http://www.diveintopython3.net/regular-expressions.html). 20 | 21 | #### *Optional Additional Readings*#### 22 | 23 | - Google Developer [Python Regular 24 | Expressions](https://developers.google.com/edu/python/regular- 25 | expressions) (Note this was written for Python2, so be aware of the 26 | differences). 27 | 28 | 29 | ### Assessment ### 30 | 31 | When you have completed and worked through the above readings, please take the [Week 6 Lesson 2 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095542). 32 | -------------------------------------------------------------------------------- /Week15/lesson1.md: -------------------------------------------------------------------------------- 1 | # Week 15 Lesson 1 # 2 | ## Introduction to Python HPC ## 3 | 4 | In this lesson, you will learn about high performance computing by using 5 | the Python language. Specifically, you will learn about Python methods 6 | for threading, process control, the IPython server's parallel processing 7 | capabilities, and finally, third party tools that can accelerate your 8 | analysis code, including commercial libraries like Numba, and the open 9 | source mpi4py. 10 | 11 | ###Objectives ### 12 | By the end of this lesson, you will be able to: 13 | 14 | -Understand the basic concepts behind optimizing Python programs. 15 | - Understand how the Python Threading library works. 16 | - Understand how the Python Multiprocessing library works. 17 | - Understand how the IPYthon cluster model works. 18 | 19 | ### Time Estimate ### 20 | 21 | Approximately 2 hours. 22 | 23 | ### Readings #### 24 | 25 | - Course Python HPC [Notebook](notebook/pyhpc.ipynb) 26 | 27 | 28 | #### *Optional Additional Readings*#### 29 | 30 | - MPI for Python [documentation](https://mpi4py.readthedocs.org/en/latest/) 31 | 32 | ### Assessment ### 33 | 34 | When you have completed and worked through the above readings, please take the [Week 15 Lesson 1 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095647). 35 | -------------------------------------------------------------------------------- /Week3/assignment/README.md: -------------------------------------------------------------------------------- 1 | # Week 3 Assignment 2 | 3 | Submit your completed assignment (**.ipynb** files) onto Moodle for peer assessment. 4 | 5 | ## Submission deadline: Saturday, September 12th, 2015, 6:00 PM 6 | 7 | ## Problem 3.1. See template [get\_informatics.ipynb](get_informatics.ipynb) 8 | 9 | ## Problem 3.2. See template: [decision\_tree.ipynb](decision_tree.ipynb). 10 | 11 | ### How to download the templates 12 | 13 | You can download the template by updating your local course repository that you created in [Week 1 Lesson 2](https://github.com/UI-DataScience/info490-fa15/blob/master/Week1/lesson2.md), e.g. `/home/data_scientist/info490`. Open a terminal, either by using _New_ -> _Terminal_ in the IPython/Jupyter notebooks server, or by using the interactive terminal mode in Docker: 14 | 15 | ```shell 16 | $ docker exec -it /bin/bash 17 | ``` 18 | 19 | You can find the container name by doing `docker ps`. At the Unix shell, go to the directory where the course repository is mounted, 20 | 21 | ```shell 22 | $ cd /home/data_scientist/info490 23 | ``` 24 | 25 | and use `git pull` to update the local repository: 26 | 27 | ```shell 28 | $ git pull 29 | ``` 30 | 31 | Now, if you navigate to `/home/data_scientist/info490/Week2/assignment`, you will discover that the `git pull` command has updated the contents of the local repository. 32 | -------------------------------------------------------------------------------- /Week1/README.md: -------------------------------------------------------------------------------- 1 | #Week 1: Introduction to Unix# 2 | ### Objectives ### 3 | 4 | ![Unix Image](images/UNIX-Licence-Plate.JPG) 5 | 6 | #####By the end of this lesson, you should be able to:###### 7 | 8 | - Know the basics about the origin of, and the disciplines that contribute to Data Science. 9 | - Understand the concept of virtualization and be able to run the course Docker container 10 | - Understand the basic Unix file system and how to create, copy, move, and delete files and directories. 11 | 12 | ### Activities and Assignments ### 13 | 14 | |Activities and Assignments | Time Estimate | Deadline* | Points| 15 | |:------| -----|-------|----------:| 16 | |**[Week 1 Introduction Video](https://mediaspace.illinois.edu/media/t/0_j4lwy6kr/33195071)** | 10 Minutes | Tuesday |20| 17 | |**[Week 1 Lesson 1: Intro to Data Science](lesson1.md)**| 2 Hours |Thursday| 20| 18 | |**[Week 1 Lesson 2: Virtualization/Docker](lesson2.md)**| 2 Hours | Thursday | 20 | 19 | |**[Week 1 Lesson 3: The Unix Shell/Jupyter Notebook Server](lesson3.md)**| 2 Hours | Thursday| 20 | 20 | |**Week 1 Quiz**| 30 Minutes | Friday | 40| 21 | 22 | *Please note that unless otherwise noted, the due time is 6pm Central time! 23 | 24 | ---------- 25 | 26 | Photo Credit: [Unix License Plate](https://commons.wikimedia.org/wiki/File%3AUNIX-Licence-Plate.JPG) By KHanger, 10 June 2009. Wikipedia. 30 July 2015. 27 | -------------------------------------------------------------------------------- /Week5/lesson3.md: -------------------------------------------------------------------------------- 1 | # Week 5 Lesson 3 # 2 | ## Python Network programming ## 3 | 4 | In this lesson, you will learn about using Python's Requests library, which allows users to interact with the network. In addition, you will also learn about what sockets are, what they are made of, and how a server and a client deal with sockets. 5 | 6 | ###Objectives ### 7 | By the end of this lesson, you will be able to: 8 | 9 | - Know the basic steps for that the server and client undergo they are using sockets 10 | - Understand the anatomy and definition of a socket. 11 | - Use basic Requests library functions. 12 | 13 | 14 | ### Time Estimate ### 15 | 16 | Approximately 2 hours. 17 | 18 | ### Readings #### 19 | 20 | - [Network Programming with Python](http://courses.cs.washington.edu/courses/cse142/12au/explorations/network%20programming/NetworkProgrammingWithPython.pdf) lecture slides. 21 | - [Requests](http://docs.python-requests.org/en/latest/user/quickstart/) Library documentation (ignore JSON sections). 22 | 23 | #### *Optional Additional Readings*#### 24 | 25 | - [Socket Programming](http://ilab.cs.byu.edu/python/) in Python (page navigation is on right side). 26 | 27 | 28 | ### Assessment ### 29 | 30 | When you have completed and worked through the above readings, please take the [Week 5 Lesson 3 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095530). 31 | -------------------------------------------------------------------------------- /Week6/lesson3.md: -------------------------------------------------------------------------------- 1 | # Week 6 Lesson 3 # 2 | ## Unix Text Processing ## 3 | 4 | In this lesson, you will learn about text processing within Unix. This will involve first learning about various way to sort through and get rid of redundant lines of text. Next, we will progress on to learning different ways to remove and merge lines of text. Then, we will find ways to compare and join files, as well as apply patches to a file. Finally, we will cover the functionality of ```tr, sed, and aspell```. 5 | 6 | ###Objectives ### 7 | By the end of this lesson, you will be able to: 8 | 9 | - Understand how to sort through lines of text and report differences of files through the use of ```sort and uniq, comm, and diff```. 10 | - Understand how to modify lines of text and files using ```cut, paste, join, and patch```. 11 | - Understand how to use ```tr, sed, and aspell``` to transform and check through lines of text. 12 | 13 | ### Time Estimate ### 14 | 15 | Approximately 2 hours. 16 | 17 | ### Readings #### 18 | 19 | - Chapter 20 of [The Linux Command Line(PDF)](http://sourceforge.net/projects/linuxcommand/?source=dlp) and try to follow allong with the examples 20 | 21 | #### *Optional Additional Readings*#### 22 | 23 | 24 | ### Assessment ### 25 | 26 | When you have completed and worked through the above readings, please take the [Week 6 Lesson 3 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095545). 27 | -------------------------------------------------------------------------------- /Week14/lesson1.md: -------------------------------------------------------------------------------- 1 | # Week 14 Lesson 1 # 2 | ## Pandas and Databases ## 3 | 4 | In this lesson, you will learn how to work with a SQL based database by 5 | using the Pandas Python library. Pandas simplifies many data tasks by 6 | hiding details that are not relevant for many projects. In this case, 7 | Pandas can hide many of the database interaction mechanisms to allow one 8 | to easily query, update, or insert new data into an existing database. 9 | 10 | ###Objectives ### 11 | By the end of this lesson, you will be able to: 12 | 13 | - Understand the basic concepts involved in using a Pandas DataFrame to interact with a database. 14 | - Understand how to query data by using Pandas. 15 | - Understand how to update data by using Pandas. 16 | - Understand how to use Pandas to create and populate new database tables. 17 | 18 | ### Time Estimate ### 19 | 20 | Approximately 1 hours. 21 | 22 | ### Readings #### 23 | 24 | - Course [IPython Notebook](notebook/intro2pandasdb.ipynb), which you can view online or (better yet) download and run locally. 25 | 26 | #### *Optional Additional Readings*#### 27 | 28 | - [Blog Post](http://pandas.pydata.org/pandas-docs/stable/comparison_with_sql.html) about Pandas and SQL 29 | 30 | ### Assessment ### 31 | 32 | When you have completed and worked through the above readings, please take the [Week 14 Lesson 1 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095635). 33 | -------------------------------------------------------------------------------- /Week5/lesson2.md: -------------------------------------------------------------------------------- 1 | # Week 5 Lesson 2 # 2 | ## Working with the Underlying File System ## 3 | 4 | In this lesson, you will learn how to write Python programs that run at 5 | a command prompt and you will learn to work with files from within a 6 | Python program. This includes opening and closing files, and reading and 7 | writing data to files. You also will learn about streams and character 8 | encoding and working with compressed files. Finally, you will learn 9 | about working with directories and file metadata. 10 | 11 | ###Objectives ### 12 | 13 | By the end of this lesson, you will be able to: 14 | 15 | - Write and execute a complete Python program. 16 | - Open and close a file from a Python program. 17 | - Read and write data from within a Python program. 18 | - Work with the file system by using Python. 19 | 20 | ### Time Estimate ### 21 | 22 | Approximately 2 hours. 23 | 24 | ### Readings #### 25 | 26 | - [Python File Input/Output Notebook](notebooks/pyfileio.ipynb) 27 | - Chapters 1 and 11 from [Dive Into Python](http://www.diveintopython3.net/index.html) 28 | 29 | #### *Optional Additional Readings*#### 30 | 31 | - Chapter 14 [Think Python](http://faculty.stedwards.edu/mikek/python/thinkpython.pdf) 32 | 33 | ### Assessment ### 34 | 35 | When you have completed and worked through the above readings, please take the [Week 5 Lesson 2 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095527). 36 | -------------------------------------------------------------------------------- /Week14/lesson3.md: -------------------------------------------------------------------------------- 1 | # Week 14 Lesson 3 # 2 | ## Working with Data ## 3 | 4 | In this lesson, you will review a worked example that demonstrates how 5 | to work with different data sets to make an interesting data 6 | visualization known as a chloropleth. To do this we need to access web 7 | resources, parse an XML-based data format, extract meaningful data from 8 | a second web accessible resource, and combine it all into a new and 9 | interesting visualization. 10 | 11 | ###Objectives ### 12 | By the end of this lesson, you will be able to: 13 | 14 | - Understand how to download and access web resources. 15 | - Understand how to use Pandas to access online data files. 16 | - Understand how to extract data from XML-based resources. 17 | - Understand how to create and display a Chloropeth visualization. 18 | 19 | ### Time Estimate ### 20 | 21 | Approximately 2 hours. 22 | 23 | ### Readings #### 24 | 25 | - Course Data [Chloropeth Visualization](notebook/dataviz.ipynb) Notebook 26 | - Python [Requests](http://docs.python-requests.org/en/latest/) 27 | 28 | #### *Optional Additional Readings*#### 29 | 30 | - Choosing [Chloropeth](http://colorbrewer2.org/) colors 31 | - [Bureau of Labor Statistics](http://www.bls.gov/lau/#tables) 32 | 33 | ### Assessment ### 34 | 35 | When you have completed and worked through the above readings, please take the [Week 14 Lesson 3 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095641). 36 | -------------------------------------------------------------------------------- /Week4/lesson1.md: -------------------------------------------------------------------------------- 1 | # Week 4 Lesson 1 # 2 | ## Unix: Working with Data ## 3 | 4 | In this lesson, you will first learn about learn about how to modify and view environment variables within Unix. Then you will go through some of the ins and outs of the vi text editor within Unix. 5 | 6 | ###Objectives ### 7 | 8 | By the end of this lesson, you will: 9 | 10 | - Be able to modify and learn about certain environment aspects in shell, with commands such as ```printenv, set, export, and alias```. 11 | - Know the difference between environment and shell variables. 12 | - Know how the environment is established. 13 | - Know the two basic categories of text editors. 14 | - Know how to open/create files in a text editor. 15 | - Know the basics of Vi commands. 16 | 17 | ### Time Estimate ### 18 | 19 | Approximately 3 hours. 20 | 21 | ### Readings #### 22 | 23 | - Chapters 11-12 in [The Linux Command Line](http://sourceforge.net/projects/linuxcommand/?source=dlp). 24 | - [The Unix Data Processing IPython Notebook](notebooks/unixdp.ipynb) 25 | 26 | #### *Optional Additional Readings*#### 27 | 28 | - [Learning the vi and Vim Editors, 7th Edition](http://proquest.safaribooksonline.com.proxy2.library.illinois.edu/book/programming/vi/9780596529833) 29 | 30 | ### Assessment ### 31 | 32 | When you have completed and worked through the above readings, please take the [Week 4 Lesson 1 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095503). 33 | -------------------------------------------------------------------------------- /Week8/lesson3.md: -------------------------------------------------------------------------------- 1 | # Week 8 Lesson 3 # 2 | ## Introduction to Pandas: Data Manipulation and Analysis## 3 | 4 | In this lesson, you will learn more advanced functionality within the 5 | Pandas library. Specifically you will learn about combining data from 6 | different Pandas DataFrames, efficiently selecting data from Pandas 7 | DataFrames, and how to use Pandas to clean data before subsequent 8 | analysis. 9 | 10 | 11 | ###Objectives ### 12 | By the end of this lesson, you will be able to: 13 | 14 | - Understand how to quickly filter data in a Pandas DataFrame 15 | - Understand how use the Pandas GroupBy and Aggregate functionality. 16 | - Understand how combine data from different Pandas data structures 17 | - Understand how to manipulate character data by using Pandas 18 | - Understand how to clean up data sources by using Pandas 19 | 20 | 21 | ### Time Estimate ### 22 | 23 | Approximately 2 hours. 24 | 25 | ### Readings #### 26 | 27 | - [Pandas Tutorial](https://github.com/jvns/pandas-cookbook) Chapters 3-7 28 | 29 | #### *Optional Additional Readings*#### 30 | 31 | - Another [Pandas Tutorial](http://nbviewer.ipython.org/github/jvns/talks/blob/master/pydatanyc2013/PyData%20NYC%202013%20tutorial.ipynb) (but do not pollute the namespace with **%pylab inline**) 32 | 33 | ### Assessment ### 34 | 35 | When you have completed and worked through the above readings, please take the [Week 8 Lesson 3 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095569). 36 | 37 | -------------------------------------------------------------------------------- /Week1/lesson1.md: -------------------------------------------------------------------------------- 1 | # Week 1 Lesson 1 # 2 | ## Intro to Data Science ## 3 | 4 | In this lesson you will learn about the history of Data Science and how Data Science, by nature, is a combination of many different disciplines. 5 | 6 | ###Objectives ### 7 | 8 | By the end of this lesson, you will: 9 | 10 | - Know the basics about the origin, development, and popularization about Data Science. 11 | - Know which disciplines contribute to Data Science 12 | - Understand how different disciplines aquire, process, and understand data. 13 | 14 | You could share with the class via a Moodle Discussion Forum post how 15 | your discipline views _Data Science_. 16 | 17 | ### Time Estimate ### 18 | 19 | Approximately 2 hours. 20 | 21 | ### Readings #### 22 | For both of these two readings, you should stop when you reach the _assignment/Exercise_ section. 23 | - Read the [Wikipedia article on A History of Data Science](https://en.wikibooks.org/wiki/Data_Science:_An_Introduction/A_History_of_Data_Science). 24 | - Read the [Wikipedia article on A Mash-up of Disciplines in Data Science](https://en.wikibooks.org/wiki/Data_Science:_An_Introduction/A_Mash-up_of_Disciplines). 25 | 26 | #### *Optional Additional Readings*#### 27 | - [Definitions of Data](https://en.wikibooks.org/wiki/Data_Science:_An_Introduction/Definitions_of_Data) 28 | 29 | ### Assessment ### 30 | 31 | When you have completed and worked through the above readings, please take the [Week 1 Lesson 1 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095467). 32 | 33 | 34 | 35 | -------------------------------------------------------------------------------- /Week6/lesson1.md: -------------------------------------------------------------------------------- 1 | # Week 6 Lesson 1 # 2 | ## Unix: Regular Expressions & Commands ## 3 | 4 | In this lesson, you will learn about regular expressions within a Unix-like environment. 5 | 6 | ###Objectives ### 7 | By the end of this lesson, you will be able to: 8 | 9 | - Understand and be able to utilize regular expressions to better streamline searches through text. 10 | - Utilize grep options. 11 | - Utilize metacharacters and literal characters for grep searches. 12 | 13 | ### Time Estimate ### 14 | 15 | Approximately 2 hours. 16 | 17 | ### Readings #### 18 | 19 | - Chapter 19 of [The Linux Command Line(PDF)](http://sourceforge.net/projects/linuxcommand/?source=dlp) and try to follow allong with the examples 20 | 21 | #### *Optional Additional Readings*#### 22 | 23 | - Chapter 6 of [Unix and Linux: Visual Quickstart Guide, Fifth Edition](http://proquest.safaribooksonline.com.proxy2.library.illinois.edu/book/operating-systems-and-server-administration/unix/9780133793871) 24 | - Chapter 8 of [Beginning Unix](http://proquest.safaribooksonline.com.proxy2.library.illinois.edu/book/operating-systems-and-server-administration/unix/9780764579943) 25 | - Chapter 5 of [Think Unix](http://proquest.safaribooksonline.com.proxy2.library.illinois.edu/book/operating-systems-and-server-administration/unix/078972376x) 26 | 27 | 28 | ### Assessment ### 29 | 30 | When you have completed and worked through the above readings, please take the [Week 6 Lesson 1 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095539). 31 | -------------------------------------------------------------------------------- /Week7/lesson1.md: -------------------------------------------------------------------------------- 1 | # Week 7 Lesson 1 # 2 | ## Introduction to Data Visualization ## 3 | 4 | In this lesson, you will learn about the art of making great 5 | visualization. While this might seem out of place, since we haven't 6 | actually shown how to make any visualizations, I want to start this week 7 | by introducing you to the ideas and results of those considered masters 8 | in the art of information visualization. 9 | 10 | 11 | ###Objectives ### 12 | By the end of this lesson, you will be able to: 13 | 14 | - Understand the power of visualizations in conveying information. 15 | - Understand the importance of aesthetics in communicating data. 16 | - Understand the perils of presentation slides to communicate technical information. 17 | 18 | ### Time Estimate ### 19 | 20 | Approximately 2 hours. 21 | 22 | ### Readings #### 23 | 24 | - The Beauty of [Data Visualizations](http://www.ted.com/talks/david_mccandless_the_beauty_of_data_visualization) 25 | 26 | - On the [Perils of Powerpoint](http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB) 27 | 28 | - [Visualizing](http://www.ted.com/talks/hans_rosling_shows_the_best_stats_you_ve_ever_seen) Statistical Information 29 | 30 | #### *Optional Additional Readings*#### 31 | 32 | - The many wonderful articles at [Flowing Data](http://flowingdata.com) 33 | 34 | 35 | ### Assessment ### 36 | 37 | When you have completed and worked through the above readings, please take the [Week 7 Lesson 1 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095551). 38 | -------------------------------------------------------------------------------- /Week1/lesson3.md: -------------------------------------------------------------------------------- 1 | # Week 1 Lesson 3 # 2 | ## The Unix Shell/JupyterHub ## 3 | 4 | In this lesson you will begin working at the Unix prompt in order to 5 | understand the bash shell, the Unix file system, and how to work with 6 | files and processes. 7 | 8 | ### Objectives ### 9 | By the end of this lesson, you will be able to: 10 | 11 | - Understand how to work at the Unix command prompt. 12 | - Understand the basic Unix file system. 13 | - Be able to create, copy, move, and delete files and directories. 14 | 15 | ### Time Estimate### 16 | 17 | Approximately 2 hours. 18 | 19 | ### Readings ### 20 | 21 | - Read chapters 1-4 from the free book [The Linux Command Line 22 | (PDF)](http://sourceforge.net/projects/linuxcommand/?source=dlp), Second 23 | Internet Edition, by William E. Shotts, Jr. 24 | - Follow along by entering the commands as directed into a terminal 25 | window within your Docker container. 26 | - Explore the INFO490 Course [Introduction to Unix](notebooks/introduction2unix.ipynb) Notebook 27 | 28 | #### *Optional Additional Readings* #### 29 | - [Guide to Unix/Why Unix-like](https://en.wikibooks.org/wiki/Guide_to_Unix/Why_Unix-like) 30 | - [Unix Tutorial for Beginners](http://www.ee.surrey.ac.uk/Teaching/Unix/) 31 | - [Introduction to Linux: A Hands-on Guide](http://www.tldp.org/LDP/intro-linux/html/index.html) 32 | 33 | ### Assessment ### 34 | 35 | When you have completed and worked through the above readings, please 36 | take the [Week 1 Lesson 3 37 | Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095470). 38 | -------------------------------------------------------------------------------- /Week10/lesson1.md: -------------------------------------------------------------------------------- 1 | # Week 10 Lesson 1 # 2 | 3 | ## Summary Statistics ## 4 | 5 | In this lesson, you will learn about summary statistics like the mean, 6 | median, and mode. You also will learn about representing a set of data 7 | as a distribution and how the summary statistics are related to a 8 | distribution. 9 | 10 | ###Objectives ### 11 | By the end of this lesson, you will be able to: 12 | 13 | - Understand the different summary statistics. 14 | - Understand how to compute the mean or average value for a data set. 15 | - Understand the concept of a Probability Density Function (PDF). 16 | - Understand the relationship between summary statistics like the mean and a distribution function. 17 | 18 | ### Time Estimate ### 19 | 20 | Approximately 2 hours. 21 | 22 | ### Readings #### 23 | 24 | - Chapter 2, Distributions, from [Think Stats2](http://www.greenteapress.com/thinkstats2/html/thinkstats2003.html) 25 | - Section on Important presentation of Probability Densities in [Introduction to Statistics](http://work.thaslwanter.at/Stats/html/statsDistributions.html#other-important-presentations-of-probability-densities). Stop at the *Distribution Functions* Header. 26 | 27 | #### *Optional Additional Readings*#### 28 | 29 | - Chapter 1, Exploratory Data Analysis, from [Think Stats2](http://www.greenteapress.com/thinkstats2/html/thinkstats2002.html) 30 | 31 | ### Assessment ### 32 | 33 | When you have completed and worked through the above readings, please take the [Week 10 Lesson 1 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095587). 34 | -------------------------------------------------------------------------------- /Week2/assignment/README.md: -------------------------------------------------------------------------------- 1 | # Week 2 Assignment 2 | 3 | **Submission deadline: September 5th, 2015, 6:00 PM** 4 | 5 | 6 | ## Problem 1. Survey. 7 | 8 | Complete this [survey](https://docs.google.com/forms/d/1gH-6Fz4-oynqSJkgkUX3rhAnri4aCRdiMTcHCiP6S3g/viewform). 9 | 10 | ## Problem 2. Values and Types. 11 | 12 | Use the IPython/Jupyter notebook template: [`types.ipynb`](types.ipynb). 13 | 14 | You can download the template by updating your local course repository that you created in [Week 1 Lesson 2](https://github.com/UI-DataScience/info490-fa15/blob/master/Week1/lesson2.md), e.g. `/home/data_scientist/info490`. Open a terminal, either by using _New_ -> _Terminal_ in the IPython/Jupyter notebooks server, or by using the interactive terminal mode in Docker: 15 | 16 | ```shell 17 | $ docker exec -it /bin/bash 18 | ``` 19 | 20 | You can find the container name by doing `docker ps`. At the Unix shell, go to the directory where the course repository is mounted, 21 | 22 | ```shell 23 | $ cd /home/data_scientist/info490 24 | ``` 25 | 26 | and use `git pull` to update the local repository: 27 | 28 | ```shell 29 | $ git pull 30 | ``` 31 | 32 | Now, if you navigate to `/home/data_scientist/info490/Week2/assignment`, you will discover that the `git pull` command has updated the contents of the local repository. 33 | 34 | Alternatively, you can directly download the [template file](https://raw.githubusercontent.com/UI-DataScience/info490-fa15/master/Week2/assignment/types.ipynb) and import it into your notebook server. 35 | 36 | Submit your completed assignment onto Moodle for peer assessment. 37 | -------------------------------------------------------------------------------- /Week2/lesson1.md: -------------------------------------------------------------------------------- 1 | # Week 2 Lesson 1 # 2 | ## Basic Unix Concepts ## 3 | 4 | In this lesson, you will learn more details about how Unix-like operating systems operate. 5 | 6 | ### Objectives### 7 | By the end of this lesson, you will: 8 | 9 | - Understand how to utilize various Unix commands. 10 | - Know the difference between the four different types of commands. 11 | - Be able to find a command's documentation. 12 | - Understand the input/output redirection process and commands for Unix. 13 | - Understand some of the inner-workings of the shell, such as expansion and quoting. 14 | - Get to know some of the more advanced keyboard tricks, such as command line editing, completion, and using history. 15 | 16 | ### Time Estimate ### 17 | Approximately 2 hours. 18 | 19 | ### Readings ### 20 | 21 | - Read chapters 5-8 from the free book [The Linux Command Line (PDF)](http://sourceforge.net/projects/linuxcommand/?source=dlp) and follow along by entering the commands as directed into a terminal window within your virtual machine. 22 | 23 | #### *Optional Additional Readings* #### 24 | 25 | - Chapter 6 of [Beginning Unix](http://proquest.safaribooksonline.com.proxy2.library.illinois.edu/book/operating-systems-and-server-administration/unix/9780764579943/6-unix-commands-in-depth/13_chap06_html#X2ludGVybmFsX0h0bWxWaWV3P3htbGlkPTk3ODA3NjQ1Nzk5NDMlMkYxM19jaGFwMDZfaHRtbCZxdWVyeT0=) 26 | 27 | 28 | ### Assessment ### 29 | 30 | When you have completed and worked through the above readings, please take the [Week 2 Lesson 1 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095479). 31 | 32 | -------------------------------------------------------------------------------- /Week5/lesson1.md: -------------------------------------------------------------------------------- 1 | # Week 5 Lesson 1 # 2 | ## Unix: Networking and Basic Commands ## 3 | 4 | In this lesson, you will learn the basics of networking commands, 5 | searching for files, and archiving within Unix-like environments. 6 | 7 | ###Objectives ### 8 | By the end of this lesson, you will: 9 | 10 | - Understand and be able to use simple networking commands, such as ```ping, netstat, ftp, wget,``` etc. 11 | - Be able to explain what the following terms mean: IP address, host and domain name, and URI. 12 | - Understand how to search for files using locate and find, and also learn other commands associated with finding files: ```xargs, touch, and stat```. 13 | - Understand how to utilize the archiving commands for Unix-like environments: ```gzip, bzip2, tar, zip, rsync,``` etc. 14 | 15 | ### Time Estimate ### 16 | 17 | Approximately 2 hours. 18 | 19 | ### Readings #### 20 | 21 | - [Unix Networking Notebook](notebooks/unixnetwork.ipynb) 22 | - Chapters 16-18 from [The Linux Command Line (PDF)](http://sourceforge.net/projects/linuxcommand/?source=dlp) and follow along by entering the commands as directed into a terminal window within your virtual machine. 23 | 24 | #### *Optional Additional Readings*#### 25 | 26 | - Chapter 7, 16, and 25 of [Practical Unix](http://proquest.safaribooksonline.com.proxy2.library.illinois.edu/book/operating-systems-and-server-administration/unix/078972250x) 27 | 28 | ### Assessment ### 29 | 30 | When you have completed and worked through the above readings, please take the [Week 5 Lesson 1 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095524). 31 | -------------------------------------------------------------------------------- /Week8/lesson2.md: -------------------------------------------------------------------------------- 1 | # Week 8 Lesson 2 # 2 | ## Introduction to Pandas: Data Access and Selection## 3 | 4 | In this lesson, you will learn about the Pandas library. Pandas is a 5 | Python library that can simplify data analysis tasks. If you know R, you 6 | will find many familiar concepts in Pandas. By using Pandas, you can 7 | more easily manipulate data by using column labels, you can identify and 8 | handle missing data, and you can quickly merge or join data sets. 9 | 10 | ###Objectives ### 11 | By the end of this lesson, you will be able to: 12 | 13 | - Understand the basic components in the Pandas library 14 | - Understand how to use a DataFrame 15 | - Understand how to read and write data between Pandas data structures and different types of files 16 | - Understand how to manipulate large data sets by column labels 17 | 18 | ### Time Estimate ### 19 | 20 | Approximately 2 hours. 21 | 22 | ### Readings #### 23 | 24 | - [Introduction to Pandas](notebooks/intro2pandas.ipynb) IPython Notebook 25 | - [Quick overview](http://pandas.pydata.org/pandas-docs/stable/overview.html) of Pandas 26 | - Pandas in [Ten Minutes](http://pandas.pydata.org/pandas-docs/stable/10min.html) 27 | - [Pandas Tutorial](https://github.com/jvns/pandas-cookbook) Chapters 1-2 28 | 29 | #### *Optional Additional Readings*#### 30 | 31 | - Another [Pandas Tutorial](http://www.gregreda.com/2013/10/26/intro-to-pandas-data-structures/) 32 | 33 | ### Assessment ### 34 | 35 | When you have completed and worked through the above readings, please take the [Week 8 Lesson 2 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095566). 36 | -------------------------------------------------------------------------------- /Week15/README.md: -------------------------------------------------------------------------------- 1 | #Week 15 Overview# 2 | 3 | ![Blue Waters Supercomputer](images/bluewatersimage.jpg) 4 | ## Introduction to High Performance Computing with Python ## 5 | 6 | In this week, you will have only one lesson, which introduces high 7 | performance computing with Python, including an introduction to the 8 | Python threading and multiprocessing libraries. We also will cover the 9 | IPython cluster model, as well as review of several commercial and 10 | open-source libraries such as the mpi4py library. 11 | 12 | ### Objectives ### 13 | 14 | #####By the end of this lesson, you should accomplish the following learning objectives:###### 15 | 16 | - Understand the benefits of high performance computing 17 | - Understand how the Python threading library can be used in a Python program. 18 | - Understand how the Python multiprocessing library can be used in a Python program. 19 | - Understand how the iPython cluster model can be used in a Python program. 20 | 21 | ### Activities and Assignments ### 22 | 23 | |Activities and Assignments | Time Estimate | Deadline* | Points| 24 | |:------| -----|-------|----------:| 25 | |**[Week 15 Introduction Video][w15v]**|10 Minutes|Monday |20| 26 | |**[Week 15 Lesson 1: Python: Introduction to HPC](lesson1.md)**| 2 Hours |Tuesday| 20| 27 | 28 | *Please note that unless otherwise noted, the due time is 6pm Central time!* 29 | 30 | ---------- 31 | [w15v]: https://mediaspace.illinois.edu/media/Week+Fifteen/0_sbxk771y/33195071 32 | 33 | Photo Credit: Blue Waters Supercomputer. From http://gladiator.ncsa.illinois.edu/Images/bluewaters/timeline/bw_front.jpg. Accessed 3 August 2015. 34 | -------------------------------------------------------------------------------- /Week4/lesson2.md: -------------------------------------------------------------------------------- 1 | # Week 4 Lesson 2 # 2 | ## Python: Strings & Lists ## 3 | 4 | In this lesson, you will learn about the difference between strings and lists, how to perform operations on these objects, and how to call various elements inside of these objects. 5 | 6 | ###Objectives ### 7 | 8 | By the end of this lesson, you will be able to: 9 | 10 | - Understand what strings are. 11 | - Manipulate strings within Python using various operators, such as ```in```, ```for``` loops and the ```if``` comparison operator. 12 | - Understand what lists are, and how their output is organized. 13 | - Understand how to call one element of a list 14 | - Manipulate lists using list operators. 15 | 16 | ### Time Estimate ### 17 | 18 | Approximately 2 hours. 19 | 20 | ### Readings #### 21 | 22 | - Chapter 8-10 [Think Python](http://faculty.stedwards.edu/mikek/python/thinkpython.pdf) 23 | - Python [Data Structures I](notebooks/pydatastructures.ipynb) Notebook. 24 | 25 | ## Optional Readings ## 26 | 27 | - Chapters 2.4 and 4 from [Dive Into Python](http://www.diveintopython3.net/index.html) 28 | - Exercises 6, 32, 34, and 38 in [Learn Python the Hard Way](http://proquest.safaribooksonline.com.proxy2.library.illinois.edu/book/programming/python/9780133124316) 29 | - Chapters 4 and 6 in [Automate the Boring Stuff with Python](http://proquest.safaribooksonline.com.proxy2.library.illinois.edu/book/programming/python/9781457189906) 30 | 31 | 32 | ### Assessment ### 33 | 34 | When you have completed and worked through the above readings, please take the [Week 4 Lesson 2 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095506). 35 | -------------------------------------------------------------------------------- /Week12/lesson1.md: -------------------------------------------------------------------------------- 1 | # Week 12 Lesson 1 # 2 | ## Relational Database ## 3 | 4 | In this lesson, you will start learning about data persistence 5 | techniques by using Python. First, you will review using basic file I/O 6 | to save Python data. Next, you will learn about Pickling, a Python 7 | technique for saving and restoring data by using a custom Python 8 | format. Third, you will learn about relational databases, including the 9 | basic relational database concepts such as database roles and the ACID 10 | test. Finally you will learn about the SQLite database that we will use 11 | to demonstrate relational database concept in this course. 12 | 13 | ###Objectives ### 14 | By the end of this lesson, you will be able to: 15 | 16 | - Understand the concept of pickling data. 17 | - Understand relational database technology 18 | - Understand database roles. 19 | - Understand the ACID test. 20 | - Understand the basic concepts behind the Sqlite database 21 | 22 | ### Time Estimate ### 23 | 24 | Approximately 2 hours. 25 | 26 | ### Readings #### 27 | 28 | - Introduction to [Data Persistence Notebook](notebook/intro2db.ipynb) 29 | 30 | #### *Optional Additional Readings*#### 31 | 32 | - [Database and SQL Tutorial](http://www.tutorialspoint.com/sql/index.htm): From the SQL-Home page through the SQL-Databases page. 33 | - SQLite Tutorial](http://www.tutorialspoint.com/sqlite/index.htm): From the SQLite-Home page through the SQLite-Commands page 34 | 35 | ### Assessment ### 36 | 37 | When you have completed and worked through the above readings, please take the [Week 12 Lesson 1 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095611). 38 | -------------------------------------------------------------------------------- /Week4/lesson3.md: -------------------------------------------------------------------------------- 1 | # Week 4 Lesson 3 # 2 | ## Python: Tuples & Dictionaries ## 3 | 4 | In this lesson, you will first learn about what dictionaries are, how to 5 | modify them, and what operations you can apply to them. Next, you will 6 | learn about tuples, how they differ from lists, and how to manipulate 7 | them with functions and operations. 8 | 9 | 10 | ###Objectives ### 11 | 12 | By the end of this lesson, you will be able to: 13 | 14 | - Know the difference between lists and dictionaries. 15 | - Understand the key-value concept of dictionaries. 16 | - Create your own dictionaries. 17 | - Understand what a tuple is and how it is different from a list. 18 | - Apply various functions and operators to a tuple. 19 | 20 | ### Time Estimate ### 21 | 22 | Approximately 3 hours. 23 | 24 | ### Readings #### 25 | 26 | - Chapters 11-13 in [Think Python](http://faculty.stedwards.edu/mikek/python/thinkpython.pdf). 27 | - Python [Data Structures II](notebooks/pydatastructures2.ipynb) Notebook. 28 | 29 | ####*Optional Additional Readings*#### 30 | 31 | - Chapters 2.5 and 2.7 in [Dive Into Python](http://www.diveintopython3.net/index.html) 32 | - Exercise 39 in [Learn Python the Hard Way](http://proquest.safaribooksonline.com.proxy2.library.illinois.edu/book/programming/python/9780133124316) 33 | - Chapters 4 and 5 in [Automate the Boring Stuff with Python](http://proquest.safaribooksonline.com.proxy2.library.illinois.edu/book/programming/python/9781457189906) 34 | 35 | ### Assessment ### 36 | 37 | When you have completed and worked through the above readings, please take the [Week 4 Lesson 3 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095509). 38 | -------------------------------------------------------------------------------- /Week7/lesson2.md: -------------------------------------------------------------------------------- 1 | # Week 7 Lesson 2 # 2 | ## Python: Plotting ## 3 | 4 | In this lesson, you will learn to work with Matplotlib, which is the 5 | standard plotting library for Python programs. You will use an iPython 6 | notebook to explore the basic Matplotlib Python statements to make and 7 | annotate a simple plot. You also will be introduced to the Seaborn 8 | plotting library, which enhances the power of MatPlotlib by simplifying 9 | the creation of aesthetically attractive visualizations and introducing 10 | new plot types. 11 | 12 | ###Objectives ### 13 | 14 | By the end of this lesson, you will be able to: 15 | 16 | - Understand how to use Matplotlib within an iPython notebook to make a simple, inline plot. 17 | - Understand how to use Matplotlib to label the plot axes and to apply a title to a plot 18 | - Understand how to use Matplotlib to change the plot limits to increase the impact of a plot. 19 | - Understand how to use Seaborn within an IPython Notebook. 20 | 21 | ### Time Estimate ### 22 | 23 | Approximately 2 hours. 24 | 25 | ### Readings #### 26 | 27 | - IPython Notebook on [Python Plotting](notebooks/info490w7l2.ipynb) 28 | 29 | #### *Optional Additional Readings*#### 30 | 31 | - Wikipedia article on [Plots][plt] which has a lot of information about different plot types. 32 | - [Matplotlib Tutorial](http://matplotlib.org/users/pyplot_tutorial.html) 33 | 34 | 35 | ### Assessment ### 36 | 37 | When you have completed and worked through the above readings, please take the [Week 7 Lesson 2 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095554). 38 | 39 | ----- 40 | [plt]: https://en.wikipedia.org/wiki/Plot_(graphics) 41 | -------------------------------------------------------------------------------- /Week3/lesson1.md: -------------------------------------------------------------------------------- 1 | # Week 3 Lesson 1 # 2 | ## Unix File System & Processes ## 3 | 4 | In this lesson, you will learn more details about how Unix-like systems deal with user permissions and how to diagnose, control, and possibly terminate unresponsive programs. 5 | 6 | ### Objectives### 7 | By the end of this lesson, you will: 8 | 9 | - Understand how the permissions system imparts system security. 10 | - Know how to utilize various permissions-based commands, such as ``` id, umask, sudo,``` etc. 11 | - Understand how to use various process-based commands, such as ```ps, top, bg, kill,``` etc. 12 | 13 | 14 | ### Time Estimate ### 15 | Approximately 2 hours. 16 | 17 | ### Readings ### 18 | 19 | - Read chapters 9-10 from the free book [The Linux Command Line (PDF)](http://sourceforge.net/projects/linuxcommand/?source=dlp) and follow along by entering the commands as directed into a terminal window within your virtual machine. 20 | 21 | #### *Optional Additional Readings* #### 22 | - Wikipedia article covering [malware](http://en.wikipedia.org/wiki/Malware) 23 | - Chapters 2.16 and 2.17 of [How Linux Works](http://proquest.safaribooksonline.com.proxy2.library.illinois.edu/book/programming/linux/9781457185519), which covers permissions and processes. 24 | - Chapter 2.7 of [The Linux Programming Interface](http://proquest.safaribooksonline.com.proxy2.library.illinois.edu/book/programming/linux/9781593272203/2dot-fundamental-concepts/id812523) which covers processes. 25 | 26 | 27 | ### Assessment ### 28 | 29 | When you have completed and worked through the above readings, please take the [Week 3 Lesson 1 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095491). 30 | -------------------------------------------------------------------------------- /LICENSE.md: -------------------------------------------------------------------------------- 1 | Copyright (c) 2009-2015 University of Illinois 2 | All rights reserved. 3 | 4 | Developed by: Robert J. Brunner 5 | Edward J. Kim 6 | University of Illinois 7 | http://lcdm.astro.illinois.edu 8 | 9 | Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal with the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: 10 | 11 | Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimers. 12 | Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimers in the documentation and/or other materials provided with the distribution. 13 | Neither the names of Robert J. Brunner, Edward J. Kim, or the University of Illinois, nor the names of its contributors may be used to endorse or promote products derived from this Software without specific prior written permission. 14 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE CONTRIBUTORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS WITH THE SOFTWARE. 15 | -------------------------------------------------------------------------------- /Week13/lesson1.md: -------------------------------------------------------------------------------- 1 | # Week 13 Lesson 1 # 2 | ## Introduction to Object Oriented Programming ## 3 | 4 | In this lesson, you will learn about object oriented programming (OOP), 5 | a paradigm in which new data types are created by using classes and 6 | access to the data in these new data types can be controlled by using 7 | methods. This enables a programmer to encapsulate the state of a system 8 | to minimize unintentional side effects, and to (potentially) improve 9 | performance. Existing classes can be extended by using inheritance, and 10 | class hierarchies can be dynamically created to support polymorphism. 11 | 12 | ###Objectives ### 13 | By the end of this lesson, you will be able to: 14 | 15 | - Understand the the fundamental concepts behind object oriented programming. 16 | - Understand how OOP can be used to create new data types. 17 | - Understand the concepts of encapsulation, polymorphism, and inheritance. 18 | 19 | ### Time Estimate ### 20 | 21 | Approximately 1 hour. 22 | 23 | ### Readings #### 24 | 25 | - [What is an Object?](http://docs.oracle.com/javase/tutorial/java/concepts/object.html) from Oracle Corp. 26 | - [What is OOP?](http://java.about.com/od/objectorientedprogramming/a/introobjects.htm) 27 | - [Python OOP 28 | Tutorial](http://www.python-course.eu/python3_object_oriented_programming.php), stop at the Methods heading 29 | (you can continue beyond this in Lessons 2 & 3). 30 | 31 | #### *Optional Additional Readings*#### 32 | 33 | - [Wikipedia article on OOP](https://en.wikipedia.org/wiki/Object-oriented_programming) 34 | 35 | ### Assessment ### 36 | 37 | When you have completed and worked through the above readings, please take the [Week 13 Lesson 1 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095623). 38 | -------------------------------------------------------------------------------- /Week3/lesson3.md: -------------------------------------------------------------------------------- 1 | # Week 3 Lesson 3 # 2 | ## Python: Conditional Statements & Iteration ## 3 | 4 | In this lesson, you will focus on the details of writing and using 5 | functions in a Python program (we will stay within the IPython notebook, 6 | primarily, although you can experiment with writing and executing Python 7 | programs in a file). First, you will learn about conditional 8 | programming. Next, you will learn how to write a function that returns 9 | one or more values. Finally, you will learn about iteration and 10 | algorithm design. 11 | 12 | ###Objectives ### 13 | 14 | By the end of this lesson, you will be able to: 15 | 16 | - Write and use conditional statements in Python. 17 | - Write Python functions that return data. 18 | - Write and use iteration statements in Python. 19 | 20 | ### Time Estimate ### 21 | 22 | Approximately 2 hours. 23 | 24 | ### Readings #### 25 | 26 | - Chapters 5-7 in [Think Python](http://faculty.stedwards.edu/mikek/python/thinkpython.pdf). 27 | - Python [Flow Control](notebooks/flowcontrolpy.ipynb) Notebook. 28 | 29 | ## Optional Readings ## 30 | 31 | - Chapter 5 of [Python Essentials](http://proquest.safaribooksonline.com.proxy2.library.illinois.edu/book/programming/python/9781784390341/python-essentials/ch05_html#X2ludGVybmFsX0h0bWxWaWV3P3htbGlkPTk3ODE3ODQzOTAzNDElMkZjaDA1X2h0bWwmcXVlcnk9Y29uZGl0aW9uYWwlMjBzdGF0ZW1lbnRz), which deals with Logic, Comparisons and Conditions. 32 | - Chapter 2 of [Automate the Boring Stuff with Python](http://proquest.safaribooksonline.com.proxy2.library.illinois.edu/book/programming/python/9781457189906) 33 | 34 | ### Assessment ### 35 | 36 | When you have completed and worked through the above readings, please take the [Week 3 Lesson 3 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095497). 37 | -------------------------------------------------------------------------------- /Week10/lesson3.md: -------------------------------------------------------------------------------- 1 | # Week 10 Lesson 3 # 2 | ## Visualizing distributions in Python ## 3 | 4 | In this lesson, you will learn to make and interpret more advanced data 5 | visualizations like histograms by using MatPlotLib. Next you will learn 6 | about improving the appearance of data visualizations to better convey 7 | information, which we can easily do by using the Seaborn library. 8 | Finally, you will learn how to use Seaborn to make visually more 9 | appealing plots, how to use Kernel Density Estimation to improve 10 | histograms, and how to make box plots and violin plots. 11 | 12 | ###Objectives ### 13 | 14 | By the end of this lesson, you will be able to: 15 | 16 | - Understand how to use Matplotlib to make and use Histograms. 17 | - Understand the importance of Beautiful Visualizations. 18 | - Understand how to use Seaborn to improve the appearance of a plot. 19 | - Understand how to make and use Box and Violin plots. 20 | - Understand how to use KDE to improve a Box plot. 21 | 22 | ### Time Estimate ### 23 | 24 | Approximately 2 hours. 25 | 26 | ### Readings #### 27 | 28 | - Introduction to [Distribution visualizations](notebook/intro2dataviz.ipynb) 29 | - Using [Seaborn to plot distributions](http://stanford.edu/~mwaskom/software/seaborn/tutorial/distributions.html) 30 | - Demonstration of [how to improve a visualization](http://darkhorseanalytics.com/blog/data-looks-better-naked/) 31 | 32 | #### *Optional Additional Readings*#### 33 | 34 | - Data [visualization in Python](http://work.thaslwanter.at/Stats/html/statsBasics.html#data-display) 35 | - Impressive [Visualizations](http://setosa.io/#/) 36 | 37 | 38 | ### Assessment ### 39 | 40 | When you have completed and worked through the above readings, please take the [Week 10 Lesson 3 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095593). 41 | -------------------------------------------------------------------------------- /Week1/lesson2.md: -------------------------------------------------------------------------------- 1 | # Week 1 Lesson 2 # 2 | ## Virtualization and Docker ## 3 | 4 | In this lesson you will learn about virtualization, and specifically the 5 | Docker technology. Note that you may have already completed some of 6 | these tasks during the Orientation Week. 7 | 8 | ###Objectives ### 9 | 10 | By the end of this lesson, you will be able to: 11 | 12 | - Understand the concept of virtualization 13 | - Install Docker 14 | - Install and run the course Docker container 15 | 16 | ### Time Estimate ### 17 | 18 | Approximately 2 hours. 19 | 20 | ### Readings #### 21 | - Read the [Wikipedia article on Virtualization](https://en.wikipedia.org/wiki/Virtualization). 22 | - Follow the _Get Started with Docker_ documentation for your particular computer type: 23 | - [Mac OSX](http://docs.docker.com/mac/started/) 24 | - [Windows](http://docs.docker.com/windows/started/) 25 | - [Linux](http://docs.docker.com/linux/started/) 26 | 27 | You can stop once you have completed _Find and run the whalesay image_. 28 | 29 | - Explore the course notes on _Working with Docker_ for your particular computer type: 30 | - [Mac OS X](Working-with-Docker-OSX.md) 31 | - [Windows](Working-with-Docker-Win.md) 32 | - Linux: Everything in the [Mac OS X](Working-with-Docker-OSX.md) documentation should work 33 | in Linux. 34 | 35 | #### *Optional Additional Readings*#### 36 | 37 | - [An introduction to the 38 | Containers](http://googlecloudplatform.blogspot.com/2015/01/in-coming- 39 | weeks-we-will-be-publishing.html) from the Google Cloud Computing Blog. 40 | - Complete any remaining steps in the _Get Started with Docker_ documentation. 41 | 42 | ### Assessment ### 43 | 44 | When you have completed and worked through the above readings, please 45 | take the [Week 1 Lesson 2 46 | Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095473). 47 | 48 | 49 | 50 | -------------------------------------------------------------------------------- /Week14/lesson2.md: -------------------------------------------------------------------------------- 1 | # Week 14 Lesson 2 # 2 | ## Data Parsing## 3 | 4 | In this lesson, you will learn about extracting information from 5 | structured data sets. This includes parsing data from XML formats such 6 | as HTML, which is the language in which web pages are written and 7 | stored. To do this you will learn about the BeautifulSoup parsing 8 | library and the libxml parsing engine. You also will review the basics 9 | of regular expressions, which can speed up the extraction of specific 10 | data from XML formatted files. 11 | 12 | ###Objectives ### 13 | By the end of this lesson, you will be able to: 14 | 15 | - Understand how to use a data parsing library like BeautifulSoup. 16 | - Understand how to find and extract information from an XML format file 17 | - Understand how to extract data from a webpage. 18 | - Understand the document object model 19 | 20 | 21 | ### Time Estimate ### 22 | 23 | Approximately 2 hours. 24 | 25 | ### Readings #### 26 | 27 | - Course IPython Notebook on [Data Parsing](notebook/intro2dp.ipynb) 28 | - [BeautifulSoup](http://www.crummy.com/software/BeautifulSoup/bs4/doc/) documentation 29 | - [Scrapy](http://scrapy.org), a new web scraping framework in Python 30 | 31 | #### *Optional Additional Readings*#### 32 | 33 | - A course [primer notebook](../Week8/notebooks/intro2pandas.ipynb) on Pandas 34 | - A [web scraping](http://nbviewer.ipython.org/url/www.unc.edu/~ncaren/Lax-1.ipynb.json) in Python tutorial 35 | - Another [web scraping](http://nbviewer.ipython.org/urls/dl.dropboxusercontent.com/u/33663928/dst4l-projects/week5/Web_Scraping_Tutorial-TotalFilm_50_Adaptations.ipynb) in Python tutorial 36 | 37 | ### Assessment ### 38 | 39 | When you have completed and worked through the above readings, please take the [Week 14 Lesson 2 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095638). 40 | -------------------------------------------------------------------------------- /Week8/lesson1.md: -------------------------------------------------------------------------------- 1 | # Week 8 Lesson 1 # 2 | ## Introduction to Numpy ## 3 | 4 | In this lesson, you will learn to work with the Numpy library, which 5 | brings fast numerical arrays and matrices to the Python programming 6 | language. You will learn how to create numnpy arrays, how to quantify 7 | the basic details about an existing array, how to index and slice these 8 | arrays, and how to apply (universal) functions to array data quickly. 9 | Finally, you will learn how to change the shape of an array, and how to 10 | make shallow and deep copies of an array. 11 | 12 | ###Objectives ### 13 | 14 | By the end of this lesson, you will be able to: 15 | 16 | - Understand how to create and use a Numpy array. 17 | - Understand how to index into and slice a Numpy array. 18 | - Understand what a ufunc is and how to apply them to a Numpy array. 19 | - Understand how to modify and copy a Numpy array. 20 | 21 | ### Time Estimate ### 22 | 23 | Approximately 3 hours. 24 | 25 | ### Readings #### 26 | 27 | - [Introduction to Numpy](notebooks/intro2numpy.ipynb) IPython Notebook 28 | 29 | - [Numpy Tutorial](http://wiki.scipy.org/Tentative_NumPy_Tutorial) (Chapters 1-4 only) 30 | or backup link to [Numpy Tutorial](http://www.cs.man.ac.uk/~barry/mydocs/MyCOMP28512/MS15_Notes/PyRefs/Tentative_NumPy_Tutorial.pdf) if the main site is down. 31 | 32 | 33 | #### *Optional Additional Readings*#### 34 | 35 | - Numpy [Cookbook](http://wiki.scipy.org/Cookbook#head-198bd222c6438301ada793ad63e0b5384ab10308) 36 | - Short [Numpy Overview](http://nbviewer.ipython.org/github/WeatherGod/AnatomyOfMatplotlib/blob/master/AnatomyOfMatplotlib-Part0-Intro2NumPy.ipynb) 37 | 38 | 39 | ### Assessment ### 40 | 41 | When you have completed and worked through the above readings, please take the [Week 8 Lesson 1 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095563). 42 | -------------------------------------------------------------------------------- /Week2/lesson3.md: -------------------------------------------------------------------------------- 1 | # Week 2 Lesson 3 # 2 | ## Introduction to Python ## 3 | 4 | In this lesson, you will learn about the basics of python terminology, and the difference between various kinds of errors you will encounter through coding in Python. Next, you will explore the various data types found in Python and learn how to apply simple operations (such as addition, subtraction, etc.) to Python variables. 5 | 6 | 7 | ### Objectives### 8 | By the end of this lesson, you will: 9 | 10 | - Know basic python terminology, such as input, output, debugging, etc. 11 | - Know the difference between the different types of errors. 12 | - Use the Python interpreter. 13 | - Describe the basic data types provided by the Python language. 14 | - Perform basic operations on Python variables. 15 | 16 | 17 | 18 | ### Time Estimate ### 19 | Approximately 2 hours. 20 | 21 | ### Readings ### 22 | 23 | - [Introduction to the Python Programming Language](notebooks/intro2py.ipynb) 24 | - Read Chapters 1-2 in [Think Python](http://faculty.stedwards.edu/mikek/python/thinkpython.pdf). 25 | - Try out Python commands and code snippets in a new IPython Notebook. 26 | 27 | #### *Optional Additional Readings* #### 28 | - Several free books, mostly written for Python 2: 29 | - [Open Tech School's Introduction to Programming with Python](http://opentechschool.github.io/python-beginners/en/index.html) 30 | - [Invent with Python](http://inventwithpython.com/) 31 | - [Building Skills in Programming](http://www.itmaybeahack.com/homepage/books/) 32 | - [Learn Python The Hard Way, 3rd Edition](http://learnpythonthehardway.org/book/) 33 | - [A Byte of Python](http://www.ibiblio.org/g2swap/byteofpython/read/) 34 | 35 | 36 | ### Assessment ### 37 | 38 | When you have completed and worked through the above readings, please take the [Week 2 Lesson 3 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095485). 39 | -------------------------------------------------------------------------------- /Week9/README.md: -------------------------------------------------------------------------------- 1 | #Week 9 Overview# 2 | 3 | ![Data Processing Comic](images/BBQ.gif) 4 | ## Introduction to Data Formats ## 5 | 6 | In this week, you will learn about different data formats. First we will discuss the text data format, which will include the .csv and .txt files. Next, we will discuss JSON files. Finally, we will discuss XML files and how to read and write them. 7 | 8 | ### Objectives ### 9 | 10 | #####By the end of this lesson, you should accomplish the following learning objectives:###### 11 | 12 | - Understand how to read and write and XML formatted file. 13 | - Understand how to read and write a CSV file. 14 | - Understand how to read and write a JSON file. 15 | 16 | ### Activities and Assignments ### 17 | 18 | |Activities and Assignments | Time Estimate | Deadline* | Points| 19 | |:------| -----|-------|----------:| 20 | |**[Week 9 Introduction Video][w9v]**|10 Minutes|Tuesday|NA| 21 | |**[Week 9 Lesson 1: Data Format: Text](lesson1.md)**| 2 Hours |Thursday| 20| 22 | |**[Week 9 Lesson 2: Data Format: JSON](lesson2.md)**| 2 Hours | Thursday | 20 | 23 | |**[Week 9 Lesson 3: Data Format: XML](lesson3.md)**| 2 Hours | Thursday| 70 | 24 | |**[Week 9 Quiz][w9q]**| 35 Minutes | Friday | 70| 25 | |**[Week 9 Assignment Submission][w9a]**| 3 Hours | Saturday | 60% of the grade from the Instructor, 40% of the grade from Peer grading | 26 | |**Week 9 Completion of Peer Review**| 3 Hours | *The following* Saturday | 50 | 27 | 28 | *Please note that unless otherwise noted, the due time is 6pm Central time! 29 | 30 | ---------- 31 | [w9v]: https://mediaspace.illinois.edu/media/Week+Nine/0_ty3bpjlk/33195071 32 | [w9a]: https://learn.illinois.edu/mod/workshop/view.php?id=1095305 33 | [w9q]: https://learn.illinois.edu/mod/quiz/view.php?id=1095308 34 | 35 | 36 | Photo Credit: Data Processing Center. By Harley Schwadron. From cartoonstock.com. Accessed 3 August 2015. 37 | 38 | -------------------------------------------------------------------------------- /Week13/README.md: -------------------------------------------------------------------------------- 1 | #Week 13 Overview# 2 | 3 | ## Object Oriented Programming (OOP) ## 4 | 5 | In this week, you will learn about object oriented programming and how 6 | to use this programming paradigm by using the Python programming 7 | language. Specifically, you will learn how to create new data types in 8 | Python by using a `class`, how to control access to the attributes of 9 | this new data type by using _methods_, and how to leverage existing 10 | objects through inheritance. 11 | 12 | ### Objectives ### 13 | 14 | #####By the end of this lesson, you should accomplish the following learning objectives:###### 15 | 16 | - Understand the basic principles behind object oriented programming. 17 | - Understand how to create and use classes in Python. 18 | - Understand how to control access to class data via methods. 19 | 20 | ### Activities and Assignments ### 21 | 22 | |Activities and Assignments | Time Estimate | Deadline* | Points| 23 | |:------| -----|-------|----------:| 24 | |**[Week 13 Introduction Video][w13v]**|10 Minutes|Tuesday|NA| 25 | |**[Week 13 Lesson 1: Object Oriented Programming](lesson1.md)**| 1 Hour |Thursday| 20| 26 | |**[Week 13 Lesson 2: Python: Introduction to OOP](lesson2.md)**| 3 Hours | Thursday | 20 | 27 | |**[Week 13 Lesson 3: Python: OOP](lesson3.md)**| 3 Hours | Thursday| 70 | 28 | |**[Week 13 Quiz][w13q]**| 35 Minutes | Friday | 70| 29 | |**[Week 13 Assignment Submission][w13a]**| 3 Hours | Saturday | 60% of the grade from the Instructor, 40% of the grade from Peer grading | 30 | |**Week 13 Completion of Peer Review**| 3 Hours | *The following* Saturday | 50 | 31 | 32 | *Please note that unless otherwise noted, the due time is 6pm Central time!* 33 | 34 | ---------- 35 | [w13v]: https://mediaspace.illinois.edu/media/Week+Thirteen/1_jcupo21k/33195071 36 | [w13a]: https://learn.illinois.edu/mod/workshop/view.php?id=1095404 37 | [w13q]: https://learn.illinois.edu/mod/quiz/view.php?id=1095401 38 | -------------------------------------------------------------------------------- /Week13/lesson2.md: -------------------------------------------------------------------------------- 1 | # Week 13 Lesson 2 # 2 | ## Introduction to Object Oriented Programming in Python ## 3 | 4 | In this lesson, you will learn about implementing object oriented 5 | programming concepts in Python. First, you will learn how to create a 6 | `class` in Python, how to add class documentation strings, how to add 7 | class attributes, and how to pass use class instances with general 8 | functions. Next, you will learn about integrating classes more fully 9 | within functions, how to support modification, and how to perform 10 | prototyping. 11 | 12 | 13 | ###Objectives ### 14 | By the end of this lesson, you will be able to: 15 | 16 | - Understand how to create a `class` in Python. 17 | - Understand how to initialize an object in Python 18 | - Understand how to use classes to define new data types 19 | - Understand ho to use these new data types within Python functions 20 | 21 | ### Time Estimate ### 22 | 23 | Approximately 3 hours. 24 | 25 | ### Readings #### 26 | 27 | - Chapter 15 from [Think Python2](http://www.greenteapress.com/thinkpython2/html/thinkpython2016.html) 28 | - Chapter 16 from [Think Python2](http://www.greenteapress.com/thinkpython2/html/thinkpython2017.html) 29 | 30 | #### *Optional Additional Readings*#### 31 | 32 | - [Improve your Python: OOP](https://www.jeffknupp.com/blog/2014/06/18/improve-your-python-python-classes-and-object-oriented-programming/) (Stopping at the Inheritance heading). 33 | - [OOP Section](http://anandology.com/python-practice-book/object_oriented_programming.html) in the Python Practice Book (up to section _4.3 Inheritance_). 34 | - Tutorialspoint [Python Object Oriented](http://www.tutorialspoint.com/python/python_classes_objects.htm) lesson (stop at _Class Inheritance_) 35 | 36 | ### Assessment ### 37 | 38 | When you have completed and worked through the above readings, please take the [Week 13 Lesson 2 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095626). 39 | -------------------------------------------------------------------------------- /Week12/lesson2.md: -------------------------------------------------------------------------------- 1 | # Week 12 Lesson 2 # 2 | ## Using SQL for Schema Manipulation ## 3 | 4 | In this lesson, you will learn how to create and delete database and 5 | database schema. First, you will earn about basic SQL Syntax, including 6 | the allowed SQL data types. While this will focus on the Sqlite3 syntax 7 | and data types, more general information will be conveyed. Next, you 8 | will learn how to create and delete databases. Next you will learn how 9 | to create database schema, including relational tables, and the columns 10 | that define the characteristics of those tables. This will also include 11 | a discussion of views, indices, and user defined functions. Finally, 12 | you will learn how to drop and alter existing tables. 13 | 14 | ###Objectives ### 15 | By the end of this lesson, you will be able to: 16 | 17 | - Understand how to create and delete databases in SQLite. 18 | - Understand how to create and delete relational database tables by using SQLite. 19 | - Understand the creation and maintenance of relational database schema. 20 | - Understand the difference between relational database tables and views. 21 | 22 | ### Time Estimate ### 23 | 24 | Approximately 2 hours. 25 | 26 | ### Readings #### 27 | 28 | - Introduction to [SQL DML Notebook](notebook/intro2sqlddl.ipynb) 29 | 30 | #### *Optional Additional Readings*#### 31 | 32 | - [Database and SQL Tutorial](http://www.tutorialspoint.com/sql/index.htm): From the SQL-Syntax page through the SQL-Drop Table page. 33 | - [SQLite Tutorial](http://www.tutorialspoint.com/sqlite/index.htm): From the SQLite-Syntax page through the SQLite-Drop Table page 34 | - [SQL Tutorial](http://www.w3schools.com/sql/) 35 | - [SQL Online Tutorial](http://sqlzoo.net/wiki/Main_Page) 36 | 37 | 38 | 39 | ### Assessment ### 40 | 41 | When you have completed and worked through the above readings, please take the [Week 12 Lesson 2 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095614). 42 | -------------------------------------------------------------------------------- /Week13/lesson3.md: -------------------------------------------------------------------------------- 1 | # Week 13 Lesson 3 # 2 | ## Object Oriented Programming in Python ## 3 | 4 | In this lesson, you will learn about more advanced object oriented 5 | programming concepts and how to use them effectively by using the Python 6 | programming language. This includes understanding the concept of classes 7 | as interfaces and specific implementations of an interface. The use of 8 | methods to encapsulate the state of an object, and finally the use of 9 | inheritance to build object oriented hierarchies. 10 | 11 | 12 | ###Objectives ### 13 | By the end of this lesson, you will be able to: 14 | 15 | - Understand the concept of class methods. 16 | - Understand how to define an interface. 17 | - Understand how to create a succesful implementation of an interface. 18 | - Understand how to use inheritance to construct object hierarchies. 19 | - Understand the benefits of using encapsulation. 20 | 21 | ### Time Estimate ### 22 | 23 | Approximately 3 hours. 24 | 25 | ### Readings #### 26 | 27 | - Chapter 17 from [Think Python2](http://www.greenteapress.com/thinkpython2/html/thinkpython2018.html) 28 | - Chapter 18 from [Think Python2](http://www.greenteapress.com/thinkpython2/html/thinkpython2019.html) 29 | 30 | #### *Optional Additional Readings*#### 31 | 32 | - [Improve your Python: OOP](https://www.jeffknupp.com/blog/2014/06/18/improve-your-python-python-classes-and-object-oriented-programming/) (starting from the Inheritance heading). 33 | - [OOP Section](http://anandology.com/python-practice-book/object_oriented_programming.html) in the Python Practice Book (from section _4.3 Inheritance_ onward). 34 | - Tutorialspoint [Python Object Oriented](http://www.tutorialspoint.com/python/python_classes_objects.htm) lesson (from _Class Inheritance_ onward) 35 | 36 | ### Assessment ### 37 | 38 | When you have completed and worked through the above readings, please take the [Week 13 Lesson 3 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095629). 39 | -------------------------------------------------------------------------------- /Week11/lesson2.md: -------------------------------------------------------------------------------- 1 | # Week 11 Lesson 2 # 2 | ## Functional Programming in Python ## 3 | 4 | In this lesson, you will learn how to apply functional programming 5 | within Python. First you will learn about employing the map, filter, and 6 | reduce methods. Second, you will learn about lambda functions, which are 7 | small, anonymous functions, that can simplify many functional 8 | programming tasks. Note that many of the readings use Python2, so keep 9 | that in mind when trying these concepts out in Python3 (for example, use 10 | the `print` function, not the `print` statement, and the `reduce` function 11 | is now in the `itertools` module). 12 | 13 | 14 | ###Objectives ### 15 | By the end of this lesson, you will be able to: 16 | 17 | - Understand the `map` function and how to use it effectively. 18 | - Understand the `filter` function and how to use it effectively. 19 | - Understand the `reduce` function and how to use it effectively. 20 | - Understand lambda functions and how to use them for functional programming. 21 | 22 | ### Time Estimate ### 23 | 24 | Approximately 2 hours. 25 | 26 | ### Readings #### 27 | 28 | - [Functional programming](http://maryrosecook.com/blog/post/a-practical-introduction-to-functional-programming) in Python2. 29 | 30 | - [Lambda functions](http://www.python-course.eu/python3_lambda.php) in Python. 31 | 32 | #### *Optional Additional Readings*#### 33 | 34 | - [Functional Programming in Python](http://alan-g.me.uk/tutor/tutfctnl.htm) article (don't worry yet 35 | about comprehensions). 36 | 37 | - Old article on Functional Programming in Python [Part 1](http://www.ibm.com/developerworks/linux/library/l-prog/index.html) 38 | and [Part 2](http://www.ibm.com/developerworks/linux/library/l-prog2/index.html). 39 | Note that are using Python2. 40 | 41 | ### Assessment ### 42 | 43 | When you have completed and worked through the above readings, please take the [Week 11 Lesson 2 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095602). 44 | -------------------------------------------------------------------------------- /Week11/README.md: -------------------------------------------------------------------------------- 1 | #Week 11 Overview# 2 | 3 | ## Functional Programming ## 4 | 5 | In this week, you will learn about functional programming. This 6 | programming paradigm employs functions to change the state of data, 7 | where the functions process input data and produce outputs, without 8 | changing the state of the rest of the program. Functional programming 9 | can result in high performance computations for certain types of 10 | problems, and can, therefore, be a useful skill to possess 11 | 12 | ### Objectives ### 13 | 14 | #####By the end of this lesson, you should accomplish the following learning objectives:###### 15 | 16 | - Understand the basic concepts behind functional programming. 17 | - Understand how to employ the `map`, `filter`, and `reduce` methods. 18 | - Understand how to write and use lambda functions. 19 | - Understand comprehensions and iterators. 20 | 21 | ### Activities and Assignments ### 22 | 23 | |Activities and Assignments | Time Estimate | Deadline* | Points| 24 | |:------| -----|-------|----------:| 25 | |**[Week 11 Introduction Video][w11v]**|10 Minutes|Tuesday|NA| 26 | |**[Week 11 Lesson 1: Functional Programming](lesson1.md)**| 2 Hours |Thursday| 20| 27 | |**[Week 11 Lesson 2: Functional Programming in Python, Part I](lesson2.md)**| 2 Hours | Thursday | 20 | 28 | |**[Week 11 Lesson 3: Functional Programming in Python, Part II](lesson3.md)**| 3 Hours | Thursday| 20 | 29 | |**[Week 11 Quiz][w11q]**| 35 Minutes | Friday | 70| 30 | |**[Week 11 Assignment Submission][w11a]**| 3 Hours | Saturday | 60% of the grade from the Instructor, 40% of the grade from Peer grading | 31 | |**Week 11 Completion of Peer Review**| 3 Hours | *The following* Saturday | 50 | 32 | 33 | *Please note that unless otherwise noted, the due time is 6pm Central time! 34 | 35 | ---------- 36 | [w11v]: https://mediaspace.illinois.edu/media/Week+Eleven/1_ofuqqrcu 37 | [w11a]: https://learn.illinois.edu/mod/workshop/view.php?id=1095362 38 | [w11q]: https://learn.illinois.edu/mod/quiz/view.php?id=1095359 39 | 40 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Welcome to INFO 490: Foundations of Data Science # 2 | 3 | ***Professor:*** Dr. Robert Brunner 4 | 5 | ***Teaching Assistants:*** 6 | 7 | 1. Avinash Balakrishnan 8 | 2. Edward Kim (Lead) 9 | 3. Sam Thrush 10 | 11 | This class is an asynchronous, online course. This course will build a practical foundation for data science 12 | by teaching students basic tools and techniques that can scale to large computational systems and massive data sets. 13 | 14 | Students will first learn how to work at a Unix command prompt before learning about source code control software 15 | like git and the github site. Next, the Python programming language will be covered, with a focus on specific aspects 16 | of the language and associated Python modules that are relevant for Data Science. Python will be introduced and used 17 | primarily via the IPython (or Jupyter) Notebooks, and will cover the Numpy, Scipy, MatPlotlib, Pandas, Seaborn, and 18 | scikit_learn Python modules. These capabilities will be demonstrated through simple data science tasks such as obtaining data, 19 | cleaning data, visualizing data, and basic data analysis. Students must have access to a fairly modern computer, ideally 20 | that supports hardware virtualization, on which they can install software. 21 | 22 | This class is open to sophomores, juniors, seniors and graduate students in any discipline. 23 | 24 | Please refer to the [course syllabus](orientation/syllabus.md) for more information about course content and grading policies. 25 | 26 | **If you have any questions, or if something is not working properly, *PLEASE* look through the FAQs wiki page (please look at the right tool bar on the Github course page and click the icon labeled "Wiki" that looks like an open book) and the _Moodle Q&A Forum_ before emailing TA or course instructor.** 27 | 28 | Click the link below to get live help on Gitter: 29 | 30 | [![Join the chat at https://gitter.im/UI-DataScience/info490-fa15](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/UI-DataScience/info490-fa15?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) 31 | -------------------------------------------------------------------------------- /Week11/lesson3.md: -------------------------------------------------------------------------------- 1 | # Week 11 Lesson 3 # 2 | ## Functional Programing in Python, Part II ## 3 | 4 | In this lesson, you will learn an alternative approach to functional 5 | programming in Python, that is specifically encouraged in Python3, 6 | iterators and comprehensions. First, you will learn about list 7 | comprehensions, which are an easy way to construct lists of items. 8 | Second, you will learn about iterators, which enable functional 9 | programming by simplifying the process of moving through items of data 10 | in a collection, like a list. Finally, you will learn moe about these 11 | constructs through a worked file system example. 12 | 13 | ###Objectives ### 14 | By the end of this lesson, you will be able to: 15 | 16 | - Understand the iterator concept and how they can be broadly applied. 17 | - Understand how to use a list comprehension to make a new list. 18 | - Understand how to use an if clause in a list comprehension. 19 | - Understand other comprehensions like dictionary or set. 20 | 21 | ### Time Estimate ### 22 | 23 | Approximately 3 hours. 24 | 25 | ### Readings #### 26 | 27 | - [Python Tutorial](https://docs.python.org/3/tutorial/datastructures.html#list-comprehensions) on list comprehensions 28 | 29 | - Python [HowTo article](https://docs.python.org/dev/howto/functional.html#iterators) on Functional Programming. 30 | Stop when you reach the _Generator_ subsection. 31 | 32 | - [Using List Comprehensions](http://getpython3.com/diveintopython3/comprehensions.html) 33 | 34 | #### *Optional Additional Readings*#### 35 | 36 | - [Python List Comprehension](http://www.python-course.eu/python3_list_comprehension.php) tutorial. 37 | 38 | - Python [HowTo article](https://docs.python.org/dev/howto/functional.html#generators) on Functional Programming. 39 | To master FP concepts in Python3, I would encourage you to read the rest of this HowTo. 40 | 41 | 42 | ### Assessment ### 43 | 44 | When you have completed and worked through the above readings, please take the [Week 11 Lesson 3 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095605). 45 | -------------------------------------------------------------------------------- /Week4/README.md: -------------------------------------------------------------------------------- 1 | #Week 4 Overview# 2 | ![python logo](../Week3/images/python-logo.png) 3 | ## Advanced Python Programming Language ## 4 | 5 | In this week, you will again be delving further into Unix and Python. First, you will learn how to view and modify parts of the environment in Unix-like environments, and then learn how to use vim and vi in Unix. Finally, you will then delve into the wondrous world of lists, dictionaries, strings, and tuples, where you will learn what these things are, when to use them, how they differ from each other, and how to manipulate them to our liking. 6 | 7 | ### Objectives ### 8 | 9 | #####By the end of this lesson, you should be able to:###### 10 | 11 | - Effectively use the Basic Python data structures: list, dictionary, string, & tuple. 12 | - Be able to modify and learn about certain environment aspects in shell, with commands such as printenv, set, export, and alias. 13 | - Know the basics of Vi commands. 14 | 15 | ### Activities and Assignments ### 16 | 17 | |Activities and Assignments | Time Estimate | Deadline* | Points| 18 | |:------| -----|-------|----------:| 19 | |**[Week 4 Introduction Video][w4v]**|10 Minutes|Tuesday|20| 20 | |**[Week 4 Lesson 1: Unix: Working with Data](lesson1.md)**| 3 Hours |Thursday| 20| 21 | |**[Week 4 Lesson 2: Python: Strings & Lists](lesson2.md)**| 2 Hours | Thursday | 20 | 22 | |**[Week 4 Lesson 3: Python: Tuples & Dictionaries](lesson3.md)**| 2 Hours | Thursday| 20 | 23 | |**[Week 4 Quiz][w4q]**| 30 Minutes | Friday | 70| 24 | |**[Week 4 Assignment Submission][w4a]**| 3 Hours | Saturday | 60% of the grade from the Instructor, 40% of the grade from Peer grading | 25 | |**Week 4 Completion of Peer Review**| 3 Hours | *The following* Saturday | 50 | 26 | 27 | *Please note that unless otherwise noted, the due time is 6pm Central time! 28 | 29 | ---------- 30 | [w4a]: https://learn.illinois.edu/mod/workshop/view.php?id=1095203 31 | [w4q]: https://learn.illinois.edu/mod/quiz/view.php?id=1095200 32 | [w4v]: https://mediaspace.illinois.edu/media/Week+Four/1_l259tvnv 33 | 34 | Photo Credit: Python Logo. Created 2001, Python.org, Accessed 31 July 2015. 35 | -------------------------------------------------------------------------------- /Week3/lesson2.md: -------------------------------------------------------------------------------- 1 | # Week 3 Lesson 2 # 2 | ## Python: Functions ## 3 | 4 | In this lesson you will learn about how to call built-in Python 5 | functions, as well as how to create your own functions. Additionally, 6 | you will learn some fundamental concepts that you should implement in 7 | the functions that you define: generalization and encapsulation. 8 | 9 | ###Objectives ### 10 | 11 | By the end of this lesson, you will be able to: 12 | 13 | - Perform a python function call. 14 | - Know the difference between different types of functions. 15 | - Be able to define new functions. 16 | - Understand how simple for loops work. 17 | - Understand why generalization should be used. 18 | 19 | ### Time Estimate ### 20 | 21 | Approximately 2 hours. 22 | 23 | ### Readings #### 24 | 25 | - Read Chapters 3-4 in [Think Python](http://faculty.stedwards.edu/mikek/python/thinkpython.pdf) and try some of the exercises shown in those two chapters. 26 | - Python [Functions](notebooks/functionpy.ipynb) Notebook. 27 | 28 | #### *Optional Additional Readings*#### 29 | 30 | - [Google Developers Python Setup Guidelines](https://developers.google.com/edu/python/set-up) on starting a Python interpreter or running a Python program at the command line. Note that in this course, the Docker image comes preinstalled with the latest version of Python3, as that is the future of the language. For the most part, there are no major differences that you will notice (the Python texts you will read are for Python 3), but since some optional readings still use Python 2, I want to make this point. 31 | - [Discussion of Differences between Python 2 and Python 3](http://python3porting.com/intro.html), and a shorter [List of Changes from Python 2 to Python 3](http://inventwithpython.com/appendixa.html). 32 | - Python website tutorial for [Python 2](https://docs.python.org/2.7/tutorial/index.html) and [Python 3](https://docs.python.org/3.4/tutorial/index.html). 33 | 34 | ### Assessment ### 35 | 36 | When you have completed and worked through the above readings, please take the [Week 3 Lesson 2 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095494). 37 | -------------------------------------------------------------------------------- /Week2/README.md: -------------------------------------------------------------------------------- 1 | #Week 2 Overview# 2 | 3 | ![Command line image](images/command-line.png) 4 | ## Introduction to Command-Line Data Science ## 5 | 6 | In this week, you will first be delving further into Unix concepts, such 7 | as commands, input/output redirection, and the unix shell. Secondly, 8 | you will be learning about iPython Notebooks and how to utilize them in 9 | this course. Finally, you will be going through the basics of the 10 | Python language and learning about different types of Python errors, 11 | different data types within Python, as well as how to do simple 12 | manipulations of Python variables. 13 | 14 | ### Objectives ### 15 | 16 | #####By the end of this lesson, you should be able to:###### 17 | 18 | - Understand how to utilize various Unix commands. 19 | - Execute Python commands in an IPython notebook. 20 | - Know basic Python terminology, such as input, output, debugging, etc. 21 | - Perform basic operations on Python variables. 22 | 23 | ### Activities and Assignments ### 24 | 25 | |Activities and Assignments | Time Estimate | Deadline* | Points| 26 | |:------| -----|-------|----------:| 27 | |**[Week 2 Introduction Video][w2v]** | 10 Minutes | Tuesday |20| 28 | |**[Week 2 Lesson 1: Basic Unix Concepts](lesson1.md)**| 3 Hours |Thursday| 20| 29 | |**[Week 2 Lesson 2: Introduction to iPython](lesson2.md)**| 2 Hours | Thursday | 20 | 30 | |**[Week 2 Lesson 3: Introduction to Python](lesson3.md)**| 2 Hours | Thursday| 20 | 31 | |**[Week 2 Quiz][w2q]**| 30 Minutes | Friday | 70| 32 | |**[Week 2 Assignment Submission][w2a] to Instructor and for Peer Grading**| 3 Hours | Saturday | 60% of the grade from the Instructor, 40% of the grade from Peer grading | 33 | |**Week 2 Completion of Peer Review**| 3 Hours | *The following* Saturday | 50 | 34 | 35 | *Please note that unless otherwise noted, the due time is 6pm Central time!* 36 | 37 | ---------- 38 | [w2q]: https://learn.illinois.edu/mod/quiz/view.php?id=1095158 39 | [w2v]: https://mediaspace.illinois.edu/media/Week+Two.mp4/0_2umvmvle/33195071 40 | [w2a]: https://learn.illinois.edu/mod/workshop/view.php?id=1095161 41 | 42 | Photo Credit: Command Line Screen Shot by Jim Hoskins. 27 Sept. 2012. http://blog.teamtreehouse.com/introduction-to-the-mac-os-x-command-line. 31 July 2015 43 | -------------------------------------------------------------------------------- /Week10/README.md: -------------------------------------------------------------------------------- 1 | #Week 10 Overview# 2 | 3 | ## Introduction to Statistical Analysis ## 4 | 5 | This week you will learn about basic statistical analysis by using 6 | distribution functions and summary statistics. First, you will learn 7 | about representing data by a distribution, and how summary statistics 8 | can simplify this description. Next you will learn about more powerful 9 | distributions functions, including ones generated directly from the data 10 | like a probability mass function and a cumulative density function, as 11 | well as more commonly used theoretical distributions that can be used to 12 | approximate data. Finally, you will learn about visualization 13 | distributions by using matplotlib and seaborn. 14 | 15 | ### Objectives ### 16 | 17 | #####By the end of this lesson, you should accomplish the following learning objectives:###### 18 | 19 | - Understand the different summary statistics. 20 | - Understand how to create and use mass and density functions to represent data sets. 21 | - Understand how theoretical distributions can be used to approximate data sets. 22 | - Understand how to visualize distributions by using Python. 23 | 24 | ### Activities and Assignments ### 25 | 26 | |Activities and Assignments | Time Estimate | Deadline* | Points| 27 | |:------| -----|-------|----------:| 28 | |**[Week 10 Introduction Video][w10v]**|10 Minutes|Tuesday|NA| 29 | |**[Week 10 Lesson 1: Statistics: Summary Measures](lesson1.md)**| 2 Hours |Thursday| 20| 30 | |**[Week 10 Lesson 2: Statistics: Distributions](lesson2.md)**| 2 Hours | Thursday | 20 | 31 | |**[Week 10 Lesson 3: Statistics: Visualization](lesson3.md)**| 2 Hours | Thursday| 70 | 32 | |**[Week 10 Quiz][w10q]**| 35 Minutes | Friday | 70| 33 | |**[Week 10 Assignment Submission][w10a]**| 3 Hours | Saturday | 60% of the grade from the Instructor, 40% of the grade from Peer grading | 34 | |**Week 10 Completion of Peer Review**| 3 Hours | *The following* Saturday | 50 | 35 | 36 | 37 | *Please note that unless otherwise noted, the due time is 6pm Central time! 38 | 39 | ---------- 40 | [w10v]: https://mediaspace.illinois.edu/media/t/1_bqgkcyum/33195071 41 | [w10a]: https://learn.illinois.edu/mod/workshop/view.php?id=1095341 42 | [w10q]: https://learn.illinois.edu/mod/quiz/view.php?id=1095338 43 | -------------------------------------------------------------------------------- /Week7/README.md: -------------------------------------------------------------------------------- 1 | #Week 7 Overview# 2 | 3 | ![conditional_risk comic from XKCD](images/conditional_risk.png) 4 | ## Introduction to Statistical Analysis ## 5 | 6 | The lessons this week will focus on learning about data visualizations. 7 | First, you will learn how visualization masters enable others to _see_ 8 | data. Next, you will start using an iPython notebook to make data 9 | visualizations. In particular, you will learn how to make general plots, 10 | and how to create and interpret scatter plots by using the Matplotlib 11 | and Seaborn libraries. 12 | 13 | ### Objectives ### 14 | 15 | #####By the end of this lesson, you should accomplish the following learning objectives:###### 16 | 17 | - Understand the power of strong visualizations 18 | - Understand how to use Matplotlib within an iPython notebook to create a new plot 19 | - Understand how to use the Seaborn library to improve the appearance of a Python plot. 20 | - Understand how to make scatter plots in Python 21 | - Understand how to use scatter plots to understand a data set 22 | 23 | 24 | ### Activities and Assignments ### 25 | 26 | |Activities and Assignments | Time Estimate | Deadline* | Points| 27 | |:------| -----|-------|----------:| 28 | |**[Week 7 Introduction Video][w7v]**|10 Minutes|Tuesday|NA| 29 | |**[Week 7 Lesson 1: Introduction to Data Visualizations](lesson1.md)**| 2 Hours |Thursday| 20| 30 | |**[Week 7 Lesson 2: Python: Plotting](lesson2.md)**| 3 Hours | Thursday | 20 | 31 | |**[Week 7 Lesson 3: Python: Data Plotting](lesson3.md)**| 3 Hours | Thursday| 70 | 32 | |**[Week 7 Quiz][w7q]**| 35 Minutes | Friday | 70| 33 | |**[Week 7 Assignment Submission][w7a]**| 3 Hours | Saturday | 60% of the grade from the Instructor, 40% of the grade from Peer grading | 34 | |**Week 7 Completion of Peer Review**| 3 Hours | *The following* Saturday | 50 | 35 | 36 | 37 | *Please note that unless otherwise noted, the due time is 6pm Central time! 38 | 39 | ---------- 40 | [w7v]: https://mediaspace.illinois.edu/media/t/1_nvk0x8np/33195071 41 | [w7a]: https://learn.illinois.edu/mod/workshop/view.php?id=1095263 42 | [w7q]: https://learn.illinois.edu/mod/quiz/view.php?id=1095266 43 | 44 | Photo Credit: [Conditional Risk](http://imgs.xkcd.com/comics/conditional_risk.png) by Randall Monroe. Accessed on 31 July 2015. 45 | -------------------------------------------------------------------------------- /Week6/README.md: -------------------------------------------------------------------------------- 1 | #Week 6 Overview# 2 | ![XKCD Extrapolating](images/regular_expressions.png) 3 | ## Regular Expressions and Text Processing ## 4 | 5 | In this week, you will begin to use Python and Unix to learn about regular expressions and their associated commands so that you can search through text more effectively. To finish up, you will learn about Unix text processing so that you can better modify and glean information from lines and files of text. 6 | 7 | ### Objectives ### 8 | 9 | #####By the end of this lesson, you should accomplish the following learning objectives:###### 10 | 11 | - Understand how to use the sub, search, and compile functions within the re library. 12 | - Understand how to effectively specify the position and structure of the string you are searching for by using ```\b, ^, $, \d, \D, ?, *, +, {n,m}, (a|b),``` etc. 13 | - Understand how to sort through lines of text and report differences of files through the use of ```sort and uniq, comm, and diff```. 14 | - Understand how to modify lines of text and files using ```cut, paste, join, and patch```. 15 | 16 | 17 | ### Activities and Assignments ### 18 | 19 | |Activities and Assignments | Time Estimate | Deadline* | Points| 20 | |:------| -----|-------|----------:| 21 | |**[Week 6 Introduction Video][w6v]**|10 Minutes|Tuesday|20| 22 | |**[Week 6 Lesson 1:Unix: Regular Expressions & Commands](lesson1.md)**| 2 Hours |Thursday| 20| 23 | |**[Week 6 Lesson 2:Python: Regular Expressions](lesson2.md)**| 3 Hours | Thursday | 20 | 24 | |**[Week 6 Lesson 3:Unix Text Processing](lesson3.md)**| 3 Hours | Thursday| 70 | 25 | |**[Week 6 Quiz][w6q]**| 35 Minutes | Friday | 70| 26 | |**[Week 6 Assignment Submission][w6a]**| 3 Hours | Saturday | 60% of the grade from the Instructor, 40% of the grade from Peer grading | 27 | |**Week 6 Completion of Peer Review**| 3 Hours | *The following* Tuesday | 50 | 28 | 29 | *Please note that unless otherwise noted, the due time is 6pm Central time! 30 | 31 | ---------- 32 | [w6a]: https://learn.illinois.edu/mod/workshop/view.php?id=1095242 33 | [w6q]: https://learn.illinois.edu/mod/quiz/view.php?id=1095245 34 | [w6v]: https://mediaspace.illinois.edu/media/Week+Six/1_9injl00l/33195071 35 | 36 | Photo Credit: [Regular Expressions Comic](http://xkcd.com/208/) by Randall Monroe. Accessed on the 23 of September. 37 | -------------------------------------------------------------------------------- /Week10/lesson2.md: -------------------------------------------------------------------------------- 1 | # Week 10 Lesson 2 # 2 | 3 | ## Distribution Functions ## 4 | 5 | In this lesson, you will learn about the different types of distribution 6 | functions that can be used to represent a data set, including both 7 | discrete and continuous values. You will lear how to construct at 8 | distribution functions from a data set, and how to use this 9 | summarization to aid in the understanding of the data under analysis. 10 | 11 | 12 | ###Objectives ### 13 | By the end of this lesson, you will be able to: 14 | 15 | - Understand probability mass functions and their use to represent discrete data. 16 | - Understand continous distribution function and how it can be used to infer relationships about data. 17 | - Understand the different distributions functions like Normal, Poisson, and Chi-Squared distribution. 18 | 19 | ### Time Estimate ### 20 | 21 | Approximately 3 hours. 22 | 23 | ### Readings #### 24 | 25 | In the Think Stats2 book, the author likes to hide details behind his 26 | custom Python code. We don't agree with this appraoch, especially when 27 | teaching the material. We encourage you to not use his code, but to use 28 | standard Python data structures (like a dictionary) to represent the 29 | distributions. 30 | 31 | - Chapter 3, Probability Mass Functions, from [Think Stats2](http://www.greenteapress.com/thinkstats2/html/thinkstats2004.html) 32 | - Chapter 4, Cumulative Distribution Functions, from [Think Stats2](http://www.greenteapress.com/thinkstats2/html/thinkstats2005.html) 33 | - Short Section on Continuous Distribution Functions in [Introduction to Statistics](http://work.thaslwanter.at/Stats/html/statsDistributions.html#continuous-distribution-functions) 34 | 35 | #### *Optional Additional Readings*#### 36 | 37 | - Introduction to [using distributions in Python](https://oneau.wordpress.com/2011/02/28/simple-statistics-with-scipy/) 38 | - [Distribution Chart](http://www.johndcook.com/blog/distribution_chart/) 39 | - Section on Distribution Functions in [Introduction to Statistics](http://work.thaslwanter.at/Stats/html/statsDistributions.html#distribution-functions) 40 | 41 | ### Assessment ### 42 | 43 | When you have completed and worked through the above readings, please take the [Week 10 Lesson 2 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095590). 44 | -------------------------------------------------------------------------------- /Week2/lesson2.md: -------------------------------------------------------------------------------- 1 | # Week 2 Lesson 2# 2 | ## Introduction to iPython ## 3 | 4 | In this lesson, you will learn how to work with an IPython Notebook (note the name is being changed to Jupiter Notebook) from within your Docker INFO490 course container. You will first learn how to view notebooks online, before downloading them locally to execute. Next you will learn how to enter basic Python commands in the notebook and execute them. Finally, you will learn about IPython magic commands and how they can give you access to the underlying operating system and increased functionality. 5 | 6 | ###Objectives ### 7 | 8 | By the end of this lesson, you will be able to: 9 | 10 | - View an IPython notebook on github via the nbviewer. 11 | - Download an IPython notebook and execute it by using your info490 Docker container. 12 | - Execute Python commands in an IPython notebook. 13 | - Understand basic IPython magic command syntax. 14 | 15 | ### Time Estimate ### 16 | 17 | Approximately 2 hours. 18 | 19 | ### Readings #### 20 | 21 | - [Introduction to the iPython Notebook](notebooks/intro2ipy.ipynb) 22 | - View other notebooks on [nbviewer](http://nbviewer.ipython.org/) website. 23 | 24 | #### *Optional Additional Readings*#### 25 | 26 | - The Berkeley python Bootcamp [IPython Notebook](http://nbviewer.ipython.org/github/profjsb/python-bootcamp/blob/master/Lectures/04_IPythonNotebookIntroduction/IPython+-+beyond+plain+Python.ipynb). 27 | - First, view this online at the provided link 28 | - Second, download the notebook (via download icon in the upper right) 29 | - Third, upload this notebook to your Docker ipython server and run the commands interactively. 30 | - Read about how to write [Markdown formatted](http://nbviewer.ipython.org/github/profjsb/python-bootcamp/blob/master/Lectures/04_IPythonNotebookIntroduction/Markdown%20Cells.ipynb) IPython cells. 31 | - Learn about [IPython extensions](https://github.com/ipython-contrib/IPython-notebook-extensions). 32 | - Learn more about the basics of iPython in Chapter 1 of [IPython Interactive Computing and Visualization Cookbook](http://proquest.safaribooksonline.com.proxy2.library.illinois.edu/book/programming/python/9781783284818) 33 | 34 | ### Assessment ### 35 | 36 | When you have completed and worked through the above readings, please take the [Week 2 Lesson 2 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095482). 37 | -------------------------------------------------------------------------------- /Week14/README.md: -------------------------------------------------------------------------------- 1 | #Week 14 Overview# 2 | 3 | ![Data Processing Comic](images/BBQ.gif) 4 | ## Introduction to Python Data Processing ## 5 | 6 | In this week, you will learn to perform basic data science tasks by 7 | using Python. First, you will learn how to effectively use Pandas to 8 | read and write data to a database directly from a `DataFrame`. Second, 9 | you will learn about web pages, and how to programmatically extract data 10 | from a web page or even websites. Finally, you will learn how to write a 11 | Python program that extract data from a website, processes an XML-based 12 | file to encode this information, in order to generate a new type of 13 | visualization known as a Chloropeth. 14 | 15 | ### Objectives ### 16 | 17 | #####By the end of this lesson, you should accomplish the following learning objectives:###### 18 | 19 | - Understand how to use Pandas to directly interact with a database 20 | - Understand how to use a parsing library to extract information from a data file. 21 | - Understand how the document object model. 22 | - Understand how to extract data from website. 23 | - Understand how to make a Chloropeth visualization. 24 | 25 | ### Activities and Assignments ### 26 | 27 | |Activities and Assignments | Time Estimate | Deadline* | Points| 28 | |:------| -----|-------|----------:| 29 | |**[Week 14 Introduction Video][w14v]**|10 Minutes|Tuesday|NA| 30 | |**[Week 14 Lesson 1: Python: Pandas Database Programming](lesson1.md)**| 1 Hours |Thursday| 20| 31 | |**[Week 14 Lesson 2: Python: Web Scraping](lesson2.md)**| 2 Hours | Thursday | 20 | 32 | |**[Week 14 Lesson 3: Python: Data Science example](lesson3.md)**| 2 Hours | Thursday| 70 | 33 | |**[Week 14 Quiz][w14q]**| 35 Minutes | Friday | 70| 34 | |**[Week 14 Assignment Submission][w14a]**| 3 Hours | Saturday | 60% of the grade from the Instructor, 40% of the grade from Peer grading | 35 | |**Week 14 Completion of Peer Review**| 3 Hours | *The following* Saturday | 50 | 36 | 37 | *Please note that unless otherwise noted, the due time is 6pm Central time! 38 | 39 | ---------- 40 | [w14v]: https://mediaspace.illinois.edu/media/t/0_sis7qq9g 41 | [w14a]: https://learn.illinois.edu/mod/workshop/view.php?id=1095425 42 | [w14q]: https://learn.illinois.edu/mod/quiz/view.php?id=1095422 43 | Photo Credit: Data Processing Center. By Harley Schwadron. From cartoonstock.com. Accessed 3 August 2015. 44 | -------------------------------------------------------------------------------- /Week12/lesson3.md: -------------------------------------------------------------------------------- 1 | # Week 12 Lesson 3 # 2 | ## Using SQL for Data Manipulation ## 3 | 4 | In this lesson, you will learn how to add, change, delete, and select 5 | data by using the SQLite database. First, you will learn about the SQL 6 | INSERT command, which you will use to insert data into a SQLite 7 | database. Second, you will learn about the SQL SELECT query command, 8 | including the use of operators, expressions, and a WHERE clause to 9 | extract data from a SQLite database. Next you will learn about making 10 | more complex SQL queries by creating compound SQL statements with the 11 | AND and OR clauses. Next, you will leaner about changing or deleting 12 | data from a SQLIte database by using the SQL UPDATE and DELETE commands. 13 | Finally, you will learn more advanced SQL commands to create more 14 | powerful queries for extracting, changing, or deleting data from a 15 | SQLite database. 16 | 17 | In this lesson, you can also try some of these SQL statements out on 18 | other databases by using the online SQL emulator located at 19 | http://sqlzoo.net/wiki/Main_Page. By changing the Engine drop-down box, 20 | you can choose to execute SQL against a MySQL, Oracle, SQL Server, 21 | PostgreSQL, Ingres, or DB2 database. 22 | 23 | ###Objectives ### 24 | By the end of this lesson, you will be able to: 25 | 26 | - Understand how to insert data into a SQL database. 27 | - Understand how to extract data from a SQL database. 28 | - Understand how to change data in a SQL database. 29 | - Understand how to delete data in a SQL database. 30 | 31 | ### Time Estimate ### 32 | 33 | Approximately 2 hours. 34 | 35 | ### Readings #### 36 | 37 | - Introduction to [SQL DML Notebook](notebook/intro2sqldml.ipynb) 38 | 39 | #### *Optional Additional Readings*#### 40 | 41 | - [Database and SQL Tutorial](http://www.tutorialspoint.com/sql/index.htm): From the SQL-Insert Query page through the SQL-Sorting Results page. 42 | - [SQLite Tutorial](http://www.tutorialspoint.com/sqlite/index.htm): From the SQLite-Insert Query page through the SQLite-Distinct page 43 | - [SQL Tutorial](http://www.w3schools.com/sql/) 44 | - [SQL Online Tutorial](http://sqlzoo.net/wiki/Main_Page) 45 | 46 | ### Assessment ### 47 | 48 | When you have completed and worked through the above readings, please take the [Week 12 Lesson 3 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095617). 49 | -------------------------------------------------------------------------------- /Week3/README.md: -------------------------------------------------------------------------------- 1 | #Week 3 Overview# 2 | ![python logo](images/python-logo.png) 3 | ## A Further Exploration into the Python Programming Language and Unix ## 4 | 5 | This week you will begin to learn how to use the Python programming 6 | language to write simple programs, and you will also learn about 7 | permissions and controlling processes within Unix-like Systems. First, 8 | you will learn about permission-related commands within Unix-like 9 | systems, such as chmod, umask, su, sudo, chown, and chgrp. Second, you 10 | will learn how to control processes within Unix-like systems through 11 | commands including top, ps, jobs, bg, fg, kill, killall, and shutdown. 12 | Third, you will learn how to perform various function calls within 13 | Python, as well as create your own functions. Finally, you will learn 14 | how to create conditional statements, value-returning functions and 15 | iterating statements. 16 | 17 | ### Objectives ### 18 | 19 | #####By the end of this lesson, you should be able to:###### 20 | 21 | - Know how to utilize various permissions-based commands, such as id, umask, sudo, etc. 22 | - Understand how to use various process-based commands, such as ps, top, bg, kill, etc. 23 | - Be able to define new functions in Python. 24 | 25 | ### Activities and Assignments ### 26 | 27 | |Activities and Assignments | Time Estimate | Deadline* | Points| 28 | |:------| -----|-------|----------:| 29 | |**[Week 3 Introduction Video][wv]** |10 Minutes|Tuesday|20| 30 | |**[Week 3 Lesson 1: Unix File System & Processes](lesson1.md)**| 2 Hours |Thursday| 20| 31 | |**[Week 3 Lesson 2: Python: Functions](lesson2.md)**| 2 Hours | Thursday | 20 | 32 | |**[Week 3 Lesson 3: Python: Conditional Statements & Iteration](lesson3.md)**| 2 Hours | Thursday| 20 | 33 | |**[Week 3 Quiz][wq]**| 35 Minutes | Friday | 70| 34 | |**[Week 3 Assignment Submission][wa]**| 3 Hours | Saturday | 60% of the grade from the Instructor, 40% of the grade from Peer grading | 35 | |**Week 3 Completion of Peer Review**| 3 Hours | *The following* Saturday | 50 | 36 | 37 | *Please note that unless otherwise noted, the due time is 6pm Central time!* 38 | 39 | ---------- 40 | [wv]: https://mediaspace.illinois.edu/media/Week+Three/0_rjhxhl68 41 | [wa]: https://learn.illinois.edu/mod/workshop/view.php?id=1095182 42 | [wq]: https://learn.illinois.edu/mod/quiz/view.php?id=1095179 43 | 44 | 45 | Photo Credit: Python Logo. Created 2001, Python.org, Accessed 31 July 2015. 46 | -------------------------------------------------------------------------------- /Week11/lesson1.md: -------------------------------------------------------------------------------- 1 | # Week 11 Lesson 1 # 2 | ## Functional Programming ## 3 | 4 | In this lesson, you will learn about functional programming, in which 5 | functions operator on data without _side effects_. In this programming 6 | paradigm data is transformed from one state to another by the 7 | application of programs. This paradigm can produce high performance 8 | since the functions can be vectorized over the data. Python, while not 9 | strictly a functional programming language, does provide considerable 10 | support for this programming paradigm. Learning functional programming 11 | techniques in Python can, there fore, improve your ability to write 12 | optimized applications. 13 | 14 | ###Objectives ### 15 | By the end of this lesson, you will be able to: 16 | 17 | - Understand the basic concepts of functional programming. 18 | - Understand how applying functions directly to data structures can improve code performance. 19 | - Understand how to use a numpy ufuncs. 20 | - Understand how to use Pandas apply. 21 | 22 | ### Time Estimate ### 23 | 24 | Approximately 2 hours. 25 | 26 | ### Readings #### 27 | 28 | - Python [HowTo article](https://docs.python.org/dev/howto/functional.html#functional-programming-howto) on Functional Programming. Stop when you reach the _Iterator_ subsection. 29 | 30 | - Numpy [ufunc](http://docs.scipy.org/doc/numpy/reference/ufuncs.html) discussion, stop after _Broadcasting_ section. 31 | 32 | - Available [numpy ufunc list](http://docs.scipy.org/doc/numpy/reference/ufuncs.html#available-ufuncs). 33 | 34 | - Pandas [apply](http://pandas.pydata.org/pandas-docs/stable/10min.html?highlight=apply#apply) method. 35 | 36 | - Pandas [apply method](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.apply.html) api. 37 | 38 | #### *Optional Additional Readings*#### 39 | 40 | - [Function application and mapping section](http://proquest.safaribooksonline.com.proxy2.library.illinois.edu/book/programming/python/9781449323592/5dot-getting-started-with-pandas/id2829411) in Python for data analysis. 41 | - Python for Data Analysis [Advanced ufunc usage](http://proquest.safaribooksonline.com.proxy2.library.illinois.edu/book/programming/python/9781449323592/12dot-advanced-numpy/id2817588) discussion. 42 | 43 | ### Assessment ### 44 | 45 | When you have completed and worked through the above readings, please take the [Week 11 Lesson 1 Assessment](https://learn.illinois.edu/mod/quiz/view.php?id=1095599). 46 | -------------------------------------------------------------------------------- /Week5/README.md: -------------------------------------------------------------------------------- 1 | #Week 5 Overview# 2 | 3 | ![XKCD sustainability comic](images/xkcd_sustainable.png) 4 | ## Networking and File Input/Output ## 5 | 6 | The lessons this week will focus on learning how to use: networking commands inside of Unix, such as ping, netstat, etc.; archiving/compression commands in Unix; and commands used to find files using given criteria. Next, we will go on to learn about writing and executing a complete program, how to open and close files, and file input/output all in Python. Finally, we will touch on the Requests library inside of Python, as well as go over what network programming in Python requires. 7 | 8 | 9 | ### Objectives ### 10 | 11 | #####By the end of this lesson, you should accomplish the following learning objectives:###### 12 | 13 | - Write and execute a complete Python program that has the ability to open a file, write or read into it, and then close the file. 14 | - Understand and be able to use simple networking commands, searching commands, and archiving commands in Unix, such as ```ping, netstat, ftp, wget,```, ```xargs, touch, and stat```, and ```gzip, bzip2, tar, zip, rsync,``` etc. 15 | - Understand what network programming in Python requires. 16 | 17 | 18 | ### Activities and Assignments ### 19 | 20 | |Activities and Assignments | Time Estimate | Deadline* | Points| 21 | |:------| -----|-------|----------:| 22 | |**[Week 5 Introduction Video][w5v]**|10 Minutes|Tuesday|20| 23 | |**[Week 5 Lesson 1: Unix: Networking and Basic Commands](lesson1.md)**| 2 Hours |Thursday| 20| 24 | |**[Week 5 Lesson 2: Working with the Underlying File System](lesson2.md)**| 3 Hours | Thursday | 20 | 25 | |**[Week 5 Lesson 3: Python: Network Programming](lesson3.md)**| 3 Hours | Thursday| 70 | 26 | |**[Week 5 Quiz][w5q]**| 35 Minutes | Friday | 70| 27 | |**[Week 5 Assignment Submission][w5a]**| 3 Hours | Saturday | 60% of the grade from the Instructor, 40% of the grade from Peer grading | 28 | |**Week 5 Completion of Peer Review**| 3 Hours | *The following* Saturday | 50 | 29 | 30 | 31 | *Please note that unless otherwise noted, the due time is 6pm Central time!* 32 | 33 | ---------- 34 | [w5q]: https://learn.illinois.edu/mod/quiz/view.php?id=1095224 35 | [w5a]: https://learn.illinois.edu/mod/workshop/view.php?id=1095221 36 | [w5v]: https://mediaspace.illinois.edu/media/t/1_ewzhn255 37 | 38 | Photo Credit: [Sustainable Comic](http://imgs.xkcd.com/comics/sustainable.png) by Randall Monroe, Accessed on 31 July 2015. 39 | -------------------------------------------------------------------------------- /Week12/README.md: -------------------------------------------------------------------------------- 1 | #Week 12 Overview# 2 | 3 | ![Relational Databases Comic](images/relational_databases.jpg) 4 | 5 | ## Introduction to Data Persistence ## 6 | 7 | This week you will learn about relational database management systems 8 | (RDBMS), still the most popular data storage technology. You will learn 9 | SQL, the language by which database users interact with a relational 10 | database. To test database and SQL concepts, we will use the open source 11 | database system known as SQLite, specifically version 3 of the SQLite 12 | database known as sqlite3. Note, most of the documentation you will read 13 | and follow online will refer to running the sqlite program at the 14 | command line, you will run the sqlite3 program both from within an 15 | IPython Notebook and at the command line. Using SQLite, you will learn 16 | about SQL data definition language (DDL) functionality and SQL data 17 | manipulation language (DML) functionality. 18 | 19 | ### Objectives ### 20 | 21 | #####By the end of this lesson, you should accomplish the following learning objectives:###### 22 | 23 | - Understand the basic concepts behind a relational database system. 24 | - Understand how to create and use a SQLite database. 25 | - Understand how to use SQL to interact with a SQLite database. 26 | 27 | ### Activities and Assignments ### 28 | 29 | |Activities and Assignments | Time Estimate | Deadline* | Points| 30 | |:------| -----|-------|----------:| 31 | |**[Week 12 Introduction Video][w12v]**|10 Minutes|Tuesday|NA| 32 | |**[Week 12 Lesson 1: Relational Databases](lesson1.md)**| 2 Hours |Thursday| 20| 33 | |**[Week 12 Lesson 2: SQL: Schema Manipulation](lesson2.md)**| 2 Hours | Thursday | 20 | 34 | |**[Week 12 Lesson 3: SQL: Data Manipulation](lesson3.md)**| 2 Hours | Thursday| 70 | 35 | |**[Week 12 Quiz][w12q]**| 35 Minutes | Friday | 70| 36 | |**[Week 12 Assignment Submission][w12a]**| 3 Hours | Saturday | 60% of the grade from the Instructor, 40% of the grade from Peer grading | 37 | |**Week 12 Completion of Peer Review**| 3 Hours | *The following* Saturday | 50 | 38 | 39 | *Please note that unless otherwise noted, the due time is 6pm Central time! 40 | 41 | ---------- 42 | [w12v]: https://mediaspace.illinois.edu/media/Week+Twelve/1_7ngyznna/33195071 43 | [w12a]: https://learn.illinois.edu/mod/workshop/view.php?id=1095383 44 | [w12q]: https://learn.illinois.edu/mod/quiz/view.php?id=1095380 45 | 46 | Photo Credit: If Libraries were like relational databases by Brian Panulla. Created 31 December 2010. http://ghostednotes.com/2010/12/31/if-libraries-were-like-relational-databases. Accessed 3 August 2015 47 | -------------------------------------------------------------------------------- /Week8/README.md: -------------------------------------------------------------------------------- 1 | #Week 8 Overview# 2 | ![Pandas logo](images/pandas_logo.png) 3 | ##Introduction to Data Analysis in Python## 4 | 5 | In this week, you will learn to perform basic data analysis by using the 6 | Python programming language. To simplify data analysis tasks when 7 | working with Python we will first learn how to create and use fast 8 | numerical arrays and matrices in Python by using the numpy library. 9 | Numpy arrays underlie many of the Python data science software stack, 10 | and thus understanding them can improve your overall efficacy. Next you 11 | will be introduced to the Pandas library. Pandas introduces several new 12 | data structures into the Python language, including the Series and the 13 | DataFrame, both of which allow you to access data by a column label in 14 | the same manner you used numerical indices with a NumPy array. Pandas 15 | goes beyond this to allow you to select, aggregate and filter data by 16 | using expressions when slicing data from a Pandas data structure. 17 | 18 | ### Objectives ### 19 | 20 | #####By the end of this lesson, you should accomplish the following learning objectives:###### 21 | 22 | - Understand how to use the Numpy library to perform fast array and matrix operations 23 | - Understand how to use the Pandas library to read and write structured data into a Python program. 24 | - Understand how to use a Pandas DataFrame to select, group, and filter data. 25 | 26 | ### Activities and Assignments ### 27 | 28 | |Activities and Assignments | Time Estimate | Deadline* | Points| 29 | |:------| -----|-------|----------:| 30 | |**[Week 8 Introduction Video][w8v]**|10 Minutes|Tuesday|NA| 31 | |**[Week 8 Lesson 1: Python: Introduction to Numpy](lesson1.md)**| 3 Hours |Thursday| 20| 32 | |**[Week 8 Lesson 2: Python: Introduction to Pandas](lesson2.md)**| 3 Hours | Thursday | 20 | 33 | |**[Week 8 Lesson 3: Python: Pandas DataFrame](lesson3.md)**| 2 Hours | Thursday| 70 | 34 | |**[Week 8 Quiz][w8q]**| 35 Minutes | Friday | 70| 35 | |**[Week 8 Assignment Submission][w8a]**| 3 Hours | Saturday | 60% of the grade from the Instructor, 40% of the grade from Peer grading | 36 | |**Week 8 Completion of Peer Review**| 3 Hours | *The following* Saturday | 50 | 37 | 38 | 39 | *Please note that unless otherwise noted, the due time is 6pm Central time! 40 | 41 | ---------- 42 | [w8v]: https://mediaspace.illinois.edu/media/Week+Eight/1_eg1flln0/33195071 43 | [w8q]: https://learn.illinois.edu/mod/quiz/view.php?id=1095287 44 | [w8a]: https://learn.illinois.edu/mod/workshop/view.php?id=1095284 45 | Photo Credit: Pandas Logo, From pandas.pydata.org. Accessed 3 August 2015. 46 | 47 | -------------------------------------------------------------------------------- /Week13/assignment/README.md: -------------------------------------------------------------------------------- 1 | # Week 13 Assignment 2 | 3 | You must use the provided templates. Submit your completed assignment (**.ipynb** files) onto Moodle. 4 | 5 | ## Announcement 6 | 7 | You must submit your assignment in two places: 8 | 9 | 1. Week 13 Assignment (Peer Assessment) 10 | 11 | 2. Week 13 Assignment (Instructors) 12 | 13 | One is just for the peer assessment, and the other one is just for instructor's grades. 14 | 15 | ## Submission deadline: Saturday, November 21, 2015, 6:00 PM 16 | 17 | ## Problem 13.1 and 13.2: Use template [OOP](oop.ipynb) 18 | 19 | Don't forget that you have to submit in **two** places, one for peer assessment and the other one for instructor's grades. 20 | 21 | ### How to download the templates 22 | 23 | You can download the template by updating your local course repository that you created in [Week 1 Lesson 2](https://github.com/UI-DataScience/info490-fa15/blob/master/Week1/lesson2.md), e.g. `/home/data_scientist/info490`. Open a terminal, either by using _New_ -> _Terminal_ in the IPython/Jupyter notebooks server, or by using the interactive terminal mode in Docker: 24 | 25 | ```shell 26 | $ docker exec -it /bin/bash 27 | ``` 28 | 29 | You can find the container name by doing `docker ps`. At the Unix shell, go to the directory where the course repository is mounted, 30 | 31 | ```shell 32 | $ cd /home/data_scientist/info490-fa15 33 | ``` 34 | 35 | Now, there are multiple ways to proceed. 36 | 37 | 1. If you don't care about the local changes and just want the local repository to exactly match the course repository: 38 | **WARNING: This will remove all your work. Back up your work before you do this.** 39 | ```shell 40 | $ git fetch --all 41 | $ git reset --hard origin/master 42 | ``` 43 | 44 | 2. Or, if you care about the changes you have made, you can [stash](https://git-scm.com/book/en/v1/Git-Tools-Stashing), pull, and then pop the stash. The changes you have made in the `info490-fa15` directory are saved and temporarily hidden, and will appear again with `git stash pop` after you do `git pull`. 45 | ```shell 46 | $ git stash 47 | $ git pull 48 | $ git stash pop 49 | ``` 50 | 51 | 3. But I think the best way is to fork the course repository and do proper version control of your own repository. The steps involved will be similar to step 1-6 in the [pull request guide](https://github.com/UI-DataScience/info490-fa15/blob/master/CONTRIBUTING.md), but you wouldn't submit a pull request at the end. If you have your own forked repository, you can [sync the fork](https://help.github.com/articles/syncing-a-fork/) by doing 52 | ```shell 53 | $ git fetch upstream 54 | $ git checkout master 55 | $ git merge upstream/master 56 | ``` 57 | -------------------------------------------------------------------------------- /Week14/assignment/README.md: -------------------------------------------------------------------------------- 1 | # Week 14 Assignment 2 | 3 | You must use the provided templates. Submit your completed assignment (**.ipynb** files) onto Moodle. 4 | 5 | ## Announcement 6 | 7 | You must submit your assignment in two places: 8 | 9 | 1. Week 14 Assignment (Peer Assessment) 10 | 11 | 2. Week 14 Assignment (Instructors) 12 | 13 | One is just for the peer assessment, and the other one is just for instructor's grades. 14 | 15 | ## Submission deadline: Saturday, December 5, 2015, 6:00 PM 16 | 17 | ## Problems 14.1 and 14.2: Use template [SVG](svg.ipynb) 18 | 19 | Don't forget that you have to submit in **two** places, one for peer assessment and the other one for instructor's grades. 20 | 21 | ### How to download the templates 22 | 23 | You can download the template by updating your local course repository that you created in [Week 1 Lesson 2](https://github.com/UI-DataScience/info490-fa15/blob/master/Week1/lesson2.md), e.g. `/home/data_scientist/info490`. Open a terminal, either by using _New_ -> _Terminal_ in the IPython/Jupyter notebooks server, or by using the interactive terminal mode in Docker: 24 | 25 | ```shell 26 | $ docker exec -it /bin/bash 27 | ``` 28 | 29 | You can find the container name by doing `docker ps`. At the Unix shell, go to the directory where the course repository is mounted, 30 | 31 | ```shell 32 | $ cd /home/data_scientist/info490-fa15 33 | ``` 34 | 35 | Now, there are multiple ways to proceed. 36 | 37 | 1. If you don't care about the local changes and just want the local repository to exactly match the course repository: 38 | **WARNING: This will remove all your work. Back up your work before you do this.** 39 | ```shell 40 | $ git fetch --all 41 | $ git reset --hard origin/master 42 | ``` 43 | 44 | 2. Or, if you care about the changes you have made, you can [stash](https://git-scm.com/book/en/v1/Git-Tools-Stashing), pull, and then pop the stash. The changes you have made in the `info490-fa15` directory are saved and temporarily hidden, and will appear again with `git stash pop` after you do `git pull`. 45 | ```shell 46 | $ git stash 47 | $ git pull 48 | $ git stash pop 49 | ``` 50 | 51 | 3. But I think the best way is to fork the course repository and do proper version control of your own repository. The steps involved will be similar to step 1-6 in the [pull request guide](https://github.com/UI-DataScience/info490-fa15/blob/master/CONTRIBUTING.md), but you wouldn't submit a pull request at the end. If you have your own forked repository, you can [sync the fork](https://help.github.com/articles/syncing-a-fork/) by doing 52 | ```shell 53 | $ git fetch upstream 54 | $ git checkout master 55 | $ git merge upstream/master 56 | ``` 57 | -------------------------------------------------------------------------------- /Week12/assignment/README.md: -------------------------------------------------------------------------------- 1 | # Week 12 Assignment 2 | 3 | You must use the provided templates. Submit your completed assignment (**.ipynb** files) onto Moodle. 4 | 5 | ## Announcement 6 | 7 | You must submit your assignment in two places: 8 | 9 | 1. Week 12 Assignment (Peer Assessment) 10 | 11 | 2. Week 12 Assignment (Instructors) 12 | 13 | One is just for the peer assessment, and the other one is just for instructor's grades. 14 | 15 | ## Submission deadline: Saturday, November 14, 2015, 6:00 PM 16 | 17 | ## Problem 12.1, 12.2, and 12.3 Use template: [SQL](sql.ipynb) 18 | 19 | Don't forget that you have to submit in **two** places, one for peer assessment and the other one for instructor's grades. 20 | 21 | ### How to download the templates 22 | 23 | You can download the template by updating your local course repository that you created in [Week 1 Lesson 2](https://github.com/UI-DataScience/info490-fa15/blob/master/Week1/lesson2.md), e.g. `/home/data_scientist/info490`. Open a terminal, either by using _New_ -> _Terminal_ in the IPython/Jupyter notebooks server, or by using the interactive terminal mode in Docker: 24 | 25 | ```shell 26 | $ docker exec -it /bin/bash 27 | ``` 28 | 29 | You can find the container name by doing `docker ps`. At the Unix shell, go to the directory where the course repository is mounted, 30 | 31 | ```shell 32 | $ cd /home/data_scientist/info490-fa15 33 | ``` 34 | 35 | Now, there are multiple ways to proceed. 36 | 37 | 1. If you don't care about the local changes and just want the local repository to exactly match the course repository: 38 | **WARNING: This will remove all your work. Back up your work before you do this.** 39 | ```shell 40 | $ git fetch --all 41 | $ git reset --hard origin/master 42 | ``` 43 | 44 | 2. Or, if you care about the changes you have made, you can [stash](https://git-scm.com/book/en/v1/Git-Tools-Stashing), pull, and then pop the stash. The changes you have made in the `info490-fa15` directory are saved and temporarily hidden, and will appear again with `git stash pop` after you do `git pull`. 45 | ```shell 46 | $ git stash 47 | $ git pull 48 | $ git stash pop 49 | ``` 50 | 51 | 3. But I think the best way is to fork the course repository and do proper version control of your own repository. The steps involved will be similar to step 1-6 in the [pull request guide](https://github.com/UI-DataScience/info490-fa15/blob/master/CONTRIBUTING.md), but you wouldn't submit a pull request at the end. If you have your own forked repository, you can [sync the fork](https://help.github.com/articles/syncing-a-fork/) by doing 52 | ```shell 53 | $ git fetch upstream 54 | $ git checkout master 55 | $ git merge upstream/master 56 | ``` 57 | -------------------------------------------------------------------------------- /Week6/assignment/README.md: -------------------------------------------------------------------------------- 1 | # Week 6 Assignment 2 | 3 | Submit your completed assignment (**.ipynb** files) onto Moodle. 4 | 5 | ## Announcement 6 | 7 | You must submit your assignment in two places: 8 | 9 | 1. Week 6 Assignment (Peer Assessment) 10 | 11 | 2. Week 6 Assignment (Instructors) 12 | 13 | One is just for the peer assessment, and the other one is just for instructor's grades. 14 | 15 | ## Submission deadline: Saturday, October 3rd, 2015, 6:00 PM 16 | 17 | ## Problem 6.1. See template: [Flights Data](flights.ipynb) 18 | 19 | ## Problem 6.2. See template: [Twitter](twitter.ipynb) 20 | 21 | Don't forget that you have to submit in **two** places, one for peer assessment and the other one for instructor's grades. 22 | 23 | ### How to download the templates 24 | 25 | You can download the template by updating your local course repository that you created in [Week 1 Lesson 2](https://github.com/UI-DataScience/info490-fa15/blob/master/Week1/lesson2.md), e.g. `/home/data_scientist/info490`. Open a terminal, either by using _New_ -> _Terminal_ in the IPython/Jupyter notebooks server, or by using the interactive terminal mode in Docker: 26 | 27 | ```shell 28 | $ docker exec -it /bin/bash 29 | ``` 30 | 31 | You can find the container name by doing `docker ps`. At the Unix shell, go to the directory where the course repository is mounted, 32 | 33 | ```shell 34 | $ cd /home/data_scientist/info490-fa15 35 | ``` 36 | 37 | Now, there are multiple ways to proceed. 38 | 39 | 1. If you don't care about the local changes and just want the local repository to exactly match the course repository: 40 | **WARNING: This will remove all your work. Back up your work before you do this.** 41 | ```shell 42 | $ git fetch --all 43 | $ git reset --hard origin/master 44 | ``` 45 | 46 | 2. Or, if you care about the changes you have made, you can [stash](https://git-scm.com/book/en/v1/Git-Tools-Stashing), pull, and then pop the stash. The changes you have made in the `info490-fa15` directory are saved and temporarily hidden, and will appear again with `git stash pop` after you do `git pull`. 47 | ```shell 48 | $ git stash 49 | $ git pull 50 | $ git stash pop 51 | ``` 52 | 53 | 3. But I think the best way is to fork the course repository and do proper version control of your own repository. The steps involved will be similar to step 1-6 in the [pull request guide](https://github.com/UI-DataScience/info490-fa15/blob/master/CONTRIBUTING.md), but you wouldn't submit a pull request at the end. If you have your own forked repository, you can [sync the fork](https://help.github.com/articles/syncing-a-fork/) by doing 54 | ```shell 55 | $ git fetch upstream 56 | $ git checkout master 57 | $ git merge upstream/master 58 | ``` 59 | -------------------------------------------------------------------------------- /orientation/Pre-Class_Activity.md: -------------------------------------------------------------------------------- 1 | #Pre-Class Activity# 2 | 3 | This [activity is on moodle](https://learn.illinois.edu/mod/forum/view.php?id=1095101), where the instructions are also shown along with the actual forum where you can complete these tasks. 4 | 5 | ## Introduce Yourself## 6 | 7 | Welcome! Before the course formally begins, I’d like you to take a few 8 | minutes to introduce yourself to the class and get to know your 9 | classmates a bit. Some of the work you do in this course, like the peer 10 | review, will involve some degree of interaction with your peers. 11 | Establishing personal interaction with your peers early in the course 12 | will make your online learning experience much more enjoyable and 13 | engaging. 14 | 15 | 16 | ## Time Estimate## 17 | Approximately 1 hour. 18 | 19 | ## Instructions## 20 | ### Part 1: Updating Your Profile### 21 | 22 | If you haven't already done so, you should update your Moodle profile. 23 | Your Moodle profile gives your classmates easy access to a little bit 24 | about you. Since your classmates can quickly access your Moodle profile 25 | almost any time they see your name listed within the course, it is a 26 | powerful tool for helping them to get to know you and vice versa. 27 | Additionally, when you update your Moodle profile photo, it shows up 28 | next to your name in discussion forums, which is a great help to those 29 | individuals who can remember faces more readily than names. So, your 30 | first task is to update your Moodle profile. 31 | 32 | Here are [Instructions for Updating Your Moodle Profile](http://publish.illinois.edu/atlas-tlt/students/editing-your-profile/) 33 | 34 | ### Part 2: Introductions ### 35 | 36 | Second, use the Pre-Class Acitivyt Moodle discussion forum to provide a brief 2-3 paragraph 37 | introduction to your classmates. In your post, please consider the following questions: 38 | 39 | - What is your name? What do you prefer to be called? 40 | - Where are you from? 41 | - What year are you in school and what is your major? 42 | - Why are you taking this class, and what, in particular, interests you about data science? 43 | - What is something you are looking forward to learning or doing in this class? 44 | - Tell us something about yourself that has nothing to do with school. 45 | 46 | 47 | Post your self-introduction by clicking the `Add a new discussion topic` button, below. 48 | 49 | Read your classmates’ postings. Pick at least one of your classmates’ 50 | postings that is most interesting to you and add your friendly comments. 51 | Also be sure to click on the name or profile picture of your other 52 | classmates to see what they have added to their profile. 53 | 54 | ## Evaluation ## 55 | 56 | 60 points total: (30 points for initial post; 30 points for reply). 57 | -------------------------------------------------------------------------------- /Week9/assignment/README.md: -------------------------------------------------------------------------------- 1 | # Week 9 Assignment 2 | 3 | You must use the provided templates. Submit your completed assignment (**.ipynb** files) onto Moodle. 4 | 5 | ## Announcement 6 | 7 | You must submit your assignment in two places: 8 | 9 | 1. Week 9 Assignment (Peer Assessment) 10 | 11 | 2. Week 9 Assignment (Instructors) 12 | 13 | One is just for the peer assessment, and the other one is just for instructor's grades. 14 | 15 | ## Submission deadline: Saturday, October 24th, 2015, 6:00 PM 16 | 17 | ## Problem 9.1. Use template: [IPython notebooks as JSON](ipynb.ipynb) 18 | 19 | ## Problem 9.2. Use template: [XML](xml.ipynb) 20 | 21 | Don't forget that you have to submit in **two** places, one for peer assessment and the other one for instructor's grades. 22 | 23 | ### How to download the templates 24 | 25 | You can download the template by updating your local course repository that you created in [Week 1 Lesson 2](https://github.com/UI-DataScience/info490-fa15/blob/master/Week1/lesson2.md), e.g. `/home/data_scientist/info490`. Open a terminal, either by using _New_ -> _Terminal_ in the IPython/Jupyter notebooks server, or by using the interactive terminal mode in Docker: 26 | 27 | ```shell 28 | $ docker exec -it /bin/bash 29 | ``` 30 | 31 | You can find the container name by doing `docker ps`. At the Unix shell, go to the directory where the course repository is mounted, 32 | 33 | ```shell 34 | $ cd /home/data_scientist/info490-fa15 35 | ``` 36 | 37 | Now, there are multiple ways to proceed. 38 | 39 | 1. If you don't care about the local changes and just want the local repository to exactly match the course repository: 40 | **WARNING: This will remove all your work. Back up your work before you do this.** 41 | ```shell 42 | $ git fetch --all 43 | $ git reset --hard origin/master 44 | ``` 45 | 46 | 2. Or, if you care about the changes you have made, you can [stash](https://git-scm.com/book/en/v1/Git-Tools-Stashing), pull, and then pop the stash. The changes you have made in the `info490-fa15` directory are saved and temporarily hidden, and will appear again with `git stash pop` after you do `git pull`. 47 | ```shell 48 | $ git stash 49 | $ git pull 50 | $ git stash pop 51 | ``` 52 | 53 | 3. But I think the best way is to fork the course repository and do proper version control of your own repository. The steps involved will be similar to step 1-6 in the [pull request guide](https://github.com/UI-DataScience/info490-fa15/blob/master/CONTRIBUTING.md), but you wouldn't submit a pull request at the end. If you have your own forked repository, you can [sync the fork](https://help.github.com/articles/syncing-a-fork/) by doing 54 | ```shell 55 | $ git fetch upstream 56 | $ git checkout master 57 | $ git merge upstream/master 58 | ``` 59 | -------------------------------------------------------------------------------- /Week10/assignment/README.md: -------------------------------------------------------------------------------- 1 | # Week 10 Assignment 2 | 3 | You must use the provided templates. Submit your completed assignment (**.ipynb** files) onto Moodle. 4 | 5 | ## Announcement 6 | 7 | You must submit your assignment in two places: 8 | 9 | 1. Week 10 Assignment (Peer Assessment) 10 | 11 | 2. Week 10 Assignment (Instructors) 12 | 13 | One is just for the peer assessment, and the other one is just for instructor's grades. 14 | 15 | ## Submission deadline: Saturday, October 31, 2015, 6:00 PM 16 | 17 | ## Problem 10.1. Use template: [PMF and CDF](cdf.ipynb) 18 | 19 | ## Problem 10.2. Use template: [Visualizing Distributions](distributions.ipynb) 20 | 21 | Don't forget that you have to submit in **two** places, one for peer assessment and the other one for instructor's grades. 22 | 23 | ### How to download the templates 24 | 25 | You can download the template by updating your local course repository that you created in [Week 1 Lesson 2](https://github.com/UI-DataScience/info490-fa15/blob/master/Week1/lesson2.md), e.g. `/home/data_scientist/info490`. Open a terminal, either by using _New_ -> _Terminal_ in the IPython/Jupyter notebooks server, or by using the interactive terminal mode in Docker: 26 | 27 | ```shell 28 | $ docker exec -it /bin/bash 29 | ``` 30 | 31 | You can find the container name by doing `docker ps`. At the Unix shell, go to the directory where the course repository is mounted, 32 | 33 | ```shell 34 | $ cd /home/data_scientist/info490-fa15 35 | ``` 36 | 37 | Now, there are multiple ways to proceed. 38 | 39 | 1. If you don't care about the local changes and just want the local repository to exactly match the course repository: 40 | **WARNING: This will remove all your work. Back up your work before you do this.** 41 | ```shell 42 | $ git fetch --all 43 | $ git reset --hard origin/master 44 | ``` 45 | 46 | 2. Or, if you care about the changes you have made, you can [stash](https://git-scm.com/book/en/v1/Git-Tools-Stashing), pull, and then pop the stash. The changes you have made in the `info490-fa15` directory are saved and temporarily hidden, and will appear again with `git stash pop` after you do `git pull`. 47 | ```shell 48 | $ git stash 49 | $ git pull 50 | $ git stash pop 51 | ``` 52 | 53 | 3. But I think the best way is to fork the course repository and do proper version control of your own repository. The steps involved will be similar to step 1-6 in the [pull request guide](https://github.com/UI-DataScience/info490-fa15/blob/master/CONTRIBUTING.md), but you wouldn't submit a pull request at the end. If you have your own forked repository, you can [sync the fork](https://help.github.com/articles/syncing-a-fork/) by doing 54 | ```shell 55 | $ git fetch upstream 56 | $ git checkout master 57 | $ git merge upstream/master 58 | ``` 59 | -------------------------------------------------------------------------------- /Week11/assignment/README.md: -------------------------------------------------------------------------------- 1 | # Week 11 Assignment 2 | 3 | You must use the provided templates. Submit your completed assignment (**.ipynb** files) onto Moodle. 4 | 5 | ## Announcement 6 | 7 | You must submit your assignment in two places: 8 | 9 | 1. Week 11 Assignment (Peer Assessment) 10 | 11 | 2. Week 11 Assignment (Instructors) 12 | 13 | One is just for the peer assessment, and the other one is just for instructor's grades. 14 | 15 | ## Submission deadline: Saturday, November 7, 2015, 6:00 PM 16 | 17 | ## Problem 11.1. Use template: [Function Application in Pandas](apply.ipynb) 18 | 19 | ## Problem 11.2. Use template: [Functional Programming](func.ipynb) 20 | 21 | Don't forget that you have to submit in **two** places, one for peer assessment and the other one for instructor's grades. 22 | 23 | ### How to download the templates 24 | 25 | You can download the template by updating your local course repository that you created in [Week 1 Lesson 2](https://github.com/UI-DataScience/info490-fa15/blob/master/Week1/lesson2.md), e.g. `/home/data_scientist/info490`. Open a terminal, either by using _New_ -> _Terminal_ in the IPython/Jupyter notebooks server, or by using the interactive terminal mode in Docker: 26 | 27 | ```shell 28 | $ docker exec -it /bin/bash 29 | ``` 30 | 31 | You can find the container name by doing `docker ps`. At the Unix shell, go to the directory where the course repository is mounted, 32 | 33 | ```shell 34 | $ cd /home/data_scientist/info490-fa15 35 | ``` 36 | 37 | Now, there are multiple ways to proceed. 38 | 39 | 1. If you don't care about the local changes and just want the local repository to exactly match the course repository: 40 | **WARNING: This will remove all your work. Back up your work before you do this.** 41 | ```shell 42 | $ git fetch --all 43 | $ git reset --hard origin/master 44 | ``` 45 | 46 | 2. Or, if you care about the changes you have made, you can [stash](https://git-scm.com/book/en/v1/Git-Tools-Stashing), pull, and then pop the stash. The changes you have made in the `info490-fa15` directory are saved and temporarily hidden, and will appear again with `git stash pop` after you do `git pull`. 47 | ```shell 48 | $ git stash 49 | $ git pull 50 | $ git stash pop 51 | ``` 52 | 53 | 3. But I think the best way is to fork the course repository and do proper version control of your own repository. The steps involved will be similar to step 1-6 in the [pull request guide](https://github.com/UI-DataScience/info490-fa15/blob/master/CONTRIBUTING.md), but you wouldn't submit a pull request at the end. If you have your own forked repository, you can [sync the fork](https://help.github.com/articles/syncing-a-fork/) by doing 54 | ```shell 55 | $ git fetch upstream 56 | $ git checkout master 57 | $ git merge upstream/master 58 | ``` 59 | -------------------------------------------------------------------------------- /Week7/assignment/README.md: -------------------------------------------------------------------------------- 1 | # Week 7 Assignment 2 | 3 | You must use the provided templates. Submit your completed assignment (**.ipynb** files) onto Moodle. 4 | 5 | ## Announcement 6 | 7 | You must submit your assignment in two places: 8 | 9 | 1. Week 7 Assignment (Peer Assessment) 10 | 11 | 2. Week 7 Assignment (Instructors) 12 | 13 | One is just for the peer assessment, and the other one is just for instructor's grades. 14 | 15 | ## Submission deadline: Saturday, October 10th, 2015, 6:00 PM 16 | 17 | ## Problem 7.1. Use template: [Trigonometric Functions](trig.ipynb) 18 | 19 | ## Problem 7.2. Use template: [Distance vs. Flight Time](distance_time.ipynb) 20 | 21 | Don't forget that you have to submit in **two** places, one for peer assessment and the other one for instructor's grades. 22 | 23 | ### How to download the templates 24 | 25 | You can download the template by updating your local course repository that you created in [Week 1 Lesson 2](https://github.com/UI-DataScience/info490-fa15/blob/master/Week1/lesson2.md), e.g. `/home/data_scientist/info490`. Open a terminal, either by using _New_ -> _Terminal_ in the IPython/Jupyter notebooks server, or by using the interactive terminal mode in Docker: 26 | 27 | ```shell 28 | $ docker exec -it /bin/bash 29 | ``` 30 | 31 | You can find the container name by doing `docker ps`. At the Unix shell, go to the directory where the course repository is mounted, 32 | 33 | ```shell 34 | $ cd /home/data_scientist/info490-fa15 35 | ``` 36 | 37 | Now, there are multiple ways to proceed. 38 | 39 | 1. If you don't care about the local changes and just want the local repository to exactly match the course repository: 40 | **WARNING: This will remove all your work. Back up your work before you do this.** 41 | ```shell 42 | $ git fetch --all 43 | $ git reset --hard origin/master 44 | ``` 45 | 46 | 2. Or, if you care about the changes you have made, you can [stash](https://git-scm.com/book/en/v1/Git-Tools-Stashing), pull, and then pop the stash. The changes you have made in the `info490-fa15` directory are saved and temporarily hidden, and will appear again with `git stash pop` after you do `git pull`. 47 | ```shell 48 | $ git stash 49 | $ git pull 50 | $ git stash pop 51 | ``` 52 | 53 | 3. But I think the best way is to fork the course repository and do proper version control of your own repository. The steps involved will be similar to step 1-6 in the [pull request guide](https://github.com/UI-DataScience/info490-fa15/blob/master/CONTRIBUTING.md), but you wouldn't submit a pull request at the end. If you have your own forked repository, you can [sync the fork](https://help.github.com/articles/syncing-a-fork/) by doing 54 | ```shell 55 | $ git fetch upstream 56 | $ git checkout master 57 | $ git merge upstream/master 58 | ``` 59 | -------------------------------------------------------------------------------- /Week4/assignment/README.md: -------------------------------------------------------------------------------- 1 | # Week 4 Assignment 2 | 3 | Submit your completed assignment (**.ipynb** files) onto Moodle for peer assessment. 4 | 5 | ## Announcement 6 | 7 | An important announcement. Starting week 4, you must submit your assignment in two places: 8 | 9 | 1. Week XX Assignment (Peer Assessment) 10 | 11 | 2. Week XX Assignment (Instructors) 12 | 13 | One is just for the peer assessment, and the other one is just for instructor's grades. 14 | 15 | ## Submission deadline: Saturday, September 19th, 2015, 6:00 PM 16 | 17 | ## Problem 4.1. See template: [Simple Stats](stats.ipynb) 18 | 19 | ## Problem 4.2. See template: [Unix data processing using IPython](unix_ipython.ipynb) 20 | 21 | Don't forget that you have to submit in **two** places, one for peer assessment and the other one for instructor's grades. 22 | 23 | ### How to download the templates 24 | 25 | You can download the template by updating your local course repository that you created in [Week 1 Lesson 2](https://github.com/UI-DataScience/info490-fa15/blob/master/Week1/lesson2.md), e.g. `/home/data_scientist/info490`. Open a terminal, either by using _New_ -> _Terminal_ in the IPython/Jupyter notebooks server, or by using the interactive terminal mode in Docker: 26 | 27 | ```shell 28 | $ docker exec -it /bin/bash 29 | ``` 30 | 31 | You can find the container name by doing `docker ps`. At the Unix shell, go to the directory where the course repository is mounted, 32 | 33 | ```shell 34 | $ cd /home/data_scientist/info490-fa15 35 | ``` 36 | 37 | Now, there are multiple ways to proceed. 38 | 39 | 1. If you don't care about the local changes and just want the local repository to exactly match the course repository: 40 | **WARNING: This will remove all your work. Back up your work before you do this.** 41 | ```shell 42 | $ git fetch --all 43 | $ git reset --hard origin/master 44 | ``` 45 | 46 | 2. Or, if you care about the changes you have made, you can [stash](https://git-scm.com/book/en/v1/Git-Tools-Stashing), pull, and then pop the stash. The changes you have made in the `info490-fa15` directory are saved and temporarily hidden, and will appear again with `git stash pop` after you do `git pull`. 47 | ```shell 48 | $ git stash 49 | $ git pull 50 | $ git stash pop 51 | ``` 52 | 53 | 3. But I think the best way is to fork the course repository and do proper version control of your own repository. The steps involved will be similar to step 1-6 in the [pull request guide](https://github.com/UI-DataScience/info490-fa15/blob/master/CONTRIBUTING.md), but you wouldn't submit a pull request at the end. If you have your own forked repository, you can [sync the fork](https://help.github.com/articles/syncing-a-fork/) by doing 54 | ```shell 55 | $ git fetch upstream 56 | $ git checkout master 57 | $ git merge upstream/master 58 | ``` 59 | -------------------------------------------------------------------------------- /Week8/assignment/README.md: -------------------------------------------------------------------------------- 1 | # Week 8 Assignment 2 | 3 | You must use the provided templates. Submit your completed assignment (**.ipynb** files) onto Moodle. 4 | 5 | ## Announcement 6 | 7 | You must submit your assignment in two places: 8 | 9 | 1. Week 8 Assignment (Peer Assessment) 10 | 11 | 2. Week 8 Assignment (Instructors) 12 | 13 | One is just for the peer assessment, and the other one is just for instructor's grades. 14 | 15 | ## Submission deadline: Saturday, October 17th, 2015, 6:00 PM 16 | 17 | ## Problem 8.1. Use template: [Simple Stats Using Pandas](pdstats.ipynb) 18 | 19 | ## Problem 8.2. Use template: [Flight Cancellations by Month](cancelled.ipynb) 20 | 21 | ![](month_cancelled.png) 22 | 23 | Don't forget that you have to submit in **two** places, one for peer assessment and the other one for instructor's grades. 24 | 25 | ### How to download the templates 26 | 27 | You can download the template by updating your local course repository that you created in [Week 1 Lesson 2](https://github.com/UI-DataScience/info490-fa15/blob/master/Week1/lesson2.md), e.g. `/home/data_scientist/info490`. Open a terminal, either by using _New_ -> _Terminal_ in the IPython/Jupyter notebooks server, or by using the interactive terminal mode in Docker: 28 | 29 | ```shell 30 | $ docker exec -it /bin/bash 31 | ``` 32 | 33 | You can find the container name by doing `docker ps`. At the Unix shell, go to the directory where the course repository is mounted, 34 | 35 | ```shell 36 | $ cd /home/data_scientist/info490-fa15 37 | ``` 38 | 39 | Now, there are multiple ways to proceed. 40 | 41 | 1. If you don't care about the local changes and just want the local repository to exactly match the course repository: 42 | **WARNING: This will remove all your work. Back up your work before you do this.** 43 | ```shell 44 | $ git fetch --all 45 | $ git reset --hard origin/master 46 | ``` 47 | 48 | 2. Or, if you care about the changes you have made, you can [stash](https://git-scm.com/book/en/v1/Git-Tools-Stashing), pull, and then pop the stash. The changes you have made in the `info490-fa15` directory are saved and temporarily hidden, and will appear again with `git stash pop` after you do `git pull`. 49 | ```shell 50 | $ git stash 51 | $ git pull 52 | $ git stash pop 53 | ``` 54 | 55 | 3. But I think the best way is to fork the course repository and do proper version control of your own repository. The steps involved will be similar to step 1-6 in the [pull request guide](https://github.com/UI-DataScience/info490-fa15/blob/master/CONTRIBUTING.md), but you wouldn't submit a pull request at the end. If you have your own forked repository, you can [sync the fork](https://help.github.com/articles/syncing-a-fork/) by doing 56 | ```shell 57 | $ git fetch upstream 58 | $ git checkout master 59 | $ git merge upstream/master 60 | ``` 61 | -------------------------------------------------------------------------------- /Week5/assignment/README.md: -------------------------------------------------------------------------------- 1 | # Week 5 Assignment 2 | 3 | ## Deeadline is extended to Sunday, 9/27, 6:00 PM due to the FAA services being down. 4 | 5 | Submit your completed assignment (**.ipynb** files) onto Moodle for peer assessment. 6 | 7 | ## Announcement 8 | 9 | You must submit your assignment in two places: 10 | 11 | 1. Week 5 Assignment (Peer Assessment) 12 | 13 | 2. Week 5 Assignment (Instructors) 14 | 15 | One is just for the peer assessment, and the other one is just for instructor's grades. 16 | 17 | ## Submission deadline: Saturday, September 26th, 2015, 6:00 PM 18 | 19 | ## Problem 5.1. See template: [Reading a CSV file](fileio.ipynb) 20 | 21 | ## Problem 5.2. See template: [Requests](requests.ipynb) 22 | 23 | Don't forget that you have to submit in **two** places, one for peer assessment and the other one for instructor's grades. 24 | 25 | ### How to download the templates 26 | 27 | You can download the template by updating your local course repository that you created in [Week 1 Lesson 2](https://github.com/UI-DataScience/info490-fa15/blob/master/Week1/lesson2.md), e.g. `/home/data_scientist/info490`. Open a terminal, either by using _New_ -> _Terminal_ in the IPython/Jupyter notebooks server, or by using the interactive terminal mode in Docker: 28 | 29 | ```shell 30 | $ docker exec -it /bin/bash 31 | ``` 32 | 33 | You can find the container name by doing `docker ps`. At the Unix shell, go to the directory where the course repository is mounted, 34 | 35 | ```shell 36 | $ cd /home/data_scientist/info490-fa15 37 | ``` 38 | 39 | Now, there are multiple ways to proceed. 40 | 41 | 1. If you don't care about the local changes and just want the local repository to exactly match the course repository: 42 | **WARNING: This will remove all your work. Back up your work before you do this.** 43 | ```shell 44 | $ git fetch --all 45 | $ git reset --hard origin/master 46 | ``` 47 | 48 | 2. Or, if you care about the changes you have made, you can [stash](https://git-scm.com/book/en/v1/Git-Tools-Stashing), pull, and then pop the stash. The changes you have made in the `info490-fa15` directory are saved and temporarily hidden, and will appear again with `git stash pop` after you do `git pull`. 49 | ```shell 50 | $ git stash 51 | $ git pull 52 | $ git stash pop 53 | ``` 54 | 55 | 3. But I think the best way is to fork the course repository and do proper version control of your own repository. The steps involved will be similar to step 1-6 in the [pull request guide](https://github.com/UI-DataScience/info490-fa15/blob/master/CONTRIBUTING.md), but you wouldn't submit a pull request at the end. If you have your own forked repository, you can [sync the fork](https://help.github.com/articles/syncing-a-fork/) by doing 56 | ```shell 57 | $ git fetch upstream 58 | $ git checkout master 59 | $ git merge upstream/master 60 | ``` 61 | -------------------------------------------------------------------------------- /orientation/README.md: -------------------------------------------------------------------------------- 1 | #Mandatory Orientation Module# 2 | 3 | In this module, you will become familiar with the course, your 4 | instructor, your classmates, and our learning environment. 5 | 6 | ## Time Estimate ## 7 | 8 | This orientation module should take approximately 2-3 hours of dedicated 9 | time to complete. 10 | 11 | ##Objectives## 12 | 13 | The goal of the orientation module is to familiarize you with the course 14 | structure and the online learning environment. The orientation also 15 | helps you obtain the technical skills required for the course. 16 | 17 | After this module, you should be able to: 18 | 19 | - Recall important information about this course. 20 | - Update your Moodle profile. 21 | - Use the technologies required in this course. 22 | 23 | ## Instructional Activities ## 24 | 25 | Below is a list of the activities and assignments you must complete in 26 | this orientation module. Click on the name of each activity for more 27 | detailed instructions. Please do these activities before starting with 28 | the normal class material. To facilitate late enrollment, these items 29 | will not be formally due until the relevant day of the second week of 30 | the course. 31 | 32 | |Activity|Time Estimate|Deadline|Points| 33 | |-----|---|---|---| 34 | |[Course Overview Video][OV]|15 Minutes|Wednesday|20| 35 | |[Course Information and Resources(Syllabus)](syllabus.md)|1 Hour|Wednesday|NA| 36 | |Read through the [Introduction to Github/Git ](notebooks/intro2github.ipynb) Notebook|1 Hour|Wednesday|NA| 37 | |Orientation Quiz| 30 minutes | Wednesday | 70| 38 | |[Pre-Class Activity: Introduce Yourself](Pre-Class_Activity.md)|30 Minutes|Wednesday|60| 39 | 40 | 41 | ## Tips for Success ## 42 | 43 | To do well in this course, remember the following: 44 | 45 | - Log in frequently to manage your messages on a near-daily basis. If 46 | you let them pile up for 3 to 4 days, you might be overwhelmed. You 47 | should start working on new weekly content right away, on Monday of each 48 | week. 49 | 50 | - Stay on top of the course work. Falling behind will make it harder to 51 | catch up, and even harder to understand new concepts that build on 52 | previous lessons. 53 | 54 | - Consider using a source code repository to archive all of your work so 55 | that you have an automatic backup copy of all your assignments. This 56 | will also be useful in the unlikely event that our server goes down and 57 | we are unable to access course submissions. In this situation, you are 58 | still required to keep current on all your tasks and must be prepared to 59 | submit completed materials as soon as submission capability is restored. 60 | 61 | - When possible, provide tips and suggestions to your peers in this 62 | class. As a learning community, we can help each other learn and grow. 63 | One way of doing this is by helping to address the questions that your 64 | peers pose. By engaging with each other, we'll all learn better. 65 | 66 | ----- 67 | [OV]: https://mediaspace.illinois.edu/media/Orientation+Video/1_4wrksitx 68 | --------------------------------------------------------------------------------