\n",
39 | "\n",
40 | "## **Live Training Template**\n",
41 | "\n",
42 | "_Enter a brief description of your session, here's an example below:_\n",
43 | "\n",
44 | "Welcome to this hands-on training where we will immerse yourself in data visualization in Python. Using both `matplotlib` and `seaborn`, we'll learn how to create visualizations that are presentation-ready.\n",
45 | "\n",
46 | "The ability to present and discuss\n",
47 | "\n",
48 | "* Create various types of plots, including bar-plots, distribution plots, box-plots and more using Seaborn and Matplotlib.\n",
49 | "* Format and stylize your visualizations to make them report-ready.\n",
50 | "* Create sub-plots to create clearer visualizations and supercharge your workflow.\n",
51 | "\n",
52 | "## **The Dataset**\n",
53 | "\n",
54 | "_Enter a brief description of your dataset and its columns, here's an example below:_\n",
55 | "\n",
56 | "\n",
57 | "The dataset to be used in this webinar is a CSV file named `airbnb.csv`, which contains data on airbnb listings in the state of New York. It contains the following columns:\n",
58 | "\n",
59 | "- `listing_id`: The unique identifier for a listing\n",
60 | "- `description`: The description used on the listing\n",
61 | "- `host_id`: Unique identifier for a host\n",
62 | "- `host_name`: Name of host\n",
63 | "- `neighbourhood_full`: Name of boroughs and neighbourhoods\n",
64 | "- `coordinates`: Coordinates of listing _(latitude, longitude)_\n",
65 | "- `Listing added`: Date of added listing\n",
66 | "- `room_type`: Type of room \n",
67 | "- `rating`: Rating from 0 to 5.\n",
68 | "- `price`: Price per night for listing\n",
69 | "- `number_of_reviews`: Amount of reviews received \n",
70 | "- `last_review`: Date of last review\n",
71 | "- `reviews_per_month`: Number of reviews per month\n",
72 | "- `availability_365`: Number of days available per year\n",
73 | "- `Number of stays`: Total number of stays thus far\n"
74 | ]
75 | },
76 | {
77 | "cell_type": "markdown",
78 | "metadata": {
79 | "colab_type": "text",
80 | "id": "BMYfcKeDY85K"
81 | },
82 | "source": [
83 | "## **Getting started**"
84 | ]
85 | },
86 | {
87 | "cell_type": "code",
88 | "metadata": {
89 | "colab_type": "code",
90 | "id": "EMQfyC7GUNhT",
91 | "outputId": "c83279e7-a23c-423a-90d7-5a53ac293a87",
92 | "colab": {
93 | "base_uri": "https://localhost:8080/",
94 | "height": 34
95 | }
96 | },
97 | "source": [
98 | "%%shell\n",
99 | "ls"
100 | ],
101 | "execution_count": 0,
102 | "outputs": [
103 | {
104 | "output_type": "stream",
105 | "text": [
106 | "sample_data\n"
107 | ],
108 | "name": "stdout"
109 | },
110 | {
111 | "output_type": "execute_result",
112 | "data": {
113 | "text/plain": [
114 | ""
115 | ]
116 | },
117 | "metadata": {
118 | "tags": []
119 | },
120 | "execution_count": 3
121 | }
122 | ]
123 | },
124 | {
125 | "cell_type": "markdown",
126 | "metadata": {
127 | "id": "J_c9BTrgXLAW",
128 | "colab_type": "text"
129 | },
130 | "source": [
131 | "### **Examples on use of markdown**\n",
132 | "\n",
133 | "#### **Images**\n",
134 | "\n",
135 | "To add images, gifs, or other assets of that kind, make sure to use the HTML `` function as in the following \n",
136 | "```\n",
137 | "
\n",
138 | "\n",
139 | "
\n",
140 | "
\n",
141 | "```\n",
142 | "\n",
143 | "- The `align` argument takes in `\"center\"`, `\"left\"`, `\"right\"`.\n",
144 | "- The `src` argument takes in the raw link of your image.\n",
145 | "- The `width` argument takes in a percentage, where `100%` is the original size of the image. \n",
146 | "\n",
147 | "\n",
148 | "#### **Formulas**\n",
149 | "\n",
150 | "To use formulas, feel free to use Latex Notation as such:\n",
151 | "\n",
152 | "$y = ax + b$\n",
153 | "\n",
154 | "You can even use color schemes like in this example, where coefficients are colored in red\n",
155 | "\n",
156 | "$y = \\color{red}a x + \\color{red}b$\n",
157 | "\n",
158 | "#### **Changing font color and size**\n",
159 | "\n",
160 | "To change or highlight specific texts in a color, you can use the following\n",
161 | "\n",
162 | "```\n",
163 | "**Example text**\n",
164 | "```\n",
165 | "\n",
166 | "Where the results will look like **Example text**.\n",
167 | "\n",
168 | "- The `color` argument takes in a HEX code for your color. "
169 | ]
170 | }
171 | ]
172 | }
--------------------------------------------------------------------------------
/assets/datacamp.svg:
--------------------------------------------------------------------------------
1 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | # **Data Processing in Shell** by **Susan Sun**
2 |
3 | Live training sessions are designed to mimic the flow of how a real data scientist would address a problem or a task. As such, a session needs to have some “narrative” where learners are achieving stated learning objectives in the form of a real-life data science task or project. For example, a data visualization live session could be around analyzing a dataset and creating a report with a specific business objective in mind _(ex: analyzing and visualizing churn)_, a data cleaning live session could be about preparing a dataset for analysis etc ...
4 |
5 | As part of the 'Live training Spec' process, you will need to complete the following tasks:
6 |
7 | Edit this README by filling in the information for steps 1 - 4.
8 |
9 |
10 | ## Step 1: Foundations
11 |
12 | This part of the 'Live training Spec' process is designed to help guide you through session design by having you think through several key questions. Please make sure to delete the examples provided here for you.
13 |
14 | ### A. What problem(s) will students learn how to solve? (minimum of 5 problems)
15 | - How to download single file and multiple files using `wget` and `curl`.
16 | - How to convert, preview, filter, and manipulate data files using `csvkit` library commands.
17 | - How to perform database operations via the commandline using `csvkit` library commands.
18 | - How to chain commands together using operators
19 | - How to install Python libraries using `pip` and execute Python scripts via the command line.
20 |
21 | ### B. What technologies, packages, or functions will students use? Please be exhaustive.
22 | - `wget`
23 | - `curl`
24 | - `csvkit` library
25 | - `pip`
26 |
27 | ### C. What terms or jargon will you define?
28 |
29 | _Whether during your opening and closing talk or your live training, you might have to define some terms and jargon to walk students through a problem you’re solving. Intuitive explanations using analogies are encouraged._
30 |
31 | - Documentation manuals
32 | - Option flags
33 | - Multi-file processing
34 | - ETL: extract, transform, load
35 | - Database operations
36 | - Package manager (`pip`)
37 | - Version control
38 |
39 | ### D. What mistakes or misconceptions do you expect?
40 |
41 | - Depending on the students' operating system, the installation will look different when they take the notebook course content and apply it locally on their machines. This can be alleviated by specifically speending time in the course to call out how each OS will handle the installations differently.
42 |
43 | - Students with less grounding in command line basics might be confused by how the course content is taught inside a notebook. Extra time spent on disambiguating this and showing screenshots of what the content will look like on a Terminal will be helpful.
44 |
45 | ### E. What datasets will you use?
46 |
47 | - Spotify music attributes dataset. Similar to the data used in [Data Processing in Shell](https://learn.datacamp.com/courses/data-processing-in-shell).
48 |
49 |
50 | ## Step 2: Who is this session for?
51 |
52 | Terms like "beginner" and "expert" mean different things to different people, so we use personas to help instructors clarify a live training's audience. When designing a specific live training, instructors should explain how it will or won't help these people, and what extra skills or prerequisite knowledge they are assuming their students have above and beyond what's included in the persona.
53 |
54 | - [ ] Please select the roles and industries that align with your live training.
55 | - [ ] Include an explanation describing your reasoning and any other relevant information.
56 |
57 | ### What roles would this live training be suitable for?
58 |
59 | *Check all that apply.*
60 |
61 | - [ ] Data Consumer
62 | - [ ] Leader
63 | - [x] Data Analyst
64 | - [x] Citizen Data Scientist
65 | - [x] Data Scientist
66 | - [x] Data Engineer
67 | - [ ] Database Administrator
68 | - [x] Statistician
69 | - [x] Machine Learning Scientist
70 | - [x] Programmer
71 | - [ ] Other (please describe)
72 |
73 | ### What industries would this apply to?
74 |
75 | Across all industries that requires product data analysts and scientists.
76 |
77 | ### What level of expertise should learners have before beginning the live training?
78 |
79 | *List three or more examples of skills that you expect learners to have before beginning the live training*
80 |
81 | - Can write a basic SELECT SQL query.
82 | - Can do basic EDA in Python and understand why this needs to be done and can interpret summary statistics output.
83 | - Has done data analysis from beginning to end (e.g. from extracting the data to cleaning data to analyzing and summarizing results)
84 | - Has very basic understanding of command line (e.g. can open terminal, do file navigation, print log)
85 |
86 |
87 | ## Step 3: Prerequisites
88 |
89 | List any prerequisite courses you think your live training could use from. This could be the live session’s companion course or a course you think students should take before the session. Prerequisites act as a guiding principle for your session and will set the topic framework, but you do not have to limit yourself in the live session to the syntax used in the prerequisite courses.
90 |
91 | - [Introduction to Shell](https://learn.datacamp.com/courses/introduction-to-shell)
92 | - [Intermediate Python for Data Science](https://learn.datacamp.com/courses/intermediate-python-for-data-science)
93 | - [Introduction to SQL](https://learn.datacamp.com/courses/introduction-to-sql)
94 |
95 |
96 | ## Step 4: Session Outline
97 |
98 | A live training session usually begins with an introductory presentation, followed by the live training itself, and an ending presentation. Your live session is expected to be around 2h30m-3h long (including Q&A) with a hard-limit at 3h30m. You can check out our live training content guidelines [here](_LINK_).
99 |
100 | ### Introduction Slides
101 | - Introduction to the webinar and instructor (led by DataCamp TA)
102 | - Introduction to the topics
103 | - Why use command line for data processing
104 | - Go over session outline
105 | - Set expectations about Q&A
106 |
107 | ### Live Training
108 | #### 1. Downloading data
109 | - Command Line 101: File navigation, manuals, option flags, interpreting output
110 | - Download a single file
111 | - Download multiple files
112 | - via file names
113 | - via wildcards
114 | - via a saved list
115 | - Q&A & break
116 | #### 2. Exploratory Data Analysis
117 | - Introducing the `csvkit` library
118 | - Unzip data
119 | - Extract data
120 | - Preview data
121 | - Filter and subset data
122 | - Q&A & break
123 | #### 3. Database Operations
124 | - Database 101: What is a database? Basic SQL queries review.
125 | - Extract data from database
126 | - Store SQL query as shell variable
127 | - Command line operators and linking commands
128 | - Pushing data back to database
129 | - Q&A & break
130 | #### 4. Python on the Command Line
131 | - Python 101: running .py files
132 | - `pip` and installing dependencies
133 | - Bringing everything together to create a mini data pipeline.
134 | - Q&A & break
135 |
136 | ### Ending slides
137 | - Recap of what we learned
138 | - The data science mindset
139 | - Call to action and course recommendations
140 |
141 |
142 |
143 | ## Authoring your session
144 |
145 | To get yourself started with setting up your live session, follow the steps below:
146 |
147 | 1. Download and install the "Open in Colabs" extension from [here](https://chrome.google.com/webstore/detail/open-in-colab/iogfkhleblhcpcekbiedikdehleodpjo?hl=en). This will let you take any jupyter notebook you see in a GitHub repository and open it as a **temporary** Colabs link.
148 | 2. Upload your dataset(s) to the `data` folder.
149 | 3. Upload your images, gifs, or any other assets you want to use in the notebook in the `assets` folder.
150 | 4. Check out the notebooks templates in the `notebooks` folder, and keep the template you want for your session while deleting all remaining ones.
151 |
152 | You can author and save your progress on your notebook using **either** of these methods.
153 |
154 | _**How to author your notebook: By directly saving into GitHub**_
155 |
156 | 1. Preview your desired notebook, press on "Open in Colabs" extension - and start developing your content in colabs _(which will act as the solution code to the session)_. :warning: **Important** :warning: Your progress will **not** be saved on Google Colabs since it's a temporary link. To save your progress, make sure to press on `File`, `Save a copy in GitHub` and follow remaining prompts.
157 | 2. Once your notebooks is ready to go, give it the name `session_name_solution.ipynb` create an empty version of the Notebook to be filled out by you and learners during the session, end the file name with `session_name.ipynb`.
158 | 3. Create Colabs links for both sessions and save them in notebooks :tada:
159 |
160 | _**How to author your notebook: By uploading notebook into GitHub**_
161 |
162 | 1. Preview your desired notebook, press on "Open in Colabs" extension - and start developing your content in colabs _(which will act as the solution code to the session)_. Once you're done, press on `file` - `download .ipynb` file - and overwrite the notebook by uploading it into GitHub.
163 | 2. Once your notebooks is ready to go, give it the name `session_name_solution.ipynb` create an empty version of the Notebook to be filled out by you and learners during the session, end the file name with `session_name.ipynb`.
164 | 3. Create Colabs links for both sessions and save them in notebooks :tada:
165 |
166 |
167 | You can check out either of those methods in action using this [recording](https://www.loom.com/share/1eeb148129244edd93fbc34bf5dc7f0d).
168 |
--------------------------------------------------------------------------------
![\"DataCamp](\"https://github.com/datacamp/python-live-training-template/blob/master/assets/datacamp.svg?raw=True\")
\n",
37 | "