├── .gitignore
├── requirements.txt
├── data
├── road_lengths.csv
└── test-set_Nov-Dec.csv
├── README.md
├── LESSON_4_LOOV_cross_validation.ipynb
├── LESSON_3_Modeling.ipynb
├── ANSWERS_LESSON_3_Modeling.ipynb
├── LESSON_2_Exploratory_data_analysis.ipynb
└── LESSON_1_data_preparation.ipynb
/.gitignore:
--------------------------------------------------------------------------------
1 | .DS_Store
2 | .ipynb_checkpoints
3 |
--------------------------------------------------------------------------------
/requirements.txt:
--------------------------------------------------------------------------------
1 | matplotlib
2 | numpy
3 | pandas
4 | geopandas
5 | seaborn
6 | contextily==1.2.0
7 | scikit-learn
8 | tqdm
9 |
--------------------------------------------------------------------------------
/data/road_lengths.csv:
--------------------------------------------------------------------------------
1 | airnow_sensor,longitude,latitude,a_road_500
2 | NJH,-104.939925,39.738578000000004,1.995
3 | i25_glo,-104.9888784,39.7859023,1.295
4 | la_casa,-105.00519109999999,39.7794672,1.7269999999999999
5 | CAMP,-104.987625,39.751184,6.85
6 | i25_denver,-105.015317,39.732146,2.195
7 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | # Time series modeling for air pollution monitoring with a focus on the calibration of low-cost sensors
2 |
3 | A data science tutorial presented by Oladimeji Mudele at [FBK Web Valley 2022](https://webvalley.fbk.eu/)
4 |
5 | # Navigating this repository
6 | `/data/`: contains the data applied in `LESSON_1_data_preparation.ipynb` notebook.
7 |
8 |
9 | Follow the lessons in the name order of the notebooks, i.e `LESSON_1` comes before `LESSON_2`.
10 |
11 | These tutorial materials have been developed based on methods and data applied in the study by [Ellen M. Considine et al.](https://www.sciencedirect.com/science/article/pii/S0269749120365222).
12 |
13 | # Instruction
14 | run `pip install -r requirements.txt`
15 |
--------------------------------------------------------------------------------
/LESSON_4_LOOV_cross_validation.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "markdown",
5 | "id": "0f6a4184",
6 | "metadata": {},
7 | "source": [
8 | "Welcome to this lab session 4 on **Time series modeling for air pollution monitoring with a focus on the\n",
9 | "calibration of low-cost sensors.**\n",
10 | "\n",
11 | "This lab session is based on the data and methods provided in the study by [Ellen M. Considine et al](https://www.sciencedirect.com/science/article/pii/S0269749120365222).\n"
12 | ]
13 | },
14 | {
15 | "cell_type": "markdown",
16 | "id": "78e512b8",
17 | "metadata": {},
18 | "source": [
19 | "In the notebook, we will focus on improving our modeling pipeline by considering cross validation."
20 | ]
21 | },
22 | {
23 | "cell_type": "markdown",
24 | "id": "5e325cf6",
25 | "metadata": {},
26 | "source": [
27 | "The question we intend to answer here is: How can we improve the experiment pipeline presented in LESSON 3 notebook.\n",
28 | "\n",
29 | "To this aim, we present leave-one-location-out cross validation. This cross validation helps us to understand how well our model generalises into new locations corresponding to the same time coverage of our training data.\n",
30 | "\n",
31 | "The idea is to split our training data into training and validation by location.\n",
32 | "\n",
33 | "**Step-by-step process**\n",
34 | "\n",
35 | "- Iterate over the monitor locations\n",
36 | "- For each location,\n",
37 | " - Select data for that location as validation data and deselect these data from training.\n",
38 | " - Fit your model on the resulting training data and predict over the validation location.\n",
39 | " - Check the model error on validation data"
40 | ]
41 | },
42 | {
43 | "cell_type": "markdown",
44 | "id": "4cbe9868",
45 | "metadata": {},
46 | "source": [
47 | "# First, lets import the libraries we will be using"
48 | ]
49 | },
50 | {
51 | "cell_type": "code",
52 | "execution_count": null,
53 | "id": "2d73e52d",
54 | "metadata": {},
55 | "outputs": [],
56 | "source": [
57 | "import math\n",
58 | "\n",
59 | "from tqdm.auto import tqdm\n",
60 | "\n",
61 | "import pandas as pd\n",
62 | "import numpy as np\n",
63 | "import matplotlib.pyplot as plt\n",
64 | "\n",
65 | "from sklearn.ensemble import RandomForestRegressor\n",
66 | "from sklearn.metrics import mean_squared_error"
67 | ]
68 | },
69 | {
70 | "cell_type": "code",
71 | "execution_count": null,
72 | "id": "49e703a1",
73 | "metadata": {},
74 | "outputs": [],
75 | "source": [
76 | "import warnings\n",
77 | "\n",
78 | "warnings.filterwarnings(\"ignore\")"
79 | ]
80 | },
81 | {
82 | "cell_type": "markdown",
83 | "id": "460e415c",
84 | "metadata": {},
85 | "source": [
86 | "# Load the data"
87 | ]
88 | },
89 | {
90 | "cell_type": "code",
91 | "execution_count": null,
92 | "id": "64da27cc",
93 | "metadata": {},
94 | "outputs": [],
95 | "source": [
96 | "data_root = \"./data/\"\n",
97 | "training_data_path = data_root + \"cleaned_training.csv\"\n",
98 | "test_data_path = data_root + \"cleaned_test.csv\""
99 | ]
100 | },
101 | {
102 | "cell_type": "code",
103 | "execution_count": null,
104 | "id": "f98ca87c",
105 | "metadata": {},
106 | "outputs": [],
107 | "source": [
108 | "training_data = pd.read_csv(training_data_path)\n",
109 | "test_data = pd.read_csv(test_data_path)"
110 | ]
111 | },
112 | {
113 | "cell_type": "code",
114 | "execution_count": null,
115 | "id": "c99523ba",
116 | "metadata": {},
117 | "outputs": [],
118 | "source": [
119 | "training_data.head()"
120 | ]
121 | },
122 | {
123 | "cell_type": "markdown",
124 | "id": "4b9bcb6c",
125 | "metadata": {},
126 | "source": [
127 | "# Utility functions"
128 | ]
129 | },
130 | {
131 | "cell_type": "code",
132 | "execution_count": null,
133 | "id": "994ad23e",
134 | "metadata": {},
135 | "outputs": [],
136 | "source": [
137 | "# More evaluation metrics can be added to the function\n",
138 | "def evaluate_model(y, y_hat):\n",
139 | " return {\"RMSE\": round(mean_squared_error(y, y_hat, squared=False), 2)}\n"
140 | ]
141 | },
142 | {
143 | "cell_type": "markdown",
144 | "id": "5f12fe30",
145 | "metadata": {},
146 | "source": [
147 | "Now, we need to get Validation data"
148 | ]
149 | },
150 | {
151 | "cell_type": "code",
152 | "execution_count": null,
153 | "id": "eadcbd49",
154 | "metadata": {},
155 | "outputs": [],
156 | "source": [
157 | "features = [\n",
158 | " \"pm_cs\",\n",
159 | " \"temp\",\n",
160 | " \"humidity\",\n",
161 | " \"a_road_500\",\n",
162 | " \"sin_time\",\n",
163 | " \"cos_time\",\n",
164 | " \"sin_month\",\n",
165 | " \"cos_month\",\n",
166 | "]\n",
167 | "\n",
168 | "# This is tagged model_4 in our last notebook"
169 | ]
170 | },
171 | {
172 | "cell_type": "code",
173 | "execution_count": null,
174 | "id": "28a389dc",
175 | "metadata": {},
176 | "outputs": [],
177 | "source": [
178 | "lolo_validation_errors = {}\n",
179 | "locations = training_data[\"cs_sensor\"].unique()\n",
180 | "\n",
181 | "for leave_sensor in tqdm(locations, total=len(locations)):\n",
182 | "\n",
183 | " train = training_data[training_data[\"cs_sensor\"] != leave_sensor]\n",
184 | " validation = training_data[training_data[\"cs_sensor\"] == leave_sensor]\n",
185 | "\n",
186 | " model = RandomForestRegressor()\n",
187 | "\n",
188 | " x_train, y_train = train[features], train[\"pm_airnow\"]\n",
189 | " x_val, y_val = validation[features], validation[\"pm_airnow\"]\n",
190 | "\n",
191 | " model.fit(x_train, y_train)\n",
192 | "\n",
193 | " y_hat_val = model.predict(x_val)\n",
194 | "\n",
195 | " error = evaluate_model(y_val, y_hat_val)\n",
196 | " lolo_validation_errors[leave_sensor] = error[\"RMSE\"]"
197 | ]
198 | },
199 | {
200 | "cell_type": "markdown",
201 | "id": "cf6b4030",
202 | "metadata": {},
203 | "source": [
204 | "The location names below shows signify the location that have been left out of training but used only to obtain validation error.\n",
205 | "\n",
206 | "| Location| Baseline RMSE| CV Random forest RMSE|\n",
207 | " |---|---|---|\n",
208 | " |**Train**|---|---|\n",
209 | " |NJH | 4.36| 2.26|\n",
210 | " |i25_glo_1|6.67|3.13|\n",
211 | " |i25_glo_2|4.55|2.72|\n",
212 | " |i25_glo_3|5.41|2.37|\n",
213 | " |la_casa|6.06|2.4|\n",
214 | "\n",
215 | " "
216 | ]
217 | },
218 | {
219 | "cell_type": "markdown",
220 | "id": "dd1ba1bf",
221 | "metadata": {},
222 | "source": [
223 | "Leaving I-25 Globeville data out increases our validation error because by removing this monitor location, we exclude samples from three CS sensors in the data. Relative to the full side of our data, this is a lot of samples."
224 | ]
225 | },
226 | {
227 | "cell_type": "markdown",
228 | "id": "86adba1b",
229 | "metadata": {},
230 | "source": [
231 | "Ideally, if applying LOLO cross validation, you want to apply it to the model evaluation and selection step in our previous notebook."
232 | ]
233 | },
234 | {
235 | "cell_type": "markdown",
236 | "id": "da06f8f5",
237 | "metadata": {},
238 | "source": [
239 | "We can represent our training performance in terms of the mean and standard deviation of all the cross validation errors as shown below."
240 | ]
241 | },
242 | {
243 | "cell_type": "code",
244 | "execution_count": null,
245 | "id": "7216115f",
246 | "metadata": {},
247 | "outputs": [],
248 | "source": [
249 | "print(\"error mean: \", round(np.mean(list(lolo_validation_errors.values())), 2))\n",
250 | "print(\"error std: \", round(np.std(list(lolo_validation_errors.values())), 2))"
251 | ]
252 | },
253 | {
254 | "cell_type": "markdown",
255 | "id": "f9edd48f",
256 | "metadata": {},
257 | "source": [
258 | "This tells us that our training RMSE of `0.85` when we use all the locations in training is too optimistic, especially for the case of generalizing to new locations over the same time period of our training data."
259 | ]
260 | },
261 | {
262 | "cell_type": "markdown",
263 | "id": "52b4a9e5",
264 | "metadata": {},
265 | "source": [
266 | "A simple way to combine these cross validated models for test/inference would be to average their outputs\n",
267 | "\n",
268 | "\n",
269 | "$final prediction = (prediction_1 + prediction_2 + prediction_3 + ... + prediction_n) / n$"
270 | ]
271 | },
272 | {
273 | "cell_type": "code",
274 | "execution_count": null,
275 | "id": "c43d3d3e",
276 | "metadata": {},
277 | "outputs": [],
278 | "source": []
279 | }
280 | ],
281 | "metadata": {
282 | "kernelspec": {
283 | "display_name": "Python 3",
284 | "language": "python",
285 | "name": "python3"
286 | },
287 | "language_info": {
288 | "codemirror_mode": {
289 | "name": "ipython",
290 | "version": 3
291 | },
292 | "file_extension": ".py",
293 | "mimetype": "text/x-python",
294 | "name": "python",
295 | "nbconvert_exporter": "python",
296 | "pygments_lexer": "ipython3",
297 | "version": "3.8.5"
298 | }
299 | },
300 | "nbformat": 4,
301 | "nbformat_minor": 5
302 | }
303 |
--------------------------------------------------------------------------------
/LESSON_3_Modeling.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "markdown",
5 | "id": "f92d1d7c",
6 | "metadata": {},
7 | "source": [
8 | "Welcome to this lab session 3 on **Time series modeling for air pollution monitoring with a focus on the\n",
9 | "calibration of low-cost sensors.**\n",
10 | "\n",
11 | "This lab session is based on the data and methods provided in the study by [Ellen M. Considine et al](https://www.sciencedirect.com/science/article/pii/S0269749120365222).\n"
12 | ]
13 | },
14 | {
15 | "cell_type": "markdown",
16 | "id": "fe605444",
17 | "metadata": {},
18 | "source": [
19 | "In the notebook, we will focus on using a regression models (linear model and random forest) to correct the readings of low-cost (CS) sensors."
20 | ]
21 | },
22 | {
23 | "cell_type": "markdown",
24 | "id": "3bc28302",
25 | "metadata": {},
26 | "source": [
27 | "We will start by presenting the use of a linear model for this purpose and we will explore different combinations of predictor variables in our model and compare the results. The idea here is to simulate a case of invastigating the best model and the features with which we get this model. \n",
28 | "\n",
29 | "In addition to a linear model, in our reference literature, a random forest model was also used. The follow-up task you will get in this session is to apply a random forest model and observe the results."
30 | ]
31 | },
32 | {
33 | "cell_type": "markdown",
34 | "id": "74df1fbe",
35 | "metadata": {},
36 | "source": [
37 | "# First, lets import the libraries we will be using"
38 | ]
39 | },
40 | {
41 | "cell_type": "code",
42 | "execution_count": null,
43 | "id": "ab4e3974",
44 | "metadata": {},
45 | "outputs": [],
46 | "source": [
47 | "import math\n",
48 | "from typing import List, Optional\n",
49 | "\n",
50 | "import pandas as pd\n",
51 | "\n",
52 | "import numpy as np\n",
53 | "import matplotlib.pyplot as plt\n",
54 | "\n",
55 | "from sklearn.linear_model import LinearRegression\n",
56 | "from sklearn.ensemble import RandomForestRegressor\n",
57 | "from sklearn.metrics import mean_squared_error"
58 | ]
59 | },
60 | {
61 | "cell_type": "code",
62 | "execution_count": null,
63 | "id": "0c826264",
64 | "metadata": {},
65 | "outputs": [],
66 | "source": [
67 | "import warnings\n",
68 | "\n",
69 | "warnings.filterwarnings(\"ignore\")"
70 | ]
71 | },
72 | {
73 | "cell_type": "markdown",
74 | "id": "b5ce04e2",
75 | "metadata": {},
76 | "source": [
77 | "# Load the data"
78 | ]
79 | },
80 | {
81 | "cell_type": "code",
82 | "execution_count": null,
83 | "id": "9cc0b0ff",
84 | "metadata": {},
85 | "outputs": [],
86 | "source": [
87 | "data_root = \"./data/\"\n",
88 | "training_data_path = data_root + \"cleaned_training.csv\"\n",
89 | "test_data_path = data_root + \"cleaned_test.csv\""
90 | ]
91 | },
92 | {
93 | "cell_type": "code",
94 | "execution_count": null,
95 | "id": "dc58d938",
96 | "metadata": {},
97 | "outputs": [],
98 | "source": [
99 | "training_data = pd.read_csv(training_data_path)\n",
100 | "test_data = pd.read_csv(test_data_path)"
101 | ]
102 | },
103 | {
104 | "cell_type": "code",
105 | "execution_count": null,
106 | "id": "1388ccba",
107 | "metadata": {},
108 | "outputs": [],
109 | "source": [
110 | "training_data.head()"
111 | ]
112 | },
113 | {
114 | "cell_type": "markdown",
115 | "id": "7a3b93c2",
116 | "metadata": {},
117 | "source": [
118 | "# Utility functions"
119 | ]
120 | },
121 | {
122 | "cell_type": "code",
123 | "execution_count": null,
124 | "id": "fa03064d",
125 | "metadata": {},
126 | "outputs": [],
127 | "source": [
128 | "# More evaluation metrics can be added to the function\n",
129 | "def evaluate_model(y, y_hat):\n",
130 | " evaluation = {\"RMSE\": round(mean_squared_error(y, y_hat, squared=False), 2)}\n",
131 | " return evaluation\n",
132 | "\n",
133 | "\n",
134 | "# To disaggregate the error metrics to the different locations\n",
135 | "def get_disaggregated_metrics(data: \"dataframe\", y_hat: Optional[List] = None):\n",
136 | "\n",
137 | " \"\"\"\n",
138 | " Disaggregate evaluation metrics into locations\n",
139 | " \"\"\"\n",
140 | " results = {}\n",
141 | "\n",
142 | " for sensor in data[\"cs_sensor\"].unique():\n",
143 | " data_sensor_idxs = data[data[\"cs_sensor\"] == sensor].index\n",
144 | " true_data = data[[\"pm_airnow\"]][data.index.isin(data_sensor_idxs)]\n",
145 | " pred_data = (\n",
146 | " data[[\"pm_cs\"]][data.index.isin(data_sensor_idxs)]\n",
147 | " if y_hat is None\n",
148 | " else y_hat[data_sensor_idxs]\n",
149 | " )\n",
150 | "\n",
151 | " eval_metrics = evaluate_model(true_data, pred_data)\n",
152 | " results[sensor] = eval_metrics\n",
153 | "\n",
154 | " return {\"RMSE\": results}\n",
155 | "\n",
156 | "\n",
157 | "def run_training_eval(\n",
158 | " features: dict,\n",
159 | " train: \"dataframe\" = training_data,\n",
160 | " test: \"dataframe\" = test_data,\n",
161 | " estimator: \"sklearn model\" = LinearRegression(),\n",
162 | "):\n",
163 | "\n",
164 | " \"\"\"\n",
165 | " Run training and evaluation on the data\n",
166 | "\n",
167 | " Purpose: fast experiment iteration\n",
168 | " \"\"\"\n",
169 | " y_train = train[[\"pm_airnow\"]]\n",
170 | " X_train = training_data[features]\n",
171 | "\n",
172 | " y_test = test[[\"pm_airnow\"]]\n",
173 | " X_test = test[features]\n",
174 | " \n",
175 | " model = estimator\n",
176 | " model.fit(X_train, y_train)\n",
177 | " y_hat_test = model.predict(X_test)\n",
178 | " y_hat_train = model.predict(X_train) # predicted value\n",
179 | "\n",
180 | " train_eval = evaluate_model(y_train, y_hat_train)\n",
181 | " test_eval = evaluate_model(y_test, y_hat_test)\n",
182 | "\n",
183 | " return {\n",
184 | " \"train_eval\": train_eval,\n",
185 | " \"test_eval\": test_eval,\n",
186 | " \"model\": model,\n",
187 | " \"y_hat_train\": y_hat_train,\n",
188 | " \"y_hat_test\": y_hat_test,\n",
189 | " }"
190 | ]
191 | },
192 | {
193 | "cell_type": "markdown",
194 | "id": "a9451c15",
195 | "metadata": {},
196 | "source": [
197 | "# How do we know we have a useful model? We need a baseline"
198 | ]
199 | },
200 | {
201 | "cell_type": "markdown",
202 | "id": "4efa70d7",
203 | "metadata": {},
204 | "source": [
205 | "Before we proceed into deriving correction models, we need to decide how we intend to evaluate our models. \n",
206 | "\n",
207 | "Our base study applied the coefficient of determination ($R^2$) and root-mean squared error (RMSE). \n",
208 | "\n",
209 | "RMSE shows us the mean distance between the predicted and actual values.\n",
210 | "\n",
211 | "For this notebook, we will be using RMSE measure which has been defined as part of our utility functions. Lower RMSE values indicate more accurate models."
212 | ]
213 | },
214 | {
215 | "cell_type": "markdown",
216 | "id": "4c880b86",
217 | "metadata": {},
218 | "source": [
219 | "We will obtain our baseline RMSE by computing the training and test RMSE between `pm_airnow` and `pm_cs`. Any model that will be useful whatsoever should do significantly better (i.e lower) than this baseline."
220 | ]
221 | },
222 | {
223 | "cell_type": "code",
224 | "execution_count": null,
225 | "id": "16cacf72",
226 | "metadata": {},
227 | "outputs": [],
228 | "source": [
229 | "baseline_train_rmse = evaluate_model(training_data[\"pm_airnow\"], training_data[\"pm_cs\"])\n",
230 | "baseline_test_rmse = evaluate_model(test_data[\"pm_airnow\"], test_data[\"pm_cs\"])"
231 | ]
232 | },
233 | {
234 | "cell_type": "code",
235 | "execution_count": null,
236 | "id": "1b24e44d",
237 | "metadata": {},
238 | "outputs": [],
239 | "source": [
240 | "print(f\"Baseline training evaluation is: {baseline_train_rmse}\")\n",
241 | "print(f\"Baseline test evaluation is: {baseline_test_rmse}\")"
242 | ]
243 | },
244 | {
245 | "cell_type": "markdown",
246 | "id": "a3f566fa",
247 | "metadata": {},
248 | "source": [
249 | "It is clear to us now that any model that learns something from our data should produce less than `5.51` and `7.08`RMSE over the training and test set, respectively. Please note that RMSE is on the scale of our data, meaning it is measured in $µg/m^3$."
250 | ]
251 | },
252 | {
253 | "cell_type": "markdown",
254 | "id": "62d7e074",
255 | "metadata": {},
256 | "source": [
257 | "# Can we disaggregate these errors to the different sensors/locations?\n",
258 | "\n",
259 | "Starting from the baseline"
260 | ]
261 | },
262 | {
263 | "cell_type": "code",
264 | "execution_count": null,
265 | "id": "a631db0d",
266 | "metadata": {},
267 | "outputs": [],
268 | "source": [
269 | "baseline_disagg_train_metrics = get_disaggregated_metrics(training_data)"
270 | ]
271 | },
272 | {
273 | "cell_type": "code",
274 | "execution_count": null,
275 | "id": "3ee601ba",
276 | "metadata": {},
277 | "outputs": [],
278 | "source": [
279 | "print(\"Disaggregated RMSE score\")\n",
280 | "baseline_disagg_train_metrics[\"RMSE\"]"
281 | ]
282 | },
283 | {
284 | "cell_type": "code",
285 | "execution_count": null,
286 | "id": "cf1afe17",
287 | "metadata": {},
288 | "outputs": [],
289 | "source": [
290 | "baseline_disagg_test_metrics = get_disaggregated_metrics(test_data)"
291 | ]
292 | },
293 | {
294 | "cell_type": "code",
295 | "execution_count": null,
296 | "id": "45a1e730",
297 | "metadata": {},
298 | "outputs": [],
299 | "source": [
300 | "print(\"Disaggregated test RMSE\")\n",
301 | "baseline_disagg_test_metrics[\"RMSE\"]"
302 | ]
303 | },
304 | {
305 | "cell_type": "markdown",
306 | "id": "c4aedf7f",
307 | "metadata": {},
308 | "source": [
309 | "Again, any model that will be generally useful should do less than these errors in each of these locations (training and test)"
310 | ]
311 | },
312 | {
313 | "cell_type": "markdown",
314 | "id": "210f6722",
315 | "metadata": {},
316 | "source": [
317 | "# Obtaining our models"
318 | ]
319 | },
320 | {
321 | "cell_type": "markdown",
322 | "id": "54e0f3ff",
323 | "metadata": {},
324 | "source": [
325 | "**model_1**: CS PM2.5
\n",
326 | "**model_2**: CS PM2.5, temperature, humidity
\n",
327 | "**model_3**: CS PM2.5, temprature, humidity, road length
\n",
328 | "**model_4**: CS PM2.5, temprature, humidity, road length, hour, month
\n",
329 | "**model_5**: CS PM2.5, temprature, humidity, road length, hour, mont, weekend
"
330 | ]
331 | },
332 | {
333 | "cell_type": "code",
334 | "execution_count": null,
335 | "id": "1f6cb3f1",
336 | "metadata": {},
337 | "outputs": [],
338 | "source": [
339 | "model_features = {\n",
340 | " \"model_1\": [\"pm_cs\"],\n",
341 | " \"model_2\": [\"pm_cs\", \"temp\", \"humidity\"],\n",
342 | " \"model_3\": [\"pm_cs\", \"temp\", \"humidity\", \"a_road_500\"],\n",
343 | " \"model_4\": [\n",
344 | " \"pm_cs\",\n",
345 | " \"temp\",\n",
346 | " \"humidity\",\n",
347 | " \"a_road_500\",\n",
348 | " \"sin_time\",\n",
349 | " \"cos_time\",\n",
350 | " \"sin_month\",\n",
351 | " \"cos_month\",\n",
352 | " ],\n",
353 | " \"model_5\": [\n",
354 | " \"pm_cs\",\n",
355 | " \"temp\",\n",
356 | " \"humidity\",\n",
357 | " \"a_road_500\",\n",
358 | " \"sin_time\",\n",
359 | " \"cos_time\",\n",
360 | " \"sin_month\",\n",
361 | " \"cos_month\",\n",
362 | " \"weekend\",\n",
363 | " ],\n",
364 | "}"
365 | ]
366 | },
367 | {
368 | "cell_type": "markdown",
369 | "id": "55865b03",
370 | "metadata": {},
371 | "source": [
372 | "###### Let us run our first example using only one feature `pm_cs` "
373 | ]
374 | },
375 | {
376 | "cell_type": "markdown",
377 | "id": "ad95ddba",
378 | "metadata": {},
379 | "source": [
380 | "We need to do the following:\n",
381 | "- Choose the features that we will be using to fit our model\n",
382 | "- Extract `x_train, x_test, y_train, y_test`, as features and reponse variables for training and test.\n",
383 | "- Fit the model\n",
384 | "- Predict\n",
385 | "- Evaluate: RMSE and scatter plot"
386 | ]
387 | },
388 | {
389 | "cell_type": "code",
390 | "execution_count": null,
391 | "id": "33b0b13e",
392 | "metadata": {},
393 | "outputs": [],
394 | "source": [
395 | "features = model_features[\"model_1\"]\n",
396 | "\n",
397 | "y_train = training_data[[\"pm_airnow\"]]\n",
398 | "X_train = training_data[features]\n",
399 | "\n",
400 | "y_test = test_data[[\"pm_airnow\"]]\n",
401 | "X_test = test_data[features]"
402 | ]
403 | },
404 | {
405 | "cell_type": "code",
406 | "execution_count": null,
407 | "id": "1ca765e3",
408 | "metadata": {},
409 | "outputs": [],
410 | "source": [
411 | "model = LinearRegression()\n",
412 | "model.fit(X_train, y_train)"
413 | ]
414 | },
415 | {
416 | "cell_type": "code",
417 | "execution_count": null,
418 | "id": "f3d56e91",
419 | "metadata": {},
420 | "outputs": [],
421 | "source": [
422 | "# Obtain the predicted value as y_hat\n",
423 | "y_hat_test = model.predict(X_test)\n",
424 | "y_hat_train = model.predict(X_train) # predicted value"
425 | ]
426 | },
427 | {
428 | "cell_type": "code",
429 | "execution_count": null,
430 | "id": "037ca7ca",
431 | "metadata": {},
432 | "outputs": [],
433 | "source": [
434 | "plt.scatter(y_test, y_hat_test, label=\"Corrected\", alpha=0.3)\n",
435 | "plt.scatter(y_test, test_data[[\"pm_cs\"]], label=\"Not corrected\", alpha=0.3)\n",
436 | "plt.xlabel(\"Airnow\")\n",
437 | "plt.ylabel(\"CS\")\n",
438 | "plt.title(\"Test data evaluation plot\")\n",
439 | "plt.legend()\n",
440 | "plt.show()"
441 | ]
442 | },
443 | {
444 | "cell_type": "markdown",
445 | "id": "b5c2ffc8",
446 | "metadata": {},
447 | "source": [
448 | "**We should only have only one model coefficient here, so we can't actually compare features. This step will be useful once we start scaling into more features**"
449 | ]
450 | },
451 | {
452 | "cell_type": "markdown",
453 | "id": "0eb6ba40",
454 | "metadata": {},
455 | "source": [
456 | "**Now, let us evaluate and compare to baseline**"
457 | ]
458 | },
459 | {
460 | "cell_type": "code",
461 | "execution_count": null,
462 | "id": "685586ab",
463 | "metadata": {},
464 | "outputs": [],
465 | "source": [
466 | "model_train_rmse = evaluate_model(y_train, y_hat_train)\n",
467 | "model_test_rmse = evaluate_model(y_test, y_hat_test)"
468 | ]
469 | },
470 | {
471 | "cell_type": "code",
472 | "execution_count": null,
473 | "id": "4cde1f5a",
474 | "metadata": {},
475 | "outputs": [],
476 | "source": [
477 | "print(\n",
478 | " f\"Resulting training evaluation is: {model_train_rmse} compared to baseline which is: {baseline_train_rmse}\"\n",
479 | ")"
480 | ]
481 | },
482 | {
483 | "cell_type": "code",
484 | "execution_count": null,
485 | "id": "279da3e4",
486 | "metadata": {},
487 | "outputs": [],
488 | "source": [
489 | "print(\n",
490 | " f\"Resulting test evaluation is: {model_test_rmse} compared to baseline which is: {baseline_test_rmse}\"\n",
491 | ")"
492 | ]
493 | },
494 | {
495 | "cell_type": "markdown",
496 | "id": "2860a9aa",
497 | "metadata": {},
498 | "source": [
499 | "Our model reduces our error both in training and test compared to the baseline we obtained initially. This shows that we have already made meaningful progress using a linear model. It is a good start.\n",
500 | "\n",
501 | "Now, let us observe the errors for each location (disaggregated)."
502 | ]
503 | },
504 | {
505 | "cell_type": "code",
506 | "execution_count": null,
507 | "id": "715f740f",
508 | "metadata": {},
509 | "outputs": [],
510 | "source": [
511 | "model_disagg_train_metrics = get_disaggregated_metrics(training_data, y_hat_train)\n",
512 | "model_disagg_test_metrics = get_disaggregated_metrics(test_data, y_hat_test)"
513 | ]
514 | },
515 | {
516 | "cell_type": "code",
517 | "execution_count": null,
518 | "id": "78d6b08d",
519 | "metadata": {},
520 | "outputs": [],
521 | "source": [
522 | "print(\"Model training RMSE\")\n",
523 | "model_disagg_train_metrics[\"RMSE\"]"
524 | ]
525 | },
526 | {
527 | "cell_type": "code",
528 | "execution_count": null,
529 | "id": "a0d3926a",
530 | "metadata": {},
531 | "outputs": [],
532 | "source": [
533 | "print(\"Model test RMSE\")\n",
534 | "model_disagg_test_metrics[\"RMSE\"]"
535 | ]
536 | },
537 | {
538 | "cell_type": "markdown",
539 | "id": "a3bc0932",
540 | "metadata": {},
541 | "source": [
542 | "Our model reduces our RMSE error with respect to the baseline; we are correcting the low-cost sensor readings with about 50% reduction in error in `i25_denver` and 30% in `CAMP`. Data from these locations have not been seen by our model. There is some progress, right? See table below.\n",
543 | "\n",
544 | " | Location| Baseline RMSE| Linear model 1 RMSE|\n",
545 | " |---|---|---|\n",
546 | " |**Train**|---|---|\n",
547 | " |NJH | 4.36| 2.28|\n",
548 | " |i25_glo_1|6.67|3.97|\n",
549 | " |i25_glo_2|4.55|3.57|\n",
550 | " |i25_glo_3|5.41|3.65|\n",
551 | " |la_casa|6.06|2.86|\n",
552 | " |**Train**|---|---|\n",
553 | " |CAMP|2.35|1.64|\n",
554 | " |i25_denver|8.04|4.47|\n",
555 | " "
556 | ]
557 | },
558 | {
559 | "cell_type": "markdown",
560 | "id": "7fee35a8",
561 | "metadata": {},
562 | "source": [
563 | "# EXERCISE "
564 | ]
565 | },
566 | {
567 | "cell_type": "markdown",
568 | "id": "b20b3494",
569 | "metadata": {},
570 | "source": [
571 | "We need to repeat this process above for other feature sets defined in `model_features`.\n",
572 | "\n",
573 | "For faster iterations, you can use `run_training_eval` function to iterate over features. Take not that the `run_training_eval` function also takes an `èstimator` argument which defaults to `LinearRegressor()`. You can change the value passed to this argument to `RandomForestRegressor` (which has alrready been imported into this notebook). This way, you can try they random forest model. \n",
574 | "\n",
575 | "Start with exploring all the possible features defined in `model_features` for `LinearRegressor()` before proceeding to `RandomForestRegressor`.\n",
576 | "\n",
577 | "Find out what your best model is and deepen your exploration of this model using things like scatter plots for camparing corrected and not corrected `pm_cs`values. You model output is the corrected `pm_cs`value.\n",
578 | "\n",
579 | "You can also find the feature importances by using `.feature_importances_`on your trained model. If you apply `run_training_eval`to fit your model, you will find the trained model in by using the `model`on the output dictionary.\n",
580 | "\n",
581 | "You can then proceed to check for disaggregated errors for your best model."
582 | ]
583 | },
584 | {
585 | "cell_type": "code",
586 | "execution_count": null,
587 | "id": "83deb9ad",
588 | "metadata": {},
589 | "outputs": [],
590 | "source": []
591 | }
592 | ],
593 | "metadata": {
594 | "kernelspec": {
595 | "display_name": "Python 3",
596 | "language": "python",
597 | "name": "python3"
598 | },
599 | "language_info": {
600 | "codemirror_mode": {
601 | "name": "ipython",
602 | "version": 3
603 | },
604 | "file_extension": ".py",
605 | "mimetype": "text/x-python",
606 | "name": "python",
607 | "nbconvert_exporter": "python",
608 | "pygments_lexer": "ipython3",
609 | "version": "3.8.5"
610 | }
611 | },
612 | "nbformat": 4,
613 | "nbformat_minor": 5
614 | }
615 |
--------------------------------------------------------------------------------
/ANSWERS_LESSON_3_Modeling.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "markdown",
5 | "id": "e5f4ebad",
6 | "metadata": {},
7 | "source": [
8 | "Welcome to this lab session 3 on **Time series modeling for air pollution monitoring with a focus on the\n",
9 | "calibration of low-cost sensors.**\n",
10 | "\n",
11 | "This lab session is based on the data and methods provided in the study by [Ellen M. Considine et al](https://www.sciencedirect.com/science/article/pii/S0269749120365222).\n"
12 | ]
13 | },
14 | {
15 | "cell_type": "markdown",
16 | "id": "6523f461",
17 | "metadata": {},
18 | "source": [
19 | "In the notebook, we will focus on using a regression models (linear model and random forest) to correct the readings of low-cost (CS) sensors."
20 | ]
21 | },
22 | {
23 | "cell_type": "markdown",
24 | "id": "5c1d5acf",
25 | "metadata": {},
26 | "source": [
27 | "We will start by presenting the use of a linear model for this purpose and we will explore different combinations of predictor variables in our model and compare the results. The idea here is to simulate a case of invastigating the best model and the features with which we get this model. \n",
28 | "\n",
29 | "In addition to a linear model, in our reference literature, a random forest model was also used. The follow-up task you will get in this session is to apply a random forest model and observe the results."
30 | ]
31 | },
32 | {
33 | "cell_type": "markdown",
34 | "id": "899192a7",
35 | "metadata": {},
36 | "source": [
37 | "# First, lets import the libraries we will be using"
38 | ]
39 | },
40 | {
41 | "cell_type": "code",
42 | "execution_count": null,
43 | "id": "6906a9d3",
44 | "metadata": {},
45 | "outputs": [],
46 | "source": [
47 | "import math\n",
48 | "from typing import List, Optional\n",
49 | "\n",
50 | "import pandas as pd\n",
51 | "\n",
52 | "import numpy as np\n",
53 | "import matplotlib.pyplot as plt\n",
54 | "\n",
55 | "from sklearn.linear_model import LinearRegression\n",
56 | "from sklearn.ensemble import RandomForestRegressor\n",
57 | "from sklearn.metrics import mean_squared_error"
58 | ]
59 | },
60 | {
61 | "cell_type": "code",
62 | "execution_count": null,
63 | "id": "d20752b8",
64 | "metadata": {},
65 | "outputs": [],
66 | "source": [
67 | "import warnings\n",
68 | "\n",
69 | "warnings.filterwarnings(\"ignore\")"
70 | ]
71 | },
72 | {
73 | "cell_type": "markdown",
74 | "id": "94d171c9",
75 | "metadata": {},
76 | "source": [
77 | "# Load the data"
78 | ]
79 | },
80 | {
81 | "cell_type": "code",
82 | "execution_count": null,
83 | "id": "ce38c6f8",
84 | "metadata": {},
85 | "outputs": [],
86 | "source": [
87 | "data_root = \"./data/\"\n",
88 | "training_data_path = data_root + \"cleaned_training.csv\"\n",
89 | "test_data_path = data_root + \"cleaned_test.csv\""
90 | ]
91 | },
92 | {
93 | "cell_type": "code",
94 | "execution_count": null,
95 | "id": "b5ca0666",
96 | "metadata": {},
97 | "outputs": [],
98 | "source": [
99 | "training_data = pd.read_csv(training_data_path)\n",
100 | "test_data = pd.read_csv(test_data_path)"
101 | ]
102 | },
103 | {
104 | "cell_type": "code",
105 | "execution_count": null,
106 | "id": "25da4f62",
107 | "metadata": {},
108 | "outputs": [],
109 | "source": [
110 | "training_data.head()"
111 | ]
112 | },
113 | {
114 | "cell_type": "markdown",
115 | "id": "71d227d3",
116 | "metadata": {},
117 | "source": [
118 | "# Utility functions"
119 | ]
120 | },
121 | {
122 | "cell_type": "code",
123 | "execution_count": null,
124 | "id": "86a6d56b",
125 | "metadata": {},
126 | "outputs": [],
127 | "source": [
128 | "# More evaluation metrics can be added to the function\n",
129 | "def evaluate_model(y, y_hat):\n",
130 | " evaluation = {\"RMSE\": round(mean_squared_error(y, y_hat, squared=False), 2)}\n",
131 | " return evaluation\n",
132 | "\n",
133 | "\n",
134 | "# To disaggregate the error metrics to the different locations\n",
135 | "def get_disaggregated_metrics(data: \"dataframe\", y_hat: Optional[List] = None):\n",
136 | "\n",
137 | " \"\"\"\n",
138 | " Disaggregate evaluation metrics into locations\n",
139 | " \"\"\"\n",
140 | " results = {}\n",
141 | "\n",
142 | " for sensor in data[\"cs_sensor\"].unique():\n",
143 | " data_sensor_idxs = data[data[\"cs_sensor\"] == sensor].index\n",
144 | " true_data = data[[\"pm_airnow\"]][data.index.isin(data_sensor_idxs)]\n",
145 | " pred_data = (\n",
146 | " data[[\"pm_cs\"]][data.index.isin(data_sensor_idxs)]\n",
147 | " if y_hat is None\n",
148 | " else y_hat[data_sensor_idxs]\n",
149 | " )\n",
150 | "\n",
151 | " eval_metrics = evaluate_model(true_data, pred_data)\n",
152 | " results[sensor] = eval_metrics\n",
153 | "\n",
154 | " return {\"RMSE\": results}\n",
155 | "\n",
156 | "\n",
157 | "def run_training_eval(\n",
158 | " features: dict,\n",
159 | " train: \"dataframe\" = training_data,\n",
160 | " test: \"dataframe\" = test_data,\n",
161 | " estimator: \"sklearn model\" = LinearRegression(),\n",
162 | "):\n",
163 | "\n",
164 | " \"\"\"\n",
165 | " Run training and evaluation on the data\n",
166 | "\n",
167 | " Purpose: fast experiment iteration\n",
168 | " \"\"\"\n",
169 | " y_train = train[[\"pm_airnow\"]]\n",
170 | " X_train = training_data[features]\n",
171 | "\n",
172 | " y_test = test[[\"pm_airnow\"]]\n",
173 | " X_test = test[features]\n",
174 | " \n",
175 | " model = estimator\n",
176 | " model.fit(X_train, y_train)\n",
177 | " y_hat_test = model.predict(X_test)\n",
178 | " y_hat_train = model.predict(X_train) # predicted value\n",
179 | "\n",
180 | " train_eval = evaluate_model(y_train, y_hat_train)\n",
181 | " test_eval = evaluate_model(y_test, y_hat_test)\n",
182 | "\n",
183 | " return {\n",
184 | " \"train_eval\": train_eval,\n",
185 | " \"test_eval\": test_eval,\n",
186 | " \"model\": model,\n",
187 | " \"y_hat_train\": y_hat_train,\n",
188 | " \"y_hat_test\": y_hat_test,\n",
189 | " }"
190 | ]
191 | },
192 | {
193 | "cell_type": "markdown",
194 | "id": "67edf58f",
195 | "metadata": {},
196 | "source": [
197 | "# How do we know we have a useful model? We need a baseline"
198 | ]
199 | },
200 | {
201 | "cell_type": "markdown",
202 | "id": "bb07d072",
203 | "metadata": {},
204 | "source": [
205 | "Before we proceed into deriving correction models, we need to decide how we intend to evaluate our models. \n",
206 | "\n",
207 | "Our base study applied the coefficient of determination ($R^2$) and root-mean squared error (RMSE). \n",
208 | "\n",
209 | "RMSE shows us the mean distance between the predicted and actual values.\n",
210 | "\n",
211 | "For this notebook, we will be using RMSE measure which has been defined as part of our utility functions. Lower RMSE values indicate more accurate models."
212 | ]
213 | },
214 | {
215 | "cell_type": "markdown",
216 | "id": "84efc6e5",
217 | "metadata": {},
218 | "source": [
219 | "We will obtain our baseline RMSE by computing the training and test RMSE between `pm_airnow` and `pm_cs`. Any model that will be useful whatsoever should do significantly better (i.e lower) than this baseline."
220 | ]
221 | },
222 | {
223 | "cell_type": "code",
224 | "execution_count": null,
225 | "id": "70f2fa7b",
226 | "metadata": {},
227 | "outputs": [],
228 | "source": [
229 | "baseline_train_rmse = evaluate_model(training_data[\"pm_airnow\"], training_data[\"pm_cs\"])\n",
230 | "baseline_test_rmse = evaluate_model(test_data[\"pm_airnow\"], test_data[\"pm_cs\"])"
231 | ]
232 | },
233 | {
234 | "cell_type": "code",
235 | "execution_count": null,
236 | "id": "7ead7988",
237 | "metadata": {},
238 | "outputs": [],
239 | "source": [
240 | "print(f\"Baseline training evaluation is: {baseline_train_rmse}\")\n",
241 | "print(f\"Baseline test evaluation is: {baseline_test_rmse}\")"
242 | ]
243 | },
244 | {
245 | "cell_type": "markdown",
246 | "id": "d4618f0f",
247 | "metadata": {},
248 | "source": [
249 | "It is clear to us now that any model that learns something from our data should produce less than `5.51` and `7.08`RMSE over the training and test set, respectively. Please note that RMSE is on the scale of our data, meaning it is measured in $µg/m^3$."
250 | ]
251 | },
252 | {
253 | "cell_type": "markdown",
254 | "id": "32f26ddc",
255 | "metadata": {},
256 | "source": [
257 | "# Can we disaggregate these errors to the different sensors/locations?\n",
258 | "\n",
259 | "Starting from the baseline"
260 | ]
261 | },
262 | {
263 | "cell_type": "code",
264 | "execution_count": null,
265 | "id": "25de818c",
266 | "metadata": {},
267 | "outputs": [],
268 | "source": [
269 | "baseline_disagg_train_metrics = get_disaggregated_metrics(training_data)"
270 | ]
271 | },
272 | {
273 | "cell_type": "code",
274 | "execution_count": null,
275 | "id": "a8cfcf8f",
276 | "metadata": {},
277 | "outputs": [],
278 | "source": [
279 | "print(\"Disaggregated RMSE score\")\n",
280 | "baseline_disagg_train_metrics[\"RMSE\"]"
281 | ]
282 | },
283 | {
284 | "cell_type": "code",
285 | "execution_count": null,
286 | "id": "5608840a",
287 | "metadata": {},
288 | "outputs": [],
289 | "source": [
290 | "baseline_disagg_test_metrics = get_disaggregated_metrics(test_data)"
291 | ]
292 | },
293 | {
294 | "cell_type": "code",
295 | "execution_count": null,
296 | "id": "5d642f72",
297 | "metadata": {},
298 | "outputs": [],
299 | "source": [
300 | "print(\"Disaggregated test RMSE\")\n",
301 | "baseline_disagg_test_metrics[\"RMSE\"]"
302 | ]
303 | },
304 | {
305 | "cell_type": "markdown",
306 | "id": "9a893283",
307 | "metadata": {},
308 | "source": [
309 | "Again, any model that will be generally useful should do less than these errors in each of these locations (training and test)"
310 | ]
311 | },
312 | {
313 | "cell_type": "markdown",
314 | "id": "c8d703fb",
315 | "metadata": {},
316 | "source": [
317 | "# Obtaining our models"
318 | ]
319 | },
320 | {
321 | "cell_type": "markdown",
322 | "id": "e2c371ca",
323 | "metadata": {},
324 | "source": [
325 | "**model_1**: CS PM2.5
\n",
326 | "**model_2**: CS PM2.5, temperature, humidity
\n",
327 | "**model_3**: CS PM2.5, temprature, humidity, road length
\n",
328 | "**model_4**: CS PM2.5, temprature, humidity, road length, hour, month
\n",
329 | "**model_5**: CS PM2.5, temprature, humidity, road length, hour, mont, weekend
"
330 | ]
331 | },
332 | {
333 | "cell_type": "code",
334 | "execution_count": null,
335 | "id": "41bf900b",
336 | "metadata": {},
337 | "outputs": [],
338 | "source": [
339 | "model_features = {\n",
340 | " \"model_1\": [\"pm_cs\"],\n",
341 | " \"model_2\": [\"pm_cs\", \"temp\", \"humidity\"],\n",
342 | " \"model_3\": [\"pm_cs\", \"temp\", \"humidity\", \"a_road_500\"],\n",
343 | " \"model_4\": [\n",
344 | " \"pm_cs\",\n",
345 | " \"temp\",\n",
346 | " \"humidity\",\n",
347 | " \"a_road_500\",\n",
348 | " \"sin_time\",\n",
349 | " \"cos_time\",\n",
350 | " \"sin_month\",\n",
351 | " \"cos_month\",\n",
352 | " ],\n",
353 | " \"model_5\": [\n",
354 | " \"pm_cs\",\n",
355 | " \"temp\",\n",
356 | " \"humidity\",\n",
357 | " \"a_road_500\",\n",
358 | " \"sin_time\",\n",
359 | " \"cos_time\",\n",
360 | " \"sin_month\",\n",
361 | " \"cos_month\",\n",
362 | " \"weekend\",\n",
363 | " ],\n",
364 | "}"
365 | ]
366 | },
367 | {
368 | "cell_type": "markdown",
369 | "id": "5da1fd60",
370 | "metadata": {},
371 | "source": [
372 | "###### Let us run our first example using only one feature `pm_cs` "
373 | ]
374 | },
375 | {
376 | "cell_type": "markdown",
377 | "id": "de32f4f2",
378 | "metadata": {},
379 | "source": [
380 | "We need to do the following:\n",
381 | "- Choose the features that we will be using to fit our model\n",
382 | "- Extract `x_train, x_test, y_train, y_test`, as features and reponse variables for training and test.\n",
383 | "- Fit the model\n",
384 | "- Predict\n",
385 | "- Evaluate: RMSE and scatter plot"
386 | ]
387 | },
388 | {
389 | "cell_type": "code",
390 | "execution_count": null,
391 | "id": "42795d99",
392 | "metadata": {},
393 | "outputs": [],
394 | "source": [
395 | "features = model_features[\"model_1\"]\n",
396 | "\n",
397 | "y_train = training_data[[\"pm_airnow\"]]\n",
398 | "X_train = training_data[features]\n",
399 | "\n",
400 | "y_test = test_data[[\"pm_airnow\"]]\n",
401 | "X_test = test_data[features]"
402 | ]
403 | },
404 | {
405 | "cell_type": "code",
406 | "execution_count": null,
407 | "id": "65877a5d",
408 | "metadata": {},
409 | "outputs": [],
410 | "source": [
411 | "model = LinearRegression()\n",
412 | "model.fit(X_train, y_train)"
413 | ]
414 | },
415 | {
416 | "cell_type": "code",
417 | "execution_count": null,
418 | "id": "be0e711a",
419 | "metadata": {},
420 | "outputs": [],
421 | "source": [
422 | "# Obtain the predicted value as y_hat\n",
423 | "y_hat_test = model.predict(X_test)\n",
424 | "y_hat_train = model.predict(X_train) # predicted value"
425 | ]
426 | },
427 | {
428 | "cell_type": "code",
429 | "execution_count": null,
430 | "id": "4a344884",
431 | "metadata": {},
432 | "outputs": [],
433 | "source": [
434 | "plt.scatter(y_test, y_hat_test, label=\"Corrected\", alpha=0.3)\n",
435 | "plt.scatter(y_test, test_data[[\"pm_cs\"]], label=\"Not corrected\", alpha=0.3)\n",
436 | "plt.xlabel(\"Airnow\")\n",
437 | "plt.ylabel(\"CS\")\n",
438 | "plt.title(\"Test data evaluation plot\")\n",
439 | "plt.legend()\n",
440 | "plt.show()"
441 | ]
442 | },
443 | {
444 | "cell_type": "markdown",
445 | "id": "55783d95",
446 | "metadata": {},
447 | "source": [
448 | "**We should only have only one model coefficient here, so we can't actually compare features. This step will be useful once we start scaling into more features**"
449 | ]
450 | },
451 | {
452 | "cell_type": "markdown",
453 | "id": "5925ffaf",
454 | "metadata": {},
455 | "source": [
456 | "**Now, let us evaluate and compare to baseline**"
457 | ]
458 | },
459 | {
460 | "cell_type": "code",
461 | "execution_count": null,
462 | "id": "ed201324",
463 | "metadata": {},
464 | "outputs": [],
465 | "source": [
466 | "model_train_rmse = evaluate_model(y_train, y_hat_train)\n",
467 | "model_test_rmse = evaluate_model(y_test, y_hat_test)"
468 | ]
469 | },
470 | {
471 | "cell_type": "code",
472 | "execution_count": null,
473 | "id": "2fbee9a4",
474 | "metadata": {},
475 | "outputs": [],
476 | "source": [
477 | "print(\n",
478 | " f\"Resulting training evaluation is: {model_train_rmse} compared to baseline which is: {baseline_train_rmse}\"\n",
479 | ")"
480 | ]
481 | },
482 | {
483 | "cell_type": "code",
484 | "execution_count": null,
485 | "id": "8b2514d7",
486 | "metadata": {},
487 | "outputs": [],
488 | "source": [
489 | "print(\n",
490 | " f\"Resulting test evaluation is: {model_test_rmse} compared to baseline which is: {baseline_test_rmse}\"\n",
491 | ")"
492 | ]
493 | },
494 | {
495 | "cell_type": "markdown",
496 | "id": "0a45dbc7",
497 | "metadata": {},
498 | "source": [
499 | "Our model reduces our error both in training and test compared to the baseline we obtained initially. This shows that we have already made meaningful progress using a linear model. It is a good start.\n",
500 | "\n",
501 | "Now, let us observe the errors for each location (disaggregated)."
502 | ]
503 | },
504 | {
505 | "cell_type": "code",
506 | "execution_count": null,
507 | "id": "9b715542",
508 | "metadata": {},
509 | "outputs": [],
510 | "source": [
511 | "model_disagg_train_metrics = get_disaggregated_metrics(training_data, y_hat_train)\n",
512 | "model_disagg_test_metrics = get_disaggregated_metrics(test_data, y_hat_test)"
513 | ]
514 | },
515 | {
516 | "cell_type": "code",
517 | "execution_count": null,
518 | "id": "84325536",
519 | "metadata": {},
520 | "outputs": [],
521 | "source": [
522 | "print(\"Model training RMSE\")\n",
523 | "model_disagg_train_metrics[\"RMSE\"]"
524 | ]
525 | },
526 | {
527 | "cell_type": "code",
528 | "execution_count": null,
529 | "id": "58ffdaa5",
530 | "metadata": {},
531 | "outputs": [],
532 | "source": [
533 | "print(\"Model test RMSE\")\n",
534 | "model_disagg_test_metrics[\"RMSE\"]"
535 | ]
536 | },
537 | {
538 | "cell_type": "markdown",
539 | "id": "e53a8ccd",
540 | "metadata": {},
541 | "source": [
542 | "Our model reduces our RMSE error with respect to the baseline; we are correcting the low-cost sensor readings with about 50% reduction in error in `i25_denver` and 30% in `CAMP`. Data from these locations have not been seen by our model. There is some progress, right? See table below.\n",
543 | "\n",
544 | " | Location| Baseline RMSE| Linear model 1 RMSE|\n",
545 | " |---|---|---|\n",
546 | " |**Train**|---|---|\n",
547 | " |NJH | 4.36| 2.28|\n",
548 | " |i25_glo_1|6.67|3.97|\n",
549 | " |i25_glo_2|4.55|3.57|\n",
550 | " |i25_glo_3|5.41|3.65|\n",
551 | " |la_casa|6.06|2.86|\n",
552 | " |**Train**|---|---|\n",
553 | " |CAMP|2.35|1.64|\n",
554 | " |i25_denver|8.04|4.47|\n",
555 | " "
556 | ]
557 | },
558 | {
559 | "cell_type": "markdown",
560 | "id": "d8c0eafd",
561 | "metadata": {},
562 | "source": [
563 | "# EXERCISE"
564 | ]
565 | },
566 | {
567 | "cell_type": "markdown",
568 | "id": "7f898479",
569 | "metadata": {},
570 | "source": [
571 | "We need to repeat this process above for other feature sets defined in `model_features`.\n",
572 | "\n",
573 | "For faster iterations, you can use `run_training_eval` function to iterate over features. Take not that the `run_training_eval` function also takes an `èstimator` argument which defaults to `LinearRegressor()`. You can change the value passed to this argument to `RandomForestRegressor` (which has alrready been imported into this notebook). This way, you can try they random forest model. \n",
574 | "\n",
575 | "Start with exploring all the possible features defined in `model_features` for `LinearRegressor()` before proceeding to `RandomForestRegressor`.\n",
576 | "\n",
577 | "Find out what your best model is and deepen your exploration of this model using things like scatter plots for camparing corrected and not corrected `pm_cs`values. You model output is the corrected `pm_cs`value.\n",
578 | "\n",
579 | "You can also find the feature importances by using `.feature_importances_`on your trained model. If you apply `run_training_eval`to fit your model, you will find the trained model in by using the `model`on the output dictionary.\n",
580 | "\n",
581 | "You can then proceed to check for disaggregated errors for your best model."
582 | ]
583 | },
584 | {
585 | "cell_type": "code",
586 | "execution_count": null,
587 | "id": "18fc8215",
588 | "metadata": {},
589 | "outputs": [],
590 | "source": [
591 | "print(\"Linear model\")\n",
592 | "for features in model_features:\n",
593 | " results = run_training_eval(model_features[features])\n",
594 | " print(f\"Results for {features}\")\n",
595 | " print(\"========================\")\n",
596 | " print({\"Train\": results[\"train_eval\"]})\n",
597 | " print({\"Test\": results[\"test_eval\"]})"
598 | ]
599 | },
600 | {
601 | "cell_type": "code",
602 | "execution_count": null,
603 | "id": "d392b961",
604 | "metadata": {},
605 | "outputs": [],
606 | "source": [
607 | "from sklearn.ensemble import RandomForestRegressor"
608 | ]
609 | },
610 | {
611 | "cell_type": "code",
612 | "execution_count": null,
613 | "id": "0c255f26",
614 | "metadata": {},
615 | "outputs": [],
616 | "source": [
617 | "print(\"Random Forest\")\n",
618 | "\n",
619 | "for features in model_features:\n",
620 | " results = run_training_eval(model_features[features], model=RandomForestRegressor())\n",
621 | " print(f\"Results for {features}\")\n",
622 | " print(\"========================\")\n",
623 | " print({\"Train\": results[\"train_eval\"]})\n",
624 | " print({\"Test\": results[\"test_eval\"]})"
625 | ]
626 | },
627 | {
628 | "cell_type": "markdown",
629 | "id": "e19e8fd7",
630 | "metadata": {},
631 | "source": [
632 | "The best model obtained is from based on random forest using `[\"pm_cs\", \"temp\", \"humidity\", \"a_road_500\", \"sin_time\" \"cos_time\", \"sin_month\", \"cos_month\"]`, i.e `model_4`\n",
633 | "\n",
634 | "The result for our best model is:\n",
635 | "\n",
636 | "`train RMSE= 0.85`
\n",
637 | "`test RMSE= 3.46`"
638 | ]
639 | },
640 | {
641 | "cell_type": "markdown",
642 | "id": "1588c366",
643 | "metadata": {},
644 | "source": [
645 | "###### Now, let us deepen our best model"
646 | ]
647 | },
648 | {
649 | "cell_type": "code",
650 | "execution_count": null,
651 | "id": "13719d03",
652 | "metadata": {},
653 | "outputs": [],
654 | "source": [
655 | "results_best_model = run_training_eval(\n",
656 | " model_features[\"model_4\"], model=RandomForestRegressor()\n",
657 | ")"
658 | ]
659 | },
660 | {
661 | "cell_type": "code",
662 | "execution_count": null,
663 | "id": "4d0b62e8",
664 | "metadata": {},
665 | "outputs": [],
666 | "source": [
667 | "plt.scatter(\n",
668 | " test_data[\"pm_airnow\"],\n",
669 | " results_best_model[\"y_hat_test\"],\n",
670 | " alpha=0.4,\n",
671 | " label=\"Corrected\",\n",
672 | ")\n",
673 | "plt.scatter(\n",
674 | " test_data[\"pm_airnow\"], test_data[\"pm_cs\"], alpha=0.4, label=\"Not corrected\"\n",
675 | ")\n",
676 | "plt.ylabel(\"AirNow\")\n",
677 | "plt.xlabel(\"CS\")\n",
678 | "plt.title(\"Test prediction from best model\")\n",
679 | "plt.legend()\n",
680 | "plt.show()"
681 | ]
682 | },
683 | {
684 | "cell_type": "markdown",
685 | "id": "47284e8a",
686 | "metadata": {},
687 | "source": [
688 | "###### Now, we should check the feature importances"
689 | ]
690 | },
691 | {
692 | "cell_type": "code",
693 | "execution_count": null,
694 | "id": "1e59a5d4",
695 | "metadata": {},
696 | "outputs": [],
697 | "source": [
698 | "feature_importances = results_best_model[\"model\"].feature_importances_"
699 | ]
700 | },
701 | {
702 | "cell_type": "code",
703 | "execution_count": null,
704 | "id": "50812096",
705 | "metadata": {},
706 | "outputs": [],
707 | "source": [
708 | "fig = plt.figure(figsize=(9, 4))\n",
709 | "plt.bar(model_features[\"model_4\"], feature_importances)\n",
710 | "plt.show()"
711 | ]
712 | },
713 | {
714 | "cell_type": "markdown",
715 | "id": "41f562a2",
716 | "metadata": {},
717 | "source": [
718 | "The most important feature for our random forest correction model is the `PM 2.5`readings from the low-cost sensor"
719 | ]
720 | },
721 | {
722 | "cell_type": "markdown",
723 | "id": "884ed3bb",
724 | "metadata": {},
725 | "source": [
726 | "**Disaggregating the errors into the different locations**"
727 | ]
728 | },
729 | {
730 | "cell_type": "code",
731 | "execution_count": null,
732 | "id": "d05208d7",
733 | "metadata": {},
734 | "outputs": [],
735 | "source": [
736 | "best_disagg_test_metric = get_disaggregated_metrics(\n",
737 | " test_data, results_best_model[\"y_hat_test\"]\n",
738 | ")\n",
739 | "best_disagg_train_metric = get_disaggregated_metrics(\n",
740 | " training_data, results_best_model[\"y_hat_train\"]\n",
741 | ")"
742 | ]
743 | },
744 | {
745 | "cell_type": "code",
746 | "execution_count": null,
747 | "id": "e66f0f55",
748 | "metadata": {},
749 | "outputs": [],
750 | "source": [
751 | "print(\"best model\")\n",
752 | "print(best_disagg_test_metric[\"RMSE\"])\n",
753 | "print(\"baseline\")\n",
754 | "print(baseline_disagg_test_metrics[\"RMSE\"])"
755 | ]
756 | },
757 | {
758 | "cell_type": "markdown",
759 | "id": "b38b872c",
760 | "metadata": {},
761 | "source": [
762 | "Compare to our baseline for test data as shown below:"
763 | ]
764 | },
765 | {
766 | "cell_type": "code",
767 | "execution_count": null,
768 | "id": "8b64ed7e",
769 | "metadata": {},
770 | "outputs": [],
771 | "source": [
772 | "print(\"best model\")\n",
773 | "print(best_disagg_train_metric[\"RMSE\"])\n",
774 | "print(\"baseline\")\n",
775 | "print(baseline_disagg_train_metrics[\"RMSE\"])"
776 | ]
777 | },
778 | {
779 | "cell_type": "markdown",
780 | "id": "c3cbc31c",
781 | "metadata": {},
782 | "source": [
783 | "We have lower errors in NJH and CAMP which are locations with lowers IQRs. They are both farther from the expressway than the other locations. "
784 | ]
785 | },
786 | {
787 | "cell_type": "markdown",
788 | "id": "55b3c5ca",
789 | "metadata": {},
790 | "source": [
791 | "Best result compared to baseline:\n",
792 | "\n",
793 | "| Location| Baseline RMSE| Random forest RMSE|\n",
794 | " |---|---|---|\n",
795 | " |**Train**|---|---|\n",
796 | " |NJH | 4.36| 0.63|\n",
797 | " |i25_glo_1|6.67|1.01|\n",
798 | " |i25_glo_2|4.55|0.93|\n",
799 | " |i25_glo_3|5.41|0.86|\n",
800 | " |la_casa|6.06|0.68|\n",
801 | " |**Test**|---|---|\n",
802 | " |CAMP|2.35|1.64|\n",
803 | " |i25_denver|8.04|4.47|\n",
804 | " "
805 | ]
806 | },
807 | {
808 | "cell_type": "markdown",
809 | "id": "fd89204c",
810 | "metadata": {},
811 | "source": [
812 | "Remember that the training results are obtained from data and locations that have already been seen by the model. Hence we have results that are over-optimistic for those locations. The model we have so far, however, shows usefulness in that it reduces error on test data (which are locations that have not been seen by our our model).\n",
813 | "\n",
814 | "How do we objectively evaluate the performance of our model in locations that are in the training data?\n",
815 | "For this, we will apply LOOO cross validation in the next notebook."
816 | ]
817 | },
818 | {
819 | "cell_type": "code",
820 | "execution_count": null,
821 | "id": "d57249ba",
822 | "metadata": {},
823 | "outputs": [],
824 | "source": []
825 | }
826 | ],
827 | "metadata": {
828 | "kernelspec": {
829 | "display_name": "Python 3",
830 | "language": "python",
831 | "name": "python3"
832 | },
833 | "language_info": {
834 | "codemirror_mode": {
835 | "name": "ipython",
836 | "version": 3
837 | },
838 | "file_extension": ".py",
839 | "mimetype": "text/x-python",
840 | "name": "python",
841 | "nbconvert_exporter": "python",
842 | "pygments_lexer": "ipython3",
843 | "version": "3.8.5"
844 | }
845 | },
846 | "nbformat": 4,
847 | "nbformat_minor": 5
848 | }
849 |
--------------------------------------------------------------------------------
/LESSON_2_Exploratory_data_analysis.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "markdown",
5 | "id": "fbffbe4c",
6 | "metadata": {},
7 | "source": [
8 | "Welcome to this lab session 2 on Time series modeling for air pollution monitoring with a focus on the\n",
9 | "calibration of low-cost sensors.\n",
10 | "\n",
11 | "This lab session is based on the data and methods provided in the study by [Ellen M. Considine et al](https://www.sciencedirect.com/science/article/pii/S0269749120365222)."
12 | ]
13 | },
14 | {
15 | "cell_type": "markdown",
16 | "id": "2d41f33d",
17 | "metadata": {},
18 | "source": [
19 | "In this notebook we perform exploratory data analysis on our cleaned data from session 1."
20 | ]
21 | },
22 | {
23 | "cell_type": "markdown",
24 | "id": "6e68c6b3",
25 | "metadata": {},
26 | "source": [
27 | "In statistics, exploratory data analysis is an approach of analyzing data sets to summarize their main characteristics, often using statistical graphics and other data visualization methods. - [Wikipedia](https://en.wikipedia.org/wiki/Exploratory_data_analysis)"
28 | ]
29 | },
30 | {
31 | "cell_type": "markdown",
32 | "id": "d6b94c2b",
33 | "metadata": {},
34 | "source": [
35 | "We will be asking ourselves the following questions:\n",
36 | " \n",
37 | "- What is the geospatial context of the locations of our sensors (and the data they collect)?\n",
38 | "- What is the length of arterial roads with 500 meter radius from each of the monitor locations?\n",
39 | "- What are the summary statistics of the airnow and CS PM2.5, and what can we observe from these statistics?\n",
40 | "- Using scatter plots and histograms to deepen our understanding of the possible disparities between airnow and CS readings. Here, we want to observe if there is a linear relationship between these readings.\n",
41 | "- What happens on weekends? Is there a significant difference?\n",
42 | "- Correlation matrix and scatter matrix between variables"
43 | ]
44 | },
45 | {
46 | "cell_type": "code",
47 | "execution_count": null,
48 | "id": "2cb09d65",
49 | "metadata": {},
50 | "outputs": [],
51 | "source": [
52 | "import pandas as pd\n",
53 | "import matplotlib.pyplot as plt\n",
54 | "import seaborn as sns\n",
55 | "import math\n",
56 | "\n",
57 | "import geopandas\n",
58 | "import contextily as cx"
59 | ]
60 | },
61 | {
62 | "cell_type": "markdown",
63 | "id": "f15ca84f",
64 | "metadata": {},
65 | "source": [
66 | "# Load the training data from disk\n",
67 | "\n",
68 | "The data is stored in a `.csv`file (with column names as keys to access the different time series variables). Pandas dataframes give use the possibility to manilpulate these kinds of files"
69 | ]
70 | },
71 | {
72 | "cell_type": "code",
73 | "execution_count": null,
74 | "id": "88dd741f",
75 | "metadata": {},
76 | "outputs": [],
77 | "source": [
78 | "data_root = \"./data/\"\n",
79 | "training_data_path = data_root + \"cleaned_training.csv\"\n",
80 | "test_data_path = data_root + \"cleaned_test.csv\"\n",
81 | "\n",
82 | "training_data = pd.read_csv(training_data_path)\n",
83 | "test_data = pd.read_csv(test_data_path)\n",
84 | "\n",
85 | "full_data = pd.concat([training_data, test_data], axis=0)"
86 | ]
87 | },
88 | {
89 | "cell_type": "code",
90 | "execution_count": null,
91 | "id": "5ba2c3b2",
92 | "metadata": {},
93 | "outputs": [],
94 | "source": [
95 | "full_data.dtypes"
96 | ]
97 | },
98 | {
99 | "cell_type": "code",
100 | "execution_count": null,
101 | "id": "76196171",
102 | "metadata": {},
103 | "outputs": [],
104 | "source": [
105 | "full_data.head()"
106 | ]
107 | },
108 | {
109 | "cell_type": "markdown",
110 | "id": "97cde018",
111 | "metadata": {},
112 | "source": [
113 | "Any NaN values?"
114 | ]
115 | },
116 | {
117 | "cell_type": "code",
118 | "execution_count": null,
119 | "id": "53b39ec5",
120 | "metadata": {},
121 | "outputs": [],
122 | "source": [
123 | "full_data.isna().sum()"
124 | ]
125 | },
126 | {
127 | "cell_type": "code",
128 | "execution_count": null,
129 | "id": "a1efe546",
130 | "metadata": {},
131 | "outputs": [],
132 | "source": [
133 | "full_data[\"date_time\"] = pd.to_datetime(full_data[\"date_time\"])"
134 | ]
135 | },
136 | {
137 | "cell_type": "markdown",
138 | "id": "b8c20ada",
139 | "metadata": {},
140 | "source": [
141 | "Now, we want to split our dataframe by sensor because we will be using these different sections of our data multiple times in this notebook."
142 | ]
143 | },
144 | {
145 | "cell_type": "markdown",
146 | "id": "7bf409bb",
147 | "metadata": {},
148 | "source": [
149 | "# 1. Plot latitude and langitude of each sensing location with basemap to provide context."
150 | ]
151 | },
152 | {
153 | "cell_type": "markdown",
154 | "id": "7a1edfc5",
155 | "metadata": {},
156 | "source": [
157 | "Here we will use a `groupby` function in pandas. It is used to group large amounts of data by descrete values contained in columns of the data."
158 | ]
159 | },
160 | {
161 | "cell_type": "markdown",
162 | "id": "f1fb5042",
163 | "metadata": {},
164 | "source": [
165 | "The `pandas.groupby.nth()` function is used to get the value corresponding the nth row for each group. To get the first value in a group, pass 0 as an argument to the `nth()` method. "
166 | ]
167 | },
168 | {
169 | "cell_type": "code",
170 | "execution_count": null,
171 | "id": "1ecfdf6c",
172 | "metadata": {},
173 | "outputs": [],
174 | "source": [
175 | "locations = (\n",
176 | " full_data[[\"airnow_sensor\", \"longitude\", \"latitude\"]]\n",
177 | " .groupby(by=\"airnow_sensor\")\n",
178 | " .nth(0)\n",
179 | " .reset_index()\n",
180 | ")"
181 | ]
182 | },
183 | {
184 | "cell_type": "code",
185 | "execution_count": null,
186 | "id": "f625f370",
187 | "metadata": {},
188 | "outputs": [],
189 | "source": [
190 | "locations"
191 | ]
192 | },
193 | {
194 | "cell_type": "markdown",
195 | "id": "55c804c3",
196 | "metadata": {},
197 | "source": [
198 | "The `contextily` package is used to retrieve web map tiles from a number of sources (OpenStreetMap, Stamen). We can use this package to add basemap to enhance `geopandas.GeoDataFrame`plots."
199 | ]
200 | },
201 | {
202 | "cell_type": "code",
203 | "execution_count": null,
204 | "id": "48a22a77",
205 | "metadata": {},
206 | "outputs": [],
207 | "source": [
208 | "geo_df = geopandas.GeoDataFrame(\n",
209 | " locations,\n",
210 | " geometry=geopandas.points_from_xy(locations.longitude, locations.latitude),\n",
211 | ")\n",
212 | "\n",
213 | "geo_df = geo_df.set_crs(\"epsg:4326\")"
214 | ]
215 | },
216 | {
217 | "cell_type": "code",
218 | "execution_count": null,
219 | "id": "7cbad40e",
220 | "metadata": {},
221 | "outputs": [],
222 | "source": [
223 | "geo_df"
224 | ]
225 | },
226 | {
227 | "cell_type": "code",
228 | "execution_count": null,
229 | "id": "6c3e83eb",
230 | "metadata": {},
231 | "outputs": [],
232 | "source": [
233 | "geo_df.dtypes"
234 | ]
235 | },
236 | {
237 | "cell_type": "code",
238 | "execution_count": null,
239 | "id": "9afad7b3",
240 | "metadata": {
241 | "scrolled": false
242 | },
243 | "outputs": [],
244 | "source": [
245 | "ax = geo_df.plot(\n",
246 | " figsize=(10, 10),\n",
247 | " alpha=1,\n",
248 | " edgecolor=\"b\",\n",
249 | " legend=True,\n",
250 | " markersize=500,\n",
251 | " legend_kwds={\"labels\": geo_df[\"airnow_sensor\"]},\n",
252 | ")\n",
253 | "\n",
254 | "for x, y, label in zip(geo_df.geometry.x, geo_df.geometry.y, geo_df[\"airnow_sensor\"]):\n",
255 | " ax.annotate(\n",
256 | " label,\n",
257 | " xy=(x, y),\n",
258 | " xytext=(3, 3),\n",
259 | " textcoords=\"offset points\",\n",
260 | " fontsize=25,\n",
261 | " color=\"r\",\n",
262 | " )\n",
263 | "plt.title(\"Locations of the AirNow (FEM) sensors used as reference in this study\")\n",
264 | "cx.add_basemap(ax, crs=geo_df.crs)"
265 | ]
266 | },
267 | {
268 | "cell_type": "markdown",
269 | "id": "75f65b4f",
270 | "metadata": {},
271 | "source": [
272 | "- The National Jewish Hospital (NJH) is farther away from the highway than the rest of the monitor locations. CAMP is also reltively far from large roads. This should have potential impact on how much and the variability of the PM2.5 atmospheric content sensed in these locations."
273 | ]
274 | },
275 | {
276 | "cell_type": "markdown",
277 | "id": "2732f829",
278 | "metadata": {},
279 | "source": [
280 | "# 2. Exploring road length variables\n",
281 | "\n",
282 | "Here we want to explore the lengths of arterial (Large) roads within 500m buffer surrounding each monitor location. We do this using a bar plot/chart.\n",
283 | "\n",
284 | "A bar chart or bar graph is a chart or graph that presents categorical data with rectangular bars with heights or lengths proportional to the values that they represent. - [Wikipedia](https://en.wikipedia.org/wiki/Bar_chart)"
285 | ]
286 | },
287 | {
288 | "cell_type": "code",
289 | "execution_count": null,
290 | "id": "6fecb997",
291 | "metadata": {},
292 | "outputs": [],
293 | "source": [
294 | "full_data_roads = (\n",
295 | " full_data.groupby(by=\"airnow_sensor\")\n",
296 | " .nth(0)\n",
297 | " .reset_index()[[\"airnow_sensor\", \"a_road_500\"]]\n",
298 | ")"
299 | ]
300 | },
301 | {
302 | "cell_type": "code",
303 | "execution_count": null,
304 | "id": "7d08dbba",
305 | "metadata": {},
306 | "outputs": [],
307 | "source": [
308 | "full_data_roads"
309 | ]
310 | },
311 | {
312 | "cell_type": "markdown",
313 | "id": "e60d0a19",
314 | "metadata": {},
315 | "source": [
316 | "### Exercise 1: Obtain a bar plot of the road length for each monitor location using the given dataframe\n",
317 | "\n",
318 | "Can you observe for \n",
319 | "\n",
320 | "**Follow up: What can you observe for from the plot?**\n",
321 | "\n",
322 | "**Follow up: What can you observe (differently) for training and test set from the plot?**\n",
323 | "\n",
324 | "Training: NJH, i25_glo, la_casa
\n",
325 | "Test: CAMP, i25_denver\n",
326 | "\n",
327 | "**Follow up: If you observed anything, how can we verify what you have observed?**"
328 | ]
329 | },
330 | {
331 | "cell_type": "code",
332 | "execution_count": null,
333 | "id": "2ab11b26",
334 | "metadata": {},
335 | "outputs": [],
336 | "source": [
337 | "plt.bar(full_data_roads[\"airnow_sensor\"], full_data_roads[\"a_road_500\"])\n",
338 | "plt.show()"
339 | ]
340 | },
341 | {
342 | "cell_type": "markdown",
343 | "id": "3fa73d67",
344 | "metadata": {},
345 | "source": [
346 | "# 3. Summary statistics of our data"
347 | ]
348 | },
349 | {
350 | "cell_type": "markdown",
351 | "id": "13a385c6",
352 | "metadata": {},
353 | "source": [
354 | "First, we obtain the summary statistics our \"true\" data, i.e the data from our monitor sensors (named `pm_airnow`) in our dataframe."
355 | ]
356 | },
357 | {
358 | "cell_type": "markdown",
359 | "id": "241c19d0",
360 | "metadata": {},
361 | "source": [
362 | "###### Training"
363 | ]
364 | },
365 | {
366 | "cell_type": "code",
367 | "execution_count": null,
368 | "id": "b7cf2b5e",
369 | "metadata": {},
370 | "outputs": [],
371 | "source": [
372 | "training_data.groupby(\"cs_sensor\")[[\"pm_airnow\"]].describe()"
373 | ]
374 | },
375 | {
376 | "cell_type": "markdown",
377 | "id": "bb731d0a",
378 | "metadata": {},
379 | "source": [
380 | "###### Test"
381 | ]
382 | },
383 | {
384 | "cell_type": "code",
385 | "execution_count": null,
386 | "id": "0f7abd25",
387 | "metadata": {},
388 | "outputs": [],
389 | "source": [
390 | "test_data.groupby(\"cs_sensor\")[[\"pm_airnow\"]].describe()"
391 | ]
392 | },
393 | {
394 | "cell_type": "markdown",
395 | "id": "7736b4af",
396 | "metadata": {},
397 | "source": [
398 | "**Observations**\n",
399 | "\n",
400 | "- NJH in training and CAMP in test data show relatively lower median values
\n",
401 | "- The IQR for both NJH and CAMP are also lower than the rest.\n"
402 | ]
403 | },
404 | {
405 | "cell_type": "markdown",
406 | "id": "dc6162c0",
407 | "metadata": {},
408 | "source": [
409 | "### Summary statistics of the CS (low-cost) sensor data?\n",
410 | "\n",
411 | "Remember, the column named `cs_sensor`provides the CS sensor tag for each data row and column named `pm_cs` contains the CA sensor data"
412 | ]
413 | },
414 | {
415 | "cell_type": "markdown",
416 | "id": "e79f50af",
417 | "metadata": {},
418 | "source": [
419 | "###### Training"
420 | ]
421 | },
422 | {
423 | "cell_type": "code",
424 | "execution_count": null,
425 | "id": "dbc1332d",
426 | "metadata": {},
427 | "outputs": [],
428 | "source": [
429 | "training_data.groupby(\"cs_sensor\")[[\"pm_cs\"]].describe()"
430 | ]
431 | },
432 | {
433 | "cell_type": "markdown",
434 | "id": "24c4efe0",
435 | "metadata": {},
436 | "source": [
437 | "Box plots show the median, quartiles ($Q_1$, $Q_2$, $Q_3$, $Q_4$), interquartile range (IQR), minimum, maximum, and outlier points.\n",
438 | "\n",
439 | "Low outlier points are defined as points below: $Q_1 − (1.5* IQR)$
\n",
440 | "High outlier points are defined as points above: $Q_3 + (1.5* IQR)$\n",
441 | "\n",
442 | "$50\\%$ of the lies between $Q_1$ and $Q_3$."
443 | ]
444 | },
445 | {
446 | "cell_type": "code",
447 | "execution_count": null,
448 | "id": "12b4f0f7",
449 | "metadata": {},
450 | "outputs": [],
451 | "source": [
452 | "training_data[[\"cs_sensor\", \"pm_cs\"]].groupby(\"cs_sensor\")[[\"pm_cs\"]].boxplot(\n",
453 | " subplots=False, figsize=(10, 7)\n",
454 | ")\n",
455 | "plt.show()"
456 | ]
457 | },
458 | {
459 | "cell_type": "markdown",
460 | "id": "724aeff9",
461 | "metadata": {},
462 | "source": [
463 | "###### Test"
464 | ]
465 | },
466 | {
467 | "cell_type": "code",
468 | "execution_count": null,
469 | "id": "8a789c11",
470 | "metadata": {},
471 | "outputs": [],
472 | "source": [
473 | "test_data.groupby(\"cs_sensor\")[[\"pm_cs\"]].describe()"
474 | ]
475 | },
476 | {
477 | "cell_type": "code",
478 | "execution_count": null,
479 | "id": "005ac1de",
480 | "metadata": {},
481 | "outputs": [],
482 | "source": [
483 | "test_data[[\"cs_sensor\", \"pm_cs\"]].groupby(\"cs_sensor\")[[\"pm_cs\"]].boxplot(\n",
484 | " subplots=False, figsize=(10, 7)\n",
485 | ")\n",
486 | "plt.show()"
487 | ]
488 | },
489 | {
490 | "cell_type": "markdown",
491 | "id": "ce04f870",
492 | "metadata": {},
493 | "source": [
494 | "- Take note of the IQR for CAMP and NJH. The IQR in these locations are lower than in the other locations. CAMP and NJH are farther away from the express way (see map)."
495 | ]
496 | },
497 | {
498 | "cell_type": "markdown",
499 | "id": "8c1d6016",
500 | "metadata": {},
501 | "source": [
502 | "# 4. Scatterplot comparing AirNow and CS PM2.5 "
503 | ]
504 | },
505 | {
506 | "cell_type": "markdown",
507 | "id": "89b52380",
508 | "metadata": {},
509 | "source": [
510 | "A scatterplot shows the relationship between two quantitative variables measure in the same unit/space"
511 | ]
512 | },
513 | {
514 | "cell_type": "code",
515 | "execution_count": null,
516 | "id": "73a3bbed",
517 | "metadata": {},
518 | "outputs": [],
519 | "source": [
520 | "plt.scatter(full_data[\"pm_airnow\"], full_data[\"pm_cs\"])\n",
521 | "plt.ylabel(\"PM2.5 - CS)\", size=15)\n",
522 | "plt.xlabel(\"PM2.5 - AirNow\", size=15)\n",
523 | "plt.title(\"Comparing AirNow and CS sensor \\n readings\", size=15)\n",
524 | "plt.axis(\"square\")\n",
525 | "plt.show()"
526 | ]
527 | },
528 | {
529 | "cell_type": "code",
530 | "execution_count": null,
531 | "id": "54cdb638",
532 | "metadata": {},
533 | "outputs": [],
534 | "source": [
535 | "ax = sns.regplot(\n",
536 | " x=full_data[\"pm_airnow\"],\n",
537 | " y=full_data[\"pm_cs\"],\n",
538 | " color=\"green\",\n",
539 | " line_kws={\"color\": \"black\"},\n",
540 | ")\n",
541 | "\n",
542 | "ax.set(xlabel=\"PM2.5 - CS\", ylabel=\"PM2.5 - AirNow\")\n",
543 | "plt.axis(\"square\")\n",
544 | "plt.show()"
545 | ]
546 | },
547 | {
548 | "cell_type": "markdown",
549 | "id": "7cc020f3",
550 | "metadata": {},
551 | "source": [
552 | "**What can we do with this plot?**
\n",
553 | "We can understand the relationship between our \"true\" PM2.5 values and the ones mesaured from our low-cost sensors\n"
554 | ]
555 | },
556 | {
557 | "cell_type": "markdown",
558 | "id": "2ed59dff",
559 | "metadata": {},
560 | "source": [
561 | "- The need for correction. A linear model is useful for this.\n",
562 | "- A linear model could find the line of best fit here. We might not even need data other confounding variables to have a good starting model."
563 | ]
564 | },
565 | {
566 | "cell_type": "markdown",
567 | "id": "6d4a5b2c",
568 | "metadata": {},
569 | "source": [
570 | "We can split our data by CS sensors to observe if there are any significant changes with respect to what has been observed over the whole data."
571 | ]
572 | },
573 | {
574 | "cell_type": "code",
575 | "execution_count": null,
576 | "id": "8e8ce65f",
577 | "metadata": {},
578 | "outputs": [],
579 | "source": [
580 | "njh_data = full_data[full_data[\"cs_sensor\"] == \"NJH\"]\n",
581 | "i25_glo1_data = full_data[full_data[\"cs_sensor\"] == \"i25_glo_1\"]\n",
582 | "i25_glo2_data = full_data[full_data[\"cs_sensor\"] == \"i25_glo_2\"]\n",
583 | "i25_glo3_data = full_data[full_data[\"cs_sensor\"] == \"i25_glo_3\"]\n",
584 | "lacasa_data = full_data[full_data[\"cs_sensor\"] == \"la_casa\"]\n",
585 | "camp_data = full_data[full_data[\"cs_sensor\"] == \"CAMP\"]\n",
586 | "i25_denver_data = full_data[full_data[\"cs_sensor\"] == \"i25_denver\"]"
587 | ]
588 | },
589 | {
590 | "cell_type": "code",
591 | "execution_count": null,
592 | "id": "bf29f0ec",
593 | "metadata": {},
594 | "outputs": [],
595 | "source": [
596 | "fig, [(ax1, ax2, ax3, ax4), (ax5, ax6, ax7, ax8)] = plt.subplots(\n",
597 | " nrows=2, ncols=4, figsize=(20, 5), sharex=True, sharey=True\n",
598 | ")\n",
599 | "\n",
600 | "ax1.scatter(njh_data[\"pm_airnow\"], njh_data[\"pm_cs\"])\n",
601 | "ax1.set_title(\"NJH\")\n",
602 | "\n",
603 | "ax2.scatter(i25_glo1_data[\"pm_airnow\"], i25_glo1_data[\"pm_cs\"])\n",
604 | "ax2.set_title(\"i-25 Globeville 1\")\n",
605 | "ax3.scatter(i25_glo2_data[\"pm_airnow\"], i25_glo2_data[\"pm_cs\"])\n",
606 | "ax3.set_title(\"i-25 Globeville 2\")\n",
607 | "ax4.scatter(i25_glo3_data[\"pm_airnow\"], i25_glo3_data[\"pm_cs\"])\n",
608 | "ax4.set_title(\"i-25 Globeville 3\")\n",
609 | "ax5.scatter(lacasa_data[\"pm_airnow\"], lacasa_data[\"pm_cs\"])\n",
610 | "ax5.set_title(\"La Casa\")\n",
611 | "ax6.scatter(camp_data[\"pm_airnow\"], camp_data[\"pm_cs\"])\n",
612 | "ax6.set_title(\"CAMP\")\n",
613 | "\n",
614 | "ax7.scatter(i25_denver_data[\"pm_airnow\"], i25_denver_data[\"pm_cs\"])\n",
615 | "ax7.set_title(\"Denver\")\n",
616 | "\n",
617 | "ax8.axis(\"off\")\n",
618 | "\n",
619 | "ax1.set_ylabel(\"Monitor sensors\", size=15)\n",
620 | "fig.text(0.5, 0.04, \"Low cost (CS) sensors\", ha=\"center\", va=\"center\", size=15)\n",
621 | "\n",
622 | "plt.show()"
623 | ]
624 | },
625 | {
626 | "cell_type": "markdown",
627 | "id": "e6c21e0b",
628 | "metadata": {},
629 | "source": [
630 | "What do we observe? Any differences across the different CS sensors\n",
631 | "\n",
632 | "- Some potential outlier measures at I-25 Globeville 1"
633 | ]
634 | },
635 | {
636 | "cell_type": "markdown",
637 | "id": "cdef8a08",
638 | "metadata": {},
639 | "source": [
640 | "# 4. Historgrams comparing AirNow and CS PM2.5 "
641 | ]
642 | },
643 | {
644 | "cell_type": "markdown",
645 | "id": "d3d5d359",
646 | "metadata": {},
647 | "source": [
648 | "What histograms can tell us about the ditribution of our data:\n",
649 | "\n",
650 | "- Is it unimodal, bimodal, or multimodal?\n",
651 | "- how widely is the distribution spread?\n",
652 | "- Do the distributions overlap for the different sensors?"
653 | ]
654 | },
655 | {
656 | "cell_type": "markdown",
657 | "id": "67aac336",
658 | "metadata": {},
659 | "source": [
660 | "###### Choosing bin size.\n",
661 | "\n",
662 | "One simple rule is [Sturge’s rule](https://www.researchgate.net/publication/230257056_Sturges'_rule)\n",
663 | "\n",
664 | "$K = 1 + 3.22 log(N)$\n",
665 | "\n",
666 | "where,\n",
667 | "\n",
668 | "K is the number of bins\n",
669 | "\n",
670 | "N is the number of observations"
671 | ]
672 | },
673 | {
674 | "cell_type": "code",
675 | "execution_count": null,
676 | "id": "e32e1129",
677 | "metadata": {},
678 | "outputs": [],
679 | "source": [
680 | "def bin_size(n_observations):\n",
681 | "\n",
682 | " return math.ceil(1 + (3.22 * math.log(n_observations)))"
683 | ]
684 | },
685 | {
686 | "cell_type": "code",
687 | "execution_count": null,
688 | "id": "c23a60fb",
689 | "metadata": {},
690 | "outputs": [],
691 | "source": [
692 | "plt.hist(\n",
693 | " [full_data[\"pm_airnow\"], full_data[\"pm_cs\"]],\n",
694 | " bins=bin_size(len(full_data)),\n",
695 | ")\n",
696 | "plt.xlabel(\"Value\", size=15)\n",
697 | "plt.ylabel(\"Frequency\", size=15)\n",
698 | "plt.legend()\n",
699 | "plt.show()"
700 | ]
701 | },
702 | {
703 | "cell_type": "code",
704 | "execution_count": null,
705 | "id": "4a709c75",
706 | "metadata": {},
707 | "outputs": [],
708 | "source": [
709 | "labels = [\"AirNow\", \"CS\"]\n",
710 | "fig, [(ax1, ax2, ax3, ax4), (ax5, ax6, ax7, ax8)] = plt.subplots(\n",
711 | " nrows=2, ncols=4, figsize=(20, 9)\n",
712 | ") # , sharex=True, sharey=True)\n",
713 | "\n",
714 | "ax1.hist(\n",
715 | " [njh_data[\"pm_airnow\"], njh_data[\"pm_cs\"]],\n",
716 | " bins=bin_size(len(njh_data)),\n",
717 | ")\n",
718 | "ax1.set_title(\"NJH\")\n",
719 | "\n",
720 | "\n",
721 | "ax2.hist(\n",
722 | " [i25_glo1_data[\"pm_airnow\"], i25_glo1_data[\"pm_cs\"]],\n",
723 | " bins=bin_size(len(i25_glo1_data)),\n",
724 | ")\n",
725 | "ax2.set_title(\"i-25 Globeville 1\")\n",
726 | "\n",
727 | "ax3.hist(\n",
728 | " [i25_glo2_data[\"pm_airnow\"], i25_glo2_data[\"pm_cs\"]],\n",
729 | " bins=bin_size(len(i25_glo2_data)),\n",
730 | ")\n",
731 | "ax3.set_title(\"i-25 Globeville 2\")\n",
732 | "\n",
733 | "ax4.hist(\n",
734 | " [i25_glo3_data[\"pm_airnow\"], i25_glo3_data[\"pm_cs\"]],\n",
735 | " bins=bin_size(len(i25_glo3_data)),\n",
736 | ")\n",
737 | "ax4.set_title(\"i-25 Globeville 3\")\n",
738 | "\n",
739 | "ax5.hist(\n",
740 | " [lacasa_data[\"pm_airnow\"], lacasa_data[\"pm_cs\"]],\n",
741 | " bins=bin_size(len(lacasa_data)),\n",
742 | ")\n",
743 | "ax5.set_title(\"La Casa\")\n",
744 | "\n",
745 | "\n",
746 | "ax6.hist(\n",
747 | " [camp_data[\"pm_airnow\"], camp_data[\"pm_cs\"]],\n",
748 | " bins=bin_size(len(lacasa_data)),\n",
749 | ")\n",
750 | "ax6.set_title(\"CAMP\")\n",
751 | "\n",
752 | "ax7.hist(\n",
753 | " [i25_denver_data[\"pm_airnow\"], i25_denver_data[\"pm_cs\"]],\n",
754 | " bins=bin_size(len(i25_denver_data)),\n",
755 | ")\n",
756 | "ax7.set_title(\"I-25 Denver\")\n",
757 | "\n",
758 | "ax8.axis(\"off\")\n",
759 | "\n",
760 | "ax1.set_ylabel(\"Probability\", size=15)\n",
761 | "fig.text(0.5, 0.04, \"Measured PM2.5 value\", ha=\"center\", va=\"center\", size=15)\n",
762 | "fig.legend(\n",
763 | " labels,\n",
764 | " loc=\"lower right\",\n",
765 | " bbox_to_anchor=(0.5, -0.04),\n",
766 | " ncol=len(labels),\n",
767 | " bbox_transform=fig.transFigure,\n",
768 | ")\n",
769 | "plt.show()"
770 | ]
771 | },
772 | {
773 | "cell_type": "markdown",
774 | "id": "5ead08a2",
775 | "metadata": {},
776 | "source": [
777 | "# What happens on weekends?"
778 | ]
779 | },
780 | {
781 | "cell_type": "code",
782 | "execution_count": null,
783 | "id": "20791f83",
784 | "metadata": {},
785 | "outputs": [],
786 | "source": [
787 | "plt.plot(camp_data[\"pm_airnow\"], label=\"AirNow\")\n",
788 | "plt.plot(camp_data[\"pm_cs\"], label=\"CS\")\n",
789 | "plt.bar(\n",
790 | " x=range(0, len(camp_data)),\n",
791 | " height=[x * 30 for x in camp_data[\"weekend\"]],\n",
792 | " alpha=0.2,\n",
793 | " label=\"weekend\",\n",
794 | ")\n",
795 | "\n",
796 | "plt.title(\"CAMP data - CS, Airnow and Bar plot show weekend days\")\n",
797 | "plt.legend()\n",
798 | "plt.show()"
799 | ]
800 | },
801 | {
802 | "cell_type": "code",
803 | "execution_count": null,
804 | "id": "b3fd0baf",
805 | "metadata": {},
806 | "outputs": [],
807 | "source": [
808 | "weekend_median = full_data[full_data[\"weekend\"] == 1][\"pm_airnow\"].median()\n",
809 | "weekday_median = full_data[full_data[\"weekend\"] != 1][\"pm_airnow\"].median()"
810 | ]
811 | },
812 | {
813 | "cell_type": "code",
814 | "execution_count": null,
815 | "id": "586ffd2e",
816 | "metadata": {},
817 | "outputs": [],
818 | "source": [
819 | "plt.bar(x=[\"week day\", \"weekend\"], height=[weekday_median, weekend_median])\n",
820 | "plt.show()"
821 | ]
822 | },
823 | {
824 | "cell_type": "markdown",
825 | "id": "979cde13",
826 | "metadata": {},
827 | "source": [
828 | "There is lower PM2.5 on weekends than on weekdays. This could be due to reduction in industrial activities, traffic on highways and other sources of pollution over the weekend"
829 | ]
830 | },
831 | {
832 | "cell_type": "code",
833 | "execution_count": null,
834 | "id": "726e3cf2",
835 | "metadata": {},
836 | "outputs": [],
837 | "source": [
838 | "corr = full_data[[\"pm_airnow\", \"pm_cs\", \"temp\", \"humidity\"]].corr()\n",
839 | "corr.style.background_gradient(cmap=\"coolwarm\")"
840 | ]
841 | },
842 | {
843 | "cell_type": "markdown",
844 | "id": "f367b18f",
845 | "metadata": {},
846 | "source": [
847 | "`pm_cs` and `pm_airnow` show high linear correlation of 0.89. "
848 | ]
849 | },
850 | {
851 | "cell_type": "code",
852 | "execution_count": null,
853 | "id": "f281d605",
854 | "metadata": {},
855 | "outputs": [],
856 | "source": [
857 | "pd.plotting.scatter_matrix(full_data[[\"pm_airnow\", \"pm_cs\", \"temp\", \"humidity\"]])\n",
858 | "plt.show()"
859 | ]
860 | },
861 | {
862 | "cell_type": "markdown",
863 | "id": "7b6fa33b",
864 | "metadata": {},
865 | "source": [
866 | "`pm_cs` and `pm_airnow` show high linear correlation. "
867 | ]
868 | }
869 | ],
870 | "metadata": {
871 | "kernelspec": {
872 | "display_name": "Python 3",
873 | "language": "python",
874 | "name": "python3"
875 | },
876 | "language_info": {
877 | "codemirror_mode": {
878 | "name": "ipython",
879 | "version": 3
880 | },
881 | "file_extension": ".py",
882 | "mimetype": "text/x-python",
883 | "name": "python",
884 | "nbconvert_exporter": "python",
885 | "pygments_lexer": "ipython3",
886 | "version": "3.8.5"
887 | }
888 | },
889 | "nbformat": 4,
890 | "nbformat_minor": 5
891 | }
892 |
--------------------------------------------------------------------------------
/LESSON_1_data_preparation.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "markdown",
5 | "id": "bab7336b",
6 | "metadata": {},
7 | "source": [
8 | "# Lab session taught by Oladimeji Mudele"
9 | ]
10 | },
11 | {
12 | "cell_type": "markdown",
13 | "id": "7ae33928",
14 | "metadata": {},
15 | "source": [
16 | "Welcome to this lab session on Time series modeling for air pollution monitoring with a focus on the\n",
17 | "calibration of low-cost sensors.\n",
18 | "\n",
19 | "This lab session is based on the data and methods provided in the study by [Ellen M. Considine et al](https://www.sciencedirect.com/science/article/pii/S0269749120365222).\n",
20 | "\n",
21 | "Rather than focus on code efficiency, software engineering practices, or theory of machine learning models, this course focuses on the **thought processes** (as generalised as possible) that apply to tackling many data science problems. Don't worry if you don't understanding certain nuances around syntax or code implementations. These problems are always easier to tackle. My advice is that you focus on the \"WHY?\".\n",
22 | "\n",
23 | "The interesting part of this course module is that, rather than work with cleaned data and do our explorations and modeling from there, we will try to simulate the real-life scenario of merging data from multple sources (sensors) and time periods, and also engineer the features we need for our prediction task.\n",
24 | "\n",
25 | "In certain events in data science, you might be lucky to have your data stuctured and prepared for you. Most often, this is not the case.\n",
26 | "\n",
27 | "Welcome to the world of data science."
28 | ]
29 | },
30 | {
31 | "cell_type": "markdown",
32 | "id": "8bb41868",
33 | "metadata": {},
34 | "source": [
35 | "In this notebook, we will prepare the training and test data. In ths process we will go through the following steps:\n",
36 | "\n",
37 | "- **Understading the problem**: This leads us to the question: \"**What data do I need?**\"\n",
38 | "- **Diving into the data**: This is simply looking at your data (each column), and asking yourself, what does it mean? Why do I need it? How would I use it for my task? What units are the data expressed in?\n",
39 | "- Aligning the different axis and features in the multi-source/multitemporal data\n",
40 | "- **Cleaning**: dealing with invalid and missing values. For example, in the case of our project, the low-cost Canary-S (CS) sensor readings register -1 as invalid values and the authors of our reference literature mentioned 1500 units of PM2.5 as the maximum threhold. We need to handle these cases. Also, we need to check for rows with missing values.\n",
41 | "- **Merging**: pull data from multiple sources into a single frame.\n",
42 | "- **Features crafting**: Based on our understanding of the problem we are trying to solve, it might be useful to take some of the information in th dsta and use them to craft features we believe will udeful for our model to learn what it needs to learn in other to produce the best results."
43 | ]
44 | },
45 | {
46 | "cell_type": "markdown",
47 | "id": "7dc52b44",
48 | "metadata": {},
49 | "source": [
50 | "# WHAT IS OUR PROBLEM?"
51 | ]
52 | },
53 | {
54 | "cell_type": "markdown",
55 | "id": "d90b46b6",
56 | "metadata": {},
57 | "source": [
58 | "We have PM2.5 readings from low cost sensors and we want to learn a model that maps them to \"true\" readings from reference FEM (AirNow) sensors . "
59 | ]
60 | },
61 | {
62 | "cell_type": "markdown",
63 | "id": "1bf57b60",
64 | "metadata": {},
65 | "source": [
66 | "# WHAT DATA DO WE NEED?"
67 | ]
68 | },
69 | {
70 | "cell_type": "markdown",
71 | "id": "14ec531b",
72 | "metadata": {},
73 | "source": [
74 | "##### The obvious and basic ones:\n",
75 | "- PM2.5 readings from AirNow sensors\n",
76 | "- PM2.5 readings from CS sensors\n",
77 | "\n",
78 | "##### What other data or features might we need? (Our experiments will determine what is eventually useful\n",
79 | "- lenght of road with certain radius from sensing location.\n",
80 | "- Time variables\n",
81 | "- temperature \n",
82 | "- humidity\n",
83 | "- day of the week (weekend or not)"
84 | ]
85 | },
86 | {
87 | "cell_type": "markdown",
88 | "id": "fd096f17",
89 | "metadata": {},
90 | "source": [
91 | "# NOW LET US DIVE INTO CODE"
92 | ]
93 | },
94 | {
95 | "cell_type": "markdown",
96 | "id": "326d3fff",
97 | "metadata": {},
98 | "source": [
99 | "We need some Python libraries to enable efficient implementations of computations. The libraries we need are `numpy`, `pandas`and the `math`module which is part of the python standard library. For now, please ensure to install both `numpy`and `pandas`."
100 | ]
101 | },
102 | {
103 | "cell_type": "markdown",
104 | "id": "22c8c8c3",
105 | "metadata": {},
106 | "source": [
107 | "Now, let us import these libraries into our workspace."
108 | ]
109 | },
110 | {
111 | "cell_type": "code",
112 | "execution_count": null,
113 | "id": "c0f01f4a",
114 | "metadata": {},
115 | "outputs": [],
116 | "source": [
117 | "import pandas as pd\n",
118 | "import numpy as np\n",
119 | "import math\n",
120 | "import matplotlib.pyplot as plt"
121 | ]
122 | },
123 | {
124 | "cell_type": "markdown",
125 | "id": "b8537f9a",
126 | "metadata": {},
127 | "source": [
128 | "Now, we are done with preparing ourselves for the data preparation phase. Here we go!"
129 | ]
130 | },
131 | {
132 | "cell_type": "markdown",
133 | "id": "14ef4dc4",
134 | "metadata": {},
135 | "source": [
136 | "# TRAINING DATA"
137 | ]
138 | },
139 | {
140 | "cell_type": "markdown",
141 | "id": "8cfa5ab7",
142 | "metadata": {},
143 | "source": [
144 | "### Load the data from disk and explain the details"
145 | ]
146 | },
147 | {
148 | "cell_type": "markdown",
149 | "id": "c4ea6ef2",
150 | "metadata": {},
151 | "source": [
152 | "Let us specify the subfolder in our project folder where you have kept all the data. "
153 | ]
154 | },
155 | {
156 | "cell_type": "code",
157 | "execution_count": null,
158 | "id": "39d14310",
159 | "metadata": {},
160 | "outputs": [],
161 | "source": [
162 | "data_root = \"./data/\" # change this to your own subfolder. Ideally, it should be the same as defined here."
163 | ]
164 | },
165 | {
166 | "cell_type": "markdown",
167 | "id": "f99c8e15",
168 | "metadata": {},
169 | "source": [
170 | "In the `data` subfolder, you will find a couple of csv files, the raw training data is in a file named `raw_collocated_data.csv`. Let us load that file."
171 | ]
172 | },
173 | {
174 | "cell_type": "code",
175 | "execution_count": null,
176 | "id": "1906b560",
177 | "metadata": {},
178 | "outputs": [],
179 | "source": [
180 | "training_data_path = (\n",
181 | " data_root + \"raw_collocated_data.csv\"\n",
182 | ") # first we define the file location using string concatenation"
183 | ]
184 | },
185 | {
186 | "cell_type": "code",
187 | "execution_count": null,
188 | "id": "e950f633",
189 | "metadata": {},
190 | "outputs": [],
191 | "source": [
192 | "raw_training_data = pd.read_csv(\n",
193 | " training_data_path\n",
194 | ") # Here we load the data as a dataframe"
195 | ]
196 | },
197 | {
198 | "cell_type": "markdown",
199 | "id": "b3ef6515",
200 | "metadata": {},
201 | "source": [
202 | "### So, what are the columns in our data and what do they mean?"
203 | ]
204 | },
205 | {
206 | "cell_type": "code",
207 | "execution_count": null,
208 | "id": "5174bae2",
209 | "metadata": {},
210 | "outputs": [],
211 | "source": [
212 | "raw_training_data.columns"
213 | ]
214 | },
215 | {
216 | "cell_type": "markdown",
217 | "id": "b0bd7a0d",
218 | "metadata": {},
219 | "source": [
220 | "It is important to ensure that you understandevery variable in the columns of your dataframe. In essense: \n",
221 | "\n",
222 | "Why do we need each of these variables (if at all)?\n",
223 | "\n",
224 | "In what way do we have to organise the data to make them most useful to us?"
225 | ]
226 | },
227 | {
228 | "cell_type": "markdown",
229 | "id": "5fc1a607",
230 | "metadata": {},
231 | "source": [
232 | "### Understanding the data content in our dataframe\n",
233 | "\n",
234 | "In summary, we need to look at each column and ask ourselves what information it contains, why and how we will need the information"
235 | ]
236 | },
237 | {
238 | "cell_type": "markdown",
239 | "id": "9e404e49",
240 | "metadata": {},
241 | "source": [
242 | "CS#: Canary sensor number #
\n",
243 | "AirNow: FEM reference data source\n",
244 | "\n",
245 | "\n",
246 | "`DateTime`: Date and time that data was registered by sensor\n",
247 | "
\n",
248 | "\n",
249 | "`PM2_5(NJH CS Collo (CS1)/Canary-S)` : PM2.5 readings from CS1 located at NJH
\n",
250 | "`tempf(NJH CS Collo (CS1)/Canary-S)` : Temperature (Farenheit) readings from CS2 located at NJH
\n",
251 | "`humidity(NJH CS Collo (CS1)/Canary-S)` : Humidity readings from CS1 located at NJH
\n",
252 | "`PM2.5 (NJH/AirNow)`: PM2.5 readings from the reference FEM sensor located at NJH\n",
253 | "
\n",
254 | "\n",
255 | "`PM2_5(I-25 Glo Collo (CS2)/Canary-S)` : PM2.5 readings from CS2 sensor located at I-25 Globeville
\n",
256 | "`tempf(I-25 Glo Collo (CS2)/Canary-S)` : Temperature (Farenheit) readings from CS2 located at I-25 Globeville
\n",
257 | "`humidity(I-25 Glo Collo (CS2)/Canary-S)` : Humidity readings from CS2 located at I-25 Globeville
\n",
258 | "
\n",
259 | "\n",
260 | "`PM2_5(I-25 Glo Collo (CS3)/Canary-S)` : PM2.5 readings from CS3 sensor located at I-25 Globeville
\n",
261 | "`tempf(I-25 Glo Collo (CS3)/Canary-S)` : Temperature (Farenheit) readings from CS3 located at I-25 Globeville
\n",
262 | "`humidity(I-25 Glo Collo (CS3)/Canary-S)` : Humidity readings from CS3 located at I-25 Globeville
\n",
263 | "
\n",
264 | "\n",
265 | "`PM2_5(I-25 Glo Collo (CS4)/Canary-S)` : PM2.5 readings from CS4 sensor located at I-25 Globeville
\n",
266 | "`tempf(I-25 Glo Collo (CS4)/Canary-S)` : Temperature (Farenheit) readings from CS4 located at I-25 Globeville
\n",
267 | "`humidity(I-25 Glo Collo (CS4)/Canary-S)` : Humidity readings from CS4 located at I-25 Globeville
\n",
268 | "`PM2.5(I-25 Globeville/AirNow)`: PM2.5 readings from the reference FEM sensor located at I-25 Globeville\n",
269 | "
\n",
270 | "\n",
271 | "`PM2_5(La Casa Collo (CS5)/Canary-S)` : PM2.5 readings from CS5 sensor located at La Casa
\n",
272 | "`tempf(La Casa Collo (CS5)/Canary-S)` : Temperature (Farenheit) readings from CS5 located at La Casa
\n",
273 | "`humidity(La Casa Collo (CS5)/Canary-S)` : Humidity readings from CS5 located at La Casa
\n",
274 | "`'PM2.5(La Casa/AirNow)`: PM2.5 readings from the reference FEM sensor located at La Casa\n",
275 | "
\n",
276 | "\n",
277 | "`PM2_5(Swansea Elementary Collo (CS7)/Canary-S)` : PM2.5 readings from CS5 sensor located at Swansea Elementary
\n",
278 | "`tempf(Swansea Elementary Collo (CS7)/Canary-S)` : Temperature (Farenheit) readings from CS5 located at Swansea Elementary
\n",
279 | "`humidity(Swansea Elementary Collo (CS7)/Canary-S)` : Humidity readings from CS5 located at Swansea Elementary
\n",
280 | "`PM2.5(I-25 Denver/AirNow)`: PM2.5 readings from the reference FEM sensor located at I-25 Denver\n",
281 | "
"
282 | ]
283 | },
284 | {
285 | "cell_type": "markdown",
286 | "id": "ab0c422e",
287 | "metadata": {},
288 | "source": [
289 | "### Units of our observations"
290 | ]
291 | },
292 | {
293 | "cell_type": "markdown",
294 | "id": "67e0477a",
295 | "metadata": {},
296 | "source": [
297 | "Temperature: Fahrenheit
\n",
298 | "Humidity: grams of water vapor per kilogram of air
\n",
299 | "PM2.5: $µg/m^3$
"
300 | ]
301 | },
302 | {
303 | "cell_type": "markdown",
304 | "id": "2e04bb94",
305 | "metadata": {},
306 | "source": [
307 | "### Checking for data types\n",
308 | "\n",
309 | "Similar vairables should be of the same type. E.g PM2.5 measures by all sensors, ideally, should come in the same datatypes. If not, we will have to cast them to a common datatype"
310 | ]
311 | },
312 | {
313 | "cell_type": "code",
314 | "execution_count": null,
315 | "id": "4b188ed7",
316 | "metadata": {
317 | "scrolled": false
318 | },
319 | "outputs": [],
320 | "source": [
321 | "raw_training_data.dtypes"
322 | ]
323 | },
324 | {
325 | "cell_type": "markdown",
326 | "id": "c14ddfdc",
327 | "metadata": {},
328 | "source": [
329 | "Since we have a Date/Time variable/column in our dataframe, it is important to ensure we let pandas know (explicitly) that this particular column column should be treated as a datetime column type.\n",
330 | "\n",
331 | "Let us cast the datetime column in our dataframe into a datetime type."
332 | ]
333 | },
334 | {
335 | "cell_type": "code",
336 | "execution_count": null,
337 | "id": "2035e87b",
338 | "metadata": {},
339 | "outputs": [],
340 | "source": [
341 | "raw_training_data[\"DateTime\"] = pd.to_datetime(raw_training_data[\"DateTime\"])"
342 | ]
343 | },
344 | {
345 | "cell_type": "markdown",
346 | "id": "18789260",
347 | "metadata": {},
348 | "source": [
349 | "As shown below, our `DateTime`column is now of the `datetime64`type."
350 | ]
351 | },
352 | {
353 | "cell_type": "code",
354 | "execution_count": null,
355 | "id": "0b22295f",
356 | "metadata": {},
357 | "outputs": [],
358 | "source": [
359 | "raw_training_data.dtypes"
360 | ]
361 | },
362 | {
363 | "cell_type": "markdown",
364 | "id": "09a3c74b",
365 | "metadata": {},
366 | "source": [
367 | "### Is this how our data should be organised/arranged?\n"
368 | ]
369 | },
370 | {
371 | "cell_type": "markdown",
372 | "id": "ff8a9ac2",
373 | "metadata": {},
374 | "source": [
375 | "The cell below helps us to acheive this need rearrangement of our data. In the cell, we create a vriable called `full_training_data` where we store our rearranged training data."
376 | ]
377 | },
378 | {
379 | "cell_type": "code",
380 | "execution_count": null,
381 | "id": "d43418c2",
382 | "metadata": {},
383 | "outputs": [],
384 | "source": [
385 | "i15_reference_reading = \"PM2.5(I-25 Globeville/AirNow)\"\n",
386 | "\n",
387 | "full_training_data = pd.DataFrame(\n",
388 | " np.vstack(\n",
389 | " [\n",
390 | " raw_training_data.iloc[:, 1:5],\n",
391 | " pd.concat(\n",
392 | " [\n",
393 | " raw_training_data.iloc[:, 5:8],\n",
394 | " raw_training_data[i15_reference_reading],\n",
395 | " ],\n",
396 | " axis=1,\n",
397 | " ),\n",
398 | " pd.concat(\n",
399 | " [\n",
400 | " raw_training_data.iloc[:, 8:11],\n",
401 | " raw_training_data[i15_reference_reading],\n",
402 | " ],\n",
403 | " axis=1,\n",
404 | " ),\n",
405 | " raw_training_data.iloc[:, 11:15],\n",
406 | " raw_training_data.iloc[:, 15:19],\n",
407 | " ]\n",
408 | " ),\n",
409 | " columns=[\"pm_cs\", \"temp\", \"humidity\", \"pm_airnow\"],\n",
410 | ")"
411 | ]
412 | },
413 | {
414 | "cell_type": "code",
415 | "execution_count": null,
416 | "id": "5462edec",
417 | "metadata": {},
418 | "outputs": [],
419 | "source": [
420 | "full_training_data[\"date_time\"] = pd.to_datetime(\n",
421 | " pd.concat([raw_training_data.iloc[:, 0]] * 5, ignore_index=True)\n",
422 | ")"
423 | ]
424 | },
425 | {
426 | "cell_type": "markdown",
427 | "id": "e7837830",
428 | "metadata": {},
429 | "source": [
430 | "Let is check the columns in our data"
431 | ]
432 | },
433 | {
434 | "cell_type": "code",
435 | "execution_count": null,
436 | "id": "d7923bb2",
437 | "metadata": {},
438 | "outputs": [],
439 | "source": [
440 | "full_training_data.columns"
441 | ]
442 | },
443 | {
444 | "cell_type": "markdown",
445 | "id": "dd51520f",
446 | "metadata": {},
447 | "source": [
448 | "We now have a dataframe with columns defined as `[\"pm_cs\", \"temp\", \"humidity\", \"pm_airnow\", \"data_time\"]`.\n",
449 | "\n",
450 | "`pm_cs`: PM2.5 measured from tne CS sensors (low cost)
\n",
451 | "`pm_airnow`: PM2.5 measured from the AirNow sensors
\n",
452 | "`date_time`: Date and time information
\n",
453 | "`temp`: temperature readings
\n",
454 | "`humidity`: humidity readings from the CS sensors
"
455 | ]
456 | },
457 | {
458 | "cell_type": "markdown",
459 | "id": "a4350fa8",
460 | "metadata": {},
461 | "source": [
462 | "Now, let us check the number of rows in our data."
463 | ]
464 | },
465 | {
466 | "cell_type": "code",
467 | "execution_count": null,
468 | "id": "164645f9",
469 | "metadata": {},
470 | "outputs": [],
471 | "source": [
472 | "print(f\"There are {len(full_training_data)} rows in our data\")"
473 | ]
474 | },
475 | {
476 | "cell_type": "code",
477 | "execution_count": null,
478 | "id": "96bc51a8",
479 | "metadata": {},
480 | "outputs": [],
481 | "source": [
482 | "full_training_data.head()"
483 | ]
484 | },
485 | {
486 | "cell_type": "markdown",
487 | "id": "5712a13a",
488 | "metadata": {},
489 | "source": [
490 | "Time to start adding new features that might be useful"
491 | ]
492 | },
493 | {
494 | "cell_type": "markdown",
495 | "id": "5762bdca",
496 | "metadata": {},
497 | "source": [
498 | "### Adding sensor location label to our data (useful as tag)\n",
499 | "\n",
500 | "This helps to keep track of the different sensing locations durihg our data exploration and modeling. \n",
501 | "\n",
502 | "Because we have the same reference monitor for the three collocated sensors at i-25 Globeville, we will have to keep track of the `airnow_sensor` and `cs_sensor` differently. Below we create these 2 columns."
503 | ]
504 | },
505 | {
506 | "cell_type": "code",
507 | "execution_count": null,
508 | "id": "2adb7b16",
509 | "metadata": {},
510 | "outputs": [],
511 | "source": [
512 | "cs_sensor = [\"NJH\", \"i25_glo_1\", \"i25_glo_2\", \"i25_glo_3\", \"la_casa\"]\n",
513 | "cs_sensor_col = np.repeat(\n",
514 | " cs_sensor, len(raw_training_data)\n",
515 | ") # note that we repeat in accordance to the raw data size.\n",
516 | "\n",
517 | "full_training_data[\"cs_sensor\"] = cs_sensor_col"
518 | ]
519 | },
520 | {
521 | "cell_type": "code",
522 | "execution_count": null,
523 | "id": "d9553823",
524 | "metadata": {},
525 | "outputs": [],
526 | "source": [
527 | "full_training_data[\"airnow_sensor\"] = full_training_data[\"cs_sensor\"].apply(\n",
528 | " lambda x: \"i25_glo\" if x in [\"i25_glo_1\", \"i25_glo_2\", \"i25_glo_3\"] else x\n",
529 | ")"
530 | ]
531 | },
532 | {
533 | "cell_type": "markdown",
534 | "id": "de396d17",
535 | "metadata": {},
536 | "source": [
537 | "So far, we have now defined the following columns in our data: `[\"pm_cs\", \"temp\", \"humidity\", \"pm_airnow\", \"data_time\", \"cs_sensor\", \"airnow_sensor\"]`"
538 | ]
539 | },
540 | {
541 | "cell_type": "markdown",
542 | "id": "d5614b1d",
543 | "metadata": {},
544 | "source": [
545 | "### Accounting for seasonal and time effects through hour, month, and weekend variables "
546 | ]
547 | },
548 | {
549 | "cell_type": "markdown",
550 | "id": "dd51316f",
551 | "metadata": {},
552 | "source": [
553 | "Because of daily, weekly, and seasonal variation in atmospheric PM2.5 that may be due to factors beyond temperature and relative humidity, we will extract hour, weekend, and month variables and convert the `hour` and `month`variables into cyclic values by taking the `cosine` and `sine` of $hour*2π/24$ and $month*2π/12$."
554 | ]
555 | },
556 | {
557 | "cell_type": "markdown",
558 | "id": "ee9c380a",
559 | "metadata": {},
560 | "source": [
561 | "###### Why do we need sine and cosine encoding of time related features?\n",
562 | "\n",
563 | "Some data are inherently cyclical. Time is an enaple of such: minutes, hours, seconds, day of week, week of month, month, season, and so on all follow cycles. \n",
564 | "\n",
565 | "**How can we let our machine learning model know that a feature is cyclical?**
\n",
566 | "By using sine and cosine transformations\n",
567 | "\n",
568 | "In the cell below, I have created a dummy data for each hour in day and have showed how we can tranform the data from linear (1-demensional) to cyclical (2-dimensional using cosine and sine transformations\n",
569 | "\n",
570 | "This [reference](https://ianlondon.github.io/blog/encoding-cyclical-features-24hour-time/) gives a good explanation. \n",
571 | "\n"
572 | ]
573 | },
574 | {
575 | "cell_type": "code",
576 | "execution_count": null,
577 | "id": "b511fe5c",
578 | "metadata": {},
579 | "outputs": [],
580 | "source": [
581 | "# Example that explains cyclical encoding\n",
582 | "\n",
583 | "time = [x for x in range(0, 24 + 1)] # create a dummy list every hour in the data\n",
584 | "\n",
585 | "time = time + time # Concatenate copies of the same list to form 2 days\n",
586 | "\n",
587 | "cos_time = [(math.cos(x) * 2 * (math.pi / 24)) for x in time]\n",
588 | "sin_time = [(math.sin(x) * 2 * (math.pi / 24)) for x in time]\n",
589 | "\n",
590 | "fig, [ax1, ax2] = plt.subplots(nrows=1, ncols=2, figsize=(11, 5.5))\n",
591 | "\n",
592 | "ax1.plot(time)\n",
593 | "ax2.scatter(sin_time, cos_time)\n",
594 | "ax1.set_title(\"Linear\", size=15)\n",
595 | "ax2.set_title(\"2-D Cosine and Sine\", size=15)\n",
596 | "ax1.set_ylabel(\"Hour\", size=15)\n",
597 | "ax1.set_xlabel(\"Sample\", size=15)\n",
598 | "ax2.set_ylabel(\"Cosine\", size=15)\n",
599 | "ax2.set_xlabel(\"Sine\", size=15)\n",
600 | "plt.suptitle(\"Comparing linear and cyclical encoding of time features\", size=15)\n",
601 | "plt.show()"
602 | ]
603 | },
604 | {
605 | "cell_type": "markdown",
606 | "id": "80bd9c98",
607 | "metadata": {},
608 | "source": [
609 | "Notice the boundary between 24 and 0. In time, we know that 24 and 0 should be the same. Cyclical encoding helps us achieve this behaviour.\n",
610 | "\n",
611 | "This approach is one of the ways to handle time information in time series data."
612 | ]
613 | },
614 | {
615 | "cell_type": "code",
616 | "execution_count": null,
617 | "id": "18cbdff7",
618 | "metadata": {},
619 | "outputs": [],
620 | "source": [
621 | "full_training_data[\"time\"] = full_training_data[\"date_time\"].dt.hour\n",
622 | "full_training_data[\"month\"] = full_training_data[\"date_time\"].dt.month\n",
623 | "\n",
624 | "\n",
625 | "full_training_data[\"weekend\"] = (\n",
626 | " full_training_data[\"date_time\"].dt.dayofweek >= 4\n",
627 | ").astype(\"int\")"
628 | ]
629 | },
630 | {
631 | "cell_type": "code",
632 | "execution_count": null,
633 | "id": "70e9367c",
634 | "metadata": {},
635 | "outputs": [],
636 | "source": [
637 | "full_training_data[\"sin_time\"] = full_training_data[\"time\"].apply(\n",
638 | " lambda x: math.sin(x) * 2 * (math.pi / 24)\n",
639 | ")\n",
640 | "full_training_data[\"cos_time\"] = full_training_data[\"time\"].apply(\n",
641 | " lambda x: math.cos(x) * 2 * (math.pi / 24)\n",
642 | ")\n",
643 | "\n",
644 | "full_training_data[\"sin_month\"] = full_training_data[\"month\"].apply(\n",
645 | " lambda x: math.sin(x) * 2 * (math.pi / 12)\n",
646 | ")\n",
647 | "\n",
648 | "full_training_data[\"cos_month\"] = full_training_data[\"month\"].apply(\n",
649 | " lambda x: math.cos(x) * 2 * (math.pi / 12)\n",
650 | ")"
651 | ]
652 | },
653 | {
654 | "cell_type": "markdown",
655 | "id": "4be40442",
656 | "metadata": {},
657 | "source": [
658 | "### Dealing with missing values (NaN) and invalid values.\n",
659 | "\n",
660 | "We need to rely on the literature or the data provider to undertstand how to filter the data and wghat to filter out and/or keep in the data."
661 | ]
662 | },
663 | {
664 | "cell_type": "code",
665 | "execution_count": null,
666 | "id": "f4bab092",
667 | "metadata": {},
668 | "outputs": [],
669 | "source": [
670 | "print(f\"We are starting with {len(full_training_data)} observations\")"
671 | ]
672 | },
673 | {
674 | "cell_type": "markdown",
675 | "id": "dee92fc6",
676 | "metadata": {},
677 | "source": [
678 | "#### Setting the threshold for invalid values"
679 | ]
680 | },
681 | {
682 | "cell_type": "code",
683 | "execution_count": null,
684 | "id": "ad38f7bf",
685 | "metadata": {},
686 | "outputs": [],
687 | "source": [
688 | "low = 0 # CS encodes invalid values as -1\n",
689 | "high = 1500"
690 | ]
691 | },
692 | {
693 | "cell_type": "code",
694 | "execution_count": null,
695 | "id": "cc3ecb8e",
696 | "metadata": {},
697 | "outputs": [],
698 | "source": [
699 | "full_training_data = full_training_data[\n",
700 | " (full_training_data[\"pm_airnow\"] > low)\n",
701 | " & (full_training_data[\"pm_cs\"] > low)\n",
702 | " & (full_training_data[\"temp\"] > low)\n",
703 | " & (full_training_data[\"humidity\"] > low)\n",
704 | " & (full_training_data[\"pm_cs\"] < high)\n",
705 | "]\n"
706 | ]
707 | },
708 | {
709 | "cell_type": "code",
710 | "execution_count": null,
711 | "id": "2fe01abf",
712 | "metadata": {},
713 | "outputs": [],
714 | "source": [
715 | "# Drop NaN values\n",
716 | "\n",
717 | "full_training_data = full_training_data.dropna(\n",
718 | " subset=[\"pm_cs\", \"temp\", \"humidity\", \"pm_airnow\"]\n",
719 | ")"
720 | ]
721 | },
722 | {
723 | "cell_type": "code",
724 | "execution_count": null,
725 | "id": "4a884dfb",
726 | "metadata": {},
727 | "outputs": [],
728 | "source": [
729 | "print(\n",
730 | " f\"Now we are left with {len(full_training_data)} hourly observation as our training data\"\n",
731 | ")"
732 | ]
733 | },
734 | {
735 | "cell_type": "markdown",
736 | "id": "a2150e1e",
737 | "metadata": {},
738 | "source": [
739 | "### Add road length variables"
740 | ]
741 | },
742 | {
743 | "cell_type": "markdown",
744 | "id": "a7d6ba88",
745 | "metadata": {},
746 | "source": [
747 | "Along with adjusting for variability in time, it is useful to investigate variability in space (location of reference sensors). The position of an air quality sensor within a city, especially relative to known sources of pollution such as highways, is likely to affect the characteristics of the air pollution in that area.\n",
748 | "\n",
749 | "In this regard, the authors of our reference study investigated the effects of lengths of different sizes of roads within a certain distance from a monitor. \n",
750 | "\n",
751 | "For this particular course, we will including the lenght of arterial (large) roads within 500m (to be tagged `aroad_500`).\n",
752 | "\n",
753 | "ALERT: The lengths here could have been measured in kilometers or Miles. What is most important is that they are all measured on the same unit, which we can believe is the case."
754 | ]
755 | },
756 | {
757 | "cell_type": "markdown",
758 | "id": "128add33",
759 | "metadata": {},
760 | "source": [
761 | "The data we load below provides us with data on lenght of arterial (large) roads within 500m of each monitor locations."
762 | ]
763 | },
764 | {
765 | "cell_type": "code",
766 | "execution_count": null,
767 | "id": "6840b08b",
768 | "metadata": {},
769 | "outputs": [],
770 | "source": [
771 | "road_data = pd.read_csv(data_root + \"road_lengths.csv\")"
772 | ]
773 | },
774 | {
775 | "cell_type": "code",
776 | "execution_count": null,
777 | "id": "881bbd14",
778 | "metadata": {},
779 | "outputs": [],
780 | "source": [
781 | "road_data.head()"
782 | ]
783 | },
784 | {
785 | "cell_type": "code",
786 | "execution_count": null,
787 | "id": "c042c780",
788 | "metadata": {},
789 | "outputs": [],
790 | "source": [
791 | "print(\"Before merging road lenghts variables\")\n",
792 | "full_training_data.head()"
793 | ]
794 | },
795 | {
796 | "cell_type": "code",
797 | "execution_count": null,
798 | "id": "ed2ca06c",
799 | "metadata": {},
800 | "outputs": [],
801 | "source": [
802 | "full_training_data = road_data.merge(full_training_data, on=\"airnow_sensor\")"
803 | ]
804 | },
805 | {
806 | "cell_type": "code",
807 | "execution_count": null,
808 | "id": "f15107f1",
809 | "metadata": {},
810 | "outputs": [],
811 | "source": [
812 | "print(\n",
813 | " \"After merging road lenghts variables. Longitude and latitude information are also added\"\n",
814 | ")\n",
815 | "full_training_data.head()"
816 | ]
817 | },
818 | {
819 | "cell_type": "markdown",
820 | "id": "dbda2af2",
821 | "metadata": {},
822 | "source": [
823 | "Now, let look below to ensure that all the columns we need in our data have been added. "
824 | ]
825 | },
826 | {
827 | "cell_type": "code",
828 | "execution_count": null,
829 | "id": "0b7a010c",
830 | "metadata": {},
831 | "outputs": [],
832 | "source": [
833 | "full_training_data.columns"
834 | ]
835 | },
836 | {
837 | "cell_type": "markdown",
838 | "id": "49de1d03",
839 | "metadata": {},
840 | "source": [
841 | "### Note about training data preparation\n",
842 | "\n",
843 | "This is our training data preparation. Take note that most real life data science objectives require **rigourous data preparation and feature engineeering**. This lesson has shown you a bit of what that looks like. Now we want to save our cleaned data in a `.csv`file."
844 | ]
845 | },
846 | {
847 | "cell_type": "markdown",
848 | "id": "3b2c819c",
849 | "metadata": {},
850 | "source": [
851 | "### Finally, save the new version of your training data as a `.csv` file\n",
852 | "\n",
853 | "We will use this file in the remaining notebooks (for exploratory data analysis and modeling)"
854 | ]
855 | },
856 | {
857 | "cell_type": "code",
858 | "execution_count": null,
859 | "id": "04127738",
860 | "metadata": {},
861 | "outputs": [],
862 | "source": [
863 | "full_training_data.to_csv(data_root + \"cleaned_training.csv\", index=False)"
864 | ]
865 | },
866 | {
867 | "cell_type": "markdown",
868 | "id": "6825775d",
869 | "metadata": {},
870 | "source": [
871 | "### Now, let us recall have has been done so far.\n",
872 | "\n",
873 | "We have taken our training data from its raw state to a cleaned state. Now, it is ready to be used for further tasks. \n",
874 | "\n",
875 | "What exactly did we do so far:\n",
876 | "\n",
877 | "1. Checked the columns in the data and interpret them.\n",
878 | "2. Checked the datatypes and cast to the right datatype where necessary.\n",
879 | "3. Reorganised our data. We collected data for each CS sensor and from the columns that contain them and stacked all these data on top of one another.\n",
880 | "4. Dealt with missing or invalid data. In our own case we dropped them, in other cases you can input them using heuristics or interpolation.\n",
881 | "5. Created features to account for seasonal/time effects (`weekend`, `cos_hour`, `sin_hour`,`cos_month`, `sin_month`).\n",
882 | "6. merged road lenghts information into our data\n"
883 | ]
884 | },
885 | {
886 | "cell_type": "markdown",
887 | "id": "a99ab9c0",
888 | "metadata": {},
889 | "source": [
890 | "# TEST DATA "
891 | ]
892 | },
893 | {
894 | "cell_type": "markdown",
895 | "id": "5e1a235b",
896 | "metadata": {},
897 | "source": [
898 | "We need to repeat the process we have implemented above for the test set. In certain cases, we might be lucky enough to have test data that is formatted exactly like our training data (thus needing the same set of cleaning operations as the training data). In other cases, we might not be so lucky. Our project is one of the latter. This gives us the opportunity to adapt and craft the appropiate set and sequence of operations needed to clean the test data and align it (in features) to the training data."
899 | ]
900 | },
901 | {
902 | "cell_type": "markdown",
903 | "id": "b9997fd3",
904 | "metadata": {},
905 | "source": [
906 | "**What is the man difference between our test data and our training data**\n",
907 | "\n",
908 | "Our test set comes in two `.csv` dataframes covering **September - October** and **November - December**, respectively. We need to **merge** these data together and restructure them for our own use."
909 | ]
910 | },
911 | {
912 | "cell_type": "markdown",
913 | "id": "b56d1ff4",
914 | "metadata": {},
915 | "source": [
916 | "First, we should load these two dataframes."
917 | ]
918 | },
919 | {
920 | "cell_type": "code",
921 | "execution_count": null,
922 | "id": "050a7390",
923 | "metadata": {},
924 | "outputs": [],
925 | "source": [
926 | "test_path_1 = data_root + \"test-set_Sept-Oct.csv\"\n",
927 | "test_path_2 = data_root + \"test-set_Nov-Dec.csv\""
928 | ]
929 | },
930 | {
931 | "cell_type": "code",
932 | "execution_count": null,
933 | "id": "7a770932",
934 | "metadata": {},
935 | "outputs": [],
936 | "source": [
937 | "test_data_1 = pd.read_csv(test_path_1)\n",
938 | "test_data_2 = pd.read_csv(test_path_2)"
939 | ]
940 | },
941 | {
942 | "cell_type": "code",
943 | "execution_count": null,
944 | "id": "071449a7",
945 | "metadata": {},
946 | "outputs": [],
947 | "source": [
948 | "test_data_1[\"DateTime\"] = pd.to_datetime(test_data_1[\"DateTime\"])\n",
949 | "test_data_2[\"DateTime\"] = pd.to_datetime(test_data_2[\"DateTime\"])"
950 | ]
951 | },
952 | {
953 | "cell_type": "code",
954 | "execution_count": null,
955 | "id": "03715ee8",
956 | "metadata": {},
957 | "outputs": [],
958 | "source": [
959 | "test_data_1.head()"
960 | ]
961 | },
962 | {
963 | "cell_type": "code",
964 | "execution_count": null,
965 | "id": "bbed66cc",
966 | "metadata": {},
967 | "outputs": [],
968 | "source": [
969 | "test_data_2.head()"
970 | ]
971 | },
972 | {
973 | "cell_type": "code",
974 | "execution_count": null,
975 | "id": "bc9b362a",
976 | "metadata": {},
977 | "outputs": [],
978 | "source": [
979 | "test_data_1.dtypes"
980 | ]
981 | },
982 | {
983 | "cell_type": "code",
984 | "execution_count": null,
985 | "id": "a84b5c04",
986 | "metadata": {},
987 | "outputs": [],
988 | "source": [
989 | "test_data_2.dtypes"
990 | ]
991 | },
992 | {
993 | "cell_type": "markdown",
994 | "id": "dfc23145",
995 | "metadata": {},
996 | "source": [
997 | "The first column (Index 0) has the date/time. Columns 1- 4 has the data from CAMP sensor, 5 - 8 has the data from i-25 Denver sensor. Columns 9 and 10 contain the reference PM2.5 for CAMP and i-25 Denver, respectively"
998 | ]
999 | },
1000 | {
1001 | "cell_type": "markdown",
1002 | "id": "a2cf352e",
1003 | "metadata": {},
1004 | "source": [
1005 | "Below, I have created a function to help us reorganise our test data\n"
1006 | ]
1007 | },
1008 | {
1009 | "cell_type": "code",
1010 | "execution_count": null,
1011 | "id": "7ed71035",
1012 | "metadata": {},
1013 | "outputs": [],
1014 | "source": [
1015 | "def organise_test_data(df):\n",
1016 | "\n",
1017 | " \"\"\"\n",
1018 | " Utility function to organise our test data\n",
1019 | "\n",
1020 | " \"\"\"\n",
1021 | "\n",
1022 | " new_data = pd.DataFrame(\n",
1023 | " np.vstack(\n",
1024 | " [\n",
1025 | " pd.concat(\n",
1026 | " [df.iloc[:, 0], df.iloc[:, 1], df.iloc[:, 3:5], df.iloc[:, 9]],\n",
1027 | " axis=1,\n",
1028 | " ),\n",
1029 | " pd.concat(\n",
1030 | " [df.iloc[:, 0], df.iloc[:, 5], df.iloc[:, 7:9], df.iloc[:, 10]],\n",
1031 | " axis=1,\n",
1032 | " ),\n",
1033 | " ]\n",
1034 | " ),\n",
1035 | " columns=[\"date_time\", \"pm_cs\", \"temp\", \"humidity\", \"pm_airnow\"],\n",
1036 | " )\n",
1037 | "\n",
1038 | " camp_sensor_column = [\"CAMP\"] * len(df)\n",
1039 | " denvor_sensor_column = [\"i25_denver\"] * len(df)\n",
1040 | " new_data[\"airnow_sensor\"] = camp_sensor_column + denvor_sensor_column\n",
1041 | "\n",
1042 | " return new_data"
1043 | ]
1044 | },
1045 | {
1046 | "cell_type": "code",
1047 | "execution_count": null,
1048 | "id": "72879e6c",
1049 | "metadata": {},
1050 | "outputs": [],
1051 | "source": [
1052 | "test_data_stacked_1 = organise_test_data(test_data_1)\n",
1053 | "test_data_stacked_2 = organise_test_data(test_data_2)"
1054 | ]
1055 | },
1056 | {
1057 | "cell_type": "code",
1058 | "execution_count": null,
1059 | "id": "25629810",
1060 | "metadata": {},
1061 | "outputs": [],
1062 | "source": [
1063 | "full_test_data = pd.concat([test_data_stacked_1, test_data_stacked_2], axis=0)"
1064 | ]
1065 | },
1066 | {
1067 | "cell_type": "code",
1068 | "execution_count": null,
1069 | "id": "b4bbbf89",
1070 | "metadata": {},
1071 | "outputs": [],
1072 | "source": [
1073 | "full_test_data[\"cs_sensor\"] = full_test_data[\"airnow_sensor\"]"
1074 | ]
1075 | },
1076 | {
1077 | "cell_type": "markdown",
1078 | "id": "a9956376",
1079 | "metadata": {},
1080 | "source": [
1081 | "## Now can we filter the NaNs and invalid values from this column"
1082 | ]
1083 | },
1084 | {
1085 | "cell_type": "code",
1086 | "execution_count": null,
1087 | "id": "06c2a3be",
1088 | "metadata": {},
1089 | "outputs": [],
1090 | "source": [
1091 | "print(f\"We are starting with {len(full_test_data)} observations\")"
1092 | ]
1093 | },
1094 | {
1095 | "cell_type": "code",
1096 | "execution_count": null,
1097 | "id": "e6183fa6",
1098 | "metadata": {},
1099 | "outputs": [],
1100 | "source": [
1101 | "full_test_data = full_test_data[\n",
1102 | " (full_test_data[\"pm_airnow\"] > low)\n",
1103 | " & (full_test_data[\"pm_cs\"] > low)\n",
1104 | " & (full_test_data[\"temp\"] > low)\n",
1105 | " & (full_test_data[\"humidity\"] > low)\n",
1106 | " & (full_test_data[\"pm_cs\"] < high)\n",
1107 | "]\n",
1108 | "\n",
1109 | "full_test_data = full_test_data.dropna(\n",
1110 | " subset=[\"pm_cs\", \"temp\", \"humidity\", \"pm_airnow\"]\n",
1111 | ")"
1112 | ]
1113 | },
1114 | {
1115 | "cell_type": "code",
1116 | "execution_count": null,
1117 | "id": "912bd9b8",
1118 | "metadata": {},
1119 | "outputs": [],
1120 | "source": [
1121 | "print(\n",
1122 | " f\"Now we are left with {len(full_test_data)} hourly observation as our training data\"\n",
1123 | ")"
1124 | ]
1125 | },
1126 | {
1127 | "cell_type": "markdown",
1128 | "id": "14a1591f",
1129 | "metadata": {},
1130 | "source": [
1131 | "## Add time and month variables to dataframe"
1132 | ]
1133 | },
1134 | {
1135 | "cell_type": "code",
1136 | "execution_count": null,
1137 | "id": "935ce6ab",
1138 | "metadata": {},
1139 | "outputs": [],
1140 | "source": [
1141 | "full_test_data[\"time\"] = full_test_data[\"date_time\"].dt.hour\n",
1142 | "full_test_data[\"month\"] = full_test_data[\"date_time\"].dt.month\n",
1143 | "full_test_data[\"weekend\"] = (full_test_data[\"date_time\"].dt.dayofweek >= 4).astype(\n",
1144 | " \"int\"\n",
1145 | ")"
1146 | ]
1147 | },
1148 | {
1149 | "cell_type": "markdown",
1150 | "id": "a48d16c4",
1151 | "metadata": {},
1152 | "source": [
1153 | "##### Cyclical encoding of time and month columns"
1154 | ]
1155 | },
1156 | {
1157 | "cell_type": "code",
1158 | "execution_count": null,
1159 | "id": "4a9e6b49",
1160 | "metadata": {},
1161 | "outputs": [],
1162 | "source": [
1163 | "full_test_data[\"sin_time\"] = full_test_data[\"time\"].apply(\n",
1164 | " lambda x: math.sin(x) * 2 * (math.pi / 24)\n",
1165 | ")\n",
1166 | "full_test_data[\"cos_time\"] = full_test_data[\"time\"].apply(\n",
1167 | " lambda x: math.cos(x) * 2 * (math.pi / 24)\n",
1168 | ")\n",
1169 | "full_test_data[\"cos_month\"] = full_test_data[\"month\"].apply(\n",
1170 | " lambda x: math.sin(x) * 2 * (math.pi / 12)\n",
1171 | ")\n",
1172 | "full_test_data[\"sin_month\"] = full_test_data[\"month\"].apply(\n",
1173 | " lambda x: math.cos(x) * 2 * (math.pi / 12)\n",
1174 | ")"
1175 | ]
1176 | },
1177 | {
1178 | "cell_type": "code",
1179 | "execution_count": null,
1180 | "id": "722cae26",
1181 | "metadata": {},
1182 | "outputs": [],
1183 | "source": [
1184 | "full_test_data = road_data.merge(full_test_data, on=\"airnow_sensor\")"
1185 | ]
1186 | },
1187 | {
1188 | "cell_type": "code",
1189 | "execution_count": null,
1190 | "id": "7557a4b9",
1191 | "metadata": {},
1192 | "outputs": [],
1193 | "source": [
1194 | "full_test_data.head()"
1195 | ]
1196 | },
1197 | {
1198 | "cell_type": "code",
1199 | "execution_count": null,
1200 | "id": "f796d458",
1201 | "metadata": {},
1202 | "outputs": [],
1203 | "source": [
1204 | "full_test_data.to_csv(data_root + \"cleaned_test.csv\", index=False)"
1205 | ]
1206 | },
1207 | {
1208 | "cell_type": "markdown",
1209 | "id": "895d6a9c",
1210 | "metadata": {},
1211 | "source": [
1212 | "Now, we are done with the data cleaning phase"
1213 | ]
1214 | }
1215 | ],
1216 | "metadata": {
1217 | "kernelspec": {
1218 | "display_name": "Python 3",
1219 | "language": "python",
1220 | "name": "python3"
1221 | },
1222 | "language_info": {
1223 | "codemirror_mode": {
1224 | "name": "ipython",
1225 | "version": 3
1226 | },
1227 | "file_extension": ".py",
1228 | "mimetype": "text/x-python",
1229 | "name": "python",
1230 | "nbconvert_exporter": "python",
1231 | "pygments_lexer": "ipython3",
1232 | "version": "3.8.5"
1233 | }
1234 | },
1235 | "nbformat": 4,
1236 | "nbformat_minor": 5
1237 | }
1238 |
--------------------------------------------------------------------------------
/data/test-set_Nov-Dec.csv:
--------------------------------------------------------------------------------
1 | "DateTime","PM2_5(CAMP Collo (CS13)/Canary-S)","PM2_52(CAMP Collo (CS13)/Canary-S)","Temp(CAMP Collo (CS13)/Canary-S)","Hmdty(CAMP Collo (CS13)/Canary-S)","PM2_5(I-25 Denver Collo (CS16)/Canary-S)","PM2_52(I-25 Denver Collo (CS16)/Canary-S)","Temp(I-25 Denver Collo (CS16)/Canary-S)","Hmdty(I-25 Denver Collo (CS16)/Canary-S)","PM2.5(CAMP/AirNow)","PM2.5(I-25 Denver/AirNow)"
2 | "2019-11-01 01:00:00",41.7772,39.6055,31.9566,80.6328,47.9812,49.1725,34.5328,54.2232,,20.4
3 | "2019-11-01 02:00:00",44.9492,42.425,31.157,87.8583,51.434,53.1133,33.4683,58.015,,22
4 | "2019-11-01 03:00:00",45.635,43.273,29.143,89.7467,53.332,55.1397,32.3287,61.0565,,23.3
5 | "2019-11-01 04:00:00",46.5482,44.2198,28.746,89.555,55.9592,57.7683,32.0738,63.35,,24.9
6 | "2019-11-01 05:00:00",43.7972,41.3442,29.222,89.3917,52.6052,54.604,31.9597,61.0929,,22.8
7 | "2019-11-01 06:00:00",42.7763,40.1937,27.872,90.0933,51.1735,52.8135,31.211,60.896,,21.7
8 | "2019-11-01 07:00:00",30.7178,27.2402,27.635,88.0517,49.858,51.5475,29.8915,62.4665,,21.1
9 | "2019-11-01 08:00:00",36.6175,33.4407,27.713,90.7483,53.623,55.626,29.687,63.9752,,23
10 | "2019-11-01 09:00:00",40.342,37.4062,27.998,92.3317,53.8105,56.0842,29.6672,65.0672,,23.1
11 | "2019-11-01 10:00:00",37.9567,35.0405,29.5948,91.8259,51.8257,53.5347,29.972,63.7682,,23.2
12 | "2019-11-01 11:00:00",32.1656,29.2033,32.7032,80.1175,43.3145,44.3613,33.7123,57.2368
13 | "2019-11-01 12:00:00",31.1135,28.7833,36.859,66.2817,35.801,35.8805,43.5658,37.8742,,16.7
14 | "2019-11-01 13:00:00",11.3197,10.3895,46.0034,56.2407,13.5273,12.8668,48.742,32.7293,,7.1
15 | "2019-11-01 14:00:00",8.7724,8.0334,49.1,51.7712,10.7753,9.9639,49.0602,34.0281,,5.7
16 | "2019-11-01 15:00:00",9.5426,8.5361,45.5221,56.8386,10.9284,9.8385,46.9426,36.9085,,5
17 | "2019-11-01 16:00:00",11.6392,10.6398,50.4729,49.7051,13.0537,11.6503,50.9565,30.6887,,6.7
18 | "2019-11-01 17:00:00",7.0465,6.2342,45.7763,52.886,11.051,9.9492,49.421,32.276,,6.4
19 | "2019-11-01 18:00:00",7.1985,6.4357,43.486,54.31,9.5577,8.9145,45.3187,36.0593,,4.2
20 | "2019-11-01 19:00:00",7.7088,6.9218,41.18,58.745,8.9348,8.2482,45.5224,35.3915,,3.7
21 | "2019-11-01 20:00:00",7.2453,6.5498,36.088,66.5367,8.8415,8.8468,41.5847,41.5395,,4.1
22 | "2019-11-01 21:00:00",9.0433,7.865,33.782,74.365,9.8664,9.828,37.4093,48.3805,,5.2
23 | "2019-11-01 22:00:00",9.9215,8.4173,32.717,79.0233,13.2525,13.2328,36.365,50.4463,,6.6
24 | "2019-11-01 23:00:00",13.0573,11.2815,31.6949,82.8746,17.2122,17.1128,34.8237,53.4348,,8
25 | "2019-11-02 00:00:00",17.1563,15.3169,30.2508,83.761,21.3707,21.508,33.704,54.131,,9.8
26 | "2019-11-02 01:00:00",16.7422,14.8395,30.074,85.6033,22.6393,22.3102,32.1102,56.0288,,9.9
27 | "2019-11-02 02:00:00",13.8525,12.2285,29.3244,84.5644,21.609,21.817,31.1898,55.9033,,9.6
28 | "2019-11-02 03:00:00",15.2995,13.0953,27.551,88.8967,23.1732,23.0137,29.5917,59.089,,10.2
29 | "2019-11-02 04:00:00",17.4857,14.9743,27.218,91.5117,27.6766,26.8238,28.6605,61.9879,,11.9
30 | "2019-11-02 05:00:00",18.4843,15.8848,27.02,91.62,25.5918,25.1693,27.4698,63.2847,,10.2
31 | "2019-11-02 06:00:00",18.33,15.3608,25.622,86.4167,26.1502,26.5992,27.1025,63.2537,,10.7
32 | "2019-11-02 07:00:00",16.057,13.4968,26.45,90.3267,26.595,26.4692,26.5677,64.7582,,11.1
33 | "2019-11-02 08:00:00",13.49,11.5462,26.906,84.9583,27.6027,27.8022,26.2755,63.5415,,12.1
34 | "2019-11-02 09:00:00",13.4265,11.3537,26.114,84.2333,24.0236,23.803,25.8897,63.633,,11
35 | "2019-11-02 10:00:00",10.6865,8.7982,26.639,78.6817,27.7052,27.3272,25.9174,62.047,,12.9
36 | "2019-11-02 11:00:00",8.8235,7.574,30.419,64.255,19.969,19.8117,32.045,51.7242,,12.4
37 | "2019-11-02 12:00:00",7.2062,6.1896,38.0709,52.0564,12.626,12.6473,42.2492,31.6248,,12.3
38 | "2019-11-02 13:00:00",6.5736,5.8591,54.8817,32.8397,9.0088,8.9385,50.9453,21.5443,,11.5
39 | "2019-11-02 14:00:00",3.7087,3.1955,66.179,24.7017,5.0645,5.149,58.4472,15.9087,,7.6
40 | "2019-11-02 15:00:00",10.476,9.9165,71.317,23.0767,4.7105,4.4733,66.1488,12.358,,6.7
41 | "2019-11-02 16:00:00",28.8516,27.4647,67.5748,30.8828,33.0685,32.0548,61.6593,22.857,,22
42 | "2019-11-02 17:00:00",31.9322,30.5598,57.172,37.9367,34.5383,33.4777,59.2837,24.9498,,20.4
43 | "2019-11-02 18:00:00",35.6364,34.0558,53.2908,44.4864,39.2833,38.1995,55.8803,29.0763,,23
44 | "2019-11-02 19:00:00",33.9638,32.2308,53.615,46.11,39.4283,38.0203,59.0973,26.3107,,24.6
45 | "2019-11-02 20:00:00",34.4585,32.5714,44.8837,57.4814,40.2367,39.374,54.8768,30.5403
46 | "2019-11-02 21:00:00",41.4458,39.015,40.691,68.6717,43.104,43.8898,46.491,41.2083
47 | "2019-11-02 22:00:00",44.3407,41.504,38.315,78.6067,47.6405,48.7293,43.45,48.4102
48 | "2019-11-02 23:00:00",47.0078,44.0227,36.56,87.5683,53.2864,54.5867,41.4418,52.5311
49 | "2019-11-03 00:00:00",49.5722,46.815,35.141,91.7333,56.7102,58.519,39.177,56.9122
50 | "2019-11-03 01:00:00",53.6457,50.8445,33.683,95.53,61.6753,62.9645,36.5578,62.3112
51 | "2019-11-03 02:00:00",48.8978,45.9873,33.527,95.2,57.4387,58.8767,35.4897,62.9285
52 | "2019-11-03 03:00:00",35.4107,32.0685,33.5651,91.8983,37.042,37.9918,34.8877,60.7098
53 | "2019-11-03 04:00:00",33.038,29.5382,32.414,92.3483,40.9267,41.9673,33.6827,62.3147
54 | "2019-11-03 05:00:00",33.018,29.1645,30.775,92.5433,52.3554,54.5679,32.4526,65.4554
55 | "2019-11-03 06:00:00",31.96,28.3908,31.289,94.2667,50.5687,52.5013,31.6734,66.8713
56 | "2019-11-03 07:00:00",38.5563,35.6362,30.593,94.36,32.6072,32.8232,31.3122,64.3435
57 | "2019-11-03 08:00:00",40.3692,36.8315,29.243,96.6667,36.5783,37.2157,31.2622,64.5948
58 | "2019-11-03 09:00:00",41.6978,38.3908,29.026,94.4133,38.2185,38.9318,30.9056,64.0956
59 | "2019-11-03 10:00:00",15.4168,13.532,33.056,84.5217,23.8643,23.6253,35.9322,53.4852
60 | "2019-11-03 11:00:00",5.1551,4.3935,41.4737,58.2,5.5946,5.2208,47.3157,33.1048
61 | "2019-11-03 12:00:00",5.2158,4.8524,56.6098,35.2673,4.4043,4.3628,55.1636,24.7011
62 | "2019-11-03 13:00:00",10.7428,10.0507,65.477,30.4783,8.452,8.1795,58.6657,22.5245
63 | "2019-11-03 14:00:00",8.8915,8.188,67.586,26.9367,7.304,7.0602,64.8533,16.4292
64 | "2019-11-03 15:00:00",18.7257,17.4642,68.867,32.6717,24.2021,23.0562,66.4056,20.0887
65 | "2019-11-03 16:00:00",14.0128,12.8737,64.791,33.0083,21.4526,20.218,66.5616,21.5328
66 | "2019-11-03 17:00:00",7.3317,6.758,69.203,30.5533,5.1544,4.8025,72.0167,13.5907
67 | "2019-11-03 18:00:00",2.4057,2.0288,66.155,27.615,2.3832,2.4325,72.1548,12.977
68 | "2019-11-03 19:00:00",3.2322,2.9258,59.189,31.8867,2.4502,2.5185,70.6269,13.7382
69 | "2019-11-03 20:00:00",3.7527,3.5197,55.322,33.4517,4.7441,4.9972,59.2789,20.5087
70 | "2019-11-03 21:00:00",9.2558,8.625,51.914,41.4517,9.192,8.4144,55.8738,25.1654
71 | "2019-11-03 22:00:00",12.342,11.2407,48.992,51.2383,14.0323,13.5574,53.423,30.7954
72 | "2019-11-03 23:00:00",9.9475,8.8627,47.678,50.4,15.8032,15.049,50.65,36.7482
73 | "2019-11-04 00:00:00",12.715,11.281,46.4745,53.9931,15.0902,14.5302,49.45,37.066
74 | "2019-11-04 01:00:00",16.3825,14.9362,44.156,60.6383,19.7003,19.1147,47.5257,40.0108
75 | "2019-11-04 02:00:00",17.599,15.879,43.223,63.1833,17.675,17.7233,46.4033,41.426
76 | "2019-11-04 03:00:00",9.3923,8.3985,42.8,64.0717,11.6463,11.2325,45.7977,41.957
77 | "2019-11-04 04:00:00",8.2125,6.9223,42.659,63.3717,9.9543,9.8077,45.087,42.463
78 | "2019-11-04 05:00:00",7.2862,6.2317,42.5797,63.0621,9.153,9.046,44.6222,43.0977
79 | "2019-11-04 06:00:00",5.8512,4.9227,40.802,69.1583,7.3753,7.1163,43.4725,45.1825
80 | "2019-11-04 07:00:00",6.7467,5.748,37.925,74.4217,7.6762,7.8002,41.2349,48.6692
81 | "2019-11-04 08:00:00",6.4636,5.6051,36.5976,80.5305,8.7548,8.7097,40.232,50.3407
82 | "2019-11-04 09:00:00",5.8654,5.1959,36.05,88.05,11.9077,11.4067,40.0841,52.0934
83 | "2019-11-04 10:00:00",6.4362,5.842,37.712,85.92,12.8468,12.471,41.7732,50.4267
84 | "2019-11-04 11:00:00",7.622,6.944,44.249,74.1567,12.945,12.6321,47.9139,41.905
85 | "2019-11-04 12:00:00",8.7058,8.0825,54.2498,51.8441,12.5217,11.3683,52.9902,34.6358,,9.2
86 | "2019-11-04 13:00:00",10.7898,10.0961,61.6258,39.5,11.3767,10.4062,57.6438,28.1769,,9.5
87 | "2019-11-04 14:00:00",7.589,7.0778,63.437,40.265,12.5,11.495,60.0628,27.5053,,11
88 | "2019-11-04 15:00:00",7.4008,7.0156,65.8034,38.3203,11.325,10.3733,62.6995,25.9113,,8.9
89 | "2019-11-04 16:00:00",7.469,7.0353,66.0759,38.869,12.1869,11.523,66.2039,23.9885,,9.3
90 | "2019-11-04 17:00:00",8.0318,7.5887,63.944,40.3367,9.9925,9.2177,65.3972,24.49,,8.1
91 | "2019-11-04 18:00:00",11.9607,11.2337,58.829,46.2617,13.3817,12.4308,63.2522,26.6175,,9
92 | "2019-11-04 19:00:00",16.9683,15.4987,50.369,60.375,20.0757,18.5963,55.4088,37.5748,,11.6
93 | "2019-11-04 20:00:00",14.6759,13.3186,45.4847,84.1407,20.4733,19.4072,49.0523,51.7077,,9.5
94 | "2019-11-04 21:00:00",13.7858,12.5923,42.539,93.6783,18.1913,16.9944,46.8861,56.5477,,9.5
95 | "2019-11-04 22:00:00",16.2145,14.0038,40.34,97.5783,18.6082,18.1089,44.1997,62.212,,10.1
96 | "2019-11-04 23:00:00",17.182,14.6722,39.692,99.27,22.909,22.4905,43.0927,65.9795,,11.7
97 | "2019-11-05 00:00:00",20.5381,18.1371,38.6376,97.8712,29.811,30.0623,42.0445,65.8477,,14.7
98 | "2019-11-05 01:00:00",27.6714,24.961,38.4293,96.3672,33.633,33.4922,40.3895,64.6922,,13.6
99 | "2019-11-05 02:00:00",29.6105,26.7974,38.331,96.7586,35.8462,36.1428,39.5047,65.084,,13.4
100 | "2019-11-05 03:00:00",29.973,26.8119,36.534,97.2755,37.4422,38.3098,38.2903,65.2767,,13.7
101 | "2019-11-05 04:00:00",26.8286,24.1895,34.5017,99.1407,39.3667,39.6575,36.941,67.6538,,14.4
102 | "2019-11-05 05:00:00",31.0225,28.2488,33.926,99.8633,41.7357,41.9907,35.2382,68.1642,,14.8
103 | "2019-11-05 06:00:00",34.6503,31.5308,33.296,99.9,43.1123,43.5405,34.4485,70.156,,15.1
104 | "2019-11-05 07:00:00",35.7675,32.8092,32.1027,98.1898,43.2723,43.9045,34.1113,69.8823,,15.4
105 | "2019-11-05 08:00:00",36.5835,33.9982,31.912,98.2183,45.0572,46.0153,33.7943,70.0605,,16.5
106 | "2019-11-05 09:00:00",40.5846,38.5111,32.7367,99.9,48.0503,49.3048,34.183,69.7223,,18
107 | "2019-11-05 10:00:00",42.151,39.693,33.684,96.38,47.9613,49.2468,38.2675,65.4682,,20.5
108 | "2019-11-05 11:00:00",44.7897,42.9334,40.6207,92.6661,47.0788,46.8635,46.8997,50.7768,,24.6
109 | "2019-11-05 12:00:00",44.5987,43.2531,55.8222,65.82,50.7487,49.6718,55.2178,39.609,,31.3
110 | "2019-11-05 13:00:00",44.6117,42.9922,64.7166,45.819,53.4228,51.3608,58.7982,33.7017,,36.1
111 | "2019-11-05 14:00:00",45.4098,43.8332,69.032,40.0933,50.2593,48.3244,62.2211,27.8952,,38.7
112 | "2019-11-05 15:00:00",43.1691,41.6571,71.3859,36.8466,52.2603,50.1347,68.5472,24.0495,,40.7
113 | "2019-11-05 16:00:00",37.1243,35.3629,74.0672,33.4931,46.7874,44.9436,72.8,20.3725,,34.2
114 | "2019-11-05 17:00:00",28.8436,26.3982,74.804,30.724,38.865,37.4013,76.3317,17.2623,,28.5
115 | "2019-11-05 18:00:00",19.7993,17.3493,66.7867,29.7593,34.8502,33.9976,74.7434,17.425,,24.5
116 | "2019-11-05 19:00:00",23.4327,20.7282,60.053,40.88,28.9215,27.9933,70.2003,19.6252,,22
117 | "2019-11-05 20:00:00",30.7143,27.447,54.275,50.0967,34.4037,33.9753,60.0375,29.759,,29.4
118 | "2019-11-05 21:00:00",29.9543,27.1565,51.002,56.385,35.7738,35.6968,55.408,35.6682,,31
119 | "2019-11-05 22:00:00",25.5807,23.0097,49.706,58.3567,40.3977,40.786,52.3098,40.6402,,33.7
120 | "2019-11-05 23:00:00",35.6657,33.0805,46.807,63.2567,37.692,37.9265,49.1477,44.1563,,26.4
121 | "2019-11-06 00:00:00",31.0348,27.9678,44.594,71.5717,27.939,27.9995,46.3728,48.6447,,16.5
122 | "2019-11-06 01:00:00",19.5373,17.0552,43.036,72.61,28.2232,27.8318,44.1528,53.281,,15.9
123 | "2019-11-06 02:00:00",14.0557,12.601,42.3872,75.3603,32.72,33.1423,41.8043,56.4123,,18.1
124 | "2019-11-06 03:00:00",11.3629,10.1124,41.0153,76.9797,29.4547,29.7576,39.5502,58.8831,,15.4
125 | "2019-11-06 04:00:00",9.8655,8.768,38.8171,82.2127,,,,,,13.8
126 | "2019-11-06 05:00:00",12.1305,10.4095,36.9637,87.8017,23.3693,23.629,35.5627,65.4313,,12.3
127 | "2019-11-06 06:00:00",11.3928,9.8608,34.699,89.9567,22.216,22.3321,34.7379,66.0093,,11.3
128 | "2019-11-06 07:00:00",4.8045,4.114,36.386,85.595,11.5318,10.9682,35.3117,64.8257,,6.4
129 | "2019-11-06 08:00:00",2.775,2.2085,38.219,75.495,14.0873,13.7428,35.2022,63.1245,,8.4
130 | "2019-11-06 09:00:00",2.7002,2.2738,38.4986,70.0379,13.8607,13.3687,35.9443,60.4393,,9.2
131 | "2019-11-06 10:00:00",3.7333,3.226,39.896,67.2467,20.9293,20.3973,41.151,51.1947,,14.7
132 | "2019-11-06 11:00:00",5.0903,4.567,47.24,53.9117,14.5169,13.6566,50.3244,36.2602,,13.6
133 | "2019-11-06 12:00:00",4.7438,4.4427,60.943,35.33,9.431,8.8785,60.0767,24.5713,,10.5
134 | "2019-11-06 13:00:00",4.5402,4.2336,74.8614,24.6729,4.9247,4.5905,66.5805,16.2718,,6.4
135 | "2019-11-06 14:00:00",23.5587,21.5507,68.528,36.5567,24.1779,23.3003,64.3095,25.4585,,16
136 | "2019-11-06 15:00:00",17.1665,15.74,60.299,46.8217,23.317,22.527,57.3892,35.1998,,12.6
137 | "2019-11-06 16:00:00",10.3067,8.9935,45.317,69.1283,13.5407,13.405,45.891,49.1882,,7.5
138 | "2019-11-06 17:00:00",13.1268,11.3065,39.002,90.72,15.3995,14.9615,40.968,59.279,,7.9
139 | "2019-11-06 18:00:00",10.4725,8.843,37.493,96.5617,14.3418,13.799,40.157,62.4533,,7.1
140 | "2019-11-06 19:00:00",2.8156,2.2612,35.8593,99.561,10.1847,9.2818,38.8468,65.4625,,5.5
141 | "2019-11-06 20:00:00",1.9655,1.538,34.355,99.9,2.0775,1.7866,38.0069,64.8826,,1.5
142 | "2019-11-06 21:00:00",2.7182,2.205,33.668,99.9,3.7959,3.1739,37.6023,64.8448,,2.3
143 | "2019-11-06 22:00:00",3.3432,2.6647,32.896,98.2183,5.133,4.8875,37.4715,65.494,,2.8
144 | "2019-11-06 23:00:00",2.677,2.1397,33.218,99.9,4.1518,3.6661,36.9197,65.5405,,2.2
145 | "2019-11-07 00:00:00",3.5383,2.8562,32.957,99.9,4.6427,4.219,36.5375,65.1735,,2.7
146 | "2019-11-07 01:00:00",2.071,1.633,32.654,99.9,3.7566,3.5131,36.3503,65.3405,,2.5
147 | "2019-11-07 02:00:00",0.7195,0.3898,32.2017,99.6259,2.0108,1.7243,35.9943,64.9733,,1.5
148 | "2019-11-07 03:00:00",0.6055,0.3658,32.081,98.84,1.1664,1.0133,35.5057,63.9874,,1.2
149 | "2019-11-07 04:00:00",0.9568,0.5997,31.82,98.955,1.6983,1.5782,34.3302,67.1157,,1.4
150 | "2019-11-07 05:00:00",1.005,0.6278,32.365,96.3733,1.528,1.3767,35.5985,65.5962,,1.6
151 | "2019-11-07 06:00:00",1.212,0.795,33.299,97.8883,1.5565,1.4928,36.1535,63.7497,,1.9
152 | "2019-11-07 07:00:00",2.4563,1.8028,32.982,93.4217,4.1111,3.9477,36.8856,62.4093,,3.5
153 | "2019-11-07 08:00:00",3.8002,3.108,34.103,97.6683,7.2036,7.3792,37.2749,61.9769,,4.8
154 | "2019-11-07 09:00:00",5.3671,4.4553,34.3493,97.6931,13.8247,13.5157,37.4107,60.781,,7.7
155 | "2019-11-07 10:00:00",11.7553,9.6219,35.8779,95.4614,16.2277,16.3822,38.2138,60.0623,,9.2
156 | "2019-11-07 11:00:00",16.6324,13.8115,38.9933,91.8259,18.2022,18.2397,40.224,57.5087,,10.3
157 | "2019-11-07 12:00:00",18.1575,15.6093,50.36,69.9618,20.1185,19.9162,46.6688,47.2248,,12.7
158 | "2019-11-07 13:00:00",16.5535,14.1983,60.032,48.4033,18.7785,18.3332,51.2512,39.8738,,12.2
159 | "2019-11-07 14:00:00",16.3553,14.5253,64.148,41.3633,16.8768,16.2707,58.5797,30.4207,,12
160 | "2019-11-07 15:00:00",20.2377,17.6973,66.089,38.525,23.1072,22.142,62.9913,25.7212,,15.3
161 | "2019-11-07 16:00:00",20.31,18.087,64.1347,42.5754,26.6188,25.586,63.2077,27.2872,,17.2
162 | "2019-11-07 17:00:00",19.4998,17.688,62.102,44.9117,23.51,22.4113,63.7737,27.4045,,15.5
163 | "2019-11-07 18:00:00",18.1308,16.5113,57.419,50.895,24.407,22.7566,61.0287,29.019,,13.9
164 | "2019-11-07 19:00:00",16.7895,14.771,49.868,63.2167,21.9605,20.7765,56.5982,35.7993,,12.1
165 | "2019-11-07 20:00:00",16.7373,14.5493,44.9834,73.122,20.0547,18.392,50.7188,43.7957,,11.5
166 | "2019-11-07 21:00:00",17.3847,15.0878,43.118,85.9067,20.8037,19.5843,47.443,49.5978,,11.6
167 | "2019-11-07 22:00:00",20.6248,17.471,40.856,91.765,24.8823,24.4413,45.9592,54.2653,,13.3
168 | "2019-11-07 23:00:00",26.3912,22.8283,39.613,91.865,28.6322,28.52,44.7225,55.8187,,15.2
169 | "2019-11-08 00:00:00",29.0652,25.0305,40.145,94.0117,33.2632,33.5708,43.7324,57.9611,,18.2
170 | "2019-11-08 01:00:00",31.2312,27.4812,39.401,94.9933,36.8353,37.4072,42.6607,59.3387,,20.4
171 | "2019-11-08 02:00:00",33.5883,29.8048,36.848,97.655,39.8744,40.2875,40.4051,62.6736,,21
172 | "2019-11-08 03:00:00",32.3095,28.9963,34.3908,97.7169,36.298,37.1677,38.3113,65.6155,,17.6
173 | "2019-11-08 04:00:00",33.541,29.2315,33.752,99.8717,43.345,44.823,35.9085,67.5783,,22.1
174 | "2019-11-08 05:00:00",36.4172,32.5005,33.6697,99.9,40.2102,41.2478,34.9198,69.7842,,19
175 | "2019-11-08 06:00:00",37.1645,33.622,31.245,96.5367,37.702,38.704,33.997,70.018,,17.2
176 | "2019-11-08 07:00:00",23.1647,19.6637,32.431,98.2183,36.0723,36.6555,33.5198,70.3207,,15.5
177 | "2019-11-08 08:00:00",14.5013,12.132,34.388,95.43,31.5348,31.3995,33.5187,68.8432,,13.6
178 | "2019-11-08 09:00:00",4.0343,3.3352,36.6832,74.225,22.0772,21.6946,34.1285,63.5139,,11.3
179 | "2019-11-08 10:00:00",6.9422,6.1428,38.762,66.2983,21.9695,21.3387,38.6945,55.1523,,14.4
180 | "2019-11-08 11:00:00",6.6118,5.9498,44.297,55.39,15.4021,14.699,48.9297,36.7675,,13.5
181 | "2019-11-08 12:00:00",6.9151,6.4027,61.0441,36.3831,7.23,6.9983,60.8495,23.1753,,10.7
182 | "2019-11-08 13:00:00",5.0845,4.6325,73.333,24.5583,3.4242,3.4455,67.9002,15.2402,,7.1
183 | "2019-11-08 14:00:00",3.0828,2.9412,80.453,19.48,2.7369,2.5363,73.519,8.4115,,5.5
184 | "2019-11-08 15:00:00",1.5277,1.3102,84.731,16.235,1.6757,1.546,80.143,5.4735,,4.2
185 | "2019-11-08 16:00:00",0.4397,0.3746,83.8369,15.1661,2.9431,2.5813,79.8216,4.8539,,4.2
186 | "2019-11-08 17:00:00",0.4083,0.3642,79.226,15.7667,3.239,2.733,78.6969,5.1797,,3
187 | "2019-11-08 18:00:00",0.5077,0.4095,72.98,18.33,2.7458,2.541,78.1505,5.8162,,3.1
188 | "2019-11-08 19:00:00",1.5095,1.2718,64.3,21.6483,5.3342,5.0085,75.1275,7.4302,,4.4
189 | "2019-11-08 20:00:00",4.1931,3.7988,60.4679,25.6052,3.0007,2.9545,66.191,11.9055,,4.7
190 | "2019-11-08 21:00:00",4.5118,4.1943,57.656,27.9217,2.3752,2.5427,63.7002,13.5233,,3.5
191 | "2019-11-08 22:00:00",3.7115,3.4608,56.612,28.745,3.7712,3.7878,60.5873,15.6368,,5.4
192 | "2019-11-08 23:00:00",6.0155,5.4782,55.232,31.3733,7.3155,7.1292,55.318,23.341,,7.8
193 | "2019-11-09 00:00:00",4.2115,3.8108,52.196,36.5067,4.5583,4.2568,51.3032,28.1023,,5.1
194 | "2019-11-09 01:00:00",4.0125,3.6775,49.667,39.045,5.0622,4.6633,49.108,30.7367,,5.1
195 | "2019-11-09 02:00:00",3.5078,3.0908,48.389,41.4617,5.5464,5.0203,47.0049,34.0587,,4.9
196 | "2019-11-09 03:00:00",1.6893,1.431,48.485,41.3817,4.629,4.2383,45.436,36.0602,,4.5
197 | "2019-11-09 04:00:00",0.9835,0.772,48.194,41.4067,3.9072,3.5936,44.8874,36.1192,,4.1
198 | "2019-11-09 05:00:00",0.7008,0.5036,47.7698,42.0559,2.52,2.5158,44.6782,35.581,,2.9
199 | "2019-11-09 06:00:00",0.4292,0.259,46.7986,41.7746,1.8247,1.6758,43.6523,37.0487,,2.6
200 | "2019-11-09 07:00:00",0.2113,0.0977,46.948,41.815,1.3775,1.3443,43.641,36.1354
201 | "2019-11-09 08:00:00",0.1925,0.1283,47.342,43.5567,1.5202,1.4287,44.2975,35.372
202 | "2019-11-09 09:00:00",0.2955,0.1653,47.369,44.045,2.6287,2.4223,46.3682,32.5873,,3.4
203 | "2019-11-09 10:00:00",0.6081,0.4259,47.9976,42.2203,2.7523,2.6866,49.9461,30.0444,,3.9
204 | "2019-11-09 11:00:00",1.0235,0.9325,53.855,38.7217,2.2818,2.3948,60.2873,20.9247,,3.5
205 | "2019-11-09 12:00:00",1.1042,0.8875,66.9137,29.0053,2.147,2.3293,66.699,16.6352,,3.9
206 | "2019-11-09 13:00:00",1.1534,1.068,78.2407,22.8036,1.4767,1.4853,71.5925,13.5352,,3.4
207 | "2019-11-09 14:00:00",0.975,0.9321,84.725,19.9521,0.7093,0.6202,79.4657,9.5892,,3.2
208 | "2019-11-09 15:00:00",1.4677,1.1665,93.3105,17.3632,1.2432,1.1707,88.1248,5.7362,,2.8
209 | "2019-11-09 16:00:00",0.8028,0.7052,93.59,15.0617,2.0033,1.622,90.075,3.0463,,3.3
210 | "2019-11-09 17:00:00",0.2863,0.2155,87.365,14.61,1.8622,1.5653,89.5635,2.7478,,1.4
211 | "2019-11-09 18:00:00",0.6108,0.5348,79.423,15.6967,1.5285,1.3542,86.6761,3.6027,,1.9
212 | "2019-11-09 19:00:00",1.2793,1.1162,72.146,18.6967,1.4023,1.199,81.7903,4.9095,,2
213 | "2019-11-09 20:00:00",4.0005,3.6225,65.327,21.3867,3.823,3.5115,71.721,7.8572,,5.1
214 | "2019-11-09 21:00:00",11.1597,10.316,59.873,25.4983,9.2968,8.5933,66.991,11.5587,,8.6
215 | "2019-11-09 22:00:00",9.0617,8.464,56.4645,28.5517,15.2463,14.7197,62.1043,15.5133,,12.9
216 | "2019-11-09 23:00:00",15.5417,14.2962,54.152,32.59,15.0063,14.5365,56.9908,21.4377,,10.7
217 | "2019-11-10 00:00:00",8.5358,7.859,52.769,35.5833,8.2092,7.7128,52.7673,25.1698,,6.9
218 | "2019-11-10 01:00:00",5.4757,5.0393,50.989,35.7017,7.2149,6.8352,51.9815,26.2284,,6
219 | "2019-11-10 02:00:00",4.9412,4.5588,50.306,39.0817,8.1705,7.558,49.4005,30.0527,,6
220 | "2019-11-10 03:00:00",4.4993,4.0918,48.818,40.705,9.0427,8.4062,46.022,35.5997,,6.8
221 | "2019-11-10 04:00:00",8.9852,7.8797,45.05,46.9917,8.9722,8.7642,44.4412,38.0433,,7.2
222 | "2019-11-10 05:00:00",4.0938,3.487,44.579,47.5867,11.8193,11.7262,43.1233,40.3182,,8.9
223 | "2019-11-10 06:00:00",4.3202,3.7141,45.2424,46.7224,9.9936,10.1389,41.9654,42.1602,,7.5
224 | "2019-11-10 07:00:00",3.8073,3.3365,44.435,48.1017,13.6925,13.4693,39.9715,46.0469,,9.1
225 | "2019-11-10 08:00:00",6.4951,5.7741,40.5271,55.939,8.6462,8.7497,38.5727,47.0118,,6.6
226 | "2019-11-10 09:00:00",6.8755,5.9825,39.494,59.9083,7.477,7.4723,37.8153,47.295,,5.7
227 | "2019-11-10 10:00:00",6.5142,5.609,41.984,56.355,9.2665,9.0093,42.7237,41.4108,,7
228 | "2019-11-10 11:00:00",6.2538,5.8223,49.115,45.1,7.738,7.141,52.938,30.0577,,5.3
229 | "2019-11-10 12:00:00",3.3242,2.9661,66.4044,28.9559,3.7692,3.7883,64.5222,18.0573,,3.5
230 | "2019-11-10 13:00:00",1.2889,1.0967,77.9305,20.6368,1.2228,1.3435,72.4892,11.9673,,2.3
231 | "2019-11-10 14:00:00",2.291,2.0605,84.682,18.222,2.1685,1.9947,78.6105,8.4417,,2.4
232 | "2019-11-10 15:00:00",4.2488,3.9397,83.141,19.5783,5.4263,5.0403,78.6485,9.4832,,4.3
233 | "2019-11-10 16:00:00",2.7714,2.531,82.4854,18.7475,3.2915,3.0692,81.6698,7.6648,,2.9
234 | "2019-11-10 17:00:00",1.792,1.5892,79.967,18.7933,2.053,1.9748,85.4764,5.8866,,3.8
235 | "2019-11-10 18:00:00",1.1082,1.0742,72.089,21.24,1.1327,1.0473,76.24,8.5355,,2.4
236 | "2019-11-10 19:00:00",1.8655,1.7257,65.594,24.775,1.9285,1.9188,69.8535,11.4682,,2.7
237 | "2019-11-10 20:00:00",3.5695,3.1763,62.198,28.1167,3.5171,3.2811,65.8213,14.6882,,3.9
238 | "2019-11-10 21:00:00",5.2115,4.728,60.611,29.1933,5.4282,5.3047,63.8015,16.3402,,5.4
239 | "2019-11-10 22:00:00",5.5855,4.998,51.635,50.6883,6.37,6.2859,54.3705,32.8308,,15.4
240 | "2019-11-10 23:00:00",7.1778,5.9723,40.735,77.345,8.5715,8.7272,43.8002,51.5368,,6.7
241 | "2019-11-11 00:00:00",9.7285,8.1028,37.142,91.7967,9.6265,9.5307,39.4525,58.8,,5.8
242 | "2019-11-11 01:00:00",1.664,1.2792,32.953,96.1667,3.6425,3.3124,36.5892,62.5873,,2.3
243 | "2019-11-11 02:00:00",0.5905,0.3325,31.1051,97.9949,0.92,0.7968,34.5395,65.1655,,1.2
244 | "2019-11-11 03:00:00",0.5248,0.4022,29.204,99.4483,0.6745,0.6098,30.9387,67.5927,,1
245 | "2019-11-11 04:00:00",0.027,0.007,26.279,98.6167,0.2037,0.3502,27.1607,68.0502,,0.6
246 | "2019-11-11 05:00:00",0.3363,0.2517,23.231,97.7033,0.5821,0.5866,25.0242,66.1032,,0.9
247 | "2019-11-11 06:00:00",1.5867,1.0935,21.752,95.9917,2.1795,1.9652,23.5139,62.0573,,1.3
248 | "2019-11-11 07:00:00",2.9293,2.0663,20.666,96.2233,4.278,4.134,22.8973,61.7363,,1.9
249 | "2019-11-11 08:00:00",4.523,3.6227,20.234,96.1183,6.8115,6.5323,22.1487,63.4677,,3
250 | "2019-11-11 09:00:00",4.4917,3.4815,19.142,95.015,6.5203,6.2558,21.6683,59.7218,,2.5
251 | "2019-11-11 10:00:00",5.6312,4.6985,19.061,94.4183,8.3988,8.3692,21.3512,58.7502,,3.2
252 | "2019-11-11 11:00:00",5.6162,4.5333,20.251,91.0017,7.5437,7.3987,21.6568,59.3712,,5.3
253 | "2019-11-11 12:00:00",6.3977,5.3202,22.442,88.8,12.7177,12.2602,23.4903,55.9467,,6
254 | "2019-11-11 13:00:00",5.051,4.1425,27.29,73.865,8.2022,7.9687,26.5182,47.6415,,3.7
255 | "2019-11-11 14:00:00",7.9933,6.9067,33.035,59.6,11.164,10.4122,30.3227,41.2975,,5.6
256 | "2019-11-11 15:00:00",9.85,8.3817,32.714,59.1733,13.9255,13.448,31.648,40.2112,,8.2
257 | "2019-11-11 16:00:00",13.4027,11.6805,37.2047,55.0051,17.9517,17.7052,32.5483,40.7653,,8.6
258 | "2019-11-11 17:00:00",15.8874,13.7359,35.2967,56.9074,18.602,18.0052,37.023,36.4442,,9.9
259 | "2019-11-11 18:00:00",16.0478,14.012,32.3966,63.5271,17.7675,17.0713,38.9525,34.9718,,9.5
260 | "2019-11-11 19:00:00",16.9307,14.3783,27.251,76.5017,18.6993,18.4397,33.9835,42.4155,,9.4
261 | "2019-11-11 20:00:00",18.3428,15.0157,25.694,85.9933,21.5712,22.0212,29.5898,52.4067,,11.1
262 | "2019-11-11 21:00:00",20.2927,16.7163,25.406,91.6167,26.7385,26.6153,28.949,55.6443,,14.3
263 | "2019-11-11 22:00:00",21.7845,17.9826,25.2003,93.2569,30.4095,30.7177,28.033,57.8848,,16.6
264 | "2019-11-11 23:00:00",25.7098,21.1287,24.958,93.4567,31.9303,32.9727,27.7788,60.1468,,16.5
265 | "2019-11-12 00:00:00",21.5673,17.4725,24.547,93.5267,32.4141,33.1249,27.097,62.2333,,16.5
266 | "2019-11-12 01:00:00",14.2545,12.0617,24.475,92.0717,26.1415,26.3275,26.193,62.8338,,13.6
267 | "2019-11-12 02:00:00",13.4007,10.893,24.071,94.1367,23.146,23.2042,25.385,63.001,,12
268 | "2019-11-12 03:00:00",10.929,8.7358,23.546,94.6567,19.2202,19.7284,24.651,64.1118,,10.6
269 | "2019-11-12 04:00:00",8.8702,7.1657,24.122,93.4083,19.4772,19.731,24.1678,64.5672,,10.8
270 | "2019-11-12 05:00:00",8.4943,6.7788,23.966,93.2917,20.1946,20.4075,23.1568,64.1279,,10.7
271 | "2019-11-12 06:00:00",6.5773,5.5017,24.782,92.0817,19.0848,19.2795,22.4037,66.3695,,9.9
272 | "2019-11-12 07:00:00",4.4447,3.5478,26.507,85.145,15.9512,15.9997,21.881,66.2523,,8.4
273 | "2019-11-12 08:00:00",3.0367,2.1228,26.848,78.2183,16.716,16.4993,23.1267,65.6668,,8.6
274 | "2019-11-12 09:00:00",2.8668,2.1495,26.771,77.785,16.8615,16.2067,24.9466,62.5192,,8.6
275 | "2019-11-12 10:00:00",2.627,2.1412,28.532,70.62,13.851,13.2453,29.4817,54.0692,,8.3
276 | "2019-11-12 11:00:00",2.8961,2.3537,33.0179,55.8386,6.4498,6.4064,39.1039,36.2913,,6.3
277 | "2019-11-12 12:00:00",2.8062,2.543,45.53,43.1217,4.4133,4.1356,45.8051,28.4595,,5.4
278 | "2019-11-12 13:00:00",2.4398,2.3447,53.618,32.9067,4.2485,4.209,50.828,23.6398,,8.8
279 | "2019-11-12 14:00:00",2.1635,2.0225,62.258,27.0617,2.6558,2.7467,58.969,17.4223,,6.4
280 | "2019-11-12 15:00:00",2.6415,2.4402,68.698,23.0067,1.502,1.5428,68.3498,11.3435,,4.4
281 | "2019-11-12 16:00:00",1.6735,1.4538,76.99,17.8433,1.8583,1.6817,77.8075,6.0547,,5.6
282 | "2019-11-12 17:00:00",2.1292,1.9167,75.449,17.1117,2.5828,2.4702,82.8455,3.0792,,6.2
283 | "2019-11-12 18:00:00",2.7722,2.5832,67.96,17.9367,1.653,1.6172,81.3548,3.1073,,4.7
284 | "2019-11-12 19:00:00",5.0944,4.8393,59.2651,24.92,3.6705,3.5811,73.5056,6.8481,,9
285 | "2019-11-12 20:00:00",12.5723,11.7872,52.022,31.8583,12.5474,11.8626,58.7954,16.9848,,20.7
286 | "2019-11-12 21:00:00",17.586,16.4379,48.56,45.6517,16.6971,15.9924,52.884,24.6622,,21.5
287 | "2019-11-12 22:00:00",19.685,17.9,47.75,51.15,17.96,18.205,50.52,26.7455,,18.6
288 | "2019-11-12 23:00:00",6.7873,6.0073,47.297,37.0683,8.4058,8.0633,47.1477,28.393,,10.5
289 | "2019-11-13 00:00:00",4.679,3.9368,46.832,36.2617,5.3407,4.9413,45.3078,31.3275,,5.7
290 | "2019-11-13 01:00:00",4.153,3.6448,45.559,36.1233,4.6115,4.1552,45.032,30.0446,,5
291 | "2019-11-13 02:00:00",1.4371,1.2212,47.1353,31.2119,3.5482,3.2245,45.3037,30.0702,,4.3
292 | "2019-11-13 03:00:00",1.7447,1.4857,47.486,32.1967,6.6125,6.1475,44.6048,33.5148,,6.3
293 | "2019-11-13 04:00:00",1.2337,0.9987,49.823,29.8417,6.1852,5.9295,43.9325,33.1213,,5.5
294 | "2019-11-13 05:00:00",1.0448,0.8462,48.403,31.0833,2.6233,2.357,46.3835,26.192,,2.9
295 | "2019-11-13 06:00:00",1.0415,0.828,46.345,34.075,2.6255,2.358,46.0618,27.1158,,2.9
296 | "2019-11-13 07:00:00",0.1785,0.1285,51.383,30.0967,0.889,0.925,51.1405,20.2415,,2.2
297 | "2019-11-13 08:00:00",0.5865,0.531,53.81,27.2833,1.7323,1.795,56.9117,13.829,,3.5
298 | "2019-11-13 09:00:00",0.7875,0.6698,50.544,27.8233,3.576,3.6485,56.0592,15.4708,,5.1
299 | "2019-11-13 10:00:00",0.7192,0.6687,54.881,29.6483,2.856,2.805,58.7387,15.7194,,5
300 | "2019-11-13 11:00:00",0.5883,0.5465,59.336,29.1717,3.206,3.2712,60.7411,16.3365,,5.5
301 | "2019-11-13 12:00:00",0.7377,0.5462,60.047,29.2767,3.0379,2.9911,60.5705,17.3194,,3.1
302 | "2019-11-13 13:00:00",2.3322,2.1603,62.639,29.0233,8.4022,8.034,63.156,16.4535,,6.6
303 | "2019-11-13 14:00:00",2.105,1.7675,64.592,29.7917,5.6987,5.3943,65.0973,17.1102,,7.7
304 | "2019-11-13 15:00:00",2.117,1.9447,66.809,28.4767,2.753,2.7046,69.36,15.0054,,6
305 | "2019-11-13 16:00:00",1.5461,1.2832,64.7346,28.1881,2.0583,2.0593,72.4163,13.0275,,4.7
306 | "2019-11-13 17:00:00",1.58,1.3958,63.2407,30.8729,2.1377,1.9228,74.5425,12.0708,,4.4
307 | "2019-11-13 18:00:00",1.8168,1.6495,58.661,35.9217,3.0543,2.8018,71.7215,14.0328,,6.2
308 | "2019-11-13 19:00:00",2.9061,2.72,51.6281,44.0644,3.0953,3.088,61.857,21.4857,,4.9
309 | "2019-11-13 20:00:00",2.037,1.7775,47.587,52.8717,1.5028,1.7877,53.3568,31.1748,,3.4
310 | "2019-11-13 21:00:00",2.4143,1.9672,45.028,59.4917,1.7656,1.7811,49.752,36.6423,,2.5
311 | "2019-11-13 22:00:00",3.046,2.6037,44.081,65.835,2.9339,2.7579,47.1844,40.8977,,3.1
312 | "2019-11-13 23:00:00",2.7395,2.4753,42.401,70.5983,3.12,2.6477,45.5308,43.7223,,2.8
313 | "2019-11-14 00:00:00",2.6463,2.1865,41.057,76.18,2.2587,1.9838,44.8615,45.6397,,2.2
314 | "2019-11-14 01:00:00",3.4938,3.0437,41.798,75.3633,2.0662,1.8142,45.5348,44.4345,,2.4
315 | "2019-11-14 02:00:00",3.3543,2.906,41.408,76.6,3.8342,3.418,45.187,45.739,,4.2
316 | "2019-11-14 03:00:00",4.0622,3.528,40.637,79.4833,4.5467,4.1195,44.8268,47.1488,,4
317 | "2019-11-14 04:00:00",3.5647,3.0375,40.313,84.1733,3.2655,3,43.797,49.7507,,2.9
318 | "2019-11-14 05:00:00",1.3321,1.0958,39.2221,91.086,2.0948,1.867,41.7587,56.4874,,1.6
319 | "2019-11-14 06:00:00",1.7263,1.4,38.537,95.0333,2.4892,2.0745,41.5823,59.4825,,1.6
320 | "2019-11-14 07:00:00",8.1279,6.7112,38.6972,95.3828,6.7872,6.2847,41.871,58.7338,,4.2
321 | "2019-11-14 08:00:00",9.486,7.9107,37.786,93.6033,9.2767,9.3085,41.3903,59.9745,,5.2
322 | "2019-11-14 09:00:00",7.4493,6.2512,36.2529,98.578,13.739,13.13,40.0985,63.0753,,6.1
323 | "2019-11-14 10:00:00",13.1298,11.159,36.2734,99.3431,16.7672,15.7138,41.1503,60.7522,,7.7
324 | "2019-11-14 11:00:00",15.2562,13.3392,40.739,92.1633,16.5055,15.8087,45.8682,49.124,,7.9
325 | "2019-11-14 12:00:00",14.7777,13.4802,53.81,58.3017,16.2955,15.637,50.7968,38.641,,9.3
326 | "2019-11-14 13:00:00",13.0141,11.8803,59.0352,41.9362,15.8178,14.7935,56.0693,31.0587,,10.5
327 | "2019-11-14 14:00:00",9.3305,8.6478,66.995,34.0517,11.9213,11.0897,61.6907,24.0778,,9.3
328 | "2019-11-14 15:00:00",4.7423,4.2775,70.7,28.3683,6.1675,5.7688,66.4428,18.722,,5.8
329 | "2019-11-14 16:00:00",3.0992,2.9413,71.708,26.6317,4.6178,4.1838,74.3462,13.2128,,5.2
330 | "2019-11-14 17:00:00",2.265,2.1276,66.2838,28.5241,2.6888,2.4943,70.4133,14.4133,,4.9
331 | "2019-11-14 18:00:00",2.4038,2.085,62.132,31.945,2.4795,2.3817,67.4015,16.6803,,4
332 | "2019-11-14 19:00:00",3.6598,3.413,56.369,39.1383,3.8956,3.9202,61.8579,21.9221,,7.1
333 | "2019-11-14 20:00:00",3.7273,3.331,52.157,46.2683,5.0303,4.8005,55.928,27.8043,,6.2
334 | "2019-11-14 21:00:00",5.8535,5.1113,49.028,52.0683,8.3,7.9963,53.238,31.3668,,9.7
335 | "2019-11-14 22:00:00",8.9341,7.7056,45.6731,57.1475,10.3856,9.6475,50.0023,36.2385,,9.8
336 | "2019-11-14 23:00:00",10.2003,8.841,43.732,63.4233,11.5282,11.0951,46.9025,40.8533,,9.3
337 | "2019-11-15 00:00:00",7.194,6.3685,42.878,68.62,9.4502,9.0613,44.559,44.9884,,8
338 | "2019-11-15 01:00:00",6.2515,5.2657,40.936,71.9,11.0513,10.9131,43.0666,47.961,,9.1
339 | "2019-11-15 02:00:00",6.5768,5.5852,40.726,71.92,13.3767,13.2675,42.4067,50.598,,9.5
340 | "2019-11-15 03:00:00",6.8188,5.8113,40.594,73.7067,11.752,11.7374,41.914,51.1911,,8.7
341 | "2019-11-15 04:00:00",11.1447,9.8255,39.595,77.0067,16.3357,15.6912,40.9512,53.8918,,10.2
342 | "2019-11-15 05:00:00",13.2342,11.83,39.6607,82.1797,17.1308,16.5987,40.252,55.7143,,10.2
343 | "2019-11-15 06:00:00",12.5235,11.1512,39.535,81.1583,14.5447,14.1856,40.5695,55.1205,,8.9
344 | "2019-11-15 07:00:00",9.9043,8.7052,40.667,79.5717,10.7682,10.6778,40.3092,56.2507,,7
345 | "2019-11-15 08:00:00",7.6788,6.8466,40.6431,79.3224,9.9284,9.7116,40.5319,55.3584,,6.9
346 | "2019-11-15 09:00:00",7.6365,6.7297,40.798,73.4117,,,,,,7.9
347 | "2019-11-15 10:00:00",4.957,4.2579,45.62,58.0439,12.9614,12.6395,43.4325,49.8641,,11.2
348 | "2019-11-15 11:00:00",4.3572,3.9293,48.578,51.7083,14.4349,13.7026,46.4254,44.1518,,15.2
349 | "2019-11-15 12:00:00",4.0152,3.6948,54.79,40.4433,12.3231,11.6244,53.79,32.5261,,15.7
350 | "2019-11-15 13:00:00",9.3838,8.6787,63.317,33.7483,8.9135,8.4712,61.586,23.2113,,14.1
351 | "2019-11-15 14:00:00",15.7053,15.0898,68.354,30.6733,9.086,8.0903,68.3762,17.6605,,12.7
352 | "2019-11-15 15:00:00",11.2025,10.5652,79.79,24.2367,6.5711,6.0229,76.151,11.3589,,9.7
353 | "2019-11-15 16:00:00",4.7835,4.3843,83.165,20.8817,4.8207,4.572,83.1044,7.4456,,6.7
354 | "2019-11-15 17:00:00",4.5407,4.2072,79.321,20.225,6.4202,6.2,88.018,6.0347,,6.5
355 | "2019-11-15 18:00:00",8.0413,7.4032,74.531,22.3533,9.84,9.4178,81.8822,7.703,,9.8
356 | "2019-11-15 19:00:00",13.0482,12.1002,67.097,32.815,12.9103,12.7925,72.4885,14.6828,,14.3
357 | "2019-11-15 20:00:00",8.8352,8.2437,60.335,44.645,12.6736,11.863,65.0926,24.0877,,13.7
358 | "2019-11-15 21:00:00",11.5635,10.821,56.303,48.5167,16.122,15.5105,61.13,27.9665,,15.6
359 | "2019-11-15 22:00:00",25.131,23.5118,52.975,52.4983,21.0593,20.2046,57.67,30.842,,19.9
360 | "2019-11-15 23:00:00",24.047,22.3233,51.397,50.76,22.6947,22.1133,55.6263,30.701,,19
361 | "2019-11-16 00:00:00",30.3593,28.2563,49.631,58.2683,24.1973,23.7688,52.5008,34.9025,,19.1
362 | "2019-11-16 01:00:00",14.9607,13.2618,48.539,53.4933,19.9667,19.5342,48.918,38.34,,14.6
363 | "2019-11-16 02:00:00",17.614,15.5503,46.058,57.8633,15.2688,14.8673,46.3335,41.0855,,11.9
364 | "2019-11-16 03:00:00",9.9251,8.5327,45.7837,55.5169,10.3452,10.3208,44.9352,40.8038,,8
365 | "2019-11-16 04:00:00",6.2677,5.2708,47.243,50.1133,9.715,9.5633,45.2392,40.2178,,7.4
366 | "2019-11-16 05:00:00",4.834,4.1487,47.738,48.29,8.637,8.6636,45.2187,39.942,,6.4
367 | "2019-11-16 06:00:00",4.5638,3.9202,47.777,48.0417,9.4823,9.4533,44.1245,42.5745,,7.2
368 | "2019-11-16 07:00:00",6.014,5.2612,47.753,49.1283,8.119,7.9828,44.6893,41.2308,,6.8
369 | "2019-11-16 08:00:00",5.4773,4.6959,45.8447,52.4186,8.0238,7.8962,44.9082,40.763,,6.4
370 | "2019-11-16 09:00:00",6.0685,5.0082,46.568,50.9517,10.6287,10.447,44.901,42.3233,,8.1
371 | "2019-11-16 10:00:00",4.4813,3.9973,49.208,46.3533,12.4367,11.8802,45.0978,42.6252,,10.7
372 | "2019-11-16 11:00:00",7.0183,6.3103,53.2797,41.7186,14.5632,14.1798,48.3217,37.8977,,11.1
373 | "2019-11-16 12:00:00",6.2297,5.7519,62.6915,30.1475,8.4515,8.4818,61.5877,19.0723,,9.2
374 | "2019-11-16 13:00:00",6.3302,5.6165,62.923,28.6783,8.5067,8.3311,64.1339,17.3195,,7.1
375 | "2019-11-16 14:00:00",4.6257,3.934,68.485,25.8783,5.6795,5.7297,68.111,15.0383,,5.1
376 | "2019-11-16 15:00:00",3.2148,2.9729,71.3207,25.1983,5.4848,5.2265,68.6083,14.8998,,5.7
377 | "2019-11-16 16:00:00",2.9269,2.5719,70.3214,25.8741,5.4032,5.1142,70.5063,14.0222,,7.6
378 | "2019-11-16 17:00:00",3.3692,2.9702,66.47,27.6887,4.51,4.2908,68.949,14.5597,,3.8
379 | "2019-11-16 18:00:00",4.1709,3.8604,63.7433,30.9833,5.1433,5.1464,64.9011,18.4392,,4.7
380 | "2019-11-16 19:00:00",3.2772,3.0175,59.711,36.5933,4.1759,4.113,61.7725,22.0784,,3.5
381 | "2019-11-16 20:00:00",1.354,1.2315,50.468,58.2067,1.9467,2.297,52.9072,37.8628,,13.7
382 | "2019-11-16 21:00:00",0.2202,0.1247,43.754,73.8067,0.7025,0.6178,46.3202,47.9662,,2.3
383 | "2019-11-16 22:00:00",0.735,0.5068,42.449,78.495,0.6343,0.5842,45.4493,49.0552,,1.3
384 | "2019-11-16 23:00:00",1.2348,0.941,41.804,79.9117,1.4357,1.4807,44.4675,49.4113,,1.6
385 | "2019-11-17 00:00:00",1.44,1.1505,40.442,83.37,1.8075,1.6813,42.789,53.1084,,1.7
386 | "2019-11-17 01:00:00",2.468,1.9046,38.5144,86.3295,4.2335,4.0138,41.9552,54.8812,,2.8
387 | "2019-11-17 02:00:00",2.8797,2.2972,39.56,89.18,5.632,5.1397,41.435,55.9088,,3.4
388 | "2019-11-17 03:00:00",3.861,3.222,37.687,90.7733,5.8803,5.7648,39.8215,59.2441,,3.4
389 | "2019-11-17 04:00:00",3.8847,3.2155,37.679,93.9633,6.4967,6.5462,38.3555,63.6383,,3.8
390 | "2019-11-17 05:00:00",3.6268,2.9938,36.479,94.3417,6.8103,7.0212,36.7975,66.2192,,3.8
391 | "2019-11-17 06:00:00",3.0644,2.5461,35.3153,93.761,6.9975,7.2952,35.599,67.4705,,3.9
392 | "2019-11-17 07:00:00",2.7668,2.2488,34.348,93.7817,7.0277,7.1998,34.8842,68.6202,,3.6
393 | "2019-11-17 08:00:00",2.3502,1.9183,34.706,94.0283,5.4712,5.5742,34.302,68.3843,,2.9
394 | "2019-11-17 09:00:00",2.0603,1.6435,32.941,91.2567,5.5008,5.7855,33.901,67.2243,,2.9
395 | "2019-11-17 10:00:00",4.0748,3.5478,33.046,91.2483,7.1273,7.1405,35.7665,64.0458,,3.7
396 | "2019-11-17 11:00:00",4.1664,3.4986,40.1,83.0797,7.999,7.4712,44.4563,47.8807,,4.5
397 | "2019-11-17 12:00:00",4.3818,4.0891,56.6214,47.9071,7.2018,6.6902,52.0097,35.3572,,4.6
398 | "2019-11-17 13:00:00",4.9261,4.507,63.3425,38.5125,5.9297,5.8922,55.372,30.0818,,4.1
399 | "2019-11-17 14:00:00",3.2687,2.9157,72.203,28.0317,3.8903,3.7123,64.6274,19.9602,,3.3
400 | "2019-11-17 15:00:00",2.7518,2.4933,72.098,25.56,3.635,3.5465,67.0272,16.5962,,4.8
401 | "2019-11-17 16:00:00",3.2872,2.921,71.204,25.67,3.6573,3.372,75.9123,11.5052,,3.6
402 | "2019-11-17 17:00:00",2.0183,1.7512,68.204,26.7617,2.9102,2.7085,80.1548,9.4741,,3.4
403 | "2019-11-17 18:00:00",3.2603,3.0572,65.156,28.615,3.9553,3.6463,70.8018,13.4692,,4
404 | "2019-11-17 19:00:00",4.901,4.5253,58.306,33.46,4.758,4.4426,64.9782,17.3425,,4.7
405 | "2019-11-17 20:00:00",5.0557,4.6093,55.136,38.0833,5.1885,5.183,58.6645,21.9972,,5.1
406 | "2019-11-17 21:00:00",7.3005,6.7468,52.984,39.635,11.1423,10.7467,56.7765,24.5985,,8
407 | "2019-11-17 22:00:00",9.4245,8.8418,53.54,42.3067,16.508,15.5842,55.2902,27.0368,,10.3
408 | "2019-11-17 23:00:00",2.244,1.9958,55.889,41.2483,5.5288,5.4627,57.4395,25.9735,,5.9
409 | "2019-11-18 00:00:00",0.5213,0.3383,58.499,40.1967,0.437,0.5462,61.6205,23.5247,,11.8
410 | "2019-11-18 01:00:00",0.0532,0.0368,58.475,42.095,0.1285,0.171,61.4787,24.8567,,0.9
411 | "2019-11-18 02:00:00",0.0228,0.0033,57.713,44.2883,0.0933,0.1297,60.7392,26.6982,,1.1
412 | "2019-11-18 03:00:00",0.0147,0.0202,58.7468,43.4983,2.4245,2.5147,56.9222,28.2122,,2.6
413 | "2019-11-18 04:00:00",0.0067,0.004,59.321,43.68,3.2512,3.271,53.5908,33.9042,,3
414 | "2019-11-18 05:00:00",0.078,0.0355,58.913,44.57,2.1364,2.1974,54.8374,33.8139,,3.3
415 | "2019-11-18 06:00:00",0.0493,0.0728,56.033,47.79,0.634,0.6532,56.4887,31.9937,,1.7
416 | "2019-11-18 07:00:00",0.0072,0.0083,56.141,48.435,0.1565,0.2207,58.6992,29.9513,,0.6
417 | "2019-11-18 08:00:00",0.5093,0.4217,53.255,52.6383,0.2825,0.4415,57.0105,31.8095,,1.5
418 | "2019-11-18 09:00:00",0.9267,0.9149,52.0274,54.8123,3.7833,3.6563,54.1885,34.8542,,4.1
419 | "2019-11-18 10:00:00",5.5343,5.2462,51.547,52.0517,5.7967,5.2905,57.219,30.2628,,6.5
420 | "2019-11-18 11:00:00",3.7673,3.4872,56.506,46.3633,4.2246,3.9508,67.5674,20.2057,,7.8
421 | "2019-11-18 12:00:00",0.7922,0.68,73.151,28.625,4.7467,4.4225,73.1068,15.0905,,6.7
422 | "2019-11-18 13:00:00",0.37,0.3005,73.63,25.1767,2.571,2.2202,72.1523,14.2846,,3.5
423 | "2019-11-18 14:00:00",0.2058,0.1615,79.472,22.9683,2.3487,2.0855,73.1927,13.2888,,3
424 | "2019-11-18 15:00:00",0.3302,0.3053,80.9352,21.6362,3.242,2.9773,75.42,11.386,,4.5
425 | "2019-11-18 16:00:00",0.3058,0.2562,77.165,21.8383,1.679,1.4877,77.4985,9.3897,,2.6
426 | "2019-11-18 17:00:00",0.2152,0.2103,73.544,22.3733,2.4233,2.1159,73.4974,10.1607,,3.1
427 | "2019-11-18 18:00:00",1.1152,1.0672,67.96,24.0833,3.5207,3.1617,71.0453,11.8218,,3.4
428 | "2019-11-18 19:00:00",1.6038,1.4925,64.67,28.4067,1.4187,1.3497,69.1564,13.7772,,5.4
429 | "2019-11-18 20:00:00",1.7979,1.6148,60.7659,34.6155,1.557,1.5017,64.2322,18.7012,,4.6
430 | "2019-11-18 21:00:00",2.1223,1.8858,56.302,42.5367,2.546,2.5885,59.5823,25.8838,,4.8
431 | "2019-11-18 22:00:00",4.0758,3.5722,55.058,46.6583,4.6325,4.7023,57.0284,29.7998,,5.5
432 | "2019-11-18 23:00:00",2.9383,2.7202,53.987,49.0817,7.4574,7.2131,55.2695,31.8505,,7.1
433 | "2019-11-19 00:00:00",3.8065,3.4678,52.583,49.3433,6.0466,5.8459,52.8602,32.4533,,7.2
434 | "2019-11-19 01:00:00",2.7225,2.5237,51.002,49.7267,3.6153,3.6462,51.172,34.1243,,4
435 | "2019-11-19 02:00:00",1.2514,1.112,49.5027,50.2169,4.6072,4.2578,48.5093,37.355,,5
436 | "2019-11-19 03:00:00",2.6207,2.188,46.712,53.6633,6.1472,5.4423,46.1302,41.2678,,5.2
437 | "2019-11-19 04:00:00",2.9847,2.5202,45.5714,55.0241,6.7458,6.3007,44.579,42.5372,,5.6
438 | "2019-11-19 05:00:00",3.4782,2.9557,44.432,57.5967,6.7188,6.4273,43.1408,44.6832,,6
439 | "2019-11-19 06:00:00",4.7897,4.1325,42.551,59.6183,7.3495,7.2347,41.7143,47.7787,,7.2
440 | "2019-11-19 07:00:00",5.4137,4.6783,40.7529,64.8661,8.781,8.874,40.987,49.3748,,7.3
441 | "2019-11-19 08:00:00",6.2352,5.3793,39.472,68.115,9.1875,9.1273,40.5057,49.8787,,7.5
442 | "2019-11-19 09:00:00",7.8249,7.1268,39.981,71.5746,12.3312,12.3087,40.503,50.7665,,9.2
443 | "2019-11-19 10:00:00",10.8772,10.0367,41.3631,70.4897,13.6167,13.408,42.431,48.2273,,10.7
444 | "2019-11-19 11:00:00",11.5058,10.4953,48.576,52.8667,12.791,12.0363,52.745,32.5445,,9.8
445 | "2019-11-19 12:00:00",7.2387,6.8302,63.082,33.2267,7.7588,7.3117,58.168,24.7542,,7.1
446 | "2019-11-19 13:00:00",3.9653,3.6454,61.401,32.5356,3.1643,3.2125,59.6338,21.5021,,5.1
447 | "2019-11-19 14:00:00",2.3243,2.1673,68.693,27.165,3.3202,3.2367,64.8477,16.666,,5.6
448 | "2019-11-19 15:00:00",1.0645,0.879,74.045,23.84,2.0553,1.836,71.6207,12.9317,,4.1
449 | "2019-11-19 16:00:00",0.5344,0.4547,73.4244,23.378,0.7595,0.8742,74.9334,10.4808,,3.8
450 | "2019-11-19 17:00:00",0.7287,0.6732,71.498,23.3667,1.0533,1.0625,73.4784,10.608,,4.4
451 | "2019-11-19 18:00:00",2.3423,2.0708,66.95,25.8733,5.7038,5.3661,68.8657,14.6926,,10.6
452 | "2019-11-19 19:00:00",8.6445,7.9475,61.537,29.9233,9.0994,8.4447,68.7223,16.1718,,10.6
453 | "2019-11-19 20:00:00",7.9675,7.4517,58.487,34.63,10.4552,9.7453,62.2082,21.0802,,13.4
454 | "2019-11-19 21:00:00",7.414,6.8912,55.589,37.5483,9.5343,9.2197,58.1811,24.0064,,11.3
455 | "2019-11-19 22:00:00",11.6288,10.8158,53.165,42.025,14.9007,13.9398,53.8952,27.7222,,12.3
456 | "2019-11-19 23:00:00",11.1995,10.4722,51.902,44.1667,11.099,10.4898,51.6865,30.756,,9.5
457 | "2019-11-20 00:00:00",7.4576,6.6788,49.4661,47.0441,12.322,11.6685,48.1803,38.0642,,11.3
458 | "2019-11-20 01:00:00",10.1683,8.9548,47.864,49.3917,17.862,17.6512,46.35,39.9142,,13.1
459 | "2019-11-20 02:00:00",12.0752,10.6862,46.454,54.6133,17.3152,16.8365,45.8672,38.9423,,13.4
460 | "2019-11-20 03:00:00",15.8047,13.7988,46.9152,53.2207,17.7177,17.2765,46.7715,39.4292,,11.5
461 | "2019-11-20 04:00:00",16.3653,14.697,48.728,50.8733,16.2047,15.7126,47.9053,37.9152,,11.7
462 | "2019-11-20 05:00:00",15.0422,13.5688,48.32,51.5367,15.6835,15.1363,47.9833,37.7243,,10.5
463 | "2019-11-20 06:00:00",14.87,13.1982,47.72,53.41,15.4052,15.1281,47.6608,38.1252,,11.2
464 | "2019-11-20 07:00:00",13.8378,12.355,47.648,54.6417,17.8295,17.2622,47.6518,39.287,,12.6
465 | "2019-11-20 08:00:00",15.1897,13.5978,46.705,55.045,18.5097,18.0808,47.9168,39.1953,,13.4
466 | "2019-11-20 09:00:00",16.429,14.9907,47.3031,57.0746,20.6398,19.5732,48.55,38.2078,,14.3
467 | "2019-11-20 10:00:00",18.7484,16.9725,48.4747,55.5544,20.8193,19.872,48.8732,38.01,,15.3
468 | "2019-11-20 11:00:00",19.8935,18.272,52.757,51.6183,25.3747,24.3817,51.3707,36.6947,,17.6
469 | "2019-11-20 12:00:00",8.9365,8.1632,59.003,42.2133,16.4858,15.8353,57.6582,28.7918,,12.5
470 | "2019-11-20 13:00:00",5.3848,4.7645,62.564,40.3567,9.651,9.2922,60.9677,26.7305,,7.3
471 | "2019-11-20 14:00:00",4.4933,4.1728,61.247,43.105,9.601,8.9908,59.7338,29.4637,,7
472 | "2019-11-20 15:00:00",4.1458,3.7563,58.522,44.8083,8.6023,8.1251,58.3389,31.3334,,7.2
473 | "2019-11-20 16:00:00",3.535,3.2358,56.561,49.3283,6.0795,5.8522,56.3498,33.884,,6
474 | "2019-11-20 17:00:00",1.7698,1.7318,51.908,53.7567,2.6857,2.6598,53.4545,35.7217,,4.8
475 | "2019-11-20 18:00:00",2.0235,1.6845,45.74,75.2067,2.7088,2.2665,47.1948,50.7135,,3.3
476 | "2019-11-20 19:00:00",3.4255,2.9933,39.571,95.9467,5.176,4.6898,42.1648,64.9855,,3.2
477 | "2019-11-20 20:00:00",5.5337,4.8708,36.1014,96.4797,11.7283,11.1107,39.6902,70.808,,4.8
478 | "2019-11-20 21:00:00",7.6443,6.5873,36.002,99.9,16.244,15.2985,38.9928,73.1148,,6.9
479 | "2019-11-20 22:00:00",7.4817,6.1553,35.834,99.9,9.023,8.6467,38.3923,70.827,,3.2
480 | "2019-11-20 23:00:00",6.2322,5.361,34.453,98.2183,8.233,8.0923,37.4107,70.0177,,2.6
481 | "2019-11-21 00:00:00",5.4628,4.661,33.916,98.2183,8.7788,8.333,37.0315,70.2288,,2.7
482 | "2019-11-21 01:00:00",5.3268,4.2558,33.344,99.9,6.1815,5.9957,35.5505,70.7398,,2.5
483 | "2019-11-21 02:00:00",4.2738,3.3384,32.7138,99.9,5.9992,5.7833,34.8815,69.759,,3.3
484 | "2019-11-21 03:00:00",3.4755,2.6785,31.982,99.9,4.1633,4.2522,33.828,67.2213,,2.7
485 | "2019-11-21 04:00:00",6.0077,4.6412,31.856,99.9,8.3237,7.99,33.9193,67.0087,,4.2
486 | "2019-11-21 05:00:00",6.6818,5.4105,31.096,98.2183,9.1105,8.7695,34.4748,65.9303,,4.6
487 | "2019-11-21 06:00:00",4.8993,4.0025,30.923,99.9,8.2811,7.8436,33.9392,66.8889,,3.9
488 | "2019-11-21 07:00:00",3.869,3.1247,30.082,98.2183,5.587,5.6268,33.341,68.2277,,2.9
489 | "2019-11-21 08:00:00",5.7933,4.8677,30.158,99.9,8.6459,8.4316,32.632,69.3692,,4.1
490 | "2019-11-21 09:00:00",7.006,6.0292,30.017,99.9,11.163,10.7543,32.3792,69.3305,,4.8
491 | "2019-11-21 10:00:00",7.6722,6.5002,30.1771,99.9,18.7552,18.503,32.494,67.9992,,8.5
492 | "2019-11-21 11:00:00",7.4532,6.4645,30.383,99.9,24.251,23.1858,32.8037,67.2452,,13.3
493 | "2019-11-21 12:00:00",6.221,5.3047,31.151,99.9,19.2868,18.3925,33.0837,65.5872,,7.8
494 | "2019-11-21 13:00:00",6.7778,5.4633,31.55,99.9,19.1983,18.4742,33.2882,65.2928,,10.8
495 | "2019-11-21 14:00:00",5.5445,4.5477,31.889,99.9,21.6138,20.9782,33.3359,66.079,,9.9
496 | "2019-11-21 15:00:00",4.4812,3.6173,31.77,96.4883,7.603,7.3253,34.2395,64.7967,,4.5
497 | "2019-11-21 16:00:00",5.8838,4.7372,33.501,94.5167,11.6993,11.3811,35.7948,61.9695,,6.6
498 | "2019-11-21 17:00:00",9.0875,7.7722,33.505,96.805,8.3797,8.2897,35.9018,60.8823,,5.6
499 | "2019-11-21 18:00:00",11.1617,9.6312,33.401,99.4733,22.1987,21.4308,34.6743,65.9855,,11.5
500 | "2019-11-21 19:00:00",13.3213,11.476,32.069,99.9,18.5222,18.2313,33.7158,68.355,,8.3
501 | "2019-11-21 20:00:00",11.4237,9.6498,31.565,99.9,13.7893,12.9417,33.689,67.2247,,6.5
502 | "2019-11-21 21:00:00",8.6298,7.2565,30.986,99.9,16.318,15.6395,33.6784,69.6543,,7.9
503 | "2019-11-21 22:00:00",10.1782,8.5875,30.842,99.9,18.5333,17.688,33.7218,70.013,,9
504 | "2019-11-21 23:00:00",10.2898,8.6433,30.499,98.2183,13.6448,13.1398,33.8528,68.8523,,7
505 | "2019-11-22 00:00:00",11.8213,10.0713,31.151,99.9,13.123,12.6923,33.8105,68.7457,,7.3
506 | "2019-11-22 01:00:00",8.0897,6.7376,31.4234,99.9,11.7792,11.2741,33.938,69.7079,,8.9
507 | "2019-11-22 02:00:00",3.5965,2.8968,31.21,98.2183,9.1287,9.04,33.8722,70.5742,,7.9
508 | "2019-11-22 03:00:00",3.461,2.6029,31.0898,98.1898,8.0853,7.9628,33.2995,72.0885,,6.5
509 | "2019-11-22 04:00:00",4.6447,3.4883,31.087,98.2183,10.1703,10.0915,33.1525,72.32,,6.4
510 | "2019-11-22 05:00:00",3.5307,2.675,31.631,99.9,8.0752,7.754,33.0905,72.4947,,5.8
511 | "2019-11-22 06:00:00",2.7227,1.9553,31.61,99.9,6.1133,6.4555,32.9248,72.8877,,4.6
512 | "2019-11-22 07:00:00",3.3762,2.4765,31.421,99.9,8.578,8.583,32.7138,72.8682,,7.6
513 | "2019-11-22 08:00:00",4.2047,3.0202,30.862,98.2183,11.6505,11.7158,32.6587,73.3248,,8
514 | "2019-11-22 09:00:00",3.4865,2.6485,31.556,99.9,12.1524,11.9478,32.7863,73.6163,,8.5
515 | "2019-11-22 10:00:00",3.978,3.1023,32.135,99.9,10.3477,10.3812,33.5543,71.3837,,11.3
516 | "2019-11-22 11:00:00",4.756,3.9709,34.3617,99.5293,8.2743,8.4185,35.2622,69.2942,,7
517 | "2019-11-22 12:00:00",5.6378,4.6475,36.623,96.69,12.287,11.7311,37.6472,63.6951,,9.5
518 | "2019-11-22 13:00:00",7.3618,6.1352,41.345,92.1267,17.6974,17.2702,40.5726,59.2095,,13.4
519 | "2019-11-22 14:00:00",13.7053,11.5632,41.615,89.4617,24.6088,23.943,41.984,55.311,,14.4
520 | "2019-11-22 15:00:00",17.9022,15.6833,44.948,84.4233,19.4748,19.0339,46.6803,46.1636,,10.2
521 | "2019-11-22 16:00:00",20.3017,17.7882,47.837,75.2233,19.6631,18.7303,46.8548,44.7392,,9.8
522 | "2019-11-22 17:00:00",20.7712,18.3278,44.063,81.46,22.0218,21.233,52.6278,38.6912,,11.4
523 | "2019-11-22 18:00:00",22.2358,19.5408,43.139,86.9183,20.6589,20.0802,52.9887,37.1434,,11
524 | "2019-11-22 19:00:00",26.0752,22.3085,37.787,95.0367,22.193,22.229,43.939,50.412,,11.2
525 | "2019-11-22 20:00:00",19.1145,16.0602,34.603,97.4583,20.6447,20.5492,38.8367,61.446,,9.3
526 | "2019-11-22 21:00:00",19.3683,16.692,33.367,98.2183,22.9083,22.925,37.0532,66.5842,,9.6
527 | "2019-11-22 22:00:00",20.6765,17.8468,33.527,99.9,32.0922,32.3347,36.2473,68.5357,,12.8
528 | "2019-11-22 23:00:00",21.4755,18.2755,31.972,98.2183,36.6058,37.651,34.7017,71.0862,,15.1
529 | "2019-11-23 00:00:00",21.4247,18.2603,31.631,99.9,42.4127,44.0515,33.2062,72.5382,,18.7
530 | "2019-11-23 01:00:00",19.5085,16.6492,31.334,99.9,25.5967,25.6273,31.8043,72.3737,,10.8
531 | "2019-11-23 02:00:00",14.2228,12.0313,30.854,99.9,21.7052,21.8693,30.5605,71.7123,,9.6
532 | "2019-11-23 03:00:00",11.8492,10.5522,29.8827,99.9,20.9418,20.9485,30.0418,71.3758,,9.2
533 | "2019-11-23 04:00:00",,,,,16.5861,16.801,29.2344,70.03,,7.7
534 | "2019-11-23 05:00:00",,,,,17.9008,17.7455,28.7753,68.5173,,7.9
535 | "2019-11-23 06:00:00",,,,,8.4939,8.6893,28.5751,67.4539,,4.4
536 | "2019-11-23 07:00:00",,,,,3.4908,3.3453,29.3558,64.67,,2.6
537 | "2019-11-23 08:00:00",,,,,3.0715,2.9137,29.6487,61.3715,,2.3
538 | "2019-11-23 09:00:00",,,,,1.7407,1.6439,30.9241,55.7731,,2.2
539 | "2019-11-23 10:00:00",1.501,1.1062,31.1943,87.6286,2.0733,1.9273,33.5563,50.291,,2.7
540 | "2019-11-23 11:00:00",5.4339,4.9827,38.5982,65.6576,2.8928,2.6495,41.2997,37.5048,,3.2
541 | "2019-11-23 12:00:00",2.0436,1.8605,46.5964,51.1436,2.7992,2.6418,47.4323,31.4512,,3.5
542 | "2019-11-23 13:00:00",1.5512,1.4508,50.894,46.0867,2.0408,1.9479,49.8544,28.9749,,2.8
543 | "2019-11-23 14:00:00",1.7969,1.4481,59.5736,37.5017,1.6038,1.8842,53.3698,26.5217,,2.8
544 | "2019-11-23 15:00:00",1.5087,1.3107,67.829,31.125,2.0302,2.1905,58.4851,23.577,,3
545 | "2019-11-23 16:00:00",1.6463,1.5052,66.566,31.585,1.6238,1.6153,63.2622,20.5668,,2.2
546 | "2019-11-23 17:00:00",1.7083,1.4297,59.6468,37.4644,2.5862,2.46,67.3745,18.3807,,3.6
547 | "2019-11-23 18:00:00",3.4062,3.0432,58.784,41.42,1.8267,1.8078,70.3168,17.3127,,2.7
548 | "2019-11-23 19:00:00",8.2672,7.7277,51.068,52.9167,2.8735,2.8262,62.474,22.659,,4.4
549 | "2019-11-23 20:00:00",12.0267,10.888,45.595,63.645,8.9105,8.3357,52.4246,35.6605,,8.9
550 | "2019-11-23 21:00:00",20.1871,18.3802,43.4224,80.8525,19.906,19.0793,48.9777,43.8785,,11.6
551 | "2019-11-23 22:00:00",25.0967,22.6792,41.777,90.9867,21.1003,21.3203,44.8223,51.5583,,12.5
552 | "2019-11-23 23:00:00",11.2807,10.1315,41.498,86.4983,16.8167,16.8475,40.7097,56.5387,,9.3
553 | "2019-11-24 00:00:00",5.1753,4.6188,41.09,76.2533,8.9198,9.3313,38.1897,58.7025,,5.7
554 | "2019-11-24 01:00:00",7.593,6.8727,39.209,78.1833,13.718,14.1327,37.2927,59.0615,,8.4
555 | "2019-11-24 02:00:00",5.2367,4.5178,38.501,77.5917,27.4648,27.4027,35.8423,60.409,,14.5
556 | "2019-11-24 03:00:00",3.363,3.0037,37.991,72.0083,12.9907,13.1813,35.5643,59.6025,,7.5
557 | "2019-11-24 04:00:00",1.7457,1.6528,38.816,63.8767,15.071,15.3707,35.3197,58.2695,,8.7
558 | "2019-11-24 05:00:00",1.2497,1.2957,39.38,59.5276,18.5856,18.5797,34.3787,59.0013,,9.8
559 | "2019-11-24 06:00:00",0.7644,0.6234,39.9292,54.4983,14.9017,14.8968,32.9727,61.0502,,7.8
560 | "2019-11-24 07:00:00",0.4497,0.481,40.241,52.3883,9.6361,9.3074,32.5807,61.5262,,5.3
561 | "2019-11-24 08:00:00",0.8572,0.778,39.602,51.53,9.3423,9.4438,31.962,61.1669,,5.4
562 | "2019-11-24 09:00:00",0.4297,0.4365,38.744,50.0667,4.4615,4.5833,33.9057,51.2698,,3.6
563 | "2019-11-24 10:00:00",0.121,0.0965,40.656,41.7633,1.0903,1.0963,41.8607,31.8675,,2
564 | "2019-11-24 11:00:00",0.1775,0.1085,45.254,36.97,1.5977,1.5505,45.0987,27.0567,,2.6
565 | "2019-11-24 12:00:00",1.2083,1.0365,56.921,30.4417,2.5139,2.5713,53.0585,22.3865,,4.2
566 | "2019-11-24 13:00:00",2.9949,2.4141,65.4424,24.322,2.492,2.5905,62.3642,16.4743,,3.6
567 | "2019-11-24 14:00:00",0.8267,0.7124,72.8848,20.7276,0.328,0.484,68.408,10.1073,,1.6
568 | "2019-11-24 15:00:00",0.5833,0.4042,76.241,16.9617,0.2785,0.291,74.9752,4.3675,,2.6
569 | "2019-11-24 16:00:00",0.2895,0.2325,71.9661,17.5932,1.3167,1.1415,75.982,4.0998,,1.1
570 | "2019-11-24 17:00:00",0.2055,0.1412,66.413,18.6567,0.5934,0.5345,77.4905,3.5656,,2.1
571 | "2019-11-24 18:00:00",0.7033,0.5518,65.1265,21.0945,0.5428,0.6793,77.2147,3.6422,,1.7
572 | "2019-11-24 19:00:00",0.7172,0.5322,59.81,21.6966,0.5718,0.699,67.0513,6.7163,,2.3
573 | "2019-11-24 20:00:00",0.4085,0.3359,56.721,22.7492,0.9002,1.1164,59.4293,10.1361,,1.6
574 | "2019-11-24 21:00:00",0.359,0.203,53.974,25.135,0.8097,1.0498,57.2707,12.8603,,1.4
575 | "2019-11-24 22:00:00",1.6286,1.519,51.4369,28.9797,1.0982,1.2885,54.632,15.5187,,2.4
576 | "2019-11-24 23:00:00",2.7473,2.78,47.6273,32.7382,2.0761,2.1953,52.4804,17.5686,,3.3
577 | "2019-11-25 00:00:00",0.6456,0.6281,46.7983,34.2111,0.6828,1.0547,54.7956,17.2725,,2.5
578 | "2019-11-25 01:00:00",2.8404,2.6282,47.4674,36.714,2.4527,2.4388,51.0562,20.356,,3.2
579 | "2019-11-25 02:00:00",3.7362,3.5347,43.832,43.2567,3.4397,3.1772,47.2705,26.2325,,4
580 | "2019-11-25 03:00:00",3.3702,3.2717,41.2197,49.2881,3.5105,3.264,44.676,30.0875,,3.1
581 | "2019-11-25 04:00:00",1.5713,1.4535,40.958,49.1633,3.7688,3.552,40.1347,37.1475,,3.6
582 | "2019-11-25 05:00:00",2.073,1.9453,37.757,56.7817,3.9443,3.5395,38.1635,42.3933,,3.6
583 | "2019-11-25 06:00:00",1.7397,1.6592,36.209,63.5183,4.817,4.7962,36.2672,46.2948,,3.8
584 | "2019-11-25 07:00:00",1.3173,1.2542,36.821,64.6417,4.8965,4.7733,35.446,47.4765,,4.1
585 | "2019-11-25 08:00:00",1.8215,1.6788,36.548,64.225,5.6573,5.6702,34.9492,50.114,,4.8
586 | "2019-11-25 09:00:00",4.876,4.2212,36.062,67.0217,7.1712,6.917,35.719,49.3603,,7.3
587 | "2019-11-25 10:00:00",3.7829,3.4109,36.1432,68.5929,11.0853,10.9915,37.7792,46.142,,11.2
588 | "2019-11-25 11:00:00",8.2922,7.6848,40.088,69.1733,8.9377,9.0897,43.407,39.865,,10.9
589 | "2019-11-25 12:00:00",4.0022,3.8246,47.1067,58.5778,7.9755,7.4198,44.7462,40.889,,5.3
590 | "2019-11-25 13:00:00",2.8072,2.7598,47.6522,58.3696,5.923,5.5297,44.9342,41.532,,3.2
591 | "2019-11-25 14:00:00",3.6472,3.5933,50.006,54.0267,8.0843,7.5721,45.9555,39.9933,,5.7
592 | "2019-11-25 15:00:00",3.9219,3.7757,45.1897,61.8741,7.8882,7.3797,43.3077,44.3992,,5.1
593 | "2019-11-25 16:00:00",2.9876,2.9262,42.1203,69.8276,7.8512,7.3863,42.532,46.7657,,4.5
594 | "2019-11-25 17:00:00",2.6565,2.4758,40.22,74.8467,5.0358,4.81,41.4167,48.9792,,3.3
595 | "2019-11-25 18:00:00",3.5305,3.2773,38.5868,81.2017,7.1598,6.9866,40.4815,52.3543,,7.4
596 | "2019-11-25 19:00:00",7.4807,6.6081,35.9507,97.2569,11.2448,10.9535,38.1732,65.0873,,5.1
597 | "2019-11-25 20:00:00",11.4358,9.6831,34.7031,99.9,15.7185,15.5636,36.792,71.6333,,7.3
598 | "2019-11-25 21:00:00",10.9953,9.3627,33.8386,98.1898,15.5907,15.5062,36.4618,71.206,,8.7
599 | "2019-11-25 22:00:00",10.9047,9.4193,33.881,99.9,15.4611,15.2953,35.7773,70.1213,,8.3
600 | "2019-11-25 23:00:00",7.6942,6.7523,33.155,99.9,13.714,14.0975,34.6823,70.9187,,7.5
601 | "2019-11-26 00:00:00",5.4192,4.5098,31.997,99.9,8.5619,8.4039,33.1211,71.779,,4.2
602 | "2019-11-26 01:00:00",4.6281,3.6285,30.8529,99.9,6.0213,6.111,31.2263,74.8062,,2.2
603 | "2019-11-26 02:00:00",3.6252,2.7692,30.119,99.9,5.1428,4.7215,30.3738,76.5842,,1.9
604 | "2019-11-26 03:00:00",1.5467,1.4332,29.429,99.9,3.7937,3.5918,29.7093,77.4938,,1.3
605 | "2019-11-26 04:00:00",2.4608,2.0667,28.831,98.2183,2.0802,1.8443,28.9338,77.7413,,0.9
606 | "2019-11-26 05:00:00",1.4577,1.2028,29.288,99.9,1.5422,1.356,28.8723,77.7817,,0.7
607 | "2019-11-26 06:00:00",0.9028,0.68,29.132,99.9,3.4507,3.2907,29.0072,76.4047,,1.3
608 | "2019-11-26 07:00:00",0.3157,0.2343,28.7857,99.9,1.3325,1.2077,28.3273,75.6575,,0.7
609 | "2019-11-26 08:00:00",,,,,2.6145,2.5278,27.771,74.8222,,1.6
610 | "2019-11-26 09:00:00",,,,,2.5757,2.66,27.3235,74.3767,,1.4
611 | "2019-11-26 10:00:00",,,,,2.5247,2.6063,27.2513,74.8872,,1.5
612 | "2019-11-26 11:00:00",,,,,2.896,2.952,27.5662,74.582,,1.8
613 | "2019-11-26 12:00:00",,,,,1.5313,1.6448,27.8468,74.5358,,12.6
614 | "2019-11-26 13:00:00",,,,,2.5323,2.6162,28.7855,71.8577,,1.4
615 | "2019-11-26 14:00:00",,,,,4.2356,4.3115,29.5315,69.4534,,2.3
616 | "2019-11-26 15:00:00",,,,,4.5822,4.7405,29.5672,64.6633,,7.5
617 | "2019-11-26 16:00:00",,,,,5.4882,5.5983,29.8945,62.1717,,4.5
618 | "2019-11-26 17:00:00",,,,,6.8643,7.0487,29.8373,61.0508,,5.3
619 | "2019-11-26 18:00:00",,,,,7.7736,7.88,29.1379,60.1549,,6.2
620 | "2019-11-26 19:00:00",,,,,5.9187,5.9413,28.2738,60.0041,,5.2
621 | "2019-11-26 20:00:00",,,,,6.4718,6.4107,26.6246,62.6087,,3.5
622 | "2019-11-26 21:00:00",,,,,6.658,6.817,26.0569,62.2893,,3.4
623 | "2019-11-26 22:00:00",,,,,9.5782,9.9723,25.37,62.282,,5.6
624 | "2019-11-26 23:00:00",,,,,21.7702,22.1072,22.9477,67.2233,,10.7
625 | "2019-11-27 00:00:00",,,,,35.158,36.1227,20.8562,69.3097,,16.1
626 | "2019-11-27 01:00:00",,,,,34.6202,35.6843,19.2113,70.1527,,15.5
627 | "2019-11-27 02:00:00",,,,,16.95,17.0635,17.4985,67.8473,,7.9
628 | "2019-11-27 03:00:00",,,,,10.1247,10.175,16.213,66.3445,,4.9
629 | "2019-11-27 04:00:00",,,,,10.5326,10.4572,15.462,66.5774,,5.4
630 | "2019-11-27 05:00:00",,,,,11.5082,11.2672,13.8823,67.1538,,5.4
631 | "2019-11-27 06:00:00",,,,,12.6228,12.4345,12.2517,67.4525,,6.2
632 | "2019-11-27 07:00:00",,,,,14.6582,13.8452,11.8427,66.8003,,6.8
633 | "2019-11-27 08:00:00",,,,,17.4445,17.116,11.7282,67.8348,,7.7
634 | "2019-11-27 09:00:00",,,,,21.0605,19.9869,12.3285,68.1048,,8.5
635 | "2019-11-27 10:00:00",,,,,23.5162,22.7077,13.3977,68.8535,,8.9
636 | "2019-11-27 11:00:00",17.5193,14.38,22.688,99.3467,21.2547,21.2565,19.194,63.4813,,9.3
637 | "2019-11-27 12:00:00",10.6532,9.1678,37.3532,68.72,14.2765,14.2648,29.4823,49.361,,7.5
638 | "2019-11-27 13:00:00",8.334,7.21,42.049,51.4155,7.2713,7.1762,33.1928,37.6028,,5.1
639 | "2019-11-27 14:00:00",11.63,10.4258,44.792,51.1033,11.0265,11.0398,36.6772,34.8352,,8.8
640 | "2019-11-27 15:00:00",24.466,22.35,42.905,56.5933,22.3753,22.012,39.5113,33.8575,,11.7
641 | "2019-11-27 16:00:00",25.0193,22.5175,39.736,64.5933,24.4188,23.5,39.4517,34.6072,,11.6
642 | "2019-11-27 17:00:00",30.401,27.3692,34.06,80.92,33.47,32.6893,33.8142,43.7392,,13.7
643 | "2019-11-27 18:00:00",17.5058,15.4998,29.843,88.1033,22.745,22.3317,29.3195,48.4942,,9.1
644 | "2019-11-27 19:00:00",28.3897,24.843,24.317,90.26,23.0228,23.0627,24.8238,54.3608,,8.5
645 | "2019-11-27 20:00:00",39.1873,33.6456,21.2262,94.0236,29.4251,28.6925,23.1226,58.7539,,9.7
646 | "2019-11-27 21:00:00",41.0302,35.6332,20.2589,96.4368,30.2693,29.6157,22.1314,60.8081,,10
647 | "2019-11-27 22:00:00",44.1902,38.1708,19.8607,98.4746,40.6613,40.3646,21.52,62.5946,,12.5
648 | "2019-11-27 23:00:00",43.5688,37.7316,19.611,98.7379,42.8147,42.799,20.78,62.8022,,13.7
649 | "2019-11-28 00:00:00",46.4883,40.491,18.694,96.2567,52.4483,53.7313,19.6737,65.1523,,17.7
650 | "2019-11-28 01:00:00",47.465,41.7812,19.187,98.2083,54.5895,56.0223,19.806,65.5453,,19.5
651 | "2019-11-28 02:00:00",49.5972,43.2743,19.364,98.8617,54.314,56.2478,19.8713,66.2618,,19.3
652 | "2019-11-28 03:00:00",42.0507,36.1553,19.672,97.2733,48.4308,49.4223,20.836,65.0805,,16.5
653 | "2019-11-28 04:00:00",35.9058,29.9342,19.9878,97.1034,45.4025,46.7755,21.0098,65.0327,,15.6
654 | "2019-11-28 05:00:00",37.164,30.8652,20.1717,97.2862,43.2542,44.7482,20.9048,66.0055,,14.8
655 | "2019-11-28 06:00:00",37.5375,31.3243,20.669,99.345,40.3445,40.5637,21.2055,65.6898,,13.6
656 | "2019-11-28 07:00:00",41.7908,34.6395,20.855,99.85,42.4713,43.7917,21.334,67.0513,,14.8
657 | "2019-11-28 08:00:00",40.8357,34.3732,20.8786,99.9,43.4977,44.7143,21.5562,67.0197,,14.9
658 | "2019-11-28 09:00:00",,,,,42.6895,43.799,21.5087,67.194,,14.4
659 | "2019-11-28 10:00:00",45.9438,39.2714,19.2371,99.9,42.3718,43.3189,22.6028,66.6961,,14.9
660 | "2019-11-28 11:00:00",,,,,44.01,45.6075,24.7952,62.1889,,16.7
661 | "2019-11-28 12:00:00",45.9754,40.4068,30.6086,93.4432,39.8507,40.7958,31.5092,51.6007,,16
662 | "2019-11-28 13:00:00",40.2725,35.7614,39.1207,73.1271,37.1632,37.707,38.935,38.9452,,16.3
663 | "2019-11-28 14:00:00",39.0644,34.6026,38.3,73.2123,38.2182,39.3085,36.4502,42.0685,,16.3
664 | "2019-11-28 15:00:00",37.17,33.0062,43.5107,62.7586,36.037,36.0652,42.1908,34.7663,,15.8
665 | "2019-11-28 16:00:00",38.1865,34.2397,38.97,62.3633,38.237,37.8887,43.5702,33.9079,,17.6
666 | "2019-11-28 17:00:00",41.4125,36.9398,33.955,75.8633,37.2402,37.0029,46.0456,30.2547,,18.4
667 | "2019-11-28 18:00:00",42.5137,38.0022,34.508,79.4883,38.3273,38.4617,44.329,30.3998,,18.8
668 | "2019-11-28 19:00:00",46.6015,41.2807,28.9369,90.0644,42.9943,43.6893,35.6513,43.5615,,19.5
669 | "2019-11-28 20:00:00",50.305,43.9823,25.712,95.22,48.87,50.1818,29.3353,55.1815,,21
670 | "2019-11-28 21:00:00",54.813,48.039,24.518,97.8833,51.7938,54.2122,27.484,60.2113,,23
671 | "2019-11-28 22:00:00",59.8452,52.442,24.104,99.3,56.4258,59.286,25.7558,62.9197,,27.1
672 | "2019-11-28 23:00:00",62.698,55.3354,23.2441,99.8339,60.7538,63.9339,25.0836,66.8841,,29.3
673 | "2019-11-29 00:00:00",64.0458,56.2947,22.3441,99.9,67.6365,70.3797,23.8323,66.893,,35.2
674 | "2019-11-29 01:00:00",60.7356,52.7489,21.7874,99.9,68.9408,71.77,22.822,66.9473,,35.6
675 | "2019-11-29 02:00:00",65.7128,57.9357,21.371,99.9,63.483,66.498,22.7661,68.3815,,30.9
676 | "2019-11-29 03:00:00",63.157,55.3007,19.919,99.9,63.634,66.3652,22.4233,67.977,,31.1
677 | "2019-11-29 04:00:00",63.199,54.624,18.2466,96.4207,65.4333,68.1607,20.9158,70.094,,32.6
678 | "2019-11-29 05:00:00",54.7653,47.5692,18.818,99.9,60.0452,62.7167,21.347,70.119,,28.9
679 | "2019-11-29 06:00:00",61.1588,53.3675,18.995,99.9,60.1816,62.7518,20.9456,69.4795,,28.9
680 | "2019-11-29 07:00:00",,,,,52.6438,54.5908,20.3227,70.6343,,23.3
681 | "2019-11-29 08:00:00",,,,,45.0892,47.2907,20.4897,70.7572,,18.9
682 | "2019-11-29 09:00:00",,,,,40.6418,42.2705,21.1647,70.7608,,16.5
683 | "2019-11-29 10:00:00",43.2367,31.7467,19.52,99.9,39.8915,41.728,21.5655,71.6282,,16.5
684 | "2019-11-29 11:00:00",44.6059,37.9294,21.1025,96.7469,41.8363,44.6108,23.8068,71.6213,,18.2
685 | "2019-11-29 12:00:00",40.53,32.65,26.3,99.9,41.3612,43.2058,27.4348,67.5902,,18.4
686 | "2019-11-29 13:00:00",,,,,39.9813,41.712,29.3957,65.2762,,18.4
687 | "2019-11-29 14:00:00",43.552,37.477,30.578,96.93,38.1189,39.6467,32.6385,58.643,,18.3
688 | "2019-11-29 15:00:00",,,,,37.561,38.9082,34.2128,55.373,,19.4
689 | "2019-11-29 16:00:00",,,,,40.7437,42.2667,34.5825,55.3623,,21
690 | "2019-11-29 17:00:00",,,,,42.902,44.2473,32.8285,59.8695,,21.3
691 | "2019-11-29 18:00:00",,,,,44.0527,45.7655,31.4125,62.7057,,20.5
692 | "2019-11-29 19:00:00",,,,,48.2972,49.3774,30.3057,65.2893,,23.1
693 | "2019-11-29 20:00:00",,,,,46.7253,48.655,28.4978,69.7705,,21.5
694 | "2019-11-29 21:00:00",,,,,45.9079,47.8851,28.8202,70.1674,,21.2
695 | "2019-11-29 22:00:00",,,,,42.8473,44.5012,29.114,72.8557,,21.8
696 | "2019-11-29 23:00:00",,,,,32.8232,33.7427,30.225,73.3475,,15.2
697 | "2019-11-30 00:00:00",,,,,39.6573,41.3257,29.784,71.7775,,21.6
698 | "2019-11-30 01:00:00",,,,,47.0038,49.2843,29.1628,72.4322,,26.7
699 | "2019-11-30 02:00:00",,,,,46.4113,48.696,28.9228,71.3865,,23.3
700 | "2019-11-30 03:00:00",,,,,31.1543,32.9355,30.1227,66.3737
701 | "2019-11-30 04:00:00",,,,,6.5483,6.7893,36.1512,39.4255
702 | "2019-11-30 05:00:00",,,,,0.2036,0.2593,39.0941,28.1376
703 | "2019-11-30 06:00:00",,,,,0.0344,0.0538,39.019,28.0744
704 | "2019-11-30 07:00:00",,,,,0.0653,0.0682,38.7102,29.9242
705 | "2019-11-30 08:00:00",,,,,0.0367,0.038,39.1997,27.5337
706 | "2019-11-30 09:00:00",,,,,0.0655,0.0475,39.063,27.0693
707 | "2019-11-30 10:00:00",0,0,32,67.3,0.19,0.2174,38.9695,28.4038
708 | "2019-11-30 11:00:00",2.3633,1.9727,38.6382,71.1515,2.3284,2.1844,40.6908,33.4957
709 | "2019-11-30 12:00:00",0.3862,0.245,39.2528,55.6966,0.6107,0.6688,40.9492,29.4788
710 | "2019-11-30 13:00:00",1.1367,0.8553,39.086,53.97,1.0147,0.8795,40.4393,29.0318
711 | "2019-11-30 14:00:00",0.2362,0.26,40.066,49.1755,0.485,0.6063,40.5012,27.9475
712 | "2019-11-30 15:00:00",0.2202,0.1719,40.17,48.2389,1.0928,1.0275,40.2673,28.2655
713 | "2019-11-30 16:00:00",0.218,0.1651,39.1171,44.4327,1.0716,1.0362,40.6418,24.539
714 | "2019-11-30 17:00:00",0.6785,0.6008,39.311,47.3367,0.7858,0.7853,40.6152,24.0637
715 | "2019-11-30 18:00:00",0.3182,0.3218,39.392,45.5883,1.3883,1.4028,42.9015,22.0512
716 | "2019-11-30 19:00:00",0.479,0.4395,34.702,48.4083,1.0132,0.9378,39.5887,24.6278
717 | "2019-11-30 20:00:00",0.7942,0.6907,33.443,52.445,0.6455,0.6087,37.0132,27.1767
718 | "2019-11-30 21:00:00",3.5258,3.001,30.896,65.285,3.722,3.6113,36.3033,32.2917
719 | "2019-11-30 22:00:00",3.25,3.01,27.77,77.95,4.5759,4.7103,29.6456,47.008
720 | "2019-11-30 23:00:00",,,,,6.4475,6.5263,28.1527,49.3392
721 | "2019-12-01 00:00:00",,,,,8.2585,8.1857,26.927,51.6115
722 | "2019-12-01 01:00:00",,,,,11.6065,11.4035,25.7713,54.1963
723 | "2019-12-01 02:00:00",,,,,12.564,12.9533,24.6187,55.8362
724 | "2019-12-01 03:00:00",,,,,7.0743,7.1623,23.433,53.9478
725 | "2019-12-01 04:00:00",,,,,4.6102,4.6063,21.9305,54.8565
726 | "2019-12-01 05:00:00",,,,,5.4557,5.3712,20.8512,57.016
727 | "2019-12-01 06:00:00",,,,,7.4127,7.3685,19.9165,59.7805
728 | "2019-12-01 07:00:00",,,,,3.2656,3.231,20.092,57.9703
729 | "2019-12-01 08:00:00",,,,,2.2293,2.3589,19.2831,57.1215
730 | "2019-12-01 09:00:00",,,,,2.219,2.102,18.4977,57.0455
731 | "2019-12-01 10:00:00",5.0814,4.0343,18.14,96.3857,3.4672,3.3948,19.8263,55.9853
732 | "2019-12-01 11:00:00",3.5964,2.8594,29.6764,81.9212,6.262,6.0898,26.915,45.5477
733 | "2019-12-01 12:00:00",5.9873,5.4385,46.025,45.9533,3.5577,3.3437,39.058,29.749
734 | "2019-12-01 13:00:00",9.031,7.9426,44.3797,45.4948,8.0962,7.5977,43.9147,25.3363
735 | "2019-12-01 14:00:00",8.6031,7.9064,49.8658,40.5678,11.6113,10.8598,44.5908,28.6407
736 | "2019-12-01 15:00:00",13.7403,12.9518,52.253,39.2167,17.1493,15.9032,46.455,27.4442
737 | "2019-12-01 16:00:00",13.7368,13.0802,45.122,47.3417,17.8782,16.3908,49.1838,26.0262
738 | "2019-12-01 17:00:00",15.9993,14.9592,38.1749,58.1695,16.7662,15.1713,49.3638,25.7687
739 | "2019-12-01 18:00:00",15.9943,14.7181,37.3597,61.6241,17.5475,16.1132,49.8762,25.0533
740 | "2019-12-01 19:00:00",15.3707,13.8663,31.003,74.315,17.6133,16.828,39.8978,36.2612
741 | "2019-12-01 20:00:00",15.5646,13.752,27.9424,86.8,17.3021,16.4738,32.8077,47.1302
742 | "2019-12-01 21:00:00",21.1048,18.67,26.6762,93.3327,23.7772,23.0089,30.1951,53.4323
743 | "2019-12-01 22:00:00",24.6757,21.61,25.6694,95.3528,38.118,39.0777,29.4407,58.6948
744 | "2019-12-01 23:00:00",35.6395,31.4077,22.946,92.9633,42.9881,44.4698,27.8074,61.9463
745 | "2019-12-02 00:00:00",31.6037,27.2717,23.123,99.1967,51.6987,53.8175,26.263,64.4707
746 | "2019-12-02 01:00:00",40.0365,34.758,22.466,99.9,54.5302,56.9053,24.8617,64.7977
747 | "2019-12-02 02:00:00",44.6075,39.1432,22.253,99.9,60.7307,63.8977,23.4347,66.2268
748 | "2019-12-02 03:00:00",46.485,41.2358,21.881,99.9,56.7813,59.1047,22.3308,66.8653
749 | "2019-12-02 04:00:00",47.5007,42.3467,21.665,99.9,58.6825,61.4037,21.7887,67.4978
750 | "2019-12-02 05:00:00",46.8633,41.6437,21.361,98.2183,55.1115,57.0347,21.1912,67.5933,,23.8
751 | "2019-12-02 06:00:00",45.9557,40.7315,22.034,99.9,51.4165,53.2812,21.3022,67.605,,21.2
752 | "2019-12-02 07:00:00",41.4525,36.2257,22.774,98.2183,46.4092,48.0688,22.115,66.7943,,18.2
753 | "2019-12-02 08:00:00",43.3487,38.3175,23.951,99.885,40.8108,41.9,23.3621,65.3416,,15.6
754 | "2019-12-02 09:00:00",41.8031,36.9514,24.1837,99.7102,39.9336,40.8464,25.0608,63.4561,,15.4
755 | "2019-12-02 10:00:00",35.8095,32.0466,26.9764,98.3614,37.4931,38.342,26.6,61.9162,,14.2
756 | "2019-12-02 11:00:00",9.9325,8.8638,37.168,76.1467,35.274,35.6961,34.9887,52.2724,,16.2
757 | "2019-12-02 12:00:00",2.0013,1.8913,50.915,46.5083,16.284,15.0572,50.3842,32.8502,,9.4
758 | "2019-12-02 13:00:00",2.5132,2.3922,53.843,44.36,7.6303,6.9037,57.0405,25.582,,5.1
759 | "2019-12-02 14:00:00",15.5422,15.1273,61.495,38.635,13.2627,11.6843,63.1918,21.018,,7.7
760 | "2019-12-02 15:00:00",19.7595,19.489,63.6107,35.5052,33.5352,31.1617,63.9368,20.2407,,16
761 | "2019-12-02 16:00:00",27.2308,26.1758,56.899,41.4167,31.448,29.7827,63.442,21.1515,,14.5
762 | "2019-12-02 17:00:00",22.1561,20.59,49.2678,48.039,24.0398,22.4677,65.8373,17.7797,,11.8
763 | "2019-12-02 18:00:00",18.3461,17.0375,48.6532,48.3161,19.2467,17.9785,62.8083,18.6353,,8.5
764 | "2019-12-02 19:00:00",20.675,18.9427,43.436,54.9667,20.9278,19.613,52.3127,25.9232,,8.8
765 | "2019-12-02 20:00:00",23.8518,21.813,41.512,59.0383,31.7962,31.6267,44.7403,39.9911,,13.7
766 | "2019-12-02 21:00:00",25.2432,22.9152,43.295,59.4333,34.7754,35.7161,42.3226,45.0061,,15.6
767 | "2019-12-02 22:00:00",33.7755,30.7097,41.321,68.3033,43.6025,44.9042,40.2205,50.2723,,21.3
768 | "2019-12-02 23:00:00",31.7473,28.543,37.717,81.0583,33.5735,34.2433,38.6637,54.755,,14
769 | "2019-12-03 00:00:00",25.5145,22.5108,36.248,90.5933,18.0605,17.7164,37.9359,54.308,,7.3
770 | "2019-12-03 01:00:00",19.5712,17.4197,34.619,92.7,24.2673,23.9598,37.7685,54.276,,10.1
771 | "2019-12-03 02:00:00",21.5287,19.537,34.751,94.135,16.7607,16.6511,38.5756,50.5652,,7.5
772 | "2019-12-03 03:00:00",25.4381,22.5605,34.5728,94.8103,14.0835,13.8793,38.8748,49.0925,,6.7
773 | "2019-12-03 04:00:00",19.7335,17.5335,33.511,93.4967,9.0412,8.95,40.1745,45.5157,,4.6
774 | "2019-12-03 05:00:00",9.8672,8.9335,35.945,90.1183,7.133,7.2883,39.5182,46.3683,,3.9
775 | "2019-12-03 06:00:00",12.4703,11.483,35.408,90.6383,5.5505,5.0657,38.9407,46.9182,,2.9
776 | "2019-12-03 07:00:00",4.5408,4.0922,37.145,85.5483,7.0284,6.8862,37.8933,49.2298,,3.8
777 | "2019-12-03 08:00:00",2.5308,2.3372,36.428,85.845,6.6823,6.7953,37.6508,49.9387,,3.6
778 | "2019-12-03 09:00:00",1.316,1.2558,38.689,79.2867,3.7167,3.4768,42.0923,43.2548,,3.2
779 | "2019-12-03 10:00:00",0.1053,0.1103,46.8729,59.8898,1.4378,1.335,50.7058,30.8515,,0.8
780 | "2019-12-03 11:00:00",0.0635,0.0318,51.455,50.46,1.1485,1.1439,53.6485,27.0633,,2.6
781 | "2019-12-03 12:00:00",0.113,0.1002,59.3407,38.0607,3.103,3.1325,56.5966,23.9859,,3.7
782 | "2019-12-03 13:00:00",0.2425,0.2039,58.4478,37.5593,1.951,2.0228,57.6541,22.497,,3.6
783 | "2019-12-03 14:00:00",0.2781,0.1871,65.1536,31.7814,2.1236,2.2864,58.5179,21.2043,,5.8
784 | "2019-12-03 15:00:00",0.7161,0.6518,71.2207,27.7107,2.803,2.724,62.3014,18.164,,4.9
785 | "2019-12-03 16:00:00",0.4241,0.2778,67.8122,28.1174,2.3143,2.4643,65.6998,15.229,,5.3
786 | "2019-12-03 17:00:00",0.3498,0.2239,58.4,31.9632,4.8068,4.1048,71.1332,12.487,,6
787 | "2019-12-03 18:00:00",1.0095,0.8392,57.2671,33.0288,3.4665,3.109,69.2248,12.5582,,5.6
788 | "2019-12-03 19:00:00",3.7266,3.3898,51.7481,38.6203,3.1946,3.2379,59.9907,16.3034,,7.4
789 | "2019-12-03 20:00:00",6.043,5.903,46.562,46.6317,6.1398,5.7703,53.2108,23.6693,,5.4
790 | "2019-12-03 21:00:00",2.4498,2.7153,45.881,44.4117,6.8359,6.3725,49.9684,26.4841,,5.3
791 | "2019-12-03 22:00:00",7.7088,6.9833,43.535,49.225,6.4243,5.9356,46.3598,31.4598,,6.2
792 | "2019-12-03 23:00:00",12.4803,11.3028,39.817,59.045,13.2813,13.2032,41.8695,45.4372,,8.6
793 | "2019-12-04 00:00:00",9.2032,8.1772,38.984,68.0067,8.1428,8.5577,38.5872,50.3892,,5.8
794 | "2019-12-04 01:00:00",3.9977,3.7105,39.166,59.4583,6.45,6.6835,36.7887,51.6842,,4.9
795 | "2019-12-04 02:00:00",2.128,2.03,39.71,56.4617,5.6167,5.757,35.3752,52.0787,,4.2
796 | "2019-12-04 03:00:00",1.3068,1.2055,37.837,57.5967,3.5302,3.419,35.1063,50.5322,,3
797 | "2019-12-04 04:00:00",0.9175,0.9273,37.702,59.5533,3.9242,3.8295,35.9145,48.4128,,3
798 | "2019-12-04 05:00:00",0.3417,0.2538,38.888,58.1817,3.4823,3.1193,36.8212,47.3557,,2.6
799 | "2019-12-04 06:00:00",0.0914,0.0783,40.7285,53.7661,3.4132,3.054,36.4122,49.934,,2.9
800 | "2019-12-04 07:00:00",0.0405,0.0288,40.526,53.99,3.3165,3.0623,36.9787,49.1185,,2.6
801 | "2019-12-04 08:00:00",0.8823,0.8085,39.332,58.21,5.4518,5.4408,36.7908,51.1178,,3.9
802 | "2019-12-04 09:00:00",1.3108,1.17,38.788,60.765,9.004,8.9876,38.0095,49.3824,,5.5
803 | "2019-12-04 10:00:00",6.7226,5.8842,38.7863,71.586,11.6448,11.0177,38.8297,49.1805,,7.3
804 | "2019-12-04 11:00:00",8.4481,7.7208,40.6942,67.2635,18.1508,17.79,41.2077,45.4743,,11.5
805 | "2019-12-04 12:00:00",13.0727,12.2911,50.864,53.3036,10.9247,10.4512,48.5078,34.4595,,9.9
806 | "2019-12-04 13:00:00",6.9066,6.6092,56.8939,38.0966,12.8428,12.2998,53.7993,27.778,,10.4
807 | "2019-12-04 14:00:00",7.4935,6.97,65.5021,32.3895,7.1968,6.9085,60.3027,21.4017,,6.8
808 | "2019-12-04 15:00:00",7.5655,7.1161,68.27,29.2196,6.199,5.775,62.2052,19.7268,,6.1
809 | "2019-12-04 16:00:00",29.1833,28.1306,63.6422,38.6981,20.7723,19.8805,66.4703,19.5087,,15.4
810 | "2019-12-04 17:00:00",36.7715,35.0578,55.286,49.155,40.1737,38.4018,68.921,20.5978,,23
811 | "2019-12-04 18:00:00",31.169,29.252,55.2884,49.866,35.2071,33.6947,68.5602,19.4762,,20
812 | "2019-12-04 19:00:00",42.0883,39.105,48.6535,65.8346,40.1068,39.434,58.9905,30.5888,,24.1
813 | "2019-12-04 20:00:00",43.2547,39.755,43.556,77.3,43.8967,44.4982,49.7743,42.2507,,23.5
814 | "2019-12-04 21:00:00",40.1898,36.4618,40.8295,83.0456,33.581,34.0087,46.4527,44.3113,,20.1
815 | "2019-12-04 22:00:00",47.8377,43.6233,37.924,88.7367,37.3587,38.4513,43.2397,49.5392,,21.8
816 | "2019-12-04 23:00:00",53.3135,48.3415,35.683,93.075,48.4405,49.8638,40.274,55.5933,,26.9
817 | "2019-12-05 00:00:00",54.7012,49.6937,36.332,97.525,49.8545,51.5075,39.5617,58.5232,,29.5
818 | "2019-12-05 01:00:00",44.1207,40.4705,38.012,96.28,48.1103,49.306,39.3513,59.2727,,28.7
819 | "2019-12-05 02:00:00",30.8602,28.0557,39.299,93.2933,13.5238,13.2761,39.4293,55.6785,,8.5
820 | "2019-12-05 03:00:00",25.7565,23.4603,39.626,92.5733,7.7717,7.896,39.8682,54.121,,5.9
821 | "2019-12-05 04:00:00",2.1598,2.1153,40.867,80.36,6.408,6.4754,40.0752,53.3674,,5.1
822 | "2019-12-05 05:00:00",2.8436,2.8064,41.0081,80.3797,8.5042,8.5092,40.3503,53.5488,,6.1
823 | "2019-12-05 06:00:00",3.1773,3.1117,41.762,82.25,21.3845,21.3585,40.2558,55.9123,,12.6
824 | "2019-12-05 07:00:00",16.2947,14.4665,41.483,86.5267,22.7302,22.6513,40.4059,56.9656,,12.1
825 | "2019-12-05 08:00:00",20.324,18.2753,40.874,89.07,21.6185,21.6344,41.8328,53.7359,,11.4
826 | "2019-12-05 09:00:00",21.4733,18.8625,40.904,85.76,21.9217,21.8852,43.3688,47.0973,,12.7
827 | "2019-12-05 10:00:00",12.9714,11.3372,41.3817,86.6121,12.8538,12.6259,45.0879,46.0052,,8.4
828 | "2019-12-05 11:00:00",5.5125,5.1661,44.2308,87.8814,19.275,19.1397,45.9912,47.6171,,11.7
829 | "2019-12-05 12:00:00",8.6194,8.016,44.6242,93.2154,24.1195,24.1212,45.8119,53.6761,,15.5
830 | "2019-12-05 13:00:00",17.3358,15.3873,43.49,97.8117,35.2912,36.4712,43.7967,67.4917,,25.4
831 | "2019-12-05 14:00:00",15.8175,14.3345,42.385,98.2183,25.5468,25.7653,43.7717,70.86,,14
832 | "2019-12-05 15:00:00",8.5203,7.6892,44.174,99.9,14.9119,14.9712,44.6134,71.1037,,9.2
833 | "2019-12-05 16:00:00",7.3852,6.8028,44.306,99.9,11.5987,11.291,44.9503,69.0877,,5.6
834 | "2019-12-05 17:00:00",6.0318,5.5437,41.954,99.9,14.1945,13.148,44.4605,65.978,,3.6
835 | "2019-12-05 18:00:00",8.1215,7.098,37.348,98.2183,14.8002,13.6772,40.1275,71.7158,,3.7
836 | "2019-12-05 19:00:00",8.6397,7.7517,37.439,99.9,13.6473,12.666,39.8562,73.6572,,3.7
837 | "2019-12-05 20:00:00",9.5495,8.3673,35.818,97.439,13.8967,12.8718,39.2393,73.3072,,4.3
838 | "2019-12-05 21:00:00",,,,,19.4838,19.1998,39.2762,73.335,,11
839 | "2019-12-05 22:00:00",,,,,16.9985,17.1817,38.936,73.2363,,12.4
840 | "2019-12-05 23:00:00",,,,,14.9607,14.9663,38.9047,72.8915,,9
841 | "2019-12-06 00:00:00",,,,,16.59,16.4395,38.7012,71.7588,,8.3
842 | "2019-12-06 01:00:00",,,,,24.7366,24.3561,37.1239,73.0816,,10.5
843 | "2019-12-06 02:00:00",,,,,34.5678,35.1705,36.1562,75.0022,,14
844 | "2019-12-06 03:00:00",,,,,35.8577,36.4931,35.5516,75.2479,,14.4
845 | "2019-12-06 04:00:00",,,,,33.6303,33.3788,34.4193,74.5752,,13.8
846 | "2019-12-06 05:00:00",,,,,33.8607,34.8361,33.5884,75.5843,,13.2
847 | "2019-12-06 06:00:00",,,,,31.279,30.9697,32.4503,75.1967,,12
848 | "2019-12-06 07:00:00",,,,,30.3272,30.3307,31.7251,75.3087,,11.3
849 | "2019-12-06 08:00:00",,,,,28.6745,28.812,31.4933,75.5995,,10.8
850 | "2019-12-06 09:00:00",,,,,32.0088,31.9877,31.1367,76.1483,,12
851 | "2019-12-06 10:00:00",40.9736,33.88,27.0255,99.9,33.3828,33.6157,31.7525,75.2305,,12.9
852 | "2019-12-06 11:00:00",28.3549,23.4994,35.0291,99.9,29.9405,29.9973,36.4603,70.9937,,12.4
853 | "2019-12-06 12:00:00",19.1012,16.0122,47.2237,99.6576,21.8277,20.4902,44.1748,58.5797,,9.8
854 | "2019-12-06 13:00:00",16.4553,14.402,49.604,92.04,14.5906,13.7568,49.1962,48.626,,8
855 | "2019-12-06 14:00:00",11.2048,10.3738,59.822,66.2583,10.7733,10.2952,55.029,38.5216,,7.8
856 | "2019-12-06 15:00:00",11.2261,10.5573,65.1169,50.0475,9.2612,8.8752,62.258,28.3725,,8.1
857 | "2019-12-06 16:00:00",19.6542,18.553,59.996,54.955,24.6843,22.6453,64.4928,26.8613,,15.2
858 | "2019-12-06 17:00:00",25.2162,23.619,51.176,76.1583,27.8462,25.6557,65.1218,27.3208,,17.6
859 | "2019-12-06 18:00:00",25.2355,23.448,50.321,84.26,30.1089,28.1523,61.1361,33.279,,15.9
860 | "2019-12-06 19:00:00",28.8128,26.3081,44.8615,96.4472,33.498,32.0385,53.3915,45.1287,,17.1
861 | "2019-12-06 20:00:00",33.7671,30.6688,39.995,96.2964,36.5382,36.5285,47.3267,55.9848,,17.7
862 | "2019-12-06 21:00:00",36.4695,32.6947,39.389,99.9,38.6403,39.2937,44.8762,60.2693,,19.7
863 | "2019-12-06 22:00:00",41.0223,36.9378,37.676,99.9,43.9734,44.9051,42.429,65.1089,,21.2
864 | "2019-12-06 23:00:00",36.6557,32.8322,36.764,99.9,39.8462,39.9772,40.8616,65.459,,19.4
865 | "2019-12-07 00:00:00",17.3576,15.2364,37.1207,99.9,30.6222,30.6393,38.8782,64.4087,,14.4
866 | "2019-12-07 01:00:00",13.8128,11.82,36.965,99.9,27.4585,27.6755,37.0583,67.1452,,12.4
867 | "2019-12-07 02:00:00",15.063,13.0875,35.18,99.9,22.0742,21.8405,35.8398,67.891,,10.5
868 | "2019-12-07 03:00:00",17.298,14.7187,34.763,99.9,31.318,31.7411,34.3557,68.2531,,14.4
869 | "2019-12-07 04:00:00",12.7662,11.2525,34.733,99.9,41.1937,42.6033,32.8535,70.404,,19
870 | "2019-12-07 05:00:00",10.443,9.2157,33.107,99.7667,23.7993,23.283,32.399,70.1267,,10.5
871 | "2019-12-07 06:00:00",5.7629,5.0903,34.5047,98.7136,30.5868,30.4465,31.551,69.6018,,13.4
872 | "2019-12-07 07:00:00",18.5312,15.8903,33.464,99.785,26.448,26.4155,31.3292,70.6865,,11.1
873 | "2019-12-07 08:00:00",21.112,18.7485,31.523,99.9,24.6723,24.739,31.2152,70.1795,,10.8
874 | "2019-12-07 09:00:00",17.8153,15.7217,31.766,99.9,14.8308,14.6795,31.7082,68.5764,,7.2
875 | "2019-12-07 10:00:00",17.1517,15.0559,26.703,82.3522,17.233,16.9105,33.3248,66.2755,,9
876 | "2019-12-07 11:00:00",0,0,-1,-1,19.1435,19.241,37.6127,60.2717,,12.7
877 | "2019-12-07 12:00:00",0,0,-1,-1,19.6515,18.8083,47.8537,44.04,,15
878 | "2019-12-07 13:00:00",0,0,-1,-1,15.0508,13.5582,53.3756,33.6121,,11.5
879 | "2019-12-07 14:00:00",0,0,-1,-1,10.1115,9.2633,63.8463,21.1433,,9.6
880 | "2019-12-07 15:00:00",0,0,-1,-1,6.497,5.8405,72.8433,13.858,,6.2
881 | "2019-12-07 16:00:00",0,0,-1,-1,1.8397,1.7215,76.787,9.38,,5.1
882 | "2019-12-07 17:00:00",0,0,-1,-1,2.6437,2.5347,70.5558,14.1292,,4.2
883 | "2019-12-07 18:00:00",0,0,-1,-1,1.6262,1.7282,67.2452,15.8492,,2.6
884 | "2019-12-07 19:00:00",0,0,-1,-1,1.6539,1.5031,63.1128,18.2433,,3.8
885 | "2019-12-07 20:00:00",0,0,-1,-1,2.188,2.456,59.2535,21.5667,,5.4
886 | "2019-12-07 21:00:00",0,0,-1,-1,3.7303,3.7158,55.655,26.7165,,5.7
887 | "2019-12-07 22:00:00",0,0,-1,-1,7.5993,7.155,52.0878,30.8457,,9.2
888 | "2019-12-07 23:00:00",0,0,-1,-1,11.4908,11.1218,48.7147,37.0257,,10.5
889 | "2019-12-08 00:00:00",0,0,-1,-1,10.7792,10.4285,45.7185,40.9897,,8.9
890 | "2019-12-08 01:00:00",0,0,-1,-1,13.5232,13.8253,43.1385,47.3758,,10.9
891 | "2019-12-08 02:00:00",0,0,-1,-1,15.9747,16.116,41.9568,48.908,,10.1
892 | "2019-12-08 03:00:00",0,0,-1,-1,11.013,11.018,40.3075,53.1522,,8.5
893 | "2019-12-08 04:00:00",0,0,-1,-1,9.4173,9.6638,39.6882,53.5978,,7
894 | "2019-12-08 05:00:00",0,0,-1,-1,14.9223,14.943,38.3433,56.039,,9
895 | "2019-12-08 06:00:00",0,0,-1,-1,9.453,9.3643,37.1227,59.0538,,5.7
896 | "2019-12-08 07:00:00",0,0,-1,-1,7.012,6.8367,36.0595,61.0117,,4.4
897 | "2019-12-08 08:00:00",0,0,-1,-1,4.3358,4.0927,36.2137,59.8605,,3.1
898 | "2019-12-08 09:00:00",0,0,-1,-1,2.6048,2.396,37.4088,56.1818,,2.2
899 | "2019-12-08 10:00:00",0,0,-1,-1,3.6548,3.2466,37.9698,55.3384,,3.1
900 | "2019-12-08 11:00:00",0,0,-1,-1,4.864,4.5662,41.8535,48.5092,,4.7
901 | "2019-12-08 12:00:00",0,0,-1,-1,4.9225,4.5163,47.4127,41.8832,,6
902 | "2019-12-08 13:00:00",0,0,-1,-1,2.8733,2.8098,54.6843,32.5246,,5.3
903 | "2019-12-08 14:00:00",0,0,-1,-1,2.2712,2.3755,56.4428,30.7628,,4.3
904 | "2019-12-08 15:00:00",0,0,-1,-1,1.1315,1.3205,62.3817,25.6197,,3.1
905 | "2019-12-08 16:00:00",0,0,-1,-1,0.4058,0.4052,75.5138,15.4653,,1.9
906 | "2019-12-08 17:00:00",0,0,-1,-1,1.3061,1.4152,74.1602,14.7716,,3
907 | "2019-12-08 18:00:00",0,0,-1,-1,0.7444,0.7692,70.361,14.2823,,3.6
908 | "2019-12-08 19:00:00",0,0,-1,-1,2.3775,2.5202,58.7531,22.9703,,4.2
909 | "2019-12-08 20:00:00",0,0,-1,-1,2.1768,2.344,53.5958,30.4166,,3.5
910 | "2019-12-08 21:00:00",0,0,-1,-1,1.5267,1.5685,53.2143,31.4265,,3
911 | "2019-12-08 22:00:00",0,0,-1,-1,0.4187,0.511,50.8107,33.6033,,1.9
912 | "2019-12-08 23:00:00",0,0,-1,-1,0.9044,0.7944,48.3477,31.4524,,1.5
913 | "2019-12-09 00:00:00",0,0,-1,-1,0.9853,1.0443,47.2703,29.4645,,3.8
914 | "2019-12-09 01:00:00",0,0,-1,-1,0.9002,0.9685,44.2898,31.2087,,2.4
915 | "2019-12-09 02:00:00",0,0,-1,-1,0.4008,0.4302,45.3695,28.8353,,1.5
916 | "2019-12-09 03:00:00",0,0,-1,-1,0.1475,0.172,46.0308,27.4192,,1.7
917 | "2019-12-09 04:00:00",0,0,-1,-1,1.65,1.3895,42.5805,40.3362,,1.9
918 | "2019-12-09 05:00:00",0,0,-1,-1,4.517,4.3767,35.2867,56.3407,,1.9
919 | "2019-12-09 06:00:00",0,0,-1,-1,7.7012,7.4868,34.13,57.2667,,3.4
920 | "2019-12-09 07:00:00",0,0,-1,-1,9.841,9.7417,33.4848,58.0145,,5.5
921 | "2019-12-09 08:00:00",0,0,-1,-1,12.6027,12.8048,32.1835,61.7025,,7.1
922 | "2019-12-09 09:00:00",0,0,-1,-1,13.4428,13.6232,31.8462,62.9987,,7.5
923 | "2019-12-09 10:00:00",0,0,-1,-1,11.5898,11.5907,31.8783,63.1743,,8
924 | "2019-12-09 11:00:00",0,0,-1,-1,13.2203,13.1143,36.5155,55.1678,,9.1
925 | "2019-12-09 12:00:00",0,0,-1,-1,8.7793,8.7695,41.7398,42.2368,,8.4
926 | "2019-12-09 13:00:00",0,0,-1,-1,4.8237,4.4942,44.8043,35.219,,8.7
927 | "2019-12-09 14:00:00",0,0,-1,-1,4.6605,4.3533,48.714,30.5417,,4.9
928 | "2019-12-09 15:00:00",0,0,-1,-1,4.0444,4.2267,52.4192,26.277,,5.9
929 | "2019-12-09 16:00:00",0,0,-1,-1,5.5598,5.3641,52.2961,26.408,,6
930 | "2019-12-09 17:00:00",0,0,-1,-1,6.0054,5.6964,49.7362,28.9041,,6.4
931 | "2019-12-09 18:00:00",0,0,-1,-1,5.6949,5.3485,50.9574,27.4633,,6.2
932 | "2019-12-09 19:00:00",0,0,-1,-1,4.7935,4.453,44.7922,33.3493,,5.7
933 | "2019-12-09 20:00:00",0,0,-1,-1,5.2275,4.9897,37.7013,43.0218,,6.8
934 | "2019-12-09 21:00:00",0,0,-1,-1,7.2183,7.3808,36.3423,45.8852,,8.9
935 | "2019-12-09 22:00:00",0,0,-1,-1,8.6025,8.5205,34.984,49.1352,,9.6
936 | "2019-12-09 23:00:00",0,0,-1,-1,10.8778,10.9118,33.6352,52.031,,8.8
937 | "2019-12-10 00:00:00",0,0,-1,-1,14.7307,14.9797,32.0122,56.491,,9.8
938 | "2019-12-10 01:00:00",0,0,-1,-1,12.7882,13.1387,32.0677,55.899,,9.5
939 | "2019-12-10 02:00:00",0,0,-1,-1,8.2722,8.0888,31.9923,55.7165,,5.7
940 | "2019-12-10 03:00:00",0,0,-1,-1,9.3785,9.0725,30.5343,58.7093,,6.3
941 | "2019-12-10 04:00:00",0,0,-1,-1,8.617,8.3658,29.2823,60.0462,,6
942 | "2019-12-10 05:00:00",0,0,-1,-1,10.0387,10.3875,27.7463,64.2413,,7.3
943 | "2019-12-10 06:00:00",0,0,-1,-1,9.2412,9.2282,26.9203,65.087,,6.9
944 | "2019-12-10 07:00:00",0,0,-1,-1,14.2235,14.3047,26.0275,66.393,,8.5
945 | "2019-12-10 08:00:00",0,0,-1,-1,17.711,17.9802,25.9417,66.5802,,9.9
946 | "2019-12-10 09:00:00",0,0,-1,-1,23.6733,23.2315,26.1633,67.7018,,13.1
947 | "2019-12-10 10:00:00",0,0,-1,-1,31.8589,31.5511,27.487,68.4303,,14
948 | "2019-12-10 11:00:00",0,0,-1,-1,32.5727,31.4697,31.7908,62.253,,13.8
949 | "2019-12-10 12:00:00",0,0,-1,-1,33.1084,32.6903,38.9215,47.8771,,19.1
950 | "2019-12-10 13:00:00",0,0,-1,-1,28.9712,28.1578,42.02,41.1105,,18.2
951 | "2019-12-10 14:00:00",0,0,-1,-1,26.9007,25.7422,44.4258,36.8465,,16.4
952 | "2019-12-10 15:00:00",0,0,-1,-1,24.3348,23.2878,48.431,31.1435
953 | "2019-12-10 16:00:00",0,0,-1,-1,23.7883,22.811,54.1145,25.4027,,16
954 | "2019-12-10 17:00:00",0,0,-1,-1,21.1662,20.0993,55.9682,22.9327,,14.4
955 | "2019-12-10 18:00:00",0,0,-1,-1,24.1812,23.3268,49.3965,29.0517,,16.6
956 | "2019-12-10 19:00:00",0,0,-1,-1,36.174,36.187,44.5248,36.3189,,29.3
957 | "2019-12-10 20:00:00",0,0,-1,-1,37.9556,38.3793,40.5387,43.0884,,30.5
958 | "2019-12-10 21:00:00",0,0,-1,-1,42.6972,43.8668,39.6463,46.024,,34.8
959 | "2019-12-10 22:00:00",0,0,-1,-1,33.7661,34.1883,37.772,47.2637,,18.4
960 | "2019-12-10 23:00:00",0,0,-1,-1,31.4825,31.4538,36.1144,50.0867,,15.5
961 | "2019-12-11 00:00:00",0,0,-1,-1,29.557,29.1538,34.9042,52.9405,,14.1
962 | "2019-12-11 01:00:00",0,0,-1,-1,24.8127,24.9512,33.8255,53.5388,,12.4
963 | "2019-12-11 02:00:00",0,0,-1,-1,21.967,21.9453,32.4135,55.9862,,11.4
964 | "2019-12-11 03:00:00",0,0,-1,-1,15.4818,15.593,31.5482,55.8338,,8.4
965 | "2019-12-11 04:00:00",0,0,-1,-1,15.751,15.776,31.0222,55.9667,,8.3
966 | "2019-12-11 05:00:00",0,0,-1,-1,13.1085,13.0638,30.0487,57.1325,,7
967 | "2019-12-11 06:00:00",0,0,-1,-1,17.1553,17.45,28.7332,58.5145,,9.4
968 | "2019-12-11 07:00:00",0,0,-1,-1,12.4115,12.6905,28.2387,59.3202,,7.2
969 | "2019-12-11 08:00:00",0,0,-1,-1,15.5368,15.5385,28.2892,59.165,,10.5
970 | "2019-12-11 09:00:00",0,0,-1,-1,15.4245,15.1675,29.6513,56.173,,13.3
971 | "2019-12-11 10:00:00",0,0,-1,-1,19.5249,19.4005,30.2997,54.623,,15.7
972 | "2019-12-11 11:00:00",0,0,-1,-1,18.379,17.643,38.2174,41.0269,,20.9
973 | "2019-12-11 12:00:00",0,0,-1,-1,6.7723,6.4535,49.7493,25.0343,,12.7
974 | "2019-12-11 13:00:00",0,0,-1,-1,3.0955,3.1248,53.6982,17.7885,,7.5
975 | "2019-12-11 14:00:00",0,0,-1,-1,3.9397,4.2018,53.6725,16.8359,,9
976 | "2019-12-11 15:00:00",0,0,-1,-1,4.1997,4.227,55.227,15.6353,,10.5
977 | "2019-12-11 16:00:00",0,0,-1,-1,3.4722,3.4217,61.991,12.2072,,9.1
978 | "2019-12-11 17:00:00",0,0,-1,-1,17.9225,17.6188,60.9133,17.314,,18
979 | "2019-12-11 18:00:00",0,0,-1,-1,45.0068,43.9072,58.3965,26.0077,,31
980 | "2019-12-11 19:00:00",0,0,-1,-1,37.817,37.5497,51.5073,34.5267,,27.4
981 | "2019-12-11 20:00:00",0,0,-1,-1,34.5385,34.8857,45.2913,40.0335,,26.5
982 | "2019-12-11 21:00:00",0,0,-1,-1,41.0083,41.765,42.7423,46.5477,,33.4
983 | "2019-12-11 22:00:00",0,0,-1,-1,16.4813,16.7713,44.4162,32.2947,,15.3
984 | "2019-12-11 23:00:00",0,0,-1,-1,1.4035,1.3297,47.1398,24.8108,,5.9
985 | "2019-12-12 00:00:00",0,0,-1,-1,5.6265,5.4555,46.1485,28.7373,,5.9
986 | "2019-12-12 01:00:00",0,0,-1,-1,2.9088,2.7715,44.7282,30.7512,,5.3
987 | "2019-12-12 02:00:00",0,0,-1,-1,12.7222,12.4025,39.3937,41.2498,,10.2
988 | "2019-12-12 03:00:00",0,0,-1,-1,13.9393,13.8198,35.5882,47.7634,,9.8
989 | "2019-12-12 04:00:00",0,0,-1,-1,12.1403,12.0682,33.7708,51.7517,,7.7
990 | "2019-12-12 05:00:00",0,0,-1,-1,10.7845,10.4097,33.965,52.2792,,7.1
991 | "2019-12-12 06:00:00",0,0,-1,-1,8.5837,8.4717,34.9112,50.5472,,5.8
992 | "2019-12-12 07:00:00",0,0,-1,-1,8.541,8.5408,35.5328,49.6977,,7
993 | "2019-12-12 08:00:00",0,0,-1,-1,6.7848,6.8727,35.2618,49.137,,6.6
994 | "2019-12-12 09:00:00",0,0,-1,-1,11.9343,12.0298,35.4967,50.0018,,10.6
995 | "2019-12-12 10:00:00",0,0,-1,-1,10.2487,10.3963,36.4775,47.5495,,9.4
996 | "2019-12-12 11:00:00",0,0,-1,-1,7.4692,7.0122,40.3272,39.448,,12.7
997 | "2019-12-12 12:00:00",0,0,-1,-1,7.0262,6.535,43.4085,35.4883,,11.5
998 | "2019-12-12 13:00:00",0,0,-1,-1,8.0267,7.4162,47.322,30.1984,,12.5
999 | "2019-12-12 14:00:00",0,0,-1,-1,6.1307,5.7297,56.0397,22.0448,,8
1000 | "2019-12-12 15:00:00",0,0,-1,-1,6.6053,6.1402,66.5755,15.2732,,9.4
1001 | "2019-12-12 16:00:00",0,0,-1,-1,3.5883,3.4757,70.1925,13.058,,6.7
1002 | "2019-12-12 17:00:00",0,0,-1,-1,2.8669,2.9031,60.3552,18.7251,,5.7
1003 | "2019-12-12 18:00:00",0,0,-1,-1,1.5882,1.6748,58.9303,19.3953,,4.5
1004 | "2019-12-12 19:00:00",0,0,-1,-1,1.7155,1.7477,56.8435,20.7435,,3.8
1005 | "2019-12-12 20:00:00",0,0,-1,-1,2.108,2.0438,54.2797,23.4765,,3.3
1006 | "2019-12-12 21:00:00",0,0,-1,-1,2.3065,2.3603,52.217,26.2777,,6.2
1007 | "2019-12-12 22:00:00",0,0,-1,-1,2.5394,2.4626,49.4539,29.9898,,7.1
1008 | "2019-12-12 23:00:00",0,0,-1,-1,5.4103,5.5352,46.0798,35.686,,8.6
1009 | "2019-12-13 00:00:00",0,0,-1,-1,5.5748,5.444,44.9002,36.606,,6.4
1010 | "2019-12-13 01:00:00",,,,,,,,,,6.9
--------------------------------------------------------------------------------