├── LICENSE ├── README.md ├── additional files ├── grid_points.png └── table-number-of-grid-points.csv ├── binder ├── environment.yml └── postBuild └── notebooks ├── ERA-NUTS-explore-with-widget.ipynb ├── exploring-ERA-NUTS.html └── exploring-ERA-NUTS.ipynb /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) 2019 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining a copy 4 | of this software and associated documentation files (the "Software"), to deal 5 | in the Software without restriction, including without limitation the rights 6 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 7 | copies of the Software, and to permit persons to whom the Software is 8 | furnished to do so, subject to the following conditions: 9 | 10 | The above copyright notice and this permission notice shall be included in all 11 | copies or substantial portions of the Software. 12 | 13 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 14 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 15 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 16 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 17 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 18 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 19 | SOFTWARE. -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # ERA-NUTS associated files 2 | 3 | This repository contains the files associated to the [ERA-NUTS](https://doi.org/10.5281/zenodo.2650191) dataset, a set of time-series of meteorological variables based on [Copernicus Climate Change Service (C3S) ERA5 reanalysis](https://climate.copernicus.eu/climate-reanalysis). 4 | 5 | An example of the analysis that can be performed with ERA-NUTS is shown [in this video](https://youtu.be/zVeF8Dv6jlE). 6 | 7 | **Important**: this dataset is still a work-in-progress, we will add more analysis and variables in the near-future. If you spot an error or something strange in the data please tell us [sending an email](mailto:matteo.de-felice@ec.europa.eu) or opening an Issue. 8 | 9 | # Data 10 | The data in NetCDF and CSV formats can be downloaded from the [Zenodo repository](https://doi.org/10.5281/zenodo.2650191) [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.2650191.svg)](https://doi.org/10.5281/zenodo.2650191) 11 | 12 | # Example notebooks 13 | In the folder `notebooks` there are two Jupyter notebooks which shows how to deal effectively with the NetCDF data in `xarray` and how to visualise them in several ways by using matplotlib or the [enlopy](https://github.com/kavvkon/enlopy) package. 14 | 15 | There are currently two notebooks: 16 | - `exploring-ERA-NUTS`: it shows how to open the NetCDF files (with Dask), how to manipulate and visualise them. A rendered version is also available as html 17 | - `ERA-NUTS-explore-with-widget`: explore interactively the datasets with [jupyter](https://jupyter.org/) and [ipywidgets](https://ipywidgets.readthedocs.io/en/stable/). (You need to download the notebook locally) 18 | 19 | Both notebooks can be edited and run online (including widgets) by clicking here: [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/energy-modelling-toolkit/era-nuts-code/master?filepath=notebooks). 20 | 21 | # Additional files 22 | 23 | In the folder `additional files` there is a map showing the spatial resolution of the ERA5 reanalysis and a CSV file specifying the number of grid points with respect to each NUTS0/1/2 region. 24 | 25 | # License 26 | 27 | This dataset hosted in Zenodo is released under [CC-BY-4.0 license](https://creativecommons.org/licenses/by/4.0/). Code, and notebooks in this repository are released under MIT license. 28 | -------------------------------------------------------------------------------- /additional files/grid_points.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/energy-modelling-toolkit/era-nuts-code/f51a452d0adc681b5c077ea6812feae0ad7d2248/additional files/grid_points.png -------------------------------------------------------------------------------- /additional files/table-number-of-grid-points.csv: -------------------------------------------------------------------------------- 1 | classification,region_name,npoints 2 | NUTS0,AL,48 3 | NUTS0,AT,163 4 | NUTS0,BE,63 5 | NUTS0,BG,197 6 | NUTS0,CH,78 7 | NUTS0,CY,12 8 | NUTS0,CZ,158 9 | NUTS0,DE,730 10 | NUTS0,DK,109 11 | NUTS0,EE,110 12 | NUTS0,EL,220 13 | NUTS0,ES,845 14 | NUTS0,FI,1001 15 | NUTS0,FR,1038 16 | NUTS0,HR,105 17 | NUTS0,HU,176 18 | NUTS0,IE,156 19 | NUTS0,IT,530 20 | NUTS0,LT,149 21 | NUTS0,LU,5 22 | NUTS0,LV,153 23 | NUTS0,ME,26 24 | NUTS0,MK,46 25 | NUTS0,NL,79 26 | NUTS0,NO,947 27 | NUTS0,PL,653 28 | NUTS0,PT,148 29 | NUTS0,RO,437 30 | NUTS0,RS,139 31 | NUTS0,SE,1265 32 | NUTS0,SI,38 33 | NUTS0,SK,95 34 | NUTS0,TR,820 35 | NUTS0,UK,531 36 | NUTS1,AL0,48 37 | NUTS1,AT1,47 38 | NUTS1,AT2,50 39 | NUTS1,AT3,66 40 | NUTS1,BE2,28 41 | NUTS1,BE3,35 42 | NUTS1,BG3,122 43 | NUTS1,BG4,75 44 | NUTS1,CH0,78 45 | NUTS1,CY0,12 46 | NUTS1,CZ0,158 47 | NUTS1,DE1,70 48 | NUTS1,DE2,136 49 | NUTS1,DE3,2 50 | NUTS1,DE4,63 51 | NUTS1,DE6,1 52 | NUTS1,DE7,43 53 | NUTS1,DE8,47 54 | NUTS1,DE9,104 55 | NUTS1,DEA,71 56 | NUTS1,DEB,38 57 | NUTS1,DEC,7 58 | NUTS1,DED,40 59 | NUTS1,DEE,42 60 | NUTS1,DEF,35 61 | NUTS1,DEG,31 62 | NUTS1,DK0,109 63 | NUTS1,EE0,110 64 | NUTS1,EL3,8 65 | NUTS1,EL4,28 66 | NUTS1,EL5,88 67 | NUTS1,EL6,96 68 | NUTS1,ES1,78 69 | NUTS1,ES2,122 70 | NUTS1,ES3,15 71 | NUTS1,ES4,368 72 | NUTS1,ES5,102 73 | NUTS1,ES6,160 74 | NUTS1,FI1,997 75 | NUTS1,FI2,4 76 | NUTS1,FR1,23 77 | NUTS1,FRB,76 78 | NUTS1,FRC,90 79 | NUTS1,FRD,57 80 | NUTS1,FRE,65 81 | NUTS1,FRF,117 82 | NUTS1,FRG,62 83 | NUTS1,FRH,55 84 | NUTS1,FRI,153 85 | NUTS1,FRJ,133 86 | NUTS1,FRK,132 87 | NUTS1,FRL,59 88 | NUTS1,FRM,16 89 | NUTS1,HR0,105 90 | NUTS1,HU1,15 91 | NUTS1,HU2,66 92 | NUTS1,HU3,95 93 | NUTS1,IE0,156 94 | NUTS1,IS0,223 95 | NUTS1,ITC,105 96 | NUTS1,ITF,123 97 | NUTS1,ITG,86 98 | NUTS1,ITH,112 99 | NUTS1,ITI,104 100 | NUTS1,LT0,149 101 | NUTS1,LU0,5 102 | NUTS1,LV0,153 103 | NUTS1,ME0,26 104 | NUTS1,MK0,46 105 | NUTS1,NL1,19 106 | NUTS1,NL2,22 107 | NUTS1,NL3,24 108 | NUTS1,NL4,14 109 | NUTS1,NO0,947 110 | NUTS1,PL2,56 111 | NUTS1,PL4,140 112 | NUTS1,PL5,60 113 | NUTS1,PL6,132 114 | NUTS1,PL7,64 115 | NUTS1,PL8,130 116 | NUTS1,PL9,71 117 | NUTS1,PT1,148 118 | NUTS1,RO1,127 119 | NUTS1,RO2,129 120 | NUTS1,RO3,68 121 | NUTS1,RO4,113 122 | NUTS1,RS1,44 123 | NUTS1,RS2,95 124 | NUTS1,SE1,127 125 | NUTS1,SE2,202 126 | NUTS1,SE3,936 127 | NUTS1,SI0,38 128 | NUTS1,SK0,95 129 | NUTS1,TR1,6 130 | NUTS1,TR2,70 131 | NUTS1,TR3,147 132 | NUTS1,TR4,88 133 | NUTS1,TR5,124 134 | NUTS1,TR6,137 135 | NUTS1,TR7,119 136 | NUTS1,TR8,123 137 | NUTS1,TR9,2 138 | NUTS1,TRC,4 139 | NUTS1,UKC,18 140 | NUTS1,UKD,33 141 | NUTS1,UKE,35 142 | NUTS1,UKF,35 143 | NUTS1,UKG,27 144 | NUTS1,UKH,39 145 | NUTS1,UKI,2 146 | NUTS1,UKJ,36 147 | NUTS1,UKK,50 148 | NUTS1,UKL,44 149 | NUTS1,UKM,180 150 | NUTS1,UKN,32 151 | NUTS2,AL01,18 152 | NUTS2,AL02,7 153 | NUTS2,AL03,23 154 | NUTS2,AT11,9 155 | NUTS2,AT12,37 156 | NUTS2,AT13,1 157 | NUTS2,AT21,20 158 | NUTS2,AT22,30 159 | NUTS2,AT31,23 160 | NUTS2,AT32,14 161 | NUTS2,AT33,25 162 | NUTS2,AT34,4 163 | NUTS2,BE21,3 164 | NUTS2,BE22,8 165 | NUTS2,BE23,5 166 | NUTS2,BE24,7 167 | NUTS2,BE25,5 168 | NUTS2,BE31,2 169 | NUTS2,BE32,10 170 | NUTS2,BE33,6 171 | NUTS2,BE34,9 172 | NUTS2,BE35,8 173 | NUTS2,BG31,33 174 | NUTS2,BG32,27 175 | NUTS2,BG33,26 176 | NUTS2,BG34,36 177 | NUTS2,BG41,36 178 | NUTS2,BG42,39 179 | NUTS2,CH01,16 180 | NUTS2,CH02,19 181 | NUTS2,CH03,3 182 | NUTS2,CH04,4 183 | NUTS2,CH05,21 184 | NUTS2,CH06,8 185 | NUTS2,CH07,7 186 | NUTS2,CY00,12 187 | NUTS2,CZ01,1 188 | NUTS2,CZ02,19 189 | NUTS2,CZ03,36 190 | NUTS2,CZ04,17 191 | NUTS2,CZ05,27 192 | NUTS2,CZ06,30 193 | NUTS2,CZ07,16 194 | NUTS2,CZ08,12 195 | NUTS2,DE11,19 196 | NUTS2,DE12,16 197 | NUTS2,DE13,16 198 | NUTS2,DE14,19 199 | NUTS2,DE21,35 200 | NUTS2,DE22,20 201 | NUTS2,DE23,18 202 | NUTS2,DE24,13 203 | NUTS2,DE25,15 204 | NUTS2,DE26,17 205 | NUTS2,DE27,18 206 | NUTS2,DE30,2 207 | NUTS2,DE40,63 208 | NUTS2,DE60,1 209 | NUTS2,DE71,13 210 | NUTS2,DE72,11 211 | NUTS2,DE73,19 212 | NUTS2,DE80,47 213 | NUTS2,DE91,18 214 | NUTS2,DE92,20 215 | NUTS2,DE93,35 216 | NUTS2,DE94,31 217 | NUTS2,DEA1,13 218 | NUTS2,DEA2,15 219 | NUTS2,DEA3,14 220 | NUTS2,DEA4,14 221 | NUTS2,DEA5,15 222 | NUTS2,DEB1,15 223 | NUTS2,DEB2,10 224 | NUTS2,DEB3,13 225 | NUTS2,DEC0,7 226 | NUTS2,DED2,17 227 | NUTS2,DED4,15 228 | NUTS2,DED5,8 229 | NUTS2,DEE0,42 230 | NUTS2,DEF0,35 231 | NUTS2,DEG0,31 232 | NUTS2,DK01,8 233 | NUTS2,DK02,17 234 | NUTS2,DK03,29 235 | NUTS2,DK04,34 236 | NUTS2,DK05,21 237 | NUTS2,EE00,110 238 | NUTS2,EL30,8 239 | NUTS2,EL41,8 240 | NUTS2,EL42,5 241 | NUTS2,EL43,15 242 | NUTS2,EL51,25 243 | NUTS2,EL52,32 244 | NUTS2,EL53,16 245 | NUTS2,EL54,15 246 | NUTS2,EL61,23 247 | NUTS2,EL62,3 248 | NUTS2,EL63,17 249 | NUTS2,EL64,28 250 | NUTS2,EL65,25 251 | NUTS2,ES11,51 252 | NUTS2,ES12,20 253 | NUTS2,ES13,7 254 | NUTS2,ES21,13 255 | NUTS2,ES22,20 256 | NUTS2,ES23,8 257 | NUTS2,ES24,81 258 | NUTS2,ES30,15 259 | NUTS2,ES41,164 260 | NUTS2,ES42,134 261 | NUTS2,ES43,70 262 | NUTS2,ES51,55 263 | NUTS2,ES52,39 264 | NUTS2,ES53,8 265 | NUTS2,ES61,141 266 | NUTS2,ES62,19 267 | NUTS2,FI19,182 268 | NUTS2,FI1B,22 269 | NUTS2,FI1C,90 270 | NUTS2,FI1D,703 271 | NUTS2,FI20,4 272 | NUTS2,FR10,23 273 | NUTS2,FRB0,76 274 | NUTS2,FRC1,60 275 | NUTS2,FRC2,30 276 | NUTS2,FRD1,35 277 | NUTS2,FRD2,22 278 | NUTS2,FRE1,24 279 | NUTS2,FRE2,41 280 | NUTS2,FRF1,17 281 | NUTS2,FRF2,52 282 | NUTS2,FRF3,48 283 | NUTS2,FRG0,62 284 | NUTS2,FRH0,55 285 | NUTS2,FRI1,74 286 | NUTS2,FRI2,31 287 | NUTS2,FRI3,48 288 | NUTS2,FRJ1,52 289 | NUTS2,FRJ2,81 290 | NUTS2,FRK1,49 291 | NUTS2,FRK2,83 292 | NUTS2,FRL0,59 293 | NUTS2,FRM0,16 294 | NUTS2,HR03,42 295 | NUTS2,HR04,63 296 | NUTS2,HU11,2 297 | NUTS2,HU12,13 298 | NUTS2,HU21,20 299 | NUTS2,HU22,20 300 | NUTS2,HU23,26 301 | NUTS2,HU31,26 302 | NUTS2,HU32,33 303 | NUTS2,HU33,36 304 | NUTS2,IE04,62 305 | NUTS2,IE05,62 306 | NUTS2,IE06,32 307 | NUTS2,IS00,223 308 | NUTS2,ITC1,44 309 | NUTS2,ITC2,5 310 | NUTS2,ITC3,10 311 | NUTS2,ITC4,46 312 | NUTS2,ITF1,17 313 | NUTS2,ITF2,10 314 | NUTS2,ITF3,24 315 | NUTS2,ITF4,29 316 | NUTS2,ITF5,18 317 | NUTS2,ITF6,25 318 | NUTS2,ITG1,44 319 | NUTS2,ITG2,42 320 | NUTS2,ITH1,16 321 | NUTS2,ITH2,10 322 | NUTS2,ITH3,33 323 | NUTS2,ITH4,14 324 | NUTS2,ITH5,39 325 | NUTS2,ITI1,41 326 | NUTS2,ITI2,15 327 | NUTS2,ITI3,17 328 | NUTS2,ITI4,31 329 | NUTS2,LT01,22 330 | NUTS2,LT02,127 331 | NUTS2,LU00,5 332 | NUTS2,LV00,153 333 | NUTS2,ME00,26 334 | NUTS2,MK00,46 335 | NUTS2,NL11,5 336 | NUTS2,NL12,8 337 | NUTS2,NL13,6 338 | NUTS2,NL21,8 339 | NUTS2,NL22,9 340 | NUTS2,NL23,5 341 | NUTS2,NL31,5 342 | NUTS2,NL32,7 343 | NUTS2,NL33,6 344 | NUTS2,NL34,6 345 | NUTS2,NL41,11 346 | NUTS2,NL42,3 347 | NUTS2,NO01,15 348 | NUTS2,NO02,137 349 | NUTS2,NO03,97 350 | NUTS2,NO04,60 351 | NUTS2,NO05,133 352 | NUTS2,NO06,115 353 | NUTS2,NO07,390 354 | NUTS2,PL21,31 355 | NUTS2,PL22,25 356 | NUTS2,PL41,64 357 | NUTS2,PL42,49 358 | NUTS2,PL43,27 359 | NUTS2,PL51,43 360 | NUTS2,PL52,17 361 | NUTS2,PL61,37 362 | NUTS2,PL62,54 363 | NUTS2,PL63,41 364 | NUTS2,PL71,40 365 | NUTS2,PL72,24 366 | NUTS2,PL81,50 367 | NUTS2,PL82,37 368 | NUTS2,PL84,43 369 | NUTS2,PL91,14 370 | NUTS2,PL92,57 371 | NUTS2,PT11,35 372 | NUTS2,PT15,7 373 | NUTS2,PT16,47 374 | NUTS2,PT17,6 375 | NUTS2,PT18,53 376 | NUTS2,RO11,63 377 | NUTS2,RO12,64 378 | NUTS2,RO21,66 379 | NUTS2,RO22,63 380 | NUTS2,RO31,66 381 | NUTS2,RO32,2 382 | NUTS2,RO41,55 383 | NUTS2,RO42,58 384 | NUTS2,RS11,5 385 | NUTS2,RS12,39 386 | NUTS2,RS21,47 387 | NUTS2,RS22,48 388 | NUTS2,SE11,17 389 | NUTS2,SE12,110 390 | NUTS2,SE21,88 391 | NUTS2,SE22,31 392 | NUTS2,SE23,83 393 | NUTS2,SE31,189 394 | NUTS2,SE32,221 395 | NUTS2,SE33,526 396 | NUTS2,SI03,23 397 | NUTS2,SI04,15 398 | NUTS2,SK01,4 399 | NUTS2,SK02,30 400 | NUTS2,SK03,30 401 | NUTS2,SK04,31 402 | NUTS2,TR10,6 403 | NUTS2,TR21,32 404 | NUTS2,TR22,38 405 | NUTS2,TR31,20 406 | NUTS2,TR32,49 407 | NUTS2,TR33,78 408 | NUTS2,TR41,52 409 | NUTS2,TR42,36 410 | NUTS2,TR51,43 411 | NUTS2,TR52,81 412 | NUTS2,TR61,60 413 | NUTS2,TR62,46 414 | NUTS2,TR63,31 415 | NUTS2,TR71,53 416 | NUTS2,TR72,66 417 | NUTS2,TR81,16 418 | NUTS2,TR82,40 419 | NUTS2,TR83,67 420 | NUTS2,TR90,2 421 | NUTS2,TRC1,4 422 | NUTS2,UKC1,6 423 | NUTS2,UKC2,12 424 | NUTS2,UKD1,16 425 | NUTS2,UKD3,3 426 | NUTS2,UKD4,6 427 | NUTS2,UKD6,6 428 | NUTS2,UKD7,2 429 | NUTS2,UKE1,8 430 | NUTS2,UKE2,20 431 | NUTS2,UKE3,4 432 | NUTS2,UKE4,3 433 | NUTS2,UKF1,9 434 | NUTS2,UKF2,11 435 | NUTS2,UKF3,15 436 | NUTS2,UKG1,13 437 | NUTS2,UKG2,12 438 | NUTS2,UKG3,2 439 | NUTS2,UKH1,25 440 | NUTS2,UKH2,7 441 | NUTS2,UKH3,7 442 | NUTS2,UKI3,1 443 | NUTS2,UKI5,1 444 | NUTS2,UKJ1,12 445 | NUTS2,UKJ2,10 446 | NUTS2,UKJ3,8 447 | NUTS2,UKJ4,6 448 | NUTS2,UKK1,14 449 | NUTS2,UKK2,14 450 | NUTS2,UKK3,8 451 | NUTS2,UKK4,14 452 | NUTS2,UKL1,30 453 | NUTS2,UKL2,14 454 | NUTS2,UKM5,14 455 | NUTS2,UKM6,96 456 | NUTS2,UKM7,27 457 | NUTS2,UKM8,4 458 | NUTS2,UKM9,39 459 | NUTS2,UKN0,32 460 | -------------------------------------------------------------------------------- /binder/environment.yml: -------------------------------------------------------------------------------- 1 | name: era5-nuts-binder 2 | channels: 3 | - conda-forge 4 | - defaults 5 | dependencies: 6 | - python=3.7 7 | - numpy>=1.12 8 | - matplotlib>=2.0 9 | - pandas>=0.22 10 | - scipy>=1.0 11 | - statsmodels 12 | - enlopy 13 | - netcdf4 14 | - xarray 15 | - ipywidgets 16 | - dask 17 | - rise 18 | - geopandas 19 | -------------------------------------------------------------------------------- /binder/postBuild: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | wget -q -O era-nuts-t2m-nuts2-hourly.nc https://zenodo.org/record/4584576/files/era-nuts-t2m-nuts2-hourly.nc?download=1 4 | wget -q -O era-nuts-ws10-nuts2-hourly.nc https://zenodo.org/record/4584576/files/era-nuts-ws10-nuts2-hourly.nc?download=1 5 | -------------------------------------------------------------------------------- /notebooks/ERA-NUTS-explore-with-widget.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "metadata": { 7 | "ExecuteTime": { 8 | "end_time": "2019-04-17T09:22:02.106975Z", 9 | "start_time": "2019-04-17T09:22:01.257262Z" 10 | } 11 | }, 12 | "outputs": [], 13 | "source": [ 14 | "import matplotlib\n", 15 | "import matplotlib.pyplot as plt\n", 16 | "import numpy as np\n", 17 | "import pandas as pd\n", 18 | "import xarray as xr\n", 19 | "import geopandas as gpd" 20 | ] 21 | }, 22 | { 23 | "cell_type": "code", 24 | "execution_count": 2, 25 | "metadata": { 26 | "ExecuteTime": { 27 | "end_time": "2019-04-17T09:22:02.862422Z", 28 | "start_time": "2019-04-17T09:22:02.860349Z" 29 | } 30 | }, 31 | "outputs": [], 32 | "source": [ 33 | "from ipywidgets import interact, IntRangeSlider, ToggleButtons, Play" 34 | ] 35 | }, 36 | { 37 | "cell_type": "code", 38 | "execution_count": 3, 39 | "metadata": {}, 40 | "outputs": [], 41 | "source": [ 42 | "# A bit of customisation\n", 43 | "matplotlib.rcParams['figure.figsize'] = (18.0, 5.0)\n", 44 | "\n", 45 | "# %matplotlib inline\n", 46 | "%config InlineBackend.figure_format='retina'\n", 47 | "\n", 48 | "np.set_printoptions(precision=3, suppress=True) # suppress scientific float notation" 49 | ] 50 | }, 51 | { 52 | "cell_type": "code", 53 | "execution_count": 4, 54 | "metadata": { 55 | "ExecuteTime": { 56 | "end_time": "2019-04-17T09:22:02.127945Z", 57 | "start_time": "2019-04-17T09:22:02.122036Z" 58 | } 59 | }, 60 | "outputs": [], 61 | "source": [ 62 | "path = r'..'\n", 63 | "# path = '.'\n", 64 | "map_df = gpd.read_file(\"https://raw.githubusercontent.com/eurostat/Nuts2json/master/pub/v2/2021/3035/20M/2.json\")" 65 | ] 66 | }, 67 | { 68 | "cell_type": "markdown", 69 | "metadata": {}, 70 | "source": [ 71 | "# Load data\n", 72 | "Load all the files as a single dataset using xarray and its dask capabilities. In this way the files are not loaded in-memory but dask/xarray load them **only** when needed. `mask_and_scale` to True replaces array values equal to `_FillValue` with NA and scale values according to the formula `original_values * scale_factor + add_offset`.\n" 73 | ] 74 | }, 75 | { 76 | "cell_type": "code", 77 | "execution_count": 5, 78 | "metadata": { 79 | "ExecuteTime": { 80 | "end_time": "2019-04-17T09:22:02.350255Z", 81 | "start_time": "2019-04-17T09:22:02.129529Z" 82 | } 83 | }, 84 | "outputs": [ 85 | { 86 | "data": { 87 | "text/plain": [ 88 | "\n", 89 | "Dimensions: (region: 309, time: 341880)\n", 90 | "Coordinates:\n", 91 | " * time (time) datetime64[ns] 1980-01-01 ... 2018-12-31T23:00:00\n", 92 | " * region (region) object 'AL01' 'AL02' 'AL03' ... 'UKM8' 'UKM9' 'UKN0'\n", 93 | "Data variables:\n", 94 | " t2m (time, region) float32 dask.array" 95 | ] 96 | }, 97 | "execution_count": 5, 98 | "metadata": {}, 99 | "output_type": "execute_result" 100 | } 101 | ], 102 | "source": [ 103 | "ds = xr.open_mfdataset(path + '/*.nc', mask_and_scale=True)\n", 104 | "ds" 105 | ] 106 | }, 107 | { 108 | "cell_type": "markdown", 109 | "metadata": {}, 110 | "source": [ 111 | "# Widget" 112 | ] 113 | }, 114 | { 115 | "cell_type": "markdown", 116 | "metadata": {}, 117 | "source": [ 118 | "The above selection method allows as to build custom interactive dashboards with a simple jupyter widgets through `ipywidget`. This process is very is simple. You just need to encapsulate you query in a function and decorate it with [interact](https://ipywidgets.readthedocs.io/en/stable/examples/Using%20Interact.html#Basic-interact) by specifying the desired range. Due to xarray and dask's out of core computation, querying and plotting this dataset is almost instant" 119 | ] 120 | }, 121 | { 122 | "cell_type": "code", 123 | "execution_count": 6, 124 | "metadata": { 125 | "ExecuteTime": { 126 | "end_time": "2019-04-17T09:22:03.221021Z", 127 | "start_time": "2019-04-17T09:22:02.863892Z" 128 | } 129 | }, 130 | "outputs": [ 131 | { 132 | "data": { 133 | "application/vnd.jupyter.widget-view+json": { 134 | "model_id": "d15d4d3884b24758acb235ae98f789c0", 135 | "version_major": 2, 136 | "version_minor": 0 137 | }, 138 | "text/plain": [ 139 | "interactive(children=(Dropdown(description='var', options=('t2m',), value='t2m'), Dropdown(description='region…" 140 | ] 141 | }, 142 | "metadata": {}, 143 | "output_type": "display_data" 144 | } 145 | ], 146 | "source": [ 147 | "@interact(var=list(ds.data_vars), \n", 148 | " region=ds.coords['region'].values,\n", 149 | " month=(1,12),\n", 150 | " year=(1980,2021))\n", 151 | "def plot_region_month(var, region, year, month):\n", 152 | " ds[var].sel(time='{}-{}'.format(year, month), region=region).plot(color='k') # year" 153 | ] 154 | }, 155 | { 156 | "cell_type": "code", 157 | "execution_count": 7, 158 | "metadata": { 159 | "ExecuteTime": { 160 | "end_time": "2019-04-17T09:22:03.221021Z", 161 | "start_time": "2019-04-17T09:22:02.863892Z" 162 | } 163 | }, 164 | "outputs": [ 165 | { 166 | "data": { 167 | "application/vnd.jupyter.widget-view+json": { 168 | "model_id": "cb6ffda7cc3f4c6fbe094c13cad38336", 169 | "version_major": 2, 170 | "version_minor": 0 171 | }, 172 | "text/plain": [ 173 | "interactive(children=(Dropdown(description='var', options=('t2m',), value='t2m'), Dropdown(description='region…" 174 | ] 175 | }, 176 | "metadata": {}, 177 | "output_type": "display_data" 178 | } 179 | ], 180 | "source": [ 181 | "@interact(var=list(ds.data_vars), \n", 182 | " region=ds.coords['region'].values,\n", 183 | " year_range=IntRangeSlider(min=1980,max=2021),\n", 184 | " timestep=ToggleButtons(options=[('week','w'),('month','m'),('year','a')]),\n", 185 | " continuous_update=False,\n", 186 | " )\n", 187 | "def plot_region_month(var, region, year_range,timestep='m'):\n", 188 | " (ds[var].sel(region=region,\n", 189 | " time=slice(str(year_range[0]), str(year_range[1])))\n", 190 | " .resample(time=timestep).mean()\n", 191 | " .plot(color='k')\n", 192 | " )" 193 | ] 194 | }, 195 | { 196 | "cell_type": "markdown", 197 | "metadata": {}, 198 | "source": [ 199 | "Querying works also for a selection of regions" 200 | ] 201 | }, 202 | { 203 | "cell_type": "code", 204 | "execution_count": 8, 205 | "metadata": { 206 | "ExecuteTime": { 207 | "end_time": "2019-04-17T09:22:05.540679Z", 208 | "start_time": "2019-04-17T09:22:03.222583Z" 209 | } 210 | }, 211 | "outputs": [ 212 | { 213 | "data": { 214 | "application/vnd.jupyter.widget-view+json": { 215 | "model_id": "bfe09283df6e4352b1699056a3205daa", 216 | "version_major": 2, 217 | "version_minor": 0 218 | }, 219 | "text/plain": [ 220 | "interactive(children=(Dropdown(description='var', options=('t2m',), value='t2m'), Dropdown(description='region…" 221 | ] 222 | }, 223 | "metadata": {}, 224 | "output_type": "display_data" 225 | } 226 | ], 227 | "source": [ 228 | "#Extract unique countries by taking the first two characters of all regions\n", 229 | "countries = set(s[0:2] for s in ds.coords['region'].values)\n", 230 | "\n", 231 | "@interact(var=list(ds.data_vars), \n", 232 | " region=countries,\n", 233 | " month=(1,12),\n", 234 | " year=(1980,2021))\n", 235 | "def plot_region_month(var, region, year, month):\n", 236 | " selection = [s for s in ds.coords['region'].values if s[0:2]==region]\n", 237 | " ds[var].sel(time='{}-{}'.format(year, month), region=selection).plot.line(x='time')" 238 | ] 239 | }, 240 | { 241 | "cell_type": "markdown", 242 | "metadata": {}, 243 | "source": [ 244 | "Another example using geopandas. Here we plot the mean of each selected month/year per region" 245 | ] 246 | }, 247 | { 248 | "cell_type": "code", 249 | "execution_count": 9, 250 | "metadata": {}, 251 | "outputs": [ 252 | { 253 | "data": { 254 | "application/vnd.jupyter.widget-view+json": { 255 | "model_id": "518cd4be015443d19212dfa78824018c", 256 | "version_major": 2, 257 | "version_minor": 0 258 | }, 259 | "text/plain": [ 260 | "interactive(children=(Dropdown(description='var', options=('t2m',), value='t2m'), IntSlider(value=1999, descri…" 261 | ] 262 | }, 263 | "metadata": {}, 264 | "output_type": "display_data" 265 | } 266 | ], 267 | "source": [ 268 | "plot_min=dict(t2m=-10)\n", 269 | "plot_max=dict(t2m=35)\n", 270 | "\n", 271 | "@interact(var=list(ds.data_vars), \n", 272 | " month=(1,12), \n", 273 | " year=(1980,2021), # Play(value=1,min=1,max=12,step=1,description='Loop months'),\n", 274 | " continuous_update=False,)\n", 275 | "def plot_region_month(var, year, month):\n", 276 | " fig, ax = plt.subplots(1, figsize=(14, 10))\n", 277 | " ax.axis('off')\n", 278 | " time_str = '{}-{}'.format(year, month)\n", 279 | " a = ds[var].sel(time=time_str).mean(dim='time').to_series().reset_index()\n", 280 | " annot_str = '{} \\nMean Temp:${:.3g}^oC$'.format(time_str,a.mean().values[0])\n", 281 | " ax.text(.7,0.9,annot_str,fontdict={'fontsize':'xx-large'},transform=ax.transAxes)\n", 282 | " \n", 283 | " (map_df.merge(a, left_on='id', right_on='region')\n", 284 | " .plot(column=var, cmap='coolwarm', vmin=plot_min.get(var,None),vmax=plot_max.get(var,None),\n", 285 | " linewidth=0.3, edgecolor='0.4',\n", 286 | " ax=ax,legend=True,)\n", 287 | " )\n" 288 | ] 289 | }, 290 | { 291 | "cell_type": "code", 292 | "execution_count": null, 293 | "metadata": {}, 294 | "outputs": [], 295 | "source": [] 296 | } 297 | ], 298 | "metadata": { 299 | "kernelspec": { 300 | "display_name": "Python 3", 301 | "language": "python", 302 | "name": "python3" 303 | }, 304 | "language_info": { 305 | "codemirror_mode": { 306 | "name": "ipython", 307 | "version": 3 308 | }, 309 | "file_extension": ".py", 310 | "mimetype": "text/x-python", 311 | "name": "python", 312 | "nbconvert_exporter": "python", 313 | "pygments_lexer": "ipython3", 314 | "version": "3.7.3" 315 | }, 316 | "toc": { 317 | "base_numbering": 1, 318 | "nav_menu": {}, 319 | "number_sections": true, 320 | "sideBar": true, 321 | "skip_h1_title": false, 322 | "title_cell": "Table of Contents", 323 | "title_sidebar": "Contents", 324 | "toc_cell": false, 325 | "toc_position": { 326 | "height": "calc(100% - 180px)", 327 | "left": "10px", 328 | "top": "150px", 329 | "width": "311px" 330 | }, 331 | "toc_section_display": true, 332 | "toc_window_display": true 333 | }, 334 | "varInspector": { 335 | "cols": { 336 | "lenName": 16, 337 | "lenType": 16, 338 | "lenVar": 40 339 | }, 340 | "kernels_config": { 341 | "python": { 342 | "delete_cmd_postfix": "", 343 | "delete_cmd_prefix": "del ", 344 | "library": "var_list.py", 345 | "varRefreshCmd": "print(var_dic_list())" 346 | }, 347 | "r": { 348 | "delete_cmd_postfix": ") ", 349 | "delete_cmd_prefix": "rm(", 350 | "library": "var_list.r", 351 | "varRefreshCmd": "cat(var_dic_list()) " 352 | } 353 | }, 354 | "types_to_exclude": [ 355 | "module", 356 | "function", 357 | "builtin_function_or_method", 358 | "instance", 359 | "_Feature" 360 | ], 361 | "window_display": false 362 | } 363 | }, 364 | "nbformat": 4, 365 | "nbformat_minor": 2 366 | } 367 | --------------------------------------------------------------------------------