├── .gitignore ├── 3.1 Evaluation of simulations Results for October 2018.ipynb ├── 3.1 Evaluation of simulations Results for September 2018.ipynb ├── 3.1 Meteorological results evaluation from WRF-Chem 2018.ipynb ├── 3.1 Toluene analysis.ipynb ├── 3.2.1 Changes in Surface O3 - October 2030.ipynb ├── 3.2.1 Changes in Surface O3 September 2030.ipynb ├── 3.2.2 Meteorological Changes in the MASP.ipynb ├── Maps Locations.ipynb ├── README.md ├── fig ├── .DS_Store ├── EDA_pairplot_pol.pdf ├── MDA8_spatial_station_oct.pdf ├── MDA8_type_rcps.pdf ├── eda_no_no2.pdf ├── rcp_2030_oct_subplot_o3.pdf └── rcp_2030_subplot_o3.pdf ├── met_rcp_changes.csv ├── mod_stats.py ├── rain └── wrf_rain.py └── wrf_extract ├── cetesb2017_latlon.dat └── wrf_extract.py /.gitignore: -------------------------------------------------------------------------------- 1 | data 2 | .old 3 | __pycache__ 4 | .ipynb_chechpoints 5 | .* 6 | .DS_Store 7 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # PythonWRF 2 | Process outputs from WRF-Chem. 3 | 4 | ## WRF-Chem model results processing 5 | WRF-Chem model outputs are files format (`wrfout__{year}-{m}-{d}_{h}:00:0`) that content matrix values for many parameters meteorological and chemical species. 6 | 7 | There are scripts and Jupyter Notebooks used to process WRF-Chem outputs: 8 | 9 | * `wrf_extract.py`: Script developed by M. Gavidia [quishqa](https://github.com/quishqa). This script extract meteorological and polutants variables such as temperature (t2), relative humidity (rh2), wind, atmospheric pressure (psfc), ozone (o3), nitrogen monoxide (no), nitrogen dioxide (no2), and carbon monoxide (co). This script requires location points as latitude and longitude. In this case, we used CETESB locations provided in its website (QUALAR). Pollutants (with the exception of carbon monoxide) were converted from ppm to $\mu$ gm$^{-3}$. After that, variables extracted were saving as dictionary and exporting to csv format. 10 | * `wrf_rain.py`: Script based on `wrf_extract.py` that extracts `rain` and `rainnc`. These variables are accumulated rain and require to be processed. Finally, the script export values based on location as dictionary saving as pickle (very useful in Python). 11 | * `mod_stats.py`: Script developed by M. Gavidia [quishqa](https://github.com/quishqa) and modified by me. This script analyze model outputs and observed values and calculate values for each statistical benchmarks. 12 | * Jupyter Notebooks: `3.1 Evaluation of simulation Results for September 2018.ipynb` for instance contents Python scripts used to obtain charts and statistical results. `3.2 Meteorological results evaluation` also is similar to before one, but it process meteorological outputs. `Maps Locations.ipynb` shows scripts to make maps locations. 13 | 14 | ## ACKNOWLEDGMENTS 15 | The author thank the CAPES (Coordenação de Aperfeiçonamento de Pessoal de Nível Superior) for the financial support of this work. I also thank the State Company for the Environment (CETESB) for the hourly data from the station networks available as open acces in its website (QUALAR). 16 | -------------------------------------------------------------------------------- /fig/.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/adelgadop/PythonWRF/3f36363d888873a7c6e4a435c5f9b9564c80347f/fig/.DS_Store -------------------------------------------------------------------------------- /fig/EDA_pairplot_pol.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/adelgadop/PythonWRF/3f36363d888873a7c6e4a435c5f9b9564c80347f/fig/EDA_pairplot_pol.pdf -------------------------------------------------------------------------------- /fig/MDA8_spatial_station_oct.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/adelgadop/PythonWRF/3f36363d888873a7c6e4a435c5f9b9564c80347f/fig/MDA8_spatial_station_oct.pdf -------------------------------------------------------------------------------- /fig/MDA8_type_rcps.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/adelgadop/PythonWRF/3f36363d888873a7c6e4a435c5f9b9564c80347f/fig/MDA8_type_rcps.pdf -------------------------------------------------------------------------------- /fig/eda_no_no2.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/adelgadop/PythonWRF/3f36363d888873a7c6e4a435c5f9b9564c80347f/fig/eda_no_no2.pdf -------------------------------------------------------------------------------- /fig/rcp_2030_oct_subplot_o3.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/adelgadop/PythonWRF/3f36363d888873a7c6e4a435c5f9b9564c80347f/fig/rcp_2030_oct_subplot_o3.pdf -------------------------------------------------------------------------------- /fig/rcp_2030_subplot_o3.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/adelgadop/PythonWRF/3f36363d888873a7c6e4a435c5f9b9564c80347f/fig/rcp_2030_subplot_o3.pdf -------------------------------------------------------------------------------- /mod_stats.py: -------------------------------------------------------------------------------- 1 | """ 2 | Modified by Alejandro Delgado 3 | Original script from Mario Gavidia 4 | """ 5 | 6 | import numpy as np 7 | import pandas as pd 8 | import matplotlib.pyplot as plt 9 | import scipy as scipy 10 | 11 | def aq_stats(data,polls): 12 | ''' 13 | Statistics for model evaluation for air quality parameters, according to Emery et al. (2017). 14 | Parameters 15 | ---------- 16 | data: pandas DataFrame 17 | data with observation (measurements) and model results, where parameters 18 | have suffixes as '_obs' and '_mod'. 19 | polls: list str 20 | name of variables. 21 | 22 | Returns 23 | ------- 24 | stats : pandas DataFrame 25 | Contain global statistics: Mean bias, Mean Gross Error, 26 | Root Mean Square Error, IOA, Normalized Mean Bias, Normalized Mean Error, 27 | Correlation coefficient using numpy, Standard deviation. Not for wind direction 28 | 29 | ''' 30 | n = {} # number of observations 31 | MB = {} # Mean Bias 32 | MAGE = {} # Mean Gross Error 33 | RMSE = {} # Root Mean Square Error 34 | NMB = {} # Normalized Mean Bias 35 | NME = {} # Normalized Mean Error 36 | IOA = {} # Index Of Agreement (Willmontt, 1982) 37 | r = {} # Correlation Coefficient based on Numpy module 38 | Mm = {} # Mean of modeling results 39 | Om = {} # Mean of measurements 40 | Msd = {} # Standard deviation of modelling 41 | Osd = {} # Standard deviation of observations 42 | 43 | for pol in polls: 44 | df = data[[pol+'_mod',pol+'_obs']].dropna() 45 | mod = df[pol+'_mod'] 46 | obs = df[pol+'_obs'] 47 | n[pol] = obs.count() 48 | MB[pol] = mod.mean()-obs.mean() 49 | MAGE[pol] = (mod-obs).abs().mean() 50 | RMSE[pol] = (((mod - obs)**2).mean())**0.5 51 | NMB[pol] = (mod - obs).sum() / obs.sum() * 100 52 | NME[pol] = (mod - obs).abs().sum() / obs.sum() * 100 53 | A = ((mod - obs)**2).sum() 54 | B = (((mod - obs.mean()).abs() + (obs - obs.mean()).abs())**2).sum() 55 | IOA[pol] = 1 - (A / B) 56 | r[pol] = np.corrcoef(mod, obs)[0,1] 57 | Mm[pol] = mod.mean() 58 | Om[pol] = obs.mean() 59 | Msd[pol] = mod.std() 60 | Osd[pol] = obs.std() 61 | res = pd.DataFrame({'n':n,'MB':MB,'MAGE':MAGE,'RMSE':RMSE,'NMB':NMB,'NME':NME,'IOA':IOA,'r':r,'Mm':Mm, 62 | 'Om':Om,'Msd':Msd,'Osd':Osd}) 63 | return res 64 | 65 | def wind_dir_diff(Mi, Oi): 66 | ''' 67 | Difference between Wind directions based in its 68 | periodic property. Based on Reboredo et al. 2015 69 | Parameters 70 | ---------- 71 | Mi : np.float 72 | Model wind direction. 73 | Oi : TYPE 74 | Observed wind direction. 75 | Returns 76 | ------- 77 | Wind difference. 78 | ''' 79 | wd_dif = Mi - Oi 80 | if Mi < Oi: 81 | if (np.abs(wd_dif) < np.abs(360 + wd_dif)): 82 | ans = wd_dif 83 | elif (np.abs(wd_dif) > np.abs(360 + wd_dif)): 84 | ans = 360 + wd_dif 85 | elif Mi > Oi: 86 | if (np.abs(wd_dif) < np.abs(wd_dif - 360)): 87 | ans = wd_dif 88 | elif (np.abs(wd_dif) > np.abs(wd_dif -360)): 89 | ans = wd_dif - 360 90 | else: 91 | ans = 0.0 92 | 93 | return(ans) 94 | 95 | def wind_dir_mb(model_df, obs_df, wd_name='wd'): 96 | ''' 97 | Calculates wind direction mean bias based in 98 | Reboredo et al. 2015 99 | Parameters 100 | ---------- 101 | model_df : pandas DataFrame 102 | DataFrame with model output. 103 | obs_df : pandas DataFrame 104 | DataFrame with observations. 105 | wd_name : str, optional 106 | Wind direction column name. The default is 'wd'. 107 | Returns 108 | ------- 109 | wd_mb : numpy.float64 110 | wind direction mean bias. 111 | ''' 112 | wd_df = pd.DataFrame({ 113 | 'mi': model_df, 114 | 'oi': obs_df}) 115 | wd_df.dropna(how="any", inplace=True) 116 | if wd_df.empty: 117 | wd_mb = np.nan 118 | else: 119 | dif = wd_df.apply(lambda row: wind_dir_diff(row['mi'], row['oi']), 120 | axis=1) 121 | wd_mb = dif.mean() 122 | return wd_mb 123 | 124 | 125 | def wind_dir_mage(model_df, obs_df, wd_name='wd'): 126 | ''' 127 | Calculate wind direction mean absolute error 128 | Parameters 129 | ---------- 130 | model_df : pandas DataFrame 131 | DataFrame with model output. 132 | obs_df : pandas DataFrame 133 | DataaFrame with observations. 134 | wd_name : str, optional 135 | wind direction column name. The default is 'wd'. 136 | Returns 137 | ------- 138 | mage : numpy.float64 139 | wind direction mean gross error. 140 | ''' 141 | wd_df = pd.DataFrame({ 142 | 'mi': model_df, 143 | 'oi': obs_df}) 144 | wd_df.dropna(how="any", inplace=True) 145 | if wd_df.empty: 146 | mage = np.nan 147 | else: 148 | dif = wd_df.apply(lambda row: wind_dir_diff(row['mi'], row['oi']), 149 | axis=1) 150 | mage = dif.abs().mean() 151 | return mage 152 | 153 | 154 | def met_stats(data,mets): 155 | ''' 156 | Model performance evaluation for meteorological parameters, 157 | according to Emery (2001), Reboredo et al (2015), Monk et al (2019). 158 | 159 | Parameters 160 | --------- 161 | data: Pandas DataFrame where meteorological parameters are and have suffix '_obs'. 162 | mets: str list 163 | meteorological parametes (tc, rh, ws, wd) 164 | 165 | Return 166 | ------ 167 | Table with statistical results for model performance evaluation as pandas DataFrame. 168 | Mean Bias (MB), Mean Absolute Gross Error (MAGE), Root Mean Square Error (RMSE), and IOA. 169 | ''' 170 | n = {} 171 | MAGE = {} 172 | RMSE = {} 173 | MB = {} 174 | IOA = {} 175 | r = {} 176 | Mm = {} 177 | Om = {} 178 | Msd = {} 179 | Osd = {} 180 | 181 | for met in mets: 182 | df = data[[met+'_mod',met+'_obs']].dropna() 183 | mod = df[met+'_mod'] 184 | obs = df[met+'_obs'] 185 | if met == 'wd': 186 | n[met] = obs.count() 187 | MB[met] = wind_dir_mb(mod,obs) 188 | MAGE[met] = wind_dir_mage(mod,obs) 189 | else: 190 | n[met] = obs.count() 191 | MAGE[met] = (mod - obs).abs().mean() 192 | RMSE[met] = (((mod - obs)**2).mean())**0.5 193 | MB[met] = mod.mean() - obs.mean() 194 | A = ((mod - obs)**2).sum() 195 | B = (((mod - obs.mean()).abs() + (obs - obs.mean()).abs())**2).sum() 196 | IOA[met] = 1 - (A / B) 197 | r[met] = np.corrcoef(mod, obs)[0,1] 198 | Mm[met] = mod.mean() 199 | Om[met] = obs.mean() 200 | Msd[met] = mod.std() 201 | Osd[met] = obs.std() 202 | res = pd.DataFrame({'n':n,'MB':MB,'MAGE':MAGE,'RMSE':RMSE,'IOA':IOA,'r':r,'Mm':Mm, 203 | 'Om':Om,'Msd':Msd,'Osd':Osd}) 204 | return res 205 | 206 | def r_pearson_sig(n, r, alpha, deg_free = 2): 207 | ''' 208 | Calculate Pearson's R significance. With a two-tail test (non-directional). 209 | Based on: 210 | https://medium.com/@shandou/how-to-compute-confidence-interval-for-pearsons-r-a-brief-guide-951445b9cb2d 211 | 212 | Parameters 213 | ---------- 214 | n : int 215 | sample size. 216 | r : float 217 | Pearson R. 218 | alpha : float 219 | test significant level. 220 | deg_free : int, optional 221 | degrees of freedom. The default is 2. 222 | 223 | Returns 224 | ------- 225 | t_cal : float 226 | Calculated t value. 227 | t_cri : float 228 | Critical t value. 229 | ''' 230 | t_cri = scipy.stats.t.ppf(1 - alpha / 2.0, deg_free) 231 | t_cal = r * np.sqrt(n - 2) / np.sqrt(1 - r**2) 232 | if t_cal > t_cri: 233 | print("Significant linear relationship") 234 | else: 235 | print("No significant linear relationship") 236 | return (t_cal, t_cri) 237 | 238 | def r_pearson_confidence_interval(n, r, alpha): 239 | ''' 240 | Calculate Pearson's R confidence intervals, using two-tail test. 241 | Based on: 242 | http://onlinestatbook.com/2/estimation/correlation_ci.html 243 | https://medium.com/@shandou/how-to-compute-confidence-interval-for-pearsons-r-a-brief-guide-951445b9cb2d 244 | 245 | Parameters 246 | ---------- 247 | n : int 248 | sample size. 249 | r : float 250 | Pearson's R. 251 | alpha : float 252 | confidence level (e.g. if 95% then alpha = 0.05). 253 | 254 | Returns 255 | ------- 256 | r_lower : float 257 | lower CI. 258 | r_upper : float 259 | upper CI. 260 | ''' 261 | alph = 0.05 / 2.0 # two-tail test: 262 | z_critical = scipy.stats.norm.ppf(1 - alph) 263 | # r to z' by Fisher's z' transform: 264 | z_prime =0.5 * np.log((1 + r) / (1 - r)) 265 | # Sample standard error: 266 | se = 1 / np.sqrt(n - 3) 267 | # Computing CI using z': 268 | ci_lower = z_prime - z_critical * se 269 | ci_upper = z_prime + z_critical * se 270 | # Converting z' back to r values: 271 | r_lower = np.tanh(ci_lower) 272 | r_upper = np.tanh(ci_upper) 273 | return (r_lower, r_upper) 274 | 275 | 276 | 277 | 278 | 279 | -------------------------------------------------------------------------------- /rain/wrf_rain.py: -------------------------------------------------------------------------------- 1 | import wrf 2 | import numpy as np 3 | import pandas as pd 4 | from netCDF4 import Dataset 5 | #import matplotlib.pyplot as plt 6 | #%matplotlib inline 7 | import glob 8 | import pickle as pkl 9 | 10 | print("Reading each wrfout...") 11 | month = input('month (e.g., 09): ') 12 | year = input('year: ') 13 | scenario = input('scenario: ') 14 | wrfout = [Dataset(i) for i in sorted(glob.glob('../wrfout_diss/wrfout_d02_'+year+'-'+month+'-*'))] 15 | 16 | print("Extracting RAINC and RAINNC variables, named as rainc and rainnc") 17 | rainc = wrf.getvar(wrfout,'RAINC',timeidx=wrf.ALL_TIMES, method='cat') 18 | rainnc = wrf.getvar(wrfout,'RAINNC',timeidx=wrf.ALL_TIMES, method='cat') 19 | 20 | print("Reading file with station location points") 21 | cetesb_stations = pd.read_csv('./cetesb2017_latlon.dat') 22 | print(cetesb_stations) 23 | 24 | # Locating stations in west_east (x) and north_south (y) coordinates 25 | stations_xy = wrf.ll_to_xy(wrfout, 26 | latitude=cetesb_stations.lat, 27 | longitude=cetesb_stations.lon) 28 | cetesb_stations['x'] = stations_xy[0] 29 | cetesb_stations['y'] = stations_xy[1] 30 | 31 | # Filter stations inside WRF domain 32 | filter_dom = (cetesb_stations.x > 0) & (cetesb_stations.x < rainc.shape[2]) & (cetesb_stations.y > 0) & \ 33 | (cetesb_stations.y < rainc.shape[1]) 34 | cetesb_dom = cetesb_stations[filter_dom] 35 | 36 | # Function to retrieve variables from WRF-Chem 37 | def cetesb_from_wrf(i, to_local=True): 38 | wrf_est = pd.DataFrame({ 39 | 'date': rainc.Time.values, 40 | 'rainc': rainc.sel(south_north=cetesb_dom.y.values[i], 41 | west_east=cetesb_dom.x.values[i]).values, 42 | 'rainnc': rainnc.sel(south_north=cetesb_dom.y.values[i], 43 | west_east=cetesb_dom.x.values[i]).values, 44 | 'code': cetesb_dom.code.values[i], 45 | 'name': cetesb_dom.name.values[i]}) 46 | if to_local: 47 | wrf_est['local_date'] = wrf_est['date'].dt.tz_localize('UTC').dt.tz_convert('America/Sao_Paulo') 48 | return(wrf_est) 49 | 50 | print("Extracting data and saving it in a dictionary") 51 | wrf_cetesb = {} 52 | 53 | for i in range(0, len(cetesb_dom)): 54 | wrf_cetesb[cetesb_dom.name.iloc[i]] = cetesb_from_wrf(i) 55 | 56 | print('Exporting to *.pickle') 57 | pkl.dump(wrf_cetesb, open('rain_'+year+'-'+month+'_'+scenario+'.pickle','wb')) 58 | print(''' 59 | !!!!!!!!!!!!!!!!! 60 | !! Succesfully !! 61 | !!!!!!!!!!!!!!!!! 62 | ''') 63 | 64 | 65 | 66 | -------------------------------------------------------------------------------- /wrf_extract/cetesb2017_latlon.dat: -------------------------------------------------------------------------------- 1 | name,code,lat,lon 2 | Americana,290,-22.7242526995067,-47.3395492909725 3 | Araçatuba,107,-21.1868410998752,-50.4393168498844 4 | Araraquara,106,-21.7825221457448,-48.1858318132219 5 | Bauru,108,-22.326608396587,-49.092759305391 6 | Cambuci,90,-23.5677084050662,-46.6122728561358 7 | Campinas-Centro,89,-22.9025248038637,-47.0572107427735 8 | Campinas-Taquaral,276,-22.8746189449543,-47.0589727566842 9 | Campinas-V.União,275,-22.9467284160848,-47.1192808618633 10 | Capão Redondo,269,-23.6683561456718,-46.7800433836767 11 | Carapicuíba,263,-23.5313950283077,-46.8357797276639 12 | Catanduva,248,-21.1419427576013,-48.9830752705603 13 | Centro,94,-23.5478061597037,-46.6424144996321 14 | Cerqueira César,91,-23.5535425616777,-46.6727047692529 15 | Cid.Universitária-USP-Ipen,95,-23.5663417755292,-46.7374142819519 16 | Congonhas,73,-23.6163200788182,-46.6634655273702 17 | Cubatão-Centro,87,-23.8790267306876,-46.418483361896 18 | Cubatão-V.Parisi,66,-23.8494161699591,-46.3886762431427 19 | Cubatão-Vale do Mogi,119,-23.8315890081254,-46.3695687908739 20 | Diadema,92,-23.6858764117137,-46.6116219323751 21 | Grajaú-Parelheiros,98,-23.776265984399,-46.6969610783657 22 | Guaratinguetá,289,-22.8019171385919,-45.1911223603662 23 | Guarulhos,118,-23.4632093829649,-46.4962135994365 24 | Guarulhos-Paço Municipal,264,-23.4555342581023,-46.5185334030574 25 | Guarulhos-Pimentas,279,-23.4401170078091,-46.4099487710006 26 | Ibirapuera,83,-23.5918419901379,-46.6606875025418 27 | Interlagos,262,-23.6805076511499,-46.6750431583082 28 | Itaim Paulista,266,-23.5015473609702,-46.4207368394923 29 | Itaquera,97,-23.5800148313619,-46.4666514114578 30 | IAG,0,-23.6512,-46.6224 31 | Jacareí,259,-23.2941992435412,-45.9682338642508 32 | Jaú,110,-22.2986196635248,-48.5674574048007 33 | Jundiaí,109,-23.192003743039,-46.8970972725384 34 | Limeira,281,-22.5636037767457,-47.4143140341751 35 | Marg.Tietê-Pte Remédios,270,-23.5187058310122,-46.7433200408467 36 | Marília,111,-22.1998094871228,-49.9599697461499 37 | Mauá,65,-23.6685489997822,-46.4660002742069 38 | Mogi das Cruzes,287,-23.518172230148,-46.1868605711067 39 | Mooca,85,-23.5497340450694,-46.6004166453369 40 | N.Senhora do Ó,96,-23.4800987059504,-46.6920519221288 41 | Osasco,120,-23.5267214190038,-46.7920776611899 42 | Parque D.Pedro II,72,-23.5448456586808,-46.6276755923666 43 | Paulínia,117,-22.772321379337,-47.1548428657284 44 | Paulínia Sul,112,-22.7868064343791,-47.1365588816311 45 | Pico do Jaraguá,284,-23.4562688997476,-46.7660977590058 46 | Pinheiros,99,-23.5614598947311,-46.7020165084763 47 | Piracicaba,113,-22.7012223402754,-47.6496526898721 48 | Pirassununga-EM,268,-22.0077128811569,-47.4275644904649 49 | Presidente Prudente,114,-22.1199367347659,-51.4087770680215 50 | Ribeirão Preto,288,-21.1539418919679,-47.828480526621 51 | S.André Capuava,100,-23.6398036991568,-46.4916367657882 52 | S.André-Centro,101,-23.6456163828539,-46.5363346692648 53 | S.André-Paço Municipal,254,-23.6569941995448,-46.5309187564731 54 | S.Bernardo-Centro,272,-23.6986710864522,-46.5462321867186 55 | S.Bernardo-Paulicéia,102,-23.6713539571073,-46.5846678858857 56 | S.José Campos,88,-23.1878873302518,-45.8711976230777 57 | S.José Campos-Jd.Satélite,277,-23.2236454817019,-45.8907999962971 58 | S.José Campos-Vista Verde,278,-23.1836973451392,-45.8308969846329 59 | S.Miguel Paulista,236,-23.4985264108007,-46.4448027795288 60 | Santa Gertrudes,273,-22.4599552691691,-47.5362983394974 61 | Santana,63,-23.5059927222475,-46.6289602999862 62 | Santo Amaro,64,-23.6549772296114,-46.7099983817602 63 | Santos,258,-23.9630572042475,-46.3211700936539 64 | Santos-Ponta da Praia,260,-23.9812951573645,-46.3005095935412 65 | São Caetano do Sul,86,-23.6184427729695,-46.5563539375078 66 | São José Do Rio Preto,116,-20.7846892839301,-49.3982777928726 67 | Sorocaba,67,-23.5024265843066,-47.4790299116269 68 | Taboão da Serra,103,-23.6093238641752,-46.7582943678197 69 | Tatuí,256,-23.3607515378568,-47.8707990664729 70 | Taubaté,280,-23.0323509589424,-45.5758050249834 71 | -------------------------------------------------------------------------------- /wrf_extract/wrf_extract.py: -------------------------------------------------------------------------------- 1 | import wrf 2 | import numpy as np 3 | import pandas as pd 4 | from netCDF4 import Dataset 5 | #import matplotlib.pyplot as plt 6 | #%matplotlib inline 7 | import glob 8 | import pickle as pkl 9 | 10 | print("Reading each wrfout...") 11 | month = input('month (e.g., 09): ') 12 | year = input('year. ') 13 | wrfout = [Dataset(i) for i in sorted(glob.glob('../wrfout_diss/wrfout_d02_'+year+'-'+month+'-*'))] 14 | 15 | print("Extracting meteorological variables...") 16 | t2 = wrf.getvar(wrfout, 'T2', timeidx=wrf.ALL_TIMES, method='cat') 17 | rh2 = wrf.getvar(wrfout, 'rh2', timeidx=wrf.ALL_TIMES, method='cat') 18 | wind = wrf.getvar(wrfout, 'uvmet10_wspd_wdir', timeidx=wrf.ALL_TIMES, method='cat') 19 | ws = wind.sel(wspd_wdir='wspd') 20 | wd = wind.sel(wspd_wdir='wdir') 21 | psfc = wrf.getvar(wrfout, 'PSFC', timeidx=wrf.ALL_TIMES, method='cat') 22 | 23 | print("Extracting polutants variables...") 24 | o3 = wrf.getvar(wrfout, 'o3', timeidx=wrf.ALL_TIMES, method='cat') 25 | no = wrf.getvar(wrfout, 'no', timeidx=wrf.ALL_TIMES, method='cat') 26 | no2 = wrf.getvar(wrfout, 'no2', timeidx=wrf.ALL_TIMES, method='cat') 27 | co = wrf.getvar(wrfout, 'co', timeidx=wrf.ALL_TIMES, method='cat') 28 | tol = wrf.getvar(wrfout, 'tol',timeidx=wrf.ALL_TIMES, method='cat') 29 | 30 | # Retrieving values from surface 31 | o3_sfc = o3.isel(bottom_top=0) 32 | co_sfc = co.isel(bottom_top=0) 33 | no_sfc = no.isel(bottom_top=0) 34 | no2_sfc = no2.isel(bottom_top=0) 35 | tol_sfc = tol.isel(bottom_top=0) 36 | 37 | print("From ppm to ug/m3...o3, no, no2, tol") 38 | # [ug/m3] = [ppm] * P * M_i / (R * T) 39 | # R = 8.3143 J/K mol 40 | # P in Pa 41 | # T in K 42 | # WRF-Chem gas units in ppmv 43 | R = 8.3144598 # J/K mol 44 | o3_u = o3_sfc * psfc * (16 * 3) / (R * t2) 45 | no_u = no_sfc * psfc * (14 + 16) / (R * t2) 46 | no2_u = no2_sfc * psfc * (14 + 2*16) / (R * t2) 47 | tol_u = tol_sfc * psfc * 92.14 / (R * t2) 48 | 49 | print("Reading file with station location points") 50 | cetesb_stations = pd.read_csv('./stations.csv') 51 | print(cetesb_stations) 52 | 53 | # Locating stations in west_east (x) and north_south (y) coordinates 54 | stations_xy = wrf.ll_to_xy(wrfout, 55 | latitude=cetesb_stations.lat, 56 | longitude=cetesb_stations.lon) 57 | cetesb_stations['x'] = stations_xy[0] 58 | cetesb_stations['y'] = stations_xy[1] 59 | 60 | # Filter stations inside WRF domain 61 | filter_dom = (cetesb_stations.x > 0) & (cetesb_stations.x < t2.shape[2]) & (cetesb_stations.y > 0) & \ 62 | (cetesb_stations.y < t2.shape[1]) 63 | cetesb_dom = cetesb_stations[filter_dom] 64 | 65 | # Function to retrieve variables from WRF-Chem 66 | def cetesb_from_wrf(i, to_local=True): 67 | wrf_est = pd.DataFrame({ 68 | 'date': t2.Time.values, 69 | 'tc': t2.sel(south_north=cetesb_dom.y.values[i], 70 | west_east=cetesb_dom.x.values[i]).values - 273.15, 71 | 'rh': rh2.sel(south_north=cetesb_dom.y.values[i], 72 | west_east=cetesb_dom.x.values[i]).values, 73 | 'ws': ws.sel(south_north=cetesb_dom.y.values[i], 74 | west_east=cetesb_dom.x.values[i]).values, 75 | 'wd': wd.sel(south_north=cetesb_dom.y.values[i], 76 | west_east=cetesb_dom.x.values[i]).values, 77 | 'o3': o3_u.sel(south_north=cetesb_dom.y.values[i], 78 | west_east=cetesb_dom.x.values[i]).values, 79 | 'no': no_u.sel(south_north=cetesb_dom.y.values[i], 80 | west_east=cetesb_dom.x.values[i]).values, 81 | 'no2': no2_u.sel(south_north=cetesb_dom.y.values[i], 82 | west_east=cetesb_dom.x.values[i]).values, 83 | 'co': co_sfc.sel(south_north=cetesb_dom.y.values[i], 84 | west_east=cetesb_dom.x.values[i]).values, 85 | 'tol': tol_u.sel(south_north=cetesb_dom.y.values[i], 86 | west_east=cetesb_dom.x.values[i]).values, 87 | 'code': cetesb_dom.code.values[i], 88 | 'name': cetesb_dom.name.values[i]}) 89 | if to_local: 90 | wrf_est['local_date'] = wrf_est['date'].dt.tz_localize('UTC').dt.tz_convert('America/Sao_Paulo') 91 | return(wrf_est) 92 | 93 | print("Extracting data and saving it in a dictionary") 94 | wrf_cetesb = {} 95 | 96 | for i in range(0,len(cetesb_dom)): 97 | wrf_cetesb[cetesb_dom.name.iloc[i]] = cetesb_from_wrf(i) 98 | 99 | print('Exporting to pickle ') 100 | pkl.dump(wrf_cetesb, open(input('scenario: ')+'_'+year+'_'+month+'.pkl','wb')) 101 | 102 | #name = '_FIN_d02.csv' #input('_name.csv: ') 103 | #def cetesb_write_wrf(df): 104 | # file_name = str(df.code[0]) + name 105 | # df.to_csv(file_name, index=False) 106 | 107 | #for k, v in wrf_cetesb.items(): 108 | # cetesb_write_wrf(v) 109 | 110 | print(''' 111 | !!!!!!!!!!!!!!!!! 112 | !! Succesfully !! 113 | !!!!!!!!!!!!!!!!! 114 | ''') 115 | --------------------------------------------------------------------------------