├── .gitignore ├── LICENSE ├── README.md ├── data └── link_to_data.txt ├── requirements.txt ├── scripts ├── 1d_spde_prob │ ├── 1d_spde.py │ ├── generate_MC_data_a=e^GRF_bounded.py │ ├── generate_data_a=e^GRF.py │ ├── inverse_problem │ │ ├── experimental_data_1d_exp_lx=0.03_var=1.ipynb │ │ ├── input_field_ground_truth.npy │ │ ├── my_model_weights.h5 │ │ ├── posterior_likscale=0.032.pdf │ │ ├── tf_probability_1d_exp_lx=0.03_var=1_inverse_prob.ipynb │ │ ├── ufipy_ground_truth.npy │ │ └── xfipy_ground_truth.npy │ ├── post_processing.ipynb │ └── uq │ │ └── 1dim_MC_code.ipynb └── 2d_spde_prob │ ├── 2d_spde.py │ ├── generate_MC_data │ ├── generate_MC_data_a=e^GRF_bounded.py │ ├── generate_MC_data_a=e^warped_bounded.py │ └── generate_MC_data_uncertain_ls_a=e^GRF_bounded.py │ ├── generate_data │ ├── generate_data_a=e^GRF.py │ ├── generate_data_a=e^stratified.ipynb │ ├── generate_data_a=e^warped.ipynb │ └── generate_data_multiple_ellxs_a=e^GRF_var0.75 │ │ ├── gen_many.py │ │ ├── generate_data.py │ │ └── heateqsolver.py │ ├── post_processing │ └── post_processing.ipynb │ └── uq │ ├── uqcase_1 │ └── 2dim_MC_code.ipynb │ ├── uqcase_2 │ └── 2dim_MC_code.ipynb │ └── uqcase_3 │ └── 2dim_MC_code.ipynb └── trained_models ├── 1d_spde_prob ├── data_file_and_dnn_architecture.txt └── my_model_weights.h5 └── 2d_spde_prob ├── 1_2dim_eg_var0.75 ├── data_file_and_dnn_architecture.txt └── my_model_weights.h5 ├── 2_warped ├── data_file_and_dnn_architecture.txt └── my_model_weights.h5 ├── 3_channelized_permeability_fields ├── data_file_and_dnn_architecture.txt └── my_model_weights.h5 ├── 4_multiple_ellxs_var0.75 ├── data_file_and_dnn_architecture.txt └── my_model_weights.h5 └── 5_merged ├── data_file_and_dnn_architecture.txt └── my_model_weights.h5 /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # C extensions 7 | *.so 8 | 9 | # Distribution / packaging 10 | .Python 11 | build/ 12 | develop-eggs/ 13 | dist/ 14 | downloads/ 15 | eggs/ 16 | .eggs/ 17 | lib/ 18 | lib64/ 19 | parts/ 20 | sdist/ 21 | var/ 22 | wheels/ 23 | *.egg-info/ 24 | .installed.cfg 25 | *.egg 26 | MANIFEST 27 | 28 | # PyInstaller 29 | # Usually these files are written by a python script from a template 30 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 31 | *.manifest 32 | *.spec 33 | 34 | # Installer logs 35 | pip-log.txt 36 | pip-delete-this-directory.txt 37 | 38 | # Unit test / coverage reports 39 | htmlcov/ 40 | .tox/ 41 | .coverage 42 | .coverage.* 43 | .cache 44 | nosetests.xml 45 | coverage.xml 46 | *.cover 47 | .hypothesis/ 48 | .pytest_cache/ 49 | 50 | # Translations 51 | *.mo 52 | *.pot 53 | 54 | # Django stuff: 55 | *.log 56 | local_settings.py 57 | db.sqlite3 58 | 59 | # Flask stuff: 60 | instance/ 61 | .webassets-cache 62 | 63 | # Scrapy stuff: 64 | .scrapy 65 | 66 | # Sphinx documentation 67 | docs/_build/ 68 | 69 | # PyBuilder 70 | target/ 71 | 72 | # Jupyter Notebook 73 | .ipynb_checkpoints 74 | 75 | # pyenv 76 | .python-version 77 | 78 | # celery beat schedule file 79 | celerybeat-schedule 80 | 81 | # SageMath parsed files 82 | *.sage.py 83 | 84 | # Environments 85 | .env 86 | .venv 87 | env/ 88 | venv/ 89 | ENV/ 90 | env.bak/ 91 | venv.bak/ 92 | 93 | # Spyder project settings 94 | .spyderproject 95 | .spyproject 96 | 97 | # Rope project settings 98 | .ropeproject 99 | 100 | # mkdocs documentation 101 | /site 102 | 103 | # mypy 104 | .mypy_cache/ 105 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2019 Predictive Science Laboratory 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Variational-elliptic-SPDE 2 | ## **Simulator-free Solution of High-Dimensional Stochastic Elliptic Partial Differential Equations using Deep Neural Networks** 3 | [SharmilaKarumuri](https://scholar.google.com/citations?user=uY1G-S0AAAAJ&hl=en), [RohitTripathy](https://scholar.google.com/citations?user=h4Qyq9gAAAAJ&hl=en), [IliasBilionis](https://scholar.google.com/citations?user=rjXLtJMAAAAJ&hl=en), [JiteshPanchal](https://scholar.google.com/citations?user=fznjeN0AAAAJ&hl=en) 4 | 5 | Our paper proposes a novel deep neural network (DNN) based solver for elliptic stochastic partial differential equations (SPDEs) characterized by high-dimensional input uncertainty. Our method is intended to overcome the curse of dimensionality associated with traditional approaches for high-dimensional SPDEs. The solution of the elliptic SPDE is parameterized using a deep residual network (ResNet) and trained on a physics-informed loss function. 6 | 7 | #### The novel features of our approach are: 8 | 9 | 1. Our method is simulator-free i.e. liberated from the requirement of a deterministic forward solver unlike the existing approaches. 10 | 2. The DNN is trained by minimizing a physics-informed loss function obtained from deriving a variational principle for the elliptic SPDE. 11 | 12 | The significance of the proposed approach is that it overcomes a fundamental limitation of existing state-of-the-art methodologies – it eliminates the requirement of a numerical solver for the deterministic forward problem whose individual evaluations are potentially very expensive. The proposed methodology is easy to implement and scalable to cases where the uncertain input data is characterized by large stochastic dimensionality. We demonstrate our solver-free approach through various examples where the elliptic SPDE is subjected to different types of high-dimensional input uncertainties. Also, we solve high-dimensional uncertainty propagation and inverse problems. 13 | 14 | ### Deep Resnet architecture: 15 | Resnet_schematic 16 | 17 | ### Results: 18 | These figures shows the comparison of the predicted SPDE solution from our trained deep ResNets and a standard finite volume method (FVM) solver for randomly sampled realizations of the input field. 19 | 20 | 2D Gaussian Random Field (GRF) of length-scales 0.05, 0.08 21 | ![test_case=6_journal-page-001](https://user-images.githubusercontent.com/30219043/69241808-90fc0e00-0b6d-11ea-8b87-0a36d3ca2be9.jpg) 22 | 2D Warped GRF 23 | ![test_case=25_journal-page-001](https://user-images.githubusercontent.com/30219043/69241842-a07b5700-0b6d-11ea-8422-9c4fea7aba61.jpg) 24 | 2D Channelized Field 25 | ![test_case=29_journal-page-001](https://user-images.githubusercontent.com/30219043/69241873-ae30dc80-0b6d-11ea-8be3-e7ea83acf6bd.jpg) 26 | 27 | More experiments and results can be found in our paper at - 28 | ### **Link for the paper:** 29 | https://doi.org/10.1016/j.jcp.2019.109120 30 | 31 | Install dependencies at [requirements.txt](https://github.com/PredictiveScienceLab/variational-elliptic-SPDE/blob/master/requirements.txt) and clone our repository 32 | ``` 33 | git clone https://github.com/PredictiveScienceLab/variational-elliptic-SPDE.git 34 | cd variational-elliptic-SPDE 35 | ``` 36 | ### Data: 37 | All the input datasets used in the paper such as one dimensional and two dimensional gaussian random fields (GRF) - 2D GRF, Warped GRF, Channelized field and Multiple lengthscales GRF etc. to train the deep resnet approximator can be downloaded from link at [/data/link_to_data.txt](https://github.com/PredictiveScienceLab/variational-elliptic-SPDE/blob/master/data/link_to_data.txt). 38 | 39 | ### Scripts: 40 | Keras-tensorflow implementation codes for building a DNN approximator of SPDE solution are at [./scripts](https://github.com/PredictiveScienceLab/variational-elliptic-SPDE/tree/master/scripts). 41 | 42 | For a 2D SPDE problem do 43 | ``` 44 | python 2d_spde.py -train_data='train_channel_nx1=32_nx2=32_num_samples=4096.npy' -test_data='test_channel_nx1=32_nx2=32_num_samples=512.npy' -nx1=32 -nx2=32 -DNN_type='Resnet' -n=300 -num_block=3 -act_func='swish' -loss_type='EF' -lr=0.0001 -max_it=75000 -M_A=100 -M_x=20 -seed=0 45 | ``` 46 | * ```-train_data``` = Training data file 47 | * ```-test_data``` = Testing data file 48 | * ```-nx1``` = Number of grid discretizations in the x1 direction. 49 | * ```-nx2``` = Number of grid discretizations in the x2 direction. 50 | * ```-DNN_type``` = Type of DNN (Resnet:Residual network, FC:Fully connected network) 51 | * ```-num_block``` = Number of blocks in the Resnet 52 | * ```-n``` = Number of neurons in each block of a Resnet 53 | * ```-act_func``` = Activation function used 54 | * ```-loss_type``` = Type of Loss to use for training (EF: Energy Functional, SR: Squared Residual) 55 | * ```-lr``` = Learning rate used 56 | * ```-max_it``` = Maximum number of iterations to train the DNN 57 | * ```-M_A``` = Number of input field images to be picked in each iteration 58 | * ```-M_x``` = Number of x=[x1,x2] locations to be picked on each of the picked image 59 | * ```-seed``` = seed number for reproducability 60 | 61 | Results are saved at ```./results``` 62 | 63 | Similar procedure as above for building a DNN approximator for the 1D SPDE problem. 64 | 65 | ### Trained models: 66 | Download the trained model weights of 1D and 2D SPDE problem from [./trained_models](https://github.com/PredictiveScienceLab/variational-elliptic-SPDE/tree/master/trained_models). Corresponding Resnet architecture and input dataset files used to train the Resnet are included in the 'data_file_and_dnn_architecture.txt' . 67 | 68 | ### Acknowledgements: 69 | We would like to acknowledge support from the Defense Advanced Research Projects Agency (DARPA) under the Physics of Artificial Intelligence (PAI) program (contract HR00111890034). 70 | 71 | ### Citation: 72 | If you use this code for your research, please cite our paper https://doi.org/10.1016/j.jcp.2019.109120. 73 | -------------------------------------------------------------------------------- /data/link_to_data.txt: -------------------------------------------------------------------------------- 1 | Link for 1d_spde_prob data: 2 | https://purdue0-my.sharepoint.com/:f:/g/personal/skarumur_purdue_edu/EphFg2rW6VNGvdSqgYeOF5EBFjBNEq1fKKJtf03TjCgF0w?e=kT8FLZ 3 | 4 | 5 | Link for 2d_spde_prob data: 6 | https://purdue0-my.sharepoint.com/:f:/g/personal/skarumur_purdue_edu/EksG3xGPbhxEpx7Q3BSIjjkBA7HSPgIKELaX_rYB8RoGtw?e=V06NTi -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | python>=2.7 2 | keras>=2.2.4 3 | tensorflow>=1.13.1 4 | GPy>=1.9.6 5 | fipy>=3.1.3 6 | matplotlib 7 | seaborn 8 | -------------------------------------------------------------------------------- /scripts/1d_spde_prob/1d_spde.py: -------------------------------------------------------------------------------- 1 | from __future__ import division 2 | import argparse 3 | 4 | ################################################################# 5 | # ====================== 6 | ## note the code is based on cell centers 7 | # ====================== 8 | #parse command line arguments 9 | parser = argparse.ArgumentParser() 10 | parser.add_argument('-train_data', dest = 'train_data', type = str, 11 | default = 'data/train_exp_nx=100_lx=0.03_v=1.0_num_samples=10000.npy', help = 'Training data file.') 12 | parser.add_argument('-test_data', dest = 'test_data', type = str, 13 | default = 'data/test_exp_nx=100_lx=0.03_v=1.0_num_samples=1000.npy', help = 'Testing data file.') 14 | parser.add_argument('-nx', dest = 'nx', type = int, 15 | default = 100, help = 'Number of FV cells in the x direction.')# number of cells/cellcenters/pixels/pixelcenters in x-direction 16 | 17 | 18 | parser.add_argument('-DNN_type', dest = 'DNN_type', type = str, 19 | default = 'Resnet', help = 'Type of DNN (Resnet:Residual network, FC:Fully connected network).') 20 | parser.add_argument('-n', dest = 'n', type = int, 21 | default = 64, help = 'Number of neurons in each block.') 22 | parser.add_argument('-num_block', dest = 'num_block', type = int, 23 | default = 1, help = 'Number of blocks.') 24 | parser.add_argument('-d', dest = 'd', type = str, 25 | default = '[5,5]', help = 'Number of neurons per layer.') 26 | parser.add_argument('-act_func', dest = 'act_func', type = str, 27 | default = 'swish', help = 'Activation function.') 28 | 29 | 30 | parser.add_argument('-loss_type', dest = 'loss_type', type = str, 31 | default = 'EF', help = 'Type of Loss to use for training (EF: Energy Functional, SR: Squared Residual).') 32 | parser.add_argument('-lr', dest = 'lr', type = float, 33 | default = 0.001, help = 'Learning rate.') 34 | parser.add_argument('-max_it', dest = 'max_it', type = int, 35 | default = 1000, help = 'Maximum number of iterations.') 36 | parser.add_argument('-M_A', dest = 'M_A', type = int, 37 | default = 10, help = 'Batch size: number of input field images in each iteration.') 38 | parser.add_argument('-M_x', dest = 'M_x', type = int, 39 | default = 10, help = 'Batch size: number of x-samples on each of the sampled image in each iteration.') ## M_x cannot be greater than nx (FOR THIS CODE) 40 | 41 | 42 | parser.add_argument('-seed', dest = 'seed', type = int, 43 | default = 0, help = 'Random seed number.') # seed for reproducability 44 | parser.add_argument('-variation', dest = 'variation', type = str, 45 | default = 'a', help = 'Model variation currently trying.') 46 | 47 | args = parser.parse_args() 48 | 49 | ################################################################# 50 | import matplotlib 51 | matplotlib.use('PS') 52 | import tensorflow as tf 53 | import random 54 | import numpy as np 55 | import os 56 | os.environ['PYTHONHASHSEED'] = '0' 57 | 58 | seed = args.seed 59 | # Setting the seed for numpy-generated random numbers 60 | np.random.seed(seed=seed) 61 | 62 | # Setting the seed for python random numbers 63 | random.seed(seed) 64 | 65 | # Setting the graph-level random seed. 66 | tf.set_random_seed(seed) 67 | 68 | os.environ['KERAS_BACKEND'] = 'tensorflow' 69 | from keras.models import Model 70 | from keras.layers import Dense, Activation, Input, concatenate, Lambda, Add 71 | from keras.utils import plot_model 72 | from keras import backend as K 73 | from keras.utils.generic_utils import get_custom_objects 74 | 75 | import matplotlib.pyplot as plt 76 | import GPy 77 | from fipy import * 78 | # import matplotlib as mpl 79 | # mpl.rcParams['figure.dpi']= 200 80 | import seaborn as sns 81 | sns.set_context('talk') 82 | sns.set_style('white') 83 | from pdb import set_trace as keyboard 84 | import sys 85 | import time 86 | ################################################################# 87 | # ------------------------------------------------------------ 88 | 89 | # loading data 90 | train_data = np.load(os.path.join(os.getcwd(),args.train_data)) 91 | test_data = np.load(os.path.join(os.getcwd(),args.test_data)) 92 | 93 | # bounding input fields from below and above 94 | lower_bound = np.exp(-5.298317366548036) # 0.005000000000000002 95 | upper_bound = np.exp(3.5) # 33.11545195869231 96 | 97 | train_data = np.where(train_data < lower_bound, lower_bound,train_data) 98 | train_data = np.where(train_data > upper_bound, upper_bound, train_data) 99 | 100 | test_data = np.where(test_data < lower_bound, lower_bound,test_data) 101 | test_data = np.where(test_data > upper_bound, upper_bound, test_data) 102 | 103 | # ------------------------------------------------------------ 104 | 105 | nx = args.nx 106 | 107 | DNN_type = args.DNN_type 108 | n = args.n 109 | num_block = args.num_block 110 | d = args.d 111 | act_func = args.act_func 112 | 113 | loss_type = args.loss_type 114 | lr = args.lr 115 | max_it = args.max_it 116 | M_A = args.M_A 117 | M_x = args.M_x 118 | 119 | variation = args.variation 120 | 121 | # ------------------------------------------------------------ 122 | # needs to be defined as activation class otherwise error 123 | # AttributeError: 'Activation' object has no attribute '__name__' 124 | class Swish(Activation): 125 | 126 | def __init__(self, activation, **kwargs): 127 | super(Swish, self).__init__(activation, **kwargs) 128 | self.__name__ = 'swish' 129 | 130 | def swish(x): 131 | return (K.sigmoid(x) * x) 132 | 133 | get_custom_objects().update({'swish': Swish(swish)}) 134 | # ------------------------------------------------------------ 135 | # BUILD DNN APPROXIMATOR 136 | # ====================== 137 | # DNN network i/p:x,A o/p:prediction 138 | 139 | x = Input(shape=(1,)) 140 | A = Input(shape=(nx,)) # input field image: conductivity image 141 | a_val = Input(shape=(1,)) # input field value: conductivity value at the corresponding 'input x' location 142 | 143 | if DNN_type == 'Resnet': 144 | x_A = concatenate([x,A]) 145 | o = Dense(n)(x_A) 146 | for i in range(num_block): 147 | z = Dense(n, activation = act_func)(o) 148 | z = Dense(n, activation = act_func)(z) 149 | o = Add()([z, o]) 150 | prediction = Dense(1)(o) 151 | print DNN_type 152 | 153 | elif DNN_type == 'FC': 154 | num_neurons_per_layer = map(int, d.strip('[]').split(',')) 155 | x_A = concatenate([x,A]) 156 | z = Dense(num_neurons_per_layer[0], activation=act_func)(x_A) 157 | for n in num_neurons_per_layer[1:]: 158 | z = Dense(n, activation=act_func)(z) 159 | prediction = Dense(1)(z) 160 | print DNN_type 161 | 162 | def myFunc(t): 163 | B1 = 1 # value_left 164 | B2 = 0 # value_right 165 | return ((B1*(1-t[0]))+(B2*t[0])+(t[0]*(1-t[0])*t[1])) 166 | 167 | u = Lambda(myFunc, output_shape=(1,))([x,prediction]) # field of interest : temperature 168 | model = Model(inputs=[x,A], outputs=u) 169 | 170 | # BUILDING LOSS FUNCTIONS 171 | # ====================== 172 | 173 | dudx = tf.gradients(u, x)[0] 174 | 175 | tf_a = a_val 176 | 177 | c = 15. 178 | f = 10. #source 179 | 180 | # loss function 181 | # ====================== 182 | if loss_type == 'EF': 183 | term_1 = 0.5 * ((tf_a * dudx ** 2) + (c*u*u)) 184 | V = term_1 - (f * u) 185 | ef_loss = tf.reduce_sum(V)/(M_x * M_A) 186 | #or 187 | # ef_loss = tf.reduce_mean(V) 188 | loss = ef_loss 189 | print('loss energy functional form') 190 | # create directories 191 | resultdir = os.path.join(os.getcwd(), 'results','loss_EF_form','DNN_type='+str(DNN_type)+'_nx='+str(nx)+'_seed='+str(seed)+'_'+str(variation)) 192 | 193 | elif loss_type == 'SR': 194 | residual = -(tf.gradients(tf_a * dudx, x)[0]) + (c*u) - f 195 | sqresi_loss = tf.reduce_sum(tf.square(residual))/(M_x * M_A) 196 | #or 197 | # sqresi_loss = tf.reduce_mean(tf.square(residual)) 198 | loss = sqresi_loss 199 | print('loss squared residual form') 200 | # create directories 201 | resultdir = os.path.join(os.getcwd(), 'results','loss_SR_form','DNN_type='+str(DNN_type)+'_nx='+str(nx)+'_seed='+str(seed)+'_'+str(variation)) 202 | 203 | if not os.path.exists(resultdir): 204 | os.makedirs(resultdir) 205 | 206 | # ====================== 207 | orig_stdout = sys.stdout 208 | q = open(os.path.join(resultdir, 'loss_output='+str(DNN_type)+'_'+str(nx)+'_'+str(seed)+'_'+str(variation)+'.txt'), 'w') 209 | sys.stdout = q 210 | start = time.time() 211 | print ("------START------") 212 | print args.train_data 213 | print args.test_data 214 | if DNN_type == 'Resnet': 215 | print (nx,DNN_type,n,num_block,act_func,loss_type,lr,max_it,M_A,M_x,seed,variation) 216 | elif DNN_type == 'FC': 217 | print (nx,DNN_type,num_neurons_per_layer,act_func,loss_type,lr,max_it,M_A,M_x,seed,variation) 218 | 219 | plot_model(model, to_file=os.path.join(resultdir,'stoch_heq_nn_fipy.pdf')) 220 | # ====================== 221 | 222 | train = tf.train.AdamOptimizer(lr).minimize(loss) 223 | 224 | init = tf.global_variables_initializer() 225 | sess = tf.Session() 226 | K.set_session(sess) 227 | sess.run(init) 228 | 229 | I=[] 230 | Loss=[] 231 | 232 | weights = sess.run(model.weights) 233 | # print weights 234 | w=[] 235 | [w.extend(weights[j].flatten()) for j in range(len(weights))] 236 | # print len(w) 237 | plt.hist(w, bins=20) 238 | plt.title('Histogram_Weights&biases_all_layers_before_training') 239 | plt.savefig(os.path.join(resultdir,'Histogram_Weights&biases_all_layers_before_training.pdf')) 240 | plt.pause(1) 241 | plt.close() 242 | 243 | # ====================== 244 | print ("--------------------------------------------------------------") 245 | 246 | #defining mesh to get cellcenters 247 | Lx = 1. # always put . after 1 248 | mesh = Grid1D(nx=nx, dx=Lx/nx) # with nx number of cells/cellcenters/pixels/pixelcenters 249 | cellcenters = mesh.cellCenters.value.T #(nx,1) matrix 250 | # print cellcenters 251 | 252 | gridnum_list = [i for i in range(nx)] # grid at cell centers 253 | # print gridnum_list 254 | # print np.shape(gridnum_list) 255 | # print type(gridnum_list) 256 | print ('*****') 257 | 258 | print ("--------------------------------------------------------------") 259 | 260 | for i in range(max_it): 261 | # Get a batch of points 262 | Xi_final = np.zeros((1, 1)) # sampled x's 263 | AAi_final = np.zeros((1, cellcenters.shape[0])) # images of input field: conductivity # or np.zeros((1, nx)) 264 | Ai_val_final = np.zeros((1, 1)) # input field values at sampled x's 265 | 266 | 267 | for t in xrange(M_A): 268 | 269 | # sampling grid locations from gridnum_list 270 | gridnum_sam = np.random.choice(gridnum_list, size=M_x, replace=False, p=None) # p: The probabilities associated with each entry in a.\ 271 | # If not given the sample assumes a uniform distribution over all entries in a. 272 | # replace: Whether the sample is with or without replacement 273 | # print gridnum_sam 274 | # print type(gridnum_sam) 275 | # print np.shape(gridnum_sam) 276 | 277 | # sampled x's coordinates 278 | Xi = np.ndarray((M_x, 1)).astype(np.float32) 279 | for j in range(M_x): 280 | Xi[j, 0] = cellcenters.reshape(1,-1)[0,gridnum_sam[j]] 281 | # print Xi 282 | # print ('########') 283 | Xi_final = np.vstack((Xi_final,Xi)) 284 | 285 | # getting input field images 286 | Ai = train_data[ random.randint(0, np.shape(train_data)[0]-1), : ].reshape(1,-1) 287 | # Ai is one image of input field: conductivity of nx cells/cellcenters/pixels/pixelcenters picked from train_data # returns (1 , nx) matrix 288 | # print Ai 289 | # print ('########') 290 | AAi = np.repeat(Ai, np.shape(Xi)[0], axis=0) # just repeating 291 | AAi_final = np.vstack((AAi_final,AAi)) 292 | 293 | # getting input field values at sampled x's to use them in loss function calculations 294 | Ai_val = np.ndarray((M_x, 1)).astype(np.float32) # or np.ndarray((np.shape(Xi)[0], 1)).astype(np.float32) 295 | for g in range(M_x): # or for g in range(np.shape(Xi)[0]): 296 | Ai_val[g,0] = Ai[0,gridnum_sam[g]] 297 | # print Ai_val 298 | # print ('#################################') 299 | Ai_val_final = np.vstack((Ai_val_final,Ai_val)) 300 | 301 | 302 | Xi_final = np.delete(Xi_final, (0), axis=0) #to delete the first row 303 | AAi_final = np.delete(AAi_final, (0), axis=0) #to delete the first row 304 | Ai_val_final = np.delete(Ai_val_final, (0), axis=0) #to delete the first row 305 | 306 | # print Xi_final 307 | # print AAi_final 308 | # print Ai_val_final 309 | # print ('done') 310 | 311 | sess.run(train,feed_dict={x:Xi_final, A:AAi_final, a_val:Ai_val_final}) 312 | l = sess.run(loss,feed_dict={x:Xi_final, A:AAi_final, a_val:Ai_val_final}) 313 | 314 | I.append(i) 315 | Loss.append(l) 316 | 317 | # display 318 | if i % 500 == 0: 319 | print ("Iteration: "+str(i)+"; Train loss:"+str(l)+";") 320 | # weights = sess.run(model.weights) 321 | # w=[] 322 | # [w.extend(weights[j].flatten()) for j in range(len(weights))] 323 | # # print len(w) 324 | # plt.hist(w, bins=20) 325 | # plt.title('Iteration:='+str(i)+'_ Histogram_Weights&biases_all_layers') 326 | # plt.savefig(os.path.join(resultdir,'Iteration='+str(i)+'_ Histogram_Weights&biases_all_layers.pdf')) 327 | # plt.pause(1) 328 | # plt.close() 329 | # # keyboard() 330 | print ("--------------------------------------------------------------") 331 | 332 | weights = sess.run(model.weights) 333 | # print weights 334 | w=[] 335 | [w.extend(weights[j].flatten()) for j in range(len(weights))] 336 | # print len(w) 337 | plt.hist(w, bins=20) 338 | plt.title('Histogram_Weights&biases_all_layers_after_training') 339 | plt.savefig(os.path.join(resultdir,'Histogram_Weights&biases_all_layers_after_training.pdf')) 340 | plt.pause(1) 341 | plt.close() 342 | model.summary() 343 | model.save(os.path.join(resultdir,'my_model.h5')) 344 | model.save_weights(os.path.join(resultdir,'my_model_weights.h5')) 345 | 346 | plt.plot(I, Loss, 'blue', lw=1.5, label='Iteration_vs_Trainloss') 347 | plt.xlabel('Iteration') 348 | plt.ylabel('Trainloss') 349 | plt.title('DNN_type='+str(DNN_type)+'_nx='+str(nx)+'_seed='+str(seed)+'_'+str(variation)+'_ Iteration Vs Trainloss') 350 | plt.savefig(os.path.join(resultdir,'DNN_type='+str(DNN_type)+'_nx='+str(nx)+'_seed='+str(seed)+'_'+str(variation)+'_ Iteration Vs Trainloss.pdf')) 351 | plt.tight_layout() 352 | plt.legend(loc='best'); 353 | plt.pause(5) 354 | plt.close() 355 | 356 | np.save(os.path.join(resultdir,'I.npy'), np.asarray(I)) 357 | np.save(os.path.join(resultdir,'Loss.npy'), np.asarray(Loss)) 358 | # np.load(os.path.join(resultdir,'I.npy')) 359 | # np.load(os.path.join(resultdir,'Loss.npy')) 360 | 361 | # end timer 362 | finish = time.time() - start # time for network to train 363 | 364 | # TESTING(checking NN solution against fipy solution) 365 | # ====================== 366 | # test cases 367 | nsamples = np.shape(test_data)[0] 368 | # validation error and relative RMS error 369 | val = [] 370 | rel_RMS_num = [] 371 | rel_RMS_den = [] 372 | # get all relative errrors and r2 scores 373 | relerrors = [] 374 | r2scores = [] 375 | # get all things for plots 376 | samples_inputfield = np.zeros((nsamples, nx)) 377 | samples_u_DNN = np.zeros((nsamples, nx)) 378 | samples_u_fipy = np.zeros((nsamples, nx)) 379 | 380 | np.random.seed(23) 381 | for i in range(nsamples): # test cases 382 | ############################################################### 383 | #FIPY solution 384 | value_left = 1 385 | value_right = 0 386 | 387 | Lx = 1. # always put . after 1 388 | 389 | # define mesh 390 | mesh = Grid1D(nx=nx, dx=Lx/nx) # with nx number of cells/cellcenters/pixels/pixelcenters 391 | 392 | # define cell and face variables 393 | phi = CellVariable(name='$T(x)$', mesh=mesh, value=0.) 394 | D = CellVariable(name='$D(x)$', mesh=mesh, value=1.0) ## coefficient in diffusion equation 395 | # D = FaceVariable(name='$D(x)$', mesh=mesh, value=1.0) ## coefficient in diffusion equation 396 | source = CellVariable(name='$f(x)$', mesh=mesh, value=1.0) 397 | C = CellVariable(name='$C(x)$', mesh=mesh, value=1.0) 398 | 399 | # apply boundary conditions 400 | # dirichet 401 | phi.constrain(value_left, mesh.facesLeft) 402 | phi.constrain(value_right, mesh.facesRight) 403 | 404 | # setup the diffusion problem 405 | eq = -DiffusionTerm(coeff=D)+ImplicitSourceTerm(coeff=C) == source 406 | 407 | source.setValue(f) 408 | C.setValue(c) 409 | 410 | # getting input field images 411 | a = test_data[ i , : ].reshape(-1,1) 412 | # 'a' is one image of input field: conductivity image of nx cells/cellcenters/pixels/pixelcenters from test_data #returns (nx,1) matrix 413 | D.setValue(a.ravel()) 414 | 415 | eq.solve(var=phi) 416 | x_fipy = mesh.cellCenters.value.T ## fipy solution (nx,1) matrix # same as cellcenters defined above 417 | u_fipy = phi.value[:][:, None] ## fipy solution (nx,1) matrix 418 | 419 | # x_face=mesh.faceCenters.value.flatten() #cell faces location i.e.edges of the element 420 | # y_face=phi.faceValue() #cell faces location i.e.edges of the element 421 | 422 | # print ('done1') 423 | ############################################################### 424 | #Neuralnet solution 425 | u_DNN = sess.run(u, feed_dict={x:x_fipy, A:np.repeat(a.T, np.shape(x_fipy)[0], axis=0)}) 426 | # print ('done2') 427 | ############################################################### 428 | val.append(np.sum((u_fipy-u_DNN)**2, axis=0)) 429 | rel_RMS_num.append(np.sum((u_fipy-u_DNN)**2, axis=0)) 430 | rel_RMS_den.append(np.sum((u_fipy)**2, axis=0)) 431 | ############################################################### 432 | from sklearn import metrics 433 | r2score = metrics.r2_score(u_fipy.flatten(), u_DNN.flatten()) 434 | relerror = np.linalg.norm(u_fipy.flatten() - u_DNN.flatten()) / np.linalg.norm(u_fipy.flatten()) 435 | r2score = float('%.4f'%r2score) 436 | relerror = float('%.4f'%relerror) 437 | relerrors.append(relerror) 438 | r2scores.append(r2score) 439 | ############################################################### 440 | samples_inputfield[i] = a.ravel() 441 | samples_u_DNN[i] = u_DNN.flatten() 442 | samples_u_fipy[i] = u_fipy.flatten() 443 | ############################################################### 444 | if i<=20: 445 | # Initialize the plot 446 | fig = plt.figure(figsize=(15,7)) 447 | 448 | try: 449 | ax1.lines.remove(lines[0]) 450 | ax2.lines.remove(lines[0]) 451 | lines2.set_visible(False) 452 | except: 453 | pass 454 | ########## 455 | ax1 = fig.add_subplot(1, 2, 1) 456 | ax1.plot(x_fipy, np.log(a), 'g', lw=1.5, label='log(Input field)') 457 | ax1.set_xlabel('$x$', fontsize=14) 458 | ax1.set_ylabel('log(Input field)', fontsize=14) 459 | ########## 460 | ax2 = fig.add_subplot(1, 2, 2) 461 | lines = ax2.plot(x_fipy, u_DNN, 'r', lw=1.5, label='DNN solution') 462 | # lines2 = ax2.plot(x_fipy, u_fipy, 'b', lw=2) 463 | # lines2 = plt.scatter(x_fipy, u_fipy, s=10, cmap='Greens', label='FVM solution') 464 | lines2 = ax2.scatter(x_fipy, u_fipy, s=25, cmap='Greens',label='FVM solution') 465 | plt.title('Rel. $L_2$ Error ='+str(relerror)+', $R^{2}$ = '+str(r2score), fontsize=14) 466 | 467 | ax2.set_xlabel('$x$', fontsize=14) 468 | ax2.set_ylabel(r'$\hat{u}$', fontsize=14) 469 | plt.legend(loc='best') 470 | plt.tight_layout() 471 | ########## 472 | 473 | # plt.suptitle('test_case='+str(i+1)+'_DNN_type='+str(DNN_type)+'_nx='+str(nx)+'_seed='+str(seed)+'_'+str(variation), fontsize=12) 474 | plt.savefig(os.path.join(resultdir,'test_case='+str(i+1)+'_DNN_type='+str(DNN_type)+'_nx='+str(nx)+'_seed='+str(seed)+'_'+str(variation)+'_nnpred-fipy.pdf')) 475 | plt.show() 476 | plt.pause(0.1) 477 | print i 478 | print ("--------------------------------------------------------------") 479 | #################################################################################################################### 480 | plt.close('all') 481 | # https://stats.stackexchange.com/questions/189783/calculating-neural-network-error 482 | # print val 483 | vali_error = np.sum(val)/(np.shape(val)[0]*np.shape(x_fipy)[0]) 484 | print ('validation_error='+str(vali_error)) 485 | 486 | # https://www.rocq.inria.fr/modulef/Doc/GB/Guide6-10/node21.html 487 | rel_RMS_error = np.sqrt(np.sum(rel_RMS_num)/np.sum(rel_RMS_den)) 488 | print ('relative_RMS_error='+str(rel_RMS_error)) 489 | 490 | 491 | np.save(os.path.join(resultdir,'cellcenters.npy'), cellcenters) # or x_fipy 492 | 493 | np.save(os.path.join(resultdir,'samples_inputfield.npy'), samples_inputfield) 494 | np.save(os.path.join(resultdir,'samples_u_DNN.npy'), samples_u_DNN) 495 | np.save(os.path.join(resultdir,'samples_u_fipy.npy'), samples_u_fipy) 496 | 497 | relerrors = np.array(relerrors) 498 | r2scores = np.array(r2scores) 499 | 500 | np.save(os.path.join(resultdir,'relerrors.npy'), relerrors) 501 | np.save(os.path.join(resultdir,'r2scores.npy'), r2scores) 502 | 503 | #plt.figure(figsize=(8, 6)) 504 | plt.hist(relerrors, alpha = 0.7, bins = 100, normed=True, label='Histogram of Rel. $L_2$ Error') 505 | plt.tight_layout() 506 | plt.legend(loc = 'best', fontsize = 14) 507 | plt.savefig(os.path.join(resultdir,'rel_errors_hist.pdf')) 508 | plt.close() 509 | 510 | plt.hist(r2scores, alpha = 0.7, bins = 100, normed=True, label='Histogram of $R^2$') 511 | plt.tight_layout() 512 | plt.legend(loc = 'best', fontsize = 14) 513 | plt.savefig(os.path.join(resultdir,'r2scores_hist.pdf')) 514 | plt.close() 515 | 516 | print "Time (sec) to complete: " +str(finish) # time for network to train 517 | print ("------END------") 518 | sys.stdout = orig_stdout 519 | q.close() 520 | ############################################################### -------------------------------------------------------------------------------- /scripts/1d_spde_prob/generate_MC_data_a=e^GRF_bounded.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | from __future__ import division 3 | import argparse 4 | import numpy as np 5 | import os 6 | import GPy 7 | import matplotlib.pyplot as plt 8 | from fipy import * 9 | from scipy.interpolate import griddata 10 | from pdb import set_trace as keyboard 11 | import time 12 | import random 13 | 14 | #parse command line arguments 15 | parser = argparse.ArgumentParser() 16 | parser.add_argument('-N', dest = 'N', type = int, 17 | default = 10000, help = 'Number of MC samples.') 18 | parser.add_argument('-nx', dest = 'nx', type = int, 19 | default = 32, help = 'Number of FV cells in the x direction.') 20 | parser.add_argument('-lx', dest = 'lx', type = float, 21 | default = 0.02, help = 'Lengthscale of the random field along the x direction.') 22 | parser.add_argument('-var', dest = 'var', type = float, 23 | default = 1., help = 'Signal strength (variance) of the random field.') 24 | parser.add_argument('-k', dest = 'k', type = str, 25 | default = 'exp', help = 'Type of covariance kernel (rbf, exp, mat32 or mat52).') 26 | parser.add_argument('-seed', dest = 'seed', type = int, 27 | default = 19, help = 'Random seed number.') 28 | args = parser.parse_args() 29 | kernels = {'rbf':GPy.kern.RBF, 'exp':GPy.kern.Exponential, 30 | 'mat32':GPy.kern.Matern32, 'mat52':GPy.kern.Matern52} 31 | 32 | num_samples = args.N 33 | nx = args.nx 34 | ellx = args.lx 35 | variance = args.var 36 | k_ = args.k 37 | assert k_ in kernels.keys() 38 | kern = kernels[k_] 39 | 40 | os.environ['PYTHONHASHSEED'] = '0' 41 | 42 | seed = args.seed 43 | # Setting the seed for numpy-generated random numbers 44 | np.random.seed(seed=seed) 45 | 46 | # Setting the seed for python random numbers 47 | random.seed(seed) 48 | 49 | #define a mean function 50 | def mean(x): 51 | """ 52 | Mean of the conductivity field. 53 | 54 | m(x) = 0. 55 | """ 56 | n = x.shape[0] 57 | return np.zeros((n, 1)) 58 | 59 | #GPy kernel 60 | k = kern(1, lengthscale = ellx, variance = variance) 61 | 62 | #defining mesh to get cellcenters 63 | Lx = 1. # always put . after 1 64 | mesh = Grid1D(nx=nx, dx=Lx/nx) # with nx number of cells/cellcenters/pixels/pixelcenters 65 | cellcenters = mesh.cellCenters.value.T #(nx,1) matrix 66 | np.save('cellcenters_nx='+str(nx)+'.npy', cellcenters) 67 | 68 | 69 | #get covariance matrix and compute its Cholesky decomposition 70 | m = mean(cellcenters) 71 | nugget = 1e-6 # This is a small number required for stability 72 | Cov = k.K(cellcenters) + nugget * np.eye(cellcenters.shape[0]) 73 | L = np.linalg.cholesky(Cov) 74 | 75 | #define matrices to save results 76 | inputs = np.zeros((num_samples, nx)) 77 | outputs = np.zeros((num_samples, nx)) 78 | 79 | start = time.time() 80 | #generate samples 81 | for i in xrange(num_samples): 82 | #display 83 | if (i+1)%100 == 0: 84 | print "Generating sample "+str(i+1) 85 | 86 | #generate a sample of the random field input 87 | z = np.random.randn(cellcenters.shape[0], 1) 88 | f = m + np.dot(L, z) 89 | sample = np.exp(f) # 'sample' is one image of input field: conductivity image 90 | 91 | # bounding input fields from below and above 92 | lower_bound = np.exp(-5.298317366548036) # 0.005000000000000002 93 | upper_bound = np.exp(3.5) # 33.11545195869231 94 | 95 | sample = np.where(sample < lower_bound, lower_bound, sample) 96 | sample = np.where(sample > upper_bound, upper_bound, sample) 97 | 98 | #FIPY solution 99 | value_left = 1 100 | value_right = 0 101 | 102 | # define cell and face variables 103 | phi = CellVariable(name='$T(x)$', mesh=mesh, value=0.) 104 | D = CellVariable(name='$D(x)$', mesh=mesh, value=1.0) ## coefficient in diffusion equation 105 | # D = FaceVariable(name='$D(x)$', mesh=mesh, value=1.0) ## coefficient in diffusion equation 106 | source = CellVariable(name='$f(x)$', mesh=mesh, value=1.0) 107 | C = CellVariable(name='$C(x)$', mesh=mesh, value=1.0) 108 | 109 | # apply boundary conditions 110 | # dirichet 111 | phi.constrain(value_left, mesh.facesLeft) 112 | phi.constrain(value_right, mesh.facesRight) 113 | 114 | # setup the diffusion problem 115 | eq = -DiffusionTerm(coeff=D)+ImplicitSourceTerm(coeff=C) == source 116 | 117 | c = 15. 118 | f = 10. #source 119 | 120 | source.setValue(f) 121 | C.setValue(c) 122 | 123 | D.setValue(sample.ravel()) 124 | 125 | eq.solve(var=phi) 126 | x_fipy = mesh.cellCenters.value.T ## fipy solution (nx,1) matrix # same as cellcenters defined above 127 | u_fipy = phi.value[:][:, None] ## fipy solution (nx,1) matrix 128 | 129 | # x_face=mesh.faceCenters.value.flatten() #cell faces location i.e.edges of the element 130 | # y_face=phi.faceValue() #cell faces location i.e.edges of the element 131 | 132 | #save data 133 | inputs[i] = sample.ravel() 134 | outputs[i] = u_fipy.flatten() 135 | 136 | #end timer 137 | finish = time.time() - start 138 | print "Time (sec) to generate "+str(num_samples)+" MC samples : " +str(finish) 139 | 140 | print np.shape(inputs) 141 | print np.shape(outputs) 142 | print inputs 143 | print outputs 144 | 145 | #save data 146 | np.save("MC_samples_inputfield_"+\ 147 | k_+"_nx="+str(nx)+\ 148 | "_lx="+str(ellx)+\ 149 | "_v="+str(variance)+\ 150 | "_num_samples="+str(num_samples)+".npy", inputs) 151 | np.save("MC_samples_u_fipy_"+\ 152 | k_+"_nx="+str(nx)+\ 153 | "_lx="+str(ellx)+\ 154 | "_v="+str(variance)+\ 155 | "_num_samples="+str(num_samples)+".npy", outputs) 156 | 157 | # END -------------------------------------------------------------------------------- /scripts/1d_spde_prob/generate_data_a=e^GRF.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | from __future__ import division 3 | import argparse 4 | import numpy as np 5 | import os 6 | import GPy 7 | import matplotlib.pyplot as plt 8 | from fipy import * 9 | from scipy.interpolate import griddata 10 | from pdb import set_trace as keyboard 11 | import time 12 | 13 | #parse command line arguments 14 | parser = argparse.ArgumentParser() 15 | parser.add_argument('-N', dest = 'N', type = int, 16 | default = 10000, help = 'Number of samples of the random inputs.') 17 | parser.add_argument('-nx', dest = 'nx', type = int, 18 | default = 100, help = 'Number of FV cells in the x direction.') 19 | parser.add_argument('-lx', dest = 'lx', type = float, 20 | default = 0.02, help = 'Lengthscale of the random field along the x direction.') 21 | parser.add_argument('-var', dest = 'var', type = float, 22 | default = 1., help = 'Signal strength (variance) of the random field.') 23 | parser.add_argument('-k', dest = 'k', type = str, 24 | default = 'exp', help = 'Type of covariance kernel (rbf, exp, mat32 or mat52).') 25 | # used seed 0 for training data, seed 23 for testing data 26 | parser.add_argument('-seed', dest = 'seed', type = int, 27 | default = 0, help = 'Random seed number.') 28 | args = parser.parse_args() 29 | kernels = {'rbf':GPy.kern.RBF, 'exp':GPy.kern.Exponential, 30 | 'mat32':GPy.kern.Matern32, 'mat52':GPy.kern.Matern52} 31 | 32 | num_samples = args.N 33 | nx = args.nx 34 | ellx = args.lx 35 | variance = args.var 36 | k_ = args.k 37 | assert k_ in kernels.keys() 38 | kern = kernels[k_] 39 | seed = args.seed 40 | 41 | np.random.seed(seed=seed) 42 | 43 | #define a mean function 44 | def mean(x): 45 | """ 46 | Mean of the conductivity field. 47 | 48 | m(x) = 0. 49 | """ 50 | n = x.shape[0] 51 | return np.zeros((n, 1)) 52 | 53 | #data directory 54 | cwd = os.getcwd() 55 | data='data' 56 | datadir = os.path.abspath(os.path.join(cwd, data)) 57 | if not os.path.exists(datadir): 58 | os.makedirs(datadir) 59 | 60 | #GPy kernel 61 | k = kern(1, lengthscale = ellx, variance = variance) 62 | 63 | #defining mesh to get cellcenters 64 | Lx = 1. # always put . after 1 65 | mesh = Grid1D(nx=nx, dx=Lx/nx) # with nx number of cells/cellcenters/pixels/pixelcenters 66 | cellcenters = mesh.cellCenters.value.T #(nx,1) matrix 67 | np.save(os.path.join(datadir, 'cellcenters_nx='+str(nx)+'.npy'), cellcenters) 68 | 69 | 70 | #get covariance matrix and compute its Cholesky decomposition 71 | m = mean(cellcenters) 72 | nugget = 1e-6 # This is a small number required for stability 73 | Cov = k.K(cellcenters) + nugget * np.eye(cellcenters.shape[0]) 74 | L = np.linalg.cholesky(Cov) 75 | 76 | #define matrices to save results 77 | inputs = np.zeros((num_samples, nx)) 78 | 79 | start = time.time() 80 | #generate samples 81 | for i in xrange(num_samples): 82 | #display 83 | if (i+1)%100 == 0: 84 | print "Generating sample "+str(i+1) 85 | 86 | #generate a sample of the random field input 87 | z = np.random.randn(cellcenters.shape[0], 1) 88 | f = m + np.dot(L, z) 89 | sample = np.exp(f) 90 | #save data 91 | inputs[i] = sample.ravel() 92 | 93 | #end timer 94 | finish = time.time() - start 95 | print "Time (sec) to generate "+str(num_samples)+" samples : " +str(finish) 96 | print inputs 97 | 98 | #save data 99 | datafile = k_+"_nx="+str(nx)+\ 100 | "_lx="+str(ellx)+\ 101 | "_v="+str(variance)+\ 102 | "_num_samples="+str(num_samples)+".npy" 103 | np.save(os.path.join(datadir,datafile), inputs) 104 | -------------------------------------------------------------------------------- /scripts/1d_spde_prob/inverse_problem/experimental_data_1d_exp_lx=0.03_var=1.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "metadata": {}, 7 | "outputs": [ 8 | { 9 | "name": "stderr", 10 | "output_type": "stream", 11 | "text": [ 12 | "/Users/sharmila/anaconda/lib/python2.7/site-packages/h5py/__init__.py:34: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n", 13 | " from ._conv import register_converters as _register_converters\n", 14 | "Using TensorFlow backend.\n" 15 | ] 16 | } 17 | ], 18 | "source": [ 19 | "from __future__ import division\n", 20 | "import argparse\n", 21 | "import matplotlib\n", 22 | "matplotlib.use('PS')\n", 23 | "%matplotlib inline\n", 24 | "import tensorflow as tf\n", 25 | "import random \n", 26 | "import numpy as np\n", 27 | "import os\n", 28 | "os.environ['PYTHONHASHSEED'] = '0'\n", 29 | "\n", 30 | "seed = 23\n", 31 | "# Setting the seed for numpy-generated random numbers\n", 32 | "np.random.seed(seed=seed)\n", 33 | "\n", 34 | "# Setting the seed for python random numbers\n", 35 | "random.seed(seed)\n", 36 | "\n", 37 | "# Setting the graph-level random seed.\n", 38 | "tf.set_random_seed(seed)\n", 39 | "\n", 40 | "os.environ['KERAS_BACKEND'] = 'tensorflow'\n", 41 | "from keras.models import Model\n", 42 | "from keras.layers import Dense, Activation, Input, concatenate, Lambda, Add\n", 43 | "from keras.utils import plot_model\n", 44 | "from keras import backend as K\n", 45 | "from keras.utils.generic_utils import get_custom_objects\n", 46 | "\n", 47 | "import matplotlib.pyplot as plt \n", 48 | "import GPy\n", 49 | "from fipy import *\n", 50 | "# import matplotlib as mpl\n", 51 | "# mpl.rcParams['figure.dpi']= 200\n", 52 | "import seaborn as sns \n", 53 | "sns.set_context('talk')\n", 54 | "sns.set_style('white')\n", 55 | "from pdb import set_trace as keyboard\n", 56 | "import sys\n", 57 | "import time \n" 58 | ] 59 | }, 60 | { 61 | "cell_type": "code", 62 | "execution_count": 2, 63 | "metadata": { 64 | "collapsed": true 65 | }, 66 | "outputs": [], 67 | "source": [ 68 | "############################\n", 69 | "# Set path to the folder:\n", 70 | "folder = r\"/Users/sharmila/Desktop/research/work/03_NN_PDE/Level3/part8_image/1_Study_EF_Vs_SR_c=15_f=10_boundedfield/data/\"\n", 71 | "# loading data\n", 72 | "train_data = np.load(os.path.join(folder, 'train_exp_nx=100_lx=0.03_v=1.0_num_samples=10000.npy'))\n", 73 | "\n", 74 | "# bounding input fields from below and above\n", 75 | "lower_bound = np.exp(-5.298317366548036) # 0.005000000000000002\n", 76 | "upper_bound = np.exp(3.5) # 33.11545195869231\n", 77 | "\n", 78 | "train_data = np.where(train_data < lower_bound, lower_bound,train_data) \n", 79 | "train_data = np.where(train_data > upper_bound, upper_bound, train_data)\n", 80 | "\n", 81 | "input_field_data = train_data \n", 82 | "###########################" 83 | ] 84 | }, 85 | { 86 | "cell_type": "code", 87 | "execution_count": 3, 88 | "metadata": {}, 89 | "outputs": [ 90 | { 91 | "name": "stdout", 92 | "output_type": "stream", 93 | "text": [ 94 | "(10000, 100)\n" 95 | ] 96 | } 97 | ], 98 | "source": [ 99 | "print input_field_data.shape" 100 | ] 101 | }, 102 | { 103 | "cell_type": "code", 104 | "execution_count": 4, 105 | "metadata": {}, 106 | "outputs": [ 107 | { 108 | "data": { 109 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAArwAAAFeCAYAAABn3sxXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsnXd4VHX2xt/pLT2BNDBEkABKkyaCCCTKyiJBREHqWhNl\nQV31t6toWEFFV0QRF1RYEQRFDCgQGx1xpYqrIIYaAiGFTMok09v9/THcy9xMybQkk+R8noeH5M69\nd74zmUzeee97zhEwDMOAIAiCIAiCINoowpZeAEEQBEEQBEE0JSR4CYIgCIIgiDYNCV6CIAiCIAii\nTUOClyAIgiAIgmjTkOAlCIIgCIIg2jQkeAmCIAiCIIg2DQlegiAIH9i8eTMyMjKwc+fOgI7XarWo\nrq4O8aoAs9mM8vJy7vtA1llWVoZBgwbhzJkz3Lb8/HyMHj0aN910E+67776gH7+vazh37lyTnJ8g\niPYNCV6CIIgm5ujRo7jzzjt5gjIUnDt3DmPHjsVPP/0U1Hn++c9/YsyYMbjhhhsAACUlJXjppZcg\nEonw4osvIicnB4MGDcK//vUv3HTTTaFYugvJycmYPHkyXnrpJVB7eIIgQo24pRdAEATR1jlx4gSq\nqqpCft6ioiJcunQpqHPs2bMHP/74I3bs2MFtO3fuHOx2O6ZPn44pU6Zw2zt37hzUfTXGww8/jHXr\n1mHLli2YMGFCk94XQRDtC3J4CYIg2jGrVq3CiBEjkJKSwm2zWCwAgMjIyGZdS2xsLMaMGYOPPvqo\nWe+XIIi2DwlegiCahEOHDmH69Ono378/hg0bhjfeeANffPEFMjIyUFJSwmVCv/nmG4wbNw69e/dG\nbm4ud/w333yDyZMno2/fvujfvz9mzZqFAwcO8O5j2bJlyMjIwB9//MHbvnPnTmRkZGDz5s3ctoyM\nDCxevBj5+fnc/Y0aNQpLly6F1WrlHX/u3Dk88cQTGDRoEAYPHoy8vDzodLqAnod//OMfWLRoEQBg\n5syZGD16NG/t+/btw+jRo9GnTx8sWLAAJSUlyMjIwKuvvupyroEDB2LGjBnc8bNnzwYAPP/888jI\nyODtW1dXh/nz5+PWW29F3759MXnyZJfn7+TJkzh69CjuuusubtuMGTNcznvo0CGXDC+7zrVr1+K9\n997D8OHD0b9/f0ybNg0HDx7kzrdx40aXnwXLe++9h4yMDF5ud+zYsTh16hTvHARBEMFCkQaCIELO\njz/+iJycHKSkpGD27Nkwm81Yv369i7AEgHnz5uHee+/FlClTEB0dDcAhhJYtW4Ybb7wRTz31FEwm\nEzZt2oSHHnoIixYtCvhy9zfffAOdToepU6ciKSkJW7ZswfLly6FQKPDYY48BAC5evIgpU6ZAIBBg\n1qxZUKlUyM/Px7Zt2wK6z8mTJ8Nms2Hr1q3Izc1F7969ebc/++yzmD59OqKiotC1a1efz3vHHXeg\nvr4ea9asweTJkzFgwADe7QsXLkTPnj3x17/+FdXV1fj444/x2GOP4euvv8Z1110HANi9ezcA4JZb\nbuGOy83NRc+ePXnn7dq1Ky5fvux2HR9//DF0Oh1mzJgBmUyGTz/9FA8//DA+/PBDDBs2DHfddRde\neeUVFBQUYOLEibxjt23bhptuuon3uAcNGgSRSIQ9e/bw1kUQBBEMJHgJggg5r7zyCqKjo5Gfn8+J\n2PHjx2PcuHEu+w4fPhwvvvgi931RURGWL1+OQYMGYfXq1ZBIJACA6dOnIzs7GwsWLMCoUaO48/pD\nRUUFCgoKkJ6eDgDIzs7GiBEjsHnzZk7wLl26FAaDAZs3b0b37t0BAPfffz8mTZqE8+fP+32f/fv3\nx6+//oqtW7fi1ltvxZAhQ3i333vvvXjyySe570tKSnw6b48ePTB48GCsWbMG/fr1Q3Z2Nu/2G2+8\nEWvXroVQ6LiQl5qain/84x/YvXs3/vKXvwAADh8+jLi4OHTs2JE7btiwYTAYDB7P25ArV65gy5Yt\nnGjNzs7Gn/70J7z++uvYtm0bIiMjkZWVhe+++w5qtRoJCQkAgN9++w0XLlzAvHnzeOdTKpXo3Lkz\njhw54tPzQBAE4QsUaSAIIqScOnUKRUVFuO+++3iitFOnThg/frzL/g1dvF27dsFms+HRRx/lxC4A\nRERE4KGHHoJOp8N///vfgNbWq1cvTuwCgFwuR3p6OmpqagAADMNg7969GDRoECd2AUClUvGKt0LJ\n0KFDm+S8Y8eO5cQuAPTp0wcAUFlZyW27dOkS0tLSgrqfMWPG8Bzajh07YsKECTh9+jQuXrwIAJgw\nYQJsNhu++eYbbr+tW7dCLBa7/RB03XXXcccSBEGEgnbl8BqNRpw4cQIdOnSASCRq6eUQRJvk559/\nBgDExMS4uJXx8fEAgPLycq4nrUAg4O136tQpAA6R2fD4mJgYAMDvv/+OPn36oK6uDoDDuXUusFKr\n1QCA6upq3jncnZNhGFgsFpSUlKC2thZarRYJCQku+7HiXa1W++TC2mw2VFZWNtrGi31OQg3rpLLI\nZDIAjr69LNXV1X7FKNzBtjJzpkuXLgAcWegrV66gU6dOiI+Px+bNmzF69GjYbDYUFBRg4MCB0Ov1\n0Ov1vONFIhHq6+tRVFTE+9BDEAThDef3XblczrutXQneEydOYNq0aS29DIJoF7z++ut4/fXX3d7m\n/Hv48ssvN7pPQ1atWoVVq1Zx3+fk5Ljd780338Sbb77JfX/gwAFkZma63dd5+9atW7F161a3+82f\nP9/jutyxfv16r7f78+HbZrP5vK+zu+ttn2B73roTpHa7HYAjD/3aa69x26uqqnjP88GDBz3+PADg\nT3/6U1BrIwiifbJ+/XoMHDiQt61dCd4OHToAcDwRSUlJLbwagmibnDp1Ck888QRmzZqFmTNn8m5b\nsWIF8vPzsX79evzvf//Dm2++iZdffhnDhw/n9tmwYQNWrlyJ1157zSXv+tVXX2HZsmXIy8vD7bff\njnXr1mH16tV49913ceONN7rs99xzz3GiKTMzE7feeisWLlzIO+ff/vY3nD17Flu3bgXDMMjOzkZa\nWhqWLVvm9r4brtcT5eXlmDZtGve+4ytiseNt2WQy8bbrdDoXJzRYEhISuDhHoBQXF7tsY7PO7M9k\n/fr1MBgMeOSRR5CTk4PLly9j7969+OKLLyCVSl2Of+aZZ3DhwgVs2rQpqLURBNG+8Pa+264EL+uk\nJCUloVOnTi28GoJom6SkpKBz587YvXs35s6di4iICACOy+d79+4F4PgdjIuLA+AQXc6/j/fccw9W\nrVqFL7/8EuPHj+ccRK1Wiy+//BJyuRzZ2dmIiIhAt27dADhiBuw57HY719IqLi6Od26FQuHyuy+T\nySAUCrntf/rTn7Bp0yaUlpZi8ODBABwxgG+//dbtehtDJBJx7z2s8+mN2NhYSCQSnDx5krfdOf/K\nwrq4vpzXHampqSgsLAzoWJaCggLMmTOH+wNTXl6Obdu2oX///pyxwL7n3njjjTh27BhKSkowduxY\nXH/99W7PWV1djc6dO9P7NEEQAeHuylm7ErwEQTQ9QqEQL730EnJzc3Hvvffi/vvvh81mw6effgqt\nVgvAkdv1RNeuXfHYY4/hgw8+wOTJk3H33XfDYrEgPz8fly9fxsKFCzkRfccdd+DVV1/F4sWLUVNT\ng7i4OBQUFHD54EB46qmn8MMPP+Cxxx7DzJkzkZCQgK+++iqoc7I53c8++wxqtRp33323x31lMhnG\njBmDgoICzJ07F7fddhsKCwuxdetWJCYm8vZlc7qsO33PPff4ta5bb70VBw4cwIULF7jcrb9YrVbc\nd999mDp1KhiG4SIcDbsvAI7iNba/sKfuD9XV1bh48SKvJzNBEESwUJcGgiBCzu233473338fUVFR\nWLp0KT7++GP8+c9/xgMPPAAAbi9jO/O3v/0Nb7zxBgQCAd5++218+OGH6Ny5M1avXo377ruP2y86\nOhqrVq1Cr1698OGHH2LZsmXo0aMHlixZEvDaO3bsiI0bNyIzMxOff/453nnnHaSnp+Oll14K+Jyj\nR49GZmYm9u7di4ULF7rEFRoyf/583H///Th8+DBeeeUVnD59Gh999BFSU1N5+/Xu3Rv33Xcfjh8/\njtdeew2lpaV+rWvUqFEAEFQLsPvvvx/Z2dn46KOP8OGHH6JXr1747LPPXPoNA8C4ceMgkUjQqVMn\nl77BLIcPHwYAbkAHQRBEKBAwwVYstCJKSkqQmZmJXbt20aUygmgi7HY7ampq3HYfmD9/Pr744gv8\n+uuvbb76vrW83zzwwAMQiURYt26dX8exj2/mzJlu3VznfdjnoLq6GiNGjEBOTg7mzJnj9pgnnngC\npaWl+Oqrr/x+LARBtG+8ve+Sw0sQREhhGAYjR47E3Llzedu1Wi327NmDXr16tXmx25rIzc3F0aNH\nceHChSa/rw0bNsBut7tMXGNRq9XYt28fNwSEIAgiVFCGlyCIkCISiTBhwgRs3LgRzz33HAYNGsQV\nnFVVVeFf//pXSy8xaH755ZdGByOwmd/q6uqwdnhvv/123HLLLVi+fHmT/WyWLl0KjUaD/fv3Izs7\n2yWawfLBBx/gxhtvpHZkBEGEHBK8BEGEnLy8PFx//fXYvHkzduzYAZlMhj59+uDll1/GzTff3NLL\nC5rPP/8cX375pU/7Xrx4kZtyFq688soryM7OxqlTp5CRkRHy89fV1eHQoUMYPXo0b4y0M5cvX8am\nTZuwceNGn3oIEwRB+ANleAmCIJoAer+h54AgiOaFMrwEQRAEQRBEu4UEL0EQRDvnt99+8zo9rqCg\nAJmZmejXrx9ycnKgVqubcXUEQRDBQ4K3HcAwDK7orrT0MgiCCDMYhkF+fj4eeughWCwWt/sUFhZi\n/vz5WLJkCQ4ePIiEhAQ8//zzzbxSgiBaCwaDISw/FJPgbQf8UPwDUt5KQXGt68x7giDaL++//z7W\nrl3rdarZtm3bkJmZib59+0Iul+PZZ5/F/v37m+UPmtFsRVGpBkaztcnviyCak9GjR6NPnz7o378/\n79/333+PWbNmYeHChS7HMAyD0aNHY/Pmzdi8eTMyMjLw7LPPuuy3fft2ZGRkYNmyZSFb7+bNmz22\nE2zItGnTcPz4cQCOKZDTpk0L2TqCgbo0tAMuai7CxthQrClGWkxaSy+HIIgw4d5770Vubi433cwd\n58+fR//+/bnvY2NjER0djaKiIm60cVOw83AxNu46g4oqHRLjVbg/8wZkDab3L6LtsHTpUm7aoTM2\nmw0LFizAP/7xD17P8gMHDkCr1WLs2LH45ptvEBMTg927d8NoNEIul3P7bdu2DSqVqlkegztqa2u5\nr8ePH4/x48e32FqcIYe3HaC36AEANYaaFl4JQRDhRMeOHSEQCLzuYzAYeH9MAUChUMBgMDTZuoxm\nKzbuOoMytQ52BihT67D++0JotN5HMhNEsITDVYWsrCwIBALs27ePt33Tpk3Izs7mfh9TU1PRpUsX\n7Nmzh9unvr4ev/zyCwYPHuzx/B9//DFGjhyJIUOGYNq0aThx4gQAwGq14p133sGIESMwZMgQzJ07\nFxUVFS7HN3R7dTodMjIyUFJSgtmzZ6O0tBRPPvkk1q5dy9vX2/k3b96Mhx9+GM899xxuvvlmZGVl\nhXzaYtgJ3saKJ3JyclwuAxDe4QSvkQQvQRD+IZfLYTQaedsMBgOUSmWT3WeZWoeKKh1vm7rWiKff\n3odvfypqcUFCtE12Hi7G3Lf24qklezH3rb3YebhlYoBSqRQTJkzg9frWaDTYsWMHpkyZwtt3/Pjx\n+Prrr7nvv/vuO4wePRpSqdTtuYuLi7F06VKsX78eBw8exC233IJFixYBAN59913s2rULn376Kfbu\n3YuoqCg8+eST8Kd77b///W+kpKRg6dKlmDlzJu+2xs7/448/YtiwYTh8+DBmzJiBhQsXwmQK3Yfc\nsBG8vhRPAMDJkyexfv16/PLLL9w/wjvk8BIEEShdu3ZFUVER9311dTU0Gg26du3aZPeZnKBCYrzr\nJdnKWgM++PI4nlqyF3MW78Fn3xeS8CVCgrurCht3nWnS19ff/vY3DBw4kPv397//nbtt8uTJ+OGH\nH1BT4/i7XVBQgL59+7r83o0dOxY//vgjtFotAEdmNjs72+N9isViWCwWbNy4EYWFhZg9ezbWr18P\nANiyZQtmz56NTp06QaFQ4IUXXsBvv/2G8+fPh+TxNnb+lJQUTJgwAWKxGBMmTIBWq0VVVVVI7hsI\nI8HrS/FEVVUVqqur0b1792ZcWevHYHVceiSHlyAIfxk3bhy2b9+Oo0ePwmQyYcmSJRgxYgRiY2Ob\n7D7lUjHuz7wBCTFyl9tsdgZ2Biiv0uPT7acwZ/Eecn2JoHF3VaGiSocytc7DEcGzZMkSHD16lPv3\nxhtvcLd16dIF/fv359zbTZs2ubi7gCOW1LdvX+zYsQPl5eUoLy/HgAEDPN5namoqVq5ciRMnTmDK\nlCkYOXIkNm3aBMChsZzHfiuVSsTGxrqNNQRCY+d3fk8Rix0lZna7PST3DYSR4L333nuxZcsW9O7d\n2+M+J0+ehEqlQk5ODm655RZMmTKFHF4fYB3eakN1C6+EIIjWQF5eHvLy8gAAPXv2xMKFCzFv3jwM\nHToUV65c4S6BNiVZg9PwztMj0SFG4XW/8io95/q25GVoonXj7qpCYrwKyQktV/w1efJkfPXVVygs\nLER5eTnuuOMOt/vdfffdKCgoQEFBAcaNG+f1nNXV1VAqlfjPf/6Dw4cP45lnnsG8efNQUVGBlJQU\nlJaWcvvqdDrU1NQgPj6edw6hUMi7Eu9cpOYNX8/fVISN4PWleMJkMqFfv36YN28efvjhB4wfPx6P\nPvooKisrm2mVrRPK8BIE4Y0hQ4bg0KFD3PcLFizAggULuO/Hjh2L77//HseOHcOHH37YbH+goiNk\nmDomA8kJKggAiITu/0awrm9zXIYm2ibsVYXkBBWEAocAvj/zBsilLdfM6o477sDly5fxwQcfYOLE\niR5zuWPGjMGxY8fwxRdfNNoR4fLly3jwwQfx+++/QyaTITY2FjKZDEqlEhMmTMDy5ctRWloKg8GA\nRYsWoVu3bi5X1dPT03HhwgWcO3cOJpMJH374IU+/SSQSLmLhjK/nbypaVVuyrKwsZGVlcd9PnToV\nn332GQ4dOtTop5r2DGV4CYJorWQNTsPwfqkoU+tQWFyNzXvOorxK73H/crUOR09WYGCvxBYVK0Tr\nw/m1lpygavHXD1u8tnr1auzYscPjfpGRkRg+fDjKysqQnp7u9Zy9e/fGM888gzlz5qC6uhopKSl4\n5513EBkZiUcffRQmkwkPPPAAtFothgwZ4iJmAaBv376YPn06Zs2aBQB4+OGHER0dzd1+zz334KWX\nXsKlS5eQlJTEbff1/E2FgPGn/K4ZOHToEObOnctzG1i+++472O12jB07ltt255134rnnnvNo9TtT\nUlKCzMxM7Nq1C506dQrpusOZSRsnYdMfm3BLp1tw4OEDLb0cgmgXtNf3G2ea4jkwmq34cu9Z7D56\nCRVVegiFAtjs1/6MiYQCMAxDvXsJoh3i7T2nVX381ev1eOutt9C9e3ekpaVhzZo1MBqNGDZsWEsv\nLazhitbI4SUIopUjl4rxwJ09cM/Ibpzr++XecyhX63jil403DO+X2uJOHUEQLU/YvwuwhRMLFizA\nxIkTUVlZiUceeQS1tbXo1asXVq5c2aT9INsClOElCKKtIZeKkZ4SjfSUaIwa0BlHT1bgzXVHefuU\nq3UoLqtDRlpcC62SIIhwIewEr7viCWdycnKQk5PT3Mtq1ThneBmGaba8DEEQRHMgl4oxsFciEuNV\nvDZSDIDF63/G5KzuFG0giHZO2HRpIJoOVvBa7Bbua4IgiLYEW2WfFM+/4ldepafODQRBkOBtDziL\nXIo1EATRVskanIZnpg1Aw4tYTT1AgCCI8IcEbztAb9EjRh4DgIZPEATRtumSHIWkMBsgQBBEy0OC\ntx1gsBjQKcrRnoM6NRAE0ZYJxwECBEG0PPQO0A7QW/RIjUzFiSsnKNJAEESbJ9wGCBAE0fLQu0Ab\nx2KzwGK3IDUyFQA5vARBtA/YtmUEQRAARRraPOzQiZTIFABUtEYQBEEQRPuDBG8bh+3QkBSRBAEE\n5PASBNHuMJqtKCrVUGsygmjHUKShjWOwOBzeCGkEYuQx5PASBNGu2Hm4GBt3nUFFlQ6J8Y4CNhpC\nQRDtD3J42zisw6uUKBGriCXBSxBEu8FotmLjrjMoU+tgZ4AytY6GUBBEO4UEbxuHFbwKiQKx8liK\nNBAE0W4oU+tQUcUfOEFDKAiifUKCt43T0OGlwRMEQbQXkhNUSKQhFARBgARvm8dZ8MYp4ijSQBBE\nu4GGUBAEwUK/9W0cti2ZUqKkSANBEO0OGkJBEARAgrfNw4s0yB1FawzDQCAQtPDKCIIgmgcaQkEQ\nBEUa2jhc0ZpYgVhFLKx2K3QWKtggCIIgCKL9QIK3jdPQ4QVovDBBEARBEO0LErxtnIZdGgAaL0wQ\nBEEQRPuCBG8bx2AxQCgQQiqSksNLEARBEES7hARvG0dv0UMpUUIgEJDDSxAEQRBEu4QEbxtHb9FD\nIVYAAOfw0vAJgiAIgiDaEyR42zh6q8PhBXDN4aVIA0EQBEEQ7YiwE7y//fYbhg8f7vH2goICZGZm\nol+/fsjJyYFarW7G1bU+2EgDAETJoiAUCCnSQBBEu8RotqKoVAOj2drSSyEIopkJG8HLMAzy8/Px\n0EMPwWKxuN2nsLAQ8+fPx5IlS3Dw4EEkJCTg+eefb+aVNj1XdFdCdi5nwSsUCBEjjyGHlyCIdsfO\nw8WY+9ZePLVkL+a+tRc7Dxe39JIIgmhGwkbwvv/++1i7di1yc3M97rNt2zZkZmaib9++kMvlePbZ\nZ7F///425fKeqz6H5LeScbDkYEjOZ7AYOMELgJu2RhAE0V4wmq3YuOsMytQ62BmgTK3Dxl1nyOkl\niHZE2Ajee++9F1u2bEHv3r097nP+/Hl069aN+z42NhbR0dEoKipqjiU2CyV1JbAzdpTUlYTkfM4O\nL+DI8ZLgJQiiPVGm1qGiij9hsqJKhzI1TZ0kiPZC2Ajejh07QiAQeN3HYDBALpfztikUChgMhqZc\nWrPCjv01WELzmPQWPRQSBfd9rDyWIg0EQbQrkhNUSIxX8bYlxquQnKDycARBEG2NsBG8viCXy2E0\nGnnbDAYDlEqlhyNaHzqzQ/AarcZG9vSNhg5vhDQCWrM2JOcmCIJoDcilYtyfeQOSE1QQChwC+J7b\nu6JMraNYA0G0E8QtvQB/6Nq1Ky++UF1dDY1Gg65du7bgqkIL5/BaQ+fwKsV8wcveB0EQRHsha3Aa\nhvdLRZlah8Liany59xwqNv+GxHgV7s+8AVmD01p6iQRBNCGtyuEdN24ctm/fjqNHj8JkMmHJkiUY\nMWIEYmNjm3Udc7+di7s/u7tJzs26r6FyeA1WftGaSqLiXGSCIIj2hFwqRnKCCl/uPccrYFv/fSE0\nWlNLL48giCYk7AVvXl4e8vLyAAA9e/bEwoULMW/ePAwdOhRXrlzBokWLmn1NJ66cwLGyY01yblaM\nhjLDyxO8UhVFGgiCaLe4K2BT1xrx9Nv78O1PRSgq1UCjNVG/XoJoY4RdpGHIkCE4dOgQ9/2CBQt4\nt48dOxZjx45t7mXx0Jg0qNBWwM7YIRSE9jMDGzcIhcNrtVthtpl5RWsR0ggYrAbY7DaIhKKg74Mg\nCKI1wRawNezQUFlrwAdfHofNzkAkFMBuZ5CUQHEHgmgrhL3DG45ojBrYGBuq9FUhPzfn8IYgw8u6\nxA0jDYDD+SUIgmhvsAVsCTFyl9tsdob7nwH16yWItgQJ3gDQmDQAgHJtecjPHcq2ZKyobRhpcL4f\ngiCI9kbW4DS88/RIdIhRNLpvuVqHoycrSPQSRCuHBK+fMAwDjbHpBa/RFnykgXWJG7YlA0CFawRB\ntGuiI2SYOiYDyQkqCACIhO77wAuFAry57iiNIyaIVk7YZXjDHaPVCIvdAgAo05aF/PxsQVmTObxX\nIw1UuEYQRHvHXauyMrUOIqGAy/KyMQc23jC8XyrkUvrTSRCtDfqt9RM2zgA0kcMbwsETrOBViK9d\ntgtVpMFsM+OS5hK6xrWdHsgEQbQ/5FIx0lOikZ4SjVEDOqNMrUNclBzHz6rx5rqjvH3ZccTpKdEt\ntFqCIAKFIg1+wsYZgCbO8IagaM2dwxuqSMOa/61B7xW9qfiNIFoxJ0+exKRJk9CvXz9kZ2fjf//7\nn9v9vvjiC2RmZmLAgAGYMmUKTpw40cwrbR5Y8RsdIcPAXok0jpgg2hAkeP2k1ljLfd1aHN6miDSU\na8thsBpI8BJEK8VkMiE3NxcTJ07EkSNHMGPGDDz++OPQ6fgfhgsLC7F48WKsWrUKR44cwejRo/Hk\nk0+20KqbDxpHTBBtC4o0+AkbaRALxe26SwN7bovNEtR5CIJoGQ4ePAihUIipU6cCACZNmoQ1a9Zg\n3759vF7nxcXFsNvtsNlsYBgGQqEQcrlrS6+2CI0jJoi2AwleP2EjDdfHXt8kRWuhdHjd9eENVaSB\nFbxmmzmo8xAE0TIUFRWha1d+Bj89PR3nz5/nbRs+fDi6dOmCP//5zxCJRFCpVFi7dm1zLrVFYccR\nL1pzhBtWQQVsBNH6oEiDn7AOb0Z8RpM4vFyXhhBmeJ0nrYUq0sA5vHZyeAmiNaLX66FQ8PvQyuVy\nGI38D9smkwndunVDfn4+fvnlF8yaNQt//etfXfZry7gbR8wWsBEE0TogwesnrMPbI6EHao21IXFi\nWRiGCelo4SaNNFjJ4SWI1oxCoXARrUajEUqlkrftvffeQ1JSEnr37g2ZTIbZs2fDYrHgp59+as7l\ntijsOGJn4qLliItqH9EOgmgLkOD1E9bhvSHuBgBAhbYiZOc22UywM3YATZfhFQvFkIlkIYs0UIaX\nIFon119/PYqKinjbioqK0K1bN9620tJSmM3XPtgKBAKIRCKIRKJmWWc44FzABjiGVFTVGvHcsv00\njIIgWgnchK7EAAAgAElEQVQkeP1EY9QgShaF1KhUAKHt1MCKULlYHrJIgwACyEQy3naVVBWySAM5\nvATROhk6dCjMZjM++eQTWCwW5OfnQ61WY/jw4bz9Ro4cifz8fPz++++wWq1YvXo1bDYbBgwY0EIr\nbxmyBqfhzTm3oUOMAjY7AwbXsrzUtYEgwh9K2/uJxqRBtCwaSRFJAEI7bY2NGSQoE1BSVwKGYSAQ\nuB936QsGqwFKidLlHBHSiNB1aaAML0G0SqRSKVauXIl//vOfWLJkCdLS0rBixQoolUrk5eUBABYs\nWIDJkyejrq4Oc+bMQV1dHXr27IlVq1YhIiKihR9B81NdZ0SVhm9GlKt1OHqyAr27JaC6zojkBBUV\nshFEGEK/lX6iMWkQLb8meEPp8LKuKyt4TTYT5OLAM2J6i55XsMaikqhCJnjJ4fXMycqT+PvOvyP/\nvnzIxLLGDyCIZqZHjx7YsGGDy/YFCxZwXwsEAjz22GN47LHHmnNpYQmb5XUuVhMKBXjjk6MQCQWw\n2xkkXe3X26NLHOKi5CSCCSJMoN9AP9EYHQ5vB2UHCCBokkhDvCIegKNwLVjB65zfZQlFpIHNGFOG\n1zP7i/ej4HQBSupKwm4Ec7m2HNM3T8fae9YiJTKlpZdDEK0CNsu7cdcZlKt1EAoFsNkZAOD+L1Pr\n8MGXx2GzMySCCSKMoN82P9GYNEiKSIJEJEGCMiG0gtcp0gA4RGWMPCbg83kSvBHSCOrD2wywP89w\nfI4KThdgV9Eu/Fr+KwlegvADdhjF0ZMVeHPdUbf7+CuCSfwSRNNDv2F+ojFqkBGfAQBIjkxuEoeX\nFbzBtibz6PBKVCitLw363ABleL0Rzs/RgUsHAISm/R1BtDfkUjEG9kp0iTd4w5MIZhiGJrcRRDNA\nXRr8hC1aA4CkiKQmcXjZSEOwnRrYorWGqKSU4W0O2A8w4fgc/VTi6KFqsplaeCUE0Tpx16rM+X9f\nsNkZ2Bnq9kAQzQE5vH7AMIwjwyu/JngL1YUhO39TOLyR0kiX7RGS4CINDMNQH14fCNcPBVX6Ku51\na7KS4CWIQGHjDWVqHZfNLSyuxpd7z6FMrYPoasZX5JT19QTb7WFgr0SKNxBEE0C/VX5gtBphsVuu\nObwqh8MbbPswFucuDUDwwyf0Fj0SVYku24MtWjPZTHB0oQw/MRdOsC56uH0oOFhykPuaHF6CCA65\nVIz0FMffhOgIGdJTojFqQGe/RbBQKMCb645SvIEgmggSvH7ATllzdnjNNjNqjbWIVcQGfX6XorUg\nIw3eMrzBRBpY5xIIz3xqc1JWX4bkyGS3t4Wrw3ug5AD3NTm8BBF6/BHBDbs9sPGG4f1SyekliBAS\nNhnekydPYtKkSejXrx+ys7Pxv//9z+1+OTk56NOnD/r378/9ay40xquC96rDywqdUOV4dWYdBBBw\nnRmaqmgtQhoBq90asBBzFrzhJub85WDJQZyvOR/QsWeqziB1SSr2F+93e3u4dmn46dJP3GhscngJ\nonlgRTArgO8amo53nxmJ/5sxEAzDjzuUq3UoLqtroZUSRNskLASvyWRCbm4uJk6ciCNHjmDGjBl4\n/PHHodO5upAnT57E+vXr8csvv3D/mgt3Di8QQsFr0UElVXHDIoKJNFTpq1BtqHab4VVJHUUWgcYa\neA5vmF2u95fpm6dj4Q8LAzq2WFMMBgxOXDnh9vZw7NJgtVtx6PIhjOoyCgA5vATRkjh3e3CGAbB4\n/c/Yebi4ZRZGEG2QsBC8Bw8ehFAoxNSpUyGRSDBp0iQkJCRg3759vP2qqqpQXV2N7t27t8g6Gzq8\noR4vrDProJKooBA7BG8wDu8z25+B1W7Fg/0fdLktQhrB3V8gtCWHt8ZYw3s8/sC+Hoo17v8ohWOX\nhuMVx6G36HF7l9sB+ObwHis7hj8q/2jqpRFEu4Tt9pAUz78aV16lx+c7T6OwuJq6NxBECAgLwVtU\nVISuXfmTqNLT03H+PP9S88mTJ6FSqZCTk4NbbrkFU6ZMCQuHt6w+RILXokOENIKbrhZohnfHuR1Y\n8+sa/H3Y39EnsY/L7SqJiru/QGhLGd56U33AgpR9PVzUXHR7ezhmeH+65GhHNqzzMMhEMp8+VD20\n5SE8/f3TTb00gmi3ZA1OwzPTBqBh7XN5lR7/t2w/5izeg8++LyThSxBBEBaCV6/XQ6FQ8LbJ5XIY\njfw/xiaTCf369cO8efPwww8/YPz48Xj00UdRWVnZLOts6PBGy6KhlChxuf5ySM6vNWt5kYZAHF6d\nWYecghx0j++OF0e86HafUEYawknM+YvZZobFbgk4lsG+HjwJ3nDs0vBTyU9IjkjGddHXQSaW+RRp\nKNYU41LdpWZYHUG0X7okRyGpQbQBABjGIXw/3X4Kcxbvwbc/FaGoVAON1oSiUg2JYILwkbAoAVUo\nFC7i1mg0QqnkX+LJyspCVlYW9/3UqVPx2Wef4dChQxg3blyTr7OhwysQCJAamRoywauz8CMNgWR4\nlx9ZjqLaIuz7yz7OKW4IRRoc1JvqAQT+GNjXg6dIQzg6vEdLj2JIpyEQCASQiWSNRhq0Zi1qjbXN\ntDqCaL+w0YaNu86gXK2Du6695VV6GlFMEAESFr8Z119/PdatW8fbVlRU5CJiv/vuO9jtdowdO5bb\nZjKZIJPJmmWdrKPnXAiWGpWKy3UhErxmR9GaTOx4PIE4vEW1RUhQJmBE2giP+4Q00hBG7qW/sA53\noLEM9vVQWl8Ki80CiUjCuz3cMrx2xo4LtRcwIWMCADgc3kYEL/varjXWeuz6QRBEaGAHWRSX1WHx\n+p9RXuVaX9DYiGISvwThnrCINAwdOhRmsxmffPIJLBYL8vPzoVarMXz4cN5+er0er776Ks6ePQuL\nxYJVq1bBaDRi2LBhzbJOjUmDKFkUREIRt60pHF6hQAiZSBZQhrfeXO+2M4MzFGlwUG92OLwBRxqu\nOrx2xo7S+lKX28OtS0NZfRnMNjO6xHQBAIfD20ikoaSuhHc8QRBNi1wqRkZaHCZndXcpZPOE84ji\nD748jqeW7MXct/ZSlweCcCIsBK9UKsXKlSvx9ddfY/DgwVi3bh1WrFgBpVKJvLw85OXlAQAmTpyI\nmTNn4pFHHsGgQYOwe/durFy50iX60FRoTBouv8uSGpmK0vpSlz6KgaAz67i4gVwsD8jh1Zq13Dk8\nEWykgY1aCAXCsBFzgcAK/mAjDYBrrMFis3DPTbh8KLhQewEArgleHxxeZ8HrTtQTBNE0ZA1Ow7Jn\nR2HqmAwkxSshACASNj7R01n8btx1hjK+BHGVsLne0aNHD2zYsMFl+4IFC3jf5+TkICcnp7mWxUNj\n1HD5XZaUyBSYbWZUGaq4CWmBwjq8AKCQKALK8Nab6hEpa8ThDVGkIVoWHTZizhN5e/Jwqe4SVmev\ndrmNzfAGE2mIU8Sh2lDtUrgWji44K3jTY9MBOD5U+ePwhupKBkEQviGXivHAnT1wz8huKFPrfBpR\n7Aw7wCItOQplah3FHIh2Db3yG8Fis0AkFEEoEKLWWOvq8EalAnBkHYMVvGyXBsAhRgKJNGjNWhdR\n3pBQRRqiZFFh7/AevnzY42AILsMbRKThpo434YfiH7wK3nDJObOCNy06DQB8KlorqSvhrjaQw0sQ\nLQM7pc2fEcWAY4DFgv8chFAgQJ3OjI5xSowe0Bn3jOpGwpdod9ArvhFGrhmJ29Nux2uZr0Fj0nC9\nd1lSI68K3vrL6JvUN+D7YRiGGzwBAAqxIqBIQ725Hp2iOnndRyFWQABBUF0a5GI55GJ52LiXnjBa\njSjXlsNmt/Gy18C1DG+gj6HOVIeusV2RoExAcS0/0uDsnofLc1RUW4REVSLX9k4mbrwPb0l9CTLi\nM3C66jQJXoIIA1jxC4AbU8yK4MLiamzec5ZX7Fanu/aBm21vtvvnS5g4shsVtxHtirDI8IYzSRFJ\nWPvrWjAM44g0eHF4g8FoNYIBExKHt7EMr0AggEqqCsrhVUqUkIgkYeNeesJkM8HG2FChq3C5LRRd\nGqJl0bgu+jpcrGsdkQY2vwv4VrR2SXMJnaM7IyUyhQQvQYQprAi+a2i62wEWDWHbmz21ZC8NtSDa\nDSR4G2F89/G4XH8ZP5f97LZoLTkiGQIIgs43so6gc4Y3IIfX1HiXBsBRuBZMhlcpUUIqkoaNmPME\n+xw6Z1FZQtGHN1oejbToNJdIg7N7Hi6xjwu1F7j8LuB70VqnyE4keAmileBpgEVD2OI2d0MtSPwS\nbRESvI3w5+5/hlAgxJbCLW6L1iQiCTqqOgbt8LICiXVnFeLAitZ8cXgBh7AOWPBarzq8QknYiDlP\nsILX3c8nmAyv2WaG0WrkHN7i2mJep45wc3htdhsuai6iS3QXbltjDq/BYkCVoQqdokjwEkRrgR1g\nkZygglAAREdIEa2SNHocub5EW4eCO42QoEzA8OuGY+PJjbDYLS4OL3B1+ESoHF6nSIO/E65MVhMs\ndkujXRrY+wk20tCaHF53Px+uD28Aop0bMy2PhkKigM6iQ42xBnGKOADhl+Et05bBYrfwIw2NOLzs\nc9YpqhOqDFUoOF0AhmEgaOx6KUEQLQo7wILtzAAAX+49i91HL6GiSu9S3MbCbnPO+k7O6o6swWnN\nun6CaApI8PpAdkY2ntn+DAC47YCQGpnqcbysr7Dik9eWzM8ML3sOXxzeCGlEUEVrrOANJHbRnPji\n8AYiSLkx07JobvrYRc1FTvDyujSEgQteVFMEAH5leNkYSKeoTqjUV0Jn0aHeXI8oWVSTrpUgiOBx\nLm4D4NLerGFxmzvKq/T4fOdpdEqMRJfkKCpuI1o1FGnwgfEZ47mv3Tq8V4dPBAMrPp0dXn/FJOtY\n+pLhDSrS4FS0Fg7upTd8cngDiDQ4O7zXRV8HALwcL/fzlKjC4jlqOHQCuNqH14vD6yx42W4kFGsg\niNaLc3Gbr0Mtyqv0+L9l+zH3rb1cxlejNVHWl2h10Mc1H+gW1w29OvTCycqTbh3elMgUqPVqmKwm\nyMSygO7DpWgtgAyvPw6vSqpCmTawUbF6i57rORwO7qU3vAle5y4N/l6qd3Z4WcHr3JqMdXhj5DFh\nJXjTYq5dmvTH4S3XlgNwOOU9Eno03UIJgmgW3A218OT6Mk5ji9lhF3Y7g6QEFe65vSu1NyNaBfTq\n9JHsjGyH4PWQ4QUc7pdzFbw/uHN4/Y00sF0HfMnwhiLSYLFZwkLMeYJhGK+RBvb5AgCr3QqJqPHC\nDhZnh7eDqgNkIhnf4b36ASZGHhMWrdsu1F5AckQy5GI5t62xPrwldSWIlcdCJVUhJTIFADm8BNHW\naDjUwjnr2zDly2Z82f9ZEcwwDBLjVbg/8wbK+xJhC0UafOTBfg9iVJdRuKnjTS63OQ+fCBRWIDl3\nafA30uCXwxtspEHsyPCGg5jzhLMY9+bwAv471c4Or1AgdOnFq7foIYAAUbIovz4UnKs+h7s/uxtV\n+ipum8lqwtxv5+J8zXm/1uhMUW0RL84AOBxeG2ODzW5ze0xJXQk3xCQ5MhkACV6CaMuwru+yZ0fh\nzbm3ISle2egxbHuzMrUO678vhEZrgtFspcgDEXb4JXitVisqKytRVVXFa8HUHrgh/gbsnrUbsYpY\nl9tCMXzCOfMJXMvw+vM8+5vhDcXgiXB2eNkPDEkRSdCatagz1fFuZ58vwP/CNWeHFwDXmoxFZ9YF\n1MniSOkRFJwuwMpjK7ltn//+OZYdXobt57b7tUZnGg6dAMDFbzzleJ0Fb4Q0AlGyKBK8BNEOkEvF\nyEiLw+Ss7khOUHnN+DqjrjUi9/VdePTVHS7tzUgEEy1No5GGsrIybNiwAfv378epU6dgt9sBAEKh\nED179sTIkSNx7733Ijk5uckXG66EwuHlujRIr3VpABxixPkytC/n8LVLg96ih52xQyjwz+hnBa/R\navTbGQ3k/gKFFbxdY7uiXFuOy3WXEdXhWocBnsPrp1PNOrxsx4Lroq/Dd2e/427XW/RQSVWQiqQ8\nYd0YrDh+/+j7eO7W5yASivDe4fcAoNGpaM5YbBbctvo2zBk8B5NvmoxLdZfwQMwDvH1kIhl3XrbT\nhDOX6i7h5uSbue9TIlNQqiXBSxDtBef2ZoXF1fhy7zmUqXUQXW1rJnLT3kxrcB1lXPDf8xAKBKjT\nmdExTonRAzrjnlHdAIBrnUb5X6Kp8fgKq66uxptvvondu3fj1ltvxQMPPIBu3bohJiYGdrsdNTU1\nOHXqFI4dO4bx48cjMzMTzz33HOLj45tz/WFBjDwGCrEiOIfXooNQIOREiELsELwGi8FnwetPhpcV\n1gaLgfvaF+yMHUarEUqJEvXmer/cy+/OfofsDdm4qeNNGHHdCDzU/yH0Tuzt8/H+wgneuK7476X/\n4nL9ZfTs0JO7vd5UjwhpBLRmrf+RBqMGKokKYqHjV6ijqiPUejVX/KazBObwssK7WFOMb89+i46q\njjhSeoT3eHyhTFuGQ5cP4ddtvyJSFgmr3eqXw2uymnBFd4VzeAHQ8AmCaIc0zPiWqXWIi5Kjus6I\nwuJqbNx5Gupa7+9NdTrvIjgxnorfiKbH46vqoYcewsyZM7FgwQJIJO6LeQYOHIhp06ZBr9fjq6++\nwoMPPoitW7c22WLDFYFAEPTwCZ1ZB5VExXUKYEWuPyKHdRJ9zfACDpfTH8HLrkchUfid4T2lPgWz\nzQyFWIF/H/k3TlefxtdTv/b5eH9xdngB/nhhO2OHzqJD56jO0Jq1/kcaTPype/GKeFjsFmjNWkTK\nIh0Or0Tld+yD3TdCGoHlR5YjQZnAiXJ/XgtqvZo73wObHM5uQ8HLvsbcOcessHUWvKmRqdh/cb/P\nayAIom3h3Ns3OkKG9JRo3No7BU+/vQ+Vtf4VWTuLYOfiN2cHmIQvEUo8XlveuHEjJk6c6FHsOqNU\nKjF16lTk5+eHdHGtiWB78eosOp5QZSMN/nRq0Jq1EAqEnDvsDfa+/C1cY9ttsaOF/RFz7GPZMWMH\nhl03zCVTG2oaCl5nB57NTLODIgKJNDh37GDbtLFC09nh9efcrNP8YL8H8d3Z77DhxAb8pe9fIBFK\n/BK8lbpKAMDrma9zPyN3RWuAe4fXuSUZC+vwtrf8PkEQnomOkGHqmAy/Rxk3hC1+Yx3gOYv3UN9f\nIqR4/PgklUr9Plkgx7QVUqNScbDkYMDH6yw6ntMakMN79RK9L/1k2fvyt3DNWfBKRVK/ogBsX2G5\nWA6VRNXkl8dZIReriEWsPJbnwLOPmxO8AUQanCeOxSsdUZ4qQxXSY9N5Gd5AHN4nBj2B5UeWw2K3\nYPbg2Vjz65qAHN7xGeMRKYvEv4/8m9eDF3CKNLhxeD0JXrPNjGpDNfd4PfHo1kfBgMGq8at8XjNB\nEK0Tb6OMr1TrEamSAgwDjc7399nyKr3Xvr9srIIiEISveHyV9OjRw+dG/H/88UfIFtRaSYlIweW6\ny34PMGBhIw0szhleX9GatT51aACcHF4/e/HyHF6RBFa71edCNL1FD4VYAYFAAJU08LZovsIKRLlY\n7hI5YeMfrOANJNIQI4/hvndxeM06JEYkQioMTPCmx6Tj0ZsfhcakQY+EHn5P3mPXkaBMQO7AXOQO\nzHXZh3V43Z33iu4KACBRlchtY3vxXq6/3KjgPX7leEC/BwRBtE68jTL2JIIFADRaM4Ruit8Az31/\nSQQTgeDxFbF69Wru699//x0ff/wxHn/8cdx0000Qi8U4efIkVqxYgRkzZjTLQsOd1KhUmGwm1Bhr\nOBHlDw2ztIFmeH3J7wLXMrzBRBqkIoejb7FZfJowZ7AauKhGhCTwwRe+whO8kam8SIOLw+tvpMGo\nQVr0Nce0oeB1zvD64x6z65CIJFgxbgW3XS6Ww2jzT/AKBUKeKG+It6K1hl0oAPCGT/RJ7OP1/vUW\nPVfQRxBE+8QXEdzYlLeG+CuCSfwSLB5fBUOHDuW+XrBgAd544w0MGzaM23bjjTeic+fOmDdvHh58\n8MGmXWUrgHXCKrQVAQlenUXHc2cDzfD60qEBCE2kQSJ05LQsdgtk8EHwWgxc+6tmd3gjU/Frxa/c\nbWxHi1i5o69yQEVrMn7RGgBuYESgXRrMNjNEApGLYx6IwxuniINIKPK4j3NbsoawXSicp891UHbg\nzt0Yeoue+0BEEATB0lAEe5ry5sn1dUdjE+BI/BKAj6OFKyoq3LYbUygU0Gg0IV9UayQpIgkAUK4t\n57W+8hWdWcedA7jm8PoTafDH4Q1FpIEVNL4KOr1Vz0U1VBJVszq8naI6oUJbAYvNAolIEpIMr3OX\nhhh5DIQCIc/hDagtmd3iVij6LXgNas519kRjDq/z4wOuPVfVhupG719v0YNxGUxKEAThHnbKG+sA\n+9r31xM0/phoiE+Cd+TIkXjhhRfwwgsvICMjAwzD4Pjx43j11Vdx1113NfUaWwWJEVcdXl1FQMfX\nm+v5Du9VYeiPyNGategc1dmnfUMdafAFg+VapEElVcFit3ACtClomOFlwKBcW47O0Z1dMrx+dVKw\nWWCwGngOr0goQqw8FlWGqw7v1Uy2WCj269xmm9nt8xGIw8s6sp7w6vA2cLABcPGIGkNNo/evt+gp\nw0sQhN801vc3EBHsLH7Xf1+IQb2SEB3R+FVJom3hk+BdsGAB5s+fj1mzZnGT1kQiEbKzs/Hiiy+G\nZCEnT55EXl4ezp49i7S0NLz88svo16+fy34FBQV4++23UVVVhSFDhuDVV19FQoJ3J6s5cHZ4A6Fh\nwVkgkQa2S4MvBOIgA65Fa4DvDq/B6hRpcBLcMSLPOdNgaBhpABwFV52jO7s4vP64sGw7tYYOaIIy\nAWq9GnbGznusFrvF52JGs80cGodXr8YNcTd43Yfrw+vG4a011ro8PpFQhGhZtM8OL2V4CYIIBnd9\nf4MVwepaI55+ex+mjskgp7ed4dNfpIiICLz11lt4+eWXUVRUBABIT09HRIRv4qoxTCYTcnNzkZub\ni/vuuw9btmzB448/jp07d0KlulbIVVhYiPnz5+Ojjz5CRkYGFi5ciOeffx4rV64MyTqCIVYeC4lQ\nErDgbShWAyla86dLAyuo/Tk/4MHh9TEOwHZpAK5liHVmndfCqmBo6PAC13rxshneQCINbEFXQwc0\nXhmPKkMV9yFCJVVx7q6nqEJDLDbPkQZ/RhSr9WoM7TTU6z7e2pJpjBq3WfRYRSxqjN4dXovNAovd\n4vdriyAIojH8EcHlap3bLHBlrQEbd53B8H6plOltR3j8SR84cMDrgcePH+e+di5wC4SDBw9CKBRi\n6tSpAIBJkyZhzZo12LdvH8aOHcvtt23bNmRmZqJv374AgGeffRZDhw6FWq1ucZdXIBAgMSIxoEiD\nxWaByWbiFZwF0pas3lzvc9Ea5/D64SAD7ovWfHZ4LQZERTiq/p0nvTUVrOCSiWRcP9lLdZd49xur\n8L9oTWO8KnjdOLwXai/wniM9HF97ErINMdvN3PPqjFwsR6W+0qf1MQwDtd6HDK+XwRMakwbpseku\n2+MUcY06vOxryt15CYIgQo03Eexp/HFFlQ5lah2vgI5o23gUvL52XhAIBEH34S0qKkLXrl1529LT\n03H+/HnetvPnz6N///7c97GxsYiOjkZRUVGLC17AEWsIxOFlxVcwDq/NboPeovc50iAUCCETyYKK\nNPid4bXyuzQA/meI/YETvGIZlBIlVBIVLtReAOD4cCAXy6/FDvzI2XpyeBMUCfi59GfuOVJJVLDa\nHZOBzDYzVGh8hLM3h9fX14LGpIHVbvW5aM3deTVG1wwv4LiS0ZjDyz5+q90Km93mtVMEQRBEU+Cc\nBXY3/rhjnBImiw0arYl697YTPP50CwsLm20Rer0eCgV/HK5cLofRyP9DbDAYIJfLedsUCgUMBv9E\nW1ORFJHETajyB1bwOscR/HVgWeHoa6SBvY9AHF4BBJCJZH5neHmRBsm1SENTYbQaIRaKuSxpl5gu\nKKp1RHK0Zi0ipBG81mq+4snhjVfGQ61Xcz8LpUTJPb++PkehKFpzHjrhDX+L1gCHw3v8ynGX7c6w\nghdwuLxKobLRNRMtg6+1E0ePHsWrr76KCxcuoFOnTnjhhReCvrJHEM0FO/54464zqKjSIVIlhdFk\nxXPv7qfeve2IxsdjXcVkMmHr1q1YtmwZamtrcfDgQVRW+naJtTEUCoWLuDUajVAq+X8oPYnghvu1\nFImqRFRo/Y80sNlMZ3dWIHCISl9FjjuXuDEUEkVAGV6lRAmBQOB3htdgMbhmeJvY4WU/OABAemw6\nimocgpftiuGvaAe8OLzKBJhsJlTqHL8XKqnKb0EdirZkPgteD23JzDYzjFaji6AHHA5vY5EGnuB1\nI6aJ8ICtnZg4cSKOHDmCGTNm4PHHH4dOx/+drKiowOOPP47c3FwcO3YMOTk5mDNnjst7MUGEM1mD\n0/DuMyPxrzm3QSETo1breM+32R0NFNn2ZU8t2Ys5i/fgs+8LYTRbW3bRREjxSfAWFxdjzJgxePfd\nd/HBBx+gvr4eGzZswLhx43DixImgF3H99ddzxXAsRUVF6NatG29b165deftVV1dDo9G4xCFaiqSI\nJFzRXYHNbvPrOM7hbZC/VUgUPkcO2CIsXzO8gCMnHIjDy8YA/M7wuuvS0MQOL0/wxqSjqLYIDMNw\nDq+/sQzAi8N7dfjERc1FAIH1Kg5FlwZfBS/782soSrnH58HhrTHUgGE8twFyFrxUuBa+ONdOSCQS\nTJo0CQkJCdi3bx9vvy1btuDWW2/FmDFjIBAIMG7cOKxZswZCoc9+CUGEBXKpGFKJCFeq3U90s9kZ\n2BmgvEqPT7efwpzFe/DtT0UoKtWQ+G0D+PSO9corryArKws7duyAROL4I7lkyRKMGTMGr732WtCL\nGDp0KMxmMz755BNYLBbk5+dDrVZj+PDhvP3GjRuH7du34+jRozCZTFiyZAlGjBiB2NjYoNcQCpIi\nkqaWH7oAACAASURBVGBjbFwvVl9hxWpDd9YfQRqow+tvhtdZtPorFvUWPa8PL9DMDm9MOrRmLaoM\nVag3OQr8Aoo0eHF4gWuCVyVRBSR4PRWt+St4G+vDKxAIIBfLXRxe7vG5cXjjFHGw2C1ef24NIw1E\neOJr7cTvv/+OxMREzJ49G0OGDMHkyZNhs9kgldIkPaL1kZygQmJ84/UUgEP4sq7v3Lf2Yufh4iZe\nHdGU+CR4f/nlF0ybNo3XR1QoFOKRRx4JumANAKRSKVauXImvv/4agwcPxrp167BixQoolUrk5eUh\nLy8PANCzZ08sXLgQ8+bNw9ChQ3HlyhUsWrQo6PsPFc7jhf3BXYYX8E/ksLGI5sjwcg6vH3EAi80C\nq93arBlek83kEmkAgAu1F1wcXn+7NCjECpesbbzS1eFl9/H1Q0FjRWvenFUWXx1ewJHj9cfhZbta\neBs+QQ5v68DX2gmNRoMvvvgCDzzwAH788UeMHz8ejz32GE3ZJFolcqkY92fegOQEx98gkVDA+78h\nrOtbptbh852nUVhcTW5vK8WnVLZSqURlZSXS0/ltik6fPo2oqKiQLKRHjx7YsGGDy/YFCxbwvh87\ndiyvVVk44Tx8ondib5+Pc5fhBa46sD4KUk8usTcU4sAzvAD8yvCyj6O5uzQ0dHgBoKimCPXmenSK\n6uS3IAXcj90FnBzeuqsOrzQwh5d1wZ1hH4fZZuayt55Q69WQiqQ+vRZkYplHh9ddf2Tn8cKdo91P\n9aMMb+vA19oJqVSKESNGcFfcpk2bhv/85z84duwYRo0a1WzrJYhQkTU4DcP7pbr07t285yzKq9zH\nHQCH4/t/y/YjiUYUt0p8cninTJmCvLw87Ny5EwBw7tw5bNy4EXl5eZg0aVKTLrA1wY4X9rc1mac4\ngj8Or6ccsDcCiTQEmuFl74eLNLRAhrdLTBcAQFFtkUuXBn+L1ty5nw0jDYFkeL0VrQG+OaaVukok\nKBN8muwmE7kRvB4yyoCjaA2A19ZkFGloHfhaO5Geng6zmf/6tdvtPl1tIIhwhW1bxvbtvWtoOpY9\nOwpTx2QgKV4JAdy7vgxzbUSxRkvvb60JnxzeJ554ApGRkXjllVdgMBiQm5uL+Ph4PPjgg3j44Yeb\neo2tBtbh9Xf4hKeCM4XYj6I1Dy6xNxRiBXf521f0Fj2iZA5X358ML+vwspEGkVAEmUjWrA5vtDwa\nsfJYFNUUcVPpBAIBxEKx323J2OfAmVh5LAQQoLjWkfNSSfzv0uCtaI19TNHw3ihdbWh86ASLTOza\nCcRTRhngO7yecH7NUqQhfHGunZgyZQq2bNnitnYiOzsbkydPxt69ezFixAisX78eJpMJQ4YMaaGV\nE0TTIJeK8cCdPXDPyG7c0Ap2YlvDj3c0orj14XOjuRkzZmDGjBnQ6/Ww2WyIjPTdSWwvREojoRAr\nAnZ4WeeUJSCH198MbwAOLyvs/cnwOg+sYFFJVU3u8DZ8TtNjHZ0anEc5S4SSkEQaREIRYhXX2nYp\nJIqQFq2xj6kxfJmyxuI1w+vO4b2a4fUmeCnS0Dpgayf++c9/YsmSJUhLS+PVTgCOSFmvXr2wYsUK\nLF68GE8//TTS09Px/vvv88a+E0RbwnloxagBnVFcVofF6392iTtU1hrw+c7T6JQYieR4FQ2wCHM8\n/lTy8/Mxfvx4SKVS5Ofnez0JxRocsOOF/RW89WaH+BIK+AkThUTR6FQr7hxXXWI2G+sLwfThBVwz\nvAzDYFfRLmSmZ7pcTm8YaQAcDmhTO7ysI8mSHpOOn8t+hsVu4Rx1iUjisyC1M3acqz6Hu264y+3t\n8Yp4VBuqIRVJIRaK/Y80eClaYx9TY6j1avRLch0e4A5vGV53Ljb7fFLRWtvA19qJ4cOHuzi/BNEe\nkEvFyEiLw+Ss7lj/faHLiOLyKj0NsGglePxJLF++HJmZmZBKpVi+fLnHEwgEAhK8TiRFJPkdaWDz\npA3x1+FViBXcVDFfCLgPr5gveFkxd7T0KO745A7snbUXt3e5nXdcw0gDcNXhbcZIA+AQvJv+2ATg\nWvxDKpLyIgcV2grUm+vRLY6fZQSA3yp+Q6W+EpnpmW7vM0GZgDPVZ7iMsr9FcSFzeBXBObwqicrt\na4nd7rPDSxlegiDaAFmD0zCoV5LLiGIWm90RemAHWDAMg45xSowe0Bn3jOrG3UYiuOXw+KyPHz8e\nIpEIAPDJJ58gOTmZGo37QFJEEs5Wn/XrGDZP2hB/M7z+5Hf9PT+Lu6I1Vsyx/YdZh7DhcUCDSIOk\n6SMNLoI39lqnEU+Rhhd2vYCfSn7CH7NdW+5tP7cdAHDH9Xe4vU+2NVlDF7y5itasditqDDXooPLe\ng9f5vO4cXndxBsDxATdOEedz0Vq4O7xLDy5F1vVZuLHjjS29FIIgwhznEcXucr0srPhlB1gU/Pc8\nhAIB6nRmnggm4du8eFSwq1evRnW1w8XJyspCbW1tsy2qNZOoCjzS0BB/B0/406EBCL4Pb0Mxx+aI\n3eU23UYamsHhlYn4LbzY1mTAtbyzVCSF2X5NkFYbq3G+5rzbKvTt57bjpo43ITky2e19stlZNloS\nUIZX5NnhbcwxrTHUgAHjV9Gai8ProQsFS2PjhVtLhtdgMeCp75/C+0ffb+mlEATRSmBHFL859zYk\nxSsbPwBAnc6CWq3ZZYobDbJoXjx+vOjZsyemT5+O6667DgzD4PHHH4dY7H739evXN9kCWxtJEUmo\n0lfBYrO4FS7u8CRW5WK5z4IhIIdXooDVboXVbvUpCmG1W2G2mT0OnmDdWneizG2kQaLCFd0Vv9bs\nDz47vCK+w2uwGGC2mVFlqOIJR71Fjx8v/ojZg2Z7vE82SuDigoewS4M3/Bk6AbhvS1ZrrPXo8AJo\n3OG16hEhjYDWrA3rSAMbPTpTfaaFV0IQRGvCOde7cdcZlKl1EAkFsNkZ7v/GKK/ScwVvXZKjyO1t\nBrxmeLdt24b6+nocO3YMN998M1Xl+kCiKhEMGFTqK5ESmeLTMfWmeq6HrzP+Znj96dAAXBOfRqvR\nJ7Hc0KVtKOa8ObweuzTUNm+kIS36WvsYrmhNyC9aY5/z0vpSnnDcX7wfJpsJd3a90+N9spEGNsPb\n3EVrlfpKAH4IXncOr1HjUuznTKwi1utVDL1Fj1h5LLRmbVhHGtjHcLrqdAuvhCCI1kigAyxYaJBF\n8+JR8MbFxWHWrFkAgMuXL2P27NmIiPDPQWyPOE9b81Xwas1adJO6Fkg5j5NtbIhAvaneq0hxByui\nDBaDT4KXFa2smGN72HIO79V4gjtx57FLQ5AZXo1Rgwu1F9A3qS9vO8MwbgWvQqJAUkQSyrXlHovW\nWDf6ct1l9Ensw23fcX4HpCIpbku7zeN6WKEZSIaXYRhY7JagitYCcXjd9eF1dsIbEqeIw8nKkx5v\n11v0iFPE4VLdpbCONLAjwC/UXoDJamp0gh1BEERD2BZmALghFqMGdMaXe89i99FLuFKtR6RKCjAM\nNDrXK32M09hicnubFp+e1UWLFjX1OtoM3PAJre+dGjzFEeRiORgwHguZnNGatUiL8e/TISs+fc3x\nsoLWufWZVCTl4gCcw+tHpCHYDO+//vsvvHv4XdT9o473ocBqt8LO2F0EL+DI8ZZryzlH3F2kAQAu\n11/mHbf93Hbcdt1tLr19nYlXXHV4pf53abDaHfPZWzrSoDFqECNzHSvMEiuPbbQtGTuWuDU4vAwY\nnKs5h14derXwigiCaAs0HGCRnOD4e8CK4IoqvUvBG+v2JlJnhyaDnsEQE8h4YU9xBNZxMlqNjQre\nQLs0sOf3BXexBOc4AJfh9RJpcClaC9LhLawq5C6dO5+bfUxuBW9sOg6UHOB1afAUaWApqy/D8SvH\n8UbWG17XE4zDy+7jrWjNV8HLCu/G8Fi01kiGV2PSwGa3QSQUudyut+jRUdURUpG0VWR4AUesgQQv\nQRChxNn9BcCJYE+DLBjGfWeHRIo8hATqMxZiElX+CV6GYbz24QV8E6QBZXhZh9fH1mSsOHUWvM5x\nAK8Or8UAiVDCK46LkEbAZDPBZrf5tW5nztecBwDUmep429k1uBO8XWO7QgABl+H1Fmlg2VW0C4Dn\ndmQswWR42X3cfbhhu034InhVEhVP/HujYVsys83sGF/cSJcGwFHc5g62k4c/GXRf2HdhX0jztuXa\ncu5DH+V4CYJoDpwL3pITVPAUVnTu7FCm1mH994XQaMPXQGgN+CR4S0tL3bZostls+P3330O+qNaM\nSqpCpDTS5+ETBqsBdsbusUsD0LjIYRiGNyrXV7gMr4+RhoYZXoA/pUxr8dKWzGpwEWHseQKNNTAM\nwwneenM97zZvDu9fB/8Vmydv5nWbcI4csMc6RxqOVxyHVCTlZXrd0dDhFQkcDqgvXRrYfYKNNPga\nZwBcB094GyvMwmbFPbUmYwWvu6EWwTDzq5l4dvuzITtfha4C6bHpSFQlkuAlCKJZ8be9mbrWiKff\n3ketzILAJ8GbmZmJmhrXzN6lS5cwderUkC+qtdNB1YGrlm8MdiRwMA6vyWaCjbEF1aXBF9xFGpzd\nUW9tyZz797KwOddAYw3VhmrO2W3o8HoTvB1VHTGhxwTeY3B2YFnH2znScLr6NLrFdXN7Cd+ZOEUc\nRAIRN5ZXIBC4nN8TXKTBTdGac7zFG5X6Sp+HTrDntTE2zmVnh4Z4c3i58cIeWpOx0/jcDbUIhiu6\nKzhYctDth+9AKNeWIykiCd3ju5PgJQii2XF2e30RvZW1BmzcdQZGs7UZVtf28Jjh/fzzz7FixQoA\nDidtwoQJLpPW6uvrccMNNzTtClshKomK13z//9s78/CoyvP937Mv2RcIIWgIYQkIEnZBFFAUoSyC\ntCqIvXCpWCqV1t2yiLbWqtDWKm70p7J8FVTErbYVELWFQhBETAAhQYUskD2ZfTm/P4b3ZM7MmZkz\n+5Lnc11cJGfOnLwnmZzcc5/7eR5/sBiAmFiVKnj9iWZ/BB1pEClac8+/+h08YTcJCtaA8B1e5u4C\nXd8Dhj/B64lKrhI4sGIO74mmExiYNzDgsZRyJT5c8CGGF3R1jZAqeJnLLObwKuVKKOXKgK+Fhs4G\nyd1BgK6ohMVhgV6ul+Tw5uhckYaADq/SuwNEqBhtRpjtZpjtZpxuPe23i4RUGjobMLZoLPQqPT48\n8WEEVkkQBBE8rL2ZZ2cHp8OJDpNQ3NY3GlBR2YDRQwqokC1IfH635s2bB41GA6fTiUceeQR33HEH\nMjK6RJlMJoNer8dll10Wk4UmE3qVXrLgZbfiw3F42TGCnbTGBGiwkQYvh/eCUAvUlswr0hCmw+su\neINxeD1xj2XYHDY4OAfUCjXOGc7B6rBCIVPgZPNJzBwwU9K6rut/nfD4HqOLfeGvaA2Q1pf5nOEc\nynuVS1on0OUcW+wW6FX6oBxeMcHLcZwgwxsph7fJ2MR//L+z/4uI4GUOb6/0XmgwNKDN7L9YjyAI\nIlqIdXawWB1Yvm4Pzrd2/Y2Wy2V4elMFFbKFgE/Bq1KpcP31rtu+ffr0wciRI31OWiOEBCN4eYfX\nT4Y3UA6SiT12G10q7n14pSBWtCbI8PopWhONNITp8Na01vAfB5Ph9cRdtLPn9cvph2ONx1DfWQ+H\n0wGrw4oBeaHdzQg20uCrI0cgwctxHM4ZzqFnWk/Ja/MshpPk8F4oWhNrTWZ1WOHknHyGN1IOb5Op\nS/DuO7MPNw29KazjdVo7YbAZUJBWwDv33zV/h9G9R4d1XIIgiHBw7+ygVSuxYNogbN35HeobDZC7\nTXGrazRg687vMLG8iJxeiUj6Lu3duxd79+71+fivf/3riC0oFdCr9KjrrJO0byQyvEx4MCEiFea4\nBpvhdS9ak5rhFY00RMDhVcqVsDvt4Tm8bpEG5naX5pTiWOMxnG0/yx9bSqRBDLVCDatTQqTBT9Ea\nEFjwtppbYXPa+E4hUuAd3gs/MykOr79Ig/tdALGWZ6HCHF6NQoP/nf1f2MdjfbJ7pffCgFzXG5kT\nTSdI8BIEkVCwuENFZQOe3lQheKy+0YDv69oxqDi4oVPdFUmCt6JC+E12OBw4c+YM2tvbMX369Kgs\nLJkJyeENI8PL2kOxZv9SCTXS4B5NkJzhtZm8XMNIZHjL8stw9NzRsDK87g4se15pTikAV463rsP1\n5iVUwevZBcIX/orWgMCC95zhHACE5PCyn5kUh1etUCNNlSZatOYueCPZlow5vFNKpmB3zW5YHdaA\nvan9wbqoFKQXoDTX1aYuWQrXampqUFISfqSDIIjkQKtWYvSQAhTkpaGusevvJQfgmc0HcePUgRRt\nkIAkwbtx40bR7U899RTsdqoW9CTWGV4mPJjzJpVQitZ0Sh3ksq7iRbEMr69IQ2FGoWBbJBzey/pc\nhm/PfevT4WWCzh/uGVv2vSjNdQne2o5anGw+iQx1RlDOqTuRKFoDoiN4+diMh8MbKB6Tq8sN7PAq\nNF4/l1BhDu/0/tPxyclP8HX91xhTNCbk47E+2b3Se0Gr1KI4uzhpBO/06dOh0+kwcOBAlJWVoays\nDIMHD8agQYOg00nrv0wQRHKhVSvxs6sH4K1PTwgGVtQ3GWkssUTCGjyxYMECbN++PVJrSRkineGN\nVqQhlD68njlcJuacnJM/51h0abA77fih7Qf0y+mHDE1G2EVrLE7AnleUUQS1Qo2z7Wf5Dg3uo4uD\nIei2ZCEWrYXk8Cq9Hd40VZpgQIgYObocSQ5vxCINpi7BCyDsWAOLNLA3MYncmuyTTz4RfL5nzx6s\nXbsWV155JZqbm/HSSy/hpptuwqhRozBt2rQ4rZIgiGgzdWwxfrtwFDz/FLGxxMue/Yz69PohLMG7\ne/duaLWBBYUUXnvtNVxxxRUYOXIk7rvvPhiN4oKxubkZgwYNwogRI/h/K1eujMgaIkVQDq+fDK/U\n6Vqt5lbB5DCpSG11xTDYDIKWZECXWHQ/X18ObyQzvD+2/QgH50C/nH7I1GSGXbTGxCYT/3qVHr0z\neuNsx1nJLcl84dn2zBfhFq2536aXintbMsDl8EqJxkhyeCPYlqzZ1Iw0VRr65/ZHYXoh9p3ZF9bx\n6jvrIYOM71k8MNcleCPV4zcSNDc3Y/ny5di5c6dge0FBAaZMmYKlS5fiueeew2effYaNGzfi4osv\nxqxZs+K0WoIgYkHfwkz0ykvz2s7RRLaASPK+J02a5OVuGQwGdHZ24sEHHwx7Ebt378aGDRvwxhtv\nID8/H7/5zW/wpz/9CatXr/bat6qqCgMGDMCHHyZu30wmeDmOC+gKdlo7oVaow5qu1WJuQbY2WxA1\nkIpOqZMcafDn8DKnGvCd4Y1klwbWkqxfTj9kqMN0eEUiDTqVDkUZRahprcHp1tO4dfitQa+RITnS\nIKFozd+bA+bwBjVpzcPhbTW3SmrNlaPNEXVEvRzeSLUlMzUhT58HmUyGcX3Ghe3w1nfWI1+fzzvZ\nZfll6LB2YPk/l+OhiQ+hV3qvSCw7LLZs2QKTyYQXX3wx4L5jxozB008/jTfeeCMGKyMIIl6waAPr\n3OD5Fp1NZFswbRDlej2QJHjvvfdewecymQwqlQpDhw5FcXH439AdO3Zg/vz5fCHGr3/9ayxatAgr\nVqyAQiGcbFVZWYmysrKwv2Y0YcLObDd79Z71pMPqeyRwMII32Pyu+9cIJ9LAxKK74BXtwysyWlil\nUEElVwmeKxV3wRuuw6tSqODgHHByTsHzijKLsL1qOzhwYTm8kZi0xtbk3o/Wk3OGc8jT5QWMI7gj\n5vD669DAyNXlClqFMTwzvBErWjM2IU+XBwAYVzQO7x17z7VNnxfS8RoMDQJR+/Pyn2N/7X78bf/f\n8NLBl/DiT17Ez8t/HpG1h8qCBQuwevVqPPDAA/jTn/7Eb7fb7aItIocNG+ZVYEwQROrBOjd8X9eO\nZzYfFGR6ga6JbNSyTIik78TcuXP5jxsbGyGXy5GbG1wbDLvdLhpTkMvlqK6uxjXXXMNvKykpgdFo\nRENDA3r3Fk6NqqqqwpkzZ3Ddddehs7MTV155JR566CFkZgbXgzaaMFFotBkDCt5Oa6dPwSt1nGyr\nuTXoDg0MnUonWfAarAZBSzKgS8wx51EpV3q5ekxIeoplwBVrCCXSUN1SDZVchaKMImRqMsNyeJmj\nanPY+O+FTqlD7/TevOsaruCVIuojUbQWTH4X8H6NtZnbJInIHG0O3x3EHU/BG8kML1sXG6xReb4S\nVxRfEdLx6jvrBdGPdHU6Xr/+day4cgVmbpmJl796Oe6CNzc3F3/961+97maVl5djwIABgoK14uJi\nHD16FCaTtN9lgiCSG/exxJv/eQyNrcK/DQ1NBtQ1GvievoREwetwOPDnP/8Z27ZtQ1ubq4o7Ly8P\nt9xyC5YsWSLpC+3fvx+LFy/22l5UVASFQiHIArNKY7GLd3p6OsaNG4c77rgDNpsNDz74IFatWoV1\n69ZJWkcscBe8efAvHjqsHaItyQDpGdsWU0vQBWsMnVIXVB9eT2HN8qlM0OXqcr1EDju+Z4YXcMUa\nQoo0tFajOLsYCrkCGZoM1HbUCh632C2Qy+SS3E7mqNqcNi+Hl8F6tYaC+3AOf4RbtNZgaAgqvwuI\ntCWztKFfTr+Az8vWZvOjft3fVEStLZmxCcVZrrtJ7E1XOHGJBkOD6CCR/rn9Ud6rHIfqD4V87Egz\nc6Zwwt/LL7+MqqoqVFVVYevWrTh9+jScTidkMhmWL18ep1USBBEPpo4txpghvbwmshXkpaEw3zvr\n252RJHj/8Ic/4NNPP8UDDzyAoUOHwul04ptvvsFzzz0Hm82Ge+65J+AxJkyYgOPHj4s+NmvWLFgs\nXX+8mNBNS/P+Ya1Zs0bw+fLly7Fw4UI4nU7I5WHV4EWMYPrb+nN4AWnjZFvMLQJxFgw6lfQMr8Fm\n8Po6vMN7QbTm6nK93Fb3XKwnaerQBG9NSw0vzHw5vFqlVlJnBeaoWh1Wrwwv4KrkD2fkrPtwDn8E\nzPAqAju8wYwVBkTakpmlRRrYG582cxu06eKCV6PUwOKwSMqyB6LJ1BVpYG8IpPQ2FoPjOJfD66PN\nXL4+H43GxtAWGgMmTJiACRMm8J9bLBb88MMPyMnJQX6+9Pw2QRCpQVa6hp/I1tBk4McOU5xBiKTv\nxvvvv48XXngBY8Z09b0sKytDnz598Nvf/laS4PVHaWkpqqur+c9ramqQmZmJnj2Ft2edTifWrVuH\nG2+8EX369AHgutirVKqEEbuA0OENRIelw293BSmCt9XcimxNaJGGsDO8CmGGN1eXi/OG817PAyAe\naVCFHmn4aeFPAbiGdohleKXEGdg5AC4B5e7w9s5wxWlCHSnMiNVo4XOGc+ipDy3S4O7wShH3TPC2\nmlsFrrKnwwu4hHw4QyIcTgdaTC18pMHdkQ+FDmsHzHazz8K0fH0+mk3NsDvtQeWh44VGo8GAAeG9\nRgmCSG5Yrreu0cA7uzW1bSjMTyPhewFJKlGv13sVjwFARkZGRITm7Nmz8dZbb+G7775DZ2cn/vrX\nv2LmzJlex5bL5Th8+DDWrl0Lo9GI8+fPY+3atYKMcSIQjOCNiMNrCr1oLZguDQarAXqleJcGJlrz\ndHlet5rdc7GehOLwtlva0WRq8nJ43VtKBSV4Lwgoq8MqWCtzswfmhp7fZcePVNGar9eCxW5Bq7k1\n+AyvW9Fap7UTZrsZubrA+Xz2evPM8bpP45PaVi8QreZWcOC8HF4p31Mx2NAJXw5vD72rVZlY2zWC\nIIhERatWoqR3Fr48fBbLnv0M9679jHrzuiFJrd5333149NFHsXPnTjQ3N6OtrQ379u3Do48+iltv\nvRU//vgj/y8UrrrqKtx555246667MHnyZGRkZOCBBx7gHx8xYgRfffzMM8/AYrFg8uTJmDlzJgYO\nHIj7778/pK8bLYJyeP1keAEEbO1ksplgcVhCz/CqgsvwevXhvSDm/GV4/UYaQnB4d9fsBgAM7zUc\ngMvhdXJOgVNtdkgXvHzRmmeG90JB3Ojeo4Nan9jxpdx+l1q0JtYr9rzR5aoHneF1c3iPNR4DAAzK\nGxTweczh9Rw+YbQZoVFoIJfJu+ISYRausW4QzOF1LzIMBTZ0wp/DCyChYw0EQRBimK12bN35Heoa\nDXBe6M27ded3MFtpKq4kn5sJyqVLl/JZPPZH9/jx41i3bh2f06uqqgppIbfeeituvVW81+mhQ10F\nJAUFBXj++edD+hqxIpYOL3PYQu7SoJTWpYHjOJ99eN2L1vJ0ebA5bYLcpt9IgzoNP7YH90Zp8zeb\n0UPfA1eVXAWgawxuu6Vd0BIulEiDyWaCQqZwtUxTqHBq2amQ30wwIjlpjQMnGhEIZcoaIHR4q867\nfncH9xgc8HnukQZ33F8jUruMBII5rbzDG2akgXd4fbw5YIL3vOE80COkL0EQBBEX6hoNaGgSmkjU\nscGFJMHrOemH8E/QGd4ADq8/wcActrD68EqINFgdVjg4h1dbMibO2ixtgnVYHVZe8PiNNATp8LZb\n2vHBiQ9wx4g7+Hwly0B3WDp41y7USIPn84IZ4uDv+MEMnvAXaQBc5xYpwauUKyGDDBa7BVWNVVDK\nlSjNKQ34PEmC16PHb6iw3sMsahFu0RqbSOfL4WXT18jhJQgi2SjMT0NBXhrqGrv+rlLHBheSBG9R\nUVdlPsdxXrdUE6lgLBGQKnhtDhssDktEHN5w2pJJcXh9ubRMeLWYWqBX6XlRa3FYugRvoEhDEBne\n7VXbYbabsWDYAn6bu8PLCEbwukcaxAZkhIvULg1WhxUqucpnRwN3wcvOmcFu0/vKpfpCJpPxI4BP\ntpzEgNwBPh1md3wKXnuX4JU6OCUQnpEG9zcooVDfWQ+FTME7xp5QpIEgiGTFfRIbdWwQIuk7cPTo\nUTz++OM4evQonE6n1+OhxhhSFamCl8UA/HVpCDStqsXkcnjDGTwhRZAwUSo2aQ1wOc1pqjRh8ZgM\njwAAIABJREFU1b/rw4CRhmAc3i1Ht6AkuwSX9bmM38YccvdODSFHGuwmUSc6HIKJNPgTm/4EZKgO\nL+B6jbFIwyU9L5H0HJ1SB7VCLSnSEHaG94LDywSq+xuUQIi1RDtnOId8fT4Ucu9CXPevw3LRBEEQ\nyYRnxwYSuy4kfRceffRRZGZm4rnnnkN6um83knARrOAN5PCKTbRihBtpkNqlgZ2LZ9Ea7/CaW5Cu\nThe9jR0o0mCym+DknJDL/N8pqO+sx6fVn+LhiQ8LRIwvh9dfVETsHMQiDZGADZ4I1I/W5vDfviuQ\n4NUqtX5fS/6O22HtwMnmk5g/ZL6k58hkMmRrs/0KXs8ev6HSZGqCXCbn26VJjTS0mFowY8sMTCqe\nhD9O/SO/vdnU7HeanEapQaYmkxxegiCSFtaxgehCkuCtqanBBx98gOLi4mivJyVgt8QDCV7mSIaT\n4Q030sC6QAQSnIEiDc2mZqSp00RdvUCDJ9jxA4m1rd9uhZNzYuGwhYLt7hleRigZXpvTVbQWjUgD\nADg4B5Qy379yLNLgC7+C1+gaKxzKgAeNUoNvz30LB+fA4PzABWuMbG22aJcGzwxv2JEGYxNydbn8\n61NK0VqntRMztszAvjP7vHLnzabmgK3XEn34BEEQhFTMVju5vZAoeIcMGYJTp06R4JUIa8kUKYc3\n2pEGwCVQ/Qk9FjvwVbTGBgMwcefu6gUaPMGO7+/7YHPYsL5iPcp7lXt1EQg3w+ve1zUaDq+7g+xv\nkEGgAQ3+BG9DZ0PQ+V2GRqHBN+e+AQAM6TFE8vN8ObwsEhDJtmTuedtAfXjNdjOuf/N67D+7H0UZ\nRXwGmNFsasbFWRf7/Zr5+nyKNBAEkfR8uv97rzzv1LHdU8tJEryzZs3C7373O1x//fW46KKLoFIJ\nXaj586XdBu1O6FX6wA7vBUcynElrLDsrpdBIDPcxyP4Eb8CiNXMLLs66uCvSYJcWaWAiN1Dh2vqK\n9TjWeAw7btrh9Vi4GV73vq7RyPC6F1mJiX6G1AyvmIA8ZzjHT4YLFla0JoMMg/ID9+BlBIo0RKot\nWZOpSRBBUMgUkEHmM9Kw8euN2FmzE/9vzv/DZ6c/w+7TuwWPt5hbAo5g7qHvgbrOurDWTRAEEU/c\ne/ICXT15J5YXdUunV9IZb9iwAVqtFp988onXYzKZjASvCFIEbyQc3lZza8j5XaDL4TXZTIAfnReo\naK3V3OrK8F4QOe7um8lmggwyUfeSRRr8Fa6dN5zHqs9W4Zp+12DWwFlej+tVeshlcoHDa7FbQoo0\nmO3msPvueiJ1UILVYQ0rwxtIxPmCvUkpzi72K8g9ydHm4HTracG2YNuSWR1WvH/8fdww+AafcYwm\nY5PAkZXJZK6R1j4iDUyoLhy2EEcajnhNTJMaaWCuN0EQRDJCPXmFSBK8u3btivY6Ug5JDm8EMrwt\n5paQ4wzs+AACtiYLVLRmd9pdGV4RkcNEkJig4SMNfhzelbtXosPSgT9f92fRY8hkMn68MCMUh9fq\nsMJkM4XslEo5vj9CjTRwHIdzhnMhdWgAupzYYPK7gLjDa7KZgmpL9q9T/8JPt/0Uu27dhSklU0T3\naTI1YUThCME2lVzl8w1Em7kNepUeKoUKubpcdFo7+TcTNodrSEqgNzX5+nzX4AmCIIgkhXryCvFZ\npeR0OiX/I7yJlcPbYmoJy5Fkt+8D3XZmDqyXw+t2Cz5dlS5etOYnLhHI4T167ihe/uplLB2z1G++\nNEOdEZG2ZNHq0gAEFryhFq21mlthc9rCyvACoQte977cwbYlazO7Bpb898f/+tyn2dTs1TOXdb4Q\nPaalDVkal3vBnFyWdWdFdoEc3h76HjDZTZKGxxAEQSQirCdvYX4a5DKgV54eU0b2ifey4oZPh3fI\nkCGSK76pD683UgSvv+4FDK1SCwfngN1pFy14ajW3ojg79AC6INLgB97hVYk7vAD8Ory+crHseEz8\ne7Lhqw1QypVYOWml3/W5O7wOpwM2py20Lg1R6sPLju+PUNuShdOD1/24UkYKu5OtzXa54naXq+vk\nnPzH7sf1F2lgr6u9Z/aKPm62mwWFcAx/wzzaLG18CzP2vCZTEwrSC/h4Q6AYkPvwiUAFbgRBEIkK\n68m7/bOT2FXxI97893Hs/upMtyxe8yl4X3/99ZBaHBEupAheJgSYSBTDvVBJKRIybzG3YLh2eMjr\nDDbS4CvDC0CQ4fV0eH1lQ/vl9AMAVDVWYQ7mCB5zOB1469u3MGPADL99UwFX4R8rAmTf11AiDWa7\nOWqCV5LD66dozVcRWLiCN5xIA+B606VX6fl1BdOWzF3wivUp9hwrzAgUafB0eJnQZf9LyfACrvw4\nCV6CIJKd3QfPoL7Jdb3trsVrPs903LhxsVxHyqFT6gL28WQCSKqr55mfBS4UrUUg0hDI4TXYDFDI\nFF5rdf/c5+AJP71tc3Q5GJA7APvP7vd67IsfvkBdZx1uHnpzwPPI1GTyt8eZwApp0prNFPlIg8RR\nuFaHlRefYkTL4eUjDUE6vOx112puRe+M3l5viqREGthzmk3NONF0wqtLhOdYYYa/ojWxSAMTuiza\nEDDSkNYDAI0XJggi+aHiNRc+M7z33nsvampqJB/o5MmTWLZsWUQWlQpIcnjtFihkCp8jTgH/LpnD\n6UC7pT2sojUmRANleH0Vnrk7kmmqtK4+vHZpkQYAGFs0Fv87+z+v7W8efRNpqjT8ZMBPAp6He4Y3\naMErF/bhjdbgiUBdGkItWmP9YplIC5YsTRaKMooCikBP3B1ewPsugFKuhEKmkOTwAuKxBs+xwgyV\n3E+G19wVafDl8EopWgNI8BIEkfyw4jV3eubqYbE5YLba47Sq2ONT8N5444245557cNttt2Hz5s04\nffq0oDiF4zicOHECW7ZswcKFC7F06VLcdNNNMVl0MiA10uDP0QP8V7qHO2UNEPbh9YfBahCNJXg5\nvErx0cL+2l2NLRqL2o5anG0/y2+zOWx4u/JtzB40W9TZ9sQ9wxus4GXnYLKb4OAcUR08cd5wHhev\nuxgVtRVe+4VatMa6CXiKQqmsnrwa/1j4j6CfF0jwAi6XN1CGV6/SI0uThb0/igheHw6vvwxvu6Xd\ny+FlwjmYojUAKT98orKyEvPnz0d5eTnmzJmDw4cP+91/7969KCsrg8Hgv282QRCJg2fxWla6GmaL\nHQ8+9wWWPfsZPt3/fbyXGBN8Ct7x48fjvffew9y5c/Hxxx9jxowZuPTSS3H55Zfjsssuw9ChQzFv\n3jz84x//wI033oiPP/4YEyZMiOXaExopgjdQ31VAouANow8vn+EViTS4bzPajaLC012guRetefbh\n9eeaji0aCwCCWMOn1Z+iydSEm4ZKexOVoe7K8IYaaWCRiIgPnnDr0vDNuW/wY/uP+Lr+a6/9AhWt\n+XJMzxvPI1OTGfDNky8KMwoxrGBY0M+TIngDdRkx2U1IU6VhXJ9xwTm8Cj8ZXrdIQ6YmEwqZwsvh\nDXRXJEubBYVMwTu8rxx8BTM2z/D7nGTDYrFgyZIlmDdvHg4cOIBFixbh7rvv9ilm29ra8MgjjwiM\nD4IgkoOpY4vx199Oxp/uuQI6jRKtnVY4ua48b3dwev2mlZVKJWbNmoVZs2aho6MDlZWVaG5uhlwu\nR15eHsrKypCe7rulVndGaqTBX8Ea4F/wMrcqEpEGT4f3cP1hjHllDL795bcYmDdQ0G7KHZ8ObxCR\nhvJe5VDKldh/dj/mDp4LAHjz2zeRpcnCtNJpks4jU5OJDmsHOI4LWvCyyV3MIY6Ww2tz2lDfWQ/A\nJco8CVS0xtYmJniZIxlL2OuO5WJFHV6FJmCGV6/SY0KfCXhsz2Not7Tzo6IB4Ez7GQAiGV65eIbX\n5rDBaDPykQaZTIZcXa5A8GZpsvzGiADXePA8fR7vnr/29WtwcqnVgnHfvn2Qy+VYsGABANfEzNdf\nfx179uzBjBne4n716tWYMWMGXn311VgvlSCICKBVK6FWKXCuWahNukue16fD60lGRgbGjRuH6dOn\nY9q0aRg9ejSJXT+wqnV/fyTDjTQwoRGNPrzHGo/B7rTj6LmjAFyRBs+WZIB3hle0aC1ApEGr1GJ4\nwXDsr3U5vG3mNmyv2o55g+dJdi0zNBlwck4YbUb+XAK9mWCwyV0sAxytDK/VYcUPbT8AgGBIBkOq\n4y8WaQg1vxsOUh1eKZGG8ReNBwfOq3jxw+8+xOUXXe71JsRXH172fWUOL+CKLzSbLxStmVskZ5V7\n6Hug0dSIDksH9p/dj6v6XiXpeclCTU0NSktLBdtKSkpQXV3tte/777+P9vZ23Hxz4AJSgiASF7E8\nb3cZRiGpH8WiRYt8TrhSqVTo0aMHpk+fjiuvvDLiC0xW2B99k83kM4NqcYTn8EYi0uCrDy9zxJjD\nJtXhVcqVkEEmbEtmC9zbdmzRWGw6sglOzomXDr6EDmsHfjX2V5LPg7mC7Zb2oB1ewOUYRsvhdS+K\nY4KXxSfcsTltUMtDELzG8yjOin0/RY1SA51SFzDDG6hoTa/SY1zROMggw94f92Jqv6kAgFPNp3C4\n/jDWXrvW63lsaponzDlnDi8AL4dX6u9Lvj4fjcZGfPHDF7A77biqJLUEr9FohE4n/L3UarUwm4U/\nr9raWvzlL3/Bli1bYLP5L7wkCCKxYXnerTu/Q0OTAT1zu88wCkkO75gxY/DVV1+hR48euOaaazB1\n6lQUFhbi4MGDKCgogE6nw29+8xu888470V5v0sD+6PuLNYSb4Y1EpEElV0EGmVekgQkEVkhmsIkX\nrXlmeGUymVehktFmDOiaji0aiw5rB440HMGf9/0ZU/tNxcjCkZLPg41n7rB28OI9GOGqVqh5wRu1\nwRMOG75vcxUHhBVpcAhfC43GxrhEGgDheGH2/XO/EyDV4c3SZmFIjyH4/IfP+cfeqXJdT+YNnuf1\nPF+RBvZGwt3hzdPn8VngZlOzZIeXjRfeWb0TGoUGEy5KrRoFnU7nJW7NZjP0+q7fc6fTiQcffBDL\nly9HQUFok/wIgkgsWJ73pmtdbSDf/PfxblG8Jsnh3bt3Lx5++GEsXLhQsH306NF47733sHnzZlx2\n2WVYu3YtbrjhhqgsNNmQIngt9vhHGmQyGXQqnW+Ht6PL4RVzqj0dXkCY2+Q4LmCkAegqXPv1J79G\nXWcd3pj7RlDn4e7w7juzD3KZnB9qIQWVosvhjUWkQUzwBipaA7wdXo7j4hZpAC4IXotL8Fadr4JK\nrhJM/tMoAju87A3bvMHz8Pjnj2P/2f0YWzQWb1e+jTG9x4hOEvRVtMa+r+454FxdLr5p+AaA63dG\n6iCJHvoeaDQ2YtfpXZhw0YSIvy7iTb9+/bBp0ybBtpqaGsycOZP/vL6+Hl9//TWqqqqwevVqfpT8\npEmT8OKLL2L06NExXTNBEJGjuw2jkOTwVlZWinZgGD16NL75xvWHZOjQoairq4vs6pIYSYI3ApEG\nlVwVUEwGQqfUeR2ftYPiHV4JbcmYs6dWqHlXz+KwwMk5A65xUN4gZKgz8Pn3n2Nk4UhcXXJ1UOfg\nLni3Vm7FlL5TghKBUY00XHBtLQ5LwAyvv7ZkbG3uP6t2SztsTltcHV72xuvIuSMY3GOw4DWhUUor\nWgOA+yfcj55pPfHbf/0W37d+jwO1B3DDYPE30L768PIZXvdIg9Yj0iDxDSKLNByuP5xycQbA1YnH\narVi48aNsNlsePvtt9HY2IiJEyfy+/Tu3RtHjhxBRUUFKioq8P777wMA9uzZQ2KXIJIYf8MoUhVJ\ngresrAwbN27k390DLmdp8+bN6N+/PwDgm2++Qa9evcJe0BNPPIGnnnrK5+NWqxWPPPIIxo4diwkT\nJmD9+vVhf81oEKtIQ7Y2O+wR0DqVzmekwT3DG7Bo7YIDrFFqeDHCBIi74yaGQq7AmKIxAIAHL38w\n6HPK0LgiDf/54T840XQCPx3y06CeH4tIQ11HHf9z9JnhDdLhDXfoRLjk6HL4SMORhiO4tOBSweOB\n2pK5C94MTQaemPIEvvzhSyzesRgAcMMQccHrqw+vWKQhV5eLDmsHrA5r0JEGDq4WXMG+AUsG1Go1\nXnnlFXz00UcYO3YsNm3ahPXr10Ov12PlypVYuXJlvJdIEESU6I7Fa5J86xUrVuDOO+/EZ599hsGD\nB4PjOBw7dgwmkwkvvvgiDh48iPvuuw+rVq0KeSEtLS146qmnsH37dtx2220+91u3bh1qa2uxc+dO\nNDU14bbbbkNxcbFoG514IjXS4O5EiRHI4Q2nYM39a/gTvBzH+Sxak8vkUMgUUClUUMpdLyeNoivD\ny3rjBhK8APDTIT+Fw+nw6er5gx3/74f/DrlMLpr79Id7pCFabcm+a/4OgOu14RlpcHJO2J12SYLX\n/WfF2maxyWCxJlubjeONx9FsasaZ9jO4tKdQ8Lq/FsTwfF3dNuI2PLf/Oew+vRvDC4ajf25/0ecF\nijR4Fq0BwI9tP8LBOaR3abjwJiJdnY7RvVPTzSwrK8Obb77ptX3NmjWi+/fp0wfHjx+P9rIIgogy\nnsVrBXlpmDupFHWNBhTmp6VkrEHSGQ0dOhT//ve/8dFHH+HEiRNQKBSYMmUKfvKTn0Cv1+PMmTPY\ntm0bysrKQl7IggULMHLkSEyb5r/v6o4dO/Dss88iIyMDGRkZuOWWW7B9+/bkFLxBRBrEREOLuSWs\n/C5Dp/Sd4bU4LGg0NvrN4aoUKoH7634bW6rDCwBLRi/BktFLQjoHVrR2uvU0pvabGrTj6e4YRjqr\nyWIKJ5tPAgAu6XEJH21gMPEmpWiNFSsCbg5vvCINGlfRGsvIhuPwAi6n/9lrn8W1m67F/CHzfT4v\n2KI1oOv7H0ykAQCuLL4y4M+FIAgi2Zg6thgTy4tQ12jAse+bsf2zU2h49wgK8tLws6sHYOrY2Hf/\niSaSJXx6ejpGjx6NnJwcOBwOlJSU8NW8ffoEbmlht9thNHqLP7lcjvT0dLz22msoKCjAQw895PMY\nbW1taGpq4mMUgKtv5ObNm6WeRsyIVKSBFbX5KlqT6lb5Q6fyzvA2m5r5DCNzJsUiDYBLLLKCNUDo\n6gUjeMPB/fjBxhkAYbeJaDm8p1pOAQCG9RzG9zdmMPEWdKTBEN9IA+vS8HWDa3Kc58S2YDK8jGtK\nr8FnP/+Mj7iI4SvD22Zpg0ahERSDst8RJniD6cMLpGacgSAIAnA5vYX5aXjy9QN8fjdVC9gknUlb\nWxseeOABfP7558jMzITD4YDBYMDo0aPxwgsvICMjI+Ax9u/fj8WLF3ttLyoqwq5duyS1vDGZXC6k\ne+9Isb6RiQDfh9fuPbKXIaVLA2sbJiZ4m0xNGJA3ILyF4oLD67ZOjuPQbGrGFRdfgd2nd+NE0wkA\n8O3wylWCDg6hOrzhoFVqoZC5pmfNLZsb9PPdHbxIZ3hZ1KPR2Ai9So+SnBKY7CbYHDbB2GEAQRet\nxd3h1WbDwTmw98xe5OnyUJheKHhcq/DdlszmsMHmtIm+rib1neT36/rsw2tu84oJeQpeqTGg4b2G\n46mpT2Fxufd1iyAIIlXwV8CWStPXJAnexx9/HI2Njfjoo4/Qr5+r1dPJkyfx0EMP4cknn8Qf/vCH\ngMeYMGFC2NkvrfZCntVs5qe8efaNTBQiFWmQyWQ+bws3GZuQp8sTeVZwaJVaPsIAuHrZ2p12XFpw\nqUDw+hqgIcXhZZGDaCGTyZCtzcbIwpEhuZ3uzmqkHV6ZTMbfgi/OKuZvt7dZ2vjb5ky8heLw6pQ6\nnz+baMPE4+fff45LCy71Kjb0N3iCvckKpcuISuEj0mBpE8QZADfB2xKcwyuXyfHA5Q8EvTaCIIhk\nghWwuXdoyM3SIjczsn8L442kLg27d+/GY489xotdAOjfvz9WrlyJnTt3Rm1xnmRnZyMvLw81NTX8\nNrHxmIkAcwnDjTQA4jlIu9OONktbRASvZ6SBid9LelwCuUwuKLYSw1+Gl43rjbbDCwCvzn4V66at\nC+m57s5qNPqtsp/zxVkX8w6ke6cG5vCG0qUhXnEGoGvoSW1HrVd+FxD2ZPZEbDKbVFRy8aK1dku7\n12uN/Y4EG2kgCILoDrACNtahQSGXoanVjPuf+yKlhlFIErzMWfVEJpPB4XBEdEGBmD17Np577jm0\ntrbi9OnT2LRpE+bMmRPTNUhB8uCJAA4vIC54We/TiGR4PSINTPAWpBegIK0gYKTB0+F178Mbq0gD\nAFxfdj0u6XlJSM9lQlMpV/IRhEjiLnjdewYz+EiDlElrbq+FeE5ZA4RT/sQEr7+itbAEr8J3htcz\n0pCpyYRCpkB1SzWA8Aa1EARBpCJTxxbj6XuuQI9sHRxOV0NGluU1W+3xXl5EkCR4r7rqKqxZs0bg\nrFZXV+Pxxx/HlClTorY4xogRI1BRUQEAuPfee9G3b19Mnz4dCxYswM9+9jNMnz496msIFnZbPGCk\nIUCGlx1LrKgM6KpADwetUivo0sCOnavLRZ/MPviuyX/RWnmvcpT3Kuc/1yiEfXjlMnnYwzGiDROa\nkY4zeB7fM9LACKZozWQzgeNc/WETxeEFxAWvRqmBg3PA7vS+YIYjeNUKNRycg/8+MNrM3pEGmUyG\nHF0Of0cl0V+LBEEQ8aC53YymNmHdUX2jARWVDSkheiVZWffffz+WLl2K6dOn89lZg8GASZMmYcWK\nFRFd0B//+EevbYcOHeI/1mq1WLNmjc8+kYmCTCaDXqWPWqSBTUKLhsPbZOw6dlFmEQ7UHgDgW5hs\n++k2weeeRWuZmsywh2NEGxZpiHTBGkNqpCFQ0VqfzD7gwOF062mU5JTgvOE8yvJDbwcYLkzwymVy\nDOkxxOtxvq2e3QKlR7VvuJEGwHtYh5jDC7hey43GRuTqchP+tUgQBBEPxLK8crkMT2+qSIlWZZIE\nb2ZmJjZu3Ihjx46huroaWq0WJSUlKCkpifb6khp/gtfutMPJOUOONDBRGs0Mb64uF30yulrOSS2M\n8ixai3bBWiRgoilaDq9A8Io5vBKL1kYVjgIAHKw76BK8xvMJEWnon9tfVLiy17fFYUEahK8f9rsR\nypsM5pjbHB6CV8ThBbp+TyjOQBAEIY77MIr6RgPkchkcTtddtFRoVeZz1T/++KPXtrS0NAwbNsxr\nn4suuigKS0t+/Ale5oAmQqSBDZ7gOA4ymUwgeIsyi/j9pDpx7oVKHdaOmOR3w4UJqGgUrAFdjuTF\nWRfzeWexDG8gwXtpwaVQyVU4WHsQMwbMgNFmTAjBKxZnAIQOryeRcHitDisvpB1OBzqsHaKCl90J\noYI1giAI37BhFBWVDXh6U4XgsWRvVeZT8F5zzTWCW39MDLnDtlVVVUVvhUmMP8ErVeAA0Y80FKQX\ngAOH+s56FGYUotnUjHR1OtQKNfpkdjm8kgWvUujwJoXglUc3w6tWqCGDjI8kAD4iDQGK1jRKDYb2\nHIqDdQfjPnQCcBX5zRo4y+c4aH+DU8LN8AIQtCZjHUF8RRrc/ycIgiDE0aqVGD2kwCveUJCXxndy\nSEZ8Ct5YthtLVfw6vBcEodRIQ6OxUbCt2dQMhUwh6mYFS2mOq61bdUs1CjMK0WRq4oWBu+D1VbTm\nibvD225pT4rbyExARTPD2zujt6A4LpSiNcAVa3j32LtxHzrBeP/m930+5h5p8CTcLg0ABK3JxMYK\nM9jrWerQCYIgiO6Me7yhocnAZ3iTNc4A+BG8RUVFvh4iJBKpSINY8/4mYxNydDkRKcDpl+Pqr3yq\n5RQuv/hyNJuaeYFQlBFCpOGCw8txHNot7SjOSvyQe7QdXq1Si4uzLuY/z9JkhVS0BgCjeo/Cq4de\nRUWt63ZTPB3eQLDvZ6QdXveiNYa/Fni8w6slh5cgCEIKLN5Q12hAYX5aUotdQGLRGhEaepVeIGrc\nCTbS4OmQNZkiM2UNAPpm94UMMr5PqUDwXsjwahQaKOQKScdjrp7daU+aSAPv8EYpw/v7q34v6O+b\nqclEu7Urwyu1aA3oKlz75OQnAOLv8PqDvaETy/CyVnjhOLzuvXiZYy4WaeCL1sjhJQiCkIxWrURJ\n7yyYrXbU1LYltfBNzlUnCXqVHnUddaKPBRtpECtai0TBGuASJX0y+wgE79CeQwG4ziHYSAITbRaH\nJWm6NPBFa1GKNEzqO0nweZbWh8MbIMMLAMMKhkEpV2JnjSt21B0dXj7DG2SkgTK8BEEQwfHp/u+9\nog3J2J5M0uAJIjQi1qVBIV60Fsk/3qW5pTjVcgqA0OEFXDleqS3JgK5zMtlM6LR2JoXDG+1IgydZ\nmixBhjdYx39oz6HotHZCKVdGJMcdLQJleJVypSSR74lYpMGfw0uClyAIInjMVju27vwOdY0GOLnk\nnr5GgjeK6JXR69LQbGqOWKQBAPpl90N1SzU4jvMS030y+wTlwjGRwzpJJIPgjXbRmieZmkyBwxtM\n0RrQFWvI1+cn9CCFQG3JQp16FmzRWv/c/lDKlRiYNzCkr0cQBNEdqWs0oKHJINjG2pMlGxRpiCKR\n7NIgVrQWaYe3vrMeDYYG2J12wbEfuPwBNHQ2SD4Wc3hZ26xkELzRHi3sSZY2S7QPr5SiNcAleDcc\n2pDQ+V0gcFuykAWvPLgMb2luKdoeaqOxwgRBEEEgNn0tWduTkeCNInqVXjCy151gB09YHVY4OSfk\nMjksdgsMNkNkHd4LnRoO1h4EILz1O7nv5KCOxUQ8a5uVDII32kVrnnhGGoIpWgNcnRqAxM7vAl2t\n7M4Zznk9ZrSHLnjF+vC2mduglCt9uvQkdgmCIIIjldqTJd+Kkwi9Sg+rwwq70y6o0AeCjzQALpGs\nU+kiOmWNwQTvgdoDrmOHIaaZiGe9gzM0SVC0FocMb4elg38TE0zRGuCabKaUKxPe4e2b3Rdl+WXY\n/M1mLB27VPBYpCMNrCNIIkc8CIIgko1UaU9GGd4owtxC1n7JnWAjDUDXbeFITlljsOFkZUz0AAAg\nAElEQVQTrLdrOMfmHd4kjDTEMsPLgUOHxTUdLJg3QIDrNfG7K36HWy69JWprjAQymQx3jLgDe8/s\nxbfnvhU8FolIg2fRWiIX8BEEQSQrrD1ZsopdgARvVGF/zMVyvMFGGoAuwcs7vBGMNOTqcpGpyeQd\n3rAEr4fDmwyClwnNWGZ4ga7cabBFawCwavIqzBw4M/KLizCLhi+CSq7ChkMbBNsj4fB6ZnjF8rsE\nQRAEQYI3ivgTvKFEGniH1xh5h1cmk6E0p5TPWoZzbHZOyZThZY5hLDO8QNd0MPZ6UMikDfdIJnqm\n9cScsjl44+s3BN0awhG8vvrwksNLEARBiEGCN4r4dXgjEGmIZIYX6MrxAhGKNCST4I1xpIF3eC+0\n0rI5bFAr1CmbP71jxB1oMjVhx/Ed/LaoRBrI4SUIgiBEIMEbRSIVafBs7RSNSAPQleNNU6VJWpcv\nvIrWkmDSWqwjDexNAIs0WB1WyS3JkpGp/abi4qyL8epXr/LbotGHlxxegiAIQgwSvFGE/TE32Lwb\nNLNb2CE5vMYmqBXqiLdZYg5vuFEJ96I1rVIb0iStWBOvSANzeK0Oa1D53WRDIVfgxktuxK6aXXA4\nHQAuCF5lZPvwkuAlCIIgxCDBG0WYi+c+YIDBIg1BtSW78Bw2ZS3St79Lc10Ob9iCV9kVaUiGOAMQ\nv6I19tqwOW0pLXgBoCS7BA7OwefEjTZjyG8wPPvwchyHdks7RRoIgiAIUUjwRhFPF88di90CGWRe\n/XnFEMvwRjq/C0Te4TXbzUkjePP1+QCAgrSCmHw90UhDEjjh4VCYUQgAqOusA8dxEY00GG1GODln\nUsRnCIIgiNhDgjeKeLp47lgdVmiUGkkurZjgjWSHBsbFWRdDIVOELabd87/JInhH9R6Fyl9WYkTh\niJh8vTRVGhQyRVfRWjdweHtn9AYA1HbU8pMDI1W0xnLyaerkG3dJEASRTJitdtTUtsFstcd7KUGR\nvB2EkwBPF88di8MS1JABQFi0NiB3QIRW2YVSrsQ1pddgXNG4sI7jnktOJsdtcI/BMftaMpkMmZrM\nblO0BgCF6Rcc3o46XqBGqg9vuMcjCIIgAvPp/u+9xgxPHVsc72VJIuEE7xNPPAGVSoUHH3xQ9PHm\n5maMHz8een3XH7ZZs2ZhzZo1sVqiZLRKLdQKtc9Ig5SCNXYcQFi0Fq4o9cU/Fv4j7GO4C/lkcXjj\nQZY2qyvD60h9h7dXei8ALoc3bMEr9440hHM8giAIwj9mqx1bd36HukZXIX5dowFbd36HieVFSTGB\nLWFW2NLSgqeeegrbt2/Hbbfd5nO/qqoqDBgwAB9++GEMVxc6WZosUYeXRRqk4C54OY7ji9YSFRK8\n0vB0eFNd8KoUKvTQ90BdZ/gOr0KugFwm5yMNJrtrfHes+igTBEF0N+oaDWhoEnadamgyoK7RgJLe\niV8wnDAZ3gULFkChUGDatGl+96usrERZWVmMVhU+7i6eO6FGGow2IywOS1SK1iKFTCbjz40Er2+y\nNFmCtmSpXrQGuHK8kXB4AZfLSw4vQRBEbCjMT0NBnrBOoiAvDYX5yVE7ETPBa7fb0d7e7vWvs7MT\nAPDaa6/h97//vSCqIEZVVRV++OEHXHfddZg4cSIeeeQRtLd7C8pEwd3Fc8fiCC3SwKasRaNoLZKw\ncyPB65ssbZf73x2K1gBXp4ZIOLyAyzFmGV6TzRT28QiCIAjfaNVK/OzqASjMT4Nc5hLAP7t6QFLE\nGYAYRhr279+PxYsXe20vKirCrl27UFAgrR1Ueno6xo0bhzvuuAM2mw0PPvggVq1ahXXr1kV6yRHB\n3cVzJ5hIAxOP5wznojZlLdJolBp0WDuSqmgt1mRpslBpqQTgej10h9vxvdN740jDET6CEI5AVSvU\nXl0aYjU4hCAIojsydWwxJpYXoa7RgML8tKQRu0AMBe+ECRNw/PjxsI/jWZy2fPlyLFy4EE6nE3J5\nwiQ0eLK0WTjZfNJru8UuPdIgk8kwue9krNu3DscajwEghzcVyNRkotXcCsBVfNUdpoQVZhSiobMB\nHZYOABRpIAiCSDa0aiWf2TVb7UkjfhN7dR44nU6sW7cON954I/r06QMAsFgsUKlUCSl2AZeL5yvD\nKzXSAAAfL/gY9/zjHmw4tAEAEjrDC3T14iXB65tLelyCZlMzDtYe7BZFa4CrNZmDc+D7tu8BhB9p\noKI1giCI+JBsLcoSUyX6QC6X4/Dhw1i7di2MRiPOnz+PtWvXYu7cufFemk8iEWkAXLdqX539Kv4+\n+++YPWg2SnNKI7nMiEMOb2AWXroQepUeLxx4oVsVrQHAqeZTAMJ3eKkPL0EQROxxb1Hm5LpalCXy\nMIqkELwjRoxARUUFAOCZZ56BxWLB5MmTMXPmTAwcOBD3339/nFfom0xNJtot7XByTsH2YCIN7iwe\nsRg7btqR8FlF6tIQmGxtNm4Zdgu2HN2CRmNj93B4L4wXPtUSvuAVy/CS4CUIgog+/lqUJSoJF2n4\n4x//6LXt0KFD/McFBQV4/vnnY7mksMjSZoEDh05rp0D8BRtpSDaYe52hoaI1f/xyzC/x8lcvw2w3\np/ykNaDL4WW59rAjDRcyvKxLQ6K/ESQIgkgFWIsyd4Gb6C3KksLhTWZYIZJnjjfYSEOyQZEGaQzv\nNRyXX3Q5AHQLh5dNW6tprQEQXuZWJVcJHF6NQgO5jC5pBEEQ0SYZW5Ql7spShCytS/C2mdvQJ7MP\nvz3USEOyQEVr0vnlmF/iPz/+J6VfDwy1Qo18fT4ajY3QKDRQyBUhH8u9D6/RZqQ4A0EQRAxxb1GW\nm6lFc7sZZqs9YUVvYq4qhWCCz3P4RMpHGsjhlcz8IfOxcvdKlGSXxHspMaEwvRCNxsawBapaoe6K\nNNhNFGcgCIKIMVq1EqfOtOLJJOjWQPf/ogyLNHh2arA6rKkteJUayCBDmipx8zyJglqhxvFfHcdv\nJ/w23kuJCSzHG67g9Yw0kMMbPJWVlZg/fz7Ky8sxZ84cHD58WHS/rVu34tprr8XIkSNxww038EXE\nBEF0b5KpWwMJ3ijDIg2eGd6UjzQoNMjQZEAmk8V7KUlBOLf2kw3WqSFcR9a9aI0Eb/BYLBYsWbIE\n8+bNw4EDB7Bo0SLcfffdMBiEVdb79u3D2rVr8Ze//AUVFRW45ZZbsGTJErS0tMRp5QRBJArJ1K2B\nBG+U4R1esUhDChet9dD34J08gnCnMN0leCPh8LIMr8luoqETQbJv3z7I5XIsWLAAKpUK8+fPR35+\nPvbs2SPYr76+HrfffjsGDx4MuVyOuXPnQqFQ4ORJ7wmSBEF0L1i3BncStVsDZXijjHvRGsPJOWF3\n2lM60vDYlMdw34T74r0MIgGJVKTBsw8vObzBUVNTg9JS4QCbkpISVFdXC7Zdf/31gs8PHjwIg8Hg\n9VyCILofrFuD58S1RCxcS7wVpRhpqjTIZXKBw8tcqVSONGRrs5GtzY73MogEJGIOr0cf3qz0rLDX\n1p0wGo3Q6YSuuFarhdls9vmckydPYtmyZVi2bBlyc3OjvUSCIJIA924NhflpCSl2AYo0RB2ZTMZP\nW2NY7BYASOlIA0H4IlpFa9SlITh0Op2XuDWbzdDrxX8uX375JW6++WYsXLgQv/jFL2KxRIIgkgSt\nWomS3lnQqpUwW+2oqW1LuMK1xJThKUaWJkvU4U3lSANB+IIVrUXC4aU+vKHTr18/bNq0SbCtpqYG\nM2fO9Nr3nXfewe9//3usWbNG9HGCIAgA+HT/917xhkRpUUYObwzI0mYJMrwWh8vhTeVIA0H4IlKR\nBrVc2IdXryTBGwzjx4+H1WrFxo0bYbPZ8Pbbb6OxsRETJ04U7Ld371489thjePnll0nsEgThk0Rv\nUUaCNwZkajIFDi9FGojujEapQXFWMS98Q0WloEhDOKjVarzyyiv46KOPMHbsWGzatAnr16+HXq/H\nypUrsXLlSgDAK6+8ApvNhjvvvBMjRozg/33++edxPgOCIBKJRG9RRpGGGJClyUJdZx3/OUUaiO7O\n3tv3hj2FTyV3Fa1xHEeRhhApKyvDm2++6bV9zZo1/Md///vfY7kkgiCSFNaizF3gJlKLMnJ4YwBF\nGghCSGFGIdLU4V0EWYbX5rTByTlJ8BIEQcQR1qKsMD8NcplLACdSi7LEWEWK41m0RpEGgggf1ofX\naDMCAA2eIAiCiDOJ3KIscVaSwmRpXA4vx3GQyWQUaSCICKCSq+DknOi0dgIIvwiOIAiCCB/WoizR\noEhDDMjUZMLmtPFRBoo0EET4qBQqAOB7XJPgJQiCSBwSrR8vObwxwH28sDZdS5EGgogAKrlL8LJ8\nPHVpIAiCSAwSsR8vObwxIEtzQfBeyPFSpIEgwofdIWG/V+TwEgRBxJ9E7cdLgjcGuDu8QFekgRxe\ngggdFmngHV4qWiMIgog7idqPlwRvDGD9RlnWkEUaKMNLEKHDRxrI4SUIgkgYWD9edxKhHy8J3hhA\nkQaCiDyeDi8JXoIgiPgj1o937qRS1DUa4hprSJiitRdeeAFbt25FZ2cnBg8ejBUrVmDgwIFe+1mt\nVqxevRqffvoplEolFi1ahLvvvjsOK5YORRoIIvJ4ZnipaI0gCCIxcO/He+z7Zmz/7BQa3j0S1wK2\nhHB43333XezYsQMbN27Evn37MH78eNx1111wOp1e+65btw61tbXYuXMntmzZgm3btuHjjz+Ow6ql\n4+nwUqSBIMKHRRqoLRlBEETioVUrUZifhu2fnUqIAraEELwtLS1YsmQJLrroIiiVStx6662ora1F\nfX291747duzAXXfdhYyMDPTt2xe33HILtm/fHodVS8czw0uRBoIIHz7SQBlegiCIhCSRCthiFmmw\n2+0wGo1e2+VyOW6//XbBtl27diE7Oxu9evUSbG9ra0NTUxP69+/PbyspKcHmzZujs+gIoZArkKZK\n84o0kMNLEKHDHN5WcysA6tJAEASRaLACNneBG68CtpgJ3v3792Px4sVe24uKirBr1y7BfqtWrcKa\nNWsglwsNaJPJBADQ6br+sGm1WpjN5iitOnJkabMEkQaVXAWZTBbnVRFE8sLeMLZb2iGXyekNJEEQ\nRILBCtjYEIqeuXpMGdknLmuJmeCdMGECjh8/7nef9957D4899hhWrFiBWbNmeT2u1WoBAGazGenp\n6fzHen3i38rM0mQJujRQwRpBhId7lwa9Sk9vIAmCIBIQVsC2/bOT2FXxI97893Hs/upMzIvXEiLD\nCwDPP/88nnzySbzwwguYN2+e6D7Z2dnIy8tDTU0Nv62mpgalpaWxWmbIZGmzuvrwOiyU3yWIMHHv\nw0txBoIgiMRm98EzqG8yxq14LSEE7zvvvIPXX38dW7Zswfjx4/3uO3v2bDz33HNobW3F6dOnsWnT\nJsyZMydGKw2dTE1mV4bXbqHbrwQRJp4OL0EQBJGYJELxWkII3pdffhkGgwHz58/HiBEj+H+nTp0C\nAIwYMQIVFRUAgHvvvRd9+/bF9OnTsWDBAvzsZz/D9OnT47l8SWRpsvjiGquTIg0EES7ufXhJ8BIE\nQSQuiTB9LSEGT/zzn//0+/ihQ4f4j7VaLdasWYM1a9ZEe1kRpTirGO8ffx92px0WO0UaCCJcWKTB\n7rTT0AmCIIgExrN4jQ2g0KpjJ0MTQvB2B4b2HAqLw4JTzadgcVCkgSDChUUaAOrBSxAEkei4T19j\nzm5NbRsK89NiInxJ8MaIYQXDAADfnPuGujQQRARgDi9AgpcgCCIZ0KqVKOmdhU/3f+/l9ka7Y0NC\nZHi7A4PzB0MGGY6eO0qRBoKIAO53SahLA0EQRHJgttqxded3MR83TII3RuhUOvTP7e8SvBRpIIiw\noUgDQRBE8hGvjg0keGPIsIJh+ObcNy6HlyINBBEWFGkgCIJIPuLVsYEEbwwZ2mMoTjafRJuljSIN\nBBEm7g4vRRoIgiCSA9axoTA/DXKZSwDHomMDFa3FkKE9h8LJOXGy+SSGFwyP93IIIqlxjwWRw0sQ\nBJE8eHZsoC4NKQbr1ODknBRpIIgwoUgDQRBE8sI6NsQKijTEkP65/XlXiiINBBEeMpkMCpkCAGjw\nBEEQRBJjttpRU9sW1U4N5PDGEKVcicH5g/F1w9fUpYEgIoBaoYbJbiKHlyAIIkmJVU9ecnhjDIs1\nkMNLEOHDCtdI8BIEQSQfsezJS4I3xgztMRQAKMNLEBGA5XipSwNBEETyEcuevCR4Y8zQni7BS5EG\ngggfcngJgiCSl1j25CXBG2Mo0kAQkYO9caSiNYIgiOQjlj15qWgtxlyUeRFWT1qNeYPnxXspBJH0\nsEgDObwEQRDJSax68pLgjTEymQyrJq+K9zIIIiWgSANBEETyE4uevBRpIAgiaaGiNYIgCEIKJHgJ\ngkhaWIaXHF6CIAjCHyR4CYJIWijSQBAEQUiBBC9BEEkLH2mgLg0EQRBJTzRHDFPRGkEQSQtzeCnD\nSxAEkdxEe8QwObwEQSQtaoUaGoUGCrki3kshCIIgQiQWI4YTRvC+8MILmDx5MkaPHo1FixbhxIkT\novs1Nzdj0KBBGDFiBP9v5cqVMV4tQRCJgEquojgDQRBEkhOLEcMJIXjfffdd7NixAxs3bsS+ffsw\nfvx43HXXXXA6nV77VlVVYcCAATh06BD/b82aNXFYNUEQ8UalUFHBWhhUVlZi/vz5KC8vx5w5c3D4\n8GHR/T788ENcffXVKC8vx1133YXGxsYYr5QgiFQmFiOGE0LwtrS0YMmSJbjooougVCpx6623ora2\nFvX19V77VlZWoqysLA6rJAgi0cjUZCJHmxPvZSQlFosFS5Yswbx583DgwAEsWrQId999NwwGoaNy\n7NgxrFq1CmvXrsW+ffuQn5+Phx9+OE6rJggiFYnFiOGYFa3Z7XYYjUav7XK5HLfffrtg265du5Cd\nnY1evXp57V9VVYUzZ87guuuuQ2dnJ6688ko89NBDyMzMjNraCYJITB6f8jjazG3xXkZSsm/fPsjl\ncixYsAAAMH/+fLz++uvYs2cPZsyYwe/3wQcf4Oqrr8bw4cMBAPfddx/Gjx+PxsZG5Ofnx2XtBEGk\nHtEeMRwzh3f//v0YM2aM17/Zs2d77bdq1Sr87ne/g1zuvbz09HSMGzcOb731Ft577z00NDRg1Soa\n1UsQ3ZE+mX1wSc9L4r2MpKSmpgalpaWCbSUlJaiurhZsq66uRv/+/fnPc3JykJWVhZqampiskyCI\n7gMbMRxpsQvE0OGdMGECjh8/7nef9957D4899hhWrFiBWbNmie7jmdddvnw5Fi5cCKfTKSqQCYIg\nCG+MRiN0OmHBn1arhdlsFmwzmUzQarWCbTqdDiaTKeprJAiCiBQJoxCff/55PPnkk3jhhRcwb948\n0X2cTieeffZZnDlzht9msVigUqlI7BIEQQSBTqfzErdmsxl6vbAI0JcI9tyPIAgikUkIlfjOO+/g\n9ddfx5YtWzB+/Hif+8nlchw+fBhr166F0WjE+fPnsXbtWsydOzeGqyUIgkh++vXr5xVLqKmpEcQX\nAKC0tFSwX3NzM9ra2rziEARBEIlMQgjel19+GQaDAfPnzxf01z116hQAYMSIEaioqAAAPPPMM7BY\nLJg8eTJmzpyJgQMH4v7774/n8gmCIJKO8ePHw2q1YuPGjbDZbHj77bfR2NiIiRMnCvabOXMm/vWv\nf6GiogIWiwVr167FlVdeiZwc6o5BEETykBCjhf/5z3/6ffzQoUP8xwUFBXj++eejvSSCIIiURq1W\n45VXXsHq1auxdu1aFBcXY/369dDr9fwwnzVr1mDw4MF4/PHH8eijj+L8+fMYPXo0nnzyyTivniAI\nIjgSQvASBEEQsaesrAxvvvmm13bP4uAZM2YIWpURBEEkGwkRaSAIgiAIgiCIaEGClyAIgiAIgkhp\nSPASBEEQBEEQKQ0JXoIgCIIgCCKl6VZFaw6HAwBQX18f55UQBJHqsOsMu+50R+iaSxBELPF33e1W\ngvf8+fMAgIULF8Z5JQRBdBfOnz+P4uLieC8jLtA1lyCIeCB23ZVxHMfFaT0xx2w24+jRo+jRowcU\nCkW8l0MQRArjcDhw/vx5DB06FFqtNt7LiQt0zSUIIpb4u+52K8FLEARBEARBdD+oaI0gCIIgCIJI\naUjwEgRBEARBECkNCV6CIAiCIAgipSHBSxAEQRAEQaQ0JHgJgiAIgiCIlIYEL0EQBEEQBJHSdHvB\nW1lZifnz56O8vBxz5szB4cOHRff78MMPcfXVV6O8vBx33XUXGhsbY7xSaUg9n61bt+Laa6/FyJEj\nccMNN6CioiLGK5WG1PNh7N27F2VlZTAYDDFaYfBIPaeKigrMnTsXI0aMwKxZs7B3794Yr1QaUs9n\n27ZtuPrqqzFq1CjcdNNNOHr0aIxXGhxHjhzBxIkTfT6eLNeERCaVrr+pdO1NpesuXW8T/3obs2st\n140xm83cFVdcwW3evJmzWq3ctm3buMsuu4zr7OwU7FdVVcWNHDmSO3z4MGcymbhHHnmEu+OOO+K0\nat9IPZ+9e/dy48aN4yorKzmHw8G9++673KhRo7jm5uY4rVwcqefDaG1t5SZPnswNHDjQ5z7xRuo5\n1dfXc6NHj+Y++eQTzul0ch988AE3atQozmQyxWnl4gTzOzR27Fiuurqaczgc3EsvvcRdddVVcVq1\nf5xOJ7dt2zZu1KhR3NixY0X3SZZrQiKTStffVLr2ptJ1l663iX29jfW1tls7vPv27YNcLseCBQug\nUqkwf/585OfnY8+ePYL9PvjgA1x99dUYPnw4tFot7rvvPnzxxRcJ5zJIPZ/6+nrcfvvtGDx4MORy\nOebOnQuFQoGTJ0/GaeXiSD0fxurVqzFjxowYrzI4pJ7Tjh07MGHCBEybNg0ymQwzZ87E66+/Drk8\nsX5lpZ7P999/D6fTCYfDAY7jIJfLE3b62Isvvog33ngDS5Ys8blPslwTEplUuv6m0rU3la67dL1N\n7OttrK+1ifXTjDE1NTUoLS0VbCspKUF1dbVgW3V1Nfr3789/npOTg6ysLNTU1MRknVKRej7XX389\n7rzzTv7zgwcPwmAweD033kg9HwB4//330d7ejptvvjlWywsJqef07bffoqCgAEuXLsW4ceNw4403\nwuFwQK1Wx3K5AZF6PhMnTkTfvn3xk5/8BMOGDcNLL72EZ555JpZLlcwNN9yAHTt2YNiwYT73SZZr\nQiKTStffVLr2ptJ1l663iX29jfW1tlsLXqPRCJ1OJ9im1WphNpsF20wmk9e7I51OB5PJFPU1BoPU\n83Hn5MmTWLZsGZYtW4bc3NxoLzEopJ5PbW0t/vKXv+APf/hDLJcXElLPqa2tDdu2bcPNN9+ML7/8\nErNnz8YvfvELtLW1xXK5AZF6PhaLBf3798fbb7+NQ4cO4ec//zl+9atf+X1txouePXtCJpP53SdZ\nrgmJTCpdf1Pp2ptK11263ib29TbW19puLXh1Op3XC8BsNkOv1wu2+boIe+4Xb6SeD+PLL7/EzTff\njIULF+IXv/hFLJYYFFLOx+l04sEHH8Ty5ctRUFAQ6yUGjdSfkVqtxpVXXomJEydCpVJh4cKF0Ov1\n+Oqrr2K53IBIPZ+//e1v6NWrF4YNGwaNRoOlS5fCZrPhv//9byyXGzGS5ZqQyKTS9TeVrr2pdN2l\n623yX28j+fvfrQVvv379vGzxmpoagX0OAKWlpYL9mpub0dbWllC3oQDp5wMA77zzDpYtW4ZVq1bh\nl7/8ZayWGBRSzqe+vh5ff/01Vq9ejdGjR2P27NkAgEmTJiVk9bPUn1FJSQmsVqtgm9PpBMdxUV9j\nMEg9n9raWsH5yGQyKBQKKBSKmKwz0iTLNSGRSaXrbypde1PpukvXWxfJfL2N6O9/qNV1qYDFYuEm\nTpzIvfHGG4KKR4PBINivsrKSGzlyJHfgwAHObDZzjz76KHfnnXfGadW+kXo+//3vf7lhw4ZxBw4c\niNNKpSH1fNz58ccfE7JamCH1nL799ltu6NCh3O7duzmHw8G98cYbfiul44XU8/m///s/buzYsdzR\no0c5m83G/f3vf+euuOIKrqOjI04rD8y+fft8Vg4nyzUhkUml628qXXtT6bpL19vkuN7G6lrbrQUv\nx7laXtx4441ceXk5N2fOHO7QoUMcx3HcihUruBUrVvD7ffTRR9y1117LjRgxgrvzzju5xsbGeC3Z\nL1LOZ/HixVxZWRlXXl4u+Ldnz554Ll0UqT8fRqJeeN2Rek5ffPEFN2fOHK68vJybO3cud/jw4Xgt\n2S9SzsfpdHIvvfQSN2XKFG7UqFHcLbfcwh0/fjyeyw6I50U4Wa8JiUwqXX9T6dqbStddut4m/vU2\nVtdaGcclmGdPEARBEARBEBGkW2d4CYIgCIIgiNSHBC9BEARBEASR0pDgJQiCIAiCIFIaErwEQRAE\nQRBESkOClyAIgiAIgkhpSPASBEEQBEEQKQ0JXoIgCIIgCCKlIcFLEARBEARBpDQkeAmCIAiCIIiU\nhgQvQfjgk08+wdChQ3H27Fl+2xNPPIGpU6eisbExjisjCIJIPeiaS0QTErwE4YNp06Zh4MCBWL9+\nPQBgw4YN+Oijj/Dqq68iPz8/zqsjCIJILeiaS0QTxerVq1fHexEEkYjIZDIUFRXh6aefhkqlwvr1\n67FhwwYMGjQo3ksjCIJIOeiaS0QTGcdxXLwXQRCJzE033YQjR45g/fr1mDRpUryXQxAEkdLQNZeI\nBhRpIAg/7N27F8eOHQPHcXRLjSAIIsrQNZeIFuTwEoQPjh07hoULF+Lhhx/Gnj17YDQasWHDhngv\niyAIIiWhay4RTcjhJQgRzp49izvuuAOLFy/G/Pnzcc899+A///kP/ve//8V7aQRBECkHXXOJaEMO\nL0F40NraiptvvhljxozBmjVr+O333nsv6urq8NZbb8VxdQRBEKkFXXOJWECCl5Q7riwAAABuSURB\nVCAIgiAIgkhpKNJAEARBEARBpDQkeAmCIAiCIIiUhgQvQRAEQRAEkdKQ4CUIgiAIgiBSGhK8BEEQ\nBEEQREpDgpcgCIIgCIJIaUjwEgRBEARBECkNCV6CIAiCIAgipSHBSxAEQRAEQaQ0/x9xqMkM3riK\nwwAAAABJRU5ErkJggg==\n", 110 | "text/plain": [ 111 | "" 112 | ] 113 | }, 114 | "metadata": {}, 115 | "output_type": "display_data" 116 | }, 117 | { 118 | "name": "stdout", 119 | "output_type": "stream", 120 | "text": [ 121 | "--------------------------------------------------------------\n" 122 | ] 123 | } 124 | ], 125 | "source": [ 126 | "###############################################################\n", 127 | "#FIPY solution\n", 128 | "value_left = 1\n", 129 | "value_right = 0\n", 130 | "\n", 131 | "Lx = 1. # always put . after 1 \n", 132 | "nx = 100\n", 133 | "# define mesh\n", 134 | "mesh = Grid1D(nx=nx, dx=Lx/nx) # with nx number of cells/cellcenters/pixels/pixelcenters\n", 135 | "\n", 136 | "# define cell and face variables\n", 137 | "phi = CellVariable(name='$T(x)$', mesh=mesh, value=0.)\n", 138 | "D = CellVariable(name='$D(x)$', mesh=mesh, value=1.0) ## coefficient in diffusion equation\n", 139 | "# D = FaceVariable(name='$D(x)$', mesh=mesh, value=1.0) ## coefficient in diffusion equation\n", 140 | "source = CellVariable(name='$f(x)$', mesh=mesh, value=1.0)\n", 141 | "C = CellVariable(name='$C(x)$', mesh=mesh, value=1.0)\n", 142 | "\n", 143 | "# apply boundary conditions\n", 144 | "# dirichet\n", 145 | "phi.constrain(value_left, mesh.facesLeft)\n", 146 | "phi.constrain(value_right, mesh.facesRight)\n", 147 | "\n", 148 | "# setup the diffusion problem\n", 149 | "eq = -DiffusionTerm(coeff=D)+ImplicitSourceTerm(coeff=C) == source\n", 150 | "\n", 151 | "c = 15.\n", 152 | "f = 10. #source\n", 153 | "\n", 154 | "source.setValue(f)\n", 155 | "C.setValue(c)\n", 156 | "\n", 157 | "# getting input field images\n", 158 | "a = input_field_data[ 5 , : ].reshape(-1,1) # just taking one sample\n", 159 | "# 'a' is one image of input field: conductivity image of nx cells/cellcenters/pixels/pixelcenters from test_data #returns (nx,1) matrix \n", 160 | "D.setValue(a.ravel())\n", 161 | "\n", 162 | "eq.solve(var=phi)\n", 163 | "x_fipy = mesh.cellCenters.value.T ## fipy solution (nx,1) matrix # same as cellcenters defined above\n", 164 | "u_fipy = phi.value[:][:, None] ## fipy solution (nx,1) matrix\n", 165 | "\n", 166 | "# x_face=mesh.faceCenters.value.flatten() #cell faces location i.e.edges of the element \n", 167 | "# y_face=phi.faceValue() #cell faces location i.e.edges of the element\n", 168 | "\n", 169 | "# print ('done1')\n", 170 | "###############################################################\n", 171 | "# Initialize the plot\n", 172 | "fig = plt.figure(figsize=(10,5))\n", 173 | "\n", 174 | "try:\n", 175 | " ax1.lines.remove(lines[0])\n", 176 | " ax2.lines.remove(lines[0])\n", 177 | " lines2.set_visible(False)\n", 178 | "except:\n", 179 | " pass\n", 180 | "##########\n", 181 | "ax1 = fig.add_subplot(1, 2, 1)\n", 182 | "ax1.plot(x_fipy, np.log(a), 'g', lw=1.5, label='log(Input field)')\n", 183 | "ax1.set_xlabel('$x$', fontsize=14)\n", 184 | "ax1.set_ylabel('log(Input field)', fontsize=14)\n", 185 | "##########\n", 186 | "ax2 = fig.add_subplot(1, 2, 2)\n", 187 | "# lines2 = ax2.plot(x_fipy, u_fipy, 'b', lw=2)\n", 188 | "# lines2 = plt.scatter(x_fipy, u_fipy, s=10, cmap='Greens', label='FVM solution')\n", 189 | "lines2 = ax2.scatter(x_fipy, u_fipy, s=25, cmap='Greens',label='FVM solution')\n", 190 | "ax2.set_xlabel('$x$', fontsize=14)\n", 191 | "ax2.set_ylabel(r'$\\hat{u}$', fontsize=14)\n", 192 | "plt.legend(loc='best')\n", 193 | "plt.tight_layout()\n", 194 | "##########\n", 195 | "plt.suptitle('ground_truth(fipy)')\n", 196 | "# plt.savefig('ground_truth(fipy)_for_expt_data.png')\n", 197 | "plt.show()\n", 198 | "\n", 199 | "print (\"--------------------------------------------------------------\")\n", 200 | "####################################################################################################################\n" 201 | ] 202 | }, 203 | { 204 | "cell_type": "code", 205 | "execution_count": 6, 206 | "metadata": { 207 | "collapsed": true 208 | }, 209 | "outputs": [], 210 | "source": [ 211 | "np.save('input_field_ground_truth.npy', (a.T).flatten())\n", 212 | "np.save('xfipy_ground_truth.npy', (x_fipy.T).flatten())\n", 213 | "np.save('ufipy_ground_truth.npy', (u_fipy.T).flatten())" 214 | ] 215 | }, 216 | { 217 | "cell_type": "code", 218 | "execution_count": 7, 219 | "metadata": { 220 | "collapsed": true 221 | }, 222 | "outputs": [], 223 | "source": [ 224 | "a = np.load('input_field_ground_truth.npy')\n", 225 | "x_fipy = np.load('xfipy_ground_truth.npy')\n", 226 | "u_fipy = np.load('ufipy_ground_truth.npy')" 227 | ] 228 | }, 229 | { 230 | "cell_type": "code", 231 | "execution_count": 8, 232 | "metadata": {}, 233 | "outputs": [ 234 | { 235 | "name": "stdout", 236 | "output_type": "stream", 237 | "text": [ 238 | "(100,)\n", 239 | "(100,)\n", 240 | "(100,)\n" 241 | ] 242 | } 243 | ], 244 | "source": [ 245 | "print a.shape\n", 246 | "print x_fipy.shape\n", 247 | "print u_fipy.shape" 248 | ] 249 | }, 250 | { 251 | "cell_type": "code", 252 | "execution_count": 11, 253 | "metadata": {}, 254 | "outputs": [ 255 | { 256 | "name": "stdout", 257 | "output_type": "stream", 258 | "text": [ 259 | "[1.01665719 0.99063377 0.96193869 1.00977752 0.9967327 0.93616409\n", 260 | " 0.94247926 0.88499612 0.82467297 0.89702066 0.82627309 0.85403733\n", 261 | " 0.84076792 0.82358127 0.76910714 0.75502232 0.79064979 0.8077229\n", 262 | " 0.71635142 0.74220147 0.76336603 0.79895011 0.7790215 0.77609703\n", 263 | " 0.7498874 0.70596455 0.7252762 0.76560805 0.73079021 0.68463428\n", 264 | " 0.75905361 0.72216237 0.72741333 0.76272434 0.68458929 0.67557567\n", 265 | " 0.61552266 0.69046765 0.70006684 0.67744345 0.67553489 0.69968927\n", 266 | " 0.67901964 0.67667017 0.66060945 0.65913227 0.69009529 0.6539962\n", 267 | " 0.64044401 0.63358467 0.64334425 0.65006683 0.65808418 0.64520239\n", 268 | " 0.62904493 0.58834116 0.65386152 0.59925661 0.61686366 0.64550558\n", 269 | " 0.61375974 0.57963595 0.62288004 0.59019097 0.57426343 0.61777706\n", 270 | " 0.59539112 0.66732123 0.53638515 0.5665077 0.57721128 0.60061597\n", 271 | " 0.54528089 0.59568973 0.54988125 0.52017278 0.53193775 0.52481327\n", 272 | " 0.50808093 0.48724559 0.4909199 0.50467817 0.47279835 0.48318282\n", 273 | " 0.38580246 0.36063373 0.43630158 0.37056565 0.35831946 0.29446733\n", 274 | " 0.27415747 0.2366451 0.21431759 0.22076607 0.13122556 0.14089704\n", 275 | " 0.06444488 0.10643745 0.02854901 0.07482388]\n" 276 | ] 277 | } 278 | ], 279 | "source": [ 280 | "# https://www.mathworks.com/matlabcentral/answers/115070-adding-noise-with-certain-standard-deviation-to-uncorrupted-data\n", 281 | "#assuming noise as Gaussian with mean 0, std 0.1047\n", 282 | "# noise = 0.1047*np.randn(1000,1);\n", 283 | "\n", 284 | "std_dev = 0.03\n", 285 | "noise = std_dev * np.random.randn(*u_fipy.shape)\n", 286 | "#or\n", 287 | "# noise = np.random.normal(0, std_dev, u_fipy.shape)\n", 288 | "\n", 289 | "u_sensor = u_fipy + noise \n", 290 | "print u_sensor" 291 | ] 292 | }, 293 | { 294 | "cell_type": "code", 295 | "execution_count": 12, 296 | "metadata": {}, 297 | "outputs": [ 298 | { 299 | "data": { 300 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfgAAAFCCAYAAAAdec9sAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xt80/W9P/DXN01zL72lN1qp5SZFfFiFlRUrImXiHMpF\npiLy8OxwUPg5cTqZChNvQ4+iDHHnoLId50DOBkwPIsfh6LTAuGgV9HATpAUpvSW9J2mSJvn+/igJ\n+bZJm5b02zR9PR+PPR7mm2/STwPjnc/l/X4LoiiKICIioqii6O8BEBERUfgxwBMREUUhBngiIqIo\nxABPREQUhRjgiYiIopCyvwcQLna7HUePHkVKSgpiYmL6ezhERER9zu12w2QyYdy4cdBoNJLnoibA\nHz16FPPnz+/vYRAREcnuvffew4QJEyTXoibAp6SkAGj/JdPT0/t5NERERH2vuroa8+fP98VAf1ET\n4L3L8unp6cjKyurn0RAREckn0NY0D9kRERFFIQZ4IiKiKMQAT0REFIUY4ImIiKIQAzwREVEUYoAn\nIiKKQgzwREREUYgBnoiIKAoxwBMREYVAFEVUVFT0+vXnz58P42i6xwAfAofLibMNFXC4nP09FCIi\n6kMvv/wynnzyyYDPvfLKK3jvvfd69b7Hjx/HvHnzfI8XLFiATZs29eq9QhU1pWr7yqdl+/HBiV2o\ntZiQakjB7NzpuHn4pC5f43A5UdVSi4y4VKiVKplGSkREfamhoQGJiYm9em1LSwva2trCPKKucQbf\nBYfLiQ9O7EK1pRYeiKi21OKDE7u6nMl/WrYfy3atwpOfvIhlu1bh07L9Mo6YiGjg+OSTTzB9+nRM\nnDgRy5cvxz333IP3338fADB16lQ8/fTTmDhxIp555hm4XC6sXbsWkydPxsSJE7F06VLU1NQAAN5/\n/33MmTPH975WqxVXXXUVKioqUFFRgQkTJuDtt9/GDTfcgIKCArz44ou+eysqKnD//ffjuuuuwz33\n3IOqqqqAY33nnXewY8cObNy4EUuXLkVFRQXGjx+PJ598EhMmTMD27ds7zco3bdqEBQsWoK6uDosW\nLUJjYyOuu+46NDQ0AABOnjyJu+++G9dddx3mzZuHCxcuhPXzlT3Af/PNNygsLAz6/EcffYSioiLk\n5eXhwQcfhNlslnF0UlUttai1mCTXai0mVFtqA97fmy8ERESRyO50obyyCXanq0/ev7y8HMuWLcPy\n5cuxb98+DBs2DIcPH5bcU1lZiZKSEixbtgzr1q1DcXExNm/ejM8++wxDhgzBI488AlEUu/1ZLS0t\nqKiowKeffor169dj8+bNvp/1yCOPYOTIkTh06BB+9atfoaSkJOB7/OxnP8Ptt9+OBQsWYN26dQAA\ni8WCzMxM7N+/H7fcckvQn5+cnIwNGzYgISEBhw8f9q0CHDp0CK+++ir2798PpVKJ9evXh/TZhUq2\nAC+KIrZt24Z//dd/DbpMcfLkSTzzzDNYs2YNDh48CKPRiKeeekquIXaSEZeKVIO0BV+qIQXphtSA\n9/f0CwERUSTa/fk5LH3tM/xizWdY+tpn2P35ubD/jJ07d+KGG27ATTfdhNjYWDz44INITZX+2zp9\n+nRoNBoYDAZs374dDz30ELKysqDVarF8+XJ88803KCsrC+nnLVq0CCqVCnl5eRg+fDjOnTuH8+fP\n4+jRo3j00UehUqlw/fXX4yc/+UmPfo/bb78dKpUKWq22R68DgFmzZuGKK66AVqvF1KlTL+sAXyCy\nBfg333wTf/rTn7B48eKg9+zYsQNFRUW49tprodFo8Pjjj2Pv3r39NotXK1WYnTsdafoUCBCQpm/f\ngw+2rx7sC0GiJoGH9IhoQLA7XdhSfBpVZis8IlBltmJL8emwz+Rra2uRkZHheywIguQxABiNRt9/\n19XVITMz0/dYp9MhMTHRt0zfnaSkJN9/K5VKeDwemEwm6HQ6GAwG33P+PyMU/mPsqfj4eN9/x8bG\nwu129/q9ApEtwN95553Yvn07rrnmmqD3lJWVYeTIkb7HiYmJiI+PR3l5uRxDDMhtzoT96CQ4jk6C\n/egkuM2ZaLLZ8M9TJ9Bks0nu9X4hSDekQgEB6YZU5BpH4NfFq7knT0QDQpXZipo6q+RaTZ0VVWZr\nkFf0TkZGBiorK32PRVHsFKwFQfD999ChQyX3W61WNDQ0IDk5GQqFQrIy3NjYGNIYUlNTYbPZJPeH\n+oUh0Bh7O46+IluAT01NlXwQgbS2tkKj0UiuabVatLa29uXQgvJ+k602OeC2xaHa5MAf9v0ND/z1\nGaz9ch0e+Osz+I/iHZK9qpuHT8Lq6Svw8vTl+E3RMpwwn+GePBENGBlGPdKS9ZJracl6ZBj1QV7R\nOzNmzMD+/fuxd+9euFwuvPvuu6iurg56/6xZs/Cf//mfqKysRGtrK1566SWMHDkSo0ePRk5ODs6e\nPYszZ87A4XDg7bff7jbeAEBWVhbGjx+Pl19+GQ6HA9988w127NgR9H6VSgWLxRL0+SuvvBJ79+6F\nw+HA+fPn8eGHH0pe63Q64XTK9+9/RJ2i12g0sNvtkmutra3Q6XT9Mp5O32QVbrQlnYKoskBQAKLK\ngpILJfj5a3+X7FWJHgU8tjhUt5i5J09EA4pGpcRdRaOQYdRDIbQH/LuKRkGjCm9W9RVXXIGXXnoJ\nzzzzDCZNmoQzZ85g6NChiI2NDXj/okWLMGXKFMybNw+FhYWor6/3BfJrr70W9913H+6//34UFRXh\nyiuvlCx/d2Xt2rUwm8344Q9/iF//+tf40Y9+FPTeW2+9Fbt27cLChQsDPv/AAw/A5XJh0qRJWLp0\nKWbNmuV77qqrrsLIkSMxceJEnDsX/jMNgQhiKEcQw+jQoUNYunQpDh061Om51atXo76+Hi+99BIA\noL6+HpMmTcKBAwe6zT2sqKhAUVERiouLkZWVFZax2p0uLH3tM9/SlKBthvrq/RD8vhaJHsBx7AaI\nrXEAgHiDClq1ErX1NqQa1cCoPWhyNfjuTzekYvX0FcyPJ6KIZne6UGW2IsOoD3twB9pPyNtsNsm2\n7KRJk/DKK690mWlFUl3Fvoiawc+YMQOffPIJSktL4XA4sGbNGkyePLnXhQUuV8dvsql6I4Q26TKV\n6NBBdFxaYWiyOFFdZ4NHBKpNDjgrc5CqC+2QHhFRpNColMgZGt8nwR1oP2R3//3348KFC/B4PPjv\n//5vOJ1O5OXl9cnPG4z6vZLdypUrAQDPP/88cnNz8cILL2DFihUwmUyYMGGCbzbfX6blZ6MwL9P3\nTfYPe13YU7kHHqUFCpcBqvqRcHhigr6+/mwqFE2ZcDjqYdcb4U7NBIbL+AsQEUWgvLw8LFq0CPPn\nz0dTUxNGjBiBN998U3KinS6P7Ev0faUvluiDabLZcOzC97g6cxi+OGrCluLTqKmzIjVJB7vDhUbL\npUMUMQoBbs+ljzjDqMfqh29EfbO9z5a+iIhocOgq9jG69EK8TodJo8YAaJ/h/2BcCo5WnMO4rGxJ\nwE+K16CuUXposMpsxaO/LUFdUyvSktsPr0zLz+6PX4OIiKIYA/xlkjSjOd2+x77ul1NQZbYiaYgG\ny97YK8kfjVEIMDW2p/15C0gU5mVyJk9ERGEVUYfsBppgtecFhQc5Q+MRb1BLDukZEzTweKQ7IjV1\nVpyrau7Tms9ERDT4cNp4GbqqPZ+d0L4X4n9IL9CMPk6vwqvvfYnaehuX7ImIKGw4g78MoTaj8aab\ndJzRpyfrIAC+tLq+qvlMRESDD2fwl8Fbe963B2/oPs/df0bvaHPjiTf2Ago3BLUVokPvq/mcMzS0\nKkxERESBMMBfppuHT8KkYRNQbalFuiE1pCI2gsIDQduCjIQkJGTXwhp3EoLaBtGhg75lTNhrPhMR\n0eDDAB8GaqXKt+feHf9T9yl6I5BuhcLT3pVO0NqgiiuHoPD05XCJiGgQ4B68jDqeuq+xmtDqkbac\nbXE1shkNEfU7h8uJsw0Vfd79cvXq1bjhhhtQUFCAhQsX4vz583C73fjd736HqVOnoqCgAE899ZSv\ni9v777+PhQsXYtmyZbj++usxbdo0/M///E+X7we0t5d97rnncMMNN+CGG27AihUr0NLSAgB44403\n8OCDD+K2227D5MmTu+wYN5AwwMso0Kn7jgId0iMiktOnZfuxbNcqPPnJi1i2axU+LdvfJz/nwIED\n+Pjjj/HRRx9h7969SE9PxxtvvIF33nkHf//73/Hee+/h73//O+x2O1544QXf6/bt24cbbrgBn3/+\nORYsWIAXXngBDocj6PsB7WXRy8rKsGPHDvzv//4vzGazr1Q6ABw8eBBr167Fzp07o6ZcLgO8jAKd\nuh+ijkOaIQUKCEg3pGJ27nSIHgXz4omoXwSr79EXM/nY2FjU1dVh69at+P777/HCCy/glVdewbZt\n2/Dzn/8cGRkZMBgMePzxx/Hhhx/C4XAAAIYOHYpZs2ZBqVRi1qxZsFgsqKurC/p+drsdu3btwuOP\nP46kpCTEx8fjiSeewMcff+xrUZ6bm4vRo0cjLi4u7L9nf+EevIyCnbr3P6S396sqLP3LZ6ipszIv\nnohkF0p9j3DxNhTbvHkz1q1bh8zMTDz11FOoqqrCr371K8TEXGrkpVQqUVlZCQCSDqNKZXsY83g8\nQd9v7NixaGtrQ2Zmpu91mZmZEEURNTU1AICUFOnkKxowwMss2Kn77IQs2J0ubCk+7SuEw1K2RCQ3\n70qj/1mgvto6rKqqwvDhw7Fp0yZYrVa89957+MUvfoHk5GSsWrUKBQUFAIC2tjacP38ew4YNw+HD\nh3v8fl9++SVUKhUqKyuRlJQEoL1Ji0Kh8D0WBCHsv19/4xJ9P/Ceuu+YUldltqKmziq55s2LJyKS\ng3elMd2QKtk6DCUFuKe+/vprPPjggzh//jz0ej2GDBmCIUOGYM6cOfiP//gP1NbWoq2tDWvXrsWi\nRYvQXfPTYO8XExODO+64A6+99hrq6+vR1NSEV155BTfddFNULcl3xGlhBMkw6pGWrJcE9LRkPfPi\niUhWvanv0Ru33norvv32W8ybNw9WqxU5OTlYt24drr76arS1teHuu+9Gc3Mzxo4di7feesu3HN/T\n9wOAp556CqtXr8Ydd9wBh8OBoqIiLF++vE9+r0jBfvARZvfn53ztZrkHT0REXWE/+AHEv5RthlEf\ndO/d4XKiqqUWGXF99+2aiIgGLgb4CORtThOMpAf9xZP4Nw+fJOMIiYgo0vGQ3QAjZ44qERENXAzw\nEairEpFd5agSERF5cYk+wnS3/C5njioREQ1cnMFHkFCW3+XMUSUiooGLM/gIEmqJSLlyVImIaODi\nDD6CBGpGE2z5PVg1PCIiIoABPqIEW35ndzkiIuopLtFHmI7L7+wuR0REvcEZfATyLr+LHoWvu5xH\nvNRdLpSZfFepdkREFP04g49gXXWXY6U7IiLqCmfwEczbXc5fd93lWOmOiIgABviIplEpcVfRKGQY\n9VAI7QH/rqJR0KiUsDtdAQ/eBUu1+77pApfsiYgGES7RR7hA3eUCtZT13pOUkNSp0p1BbcC6g+/A\nZDFzyZ6IaJBggB8A/LvL2Z0u38E7oH2f/o87j+Mvu0+htt6GtGQ9rs2fAFH8ArVWM1J0yXC4Hai5\nOKv3LtlPGjaBOfRERFGMAX6ACXTwrsniRJPF6XvetkcFjaYADlsdbImxsGSWSO4PVB2PiIiiC/fg\nBwD/lLdAB+8AAAo3BG0zoHCjyeJEjdkJty0OpqpYwKGT3BqnTGBzGiKiKMcZfIQLlPJ2V9Eo3x58\napIOVk0ZnEmnIKhtEB06uKqGw22+ODv3xMB5IQfKjDLf886WHDRZHThdcwbjsrIRr9N1PQgiIhpw\nGOAjmH/KG3Bp/3z19BV+h+pi8fjHn8LlsgEABK0NqsxytNZnAJ4YAIDbnAV3fcalLwBJVXh4+3Pw\nxFqhOGTA5KGT8VDR7f32exIRUfhxiT6CddVdznvwrsFRjxZXo/SFahtS00UoBCA9WYcEgwrwxEBs\njQMAKIeWQVRbISgAUWXBnso9aLLZ5Pq1iIhIBpzBRzBvdzn/lLeO3eUC3ZNuSMELS36M+sY2ZBj1\n2Hfkgm9JP97oQqtKGsw9SguOXfgek0aN6ftfioiIZCHbDP748eOYO3cu8vLyMHPmTBw5ciTgfVu3\nbkVRURHGjx+Pe+65B0ePHpVriBEnWHc5//S2YPfE63TIGRoPjUqJafnZWPfLKVj72BS8uvjHULgM\nkp+jcBlwdeawHo2Nte6JiCKbLDN4h8OBxYsXY/HixfjpT3+K7du3Y8mSJdi9ezf0+ksnwk+ePIlX\nX30Vf/7zn5GdnY3f//73eOSRR1BcXCzHMCNSx+5ygXLXQ7nHP5d+8tDJ2FO5Bx6lBQpX+x68WqlC\neWWTr5hOV1jrnogo8skS4A8ePAiFQoF7770XADB37ly8++67KCkpwW233ea779y5c/B4PHC73RBF\nEQqFAhqNRo4hRjRvd7nLvcfroaLbcZ+tCMcufI+rM4fhi6MmLH0ttJa0wQ7+9bRwjsPlRFVLLTLi\nAn8hISKiyyNLgC8vL8eIESMk13JyclBWVia5VlhYiCuvvBI/+clPEBMTA71ejz/96U9yDHHQidfp\nMGnUmICV8bYUn0ZhXmbAmXxXB/9C/YLBFQAior4nyx68zWaDVquVXNNoNLDb7ZJrDocDI0eOxLZt\n23D48GHcf//9+PnPf97pPgqfrlrS+vPuuSdpE5BqSJE81/HgX1fY7Y6ISB6yzOC1Wm2nIG2326Hr\nUGDld7/7HdLT03HNNdcAAB566CFs2bIF+/fvx9SpU+UY6qDjrYznH9DTkvVIGqLx7ckfqPhcMuPO\nNbavxvjPwENdZg/HCgAREXVPlgA/fPhwbNq0SXKtvLwcM2bMkFyrrKyUzPQFQUBMTAxiYmLkGOag\n5G1J69+d7uqcJCx7Y297pTyjGhi1B02uBgDw7b3/pmgZGuyNQQ/1BRNK6h8REV0+WZboCwoK4HQ6\nsXHjRrS1tWHbtm0wm80oLCyU3DdlyhRs27YNx44dg8vlwjvvvAO3243x48fLMcxByz+NbvXDN+JY\neT2qzFZ4RKDGYkJTW4Pk/lqLCQ32RmQnZPX4gFywtD4AkrQ7puEREV0eWWbwKpUKGzZswLPPPos1\na9YgOzsb69evh06nw8qVKwEAzz//PO6++240Nzfj4YcfRnNzM3Jzc/H73/8eBoOhm59Al8ubRlde\n2STZkxcdenjsOii0l4rjxCkTkKBKkqTV2Z0uSc/6rnRM69v/fSmW7Vol2QI4YT7DQ3hERJdBEEVR\n7O9BhENFRQWKiopQXFyMrCzu5faUN20tUZ2EZa/vl+zJK40XEJNxxlfLPrZ+NAz24b7+81fnJOFY\neb0kzc5bKz/YFwDv46SEWDz96cuSJfsYIQZu0e17nG5IxerpK5hOR0TUQVexj6VqqVPa2rX5E4DP\n9aipsyIpXoM6cyZc9em+AO/wxMCC9hl9ldmK2nob3B7R9/iPO4/jL7tPSb4AHD1bC5PVhBR9CsZd\nmer7QmDMaENLlvTQnX9wB3p3CE+OPHvm8hNRJGOAH+QCFa4BSrH6kSdQ39iGpCEaLHtjL6rMVl+z\nmo68wd2ryeJEk6V977zKbIUJpxCTcQaxahsaHDp8Vj4CLnMmAKC2WgFtig5QX1oxEKCACI/vcU8P\n4fU2z74nAZu5/EQU6dhNbpALlrZW21oDQdsCjUbAXUWjkGHUS7vT+YlRCMF/gMKNmIwzUGhtEBSA\nQmtDTMYZQHFxln6xX72nVQfRA3hadVA2ZSFNnxK0/n5Xeptn/2nZfizbtQpPfvIilu1ahU/L9of9\nZxARyYkz+EEuUNqaQW3AuoPvwGQx+2an6345xbeH7t+druMefGqSDnaHC40XZ/CC2gpBLe1eJ6ht\n7cv9F1cEOvarbxNj8OS0iWhxN+LqzGGI71AvoSu9ybPvafld5vIT0UDAAD/IedPWvMvNKXoj7C47\nai4GMP9g521WMy0/u8tDdP5fAFINKXAIQ2BHs+9naoUhSDAYUWt3SL4QeAN+nEGFdX/+v4t7+BW4\nq2gUbrw+I6Tl897k2fc0YDOXn4gGAgZ4kqStOVxtWFm8WvJ8oGDn352u4+Np+dn4wbgUHK04h3FZ\n2fiqegjeP/431FrNSNUbMWfsrSiYkx/4C8HFgF9dd+kQ38ZDn+B/as7DZLv0+mD73R2/sIRSaa+n\nAbs3P4OISG4M8ATgUjc6h8t52bNTyQG00+3B79Vbf92pnW2gFQFHmxtPvLH30psp3LDGnUSrrT3g\n11hN2Hz4oy6714XSPrfj797TgN3Tn0FEJDcGeJK43NlpV/vZXe1Pe1cA7E6XpDZ+oD38prYGfFd7\nATokBS2s05P2uUDvAnZPfwYRkZwY4KmTy5mdXu4BtI618RM1SbA4dBD8Kul57Dq8+l8nUN/Y1m3/\neq9QUuAYsIkomjDAU0C9DXbhOIDmv2SfNESDpe9UwoqTvlP27qoRMDe0AbjUv/4HY9NR32wPeOiv\nYze83uass7ANEQ0kDPAUVuE6gOZ/aG/BxFvwl39cAZPVjAR1IuobXJJ7q8xW/GJtMRocdZ0q5QXq\nhtdVCpw//4C+//tSFrYhogGFAZ7CLtwH0DrO6L2V9byUxgtoCVIpr8ZigqatAfCrxVN9ccsgTZce\ntEGO/0FBb+pgk6Pl4utD/5JARNRfGOApLDouX4d7P9t/Ri/Zo09QwnKxUh6A9r36jDNw1acDnpiA\n3fAEpx7fHLNhx57PAjbISUqIlRwUrLGaOo2HhW2IKNIxwNNlk7suu/+M3oo6PL/nI8nzkkp5nhi4\nqoZDmVHm28N3VV2JD86ehamxFUDnBjmBGuB0xMI2RBTpGODpsvS0zGu4CAoPBG0LhmlTER+b6Ntj\nBwJUyrMPR+OxS6VwjUP0qGtqlbyff4Oc2moFdKl6iCqL7/kh6jhoYtQw2eqQqjeysA0RRTwGeLos\n/VGXveOKwfVZY3Dc9F0IlfJikJ6sx+ybRuCDkjOSfXwJTwwcFVdCnVkOT6wVCpcBqeJ1MJ1NhMNq\nhl1vhDs1E/aswD3ug+XmExHJif8K0WWRuy574Pa2wKppv0KDvbHbSnne4BurVHQqj+ttkAMAqL8C\nNr8GOMehhNvjABCHapsjYM9778n9UHPzA/1uPUnDY9oeEXWFAZ4ui9x12YOtGDTYG0OqlOfVMej7\n18NPitegrtEOIOZSxzt03fO+tt4Gt0f0PQ6Um9+Vnp5jYD96IuoOAzxdNjnrsodzxaBjg5yuUvFi\nFIIvgAfS8bkqsxWP/rYEdU2tnU7pewO+dwaepE3o0TmGvjr3wBUBoujCAE9hIVeZ13CuGHQMaMFS\n8brreQ9c/AIAFwS1FaJDjxgog57ST0vW49r8Vpxs/QK1VjMSNQlosDdIxtbVOYa+OPfAFQGi6MMA\nTwNOOFYMugtoPel5n5ashzHHhDNtX8GjtEDRpofjQg5gvhRsJUv69c1ovLAfULevENTbGyBAAcDj\nu7+rVYlwn3vor0wIIupbiv4eAFFveFcMejtz9wY0D0RfQHO4nJL7vDN67/65/+Np+dlY98spWPvY\nFKx+ZBIscSchqiwQFICotkKddRZQuAP+fEFthRgrPcEvih4IbVqIHkBwGnCVZgLsThf+eeoEmmzS\nbnreVYx0QyoUEJBuSMXs3OkAgLMNFb7fw+FySh4H09WKABENXJzB06ATriVub8A/21DR6f1ElRUp\nGS7UN1hh1BnhsMO3pC869BADdMhznPghhFgHRIcO+0yV2FP5TPuKwCEDJg+djPsKinC04hzGZWV3\nWsXY/30plu1a5VuRyDWOwAnzmZCW3OXOhCAieTDA06AT7oAW6P2GqA3QjPo/WG110OqNuE77A3z9\nub59Dz8xDtb60XAmnfKrrjcccKkgulSAwo22pFNQqGwQAIgqC0oqd2PPX0vgUVp9AX/hjT+GxxYH\nu8rVaYndZK2DW3T7Hne15C53JgQRyYMBngadcAe0ju/nbU5TazMDaK9lLwilWP3IE6hvbLu4hz8a\nf/lHNkxWM4y65PYZPtpn+ILaCkEtXZYXY52A4LwU8C+U4PBrSpjqnAFL63qDu1d3KxRyZkIMJsxM\noP7EAE+DUrgDmv/7OVxtWFm8WvJ8rcWERmc9coa2B9iu8vBT9EY0t+l9h/ACEWOtqLXWQRTjUFut\ngDZFJ71fFADhUupenDIBWoUB/zx1AuOyshGv03WqvCdXJsRgwcwE6m8M8DRohTuged/P4XKGtAUQ\nLA8/w6jHH/a6sKdyDzxKCwSXHoLQBsReOiwnOnQQHbr2B54YOC/kSBrqeFqSoIir9z1ubjXg4e2r\nfHv6I2Kvh+lsIkxWE1L0Kbh7am6XWQPhKr07WGa0zEygSMAATxRmvd0C8A/4DxXdjvtsRTh24Xtc\nnTkMmw4U+wK+wmWAqn4kHJ4Y32vd5iy4/UrrwhMDKNztj9vUUOcelOzpn3IeADKUiFW1osGhwx/2\nncNfdg+XlN49era2yy8APRWOGe1A+YLQHz0aiDpigCfqA+HYAojX6TBp1BgAnQP+F0dNAWvpe0vr\ntlfeay+1K2ibO+3pC7Ft7asCAAStDW1Jp1B9LAUQY1BltsKEU4jJOINYtS3gF4BAlfkC6W21vkAG\n0pI3MxMoEjDAE/WRcG8B+Af8rvbwO1beC2VPX1Db2mf7rXGAwo2YjDNQXEzjC/QFoGNlPm9zHf9l\n/QMVn+P943/rslrf900XEKuI7XZG3tsl7/6a8TMzgSIBAzzRANXVHn7HPfSOe/qxsR640Op7L6FN\n79vTD3SKX1DbIGhaAFEB0aHv1GxnS/FptLk9eL/kW5isJhgNiXBm74VdaAYQuFqfQW3A6/v/CyZb\nna/Nb7AZeW+WvPt7xs/MBOpvDPBEUaJjx7yu9vS/qj7im12n6o0YY/wBvr6gbV/yN6TAIQyBHc2+\n9xJcKqhGfg1B1erL23f7leKtNlux+fPdaE0/iVi1DY1ODQTYIfiNTxQ9EFxaeGJaIbj0sIkuNAst\nANpTCTcxUGYvAAAgAElEQVQf/gjXp+f5Ugn9v6QkJST1aMk7Ug65MTOB+hMDPNEg4b/EH2h2ab/R\nf3l9iO8LQIouGS0KG1o9l5bsVZnlaG1MhRBrh+jQIyleDUv8yUvL+ho7RI80Vc+/Wh8UbqhzD0Lw\n+wbQ1NaAR//zf1FXHdtpm6G93v8YCG0230HDqzTBgzUPuRExwBMNWh1nl/4z/u7y+qG2QnfNAXhi\nWqFwGTA6+VocaZYu6wMiPHYNBJU9YLW+QOV6TSYRgqYZVfVu1NbbfG14q8xW1Nbr4UaBL1Pg6wot\nmiY6UN9s73TQj4fciBjgiSiIrvL6Y4QYuGNbfWl3VZ5TGBKbgGZXo+8erTAEquobYbY0dqrWB08M\n3FUjgIwzvoDtbklsn9X7l+/12wZoD/YxvkyBKrMVj/62BHVNrZKDft6x9+aQ20BJwyMKBQM8EXWp\nY7BM1CagvlV6It5kNWPu1T9BydlDvn39OWNvRcGc/C5O+v8QR88Oh8lqRrI+Hi1DSwD1pW0A5dAy\nuOsz2nP64U39u7TkH6MQYGpsPyjoPej3g7Hpvhl9Tw+59fehPKJwY4AnkslAnh36B8tETQJ+Xby6\n0/L37WN+hNvH/KhTQA3lpH9bbAN+/Y+PJT9TobYhNV2EuQqd9uST4jWoa7RL7q8yW/GLtcVocNT5\nivNMy88Oac99oKXhEYWCAZ5IBtEwO/Tfs+9q+burgBrspL/DpUV6h22A9LhUvLDkx4FP1Q/RYNkb\ne1FlvpTbrzReQItfcZ6Nh86jMO9nkr35YAF5IKbhEXWHAZ6oj0VKylY4hTvHO9ieebxOh3jdpfv8\nvyDcVTTKt+SfmKCEpUNxHitO4nSFGQaN1ld4J1hA7umhvGj8M6XoI1uAP378OFauXInvvvsO2dnZ\neO6555CXl9fpvtLSUqxatQpnz55FVlYWli9fjoKCArmGSRR20ZqyFe4c755+afBf8reiDs/v+Ujy\nvEJjw2/f34e66likGtXAqD1ocrWfHfAGZP+8+54cyovWP1OKLrIEeIfDgcWLF2Px4sX46U9/iu3b\nt2PJkiXYvXs39Hq9776amhosWbIEv/nNb3DLLbdg586dePjhh7Fv3z5oNBo5hkoUdkzZCl1PvzQI\nCg8EbQuGaVMRH5voC+AAAKcephoFIAI1FhM0bQ3wr7xT3VKLx9Z/DHOV0ncK/4Wbn/AVAwrUUtdL\nzj9T7vNTb8kS4A8ePAiFQoF7770XADB37ly8++67KCkpwW233ea7b/v27Zg0aRKmT58OAJgxYwZy\ncnKgUCjkGCZRn2Bd8r7RcQ/8+qwxOG76rr32vToJ1WWXTuCLDj08dp1vCR8APA4daqsFQIRffX3l\nxfr6FZ0K7YSShgcAZxsqwhaMQ9nn5xcACkaWAF9eXo4RI0ZIruXk5KCsrExy7dixY0hLS8NDDz2E\n0tJSXHnllVixYgVUKv6lpYGNdcnDK9AeOACsmvYrNNgbkaBKwrIT+1GFi4fwPDFwVQ2HMqNMmmfv\n13K3Y3392nob3HBB0FhRVe/GluLTKMzL9M3kO/6Z7v++FMt2rQrbobtQ9vl50I+6IsvU2GazQavV\nSq5pNBrY7dI0l6amJmzduhXz5s3Dvn37cMcdd+CBBx5AU1OTHMMk6lPe5WcG99A5XE6cbaiAw+WU\nXA+2B95gb0R2QhbidTrcVTQKGUY9FAKQnqxDnH04HMdu8P0P9Vd0/cOTzkN99T+hvno/1Ff/E7X4\nFueqmlFe2QS70wVAuqXgDcYeiL5g3HHcPdHVPr/3swn3z6ToIssMXqvVdgrmdrsdOp1Ock2lUmHy\n5MkoLCwEAMyfPx9/+MMf8NVXX+Hmm2+WY6hEFCG6mp2GsgcevKVuDNI75NWnJulgd7jQeHEGD4Ub\nyqFlUGik9fdXbz4EU52z05J9Xxy66+535EE/6o4sAX748OHYtGmT5Fp5eTlmzJghuZaTk4Pvv/9e\ncs3j8UAURRDR4NHd8nSo5xq6a6nbZLPhaMU5jMvKxhdHTb60O2O6By0dWuZCZUWttQ6iGNepcl5P\nu92ForvfkYc3qTuyBPiCggI4nU5s3LgR99xzD7Zv3w6z2eybqXvNnDkTd999Nz777DNMnjwZ7733\nHhwOByZOnCjHMIkoQoQyO+3NuQb/gC9ZITjdHjzX/XLKxfa0sXj60/+TBE+PXQexTQ1B2wzRoe9U\nC//a/AkQxS98pXp7c5Cy44G5rn5HHt6k7sgS4FUqFTZs2IBnn30Wa9asQXZ2NtavXw+dToeVK1cC\nAJ5//nmMHTsW69evx6uvvopHH30UOTk5ePPNNyWpdEQU/UKdnfY2F7+rFQLvFwD/4JmiN6K+SQv4\nNcNxV42AyZwJoP1Qnm2PChpNARy2Otj1RrhTM2HPCpxmF0iwLYmufkce3qSuCGKUrH9XVFSgqKgI\nxcXFyMri/hPRQNeXJ8TPNlTgyU9ehAeX/vlTQMDL05dLgqnD5fTV3//lzhclefaeVl37YT2/k/j+\n4g0qaNXetLv2PftgWwSj0jPwm72/lZbqNaRi9fQVDNrUpa5iH0vVElFE6svZaU9XCM42VKDFrxUu\n0F4pT1DbfO1rO+qYdteeZ3/KF/CNOSacafsKHqUFglsLxLZKXs8Dc3S5WEGGiCJWX6UWevev0w2p\nUEBAuiG1y/1r7xcCf/GxiUgzGH1peAmGrsfYZHGius4GjwhU1TfjlKMUosoCQYH24O4RJPfHKROQ\nbkiF3emSpOYRhYozeCIalHqyQhDsQFvB7YH73XdKu+tAUFshqKSn9EWIEO0aCCo7RIcOzpYc/OOL\nC/jgszMBq+kRdYcBnogGrZ4c0gv2hSBYGl5XAV906CE6dRA0l4K86NDBceKHEGId7QHeE4Otjadh\namxfuu+Ymhfs4B5L15IXAzwRUYi6+0LQVd69f8BPSxoCo3oCzjjb9+AVLgNU9SPhcKkgutqDcnKC\nBnVN0n35KrMVv1hbjAZHHVL0Kbh7aq7kZxyo+BzvH/+bL1VvzthbWbp2EGOAJyLqI90X2pnp617n\nX2gnLVmP2TeNwAclZ1BltvreT2m8gJaMM4hV29Dg0OEP+87hL7uHo7behlSjGo4rP4VdaAYA1FhN\n2Hz4I0lL3O5S9Si68E+biEgm/gEfAOJ1OkwaNQZA4C8AsUqFL+gnJihhyTjj64gnaG1oSzqF6mMp\ngBiDGosJarEZgt9Zvaa2Bjz6n/+LuupY7uEPQgzwREQy6W5/vOMXAP+gb0Udnt/zkeR+QX0pVU90\n6CE6dBD8W+LadTDVKHwtcUPZw6fowT9dIiIZ9LZwjzfoO1xaxMcmSortCG3tQR0A4ImBu2oEkHEm\naEvcQHv4nNFHL+bBExH1sXC0dlUrVbj3uhlI06dAgIA0fQpuyrwJGUlDoBCADKMeU3J+iMTqH6Ht\n+A1IqJqGOPtwyXsojRfQklWM2LH70ZD+d2w89Alqm1rwz1Mn0GSzBfnJNFB1O4P//e9/j6uvvhq5\nublISEiQY0xERFElXK1dA6Xq2W+U1ru3O12Sk/t/+ccJmKwmxGviYe2wh2+N+T88vP0kPLE2KA4Z\nMHnoZDxUdHtYf3fqP90G+H379mHDhg1obm5Geno6xo4di2uuuQZTp07F6NGj5RgjEdGAFs7Wrh1T\n9Tru2/s/jjFegGbcfqitZmg0CWhtlc7Shdg2iEIbBACiyoI9lXtwn60I8Tpdj8dFkafbJfo//vGP\nOHToEHbv3o0VK1Zg9OjR+OKLLzB37lz84he/QGtra3dvQUQ0qPW0NG44eLcFaqwmiBDRYG+AIHT9\nT75HacGxC9/32ZhIXiEfssvMzERmZiamTZsGAKivr8fjjz+O9evX47HHHuuzARIRRQO5W7sG2hYQ\n4UGyNhH1rY0wapNhbmmBqHT4nle4DBiWnIJ/njqBcVnZnMkPcL0+RZ+UlISnn34aDzzwAAM8EVEI\netu/vjcCbQukG1Lxm6JlaLA3It2Qit+X7MKeyj2+anopsUPxy53/3v744p78wht/3GVPe5bGjVyX\nlSY3dOhQmEym7m8kIiJZBWuQM0RjwBCNAQDwUNHtuM9WhGMXvsew5BT8cue/t3e4Q/uefMmFEhx+\nTQlTnTNgoZzepv6RPLoN8BMmTEBubi7Gjh3r+9+IESOgUCjw4Ycf4oorrpBjnERE1EOhbAt4q+n9\n89SJ9t70fs+JsVbUWusginGdCuUkJcT6Uv8A+FL/Jg2bwJl8hOg2wL/11ls4ceIEjh8/jnfffRen\nT5+GIAjQaDRwOp1Yu3atHOMkIqJeCHVbYFxWNhSHDBBVFt810aG7VEgH7YVyHv1tCeqaWmHMaENL\n1uWn/lHf6TbAjx8/HuPHj/c9drlcOHPmDOrq6jB69GgYjcY+HSAREfW9eJ0Ok4dOluzJq+pHwgFA\n0DZDdOgRAyVMzRYIGitqazXQpeolXwh6m/pHfaPHe/BKpRJXXXVVX4yFiIj6kf+e/NWZw7DpQDH2\npFwM+G16OJsSoI5r8JXCdTYmIn2YGg32et8efMfleR7C6z+sRU9ERD7ePXmHy4lv7aWXDt2prYg1\n2gCFCKC9El5MjALP3bwCrR5LwD1+HsLrX6xFT0REnQTKo/cGdy9RZUWrx4LshKyAM/fLrb9Pl4cB\nnoiIOvHm0fuLEWIkj9MNKUhQJaG8sgl2pwtAe2A/21CBc40XgtbfJ3lwiZ6IiDoJlEefaxyBE+Yz\nvsdXaSZg2ev7UVNnRVqyHtfmt+JbeylqLSak6I2IUxvQ5GjxvScP4cmLAZ6IiAIKlEfvcDlRbalF\ngioJy17fjyqzFQBQVd+MpsoDvlP1NVYThqjjkGZIgcliDnoIj/oOAzwREQXVMY/e+7i8sgk1dVbf\ndUFt7VQox+Kw4Ikbl0AVEytL/X2S4h48ERH1WIZRj7Rkve+x6NBD4TJI7kk1pGBYfGbAQ3jU9xjg\niYioxzQqJe4qGoUMox4KAchIGoLJQyfL2hKXusYleiIi6pVp+dkozMuUdJtzuKbL1hKXusYAT0RE\nvaZRKZEzNN73WPQo4LHFQdRxgbi/McATEVFY7P78HLYUn/alzXVsL0vy4lcsIiK6bHanC1uKT6PK\nbIVHhK+9rLcATqi8hXJY8e7ycQZPRESXrcpslaTNAUBNnRVVZqtkCb8rrF0fXpzBExHRZeuYNgcA\nacl6ZBj1QV4hxdr14ccAT0REl61T2pyxfQ8egKRWfTCBmtuwdv3l4RI9ERGFRce0uX1HLmDpa5+F\ndOjO29zGP6Czdv3l4QyeiIjCxj9trieH7rzNbVgoJ3w4gyciorALdujuXFUzVLExvsI4/gI1t6He\nY4AnIqKw8x6683abA4A4vQqvvvclauttQZfsOza3od6TbYn++PHjmDt3LvLy8jBz5kwcOXKky/sP\nHDiAMWPGwGq1dnkfERFFno6H7tKTdRAAVNfZLitPnkInS4B3OBxYvHgx5syZgy+++AILFizAkiVL\nggbvpqYmLF++HKIoyjE8IiLqA9Pys7Hul1Ow9rEp+OX88Wi2SlPevHny1DdkCfAHDx6EQqHAvffe\ni9jYWMydOxdGoxElJSUB73/22Wdx2223yTE0IiLqQ95Dd1dmDLmsPHnqOVkCfHl5OUaMGCG5lpOT\ng7Kysk73fvjhh2hubsa8efPkGBoREcngcvPkqedkOWRns9mg1Wol1zQaDex2u+RaZWUlXn/9dWze\nvBltbW1yDI2IiGQyLT8bPxiXgqMV5zAuKxtfHDWFnCdPPSdLgNdqtZ2Cud1uh06n8z32eDx44okn\n8OijjyItLQ0VFRVyDI2IiGTiX2s+5ZQRzeVXoN7cXsjGe+iuMC+zU/oc9Y4sS/TDhw9HeXm55Fp5\neTlGjhzpe1xdXY2vv/4azz77LCZMmIA77rgDAHDTTTehtLRUjmESEVEf6VhrvsZqgjXuJKBw++7h\nobvwkuVrUkFBAZxOJzZu3Ih77rkH27dvh9lsRmFhoe+eoUOH4ptvvvE9rqioQFFREUpKSqDX8xAG\nEdFAFqjWvEJjg6C2QWyNA8BDd+EmywxepVJhw4YN2LlzJ/Lz87Fp0yasX78eOp0OK1euxMqVK+UY\nBhER9RNvrXl/8bGJSDMYuzx0x/7wvSeIUZJs7p3xFxcXIyuLVZCIiCJNoH7vBVn5kuY0W4pP+w7d\nXZvfim/tpewP34WuYh9PMhARkSwC1Zp3uJwQtC1wuGJ9zWkAoKq+GU2VByCqLADg6w8/adgE1qgP\nEQM8ERHJxr/WvP+MPlGThFpkAGh/TlBb4VFaIPi9ttZiwvdNFxCriEVGHJvRdIcBnoiIZOd/qh4A\n6ux1UGc5YKvPADwxEB16CG16QH3pVL1aocW6g+/AZDFzyT4E7AdPRESyC3SqXlRZkZoutjenSYyD\n0jwKnlYdRA/gadXC7nSjxmKCB6JvyZ6H74LjDJ6IiGTnPVXvncEDQLohBS8s+THqG9vgaHPjiTds\n8FSlQFDbAIUb6tyDnZbsqy21bC8bBGfwREQkO7VShdm505FuSIUCAtINqZidOx3xOp20OY0nBmJr\nHMTWuPYlez9xygSkG1L76TeIfJzBExFRvwh0qt7L25zGmzaXmhgHq3kUnEmn2ovjOHRwtuRA9HCe\nGgwDPBER9Rv/U/UdTcvPRmFeJqrM1k5L9qJDhzYxBlVmK3KGxss86oGBAZ6IiCKWt5+83elCWrIe\nVWbrpdK2Rpa27QrXNoiIKOIF6yfPznPB8ZMhIqIBwX/JPsOoh0alhN3pkjymS/hpEBHRgOFdsgeA\n3Z+fk9Suv6toFKblZ/fzCCMHl+iJiGjAsTtdvtr1HhGoMluxpfi0rwsdMcATEdEAVGW2oqbOKrlW\nU2f1NashBngiIhqAMoz69kI4ftKSeareHwM8ERENODxV3z1+EkRENCBNy8/GD8al4GjFOYzLyka8\nTtffQ4ooDPBERDQg+feTTz3N9rEdcYmeiIgGHP9+8mwfGxgDPBERDTiB+sl728dSOwZ4IiIacLz9\n5P2lGlLYPtYPAzwREQ04wfrJA8DZhgou1YOH7IiIaIDq2E9+//elWLZrVfuhOwMP3XEGT0REA5Z/\nP3keupNigCciogGPh+46Y4AnIqIBj4fuOmOAJyKiAU+tVOEqzQQITgNEDyA4DbhKMwFqpaq/h9Zv\neMiOiIgGPLvTha8/18JWXwBBbYPo0OHrCi3sN7oGbX16zuCJiGjA87WP9cRAbI0DPDGDvn0sAzwR\nEQ14bB/bGQM8ERENeF21j3W4nIOy+M3g3JggIqKoMy0/G4V5magyW5Fh1EOjUuLTsv14//jfUGs1\nI1VvxJyxtw6a4jecwRMRUdTQqJTIGRrvm7lvPvwRaqwmiBBRYzVh8+GPBs1MngGeiIii0rmGSjS1\nNUiuNbU14FxDVT+NSF4M8EREFJVEux4eu05yzWPXQexwLVr36LkHT0REUSk7LRH6ljGw4qQvN17f\nMgbZaYm+ez4t248PTuyKygY1nMETEVFU0qiUWDDxFiRW/whtx29AYvWPsGDiLXC4nPjnqROobW6M\n6gY1nMETEVHU6niy/g97P8aGv+6BR2mB4NYCsa2S+70Narwd6gYy2Wbwx48fx9y5c5GXl4eZM2fi\nyJEjAe/bsmULbrnlFlx//fW48847UVpaKtcQiYgoCnlP1jtcTuyp3ANRZYGgQHtw9wiSe6OpQY0s\nAd7hcGDx4sWYM2cOvvjiCyxYsABLliyB1SotIXjw4EGsWbMGr7/+OkpLS3Hfffdh8eLFaGhoCPLO\nREREoTlacQ4epUVyTYSIBHUCFBCQbkjF7NzpUdOgRpYAf/DgQSgUCtx7772IjY3F3LlzYTQaUVJS\nIrmvuroaCxcuRG5uLhQKBWbPno2YmBh89913cgyTiIii2LisbChcBsk1hcuA30x9Ai9PX47V01dE\nzQE7QKY9+PLycowYMUJyLScnB2VlZZJrs2bNkjz+8ssvYbVaO72WiIiop+J1OkweOhl7Ktv34BUu\nAyYPnYzUIQkAEvp7eGEnS4C32WzQarWSaxqNBna7PehrvvvuOyxduhRLly5FUlJSXw+RiIgGgYeK\nbsd9tiIcu/A9rs4chnidrvsXDVCyLNFrtdpOwdxut0MX5IPdt28f5s2bh/nz5+OBBx6QY4hERDRI\nxOt0mDRqjC+4250ulFc2we509fPIwkuWGfzw4cOxadMmybXy8nLMmDGj071//etfsWrVKjz//PMB\nnyciIgqX3Z+fw5bi06ipsyItub0D3bT87P4eVljIMoMvKCiA0+nExo0b0dbWhm3btsFsNqOwsFBy\n34EDB/Dcc8/h7bffZnAnIqI+ZXe6sKX4NKrMVnhEoMpsxZbi01Ezk5clwKtUKmzYsAE7d+5Efn4+\nNm3ahPXr10On02HlypVYuXIlAGDDhg1oa2vDokWLcN111/n+t2fPHjmGSUREg0iV2YqaOmm6dk2d\nFVVma5BXDCyyVbIbM2YM/vznP3e6/vzzz/v++7/+67/kGg4REQ1yGUY90pL1koCelqxHhlHfj6MK\nH9aiJyKiQUmjUuKuolHIMOqhENoD/l1Fo6BRRUcV9+j4LYiIiHqhY636aAnuAGfwREQ0yHlr1XuD\ne7SkzUXPVxUiIqLLFE1pc5zBExERIfrS5hjgiYiIEH1pcwzwREREuJQ25y8tWY+khFicbaiAw+Xs\np5H1DvfgiYiIcCltzn8P/tr8Vvz6H/+OWqsZqXoj5oy9dcC0lGWAJyIiumhafjZ+MC4FRyvOYVR6\nBlb8/RU0uRoAADVWEzYf/giThk2AWqnq55F2jwGeiIjook/L9uODE7tQazEhQZOAprYGQLj0fFNb\nA841VGF0SuSfrOcePBEREQCHy4kPTuxCtaUWHoiotzdAFAXJPR67DqJ9YPSQZ4AnIiICUNVSi1qL\nSXJNEER47BqIHsDTqoO+ZQyy0xL7aYQ9wyV6IiIiABlxqUg1pKDaUuu7Fh+bCPHcJJgtjUjRG3H3\n1FwAQHllU8SXto3ckREREclIrVRhdu503x58qiEFs3Ono+D2fF+t+n1HLmDpa58NiEp3DPBEREQX\n3Tx8EiYNm4BqSy3SDam+0/I5Q+Mlle6AS5XufjAuBQ2OemTEpUbU6XoGeCIiIj9qpQrZCVmdrgeq\ndFeLb/Hk3z9Dg73eN+OPlDx5HrIjIiIKQadKdwo31FlnUWevgwciqi21+ODEroipeMcAT0REFAJv\npbsMox4KAUhN90BUdZjRW0ySQ3r9iUv0REREIZqWn43CvExUma1ISojF4x9/5at0BwBxygSkG1L7\ncYSXcAZPRETUAxqVEjlD46FWquCszIGnVefLk3dW5kD0REZo5QyeiIioF6rMVjSeS4VHSIagtkF0\n6NAmxqDKbEXO0Pj+Hh5n8ERERL3hO3TniYHYGgd4YpCWrEeGUd/9i2XAAE9ERNQLHQ/dZRjbC990\nrG7ncDn7pZ88l+iJiIh6yf/QXaDStf7d6eTOk+cMnoiIqAc6zsi9h+4Czdz9u9PJnSfPGTwREVGI\nQpmR250uVJmtaItt6NSdzpsnH6hSXrgxwBMREYXAf0YOwDcjnzRsgq8G/e7Pz2FL8WnU1FmRalQj\nblSCJE8+1ZAiW548l+iJiIhCEKhfvH/lOv9mNB4RqDY54KzMQZo+BQoISDekYnbudNka0nAGT0RE\nFIJA/eL9Z+SBmtE0nkvF8lmzoIlzSLrTyYEzeCIiohB4+8WnG1IDzsg7NaMBkJasR3ZaIrITsmRv\nJcsZPBERUYiC9YsHLuXFe/fg05ID58XLhQGeiIioB4L1iwe6z4uXEwM8ERFRGHnz4vsb9+CJiIjC\nqL9K03bEGTwREVGY9Gdp2o44gyciIgqDYKVpm2w2lFc2we50yToezuCJiIjCIFAhnGqLCY+t/xjm\nKqXvVP20/GxZxsMZPBERURh4C+H4E5x61FYL8IjthXC2FJ+WbSbPAE9ERBQGHQvhJGuS4ai4EvDE\n+O6pqbOiymwN/iZhJFuAP378OObOnYu8vDzMnDkTR44cCXjfRx99hKKiIuTl5eHBBx+E2WyWa4hE\nRESX5ebhk7B6+gq8PH05/v1Hy5GKqyTPpyXrkWHUB3l1eMkS4B0OBxYvXow5c+bgiy++wIIFC7Bk\nyRJYrdJvMSdPnsQzzzyDNWvW4ODBgzAajXjqqafkGCIREVFYeAvhxOt0uKtoFDKMeiiE9lK2cla2\nk+WnHDx4EAqFAvfeey8AYO7cuXj33XdRUlKC2267zXffjh07UFRUhGuvvRYA8Pjjj6OgoABmsxlG\no1GOoRIREYVNf1a2k2UGX15ejhEjRkiu5eTkoKysTHKtrKwMI0eO9D1OTExEfHw8ysvL5RgmERFR\n2Hkr28ldtlaWAG+z2aDVaiXXNBoN7Ha75Fprays0Go3kmlarRWtra5+PkYiIKJrIEuC1Wm2nYG63\n26HT6STXggX9jvcRERFR12QJ8MOHD++0zF5eXi5ZjgeAESNGSO6rr69HU1NTp+V9IiIi6posAb6g\noABOpxMbN25EW1sbtm3bBrPZjMLCQsl9M2bMwCeffILS0lI4HA6sWbMGkydPRmJiohzDJCIiihqy\nBHiVSoUNGzZg586dyM/Px6ZNm7B+/XrodDqsXLkSK1euBADk5ubihRdewIoVK1BQUIDa2lq89NJL\ncgyRiIgoqgiiKIr9PYhwqKioQFFREYqLi5GVldXfwyEiIupzXcU+lqolIiKKQgzwREREUYgBnoiI\nKApFTT94t9sNAKiuru7nkRAREcnDG/O8MdBf1AR4k8kEAJg/f34/j4SIiEheJpMJ2dnZkmtRc4re\nbrfj6NGjSElJQUxMTPcvICIiGuDcbjdMJhPGjRvXqdR71AR4IiIiuoSH7IiIiKIQAzwREVEUYoAn\nIiKKQgzwREREUYgBnoiIKAoxwBMREUUhBngAx48fx9y5c5GXl4eZM2fiyJEjAe/76KOPUFRUhLy8\nPJup2GoAAAbjSURBVDz44IMwm80yjzRyhfoZbtmyBbfccguuv/563HnnnSgtLZV5pJEr1M/Q68CB\nAxgzZgysVqtMI4xsoX5+paWlmD17Nq677jrcfvvtOHDggMwjjVyhfoZbt25FUVERxo8fj3vuuQdH\njx6VeaSR75tvvkFhYWHQ52WJJ+IgZ7fbxRtvvFF87733RKfTKW7dulX84Q9/KFosFsl9J06cEK+/\n/nrxyJEjYmtrq7h8+XLx3/7t3/pp1JEl1M/wwIED4sSJE8Xjx4+LbrdbfP/998Xx48eL9fX1/TTy\nyBHqZ+jV2NgoTpkyRRw9enTQewaTUD+/6upqccKECeLf/vY30ePxiDt27BDHjx8vtra29tPII0dP\n/i3Mz88Xy8rKRLfbLb711lvi1KlT+2nUkcfj8Yhbt24Vx48fL+bn5we8R654Muhn8AcPHoRCocC9\n996L2NhYzJ07F0ajESUlJZL7duzYgaKiIlx77bXQaDR4/PHHsXfvXs7iEfpnWF1djYULFyI3NxcK\nhQKzZ89GTEwMvvvuu34aeeQI9TP0evbZZ3HbbbfJPMrIFernt337dkyaNAnTp0+HIAiYMWMG3n33\nXSgUg/6fwpA/w3PnzsHj8cDtdkMURSgUik4V1AazN998E3/605+wePHioPfIFU8G/d/q8vJyjBgx\nQnItJycHZWVlkmtlZWUYOXKk73FiYiLi4+NRXl4uyzgjWaif4axZs7Bo0SLf4y+//BJWq7XTawej\nUD9DAPjwww/R3NyMefPmyTW8iBfq53fs2DGkpaXhoYcewsSJE3H33XfD7XZDpVLJOdyIFOpnWFhY\niCuvvBI/+clPcM011+Ctt97Cq6++KudQI9qdd96J7du345prrgl6j1zxZNAHeJvNBq1WK7mm0Whg\nt9sl11pbWzt9S9VqtWhtbe3zMUa6UD9Df9999x2WLl2KpUuXIikpqa+HGPFC/QwrKyvx+uuv48UX\nX5RzeBEv1M+vqakJW7duxbx587Bv3z7ccccdeOCBB9DU1CTncCNSqJ+hw+HAyJEjsW3bNhw+fBj3\n338/fv7zn3f5//fBJDU1FYIgdHmPXPFk0Ad4rVbb6S+m3W6HTqeTXAsW9DveNxiF+hl67du3D/Pm\nzcP8+fPxwAMPyDHEiBfKZ+jxePDEE0/g0UcfRVpamtxDjGih/h1UqVSYPHkyCgsLERsbi/nz50On\n0+Grr76Sc7gRKdTP8He/+x3S09NxzTXXQK1W46GHHkJbWxv2798v53AHNLniyaAP8MOHD++0LFJe\nXi5ZPgGAESNGSO6rr69HU1MTl5cR+mcIAH/961+xdOlSPPPMM/h//+//yTXEiBfKZ1hdXY2vv/4a\nzz77LCZMmIA77rgDAHDTTTcN+myEUP8O5uTkwOl0Sq55PB6I7LkV8mdYWVkp+QwFQUBMTAy7ePaA\nXPFk0Af4goICOJ1ObNy4EW1tbdi2bRvMZnOn9IYZM2bgk08+QWlpKRwOB9asWYPJkycjMTGxn0Ye\nOUL9DA8cOIDnnnsOb7/9NmbMmNFPo41MoXyGQ4cOxTfffIPS0lKUlpbiww8/BACUlJRgwoQJ/TX0\niBDq38GZM2di3759+Oyzz+DxeLBx40Y4HA5MnDixn0YeOUL9DKdMmYJt27bh2LFjcLlceOedd+B2\nuzF+/Ph+GvnAI1s8Cfu5/AHoxIkT4t133y3m5eWJM2fOFA8fPiyKoig+/fTT4tNPP+27b+fOneIt\nt9wiXnfddeKiRYtEs9ncX0OOOKF8hj/72c/EMWPGiHl5eZL/lZSU9OfQI0aofw+9zp8/zzQ5P6F+\nfnv37hVnzpwp5uXlibNnzxaPHDnSX0OOOKF8hh6PR3zrrbfEm2++WRw/frx43333id9++21/Djsi\nHTx4UJIm1x/xhP3giYiIotCgX6InIiKKRgzwREREUYgBnoiIKAoxwBMREUUhBngiIqIoxABPREQU\nhRjgiYiIohADPBERURRigCeiXnvllVckPQVefvll3H///Z3qvROR/JT9PQAiGrgWLVqEadOm4fjx\n4/j666+xb98+bN68mf3ViSIAS9US0WV544038Mknn8BisWDz5s3IyMjo7yEREbhET0SXKTc3F6dO\nncJjjz3G4E4UQTiDJ6Je+/bbb7Fw4UIUFhaiuroaf/zjH/t7SER0EWfwRNQrNTU1WLJkCZ577jk8\n88wzOHXqFA4dOtTfwyKiixjgiajHLBYLFi1ahH/5l39BUVERtFotFi5ciN/+9rf9PTQiuohL9ERE\nRFGIM3giIqIoxABPREQUhRjgiYiIohADPBERURRigCciIopCDPBERERRiAGeiIgoCjHAExH9/42C\nUTAMAQBapSaODGO7xgAAAABJRU5ErkJggg==\n", 301 | "text/plain": [ 302 | "" 303 | ] 304 | }, 305 | "metadata": {}, 306 | "output_type": "display_data" 307 | } 308 | ], 309 | "source": [ 310 | "# Initialize the plot\n", 311 | "fig = plt.figure(figsize=(8,5))\n", 312 | "##########\n", 313 | "ax3 = fig.add_subplot(1, 1, 1)\n", 314 | "lines2 = ax3.scatter(x_fipy[:,None], u_fipy[:,None],s=25, cmap='Greens',label='ground truth')\n", 315 | "lines3 = ax3.scatter(x_fipy[:,None], u_sensor[:,None],s=25,cmap='Greens',label='sensor')\n", 316 | "ax3.set_xlabel('$x$', fontsize=12)\n", 317 | "ax3.set_ylabel(r'$U$', fontsize=12)\n", 318 | "plt.legend(loc='best')\n", 319 | "plt.show()\n", 320 | "##########" 321 | ] 322 | }, 323 | { 324 | "cell_type": "code", 325 | "execution_count": 13, 326 | "metadata": {}, 327 | "outputs": [ 328 | { 329 | "name": "stdout", 330 | "output_type": "stream", 331 | "text": [ 332 | "(0.20500000000000002, 0.305, 0.405, 0.505, 0.605, 0.705, 0.805)\n" 333 | ] 334 | } 335 | ], 336 | "source": [ 337 | "print (x_fipy[20],x_fipy[30],x_fipy[40],x_fipy[50],x_fipy[60],x_fipy[70],x_fipy[80])" 338 | ] 339 | }, 340 | { 341 | "cell_type": "code", 342 | "execution_count": 14, 343 | "metadata": {}, 344 | "outputs": [ 345 | { 346 | "name": "stdout", 347 | "output_type": "stream", 348 | "text": [ 349 | "(0.7633660270634811, 0.7590536094026412, 0.6755348939987039, 0.6433442468899135, 0.6137597421640841, 0.5772112819430778, 0.4909199041904633)\n" 350 | ] 351 | } 352 | ], 353 | "source": [ 354 | "print (u_sensor[20],u_sensor[30],u_sensor[40],u_sensor[50],u_sensor[60],u_sensor[70],u_sensor[80])" 355 | ] 356 | }, 357 | { 358 | "cell_type": "code", 359 | "execution_count": null, 360 | "metadata": { 361 | "collapsed": true 362 | }, 363 | "outputs": [], 364 | "source": [] 365 | } 366 | ], 367 | "metadata": { 368 | "kernelspec": { 369 | "display_name": "Python 2", 370 | "language": "python", 371 | "name": "python2" 372 | }, 373 | "language_info": { 374 | "codemirror_mode": { 375 | "name": "ipython", 376 | "version": 2 377 | }, 378 | "file_extension": ".py", 379 | "mimetype": "text/x-python", 380 | "name": "python", 381 | "nbconvert_exporter": "python", 382 | "pygments_lexer": "ipython2", 383 | "version": "2.7.13" 384 | } 385 | }, 386 | "nbformat": 4, 387 | "nbformat_minor": 2 388 | } 389 | -------------------------------------------------------------------------------- /scripts/1d_spde_prob/inverse_problem/input_field_ground_truth.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PredictiveScienceLab/variational-elliptic-SPDE/967d8b6e5a4fb123eef199c86b7ec441814b08b0/scripts/1d_spde_prob/inverse_problem/input_field_ground_truth.npy -------------------------------------------------------------------------------- /scripts/1d_spde_prob/inverse_problem/my_model_weights.h5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PredictiveScienceLab/variational-elliptic-SPDE/967d8b6e5a4fb123eef199c86b7ec441814b08b0/scripts/1d_spde_prob/inverse_problem/my_model_weights.h5 -------------------------------------------------------------------------------- /scripts/1d_spde_prob/inverse_problem/posterior_likscale=0.032.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PredictiveScienceLab/variational-elliptic-SPDE/967d8b6e5a4fb123eef199c86b7ec441814b08b0/scripts/1d_spde_prob/inverse_problem/posterior_likscale=0.032.pdf -------------------------------------------------------------------------------- /scripts/1d_spde_prob/inverse_problem/ufipy_ground_truth.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PredictiveScienceLab/variational-elliptic-SPDE/967d8b6e5a4fb123eef199c86b7ec441814b08b0/scripts/1d_spde_prob/inverse_problem/ufipy_ground_truth.npy -------------------------------------------------------------------------------- /scripts/1d_spde_prob/inverse_problem/xfipy_ground_truth.npy: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PredictiveScienceLab/variational-elliptic-SPDE/967d8b6e5a4fb123eef199c86b7ec441814b08b0/scripts/1d_spde_prob/inverse_problem/xfipy_ground_truth.npy -------------------------------------------------------------------------------- /scripts/2d_spde_prob/2d_spde.py: -------------------------------------------------------------------------------- 1 | from __future__ import division 2 | import argparse 3 | 4 | ################################################################# 5 | # ====================== 6 | ## note the code is based on cell centers 7 | # ====================== 8 | #parse command line arguments 9 | parser = argparse.ArgumentParser() 10 | parser.add_argument('-train_data', dest = 'train_data', type = str, 11 | default = 'data/train_exp_nx1=32_nx2=32_lx1=0.05_lx2=0.08_v=0.75_num_samples=10000.npy', help = 'Training data file.') 12 | parser.add_argument('-test_data', dest = 'test_data', type = str, 13 | default = 'data/test_exp_nx1=32_nx2=32_lx1=0.05_lx2=0.08_v=0.75_num_samples=2000.npy', help = 'Testing data file.') 14 | parser.add_argument('-nx1', dest = 'nx1', type = int, 15 | default = 32, help = 'Number of FV cells in the x1 direction.')# number of cells/cellcenters/pixels/pixelcenters in x1-direction 16 | parser.add_argument('-nx2', dest = 'nx2', type = int, 17 | default = 32, help = 'Number of FV cells in the x2 direction.')# number of cells/cellcenters/pixels/pixelcenters in x2-direction 18 | 19 | 20 | parser.add_argument('-DNN_type', dest = 'DNN_type', type = str, 21 | default = 'Resnet', help = 'Type of DNN (Resnet:Residual network, FC:Fully connected network).') 22 | parser.add_argument('-n', dest = 'n', type = int, 23 | default = 64, help = 'Number of neurons in each block.') 24 | parser.add_argument('-num_block', dest = 'num_block', type = int, 25 | default = 1, help = 'Number of blocks.') 26 | parser.add_argument('-d', dest = 'd', type = str, 27 | default = '[5,5]', help = 'Number of neurons per layer.') 28 | parser.add_argument('-act_func', dest = 'act_func', type = str, 29 | default = 'swish', help = 'Activation function.') 30 | 31 | 32 | parser.add_argument('-loss_type', dest = 'loss_type', type = str, 33 | default = 'EF', help = 'Type of Loss to use for training (EF: Energy Functional, SR: Squared Residual).') 34 | parser.add_argument('-lr', dest = 'lr', type = float, 35 | default = 0.001, help = 'Learning rate.') 36 | parser.add_argument('-max_it', dest = 'max_it', type = int, 37 | default = 1000, help = 'Maximum number of iterations.') 38 | parser.add_argument('-M_A', dest = 'M_A', type = int, 39 | default = 10, help = 'Batch size: number of input field images in each iteration.') 40 | parser.add_argument('-M_x', dest = 'M_x', type = int, 41 | default = 10, help = 'Batch size: number of x-samples=[x1,x2] on each of the sampled image in each iteration.') ## M_x cannot be greater than nx1*nx2 (FOR THIS CODE) 42 | 43 | 44 | parser.add_argument('-seed', dest = 'seed', type = int, 45 | default = 0, help = 'Random seed number.') # seed for reproducability 46 | parser.add_argument('-variation', dest = 'variation', type = str, 47 | default = 'a', help = 'Model variation currently trying.') 48 | 49 | args = parser.parse_args() 50 | 51 | ################################################################# 52 | import matplotlib 53 | matplotlib.use('PS') 54 | import tensorflow as tf 55 | import random 56 | import numpy as np 57 | import os 58 | os.environ['PYTHONHASHSEED'] = '0' 59 | 60 | seed = args.seed 61 | # Setting the seed for numpy-generated random numbers 62 | np.random.seed(seed=seed) 63 | 64 | # Setting the seed for python random numbers 65 | random.seed(seed) 66 | 67 | # Setting the graph-level random seed. 68 | tf.set_random_seed(seed) 69 | 70 | os.environ['KERAS_BACKEND'] = 'tensorflow' 71 | from keras.models import Model 72 | from keras.layers import Dense, Activation, Input, concatenate, Lambda, Add 73 | from keras.utils import plot_model 74 | from keras import backend as K 75 | from keras.utils.generic_utils import get_custom_objects 76 | 77 | import matplotlib.pyplot as plt 78 | import GPy 79 | from fipy import * 80 | # import matplotlib as mpl 81 | # mpl.rcParams['figure.dpi']= 300 82 | import seaborn as sns 83 | sns.set_context('talk') 84 | sns.set_style('white') 85 | from pdb import set_trace as keyboard 86 | import sys 87 | import time 88 | ################################################################# 89 | # ------------------------------------------------------------ 90 | 91 | # loading data 92 | train_data = np.load(os.path.join(os.getcwd(),args.train_data)) 93 | test_data = np.load(os.path.join(os.getcwd(),args.test_data)) 94 | 95 | # bounding input fields from below and above 96 | lower_bound = np.exp(-5.298317366548036) # 0.005000000000000002 97 | upper_bound = np.exp(3.5) # 33.11545195869231 98 | 99 | train_data = np.where(train_data < lower_bound, lower_bound,train_data) 100 | train_data = np.where(train_data > upper_bound, upper_bound, train_data) 101 | 102 | test_data = np.where(test_data < lower_bound, lower_bound,test_data) 103 | test_data = np.where(test_data > upper_bound, upper_bound, test_data) 104 | 105 | # ------------------------------------------------------------ 106 | 107 | nx1 = args.nx1 108 | nx2 = args.nx2 109 | 110 | DNN_type = args.DNN_type 111 | n = args.n 112 | num_block = args.num_block 113 | d = args.d 114 | act_func = args.act_func 115 | 116 | loss_type = args.loss_type 117 | lr = args.lr 118 | max_it = args.max_it 119 | M_A = args.M_A 120 | M_x = args.M_x 121 | 122 | variation = args.variation 123 | 124 | # ------------------------------------------------------------ 125 | # needs to be defined as activation class otherwise error 126 | # AttributeError: 'Activation' object has no attribute '__name__' 127 | class Swish(Activation): 128 | 129 | def __init__(self, activation, **kwargs): 130 | super(Swish, self).__init__(activation, **kwargs) 131 | self.__name__ = 'swish' 132 | 133 | def swish(x): 134 | return (K.sigmoid(x) * x) 135 | 136 | get_custom_objects().update({'swish': Swish(swish)}) 137 | # ------------------------------------------------------------ 138 | # BUILDING DNN APPROXIMATOR 139 | # ====================== 140 | # DNN network i/p:x1,x2,A o/p:prediction 141 | 142 | x1 = Input(shape=(1,)) 143 | x2 = Input(shape=(1,)) 144 | A = Input(shape=(nx1*nx2,)) # input field image: conductivity image 145 | a_val = Input(shape=(1,)) # input field value: conductivity value at the corresponding 'input(x1,x2)' location 146 | 147 | if DNN_type == 'Resnet': 148 | x1_x2_A = concatenate([x1,x2,A]) 149 | o = Dense(n)(x1_x2_A) 150 | for i in range(num_block): 151 | z = Dense(n, activation = act_func)(o) 152 | z = Dense(n, activation = act_func)(z) 153 | o = Add()([z, o]) 154 | prediction = Dense(1)(o) 155 | print DNN_type 156 | 157 | elif DNN_type == 'FC': 158 | num_neurons_per_layer = map(int, d.strip('[]').split(',')) 159 | x1_x2_A = concatenate([x1,x2,A]) 160 | z = Dense(num_neurons_per_layer[0], activation=act_func)(x1_x2_A) 161 | for n in num_neurons_per_layer[1:]: 162 | z = Dense(n, activation=act_func)(z) 163 | prediction = Dense(1)(z) 164 | print DNN_type 165 | 166 | def myFunc(t): 167 | return ((1-t[0])+(t[0]*(1-t[0])*t[2])) # dirichlet on left and right sides of plate 168 | 169 | u = Lambda(myFunc, output_shape=(1,))([x1,x2,prediction]) # field of interest : temperature 170 | model = Model(inputs=[x1,x2,A], outputs=u) 171 | 172 | # BUILDING LOSS FUNCTIONS 173 | # ====================== 174 | 175 | dudx1 = tf.gradients(u, x1)[0] 176 | dudx2 = tf.gradients(u, x2)[0] 177 | 178 | tf_a = a_val 179 | 180 | c = 0. 181 | f = 0. # source 182 | 183 | # loss function 184 | # ====================== 185 | if loss_type == 'EF': 186 | term_1 = 0.5 * ((tf_a * (dudx1 ** 2 + dudx2 ** 2)) + (c*u*u)) 187 | V = term_1 - (f * u) 188 | ef_loss = tf.reduce_sum(V)/(M_x * M_A) 189 | #or 190 | # ef_loss = tf.reduce_mean(V) 191 | loss = ef_loss 192 | print('loss energy functional form') 193 | # create directories 194 | resultdir = os.path.join(os.getcwd(), 'results','loss_EF_form','DNN_type='+str(DNN_type)+'_nx1*nx2='+str(nx1*nx2)+'_seed='+str(seed)+'_'+str(variation)) 195 | 196 | elif loss_type == 'SR': 197 | residual = -(tf.gradients(tf_a * dudx1, x1)[0]) - (tf.gradients(tf_a * dudx2, x2)[0]) + (c*u) - f 198 | sqresi_loss = tf.reduce_sum(tf.square(residual))/(M_x * M_A) 199 | #or 200 | # sqresi_loss = tf.reduce_mean(tf.square(residual)) 201 | loss = sqresi_loss 202 | print('loss squared residual form') 203 | # create directories 204 | resultdir = os.path.join(os.getcwd(), 'results','loss_SR_form','DNN_type='+str(DNN_type)+'_nx1*nx2='+str(nx1*nx2)+'_seed='+str(seed)+'_'+str(variation)) 205 | 206 | if not os.path.exists(resultdir): 207 | os.makedirs(resultdir) 208 | 209 | # ====================== 210 | orig_stdout = sys.stdout 211 | q = open(os.path.join(resultdir, 'loss_output='+str(DNN_type)+'_'+str(nx1*nx2)+'_'+str(seed)+'_'+str(variation)+'.txt'), 'w') 212 | sys.stdout = q 213 | start = time.time() 214 | print ("------START------") 215 | print args.train_data 216 | print args.test_data 217 | 218 | if DNN_type == 'Resnet': 219 | print (nx1,nx2,DNN_type,n,num_block,act_func,loss_type,lr,max_it,M_A,M_x,seed,variation) 220 | elif DNN_type == 'FC': 221 | print (nx1,nx2,DNN_type,num_neurons_per_layer,act_func,loss_type,lr,max_it,M_A,M_x,seed,variation) 222 | 223 | plot_model(model, to_file=os.path.join(resultdir,'stoch_heq_nn_fipy.pdf')) 224 | # ====================== 225 | 226 | train = tf.train.AdamOptimizer(lr).minimize(loss) 227 | 228 | init = tf.global_variables_initializer() 229 | sess = tf.Session() 230 | K.set_session(sess) 231 | sess.run(init) 232 | 233 | I=[] 234 | Loss=[] 235 | 236 | weights = sess.run(model.weights) 237 | # print weights 238 | w=[] 239 | [w.extend(weights[j].flatten()) for j in range(len(weights))] 240 | # print len(w) 241 | plt.hist(w, bins=20) 242 | plt.title('Histogram_Weights&biases_all_layers_before_training') 243 | plt.savefig(os.path.join(resultdir,'Histogram_Weights&biases_all_layers_before_training.pdf')) 244 | plt.pause(1) 245 | plt.close() 246 | 247 | # ====================== 248 | print ("--------------------------------------------------------------") 249 | 250 | #defining mesh to get cellcenters 251 | Lx1 = 1. # always put . after 1 252 | Lx2 = 1. # always put . after 1 253 | mesh = Grid2D(nx=nx1, ny=nx2, dx=Lx1/nx1, dy=Lx2/nx2) # with nx1*nx2 number of cells/cellcenters/pixels/pixelcenters 254 | cellcenters = mesh.cellCenters.value.T # (nx1*nx2,2) matrix 255 | # print cellcenters 256 | 257 | gridnum_list = [i for i in range(nx1*nx2)] # grid at cell centers 258 | # print gridnum_list 259 | # print np.shape(gridnum_list) 260 | # print type(gridnum_list) 261 | print ('*****') 262 | 263 | print ("--------------------------------------------------------------") 264 | 265 | for i in range(max_it): 266 | # Get a batch of points 267 | X1i_final=np.zeros((1, 1)) # sampled x's X1 coordinates 268 | X2i_final=np.zeros((1, 1)) # sampled x's X2 coordinates 269 | AAi_final = np.zeros((1, cellcenters.shape[0])) # images of input field: conductivity # or np.zeros((1, nx1*nx2)) 270 | Ai_val_final = np.zeros((1, 1)) # input field values at sampled x's 271 | 272 | for t in xrange(M_A): 273 | 274 | # sampling grid locations from gridnum_list 275 | gridnum_sam = np.random.choice(gridnum_list, size=M_x, replace=False, p=None) # p: The probabilities associated with each entry in a.\ 276 | # If not given the sample assumes a uniform distribution over all entries in a. 277 | # replace: Whether the sample is with or without replacement 278 | # print gridnum_sam 279 | # print type(gridnum_sam) 280 | # print np.shape(gridnum_sam) 281 | 282 | # sampled x's coordinates 283 | Xi = np.ndarray((M_x, 2)).astype(np.float32) 284 | for j in range(M_x): 285 | Xi[j, :] = cellcenters[gridnum_sam[j],:] 286 | # print Xi 287 | # print ('########') 288 | X1i_final = np.vstack((X1i_final,Xi[:,0][:,None])) 289 | X2i_final = np.vstack((X2i_final,Xi[:,1][:,None])) 290 | 291 | # getting input field images 292 | Ai = train_data[ random.randint(0, np.shape(train_data)[0]-1), : ].reshape(1,-1) 293 | # Ai is one image of input field: conductivity of nx1*nx2 cells/cellcenters/pixels/pixelcenters picked from train_data # returns (1 , nx1*nx2) matrix 294 | # print Ai 295 | # print ('########') 296 | AAi = np.repeat(Ai, np.shape(Xi)[0], axis=0) # just repeating 297 | AAi_final = np.vstack((AAi_final,AAi)) 298 | 299 | # getting input field values at sampled x's to use them in loss function calculations 300 | Ai_val = np.ndarray((M_x, 1)).astype(np.float32) # or np.ndarray((np.shape(Xi)[0], 1)).astype(np.float32) 301 | for g in range(M_x): # or for g in range(np.shape(Xi)[0]): 302 | Ai_val[g,0] = Ai[0,gridnum_sam[g]] 303 | # print Ai_val 304 | # print ('#################################') 305 | Ai_val_final = np.vstack((Ai_val_final,Ai_val)) 306 | 307 | X1i_final=np.delete(X1i_final, (0), axis=0) #to delete the first row 308 | X2i_final=np.delete(X2i_final, (0), axis=0) #to delete the first row 309 | AAi_final = np.delete(AAi_final, (0), axis=0) #to delete the first row 310 | Ai_val_final = np.delete(Ai_val_final, (0), axis=0) #to delete the first row 311 | 312 | # print X1i_final 313 | # print X2i_final 314 | # print AAi_final 315 | # print Ai_val_final 316 | # print ('done') 317 | 318 | sess.run(train, feed_dict={x1:X1i_final, x2:X2i_final, A:AAi_final, a_val:Ai_val_final}) 319 | l = sess.run(loss, feed_dict={x1:X1i_final, x2:X2i_final, A:AAi_final, a_val:Ai_val_final}) 320 | 321 | I.append(i) 322 | Loss.append(l) 323 | 324 | # display 325 | if i % 500 == 0: 326 | print ("Iteration: "+str(i)+"; Train loss:"+str(l)+";") 327 | # weights = sess.run(model.weights) 328 | # w=[] 329 | # [w.extend(weights[j].flatten()) for j in range(len(weights))] 330 | # # print len(w) 331 | # plt.hist(w, bins=20) 332 | # plt.title('Iteration:='+str(i)+'_ Histogram_Weights&biases_all_layers') 333 | # plt.savefig(os.path.join(resultdir,'Iteration='+str(i)+'_ Histogram_Weights&biases_all_layers.pdf')) 334 | # plt.pause(1) 335 | # plt.close() 336 | # # keyboard() 337 | print ("--------------------------------------------------------------") 338 | 339 | weights = sess.run(model.weights) 340 | # print weights 341 | w=[] 342 | [w.extend(weights[j].flatten()) for j in range(len(weights))] 343 | # print len(w) 344 | plt.hist(w, bins=20) 345 | plt.title('Histogram_Weights&biases_all_layers_after_training') 346 | plt.savefig(os.path.join(resultdir,'Histogram_Weights&biases_all_layers_after_training.pdf')) 347 | plt.pause(1) 348 | plt.close() 349 | model.summary() 350 | model.save(os.path.join(resultdir,'my_model.h5')) 351 | model.save_weights(os.path.join(resultdir,'my_model_weights.h5')) 352 | 353 | plt.plot(I, Loss, 'blue', lw=1.5, label='Iteration_vs_Trainloss') 354 | plt.xlabel('Iteration') 355 | plt.ylabel('Trainloss') 356 | plt.title('DNN_type='+str(DNN_type)+'_nx1*nx2='+str(nx1*nx2)+'_seed='+str(seed)+'_'+str(variation)+'_ Iteration Vs Trainloss') 357 | plt.savefig(os.path.join(resultdir,'DNN_type='+str(DNN_type)+'_nx1*nx2='+str(nx1*nx2)+'_seed='+str(seed)+'_'+str(variation)+'_ Iteration Vs Trainloss.pdf')) 358 | plt.tight_layout() 359 | plt.legend(loc='best'); 360 | plt.pause(5) 361 | plt.close() 362 | 363 | np.save(os.path.join(resultdir,'I.npy'), np.asarray(I)) 364 | np.save(os.path.join(resultdir,'Loss.npy'), np.asarray(Loss)) 365 | # np.load(os.path.join(resultdir,'I.npy')) 366 | # np.load(os.path.join(resultdir,'Loss.npy')) 367 | 368 | # end timer 369 | finish = time.time() - start # time for network to train 370 | 371 | # TESTING(checking NN solution against fipy solution) 372 | # ====================== 373 | # test cases 374 | nsamples = np.shape(test_data)[0] 375 | # validation error and relative RMS error 376 | val = [] 377 | rel_RMS_num = [] 378 | rel_RMS_den = [] 379 | # get all relative errrors and r2 scores 380 | relerrors = [] 381 | r2scores = [] 382 | # get all things for plots 383 | samples_inputfield = np.zeros((nsamples, nx1*nx2)) 384 | samples_u_DNN = np.zeros((nsamples, nx1*nx2)) 385 | samples_u_fipy = np.zeros((nsamples, nx1*nx2)) 386 | 387 | cmap = matplotlib.colors.LinearSegmentedColormap.from_list("", ["navy","blue","deepskyblue","limegreen","yellow","darkorange","red","maroon"]) 388 | 389 | np.random.seed(23) 390 | for i in range(nsamples): # test cases 391 | ############################################################### 392 | # FIPY solution 393 | value_left=1. 394 | value_right=0. 395 | value_top=0. 396 | value_bottom=0. 397 | 398 | Lx1 = 1. # always put . after 1 399 | Lx2 = 1. # always put . after 1 400 | 401 | # define mesh 402 | mesh = Grid2D(nx=nx1, ny=nx2, dx=Lx1/nx1, dy=Lx2/nx2) # with nx1*nx2 number of cells/cellcenters/pixels/pixelcenters 403 | 404 | # define cell and face variables 405 | phi = CellVariable(name='$T(x)$', mesh=mesh, value=0.) 406 | D = CellVariable(name='$D(x)$', mesh=mesh, value=1.0) ## coefficient in diffusion equation 407 | # D = FaceVariable(name='$D(x)$', mesh=mesh, value=1.0) ## coefficient in diffusion equation 408 | source = CellVariable(name='$f(x)$', mesh=mesh, value=1.0) 409 | C = CellVariable(name='$C(x)$', mesh=mesh, value=1.0) 410 | 411 | # apply boundary conditions 412 | # dirichet 413 | phi.constrain(value_left, mesh.facesLeft) 414 | phi.constrain(value_right, mesh.facesRight) 415 | 416 | # homogeneous Neumann 417 | phi.faceGrad.constrain(value_top, mesh.facesTop) 418 | phi.faceGrad.constrain(value_bottom, mesh.facesBottom) 419 | 420 | # setup the diffusion problem 421 | eq = -DiffusionTerm(coeff=D)+ImplicitSourceTerm(coeff=C) == source 422 | 423 | source.setValue(f) 424 | C.setValue(c) 425 | 426 | # getting input field images 427 | a = test_data[ i , : ].reshape(-1,1) 428 | # 'a' is one image of input field: conductivity image of nx1*nx2 cells/cellcenters/pixels/pixelcenters from test_data # returns (nx1*nx2,1) matrix 429 | D.setValue(a.ravel()) 430 | 431 | eq.solve(var=phi) 432 | x_fipy = mesh.cellCenters.value.T ## fipy solution (nx1*nx2,2) matrix # same as cellcenters defined above 433 | u_fipy = phi.value[:][:, None] ## fipy solution (nx1*nx2,1) matrix 434 | 435 | x1_f = x_fipy[:,0][:,None] # x1-coordinates of cell centers (nx1*nx2,1) matrix 436 | x2_f = x_fipy[:,1][:,None] # x2-coordinates of cell centers (nx1*nx2,1) matrix 437 | # print ('done1') 438 | ############################################################### 439 | #Neuralnet solution 440 | u_DNN = sess.run(u, feed_dict={x1:x1_f, x2:x2_f, A:np.repeat(a.T, np.shape(x1_f)[0], axis=0)}) 441 | # print ('done2') 442 | ################################################################# 443 | val.append(np.sum((u_fipy-u_DNN)**2, axis=0)) 444 | rel_RMS_num.append(np.sum((u_fipy-u_DNN)**2, axis=0)) 445 | rel_RMS_den.append(np.sum((u_fipy)**2, axis=0)) 446 | ############################################################### 447 | from sklearn import metrics 448 | r2score = metrics.r2_score(u_fipy.flatten(), u_DNN.flatten()) 449 | relerror = np.linalg.norm(u_fipy.flatten() - u_DNN.flatten()) / np.linalg.norm(u_fipy.flatten()) 450 | r2score = float('%.4f'%r2score) 451 | relerror = float('%.4f'%relerror) 452 | relerrors.append(relerror) 453 | r2scores.append(r2score) 454 | ############################################################### 455 | samples_inputfield[i] = a.ravel() 456 | samples_u_DNN[i] = u_DNN.flatten() 457 | samples_u_fipy[i] = u_fipy.flatten() 458 | ############################################################### 459 | if i<=20: 460 | # Initialize the plot 461 | fig = plt.figure(figsize=(18,6)) 462 | 463 | try: 464 | ax1.lines.remove(c1[0]) 465 | ax2.lines.remove(c2[0]) 466 | ax3.lines.remove(c3[0]) 467 | 468 | except: 469 | pass 470 | 471 | ax1 = fig.add_subplot(1, 3, 1) 472 | c1 = ax1.contourf( x1_f.reshape((nx1,nx2)), x2_f.reshape((nx1,nx2)), np.log(a).reshape((nx1,nx2)), 100, cmap=cmap) # set levels automatically 473 | # This is the fix for the white lines between contour levels (https://stackoverflow.com/questions/8263769/hide-contour-linestroke-on-pyplot-contourf-to-get-only-fills) 474 | for j in c1.collections: 475 | j.set_edgecolor("face") 476 | plt.colorbar(c1) 477 | plt.title('$log(Input \ field)$', fontsize=14) 478 | plt.xlabel('$x1$', fontsize=12) 479 | plt.ylabel('$x2$', fontsize=12) 480 | 481 | 482 | ax2 = fig.add_subplot(1, 3, 2) 483 | c2 = ax2.contourf( x1_f.reshape((nx1,nx2)), x2_f.reshape((nx1,nx2)), u_fipy.reshape((nx1,nx2)), 100, cmap=cmap) # set levels as previous levels 484 | # This is the fix for the white lines between contour levels (https://stackoverflow.com/questions/8263769/hide-contour-linestroke-on-pyplot-contourf-to-get-only-fills) 485 | for j in c2.collections: 486 | j.set_edgecolor("face") 487 | plt.colorbar(c2) 488 | plt.title('$FVM \ solution$',fontsize=14) 489 | plt.xlabel('$x1$', fontsize=12) 490 | plt.ylabel('$x2$', fontsize=12) 491 | 492 | 493 | ax3 = fig.add_subplot(1, 3, 3) 494 | c3 = ax3.contourf( x1_f.reshape((nx1,nx2)), x2_f.reshape((nx1,nx2)), u_DNN.reshape((nx1,nx2)), 100, cmap=cmap) # set levels automatically 495 | # This is the fix for the white lines between contour levels (https://stackoverflow.com/questions/8263769/hide-contour-linestroke-on-pyplot-contourf-to-get-only-fills) 496 | for j in c3.collections: 497 | j.set_edgecolor("face") 498 | plt.colorbar(c3) 499 | plt.title('$DNN \ solution: \ Rel. L_2 \ Error$ = '+str(relerror)+', $R^{2}$ = '+str(r2score), fontsize=14) 500 | plt.xlabel('$x1$', fontsize=12) 501 | plt.ylabel('$x2$', fontsize=12) 502 | 503 | plt.tight_layout() 504 | # plt.suptitle('test_case='+str(i+1)+'_DNN_type='+str(DNN_type)+'_nx1*nx2='+str(nx1*nx2)+'_seed='+str(seed)+'_'+str(variation), fontsize=12) 505 | plt.savefig(os.path.join(resultdir,'test_case='+str(i+1)+'_DNN_type='+str(DNN_type)+'_nx1*nx2='+str(nx1*nx2)+'_seed='+str(seed)+'_'+str(variation)+'_nnpred-fipy.pdf')) 506 | plt.show() 507 | plt.pause(0.1) 508 | print i 509 | print ("--------------------------------------------------------------") 510 | #################################################################################################################### 511 | plt.close('all') 512 | # https://stats.stackexchange.com/questions/189783/calculating-neural-network-error 513 | # print val 514 | vali_error = np.sum(val)/(np.shape(val)[0]*np.shape(x_fipy)[0]) 515 | print ('validation_error='+str(vali_error)) 516 | 517 | # https://www.rocq.inria.fr/modulef/Doc/GB/Guide6-10/node21.html 518 | rel_RMS_error = np.sqrt(np.sum(rel_RMS_num)/np.sum(rel_RMS_den)) 519 | print ('relative_RMS_error='+str(rel_RMS_error)) 520 | 521 | 522 | np.save(os.path.join(resultdir,'cellcenters.npy'), cellcenters) # or x_fipy 523 | 524 | np.save(os.path.join(resultdir,'samples_inputfield.npy'), samples_inputfield) 525 | np.save(os.path.join(resultdir,'samples_u_DNN.npy'), samples_u_DNN) 526 | np.save(os.path.join(resultdir,'samples_u_fipy.npy'), samples_u_fipy) 527 | 528 | relerrors = np.array(relerrors) 529 | r2scores = np.array(r2scores) 530 | 531 | np.save(os.path.join(resultdir,'relerrors.npy'), relerrors) 532 | np.save(os.path.join(resultdir,'r2scores.npy'), r2scores) 533 | 534 | #plt.figure(figsize=(8, 6)) 535 | plt.hist(relerrors, alpha = 0.7, bins = 100, normed=True, label='Histogram of Rel. $L_2$ Error') 536 | plt.tight_layout() 537 | plt.legend(loc = 'best', fontsize = 14) 538 | plt.savefig(os.path.join(resultdir,'rel_errors_hist.pdf')) 539 | plt.close() 540 | 541 | plt.hist(r2scores, alpha = 0.7, bins = 100, normed=True, label='Histogram of $R^2$') 542 | plt.tight_layout() 543 | plt.legend(loc = 'best', fontsize = 14) 544 | plt.savefig(os.path.join(resultdir,'r2scores_hist.pdf')) 545 | plt.close() 546 | 547 | print "Time (sec) to complete: " +str(finish) # time for network to train 548 | print ("------END------") 549 | sys.stdout = orig_stdout 550 | q.close() 551 | ############################################################### 552 | -------------------------------------------------------------------------------- /scripts/2d_spde_prob/generate_MC_data/generate_MC_data_a=e^GRF_bounded.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | from __future__ import division 3 | import argparse 4 | import numpy as np 5 | import os 6 | import GPy 7 | import matplotlib.pyplot as plt 8 | from fipy import * 9 | from scipy.interpolate import griddata 10 | from pdb import set_trace as keyboard 11 | import time 12 | import random 13 | 14 | #parse command line arguments 15 | parser = argparse.ArgumentParser() 16 | parser.add_argument('-N', dest = 'N', type = int, 17 | default = 10000, help = 'Number of MC samples.') 18 | # x=[x1, x2] 19 | parser.add_argument('-nx1', dest = 'nx1', type = int, 20 | default = 32, help = 'Number of FV cells in the x1 direction.') 21 | parser.add_argument('-nx2', dest = 'nx2', type = int, 22 | default = 32, help = 'Number of FV cells in the x2 direction.') 23 | parser.add_argument('-lx1', dest = 'lx1', type = float, 24 | default = 0.02, help = 'Lengthscale of the random field along the x1 direction.') 25 | parser.add_argument('-lx2', dest = 'lx2', type = float, 26 | default = 0.02, help = 'Lengthscale of the random field along the x2 direction.') 27 | parser.add_argument('-var', dest = 'var', type = float, 28 | default = 1., help = 'Signal strength (variance) of the random field.') 29 | parser.add_argument('-k', dest = 'k', type = str, 30 | default = 'exp', help = 'Type of covariance kernel (rbf, exp, mat32 or mat52).') 31 | parser.add_argument('-seed', dest = 'seed', type = int, 32 | default = 19, help = 'Random seed number.') 33 | args = parser.parse_args() 34 | kernels = {'rbf':GPy.kern.RBF, 'exp':GPy.kern.Exponential, 35 | 'mat32':GPy.kern.Matern32, 'mat52':GPy.kern.Matern52} 36 | 37 | num_samples = args.N 38 | nx1 = args.nx1 39 | nx2 = args.nx2 40 | ellx1 = args.lx1 41 | ellx2 = args.lx2 42 | variance = args.var 43 | k_ = args.k 44 | assert k_ in kernels.keys() 45 | kern = kernels[k_] 46 | 47 | os.environ['PYTHONHASHSEED'] = '0' 48 | 49 | seed = args.seed 50 | # Setting the seed for numpy-generated random numbers 51 | np.random.seed(seed=seed) 52 | 53 | # Setting the seed for python random numbers 54 | random.seed(seed) 55 | 56 | #define a mean function 57 | def mean(x): 58 | """ 59 | Mean of the conductivity field. 60 | 61 | m(x) = 0. 62 | """ 63 | n = x.shape[0] 64 | return np.zeros((n, 1)) 65 | 66 | #GPy kernel 67 | k=kern(input_dim = 2, 68 | lengthscale = [ellx1, ellx2], 69 | variance = variance, 70 | ARD = True) 71 | 72 | #defining mesh to get cellcenters 73 | Lx1 = 1. # always put . after 1 74 | Lx2 = 1. # always put . after 1 75 | mesh = Grid2D(nx=nx1, ny=nx2, dx=Lx1/nx1, dy=Lx2/nx2) # with nx1*nx2 number of cells/cellcenters/pixels/pixelcenters 76 | cellcenters = mesh.cellCenters.value.T # (nx1*nx2,2) matrix 77 | np.save('cellcenters_nx1='+str(nx1)+'_nx2='+str(nx2)+'.npy', cellcenters) 78 | 79 | 80 | #get covariance matrix and compute its Cholesky decomposition 81 | m = mean(cellcenters) 82 | nugget = 1e-6 # This is a small number required for stability 83 | Cov = k.K(cellcenters) + nugget * np.eye(cellcenters.shape[0]) 84 | L = np.linalg.cholesky(Cov) 85 | 86 | #define matrices to save results 87 | inputs = np.zeros((num_samples, nx1*nx2)) 88 | outputs = np.zeros((num_samples, nx1*nx2)) 89 | 90 | start = time.time() 91 | #generate samples 92 | for i in xrange(num_samples): 93 | #display 94 | if (i+1)%100 == 0: 95 | print "Generating sample "+str(i+1) 96 | 97 | #generate a sample of the random field input 98 | z = np.random.randn(cellcenters.shape[0], 1) 99 | f = m + np.dot(L, z) 100 | sample = np.exp(f) # 'sample' is one image of input field: conductivity image 101 | 102 | # bounding input fields from below and above 103 | lower_bound = np.exp(-5.298317366548036) # 0.005000000000000002 104 | upper_bound = np.exp(3.5) # 33.11545195869231 105 | 106 | sample = np.where(sample < lower_bound, lower_bound, sample) 107 | sample = np.where(sample > upper_bound, upper_bound, sample) 108 | 109 | # FIPY solution 110 | value_left=1. 111 | value_right=0. 112 | value_top=0. 113 | value_bottom=0. 114 | 115 | # define cell and face variables 116 | phi = CellVariable(name='$T(x)$', mesh=mesh, value=0.) 117 | D = CellVariable(name='$D(x)$', mesh=mesh, value=1.0) ## coefficient in diffusion equation 118 | # D = FaceVariable(name='$D(x)$', mesh=mesh, value=1.0) ## coefficient in diffusion equation 119 | source = CellVariable(name='$f(x)$', mesh=mesh, value=1.0) 120 | C = CellVariable(name='$C(x)$', mesh=mesh, value=1.0) 121 | 122 | # apply boundary conditions 123 | # dirichet 124 | phi.constrain(value_left, mesh.facesLeft) 125 | phi.constrain(value_right, mesh.facesRight) 126 | 127 | # homogeneous Neumann 128 | phi.faceGrad.constrain(value_top, mesh.facesTop) 129 | phi.faceGrad.constrain(value_bottom, mesh.facesBottom) 130 | 131 | # setup the diffusion problem 132 | eq = -DiffusionTerm(coeff=D)+ImplicitSourceTerm(coeff=C) == source 133 | 134 | c = 0. 135 | f = 0. # source 136 | 137 | source.setValue(f) 138 | C.setValue(c) 139 | 140 | D.setValue(sample.ravel()) 141 | 142 | eq.solve(var=phi) 143 | x_fipy = mesh.cellCenters.value.T ## fipy solution (nx1*nx2,2) matrix # same as cellcenters defined above 144 | u_fipy = phi.value[:][:, None] ## fipy solution (nx1*nx2,1) matrix 145 | 146 | #save data 147 | inputs[i] = sample.ravel() 148 | outputs[i] = u_fipy.flatten() 149 | 150 | #end timer 151 | finish = time.time() - start 152 | print "Time (sec) to generate "+str(num_samples)+" MC samples : " +str(finish) 153 | 154 | print np.shape(inputs) 155 | print np.shape(outputs) 156 | print inputs 157 | print outputs 158 | 159 | #save data 160 | np.save("MC_samples_inputfield_"+\ 161 | k_+"_nx1="+str(nx1)+\ 162 | "_nx2="+str(nx2)+\ 163 | "_lx1="+str(ellx1)+\ 164 | "_lx2="+str(ellx2)+\ 165 | "_v="+str(variance)+\ 166 | "_num_samples="+str(num_samples)+".npy", inputs) 167 | np.save("MC_samples_u_fipy_"+\ 168 | k_+"_nx1="+str(nx1)+\ 169 | "_nx2="+str(nx2)+\ 170 | "_lx1="+str(ellx1)+\ 171 | "_lx2="+str(ellx2)+\ 172 | "_v="+str(variance)+\ 173 | "_num_samples="+str(num_samples)+".npy", outputs) 174 | 175 | # END -------------------------------------------------------------------------------- /scripts/2d_spde_prob/generate_MC_data/generate_MC_data_a=e^warped_bounded.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | from __future__ import division 3 | import argparse 4 | import numpy as np 5 | import os 6 | import GPy 7 | import matplotlib.pyplot as plt 8 | from fipy import * 9 | from scipy.interpolate import griddata 10 | from pdb import set_trace as keyboard 11 | import time 12 | import random 13 | 14 | seed=19 15 | os.environ['PYTHONHASHSEED'] = '0' 16 | # Setting the seed for numpy-generated random numbers 17 | np.random.seed(seed=seed) 18 | # Setting the seed for python random numbers 19 | random.seed(seed) 20 | 21 | num_samples=100000 # Number of MC samples. 22 | 23 | nx1=32 24 | nx2=32 25 | 26 | kern_1=GPy.kern.RBF 27 | ellx1_1=2 28 | ellx2_1=2 29 | variance_1=0.25 30 | 31 | kern_2=GPy.kern.RBF 32 | ellx1_2=0.1 33 | ellx2_2=0.1 34 | variance_2=0.75 35 | 36 | #define a mean function 37 | def mean_1(x): 38 | return x 39 | 40 | #define a mean function 41 | def mean_2(x): 42 | n = x.shape[0] 43 | return np.zeros((n, 1)) 44 | 45 | #GPy kernel 46 | k_1=kern_1(input_dim = 2, 47 | lengthscale = [ellx1_1, ellx2_1], 48 | variance = variance_1, 49 | ARD = True) 50 | 51 | # GPy kernel 52 | k_2=kern_2(input_dim = 2, 53 | lengthscale = [ellx1_2, ellx2_2], 54 | variance = variance_2, 55 | ARD = True) 56 | 57 | #defining mesh to get cellcenters 58 | Lx1 = 1. # always put . after 1 59 | Lx2 = 1. # always put . after 1 60 | mesh = Grid2D(nx=nx1, ny=nx2, dx=Lx1/nx1, dy=Lx2/nx2) # with nx1*nx2 number of cells/cellcenters/pixels/pixelcenters 61 | cellcenters = mesh.cellCenters.value.T # (nx1*nx2,2) matrix 62 | 63 | np.save('cellcenters_nx1='+str(nx1)+'_nx2='+str(nx2)+'.npy', cellcenters) 64 | 65 | print cellcenters 66 | 67 | #define matrices to save results 68 | inputs = np.zeros((num_samples, nx1*nx2)) 69 | outputs = np.zeros((num_samples, nx1*nx2)) 70 | 71 | start = time.time() 72 | 73 | #generate samples 74 | for i in xrange(num_samples): 75 | #display 76 | if (i+1)%10000 == 0: 77 | print "Generating sample "+str(i+1) 78 | 79 | #get covariance matrix and compute its Cholesky decomposition 80 | m_1 = mean_1(cellcenters) 81 | nugget = 1e-6 # This is a small number required for stability 82 | Cov_1 = k_1.K(cellcenters) + nugget * np.eye(cellcenters.shape[0]) 83 | L_1 = np.linalg.cholesky(Cov_1) 84 | 85 | #generate a sample 86 | z_1 = np.random.randn(cellcenters.shape[0], 1) 87 | f_1 = m_1 + np.dot(L_1, z_1) 88 | 89 | # print f_1 90 | # print np.shape(f_1) 91 | 92 | #get covariance matrix and compute its Cholesky decomposition 93 | m_2 = mean_2(f_1) 94 | nugget = 1e-6 # This is a small number required for stability 95 | Cov_2 = k_2.K(f_1) + nugget * np.eye(f_1.shape[0]) 96 | L_2 = np.linalg.cholesky(Cov_2) 97 | 98 | #generate a sample 99 | z_2 = np.random.randn(f_1.shape[0], 1) 100 | f_2 = m_2 + np.dot(L_2, z_2) 101 | 102 | # print f_2 103 | # print np.shape(f_2) 104 | 105 | sample = np.exp(f_2)# 'sample' is one image of input field: conductivity image 106 | 107 | # bounding input fields from below and above 108 | lower_bound = np.exp(-5.298317366548036) # 0.005000000000000002 109 | upper_bound = np.exp(3.5) # 33.11545195869231 110 | 111 | sample = np.where(sample < lower_bound, lower_bound, sample) 112 | sample = np.where(sample > upper_bound, upper_bound, sample) 113 | 114 | # FIPY solution 115 | value_left=1. 116 | value_right=0. 117 | value_top=0. 118 | value_bottom=0. 119 | 120 | # define cell and face variables 121 | phi = CellVariable(name='$T(x)$', mesh=mesh, value=0.) 122 | D = CellVariable(name='$D(x)$', mesh=mesh, value=1.0) ## coefficient in diffusion equation 123 | # D = FaceVariable(name='$D(x)$', mesh=mesh, value=1.0) ## coefficient in diffusion equation 124 | source = CellVariable(name='$f(x)$', mesh=mesh, value=1.0) 125 | C = CellVariable(name='$C(x)$', mesh=mesh, value=1.0) 126 | 127 | # apply boundary conditions 128 | # dirichet 129 | phi.constrain(value_left, mesh.facesLeft) 130 | phi.constrain(value_right, mesh.facesRight) 131 | 132 | # homogeneous Neumann 133 | phi.faceGrad.constrain(value_top, mesh.facesTop) 134 | phi.faceGrad.constrain(value_bottom, mesh.facesBottom) 135 | 136 | # setup the diffusion problem 137 | eq = -DiffusionTerm(coeff=D)+ImplicitSourceTerm(coeff=C) == source 138 | 139 | c = 0. 140 | f = 0. # source 141 | 142 | source.setValue(f) 143 | C.setValue(c) 144 | 145 | D.setValue(sample.ravel()) 146 | 147 | eq.solve(var=phi) 148 | x_fipy = mesh.cellCenters.value.T ## fipy solution (nx1*nx2,2) matrix # same as cellcenters defined above 149 | u_fipy = phi.value[:][:, None] ## fipy solution (nx1*nx2,1) matrix 150 | 151 | #save data 152 | inputs[i] = sample.ravel() 153 | outputs[i] = u_fipy.flatten() 154 | 155 | #end timer 156 | finish = time.time() - start 157 | print "Time (sec) to generate "+str(num_samples)+" MC samples : " +str(finish) 158 | 159 | print np.shape(inputs) 160 | print np.shape(outputs) 161 | print inputs 162 | print outputs 163 | 164 | #save data 165 | np.save("MC_samples_inputfield_warped_double_rbf"+\ 166 | "_nx1="+str(nx1)+\ 167 | "_nx2="+str(nx2)+\ 168 | "_num_samples="+str(num_samples)+".npy", inputs) 169 | np.save("MC_samples_u_fipy_warped_double_rbf"+\ 170 | "_nx1="+str(nx1)+\ 171 | "_nx2="+str(nx2)+\ 172 | "_num_samples="+str(num_samples)+".npy", outputs) 173 | 174 | # END -------------------------------------------------------------------------------- /scripts/2d_spde_prob/generate_MC_data/generate_MC_data_uncertain_ls_a=e^GRF_bounded.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | from __future__ import division 3 | import argparse 4 | import numpy as np 5 | import os 6 | import GPy 7 | import matplotlib.pyplot as plt 8 | from fipy import * 9 | from scipy.interpolate import griddata 10 | from pdb import set_trace as keyboard 11 | import time 12 | import random 13 | from scipy.stats import truncnorm 14 | 15 | #parse command line arguments 16 | parser = argparse.ArgumentParser() 17 | parser.add_argument('-N', dest = 'N', type = int, 18 | default = 10000, help = 'Number of MC samples.') 19 | # x=[x1, x2] 20 | parser.add_argument('-nx1', dest = 'nx1', type = int, 21 | default = 32, help = 'Number of FV cells in the x1 direction.') 22 | parser.add_argument('-nx2', dest = 'nx2', type = int, 23 | default = 32, help = 'Number of FV cells in the x2 direction.') 24 | parser.add_argument('-var', dest = 'var', type = float, 25 | default = 1., help = 'Signal strength (variance) of the random field.') 26 | parser.add_argument('-k', dest = 'k', type = str, 27 | default = 'exp', help = 'Type of covariance kernel (rbf, exp, mat32 or mat52).') 28 | parser.add_argument('-seed', dest = 'seed', type = int, 29 | default = 19, help = 'Random seed number.') 30 | args = parser.parse_args() 31 | kernels = {'rbf':GPy.kern.RBF, 'exp':GPy.kern.Exponential, 32 | 'mat32':GPy.kern.Matern32, 'mat52':GPy.kern.Matern52} 33 | 34 | num_samples = args.N 35 | nx1 = args.nx1 36 | nx2 = args.nx2 37 | variance = args.var 38 | k_ = args.k 39 | assert k_ in kernels.keys() 40 | kern = kernels[k_] 41 | 42 | os.environ['PYTHONHASHSEED'] = '0' 43 | 44 | seed = args.seed 45 | # Setting the seed for numpy-generated random numbers 46 | np.random.seed(seed=seed) 47 | 48 | # Setting the seed for python random numbers 49 | random.seed(seed) 50 | 51 | #define a mean function 52 | def mean(x): 53 | """ 54 | Mean of the conductivity field. 55 | 56 | m(x) = 0. 57 | """ 58 | n = x.shape[0] 59 | return np.zeros((n, 1)) 60 | 61 | 62 | #defining mesh to get cellcenters 63 | Lx1 = 1. # always put . after 1 64 | Lx2 = 1. # always put . after 1 65 | mesh = Grid2D(nx=nx1, ny=nx2, dx=Lx1/nx1, dy=Lx2/nx2) # with nx1*nx2 number of cells/cellcenters/pixels/pixelcenters 66 | cellcenters = mesh.cellCenters.value.T # (nx1*nx2,2) matrix 67 | np.save('cellcenters_nx1='+str(nx1)+'_nx2='+str(nx2)+'.npy', cellcenters) 68 | 69 | # https://stackoverflow.com/questions/18441779/how-to-specify-upper-and-lower-limits-when-using-numpy-random-normal 70 | # X = truncnorm((lower - mu) / sigma, (upper - mu) / sigma, loc=mu, scale=sigma) 71 | #define the distribution of truncated normals for lengthscales 72 | l1rv = truncnorm((0.07-0.1)/0.03, (0.13-0.1)/0.03, 0.1, 0.03) 73 | l2rv = truncnorm((0.47-0.5)/0.03, (0.53-0.5)/0.03, 0.5, 0.03) 74 | 75 | lx1s = l1rv.rvs(num_samples) 76 | lx2s = l2rv.rvs(num_samples) 77 | ls = np.array(zip(lx1s, lx2s)) 78 | 79 | #define matrices to save results 80 | inputs = np.zeros((num_samples, nx1*nx2)) 81 | outputs = np.zeros((num_samples, nx1*nx2)) 82 | 83 | start = time.time() 84 | #generate samples 85 | for i in xrange(num_samples): 86 | #display 87 | if (i+1)%100 == 0: 88 | print "Generating sample "+str(i+1) 89 | 90 | l1sample = ls[i][0] 91 | l2sample = ls[i][1] 92 | 93 | #GPy kernel 94 | k=kern(input_dim = 2, 95 | lengthscale = [l1sample, l2sample], 96 | variance = variance, 97 | ARD = True) 98 | 99 | #get covariance matrix and compute its Cholesky decomposition 100 | m = mean(cellcenters) 101 | nugget = 1e-6 # This is a small number required for stability 102 | Cov = k.K(cellcenters) + nugget * np.eye(cellcenters.shape[0]) 103 | L = np.linalg.cholesky(Cov) 104 | 105 | #generate a sample of the random field input 106 | z = np.random.randn(cellcenters.shape[0], 1) 107 | f = m + np.dot(L, z) 108 | sample = np.exp(f) # 'sample' is one image of input field: conductivity image 109 | 110 | # bounding input fields from below and above 111 | lower_bound = np.exp(-5.298317366548036) # 0.005000000000000002 112 | upper_bound = np.exp(3.5) # 33.11545195869231 113 | 114 | sample = np.where(sample < lower_bound, lower_bound, sample) 115 | sample = np.where(sample > upper_bound, upper_bound, sample) 116 | 117 | # FIPY solution 118 | value_left=1. 119 | value_right=0. 120 | value_top=0. 121 | value_bottom=0. 122 | 123 | # define cell and face variables 124 | phi = CellVariable(name='$T(x)$', mesh=mesh, value=0.) 125 | D = CellVariable(name='$D(x)$', mesh=mesh, value=1.0) ## coefficient in diffusion equation 126 | # D = FaceVariable(name='$D(x)$', mesh=mesh, value=1.0) ## coefficient in diffusion equation 127 | source = CellVariable(name='$f(x)$', mesh=mesh, value=1.0) 128 | C = CellVariable(name='$C(x)$', mesh=mesh, value=1.0) 129 | 130 | # apply boundary conditions 131 | # dirichet 132 | phi.constrain(value_left, mesh.facesLeft) 133 | phi.constrain(value_right, mesh.facesRight) 134 | 135 | # homogeneous Neumann 136 | phi.faceGrad.constrain(value_top, mesh.facesTop) 137 | phi.faceGrad.constrain(value_bottom, mesh.facesBottom) 138 | 139 | # setup the diffusion problem 140 | eq = -DiffusionTerm(coeff=D)+ImplicitSourceTerm(coeff=C) == source 141 | 142 | c = 0. 143 | f = 0. # source 144 | 145 | source.setValue(f) 146 | C.setValue(c) 147 | 148 | D.setValue(sample.ravel()) 149 | 150 | eq.solve(var=phi) 151 | x_fipy = mesh.cellCenters.value.T ## fipy solution (nx1*nx2,2) matrix # same as cellcenters defined above 152 | u_fipy = phi.value[:][:, None] ## fipy solution (nx1*nx2,1) matrix 153 | 154 | #save data 155 | inputs[i] = sample.ravel() 156 | outputs[i] = u_fipy.flatten() 157 | 158 | #end timer 159 | finish = time.time() - start 160 | print "Time (sec) to generate "+str(num_samples)+" MC samples : " +str(finish) 161 | 162 | print np.shape(inputs) 163 | print np.shape(outputs) 164 | print inputs 165 | print outputs 166 | 167 | #save data 168 | np.save("Uncertain_truncated_normal_sampled_ls.npy",ls) 169 | 170 | np.save("MC_samples_inputfield_"+\ 171 | k_+"_nx1="+str(nx1)+\ 172 | "_nx2="+str(nx2)+\ 173 | "_uncertain_ls"+\ 174 | "_v="+str(variance)+\ 175 | "_num_samples="+str(num_samples)+".npy", inputs) 176 | np.save("MC_samples_u_fipy_"+\ 177 | k_+"_nx1="+str(nx1)+\ 178 | "_nx2="+str(nx2)+\ 179 | "_uncertain_ls"+\ 180 | "_v="+str(variance)+\ 181 | "_num_samples="+str(num_samples)+".npy", outputs) 182 | 183 | # END -------------------------------------------------------------------------------- /scripts/2d_spde_prob/generate_data/generate_data_a=e^GRF.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | from __future__ import division 3 | import argparse 4 | import numpy as np 5 | import os 6 | import GPy 7 | import matplotlib.pyplot as plt 8 | from fipy import * 9 | from scipy.interpolate import griddata 10 | from pdb import set_trace as keyboard 11 | import time 12 | 13 | #parse command line arguments 14 | parser = argparse.ArgumentParser() 15 | parser.add_argument('-N', dest = 'N', type = int, 16 | default = 10000, help = 'Number of samples of the random inputs.') 17 | # x=[x1, x2] 18 | parser.add_argument('-nx1', dest = 'nx1', type = int, 19 | default = 32, help = 'Number of FV cells in the x1 direction.') 20 | parser.add_argument('-nx2', dest = 'nx2', type = int, 21 | default = 32, help = 'Number of FV cells in the x2 direction.') 22 | parser.add_argument('-lx1', dest = 'lx1', type = float, 23 | default = 0.02, help = 'Lengthscale of the random field along the x1 direction.') 24 | parser.add_argument('-lx2', dest = 'lx2', type = float, 25 | default = 0.02, help = 'Lengthscale of the random field along the x2 direction.') 26 | parser.add_argument('-var', dest = 'var', type = float, 27 | default = 1., help = 'Signal strength (variance) of the random field.') 28 | parser.add_argument('-k', dest = 'k', type = str, 29 | default = 'exp', help = 'Type of covariance kernel (rbf, exp, mat32 or mat52).') 30 | # used seed 0 for training data, seed 23 for testing data 31 | parser.add_argument('-seed', dest = 'seed', type = int, 32 | default = 0, help = 'Random seed number.') 33 | args = parser.parse_args() 34 | kernels = {'rbf':GPy.kern.RBF, 'exp':GPy.kern.Exponential, 35 | 'mat32':GPy.kern.Matern32, 'mat52':GPy.kern.Matern52} 36 | 37 | num_samples = args.N 38 | nx1 = args.nx1 39 | nx2 = args.nx2 40 | ellx1 = args.lx1 41 | ellx2 = args.lx2 42 | variance = args.var 43 | k_ = args.k 44 | assert k_ in kernels.keys() 45 | kern = kernels[k_] 46 | seed = args.seed 47 | 48 | np.random.seed(seed=seed) 49 | 50 | #define a mean function 51 | def mean(x): 52 | """ 53 | Mean of the conductivity field. 54 | 55 | m(x) = 0. 56 | """ 57 | n = x.shape[0] 58 | return np.zeros((n, 1)) 59 | 60 | #data directory 61 | cwd = os.getcwd() 62 | data='data' 63 | datadir = os.path.abspath(os.path.join(cwd, data)) 64 | if not os.path.exists(datadir): 65 | os.makedirs(datadir) 66 | 67 | #GPy kernel 68 | k=kern(input_dim = 2, 69 | lengthscale = [ellx1, ellx2], 70 | variance = variance, 71 | ARD = True) 72 | 73 | #defining mesh to get cellcenters 74 | Lx1 = 1. # always put . after 1 75 | Lx2 = 1. # always put . after 1 76 | mesh = Grid2D(nx=nx1, ny=nx2, dx=Lx1/nx1, dy=Lx2/nx2) # with nx1*nx2 number of cells/cellcenters/pixels/pixelcenters 77 | cellcenters = mesh.cellCenters.value.T # (nx1*nx2,2) matrix 78 | np.save(os.path.join(datadir, 'cellcenters_nx1='+str(nx1)+'_nx2='+str(nx2)+'.npy'), cellcenters) 79 | 80 | 81 | #get covariance matrix and compute its Cholesky decomposition 82 | m = mean(cellcenters) 83 | nugget = 1e-6 # This is a small number required for stability 84 | Cov = k.K(cellcenters) + nugget * np.eye(cellcenters.shape[0]) 85 | L = np.linalg.cholesky(Cov) 86 | 87 | #define matrices to save results 88 | inputs = np.zeros((num_samples, nx1*nx2)) 89 | 90 | start = time.time() 91 | #generate samples 92 | for i in xrange(num_samples): 93 | #display 94 | if (i+1)%100 == 0: 95 | print "Generating sample "+str(i+1) 96 | 97 | #generate a sample of the random field input 98 | z = np.random.randn(cellcenters.shape[0], 1) 99 | f = m + np.dot(L, z) 100 | sample = np.exp(f) 101 | #save data 102 | inputs[i] = sample.ravel() 103 | 104 | #end timer 105 | finish = time.time() - start 106 | print "Time (sec) to generate "+str(num_samples)+" samples : " +str(finish) 107 | print inputs 108 | 109 | #save data 110 | datafile = k_+"_nx1="+str(nx1)+\ 111 | "_nx2="+str(nx2)+\ 112 | "_lx1="+str(ellx1)+\ 113 | "_lx2="+str(ellx2)+\ 114 | "_v="+str(variance)+\ 115 | "_num_samples="+str(num_samples)+".npy" 116 | np.save(os.path.join(datadir,datafile), inputs) 117 | -------------------------------------------------------------------------------- /scripts/2d_spde_prob/generate_data/generate_data_multiple_ellxs_a=e^GRF_var0.75/gen_many.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | import argparse 3 | from heateqsolver import SteadyStateHeat2DSolver 4 | import numpy as np 5 | import os 6 | import GPy 7 | import matplotlib.pyplot as plt 8 | from fipy import * 9 | from scipy.interpolate import griddata 10 | from pdb import set_trace as keyboard 11 | import time 12 | # from kle import KarhunenLoeveExpansion 13 | import subprocess 14 | from multiprocessing import Pool 15 | np.random.seed(1) 16 | 17 | def func(case): 18 | cmd = ['python', 'generate_data.py', '-k', case[0], \ 19 | '-nx', str(case[1]),\ 20 | '-ny', str(case[2]),\ 21 | '-lx', str(case[3]),\ 22 | '-ly', str(case[4]),\ 23 | '-N', str(case[5])] 24 | assert subprocess.call(cmd) == 0, 'Failed to run case: '+str(case) 25 | 26 | 27 | if __name__ == '__main__': 28 | #covfuncs = ['mat52', 'rbf', 'mat32', 'exp'] 29 | covfuncs = ['exp'] 30 | # # generate data at design lengthscales 31 | # Lgrid = [] 32 | # c = 0 33 | # n_lx = 60 #number of lengthscale pairs 34 | # while True: 35 | # u = np.random.rand(3) 36 | # if u[2] > np.exp(-1.5*np.sum(u[:2])): 37 | # continue 38 | # else: 39 | # Lgrid.append(u[:2]) 40 | # c += 1 41 | # if c == n_lx: 42 | # break 43 | # Lgrid = np.array(Lgrid) 44 | # Lgrid = 0.035 + Lgrid*(1. - 0.035) 45 | # Neach = 150 # Neach = 500 for training dataset # Neach = 150 for testing dataset 46 | # N = Lgrid.shape[0] * Neach 47 | 48 | # generate test data from arbitrary lengthscales 49 | Neach = 100 50 | lx = np.linspace(0.035, 1., 10) 51 | Lx, Ly = np.meshgrid(lx, lx) 52 | Lgrid = np.hstack([Lx.flatten()[:, None], Ly.flatten()[:, None]]) 53 | 54 | #grid size 55 | nx = 32 56 | ny = 32 57 | cases = [(k, nx, ny, l[0], l[1], Neach) for k in covfuncs for l in Lgrid] 58 | 59 | #generate some data 60 | pool = Pool(4) 61 | pool.map(func, cases) 62 | 63 | for k in covfuncs: 64 | i = 0 65 | datafiles = [x for x in os.listdir(os.path.join(os.getcwd(), 'data')) if k in x and 'lx' in x] 66 | for datafile in datafiles: 67 | data = np.load(os.path.join(os.getcwd(), 'data', datafile)) 68 | if i == 0: 69 | inputs = data['inputs'] 70 | outputs = data['outputs'] 71 | Ellx = [float(data['lx'])] 72 | Elly = [float(data['ly'])] 73 | else: 74 | inputs = np.vstack([inputs, data['inputs']]) 75 | outputs = np.vstack([outputs, data['outputs']]) 76 | Ellx.append(float(data['lx'])) 77 | Elly.append(float(data['ly'])) 78 | i += 1 79 | # np.savez(os.path.join(os.getcwd(), 80 | # 'data', 'train_data_var0.75_'+k+'.npz'), inputs = inputs, 81 | # outputs = outputs, 82 | # lx = np.array(Ellx), 83 | # ly = np.array(Elly), 84 | # Neach = Neach) 85 | # np.savez(os.path.join(os.getcwd(), 86 | # 'data', 'test_data_var0.75_'+k+'.npz'), inputs = inputs, 87 | # outputs = outputs, 88 | # lx = np.array(Ellx), 89 | # ly = np.array(Elly), 90 | # Neach = Neach) 91 | np.savez(os.path.join(os.getcwd(), 92 | 'data', 'test_arbitrary_data_var0.75_'+k+'.npz'), inputs = inputs, 93 | outputs = outputs, 94 | lx = np.array(Ellx), 95 | ly = np.array(Elly), 96 | Neach = Neach) 97 | 98 | 99 | 100 | 101 | 102 | 103 | 104 | -------------------------------------------------------------------------------- /scripts/2d_spde_prob/generate_data/generate_data_multiple_ellxs_a=e^GRF_var0.75/generate_data.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | import argparse 3 | from heateqsolver import SteadyStateHeat2DSolver 4 | import numpy as np 5 | import os 6 | import GPy 7 | import matplotlib.pyplot as plt 8 | from fipy import * 9 | from scipy.interpolate import griddata 10 | from pdb import set_trace as keyboard 11 | import time 12 | 13 | #parse command line arguments 14 | parser = argparse.ArgumentParser() 15 | parser.add_argument('-N', dest = 'N', type = int, 16 | default = 1000, help = 'Number of samples of the random inputs') 17 | parser.add_argument('-nx', dest = 'nx', type = int, 18 | default = 32, help = 'Number of FV cells in the x direction.') 19 | parser.add_argument('-ny', dest = 'ny', type = int, 20 | default = 32, help = 'Number of FV cells in the y direction.') 21 | parser.add_argument('-lx', dest = 'lx', type = float, 22 | default = 0.02, help = 'Lengthscale of the random field along the x direction.') 23 | parser.add_argument('-ly', dest = 'ly', type = float, 24 | default = 0.02, help = 'Lengthscale of the random field along the y direction.') 25 | parser.add_argument('-var', dest = 'var', type = float, 26 | default = 0.75, help = 'Signal strength (variance) of the random field.') 27 | parser.add_argument('-k', dest = 'k', type = str, 28 | default = 'rbf', help = 'Type of covariance kernel (rbf, exp, mat32 or mat52)') 29 | args = parser.parse_args() 30 | kernels = {'rbf':GPy.kern.RBF, 'exp':GPy.kern.Exponential, 31 | 'mat32':GPy.kern.Matern32, 'mat52':GPy.kern.Matern52} 32 | 33 | num_samples = args.N 34 | nx = args.nx 35 | ny = args.ny 36 | ellx = args.lx 37 | elly = args.ly 38 | variance = args.var 39 | k_ = args.k 40 | assert k_ in kernels.keys() 41 | kern = kernels[k_] 42 | 43 | #define a mean function 44 | def mean(x): 45 | """ 46 | Mean of the permeability field. 47 | 48 | m(x) = 0. 49 | """ 50 | n = x.shape[0] 51 | return np.zeros((n, 1)) 52 | 53 | def q(x): 54 | n = x.shape[0] 55 | s = np.zeros((n)) 56 | return s 57 | 58 | #data directory 59 | cwd = os.getcwd() 60 | data='data' 61 | datadir = os.path.abspath(os.path.join(cwd, data)) 62 | if not os.path.exists(datadir): 63 | os.makedirs(datadir) 64 | 65 | #GPy kernel 66 | k=kern(input_dim = 2, 67 | lengthscale = [ellx, elly], 68 | variance = variance, 69 | ARD = True) 70 | 71 | ##define the solver object 72 | solver = SteadyStateHeat2DSolver(nx=nx, ny=ny) 73 | cellcenters = solver.mesh.cellCenters.value.T 74 | np.save(os.path.join(datadir, 'cellcenters.npy'), cellcenters) 75 | 76 | #get source field 77 | source = q(cellcenters) 78 | 79 | #get covariance matrix and compute its Cholesky decomposition 80 | m=mean(cellcenters) 81 | C=k.K(cellcenters) + 1e-6*np.eye(cellcenters.shape[0]) 82 | L=np.linalg.cholesky(C) 83 | 84 | #define matrices to save results 85 | inputs = np.zeros((num_samples, nx, ny)) 86 | outputs = np.zeros((num_samples, nx, ny)) 87 | 88 | start = time.time() 89 | #generate samples 90 | for i in xrange(num_samples): 91 | #display 92 | if (i+1)%100 == 0: 93 | print "Generating sample "+str(i+1) 94 | 95 | #generate a sample of the random field input 96 | z =np.random.randn(cellcenters.shape[0], 1) 97 | f = m + np.dot(L, z) 98 | sample = np.exp(f[:, 0]) 99 | 100 | #solve the PDE 101 | solver.set_coeff(C=sample) #set diffusion coefficient. 102 | solver.set_source(source=source) #set source term. 103 | solver.solve() 104 | 105 | #save data 106 | inputs[i] = f.reshape((nx, ny)) 107 | outputs[i] = solver.phi.value.reshape((nx, ny)) 108 | 109 | #end timer 110 | finish = time.time() - start 111 | print "Time (sec) to generate "+str(num_samples)+" samples : " +str(finish) 112 | 113 | #save data 114 | datafile = k_+"_lx_"+str(ellx).replace('.', '')+\ 115 | "_ly_"+str(elly).replace('.', '')+\ 116 | "_v_"+str(variance).replace('.', '')+".npz" 117 | 118 | np.savez(os.path.join(datadir, datafile), inputs=inputs,\ 119 | outputs=outputs,\ 120 | nx=nx, ny=ny, lx=ellx, ly=elly,\ 121 | var=variance) 122 | 123 | 124 | 125 | -------------------------------------------------------------------------------- /scripts/2d_spde_prob/generate_data/generate_data_multiple_ellxs_a=e^GRF_var0.75/heateqsolver.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | """ 3 | We define a class ``SteadyStateHeat2DSolver'' which 4 | solves the steady state heat equation 5 | in a two-dimensional square grid. 6 | 7 | For now we don't add any forcing function and the 8 | boundary conditions are Dirichlet. 9 | 10 | """ 11 | 12 | 13 | __all__ = ['SteadyStateHeat2DSolver'] 14 | 15 | import fipy 16 | import numpy as np 17 | from pdb import set_trace as keyboard 18 | from scipy.interpolate import griddata 19 | import matplotlib.pyplot as plt 20 | 21 | 22 | class SteadyStateHeat2DSolver(object): 23 | 24 | """ 25 | Solves the 2D steady state heat equation with dirichlet boundary conditions. 26 | It uses the stochastic model we developed above to define the random conductivity. 27 | _ 28 | Arguments: 29 | nx - Number of grid points along x direction. 30 | ny - Number of grid points along y direction. 31 | value_left - The value at the left side of the boundary. 32 | value_right - The value at the right side of the boundary. 33 | value_top - The value at the top of the boundary. 34 | value_bottom - The value at the bottom of the boundary. 35 | """ 36 | 37 | def __init__(self, nx=100, ny=100, value_left=1., 38 | value_right=0., value_top=0., value_bottom=0., 39 | q=None): 40 | """ 41 | ::param nx:: Number of cells in the x direction. 42 | ::param ny:: Number of cells in the y direction. 43 | ::param value_left:: Boundary condition on the left face. 44 | ::param value_right:: Boundary condition on the right face. 45 | ::param value_top:: Boundary condition on the top face. 46 | ::param value_bottom:: Boundary condition on the bottom face. 47 | ::param q:: Source function. 48 | """ 49 | #set domain dimensions 50 | self.nx = nx 51 | self.ny = ny 52 | self.dx = 1. / nx 53 | self.dy = 1. / ny 54 | 55 | #define mesh 56 | self.mesh = fipy.Grid2D(nx=self.nx, ny=self.ny, dx=self.dx, dy=self.dy) 57 | 58 | #get the location of the middle of the domain 59 | #cellcenters=np.array(self.mesh.cellCenters).T 60 | #x=cellcenters[:, 0] 61 | #y=cellcenters[:, 1] 62 | x, y = self.mesh.cellCenters 63 | x_all=x[:self.nx] 64 | y_all=y[0:-1:self.ny] 65 | loc1=x_all[(self.nx-1)/2] 66 | loc2=y_all[(self.ny-1)/2] 67 | self.loc=np.intersect1d(np.where(x==loc1)[0], np.where(y==loc2)[0])[0] 68 | 69 | #get facecenters 70 | X, Y = self.mesh.faceCenters 71 | 72 | #define cell and face variables 73 | self.phi = fipy.CellVariable(name='$T(x)$', mesh=self.mesh, value=1.) 74 | self.C = fipy.CellVariable(name='$C(x)$', mesh=self.mesh, value=1.) 75 | self.source=fipy.CellVariable(name='$f(x)$', mesh=self.mesh, value=0.) 76 | 77 | #apply boundary conditions 78 | #dirichet 79 | self.phi.constrain(value_left, self.mesh.facesLeft) 80 | self.phi.constrain(value_right, self.mesh.facesRight) 81 | 82 | #homogeneous Neumann 83 | self.phi.faceGrad.constrain(value_top, self.mesh.facesTop) 84 | self.phi.faceGrad.constrain(value_bottom, self.mesh.facesBottom) 85 | 86 | #setup the diffusion problem 87 | self.eq = -fipy.DiffusionTerm(coeff=self.C) == self.source 88 | 89 | def set_source(self, source): 90 | """ 91 | Initialize the source field. 92 | """ 93 | self.source.setValue(source) 94 | 95 | def set_coeff(self, C): 96 | """ 97 | Initialize the random conductivity field. 98 | """ 99 | self.C.setValue(C) 100 | 101 | def solve(self): 102 | self.eq.solve(var=self.phi) 103 | 104 | def ObjectiveFunction(self): 105 | """ 106 | We look at the temperature in the middle of the domain. 107 | """ 108 | return self.phi.value[self.loc] 109 | 110 | def NeumannSpatialAverage(self): 111 | """ 112 | Spatial average of the independent variable on the right side 113 | Neumann boundary. 114 | """ 115 | loc = np.where(np.int32(self.mesh.facesRight.value) == 1)[0] 116 | val = self.phi.faceValue.value[loc] 117 | return np.mean(val) 118 | 119 | 120 | def RandomField(self): 121 | facecenters=np.array(self.mesh.faceCenters).T 122 | xf=facecenters[:, 0] 123 | yf=facecenters[:, 1] 124 | zf=self.C.value 125 | xif=yif=np.linspace(0.01, 0.99, 32) 126 | zif=griddata((xf, yf), zf, (xif[None,:], yif[:,None]), method='cubic') 127 | return zif 128 | -------------------------------------------------------------------------------- /trained_models/1d_spde_prob/data_file_and_dnn_architecture.txt: -------------------------------------------------------------------------------- 1 | Data file used to train this DNN: 2 | Train data- 3 | 'data_1d_spde_prob/train_exp_nx=100_lx=0.03_v=1.0_num_samples=10000.npy' 4 | Test data- 5 | 'data_1d_spde_prob/test_exp_nx=100_lx=0.03_v=1.0_num_samples=1000.npy' 6 | 7 | 8 | DNN architecture: 9 | DNN type = Resnet 10 | Number of neurons in each block -n=400 11 | Number of blocks -num_block=3 12 | Number of dense layers in each block=2 13 | 14 | 15 | To run: 16 | python 1d_spde.py -train_data='train_exp_nx=100_lx=0.03_v=1.0_num_samples=10000.npy' -test_data='test_exp_nx=100_lx=0.03_v=1.0_num_samples=1000.npy' -nx=100 -DNN_type='Resnet' -n=400 -num_block=3 -act_func='swish' -loss_type='EF' -lr=0.0001 -max_it=80000 -M_A=100 -M_x=15 -seed=4 17 | 18 | -------------------------------------------------------------------------------- /trained_models/1d_spde_prob/my_model_weights.h5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PredictiveScienceLab/variational-elliptic-SPDE/967d8b6e5a4fb123eef199c86b7ec441814b08b0/trained_models/1d_spde_prob/my_model_weights.h5 -------------------------------------------------------------------------------- /trained_models/2d_spde_prob/1_2dim_eg_var0.75/data_file_and_dnn_architecture.txt: -------------------------------------------------------------------------------- 1 | Data file used to train this DNN: 2 | Train data- 3 | 'data_2d_spde_prob/train_exp_nx1=32_nx2=32_lx1=0.05_lx2=0.08_v=0.75_num_samples=10000.npy' 4 | Test data- 5 | 'data_2d_spde_prob/test_exp_nx1=32_nx2=32_lx1=0.05_lx2=0.08_v=0.75_num_samples=2000.npy' 6 | 7 | 8 | DNN architecture: 9 | DNN type = Resnet 10 | Number of neurons in each block -n=350 11 | Number of blocks -num_block=3 12 | Number of dense layers in each block=2 13 | 14 | 15 | To run: 16 | python 2d_spde.py -train_data='train_exp_nx1=32_nx2=32_lx1=0.05_lx2=0.08_v=0.75_num_samples=10000.npy' -test_data='test_exp_nx1=32_nx2=32_lx1=0.05_lx2=0.08_v=0.75_num_samples=2000.npy' -nx1=32 -nx2=32 -DNN_type='Resnet' -n=350 -num_block=3 -act_func='swish' -loss_type='EF' -lr=0.0001 -max_it=75000 -M_A=100 -M_x=20 -seed=2 -------------------------------------------------------------------------------- /trained_models/2d_spde_prob/1_2dim_eg_var0.75/my_model_weights.h5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PredictiveScienceLab/variational-elliptic-SPDE/967d8b6e5a4fb123eef199c86b7ec441814b08b0/trained_models/2d_spde_prob/1_2dim_eg_var0.75/my_model_weights.h5 -------------------------------------------------------------------------------- /trained_models/2d_spde_prob/2_warped/data_file_and_dnn_architecture.txt: -------------------------------------------------------------------------------- 1 | Data file used to train this DNN: 2 | Train data- 3 | 'data_2d_spde_prob/train_warped_double_rbf_nx1=32_nx2=32_num_samples=10000.npy' 4 | Test data- 5 | 'data_2d_spde_prob/test_warped_double_rbf_nx1=32_nx2=32_num_samples=1000.npy' 6 | 7 | nx1=32 8 | nx2=32 9 | 10 | kern_1=GPy.kern.RBF 11 | ellx1_1=2 12 | ellx2_1=2 13 | variance_1=0.25 14 | 15 | kern_2=GPy.kern.RBF 16 | ellx1_2=0.1 17 | ellx2_2=0.1 18 | variance_2=0.75 19 | 20 | 21 | DNN architecture: 22 | DNN type = Resnet 23 | Number of neurons in each block -n=300 24 | Number of blocks -num_block=5 25 | Number of dense layers in each block=2 26 | 27 | 28 | To run: 29 | python 2d_spde.py -train_data='train_warped_double_rbf_nx1=32_nx2=32_num_samples=10000.npy' -test_data='test_warped_double_rbf_nx1=32_nx2=32_num_samples=1000.npy' -nx1=32 -nx2=32 -DNN_type='Resnet' -n=300 -num_block=5 -act_func='swish' -loss_type='EF' -lr=0.0001 -max_it=75000 -M_A=100 -M_x=20 -seed=2 30 | 31 | -------------------------------------------------------------------------------- /trained_models/2d_spde_prob/2_warped/my_model_weights.h5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PredictiveScienceLab/variational-elliptic-SPDE/967d8b6e5a4fb123eef199c86b7ec441814b08b0/trained_models/2d_spde_prob/2_warped/my_model_weights.h5 -------------------------------------------------------------------------------- /trained_models/2d_spde_prob/3_channelized_permeability_fields/data_file_and_dnn_architecture.txt: -------------------------------------------------------------------------------- 1 | Data file used to train this DNN: 2 | Train data- 3 | 'data_2d_spde_prob/train_channel_nx1=32_nx2=32_num_samples=4096.npy' 4 | Test data- 5 | 'data_2d_spde_prob/test_channel_nx1=32_nx2=32_num_samples=512.npy' 6 | 7 | 8 | DNN architecture: 9 | DNN type = Resnet 10 | Number of neurons in each block -n=300 11 | Number of blocks -num_block=3 12 | Number of dense layers in each block=2 13 | 14 | 15 | To run: 16 | python 2d_spde.py -train_data='train_channel_nx1=32_nx2=32_num_samples=4096.npy' -test_data='test_channel_nx1=32_nx2=32_num_samples=512.npy' -nx1=32 -nx2=32 -DNN_type='Resnet' -n=300 -num_block=3 -act_func='swish' -loss_type='EF' -lr=0.0001 -max_it=75000 -M_A=100 -M_x=20 -seed=0 17 | 18 | -------------------------------------------------------------------------------- /trained_models/2d_spde_prob/3_channelized_permeability_fields/my_model_weights.h5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PredictiveScienceLab/variational-elliptic-SPDE/967d8b6e5a4fb123eef199c86b7ec441814b08b0/trained_models/2d_spde_prob/3_channelized_permeability_fields/my_model_weights.h5 -------------------------------------------------------------------------------- /trained_models/2d_spde_prob/4_multiple_ellxs_var0.75/data_file_and_dnn_architecture.txt: -------------------------------------------------------------------------------- 1 | Data file used to train this DNN: 2 | Train data- 3 | 'data_2d_spde_prob/train_exp_nx1=32_nx2=32_multiple_ellxs_v=0.75_total_num_samples=30000.npy' 4 | Test data- 5 | 'data_2d_spde_prob/test_exp_nx1=32_nx2=32_multiple_ellxs_v=0.75_total_num_samples=9000.npy' 6 | 7 | 8 | DNN architecture: 9 | DNN type = Resnet 10 | Number of neurons in each block -n=500 11 | Number of blocks -num_block=3 12 | Number of dense layers in each block=2 13 | 14 | 15 | To run: 16 | python 2d_spde.py -train_data='train_exp_nx1=32_nx2=32_multiple_ellxs_v=0.75_total_num_samples=30000.npy' -test_data='test_exp_nx1=32_nx2=32_multiple_ellxs_v=0.75_total_num_samples=9000.npy' -nx1=32 -nx2=32 -DNN_type='Resnet' -n=500 -num_block=3 -act_func='swish' -loss_type='EF' -lr=0.0001 -max_it=75000 -M_A=100 -M_x=20 -seed=2 17 | 18 | 19 | -------------------------------------------------------------------------------- /trained_models/2d_spde_prob/4_multiple_ellxs_var0.75/my_model_weights.h5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PredictiveScienceLab/variational-elliptic-SPDE/967d8b6e5a4fb123eef199c86b7ec441814b08b0/trained_models/2d_spde_prob/4_multiple_ellxs_var0.75/my_model_weights.h5 -------------------------------------------------------------------------------- /trained_models/2d_spde_prob/5_merged/data_file_and_dnn_architecture.txt: -------------------------------------------------------------------------------- 1 | Data file used to train this DNN: 2 | Train data- 3 | 'data_2d_spde_prob/train_merged_nx1=32_nx2=32.npy' 4 | Test data- 5 | 'data_2d_spde_prob/test_merged_nx1=32_nx2=32.npy' 6 | 7 | 8 | DNN architecture: 9 | DNN type = Resnet 10 | Number of neurons in each block -n=500 11 | Number of blocks -num_block=2 12 | Number of dense layers in each block=2 13 | 14 | 15 | To run: 16 | python 2d_spde.py -train_data='train_merged_nx1=32_nx2=32.npy' -test_data='test_merged_nx1=32_nx2=32.npy' -nx1=32 -nx2=32 -DNN_type='Resnet' -n=500 -num_block=2 -act_func='swish' -loss_type='EF' -lr=0.0001 -max_it=75000 -M_A=100 -M_x=20 -seed=2 17 | 18 | -------------------------------------------------------------------------------- /trained_models/2d_spde_prob/5_merged/my_model_weights.h5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/PredictiveScienceLab/variational-elliptic-SPDE/967d8b6e5a4fb123eef199c86b7ec441814b08b0/trained_models/2d_spde_prob/5_merged/my_model_weights.h5 --------------------------------------------------------------------------------