├── README.md ├── advanced_tutorial.ipynb ├── clustering.ipynb ├── data ├── emit_probability.txt ├── honglou.txt ├── initial_vector.txt ├── story.txt ├── transition_probability.txt └── xiyouji_wuchengen.txt ├── dimension_reduction.ipynb ├── graph_model.ipynb ├── requirements.txt └── semi_supervise.ipynb /README.md: -------------------------------------------------------------------------------- 1 | # 说明 2 | 代码库选取sklearn例子配合课程学习 3 | 4 | # 使用说明 5 | 1. 下载代码 git clone https://github.com/lixinsu/tutorials2018.git 6 | 2. 安装python3.5 建议直接使用anaconda(包含常用的包) https://www.anaconda.com/download/#macos 7 | 3. pip install -r requirements.txt 安装依赖 8 | 4. 在tutorials2018 目录下打开 jupyter notebook，通过浏览器访问其显示的网址 9 | 10 | 11 | # 问题反馈 12 | 1. 请在github上提交issue反馈问题 13 | 2. 目前代码大部分直接来自sklearn例子，如有更有趣的demo展示例子，欢迎pull request。 14 | -------------------------------------------------------------------------------- /advanced_tutorial.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "metadata": {}, 7 | "outputs": [], 8 | "source": [ 9 | "%matplotlib inline" 10 | ] 11 | }, 12 | { 13 | "cell_type": "markdown", 14 | "metadata": {}, 15 | "source": [ 16 | "\n", 17 | "Advanced: Making Dynamic Decisions and the Bi-LSTM CRF\n", 18 | "======================================================\n", 19 | "\n", 20 | "Dynamic versus Static Deep Learning Toolkits\n", 21 | "--------------------------------------------\n", 22 | "\n", 23 | "Pytorch is a *dynamic* neural network kit. Another example of a dynamic\n", 24 | "kit is `Dynet `__ (I mention this because\n", 25 | "working with Pytorch and Dynet is similar. If you see an example in\n", 26 | "Dynet, it will probably help you implement it in Pytorch). The opposite\n", 27 | "is the *static* tool kit, which includes Theano, Keras, TensorFlow, etc.\n", 28 | "The core difference is the following:\n", 29 | "\n", 30 | "* In a static toolkit, you define\n", 31 | " a computation graph once, compile it, and then stream instances to it.\n", 32 | "* In a dynamic toolkit, you define a computation graph *for each\n", 33 | " instance*. It is never compiled and is executed on-the-fly\n", 34 | "\n", 35 | "Without a lot of experience, it is difficult to appreciate the\n", 36 | "difference. One example is to suppose we want to build a deep\n", 37 | "constituent parser. Suppose our model involves roughly the following\n", 38 | "steps:\n", 39 | "\n", 40 | "* We build the tree bottom up\n", 41 | "* Tag the root nodes (the words of the sentence)\n", 42 | "* From there, use a neural network and the embeddings\n", 43 | " of the words to find combinations that form constituents. Whenever you\n", 44 | " form a new constituent, use some sort of technique to get an embedding\n", 45 | " of the constituent. In this case, our network architecture will depend\n", 46 | " completely on the input sentence. In the sentence \"The green cat\n", 47 | " scratched the wall\", at some point in the model, we will want to combine\n", 48 | " the span $(i,j,r) = (1, 3, \\text{NP})$ (that is, an NP constituent\n", 49 | " spans word 1 to word 3, in this case \"The green cat\").\n", 50 | "\n", 51 | "However, another sentence might be \"Somewhere, the big fat cat scratched\n", 52 | "the wall\". In this sentence, we will want to form the constituent\n", 53 | "$(2, 4, NP)$ at some point. The constituents we will want to form\n", 54 | "will depend on the instance. If we just compile the computation graph\n", 55 | "once, as in a static toolkit, it will be exceptionally difficult or\n", 56 | "impossible to program this logic. In a dynamic toolkit though, there\n", 57 | "isn't just 1 pre-defined computation graph. There can be a new\n", 58 | "computation graph for each instance, so this problem goes away.\n", 59 | "\n", 60 | "Dynamic toolkits also have the advantage of being easier to debug and\n", 61 | "the code more closely resembling the host language (by that I mean that\n", 62 | "Pytorch and Dynet look more like actual Python code than Keras or\n", 63 | "Theano).\n", 64 | "\n", 65 | "Bi-LSTM Conditional Random Field Discussion\n", 66 | "-------------------------------------------\n", 67 | "\n", 68 | "For this section, we will see a full, complicated example of a Bi-LSTM\n", 69 | "Conditional Random Field for named-entity recognition. The LSTM tagger\n", 70 | "above is typically sufficient for part-of-speech tagging, but a sequence\n", 71 | "model like the CRF is really essential for strong performance on NER.\n", 72 | "Familiarity with CRF's is assumed. Although this name sounds scary, all\n", 73 | "the model is is a CRF but where an LSTM provides the features. This is\n", 74 | "an advanced model though, far more complicated than any earlier model in\n", 75 | "this tutorial. If you want to skip it, that is fine. To see if you're\n", 76 | "ready, see if you can:\n", 77 | "\n", 78 | "- Write the recurrence for the viterbi variable at step i for tag k.\n", 79 | "- Modify the above recurrence to compute the forward variables instead.\n", 80 | "- Modify again the above recurrence to compute the forward variables in\n", 81 | " log-space (hint: log-sum-exp)\n", 82 | "\n", 83 | "If you can do those three things, you should be able to understand the\n", 84 | "code below. Recall that the CRF computes a conditional probability. Let\n", 85 | "$y$ be a tag sequence and $x$ an input sequence of words.\n", 86 | "Then we compute\n", 87 | "\n", 88 | "\\begin{align}P(y|x) = \\frac{\\exp{(\\text{Score}(x, y)})}{\\sum_{y'} \\exp{(\\text{Score}(x, y')})}\\end{align}\n", 89 | "\n", 90 | "Where the score is determined by defining some log potentials\n", 91 | "$\\log \\psi_i(x,y)$ such that\n", 92 | "\n", 93 | "\\begin{align}\\text{Score}(x,y) = \\sum_i \\log \\psi_i(x,y)\\end{align}\n", 94 | "\n", 95 | "To make the partition function tractable, the potentials must look only\n", 96 | "at local features.\n", 97 | "\n", 98 | "In the Bi-LSTM CRF, we define two kinds of potentials: emission and\n", 99 | "transition. The emission potential for the word at index $i$ comes\n", 100 | "from the hidden state of the Bi-LSTM at timestep $i$. The\n", 101 | "transition scores are stored in a $|T|x|T|$ matrix\n", 102 | "$\\textbf{P}$, where $T$ is the tag set. In my\n", 103 | "implementation, $\\textbf{P}_{j,k}$ is the score of transitioning\n", 104 | "to tag $j$ from tag $k$. So:\n", 105 | "\n", 106 | "\\begin{align}\\text{Score}(x,y) = \\sum_i \\log \\psi_\\text{EMIT}(y_i \\rightarrow x_i) + \\log \\psi_\\text{TRANS}(y_{i-1} \\rightarrow y_i)\\end{align}\n", 107 | "\n", 108 | "\\begin{align}= \\sum_i h_i[y_i] + \\textbf{P}_{y_i, y_{i-1}}\\end{align}\n", 109 | "\n", 110 | "where in this second expression, we think of the tags as being assigned\n", 111 | "unique non-negative indices.\n", 112 | "\n", 113 | "If the above discussion was too brief, you can check out\n", 114 | "`this `__ write up from\n", 115 | "Michael Collins on CRFs.\n", 116 | "\n", 117 | "Implementation Notes\n", 118 | "--------------------\n", 119 | "\n", 120 | "The example below implements the forward algorithm in log space to\n", 121 | "compute the partition function, and the viterbi algorithm to decode.\n", 122 | "Backpropagation will compute the gradients automatically for us. We\n", 123 | "don't have to do anything by hand.\n", 124 | "\n", 125 | "The implementation is not optimized. If you understand what is going on,\n", 126 | "you'll probably quickly see that iterating over the next tag in the\n", 127 | "forward algorithm could probably be done in one big operation. I wanted\n", 128 | "to code to be more readable. If you want to make the relevant change,\n", 129 | "you could probably use this tagger for real tasks.\n", 130 | "\n" 131 | ] 132 | }, 133 | { 134 | "cell_type": "code", 135 | "execution_count": null, 136 | "metadata": {}, 137 | "outputs": [], 138 | "source": [ 139 | "# Author: Robert Guthrie\n", 140 | "\n", 141 | "import torch\n", 142 | "import torch.autograd as autograd\n", 143 | "import torch.nn as nn\n", 144 | "import torch.optim as optim\n", 145 | "\n", 146 | "torch.manual_seed(1)" 147 | ] 148 | }, 149 | { 150 | "cell_type": "markdown", 151 | "metadata": {}, 152 | "source": [ 153 | "Helper functions to make the code more readable.\n", 154 | "\n" 155 | ] 156 | }, 157 | { 158 | "cell_type": "code", 159 | "execution_count": null, 160 | "metadata": {}, 161 | "outputs": [], 162 | "source": [ 163 | "def argmax(vec):\n", 164 | " # return the argmax as a python int\n", 165 | " _, idx = torch.max(vec, 1)\n", 166 | " return idx.item()\n", 167 | "\n", 168 | "\n", 169 | "def prepare_sequence(seq, to_ix):\n", 170 | " idxs = [to_ix[w] for w in seq]\n", 171 | " return torch.tensor(idxs, dtype=torch.long)\n", 172 | "\n", 173 | "\n", 174 | "# Compute log sum exp in a numerically stable way for the forward algorithm\n", 175 | "def log_sum_exp(vec):\n", 176 | " max_score = vec[0, argmax(vec)]\n", 177 | " max_score_broadcast = max_score.view(1, -1).expand(1, vec.size()[1])\n", 178 | " return max_score + \\\n", 179 | " torch.log(torch.sum(torch.exp(vec - max_score_broadcast)))" 180 | ] 181 | }, 182 | { 183 | "cell_type": "markdown", 184 | "metadata": {}, 185 | "source": [ 186 | "Create model\n", 187 | "\n" 188 | ] 189 | }, 190 | { 191 | "cell_type": "code", 192 | "execution_count": null, 193 | "metadata": {}, 194 | "outputs": [], 195 | "source": [ 196 | "class BiLSTM_CRF(nn.Module):\n", 197 | "\n", 198 | " def __init__(self, vocab_size, tag_to_ix, embedding_dim, hidden_dim):\n", 199 | " super(BiLSTM_CRF, self).__init__()\n", 200 | " self.embedding_dim = embedding_dim\n", 201 | " self.hidden_dim = hidden_dim\n", 202 | " self.vocab_size = vocab_size\n", 203 | " self.tag_to_ix = tag_to_ix\n", 204 | " self.tagset_size = len(tag_to_ix)\n", 205 | "\n", 206 | " self.word_embeds = nn.Embedding(vocab_size, embedding_dim)\n", 207 | " self.lstm = nn.LSTM(embedding_dim, hidden_dim // 2,\n", 208 | " num_layers=1, bidirectional=True)\n", 209 | "\n", 210 | " # Maps the output of the LSTM into tag space.\n", 211 | " self.hidden2tag = nn.Linear(hidden_dim, self.tagset_size)\n", 212 | "\n", 213 | " # Matrix of transition parameters. Entry i,j is the score of\n", 214 | " # transitioning *to* i *from* j.\n", 215 | " self.transitions = nn.Parameter(\n", 216 | " torch.randn(self.tagset_size, self.tagset_size))\n", 217 | "\n", 218 | " # These two statements enforce the constraint that we never transfer\n", 219 | " # to the start tag and we never transfer from the stop tag\n", 220 | " self.transitions.data[tag_to_ix[START_TAG], :] = -10000\n", 221 | " self.transitions.data[:, tag_to_ix[STOP_TAG]] = -10000\n", 222 | "\n", 223 | " self.hidden = self.init_hidden()\n", 224 | "\n", 225 | " def init_hidden(self):\n", 226 | " return (torch.randn(2, 1, self.hidden_dim // 2),\n", 227 | " torch.randn(2, 1, self.hidden_dim // 2))\n", 228 | "\n", 229 | " def _forward_alg(self, feats):\n", 230 | " # Do the forward algorithm to compute the partition function\n", 231 | " init_alphas = torch.full((1, self.tagset_size), -10000.)\n", 232 | " # START_TAG has all of the score.\n", 233 | " init_alphas[0][self.tag_to_ix[START_TAG]] = 0.\n", 234 | "\n", 235 | " # Wrap in a variable so that we will get automatic backprop\n", 236 | " forward_var = init_alphas\n", 237 | "\n", 238 | " # Iterate through the sentence\n", 239 | " for feat in feats:\n", 240 | " alphas_t = [] # The forward tensors at this timestep\n", 241 | " for next_tag in range(self.tagset_size):\n", 242 | " # broadcast the emission score: it is the same regardless of\n", 243 | " # the previous tag\n", 244 | " emit_score = feat[next_tag].view(\n", 245 | " 1, -1).expand(1, self.tagset_size)\n", 246 | " # the ith entry of trans_score is the score of transitioning to\n", 247 | " # next_tag from i\n", 248 | " trans_score = self.transitions[next_tag].view(1, -1)\n", 249 | " # The ith entry of next_tag_var is the value for the\n", 250 | " # edge (i -> next_tag) before we do log-sum-exp\n", 251 | " next_tag_var = forward_var + trans_score + emit_score\n", 252 | " # The forward variable for this tag is log-sum-exp of all the\n", 253 | " # scores.\n", 254 | " alphas_t.append(log_sum_exp(next_tag_var).view(1))\n", 255 | " forward_var = torch.cat(alphas_t).view(1, -1)\n", 256 | " terminal_var = forward_var + self.transitions[self.tag_to_ix[STOP_TAG]]\n", 257 | " alpha = log_sum_exp(terminal_var)\n", 258 | " return alpha\n", 259 | "\n", 260 | " def _get_lstm_features(self, sentence):\n", 261 | " self.hidden = self.init_hidden()\n", 262 | " embeds = self.word_embeds(sentence).view(len(sentence), 1, -1)\n", 263 | " lstm_out, self.hidden = self.lstm(embeds, self.hidden)\n", 264 | " lstm_out = lstm_out.view(len(sentence), self.hidden_dim)\n", 265 | " lstm_feats = self.hidden2tag(lstm_out)\n", 266 | " return lstm_feats\n", 267 | "\n", 268 | " def _score_sentence(self, feats, tags):\n", 269 | " # Gives the score of a provided tag sequence\n", 270 | " score = torch.zeros(1)\n", 271 | " tags = torch.cat([torch.tensor([self.tag_to_ix[START_TAG]], dtype=torch.long), tags])\n", 272 | " for i, feat in enumerate(feats):\n", 273 | " score = score + \\\n", 274 | " self.transitions[tags[i + 1], tags[i]] + feat[tags[i + 1]]\n", 275 | " score = score + self.transitions[self.tag_to_ix[STOP_TAG], tags[-1]]\n", 276 | " return score\n", 277 | "\n", 278 | " def _viterbi_decode(self, feats):\n", 279 | " backpointers = []\n", 280 | "\n", 281 | " # Initialize the viterbi variables in log space\n", 282 | " init_vvars = torch.full((1, self.tagset_size), -10000.)\n", 283 | " init_vvars[0][self.tag_to_ix[START_TAG]] = 0\n", 284 | "\n", 285 | " # forward_var at step i holds the viterbi variables for step i-1\n", 286 | " forward_var = init_vvars\n", 287 | " for feat in feats:\n", 288 | " bptrs_t = [] # holds the backpointers for this step\n", 289 | " viterbivars_t = [] # holds the viterbi variables for this step\n", 290 | "\n", 291 | " for next_tag in range(self.tagset_size):\n", 292 | " # next_tag_var[i] holds the viterbi variable for tag i at the\n", 293 | " # previous step, plus the score of transitioning\n", 294 | " # from tag i to next_tag.\n", 295 | " # We don't include the emission scores here because the max\n", 296 | " # does not depend on them (we add them in below)\n", 297 | " next_tag_var = forward_var + self.transitions[next_tag]\n", 298 | " best_tag_id = argmax(next_tag_var)\n", 299 | " bptrs_t.append(best_tag_id)\n", 300 | " viterbivars_t.append(next_tag_var[0][best_tag_id].view(1))\n", 301 | " # Now add in the emission scores, and assign forward_var to the set\n", 302 | " # of viterbi variables we just computed\n", 303 | " forward_var = (torch.cat(viterbivars_t) + feat).view(1, -1)\n", 304 | " backpointers.append(bptrs_t)\n", 305 | "\n", 306 | " # Transition to STOP_TAG\n", 307 | " terminal_var = forward_var + self.transitions[self.tag_to_ix[STOP_TAG]]\n", 308 | " best_tag_id = argmax(terminal_var)\n", 309 | " path_score = terminal_var[0][best_tag_id]\n", 310 | "\n", 311 | " # Follow the back pointers to decode the best path.\n", 312 | " best_path = [best_tag_id]\n", 313 | " for bptrs_t in reversed(backpointers):\n", 314 | " best_tag_id = bptrs_t[best_tag_id]\n", 315 | " best_path.append(best_tag_id)\n", 316 | " # Pop off the start tag (we dont want to return that to the caller)\n", 317 | " start = best_path.pop()\n", 318 | " assert start == self.tag_to_ix[START_TAG] # Sanity check\n", 319 | " best_path.reverse()\n", 320 | " return path_score, best_path\n", 321 | "\n", 322 | " def neg_log_likelihood(self, sentence, tags):\n", 323 | " feats = self._get_lstm_features(sentence)\n", 324 | " forward_score = self._forward_alg(feats)\n", 325 | " gold_score = self._score_sentence(feats, tags)\n", 326 | " return forward_score - gold_score\n", 327 | "\n", 328 | " def forward(self, sentence): # dont confuse this with _forward_alg above.\n", 329 | " # Get the emission scores from the BiLSTM\n", 330 | " lstm_feats = self._get_lstm_features(sentence)\n", 331 | "\n", 332 | " # Find the best path, given the features.\n", 333 | " score, tag_seq = self._viterbi_decode(lstm_feats)\n", 334 | " return score, tag_seq" 335 | ] 336 | }, 337 | { 338 | "cell_type": "markdown", 339 | "metadata": {}, 340 | "source": [ 341 | "Run training\n", 342 | "\n" 343 | ] 344 | }, 345 | { 346 | "cell_type": "code", 347 | "execution_count": null, 348 | "metadata": {}, 349 | "outputs": [], 350 | "source": [ 351 | "START_TAG = \"\"\n", 352 | "STOP_TAG = \"\"\n", 353 | "EMBEDDING_DIM = 5\n", 354 | "HIDDEN_DIM = 4\n", 355 | "\n", 356 | "# Make up some training data\n", 357 | "training_data = [(\n", 358 | " \"the wall street journal reported today that apple corporation made money\".split(),\n", 359 | " \"B I I I O O O B I O O\".split()\n", 360 | "), (\n", 361 | " \"georgia tech is a university in georgia\".split(),\n", 362 | " \"B I O O O O B\".split()\n", 363 | ")]\n", 364 | "\n", 365 | "word_to_ix = {}\n", 366 | "for sentence, tags in training_data:\n", 367 | " for word in sentence:\n", 368 | " if word not in word_to_ix:\n", 369 | " word_to_ix[word] = len(word_to_ix)\n", 370 | "\n", 371 | "tag_to_ix = {\"B\": 0, \"I\": 1, \"O\": 2, START_TAG: 3, STOP_TAG: 4}\n", 372 | "\n", 373 | "model = BiLSTM_CRF(len(word_to_ix), tag_to_ix, EMBEDDING_DIM, HIDDEN_DIM)\n", 374 | "optimizer = optim.SGD(model.parameters(), lr=0.01, weight_decay=1e-4)\n", 375 | "\n", 376 | "# Check predictions before training\n", 377 | "with torch.no_grad():\n", 378 | " precheck_sent = prepare_sequence(training_data[0][0], word_to_ix)\n", 379 | " precheck_tags = torch.tensor([tag_to_ix[t] for t in training_data[0][1]], dtype=torch.long)\n", 380 | " print(model(precheck_sent))\n", 381 | "\n", 382 | "# Make sure prepare_sequence from earlier in the LSTM section is loaded\n", 383 | "for epoch in range(\n", 384 | " 300): # again, normally you would NOT do 300 epochs, it is toy data\n", 385 | " for sentence, tags in training_data:\n", 386 | " # Step 1. Remember that Pytorch accumulates gradients.\n", 387 | " # We need to clear them out before each instance\n", 388 | " model.zero_grad()\n", 389 | "\n", 390 | " # Step 2. Get our inputs ready for the network, that is,\n", 391 | " # turn them into Tensors of word indices.\n", 392 | " sentence_in = prepare_sequence(sentence, word_to_ix)\n", 393 | " targets = torch.tensor([tag_to_ix[t] for t in tags], dtype=torch.long)\n", 394 | "\n", 395 | " # Step 3. Run our forward pass.\n", 396 | " loss = model.neg_log_likelihood(sentence_in, targets)\n", 397 | "\n", 398 | " # Step 4. Compute the loss, gradients, and update the parameters by\n", 399 | " # calling optimizer.step()\n", 400 | " loss.backward()\n", 401 | " optimizer.step()\n", 402 | "\n", 403 | "# Check predictions after training\n", 404 | "with torch.no_grad():\n", 405 | " precheck_sent = prepare_sequence(training_data[0][0], word_to_ix)\n", 406 | " print(model(precheck_sent))\n", 407 | "# We got it!" 408 | ] 409 | }, 410 | { 411 | "cell_type": "markdown", 412 | "metadata": {}, 413 | "source": [ 414 | "Exercise: A new loss function for discriminative tagging\n", 415 | "--------------------------------------------------------\n", 416 | "\n", 417 | "It wasn't really necessary for us to create a computation graph when\n", 418 | "doing decoding, since we do not backpropagate from the viterbi path\n", 419 | "score. Since we have it anyway, try training the tagger where the loss\n", 420 | "function is the difference between the Viterbi path score and the score\n", 421 | "of the gold-standard path. It should be clear that this function is\n", 422 | "non-negative and 0 when the predicted tag sequence is the correct tag\n", 423 | "sequence. This is essentially *structured perceptron*.\n", 424 | "\n", 425 | "This modification should be short, since Viterbi and score\\_sentence are\n", 426 | "already implemented. This is an example of the shape of the computation\n", 427 | "graph *depending on the training instance*. Although I haven't tried\n", 428 | "implementing this in a static toolkit, I imagine that it is possible but\n", 429 | "much less straightforward.\n", 430 | "\n", 431 | "Pick up some real data and do a comparison!\n", 432 | "\n", 433 | "\n" 434 | ] 435 | } 436 | ], 437 | "metadata": { 438 | "kernelspec": { 439 | "display_name": "Python 3", 440 | "language": "python", 441 | "name": "python3" 442 | }, 443 | "language_info": { 444 | "codemirror_mode": { 445 | "name": "ipython", 446 | "version": 3 447 | }, 448 | "file_extension": ".py", 449 | "mimetype": "text/x-python", 450 | "name": "python", 451 | "nbconvert_exporter": "python", 452 | "pygments_lexer": "ipython3", 453 | "version": "3.6.2" 454 | } 455 | }, 456 | "nbformat": 4, 457 | "nbformat_minor": 1 458 | } 459 | -------------------------------------------------------------------------------- /data/initial_vector.txt: -------------------------------------------------------------------------------- 1 | A,109351,0.060112 2 | B,137125,0.075380 3 | C,101101,0.055577 4 | D,135524,0.074500 5 | F,6781,0.003728 6 | G,92134,0.050648 7 | I,11794,0.006483 8 | J,10395,0.005714 9 | K,24924,0.013701 10 | L,3962,0.002178 11 | M,1985,0.001091 12 | P,6305,0.003466 13 | S,1169907,0.643117 14 | W,897,0.000493 15 | X,6931,0.003810 16 | Z,5,0.000003 17 | -------------------------------------------------------------------------------- /data/transition_probability.txt: -------------------------------------------------------------------------------- 1 | A,A,0.0 2 | A,B,0.0 3 | A,C,0.1840012546588435 4 | A,D,0.008146057050075648 5 | A,F,0.018570795970331008 6 | A,G,0.5422801579394073 7 | A,I,0.030351673493486844 8 | A,J,0.014594634488357504 9 | A,K,0.17762648068194398 10 | A,L,0.013118565260710728 11 | A,M,0.002601572013727444 12 | A,P,0.008708808443115982 13 | A,S,0.0 14 | A,W,0.0 15 | A,X,0.0 16 | A,Z,0.0 17 | B,A,0.023407214052097173 18 | B,B,0.0 19 | B,C,0.0 20 | B,D,0.0 21 | B,F,0.0 22 | B,G,0.0 23 | B,I,0.0 24 | B,J,0.0 25 | B,K,0.0 26 | B,L,0.0 27 | B,M,0.0 28 | B,P,0.0 29 | B,S,0.0 30 | B,W,0.0 31 | B,X,0.0 32 | B,Z,0.9765927859479028 33 | C,A,0.0 34 | C,B,0.0 35 | C,C,0.29109735029503486 36 | C,D,0.6420093782953041 37 | C,F,0.001592964034910592 38 | C,G,0.021404051962038095 39 | C,I,0.009535348096295799 40 | C,J,0.017982544703955487 41 | C,K,0.0005945569989455027 42 | C,L,0.001895851562675282 43 | C,M,0.0029166947118081264 44 | C,P,0.009355859190953759 45 | C,S,0.0 46 | C,W,0.0016154001480783468 47 | C,X,0.0 48 | C,Z,0.0 49 | D,A,0.0 50 | D,B,0.8602186856864562 51 | D,C,0.0344759869919721 52 | D,D,0.050845993118941765 53 | D,F,0.00106043705716934 54 | D,G,0.00799648091998806 55 | D,I,0.003739022512685969 56 | D,J,0.0017988154525316953 57 | D,K,0.00021994250074623348 58 | D,L,0.0008247843777983755 59 | D,M,0.0013903508082886902 60 | D,P,0.0009426107174838578 61 | D,S,0.0 62 | D,W,0.0004634502694295634 63 | D,X,0.036023439586508096 64 | D,Z,0.0 65 | F,A,0.0 66 | F,B,0.0 67 | F,C,0.2253423067997215 68 | F,D,0.6187050359712231 69 | F,F,0.03202599210953817 70 | F,G,0.029473195637038756 71 | F,I,0.021350661406358783 72 | F,J,0.03782780227430958 73 | F,K,0.0011603620329542817 74 | F,L,0.0016245068461359943 75 | F,M,0.0011603620329542817 76 | F,P,0.020190299373404504 77 | F,S,0.0 78 | F,W,0.011139475516361104 79 | F,X,0.0 80 | F,Z,0.0 81 | G,A,0.0 82 | G,B,0.0 83 | G,C,0.2934756565075714 84 | G,D,0.5146875599003259 85 | G,F,0.015370423615104467 86 | G,G,0.062032777458309374 87 | G,I,0.03718612229250527 88 | G,J,0.03332854130726471 89 | G,K,0.002180371861222925 90 | G,L,0.016951792217749666 91 | G,M,0.009056929269695228 92 | G,P,0.014052616446233467 93 | G,S,0.0 94 | G,W,0.0016772091240176347 95 | G,X,0.0 96 | G,Z,0.0 97 | I,A,0.0 98 | I,B,0.0 99 | I,C,0.2777777777777778 100 | I,D,0.6136514247846255 101 | I,F,0.0037552462999779104 102 | I,G,0.02982107355864811 103 | I,I,0.019770267285177823 104 | I,J,0.019991164126352993 105 | I,K,0.0003313452617627568 106 | I,L,0.004307488402915839 107 | I,M,0.006185111552904793 108 | I,P,0.023194168323392977 109 | I,S,0.0 110 | I,W,0.0012149326264634416 111 | I,X,0.0 112 | I,Z,0.0 113 | J,A,0.0 114 | J,B,0.0 115 | J,C,0.2318202871699861 116 | J,D,0.575845298749421 117 | J,F,0.007989810097267253 118 | J,G,0.0111162575266327 119 | J,I,0.00984251968503937 120 | J,J,0.08186660490968041 121 | J,K,0.0004631773969430292 122 | J,L,0.011000463177396943 123 | J,M,0.005442334414080593 124 | J,P,0.061949976841130155 125 | J,S,0.0 126 | J,W,0.0026632700324224177 127 | J,X,0.0 128 | J,Z,0.0 129 | K,A,0.0 130 | K,B,0.8048484848484848 131 | K,C,0.04694949494949495 132 | K,D,0.08226262626262626 133 | K,F,0.0005252525252525252 134 | K,G,0.007434343434343434 135 | K,I,0.002585858585858586 136 | K,J,0.002303030303030303 137 | K,K,0.0006464646464646465 138 | K,L,0.00012121212121212121 139 | K,M,0.001292929292929293 140 | K,P,0.00101010101010101 141 | K,S,0.0 142 | K,W,0.0 143 | K,X,0.05002020202020202 144 | K,Z,0.0 145 | L,A,0.0 146 | L,B,0.0 147 | L,C,0.3475330277702885 148 | L,D,0.3979509301698571 149 | L,F,0.005122674575357239 150 | L,G,0.018333782690752223 151 | L,I,0.01563763817740631 152 | L,J,0.12968455109193852 153 | L,K,0.00026961445133459155 154 | L,L,0.004853060124022647 155 | L,M,0.0026961445133459154 156 | L,P,0.07791857643569695 157 | L,S,0.0 158 | L,W,0.0 159 | L,X,0.0 160 | L,Z,0.0 161 | M,A,0.3597122302158273 162 | M,B,0.0 163 | M,C,0.20467625899280575 164 | M,D,0.27266187050359714 165 | M,F,0.00035971223021582735 166 | M,G,0.002517985611510791 167 | M,I,0.003956834532374101 168 | M,J,0.0460431654676259 169 | M,K,0.0 170 | M,L,0.027338129496402876 171 | M,M,0.03093525179856115 172 | M,P,0.05143884892086331 173 | M,S,0.0 174 | M,W,0.00035971223021582735 175 | M,X,0.0 176 | M,Z,0.0 177 | P,A,0.0 178 | P,B,0.0 179 | P,C,0.33455210237659966 180 | P,D,0.4040219378427788 181 | P,F,0.014218972171440178 182 | P,G,0.010968921389396709 183 | P,I,0.040625634775543366 184 | P,J,0.08632947389802965 185 | P,K,0.0 186 | P,L,0.013609587649807029 187 | P,M,0.005890717042453788 188 | P,P,0.08795449928905139 189 | P,S,0.0 190 | P,W,0.0018281535648994515 191 | P,X,0.0 192 | P,Z,0.0 193 | S,A,0.008181938643634368 194 | S,B,0.0 195 | S,C,0.0025047453351161684 196 | S,D,8.948592059501254e-05 197 | S,F,0.00023920274928282198 198 | S,G,0.010931049377298455 199 | S,I,0.0004095701750310189 200 | S,J,0.0001824136073667563 201 | S,K,0.0034684398646009184 202 | S,L,0.0001806927242783907 203 | S,M,1.9790155516204696e-05 204 | S,P,7.399797279972191e-05 205 | S,S,0.0 206 | S,W,0.0 207 | S,X,0.0 208 | S,Z,0.9737186734744802 209 | W,A,0.0 210 | W,B,0.0 211 | W,C,0.2827586206896552 212 | W,D,0.3448275862068966 213 | W,F,0.052873563218390804 214 | W,G,0.2413793103448276 215 | W,I,0.02528735632183908 216 | W,J,0.020689655172413793 217 | W,K,0.0 218 | W,L,0.009195402298850575 219 | W,M,0.0 220 | W,P,0.022988505747126436 221 | W,S,0.0 222 | W,W,0.0 223 | W,X,0.0 224 | W,Z,0.0 225 | X,A,0.0 226 | X,B,0.0 227 | X,C,0.20140796703296704 228 | X,D,0.008585164835164836 229 | X,F,0.015625 230 | X,G,0.5103021978021978 231 | X,I,0.02712912087912088 232 | X,J,0.013221153846153846 233 | X,K,0.20140796703296704 234 | X,L,0.01356456043956044 235 | X,M,0.002918956043956044 236 | X,P,0.005837912087912088 237 | X,S,0.0 238 | X,W,0.0 239 | X,X,0.0 240 | X,Z,0.0 241 | Z,A,0.004819724827162991 242 | Z,B,0.0 243 | Z,C,0.0 244 | Z,D,0.0 245 | Z,F,0.0 246 | Z,G,0.0 247 | Z,I,0.0 248 | Z,J,0.0 249 | Z,K,0.0 250 | Z,L,0.0 251 | Z,M,0.0 252 | Z,P,0.0 253 | Z,S,0.0 254 | Z,W,0.0 255 | Z,X,0.0 256 | Z,Z,0.995180275172837 257 | -------------------------------------------------------------------------------- /graph_model.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 73, 6 | "metadata": { 7 | "scrolled": true 8 | }, 9 | "outputs": [], 10 | "source": [ 11 | "import jieba\n", 12 | "import re\n", 13 | "import jieba.posseg as pseg" 14 | ] 15 | }, 16 | { 17 | "cell_type": "markdown", 18 | "metadata": {}, 19 | "source": [ 20 | "# Demonstration of inference in HMM. \n", 21 | "`./data/initial_vector.txt` contains initial probability $\\pi$ \n", 22 | "`./data/emit_probability.txt` contains the transition probabilities $A$ \n", 23 | "`./data/emit_probability.txt` contains the emission probability $p(y_{t+1}|y_t)$ \n", 24 | "This is a program for recognizing chinese organization, the calculation of above probability is based on `People's daily`\n" 25 | ] 26 | }, 27 | { 28 | "cell_type": "code", 29 | "execution_count": 114, 30 | "metadata": {}, 31 | "outputs": [ 32 | { 33 | "name": "stdout", 34 | "output_type": "stream", 35 | "text": [ 36 | "['始##始', '人民日报', '出版社', '头版头条', '末##末']\n", 37 | "['S', 'I', 'D', 'A', 'A']\n", 38 | "===识别的实体===\n", 39 | "人民日报出版社\n", 40 | "--------------------------------------------------------------------------------\n", 41 | "['始##始', '新', '中国', '成立', '啦', '末##末']\n", 42 | "['S', 'I', 'D', 'A', 'A']\n", 43 | "===识别的实体===\n", 44 | "新中国\n", 45 | "--------------------------------------------------------------------------------\n", 46 | "['始##始', '中国移动', '研究所', '在', '京', '召开大会', '末##末']\n", 47 | "['S', 'I', 'D', 'A', 'A']\n", 48 | "===识别的实体===\n", 49 | "中国移动研究所\n", 50 | "--------------------------------------------------------------------------------\n" 51 | ] 52 | } 53 | ], 54 | "source": [ 55 | "import jieba\n", 56 | "class OrgRecognize:\n", 57 | " def __init__(self):\n", 58 | " self.hidden_states = [\"A\", \"B\", \"C\", \"D\",\"F\",\"G\",\"I\",\"J\",\"K\",\"L\",\"M\",\"P\",\"S\",\"W\",\"X\",\"Z\"]\n", 59 | " self.initial_vector = self.load_initial_vector()\n", 60 | " self.transision_matrix = self.load_transition_matrix(hidden_states=self.hidden_states)\n", 61 | " self.emission_matrix = self.load_emission_matrix(hidden_states=self.hidden_states)\n", 62 | "\n", 63 | " def load_patterns(self):\n", 64 | " \"\"\"\n", 65 | " organization pattern\n", 66 | " :return: list all patterns\n", 67 | " \"\"\"\n", 68 | " result = []\n", 69 | " with open(\"./data/nt.pattern.txt\", \"r\") as file:\n", 70 | " datas = file.readlines()\n", 71 | " for line in datas:\n", 72 | " result.append(line.strip())\n", 73 | " return result\n", 74 | "\n", 75 | " def load_transition_matrix(self,hidden_states):\n", 76 | " \"\"\"\n", 77 | " Load transition matrix\n", 78 | " :return: dict：key first state，value dict--key is next state，value is the corresponding probability\n", 79 | " \"\"\"\n", 80 | " result = {x: {} for x in hidden_states}\n", 81 | " with open(\"./data/transition_probability.txt\",\"r\") as file:\n", 82 | " datas = file.readlines()\n", 83 | " for line in datas:\n", 84 | " split_line = line.strip().split(\",\")\n", 85 | " result[split_line[0]][split_line[1]] = split_line[2]\n", 86 | " return result\n", 87 | " \n", 88 | " def load_initial_vector(self):\n", 89 | " \"\"\"\n", 90 | " Load initial probabilities\n", 91 | " :return: dict：key is the hidden state，value is the probability\n", 92 | " \"\"\"\n", 93 | " result = {}\n", 94 | " with open(\"./data/initial_vector.txt\",\"r\") as file:\n", 95 | " datas = file.readlines()\n", 96 | " for line in datas:\n", 97 | " split_line = line.strip().split(\",\")\n", 98 | " result[split_line[0]] = split_line[2]\n", 99 | " return result\n", 100 | "\n", 101 | " def load_emission_matrix(self,hidden_states):\n", 102 | " \"\"\"\n", 103 | " Load emission matrix\n", 104 | " :param hidden_states: list of hidden state\n", 105 | " :return: dict：key is the hidden state，value is a dict key is observable var，value is the probability\n", 106 | " \"\"\"\n", 107 | " result = {x:{} for x in hidden_states}\n", 108 | " with open(\"./data/emit_probability.txt\",\"r\") as file:\n", 109 | " datas = file.readlines()\n", 110 | " for line in datas:\n", 111 | " split_line = line.strip().split(\",\")\n", 112 | " result[split_line[0]][split_line[1]] = split_line[2]\n", 113 | " return result\n", 114 | " def get_observed_states(self,sentence):\n", 115 | " return sentence\n", 116 | "\n", 117 | " def viterbi(self, input_sentence):\n", 118 | " \"\"\"\n", 119 | " Inference the best hidden state sequence\n", 120 | " input_setence: observable tokens\n", 121 | " :return: best state sequence\n", 122 | " \"\"\"\n", 123 | " hidden_states=self.hidden_states\n", 124 | " initial_probability=self.initial_vector\n", 125 | " transition_probability=self.transision_matrix\n", 126 | " emit_probability=self.emission_matrix\n", 127 | " observed_states = self.get_observed_states(sentence=input_sentence)\n", 128 | " self.observed_states = observed_states\n", 129 | " result = []\n", 130 | " compute_recode = [] #记录每一次的计算结果\n", 131 | " #初始化\n", 132 | " tmp_result = {}\n", 133 | " for state in hidden_states:\n", 134 | " if observation[0] in emit_probability[state] :\n", 135 | " tmp_result[state] = eval(initial_probability[state])*eval(emit_probability[state][observation[0]])\n", 136 | " else:\n", 137 | " tmp_result[state] = 0\n", 138 | " compute_recode.append(tmp_result)\n", 139 | "\n", 140 | " #对于之后的词语，继续计算\n", 141 | " for index,word in enumerate(observation[1:]):\n", 142 | " tmp_result = {}\n", 143 | " for current_state in hidden_states:\n", 144 | " #取最大值：上一次的所有状态(x)*转移到当前状态（current_state）*发射概率\n", 145 | " if word in emit_probability[current_state]:\n", 146 | " tmp_result[current_state] = max([compute_recode[index][x]*eval(transition_probability[x][current_state])*\n", 147 | " eval(emit_probability[current_state][word]) for x in hidden_states])\n", 148 | " else:\n", 149 | " tmp_result[current_state] = 0\n", 150 | " compute_recode.append(tmp_result)\n", 151 | "\n", 152 | " #返回概率最大的标签序列\n", 153 | " tag_sequence = []\n", 154 | " for recode in compute_recode:\n", 155 | " tag_sequence.append(max(recode, key=recode.get))\n", 156 | " return tag_sequence\n", 157 | " def get_organization(self, observation, sequence, patterns):\n", 158 | " \"\"\"\n", 159 | " 得到识别的机构名\n", 160 | " :param observation: 单词序列\n", 161 | " :param sequence: 标注序列\n", 162 | " :param patterns: 模式串\n", 163 | " :return: list，机构名\n", 164 | " \"\"\"\n", 165 | " org_indices = [] # 存放机构名的索引\n", 166 | " orgs = [] # 存放机构名字符串\n", 167 | " tag_sequence_str = \"\".join(tag_sequence) # 转为字符串\n", 168 | " for pattern in patterns:\n", 169 | " if pattern in tag_sequence_str:\n", 170 | " start_index = (tag_sequence_str.index(pattern))\n", 171 | " end_index = start_index + len(pattern)\n", 172 | " org_indices.append([start_index,end_index])\n", 173 | " if len(org_indices)!=0:\n", 174 | " for start,end in org_indices:\n", 175 | " orgs.append(\"\".join(observation[start:end]))\n", 176 | " return orgs\n", 177 | "\n", 178 | " \n", 179 | "def ner(orgrecog, setence):\n", 180 | " sentence = [\"始##始\"]\n", 181 | " sentence.extend(list(jieba.cut(sentence_str)))\n", 182 | " sentence.append(\"末##末\")\n", 183 | " print(sentence)\n", 184 | " tag_sequence = orgrecog.viterbi(sentence)\n", 185 | " print( tag_sequence )\n", 186 | " patterns = orgrecog.load_patterns()\n", 187 | " results = orgrecog.get_organization(sentence,tag_sequence,patterns)\n", 188 | " \n", 189 | " if len(results) == 0:\n", 190 | " print (\"未识别到机构名\")\n", 191 | " print (tag_sequence)\n", 192 | " else:\n", 193 | " print('===识别的实体===')\n", 194 | " for result in results:\n", 195 | " print (result)\n", 196 | " print('-'*80)\n", 197 | " \n", 198 | "if __name__ == '__main__':\n", 199 | " orgrecog = OrgRecognize() \n", 200 | " # Note the corpus is too old, so some new organization cannot be recognized\n", 201 | " sentence_str = \"人民日报出版社头版头条\"\n", 202 | " ner(orgrecog, sentence_str)\n", 203 | " sentence_str = \"新中国成立啦\"\n", 204 | " ner(orgrecog, sentence_str)\n", 205 | " sentence_str = \"中国移动研究所在京召开大会\"\n", 206 | " ner(orgrecog, sentence_str)\n" 207 | ] 208 | }, 209 | { 210 | "cell_type": "markdown", 211 | "metadata": {}, 212 | "source": [ 213 | "# Demonstration of CRF for sequence labeling\n", 214 | "reference: [link](http://www.albertauyeung.com/post/python-sequence-labelling-with-crf/) \n", 215 | "The task is to identify country names, the code include training and test." 216 | ] 217 | }, 218 | { 219 | "cell_type": "code", 220 | "execution_count": 135, 221 | "metadata": { 222 | "scrolled": false 223 | }, 224 | "outputs": [ 225 | { 226 | "name": "stdout", 227 | "output_type": "stream", 228 | "text": [ 229 | "blocker (N) || energy (N) || corp (N) || said (I) || an (I) || offering (I) || of (I) || 20 (I) || mln (I) || common (I) || shares (I) || is (I) || underway (I) || at (I) || 2.625 (I) || dlrs (I) || per (I) || share (I) || through (I) || underwriters (I) || led (I) || by (I) || drexel (N) || burnham (N) || lambert (N) || inc (N) || and (I) || alex. (N) || brown (N) || and (N) || sons (N) || inc (N) || absb. (I) || the (I) || company (I) || is (I) || offering (I) || 19.7 (I) || mln (I) || shares (I) || and (I) || shareholders (I) || the (I) || rest. (I) || before (I) || the (I) || offering (I) || it (I) || had (I) || about (I) || 33.6 (I) || mln (I) || shares (I) || outstanding. (I)\n", 230 | " precision recall f1-score support\n", 231 | "\n", 232 | " I 0.98 0.98 0.98 2805\n", 233 | " N 0.85 0.89 0.87 437\n", 234 | "\n", 235 | " micro avg 0.96 0.96 0.96 3242\n", 236 | " macro avg 0.92 0.93 0.92 3242\n", 237 | "weighted avg 0.96 0.96 0.96 3242\n", 238 | "\n" 239 | ] 240 | } 241 | ], 242 | "source": [ 243 | "import codecs\n", 244 | "import numpy as np\n", 245 | "import nltk\n", 246 | "import pycrfsuite\n", 247 | "from bs4 import BeautifulSoup as bs\n", 248 | "from bs4.element import Tag\n", 249 | "from sklearn.model_selection import train_test_split\n", 250 | "from sklearn.metrics import classification_report\n", 251 | "\n", 252 | "# Read data file and parse the XML\n", 253 | "with codecs.open(\"n3-collection/reuters.xml\", \"r\", \"utf-8\") as infile:\n", 254 | " soup = bs(infile, \"html5lib\")\n", 255 | "\n", 256 | "docs = []\n", 257 | "for elem in soup.find_all(\"document\"):\n", 258 | " texts = []\n", 259 | "\n", 260 | " # Loop through each child of the element under \"textwithnamedentities\"\n", 261 | " for c in elem.find(\"textwithnamedentities\").children:\n", 262 | " if type(c) == Tag:\n", 263 | " if c.name == \"namedentityintext\":\n", 264 | " label = \"N\" # part of a named entity\n", 265 | " else:\n", 266 | " label = \"I\" # irrelevant word\n", 267 | " for w in c.text.split(\" \"):\n", 268 | " if len(w) > 0:\n", 269 | " texts.append((w, label))\n", 270 | " docs.append(texts)\n", 271 | "\n", 272 | "\n", 273 | "data = []\n", 274 | "for i, doc in enumerate(docs):\n", 275 | "\n", 276 | " # Obtain the list of tokens in the document\n", 277 | " tokens = [t for t, label in doc]\n", 278 | "\n", 279 | " # Perform POS tagging\n", 280 | " tagged = nltk.pos_tag(tokens)\n", 281 | "\n", 282 | " # Take the word, POS tag, and its label\n", 283 | " data.append([(w, pos, label) for (w, label), (word, pos) in zip(doc, tagged)])\n", 284 | "\n", 285 | "\n", 286 | "def word2features(doc, i):\n", 287 | " word = doc[i][0]\n", 288 | " postag = doc[i][1]\n", 289 | "\n", 290 | " # Common features for all words\n", 291 | " features = [\n", 292 | " 'bias',\n", 293 | " 'word.lower=' + word.lower(),\n", 294 | " 'word[-3:]=' + word[-3:],\n", 295 | " 'word[-2:]=' + word[-2:],\n", 296 | " 'word.isupper=%s' % word.isupper(),\n", 297 | " 'word.istitle=%s' % word.istitle(),\n", 298 | " 'word.isdigit=%s' % word.isdigit(),\n", 299 | " 'postag=' + postag\n", 300 | " ]\n", 301 | "\n", 302 | " # Features for words that are not\n", 303 | " # at the beginning of a document\n", 304 | " if i > 0:\n", 305 | " word1 = doc[i-1][0]\n", 306 | " postag1 = doc[i-1][1]\n", 307 | " features.extend([\n", 308 | " '-1:word.lower=' + word1.lower(),\n", 309 | " '-1:word.istitle=%s' % word1.istitle(),\n", 310 | " '-1:word.isupper=%s' % word1.isupper(),\n", 311 | " '-1:word.isdigit=%s' % word1.isdigit(),\n", 312 | " '-1:postag=' + postag1\n", 313 | " ])\n", 314 | " else:\n", 315 | " # Indicate that it is the 'beginning of a document'\n", 316 | " features.append('BOS')\n", 317 | "\n", 318 | " # Features for words that are not \n", 319 | " # at the end of a document\n", 320 | " if i < len(doc)-1:\n", 321 | " word1 = doc[i+1][0]\n", 322 | " postag1 = doc[i+1][1]\n", 323 | " features.extend([\n", 324 | " '+1:word.lower=' + word1.lower(),\n", 325 | " '+1:word.istitle=%s' % word1.istitle(),\n", 326 | " '+1:word.isupper=%s' % word1.isupper(),\n", 327 | " '+1:word.isdigit=%s' % word1.isdigit(),\n", 328 | " '+1:postag=' + postag1\n", 329 | " ])\n", 330 | " else:\n", 331 | " # Indicate that it is the 'end of a document'\n", 332 | " features.append('EOS')\n", 333 | "\n", 334 | " return features\n", 335 | "\n", 336 | "# A function for extracting features in documents\n", 337 | "def extract_features(doc):\n", 338 | " return [word2features(doc, i) for i in range(len(doc))]\n", 339 | "\n", 340 | "# A function fo generating the list of labels for each document\n", 341 | "def get_labels(doc):\n", 342 | " return [label for (token, postag, label) in doc]\n", 343 | "\n", 344 | "\n", 345 | "X = [extract_features(doc) for doc in data]\n", 346 | "y = [get_labels(doc) for doc in data]\n", 347 | "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)\n", 348 | "\n", 349 | "trainer = pycrfsuite.Trainer(verbose=False)\n", 350 | "\n", 351 | "# Submit training data to the trainer\n", 352 | "for xseq, yseq in zip(X_train, y_train):\n", 353 | " trainer.append(xseq, yseq)\n", 354 | "\n", 355 | "# Set the parameters of the model\n", 356 | "trainer.set_params({\n", 357 | " # coefficient for L1 penalty\n", 358 | " 'c1': 0.1,\n", 359 | "\n", 360 | " # coefficient for L2 penalty\n", 361 | " 'c2': 0.01, \n", 362 | "\n", 363 | " # maximum number of iterations\n", 364 | " 'max_iterations': 200,\n", 365 | "\n", 366 | " # whether to include transitions that\n", 367 | " # are possible, but not observed\n", 368 | " 'feature.possible_transitions': True\n", 369 | "})\n", 370 | "\n", 371 | "# Provide a file name as a parameter to the train function, such that\n", 372 | "# the model will be saved to the file when training is finished\n", 373 | "trainer.train('crf.model')\n", 374 | "\n", 375 | "# Generate predictions\n", 376 | "tagger = pycrfsuite.Tagger()\n", 377 | "tagger.open('crf.model')\n", 378 | "y_pred = [tagger.tag(xseq) for xseq in X_test]\n", 379 | "\n", 380 | "# Let's take a look at a random sample in the testing set\n", 381 | "i = 12\n", 382 | "outs = []\n", 383 | "for x, y in zip(y_pred[i], [x[1].split(\"=\")[1] for x in X_test[i]]):\n", 384 | " outs.append(\"%s (%s)\" % (y, x))\n", 385 | "print(' || '.join(outs))\n", 386 | "\n", 387 | "# Create a mapping of labels to indices\n", 388 | "labels = {\"N\": 1, \"I\": 0}\n", 389 | "\n", 390 | "# Convert the sequences of tags into a 1-dimensional array\n", 391 | "predictions = np.array([labels[tag] for row in y_pred for tag in row])\n", 392 | "truths = np.array([labels[tag] for row in y_test for tag in row])\n", 393 | "\n", 394 | "# Print out the classification report\n", 395 | "print(classification_report(\n", 396 | " truths, predictions,\n", 397 | " target_names=[\"I\", \"N\"]))" 398 | ] 399 | }, 400 | { 401 | "cell_type": "markdown", 402 | "metadata": {}, 403 | "source": [ 404 | "# Demonstration of PLSA\n", 405 | "Here we provide a implementation of PLSA from [link](https://github.com/isnowfy/plsa.git)" 406 | ] 407 | }, 408 | { 409 | "cell_type": "code", 410 | "execution_count": 141, 411 | "metadata": {}, 412 | "outputs": [ 413 | { 414 | "name": "stdout", 415 | "output_type": "stream", 416 | "text": [ 417 | "0 iter\n", 418 | "likelihood 2.123584 \n", 419 | "1 iter\n", 420 | "likelihood -12.995615 \n", 421 | "2 iter\n", 422 | "likelihood -11.750363 \n", 423 | "3 iter\n", 424 | "likelihood -10.234484 \n", 425 | "4 iter\n", 426 | "likelihood -8.967368 \n", 427 | "5 iter\n", 428 | "likelihood -8.107269 \n", 429 | "6 iter\n", 430 | "likelihood -7.478987 \n", 431 | "7 iter\n", 432 | "likelihood -7.054530 \n", 433 | "8 iter\n", 434 | "likelihood -6.857159 \n", 435 | "9 iter\n", 436 | "likelihood -6.811286 \n", 437 | "10 iter\n", 438 | "likelihood -6.807646 \n", 439 | "11 iter\n", 440 | "likelihood -6.807588 \n", 441 | "12 iter\n", 442 | "likelihood -6.807588 \n", 443 | "13 iter\n", 444 | "likelihood -6.807588 \n", 445 | "0 iter\n", 446 | "likelihood 4.392996 \n", 447 | "1 iter\n", 448 | "likelihood -14.747961 \n", 449 | "2 iter\n", 450 | "likelihood -14.542116 \n", 451 | "3 iter\n", 452 | "likelihood -14.353453 \n", 453 | "4 iter\n", 454 | "likelihood -14.086266 \n", 455 | "5 iter\n", 456 | "likelihood -13.601759 \n", 457 | "6 iter\n", 458 | "likelihood -12.697529 \n", 459 | "7 iter\n", 460 | "likelihood -11.290563 \n", 461 | "8 iter\n", 462 | "likelihood -9.747169 \n", 463 | "9 iter\n", 464 | "likelihood -8.513129 \n", 465 | "10 iter\n", 466 | "likelihood -7.593646 \n", 467 | "11 iter\n", 468 | "likelihood -7.028783 \n", 469 | "12 iter\n", 470 | "likelihood -6.834483 \n", 471 | "13 iter\n", 472 | "likelihood -6.808457 \n", 473 | "14 iter\n", 474 | "likelihood -6.807592 \n", 475 | "15 iter\n", 476 | "likelihood -6.807588 \n", 477 | "16 iter\n", 478 | "likelihood -6.807588 \n" 479 | ] 480 | } 481 | ], 482 | "source": [ 483 | "\n", 484 | "import math\n", 485 | "import operator\n", 486 | "import random\n", 487 | "import gzip\n", 488 | "import sys\n", 489 | "import marshal\n", 490 | "from functools import reduce\n", 491 | "\n", 492 | "def cos_sim(p, q):\n", 493 | " sum0 = sum(map(lambda x:x*x, p))\n", 494 | " sum1 = sum(map(lambda x:x*x, q))\n", 495 | " sum2 = sum(map(lambda x:x[0]*x[1], zip(p, q)))\n", 496 | " return sum2/(sum0**0.5)/(sum1**0.5)\n", 497 | "\n", 498 | "def _rand_mat(sizex, sizey):\n", 499 | " ret = []\n", 500 | " for i in range(sizex):\n", 501 | " ret.append([])\n", 502 | " for _ in range(sizey):\n", 503 | " ret[-1].append(random.random())\n", 504 | " norm = sum(ret[-1])\n", 505 | " for j in range(sizey):\n", 506 | " ret[-1][j] /= norm\n", 507 | " return ret\n", 508 | "\n", 509 | "class Plsa:\n", 510 | "\n", 511 | " def __init__(self, corpus, topics=2):\n", 512 | " self.topics = topics\n", 513 | " self.corpus = corpus\n", 514 | " self.docs = len(corpus)\n", 515 | " self.each = list(map(sum, map(lambda x:x.values(), corpus)))\n", 516 | " self.words = max(reduce(operator.add, map( lambda x:list(x.keys()), corpus)))+1\n", 517 | " self.likelihood = 0\n", 518 | " self.zw = _rand_mat(self.topics, self.words)\n", 519 | " self.dz = _rand_mat(self.docs, self.topics)\n", 520 | " self.dw_z = None\n", 521 | " self.p_dw = []\n", 522 | " self.beta = 0.8\n", 523 | "\n", 524 | " def save(self, fname, iszip=True):\n", 525 | " d = {}\n", 526 | " for k, v in self.__dict__.items():\n", 527 | " if hasattr(v, '__dict__'):\n", 528 | " d[k] = v.__dict__\n", 529 | " else:\n", 530 | " d[k] = v\n", 531 | " if sys.version_info[0] == 3:\n", 532 | " fname = fname + '.3'\n", 533 | " if not iszip:\n", 534 | " marshal.dump(d, open(fname, 'wb'))\n", 535 | " else:\n", 536 | " f = gzip.open(fname, 'wb')\n", 537 | " f.write(marshal.dumps(d))\n", 538 | " f.close()\n", 539 | "\n", 540 | " def load(self, fname, iszip=True):\n", 541 | " if sys.version_info[0] == 3:\n", 542 | " fname = fname + '.3'\n", 543 | " if not iszip:\n", 544 | " d = marshal.load(open(fname, 'rb'))\n", 545 | " else:\n", 546 | " try:\n", 547 | " f = gzip.open(fname, 'rb')\n", 548 | " d = marshal.loads(f.read())\n", 549 | " except IOError:\n", 550 | " f = open(fname, 'rb')\n", 551 | " d = marshal.loads(f.read())\n", 552 | " f.close()\n", 553 | " for k, v in d.items():\n", 554 | " if hasattr(self.__dict__[k], '__dict__'):\n", 555 | " self.__dict__[k].__dict__ = v\n", 556 | " else:\n", 557 | " self.__dict__[k] = v\n", 558 | "\n", 559 | " def _cal_p_dw(self):\n", 560 | " self.p_dw = []\n", 561 | " for d in range(self.docs):\n", 562 | " self.p_dw.append({})\n", 563 | " for w in self.corpus[d]:\n", 564 | " tmp = 0\n", 565 | " for _ in range(self.corpus[d][w]):\n", 566 | " for z in range(self.topics):\n", 567 | " tmp += (self.zw[z][w]*self.dz[d][z])**self.beta\n", 568 | " self.p_dw[-1][w] = tmp\n", 569 | "\n", 570 | " def _e_step(self):\n", 571 | " self._cal_p_dw()\n", 572 | " self.dw_z = []\n", 573 | " for d in range(self.docs):\n", 574 | " self.dw_z.append({})\n", 575 | " for w in self.corpus[d]:\n", 576 | " self.dw_z[-1][w] = []\n", 577 | " for z in range(self.topics):\n", 578 | " self.dw_z[-1][w].append(((self.zw[z][w]*self.dz[d][z])**self.beta)/self.p_dw[d][w])\n", 579 | "\n", 580 | " def _m_step(self):\n", 581 | " for z in range(self.topics):\n", 582 | " self.zw[z] = [0]*self.words\n", 583 | " for d in range(self.docs):\n", 584 | " for w in self.corpus[d]:\n", 585 | " self.zw[z][w] += self.corpus[d][w]*self.dw_z[d][w][z]\n", 586 | " norm = sum(self.zw[z])\n", 587 | " for w in range(self.words):\n", 588 | " self.zw[z][w] /= norm\n", 589 | " for d in range(self.docs):\n", 590 | " self.dz[d] = [0]*self.topics\n", 591 | " for z in range(self.topics):\n", 592 | " for w in self.corpus[d]:\n", 593 | " self.dz[d][z] += self.corpus[d][w]*self.dw_z[d][w][z]\n", 594 | " for z in range(self.topics):\n", 595 | " self.dz[d][z] /= self.each[d]\n", 596 | "\n", 597 | " def _cal_likelihood(self):\n", 598 | " self.likelihood = 0\n", 599 | " for d in range(self.docs):\n", 600 | " for w in self.corpus[d]:\n", 601 | " self.likelihood += self.corpus[d][w]*math.log(self.p_dw[d][w])\n", 602 | "\n", 603 | " def train(self, max_iter=100):\n", 604 | " cur = 0\n", 605 | " for i in range(max_iter):\n", 606 | " print ('%d iter' % i)\n", 607 | " self._e_step()\n", 608 | " self._m_step()\n", 609 | " self._cal_likelihood()\n", 610 | " print ('likelihood %f ' % self.likelihood)\n", 611 | " if cur != 0 and abs((self.likelihood-cur)/cur) < 1e-8:\n", 612 | " break\n", 613 | " cur = self.likelihood\n", 614 | "\n", 615 | " def inference(self, doc, max_iter=100):\n", 616 | " doc = dict(filter(lambda x:x[0]cos_sim(p.dz[0], p.dz[2])\n", 672 | " assert p.post_prob_sim(p.corpus[0], p.dz[1])>p.post_prob_sim(p.corpus[0], p.dz[2])\n", 673 | "\n", 674 | "def test_inference():\n", 675 | " corpus = [{0:2,3:5},{0:5,2:1},{1:2,4:5}]\n", 676 | " p = Plsa(corpus)\n", 677 | " p.train()\n", 678 | " z = p.inference({0:4, 6:7})\n", 679 | " assert abs(cos_sim(p.dz[0], p.dz[1])-cos_sim(p.dz[0], z))<1e-8\n", 680 | "test_train()\n", 681 | "test_inference()\n" 682 | ] 683 | }, 684 | { 685 | "cell_type": "markdown", 686 | "metadata": {}, 687 | "source": [ 688 | "# Demonstration of LDA for text analysis\n", 689 | "\n", 690 | "Analyze the topic in chapters of 《Journey to the West》 " 691 | ] 692 | }, 693 | { 694 | "cell_type": "code", 695 | "execution_count": 115, 696 | "metadata": {}, 697 | "outputs": [], 698 | "source": [ 699 | "def load_weicheng(filename='story.txt'):\n", 700 | " lines = open(filename).readlines()\n", 701 | " books = []\n", 702 | " for line in lines:\n", 703 | " line = line.strip()\n", 704 | " if not line:\n", 705 | " continue\n", 706 | " if line.startswith('第') and line.endswith('章'):\n", 707 | " books.append(line)\n", 708 | " else:\n", 709 | " books[-1] += line\n", 710 | " for i in range(len(books)):\n", 711 | " books[i] = ' '.join(jieba.cut(books[i]))\n", 712 | " return books\n", 713 | "\n", 714 | "def load_honglou(filename='honglou.txt'):\n", 715 | " lines = open(filename).readlines()\n", 716 | " book = [[]]\n", 717 | " for line in lines:\n", 718 | " line = line.strip()\n", 719 | " if not line:\n", 720 | " continue\n", 721 | " if '书香屋' in line:\n", 722 | " book[-1].pop()\n", 723 | " book.append([])\n", 724 | " else:\n", 725 | " book[-1].append(line)\n", 726 | " return [' '.join(jieba.cut(''.join(chp))) for chp in book]\n", 727 | "\n", 728 | "def load_xiyou(filename='xiyouji_wuchengen.txt'):\n", 729 | " \n", 730 | " def _is_sep(line):\n", 731 | " pat = re.compile(u'第.{1,3}回')\n", 732 | " if pat.search(line):\n", 733 | " return True\n", 734 | " return False\n", 735 | " \n", 736 | " lines = open(filename).readlines()\n", 737 | " book = [[]]\n", 738 | " for line in lines:\n", 739 | " line = line.strip()\n", 740 | " if not line:\n", 741 | " continue\n", 742 | " if _is_sep(line):\n", 743 | " book.append([])\n", 744 | " else:\n", 745 | " book[-1].append(line)\n", 746 | " print(len(book))\n", 747 | " book = [''.join(chp) for chp in book if len(chp)]\n", 748 | " book_noun = []\n", 749 | " for chp in book:\n", 750 | " words = pseg.cut(chp)\n", 751 | " book_noun.append( ' '.join([word for word, flag in words if flag[0] == 'n']))\n", 752 | " # book_noun.append( ' '.join(jieba.cut(chp)))\n", 753 | " return book_noun" 754 | ] 755 | }, 756 | { 757 | "cell_type": "code", 758 | "execution_count": 117, 759 | "metadata": { 760 | "scrolled": false 761 | }, 762 | "outputs": [ 763 | { 764 | "name": "stdout", 765 | "output_type": "stream", 766 | "text": [ 767 | "122\n", 768 | "\n" 769 | ] 770 | }, 771 | { 772 | "name": "stderr", 773 | "output_type": "stream", 774 | "text": [ 775 | "/users/sulixin/anaconda2/envs/py36/lib/python3.6/site-packages/sklearn/decomposition/online_lda.py:314: DeprecationWarning: n_topics has been renamed to n_components in version 0.19 and will be removed in 0.21\n", 776 | " DeprecationWarning)\n" 777 | ] 778 | }, 779 | { 780 | "name": "stdout", 781 | "output_type": "stream", 782 | "text": [ 783 | "\n", 784 | "LDA\n", 785 | "Topic 0:三藏 师父 行者 形容 大圣 这里 一个 八戒 意思 后来 不知 唐僧 那里 不是 所以 古代 小龙 土地 故事 甚么\n", 786 | "Topic 1:行者 八戒 师父 三藏 一个 大圣 沙僧 妖精 那里 怎么 唐僧 我们 不知 不是 菩萨 和尚 呆子 两个 只见 三个\n", 787 | "Topic 2:意思 这里 佛教 道教 形容 后来 古代 一种 传说 称为 故事 比喻 神仙 所以 一个 这是 就是 地方 又称 认为\n", 788 | "Topic 3:行者 八戒 师父 三藏 一个 唐僧 沙僧 怎么 那里 我们 大圣 和尚 妖精 不知 两个 菩萨 甚么 长老 不是 国王\n", 789 | "Topic 4:菩萨 太宗 御弟 南无 女王 袈裟 玄奘 三藏 长老 唐王 太师 圣僧 法师 行者 真经 锡杖 唐僧 取经 师父 徒弟\n", 790 | "Topic 5:大圣 菩萨 行者 天王 悟空 玉帝 一个 猴王 如来 不知 太宗 只见 那里 如何 两个 太子 大王 陛下 哪吒 龙王\n", 791 | "Topic 6:行者 菩萨 一个 这里 八戒 甚么 师父 唐僧 称为 就是 不知 袈裟 乃是 后来 猴王 两个 原来 那个 一种 今日\n", 792 | "Topic 7:八戒 国王 公主 三藏 师父 长老 唐僧 行者 菩萨 沙僧 两个 徒弟 和尚 陛下 如何 意思 不敢 不知 驸马 一个\n", 793 | "Topic 8:光蕊 玄奘 丞相 婆婆 母亲 龙王 我儿 和尚 打死 一个 师父 今日 夫人 只见 父母 长老 唐王 孩儿 夜叉 报仇\n", 794 | "Topic 9:八戒 行者 菩萨 师父 一个 佛教 大圣 意思 这里 三藏 后来 悟空 怎么 取经 甚么 不知 不是 故事 形容 沙僧\n" 795 | ] 796 | } 797 | ], 798 | "source": [ 799 | "from sklearn.feature_extraction.text import TfidfVectorizer, CountVectorizer\n", 800 | "from sklearn.datasets import fetch_20newsgroups\n", 801 | "from sklearn.decomposition import NMF, LatentDirichletAllocation\n", 802 | "\n", 803 | "def display_topics(model, feature_names, no_top_words):\n", 804 | " for topic_idx, topic in enumerate(model.components_):\n", 805 | " print (\"Topic %d:\" % (topic_idx) + \" \".join([feature_names[i]\n", 806 | " for i in topic.argsort()[:-no_top_words - 1:-1]]) )\n", 807 | "\n", 808 | "\n", 809 | "documents = load_xiyou()\n", 810 | "print()\n", 811 | "no_features = 1000\n", 812 | "\n", 813 | "# NMF is able to use tf-idf\n", 814 | "tfidf_vectorizer = TfidfVectorizer(max_df=0.95, min_df=2, max_features=no_features, stop_words='english')\n", 815 | "tfidf = tfidf_vectorizer.fit_transform(documents)\n", 816 | "tfidf_feature_names = tfidf_vectorizer.get_feature_names()\n", 817 | "\n", 818 | "# LDA can only use raw term counts for LDA because it is a probabilistic graphical model\n", 819 | "tf_vectorizer = CountVectorizer(max_df=0.95, min_df=2, max_features=no_features, stop_words='english')\n", 820 | "tf = tf_vectorizer.fit_transform(documents)\n", 821 | "tf_feature_names = tf_vectorizer.get_feature_names()\n", 822 | "\n", 823 | "no_topics = 10\n", 824 | "\n", 825 | "\n", 826 | "# Run LDA\n", 827 | "lda = LatentDirichletAllocation(n_topics=no_topics, max_iter=20, learning_method='online', learning_offset=50.,random_state=0).fit(tf)\n", 828 | "\n", 829 | "no_top_words = 20\n", 830 | "print('\\nLDA')\n", 831 | "display_topics(lda, tf_feature_names, no_top_words)" 832 | ] 833 | } 834 | ], 835 | "metadata": { 836 | "kernelspec": { 837 | "display_name": "Python 3", 838 | "language": "python", 839 | "name": "python3" 840 | }, 841 | "language_info": { 842 | "codemirror_mode": { 843 | "name": "ipython", 844 | "version": 3 845 | }, 846 | "file_extension": ".py", 847 | "mimetype": "text/x-python", 848 | "name": "python", 849 | "nbconvert_exporter": "python", 850 | "pygments_lexer": "ipython3", 851 | "version": "3.6.2" 852 | } 853 | }, 854 | "nbformat": 4, 855 | "nbformat_minor": 2 856 | } 857 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | scikit-learn==0.19.1 2 | beautifulsoup4==4.6.3 3 | python-crfsuite==0.9.6 4 | nltk==3.3 5 | scipy==0.19.1 6 | numpy==1.14.2 7 | jupyter==1.0.0 8 | jupyter-client==5.1.0 9 | jupyter-console==5.2.0 10 | jupyter-core==4.4.0 11 | -------------------------------------------------------------------------------- /semi_supervise.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Decision boundary of label propagation versus SVM on the Iris dataset\n", 8 | "\n", 9 | "Comparison for decision boundary generated on iris dataset between Label Propagation and SVM.\n", 10 | "This demonstrates Label Propagation learning a good boundary even with a small amount of labeled data." 11 | ] 12 | }, 13 | { 14 | "cell_type": "code", 15 | "execution_count": 2, 16 | "metadata": {}, 17 | "outputs": [ 18 | { 19 | "name": "stderr", 20 | "output_type": "stream", 21 | "text": [ 22 | "/users/sulixin/env/lib/python3.5/site-packages/scipy/sparse/csgraph/_laplacian.py:72: FutureWarning: Conversion of the second argument of issubdtype from `int` to `np.signedinteger` is deprecated. In future, it will be treated as `np.int64 == np.dtype(int).type`.\n", 23 | " if normed and (np.issubdtype(csgraph.dtype, int)\n" 24 | ] 25 | }, 26 | { 27 | "data": { 28 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW4AAAEMCAYAAADknlzeAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzsnWd4VEUXgN/Zkmw2FQglhE7ovRep\n0rugVBFBKQoIKEUUUBAEpaooTVCxoRQBld5CR0INPXQIkN6zu9k234+7hCQELB8Qyn2fJ092Z+bO\nnHv33HPPnClXSClRUVFRUXly0OS0ACoqKioq/w7VcKuoqKg8YaiGW0VFReUJQzXcKioqKk8YquFW\nUVFRecJQDbeKiorKE0aOG24hRLAQov+jPjYnEUI0EUKEZ/h+SgjRJAdFeug8qb/V/4Oq28+Mbl8R\nQjR/lG0+MMOdE8LfDyGEnxDiGyFEhBAiWQgRJoQYm9NyZYeUsoKUMvhB1yuEaCqEOCGESBBCxAoh\nVgshAjPku7uuUZLrOr2TIa+wEOKAECJOCDErS70bhBA1H7S8Gep/3HTpcZNH1W3lAeEUQqRk+Hs1\nQ35ul76nCiGuCiF6Zcir4nqgxGTReb0Q4i8hROEHLW+GNqQQIuj/rSfHPe6HyBzACygH+AIdgQv/\npSIhhO4ByvUoOQ20klL6AQWB88D8DPkTgVJAUaApMEYI0dqV9x6wFCgOvHDbUAshugOXpZSHHskZ\nqGSHqtsKN6WUXhn+lmbI+wqwAvmBl4H5QogKrrxpwCigCjBOCFHAlf4OsEpKef0Ryf+feeiGWwiR\nSwjxpxAiWggR7/pcKEuxkkKIgy7Pb60QIneG4+sKIfa5vMbj/6LbVQv4WUoZL6V0SinPSilXZqhX\nCiGGCSEuuZ68M4QQGldeXyHEXiHEHCFELIqBQwjxmhDijOs8Ngkhimao73MhxHXXORwWQjTMkOch\nhPjOddxpl2wZr1G6RyeEmCiEWC6E+N7lTZ3K6N0KIaoLIY668lYIIX4VQkzJ7gJIKSOllDczJDmA\njE/7V4HJrmt0Bvga6OvKKw5sl1ImAiFACSGEDzAWeP/vLr4QooUQ4qwQIlEI8SUgMuSVFEJsd/UC\nYoQQPwkh/Fx5PwBFgD9cXtQYV/oKl4eZKITYleEmzDFU3c453b4fQghP4EVggpQyRUq5B/gdeMVV\n5LZu30BxZoq4zvdFlIfi39X/ilC8+FghxLgsebWFEPtdv+ktIcSXQgg3V94uV7HjLt3u/g916G6k\nlA/kD7gCNM8mPY/rghgBb2AFsCZDfjBwA6gIeAKrgB9deYFALNAW5SHTwvU9b4Zj+99DnsXAKaAf\nUCqbfAnsAHKjGIqw23WhGC878BagAzyATiheTTlX2nhgX4b6ervOVQeMBCIAgyvvE2C3q63CwEkg\nPLtrh3IjWVznrEXxDg648tyAq8BwQA90QfEqptzndykCJABOwAb0daXncl2D/BnKvgSccH2eAQwF\n/FCUuwLwOfDqP9AFfyDZVZ8eeNt1PW9f3yDXb+kO5AV2AZ/dT5eA11z64w58Bhx7ULqr6vaTp9tA\nE1d+JHAZxeB6uvKqAaYs5UcBf7g+rwA6AIVc55IHWAM0/ge6UB5IARq5dHG263rePscaQF3XtSoG\nnAFGZPltgv6pDt1Tjoet3NmUqwrEZ1HuT7JcGKvrh30X+CHL8ZtwGQ/ur9weKJ7hYRSDdQFok+UC\nts7wfTCwLYNyX8tS3wbg9QzfNYAJKHqP9uOBKq7Pl7K0NfBvlHtrluthdn1uhGIIRIb8PfdS7izy\n5HZdz7qu74Vd18CQoUwL4EqG8r8Cx1EMbzXuGIOfUYzt0Hu01QfXDen6LoDw+/xWLwBH/6kuoTxM\nJOD7oPRX1e0nS7eBAq7jNSge9C5goSuvIRCRpfwAINj1uSiwHjgC9EQJNd3u6a0FdgJd79HuB8Av\nGb57un7TbPUDGAGszvLbBGVXNjsdutffowiVGIUQC11diySUC+wnhNBmKJYxpnQV5Ynrj3KBu7q6\nHQlCiASgARDwd+1KKc1SyqlSyhooT7XlwIqMXdVs2i14jzxcsnyeQY44FIMU6DrPUa6uZqIr39d1\nDrjqzdrW/YjI8NkEGIQSiywI3JCuX/gecmaLlDIOJWa91lVXiivLJ0MxHxRPGSllnJSyu5SyCoqn\nPRfFSxuL4lU1B94QQpTLprlM5+uSN/27ECK/EOIXIcQNl078yJ1rdRdCCK0Q4hMhxEVX+SuurHse\n8yhQdRvIId2WUkZIKU9LJVR0GRiD4rmCots+WQ7JqNtXpZRtpZTVUQz1ZBSPfCaKs9IRmJ3let4m\nq26novSUABBClHaFOyJcOjGV++v2P9Ghu3gUg5MjgTJAHSmlD8qTFTLEPFG8v9sUQfEiYlAu0A9S\nSr8Mf55Syk/+jQBSytsX0BPl6XyvdjPGgzMqEC5ZBmWRxUNKuc8V8xsDdANySWUwMDHDOd7Kpq3/\nwi0gUAhxr2v3d+iAfICPlDLeVV+VDPlVULrgWRmI4kGfBCoBh6SUVuCE63t2cqbL5ZI3o5xTUa5v\nJZdO9CazPmS99r1QuvPNUYxGsdtV3+tEHxGqbj8+ui25Y8/CAJ0QolSG/Hvp9gfA11LKSO7odiJK\nDzG72R9ZdduI8vC8zXzgLEoIywelZ3Q/Pf0nOnQXD9pw64UQhgx/OpS4jRlIcD3BPszmuN5CiPKu\ni/ARsFJK6UDxxDoIIVq5vC6DUKYB/W3wXggxQQhRSwjhJoQwoMTOEoBzGYqNdg0OFHbl/3qfKhcA\n7wnXoJgQwlcI0dWV540S54pGUZgPyPzEX+46NpdL9rf+Tv57sB9lgHGoEEInhOgE1L5XYSFEFyFE\nGSGERgiRFyUed9TlfQN8D4x3yVUWpTv5XZY68gFDcA1iocQTmwohvICaKF3lrKwDKrja1wHDULq2\nt/FG8YoShTI9cXSW4yOBElnKp6F4NkYUQ/WoUXX78dLtpkKIokKhMEqsfS2ke8G/AR8JITyFEM+h\nPPh/yFJHeZRY+e2ZVpeB54UQ+VFmW13LpumVQHshRAOhDDp+RGY76g0kASmue+rNLMdnp9t/p0N3\n8aAN93qXELf/JqIMJHmgeBkHgI3ZHPcDisGIAAwoNzpSmZbTCeWpFY3iGYz+h3JL4FtXuzdR4rft\npJQpGcqsRYkTHkMxNkvuWZmUq4FPgV9cXZqTQBtX9ibXeYWhdBUtZO7mTXKlXwY2k0WB/ikuL7cL\n8DrKjdob+BPFqGVHoEuuZBTv2Al0zpD/IXDRJdtOYIaUMuvvMxP4KMN1mwY8j3J+f8hspgVKKWOA\nrig3UyzKTbA3Q5FJQHUUz20dyk2WkWkoD5QEIcQolAfMVZQY6GkUPXrUqLr9eOl2NWAfkOr6fwLX\ntXUxGOW3iQKWAW9KKbN63F8Bw10PUlCmwA5D8cynSikjspTHVccQlHGeWyjx/vAMRUah9BCTUWZp\nZX1gTgSWunS7G/9Mh+5CZA4pPTsIISRKd+Y/zX99XBBC/AUskFJ+m9OyqDweqLr99PM0L8B5KhFC\nNBZCFHB1J18FKvMPn9IqKo8zqm7/c57kVVPPKmVQ4oqeKPHll6SUt3JWJBWVB4Kq2/+QZzZUoqKi\novKkooZKVFRUVJ4wVMOtoqKi8oTxUGLcy46Gq/GXxxDPGS1yWoQHQsefz+TIwhtVrx9PGravx5HG\nXjktxv/Nv9Fr1eN+RggKT85pEVRUHgpPg9H+t6iG+xnhuxl7cloEFRWVB4RquJ8RWhWandMiqKg8\ncLonzsxpEXIE1XA/AzRsXy+nRVBReSj8uWhTTouQI6iG+xngWYwBqqg8zaiGW0VF5Ylkz/ANOS1C\njqEa7qecp2UKoIpKVp7lcRvVcKuoqDxxPOvjNqrhfoqZFdrm7wupqDyBPOvjNqrhfor5YJM9p0VQ\nUXngeOValtMi5Diq4X5Keda7kipPL/L9b3JahBxHNdxPKc96V1Ll6eRZXXCTFdVwP4WoM0lUnlae\n1QU3WVEN91OGGiJReVpRHZI7qIb7KUMNkag8jaghksyohvspQvVIVJ5W1BBJZlTDraKi8ljzLC9t\nvxeq4X7ASCm5dv4MV8NO43Q6H1m7qret8rAxp6Zw4eRRYiMf7YvXn+Wl7ffioby67Fnl8pkTLPpg\nBNJhQ6vVkmazM2DibMpUrfVQ250V2oYPHmoLKs8yUkpWL57P2m/modMVwW4Lp3yt+gz7ZCYeng93\nTEV1SLJH9bgfEBZTKrNH9OWTKZO4cukiF8+HsfCruXw+agDJCfEPtW11haTKw2T/5t/549uV2NK2\nYE7dhM16iFMhniyYOP6htquukLw3quF+QBzcvoFatWrRo0cPhBAIIWjfvj2tW7dm38a1D61d9V2S\nKg+bP5f+RJplLFDIleKB3foRR3dtJjU58aG1q66QvDeq4X5AJMXFUCqo5F3ppUqWICku+qG1e2tZ\nl4dWt4oKKLoNRbOk+qDRepGa9HAMt7pB2v1RY9z/kfjoCDb8/D0XQs8QWKIIFWrVZPX86Uz/9FMM\nBgMANpuNFat+o/PQ9x+KDN0TZ/LnQ6lZ5Vnm9KH97PlzBZbkJMrUakDZGjU4sOl3nM7yGUodwt2g\nxb9A4EORQQ3/3R/VcP8HIq5dZvwrXbFa2mC39SIsNJQ968ZTpkp5mjzfjHdHj0Kn0zFrzmd4+QdQ\noXaDhyKHOrdV5UGzcdkSti1bwpjRowgICOCb75YSdfUaBuN20ixmHPZWIM7h5v4Ffd/9EI1W+8Bl\nUAfb/x7VcN8Du83K0d3biLp5nRLlKlG2el2EEAD8/PkczKl9kc7hADgdrUlzlCcm8iva9enNlFlf\n4HQ6qNa4FU0790SjUSNSKo8P8dERHAreDFJSo0lLcucLACAlKYHVi+Zw4vhxihZVQiMvvfQSrdu2\no8bzbYmPiefMoenkDQygY98FlK5S86HId7BoM/5EdUruh2q4syH65nUmvvYy5pQC2G3l0ep+oUjp\nAoybvwQ3gwenQvYine9lOaoNkdeHU6d5O5q+0OOhy6hOk1L5L+xY/SvfTZ+MEC1ACn6cPYPeI9+n\nRddeXDhxlGrVqqcbbQAhBH37vMKX3/7E0E8XPBIZ1Z7k36O6gtkwb/z7JMR0x2Jahd02mTTzdq6c\n8Wb1kvkAeHj6ApFZjopHaLW4ubunp9htVq6FnXngCxZUo63yX4iNuMl3n07GlvYnVssXWNM+x2bd\nwI+zPyHqxjU8vX25desWUspMx4WHh2P08cuUlhgXw5Vzp7CYTQ9MvlmhbVTd/oeohjsLpuQkLpwM\nQToHZkjVYrO+xe4/lKHAVj164Gb4GEhy5aehd5tI3Rad0OndANiz7jdGtKvH4g+GMeHl1swe0Zek\n+Lj/Wz5VsVX+Kwe3rQfaARlnPxVDOjrw19b1BFWqhk0K5s6dm268z58/z+w5n9GwQzcA0sxmPhvz\nNsPaNmFy/xG80awOa5b8/554UHiyOiD5L1ANdxacTocrlp110EWH06EoVrver1OvZWn0bnUwenXD\nzb02Zaul8tp7EwAIO36IVV99wtbNGzl/7gwRN2/SoHplFox/6z/L1bB9PdVoq/xfOJ0OpLw7Oiql\nHqfDjhCCYdMX8vm8hQSVLkO95xpQs1Zt2r82jFKVqwOwZOokju5Kw2YNwZy6HatlI2uXLGffxjX/\nWS7PGS3Uaa3/EjXGnQUv31wUKlmBK+d+BvmqK1Wi039NmWpVGNW5KXGRN9Dp9RQvW5oO/foSWLw0\nBYoUT68j+LefeG/su1SrVg0Ad3d3pk//lEKFi3Dr2mUCMpT9J3jOaKFu16ryf1OjcUuWz3sBGAEU\ncKVGodGtJfJ6c4Y2r05qajJGLx+qN21F7WZtGFCxWvqydosplQNbfsdu3Q/c1scipFnG8fu3X1G/\n9Qv/Sp6g8GTVYP9HVI87GwZP/hhP789w9xgIfI7B2Jnc+UIJ3buZ117pyfnzYezZvYtC/n78POvj\nTEYbICk2mtKlS2dK0+v1FC5ShMSYqH8sR/fEmaqXrfLAKFCkOJ37v4mbeys02o/QaCfj5t6SIkHF\nOLVvO0uXfkN4+HUWLfiKQ1v+4Pzxw5n2IklNTkIjPIDcWWouTkLsP9drr1zLVC/7/0T1uDNwNew0\nc98bijkhEb1eQ6V6Av/8KZSo0I+da5bTuH5VJk/+CIBChQqxfv2f5M9fgKO7t1Ot4fNIKTmyaysR\nN+MZNOgdZs9OpUuXLmg0Gq5du0ZY2DneLF3+b6RQ8JzR4rFaXJNgsbPpQjxXk83kM7rTumQu8nvq\nCbmZwr7wRJxAnQAf6hX2RuOaNqny+PDH0vls+ukbpNNJngLeVKxjwcs3F3Wa/8jUAd34/fc1NG7c\nGFCmAGq1WvoPeIMuA0cAylTB3X+uxuFwAm8AI4FSAAjNOsrV+GcbqXVPnMmfMx6vWSNnY8xsvxJP\nqt1BFX8vmhT3xeqQbL4Yz4UEE7kNbrQqkYvCvu5/X9kjQjXcLs4dC2HG0Fdo0aIVPXt248SJk3z2\n2RwqN2xOrxFjWTVvNu+80TfTMe7u7tSuXZcTf+2iWsPnWTLlQ/Zu3Eea+TVAQ8+e42jQYAmvv/4y\nH0ycRMfXhmL08r6vHI+jhx2ZYuX9HVd4rrKkfV3J6SsmxmxNoFI+T25YUunRXKLVwPJtqRyKSGZY\nrYLpc95Vcp7pw17l4vEQRo0aTenSpVi69Ad2r1vJ+CWryFeoKMnJiTRq1CjTMc2aNSM5SdkcLSEm\nivd7dSE1uQYO+2TgNNARGIFGG4+7YRndBq/8WzkeN2cEYMOFOFadi6ZXC4m/L/y518SW7fEkWx1U\nKe2kXXPJxRsmxgcnMrx2INUDHo+QpWq4Xcwd+xa9er3M118vBKB79+40a/Y8HTp0JCEuFqmBRYsW\n07hx4/R5rg6Hg2PHjtD97XZcCzvDng3rsVp2AYpxttu7sHPXc0QkxNBl6HvUaNzynu3vGb7hsd13\n+JfTUXRp4mRgB+V7i5rg5yVZuimF36eCl4eS3rq2pMfEFM7GmCmX15hzAqukk5KcyLnD+9m5cye1\nailecY8ePejatTufj36DETMXode7MWPGDIYNG5a+XUNISAieXj4A/Pb1PJITmruMNkAnoB4a7Zs0\nbN+JF15fTf5CWfcyuUPD9vUeyzGaVKuDH0Oj+XGCpFBeJa1VLUnn8VYaVoMxvVwFa0G10pJPlkZQ\nrUDJx8IpUWPcLuxpqQwY8HqmtCZNmqDRaBjWpiExt4qzd68vpUpVZuLEKURHR9OvXz/sDkn91i9w\n4uBunI623DbaCh5I2ZVytRrc12h7zmjx2BptgGMRJjrUz5ym0UCz6neMNoDBDVrUkhyNSH20Aqrc\nk20rvsffP1+60b7NgAGvY0qMY/awV+jWrSurV6+mePHiHD9+nL/++otXXulDVZfOHt21G4e9W5aa\nG6PTGegy4I37Gu3HeWD9XIyZMoVFutEGRa8l0DHLLhV1yoHZ7iAq1fZIZbwXT7XHHX0zHKfTQb7A\nIvd9SkopQQoiIzMvqomNjSU5OQ34BWx1ALDZIpk0qSlTpnyMXt8ANA5s1jSMXj5oddHYs/yuen0M\nRu+gbNvNqVF1KSURKTZ0GkFeT/3fljfqNcQmOQnIcydNp4WohLvLxiYK/PUPfv8KlTvYrGlEhV/D\nN48/Xr657lvWJ3c+EhMTsNvt6HR3bvc1a9ZQsGBBQkIO4uWlGNalS5fy3HMNcDpzYbUGUEGv9Jo8\nvHwgMiZLzWacTjMGY/ZGuXvizBxZAWmxO4lOtZHbQ4en2/310OimJS5JIiVkNA86LcQlZS5rsYLF\nBh6PiW4/lR53+KUwRr3YntEvduTdri/ydqfWXDodmm3Z86FHGNGhBUlJKYwe/S5xccoiGafTSb9+\nrwMVgToZjsgPDMDh6IXF8j1ClObkwb3Uer41sBfYk6HsYYTYQP1WHe9qN6dG1c/FmBmx+RLjd15m\n1NZLjN1+mRtJ1vse06yYH3NXCUwW5bvdAScuwZEwCDl7p9yJS7DjKDQocv84vsp/Z+uKpYxoV5d5\n7w5kZKeGLP5oFFaL+a5yUkrWLpnLz7Mn4XRKJk+ekr6oJjIykpUrVzJx4ofpRhugT58+eHr6YzZ/\ng8Mxl73rfwegdc9uuHvMAG4/qR1oddMpW70e3n53Pzg8Z7R45EZbSsnKs4kMWH+d2SesDNpwnW9P\nJOBwynseUzqPAbtVy2+74fZi0ZsxitH+arUgydVxdDhh/hqolN8DH/fHw3A/dR53mtnMR6/3JiXp\nbZA9AUFU+Bo+HtSXL9bvwNPbN71sUnws0wa/hsU0DfiCy5cdBAYWolq1mly8GEZiogmokk0rHoDD\n9dmA3WbFy8ePUZ8tYM7IoUhZCNAh5UWGTp1DrrwF0o/MybmriRY7U/dcY+wrkmbVFYVctTONSeuv\n8lWbIPTa7Hslncrk4eYRK+3HJlOhqODCTUlRHwMj6+Zm3MJbBPiDVgPXImF47YLkMf69F6/y7wnZ\nsZFtv3zD3t27KFeuHAkJCfQfMJCfZk2i37hPMpXdvuonTu/ZzMoVyxkwYABz537J4sWLKVq0OEeP\nHsbPzxcPD49MxwghcHc3AnbAgMOhPNCbvNCdK+fC2Lm2Hjq3Gjgd5ylYrCBDpy7MdPys0DY5tvpx\n08Ukjqf5EnpqP0WLFiUqKooeL3Xh17MX6VXeL9tjNEIw9rnCfLLuOr9udZDHR3DmmqRnBX+izDY6\nvp9IpWKCyxESf4MbY+oVfMRndW9E1n0JHgTLjoY/+Er/IXs3rGbJx2uwmH7MlO5meJOX365Ni659\n0tPW/fA1v355EI24iV5/CSEEJlNRnM7nABuwA7gJbOXOMmET0AboBWwEQgksUZZug9+k1vNtsNus\nnD1yEKfTQdnqdXBzN6S3l9MzRtaejSXSLYZJr2X+eQZO19AyIIB6he/vKUemWLmWaCW/l54irqlR\nNofkdLQJp4QK+Txw0z78TlzHn8/kyOhQTuo1wPTBvRg3chgvvfRSelpcXBxFixXns3UHMs1YGtej\nJR1at2Dp0lUkJsbh5uZBq1bPUa5cOcLDI/n11+XUq1eD4OBgtK6tWbdt20aHDq9iNvcDFqPVmajZ\ntCW9hr9D3oKFiY+O4PKZk+TOH0CxMhUyyZbTuv32jii+W76Ghg0bpqddunSJ6pUr8m37Img191YZ\np5ScizGTanNS1t8DL1eIJdZk41J8GnmMOkrkMtzz+AfFv9Hrp87jjo+OxGYtcVe61VKSuCwx7FvX\nruKm30uvXj0ZNWo5FouFceM+YMeOXzGZDMAM4ALQAXgRyAX8BPgAs4HJwDxuXArlq/Ef0G9sKo07\nvkTFOnfvv53Tig0Qb7FTvNjdtqdYgCTO/PeeUn4vN/J7uWVK02sFVQp4PjAZVe5NfEwkZcqUyZSW\nO3duvH18SE6Iy2S4b127zBdffANMBZ7Daj3GH3+MYevWXVgsjZFyNUeOfEiFClXo1+8Vzp49y6+/\nrsRsDgQ2AItw2PNxcNsKTv7VhZm/bSBX3gKZeo+3eRx0OzbZfNe1KVasGGarDbtT3tdwa4TIdhZU\nHqP+se09PnWGO6hSdXT6UTjsE4DbF92BwbgF3zwv8fHAniRER5ArfwBpFhOVKlVi0aIF6YOXa9f+\nRtGixTCZfIFNQAqKd32KXPmu4V+gIOdDLwBTgK6u+gOwWvxZ9sUQGrbvctf+28pUP+XzxTgLwdfN\npDoE5f2gcVEfbE7J9suJhMWbMGg0gMDitFPC10jzEn7pHsD/S+k8Hqw7kkCflpLbIlptsO8kjK7j\ncf+DVXKcoErVWb1mDZUqVUpPO3LkCHaHgyM7t7B1xa/YbQ5qPt8Yu8OAoqO3w3IBQG4slt5I6Qcs\nxmQqwrlzBZk48XNKVSuNu6cvZvN14Bi3l7RL50islgi2LP+Rl94YcZdMt422zSHZey2J0HgnBi00\nKWSgdB4PriWmsfVSPAlWG55aPRaHHY1GUK+gLzUKej6wqXVl83mxdu1aBgwYkJ62efNmCufxxu0e\nIcAnmaducLJM1Vp4+miBbsAuYB/wCnCLFV9+Su1KZflkyodULV2CK2dO0aVLp0zKo9VqadGiFWAG\n1gIFgVvAeeKjojkfmuYqmXnBAlQnNSkWS2pKptRZoW3Sp/ptv5zEtJAEavUYzMvvTuW4thgf7I5i\nzNbLXLBH07RRCnmLJ7E3PJGiQanc1MQwcsslYk0PZgpSnUBvnBY33l0gOBIGB07DW58JSvl5EpT7\n4XcFVf4/Gr3Qk08/+ZTx4ydw+PBhvv/+e1q1agXCwI+zv+LWta5E3xrApmW7sdvMQOMsNdRGyjTq\n17+I0XgQKAtswGLx5cT+v0iIKQVU584+JAo2ayPCjp2+S56G7esp+Q7JtL9iOOAoSM/RH/Pcy28x\n83Ay8w5FMCH4Cv7FE2jaKJUoTQJnE1OoUCWZH87eYOGRB7fdcddSRt4d+TYzpk/nyJEjLJg/n1d6\n9aBX6Qf3cHiceOpi3NcvnGV8717YrK8DWwAn0Bwvr6+ZOvUj3nrrzg59JUqUpEKFSixZsoiNGzei\n1+tp27Yt9eo14MyZVkANYBywG3gVxYBvBDoDw4HmGVq+iIdnZ74OPpT+OqdZoW1Y834CRxp7YbY5\nGbThOvtDDlOuXDlAmblSqkQRKgbe4INX79R04DR8ugxWTYIvVkHsNV/eqBFw17la7U5WnY4lymSj\nfmFvagX+/WwOi93JH2FxHLyVhE5Ag0J+tA7Kdd+u5OPGsxrj/m76ZLaujMZN54Wb2zGczgBSUvID\nK4H9QD5XSSsaTWWczvmAL3AGKA7kQYi2pKUl0rXrK/zxR3GczteAWsB7QF2gB/AXd3qroNVOpdlL\nVvq+OyE97bbRBphZxEGIM5Bd+w6kx8vPnTtH1cplmTcCKruGh6SEcUsgqCB0fx66fSAYXacIpfLc\n3du7Gm/hj7B43HQaupTLhb/R7a4yWbmSYOH3iyauJTso4KmlXTHDE7UQ7JmOcYce2IWU7YBhrj8A\nO6mpM+nfv3+mso0bN+LHH38hMLAkBkNTIA2LZQBarQTeQlHeNOCq63M/lEs2CJgA+ANVgUu4G4bT\n9uXXMr2Db837yvSp6jtT+LZj00WJAAAgAElEQVSMoGyZ0ulGG0Cj0SCx0SWL816nnBLCuBEDnRrA\n0NmZvXiAY7dS+XT/dQrlhWIF4LOQJPKc1DG7RQl093lVmkGnoWt5f7qW9/+7S6nymHF4RzBO+xdY\n7JWxWG6njkSrbYzDkS9DSTekNGE0DsHb25dGjZpy8OA3xMREUKdOA/R6PUOGvE5w8AckJg4GkoHu\ngDtQHngXGI9i9Nej0/9M6553tm2dFdqGhhlaO2+sSf/endONNoDJZCK3j5bKJR3paUJAx/qwZB28\n1haa15IcuZV6l+Geuf8mB28kUbeCloQ0eHN9PC+WyUOPSnm5H8X8DAyr8Wz0HJ8Kw22zphEXFYFv\nnrx4GL3QauNcC2EiUTxuf3Q6HRcvXqRixYrpxzmdTpxOPU7nJlJSirlSj2K390SZPWJw/fdE2Z87\n2lWmPUrseyAQi06vo12fN+g8cAig7H5W7bkvMslY7riZpd5xSCkzdd30Oj0JWeyy1Q7mNPBwh+tR\nYLE7cDgcxFqcuGsFvgYdcw5eZ/AL0ON55RhzGrw23c7sfTcZWT+QGJMNTzcteo0g3mInl0GHu04x\n6FJKYs12tEKQy+OpUIGnluSEeCymFPIUCMTD0xuIQ5nxdBNllz43nM4bgBW445UaDEaaN2/EqlWr\n0Ol0SCkZOnQoiYmJAMTExOB0eqFES/Uoc7TzA/OBD4HagJ2CxcrRf8LXmXbAvO2Q3Mbz6B6iWz2X\nKc3b25sUsxOHU5kqepuEFDC6bGt0AjhSrKTZncRb7OT20HH0VipHIpJYPhEC/RWjH3oR3pgTy3NF\nvMlj1JOU5iCPh55UmwObQ+Jv1KXfU1aHkzizHT+DDoPuqYsEp/NE37VSSv5cupjVi79CSg+czmQa\ntO2Mw7EdL69WOJ03EMKJEJCS4k6VKrVp06Y9P/64CKPRSHDwHpzObkCxDLVWAxqihESuur57odVd\nwWHfgzLYUwSlS2kARrJw20E8XAsZbodHbuOUkjlJiXybmoJM8+Trr79m4EDl7To3btwgJiaZ+WsE\nVYMkXh5Kd3LJeqhYHDwN8NlKZcXWK7+fRwM4paCQjx67hK5N7kjt4Q6DOsCk71J49fdz6ASYbaDT\ngI8NUoWgQ9ncVA30YsGRm8SY7DicUDyXO0NqFqSA1993RVUeHUnxcXw3dSwnD+7FaDTi5mGkfJ1G\nRIW/j1Zrwmh0JzU1Fq1Wg7u7G2ZzddLSxuB09gHOo9E4mDZtWvpqSSEEU6ZMITAwkIiICMaN+5jk\n5GHAcRQzMBGYi7JlwzTgBsXKpjL157XpMmV1SG7Y7YyJj+OIzcr2T6fTvXuP9H18/jpwAKTgh02S\nV1sr3nZCCny7AQa0h5OXYedxcDqTOHArBR/vXJjNZgwaOy82FgT634lKVS4JVUrC21tuYHDTYdCa\nSU0TIHQYDUb8DBr6V/TmbKyJ1Wdj8XCHFDO0CvLj5Yr5nqgw4D/liTbcu/9cxW+LfiHNsgZlnnUk\ne9YPwcOoY/Kk1xkyZAhCCJYu/Z6hQ9/HbF7HunWfUqJEJaRMIc0KmfcWuY0W+Ahl14Iy6PS1qN2s\nFTZrCUK2N0Yx5nEIcYMhH0/Hw8srXakbZqlpYUoy+1NTOQGkpqbS7u23mTVjOoUCAwkJCaFLGV/i\nzBo6jE2kTFFJeDSkWqBkQWj7LjSpCkvGwJ4T8PGPsGS05MvVVqIvQlZ9NLpD8QB4oQHMWwurPoAl\nayDtCHxplXQ4E8uai3GMfVnSsiY4JfyyzcLELVf5sk0QuqdQwZ9Uvhr7Bs/Xr8PWtSswGo1s27aN\nzl1eJHduTzZt2kLlypWJjY2lf//+BAQEMHjwYFq2bEVk5GKczutIdJlWRQJ4eHhgs9koUaIEQuRF\niPm4Ga7Sb+xkls6YiTmlKlAZOIxPrtyMX7Q+/diMMW0Ah5S8GhtNb4eDzcDChAQqly1LpRo1SImP\nJerWDd6uVZDvd0eyereNwHxaQi86yOOr47uNdm7EwLQBUL00zFmpA78azP58PtWqVMTT/e6VvPny\neDLjjY/ZvnUD7qadvNjAwpiv9cyeuwSAV3v3Ir+fje/elxTOp3jz4xYlsOK0hh4V7x9ieRJ5ogcn\n33mhLRHX3ifz6PkS6tXbwL59ezOVbdeuK+vX++HldR6HIwSnlKRZBEoYZC93RtIjgAbAa4AfXn5L\nmLx0OfkLFwMg8voVtv/2M56+frTu8RrNXmp6Xxlr37rBJim5PYHLASwCJug1zGlfAl+D8uyMTrUx\nfONFejSHvH7w3QZoWQv2nlT2SWhQSQmHBOSBvq3g+ZEwfRDUc62DkBLeXQTli0Lf1jB8Lhw9rxx3\nKBRCrPAlEFodPh2UWcb+nwjaFipInUJPxlL1p31w8lrYGeaO7s/Vy5cyxY3LlS/PxA8/pHv37ulp\nMTExBAUFMXz4cH744QciIiOxWfXodA4GDx7IrFmz0st+8cVcJkyYQEjIQXq/0gdjQDH6jpmMu2sF\n5cFt6wk7fogKtRpQraESg8su7Aewy2JhVnwshzPYj1ighwCK+TC4dgBajUBKyeozsSwPS2Xnzp2M\nHjUcvWk/bm4eHL+oIbePhna1kln4pzvHToQxa9Ysln0/l1WTZHpIJTIeek01cOr0Bdzd3SlcuDB5\nfDU0rmjiurk6u/YepkBebyb1SaFmhqnc16Og71TBd51KPxF7xD8zg5MJMbeAMllS06hevdpdZWvX\nrsj27TNZsGABU6YkcO6cHaV7+AvQFOiNEidcBpRDo/0Lvdsl3pm1KN1oA+QvXIwBE4sryrzyh/vK\n55SSCCnJ+OoELdAHGGF3phttgLyeegbVCGDelluULgQ6HZy7DuN6KzvwrdqlzDapXQ70evBwg9EL\noG0dZXAy+Jiyh0j3vkp9FYqBj1Gpw6GDG1awa6BcNhu5lQyEGJP6otbHhdjIm5QuUyaT0QYwm0xU\nqJB5xaK/vz8Gg4Hg4GCmT59Ot259kbIHdntLFiwYTmjoKTp2bMfOnQdZv34rFksqvfu8iskOgzMY\nbYDazdpSu1lb4G4POys3nQ6yvhIkD9BJKkHG2+EJIQRdyvtzLMpKu7ZtMBoEySkejHjnXT7r0Ysr\nV64weuRQjMZb3Lhxg7Fjx7L463l0nwxdGtix2DT8ccDAhx9OIjAwEIC8efPSr18/Zs6cgbv7OQBS\nTGmUzLIivVBeMFklNofEXff4G+5/wxNpuJ0OByE7NuLm7oXFtAVlnvZtdKxZs5bPPvssPb7ndDr5\n5ZflDBkyiP37/+Ls2YsoM0PKAV8AB4APULzvxWi0/WnaqQVdBs3LtFLM6XCQ0rI6f5jN/K7R0MXo\nSSU3N3ZbLMxJSiTZ6SS/TkeApxEdglZCQ2WNlp+cDm4IOKuHPDY4KSCXh2Da7nAiks1oNRqal8xF\n21K5KZ7LnRl7wkm02/n1Q3DXK3uKVCsFu0OVxTIrdkCKBT7qp8QMtx6GptVgxEtKrNvpVEIr/dvB\nO92g3XtwDtgjITVEmY61+5QBncZJ0ypW9p2Cd2rdPRrvcEoOhCdzNCoFHeCm0ZLqsJPPw43mJfwe\n21VlTzKJsdGcOXyQv/bvIy4ujty577wmzM3dnTVr1mYaYD9+/DgpKSlMnDiR/v0H4u4egMVSF6iD\nybSDrVuXsXv3eKzWfkg5GYNxHHU69aZ2s7bo9PpM7Qav/ZX4yF3U2nyGih5GvIHPUpLZrNEgnE5K\n2e24e3oS4HBQS6NlloSdwAotmDSKbq/wMOBjgXHbrpLscJLLTUffKnn5qElBtlxM4LsT8bw+YCAT\nJnwIgI+PD23bvcAXn89kwvhx+OfNR+UqNWnfvj1fzv0Cd4OBT2dOom/fvunna7fbGT9+PLly+TFp\n4of8+uuv6LWSTSHKvXIhHArlg8J5oaCvLtsFOAlmO1uvJBNhERixYkeLDQ2VcmupX9j7sQ8baidO\nnPjAKz0ZkfTgK3XhsNv59K2BbP51F6bkWsA8lE2fvIDtwFQcDg07duykRIki3Lx5k8GDh3H8+BGO\nHz/H/v15cDp7o7zF42OgFVAJyAscAXaRt6CG9+Z/6xrFv9PuDw3KsteUSke7DZ3NxniTif3WNOal\nJNPZ6UTq9ZzKlYtOo0YR2LQJc44ewWC18g029NWhTBM4ooODCdC1GZyNtCK1kpZ1nSw/lMrRG6m8\nUM4fg16L8EmhbV3Fix6zEPafhpcaQ/GC8M0GJb69/xQ8X00JqRwOg1+2Q9H88OVqsNlh8AvKCP6W\nQ/CLCZ5vBseuGDgeHki3vhPwL/wcn3x9HAPwcsU8mWa7OJyST/dd50RiPNUrpbHxRBq++Sw8V9NK\nhMXCgj0JVM7nSe4cmJVS5sWhkx55ozxcvQYIv3iO93t2Jux4QZBWVq9eQokSJUhJSWHy5Cns3LmT\nPXtCcDrt5MmTmx07dtCnTx/8/PxYuXIlr7/+Gh06NCYsbB4WywHs9o5ADRyOv1Dukc/oMewtnu/S\nK9O01fCL55j8WhcqF8pL6+ZdOOZ0MOXUSX6y2zifPx99h73F7tBQfGtUp+OI4UTl9Wf6oRCE085P\nHlCpNfhWhuUxggJB5Wnb9RW2HzhEzbJmcuW2s3BvPAGebjQt7ktItJMR746jRIkSnDp1irp1qmFM\n20fHejZu3LjGlp0n8fXLw8mTJxk+fASVKlVm6tSphIeHk5iYSP/+/fnwww+pVasWuXPnYeHChRze\ns5zaZez8vE3ZE75mGbh4Q7lPXiiVh7JZ5nJfS0zj/Z2RlGvcDt+iZdl06Azt+wyidouO/H7wLLvO\nR1C/oPGRD2r+G71+4mLcBzb/wcJJX5NmXgOMRZnuZ0ExxEVR3oO3Eo1mAF5e65AyDZPJiNMZBoxC\nyoEZapsLnAQWAtOBpcBr6PRLmPnbOvIFFkkvmdCsCt8lxLNHyvSlCedRhin/QHlndlMfH/4KDeXQ\noUM4HA7q1atHpfIlea2NjZczrNX5biOcvQYf94e3PlfeKNO4CnQaBz3L+6MRgk3hMfw2WbL1MPy0\nFRaPVvYJBgiPhm6TlMGdxq7NC6WEMQsUA96zGfRuoXjfaTZoMRJG94SCeeDTVQU5ceo8RqOizFFR\nUZQKKkn/Ct40LOaTHgvccy2J9ddvsWSsZNpPkMsbhna+cw6/74VVm92Z2vTfvbH+QfC0xrgn9utN\nWGhLkF1RFsUMwscnGCmTSEtrgNV6ACiOweCBXn8M8MRsjsDNLZGwsLD0UILFYqF8+VpcvjwWZfyn\nGuXLF6F16yZs23eQd+cty9TunLfb0OfFgQzNsDitT58+BAcHc+7cOSZOnEhCQgJvv/02ISEhFC5c\nmEuXLjH4zddZ/RHkc+3smmaDfjO9+GzecoKCgqhVoxJrJ6exOQTmrdEwsWERlp5KotdbYxk1ejQt\nmjWgev59dGty57Iu+gNW7Pbgxq3Y9N0LIyIiKFmyJJUqVWL8+PG0b98egBUrVjBo4OtsmJrM5O+h\nQG54s9Od8/ptFyzbqGds/cKZZk1N2R9Ln5ETePXVvhQtWpS9e/emr69wOBw0aVCP6vI6zUpkv6vg\nw+Lf6PUTN9HxwNbtpJl7oMw93Q68jTL3dCfwPYoxT8AprSQlrSY5eSMORyekNCFl7yy19UbZ+W8L\nypDhHGAkGm0rju8LTi/VsH09gs0mXstgtEGJiPuhRMg3AtVr1aJ6+fIs6dePH/r3p1Lp0iSbbLyY\nZYFN54ZKKEOrURbY7D0JuX2gcVUNP5+JY93VaExpkjkrlClTHerfMdqgxO4qFAXHnbUNCKGsRpMo\nUwkNbsr0q4+WglYLHerBvlNa+vQdkG60AfLly0e7du344fQt3t58iWjXGz6ORCTTsaFEp1Xk65xl\nukzbunAhJg2TzYHK/4/Dbuf8ib2urYgPARWAESQlrSE5eTtW60fAQIRmHxbLKyQnB5OcvASHE5o2\nfT7daAMYDAYGD34VN7dNwDiMRg2nTh1m2rRpnD5yEGuaJVO7R3efpn+GPT4AjEYjgwYNwsPDg3Xr\n1hF5+TJNqldnw+DBDO/QgYnvvUetMneMNihhvTY1Uvjzj9WUKlWKGtWrcCRM0d9ks5PRW69gFnFM\nnvIRGzduJHjnfjo9l/lZ+GJjSEuzZdpytkCBArRq1Yq42BiqVq0KKK9WGzJkCBWLJKPTwr5Td+to\nh/pwLd7G6K2XWHw0AiklDqfk6PVY+vcfwL59+6hSpUqmRXFarZaBg9/iWPx/+RUfHU9cjNvD04gQ\nia6Nz40oCweKZCiRihCCkhX/4vKZ+QghCCxZgatntUCS65jbJAISIYYj5TuA8qomjSYRg0fmHe88\nNBqy/paeKMtw7Cg+f/D27eyQkiCUfsAtoKYWkk2KIb1NskmZunf7s4frc2KqoFqQk+mD4GokfP0n\nHDgDJbLZBjjJpNwomdJSwd8H3l8MdjvYneDpBr6u+eEebk7iYqPvqislOYahXSA82sasv8L5uEkx\n3LUakk2uc3cn/fNtTBblYfG4xwKfFIRGg1brht2ZjBLWyOb1QsQTUCSA5IQ3sZhMCOGgQNEixMbd\nbWWioqJwOpeTP39h9u7dB0BSUhJarQ6N5o4X0KhTA9y1WhITE9PfNwnKqt6YGOWtN0mJidw6f57z\nViuJKHtkvpKayoVsXn6TbNaS11t5V2V8fAIe7sr0ViFg4Ugokg8OnE5hQL8XAScp5sx6nGwCfTZv\nmbFYLCCgVFAQOr0eKZ14YKW68qJ5PNyUudsZSbWAXge/TYFB0xPYdMGdlkF+6HU6EhMT8fT0JD7+\n7msXFxeHu8jR3Q3+lifO436+84vo3b8FbgAvoWy9envjJycwHYme8Yu+Yf7WA8zbvI9pP6+kQJEy\nKHOzb3uIdpRtWTVIuRoY7Er/C+k8QI0mihG/PbrexejJl0JwPYMsx121NdLDZB3o3CUt3aGwXkcx\nvZ5GBoGbDmYvVwZNQIlZf7UG2tWF2CT4cYvy+eh5OHTOwYkrBlq9687bC305dMGI3aHlxy0QEXen\n3eBjynL4c+F33tyRYlYGKvu2gQVvK2kfNS7Mtx3LgEPHugPKi1B//HEpZ86cSa9r9+7d7Nq1mzPX\nlBj59aQ0+v95HqNOy6/bBBFxinxfrlaW4YMy+Pnlaqhb2OuR7L/9LKDRaKjbsjM6/TSUjZ5MwOoM\nJaJwc1tI/oL+zN+6l8//3MKi4EO8P/9Hjh09wpYtW9JLXrx4kQULFlC5clWuXTtFyZIlcTqdvPf+\nOOq36pA+KNmwfT00QvCCtzfvjRyJ06koqc1m48ypUyxcuJDKFYOIjgon1GmliIeGap6eFHDXsEEj\nCbuuzHS6TXg0/L7fjd6v9GXlypVERlynSkn4fKViQEcu9KTTBAPTlhlITBXotTBnhaJPoNwbn68S\nWG1OwsLC0usNDg4mJCSEw4eP8ubgNwn00rGkTSAj6xXkl62CqHilBzhvrTK2A0qd89Yq02Xf+8aT\niGQDS08lMXFvLDUDvRk7ehR16tQhKSmJZcvuhI4iIiKY+ek0GgU+3gvSnrgYN8Cf3y9mxbw52KwV\ngUsoO/k1RIlz50FoQlmy6zgG4x2vOS46klGd22Mx2VFujBDAiUZXCKf9Ajp9bfRuEilP8/bMuVSq\n2+iuOayLU5L5LCmRRkIQi7JTt8VDS5WSdj7sC35ecOwCjF3sjs0GU/unUaEYjJ4PlyOgbBFlxZhO\nqyWosODoOTt5/cDXS3Aj1kDrtp2Ijo7m+++/p2DBghw/fpwuXbrQqOFzrFj+C+WK2EhKVbzxgLwF\niYq7hb8PFM4vOHLOiZ+3oGRhT46FpaFBsLBtUYx6LVcT0vh4zzXy+EkkksvRbtSuVRvpNHPq5Elq\nlrKQkAqT+ipzyMOuw6h5gnK+3hy4mUwhf0lEghKuqV4azlxVPO7GhfzoW/Xu/ZkfNk9rjNuUnMQn\nQwdx9dwV7PZCSGcoWm1xjMai2Gx7adOmFVEmM0Omzct03MZl37Diy08oV64i3t7eHDiwF7vDH6OH\nE50+jcZNmnLs2DG88+TjrRmL8PLxyzTdL8np5A2LmXCDOzVr1GDfgQO4paURYTPxbk8nbeooRvX7\nzVpW7zNSsoCVD/ukcS1SGTgPyA3eBjh2CcoHlcapd+fCxQuUCjATEQf5/CAmxYMx701h8JChCCFY\nuHABkz8cQ8HcFm7EQJlCcOIyOJwa/MxOYgwG6tWrR3JyMidPnqRmpUrERUYSHRtLNX8NQ+oroaHV\nZ2NYeSaWqiUk524ocfbKJZV7JI8PRCZ6MOb9KQwefKfdj8a/T0FvdxKkHovNidVmo0yZMgQEBLBt\n2zZwOpjWOB+FfNwf5s99F/9Gr59Iww2QGBfDnFFDOB+aC+kcjjJUWBQ4Q7GyK5n688psj9u9bhUX\nThyhdJWaePvlJiUpgaKlyxN+MQytTkelOo1w9/Dg7LEP2Dnqd6477FTUahmi1VFe70aMw8E+axpe\nQlBGp6NJfCQbZ2R+2/mqnbA8GKoEweFz4OsFNUvDD5uhy4vdOH7gAInJydStX5/NwcHpI+UlS5Yk\nNDSUQoUKpde1Zs0aZs+ezcqVK+nRoxsHdu5iq1NyFSVANAq4nC8fS7//HrPZjMPhoEWLFrzSszuF\no4/RMkgZYHE4JaGRJpKtDor7ufFTaAxWjxSmvwGdJ8CyDyB/hnhl8DH4brU7o+sVZnzwZca84iBf\nLjgfrsTYfT2VxQ3fdiz9yEffn1bDDco2DmeP/sXMt17lu+++xcfHh4SEBBo0aEC3Hj2p0rILjTp0\nveu4lKRE1v+wiDSLiXotOhB9KxwPL298c/tz88oF8hcuRskKVXHz+J4DtT9lo9mEBFp5GHnDyxsP\nIQi12bhkt1NWr+N7UwqR5UxMyxz6putEeK4ChIUrvcDShZWw344Q6Ch17NDp8BKCkmYzW7UwdzhE\nxcPua8+xeeueTHW1b/M8NfLtoEh+eHM2dHYo27glAIVRApctUPbnvAAUAkoDpTSCRV2CMLpCKglm\nOyeiTBj1GnzctUzaeY0hL0qM7tm327Zlc4IST+GmFfx4wcaVa9cJDg4mISGBJk2aMPfzzzj1x7e8\nVjk3j5JnYgGOb25/hn/6OeNefhFTyudYLY3Ruy1Dp9/EwA9/vOdxDdu9SMN2L96VHli8VPrnqGZV\n+VanYfpXX1K1alU2rF9PrwkT+MnTiwp6Nzp6KHHy0zYr/t6ZjTYoXuvNWGhURVmlGBEH89YoHvmB\n5ctZiLLL97fr1qH38eHVV18lLS0NrVabyWgDVK5cmatXr5IvXz66devB0cOHmWN30sJkYrGA40Iw\noFtXZV/mDFSvXZdTKw6lf9dqBNUC7vRAhtYJYELwFcbMtyHJbLQBggIhKtWGn0FHssVJqULg70v6\nIgcpweqQmO3OB/aiBxVlwUq56nXp9/40Bg8ZysCBAygYEMALXV5EuntSv/UL2R7n5eNLtyGj078H\nVa6e/rl4OWXd7sxjrfEcdJ48Nitfo8RJZ6Qk0zfNwi/++aji5kYVNyVEoNEoi7juascDNh2Cd7oq\nOrL3pBKiswnweb0/6994g+joaAa/+SYFEi5Qswx8sx6qVqtzV11VqtflVugOOj6nzLveEK0jSEoC\nHA5m6gQeTskUp/K67loZjvMRkJTmSDfcfh46Ghb1Sc8fUTeQL1bfoFhBSfMX7m63Wq3aXPr9GKVy\ne1ChbHEMBgOtW7dOz69avQa7Vi3N9jo/LjzRAcpceQsw87eN9HirLvVbH+WF1wsxa/Wmu96H90+Z\nFdqG+u3qMstmZeWff9KjRw/Kli3L2++8wwfTpvFlhmkcqU4nwWYzUclKvDkjSzdB69ow5AVFuRtU\ngi+GQaIJFqPMHK8ItAPc0lIZNvRNrly5gtFoJCQkJFNd69evp2bNmgCsWrmckqVKU3fSJN7V60gs\nlYth9QLYuH4djgyySSn5Y/VKgv7H3nmGV1F0AfidW5J70wuBhBaadEQBKVIVRBAEATsWUBS7Aiqg\nFEUEFBHbJ1hARQQBEQQEaQrSUap06R2SkJ7cPt+P2ZCbS0INpu37PHlyd3Z3Znb37NkzZ87MROTd\n1AvyMzLm9so0DS+DkLDzcM79a/7h/OIK1SItrP4n5/6t+yEiwEiguUiLUKGlRafuDJwwnZ0nE5m3\ncj0t7nuCfh9OzjFo5kpo2bkZr752hkSXk5+AZkATYCZgc7lY6RVpst/pZFOmg983ZfehgJpyYf9x\nGP+8CmGtHKPCTnt1BD+T4PMJE6hfvz5NmjShV+/enMuw8PkvgoplYNGvc3PIqMfjYdGC2dSppOYV\nOREH382cyY727Rkd4E/bJjHcGBPErz5D1XcCTg9EWvO+D43KBvFe28qEuoKZN/fCcn/9ZQ43RFip\nHO7Ppi3bzs+YmMX8uT9TKbBwd04WWYs7i4CgYDo81AseurrzvX19LYGzHg8ZAho3bpzjuM6dO/PB\nkKFgsXLG7ea+uDNU8ngICw9hwEQ3z3dJp0JpWL4ZDp2CJ+/KWU6pUNWjvucEtAWeNcHSIHjgNjc2\n5wbu7tiKW5q2oXv37owfP5769euzcOFCRowYweeff07fp59iw4YNNG7ckDk/zaRp1Uh63RyOlJJl\nx+Pp3vVu3hz2FiaTiffHjCIz7gS31CzNxfA3GWhfLRyTQTBwwhleuV9SvbwKrfpqnmB4K3V+hyqR\njJtxHIcLmtWGXUfg/elwT7XwYrm6SGGhfNUaPPjSG5d9/OqXF+XYzlp5KYutDgcdpcS7fWQAOkrJ\nVqeT2yxW5mSkMyIpiUpITp+Ad7+B++5Q8+V8OseI0eimZsUc2dLqRvj6V6XoTpw4QatWrbj55pt5\nf+w4Vv25kvdmzKd02El6dO/Km0OGI4Rg9KiRONOPg4S+H0JQYAAbN25k9drV9G9amnplAqkQ6s/w\nM+kYXIJ7kOwEXhWCV4JD6fZsBxZ8uTjPexET7Ee/ZmV5e00893XvxuAhQxFC8N6od/Ekn+XmulEY\nDYLqkf60atWK8ePHEy3GjYkAACAASURBVB0dzaRJk/hl3nw+bvff991cCUXWx32ljNvekWGLs+fj\ncHskO85mkOZw8+B+SZQ2kmy/08ldSeeYNnMmXbtmL2s2b948RvV+gp8sVgYlniM6M4PXger+/nz4\n+ed8NfEjTp06TXhEKf79dx9PdHTzuJf3wuGEdq/CHJsKSOwZAlPfyZ6bOD4Z7hkCkWF+mE0mUjPc\nmE0G0m0CIQxUiLITHeZk+34rXW8I4a5qYQgBu+MyOZvu5HCyg+3nJB4paVLGj3tqhJxvSl4Of51I\nY/7+eM6mO6kabqFHjSiqaBb353+fxBCegs2pOi6zFm/YvcfCO20qXctjuSqKs4/7Sshrkd5km4sd\nZzMIMBu5sYwaAeiRkt8XnGBVWhrzUf7iLLogaB0Wxt0WK03PnGKllCxBjSOOMcI8kwrnw2ngkPAw\nf5QyRLJYvhlGTRUkpXl44oknKFOmDKNHjz6//7PPPmPIm4OJDs0gMcMfj0ciPQ4ybR78/Ux07dGT\nmTNnUSvKyv3Vg6hZykq6w832MxkkZrrYfSqNAwk2qlWMoGe8h45ajPfm1rnEI/pgd3mYuy+Zdacd\nSKBJGT+6VQ/FajZgc3l46tejvNhvAIsXLyYpKYl27dqxce1qWgYkcHvl0Evmn5+UiM7JK8FXwI8m\n2xm1+ihhIZKoUNj6r4e+1mDiHG7mZ2bQBNhtNBJcqRKL/vyTc+fOcU+HDryWkUknawC3nDrBGimp\nAjxmteJs357/TZ7Mxo0b6dmzJ2XKlOHY4b188KyHhtVVPOm4mQZWbDPSNsNJJSC9I/TtmrOer02A\nLQcDiYgIof6N9fhz9Tqkx0X/HpmYjDDme8G49lUoFWAmLt3JqPUJ+IdGUK1aNVatXsNtscE8Xjc0\n363gJ+btY9JgD+W9Zsd0uqDNy/DdPdWx/sfukpKouIPCVciafGPyRY+btzeBGbviuamKIDFNEpdo\noO/N0Uzdk0ZAqfLExETz17p19HG5GO1wMAkYIQR/lInhL4eDCecSWIPkLGoiiPGoJbGfNcMsA4SE\nqBGK7/ZRURv/Hlejf1NS4OcFC+jduzd///03FStmm+V2u52AgAACrX40bFAXg8HAX39vw2Qys33H\nbt4Y9DqH1y/j1caRAPxxKIlJW89Qt5LA5oDDp+HVZuWpWzp/lyHbfCqNJWlRrP1rc470KVOmMPnd\ngfRvWHhHThZ5V8mlqHY8Fe8lST1S8v7aY/S5x01XbdGO+GToNTKV0EwVXBgCSLebNw8coG7Fivj5\nW3jZ359OWqekR6qhPAATMzN5eckSYmNicBkMLF++nFtvvZV58+bx/LNPkJSYgMMJsbHlCAmycSwg\nlUUJNu7xGdACsO9UAD0feZxPPv0Ug8FARkYGbdu2ZeT3GygbbGJQ8xhKaRM7fbolmSdfepXBb7yB\nEIKkpCRaN2/KysMptMlnS8HfJEi35UyzOZQVpodx5z9Z7jtvi/Jyvhi74zKYfyCeH4dLoiPUGUv+\ncvPu1JMMGfYOAwcpWTl37hyNGzdm4oED1DGZmRoeQaDBgB8Qp5VUGvgVeAZ4CihfBha8puKxP54N\n97ypfjtd0MoJBwwGBvTsSabLRWpqao56HT58GD8/P5YuX0GTJqqzcNOmTbRs2ZIaN1Tj1tgwnrtZ\nKcnjKXa+2X6GyYMklWNUXTbuhsFfHOOLTjfkq5HgbzSQlnbhsoDJycn46QNwCpZvx+YMBfo3wYbB\n7KbLrdlppULhsbugnJ9S2gACtYCTdLv5JSiYR63ZX3sPkqGoBaQCgC8zM2nocHBHu3bceqvKuEuX\nLhw5dpa773mI1wa+yeA338ZPOhnaNpZ328WyaIPg6JnsOmzcDXGJLkaPGYNBWzMyICCAsWPHYrUG\n8+EdVc4vfHomzcHxVCevDxx43roOCwtj2Ih3WXUm/4egt6oYysS5ApeWtZTwxTxoUl4fgHMtPJD8\nAdWOp1LteCqBY+84/7e5ddBluQF8+eNIEg+2lUR7RbG1vwVKhUoa3dLkvKxEREQwcuRIGkZGMrt0\nGaprHZ4CNRv9bO3cRsAiwGyBl3oot57ZBK8+APNGgdsB2xzgkoIHA4NYGhDEfSYTQ15/HZdLuSWl\nlPTp8ySdO3c+r7QBGjZsSI8ePagdbuLFBuHn3Xorj6Rw962Syl5rYzeuBXUqCf46eaGSvRZqlrIS\nf+Y0s2fPPp8WFxfHh++PoUVM4R6AU+wt7hYfd4Sx2Z00NpcHi0Xz2XkRFqTmq/bGDwgUAt/1OJyo\neUqqAW2ATSgL/I7SOTsDDQYDFStW5Luvv0akpfFNQBA1/kynIeDxD6XXO0k0ukFgd8L2QxKjyUBg\nYM6h9pGRkbjdnhzuD5tLEhwUeH7aWu9jM135byn0qFWKcettdB2cSUNtAI5ZmhnSonB34BQmVr+8\n6ILOwgUA5N3BdqUk212E5qLvQwMFGRk5m3iRkZE4yPkS2IByqCW2P0NNnLYYNStQaE6xJCwITGZo\n7oLmFn+eCApGCMHggCCeW7eOqmXL0apVSzZs/pszZ05Tv3fOhbpBzZOz3ZNTXu0uN6Vyu4Ygic3p\nucQduDKMBsGrjSN59snH+WjsGKJjYli6bDmdqoVyU0zgpTMoQIqVufTXH78x9LGHeanTHUwcPpi4\nk2qAeuens3sJg/2MHD4tOXAy+zyPB2b8AXYfd8ASwCIEsT4T2gvU2Mu5qAmmPkfN7j37p59yzH2Q\nlpbG1G++oXdmJitDwqjhFcr1UEAQqyJiuOdkOI/Eh7M+IpoAs5mffso5cOiLCROoKCWdn77z/HWU\nD/HDlp7KypUrzx8npeSLCf/jpoj8d//6GQ0Mbl6RQU1jqWYow1P1KvB+28o5FoLQycbXgg4ce8cF\nSvtKyHC6+XHnOV5bGc8bqxOYvy8Rt+fCD7TbY2LOatP56RVArQKz55jEzy/bgpRS8tVnn9HGlXPx\njBSPh8OoCdP6AR1RHZQBbsGsFTnLWv0PBGLgh1Kl+TSiFGbNsLAKwTfWQCZIuGn574xJyeCLAbcx\nbdo0kpKy519JSUlhypQptKuS063XIDqYhWuVMZNFQoqKF78eyrRahIUJd5ajld8pyp3+m3G3x/BA\nrf+2U/JqKDadkwunfsOszydjtw0CKmMwLMQS+COjp89lSvxT5yNKNp9KY9qxAJISj9O9uZ1SoW7m\nrQJxClIcaj2dB1CD578Rgk8jImnpn3ORgWonj1MaaIUa2fU3MANI9fOjVFQU/fr3x2w2M2HchzRM\nSWGUxXpZHYavJ53jF+DJJ5+kQYMGzJ8/n8W//cYHfhY6e83oN7rqMCINI/jfliT6PN2XGrVqM3P6\nDxzcsZm3m0cRWMwHxBSWzslx2zsC5IhWym+cbsnQ1Wepd+ttvNxvABkZGbwzfCiGuIP0a5RzZN+4\nTYmkWMIR9mN0bJROQgrMWmnmjg5dWbJ0OX369KFmzZpMnTKNhC1/MyMgkCBDtu02LiWZRdFlSD55\nkufsdqLdbr4PDCQpOpqDxw9yY1VJ+1vg4DFYsEbwRVAkzfwvXIDDFyklDdNSsIaH88orr2AwGPjo\no49ISUxgUseyOZYVk1Ly0cYTnLCl062VxOaAmb8L2laM4P46xW/tSG9KXFSJw5ZJ37ZNsGfOB6qc\nTzca36V11wz6DBlBteOpfDt2NU/sG0671GTmL17MmNHvsG3FckZluHkA5f74BjX11G0WKy+GhFLJ\ndKFVedPJ44xANS23opYpvhl4GHjAGkCGxYIQgo5AG39LDqW93m7jo7QUUvBwu9nKS0HBIAQLMjIY\n4Xby5XffsXXrVg4fPkzDhg1JS01lw/jxfG7J2aO+uXUQJ1IcLD+STrJTUD0E2lQKwd9UrBpRuVJQ\nituz4hW5KbUvp6Z3/8/K/PNwCus95Vi1bsN5ObLZbFSNrcBrDYKoEp6tOH/ek4izeivuvqc7gwe+\nQrVqNXhv7HgaNGjA3r17GTZsGCuWL6FW+N18lf47Vh9jYllmJs/bM1myZAmzpk4lKT6e9t26sWjR\nIhb8+CP3+vtzzuSmrDTxkCWIil7vhsvj4X9paSx1u7BKDy8GBNLKYuWU282HKclsjCrF519+yezZ\ns5FS0q1bNwa99BKx1s48VS+nu8gjJX+dSOOvUymYDIJWFcOoHZW/ESWFkRIXVXL66GEMhtJ4K20A\nt7s9uzepJZJO11vA4AOfgNFID4uVgS+9TMUqlXkhw31+4TM/4EXU6Kzy/v65Km0Au9nMQKeTj4GR\nKB/3U4BTCAaEhhFiyF15jk5OZLIzna7NoEwk/LI6lZmJaZR2GbC63biDgujevTvdu2crhgMHDvDF\n+2PBx7BpsDINWgfxWL3C3YlSnFADPvLPJ3057E920f3JB3N8/C0WC+3b38m+vctyKO52lUN4fclv\nVIitRHKqg2k//kRUlLJSa9SowY8//ojF34/qNRpjPfzHBWUZBARarfTr148PP/yQ6OhovvnmG+bM\nmUOD4GCGW3N3VTg8HtqmpmAsHcVTffty+sQJnp48mRbp6fzlsFNNSrrfey933nlnjqkZ1qxeje2z\n/9HgXEiOzliDEDQpH1xkFq8uCIqFeRYaWQqX8zSQ7rPnX0pFX9iBNszfQodDh1jx669svmAv7BKC\naEPe7obKQUH0eeUVRoSGUkcInrdaue/55wn0txCQh0sk2eNhkiOdLwbAaw/DY3fCj29BaGmJn9vN\n7wBOJ6dOncpx3p49e4g25/4BabAyf3vZdQofYX6wY9uWC9J379pxwbDvEH8j77QszbZ5U5Aed47p\newEOHjyInzmQIYfG5FpWtNGIRQjKlStH165dadCgAfPmzeP+Hj2oRt7G4OjUFAIrlGfn7t0MHDiQ\n8Z98wrp161iJZLKUPA/s3Xzhm7Zr82ZitPdMl+Uro5go7ihubNYGs98gsiOs/8HPMo4uvXtdcLxB\nCHoHBPJbaDh/CMEM1EzeTuAj4KgQ3G7J23fXS8LSefNYvmkTqR4PWw8fZu+2bTxstWLKQ3HPyEij\nQqmcE/eYjNBTW2LMCvQCnurZ8/wE9nv37qX/s8/x+EUcTzEP/Zz3Tp0iz22VQpg7Zw4//fQTUkqc\nTicfjH2fk0cO0SCXzrqoQDPP3hxOzxoBPN/3KQ4dOgTAmTNneOqRR3jS35jDp+xNLZOZci4XVWNj\nz6/x+MknnzD/55/pacp7bpCVBsHL/fvj7589N069evWoV7cuRqAHsHnDBr764gvcbjdut5vJkyax\nbtUqOnutdKMr78unWLhKAJ5/9z0mDn+TzasaYzSGYjQ5eHTAIOo0bp7nOZFGI5MioxicmMDLbjdO\noLrJzPcRkfhdpDPxPmsA55KSaVyvHmGhoSQkJXFfQCADLNY8zwkQBmy+cYWogSxGTTG/Z7fTf+1a\nqpUvjyU0FFdaGi9arHS15u3faxj8hRZWplMcCbeaGNS0FINeeJrnn3kah8NJbJiFIbeWuuh0undU\nCUX4d6JRnTqEhoZyLimJBwMCeeUiMiqEYIK/lYFTvqfsl18RFGDF5HDwgZ9/jogoX4xSkp7u29qF\njMxMzKiVohZnZvLkgAEMevVV3EJQ2Wjk+8CgHJ2joJT31cSwlzSKjeK2BATyytiPSEtOJDU5iaiY\n8jlmUvNeEMGbEIMgMiSEf1NTMQClQkLydHdkIYTgGYuVXv4WTrldRIVFXCCAvtxvDeCdc0ks3wxt\ntRk3U9Jh8q8Q5lAdnRbgM7udGGBWwjlmly6D/yXqcrLnbBZXHXZN4WY6hZuapayMu83C2XQnJoMg\nMiBvJbr4eH8GHxgBwP70yQTYbJzOzCQIKGX2w3iJKJBSRiOTrAHE+/mTJiUVgyx5WuhZ9DAYeP+9\n9+jZs+d5n/qiRYs4dOgQB7Rj6gDL09O5FegaEsrTAXkr55iHfv5PO4CLIsUiquRy8J4FMIsUj4d2\nqckMfOcd+j77LA6Hg9HvvMMvX37JosBgjPk858dP6ekMzkikRnmIDodVO6GGMFHJY2Sj08GdUrJL\nCM4IwfelShObR+dobpQkK6WgokrmPVyr0Mm1N96uhnkZ6YxLSmI6ksbAbuARIbgrKJi+wSF55nG1\nPJCUyDYBnTp04Mzp02zctIm+RjOzM9OpJCXVpGQh0M5qZVRYxCU/BiVJnrMoceGAl2Lc9o7MfePC\nxVenpKextWULZs6ffz5NSkmTunV58fQZbrtIs/JqSfR4+DQ1hXiPmwesgTTXfOm7nA42OxxEG420\n8bfk6SvPi7I/9LjoNJfFCV1x58Tbys7i7jOnGe120cErbSfQVgg2Rpe9pOK8GjbabfyQkU64wcBL\nQcFEGE04pOQPm404j5vGfv7nh9dfDiVNeZe4cMBLkZvSBjgKNGrZMkeaEIJGTZtyZPb16fQLNxgY\nFnrhrGO1zX7UNl99aN/JnrOhhAm6Tu5KG+Cwx00jn7TaQLKU2KW8IIY7P2jsb6GxjyvGTwjutF6d\nAdT56TtLjDFypRSLqJKLkZuLJItaQrBs3rwcaW63m9+XLqXWRXrRCyt6r3zJIzelDVDbZGKZT9pa\nIMZgwFJEFr842XP2pQ8qoRRrxZ01h3FedLIGcHT7P7z47LPs37+fHTt28HCPHkSlptLYr2gObFl8\nvH9BV0HnP+JiH+qXQkJ5GcE04AxqQque2uoxRWnVIt0YyZ1irbjziiTJwiIEMwICSZ0xk5Y33USn\nW28lauWfTLIEFCnh9mbwgRG68i4BeE+clhvN/S18GhnJRLOZOkLwtsnEm2ERdA8s3LPe5YauvC+k\n2Pq4L+Yi8SbSaGSkNYCRF4mVLmoMPjCCzeV1f3dx5nLcCLf6W7g16tKTQBUFFh/vr4e8elEsLe5L\nuUhKAvqIyuJLSbRA8/Lll1SKneIOCp9+SRdJSSDmmfYFXQWd60BJVNpZ6MZINsVOcetKO5tL+UF1\nihYlWWmDbox4U6wU9+X6tUsKJ3vO1pV3MUHvcFaU9I9XFsVGcWetRqKTEz0Wtnig+3iz0Y2RYqS4\n8xodqaNbKUUd/fnlRDdGioni1l0kOsUV3UWSOyX9Y1bkFbeutC+Pki7oRRXdRZI3JVmmi7Ti1uO1\nr4ySLOhFEf15XZqS2iIp0opbD/27cvRY2KKBrrQvj5LaIimyilt3kVwdeixs4aekWpFXS0mMMimS\nilsP/bs2dGuucFNSrcir5WTP2SXuY1fkFHdeq9noXBkl0UopCugf1aujpH3sipzi1pV2/qDHwhY+\n9I/ptVGS7l+RUty6Xzt/0a27woX+Mb02SpLLpMgobt2vrVOc0T+i+UNJcZkUGcWtu0iuDyWpeVlY\n0ZV2/lISZLpIKG7dRXL9KEnNS52SQUmQaSGlLOg66Ojo6OhcAUXC4tbR0dHRyUZX3Do6OjpFDF1x\n6+jo6BQx/nPFLYRYIYTo81+fW5AIIdoIIY57be8UQrQpwCoVKL73Q6fwIIRIE0JUucj+w0KIdteQ\nfy8hxOqL7C8jhPhTCJEqhBiXy/5vhRAjr7b8/KKgZfiqFfe1PsD8RggRJoSYLIQ4rT30fUKIQQVd\nr9yQUtaRUq7I73yFEDFCiHlCiJNCCCmEqOSz31+7Rynafervs7+tEGKPECJDCPGHECLWa99rQoh4\n7aNTzyu9uRBibn5fi1f+F33RiypCiBZCiLVCiGQhxDkhxBohxC1CiKZCiHQhRFAu52wRQryg/fYT\nQrwlhPhXO/6w9mwrXUu9pJRBUsqDWhkFoSSfBuKBECnlgP+47CJDcXKVjAeCgFpAKNAF2H81GQkh\nTPlYr/8SD/Ab0COP/W8BNwCxwG3A60KIDgBCiFLAz8BQIAL4G5ih7YsBngSqABOA0Vq6CRgHvHJd\nrqaYIoQIARYAn6LudTngbcAupVwPHAfu9TmnLlAbyJqE/ieUjD+Mkvf6wCag7X9wCVfFZb5XscAu\neZ3D3YrwO66QUl7VH3AYaJdLejhKKOOARO13ea/9K1Av/kYgBfgFiPDa3xRYCyQB24A2Puf2yaM+\nO4B7LlJfCbwEHER90ccCBm1fL2ANSvknACO19CeA3dp1LAZivfL7GDimXcMmoKXXPivwrXbeLuA1\n4Hhu9w6lTGcCU4BUYCfQyOvYBsAWbd8slDIdeYlnY9Kut5JP+kmgvdf2O8CP2u+ngbVe+wKBTKAm\n0ASYrqXXRL1YAK8Cb1yGrFzqfgwCDmjXuAvopqXXAmyAG0gDkrT0Tto9SdGewVtXK8cF8Qc0yrqW\nPPa/Afzuk/Y+MEf73U57NhUus7zewHyv7X+BWV7bx4CbvN6Tapo8OAGHdu/ne8nuq8B2IFmTR0se\n5fbC573ySvtMO38P0FY7/lufMnPTL9+S/X4GA38AnwAC8Ac+AI4CZ4CJgFU7tg3qgzgQOA1875U2\nADgLnAJ6e5V1yfwKTIauQfgO53FjI1EWX4B2Y2cBc732rwBOAHVRymE2MFXbV057wHehWgN3aNtR\nXufmpbi/Rim93sANueyX2kOOACoC+7Ly0oTJBbyIUnpWoCvKYq+lpQ0hp2J7RLtWk/bgT2cJMDAG\nWKWVVQH1UbmY4rZp12xEfdTWa/v8gCPAy4AZ6I4S6itW3KgPqgTKeKXdC/yj/f4YmOCTzw7tWUZq\nv8OAF7RnWgFllftdhqxc6n7cB5TVnvkDQDoQ4/VsVvvk1waopx1/I+qlyvOjXdj+gBBNrr8DOgLh\nPvsraPJYQds2oBTMPV73c+UVlFcFZQgZtPt8JOv+a/sSyTZiJFBN+/2tr6xpsrtRyycCZdg8k0e5\nvbjwvcpK66fJ9AMoBR6RV5k+eX6L+gBEavUY6bVvPDBPq1cwMB8Y7SUzLuA9lEK2eqWN0OpyF5CR\n9TwuI7/io7hzOe4mINFrewUwxmu7NkoZGVFfw+99zl8MPO51bl6K24qyVDahvtr7gY5e+yXQwWv7\nOWC5l4Ad9clvEfCk17ZBe6ixeZSfCNTXfh/0KetpLq64l/ncj0ztdyvUR0547V99McHWjslNcVfQ\n0ixeaXcAh7Xfk7yfi5a2Buil/X4I2Kzdl1iUW6Ut6sVbiWo5lc+jPhe9H7kcvxXo6vVsVl/iej8C\nxv/XL8+1/KEMgm9RCtmFUhDeH9VlaK0Z7TnFAWZt+yu0ltIVlHcM1Xp7EPgSpfRqogydeT7vyaUU\n9yNe2+8DE/MoM7f3qheq5ect0xuBR/Mq0+f8b4HJqI//a17pAvXBr+qV1gw4pP1ug9Iz3vLfBtVy\nMXmlnUW1+i8nvwJT3Pnu4xZCBAghvhBCHBFCpAB/AmFCCKPXYce8fh9Bfe1KoRTCfUKIpKw/oAUQ\nc6lypZSZUspRUsqGqK/xTGCWECLiIuWWzWMfWl0+9qrHOdTDLKdd56tCiN1a51ISys9YSju3bC5l\nXYzTXr8zAIvmgysLnJCapORRz8sla0KMEK+0EJR7Imt/CDk5v19KOV1K2UBK2RHVWrKj3BUfAHej\nrPAP8ij7ovdDCPGYEGKr172uS/a9vAAhRBOt8zROCJEMPHOx4wsjUsrdUspeUsryqOsti/oAZfEd\n8Kj2+1GUonZq2wlcxjvhw0qUsmml/V4BtNb+Vl5hXr7yekFHqhe5yauvTPu+i5eiE8pQm+iVFoVq\n5W/ykqPftPQs4qSUNp+8EqSULq/trOu5nPwKjOvROTkAqAE0kVKGoAQFlNLLooLX74ooCzke9ZC/\nl1KGef0FSinHXEkFpJQpwCiUK6byRco96X2aTzbHgL4+dbFKKdcKIVoCrwP3o5pVYajmXtY1nsql\nrKvhFFBOCJHXvbtspJSJWn71vZLro9xLaP/P7xNCBAJVvfZnpVtR93YAqqPzmHa//0K5LfK6jlzv\nhxa58hXKBROp3csdZN9L3+cCMA1loVaQUoaiXmCRy3FFAinlHpQlWdcr+WegvBDiNpSL7DuvfcuA\nxkKI8ldQTJbibqn9XsmlFXdu9/5KyS0PX5n2fRcvxVcoJbpQk1NQ+iMTqOP1voZKKb0/KldyPZeT\nX4FxrYrbLISweP2ZUL6gTCBJs3aH53LeI0KI2kKIAJR/6ScppRuYCtwthLhTCGHU8mxzOQIqhBiq\nhVP5CSEsKL9wErDX67DXhBDhQogK2v4ZF8lyIjBYCFFHyz9UCHGfti8Y1byNA0xCiGHktFZnaueG\na3V/8VL1z4N1qI65F4QQJiFEV6DxxU7Qrt1f2/TXtrOYAgzR6lUTeAqlMADmAHWFED20c4YB2zWl\n4s0Q4Fsp5UlUp00NIUQZVJTKwTyqdbH7EYh6oeK0+vcmpwI7g1Jgfl5pwcA5KaVNCNEYFVlRZBBC\n1BRCDMiSa00eHwLWZx0jpUxHRY58AxyRUv7ttW8ZsBSYI4RoqMlGsBDiGSHEE3kUuxL1jKxSyuOo\nPocOqNbpljzOOYPygec3pYGXhBBm7Z2qBSy8wjxeQL3b84UQVimlB6XQxwshSgMIIcoJIa5qqsD8\nzi+/uVbFvRClpLP+3kI196yoL9Z61JfRl+9RCuM0YEFFeyClPIbqFHwD9SIfQ0UgXE49JUrI41Ff\n7zuATlJK7zkzf0H5wLcCv6L8urlnJuUcVEfGj5rLZweqIwmU3/03VAfnEVTnoneT8G0t/RCwRLve\nK0ZK6UBZW0+iPkKPoKJ07Bc5LZNst8gebTuL4ajojSOoF3mslPI3raw4VEfkuyh/fROUP/Q8mrJv\nj+rFR0p5CtVRthP1DAfnUac874eUchcqpHAdSlHUQ/nWs/hdy/+0ECJeS3sOGCGESEV9YGZe5H4U\nRlJR93eDECId9Z7sQLVivPkO5bKbkkse96Levxmo1t4OVLTKstwKlFLuQ8nFKm07BfWhXaMZTbkx\nCaituQryM1Z/A6q1Fo+St3ullAlXkoHmanka1Ufwi2ZsDET1ba3X3tllqNb/1ZLf+eUbJWZ2QCGE\nREWbXFVsd2FBCLEB1Rn0TUHXRUdHp2AoTgNwiiVCiNZCiGitOfw4yo+cWytGR0enhFC0Rw+VDGqg\nXAGBqKbtvZqLork8BAAAIABJREFUQkdHp4RSYlwlOjo6OsUF3VWio6OjU8TQFbeOjo5OEeO6+Lin\nbzmu+18KIYFj7yjoKuQLXabtLpDBNvMeriUXH+9Pi487XvpgnXxj9cuLuLP8hwVdjevOlci13jlZ\nQlDCX9C1KNqkv7aUFgVdCR0ddFeJjo6OTpFDV9wlhJLQ1NQpnuiyeyG64i4BFBffto6OjkJX3Do6\nOjpFDF1x6+joFFpWv7yooKtQKNEVdzFHd5PoFGV0/3bu6Iq7GBMUPv3SB+no6BQ5dMVdjPntsbCC\nroKOzlWjGx55oyvuYkpQ+HS9malTpNENj7zRFXcxRb4xuaCroKNzTeiGR97oirsY8kByXout6+gU\nDcZt1+eDuRi64i6GLPhycUFXQUfnmhi22FXQVSjU6Iq7mKHHveroFH90xV3M0P2COkWdlp2bFXQV\nCj264i5G6INtdIoDm1sHFXQVCj264i4m6J05OsWBasdTC7oKRQJ9IYV8Zs/mDaxe+Cset5tbO3Sk\nTuPmCHH9F2zRO3N0rie2zAzWLJzL7k1bKF2+LG27P0BkdNl8L+fU9O75nmdxRFfc+ci0j8eydMYc\n7PZHQJpYt+RNmndoTZ+hI65rubqLROd6kpacyJBH7iP5XHnsme0xmfey6IeODPzsK2re3DjfytFX\nabp8dFdJPnHi0H4W/zgNu20hyBeBZ7FnLmL1oiUc2Ln1upWru0h0rjdzJk3k3NmG2DO/Bx7F5RyJ\nPXMsE4a+gZT5t7ys3rF++eiKO5/YtuYPpLwLiPRKDcZp78bmP3+/buXqLhKd683GZctxOR/1Se1A\ncsI5Ek6fzJcy9FbjlaEr7mvA7XKRGHcGp8OOv8WK0XBhx4rRlIwlIKAAaqejc/XYMtJJTohDSom/\nxQqk+BxhxyMd+Fks11yWHv535eg+7qtkyYypzPz8I5wONwbhos09D+DxLAe2AjdpR+1BGObTrP1v\n16UODyR/wILrkrNOSSU9NZkv3hrK1tVLQRiJiCrLTS2aEX96PA5bQyAAkBiM/6Nq7ZsJCY+8VJaX\nRA//u3J0xX0VrP3tF6Z9/CUO2w9AbeA4f8x9gQatbmfr6ocRxptBGnG7/+LpYe9SKqbcdamHPrRd\nJ795/8VnObS7Ei7n30AwZ08sYcUvA6jVsAm7/m6K0dACxF5CIwUvjvn2msvTXSRXh66482Db2hXM\n+WoScSePU7VuPe595lkq3lALgNlffoXDNhKltAHK47CNZ9uau/l00Sp2/b0Wj9tNvWYfERgcel3q\nV+14KqeuS846xRmHLZMF33/Nn/MXgpS07NyRzo89jb/VyrH9eziybz8u5w9kq4Y7cTm3ULpsBo9O\nH8jBnduIKPMwNRs0wWC4Nk9ry87NdGv7KtEVdy6sWjCHSaPG4LC9AdRh04rf+Wf9Q7z1zTRiq9cm\n8exxspV2FpVwOu0YDAYat73rutdRj3fVuVI8Hg+jnn2Sw3usOOzvAAbmfTuRrat78fZ304k/dQKj\nqQa+asHjrsOpI3MpW6kqZStVzbf66Er76tEVtw8ej4epH76Pw/YF0BAAKWtgt5mY8emnvP7pBCpW\nr8e+rb8DD3mduZGg0FJYg4IB1XG5aeUStq1dR2hEOK279qBM+dhrrp9upehcLbv+WsvRfXE47EsB\nIwBOe0NOHLqTf9atpGL12rgcW4BUIPj8eWa/36nZoO757eSEOFbMncXpYyeocfON3HpnF/ws1iuq\ni+4iuTb0qBIfUs7FY8vIIEtpn0e2Z/8OFY/98Muv4GcZBXwPHAXm42d5np79BiCEwOmwM6LPY0wY\nNpE/5lRi/pRUBt7fhU0rl15T3QLH3qErbZ2rZv+OLTjst5OltBUGbBlt2b9jK5FlYmh+Vzf8LY8A\n64D9GIyjsASspt19jwBwYOdW+nVtz89fHWHlvGp89/6vvH5/F9KSEy+7HnoUybWjK24fAoJDEMIN\nnPbZs5fwqBjOHDvMphWLad2lE7UbLSUo7F6q1J7Cy++PpsVd3QBYMXcGR/YJ7JnzgKdxO0fgsH3L\nhKGDcDkdV1ynlp2b6RaKzjUTWaYsZsveC9L9rfuILBPDhqW/Yg000fyuGpQpP5yQ8Edo3jGJUdN+\nJiQ8AoAJQ9/ElvEWTscHQB/smVNJONOQn7/6/LLroRsf147uKvHBz99C6673s3LeqzhsHwGlgH/x\nt4zA3xJBv67tgAbAOYQ4zvPvjuHWDvfkyGPNomU4bI+T07JphJSlObjrH6rX97HmL8LqlxcRqAu6\nTj7QuO1dfD/uPeziW5CPaKkzMBo3M2XcFmzpTuBGYDMh4eF8+MsiAoJCzp+fFH+WuJNHga5euQrc\nzsfZsPRZHnv1zUvWQTdA8gfd4vbB7XbTsnMXmtxRFrNfC/ytt2ANvJeGbW5m/45/gZXAbOAPpBzH\nZ28OxGGz5cjDZDYC53xylng86fj5+19WPcZt70jg2DsK1TBgj5Sk2t24PTmHOdtcHjKdngKqlc7l\n4nRk0mfo25SrPAuz342Y/epTvup0rIFB2NKbosYgTAO2kZJYmVHPPpHjfKPJhMfjAtJ9ck7H7Hfp\ngTiFWWk73B7SHO4caVKTd5cn/4b15xe6xe3FB688yeY/1wBqGHlQaCnenDiZclWqMfD+rsAzQEWv\nM7qA/Ijfpk+iS+/niT91ggnDh7B382qUgv8FGANUAWYRHGYmtkadS9aj2vHUQjeUfdnBJGbsiiPd\n4cFsEHSuHk6riqF8vfUU209nAlC7tJWnbo6hbLBfAddWx5vEuLO81qMzGWkJgADMNLq9Bb0HvkNw\neCkea1wdZYxkqQN/4C0O7ux0Po91Sxbw/Qfv4Xa7Uf0/vYHXATd+lo+4vUfOVqcvhXUCqQynm0lb\nT7PmaBpSSiqE+fFE/WgSM11M3XGGZJsbgxC0rxpGz7qlMRqu/0yfl4OuuDW+fW8om/9cD3wHNANO\nkpb8MiOe6sWkPzdiy7Cj3Ca+RJCalIjL6WB474dIjr8Xj2ciqjHzDdAJ/4Dq+Pmd4NWPvr3oFK/j\ntndk2GJXoYvPXnM0hdn/nmHsC5I6leDoGcmbXyewcP857r9d8mF7EAJ+WpHJsN+O8GmHqljNemOu\nsNCvSwcc9huBD4DSwEr+/v0pKlSbStdeL6AMlTCfsyIAJwD/bFjFF2+9jcP2OdAEOAG8gMHYAZM5\nkTq3NOSunk/mWX5Q+HTuLD85/y8sHxi3/jgxFTKZ/ywEB8AfWxy88+1RrP6CUU9Lbr4BTp+TjPg2\niSn/SHrXjy7oKgO6q+Q8y36aBwwHbkVZJeWAL8hMS+CnieNxOuwo6/k9sudtOA5s4vbuPdn85zIy\n06LxePqhhgVbgGcxmRvRuksd/rd41fkBPLlRGK3sLObtj+f1h5XSBqhYBjo1g9KRkj6dweIH/mbo\neQfUquRhzVHfeS10Copdf6/DYU8FPgfKoGS7DfAa87+ZwtQPx4AIBe4CFnqd+QMhEWq+7Tlffo3D\nNhhoqp1fHvgSIY4weMLnvPbxBExmc551kG8UTqV9NNnO4WQbbz4KYUFgNEC7hlA6Al7oIWlQXRkk\nMZHwTh/JsgPJhcYlqFvcGh63G6jnkxoJWJn3zXJczndRsa2TgfbA/cAkSkVXICa2Mn+vWIzT4Xs+\nuJw3YQ1wYjLn7j4ICp+OfGNyobOyvTmd6qJmxZxpDifcWOXCY2tWkpw55PxvKqZzSXZvWgdE4x2X\nraiAy2njjzkmkD+gjJC3gQWAGZjPPU8MAuDM8aOoTktvSmMyhxMWGXXR8guzX/tsmpNqZQUmo0+f\njQNq+Qy5KBUKQVZItruw5vEu/5cUW4v7yL5dTP/4PaaOH83+HVsueuy+bZsAD8ov7c2fgBuXczbQ\nFmgMTEBZ42uBCSSfSyDu5HEq1aiN2W+Vlk8WEkvAKirXyt2v/UDyBwVijeyJz+S7bWeYuv0sR5Ls\nlzy+Srgf63flTPMzwdqd4PG6XClhw05B5fDL64DVuXJcTgdrF//CN6PfYsGUL0lJTMjzWIctE7fb\nA5ziwvDWL4BuuN3DgZuBu1F+7iVABeBN1i5eDkCV2nVArPA5fz9C2Igonbfr4L9W2naXh+UHk5i0\n5TQL/z1Huk9noy+xYf7sOipJzxlbgL8frNuZM+3wabA5BJHWwmHrFo5a5DNzv57A3EmTcDrvB+nH\nslnPcMd9PejZ7/ULjv1mzAj+nPcb6hv2IcrN0R7YD7yA8nd7T8sqgC7AP0ArTOamHNi5lcZt7yIm\nNoRjB17E5XgBMGE0f0F4VCYNWl0owC07N2NBAYT5Td56mg2nkrm7ucTuhLf+TKR7jVLcXSPvWd7u\nq1Wa92YeAySNa8LuozB1MfgbTAyd5Kb3XRKjAaYugfRUM43L+Vp3OvlBRloqw3s9RPwpK/bMdpj9\n9/DzV+0Y8sW3VKldP8exyQlxDHn0flISw1BuuweAUUAsyqreC/T3KaE8UB3lSrmBw3vGAHDvs8+x\nY8ND2G3+qHdjH/6W4fTo+wJmv9w/0v+10k7KdDFkxWHKR7tpUlfyzwHB7MXxjGhdiXIhuVvIUYFm\nmpUPpt8nqbzQQxIZCgvWQlKyYPKv4G+WtKoPB0/CuB8F99aMxGwsHLZusVPcZ44dZs7XE3A6fkf5\n9MBh68PSWW1pflcnKnlFdfy7fTMr5y3CYZsBPIJyjUwE3gWsQCWUApcohZ3FAVTz8ygu5zZczo4Y\nDAaGfvUdP038lNULn0B63DRt34H7npuWw/9XkKPG9sZnsuFUMtOHS4K1b9H9t0keejueZhVCKBWQ\nu5+ydlQAA2+twE/L4vhklp3oYDO96pXipuhAZu2Ko9/HKXg80LRcMG+3jsJUSHreixu/TJ7I2eNV\ncTo+BQROOzjtc/jsjUGMm7MwR8f3Dx99QFLcbbjdtwDPo2S7L2ADolCdkf8CrbxKsAPHtP3zsFiD\nSU9NJrZ6bYZ+/T3TP/mYQ7s+JjQymm59XqRFp2651rMg3CPTdp6lRQMXAx7QEu6Q/LBMMmndKYa1\nzHuqib4NYpi/z5+3vk4k3e7hpuhA3m8XRarDw6w1Z/lqno1SgSa6VY2kTaXrM2Hc1SDyc+mhLKZv\nOV5ggY+Lpk3ix0/+xekYmyPdYHiHbk8F0KPvK+fTpn08hgXfrQd2AjegBPc4yhL5HWWVuIGngedQ\nvr/fgZdRlvgqhKiF2f8IVWrX4tWP/kdAUO7WZm4K+78eQfb9trOEVjrHs11zpg/7WhBLaTpUC/9P\n63O1dJm2u0C+DAUp1wAvd25P3MmxqAFgWXgw+9fnwzkLcyze27v5TdgzQ4EzqL6b3UAgMBR4EwhC\ndbJPB+oDGSgf9yFU1Igbs18MiD08+OKrdHy412XVsaB82r1+2cd3b3oo6xX4ZXPAba/A9B7VC42l\nfDGuRK6LncVtMpkQBtsF6cJgw2QOxePxkJp0juCwCJLizwC7gLlkd778ATyBahJuBg4DA4HPMRgt\nSI8dhAfpOQ1sQspAHDYX+3e8xuRRI3hh1NgLyvZV2i4p8e2+k1LicEtMBoEQ4HBL/I0iX1eINxoE\n9lxG3NsdYLLoVnJhx2gyoyxmb1xIjwuT2YwtMwOX00lQSChOeybKGFmCcvU5gZeAV4EpqFjsXwC1\nJJkwZCI9BlTrcgjwKE6HAI4x47N7qFyr9kUXBs6S8XXNAzAZRI54Z7dH4vJI/E0GnG4JyHxXpEYD\n2H1eKqdLRYXk5ztUWCh2ivuW2zvy/YfvoyyMrPC7QyB+ZvffjZjxv09AOkD4ExRiBe4lZ4/5bUAj\nYBHKUrGjfH8PYQ2cj8MmcDoygBEoCwbAhMsxhI3Lm+J0jMzh9/NW2hkeD6OTk5idmYEduGGhH481\njiHV7mbqjtOcSHZhNiphc7igTLCJh+pE0aJi/jTRWlQMYdjKc9x/mzxvmew5Chv2QO9Oul+6sHNb\nty7M+nw8LmcjQPPbiq+Ijq3CGw/fR2LcEUBisUbh8RhQFnRW/4wZJbMLUS6TZFS/TksMJjtlyp3l\n1JHDqLEKj5LtGqyAw/4Uy2bNylVxB4VP5+bmn7DN4WBkUiKbfnJiFoLWFYJ5uGFpft6TwNIDSdhd\nkmCrIDlDYhDQoFwAfW6KoXRg3mGEV0LLiqF8OS+JkU+p/hYpYdKv0LRCULF03RU7xR0UGo6/NQCX\nowuQNRPaUjxuD9vX70dZ13VBbiMtuTvKV+1LDNAaZd3UAe4EniE9JRXl/x6D8gN6E4bH48bldJ5X\n3OO2d6Sl1xH9ExMIttvZh3o9ZiU76Pv7EUwWwYgnJU1rw4l4GPk9VK8Areu7GPLVaQJMRhqUvXa3\nSsVQf+6rFcXD78TRvI4K6ftrL7xwSwzB/sZLZ6BToFSqURu3+1OUm64tsAfkXk4cFEhPd2A+YMWW\nOQ2lpC+UURX1NByl1L8G1uFxfc6pI5Fa+kJy9ucAsjRpyZsuqE+WUXLC5aJXQhwfSMnDQKKUvHYs\nlYEJ6VSO9fDjW5LSYbDqH8nI72HsM7BpbwbDVhzmkw5V8csH6/uB2lGMXpPJfUPtNKoJOw8JnDYj\nw1sVjgEz+U2xU9xbVi3D7YxFCeVSlI/6LaSnOWoQQta8wvVRFvU01G1YjrJK7kBZ21+jFktohWpe\nNgVCUHNwrwdmkrNXfj5lK9XGGpitYOe+kXT+92GXi412O8dQA4rRchppgm7dJM20PtPyUfBuH7h3\nOPS9G/o/IPnx1/hcFfesnQksO5KA3e0hyupH35tjqBZ58XmRO90QQdNywWw6mY7JKniyU5CutIsI\ncydNRnreAmoCW4B2wHakZxbKoMhSuL1RA8X+B2Si3IGVUVMvRAI9gHjgB+Aj1HsxEdUZ/x5wBBV9\nAuDBzzKLW27PXhwky8rOYlp6Gj2l5HFtuzQwxiOpbHcz7SkV/wzQuj4cOwtzVsHbvWHzXg9rj6Ve\n0OmXYnPx8YaTHErNQCCoFxnEc7fE4GfKW8FbzQbebh3L7vhMDiXaeegGP+pHB2Aohm4SKIaK+/Sx\nIzgd9VEC+qCWmjUxTn2fo2sBe4A1QD+UW2QcSkG31o6JQMXAOlFx3AADgG6o+NiWCLEFP8ss+gzJ\njsn29WsfdrmoJwT+Pp3BGQJqV8pZq8gQNZIrLkntO5Z8Yaz1+PUn2BafykvdoUJpWLbJwZAVRxjZ\nJvaSyjsywEz7ar5DnHUKO6ePHkYtRF2DbFn+FdVZ6augbCij5AWUibAZeB/oqe2/STvXBaShjBgB\nDAK6A32A0vhbZhETm0nLzmrFpdw62Y+4nNzvk3YCiA7NVtpZ1KkEv2vDKmpXkZw8mlO2bS4PLy0+\nQO0qkhH3g80h+XJ+KgOWZfJph2p53xyUL7t2VAC1owIuelxxoMgrbqfDzl9//Mah3buIqRhLmQqx\nmP1+we3aS7ZglkbFsg5GKd3S2tkngbIoAc/6mrfU/v4BQoEk7Zgsq/w+1ERTi4FvgUE0aNWCRwbM\nO7/Cja9wZ3o87HM5+UtKn7VFwOqBjbuhRoXstBPxkJIO0RHw20awuyXfbDmDUUCAn5FGMYGsO57K\n1Dehcow656Zqyq83Zs0xuteMIiHTSYDZgNkgSMp0UT7MQvMKwfibDOw/Z2PDiVRMAppXDKF8iD5g\npjCScOYUaxbNJT05mRtvbUVsjRokxq0EDqIUcTiQiFr0YB7QEdVqBNX/8jqct4Mboizq97Xt1agW\npQllqKxHuWAeR7kHP0SIbTz66mBadOpGRPScHFZ2FoddLhI9ksXAw17p5YDTSZCQogyRLP7aAzeU\nU7K6ajucS0jGKAzYPB5iQ/z591wmpSMl45+HrCUtm9SCjgNdfLTuJJXC/El2uCllNWkzVUKjckHU\nKGXF6VYW/OFkOzFBfrSsGFJs58wp0uGAqUmJDHv8AZITwrBltMbfuh2z3xaEEKQmpaMsbgH8iHKR\nlAaWoSbbMaB62V8GXvTJ+XWUQ2MNyr9dFZPfMFwOB6pjpyfqhRlBaMQxJixbBeRujZxwuXgw/iy1\npSRRSgTwMepTMAUYYxJIM7zYQ9K6Phw6De9PV83K2GgY+6P6vW4XxERAzfKwcD2EhsCC0TnL2rof\nhk2GtEyoURFKhcDKLfCAE46aBPvMRupXDGTDqRQ6NVMDcBauFzxUJ4qO1SKu9XH8Z5SEcMBNK5fy\n6aD+eGRnXI7S+FvnUalGDHu3bQFZBaWkt6Jk9F5gE8ot+BkwCzU1w1/knBjNg3KXDNSO+xyTaQFm\nyxIy0zyoAWgNUB+C17j3mT50f/qVPMcezExP493kJLoDP6PeoqeAOGCgn4UDAW4sIS76PygpHwXL\n/oavFsDIPrD4L6XE61aBNduh/S1wKk6w84jkqbvhEZ+owmGT4d8TcDIeOjaB7Qcg/Sw86IAZRkGd\nCsHsSc4gqpSbxrUlOw8Kdh82MKJ1LDFFZLbKK5HrIq24vx45jJXzXLhdY8huKo5BiElIuRKlHgES\nUP7Aaahok/4Ig5WwyAAS45qifIHedITz3ugEKtdqyKMDBoAQfDrodRLjTiCEiRturM+g/33NHffn\nHbv6XEI8N9ttvI2y/cehvIqpQFt/Cx3alSXD6eaHnWfYecZGZAi4PZCYBrUqwl1NIbYMVCkLT38A\nrz8EyWkwYgosG6cmeMpi/lr4YysMfRTue0sJf62K8O7ncMSpPJ8Lg2DWCAjVAmJOxEPPdwSf3FmF\nyDwG4BQ2irvidthtPNO2KbaM78iO2bZhMDYHbsbj/opsec9S0vNRLpGNhEWWJSUxCY/nO7Lde6Bk\nuhUqSioOg1HSuksPHnqpP/O/ncCiabNwOVMx+4dwz5O96NbnpTyV9jm3m1ZnTrERNdbyIKo9uwAI\nE4KHA4NoclcMSw4k8dOeODKdkugIiE9SAYdtG0DzeqpPJyEZRk+DuSOh2xBoVAOGPZ6zvKc/gIfb\nKaPkywWqg3PaQrhxKwzyQGUztL0VBnqZ/VN+g3V/WxnS4trXev0vKDGK+6k2jUlPmQl4rzw9EdiB\nsii8eRs198gZVBPzNGp0ZAZKcbdHidR0YCTwCma/ndRv7qb/uAubiHDpUZBSSqqdOsFZlNMlizOo\nCNtdZcvnGITT/7cDlC3r5LaGMH+dEmiTSfkJj5yGZnUg2KqE8/Z+0PJGGNxTKe9Dp+DlT9V2szow\nbgb8sgbMJgg2wk8pME1Aagd43mfq5CFfCaqIMtxZRPzexV1x/7NhFR+9Op7M9F989jQDPkWFq2bh\nRoWzBqLkOR4l0+EoqZuOipxKQo2gXAf8gp/lCV4d/wF1m7TItQ6Xku2fM9L5IzmJOT76YxRwMCCQ\nt8LCGV11GHeW/5CEDCfPLTzA4J6waCNEhcKKbXBDeThzDsKDITkdRvWBc6nw+kR4ry80rwseqToz\npyyG2SOU++T2fipGOzYa5Fn41wFhZvhuuPoQZKEPwCmkGAxGshY9yEbABcNbQHU8uoANKMEfgHJ7\nbAReAV7Tzo0BamM0vk+9pnfw3DvvX5BTs05N1aJkXj3WHo8HB2AxGPBIiUQ5Y4xetXGhbridbI96\n+15tWTh5KSaDgRG3x/LG8sO8u9uFxQ8GPQwdGqti9h2Dp8dBi7pZ1w57jsCdrynrOdMBz3ThfHSK\nBHp1UP+nLM5+0E6tMm6PulqDAVxuMF7E2HZ7VOytEAKXR2IspoMaCgtKrnOTYcGF8u5BdUQORLUQ\nh6EG3VQDxqL6ayJQoyQ7AU78Ax7mkX6DclXabpeLNve0zJHm8HgwACaDAaeUmACjEOdr6EHJmVGr\niVFT5i//+xa7ygYRGWDm+UYxfPDjKVxSdbxPH6r6cDwe/t/eeYdHUXUN/He3pvdAQujSmwrSiygq\nHRS7KHZey2tBfVUQBLFhAcsnihWQJoioiAURBAt2VAQFlF4SElJI32y53x9nQjabAqEoG+f3PPs8\nuzszd+7M3jl77mmX1z+C+SsliSbUYfhq5ku4qtaQnADP3y5KiMcr+80aD/e/AqnG7VBKtkHZPh5v\n6SNa9Vj1+jRWi0JrjU9z0iyUcDiCUnDv+vMPXn/0EfJy0hFny6uUicLdiB37T0SvLf3ubWAU0AsZ\nxHOQyJOLkUp/jyM1SO7GZr+AxxcsI6Vpi3LnTeh/Bo/lZDPKXUIIcGFoGHdGRvHfrEzWedwUIZGy\nhcb+5zqdnO10cmmJi2122FUCcU7IdYNHQ7vsPeT2n47FAimRVu7v1ZDnBzZjycYDrM08wMCu0s5f\ne+H5JVDogs/WwYYHoMgFffvA4jWQminaS0PD57rvgDg137gXUhJhwUq4R8FaC1i/gG2p8P0mGdzd\n2sCPf2iuHlIx3HBHTjGz16fxa2oxVgURoYqcAk18uJXzW8YzuHmsKcCPI1prPnt7HotffpGiggxg\nNVLwCWTMHgSeAeZS5oSciaSv/wosRsTow0hM9r3Iqk2DgP9Dqe9ocWovxs14vVySmNaaVW/P4bMn\nJ7Db56OJxcIdUdEkWKzcn53JHkMQxyB6e6JSXB4ewZdac74dPvWCW0OCE9KKwVlSyLzMQgpcEPYu\ndEqK4K7u9ejVMJKbPv6LGwb7SIoTofzuV2Liyy+EG58W4Xz26TLWv94ggrdHkgh7gLdXQ7MUccqP\nuRjGvgK9fVDghYdmiSll536oGweN60LnlDDs1vJjVGvN8q05vLv5APtzvcSEKYrcmhIPtE8O4er2\nSZwSd/il2P5Jgk5wZ2fs56HrRlJUMAYRuuORwX0m4qz5EykuNRxJwNHAJ0AJoom8jpSx/AHRtK1I\n+FMHxE44nPCoeJIblw89ajGwC4MOpDNRaz5GXJP3FhVyXlEhLRHX0CWIK/MBo9VnXC6mKQiNkalf\n28bw61aYOBNuHCID97l3RJtY9rWX+1duZ+bwFsSF2Q9Z5w8chFuflf2n3gJ5hTB1EeRshN+2w8z7\nRDiv+QXueUmiS37bBjcPlwUPQKaiuyJh7n/gP0/Dqc1gymjJzpyxFKJDrUQ4ysdyZxV5mLRmF6OH\n+7gsSs4HA+aCAAAgAElEQVQ56RrNGS1h824vD83MwOvTDG9VdVVBk5qxYtE8Fjw3E1fxy0gCzY2I\nolEHiZByIWpBX+NVOt5DkEd5HWIumYs8GysRk0kSMAFNDm27jq5Q0W/VojdZ++QE3taaM4CvfD5G\n5mSTiejvpyO5lK8jfwGbtOaG/Dy0A3ydYOmFEOoUJWLeCph+p2jRBcWSizDulXyeXruPe3vVp26E\nncRoCQFc8iUsXAWPj4Y2jcS5ft8rsGY93DhY8hlcbpj+HlzxsIS9pmXBC3dIv+vEgFdD7wuhnw3+\n71147Ebo1EIygse9Cn2SK4YGLt+aw8c703niVs2sT8Bq0Yy5GOIi4aPvipn89k6eOrfpccvqPBGc\n/IafAFa+swC3eyBwDaI1T0W05YZIzOoSIB0Rwl2BzojdOwTRVkpjXrsg2vp0RLjPRcKi1uIqimbD\nd18eOufU9QNZUJDPCK25xWgpGfHB5yG5mFuQML9WSCzLCCRvTTng8RuhXRPRHk5rBuOvgrkrYGgP\nOL8XfPoD/O9yiImC2z/Yxmfbs/h+E2TnwXtfwVmnw0VnyiozCdHw8HVgs8L1g0Q4Wy1wdkcR7pt2\nwfwJcHFf6fuONNHA/+8O+OVPOf/1gyAsRLSY+y4HR4iXCZ/vYGN64aFrXrE1m7M7aS7uK9PYey6F\nzq3kGlo1hEdu1Ly3JbPCwsEmR4fWmiWvvIir+FlkJG1FQvQGIjPHJSh1BaJRP4uE9t2IREs5kGze\neCTFfTTQE9HA9yFqxU2gF7J8wRz8/VrhMfNZ8dSDzNaazsiT0RspXHw2pVV6JAVtNaIePQIMAcLC\nxYkYGyl+livPhTNawbe/S4LNlt3gsMMTN8G6tHweWrWDjHwP738tSsvsT2DiNaLQKAWnN4fJ10KY\nE0b1lz+DmAgYN1I08cQYeHuSCHCAZd9AmAMuPBPe+RImjBLHplKyEMIT/4H3Nh9gzq/7ySn2lN3n\nzQd46HpNRKhEpzx8HdSNlXMM7wmDe2g+2Rq42PfJRdAJ7p2bt+Ep6WR82orYq3sig7g/0AalfDhC\nbkCmkxHYndOQBJyOAa11Qsq2XoHEbc8DEvF6OpK6c9uhvd4bl8N2t5tAd01pFHg8UkcwD4nsvgkJ\nMnwHOOgSoe1P+6YynQPZtjNNBtsZzSCrxMM5fVz06wSjHpPFCgKPt1lFeGYFrBB26imiRU+ZDyvX\nwZwVcN0T4tysGwu70iX8yh+loGNLSKrvYtr3u1mz46BcW4GL9qfIA75zv/TZn1PqQbHbR5Hn5FjK\nKdjxejzk5exDEmu2IwI5HpnH3QC0QOsuWG27sDvmA/WxWHagLD8jz0CgyaojMtMcgESSXA10oKgg\nG1exLO7ce0h32vd8nr0+Xzl3J4jL82zj/W+IeqQRI0xPROVp37ScmweA9k1kvNis0KoR7Nov5Rvc\nXii2F3PLRV427YIx0yEtWzRtf9o1EXOHP0rJLHHlOpj5Caz+BSbNgkWrYYSRJ7erkjFael53TDb3\nfradnCIPXg0ZeV7aNJIszpYNRGCXu4ZTYF/+4RcY+ScJOsHdtG1L7M5vjE+tEPu0P+sBGzdNuoPT\neq6ifbdPGP3g9Vgs0YgG4883yNQyAln9JgHwYbV9R/1TWgJl3vUWDgerA46uhwjvzYiLczvyeGwy\nenEFEO6EdX+WP+6nLdCsXtn7U1LESfPtZujSRlaXSY4TDXfTzoqrcbg9sHE7xEWV//7HzdC9jWjg\nj8+DmR9Cj+Ro3G5FYTE0TYZ1W8of4/PBr3/BkO4wvI/mpR9T+WZ3LikRTn7aJE/lKfWkn/5s2gXh\nDgthtTTB4e/GZrcTHd8QGUnNEZt1gATjC5q1b8WwaxvQquMCeg1JZcQNtyAO98DVXlYioa+3IAtg\nA/xMRFQd4pPfOzSu7UrRyGLh64CjLUjVkl8R63prJC3nc0SADwJ+/lOc3P78tEXGi8stY7RpPdiw\nXYTjBWeKqWNgFygshhC7mA4Djw/MtvT54PcdYj5ZuQ4mz4Yff7PSMyWS9CwZo00rGaMbtssM9ar+\nkBjv5am1e0nLLyE52sqvW6FxEvy+U6JPyvVhEzSIPLlt3NZJkyYd90Y3pOUe/0YNUpo0Y+XiR/C4\nXaDPRmzcjRDjxQ/AGJQ6yI0THqPv+RfSe/BQGjZvzcHMVLb9Pg0ZgglIaODtiKYi8bGQic3+EPUa\nF3LZbXcTGfcWyW98B0Azm41JhQW4kL+L3UjCwU5ghh3i20KvTvB6Nqz3wm8KPoqSf++3VolDJS5K\nsiQfnQvXDZQCT4vXiOnixfclpnpXOhQUiXPluz/kmnfsB69XHDL7s2XgpmXBhm2i1YQ6xBn56jL4\n7wWSKr9rn4VXhzSna4Mo9uW5WPaTm/5dxa6YliUPV3YePLsYcgrg0x/BqqDvabBiSz67czxs3uPD\n7YOOzSUpqGEdqBMr08uxr8K5jeJoVzecv5uWF/73ob/9pJzYcQ0QGRPN+m8fxOvpjKShL0RS0S1I\nutZcEpJTuHnyE5w57HzO6NuPVh278tHc1/C41xn7epBQ2KWICeVi5Nn4HkfIGCZaCzhv7rpy541W\nFu4tcdEWsaZ/ioT1bbXCvBDo3wdsyfB/aSJE59jB2kIUiJ+2SK6Axyumj683wBX9ZNaXkiDCcfzr\nEB8tjvXkeBHq3/4u7z/6Tp6N2Ej4dqM4GAuLoagYmqZIeOBTC2X8n99LtO2uSdFMPLMRLePDeGVt\nDj6t6dYWnn5LzCiJMaKMTH5TNPgZS8UJn5Dg4bW1B2kVH8bib9yc1lyet4+/k9ISFiXOz4Wfw21d\n6hFm/3tr+NRkXAdlHHfaru3MmfokP3/5KWJxjkXytRoBl2N3Ps3rX/xSYeXpN59+iBWLluD15KIs\nEdhsHnw+L+FRdSnKz8Rmd9JjwDAuv+MewiIiaTSgM3u9Hlra7MRZrWz3eHjqYA6rXcWEK8WAkBAW\neYuYcDWca8w1PV64Y5oI4VkTZNmjP3bCkq9gbwbUs0OeB4o0hNslwsRugyE94M89kphwmTFH1Roe\nnCl2vKx8+Oo3GVw2LQ9NmAK3A1weiLCDxw02BQOUePofGtCEhtFOPD7N4j8OsHJ7DrlFXuLCrWQW\nenHaJcFn8+6K553wukLlRJHv9vDjvgLaNxXN5M+98kA2SYLs/aFMqGZ1kRNFbY7jXrt8KfOffZas\n/dsRJ3sJonn3AZVCn6HF3DTpsXLHHMw6wGM3X8fuPzcBXmz2CLQuxGYPxeGMojA/nYTkRMbl5TMi\nLAyv1qx3l6CBDnYHNqVYWljAjLxctnq9tLLZiLJa+c3mYuHksoiOHWlw5SMwegic3UmE3he/SGy2\nxwP1rJDqBqdV9P8iL9SLF//MGx/DvPESAgiQng0jH4VrB8DiL8QPE+IArzHJCLNDrpbxHq6gyAP1\n7dDFBT9H2Xl8sORu7M0tYf7G/azbV4jNAqEORWaBjyZJMous7LxXTFaMaJnA6l3Z7DnoYUAXUZLy\ni8RGvnMfXNqyHj0aBExpTzD/mgSc5W/NZsHziykpno24AvNxOO+g95B6XP/ApKNu12KfyewzHuUH\nl4sWSrFRa0aGR3B/VHS58LetHjcjCvazfGp5W983G+HhN0XQNaoL6TkycNIywZojlvckjPV1QuDp\nW6V+w9Bxkg3pb3PbvFu02yWTYdYn8OYy2OKWRzoP6Gm1cZrXw5sB1zBGQUa7BC5pl0BlfLM7j/mb\n9zHlP5obn678vPe9YGX6wObcsGwLL93jo7FfhcziEug3BuZc0AJnNVXbTgS1WXCDOND+d9EQ0nb2\nx+e7DYkY+Q1HyJVMfH02TVq3r1F7U9cPPFSp8geXizuzM4k0yi/kKMW02Hi6O8tHmozLzcLRp7DC\nakm3PiuRTWlZ4hj/ay/0O13S2Z0lkgp3AInnLomET56CeZ+JPXnsyPJtPfWWaMdX94chY6F3tqQL\n2ZDah2cjsTQ9/I7xAokWxdShVWf6PvzlTpo1LyIxBvZlVjzvk/PBmZNI87gQFm3dy6wHytt73vsK\nvvw6nLu6NuDv5F+TgHPepaPISE1jxaI+2OyN8JTspGOf87jqnvuPus3eQ7pzT1YmCS4XewCn1mQA\ngwoLmG+zMdKvbKtC7H2BbNolTpHZY2Xq5vVJUfe3Vondezoy+d0EdNUy/VRKNN3A/1Gty9xOWotF\nvhUi+PcqaNyoOZZtf1Tow+Fcht0bRLK/MJ7RTx3A7a38vKVntqDKreZett3kRKCUYuyLrzH1zv+y\nb8ebWGxxKA5w3biJNRbavYd0P1QT/qDPx41ZB5ilNUOM75ZrzcisA3xeN4lYS3nTQGUBQ7szxHT2\n2v8kYiQrF259Dtw+8Ri1R8beWOBVX+n1VN6W/3c+LU7QBsgcOs9uIcqi8LnK2+618aouf+COLvV5\n7vs9vLe/iHMCva6UPVOqimuUsX5y5ycEtWdJKcWVY+5j+vKvGPviIzy3bBW3PzENh7NmjoWp6wfK\nAB/SnUKfj4+Ki3iGsrrZicAUrVmQn3/omOVFRYzOSMflho/9fJ5uj9jJbhxcFrZktcD1gwElcS8W\nYBfwoAMK3XD7/8EziyWEaeHnZW35fGI3PKcT5OTDglXQwS0ls/ZYYeqQptzZvoT3kWjeUnYD8zX0\naFj9qjbnt0zgtaEtaBrn4K1V5c876yNFj/oyVexRP5I3PqKc8J67Ak5LDvvbte1/C/F1k3lswTtM\nWfg2419+ipdXfUvPgcMPf6AfgWnrHxYVchYcEtog4/FcDUuLxEZRrDWPHszm7fxC3lsj1f1KWb9N\nao3cfYkIbRC/ze0jxNFY+pfyLvCRE7ILYPBYeSZW/gR7MsraSs2EFT/K2F71s5gplgB9FKSkhPPy\nBc05s0Usj1tVuVzRl4EGUQ7iQqvWOaOcVib0bsSkMxuy+mdV4byf/qjo0SCS1olhpGaKU7+UwmJY\n8JmiR8rJszBwZQS1xl1KRFQMzdqdfuT7xy7goTVvHpo++if4FhopvYG18hoA2Vok1+fFRUzIzmI8\nmqdKYPo8WLUW6qXAinWSqlsvwEJhtUB8JOwrELdTHzsMOAdWnCvOmpfeF5vh6x/B5z+LU2XNr4Yt\nOwTOHy/lMXcWw6VexV29UqgbIRWmruxcl44/ZXGJz40Nid69LSqam+4ZxrJXlld7L0LtFu7sUp9J\nn+3kmw0+WjbUfLNBEYaTG3vLRbSIC2P6Tzlc8YhErWzcIaaUYc2rr/ttcuwkNWxy+J0qobJaI9k+\nHw0rmSo1QpNt/CvflZWJ1VXM5UCqC64YL/bsXJfUF7HbKkZ91I0t01w/AP4bDuNvEHvx7ztg/Bsy\ndi+fLPkGINEhbRuL0/unLRDlhAs8oMIdPNitHnarYljrOJ5IK6BtdjFDvJrfrBbWWxUP9qzHkdA6\nMYxRHeow6tF0zjoNUPJsXdY28VDFwPqRDsZML6ZnO4lAWfEjuEo0TWNO7lLHQW3jrglf3fEx/etP\nY/meuxi7dTJriotZ4MnnIF56EcqosAiKtWZmfh6LCvI5FcldM7LOmQT8ERLK1Lh4Lknfz10eN8MQ\nd+hiJKJ8n/F63QZd20sFs1J27ofLH4b33FJkfl4reHpM2Xat4YIJkBwLWskUtG4c7EkXwX7f5WJu\nSbZGcc3pdYhy2vgjo5Dl27LIcXloEBFCpMOG02aha/0IkiJqVsqyxOvj2z35ZBS4OSU2hA5+q4c8\nsHo7VwxxERECW/ZIIZ+UBLjzOQuvDW3+t68yUttt3MeC1TGL33o8xRqKiUBxqS2CfiGhfOdy8WLe\nQX4rKeE/SM5wPGKLbq8UU+ISqGO1cmF6GruQavMPI7EsHxv7rQVWh8DTN0GX1mXnfOFdWLwKckug\nmxMuuU6ik0rZtAtueFIc4Rt2AFqclt9slISdq86Fu16E285IoWv9CHxa89m2HH5Iy8NugYaRYTit\nioRwO93rR9Z4lpdR4ObbPXloDV3rRxxSeFLzShj7+XbmP6j5cj3kFUnt72VrwZsex5Ud6hym5ePL\nv8bGfaRcevBpwuuL5jl262ReLczjdV8u1wzTJMbAR1+6WbylgMJiH8O0ZhawESnJMxrRkN9WisVR\nMn3a7PXQCwm2mooEXN0LrLTDrgS46xzRnP83A4b1EAfJqx9AhEdKq7a3Qvs25fuoFDRIlLjW6wZJ\ngs2X6yVW9ur+8PbnCqvbyS3dk7FZFKu35zBn436uGaRpWAdW/uRi5QYLT5zdhJhqppFV4bBa6NOo\nci/6npwSOjYXjaRHu7Lvi9w+Ct2+CunyJv8MZwzqykW56cQ39XBBHykNPPGDEhbmOPi1sIT70dyD\npJmdiqSzz1SKNg4nXRwOVrmKaYvYmIcg+ca3IeP7cTv06QpXxUpxpyvPk+SVNb+IqdDiliITf3gl\nO9eflg0AJSUarh0ghc1mfQIxkdCttUQwXXtqHXo0jMTr0zz29W6s4cVcNkRTXAJvflJMy8goLmpb\nuaP9cCSG2xnasmK9+b25JbSsr0iM0YzoU/b9ngx4f1vxUZ3r7+JfIbj3jXznUM7kQZ+PZwsOMn9y\nWYhQz3Zw9wteLBvgJWO/AUh09wDgirBwlkZGkWwVARWGrB9yEeJsbIIs+LTeCcvGSVhTv45i6548\nGyLccL9LkpInxsbxSl4uP/zu4er+ZX30+WRgP/Gfsgp/3dpI5b+FK+GS1omc2yEGm0Xh9mpmrU/n\nhbv0oZVzerTTTJnn5f0tmVx9at3jev8axDj45S8X53Qq+27LbggzE3BOGnoP6c6cokKiGnh58r9l\nUU4922mGj3OxnLLF+AYC1wFPWSzcEhXNiNAwlFIkWiysQ7TrEOB9JBP4ARsMOFOKOoGUE35lGcz9\nFNoVw30++MLuoFNoKN+VHOSXv8pr3Mu+gfAQmHW/lG0A6NNBzH8ffO5gTOck2taRmiI/7MunUBUz\ne4ys1g7Q51TNBeNyGdgs7riu1pQS5WDzT7KgiNMvQOWXP6F+xMmdgPOve+p+LSnhlGR1SGiDDPLB\nPcEX8Ft1Q4rCXx4RcUhoA+Rqze2Ulf3RSIRIv45lCxtEhonmfGEfGOySPLbBoaEMDQ1jYUId9u6w\nMOM9cTqmZ8PENyQGvFuAJj6gi0SoDG4Rd2g17L25LqLCyy93BjCgK/yeWXDM9yiQES0TeXqB4qvf\nxH7/61YY94rEwtbWxViDiVJ79lpdxDnddbnQ1IRoSSvPCTjmKiDeYuHisHCsxgEHfD6ikFo7fyJj\nuwQotsHgbmXHtmoI026RCKcrfTAdxa1R0VwfEcmTYfE89abi6w0yVn75C55ZBOedUV44OuySQem0\n2A4JbYANGQWc16VMaIMI/R7tYGN6YCbpsZEc6aB9nXDGvqzYtV+KYi1cBZ98Z2FAs9jjeq7jTa3T\nuDP27SbnQAYNmrUkJKxiVl+URbErW+Pzla1pB5IAYA/IGi4AcrQmWpX/f7MjNsJJwPlI7OpZwPb0\niv3ZniZJzDeER3CbYWoJs1hYHF2Hx77OYcgqL1Y8dEsJx+vL52BBWcIDQGoWBFoiIp1WsvMragqp\nmRB5AswWHZMjuOn0FKYv3M/dmW6So2wMbxHPuU2DY+GF2oDX42Hnlt+x2e00aNbqUDicvxPS5ZVx\n7I/WUhMkMEZiJ1QI/3Og8CLhpr0QYX8ekmecliW1P0rJL4LsEnjFauXZ6Fi6GHHg/UJCeVTH8cwb\nOdxd6KVhqJXmHgt70yvWF9+9H6JDyvchymEj9UCFXUnNVLRLOv5j+/bO9ViwMYPrphykwOXj9Hqh\nTD4ziYSTfDWoWiO48w9mM+3uO9m6cT02ewpe9w4uuvlOhoy6odx+RT6NzwWzP4JRgyTaY9s+mPkR\nhLlhD7KwUwlSEa2300mctfyAKURClz5GBnUhsgrlr1vFa933NNHiv/0d1m1QfFk3ifiANupZbbwQ\nkSBlUgCK4dwkeGJePhOvFc09K1fSeE9NLF8rOz7MTquEEF5YUsQdF0lBn9RMePl9xag2J2btyM4p\nEXROqViz2+TEs/6bNbww7l487ki0LiIqNpQJrw/iguvfLr+jGxZ9JuOvRQMxv81fCbn5Ml57IkrH\nduBhpZgUUf733OB2UYw4LVOR6fgGoEcxTFsk4aqJMaJJP7cQBoSFMD2iot25f2go/UNDD/1b7PZ4\nOGtzGmt+FROJUpIF/PXv8Fz/xHLHntU4mrtWZHJeF83pzeWP56PvYGcqnNHx+JdXsFstjOpQl1Ed\njq958URTawT3c/fdzZ+/NcbrfgO3ywHs5J0Zl1OvcWOmPhZzKPRPKUVKCfz4KbyzCpzhkJMF0zyy\nmFkbJPQvAzjV4eCZ2Ir1pi1IwZ3mxv6bgX5IyvmDb0B8OIRYFIUFipcj4yoI7apok+FhXToMuwca\nxcFfGRCuYVBYxeNv65zCM9/tZdC9xSTFKnalay5qE0/X+tXHbpsEF5lp+5h2922UFL+CiF5NRtEi\nJg56gEFJsTj97CKxFsUFxXD7ExAbC5mFUL8EVpVIedYkRCnZheKOyCjOCikf16dR1EGipF5A6pbs\nQHw5b2fD+Q9A8zqKfVmajnYnz4UfmZIQbbHgcMNTr8GL4ZLakpcPdjc0u2IeBR9ce2jfxHA7d3ZJ\n4f6X9hEbCcVujfJaGd+7flAsP/Z3USsEd3ZGGlt++QGv+2Uk1gOgEa7iu/jwzXmE3V62byeHgwyl\neN6l+dAFWwokoaXUynYrsqzwbVHRXBtRuRC0IQtLbUbqcDdCqnl/AqyJSWKfzycrAcbaD9kPAfJ9\nPq7LzuAX7cbtgboOxcTQOOItFp48mM3PHg97ERPN7lQph7UOuK+wgKVrJPmndI3KKKeNiX0akZpX\nQnaxh8ZdnX97URyTE88XH7yDzzsMEdogYu9S3MxmVfFuBoaWCd8BYeFMdbn4w63plA7PI85zhVT1\n+wJxTH5fN4noSpSJ0+wOnkHGcTxiKmmFpJwnaivvJ9ThT5eHpCgrDW3lRcfnxUU8UJhFqltjt0A7\nq53XohNYWFjAjPw8egAflsDPJeIT6giMVIrvbx5I20FJ5drqVC+CV4Y0Z2tWMXaLokms01xpKYBa\nIbjzsrOx2hJxlwR6ghuQk3mACR0+BsrKWD4bF89FWZkka81IyoQ2yPJMHak+pdaKOCZ/QQrOf4WU\nqncDDouFDrbKb+uwnP1EN/Qy+wqZcn74jeb2dzMJccOjiEMoHjG/lJZuaow4jUpZvucu+tefduhz\ncqTjUDKBSe0jOyMTj7thhe89uhGZvh3lvjvHGcKqkFA6FReRpTXnUD5xuydS68NRxdguQpOMRFIN\nR7TzORhlFiwWYq1WulQi8H8vKWF0Xiajh8LwXlJ1ctpCN2f/lUrrEvnzcCLPjX8GehOtOeDzMWR0\n/wqJYjaLomWCmeBVFbVi7pHcuClK5QC/l/vealvKqT27Vdi/lzOEVXWSOC0snHmUr2RcACxD0dVR\nddhRT2cIdyOxrmsRzXgG0MxqJbKKh+Inl4s9Pi/P3Ca1gyPD4LJ+MKwnnGGFm5Gp6cqA4xYCXf0K\nAI3dOrnKfpnUPtp17YozdBkVR+nnFcaoUorHY+OYHp9IQ6uVtwLaWgq0ttoItVT+2J9md5CJ1OH2\nIIWeHkVS2buHVVwCrJTJeTkMOENWrYkOl1KuT98CxRZZrvhyZJUo/6UJSoAlStHN4WTfyHcOex9M\nylMrBLfd4eSqe8biCLkKiTz9Apv9PsKjPmPYNTdWeky81cqj0TEkOhwMQLEUWAT0VYqzQkNoba/a\nq3xnVDTPKoVGqoE3Ae5UinHRVS+e+7XLRfNkWY7Jny6twWUXzegp4EpkSbRViHP0BaW4Pap8YkzH\nNfmY/DvodOa5tPVsI5SLkUrZ7xOmBjE01EbzKsboaQ4Hz8Yl8KhSPICMpSnAaKUYG1N1JFCc1crN\nEZGMVorByPj7DNhmtXJpJRFapeyzuDkjIIzVYYfWDcTJ2R3RtPshq0K9i5gjmzkcdHPIbNEc0zWj\nVphKAPoOv5i69RuwbM4cstKW0r57ZwZftZTouKqzraxK8Vp8Ii/l5TK2sACrUlwSHsGo8OqjJ1rY\n7byXWJc38vOYUFJCQ5uNuZGRtLVXbbLo4nQwPU3KoYb47fbzFggxIqWGICaYJxFNZ3hoGO9HRpFS\nieml45r8Q/Zuk9pL3/N70y0hgnkFm3i36G6cCq4M0wwPrd6M0Npu552EOjx2MIfFHjfJVhuzo2Po\n4KjerHZrVDQtHQ4W5eeT6/PROzSUB8MjCK9CSweo67Pxy2Y3A7qUfefxwuY94hBViA/oTeAhIF8p\nboqK5pKwcNN2fZTUGsEN0LpTN1p3qmgambp+YLlCUv7Mys9jZn4eI4AiFE/nHiTWYmF4NRoGQEOb\njUkxRx6k380ZQp0CC/fP8HHvFZIY8fH38M4XEOqVqeRQxJ64D7g8PIL7o6uPkw60d5vULkpjtEOU\n4vqICK6vwf90ns/H3dlZlHg9DNKa77WbMdmZzEuoQ9JhopzOCQnlnJAjty+Pj4zlou/SadYAhvaU\npLJpb4HNBw+imI6mCRAOpCrF/IREWlei5JjKyJFTqwR3TfnD7ebVvDzWAykAaDYAvXKy6eUMOeIw\nviNlWUwSV21N56KJHtweqBOimBIeQ6LFyuTcHEZ4PNRRimsiIrm5iogWf8Zuncy6+uZAr41UVt2v\nJjyTe5BWHjezMRyUWvOA18tDOVm8FJ94mKNrxqkOB89GxDHp3WyeeEtjs0Bru41Po+OZV1BAz8IC\nMrWms83GjOjYSoV2KaYycmT8K6oDVvUQPJ17EGd+Hk8EfH+pUnSOiuGy8BOznqLP58MH2AKmnz6t\na5xCnjrjU1IXjDiOvTv5qe3VAY9VaAN0Sd3Laq1p4fddLrJy0sbklCojS44Vj8+HBbAcw9j+t2rd\nNRnXtcI5WR3VPQReo/Z2IDbAW+naNscHi8VSQWgDR1X3I/mm845Hl0xOEo6H0AZZhSZwbNsoW0Hm\nRAGXiroAAApvSURBVGGzWCoIbajZ2E6+fMnx7FKtpNYL7uoYEBrGbKXI9PtuG/Ch1vSrgY3vn8b0\nyNcOpq4feNza6h8SylTKC+nngTMdjnLZlicjpjJyeGq14D6c9nKqw8GFYeG0R3EvcDvQBbg/Kuaw\nDpyTjSGj+x9+J5OTFv8FfY8Hd0dF87nVyllK8TAwRCleslh4MObE1LI53phad/XUWsF9pFPO/0XH\nMDMxEW9EJKGRUbxbJ4krI4LPxmYmMQQvEbELjqvQBonJXlYniUtiYkkPj+Cs6BhW1EmiURVZvScb\nptZdPcHxK9aQiNgFNdq/rd1RbQx2sGCGUwUnp/d8/oS061CKYaFhDAutOuvxZKayVHgToVZq3Cfq\nQQgGlu+565/ugkkNOF7OyNqIOYusmlonuP/tD4JZyyR4qOnM8N+I6XivnFoluI+nVz6YMQd7cPBv\nnhmaHBu1RnCfCAdPMGN65U9u/u0zw5pgKiIVqTWC29ReymN65U9eTBOJybFSKwS3qb1UjhnbfXJi\nKhk1x5xBlifoBbepvVTNvpHvmMLbpFZgziDLE/SC29ReqscMqTq5MGeHR4+pdZcR1ILbfAiODNO5\nY1IbMLXuMoJWcJsmkpphJub885iKhsnxIigF99T1A00TSQ0Zu3WyKbxNgh7TXCIEpeA247WPDjOr\n0iTYMc0lQtAJbnO6eWyYGouJSfATVILbFNrHTvJN55nC+x/AHLvHDzPENcgEt8nxwZxumgQzZohr\nEAluU2M5vphai4lJ8BIUgtsU2scfU2sxMQleTnrBbcZrnzjMxBwTk+BEaa0Pv5eJiYmJyUnDSa9x\nm5iYmJiUxxTcJiYmJkGGKbhNTExMggxTcJuYmJgEGabgNjExMQkyTMFtYmJiEmSYgvsfQik1Syn1\nyBHuu1opdcNRnucfOfYw7c5QSk043u0GA0qpxkoprZSy/U3n26GUOucY26hyHCilGiql8pVS1mM5\nh0nNMAX3UWI8fM0CvpuklJr7T/UpWNBa36S1fvhI9q3JH5zJ34/WepfWOkJr7YUT92dvUh5TcJuY\nAH+XBnwsBEMfTf4eTMF9glBK9VVK7VFK3a2USldKpSqlrq1i31il1DKlVIZSKtt4Xz9gt1OUUt8r\npXKVUu8rpeL8ju+mlFqrlMpRSv2qlOpbTb+uU0r9YZxnuVKqkd+2c5VSm5RSB5VSLwCqmnYmKaUW\nK6UWKqXylFLrlFKn+m1vbWhfOUqpjUqpYX7bDmnR1d0npdRoYCRwrzEd/8D4/j6l1F7jvJuVUv2q\n6ONgpdTPxj3brZSa5Let1GRxvVJqF7DqKO5lA6XUEuN3yzTuGUopi1JqvFJqp3FNbyqloqtoo55S\naqlSKksp9ZdS6sZK7vFcpVQucI3R9v1Kqa3GORcFjIWrjPNmKqUeqKbvTYxrtBifX1VKpfttn6OU\nutPvkEZKqa+Ne/6pUioh4D7alFKPAr2BF4zfq/R+tFJKrTCucbNS6pKq+mVyhGitzddRvAANNAv4\nbhIw13jfF/AAkwE7MAgoBGKN7bOAR4z38cCFQBgQCbwNvOfX7mpgL9AOCAfe8TtPCpBptG8BzjU+\nJ/ode4PxfjjwF9AasAHjgbXGtgQgD7jI6O8Yo/83VHH9kwC33/73ANuN93bjPOMAB3C20XbLSq79\niO+T8bklsBuoZ3xuDJxSRR/7Au2N+9IB2A+c73ecBt407mno4e5lQNtW4FfgGeP4EKCXse064/qb\nAhHAEmBOwHltxucvgBeN408DMoCzA+7x+UZ/QoE7gG+B+oATeBlYYOzfBsgH+hjbphn39pwq7s8u\noJPxfjOwDWjtt+10vzG0FWhh9GE1MKWK61mN35gx7s1u4FpkzJ0OHADa/NPPcDC//vEOBOuLIxPc\nRaUD2vguHehmvJ+Fn0AKaOc0INvv86EHxfjcBigxhMd9pULBb/ty4Gq/Y0sF98fA9X77WRAh2QgY\nBXzrt00Be6hecH8b0FYqonH1BtIAi9/2BcCkwGuv6X0CmhnbzwHsNfzNngWeMd6XCpymfturvZcB\n33dHhKytkm0rgVv8PrdEBLDNX9ABDQAvEOm37+PALL97/EVA238A/fw+J/u1/SDwlt+2cGOcVCW4\n5wB3AUmI4H4SuAloAuSU/n7GGBrvd9wtwCcB97EqwX0p8GXAeV8GJv7Tz3Awv0xTydHjRTREf+zI\nQ1RKptba4/e5ENHAyqGUClNKvWxMcXMRLSxGlffU7/Z7v9M4VwIidC82pr05SqkcoBfyQAfSCHjO\nb78sRECnAPX8z6HlCdtdSRv++O/vQwR9vdK2jO/8+5xSRTtHdJ+M8/wF3IkItXSl1FtKqXqV7auU\n6qqU+twwZRxEhFJCVddAze5lA2BnQL9LqYdcbyk7EcFat5L9srTWeQH7+t+nwN+gEfCuX//+QMZi\nXSr+hgXIjKEq1iB/nH2QMbcaONN4fRnw+6X5va/y96mERkDXgHs6EvmzMDlKTMF99OxCtA1/mlD+\ngT1S7ka0sq5a6yjkQYLyNuYGfu8bIn8QB5AHdY7WOsbvFa61nlLJeXYD/wnYN1RrvRbRlg+dQyml\nAs5ZGf77W5Dp+z7j1aDUfurX572Haa8yKpSv1FrP11r3QoSCBp6o4tj5wFKggdY6GphBRbu9f/s1\nvZcNVeUOw31G30ppiJgs9leyX5xSKjJgX//7FHj9u4GBAX0M0VrvpeJvGIaY4apiDTI76mu8/wro\niQjuNdUcVx2V9XdNQH8jtNY3H2X7JpiC+1hYCIxXStU3HEbnAEOBxUfRViRiLsgxHE0TK9nnSqVU\nG+NhnAws1hKCNRcYqpTqr5SyKqVCDIdfoHMTRHCNVUq1BVBKRSulLja2fQi0VUqNMITR7RxeK+rk\nt/+dgAuxv36HaGX3KqXshoNvKPDWkd4QP/YjtmKMPrdUSp2tlHICxch981VxbCSi0RYrpboAVxzm\nXDW5l98jgnKKUirc2LensW0BMMZwAEYAjwELA7VzrfVuYC3wuHF8B+B6ox9VMQN4VBlOZaVUolJq\nuLFtMTBEKdVLKeVAxkmVz7jW+k/k/l2JCNdc5H5fyNEL7nK/F7AMaGE4Te3Gq7NSqvVRtm+CKbiP\nhcnIQ/cVkI3YB0dqrTccRVvPIk6fA4jg+6SSfeYg9t40xJF1Oxx6+IcjjsAMRMP5H5X8tlrrdxHt\n9C3DJLMBGGhsOwBcDExBptfNga8P0+/3ERtmNnAVMEJr7dZalyCCeqBxTS8Co7TWmw57JyryOtDG\nmGa/hzjdphjtpgF1gLFVHHsLMFkplYfYfxdVd6Ia3ksvco3NkNnXHuReALyB/F5fIA7bYuC2Kk57\nOTJz2we8i9h+P6umm88hs4hPjev6Fuhq9GkjcCsy00hFfpc91V0zIqAzjWsv/ayAdYc5rrr+XaQk\naul5wwx0HnAZco1pyBh0HmX7JpgLKZgcJUpC65ppra/8p/tiYvJvw9S4TUxMTIIMU3CbmJiYBBmm\nqcTExMQkyDA1bhMTE5MgwxTcJiYmJkGGKbhNTExMggxTcJuYmJgEGabgNjExMQkyTMFtYmJiEmT8\nPw5yCcq5EUo2AAAAAElFTkSuQmCC\n", 29 | "text/plain": [ 30 | "" 31 | ] 32 | }, 33 | "metadata": {}, 34 | "output_type": "display_data" 35 | } 36 | ], 37 | "source": [ 38 | "import numpy as np\n", 39 | "import matplotlib.pyplot as plt\n", 40 | "from sklearn import datasets\n", 41 | "from sklearn import svm\n", 42 | "from sklearn.semi_supervised import label_propagation\n", 43 | "\n", 44 | "rng = np.random.RandomState(0)\n", 45 | "\n", 46 | "iris = datasets.load_iris()\n", 47 | "\n", 48 | "X = iris.data[:, :2]\n", 49 | "y = iris.target\n", 50 | "\n", 51 | "# step size in the mesh\n", 52 | "h = .02\n", 53 | "\n", 54 | "y_30 = np.copy(y)\n", 55 | "y_30[rng.rand(len(y)) < 0.3] = -1\n", 56 | "y_50 = np.copy(y)\n", 57 | "y_50[rng.rand(len(y)) < 0.5] = -1\n", 58 | "# we create an instance of SVM and fit out data. We do not scale our\n", 59 | "# data since we want to plot the support vectors\n", 60 | "ls30 = (label_propagation.LabelSpreading().fit(X, y_30),\n", 61 | " y_30)\n", 62 | "ls50 = (label_propagation.LabelSpreading().fit(X, y_50),\n", 63 | " y_50)\n", 64 | "ls100 = (label_propagation.LabelSpreading().fit(X, y), y)\n", 65 | "rbf_svc = (svm.SVC(kernel='rbf', gamma=.5).fit(X, y), y)\n", 66 | "\n", 67 | "# create a mesh to plot in\n", 68 | "x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1\n", 69 | "y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1\n", 70 | "xx, yy = np.meshgrid(np.arange(x_min, x_max, h),\n", 71 | " np.arange(y_min, y_max, h))\n", 72 | "\n", 73 | "# title for the plots\n", 74 | "titles = ['Label Spreading 30% data',\n", 75 | " 'Label Spreading 50% data',\n", 76 | " 'Label Spreading 100% data',\n", 77 | " 'SVC with rbf kernel']\n", 78 | "\n", 79 | "color_map = {-1: (1, 1, 1), 0: (0, 0, .9), 1: (1, 0, 0), 2: (.8, .6, 0)}\n", 80 | "\n", 81 | "for i, (clf, y_train) in enumerate((ls30, ls50, ls100, rbf_svc)):\n", 82 | " # Plot the decision boundary. For that, we will assign a color to each\n", 83 | " # point in the mesh [x_min, x_max]x[y_min, y_max].\n", 84 | " plt.subplot(2, 2, i + 1)\n", 85 | " Z = clf.predict(np.c_[xx.ravel(), yy.ravel()])\n", 86 | "\n", 87 | " # Put the result into a color plot\n", 88 | " Z = Z.reshape(xx.shape)\n", 89 | " plt.contourf(xx, yy, Z, cmap=plt.cm.Paired)\n", 90 | " plt.axis('off')\n", 91 | "\n", 92 | " # Plot also the training points\n", 93 | " colors = [color_map[y] for y in y_train]\n", 94 | " plt.scatter(X[:, 0], X[:, 1], c=colors, edgecolors='black')\n", 95 | "\n", 96 | " plt.title(titles[i])\n", 97 | "\n", 98 | "plt.suptitle(\"Unlabeled points are colored white\", y=0.1)\n", 99 | "plt.show()" 100 | ] 101 | }, 102 | { 103 | "cell_type": "markdown", 104 | "metadata": {}, 105 | "source": [ 106 | "# Useful resources" 107 | ] 108 | }, 109 | { 110 | "cell_type": "markdown", 111 | "metadata": {}, 112 | "source": [ 113 | "## semi-supervise learning in deep learning\n", 114 | "### Tutorial\n", 115 | " - [EMLo](https://allennlp.org/elmo)\n", 116 | " - [BERT](https://github.com/google-research/bert)\n", 117 | " - [Transformer](https://github.com/harvardnlp/annotated-transformer)\n", 118 | " - [Semi-supervise in NLP](http://web.stanford.edu/class/cs224n/lectures/lecture17.pdf)" 119 | ] 120 | } 121 | ], 122 | "metadata": { 123 | "kernelspec": { 124 | "display_name": "Python 3", 125 | "language": "python", 126 | "name": "python3" 127 | }, 128 | "language_info": { 129 | "codemirror_mode": { 130 | "name": "ipython", 131 | "version": 3 132 | }, 133 | "file_extension": ".py", 134 | "mimetype": "text/x-python", 135 | "name": "python", 136 | "nbconvert_exporter": "python", 137 | "pygments_lexer": "ipython3", 138 | "version": "3.5.4" 139 | } 140 | }, 141 | "nbformat": 4, 142 | "nbformat_minor": 2 143 | } 144 | --------------------------------------------------------------------------------