├── .gitignore ├── CODE_OF_CONDUCT.md ├── CONTRIBUTING.md ├── LICENSE ├── NOTICE ├── README.md └── notebooks ├── part1 └── Intro_to_gluonts.ipynb ├── part2 └── descriptive_stats.ipynb ├── part3 └── twitter_volume_forecast.ipynb └── part4 ├── deepar_details.ipynb ├── entry_point └── train.py ├── images ├── prediction.png └── training.png └── twitter_volume_sagemaker.ipynb /.gitignore: -------------------------------------------------------------------------------- 1 | # Jupyter Notebook 2 | .ipynb_checkpoints 3 | 4 | .DS_Store 5 | 6 | __pycache__/ 7 | -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | ## Code of Conduct 2 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 3 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 4 | opensource-codeofconduct@amazon.com with any additional questions or comments. 5 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing Guidelines 2 | 3 | Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional 4 | documentation, we greatly value feedback and contributions from our community. 5 | 6 | Please read through this document before submitting any issues or pull requests to ensure we have all the necessary 7 | information to effectively respond to your bug report or contribution. 8 | 9 | 10 | ## Reporting Bugs/Feature Requests 11 | 12 | We welcome you to use the GitHub issue tracker to report bugs or suggest features. 13 | 14 | When filing an issue, please check existing open, or recently closed, issues to make sure somebody else hasn't already 15 | reported the issue. Please try to include as much information as you can. Details like these are incredibly useful: 16 | 17 | * A reproducible test case or series of steps 18 | * The version of our code being used 19 | * Any modifications you've made relevant to the bug 20 | * Anything unusual about your environment or deployment 21 | 22 | 23 | ## Contributing via Pull Requests 24 | Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that: 25 | 26 | 1. You are working against the latest source on the *master* branch. 27 | 2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already. 28 | 3. You open an issue to discuss any significant work - we would hate for your time to be wasted. 29 | 30 | To send us a pull request, please: 31 | 32 | 1. Fork the repository. 33 | 2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change. 34 | 3. Ensure local tests pass. 35 | 4. Commit to your fork using clear commit messages. 36 | 5. Send us a pull request, answering any default questions in the pull request interface. 37 | 6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation. 38 | 39 | GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and 40 | [creating a pull request](https://help.github.com/articles/creating-a-pull-request/). 41 | 42 | 43 | ## Finding contributions to work on 44 | Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any 'help wanted' issues is a great place to start. 45 | 46 | 47 | ## Code of Conduct 48 | This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 49 | For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 50 | opensource-codeofconduct@amazon.com with any additional questions or comments. 51 | 52 | 53 | ## Security issue notifications 54 | If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue. 55 | 56 | 57 | ## Licensing 58 | 59 | See the [LICENSE](LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution. 60 | 61 | We may ask you to sign a [Contributor License Agreement (CLA)](http://en.wikipedia.org/wiki/Contributor_License_Agreement) for larger changes. 62 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | 2 | Apache License 3 | Version 2.0, January 2004 4 | http://www.apache.org/licenses/ 5 | 6 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 7 | 8 | 1. Definitions. 9 | 10 | "License" shall mean the terms and conditions for use, reproduction, 11 | and distribution as defined by Sections 1 through 9 of this document. 12 | 13 | "Licensor" shall mean the copyright owner or entity authorized by 14 | the copyright owner that is granting the License. 15 | 16 | "Legal Entity" shall mean the union of the acting entity and all 17 | other entities that control, are controlled by, or are under common 18 | control with that entity. For the purposes of this definition, 19 | "control" means (i) the power, direct or indirect, to cause the 20 | direction or management of such entity, whether by contract or 21 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 22 | outstanding shares, or (iii) beneficial ownership of such entity. 23 | 24 | "You" (or "Your") shall mean an individual or Legal Entity 25 | exercising permissions granted by this License. 26 | 27 | "Source" form shall mean the preferred form for making modifications, 28 | including but not limited to software source code, documentation 29 | source, and configuration files. 30 | 31 | "Object" form shall mean any form resulting from mechanical 32 | transformation or translation of a Source form, including but 33 | not limited to compiled object code, generated documentation, 34 | and conversions to other media types. 35 | 36 | "Work" shall mean the work of authorship, whether in Source or 37 | Object form, made available under the License, as indicated by a 38 | copyright notice that is included in or attached to the work 39 | (an example is provided in the Appendix below). 40 | 41 | "Derivative Works" shall mean any work, whether in Source or Object 42 | form, that is based on (or derived from) the Work and for which the 43 | editorial revisions, annotations, elaborations, or other modifications 44 | represent, as a whole, an original work of authorship. For the purposes 45 | of this License, Derivative Works shall not include works that remain 46 | separable from, or merely link (or bind by name) to the interfaces of, 47 | the Work and Derivative Works thereof. 48 | 49 | "Contribution" shall mean any work of authorship, including 50 | the original version of the Work and any modifications or additions 51 | to that Work or Derivative Works thereof, that is intentionally 52 | submitted to Licensor for inclusion in the Work by the copyright owner 53 | or by an individual or Legal Entity authorized to submit on behalf of 54 | the copyright owner. For the purposes of this definition, "submitted" 55 | means any form of electronic, verbal, or written communication sent 56 | to the Licensor or its representatives, including but not limited to 57 | communication on electronic mailing lists, source code control systems, 58 | and issue tracking systems that are managed by, or on behalf of, the 59 | Licensor for the purpose of discussing and improving the Work, but 60 | excluding communication that is conspicuously marked or otherwise 61 | designated in writing by the copyright owner as "Not a Contribution." 62 | 63 | "Contributor" shall mean Licensor and any individual or Legal Entity 64 | on behalf of whom a Contribution has been received by Licensor and 65 | subsequently incorporated within the Work. 66 | 67 | 2. Grant of Copyright License. Subject to the terms and conditions of 68 | this License, each Contributor hereby grants to You a perpetual, 69 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 70 | copyright license to reproduce, prepare Derivative Works of, 71 | publicly display, publicly perform, sublicense, and distribute the 72 | Work and such Derivative Works in Source or Object form. 73 | 74 | 3. Grant of Patent License. Subject to the terms and conditions of 75 | this License, each Contributor hereby grants to You a perpetual, 76 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 77 | (except as stated in this section) patent license to make, have made, 78 | use, offer to sell, sell, import, and otherwise transfer the Work, 79 | where such license applies only to those patent claims licensable 80 | by such Contributor that are necessarily infringed by their 81 | Contribution(s) alone or by combination of their Contribution(s) 82 | with the Work to which such Contribution(s) was submitted. If You 83 | institute patent litigation against any entity (including a 84 | cross-claim or counterclaim in a lawsuit) alleging that the Work 85 | or a Contribution incorporated within the Work constitutes direct 86 | or contributory patent infringement, then any patent licenses 87 | granted to You under this License for that Work shall terminate 88 | as of the date such litigation is filed. 89 | 90 | 4. Redistribution. You may reproduce and distribute copies of the 91 | Work or Derivative Works thereof in any medium, with or without 92 | modifications, and in Source or Object form, provided that You 93 | meet the following conditions: 94 | 95 | (a) You must give any other recipients of the Work or 96 | Derivative Works a copy of this License; and 97 | 98 | (b) You must cause any modified files to carry prominent notices 99 | stating that You changed the files; and 100 | 101 | (c) You must retain, in the Source form of any Derivative Works 102 | that You distribute, all copyright, patent, trademark, and 103 | attribution notices from the Source form of the Work, 104 | excluding those notices that do not pertain to any part of 105 | the Derivative Works; and 106 | 107 | (d) If the Work includes a "NOTICE" text file as part of its 108 | distribution, then any Derivative Works that You distribute must 109 | include a readable copy of the attribution notices contained 110 | within such NOTICE file, excluding those notices that do not 111 | pertain to any part of the Derivative Works, in at least one 112 | of the following places: within a NOTICE text file distributed 113 | as part of the Derivative Works; within the Source form or 114 | documentation, if provided along with the Derivative Works; or, 115 | within a display generated by the Derivative Works, if and 116 | wherever such third-party notices normally appear. The contents 117 | of the NOTICE file are for informational purposes only and 118 | do not modify the License. You may add Your own attribution 119 | notices within Derivative Works that You distribute, alongside 120 | or as an addendum to the NOTICE text from the Work, provided 121 | that such additional attribution notices cannot be construed 122 | as modifying the License. 123 | 124 | You may add Your own copyright statement to Your modifications and 125 | may provide additional or different license terms and conditions 126 | for use, reproduction, or distribution of Your modifications, or 127 | for any such Derivative Works as a whole, provided Your use, 128 | reproduction, and distribution of the Work otherwise complies with 129 | the conditions stated in this License. 130 | 131 | 5. Submission of Contributions. Unless You explicitly state otherwise, 132 | any Contribution intentionally submitted for inclusion in the Work 133 | by You to the Licensor shall be under the terms and conditions of 134 | this License, without any additional terms or conditions. 135 | Notwithstanding the above, nothing herein shall supersede or modify 136 | the terms of any separate license agreement you may have executed 137 | with Licensor regarding such Contributions. 138 | 139 | 6. Trademarks. This License does not grant permission to use the trade 140 | names, trademarks, service marks, or product names of the Licensor, 141 | except as required for reasonable and customary use in describing the 142 | origin of the Work and reproducing the content of the NOTICE file. 143 | 144 | 7. Disclaimer of Warranty. Unless required by applicable law or 145 | agreed to in writing, Licensor provides the Work (and each 146 | Contributor provides its Contributions) on an "AS IS" BASIS, 147 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 148 | implied, including, without limitation, any warranties or conditions 149 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 150 | PARTICULAR PURPOSE. You are solely responsible for determining the 151 | appropriateness of using or redistributing the Work and assume any 152 | risks associated with Your exercise of permissions under this License. 153 | 154 | 8. Limitation of Liability. In no event and under no legal theory, 155 | whether in tort (including negligence), contract, or otherwise, 156 | unless required by applicable law (such as deliberate and grossly 157 | negligent acts) or agreed to in writing, shall any Contributor be 158 | liable to You for damages, including any direct, indirect, special, 159 | incidental, or consequential damages of any character arising as a 160 | result of this License or out of the use or inability to use the 161 | Work (including but not limited to damages for loss of goodwill, 162 | work stoppage, computer failure or malfunction, or any and all 163 | other commercial damages or losses), even if such Contributor 164 | has been advised of the possibility of such damages. 165 | 166 | 9. Accepting Warranty or Additional Liability. While redistributing 167 | the Work or Derivative Works thereof, You may choose to offer, 168 | and charge a fee for, acceptance of support, warranty, indemnity, 169 | or other liability obligations and/or rights consistent with this 170 | License. However, in accepting such obligations, You may act only 171 | on Your own behalf and on Your sole responsibility, not on behalf 172 | of any other Contributor, and only if You agree to indemnify, 173 | defend, and hold each Contributor harmless for any liability 174 | incurred by, or claims asserted against, such Contributor by reason 175 | of your accepting any such warranty or additional liability. 176 | -------------------------------------------------------------------------------- /NOTICE: -------------------------------------------------------------------------------- 1 | Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved. 2 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | This repository contains the material for the workshop ["AIM347 - Time-series prediction using GluonTS and Amazon SageMaker"](https://www.portal.reinvent.awsevents.com/connect/sessionDetail.ww?SESSION_ID=98535). 2 | 3 | [Part 1](notebooks/part1/Intro_to_gluonts.ipynb) gives a brief overview of basic GluonTS components. 4 | 5 | ### Lab 1 6 | Tutorial [part 2](notebooks/part2/descriptive_stats.ipynb) is optional. It gives a high level overview of the time-series modeling pipeline and you will get more familiar with the dataset. 7 | 8 | In [part 3](notebooks/part3/twitter_volume_forecast.ipynb), you will learn how to run a time-series forecast using GluonTS. You will create a baseline with a seasonal naive predictor and train a DeepAR model. 9 | 10 | ### Lab 2 11 | In [part 4](notebooks/part4/twitter_volume_sagemaker.ipynb) you will use Amazon SageMaker to train and tune your DeepAR model. You will use SageMaker automatic model tuner to find better hyperpameters and then deploy the tuned model as an endpoint. 12 | -------------------------------------------------------------------------------- /notebooks/part1/Intro_to_gluonts.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": { 6 | "slideshow": { 7 | "slide_type": "slide" 8 | } 9 | }, 10 | "source": [ 11 | "# An overview of GluonTS" 12 | ] 13 | }, 14 | { 15 | "cell_type": "markdown", 16 | "metadata": { 17 | "slideshow": { 18 | "slide_type": "slide" 19 | } 20 | }, 21 | "source": [ 22 | "### Opensource python toolkit based on Apache MXNet and Gluon\n", 23 | "\n", 24 | "- Implementations of state of the art models\n", 25 | "- Baseline models\n", 26 | "- Tools for model evaluation and comparison\n", 27 | "- Dataloaders and iterators" 28 | ] 29 | }, 30 | { 31 | "cell_type": "markdown", 32 | "metadata": { 33 | "slideshow": { 34 | "slide_type": "slide" 35 | } 36 | }, 37 | "source": [ 38 | "### Models:\n", 39 | "- DeepAR\n", 40 | "- Deep factor\n", 41 | "- DeepState\n", 42 | "- Wavenet\n", 43 | "- Transformer models \n", 44 | "- Seq-2-seq models\n", 45 | "- etc." 46 | ] 47 | }, 48 | { 49 | "cell_type": "markdown", 50 | "metadata": { 51 | "slideshow": { 52 | "slide_type": "slide" 53 | } 54 | }, 55 | "source": [ 56 | "## Components of GluonTS" 57 | ] 58 | }, 59 | { 60 | "cell_type": "markdown", 61 | "metadata": { 62 | "slideshow": { 63 | "slide_type": "notes" 64 | } 65 | }, 66 | "source": [ 67 | "### Dataset\n", 68 | "- **Dataset** abstraction for providing uniform access to data across different input formats\n", 69 | "- In GluonTS, any Dataset is just an Iterable over dictionaries mapping string keys to arbitrary values\n", 70 | "- There are two variations of dataset: **FileDataSet** and **ListDataset**\n", 71 | "- In order to implement your own dataset:\n", 72 | " - you will need to derive a class from `gluonts.dataset.common.Dataset`, and\n", 73 | " - implement `__iter__`, and `__len__`\n", 74 | " - more information can be found [here](https://github.com/awslabs/gluon-ts/blob/master/src/gluonts/dataset/common.py).\n" 75 | ] 76 | }, 77 | { 78 | "cell_type": "markdown", 79 | "metadata": { 80 | "slideshow": { 81 | "slide_type": "notes" 82 | } 83 | }, 84 | "source": [ 85 | "- A dataset object requires start and target (pandas timestamp)\n", 86 | "- Please note that your data does not require to have a timestamp field but should be aggregated as ***fixed-length intervals***.\n", 87 | "- You shall need to pass a frequency parameter, which is compatible with pandas frequency.\n", 88 | "- **ListDataset** to access data stored in memory as a list of dictionaries." 89 | ] 90 | }, 91 | { 92 | "cell_type": "markdown", 93 | "metadata": { 94 | "slideshow": { 95 | "slide_type": "slide" 96 | } 97 | }, 98 | "source": [ 99 | "### Dataset" 100 | ] 101 | }, 102 | { 103 | "cell_type": "markdown", 104 | "metadata": { 105 | "slideshow": { 106 | "slide_type": "fragment" 107 | } 108 | }, 109 | "source": [ 110 | "```python\n", 111 | "from gluonts.dataset.common import ListDataset\n", 112 | "\n", 113 | "training_data = ListDataset(\n", 114 | " [{\"start\": df.index[0], \"target\": df.value[:\"2015-04-05 00:00:00\"]}],\n", 115 | " freq = \"5min\"\n", 116 | ")\n", 117 | "```" 118 | ] 119 | }, 120 | { 121 | "cell_type": "markdown", 122 | "metadata": { 123 | "slideshow": { 124 | "slide_type": "slide" 125 | } 126 | }, 127 | "source": [ 128 | "### The Estimator and the Predictor\n", 129 | "- An **Estimator** represents a model that can be trained on a dataset to yield\n", 130 | "- a **Predictor**, which can later be used to make predictions on unseen data.\n" 131 | ] 132 | }, 133 | { 134 | "cell_type": "markdown", 135 | "metadata": { 136 | "slideshow": { 137 | "slide_type": "fragment" 138 | } 139 | }, 140 | "source": [ 141 | "```python\n", 142 | "from gluonts.model.deepar import DeepAREstimator\n", 143 | "from gluonts.trainer import Trainer\n", 144 | "\n", 145 | "estimator = DeepAREstimator(freq=\"5min\", \n", 146 | " prediction_length=36, \n", 147 | " trainer=Trainer(epochs=10))\n", 148 | "```" 149 | ] 150 | }, 151 | { 152 | "cell_type": "markdown", 153 | "metadata": { 154 | "slideshow": { 155 | "slide_type": "notes" 156 | } 157 | }, 158 | "source": [ 159 | "```python\n", 160 | "from gluonts.model.deepar import DeepAREstimator \n", 161 | "```\n", 162 | "- `DeepArEstimator` is an abstraction for models. \n", 163 | "- Other supported estimators include:\n", 164 | " - Seq2SeqEstimator\n", 165 | " - CanonicalRNNEstimator\n", 166 | " - SimpleFeedForwardEstimator\n", 167 | " - WaveNetEstimator" 168 | ] 169 | }, 170 | { 171 | "cell_type": "markdown", 172 | "metadata": { 173 | "slideshow": { 174 | "slide_type": "slide" 175 | } 176 | }, 177 | "source": [ 178 | "### Predictor\n", 179 | "- you can call your estimator's **train** methods to kick-off training.\n", 180 | "- train will return a **predictor** object after completes the training.\n", 181 | "- You can then use the predictor object in order to make inference." 182 | ] 183 | }, 184 | { 185 | "cell_type": "markdown", 186 | "metadata": { 187 | "slideshow": { 188 | "slide_type": "fragment" 189 | } 190 | }, 191 | "source": [ 192 | "```python\n", 193 | "predictor = estimator.train(training_data)\n", 194 | "```" 195 | ] 196 | }, 197 | { 198 | "cell_type": "markdown", 199 | "metadata": { 200 | "slideshow": { 201 | "slide_type": "slide" 202 | } 203 | }, 204 | "source": [ 205 | "### Evaluation\n", 206 | "- You can use the predictor in order to create a forecast:\n" 207 | ] 208 | }, 209 | { 210 | "cell_type": "markdown", 211 | "metadata": { 212 | "slideshow": { 213 | "slide_type": "fragment" 214 | } 215 | }, 216 | "source": [ 217 | "```python\n", 218 | "from gluonts.evaluation.backtest import make_evaluation_predictions\n", 219 | "\n", 220 | "forecast_it, ts_it = make_evaluation_predictions(\n", 221 | " test_data, \n", 222 | " predictor, \n", 223 | " num_samples=100)\n", 224 | "forecasts = list(forecast_it)\n", 225 | "tss = list(ts_it)\n", 226 | "```" 227 | ] 228 | }, 229 | { 230 | "cell_type": "markdown", 231 | "metadata": { 232 | "slideshow": { 233 | "slide_type": "notes" 234 | } 235 | }, 236 | "source": [ 237 | "```python\n", 238 | "def plot_forecasts(tss, forecasts, past_length):\n", 239 | " for target, forecast in islice(zip(tss, forecasts)):\n", 240 | " ax = target[-past_length:].plot(figsize=(12, 5), linewidth=2)\n", 241 | " forecast.plot(color='g')\n", 242 | " plt.grid(which='both')\n", 243 | " plt.legend([\"observations\", \"median prediction\", \"90% confidence interval\", \"50% confidence interval\"])\n", 244 | " plt.show()\n", 245 | "```" 246 | ] 247 | }, 248 | { 249 | "cell_type": "markdown", 250 | "metadata": { 251 | "slideshow": { 252 | "slide_type": "slide" 253 | } 254 | }, 255 | "source": [ 256 | "### Plotting the forecast\n", 257 | "- Plotting the forecast gives you a qualitative feel for the model output." 258 | ] 259 | }, 260 | { 261 | "attachments": { 262 | "image.png": { 263 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAArwAAAE2CAYAAACDeUFgAAAgAElEQVR4Aey9B5wcxZU//iaH3dUGrVY5AxIIIRAIIYIxwRhOPjJ/GziSDcYGLLAxsgm2MYYzxuc74Ig2wYAx3IdoDg7wT+QgQCCCEAiBclhptWzenTzz/3yrp2Z7Zntiz0z3zLz6aNXd1VWvXn1rwpvX33rPEovFYsSFEWAEGAFGgBFgBBgBRoARqFIErFU6L54WI8AIMAKMACPACDACjAAjIBBgg5dfCIwAI8AIMAKMACPACDACVY0AG7xVvbw8OUaAEWAEGAFGgBFgBBgBNnj5NcAIMAKMACPACDACjAAjUNUI2I2e3dDQEPUO9CWpEQ1FxLXVYSP1OSrlNc5xX12nbi/vo31qO9Ep3l/K02qTWlfN8tU4pJ7nOu9c2zH+CgISL3lUv2YlRuq61HayTep6yXrZHkcU2S71Pq612qTWSXnq/vJcyla3ST1HW9lOa0x1e3kf7VGf2ld9X56ntkntl0m+1Cu1jVq2HvmpfaVc9bipbfLRP7VvpciHnigSh/il5pqr10Zio+6rvo96eS3bao0h67TapNZJeWod0T+1Xep9qQuOcjytOiPkS13VemnpJutkOzln1Kvr5LlsjyOKuh7Xsr+sT71O10a2V9+vFfnquavnL+u1Xj9ohyLbxC+rHv/mMePI61S+N+SccTTc4L3tttvo6quvpm9/+9sUCoUSug0ODorzurq6RB2flBeBrq4uamlpKe+gPFoSAliDsWPHJr03khrwRUkRAP4ul4uampp4DUqKdHrh/DmUHpty3JHfxfweKAfa2mPwe0AbF63aTz75hN5//33yTpw44rbhBi+M3F/84hd0/fXXJyn3xZrPyeOsoykzpiTV80X5EFi+fDktWrSofAPySEkItHfsoPat22n+/PlJ9XxRPgRWrlxJ06dNo2b+4Vc+0FNGWvHuclqwkD+HUmAp2+W6desoHArSrNl7lm1MHigZAXwOzZ49m7xeb/INvhqBAJyn6YppObzBUJhCsWGPb7oJcH1pEQj6AqUdgKWnRQCPoSIhxj8tQGW4Afz94WAZRuIhtBAA5S0c1brDdeVCAMYuvo+5GIdAIBCggbBpzTXjgMlzZFMgGAyO/EJxOgx3PucJJTdnBIqLADhZNoeruEJZWl4IAP9U/lteArgxI8AIMAJFQKDezr/89MJoCqvS6XSOmAf/ohwBiSEVYXuERq6OIarU3KBsaNXckvOEUxGw8w++VEgq4RpPBsM2O9kjYXGksOpJFdYU1/Gj3WIbbhuLkLwW8wwHxLV6zpCr7i/lyKPob48QxYbHgR4o4VhEGRcbmOJ1sl6OK3SOt0vSP/5aFHX2CNnDtoQ8KUvKV19Drrok6R/XA9+zannq+cn+Ab9fiJF64iJ1nATmeegv5Yl5lQD/TPL16u/05Pf5YAqDV8vDKxbTwaaW+o1iyDk+NLgYhgAeZXFhBGoWAbWhVLMgVNbEsfegr6s7obTd4RQcYFTI82zHROcMfbTkpdbhGpSM1PGkfK16WZcqSy0nXX9131zapGuPvhgPBW1ampuoo2NnYh7pZMv26r5qvVPHk9epRylfLU+rTbY69M9nfPV46nOtcXC/rW1sXvsrTGHwanl4MRkuxiOg/AJL/oVqvFa1owEiBHBhBBgBRsBIBHKlGMLYDQ0EafrUGZSv983I+Zl9bHDZecNa8ioBky2bN5GrvlEzBFlya+XKtBxeLWW5jhFgBBgBRqB2EJCPc2tnxuacaa4UQxi7jnonG7vmXMaq0kr+AJC0iFwmZwqDlz28uSwVt6lFBJjSYOyqI0qDDIpvrCa1ObrgRNbm1Ct21m47UxFLsXhDQSUhVylkV6pMQXUAXznHYgqDNx2HV/JQcpwLN2MEqg4BpjQYu6QcpcFY/Hn0ykLA5rKZMoxfd1cXIba/kQUGKx7DS8MV59AJx1xKPp7MXORVQ5t8bURTGLzpPLyw3rkYiwB2j3IxDgGOw2sc9hiZPbzG4s+jmwOBXDm8/pidzOjhffmVV+jfzjrbUDC7d+2gv//979Tbs0vo0d7eTm+8+RZ93b41q154fM+c6JEwwUbMh/Zkik1rWh7eXN9gIyHgmmIgwAknioGiPhlmi8P73oYu6veHaNHM1pw3CehDwPje7OE1eA04LJnBC6AMnyuH1xnoJb9zlOE6b9u2jT7//HOqq6ujefPmkdvtFinCkbEMaXr33HNPmjhxIuF7bsOm9fTVuvXkaB1PcyaNFfXwun788ceEtMqTJ00UWeZQh01Sg0M+klSztrY2mjlzppivjE6BjHTw3G7Zuk3Uz9lnbxrfNo7eeecduvvuu8X4Jxz/ryLCwOLjv0ON9QpeyGi3YcMGofP++84XBi7m4ff7RT3GPHjRokRUglWrVtHOnTuTxjAceAMUAO0pV9eoKQxeLQ9vrm8wA/CtiSHlr0nEBiRHTUzZdJM0W6a13/zjU/rn6p209t+Po5lj6k2HVykUYg9vKVDNQyaHJcsDLOObWt11CSXW7RqgTV0+qncmR/kJRaLksFlJHtFBfS4FyDp5lPUHTm+Rp5pHGIm//OUvqaOjQ9z/xje+QUceeSTBoLzjjjtozZo1NH36dLr1llto67Zt9POf/1y0a2hooPPOO4+8Hg/93/PP01NPPSWMzXHjxtEVP7+cPM46WnLpZcJghRG9ZcsWmjx5Mt16660igsJ//cd/Ery2MGqXvfQyvfLKK9Tf308LFiygy3/2MzH+Rx99JGTCEJ82bRrdf//99Jvf/IZaWlro2muvTRi25557Lp13wY/olltuEYY3dNu0aRMtXLiQbrrpJmF4X3nllcKQh0F82WWX0fijx2niwZXDCJjC4NXy8EJFpjQML5RRZwhkneuvJ6N0rOZxzZRpLRxTkPaHaifjD3t4q/ndxXMrJQJLH/+EnnxhDdEod1GH2Xn7ydSWRib4sa+++ip1d3fTiy++SPC6nnvWOTR27Fihwz333COO559/Pj3x5JM0fvx42r59O7377ruJsF/PPfcc3XnnncIARb/f/va3dPU1v6K/P/Sw8PiedtpptHTpUgIv+IqlS4VhfOaZZwpjd+7cuULOGaefTjC04YGFIbto0SK64PzzafPmzfSTSy4WHuPly5eTx+Oh1rFj6FdXXUNTp06lhx56SOh8/OJ/paOPOpJaW1sJxu5/33Gb8BLvvvvuov/7H6wkGOLQDZ7qWi35cnhNYfDW6mJVxLzhYXF6K0JVVrK0CNgtivxAmHndpUWapUsE8uHnyT58NA6BqH+QvJ5WocARs9poMBihMfXFjSXuTvEYJ802HBBUhhNPPFFUgy5w1FFHETyr++yzT6LpQQcdJOpAKZg1axYtWbKEUHfccceJNjCAH3jgAfL5fMJLCyN3y/atNHv2bDrssMNEm+aWFuGZBXUCFAZsMAZVAUY2vL1vv/22MErhWYYXdsjnE8eu7p6EHugDzjO8zieddJKoB/2hp6eHZLsjjjgi0R4nePr9L8cdRy+88ILwDu+333508mmnCIM4qWENXFQkh1eL0lADa1UZU2QOnWHrBA6vWTat9flDBA+v3W4lH4fHMew1UWsDc1iyylpxNaXhkiN3I/yVu8Bru3btWjEs0vHiHN5QeHJl2bhxo/DEwrh8/PHHCdzeX//61/Taa6/RRRddJKgD8Laqy4p3l4tL8HplAbUBXN/rb/h3amxsFJ7b22+/nT777DN68803RbNvf/vbwuOMz3PwcOvrh+lgkgvc1NQkPMRSruQc41o9Hq5DoZDg8UI/GNqX//wKQXeAJ1nGppVyauFYcRzedJSGWlgss8+RM60Zu0JmoTT4gxEKhBTPbiAS5zYYCw2PXgsI8P4BU6xyrpvII4EI+R1KSlwjFIfBB/rAI488Qk888YTgzcJbCu/pH/7wB1G3Y8cO4d39za+ups3rN9M/X/6noDbsu+++hHswYuHJBW0BnFnQI3Ac1dgkPK/qeR35raOEp/XRRx+lv/71r+IWjGsYrA8//LDwEL/88ssED/HohmZxDUP1e9/7njBk4UFGymBwh59++mlBcYDH+PDDD6fp06ZRZ2enMKTlmAMDA+IUG9ZgyMPIht6gRnAZRiBd7HSmNAxjxGcqBDhKgwoMg06xac0sJRiJUjiicHdj0drh8JoF/5rVI1SzMzfVxHPdRI44vEaX+fPn009/+lOxIQzGJzaFwWC/8MILBVcX3lcYmAsWLiLwaB977DFyOBy011570R9vukl4ScGNRb8///nPgkM7Z84cmj1rFoFeIPnAmCc8xKBCoMD4RTnllFOE4Qwe8PHHHy90wQY1bASH4f3ggw8Kvu6BB8ynY489VuxVQh8Y1jDUm5ubBe8XlAkY3/D+ylBvaNfa1Eo7d20TBjIMZswRm+Jq0bsrANf4L913pyUWixnqrrnhhhvEr57rr78+SW38gsECypAfSTf5oiwI4MNAhkcpy4A8SBIC2G28fetm8cGcdMOAi9Xb++iCB1bQqm299OgPF9HifcYboEX5h8RjzEnTp9ckP678aI8cUYaHgteOizEI4LE5DF5syMpW4DFFamEYglyKhwDeB2zQjsQT/GhQWNTYgEJy3333aW7mM0XiCS1KQ66PUEZCwDWMQHUgYKY4vIOBcAJUn4k8zwmlSnRiFkpJiabHYhmBoiJgBg9vUSdkImH81HXkYlRklAatTWu5PkIZCQHXFBMBZFrjsGTFRDR3Wekey+QuoXgtEQszGOfu+mvI4DXLpsHirWSFSeJNsxW2YJR4/F5xiptcYQ4ROnKB8g1dawoOr5aHF1PLdzIj4eAa3QjEihtSRrc+NSbALAaX2sM7oPL2VvtysIfX4BXmxBMGL0B+wyMsmd9hPI83P60ro7U3Uzi2ypiC4VqagtJgOAqsQFoElCgNaW/zjRIiYCZKQygSI+nZDXOUhhKuOotmBMyHQK4Uw6Cr0XzKV4lGSKrBRR8CpjB4tSgN+qbFvRmBykfATJSGXr+yXd5pV+JE1cqHLzzsZlqHyn9V5zcDTjyRH16lap0rxdAZ6GVKQ4kWgZ1P+oE1hcGbjtKgf3osgRGobATMQmnwxWkMsViEhoLDG9gqG93ctE8X0zG33txKDwKceEIPeuXvq048Uf7RizsiIgAsW7ZMCEWkCqQcNrIgrFlqgY6IpgQHBCJb4S/d5jZEekAWOFkwJzk/WVftR1MYvFog4xFKvjvwtORwHSNQyQiYhUPqC0VJphRGTN5aKcCfPby1sto8T70ICA5v2LjEE3r1V/dHut8rr7xSGJODQ74RSSfUbUt9rg67pR7rnXfeoTvuuIMCA73kHxoQf9jcplUwn4f++mDiFuYkM70lKivsJF8bURsZE0waj1By5Q2ZQN2qVSGftH1VC4KBEzOLh1dNYQCft1YK8GcPb62sNs8zHQK5fhfDwyuTJKSTVcp6eDDffuMt8vv9hDjmRxx+GE2YNEVkPZs4cSKdeeaZYnh8nj312KOiDWL9I6EDCrygf//736mrq4v6+vpExjVQCZDsYY/dZog28KoiixtS/CIjm+wLT+v69etFG6QuRja11DwCiFO8bec20Q4Z1RYvXiwyw2HcN1etoV1ffU6ov3jJJdTYNCaho1r37q4u+vsjjyRSDiPbmsvtFpngsNFfbm5Dpjdg0NLSQscdd5wwjJE6ua6ujpD0AvHFZZF4QG9kboNe0Alpk4HF6tWrhZwzzjgjKeat7G/kMR/akykM3nQcXo7SYNzLSG3gGKcFj2wWD28wEiFsVrPbrNTvD1FYZFur/t3YZsGf3wmMgJEI5MrhhaEpfyCu715PH7Z/WLDaDouDQrHkVHtN7iY6bPyhImuZluDVn3wqDE1kWoPxiSxqJ554Im3YsIGeeeYZkbXsW0ceTc//37PCmIPt8corr4jMaBdffLHIznbXXXeJ1L4fffSRMA5BJXj08UeFrBUrVlBvb69I+YvxkQ4Yc4YhDU8rUgwjmRbq29vb6XfXXUfImCbLig9X0C9/+UuRQQ3pgJcsWUJ33323SCN88b99l2bMmEFtbW3iqdLfH7yfYICivPfeeyLD29FHH02/+vWvhb7IsPbll1/S5MmTRZuH/vawOF511VViHpgXsrTB4D3s0EMSHupNmzbRrFmz6ONnnhV6Pvvss4SxkBoZBjwMd5SDFy0SGd9wjjHgId65cyddffXV4r4Z/oONmI9TzhQGrxmAYx2SEZC/EvP59ZQsga/0IoAvDrN4ePt8YYqS4tkNR2pntzDwZ0qD3leyjv4ch1cHeOXv6na7E4M+t+Y5uvLlKykQCZDLNpJ/mmiY40koGqXDph5Mi05/Jm1seHgvkcb3pptuElIXLFggjL577rlHeGXvv/9+YcjdeOONIk3wnnvuSS+88ALdfPPNdPRRRwrj9+Z/3ExHzTiKbr/9doLxiwKjEX8oSF0MryeMv7ffflukA4bB63K5CFm+li5dKgzgSy+9lD5YuZJgpMoCfGCE/uKKy0UGzbPOOouef/55Ovfcc0UTGMDwrsKLfMstt9AJJ5xA0PHVV1+lBx54gBob6giG+GNPPi6y2WEs6IIyatQocewd6BN6w+iWHmbwen/0ox8JA1diA0yAFbziMHbvvecvNGv2nsIb/vvf/57++c9/irTK++23nzBy4dW+9tprTWXwignn8Z8pDF7etJbHinHTmkEAhpZZPIz+UIiEU5eIojEL1QqtAfhLj1XNvPDMNFGOw2um1ciqSyQQSbxfjph5BP2H4z8SfUABcDiUKC+JygwnWu0n1E0ge9hGlEEMvKQoMOTmzZsnDFRcS2O8q7ubtm7dSvDu4pE9UtPCAP5qnUJHWDROSWMNLyj+ZGloaBCnoAo8+OCDtPvuuwsPcHNzs6iH/L322kucw+sLD24qRxb1RxxxBLW0tol2hx9+uDCcN2/eTAcddBC1tSpGNbi5U6dOFV5Z6HjwwQcLr/PmrdsF5UDSRtAfBrO6yB/o0tjFPXip4ZnG+Ch4ggv9UPq6uoWeMHZRQN2A3jCc4UUGxUGWCRMmyFNTHKuGw2sKNFkJ4kxrxr4IzOLhHQhg05qyWS0YPxqLTPlGl18g5RuRR2IEKhcBaYzt3bY34c/IAsNtcHAwSQVvQx3ts88+gic7vm1c4h4iHMAgXL3qY+F9/eKLLwh/skgD8ZFHHqHTTjuNzj//fOEFxrUs4L1mKjCK4RWGZxcFnFrQCEBj8Pl8ia4wOmHwwxur3rCGyAowzmVBf5RA3JDFOX6gYx7g74L7iyIjN0gDXD7Bxb1RLc3COw2vMoxkGNXwVjfWKx5jIcCk/4HSkM9TaFN4eLU4vLmS5E26DtWjFmdaM3QtzeLhlR5dl91KtZRpDYvPHl5D3wI8eAUhEG4Okz9kbJQGuXEMP1S7uxXvJSDs6ekRBiyMXBis5551jvAAw0iFFxMbsrC566eXXyGOoATIAtoAOKwoixYtErxbUBpABRg7dqyo37FjOOQXPI8YG2OqCwxJGJ0wZLHZDDIvuugighGO/r39inG+YOEi4UG+8MILE55e6Aa6A7zWSy69VNSDpwtPMAo22aHAUL3gggsIlAoY0zDUv3/BD2jKpAmCDgGO8cKFC4WBjTGxOe70008X7eGhBn7gQMPDi/tyDjCit2/fLsYw03/5cHht14KUYWB54403xG7Ab33rW0latLfvIAvFqLV1TFI9X5QPATz2mTR2LNkcpvhdVL6Jm2SkQf8QDfT1i0duRqv08LubaEOnjxw2C41rdNO/7D2O6lzV/7rYunkj1dePSvDjjF6HWhs/Fo5S+472xMacWpu/Geb7dWcn2WzWnL6L+9v7qc7rSTwuL7f+VqtVGKDg2drtdmptbRVGH/i3LqeDZsycKby7s/eeR9FwkL7++mtBs9h7771pzpw5hGNnZ6cwBg877DA69dRTaY899hDz2W233URfeEFhzIKeAc4u+LvY1AX+MPiuaG+z2gRXd995+yTh9uGHHwrP7fTp02lgYIAuueQSQVWwWaxUX19PCw8+iBrq6gVsRx9xFHXs6hAcXXh7v/GNb4jvgsMOPZQ2bNxI4XBYeIox5m577k2j6uvE2KB0HHLIIbRr1y7RF17lbx7+TWqbOJXsVhLeX1A10A5/c/acJeYNAx5zhy323e9+l8KBoMBv7zl7iTmo8Sv3uqYbD+sAqomaKvO3v/2NTjr15ASO6r6WWCxmaIwh/NqAK//6669X6yUCKMOVr+ahJDXgi5IjgN2a++87P+2O2JIrUOMDgIO2dcMG8XjNaCiOveUNem/918LI3XdyE917zgHUNmp4g4rR+pVq/JUrV4ovUPlosFTjsFxtBGRoJHjVuBiDAB6jI0rD3LlzsyqAsFuOeqfYUJW1cQ02AP/3qaeeov++4zbGqAjrDxoGONhq2gd+hNx3330JOod6GFO4aHjTmnpJzHWOINZOc6lUU9qYhdJgif8uBo83XEOJJzgOr8FvtwybkwzWjIdPg4Dk8Ka5XdPVB+w/X3h+88UIsXfV4c1qGkQdkzeFwatDf+5aYgSU/N3VH2+1xDAWJN5sm6UsFrwOYmLzWq3knjDLD46CXkDV0Ck5DGs1zKjq5+CvkkxrpVgoREKQ0RDykY/EElxGIpBvlAZTpBbW2rQ2cmpcwwjUHgJmiNLQh0QTMSKbNUbYtOYPRSgiY5RV+ZKYAf8qh5inV0UIWGJ+QzOtVRGUI6bCyaBGQEL5JiczhYdXi9LAURpGLi7X1B4CZvAw9vlCFIlEyeO0kpUsFKwV9y6RaeIg194rPz5jTjxRUUtf39xKdZ66itKZla0dBLIavCAFI/UdwnIgCDFyPCOsB7J0IDgyCnYcIuyFjGmHewjpgTAaSNdXSMk1lWEhsrkPI1AJCJgl0xp4u/DqNrgcFAhHxHkl4Mc6VgECnHjCFIuYiwNqY+9GenfjcvJ46mh03eiy6X3IlEPKNpZhA9ldpI6da5geFT5wVoO3q7NDTBGhNpAN5FdXXUO//e1vRTo+xGVDqAyZcg8Nn3vuOWEgY6ccYsQh6DPS32Uq6SgN+bqrM43B9xiBSkTADB5ef0hJODG+yUPtPT4aDIQTSSgqEdN8dAalwWxc6nz057aMQDEQyMUBte7rdbRs48s0xrOIRjfHEzZEC2RNWqNEqX016pwWJy0Yc0BSJKFly5aJNLzYvf+D75+X4Mwi4geisLa3t9OJJ55Ip5xyioAGNgtSDiMWraxDdJbPP/9chBwrBn5aMrAR7U//+Z8izi30QSY0ZFtLjQiDSFbHH398TlEytMYppA6ROf7xzP9mtN0QRQhZ2ibP2KOkxjiiRSH+75FHHKF7415Wg3fO3HmJsEiYIII1I5czAhqffPLJibR9ABULiMwfWLyrr75a5GRGCr5LLrs8LSDIK63OMCIXJ5dflLItH0uHAFIScmEEgMC4UW7q6PPTYDBCXmfWj46qAA0/OLzxFJxVMaEKmwTCDSF2KBfjEMjV8dTkaaLWhlYaHxogt1OJJesPhJNipOY6C5FW2Jk9RIfNYksydmGo/fGPf6SjjjpKxLv943/8STyNhrELoxZJFWC7IDsanHh4fT366KMifi4cdEhAgSQQCGuFJ9TFLsh4Jr9T77jzTmHsItUwnpJjA5ZMU6we97bbbhN65RIWTt1Pz3lb21hCHOJMZeO6DcLBmRpSNlOfQu7hxwfiF+PHgJLEOVmKOiQZ7qhj8ia3JMr6raUW1rlTCWS828wZ4sVzzjnniKDJMHCPPvpoEQwZ1IeTTjpJjIOgxggK3L5lY1I8XRjO9/3lXpFlBMYz8kEjDR4CH0uFcY5Aznih4sUv6810XkydxATj/2HBxBs+x2Mx9ZBjQya883ixZSuyTyFHyC50TdFXjlnuczUmenRQy1GfQyYCk2MNkPJS/XpQt8O5nvFz6bu2K0R9/jANDnRSJKpE7Fi15jPq9CpfSIWun3ps9Zxkfa7HUmEAufg8Q2YkZEiSReolr1OP8n4uR/QtFD/0lWPke67ul29ftFcXtax8z6Uc2U8eZT2OyD6FzyH1PTOcQ7di6aGer5SZ67GYeqSOCdmdISJrf5eIxav+HJJt0QbJJlZve5/CQyEiB1FfcEBMyUnuxOsbFcFYkOCVlUVey6Osx1G+L9R1OFe3DVFI2AiyzRtvviXS9OKpMmyHJUuWEDy+yIb2zDPP0B133CFsEWQ6u+WWW+gHP/iBcLjBQQdPKmiaSNSAgqxmkJFU4tSCJ554QhjNuHfeeeeJtiveXU53/+VeYcTKOukp3rhxozDajj32WJGS+J/vriQYspMmTSIYvPDubtiwgTzOOpoyY4pIWQwDHO1R5OcPxnjgoYdFe9w76bTvUW/PLnrorw+KdmvXrhWG/GU/uZQQUjQw0Cu8yJs2bRKZ3a7+9TUis9rz//u0oJ0i2xsSZyDOtZwrbL6t27bR6tWrRT08rPCCo0AO7D7Ye4gpDLorbDh4xyEH7fADAkW2A/5I0QzPOrK+Ya7HHHOMkIF2WAskCtlzzz3FmuBHCWxHZIqDtxtzx5oM9PfRUHAckYrmhB8JahsFr0lkhkuXHTOrwSs0JxJgXHnllSIlH8JqYIIy7dxdd90ljNOW5qYRXw7w4GKC6oLUd/iVhXvIlIECBfGiBE1CTG5gQPyyB10CdbLgnrxWn+O++rpSzrXmlToX9bV6Xup6vefoj6KWjy97vCFS6y0tFrJ32ykUG44ZpO6X2j7fa7WsSjlPnWOuekcCEcLO5iDZkrBHf5RNg1bac7SXor4IxSxKHdJ3xrpi5LA4xBrIY65jar1/MvX9oKeXAuGvqbW5jTb3d4g4vC1eElQm+X7NV6ZsLyaZ8rpLxVJ9rdZTXa/nXOKHI7CVr21c4z2ADEiNdcr7AK95dXv5HsikV6pu2a7VsnI5J1c9UUAxMNTvTbWeqfOS87C5bJqfp7mMq26TOif1vYznMTs5A72ar3+8RvCF1tVFie8GzFUZKDUAACAASURBVFWrfcHjx79bMupYwjbQG0U9fqbrTO3U9/I5z9Y20tlB4bq6jO93yHA1jyIaIOp1e6nV7tQ0WNXGLuYpr+VRgJHlP3Vbt9MtDGyKuYQhBIOqo0OhYQb8ftqyZYswiGCg4bG4TGSFrGqPPfYYzZ41S+xNAgVzwYIF4nUGg+13/64kwoLxB0PQbrEJAxI8WuxRQkph0AxQQJ3AXqdbb7tDnMNzfOedd4pz/FgGjQLyDzzwQGHQof2UKVNEFjPYPMAOxi480MhqBqPx+eefF3RRmf0Mzj88QX/siaeEEQrbCXrCMITdBYP9F7/4hfBUw3sNA/Lggw+mX/361+L9De+oNJo//uA9evfdd+l73/ueMELvvfdemjZzujCEJfQwnP/whz8I4/zll18W5xgPNh/svf3nzxdzgB0HYxnzgGErn/DDrsM+rsmTJhKMfXiBzzrrLNH21ltvFZhiTYAtfojAzoNMOD/xBzkwqJGJDmuKgicNwH+IXIRwqTDoUYSNEv9csLrrhC2ajoaWs8GLX0rYtAZ6AorkuuAcv4zwywnWPkDFIqPAOwVrG2CoC15E6v6w/OWGN9kO1jwmmMpnkff5WHoE8MEhPyBGjKb1bGFEI67QgwB+BO4R2k4TJ8wcKaaM+E/4Gs+UN9KU0V76qsNLoYifmsftQRPHKI8tRypX4TUqbDt7OmnSxIm6uWOlR0T1GavSP2ncdPVJjYy4UOmuMXxfb0/K90Dm9hoiuEonAvjhke27eDfrHvTR15+Qx91LNssoYYiCckCWMEWiFnI6YhQMWXLSRLYV/eM9bPYwRcJ2IY9iytFmC5PX4VVaOL105LeOEvuLYFxJAw834YFW/8jGPVwjmcNPLrmYlr30sjDGwNsFvWCof5Auue6ShPcTMuCXhoEGL/A3v/nNJH4rvJSgZuKI8t577wmj9dRTThZG6TlnnSmooaiHxxTeY6T3xdNwnKMO9tXkKVNpyaWX0dlnny3GxnjwBKO8/MorQiZ+hMNAhIEJgxNjTJs2LRFQADYXvJ7wmMJwfP211xKfX5CHTG8wqFFge0GO1ENUxo1IabfBuF64cKGw2Q4+7BBaevkV9NWXXwjKA54+wthGgVf9oYceIuzJgmEMwx3ea9A09tlnH/rpz38m7DxgD+MZT/pXrvhA0DUWH/8doSPWBJ5teI1h5F9w/vniBwFsRvwwQFE279nEesBGHLZRsn8uZDV4wTm5+b9vEYNJ4MWoqv8kBxdKYSEAKKx+WPaYeKY3ChZHKywZSPK5codUqvBpkRFA7D/eHVpkUCtM3NeDAQpHYoK3izi8DpuVen3D3v0Km05e6mLTGgfSzwuyojbGF3RY2TNZVLksLHcEYOzmsmnNH/ILSoMz5CCK2x6RWIQophi5uRq70Ey2Ff3jqkaEsSzlxY+x5KRIcJzdeOONwqCDZxAGLGwS6C8NJoiDU05ey2QQ4P/C2ALNAd5W8Hnh3YQHVSaL6B3oE/1gTKpLZ2enMFhlHYxXlK7uHtp9991pVGOTuIbxKD2WMO5g/8giHYXwRMNziwKbqrW1VZyvWbNGeD+hFwrsMXiEYSfBOMWTcxSMAblwGqKoM7TB6w0j94QTThBGMrzRoB7M2Wdv0Vb+J3WR13I+2KQGjypoRijS9sM53qswbIENDHBwoGEgP/nYEwID6dSE5xle4tWffCoMcni8cQ8/FmBAw8MLD/jrr79OXXEGAGSnK7gHnHIpWQ3ex558XFjwsPDxqwnKgX8BCx2gil9Jzc10zdVXifHwSwHe4EMPPVRY6D/96U9z0YPbmBQBzrRm3MLgsYwZEh/gR084GiOnzUp2m7KDCAZwLRQzRMmoBZx5jtWDQLApRHV2BykGamnnpTaI5UhwsJ1//vnCAAO1ADYMPkunTp1KMGphvMKDCJtGFtAFHn/iSeHpxJNpOOrwGB5Prz9d/VnC4IVRKb2Q4KzKAsMRj/Cl8QVnH4w26UnW+sEgDW4pA0cYpPD8wkMLpyGoEjCmUWDoYvMW5qYu2P+Egr7S8AOXdVSL8kgHMoa9oCS8wqCZquXAsakuau+4ul6ew/sKT7O6YOy2tjbhmVbXA08U6TxDO3imH3jgAWEwg++MAu8wbEzoBY40qA4owEnOS1So/sOPsXxKVoMXuZ/BV5G/GnCEyx+Grfx1gjbyFxAWCRwNLBgWDjyNbCVdWLJ8J5NtHL6fOwKc1SV3rKq9JYxbxOL1OKzU5FU2nAwEI9U+bTE/M/zgqAmg002SE0+kQ8a09c4eB0VcMC1GfkbITW+pyqfWq6/luTzKvrGwjewpgRwQReai7/9bgr6AvUbwHuL7DIbVqaf9fzRhwgSxmR6b1mQBXQDGJB7PwyiGhxOcXhjFyDEgCwyvCy/4Af32dzcIXi7q4eA75eSTCbxXOANhDE6ePJnOOvds8cgedhJ0R4EhDY+zPJc2FIxC8F47OnbSDdf/ji7/+RXC4Js3b54weHEf9ABQEb7zne8IIxCG929+dbXwHoPrK59EQSYM2tENzWLj10UXXSTmC9sN3GRwd8H3PfXUUwVO8Mb+7rrrEtEjoBv6QyaK/AEgLuJzkGHC4AmHBxvyLrzwwgRfGW2BAWxBzB3zVjvPvvvd7wqeMfA9eNEiIRqb1ECLfemll8RmULn3C/OBVzrqVwxnqQeO8G6DX51ryWrwSnd/qkD1L4bUezB68Zdr0aI0oC9TGnJFsPjtmMZQfEwLkWgGD2OPT/kV7XHZhdELL284PPLLrJD5mb2PGfA3O0Yl1U+1I7uk47Bw3Qj4w34asA3Qdkc7TQhOENEUCLFz1SVmHUlhRBuN+oRdYI0qfXAMD29gHwr4yRtNfpTd5LCIDfUwBmHgfetIxeGG77N//ZfvCE1wD4/d1VRLeE/hyEOBzQM+KjZ2wSCTj+LlNBYsXER/+o8/EiJCgJ6AqFWgDYCjuvCNhcJwhXGKfvMX7C+eisOjiSINVJwjfBo8oijYsIV7zROmU9sotzhf+9V62mO3GeIRP/SFsQ3DFMY55gDP5267zyKX200I/9rYNEbIAqcXBeHP4C3FU3kYjcADRjD0glH9/gdKBCbQIVz1yqZc0ZFIUBEgEwX7rY44XAlRBo7xVVddReMnTRBz/s1vfiOoB9gkB5sQdJJXX31V9INc6Hz2904T9AYZjg03EWINHvG21pYE5eKMs88TVAgY2pg3PnuhO3jZoEqMHj9JyE39LxyLCD5var3WtSUWixn6bBKPDGDJS7K3VBJcDoCVybCWbflYGgRApN9/3/lJv/xKMxJL1UIApP72rduTYl1rtSt13W+eWU3//dKX9OezD6DX1+6kR1dso/vPXUCL9xlf6qENl48nVfhwT/3SM1yxGlEAj4g//vjjvBwoNQJN2aYJjyceyWeLA9sT6KHPV35OzmlOGu0dTdFolKxWhQIV6feQrcEn6qA46tX3USevcZRF9pfXOEpZsm5a4zR5WrVH+cSVHVHJSwy6Boxi2IqyICLGXx96QPMzO6uHVwop5TEdpaGUY7JsRsDsCJiFw+sLRigUiVKz10Fuh/IM0ReqDQ8vZ1oz+7uE9SsHArkkgmpyNYnwVm67k5oblcf2Cd2SHYiJ6oJOiimrIAUM6BQOJBl1BmhQUUOmC0tmihw2iUcXKZAyhzcFEAMuZaw7A4bmIYnEYx2jgRgKhoUKboeNPE6FLyXrjNatHOOnC2JejrF5DEbADAhobbrS0iudoaHVluvyQ0B6efPrxa3VCJjaw8scXvVSGXOuJpobo0Ftj2qGTVPBcJSGAhFh7NY5bYTQZLXy4cscXoPff7xpzeAFyG94cEv5B2J+mOXamr+LRyKVr1PUFB7ekdPgGkaAETALAgMBxcPrsttELF7oVSthyczwg8MsrwND9OBNa4bAXuigLc3NYmOU7K8Od6U+x331tfpc9k095tImtU81Xas3fVXTvPTMJZNTVOv1YgoPbzpKgx4guC8jUA0ImMHD2O+XlAbE4Y0HkY9EE3EVqwHndHMwA/7pdON6RsBMCDy79ln6x6f/oNaGVvI6hzcRlUrHoeAQtTa00E/mLan6jdUIvQUDjo3e3F9NWliZwuDNfQrckhFgBIxAwOtSqAzYJex2gNKgGMFG6MJj1g4C+cTYrB1Uyj/TXDatbelqp7e2rKAp9WPJ4VHidZda07reOqJ5w6MggcQFP/wh9ff3i0rEiJUZYhH15icXXSJiwh5//PGJ1MD33HOPiP962mmnJeqQFnf16tWJ6+ERineGCAPXXnutSL6AeMGIw4vsZOpwaRgNyRgQSzefUK96tUR0GiQXS42epZaLhBebN2+mefsfWNJsrFgLJPI4+bRTNCMvqHXKdm4Kg1crSkMub7Bsk+P7jEAlIwAunBkeqYeiMZFOGFjKTGuVjGs+unOUhnzQKn5bxNjkYjwCuWxaG+X1UqPHQx6vl9wOl1C6lDFPA6EANVgbksBB8oW33npLZO1CAgiZMQyG8O233iZS9iIBxVNPPUXLli2jyZMmivS2MDwRhhOGHlLnIsHDTy65OEl2sS/+67/+S2R/QxKGSRMnisQRMj2weizoiWQR5SyIG4y4wpkKstUBp1Ib4kg6gRTRxdgQaQoOrxalAW+wfAnJmRaH7xWGQNjOXziFIae/F97gRj9Sx+a0SDRGdS67SC2M9MIoCFVWK4U34Ri40imZtAzUhIfOgsBQaChLi+Ledjlc5CNfklB8Zk6aNElkeIUhhoQIKMhghoQIMGwXL15MBx10kPBgys6oGzVqFG3YsEGkHUaSBpk9VrZRH2+//XaRie3QQw8VaXBxD4YpYsAiQ9vDDz8smq94d7nwkl5yySWi/pprrhH1aIv0uf/zP/8jEjd8sHIlPfnYE8LLiwZoBznSwyqzs8HbiQxpGAc6oMCYR4IJJMtAPcaS/FV4tXEP9fAU4xoFfVEHWZCZWtasWUM3XHe9qIauaAe56vniRwMyo0FPqQvmjTbqdpgn+uIPOkIP1MmCeuggdUVfjIf4z7Ko4+zKukKOpjB4tTy8mEwmQnIhk+U++SNgD+eeti9/6dzD7Agg3m4gFKEGt/I68DiUI+rDqgDxZp9HofrhB0cxPAuFjl/z/ZSMrDUPAwMwEgF4eD3kodTQmVu3bk0Yc/DaovT1Kul2peGEZAVIaDJ5xh7CywqjDSlwkXYYXktkE0tXYISiDdLmwoA++LBDCIbtAw88QKBFoP6RRx4RRpzbWy+8zRgP9S+++KIwhpFZDUm1YHQee+yxYqh33nlHHEGxQCpetAe9obOzU9Sv714vvNG/uOJyuvWWm0UqZBijXUMhMdYee+wh9MHj/9ffeoOQtGXp5VcQ6q+/7lpacslFhBjJ6LNp0yYh40c/+pEw/Dev35w0XXhVX3vtNVEHrze85kgdDOoF5gYjG9nkkInu97//vZgDjFhgeu89fxFZ5F555RXhMUfkDtxDNrsffP884UHHPRQYuZgrCjLFXfHzy0X/Aw88kB5/4kkxh+bmZnEUjVL+y9cpagpKQ8oc+NJECMDDWx42lokmbRJVzEBp8AXDFI5EyetUXG3g7zpsFgpHasfDa5KXA6vBCBiGQL4UQ5vVTuFoaXn+8PCiqMN1NY8ZR39762/U3NMsHoPDIEXWLfxwheElC6gOuMaehMt/9jM64V8X06jGJpEueL/99hOG25/+9Cdh4CG1riww9MBbhfdY/Sj/vr/cK5rAm4oCIw2P/GHozpo1S3Bz4W2eOnWq4KMiFfG4ceOEEQjDt6uzQ/Tzejz02GOP0Q9/+EMhH5lOMQeU5f+3XBjaMCobGhqEEQpjFimRW1tbafHx3xEcV+gFI3XW9ClCB3hhpaEPIxgpfWF4r1+/XhibSOW76vNVNGXGFDEO/gM+oHaggBqC1L7ItAc5H374IX315ReE1MEY++ijlfTNGAc0DRjLHR0d9Mknn4h5IwXyIYccQqecfLJII/z973+f7rjjDgKHGUb+jBkzhPGMtYCR+/rrr9P27dvF+Bf9+MdCB6m/uMjhv3RP5Uxh8GpRGnKYEzcpIQLykUgJh2DRWRAwA6UhEFbSfDptFrLZrITkEyj+UCnZeVmA4duMACNQVgRy4fB6HaWPzJBt0jCajppxlGi258xpgpf79htv0d5z9iLQFGQJBAIJgw7G54KFi0RbGKlnnnmm4Phefvnl9PLLLwvvLe6jgCOMAmNPXXw+X0Ie6iV3GOPAsK3zekRztQ4wuHEf37X4mJV9urq6ErLUkQawSQyb2k466SSqr68n/AgZ1dIsnkDBOIX3FgVjDA4OUpCUz2q1sRjw+4Whf9lllwmD2uFwCCMWhnmmAgMbBR5VGLC9/YPiGvOW0SNgTJ9zzjl06aWXCm855jx5ylSxaVBg4KkTffbecw719PQIYxwe+ClTpggdQHeADHiMwdkFZaKru1v0SfdfOhZAuqdypqA0pJsM1xuHgHyjMaXBuDUww8iDgTAFI7GEoQudrBYr+UJRCkWq3+jFprV03gIzrE/V68CJJypmicvN4QUwqRxeGEyydA346MsvvxQ0hfqGUYSNWJKvisf1MB5lwSN93DvnrDOFsQijU3pwOzq7ZLNElIAXXnghUYcTeIXxaB5GKQrOYRRLT7L8wQADVxYYjij4rrWrLDHQBN59911xD57agYEBcX7E4YfRZ599JtI3w9sKjrHWJjc0Bj3D21AnDGdJ60B9xB0VusKzCo8z5IyfPE3IV/+n1lNdL887aXizoKSUwNDGvCETsqdPnZHwLKOfbIf5grIAagTaS57122+/LYxfXMMgBu3CYVGeLKrXVepQyFEFcyHdi9MnHYe3ONJZih4E5ItUjwzuWxgCMLSyffAUJjn3XjIGb3OdU1AZZGrhYLg2KA3M4c39tVKSlpx4oiSwllJoOFZaKoPUXXJ45TWO2GwlN00h+sE3vvENAi0AXFh4H0EPAF8XBq18XI5+Kz5cIYwveHLn7LO38GIipFlvb6943K4eY+nPlgovKuTgD/xUGM8w4k488URh8O211150xhlniHHUn+HwiOIPBaHT4IlFWddvFUYeNtf97rrrBG0AshEaDBxetINumM+5Z50j5gj+L+YbCUTE94T0PkuZ4MReeeWVYs5yI1hoICzCe8EjLHFactGFBM9vasHcZZFh3nANz3RDoJO+cchhAoeZUyeJTWughmDeEpcf/PB84f0FJUItCzJAEwEvGLIOPvhgMQx0feaZZ0R/cHyxUS8UKy6J3xKLxQx109xwww0CDLkbUQK8atUq4bLPtFNStuVjaRDAL8N58+Yl/UorzUgsVQsBeAvAr5K/gLXalLrupc930k8eWUn7Tx1Nd5+1P23oHKRz7nuPxje66d5zDqC2UYqXotR6GCUfHhZ4alJjYxqlT62NC88OOIvS21Zr8zfDfLFbHh5KeO4ylbvev4v+/P5faOKoCWS32Elt+Mpr9VEtS7bFfRT1tTxHvbwv2zS6G+m+4+5NJGTA62XL5k1CBh5341G9fKSPiDPtWzaKx/KgAoxvGyfa4b+OPj9F/D2JOmym6ty5i1rHjknUJRrHIyNs3bZN2ChtbWMFN1U9Nh7lY1zU9Q70iQ1ZoFtALqgHoFGoz7Xa9XV1C8oCjuoxwJGVm7UwDtld1L1rB41paRU4gGeMgjFQ8D0y0N8nggCAL4yCNjCuUeD9Tv18gz6IC4x6qRvwAoa9PbuEZxnzgxzgIHFCW4m/Y6yLZjTPSOovBoz/Bw4vvNDqddiybi0NhSJi3mgGo53CAYGhup2Ug9emxFrW4YfAfffdN2JOuG8KDm86D286foacGB/LgAA/UiwDyOYdwh+nLngcysMgbFpDQWxeLoxAqRHgxBOlRrh48s+edzYd4DqQGhweGj1lTPEEZ5Ek6XdoBiMsnZMMBqc0+FJFih/uo4YNYBhXWgaW7AdjUhqUsk5rbNThTxa1TPW5Vjt5H0cYkjA20U5rDt6JE+UQI/RSjNnh+2iopX9CQBxHqbdaN2DoVf1QSJWDtqn4q/urx9Cax+SZe6ibKOfOZAzVDfK1EU1h8KonwOfmQkC9A9ZcmtWGNurHYUbMuNcfonAkRvXu5I+KYKg2KA3M4TXiVTc8JieeGMbCyLNcojRg09r4lrGESAPNXsW7aKTOPDYjkIqAKTi86aI0SLd9qtJ8XXoEOEpD6THOZQS5czeXtqVog5Bk4WiMEH9X/Lp3IgGFhfyhCNXAnjURzijdjt9S4M0yGQEzIiA3XWXTDXxSySXN1pbv54eA4nzKr0+1t87XRjSFwVvti1KJ81M/JqpE/atFZ6NTC/f5woTQZHKzmjO+nRiRG2qhsIfX4FXmTGsGL0B+wzvqnRTu7xOP3/Prya2zIcDfyckIgeYhKA150C6Tn1MmyyvblRaHF49Q8uVnlE1hHogRKBMCRqcWDkaUOLxep/JRgaQTKPDw1kLhKA0Gr3JxN2kbPJnqHx5803Yi2rJ+bWKy8nsc3jicp3rltOoSnVUn6nbyXB5lM/W1PJfH1Dbp6tFO3sMRReqsrpd1qXJT+6vbqfur5ar7aJ3DuFPTSlL10uqDceV4uI+ivtY6V9fFu4g+ONeSlypT9tE6qmXLcxxRJEayHnXq83TyxKZEpxJvWKtNap0pDF4tSgMeoUgwUpXm6/IhwJnWyod16khmiP86lMi0pnyo2K1WstushPi8kRpILcwe3tRXJV8zApkRgNErNlphk5XTlvD24lwmKRDJFmx2cR/SZL08RzhMtEcRm7XicqQ8ecRjfng+0QZF1quPqNeSr+6rbq8eD33TtZMy1e1T5ci+6vmiPepRpO6p/eS17I+ISbNnzRKbzQrpnzq+lC+P0AXnck7p2kt91PfRV/aT53L9pK7p5on2UmY6HNFG3pPnOELffIspDF4tD2++E+H2JUIgpqRvLJF0FpsBAXBHjaY0+IIRCkWiVO9SPirwIWO3WihKtUFpYA9vhhco32IEMiAgDRJ5RFP5WB5Hdcp6WS/bqO/J/tpHxeiR99BfnsujlIkjyvDYyX1l+9QjyYxlcQNL3pc6y2vtozLG8JhSv2FjTbvfyHZx9ePzy6//yPGl/OGjxEYe5Rok6zdyPrK91G/kWEqfZDnqcbPdV7dNPpdj5no0NYdXurlznQy3Kz4C8pdo8SWzxFwQiGf2zaVpSdr44tQFmVJYDuILKlQHec1HRqAkCOTBzyvJ+CxUIKB+nM6QGIOAK56ZzZjRq2NUUxi8WpQGwMuUhup4kfEsCkfA6CgNA/4wDQUi1OxN3j3kCyqb2QqfGfdkBHJAgDOt5QBS6ZvkGqWh9JrU7gha2dBqF43CZm4Kg7cw1bkXI1DdCJiBw+sLKZ5ctYfXpU78Xt1LwLNjBBgBRsAUCLCHN/dlSPfdaQqDlzm8uS8kt6wdBMDhNTrxRDCsbAZx2IY/KrBpDQVZ2LgwAowAI8AIlB4B9vDmjnG62OnD32K5yyp6y3SUBubwFh3qvAVypqO8IStqB6MpDdgdi1IX37SGcxmLNxA3hos6YRbGCDACjAAjMAIB9vCOgCTvClMYvOk8vMzhzXs9i9ZBGjpFE8iCCkLA6CgNUNrrGt4RLK7jMXmRcpgLI8AIMAKMQOkRYA+vfoxNYfDqnwZLKDYCMoSI3ZJs7BR7HJaXGQGjE08gtbDHaRexd6WmSD4BisNA3Psr66vxaIYfHNWIa85z4igNOUPFDasbAfbw6l9fUxi86SgN+qfHEhgBRqBQBPr8IQrHiOpE7N1hKYjDa7HYKFYDiSfwgyPdBohhRPisZAhwlIaSQcuCKwsB9vDqXy9TGLz6p8ESSoUAMq1xMQ4BIz2M/mCEAqFIEn8XSLgdFrJZYzWxaQ34p9sAYdyrgkdmBBiBWkOAPbz6V9wUBq8Wh5cDXetf3KJI4ExrRYGxECHwLBpJaegPhAmUBoQkU4ciczuUmLwyKUUhc6uUPkbiXykYsZ6MACNQegTYw6sfY1OkFtaiNHCga/2LWwwJSqY15vEWA8t8ZRjtWUTYsWAkRqPcyR8T4PCi+ONZ2PKdVyW1N9LDXkk4lUpX3kNQKmTzk8sOqPzwKkVr9vDqR9W0Hl5MjaM06F9gllDZCBhpcMmwY054mq3DHxUep+LxBd2h2qN5MIfX2PcPh0U0Fn85OjugJBLGHdnDqx/74W8x/bIKlqDl4S1YGHcsKgJhW7J3r6jCWVhGBIymNPiCEeHFbXTbKe7UFfo6bTaC/YssbDKaR8aJVPhNoz3tFQ6fPvWTM1rrk8W9GYEKRoA9vPoXzxQGr/5psISSIWAJlEw0C86MgNGGFigL0ViUnHYb2ZIyrSmUhmAkWvUe3swrxHdLjkCo5CPwAIxARSDAHl79y5TVfTc0NETr1q2jUChEDoeDZs6cSV6vV3zRrfviM/IPDdCk6dNpfNs4oQ0ecW5Zv5a6untoypQpNHHixKxaam1ay9qJG5QFAXvYRsRelrJgrTWIkZSG7qEQ+YJRavQkf0w4bVaykoVqIfGEkfhrvR64jhFgBGoTAfbw6l/35G8yDXkff/wxXXHFFeJOXV0dHX/88XTeBT+iV/7fC/TLX/6S3G43zZs3j2699VZhCH/8wXu0ZMkS8vv9NGvWLLrh+t/RrNl7akgermJKwzAWZjsDpcFpNqVqRB+jKQ3w4EYpRl6nneRGNUDvcSgPhoKR6g9Zx1EaDH6zceIJgxeAhzcLAvDwwtnIpXAEslIaps2cTv/7zDP05ptv0q233EyPPPIIvf36K3T//ffTfff8mVasWEEtLS102223Ca/vww8/TKeddhqtWrWK9ttvP7r6ml9l1A5eYy0PL+8KzQhb2W7WAkezbGBW2EC+QJiQW2KUx06j3MNufo/LTh6nlcDx5cIIMALVjQA2j/P3sfFrzB5e/WuQ1cMrqQpyqHHjxtHGjRtpx44dtGDhIlENDy8M4TPPPJN6e3vphOP/e+hhLQAAIABJREFUVdTPnj2bli9fLgxhteEEmsSaNWsoEAjQ2rVrhcGMOslRCdm91NfbQ25vPXV3dcmh+VgGBPzhILntwz7d9o4dSddlUIGHiCPQ19Ut3iNGvAfcNqKvB4MUCEeFNurXweDAgKjr9wept2cXBVSvl2pbPHxGDfT3UXeXp9qmVhHzwecRihHvgYoAqAxK4rsZlEZegzKAnWGI7u7uhI2UoRnfyoBAVoNX3ffxJ56kpqYmmjd3jjBs5T3U9fT0CBqDz+ejUS3N4hboDvgjpId0Drviewf66NFHHyXQJT755BNavHgxtbe3U0dHhxRJg4OD1Ns/KDjC8e/cxD0+KQ8CXfxjozxApxml3+oi6+AgrfniizQtSlftcrloy5ZdYoCd27fQxnW94txuJRrsJwpFYrS1Yyet2mChZgpQNb5HMVe8B4bGDlHXFz2lA5slZ0QAa2DEeyCjUjV0E9/FKPjxx8UYBPi7ODfc8b2F12u6dPA5G7ygKjz//POC3qD14QN+b2oBjxd/qQVe45tuuklU33777cLYxWY4/MnyxZrPyeOsoykzpsgqPpYZAXjnFy1SvPhlHpqHIyJ4Vdu3Es2fP98QPP6v/TPyfbyGFs6dR4sOmprQoe/zneRa+RG1No+mQ/ebV9WhyVauXEltzbNp0ozhH+wJIPik5AgEfQH64KOV/DlUcqTTD4BN6+FQMOtenPQS+I5eBFa8u5zmzJ9HXgd/DmXDErZoughHWTm8EL5s2TJBWbj3nr9Qc0sLwYpubGwkPOpAAcVh8uTJ1NLcTB6Ph1au+EDUg/aARyGZiNbwDGsVBLoOxTgmjRY25awbCilrXM4xeSwFAbxpjYwS0OcLC0WsViUMWeq6IA5vGCTfKi7A3xbqq+IZmntqnHjC+PWBscvFWATEEzQ2h3QvQlYPL7x8Z511Fp1yyim0Zes2+nBrHy2Y3krHHXccLV26VPzyfuedd+iiiy4SxvBBBx0kNrTBs/vhhx+KqA7ZtNSK0sAk+WyolfY+PCtcjEXA6CgN/lCIhgIRalRtWAMiboeSajoYjghqg7EolXZ0jtJQWnyzSucoDVkhKkcDzrRWDpQzj8FRGjLjk8vdrB5ecEdANcBmtD/+8Y/05tMPkdtD9P0LfiA2rt155510+OGHJx45nXH66YSNbTfeeCNNnz6dLr744qx6aEVpwBuMf1lmha5kDZwel5At4vCWbBQWnAkBoz28/lBMqOeOhyGTujpsVmH0VntaYczXSA+7xLumj9j/wYURYASIozTofxFk9fBiQxn+tMrjjz8+ohqUB4Qoy6doeXjRH+FQuBiLAMfhNRZ/Iz2MoCygSI+uRKLOpXxshKKKQSzrq/FoJP7ViCfPiRFgBApDgD28heGm7pXVw6tuzOc1iAB7WGpw0ZUpDwXDZIlGyeNUKAwSCHs8zXAQHONI9XN45bz5WH4E7Jbk1175NeARGQFGoFoQMIXBq0VpqBaAK34ezKEzbAnB4TX6kXrMaiV7yqa1OqfyseEPVX/iCXh40+34NeyFUUMD86a1GlpsnmpGBJjSkBGenG6awuBNR2nIaQbcqKQI2CPKTv2SDsLC0yJg5CP1wWCEvC4b2awjPyacNgsFI9VPaUi7MHyjPAgMJ/grz3g8iiYCvIlcE5ayVsrEXGUdtMoGG/lNVmUT5OkUhgBHaSgMt2L2MtqzaInFqMk7kkfvcSocXnh42eYt5oqzrBEIcCimEZAYUcFRGoxAPXlM9vAm41HIlSkMXqY0FLJ0pe0jozSUdhSWng0BoygNfX7F0gCdITVKgycelmwwwN7/bOvH9xkBRoARKAYC7OHVj6IpDF4tSgM/QtG/uCyhshEwMg4vUgcHwtiwNvIjwuu0ETauhaMxilR54onKfgWx9owAI1AtCLCHV/9Kjvw20y8zbwlaHl6Ow5s3jCXpELZX/8akkgBXBKFGUhp8wbAwaB027V3y8PzCIObCCJQUAd40W1J4WXjlIMAeXv1rZQqDV8vDi6lxHF79C6xbQkxJQKFbDgsoCIFAwJjA+zBmw5EoIcmEy679MQGjmAsjUFIEOCxiSeFl4ZWDAHt49a+V9jeZfrl5SdDy8OYlgBuXDAGO0lAyaHMS7HIZ84PDH4qKKAyj3Nq5aWzxWLzV7uUFhxrUEi6MQC0jwBTDWl796pm7KQzedB7e6oGZZ8IIVBYCkpubzrtri8fmhWFczYXj8Bq7upx4wlj85egcpUEiYdyRKQ36sTeFwcseXv0LyRKqEwGjKA0wZEORqNic5ox7c9UI1zltgu4QCDPHW40LnxcXAU48UVw8WVrlIsCUBv1rZwqDN52HNxwK6p8hS2AEKhgBoygNA8EIRaIxakhDaYDn12KxUZgD8Vbwq4tVZwQYgUpBgD28+lfKFAZvOg8vb1rTv8B6JbCHRS+C+vobFYc3Fo0SvLfIsib5uuqZoD4WiwgvsLqezxkBRqD6EGAOr/Fryh7e3NfAH9Pee2IKg1fLw8tvsNwXl1tWJwJGxuEFpQHe20aP9geHx6F4eJFtjQsjUDIEOCxZyaDNRzBzePNBqzRt2cObO65ui3YEIe1vs9zlFqWlloeX32BFgZaFVDACiMNrlIfXF4pQlGLkddrJYbOMQNHtsJDNGqNq37TGURpGLH15KzgsWXnx5tFMiwB7ePUvjWk9vJgaUxr0L7BeCbxLWi+C+vojSoARBZ5bmUTNbh35MeF2OIRaMIyruXCUhmpeXZ4bI1A5CLCHV/9ajfwm0y+TJVQBAkPB6jZkqmCJSjqFQCgSTy2sHYMWXl9sXBvi5BMlXYdaF84/uGv9FcDzZwSKh4ApDF4tSkPxpsiSCkHA61QMHU4tXAh6xekDDq9xlAYlvm6dU5v15HHaCI7fEKcXLs5isxRNBHjTrCYsXFmDCDClQf+im8Lg1dq0pn9qLKEoCHBq4aLAWKgQoygNwUiUIlEL2W0Wkj9+1HNw2pQfRL4qTzzBHF71qhtwrjBnDBiYh1QjwJvI1WgYc86UBv24m8Lg1T8NllAqBDi1cKmQzS4Xm9aMKqAqBMMhTWMXOsEQtpKyma2a6S/M4TXqFRgfN2Tw+Dy8QIA3kRv/QmAPr/41MIXBy5QG/QtZKglhm/Yj7VKNx3KTETCK0hCKJ5RwxFMIJ2tFJLOvVTuHlz28qSvP14wAI2AEAuzh1Y+6KQxeLUoDP0LRv7hFkWAJFEUMC8kfASPj8IYjUQpGYuRxaf/gQRxeFFAfqrmwh7eaV5fnxghUDgLs4dW/VqYweLU8vHiEwqmF9S+wXgn2sPYufb1yuX92BIykNPT7gxQOR6nRrU2idDniHN4qj+bBHt7sr9OStuDEEyWFl4VXDgLs4dW/VqYweLU8vJgax+HVv8B6JTClQS+C+vobRWnwh2JCca2kE7hhtVrI47QSx+HVt77cOwsCnHgiC0B8u1YQYA+v/pU2hcGrfxosoVQI8Ka1UiGbm1yjojSEozGyRKNUl4bSID2/4UiE+DWS21pyK0agUhFgimGlrhzrrUbAcIPXEc/YpFaKzxkBRsBYBLBpLZIDXQGeYL/F8I8RY8Hi0UuHgDajpnTjsWRNBDhKgyYsZa1kSoN+uA3/pgqFOO6M/mUsnQROPFE6bHORbBSlIRiOkNXtICSY0Cpuh5WsFlAaqnvTmtbcua6MCPDXQxnB5qHMjABTGvSvjuEGb6Yp8Ka1TOiU5x5vWisPzulGMZLSkE4n1LsdNrLFQ5bJEGaZ2vM9RoARYAQYgcIRYA9v4djJnqY2eKWSfDQOAU7taRz2GNk4D2+UvC4buezaHl7p+eU4vMa+Pnh0RqAcCDCHtxwoZx6DPbyZ8cnlrmkNXrzBOEpDLkvIbaoVAcThNaqEojHyOLVj8EqdHDYrDebA85XtK/HIcXgNXjUOS2bwAijDM4fXFMvASuhEwLQGL8fh1bmy3L3iEUAcXiMoDUgVbInFyGVP//FQ51J2EwUNTH9c8QvME8iOAIcly44Rt6gJBJjSoH+Z03+j6ZedswStxBPozB7enCEsXUP2sJQOW5NKDkeVjWh2q4WwOU2rwBZ22iyE8GVcGAFGgBFgBEqLAFMa9OOr/W2mX25eEtIlnuBNa3nBWJLGHGO1JLDmLNQIDu9gICKoCuli8EJ56f2FNzhS5emFc14sblh0BOwW42g9RZ8MC2QEGAFDETCFwauFAHN4tVDhulpCABxeIygNkWiUwpGoiMSQDm+nTfno8DOlIR1EXF8EBHjTbBFAZBFVgQBTGvQvoykMXi1KA5Pk9S9uMSRwHN5ioFiYDHB4jSi9vrAYFpQF6clN1cMdj88bCEcoUsWsBnjYjdw8mIp7zV1z4glTLDlHaTB+GZjSoH8NTGHwpqM06J8eS9CNQMylWwQLKAwBGFpGUBqgbTASozp3emtjlNtBdpuVwtVs7RIJD7tRPzwKe9VUWS9OPGGKBWUHlPHLwB5e/WtgCoM33TSYw5sOmfLVM4e3fFinjmRUlIbBgOLhRWIJmzXzR0S1hyVjD2/qq5KvGQFGwAgEQnavEcNW1ZgZv826u7rokksuoQULFtA111wjJo66G264gQ499FD69re/Teeffz6tW7dO3BsaGhL35s6dS0uXLiVsaMmlaFEa0I+jNOSCHrdhBIqLQCgSJXBzG912slkyywbXF5zfai1GcKirFUueFyPACBSOQL29ej9nC0clv56ZI8sT0b777kter5deffVVIdkfDtKaNWvo4IMPppNOOokQnmj8+PHi3lNPPUWvv/463XjjjfT888/TtddcSTfddFNWjZjSkBUiblCjCBhBaYCxG41FRepgW3xzmhb8LgMTY2jpw3VViACHRazCReUpMQLGIJDR4G1uaaEzzj6P3n79Ffr444+FhnjMCgP3mGOOoUWLFiW0bu/YQcuXL6fjjz+eFi9eTH6/n/785z8TvL4wmNOVpqYm8vl8I24zSX4EJIZUOD3M4TUEeCLyejyGRGmwWK3kC0appc5JngxGrTuemCIQZs+DUa+Rah/XG98cWe3zNPP8+EmrmVeHdcsHgYwGLwThAycQCCRkYiPN+vXr6cc//jHNmjWLTj/9dDrzzDOpc+cuYeTOnz9ftJ0wYQK1tbXRls2baNbsPRP9cbJt2zb6/e9/Tx999JGgQ8BABi2io6NDtMP3aG//oDjv6uygXL9P49+/Se3T1aXKVLdTnycprrrI1Ab3UuXLrtnuoZ26r9Y4WjLU7dTnctzUo7pNOnldXV3iR4y6rVqOup+6jfpc3R7nso88pt5XtynkHvrowU89plqO1CtVvrq9PM93bur26vNNg1ZqpX5a8e5yITpVH1Sq28trHNVtU9uo28m2sg2O72/qQzVt37WD3vyQqCGqvP/VbXDfEe0TG9f+8MxHdN5etoQusp0QEv8PdShqveK3RL9091LbyGt1+0yyZft0bWR9qjzoiXsdnV2E98HWhg1SVOIo5yJlyOtEA9WJbKOqSmAh75Wrv5ybejzooFWv1hfnsl1qvbyHY6pcrTp1G3VfnKOodZGfQ/Fb4pAJs2w6po4t5WrJTFeXKkPdTn0uZaceM7XBvVT5sn+2e2in7qs1jpYM2U6OI+WgPvFd3N2TuC3bq8eSN7Xk53pPjqtur1WXOq5aH/W5lJN6zNTGbPpDH7wH4HTMpBvmWMh99EFJxVSpVf6XbdR1sr28J6/VbeS5bCOvcZTt5T15rW4jz2UbeZ3aX/ZFu8HBwbSRdbIavOoBcN5YP4quuuoqsXt889bt9Mgjj5Db7aaDDjpIGMapig0OjfTejmlppZ9ccjHh3n333Sc8wPAatzQ3i+EQfgO0CYfDQTNnzqTU3YmSvO0IDyWph3p1HeQM0RDZ/FZKvYeOuD8Qtib1kQIj7qjol9ounRy1jlr6pfZLvcY4qXXl0l/OORWPcPQLmjZzOrntziTd1Nik6ixlqduo5arrZVvcNxt+WvNKrZN6y6OcD9qhqF+LqX1Tr9FeLQfnLZs3UVd3lHbbfVYS/upx1GOgTz6v93Rr8dHgRiLaQNPGT6C508eJ9ZdjqnW8dqKVfvTQB/T06i5aMH03OnXfcbKZOKKt1vtLa+5qXdT9tNpCeGp9Pv3Vc5AKp8qT9XgP4Ic7Pp9SX6Nq7PMZX6/+evprzRN1VvtA4vNOS35qv9RrrT7AOZ/XI2TIIvEEhQ5l9qxZ4qg1bup6og2Ken1S+6Veo31qXTH0h1zIke+D1DHk/dTXltH6Sz2hO5xR/qEBmrrnflArCVez6i9fP1K/SsNfrT/mgM+hKVOmiKd+ck6pr5l8Xu9yfSELRT1ettcr2pulf6oe0L2uro7SRdbJyeB1uYYfa4OeIL24CxYS7dixg55++mk67rjjCO3gEUGBlQ2qwvhJE8S1+j88Jpde32XLllFfX58wetXUBxi7uJZ/6v7D5+7h08RZcp2XvIR/Skm+hzrl1sh60T7RL7XdyPZq3eODEVFqu2zX6Jncplz6S51T8RjfpjZiVLqpsEnVWcoaxj0Fv6S+SmtT4peyFoqmKgzw+onTdeQxMXdxktx2JE6p94flofuolmbxoxDUIqWMbJ8qM+/Xi8ZaeLx1ItMaKA3J669oIefa3EJ0yVG709VPrqK7395OR86dQjPH1Md1jbcVh+x6i2YqXZRT2U8ek0SPeK+kfb1prKOcQ7LEkePgM83bUJfmcyilfRn1V/QudPyUfkKYW75JVZCktst2ja7JbfJ+PapGx3oqG5830PB7YOQYqMltPZN1S9VVGTq5jV795XSyvZ5NqX98LaEbKIZtoyQ28ihnx/gPI5GMTbFeP5Df3Nyc9Dof+ZpJHnvk6zvL/Yr5/JJoy/nIo6xPf8wYpUF2g8WsLjL6Ao6bNm2ixsZGsRDTp08nbFxD+eKLL6i/v1/zC1MtKxQKEW9aUyNirnNwsLkYh4ARm9YCoQhFokp4BvleT4fAGQdOodMOmEw7+/x01ZOrqM9fXYFTjcA/HdY1WR8eptPV5PxNMGmEB+U4vMYvhNqja7w2lalBRg8vQpAtufRSghe2s7OTvvOd7wjO7gsvvEAbNiicNkRx+N1114nZn3rKybTk0sto9913F/xe8HRzKenCkuXSl9uUGAHeJV1igDOLl9ykzK2Ke9cXilIsFiGv0y44/Nmk/+K42bRqaw+9ve5ruuvVdbT02NnZuvB9RoARqCAEeBN5BS0Wq5oWgYwGr6u+ka65+irxJyW0tY2lwxYdRr6gsqkMj13l4ybQFP760APUvnU7jR07liZOnCi7ZTyyhzcjPMbetMDDonrWYaw2NTe6mk5UrskPBcOEWLzODCHJ1LqMrnPSb0/Ym77/1xX0lzfW0+J9JtCcCaPUTSr2nOPwGrx0/IPb4AVQhmcPr/HLAH4qF30IZDR4EaFBcm3Vw0gDV10nz8H50+L9yft8rCwE7GEbUfoMs5U1mQrTNh3xvtTTCEaUMGP2bFknVIocOL2FDp45mp7/dAdt7BysGoOXKQ2qRTbilCkNRqDOY5oQAVAaRvJ2TaioCVRCNDGtkhOHV6tjMeuY0lBMNFlWNSFghMEVisRoKBChelfG38MjYN6trYEa3Db6Ymf/iHuVWsEe3kpdOdabEaguBNjDm/t6pnMWmcLgTUdpAFmeCyPACJQXAV88Jbg7za/kdNpMavGIW1u7h3JOK55Ollnq1T84sCHv3jfW0xc7B8yiXtXrYbdoe2qqfuI8QUaAESg6AqYweNN5eDnDS9HXO2+BYVt+Xr68B+AOGREwwsPoDymRFjJlWdNSercx9eSw2ejLKjUIn/2kna5/7nN67P3NVWPQa62jmerCsYiZ1GFdGAHDEOAoDfqhN4XBqzUN3hWqhUr56+yRcPkH5REFAul4SKWGxx+KkSUaJbcjv4+HaaOVzY2bu4YoHK2OdMPqHxxdA0ERn7hrMFg18yv1a0m3fN4/oBvCYgjg7+NioKhPBlMa9OGH3vl9o+kfT1OCFqUBu0KZ0qAJV1krw3b2sJQVcNVg4CGpH6mrbpX0FGHJIsEI5UtpQGD6Ro+DenxB2tVfPfFTJR8Mm/nCkSgN+BHFIlbSNWDhcQSqK6xzxS4rR2kwfunYw6t/DUxh8OqfBksoGQL8hVMyaHMRrPYw5tK+GG0i0SjZnDZy5BiWTD3m1NF1ZCWLiNSgrq+Gc4Rrs1hsJH4QxCNZVMO8eA6MACNgfgTYw6t/jUxh8GpxePEIhTm8+heYJVQ2AkZ4eCPRGMWsVsonLJlEeUqLR/RDaLJqK32+MAXDIYLhK0O3VdsceT6MACNgTgTYw6t/XUxh8KajNOifHkvQjQBz6HRDWGkCwtEYeV2F7Y6f3OIll91Gm7t9lTbtrPrC0EUZCjCvPStYRWrAURqKBCSLqXgE2MOrfwlNYfBqeXj1T40lFAUBpjQUBcZChRhBaQiGo4LOYLda8lZ797Z60Wf9rsGqiGQAD7vcPDgUjFAwEhMb1wJG5HzOezUqvwNHaaj8NeQZFAcB9vDqx9EUBm+6afCmtXTIlLGeU3uWEWzjh4JRF4rGyG6zks2a/8fDtNY6MYlNX1cfpUFuVANGbO+W6bXKT5jKBHTmYThKQ2Z8+G5lIJD/N1oJ5qVFacAwzOEtAdg5igz6lF32HJYsR8BK0AyexXJzeBFOzBKLUZ3TJozefKc1vtEjoju09/powF/5jwfUHvZgOEJOm4V6fUERrSFfbLh9AQhU/kuogEmbrwtHaTB+TZjSoH8NTGHw6p8GSyg2Ak6Pq9giWV6eCCAcltrgyrN7Qc39wQjhcX0hG9YwoNdpo/GNbjH21p7q4vGC24zIFb0+5vAW9OLiTowAI1AwAkxpKBi6REdTGLzM4U2sB58wAoYiIGLNRmPU6HGSvcBPh+mtSoph8HirqYDbjIJYvIO8ca2alpbnwgiYHgH28OpfogK/0vQPrJaQjtLAHF41SnxeawgYQWmAdxcGHR7duwq0eKe1NoilQsa1ailqbvNgkJOxVMu68jxyQ4A5vLnhVMpW7OHNHV250Ti1hykMXi0PL8fhTV0qvq41BAyhNISiIhIBNq0VWmaOqSOHDckn+is+UgM41G67kwaCMcFtjkSVyBX9VcBPLnR9uV/tIcAcXuPXnD28ua+BzI6Z2qPwb7VUSTqutTy8/AbTAWgRu3Jq4SKCWQGiAmHFe+l12QvWdno8UsOGzsrn8EoOdb/PL7jNNmtMeL/9IYXeUDBI3DE3BDhKTG44cauqR4A9vPqX2BQGr5aHV//UWEJREIjx5rWi4FiAECMoDT6EJYtEyWG1kLNAL+/EZi9ZLVbaUEXZ1kDfDcWGk3F8PahEMSlgWblLPgiEGed84OK21YsAe3j1r60pDN5002AObzpkylfPYcnKh3XqSEZQGqBDKBIR0RZS9cn1enSdk9oaXARvMcKTVXpx24gGglGKhIOJqYQ4EG8CCz5hBBiB0iPAHl79GJvC4NWiNGBqHIdX/wKzBEYgHwT8oQj5glGqc9nJVqCHF+PJBBQbq8TL2zsUoGBE8TaK0GT+cMXzk/N5XXBbRoARMBYB9vDqx98UBq/+abAERqA6EQgEyvtIF9zUKMV0eXixEpNbvGJBvuwYqPiF8UeIwmFQPRRec9Rqp6Egx+Itx8LaLcM0knKMx2NoI8BRGrRxKWcte3j1o134zhT9YyckMIc3AQWfMAJJCLhc5eVQ+0LKpjWrJUYN1liSLvlcIFIDypYqCE0GSkOfP0yRaIw8TitRMEp9nHwin5dDwW3DMQ4BVzB4RezIm8iLCGaBotjDWyBwqm6m8PCmozQwh1e1Ugadhm2m+E1k0OyNH7b8Hl6F0tDgdpKetZ85pl6EJsPGNcSwrdQiUzsjIQeSTYypt4r4xPDwyh8HlTq3itDbURFaspKMQMkRYA+vfohNYfCm8/Ayh1f/AhcqIeiLP0q3lPeReqH6Vmu/cnt4A3EPr8ep71Hy1NEKpQEcXqQbrtQiw5L5wySoHq0NDdTgtnF64XItaKhcA/E4jIC5EWAPr/71MYXBq38aLKHYCDg9yqN0e7hyjZViY1JueemyxZRSD18oSr44P1WPodpS5yKv00G7BgLU0ecvpcollS09vD2Dfhrwx2hKs4ccNhv1DAVJphouqQIsnBFgBBgBImIPr/6XgSkM3nSUBv3TYwmMQOUigLBk0uAq1yzw6B6lyePUPSS8vCLj2teVm2IYHl6bs47CkRgFwyFq8jqpzmkjP4cl0/36yEkAJ57ICSZuVP0IsIdX/xqbwuDVPw2WUCoE9PA4S6UTyy0dAjL6gN2mpNAtdCR4h2ePa6BQJEYrN3fnJGZr9xC99VWn6Ti/keAgDQbDNBSI0LhGN7kdNhr0hzhSQ06rqrMRJ57QCWBxunOUhuLgqEcKe3j1oKf0NYXBq8Xh5TeY/sUtigT+wikKjIUKkRzSQvvn2w8GKgw7PXQGOebBM1vF6cpN2Q1ebGy79aWv6NibX6cnV26VIkxxhIcXGegs0Si11jtFjOIe33ASClMoyUowAiVEgKM0lBDcHEWzhzdHoDI0M4XBq0VpwBuMozRkWLly3eJHiuVCesQ4RnB4w3FKg11H0gk5kXmTG8Xpp9t6s3ptEfGgvVfh+v759fXU5zfPbiV4eP0hRZ+JTR7xY6DfHyFmNciVLt2R4/CWDluWXFkIsIdX/3qZwuDV8vBiahylQf8C65WQa2rh1dv76K7X1pnKUNE7d6P7G8HhhYcXxVkEgxcb18Y3egje0K86+jPCGYlExUYwj9NOCGV216vrMrYv901/KEYxq5Wa65xks1qEroEQJ58o9TpwHN5SI8zyKwUB9vDqXylTGLz6p8ESSoVALl848Mb98olP6Nf/WE1vrO0slSo1KbfclAZEH4j6Q9Ts1R8AFbSIA6Y1Cx7vig1dGdcPm+W+HgiIGLeBcJSe/nBbwuObsWMZbiLTGqJX2O1WavQ4yGbvualOAAAgAElEQVS1CtpHGYau+SHYw2uOlwBTDM2xDqyFPgRMYfBqURr0TYt7Fw2BHOye9t4Avbuhi1rrHcI7V7SxWVDZozRIyC3W4nw0HDCtRUQ4+HBLjxSteYSRC0rD2FFuOnHfCbRqWy/d9vKXmm3LXekPB5FcjZw2CzW47ORxWMWPAqRh5lJaBHL5wV1aDVg6EGAOr/GvA6Y06F+D4nyr6deDJZgVgRyolG+tU7x3viCMFp9ZZ1KRepXbwwtPJopLZ5QGCfbC6S2EiA+5bFzrHAgQQpn94rjZBGrDs5+0E6gyRpehoIXC4bAISWazWcntsJDV7aBQnO9stH48PiPACFQ/Akxp0L/GpjB403F49U+PJehGIIuHF7vrX1y1jTxOq8hEJTce6R6XBRiCQCQaFcacowgcXkwAHN4x9S7B4123ayDtnHp9oXh0CDshLfGSo3ajT7f20p2vfpW2T9luBAZEkglkWENB2mWUnqEcfg2KlvwfI8AIMAL6EGAPrz780NsUBm86SgNHadC/wLolxJSMa+nkwKP73oYuspKF4OHdHt9pn6491+eHQLkTT0SiMcFV1RuHVz3LA6Y1iUvQXtKV7sEgeV02QRdAm9MPnEKTRnvo5TU7xesrXb9y1O8KOikUjYnMcUikgbTL4PP2miiSRDlwMGIM5vAagTqPaUYE2MOrf1VMYfCm8/BylAb9C1yohKAvILpmi9Lw2peKETN/arPw8vYNcXzSQjFP7YewZOWmNISjMcFVTdWl0GtsXDtweqv4MfT+xvQG787+ADntDhrToPzAgmf4imNm0xftA/S3dzYUOrzufg6Hg/r6dtFQICzCkdmtVpGFDnzergF+resGOIsA5vBmAYhv1wwC7OHVv9SmMHi1psG7QrVQKV+d05PZswtNQGd46oPNIhD/4nmTaGKjRyiIjFlc9COAsGTlLsFwVPBn7VZ9mdbUeu83pVnQXT7arL1xDa8jGI9ep4VG1yuvOxjKx80dR6MbXPT2um4y6jWFz6HeoJMQn9jrsotpuShIoHzIrHTqufI5I1CNCPD3sfGryh5e/WuQ0eAdGhqiJ554gm6//XZatmxZYjR4/+655x5Rv3z58kQ9Tp577jm66aabktonNdC40KI0cOIJDaBMVgU6w0dbeqilzknH7NkqjJVgJEYd/Yp32GTqVqQ65aQ0ILwcHt277FYReqtYgE1vrRNe0R19vrSG666BAMHIbnQrRiXGhpf3gsOm04ebuumvb28sljoFycHrus5pE15el8dLSMyBUGpmSpBR0MS4EyOQAwIcpSEHkErchD28+gHOaPD2DvTR008/TTfffDNdeeWVYjQYu489+Tjdfffd9Pzzz9Of/vQn2rZtm7gHY/faa6+ll156ie666y56af1L+jVkCaZF4NU1HYToVftOHU2Tmr00vtFNvf4otfcoGbNMq3iFKFZuSoM/GKFgKEJuR8aPhYLQQ7QGq8VK76wfSWsIR5UYvHa7g8bFnxJgEHh5T9h3ohjvtbWdhhiX+KKPBJBpLSJi8Cp62cWPgj4fJ54o6MXAnRgBRiBvBNjDmzdkIzpk/GYb3dBMt95yCz344IPkcimPGjdsWi+uH330UXr22Wdp9uzZdMstt9BQaIieeuopuvDCC+nFF1+kAw88kG768U0jBkytaGpqIi0OLx6h1DeMSm3O12VGIBO14emPtpPT7qTv7j9BaDVltJfC4RDtHCj97vWOgTA990k7dfSxcV3sl4TLbqMGr7toYmG4HrRbGw0GwiK+bqpg8GJ3DYYJuS7GNdcl3T5wegudc8g0Wrmplx5bsSXpXjkumrwu6rc2UCAcoXqX8nGJpBz4UQBKA3TnUjoEvF5v6YSz5JwQwF4apjTkBBU3MjkCw88PNRSFsYO/wcFBqqtTvoi6unto+/btNHPmTNFj0aJFdOutt1J3Rzf19PTQMUceI+rnzJkjPL3g5+ELT126u7ro5VdeoXXr1ok2u+++O7V37KCh/sFEM4yDP7+fDZoEKGU+wbp/sebzpBTPTlJ4pduCHkFnQOap1vAuWreul+rDvYKruXZLB21eTxSKlcbwdXuIHlvRQVe+2ENXL55K58xWXsZBSn6dlRmuog/X29tLa7tCNGbd2oTsUs0R64o1hScTpWPLp0RdrUVbw4aIj7Ah7r31X9OWdWtJzgPj9gwFqL1bee9/vf0r2hKsp5jFLcbG/X2aw/TAoJ/+79Md9M0JSurjBCAlPunq/Jq6O/tE8gz/kE98Zu3a1kn4UYAQfJ+s204Tnb7EfEqsTs2JR6QefA7hu4KLMQh0dXZQOEo1uQbyc3F9f4wWjg6Rw+Io2mdiPquJ98CWzZuSvovz6V9tbfG5oBXUAOuDgqejWiWjwSs7BALDnEwAn2qEqusamuplN2poaKDAQC95W1oSdThB5iJ8gK1du5a6upRHnNiggy947IpGgUwY2eARcyk/AqGQYqwODvnI4Rh+dCtX4+F3NgjO5T6TGkUWHsnxikaJ1u7oIF+wLqfsPBhHrnmuswyH7LSuo4/8/d309mdEB7aOpfFFSIWb6/ilbKfGA++7VhqknqHSeLnUY2Fd+4YGKOQfIrfDQ11DRC7qLdpUZ9aFBNf7q10D9HnnQGK9MG77UEgkLGn0yPi2+LxRPnPCDjt9Y7KLFs5ooXc3bqP//dRCx81qyum1VSzlt/SSMNYhT/15hFBqsZ6t1FM//JmXbUx4jFFg5HPJjoD8HFLjnr0XtygmAjB28X1ci2uAz6ebXv6Knnq/h/7r1Ol01Izc3+vFXAPI6uvtIbfXuPGLPR+98qTNoZaDJxF4rabb8J2TwasWCCPU7U5+3Ik6aVkP+XzUHO/Q399PWryT8W3jaOnSpaIVNsS1t7fTxIkTxZ8cS1IaUM/FGASwIXH+/Pmag/e85yOP82s69+BpNHfOONHGOrqPfvX/lpHFOY5mzd5Ts1+xKmMrfWR199H77SGyN02mubPGFEu0aeTgqUf71u00d+7csujk3DlAMUcXjal30r5z9qG2+rw/HjLquc+H79GHm7upqW0azZ0+/CN47ECYgk9tp1FuOx2zUPv1dnb/Orr4b120PVRX8teWehLwRteNctBQsJ32mT6O5s7djbzjBsj9zuuErGtz9juARrmzZGdRCfz7e5tp/6ktNHcmf3GpYMl4is+hcr0HMipSozfhnIJHrdSf6WaFd8KqEFndgzRqzESaO3eKIWr6hwZozn4HjnhabogyJh9UshG01MyJgDZ+/PiEF66luYkmTJggKAgQuHr1agIloXVcK3k8Hnr11VfFOBs3Kruqs3Gw/n/2vgM8rqtM+53e1Itl2XLvJY6d2OkJSSAhIZTQAoRefmAh7PIv+1A2C7v//mF3YZeysPsDC0sJbEiAEIckpDqxU9x7l6xu9T6SRtNn/uf9Zs7Mnas70siWZDnWeR7ptnPPOffcO/d+5z3v936kQWRTaRgeuvBhRY067VLap/R4tdfc6wthd30PbBYLrlycNlwKXVbRUu2eYpWG6s5hNPWOSKCC7u5h7Kju0jZP1hnV64GnTl7woAWjGjbBHdOp0uAdCQmlgQoE0Wga1Z9gkw2zk9q0am6ecGGPtWTKk/X7AiJLRnWIbOmW1RWYV8xAFF3g/T+XdLRlAPc/dgxcTiT5BnsRj2dKxNktDvgCYUzkWX+ttgefeXA/vrH12ESqv6TzXoqo4ky74TR2jdC0mdbOqWgP5RD5rWHqGfRPRRU5lUmUnbPls+n8eiD7FyY5fUcE9hvf+IY4on3zm98Ep7jf/va342Mf/ijuu+8+7N27VxzVaNjecccdoujA/S+99BI+/elP59Q6I6c1nmjE0cipwNlMk9YDEctolO9wc79IMi2fk4dST2IamhVazGZUFFhFs5RG8VSlnqEAmvtGUJJnR1VlAV6p6YY+bO1PdtTj6/+1G48fTiiITFVbprJcTstMZ+CJcDQml0POvWXyZHhTXVRZ4BQubLvO0ZCGNlOBKztSuqDYhU/csASHG3rx+/3NqTInsvKnI234p1/uw/2PHZ+ws6MvFEVRkjZT5LaLLNlgYGKDgtPtgyjLc6Cu2yca1hNp+yWb1zq+Hvgl2zezFz7lPeALxcTZlhV5/VP3TcvlQsLWqaG25VL36yXPmAYvL5II7OLFi8WoJce2sLAQH//4x7FlyxbpA66r6aZ3vvf9+NjHPiZc3He+851497vfnVM/GSG8OZ04m2nqeyAymmu4q75XNEmvX16WUT+NpPnFHlCztGMKQwzTWeh0xxDetmEe7lw/Fy8daYM2bC2RtCePtsJS5oHXH74gclYZHXOOG9mI9+dY3Lin0WGNagR2qzHhf9wCxshAI3pxmUecGpuTiInKrlBSFWVN7dcuef7taytk16HmgXMyGPt8IXkm/ny0Hf/1Sr22+DHXW0eiiERicCYdIVw2izz/fLYmkrxJA7l3OIg+3+jf1UTKumTyGrx/Lplrn73QC94DvmA4pSsfjk6vw6z+4vOsCUBCv392O/ceGA3fac4lanv//fdr9qRXH3jggfRGco0fpc9//vPyN+rgGDuyIbxjnDJ7aJp6wCiW/YunusAf/8063iw5jXMLnGgf8KPD68e6eVMjK0djOtQ3guuWl6HMY8dPn6/B7rpu3HtVgl/1273NEpLWarfKlPOgPzwhnuU0de241RDhnU5KQyAcEwTWZTMLP3XcBk4wQ1WJR0IMn6VHnCb1+oIS3Y2KH2OlRaVu3LK+Egeb+rGrrgdvXJMwgMc6Rx2jfF2rN4SSpM7vt54+jU0LinHXhkqVJesyEDIhFuAzlHhd8j3HNBKKi/d61hN1B/jcxhCHGSbUdPlEu1qXZXZztgdmZA9cqrJkA/5IyuANJB25Z+QNmm1UTj0wLsKbUynnmckI4b1Uf2Dn2ZVTfjo5TXU9wyjPc2D13EyDlsjXwhKXoIREYfVpssLDks4AuwWFThtuWFGOK1cV4YkjHSCNYm9DH363vwWrKvMwr9Ap0lFE9i7GNN2BJ/zhqKgROG0m2KaA01CWZxdktHs4lEJoye1lAAfyd1VY4Wz3isFN3nH5PDS2eoXLmy2f0X7WWdfZgysWFYujJfN85dGjGVQYPj96aowqy+y0ocCcRnTJcx7wBYS+o/KMt9Q+/zUdl6Z/wnefr8F9Dx2cMKVkvL6dPT61PXCpcnj9wYjMErodFgTCFxbhnY20dv7P+IwweI0QXv7ASJafTTOrB14+0yMI1fr5haM8Rol8MRwskcI2XbQ1eqd/6XdHJuwwpL96OtHRcCguccPlsEobbl1dJUbQr3Y24sFdjegdCuJzN68QY5jUh6EJci31dV6o7elGeBkqNyhT92MjrefaHzSiF5S4xTGOYalV4jrDCs/JH5+vyUAUeYVOQXknEtaXMw713SEsLHbhK3euxl/fvgon6nvxD386if9+pV4cHD/30GF8+Gd7sO1Up2oaRpK6xNwRtblS+/msE/UlBSTXRBkzyvYx6VFuVQYHihwEvB4Tr4u/0R+9VIfj7UM5XaLRDFNOJ85mmu2BSeiB/pGQ0JlYlD8cu6C/TSPFq0m4xNdlEYG4MXlhRhi8Rggv78Ks09rMexZJHbBaTLh59RzDxtGRjEZTh8ajldPJv93TDEZme+Q8o2WdHQ6jpd+PpeV5KPMkDLONC4tQXp6HH247Ix/Ut2yoxAeuXiihjqnioIIpGDZ4hu+cTqc1ohnRmAkuu2XKIogtKvUgFo+lON4MK0wOr8tuRrE77QCZ7bYsm5OHuzdVYW+jF08ebc+WbdT+1gE/+juGQJ4wHS0/+4ZluPGySvxmey0+9V+78fWHD2PrwbPYdbIzAz3mwLs/6hdFEJsl/bokz5mor38CxilRZv42SGvgQEyfiC4T/fzxjpkTZIFtyoZ669s/3jYHNuzD0nwH9jeMDjFtdH5Ep45hlOdS23ewuf+i9Uu42O4VgRtrUj2GkRWHA+lZnum+llmEN/ced5qMHYrTb/Dcy5rNeYn2ABGanXX9cvVXa3RUtd2xqMQNfyiC1v40gjcUjOB4qxd2iwm1XecmKaXqIA+yoccnxuz84oTX6i2r5oDtUc5QH71ucVo9IhRF5xTLpKm2XexLIhiU3yISq3iqk3lNDMNLygsT7yETueB0/qLEnVJBkANZ/tFYffO6CvT1DglvO0u2UbuVEyURZqbKQif++V2X4QM3LcUX3rIKv/789fj6W9fIwIkoq1ZlJOK3CceYAz2VKMFnisUwMJLbB5BlUgWjLM8mKK+SOlLlcUnHy60HW/GH/dMfQlnbDrXOPiDX+X0/2T0pRjidDdkHpK8ca/XmZrRNzWSDusSLbnmibRBfefQYfrw9d6fLi+4iZ0iD+b3TKjOEIjHwWzabLt4eMMZ9p/l6jCgN09yE2eqy9ABlyRTuVts1JC8A0haWlRsL588tTAQloaOYSnVdw+gZDgq6QzSPlATyMc8l0XBo7/XhnZsqU0YtjRdyM598oQZvecsa3L4u4cxECShLnj2FJp5LfRfyHHJ4p9NpjddKg8RumXyVBtWPNDj9oRgaexMGbzQaE541kT/17Ki82ZabFhaLYXr47GDOz1JLvw/mAqfIgqlyqTLCspRxTyRz66E2cbrkoA0eO1SkLxq4pF2o5LZbYbFb4c0R8VEUjkUlHhDp5YeU9anfET+uh5sHBP1kHdxW7VJ1TveS8oM763pxgtrHHYMyaP3ft61MtXmi7anvHoZ6L1ALOSev9/RrZKLVvS7zH2kZwMEmLyxmE758x6rX5TXOlIsimsvvFYEavp8CkZjQ9S5U+2YpDeff8zMC4c1GaTj/y8tewn+8WJvB1cue8xI/opEF4seP6cYVmXJk2h4iekNDk7ql/Gjz77W6XglIQUcfokZHGKv1HBMN5+hwCFXFnowS7tm8AH/7sS340u2rUooMVSUuuFy2i9bg5QVOJ6WBU3ZMU2VosdwFxW6Z0leIK3nDdCrkR4X6trmkJWUe3LO5SqK2PXu8Y9xTSKlp6PGjqtQlyK72BO218tmlEkNTnw9efyaS47BaRGdanVvAICsWE0gDySX1DAURjkZRlu/EmsoCQdFPa3ispHbsb0xM84eiwRQCnkvZU5XnVPsQznQN40O3r8I1y8rwn0m0V8txnkjdh88OyHvBbIbcc/UMTKSMSz1v33AIbrsJg0nt6unqj0vRiZzvJgIsLrsVHodVAs0MTHO/a+/vLKVB2xvntj4jDN5za/q5n/XU0Xb81W8P4VO/2o+//O0BcHs2ZekBzZTia7UJg/e6ZdkNXgafIOIaEUMmCHr+v3S6C5WFnIqeC6I8O+t6slQ2/m5x9rFbRqGBlED75jsvA52aVKrId0hwCiLKNHoutpQtHvhUXQdf8CPBqHC0p6oOavES4SXqr1LXUFACOWiDmKhjRksaqTevmoNhb0AGU0Z5tPuIqFLZg+jqWCgyZx3I8T3b54f6sPWEgYGIRQxhpy39uiQKbrfaoLR1tfUZrXcknz/+NhhxLhyzZBi1RJKofkKjOxy1ojFJ+TAqa7r2ne4YFPm/D1+zED/+0BVC/TjU1C/vzXPh9ZJ7z+vbvKhE0PLd5/EemK4+mGn1dA8n9JupdT6d6VJUaeCgl5rZ/M0y8A0DzVxIf5BZhPf8n/j0G/z8yzrnEqab0sCP7j+/+zKRQfrhn6vx2d8cwKcf3D+L+BrdweSUIj3iDzX3CzK1ZXGxUU7ZZ7eaxROfLwcaMjQcyE1cNTcfX7h1uRhURg47WQvUHGAbyP2kQsP8orTHvCZLxmqxx47yPKcgvBcr92o6KQ3KASvPMXVMp/I8uxg9KmgDnbhIH/AktW0zbuAYG5sWFmHNkmIcaOoHeY1jpYRCwzDoMEc6zliJWsA0pH0a5DYa98Pj1Iz8kig4kTai4pzFGC91eBOGysIStzhc2sxR1HSmZzqI9lLBgf1BJPhsf6ZW8XjlT/ZxDhKPtnixeEmJDBJIvfiXd2/ER69fLIooz59MK1nkUjcNZPYTDQdy7hlV70hbYt9Y51ML3uEYX71jrDJeL8fYf9Q4V4no42yauh7gO+Bsvx+VRS4sm5MvvgbDmvfC1NU8W/JU9cCMMHiNKA1TOYVCNPDLd6zGrz5xFX748auwYk6eBC/4698dwR8OtExVX1/U5R5r8YoE07LyfMwpSPB0jS6IDk/zixNGBQ1bOqowrZ6bD0qZUR+XL+1zQYiIghH5okLDWFG5VLuqilyggUUjmVJpF1uabh1exalUEcWmor8YnIRGDxEqfrB7hxPSgxycTCTRcH3v5oU4WtuDxw6N/ZulQgONWDrMaSkMRvXJcxWKpsTm+4YSIz6n1SyGujqHlAYmDhKs0fFpDaTiENkmwszfAvv6TGeCx0xDhrMe5AgvLzcOzqHqna4laUcn2oZF2o8AARP77oNXL5Rw3o8fajWcNeH7kzJv+kEADfqRUFjeAZyFsVki2F/fDVI5xkrU1f7FgW7hao+V71I4xr5qTjoDk2t/seqLT/ReDQQHUN8//U56Q4Ew2vr9ojnPWRmCN2qgPtFrmIz8s5SG8+/FGWHwGiG806HDS8P3vluX46cf3YxvffBKmW7/xuPH5IV9/l17cZeQ+mAlY9kzXC/TWPxdHqcnPqeGqU9K+gI/WAwAoZyDblhejlavP2UIT6SXyPnjuZxiUgoNY51Pw5ycyTZvIDVFPVb+mXZsuikNjCREbVkGEJmqpNXi5fPBQQw5clQvmEii8cVQw0T76Wg2Fsor6GooivHQXdbPPHRuUzrSFocHfl8cbqcddo0sGZ3WaKDyAxgwjf8aVWgcn10izeQEEz1SvzNGjyNP8D2bFwjPtbkvjeRNpF8mK++hs/3oax8Q9ZMCDbp97bIyoQ09f6xdlBa09XEQ+93nqvG5hw6BdAhtYnnk9a+uLBCj1213yfUrJzZtXrXOvvnWM6fxyKkg/urhw7PUMyCF8EZj8UtGfeZE5wnsOrsLI+HpnfUgSBIbDMj3hjMzpHup36t6RmeXF1cPjP+mnobrMUJ4We106fByuo4er//8rg3oHIzggadOgU5tuSY6Yn37mdOvK0NZIWEKvSIPNxKNg57tYyWexwAC5INWdwzj1TPd8sJQ51FNoaWXBm9C3myssvTHiJK1dvuwpMyVUmjQ59Fv07ueYYhzlY/Sn3+ht4PBxFT4dLRDRRLSclUnu14OiJQWL5F3hhUmr7PYk33WIFsbOIj6i5uX4UB1l+gvZ8vX3OcTtY6x+LvqXEqjFXmcGTrSPOaxmTLCLRe7KVVmFuc2hYyrMoyW5PBSf5dt4ICCKDcHhVRvoNF8smNQeMLv2DhP+O+K82tU1lTv40f9ZNug9NmayvyM6vj7VpKEL57uyjAAfrKjXvSFI5EYdib5/upkvguGA3FcXpUIWEOUm2ksB1b2DQfMTJRr++IjhyRAiFYyTg5eIv84ONAOEPqSfN7X8+XzWWwcaESfr2/SUN5GbyNaBseeFWKfCq0o6S9S6nEIGHAhDd6w9dyUjV7Pz8dEr21GGLxGCO9EL2Qy8hPt/d77LhMk4jvPVedswLb2j+Brjx7Dr3Y1TUYzZlwZRKfouc7Qr5dVFY7bvnnFCekpSg9Vd44IB4rIFtMVCxP833Ph8dKZyEihYawGSbhauwXkcV6MaTr5i9ThZZpKSgPLJ1pCI5HSZAwrzMRQ1RNNNL7et2UBFs8vxO/3n00ZR9py6KxIzdv55eTvjm9UUwGixGNJ6UgPhBOvSNZFdFol9hG1g8nhpbTaeIlR1sxIKFGwLAbQYHmk/Jxs8wp/l+gnKRXkEfsC4dx0aser+ByOcyBCxPyq5WVQdAZtMe++sgobl5TiwV1NKcc75qcBzKl2BpZ49kRHBuWhqdcHizmeKm9DVRE4sCLym82I2H66S1D0GxZZ8YtPbJEB999vPYGvPXo0QydZ27bX8zp9IlTSBm9R+6ZyOZUUw7HaXT9wCiORQdhsNhzpOHLOKC8N3NeaX8OTNU/i+drnsadlz7hGLwEWe55D3lcCAogu74XTycuzjv+eGasvZ48BM8LgzYbwXogb9JFrF+P779+ISCyeswF7POk0Q2WCc+GmXojrnEidis5wxcKilOTXWOdTHYF9QUOZjj3k7qpUmmfH+qpCMUIm2leiZWq35Izusk4aOYyI1X4RqjSoPpvupTai2GTXTWOPRiVnAOq7fRjwh8TwUZzYidZHw4kob2NDH36zu2HU6URKc1FoUCdSGq3QZZfQv9rQxTRCiU6rxGh07KeRHJxY6AAmhmCeI0UXIQ+diVP/u+p7pQ84GORAgOgnOc76IC003ulcO5HZJ9VeLomM3v/YMZmN0u7Xr3P25XiLF5sXF8GIBsIZseuWFaOlfTA1U/Oj7bWgggPDNq+fV4Dt1V1oSTpY8Xc+EvKL4ZCfdIik06Ey+PX1c5tG8HMnO0Ux5JpKC/he/tlHN+P6FWV49mQHdiclEvXnPn64Vfpoou8WfTkzcbtdE66dzwl/O9OVLpRKQ31vvUR/5HWeC8rL3/C2+m3Y0bgDB1sPorrvFAKhABoH68Xo7RnJrhjUNeiXwSe/IQoEIK93oon84wPtByZ62qj83uFMmtCoDLM7xu2B9Bt83KxTl2GmILzqCu/eOF8edDoFaD966rh2yePVHUMSepT6mVptTW2+i3mdHy++YLOFE9ZfG2kENBCYKgqcuGpxWiqMCBanRDsHAxPi8bKfiTwxhLAR6qRvg9qmJjARJ/J/x7uX6pyZtJxOSkM0FpPBwVRf/4ISlzhwkcPKjwoNH04Znmsi4njlZZV4eF/rKKUVcoTJFc5FoYH1k3JAR0cO1gJJ9YW+UDLcssY5jc83dXjJS6XxPlZSyFxZfvoaV1UkpvQ51b+nvk84vXTmyjfHReWE8kdaA4flMzrZT5+vwS93NqK6My3rpuqmYX3vT3fLtL/ap12ebs3dT0cAACAASURBVB/ED7adwf/saTY8X+U91T4o3MXLq9JBOdQxtbxpZYU4rz17ohMP7W3G7vo+LCx14y/fuAK3rkmEHVe0BjqaBsIWbKgqhDOpxsGZIrfdBnKXKV2oT32+IBhCl8/GxkXlcviNayrw1g3zZJ3IsD7RoP/DgVb8fEedoM/64xf7tlLuqCgwC9rdNTy+s+R0XHPIn0aeJ7M+GqM9odZUkQrlzaU+5ej2fP2fcLLzpBjLPN9jzRe0mEsavTSEmVclrtM4fW7PQXQOhsBvGQfBAgLYLWC0tVwS205D+0+n/iTLnY07BV02ms0gN/l41/Gs6DUN5m5/E15ueUlQam17c2nLbJ50D8wIgzcbwhsJT98INt0lCW/kqhI3OG00nh4mP4p8adPxZiRkSk3xacu7mNeHYib5mPEabhoj4IT2GqnFy4haRHmJ6N22Im3w0gGGBi+jpSl+nvbcbOtKoYHT4bkoNKhyODqnQxSntX3B0R9WlW+mLqeT0kBHGLdj6hzWVB+XeBLSZK39PrQOBMXwmcg9VeWoJRHHT924BN3dw/jlzqaMKXKGlc5VoYHlUQuYjo5nu4dTYUTJSaWTmt2VNljJO2bKxWubhiulxhYm1Ut43urKfDH6Sfs52jogCBJnQhjZkIYjJZE4wNMmBoLIK3QKYv3gztFoNp33fvtyPZ470WmoaiBIazAqyOCxlvRHXlsHjUbSjYrn5ouDmfaYdv0tl1Vi44IioS7801OnJBIb0V3+vm9fO1cGGH882CL3gu1mPxG5Vig58/G3yZk0rSazqoMGNAcSayoLkR9LG1R0NKbahQojrvJzSS960ssYbOaVmm4ZtGiPX8zrNJQ4xU7HxjWVCT+K/uGZIUu2r3u/GGKT3b+kMITCaRoRyyfKe2zg+JhVPfDUSXzml6fwxxPb0OZtEwOXxq4+0ejl8ZfrXxaDk5SHp6qfwr627djrO4hTXa0SKIeDYIYVJ71BH5BGXya3ea/Y9sMdh3HGWy2IMutv6G3A07VbMwxsGrpPn3karza/ihcbXoTemCcVgw57/rAJXcEeQanZxslAjI3a/nrfNyMM3mwI73Q5relvMh9YygMx1RggKdr81HflB4upZzicoa2pzTfeOqcrlSf3eHmn8/iBxj54AzHRISzJEYUj1ZEGjMlkQVWxK8NQYNsvqypC3GyeEP2DCG1tt08+krkoNKg+4suqLM8pfUtt4IstTacOr0If+HKfysTBIZ22hgJR0OBjItXlfNI7Ns7HLZfPw9ZDLfjDgbOpolr6RoAcFRrUScVui3DF6aU9FLXCFGO45cxXJXVkOc1JJyJq546V+pPRmbRG/YI8m2jRchaJylzLyj0yOOQAsbIgYQiSiqESDVH2lQS78IfxypmejN8PjdmfvlIvBjGNSyNnMM5EMVG1gqi3USIKzAH8VUtLsag0u5MM23n10sRAlvxcOhC+dUOlFEnDnfd3X2OfOOWRtkEDft28hMOaqpfnkxupH/jyOeSsErWZ71hVpLLLkrM71G1W8lzag6yDkeH4fHFJesPrKfEdyJDCl3FgFItDBaG4kNdIg+xkz1Hsbts5aU5lvB6inh3DHanQ3uoaaTie7j6dFQ1lFED+Dl6q7kZduxPOcTTFWV6br02MSlIeaFDbrVRkMKNvOC79bLGOyIwlZ3Vy0d3u8DXhTN8Z2E32FKLM9rMuosrb6ralDGwaszS6GcKcBvEzjc+mDGKixESg2SYmq8kpZXCbiDERZL2BrPppdmncA5lvceM8U77XCOG9UCR5Xixf5qvnFsg0frYPg+oUvoSIOBCtYCJiNdHEqcgvPnIYf//48QxHj4mWMxX5d9R0gyL5t64ql37JpQ5OWxIpC0XC2KyhM6hzadxsWlQ8IT1eqiwQweNHONeIXKyPhgVfVDQefMGJ869Umy/Ecjp1eMnTZh/xY6roKFN1zZym5gwAExETUgO0xuC51Mvf3+duSQQ2+cG22pRMGa/JXuIWqkKu5c4pcAF2i8zuqHDLeo4xEUqivKQ0jJfaBvzyLiEflu8WJkFyk31AebMrF6WDudB5jcYb3wsq0cg72DyAqmIH7t44T7izv93brA7j0QMtYsRy+pXvLP2UP8siJ5gIPhFrrqsBTqoQQGhGja1e3LC8VOQFtcf069RB5ozLcLcPn715WSo/r5FqLJwGfvpYhwSw4OB3eZLGocrZtIB5TEJd0LaFFIcXT3fCbDLj1ssWquyy5BQzB7EMrctBgDb1+MKC8nJGh4MQBsfQlqvNezGuc5aKaeOChE/EdIcX1vcZjVKimIOhYdjjTjEaJ2u6/VTPKQyFhsTA09dLg4+oKBFZrdoCUc+fvLZdvsdUUGnqcsCSw4QVjc1oPJpCgu2WxAA2FHYgHOvHgdYD8i4odDtzojQ09DfAF/IZtl2hykR0j3UdS6G/NIb51zTQJAYxaQy7W3ZjKNwxqhyVl1SN584+n9X41/fb7PYMcVozQninQ4d3rAeAjjVMWpTFKD+9rPnj4geLH11OvetfxEbnafdRAuiRvWfx9PGOUdqW2nyTuc4P4FjapaquXXW9gsQxlGuuicbAR69bLHy+D12d+cFiGTRurllaIteaa8Qm+YCHohMyXFgXp6FpBPQy6ltSESDX67jQ+ajDO10I75+PtUsQh7s3zQcpAlOZOK2tgpOwHqvFnJMz5Hht4jT7l968CgeOteMnO84IAkoElTrQakA6Xhk8Tt43qQPkTJJjnC3FTUkd3nGCmtBxjpJ+NNS0aU1lgWzy/bF5SWnqEPPRyY2DaZXIA6ZTGGkB99+1Rna/eKpL3jVEd//l6dPYsrgEn3nDUjlGbrA20Vg61jog10bKREPviOjgavNwXUV/U051+uPa7VUVCSrJ//3IlRI2XHuM94KDml/tOCUOgKsq3ChOButQ+S5PGm6M6KYGAjx2uDmh2Uv6Ages2kT0lmAEnfr0VIiajkExsqn1/aY1cwQ5fv5kh/b0i3qdYW6ZOMNFWhB9Rib6rZnMDqBRSlRS8WJpiL7a9Op5DzKIbBLFpSFqlGjwERUlIksElMoLRDufPnUIe2o98j3mrElLnxnR8cejowxK1hkbKcBQwIbi/Ah6QtU41PsnuG1Aj68Lrx46YtQs2UcDnMYq0d1sie1XSbvOferaeD1izMcTdgjKY7CmT5PTmZf9/0LtC2CfTXY6H/SYgyH+zbQ0YxFedtSFojSwbqIR/EgZOYeom6imGYtcdhFj58eI04n6F7HKn215+GxiWpccNHpsT0fiB/K9P34NSoHBqM4mn1k++nSSWF+R+bE2yq/dR91dcvqMorLRIL5zfaUIeT91tH1cVJvTVN99rgZrVpVJWFJtPbmsl+fbxQnHn4NHfS7lTWceiy3NG52qejn44TQ2jbyJDGzOpz1Ly9JGtXucacdc66HR9De3r8QtWxbgF6814etbjwsyw8Gr3tgcq8wFxW4ZJJEzSceVaChqGMqaU+5ES+nsN1bqHkogkSUa6TW2legoE+XNqE+rEvnCdOyj45oyaMirJTq7fE4eqEzx8esXge8NBnog0usdCYuz2KdvXCr82bquoQzKQ1PfCE419OO2tRV42+VzQapSbWeC4qDqpeF8vHVIZN6W5zjooXrC3921dtSsCykOHGjUDURTAScYZU+bGKCG706CCt99viZ1iPJmpJDcuKIckXimxUJEeEVFHmj81WjaT4dUbpPjevu6ufjg1YtERYLvl9dDIupNyhufOT7LnOUKR63oSUYqnO5rpIF1sO0gQqb0oIxtqO6pxqnew+fUHN5DorbP1j6bFd1VBdPY4x+NbNZJtPNofcIRdF5RCMX5ASBuR+eATeTw1Hm5LENRM7yx0QPduHkQvpAV2/sOSzuNDDoitIPBhIxaLnUZ5SENg0oSWoP/UIMNRsqa7APWyT6joT1ZiQb3Ew1PJpzpcpjFUvWyT3guqRozMWW+gS5QC40QXjblQjmtse7KQoeMFMmtzebdT49NTjNyCpge1vQEp1SM9kU8XpfyJXagqV8+ZozkQmQmW33jlZXrcV4TDZzq9mExCrLV1+GLidF/3bIKWE05zA3pGkDDNlu6Zlkp3rO5Srh6v9ufXQScH+GvPHpMPuifu3nFuIEvjOojGsLoWXonIKO8l+K+Z493YE9DH+7eVCX6sFPdBzT2+NHmtDNpAeeiwZutjRxg/d+71wuS+UxyxoS8TyN5rWxlcAaCzwwdozhjY7FbYNJIkqnzlOPaeGGrVdAJyvVpE6W5OMil4aul6ZBCvbA0T1BMOmFxWv5w84AY4auSARvet2WRIOMP7WkGAz6QD0tjl9fPdxH57iqsN+skf5fXQaWVa5aVo79jCNUag5F5OJvz8qlOMYq1UoLaNuvXtcis9hj3k9aQn0R1icoaRfC7/661YsT9+7YaUZfgtTLIDdNt6+Zqi5R1vlM4WCKVhDrOKtEh9UznsNy3lRV5UveaJcXYWdcrlAmV72JdKrSfuuI0+tUArmco0+CcquvTUgyJ/NEpazA8IOiuqpPGF5HNfa37Jow40nmLigqKR6vKHG/JOj12D4JRJw415KPQHcMtlw+gqiQGf9CK1t7M39x45dltcUGFhwOJ711pQdrvw2NLm0vHunfjUPuhjOJI5yDv+HxTNGYSY57XRu3qIw0e/OmIR5Y0grlPm5iPhj8NTQ4Yckk0jrOhwqSGcABBFH1XyyviaMdys+Vnfbx25nns1GNCcxkIDCASzf79z6WNU5EnfQenovQcyzTi8PLUC4nwSkSoEo94V+v1MNVl0QmqumMQRKioobm03IO+4VDGi1jlzbYk344E+1VzC+SjRceUYy3ebNknZT/ljTglSMSIKNEjO88YlvtyY7+Q9ilBpPVQN8w8wZ38wH/6pqXSDnpz07DVJxri//5CtUTS+uSNS/CJG5bos+S0TScgXit1fMfi9P33K/WG7cipkinKNB2UhtfqetHfNwJG+dIaXlN0SVKsQjfJXyUCP5mJswtfun2V0AL8/rDQGbIZZkb1kmPOfiCqOhhIeP07DBz58pLI9HjanCrohDJSVJ1EQX/ykc34P+9Yp3bJkhx4KjqMhMJicBPdO3R2QCT+lCHKc8nlZdhsDhxIRVGzKXwXke+uwvuqAe6C8jxcuagEi0rcMgDU+yfsbegRNPuGFWUZFIOMxk1gg7QGIrjkH5OeYHQP7tpQif+49woJykHD/TO/PiDhw4kwkzJhNNBm9DeWqaVtkGvN6+V9SwxwnPjYdUtwor4XD+9NOzFOoPkzKisD59CZVFFzVCju6YogqdXhrRk6IwYRebv6ROOL/FVRWBhHrkwZSY+efBR7218RA4vn828iKR6xiDFIVY+q0gAWloewaE5QVD5y5fFq6wsEIwgGLXA4MmcXmMc7kjaZyMGloa5SXV+dOKXRAJ+s5LLHhItMpLq21SmGvVHZ7DPykNkmSqGNxaWmsftc3XPY2bRzlBFLGgUd4pikzJhJ7gvLJYrMe0XDmtfNcrjO+qgcwTwdI22Jc002WC3GlBSj9k/XvvTdm64aDeoxQni1I0qDU6Zl14qKxIObLSqYmhJUig50+FEoba4NpPHJj9Pdm+bh3qsX4ejJzgxkJtdyJpKPiDJ1cN92+TxBp7+9bbSuJ43N5iG7RDoi33Yq0rXLyvC/blyCHdXdMi2rr2ProVb8vxdqceNllfjG29YafjD15xhtzy10oSSprWqk+clzOPX5neer8fNXG6YcYWd97F8a2GNNuU6H0xq1Tvk8rFtaCiKO05XmFaU/lkRTJzt95LpFeO/mBaK2MBF0l+1g/uKkwds7HE/rcOoamee0ykCqfyT7i53Gpgo6QWROm2gA3nvVwlT0QXWMg23ybOkMS1lEGne763ok9PD68gQNgue+/+pFQqkgleqTNya4uyyDBiGDrSiDkAEgOLBlPhqRlF1bVZknzmRqoEk+/866xHNAo3gykqI1sKyxtLNp9DKoRL7TIiobjEjHABPZEsuiQ5wW3SSVrL7LJ5QGNbP0prUVotu9u753xg1ks11btv1KwaeyyCXSbgzFTam76Y4gSZSPxixTNsOUKC8Rwq5wt+HlqGlvZSQRSfSNRLKWZ1iIZqcvAuw+XSSDpo1Lh0GUdn5pUH4/Lb3OUdJmmlMNV4kWDwdNyLcBec6E0csl+bz8TfYP25DncArlgFP3CvlsH2oHYpNnUvE6Ogbs4DVQKaJnKP3ONGw4IG3SOr7p89EQZpuJFDMfOdCq/TzGbSbtvVXrRJF5r3hvWQaNXa5Teo3HmMjnhnlsipdkvED/Ju/unMcFGCG82hHleRR9zqfyg7KyolC8q0XaSFcSkUI6UtGQ4nQdE52jiCTyRTwWkqiK4sfw5ZpueSnfsmqOTEVa8uwiP5TL+aqciS758Wvr9+PztyzHvVcvFE/5bz19KqPNRJmJMHFaMlc5som2g31MDiADQ1BDVOtER2Pw7x8/gcpSD75yx+qUB/hE62D+yiKnIE3tA34xHozK2FnXI3JNdKCZDL1e3j9GxCL/2Chx1uDLj57A97edycoTp9PaVKfHD7eJs9r7r1ow5c5q2mshR5MoLx22iAJOdqLR84/vWIcn738T7tlcNaHi+VwSKaShSfknotBGUm35TpsYXt6k7JhRJSramELmjPLo97F+9g3rru/xiYY0Z2TI39XOtFyzpATP/fVN+I97N2Ug8xzgMZohebzkABNhbmnqT9FVlhTYsHFBsSDYyt+ANIID1V0gKqscdvXtmug2r+Pf7tmIZ75407hlMqjETz+yRWa6SFdgO5gi1tG/AVJJ6FRHI1cZgozax7SkzCVL/uN7mff+eKsXDFN8MScqfTisFqH/sF/5PNH4Uhzvqb42BUDR2CWKp4ygbPWG4iHh1hrxXGksHe5KSIDxfJY1XnnZ6uF+TvmLZvMCn6C73Oe0xzE3OX5v7raPogGMVR6P9Q9ZRTKvOC89mM1zxBGNpKX62GZybammwOl8GryTie6yHXXtTqFmOK1u+U73DSWoFnpag7oe1ZdCRzi7K0Ovl98kSqLROFX9zXw0conW0uFQe0yVqZaqbHXdvHauK6dFlW8mL2eEwWuE8LLTLiSHl/WrF78RpUGmGZsHkO+wCaLC/JTMITUh8SIeTXrXPwiNvSN44VSXGJWChhQ5JUY9+ZRqOlJ/zvlu8wVJPti8YpdIfNGxjMjPk0fbsfVwq/DdfrO7SZxI+EKlIc4X7FQl8g2J8h5u6BXP+gd3NeKOf38FX3vsuHAo6XVOBOh80px8h0z3NvX5sgqHk99LJI7OMEqK6nzq5P37znPV+Nfn0s442vIYhWo4GMKRswP44TbjPMw/lZQGDrj4rNFZjfd5OhMdmFTkMb3k12S1g05RfHbUVP9EyuXglYl0AWru0tjQJzoQUWHCP4ZKQ89QUJA4Gigq6IK+HKNtKmVQmowDMHJx6SdAZzVt4u+ShqJ+Px3eGNyFPF4astSkpcyaGpjTaF47rwDdrV5RUGCZfA6ICvP3OJm/d1IZSDHJpUzme+Qz1+Cxz12XRr3T9kbq0qnUQN1iUsoULUPoDPkOASlURtZ53fIyoescaRkdmU3luxiWLf0+CYSkaDFUEmGaDqc1Ugv51+htFESv2JUnKGq2fqMRlEJ5RzIHGpwKJypIOgTznW8i1/ZIo124u0R3mVSwCtIbyOM92+PMSZ6MiKpKdE5zOSJwO2LwBU1yvaQ4jETCUPxe5uU10NA92X1SKAXq/MlakpJBH4E8Z1i47rwWcnzHS2wXjVc6FhKJ5cBjV8v2FG2E5zMP/2j0Mg/pDNzOJalzjfIaUV2M8l2IfTPC4DVCeDmivJAcXt4MTp3xYTMyeMVhralfphm3LElMAfJFTKkhCoLnEmKYL2l+dNZW5ssHgR/om1aW4URzv0gQTcUDQWF5ShxtXFAkntBEwsil5cf7gSdP4NMPHhBHtudOdCT5u9mnFierfQwNu3J+gYSGJar7/PEOXLmwCH/6wg0ibXa+9bBfC5xWnO3zixGhL4/0gr6RsKB1NBImIxFx4sCHswOkDWgTBx1ElJXeLTVHjagNU01peOF0F/bW9wqaRu3X6Ux0YKLWKxOfxZmWaFDw98zE34ZRopoC0UYOkLLNyNBhjWG5J6ozTMOGxjQRTD4rHLSRqpBL4m+aMzN9Xj9erukUJ9qqyoKM80V2zG5B+2BA5Lt21/fgDavKp5XWYnQtNPRp+I6V+OwQ7Q5GokJVYN+TvkFeqwIp1PnkAnNAR330bPdI5TVacoaG4ZrHUrMxOm+y91EthElx34vdNnk/U0lkqpPNZENdjxfHO4+LKsMz+0rw0tG0qohR/cpwquutS8lT0aAajw5hVNZ4+7y+xFR/SX56NoAUBPJ4g2E7egcTv+Ns5dDQVdQF5iGKKlxdU0iQYqPzbNa00UnuLNHOyU6keRBpZlq/cFBmw3gtWoN7rDp5D9g2BsGgMxmX6r5oz1P5tPvOZ53KHWadIsv5lDeZ5479JExmTWOUZYTwXmhKA5vLjw55ZUQSaBQpbhiPne0bkVCP/LCo/XwRr5qbJ+E2x1MEYHmvnulBXrknIzgDP/6UQZoqRIJIDg1eTmHnJVUU3nVFFchzo1rC8nK7yBbRk9s82I7lc3L7yI5xe8c9RCcbKjB84/HjuHbZHOHzrZ1XmHLQGLeAHDJwipohZo0cjMiTpIi7oHWhiCDg56NFyw8rpZYokcSP8vbq7jRilQyBSmR//bwC3LNlAf7q4cP48Y46ULmC7VRpqnV4Oc0rzmqb5mfUq+qfyiXRt7+4eTkYvIC80pmWyDHmjA1nQxhRjRHB9MmV3DdW8AnlXa8NOqEvx2ibhjSNG3Lt+UdDTg2sjfLr962Yk4eSQpfMIPHYuvmFUAoP3KYhSBUDGood3kbUNg+Idvb5PPf6NkzVNp+dpWUema3h7ywSiwl9w223jeIKS5CbhcVCCyHAwHfNRNKzJzrx2xfOCKWKs3B6pJqGMCXVqObhsFkQjcZETo5KIUTfJyPxfcKBDwNxqIiEHEBx9oH1TnWyOCzoGN6HiDOG3v5SbD/pknfzgrIA1i7wZzXAaEgRzV1VtgpumzulwGBkdJ3rNagpfo89kmG0csp/WWUAgciI6PES9eU+LTqqtvfV5OFwnUfUHS5fkjBch0ecoLxZoTut0qA1ivXtncxrYtk0wpu7PcLbrSoJYHNBERpMITR25iMQGkRhmlmhb4rhNtFeyp1pr98w4+t854wweI0QXvb7hUZ4acBSHogoHVFe7cuSsegTIuhpg5AvQ04bMsiBEnDP9vwQbXypuksiW3H6USUavHQg2t84INzOyTYGOD3K9jHKkXp5c/l/3rEe//u2lYJYlTgAt9uNfXtaYY3yB2+McKk2T8aSCgyUISpLeshPRpnaMhTC1mnwgTjeNiiDGpWfjjzjoUwqb7alipLFGQJOK//1bStTWXkPyA9dPz8fn33DMuFxP3GkDT/aXiuapqmM7Pkp0uElX5qSenzWLtdNlWvrn8p1GvdaA38q65po2aQgFLrsYvBy4GKUyJXlgJizJsOBcOr3pM1LhJeJXP+JJL5bGFWQxi6pDWyPGljnUg5npzhLwgEuDXIGY+BMh0qLS93C4332RIcgR4vnF+Iqg6iIKv+FWhqpNLAtnJHgDAql4wKhqPCR+R4lhUKbGLhi8+Ji/PK1RuHyat/h2nxG61TMIRDASH3sp/dcWSWUD21eaiBvPdiKQje90s2IRGMiofjLnU3ybj8XOo22fK7ToO8ZThi8ygFTOXqyD6Y6BQJp9LK21SM81j5fQJy3bNb0MaN2EO2jTi4jsjGE72QbhnQgY1pQlu4Hi8kCiyVhqAqPN24HebxLKjIHB4zC1jlgxY5jZegZDsO3jzMLiQAOpC7QSY0qCf6QWZbq+oiyhiNp+oPaP9lL9nXfkB3XrB7A3NJuWF3l6Ok6t3rZ79GZ60s22V2XtbzRsEXWrFN3wAjhnbraJlbyyjlu4U5plRpEh7ClXxAgIifapKac1BSU9ph2/VT7oKAqlBnSoirk4123rBR76vtwoCnh+ag9L5f1h/Y2487vvyxcWOWFzfOIKvMDTCcxvdc0DQ+2gx9FGrtMkVgiBGoudZ5vHhrdNO6nygCifiU5ii19o7nV5AHyw8F7R+4mUaNzmf5UfcAPlAoDyn1EZ9R94D3gFDXrubwqgTZ98U0rZfBExz3t1CkpDVPF4X3lTLdEuptMJyV1/a+HJY1ZPpMi32e3gPQFfSIiT4c7ar0OZQlqwqATHPRQCmwiiYoONHiJ4hHVmyjlhIbRmnkFYuwydDMpANpEQ4w8Xg5+KaV4w4ryGUkt0QeeUNfA2TdSgvjbotINA29QfUKfyFcmiMCZjOOtg/rDY27va+iTEM6kk/AdwZkabSLd4YmjbSIn+cQXbsCur92KXX/7Jty+rgIP7W5KcKe1J5zjOlFc+lPQyFUgBQdEnIGggc13ylQmUhqY+obIh7UhaglgJOAUZHc8w48OTZxKJ50hl0RjdbykzdPrTQwkSwtDEEPXZJHAD2e6mlHoAlbP8ws9QfF4ieqqP6K+DFZBY5d92T9sxpP7ynC4Pg/uZFgzUheMUFEGp5jKxLaxr2H2Y9X8ERSZClHpcgogRW1hozZNZXsmWnZshlrXU3vXcuylbAhvjqdPaba0UkPaUBqKmSQiEREgPTpW6LZj6RyPTDVpPWi5TmOGfE4uGcqV3DK9wwkvhkhFbDCAE6256/HypUdVgNu+9zLu/+MxvFrbI1xYeuGrROUFOioR+VEIgTqWdWnKHBVnzTfDD6yqyEdVqQuHz/aPiuzGaV0GACB/uqLAKahRNvmyXC6TnE0avET/blxRBq8/lPISpwLEyzU9Ug+DADDRUegv37hceNv//WpDqgpSGqYK4WXgkdBwENcvL019RFMVz66gPM8uBhUjqdmtp0K50wAAIABJREFUFugUxaSHZJA4J1+enWycfQaOYKKBNpFEBzcOPmnQ0Nim5OFEEg2jdZUFiIYiEhBn3bzR53OAR1UZSilSkm4y0MiJtPF88pLyQWdbGrykHXAQTx10o8SBPAe7TZpAFUb5tPs44CU6Tvkz6lOz/7ceaslQVHnsUCuaWwdFTpIzQqyHg/ZbVldIv/LdOxnGKGkxVAlhYCOVOCBiwBYOyAbHQXn57TmfdpjiAdiK46IYUNdlRaEDIpNFpHNo2D6mA5tqby5LGqwMZvHsmWfxVM1T6PZ1ixGrPVflaepvEr1fGoVUfKGaAs/d27QXT9c+jZebX8afa/4MO7oRjzlFwzajHAvQM2jFrhqPqDm894YO3HzZIAaG3TjeVAK3MwAGndAa9KyDhjB5tYoioS3zfNZ5XaQx0BhP0Bns6ByMY6nmZ+spGRDFGDqyUXuYeWdimnVaG+euzFSElx8NcucYYri2Kx2Gky+gIy1DYijonSQ8djOWzcmXKXIl+UMk4G0/eEVE1ekURnF1qiLwJUpjR5+I5ixfXgq+MPUOT/q8avsf/nRCOLDU62SZ//yuDdi0qBiMYMSpOSYiIfxAkHesHKbU+dmW1sj4I+5s586k/bzmlRUFeOVMD7o14Tj5YaNRQsR18+IScW4jfYVOieeaSFfgVHRZnlNCnHr9EWxLyiIRKWKwEk5Ra5H9D1y1UJAiIq90GFRpIggvaRSKSqHON1ryeSCdYcPysmnhaBu1Yabvo/GX70ygukR/9GFxVftpKBJto0Sh0awA+f+UNcvPQotQ5eiXfPfQIKVyCJ3mWM9EE2dxXC6bDG5XGzi88fmjcx4Hglpa1UTrmcr82SgNik7G3xnBAzqsUbnBKHGwwXch/S600odGedW+Q839eP5kp/QLqV63rJ6DXSc78WpNQnWAoAVpDleumiPH1HlcvvvK+SDf95c7G0VHWXvsXNYZFpqJ7wxt4qwVn4+2gey0gr0NfbjxW9vwwJOntKdOeD3cb5IACJTlWrdgBMvmREQf9my/8feBRtt4SWuwcf1g+wE8XfMszg6dReNQA56pfUb2sRwahL7IEHY07ZA8NGj/eOqPYhRSkqzZ/6LsP9RzCIzyNRjxSjnH/E9LM3zhGIj6dgz1YDA0LGj1zlMFMoNCNYe8/DrEPX9GReWz8DEqsSkBcGmd01gQDWEmGpy5JLZb/pLIsv4cHqMmbn1fPfadPYKas3XSzubWEUQihViyYASFrkRbiPRy4EODm9rDMxnlHQoPpZwV9dd8IbdHz9NdgNZkQ3gvtCwZu4IfjRjiKfkejpT/5enTcNtNgt6pKSbVbZQwourCY6QsdA/LC+mzvzkgDhZ0UiKXlPqaNJTfvG6uocFLtOCdm6rw3adP4ZtPncK337MhwzhSdaklp8v3N/aLk9T33rdRXsBEnzg1/51nq/HIvrOCJJ9sS/J3DZwvVFmv1yU/FktK3XjhYAtIJ1FcPw5kaJTQyYf3x26ziAPMeNJkNCyJ4nJwoqdhKDoDEZktizm4sMqHj/eJ/F1+rBkKVptofHzyhiX4zC/34eF9Z3H7urmYiEoDP+RfeOigTHlR+1Rdn7YOtU6EmYOfz9+ybJRXu8ozuwQYSS0m3FyrhHM16hMaijRMXzvTg0/esDQDLVeDDxomjJ420cRyib56nDTmMikJuZS1cWExKDtIo9bofBplf3HzMuGeni9nPZf2nEuebJSGtFJDTJxw2VdG18g6+c5l8Bw65R5pGRjzt6HaSHS3saEPH7lusZTLviouccvA9e4rqsTIpqPfDz9+1ajy2I5rlpbKM0HnVKNZPFVPLktSTojoKikynsMZAL7T6Bxt5IjLPHz+/v7x46jrHsGrZ7rF2B/rvTBWW04OlKKlD6gqjYHyX5z2P1WboBPYLTGEwgnDjqoA2xu3S1HXz78eTrvTUK6L6O1IKGHIDwT70TLYij5/nxiqtrgDDrNT1ve17UfHUCc8Do+E7aUxq1I4WIFozIFCTwCne04jGE8YozzfBgdgIhI9iKLCkxjwrsUfdxehbN6f4HLE0d+3AINdn0SBuxudpl/iqepEW6LOJsxdMAzviBveQCtaBq5EgY2DzYQChIowToOzxEWjU7UmvaQRy0TEeSQYQNtQa+rg8pIVotXLPIPBQZzuOI2G4QYx0pnJBhvCCGOg9dNieC+f7xNnM/+gCaWFcbicdMJzIhAyodCd6YSXqmQaVpRSREl+JCUDx2p5XUd6DwO9wPWLrkcZxlZcmYamZlQxIwzejBbNsA1RanDYBBXk1BAdkDiyp3H0V29cMaq1dCxZWZEvnNAfb6+TJTl8nLKmV7o28aWdLfFDxRfW73Y3iUH1z+/eMMqwUuc29fnFqWHjgiLctKIslY/Rl/ii++krDSI9RWkeOldQOi3XxA/OxNxtci15+vMR5eVHi6gHeXa8VzWdw6K9e92yYvkwEU1r7vML71LfQkqHbTvdiZPtQ6C+KtOtq+fIgESblx8honoLS1yihHHjinKp8+ljHSCCS07gtUsTklza80hx2LikVPISQbqi3I6WcG6Ukof2NEnEOqvVjMcOtWDdvLXaojPWOWtAOsN1y3LTR804+RLaoKHEqXAX9XbVl053/WIIeewy/d3nC2WgcAp50yNzuiKybi6bk4d/vedyeddkzTTGAQ7E/vq2VVlzcLB+362j32FZT5hBB9h2SquR8kFnMcoOZuM583dOX4n/fPo0qjvSM3XZLoe0L/7+iufmpxz5+L64bW0FnjnegW8+dVIAhitXFY1Cd1WZpEHwO/GrnY1CichmjKv8Yy0JXDBV0cLSJM5KxeIxGDniEpj50fY6HEhqOHNmj9+uczF4zS4LWkdCONNejLdd0Q8qGZATy/C7XFKpgYZOx3CH6L7WDp6Bw+SUCF5vWPQGlHvKU60mFeF493ExblM7qXcdDYrByvNoqDLJOhWRhs7CMeLIzEMfk8BCMXaDqJNvlMqvzmcZZosfMdduxL1L4R+6Ej1tgM1zBGHf5QiGzPAUH4fF0YxgNOF8brYArsJ9cOS50BDrR7zeh9uLbkPBUkZVTMy2kGpAdYi5RSFENVxehiSmgc9+2Nu6d9Q1sj1HO4/BZXPBZXXBH/GLoRuMBOGwJnSVw6ZhhAMLEQgUw+Xqx0DQh2icknQlKLJH4bGZ4TUDbYN+xC0DcDucEviBCLlCfBVyrralM3X/iCqfiwMhkfvaNgd+/+pcrJg3jHde2ysUjGjEKtJsNHZfaHgBmys362qcGZszgsObrSsutEoD20WjtCrpcLK7rhfffa5aps8+ddPSrJw3ooX0jKajG/lVjBTGDw8/QNo/PTqs7Qfmo7TNbZdV4uc76qReoylTnsPwklR9oCGrpMa4n3yytySDNnx963GhNhCRIuJ4KSY6Ay4tt+PF010pOR9KFVHLklxtJvY7nXwU91L104vV3fKh+/3+FpD/yih0nE5lWdr7wnVGRUrEvU84PjHEKZ3injraht0NvTLFzLDK+sSP4seuW4xTbYN45ECCe50Lh5cUiAd3NQl3nEb2f7/SkEGL0NZDY5fBDDgVO5GBj7aMS2VdpMQcFkHXsv1WuV8pqRA91KaaziF5TmjwZjOYtfn163wW77thkaD9+mO5brN92drOMsY7nms9U5UvG6WB9ZGzyyl9Gr1U0mB/ZUsr5xbI4OVM5/gGL2c/aBwS9eYMDRONZg5caVzzHUDKwz2bl2Y1IHkukd3jLV4ZDGVrVy77aawy0ZFSJd43Ir78zhAB1qfnTnTiB9vOyGCJEQfbvAEZbOvz5bId8AO9HYl+oHMYEyW68p1h0bgl2scpeVIQzg7XpgzVTn+H7DvScRg0dMnLfb7xSVFMGB6eKwYsqQc0dmOxeOq8jDaZggj7F4H59XlikSJQg9dszfzdZZzPqGt5NSiZ/wu48g/A770BQ133ivFL5NdV9CpiSWNXex7LpAHdOtiGF70voa6nFd6h0wjF+iSq21O1j+K1xmMS0MI7YkVDpwNWkxPd/kahXZCSwWvzhwOpJdd5rewXlsslkzJ2zZYhhEdWwtfzDpij85BfdBBHeh9GU19COYJ5l5UlvgsvnWrD1hMv448ntuIPJ/+AbfU7hPJBY5ftOViXl1UujuXQ2FVItDQih3/MH4qMgHQLqkfUtIfxxIEdQsXg/d/Tvgd7WvZJSRbzzLQxZkSrZiqHVz0DiqLwrWdO42y/X0b69161UB0eteSLiSgenZa+eucavOfKBWN+dEYVkNxBA4hyYTSW/u25GvADfN+tmSgxs5IXSo1ZTuvpP26fvnGpTK1xio7pns3FEjlKNi6xf/wALSsvxqN7m1NauwwQ4feHU6L8C0sZ7tYqIV213bOztlvu/Ts3VeJj1y8F78X7frJLqC7tXn/GdCoRGU5BKn43BxlEfCl/xrShqnDUfVJ1UZqN3vX767vx+D4/NlWNjcZzFuAnL9dLqOhffGKLFPPxn+8T/iC98LVSVDzICG+HmvrxpTevSrVP1T27zOyBD1y9EC39h/GmpWnpwcwciS1y5on+72/sw1s3VKbkw+jURA9wRnvT/y6NyjHapw0lbHT89b4vG6WB100/BEaVYx9rQwob9UlFvkPCLVNzm9SisRDXXfW96O8Ywi1vXZsBatx52VwZtNKYJCeYA9mxElFezrD9dk+zRDI8F6SfikBK2m6RDuEt9STUE/j+0SbSm772x6PSN1+6fZUY7Q/ubBQAhrNbRn4j2vP168eHnDjU4sP6cgfmlwaxvfYo9rb3w2X/sBg9P935ElxFJxCMkmOcQCrFOE3SEg63HZFABMHYIPwDN8DbcyssJhfy5zwkxijrM5vTgRy4TXqCLVYM/8BtGBq4QvIXzf8RzI5EqHaWb4slInDa3dkjVaprsbmaYLF3iXHs638jTOYAovZTsNNYjifarPLKMrnPbAkJwtxxsgOhSB5gWQlTqBCIOXGg8yX0BwZgHbgBL5/ehKLiA5iz6CV0+gdTxnssnjTkNZdHQzq1X1PpUNe7EBzaLNxdu+cE7HlHwT4jX3leZSVOtp3E4UF+QxYjHFiCeNGrYkwzD6kevpAPC9y349DpCjBSHJ3b3n61sdITKSXNvc0Yig0JEs9mbKncgqUlS0dRUIjq0kmPHOszA0fRMbgYHvsmBMJ21A+Z0ed4FdFAvlAxNJcDq+nCmZfRYDoIibZNMwLhzcbh1Tb0Qq3zQ7W0LE94vERs6cX/lTvXjNkcvoR/+fEt+OEHrsBHrl18zh87VsKX0/13rRUDmk4QygFN2wARtw9FDQ0YOt+8g4EF8uwi3cMpWKIVl2q6rKoQcbNZuLREYyn5U5DnSKEnHDSQ1kAHFy1yS+4tJaLet2URqOXJjxe9xJm0HvqUJGO0NpvFgsqihJMJp8avXVYq5/MjPdYHh2ghNYk5Ffm7E71wJ2kv/Eh/+5nTEqBCq/5BKbPH95/F+65aIM8aB1dcJxWGx/SJz48pFptVZ9B3jME2EcN7lizFgoKxX5NXLy0DebrkRiuPeRrAp9q9grAZ0VcMqpvdNcEeoHrFohKP8NbVDE22IkhNu3JRsfw2+VvOlvg7e/FUFxYvKRnlyEcj+foVZfL7uXvTPHkPZCuH+9++tlyoFH8+2o6Tbbkr7mjLjFisKVUZUhi0icBKLAYBRNR+0jH+8YkT4jPy1g3zU6G1OdNHYOSpY+0qa87Lrj4vukbiCNobsL3lYexrfw12ZzMK8+sRCixGMGJBKJwpjaYMWBp35KQqY3e49y6Y48WinBDouwvxSDFgoARkDqxCpO+9GPFeL8YuGxsa3iAor2r4iD9BC7PacpSbM43AVfwciuf/SIxtUhcMjV1VAZdJw5dUg7gpALc9ALKbotGEkVzT3YqjnTGEQk4EY36hOjg0xrvqB22RXE/tNwWlD3pb34eA9wbhJHuKt6Fg7q9ho3EfdwjXNzwcQUNnA+zuatisXoSD86RIoXHEHdIvtU3r8erRRWgb4LfejDNteaJEoa2bCDCN3R1nnwYd/Gp6zoixTLSZhjWNWn1qHegRdJ73fcA/hGhwkWThfeQ6730wNnqWIRJPB+3QlznV2wyWYpTGfpMbnTEF+7IhvDPBaY2XS6SOLxZSAe69evzIUDQoGWVnsiLtECG6c/1cQeZIXdAmZbQxYluR29iQpdHNgBhFHidWTjSqVZJbpK3zYl5nwA3KxlELd1ddj3wsyO9TkbSqSjwixE8erpIm45QiHdHoCU5tVJWIGJO6QA99bWKgkmgsnpIR4vNw08pymXol13A8B6E3rZkjxvSxLuDR47144KmTeMd/vIafvlKP7zxXjTd/72WRoHv8cKvsWzI3X6gQbAMHaKRFzC/34P9tP5Oh60s6A5F+8oQV+qxt9+y6cQ/QeXCsxEEKB0BU31C83Qd3NQry+LHrF2eghGOVM3tsdA+MRWnwOCwyG8KZtPFULDh44UC1u3sYpDFlSzSGd1R3CzeX4IA+0THx2b+5OfV70x/XbhOd57t7XrELX3n0mCFYoc1vtN7nY0jkiOg96ykbHEjro63ta+wXhzr6K/zdW9PAzC2rEhKICTqWMfplVD/ffS83DIO+AUREB8NnEQwDUUsfwuZ2MfTiUY12llEhpiB8PW+FGLswIb/8UaEXRCNFGO59E0hN0Bq9vt7b0Nv2fniHlsLmaENe6VNSati3Hogn3r+mWAliMadweEOWdkGEvcEBaP/0TaGRyT+ivaQ5TDQ5LBEMhYJirDusiT6MRgrQN1gJu7MfAd86hPwLYDaPNv6y1cW8I97NiPo3wWIdEEO8qHR7gqahBgKWtOJF2BSE29UrxbEu0j383i3wdd8jgwMauk7PCYkS53JEQM1elWjsDgZGsK/mBFrPvkHOVYY3DWfSLegkqJwOqYrBdVJVSMGg8R2L2hP3y+wHYi5Z90dsKQNelafqnGnL88KcaWwxEhdHofopOx7T78t28UYIr91mveCR1lR7KenDl+qaykKJjKX2T+eSU+3U5tWHLOZ0FtHIVXOdhsLrbCPvw/ffv1EcsSZq6ExXpLXp6ksGC1hQ7Ma+xj6sSDoXknKg0BNOGzJEKZ3ZKC8Gjx2MqkfO7pWLSjPoIGso9WSySnhW9bwT4WNkrAUl7gxOIfnDRNkpWTfWdCr7gYb0B69ehK8/fhz/visOq6VfKBJ3rq9AIBzH08c7RG6OeYnu08Hx5qXpjzMVHj523RI88MRJkLvN8MWcaqdwPjmFX33L6vP2HJ+u+zUT6qEe8niJNBUOJviscNaA08qcAbhj3dzxTp09PkYPjEVp4EDyq3esxruvqBp3EMkq5PcqFDCfYY0cEJL3ysRZGKPvFwc2E6EmMJIijcyth9tw/2PH8aMPXTGKZmTYmOROpfhSWeQC6Q1aiosKM9w3kkBX6ajGQTC5ve+9Yn7GTB4d/K5fUS7GPvnJNMRzSWz7jupBLJjTC6enVk5JcE5DYpTZ7QFEAosNebCSOWnskkbgsIVgL3pCeLPxcC28kWLh1FqdjXDk74UpViwGMA1H6l67C1+TvHQ8Cw5vElSTRh6N1WCgXNDQvthL6O6qRyyeiTCzbrOpDQ6rBxV5hci3FaaMslyue1SeuGPUlD3zRMOFCQTaFBBDmM5wfs9ROMzBFDo8qizNDku0ROgJNB6dJU/BXXgAYXKKjWgWyfM40ACWChdZUxSicT9K5z0s1I1QeB68Ax+WkMlrqwaEs0unulebXkXdwFxBk0mLKKr64ai6avtqUzQHxTNOqF4EYYpp3mdmPxxmF6yxYsCAGnIhKQ3aftGun7PBe/DgQdx1113YsGED+vr68OY3vxkPPPAAXnjhBXzta18DwxF++MMfxpe//GVtfTmvh8IXDg7XN5IGyhNfuFG0NI1egvr8U7HNF5a5wDkqZDENnu7hoBgwxWM4o41nZE1Fm2dimURJGNKXBi+nnYlg0MBU6An7iSgs+bZET5heYyStQFSmOLXOR2vnFcJhiQtyowYG/EBRxk4rEs8yls/JxzNfvClnPdY71lfgf/Y0Cbrz0esWgzq95A0z3dc1JAgvP6I3r5ojaJP2Q8g8H7l2kYRSpe5wx7Ya/GBbjTi4MNAA1RlmU+49MB7Cy5LYp9TN5rPSPuCXmYMv37FqFt3NvZuNcxpPWqXykrLFv1wSKQAMpU2dbT2Pl9uUgKS83OdvXTqpwAYVduiHQV3ff3ziJB64e31ObaYBzgErExFs/W+c2s4MwOFLRlrj7B/fCRxoUTpNm2ik371xHj7/m4OiNKMMXtbBgEVEv/X+IRzE76jpRihQDE9xg6gZaB28rPYumKz9iAarZFqeagf9I1F0DnvhsVmxoLgAvr5bhZcaRwCO0ofEoDvbP4iekdMocnTCFvmQcHrzI0UIBErgi7YjYNsFh6seIVs3BnyUB/TAkXdIDF4alHbXWZlK77f9DH7TUzDDhjJ3meTTXjPb4Q8PoLE/QWExm2ywmc0wm13SPpWX5WvTcNAnZRU6dNrXpiDKirrQ6XMhECyBxwNpB2kZrsJX4cAy9A0vhbNgFZzF1dBEZBbKgcuWQFFTdZmCGBpaiWhwPsjZdTi7EYtl0lZSeTUrFkcTLObNYmi73LVCpaDyBM/n/WCKoU3Q785BE051AmsqwuJU1tw/LP1tI6c46hGElgMKfVLyb0KZ4MEkBzkStcM3vBQmJBwpSSvxRAoMHQcvJKVBfz1q+5wNXhq5VVVVeOyxxxAMBOBwOlF9+hQefvhhfPWrX8X6dWvxr//2HfzP//wPPvjBD6r6DJfZKA0zQaVBNfhCG4wMTUqU90ynT+TKlDMSX9Q0et92uRtF8hSrFk/Okuj9+D/ByalrukrZvLgU1MekcxlRXIX8qPqLPXbZz48UE1EOorZ61IcfkQXFLtFbPtMfxDqXQz6kZiQkyVR5XHKgNJFniEb4Hz+0DHU9Xtx+dabEGI99//2b8I23rYUlqceprYvrrOtHH7wCnOLc29iHvfW9YsQT/b18wThTkPrCLuFtBv7IBeHlzAGjFzKgjMtuFsoIJetm0/n1wGQGvpEAFAuLxfHs5682yMwIUWJy4r/19OkEF/6aRfi7u9adX6N1Z3Mw/Z17LsenHjyA3+5tlvc4JSr1spRaMIXOZV98+JA4ypLORudjfWIwlIokv5zfAcqgMVEDWA3gtefw/cWIdFSBYX7OSDzw5Al0eIM43T4oRrWWbnW81Ysnj7ZiQXEEl5V34pWRqERZY5k0zOgExunz/mAnuryHxbhU9fHVORQcQnloKaKhuSio+I0Yuyc7WhEIWmGx2TAQOQWz8x8QDdvR4QcsthBilsQ710+8K4l59YxQpeBniNt/C3t4OawDTfScwIgpDLetCItKKlJOYqp+Or2tLC1CMF4hclk0Yn1JEC1Grq0/lkKFE+WrMxNL7nPZvFhevDCFDJN+QHIB+brdXhc8JUhN7xOlDkcGEPG/B5HQHASCCcoEnetIQ1C6wsLvjTuEghEIBtDZPwKndRustnqYog0otGqMbFIaDJBeGvyOqh9LQ8krthMBjwdgoqRbMtGIDVsPIB67AVR0qBnegb7hIIK+q4RDTQoJKSGCmOcfM6xHlaWWQtWIOWGKu+BwtcFi7Re1C0dgIRzuWsSSvGaVfyYuz9ng1V6Mx+WR0WdtXT1Onz6Nn/3sZ3J45cqVeOaZZwwN3pGRhLag3++HojRwRMmkfvjk8KqpYi6JonGUy6kdJrWuKBXcH7FG4ba5U3kko+af9hzuTiBz6QzquNrDslUeta7ysC71Qlb7eB7Xjdqrjqn2qzqMlqoMHmN/kDtKOgKDWRBFpMHLcmi0UaGBjnWqDWqp+mUkPDKqnTzGpPpU5UnVq/hDkSBC0QSHUXv9zK/6Wdvnqm51TVLeGPXz2uS+JdujttX56hr0faavR+XnUl2Dum/aNnCdesVzC1zyoi8viIswvWoD659f5JCQou0DAbTkj4hO77Iii9AUWL62LVRUaOrziaD9kiKr3A8aPERjtfm07VXXpJ5tVTfLVm1mHqfTiWJkyp6p426XA257+jmXMnTPPiXqGCHqljWFQHgJ+oKQMJzcz7pV0vZ5tnuZdX+SzqSuNWs+XdvUs8Slum5tH6n1bOWp42qpytFei7o+lUdtc6ndp9qi3a/K056TukZNm9W5pR6HTHXTsdUfiuHDmyswx5n43fA8bX3aMnl+Lm3O1g/687PmG6P/1TtAnqHk+061Vy21fcN1VQ/X1Xlcz+Va9GVpt1W5qb6OJ55Tta3ao70/2vazLCaVL7kp2/nmOG5bO0cMXsr4MSTxPZurRAXhV7uacPvqAvz92xPoK+tTZejbpPaz7GztUOeo7wDf199+23J89pEQfrCtFtTNpmOrSnSWJe9WhZBmcCPKHlLp4RtvXStghuoDdU40GkdZfj4GR0IS2GbroVYsyjfJOcyr2sZ7wnZUFblw97oSPF09gA//bI8Y0xykERGmFBuNcfKWmZ9pe3UXTjX0476bSuEsL4Gzw5kw1EIBQXFjsQYE47sRN3sAP1BeUIb5hWWwxfPQE+xGQ3cPOvEI8pztKPfUQhm7S8rLUOYoF4rAiD+Cbl86esOCYs6aRRCMWmU5HE/IyAVCAbR2J4zBEX8YbkYQjH0Cle4m5Fk7EEtGgDBbzLJuM9lEGcIBBwqshZjjTmj98rocloSmL9clWEU0LaFgMyfgnaGwV9pfE6vGkuIlKHAUwGQLQKmi8/duiZYIum21ehN0j0gJfPYARoaXwpF3HETAvcEE4s02m6w+EGWmgc/EbdiaEDTZEIuF0dXtgcnaJO3Ld+Sj2GNDOJLIy2hyKiUQWb/wlZUhTyR7cfEizHGnlUPinnrh9ZK6MDDyPMh7JjWE7TXbegVZptOZLa9B+kqVr+1LtY/LMIZEeYFR6EZsf5ZD1tgNCaM/2a/caYslguTYfFGE7Jm2Gn/ODM2VAAAgAElEQVQ76tvD3y2f0Vx/v+r55zna94z6Par9bEM2lYbE10aaPrF/FRUVaGlpwe233465c+fib//2b0cV4PF40NWVCMWoPdje1YFv/uMDOHz4MOrq6oQaQXS4Lzn1wLw+X4Jnpd2nLeNSXC+yjWCfN4DXDh6Aa7AcRJ8O1vbDnudAZKgb+/Z04zwi4o7qUqL4R44cGbX/Yt9hNQPOuF80PCsKTDh5uhrB/Jj0HY/FhwdF1/OZPUdx4jQw4I9hdakFO/YdwSJP+sXDfgh7B8XA2XXkFMri5ThY2wFSPkd66nDgcPeormL5ud4j/gaCwSAiB/amypnI+TxJn1+/nSp4dsWwB3gPvKdOobm52fC42sl+nWMeFMdWh9WC/NgQjhw9mPO9VuXMLkf3AN9DBw4fHH3gHPYwzMZXr3dhd3sUp3sTwRkY+fLKhUW4eVEEg43HsKv+HArO4RQ+I3cvi2ArPOj2RWE2JZyRGDyirntI+LWUM+SAic6wd6+w4W3LgLoTh0aVzrL64YAbQ2gJWPCfzx3A6Y4INlfaYO4+gwOjXz3yLlie75PZKsprcnbqnrVurLD24R8CNjxxtA0Lrd24c808nOgcwe93dWHNkmJwQsganIc8Uz5O9NYKkkujcUnZYsR9G9DXuxEb5y3GTSXNMqBmY8+YyvFo5yH0mX+OwooBtI34BNn9yMJ3YX3B+vT1FEF0bbnDbhnbh/6gy46d9ZUotdnRO5RwDPvAnHnjnpeubGJrx13H8eu2X6OhvwH3VNyDTWUbsSdkxvFIF24q34jSwrV4qrYC80sG8d6yO+U6Hu3tQ2skjkBoCM19reII5rIVwR2/GSuKHCj3pBVC7LFi7G+8AvnOON64wY1AzyEcjhxBTW81ugf98idGMYCv938N8QgN4tH8c9I5PLYyNPY34UbXTan+DbljeLQ3joFIFbYU/xWGvF7s8M9DcX4AGypDeLnGgSLTFtxbVQmqQIyXaHTvjO3HEcf/gimWaIfLBcw3fQTvzU8HF2kva8Orna/h+/t+hA8suFe+YeOVPVnH+bvgOzubSsM5G7zLli3DwSOHUJpfjCf+/CT+6Z/+CZs2bTJstxotq4OVc+bi29/9dyASxPe+9z14vV6sWp32KGW+Y8eOgY5rar++DObR7tOu64+petVSn1ftV0t1XC3V/mxLlU8tmU+7brSdbZ+2Dm0ZXL/G24DHdu6DtXg9rrhipdRhOngAHnsIW9atwJblCW6m9jxteWrd6Lh+H7f5kbn88svhdqeVCYzarc7lyE2hA2qfqlO7HOuYtnyVTy1VGfpt7leIs/Z8ld9o3839p7G75TSuXLgQt127PmMK8FSsEc80noSruBLNoQh8oTbcetlqvP2GJanrU2XHynuw7ew+tEXyceX69fDvGobD6scd1109yjFM3261rZaqTLVsbW1FW0sztlx9rdqVWmrP0fZ7KkNyReVTS/1xbmc7pu1TbR3a/bmer69Du61dZ3naurTH1Lr2+FjXk8qfnI1QedV+ta2vU7t/355dqFqyBHxnGZ2n3Ve5cgRtDx8WhO3tGu1tbR5Vtnafdp3H9dv69k1m/49Xl2qvtEHTj2PdA1WmWo7X3mxl8TyEIYPua69N/wZUubn0lTavuhYWxf2ne/34r5frcbJtENSrJa/VKL9+n35b3w79ce31bbkauC85WxkwJQy8cDSOuq5hkE9L+hRnCe7eNB+funEJnPGYIM2q7dolndReOGvFvtZmHGpn6GEb3n7VSlx77TJttoxrKlk6jGNDJ1HstuBrb1mXcsA7m1eH+x7cj9qh+Vi2ai0ODZ/FwbZmfO6qAjS4H8NP9z4sxhYNXSKJdAILIwiLJQhnfBmONxWiYfAQ3IX7YbcOw9f9ATgD74bbVY+G7kSo4ZWVBag2VePowFFpH8vyh8xw2RMgAlUCuI8II5FaldT+QMgNs/lj6Ak6EYNDFByeGHlMZctYMsiDljPLclkOk1pXdVE2TZtUeF/uW122WsIWP9j8R/yu83fC/7WVXo9XO4BynwWByEcxEN+Hh4f+KPzlhngPYAG6eoFi51K4Q2+Ca+RWcWjzdnUiUvEQbM5mUXIY6rkC1uD1MOc9iSO9f0IwRuqDHavLV4PR1wJRcmvLQBT38uINqBupw3AwHXyE3GNGdmMoZdImSCGhgb46ukpQbbZ/2HkTokObJUKgJToPDrsXMc9T2DbwEqL2D6Dbvxq/aPq9RKRTfaD6hdvqPnSNdKJ9sEMGLXYsxxLXFvSZtqJ7cDuOjLQg2OtC2NyPnqFeMdR57g0V1+OKK65QxWYs1W9ELbUHtb8X7jfKY7RPlUGgNVs6Z4OXRpAyhDZu3Ig9e/agoaFB0F5V2cDAAObMmWP4gxXDyJ4wpIw4vHqVBsLW+qTdp11nPv229tyxjmnPHS+fKlPlU0ttGfo8atsoj/aY/jjLpuMa5ceop8jUFYiKgwyjh+U707dS2w59mfpy1XH9OWrbSBZIHdOfq4zdbHXo86tt/VKVr1+qfGq/2uaS9AqVjI7r9737yiocax3EnesrM4xdlkGZskKXHU29FKn3CeJy3fLSUcYu86pwpnSEIR+YyInTZsHistE/On0b1LZaqvZrl9kirWnP0fa79lyuq3xqqT+uzaM/pu1TbR3a/bmer69fu61dZ3naurTH1Lr2uL7N2vak8mueDe1x7bnjlZntPFUHj3Pa+tG/uE5brKxr86iD2n3adR7Xb3Oftn2T2f/j1aXaK23Q9KO2Pdo8XFdlquV47c1WFs8To1dXgSpXW5c2i7Y8bV5tHu7fUOXAf9yb+UE2yq/fp9/Wt0N/XNse5k1sWzJ8I0qXlIypz61tu1qnA21B0lGZEdfoU/COjfPV4dRS2x5y+3/9yatSx9QKwyYzqufzZ7yi9U1OL/m+N62ej4hnI7YWPI6y/KoMrqwDTgRtNXDk70e4/3pQiYFBIpzOfnEwszsbsaLSB7+pAJZ4IRzWNK2A3NYgCQKmOIIaSgGNUtIMHBLEQrUOEmDBYQ3C4mhBMETFBcDmbBCDWVQceA6ltRiwIsl7ZaQz5XSljF2WqNbVMl1LYk1rADtswOWVqzES8aLfF0af34uoaTtGQgmjFnnbQKVQU2sC8XQ6HDAH3oSC0AdhC7lFScKZfwDRSDFCvnWwDW+A29EHRJFQZ+DvxV2DMMhTTgdvoBKGqGGYgmLU3p7/ZjyHZ9HjSEiSZbTZxHAfTqFe1PTW4XRntRjqPJ/BK6jvi+BaxMwB9Fh+jZGBxAAE5p9jOL4cnkiFOJ1RcoyyYuwX3h+mcDyIpr5OQfVJxyBSXRS8D8h/FJWe5TAFStEV2o6j7QmKBlFopyOCVfl34bblf2NorLJc9UyqpfZ69L8XozxG+7RlZFtPW0nZcmTZT1rCyJAP7nwPdu/ejfr6enz2s5/F1q1bxVGN6g00gO++++4sJaR3Kw5veg8wk1QatO26kOvkedH7n4gER/fU/FQKDXNz9FSeaPvJRXu9Oa2xD7K9+HmM/Fv+6HbW9UqENPa7UkjgcW0iL3BZeb5MSe6u7xNvbPLi6AwzGYm0ldk02wOzPTDbA/oeIPexMvne///tfQt4XFW1/8o8M0kmmUzSNK+mbdIHfdCWlvQBBQpUFOGCUvlUEJWLj88/XsWL8ImAFxFF5Xq9KoKK4hP1u6iI4kUvj/Is0NoXj5bSNn03SR95TzKvZP7fb0325MzJmcwkM5kz7az9fcmcs89+nd8+Z5/fXnvttbBv4OwZnpjEVp822TnGQ3gWvPGRLfTgc3vYcQXsgV993kJqPVZJl3o20osDL5DTObIxCmUyMat8golVz4k1ZAs1sm5oZMhJhd7fkquklQoCZdFNUSPmZCk0FCQ7RQmWtm0gWtjgpQ+KuMIagcUX3VQI97/KwgD0dEHUFNkFcQMJTDsMk+ciaxWVlPdSreN8aj/yARqy7aNwQXSTYKToabI5jrPOLdL1tq+iYKiIvbkVl71ExZVPUMjfwFYRwr4LKVjcQrBgMBSqYO9sZPETbNk6R1SJR5rN9UctIiCSJwOazWkjCaN9MaeiiSXSUMOoKa0m+1CYfLZnOFmf5QUK0h4mpNBLPtnRxGR1R+f2mJWLQuuIBBkSZqhJIEBtAgTXd2IV+UN+1k+GjrI7cjlZAtdSoOxrNBBpoYYppeQKraKje6+mHxTsoaaqJdRkIKzUtjubxxMmvFs2baa77rqLnE4neTweuummm2jt2rUEcfLnPve5mFmydevWJb0fIwkvMuWSlYakN5GFBDMqimhutZtgagq2Prv6gzELDdiINCkBqz2TVPSktDcDhUJSAlfAbJbMZqFzmsoJRu6NAmaas6eWMOH9v7fa2IGF3iSZUb5U4lIxh5VKOZJm4ggkkrBPvETJKQhkBgGMPTC1pux+wwtkOuFd86vpXxbX0l+3H2X36sp8YagviSOFiJOllvb6X1EkVM22dGG1APZy/YEC3sSmpKZRQusgT6mDfP0RQ9u2SJPIgQHK7GofYJNctsKDrJ4wrWwaLZiygHad3EVtfW3U0R/vCCgdTPR5QVQL7UEKBpcSyJPD1UKl5ZvI5gjQQMBBFmsvFZZupGDYQm7vq2SH2+OIk60YBIvfJN/Jy6nj5NlcLKxXhNz/pGJ2l5waOVfEX98udY5Nd1A5AVHFHySukOSqgA2DUGeBRQVvURFZeh+iDvoWnejfHrO9C5UGBEwgcAxdbViXiIQK2fSaLzJAIOkI6Ofw8bXkLbmWymofZnvMJ9vOpo5gAa1o9I7LMhEXOMn/Jkx4YYMXf/oAXatNmzbpoyd0niue1ibU+EnIVFVayNYa/rShj072BdnhRN9xH82Z6o5b8sxk1UYqDZksP1fLggkjZYsXZsy09nf1bYbVB3wk1u+KbtBs0Pm816cfz7kQrvGglfm0ImHPPKZSYuYQgLc1t8tGs6pKUnK+MVbNmOjD4c3f32xjm+OXnhl1MpBoA5C2LCa0ESdFLB3knvpo7BIsNmjJLojsmplraHr5dNp7ZA9t7d7Grm1VBkV0QbQSqRzAuQJMf8H1blXRNFozYw1nn1I8heAd7O2j79COzh0J8yMxSDVCKvVxQvVvmOhZLQGCpziY5oKkGWbaIO0GkYQnt4q6qGRUmRWD/WJ4UIOtYfaqZgmwZzWoM7CXOQPzY6iSrUioupNIeFUy6FefMXUum2Oz24Js6cHXvZCmlIUYs6FB6Ob2st1euAeuHfoCuSq/yTrD0BdGgD4wpLp1pVEVmcCQnyyhUuroi7DKitUW9VYIKS+8zCFASh3xzSFf71yaWxuk5ZUjkmlOkAP/Jkx4M9l2I5WGTJZ/OpUF82MI2OAAD2vksE54Get0wiXT9wITcMUOK/X4hwiEVq9XpK0POr8I/tAQS3jhZS1TQQhXppCcWDmYcKRih3dipUsuQSA9BDD23HvVIjYtll5J0dwwY3j/tUuprNA+IekcCKSWTIYK+qLqDGzm0kEXTl1DjRUNRBShOfVzqaluFm1v20Ztve1kt9vIQlayWWzkIhfZhs2joWWd/i461H0o2kjXASaV09zT6KIZF8XddrHNTcsaltEZ1WewV7FDvYe4Pfp2YUMbgtflpfphUoc2HOs/xkRZqzqADWSsT6tqsvpo0D+DyDLAEk7Yp9U65FDJ9L/AwlvxT+psa6RwsJqdTcCWcaIAHOHhDBvSVEgm4cV9YcMeVDti6idWorKpe6P9MFwQ2ss2fF0trFrhCM8np2sHjXK2MZzeZQtR/6CTLIO1ZLW18AY1Jo8WP3ncsDzRRIM955E1OI+sg0FaVNNGs+bE68erezDzNycIbyKVBjOBydW6QcSmTClhrz2dviDV15SyK8nJai9s252OOrzJ8FI6u3OnFtEU99jLTZCuFDls5A8F2QXw7KropCRZHcmug2ilasIsWVlyfWIIyIRjYrhJruwggIn4NctBIDMTkpWn1A0ggQWxUhJSbe2KTCIuMDgspYwQra5vpqbqWTRiUjZqY3lp7WKiyDAVKQiPOh6MDJK1wEo9oS6W3u7r20cum4tWT1/NLnO1davj0sIievfsdzOZfr39jWi0lchld7HkcmbZTKp0e6nU7iFV/mD1IB33Hae3jr8VW94fCA2Qp9BD+IUVBQRIdRHgPa7A2s3HqfwLQD/ZtYPsrj1kHaolj2c3OdhCRekoko3yYtgO6z6H7YNEQwHC5jIVYmmGIyAV18fxpQQSZNxLKFBL/b5GchdvS+o8Ak43oOULAuyw2yls6yEf7SW4gx7suyhKiL1/IvK2k8VyuWpmzvzmBOE1kvDqrTTkDGImN2Sa18WzeXjiQQC5qkxCyNJqMs8uMyexTKstWcwMV55Y4oPEo8JhtJtgpDFQdwDpRZ9YCiysVzdyNb0j6MhLMA8BSHhFl9o8/Mkmz7+J6HPVWiP+3qJymlkyk6Wvuztgk3cgpjoAEgxCOds7i/P1Bvuo299NHQMdbEFgfsUCJq4gmNowOITxdTguYnyMPJDeLp++nBYFF3F2uz3x5pJomURLa5bRrMom2nNiL9W66whqDyDPqg36X1xfUxxVkQiFQmR3BIkisJt8nN44/jrtOR7dqAbiVzW1hQrsyQlvbJJQUEg1xTX0rpVdVBDeSV73HCqwVXHbIMGGdzfgiQDiGrM4Mfz5sYWsxKTZMvb3SIttomP0FVmHKFi8nQa6o84jxnJrzBLhwXgPnX5/IatzQKUD7pUtlkIash6lUs8zNDh0RqKqTY3PCcJrhACsNMimtdHIwNwVNkVtPRidZeK8fNg0zejU6cewJ7nE40r6FeRoCXCxed/Vi9nsmMM1MqM2ai6kIlhaBOG1WgoIkxIJpwcCkPCKSoOJfRkeWc41sRV5X7XVa6UVtJKWVF1AZcPD2+LqJUwEsVkMKggNFQ1UXRLV+wWRVMQSerUgqwiKYE4UUOQfi+jqy0V61A3iq+pPtQ1cTyT68cN91XkqaXfZQfpbRxSAwNAAuZ3trL+r1Dj09Su3wpASN9c0R9U5IM0eVlMYHHJT87TFNBi2MTYg1gdPHqQ2fxsNhAdYko0y7SU28ljKqbc7apIUcUrHWen5Qo3BULqraRTSTnVVxyYlR4a6aJtlgDejQd9Yq8qhycaHQ2EPORx+VuNQ16DOYXUO6ysPuchd8QyRo4NdMKs0ufSbE4RXVBpSfyRg7gqbooKD0XUO6ItiM5uEzCOg9SufrHRYz0CAD/tMmSRDebKkngz5yb0uEt7JxVdKz30EQpEQnVd2NvWX4ZszSMOfHm54tbuSiSAIG4KWTKpjRXbNvFPVlom2gfMPEhPWek8FHWsnclpcbO/WF/BRb6iXpbNwmqHcIYOAglxCIr6oZhHb0x3EjjFIs1mSHW1NMDQi1YaEGX8Oe4RJsGpv6YlSuqD+Yuqv76L+gJ/6g/3UFehk3WZf0MfkuMs/4sUN+ZRkWZUBMjvDPZOW1y3niYnVFqYmj432t3VRt6+Qzi9bTK2BA3FSe+RFPvtQObsQLrB1xtQ4mBwPOchq7ybYXIaN5OKKp6g74KDyQg+V2DOj2qfan4nfnCC8iW5ErDQYIwOSiw1V3f0h2bBmDFHWY8+fXUkL6spo9ayKrNctFU4uAiLhnVx8xyo9X63EjIVJtq/ZC6JSTl7i16kRQHUgqpMbr6aQ7TZmoz7cK4jo9KoA7T7aTpcvLaHFM8+hcLCMeoO9MWk3TKNBNQHm0pprm6nUWcrNGw/p1pJg7b1h8gC9Y22Ag5YOXwc9u/9Z2tOzO2aLWOtpDukXVCykc+vOjRJvSOCHC6n3DlG3j6jEsoauWdxHu48dpDePv0mtfa0xKxGD4VIm+LC7AO96dqqkUpeLLWwMOQ5QWe0veBMcSPDCyjNpbeO7qMhWrm1mThznBOE10uEFOqLSYPyMNFYWU3WZkwlvZYnouBmjlN1YeNn6n0+vzKh0F3cgZsmy249GtYkOrxEq2YkL6/Q9s1Or1CIIGCMAIrpomp/m1+9l1YrOPgtZLT1kLbCxZHaWewqdCIQoGAlSsaM4ptZhXNrEY7XkGWQb6herQb5bwrS/dx97mMOmQqhUwCnH/PL5bL0CNaq8uBeop4DAb9hTRAeOOal5Th81ehvZbNzmg5tp87EtI1YihmqpzN1C04vs1DzrPVTkLKSX9m0gWMIIWFv5ZhZPWUJn1S6OeYkjyq39PzlBeCfe7fmZs6mqhDwuB5UV2SddwosPztgarPnZB0Z3nUlVBqPyJS77CIgOb/Yxj6sxD/cPxN1/jpx0BwJkL5MvAbqjwDZIdo03Ju2mu+4hYvJZWFAYI5aZ7kJFWPXlljpK2Cbxk3sGqH2gjaiAWIVh4ZSFTGD16XEOtYa6igCV2gtp91ELbXonwKSXwjYmyDAN99KRlzhrV6+HPG6i5jkLYmos753zXmrpaKFNrZtYdWNV01Ly9YcJqh2mhgJjG8A5QXhFh3d8jwbcQL5rQTW7kjyjJrpcMr4SxpFaPjjjACvzSUWHN/OYjqdEkbCPB61JSAtPjxJMR6DM6aRumF3UqTRMtGFQj0DIVHkTbcdk5VOkVP1OVj3ackG8IVW+YPoFtPHIRj5url8WM72mTauOoXs91ROiy5pb6X9erKGXd5ZRU42fvO4wgfSyyTgievGdHqIhFxXbe2hmhZcGgiOSYkiEo7aVo6UGE5BNVWdWfiPG+5pygvAmUmkQHd7Ej8ZXr4j6E0+cIkNX8tC1cIaQk2JOAwQw4RCVhtOgI+UWJoxAyBFVm8sEOQXRrS2rJegFH/UdnXCbJGNiBNi02ow1rDsMwj0W6cY1h41YTePA3A56bZeX/vhSBX1s7TFy2MM0OEhMenvbBunp/YVU7g6T3VZAff4R02hc/vAmPLMFu4lRiV6xJEuQjeuJJLyiw5sN9KUOQUAQSIQAJLyyaS0ROhKfLwhApSHdoDa9XTDjArpi3hU02zvbsEgl/TW8SES4niyN1RIh/OVrwEa5sYiuFhfo8jpdYbpocTfVVfbT3mM2enlnKTmsUbUESI4LnF7yW1upojRMvsAI2dWWo44dCaSr6rqZvzlBeM0EQOoWBHIVAZEsmt8zIuE1tw/ESoO5+HPtgT7qLkxv85EiqKsbVrOnM5R7cePFdGbVmVwFbPUiDWz3eou9/JvozgsdhSwlRlqjgHL8gTD/GV3Ph7hUya7CAqQXagyXN58gp81Cr77tYdJrHYb4ZI+NHAVDVFKYmjUOizU3qWVOqzSozpBfExEQHV7TwIdkUXRITYNfKs4BBMRKg/mdAPWDhsgg9afZlKV1S2mOO16qe27DuVRoL6T9nfvZha/H5aHpZdNpe9t22t2x27DG+VPmE/Jtbt1Mbx9/m01yqYRQuwBhLrQVkj/sZ3NhWscXKt1EfkGkM6HWMd66laQam+UmOzRMCdKaM3vor1tK6X83R82KQae3s9dG1R6i8pLkSvU5ocObAKicILwJ2ibRuYCA6PDmQi9IG0xCQCYcJgGvqpUJt0LCtF+r00rOsIfC9g6CJHA8ASQRBGhuxTw6q+YscthHm9FcVrOM8KcNq6atItizhX1ZLcmEVBflICDP1OKptOXIFnb84La7qc5TR03eJvI4PXS45zAd6DoQLWegI+Ulfm070H7oHM8on0GbjmxiKbS2Pdq0k3UctQIxUjrahJBuO/QEHn0LO8PnzuuhQMBKr+0p5o1sc2p7yBeEd7gged2DrNc70hrjI9OtNBg3i3KC8Brp8DrsOdG0BLCd/tHBgajOliwpmtfXUGkQKw3m4Y+aBX9z8afkAiWTG3j6V+/3+2n2lNnU6e6kbce2xNwEp3LnILvVRbV00cyLCA4SUg1Iu7h6Mb10MGoSS+UDEdaWU19aT/gDucWvNqhriHvz2Ju0fu/6cZFERSxBsBvLG7no5w48G2eSTFtfJo/1ZBRlR8JW6qcuqi6uJZD7dDf9QTUEqg+QgI/o3dpZ9/k9zR1UURZkqw072x1MgJc0hMlphbmv5NwMKg22sJU01tsyCc+Ey0re8gkXnXpGIysNwVA49QIkZcYRcLhGz8QzXokUOCYCotIwJjxZuyib1rIGtVSUgwjAWlKlp5IaqhtYzUBPxhQx1EocVdxsz1xaO2ttHElN9RYXVi2k/V37mcwij9vhpkbPPMPserKrT4Sy2vvaaUf7jlGkF20FMQfp094D4pZULY2R3dnli6i1t5V2deyMM/Wl7lWbV19/queqLJBRBH9wxJ6szU40w9XIEm4Q8Gdankmo9pGoPpSPspfWLqXywnK+H6h+wC3x8f7jnC0qUY6wPV6oMzy7vYzeOlTEFhqKi2xjSvlRfqnDQwunLqRc5BA5QXiNJLxAXqw0JHpsJV4QEASygQBUGmTzYDaQljpyGYGBoI8aihrYsgKkvMrxAqSDIDiQuvaH+qkn1MW34aBCmj91Pq2qX0NFduPNZanc78r6lfSPPf9g1YZVTauoyDHxsiAdBrHb39MSk1Kj/TPKGll/uKWzhdsP970gbohfVjeiagHHQmgPSG/sPiOFNLNiJgXCgRhhNLovRWTHIsWKjNa4a2iGZwYX0+3vpk5/J/l6fKR0l1X5aBvUPuBpLdWA+kF2lQqJklz3Bwdpa9urtOP4Dr43OLGASTJsZHv/qpO0pLGPipypOZNAO2eURdufaruylS4nCK+RhDdbAEg9gkCuIiAqDeb3DFQaRMJrfj9IC8xFQK24zq2cS0wMAz3kKHAwKUScp9BD7b526hroYlJZXVLNG8vSbXVlUSWTv7LCspikdaJlgpSvqF9BHbs7qGe4/dAtXj19Nev8zuycSVtbt9LR7qh9YKgyQBdYG9AeEGeoR1Q67dRQeSbfJ8j+Yzsfo7b+ozEyrc2HjXQI0ElGUMQXJFe5IYauMLCENFofXjn5Ci2bsjQuGm1DWyDphWoCNrfp9X1VBtSDOpvrmmNkV13DLyYS2AgIsv3KocK+7t8AABsHSURBVFeYSKN/VTvnTxug4KAlqXQX97BgapZ8BGhvIMXjnCC8KbZVkgkCeYWAqDSY390i4TW5D2yiWmVyD8RVD9UBSPAgCcQvSKHSqU2mVhBX0DhOQMRAKDMR0EaQRJA6vcQU0s7q4un0yuHn2MqDkn7q6+X4JiKQehBgBGAAneOOvVFCq/KwBLm0kS5uupj6An20rW0bq2ggHpJkEGGQeUh0jYiuKge/sFiid+6Mtuwr38eqGkrdXZFUJVVWhBom4JRkV1uu9hjleYu89ELLC7zhTxHlPv/YknVVF09+dJMEbflmH+cE4U2k0mA2OFI/vAuOfskEF0EgXxAQCa/JPR1O3+GByXdwWlSv3UQOCR6IjSJ72bhBRaozUReIJSTSRgQdkk7YBx4roC1G5BT6xS2eFtrduZvd+oIEFtvdLFWGNBZ/qBMS8n2d+5hUZwJHqI1AVaNDZ4kCurpel5ecNidLbpORXXXPaCccgygdYUV61XX9L66DVC+pXmKIiz69mec5QXiNVBq0L5iZAOV93RGRsJj1DIhKg1nIj9QrEt4RLOQofxFQKg1AAIToVA9GZDfdewJZhq1hWE9QJPCcunNGEWtIUVlKPM4KA34/FRWNtnSBeuG9Dvq8gcGRCSI2pUFaO9H+AvGHXWSYY1MSaX2TEU8RC0u7Qd5zPeQE4TUCSfuCGV2XuOwgYBuEtYyxlzOy05L8q0VUGnKjz0WH17x+ELOI5mEvNY8fARBpuEzecGgDLa5aTLDskI0AaftkSNwhFXZanbSx9UXqHOglGrKw7rayagETaVDtANmdjElEprHLCcKbSKVBrDRkurulPEFAEBgvAmKlYbyIZS69eFrLHJZSUnYQAPmDigGkvbDskKngLIyaKstUeamWo1RAduw8RtbqThoIDxAcSxQ7i08ZoqvuNScIr2qM/AoCgkA8AoHAyBJV/BU5ywYCosObDZTHqCNzfGGMSuRSMgRExTAZQiPXIen0zvbGNvONXEnvKJFKQ3qlppYb91S/YsSxBxxT5aKd3WR3Y0mWIBvXjXR4US8MXksQBPIZAadTdKjN7H9xLWwm+kTiac1k/IerFxXD8fVDJjfZqZrNkvCq+rW/pyLZRftzgvAmUmnQAizH5iAQtsoigDnIR2sV17Zmoi+uhc1FX2oXBAQBhQAkvBLSQyAnCK+RhBdLKKLDm17nppMbSxYcCmRJPR0c080rEsZ0EUwvv1hpSA8/yS0ICAKZQSCXJLyZuaPsl5IThNfotmUJxQiV7MWpJQtbWCw0ZA/1+Jpks1Q8HmaciYTdDNSlTkFAENAjIBJePSLjP88JwisqDePvOMlx+iMAc1hCuMztZ5Gwm4u/1C4ICAJRBETCm/6TkBOEN9FtyKa1RMhIvCAgCGQDAbHSkA2UpY5cR0CsNOR6D0n7UkEgJwivkQ4vGi86vKl0oaQ5nREQCaO5vSv4m4u/OJ4wF39Vu6gYKiTM+xWVhvSxzwnCm/5tSAmThYBYaZgsZJOXK66Fk2M02SlEpWSyER67fHE8MTY+cjV/EBCVhvT7OicIr+jwpt+Rk1ZCWKw0TBq2SQoWl7ZJAMrCZbHSkAWQpQpBQBBIioBIeJNClDRBThDeRCoNosObtP8mP4F4Opp8jKWGnEVAJLw52zXSMEEgrxAQCW/63Z0TXgUqKytH3Unl1ClUVlI6Kl4isodAVVVVxt0jZq/1p35NdXV1p/5NnOJ3UFvfQDVV1af4XZy6zS8qKiKMQxLMQ2Baw3QS6aJ5+KNm/hYXFZnbiFOkdrfbnbClphPegYEBuv/+++nEiRNxjezv7+dzDHgSzEGgo6ODvF6vOZVLrYR3wO/351Uf4J7HeueTXc/0Y4N3oLCwcMw2ZbpOKS8eARmH4vHI9pl8i7ON+Oj6ZBwajUmimJdffjnRJTKd8F511VXcuNLSeGnuM888Q2Dqy5cvT9h4/YVQKMRRdrvxOjyuq2vaY1WONk57jOv6c5VH/eqv689VOvVrdF0bpz1GHv25KifRb7L0Rtf1cU8++SRdeeWVpO8bVac+vYrH71jXEl3X5tEeJ0qvrU9/rM+fynVtHu1xturXt/Gdd96ht99+m6644gr9paTn+vbrM+iv68/TTY/82jK1x/pr+roycT7e+hKl37BhA51xxhnU0NCQsFmJ8qoMya6rdPpfbT7tMdLpz/V59Wn06bXn2mN9vlTOk9VtVIZRHm2calNPTw9t376drr76au3luPtXaVWC8Z6rfPpfbTnaY6TTn+vz6tPo02vPtcf6fKmcJ6vbqAx9nrHagG+xz+dLOA6Nldeobn16fVvUuTad9tioTJVH+6vNoz3W5x/rmj6t0bm2TnWcrEyVLtGvPv/69evpwgsvJKPVcH3a8Z6n0oZkZRqVoc2jPUZa7bn2WH8tlXN93SUlJfqo2LnphHfp0qWEP31AXJm7mJpXrNJfkvMsIQDp7ic+8Yks1SbV6BE42HKQNm3dROvWrdNfkvMsIQC1kovedbGoNWQJb6NqZsyYIeOQETBZiisuLqbW1la69dZbs1SjVKNHAN/ij157HSkPqPrrcj6CwAsvvDByojsynfDq2hM7XbZ0KYmSdgwOUw4uu+JyU+qVSqMIVFZX0jnnnStwmIjAey+9VMYhE/FH1TIOmdsBILtdXV3mNiLPa1931VUEE6GOPMchldsf61nNWcJbLrqjqfTtpKaRzTqTCm/SwqHLOpY+a9ICJEHaCMg4lDaEaRcg41DaEKZVwLx584TwpoVg+pllHEodwyVLlhBs2BuFgkgkEjG6IHGCgCAgCAgCgoAgkN8IdHZ0MABCuvL7OThV7r71WFtCFbSs2OHdu3cv62B94AMfoKeffjoON+wAhW7QD3/4w7h4ddIfHKRvf/vb9O53v5seeeQRFc2/yIP4r3/967yjPe6inMQh8Le//Y2AP3Ryjxw5EruGvvnsZz/L14Cz2pEbS0BEeICQ5rrrrovrP6RV8fq+0eaXY2Jc77jjDsZZ+6zvensn4wpstfF6zIBv7FkPDsYuo18Rj35AP0kYG4E//vGPjDfeBS3eGJcQh37AO2EU8LyjD4E3cNcGjEHI/9Of/lQbLcc6BIAhsALO+HvllVc4BZ5vnGN80vaLLju98cYbjDPS4d1RAWOaKlP/jVFp5HdiCIDoCtmdGHZj5VK8Ro0ZeG7xXGMcwTsSHDB2+qS+x0i36bXo+4N6tN9jjHP5GsZaEcoK4UVHQOl6165d9Pvf/z6uH7D78Lvf/S69+uqrcfHq5Lv3fZM2btxI559/Pj344IOkOhIPCfJcfPHFBCXlxx57TGWRXwMEoNfS2NhIv/zlL+ngwYOcAv1y1113kcfj4R2giey+3nrzLZzmzDPPpPvuu48w40demJOD2SzE/+53v+OPkUHVEkUUZ8fyF7/4BWOCj/R9//kdmj59Oq1cuZKfZ6OPPcgVnn28A1u3bqVv3P0fnB9k4ec//znHd3d303f/878IE0QJxgjgA4Ln1X6BnZ/3v/zlL0xQQZwwLsEag9PppB//+MeGkwdMzNva2njMuffee2nLli1cET5c+/btY4sy6FshXMb4I7azs5Mv4nnHH8Yf4I93AednnXUW4ZugxnltSfjQ33bbbZymurqabr/jTr6M+O9973tUVlbG79KPfvSjhJMWbXlyLAiYhQAILXgN+AvGG4wZ7e3tNHPmTB6bMLY/9PBPDQVQ//b/PsvfY1iwuuMrd8XGKrxLCPge/+QnP4mbEJp1nzlXL1QaJjv4AmGu4p577onccMMNseo2bNgQQdwll1wSueWWW2Lx6uDw4cORyy67LPKHP/yBo26//XZOh/KQ56GHHuJ4XP/IRz4S2bNnj8oqvzoEfD4fxwA34I5w//33R4DpWOHtnTsiZ599dkT1IfoPeB9tb40sXLgw0nHyZKysG2+8cayi8voa8EMfAHv0AQKOgaEK3/rWtyLr1q1Tp/wLfPFuqPcD2OOdQFnoO5X+9ddf5+OnnnoqLr+cjCAQ6PdH+ruizyuOgTcwxHOr8EVq9I8ac1RujC2I37x5M0fhPcD7g37F+6F9p1CWei9UfvmNIoDnVo0liFHjuxav3/zmNzye6zHDs33uuedyNMrBs7/x1Q3cJ9r3CPjjuyIhMQIKSzz7GMsRgCX6A7iq51xfghp30A/oJ23A+4B4vFdIJ8EYAV/QFzfGAC88s1r+kojTYJzHOKTeF/TVE088wefa7zGef3kHRuOfFQlvkSOqQAwnEzDijgBJFCRXMDmDWQ4MK3N8fz8vaWGGgzjM5Bvqa/kaZv+QLB5qeYdt9NbU1HA8JAMo+9ixY3wu/0YjoDY/wZ6iCrCz63K5WKUEy4FK1QEYY+kWktw9e1to8eLFROHo8gpsAULK2NMRldSopS6Ykdu9e7cqWn51COAdQB+o5xyXgStwb25uptWrV/OM/wcP3M85IeFSUi70x4oVKzgeEgC8M7BNir6DVBKhqamJ3xWsokgwRgAmfVxlUUcqOMZKB2xMQ2qrt2+pJJGQ3kLFARLc2bNnsxQRpWPMwe51jEVYvfKWe7hSSOFbWlqof2DAuBF5HssbMYe/B4ACUq3a2tq4JXM82+q5BvYYi/C92L9/P51zzjkxBCHhWv/8ixQIBGL9gotz5szhMSqWUA7iEMAyOFbqYN8bKx5YGTq09x368UM/I2z4wUrgAw88EJMcajNjNXbbtm2c91e/+lVMzRArU5DMo0ysnPz2t7/VZpNjDQKH9h5g/oLnHuG8887j70I4FIyleu211wj8Bn9YwcA4hO/x66+/Hv0eD6fEO/DWW2/Rnt27+D1Slq3wvYANd1nxi0HKB1khvPFVRs8ee/T3FAwG6dprryXY+VNEmGxOtssLQ+99fX0cb7U7OZNKc+jwERbpq3IT7chT1+U3ioDRww91EHwgsAzy+c9/PjrI2Zy0atUqNseEAVHhrnCEekQwFI77yNgsxMbJVRr5TY4A3HXi2cck4sMf/jAT2C2bNnNG9AnIrSJOqg+U45SjR49yuphTEFv0HdFOaJK3IH9TYDKBjwrGH0wcYPxcG0CiEDCR85aXE1RGQNZUPyAPxq+Ozi7+eKm8DruNSYQ/krMGcFRTTf+F3jk+zDyhHm4NhCBw9vGvn7yBY4A9xiIE9InDEW+YCf2GZx7vkQroG4xbEowR2PjPLfzMQkUHYz4m1F+999tsieGee+6hu+65l7H+06PxeqBQPQGJ+sxnPsNCEvQbBFNQFVKOEVDmxz/+cVbPAlGTMBoBX/8Ajzn4ZiJAjQrPNuIRoN4APN///vfzmFNWUsrjEMgsBHvadwDPPd6B7l5f3DiEeKS1DYa5TPkXRcC0URkdCn1SSEMwowe5xewSSttqgIMEBgOX+vjgGB8dvZmU/t4RqaV0bGIE9A8/Pt7AGpgjgPTu37uPVq2qpssuu4zjoBeHj70KeIkw6ywrLmPpjIrHCwddYAmpI/D2rl0Enc9NmzZxJuD3pS99ibFHXyCoDVTKtqC/v4/jQYjx8VEuuQN90T6CtFHC2Ajgg/LNb36TbrrpJoLeOp7pRGHt2rV8Ce8KpPN9vT1EVMfkAB8e+Ljv7e2NZe/p7qLy8nIqLJAPTQwUgwPoMEKyjhUNjOkIkDxCNxq6iGrjCVaQMBZhso73Y+fO6EY1TBbxoUccPu7alRP0p5qYGFSd11GQEmJlAqulCJVTp7AXwV//+texVSSsRmFfwYEDBzgNpLf4TpwIRcnZrKZGjscYBG+Q21/fwqtLWHlCwCQR+2vwLVH9yBfkHyMwdepUHnPCQ1FA8OyC9NbU17LeLZ5/jE2KB+H9UOOQmmgrKPFdwDuAMrXjEMpEvNjuVUhFf7Mq4YU0Sg1ut3zxZv7YYxYzd+5cqq+v5xdF2zzMIA8dOhTbHIIleAR8pEB+IcpH+Pvf/x594Zpm8rn8M0YAy7j4OOAjjQDJojYAUzXrVPELFi3kJSx1/uc//5kHRrenhF8yRciee+652Auq0srvaASqKr0x99a4evjw4VgiDF76D3XNtBk8cOEZR9j+xlusOtI0dz6fY8KIAPKMJd/zVoujCgYkwT9sNMOmJnxQIN1FwLIgVBYQQAggRceYpA14VzAxh0QXAcu2+MBDlQQfF6w6ITz+178xWQDplWCMAKTrmKw99JOfxAgRxpFv3fcdXmpXH3dtbpCwObMa6fnnn49F43vwnve8h+BKFIQBkkoEkC3ESxiNgD8cXTZXK0XaFPqxR+G5/OylTMZCJ1o5ua0kOqkG+UI4diKqjog+QCguckUllhr1Ob4g/xiB8inVPNlT/AXfVKhuDoUG6Z6vf4Ml6Gps0kMGr48QFqp+xDuACcbcWXMIq36YzCBgEz/UrpQ6qb6cfD3PioQXpmSwu/Yf//gH44wX6Wt33x374OBDgpcNHYflkf/+wff4GAPfpz71KbbigI8UdF4+PbzU9R933k5f/drX6dFHH2XyC90imU0mfoxh1QJYPfvss6wvCr24q65eR3d+OWpmCTkx+Vhw5mL+cGCWeduXvsSYfuhDH6JLLrkkRpahR4eJyxe+8AXCNUgVMYBiV6kEYwRApG659Vb+YIOYwqTM9ddfTzfccAP3B2bjCN///vf5F6QAeqSQvn/wgx+kL37xi6zri48KlhQxkGHpEMQB0mC8Pzife8Y84wZILCOAcQjvAAJ0ELF/4JOf+ATd+ZWvML6Y9EGvGs84+uyBBx9kvURgDHzxXoDgQqJyyUWXcDmIg04kfhFgUUNN7DlC/sUQwLI4xgzoTH/u859nSRfeA2AH3VAVMAnB8jjS/+zhn7MO46zZc+mCCy7g9wUTd/QRJhz4nkB3FOoRWJGC/u8111yjipJfDQJYHgdR7enBSgUxycIvxg/tSh7UddQz3LwiqlKiiunvPEZUPzXmjAISX7w3Kj8mhRinytwjaiYqr/wSj90YhzDWw0IDsHr0T38gWF94/PHHedwHYcVzfPO//zuT24cf+hmPPxD2wSzix6/7GKvyQO0NgkEIs7A6+LGPfYy/0263WzwUGjxsWXE8AV2eDS++HKseg5IiTYjEoAadUHxUsHT11taN5K2s4sEM16EnBEnWokWLOI0qCNIaLHFByX7xsuUym1HAGPxi0oHlJwQMTlALwQQDkhWQJsRdeumlPMihDza8sD7WR/ig8KzS7+dZo9Z8GXTukHfhgvlCtgxwV1HAELqJ6qPAE7zmZVRoc9Cz69dzMgxe6BME7Tuhzl986WVasGBBnCQd/QeJFggzpJDqI8WFyL9RCGAsgSQE+OO5xYZYfNCxYRM4Ilx04YW8iQqT783bttCMppmxybQyHwTpCciWCmqMUu+VipffeAQwidjwyiv8kYeqGvoAqmwYg/DhV+prWAlBvyD9pt376ZIV0fcC5//75JP8vJ+zalVssxvi1XvUfFYzNTQ2xFcsZzEEIPyAKtVLL73E39bvfOc7vMqB/RxPPPEEY/7JT32KJ+RKtQ2ZgTHiIT3HRBx/eI9gnvLyyy/ncen222/nTYZYrcJGKxmPYrCPOtCPGTjHZBoB7wUEfCCzCBAK4hh4ar/HcD2vBH34bq9/6u/yPWbEjP9lhfAaVy2xgoAgIAgIAoKAIJBNBDC5gzQQEnUIn7BhFlL2b3zjGyxhBImFFPHLX/4ydff1EDavQcUHK65YeYL+OwgZ9HzvvvtunqSDrEFKryaT0MU2Uk3J5n1KXYKAHgEhvHpE5FwQEAQEAUFAEDiNEVCrrkqKCMmhigPhVdJzSBOxioTNbZAkQoq4ffNGJrb6VT21iqhdqTqNIZRbOwUREMJ7CnaaNFkQEAQEAUFAEBAEBAFBIHUEsmqlIfVmSUpBQBAQBAQBQUAQEAQEAUEgMwgI4c0MjlKKICAICAKCgCAgCAgCgkCOIiCEN0c7RpolCAgCgoAgIAhkAgFsVIMpRFhCgvUEFaB3iw1qcG/+yCOPqGjDX+SDNQZYa5AgCJyKCGTFDu+pCIy0WRAQBAQBQUAQOB0QsNitbE4M5gs3btzIt4TNaLADC1vUMLEHBwiwL21kXQEb2pAPtvRhVk5rrux0wEfuIT8QEMKbH/0sdykICAKCgCCQpwjAwoJyIa+8NsIs2e7du+nb//U9tmG/detWJr1GhPfZp55he/dXXnklO89RhBdSXxBhEGnYUoYDFrjEha1f2H2H5y+YPrv9K3fE7MXmaRfIbecAAqLSkAOdIE0QBAQBQUAQEASyiQA8Ofp8vpjDptLSUibARm2AnV2Hw0E333wzO0eAYxwEqES8/PLLdMsXb2Z7vg888ACbN2tvb2f3tu973/vo3z57I8HDmwRBwGwEhPCa3QNSvyAgCAgCgoAgkKMIgNzC0cSaNWtiXh6ffibqnntgYIDVG+DS/MYbb2S3uG/sO8qu5uEGGt47cU08ruVo5+ZZs0SlIc86XG5XEBAEBAFBID8RUK6bcfcul4uKi4tjQIRCIZo9eza7Nf/IdR8lr9dL8JiGPHAh/9prr7F6An7hnEIFpFMB8aETrXxaXV0tRFcBI785gYAQ3pzoBmmEICAICAKCgCAwuQg4nc5YBfPmzSO3281qCdi0tm/fPlq5ciVNa5xDDz/8MDnsNprWMJ1dBkOv947bv0zBUJiJ7/PPP0/wwgbd3Q0bNnCZShI8q6mR3RYjMjgQIIdrpM5Y5XIgCJiAgKg0mAC6VCkICAKCgCAgCGQLAVhkgEmxO++8kx5//HE+Li5y0fXXX0/XXHMNrVixgiCRxXGRw8rmy6CKgI1oLS0tBF1cnMOs2TUfvZ5VF5588kmCG+Hu7m7eEHf5v1wRU2FAHIKQ3Wz1sNSTCgLiWjgVlCSNICAICAKCgCBwiiLQHxykE4ePUCgS4juA6kH5lGo+PnloL/WHBlmaq9e1hRS3u6+HN51pr8FMGTaiffrTn2Y1h9tuu431fGtqaliNQdnqLdeoO5yi0EmzTyMERKXhNOpMuRVBQBAQBAQBQUCPAKS2DY0N+mg+L2qaYxiPSJBcLdFVCWHmDAGb1kBy6+rq1CX+FaIbB4ec5AgCIuHNkY6QZggCgoAgIAgIAqcSApDkOgsLDUnxqXQf0tb8QEAIb370s9ylICAICAKCgCAgCAgCeYuAbFrL266XGxcEBAFBQBAQBAQBQSA/EBDCmx/9LHcpCAgCgoAgIAgIAoJA3iIghDdvu15uXBAQBAQBQUAQEAQEgfxAQAhvfvSz3KUgIAgIAoKAICAICAJ5i8D/B+Y8sbnxhANNAAAAAElFTkSuQmCC" 264 | } 265 | }, 266 | "cell_type": "markdown", 267 | "metadata": { 268 | "slideshow": { 269 | "slide_type": "fragment" 270 | } 271 | }, 272 | "source": [ 273 | "```python\n", 274 | "plot_forecasts(tss, forecasts, past_length=150, num_plots=3)\n", 275 | "```\n", 276 | "![image.png](attachment:image.png)" 277 | ] 278 | }, 279 | { 280 | "cell_type": "markdown", 281 | "metadata": { 282 | "slideshow": { 283 | "slide_type": "slide" 284 | } 285 | }, 286 | "source": [ 287 | "### Evaluator class\n", 288 | "- **Evaluator** provides a multi-metric quantitative evaluation of the model." 289 | ] 290 | }, 291 | { 292 | "cell_type": "markdown", 293 | "metadata": { 294 | "slideshow": { 295 | "slide_type": "fragment" 296 | } 297 | }, 298 | "source": [ 299 | "```python\n", 300 | "from gluonts.evaluation import Evaluator\n", 301 | "\n", 302 | "evaluator = Evaluator(quantiles=[0.5], seasonality=1)\n", 303 | "agg_metrics, item_metrics = evaluator(iter(tss), iter(forecasts), num_series=len(test_data))\n", 304 | "```" 305 | ] 306 | }, 307 | { 308 | "cell_type": "markdown", 309 | "metadata": { 310 | "slideshow": { 311 | "slide_type": "slide" 312 | } 313 | }, 314 | "source": [ 315 | "```python\n", 316 | "agg_metrics\n", 317 | "\n", 318 | "{'MSE': 163.59102376302084,\n", 319 | " 'abs_error': 1090.9220886230469,\n", 320 | " 'abs_target_sum': 5658.0,\n", 321 | " 'abs_target_mean': 52.38888888888889,\n", 322 | " 'seasonal_error': 18.833625618877182,\n", 323 | " 'MASE': 0.5361500323952336,\n", 324 | " 'sMAPE': 0.21201368270827592,\n", 325 | " 'MSIS': 21.446000940010823,\n", 326 | " 'QuantileLoss[0.5]': 1090.9221000671387,\n", 327 | " 'Coverage[0.5]': 0.34259259259259256,\n", 328 | " 'RMSE': 12.790270668090681,\n", 329 | " 'NRMSE': 0.24414090352665138,\n", 330 | " 'ND': 0.19281054942082837,\n", 331 | " 'wQuantileLoss[0.5]': 0.19281055144346743,\n", 332 | " 'mean_wQuantileLoss': 0.19281055144346743,\n", 333 | " 'MAE_Coverage': 0.15740740740740744}\n", 334 | "```" 335 | ] 336 | }, 337 | { 338 | "cell_type": "markdown", 339 | "metadata": { 340 | "slideshow": { 341 | "slide_type": "slide" 342 | } 343 | }, 344 | "source": [ 345 | "# Lab 1\n", 346 | " \n", 347 | "Optional: [Descriptive statistics](../part2/descriptive_stats.ipynb)\n", 348 | " \n", 349 | " \n", 350 | "[Build and train a DeepAR model with GluonTS](../part3/twitter_volume_forecast.ipynb)\n" 351 | ] 352 | }, 353 | { 354 | "cell_type": "code", 355 | "execution_count": null, 356 | "metadata": { 357 | "slideshow": { 358 | "slide_type": "notes" 359 | } 360 | }, 361 | "outputs": [], 362 | "source": [] 363 | } 364 | ], 365 | "metadata": { 366 | "celltoolbar": "Slideshow", 367 | "kernelspec": { 368 | "display_name": "Python 3", 369 | "language": "python", 370 | "name": "python3" 371 | }, 372 | "language_info": { 373 | "codemirror_mode": { 374 | "name": "ipython", 375 | "version": 3 376 | }, 377 | "file_extension": ".py", 378 | "mimetype": "text/x-python", 379 | "name": "python", 380 | "nbconvert_exporter": "python", 381 | "pygments_lexer": "ipython3", 382 | "version": "3.7.3" 383 | } 384 | }, 385 | "nbformat": 4, 386 | "nbformat_minor": 4 387 | } 388 | -------------------------------------------------------------------------------- /notebooks/part2/descriptive_stats.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Optional: Descriptive Statistics\n", 8 | "\n", 9 | "This lab is optional. \n", 10 | "\n", 11 | "The goal of this quick starter guide is to very quickly go through time-series modeling pipeline. This tutorial does not intend to get deep into concepts of time-series modeling and only uses a very clean dataset. To use an analogy from learning music, the intention is to help the reader to play a few basic songs on the instrument before being disheartened by musical theory.\n" 12 | ] 13 | }, 14 | { 15 | "cell_type": "code", 16 | "execution_count": null, 17 | "metadata": {}, 18 | "outputs": [], 19 | "source": [ 20 | "import datetime\n", 21 | "\n", 22 | "import matplotlib.pyplot as plt\n", 23 | "import pandas as pd\n", 24 | "from scipy.stats import pearsonr\n", 25 | "from sklearn.metrics import mean_squared_error" 26 | ] 27 | }, 28 | { 29 | "cell_type": "markdown", 30 | "metadata": {}, 31 | "source": [ 32 | "# The Strategy\n", 33 | "The dataset that is used in this tutorials includes tweet volume for the hashtag **#AMZN**. We load the csv file directly into a `pandas` dataframe, run a few quick checks and descriptive statistics to have an idea of what the dataset is all about and we build a baseline neural model using MLP. " 34 | ] 35 | }, 36 | { 37 | "cell_type": "markdown", 38 | "metadata": {}, 39 | "source": [ 40 | "# The Dataset\n", 41 | "The dataset is located in [Numenta Anomaly Detection Benchmark](https://raw.githubusercontent.com/numenta/NAB/master/data/realTweets/Twitter_volume_AMZN.csv). The dataset is based on real-time streaming data from Twitter with **#AMZN** hashtag." 42 | ] 43 | }, 44 | { 45 | "cell_type": "code", 46 | "execution_count": null, 47 | "metadata": {}, 48 | "outputs": [], 49 | "source": [ 50 | "url = \"https://raw.githubusercontent.com/numenta/NAB/master/data/realTweets/Twitter_volume_AMZN.csv\"" 51 | ] 52 | }, 53 | { 54 | "cell_type": "markdown", 55 | "metadata": {}, 56 | "source": [ 57 | "## Loading the data\n", 58 | "The dataset includes only two columns, *timestamp* and *value*. Each line represents a five-minute aggregate of AMZN tweet volume." 59 | ] 60 | }, 61 | { 62 | "cell_type": "code", 63 | "execution_count": null, 64 | "metadata": {}, 65 | "outputs": [], 66 | "source": [ 67 | "df = pd.read_csv(url)\n", 68 | "df.head()" 69 | ] 70 | }, 71 | { 72 | "cell_type": "markdown", 73 | "metadata": {}, 74 | "source": [ 75 | "## Checking for missing data\n", 76 | "`pandas.isna()` returns `True` or `False` for every record in the dataframe. True indicates that at a certain position in the dataset, a value is missing. The output of `isna()` is a DataFrame, so we can aggregate the values using `sum` to retrieve the total number of missing values per column." 77 | ] 78 | }, 79 | { 80 | "cell_type": "code", 81 | "execution_count": null, 82 | "metadata": {}, 83 | "outputs": [], 84 | "source": [ 85 | "df.isna().sum()" 86 | ] 87 | }, 88 | { 89 | "cell_type": "markdown", 90 | "metadata": {}, 91 | "source": [ 92 | "There are no missing values in the dataset" 93 | ] 94 | }, 95 | { 96 | "cell_type": "markdown", 97 | "metadata": {}, 98 | "source": [ 99 | "### Which year/month do we have data from?\n", 100 | "First we check what years we have data for. As you can see data is only from 2015.\n", 101 | "We also have data only for February till April. February data seems to be incomplete as well" 102 | ] 103 | }, 104 | { 105 | "cell_type": "code", 106 | "execution_count": null, 107 | "metadata": {}, 108 | "outputs": [], 109 | "source": [ 110 | "dt = pd.to_datetime(df.timestamp)\n", 111 | "dt.groupby([dt.dt.year, dt.dt.month]).count()" 112 | ] 113 | }, 114 | { 115 | "cell_type": "markdown", 116 | "metadata": {}, 117 | "source": [ 118 | "## Converting Timestamp to Readable Dates" 119 | ] 120 | }, 121 | { 122 | "cell_type": "code", 123 | "execution_count": null, 124 | "metadata": {}, 125 | "outputs": [], 126 | "source": [ 127 | "df['timestamp_readable'] = df['timestamp'].apply(lambda x: datetime.datetime.strptime(x, '%Y-%m-%d %H:%M:%S').ctime()[:-4]).astype('str')\n", 128 | "df.head()\n" 129 | ] 130 | }, 131 | { 132 | "cell_type": "markdown", 133 | "metadata": {}, 134 | "source": [ 135 | "This will help us uncover seasonnality events" 136 | ] 137 | }, 138 | { 139 | "cell_type": "markdown", 140 | "metadata": {}, 141 | "source": [ 142 | "## Basic Descriptive Statistics\n", 143 | "Using `describe` we can get a quick descriptive report on the dataset. Since the data includes only one numerical column, we can improve the readability, using `no.transpose()` or `np.T`.\n", 144 | "\n", 145 | "`df.describe` by default only includes numerical values, but we could pass `include='all'` to the function to include all field. In the case of timestamp, the output would be of no interest to us, so we do not change default values." 146 | ] 147 | }, 148 | { 149 | "cell_type": "code", 150 | "execution_count": null, 151 | "metadata": {}, 152 | "outputs": [], 153 | "source": [ 154 | "df.describe().T" 155 | ] 156 | }, 157 | { 158 | "cell_type": "markdown", 159 | "metadata": {}, 160 | "source": [ 161 | "**Notable Observations**\n", 162 | "- There are 15831 rows of data\n", 163 | "- min==0.0, therefore there exists at least one row with `value==0`. It could be possible that missing values are replaced with zeros, so we would like to check that the total number of zeros are not significant. We run a count on `df.value==0`, and if the output shows only a small number of rows are zero, then we accept those zeros and acceptable, accurate, or ignorable.\n", 164 | "- In this case, only 28 rows have value of zero, so we keep them as they are." 165 | ] 166 | }, 167 | { 168 | "cell_type": "code", 169 | "execution_count": null, 170 | "metadata": {}, 171 | "outputs": [], 172 | "source": [ 173 | "df[df.value==0].count()" 174 | ] 175 | }, 176 | { 177 | "cell_type": "markdown", 178 | "metadata": {}, 179 | "source": [ 180 | "## Plotting the Data\n", 181 | "The data is showing signs of seasonality, indeed we notice that we have a peak for each day, most likely corresponding to day time / night time.\n", 182 | "We can also see that the activity seems more limited on the Saturday and Sunday" 183 | ] 184 | }, 185 | { 186 | "cell_type": "code", 187 | "execution_count": null, 188 | "metadata": {}, 189 | "outputs": [], 190 | "source": [ 191 | "fourteen_days = 24*60*14//5 # Seven days of the data that is sampled every 5 minutes" 192 | ] 193 | }, 194 | { 195 | "cell_type": "code", 196 | "execution_count": null, 197 | "metadata": {}, 198 | "outputs": [], 199 | "source": [ 200 | "ax = df[:fourteen_days].plot(y='value', figsize=(20,10), sharex=False)\n", 201 | "o = ax.set_xticklabels([df.timestamp_readable[int(x)] if x >= 0 else '' for x in ax.get_xticks()])" 202 | ] 203 | }, 204 | { 205 | "cell_type": "markdown", 206 | "metadata": {}, 207 | "source": [ 208 | "## Are there any patterns in the data if we reduce granularity? \n", 209 | "Very often short term data is far too jittery and all that can be observed is white noise. There are several techniques to address the issue and try to understand whether there are underlying patterns in the data.\n", 210 | "Aggregation and smoothing are amongst those techniques.\n", 211 | "\n", 212 | "In the next few cells, the data has been aggregated to hourly average per month in order to investigate if there are patterns hidden in the data.\n", 213 | "\n", 214 | "The data in the dataset sums the number of tweets every five minutes. It is useful for short term prediction of immediate actions. Such data is similar to that of financial data such as stock prices.\n", 215 | "\n", 216 | "Aggregated data will reduce granularity and helps us predict seasonal and long term non-stationary time series. This is more similar to use cases such as forecasting weather patterns." 217 | ] 218 | }, 219 | { 220 | "cell_type": "code", 221 | "execution_count": null, 222 | "metadata": {}, 223 | "outputs": [], 224 | "source": [ 225 | "df.plot()" 226 | ] 227 | }, 228 | { 229 | "cell_type": "markdown", 230 | "metadata": {}, 231 | "source": [ 232 | "Plotting several hours, hour by hour to understand if they are any internal patterns. As the data has a 5 minute frequency, 12 * 5 minutes represents 1 hour." 233 | ] 234 | }, 235 | { 236 | "cell_type": "markdown", 237 | "metadata": {}, 238 | "source": [ 239 | "We then plot a histogram of data to understand how the data is distributed. It could be observed from the previous plot that most data were around 0-100. From the histogram we can see the point more clearly." 240 | ] 241 | }, 242 | { 243 | "cell_type": "code", 244 | "execution_count": null, 245 | "metadata": {}, 246 | "outputs": [], 247 | "source": [ 248 | "df.hist(column=['value'], bins=100)" 249 | ] 250 | }, 251 | { 252 | "cell_type": "markdown", 253 | "metadata": {}, 254 | "source": [ 255 | "### Daily Patterns" 256 | ] 257 | }, 258 | { 259 | "cell_type": "code", 260 | "execution_count": null, 261 | "metadata": {}, 262 | "outputs": [], 263 | "source": [ 264 | "df.timestamp = pd.to_datetime(df.timestamp)\n", 265 | "df = df.set_index('timestamp')" 266 | ] 267 | }, 268 | { 269 | "cell_type": "code", 270 | "execution_count": null, 271 | "metadata": {}, 272 | "outputs": [], 273 | "source": [ 274 | "df.iloc[28]" 275 | ] 276 | }, 277 | { 278 | "cell_type": "code", 279 | "execution_count": null, 280 | "metadata": {}, 281 | "outputs": [], 282 | "source": [ 283 | "offset = 28 # We offset to 28 to start the day closest to 12am" 284 | ] 285 | }, 286 | { 287 | "cell_type": "markdown", 288 | "metadata": {}, 289 | "source": [ 290 | "Plotting separate days, day by day, to understand daily patterns. 288/12=24 hour." 291 | ] 292 | }, 293 | { 294 | "cell_type": "code", 295 | "execution_count": null, 296 | "metadata": {}, 297 | "outputs": [], 298 | "source": [ 299 | "timestamps_per_day = 24*60//5" 300 | ] 301 | }, 302 | { 303 | "cell_type": "code", 304 | "execution_count": null, 305 | "metadata": {}, 306 | "outputs": [], 307 | "source": [ 308 | "fig, axes = plt.subplots(nrows=2, ncols=2, figsize=(14,14), sharey=True)\n", 309 | "\n", 310 | "df[offset : timestamps_per_day + offset].plot(ax=axes[0,0], title='day 1')\n", 311 | "df[1 * timestamps_per_day + offset: 2 * timestamps_per_day + offset].plot(ax=axes[0,1], title='day 2')\n", 312 | "df[2 * timestamps_per_day + offset: 3 * timestamps_per_day + offset].plot(ax=axes[1,0], title='day 3')\n", 313 | "df[3 * timestamps_per_day + offset: 4 * timestamps_per_day + offset].plot(ax=axes[1,1], title='day 4')" 314 | ] 315 | }, 316 | { 317 | "cell_type": "markdown", 318 | "metadata": {}, 319 | "source": [ 320 | "We can try to compute the moving average to remove the noise" 321 | ] 322 | }, 323 | { 324 | "cell_type": "code", 325 | "execution_count": null, 326 | "metadata": {}, 327 | "outputs": [], 328 | "source": [ 329 | "fig, axes = plt.subplots(nrows=2, ncols=2, figsize=(14,14), sharey=True)\n", 330 | "\n", 331 | "df[offset : timestamps_per_day + offset].value.rolling(window=12).mean().plot(ax=axes[0,0], title='day 1')\n", 332 | "df[1 * timestamps_per_day + offset: 2 * timestamps_per_day + offset].value.rolling(window=12).mean().plot(ax=axes[0,1], title='day 2')\n", 333 | "df[2 * timestamps_per_day + offset: 3 * timestamps_per_day + offset].value.rolling(window=12).mean().plot(ax=axes[1,0], title='day 3')\n", 334 | "df[3 * timestamps_per_day + offset: 4 * timestamps_per_day + offset].value.rolling(window=12).mean().plot(ax=axes[1,1], title='day 4')" 335 | ] 336 | }, 337 | { 338 | "cell_type": "markdown", 339 | "metadata": {}, 340 | "source": [ 341 | "Now if we average this across all days we can see the daily pattern even more clearer" 342 | ] 343 | }, 344 | { 345 | "cell_type": "code", 346 | "execution_count": null, 347 | "metadata": {}, 348 | "outputs": [], 349 | "source": [ 350 | "o = df.value.groupby(df.index.hour).mean().plot(title=\"Overall mean across the hours of the day\")" 351 | ] 352 | }, 353 | { 354 | "cell_type": "markdown", 355 | "metadata": {}, 356 | "source": [ 357 | "In this notebook you have investigated the Twitter volume dataset. Now it is time to move to the [next](../part3/twitter_volume_forecast.ipynb) tutorial where you will train a DeepAR model on this dataset." 358 | ] 359 | } 360 | ], 361 | "metadata": { 362 | "kernelspec": { 363 | "display_name": "conda_python3", 364 | "language": "python", 365 | "name": "conda_python3" 366 | }, 367 | "language_info": { 368 | "codemirror_mode": { 369 | "name": "ipython", 370 | "version": 3 371 | }, 372 | "file_extension": ".py", 373 | "mimetype": "text/x-python", 374 | "name": "python", 375 | "nbconvert_exporter": "python", 376 | "pygments_lexer": "ipython3", 377 | "version": "3.6.5" 378 | } 379 | }, 380 | "nbformat": 4, 381 | "nbformat_minor": 4 382 | } 383 | -------------------------------------------------------------------------------- /notebooks/part3/twitter_volume_forecast.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Modeling and forecasting of twitter volume timeseries\n", 8 | "After understanding our data in the previous section, [descriptive statistics](../part2/descriptive_stats.ipynb), we now want to quickly run a time-series forecast using [gluonts](https://github.com/awslabs/gluon-ts).\n", 9 | "In this example we use the same dataset as before and create first a baseline (seasonal naive estimator). Afterwards we create and train a [DeepAR](https://arxiv.org/abs/1704.04110) model and compare it to the baseline. \n", 10 | "\n", 11 | "Let's first check that GluonTS is installed and that we have the correct MXNet version:" 12 | ] 13 | }, 14 | { 15 | "cell_type": "code", 16 | "execution_count": null, 17 | "metadata": {}, 18 | "outputs": [], 19 | "source": [ 20 | "import mxnet\n", 21 | "import gluonts\n", 22 | "print(mxnet.__version__)" 23 | ] 24 | }, 25 | { 26 | "cell_type": "markdown", 27 | "metadata": {}, 28 | "source": [ 29 | "If your MXNet version is not 1.4.1 or GluonTS is not installed, then please uncomment and execute the following lines." 30 | ] 31 | }, 32 | { 33 | "cell_type": "code", 34 | "execution_count": null, 35 | "metadata": {}, 36 | "outputs": [], 37 | "source": [ 38 | "#! pip install mxnet==1.4.1\n", 39 | "#! pip install gluonts" 40 | ] 41 | }, 42 | { 43 | "cell_type": "code", 44 | "execution_count": null, 45 | "metadata": {}, 46 | "outputs": [], 47 | "source": [ 48 | "import pandas as pd\n", 49 | "import numpy as np\n", 50 | "import matplotlib.pyplot as plt\n", 51 | "from mxnet import gpu, cpu\n", 52 | "from mxnet.context import num_gpus\n", 53 | "from gluonts.dataset.util import to_pandas\n", 54 | "from gluonts.model.deepar import DeepAREstimator\n", 55 | "from gluonts.model.simple_feedforward import SimpleFeedForwardEstimator\n", 56 | "from gluonts.dataset.common import ListDataset\n", 57 | "from gluonts.trainer import Trainer\n", 58 | "from gluonts.evaluation.backtest import make_evaluation_predictions, backtest_metrics\n", 59 | "import pathlib\n", 60 | "import json\n", 61 | "import boto3\n", 62 | "import csv" 63 | ] 64 | }, 65 | { 66 | "cell_type": "markdown", 67 | "metadata": {}, 68 | "source": [ 69 | "### Setting up hyperparameters\n", 70 | "Here we just set the number of epochs and rely on default values for the rest of the parameters in order to make the example more understandable." 71 | ] 72 | }, 73 | { 74 | "cell_type": "code", 75 | "execution_count": null, 76 | "metadata": {}, 77 | "outputs": [], 78 | "source": [ 79 | "EPOCHS = 20" 80 | ] 81 | }, 82 | { 83 | "cell_type": "markdown", 84 | "metadata": {}, 85 | "source": [ 86 | "### Loading the data" 87 | ] 88 | }, 89 | { 90 | "cell_type": "code", 91 | "execution_count": null, 92 | "metadata": {}, 93 | "outputs": [], 94 | "source": [ 95 | "url = \"https://raw.githubusercontent.com/numenta/NAB/master/data/realTweets/Twitter_volume_AMZN.csv\"" 96 | ] 97 | }, 98 | { 99 | "cell_type": "code", 100 | "execution_count": null, 101 | "metadata": {}, 102 | "outputs": [], 103 | "source": [ 104 | "df = pd.read_csv(filepath_or_buffer=url, header=0, index_col=0)\n", 105 | "df.head()" 106 | ] 107 | }, 108 | { 109 | "cell_type": "code", 110 | "execution_count": null, 111 | "metadata": {}, 112 | "outputs": [], 113 | "source": [ 114 | "df[:100].plot(figsize=(10,5))" 115 | ] 116 | }, 117 | { 118 | "cell_type": "markdown", 119 | "metadata": {}, 120 | "source": [ 121 | "### Plotting forecast helper function\n", 122 | "Often it is interesting to tune or evaluate the model by looking at error metrics on a hold-out set. For other machine learning tasks such as classification, one typically does this by randomly separating examples into train/test sets. For forecasting it is important to do this train/test split in time rather than by series.\n", 123 | "\n", 124 | "The below function plots a forecast for a given data and a given predictor. Let's dive deeper into components of this funciton.\n", 125 | "`from gluonts.model` includes a number of implemented models. Each model has an estimator. An estimator accepts a series of models and hyperparameters. Parameters include a trainer that accepts optimization parameters. The estimator also accepts parameters such as context (CPU, GPU), number of layers, context length, time-series frequency, and prediction length amongst others. The context length defines how many past time steps will be taken into account to make a prediction. The default value is the prediction_length.\n", 126 | "\n", 127 | "The estimator has a `train` method that is used for fitting the data. The `train` method returns a predictor that can be used to forecast based on input data.\n", 128 | "\n", 129 | "`plot_forecast` function accepts a `predictor` object and an iterable dataset to plot the data and the forecast.\n", 130 | "It will call `make_evaluation_predictions` that takes the predictor and test_data and creates the forecasts." 131 | ] 132 | }, 133 | { 134 | "cell_type": "code", 135 | "execution_count": null, 136 | "metadata": {}, 137 | "outputs": [], 138 | "source": [ 139 | "from pylab import rcParams\n", 140 | "rcParams['figure.figsize'] = 15, 8\n", 141 | "def plot_forecast(predictor, test_data):\n", 142 | " prediction_intervals = (50.0, 90.0)\n", 143 | " legend = [\"observations\", \"median prediction\"] + [f\"{k}% prediction interval\" for k in prediction_intervals][::-1]\n", 144 | " forecast_it, ts_it = make_evaluation_predictions(\n", 145 | " dataset=test_data, \n", 146 | " predictor=predictor, \n", 147 | " num_samples=100, \n", 148 | " )\n", 149 | " fig, ax = plt.subplots(1, 1, figsize=(10, 7))\n", 150 | " list(ts_it)[0][-336:].plot(ax=ax) \n", 151 | " list(forecast_it)[0].plot(prediction_intervals, color='g')\n", 152 | " plt.grid(which=\"both\")\n", 153 | " plt.legend(legend, loc=\"upper left\")\n", 154 | " plt.show()" 155 | ] 156 | }, 157 | { 158 | "cell_type": "markdown", 159 | "metadata": {}, 160 | "source": [ 161 | "### Dataset\n", 162 | "`gluonts.dataset.common` has a class `ListDataset`. GluonTS does not require this specific format for a custom dataset that a user may have. The only requirements for a custom dataset are to be iterable and have a \"target\" and a \"start\" field. To make this more clear, assume the common case where a dataset is in the form of a `numpy.array` and the index of the time series in a `pandas.Timestamp`. \n", 163 | "\n", 164 | "In this example we are using a `gluonts.dataset.common.ListDataset`. A `ListDataset` consist of a list of of dictionaries with the following format:\n", 165 | "```\n", 166 | "{'start': Timestamp('2019-07-26 00:00:00', freq='D'),\n", 167 | " 'cat': [5, 4, 42, 17, 0, 0, 0],\n", 168 | " 'target': array([0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 2., 0.], dtype=float32)},\n", 169 | " {'start': Timestamp('2019-07-26 00:00:00', freq='D'),\n", 170 | " 'cat': [8, 7, 32, 13, 0, 0, 0],\n", 171 | " 'target': array([4., 3., 5., 2., 5., 2., 3., 7., 4., 3., 3., 2.], dtype=float32)}\n", 172 | "```\n", 173 | "Each dictionary contains one time series and we need to pass *start* as `pandas.index` and a *target* as an iterable set of timestamp values from our pandas dataframe. We can also indicate categorical features in the field `cat`.\n", 174 | "\n", 175 | "In the followng cell we build a training dataset ending at April 5th, 2015 and a test dataset that will be used forecast the hour following the midnight on April 15th, 2015. GluonTS requires the full timeseries to be in the test dataset. So test and train data will start at February 26 2015. GluonTS will then cut out the `n` last elements from test dataset, in order to predict those. `n` is equal the prediction length. " 176 | ] 177 | }, 178 | { 179 | "cell_type": "code", 180 | "execution_count": null, 181 | "metadata": {}, 182 | "outputs": [], 183 | "source": [ 184 | "training_data = ListDataset([{\"start\": df.index[0], \n", 185 | " \"target\": df.value[: \"2015-04-05 00:00:00\"]}], \n", 186 | " freq=\"5min\")\n", 187 | "\n", 188 | "test_data = ListDataset([{\"start\": df.index[0], \n", 189 | " \"target\": df.value[:\"2015-04-15 00:00:00\"]}], \n", 190 | " freq=\"5min\")" 191 | ] 192 | }, 193 | { 194 | "cell_type": "markdown", 195 | "metadata": {}, 196 | "source": [ 197 | "## Create a baseline: seasonal naive predictor\n", 198 | "\n", 199 | "Before training complex deep learning models, it is best to come up with simple base models. Such a baseline could for instance be: that the future volumes will be the same like in the last 5 minutes. This naive estimation works remarkably well for many economic and financial time series. Because a naïve forecast is optimal when data follows a random walk, these are also called random walk forecasts\n", 200 | "\n", 201 | "GluonTS provides a [seasonal naive predictor](https://gluon-ts.mxnet.io/api/gluonts/gluonts.model.seasonal_naive.html). The seasonal naive method sets each forecast to be equal to the last observed value from the same season. So the model assumes that the data has a fixed seasonality (in this case, 300 time steps correspond to nearly a day), and produces forecasts by copying past observations based on it.\n" 202 | ] 203 | }, 204 | { 205 | "cell_type": "code", 206 | "execution_count": null, 207 | "metadata": {}, 208 | "outputs": [], 209 | "source": [ 210 | "from gluonts.model.seasonal_naive import *\n", 211 | "from gluonts.evaluation import Evaluator\n", 212 | "\n", 213 | "naive_predictor = SeasonalNaivePredictor(freq='5min', \n", 214 | " prediction_length=36,\n", 215 | " season_length=300)" 216 | ] 217 | }, 218 | { 219 | "cell_type": "markdown", 220 | "metadata": {}, 221 | "source": [ 222 | "Below we are using [gluonts.evaluation.Evaluator](https://gluon-ts.mxnet.io/api/gluonts/gluonts.evaluation.html) to create an aggregated evaluation metrics of the model we have trained. It produces some commonly used error metrics such as MSE, MASE, symmetric MAPE, RMSE, and (weighted) quantile losses. \n", 223 | "\n", 224 | "The Evaluator returns both a dictionary and a pandas DataFrame. You can use the python dictionary, first output, or the pandas DataFrame, the second output, depending on what you would like to do. The dictionary item includes more values." 225 | ] 226 | }, 227 | { 228 | "cell_type": "code", 229 | "execution_count": null, 230 | "metadata": {}, 231 | "outputs": [], 232 | "source": [ 233 | "forecast_it_baseline, ts_it_baseline = make_evaluation_predictions(test_data, naive_predictor, num_samples=100)\n", 234 | "forecasts_baseline = list(forecast_it_baseline)\n", 235 | "tss_baseline = list(ts_it_baseline)\n", 236 | "evaluator = Evaluator()\n", 237 | "agg_metrics_baseline, item_metrics = evaluator(iter(tss_baseline), iter(forecasts_baseline), num_series=len(test_data))" 238 | ] 239 | }, 240 | { 241 | "cell_type": "markdown", 242 | "metadata": {}, 243 | "source": [ 244 | "Now we plot the forecasts and we can see that the naive estimator just copies the values from last day. It will also just give single point forecast." 245 | ] 246 | }, 247 | { 248 | "cell_type": "code", 249 | "execution_count": null, 250 | "metadata": {}, 251 | "outputs": [], 252 | "source": [ 253 | "plot_forecast(naive_predictor, test_data)" 254 | ] 255 | }, 256 | { 257 | "cell_type": "markdown", 258 | "metadata": {}, 259 | "source": [ 260 | "Print the baseline metrics:" 261 | ] 262 | }, 263 | { 264 | "cell_type": "code", 265 | "execution_count": null, 266 | "metadata": {}, 267 | "outputs": [], 268 | "source": [ 269 | "agg_metrics_baseline" 270 | ] 271 | }, 272 | { 273 | "cell_type": "markdown", 274 | "metadata": {}, 275 | "source": [ 276 | "Which metric to choose depends on your business use case. For instance is underestimation more problematic than overestimation?\n", 277 | "\n", 278 | "For instance, the lower the Root-Mean-Squared Error (RMSE) the better - a value of 0 would indicate a perfect fit to the data. But RMSE is dependent on the scale of the data being used. Dividing the RMSE by the range of the data, gives an average error as a proportion of the data's scale. This is called the Normalized Root-Mean-Squared Error (NRMSE). However, the RMSE and NRMSE are very sensitive to outliers. \n", 279 | "\n", 280 | "Percentage errors like MAPE, sMAPE are unit-free and are frequently used to compare forecast performances between data sets.\n", 281 | "\n", 282 | "## DeepAR \n", 283 | "\n", 284 | "Amazon SageMaker DeepAR is a methodology for producing accurate probabilistic forecasts, based on training an auto-regressive recurrent network model on a large number of related time series. DeepAR produces more accurate forecasts than other state-of-the-art methods, while requiring minimal manual work.\n", 285 | "\n", 286 | "* The DeepAR algorithm first tailors a `Long Short-Term Memory` ([LSTM](https://en.wikipedia.org/wiki/Long_short-term_memory))-based recurrent neural network architecture to the data. DeepAR then produces probabilistic forecasts in the form of `Monte Carlo` simulation. \n", 287 | "* `Monte Carlo` samples are empirically generated pseudo-observations that can be used to compute consistent quantile estimates for all sub-ranges in the prediction horizon.\n", 288 | "* DeepAR also uses item-similarity to handle the `Cold Start` problem, which is to make predictions for items with little or no history at all.\n", 289 | "\n", 290 | "In this notebook you will learn how to use GluonTS to train a DeepAR model on your own dataset and a very detailed understanding of the implementation details of DeepAR won't be necessary. But if you you would like to learn more about DeepAR, then check out [this](../part4/deepar_details.ipynb) notebook or the [paper](https://arxiv.org/abs/1704.04110).\n", 291 | "\n", 292 | "To train a DeepAR in GluonTS, we first need to create an estimator object. An estimator object represents the network, contains a trainer, which in turn includes batch size, initializer, context, learning rate and other training specific hyperparameters. The estimator object also includes frequency of timestamp, prediction length to express how many steps we want to predict, and structural parameters such as number of layers. The estimator also crucially includes a `train` method. The train method is used to fit a model to a given dataset and returns a predictor object, which can be used to predict/forecast values.\n", 293 | "\n", 294 | "The frequnecy parameter needs to be the same as accepted frequencies by pandas. For more information on pandas use of frequency please refer to the [documentation of pandas date_range.](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.date_range.html)\n", 295 | "\n", 296 | "Finally the `prediction_length` is set to 36. We aim to predict tweets for the next 3 hours and as the data has `freq=5min`, we opt 36 steps, which is 36x5min = 180min or three hours." 297 | ] 298 | }, 299 | { 300 | "cell_type": "code", 301 | "execution_count": null, 302 | "metadata": {}, 303 | "outputs": [], 304 | "source": [ 305 | "from gluonts.distribution import DistributionOutput, StudentTOutput\n", 306 | "\n", 307 | "deepar_estimator = DeepAREstimator(freq=\"5min\", \n", 308 | " prediction_length=36,\n", 309 | " distr_output=StudentTOutput(),\n", 310 | " trainer=Trainer(epochs=EPOCHS))" 311 | ] 312 | }, 313 | { 314 | "cell_type": "markdown", 315 | "metadata": {}, 316 | "source": [ 317 | "DeepAR has a lot of different hyperparameters and in [lab 4](../part4/twitter_volume_sagemaker.ipynb) we will tune some of them. In this notebook we will just use the default values.\n", 318 | "\n", 319 | "\n", 320 | "We simply call `train` method of the `deepar_estimator` we just created and pass our iterable training data to the train method. The output is a predictor object. " 321 | ] 322 | }, 323 | { 324 | "cell_type": "code", 325 | "execution_count": null, 326 | "metadata": {}, 327 | "outputs": [], 328 | "source": [ 329 | "deepar_predictor = deepar_estimator.train(training_data=training_data)" 330 | ] 331 | }, 332 | { 333 | "cell_type": "markdown", 334 | "metadata": {}, 335 | "source": [ 336 | "### Test the model\n", 337 | "\n", 338 | "We use the `plot_forecast` function that was implemented earlier in this notebook and pass predictor object and test data. You will notice the green print in the forecast in different confidence intervals." 339 | ] 340 | }, 341 | { 342 | "cell_type": "code", 343 | "execution_count": null, 344 | "metadata": {}, 345 | "outputs": [], 346 | "source": [ 347 | "plot_forecast(predictor=deepar_predictor, test_data=test_data)" 348 | ] 349 | }, 350 | { 351 | "cell_type": "markdown", 352 | "metadata": {}, 353 | "source": [ 354 | "### Save the model\n", 355 | "Both training and prediction networks can be saved using `estimator.serialize_prediction_net` and `estimator.serialize` respectively." 356 | ] 357 | }, 358 | { 359 | "cell_type": "code", 360 | "execution_count": null, 361 | "metadata": {}, 362 | "outputs": [], 363 | "source": [ 364 | "import os\n", 365 | "os.makedirs('deepar', exist_ok=True)\n", 366 | "deepar_predictor.serialize_prediction_net(pathlib.Path('deepar'))\n", 367 | "deepar_predictor.serialize(pathlib.Path('deepar'))" 368 | ] 369 | }, 370 | { 371 | "cell_type": "markdown", 372 | "metadata": {}, 373 | "source": [ 374 | "### Evaluation\n", 375 | "Below we are using `gluonts.evaluation.Evaluator` to create an aggregated evaluation metrics of the model we have trained. The `Evaluator` accepts predictions and calculates multiple evaluation metrics such as \"MSE\" and \"Quantile Loss\". The `Evaluator` returns both a dictionary and a pandas DataFrame. You can use the python dictionary, first output, or the pandas DataFrame, the second output, depending on what you would like to do. The dictionary item includes more values." 376 | ] 377 | }, 378 | { 379 | "cell_type": "code", 380 | "execution_count": null, 381 | "metadata": {}, 382 | "outputs": [], 383 | "source": [ 384 | "from gluonts.evaluation.backtest import make_evaluation_predictions\n", 385 | "from gluonts.evaluation import Evaluator\n", 386 | "\n", 387 | "forecast_it, ts_it = make_evaluation_predictions(dataset=test_data, \n", 388 | " predictor=deepar_predictor, \n", 389 | " num_samples=100)\n", 390 | "deepar_agg_metrics, item_metrics = Evaluator(quantiles=[0.1, 0.5, 0.9])(\n", 391 | " ts_it, \n", 392 | " forecast_it, \n", 393 | " num_series=len(training_data))\n", 394 | "deepar_agg_metrics" 395 | ] 396 | }, 397 | { 398 | "cell_type": "markdown", 399 | "metadata": {}, 400 | "source": [ 401 | "We can see that DeepAR produces much better predictions than the naive estimator. " 402 | ] 403 | }, 404 | { 405 | "cell_type": "code", 406 | "execution_count": null, 407 | "metadata": {}, 408 | "outputs": [], 409 | "source": [ 410 | "print(deepar_agg_metrics[\"MSE\"],agg_metrics_baseline[\"MSE\"])" 411 | ] 412 | }, 413 | { 414 | "cell_type": "markdown", 415 | "metadata": {}, 416 | "source": [ 417 | "If you still have time left you can proceed to the next section, where you will train a multi layer perceptron." 418 | ] 419 | }, 420 | { 421 | "cell_type": "markdown", 422 | "metadata": {}, 423 | "source": [ 424 | "### Additional: Comparison with Mulitlayer Perceptron\n", 425 | "We now use another estimator, `SimpleFeedForwardEstimator`, to make the same forecast. This model is using a simple MLP or a feed forward network to reach the same goal. At the end we shall compare the results of the models." 426 | ] 427 | }, 428 | { 429 | "cell_type": "code", 430 | "execution_count": null, 431 | "metadata": {}, 432 | "outputs": [], 433 | "source": [ 434 | "mlp_estimator = SimpleFeedForwardEstimator(freq=\"5min\", \n", 435 | " prediction_length=36, \n", 436 | " trainer=Trainer(epochs=EPOCHS))\n", 437 | "mlp_predictor = mlp_estimator.train(training_data=training_data)" 438 | ] 439 | }, 440 | { 441 | "cell_type": "code", 442 | "execution_count": null, 443 | "metadata": {}, 444 | "outputs": [], 445 | "source": [ 446 | "plot_forecast(predictor=mlp_predictor, test_data=test_data)" 447 | ] 448 | }, 449 | { 450 | "cell_type": "markdown", 451 | "metadata": {}, 452 | "source": [ 453 | "\n", 454 | "The code snippet below, is using the same mechanism we have used before for evaluation, except the function accepts data and a list of predictors as well as a textual name for the predictors to use as column name in the pandas DataFrame output. It then loops over predictors, performs evaluation, converts the evaluation dictionary into a pandas DataFrame, and appends the output of evaluation to a dataframe as as a new column." 455 | ] 456 | }, 457 | { 458 | "cell_type": "code", 459 | "execution_count": null, 460 | "metadata": {}, 461 | "outputs": [], 462 | "source": [ 463 | "from gluonts.evaluation.backtest import make_evaluation_predictions\n", 464 | "from gluonts.evaluation import Evaluator\n", 465 | "def evaluat_models_from_dict(data, predictors, predictor_names, num_samples=100):\n", 466 | " '''\n", 467 | " Comparing results of multiple models.\n", 468 | " Parameters:\n", 469 | " data: the dataset on which we are performing the evaluation.\n", 470 | " predictors: A list of predictor objects\n", 471 | " predictor_names: A list of textual names for the predictors that have an ordered one-to-one\n", 472 | " relationship with the predictors.\n", 473 | " num_samples (default=100): what sample size from the evaluation dataset.\n", 474 | " Output: pandas dataframe to an evaluation column per predictor.\n", 475 | " '''\n", 476 | " df = pd.DataFrame()\n", 477 | " for (predictor, predictor_name) in zip(predictors, predictor_names):\n", 478 | " forecast_it, ts_it = make_evaluation_predictions(data, \n", 479 | " predictor=predictor, \n", 480 | " num_samples=num_samples)\n", 481 | " deepar_agg_metrics, item_metrics = Evaluator(quantiles=[0.1, 0.5, 0.9])(\n", 482 | " ts_it, \n", 483 | " forecast_it, \n", 484 | " num_series=len(data))\n", 485 | " \n", 486 | " evaluation = pd.DataFrame.from_dict(deepar_agg_metrics, orient='index', columns=[predictor_name])\n", 487 | " if df.empty:\n", 488 | " df = evaluation.copy()\n", 489 | " else:\n", 490 | " df.insert(loc=len(df.columns), column=predictor_name, value=evaluation.values)\n", 491 | " return df\n", 492 | "evaluat_models_from_dict(data=test_data, \n", 493 | " predictors=[deepar_predictor, mlp_predictor, naive_predictor], \n", 494 | " predictor_names = ['deepar', 'mlp', 'naive predictor'])" 495 | ] 496 | }, 497 | { 498 | "cell_type": "markdown", 499 | "metadata": {}, 500 | "source": [ 501 | "### Additional: Accessing weights and model parameters\n", 502 | "You can get access to the network structure and parameters. `DeepARNetwork` is derived from `mxnet.gluon.block.HybridBlock`. " 503 | ] 504 | }, 505 | { 506 | "cell_type": "code", 507 | "execution_count": null, 508 | "metadata": {}, 509 | "outputs": [], 510 | "source": [ 511 | "gluonts.model.deepar._network.DeepARTrainingNetwork.__bases__[0].__bases__[0]" 512 | ] 513 | }, 514 | { 515 | "cell_type": "markdown", 516 | "metadata": {}, 517 | "source": [ 518 | "We can now call `DeepARTrainingNetwork.collect_params()`, which returns a `mxnet.gluon.parameter.ParameterDict` object. for more information how to query `ParameterDict`, plese refer to [mxnet documentation.](https://mxnet.incubator.apache.org/api/python/gluon/gluon.html#mxnet.gluon.ParameterDict)" 519 | ] 520 | }, 521 | { 522 | "cell_type": "code", 523 | "execution_count": null, 524 | "metadata": {}, 525 | "outputs": [], 526 | "source": [ 527 | "deepar_predictor.prediction_net.collect_params() " 528 | ] 529 | }, 530 | { 531 | "cell_type": "code", 532 | "execution_count": null, 533 | "metadata": {}, 534 | "outputs": [], 535 | "source": [] 536 | } 537 | ], 538 | "metadata": { 539 | "kernelspec": { 540 | "display_name": "conda_python3", 541 | "language": "python", 542 | "name": "conda_python3" 543 | }, 544 | "language_info": { 545 | "codemirror_mode": { 546 | "name": "ipython", 547 | "version": 3 548 | }, 549 | "file_extension": ".py", 550 | "mimetype": "text/x-python", 551 | "name": "python", 552 | "nbconvert_exporter": "python", 553 | "pygments_lexer": "ipython3", 554 | "version": "3.6.5" 555 | } 556 | }, 557 | "nbformat": 4, 558 | "nbformat_minor": 4 559 | } 560 | -------------------------------------------------------------------------------- /notebooks/part4/deepar_details.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "## ....To Read While the Endpoint is Deploying or While your Hyperparameter Tuning Jobs are Running....\n", 8 | "\n", 9 | "It is important to elaborate on the DeepAR model's architecture by walking through an example. When interested in quantifying the confidence of the estimates produced, then it's probabilistic forecasts that are wanted. If we have real-valued, it is recommended to opt for the Gaussian likelihood:\n", 10 | "$$\\ell(y_t|\\mu_t,\\sigma_t)=\\frac{1}{\\sqrt{2\\pi\\sigma^2}}\\exp{\\frac{-(y_t-\\mu_t)^2}{2\\sigma^2}}.$$\n", 11 | "\n", 12 | "$\\theta$ represents the `parameters of the likelihood`. In the case of Gaussian, $\\theta_t$ will represent the mean and standard deviation: $$\\theta_t = \\{\\mu_{t},\\sigma_{t}\\}.$$\n", 13 | "\n", 14 | "The neural network’s last hidden layer results in $h_{d,t}$. This $h_{d,t}$ will undergo 1 activation function per likelihood parameter. For example, for the Gaussian likelihood, $h_{d,t}$ is transformed by an affine activation function to get the mean:\n", 15 | "$$\\mu_{t} = w_{\\mu}^T h_{d,t} + b_{\\mu},$$\n", 16 | "and then $h$ is transformed by a softplus activation to get the standard deviation:\n", 17 | "$$\\sigma_t = \\log\\left(1 + \\exp(w_{\\sigma}^T h_{d,t} + b_{\\sigma})\\right).$$\n", 18 | "\n", 19 | "The `activation parameters` are the $w_{\\mu},b_{\\mu},w_{\\sigma},b_{\\sigma}$ parameters within the activation functions. The neural network is trained to learn the fixed constants of the activation parameters. Since the $h_{d,t}$ output vary given each time-step's input, this still allows the likelihood parameters to vary over time, and therefore capture dynamic behaviors in the time series data.\n", 20 | "\n", 21 | "![DeepAR Training](images/training.png)\n", 22 | "\n", 23 | "From the above diagram, the input at each time-step is the data point preceding the current time-step’s data, as well as the previous network’s output. For simplicity, on this diagram you aren’t shown covariates which would also be inputs.\n", 24 | "\n", 25 | "The LSTM layers and the final hidden layer produces the $h_{i,t}$ value, which will undergo an activation function for each parameter of the specified likelihood. To learn the activation function parameters, the neural network takes the $h_{i,t}$ at time $t$ and the data up until time $t$, and performs Stochastic Gradient Descent (SGD) to yield the activation parameters which maximize the likelihood at time $t$. The output layer uses the SGD-optimized activation functions to output the maximum likelihood parameters.\n", 26 | "\n", 27 | "This is how DeepAR trains its model to your data input. Now you want DeepAR to give you probabilistic forecasts for the next time-step.\n", 28 | "\n", 29 | "![DeepAR Forecast](images/prediction.png)\n", 30 | "\n", 31 | "During prediction the input of the current time will be processed by the trained LSTM layers, and subsequently get activated by the optimized activation functions to output the maximum-likelihood theta parameters at time $t+1$. \n", 32 | "\n", 33 | "Now that DeepAR has completed the likelihood with its parameter estimates, DeepAR can simulate `Monte Carlo (MC) samples` from this likelihood and produce an empirical distribution for the predicted datapoint - the probabilistic forecasts. The Monte Carlo samples produced at time $t+1$ are used as input for time $t+2$, etc, until the end of the prediction horizon. " 34 | ] 35 | }, 36 | { 37 | "cell_type": "code", 38 | "execution_count": null, 39 | "metadata": {}, 40 | "outputs": [], 41 | "source": [] 42 | } 43 | ], 44 | "metadata": { 45 | "kernelspec": { 46 | "display_name": "Python 3", 47 | "language": "python", 48 | "name": "python3" 49 | }, 50 | "language_info": { 51 | "codemirror_mode": { 52 | "name": "ipython", 53 | "version": 3 54 | }, 55 | "file_extension": ".py", 56 | "mimetype": "text/x-python", 57 | "name": "python", 58 | "nbconvert_exporter": "python", 59 | "pygments_lexer": "ipython3", 60 | "version": "3.7.3" 61 | } 62 | }, 63 | "nbformat": 4, 64 | "nbformat_minor": 2 65 | } 66 | -------------------------------------------------------------------------------- /notebooks/part4/entry_point/train.py: -------------------------------------------------------------------------------- 1 | import os 2 | os.system('pip install pandas') 3 | os.system('pip install gluonts') 4 | import pandas as pd 5 | import pathlib 6 | import gluonts 7 | import numpy as np 8 | import argparse 9 | import json 10 | from mxnet import gpu, cpu 11 | from mxnet.context import num_gpus 12 | from gluonts.dataset.util import to_pandas 13 | from gluonts.model.deepar import DeepAREstimator 14 | from gluonts.model.simple_feedforward import SimpleFeedForwardEstimator 15 | from gluonts.evaluation.backtest import make_evaluation_predictions, backtest_metrics 16 | from gluonts.evaluation import Evaluator 17 | from gluonts.model.predictor import Predictor 18 | from gluonts.dataset.common import ListDataset 19 | from gluonts.trainer import Trainer 20 | 21 | 22 | def train(epochs, prediction_length, num_layers, dropout_rate): 23 | 24 | #create train dataset 25 | df = pd.read_csv(filepath_or_buffer=os.environ['SM_CHANNEL_TRAIN'] + "/train.csv", header=0, index_col=0) 26 | 27 | training_data = ListDataset([{"start": df.index[0], 28 | "target": df.value[:]}], 29 | freq="5min") 30 | 31 | #define DeepAR estimator 32 | deepar_estimator = DeepAREstimator(freq="5min", 33 | prediction_length=prediction_length, 34 | dropout_rate=dropout_rate, 35 | num_layers=num_layers, 36 | trainer=Trainer(epochs=epochs)) 37 | 38 | #train the model 39 | deepar_predictor = deepar_estimator.train(training_data=training_data) 40 | 41 | #create test dataset 42 | df = pd.read_csv(filepath_or_buffer=os.environ['SM_CHANNEL_TEST'] + "/test.csv", header=0, index_col=0) 43 | 44 | test_data = ListDataset([{"start": df.index[0], 45 | "target": df.value[:]}], 46 | freq="5min") 47 | 48 | #evaluate trained model on test data 49 | forecast_it, ts_it = make_evaluation_predictions(test_data, deepar_predictor, num_samples=100) 50 | forecasts = list(forecast_it) 51 | tss = list(ts_it) 52 | evaluator = Evaluator(quantiles=[0.1, 0.5, 0.9]) 53 | agg_metrics, item_metrics = evaluator(iter(tss), iter(forecasts), num_series=len(test_data)) 54 | 55 | print("MSE:", agg_metrics["MSE"]) 56 | 57 | #save the model 58 | deepar_predictor.serialize(pathlib.Path(os.environ['SM_MODEL_DIR'])) 59 | 60 | return deepar_predictor 61 | 62 | 63 | def model_fn(model_dir): 64 | path = pathlib.Path(model_dir) 65 | predictor = Predictor.deserialize(path) 66 | 67 | return predictor 68 | 69 | 70 | def transform_fn(model, data, content_type, output_content_type): 71 | 72 | data = json.loads(data) 73 | df = pd.DataFrame(data) 74 | 75 | test_data = ListDataset([{"start": df.index[0], 76 | "target": df.value[:]}], 77 | freq="5min") 78 | 79 | forecast_it, ts_it = make_evaluation_predictions(test_data, model, num_samples=100) 80 | #agg_metrics, item_metrics = Evaluator()(ts_it, forecast_it, num_series=len(test_data)) 81 | #response_body = json.dumps(agg_metrics) 82 | response_body = json.dumps({'predictions':list(forecast_it)[0].samples.tolist()[0]}) 83 | return response_body, output_content_type 84 | 85 | 86 | def parse_args(): 87 | parser = argparse.ArgumentParser() 88 | parser.add_argument('--epochs', type=int, default=10) 89 | parser.add_argument('--prediction_length', type=int, default=12) 90 | parser.add_argument('--num_layers', type=int, default=2) 91 | parser.add_argument('--dropout_rate', type=float, default=0.1) 92 | return parser.parse_args() 93 | 94 | if __name__ == '__main__': 95 | args = parse_args() 96 | train(args.epochs, args.prediction_length, args.num_layers, args.dropout_rate) 97 | 98 | -------------------------------------------------------------------------------- /notebooks/part4/images/prediction.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-sagemaker-time-series-prediction-using-gluonts/28541a616924c042e062c2149cd3a7236f20b32b/notebooks/part4/images/prediction.png -------------------------------------------------------------------------------- /notebooks/part4/images/training.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/aws-samples/amazon-sagemaker-time-series-prediction-using-gluonts/28541a616924c042e062c2149cd3a7236f20b32b/notebooks/part4/images/training.png -------------------------------------------------------------------------------- /notebooks/part4/twitter_volume_sagemaker.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Hyperparameter Tuning with SageMaker Automatic Model Tuner\n", 8 | "\n", 9 | "In [part3](../part3/twitter_volume_forecast.ipynb) we have trained a DeepAR model. Apart from prediction_length, time freqency and number of epochs we did not specify any other hyperparameters. DeepAR has many hyperparameters and in this section we will use SageMaker automatic model tuner to find the right set for our model. Here is a short list of some hyperparameters and their default values in GluonTS DeepAR:\n", 10 | "\n", 11 | "| Hyperparameters | Value |\n", 12 | "|--------------------------|---------------------------|\n", 13 | "| epochs | 100 |\n", 14 | "| context_length | prediction_length |\n", 15 | "| batch size | 32 |\n", 16 | "| learning rate | $1e-3$ |\n", 17 | "| LSTM layers | 2 |\n", 18 | "| LSTM nodes | 40 |\n", 19 | "| likelihood | StudentTOutput() |\n", 20 | "\n", 21 | "\n", 22 | "We also need to choose a likelihood model. For example, we choose negative binomial likelihood or StudentT for count data. Other likelihood models can also readily be used as long as samples from the distribution can cheaply be obtained and the log-likelihood and its gradients with respect to the parameters can be evaluated. For example:\n", 23 | "\n", 24 | "- **Gaussian:** Use for real-valued data.\n", 25 | "- **Beta:** Use for real-valued targets between 0 and 1 inclusive.\n", 26 | "- **Negative-binomial:** Use for count data (non-negative integers).\n", 27 | "- **Student-T:** An alternative for real-valued data that works well for bursty data.\n", 28 | "- **Deterministic-L1:** A loss function that does not estimate uncertainty and only learns a point forecast.\n", 29 | "\n", 30 | "Refer to the [documentation](https://gluon-ts.mxnet.io/api/gluonts/gluonts.model.deepar.html) for a full description of the available parameters. In this notebook your will learn how to train your GluonTS model on Amazon SageMaker and to tune it with automatic model tuner." 31 | ] 32 | }, 33 | { 34 | "cell_type": "code", 35 | "execution_count": null, 36 | "metadata": {}, 37 | "outputs": [], 38 | "source": [ 39 | "import pandas as pd\n", 40 | "import gluonts\n", 41 | "import numpy as np\n", 42 | "import matplotlib.pyplot as plt\n", 43 | "import pathlib\n", 44 | "import json\n", 45 | "import boto3\n", 46 | "import s3fs\n", 47 | "import csv\n", 48 | "import sagemaker" 49 | ] 50 | }, 51 | { 52 | "cell_type": "markdown", 53 | "metadata": {}, 54 | "source": [ 55 | "### Upload data to Amazon S3\n", 56 | "In order to run the model training with Amazon SageMaker, we need to upload our train and test data to Amazon S3. In the following code cell, we define SageMaker default bucket where data will be uploaded to. " 57 | ] 58 | }, 59 | { 60 | "cell_type": "code", 61 | "execution_count": null, 62 | "metadata": {}, 63 | "outputs": [], 64 | "source": [ 65 | "sagemaker_session = sagemaker.Session()\n", 66 | "s3_bucket = sagemaker_session.default_bucket()\n", 67 | "\n", 68 | "s3_train_data_path = \"s3://{}/gluonts/train\".format(s3_bucket)\n", 69 | "s3_test_data_path = \"s3://{}/gluonts/test\".format(s3_bucket)\n", 70 | "\n", 71 | "print(\"Data will be uploaded to: \", s3_bucket)" 72 | ] 73 | }, 74 | { 75 | "cell_type": "markdown", 76 | "metadata": {}, 77 | "source": [ 78 | "Now we download the file and split it into training and test data. Afterwards we write it to a csv." 79 | ] 80 | }, 81 | { 82 | "cell_type": "code", 83 | "execution_count": null, 84 | "metadata": {}, 85 | "outputs": [], 86 | "source": [ 87 | "url = \"https://raw.githubusercontent.com/numenta/NAB/master/data/realTweets/Twitter_volume_AMZN.csv\"\n", 88 | "df = pd.read_csv(filepath_or_buffer=url, header=0, index_col=0)\n", 89 | "\n", 90 | "train = df[: \"2015-04-05 00:00:00\"]\n", 91 | "train.to_csv(\"train.csv\")\n", 92 | "\n", 93 | "test = df[: \"2015-04-15 00:00:00\"]\n", 94 | "test.to_csv(\"test.csv\")" 95 | ] 96 | }, 97 | { 98 | "cell_type": "markdown", 99 | "metadata": {}, 100 | "source": [ 101 | "The following function will create a `train` and `test` folder in the S3 bucket and upload the csv files." 102 | ] 103 | }, 104 | { 105 | "cell_type": "code", 106 | "execution_count": null, 107 | "metadata": {}, 108 | "outputs": [], 109 | "source": [ 110 | "s3 = boto3.resource('s3')\n", 111 | "def copy_to_s3(local_file, s3_path, override=False):\n", 112 | " assert s3_path.startswith('s3://')\n", 113 | " split = s3_path.split('/')\n", 114 | " bucket = split[2]\n", 115 | " path = '/'.join(split[3:])\n", 116 | " buk = s3.Bucket(bucket)\n", 117 | " \n", 118 | " if len(list(buk.objects.filter(Prefix=path))) > 0:\n", 119 | " if not override:\n", 120 | " print('File s3://{}/{} already exists.\\nSet override to upload anyway.\\n'.format(s3_bucket, s3_path))\n", 121 | " return\n", 122 | " else:\n", 123 | " print('Overwriting existing file')\n", 124 | " with open(local_file, 'rb') as data:\n", 125 | " print('Uploading file to {}'.format(s3_path))\n", 126 | " buk.put_object(Key=path, Body=data)\n", 127 | " \n", 128 | "copy_to_s3(\"train.csv\", s3_train_data_path + \"/train.csv\")\n", 129 | "copy_to_s3(\"test.csv\", s3_test_data_path + \"/test.csv\")" 130 | ] 131 | }, 132 | { 133 | "cell_type": "markdown", 134 | "metadata": {}, 135 | "source": [ 136 | "Let's have a look to what we just wrote to S3. With `s3fs` we can have a look on the files in the bucket." 137 | ] 138 | }, 139 | { 140 | "cell_type": "code", 141 | "execution_count": null, 142 | "metadata": {}, 143 | "outputs": [], 144 | "source": [ 145 | "s3filesystem = s3fs.S3FileSystem()\n", 146 | "with s3filesystem.open(s3_train_data_path + \"/train.csv\", 'rb') as fp:\n", 147 | " print(fp.readline().decode(\"utf-8\")[:100] + \"...\")" 148 | ] 149 | }, 150 | { 151 | "cell_type": "markdown", 152 | "metadata": {}, 153 | "source": [ 154 | "### Train DeepAR model with Amazon SageMaker\n", 155 | "\n", 156 | "Since SageMaker will automatically spin up instances for us, we need to provide a role. " 157 | ] 158 | }, 159 | { 160 | "cell_type": "code", 161 | "execution_count": null, 162 | "metadata": {}, 163 | "outputs": [], 164 | "source": [ 165 | "import sagemaker\n", 166 | "from sagemaker.mxnet import MXNet\n", 167 | "\n", 168 | "sagemaker_session = sagemaker.Session()\n", 169 | "role = sagemaker.get_execution_role()" 170 | ] 171 | }, 172 | { 173 | "cell_type": "markdown", 174 | "metadata": {}, 175 | "source": [ 176 | "Now we define the MXNet estimator. An [estimator](https://sagemaker.readthedocs.io/en/stable/estimators.html) is a higher level interface to define the SageMaker training. It takes several parameters like the [training](entry_point/train.py) script, which defines our DeepAR model. We indicate the train instance type on which we want to execute our model training. Here we choose `ml.m5.xlarge` which is a CPU instance. We need to provide the role so that SageMaker can spin up the instance for us. We also indicate the framework and python version for MXNet. Afterwards we provide a dictionary of hyperparameters that will be parsed in the [training](entry_point/train.py) script to set the hyperparameters of our model. During hyperparameter tuning SageMaker will adjust the hyperparameters passed into our training job." 177 | ] 178 | }, 179 | { 180 | "cell_type": "code", 181 | "execution_count": null, 182 | "metadata": {}, 183 | "outputs": [], 184 | "source": [ 185 | "mxnet_estimator = MXNet(entry_point='train.py',\n", 186 | " source_dir='entry_point',\n", 187 | " role=role,\n", 188 | " train_instance_type='ml.m5.xlarge',\n", 189 | " train_instance_count=1,\n", 190 | " framework_version='1.4.1', py_version='py3',\n", 191 | " hyperparameters={\n", 192 | " 'epochs': 1, \n", 193 | " 'prediction_length':12,\n", 194 | " 'num_layers':2, \n", 195 | " 'dropout_rate': 0.2,\n", 196 | " })" 197 | ] 198 | }, 199 | { 200 | "cell_type": "markdown", 201 | "metadata": {}, 202 | "source": [ 203 | "We are ready to start the training job. Once we call `fit`, SageMaker will spin up an `ml.m5.xlarge` instance, download the MXNet docker image, download the train and test data from Amazon S3 and execute the `train` function from our `train.py` file. \n", 204 | "\n", 205 | "While the model is training you may want to have a look at [train.py](entry_point/train.py) file. The file follows a certain structure and has the following functions:\n", 206 | "- `train`: defines the training procedure as we defined it in [lab 3](../notebooks/twitter_volume_forecast.ipynb) So in our case it creates the ListDataset, the DeepAR estimator and performs the training. It also performs the evaluation and prints the MSE metric. This is necessary for the hyperparameter tuning later on.\n", 207 | "- `model_fn`: used for inference. Once the model is trained we can deploy it and this function will load the trained model.\n", 208 | "- `transform_fn`: used for inference. If we send requests to the endpoint, the data will by default be encoded as json string. We decode the data from json into a Pandas data frame. We then create the ListDataset and perform inference. The forecasts will be sent back as a json string." 209 | ] 210 | }, 211 | { 212 | "cell_type": "code", 213 | "execution_count": null, 214 | "metadata": {}, 215 | "outputs": [], 216 | "source": [ 217 | "mxnet_estimator.fit({\"train\": s3_train_data_path, \"test\": s3_test_data_path})" 218 | ] 219 | }, 220 | { 221 | "cell_type": "markdown", 222 | "metadata": {}, 223 | "source": [ 224 | "### SageMaker Automatic Model Tuner" 225 | ] 226 | }, 227 | { 228 | "cell_type": "markdown", 229 | "metadata": {}, 230 | "source": [ 231 | "Now that we are able to run our DeepAR model with SageMaker, we can start tuning its hyperparameter. In the following section we define the `HyperparameterTuner`, which takes the following hyperparameters:\n", 232 | "- `epochs`: number of training epochs. If this value is too large we may overfit the training data, which means the model achieves good performance on the trasining dataset but bad performance on the test dataset.\n", 233 | "- `prediction_length`: how many time units shall the model predict\n", 234 | "- `num_layers`: number of RNN layers\n", 235 | "- `dropout_rate`: dropouts help to regularize the training because they randomly switch off neurons. \n", 236 | "\n", 237 | "You can find more information about DeepAR parameters [here](https://gluon-ts.mxnet.io/api/gluonts/gluonts.model.deepar.html) \n", 238 | "\n", 239 | "Next we have to indicate the metric we want to optimize on. We have to make sure that our training job prints those metrics. [train.py](entry_point/train.py) prints the MSE value of evaluated test dataset. These printouts will appear in Cloudwatch and the automatic model tuner will then retrieve those outputs by using the regular expression indicated in `Regex`. \n", 240 | "Next we indicate the `max_jobs` and `max_parallel_jobs`. Here we will run 10 jobs in total and in each step we will start 5 parallel jobs." 241 | ] 242 | }, 243 | { 244 | "cell_type": "code", 245 | "execution_count": null, 246 | "metadata": {}, 247 | "outputs": [], 248 | "source": [ 249 | "from sagemaker.tuner import HyperparameterTuner, ContinuousParameter, IntegerParameter \n", 250 | "\n", 251 | "tuner = HyperparameterTuner(estimator=mxnet_estimator, \n", 252 | " objective_metric_name='loss',\n", 253 | " hyperparameter_ranges={\n", 254 | " 'epochs': IntegerParameter(5,20),\n", 255 | " 'prediction_length':IntegerParameter(5,20),\n", 256 | " 'num_layers': IntegerParameter(1, 5),\n", 257 | " 'dropout_rate': ContinuousParameter(0, 0.5) },\n", 258 | " metric_definitions=[{'Name': 'loss', 'Regex': \"MSE: ([0-9\\\\.]+)\"}],\n", 259 | " max_jobs=10,\n", 260 | " max_parallel_jobs=5,\n", 261 | " objective_type='Minimize')" 262 | ] 263 | }, 264 | { 265 | "cell_type": "markdown", 266 | "metadata": {}, 267 | "source": [ 268 | "`tuner.fit` will start the automatic model tuner. You can go now to the SageMaker console and check the training jobs or proceed to the next cells, to get some real time results from the jobs. \n", 269 | "\n", 270 | "The search space grows exponentially with the number of hyperparameters. Assuming 5 parameters where each one has 10 discrete options we end up with $10^5$ possible combinations. Clearly we do not want to run $10^5$ jobs. Automatic model tuner will use per default Bayesian optimization which is a combination of explore and exploit. That means after each training job it will evaluate whether to jump into a new area of the search space (explore) or whether to further exploit the local search space. You can find some more information [here](https://docs.aws.amazon.com/sagemaker/latest/dg/automatic-model-tuning-how-it-works.html)" 271 | ] 272 | }, 273 | { 274 | "cell_type": "code", 275 | "execution_count": null, 276 | "metadata": {}, 277 | "outputs": [], 278 | "source": [ 279 | "tuner.fit({'train': s3_train_data_path, \"test\": s3_test_data_path})" 280 | ] 281 | }, 282 | { 283 | "cell_type": "markdown", 284 | "metadata": {}, 285 | "source": [ 286 | "We can track the status of the hyperparameter tuning jobs by running the following code. Get the name of your job from the sagemaker console." 287 | ] 288 | }, 289 | { 290 | "cell_type": "code", 291 | "execution_count": null, 292 | "metadata": {}, 293 | "outputs": [], 294 | "source": [ 295 | "tuning_job_name = tuner.latest_tuning_job.job_name" 296 | ] 297 | }, 298 | { 299 | "cell_type": "markdown", 300 | "metadata": {}, 301 | "source": [ 302 | "Now we retrieve information about the training jobs from SageMaker and we can see how many have already completed." 303 | ] 304 | }, 305 | { 306 | "cell_type": "code", 307 | "execution_count": null, 308 | "metadata": {}, 309 | "outputs": [], 310 | "source": [ 311 | "sage_client = boto3.Session().client('sagemaker')\n", 312 | "\n", 313 | "# run this cell to check current status of hyperparameter tuning job\n", 314 | "tuning_job_result = sage_client.describe_hyper_parameter_tuning_job(HyperParameterTuningJobName=tuning_job_name)\n", 315 | "\n", 316 | "status = tuning_job_result['HyperParameterTuningJobStatus']\n", 317 | "if status != 'Completed':\n", 318 | " print('Reminder: the tuning job has not been completed.')\n", 319 | " \n", 320 | "job_count = tuning_job_result['TrainingJobStatusCounters']['Completed']\n", 321 | "print(\"%d training jobs have completed\" % job_count)\n", 322 | " \n", 323 | "is_minimize = (tuning_job_result['HyperParameterTuningJobConfig']['HyperParameterTuningJobObjective']['Type'] != 'Maximize')\n", 324 | "objective_name = tuning_job_result['HyperParameterTuningJobConfig']['HyperParameterTuningJobObjective']['MetricName']" 325 | ] 326 | }, 327 | { 328 | "cell_type": "markdown", 329 | "metadata": {}, 330 | "source": [ 331 | "In the following cell, we retrieve information about training jobs that have already finished. We will plot their hyperparameters versus objective metric." 332 | ] 333 | }, 334 | { 335 | "cell_type": "code", 336 | "execution_count": null, 337 | "metadata": {}, 338 | "outputs": [], 339 | "source": [ 340 | "import pandas as pd\n", 341 | "\n", 342 | "job_analytics = sagemaker.HyperparameterTuningJobAnalytics(tuning_job_name)\n", 343 | "\n", 344 | "full_df = job_analytics.dataframe()\n", 345 | "\n", 346 | "if len(full_df) > 0:\n", 347 | " df = full_df[full_df['FinalObjectiveValue'] > -float('inf')]\n", 348 | " if len(df) > 0:\n", 349 | " df = df.sort_values('FinalObjectiveValue', ascending=is_minimize)\n", 350 | " print(\"Number of training jobs with valid objective: %d\" % len(df))\n", 351 | " print({\"lowest\":min(df['FinalObjectiveValue']),\"highest\": max(df['FinalObjectiveValue'])})\n", 352 | " pd.set_option('display.max_colwidth', -1) # Don't truncate TrainingJobName \n", 353 | " else:\n", 354 | " print(\"No training jobs have reported valid results yet.\")\n", 355 | " \n", 356 | "df" 357 | ] 358 | }, 359 | { 360 | "cell_type": "markdown", 361 | "metadata": {}, 362 | "source": [ 363 | "Once the hyperparameter tuning job has finished we will plot all results. " 364 | ] 365 | }, 366 | { 367 | "cell_type": "code", 368 | "execution_count": null, 369 | "metadata": {}, 370 | "outputs": [], 371 | "source": [ 372 | "import bokeh\n", 373 | "import bokeh.io\n", 374 | "bokeh.io.output_notebook()\n", 375 | "from bokeh.plotting import figure, show\n", 376 | "from bokeh.models import HoverTool\n", 377 | "\n", 378 | "ranges = job_analytics.tuning_ranges\n", 379 | "figures = []\n", 380 | "\n", 381 | "class HoverHelper():\n", 382 | "\n", 383 | " def __init__(self, tuning_analytics):\n", 384 | " self.tuner = tuning_analytics\n", 385 | "\n", 386 | " def hovertool(self):\n", 387 | " tooltips = [\n", 388 | " (\"FinalObjectiveValue\", \"@FinalObjectiveValue\"),\n", 389 | " (\"TrainingJobName\", \"@TrainingJobName\"),\n", 390 | " ]\n", 391 | " for k in self.tuner.tuning_ranges.keys():\n", 392 | " tooltips.append( (k, \"@{%s}\" % k) )\n", 393 | "\n", 394 | " ht = HoverTool(tooltips=tooltips)\n", 395 | " return ht\n", 396 | "\n", 397 | " def tools(self, standard_tools='pan,crosshair,wheel_zoom,zoom_in,zoom_out,undo,reset'):\n", 398 | " return [self.hovertool(), standard_tools]\n", 399 | "\n", 400 | "hover = HoverHelper(job_analytics)\n", 401 | "\n", 402 | "for hp_name, hp_range in ranges.items():\n", 403 | " categorical_args = {}\n", 404 | " if hp_range.get('Values'):\n", 405 | " # This is marked as categorical. Check if all options are actually numbers.\n", 406 | " def is_num(x):\n", 407 | " try:\n", 408 | " float(x)\n", 409 | " return 1\n", 410 | " except:\n", 411 | " return 0 \n", 412 | " vals = hp_range['Values']\n", 413 | " if sum([is_num(x) for x in vals]) == len(vals):\n", 414 | " # Bokeh has issues plotting a \"categorical\" range that's actually numeric, so plot as numeric\n", 415 | " print(\"Hyperparameter %s is tuned as categorical, but all values are numeric\" % hp_name)\n", 416 | " else:\n", 417 | " # Set up extra options for plotting categoricals. A bit tricky when they're actually numbers.\n", 418 | " categorical_args['x_range'] = vals\n", 419 | "\n", 420 | " # Now plot it\n", 421 | " p = figure(plot_width=500, plot_height=500, \n", 422 | " title=\"Objective vs %s\" % hp_name,\n", 423 | " tools=hover.tools(),\n", 424 | " x_axis_label=hp_name, y_axis_label=objective_name,\n", 425 | " **categorical_args)\n", 426 | " p.circle(source=df, x=hp_name, y='FinalObjectiveValue')\n", 427 | " figures.append(p)\n", 428 | "show(bokeh.layouts.Column(*figures))" 429 | ] 430 | }, 431 | { 432 | "cell_type": "markdown", 433 | "metadata": {}, 434 | "source": [ 435 | "Running hyperparamter tuning jobs may take a while so in the meantime freel free to check out [this notebook](deepar_datails.ipynb) that gives more in depth details about DeepAR." 436 | ] 437 | }, 438 | { 439 | "cell_type": "markdown", 440 | "metadata": {}, 441 | "source": [ 442 | "Now that we have found a model with good hyperparameters we can deploy it. Note: This endpoint will take approximately 5-8 minutes to launch. " 443 | ] 444 | }, 445 | { 446 | "cell_type": "code", 447 | "execution_count": null, 448 | "metadata": { 449 | "scrolled": true 450 | }, 451 | "outputs": [], 452 | "source": [ 453 | "tuned_endpoint = tuner.deploy(instance_type=\"ml.m5.xlarge\", initial_instance_count=1)" 454 | ] 455 | }, 456 | { 457 | "cell_type": "markdown", 458 | "metadata": {}, 459 | "source": [ 460 | "Now we can send some test data to the endpoint, but first we convert the Numpy arrays `test.value` and `test.index` to lists and add them to a dictionary. SageMaker will encode them as a json string when they are sent to the endpoint. Let's compare how much better our predictions are:" 461 | ] 462 | }, 463 | { 464 | "cell_type": "code", 465 | "execution_count": null, 466 | "metadata": {}, 467 | "outputs": [], 468 | "source": [ 469 | "input_data = {'value': test.value.tolist(), 'timestamp': test.index.tolist() }\n", 470 | "result = tuned_endpoint.predict(input_data)" 471 | ] 472 | }, 473 | { 474 | "cell_type": "markdown", 475 | "metadata": {}, 476 | "source": [ 477 | "When we call `endpoint.predict()`, SageMaker will execute the `transform_fn` in the [train.py](entry_point/train.py) file. As discussed above, this function will decode the json string into a Pandas frame. Afterwards it creates the `ListDataset` and performs inference. The endpoint will then return forecasts. Let's have a look at the result" 478 | ] 479 | }, 480 | { 481 | "cell_type": "code", 482 | "execution_count": null, 483 | "metadata": {}, 484 | "outputs": [], 485 | "source": [ 486 | "result" 487 | ] 488 | }, 489 | { 490 | "cell_type": "markdown", 491 | "metadata": {}, 492 | "source": [ 493 | "In this notebook you have learnt how to build and train a DeepAR model with GluonTS, how to tune and deploy it with Amazon SageMaker." 494 | ] 495 | }, 496 | { 497 | "cell_type": "markdown", 498 | "metadata": {}, 499 | "source": [ 500 | "### Delete the endpoint\n", 501 | "Remember to delete your Amazon SageMaker endpoint once it is no longer needed." 502 | ] 503 | }, 504 | { 505 | "cell_type": "code", 506 | "execution_count": null, 507 | "metadata": {}, 508 | "outputs": [], 509 | "source": [ 510 | "tuned_endpoint.delete_endpoint()" 511 | ] 512 | }, 513 | { 514 | "cell_type": "markdown", 515 | "metadata": {}, 516 | "source": [ 517 | "# Challenge\n", 518 | "Now it is your turn to find even better hyperparameters for the model. Go to [documentation](https://gluon-ts.mxnet.io/api/gluonts/gluonts.model.deepar.html) and try out other hyperparameters." 519 | ] 520 | } 521 | ], 522 | "metadata": { 523 | "kernelspec": { 524 | "display_name": "conda_python3", 525 | "language": "python", 526 | "name": "conda_python3" 527 | }, 528 | "language_info": { 529 | "codemirror_mode": { 530 | "name": "ipython", 531 | "version": 3 532 | }, 533 | "file_extension": ".py", 534 | "mimetype": "text/x-python", 535 | "name": "python", 536 | "nbconvert_exporter": "python", 537 | "pygments_lexer": "ipython3", 538 | "version": "3.6.5" 539 | } 540 | }, 541 | "nbformat": 4, 542 | "nbformat_minor": 4 543 | } 544 | --------------------------------------------------------------------------------