├── LICENSE ├── README.md ├── ch02 ├── README.md ├── real-data.ipynb └── synthetic-data.ipynb ├── ch03 ├── README.md ├── mf.py └── naive-vs-ips.ipynb ├── ch04 ├── README.md ├── evaluate.py ├── loss.py ├── model.py ├── naive-vs-ips.ipynb ├── position-bias-effects.ipynb ├── theta-misspecification.ipynb ├── train.py └── utils.py ├── ch05 ├── README.md ├── evaluate.py ├── loss.py ├── model.py ├── naive-vs-ips.ipynb ├── objective-misspecification.ipynb ├── train.py └── utils.py ├── poetry.lock └── pyproject.toml /LICENSE: -------------------------------------------------------------------------------- 1 | BSD 3-Clause License 2 | 3 | Copyright (c) 2021, ghmagazine 4 | All rights reserved. 5 | 6 | Redistribution and use in source and binary forms, with or without 7 | modification, are permitted provided that the following conditions are met: 8 | 9 | 1. Redistributions of source code must retain the above copyright notice, this 10 | list of conditions and the following disclaimer. 11 | 12 | 2. Redistributions in binary form must reproduce the above copyright notice, 13 | this list of conditions and the following disclaimer in the documentation 14 | and/or other materials provided with the distribution. 15 | 16 | 3. Neither the name of the copyright holder nor the names of its 17 | contributors may be used to endorse or promote products derived from 18 | this software without specific prior written permission. 19 | 20 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 21 | AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 22 | IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 23 | DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE 24 | FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 25 | DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 26 | SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 27 | CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 28 | OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 29 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 30 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ## 施策デザインのための機械学習入門 2 | 3 | 技術評論社発行の書籍『施策デザインのための機械学習入門』（[Amazon](https://www.amazon.co.jp/dp/4297122243/)）のサンプルコードです。 4 | 5 | 6 | 7 | ## 書籍情報 8 | 9 | - 紙版発売: 2021年8月4日 / 電子版発売: 2021年7月30日 10 | - 齋藤優太，安井翔太　著，株式会社ホクソエム　監修 11 | - A5判／336ページ 12 | - 定価3,278円（本体2,980円＋税10%） 13 | - ISBN 978-4-297-12224-9 14 | - 出版社サポートサイト: https://gihyo.jp/book/2021/978-4-297-12224-9 15 | 16 | ## ディレクトリ構成 17 | 18 | |ディレクトリ| 内容 | 19 | |:----|:-------| 20 | | [ch02](ch02/) |「2.3節 Open Bandit Pipelineを用いた実装」で用いた実装 | 21 | | [ch03](ch03/) |「3.5節 Pythonによる実装とYahoo! R3データを用いた性能検証」で用いた実装 | 22 | | [ch04](ch04/) |「4.3節 PyTorchを用いた実装と簡易実験」で用いた実装 | 23 | | [ch05](ch05/) |「5.4節 PyTorchを用いた実装と簡易実験」で用いた実装 | 24 | 25 | 26 | ## 動作環境 27 | 本書で用いたPython環境は[poetry](https://python-poetry.org/docs/)を用いて構築しています。リポジトリを`git clone`し、フォルダ直下で`poetry install`を実行すると、本書と同じ環境を構築できます。 28 | 29 | ```bash 30 | # リポジトリをclone 31 | git clone https://github.com/ghmagazine/ml_design_book.git 32 | cd ml_design_book 33 | 34 | # poetryで環境構築 35 | poetry install 36 | 37 | # jupyter labを立ち上げ 38 | poetry run jupyter lab 39 | ``` 40 | 41 | Pythonおよび利用パッケージのバージョンは以下の通りです。 42 | 43 | ``` 44 | [tool.poetry.dependencies] 45 | python = "^3.9" 46 | torch = "^1.9.0" 47 | scikit-learn = "^0.24.2" 48 | numpy = "^1.20.3" 49 | matplotlib = "^3.4.2" 50 | seaborn = "^0.11.1" 51 | tqdm = "^4.61.1" 52 | pytorchltr = "^0.2.1" 53 | pandas = "^1.2.4" 54 | obp = "^0.4.1" 55 | jupyterlab = "^3.0.16" 56 | ``` 57 | 58 | これらのパッケージのバージョンが異なると、使用方法や挙動が本書執筆時点と異なる場合があるので、注意してください。 59 | -------------------------------------------------------------------------------- /ch02/README.md: -------------------------------------------------------------------------------- 1 | ## 第2章 2 | 3 | ### データセット 4 | 5 | 実データを用いた簡易実験を行うには、Open Bandit Datasetを[https://research.zozo.com/data.html](https://research.zozo.com/data.html)から取得し、ディレクトリを以下の通りに配置します。 6 | 7 | ``` 8 | ch2/ 9 | ├──open_bandit_dataset/ 10 | ``` 11 | なお本書でも補足した通りオリジナルのデータセットは11GBあるため、最初はお試しのスモールサイズデータを使ってみると良いかもしれません。お試しのデータセットの使い方も本書にて補足しています。 12 | ### Open Bandit Pipelineを用いた実装 13 | 14 | - [`synthetic-data.ipynb`](./synthetic-data.ipynb): 人工データを用いて意思決定モデルの学習とその性能評価を行う流れを実装. 15 | - [`real-data.ipynb`](./real-data.ipynb): 実データ（Open Bandit Dataset）を用いて意思決定モデルの学習とその性能評価を行う流れを実装. 16 | -------------------------------------------------------------------------------- /ch02/real-data.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "## 2章 Open Bandit Datasetを用いた意思決定モデルの学習/評価の実装\n", 8 | "\n", 9 | "この実装例は主に次の3つのステップで構成される。\n", 10 | "\n", 11 | "1. データの前処理: Open Bandit DatasetのうちBernoulliTSモデルで収集されたデータを読み込んで前処理を施す。\n", 12 | "2. 意思決定モデルの学習: トレーニングデータを用いてIPWLearnerに基づいた意思決定モデルを学習し、バリデーションデータに対して行動を選択する。\n", 13 | "3. 意思決定モデルの性能評価: 学習された意思決定モデルの性能をバリデーションデータを用いて評価する。\n", 14 | "\n", 15 | "このような分析手順を経ることにより「**ZOZOTOWNのファッションアイテム推薦枠において、データ収集時に使われていたBernoulliTSモデルをこれからも使い続けるべきなのか、はたまたOPLで新たに学習したIPWLearnerに基づく意思決定モデルへの切り替えを検討すべきなのか**」という問いに答えることを目指す。" 16 | ] 17 | }, 18 | { 19 | "cell_type": "code", 20 | "execution_count": 1, 21 | "metadata": {}, 22 | "outputs": [], 23 | "source": [ 24 | "# 必要なパッケージをインポート\n", 25 | "from pathlib import Path\n", 26 | "\n", 27 | "from sklearn.ensemble import RandomForestClassifier\n", 28 | "from sklearn.linear_model import LogisticRegression\n", 29 | "\n", 30 | "import obp\n", 31 | "from obp.dataset import OpenBanditDataset\n", 32 | "from obp.policy import IPWLearner\n", 33 | "from obp.ope import (\n", 34 | " OffPolicyEvaluation, \n", 35 | " RegressionModel,\n", 36 | " InverseProbabilityWeighting as IPS,\n", 37 | " DoublyRobust as DR\n", 38 | ")" 39 | ] 40 | }, 41 | { 42 | "cell_type": "code", 43 | "execution_count": null, 44 | "metadata": {}, 45 | "outputs": [], 46 | "source": [] 47 | }, 48 | { 49 | "cell_type": "markdown", 50 | "metadata": {}, 51 | "source": [ 52 | "## (1) Data Loading and Preprocessing\n", 53 | "\n", 54 | "[Open Bandit Dataset（約11GB）](https://research.zozo.com/data.html)をダウンロードし、\"./open_bandit_dataset\"におく。" 55 | ] 56 | }, 57 | { 58 | "cell_type": "code", 59 | "execution_count": 2, 60 | "metadata": {}, 61 | "outputs": [ 62 | { 63 | "output_type": "stream", 64 | "name": "stderr", 65 | "text": [ 66 | "/Users/usaito/.pyenv/versions/3.8.2/lib/python3.8/site-packages/numpy/lib/arraysetops.py:583: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison\n mask |= (ar1 == a)\n" 67 | ] 68 | }, 69 | { 70 | "output_type": "execute_result", 71 | "data": { 72 | "text/plain": [ 73 | "dict_keys(['n_rounds', 'n_actions', 'action', 'position', 'reward', 'pscore', 'context', 'action_context'])" 74 | ] 75 | }, 76 | "metadata": {}, 77 | "execution_count": 2 78 | } 79 | ], 80 | "source": [ 81 | "# ZOZOTOWNのトップページ推薦枠でBernoulli Thompson Sampling (bts)が収集したデータをダウンロードする\n", 82 | "# `data_path=None`とすると、スモールサイズのお試しデータセットを用いることができる\n", 83 | "dataset = OpenBanditDataset(\n", 84 | " behavior_policy=\"bts\", # データ収集に用いられた意思決定モデル\n", 85 | " campaign=\"men\", # キャンペーン. \"men\", \"women\", or \"all\" (\"all\"はデータ数がとても多いので注意)\n", 86 | " data_path=Path(\"./open_bandit_dataset\"), # データセットのパス\n", 87 | ")\n", 88 | "\n", 89 | "# デフォルトの前処理を施したデータを取得する\n", 90 | "# タイムスタンプの前半70%をトレーニングデータ、後半30%をバリデーションデータとする\n", 91 | "training_data, validation_data = dataset.obtain_batch_bandit_feedback(\n", 92 | " test_size=0.3,\n", 93 | " is_timeseries_split=True\n", 94 | ")\n", 95 | "\n", 96 | "# training_dataの中身を確認\n", 97 | "training_data.keys()" 98 | ] 99 | }, 100 | { 101 | "cell_type": "code", 102 | "execution_count": 3, 103 | "metadata": {}, 104 | "outputs": [ 105 | { 106 | "output_type": "execute_result", 107 | "data": { 108 | "text/plain": [ 109 | "34" 110 | ] 111 | }, 112 | "metadata": {}, 113 | "execution_count": 3 114 | } 115 | ], 116 | "source": [ 117 | "# 行動（ファッションアイテム）の数\n", 118 | "dataset.n_actions" 119 | ] 120 | }, 121 | { 122 | "cell_type": "code", 123 | "execution_count": 4, 124 | "metadata": {}, 125 | "outputs": [ 126 | { 127 | "output_type": "execute_result", 128 | "data": { 129 | "text/plain": [ 130 | "4077727" 131 | ] 132 | }, 133 | "metadata": {}, 134 | "execution_count": 4 135 | } 136 | ], 137 | "source": [ 138 | "# データ数\n", 139 | "dataset.n_rounds" 140 | ] 141 | }, 142 | { 143 | "cell_type": "code", 144 | "execution_count": 5, 145 | "metadata": {}, 146 | "outputs": [ 147 | { 148 | "output_type": "execute_result", 149 | "data": { 150 | "text/plain": [ 151 | "27" 152 | ] 153 | }, 154 | "metadata": {}, 155 | "execution_count": 5 156 | } 157 | ], 158 | "source": [ 159 | "# デフォルトの前処理による特徴料の次元数\n", 160 | "dataset.dim_context" 161 | ] 162 | }, 163 | { 164 | "cell_type": "code", 165 | "execution_count": 6, 166 | "metadata": {}, 167 | "outputs": [ 168 | { 169 | "output_type": "execute_result", 170 | "data": { 171 | "text/plain": [ 172 | "3" 173 | ] 174 | }, 175 | "metadata": {}, 176 | "execution_count": 6 177 | } 178 | ], 179 | "source": [ 180 | "# 推薦枠におけるポジションの数\n", 181 | "dataset.len_list" 182 | ] 183 | }, 184 | { 185 | "cell_type": "code", 186 | "execution_count": null, 187 | "metadata": {}, 188 | "outputs": [], 189 | "source": [] 190 | }, 191 | { 192 | "cell_type": "markdown", 193 | "metadata": {}, 194 | "source": [ 195 | "### 意思決定モデルの学習\n", 196 | "\n", 197 | "トレーニングデータを用いてIPWLearnerとランダムフォレストの組み合わせに基づく意思決定モデルを学習し、バリデーションデータに対して行動を選択する。" 198 | ] 199 | }, 200 | { 201 | "cell_type": "code", 202 | "execution_count": 7, 203 | "metadata": {}, 204 | "outputs": [ 205 | { 206 | "output_type": "stream", 207 | "name": "stdout", 208 | "text": [ 209 | "CPU times: user 4min 3s, sys: 24.7 s, total: 4min 27s\nWall time: 4min 28s\n" 210 | ] 211 | } 212 | ], 213 | "source": [ 214 | "%%time\n", 215 | "# 内部で用いる分類器としてランダムフォレストを指定したIPWLearnerを定義する\n", 216 | "new_decision_making_model = IPWLearner(\n", 217 | " n_actions=dataset.n_actions, # 行動の数\n", 218 | " len_list=dataset.len_list, # 推薦枠の数\n", 219 | " base_classifier=RandomForestClassifier(\n", 220 | " n_estimators=300, max_depth=10, min_samples_leaf=5, random_state=12345\n", 221 | " ),\n", 222 | ")\n", 223 | "\n", 224 | "# トレーニングデータを用いて、意思決定意思決定モデルを学習する\n", 225 | "new_decision_making_model.fit(\n", 226 | " context=training_data[\"context\"], # 特徴量（X_i）\n", 227 | " action=training_data[\"action\"], # 過去の意思決定モデルによる行動選択\n", 228 | " reward=training_data[\"reward\"], # 観測される目的変数\n", 229 | " position=training_data[\"position\"], # 行動が提示された推薦位置（ポジション）\n", 230 | " pscore=training_data[\"pscore\"], # 過去の意思決定モデルによる行動選択確率(傾向スコア)\n", 231 | ")\n", 232 | "\n", 233 | "# バリデーションデータに対して行動を選択する\n", 234 | "action_dist = new_decision_making_model.predict(\n", 235 | " context=validation_data[\"context\"],\n", 236 | ")" 237 | ] 238 | }, 239 | { 240 | "cell_type": "code", 241 | "execution_count": null, 242 | "metadata": {}, 243 | "outputs": [], 244 | "source": [] 245 | }, 246 | { 247 | "cell_type": "markdown", 248 | "metadata": {}, 249 | "source": [ 250 | "### 意思決定モデルの性能評価\n", 251 | "学習した新たな意思決定モデル(IPWLearner)の性能を、バリデーションデータとIPWおよびDR推定量により評価する。" 252 | ] 253 | }, 254 | { 255 | "cell_type": "code", 256 | "execution_count": 8, 257 | "metadata": {}, 258 | "outputs": [ 259 | { 260 | "output_type": "stream", 261 | "name": "stdout", 262 | "text": [ 263 | "CPU times: user 6min 54s, sys: 1min 43s, total: 8min 38s\nWall time: 2min 4s\n" 264 | ] 265 | } 266 | ], 267 | "source": [ 268 | "%%time\n", 269 | "# DR推定量を用いるのに必要な目的変数予測モデルを得る\n", 270 | "# opeモジュールに実装されている`RegressionModel`に好みの機械学習手法を与えば良い\n", 271 | "regression_model = RegressionModel(\n", 272 | " n_actions=dataset.n_actions, # 行動の数\n", 273 | " len_list=dataset.len_list, # 推薦枠内のポジションの数\n", 274 | " base_model=LogisticRegression(C=100, max_iter=10000, random_state=12345), # ロジスティック回帰を使用\n", 275 | ")\n", 276 | "\n", 277 | "# `fit_predict`メソッドにより、バリデーションデータにおける期待報酬を推定\n", 278 | "estimated_rewards_by_reg_model = regression_model.fit_predict(\n", 279 | " context=validation_data[\"context\"], # 特徴量（X_i）\n", 280 | " action=validation_data[\"action\"], # 過去の意思決定モデルによる行動選択\n", 281 | " reward=validation_data[\"reward\"], # 観測される目的変数\n", 282 | " position=validation_data[\"position\"], # 行動が提示された推薦位置（ポジション）\n", 283 | " random_state=12345,\n", 284 | ")" 285 | ] 286 | }, 287 | { 288 | "cell_type": "code", 289 | "execution_count": 9, 290 | "metadata": {}, 291 | "outputs": [], 292 | "source": [ 293 | "# 意思決定モデルの性能評価を一気通貫で行うための`OffPolicyEvaluation`を定義する\n", 294 | "ope = OffPolicyEvaluation(\n", 295 | " bandit_feedback=validation_data, # バリデーションデータ\n", 296 | " ope_estimators=[IPS(), DR()] # 使用する推定量\n", 297 | ")" 298 | ] 299 | }, 300 | { 301 | "cell_type": "code", 302 | "execution_count": 10, 303 | "metadata": {}, 304 | "outputs": [ 305 | { 306 | "output_type": "display_data", 307 | "data": { 308 | "text/plain": "
", 309 | "image/svg+xml": "\n\n\n\n \n \n \n \n 2021-06-30T12:28:29.742065\n image/svg+xml\n \n \n Matplotlib v3.3.4, https://matplotlib.org/\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n\n", 310 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgIAAAGTCAYAAABal3q3AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAABM6ElEQVR4nO3dd1hUZ/o38O8ADmWQJmIBDUoRQQWxi1GiWNaexJrEGDVZW8gvUaO7ZhM1pmCMba2brCbqGhXU2LJqRBQLERtKBAQbSBHpfWgz8/7hy6wjzHAGZkCc7+e6vOQ85znPuUc9nnvOeYpIoVAoQERERAbJqLEDICIiosbDRICIiMiAMREgIiIyYEwEiIiIDBgTASIiIgPGRICIiMiAMREgIiIyYEwEiIiIDBgTASIiIgNmIrRiWloa/vzzT8TFxSErKwuFhYUQi8WwsrKCs7MzvLy80KVLF4jFYn3GS0RERDokqm2K4UuXLuH333/HnTt3am1MIpHA398fI0aMgIODg86CJCIiIv1Qmwjcvn0bu3btQlJSEiwsLNCrVy94eHjAxcUFNjY2sLS0RHl5OQoLC5GWloaEhARER0fj7t27MDExwV/+8he88cYbsLCwaOjPRERERAKpTQQmT56MDh06YNy4cejZsyeaNWsmqMHHjx/j9OnTOH36NMaNG4cJEyboNGAiIiLSHbWJwJUrV9C7d+86N5yXl4eMjAy4u7vXuQ0iIiLSr1r7CBAREdHLS/CogZdNWlpaY4dARETUINq2bat2H+cRICIiMmAanwh8+OGHWjcoEomwcePGOgdEREREDUdjIpCZmdlQcRAREVEj0NhZsK6JQMuWLescUENhHwEiIjIUmvoIaHwi0BRu6ERERFR37CxIRERkwDQ+EZDL5Vi/fj1EIhECAwNhYlJz9crKSmzcuBEikQgff/yxPuIkIiIiPdD4RCAyMhKRkZHo2bOn2iQAAExMTNCrVy/88ccfuHz5ss6DJCIiIv3QmAj88ccfsLOzw4ABA2ptyM/PD3Z2drh48aLOgiMiIiL90pgI3L9/H15eXhCJRLU2JBKJ0KVLFzx48EBnwREREZF+aUwE8vLy0KJFC8GN2dnZIT8/v95BERERUcPQmAiYmJigoqJCcGMVFRUa+xIQERHRi0XjXdvW1hZJSUmCG0tKSoKtrW29gyIiohfLqlWrkJWVBXt7eyxZsqSxwyEd0vhEoFOnToiNjUV6enqtDaWnpyM2NhYeHh46C46IiF4MWVlZSE9PR1ZWVmOHQjqmMREYOnQo5HI51q5dq/Hdf0FBAdatWwe5XI6AgACdB0lERET6ofHVgKurKwICAhAaGooFCxZg6NCh6NKlC+zs7AAAOTk5uH37NkJDQ1FYWIihQ4fC1dVVqwDS09Nx9OhRJCQkIDk5GZ07d8by5csFHRsZGYnDhw/j0aNHMDU1hYuLCxYuXAgzMzOtYiAiIjJUtfbsmzlzJuRyOcLCwvDrr7/i119/rbHekCFDMHPmTK0DSE5ORlRUFNzc3CCTyQQfd+bMGezYsQNjx47FO++8g+LiYty+fRtyuVzrGIiIiAxVrYmAsbExZs+eDX9/f5w+fRrx8fHIy8sDANjY2MDDwwMBAQHo1KlTnQLo0aMHevXqBQBYs2YNCgsLaz2moKAAO3fuxIwZM1ReRfTu3btOMRARERkqwWP9OnXqVOebvSZGRtqve/THH38AAPz9/XUcDRERkWFpkoP+7969i7Zt2yIsLAyHDh1Cfn4+OnTogOnTp+slWSEiInpZNclliPPz85GWloaDBw/i7bffxpIlS2BqaopvvvlG+dqCXk6rVq3Cp59+ilWrVjV2KEREL4Um+URAoVCgtLQUCxYsgI+PDwDA3d0d8+fPx8mTJzFlypRqx4SGhiI0NBQAEBQUBHt7+4YMmXQkNzcX6enpMDY25t8hUQMyNjZW/s5r7+XSJBMBiUQCkUgET09PZZmFhQU6duyIlJSUGo8JCAhQ6VjISTGapqqRJTKZjH+HRA2I117T1rZtW7X7muSrAUdHRygUimrlCoWiTp0PiYiIDFWTvGv26NEDAHD79m1lWUlJCR48eIBXXnmlscIiIiJqchr91UBZWRmioqIAPJ2pUCqV4vLlywCA7t27w9TUFIGBgfD09MTcuXMBAC4uLujZsye2bduGt956C1ZWVjhy5AiMjY0xfPjwRvssRERETU29EoGMjAzlO3knJyc4ODho3UZ+fj7Wrl2rUla1vWnTJjg4OEAul1ebMfCjjz7C7t27sWvXLpSVlcHDwwPLli2DpaVlHT8NERGR4alTIiCVSrFt2zblN/cq/fr1w5w5c7Sa69/BwQHBwcEa62zevLlamZmZGT744AN88MEHgs9FREREquqUCGzfvh3R0dGYNGkSOnbsiIqKCly7dg3h4eEwNTVVPsInIiKiF5vGRKCsrAympqbVyq9evYr3338fr776qrKsd+/eKCsrw5UrV5gIEBERNREaRw0sWrRIpWd+FZlMBnNz82rl5ubmXP2PiIioCdH4RMDNzQ0rV67EkCFDMG3aNOXNv0uXLti+fTtKS0vRoUMHVFRU4Pr16wgPD1cO7SMiIqIXn8ZE4KOPPsKAAQPw448/IioqCn/961/RvXt3vP/++1i9ejU2btyoUr9jx46YOXOmXgMmIiIi3am1s6Cvry/WrFmDXbt2ISgoCK+++iree+89rFq1CtHR0UhNTQXwdPhg165d9R4wERER6Y6gUQMWFhaYM2cO+vfvjx9++AELFy7ErFmz0Lt3b3Tr1k3fMRIREZGeaDXFcLdu3fD999+jd+/eWLNmDdatW4eCggJ9xUZERER6JigRKCgowIMHD1BQUAAzMzPMmjULy5cvR2JiIj755BNcvHhR33ESERGRHmh8NVBaWoqtW7eqzCDYp08fzJs3D507d8bq1auxb98+bN68GREREfjrX/8KGxsbfcdMREREOqLxicAvv/yCy5cvY9CgQZg1axb8/f0RGRmJPXv2AADEYjHeffddrFy5Eunp6fjkk09w9uzZBgmciIiI6k/jE4GrV68qnwBUkUqluHbtGmbNmqUsc3V1xXfffYcDBw7gxx9/xGuvvaa/iImIiEhnap1iuEWLFiplLVq0qHG2QRMTE0yZMgV9+/bVbYRERESkNxpfDbi5ueH8+fO4c+cOKisrkZCQgAsXLsDNzU3tMc7OzrqOkYiIiPRE4xOBGTNmYMWKFVi2bJmyzM7ODu+9956+4yIiIqIGoDERaN26NdavX4/r168jKysL9vb28PX1hZmZWUPFR0RERHpU68yCpqam6N+/f0PEQkRERA1Mq5kFiYiI6OUiaK2Bmly7dg1xcXEoKyuDg4MD+vfvD3t7e13GRkRERHqmMRH45Zdf0K1bN3Tp0kVZVlxcjO+++w537txRqbt//37Mnj0bAwcO1E+kREREpHMaE4EjR45ALBarJAL/+te/cOfOHTg4OMDPzw9WVlZISEjAH3/8gW3btsHZ2Rnt27fXe+BERERUf1q9GkhPT0dkZCQ6dOiAZcuWwdzcHAAwcuRI+Pr6YvPmzfjvf/+LOXPm6CVYIiIi0i2tOgvGxcUBAKZOnapMAqoMHDgQrq6uiI2N1V10REREpFdaJQJ5eXkAABcXlxr3u7i4ICcnp95BERERUcPQKhGoegrQrFmzGvc3a9YMIpGo/lERERFRg6i1j0BMTIzy5/T0dABAZmYmnJycqtXNzs5G8+bNtQogPT0dR48eRUJCApKTk9G5c2csX75c8PFyuRxLly7FgwcPsGTJEvTo0UOr8xMRERmyWhOB2NjYau/9b9y4UWMi8ODBAzg6OmoVQHJyMqKiouDm5gaZTKbVsQAQFhaG7OxsrY8jIiKiWhKBZxcbepaVlVW1sgcPHkAmk6Fr165aBdCjRw/06tULALBmzRoUFhYKPraoqAh79+7F22+/jW3btml1XiIiIqolEfD09BTcUMeOHbF582atAzAyqvssx/v370enTp1U5jkgIiIi4ZrsWgNJSUk4e/Ys3n333cYOhYiIqMnSakIhmUyGJ0+eoLi4GCKRCNbW1mjZsqW+YtNox44dGDFiBFq3bo2MjIxa64eGhiI0NBQAEBQUxHURmihjY2Pl7/w7JGo4vPZeXoISgStXruDUqVOIi4ur1qHPysoKfn5+GD9+PGxsbPQRYzWXLl1CWloalixZIviYgIAABAQEKLezsrL0ERrpWdW/P5lMxr9DogbEa69pa9u2rdp9GhMBhUKBLVu24Pz589X22dvbw8zMDOnp6Thx4gQuXLiATz/9FB4eHvWPWIPKykr85z//wbhx46BQKFBcXAypVAoAKCsrg1QqrTbrIREREdVMYyIQGhqK8+fPw9fXF5MnT0arVq3w5MkTBAcHIz4+Hp999hlatmyJS5cuYffu3Vi1ahXWrFkDOzs7vQVcVlaG7Oxs7Nq1C7t27VLZt379erRq1QobN27U2/mJiIheJhoTgbCwMDg5OWHRokXK90POzs5YuHAhFi9ejF9++QWLFi2Cv78/nJ2d8fe//x2HDx/GzJkz9RawmZlZtWGNeXl52LBhA6ZOncoRBERERFrQmAikpKRgyJAhyiSgirGxMbp27Yrw8HBlmbOzM3x9fREVFaVVAGVlZcpjcnJyIJVKcfnyZQBA9+7dYWpqisDAQHh6emLu3LkwNjaGl5eXShtVnQXbt28PNzc3rc5PRERkyDQmAiKRCOXl5TXuKy8vR0VFhUqZo6Mjbt68qVUA+fn5WLt2rUpZ1famTZvg4OAAuVwOuVyuVbtERERUO42JQLt27XDt2jW89dZbsLS0VJYXFRXh2rVraNOmjUr90tJSiMVirQJwcHBAcHCwxjq1TVQkpA0iIiKqTmMi8Nprr+HHH3/E0qVLMXr0aDg4OCAjIwO//fYb8vPzMXr0aJX6ycnJaN26tV4DJiIiIt3RmAgEBAQgNjYWly5dwvbt21X2+fj4qCQCUqkU5eXl6N+/v34iJSIiIp2rdUKhjz76CH379sWVK1eQn5+P5s2bw9fXF/3791dZJ8Dc3Bxff/21XoMlIiIi3RI0s2Dv3r3Ru3dvfcdCRFRn7+38o7FDeKnJCp5O3JZeIOWftR79PL1fg5+zyS46RERERPXHRICIiMiAMREgIiIyYEwEiIiIDBgTASIiIgPGRICIiMiACRo+SMI8/vT9xg7hpVeZWf7/f3/CP289arP6340dAhE1ED4RICIiMmBaJwKxsbE4cOCA1vuIiIjoxaN1IhATE4OQkBCt9xEREdGLh68GiIiIDBgTASIiIgPGRICIiMiACRo+mJWVpfy5uLi4WhkA2Nvb6zAsIiIiagiCEoH58+drLBOJRNi3b5/uoiIiIqIGISgRePPNNyESiQA8HSIYGxuLCRMm6DUwIiIi0j9BicCkSZOUP4eEhCA2NhYTJ07UW1BERETUMNhZkIiIyIAxESAiIjJgTASIiIgMmNaJgEKhqNM+IiIievFovQzxpEmTVDoPCt2nTnp6Oo4ePYqEhAQkJyejc+fOWL58ucZj7t27h99//x1xcXHIzc1FixYtMGDAAIwbNw5isVir8xMRERkyrRMBXUtOTkZUVBTc3Nwgk8kEHRMREYEnT55g3LhxaNOmDZKSkrB//34kJSVh0aJFeo6YiIjo5dHoiUCPHj3Qq1cvAMCaNWtQWFhY6zHjx4+HlZWVctvLywtisRg//PADMjMz0bJlS73FS0RE9DJR20egvLy83o0LacPISPv+is8mAVWcnZ0BALm5uVq3R0REZKjU3oXnz5+P//73v6ioqNC60cTERHz33Xc4evRovYLTRkJCAkQiEVq1atVg5yQiImrq1L4a8Pb2xs6dOxESEoL+/fujX79+cHd3V9sZ78mTJ7h16xbCw8Nx79492NvbY+zYsXoL/Fl5eXk4dOgQBg4cCGtr6xrrhIaGIjQ0FAAQFBSkl0WSHuu8RaLGwUXEiBpHY1x7ahOBDz/8ECNGjMC+ffuUN1EjIyM4OTnBxsYGEokEFRUVKCoqQlpaGgoKCgA8fWw/depUjBo1Cs2aNdP7B6isrMS6detgZmaG6dOnq60XEBCAgIAA5fbzqycS0f/w+iBqHPq69tq2bat2n8bOgq6urvjHP/6Bx48fIywsDLdv30ZiYiIePXqkUs/Kygp9+vRR/jIxaZg+iAqFAps2bUJycjJWrlwJS0vLBjkvERHRy0LQHbtNmzZ4++23AQBlZWXIyclBYWEhxGIxrK2tYWtrq9cg1fn5559x9epVfP7553B0dGyUGIiIiJoyrb+6m5qaok2bNmjTpo0+4hHs119/xcmTJ/HJJ5/Aw8OjUWMhIiJqqhp9HoGysjJERUUBAHJyciCVSnH58mUAQPfu3WFqaorAwEB4enpi7ty5AICLFy9i79698Pf3h52dHRISEpTttW7dusbhhURERFRdoycC+fn5WLt2rUpZ1famTZvg4OAAuVwOuVyu3H/r1i0AwLlz53Du3DmVY+fNmwd/f3+9xkxERPSyaPREwMHBAcHBwRrrbN68WWV7/vz5mD9/vj7DIiIiMghchpiIiMiAMREgIiIyYEwEiIiIDBgTASIiIgOmdWfByspK3L59GykpKSgtLcWECRMAPF1pUCqVonnz5nVaUZCIiIganlaJwM2bN7F161bk5eUpy6oSgcTERHz++ecIDAzEgAEDdBokERER6Yfgr+7379/H6tWrIRKJMH36dPj5+ansd3d3h4ODA65cuaLzIImIiEg/BCcCBw8ehFgsRlBQEEaOHFnjFMMuLi5ISkrSaYBERESkP4ITgfj4ePTq1Qs2NjZq69jb26u8NiAiIqIXm+BEoLS0tNY5/MvKylSmAiYiIqIXm+BEwM7ODsnJyRrrJCYmolWrVvUOioiIiBqG4ETAx8cHt27dwp07d2rcHxUVhYSEBPj6+uosOCIiItIvwcMHX3/9dUREROCrr77CiBEjkJmZCQC4ceMGYmNjcerUKdjY2GD06NF6C5aIiIh0S3AiYGdnh88++wzr1q3DsWPHlOWrVq0CALRq1QqLFi2qtR8BERERvTi0mlCoY8eO2LBhA27cuIGEhAQUFhbCwsICbm5u6NWrF4yNjfUVJxEREemB1lMMGxkZoWfPnujZs6c+4iEiIqIGxEUBiIiIDJjgJwLh4eGCGx00aFCdgiEiIqKGJTgR2LJli+BGmQgQERE1DYITgblz59ZYXlJSgnv37iEiIgK9e/fmPAJERERNiOBEwN/fX+P+1157TbkgERERETUNOuss2LVrV3h7e2P//v26apKIiF4UZs0BC+unv9NLRevhg5q0bdsWp0+f1mWTRET0AjD2HdvYIZCe6HT4YEpKii6bIyIiIj2r9xMBuVyO7OxsnDlzBlFRUejevbsu4iIiIqIGIDgRmDx5cq11LC0t8c4772gVQHp6Oo4ePYqEhAQkJyejc+fOWL58ea3HlZSU4Oeff8bVq1chl8vRo0cPzJgxA82b8/0VERGRUIITgc6dO0MkElUrF4lEkEgkcHV1xWuvvab1okPJycmIioqCm5sbZDKZ4OPWrVuHtLQ0zJ49G0ZGRtizZw9Wr16NL7/8UqvzExERGTLBiYCQb+l10aNHD/Tq1QsAsGbNGhQWFtZ6TEJCAm7duoXly5fD09MTwNPVEZcuXYro6Gh069ZNL7FS47M1BgDR//+diIjqS6ejBurCyEj7/opRUVGwtrZWJgEA4OrqCgcHB9y8eZOJwEvsAztxY4dARPRSaZKLDqWmpsLR0bFauaOjI1JTUxshIiIioqZJ7RMBbdYWeJZIJFI7HbGuFBcXw8LColq5RCJBRkZGjceEhoYiNDQUABAUFAR7e3udx/VY5y0SNQ59XB9EVLvGuPbUJgLarDb4PH0nAnUREBCAgIAA5XZWVlYjRkP0YuP1QdQ49HXttW3bVu0+tYnApk2b9BKMLkgkkho7FRYXF0MikTRCRERERE2T2kSgZcuWDRmHVhwdHXHmzJlq5WlpacoRCERERFS7JtlZsHv37sjLy8OdO3eUZffv38eTJ0/g4+PTeIERERE1MXUaPiiXy1FQUIDKysoa92vT2aGsrAxRUVEAgJycHEilUly+fBnA0xu+qakpAgMD4enpqex74O7uDm9vb2zatAnTpk2DSCTCnj174OHhwaGDREREWtAqEXj06BH27NmDmJgYVFRU1FhHJBJh3759gtvMz8/H2rVrVcqqtjdt2gQHBwfI5XLI5XKVOh9//DF27tyJrVu3QqFQwNfXFzNmzNDm4xARERk8wYlASkoK/vGPfwAAunXrhuvXr+OVV16BtbU1Hj58iMLCQnh5eWk99MHBwQHBwcEa62zevLlamUQiwbx58zBv3jytzkdERET/IzgROHToEGQyGb799lu0b98ekydPRu/evTFhwgSUlpbip59+QlRUFG/MRERETYjgzoIxMTHw9fVF+/btlWUKhQIAYGZmhr/+9a+QSCTYv3+/7qMkIiIivRCcCBQWFqJNmzb/O9DICGVlZcptY2NjeHl5ITo6WrcREhERkd4ITgQsLS1RWlqq3Laysqo2A5KJiQlKSkp0Fx0RERHpleBEoFWrVirz+Hfo0AF//vkn8vPzAQClpaW4du0aHBwcdB8lERER6YXgzoLe3t44cuQISktLYWZmhmHDhiEqKgqLFy9Gp06d8ODBA2RmZuLdd9/VZ7xERESkQ4ITgSFDhqBt27YoLy+HmZkZfH19MX36dISEhCAyMhJisRjjxo3DX/7yF33GS0RERDqkMRFYvHgxAgIC8Oqrr8LW1hb9+/dX2T9y5EiMGDECBQUFsLa2hkgk0muwREREpFsa+wgkJSVh+/btmD17NrZt24a7d+9Wb8DICDY2NkwCiIiImiCNTwRWrlyJ0NBQXL58GWfPnsXZs2fRvn17DBkyBAMHDoSFhUVDxUlERER6oDERcHd3h7u7O2bMmIELFy4gLCwMDx8+xE8//YQ9e/agb9++GDJkCDw8PBoqXiIiItIhQZ0Fzc3NMWzYMAwbNgyJiYkIDQ3FpUuXcP78eZw/fx5OTk7KpwSWlpb6jpmIiIh0RPA8AlWcnZ3x/vvv41//+hfmzZuHTp06ISUlBTt37sScOXOwceNGfcRJREREeqB1IlBFLBZj0KBB+PLLL7Fu3Tp4eHigoqICFy9e1GV8REREpEeC5xGoSVFREcLDwxEWFoaUlBQAYAdCIiKiJqROicDt27cRGhqKq1evorKyEgDg5uaGgICAanMNEBER0YtLcCKQl5eHs2fPIiwsTLnmgEQiQUBAAAICAtCuXTu9BUlERET6oTERUCgUuHHjBs6cOYOoqCjI5XIAgIeHB4YMGYK+fftCLBY3SKBERESkexoTgXnz5iEnJwfA02WIBw4ciICAADg6OjZIcERERKRfGhOBnJwceHp6Kr/9m5jUq28hERERvWA03tnXr1+PNm3aNFQsRERE1MA0ziPAJICIiOjlVucJhYiIiKjpYyJARERkwJgIEBERGbBGHwaQkpKCHTt2ICEhARKJBIMHD8bEiRNhZKQ5R7l//z727t2L+/fvAwA6duyIKVOmwM3NrSHCJiIieik06hOBoqIirFy5EiKRCIsXL8abb76J48ePIzg4WONxWVlZWLlyJWQyGQIDAxEYGAiZTIavvvoKmZmZDRQ9ERFR0yf4iUBkZCR69epV6zd1bZw+fRrl5eVYuHAhLCws0K1bN0ilUoSEhGDs2LFqFzC6ceMGpFIpPv30U2Udd3d3zJo1C1FRURg2bJjOYiQiInqZCb6rr127FvPmzcP+/fuRlZWlk5PfvHkT3t7eKjd8Pz8/lJeXIzY2Vu1xMpkMxsbGMDU1VZaZmZnB2NgYCoVCJ7EREREZAsGJwPDhw1FWVoZDhw4hMDAQQUFBuH79er1uvKmpqWjbtq1Kmb29PUxNTZGWlqb2uD59+sDU1BS7du1Cfn4+8vPzsXPnTkgkEvTr16/O8RARERkawa8GZs6ciXfeeQcRERE4ffo0oqKiEBUVBTs7OwwZMgSDBw+GnZ2dVicvLi6GRCKpVi6RSFBUVKT2ODs7OyxbtgxBQUE4ceIEAMDW1hafffYZrKysajwmNDQUoaGhAICgoCDY29trFasQj3XeIlHj0Mf1QUS1a4xrT6tRA2KxGP7+/vD398ejR48QGhqKCxcuICQkBAcPHoSvry+GDh0KHx8fPYX7VG5uLtauXYuOHTtizpw5AICTJ08iKCgIX331VY1/kFXLJVfR1esNopcRrw+ixqGva+/5p+/PqvPwwfbt26s8Jdi/fz+uXbuGa9euwd7eHsOHD8ewYcNgZmamtg2JRIKSkpJq5cXFxbC0tFR73NGjRyGTybBgwQLlQkhdunTBRx99hKNHj2LmzJl1/VhEREQGpV5DAEpLS3H+/HmcPHlSuVyxs7MzioqKsGfPHnzyySdITExUe7yjoyNSU1NVyrKyslBWVqYxe0lLS4OTk5PKaogmJiZo164dnjx5Up+PREREZFDq9ETg4cOHOH36NC5duoTS0lKIxWIMHjwYw4cPh7OzM0pLS3Hq1CkEBwfjp59+wooVK2psx8fHB0ePHoVUKoW5uTkAICIiAmKxGJ6enmrPb29vj6ioKFRWViqTgYqKCjx69Ag9evSoy0ciIiIySIITgbKyMly6dAmnT5/GgwcPADz9Rj906FAMGjRIZQigmZkZxo0bh+zsbISFhaltc+jQoThx4gS+//57jBs3DhkZGQgJCcHo0aNV2gsMDISnpyfmzp0LABgyZAjCwsKwevVqDB8+HAqFAqdOnUJeXp5KPwAiIiLSTHAiMHv2bEilUhgZGaFPnz4YPnw4vLy8NB5jZ2eHiooKtfstLS3xxRdfYPv27Vi1ahUkEglGjRqFSZMmqdSTy+WQy+XK7Y4dO2Lp0qU4cOAANm7cCOBpn4V//OMfcHZ2FvqRiIiIDJ7gRMDc3ByjR49GQEAAbGxsBB0zbNgw+Pn5aazj5OSEZcuWaayzefPmamVdu3ZF165dBcVBRERENROcCGzevFnr6YUtLCzUThNMREREjU/wnV2XawwQERHRi0Hw3f3gwYOYOnWqcpjg83JycjB16lQcPnxYV7ERERGRnglOBK5fvw5PT0+10wjb2dmhS5cuuHr1qs6CIyIiIv0SnAikp6fDyclJYx1HR0ekp6fXOygiIiJqGIITgfLycpVlf2siFotRWlpa76CIiIioYQhOBFq0aIG7d+9qrHP37l2tVyAkIiKixiM4EfD29kZsbCwiIiJq3H/p0iXExsbqfeVBIiIi0h3B8wiMHz8eFy9exIYNGxAREQEfHx/Y2dkhJycHUVFRuHbtGiwtLTF+/Hg9hktERES6JDgRsLOzw2effYa1a9fi6tWr1UYHtGzZEgsWLECLFi10HiQRERHph1arD7q4uGDDhg24fv067t69i+LiYkgkEri5uaFHjx4qywITERHRi0/rO7eJiQn69OmDPn366CMeIiIiakCcN5iIiMiAqX0iEB4eDgDo3bs3zM3NldtCDBo0qP6RERERkd6pTQS2bNkCAHBzc4O5ublyWwgmAkRERE2D2kRg7ty5AABbW1uVbSIiInp5qE0E/P39NW4TERFR08fOgkRERAaMiQAREZEBU/tq4MMPP6xTgyKRCBs3bqxzQERERNRw1CYCCoWiTg3W9TgiIiJqeGoTgc2bNzdkHERERNQI2EeAiIjIgNU5EZBKpcjKykJJSYku4yEiIqIGpNWiQzKZDMeOHcOZM2eQkZGhLHdwcMCQIUMwZswYGBsb6zxIIiIi0g/BiUBlZSW+/vprxMbGQiQSwd7eHjY2NsjLy0NmZib27t2Lmzdv4h//+IdWyxGnpKRgx44dSEhIgEQiweDBgzFx4kQYGdX+sCIyMhKHDx/Go0ePYGpqChcXFyxcuBBmZmaCz09ERGTIBN+xjx8/jtjYWPj6+uLdd99FmzZtlPvS09Oxa9cuXL9+HcePH8f48eMFtVlUVISVK1fCyckJixcvRnp6Onbv3g2FQoEpU6ZoPPbMmTPYsWMHxo4di3feeQfFxcW4ffs25HK50I9ERERk8AQnAhcvXkS7du3w6aefVvu23rp1ayxatAiLFy/GhQsXBCcCp0+fRnl5ORYuXAgLCwt069YNUqkUISEhGDt2LCwsLGo8rqCgADt37sSMGTMQEBCgLO/du7fQj0NERETQorNgeno6fHx81D6yNzIygo+PD548eSL45Ddv3oS3t7fKDd/Pzw/l5eWIjY1Ve9wff/wBgOsfEBER1ZfgJwImJiYoLS3VWKesrEyrzoKpqanw8vJSKbO3t4epqSnS0tLUHnf37l20bdsWYWFhOHToEPLz89GhQwdMnz4dnTp1Enx+IiIiQyc4EXjllVcQGRmJSZMmwcrKqtr+goICXL58Gc7OzoJPXlxcDIlEUq1cIpGgqKhI7XH5+flIS0vDwYMH8c4776B58+Y4cuQIvvnmG2zYsAE2NjbVjgkNDUVoaCgAICgoCPb29oLjFOqxzlskahz6uD6IqHaNce0JTgSGDx+ODRs24O9//zvefPNNeHl5wdbWFnl5eYiJicGhQ4dQUFCAGTNm6DNeAE+nMS4tLcWCBQvg4+MDAHB3d8f8+fNx8uTJGjsaBgQEqPQnyMrK0nucRE0Vrw+ixqGva69t27Zq9wlOBPr374/ExEQcOXIE//rXv2qsM3bsWPTv319wYBKJpMYJiYqLi2FpaanxOJFIBE9PT2WZhYUFOnbsiJSUFMHnJyIiMnRaTSj01ltvoWfPnggLC0NiYiJKSkpgYWEBZ2dnDB48GO7u7lqd3NHREampqSplWVlZKCsr05i9ODo61ri4kUKhEDT/ABERET0lOBEoLCyESCSCu7u71jd8dXx8fHD06FFIpVKYm5sDACIiIiAWi1W+7T+vR48eOHDgAG7fvg1fX18AQElJCR48eIAxY8boJDYiIiJDUGsicPXqVezatUs5pXDr1q0xbdo09OzZs94nHzp0KE6cOIHvv/8e48aNQ0ZGBkJCQjB69GiVIYWBgYHw9PTE3LlzAQAuLi7o2bMntm3bhrfeegtWVlY4cuQIjI2NMXz48HrHRUREZCg0PkdPSEjAmjVrVNYVSE9Px5o1a5CQkFDvk1taWuKLL76AXC7HqlWrEBwcjFGjRmHSpEkq9eRyebUZAz/66CP06tULu3btwpo1a2BiYoJly5Zp7FtAREREqkSKml62/39r165FZGQk3nzzTYwYMQIKhQInT57EoUOH0KdPHyxYsKAhY9UpTfMU1NXjT9/XeZtEjaHN6n83dghae2/nH40dAlG9/Ty9n17arfOogbt378LDw0PlG/rkyZMRGxurkycCRERE1Lg0vhrIz8+Hm5tbtXI3NzcUFBToLSgiIiJqGBoTAZlMVuOSvqamppDJZHoLioiIiBoGB90TEREZsFqHD547dw4xMTEqZZmZmQCAFStWVKsvEonwxRdf6Cg8IiIi0qdaE4HMzEzljf95mpYKJiIiohefxkRg2bJlDRUHERERNQKNiYCmaX6JiIio6WNnQSIiIgPGRICIiMiAMREgIiIyYEwEiIiIDBgTASIiIgPGRICIiMiAMREgIiIyYEwEiIiIDJjaCYUOHDhQ50YnTJhQ52OJiIio4ahNBEJCQurcKBMBIiKipkFtIlDTOgPHjx9HVFQUXn31VXh6esLGxgZ5eXmIiYnBxYsX4evri1GjRuk1YCIiItIdtYnA8+sMhIeH488//8TXX3+Njh07quzz9/fHiBEjsGzZMvTp00c/kRIREZHOCe4s+Ntvv6Ffv37VkoAqLi4u6NevH3777TedBUdERET6JTgRSEtLg62trcY6tra2SEtLq3dQRERE1DAEJwLm5uaIj4/XWCc+Ph5mZmb1DoqIiIgahuBEwNfXF3Fxcdi1axekUqnKPqlUil27duHOnTvo0aOHzoMkIiIi/VDbWfB5b731FmJjY/Hbb78hLCwMzs7OsLa2Rn5+PhITEyGVSuHg4ICpU6fqM14iIiLSIcGJgLW1Nb755hv88ssvuHjxIuLi4pT7xGIxhgwZgqlTp6J58+ZaBZCSkoIdO3YgISEBEokEgwcPxsSJE2FkJOxhhVwux9KlS/HgwQMsWbKETySIiIi0IDgRAIDmzZtj9uzZeP/995GamoqSkhJYWFjA0dERxsbGWp+8qKgIK1euhJOTExYvXoz09HTs3r0bCoUCU6ZMEdRGWFgYsrOztT43ERERaZkIVDE2Nkb79u3rffLTp0+jvLwcCxcuhIWFBbp16wapVIqQkBCMHTsWFhYWGo8vKirC3r178fbbb2Pbtm31joeIiMjQaL3oUGVlJW7evInjx4+rrEdQXl6O/Px8yOVywW3dvHkT3t7eKjd8Pz8/lJeXIzY2ttbj9+/fj06dOqFLly7afQgiIiICoOUTgZs3b2Lr1q3Iy8tTllWtK5CYmIjPP/8cgYGBGDBggKD2UlNT4eXlpVJmb28PU1PTWucjSEpKwtmzZ/H9999r8xGIiIjoGYITgfv372P16tVo3rw5pk+fjnv37uHSpUvK/e7u7nBwcMCVK1cEJwLFxcWQSCTVyiUSCYqKijQeu2PHDowYMQKtW7dGRkZGrecKDQ1FaGgoACAoKAj29vaCYtTGY523SNQ49HF9EFHtGuPaE5wIHDx4EGKxGEFBQbCxsalxdUIXFxc8fPhQpwHW5NKlS0hLS8OSJUsEHxMQEICAgADldlZWlj5CI3op8Pogahz6uvbatm2rdp/gPgLx8fHo1asXbGxs1Naxt7dXeW1QG4lEgpKSkmrlxcXFsLS0rPGYyspK/Oc//8G4ceOgUChQXFysnOCorKys2mRHREREpJ7gJwKlpaWwsrLSWKesrEyrzoKOjo5ITU1VKcvKykJZWZna7KWsrAzZ2dnYtWsXdu3apbJv/fr1aNWqFTZu3Cg4BiIiIkMmOBGws7NDcnKyxjqJiYlo1aqV4JP7+Pjg6NGjkEqlMDc3BwBERERALBZXWwa5ipmZGZYtW6ZSlpeXhw0bNmDq1KkcQUBERKQFwa8GfHx8cOvWLdy5c6fG/VFRUUhISICvr6/gkw8dOhTNmjXD999/j+joaISGhiIkJASjR49WGVIYGBiIrVu3Ang6h4GXl5fKLzc3NwBA+/btlT8TERFR7QQ/EXj99dcRERGBr776CiNGjEBmZiYA4MaNG4iNjcWpU6dgY2OD0aNHCz65paUlvvjiC2zfvh2rVq2CRCLBqFGjMGnSJJV6crlcq1cOREREJIxIoVAohFZ+8OAB1q1bV+NwvVatWmHRokU6mXGwIdQ2T0FdPP70fZ23SdQY2qz+d2OHoLX3dv7R2CEQ1dvP0/vppV1Nowa0mlCoY8eO2LBhA27cuIGEhAQUFhbCwsICbm5u6NWrV53WGyAiIqLGo/VaA0ZGRujZsyd69uypj3iIiIioAQnuLLhixQqEh4drrHP+/HmsWLGi3kERERFRwxCcCMTGxio7CKqTlZUlaLEgIiIiejFovfqgJuXl5ewnQERE1IRo3UegJgqFAllZWYiKikKLFi100SQRERE1AI2JwOTJk1W2Q0JCalxs6Fmvv/56/aMiIiKiBqExEejcuTNEIhGAp30E7O3t4eDgUK2ekZERLC0t0bVrVwwePFg/kRIREZHOaUwEli9frvx58uTJeO211zBhwgR9x0REREQNRHAfgU2bNkEikegzFiIiImpgghOBli1b6jMOIiIiagRajxrIzc3Fn3/+iZycHFRWVtZYh68PiIiImgatEoHg4GAcPnwYMplMYz0mAkRERE2D4ETgwoULOHjwILp06YLhw4djzZo1GDRoELy9vRETE4OzZ8+ib9++GDp0qD7jJSIiIh0SnAj8/vvvsLOzw9KlS5WzBzo4OMDPzw9+fn7o3bs3goKC4Ofnp7dgiYiISLcETzH86NEjdO/eXWUKYblcrvzZx8cH3t7eOHbsmG4jJCIiIr0RnAjIZDI0b95cuS0Wi1FSUqJSp127dkhMTNRZcERERKRfghMBW1tb5ObmKrft7e2RlJSkUic3N5eLDhERETUhghMBZ2dnJCcnK7e9vLxw584dnD9/HqWlpbhx4wYuX76MDh066CVQIiIi0j3BiUCPHj2QnJyMjIwMAMD48eNhYWGBzZs3Y/r06Vi1ahWA6gsVERER0YtL8KgBf39/+Pv7K7ft7e3x7bff4tixY3jy5AlatmyJ4cOHo3379vqIk4iIiPRA65kFn+Xg4IBZs2bpKhYiIiJqYIJfDRAREdHLR+snAnK5HDk5ORrXGvD09Kx3YERERKR/WiUCR48exbFjx1BQUKCx3v79++sVFBERETUMwYlAcHAwDh48CEtLSwwaNAh2dnY6mTMgJSUFO3bsQEJCAiQSCQYPHoyJEyfCyEj9W4t79+7h999/R1xcHHJzc9GiRQsMGDAA48aNg1gsrndMREREhkJwInD27Fk4ODhg1apVsLCw0MnJi4qKsHLlSjg5OWHx4sVIT0/H7t27oVAoMGXKFLXHRURE4MmTJxg3bhzatGmDpKQk7N+/H0lJSVi0aJFOYiMiIjIEghOBwsJCDB06VGdJAACcPn0a5eXlWLhwISwsLNCtWzdIpVKEhIRg7Nixas81fvx4WFlZKbe9vLwgFovxww8/IDMzEy1bttRZjERERC8zwaMGWrdujeLiYp2e/ObNm/D29la54fv5+aG8vByxsbFqj3s2Caji7OwMACrTIBMREZFmghOBYcOG4fr168jLy9PZyVNTU9G2bVuVMnt7e5iamiItLU2rthISEiASidCqVSudxUdERPSyE/xqYNiwYXj8+DE+//xzvPnmm+jYsaPaR/f29vaC2iwuLoZEIqlWLpFIUFRUJDQ05OXl4dChQxg4cCCsra1rrBMaGorQ0FAAQFBQkOAYtfFY5y0SNQ59XB9EVLvGuPa0Gj74yiuv4Ny5c9i6davaOiKRCPv27at3YEJVVlZi3bp1MDMzw/Tp09XWCwgIQEBAgHI7KyurIcIjapJ4fRA1Dn1de88/fX+W4ETgzJkz+OGHH2BsbAwvLy/Y2trWe/igRCJBSUlJtfLi4mJYWlrWerxCocCmTZuQnJyMlStXCjqGiIiI/kdwInDs2DFYW1vjq6++goODg05O7ujoiNTUVJWyrKwslJWVacxeqvz888+4evUqPv/8czg6OuokJiIiIkMiuLNgZmYm+vbtq7MkAAB8fHxw69YtSKVSZVlERATEYnGt0xT/+uuvOHnyJAIDA+Hh4aGzmIiIiAyJ4ETAzs5O7doCdTV06FA0a9YM33//PaKjoxEaGoqQkBCMHj1apSNiYGCgSr+EixcvYu/evcoZDhMSEpS/apv+mIiIiP5H8KuBQYMG4cyZM5BKpTA3N9fJyS0tLfHFF19g+/btWLVqFSQSCUaNGoVJkyap1JPL5ZDL5crtW7duAQDOnTuHc+fOqdSdN28e/P39dRIfERHRy06kUCgUQirKZDKsX78e2dnZePvtt9GxY0edJQSNQdt5CoR4/On7Om+TqDG0Wf3vxg5Ba+/t/KOxQyCqt5+n99NLuzoZNfDWW28pf/7yyy/V1mvo4YNERERUd4ITgc6dO0MkEukzFiIiImpgghOB5cuX6zEMIiIiagyCRw0QERHRy4eJABERkQFT+2rgwIEDAIARI0bA0tJSuS3EhAkT6h8ZERER6Z3aRCAkJAQA0L9/f1haWiq3hWAiQERE1DSoTQSWLVsG4H9LIlZtExER0ctDbSLw/Fz/tc39T0RERE2P4M6C4eHhSEpK0ljn0aNHCA8Pr3dQRERE1DAEJwJbtmzB1atXNda5du0atmzZUu+giIiIqGHodPigXC7n7INERERNiE4TgbS0NEgkEl02SURERHqkcYrh5x/zX716FRkZGdXqyeVyZGdnIy4uDr6+vrqNkIiIiPRGYyLwfMe/xMREJCYmqq3v5uaG6dOn6yQwIiIi0j+NicCmTZsAAAqFAoGBgRg5ciRGjhxZrZ6RkREkEgnMzMz0EyURERHphcZEoGXLlsqfJ0yYAC8vL5UyIiIiatoEL0M8ceJEfcZBREREjUBwIvDw4UMkJCTg1VdfhYWFBQCgtLQU//73v3Ht2jWYmppi3LhxNb46ICIioheT4OGDR44cwaFDh5RJAAD88ssvuHDhAhQKBQoLC7Fz507cunVLL4ESERGR7glOBO7fvw8vLy/ldmVlJcLDw+Hq6ooff/wRmzZtgpWVFU6cOKGXQImIiEj3BCcCBQUFaNGihXL7wYMHKC0tRUBAAMRiMezs7NCzZ89a1yMgIiKiF4dWMwvKZDLlz3fu3AGguiqhlZUVCgoKdBQaERER6ZvgRMDe3h53795Vbl+9ehUtWrRAq1atlGW5ubmwtLTUbYRERESkN4JHDfTr1w8hISFYs2YNmjVrhoSEBIwaNUqlTmpqqkpiQERERC82wYnA6NGjcevWLVy5cgUA4OzsjAkTJij3Z2Rk4N69e3j99de1CiAlJQU7duxAQkICJBIJBg8ejIkTJ8LISPPDipKSEvz888+4evUq5HI5evTogRkzZqB58+ZanZ+IiMiQCU4EzMzMsHLlSjx69AgA4OTkVO1mvWjRIri4uAg+eVFREVauXAknJycsXrwY6enp2L17NxQKBaZMmaLx2HXr1iEtLQ2zZ8+GkZER9uzZg9WrV+PLL78UfH4iIiJDJzgRqNK+ffsayx0cHODg4KBVW6dPn0Z5eTkWLlwICwsLdOvWDVKpFCEhIRg7dqzKnAXPSkhIwK1bt7B8+XJlZ0U7OzssXboU0dHR6Natm3YfioiIyEBpfP4eGxuLrKwswY0lJSVVW7FQk5s3b8Lb21vlhu/n54fy8nLExsaqPS4qKgrW1tYqIxZcXV3h4OCAmzdvCj4/ERGRodOYCKxYsQLnzp1TKTt8+DBmzpxZY/0rV65gy5Ytgk+empqKtm3bqpTZ29vD1NQUaWlpGo9zdHSsVu7o6IjU1FTB5yciIjJ0Wr8aqKioQHFxsU5OXlxcDIlEUq1cIpGgqKhI43E1vTaQSCTIyMio8ZjQ0FCEhoYCAIKCgqolILrQds9/dd4mEQnz+9/fbOwQiJokrSYUasoCAgIQFBSEoKCgxg6F6ulvf/tbY4dAZJB47b2cGjURkEgkKCkpqVZeXFyscWIiiUQCqVRa43E1PWEgIiKimjVqIlDTO/2srCyUlZVpfHSvri9AWlpajX0HiIiIqGaNmgj4+Pjg1q1bKt/uIyIiIBaLVUYEPK979+7Iy8tTrncAPF0d8cmTJ/Dx8dFnyPQCCAgIaOwQiAwSr72XU6MmAkOHDkWzZs3w/fffIzo6GqGhoQgJCcHo0aNVOgMGBgZi69atym13d3d4e3tj06ZNiIyMxJUrV/DPf/4THh4enEPAAPA/I6LGwWvv5SRSKBQKdTsnT55cp0b3798vuG5KSgq2b9+uMsXwpEmTVGYtnD9/Pjw9PTF//nxlWXFxMXbu3IkrV65AoVDA19cXM2bMgJWVVZ1iJiIiMkSNnggQERFR49GYCBDp0qRJkwAAmzZtUpmOev78+cjMzFSpa2pqitatW6N3794YM2YMzMzMAAAnTpzATz/9hL59+2LBggXVzpGWloaPP/4YADBr1iwMHz68Wp3Q0FD88MMP6N69O/7+97/r6uMRNSlV12MVY2NjmJubw9raGh06dICPjw/69euHZs2a1Xi8uuvWwcEB3bt3x9ixY/mEtonQekIhIn3x9vaGjY0NACAnJwcJCQkICQnB5cuX8eWXX0IikaBz584AgPj4+BrbeLYD6Z07d2pMBOLi4gBA2RaRIevTpw/MzMygUCgglUqRkZGBiIgIXLx4EXv27MG8efPg7e2t9vhnr9u8vDwkJCTg6NGjuHTpEr7++mvY2dk10CehumIiQC+M8ePHw8vLS7mdkZGBFStWIDk5GYcOHcK0adPQvn17WFhYIDc3F+np6WjdurVKG3FxcRCJRGjXrp1KUvCsqnImAkTAtGnTqi0Yl5eXh4MHD+LUqVP49ttvsXTpUrUdsZ+/bnNzc/Hll18iNTUVwcHBmDNnjl7jp/ozmJkFqelxcHBQPr68evUqAMDIyAidOnUCgBpv9Hfu3IGTkxN8fX2RnZ1dbcrp7OxsZGZmQiwWa7VkNpEhsbGxwaxZszB58mTI5XJs3rwZlZWVgo61tbXFxIkTAQC3bt3SZ5ikI0wE6IXWoUMHAFBZBbPqm3zVI/4qubm5ePLkCTp16gQPDw8A1ZOFqmPc3NxgYsIHYkSavP7662jZsiVyc3Pxxx9/CD7OyckJAJCfn6+v0EiHmAjQC61qsqlnOyzVdpPv3Lkz3N3dIRKJqiULfC1AJJyRkRH69u0LAIiJiRF8XGlpKQDA2tpaL3GRbjERoBfa9evXAQDt27dXlrm6uqJZs2Z4/Pgx8vLylOVVN3kPDw9YWlrCycmpWrLwbB0iqp2zszMAaLXE+82bNwFAYydDenEwEaAXUk5ODo4dO4bjx48DAIYNG6bcZ2JiAldXVwDVRwm0aNECLVu2BAB06tQJqampKCgoAPB0Eqrk5GQYGxvD3d29oT4KUZPWvHlzANC4NHyV3NxcnDx5EkeOHEGbNm3qPBcNNSy+JKUXxooVK6qViUQivP7663j11VdVyjt37oy4uDjExcWhb9++KCkpQVJSEvr166es4+HhgdDQUNy5cwe9e/dGfHw8FAoFOnTooJyXgIjqp6br1sXFBV988QXMzc0bISLSFhMBemFUjUcWiUQQi8Vo3bo1evbsWW2IIPC/d/xVTwQSEhKgUChUHvlXjS6Ii4tD7969OX8AUR1UPVFTtzR81XUrk8mQkZGBhIQE3L9/Hz/99BPmzZvXkKFSHTERoBfG8+ORNXF3d4eRkRGSkpIglUprvMm3atUKtra2ymSBHQWJtJeYmAjgfyMBnvf8dRsTE4NvvvkG586dg6+vr7KzIb242EeAmiRzc3M4OztDLpcjPj4ed+7cgUQiQbt27VTqderUCYmJiSgoKMD9+/chEonYUZBIILlcjsuXLwMAunTpIugYLy8vTJgwAQCwd+9eyOVyvcVHusFEgJqsqm/2f/75J+7du6d8SvCsTp06QSaT4cSJE6isrISTk5PaR5xEpOrQoUPIyspCixYt0KdPH8HHjRo1CjY2Nnj8+DEuXbqkxwhJF5gIUJNVlQiEhYWhoqJC2SfgWVXf/k+ePKlyDBGpl5eXhx07diA4OBjGxsaYP3++VhNwicVijB8/HgDw66+/gmvbvdjYR4CarKqbfHFxscr2s5ydnWFqaqqsw0SASNXu3buViw6VlpYiIyMDjx49glwuh52dHebNmyf4tcCzAgICcOTIEaSkpODKlStaPVGghsVEgJosKysrODo6IjU1VWVugWcZGxvD1dVVOSsaEwEiVZGRkQBUlyHu378/fH190adPH7XLENem6qnATz/9hEOHDjEReIGJFHxmQ0REZLDYR4CIiMiAMREgIiIyYEwEiIiIDBgTASIiIgPGRICIiMiAMREgIiIyYEwEiIiIDBgTASJqFDExMZg0aRImTZrU2KEQGTTOLEgGr7y8HOHh4bh+/TqSkpJQUFAAExMT2NnZwcPDA35+frVOsTp//nxkZmZWKzczM0PLli3RuXNnjBgxosalXJcvX47Y2FhBsXp6emL58uWC6tYWW00GDRqE+fPna9X+84qLi/Hbb78BeLr4jEQiqVd7L6Jz584hIyMDXl5egpfOJnpRMREggxYdHY2tW7ciOztbWWZubo7KykqkpqYiNTUVZ86cQffu3fHhhx+iefPmGttr1qwZLCwsAAAKhQKFhYVITk5GcnIyzpw5gw8++ACDBw+u8VhjY+NaV0asz8qJz8amTm37hSguLsaBAwcAAP7+/moTAVNTU7Rt27be52sM586dUyZvTASoqWMiQAYrIiICGzduhEwmg52dHSZNmoTevXsrb7apqak4ffo0Tp06haioKHz22WdYuXIlrK2t1bbZv39/lW/U5eXluH79Onbs2IH8/Hz88MMPcHFxwSuvvFLt2E6dOmn9bV8bz8fW2FxdXbF+/frGDoPI4LGPABmklJQUbN26FTKZDO3bt8d3332HwYMHq3zjdnR0xHvvvYdPP/0UJiYmSE9Pxz//+U+tziMWi9GvXz8EBgYCAORyOX7//XedfhYiovrgEwEySPv27UNZWRmaNWuGBQsWwMrKSm1dX19fvPHGGwgODsaff/6JGzduwNfXV6vzdevWDba2tsjNzcX9+/frG36Dys7OxrFjxxAdHY3MzEzIZDI0b94cNjY26Ny5MwYMGKBc+fH5/g4ffvihSlvP9nGIiYnBihUrAADBwcEq9c6dO4ctW7agZcuW2Lx5M+Li4nDkyBHcu3cPZWVlaNOmDUaMGKHymuXGjRv47bffkJiYiLKyMrRr1w5jxoxB//79a/xcGRkZiIiIQExMDDIyMpCTkwMAsLe3h7e3N0aPHg17e/sa46py4MAB5WuQKps2bYKDg4NyWy6X49y5c7hw4QIePXoEqVSK5s2bo1OnThg+fLjaVwtVf5YTJkzAG2+8gRMnTuDSpUtIT09HSUkJli1bpjw2NTUVx48fR2xsLLKzs6FQKGBlZQU7Ozt4eXlh0KBBcHR0rPE8REwEyODk5ubi6tWrAAA/Pz9B76lHjx6NY8eOQSqV4tSpU1onAgBgZ2eH3NxcSKVSrY9tLImJiVixYgWKi4sBAEZGRjA3N0deXh5yc3Px8OFDFBcXKxMBS0tLNG/eHIWFhQCA5s2bw8jofw8e69LH4cyZM/jhhx8APO2/UVZWhsTERGzbtg3p6el46623EBwcjAMHDkAkEsHc3Bzl5eW4f/8+1q9fj6KiIgwbNqxau1u2bFEmLSYmJjA3N0dRUZGyb8i5c+fwt7/9DR4eHspjxGIxrK2tUVRUBJlMBlNTU5iZmam0++znLSkpwerVq5XLYD/753f58mVcvnwZY8aMwbRp09R+/oqKCqxYsQLx8fEwNjaGmZkZRCKRcn90dDRWrVqFiooKAFDWyc7ORnZ2Nu7evQsTExOOziC1mAiQwYmJiUHV6ttC10g3MzNDt27dEBkZibi4OMhkMhgbG2t13qqe+/Xp8NfQdu/ejeLiYnTo0AGzZs2Cm5sbRCIRKisrkZmZiWvXruHZlcwXLVqEjIwM5ZOAb7/9VuXbsbYKCgqwfft2jBgxAm+++SasrKxQVFSEnTt3Ijw8HEeOHIFEIsGhQ4cwZcoUjBgxAhYWFsjNzcXWrVtx8+ZN7N69GwMGDKjWEdLZ2Rn9+vVDt27d0KpVKxgZGUEmk+Hhw4cIDg7GzZs3sW7dOmzcuBFisRjA034W/fv3V35bHzNmjMYb7NatWxETEwMTExNMmzYNgwcPhqmpKfLy8rB3716cPXsWx44dQ6tWrWpMVgDg1KlTAIB58+ahf//+EIvFKCwsVCYDP/74IyoqKuDt7Y1p06ahffv2AJ72T3ny5AkiIyOrPdkgehYTATI4KSkpyp87dOgg+DhnZ2dERkaitLQUmZmZaN26teBjL1++jIKCAgCAm5tbjXXi4+PxwQcfaGxnxowZah911yYiIgI3b97UWGfRokXo1KmTSkwAMGvWLLi7uyvLTUxM0KZNG4wZM6ZOsQhVVlaGwYMHY8aMGcoyS0tLzJ07F3FxccjIyMCePXswZcoUvPHGG8o6tra2+PjjjzF79myUlZXh2rVrGDhwoErb7733XrXzGRsbw9XVFX/729+wZMkSJCUl4fLly9WOFeLu3buIjIwEAMycORMBAQHKfTY2Npg7dy5KSkoQGRmJ/fv3w9/fX5lwPKu0tBSLFy9Gz549lWVVo1fy8/Px5MkTAE8TBVtbW2UdsViMdu3aoV27dlrHToaFnQXJ4FQ9tga0+3b+7NDBoqKiWusrFApkZmbixIkT2Lp1K4CnN9Dhw4fXWF8mkyE/P1/jr/LycsHxPq+ioqLW9isrK1WOqRr6l5ubW+fz1tf48eOrlRkZGSnndmjWrBlGjhxZrY6FhYUyeXn06JFW5zQyMoK3tzcA4M6dO1pG/FRERAQAoEWLFmqHjE6ePBnA03+T0dHRNdZp166dShLwLHNzc+WTgcb8O6KmjU8EiHQoPDwc4eHhNe4zMzPD/Pnz0aZNmxr312WyIG3UZbIgX19fnDlzBps3b0Z8fDx69uwJFxcXmJqa6ilKVZaWlmqfvNjY2AAAnJycqr2nr1I11FNd4hYXF4ewsDDcvXsX2dnZKCsrq1anqhOhth48eADg6TwDz/YbeJaTkxPs7OyQk5ODBw8e1HjDf/YJzfPEYjG6du2K6OhofPPNNxg6dCh8fX3RoUMHmJjwv3cShv9SyOA8/83ezs5O0HFCniQ8O2mPSCSCqakp7O3t0blzZwwZMgQtWrSoR+QN75133kF6ejpiYmJw/PhxHD9+HEZGRnB2doavry8CAgIE//nVhbm5udp9VTdXTXWq+nHIZLJq+/7zn//g6NGjKu1JJBLlDbS0tBRlZWU1JgdC5OfnA0Ctfz4tWrRATk6Osv7zNI1oAYA5c+Zg1apVSEpKwsGDB3Hw4EGYmJjAxcUFvXr1qjYsluh5TATI4Dw7ze+DBw8E38gePnwI4H/TBtfkRZu0p74kEgmWLVuGO3fu4Nq1a4iPj8eDBw+Uv44ePYo5c+ZgwIABjR2qVqKjo5VJwLBhwzBs2DA4OTmpfHPft28fDh06pNIZsjGoe5pQxd7eHqtWrUJ0dDSioqIQHx+PpKQkxMfHIz4+Hr/++isWLlxY6zTZZLiYCJDB8fLygkgkgkKhQGRkpNr3r88qLS3Fn3/+CQDo3Lmz1iMGmjoPDw/lMLry8nJER0dj3759ePToEbZu3YouXbooH9U3BZcuXQIAeHt74/3336+xTl5eXr3OYW1tjbS0NJXpq2tStV/TjJW1MTIygo+PD3x8fAAAUqkU169fxy+//IKsrCxs2LABW7du5esCqhE7C5LBsbW1Ra9evQA87dCVlpZW6zHHjx9Xjv9XN8zLUIjFYvTs2ROLFi0C8LQT4rMd6mr7BvsiqLr5qhs1olAolGP/a/LsOH51OnbsCODpcFW5XF5jndTUVGUfBBcXl1rbFMrc3BwDBgzAnDlzADx9TaFth0kyHC/+FUukB5MnT4ZYLEZFRQXWrl2rHNpXk6ioKBw6dAjA06cJdZlMqCmSyWRqb2AAVIa6PXvzf/adfdVERC+aqn4cSUlJNe4/ffq0clheTao+o6bP5+fnB+BpZ8OwsLAa6+zfvx/A034rXbt2rT3w5zw/yuN5z/4dCUleyDAxESCD1K5dO8yZMwdGRkZ49OgRlixZgrCwMJX/2NPS0rBz50589913qKysRKtWrfB///d/BvMfanZ2Nv7v//4PBw8exMOHD1U63CUlJWHjxo0Anq4i6OnpqdwnkUiU/S7Onj1bY0e9xlb1CD0qKgoHDhxAaWkpgKc39kOHDmHHjh0aV5qsmrQnKipK7agCV1dX5YRVO3bswMmTJ5UdD/Py8rBt2zZcvnwZwP8SU23Fx8dj0aJFOH78OFJSUpSJm0KhQHx8PP79738DeNohsaaFrogA9hEgAzZgwABYWloqlyHetm0btm3bBgsLC1RUVCinbAWevksODAystQd3fQiZUAh4OpNcXQiZUMje3h7ffvutcvvJkyfYv38/9u/fDyMjI1hYWKC0tFT5TdTExATz58+v1it96NCh2L9/P06ePIkzZ87AysoKRkZGcHNzw8cff1yn+HVp4MCBCA8PR1xcHIKDgxESEgILCwuUlJRAoVDA19cXzs7OyidBzxs0aBCOHTuG9PR0zJ07F1ZWVsob+ZdffqkcHTJ37lwUFhYiNjYWO3bswM6dO2FmZqY8DwCMGTOmXq+bHj16hF27dmHXrl0wNjZWfo6qBMzc3BwfffRRk3hlQ42DiQAZNB8fH2zcuBHnzp3D9evXkZSUhMLCQpiYmCiH/fn5+dXpsa22qiYU0peqCYU0efZbqZ2dHRYvXoyYmBgkJCQoh7gZGxujdevW8PLywsiRI2ucF+H111+Hubk5Lly4oHwPrlAo1I62aGgmJib47LPPcPjwYVy6dEk5/bOrqysGDRqEgICAaosJPatNmzZYtmwZDh8+jLt37yrXHgBUhypaWFjgiy++UC46lJiYiNLSUtjY2MDd3R0jRoxQu+iQEC4uLvjkk08QExODe/fuITc3FwUFBWjWrBnatWuHbt26YeTIkXod4klNn0jR2GNjiIiIqNHwWREREZEBYyJARERkwJgIEBERGTAmAkRERAaMiQAREZEBYyJARERkwJgIEBERGTAmAkRERAaMiQAREZEBYyJARERkwP4fVrj3oAL2lZAAAAAASUVORK5CYII=\n" 311 | }, 312 | "metadata": {} 313 | } 314 | ], 315 | "source": [ 316 | "# 内部で用いる分類器としてロジスティック回帰を指定したIPWLearnerの性能をOPEにより評価\n", 317 | "ope.visualize_off_policy_estimates(\n", 318 | " action_dist=action_dist, # evaluation_policy_aによるバリデーションデータに対する行動選択\n", 319 | " estimated_rewards_by_reg_model=estimated_rewards_by_reg_model,\n", 320 | " is_relative=True, # 過去の意思決定モデルの性能に対する相対的な改善率を出力\n", 321 | " random_state=12345,\n", 322 | ")" 323 | ] 324 | }, 325 | { 326 | "cell_type": "markdown", 327 | "metadata": {}, 328 | "source": [ 329 | "ここで得られた意思決定モデルの性能評価の結果から、データ収集時に用いられていたBernoulliTSモデルからIPWLearnerによる特徴量の情報を活用した個別化推薦に切り替えることで、クリック確率（意思決定モデルの性能）を30%程度向上させられる可能性が示唆された。IPWLearnerの性能について推定された95%信頼区間の下限も、1.0付近（ベースラインであるBernoulliTSモデルの性能と同程度）であるため、大きな失敗はしなさそうである。この性能評価の結果に基づき、IPWLearnerを実環境にいきなり導入したり、IPWLearnerが有望な意思決定モデルであることに対して自信を持った上で、安全にA/Bテストに進んだりできる。" 330 | ] 331 | }, 332 | { 333 | "cell_type": "code", 334 | "execution_count": null, 335 | "metadata": {}, 336 | "outputs": [], 337 | "source": [] 338 | } 339 | ], 340 | "metadata": { 341 | "kernelspec": { 342 | "name": "python3", 343 | "display_name": "Python 3.8.2 64-bit ('3.8.2': pyenv)" 344 | }, 345 | "language_info": { 346 | "codemirror_mode": { 347 | "name": "ipython", 348 | "version": 3 349 | }, 350 | "file_extension": ".py", 351 | "mimetype": "text/x-python", 352 | "name": "python", 353 | "nbconvert_exporter": "python", 354 | "pygments_lexer": "ipython3", 355 | "version": "3.8.2" 356 | }, 357 | "metadata": { 358 | "interpreter": { 359 | "hash": "a588998c237fcc28dc215a10a422972d26151263dec0bff02e1a95f6e2b22b77" 360 | } 361 | }, 362 | "interpreter": { 363 | "hash": "a588998c237fcc28dc215a10a422972d26151263dec0bff02e1a95f6e2b22b77" 364 | } 365 | }, 366 | "nbformat": 4, 367 | "nbformat_minor": 4 368 | } -------------------------------------------------------------------------------- /ch02/synthetic-data.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "## 2章: Open Bandit Pipelineを用いた意思決定モデルの学習/性能評価の実装\n", 8 | "\n", 9 | "この実装例は主に次のステップで構成される。\n", 10 | "\n", 11 | "1. ある古い意思決定モデル$\\pi_b$が稼働することで収集されたログデータを模した工データを生成する\n", 12 | "2. トレーニングデータを用いて意思決定モデルを学習し、バリデーションデータに対して行動を選択する\n", 13 | "3. 学習した意思決定モデルの性能を、バリデーションデータを用いて推定する" 14 | ] 15 | }, 16 | { 17 | "cell_type": "code", 18 | "execution_count": 1, 19 | "metadata": {}, 20 | "outputs": [], 21 | "source": [ 22 | "# 必要なパッケージやモジュールをインポート\n", 23 | "from sklearn.linear_model import LogisticRegression\n", 24 | "\n", 25 | "from obp.dataset import (\n", 26 | " SyntheticBanditDataset,\n", 27 | " logistic_reward_function,\n", 28 | " linear_behavior_policy\n", 29 | ")\n", 30 | "from obp.policy import IPWLearner, Random\n", 31 | "from obp.ope import (\n", 32 | " OffPolicyEvaluation, \n", 33 | " RegressionModel,\n", 34 | " InverseProbabilityWeighting as IPS,\n", 35 | " DoublyRobust as DR\n", 36 | ")" 37 | ] 38 | }, 39 | { 40 | "cell_type": "code", 41 | "execution_count": null, 42 | "metadata": {}, 43 | "outputs": [], 44 | "source": [] 45 | }, 46 | { 47 | "cell_type": "markdown", 48 | "metadata": {}, 49 | "source": [ 50 | "### 1. 人工データの生成" 51 | ] 52 | }, 53 | { 54 | "cell_type": "code", 55 | "execution_count": 2, 56 | "metadata": {}, 57 | "outputs": [ 58 | { 59 | "output_type": "execute_result", 60 | "data": { 61 | "text/plain": [ 62 | "{'n_rounds': 10000,\n", 63 | " 'n_actions': 3,\n", 64 | " 'context': array([[-0.20470766, 0.47894334, -0.51943872],\n", 65 | " [-0.5557303 , 1.96578057, 1.39340583],\n", 66 | " [ 0.09290788, 0.28174615, 0.76902257],\n", 67 | " ...,\n", 68 | " [ 0.42468038, 0.48214752, -0.57647866],\n", 69 | " [-0.51595888, -1.58196174, -1.39237837],\n", 70 | " [-0.74213546, -0.93858948, 0.03919589]]),\n", 71 | " 'action_context': array([[1, 0, 0],\n", 72 | " [0, 1, 0],\n", 73 | " [0, 0, 1]]),\n", 74 | " 'action': array([0, 1, 0, ..., 0, 0, 2]),\n", 75 | " 'position': None,\n", 76 | " 'reward': array([0, 1, 1, ..., 0, 0, 0]),\n", 77 | " 'expected_reward': array([[0.62697512, 0.66114455, 0.66545218],\n", 78 | " [0.73402729, 0.92955625, 0.94007301],\n", 79 | " [0.72522191, 0.79973865, 0.85946747],\n", 80 | " ...,\n", 81 | " [0.74929842, 0.68243742, 0.77801157],\n", 82 | " [0.3583225 , 0.25252838, 0.16625489],\n", 83 | " [0.41919738, 0.52158296, 0.41624562]]),\n", 84 | " 'pscore': array([0.25534252, 0.36715004, 0.25534252, ..., 0.25534252, 0.25534252,\n", 85 | " 0.37750744])}" 86 | ] 87 | }, 88 | "metadata": {}, 89 | "execution_count": 2 90 | } 91 | ], 92 | "source": [ 93 | "# `SyntheticBanditDataset`を用いて人工データを生成する\n", 94 | "dataset = SyntheticBanditDataset(\n", 95 | " n_actions=3, # 人工データにおける行動の数\n", 96 | " dim_context=3, # 人工データにおける特徴量の次元数\n", 97 | " reward_function=logistic_reward_function, # 目的変数を生成する関数\n", 98 | " behavior_policy_function=linear_behavior_policy, # 過去の意思決定モデル\\pi_bによる行動選択確率を生成する関数\n", 99 | " random_state=12345,\n", 100 | ")\n", 101 | "\n", 102 | "# トレーニングデータとバリデーションデータを生成する\n", 103 | "training_data = dataset.obtain_batch_bandit_feedback(n_rounds=10000)\n", 104 | "validation_data = dataset.obtain_batch_bandit_feedback(n_rounds=10000)\n", 105 | "\n", 106 | "# `training_data`の中身を確認する\n", 107 | "training_data" 108 | ] 109 | }, 110 | { 111 | "cell_type": "code", 112 | "execution_count": null, 113 | "metadata": {}, 114 | "outputs": [], 115 | "source": [] 116 | }, 117 | { 118 | "cell_type": "markdown", 119 | "metadata": {}, 120 | "source": [ 121 | "### 2. 意思決定モデルの学習（Off-Policy Learning; OPL）\n", 122 | "\n", 123 | "トレーニングデータを用いて次の2つの意思決定意思決定モデルを学習し、バリデーションデータに対して目的変数を最大化する行動を選択する。\n", 124 | "\n", 125 | "1. IPWLearner+ロジスティック回帰\n", 126 | "2. ランダム意思決定モデル" 127 | ] 128 | }, 129 | { 130 | "cell_type": "code", 131 | "execution_count": 3, 132 | "metadata": {}, 133 | "outputs": [ 134 | { 135 | "output_type": "stream", 136 | "name": "stdout", 137 | "text": [ 138 | "CPU times: user 66.3 ms, sys: 1.8 ms, total: 68.1 ms\nWall time: 66.9 ms\n" 139 | ] 140 | } 141 | ], 142 | "source": [ 143 | "%%time\n", 144 | "# 「IPWLearner+ロジスティック回帰」を定義\n", 145 | "ipw_learner = IPWLearner(\n", 146 | " n_actions=dataset.n_actions,\n", 147 | " base_classifier=LogisticRegression(C=100, random_state=12345)\n", 148 | ")\n", 149 | "\n", 150 | "# トレーニングデータを用いて、意思決定意思決定モデルを学習\n", 151 | "ipw_learner.fit(\n", 152 | " context=training_data[\"context\"], # 特徴量\n", 153 | " action=training_data[\"action\"], # 過去の意思決定モデル\\pi_bによる行動選択\n", 154 | " reward=training_data[\"reward\"], # 観測される目的変数\n", 155 | " pscore=training_data[\"pscore\"], # 過去の意思決定モデル\\pi_bによる行動選択確率(傾向スコア)\n", 156 | ")\n", 157 | "\n", 158 | "# バリデーションデータに対して行動を選択する\n", 159 | "action_choice_by_ipw_learner = ipw_learner.predict(\n", 160 | " context=validation_data[\"context\"],\n", 161 | ")" 162 | ] 163 | }, 164 | { 165 | "cell_type": "code", 166 | "execution_count": 4, 167 | "metadata": {}, 168 | "outputs": [ 169 | { 170 | "output_type": "stream", 171 | "name": "stdout", 172 | "text": [ 173 | "CPU times: user 300 µs, sys: 94 µs, total: 394 µs\nWall time: 299 µs\n" 174 | ] 175 | } 176 | ], 177 | "source": [ 178 | "%%time\n", 179 | "# ランダム意思決定モデルを定義\n", 180 | "random = Random(n_actions=dataset.n_actions)\n", 181 | "\n", 182 | "# バリデーションデータに対する行動選択確率を計算する\n", 183 | "action_choice_by_random = random.compute_batch_action_dist(\n", 184 | " n_rounds=validation_data[\"n_rounds\"]\n", 185 | ")" 186 | ] 187 | }, 188 | { 189 | "cell_type": "code", 190 | "execution_count": null, 191 | "metadata": {}, 192 | "outputs": [], 193 | "source": [] 194 | }, 195 | { 196 | "cell_type": "markdown", 197 | "metadata": {}, 198 | "source": [ 199 | "### 3. 意思決定モデルの性能評価（Off-Policy Evaluation; OPE）\n", 200 | "\n", 201 | "2つの意思決定意思決定モデルの性能を、バリデーションデータを用いて評価する。オフライン評価には、IPS推定量とDR推定量を用いる。" 202 | ] 203 | }, 204 | { 205 | "cell_type": "code", 206 | "execution_count": 5, 207 | "metadata": {}, 208 | "outputs": [ 209 | { 210 | "output_type": "stream", 211 | "name": "stdout", 212 | "text": [ 213 | "CPU times: user 60.8 ms, sys: 23.4 ms, total: 84.2 ms\nWall time: 14 ms\n" 214 | ] 215 | } 216 | ], 217 | "source": [ 218 | "%%time\n", 219 | "# DR推定量に必要な目的変数予測モデルを得る\n", 220 | "# opeモジュールに実装されている`RegressionModel`に好みの機械学習手法を与えば良い\n", 221 | "regression_model = RegressionModel(\n", 222 | " n_actions=dataset.n_actions, # 行動の数\n", 223 | " base_model=LogisticRegression(C=100, random_state=12345), # ロジスティック回帰を使用\n", 224 | ")\n", 225 | "\n", 226 | "# `fit_predict`メソッドにより、バリデーションデータにおける期待報酬を推定\n", 227 | "estimated_rewards_by_reg_model = regression_model.fit_predict(\n", 228 | " context=validation_data[\"context\"], # 特徴量\n", 229 | " action=validation_data[\"action\"], # 過去の意思決定モデル\\pi_bによる行動選択\n", 230 | " reward=validation_data[\"reward\"], # 観測される目的変数\n", 231 | " random_state=12345,\n", 232 | ")" 233 | ] 234 | }, 235 | { 236 | "cell_type": "code", 237 | "execution_count": 6, 238 | "metadata": {}, 239 | "outputs": [], 240 | "source": [ 241 | "# 意思決定モデルの性能評価を一気通貫で行うための`OffPolicyEvaluation`を定義する\n", 242 | "ope = OffPolicyEvaluation(\n", 243 | " bandit_feedback=validation_data, # バリデーションデータ\n", 244 | " ope_estimators=[IPS(estimator_name=\"IPS\"), DR()] # 使用する推定量\n", 245 | ")" 246 | ] 247 | }, 248 | { 249 | "cell_type": "code", 250 | "execution_count": 7, 251 | "metadata": {}, 252 | "outputs": [ 253 | { 254 | "output_type": "display_data", 255 | "data": { 256 | "text/plain": "
", 257 | "image/svg+xml": "\n\n\n \n \n \n \n 2021-06-30T12:15:16.902441\n image/svg+xml\n \n \n Matplotlib v3.4.2, https://matplotlib.org/\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n\n", 258 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgIAAALgCAYAAADmyR9FAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAACQLUlEQVR4nOzdeVyU5f4//teADssg64CyaCiyCAaIhAsmiqCeJNFybbPSk1nRr5NlnzbJqBOUVpZLp5JOejQV9bhmxog7LqigySKWggKyqcjOADO/P/wyxxEY7oEZEOf1fDx65Fz3dV33e3p4d7/nvq9FpFQqlSAiIiKDZNTVARAREVHXYSJARERkwJgIEBERGTAmAkRERAaMiQAREZEBYyJARERkwJgIEBERGTAmAkTULiKRCCKRqNXypn+MjY0hlUoRGhqKDRs2NKvf2NiIH374ASEhIbC1tUXPnj3h4OAAX19fzJs3Dzt37uyMr0NksERcUIiI2qMpCbj3fyFN5dHR0QCA+vp6ZGVlYceOHWhsbMQ//vEPfPnllwDuJAERERH47bffYG1tjUmTJsHFxQVyuRzp6ek4cuQIAgICcPTo0U78ZkSGhYkAEbVLW4nAveX79+9HeHg4AODy5ctwdXXFf/7zHzz77LPw8/PDoUOHYGVlpdamuroaJ0+exNixY/X1NYgMHl8NEFGnGDduHLy8vKBUKpGSkgIASE5OBgA8//zzzZIAADA3N2cSQKRnTASIqNM0PSVoempgZ2cHAMjOzu6ymIgMneBXAwUFBfjjjz+QmZmJ0tJSVFRUQCwWw9LSEq6urvDx8cHgwYMhFov1HTMR3Qe0fTUgk8kwfvx4AMCVK1fw0EMPITU1FcOGDUNDQwOefvppTJ06FUOHDsVDDz3UCd+AiAABicCxY8fw+++/Iysrq83OJBIJxowZg4kTJ8LBwUFnQRLR/UebwYIXL17E9u3bmw0WBIDNmzfj//v//j8UFhaqymxtbTF69Gi8+OKLePzxx/X9VYgMWquJwIULF7B27Vrk5ubC3NwcjzzyCLy8vODm5gZra2tYWFhALpejoqICBQUFyM7Oxvnz53Hp0iX06NEDf/vb3/DEE0/A3Ny8s78TEXWCthKBuz9bW1vDz88Pc+fOxTPPPNOsr/r6ehw4cABHjx5Famoqjh49irKyMgDAc889h3//+98tTlUkoo5rNRGYOXMm+vfvj8jISAQGBqJnz56COrx+/ToSExORmJiIyMhITJs2TacBE9H9QdtXA9pobGzE1q1b8eKLL6Kqqgr//e9/MWXKlHb3R0StazUROHXqFIKCgtrdcVlZGYqLi+Hh4dHuPojo/qXPRKDJhx9+iE8++QRRUVH45ptvOtwfETXXo7UDHUkCAMDa2hrW1tYd6oOIDFuvXr0A6CapIKKWcfogEXWZX375BYmJiVAoFM2OFRYW4ocffgAAjB49urNDIzIYrT4RICLSt5MnT2L58uXo06cPRo0ahf79+wO4M71wz549qKmp4VgjIj3TOH3wtdde075DkQjffvtth4IiovufLsYIXLt2DTt37oRMJkNGRgauX7+O2tpa2NnZYciQIXjqqafw1FNPwciIDy+J9EVjIjBz5sx2dbpp06Z2B0RERESdR2MiUFJS0q5O7e3t2x0QERERdR7uPkhERGTA+OKNiIjIgGmcNaBQKPD1119DJBIhKioKPXq0XL2hoQHffvstRCIR3njjDX3ESURERHqg8YnAyZMncfLkSQQGBraaBABAjx498Mgjj+D48eM4ceKEzoMkIiIi/dCYCBw/fhy2trYYNWpUmx0FBwfD1tYWR48e1VlwREREpF8aXw389ddf8PHxEbTrl0gkwuDBg5Genq6z4PSpoKCgq0MgIiLqFE5OTq0e0/hEoKysDHZ2doJPZGtri9u3bwuPjIiIiLqUxkSgR48eqK+vF9xZfX29xrEEREREdH/RmAjY2NggNzdXcGe5ubmwsbHpcFBERETUOTQmAp6ensjIyEBhYWGbHRUWFiIjIwNeXl46C46IiIj0S2MiEB4eDoVCgS+//FLju//y8nJ89dVXUCgUCAsL03mQREREpB8aX+gPHDgQYWFhkMlkePPNNxEeHo7BgwfD1tYWAHDz5k1cuHABMpkMFRUVCA8Px8CBAzslcCIiIuq4NvcaaGxsxI8//oikpCSNHY0bNw7z5s3rNtuFcvogEREZCk3TBwVvOnTx4kUkJibi4sWLKCsrAwBYW1vDy8sLYWFh8PT01EmwnYWJABERGQqdJAIPGiYCRERkKNq9oBARERE92JgIEBERGTAuA0jdSlxcHEpLSyGVSvHOO+90dThERN0eEwHqVkpLSwUtcEVERMLw1QAREZEBYyJARERkwJgIEBERGbAOjREoLi5GXl4eAMDFxQUODg46CYqIiIg6R7sSgZqaGnz33Xc4ceKEWvmIESPw8ssvw9TUVCfBERERkX61KxFYs2YNzp8/jxkzZmDAgAGor6/H6dOncejQIZiYmGDBggW6jpOIiIj0QGMiUFdXBxMTk2blKSkpmDdvHh599FFVWVBQEOrq6nDq1CkmAkRERN2ExsGCb731Fi5cuNCsvLGxEWZmZs3KzczMoFAodBcdERER6ZXGJwLu7u6IiYnBuHHj8Oyzz6pu/oMHD8aaNWtQW1uL/v37o76+HmfOnMGhQ4cwdOjQTgmciIiIOk5jIvD6669j1KhR+OGHH5CamoqXXnoJQ4YMwbx58/DFF1/g22+/Vas/YMAAvPjii3oNmIiIiHRH0DbE1dXVWLt2LQ4cOIBHH30Uzz//PCwsLHD+/Hnk5+cDuDN98OGHH9Z7wLqij22Ir789T+d9krrPS+QobVRCaizCIntxV4fzwHL84seuDoGIdEjTNsSCZg2Ym5vj5ZdfxsiRI/H9999j4cKFmDt3LoKCguDr66uzQImIiKhzaTV90NfXF0uXLsX69euxbNkyDB8+HHPnzoWlpWW7A8jLy0N8fDyys7MhkUgQGhqK6dOnw8io9XGMmzdvxpYtW1o8Nnv2bEydOrXd8RARERkSQYlAeXm5autXS0tLzJ07FyNHjsR3332Hf/zjH3jhhRcwatQorU9eWVmJmJgYuLi4YNGiRSgsLMS6deugVCoxa9asVtuNGzcO/v7+amUpKSnYsWMHhgwZonUcREREhkpjIlBbW4vVq1errSA4bNgwvPLKKxg0aBC++OILbNy4EStXrkRycjJeeuklWFtbCz55YmIi5HI5Fi5cCHNzc/j6+qKmpgYJCQmYPHkyzM3NW2xnZ2cHOzs7tbKtW7fC2dkZrq6ugs9PRERk6DSuI7BhwwacOHECISEhmDt3LsaMGYOTJ09i/fr1AACxWIznnnsOMTExKCwsxD/+8Q8cOHBA8MnT0tLg5+endsMPDg6GXC5HRkaG4H4qKipw/vx5BAcHC25DREREbTwRSElJUT0BaFJTU4PTp09j7ty5qrKBAwfi888/x5YtW/DDDz9g7Nixgk6en58PHx8ftTKpVAoTExOtRvWfPHkSjY2NTASIiIi01OYSw/c+grezs2txtcEePXpg1qxZGD58uOCTV1VVQSKRNCuXSCSorKwU3M+xY8fQv39/ODo6tlpHJpNBJpMBAGJjYyGVSgX3L9R1nfdI1DX0cX0Q0f2pzZUFDx8+jGHDhmHgwIG4fPkyjhw5And391bbdPY7+lu3biEjIwNPP/20xnphYWEICwtTfS4tLdV3aETdFq8PogdLu9cReOGFF7BkyRJER0erymxtbfH888/rJDCJRILq6upm5VVVVbCwsBDUx/HjxwEAI0eO1ElMREREhkRjItCnTx98/fXXOHPmjGr6YEBAAExNTXVycmdnZ9XKhE1KS0tRV1enMXu527Fjx+Dl5cVHmQbCxhgARP/v30RE1FFtriNgYmKit1/b/v7+2LlzJ2pqalQbGiUnJ0MsFsPb27vN9sXFxbh06RLmzePSvobi77ZcVpiISJc0Th/Ut/DwcPTs2RNLly7F+fPnIZPJkJCQgIiICLUphVFRUVi9enWz9snJyTA2NtZqgCIRERH9j1ZLDN/t9OnTyMzMRF1dHRwcHDBy5EitH89bWFhg8eLFWLNmDeLi4iCRSDBp0iTMmDFDrZ5CoYBCoWjW/tixYxg8eHCHljgmIiIyZBp3H9ywYQN8fX0xePBgVVlVVRU+//xzZGVlqdXt0aMH5s+fj9GjR+svWh3i7oNErePug0QPlnbPGtixYwfEYrFaIvCvf/0LWVlZcHBwQHBwMCwtLZGdnY3jx4/ju+++g6urK/r166e76ImIiEhvtHo1UFhYiJMnT6J///6Ijo5WDfB77LHHEBAQgJUrV+LXX3/Fyy+/rJdgiYiISLe0GiyYmZkJ4M5Wv01JQJPRo0dj4MCBWu0RQERERF1Lq0SgrKwMAODm5tbicTc3N9y8ebPDQREREVHn0CoRaHoK0LNnzxaP9+zZEyKRqONRERERUadoc4xAenq66s+FhYUAgJKSEri4uDSre+PGDfTq1UuH4REREZE+tZkIZGRkNHvvf/bs2RYTgcuXL8PZ2Vl30RER0X0hLi5OtdT8O++809XhkA5pTATu3mzobi0t4HP58mU0Njbi4Ycf1k1kRER03ygtLVU9FaYHi8ZEQMh6/00GDBiAlStXdjggIiIi6jxdutcAERERdS2tFhRqbGxEUVERqqqqIBKJYGVlBXt7e33FRkQk2PM/H+/qEB5ojeU1AIDC8hr+t9ajf88Z0ennFJQInDp1Cvv27UNmZiYaGxvVjllaWiI4OBhTpkyBtbW1PmIkIiIiPdGYCCiVSqxatQqHDx9udkwqlcLU1BSFhYXYu3cvjhw5grfffhteXl56C5aIiIh0S2MiIJPJcPjwYQQEBGDmzJno3bs3ioqKsHnzZly8eBHvv/8+7O3tcezYMaxbtw5xcXFYtmwZbG1tOyt+IiIi6gCNgwWTkpLg4uKCt956C66urjAzM4OrqysWLlwIa2trbNiwAT179sSYMWPw4Ycfora2Ftu3b++k0ImIqNOY9gLMre78mx4oGp8I5OXlYdy4cTA2NlYrNzY2xsMPP4xDhw6pylxdXREQEIDU1FT9REpERF3GOGByV4dAeqLxiYBIJIJcLm/xmFwuR319vVqZs7MzNx0iIiLqRjQmAn379sXp06dRWVmpVl5ZWYnTp0/D0dFRrby2thZisVj3URIREZFeaHw1MHbsWPzwww947733EBERAQcHBxQXF2PPnj24ffs2IiIi1Opfu3YNffr00WvAREREpDsaE4GwsDBkZGTg2LFjWLNmjdoxf39/tUSgpqYGcrkcI0eO1E+kREREpHNtLij0+uuvY/jw4Th16hRu376NXr16ISAgACNHjoSR0f/eLJiZmeHTTz/Va7BERESkW4JWFgwKCkJQUJC+YyEiIqJOxk2HiIiIDBgTASIiIgPGRICIiMiAMREgIiIyYIIGC+pTXl4e4uPjkZ2dDYlEgtDQUEyfPl1tRkJrTp48ie3bt+Pq1aswMTGBm5sbFi5cCFNT006InIiIqPvr0kSgsrISMTExcHFxwaJFi1BYWIh169ZBqVRi1qxZGtvu378f8fHxmDx5Mp555hlUVVXhwoULUCgUnRQ9ERFR99eliUBiYiLkcjkWLlwIc3Nz+Pr6oqamBgkJCZg8eTLMzc1bbFdeXo6ff/4ZL7zwAsLCwlTlnOJIRESknS4dI5CWlgY/Pz+1G35wcDDkcjkyMjJabXf8+HEAwJgxY/QdIhER0QNN60QgIyMDW7Zs0fpYS/Lz8+Hk5KRWJpVKYWJigoKCglbbXbp0CU5OTkhKSsLLL7+M2bNn47333sPFixcFn5uIiIja8WogPT0dW7ZswbRp07Q61pKqqipIJJJm5RKJpNmOh3e7ffs2CgoKsHXrVjzzzDPo1asXduzYgX/+859Yvnw5rK2tm7WRyWSQyWQAgNjYWEilUkExauO6znsk6hr6uD6IqG1dce11+ayB9lAqlaitrcWbb74Jf39/AICHhwdeffVV/Pbbby0ONAwLC1MbT1BaWtpZ4RJ1O7w+iLqGvq69e5++361LxwhIJBJUV1c3K6+qqoKFhYXGdiKRCN7e3qoyc3NzDBgwAHl5eXqJlYiI6EHUpYmAs7Mz8vPz1cpKS0tRV1enMXtxdnaGUqlsVq5UKgWtP0BERER3CLprlpaWqv6pqqpqVtbeRxn+/v44d+4campqVGXJyckQi8Vqv/bvNXToUADAhQsXVGXV1dW4fPkyHnrooXbFQkREZIgEjRF49dVXNZaJRCJs3LhR65OHh4dj7969WLp0KSIjI1FcXIyEhARERESoTSmMioqCt7c3FixYAABwc3NDYGAgvvvuOzz11FOwtLTEjh07YGxsjAkTJmgdBxERkaESlAg8+eSTEIlEAO5MEczIyBA8M0ATCwsLLF68GGvWrEFcXBwkEgkmTZqEGTNmqNVTKBTNVgx8/fXXsW7dOqxduxZ1dXXw8vJCdHS0xrEFREREpE5QInD3jTkhIQEZGRmYPn26TgJwcXFBdHS0xjorV65sVmZqaoq///3v+Pvf/66TOIiIiAwRR9YREREZMCYCREREBoyJABERkQHTOhFoaf6+kGNERER0/9F6ieEZM2Y0G9Uv5BgRERHdf/hqgIiIyIAxESAiIjJgrSYCcrm8w53rog8iIiLSn1YTgVdffRW//vor6uvrte40JycHn3/+OXbu3Nmh4IiIiEi/Wh0s6Ofnh59//hkJCQkYOXIkRowYAQ8PD4jF4hbrFxUV4dy5czh06BD+/PNPSKVSTJ48WW+BExERUce1mgi89tprmDhxIjZu3AiZTAaZTAYjIyO4uLjA2toaEokE9fX1qKysREFBAcrLywEAlpaWmD17NiZNmoSePXt22hchIiIi7WmcPjhw4EB88MEHuH79OpKSknDhwgXk5OTg6tWravUsLS0xbNgw1T89emg9K5GIiIi6gKA7tqOjI55++mkAQF1dHW7evImKigqIxWJYWVnBxsZGr0ESERGRfmj9093ExASOjo5wdHTURzxERETUibiOABERkQFjIkBERGTAmAgQEREZMCYCREREBoyJABERkQFjIkBERGTAmAgQEREZMK3XEWhoaMCFCxeQl5eH2tpaTJs2DcCdnQZramrQq1cvGBkxvyAiIuoOtEoE0tLSsHr1apSVlanKmhKBnJwcfPjhh4iKisKoUaN0GiQRERHph+Cf7n/99Re++OILiEQizJkzB8HBwWrHPTw84ODggFOnTuk8SCIiItIPwYnA1q1bIRaLERsbi8cee6zFJYbd3NyQm5ur0wCJiIhIfwQnAhcvXsQjjzwCa2vrVutIpVK11wZERER0fxM8RqC2thaWlpYa69TV1UGhUGgVQF5eHuLj45GdnQ2JRILQ0FBMnz5d44DD4uJivPbaa83KR44ciTfeeEOr8xMRERkywYmAra0trl27prFOTk4OevfuLfjklZWViImJgYuLCxYtWoTCwkKsW7cOSqUSs2bNarP9s88+C09PT9XnthIVIiIiUic4EfD390diYiKysrLg5eXV7Hhqaiqys7MRGRkp+OSJiYmQy+VYuHAhzM3N4evri5qaGiQkJGDy5MkwNzfX2N7JyQkeHh6Cz0dERETqBI8RmDp1KiQSCT755BP85z//QV5eHgDg7Nmz+M9//oMvv/wS1tbWiIiIEHzytLQ0+Pn5qd3wg4ODIZfLkZGRocXXICIiovbQ6tXA+++/j6+++gq7du1SlcfFxQEAevfujbfeekurx/P5+fnw8fFRK5NKpTAxMUFBQUGb7VetWoXKykpYWVkhODgYs2fPhlgsFnx+IiIiQ6fVgkIDBgzA8uXLcfbsWWRnZ6OiogLm5uZwd3fHI488AmNjY61OXlVVBYlE0qxcIpGgsrKy1XY9e/bEhAkT4OfnBzMzM6Snp2PHjh0oKirCokWLWmwjk8kgk8kAALGxsZBKpVrFKsR1nfdI1DX0cX0QUdu64trTeolhIyMjBAYGIjAwUB/xCGJjY4O5c+eqPvv4+MDa2ho//vgjcnJy4Orq2qxNWFgYwsLCVJ9LS0s7I1SibonXB1HX0Ne15+Tk1OqxLt0UQCKRoLq6ull5VVUVLCwstOpr+PDhAIDLly/rJDYiIiJDIPiJwKFDhwR3GhISIqies7Mz8vPz1cpKS0tRV1enMXvRRCQStasdERGRIRKcCKxatUpwp0ITAX9/f+zcuRM1NTUwMzMDACQnJ0MsFsPb21vw+QDgxIkTAO6MYyAiIiJhBCcCCxYsaLG8uroaf/75J5KTkxEUFISAgADBJw8PD8fevXuxdOlSREZGori4GAkJCYiIiFCbUhgVFQVvb29VDJs3b0ZtbS08PT1hZmaGzMxM7Ny5E0FBQXjooYcEn5+IiMjQCU4ExowZo/H42LFjVRsSCWVhYYHFixdjzZo1iIuLg0QiwaRJkzBjxgy1egqFQm3pYmdnZ+zatQv79++HXC6HVCrF5MmT8cQTTwg+NxEREQEipVKp1FVnn3/+OWpqahAdHa2rLvVGyDoF2rr+9jyd90nUFRy/+LGrQ9Da8z8f7+oQiDrs33NG6KXfTps14OTkxFH7RERE3YhOE4GmZYeJiIioe9B6QaF7KRQK3LhxA/v370dqaiqGDBmii7iIiIioEwhOBGbOnNlmHQsLCzzzzDMdCoiIiIg6j+BEYNCgQS0u1iMSiSCRSDBw4ECMHTtWq02HiIiIqGsJTgQ++ugjPYZBREREXaFL9xogIiKirsVEgIiIyIC1+mpAm70F7iYSiVpdjpiIiIjuL60mAtrsNngvJgJERETdQ6uJwIoVKzozDiIiIuoCrSYC9vb2nRkHERERdQEOFiQiIjJg7VpiWKFQoLy8HA0NDS0el0qlHQqKiIiIOodWicDVq1exfv16pKeno76+vsU6IpEIGzdu1ElwREREpF+CE4G8vDx88MEHAABfX1+cOXMGDz30EKysrHDlyhVUVFTAx8eHTwOIiIi6EcGJwLZt29DY2IjPPvsM/fr1w8yZMxEUFIRp06ahtrYWP/30E1JTU/HKK6/oM14iIiLSIcGDBdPT0xEQEIB+/fqpypRKJQDA1NQUL730EiQSCTZt2qT7KImIiEgvBCcCFRUVcHR0/F9DIyPU1dWpPhsbG8PHxwfnz5/XbYRERESkN4ITAQsLC9TW1qo+W1paorS0VK1Ojx49UF1drbvoiIiISK8EJwK9e/dGcXGx6nP//v3xxx9/4Pbt2wCA2tpanD59Gg4ODrqPkoiIiPRC8GBBPz8/7NixA7W1tTA1NcX48eORmpqKRYsWwdPTE5cvX0ZJSQmee+45fcZLREREOiQ4ERg3bhycnJwgl8thamqKgIAAzJkzBwkJCTh58iTEYjEiIyPxt7/9TZ/xEhERkQ5pTAQWLVqEsLAwPProo7CxscHIkSPVjj/22GOYOHEiysvLYWVlBZFIpNdgiYiISLc0jhHIzc3FmjVrMH/+fHz33Xe4dOlS8w6MjGBtbc0kgIiIqBvS+EQgJiYGMpkMJ06cwIEDB3DgwAH069cP48aNw+jRo2Fubt5ZcRIREZEeaEwEPDw84OHhgRdeeAFHjhxBUlISrly5gp9++gnr16/H8OHDMW7cOHh5ebU7gLy8PMTHxyM7OxsSiQShoaGYPn06jIyETWhQKBR47733cPnyZbzzzjsYOnRou2MhIiIyNIIGC5qZmWH8+PEYP348cnJyIJPJcOzYMRw+fBiHDx+Gi4uL6imBhYWF4JNXVlYiJiYGLi4uWLRoEQoLC7Fu3ToolUrMmjVLUB9JSUm4ceOG4HMSERHR/wheR6CJq6sr5s2bh3/961945ZVX4Onpiby8PPz88894+eWX8e233wruKzExEXK5HAsXLoSvry/Gjx+PadOmYffu3YIWJqqsrMQvv/yC2bNna/s1iIiICO1IBJqIxWKEhITg448/xldffQUvLy/U19fj6NGjgvtIS0uDn5+f2liD4OBgyOVyZGRktNl+06ZN8PT0xODBg9v1HYiIiAyd4HUEWlJZWYlDhw4hKSkJeXl5AKDVAML8/Hz4+PiolUmlUpiYmKCgoEBj29zcXBw4cABLly7VPnAiIiIC0M5E4MKFC5DJZEhJSUFDQwMAwN3dHWFhYc3WGtCkqqoKEomkWblEIkFlZaXGtvHx8Zg4cSL69OmjtvQxERERCSc4ESgrK8OBAweQlJSkuvFKJBKEhYUhLCwMffv21VuQ9zp27BgKCgrwzjvvCG4jk8kgk8kAALGxsZBKpTqP67rOeyTqGvq4PoiobV1x7WlMBJRKJc6ePYv9+/cjNTUVCoUCAODl5YVx48Zh+PDhEIvF7T65RCJpcVBgVVVVq7MPGhoa8J///AeRkZFQKpWoqqpCTU0NAKCurg41NTUwMzNr1q4pYWly786JRPQ/vD6Iuoa+rj0nJ6dWj2lMBF555RXcvHkTwJ1tiEePHo2wsDA4OzvrJDBnZ2fk5+erlZWWlqKurq7VoOvq6nDjxg2sXbsWa9euVTv29ddfo3fv3lrNXCAiIjJkGhOBmzdvwtvbW/Xrv0ePDo0tbMbf3x87d+5U+xWfnJwMsVgMb2/vFtuYmpoiOjparaysrAzLly/H7NmzOYOAiIhICxrv7F9//TUcHR31dvLw8HDs3bsXS5cuRWRkJIqLi5GQkICIiAi12QdRUVHw9vbGggULYGxs3GymQdOYhX79+sHd3V1v8RIRET1oNK4joM8kALjzumHx4sVQKBSIi4vD5s2bMWnSJMyYMUOtnkKhUI1PICIiIt3R7bP+dnBxcWn2qP9eK1eu1HjcwcEBmzdv1mVYREREBqHdKwsSERFR98dEgIiIyIAxESAiIjJgTASIiIgMmOBE4OTJkxy5T0RE9IARPGvgyy+/hI2NDcaOHYtx48ZxLXIiIqIHgOAnAhMmTEBdXR22bduGqKgoxMbG4syZM1AqlfqMj4iIiPRI8BOBF198Ec888wySk5ORmJiI1NRUpKamwtbWFuPGjUNoaChsbW31GSsRERHpmFYLConFYowZMwZjxozB1atXIZPJcOTIESQkJGDr1q0ICAhAeHg4/P399RQuERER6VK7Vxbs16+f2lOCTZs24fTp0zh9+jSkUikmTJiA8ePHw9TUVJfxEhERkQ51aPpgbW0tDh8+jN9++021XbGrqysqKyuxfv16/OMf/0BOTo4u4iQiIiI9aNcTgStXriAxMRHHjh1DbW0txGIxQkNDMWHCBLi6uqK2thb79u3D5s2b8dNPP2HJkiW6jpuIiIh0QHAiUFdXh2PHjiExMRGXL18GADg7OyM8PBwhISFq2wabmpoiMjISN27cQFJSku6jJiIiIp0QnAjMnz8fNTU1MDIywrBhwzBhwgT4+PhobGNra4v6+voOB0lERET6ITgRMDMzQ0REBMLCwmBtbS2ozfjx4xEcHNze2IiIiEjPBCcCK1euhJGRdmMLzc3N1V4ZEBER0f1F8J1d2ySAiIiI7n+C7+5bt27F7NmzVdME73Xz5k3Mnj0b27dv11VsREREpGeCE4EzZ87A29u71WWEbW1tMXjwYKSkpOgsOCIiItIvwYlAYWEhXFxcNNZxdnZGYWFhh4MiIiKiziE4EZDL5TAxMdFYRywWo7a2tsNBERERUecQnAjY2dnh0qVLGutcunSJOxASERF1I4ITAT8/P2RkZCA5ObnF48eOHUNGRgZ3HiQiIupGBK8jMGXKFBw9ehTLly9HcnIy/P39YWtri5s3byI1NRWnT5+GhYUFpkyZosdwiYiISJcEJwK2trZ4//338eWXXyIlJaXZ7AB7e3u8+eabsLOz03mQREREpB9a7T7o5uaG5cuX48yZM7h06RKqqqogkUjg7u6OoUOHokePdm1mSERERF1E6zt3jx49MGzYMAwbNkwnAeTl5SE+Ph7Z2dmQSCQIDQ3F9OnTNa5keO3aNaxduxZXr15FRUUFrKys4Ofnh5kzZ8LGxkYncRERERmCLv0JX1lZiZiYGLi4uGDRokUoLCzEunXroFQqMWvWrFbbVVdXw8HBASEhIbCxsUFxcTG2bNmCy5cv47PPPoOxsXEnfgsiIqLuq9VE4NChQwCAoKAgmJmZqT4LERISIqheYmIi5HI5Fi5cCHNzc/j6+qKmpgYJCQmYPHlyqxsWeXp6wtPTU/XZx8cHdnZ2+OSTT5Cbm4sBAwYIjpWIiMiQtZoIrFq1CgDg7u4OMzMz1WchhCYCaWlp8PPzU7vhBwcHY/369cjIyEBgYKDgc1pYWAAAGhoaBLchIiIydK0mAgsWLAAA1Tv3ps+6lJ+fDx8fH7UyqVQKExMTFBQUtNleoVBAoVCguLgYGzZsgJubGwYOHKjzOImIiB5UrSYCY8aM0fhZF5pmHdxLIpGgsrKyzfafffYZzp07BwAYMGAA3n33XW6XTEREpIVuPd/vxRdfRGVlJa5fv45t27bhn//8J2JiYiAWi5vVlclkkMlkAIDY2FhIpVKdx3Nd5z0SdQ19XB9E1LauuPa6NBGQSCSorq5uVl5VVaV656+Jo6MjgDvjGAYNGoTXXnsNR48eRWhoaLO6YWFhCAsLU30uLS3tQOREDzZeH0RdQ1/XnpOTU6vHWk0EXnvttXadTCQS4dtvvxVU19nZGfn5+WplpaWlqKur0xh0S+zt7WFhYYHi4mKt2hERERmyVl+oK5XKdv2jUCgEn9zf3x/nzp1DTU2Nqiw5ORlisRje3t5afZGCggJUVFTAwcFBq3ZERESGrNUnAitXrtT7ycPDw7F3714sXboUkZGRKC4uRkJCAiIiItSmFEZFRcHb21s1c2Ht2rUwNjaGu7s7zM3NkZ+fj507d6J3794YOXKk3uMmIiJ6UHTpGAELCwssXrwYa9asQVxcHCQSCSZNmoQZM2ao1WuaJtjEzc0Nv/32G2QyGerr6yGVSjFs2DBMmTIFpqamnf01iIiIui2RUqlUtqdhTU0NqqqqYG5u3uoKgPczIesUaOv62/N03idRV3D84seuDkFrz/98vKtDIOqwf88ZoZd+2zVYsCWNjY3YtWsX9u/frzYoz8HBAePGjcPjjz/Odf6JiIi6EcGJQENDAz799FNkZGRAJBJBKpXC2toaZWVlKCkpwS+//IK0tDR88MEH3I6YiIiomxB8x969ezcyMjIQEBCA5557TjWHHwAKCwuxdu1anDlzBrt378aUKVP0ESsRERHpmOD1eI8ePYq+ffvi7bffVksCAKBPnz5466230LdvXxw5ckTnQRIREZF+CE4ECgsL4e/v3+pa/kZGRvD390dRUZHOgiMiIiL9EpwI9OjRA7W1tRrr1NXVcbAgERFRNyI4EXjooYdw8uRJlJeXt3i8vLwcJ06cgKurq65iIyIiIj0TnAhMmDAB5eXlePfdd5GUlISioiLI5XIUFxfjwIEDeP/991FeXo4JEyboM14iIiLSIcGzBkaOHImcnBzs2LED//rXv1qsM3nyZC7xS0RE1I1oNeH/qaeeQmBgIJKSkpCTk4Pq6mqYm5vD1dUVoaGh8PDw0FecREREpAeCE4GKigqIRCJ4eHjwhk9ERPSAaDMRSElJwdq1a1VLCvfp0wfPPvssAgMD9R4cERER6ZfGwYLZ2dlYtmyZ2r4ChYWFWLZsGbKzs/UeHBEREemXxkRg9+7dUCqVePLJJ/HDDz/g+++/xxNPPAGFQoHdu3d3VoxERESkJxpfDVy6dAleXl6YMWOGqmzmzJnIyMjgEwEiIqIHgMYnArdv34a7u3uzcnd391YXFiIiIqLuQ2Mi0NjYCFNT02blJiYmaGxs1FtQRERE1DkEryxIRERED542pw8ePHgQ6enpamUlJSUAgCVLljSrLxKJsHjxYh2FR0RERPrUZiJQUlKiuvHfKyMjQ+cBERERUefRmAhER0d3VhxERETUBTQmAt7e3p0VBxEREXUBDhYkIiIyYEwEiIiIDBgTASIiIgPGRICIiMiAtTl9UN/y8vIQHx+P7OxsSCQShIaGYvr06TAyaj1H+fPPP/H7778jMzMTt27dgp2dHUaNGoXIyEiIxeJOjJ6IiKh769JEoLKyEjExMXBxccGiRYtQWFiIdevWQalUYtasWa22S05ORlFRESIjI+Ho6Ijc3Fxs2rQJubm5eOuttzrxGxAREXVvXZoIJCYmQi6XY+HChTA3N4evry9qamqQkJCAyZMnw9zcvMV2U6ZMgaWlpeqzj48PxGIxvv/+e5SUlMDe3r6zvgIREVG31qVjBNLS0uDn56d2ww8ODoZcLte4auHdSUATV1dXAMCtW7d0HicREdGDqtUnAlu2bGl3p9OmTRNULz8/Hz4+PmplUqkUJiYmKCgo0Oqc2dnZEIlE6N27t1btiIiIDFmriUBCQkK7OxWaCFRVVUEikTQrl0gkqKysFHy+srIybNu2DaNHj4aVlZXgdkRERIau1USgpX0Gdu/ejdTUVDz66KPw9vaGtbU1ysrKkJ6ejqNHjyIgIACTJk3Sa8D3amhowFdffQVTU1PMmTOn1XoymQwymQwAEBsbC6lUqvNYruu8R6KuoY/rg4ja1hXXXquJwL37DBw6dAh//PEHPv30UwwYMEDt2JgxYzBx4kRER0dj2LBhgk8ukUhQXV3drLyqqgoWFhZttlcqlVixYgWuXbuGmJgYjW3CwsIQFham+lxaWio4TiJDw+uDqGvo69pzcnJq9ZjgwYJ79uzBiBEjmiUBTdzc3DBixAjs2bNHcGDOzs7Iz89XKystLUVdXZ3GoJv8+9//RkpKChYtWgRnZ2fB5yUiIqI7BCcCBQUFsLGx0VjHxsZGq0F+/v7+OHfuHGpqalRlycnJEIvFbe58+N///he//fYboqKi4OXlJficRERE9D+CEwEzMzNcvHhRY52LFy/C1NRU8MnDw8PRs2dPLF26FOfPn4dMJkNCQgIiIiLUphRGRUVh9erVqs9Hjx7FL7/8gpCQENja2iI7O1v1T3l5ueDzExERGTrBCwoFBATg4MGDWLt2LaZPnw4zMzPVsaZFgLKysjB27FjBJ7ewsMDixYuxZs0axMXFQSKRYNKkSZgxY4ZaPYVCAYVCofp87tw5AMDBgwdx8OBBtbqvvPIKxowZIzgGIiIiQyZSKpVKIRVv376NDz74AMXFxTAzM4OrqyusrKxw+/Zt5OTkoKamBg4ODvjkk0+6xRQ+bdcpEOL62/N03idRV3D84seuDkFrz/98vKtDIOqwf88ZoZd+NY27E/xEwMrKCv/85z+xYcMGHD16FJmZmapjYrEY48aNw+zZs9GrV6+ORUtERESdRqu9Bnr16oX58+dj3rx5yM/PR3V1NczNzeHs7AxjY2N9xUhERER60q5Nh4yNjdGvXz9dx0JERESdTOtEoKGhARcuXEBeXh5qa2tVywnL5XLU1NSgV69eMDLq0r2MiIiISCCtEoG0tDSsXr0aZWVlqrKmRCAnJwcffvghoqKiMGrUKJ0GSURERPoh+Kf7X3/9hS+++AIikQhz5sxBcHCw2nEPDw84ODjg1KlTOg+SiIiI9ENwIrB161aIxWLExsbiscceg6OjY7M6bm5uyM3N1WmAREREpD+CE4GLFy/ikUcegbW1dat1pFKp2msDIiIiur8JTgRqa2thaWmpsU5dXZ3aCoBERER0fxOcCNja2uLatWsa6+Tk5KB3794dDoqIiIg6h+BEoGmnwKysrBaPp6amIjs7GwEBAToLjoiIiPRL8PTBqVOnIjk5GZ988gkmTpyIkpISAMDZs2eRkZGBffv2wdraGhEREXoLloiIiHRLcCJga2uL999/H1999RV27dqlKo+LiwMA9O7dG2+99Vab4wiIiIjo/qHVgkIDBgzA8uXLcfbsWWRnZ6OiogLm5uZwd3fHI488wv0GiIiIuhmtlxg2MjJCYGAgAgMD9REPERERdSLBgwWXLFmCQ4cOaaxz+PBhLFmypMNBERERUecQnAhkZGSoBgi2prS0FBkZGR0OioiIiDqHTrcJlMvlHCdARETUjWg9RqAlSqUSpaWlSE1NhZ2dnS66JCIiok6gMRGYOXOm2ueEhAQkJCRo7HDq1Kkdj4qIiIg6hcZEYNCgQRCJRADujBGQSqVwcHBoVs/IyAgWFhZ4+OGHERoaqp9IiYiISOc0JgIfffSR6s8zZ87E2LFjMW3aNH3HRERERJ1E8BiBFStWQCKR6DMWIiIi6mSCEwF7e3t9xkFERERdQOtZA7du3cIff/yBmzdvoqGhocU6fH1ARETUPWiVCGzevBnbt29HY2OjxnpMBIiIiLoHwYnAkSNHsHXrVgwePBgTJkzAsmXLEBISAj8/P6Snp+PAgQMYPnw4wsPDtQogLy8P8fHxyM7OhkQiQWhoKKZPnw4jo9bXOmpoaMAvv/yCS5cu4a+//kJ9fT02b96s1XmJiIhIi5UFf//9d9ja2uK9995DUFAQAMDBwQHBwcF46aWX8M477+D48eOorq4WfPLKykrExMRAJBJh0aJFePLJJ7F79+42b+p1dXVISkqCiYkJPD09BZ+PiIiI1Al+InD16lUEBwerLSGsUChUf/b394efnx927doleGfCxMREyOVyLFy4EObm5vD19UVNTQ0SEhIwefJkmJubt9hOIpEgPj4eIpEIv/32Gy5cuCD0axAREdFdBD8RaGxsRK9evVSfxWJxs1//ffv2RU5OjuCTp6Wlwc/PT+2GHxwcDLlc3ubmRU0LHREREVH7CU4EbGxscOvWLdVnqVSK3NxctTq3bt3SatOh/Px8ODk5qZVJpVKYmJigoKBAcD9ERETUPoITAVdXV1y7dk312cfHB1lZWTh8+DBqa2tx9uxZnDhxAv379xd88qqqqhYXKZJIJKisrBTcDxEREbWP4DECQ4cOxY8//oji4mI4ODhgypQpOH78OFauXImVK1fe6axHj2YbFd0vZDIZZDIZACA2NhZSqVTn57iu8x6JuoY+rg8ialtXXHuCE4ExY8ZgzJgxqs9SqRSfffYZdu3ahaKiItjb22PChAno16+f4JNLJJIWZxlUVVXBwsJCcD9ChIWFISwsTPW5tLRUp/0TPUh4fRB1DX1de/e+hr+b1isL3s3BwQFz585td3tnZ2fk5+erlZWWlqKurk5j0ERERKQbgscI6IO/vz/OnTuHmpoaVVlycjLEYjG8vb27MDIiIiLDoPUTAYVCgZs3b2rca0DoTTw8PBx79+7F0qVLERkZieLiYiQkJCAiIkJtSmFUVBS8vb2xYMECVVlqairq6upU0xVPnDgBAHBzc+MGSURERAJplQjs3LkTu3btQnl5ucZ6mzZtEtSfhYUFFi9ejDVr1iAuLg4SiQSTJk3CjBkz1OopFAq1xYsA4Mcff0RJSYnq85dffgkAeOWVV9TGMhAREVHrBCcCmzdvxtatW2FhYYGQkBDY2tpqtWZAa1xcXBAdHa2xTtOshLbKiIiISDuCE4EDBw7AwcEBcXFxrS79S0RERN2L4MGCFRUVCAwMZBJARET0ABGcCPTp0wdVVVX6jIWIiIg6meBEYPz48Thz5gzKysr0GA4RERF1JsFjBMaPH4/r16/jww8/xJNPPokBAwa0+pqAy5MSERF1D1pNH3zooYdw8OBBrF69utU6IpEIGzdu7HBgREREpH+CE4H9+/fj+++/h7GxMXx8fGBjY6OT6YNERETUdQQnArt27YKVlRU++eQTODg46DMmIiIi6iSCBwuWlJRg+PDhTAKIiIgeIIITAVtb21b3FiAiIqLuSXAiEBISgtTUVLWdAomIiKh7E5wITJ06FQMHDkRMTAzS09OZEBARET0ABA8WfOqpp1R//vjjj1utx+mDRERE3YfgRGDQoEEQiUT6jIWIiIg6meBE4KOPPtJjGERERNQVBI8RICIiogcPEwEiIiID1uqrgS1btgAAJk6cCAsLC9VnIaZNm9bxyIiIiEjvWk0EEhISAAAjR46EhYWF6rMQTASIiIi6h1YTgejoaAD/21K46TMRERE9OFpNBLy9vTV+JiIiou5P8GDBQ4cOITc3V2Odq1ev4tChQx0OioiIiDqH4ERg1apVSElJ0Vjn9OnTWLVqVYeDIiIios6h0+mDCoWCqw8SERF1IzpNBAoKCiCRSHTZJREREemRxiWG733Mn5KSguLi4mb1FAoFbty4gczMTAQEBOg2QiIiItIbjYnAvQP/cnJykJOT02p9d3d3zJkzRyeBERERkf5pTARWrFgBAFAqlYiKisJjjz2Gxx57rFk9IyMjSCQSmJqaah1AXl4e4uPjkZ2dDYlEgtDQUEyfPh1GRprfWlRXV+Pf//43UlJSoFAoMHToULzwwgvo1auX1jEQEREZKo2JgL29verP06ZNg4+Pj1pZR1VWViImJgYuLi5YtGgRCgsLsW7dOiiVSsyaNUtj26+++goFBQWYP38+jIyMsH79enzxxRf4+OOPdRYfERHRg07wNsTTp0/X+ckTExMhl8uxcOFCmJubw9fXFzU1NUhISMDkyZNhbm7eYrvs7GycO3cOH330kWqhI1tbW7z33ns4f/48fH19dR4rERHRg0jwrIErV65g3759qK6uVpXV1tZixYoVeP755zF//nz8+uuvWp08LS0Nfn5+ajf84OBgyOVyZGRktNouNTUVVlZWaqsdDhw4EA4ODkhLS9MqBiIiIkMmOBHYsWMHtm3bpnbT3rBhA44cOQKlUomKigr8/PPPOHfunOCT5+fnw8nJSa1MKpXCxMQEBQUFGts5Ozs3K3d2dkZ+fr7g8xMRERk6wa8G/vrrL/j4+Kg+NzQ04NChQxg4cCCio6NRWVmJd955B3v37oWfn5+gPquqqlpcd0AikaCyslJju5ZeG0gkkhanNwKATCaDTCYDAMTGxjZLQHTBab12T0SISHd+f/fJrg6BqFsS/ESgvLwcdnZ2qs+XL19GbW0twsLCIBaLYWtri8DAwDb3I+gqYWFhiI2NRWxsbFeHQh30f//3f10dApFB4rX3YNJqZcHGxkbVn7OysgCo70poaWmJ8vJywf1JJBK1MQdNqqqqYGFhobFdTU1Ni+24siEREZFwghMBqVSKS5cuqT6npKTAzs4OvXv3VpXdunVL4w38Xi290y8tLUVdXZ3GR/etjQUoKChocewAERERtUxwIjBixAhkZ2dj2bJl+Oabb5CdnY3hw4er1cnPz1dLDNri7++Pc+fOqf26T05OhlgsVnvScK8hQ4agrKxM9VQCuDOGoaioCP7+/oLPT91TWFhYV4dAZJB47T2YBCcCERER8PDwwKlTp3Ds2DG4urpi2rRpquPFxcX4888/Nd7A7xUeHo6ePXti6dKlOH/+PGQyGRISEhAREaE2GDAqKgqrV69Wffbw8ICfnx9WrFiBkydP4tSpU/jmm2/g5eXFNQQMAP9nRNQ1eO09mERKpVKpTYOrV68CAFxcXNSWAS4uLkZubi7c3Nxga2sruL+8vDysWbNGbYnhGTNmqPX96quvwtvbG6+++qqqrKqqCj///DNOnToFpVKJgIAAvPDCC7C0tNTm6xARERk0rRMBIiIienBoXEcgIyMDDg4OkEqlgjrLzc1FTk4OQkJCdBIcaWfGjBkA7mwW5eDgoCp/9dVXUVJSolbXxMQEffr0QVBQEB5//HHVhlF79+7FTz/9hOHDh+PNN99sdo6CggK88cYbAIC5c+diwoQJzerIZDJ8//33GDJkCN59910AwObNm7FlyxaEhISoPdkhojuart+7mZmZqa7TiIgImJiYdEFkbTt48CBWrVqFadOmtfg96P6mcYzAkiVLcPDgQbWy7du348UXX2yx/qlTp7Bq1SqdBUe65efnh5CQEISEhMDDwwOFhYVISEjA+++/j6qqKgDAoEGDAAAXL15ssY+7B2je/ee7ZWZmqvVFRMINGzYMISEhGD16NAYMGIC8vDxs2rQJH3zwAWpra7s6PHoACV5ZsEl9fb3qpkHdy5QpU9RWhywuLsaSJUtw7do1bNu2Dc8++yz69esHc3Nz3Lp1C4WFhejTp49aH5mZmRCJROjbt2+riUBTORMBIu09++yzak/0rl+/jsWLFyM3Nxe//vornnjiiS6Mjh5EWi0oRA8WBwcH1WO8lJQUAICRkRE8PT0BtPyLPysrCy4uLggICMCNGzeaLel848YNlJSUQCwWw83NTc/fgOjB5+joiEmTJgEAN1UjvdD6iQA9WPr37w/gzkJOTQYNGoTU1FRkZmZizJgxqvJbt26hqKgIYWFh8PLyAnAnMbj710vTawF3d3f06NGxv16XLl3Crl27kJWVhYqKClhZWcHf3x/Tpk1rNm6lsrISR48exZkzZ1BQUICysjKIxWK4urpiwoQJzda8AICVK1fi0KFDiI6ORn19PXbs2IErV66guroaP/30E1JSUlTvPUNDQ/HLL7+o1r1wdHTE448/3up4mNLSUuzYsQNpaWm4efMmxGIxPD09MXXqVFWi1SQ9PR1LlixBSEgInn76aWzcuBFpaWkoKyvDs88+q7oJkOHq27cvAOD27dvNjp05cwYpKSm4ePEibt68icbGRtjb2yMoKAiRkZHN9mW5++/bnDlzsHHjRqSkpKCiogL29vYICwtDREQERCJRs3NlZWUhISEBly5dgpGREdzd3dscE1BZWYnt27cjJSUFpaWlqh8JERERLa77MmPGDNjb2+Obb77Bzp07cfDgQZSWlsLW1hYTJkxAREQEACAnJwebN29GVlYW6uvr4enpiTlz5qj+W5FwTAQMXNNiTj179lSV3X2Tv9vd7/49PDwgEomQmZmJ0aNHq+ro6rXAvn37EB8fDwBwc3ODl5cXrl+/jqSkJJw+fRofffQRXFxcVPUvXryI+Ph42Nvbo0+fPnB3d8etW7eQlZWF9PR0zJw5E08+2fKmNEePHkVSUhIGDBgAf39/FBUVqf1PsLS0FO+++y7EYjEGDx6MsrIyZGZmYuXKlVAoFBg7dqxaf9nZ2fjss89QVVUFJycnDBkyBBUVFTh37hzS0tLw+uuvY+TIkc3iKC8vx7vvvovGxkZ4eXlBLpfft4PDqHM1XadWVlbNjq1atQr19fXo168f+vbti7q6Oly+fBn//e9/cfbsWcTExKgGA9+tqqoKH3zwAaqqqjBo0CBUVVUhKysL69atQ01NTbMb/JkzZ7B06VI0Njaqtn2/evUqoqOj1X4w3O3mzZuIjo5GUVER7Ozs8Mgjj6C8vBwXLlzA+fPn8cwzz2Dy5Mkttl2+fDnS0tLg4+MDR0dHpKenY+3ataitrcXgwYPx6aefok+fPvD19cW1a9dw/vx5fPTRR/jyyy9b/O9ErWMiYODOnDkDAOjXr5+qbODAgejZsyeuX7+OsrIyWFtbA/jfTd7LywsWFhZwcXFplizcXae9srOz8dNPP8Ha2hpvv/02Bg4cqDqWlJSE7777DqtXr8ann36qKnd2dsYnn3wCDw8Ptb4KCwvx8ccfIyEhAaNHj4a9vX2z8+3fvx9vvPFGizdn4M6I6IkTJ+L5559XrW9x4sQJfPnll9iyZYtaIlBdXY1ly5ahuroar732mlqS9Ndff+GTTz7Bd999h8GDBzdb8yI1NRVBQUF4/fXXIRaLtfgvRg+6plcCLf2Cnj9/Pvz8/NSSRrlcjvj4eCQlJWHPnj0tJsGnT59u9vft0qVL+PDDD7Fr1y5MnjxZlUDU1NRg9erVaGxsxIIFC1R/55VKJTZs2IAdO3a0GPf333+PoqIijBo1Cq+88orqKWFWVhY+/fRTrF+/HoMHD8aAAQPU2jW9Xvzmm29U//+5du0a3nnnHezYsQMHDhzA7Nmz8dhjjwEAFAoFVqxYgaNHj2Lfvn2cuaAljhEwUDdv3sSuXbuwe/duAMD48eNVx3r06KG6+d47S8DOzk51M/X09ER+fr5qo6mqqipcu3YNxsbGzW7I2ti+fTsUCgX+/ve/qyUBABAaGorAwEBcunQJV65cUZX36dOnxXP26dMHTzzxBBQKBU6fPt3i+QICAlpNAgDA3t4ezz77rNoiV8OHD0ffvn1RUlKiNjXzwIEDuHXrFh577DG1JAC482TjySefRG1tLQ4fPtzsPD179sSLL77IJIAA3LnJlpaWYvPmzThy5Ag8PDxUN767BQUFNXtyJBaL8eKLL8LY2BinTp1qsX8zMzO89NJLan/f3N3d4e/vr3qq0OTEiRMoLy/HoEGD1BJfkUiEmTNnqu1M26SoqAhnz56FqakpXnzxRbVXhV5eXhg/fjyUSiV+++23FuN74YUXVEkAcOf1yJAhQ1BXVwdbW1u1/xZGRkaqJwtNTy5JuDafCCQkJCAhIaFZ+cyZM/USEOnPkiVLmpWJRCJMnToVjz76qFr5oEGDkJmZiczMTAwfPhzV1dXIzc3FiBEjVHW8vLwgk8mQlZWFoKAgXLx4EUqlEv3792/xUaQQCoUCFy5cgImJSav7Rnh5eeH06dP4888/VWMcmtqmp6fj4sWLuHXrFurr66FUKlFWVgbgzujrlgQGBmqMycfHR+3VSRNHR0dcu3YNt27dUiVH58+fB3BnClhLml6Z/Pnnn82O9e/fX6tVOenB9NprrzUrCwgIwFtvvdXquJvCwkKkpqbi+vXrqK2tRdM6cT169EBhYWGLbQYMGNDiSqxOTk44e/Ysbt68qSprurkGBwc3q9+jRw8MGzYMv/76q1p5048If3//FjejCwkJwa5du1q8cRsbG2Pw4MHNypv2svHz82t2rGmG091xkzB8NWBA/Pz8YG1tDZFIBLFYjD59+iAwMLDZFEHgfzespos5OzsbSqVS7ZF/06C3zMxMBAUF6WT9gIqKCtVc6aeeeqrNuk1u3ryJuLg4tacE92ptDnZbC2a19GsHuPOLCrgzpbZJ0yyKxYsXa+zz7tiFxkGGYdiwYTA1NUVDQwMKCgpw5coVnD17Fv/9738xffr0ZvX/85//YNeuXdB2kdjW/l43JfENDQ2qslu3bgFo/e/o3QOG723T0uu4u9u0dOO2trZWewJ3b2wtJcwtxU3CaEwENm3a1FlxUCe4dx0BTTw8PGBkZITc3FzU1NS0eJPv3bs3bGxsVMmCLgYKNv3PzNTUtNVf1U3uHiz43Xff4cqVK3jkkUcQGRkJJycnmJubw8jICOfOncOnn37a6v8oW/q1f7eWRk+3Ff/w4cM1DvRrabvstuIgw3DvOgJHjx7Ft99+iy1btmDIkCFqr8uSk5Oxc+dO2NraYs6cOfDw8ICVlZXqycH8+fNVN+R7afP3urO1lARoc5y0wycC1CIzMzO4urri8uXLuHjxIrKysiCRSJpNzfH09ERKSgrKy8vx119/QSQSdWigYK9evdCzZ0+IRCK88sorgv5nVVtbi3PnzsHKygoLFy5s9j+J1h6N6oOdnR0KCgowZcqUZgOgiNpj1KhRyMjIgEwmw/r16xEdHa06dvLkSQDA3//+dwwdOlStXW1treq1WEfZ2NgAUJ9mfLd7lzC/u01Lx4D/PT3j67Cux7SKWtX0y/6PP/7An3/+qXpKcDdPT080NjZi7969aGhogIuLS4vvA4UyNjaGj48Pampq8McffwhqU11dDaVSCRsbmxZ/KSQnJ7c7Hm09/PDDANDqAC2i9pg+fTp69uyJ9PR0tQG8Tau8tvSYPzk5WevXBa1pSu6PHz/e7FhjY6MqIWmpTVpaGiorK5sdbxowyxVIux4TAWpV0wWalJSkWrDjXk0Xe9PIX11c1E888QREIhFWr16NCxcuNDteW1uLpKQkyOVyAHfeJ0okEly7dk1t4JFSqcR///vfTh1FHB4eDisrK+zYsQOJiYlQKBRqxxsbG5GWlqbazptICBsbG4SHhwMAtm3bpip3cnICAPz+++9qN/2cnBxs2LBBZ+cfMWIEevXqhfT0dLX9Z5RKJTZv3tzik4LevXsjICAAtbW1iI+PV3t3n52djX379kEkEmHixIk6i5Pah68GqFVNN/mmXx0tPfJ3dXWFiYlJs02LWnP27Fm8//77rR7/9NNP4eXlhblz5yI+Ph4ff/wx+vbtC0dHR/To0QMlJSXIyclBfX09hg0bBrFYDCMjI0RGRmLDhg1YsmQJfHx80KtXL1y5cgVFRUWIiIhQTZPUN4lEgrfffhtxcXH44YcfsG3bNvTt2xcWFhYoKyvDlStXUFVVhbfeektt7QaitkyZMgUymQxpaWm4fPkyBgwYgL/97W84dOgQZDIZMjIy4Orqitu3b6tm+1y6dKnVR/PaMDMzw8svv4xly5Zh1apVSExMVC0oVFBQgHHjxmH//v3N2r300ktYvHgxjh49iqysLHh4eKCiogLp6elQKBR4+umn+QrtPsBEgFplaWkJZ2dn5Ofnq60tcDdjY2MMHDgQ6enpANpOBCoqKlocMX+v8ePHw8PDA3v27EFGRgbOnj0LExMT2NraYtSoURg2bJja0qlTpkyBnZ0d9uzZg+zsbFW8CxYsQGNjY6clAsCdgZbLli3D7t27VUs1A3eeXAwaNAhBQUHw9fXttHjowWBtbY3x48dj9+7d2LZtG9566y04Ojris88+w4YNG5CdnY3Tp0+jd+/eePrppzFp0iRERUXp7PyPPPIIoqOjsXnzZvz555/Iy8vDwIEDMX/+fBQUFLSYCNja2uKzzz7D9u3bcerUKZw6dQomJibw8fFBREQEhgwZorP4qP1ESl29RCIiIqJuh2MEiIiIDBgTASIiIgPGRICIiMiAMREgIiIyYEwEiIiIDBgTASIiIgPGRICIiMiAMREgIiIyYEwEiIiIDBgTASIiIgPGRICIiMiAMREgIiIyYEwEiIiIDBgTASIiIgPGRICIiMiAMREgIiIyYEwEiIiIDBgTASJqN5FIpPaPiYkJ7O3tERAQgHnz5mHv3r1obGxsse3zzz/frL25uTm8vb2xcOFClJSUdPK3ITJMIqVSqezqIIioexKJRACA6OhoAEBjYyPKysqQnp6OY8eOQS6XIzAwEOvXr4eHh4da2+effx4///wzIiMj4e/vDwAoKirCr7/+iqtXr+Khhx7CmTNnYGdn16nficjQ9OjqAIio+/voo4+alRUVFSEqKgoJCQkICwvD6dOn4eDg0KzelClT8Pzzz6s+19bWYvjw4Th37hxWrFihSjKISD/4aoCI9KJ3797YuHEjxowZg2vXruGf//ynoHampqZ4+umnAQApKSn6DJGIoMUTgYKCAvzxxx/IzMxEaWkpKioqIBaLYWlpCVdXV/j4+GDw4MEQi8X6jJeIuhEjIyN88MEHOHjwIH755Rd89dVXqtcJQvTs2VOP0RERICAROHbsGH7//XdkZWW1WufChQvYvXs3JBIJxowZg4kTJ7b4CJCIDM+oUaPQo0cPFBcXIycnB/3799dYv6amBuvWrVO1JSL9ajURuHDhAtauXYvc3FyYm5sjJCQEXl5ecHNzg7W1NSwsLCCXy1FRUYGCggJkZ2fj/Pnz2LNnD/bt24e//e1veOKJJ2Bubt6Z34eI7jMmJiaws7NDUVERSkpKmiUC27dvR05ODgCguLgYu3fvxrVr1zB69GgsWLCgCyImMiytJgIxMTHo378/3njjDQQGBrb4iM7MzAxmZmZwcHCAv78/ZsyYgevXryMxMRH79u2Dqakppk2bptcvQET3v6bJSS29FtixYwd27NihVhYeHo49e/bw1QBRJ2h1sODChQsRGxuLESNGaHUxOjo64rnnnsO3334LX19fnQRJRN1XbW0tbt68CQCwt7dvdvynn36CUqlEQ0MDsrOzMXPmTCQmJvJpAFEnaTURCAoK6lDH1tbWzeYNE5HhOXr0KBoaGtC7d2+4urq2Ws/Y2Bju7u7YsGEDhg0bhjVr1mDnzp2dFyiRgeL0QSLSG4VCgU8//RQA8NRTTwlqY2RkhOXLlwMA3nnnnVZXJiQi3WAiQER6UVxcjFmzZuHgwYPo168f3nvvPcFthw0bhoiICGRlZWHt2rV6jJKINE4ffO2117TuUCQS4dtvv213QETU/TStLKhQKFRLDB89ehRyuRxBQUFYv349pFKpVn1+/PHH2LNnD5YsWYKnn36aa5QQ6YnGRICbfhCREEuWLAEAiMVi9OrVCw899BCee+45PPnkkxg/fjyMjLR/+DhkyBBMnToV27Ztw7/+9S9ERUXpOmwiQhubDrU3EWhpZDARERHdf7j7IBERkQHjYEEiIiIDpnGMgEKhwNdffw2RSISoqCj06NFy9YaGBnz77bcQiUR444039BEnERER6YHGJwInT57EyZMnERgY2GoSAAA9evTAI488guPHj+PEiRM6D5KIiIj0Q2MicPz4cdja2graASw4OBi2trY4evSozoIjIiIi/dL4auCvv/6Cj4+PoP3DRSIRBg8ejPT0dJ0Fp08FBQVdHQIREVGncHJyavWYxicCZWVlsLOzE3wiW1tb3L59W3hkRERE1KU0JgI9evRAfX294M7q6+s1jiUgIiKi+4vGRMDGxga5ubmCO8vNzYWNjU2HgyIiIqLOoTER8PT0REZGBgoLC9vsqLCwEBkZGfDy8tJZcERERKRfGhOB8PBwKBQKfPnllxrf/ZeXl+Orr76CQqFAWFiYzoMkIiIi/dD4Qn/gwIEICwuDTCbDm2++ifDwcAwePBi2trYAgJs3b+LChQuQyWSoqKhAeHg4Bg4c2CmBExERUce1uddAY2MjfvzxRyQlJWnsaNy4cZg3b167dhnrCpw+SEREhkLT9EHBmw5dvHgRiYmJuHjxIsrKygAA1tbW8PLyQlhYGDw9PXUSbGdhIkBERIZCJ4nAg4aJABERGYp2LyhEREREDzYmAkRERAaMiQAREZEB43rA1K3ExcWhtLQUUqkU77zzTleHQ0TU7TERoG6ltLRU0EqXREQkDF8NEBERGTAmAkRERAasQ68GiouLkZeXBwBwcXGBg4ODToIiIiKiztGuRKCmpgbfffcdTpw4oVY+YsQIvPzyyzA1NdVJcERERKRf7UoE1qxZg/Pnz2PGjBkYMGAA6uvrcfr0aRw6dAgmJiZYsGCBruMkIiIiPdCYCNTV1cHExKRZeUpKCubNm4dHH31UVRYUFIS6ujqcOnXKYBOB62/P6+oQHngNJfL/9+8i/vfWI8cvfuzqEIiok2gcLPjWW2/hwoULzcobGxthZmbWrNzMzAwKhUJ30REREZFeaXwi4O7ujpiYGIwbNw7PPvus6uY/ePBgrFmzBrW1tejfvz/q6+tx5swZHDp0CEOHDu2UwImIiKjjNCYCr7/+OkaNGoUffvgBqampeOmllzBkyBDMmzcPX3zxBb799lu1+gMGDMCLL76o14CJiIhId9ocLBgQEIBly5Zh7dq1iI2NxaOPPornn38ecXFxOH/+PPLz8wHcmT748MMP6z1gIiIi0h1BswbMzc3x8ssvY+TIkfj++++xcOFCzJ07F0FBQfD19dV3jEQqNsYAIPp//yYioo4SKZVKpTYNamtrsX79evz+++8YPnw45s6dC0tLS33FpzcFBQU675Oj2OlBwVkDRA8WJyenVo8JSgTKy8tVO7413fQzMzPx3XffobKyEi+88AJGjRrVruDy8vIQHx+P7OxsSCQShIaGYvr06TAyan1Cw+bNm7Fly5YWj82ePRtTp05t87xMBIhax0SA6MGiKRHQ+GqgtrYWq1evVltBcNiwYXjllVcwaNAgfPHFF9i4cSNWrlyJ5ORkvPTSS7C2thYcWGVlJWJiYuDi4oJFixahsLAQ69atg1KpxKxZs1ptN27cOPj7+6uVpaSkYMeOHRgyZIjg8xMRERk6jYnAhg0bcOLECYSEhGDgwIH466+/cPDgQVhZWWHu3LkQi8V47rnnMHLkSKxatQr/+Mc/8Nxzz2Hs2LGCTp6YmAi5XI6FCxfC3Nwcvr6+qKmpQUJCAiZPngxzc/MW29nZ2cHOzk6tbOvWrXB2doarq6uwb05ERESaFxRKSUlRPQEYP348FixYgGHDhuH06dNq9QYOHIjPP/8cEyZMwA8//CD45GlpafDz81O74QcHB0MulyMjI0NwPxUVFTh//jyCg4MFtyEiIiIBSwzf+8vbzs6uxdUGe/TogVmzZmH48OGCT56fnw8fHx+1MqlUChMTE63e4Z88eRKNjY1MBIiIiLSk8YmAu7s7Dh8+jKysLDQ0NCA7OxtHjhyBu7t7q220eTRfVVUFiUTSrFwikaCyslJwP8eOHUP//v3h6OgouA0RERG18UTghRdewJIlSxAdHa0qs7W1xfPPP6/vuAS7desWMjIy8PTTT2usJ5PJIJPJAACxsbGQSqU6j+W6znsk6hr6uD6I6P6kMRHo06cPvv76a5w5c0Y1fTAgIACmpqY6OblEIkF1dXWz8qqqKlhYWAjq4/jx4wCAkSNHaqwXFhaGsLAw1efS0lItIiUyLLw+iB4s7Z4+CAAmJiZt3mTby9nZWbVEcZPS0lLU1dVpDPpux44dg5eXF3/BEBERtYPGMQL65u/vj3PnzqGmpkZVlpycDLFYDG9v7zbbFxcX49KlSxwkSERE1E6C9hpoyenTp5GZmYm6ujo4ODhg5MiRWv8qDw8Px969e7F06VJERkaiuLgYCQkJiIiIUJtSGBUVBW9vbyxYsECtfXJyMoyNjbWaqUBERET/0+aCQr6+vhg8eLCqrKqqCp9//jmysrLU6m7atAnz58/H6NGjBZ/cwsICixcvxpo1axAXFweJRIJJkyZhxowZavUUCgUUCkWz9seOHcPgwYO75V4HRERE9wONew3MnDkT06dPx7Rp01RlX375JU6ePAkHBwcEBwfD0tIS2dnZOH78OIyNjREbG4t+/fp1SvAdwb0GiFrHvQaIHiwdGix4t8LCQpw8eRL9+/dHdHQ0zMzMAACPPfYYAgICsHLlSvz66694+eWXOxYxERERdQqtBgtmZmYCuLPDX1MS0GT06NEYOHCgVksDExERUdfSKhEoKysDALi5ubV43M3NDTdv3uxwUERERNQ5tEoEmp4C9OzZs8XjPXv2hEgk6nhURERE1CnaHCOQnp6u+nNhYSEAoKSkBC4uLs3q3rhxA7169dJheERERKRPbSYCGRkZzd77nz17tsVE4PLly3B2dtZddERERKRXGhOBuzcbultL8/YvX76MxsZGPPzww7qJjIiI7htxcXGqPWfeeeedrg6HdEhjIiBkmd8mAwYMwMqVKzscEBER3X9KS0tVr4fpwdKlew0QERFR19JqQaHGxkYUFRWhqqoKIpEIVlZWsLe311dsREREpGeCEoFTp05h3759yMzMRGNjo9oxS0tLBAcHY8qUKbC2ttZHjERERKQnGhMBpVKJVatW4fDhw82OSaVSmJqaorCwEHv37sWRI0fw9ttvw8vLS2/BEhG15vmfj3d1CA+0xvI728UXltfwv7Ue/XvOiE4/p8ZEQCaT4fDhwwgICMDMmTPRu3dvFBUVYfPmzbh48SLef/992Nvb49ixY1i3bh3i4uKwbNky2Nradlb8RERE1AEaBwsmJSXBxcUFb731FlxdXWFmZgZXV1csXLgQ1tbW2LBhA3r27IkxY8bgww8/RG1tLbZv395JoRMREVFHaUwE8vLy8PDDD8PY2Fit3NjYGA8//LDaqoOurq4ICAhAamqqfiIlIiIindOYCIhEIsjl8haPyeVy1NfXq5U5Oztz0yEiogeRaS/A3OrOv+mBonGMQN++fXH69Gk89dRTsLCwUJVXVlbi9OnTcHR0VKtfW1sLsVisn0iJiKjLGAdM7uoQSE80JgJjx47FDz/8gPfeew8RERFwcHBAcXEx9uzZg9u3byMiIkKt/rVr19CnTx+9BkxERES6ozERCAsLQ0ZGBo4dO4Y1a9aoHfP391dLBGpqaiCXyzFy5Ej9REpEREQ61+aCQq+//jqGDx+OU6dO4fbt2+jVqxcCAgIwcuRIGBn9b4iBmZkZPv30U70GS0RERLolaGXBoKAgBAUF6TsWIiIi6mTcdIiIiMiAMREgIiIyYEwEiIiIDBgTASIiIgPGRICIiMiACZo1oE95eXmIj49HdnY2JBIJQkNDMX36dLWpia05efIktm/fjqtXr8LExARubm5YuHAhTE1NOyFyIiKi7q9LE4HKykrExMTAxcUFixYtQmFhIdatWwelUolZs2ZpbLt//37Ex8dj8uTJeOaZZ1BVVYULFy5AoVB0UvRERETdX5cmAomJiZDL5Vi4cCHMzc3h6+uLmpoaJCQkYPLkyTA3N2+xXXl5OX7++We88MILCAsLU5VzrQMiIiLtaD1GICMjA1u2bNH6WEvS0tLg5+endsMPDg6GXC5HRkZGq+2OHz8OABgzZozgcxEREVFzWicC6enpSEhI0PpYS/Lz8+Hk5KRWJpVKYWJigoKCglbbXbp0CU5OTkhKSsLLL7+M2bNn47333sPFixcFn5uIiIi6eNZAVVUVJBJJs3KJRILKyspW292+fRsFBQXYunUrnn76abzzzjswMTHBP//5T5SVlekxYiIiogdLl88aaA+lUona2lq8+eab8Pf3BwB4eHjg1VdfxW+//dbiQEOZTAaZTAYAiI2NhVQq1Xlc13XeI1HX0Mf1QURt64prr0sTAYlEgurq6mblVVVVsLCw0NhOJBLB29tbVWZubo4BAwYgLy+vxTZhYWFqAwtLS0s7EDnRg43XB1HX0Ne1d+9r+LsJSgTuDqyqqqpZGdC+LMbZ2Rn5+fnNzlVXV6cxaGdnZyiVymblSqVS0PoDREREdIegRODVV1/VWCYSibBx40atT+7v74+dO3eipqYGZmZmAIDk5GSIxWK1X/v3Gjp0KLZs2YILFy4gICAAAFBdXY3Lly/j8ccf1zoOIiIiQyUoEXjyySchEokA3JkimJGRgWnTpnX45OHh4di7dy+WLl2KyMhIFBcXIyEhAREREWpTCqOiouDt7Y0FCxYAANzc3BAYGIjvvvsOTz31FCwtLbFjxw4YGxtjwoQJHY6LiIjIUAhKBGbMmKH6c0JCAjIyMjB9+vQOn9zCwgKLFy/GmjVrEBcXB4lEgkmTJqmdDwAUCkWzFQNff/11rFu3DmvXrkVdXR28vLwQHR2tcWwBERERqevyWQMuLi6Ijo7WWGflypXNykxNTfH3v/8df//73/UVGhER0QOPI+uIiIgMGBMBIiIiA6Z1ItDStD0hx4iIiOj+o/UYgRkzZjQbzCfkGBEREd1/+GqAiIjIgDERICIiMmCtJgJyubzDneuiDyIiItKfVhOBV199Fb/++ivq6+u17jQnJweff/45du7c2aHgiIiISL9aHSzo5+eHn3/+GQkJCRg5ciRGjBgBDw8PiMXiFusXFRXh3LlzOHToEP78809IpVJMnjxZb4ETERFRx7WaCLz22muYOHEiNm7cCJlMBplMBiMjI7i4uMDa2hoSiQT19fWorKxEQUEBysvLAQCWlpaYPXs2Jk2ahJ49e3baFyEiIiLtaZw+OHDgQHzwwQe4fv06kpKScOHCBeTk5ODq1atq9SwtLTFs2DDVPz16dPnKxURERCSAoDu2o6Mjnn76aQBAXV0dbt68iYqKCojFYlhZWcHGxkavQRIREZF+aP3T3cTEBI6OjnB0dNRHPERERNSJuI4AERGRAWMiQEREZMCYCBARERkwJgJEREQGjIkAERGRAWMiQEREZMCYCBARERkwrdcRaGhowIULF5CXl4fa2lpMmzYNwJ2dBmtqatCrVy8YGTG/ICIi6g60SgTS0tKwevVqlJWVqcqaEoGcnBx8+OGHiIqKwqhRo3QaJBEREemH4J/uf/31F7744guIRCLMmTMHwcHBasc9PDzg4OCAU6dO6TxIIiIi0g/BicDWrVshFosRGxuLxx57rMUlht3c3JCbm6vTAImIiEh/BCcCFy9exCOPPAJra+tW60ilUrXXBkRERHR/E5wI1NbWwtLSUmOduro6KBSKDgdFREREnUPwYEFbW1tcu3ZNY52cnBz07t1bqwDy8vIQHx+P7OxsSCQShIaGYvr06RpnHhQXF+O1115rVj5y5Ei88cYbWp2fiIjIkAlOBPz9/ZGYmIisrCx4eXk1O56amors7GxERkYKPnllZSViYmLg4uKCRYsWobCwEOvWrYNSqcSsWbPabP/ss8/C09NT9bmtJxZERESkTnAiMHXqVCQnJ+OTTz7BxIkTUVJSAgA4e/YsMjIysG/fPlhbWyMiIkLwyRMTEyGXy7Fw4UKYm5vD19cXNTU1SEhIwOTJk2Fubq6xvZOTEzw8PASfj4iIiNQJHiNga2uL999/HzY2Nti1axdOnDgBAIiLi8OuXbtgY2OD999/X6tf5WlpafDz81O74QcHB0MulyMjI0OLr0FERETtodWCQgMGDMDy5ctx9uxZZGdno6KiAubm5nB3d8cjjzwCY2NjrU6en58PHx8ftTKpVAoTExMUFBS02X7VqlWorKyElZUVgoODMXv2bIjFYq1iICIiMmRaLzFsZGSEwMBABAYGdvjkVVVVkEgkzcolEgkqKytbbdezZ09MmDABfn5+MDMzQ3p6Onbs2IGioiIsWrSow3EREREZCq0TgfuBjY0N5s6dq/rs4+MDa2tr/Pjjj8jJyYGrq2uzNjKZDDKZDAAQGxsLqVSq87iu67xHoq6hj+uDiNrWFdee4ETg0KFDgjsNCQkRVE8ikaC6urpZeVVVFSwsLASfDwCGDx+OH3/8EZcvX24xEQgLC0NYWJjqc2lpqVb9ExkSXh9EXUNf156Tk1OrxwQnAqtWrRJ8QqGJgLOzM/Lz89XKSktLUVdXpzFoTUQiUbvaERERGSLBicCCBQtaLK+ursaff/6J5ORkBAUFISAgQPDJ/f39sXPnTtTU1MDMzAwAkJycDLFYDG9vb8H9AFDNYhgwYIBW7YiIiAyZ4ERgzJgxGo+PHTtWtSGRUOHh4di7dy+WLl2KyMhIFBcXIyEhAREREWpTCqOiouDt7a1KRjZv3oza2lp4enrCzMwMmZmZ2LlzJ4KCgvDQQw8JPj8REZGh09lgwYcffhh+fn7YtGkToqOjBbWxsLDA4sWLsWbNGsTFxUEikWDSpEmYMWOGWj2FQqG2h4GzszN27dqF/fv3Qy6XQyqVYvLkyXjiiSd09XWIiIgMgk5nDTg5OSExMVGrNi4uLm0mDitXrlT7HBwcjODgYK3jIyIiInWCVxYUIi8vT5fdERERkZ51+ImAQqHAjRs3sH//fqSmpmLIkCG6iIuIiIg6geBEYObMmW3WsbCwwDPPPNOhgIiIiKjzCE4EBg0a1OIcfZFIBIlEgoEDB2Ls2LHcCpiIiKgbEZwIfPTRR3oMg4iIiLqCTgcLEhERUffCRICIiMiAtfpqQJu9Be4mEolaXY6YiIiI7i+tJgLa7DZ4LyYCRERE3UOricCKFSs6Mw4iIiLqAq0mAvb29p0ZBxEREXUBDhYkIiIyYO1aYlihUKC8vBwNDQ0tHpdKpR0KioiIiDqHVonA1atXsX79eqSnp6O+vr7FOiKRCBs3btRJcERERKRfghOBvLw8fPDBBwAAX19fnDlzBg899BCsrKxw5coVVFRUwMfHh08DiIiIuhHBicC2bdvQ2NiIzz77DP369cPMmTMRFBSEadOmoba2Fj/99BNSU1Pxyiuv6DNeIiIi0iHBgwXT09MREBCAfv36qcqUSiUAwNTUFC+99BIkEgk2bdqk+yiJiIhILwQnAhUVFXB0dPxfQyMj1NXVqT4bGxvDx8cH58+f122EREREpDeCEwELCwvU1taqPltaWqK0tFStTo8ePVBdXa276IiIiEivBCcCvXv3RnFxsepz//798ccff+D27dsAgNraWpw+fRoODg66j5KIiIj0QvBgQT8/P+zYsQO1tbUwNTXF+PHjkZqaikWLFsHT0xOXL19GSUkJnnvuOX3GS0RERDokOBEYN24cnJycIJfLYWpqioCAAMyZMwcJCQk4efIkxGIxIiMj8be//U2f8RIREZEOaUwEFi1ahLCwMDz66KOwsbHByJEj1Y4/9thjmDhxIsrLy2FlZQWRSKTXYImIiEi3NI4RyM3NxZo1azB//nx89913uHTpUvMOjIxgbW3NJICIiKgb0vhEICYmBjKZDCdOnMCBAwdw4MAB9OvXD+PGjcPo0aNhbm7eWXESERGRHmhMBDw8PODh4YEXXngBR44cQVJSEq5cuYKffvoJ69evx/DhwzFu3Dh4eXl1VrxERESkQ4IGC5qZmWH8+PEYP348cnJyIJPJcOzYMRw+fBiHDx+Gi4uL6imBhYWFVgHk5eUhPj4e2dnZkEgkCA0NxfTp02FkJGxmo0KhwHvvvYfLly/jnXfewdChQ7U6PxERkSETvI5AE1dXV8ybNw//+te/8Morr8DT0xN5eXn4+eef8fLLL+Pbb78V3FdlZSViYmIgEomwaNEiPPnkk9i9ezc2b94suI+kpCTcuHFD269BREREaEci0EQsFiMkJAQff/wxvvrqK3h5eaG+vh5Hjx4V3EdiYiLkcjkWLlwIX19fjB8/HtOmTcPu3bsFrVBYWVmJX375BbNnz27v1yAiIjJo7U4EgDs34j179mDZsmXIysoCAK0GEKalpcHPz0+tTXBwMORyOTIyMtpsv2nTJnh6emLw4MHaB09ERETCFxS624ULFyCTyZCSkoKGhgYAgLu7O8LCwpqtNaBJfn4+fHx81MqkUilMTExQUFCgsW1ubi4OHDiApUuXav8FiIiICIAWiUBZWRkOHDiApKQk1Z4DEokEYWFhCAsLQ9++fbU+eVVVFSQSSbNyiUSCyspKjW3j4+MxceJE9OnTR20PBCIiIhJOYyKgVCpx9uxZ7N+/H6mpqVAoFAAALy8vjBs3DsOHD4dYLO6UQO927NgxFBQU4J133hHcRiaTQSaTAQBiY2MhlUp1Htd1nfdI1DX0cX0QUdu64trTmAi88soruHnzJoA72xCPHj0aYWFhcHZ21snJJRJJi4MCq6qqWp2G2NDQgP/85z+IjIyEUqlEVVUVampqAAB1dXWoqamBmZlZs3ZNTy6a3LuFMhH9D68Poq6hr2vPycmp1WMaE4GbN2/C29tb9eu/R492DSlolbOzM/Lz89XKSktLUVdX12rQdXV1uHHjBtauXYu1a9eqHfv666/Ru3dvraYwEhERGTKNd/avv/4ajo6Oeju5v78/du7cqfYrPjk5GWKxGN7e3i22MTU1RXR0tFpZWVkZli9fjtmzZ3MGARERkRY0JgL6TAIAIDw8HHv37sXSpUsRGRmJ4uJiJCQkICIiQm1KYVRUFLy9vbFgwQIYGxs3m2nQNFiwX79+cHd312vMRERED5IOrSPQURYWFli8eDEUCgXi4uKwefNmTJo0CTNmzFCrp1AoVAMViYiISHd0+9K/HVxcXJo96r/XypUrNR53cHDQalliIiIiuqNLnwgQERFR12IiQEREZMCYCBARERkwwYnAyZMnOWCPiIjoASN4sOCXX34JGxsbjB07FuPGjeMSpERERA8AwU8EJkyYgLq6Omzbtg1RUVGIjY3FmTNnoFQq9RkfERER6ZHgJwIvvvginnnmGSQnJyMxMRGpqalITU2Fra0txo0bh9DQUNja2uozViIiItIxrdYREIvFGDNmDMaMGYOrV69CJpPhyJEjSEhIwNatWxEQEIDw8HD4+/vrKVwiIiLSpXYvKNSvXz+1pwSbNm3C6dOncfr0aUilUkyYMAHjx4+HqampLuMlIiIiHerQ9MHa2locPnwYv/32m2q7YldXV1RWVmL9+vX4xz/+gZycHF3ESURERHrQricCV65cQWJiIo4dO4ba2lqIxWKEhoZiwoQJcHV1RW1tLfbt24fNmzfjp59+wpIlS3QdNxEREemA4ESgrq4Ox44dQ2JiIi5fvgwAcHZ2Rnh4OEJCQtR2CzQ1NUVkZCRu3LiBpKQk3UdNREREOiE4EZg/fz5qampgZGSEYcOGYcKECc22A76Xra0t6uvrOxwkERER6YfgRMDMzAwREREICwuDtbW1oDbjx49HcHBwe2MjIiIiPROcCKxcuRJGRtqNLTQ3N1d7ZUBERET3F8F3dm2TACIiIrr/Cb67b926FbNnz1ZNE7zXzZs3MXv2bGzfvl1XsREREZGeCU4Ezpw5A29v71aXEba1tcXgwYORkpKis+CIiIhIvwQnAoWFhXBxcdFYx9nZGYWFhR0OioiIiDqH4ERALpfDxMREYx2xWIza2toOB0VERESdQ3AiYGdnh0uXLmmsc+nSJe5ASERE1I0ITgT8/PyQkZGB5OTkFo8fO3YMGRkZ3HmQiIioGxG8jsCUKVNw9OhRLF++HMnJyfD394etrS1u3ryJ1NRUnD59GhYWFpgyZYoewyUiIiJdEpwI2Nra4v3338eXX36JlJSUZrMD7O3t8eabb8LOzk7nQRIREZF+aLX7oJubG5YvX44zZ87g0qVLqKqqgkQigbu7O4YOHYoePdq1mSERERF1Ea3v3D169MCwYcMwbNgwfcRDREREnajLf8Ln5eUhPj4e2dnZkEgkCA0NxfTp0zUuaXzt2jWsXbsWV69eRUVFBaysrODn54eZM2fCxsamE6MnIiLq3lpNBA4dOgQACAoKgpmZmeqzECEhIYLqVVZWIiYmBi4uLli0aBEKCwuxbt06KJVKzJo1q9V21dXVcHBwQEhICGxsbFBcXIwtW7bg8uXL+Oyzz2BsbCw4ViIiIkPWaiKwatUqAIC7uzvMzMxUn4UQmggkJiZCLpdj4cKFMDc3h6+vL2pqapCQkIDJkye3unOhp6cnPD09VZ99fHxgZ2eHTz75BLm5uRgwYIDgWImIiAxZq4nAggULAED1qL3psy6lpaXBz89P7YYfHByM9evXIyMjA4GBgYL7srCwAAA0NDToPE4iIqIHVauJwJgxYzR+1oX8/Hz4+PiolUmlUpiYmKCgoKDN9gqFAgqFAsXFxdiwYQPc3NwwcOBAncdJRET0oOrSwYJN0w/vJZFIUFlZ2Wb7zz77DOfOnQMADBgwAO+++67GQYZERESkrstnDXTEiy++iMrKSly/fh3btm3DP//5T8TExEAsFjerK5PJIJPJAACxsbGQSqU6j+e6znsk6hr6uD6IqG1dce21mgi89tpr7epQJBLh22+/FVRXIpGgurq6WXlVVZXqnb8mjo6OAO4MaBw0aBBee+01HD16FKGhoc3qhoWFISwsTPW5tLRUUIxEhojXB1HX0Ne15+Tk1OqxVhMBpVLZrpNp087Z2Rn5+flqZaWlpairq9MYdEvs7e1hYWGB4uJirdoREREZslYTgZUrV+r95P7+/ti5cydqampgZmYGAEhOToZYLIa3t7dWfRUUFKCiogIODg76CJWIiOiB1KVjBMLDw7F3714sXboUkZGRKC4uRkJCAiIiItSmFEZFRcHb21s1hXHt2rUwNjaGu7s7zM3NkZ+fj507d6J3794YOXJkV30dIiKibqfdiUBNTQ2qqqpgbm7e6sI/bbGwsMDixYuxZs0axMXFQSKRYNKkSZgxY4ZavaZpgk3c3Nzw22+/QSaTob6+HlKpFMOGDcOUKVNgamra3q9ERERkcERKLV7qNzY2YteuXdi/f7/au3gHBweMGzcOjz/+eLdZ3lfIOgXauv72PJ33SdQVHL/4satD0NrzPx/v6hCIOuzfc0bopd92DRa8V0NDAz799FNkZGRAJBJBKpXC2toaZWVlKCkpwS+//IK0tDR88MEH3I6YiIiomxB8x969ezcyMjIQEBCA5557TjV1DwAKCwuxdu1anDlzBrt378aUKVP0ESsRERHpmOBl+I4ePYq+ffvi7bffVksCAKBPnz5466230LdvXxw5ckTnQRIREZF+CE4ECgsL4e/v3+oSvkZGRvD390dRUZHOgiMiIiL9EpwI9OjRA7W1tRrr1NXVdZvBgkRERKRFIvDQQw/h5MmTKC8vb/F4eXk5Tpw4AVdXV13FRkRERHomOBGYMGECysvL8e677yIpKQlFRUWQy+UoLi7GgQMH8P7776O8vBwTJkzQZ7xERESkQ4JnDYwcORI5OTnYsWMH/vWvf7VYZ/LkyVzZj4iIqBvRasL/U089hcDAQCQlJSEnJwfV1dUwNzeHq6srQkND4eHhoa84iYiISA8EJwIVFRUQiUTw8PDgDZ+IiOgB0WYikJKSgrVr16qWFO7Tpw+effZZBAYG6j04IiIi0i+NgwWzs7OxbNkytX0FCgsLsWzZMmRnZ+s9OCIiItIvjYnA7t27oVQq8eSTT+KHH37A999/jyeeeAIKhQK7d+/urBiJiIhITzS+Grh06RK8vLzUtgWeOXMmMjIy+ESAiIjoAaDxicDt27fh7u7erNzd3b3VhYWIiIio+9CYCDQ2NsLU1LRZuYmJCRobG/UWFBEREXUOwSsLEhER0YOnzemDBw8eRHp6ulpZSUkJAGDJkiXN6otEIixevFhH4REREZE+tZkIlJSUqG7898rIyNB5QERERNR5NCYC0dHRnRUHERERdQGNiYC3t3dnxUFERERdgIMFiYiIDBgTASIiIgPGRICIiMiAMREgIiIyYEwEiIiIDFib6wjoW15eHuLj45GdnQ2JRILQ0FBMnz4dRkat5yh//vknfv/9d2RmZuLWrVuws7PDqFGjEBkZCbFY3InRExERdW9dmghUVlYiJiYGLi4uWLRoEQoLC7Fu3ToolUrMmjWr1XbJyckoKipCZGQkHB0dkZubi02bNiE3NxdvvfVWJ34DIiKi7q1LE4HExETI5XIsXLgQ5ubm8PX1RU1NDRISEjB58mSYm5u32G7KlCmwtLRUffbx8YFYLMb333+PkpIS2Nvbd9ZXICIi6tZaTQS2bNnS7k6nTZsmqF5aWhr8/PzUbvjBwcFYv349MjIyEBgY2GK7u5OAJq6urgCAW7duMREgIiISqNVEICEhod2dCk0E8vPz4ePjo1YmlUphYmKCgoICrc6ZnZ0NkUiE3r17a9WOiIjIkLWaCLS0z8Du3buRmpqKRx99FN7e3rC2tkZZWRnS09Nx9OhRBAQEYNKkSYJPXlVVBYlE0qxcIpGgsrJScD9lZWXYtm0bRo8eDSsrK8HtiIiIDF2ricC9+wwcOnQIf/zxBz799FMMGDBA7diYMWMwceJEREdHY9iwYfqJtBUNDQ346quvYGpqijlz5rRaTyaTQSaTAQBiY2MhlUp1Hst1nfdI1DX0cX0QUdu64toTPFhwz549GDFiRLMkoImbmxtGjBiBPXv2YPTo0YL6lEgkqK6ublZeVVUFCwuLNtsrlUqsWLEC165dQ0xMjMY2YWFhCAsLU30uLS0VFCORIeL1QdQ19HXtOTk5tXpM8IJCBQUFsLGx0VjHxsZGq3f7zs7OyM/PVysrLS1FXV2dxqCb/Pvf/0ZKSgoWLVoEZ2dnweclIiKiOwQnAmZmZrh48aLGOhcvXoSpqangk/v7++PcuXOoqalRlSUnJ0MsFre5BfJ///tf/Pbbb4iKioKXl5fgcxIREdH/CE4EAgICkJmZibVr16rduAGgpqYGa9euRVZWFoYOHSr45OHh4ejZsyeWLl2K8+fPQyaTISEhAREREWpTCqOiorB69WrV56NHj+KXX35BSEgIbG1tkZ2drfqnvLxc8PmJiIgMneAxAk899RQyMjKwZ88eJCUlwdXVFVZWVrh9+zZycnJQU1MDBwcHzJ49W/DJLSwssHjxYqxZswZxcXGQSCSYNGkSZsyYoVZPoVBAoVCoPp87dw4AcPDgQRw8eFCt7iuvvIIxY8YIjoGIiMiQiZRKpVJo5YqKCmzYsAFHjx6FXC5XlYvFYjz66KOYPXs2evXqpZdAdU3bdQqEuP72PJ33SdQVHL/4satD0NrzPx/v6hCIOuzfc0bopV9N4+60WmK4V69emD9/PubNm4f8/HxUV1fD3Nwczs7OMDY27nCgRERE1LnatdeAsbEx+vXrp+tYiIiIqJNpnQg0NDTgwoULyMvLQ21trWo5YblcjpqaGvTq1UvjFsJERER0/9AqEUhLS8Pq1atRVlamKmtKBHJycvDhhx8iKioKo0aN0mmQREREpB+Cf7r/9ddf+OKLLyASiTBnzhwEBwerHffw8ICDgwNOnTql8yCJiIhIPwQnAlu3boVYLEZsbCwee+wxODo6Nqvj5uaG3NxcnQZIRERE+iM4Ebh48SIeeeQRWFtbt1pHKpWqvTYgIiKi+5vgRKC2thaWlpYa69TV1akt/ENERET3N8GJgK2tLa5du6axTk5ODnr37t3hoIiIiKhzCE4EmjYIysrKavF4amoqsrOzERAQoLPgiIiISL8ETx+cOnUqkpOT8cknn2DixIkoKSkBAJw9exYZGRnYt28frK2tERERobdgiYiISLcEJwK2trZ4//338dVXX2HXrl2q8ri4OABA79698dZbb7U5joCIiIjuH1otKDRgwAAsX74cZ8+eRXZ2NioqKmBubg53d3c88sgj3G+AiIiom9F6iWEjIyMEBgYiMDBQH/EQERFRJxI8WHDJkiU4dOiQxjqHDx/GkiVLOhwUERERdQ7BiUBGRoZqgGBrSktLkZGR0eGgiIiIqHPodJtAuVzOcQJERETdiNZjBFqiVCpRWlqK1NRU2NnZ6aJLIiIi6gQaE4GZM2eqfU5ISEBCQoLGDqdOndrxqIiIiKhTaEwEBg0aBJFIBODOGAGpVAoHB4dm9YyMjGBhYYGHH34YoaGh+omUiIiIdE5jIvDRRx+p/jxz5kyMHTsW06ZN03dMRERE1EkEjxFYsWIFJBKJPmMhIiKiTiY4EbC3t9dnHERERNQFtJ41cOvWLfzxxx+4efMmGhoaWqzD1wdERETdg1aJwObNm7F9+3Y0NjZqrMdEgIiIqHsQnAgcOXIEW7duxeDBgzFhwgQsW7YMISEh8PPzQ3p6Og4cOIDhw4cjPDxcn/ESERGRDglOBH7//XfY2trivffeU60e6ODggODgYAQHByMoKAixsbEIDg7WKoC8vDzEx8cjOzsbEokEoaGhmD59OoyMWl/0sKGhAb/88gsuXbqEv/76C/X19di8ebNW5yUiIiItlhi+evUqhgwZoraEsEKhUP3Z398ffn5+2LVrl+CTV1ZWIiYmBiKRCIsWLcKTTz6J3bt3t3lTr6urQ1JSEkxMTODp6Sn4fERERKRO8BOBxsZG9OrVS/VZLBajurparU7fvn2RmJgo+OSJiYmQy+VYuHAhzM3N4evri5qaGiQkJGDy5MkwNzdvsZ1EIkF8fDxEIhF+++03XLhwQfA5iYiI6H8EPxGwsbHBrVu3VJ+lUilyc3PV6ty6dUurTYfS0tLg5+endsMPDg6GXC5vcxfDphUPiYiIqP0EJwKurq64du2a6rOPjw+ysrJw+PBh1NbW4uzZszhx4gT69+8v+OT5+flwcnJSK5NKpTAxMUFBQYHgfoiIiKh9BCcCQ4cOxbVr11BcXAwAmDJlCszNzbFy5UrMmTMHcXFxAJpvVKRJVVVVi6sVSiQSVFZWCu6HiIiI2kfwGIExY8ZgzJgxqs9SqRSfffYZdu3ahaKiItjb22PChAno16+fPuLsMJlMBplMBgCIjY2FVCrV+Tmu67xHoq6hj+uDiNrWFdee1isL3s3BwQFz585td3uJRNJswCFw50mBhYVFR0JrJiwsDGFhYarPpaWlOu2f6EHC64Ooa+jr2rv3NfzdBL8a0AdnZ2fk5+erlZWWlqKurk5j0ERERKQbWj8RUCgUuHnzpsa9Bry9vQX15e/vj507d6KmpgZmZmYAgOTkZIjFYsF9EBERUftplQjs3LkTu3btQnl5ucZ6mzZtEtRfeHg49u7di6VLlyIyMhLFxcVISEhARESE2pTCqKgoeHt7Y8GCBaqy1NRU1NXVIScnBwBw4sQJAICbmxt3SiQiIhJIcCKwefNmbN26FRYWFggJCYGtra1Wawa0xMLCAosXL8aaNWsQFxcHiUSCSZMmYcaMGWr1FAqF2iqGAPDjjz+ipKRE9fnLL78EALzyyitqgxqJiIiodYITgQMHDsDBwQFxcXGtrvjXHi4uLoiOjtZYZ+XKlYLKiIiISDuCBwtWVFQgMDBQp0kAERERdS3BiUCfPn1QVVWlz1iIiIiokwlOBMaPH48zZ86grKxMj+EQERFRZxI8RmD8+PG4fv06PvzwQzz55JMYMGBAq68JuCoZERFR96DV9MGHHnoIBw8exOrVq1utIxKJsHHjxg4HRkRERPonOBHYv38/vv/+exgbG8PHxwc2NjYdnj5IREREXUtwIrBr1y5YWVnhk08+gYODgz5jIiIiok4ieLBgSUkJhg8fziSAiIjoASI4EbC1tW11bwEiIiLqngQnAiEhIUhNTUVNTY0+4yEiIqJOJDgRmDp1KgYOHIiYmBikp6czISAiInoACB4s+NRTT6n+/PHHH7daj9MHiYiIug/BicCgQYMgEon0GQsRERF1MsGJwEcffaTHMIiIiKgrCB4jQERERA8eJgJEREQGrNVXA1u2bAEATJw4ERYWFqrPQkybNq3jkREREZHetZoIJCQkAABGjhwJCwsL1WchmAgQERF1D60mAtHR0QD+t6Vw02ciIiJ6cLSaCHh7e2v8TERERN2f4MGChw4dQm5ursY6V69exaFDhzocFBEREXUOwYnAqlWrkJKSorHO6dOnsWrVqg4HRURERJ1Dp9MHFQoFVx8kIiLqRnSaCBQUFEAikeiySyIiItIjjUsM3/uYPyUlBcXFxc3qKRQK3LhxA5mZmQgICNBthERERKQ3GhOBewf+5eTkICcnp9X67u7umDNnjk4CIyIiIv3TmAisWLECAKBUKhEVFYXHHnsMjz32WLN6RkZGkEgkMDU11TqAvLw8xMfHIzs7GxKJBKGhoZg+fTqMjDS/taiursa///1vpKSkQKFQYOjQoXjhhRfQq1cvrWMgIiIyVBoTAXt7e9Wfp02bBh8fH7WyjqqsrERMTAxcXFywaNEiFBYWYt26dVAqlZg1a5bGtl999RUKCgowf/58GBkZYf369fjiiy/w8ccf6yw+IiKiB53gbYinT5+u85MnJiZCLpdj4cKFMDc3h6+vL2pqapCQkIDJkyfD3Ny8xXbZ2dk4d+4cPvroI9VCR7a2tnjvvfdw/vx5+Pr66jxWIiKiB5HgWQNXrlzBvn37UF1drSqrra3FihUr8Pzzz2P+/Pn49ddftTp5Wloa/Pz81G74wcHBkMvlyMjIaLVdamoqrKys1FY7HDhwIBwcHJCWlqZVDERERIZMcCKwY8cObNu2Te2mvWHDBhw5cgRKpRIVFRX4+eefce7cOcEnz8/Ph5OTk1qZVCqFiYkJCgoKNLZzdnZuVu7s7Iz8/HzB5yciIjJ0ghOBv/76Cz4+PqrPDQ0NOHToEAYOHIgffvgBK1asgKWlJfbu3Sv45FVVVS2uOyCRSFBZWamxXUuvDSQSCaqqqgSfn4iIyNAJHiNQXl4OOzs71efLly+jtrYWYWFhEIvFsLW1RWBg4H37aF4mk0EmkwEAYmNjmz2J0AWn9dq9GiEi3fn93Se7OgSibkmrlQUbGxtVf87KygKgviuhpaUlysvLBfcnkUjUxhw0qaqqgoWFhcZ2NTU1LbZrbWXDsLAwxMbGIjY2VnB8dH/6v//7v64Ogcgg8dp7MAlOBKRSKS5duqT6nJKSAjs7O/Tu3VtVduvWLY038Hu19E6/tLQUdXV1Gn+xtzYWoKCgoMWxA0RERNQywYnAiBEjkJ2djWXLluGbb75BdnY2hg8frlYnPz9fLTFoi7+/P86dO6f26z45ORlisVjtScO9hgwZgrKyMtVTCeDOGIaioiL4+/sLPj8REZGhE5wIREREwMPDA6dOncKxY8fg6uqKadOmqY4XFxfjzz//1HgDv1d4eDh69uyJpUuX4vz585DJZEhISEBERITaYMCoqCisXr1a9dnDwwN+fn5YsWIFTp48iVOnTuGbb76Bl5cX1xAwAGFhYV0dApFB4rX3YBIplUqlNg2uXr0KAHBxcVFbBri4uBi5ublwc3ODra2t4P7y8vKwZs0atSWGZ8yYodb3q6++Cm9vb7z66quqsqqqKvz88884deoUlEolAgIC8MILL8DS0lKbr0NERGTQtE4EiIiI6MGhcfpgRkYGHBwcIJVKBXWWm5uLnJwchISE6CQ40s6MGTMA3NksysHBQVX+6quvoqSkRK2uiYkJ+vTpg6CgIDz++OOqDaP27t2Ln376CcOHD8ebb77Z7BwFBQV44403AABz587FhAkTmtWRyWT4/vvvMWTIELz77rsAgM2bN2PLli0ICQlRe7JDRHc0Xb93MzMzU12nERERMDEx6YLI2nbw4EGsWrUK06ZNa/F70P1N4xiBJUuW4ODBg2pl27dvx4svvthi/VOnTmHVqlU6C450y8/PDyEhIQgJCYGHhwcKCwuRkJCA999/X7UQ06BBgwAAFy9ebLGPuwdo3v3nu2VmZqr1RUTCDRs2DCEhIRg9ejQGDBiAvLw8bNq0CR988AFqa2u7Ojx6AAleUKhJfX09V+/rpqZMmaK2OmRxcTGWLFmCa9euYdu2bXj22WfRr18/mJub49atWygsLESfPn3U+sjMzIRIJELfvn1bTQSaypkIEGnv2WefVXuid/36dSxevBi5ubn49ddf8cQTT3RhdPQg0mpBIXqwODg4qB7jpaSkAACMjIzg6ekJoOVf/FlZWXBxcUFAQABu3LiB4uJiteM3btxASUkJxGIx3Nzc9PwNiB58jo6OmDRpEgDctyu3Uvem9RMBerD0798fwJ2FnJoMGjQIqampyMzMxJgxY1Tlt27dQlFREcLCwuDl5QXgTmJw96+XptcC7u7u6NGjY3+9Ll26hF27diErKwsVFRWwsrKCv78/pk2b1mzcSmVlJY4ePYozZ86goKAAZWVlEIvFcHV1xYQJE5qteQEAK1euxKFDhxAdHY36+nrs2LEDV65cQXV1NX766SekpKSo3nuGhobil19+Ua174ejoiMcff7zV8TClpaXYsWMH0tLScPPmTYjFYnh6emLq1KmqRKtJeno6lixZgpCQEDz99NPYuHEj0tLSUFZWhmeffVZ1EyDD1bdvXwDA7du3mx07c+YMUlJScPHiRdy8eRONjY2wt7dHUFAQIiMjm+3Lcvfftzlz5mDjxo1ISUlBRUUF7O3tERYWhoiICIhEombnysrKQkJCAi5dugQjIyO4u7u3OSagsrIS27dvR0pKCkpLS1U/EiIiIlpc92XGjBmwt7fHN998g507d+LgwYMoLS2Fra0tJkyYgIiICABATk4ONm/ejKysLNTX18PT0xNz5sxR/bci4ZgIGLimxZx69uypKrv7Jn+3u9/9e3h4QCQSITMzE6NHj1bV0dVrgX379iE+Ph4A4ObmBi8vL1y/fh1JSUk4ffo0PvroI7i4uKjqX7x4EfHx8bC3t0efPn3g7u6OW7duISsrC+np6Zg5cyaefLLlteiPHj2KpKQkDBgwAP7+/3979x9TVfkHcPx9+U1I/Jrjh+KQ0eUySIT4ZVGtmszMKdmcK2sVTUlnri1cc5WkyZhbuPVLWG20VbIFDiSgMhFXMTLCKyCXHxcEjUSYDgi6cAXuvd8/2DlxuOcqGFnfeF5/Ps899zy7O/c8n/M8n+c5qxkYGFDcBK9fv86+fftwc3MjJiaG4eFh2tra+Oijj7BarTzyyCOK7zMajeTm5mIymQgJCSEuLo7R0VGamppobGxkz5493H///XbtGBkZYd++fVgsFnQ6HRMTE//a5DDhzpL+pz4+PnZ1R48eZXJykhUrVhAaGsqNGzfo7u6mrKwMvV7PO++8IycDz2QymXjzzTcxmUxERUVhMplob2/n888/Z3x83K6DP3fuHO+++y4Wi0V+7fuvv/5Kdna24oFhpsHBQbKzsxkYGCAgIIDExERGRkZoaWmhubmZZ599lo0bN6oe+95779HY2Eh0dDTBwcEYDAY+++wzzGYzMTEx5OTkEBQUxKpVq+jt7aW5uZm3336bI0eOqP5OgmMiEFjkzp07B8CKFSvksoiICFxdXbl69SrDw8P4+voCf3byOp2OJUuWsHz5crtgYeZnbpfRaOTTTz/F19eXvXv3EhERIdfV1NRQUFBAfn4+OTk5cvmyZcs4dOgQWq1W8V39/f0cPHiQkpISHnroIZYuXWp3vtOnT/Pqq6+qds4wnRG9bt06XnjhBXl/i7Nnz3LkyBGOHz+uCATGxsbIy8tjbGyM3bt3K4KkixcvcujQIQoKCoiJibHb8+L8+fMkJSWxZ88e3Nzc5vGLCf910pSA2hN0ZmYmsbGxiqBxYmKCwsJCampqqKqqUg2CGxoa7K63zs5O3nrrLSoqKti4caMcQIyPj5Ofn4/FYmHnzp3yNW+z2SgqKqK8vFy13R9//DEDAwOkpqaya9cueZSwvb2dnJwcjh07RkxMDOHh4YrjpOnF999/X77/9Pb28vrrr1NeXs6ZM2d4+umnWb9+PQBWq5UPP/yQ2tpaTp48KVYuzJPIEVikBgcHqaiooLKyEoC0tDS5zsXFRe58Z68SCAgIkDvTyMhIrly5Ir9oymQy0dvbi7Ozs12HPB8nTpzAarWyfft2RRAA8Oijj5KQkEBnZyc9PT1yeVBQkOo5g4KC2Lx5M1arlYaGBtXzxcfHOwwCAJYuXcpzzz2n2OQqJSWF0NBQrl27pliaeebMGYaGhli/fr0iCIDpkY2nnnoKs9nMDz/8YHceV1dXMjIyRBAgANOd7PXr1ykuLubHH39Eq9XKHd9MSUlJdiNHbm5uZGRk4OzsTH19ver3e3p6smPHDsX1ds8997B69Wp5VEFy9uxZRkZGiIqKUgS+Go2GrVu3Kt5MKxkYGECv1+Ph4UFGRoZiqlCn05GWlobNZuPbb79Vbd+LL74oBwEwPT0SFxfHjRs38Pf3V/wWTk5O8siCNHIpzN0tRwRKSkooKSmxK9+6devf0iDh73PgwAG7Mo1Gw5NPPsmDDz6oKI+KiqKtrY22tjZSUlIYGxvj8uXLrFmzRv6MTqejurqa9vZ2kpKS6OjowGazsXLlStWhyLmwWq20tLTg7u7u8L0ROp2OhoYGurq65BwH6ViDwUBHRwdDQ0NMTk5is9kYHh4GprOv1SQkJNy0TdHR0YqpE0lwcDC9vb0MDQ3JwVFzczMwvQRMjTRl0tXVZVe3cuXKee3KKfw37d69264sPj6erKwsh3k3/f39nD9/nqtXr2I2m5H2iXNxcaG/v1/1mPDwcNWdWENCQtDr9QwODsplUuf6wAMP2H3excWF5ORkvv5a+Rp26SFi9erVqi+je/jhh6moqFDtuJ2dnYmJibErl95lExsba1cnrXCa2W5hbsTUwCISGxuLr68vGo0GNzc3goKCSEhIsFsiCH92WNKf2Wg0YrPZFEP+UtJbW1sbSUlJC7J/wOjoqLxW+plnnrnlZyWDg4McPnxYMUowm6M12LfaMEvtaQemn6hgekmtRFpFsX///pt+58y2z7UdwuKQnJyMh4cHU1NT9PX10dPTg16vp6ysjC1btth9/osvvqCiooL5bhLr6LqWgvipqSm5bGhoCHB8jc5MGJ59jNp03Mxj1DpuX19fxQjc7LapBcxq7Rbm5qaBwJdffnmn2iHcAbP3EbgZrVaLk5MTly9fZnx8XLWTDwwMxM/PTw4WFiJRULqZeXh4OHyqlsxMFiwoKKCnp4fExEQ2bdpESEgId911F05OTjQ1NZGTk+PwRqn2tD+TWvb0rdqfkpJy00Q/tddl36odwuIwex+B2tpaPvjgA44fP05cXJxiuqyuro6vvvoKf39/nn/+ebRaLT4+PvLIQWZmptwhzzaf6/pOUwsC5lMvzI8YERBUeXp6EhYWRnd3Nx0dHbS3t+Pl5WW3NCcyMpJffvmFkZERLl68iEaj+UuJgt7e3ri6uqLRaNi1a9ecblZms5mmpiZ8fHx47bXX7G4SjoZG/w4BAQH09fWRnp5ulwAlCLcjNTWV1tZWqqurOXbsGNnZ2XLdzz//DMD27du57777FMeZzWZ5Wuyv8vPzA5TLjGeavYX5zGPU6uDP0TMxHfbPE2GV4JD0ZH/hwgW6urrkUYKZIiMjsVgsfPPNN0xNTbF8+XLV+cC5cnZ2Jjo6mvHxcS5cuDCnY8bGxrDZbPj5+ak+KdTV1d12e+br3nvvBXCYoCUIt2PLli24urpiMBgUCbzSLq9qw/x1dXXzni5wRAruf/rpJ7s6i8UiByRqxzQ2NvLHH3/Y1UsJs2IH0n+eCAQEh6Q/aE1Njbxhx2zSn13K/F2IP/XmzZvRaDTk5+fT0tJiV282m6mpqWFiYgKYnk/08vKit7dXkXhks9koKyu7o1nEa9euxcfHh/Lyck6dOoXValXUWywWGhsb5dd5C8Jc+Pn5sXbtWgBKS0vl8pCQEAC+++47Rad/6dIlioqKFuz8a9aswdvbG4PBoHj/jM1mo7i4WHWkIDAwkPj4eMxmM4WFhYq5e6PRyMmTJ9FoNKxbt27B2incHjE1IDgkdfLSU4fakH9YWBju7u52Ly1yRK/X88Ybbzisz8nJQafT8dJLL1FYWMjBgwcJDQ0lODgYFxcXrl27xqVLl5icnCQ5ORk3NzecnJzYtGkTRUVFHDhwgOjoaLy9venp6WFgYIANGzbIyyT/bl5eXuzdu5fDhw/zySefUFpaSmhoKEuWLGF4eJienh5MJhNZWVmKvRsE4VbS09Oprq6msbGR7u5uwsPDefzxx/n++++prq6mtbWVsLAwfv/9d3m1T2dnp8Oh+fnw9PTk5ZdfJi8vj6NHj3Lq1Cl5Q6G+vj4ee+wxTp8+bXfcjh072L9/P7W1tbS3t6PVahkdHcVgMGC1Wtm2bZuYQvsXEIGA4NDdd9/NsmXLuHLlimJvgZmcnZ2JiIjAYDAAtw4ERkdHVTPmZ0tLS0Or1VJVVUVrayt6vR53d3f8/f1JTU0lOTlZsXVqeno6AQEBVFVVYTQa5fbu3LkTi8VyxwIBmE60zMvLo7KyUt6qGaZHLqKiokhKSmLVqlV3rD3Cf4Ovry9paWlUVlZSWlpKVlYWwcHB5ObmUlRUhNFopKGhgcDAQLZt28YTTzzBK6+8smDnT0xMJDs7m+LiYrq6uvjtt9+IiIggMzOTvr4+1UDA39+f3NxcTpw4QX19PfX19bi7uxMdHc2GDRuIi4tbsPYJt09jW6hJJEEQBEEQ/u+IHAFBEARBWMREICAIgiAIi5gIBARBEARhEROBgCAIgiAsYiIQEARBEIRFTAQCgiAIgrCIiUBAEARBEBYxEQgIgiAIwiImAgFBEARBWMREICAIgiAIi9j/AFhTGD+rghHTAAAAAElFTkSuQmCC\n" 259 | }, 260 | "metadata": {} 261 | } 262 | ], 263 | "source": [ 264 | "# IPWLearner+ロジスティック回帰の性能をIPS推定量とDR推定量で評価\n", 265 | "ope.visualize_off_policy_estimates_of_multiple_policies(\n", 266 | " policy_name_list=[\"IPWLearner\", \"Random\"],\n", 267 | " action_dist_list=[\n", 268 | " action_choice_by_ipw_learner, # IPWLearnerによるバリデーションデータに対する行動選択\n", 269 | " action_choice_by_random, # ランダム意思決定モデルによるバリデーションデータに対する行動選択\n", 270 | " ],\n", 271 | " estimated_rewards_by_reg_model=estimated_rewards_by_reg_model, # DR推定量に必要な期待報酬推定値\n", 272 | " random_state=12345,\n", 273 | ")" 274 | ] 275 | }, 276 | { 277 | "cell_type": "markdown", 278 | "metadata": {}, 279 | "source": [ 280 | "### どの推定量を信じたとしても、IPWLearner(new_decision_making_model)がランダム意思決定モデル(random)の性能を上回るという結果が得られた" 281 | ] 282 | }, 283 | { 284 | "source": [ 285 | "### 4. 最後に2つの意思決定モデルの真の性能を確認する" 286 | ], 287 | "cell_type": "markdown", 288 | "metadata": {} 289 | }, 290 | { 291 | "cell_type": "code", 292 | "execution_count": 8, 293 | "metadata": {}, 294 | "outputs": [ 295 | { 296 | "output_type": "stream", 297 | "name": "stdout", 298 | "text": [ 299 | "IPWLearner+ロジスティック回帰の性能: 0.725425479831354\nランダム意思決定モデルの性能: 0.6525795707041967\n" 300 | ] 301 | } 302 | ], 303 | "source": [ 304 | "# ipw_learnerとrandomの真の性能を計算する\n", 305 | "# これは、`SyntheticBanditDataset`の`cal_ground_truth_policy_value`メソッドを呼び出すことで計算できる\n", 306 | "performance_of_ipw_learner = dataset.calc_ground_truth_policy_value(\n", 307 | " expected_reward=validation_data['expected_reward'], # バリデーションデータにおける期待報酬\n", 308 | " action_dist=action_choice_by_ipw_learner, # 評価対象の意思決定モデルによる行動選択確率\n", 309 | ")\n", 310 | "performance_of_random = dataset.calc_ground_truth_policy_value(\n", 311 | " expected_reward=validation_data['expected_reward'], # バリデーションデータにおける期待報酬\n", 312 | " action_dist=action_choice_by_random, # 評価対象の意思決定モデルによる行動選択確率\n", 313 | ")\n", 314 | "\n", 315 | "print(f'IPWLearner+ロジスティック回帰の性能: {performance_of_ipw_learner}')\n", 316 | "print(f'ランダム意思決定モデルの性能: {performance_of_random}')" 317 | ] 318 | }, 319 | { 320 | "cell_type": "code", 321 | "execution_count": null, 322 | "metadata": {}, 323 | "outputs": [], 324 | "source": [] 325 | } 326 | ], 327 | "metadata": { 328 | "kernelspec": { 329 | "name": "python3", 330 | "display_name": "Python 3.9.5 64-bit ('mldesignbook-ucg6RFfZ-py3.9': poetry)" 331 | }, 332 | "language_info": { 333 | "codemirror_mode": { 334 | "name": "ipython", 335 | "version": 3 336 | }, 337 | "file_extension": ".py", 338 | "mimetype": "text/x-python", 339 | "name": "python", 340 | "nbconvert_exporter": "python", 341 | "pygments_lexer": "ipython3", 342 | "version": "3.9.5" 343 | }, 344 | "metadata": { 345 | "interpreter": { 346 | "hash": "a588998c237fcc28dc215a10a422972d26151263dec0bff02e1a95f6e2b22b77" 347 | } 348 | }, 349 | "interpreter": { 350 | "hash": "0179470337d0da01265aacbae97f9e7b0116efe29a7e88412a61b2f5e9549d1a" 351 | } 352 | }, 353 | "nbformat": 4, 354 | "nbformat_minor": 4 355 | } -------------------------------------------------------------------------------- /ch03/README.md: -------------------------------------------------------------------------------- 1 | ## 第3章 2 | ### Pythonによる実装 3 | - [`mf.py`](./mf.py): IPS推定量に対応できるMatrix Factorizationを実装. 4 | 5 | ### 簡易実験 6 | - [`naive-vs-ips.ipynb`](./naive-vs-ips.ipynb): 嗜好度合いデータの観測構造にバイアスが存在する状況で、ナイーブ推定量とIPS推定量の挙動を検証. 7 | -------------------------------------------------------------------------------- /ch03/mf.py: -------------------------------------------------------------------------------- 1 | from typing import Tuple, Optional, List 2 | from dataclasses import dataclass 3 | 4 | import numpy as np 5 | from sklearn.metrics import mean_squared_error as calc_mse 6 | from sklearn.utils import check_random_state 7 | from tqdm import tqdm 8 | 9 | 10 | @dataclass 11 | class MatrixFactorization: 12 | """MatrixFactorization. 13 | 14 | パラメータ 15 | ---------- 16 | k: int 17 | ユーザ・アイテムベクトルの次元数. 18 | 19 | learning_rate: float 20 | 学習率. 21 | 22 | reg_param: float 23 | 正則化項のハイパーパラメータ. 24 | 25 | random_state: int 26 | モデルパラメータの初期化を司る乱数. 27 | 28 | """ 29 | 30 | k: int 31 | learning_rate: float 32 | reg_param: float 33 | alpha: float = 0.001 34 | beta1: float = 0.9 35 | beta2: float = 0.999 36 | eps: float = 1e-8 37 | random_state: int = 12345 38 | 39 | def __post_init__(self) -> None: 40 | self.random_ = check_random_state(self.random_state) 41 | 42 | def fit( 43 | self, 44 | train: np.ndarray, 45 | val: np.ndarray, 46 | test: np.ndarray, 47 | pscore: Optional[np.ndarray] = None, # 傾向スコア (Propensity Score; pscore) 48 | n_epochs: int = 10, 49 | ) -> Tuple[List[float], List[float]]: 50 | """トレーニングデータを用いてモデルパラメータを学習し、バリデーションとテストデータに対する予測誤差の推移を出力. 51 | 52 | パラメータ 53 | ---------- 54 | train: array-like of shape (データ数, 3) 55 | トレーニングデータ. (ユーザインデックス, アイテムインデックス, 嗜好度合いデータ)が3つのカラムに格納された2次元numpy配列. 56 | 57 | val: array-like of shape (データ数, 3) 58 | バリデーションデータ. (ユーザインデックス, アイテムインデックス, 嗜好度合いデータ)が3つのカラムに格納された2次元numpy配列. 59 | 60 | test: array-like of shape (データ数, 3) 61 | テストデータ. (ユーザインデックス, アイテムインデックス, 嗜好度合いデータ)が3つのカラムに格納された2次元numpy配列. 62 | 63 | pscore: array-like of shape (ユニークな嗜好度合い数,), default=None. 64 | 事前に推定された嗜好度合いごとの観測されやすさ, 傾向スコア. P(O=1|R=r). 65 | Noneが与えられた場合, ナイーブ推定量が用いられる. 66 | 67 | n_epochs: int, default=10. 68 | 学習におけるエポック数. 69 | 70 | """ 71 | 72 | # 傾向スコアが設定されない場合は、ナイーブ推定量を用いる 73 | if pscore is None: 74 | pscore = np.ones(np.unique(train[:, 2]).shape[0]) 75 | 76 | # ユニークユーザとユニークアイテムの数を数える 77 | n_users = np.unique(train[:, 0]).shape[0] 78 | n_items = np.unique(train[:, 1]).shape[0] 79 | 80 | # モデルパラメータを初期化 81 | self._initialize_model_parameters(n_users=n_users, n_items=n_items) 82 | 83 | # トレーニングデータを用いてモデルパラメータを学習 84 | val_loss, test_loss = [], [] 85 | for _ in tqdm(range(n_epochs)): 86 | self.random_.shuffle(train) 87 | for user, item, rating in train: 88 | # 傾向スコアの逆数で予測誤差を重み付けて計算 89 | err = rating - self._predict_pair(user, item) 90 | err /= pscore[rating - 1] 91 | grad_P = err * self.Q[item] - self.reg_param * self.P[user] 92 | self._update_P(user=user, grad=grad_P) 93 | grad_Q = err * self.P[user] - self.reg_param * self.Q[item] 94 | self._update_Q(item=item, grad=grad_Q) 95 | 96 | # バリデーションデータに対する嗜好度合いの予測誤差を計算 97 | # 傾向スコアが与えられた場合はそれを用いたIPS推定量で、そうでない場合はナイーブ推定量を用いる 98 | r_hat_val = self.predict(data=val) 99 | inv_pscore_val = 1.0 / pscore[val[:, 2] - 1] # 傾向スコアの逆数 100 | val_loss.append( 101 | calc_mse(val[:, 2], r_hat_val, sample_weight=inv_pscore_val) 102 | ) 103 | # テストデータにおける嗜好度合いの予測誤差を計算 104 | r_hat_test = self.predict(data=test) 105 | test_loss.append(calc_mse(test[:, 2], r_hat_test)) 106 | 107 | return val_loss, test_loss 108 | 109 | def _initialize_model_parameters(self, n_users: int, n_items: int) -> None: 110 | """モデルパラメータを初期化.""" 111 | self.P = self.random_.rand(n_users, self.k) / self.k 112 | self.Q = self.random_.rand(n_items, self.k) / self.k 113 | self.M_P = np.zeros_like(self.P) 114 | self.M_Q = np.zeros_like(self.Q) 115 | self.V_P = np.zeros_like(self.P) 116 | self.V_Q = np.zeros_like(self.Q) 117 | 118 | def _update_P(self, user: int, grad: np.ndarray) -> None: 119 | "与えられたユーザのベクトルp_uを与えられた勾配に基づき更新." 120 | self.M_P[user] = self.beta1 * self.M_P[user] + (1 - self.beta1) * grad 121 | self.V_P[user] = self.beta2 * self.V_P[user] + (1 - self.beta2) * (grad ** 2) 122 | M_P_hat = self.M_P[user] / (1 - self.beta1) 123 | V_P_hat = self.V_P[user] / (1 - self.beta2) 124 | self.P[user] += self.alpha * M_P_hat / ((V_P_hat ** 0.5) + self.eps) 125 | 126 | def _update_Q(self, item: int, grad: np.ndarray) -> None: 127 | "与えられたアイテムのベクトルq_iを与えられた勾配に基づき更新." 128 | self.M_Q[item] = self.beta1 * self.M_Q[item] + (1 - self.beta1) * grad 129 | self.V_Q[item] = self.beta2 * self.V_Q[item] + (1 - self.beta2) * (grad ** 2) 130 | M_Q_hat = self.M_Q[item] / (1 - self.beta1) 131 | V_Q_hat = self.V_Q[item] / (1 - self.beta2) 132 | self.Q[item] += self.alpha * M_Q_hat / ((V_Q_hat ** 0.5) + self.eps) 133 | 134 | def _predict_pair(self, user: int, item: int) -> float: 135 | """与えられたユーザ・アイテムペア(u,i)の嗜好度合いを予測する.""" 136 | return self.P[user] @ self.Q[item] 137 | 138 | def predict(self, data: np.ndarray) -> np.ndarray: 139 | """与えられたデータセットに含まれる全ユーザ・アイテムペアの嗜好度合いを予測する.""" 140 | r_hat_arr = np.empty(data.shape[0]) 141 | for i, row in enumerate(data): 142 | r_hat_arr[i] = self._predict_pair(user=row[0], item=row[1]) 143 | return r_hat_arr 144 | -------------------------------------------------------------------------------- /ch04/README.md: -------------------------------------------------------------------------------- 1 | ## 第4章 2 | 3 | ### PyTorchを用いた実装 4 | - [`evaluate.py`](./evaluate.py): テストデータにおけるnDCG@10を計算するための関数を実装. 5 | - [`loss.py`](./loss.py): IPS推定量に基づくリストワイズ損失関数を実装. 6 | - [`model.py`](./model.py): 多層パーセプトロンに基づくスコアリング関数を実装. 7 | - [`utils.py`](./utils.py): ポジションバイアスが存在するクリックデータを生成するための関数を実装. 8 | 9 | 10 | ### 半人工データを用いた簡易実験 11 | - [`naive-vs-ips.ipynb`](./naive-vs-ips.ipynb): ポジションバイアスが存在する状況においてナイーブ推定量とIPS推定量の性能差を検証. 12 | - [`position-bias-effects.ipynb`](./position-bias-effects.ipynb): ポジションバイアスの大きさがランキング性能に与える影響を検証. 13 | - [`theta-misspecification.ipynb`](./theta-misspecification.ipynb): ポジションバイアスの大きさを見誤ったときのランキング性能の変化を検証. 14 | -------------------------------------------------------------------------------- /ch04/evaluate.py: -------------------------------------------------------------------------------- 1 | from torch import nn 2 | from torch.utils.data import DataLoader 3 | from pytorchltr.evaluation.dcg import ndcg 4 | from pytorchltr.datasets.svmrank.svmrank import SVMRankDataset 5 | 6 | from utils import convert_rel_to_gamma 7 | 8 | 9 | def evaluate_test_performance(score_fn: nn.Module, test: SVMRankDataset) -> float: 10 | """与えられたmodelのランキング性能をテストデータにおける真の嗜好度合い情報(\gamma)を使ってnDCG@10で評価する.""" 11 | loader = DataLoader( 12 | test, batch_size=1024, shuffle=False, collate_fn=test.collate_fn() 13 | ) 14 | ndcg_score = 0.0 15 | for batch in loader: 16 | gamma = convert_rel_to_gamma(relevance=batch.relevance) 17 | ndcg_score += ndcg( 18 | score_fn(batch.features), gamma, batch.n, k=10, exp=False 19 | ).sum() 20 | return float(ndcg_score / len(test)) 21 | -------------------------------------------------------------------------------- /ch04/loss.py: -------------------------------------------------------------------------------- 1 | from typing import Optional 2 | 3 | from torch import ones, FloatTensor 4 | from torch.nn.functional import log_softmax 5 | 6 | 7 | def listwise_loss( 8 | scores: FloatTensor, # f_{\phi}(u,i) 9 | click: FloatTensor, # CTR(u,i,k) 10 | num_docs: FloatTensor, 11 | pscore: Optional[FloatTensor] = None, # \theta(k) 12 | ) -> FloatTensor: 13 | """リストワイズ損失. 14 | 15 | パラメータ 16 | ---------- 17 | scores: FloatTensor 18 | スコアリング関数の出力. 19 | 20 | click: FloatTensor 21 | クリック有無データ. Implicit Feedback. 22 | 23 | num_docs: FloatTensor 24 | クエリごとのドキュメントの数. 25 | 26 | pscore: Optional[FloatTensor], default=None. 27 | 傾向スコア. Noneの場合は、ナイーブ推定量に基づいた損失が計算される. 28 | 29 | """ 30 | if pscore is None: 31 | pscore = ones(click.shape[1]) 32 | listwise_loss = 0 33 | for scores_, click_, num_docs_ in zip(scores, click, num_docs): 34 | listwise_loss_ = (click_ / pscore) * log_softmax(scores_, dim=0) 35 | listwise_loss -= listwise_loss_[:num_docs_].sum() 36 | return listwise_loss / len(scores) 37 | -------------------------------------------------------------------------------- /ch04/model.py: -------------------------------------------------------------------------------- 1 | from dataclasses import dataclass 2 | from typing import Tuple 3 | 4 | from torch import nn, FloatTensor 5 | 6 | 7 | @dataclass(unsafe_hash=True) 8 | class MLPScoreFunc(nn.Module): 9 | """多層パーセプトロンによるスコアリング関数. 10 | 11 | パラメータ 12 | ---------- 13 | input_size: int 14 | 特徴量ベクトルの次元数. 15 | 16 | hidden_layer_sizes: Tuple[int, ...] 17 | 隠れ層におけるニューロンの数を定義するタプル. 18 | (10, 10)が与えられたら、ニューロン数が10個の隠れ層を持つ多層パーセプトロンが定義される. 19 | 20 | activation_func: torch.nn.functional, default=torch.nn.functional.elu 21 | 活性化関数. 22 | 23 | """ 24 | 25 | input_size: int 26 | hidden_layer_sizes: Tuple[int, ...] 27 | activation_func: nn.functional = nn.functional.elu 28 | 29 | def __post_init__(self) -> None: 30 | super().__init__() 31 | self.hidden_layers = nn.ModuleList() 32 | self.hidden_layers.append( 33 | nn.Linear(self.input_size, self.hidden_layer_sizes[0]) 34 | ) 35 | for hin, hout in zip(self.hidden_layer_sizes, self.hidden_layer_sizes[1:]): 36 | self.hidden_layers.append(nn.Linear(hin, hout)) 37 | self.output = nn.Linear(self.hidden_layer_sizes[-1], 1) 38 | 39 | def forward(self, x: FloatTensor) -> FloatTensor: 40 | h = x 41 | for layer in self.hidden_layers: 42 | h = self.activation_func(layer(h)) 43 | return self.output(h).flatten(1) # f_{\phi}, (batch_size, number_of_documents) 44 | -------------------------------------------------------------------------------- /ch04/train.py: -------------------------------------------------------------------------------- 1 | from typing import List, Optional 2 | 3 | from torch import nn, optim 4 | from torch.utils.data import DataLoader 5 | from tqdm import tqdm 6 | from pytorchltr.datasets.svmrank.svmrank import SVMRankDataset 7 | 8 | from evaluate import evaluate_test_performance 9 | from loss import listwise_loss 10 | from utils import convert_rel_to_gamma, convert_gamma_to_implicit 11 | 12 | 13 | def train_ranker( 14 | score_fn: nn.Module, 15 | optimizer: optim, 16 | estimator: str, 17 | train: SVMRankDataset, 18 | test: SVMRankDataset, 19 | batch_size: int = 32, 20 | n_epochs: int = 30, 21 | pow_true: float = 1.0, 22 | pow_used: Optional[float] = None, 23 | ) -> List: 24 | """ランキングモデルを学習するための関数. 25 | 26 | パラメータ 27 | ---------- 28 | score_fn: nn.Module 29 | スコアリング関数. 30 | 31 | optimizer: optim 32 | パラメータ最適化アルゴリズム. 33 | 34 | estimator: str 35 | スコアリング関数を学習するための目的関数を観測データから近似する推定量. 36 | 'naive', 'ips', 'ideal'のいずれかしか与えることができない. 37 | 'ideal'が与えられた場合は、真の嗜好度合いデータ（Explicit Feedback）をもとに、ランキングモデルを学習する. 38 | 39 | train: SVMRankDataset 40 | （オリジナルの）トレーニングデータ. 41 | 42 | test: SVMRankDataset 43 | （オリジナルの）テストデータ. 44 | 45 | batch_size: int, default=32 46 | バッチサイズ. 47 | 48 | n_epochs: int, default=30 49 | エポック数. 50 | 51 | pow_true: float, default=1.0 52 | ポジションバイアスの大きさを決定するパラメータ. クリックデータの生成に用いられる. 53 | pow_trueが大きいほど、ポジションバイアスの影響（真の嗜好度合いとクリックデータの乖離）が大きくなる. 54 | 55 | pow_used: Optional[float], default=None 56 | ポジションバイアスの大きさを決定するパラメータ. ランキングモデルの学習に用いられる. 57 | Noneが与えられた場合は、pow_trueと同じ値が設定される. 58 | pow_trueと違う値を与えると、ポジションバイアスの大きさを見誤ったケースにおけるランキングモデルの学習を再現できる. 59 | 60 | """ 61 | assert estimator in [ 62 | "naive", 63 | "ips", 64 | "ideal", 65 | ], f"estimator must be 'naive', 'ips', or 'ideal', but {estimator} is given" 66 | if pow_used is None: 67 | pow_used = pow_true 68 | 69 | ndcg_score_list = list() 70 | for _ in tqdm(range(n_epochs)): 71 | loader = DataLoader( 72 | train, 73 | batch_size=batch_size, 74 | shuffle=True, 75 | collate_fn=train.collate_fn(), 76 | ) 77 | score_fn.train() 78 | for batch in loader: 79 | if estimator == "naive": 80 | click, theta = convert_gamma_to_implicit( 81 | relevance=batch.relevance, pow_true=pow_true, pow_used=pow_used 82 | ) 83 | loss = listwise_loss( 84 | scores=score_fn(batch.features), click=click, num_docs=batch.n 85 | ) 86 | elif estimator == "ips": 87 | click, theta = convert_gamma_to_implicit( 88 | relevance=batch.relevance, pow_true=pow_true, pow_used=pow_used 89 | ) 90 | loss = listwise_loss( 91 | scores=score_fn(batch.features), 92 | click=click, 93 | num_docs=batch.n, 94 | pscore=theta, 95 | ) 96 | elif estimator == "ideal": 97 | gamma = convert_rel_to_gamma(relevance=batch.relevance) 98 | loss = listwise_loss( 99 | scores=score_fn(batch.features), click=gamma, num_docs=batch.n 100 | ) 101 | optimizer.zero_grad() 102 | loss.backward() 103 | optimizer.step() 104 | score_fn.eval() 105 | ndcg_score = evaluate_test_performance(score_fn=score_fn, test=test) 106 | ndcg_score_list.append(ndcg_score) 107 | 108 | return ndcg_score_list 109 | -------------------------------------------------------------------------------- /ch04/utils.py: -------------------------------------------------------------------------------- 1 | from typing import Tuple 2 | 3 | from torch import LongTensor, FloatTensor, arange, bernoulli 4 | 5 | 6 | def convert_rel_to_gamma( 7 | relevance: LongTensor, max_rel_value: int = 4, eps: float = 0.1 8 | ) -> FloatTensor: 9 | """元データの嗜好度合いラベルを[0,1]-スケールに変換する.""" 10 | gamma = 1.0 - eps 11 | gamma *= (2 ** relevance.float()) - 1 12 | gamma /= (2 ** max_rel_value) - 1 13 | gamma += eps 14 | return gamma 15 | 16 | 17 | def convert_gamma_to_implicit( 18 | relevance: LongTensor, 19 | pow_true: float = 1.0, 20 | pow_used: float = 1.0, 21 | ) -> Tuple[FloatTensor, FloatTensor]: 22 | """[0,1]-スケールの嗜好度合いをPosition-based Modelをもとにクリックデータに変換する.""" 23 | gamma = convert_rel_to_gamma(relevance=relevance) 24 | theta_true = (0.9 / arange(1, gamma.shape[1] + 1)) ** pow_true 25 | theta_used = (0.9 / arange(1, gamma.shape[1] + 1)) ** pow_used 26 | click = bernoulli(gamma * theta_true) 27 | return click, theta_used 28 | -------------------------------------------------------------------------------- /ch05/README.md: -------------------------------------------------------------------------------- 1 | ## 第5章 2 | 3 | ### PyTorchを用いた実装 4 | - [`evaluate.py`](./evaluate.py): テストデータにおけるnDCG@10を計算するための関数を実装. 5 | - [`loss.py`](./loss.py): IPS推定量に基づくリストワイズ損失関数を実装. 6 | - [`model.py`](./model.py): 多層パーセプトロンに基づくスコアリング関数を実装. 7 | - [`utils.py`](./utils.py): 半人工データを生成するための関数を実装. 8 | 9 | 10 | ### 半人工データを用いた簡易実験 11 | - [`naive-vs-ips.ipynb`](./naive-vs-ips.ipynb): 推薦枠内で定義されるKPIを扱う状況においてナイーブ推定量とIPS推定量の性能差を検証. 12 | - [`objective-misspecification.ipynb`](./objective-misspecification.ipynb): ランキングシステム構築のための方針の誤設定が性能に与える影響を検証. 13 | -------------------------------------------------------------------------------- /ch05/evaluate.py: -------------------------------------------------------------------------------- 1 | from torch import nn 2 | from torch.utils.data import DataLoader 3 | from pytorchltr.evaluation.dcg import ndcg 4 | from pytorchltr.datasets.svmrank.svmrank import SVMRankDataset 5 | 6 | from utils import ( 7 | convert_rel_to_mu, 8 | convert_rel_to_mu_zero, 9 | ) 10 | 11 | 12 | def evaluate_test_performance( 13 | score_fn: nn.Module, test: SVMRankDataset, objective: str 14 | ) -> float: 15 | """与えられたスコアリング関数のランキング性能をテストデータにおける目的変数の期待値を使ってnDCG@10で評価する.""" 16 | loader = DataLoader( 17 | test, batch_size=1024, shuffle=False, collate_fn=test.collate_fn() 18 | ) 19 | ndcg_score = 0.0 20 | for batch in loader: 21 | mu = convert_rel_to_mu(batch.relevance)[0] 22 | mu_zero = convert_rel_to_mu_zero(batch.relevance)[0] 23 | outcome = mu if objective == "via-rec" else (mu - mu_zero) 24 | ndcg_score += ndcg( 25 | score_fn(batch.features), outcome, batch.n, k=10, exp=False 26 | ).sum() 27 | return float(ndcg_score / len(test)) 28 | -------------------------------------------------------------------------------- /ch05/loss.py: -------------------------------------------------------------------------------- 1 | from typing import Optional 2 | 3 | from torch import ones_like, FloatTensor 4 | from torch.nn.functional import log_softmax 5 | 6 | 7 | def listwise_loss( 8 | scores: FloatTensor, # f_{\phi} 9 | click: FloatTensor, # C(u,i,k) 10 | conversion: FloatTensor, # R(u,i) 11 | num_docs: FloatTensor, 12 | recommend: Optional[FloatTensor] = None, # \mathbb{I}\{ K(u,i) \neq 0 \} 13 | pscore: Optional[FloatTensor] = None, # CTR(u,i) 14 | pscore_zero: Optional[FloatTensor] = None, # e(u,i,0) 15 | ) -> FloatTensor: 16 | """リストワイズ損失. 17 | 18 | パラメータ 19 | ---------- 20 | scores: FloatTensor 21 | スコアリング関数の出力. f_{\phi}. 22 | 23 | click: FloatTensor 24 | クリック発生有無データ. C(u,i,k). 25 | 26 | conversion: FloatTensor 27 | コンバージョン発生有無データ（クリック発生あとに観測される目的変数）. R(u,i). 28 | 29 | num_docs: FloatTensor 30 | クエリごとのドキュメントの数. 31 | 32 | recommend: FloatTensor, default=None. 33 | 推薦有無を表すインディケータ. \mathbb{I}\{ K(u,i) \neq 0 \} 34 | 35 | pscore: Optional[FloatTensor], default=None. 36 | 傾向スコア. CTR(u,i). 37 | Noneが与えられた場合はナイーブ推定量に基づいた損失が計算される. 38 | 39 | pscore_zero: Optional[FloatTensor], default=None. 40 | アイテムがユーザに推薦されない確率. e(u,i,0). 41 | Noneが与えられた場合はナイーブ推定量に基づいた損失が計算される. 42 | 43 | """ 44 | if recommend is None: 45 | recommend = ones_like(click) 46 | if pscore is None: 47 | pscore = ones_like(click) 48 | if pscore_zero is None: 49 | pscore_zero = ones_like(click) 50 | listwise_loss = 0 51 | for scores_, click_, conv_, num_docs_, recommend_, pscore_, pscore_zero_ in zip( 52 | scores, click, conversion, num_docs, recommend, pscore, pscore_zero 53 | ): 54 | weight = ((click_ / pscore_) - ((1 - recommend_) / pscore_zero_)) * conv_ 55 | listwise_loss -= (weight * log_softmax(scores_, dim=-1))[:num_docs_].sum() 56 | return listwise_loss / len(scores) 57 | -------------------------------------------------------------------------------- /ch05/model.py: -------------------------------------------------------------------------------- 1 | from dataclasses import dataclass 2 | from typing import Tuple 3 | 4 | from torch import nn, FloatTensor 5 | 6 | 7 | @dataclass(unsafe_hash=True) 8 | class MLPScoreFunc(nn.Module): 9 | """多層パーセプトロンによるスコアリング関数. 10 | 11 | パラメータ 12 | ---------- 13 | input_size: int 14 | 特徴量ベクトルの次元数. 15 | 16 | hidden_layer_sizes: Tuple[int, ...] 17 | 隠れ層におけるニューロンの数を定義するタプル. 18 | (10, 10)が与えられたら、ニューロン数が10個の隠れ層を持つ多層パーセプトロンが定義される. 19 | 20 | activation_func: torch.nn.functional, default=torch.nn.functional.elu 21 | 活性化関数. 22 | 23 | """ 24 | 25 | input_size: int 26 | hidden_layer_sizes: Tuple[int, ...] 27 | activation_func: nn.functional = nn.functional.elu 28 | 29 | def __post_init__(self) -> None: 30 | super().__init__() 31 | self.hidden_layers = nn.ModuleList() 32 | self.hidden_layers.append( 33 | nn.Linear(self.input_size, self.hidden_layer_sizes[0]) 34 | ) 35 | for hin, hout in zip(self.hidden_layer_sizes, self.hidden_layer_sizes[1:]): 36 | self.hidden_layers.append(nn.Linear(hin, hout)) 37 | self.output = nn.Linear(self.hidden_layer_sizes[-1], 1) 38 | 39 | def forward(self, x: FloatTensor) -> FloatTensor: 40 | h = x 41 | for layer in self.hidden_layers: 42 | h = self.activation_func(layer(h)) 43 | return self.output(h).flatten(1) # f_{\phi}, (batch_size, number_of_documents) 44 | -------------------------------------------------------------------------------- /ch05/train.py: -------------------------------------------------------------------------------- 1 | from typing import List 2 | 3 | from torch import nn, optim 4 | from torch.utils.data import DataLoader 5 | from tqdm import tqdm 6 | from pytorchltr.datasets.svmrank.svmrank import SVMRankDataset 7 | 8 | from evaluate import evaluate_test_performance 9 | from loss import listwise_loss 10 | from utils import ( 11 | convert_rel_to_mu, 12 | convert_rel_to_mu_zero, 13 | generate_click_and_recommend, 14 | ) 15 | 16 | 17 | def train_ranker( 18 | score_fn: nn.Module, 19 | optimizer: optim, 20 | estimator: str, 21 | objective: str, 22 | train: SVMRankDataset, 23 | test: SVMRankDataset, 24 | batch_size: int = 32, 25 | n_epochs: int = 30, 26 | ) -> List: 27 | """ランキングモデルを学習するための関数. 28 | 29 | パラメータ 30 | ---------- 31 | score_fn: nn.Module 32 | スコアリング関数. 33 | 34 | optimizer: optim 35 | パラメータ最適化アルゴリズム. 36 | 37 | estimator: str 38 | スコアリング関数を学習するための目的関数を近似する推定量. 39 | 'naive', 'ips-via-rec', 'ips-platform'のいずれかしか与えることができない. 40 | 41 | objective: str 42 | 推薦枠内経由('via_rec')のKPIを扱う場面か、プラットフォーム全体('platform')で定義されたKPI扱う場面かを指定. 43 | 'via_rec', 'platform'のいずれかしか与えることができない. 44 | 45 | train: SVMRankDataset 46 | （オリジナルの）トレーニングデータ. 47 | 48 | test: SVMRankDataset 49 | （オリジナルの）テストデータ. 50 | 51 | batch_size: int, default=32 52 | バッチサイズ. 53 | 54 | n_epochs: int, default=30 55 | エポック数. 56 | 57 | """ 58 | assert estimator in [ 59 | "naive", 60 | "ips-via-rec", 61 | "ips-platform", 62 | ], f"estimator must be 'naive', 'ips-via-rec', 'ips-platform', but {estimator} is given" 63 | assert objective in [ 64 | "via-rec", 65 | "platform", 66 | ], f"objective must be 'via-rec' or 'objective', but {objective} is given" 67 | 68 | ndcg_score_list = list() 69 | for _ in tqdm(range(n_epochs)): 70 | loader = DataLoader( 71 | train, 72 | batch_size=batch_size, 73 | shuffle=True, 74 | collate_fn=train.collate_fn(), 75 | ) 76 | score_fn.train() 77 | for batch in loader: 78 | conversion = convert_rel_to_mu(batch.relevance)[1] 79 | conversion_zero = convert_rel_to_mu_zero(batch.relevance)[1] 80 | click, pscore, recommend, pscore_zero = generate_click_and_recommend( 81 | batch.relevance 82 | ) 83 | conversion_obs = conversion * click + conversion_zero * (1 - recommend) 84 | scores = score_fn(batch.features) 85 | if estimator == "naive": 86 | loss = listwise_loss( 87 | scores=scores, 88 | click=click, 89 | conversion=conversion_obs, 90 | num_docs=batch.n, 91 | ) 92 | elif estimator == "ips-via-rec": 93 | loss = listwise_loss( 94 | scores=scores, 95 | click=click, 96 | conversion=conversion_obs, 97 | num_docs=batch.n, 98 | recommend=None, 99 | pscore=pscore, 100 | pscore_zero=None, 101 | ) 102 | elif estimator == "ips-platform": 103 | loss = listwise_loss( 104 | scores=scores, 105 | click=click, 106 | conversion=conversion_obs, 107 | num_docs=batch.n, 108 | recommend=recommend, 109 | pscore=pscore, 110 | pscore_zero=pscore_zero, 111 | ) 112 | optimizer.zero_grad() 113 | loss.backward() 114 | optimizer.step() 115 | score_fn.eval() 116 | ndcg_score = evaluate_test_performance( 117 | score_fn=score_fn, test=test, objective=objective 118 | ) 119 | ndcg_score_list.append(ndcg_score) 120 | 121 | return ndcg_score_list 122 | -------------------------------------------------------------------------------- /ch05/utils.py: -------------------------------------------------------------------------------- 1 | from typing import Tuple 2 | 3 | from torch import LongTensor, FloatTensor, bernoulli, arange 4 | 5 | 6 | def convert_rel_to_mu( 7 | relevance: LongTensor, 8 | max_rel_value: int = 4, 9 | eps: float = 0.1, 10 | ) -> Tuple[FloatTensor, FloatTensor]: 11 | """元データの嗜好度合いラベルを[0,1]-スケールの\mu(u,i)に変換する.""" 12 | mu = 1 - eps 13 | mu *= (2 ** relevance.float()) - 1 14 | mu /= (2 ** max_rel_value) - 1 15 | mu += eps 16 | conversion = bernoulli(mu) 17 | return mu, conversion # \mu(u,i), R(u,i) | C(u,i,\cdot)=1 18 | 19 | 20 | def convert_rel_to_mu_zero( 21 | relevance: LongTensor, 22 | max_rel_value: int = 4, 23 | eps: float = 0.1, 24 | ) -> Tuple[FloatTensor, FloatTensor]: 25 | """元データの嗜好度合いラベルを[0,1]-スケールの\mu^{(0)}(u,i)に変換する.""" 26 | mu_zero = 1 - eps 27 | mu_zero *= (2 ** relevance.float()) - 1 28 | mu_zero /= (2 ** max_rel_value) - 1 29 | mu_zero += eps 30 | mu_zero += 0.1 * (relevance == 3).float() 31 | mu_zero += 0.05 * (relevance == 2).float() 32 | mu_zero -= 0.1 * (relevance == 1).float() 33 | mu_zero -= 0.05 * (relevance == 0).float() 34 | conversion_zero = bernoulli(mu_zero) 35 | return mu_zero, conversion_zero # \mu^{(0)}(u,i), R(u,i) | C(u,i,\cdot)=0 36 | 37 | 38 | def generate_click_and_recommend( 39 | relevance: LongTensor, 40 | ) -> Tuple[FloatTensor, FloatTensor, FloatTensor, FloatTensor]: 41 | """推薦枠内でのクリック発生有無・推薦確率・推薦有無情報を適当に生成する（他の生成方法も十分あり得る）.""" 42 | num_items = relevance.shape[1] 43 | pscore = 0.9 / arange(1, num_items + 1) 44 | recommend = bernoulli(pscore) 45 | mu = convert_rel_to_mu(relevance)[0] 46 | click = bernoulli(mu) * recommend # \mu(u,i)が大きいとクリックが発生しやすいとする 47 | pscore_zero = 1.0 - pscore 48 | pscore = mu * pscore # 推薦枠内でクリックが発生する確率x推薦される確率 49 | return click, pscore, recommend, pscore_zero 50 | -------------------------------------------------------------------------------- /pyproject.toml: -------------------------------------------------------------------------------- 1 | [tool.poetry] 2 | name = "mldesignbook" 3 | version = "0.1.0" 4 | description = "" 5 | authors = ["usaito "] 6 | 7 | [tool.poetry.dependencies] 8 | python = "^3.9" 9 | torch = "^1.9.0" 10 | scikit-learn = "^0.24.2" 11 | numpy = "^1.20.3" 12 | matplotlib = "^3.4.2" 13 | seaborn = "^0.11.1" 14 | tqdm = "^4.61.1" 15 | pytorchltr = "^0.2.1" 16 | pandas = "^1.2.4" 17 | obp = "^0.4.1" 18 | jupyterlab = "^3.0.16" 19 | 20 | [tool.poetry.dev-dependencies] 21 | flake8 = "^3.9.2" 22 | black = "^21.6b0" 23 | 24 | [build-system] 25 | requires = ["poetry-core>=1.0.0"] 26 | build-backend = "poetry.core.masonry.api" 27 | --------------------------------------------------------------------------------