├── .gitignore ├── 1_Concepts.ipynb ├── 2_Fine_tuning.ipynb ├── README.md └── slide.pdf /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # C extensions 7 | *.so 8 | 9 | # Distribution / packaging 10 | .Python 11 | build/ 12 | develop-eggs/ 13 | dist/ 14 | downloads/ 15 | eggs/ 16 | .eggs/ 17 | lib/ 18 | lib64/ 19 | parts/ 20 | sdist/ 21 | var/ 22 | wheels/ 23 | share/python-wheels/ 24 | *.egg-info/ 25 | .installed.cfg 26 | *.egg 27 | MANIFEST 28 | 29 | # PyInstaller 30 | # Usually these files are written by a python script from a template 31 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 32 | *.manifest 33 | *.spec 34 | 35 | # Installer logs 36 | pip-log.txt 37 | pip-delete-this-directory.txt 38 | 39 | # Unit test / coverage reports 40 | htmlcov/ 41 | .tox/ 42 | .nox/ 43 | .coverage 44 | .coverage.* 45 | .cache 46 | nosetests.xml 47 | coverage.xml 48 | *.cover 49 | *.py,cover 50 | .hypothesis/ 51 | .pytest_cache/ 52 | cover/ 53 | 54 | # Translations 55 | *.mo 56 | *.pot 57 | 58 | # Django stuff: 59 | *.log 60 | local_settings.py 61 | db.sqlite3 62 | db.sqlite3-journal 63 | 64 | # Flask stuff: 65 | instance/ 66 | .webassets-cache 67 | 68 | # Scrapy stuff: 69 | .scrapy 70 | 71 | # Sphinx documentation 72 | docs/_build/ 73 | 74 | # PyBuilder 75 | .pybuilder/ 76 | target/ 77 | 78 | # Jupyter Notebook 79 | .ipynb_checkpoints 80 | 81 | # IPython 82 | profile_default/ 83 | ipython_config.py 84 | 85 | # pyenv 86 | # For a library or package, you might want to ignore these files since the code is 87 | # intended to run in multiple environments; otherwise, check them in: 88 | # .python-version 89 | 90 | # pipenv 91 | # According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control. 92 | # However, in case of collaboration, if having platform-specific dependencies or dependencies 93 | # having no cross-platform support, pipenv may install dependencies that don't work, or not 94 | # install all needed dependencies. 95 | #Pipfile.lock 96 | 97 | # poetry 98 | # Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control. 99 | # This is especially recommended for binary packages to ensure reproducibility, and is more 100 | # commonly ignored for libraries. 101 | # https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control 102 | #poetry.lock 103 | 104 | # pdm 105 | # Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control. 106 | #pdm.lock 107 | # pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it 108 | # in version control. 109 | # https://pdm.fming.dev/#use-with-ide 110 | .pdm.toml 111 | 112 | # PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm 113 | __pypackages__/ 114 | 115 | # Celery stuff 116 | celerybeat-schedule 117 | celerybeat.pid 118 | 119 | # SageMath parsed files 120 | *.sage.py 121 | 122 | # Environments 123 | .env 124 | .venv 125 | env/ 126 | venv/ 127 | ENV/ 128 | env.bak/ 129 | venv.bak/ 130 | 131 | # Spyder project settings 132 | .spyderproject 133 | .spyproject 134 | 135 | # Rope project settings 136 | .ropeproject 137 | 138 | # mkdocs documentation 139 | /site 140 | 141 | # mypy 142 | .mypy_cache/ 143 | .dmypy.json 144 | dmypy.json 145 | 146 | # Pyre type checker 147 | .pyre/ 148 | 149 | # pytype static type analyzer 150 | .pytype/ 151 | 152 | # Cython debug symbols 153 | cython_debug/ 154 | 155 | # PyCharm 156 | # JetBrains specific template is maintained in a separate JetBrains.gitignore that can 157 | # be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore 158 | # and can be added to the global gitignore or merged into this file. For a more nuclear 159 | # option (not recommended) you can uncomment the following to ignore the entire idea folder. 160 | #.idea/ 161 | -------------------------------------------------------------------------------- /1_Concepts.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "colab": { 6 | "provenance": [], 7 | "gpuType": "T4", 8 | "include_colab_link": true 9 | }, 10 | "kernelspec": { 11 | "name": "python3", 12 | "display_name": "Python 3" 13 | }, 14 | "language_info": { 15 | "name": "python" 16 | }, 17 | "accelerator": "GPU" 18 | }, 19 | "cells": [ 20 | { 21 | "cell_type": "markdown", 22 | "metadata": { 23 | "id": "view-in-github", 24 | "colab_type": "text" 25 | }, 26 | "source": [ 27 | "\"Open" 28 | ] 29 | }, 30 | { 31 | "cell_type": "markdown", 32 | "source": [ 33 | "# 환경설정하기" 34 | ], 35 | "metadata": { 36 | "id": "IC-upOpwnQ8c" 37 | } 38 | }, 39 | { 40 | "cell_type": "markdown", 41 | "source": [ 42 | "### 1. GPU 사용 설정\n", 43 | "메뉴 바에서 [런타임] -> [런타임 유형 변경] -> [하드웨어 가속기] 항목에서 GPU 선택\n", 44 | "\n", 45 | "※ Colab GPU 하루 최대 12시간까지 사용 가능\n" 46 | ], 47 | "metadata": { 48 | "id": "uZzOtYQiDPNU" 49 | } 50 | }, 51 | { 52 | "cell_type": "markdown", 53 | "source": [ 54 | "### 2. 한글 설정\n", 55 | "\n", 56 | "주의: 아래 apt-get 설치 코드가 현재 런타임에는 바로 반영되지 않을 수 있습니다.\n", 57 | "아래 코드 셀을 실행한 뒤 [런타임] -> [런타임 다시 시작] 을 통해 설치된 패키지가 현재 실행 환경에 반영되도록 해주세요." 58 | ], 59 | "metadata": { 60 | "id": "_Mr8NUV0-cti" 61 | } 62 | }, 63 | { 64 | "cell_type": "code", 65 | "source": [ 66 | "!sudo apt-get install -y fonts-nanum\n", 67 | "!sudo fc-cache -fv\n", 68 | "!rm ~/.cache/matplotlib -rf" 69 | ], 70 | "metadata": { 71 | "id": "4YtZhTaN-eXB" 72 | }, 73 | "execution_count": null, 74 | "outputs": [] 75 | }, 76 | { 77 | "cell_type": "code", 78 | "source": [ 79 | "import matplotlib.pyplot as plt\n", 80 | "\n", 81 | "plt.rc('font', family='NanumBarunGothic')" 82 | ], 83 | "metadata": { 84 | "id": "gGAZEy0A-i15" 85 | }, 86 | "execution_count": 2, 87 | "outputs": [] 88 | }, 89 | { 90 | "cell_type": "markdown", 91 | "source": [ 92 | "### 3. 라이브러리 설치하기" 93 | ], 94 | "metadata": { 95 | "id": "mN7IlHVCff2Q" 96 | } 97 | }, 98 | { 99 | "cell_type": "code", 100 | "source": [ 101 | "! pip install torch transformers datasets" 102 | ], 103 | "metadata": { 104 | "id": "krbSJqrrnEav" 105 | }, 106 | "execution_count": null, 107 | "outputs": [] 108 | }, 109 | { 110 | "cell_type": "markdown", 111 | "source": [ 112 | "### 4. 편집기 설정\n", 113 | "메뉴 바에서 [도구] -> [설정] -> [편집기] 에서 \"행 번호 표시\" 와 \"들여쓰기 가이드 표시\" 등 유용한 기능을 설정하세요." 114 | ], 115 | "metadata": { 116 | "id": "A2ZQk44ciRBK" 117 | } 118 | }, 119 | { 120 | "cell_type": "markdown", 121 | "source": [ 122 | "# 다국어 BERT (mBERT) 의 Attention 점수 확인하기\n" 123 | ], 124 | "metadata": { 125 | "id": "RtRSJp4srq6N" 126 | } 127 | }, 128 | { 129 | "cell_type": "markdown", 130 | "source": [ 131 | "### 모델 허브에서 모델 다운로드하기\n", 132 | "* Hugging Face model hub: https://huggingface.co/bert-base-multilingual-cased" 133 | ], 134 | "metadata": { 135 | "id": "GZ4PWnoRuei3" 136 | } 137 | }, 138 | { 139 | "cell_type": "code", 140 | "source": [ 141 | "from transformers import AutoModel\n", 142 | "\n", 143 | "model = AutoModel.from_pretrained(\"bert-base-multilingual-cased\")" 144 | ], 145 | "metadata": { 146 | "id": "RLj6pMF4F-hv" 147 | }, 148 | "execution_count": 4, 149 | "outputs": [] 150 | }, 151 | { 152 | "cell_type": "code", 153 | "source": [ 154 | "from transformers import AutoTokenizer\n", 155 | "\n", 156 | "tokenizer = AutoTokenizer.from_pretrained(\"bert-base-multilingual-cased\")" 157 | ], 158 | "metadata": { 159 | "id": "UPvmyL1-XOAC" 160 | }, 161 | "execution_count": 5, 162 | "outputs": [] 163 | }, 164 | { 165 | "cell_type": "markdown", 166 | "source": [ 167 | "### 문장 토크나이즈 해보기" 168 | ], 169 | "metadata": { 170 | "id": "3vHUkt34nifP" 171 | } 172 | }, 173 | { 174 | "cell_type": "code", 175 | "source": [ 176 | "example_text = \"가는 말이 고와야 오는 말도 곱다\"\n", 177 | "tokens = tokenizer.tokenize(example_text, add_special_tokens=True)\n", 178 | "token_ids = tokenizer.convert_tokens_to_ids(tokens)\n", 179 | "for token, token_id in zip(tokens, token_ids):\n", 180 | " print(f\"{token:>3} :: {token_id}\")" 181 | ], 182 | "metadata": { 183 | "colab": { 184 | "base_uri": "https://localhost:8080/" 185 | }, 186 | "id": "07gyXSamnhz5", 187 | "outputId": "0ffef602-b5ff-4c7f-fb2d-059bb9026d94" 188 | }, 189 | "execution_count": 6, 190 | "outputs": [ 191 | { 192 | "output_type": "stream", 193 | "name": "stdout", 194 | "text": [ 195 | "[CLS] :: 101\n", 196 | " 가 :: 8843\n", 197 | "##는 :: 11018\n", 198 | " 말 :: 9251\n", 199 | "##이 :: 10739\n", 200 | " 고 :: 8888\n", 201 | "##와 :: 12638\n", 202 | "##야 :: 21711\n", 203 | " 오 :: 9580\n", 204 | "##는 :: 11018\n", 205 | " 말 :: 9251\n", 206 | "##도 :: 12092\n", 207 | " 곱 :: 8894\n", 208 | "##다 :: 11903\n", 209 | "[SEP] :: 102\n" 210 | ] 211 | } 212 | ] 213 | }, 214 | { 215 | "cell_type": "markdown", 216 | "source": [ 217 | "### 토크나이저의 사전\n", 218 | "* 토크나이저의 사전은 학습 데이터에 등장하는 단어 통계를 학습한 결과물입니다.\n", 219 | "* 아래 예시 문장을 토크나이즈하여 학습 데이터에서 발견되지 않은 단어와 더 많이 발견된 단어를 확인해보세요.\n", 220 | " * 많이 발견된 단어: '##르고', '##기가', '##에는'\n", 221 | " * 발견되지 않은 단어: '닳도록' --> '[UNK]'" 222 | ], 223 | "metadata": { 224 | "id": "Uq7J5P3cggx0" 225 | } 226 | }, 227 | { 228 | "cell_type": "code", 229 | "source": [ 230 | "example_text = \"동해물과 백두산이 마르고 닳도록\"\n", 231 | "# 토크나이저 호출해보기\n", 232 | "tokens = tokenizer.tokenize(example_text, add_special_tokens=True)\n", 233 | "print(tokens)\n", 234 | "\n", 235 | "example_text = \"나 보기가 역겨워 가실 때에는\"\n", 236 | "# 토크나이저 호출해보기\n", 237 | "tokens = tokenizer.tokenize(example_text, add_special_tokens=True)\n", 238 | "print(tokens)" 239 | ], 240 | "metadata": { 241 | "id": "DNDR1uy6ga6l", 242 | "colab": { 243 | "base_uri": "https://localhost:8080/" 244 | }, 245 | "outputId": "ed513f50-e020-44bb-d9d6-0a93fed126c4" 246 | }, 247 | "execution_count": 7, 248 | "outputs": [ 249 | { 250 | "output_type": "stream", 251 | "name": "stdout", 252 | "text": [ 253 | "['[CLS]', '동', '##해', '##물', '##과', '백', '##두', '##산', '##이', '마', '##르고', '[UNK]', '[SEP]']\n", 254 | "['[CLS]', '나', '보', '##기가', '역', '##겨', '##워', '가', '##실', '때', '##에는', '[SEP]']\n" 255 | ] 256 | } 257 | ] 258 | }, 259 | { 260 | "cell_type": "markdown", 261 | "source": [ 262 | "### 모델 구조 확인\n", 263 | "* BERT 모델 구조 사용\n", 264 | "* 모델에 입력할수 있는 토큰 최대 길이: 512\n", 265 | " * `model.config.max_position_embeddings`\n", 266 | "* Embeddings Layer + 12 x Encoder Layer + Pooler Layer\n", 267 | " * `(embeddings)`: Embeddings Layer: 119,547 -> 768\n", 268 | " * `(encoder)`: 12 x Encoder Layer: 768 -> 768\n", 269 | " * `(pooler)`: Pooler Layer: 768 -> 768" 270 | ], 271 | "metadata": { 272 | "id": "mxJH3v2hLAIq" 273 | } 274 | }, 275 | { 276 | "cell_type": "code", 277 | "source": [ 278 | "print(repr(model.config))\n", 279 | "print(repr(model))" 280 | ], 281 | "metadata": { 282 | "colab": { 283 | "base_uri": "https://localhost:8080/" 284 | }, 285 | "id": "3W5E_AGzK2TA", 286 | "outputId": "c7f05916-a12e-4a8b-f1d9-653ec1ee59a4" 287 | }, 288 | "execution_count": 8, 289 | "outputs": [ 290 | { 291 | "output_type": "stream", 292 | "name": "stdout", 293 | "text": [ 294 | "BertConfig {\n", 295 | " \"_name_or_path\": \"bert-base-multilingual-cased\",\n", 296 | " \"architectures\": [\n", 297 | " \"BertForMaskedLM\"\n", 298 | " ],\n", 299 | " \"attention_probs_dropout_prob\": 0.1,\n", 300 | " \"classifier_dropout\": null,\n", 301 | " \"directionality\": \"bidi\",\n", 302 | " \"hidden_act\": \"gelu\",\n", 303 | " \"hidden_dropout_prob\": 0.1,\n", 304 | " \"hidden_size\": 768,\n", 305 | " \"initializer_range\": 0.02,\n", 306 | " \"intermediate_size\": 3072,\n", 307 | " \"layer_norm_eps\": 1e-12,\n", 308 | " \"max_position_embeddings\": 512,\n", 309 | " \"model_type\": \"bert\",\n", 310 | " \"num_attention_heads\": 12,\n", 311 | " \"num_hidden_layers\": 12,\n", 312 | " \"pad_token_id\": 0,\n", 313 | " \"pooler_fc_size\": 768,\n", 314 | " \"pooler_num_attention_heads\": 12,\n", 315 | " \"pooler_num_fc_layers\": 3,\n", 316 | " \"pooler_size_per_head\": 128,\n", 317 | " \"pooler_type\": \"first_token_transform\",\n", 318 | " \"position_embedding_type\": \"absolute\",\n", 319 | " \"transformers_version\": \"4.32.0\",\n", 320 | " \"type_vocab_size\": 2,\n", 321 | " \"use_cache\": true,\n", 322 | " \"vocab_size\": 119547\n", 323 | "}\n", 324 | "\n", 325 | "BertModel(\n", 326 | " (embeddings): BertEmbeddings(\n", 327 | " (word_embeddings): Embedding(119547, 768, padding_idx=0)\n", 328 | " (position_embeddings): Embedding(512, 768)\n", 329 | " (token_type_embeddings): Embedding(2, 768)\n", 330 | " (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n", 331 | " (dropout): Dropout(p=0.1, inplace=False)\n", 332 | " )\n", 333 | " (encoder): BertEncoder(\n", 334 | " (layer): ModuleList(\n", 335 | " (0-11): 12 x BertLayer(\n", 336 | " (attention): BertAttention(\n", 337 | " (self): BertSelfAttention(\n", 338 | " (query): Linear(in_features=768, out_features=768, bias=True)\n", 339 | " (key): Linear(in_features=768, out_features=768, bias=True)\n", 340 | " (value): Linear(in_features=768, out_features=768, bias=True)\n", 341 | " (dropout): Dropout(p=0.1, inplace=False)\n", 342 | " )\n", 343 | " (output): BertSelfOutput(\n", 344 | " (dense): Linear(in_features=768, out_features=768, bias=True)\n", 345 | " (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n", 346 | " (dropout): Dropout(p=0.1, inplace=False)\n", 347 | " )\n", 348 | " )\n", 349 | " (intermediate): BertIntermediate(\n", 350 | " (dense): Linear(in_features=768, out_features=3072, bias=True)\n", 351 | " (intermediate_act_fn): GELUActivation()\n", 352 | " )\n", 353 | " (output): BertOutput(\n", 354 | " (dense): Linear(in_features=3072, out_features=768, bias=True)\n", 355 | " (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n", 356 | " (dropout): Dropout(p=0.1, inplace=False)\n", 357 | " )\n", 358 | " )\n", 359 | " )\n", 360 | " )\n", 361 | " (pooler): BertPooler(\n", 362 | " (dense): Linear(in_features=768, out_features=768, bias=True)\n", 363 | " (activation): Tanh()\n", 364 | " )\n", 365 | ")\n" 366 | ] 367 | } 368 | ] 369 | }, 370 | { 371 | "cell_type": "markdown", 372 | "source": [ 373 | "### 모델 크기 계산해서 `pytorch_model.bin` 크기와 비교해보기\n", 374 | "\n", 375 | "* 파라미터 개수 구하기\n", 376 | "* 파라미터 크기 확인\n", 377 | " * `torch.float16` -> 개당 2 byte\n", 378 | " * `torch.float32` -> 개당 4 byte\n", 379 | "* 파라미터 개수 x 파라미터 크기 = 모델의 최소 크기\n", 380 | " * mBERT: 178M * 4byte = 711 MB\n", 381 | " * `pytorch_model.bin` 크기와 거의 동일\n", 382 | "\n", 383 | "(참고) mixed precision 사용하는 경우도 있음" 384 | ], 385 | "metadata": { 386 | "id": "-fJmpuw8GWBj" 387 | } 388 | }, 389 | { 390 | "cell_type": "code", 391 | "source": [ 392 | "type_params = set([t.dtype for t in model.parameters()])\n", 393 | "type_params" 394 | ], 395 | "metadata": { 396 | "colab": { 397 | "base_uri": "https://localhost:8080/" 398 | }, 399 | "id": "2NkaD2M0H00E", 400 | "outputId": "38001fae-a856-4475-9ac5-5e0cd655bf03" 401 | }, 402 | "execution_count": 9, 403 | "outputs": [ 404 | { 405 | "output_type": "execute_result", 406 | "data": { 407 | "text/plain": [ 408 | "{torch.float32}" 409 | ] 410 | }, 411 | "metadata": {}, 412 | "execution_count": 9 413 | } 414 | ] 415 | }, 416 | { 417 | "cell_type": "code", 418 | "source": [ 419 | "num_params = sum(t.numel() for t in model.parameters())\n", 420 | "print(f\"number of params : {num_params:,}\")\n", 421 | "print(f\"total param size : {num_params * 4:,}\")" 422 | ], 423 | "metadata": { 424 | "colab": { 425 | "base_uri": "https://localhost:8080/" 426 | }, 427 | "id": "_rwpGMrzHVG-", 428 | "outputId": "0a31e5db-b05d-4212-d6cc-5dae8c71e894" 429 | }, 430 | "execution_count": 10, 431 | "outputs": [ 432 | { 433 | "output_type": "stream", 434 | "name": "stdout", 435 | "text": [ 436 | "number of params : 177,853,440\n", 437 | "total param size : 711,413,760\n" 438 | ] 439 | } 440 | ] 441 | }, 442 | { 443 | "cell_type": "markdown", 444 | "source": [ 445 | "### mBERT 를 이용한 텍스트 임베딩 생성\n" 446 | ], 447 | "metadata": { 448 | "id": "OsU_EXHYXB0s" 449 | } 450 | }, 451 | { 452 | "cell_type": "code", 453 | "source": [ 454 | "import torch\n", 455 | "\n", 456 | "if torch.cuda.is_available():\n", 457 | " device = torch.device(\"cuda\")\n", 458 | "else:\n", 459 | " device = torch.device(\"cpu\")\n", 460 | "\n", 461 | "# 모델과 입력 GPU 메모리에 올리기\n", 462 | "model = model.to(device)\n", 463 | "\n", 464 | "example_text = \"가는 말이 고와야 오는 말도 곱다\"\n", 465 | "tokenized_inputs = tokenizer(example_text, return_tensors='pt')\n", 466 | "\n", 467 | "input_ids = tokenized_inputs.input_ids.to(device)\n", 468 | "attention_mask = tokenized_inputs.attention_mask.to(device)\n", 469 | "\n", 470 | "# 모델 임베딩\n", 471 | "model_out = model(input_ids=input_ids, attention_mask=attention_mask, output_attentions=True)" 472 | ], 473 | "metadata": { 474 | "id": "W4iCFTZ-Bt1H" 475 | }, 476 | "execution_count": 11, 477 | "outputs": [] 478 | }, 479 | { 480 | "cell_type": "markdown", 481 | "source": [ 482 | "### mBERT 모델의 Attention 점수 확인해보기\n", 483 | "\n", 484 | "* 각 층별 Attention 행렬 크기: `batch_size * num_heads * sequence_length * sequence_length`\n", 485 | "([참고](https://huggingface.co/docs/transformers/main_classes/output#transformers.modeling_outputs.BaseModelOutput.attentions))\n", 486 | "* multi-head attention: attention 연산을 병렬화하는 기법\n", 487 | "* Transformers 의 `ModelOutput` 출력에 달린 `attentions` 는 이미 multi-head attention 에 대한 softmax, weighted average 연산을 마친 값" 488 | ], 489 | "metadata": { 490 | "id": "Qad3v5J2XedR" 491 | } 492 | }, 493 | { 494 | "cell_type": "code", 495 | "source": [ 496 | "attns = model_out.attentions\n", 497 | "len(attns), attns[0].shape" 498 | ], 499 | "metadata": { 500 | "colab": { 501 | "base_uri": "https://localhost:8080/" 502 | }, 503 | "id": "dvehsz0gXkD5", 504 | "outputId": "6d20bf01-65db-403c-92a0-e2c795fab3ee" 505 | }, 506 | "execution_count": 12, 507 | "outputs": [ 508 | { 509 | "output_type": "execute_result", 510 | "data": { 511 | "text/plain": [ 512 | "(12, torch.Size([1, 12, 15, 15]))" 513 | ] 514 | }, 515 | "metadata": {}, 516 | "execution_count": 12 517 | } 518 | ] 519 | }, 520 | { 521 | "cell_type": "code", 522 | "source": [ 523 | "import matplotlib.pyplot as plt\n", 524 | "import matplotlib.ticker as ticker\n", 525 | "\n", 526 | "\n", 527 | "def plot_attention(attention, tokens):\n", 528 | " fig = plt.figure(figsize=(10, 10))\n", 529 | " ax = fig.add_subplot(1, 1, 1)\n", 530 | "\n", 531 | " attention = attention[:len(tokens), :len(tokens)]\n", 532 | "\n", 533 | " ax.matshow(attention, cmap='viridis', vmin=0.0)\n", 534 | "\n", 535 | " fontdict = {'fontsize': 14}\n", 536 | "\n", 537 | " ax.set_xticklabels([''] + tokens, fontdict=fontdict, rotation=90)\n", 538 | " ax.set_yticklabels([''] + tokens, fontdict=fontdict)\n", 539 | "\n", 540 | " ax.xaxis.set_major_locator(ticker.MultipleLocator(1))\n", 541 | " ax.yaxis.set_major_locator(ticker.MultipleLocator(1))\n", 542 | "\n", 543 | " ax.set_xlabel('Input text')\n", 544 | " ax.set_ylabel('Output text')\n", 545 | " plt.suptitle('Attention weights')" 546 | ], 547 | "metadata": { 548 | "id": "N6Hrtxlxawy7" 549 | }, 550 | "execution_count": 13, 551 | "outputs": [] 552 | }, 553 | { 554 | "cell_type": "markdown", 555 | "source": [ 556 | "#### 마지막 레이어의 attention hitmap 출력하기" 557 | ], 558 | "metadata": { 559 | "id": "akG2zTTRCTAA" 560 | } 561 | }, 562 | { 563 | "cell_type": "code", 564 | "source": [ 565 | "layer_idx = -1\n", 566 | "sum_heads = attns[layer_idx][0, 0, :, :]\n", 567 | "for head_idx in range(1, model.config.num_attention_heads):\n", 568 | " sum_heads = sum_heads + attns[layer_idx][0, head_idx, :, :]\n", 569 | "plot_attention(sum_heads.cpu().detach().numpy(), tokenizer.tokenize(example_text, add_special_tokens=True))" 570 | ], 571 | "metadata": { 572 | "id": "3-wHxrEqbfni", 573 | "colab": { 574 | "base_uri": "https://localhost:8080/", 575 | "height": 1000 576 | }, 577 | "outputId": "7d305076-5de8-4f4d-edc8-5bf844f8d8de" 578 | }, 579 | "execution_count": 14, 580 | "outputs": [ 581 | { 582 | "output_type": "stream", 583 | "name": "stderr", 584 | "text": [ 585 | ":15: UserWarning: FixedFormatter should only be used together with FixedLocator\n", 586 | " ax.set_xticklabels([''] + tokens, fontdict=fontdict, rotation=90)\n", 587 | ":16: UserWarning: FixedFormatter should only be used together with FixedLocator\n", 588 | " ax.set_yticklabels([''] + tokens, fontdict=fontdict)\n" 589 | ] 590 | }, 591 | { 592 | "output_type": "display_data", 593 | "data": { 594 | "text/plain": [ 595 | "
" 596 | ], 597 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAA2cAAAOSCAYAAADwFKjPAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAACHAklEQVR4nOzdf5xM9eLH8ffZ33aNRbHW/vQrIalsCksU+kH5UYlS1E2lbj+ubkr3JkpX2/cWin7eUqhLRCX9DrUU1s8U+bW7lmQp2m2W3TXmfP/QzjV2l11m95yZfT0fj3k8nHM+c877OGvMe8+ZM4ZpmqYAAAAAAJYKsjoAAAAAAIByBgAAAAC2QDkDAAAAABugnAEAAACADVDOAAAAAMAGKGcAAAAAYAOUMwAAAACwAcoZAAAAANgA5QwAAAAAbIByBgCoMbp27aru3btbHeO07NmzR7GxsXr88ccr9bwlS5bIMAxlZ2dXTTAAwGmjnAEATkmnTp1kGIZeffVVq6NU2M8//6xdu3ZZHeO0HDx4UL/99pt2795tdRQAgI9RzgAAlbZr1y4tX75cvXr10ty5c8scM3bsWCUnJ1d4vi+Vd5bo+++/17p166p021WtadOm2rNnj15++eUq20ZycrLGjh1bZesHAJSNcgYAqLS5c+fqnHPO0QMPPKDFixfrt99+szpShURFRSkqKsrqGKetfv36Cg4OtjoGAMDHKGcAgEqbO3eu+vTpo8suu0yRkZGaP3++Z1l2drYMw9C4ceO0Y8cOGYYhwzD05ptvljl/yZIlnue+/fbbat++vSIiItSgQQMNGzZM+/bt8ywfNmyYLr74Yv3www/q1auXIiMj1ahRIz366KMyTVPS0bM+JZ8ra9KkiQzD8JwF6tatm1JTU7325eOPP1anTp0UGRmpevXqqX///tq8ebPXGMMw9PLLL+vVV19Vy5YtFR4ernPPPVeLFi0q8+9n586dMgxDs2fP9prfunVrde7c2Wve5MmTVadOHR05ckSSlJeXp7/97W+Kj49XeHi4WrZsqSlTppTK889//tMznZWVpX79+snhcCg6Olo33HCD5+zhsX+/klRQUKD77rtPDRs2VGRkpK644grt3LlT0tGzmoZhaMeOHRo3bpwMw/Cc5Tx8+LAefvhhJSUlKTIyUq1atdI//vEPFRQUlPl3AACoPMoZAKBSfv75Z3377be6+uqrFRYWpt69e2vOnDme5XFxcdqwYYNGjBihxo0ba8OGDdqwYYP69+9f5vwLL7xQkvR///d/GjZsmHr16qWvvvpKU6dO1TfffKOePXvq8OHDnvXn5OTo0ksv1UUXXaQlS5bo4Ycf1jPPPOPJ8Pnnn+uNN96QJH322WfasGGD7r777jL35d1331WfPn3Utm1bffzxx5o+fbp+++03dezYUdu3b/caO3XqVD311FN66qmn9NVXX6lJkya6/vrrVVhYWGq9CQkJSkpK0sqVKz3zsrKy9NNPP2n58uVeZxozMjLUqVMnBQcHq7CwUJdddpnmz5+vCRMmaMmSJRo8eLD+9re/6ZlnnilzH5xOp7p3765NmzbpnXfe0aeffqqQkBANHjy4zPG33HKLfvrpJ82ZM0fvvfeetm/f7vn7ufvuu7VhwwY1btxYI0aM0IYNG/T5559LktLS0vTCCy/o6aef1tKlSzV69Gh99NFHWrt2bZnbAQCcAhMAgEqYPHmy2aBBA/PIkSOmaZrmu+++a4aEhJi//vqr17jHH3/cTEpKKvX8subv3LnTDA0NNSdMmOA1PyMjw5Rkzp071zRN0xw6dKgpyXz++ee9xl1xxRVm//79PdOLFy82JZlZWVle4y655BKzc+fOpmmapsvlMhs3bmzefPPNXmMKCwvNJk2amDfccINnniTT4XCYu3bt8sz75ZdfTMMwzA8++KDUPpqmad58881mamqqZ/r55583r7zySjMuLs6cMWOGZ/5ZZ51ljh8/3jRN03z66afNyMhIc9u2bV7ruueee8x69eqZhw8f9uT5xz/+YZqmaT733HOmJHPt2rVez+nXr58pyVy8eLHX38nFF19sulwuz7h33nnHDAoKMg8cOOCZl5SUZD7++ONe6+vdu7d53nnnec0rLi428/Pzy9x/AEDlceYMAFApc+fO1VVXXaWgoKP/hVx55ZUKCQnR+++/f8rrfO+992SaZqkzXCkpKapdu7aWL1/umVe3bt1S41q3bq0dO3ZUapurV6/W7t27NWjQIK/54eHhuvbaa7Vw4UK53W7P/GHDhikuLs4z3ahRI9WrV6/c7Xbp0kVr1qzxXK64cOFC9enTR3369NHChQslHb2EcevWrerSpYskadasWerdu7eaNWvmta7u3bvrwIEDpS63lKQvvvhCbdq00Xnnnec1f8SIEWXmeuihh7w+r9amTRu53W7PpY3l6dChg77//nu9+OKLnr+X0NBQORyOEz4PAFBxlDMAQIX98ssvWrZsmXr16qXCwkIVFhYqJCREl156qdeljZW1efNmuVwu1a9fXyEhIV4Pp9PpdRng2WefXepmGA6HQ/v376/UNjMzMyWpzDtHJicny+l0en3erU2bNqXGnWi7Xbt21cGDB/XDDz+ooKBAS5YsUe/evXX11Vfrs88+k8vl0qpVqxQaGqoOHTp4/h7mzp1b6u9g4MCBklTmjVeys7PL3IfjC155+1GnTh1JOunf38MPP6ybbrpJ99xzj84++2y98cYbcrlcJ3wOAKByQqwOAADwH/PmzZPb7dZNN91UallISIj279+v+vXrV3q9pmkqOjpaS5cuLXN5vXr1PH8ODw8vdx2VYRiGJKm4uLjUsrLmVXa7LVu2VMOGDbVy5UplZ2erRYsWSkpKUkxMjIqKivTtt98qIyNDKSkpioiI8Kzr5ptv1kMPPVTmOps0aVLm/LKylezf6e7Hsc+bPn267rrrLqWlpekvf/mLXn31VX3++eeeggcAOD2UMwBAhc2ZM0eXX355qe/AOnz4sHr06KH3339ft912W6XX27RpU+Xl5SkxMbHa3ug3bdpUkrRjx45SlwTu2LFDtWvXVoMGDU5rG126dNHKlStlGIZ69+4tSYqIiFCPHj20cOFCbdu2zXNJY0mmP/74Q+ecc06Ft5GQkFDmpZUnu0zxVHXq1EkffPCBPvvsM1111VV64YUX9I9//KNKtgUANQ2XNQIAKiQ3N1fp6ekaPHiwLr74Yq9Hly5d1K1bN69LG0NCQrw+s3Wi+f369VNQUJCmTp1aanxxcXGZd0Q8kZCQo797LGv7JS644ALFx8dr5syZXvOLioo0e/ZsXX311Z7P1Z2qrl27auXKlZ7Pm5W4+uqr9dFHHykjI8OrnA0YMECffPKJsrKySq0rLy+v3G2sWbNGmzZt8po/ffr0U85d1jE6/pLKXr16KSoqSr/88sspbwcA4I0zZwCACpk3b57XGaDjXX311Ro5cqQOHDigevXqKTk5Wbt379abb76psLAwBQcH64Ybbih3/iOPPKLHHntM+/bt09VXX63g4GCtX79eU6ZM0XvvvVeps0lJSUmSpJdfflkdO3ZUZmamHnzwQa8xwcHBmjx5sq677jrdeeedGjx4sJxOp5555hkVFxdr/Pjxp/6X9acuXbro/vvvV/369dWxY0fP/N69e+uOO+6QYRhe33v20EMPae7cuerSpYvGjh2rtm3bat++fVq0aJEyMjKUnp5eahv33HOPJk2apD59+uiZZ55RgwYNNGfOHH388ceSyr+88USSk5P16aefqn379tq2bZsGDBigNm3a6C9/+Yv69++vqKgoTZ8+XQcPHix1QxUAwKnjzBkAoELmzJmjjh076swzzyxz+dVXX63Dhw977tp43XXX6eqrr9a9996re++9VwcPHjzh/KeeekqvvfaalixZoiuvvFL9+vXTG2+8oVtvvVUtW7asVNaEhAQ9/vjjmjZtmoYMGaKtW7eWOW7AgAFasGCB1q9fr6uuukpDhgxRvXr19O2333ouezwd7dq1U3R0tK644gqvm5jExsYqJSVFbdq0Ud26dT3z69Spo2XLlqlfv34aO3asunbtqjvvvFM//vijxo0bV+Y26tatq6+//lpnn322hg0bpquuukqZmZmeM2eRkZGVzv3kk0+qoKBAN9xwg6ZNm6aQkBC98sor+v777zVo0CBddtllWrVqld57771SX+oNADh1hlnZT1ADAADbW7Jkibp3767du3crNjbW6jgAgArgzBkAAH6svN+xzpkzR2eddRbFDAD8CJ85AwDAj+3evVt9+vTRiBEjdO655yo/P18fffSRXnnlFb3zzjtWxwMAVALlDAAAP1avXj2lpqYqLS1NP//8s8LCwtSuXTvNmzdP11xzjdXxAACVwGfOAAAAAMAG+MwZAAAAANgA5QwAAAAAbIByBgAAAAA2QDkDAAAAABugnAEAAACADVDOAAAAAMAGKGcAAAAAYAOUMwAAAACwAcoZAAAAANgA5QwAAAAAbIByBgAAAAA2QDkDAAAAABugnAEAAACADVDOAAAAAMAGKGcAAAAAYAOUMwAAAACwAcoZAAAAANgA5QwAAAAAbIByBgAAAAA2QDkDAAAAABugnAEAAACADVDOAAAAAMAGKGcAAAAAYAOUMwAAAACwAcoZAAAAANgA5QwAAAAAbIByBgAAAAA2QDkDAAAAABugnAEAAACADYRYHQCn5rbbbvPJev7xj3+oWbNmPlkXAAAAYEdNmzY97XUYhqE5c+boggsu8EGicrZhmqZZZWtHlQkKOv2TnoZhKD09XZ06dfJBIgAAAMCegoKC1LBhQ9WqVeuUnm+apnbu3Fnl7505c+bH3njjDXXv3v2UnltQUKC2bdv6OBEAAABgT2+88YauuuqqU3qu0+lUnTp1fJyoNMqZH2vYsKGSkpJO6bkFBQXipCkAu9m/f79q166tsLAwq6MAAP505MgR5ebmqmHDhgoJ8c/6EB0drdDQ0FN+flBQkKKjo6t8/7khiJ9au3atunTpcsrPj4yM1Nq1a3Xeeef5LhTKlJWVpcjISH399ddWRwFsr0GDBnrzzTetjnHK9uzZo4YNG+qzzz7zmt+wYUNNmzbNolQoC6/N/uWzzz7TpZdeanWMGuuHH35QQkKCVq5caXWUU3bgwAH17NnzlJ8fGRmpAwcOqEOHDj5MVZp/Vl+oXbt2p/V8wzBOex2oGLfbrcLCQh05csTqKIDt+fsZ/SNHjujXX39VUVGR1/xff/1Vhw4dsigVysJrs3/Zs2cPRdpi/v76fKxVq1Zpzpw52rt3r2JjY9W/f39deOGFVseSRDnza263W9nZ2WrQoIEcDkep5Tt37tS4ceO0atUqhYaGqlevXho1apSio6MtSFuzPPfcc7ryyivVqlWrE45btmyZ3nvvPT333HPVlKzmmD17tm688cbTWodhGHK5XD5KBOloSdm4ceMJx2zdulXffPPNCcd07drVl7FOWckb/BIHDx6UJBUVFXn+XPKG5vDhw555x4uMjKzipJB4bbYTl8ul/Px81a9fv9SyX375RVu3brXNv/Oa4vjXs2MdOnRIhmGosLCw3NexEnZ/PRs/frzGjh0rt9vtmZeWlqZ77rlHzz//vIXJjqKc+bE33nhDd999t1avXl3q5h5bt25Vx44ddeDAAc8bg9WrV+uDDz7QihUrFBUVZUXkGuHIkSN66KGHdOaZZ570DcC6des0efJk3gBUgYSEBPXv31+GYVgdBcf46quvNHjw4BMel2effbbcfxOmacowDNuc7ZgzZ06ZvwQYNGhQqXkjR47UyJEjS83nlwDVg9dme1m4cKEGDBignTt3qnHjxl7L5s2bp/vuu882/85rivJez0qYpnnSywLt/nq2dOlSjRkzRj179tTDDz+shIQE/fzzz5oyZYqmTp2qtm3bavjw4ZZmpJz5sblz56p///5l3nVx0KBB2r9/v7p166ZXXnlFDRs21AcffKARI0Zo4sSJ+uc//2lB4pojkE79+6tOnTrxNRE2dNFFFwXUZ6+aNGmim2++mV8C+Alem4Hy1YTXs5dfflktW7bUxx9/rODgYElSixYt1K1bN11xxRX697//TTnDqdu8eXOZJeu///2v1q5dqyZNmujDDz9U7dq1JUm33HKLVqxYoXnz5lHOAFgiOTlZycnJlXrOX//6VyUkJOjhhx+umlCnoUOHDlX+4XAAqA6VfT3Lz89XZGSkX929cdmyZbrjjjs8xexYDzzwgHr37q09e/aoUaNGFqQ7irs1+rE9e/bozDPPLDX/6aeflmEYGjdunKeYlbj44ou1devW6ooIACe1b98+/fjjj3I6nWUuz8vL09KlS6s5FQDgRBISEjRlyhSrY1TKL7/8oubNm5e57JxzzpFpmvr555+rOZU3/6m6KCUmJka5uble8xYtWqQNGzYoPj5egwcPLvWcqKioUncRA2qCH3/8UW+88YbS09O1a9cuFRcXq379+mrXrp369++vG264oczfpKHqbNu2TXfeeaeWLFkiSQoJCdGdd96pf/3rX16/WGrUqNFJbyJiJ99++62WLl1a6ufsiiuu4IZMQCW88MILnktRV6xYYXGamiMnJ0dLly7VoEGDFBRU/nmcVq1aac2aNdWY7PQVFxeXe9+FevXqSZLl75MpZ36sXbt2mj17tu644w5JR3+YHn74YRmGoYceeqjMN5pZWVmeHz6gphgzZowmTJigI0eOqG7duoqPj1dkZKR2796t+fPna968eXrmmWe0YMECJSQkWB23Rti9e7dSU1O1f/9+9evXT02bNtX333+vKVOmaNu2bVq4cKHncw9169bV/v37LU58ct9++62GDx+un376qczPNkVERGjkyJF64oknTviGB8DRzwfef//9XvMC+bNQdvLdd9/p5ptvVr9+/U5458Wzzz5bP/30UzUm8421a9eWeSlmyZ0qV65cWeaVHL169arybBLlzK/de++96tWrl7p27ap27drpiy++0JYtW9S2bVvdeeedZT5n1qxZZd5ABAhU06ZN0/jx43XNNdfoiSee0Lnnnuu1fO/evXrrrbf0xBNPqG/fvlq9ejVvAKrBE088oT/++EPfffed2rdv75k/e/ZsDR48WJ9++qmuvPJKSZLD4dCBAwesilohq1evVo8ePeRwODR27Fhdcsklio2NVWRkpPbs2aO1a9fqzTff1IQJE5Sbm6vXXnvN6siArRmGoY8//tgz/fnnn2vSpEnWBUIpSUlJ+vzzz62OUWljxow54fIHH3zQa7q67xJMOfNjPXr00FNPPaWxY8d6Po/RpUsXzZgxQ2FhYaXG/9///Z9Wr16tV199tbqjApaZPHmyunTpovfff7/M5Q0bNtRDDz2k5ORkDRo0SEuWLFH37t2rN2QNtGDBAt12221exUySbrjhBk2ZMkUPP/yw5syZI0nasmWLCgoKrIhZYf/4xz/UsGFDrVixQjExMV7L4uLi1L59e91+++36+9//rokTJ+pvf/ubWrdubVFawD9cfvnlnj/v2bPHwiQoS0xMjPbt22d1jEp5/PHHrY5wUpQzPzd69Gjddddd2rJli+Li4hQfH1/muCNHjqhRo0Z66623dNNNN1VzSsA6mzdv1tixY0867pprrpFpmtqyZQvlrBr89ttv5d61MTk5WW+//bZ++OEHz7ySLz+NiIiopoSVs2zZMv39738vVcyO99hjj+m5557Td999RzkDxKWK/iwyMlJut1sHDx60/RdPl6CcoVrUq1dPF1100QnHBAcH6+abb9b+/fu1dOlSde3atZrSBb6XXnpJWVlZnumS099l2bx5sxYuXOiZ5g50Va9Ro0batGnTScf9+OOPknTSN9fwjeTkZC1fvrzUfLfbrVWrVunOO+/U008/LenoWbahQ4faupyFhYXp4MGDJx1X8jmGsq5ugG/x2mx/pmnq7rvvVq1atbzmc1dp/xAaGirp6D0P/KWc5efnq3bt2pX63K/b7ZbT6VSdOnWqMNn/UM5qmO+++07XXHNNtV03WxO88847WrZsWYXGrlq1Sn//+9+95vFbw6p1880361//+pfOP/983XvvvWW+IK9Zs0Y33XSTYmJiqu0DvzXdoEGD9OSTT2rq1Km65557JEkul0sPPvigtmzZotdee81zZ0OHw+FZble9evXSa6+9psGDB+u8884rc0xxcbHuv/9+hYaGcna2GvDa7B8++OCDMufz92+9ih4Df/py93r16mnBggW66qqrvOYfPnxYjz/+uEaMGFHqxmCffPJJtb53ppwBp2nGjBllfh6mrEtML7/8cqWnp1dHLPzp0Ucf1bJly/S3v/1NaWlp6tKlixo3bqxatWopNzdXa9as0ffff6/IyEgtWLDAb3775+8efvhhffjhh7rvvvv01FNPKTExUdu2bdOBAwd0//33KzU1tdRz3G63BUkr5umnn9aiRYt00UUXqX///rrkkku8fs7Wrl2rWbNmKTc3V08++WS5l6DDd3httrdrrrlGhw4dsjoGymGapi644IITFrT8/PxqTOQb5RXJ4uJipaWlqU+fPpbftZlyBpym8j43c6ySF7czzzyzzC8OR9WJiIjQF198oZdfflkvvfSS5yYTJaKjozVs2DA99thjFTqW8I1atWrpm2++0ZNPPql3331XGzZsUNOmTfXcc8/plltusTpepSUlJWn58uW6//77NXfuXL377rteb2pM01RiYqLeeOMNDR061MKkNQevzfZmGIbCw8OtjoEyREVFKSkpyXNr+fKEhoYqMTExYL4j1C5nAClnQDWwyz/4miooKEh333237r77bv3+++/atWuXDh8+rHr16lHILFS7dm2lpaUpLS3tpGP94d9QkyZN9OGHH2rPnj367rvv9PPPP+vw4cOqW7eu2rZtq/bt23Opls34w88VjqpXrx430akmffr0UZ8+fayOUWNRzoAq8N133yklJUWhoaFq1qyZrS/Hqmk2bdrkOTawj2P/zRzvqquu0r59+3TGGWdYkKzyGjVqpEaNGqlPnz78nNkMr832daLXAOnoZZDXXHNNNaeCdPJjA9+q+K1KAFTIkSNHlJqaqg0bNnjmHT58WP/+97/97vtAAg3Hxp5OdlxCQ0P9pphJ/JzZFcfFvjg29sWxqX6cOfNjjz76aKWfs3379ipIgieeeEKJiYlKSkpS48aNS92yubi4WA8//LA6deqkBg0aWJi05uHY2FOgHZdA259AwXGxL46NfXFsrEU582Ml3wFUWXzmwfeeeOIJud1uz9+tYRjq16+f2rRpo5YtWyo5OVmmaWrv3r1yu92V+n4NnB6OjT0F2nEJtP0JFBwX++LY2FegH5tJkyZp7ty5XvNcLpcMw9C//vUvNWzY0GvZrl27qjOeDJNPw/qtr7/++pSfe8kll/gwie8cOnRI8+fPV9euXf3qVtMul0s7d+5UTk6OsrKydNttt2nQoEHKy8vTxo0blZOT4xkbEhKixMRENW3aVM2aNdOLL75oYfLAx7Gxp0A7LoG2P4GC42JfNeHY/Prrr8rMzFSHDh2sjlIpgXxsTrVIGoZRbd9zRjmDrfz8889KTEzUnDlzNGDAAKvjnJIjR44oNDRUq1at0gUXXCBJ2rdvn2JiYjRq1CiFhoYqMzNT27dvV1ZWlnJzcy1OXL4DBw5o06ZNuvDCCwPig8CBdGwq4q233tJtt91m+y+dD7TjEmj7czL8nFU/Xpvte2yO9corr+juu++2/b+NEwm0Y7Njx45Tfm5SUpIPk5SPyxr9WFFRkR577DGFh4friSeeOOHlisXFxRo+fLhq1aqll19+uRpTVp4//r6gV69enuuz4+LiZBiGDh8+7Fle8sXG11xzjTp16mRVzEpbtGiRBg4cqKysLCUmJlod55QE6rHxd4F2XAJtfwJFoB4XXptRlQL52FRXwTodlDM/9sorr+jZZ5/V/PnzT/o5srCwMA0YMEADBgxQjx49dN1111VTytL27NmjNWvWlLnst99+kyStWbNGERERJ1zPVVdd5fNsp+qPP/7QJ598otzcXM+tmTt37qyEhAS1atVKzZs3l2EY+vLLLyVJzZo1U0xMjJWRa4xAOjbZ2dn67LPPVKdOHdWrV08xMTGKjY1Vo0aNrI5WaYF0XKTA2x/p6G/HZ8yYoW3btik2NlaDBg1SixYtrI5VKYF4XAJFoBybzMxMLVu2TLVr11aDBg0UFxdX6ouZDx06pBdeeKHcdYwaNao6olZYoByb0/H7778rNDRUUVFR1b5tLmv0Y5dccokiIiL02WefVfg53bt3V3R0tN5///2qC3YSs2fP1uDBg8stlCU/kidaXp3X/lbG4cOHtWPHDp111ll68MEHVVxcrC1btmjz5s3Kzs6W9L/9qlWrlpo2barvv//ewsSlzZkzx3N73HXr1un111/XE088oXr16kmSEhIS1Lp161I/d3feeacWLFigJ554otzybaVAODYLFy7U1VdfXerfRkREhJo0aaJWrVqpffv2Gjx4sJKSkvzicrNAOC7HCpT9+eqrr3T99dfr999/98wLDQ3V5MmTddddd3mN5eesevDabM9jU9Z7mpCQEM2dO1e7d+/W3XffrV9++cXzSzTDMLyuELLr+xnJ/49NWQ4cOKD58+erc+fOatmyZanlH374oUaOHKmsrCxJ0sUXX6znn39e7du3r7aMnDnzYxs3btTYsWMr9Zz+/fvr3//+d9UEqqCLLrpI06ZNszRDVQkNDVWTJk0kSYMHD/Zcn+10OlWnTh29/fbbql+/vjIzM7Vt2zbPP347+de//qX169d7zXvsscc8f+7Ro4f+8pe/6K9//atnnmEYGjZsmH777bdSz7WLQDg2Jd555x1FRERo37592rdvn/bs2aOffvpJ3333nebNm6fGjRvrlltusTpmhQTScZECY392796ta6+9VtHR0XrttdfUoUMHbd26VaNHj9a9996rdu3aqWPHjlbHrJRAOC68Ntv32EjStGnTVFRUpN27d2vz5s1yOByeZWeeeaYyMjJkmqY6dOigJ554QldffbXmzJmjCRMmWJj6xALl2BzrhRde0MSJE7V169ZSyxYuXKhrr71WR44cUVxcnM4880ytXr1al156qdavX6/k5ORqyUg582N5eXmKi4ur1HMSEhIs/9LA5OTkavsBt0JwcLDee+89r+uaS36zlJSUZPvrs//zn//I6XSWu7xevXratGmTpKO/Xf/9998tvUy2Mvz92EhH81544YVq2rRpmct37NjhdzcICITjcix/358XXnhBRUVF+uKLL3TWWWdJOvp/x+LFi3Xuuedq/PjxWrhwocUpK8/fjwuvzfZlGIb69Omj+vXre83fvHmzpKN3CDz2zEtiYqLatWun5cuXV2vOU+Hvx+Z4n332mYYMGaIzzzzTa/7Bgwd1++23y+126+9//7vS0tJkGIa2bdumSy65RM8880y13YmScubH6tSpo/3791fqOU6nU2FhYVWUCJLkdrv1yCOP6KOPPtIZZ5whSYqKitKePXsUHR1tcbqTq8ip+02bNskwDHXu3Nnysl8Z/n5sKsIfPux8vEA7Lv6+P5988ol69+7tKWYlIiMjdffdd+vRRx/VkSNHvD5T4w/8/bjw2gwrBNqxycrK0j333FNq/uTJk5Wbm6suXbromWee8cxv3ry5hg8frnfeeafaMvrXt8bBS+vWrT0fxqyo5cuX2+as1eLFizVkyBCNGzdOixcvVnFxsdWRTllOTo7nQ7OmaWrr1q36448/PMsLCgoUGxur1atXWxWxxuLY2FOgHZdA2p+tW7fqoosuKnPZhRdeqOLiYt166626++67dffdd2v69OnVnLDiAum4BBqOjX0F8rHJy8tTrVq1vOYVFxfr+eefl2EYGj9+fKnntG7dWjt37qyuiJw582cDBgzQqFGj9PDDD6tdu3YnHf/LL79oxowZZf7GwAq//vqr5zcRhmGobt26Gjp0qP7+97+rcePGFqernOTkZAUHBys2NlYJCQkyDENvvPGGtm3bprPPPluxsbGeG5n4g0WLFpW77NJLL63GJKcv0I5NWT777LMy/5O044f/SwTacQmk/SkqKvLcKvt4JfNnzpzpNd+u+xVIx0XitdkfrFixwnPn6Q0bNlic5tQE6rGRjl6inZmZ6TXvv//9r3Jzc5WSkqIuXbqUek513zuRcubH7rrrLk2cOFF9+vTR3Llzy/1Np3T0cyj9+/dXcHCwRo4cWY0pyxcSEiLDMLR8+XKtWbNG7777riZNmqRXX31V//jHPzR69GirI1bYa6+9ppycHM/DNE299dZbevHFFz0vXoZh6L777lPbtm3VrFkzNW3aVE2bNj3hcbPCkSNH1KNHjzJfdA3DkMvlsiDVqQukY1Oe999/X6+88kqZy+z6n2egHZdA2p/69etr9+7dZS775ZdfZBiG0tPTPZc4zZs3T//85z+rM2KFBdJx4bXZvsfmWI888oi++eYbSfLbAhOox0aSunTpotdee01/+ctfVLduXW3evFmPPPKIDMPwusHOsX744Qc1bNiw+kKa8Gtr164169evbwYHB5t9+/Y133jjDXPFihXmxo0bzZUrV5pvv/22eeutt5pRUVFmeHi4+dlnn1kd2eP99983g4KCzPz8fM+81atXm126dDENwzB79epl/vHHHxYmPDWHDx82DcMwV69ebe7du9dcsmSJOWnSJNMwDLNt27ZmfHy8GRQUZBqGYQYFBVkdtxSXy2UahmH+9a9/Nd98803P46677vLknTVrlhkUFGQWFRWZu3btMoOCgsyCggLzP//5jy33qYS/H5uPPvrIDAoKMrdv3+41Pysry1y+fHmpxz//+U9b7sfx/P24HM/f9+eqq64y27VrV+ayu+66y2zcuLHXvDfffNOW+3E8fz8uvDbb89iU/J3/9ttvpmma5ty5c81JkyaZkyZNMgcOHFgqr2EY5ltvvWWapmm+/PLLttuf4/nzsSnLTz/9ZEZERJh16tQxzznnHDM8PNw0DMO84YYbyhxfUFBgxsfHm9ddd121ZaScBYAdO3aY11xzjRkUFFTmwzAMs3PnzubatWutjuqlpJzt37+/1LInnnjCDAoKMrt06WIWFBRYkK5y3n77bTM9Pd3Myckxi4qKTMMwzDVr1niWO51O0zAMc9myZaZpmmZhYaG5ceNGc+HChVZFLlfJG4D33nvPa37Jf0DH/tkf3gAE0rEpr5yVx85vmgPpuJhmYO3P22+/bRqGYT777LNe87/88kszLCzMHDVqlNd8fs6qB6/N9jw2x5ezY5VVvgzDMKdPn17ucjsIlGNTnkWLFpnt27c3IyIizGbNmpmPPfaYWVhYWObY66+/3jQMw/z888+rLR+XNQaAxMREffDBB8rOztYXX3yhLVu2KC8vT7Vr11azZs3UrVs3tWnTxuqY5Srryxcfe+wxxcTE6K677tLw4cP19ttvW5Cs4oYMGeI51R8UFCTDMPToo4/q4osvVuvWrZWYmCjpf5eYhYeHq1WrVmrVqpVlmX3N7XZ7PkBsJxwbewq04xJI+3PjjTfq9ddf10MPPaSFCxfqwgsv1LZt2/TBBx8oMTFRjz76qNURKyyQjsup4rXZevn5+br99tslHd2fF198UZ988om2b99ucbKyBfqx6d69u1atWnXScaZp6t///reeffZZJSQkVEOyoyhnfqp+/fqaPXu2evbs6ZmXnJys4cOHV+j5Bw8eVHx8vD799FN16NChqmJWiFnOBy3vuOMOZWdnKy0tTb169dLQoUOrOVnF/fTTT9qxY4dycnKUnZ2tp556SllZWfruu++Un5/veQHr3r27EhMTPddnN2vWzDafATwdpmna9pa6gXZsyvv34m8C7bgE2v58+OGHuv/++zVz5kwtXrxYhmHo8ssv16uvvmrbf+tlCbTjUlm8NlePk70uHzp0SHPnzvVMr1ixQitWrJBkz88FB9KxOVZZ751PxDAMTxGVqvG9c7Wdo4NPGYZhfvzxx6f8/ONPSVuh5LLG3Nzccse4XC7zwgsvNBs2bGjm5eVVY7pTV3LpyerVq03TNM09e/aYn376qWkYhnnTTTeZgwYNMjt06GCeccYZtrycoST/vHnzvOYfe+nM8uXLzSFDhpgul8s8cOCAOWTIEK+HXfn7sXG73eVeelEWO19udix/Py7HC6T9cTqd5qZNm8q8ZKsEP2fVg9dmex6bzMxM8z//+Y9ZXFxcapldL1usDH8+Nsfzl/fOnDnzY7fddlup72qoKNMGdxCqX7++OnTocMIvxQ4ODtaUKVPUuXNnrV27Vpdcckk1Jjw1QUFBuvrqqxUVFSVJiomJUWpqqiTp7rvvVqdOnTxj8/PzLclYESNGjNCDDz7omS4oKPD8+aKLLvLckalu3bqaMWNGtec7Ff5+bAzDUHh4uNUxfM7fj8vxAml/oqKidPbZZ1sdwycC5bjw2myvY9OkSRP95S9/sTpGlfHnY1MWf3jvTDnzU4mJiTIM47QucUpMTFRERIQPU1VOly5d9N133510XIcOHbRy5Uqdf/751ZDq9BmGoQ8++MBrXkREhJ555plS1yzXqVOnOqNViGEY6tChQ5kvQE2bNrUgke/4+7GprFatWunuu++2OsZJBdpxCbT9ORl+zqoHr81H2fHYlKdx48bq2rWr1TFOSyAdG39572yYp5MQAAAAAOATQVYHAAAAAABQzgAAAADAFihnAAAAAGADlLMAU1RUpLFjx6qoqMjqKKctkPZFCqz9YV/sK5D2h32xr0DaH/bFvgJpf9gX+7Lb/nBDkACTn5+v6Oho5eXl2f6uOScTSPsiBdb+sC/2FUj7w77YVyDtD/tiX4G0P+yLfdltfzhzBgAAAAA2QDkDAAAAABvgS6iridvt1u7du+VwOKr028VLvp3dH76l/WQCaV+kwNof9sW+Aml/2Bf7CqT9YV/sK5D2h32xr+rYH9M09ccff6hx48YKCjrxuTE+c1ZNdu3aVeqb1AEAAADUDDt37lR8fPwJx3DmrJo4HA5JUtO/jlFweITFaU5f/JR1VkfwLXfg/I4iKMr/f75KmIddVkfwLX4XZkuFnVpZHcGnaq3aZnUE3wkOoLcp0Q6rE/hWFV4FVN0Mt9vqCD5zKLm+1RF86o+EMKsj+MSR4kL9OOtJTx84kQB61bO3kksZg8MjAqKchRihVkfwLSNw3jQHGYHxQiZJphFoH4sNnJ+zQBIS6v+vyccKCaDXAAUF0NuU4HCrE/hWIJUzBU45CwkJrNez4LAAej2TKvTRpkB75wMAAAAAfolyBgAAAAA2QDkDAAAAABugnAEAAACADVDOAAAAAMAGKGcAAAAAYAOUMwAAAACwAcoZAAAAANgA5QwAAAAAbIByBgAAAAA2QDkDAAAAABugnAEAAACADVDOAAAAAMAGKGcAAAAAYAOUMwAAAACwAduUs+TkZBmGIcMwVLduXavjeKxbt86TyzAMDRs2zOpIAAAAAAKQbcqZJDVs2FALFizQ7NmzveYfOnRIkyZNUpcuXXTGGWcoNDRUZ555prp27arJkyersLBQkrRkyRIZhqFHHnmkQtv78ssvdc011ygpKUkRERFKTk7W5Zdfrtdee00ul0uS1LRpUy1YsECvv/66b3cWAAAAAI4RYnWAY9WqVUt9+vTxmrdu3Tr17dtXP//8s6677joNHTpU9evX188//6wPP/xQDzzwgHbu3Kl///vfldrWhAkT9Oijj6pbt24aPXq0YmJi9Ouvv+rzzz/XiBEj1L17dzVv3lx16tRRnz59lJ2d7cM9BQAAAABvtipnx9uxY4cuu+wyhYeHa+XKlbrgggu8lt977716//33lZGRUan17t+/X48//rg6deqkr776SkFB/zuBOHz4cK1fv14NGjTwyT4AAAAAQEXYupzdd999OnDggJYuXVqqmJXo16+frr766kqtd8uWLTp8+LDOO+88r2JWol27dqeUFwAAAABOla0+c3asrKwsLViwQL1791anTp1OODY4OLhS646Li5MkffHFFzp06NApZwQAAAAAX7FtOfviiy9kmqb69evn83UnJCSoS5cu2rp1q7p06aKvvvrK59soKipSfn6+1wMAAAAAymPbcrZlyxZJUps2bapk/TNmzNA555yj1atXq0ePHjr33HP1zjvvyDRNn6x/woQJio6O9jwSEhJ8sl4AAAAAgcm25SwvL0+S5HA4qmT9SUlJysjI0KRJkxQXF6cNGzbopptuUvv27T3F8HSMHj1aeXl5nsfOnTt9kBoAAABAoLJtOatdu7Ykyel0Vtk2IiIidP/99ysrK0tvvfWWkpOTtXbtWl188cXasWPHaa07PDxcderU8XoAAAAAQHlsW86aNWsmSdq6dWuVbys0NFS33HKL1q9fr27duunAgQMaN25clW8XAAAAAErYtpx169ZNkvTll19W2zbr1KmjV155RZK0atWqatsuAAAAANi2nJ1zzjnq1KmT3n33XWVlZZ1wbFFRUaXW7XK5yl3WsGFDSUfPpgEAAABAdbFtOZOkKVOm6MiRI+rfv792795d5pgPPvhAw4cPr9R633nnHT3zzDNl3pnxww8/lCT17Nmz8oEBAAAA4BSFWB3gRM4//3x98MEHuuGGG9SyZUvdfPPNuvjii1W7dm3t3LlT8+fP19dff63bb7/d63nbt2/XRx99VOY6O3bsKEl6+OGH9e6772rQoEFq0qSJCgoKtGLFCk2bNk1nn322Hn744SrfPwAAAAAoYetyJklXXHGFtmzZoueee04ff/yxZsyYoeLiYjVo0EAXXHCBZs2apeuvv97rOXPnztXcuXPLXN/ixYt14403qrCwULNmzVJaWpoOHDig8PBwtWjRQg8//LBGjhxZZbfwBwAAAICy2KqcmaYpp9MpwzAUFRXlmR8TE6O0tDSlpaWd8PndunWr8JdI33HHHbrjjjtOOs7tduvgwYM6ePBghdYLAAAAAKfCVuUsJydHDodD0dHR+v33362OI0n6/vvvdf7551sdAwAAAECAs005mzt3rgoLCyVJISG2iaUWLVooPT3dMx0TE2NhGgAAAACByjYtKCUlxeoIZYqKilJqaqrVMQAAAAAEOFvfSh8AAAAAagrKGQAAAADYAOUMAAAAAGyAcgYAAAAANkA5AwAAAAAboJwBAAAAgA1QzgAAAADABihnAAAAAGADlDMAAAAAsAHKGQAAAADYAOUMAAAAAGyAcgYAAAAANhBidYCaJuGLPIUEF1od47RtH3eB1RF8qsULO6yO4DNmof//fJUwjMD6/ZFRL9rqCD7j3rPX6gg+UyvrgNURfCrnzjZWR/CZxDe3WR3Bdw7kWZ3Ap8yCg1ZH8J2wMKsT+Ez47j1WR/CphtmxVkfwCdeRogqPDax3PgAAAADgpyhnAAAAAGADlDMAAAAAsAHKGQAAAADYAOUMAAAAAGyAcgYAAAAANkA5AwAAAAAboJwBAAAAgA1QzgAAAADABihnAAAAAGADlDMAAAAAsAHKGQAAAADYAOUMAAAAAGyAcgYAAAAANkA5AwAAAAAboJwBAAAAgA1QzgAAAADABihnpyArK0uzZs3S7t27rY4CAAAAIEBQzk7B119/rcGDB2vNmjVWRwEAAAAQIEKsDmAX33zzjfbv319qvtvtlsvl0plnnqlLL73UgmQAAAAAagLK2Z9GjRqlFStWlLt88ODBlDMAAAAAVYZy9qePP/5YxcXFMgxDhmEoODhYISEh2rx5sy666CK1aNHC6ogAAAAAAhjl7E/169cvc/7atWslSV27dq3OOAAAAABqGG4IchLvvvuu6tatq86dO1sdBQAAAEAAo5ydwJo1a7Ro0SLdcsstioiIKLX8t99+065du7Rr1y4dPHjQgoQAAAAAAgXlrBymaeqBBx5QrVq19NBDD5U5ZtiwYUpISFBCQoLmzZvntayoqEj5+fleDwAAAAAoD585K8f48eOVnp6uJ598UvHx8WWOeeSRR9S+fXtJUocOHbyWTZgwQePGjavynAAAAAACA+WsDK+++qoef/xxXXbZZXr00UfLHde5c2f16dOnzGWjR4/WyJEjPdP5+flKSEjweVYAAAAAgYFydgyXy6UxY8ZowoQJ6tSpk+bNm6egoFO78jM8PFzh4eE+TggAAAAgUPGZsz+tXLlSnTp10oQJE3TDDTfos88+U506dayOBQAAAKCG4MyZjt6VsVOnTqpXr55ef/113XbbbVZHAgAAAFDDUM4kXXDBBfriiy90/vnnq27dulbHAQAAAFADUc7+1L17d6sjAAAAAKjB+MwZAAAAANgA5QwAAAAAbIBydgqGDRsm0zTL/Y4zAAAAAKgsyhkAAAAA2ADlDAAAAABsgHIGAAAAADZAOQMAAAAAG6CcAQAAAIANUM4AAAAAwAYoZwAAAABgA5QzAAAAALAByhkAAAAA2ADlDAAAAABsgHIGAAAAADZAOQMAAAAAG6CcAQAAAIANUM4AAAAAwAYoZwAAAABgAyFWB6hpgn79XUFB4VbHOG0tJv1mdQSf2jg+weoIPtPqkSyrI/jOEZfVCXzK3H/A6gg+Y4SFWh3BZ8xaYVZH8Kmk6ZlWR/CZ7Xc3tzqCzzR99gerI/hWcLDVCXzGfajQ6gg+E0ivzZJkOA9aHcEnDHdxhcdy5gwAAAAAbIByBgAAAAA2QDkDAAAAABugnAEAAACADVDOAAAAAMAGKGcAAAAAYAOUMwAAAACwAcoZAAAAANgA5QwAAAAAbIByBgAAAAA2QDkDAAAAABugnAEAAACADVDOAAAAAMAGKGcAAAAAYAOUMwAAAACwAcoZAAAAANgA5QwAAAAAbIByBgAAAAA2QDn7U1ZWllJTUzV16lSrowAAAACogfyynG3evFmGYWjmzJmSpNdff12GYSg7O7vU2I4dO6pHjx6SpMOHDys4OFhjx44tNa6goEDLli1TVlZWVUYHAAAAgDL5ZTnbsWOHJCkxMdEzHRQUpPj4+DLHlozbuXOn3G63ZxoAAAAA7MIvy1lOTo6k/5WznJwcxcbGKiQkxGtccXGxcnNzvcYd+zwAAAAAsIuQkw+xD7fbLbfb7TlT1qhRI7lcLuXk5CghIUEul0uGYSg4ONgz3+12Kz4+Xi6Xy3PZY1xcnFwul4KDg2UYhtc2TNOUy+U6aZayngsAAAAAp8qvzpz169dPoaGhGj9+vNxut2rVqqXQ0FAtXrxYy5cvV2hoqM4++2zNmzdPoaGhatGihSRp+PDhCg0N1a233ipJat26tUJDQ/Xtt9+W2sZzzz2n0NDQkz6effbZat13AAAAAIHNr86cpaWlacyYMbrpppuUnJysp556SpLUrVs33XLLLbrtttvkcDhUv359ZWRkaNasWXruuee0dOlShYWF6dlnn9XSpUs1f/58SVLbtm09646Pj9e0adMqnOXCCy884fKioiIVFRV5pvPz8yuzqwAAAABqGL8qZ61atZIk7dmzR4MGDVJKSor27t2rgoICde3aVSkpKZ6xDRo00PTp05WQkKBOnTpJkvLy8tSmTRuvcSXq1q2rYcOG+SzrhAkTNG7cOJ+tDwAAAEBg86tyJh0tZvn5+Z6itnHjRklSy5YtS43dsmWLZ1zJ2L59+3qN2bZt22nlCQoKUtOmTUvNHz16tEaOHOmZzs/PV0JCwmltCwAAAEDg8pty5nK5VFhYqPXr10uSkpKS5HQ6PdNxcXFyOp2KiopSQUGBpKPfh9a7d285nU4dOnRIOTk5Sk5OltPpVFhYmMLCwjyfSztV4eHhKiwsLHN+eHj4aa0bAAAAQM3hN+Vs4sSJGjVqlGe65FLFEjExMZKOnllr1KiRZ/7UqVM1depUz/TIkSM1cuRIpaWladSoUfriiy9OK1dwcPBpPR8AAAAAJD8qZwMHDlTHjh310ksvac6cOVq0aJEkacSIEQoLC9PkyZMlSXXq1FF6erp27NihIUOGaMyYMerZs6cWLFigZ555RnPnzlVMTIznjFmPHj0kSfv27dNnn32m9u3be10KCQAAAADVwW9upZ+UlKTU1FQdOXJEiYmJSk1NVWpqqg4cOKC2bdt6pmvVqqXU1FSdccYZkqSePXsqNTVVoaGhCgkJUd++fZWamuo501Zi8+bNuvnmm7Vw4UIrdg8AAABADec35axEZmammjVrJkkqLCzU7t27PdPHj5PkWZaZmanExESFhPjNyUIAAAAANYjflbPt27d77o6YlZUl0zTLvFvi9u3bVatWLcXGxpZ6HgAAAADYjV+dRsrLy9P+/fs9Javk7FhZpSszM1NNmjTxmh4wYMBJt7Fs2TJFREScdNx5552n1NTUikYHAAAAgBPyq3IWHR0t0zQ907179/aaPtb8+fO9pvft21ehbbz//vt6//33Tzru/vvvp5wBAAAA8Bm/KmdVKTU1tdyiBwAAAABVze8+cwYAAAAAgYhyBgAAAAA2QDkDAAAAABugnAEAAACADVDOAAAAAMAGKGcAAAAAYAOUMwAAAACwAcoZAAAAANgA5QwAAAAAbIByBgAAAAA2QDkDAAAAABugnAEAAACADVDOAAAAAMAGKGcAAAAAYAMhVgeoacyIcJnB4VbHOG3m7lyrI/jU2ZOcVkfwmW0jz7I6gs+0eD7T6gg+5c7/w+oIPmOEBM5/H0ciQ62O4FsHfrc6gc80ezlwXgN+ubGN1RF8qtGczVZH8BkjMtLqCL5juq1O4FvhYVYn8A23WeGhnDkDAAAAABugnAEAAACADVDOAAAAAMAGKGcAAAAAYAOUMwAAAACwAcoZAAAAANgA5QwAAAAAbIByBgAAAAA2QDkDAAAAABugnAEAAACADVDOAAAAAMAGKGcAAAAAYAOUMwAAAACwAcoZAAAAANgA5QwAAAAAbIByBgAAAAA2QDkDAAAAABugnAEAAACADVDOJB05ckQul6vch2manrGrVq3Seeedp9mzZ1uYGAAAAECgoZxJ6ty5s0JDQ8t9fPXVV56xTqdT69ev1759+yxMDAAAACDQhFgdwC4aNGigF1980WvekiVLNHXqVIsSAQAAAKhJKGd/ioyM1HXXXec1z+l0ljt+3bp1mjt3riSpe/fuOuOMM6o0HwAAAIDAxmWNp+j111/X9ddfr+uvv16bNm2yOg4AAAAAP0c5O0Xjxo1TVlaWsrKydOGFF1odBwAAAICf47LGPxUUFGjWrFle81asWFHu+Pr16ys5Obnc5UVFRSoqKvJM5+fnn3ZGAAAAAIGLcvanX3/9VYMHD/bZ+iZMmKBx48b5bH0AAAAAAhuXNUpavny5TNMs99GjRw/P2Hr16umyyy5TQkLCCdc5evRo5eXleR47d+6s6t0AAAAA4Mc4c3aM5cuXKzs7+6Tjbr/9drVu3fqEY8LDwxUeHu6jZAAAAAACHeXsGFOmTNHbb79dobEvvPCCWrRoUcWJAAAAANQUXNZ4jGnTpunQoUMnfHz11VdWxwQAAAAQgDhzdgyn06mCgoITjsnLy6umNAAAAABqEsrZMe69994KX9YIAAAAAL5EOSvD77//ruDg4BOO4WYfAAAAAHyJclaG2bNnKyjoxB/HO+OMM9S/f/9qSgQAAAAg0FHOynDnnXeedEy7du0oZwAAAAB8hrs1HmPmzJkn/DLqYx/r1q2zOi4AAACAAEI5AwAAAAAboJwBAAAAgA1QzgAAAADABihnAAAAAGADlDMAAAAAsAHKGQAAAADYAOUMAAAAAGyAcgYAAAAANkA5AwAAAAAboJwBAAAAgA1QzgAAAADABihnAAAAAGADlDMAAAAAsAHKGQAAAADYQIjVAWoaMyJcZnC41TFOm7uwyOoIPhW0LdvqCD7TfFqx1RF8JnvqmVZH8KnEm/+wOoLPmC6X1RF8prhemNURfCq8+LDVEXznwO9WJ/CZ2AU7rI7gU/ldW1gdwWccX22yOoLvBAdbncC3AmV/jIrvB2fOAAAAAMAGKGcAAAAAYAOUMwAAAACwAcoZAAAAANgA5QwAAAAAbIByBgAAAAA2QDkDAAAAABugnAEAAACADVDOAAAAAMAGKGcAAAAAYAOUMwAAAACwAcoZAAAAANgA5QwAAAAAbIByBgAAAAA2QDkDAAAAABugnAEAAACADVDOAAAAAMAGKGenYNmyZZo1a5bVMQAAAAAEEMrZKZg4caIGDx5sdQwAAAAAAcQvy9nmzZtlGIZmzpwpSXr99ddlGIays7NLje3YsaN69OghSTp8+LCCg4M1duzYUuMOHTqkMWPGKDk5WREREWrXrp3eeOONqtwNAAAAAPDwy3K2Y8cOSVJiYqJnOigoSPHx8WWOLRm3c+dOud1uz3SJI0eOqE+fPho/fry6dOmixx9/XHXq1NFf/vIX/fOf/6zivQEAAAAAPy1nOTk5kv5XznJychQbG6uQkBCvccXFxcrNzfUad+zzSrzyyitatGiR/v3vf2vGjBkaPXq0vv76a11zzTWaMGGCNmzYUNW7BAAAAKCG86ty5na75XK5PGfKGjVqJJfLpZycHCUkJMjlcunIkSOS5JnvdrsVHx8vl8vluewxLi5OLpdLpmlKkl566SXFxMTovvvu82wrKChITzzxhNxuN5c3AgAAAKhyIScfYh/9+vXTggULPNO1atXyWh4aGqrmzZsrLS1N1157rWf+8OHDNXz4cM9069atJUlLly5V69at9cMPP+i6664rdeatXbt2qlu3rtLT06tidwAAAADAw6/OnKWlpSkjI0NnnXWWevXqpYyMDGVkZCgqKkojRoxQRkaGPvroI3Xp0kUZGRl68MEHZRiGli1bpoyMDA0aNEjx8fGe56WkpGjXrl2SpMaNG5e5zfj4eM8YAAAAAKgqfnXmrFWrVpKkPXv2aNCgQUpJSdHevXtVUFCgrl27KiUlxTO2QYMGmj59uhISEtSpUydJUl5entq0aeM17vDhw5JKn4UrERkZqeLi4kpnLSoqUlFRkWc6Pz+/0usAAAAAUHP41Zkz6Wgxy8/P9xS1jRs3SpJatmxZauyWLVs840rGHj+uXr16kqTc3Nwyt7d7927Vr1+/0jknTJig6OhozyMhIaHS6wAAAABQc/hNOXO5XHI6nVq/fr0kKSkpyWs6Li5OTqdTpmnK6XTK6XRq8+bNat68uZxOp/bt26ecnBwlJyfL6XR6zoYlJycrOjpaa9asKbXN3Nxc7d69W+3atat03tGjRysvL8/z2Llz52nsPQAAAIBA5zflbOLEiXI4HLriiiskSZ06dZLD4dADDzwgSYqJiZHD4dDevXvlcDjkcDiUnZ2tqVOnyuFwqGHDhjJNUyNHjpTD4dCkSZMkSYZhaMCAAfr++++1atUqr21Onz5dbrdb/fr1q3Te8PBw1alTx+sBAAAAAOXxm3I2cOBApaen68Ybb1RoaKjS09OVnp6uc845RxdccIFnuk6dOkpPT9fMmTMlSWPGjFF6erpGjRolSZo7d67S09M1dOhQz7ofe+wxRUVFaciQIVq9erUKCws1Z84cjRs3Tuecc44GDRpkyT4DAAAAqDn85oYgSUlJSkpK0pQpU5SYmKjU1FRJ0oEDB9SjRw/PtCSlpqbK6XRKknr27KnU1FR9+umnCgkJUd++fUvdMr9Jkyb68MMPNXjwYK+bhZx77rn68MMPFRoaWg17CAAAAKAm85tyViIzM1PNmjWTJBUWFmr37t2e6ePHSfIsy8zMVGJiYqliVuLSSy/V9u3b9fnnnys3N1dnnXWWunXrpuDg4CraEwAAAAD4H78rZ9u3b9fAgQMlSVlZWTJNU02bNi1zXK1atRQbG+uZLmvcsWrXrq0BAwb4PjQAAAAAnITffOZMOvo9Zfv37/eUrJKzY2WVrszMTDVp0sRr+mTlDAAAAACs4ldnzqKjo2Wapme6d+/eXtPHmj9/vtf0vn37qjQbAAAAAJwOvypndjF37lyrIwAAAAAIMH51WSMAAAAABCrKGQAAAADYAOUMAAAAAGyAcgYAAAAANkA5AwAAAAAboJwBAAAAgA1QzgAAAADABihnAAAAAGADlDMAAAAAsAHKGQAAAADYAOUMAAAAAGyAcgYAAAAANkA5AwAAAAAboJwBAAAAgA1QzgAAAADABkKsDlDTBBUcVFDQEatjnL6oSKsToBzmz3usjuAzyQ/UtzqCTz3+49dWR/CZcV37Wx3BZyI3/2p1BJ8yw0KtjuA7wcFWJ/AZ9+95VkfwqdqfB9D+xDa0OoHv7P3N6gS+5XZbncA3KrEfnDkDAAAAABugnAEAAACADVDOAAAAAMAGKGcAAAAAYAOUMwAAAACwAcoZAAAAANgA5QwAAAAAbIByBgAAAAA2QDkDAAAAABugnAEAAACADVDOAAAAAMAGKGcAAAAAYAOUMwAAAACwAcoZAAAAANgA5QwAAAAAbIByBgAAAAA2QDkDAAAAABugnAEAAACADVDOAAAAAMAGQqwOYCdut1sff/xxhce3adNGTZo0qcJEAAAAAGoKytkxiouLdfXVV1d4/IsvvqgRI0ZUYSIAAAAANQXl7Bjh4eHaunXrScetWrVKgwcPVp06daohFQAAAICagHJ2DMMw1Lx585OO++GHHyRJ9erVq+pIAAAAAGoIbghyCvbt2ydJiomJsTgJAAAAgEDBmbNTUFLOYmNjyx1TVFSkoqIiz3R+fn6V5wIAAADgvzhzdgJZWVm666679Mknn3jN37NnjyIiIk5YziZMmKDo6GjPIyEhoarjAgAAAPBjlLMTyM3N1SuvvKLVq1d7zf/pp5/UpEkTGYZR7nNHjx6tvLw8z2Pnzp1VHRcAAACAH+OyxlPw448/6uKLLz7hmPDwcIWHh1dTIgAAAAD+jjNnlZSdna3du3frggsusDoKAAAAgADCmbNKCg0N1bRp03TJJZdYHQUAAABAAKGcVVJcXJyGDRtmdQwAAAAAAYZyJmnFihXatGlTqfnbt2+XJK1du1ZvvvnmCddBYQMAAABwOihnkmbMmKGpU6eWu3zevHmaN2/eCddBOQMAAABwOrghiKQpU6bINM3TegAAAADA6aCcAQAAAIANUM4AAAAAwAYoZwAAAABgA5QzAAAAALAByhkAAAAA2ADlDAAAAABsgHIGAAAAADZAOQMAAAAAG6CcAQAAAIANUM4AAAAAwAYoZwAAAABgA5QzAAAAALAByhkAAAAA2ADlDAAAAABsgHIGAAAAADZAOQMAAAAAGwixOkBNY4aEyAz2/792s7DI6gg+FRTtsDqCz7j/cFodwWfc+361OoJPjf7LXVZH8Jl+n35pdQSfWXB7N6sj+FRQ9k6rI/hMUO0oqyP4jDvvD6sj+FaQYXUCnzEO5FkdwWeM6DpWR/CtQHm/6S6u8FDOnAEAAACADVDOAAAAAMAGKGcAAAAAYAOUMwAAAACwAcoZAAAAANgA5QwAAAAAbIByBgAAAAA2QDkDAAAAABugnAEAAACADVDOAAAAAMAGKGcAAAAAYAOUMwAAAACwAcoZAAAAANgA5QwAAAAAbIByBgAAAAA2QDkDAAAAABugnAEAAACADVDOKikrK0uzZs3S3r17rY4CAAAAIIBQzirp66+/1uDBg7Vx40arowAAAAAIIH5ZzjZv3izDMDRz5kxJ0uuvvy7DMJSdnV1qbMeOHdWjRw9J0uHDhxUcHKyxY8eWGnfw4EGNHTtWLVu2VHh4uBo3bqw777xTP//8c1XuCgAAAABI8tNytmPHDklSYmKiZzooKEjx8fFlji0Zt3PnTrndbs90iT/++EOpqakaN26c2rRpo/Hjx6t///6aMWOGLrjgAm3ZsqWK9wgAAABATRdidYBTkZOTI+l/5SwnJ0exsbEKCfHeneLiYuXm5nqNO/Z5JcaPH6+1a9dq8uTJuu+++zzzb731VnXs2FF33nmnFi9eXGX7AwAAAAB+debM7XbL5XJ5zpQ1atRILpdLOTk5SkhIkMvl0pEjRyTJM9/tdis+Pl4ul8tz2WNcXJxcLpdM05QkzZ49WzExMbr33nu9tpeSkqKrr75aS5YsUW5ubrXuKwAAAICaxa/OnPXr108LFizwTNeqVctreWhoqJo3b660tDRde+21nvnDhw/X8OHDPdOtW7eWJC1dulSdO3fW3r17dc4558gwjFLbbNq0qSRp7969iomJ8en+AAAAAEAJvzpzlpaWpoyMDJ111lnq1auXMjIylJGRoaioKI0YMUIZGRn66KOP1KVLF2VkZOjBBx+UYRhatmyZMjIyNGjQIMXHx3uel5KSIuloAdu2bZuKi4tLbXPDhg0KCQlRUlJSpbIWFRUpPz/f6wEAAAAA5fGrctaqVSulpKRoz549uvjii5WSkqLExEQVFBSoa9euSklJUcuWLdWgQQOlpKSouLhYCQkJ6tSpk1JSUpSXl6c2bdooJSVFKSkpCg8PlyQ99NBDOnDggP7+9797LouUjl7u+Pnnn+uOO+5QnTp1vLJ0795dhmHIMAz9+uuvpbJOmDBB0dHRnkdCQkLV/uUAAAAA8Gt+dVmjJO3Zs0f5+flq1aqVJHm+b6xly5alxm7ZssUzrmRs3759S40bOnSo1q1bp0mTJunzzz9XSkqKsrOztWzZMvXq1UsTJkwo9Zy//e1vOuussyRJtWvXLrV89OjRGjlypGc6Pz+fggYAAACgXH5TzlwulwoLC7V+/XpJUlJSkpxOp2c6Li5OTqdTUVFRKigokHT0+9B69+4tp9OpQ4cOKScnR8nJyXI6nQoLC1NYWJhn/RMnTlT//v319ttv69VXX1WbNm20YMECXXnllQoODi6V55prrlG3bt3KzRseHu45MwcAAAAAJ+M3lzVOnDhRDodDV1xxhSSpU6dOcjgceuCBByRJMTExcjgc2rt3rxwOhxwOh7KzszV16lQ5HA41bNhQpmlq5MiRcjgcmjRpUqltdO3aVWlpaZKO3qmxT58+ZRYzAAAAAPA1vylnAwcOVHp6um688UaFhoYqPT1d6enpOuecc3TBBRd4puvUqaP09HTNnDlTkjRmzBilp6dr1KhRkqS5c+cqPT1dQ4cOtXJ3AAAAAMCL31zWmJSUpKSkJE2ZMkWJiYlKTU2VJB04cEA9evTwTEtSamqqnE6nJKlnz55KTU3Vp59+qpCQEPXt29fzZdXFxcX6+OOPvbZT8rwtW7Zo8uTJ+uOPP/T7779rz549io6O1oUXXlgduwsAAACghvGbclYiMzNTzZo1kyQVFhZq9+7dnunjx0nyLMvMzFRiYqKnmElHb9LRv3//MrezYsUK/fDDD6pXr57q16+vxo0bq02bNr7eHQAAAACQ5IflbPv27Ro4cKAkKSsrS6Zper4o+vhxtWrVUmxsrGf6+HFnnHGGfvnlF890UFCQQkJCFBERocjIyDK3/+abb/poTwAAAADgf/yqnOXl5Wn//v2eklVydqyscpaZmakmTZp4TQ8YMMBrjGEYatSoURUmBgAAAICK8atyFh0dLdM0PdO9e/f2mj7W/Pnzvab37dtXpdkAAAAA4HT4zd0a7WLYsGEyTfOE33EGAAAAAJVFOQMAAAAAG6CcAQAAAIANUM4AAAAAwAYoZwAAAABgA5QzAAAAALCBSpez559/vtS8zMxM3XXXXT4JBAAAAAA1UaXL2dNPP11qXnJysmbMmOGTQAAAAABQE1XoS6gXL16sxYsXS5L++OMPjRkzxmv53r17Va9ePd+nAwAAAIAaokLlLCcnR+np6ZKkw4cPe/5cIjIyUi+++KLv0wEAAABADVGhcjZ06FANHTpUkhQbG+s5iwYAAAAA8I1Kf+Zszpw5Zc7/9ttvTzsMAAAAANRUlS5nffv21TvvvOM17/nnn1efPn18FgoAAAAAapoKXdZ4rC+//FIDBw7UihUr9OSTT2rEiBHasGGDli1bVhX5AAAAAKBGqPSZs/PPP1+rV69WZmamGjdurODgYK1YsUKtWrWqinwAAAAAUCNUupxJ0vfff6/vv/9eXbt21cqVK7VlyxZf5wIAAACAGqXSlzU++eSTmjp1qqZPn65evXrp3XffVY8ePTR27Fjdc889VZExsISFSsGhVqc4bebhYqsj+JQ77w+rI/iO6bY6gc8YdaOtjuBTYcs3WR3BZxYOSbU6gs98vvAtqyP41OVx51sdwWfcfzitjuA7AfTaLEmGUem3kLZlFh+2OoLPmPEOqyP4lHGwyOoIPmEeCZFyKza20mfO3n//fS1fvly9evWSJA0cOFDffPONpk6dWtlVAQAAAAD+VOlytmzZMjVs2FCrV6/2zGvVqpUyMjJ8GgwAAAAAapJKl7MNGzYoISFBl156qWfexIkT9corr/g0GAAAAADUJJUuZ4888oj+85//KCoqyjPv2muv1RtvvOHTYAAAAABQk1S6nG3dulX9+/eXaZqeeYmJifrtt998GgwAAAAAapJKl7MjR46UmldQUOCTMAAAAABQU1W6nF100UVatmyZDMOQdLSsjRo1Sh06dPB5OAAAAACoKSr9JRUTJ07UgAED9Pvvv6tfv35at26dDh48qK+//roq8gEAAABAjVDpM2dJSUlatGiRHn/8ccXExOi2227T999/r1atWlVFPgAAAACoESp95uz555/Xfffdp4cfftgzLzMzU3fddZdefvlln4YDAAAAgJqi0mfOnn766VLzkpOTNWPGDJ8EAgAAAICaqEJnzhYvXqzFixdLkv744w+NGTPGa/nevXtVr14936cDAAAAgBqiQuUsJydH6enpkqTDhw97/lwiMjJSL774ou/TAQAAAEANUaFyNnToUA0dOlSSFBsb6zmLBgAAAADwjUp/5mzLli1VkQMAAAAAarRKlzOHw1EVOQAAAACgRqt0OavpVqxYoVmzZsnlclkdBQAAAEAAqXQ5mz9/fql5O3bs0FNPPeWTQHb30ksvafDgwSosLLQ6CgAAAIAAUulyds8995SaFx8frwkTJvgkUEVs3rxZhmFo5syZkqTXX39dhmEoOzu71NiOHTuqR48eko7eaTI4OFhjx44tNS47O1uDBw9W/fr1Vbt2bV1++eVatWpVVe4GAAAAAHhU6G6N69at05o1ayRJBw8e1LRp02Sapmf53r17VatWrapJWIYdO3ZIkhITEz3TQUFBio+PL3PsFVdcIUnauXOn3G6353klMjMz1bFjRxUVFen2229XVFSUpk2bpi5duuiLL75QampqFe8RAAAAgJquQuXs66+/1qRJkyRJBQUFeuKJJ7yWR0ZGavz48T4PV56cnBxJ/ytnOTk5io2NVUiI9+4UFxcrNzfXa9yxzyvxwAMPaP/+/Vq+fLnat28vSRoxYoTOPfdcDR8+XBs3bpRhGF7P2b17tyIjIxUcHKzY2Fjf7yQAAACAGqVC5ez+++/X/fffL+no95xlZWVVaajyuN1uud1uz5myRo0ayeVyKScnRwkJCXK5XDIMQ8HBwZ75brdb8fHxcrlcnsse4+Li5HK5FBwcrD179mjhwoUaMGCAp5hJUqNGjfTXv/5Vjz/+uJYtW1bq7FnLli0lSWeccYZ+/fXXavs7AAAAABCYKlTOjvXVV19VRY4K6devnxYsWOCZPv5SytDQUDVv3lxpaWm69tprPfOHDx+u4cOHe6Zbt24tSVq6dKlyc3PldrvVrVu3Utu75JJLPOOOL2f//e9/FRERofDw8NPeLwAAAACodDm75557Sl3iV2LRokWnHehE0tLSNGbMGN10001KTk723CGyW7duuuWWW3TbbbfJ4XCofv36ysjI0KxZs/Tcc89p6dKlCgsL07PPPqulS5d67jjZtm1bvfrqq5Kkxo0bl9peyeWKu3btKrWsT58+ql27dlXtKgAAAIAaptLlbNiwYZ4/m6apzZs369VXX9UDDzzgw1hla9WqlSRpz549GjRokFJSUrR3714VFBSoa9euSklJ8Yxt0KCBpk+froSEBHXq1EmSlJeXpzZt2niNO3z4sKTSZ+Gko5+lk45+dq2yioqKVFRU5JnOz8+v9DoAAAAA1ByVLmdDhw4tNa9Xr16aMmWKTwKdzJ49e5Sfn+8pahs3bpT0v8+AHWvLli2ecSVj+/bt6zWmbt26kqTc3NwytyVJ9evXr3TOCRMmaNy4cZV+HgAAAICaqdLfc1aW7t27a9myZb5YVblcLpecTqfWr18vSUpKSvKajouLk9PplGmacjqdcjqd2rx5s5o3by6n06l9+/YpJydHycnJcjqdnrNhbdu2lSStXbu21DbXrVsnSWrXrl2pZQ6HQ4Zh6Mwzzywz7+jRo5WXl+d57Ny587T/DgAAAAAELp+Usx9++EGhoaG+WFW5Jk6cKIfD4fnOsk6dOsnhcHgup4yJiZHD4dDevXvlcDjkcDiUnZ2tqVOnyuFwqGHDhjJNUyNHjpTD4fB8NUD79u2VmJioWbNmqbCw0Gubb775psLDwz3bPNazzz6rF154QWlpaWXmDQ8PV506dbweAAAAAFCeSl/W2KVLF68bghQUFOiHH37Qk08+6dNgxxs4cKA6duyol156SXPmzPHcfGTEiBEKCwvT5MmTJUl16tRRenq6duzYoSFDhmjMmDHq2bOnFixYoGeeeUZz585VTEyMWrRoIUkKCgrSuHHjdOutt+qmm27S5MmTVbt2bY0fP17Lli3TI488onr16pXKc8cdd3BDEAAAAAA+U+ly1qNHD6/p6OhodezYURdddJHPQpUlKSlJSUlJmjJlihITEz23tj9w4IB69Ojhdav71NRUOZ1OSVLPnj2VmpqqTz/9VCEhIerbt2+pL6seNmyYdu3apXHjxmnevHme+bfeemuVl04AAAAAkE6hnD3++OOSpO3bt+vXX39VcnKyYmJifB6sPJmZmWrWrJkkqbCwULt37/ZMHz9OkmdZZmamEhMTSxWzEv/85z91yy23aPHixSouLlbHjh11zjnnVNFeAAAAAIC3SpeznJwcDRgwQGvXrpVpmjIMQ/369dNbb71VLZf5bd++XQMHDpQkZWVlyTRNNW3atMxxtWrV8nxX2fbt28scd6zExMQy70YJAAAAAFWt0jcEueeee5SYmKgtW7bo4MGDWrNmjZxOp+67776qyOclLy9P+/fv95SskrNjZZWuzMxMNWnSxGv6ZOUMAAAAAKxS6TNnGRkZysrK8nxpc7t27TRr1iw1b97c5+GOFx0dLdM0PdO9e/f2mj7W/Pnzvab37dtXpdkAAAAA4HRU+sxZWFhYqc9t1atXTw6Hw2eh7OzNN9+UaZrcqREAAACAT1W6nI0aNUpjx471mvfOO++oZ8+evsoEAAAAADVOpS9rnDVrllauXKnZs2erRYsWys/P18qVK3Xuueeqa9euXmO/+eYbnwUFAAAAgEBW6XLWs2fPUmfJevXq5bNAAAAAAFATVbqc3XXXXaW+1yw3N1fff/89lzYCAAAAwCmq9GfOzj///FLzoqKidOONN/okEAAAAADURBU6c/b7779r//79kiSXy+X58ucSubm5KiwsrJqEAAAAAFADVKicTZ48WePGjZNhGJLk9Z1mpmnKMAzdeuutVZMQAAAAAGqACpWzBx54QMOGDZNpmurQoYMyMjK8lkdGRqpBgwZVEhAAAAAAaoIKlbPo6GhFR0dLkv71r38pKSmpSkMBAAAAQE1T6bs17t69W0888USZy8aMGXPagQAAAACgJqp0OcvKyvL82TRNbd68WatXr1a/fv18mQsAAAAAapRKl7Np06aVmjdx4kT98ssvPgkEAAAAADVRpb/nrCwPPPCAZs2a5YtVAQAAAECN5JNylpeXp6KiIl+sCgAAAABqpEpf1nj8TT8KCgr0ySef6IorrvBZKAAAAACoaSpdztLT072mo6OjNXToUN17770+CxXIXHXCpZAIq2OcNlevFKsj+FT4sk1WR/Ad07Q6gc+YBw9ZHcG3goOtTuA7GzZbncBnrura3+oIPrX97WirI/jMWWPyrI7gM8ahwLrCyCwstDqC74RU+u2wfe35zeoEPuVOaGh1BJ9wH6n4xYqV/mlcvHhxZZ8CAAAAADiJSpWzNWvWaObMmdq0aZNM01TLli110003qUOHDlWVDwAAAABqhAqfYxs1apQuuugi/fjjj2rZsqXOPvtsbdmyRZ07d9bIkSOrMiMAAAAABLwKnTl744039Omnn+rHH3/UWWed5bVs69atuuqqq9S6dWvdfvvtVRISAAAAAAJdhc6cvfjii5o+fXqpYiZJLVq00OzZs/Xiiy/6PBwAAAAA1BQVKme7d+/WeeedV+7yCy64QHv27PFVJgAAAACocSr8mTOXy3VKywAAAAAAJ1ehctapUydNnz693OWvvvqqLrroIp+FAgAAAICapkI3BBk3bpy6du2qHTt2aMSIEWrUqJEkae/evXrxxRc1efJkffPNN1UaFAAAAAACWYXOnLVp00affPKJPvzwQ8XFxSk6OlrR0dGKjY3VvHnztHDhQrVt27aqswIAAABAwKrwl1B36NBBa9eu1Y8//qiNGzdKklq2bKlzzz23ysIBAAAAQE1R4XJWok2bNmrTpk1VZAEAAACAGqvCd2sEAAAAAFQdyhkAAAAA2ADlDAAAAABsgHIGAAAAADZAOQMAAAAAG6CcAQAAAIANUM4AAAAAwAYoZ2UwTVPvvPOOrrrqKjVo0EDBwcGKjIxUu3bt9Mgjj+jnn3+2OiIAAACAAFPpL6EOdE6nU3379tWiRYvUtm1b3X777WrcuLEOHjyoFStWaOLEiXrppZf03nvvqUePHlbHBQAAABAgKGfHeeCBB7Ro0SI988wzeuihh0ot/+mnn9SrVy9dd9112rx5s2JiYixICQAAACDQcFnjMZxOp2bMmKFLLrmkzGImSWeffbaee+455eXlacaMGdWcEAAAAECgopwdY+fOnSouLlaHDh1OOK5k+bZt26ojFgAAAIAagMsaj9GwYUMZhnHS0lWyPDY2ttwxRUVFKioq8kzn5+f7JiQAAACAgMSZs2OcccYZuvzyy/XBBx/oyy+/LHNMQUGBRo8erZCQEA0aNKjcdU2YMEHR0dGeR0JCQlXFBgAAABAAKGfHefXVV5WYmKgrr7xSd911lxYsWKBVq1bpm2++0XPPPadzzz1Xq1at0qRJk9SyZcty1zN69Gjl5eV5Hjt37qzGvQAAAADgb7is8TgJCQlas2aN/vWvf2nmzJl65ZVXPMtCQ0N16aWX6q233lJqauoJ1xMeHq7w8PCqjgsAAAAgQHDmrAz16tXT//3f/+mXX37R0qVLJUk33nijnE6nPv3005MWMwAAAACoLMrZSZxxxhmSJIfDobCwMIvTAAAAAAhUlDMAAAAAsAE+cyZpxYoV2rRpU5nLfvnlF0nS5s2b9eabb5a7jmHDhlVBMgAAAAA1BeVM0owZMzR16tQTjlmyZImWLFlS7nLKGQAAAIDTwWWNkqZMmSLTNE/rAQAAAACng3IGAAAAADZAOQMAAAAAG6CcAQAAAIANUM4AAAAAwAYoZwAAAABgA5QzAAAAALAByhkAAAAA2ADlDAAAAABsgHIGAAAAADZAOQMAAAAAG6CcAQAAAIANUM4AAAAAwAYoZwAAAABgA5QzAAAAALAByhkAAAAA2ADlDAAAAABsIMTqADVNyP6DCgk+YnWM0xaUkWN1BJ9yXdTa6gg+E5yxyeoIKE9xsdUJfMcInN/tGYUBdFwknfXUIasj+MwFc7dZHcFn1vRoaHUE3woKtjqBz5h/OK2O4DNBdRxWR/Apo9BldQSfMI5U/L1/4PzvCgAAAAB+jHIGAAAAADZAOQMAAAAAG6CcAQAAAIANUM4AAAAAwAYoZwAAAABgA5QzAAAAALAByhkAAAAA2ADlDAAAAABsgHIGAAAAADZAOQMAAAAAG6CcAQAAAIANUM4AAAAAwAYoZwAAAABgA5QzAAAAALAByhkAAAAA2ADlDAAAAABsgHIGAAAAADZAOftTVlaWUlNTNXXqVKujAAAAAKiB/LKcbd68WYZhaObMmZKk119/XYZhKDs7u9TYjh07qkePHpKkw4cPKzg4WGPHji01rqCgQMuWLVNWVlZVRgcAAACAMvllOduxY4ckKTEx0TMdFBSk+Pj4MseWjNu5c6fcbrdnGgAAAADswi/LWU5OjqT/lbOcnBzFxsYqJCTEa1xxcbFyc3O9xh37PAAAAACwi5CTD7EPt9stt9vtOVPWqFEjuVwu5eTkKCEhQS6XS4ZhKDg42DPf7XYrPj5eLpfLc9ljXFycXC6XgoODZRiG1zZM05TL5TpplrKeCwAAAACnyq/OnPXr10+hoaEaP3683G63atWqpdDQUC1evFjLly9XaGiozj77bM2bN0+hoaFq0aKFJGn48OEKDQ3VrbfeKklq3bq1QkND9e2335baxnPPPafQ0NCTPp599tlq3XcAAAAAgc2vzpylpaVpzJgxuummm5ScnKynnnpKktStWzfdcsstuu222+RwOFS/fn1lZGRo1qxZeu6557R06VKFhYXp2Wef1dKlSzV//nxJUtu2bT3rjo+P17Rp0yqc5cILLzzh8qKiIhUVFXmm8/PzK7OrAAAAAGoYvypnrVq1kiTt2bNHgwYNUkpKivbu3auCggJ17dpVKSkpnrENGjTQ9OnTlZCQoE6dOkmS8vLy1KZNG69xJerWrathw4b5LOuECRM0btw4n60PAAAAQGDzq3ImHS1m+fn5nqK2ceNGSVLLli1Ljd2yZYtnXMnYvn37eo3Ztm3baeUJCgpS06ZNS80fPXq0Ro4c6ZnOz89XQkLCaW0LAAAAQODym3LmcrlUWFio9evXS5KSkpLkdDo903FxcXI6nYqKilJBQYGko9+H1rt3bzmdTh06dEg5OTlKTk6W0+lUWFiYwsLCPJ9LO1Xh4eEqLCwsc354ePhprRsAAABAzeE35WzixIkaNWqUZ7rkUsUSMTExko6eWWvUqJFn/tSpUzV16lTP9MiRIzVy5EilpaVp1KhR+uKLL04rV3Bw8Gk9HwAAAAAkPypnAwcOVMeOHfXSSy9pzpw5WrRokSRpxIgRCgsL0+TJkyVJderUUXp6unbs2KEhQ4ZozJgx6tmzpxYsWKBnnnlGc+fOVUxMjOeMWY8ePSRJ+/bt02effab27dt7XQoJAAAAANXBb26ln5SUpNTUVB05ckSJiYlKTU1VamqqDhw4oLZt23qma9WqpdTUVJ1xxhmSpJ49eyo1NVWhoaEKCQlR3759lZqa6jnTVmLz5s26+eabtXDhQit2DwAAAEAN5zflrERmZqaaNWsmSSosLNTu3bs908ePk+RZlpmZqcTERIWE+M3JQgAAAAA1iN+Vs+3bt3vujpiVlSXTNMu8W+L27dtVq1YtxcbGlnoeAAAAANiNX51GysvL0/79+z0lq+TsWFmlKzMzU02aNPGaHjBgwEm3sWzZMkVERJx03HnnnafU1NSKRgcAAACAE/KrchYdHS3TND3TvXv39po+1vz5872m9+3bV6FtvP/++3r//fdPOu7++++nnAEAAADwGb8qZ1UpNTW13KIHAAAAAFXN7z5zBgAAAACBiHIGAAAAADZAOQMAAAAAG6CcAQAAAIANUM4AAAAAwAYoZwAAAABgA5QzAAAAALAByhkAAAAA2ADlDAAAAABsgHIGAAAAADZAOQMAAAAAG6CcAQAAAIANUM4AAAAAwAYoZwAAAABgA5QzAAAAALCBEKsD1Dhut2S4rU5x2swjR6yO4FOhP2RZHcFnjpzf0uoIPhP0Y+AcF0kyDx2yOgLKEmCvZ+b2HVZH8Jk11zW3OoLP1P/wgNURfOrAEIfVEXzGqB1pdQTfOeyyOoFPGW7/f88sVW4/OHMGAAAAADZAOQMAAAAAG6CcAQAAAIANUM4AAAAAwAYoZwAAAABgA5QzAAAAALAByhkAAAAA2ADlDAAAAABsgHIGAAAAADZAOQMAAAAAG6CcAQAAAIANUM4AAAAAwAYoZwAAAABgA5QzAAAAALAByhkAAAAA2ADlDAAAAABsgHIGAAAAADZAOQMAAAAAG6CcSTpy5IhcLle5D9M0PWNXrVql8847T7Nnz7YwMQAAAIBAQzmT1LlzZ4WGhpb7+OqrrzxjnU6n1q9fr3379lmYGAAAAECgCbE6gF00aNBAL774ote8JUuWaOrUqRYlAgAAAFCTUM7+FBkZqeuuu85rntPpLHf8unXrNHfuXElS9+7ddcYZZ1RpPgAAAACBjcsaT9Hrr7+u66+/Xtdff702bdpkdRwAAAAAfo5ydorGjRunrKwsZWVl6cILL7Q6DgAAAAA/x2WNfyooKNCsWbO85q1YsaLc8fXr11dycnK5y4uKilRUVOSZzs/PP+2MAAAAAAIX5exPv/76qwYPHuyz9U2YMEHjxo3z2foAAAAABDYua5S0fPlymaZZ7qNHjx6esfXq1dNll12mhISEE65z9OjRysvL8zx27txZ1bsBAAAAwI9x5uwYy5cvV3Z29knH3X777WrduvUJx4SHhys8PNxHyQAAAAAEOsrZMaZMmaK33367QmNfeOEFtWjRoooTAQAAAKgpuKzxGNOmTdOhQ4dO+Pjqq6+sjgkAAAAgAHHm7BhOp1MFBQUnHJOXl1dNaQAAAADUJJSzY9x7770VvqwRAAAAAHyJclaG33//XcHBwSccw80+AAAAAPgS5awMs2fPVlDQiT+Od8YZZ6h///7VlAgAAABAoKOcleHOO+886Zh27dpRzgAAAAD4DHdrPMbMmTNP+GXUxz7WrVtndVwAAAAAAYRyBgAAAAA2QDkDAAAAABugnAEAAACADVDOAAAAAMAGKGcAAAAAYAOUMwAAAACwAcoZAAAAANgA5QwAAAAAbIByBgAAAAA2QDkDAAAAABugnAEAAACADVDOAAAAAMAGKGcAAAAAYAOUMwAAAACwgRCrA9Q0ZlSEzOBwq2OcPvcRqxP41JF8p9URfCb4px1WR/CdxMZWJ/Ap98YtVkfwHcO0OoHvmAG0L5JMl8vqCL6T+6vVCXxm74NNrY7gU5d98K3VEXzmm17NrI7gO0GBdd7FHR5qdQSfcB9xV3hsYB1BAAAAAPBTlDMAAAAAsAHKGQAAAADYAOUMAAAAAGyAcgYAAAAANkA5AwAAAAAboJwBAAAAgA1QzgAAAADABihnAAAAAGADlDMAAAAAsAHKGQAAAADYAOUMAAAAAGyAcgYAAAAANkA5AwAAAAAboJwBAAAAgA1QzgAAAADABihnAAAAAGADlDMAAAAAsAHKGQAAAADYQIjVAU7F5s2bdfbZZ2vGjBkaMmSIXn/9dd1+++3KyspScnKy19iOHTsqKipKX375pQ4fPqyIiAg99thjGjt2rNc4t9utDz/8sMIZrrrqKoWFhflgbwAAAADAT8vZjh07JEmJiYme6aCgIMXHx5c59oorrpAk7dy5U2632/O8YxUXF6t///4VzrBv3z6deeaZpxIfAAAAAErxy3KWk5Mj6X/lLCcnR7GxsQoJ8d6d4uJi5ebmeo079nllueaaazR16tSTZqhfv/4pZQcAAACAsvhVOXO73XK73Z4zZY0aNZLL5VJOTo4SEhLkcrlkGIaCg4M9891ut+Lj4+VyuZSdnS1JiouLk8vlUnBwsAzD8NpGrVq1yjwDBwAAAABVya/KWb9+/bRgwQLPdK1atbyWh4aGqnnz5kpLS9O1117rmT98+HANHz7cM926dWtJ0tKlS9W5c+cqTg0AAAAAJ+dXd2tMS0tTRkaGzjrrLPXq1UsZGRnKyMhQVFSURowYoYyMDH300Ufq0qWLMjIy9OCDD8owDC1btkwZGRkaNGiQ4uPjPc9LSUmpsqxFRUXKz8/3egAAAABAefzqzFmrVq0kSXv27NGgQYOUkpKivXv3qqCgQF27dvUqWw0aNND06dOVkJCgTp06SZLy8vLUpk2bKi1lJSZMmKBx48ZV+XYAAAAABAa/KmfS0WKWn5/vKWobN26UJLVs2bLU2C1btnjGlYzt27fvCde/detWTZky5aQ5+vfvr7i4uHKXjx49WiNHjvRM5+fnKyEh4aTrBQAAAFAz+U05c7lcKiws1Pr16yVJSUlJcjqdnum4uDg5nU5FRUWpoKBA0tHvQ+vdu7ecTqcOHTqknJwcJScny+l0KiwsrMzvKVuzZo3WrFlz0jznnHPOCctZeHi4wsPDT2VXAQAAANRAfvOZs4kTJ8rhcHi+s6xTp05yOBx64IEHJEkxMTFyOBzau3evHA6HHA6HsrOzNXXqVDkcDjVs2FCmaWrkyJFyOByaNGmS1/ojIiJkmqbX45NPPvFs+/hl3bp1q8a9BwAAABDo/KacDRw4UOnp6brxxhsVGhqq9PR0paen65xzztEFF1zgma5Tp47S09M1c+ZMSdKYMWOUnp6uUaNGSZLmzp2r9PR0DR061MrdAQAAAAAvfnNZY1JSkpKSkjRlyhQlJiYqNTVVknTgwAH16NHDMy1JqampcjqdkqSePXsqNTVVn376qUJCQtS3b99SX1YNAAAAAFbzu5aSmZmpZs2aSZIKCwu1e/duz/Tx4yR5lmVmZioxMbFUMfvggw904MCBMre1YcMGSdLKlSv15ptvljmmUaNGnkstAQAAAOBU+V052759uwYOHChJysrKkmmaatq0aZnjatWqpdjYWM90WeP+8Y9/6McffzzhNv/73//qv//9b5nLLrnkEsoZAAAAgNPmV+UsLy9P+/fv95SskrNjZZWuzMxMNWnSxGt6wIABpcb98MMPVZQWAAAAACrOr8pZdHS0TNP0TPfu3dtr+ljz58/3mt63b1+VZgMAAACA0+E3d2sEAAAAgEBGOQMAAAAAG6CcAQAAAIANUM4AAAAAwAYoZwAAAABgA5QzAAAAALAByhkAAAAA2ADlDAAAAABsgHIGAAAAADZAOQMAAAAAG6CcAQAAAIANUM4AAAAAwAYoZwAAAABgA5QzAAAAALAByhkAAAAA2ADlDAAAAABsIMTqADWNOzxY7pAA+GsPCrY6gU8ZwYGzP+Zhl9URfMbclm11BJ8Kjq5jdQSfCaSfs0BjBML/MX8yDMPqCD4Tsi/f6gg+9c1VLa2O4DMdP9tmdQSfWTbkPKsj4DRx5gwAAAAAbIByBgAAAAA2QDkDAAAAABugnAEAAACADVDOAAAAAMAGKGcAAAAAYAOUMwAAAACwAcoZAAAAANgA5QwAAAAAbIByBgAAAAA2QDkDAAAAABugnAEAAACADVDOAAAAAMAGKGcAAAAAYAOUMwAAAACwAcoZAAAAANgA5QwAAAAAbCDE6gB2kZmZqZUrV1Z4/IUXXqhmzZpVYSIAAAAANQnl7E+LFi3S8OHDKzz+zTffpJwBAAAA8BnK2Z9uvvlm9evX76TjFi5cqGHDhql27dpVHwoAAABAjUE5+1N4eLjCw8NPOq6oqEiS1KBBg6qOBAAAAKAG4YYg5Zg7d666deum7777zmv+7t27JUlxcXFWxAIAAAAQoChn5di1a5e+/vpr/fbbb17zN2/erJCQECUmJlqUDAAAAEAg4rLGSho+fLgGDRqk0NDQE44rKiryXAIpSfn5+VUdDQAAAIAfq/Hl7MMPP/Rcqnisb7/9VpL0ySefaNeuXaWWv/zyy54/DxgwQA0bNvRaPmHCBI0bN87HaQEAAAAEqhpfzp577jl9/fXX5S5/8cUXT7qO8847r1Q5Gz16tEaOHOmZzs/PV0JCwqkHBQAAABDQanw5W7JkiefPe/bsUUREhOrWrVvueJfLpV9//VUOh0NRUVHljqvo3R8BAAAAQOKGIF5iY2N1++23n3DMunXrFBsbqxdeeKGaUgEAAACoCShnAAAAAGADNf6yxuO99957MgzD6hgAAAAAahjK2XE6dOigESNGlLs8JydHjz/+eDUmAgAAAFATUM6O06RJEw0bNqzc5evWraOcAQAAAPA5ytlxtm/f7vUdZscr6zvPAAAAAOB0Uc6Os2rVKq1atcrqGAAAAABqGMrZMUzTtDoCAAAAgBqKW+kDAAAAgA1QzgAAAADABihnAAAAAGADlDMAAAAAsAHKGQAAAADYAOUMAAAAAGyAcgYAAAAANkA5AwAAAAAboJwBAAAAgA1QzgAAAADABihnAAAAAGADlDMAAAAAsAHKGQAAAADYAOUMAAAAAGyAcgYAAAAANhBidYCaxh0eLHdIsNUxTluQ+4jVEXzKtDqAD5mHAufYGEGG1RF86ki+0+oIPmOEBs5/H8XNY62O4FNBy/ZaHcFngiPCrY7gM+YvgXNcJEkBdGzS7+hgdQSfaf/WOqsj+NSKv19odQSfcLkqPpYzZwAAAABgA5QzAAAAALAByhkAAAAA2ADlDAAAAABsgHIGAAAAADZAOQMAAAAAG6CcAQAAAIANUM4AAAAAwAYoZwAAAABgA5QzAAAAALAByhkAAAAA2ADlDAAAAABsgHIGAAAAADZAOQMAAAAAG6CcAQAAAIANUM4AAAAAwAYoZwAAAABgA5SzSlqxYoVmzZoll8tldRQAAAAAAYRyVkkvvfSSBg8erMLCQqujAAAAAAggIVYHOBWbN2/W2WefrRkzZmjIkCF6/fXXdfvttysrK0vJycleYzt27KioqCh9+eWXOnz4sCIiIvTYY49p7NixnjELFixQQUFBudvr2LGjkpKSqmhvAAAAAMBPy9mOHTskSYmJiZ7poKAgxcfHlzn2iiuukCTt3LlTbrfb87wS9957r2edZZkxYwblDAAAAECV8stylpOTI+l/5SwnJ0exsbEKCfHeneLiYuXm5nqNO/Z5xzrrrLO0evXqMrcXERHhs+wAAAAAUBa/Kmdut1tut9tzpqxRo0ZyuVzKyclRQkKCXC6XDMNQcHCwZ77b7VZ8fLxcLpeys7MlSXFxcXK5XAoODpZhGJIkwzBUu3ZtC/cOAAAAQE3mV+WsX79+WrBggWe6Vq1aXstDQ0PVvHlzpaWl6dprr/XMHz58uIYPH+6Zbt26tSRp6dKl6ty5cxWnBgAAAICT86u7NaalpSkjI0NnnXWWevXqpYyMDGVkZCgqKkojRoxQRkaGPvroI3Xp0kUZGRl68MEHZRiGli1bpoyMDA0aNEjx8fGe56WkpJxylg0bNmjdunVat26dDh8+XGp5UVGR8vPzvR4AAAAAUB6/OnPWqlUrSdKePXs0aNAgpaSkaO/evSooKFDXrl29ylaDBg00ffp0JSQkqFOnTpKkvLw8tWnT5rRKWYmSdUpHbzRy/M1IJkyYoHHjxp32dgAAAADUDH515kw6Wszy8/M9RW3jxo2SpJYtW5Yau2XLFs+4krFljTsVn3/+udLT05Wenq6GDRuWWj569Gjl5eV5Hjt37vTJdgEAAAAEJr85c+ZyuVRYWKj169dLkpKSkuR0Oj3TcXFxcjqdioqK8nxn2ebNm9W7d285nU4dOnRIOTk5Sk5OltPpVFhYmMLCwk45T8eOHU94A5Hw8HCFh4ef8voBAAAA1Cx+c+Zs4sSJcjgcnu8s69SpkxwOhx544AFJUkxMjBwOh/bu3SuHwyGHw6Hs7GxNnTpVDodDDRs2lGmaGjlypBwOhyZNmlSp7Zum6eM9AgAAAID/8ZszZwMHDlTHjh310ksvac6cOVq0aJEkacSIEQoLC9PkyZMlSXXq1FF6erp27NihIUOGaMyYMerZs6cWLFigZ555RnPnzlVMTIxatGjhtf78/HzNnDlTknT48GEdPHhQubm52rFjh3766Sdt3LiRSxMBAAAAVBm/KWdJSUlKSkrSlClTlJiYqNTUVEnSgQMH9P/t3VtsVGXbxvFr1ZaBQjfuhg7D1LGAxpTwJigbsUGJEqZSU6mhBwoJJmKTvrE0mEoUA60mGDcQQI2k8aCiJlBpMKJJDwQ+KpGIiRCCGTdt6cfUUqoWqC0UaLu+A9OJI1PoR6ddT6f/X7IO1qxnPc99Q5jkYm3mscceC+9LUk5Ojjo7OyVJixcvVk5Ojmpra5WYmKj8/Pxrfqxaks6cOaOVK1eG9xMSEnTnnXcqMzNT99xzjwoLCzVx4sRh7hIAAADAWDVqwlm/xsZGTZs2TZLU3d2tlpaW8P6/x0kKH2tsbFRmZmbUYHbs2DH19vZK+vvHqF0ul5KTk5WQMGru+gQAAAAwyo26cNbQ0KDCwkJJ0qlTp2TbtrKysqKOmzBhgjweT3g/2jhJuvXWW4evYAAAAAAYhFF1aejChQtqb28Ph6z+q2PRQldjY6PuvvvuiP2BwhkAAAAAOG1UXTlLS0uLeGvi0qVLB3yL4t69eyP2f//992GtDQAAAACGYlRdOTNBVVWVbNu+7m+cAQAAAMD/F+EMAAAAAAxAOAMAAAAAAxDOAAAAAMAAhDMAAAAAMADhDAAAAAAMQDgDAAAAAAMQzgAAAADAAIQzAAAAADAA4QwAAAAADEA4AwAAAAADEM4AAAAAwACEMwAAAAAwAOEMAAAAAAxAOAMAAAAAAyQ6XcBY8/t/JugW13inyxiyjDrL6RJiy+5zuoKYsRKTnC4BA7BucbqC2LGv9jhdQsz0ueLoL0bSLePGOV1CzNjdl50uIWas8S6nS4it3l6nK4iZxP9tc7qEmDlcMd/pEmKq/b9dTpcQE70XL0v/M7ixXDkDAAAAAAMQzgAAAADAAIQzAAAAADAA4QwAAAAADEA4AwAAAAADEM4AAAAAwACEMwAAAAAwAOEMAAAAAAxAOAMAAAAAAxDOAAAAAMAAhDMAAAAAMADhDAAAAAAMQDgDAAAAAAMQzgAAAADAAIQzAAAAADAA4QwAAAAADEA4AwAAAAADEM4AAAAAwADGhDO/3y/LsmRZltLT050uJ0J6enq4Nr/f73Q5AAAAAOKQMeFMktxut/bt26fdu3eHP/vpp5+0evVq3XvvvUpOTpbH49GCBQtUUVGh9vb28LimpqZwgLredv78eUlSeXl5xOcul0t+v1/PPPOMvvnmm4i6du/erX379sntdo/InwMAAACAsSfR6QL+acKECcrLywvvHzx4ULm5ucrMzNTq1auVlZWlS5cu6ciRI9qyZYu8Xq+ee+65iDkeffRRlZaWDrjGpEmTIvYrKio0e/ZsdXV1qb6+Xjt37tTChQtVUVGhDRs2SJKWLFkSrg8AAAAAhoNR4ezfysrKlJSUpCNHjuj2228Pf75ixQpt3LhRf/755zXnTJ06NSLg3cjcuXMVCATC+y+99JKWLl2q8vJyBQIBzZ07d2hNAAAAAMAgGHVb47+dPHlSmZmZEcGsn9vt1n333RfzNZOSkrRu3TrZtq3PPvss5vMDAAAAQDRGhzOv16tff/1VwWBwRNf1+XySpLa2thFdFwAAAMDYZXQ4e/rpp3X16lUtWrRIVVVVunz58ois+9tvv0mSPB7PTc9x+fJldXR0RGwAAAAAMBCjw9n69eu1fPlynT17Vs8++6zuuusubdq0SV1dXQOe09PTo87OzqjblStXBrXu+++/L0latmzZTdf+xhtvKC0tLbz1X40DAAAAgGiMDmfjx49XdXW1Pv/8c91///06e/as1q9fr2nTpunLL7+Mes6nn36qlJSUqFv/2xf/qbu7W52dnQqFQqqtrVUgEFBNTY2Ki4s1b968m6795Zdf1oULF8JbKBS66bkAAAAAxD+j39bYLz8/X/n5+Tp06JBeffVVHT58WPn5+aqpqdGTTz4ZMTY3N1evvPJK1HmiXb3699Wx6dOn64MPPlBRUdGQana5XHK5XEOaAwAAAMDYMSrCWb+HH35YdXV1Kisr0+bNm7V27dprwpnb7VZOTs6g53znnXc0b948uVwueb1eTZkyJcZVAwAAAMCNGX1bYzSWZemtt96S1+vVqVOn1N7ePqT5srOzlZOTozlz5hDMAAAAADjG2HDW19cn27ajHktISNAdd9wh6e/fJQMAAACA0c7YcBYKhVRYWBj1zYz19fUKBoOaN2+eUlJSHKgOAAAAAGLL2GfOLMvSnj17dOTIEa1atUozZ86UZVkKBoOqrKzUuHHjtH379mvOa25uHvBNjpIUCASUmGhs2wAAAADGKGNTSmZmpmpra1VZWamPPvpIbW1tkv5+4+ITTzyhdevWKSsr65rz9u/fr/379w8477lz55Senj5cZQMAAADATTEqnNm2rc7OTlmWpYkTJ2rJkiVasmTJoM71+/0DPqMWTXl5ucrLywc1tqurS7Zty7ZtWZY16DUAAAAAYLCMeubs9OnTSklJkdfrdbqUCF6vVykpKTp9+rTTpQAAAACIU8ZcOduzZ4+6u7slybhnwmpra9XT0yNJGj9+vMPVAAAAAIhHxqSgBx54wOkSBjR//nynSwAAAAAQ54y6rREAAAAAxirCGQAAAAAYgHAGAAAAAAYgnAEAAACAAQhnAAAAAGAAwhkAAAAAGIBwBgAAAAAGIJwBAAAAgAEIZwAAAABgAMIZAAAAABiAcAYAAAAABiCcAQAAAIABEp0uYKywbVuS1Hul2+FKYqPHvup0CRiAZTtdAcYC2+51uoSY6emJj+/lflYcfT9bThcQQ5YdT93EF6vvFqdLiJmeq/H1fdZ78bLTJcREfx/9eeB6LHswozBkzc3N8vl8TpcBAAAAwAGhUEhTp0697hjC2Qjp6+tTS0uLUlJSZFnD979nHR0d8vl8CoVCSk1NHbZ1RkI89SLFVz/0Yq546odezBVP/dCLueKpH3ox10j0Y9u2/vrrL02ZMkUJCdd/qozbGkdIQkLCDZNyLKWmpsbFPxgpvnqR4qsfejFXPPVDL+aKp37oxVzx1A+9mGu4+0lLSxvUOF4IAgAAAAAGIJwBAAAAgAEIZ3HG5XJp48aNcrlcTpcyZPHUixRf/dCLueKpH3oxVzz1Qy/miqd+6MVcpvXDC0EAAAAAwABcOQMAAAAAAxDOAAAAAMAAhDMAAAAAMADhDAAw5vj9fn344Ycjtt7WrVv1ww8/DHi8qqpKBw4ciNl6sZ4PADAyCGcAAAwzwhkAYDAIZwAAAABgAMIZAGBMW7VqldasWaOSkhL5fD5NnjxZRUVFunr1qiSpvLxcDz30kN599135fD653W7l5eWpqakpfDwnJydizqqqKj3yyCP68ccflZGRoVAopBdffFEZGRmqqakJj+vo6FBGRoa+/fZbbd26VRkZGdq6daskqampScuWLZPH49HUqVO1aNEifffdd7p06ZJmz56t1157LTzPsWPHdNttt6murm7A+QAA5iOcAQDGvB07dig7O1tNTU06fvy49u7dq127doWPHz16VC0tLaqvr9epU6eUlJSkgoIC9fX1XXfe7Oxstba2yufzafPmzWptbdVTTz0VPp6amqrW1lYtWLBApaWlam1tVWlpqc6fP6+CggKtXbtWZ86cUXNzs1544QXl5uaqt7dXNTU1eu+991RXV6cLFy5o+fLl2rZtmxYuXBh1PgDA6JDodAEAADgtLy9PRUVFkiSPx6OcnBx9//33WrlypSTJ5/Np06ZNsixLLpdLb7/9tmbMmKGGhoZhqWfHjh0KBoNavnx5xOdXrlxRMBjUnDlztHPnTq1YsUKzZs1SIBAI1woAGL0IZwCAMW/mzJkR+8nJyero6Ajvz5gxQ5Zlhfe9Xq8k6Y8//og6n23bQ6qnsbFRixcv1hdffDHgmEAgoOzsbB04cEAtLS1DWg8AYAZuawQAjHn/DF7R/DOoSdIvv/wi6e9X8qelpamrqyvi+M8//zykerxer44fPx5+7i2a6upqNTQ0qKCgQMXFxUNaDwBgBsIZAAA3cPToUX3yySeybVvnzp1TWVmZ8vPz5fF49OCDD+rkyZM6ceKEJOngwYOqrKyMOD85OVltbW3q6ekJv0gk2vHe3l41NDTo+eef18WLF1VSUqLOzk5J0unTp7VlyxZJUjAYVHFxsXbt2qXKykqdOHFC27ZtG3A+AMDoQDgDAOAGZs2apYMHD8rv9ysrK0sej0c7d+6UJM2fP1+vv/66Hn/8cbndbm3fvl1r1qyJOL+kpERbtmxRamqqPv7442vmLyoq0ldffaVJkybpzTfflMfj0aFDh9Tc3Kzp06dr8uTJCgQC6uvrU2dnpwoKCrRhwwbNnj1bycnJqq6u1saNG3X48OGo8wEARgfLHuqN8QAAxLHy8nJ9/fXX4eADAMBw4coZAAAAABiAcAYAAAAABiCcAQAAAIABeOYMAAAAAAzAlTMAAAAAMADhDAAAAAAMQDgDAAAAAAMQzgAAAADAAIQzAAAAADAA4QwAAAAADEA4AwAAAAADEM4AAAAAwACEMwAAAAAwwP8BK92okU6AaVoAAAAASUVORK5CYII=\n" 598 | }, 599 | "metadata": {} 600 | } 601 | ] 602 | } 603 | ] 604 | } -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # SIGPL 2023 튜토리얼: 손에 잡히는 프로그래밍 언어 모델 구현 2 | 3 | ## 준비물 4 | - Google 계정 5 | - Github 계정 6 | 7 | ## 강의 자료 8 | - [슬라이드](slide.pdf) 9 | - [1교시 실습 자료](1_Concepts.ipynb) 10 | - [2교시 실습 자료](2_Fine_tuning.ipynb) 11 | 12 | ## Colab 에서 ipynb 사용 방법 13 | 1. Colab 에 GitHub 계정 연결: 14 | * 구글 Colab 페이지 접속 https://colab.research.google.com/ 15 | * 우측 상단의 :gear: 표시 클릭 16 | * GitHub 탭에서 계정 연결 17 | 2. 파일 상단에 있는 ![image](https://colab.research.google.com/assets/colab-badge.svg) 배지 클릭 18 | 3. Colab 에서 ![image](https://github.com/prosyslab/sigpl23-tutorial/assets/17640199/d72426d3-36fe-4d88-89ac-826a4b64dfb0) 버튼 클릭하여 개인 드라이브로 복사 19 | * 본인의 구글 드라이브 내 `Colab Notebooks` 디렉토리에 저장됨 20 | -------------------------------------------------------------------------------- /slide.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/prosyslab/sigpl23-tutorial/b02623147dbb8bb165e88260057eecf2824129ce/slide.pdf --------------------------------------------------------------------------------