├── Chapter3 ├── 3_1_인공신경망의_구성요소.ipynb ├── 3_2_1_범죄율로_집값_예측하기_기초회귀문제.ipynb ├── 3_2_2_나만의_데이터셋을_만들어_집값_예측하기.ipynb ├── 3_3_간단한_과일_분류_문제_비틀기(이미지분류기초문제).ipynb ├── 3_4_뉴스_기사_분류하기(텍스트분류기초문제).ipynb ├── 3_5_학습을_더_쉽고_빠르고_정확하게.ipynb └── README.md ├── Chapter4 ├── 4_1_1_트랜스퍼러닝_ResNet.ipynb ├── 4_1_2_트랜스퍼러닝_메트릭러닝.ipynb ├── 4_2_자기지도학습과_준지도학습.ipynb └── README.md ├── Chapter5 ├── 5_1_실전딥러닝_이미지분류.ipynb ├── 5_2_실전딥러닝_객체검출.ipynb ├── 5_3_1_실전딥러닝_질의응답.ipynb ├── 5_3_2_실전딥러닝_긍부정분석.ipynb ├── 5_4_딥러닝과_마이크로_서비스.ipynb ├── 5_5_인공지능을_위한_인공지능.ipynb └── README.md ├── Data ├── README.md └── dataset.zip └── README.md /Chapter3/3_1_인공신경망의_구성요소.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "colab": { 6 | "name": "3_1_인공신경망의_구성요소.ipynb", 7 | "provenance": [], 8 | "collapsed_sections": [] 9 | }, 10 | "kernelspec": { 11 | "name": "python3", 12 | "display_name": "Python 3" 13 | } 14 | }, 15 | "cells": [ 16 | { 17 | "cell_type": "markdown", 18 | "metadata": { 19 | "id": "vXmzVZqzqGhu", 20 | "colab_type": "text" 21 | }, 22 | "source": [ 23 | "# 3.1 인공신경망의 구성요소\n", 24 | "\n", 25 | "- 작성자: [김찬란](https://github.com/seriousran)" 26 | ] 27 | }, 28 | { 29 | "cell_type": "code", 30 | "metadata": { 31 | "id": "DHG7NuYfoOiU", 32 | "colab_type": "code", 33 | "outputId": "0657f1d7-40ba-4731-c3fd-db1716ee8f05", 34 | "colab": { 35 | "base_uri": "https://localhost:8080/", 36 | "height": 89 37 | } 38 | }, 39 | "source": [ 40 | "# 코드 3-1 벡터 예제\n", 41 | "import numpy as np\n", 42 | "\n", 43 | "x = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12])\n", 44 | "print(x.shape)\n", 45 | "print(x.ndim)\n", 46 | "print(x.dtype)\n", 47 | "print(x)" 48 | ], 49 | "execution_count": 1, 50 | "outputs": [ 51 | { 52 | "output_type": "stream", 53 | "text": [ 54 | "(12,)\n", 55 | "1\n", 56 | "int64\n", 57 | "[ 1 2 3 4 5 6 7 8 9 10 11 12]\n" 58 | ], 59 | "name": "stdout" 60 | } 61 | ] 62 | }, 63 | { 64 | "cell_type": "code", 65 | "metadata": { 66 | "id": "FfThw6VqoVsB", 67 | "colab_type": "code", 68 | "outputId": "a175cb3c-4752-4cff-dbca-c64ead5a7c70", 69 | "colab": { 70 | "base_uri": "https://localhost:8080/", 71 | "height": 125 72 | } 73 | }, 74 | "source": [ 75 | "# 코드 3-2 행렬 변환 예제\n", 76 | "\n", 77 | "A = x.reshape((3,4))\n", 78 | "print(A.shape)\n", 79 | "print(A.ndim)\n", 80 | "print(A.dtype)\n", 81 | "print(A)" 82 | ], 83 | "execution_count": 2, 84 | "outputs": [ 85 | { 86 | "output_type": "stream", 87 | "text": [ 88 | "(3, 4)\n", 89 | "2\n", 90 | "int64\n", 91 | "[[ 1 2 3 4]\n", 92 | " [ 5 6 7 8]\n", 93 | " [ 9 10 11 12]]\n" 94 | ], 95 | "name": "stdout" 96 | } 97 | ] 98 | }, 99 | { 100 | "cell_type": "code", 101 | "metadata": { 102 | "id": "5mXYYM_IolEa", 103 | "colab_type": "code", 104 | "outputId": "ca03215f-6d5c-4ecb-e6f0-f3dd515729ee", 105 | "colab": { 106 | "base_uri": "https://localhost:8080/", 107 | "height": 125 108 | } 109 | }, 110 | "source": [ 111 | "# 코드 3-3 포트란식 행렬 변환 예제\n", 112 | "\n", 113 | "A = x.reshape((3,4), order='F') # Fortran-like index ordering\n", 114 | "print(A.shape)\n", 115 | "print(A.ndim)\n", 116 | "print(A.dtype)\n", 117 | "print(A)" 118 | ], 119 | "execution_count": 3, 120 | "outputs": [ 121 | { 122 | "output_type": "stream", 123 | "text": [ 124 | "(3, 4)\n", 125 | "2\n", 126 | "int64\n", 127 | "[[ 1 4 7 10]\n", 128 | " [ 2 5 8 11]\n", 129 | " [ 3 6 9 12]]\n" 130 | ], 131 | "name": "stdout" 132 | } 133 | ] 134 | }, 135 | { 136 | "cell_type": "code", 137 | "metadata": { 138 | "id": "NP_InkoSpdk6", 139 | "colab_type": "code", 140 | "outputId": "ed912353-b234-48e3-b76f-48c129a204c9", 141 | "colab": { 142 | "base_uri": "https://localhost:8080/", 143 | "height": 215 144 | } 145 | }, 146 | "source": [ 147 | "# 코드 3-4 다차원 행렬 변환 예제\n", 148 | "\n", 149 | "A = x.reshape((3, 2, 2))\n", 150 | "print(A.shape)\n", 151 | "print(A.ndim)\n", 152 | "print(A.dtype)\n", 153 | "print(A)" 154 | ], 155 | "execution_count": 4, 156 | "outputs": [ 157 | { 158 | "output_type": "stream", 159 | "text": [ 160 | "(3, 2, 2)\n", 161 | "3\n", 162 | "int64\n", 163 | "[[[ 1 2]\n", 164 | " [ 3 4]]\n", 165 | "\n", 166 | " [[ 5 6]\n", 167 | " [ 7 8]]\n", 168 | "\n", 169 | " [[ 9 10]\n", 170 | " [11 12]]]\n" 171 | ], 172 | "name": "stdout" 173 | } 174 | ] 175 | }, 176 | { 177 | "cell_type": "code", 178 | "metadata": { 179 | "id": "X-tnWbHmpgga", 180 | "colab_type": "code", 181 | "outputId": "cf149594-5cbb-4aa7-c75f-00448a418f76", 182 | "colab": { 183 | "base_uri": "https://localhost:8080/", 184 | "height": 489 185 | } 186 | }, 187 | "source": [ 188 | "# 코드 3-5 케라스 함수형 API 예제\n", 189 | "\n", 190 | "from keras.layers import Input, Dense\n", 191 | "from keras.models import Model\n", 192 | "\n", 193 | "inputs = Input(shape=(512,))\n", 194 | "\n", 195 | "x1 = Dense(64, activation='relu')(inputs)\n", 196 | "x2 = Dense(64, activation='relu')(x1)\n", 197 | "outputs = Dense(10, activation='softmax')(x2)\n", 198 | "\n", 199 | "model = Model(inputs=inputs, outputs=outputs)\n", 200 | "model.compile(optimizer='rmsprop',\n", 201 | " loss='categorical_crossentropy',\n", 202 | " metrics=['accuracy'])\n", 203 | "model.fit(data, labels) \n", 204 | "\n", 205 | "# 아직 준비된 데이터가 없기 때문에 실행할 경우 에러가 나는 것이 맞습니다." 206 | ], 207 | "execution_count": 5, 208 | "outputs": [ 209 | { 210 | "output_type": "stream", 211 | "text": [ 212 | "Using TensorFlow backend.\n" 213 | ], 214 | "name": "stderr" 215 | }, 216 | { 217 | "output_type": "display_data", 218 | "data": { 219 | "text/html": [ 220 | "

\n", 221 | "The default version of TensorFlow in Colab will soon switch to TensorFlow 2.x.
\n", 222 | "We recommend you upgrade now \n", 223 | "or ensure your notebook will continue to use TensorFlow 1.x via the %tensorflow_version 1.x magic:\n", 224 | "more info.

\n" 225 | ], 226 | "text/plain": [ 227 | "" 228 | ] 229 | }, 230 | "metadata": { 231 | "tags": [] 232 | } 233 | }, 234 | { 235 | "output_type": "stream", 236 | "text": [ 237 | "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:66: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.\n", 238 | "\n", 239 | "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:541: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.\n", 240 | "\n", 241 | "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:4432: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.\n", 242 | "\n", 243 | "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/optimizers.py:793: The name tf.train.Optimizer is deprecated. Please use tf.compat.v1.train.Optimizer instead.\n", 244 | "\n", 245 | "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:3576: The name tf.log is deprecated. Please use tf.math.log instead.\n", 246 | "\n" 247 | ], 248 | "name": "stdout" 249 | }, 250 | { 251 | "output_type": "error", 252 | "ename": "NameError", 253 | "evalue": "ignored", 254 | "traceback": [ 255 | "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", 256 | "\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)", 257 | "\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m()\u001b[0m\n\u001b[1;32m 13\u001b[0m \u001b[0mloss\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'categorical_crossentropy'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 14\u001b[0m metrics=['accuracy'])\n\u001b[0;32m---> 15\u001b[0;31m \u001b[0mmodel\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfit\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdata\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlabels\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 16\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 17\u001b[0m \u001b[0;31m# 아직 준비된 데이터가 없기 때문에 실행할 경우 에러가 나는 것이 맞습니다.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", 258 | "\u001b[0;31mNameError\u001b[0m: name 'data' is not defined" 259 | ] 260 | } 261 | ] 262 | }, 263 | { 264 | "cell_type": "code", 265 | "metadata": { 266 | "id": "fhYn-HCiy2Lh", 267 | "colab_type": "code", 268 | "colab": {} 269 | }, 270 | "source": [ 271 | "# 코드 3-6 Input 레이어의 예\n", 272 | "\n", 273 | "inputs = Input(shape=(512,))" 274 | ], 275 | "execution_count": 0, 276 | "outputs": [] 277 | }, 278 | { 279 | "cell_type": "code", 280 | "metadata": { 281 | "id": "SzdLbvPR4t2y", 282 | "colab_type": "code", 283 | "colab": { 284 | "base_uri": "https://localhost:8080/", 285 | "height": 35 286 | }, 287 | "outputId": "bdc2ae27-d64b-4ec2-d249-f9b7c3ea80a9" 288 | }, 289 | "source": [ 290 | "# 코드 3-7 Dense 레이어의 예-1\n", 291 | "\n", 292 | "Dense(64, input_shape=(512,))" 293 | ], 294 | "execution_count": 7, 295 | "outputs": [ 296 | { 297 | "output_type": "execute_result", 298 | "data": { 299 | "text/plain": [ 300 | "" 301 | ] 302 | }, 303 | "metadata": { 304 | "tags": [] 305 | }, 306 | "execution_count": 7 307 | } 308 | ] 309 | }, 310 | { 311 | "cell_type": "code", 312 | "metadata": { 313 | "id": "OCDRXAXH47Gs", 314 | "colab_type": "code", 315 | "colab": { 316 | "base_uri": "https://localhost:8080/", 317 | "height": 448 318 | }, 319 | "outputId": "bb9ccb08-a873-432b-d8ad-b90359874308" 320 | }, 321 | "source": [ 322 | "# 코드 3-8 Dense 레이어의 예-2\n", 323 | "\n", 324 | "Dense(10, activation='relu')(x)\n", 325 | "\n", 326 | "# 아직 준비된 데이터가 없기 때문에 실행할 경우 에러가 나는 것이 맞습니다." 327 | ], 328 | "execution_count": 8, 329 | "outputs": [ 330 | { 331 | "output_type": "error", 332 | "ename": "ValueError", 333 | "evalue": "ignored", 334 | "traceback": [ 335 | "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", 336 | "\u001b[0;31mValueError\u001b[0m Traceback (most recent call last)", 337 | "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/keras/engine/base_layer.py\u001b[0m in \u001b[0;36massert_input_compatibility\u001b[0;34m(self, inputs)\u001b[0m\n\u001b[1;32m 272\u001b[0m \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 273\u001b[0;31m \u001b[0mK\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mis_keras_tensor\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 274\u001b[0m \u001b[0;32mexcept\u001b[0m \u001b[0mValueError\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", 338 | "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py\u001b[0m in \u001b[0;36mis_keras_tensor\u001b[0;34m(x)\u001b[0m\n\u001b[1;32m 497\u001b[0m raise ValueError('Unexpectedly found an instance of type `' +\n\u001b[0;32m--> 498\u001b[0;31m \u001b[0mstr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtype\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;34m'`. '\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 499\u001b[0m 'Expected a symbolic tensor instance.')\n", 339 | "\u001b[0;31mValueError\u001b[0m: Unexpectedly found an instance of type ``. Expected a symbolic tensor instance.", 340 | "\nDuring handling of the above exception, another exception occurred:\n", 341 | "\u001b[0;31mValueError\u001b[0m Traceback (most recent call last)", 342 | "\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m()\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mDense\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m10\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mactivation\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'relu'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 3\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 4\u001b[0m \u001b[0;31m# 아직 준비된 데이터가 없기 때문에 실행할 경우 에러가 나는 것이 맞습니다.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", 343 | "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/keras/engine/base_layer.py\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, inputs, **kwargs)\u001b[0m\n\u001b[1;32m 406\u001b[0m \u001b[0;31m# Raise exceptions in case the input is not compatible\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 407\u001b[0m \u001b[0;31m# with the input_spec specified in the layer constructor.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 408\u001b[0;31m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0massert_input_compatibility\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minputs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 409\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 410\u001b[0m \u001b[0;31m# Collect input shapes to build layer.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", 344 | "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/keras/engine/base_layer.py\u001b[0m in \u001b[0;36massert_input_compatibility\u001b[0;34m(self, inputs)\u001b[0m\n\u001b[1;32m 277\u001b[0m \u001b[0;34m'Received type: '\u001b[0m \u001b[0;34m+\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 278\u001b[0m \u001b[0mstr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtype\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;34m'. Full input: '\u001b[0m \u001b[0;34m+\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 279\u001b[0;31m \u001b[0mstr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minputs\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;34m'. All inputs to the layer '\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 280\u001b[0m 'should be tensors.')\n\u001b[1;32m 281\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n", 345 | "\u001b[0;31mValueError\u001b[0m: Layer dense_5 was called with an input that isn't a symbolic tensor. Received type: . Full input: [array([ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12])]. All inputs to the layer should be tensors." 346 | ] 347 | } 348 | ] 349 | }, 350 | { 351 | "cell_type": "code", 352 | "metadata": { 353 | "id": "hr71451y5Nqo", 354 | "colab_type": "code", 355 | "colab": {} 356 | }, 357 | "source": [ 358 | "# 코드 3-9 간단한 모델의 예\n", 359 | "\n", 360 | "inputs = Input(shape=(3,))\n", 361 | "fc1 = Dense(4, activation='relu')(inputs)\n", 362 | "outputs = Dense(1, activation='softmax')(fc1)\n", 363 | "model = Model(inputs, outputs)" 364 | ], 365 | "execution_count": 0, 366 | "outputs": [] 367 | } 368 | ] 369 | } -------------------------------------------------------------------------------- /Chapter3/3_3_간단한_과일_분류_문제_비틀기(이미지분류기초문제).ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "colab": { 6 | "name": "3_3_간단한_과일_분류_문제_비틀기(이미지분류기초문제).ipynb", 7 | "provenance": [], 8 | "collapsed_sections": [], 9 | "toc_visible": true 10 | }, 11 | "kernelspec": { 12 | "name": "python3", 13 | "display_name": "Python 3" 14 | }, 15 | "accelerator": "GPU" 16 | }, 17 | "cells": [ 18 | { 19 | "cell_type": "markdown", 20 | "metadata": { 21 | "id": "PtOk6uylElp-", 22 | "colab_type": "text" 23 | }, 24 | "source": [ 25 | "# 3.3 간단한 과일 분류 문제 비틀기(이미지 분류 기초 문제)\n", 26 | "\n", 27 | "- 작성자: [김찬란](https://github.com/seriousran)" 28 | ] 29 | }, 30 | { 31 | "cell_type": "code", 32 | "metadata": { 33 | "id": "272hv_NGQ1rK", 34 | "colab_type": "code", 35 | "outputId": "cd9c4a7b-5df7-4e55-8dc1-0574cffd9160", 36 | "colab": { 37 | "base_uri": "https://localhost:8080/", 38 | "height": 192 39 | } 40 | }, 41 | "source": [ 42 | "# 코드 3-56 이미지 분류를 위한 임포트 및 구글 드라이브 마운트\n", 43 | "\n", 44 | "import keras\n", 45 | "from keras.preprocessing.image import ImageDataGenerator\n", 46 | "from keras.models import Model\n", 47 | "from keras.layers import Dense, Dropout, Activation, Flatten, Input, Conv2D, MaxPooling2D\n", 48 | "import matplotlib.pyplot as plt\n", 49 | "\n", 50 | "from google.colab import drive\n", 51 | "drive.mount('/content/drive')" 52 | ], 53 | "execution_count": 1, 54 | "outputs": [ 55 | { 56 | "output_type": "stream", 57 | "text": [ 58 | "Using TensorFlow backend.\n" 59 | ], 60 | "name": "stderr" 61 | }, 62 | { 63 | "output_type": "display_data", 64 | "data": { 65 | "text/html": [ 66 | "

\n", 67 | "The default version of TensorFlow in Colab will soon switch to TensorFlow 2.x.
\n", 68 | "We recommend you upgrade now \n", 69 | "or ensure your notebook will continue to use TensorFlow 1.x via the %tensorflow_version 1.x magic:\n", 70 | "more info.

\n" 71 | ], 72 | "text/plain": [ 73 | "" 74 | ] 75 | }, 76 | "metadata": { 77 | "tags": [] 78 | } 79 | }, 80 | { 81 | "output_type": "stream", 82 | "text": [ 83 | "Go to this URL in a browser: https://accounts.google.com/o/oauth2/auth?client_id=947318989803-6bn6qk8qdgf4n4g3pfee6491hc0brc4i.apps.googleusercontent.com&redirect_uri=urn%3aietf%3awg%3aoauth%3a2.0%3aoob&response_type=code&scope=email%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdocs.test%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive.photos.readonly%20https%3a%2f%2fwww.googleapis.com%2fauth%2fpeopleapi.readonly\n", 84 | "\n", 85 | "Enter your authorization code:\n", 86 | "··········\n", 87 | "Mounted at /content/drive\n" 88 | ], 89 | "name": "stdout" 90 | } 91 | ] 92 | }, 93 | { 94 | "cell_type": "markdown", 95 | "metadata": { 96 | "id": "-jDaCbAdS1Cg", 97 | "colab_type": "text" 98 | }, 99 | "source": [ 100 | "## 3.3.1 컨볼루션과 Conv2D 레이어" 101 | ] 102 | }, 103 | { 104 | "cell_type": "code", 105 | "metadata": { 106 | "id": "GvLNs-3MFB_8", 107 | "colab_type": "code", 108 | "colab": { 109 | "base_uri": "https://localhost:8080/", 110 | "height": 71 111 | }, 112 | "outputId": "7282cacb-4176-4f5e-92d3-8cdf7a725473" 113 | }, 114 | "source": [ 115 | "# 코드 3-57 Conv2D 레이어\n", 116 | "\n", 117 | "Conv2D(64, (3, 3), padding='same', activation='relu')" 118 | ], 119 | "execution_count": 2, 120 | "outputs": [ 121 | { 122 | "output_type": "stream", 123 | "text": [ 124 | "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:66: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.\n", 125 | "\n" 126 | ], 127 | "name": "stdout" 128 | }, 129 | { 130 | "output_type": "execute_result", 131 | "data": { 132 | "text/plain": [ 133 | "" 134 | ] 135 | }, 136 | "metadata": { 137 | "tags": [] 138 | }, 139 | "execution_count": 2 140 | } 141 | ] 142 | }, 143 | { 144 | "cell_type": "markdown", 145 | "metadata": { 146 | "id": "RWE-rcPoS6ke", 147 | "colab_type": "text" 148 | }, 149 | "source": [ 150 | "## 3.3.2 Pooling, DropOut, Flatten 레이어" 151 | ] 152 | }, 153 | { 154 | "cell_type": "code", 155 | "metadata": { 156 | "id": "-LMijVWeFB9w", 157 | "colab_type": "code", 158 | "colab": { 159 | "base_uri": "https://localhost:8080/", 160 | "height": 35 161 | }, 162 | "outputId": "fd19a414-b37d-4758-e8a8-ca0ea6576f15" 163 | }, 164 | "source": [ 165 | "# 코드 3-58 MaxPooling2D 레이어\n", 166 | "\n", 167 | "MaxPooling2D((2,2))" 168 | ], 169 | "execution_count": 3, 170 | "outputs": [ 171 | { 172 | "output_type": "execute_result", 173 | "data": { 174 | "text/plain": [ 175 | "" 176 | ] 177 | }, 178 | "metadata": { 179 | "tags": [] 180 | }, 181 | "execution_count": 3 182 | } 183 | ] 184 | }, 185 | { 186 | "cell_type": "code", 187 | "metadata": { 188 | "id": "ND46DkTKFB7X", 189 | "colab_type": "code", 190 | "colab": { 191 | "base_uri": "https://localhost:8080/", 192 | "height": 35 193 | }, 194 | "outputId": "4d924993-df89-4f62-eea7-b8afa8dc15bf" 195 | }, 196 | "source": [ 197 | "# 코드 3-59 Dropout 레이어\n", 198 | "\n", 199 | "Dropout(0.25)" 200 | ], 201 | "execution_count": 4, 202 | "outputs": [ 203 | { 204 | "output_type": "execute_result", 205 | "data": { 206 | "text/plain": [ 207 | "" 208 | ] 209 | }, 210 | "metadata": { 211 | "tags": [] 212 | }, 213 | "execution_count": 4 214 | } 215 | ] 216 | }, 217 | { 218 | "cell_type": "code", 219 | "metadata": { 220 | "id": "6Xe7tW5mFB5J", 221 | "colab_type": "code", 222 | "colab": { 223 | "base_uri": "https://localhost:8080/", 224 | "height": 35 225 | }, 226 | "outputId": "398c14cc-d14e-4e5a-8352-00673655531e" 227 | }, 228 | "source": [ 229 | "# 코드 3-60 Flatten 레이어\n", 230 | "\n", 231 | "Flatten()" 232 | ], 233 | "execution_count": 5, 234 | "outputs": [ 235 | { 236 | "output_type": "execute_result", 237 | "data": { 238 | "text/plain": [ 239 | "" 240 | ] 241 | }, 242 | "metadata": { 243 | "tags": [] 244 | }, 245 | "execution_count": 5 246 | } 247 | ] 248 | }, 249 | { 250 | "cell_type": "markdown", 251 | "metadata": { 252 | "id": "zhT9Cd26S_g6", 253 | "colab_type": "text" 254 | }, 255 | "source": [ 256 | "## 3.3.3 Dataset Generator" 257 | ] 258 | }, 259 | { 260 | "cell_type": "code", 261 | "metadata": { 262 | "id": "eJR_7TkkFB26", 263 | "colab_type": "code", 264 | "colab": { 265 | "base_uri": "https://localhost:8080/", 266 | "height": 35 267 | }, 268 | "outputId": "24d00832-2cf4-46f2-d4be-0e0a480b94f9" 269 | }, 270 | "source": [ 271 | "# 코드 3-61 셀에서 실행하는 ls 명령어\n", 272 | "\n", 273 | "!ls \"/content/drive/My Drive/dataset/fruit_1\"" 274 | ], 275 | "execution_count": 6, 276 | "outputs": [ 277 | { 278 | "output_type": "stream", 279 | "text": [ 280 | "'Apple Braeburn' Banana\n" 281 | ], 282 | "name": "stdout" 283 | } 284 | ] 285 | }, 286 | { 287 | "cell_type": "code", 288 | "metadata": { 289 | "id": "h_sJsoH4FB0d", 290 | "colab_type": "code", 291 | "colab": {} 292 | }, 293 | "source": [ 294 | "# 코드 3-62 이미지 데이터 생성기\n", 295 | "\n", 296 | "train_datagen = ImageDataGenerator( \n", 297 | " rescale=1./255,\n", 298 | " validation_split=0.2\n", 299 | ") " 300 | ], 301 | "execution_count": 0, 302 | "outputs": [] 303 | }, 304 | { 305 | "cell_type": "code", 306 | "metadata": { 307 | "id": "uqsedeGvUhnj", 308 | "colab_type": "code", 309 | "outputId": "22c44359-5935-41cd-a25d-2a7a89269aa2", 310 | "colab": { 311 | "base_uri": "https://localhost:8080/", 312 | "height": 53 313 | } 314 | }, 315 | "source": [ 316 | "# 코드 3-63 디렉토리로부터 데이터 가져오는 생성기\n", 317 | "\n", 318 | "train_generator = train_datagen.flow_from_directory( \n", 319 | " \"/content/drive/My Drive/dataset/fruit_1\",\n", 320 | " target_size=(64, 64),\n", 321 | " batch_size=32,\n", 322 | " class_mode='categorical',\n", 323 | " subset='training'\n", 324 | ")\n", 325 | "\n", 326 | "validation_generator = train_datagen.flow_from_directory( \n", 327 | " \"/content/drive/My Drive/dataset/fruit_1\",\n", 328 | " target_size=(64, 64),\n", 329 | " batch_size=32,\n", 330 | " class_mode='categorical',\n", 331 | " subset='validation'\n", 332 | ")" 333 | ], 334 | "execution_count": 8, 335 | "outputs": [ 336 | { 337 | "output_type": "stream", 338 | "text": [ 339 | "Found 786 images belonging to 2 classes.\n", 340 | "Found 196 images belonging to 2 classes.\n" 341 | ], 342 | "name": "stdout" 343 | } 344 | ] 345 | }, 346 | { 347 | "cell_type": "code", 348 | "metadata": { 349 | "id": "87ZYYntRHz7f", 350 | "colab_type": "code", 351 | "colab": { 352 | "base_uri": "https://localhost:8080/", 353 | "height": 125 354 | }, 355 | "outputId": "70c5f563-fa91-4474-976e-a2f797459c1c" 356 | }, 357 | "source": [ 358 | "# 코드 3-64 one-hot 인코딩\n", 359 | "\n", 360 | "from keras.utils import to_categorical\n", 361 | "import numpy as np\n", 362 | "data = np.array([0,1,2,1,0])\n", 363 | "print(data)\n", 364 | "\n", 365 | "encoded = to_categorical(data)\n", 366 | "print(encoded)" 367 | ], 368 | "execution_count": 9, 369 | "outputs": [ 370 | { 371 | "output_type": "stream", 372 | "text": [ 373 | "[0 1 2 1 0]\n", 374 | "[[1. 0. 0.]\n", 375 | " [0. 1. 0.]\n", 376 | " [0. 0. 1.]\n", 377 | " [0. 1. 0.]\n", 378 | " [1. 0. 0.]]\n" 379 | ], 380 | "name": "stdout" 381 | } 382 | ] 383 | }, 384 | { 385 | "cell_type": "code", 386 | "metadata": { 387 | "id": "EGxg0JoaHz40", 388 | "colab_type": "code", 389 | "colab": { 390 | "base_uri": "https://localhost:8080/", 391 | "height": 35 392 | }, 393 | "outputId": "6fce7ff1-3272-442f-a018-c24869a31b80" 394 | }, 395 | "source": [ 396 | "# 코드 3-65 one-hot 디코딩\n", 397 | "\n", 398 | "inverted = []\n", 399 | "for en in encoded:\n", 400 | " inverted.append(np.argmax(en))\n", 401 | "print(inverted)" 402 | ], 403 | "execution_count": 10, 404 | "outputs": [ 405 | { 406 | "output_type": "stream", 407 | "text": [ 408 | "[0, 1, 2, 1, 0]\n" 409 | ], 410 | "name": "stdout" 411 | } 412 | ] 413 | }, 414 | { 415 | "cell_type": "markdown", 416 | "metadata": { 417 | "id": "f7Ebfm24TEF5", 418 | "colab_type": "text" 419 | }, 420 | "source": [ 421 | "# 3.3.4. 기초 분류 모델" 422 | ] 423 | }, 424 | { 425 | "cell_type": "code", 426 | "metadata": { 427 | "id": "B6TtjueiVeR9", 428 | "colab_type": "code", 429 | "colab": {} 430 | }, 431 | "source": [ 432 | "# 코드 3-66 이미지 분류 모델 생성 함수\n", 433 | "\n", 434 | "def create_model():\n", 435 | " inputs = Input(shape=(64, 64, 3))\n", 436 | " \n", 437 | " conv_1 = Conv2D(64, (3, 3), padding='same', activation='relu')(inputs)\n", 438 | " pool_1 = MaxPooling2D((2,2))(conv_1)\n", 439 | " conv_2 = Conv2D(32, (3, 3), padding='same', activation='relu')(pool_1)\n", 440 | " pool_2 = MaxPooling2D((2,2))(conv_2)\n", 441 | " drop_2 = Dropout(0.25)(pool_2)\n", 442 | " flat_1 = Flatten()(drop_2)\n", 443 | " fc_1 = Dense(16)(flat_1)\n", 444 | " \n", 445 | " outputs = Dense(2, activation='softmax')(fc_1)\n", 446 | " model = Model(inputs=inputs, outputs=outputs)\n", 447 | " return model" 448 | ], 449 | "execution_count": 0, 450 | "outputs": [] 451 | }, 452 | { 453 | "cell_type": "code", 454 | "metadata": { 455 | "id": "PbpKgAk1douK", 456 | "colab_type": "code", 457 | "outputId": "17348465-7fed-40f5-e88f-a2948b6cbaf4", 458 | "colab": { 459 | "base_uri": "https://localhost:8080/", 460 | "height": 775 461 | } 462 | }, 463 | "source": [ 464 | "# 코드 3-67 모델 생성 및 컴파일\n", 465 | "\n", 466 | "_model_1 = create_model()\n", 467 | "_model_1.compile(loss='categorical_crossentropy',\n", 468 | " optimizer='adam',\n", 469 | " metrics=['accuracy'])\n", 470 | "_model_1.summary()" 471 | ], 472 | "execution_count": 12, 473 | "outputs": [ 474 | { 475 | "output_type": "stream", 476 | "text": [ 477 | "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:541: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.\n", 478 | "\n", 479 | "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:4432: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.\n", 480 | "\n", 481 | "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:4267: The name tf.nn.max_pool is deprecated. Please use tf.nn.max_pool2d instead.\n", 482 | "\n", 483 | "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:148: The name tf.placeholder_with_default is deprecated. Please use tf.compat.v1.placeholder_with_default instead.\n", 484 | "\n", 485 | "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:3733: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.\n", 486 | "Instructions for updating:\n", 487 | "Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.\n", 488 | "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/optimizers.py:793: The name tf.train.Optimizer is deprecated. Please use tf.compat.v1.train.Optimizer instead.\n", 489 | "\n", 490 | "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:3576: The name tf.log is deprecated. Please use tf.math.log instead.\n", 491 | "\n", 492 | "Model: \"model_1\"\n", 493 | "_________________________________________________________________\n", 494 | "Layer (type) Output Shape Param # \n", 495 | "=================================================================\n", 496 | "input_1 (InputLayer) (None, 64, 64, 3) 0 \n", 497 | "_________________________________________________________________\n", 498 | "conv2d_2 (Conv2D) (None, 64, 64, 64) 1792 \n", 499 | "_________________________________________________________________\n", 500 | "max_pooling2d_2 (MaxPooling2 (None, 32, 32, 64) 0 \n", 501 | "_________________________________________________________________\n", 502 | "conv2d_3 (Conv2D) (None, 32, 32, 32) 18464 \n", 503 | "_________________________________________________________________\n", 504 | "max_pooling2d_3 (MaxPooling2 (None, 16, 16, 32) 0 \n", 505 | "_________________________________________________________________\n", 506 | "dropout_2 (Dropout) (None, 16, 16, 32) 0 \n", 507 | "_________________________________________________________________\n", 508 | "flatten_2 (Flatten) (None, 8192) 0 \n", 509 | "_________________________________________________________________\n", 510 | "dense_1 (Dense) (None, 16) 131088 \n", 511 | "_________________________________________________________________\n", 512 | "dense_2 (Dense) (None, 2) 34 \n", 513 | "=================================================================\n", 514 | "Total params: 151,378\n", 515 | "Trainable params: 151,378\n", 516 | "Non-trainable params: 0\n", 517 | "_________________________________________________________________\n" 518 | ], 519 | "name": "stdout" 520 | } 521 | ] 522 | }, 523 | { 524 | "cell_type": "code", 525 | "metadata": { 526 | "id": "oENQ_lpjc7UW", 527 | "colab_type": "code", 528 | "outputId": "f4f61784-987f-4f09-c6be-1098e8702c41", 529 | "colab": { 530 | "base_uri": "https://localhost:8080/", 531 | "height": 539 532 | } 533 | }, 534 | "source": [ 535 | "# 코드 3-68 생성기를 이용한 학습\n", 536 | "\n", 537 | "history = _model_1.fit_generator(\n", 538 | " train_generator,\n", 539 | " epochs=5,\n", 540 | " steps_per_epoch=72,\n", 541 | " validation_data=validation_generator,\n", 542 | " validation_steps=72)" 543 | ], 544 | "execution_count": 13, 545 | "outputs": [ 546 | { 547 | "output_type": "stream", 548 | "text": [ 549 | "WARNING:tensorflow:From /tensorflow-1.15.0/python3.6/tensorflow_core/python/ops/math_grad.py:1424: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n", 550 | "Instructions for updating:\n", 551 | "Use tf.where in 2.0, which has the same broadcast rule as np.where\n", 552 | "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:1033: The name tf.assign_add is deprecated. Please use tf.compat.v1.assign_add instead.\n", 553 | "\n", 554 | "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:1020: The name tf.assign is deprecated. Please use tf.compat.v1.assign instead.\n", 555 | "\n", 556 | "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:3005: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.\n", 557 | "\n", 558 | "Epoch 1/5\n", 559 | "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:190: The name tf.get_default_session is deprecated. Please use tf.compat.v1.get_default_session instead.\n", 560 | "\n", 561 | "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:197: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.\n", 562 | "\n", 563 | "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:207: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead.\n", 564 | "\n", 565 | "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:216: The name tf.is_variable_initialized is deprecated. Please use tf.compat.v1.is_variable_initialized instead.\n", 566 | "\n", 567 | "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:223: The name tf.variables_initializer is deprecated. Please use tf.compat.v1.variables_initializer instead.\n", 568 | "\n", 569 | "72/72 [==============================] - 389s 5s/step - loss: 0.0194 - acc: 0.9902 - val_loss: 5.5738e-07 - val_acc: 1.0000\n", 570 | "Epoch 2/5\n", 571 | "72/72 [==============================] - 6s 82ms/step - loss: 1.1924e-07 - acc: 1.0000 - val_loss: 5.0249e-07 - val_acc: 1.0000\n", 572 | "Epoch 3/5\n", 573 | "72/72 [==============================] - 6s 83ms/step - loss: 1.1921e-07 - acc: 1.0000 - val_loss: 5.1433e-07 - val_acc: 1.0000\n", 574 | "Epoch 4/5\n", 575 | "72/72 [==============================] - 6s 80ms/step - loss: 1.1949e-07 - acc: 1.0000 - val_loss: 5.1650e-07 - val_acc: 1.0000\n", 576 | "Epoch 5/5\n", 577 | "72/72 [==============================] - 6s 78ms/step - loss: 1.1957e-07 - acc: 1.0000 - val_loss: 5.1577e-07 - val_acc: 1.0000\n" 578 | ], 579 | "name": "stdout" 580 | } 581 | ] 582 | }, 583 | { 584 | "cell_type": "code", 585 | "metadata": { 586 | "id": "ZJBJvYSCtAPO", 587 | "colab_type": "code", 588 | "outputId": "263e625e-9f25-4b3f-fad1-7b15e9f970f2", 589 | "colab": { 590 | "base_uri": "https://localhost:8080/", 591 | "height": 295 592 | } 593 | }, 594 | "source": [ 595 | "# 코드 3-69 학습 히스토리 정확도 그래프 그리기\n", 596 | "\n", 597 | "plt.plot(history.history['acc'], '-') \n", 598 | "plt.plot(history.history['val_acc'], \"-.\") \n", 599 | "plt.title('model accuracy') \n", 600 | "plt.ylabel('accuracy') \n", 601 | "plt.xlabel('epoch') \n", 602 | "plt.legend(['train', 'test'], loc='upper left') \n", 603 | "plt.show()" 604 | ], 605 | "execution_count": 14, 606 | "outputs": [ 607 | { 608 | "output_type": "display_data", 609 | "data": { 610 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEWCAYAAABxMXBSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjAsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy8GearUAAAgAElEQVR4nO3deXyV5Z3//9fbEDbZFHCBqFCLCyiC\nRLS11r3Fuu9LdWrn19rNaTszOmrbsVPn61gfP8fpt3axtkNHp1YJWC1arLhA0boRZBFccWsSkH1f\nhITP94/7Dh5CICeQOyfJeT8fjzw457qv6z6f+5Dkk/u+7vO5FBGYmZnla49CB2BmZu2LE4eZmTWL\nE4eZmTWLE4eZmTWLE4eZmTWLE4eZmTWLE4fZTkj6H0n/J8++70s6LeuYzArNicPMzJrFicOsCEjq\nVOgYrONw4rB2L71EdL2kOZLWSfpvSftKelzSGklPSdorp/85kuZJWilpqqTDc7aNlPRKOm4c0LXB\na50laVY69nlJw/OM8UxJMyWtllQl6d8abP9Mur+V6far0/Zukv5T0geSVkl6Lm07SVJ1I+/Daenj\nf5M0QdLvJK0GrpY0WtIL6WsslPQzSZ1zxg+T9KSk5ZIWSfqepP0krZfUN6ff0ZKWSCrN59it43Hi\nsI7iQuB04BDgbOBx4HtAf5Lv828DSDoEeAD4brptEvCopM7pL9FHgP8F9gbGp/slHTsSGAt8DegL\n/AqYKKlLHvGtA/4O6AOcCXxD0nnpfg9K470rjWkEMCsddwcwCvh0GtO/AFvyfE/OBSakr3k/UAf8\nI9AP+BRwKvDNNIaewFPAn4EBwCeBpyPiQ2AqcEnOfq8CHoyIzXnGYR2ME4d1FHdFxKKIqAGeBV6K\niJkRsRF4GBiZ9rsU+FNEPJn+4rsD6Ebyi/k4oBT4SURsjogJwPSc17gG+FVEvBQRdRFxL/BROm6n\nImJqRLwaEVsiYg5J8jox3XwF8FREPJC+7rKImCVpD+Dvge9ERE36ms9HxEd5vicvRMQj6WtuiIgZ\nEfFiRNRGxPskia8+hrOADyPiPyNiY0SsiYiX0m33AlcCSCoBLidJrlaknDiso1iU83hDI897pI8H\nAB/Ub4iILUAVMDDdVhPbVv78IOfxQcA/p5d6VkpaCRyQjtspScdKmpJe4lkFfJ3kL3/SfbzTyLB+\nJJfKGtuWj6oGMRwi6TFJH6aXr/4jjxgA/ggMlTSY5KxuVUS8vIsxWQfgxGHFZgFJAgBAkkh+adYA\nC4GBaVu9A3MeVwG3RkSfnK/uEfFAHq/7e2AicEBE9AbuBupfpwo4uJExS4GNO9i2DuiecxwlJJe5\ncjUsff1L4A1gSET0IrmUlxvDJxoLPD1rqyA567gKn20UPScOKzYVwJmSTk0nd/+Z5HLT88ALQC3w\nbUmlki4ARueM/TXw9fTsQZL2TCe9e+bxuj2B5RGxUdJokstT9e4HTpN0iaROkvpKGpGeDY0F7pQ0\nQFKJpE+lcypvAV3T1y8FfgA0NdfSE1gNrJV0GPCNnG2PAftL+q6kLpJ6Sjo2Z/t9wNXAOThxFD0n\nDisqEfEmyV/Od5H8RX82cHZEbIqITcAFJL8gl5PMh/whZ2wl8FXgZ8AKYH7aNx/fBG6RtAa4mSSB\n1e/3b8AXSJLYcpKJ8aPSzdcBr5LMtSwHbgf2iIhV6T5/Q3K2tA7Y5i6rRlxHkrDWkCTBcTkxrCG5\nDHU28CHwNnByzva/kkzKvxIRuZfvrAjJCzmZWT4kPQP8PiJ+U+hYrLCcOMysSZKOAZ4kmaNZU+h4\nrLB8qcrMdkrSvSSf8fiuk4aBzzjMzKyZfMZhZmbNUhSFz/r16xeDBg0qdBhmZu3KjBkzlkZEw88H\nFUfiGDRoEJWVlYUOw8ysXZHU6K3XvlRlZmbN4sRhZmbN4sRhZmbNUhRzHI3ZvHkz1dXVbNy4sdCh\nZKpr166UlZVRWuo1d8ysZRRt4qiurqZnz54MGjSIbYuhdhwRwbJly6iurmbw4MGFDsfMOohML1VJ\nGitpsaS5O9guST+VNF/Jsp9H52z7kqS3068v5bSPkvRqOuan2sXf+hs3bqRv374dNmkASKJv374d\n/qzKzFpX1nMc/wOM2cn2M4Ah6dc1JOsFIGlv4IfAsSRlrX+oj9eM/iVJhdL6cTvb/0515KRRrxiO\n0cxaV6aXqiJimqRBO+lyLnBfuuLai5L6SNofOAl4MiKWA0h6EhgjaSrQKyJeTNvvA84jWa85G0vf\nbrpP117QY9+P+3ffG7r3hbpaWPFe0+Mb9u+xD3TtDZs3wqqqpsc37N9zf+jSAz5aC2sWwtrF8Nvr\ndjz+1JvhwGPhby/B07fA2T+BfkPgzcfh+Z81/fpp/6oXHoIXfsajh/6YjaV9GLr4MYYtfqzJ4Q37\njz/ibgBG1fyOT6x4rsnxuf0HrHmVRw+7HYDPfPBz9l/z6k7HbuzUe5v+XWtX8dTB3wPgtHf+g702\n/G2n41d0O3Cb/hs79ea5g74FwNlv3EDX2lU7Hb+w55Hb9F/Q80hmDLwSgIvnfn2nYwHe3esz2/Sf\nt89ZvLbPWXTdvJKz37yxyfEN+88Y8EXe3fsE9trwAae9c1uT4xv2f+7Ab7Kw13D2Xz2Hz/ztF02O\nb9j/qYNvYkW3g/jE8mcZteD+Jsc37O/vvW2/98YfcTdf+vQg+vZoaqmW5in0HMdAtl3esjpt21l7\ndSPt25F0DclZDAceeGBjXQpq5arV/P6hR/nm33+xWeO+cNlX+P2v7qRP714ZRbbr/vfFDzhl5QZ+\n89y7rKAXF+6xiD4lG5oc17D/XVPmA/DVkqV02aPp8bn9R2otdy1MnnctWcHRTYxfESXb9N9La7jr\nb8nz/TutYrB2Pv69Fau26b8itnDXu8nzwzutZa8mxr+yfMU2/WcuXsqv30qef7q06WP/6/Jt+z+1\nbBEPzZvPXqzm6DzGN+z/2LIFPL1lPp/QAg7v1PT4hv0nLK3mlejO0apmcB7jG/b/3Usf8G5s5tQ9\nFrBfHt87Dfv7e2/b7727psznnBEDWzxxZF7kMD3jeCwijmhk22PAjyPiufT508ANJGccXSPi/6Tt\n/0qybvTUtP9pafsJwA0RcdbOYigvL4+Gnxx//fXXOfzww3fn0HbL+++/z1lnncXcudtO/9TW1tKp\nU8vm89Y41vmL13DandP43hcO45rPNrbSqZm1N5JmRER5w/ZCf46jhmS953pladvO2ssaaW93brzx\nRt555x1GjBjBMcccwwknnMA555zD0KFDATjvvPMYNWoUw4YN45577tk6btCgQSxdupT333+fww8/\nnK9+9asMGzaMz33uc2zY0PRfSFmpqKym0x7i/JFlTXc2s3at0JeqJgLXSnqQZCJ8VUQslPQE8B85\nE+KfA26KiOWSVks6DngJ+DuSJUB3y48encdrC1bv7m62MXRAL3549rAdbv/xj3/M3LlzmTVrFlOn\nTuXMM89k7ty5W2+bHTt2LHvvvTcbNmzgmGOO4cILL6Rv377b7OPtt9/mgQce4Ne//jWXXHIJDz30\nEFdeeWWLHkc+Ntdt4Q+vVHPKYfvQv2fLnhKbWduTaeKQ9ADJZad+kqpJ7pQqBYiIu4FJJGstzwfW\nA19Oty2X9O8k6ywD3FI/UU6yzvL/AN1IJsWzmxhvRaNHj97msxY//elPefjhhwGoqqri7bff3i5x\nDB48mBEjRgAwatQo3n///VaLN9czbyxm6dpNXHrMAU13NrN2L+u7qi5vYnsA39rBtrHA2EbaK4Ht\n5kt2x87ODFrLnnvuufXx1KlTeeqpp3jhhRfo3r07J510UqOfxejS5eO/7ktKSgp2qWp8ZRX79OzC\niYdsV33ZzDqgQs9xFK2ePXuyZk3jq3CuWrWKvfbai+7du/PGG2/w4osvtnJ0+Vu8eiNT3lzChaPK\n6FTibyezYlDoOY6i1bdvX44//niOOOIIunXrxr777rt125gxY7j77rs5/PDDOfTQQznuuOMKGOnO\nTXilmrotwSXlvkxlViyKYs3xtng7bmvK6lgjglP+8y/079GFiq9/qsX3b2aF1VZvx7V2bPr7K3hv\n6Tou8aS4WVFx4rBdVlFZRY8unfjCkfsVOhQza0VOHLZL1mzczJ/mLOTso/ane2dPlZkVEycO2yV/\nmrOQDZvruNiT4mZFx4nDdsm4yiqG7NODkQf0KXQoZtbKnDis2d5etIaZf1vJJeUHeL0PsyLkxFEg\nK1eu5Be/aHq9gsb85Cc/Yf369S0cUf4qKquSgoZHN1rR3sw6OCeOAmmviSMpaFjDqYfvQ78WrvFv\nZu2Db4cpkNyy6qeffjr77LMPFRUVfPTRR5x//vn86Ec/Yt26dVxyySVUV1dTV1fHv/7rv7Jo0SIW\nLFjAySefTL9+/ZgyZUqrxv3064tZts4FDc2KmRNHvd+e2XSfQz4Px3/74/4jroCRX4R1y6Di77bt\n++U/7XRXuWXVJ0+ezIQJE3j55ZeJCM455xymTZvGkiVLGDBgAH/6U7KvVatW0bt3b+68806mTJlC\nv379duVId0t9QcPPDnFBQ7Ni5UtVbcDkyZOZPHkyI0eO5Oijj+aNN97g7bff5sgjj+TJJ5/khhtu\n4Nlnn6V3794FjXPR6o1MeXMxF7mgoVlR8xlHvSbOEHbaf8++zR+fIyK46aab+NrXvrbdtldeeYVJ\nkybxgx/8gFNPPZWbb755l19nd02YUc2WwAUNzYqc/2wskNyy6p///OcZO3Ysa9euBaCmpobFixez\nYMECunfvzpVXXsn111/PK6+8st3Y1hIRjK+sYvTgvRnUb8+mB5hZh5X1CoBjgP8LlAC/iYgfN9h+\nEMliTf2B5cCVEVGdbrsdqJ94+PeIGJe2nwLcAXQGZgD/X0TUZnkcWcgtq37GGWdwxRVX8KlPJRVm\ne/Towe9+9zvmz5/P9ddfzx577EFpaSm//OUvAbjmmmsYM2YMAwYMaLXJ8ZffW877y9bzD6cMaZXX\nM7O2K7Oy6pJKgLeA04FqkmVgL4+I13L6jAcei4h704Tw5Yi4StKZwHeBM4AuwFTgVGAt8AFwakS8\nJekW4IOI+O+dxeKy6rt/rP9cMZsn5n3Iy98/1bWpzIpEIcqqjwbmR8S7EbEJeBA4t0GfocAz6eMp\nOduHAtMiojYi1gFzgDFAX2BTRLyV9nsSuDDDYzCSgoaTXl3I2UcNcNIws0wTx0CgKud5ddqWazZw\nQfr4fKCnpL5p+xhJ3SX1A04GDgCWAp0k1WfAi9L27Ui6RlKlpMolS5a0yAEVq8fSgoaXlJcVOhQz\nawMKPTl+HXCipJnAiUANUBcRk4FJwPPAA8ALaXsAlwH/JellYA1Q19iOI+KeiCiPiPL+/Rv/zEEx\nrH7YEsc4bnoVh+zbgxEuaGhmZJs4atj2bKAsbdsqIhZExAURMRL4ftq2Mv331ogYERGnAyKZLyEi\nXoiIEyJiNDCtvr25unbtyrJlyzp08ogIli1bRteuXXd5H28tWsOsKhc0NLOPZXnBejowRNJgkoRx\nGXBFbof0MtTyiNgC3ERyh1X9xHqfiFgmaTgwHJicbtsnIhZL6gLcANy6K8GVlZVRXV1NR7+M1bVr\nV8rKdv0SU8X0tKDhSBc0NLNEZokjImolXQs8QXI77tiImJfeCVUZEROBk4DbJAXJ2cO30uGlwLPp\nX7irSW7Trb/l9npJZ5GcLf0yIp5hF5SWljJ48OBdPLrisKl2Cw/PrOG0w/elrwsamlkq01tkImIS\nyVxFbtvNOY8nABMaGbeR5M6qxvZ5PXB9y0ZqjXnmjUUuaGhm2yn05Li1YRWV1ezXqyufPcQFDc3s\nY04c1qgPV21k6puLuXDUQEr28KS4mX3MicMa9dArSUHDi0f5MpWZbcuJw7YTEVRUVnGsCxqaWSOc\nOGw7L723nA+WrfekuJk1yonDtlNRWUXPLp0444j9Cx2KmbVBThy2jdX1BQ1HDKBb55JCh2NmbZAT\nh23jsdkL2bh5i1f5M7MdcuKwbYyrrOLQfXtyVFlh1zc3s7bLicO2evPDNcyuWsnF5WUuaGhmO+TE\nYVtVVFZRWiIuONrrbpjZjjlxGPBxQcPTh+7L3nt2LnQ4ZtaGOXEYAE+/vojl6zZxsSfFzawJThwG\nJJPi+/XqymeHuKChme2cE4excNUGpr21hItGlbmgoZk1yYnDeGhGWtCw3JPiZta0TBOHpDGS3pQ0\nX9KNjWw/SNLTkuZImiqpLGfb7ZLmpl+X5rSfKukVSbMkPSfpk1keQ0e3ZUtQUVnNcZ/Ym4P6uqCh\nmTUts8SRrhv+c+AMktX8LpfUcFW/O4D7ImI4cAtwWzr2TOBoYARwLHCdpF7pmF8CX4yIEcDvgR9k\ndQzF4KX3lvO35S5oaGb5y/KMYzQwPyLejYhNwIPAuQ36DAXq1wyfkrN9KDAtImojYh0wBxiTbgug\nPon0BhZkFH9RGJ8WNBwzzAUNzSw/WSaOgUBVzvPqtC3XbOCC9PH5QE9JfdP2MZK6S+oHnAzU/0n8\nFWCSpGrgKuDHjb24pGskVUqqXLJkSYscUEezeuNmJs1dyDkuaGhmzVDoyfHrgBMlzQROBGqAuoiY\nDEwCngceAF4A6tIx/wh8ISLKgN8Cdza244i4JyLKI6K8f3/fYtqYR2cvcEFDM2u2LBNHDR+fJQCU\npW1bRcSCiLggIkYC30/bVqb/3hoRIyLidEDAW5L6A0dFxEvpLsYBn87wGDq0iulVHLZfT4a7oKGZ\nNUOWiWM6METSYEmdgcuAibkdJPWTVB/DTcDYtL0kvWSFpOHAcGAysALoLemQdMzpwOsZHkOH9caH\nq5ldvYqLyw9wQUMza5ZOWe04ImolXQs8AZQAYyNinqRbgMqImAicBNwmKYBpwLfS4aXAs+kvtNXA\nlRFRCyDpq8BDkraQJJK/z+oYOrKK6dWUlojzRzacdjIz27nMEgdAREwimavIbbs55/EEYEIj4zaS\n3FnV2D4fBh5u2UiLS1LQsJrPDd3PBQ3NrNkKPTluBfDU64tYsX6zPyluZrvEiaMIjZtexf69u3KC\nCxqa2S5w4igyC1ZuYNrbLmhoZrvOiaPIPDSjmgi4eJQ/u2Fmu8aJo4hs2RKMn1HNpz7RlwP7di90\nOGbWTjlxFJEX31vmgoZmttucOIrI+MpqenbtxJgj9it0KGbWjjlxFIlVGzYz6dWFnDtiAF1LXdDQ\nzHadE0eReHT2Aj6qdUFDM9t9ThxFoqIyKWh45EAXNDSz3ePEUQReX7iaOdWruPQYFzQ0s93nxFEE\nKiqr6FyyB+eNcEFDM9t9Thwd3Ee1dTwys4bTh+3LXi5oaGYtwImjg3vqtcWsWL/Zk+Jm1mKcODq4\ncZVVDOjdlc98sl+hQzGzDsKJowNbsHIDz7qgoZm1sEwTh6Qxkt6UNF/SjY1sP0jS05LmSJoqqSxn\n2+2S5qZfl+a0PytpVvq1QNIjWR5DezahvqChL1OZWQvKLHFIKgF+DpxBsprf5ZIarup3B3BfRAwH\nbgFuS8eeCRwNjACOBa6T1AsgIk6IiBERMQJ4AfhDVsfQniUFDav49MF9OWBvFzQ0s5aT5RnHaGB+\nRLwbEZuAB4FzG/QZCjyTPp6Ss30oMC0iaiNiHTAHGJM7ME0kpwA+42jEi+8uo2r5Bhc0NLMWl2Xi\nGAhU5TyvTttyzQYuSB+fD/SU1DdtHyOpu6R+wMlAw9+A5wFPR8Tqxl5c0jWSKiVVLlmyZDcPpf2p\nqKyiZ9dOfH6YCxqaWcsq9OT4dcCJkmYCJwI1QF1ETAYmAc8DD5BckqprMPbydFujIuKeiCiPiPL+\n/YtridRVGzbz+NwPOW/EQBc0NLMWl2XiqGHbs4SytG2riFgQERdExEjg+2nbyvTfW9O5jNMBAW/V\nj0vPQkYDf8ow/nZrogsamlmGskwc04EhkgZL6gxcBkzM7SCpn6T6GG4CxqbtJeklKyQNB4YDk3OG\nXgQ8FhEbM4y/3aqYXsXh+/fiiIG9Ch2KmXVAeSUOSX+QdGbOL/kmRUQtcC3wBPA6UBER8yTdIumc\ntNtJwJuS3gL2BW5N20uBZyW9BtwDXJnur95l7OQyVTF7bcFqXq1ZxaXlZS5oaGaZ6JRnv18AXwZ+\nKmk88NuIeLOpQRExiWSuIrft5pzHE4AJjYzbSHJn1Y72e1KecRed+oKG57qgoZllJK8ziIh4KiK+\nSPLZiveBpyQ9L+nLkkqzDNDy91FtHY/MquFzLmhoZhnK+9JTOudwNfAVYCbwf0kSyZOZRGbN9uRr\ni1jpgoZmlrG8LlVJehg4FPhf4OyIWJhuGiepMqvgrHnGTa9iYJ9uHO+ChmaWoXznOH4aEVMa2xAR\n5S0Yj+2impUbeG7+Uv7hlCEuaGhmmcr3UtVQSX3qn0jaS9I3M4rJdsGEyrSg4aiypjubme2GfBPH\nV+s/mAcQESuAr2YTkjVXfUHD4z/pgoZmlr18E0eJcj4UkFa+9W07bcQL7y6jesUGT4qbWavId47j\nzyQT4b9Kn38tbbM2oKKyil4uaGhmrSTfxHEDSbL4Rvr8SeA3mURkzbJqfVLQ8LJjDnBBQzNrFXkl\njojYAvwy/bI2ZOLsGja5oKGZtaJ8P8cxhGR1vqFA1/r2iPhERnFZnsZVVjF0/14cMbB3oUMxsyKR\n7+T4b0nONmpJFlW6D/hdVkFZfuYtWMXcmtVe5c/MWlW+iaNbRDwNKCI+iIh/A87MLizLx/jKajp3\n2oNzRwwodChmVkTynRz/KC2p/raka0kWZOqRXVjWlI2b63h4Zg2fH7Yffbr7zmgzaz35nnF8B+gO\nfBsYBVwJfCmroKxpT762iFUbNnNJuT8pbmatq8nEkX7Y79KIWBsR1RHx5Yi4MCJezGPsGElvSpov\n6cZGth8k6WlJcyRNlVSWs+12SXPTr0tz2iXpVklvSXpd0rebcbwdRkVlWtDwYBc0NLPW1WTiiIg6\n4DPN3XGacH4OnEFyN9blkhouznQHcF9EDAduIblzC0lnkpRsHwEcC1wnqX4d1KtJ1jI/LCIOBx5s\nbmztXfWK9Tw3fykXjSpjDxc0NLNWlu8cx0xJE4HxwLr6xoj4w07GjAbmR8S7AJIeBM4FXsvpMxT4\np/TxFOCRnPZp6XKxtZLmAGOACpIPIV6RfraEiFic5zF0GBNmVANwsS9TmVkB5DvH0RVYBpwCnJ1+\nndXEmIFAVc7z6rQt12zggvTx+UDPdMGo2cAYSd0l9SO5Bbj+ntODgUslVUp6PP2MSdHYsiUYX1nN\n8Qf3o2wvFzQ0s9aX7yfHv5zR618H/EzS1cA0kru16iJisqRjgOeBJcALQF06pguwMSLKJV0AjAVO\naLhjSdcA1wAceOCBGYXf+p5/Zxk1KzdwwxmHFToUMytS+X5y/LdANGyPiL/fybAaPj5LAChL23LH\nLyA945DUA7iwvnx7RNwK3Jpu+z3wVjqsGqi/RPYwyYcTtxMR9wD3AJSXl28Xe3tVUVlF726lfG7o\nvoUOxcyKVL5zHI/lPO5KcllpQRNjpgNDJA0mSRiXAVfkdkgvQy1P5ytuIjl7qJ9Y7xMRyyQNB4YD\nk9Nhj5BcunoPOJGPE0qHt2r9Zv4870Mud0FDMyugfC9VPZT7XNIDwHNNjKlNPyz4BFACjI2IeZJu\nASojYiJwEnCbpCC5VPWtdHgp8Gy6BMhq4Mp0ohzgx8D9kv4RWAt8JZ9j6Aj+WF/Q0CVGzKyA8j3j\naGgIsE9TnSJiEjCpQdvNOY8nABMaGbeR5M6qxva5kiItdzJuehXDBvRi2AAXNDSzwsl3jmMN285x\nfEiyRoe1krk1q5i3YDW3nDus0KGYWZHL91JVz6wDsZ0bX1mVFDQ8quEdzWZmrSuvz3FIOl9S75zn\nfSSdl11Ylmvj5joembWAMcP2o3f30kKHY2ZFLt8PAP4wIlbVP0nnGX6YTUjW0OStBQ09KW5mhZdv\n4mis365OrFszVUxPChp++uC+hQ7FzCzvxFEp6U5JB6dfdwIzsgzMElXL1/PXd5ZycbkLGppZ25Bv\n4vgHYBMwjqQa7UY+/syFZai+oOFFo1zQ0MzahnzvqloHbLeehmVry5ZgwoxqPvNJFzQ0s7Yj37uq\nnpTUJ+f5XpKeyC4sA/jrO0upWbnBk+Jm1qbke6mqX33xQYCIWEEenxy33VNRWU2f7qV8bpgLGppZ\n25Fv4tgiaWttckmDaKRarrWcles38cS8DzlvxEC6dHJBQzNrO/K9pfb7wHOS/gKIZP2LazKLyvjj\nrAVJQUNfpjKzNibfyfE/SyonSRYzSUqbb8gysGI3bnoVRwzsxdABvZrubGbWivItcvgV4DskizHN\nAo4jWZXvlOxCK15za1bx2sLV/LsLGppZG5TvHMd3gGOADyLiZGAksHLnQ2xXVaQFDc9xQUMza4Py\nTRwb0zUykNQlIt4ADs0urOK1cXMdj8ys4YwjXNDQzNqmfBNHdfo5jkeAJyX9EfigqUGSxkh6U9J8\nSdt9gFDSQZKeljRH0lRJZTnbbpc0N/26NKf9fyS9J2lW+jUiz2NoF56Y9yGrN9Z6UtzM2qx8J8fP\nTx/+m6QpQG/gzzsbk64b/nPgdKAamC5pYkS8ltPtDuC+iLhX0inAbcBVks4EjgZGAF2AqZIej4jV\n6bjr09UDO5yKyirK9urGpz7hgoZm1jble8axVUT8JSImRsSmJrqOBuZHxLtp3weBcxv0GQo8kz6e\nkrN9KDAtImrTcidzgDHNjbW9qVq+nr/OX8bFow5wQUMza7OanTiaYSBQlfO8Om3LNRu4IH18PtBT\nUt+0fYyk7pL6AScDuddubk0vb/2XpC6NvbikayRVSqpcsmRJSxxP5sbPqEaCi8pd0NDM2q4sE0c+\nrgNOlDQTOBGoAeoiYjIwCXgeeIDk1t+6dMxNwGEkd3ntzQ7WPo+IeyKiPCLK+/fvn+1RtIC6LcGE\nyipOGNKfgX26FTocM7MdyjJx1LDtWUJZ2rZVRCyIiAsiYiTJp9PrVxckIm6NiBERcTrJp9XfStsX\nRuIj4Lckl8Tavb/OX8qCVRu5xGcbZtbGZZk4pgNDJA2W1Bm4DJiY20FSP0n1MdwEjE3bS9JLVkga\nDgwHJqfP90//FXAeMDfDY//nb60AABBBSURBVGg1FZVV9OleyulDXdDQzNq2zJZ/jYhaSdcCTwAl\nwNiImCfpFqAyIiYCJwG3SQpgGh8vDlUKPJvkBlYDV0ZEbbrtfkn9Sc5CZgFfz+oYWsuKdZuYPG8R\nVxx7oAsamlmbl+m64RExiWSuIrft5pzHE4DtbqtNP2w4dAf77HBlTv44q4ZNdS5oaGbtQ6Enx4te\nRDCuspojB/Z2QUMzaxecOApsbs1qXl+4mkuO8dmGmbUPThwFVlFZRZdOe3DOUQMKHYqZWV6cOApo\n4+Y6HpmVFjTs5oKGZtY+OHEU0BPzPmSNCxqaWTvjxFFA46ZXccDe3TjOBQ3NrB1x4iiQquXref4d\nFzQ0s/bHiaNAxldWJQUNR7nEiJm1L04cBVC3JZgwo5rPDunPABc0NLN2xomjAJ7bWtDQk+Jm1v44\ncRRARWUVe3Uv5bSh+xQ6FDOzZnPiaGUr1m3iyXmLOG/kQBc0NLN2yYmjlT08MyloeKlLjJhZO+XE\n0YoigorKKoaX9eaw/VzQ0MzaJyeOVvRqzSre+HCNJ8XNrF1z4mhF9QUNz3ZBQzNrxzJNHJLGSHpT\n0nxJNzay/SBJT0uaI2mqpLKcbbdLmpt+XdrI2J9KWptl/C1p4+Y6/jhrAV84cn8XNDSzdi2zxCGp\nBPg5cAbJan6XS2q4qt8dwH0RMRy4BbgtHXsmcDQwAjgWuE5Sr5x9lwN7ZRV7Fv48NyloeHG5Pylu\nZu1blmcco4H5EfFuRGwCHgTObdBnKPBM+nhKzvahwLSIqI2IdcAcYAxsTUj/P/AvGcbe4sZNr+LA\nvbtz3GAXNDSz9i3LxDEQqMp5Xp225ZoNXJA+Ph/oKalv2j5GUndJ/YCTgfoZ5WuBiRGxcGcvLuka\nSZWSKpcsWbKbh7J7/rZsPS+8u4xLystc0NDM2r1CT45fB5woaSZwIlAD1EXEZGAS8DzwAPACUCdp\nAHAxcFdTO46IeyKiPCLK+/fvn9kB5GP8jCr2EFzogoZm1gFkmThq+PgsAaAsbdsqIhZExAURMRL4\nftq2Mv331ogYERGnAwLeAkYCnwTmS3of6C5pfobHsNu2FjQ8pD/793ZBQzNr/7JMHNOBIZIGS+oM\nXAZMzO0gqZ+k+hhuAsam7SXpJSskDQeGA5Mj4k8RsV9EDIqIQcD6iPhkhsew2559ewkLXdDQzDqQ\nTlntOCJqJV0LPAGUAGMjYp6kW4DKiJgInATcJimAacC30uGlwLOSAFYDV0ZEbVaxZml8ZTV779mZ\n0w7ft9ChmJm1iMwSB0BETCKZq8htuznn8QRgQiPjNpLcWdXU/nu0QJiZWb5uE5Nf+5CrjhtE506F\nnk4yM2sZ/m2WoYdn1rC5LlzQ0Mw6FCeOjEQE4yurOKqsN4fu17PQ4ZiZtRgnjozMqU4LGvpsw8w6\nGCeOjFRUVtG11AUNzazjceLIwIZNdUyctYAvHLE/vbq6oKGZdSxOHBn487yFrPmo1pepzKxDcuLI\nwLjpVRzUtzvHDt670KGYmbU4J44W9sGydbz47nIuKT+A9AOMZmYdihNHCxtfWZ0UNDzaBQ3NrGNy\n4mhB9QUNTzykP/v17lrocMzMMuHE0YKmvb2ED1e7oKGZdWxOHC1ofGUVe+/ZmVNd0NDMOjAnjhay\nbO1HPPnaIs4fOdAFDc2sQ/NvuBZSX9DQl6nMrKNz4mgBEUFFZRVHHdDHBQ3NrMNz4mgBs6tX8dai\ntVzqsw0zKwKZJg5JYyS9KWm+pBsb2X6QpKclzZE0VVJZzrbbJc1Nvy7Naf9vSbPTMRMkFXwxp/qC\nhmcdtX+hQzEzy1xmiUNSCfBz4AyS1fwul9RwVb87gPsiYjhwC3BbOvZM4GhgBHAscJ2kXumYf4yI\no9IxfwOuzeoY8rFhUx2PzlrAF450QUMzKw5ZnnGMBuZHxLsRsQl4EDi3QZ+hwDPp4yk524cC0yKi\nNiLWAXOAMQARsRpAST2PbkBkeAxNenxuUtDQl6nMrFhkmTgGAlU5z6vTtlyzgQvSx+cDPSX1TdvH\nSOouqR9wMrD1N7Ok3wIfAocBdzX24pKukVQpqXLJkiUtcTyNGje9ikF9uzPaBQ3NrEgUenL8OuBE\nSTOBE4EaoC4iJgOTgOeBB4AXgLr6QRHxZWAA8DpwacOdpn3uiYjyiCjv379/JsG/v3QdL723nItd\n0NDMikiWiaOGnLMEoCxt2yoiFkTEBRExEvh+2rYy/ffWiBgREacDAt5qMLaO5PLXhdkdws6Nn1Hl\ngoZmVnSyTBzTgSGSBkvqDFwGTMztIKmfpPoYbgLGpu0l6SUrJA0HhgOTlfhk2i7gHOCNDI9hh+oL\nGp506D4uaGhmRaVTVjuOiFpJ1wJPACXA2IiYJ+kWoDIiJgInAbdJCmAa8K10eCnwbHr5ZzVwZbq/\nPYB70zusRDIX8o2sjmFnpr21hEWrP+JH5/hsw8yKS2aJAyAiJpHMVeS23ZzzeAIwoZFxG0nurGrY\nvgU4vuUjbb5x06vou2dnTjnMBQ3NrLgUenK8XVq29iOeet0FDc2sOPm33i54eGYNtVuCS47xZzfM\nrPg4cTRTRDBuehUjDujDIfu6oKGZFR8njmaaVbWStxev5VKfbZhZkXLiaKaKymq6lZZw1nAXNDSz\n4uTE0QzrN9Xy6OykoGFPFzQ0syLlxNEMj7/6IWs/qvVlKjMrak4czTCusorB/fbkmEF7FToUM7OC\nceLI03tL1/Hye8u5uLzMBQ3NrKg5ceRpfKULGpqZgRNHXmrrtvDQK9WcfOg+7NvLBQ3NrLg5ceRh\n2ttJQcOLvcqfmZkTRz7GTa+iX4/OnHr4PoUOxcys4Jw4mrB07Uc8/fpizh85kNISv11mZv5N2ISH\nX0kLGvoylZkZ4MSxUxFBRWUVIw/swxAXNDQzAzJOHJLGSHpT0nxJNzay/SBJT0uaI2mqpLKcbbdL\nmpt+XZrTfn+6z7mSxkrKrPbHzPqChj7bMDPbKrPEIakE+DlwBslqfpdLariq3x3AfRExHLgFuC0d\neyZwNDACOBa4Ll0uFuB+4DDgSKAb8JWsjmF8ZVVS0PCoAVm9hJlZu5PlGcdoYH5EvBsRm4AHgXMb\n9BkKPJM+npKzfSgwLSJqI2IdMAcYA8lytJECXgYy+0TegXvvydXHD6JHl0xX2DUza1eyTBwDgaqc\n59VpW67ZwAXp4/OBnpL6pu1jJHWX1A84GdjmelF6ieoq4M+NvbikayRVSqpcsmTJLh3AN046mBvG\nHLZLY83MOqpCT45fB5woaSZwIlAD1EXEZGAS8DzwAPACUNdg7C9IzkqebWzHEXFPRJRHRHn//v0z\nOwAzs2KTZeKoYduzhLK0bauIWBARF0TESOD7advK9N9bI2JERJwOCHirfpykHwL9gX/KMH4zM2tE\nloljOjBE0mBJnYHLgIm5HST1k1Qfw03A2LS9JL1khaThwHBgcvr8K8DngcsjYkuG8ZuZWSMySxwR\nUQtcCzwBvA5URMQ8SbdIOiftdhLwpqS3gH2BW9P2UuBZSa8B9wBXpvsDuDvt+4KkWZJuzuoYzMxs\ne0puTurYysvLo7KystBhmJm1K5JmRER5w/ZCT46bmVk748RhZmbN4sRhZmbNUhRzHJKWAB/s4vB+\nwNIWDKelOK7mcVzN47iap6PGdVBEbPdBuKJIHLtDUmVjk0OF5riax3E1j+NqnmKLy5eqzMysWZw4\nzMysWZw4mnZPoQPYAcfVPI6reRxX8xRVXJ7jMDOzZvEZh5mZNYsTh5mZNYsTRyqP9dG7SBqXbn9J\n0qA2EtfVkpakBR9npdWDs45prKTFkubuYLsk/TSNeY6ko7OOKc+4TpK0Kue9apUCmZIOkDRF0muS\n5kn6TiN9Wv09yzOuVn/PJHWV9LKk2WlcP2qkT6v/POYZV6v/POa8domkmZIea2Rby75fEVH0X0AJ\n8A7wCaAzyQqEQxv0+SZwd/r4MmBcG4nrauBnrfx+fZZkTfi5O9j+BeBxknVUjgNeaiNxnQQ8VoDv\nr/2Bo9PHPUnWlmn4/9jq71mecbX6e5a+Bz3Sx6XAS8BxDfoU4ucxn7ha/ecx57X/Cfh9Y/9fLf1+\n+Ywjkc/66OcC96aPJwCnSlIbiKvVRcQ0YPlOupwL3BeJF4E+kvZvA3EVREQsjIhX0sdrSJYZaLiM\ncqu/Z3nG1erS92Bt+rQ0/Wp4F0+r/zzmGVdBSCoDzgR+s4MuLfp+OXEk8lkffWufSNYGWQX0bQNx\nAVyYXt6YIOmARra3tnzjLoRPpZcaHpc0rLVfPL1EMJLkr9VcBX3PdhIXFOA9Sy+7zAIWA09GxA7f\nr1b8ecwnLijMz+NPgH8BdrS4XYu+X04c7d+jwKCIGA48ycd/Vdj2XiGpvXMUcBfwSGu+uKQewEPA\ndyNidWu+9s40EVdB3rOIqIuIESRLTo+WdERrvG5T8oir1X8eJZ0FLI6IGVm/Vj0njkST66Pn9pHU\nCegNLCt0XBGxLCI+Sp/+BhiVcUz5yOf9bHURsbr+UkNETAJKJfVrjdeWVEryy/n+iPhDI10K8p41\nFVch37P0NVcCU4AxDTYV4uexybgK9PN4PHCOpPdJLmefIul3Dfq06PvlxJFocn309PmX0scXAc9E\nOtNUyLgaXAc/h+Q6daFNBP4uvVPoOGBVRCwsdFCS9qu/ritpNMn3f+a/bNLX/G/g9Yi4cwfdWv09\nyyeuQrxnkvpL6pM+7gacDrzRoFur/zzmE1chfh4j4qaIKIuIQSS/I56JiCsbdGvR96vTrg7sSCKi\nVlL9+uglwNhI10cHKiNiIskP2P9Kmk8yAXtZG4nr20rWcK9N47o667gkPUByt00/SdXAD0kmComI\nu4FJJHcJzQfWA1/OOqY847oI+IakWmADcFkrJH9I/iK8Cng1vT4O8D3gwJzYCvGe5RNXId6z/YF7\nJZWQJKqKiHis0D+PecbV6j+PO5Ll++WSI2Zm1iy+VGVmZs3ixGFmZs3ixGFmZs3ixGFmZs3ixGFm\nZs3ixGHWximpULtdxVOzQnHiMDOzZnHiMGshkq5M12uYJelXaUG8tZL+K12/4WlJ/dO+IyS9mBbD\ne1jSXmn7JyU9lRYVfEXSwenue6RF896QdH8rVGY22yEnDrMWIOlw4FLg+LQIXh3wRWBPkk/vDgP+\nQvJpdoD7gBvSYniv5rTfD/w8LSr4aaC+7MhI4LvAUJL1WY7P/KDMdsAlR8xaxqkkBe2mpycD3UhK\nb28BxqV9fgf8QVJvoE9E/CVtvxcYL6knMDAiHgaIiI0A6f5ejojq9PksYBDwXPaHZbY9Jw6zliHg\n3oi4aZtG6V8b9NvVGj8f5Tyuwz+7VkC+VGXWMp4GLpK0D4CkvSUdRPIzdlHa5wrguYhYBayQdELa\nfhXwl3QVvmpJ56X76CKpe6sehVke/FeLWQuIiNck/QCYLGkPYDPwLWAdyYI/PyC5dHVpOuRLwN1p\nYniXj6vhXgX8Kq1suhm4uBUPwywvro5rliFJayOiR6HjMGtJvlRlZmbN4jMOMzNrFp9xmJlZszhx\nmJlZszhxmJlZszhxmJlZszhxmJlZs/w/IRkfQDAr6yIAAAAASUVORK5CYII=\n", 611 | "text/plain": [ 612 | "
" 613 | ] 614 | }, 615 | "metadata": { 616 | "tags": [] 617 | } 618 | } 619 | ] 620 | }, 621 | { 622 | "cell_type": "markdown", 623 | "metadata": { 624 | "id": "F4fHvLVITI89", 625 | "colab_type": "text" 626 | }, 627 | "source": [ 628 | "## 3.3.5 사과, 바나나, 그런데 레몬?" 629 | ] 630 | }, 631 | { 632 | "cell_type": "code", 633 | "metadata": { 634 | "id": "fg1vucC-Z3aW", 635 | "colab_type": "code", 636 | "outputId": "47100d37-8b37-435f-ab48-16ad2053476e", 637 | "colab": { 638 | "base_uri": "https://localhost:8080/", 639 | "height": 35 640 | } 641 | }, 642 | "source": [ 643 | "# 코드 3-70 시험을 위한 생성기\n", 644 | "\n", 645 | "test_generator = train_datagen.flow_from_directory( \n", 646 | " \"/content/drive/My Drive/dataset/fruit_2/\",\n", 647 | " target_size=(64, 64),\n", 648 | " batch_size=32,\n", 649 | " class_mode=None,\n", 650 | " shuffle=False\n", 651 | ")" 652 | ], 653 | "execution_count": 15, 654 | "outputs": [ 655 | { 656 | "output_type": "stream", 657 | "text": [ 658 | "Found 492 images belonging to 1 classes.\n" 659 | ], 660 | "name": "stdout" 661 | } 662 | ] 663 | }, 664 | { 665 | "cell_type": "code", 666 | "metadata": { 667 | "id": "KRZCKhsRLTi1", 668 | "colab_type": "code", 669 | "colab": { 670 | "base_uri": "https://localhost:8080/", 671 | "height": 53 672 | }, 673 | "outputId": "7e6e62bc-8d70-438f-f0c8-238feccc0e20" 674 | }, 675 | "source": [ 676 | "# 코드 3-71 학습한 모델과 생성기를 이용한 예측\n", 677 | "\n", 678 | "probs = _model_1.predict_generator(test_generator)" 679 | ], 680 | "execution_count": 16, 681 | "outputs": [ 682 | { 683 | "output_type": "stream", 684 | "text": [ 685 | "/usr/local/lib/python3.6/dist-packages/keras/utils/data_utils.py:610: UserWarning: The input 1 could not be retrieved. It could be because a worker has died.\n", 686 | " UserWarning)\n" 687 | ], 688 | "name": "stderr" 689 | } 690 | ] 691 | }, 692 | { 693 | "cell_type": "code", 694 | "metadata": { 695 | "id": "x8TqAnUAXgKl", 696 | "colab_type": "code", 697 | "colab": { 698 | "base_uri": "https://localhost:8080/", 699 | "height": 269 700 | }, 701 | "outputId": "1e5e0287-8a99-428b-c5b4-5f3e743d3b9c" 702 | }, 703 | "source": [ 704 | "# 코드 3-72 최대 예측값의 위치값을 활용하여 예측 결과 획득\n", 705 | "\n", 706 | "pred = np.argmax(probs, axis=1)\n", 707 | "print(pred)" 708 | ], 709 | "execution_count": 17, 710 | "outputs": [ 711 | { 712 | "output_type": "stream", 713 | "text": [ 714 | "[1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n", 715 | " 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n", 716 | " 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n", 717 | " 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n", 718 | " 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n", 719 | " 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n", 720 | " 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0\n", 721 | " 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n", 722 | " 0 0 0 0 0 0 0 1 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n", 723 | " 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n", 724 | " 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n", 725 | " 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0\n", 726 | " 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n", 727 | " 0 0 0 0 0 0 0 0 0 0 0]\n" 728 | ], 729 | "name": "stdout" 730 | } 731 | ] 732 | }, 733 | { 734 | "cell_type": "markdown", 735 | "metadata": { 736 | "id": "xzJgwiWITO07", 737 | "colab_type": "text" 738 | }, 739 | "source": [ 740 | "## 3.3.6 학습, 검증, 그리고 시험" 741 | ] 742 | }, 743 | { 744 | "cell_type": "code", 745 | "metadata": { 746 | "id": "TthBM5WeLkMF", 747 | "colab_type": "code", 748 | "colab": {} 749 | }, 750 | "source": [ 751 | "# 코드 7-73 이른 종결\n", 752 | "\n", 753 | "from keras.callbacks import EarlyStopping\n", 754 | "es = EarlyStopping(monitor='val_loss', mode='min', patience=3)" 755 | ], 756 | "execution_count": 0, 757 | "outputs": [] 758 | }, 759 | { 760 | "cell_type": "markdown", 761 | "metadata": { 762 | "id": "FyHWc8rZTR1Q", 763 | "colab_type": "text" 764 | }, 765 | "source": [ 766 | "## 3.3.7 모델 저장하고 불러오기" 767 | ] 768 | }, 769 | { 770 | "cell_type": "code", 771 | "metadata": { 772 | "id": "qqFZvelKjfhL", 773 | "colab_type": "code", 774 | "colab": {} 775 | }, 776 | "source": [ 777 | "# 코드 7-74 모델 저장하기\n", 778 | "\n", 779 | "_model_1.save(\"model.h5\")" 780 | ], 781 | "execution_count": 0, 782 | "outputs": [] 783 | }, 784 | { 785 | "cell_type": "code", 786 | "metadata": { 787 | "id": "GoYpLJdz4Fwl", 788 | "colab_type": "code", 789 | "colab": { 790 | "base_uri": "https://localhost:8080/", 791 | "height": 485 792 | }, 793 | "outputId": "71b037fb-244b-4f33-a946-29c95163a4e0" 794 | }, 795 | "source": [ 796 | "# 코드 7-75 모델 불러오기\n", 797 | "\n", 798 | "from keras.models import load_model\n", 799 | "\n", 800 | "_model_2 = load_model('model.h5')\n", 801 | "_model_2.summary()" 802 | ], 803 | "execution_count": 20, 804 | "outputs": [ 805 | { 806 | "output_type": "stream", 807 | "text": [ 808 | "Model: \"model_1\"\n", 809 | "_________________________________________________________________\n", 810 | "Layer (type) Output Shape Param # \n", 811 | "=================================================================\n", 812 | "input_1 (InputLayer) (None, 64, 64, 3) 0 \n", 813 | "_________________________________________________________________\n", 814 | "conv2d_2 (Conv2D) (None, 64, 64, 64) 1792 \n", 815 | "_________________________________________________________________\n", 816 | "max_pooling2d_2 (MaxPooling2 (None, 32, 32, 64) 0 \n", 817 | "_________________________________________________________________\n", 818 | "conv2d_3 (Conv2D) (None, 32, 32, 32) 18464 \n", 819 | "_________________________________________________________________\n", 820 | "max_pooling2d_3 (MaxPooling2 (None, 16, 16, 32) 0 \n", 821 | "_________________________________________________________________\n", 822 | "dropout_2 (Dropout) (None, 16, 16, 32) 0 \n", 823 | "_________________________________________________________________\n", 824 | "flatten_2 (Flatten) (None, 8192) 0 \n", 825 | "_________________________________________________________________\n", 826 | "dense_1 (Dense) (None, 16) 131088 \n", 827 | "_________________________________________________________________\n", 828 | "dense_2 (Dense) (None, 2) 34 \n", 829 | "=================================================================\n", 830 | "Total params: 151,378\n", 831 | "Trainable params: 151,378\n", 832 | "Non-trainable params: 0\n", 833 | "_________________________________________________________________\n" 834 | ], 835 | "name": "stdout" 836 | } 837 | ] 838 | } 839 | ] 840 | } -------------------------------------------------------------------------------- /Chapter3/README.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /Chapter4/4_1_1_트랜스퍼러닝_ResNet.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "colab": { 6 | "name": "4_1_트랜스퍼러닝_ResNet.ipynb", 7 | "provenance": [], 8 | "collapsed_sections": [], 9 | "toc_visible": true, 10 | "machine_shape": "hm" 11 | }, 12 | "kernelspec": { 13 | "name": "python3", 14 | "display_name": "Python 3" 15 | }, 16 | "accelerator": "GPU" 17 | }, 18 | "cells": [ 19 | { 20 | "cell_type": "markdown", 21 | "metadata": { 22 | "id": "OpBd_VwTg-bu", 23 | "colab_type": "text" 24 | }, 25 | "source": [ 26 | "# 4.1 트랜스퍼 러닝 (ResNet)\n", 27 | "\n", 28 | "- 작성자: [김찬란](https://github.com/seriousran)" 29 | ] 30 | }, 31 | { 32 | "cell_type": "markdown", 33 | "metadata": { 34 | "id": "ZSQP24KGZPnQ", 35 | "colab_type": "text" 36 | }, 37 | "source": [ 38 | "## 4.1.2 ResNet 살펴보기" 39 | ] 40 | }, 41 | { 42 | "cell_type": "code", 43 | "metadata": { 44 | "id": "PsVsqTymZEsH", 45 | "colab_type": "code", 46 | "colab": {} 47 | }, 48 | "source": [ 49 | "# 코드 4-1 케라스에 구현되어 있는 ResNet\n", 50 | "\n", 51 | "keras.applications.resnet.ResNet50(include_top=True, weights='imagenet', input_tensor=None, input_shape=None, pooling=None, classes=1000)\n", 52 | "keras.applications.resnet.ResNet101(include_top=True, weights='imagenet', input_tensor=None, input_shape=None, pooling=None, classes=1000)\n", 53 | "keras.applications.resnet.ResNet152(include_top=True, weights='imagenet', input_tensor=None, input_shape=None, pooling=None, classes=1000)\n", 54 | "keras.applications.resnet_v2.ResNet50V2(include_top=True, weights='imagenet', input_tensor=None, input_shape=None, pooling=None, classes=1000)\n", 55 | "keras.applications.resnet_v2.ResNet101V2(include_top=True, weights='imagenet', input_tensor=None, input_shape=None, pooling=None, classes=1000)\n", 56 | "keras.applications.resnet_v2.ResNet152V2(include_top=True, weights='imagenet', input_tensor=None, input_shape=None, pooling=None, classes=1000)" 57 | ], 58 | "execution_count": 0, 59 | "outputs": [] 60 | }, 61 | { 62 | "cell_type": "markdown", 63 | "metadata": { 64 | "id": "HiXgZCueZSBC", 65 | "colab_type": "text" 66 | }, 67 | "source": [ 68 | "## 4.1.3 인공신경망 내부 들여다보기" 69 | ] 70 | }, 71 | { 72 | "cell_type": "code", 73 | "metadata": { 74 | "id": "mL_dSqUWgUIE", 75 | "colab_type": "code", 76 | "colab": {} 77 | }, 78 | "source": [ 79 | "# 코드 4-2 가중치 시각화를 위한 함수들\n", 80 | "\n", 81 | "import keras\n", 82 | "import keras.backend as K\n", 83 | "from keras.applications.resnet_v2 import ResNet50V2\n", 84 | "\n", 85 | "def normalize(x):\n", 86 | " return x / (K.sqrt(K.mean(K.square(x))) + K.epsilon())\n", 87 | "\n", 88 | "\n", 89 | "def deprocess_image(x):\n", 90 | " x -= x.mean()\n", 91 | " x /= (x.std() + K.epsilon())\n", 92 | " x *= 0.25\n", 93 | "\n", 94 | " x += 0.5\n", 95 | " x = np.clip(x, 0, 1)\n", 96 | "\n", 97 | " x *= 255\n", 98 | " x = np.clip(x, 0, 255).astype('uint8')\n", 99 | " return x\n", 100 | "\n", 101 | "\n", 102 | "def process_image(x, former):\n", 103 | " return (x / 255 - 0.5) * 4 * former.std() + former.mean()\n", 104 | "\n", 105 | "\n", 106 | "def visualize_layer(model,\n", 107 | " layer_name,\n", 108 | " step=1.,\n", 109 | " epochs=15,\n", 110 | " upscaling_steps=9,\n", 111 | " upscaling_factor=1.2,\n", 112 | " output_dim=(412, 412),\n", 113 | " filter_range=(0, None)):\n", 114 | "\n", 115 | " def _generate_filter_image(input_img,\n", 116 | " layer_output,\n", 117 | " filter_index):\n", 118 | " s_time = time.time()\n", 119 | "\n", 120 | " loss = K.mean(layer_output[:, :, :, filter_index])\n", 121 | "\n", 122 | " grads = K.gradients(loss, input_img)[0]\n", 123 | " grads = normalize(grads)\n", 124 | " iterate = K.function([input_img], [loss, grads])\n", 125 | "\n", 126 | " intermediate_dim = tuple(\n", 127 | " int(x / (upscaling_factor ** upscaling_steps)) for x in output_dim)\n", 128 | " input_img_data = np.random.random(\n", 129 | " (1, intermediate_dim[0], intermediate_dim[1], 3))\n", 130 | " input_img_data = (input_img_data - 0.5) * 20 + 128\n", 131 | "\n", 132 | " for up in reversed(range(upscaling_steps)):\n", 133 | " for _ in range(epochs):\n", 134 | " loss_value, grads_value = iterate([input_img_data])\n", 135 | " input_img_data += grads_value * step\n", 136 | "\n", 137 | " if loss_value <= K.epsilon():\n", 138 | " return None\n", 139 | "\n", 140 | " intermediate_dim = tuple(\n", 141 | " int(x / (upscaling_factor ** up)) for x in output_dim)\n", 142 | " img = deprocess_image(input_img_data[0])\n", 143 | " img = np.array(Image.fromarray(img).resize(intermediate_dim,\n", 144 | " Image.BICUBIC))\n", 145 | " input_img_data = [process_image(img, input_img_data[0])]\n", 146 | "\n", 147 | " img = deprocess_image(input_img_data[0])\n", 148 | " e_time = time.time()\n", 149 | " print('Costs of filter {:3}: {:5.0f} ( {:4.2f}s )'.format(filter_index,\n", 150 | " loss_value,\n", 151 | " e_time - s_time))\n", 152 | " return img, loss_value\n", 153 | "\n", 154 | " def _draw_filters(filters, n=None):\n", 155 | " if n is None:\n", 156 | " n = int(np.floor(np.sqrt(len(filters))))\n", 157 | "\n", 158 | " filters.sort(key=lambda x: x[1], reverse=True)\n", 159 | " filters = filters[:n * n]\n", 160 | "\n", 161 | " MARGIN = 0\n", 162 | " width = n * output_dim[0] + (n - 1) * MARGIN\n", 163 | " height = n * output_dim[1] + (n - 1) * MARGIN\n", 164 | " stitched_filters = np.zeros((width, height, 3), dtype='uint8')\n", 165 | "\n", 166 | " for i in range(n):\n", 167 | " for j in range(n):\n", 168 | " img, _ = filters[i * n + j]\n", 169 | " width_margin = (output_dim[0] + MARGIN) * i\n", 170 | " height_margin = (output_dim[1] + MARGIN) * j\n", 171 | " stitched_filters[\n", 172 | " width_margin: width_margin + output_dim[0],\n", 173 | " height_margin: height_margin + output_dim[1], :] = img\n", 174 | "\n", 175 | " save_img('/content/drive/My Drive/0_colab/conv/resnet_{0:}_{1:}x{1:}.png'.format(layer_name, n), stitched_filters)\n", 176 | " assert len(model.inputs) == 1\n", 177 | " input_img = model.inputs[0]\n", 178 | "\n", 179 | " layer_dict = dict([(layer.name, layer) for layer in model.layers[1:]])\n", 180 | "\n", 181 | " output_layer = layer_dict[layer_name]\n", 182 | " assert isinstance(output_layer, Conv2D)\n", 183 | "\n", 184 | " filter_lower = filter_range[0]\n", 185 | " filter_upper = (filter_range[1]\n", 186 | " if filter_range[1] is not None\n", 187 | " else len(output_layer.get_weights()[1]))\n", 188 | "\n", 189 | " assert(filter_lower >= 0\n", 190 | " and filter_upper <= len(output_layer.get_weights()[1])\n", 191 | " and filter_upper > filter_lower)\n", 192 | " print('Compute filters {:} to {:}'.format(filter_lower, filter_upper))\n", 193 | "\n", 194 | " processed_filters = []\n", 195 | " for f in range(filter_lower, filter_upper):\n", 196 | " img_loss = _generate_filter_image(input_img, output_layer.output, f)\n", 197 | "\n", 198 | " if img_loss is not None:\n", 199 | " processed_filters.append(img_loss)\n", 200 | "\n", 201 | " if len(processed_filters) == 9:\n", 202 | " break\n", 203 | "\n", 204 | " print('{} filter processed.'.format(len(processed_filters)))\n", 205 | " _draw_filters(processed_filters)" 206 | ], 207 | "execution_count": 0, 208 | "outputs": [] 209 | }, 210 | { 211 | "cell_type": "code", 212 | "metadata": { 213 | "id": "Ep6im76PyRGA", 214 | "colab_type": "code", 215 | "outputId": "704f6c7d-2ac9-4a70-aeff-882b8160a96e", 216 | "colab": { 217 | "base_uri": "https://localhost:8080/", 218 | "height": 503 219 | } 220 | }, 221 | "source": [ 222 | "resnet_v2 = ResNet50V2(include_top=False, weights='imagenet',\n", 223 | " input_shape = (32,32,3))\n", 224 | "visualize_layer(resnet_v2, 'conv1_conv')" 225 | ], 226 | "execution_count": 0, 227 | "outputs": [ 228 | { 229 | "output_type": "stream", 230 | "text": [ 231 | "Compute filters 0 to 64\n", 232 | "Costs of filter 0: 38 ( 6.71s )\n", 233 | "Costs of filter 1: 168 ( 6.59s )\n", 234 | "Costs of filter 5: 25 ( 6.57s )\n", 235 | "Costs of filter 7: 18 ( 6.80s )\n", 236 | "Costs of filter 8: 36 ( 6.78s )\n", 237 | "Costs of filter 10: 160 ( 6.72s )\n", 238 | "Costs of filter 13: 8 ( 6.70s )\n", 239 | "Costs of filter 14: 7 ( 6.69s )\n", 240 | "Costs of filter 15: 24 ( 6.69s )\n", 241 | "Costs of filter 16: 45 ( 6.79s )\n", 242 | "Costs of filter 17: 28 ( 6.82s )\n", 243 | "Costs of filter 20: 29 ( 6.74s )\n", 244 | "Costs of filter 24: 12 ( 6.82s )\n", 245 | "Costs of filter 27: 15 ( 6.68s )\n", 246 | "Costs of filter 30: 5 ( 6.90s )\n", 247 | "Costs of filter 31: 110 ( 6.75s )\n", 248 | "Costs of filter 32: 16 ( 6.67s )\n", 249 | "Costs of filter 37: 51 ( 6.72s )\n", 250 | "Costs of filter 40: 28 ( 6.90s )\n", 251 | "Costs of filter 41: 7 ( 6.77s )\n", 252 | "Costs of filter 44: 155 ( 7.01s )\n", 253 | "Costs of filter 45: 16 ( 6.75s )\n", 254 | "Costs of filter 46: 16 ( 6.85s )\n", 255 | "Costs of filter 49: 153 ( 6.90s )\n", 256 | "Costs of filter 58: 59 ( 6.90s )\n", 257 | "25 filter processed.\n" 258 | ], 259 | "name": "stdout" 260 | } 261 | ] 262 | }, 263 | { 264 | "cell_type": "code", 265 | "metadata": { 266 | "id": "1vbZwqWPyQ-q", 267 | "colab_type": "code", 268 | "outputId": "31de1bea-1fa3-4a94-a6fd-bab75684b9f6", 269 | "colab": { 270 | "base_uri": "https://localhost:8080/", 271 | "height": 503 272 | } 273 | }, 274 | "source": [ 275 | "visualize_layer(resnet_v2, 'conv2_block2_3_conv')" 276 | ], 277 | "execution_count": 0, 278 | "outputs": [ 279 | { 280 | "output_type": "stream", 281 | "text": [ 282 | "Compute filters 0 to 256\n", 283 | "Costs of filter 3: 133 ( 7.88s )\n", 284 | "Costs of filter 7: 28 ( 8.02s )\n", 285 | "Costs of filter 8: 26 ( 7.93s )\n", 286 | "Costs of filter 14: 65 ( 8.04s )\n", 287 | "Costs of filter 22: 102 ( 8.16s )\n", 288 | "Costs of filter 25: 251 ( 8.10s )\n", 289 | "Costs of filter 31: 56 ( 8.45s )\n", 290 | "Costs of filter 34: 103 ( 8.50s )\n", 291 | "Costs of filter 35: 95 ( 8.58s )\n", 292 | "Costs of filter 36: 50 ( 8.43s )\n", 293 | "Costs of filter 40: 41 ( 8.48s )\n", 294 | "Costs of filter 43: 15 ( 8.71s )\n", 295 | "Costs of filter 46: 113 ( 8.86s )\n", 296 | "Costs of filter 47: 176 ( 8.66s )\n", 297 | "Costs of filter 51: 91 ( 8.71s )\n", 298 | "Costs of filter 53: 87 ( 9.03s )\n", 299 | "Costs of filter 63: 54 ( 9.20s )\n", 300 | "Costs of filter 66: 50 ( 9.20s )\n", 301 | "Costs of filter 74: 44 ( 9.45s )\n", 302 | "Costs of filter 77: 94 ( 9.54s )\n", 303 | "Costs of filter 79: 122 ( 9.31s )\n", 304 | "Costs of filter 82: 91 ( 9.27s )\n", 305 | "Costs of filter 84: 43 ( 9.60s )\n", 306 | "Costs of filter 85: 32 ( 9.46s )\n", 307 | "Costs of filter 105: 132 ( 9.77s )\n", 308 | "25 filter processed.\n" 309 | ], 310 | "name": "stdout" 311 | } 312 | ] 313 | }, 314 | { 315 | "cell_type": "code", 316 | "metadata": { 317 | "id": "tZ5BhZrCyQ8W", 318 | "colab_type": "code", 319 | "outputId": "7397f5d6-ae11-46fd-cd1f-4459a147fdc9", 320 | "colab": { 321 | "base_uri": "https://localhost:8080/", 322 | "height": 503 323 | } 324 | }, 325 | "source": [ 326 | "visualize_layer(resnet_v2, 'conv2_block3_3_conv')" 327 | ], 328 | "execution_count": 0, 329 | "outputs": [ 330 | { 331 | "output_type": "stream", 332 | "text": [ 333 | "Compute filters 0 to 256\n", 334 | "Costs of filter 7: 47 ( 10.54s )\n", 335 | "Costs of filter 9: 38 ( 10.63s )\n", 336 | "Costs of filter 10: 47 ( 10.71s )\n", 337 | "Costs of filter 12: 132 ( 10.72s )\n", 338 | "Costs of filter 14: 81 ( 10.61s )\n", 339 | "Costs of filter 16: 50 ( 10.90s )\n", 340 | "Costs of filter 17: 151 ( 11.24s )\n", 341 | "Costs of filter 24: 55 ( 11.15s )\n", 342 | "Costs of filter 31: 112 ( 11.17s )\n", 343 | "Costs of filter 35: 73 ( 11.38s )\n", 344 | "Costs of filter 40: 50 ( 11.42s )\n", 345 | "Costs of filter 41: 34 ( 11.65s )\n", 346 | "Costs of filter 45: 150 ( 11.50s )\n", 347 | "Costs of filter 47: 181 ( 11.73s )\n", 348 | "Costs of filter 48: 58 ( 11.69s )\n", 349 | "Costs of filter 49: 55 ( 11.94s )\n", 350 | "Costs of filter 51: 162 ( 12.02s )\n", 351 | "Costs of filter 68: 82 ( 12.33s )\n", 352 | "Costs of filter 72: 83 ( 12.44s )\n", 353 | "Costs of filter 73: 129 ( 12.58s )\n", 354 | "Costs of filter 74: 68 ( 12.39s )\n", 355 | "Costs of filter 79: 20 ( 12.51s )\n", 356 | "Costs of filter 81: 106 ( 12.53s )\n", 357 | "Costs of filter 84: 12 ( 12.73s )\n", 358 | "Costs of filter 85: 52 ( 12.95s )\n", 359 | "25 filter processed.\n" 360 | ], 361 | "name": "stdout" 362 | } 363 | ] 364 | }, 365 | { 366 | "cell_type": "code", 367 | "metadata": { 368 | "id": "YHiQgGhTyQ5f", 369 | "colab_type": "code", 370 | "outputId": "32f49a6a-282d-408e-e7b5-469d3d60649c", 371 | "colab": { 372 | "base_uri": "https://localhost:8080/", 373 | "height": 503 374 | } 375 | }, 376 | "source": [ 377 | "visualize_layer(resnet_v2, 'conv3_block1_3_conv')" 378 | ], 379 | "execution_count": 0, 380 | "outputs": [ 381 | { 382 | "output_type": "stream", 383 | "text": [ 384 | "Compute filters 0 to 512\n", 385 | "Costs of filter 4: 55 ( 13.57s )\n", 386 | "Costs of filter 7: 26 ( 13.35s )\n", 387 | "Costs of filter 14: 15 ( 13.62s )\n", 388 | "Costs of filter 16: 66 ( 13.75s )\n", 389 | "Costs of filter 18: 49 ( 13.95s )\n", 390 | "Costs of filter 19: 31 ( 13.83s )\n", 391 | "Costs of filter 31: 25 ( 14.64s )\n", 392 | "Costs of filter 32: 48 ( 14.38s )\n", 393 | "Costs of filter 39: 83 ( 14.56s )\n", 394 | "Costs of filter 44: 40 ( 15.07s )\n", 395 | "Costs of filter 47: 32 ( 14.98s )\n", 396 | "Costs of filter 50: 62 ( 15.16s )\n", 397 | "Costs of filter 55: 20 ( 15.31s )\n", 398 | "Costs of filter 63: 21 ( 15.77s )\n", 399 | "Costs of filter 66: 31 ( 15.43s )\n", 400 | "Costs of filter 70: 81 ( 15.64s )\n", 401 | "Costs of filter 83: 52 ( 16.47s )\n", 402 | "Costs of filter 84: 107 ( 16.30s )\n", 403 | "Costs of filter 85: 80 ( 16.56s )\n", 404 | "Costs of filter 87: 16 ( 16.51s )\n", 405 | "Costs of filter 89: 45 ( 16.65s )\n", 406 | "Costs of filter 91: 55 ( 16.72s )\n", 407 | "Costs of filter 92: 107 ( 16.49s )\n", 408 | "Costs of filter 95: 58 ( 16.75s )\n", 409 | "Costs of filter 98: 39 ( 16.97s )\n", 410 | "25 filter processed.\n" 411 | ], 412 | "name": "stdout" 413 | } 414 | ] 415 | }, 416 | { 417 | "cell_type": "code", 418 | "metadata": { 419 | "id": "5TdyKjjMo79b", 420 | "colab_type": "code", 421 | "outputId": "bc7ed142-95f4-4a06-aaaa-d3de83d62861", 422 | "colab": { 423 | "base_uri": "https://localhost:8080/", 424 | "height": 377 425 | } 426 | }, 427 | "source": [ 428 | "visualize_layer(resnet_v2, 'conv3_block4_3_conv')" 429 | ], 430 | "execution_count": 0, 431 | "outputs": [ 432 | { 433 | "output_type": "stream", 434 | "text": [ 435 | "Compute filters 0 to 512\n", 436 | "Costs of filter 1: 169 ( 18.37s )\n", 437 | "Costs of filter 3: 43 ( 18.60s )\n", 438 | "Costs of filter 7: 35 ( 19.17s )\n", 439 | "Costs of filter 12: 45 ( 18.92s )\n", 440 | "Costs of filter 29: 45 ( 19.96s )\n", 441 | "Costs of filter 32: 24 ( 20.40s )\n", 442 | "Costs of filter 34: 46 ( 20.53s )\n", 443 | "Costs of filter 38: 77 ( 20.58s )\n", 444 | "Costs of filter 39: 64 ( 20.96s )\n", 445 | "Costs of filter 41: 150 ( 20.97s )\n", 446 | "Costs of filter 47: 187 ( 21.23s )\n", 447 | "Costs of filter 50: 16 ( 21.48s )\n", 448 | "Costs of filter 51: 45 ( 21.68s )\n", 449 | "Costs of filter 58: 29 ( 22.04s )\n", 450 | "Costs of filter 66: 76 ( 22.46s )\n", 451 | "Costs of filter 68: 57 ( 22.61s )\n", 452 | "Costs of filter 73: 69 ( 22.79s )\n", 453 | "Costs of filter 75: 41 ( 23.23s )\n", 454 | "Costs of filter 76: 34 ( 23.09s )\n" 455 | ], 456 | "name": "stdout" 457 | } 458 | ] 459 | }, 460 | { 461 | "cell_type": "code", 462 | "metadata": { 463 | "id": "arqss_p3o76s", 464 | "colab_type": "code", 465 | "outputId": "7358a94f-32f2-4890-b23e-af960e01af03", 466 | "colab": { 467 | "base_uri": "https://localhost:8080/", 468 | "height": 503 469 | } 470 | }, 471 | "source": [ 472 | "visualize_layer(resnet_v2, 'conv4_block1_3_conv')" 473 | ], 474 | "execution_count": 0, 475 | "outputs": [ 476 | { 477 | "output_type": "stream", 478 | "text": [ 479 | "Compute filters 0 to 1024\n", 480 | "Costs of filter 1: 47 ( 5.51s )\n", 481 | "Costs of filter 7: 13 ( 4.56s )\n", 482 | "Costs of filter 10: 47 ( 4.75s )\n", 483 | "Costs of filter 12: 17 ( 4.90s )\n", 484 | "Costs of filter 16: 39 ( 5.18s )\n", 485 | "Costs of filter 18: 39 ( 5.35s )\n", 486 | "Costs of filter 19: 14 ( 5.43s )\n", 487 | "Costs of filter 25: 43 ( 5.76s )\n", 488 | "Costs of filter 26: 70 ( 5.81s )\n", 489 | "Costs of filter 27: 21 ( 6.17s )\n", 490 | "Costs of filter 29: 88 ( 6.01s )\n", 491 | "Costs of filter 33: 19 ( 6.48s )\n", 492 | "Costs of filter 36: 51 ( 6.68s )\n", 493 | "Costs of filter 40: 15 ( 6.93s )\n", 494 | "Costs of filter 41: 10 ( 6.96s )\n", 495 | "Costs of filter 43: 35 ( 7.17s )\n", 496 | "Costs of filter 57: 25 ( 8.09s )\n", 497 | "Costs of filter 64: 23 ( 8.92s )\n", 498 | "Costs of filter 65: 30 ( 8.60s )\n", 499 | "Costs of filter 71: 11 ( 9.04s )\n", 500 | "Costs of filter 76: 30 ( 9.44s )\n", 501 | "Costs of filter 83: 7 ( 9.88s )\n", 502 | "Costs of filter 89: 43 ( 10.29s )\n", 503 | "Costs of filter 93: 78 ( 10.55s )\n", 504 | "Costs of filter 104: 56 ( 11.38s )\n", 505 | "25 filter processed.\n" 506 | ], 507 | "name": "stdout" 508 | } 509 | ] 510 | }, 511 | { 512 | "cell_type": "code", 513 | "metadata": { 514 | "id": "SyyGR_2go748", 515 | "colab_type": "code", 516 | "outputId": "d7978454-f3a9-4467-8742-3e08d8c2e174", 517 | "colab": { 518 | "base_uri": "https://localhost:8080/", 519 | "height": 125 520 | } 521 | }, 522 | "source": [ 523 | "visualize_layer(resnet_v2, 'conv4_block6_3_conv')" 524 | ], 525 | "execution_count": 0, 526 | "outputs": [ 527 | { 528 | "output_type": "stream", 529 | "text": [ 530 | "Compute filters 0 to 1024\n", 531 | "Costs of filter 3: 35 ( 58.30s )\n", 532 | "Costs of filter 47: 61 ( 61.91s )\n", 533 | "Costs of filter 56: 378 ( 64.66s )\n", 534 | "Costs of filter 274: 103 ( 90.64s )\n", 535 | "Costs of filter 319: 21 ( 95.52s )\n" 536 | ], 537 | "name": "stdout" 538 | } 539 | ] 540 | }, 541 | { 542 | "cell_type": "code", 543 | "metadata": { 544 | "id": "W_hKx_MSo71N", 545 | "colab_type": "code", 546 | "colab": {} 547 | }, 548 | "source": [ 549 | "visualize_layer(resnet_v2, 'conv5_block1_3_conv')" 550 | ], 551 | "execution_count": 0, 552 | "outputs": [] 553 | }, 554 | { 555 | "cell_type": "code", 556 | "metadata": { 557 | "id": "Rzps6o0Ao7xE", 558 | "colab_type": "code", 559 | "colab": {} 560 | }, 561 | "source": [ 562 | "visualize_layer(resnet_v2, 'conv5_block3_3_conv')" 563 | ], 564 | "execution_count": 0, 565 | "outputs": [] 566 | } 567 | ] 568 | } -------------------------------------------------------------------------------- /Chapter4/4_1_2_트랜스퍼러닝_메트릭러닝.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "colab": { 6 | "name": "4_1_2_트랜스퍼러닝_메트릭러닝.ipynb", 7 | "provenance": [], 8 | "collapsed_sections": [], 9 | "toc_visible": true, 10 | "machine_shape": "hm" 11 | }, 12 | "kernelspec": { 13 | "name": "python3", 14 | "display_name": "Python 3" 15 | }, 16 | "accelerator": "GPU" 17 | }, 18 | "cells": [ 19 | { 20 | "cell_type": "markdown", 21 | "metadata": { 22 | "id": "OpBd_VwTg-bu", 23 | "colab_type": "text" 24 | }, 25 | "source": [ 26 | "# 4.1 트랜스퍼러닝(이어서)\n", 27 | "- 작성자: [김찬란](https://github.com/seriousran)" 28 | ] 29 | }, 30 | { 31 | "cell_type": "markdown", 32 | "metadata": { 33 | "id": "gKr1x8Sp9So1", 34 | "colab_type": "text" 35 | }, 36 | "source": [ 37 | "## 4.1.4 구분하는 능력을 더 강력하게 학습하기 - 메트릭 러닝" 38 | ] 39 | }, 40 | { 41 | "cell_type": "code", 42 | "metadata": { 43 | "id": "iR8MGU_x8tIQ", 44 | "colab_type": "code", 45 | "colab": {} 46 | }, 47 | "source": [ 48 | "# 코드 4-4 트리플렛 손실 함수\n", 49 | "\n", 50 | "def triplet_loss(y_true, y_pred):\n", 51 | " margin = K.constant(1)\n", 52 | " return K.mean(K.maximum(K.constant(0), K.square(y_pred[:,0,0]) - 0.5 * (K.square(y_pred[:,1,0]) + K.square(y_pred[:,2,0])) + margin))" 53 | ], 54 | "execution_count": 0, 55 | "outputs": [] 56 | }, 57 | { 58 | "cell_type": "markdown", 59 | "metadata": { 60 | "id": "ixLD207U8s4p", 61 | "colab_type": "text" 62 | }, 63 | "source": [ 64 | "## 4.1.5 트리플렛 데이터셋분할하기" 65 | ] 66 | }, 67 | { 68 | "cell_type": "code", 69 | "metadata": { 70 | "id": "55OBoLXS-2zT", 71 | "colab_type": "code", 72 | "colab": { 73 | "base_uri": "https://localhost:8080/", 74 | "height": 127 75 | }, 76 | "outputId": "e3e1b67b-9ed8-41ed-96e9-b0cf15747a10" 77 | }, 78 | "source": [ 79 | "from google.colab import drive\n", 80 | "drive.mount('/content/drive')" 81 | ], 82 | "execution_count": 5, 83 | "outputs": [ 84 | { 85 | "output_type": "stream", 86 | "text": [ 87 | "Go to this URL in a browser: https://accounts.google.com/o/oauth2/auth?client_id=947318989803-6bn6qk8qdgf4n4g3pfee6491hc0brc4i.apps.googleusercontent.com&redirect_uri=urn%3aietf%3awg%3aoauth%3a2.0%3aoob&response_type=code&scope=email%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdocs.test%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive.photos.readonly%20https%3a%2f%2fwww.googleapis.com%2fauth%2fpeopleapi.readonly\n", 88 | "\n", 89 | "Enter your authorization code:\n", 90 | "··········\n", 91 | "Mounted at /content/drive\n" 92 | ], 93 | "name": "stdout" 94 | } 95 | ] 96 | }, 97 | { 98 | "cell_type": "code", 99 | "metadata": { 100 | "id": "AOI6CO7b9krb", 101 | "colab_type": "code", 102 | "colab": {} 103 | }, 104 | "source": [ 105 | "# 코드 4-5 폴더 이름 가져오기 및 정렬\n", 106 | "\n", 107 | "import os\n", 108 | "import glob\n", 109 | "from shutil import copyfile\n", 110 | "from random import shuffle\n", 111 | "\n", 112 | "folder_list = glob.glob(\"/content/drive/My Drive/colab/oxford-flowers-17/*/\")\n", 113 | "folder_list.sort()" 114 | ], 115 | "execution_count": 0, 116 | "outputs": [] 117 | }, 118 | { 119 | "cell_type": "code", 120 | "metadata": { 121 | "id": "E1x6FEifE0p9", 122 | "colab_type": "code", 123 | "colab": {} 124 | }, 125 | "source": [ 126 | "# 코드 4-6 이미지 파일 이름 가져오기\n", 127 | "\n", 128 | "whole_img_list = glob.glob(\"/content/drive/My Drive/colab/oxford-flowers-17/*/*.jpg\")" 129 | ], 130 | "execution_count": 0, 131 | "outputs": [] 132 | }, 133 | { 134 | "cell_type": "code", 135 | "metadata": { 136 | "id": "xH-GmccxGdU0", 137 | "colab_type": "code", 138 | "colab": { 139 | "base_uri": "https://localhost:8080/", 140 | "height": 35 141 | }, 142 | "outputId": "6652d6d4-3784-4324-c2b7-007254c9ac99" 143 | }, 144 | "source": [ 145 | "# 코드 4-7 앵커 데이터 생성\n", 146 | "\n", 147 | "BASE_PATH = \"/content/drive/My Drive/colab/\"\n", 148 | "\n", 149 | "if not os.path.exists(BASE_PATH + 'flowers_anchor'):\n", 150 | " os.mkdir(BASE_PATH + 'flowers_anchor')\n", 151 | "if not os.path.exists(BASE_PATH + 'flowers_anchor/images'):\n", 152 | " os.mkdir(BASE_PATH + 'flowers_anchor/images')\n", 153 | "\n", 154 | "flower_list = []\n", 155 | "for folder in folder_list:\n", 156 | " img_list = glob.glob(folder + \"*.jpg\")\n", 157 | " tmp_list = img_list.copy()\n", 158 | " flower_list.append(img_list)\n", 159 | "\n", 160 | "for i, flower in enumerate(flower_list):\n", 161 | " for j, img in enumerate(flower):\n", 162 | " copyfile(img, '/'.join(img.split('/')[:5]) + '/flowers_anchor/images/image_' + str(i*len(flower) + j).zfill(4) + '.jpg')\n", 163 | "\n", 164 | "positive_list = glob.glob(\"/content/drive/My Drive/colab/flowers_anchor/images/*.jpg\")\n", 165 | "len(positive_list)" 166 | ], 167 | "execution_count": 16, 168 | "outputs": [ 169 | { 170 | "output_type": "execute_result", 171 | "data": { 172 | "text/plain": [ 173 | "1360" 174 | ] 175 | }, 176 | "metadata": { 177 | "tags": [] 178 | }, 179 | "execution_count": 16 180 | } 181 | ] 182 | }, 183 | { 184 | "cell_type": "code", 185 | "metadata": { 186 | "id": "6q5klC8pO3f8", 187 | "colab_type": "code", 188 | "colab": { 189 | "base_uri": "https://localhost:8080/", 190 | "height": 35 191 | }, 192 | "outputId": "21b1680a-df83-4e71-ea65-69ae3b7b8f9f" 193 | }, 194 | "source": [ 195 | "# 코드 4-8 긍정 데이터 생성\n", 196 | "\n", 197 | "def shuffle_not_in_place(input_list):\n", 198 | " randomized_list = input_list[:]\n", 199 | " while True:\n", 200 | " shuffle(randomized_list)\n", 201 | " for a, b in zip(input_list, randomized_list):\n", 202 | " if a == b:\n", 203 | " break\n", 204 | " else:\n", 205 | " return randomized_list\n", 206 | "\n", 207 | "if not os.path.exists(BASE_PATH + 'flowers_positive'):\n", 208 | " os.mkdir(BASE_PATH + 'flowers_positive')\n", 209 | "if not os.path.exists(BASE_PATH + 'flowers_positive/images'):\n", 210 | " os.mkdir(BASE_PATH + 'flowers_positive/images')\n", 211 | "\n", 212 | "flower_list = []\n", 213 | "for folder in folder_list:\n", 214 | " img_list = glob.glob(folder + \"*.jpg\")\n", 215 | " img_list = shuffle_not_in_place(img_list)\n", 216 | " flower_list.append(img_list)\n", 217 | "\n", 218 | "for i, flower in enumerate(flower_list):\n", 219 | " for j, img in enumerate(flower):\n", 220 | " copyfile(img, '/'.join(img.split('/')[:5]) + '/flowers_positive/images/image_' + str(i*len(flower) + j).zfill(4) + '.jpg')\n", 221 | "\n", 222 | "positive_list = glob.glob(\"/content/drive/My Drive/colab/flowers_positive/images/*.jpg\")\n", 223 | "len(positive_list)" 224 | ], 225 | "execution_count": 17, 226 | "outputs": [ 227 | { 228 | "output_type": "execute_result", 229 | "data": { 230 | "text/plain": [ 231 | "1360" 232 | ] 233 | }, 234 | "metadata": { 235 | "tags": [] 236 | }, 237 | "execution_count": 17 238 | } 239 | ] 240 | }, 241 | { 242 | "cell_type": "code", 243 | "metadata": { 244 | "id": "wGcBr1GLVQ8O", 245 | "colab_type": "code", 246 | "colab": { 247 | "base_uri": "https://localhost:8080/", 248 | "height": 35 249 | }, 250 | "outputId": "38a1778a-47c5-4c23-b0d6-c0343a162b24" 251 | }, 252 | "source": [ 253 | "# 코드 4-9 부정 데이터 생성\n", 254 | "\n", 255 | "folder_names = [folder.split('/')[6] for folder in folder_list]\n", 256 | "\n", 257 | "if not os.path.exists(BASE_PATH + 'flowers_negative'):\n", 258 | " os.mkdir(BASE_PATH + 'flowers_negative')\n", 259 | "if not os.path.exists(BASE_PATH + 'flowers_negative/images'):\n", 260 | " os.mkdir(BASE_PATH + 'flowers_negative/images')\n", 261 | "\n", 262 | "whole_shuffled = whole_img_list.copy()\n", 263 | "shuffle(whole_shuffled)\n", 264 | "i=0\n", 265 | "j=0\n", 266 | "while j < len(folder_names):\n", 267 | " if folder_names[j] == whole_shuffled[i].split('/')[6]:\n", 268 | " tmp = whole_shuffled.pop(i)\n", 269 | " whole_shuffled.append(tmp)\n", 270 | " else:\n", 271 | " i+=1\n", 272 | " if i%80 == 0:\n", 273 | " j+=1\n", 274 | "\n", 275 | "for i, img in enumerate(whole_shuffled):\n", 276 | " copyfile(img, '/'.join(img.split('/')[:5]) + '/flowers_negative/images/image_' + str(i).zfill(4) + '.jpg')\n", 277 | "\n", 278 | "negative = glob.glob(\"/content/drive/My Drive/colab/flowers_negative/images/*.jpg\")\n", 279 | "len(negative)" 280 | ], 281 | "execution_count": 18, 282 | "outputs": [ 283 | { 284 | "output_type": "execute_result", 285 | "data": { 286 | "text/plain": [ 287 | "1360" 288 | ] 289 | }, 290 | "metadata": { 291 | "tags": [] 292 | }, 293 | "execution_count": 18 294 | } 295 | ] 296 | } 297 | ] 298 | } -------------------------------------------------------------------------------- /Chapter4/README.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /Chapter5/5_3_2_실전딥러닝_긍부정분석.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "colab": { 6 | "name": "5_3_2_실전딥러닝_긍부정분석.ipynb", 7 | "provenance": [], 8 | "collapsed_sections": [], 9 | "machine_shape": "hm" 10 | }, 11 | "kernelspec": { 12 | "name": "python3", 13 | "display_name": "Python 3" 14 | }, 15 | "accelerator": "GPU" 16 | }, 17 | "cells": [ 18 | { 19 | "cell_type": "markdown", 20 | "metadata": { 21 | "id": "GMG5r1LJEqXm", 22 | "colab_type": "text" 23 | }, 24 | "source": [ 25 | "# 5.3.4 텐서플로 2.0과 케라스를 이용한 영화 리뷰 긍부정(감정) 분석\n", 26 | "\n", 27 | "- 작성자: [김찬란](https://github.com/seriousran)" 28 | ] 29 | }, 30 | { 31 | "cell_type": "code", 32 | "metadata": { 33 | "id": "G6aw0MJp7Ojw", 34 | "colab_type": "code", 35 | "outputId": "04c0e021-73ef-4f89-c0fc-fc6f01f8c238", 36 | "colab": { 37 | "base_uri": "https://localhost:8080/", 38 | "height": 442 39 | } 40 | }, 41 | "source": [ 42 | "# 코드 5-19 패키지 설치 및 데이터셋 다운로드\n", 43 | "\n", 44 | "!git clone https://github.com/kpe/bert-for-tf2.git\n", 45 | "!pip install -r bert-for-tf2/requirements.txt\n", 46 | "!pip install sentencepiece\n", 47 | "\n", 48 | "!git clone https://github.com/e9t/nsmc\n", 49 | "!wget https://storage.googleapis.com/bert_models/2018_11_23/multi_cased_L-12_H-768_A-12.zip\n", 50 | "!unzip multi_cased_L-12_H-768_A-12.zip\n", 51 | "#!pip install ujson\n", 52 | "\n", 53 | "import sys\n", 54 | "import pandas as pd\n", 55 | "import numpy as np\n", 56 | "import random\n", 57 | "import matplotlib.pyplot as plt\n", 58 | "\n", 59 | "sys.path.append(\"bert-for-tf2/\")\n", 60 | "\n", 61 | "import tensorflow as tf\n", 62 | "import keras\n", 63 | "\n", 64 | "import bert\n", 65 | "from bert.model import BertModelLayer\n", 66 | "from bert.loader import params_from_pretrained_ckpt, load_stock_weights\n", 67 | "from bert.tokenization.bert_tokenization import FullTokenizer" 68 | ], 69 | "execution_count": 0, 70 | "outputs": [ 71 | { 72 | "output_type": "stream", 73 | "text": [ 74 | "fatal: destination path 'bert-for-tf2' already exists and is not an empty directory.\n", 75 | "Requirement already satisfied: py-params>=0.7.3 in /usr/local/lib/python3.6/dist-packages (from -r bert-for-tf2/requirements.txt (line 4)) (0.8.2)\n", 76 | "Requirement already satisfied: params-flow>=0.7.1 in /usr/local/lib/python3.6/dist-packages (from -r bert-for-tf2/requirements.txt (line 5)) (0.7.4)\n", 77 | "Requirement already satisfied: tqdm in /usr/local/lib/python3.6/dist-packages (from params-flow>=0.7.1->-r bert-for-tf2/requirements.txt (line 5)) (4.28.1)\n", 78 | "Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from params-flow>=0.7.1->-r bert-for-tf2/requirements.txt (line 5)) (1.17.5)\n", 79 | "Requirement already satisfied: sentencepiece in /usr/local/lib/python3.6/dist-packages (0.1.85)\n", 80 | "fatal: destination path 'nsmc' already exists and is not an empty directory.\n", 81 | "--2020-02-07 00:56:04-- https://storage.googleapis.com/bert_models/2018_11_23/multi_cased_L-12_H-768_A-12.zip\n", 82 | "Resolving storage.googleapis.com (storage.googleapis.com)... 74.125.195.128, 2607:f8b0:400e:c07::80\n", 83 | "Connecting to storage.googleapis.com (storage.googleapis.com)|74.125.195.128|:443... connected.\n", 84 | "HTTP request sent, awaiting response... 200 OK\n", 85 | "Length: 662903077 (632M) [application/zip]\n", 86 | "Saving to: ‘multi_cased_L-12_H-768_A-12.zip.1’\n", 87 | "\n", 88 | "multi_cased_L-12_H- 100%[===================>] 632.19M 47.6MB/s in 13s \n", 89 | "\n", 90 | "2020-02-07 00:56:17 (50.3 MB/s) - ‘multi_cased_L-12_H-768_A-12.zip.1’ saved [662903077/662903077]\n", 91 | "\n", 92 | "Archive: multi_cased_L-12_H-768_A-12.zip\n", 93 | "replace multi_cased_L-12_H-768_A-12/bert_model.ckpt.meta? [y]es, [n]o, [A]ll, [N]one, [r]ename: N\n" 94 | ], 95 | "name": "stdout" 96 | }, 97 | { 98 | "output_type": "display_data", 99 | "data": { 100 | "text/html": [ 101 | "

\n", 102 | "The default version of TensorFlow in Colab will soon switch to TensorFlow 2.x.
\n", 103 | "We recommend you upgrade now \n", 104 | "or ensure your notebook will continue to use TensorFlow 1.x via the %tensorflow_version 1.x magic:\n", 105 | "more info.

\n" 106 | ], 107 | "text/plain": [ 108 | "" 109 | ] 110 | }, 111 | "metadata": { 112 | "tags": [] 113 | } 114 | }, 115 | { 116 | "output_type": "stream", 117 | "text": [ 118 | "Using TensorFlow backend.\n" 119 | ], 120 | "name": "stderr" 121 | } 122 | ] 123 | }, 124 | { 125 | "cell_type": "code", 126 | "metadata": { 127 | "id": "GIo6OQ_B_v0A", 128 | "colab_type": "code", 129 | "outputId": "be65d7d2-f73e-4351-bdf6-60287b491b67", 130 | "colab": { 131 | "base_uri": "https://localhost:8080/", 132 | "height": 204 133 | } 134 | }, 135 | "source": [ 136 | "df_train = pd.read_csv(\"nsmc/ratings_train.txt\", sep='\\t', nrows=10000)\n", 137 | "df_test = pd.read_csv(\"nsmc/ratings_test.txt\", sep='\\t', nrows=10000)\n", 138 | "df_train.head()" 139 | ], 140 | "execution_count": 0, 141 | "outputs": [ 142 | { 143 | "output_type": "execute_result", 144 | "data": { 145 | "text/html": [ 146 | "
\n", 147 | "\n", 160 | "\n", 161 | " \n", 162 | " \n", 163 | " \n", 164 | " \n", 165 | " \n", 166 | " \n", 167 | " \n", 168 | " \n", 169 | " \n", 170 | " \n", 171 | " \n", 172 | " \n", 173 | " \n", 174 | " \n", 175 | " \n", 176 | " \n", 177 | " \n", 178 | " \n", 179 | " \n", 180 | " \n", 181 | " \n", 182 | " \n", 183 | " \n", 184 | " \n", 185 | " \n", 186 | " \n", 187 | " \n", 188 | " \n", 189 | " \n", 190 | " \n", 191 | " \n", 192 | " \n", 193 | " \n", 194 | " \n", 195 | " \n", 196 | " \n", 197 | " \n", 198 | " \n", 199 | " \n", 200 | " \n", 201 | "
iddocumentlabel
09976970아 더빙.. 진짜 짜증나네요 목소리0
13819312흠...포스터보고 초딩영화줄....오버연기조차 가볍지 않구나1
210265843너무재밓었다그래서보는것을추천한다0
39045019교도소 이야기구먼 ..솔직히 재미는 없다..평점 조정0
46483659사이몬페그의 익살스런 연기가 돋보였던 영화!스파이더맨에서 늙어보이기만 했던 커스틴 ...1
\n", 202 | "
" 203 | ], 204 | "text/plain": [ 205 | " id document label\n", 206 | "0 9976970 아 더빙.. 진짜 짜증나네요 목소리 0\n", 207 | "1 3819312 흠...포스터보고 초딩영화줄....오버연기조차 가볍지 않구나 1\n", 208 | "2 10265843 너무재밓었다그래서보는것을추천한다 0\n", 209 | "3 9045019 교도소 이야기구먼 ..솔직히 재미는 없다..평점 조정 0\n", 210 | "4 6483659 사이몬페그의 익살스런 연기가 돋보였던 영화!스파이더맨에서 늙어보이기만 했던 커스틴 ... 1" 211 | ] 212 | }, 213 | "metadata": { 214 | "tags": [] 215 | }, 216 | "execution_count": 3 217 | } 218 | ] 219 | }, 220 | { 221 | "cell_type": "code", 222 | "metadata": { 223 | "id": "AwDF2m91Aqpm", 224 | "colab_type": "code", 225 | "colab": {} 226 | }, 227 | "source": [ 228 | "# 코드 5-20 데이터셋 임의 순열\n", 229 | "\n", 230 | "data_train = df_train[['label', 'document']].values\n", 231 | "data_test = df_test[['label', 'document']].values\n", 232 | "\n", 233 | "shuffled_train = random.sample(list(data_train), len(data_train))\n", 234 | "training_set = shuffled_train\n", 235 | "shuffled_test = random.sample(list(data_test), len(data_test))\n", 236 | "testing_set = shuffled_test\n", 237 | "\n", 238 | "SEQ_LEN = 256\n", 239 | "CLASS = 2" 240 | ], 241 | "execution_count": 0, 242 | "outputs": [] 243 | }, 244 | { 245 | "cell_type": "code", 246 | "metadata": { 247 | "id": "u_i7-xcA7wAe", 248 | "colab_type": "code", 249 | "colab": {} 250 | }, 251 | "source": [ 252 | "# 코드 5-21 BERT를 위한 토크나이징\n", 253 | "\n", 254 | "tokenizer = FullTokenizer('multi_cased_L-12_H-768_A-12/vocab.txt', do_lower_case=False)\n", 255 | "\n", 256 | "train_tokens = []\n", 257 | "for row in training_set:\n", 258 | " train_tokens.append( [\"[CLS]\"] + tokenizer.tokenize(str(row[1])) + [\"[SEP]\"] )\n", 259 | "\n", 260 | "train_token_ids = list(map(tokenizer.convert_tokens_to_ids, train_tokens))\n", 261 | "train_token_ids = map(lambda tids: tids + [0] * (SEQ_LEN - len(tids)), train_token_ids)\n", 262 | "train_token_ids = np.array([np.array(xi) for xi in list(train_token_ids)])\n", 263 | "\n", 264 | "test_tokens = []\n", 265 | "for row in testing_set:\n", 266 | " test_tokens.append( [\"[CLS]\"] + tokenizer.tokenize(str(row[1])) + [\"[SEP]\"] )\n", 267 | "\n", 268 | "test_token_ids = list(map(tokenizer.convert_tokens_to_ids, test_tokens))\n", 269 | "test_token_ids = map(lambda tids: tids + [0] * (SEQ_LEN - len(tids)), test_token_ids)\n", 270 | "test_token_ids = np.array([np.array(xi) for xi in list(test_token_ids)])\n", 271 | "\n", 272 | "x_train = train_token_ids\n", 273 | "y_train = np.array(training_set)[:,0]\n", 274 | "x_test = test_token_ids\n", 275 | "y_test = np.array(testing_set)[:,0]" 276 | ], 277 | "execution_count": 0, 278 | "outputs": [] 279 | }, 280 | { 281 | "cell_type": "code", 282 | "metadata": { 283 | "id": "Z253nRx1Btkl", 284 | "colab_type": "code", 285 | "outputId": "bfbaeff2-3a9b-49ca-d849-bd84721de8c0", 286 | "colab": { 287 | "base_uri": "https://localhost:8080/", 288 | "height": 865 289 | } 290 | }, 291 | "source": [ 292 | "# 5-22 BERT 기반의 분류 모델 생성\n", 293 | "\n", 294 | "from tensorflow.keras.layers import Input, Flatten, Dense, Dropout, Lambda\n", 295 | "from tensorflow.keras.models import Model\n", 296 | "from tensorflow.keras.optimizers import Adam\n", 297 | "\n", 298 | "bert_params = params_from_pretrained_ckpt(\"multi_cased_L-12_H-768_A-12\")\n", 299 | "bert_layer = BertModelLayer.from_params(bert_params, name=\"bert\")\n", 300 | "bert_layer.apply_adapter_freeze()\n", 301 | "\n", 302 | "def create_model(max_seq_length, classes):\n", 303 | " inputs = Input(shape=(max_seq_length,), dtype='int32', name='input_ids')\n", 304 | " bert = bert_layer(inputs)\n", 305 | " cls_out = Lambda(lambda seq: seq[:, 0, :])(bert)\n", 306 | " dr_1 = Dropout(0.3)(cls_out)\n", 307 | " fc_1 = Dense(256, activation=tf.nn.relu)(dr_1)\n", 308 | " dr_2 = Dropout(0.3)(fc_1)\n", 309 | " outputs = Dense(classes, activation='softmax')(dr_2)\n", 310 | "\n", 311 | " model = Model(inputs, outputs)\n", 312 | "\n", 313 | " return model\n", 314 | "\n", 315 | "model = create_model(SEQ_LEN, CLASS)\n", 316 | "model.build(input_shape=(None, SEQ_LEN))\n", 317 | "\n", 318 | "load_stock_weights(bert_layer, \"multi_cased_L-12_H-768_A-12/bert_model.ckpt\")\n", 319 | "\n", 320 | "def flatten_layers(root_layer):\n", 321 | " if isinstance(root_layer, keras.layers.Layer):\n", 322 | " yield root_layer\n", 323 | " for layer in root_layer._layers:\n", 324 | " for sub_layer in flatten_layers(layer):\n", 325 | " yield sub_layer\n", 326 | "\n", 327 | "for layer in flatten_layers(bert_layer):\n", 328 | " if layer.name in [\"LayerNorm\", \"adapter-down\", \"adapter-up\"]:\n", 329 | " layer.trainable = True\n", 330 | " else:\n", 331 | " layer.trainable = False\n", 332 | "\n", 333 | "bert_layer.embeddings_layer.trainable = False\n", 334 | "\n", 335 | "model.compile(loss='sparse_categorical_crossentropy', optimizer=Adam(lr=0.00001), metrics=['accuracy'])\n", 336 | "\n", 337 | "print(model.summary())" 338 | ], 339 | "execution_count": 0, 340 | "outputs": [ 341 | { 342 | "output_type": "stream", 343 | "text": [ 344 | "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/keras/initializers.py:119: calling RandomUniform.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.\n", 345 | "Instructions for updating:\n", 346 | "Call initializer instance with the dtype argument instead of passing it to the constructor\n", 347 | "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.\n", 348 | "Instructions for updating:\n", 349 | "If using Keras pass *_constraint arguments to layers.\n", 350 | "WARNING:tensorflow:From bert-for-tf2/bert/layer.py:20: The name tf.keras.initializers.TruncatedNormal is deprecated. Please use tf.compat.v1.keras.initializers.TruncatedNormal instead.\n", 351 | "\n", 352 | "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/keras/initializers.py:94: calling TruncatedNormal.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.\n", 353 | "Instructions for updating:\n", 354 | "Call initializer instance with the dtype argument instead of passing it to the constructor\n", 355 | "Done loading 196 BERT weights from: multi_cased_L-12_H-768_A-12/bert_model.ckpt into (prefix:bert). Count of weights not found in the checkpoint was: [0]. Count of weights with mismatched shape: [0]\n", 356 | "Unused weights from checkpoint: \n", 357 | "\tbert/embeddings/token_type_embeddings\n", 358 | "\tbert/pooler/dense/bias\n", 359 | "\tbert/pooler/dense/kernel\n", 360 | "\tcls/predictions/output_bias\n", 361 | "\tcls/predictions/transform/LayerNorm/beta\n", 362 | "\tcls/predictions/transform/LayerNorm/gamma\n", 363 | "\tcls/predictions/transform/dense/bias\n", 364 | "\tcls/predictions/transform/dense/kernel\n", 365 | "\tcls/seq_relationship/output_bias\n", 366 | "\tcls/seq_relationship/output_weights\n", 367 | "Model: \"model\"\n", 368 | "_________________________________________________________________\n", 369 | "Layer (type) Output Shape Param # \n", 370 | "=================================================================\n", 371 | "input_ids (InputLayer) [(None, 256)] 0 \n", 372 | "_________________________________________________________________\n", 373 | "bert (BertModelLayer) (None, 256, 768) 177261312 \n", 374 | "_________________________________________________________________\n", 375 | "lambda (Lambda) (None, 768) 0 \n", 376 | "_________________________________________________________________\n", 377 | "dropout_37 (Dropout) (None, 768) 0 \n", 378 | "_________________________________________________________________\n", 379 | "dense (Dense) (None, 256) 196864 \n", 380 | "_________________________________________________________________\n", 381 | "dropout_38 (Dropout) (None, 256) 0 \n", 382 | "_________________________________________________________________\n", 383 | "dense_1 (Dense) (None, 2) 514 \n", 384 | "=================================================================\n", 385 | "Total params: 177,458,690\n", 386 | "Trainable params: 85,251,842\n", 387 | "Non-trainable params: 92,206,848\n", 388 | "_________________________________________________________________\n", 389 | "None\n" 390 | ], 391 | "name": "stdout" 392 | } 393 | ] 394 | }, 395 | { 396 | "cell_type": "code", 397 | "metadata": { 398 | "id": "t-AsBEa6CK0u", 399 | "colab_type": "code", 400 | "outputId": "bda0fd09-0501-441a-ee34-9bef8d445e75", 401 | "colab": { 402 | "base_uri": "https://localhost:8080/", 403 | "height": 377 404 | } 405 | }, 406 | "source": [ 407 | "# 코드 5-23 콜백을 활용한 모델 학습\n", 408 | "\n", 409 | "checkpointName = \"bert_fine-tuning.ckpt\"\n", 410 | "\n", 411 | "cp_callback = tf.keras.callbacks.ModelCheckpoint(filepath=checkpointName,\n", 412 | " save_weights_only=True,\n", 413 | " verbose=1)\n", 414 | "\n", 415 | "history = model.fit(x_train, y_train, \n", 416 | " epochs=4, batch_size=16,\n", 417 | " validation_data=(x_test, y_test),\n", 418 | " verbose=1, callbacks=[cp_callback]\n", 419 | ")" 420 | ], 421 | "execution_count": 0, 422 | "outputs": [ 423 | { 424 | "output_type": "stream", 425 | "text": [ 426 | "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/math_grad.py:1375: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n", 427 | "Instructions for updating:\n", 428 | "Use tf.where in 2.0, which has the same broadcast rule as np.where\n", 429 | "Train on 10000 samples, validate on 10000 samples\n", 430 | "Epoch 1/4\n", 431 | " 9984/10000 [============================>.] - ETA: 0s - loss: 0.6464 - acc: 0.6074\n", 432 | "Epoch 00001: saving model to bert_fine-tuning.ckpt\n", 433 | "10000/10000 [==============================] - 456s 46ms/sample - loss: 0.6462 - acc: 0.6075 - val_loss: 0.5222 - val_acc: 0.7364\n", 434 | "Epoch 2/4\n", 435 | " 9984/10000 [============================>.] - ETA: 0s - loss: 0.4732 - acc: 0.7785\n", 436 | "Epoch 00002: saving model to bert_fine-tuning.ckpt\n", 437 | "10000/10000 [==============================] - 448s 45ms/sample - loss: 0.4737 - acc: 0.7784 - val_loss: 0.4378 - val_acc: 0.7904\n", 438 | "Epoch 3/4\n", 439 | " 9984/10000 [============================>.] - ETA: 0s - loss: 0.3692 - acc: 0.8375\n", 440 | "Epoch 00003: saving model to bert_fine-tuning.ckpt\n", 441 | "10000/10000 [==============================] - 448s 45ms/sample - loss: 0.3689 - acc: 0.8375 - val_loss: 0.4333 - val_acc: 0.8030\n", 442 | "Epoch 4/4\n", 443 | " 9984/10000 [============================>.] - ETA: 0s - loss: 0.2983 - acc: 0.8743\n", 444 | "Epoch 00004: saving model to bert_fine-tuning.ckpt\n", 445 | "10000/10000 [==============================] - 448s 45ms/sample - loss: 0.2983 - acc: 0.8743 - val_loss: 0.4350 - val_acc: 0.7992\n" 446 | ], 447 | "name": "stdout" 448 | } 449 | ] 450 | }, 451 | { 452 | "cell_type": "code", 453 | "metadata": { 454 | "id": "hL8PCXS30tgP", 455 | "colab_type": "code", 456 | "outputId": "78792e2d-00da-414d-becd-e1a2dce0355f", 457 | "colab": { 458 | "base_uri": "https://localhost:8080/", 459 | "height": 404 460 | } 461 | }, 462 | "source": [ 463 | "# 코드 5-24 결과 그래프 그리기 \n", 464 | "\n", 465 | "plt.figure(figsize=(9,6))\n", 466 | "plt.plot(history.history['loss'], '-', linewidth=2)\n", 467 | "plt.plot(history.history['val_loss'], '-.', linewidth=2)\n", 468 | "plt.title('model loss')\n", 469 | "plt.ylabel('loss')\n", 470 | "plt.xlabel('epoch')\n", 471 | "plt.legend(['train', 'test'], loc='upper left')\n", 472 | "plt.show()" 473 | ], 474 | "execution_count": 0, 475 | "outputs": [ 476 | { 477 | "output_type": "display_data", 478 | "data": { 479 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAi8AAAGDCAYAAADj4vBMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nOzdd3hUZf7+8fdnJo1ACJDQW0BA6S00\nEUVFxYauuAh2VwF31152db/u6ur+dt21d0XsDVBXRXEVFbDSAgJSpUOoIbT0+vz+OIMJiNIynExy\nv65rLjPPnHPmnnjB3DynmXMOERERkUgR8DuAiIiIyKFQeREREZGIovIiIiIiEUXlRURERCKKyouI\niIhEFJUXERERiSgqLyJSqZjZy2b2j4Ncdo2ZDTrS7YhIZFF5ERERkYii8iIiIiIRReVFRA5ZaHfN\n7Wa2wMxyzOwFM2toZv8zsywz+9zM6pZbfoiZLTKznWY2zczal3utu5nNDa03Hojb573OMbN5oXW/\nM7Muh5l5pJmtMLPtZjbRzJqExs3MHjGzrWa228x+MLNOodfOMrPFoWwbzOy2w/qFiUiFUnkRkcM1\nFDgNaAecC/wP+AtQH+/vlhsAzKwd8BZwU+i1j4EPzSzGzGKA94HXgHrA26HtElq3O/AiMBpIAp4D\nJppZ7KEENbNTgH8Bw4DGwFpgXOjl04ETQ58jMbRMZui1F4DRzrkEoBMw5VDeV0TCQ+VFRA7XE865\nLc65DcDXwEzn3PfOuXzgPaB7aLmLgEnOuc+cc0XAg0AN4HigLxANPOqcK3LOvQPMLvceo4DnnHMz\nnXMlzrlXgILQeofiEuBF59xc51wBcCfQz8xSgCIgATgOMOfcEufcptB6RUAHM6vtnNvhnJt7iO8r\nImGg8iIih2tLuZ/z9vO8VujnJngzHQA450qB9UDT0Gsb3N53iF1b7ueWwK2hXUY7zWwn0Dy03qHY\nN0M23uxKU+fcFOBJ4Clgq5mNMbPaoUWHAmcBa83sSzPrd4jvKyJhoPIiIuG2Ea+EAN4xJngFZAOw\nCWgaGtujRbmf1wP/zzlXp9wj3jn31hFmqIm3G2oDgHPucedcT6AD3u6j20Pjs51z5wEN8HZvTTjE\n9xWRMFB5EZFwmwCcbWanmlk0cCverp/vgOlAMXCDmUWb2QVA73LrPg9ca2Z9QgfW1jSzs80s4RAz\nvAVcZWbdQsfL/BNvN9caM+sV2n40kAPkA6WhY3IuMbPE0O6u3UDpEfweRKSCqLyISFg555YBlwJP\nANvwDu491zlX6JwrBC4ArgS24x0f899y66YBI/F26+wAVoSWPdQMnwN/Bd7Fm+05Bhgeerk2Xkna\ngbdrKRN4IPTaZcAaM9sNXIt37IyI+Mz23tUsIiIiUrlp5kVEREQiisqLiIiIRBSVFxEREYkoKi8i\nIiISUVReREREJKJE+R2goiQnJ7uUlBS/Y4iIiEgFmTNnzjbnXP19x6tMeUlJSSEtLc3vGCIiIlJB\nzGzt/sa120hEREQiisqLiIiIRBSVFxEREYkoVeaYl/0pKioiPT2d/Px8v6OEXVxcHM2aNSM6Otrv\nKCIiImFVpctLeno6CQkJpKSkYGZ+xwkb5xyZmZmkp6fTqlUrv+OIiIiEVZXebZSfn09SUlKVLi4A\nZkZSUlK1mGESEREJa3kxs8FmtszMVpjZHb+wzDAzW2xmi8zszXLjJWY2L/SYeAQZDnfViFJdPqeI\niEjYyouZBYGngDOBDsAIM+uwzzJtgTuB/s65jsBN5V7Oc851Cz2GhCtnuO3cuZOnn376kNc766yz\n2LlzZxgSiYiIRLZwzrz0BlY451Y55wqBccB5+ywzEnjKObcDwDm3NYx5fPFL5aW4uPhX1/v444+p\nU6dOuGKJiIhErHCWl6bA+nLP00Nj5bUD2pnZt2Y2w8wGl3stzszSQuPn7+8NzGxUaJm0jIyMik1f\nQe644w5WrlxJt27d6NWrFwMGDGDIkCF06OBNQp1//vn07NmTjh07MmbMmJ/WS0lJYdu2baxZs4b2\n7dszcuRIOnbsyOmnn05eXp5fH0dERMR3fp9tFAW0BQYCzYCvzKyzc24n0NI5t8HMWgNTzOwH59zK\n8is758YAYwBSU1Pdr71Ryh2TwpGfNfef/auv33///SxcuJB58+Yxbdo0zj77bBYuXPjTWUEvvvgi\n9erVIy8vj169ejF06FCSkpL22sby5ct56623eP755xk2bBjvvvsul156aVg+j4iISGUXzpmXDUDz\ncs+bhcbKSwcmOueKnHOrgR/xygzOuQ2h/64CpgHdw5j1qOndu/depzM//vjjdO3alb59+7J+/XqW\nL1/+s3VatWpFt27dAOjZsydr1qw5WnFFREQqnXDOvMwG2ppZK7zSMhy4eJ9l3gdGAC+ZWTLebqRV\nZlYXyHXOFYTG+wP/OZIwB5oh2Z9S5/hxcxaFJaUkxEXTsl48gcCRndVTs2bNn36eNm0an3/+OdOn\nTyc+Pp6BAwfu93Tn2NjYn34OBoPabSQiItVa2GZenHPFwHXAp8ASYIJzbpGZ3Wtme84e+hTINLPF\nwFTgdudcJtAeSDOz+aHx+51zi8OV9ZcEzGiZVJOoQICs/CJWZ+ZQUvqre6d+JiEhgaysrP2+tmvX\nLurWrUt8fDxLly5lxowZFRFbRESkSgvrMS/OuY+Bj/cZ+1u5nx1wS+hRfpnvgM7hzHawasQEaV2/\nJqu35ZBTUMzqbTm0So4nGDi43peUlET//v3p1KkTNWrUoGHDhj+9NnjwYJ599lnat2/PscceS9++\nfcP1MURERKoM8/pD5EtNTXVpaWl7jS1ZsoT27dtXyPYLikpYtS2HopJSasQEaZVUk6hg5bpAcUV+\nXhEREb+Z2RznXOq+45Xr27cSi40Ockz9msREBcgr9IpMcUmp37FERESqHZWXQxATFeSY5FrERgXJ\nLyphZYY3EyMiIiJHj8rLIYqOCtC6fk3iooMUFJewKiObwmIVGBERkaNF5eUwRAcDtE6uSY3oIAXF\npaECU+J3LBERkWpB5eUwRQUDtEquSXxMFIUlpazMyKGgSAVGREQk3FRejoBXYOKpGRNFUajA5KvA\niIiIhJXKyxEKBgKkJNekVmwUxaWlrMrIIa+w7I7Rv3RX6YPx6KOPkpubW1FRRUREqgSVlwoQDBgp\nSTVJiIv2Csy2HHJDBUblRUREpGL5fVfpKiMQMFomxbMuM5fd+UWszsghJbkmd9xxBytXrqRbt26c\ndtppNGjQgAkTJlBQUMBvfvMb/v73v5OTk8OwYcNIT0+npKSEv/71r2zZsoWNGzdy8sknk5yczNSp\nU/3+iCIiIpVC9Sov9yQe2vKNu8Lor36+/j279rt4wIwWSfGkb89jZ14hq7fl8Ne/38fChQuZN28e\nkydP5p133mHWrFk45xgyZAhfffUVGRkZNGnShEmTJgHePY8SExN5+OGHmTp1KsnJyYfzaUVERKok\n7TaqYAEzmterQd34GEqdI31HHqWhWzBMnjyZyZMn0717d3r06MHSpUtZvnw5nTt35rPPPuPPf/4z\nX3/9NYmJh1iyREREqpFqNvOy/xmTil7fzGhWtwZmsMFBYYljV14RzjnuvPNORo8e/bN15s6dy8cf\nf8xdd93Fqaeeyt/+9rf9bFlEREQ08xImZkbTOjVo3rAeudlZrMvMpf9Jp/Diiy+SnZ0NwIYNG9i6\ndSsbN24kPj6eSy+9lNtvv525c+cCkJCQQFZWlp8fQ0REpNKpXjMvR5mZ0bFVU/r0O57fnNqXE04e\nxPlDh9GvXz8AatWqxeuvv86KFSu4/fbbCQQCREdH88wzzwAwatQoBg8eTJMmTXTAroiISIi50PEY\nkS41NdWlpaXtNbZkyRLat2/vU6Iyzjm2ZhWwZXc+AE3r1CCpVmyFv09l+bwiIiIVwczmOOdS9x3X\nbqOjwMxoWDuORolxAGzYmce27AKfU4mIiEQmlZejqEFCHE3q1ABg4848MrLyfU4kIiISeVRejrLk\nWrE0DRWYTbvy2bI7n6qy605ERORoqPLlpTIWg6RasTSvG48BW3ZXTIGpjJ9TREQkHKp0eYmLiyMz\nM7NSfrHXrRlD83rxGMbWrAI27Tr8AuOcIzMzk7i4uApOKSIiUvlU6VOlmzVrRnp6OhkZGX5H+UVF\nRSVszylki4O1sUESa8RgdujbiYuLo1mzZhUfUEREpJKp0uUlOjqaVq1a+R3jgKYu28q1r82hoLiU\nC3s2499DuxAMHEaDERERqQaq9G6jSHHysQ146cpe1IgO8s6cdG4eP4+iklK/Y4mIiFRKKi+VxPFt\nknn16t7Uio1i4vyNXPfmXAqLVWBERET2pfJSifRKqcfr1/ShdlwUny7awrWvzyG/qMTvWCIiIpWK\nyksl0615Hd4c2Ze68dFMWbqVa15JI7ew2O9YIiIilYbKSyXUqWki40f3I7lWLN+s2MaVL84mu0AF\nRkREBFReKq12DROYMLovjWrHMWvNdi57YSa78or8jiUiIuI7lZdKrHX9WkwY3Y9mdWvw/bqdXDJ2\nBjtyCv2OJSIi4iuVl0quRVI8E0b3IyUpnoUbdjN8zAwysnRHahERqb5UXiJAkzo1mDC6H20a1GLZ\nliwuGjOdzbt0R2oREameVF4iRIPacYwb1ZfjGiWwKiOHYc9NJ31Hrt+xREREjrqwlhczG2xmy8xs\nhZnd8QvLDDOzxWa2yMzeLDd+hZktDz2uCGfOSJFcK5Zxo/rSuWki67bnctFzM1izLcfvWCIiIkdV\n2MqLmQWBp4AzgQ7ACDPrsM8ybYE7gf7OuY7ATaHxesDdQB+gN3C3mdUNV9ZIUic+hjdG9qFHizps\n2JnHsOems2Jrtt+xREREjppwzrz0BlY451Y55wqBccB5+ywzEnjKObcDwDm3NTR+BvCZc2576LXP\ngMFhzBpRasdF8+rVfejTqh5bswoYPmY6Szfv9juWiIjIURHO8tIUWF/ueXporLx2QDsz+9bMZpjZ\n4ENYt1qrFRvFy1f1ZkDbZLZlFzJ8zAx+SN/ldywREZGw8/uA3SigLTAQGAE8b2Z1DnZlMxtlZmlm\nlpaRkRGmiJVXjZggz1+eyqD2DdiZW8TFY2cwZ+0Ov2OJiIiEVTjLywagebnnzUJj5aUDE51zRc65\n1cCPeGXmYNbFOTfGOZfqnEutX79+hYaPFHHRQZ6+pCdndmpEVn4xl78wk5mrMv2OJSIiEjbhLC+z\ngbZm1srMYoDhwMR9lnkfb9YFM0vG2420CvgUON3M6oYO1D09NCb7ERMV4IkR3Tm/WxNyCku44qVZ\nfL28+s1EiYhI9RC28uKcKwauwysdS4AJzrlFZnavmQ0JLfYpkGlmi4GpwO3OuUzn3HbgPrwCNBu4\nNzQmvyAqGOChYd24KLU5+UWlXP1KGl8s2eJ3LBERkQpnzjm/M1SI1NRUl5aW5ncM35WWOu6euIjX\nZqwlOmg8MaI7gzs19juWiIjIITOzOc651H3H/T5gVypYIGDce15HRg5oRVGJ449vfs8H8352uJCI\niEjEUnmpgsyMv5zVnutPaUNJqeOm8fOYkLb+wCuKiIhEAJWXKsrMuPX0Y7nt9HY4B396ZwGvzVjr\ndywREZEjpvJSxV13SlvuOrs9AH99fyFjv17lcyIREZEjo/JSDVwzoDX3ndcRgH9MWsJTU1f4nEhE\nROTwqbxUE5f1S+E/Q7tgBg98uoyHJy+jqpxpJiIi1YvKSzUyrFdzHhnWjWDAeHzKCv71v6UqMCIi\nEnFUXqqZ87s35ckR3YkKGGO+WsU9ExdRWqoCIyIikUPlpRo6s3Njnr20JzHBAK9MX8tf3vuBEhUY\nERGJECov1dSgDg0Ze0UqcdEBxs1ez21vz6e4pNTvWCIiIgek8lKNndiuPi9f1Zv4mCDvfb+BG8fN\no0gFRkREKjmVl2qub+skXru6NwmxUUz6YRO/f30uBcUlfscSERH5RSovQs+W9XhjZB8Sa0Tz+ZIt\njHx1DnmFKjAiIlI5qbwIAF2a1WHcqL4k1Yzhqx8zuOrlWeQUFPsdS0RE5GdUXuQn7RvXZvzovjRI\niGXGqu1c/uIsducX+R1LRERkLyovspc2DRKYMLofTRLjmLN2B5eOncnO3EK/Y4mIiPxE5UV+JiW5\nJuNH96NFvXgWpO9ixPMzycwu8DuWiIgIoPIiv6B5vXgmjO5H6+SaLNm0m+FjZrB1d77fsURERFRe\n5Jc1Soxj3Oi+tGtYi+Vbs7lozAw27szzO5aIiFRzKi/yqxokxDFuVD86NqnN6m05DHtuOuu35/od\nS0REqjGVFzmgejVjePOavnRrXof0HXkMe246qzKy/Y4lIiLVlMqLHJTE+Gheu7o3vVLqsmlXPheN\nmcHyLVl+xxIRkWpI5UUOWkJcNK/8rjf92ySRkVXARWNmsGjjLr9jiYhINaPyIockPiaKF67oxcnH\n1md7TiEjxsxg/vqdfscSEZFqROVFDllcdJBnL+vJ6R0asju/mEvGziRtzXa/Y4mISDWh8iKHJTYq\nyFOX9ODcrk3ILijm8hdn8d3KbX7HEhGRakDlRQ5bdDDAoxd1Y2iPZuQWlnDVS7OZtmyr37FERKSK\nU3mRIxIMGA9c2IWL+7SgoLiUUa/OYfKizX7HEhGRKkzlRY5YIGD8v/M7cVX/FApLSvnDG3OZtGCT\n37FERKSKUnmRCmFm/O2cDvx+4DEUlzquf2su/52b7ncsERGpglRepMKYGX8641huHtSOUge3vj2f\nt2at8zuWiIhUMSovUqHMjBsHteWOM4/DObjzvz/wyndr/I4lIiJViMqLhMW1Jx3D3ed2AODuiYt4\n7suVPicSEZGqQuVFwuaq/q345286Ywb/+t9SHv9iOc45v2OJiEiEC2t5MbPBZrbMzFaY2R37ef1K\nM8sws3mhxzXlXispNz4xnDklfC7u04IHL+xKwODhz37kgU+XqcCIiMgRiQrXhs0sCDwFnAakA7PN\nbKJzbvE+i453zl23n03kOee6hSufHD1DezYjJirATePn8fS0leQXlfLXc9pjZn5HExGRCBTOmZfe\nwArn3CrnXCEwDjgvjO8nldi5XZvw9CU9iA4aL367mrveX0hpqWZgRETk0IWzvDQF1pd7nh4a29dQ\nM1tgZu+YWfNy43FmlmZmM8zs/P29gZmNCi2TlpGRUYHRJRzO6NiIMZenEhsV4I2Z6/jTuwsoUYER\nEZFD5PcBux8CKc65LsBnwCvlXmvpnEsFLgYeNbNj9l3ZOTfGOZfqnEutX7/+0UksR+TkYxvw0pW9\nqBEd5J056dw8fh5FJaV+xxIRkQgSzvKyASg/k9IsNPYT51ymc64g9HQs0LPcaxtC/10FTAO6hzGr\nHEXHt0nm1at7Uys2ionzN3Ldm3MpLFaBERGRgxPO8jIbaGtmrcwsBhgO7HXWkJk1Lvd0CLAkNF7X\nzGJDPycD/YF9D/SVCNYrpR6vX9OH2nFRfLpoC9e+Pof8ohK/Y4mISAQIW3lxzhUD1wGf4pWSCc65\nRWZ2r5kNCS12g5ktMrP5wA3AlaHx9kBaaHwqcP9+zlKSCNeteR3eHNmXuvHRTFm6lWteSSO3sNjv\nWCIiUslZVbnmRmpqqktLS/M7hhyGH7dkcfHzM9mWXUDvlHq8eFUvasWG7Sx+ERGJEGY2J3T86178\nPmBXhHYNE5gwui+Nascxa812LnthJrvyivyOJSIilZTKi1QKrevXYsLofjSrW4Pv1+3kkrEz2JFT\n6HcsERGphFRepNJokRTPhNH9SEmKZ+GG3QwfM4OMrIIDrygiItWKyotUKk3q1GDC6H60aVCLZVuy\nuGjMdDbvyvc7loiIVCIqL1LpNKgdx7hRfTmuUQKrMnIY9tx00nfk+h1LREQqCZUXqZSSa8UyblRf\nujRLZN32XC56bgZrtuX4HUtERCoBlReptOrEx/D6NX3o2bIuG3bmMey56azYmu13LBER8ZnKi1Rq\nteOiefV3venbuh5bswoYPmY6Szfv9juWiIj4SOVFKr2asVG8dGVvBrRNZlt2IcPHzOCH9F1+xxIR\nEZ+ovEhEqBET5PnLUxnUvgE7c4u4eOwM5qzd4XcsERHxgcqLRIy46CBPX9KTMzs1Iiu/mMtfmMnM\nVZl+xxIRkaNM5UUiSkxUgCdGdOf8bk3IKSzhipdm8c3ybX7HEhGRo0jlRSJOVDDAQ8O6cVFqc/KL\nSvndK7OZsnSL37FEROQoUXmRiBQMGP+6oDOX9W1JYXEpo1+bwycLN/sdS0REjgKVF4lYgYBx73kd\nGTmgFUUljj++OZcP5m3wO5aIiISZyotENDPjL2e15/pT2lBS6rhp/DzeTlvvdywREQkjlReJeGbG\nracfy22nt8M5uP2dBbw+Y63fsUREJExUXg7G/PEweywU6e7Gldl1p7TlrrPbA3DX+wt54ZvVPicS\nEZFwUHk5kOJC+OLvMOlWeLQzfP0w5OvqrpXVNQNac995HQG476PFPDV1hc+JRESkoqm8HEggCGf8\nExp1gZytXpF5pDN8/nfIzvA7nezHZf1S+M/QLpjBA58u4+HPfsQ553csERGpIFZV/lJPTU11aWlp\n4XsD52DlF/D1I7D2G28sKg66Xwb9b4A6LcL33nJY3v9+A7e+PZ+SUsfoE1tzx5nHYWZ+xxIRkYNk\nZnOcc6n7jmvm5WCZQZtBcNUkuPozaHcmFOfD7OfhsW7w39GwdanfKaWc87s35ckR3YkKGM99tYq/\nf7iY0tKqUdZFRKozlZfD0bw3XDwOfj8dOg/zxhaMg6f7wFsXa3dSJXJm58Y8e2lPYoIBXv5uDf/3\n/g8qMCIiEU7l5Ug07ABDn4cb5kLq1RCMhc0/QI06fieTcgZ1aMjYK1KJiw7w1qz13Pb2fIpLSv2O\nJSIih0nlpSLUTYFzHoabfoALX4BgtDeenQEvnglLPvQ1nsCJ7erz8lW9iY8J8t/vN3DjuHkUqcCI\niEQklZeKlNDQ26W0R9qLsO47+P4N/zLJT/q2TuK1q3uTEBvFpB828Yc35lJQXOJ3LBEROUQqL+F0\n/PVw5gNw0p/KxtLTYMYzUJjjX65qrGfLerwxsg+JNaL5bPEWRr06h/wiFRgRkUii8hJOMfHQZxQ0\n7VE2Nu1++OQO74J3Xz4AeTv8y1dNdWlWh3Gj+pJUM4Yvf8zgqpdmk1NQ7HcsERE5SCovR1vqVdC0\nJ+RmwtR/wCOdYPJfIWuz38mqlfaNazN+dF8aJMQyfVUmV7w4i935RX7HEhGRg6DycrQddzZc8wVc\nPhFaD4TCbPjucXi0C3x4E2zX/XiOljYNEpgwuh9NEuNIW7uDy8bOZGduod+xRETkAFRe/GAGrU+C\nyz+AkVOg/blQUghzXoInesA7V8PmhX6nrBZSkmsyfnQ/WtSLZ376LkY8P5PM7AK/Y4mIyK9QefFb\n055w0evwx5nQ7RKwACx8B57tD28Mg8yVfies8prXi2fC6H60Tq7Jkk27GT5mBlt36w7iIiKVlcpL\nZVH/WDj/abhhHvQeDVE1YOUU7/5JEnaNEuMYN7ov7RrWYvnWbC4aM4ONO/P8jiUiIvuh8lLZ1GkO\nZ/0Hbl4Iv30JEpt646UlMOFyWPiu97NUuAYJcYwb1Y+OTWqzelsOw56bzvrtuX7HEhGRfYS1vJjZ\nYDNbZmYrzOyO/bx+pZllmNm80OOacq9dYWbLQ48rwpmzUqqZ7B0Ls8eyj2HxB/DZPeB0ZdhwqVcz\nhjev6Uu35nVI35HHsOemsyoj2+9YIiJSTtjKi5kFgaeAM4EOwAgz67CfRcc757qFHmND69YD7gb6\nAL2Bu82sbriyRoQ2p8HZD8Ogu/e+/cB3T0KBvlwrUmJ8NK9d3ZteKXXZtCufi8bMYPmWLL9jiYhI\nSDhnXnoDK5xzq5xzhcA44LyDXPcM4DPn3Hbn3A7gM2BwmHJGhug46HU1dL6wbGzmMzD5/+DRTjD1\nX5C73b98VUxCXDSv/K43/dskkZFVwEVjZrBo4y6/Y4mICOEtL02B9eWep4fG9jXUzBaY2Ttm1vxQ\n1jWzUWaWZmZpGRkZFZU7crTsD837eFfp/fJ+74J3n/wFdm3wO1mVEB8TxQtX9OLkY+uzPaeQEWNm\nMH/9Tr9jiYhUe34fsPshkOKc64I3u/LKoazsnBvjnEt1zqXWr18/LAErtTanwu8+hSs/hjaDoCgH\nZjwFj3WFD66DbSv8Thjx4qKDPHtZT07v0JDd+cVcMnYmaWs0wyUi4qdwlpcNQPNyz5uFxn7inMt0\nzu25IthYoOfBrishZpDSHy59F0Z/BR1/A6XF8P1r8GQqTLgCNs33O2VEi40K8tQlPTi3axOyC4q5\n/MVZfLdym9+xRESqrXCWl9lAWzNrZWYxwHBgYvkFzKxxuadDgCWhnz8FTjezuqEDdU8PjcmvadwV\nfvsyXD8HelwOgShY/D48dyK8dgFs/N7vhBErOhjg0Yu6MbRHM3ILS7jqpdlMW7bV71giItVS2MqL\nc64YuA6vdCwBJjjnFpnZvWY2JLTYDWa2yMzmAzcAV4bW3Q7ch1eAZgP3hsbkYCQdA0OegJsWQL/r\nILomrPwCcjL9ThbRggHjgQu7cHGfFhQUlzLq1TlMXqQbaoqIHG3mnPM7Q4VITU11aWlpfseonHK3\nexe363WNt5sJ4Iv7vKv6drwAglH+5oswzjnu/WgxL327hqiA8djw7pzdpfGBVxQRkUNiZnOcc6n7\njutbqzqIrwe9R5Y9z1wJ3zwMFvTOWErc30lg8kvMjL+d04G46CDPTFvJ9W/NpaC4Kxf0aOZ3NBGR\nasHvs43ED4nN4JxHYcCt5W4/UAqzx0L+bn+zRQgz409nHMvNg9pR6uDWt+fz1qx1fscSEakWNPNS\nHUXFQs997riw9COYdCt8fi/0vgb6/B5qVcPTzw+BmXHjoLbERge4/39LufO/P1BYXMoVx6f4HU1E\npErTzIt4ajX0diEV7IKvH4JHO8PHt8NOzSYcyLUnHcPd53p3vrh74iKe+3Klz4lERKo2lRfxtOgD\nV30Mv5sM7c6E4jyYNQYe7w7vXQtbl/qdsFK7qn8r/vmbzpjBv/63lMe/WE5VORheRKSyUXmRvbXo\nAxePg99/B51/693Bev5b8OE+vSUAACAASURBVHQfGHcJpM/xO2GldXGfFjx4YVcCBg9/9iMPfLpM\nBUZEJAxUXmT/GnaEoWPh+rmQ+jsIxnrHxYw9BV45F9Z863fCSmloz2Y8Nrw7wYDx9LSV3PfREhUY\nEZEKpvIiv65eKzjnEbjpB+h/I8QkwOqvYP0Mv5NVWud2bcLTl/QgOmi8+O1q7np/IaWlKjAiIhVF\n5UUOTkJDOO1euHkhnPo374J3e8x7C+a9CSVF/uWrZM7o2Igxl6cSGxXgjZnr+PO7CyhRgRERqRAq\nL3JoatTxrg8Tl+g9L8qHz++G93/vzcjIT04+tgEvXdmLGtFB3p6Tzi0T5lFcUup3LBGRiKfyIkcm\nEIRBf/duM3DMKWXjiydC3k7/clUSx7dJ5tWre1MrNooP5m3kuje/p7BYBUZE5EgcVHkxsxvNrLZ5\nXjCzuWZ2erjDSQQIRkO3EfDbl8rum5S5Et6+Ah7pBJ/9DbK2+JvRZ71S6vH6NX2oHRfFJ4s2c+3r\nc8gvKvE7lohIxDrYmZffOed2A6cDdYHLgPvDlkoiW3EBpAyAwiz49jHvgncf3QzbV/udzDfdmtfh\nzZF9qRsfzZSlWxn5ahp5hSowIiKH42DLS+if1JwFvOacW1RuTGRvDTvAFRNh5BQ47hwoKYC0F+GJ\nnvDuNbBlkd8JfdGpaSLjR/cjuVYsXy/fxhUvzSK7oNjvWCIiEedgy8scM5uMV14+NbMEQDvu5dc1\n7QnD34A/zoKuF3u7lX54G545Ht68CNbN9DvhUdeuYQITRvelUe04Zq3ezmUvzGRXns7SEhE5FHYw\nF9AyswDQDVjlnNtpZvWAZs65BeEOeLBSU1NdWlqa3zHk1+xcB989CXNf9W4/AN79lAbcAm0G+Zvt\nKFuXmcvFY2eQviOPTk1r8/zlqTROrOF3LBGRSsXM5jjnUvcdP9iZl37AslBxuRS4C9hVkQGlGqjT\nAs76j3fBuwG3QWwirP0W5rzid7KjrkVSPBNG9yMlKZ6FG3Zz6kNf8tTUFRQU6zgYEZEDOdiZlwVA\nV6AL8DIwFhjmnDsprOkOgWZeIlD+bu9YmGNOhsZdvbF1MyBjGXQdDlGx/uY7CrZm5XP3B4v438LN\nALRMiudv53TglOMaYKbDykSkejvSmZdi57Wc84AnnXNPAQkVGVCqobjacMJNZcUFYMo/4MMbvDta\nVwMNEuJ45tKevHFNH9o2qMXazFyufiWNq16ezcqMbL/jiYhUSgdbXrLM7E68U6QnhY6BiQ5fLKmW\nnIMeV0DzvtD9srLxdTMgd7t/uY6C/m2S+fjGAfztnA4kxEUxbVkGgx/9in99vISsfB3QKyJS3sHu\nNmoEXAzMds59bWYtgIHOuVfDHfBgabdRFVWUD491gYJs6HklHH8d1G7id6qw2pZdwIOfLmN82nqc\ng/oJsdx55nGc360pgYB2JYlI9fFLu40OqryENtAQ6BV6Oss5t7UC8x0xlZcqavdGmHg9rPjcex6I\n9o6HOeFmSDrG32xhtiB9J3dPXMT367zbLPRoUYd7hnSkS7M6PicTETk6jqi8mNkw4AFgGt7F6QYA\ntzvn3qngnIdN5aWK2zgPvnkEFn8AOMCg4/leiSl/zEwVU1rqeO/7Ddz/yVIysgowg4tSm3PbGceS\nXKvqH9AsItXbkZaX+cBpe2ZbzKw+8LlzrtJ8a6i8VBPbVsC3j8L8cVAaOhakzSA44RZoeXzZ/ZWq\nmKz8Ip6csoIXv11NUYkjIS6KW05rx6V9WxId1P1VRaRqOtLy8oNzrnO55wFgfvkxv6m8VDO7NsD0\np2DOS1CU64017+OVmHZnVNkSszIjm3s/XMyXP2YA0K5hLe4+tyP92yT7nExEpOIdaXl5AO8aL2+F\nhi4CFjjn/lyhKY+Ayks1lbsdZj4HM5+F/J1QrzVclwaBoN/JwsY5xxdLtnLvR4tZt90rbmd2asRf\nzmpP83rxPqcTEak4FXHA7lCgf+jp18659yow3xFTeanmCrJhzsvemUidLvDGsjbD0knQ7RKIjvM1\nXjjkF5XwwjereXLKCvKKSoiNCvD7gcdw7UnHEBdddcubiFQfR1xeKjuVF/mZyXfBd09A90vhvKf8\nThM2m3bl8a+PlzJx/kYAmtapwV1nt2dwp0a6Sq+IRLTDusKumWWZ2e79PLLMbHf44opUgOZ9oVEX\n6HVN2di2FZCzzb9MYdA4sQaPj+jOhNH9aN+4Nht25vH7N+ZyydiZ/Lgly+94IiIVTjMvUrU5t/fB\nu6+cC+tnQ4/L4fjroU5z/7KFQUmp461Z63hw8jJ25hYRDBiX9W3Jzae1I7GGLootIpHlSO9tJBKZ\nyheXojyIjofiPJj1HDzeDd77vXcjyCoiGDAu7duSabcN5LK+LXHO8fJ3azj5wWmMm7WOktKq8Y8V\nEaneNPMi1c/mhd4F7xb9F1wpYHDc2TDgFmja0+90FWrxxt3c8+EiZq327g3VuWki9wzpSM+WdX1O\nJiJyYDpgV2Rf21fBt4/DvDegpNAbaz3Qu1ZMqxOrzLVinHN8tGAT//x4CZt25QNwQfem3HHmcTSo\nXfXOwhKRqsOX8mJmg4HHgCAw1jl3/y8sNxR4B+jlnEszsxRgCbBnPn+Gc+7aX3svlRc5bFmbvQve\npb0IhdneWNOeXok59iwIVI29q7mFxTw9dSVjvlpFYUkpNWOC3HBqW67q34qYqKrxGUWkajnq5cXM\ngsCPwGlAOjAbGOGcW7zPcgnAJCAGuK5cefnIOdfpYN9P5UWOWN4OmDUWZj4DuZkQjIWbfoCEhn4n\nq1BrM3P4x6QlfLZ4CwCtk2vy13M7cPKxDXxOJiKyNz8O2O0NrHDOrXLOFQLjgPP2s9x9wL+B/DBm\nETmwGnXhpNu9wjL4394xMHuKS0kxzH3VO+g3wrVMqsnzl6fyyu9607p+TVZty+Gql2Zz9cuzWbMt\nx+94IiIHFM7y0hRYX+55emjsJ2bWA2junJu0n/Vbmdn3ZvalmQ0IY06RvcXUhL7XwsA7ysYWvQcT\nr4dX99e/I9NJ7erzyY0n8n9ntadWbBRfLN3K6Y98xX8+WUpOQbHf8UREfpFvO7pDN3d8GLh1Py9v\nAlo457oDtwBvmlnt/WxjlJmlmVlaRkZGeANL9VarPjTpDt0uLhvL3Q5ZW/zLVAFiogKMPLE1U247\niQt7NqOwpJSnp63klIem8cG8DVSVA/pFpGoJ5zEv/YB7nHNnhJ7fCeCc+1foeSKwEggdIUkjYDsw\nxDmXts+2pgG37Tteno55kbBzzju1es9NHyffBTPHeLcf6H8D1E3xNV5F+H7dDu6ZuIj56bsA6JVS\nl3uGdKRjk0Sfk4lIdeTHMS+zgbZm1srMYoDhwMQ9Lzrndjnnkp1zKc65FGAGoeJiZvVDB/xiZq2B\ntsCqMGYVOTCzve9WnZ0BJQWQ9gI83gPeHQlbFv/y+hGge4u6vPeH/vxnaBeSasYwe80Ozn3iG/7v\nvR/YnlPodzwRESCM5cU5VwxcB3yKd9rzBOfcIjO718yGHGD1E4EFZjYP7xTqa51z28OVVeSwXPAc\n/GEmdB3hPf9hAjzTD94cDutn+ZvtCAQCxrBezZly20CuPqEVZsYbM9dx8oPTeHX6GopLSv2OKCLV\nnC5SJ1IRdq7z7mA991UoDp041/IEGHAzHHNqRF/wbvmWLP7+4WK+WeHd0PK4RgncM6QjfVsn+ZxM\nRKo6XWFX5GjIzvCuEzNrLBR4x43QqIt32nX7IXvvdoogzjk+XbSFf0xaTPoO73Txc7o05i9ntadJ\nnRo+pxORqkrlReRoyt/lXbF3+tOQs9Ubu+p/0PJ4f3MdofyiEsZ8tYqnp60gv6iUGtFB/jDwGEae\n2Jq46MgsZiJSeam8iPihKM+7d9Kab+HCF8t2Hy39GFqf5F1TJgJt2JnHPz9ewqQFmwBoXq8Gfz27\nA6d1aIhF8C4yEalcVF5EKouMZfBUb0hoAjfOg6hYvxMdtu9WbuPvExezbEsWAAPaJnP3uR1p06CW\nz8lEpCr4pfIS5UcYkWqtKBea9YZGncuKy9rvvFOto2IgWO4RFQvBaO8+S8Hovcfang7tz/XW374a\nFr4D9VpDp6HeWHEhLP7AW3av7cSUe5/Ycq/HQFQcxMQf9Ec5/phkJt1wAm/MXMdDk5fx9fJtDH70\nK648PoUbBrWldlx0Bf/yRERUXkSOvibd4erJUFLuuikF2bA7/dC2U7N+WXnJXAlT/gHHnFJWXgqy\n4L/XHNo2jzkFLnvP+zl3Ozzaxbu68A3fly3z+tCyG1cGo4mKiuWKYAzD20WxeGs+P2YWUjQjyKQ5\nsXRv1ZB2LZsROOm2svXnj/euj9PxAogNzdBsmu/dGPNnheoXClcVudO3iBwelRcRP5jtvbuo1Yne\nDSFLiqC4wCs2ex7FBd54ScHerzfqUrZ+3ZZwwi3ezMsegaBXZEoKvVmYg9lmbELZ+sUFUJgFhfvM\nxGxeCNmbf/aRYoHuQPc9x+06YBXsWF2Xta2voVvzOt7453dD1ibvFPI95WXa/bDs40P4/QW931di\n6HZpE6+HNd/A2Q/DMSd7Y/PHeY9fm2kqP6sVFQvHnV32O9y6BLYth/rHQf123ljeTtixptz6Mfsp\nVsGIPjVewsQ5788UDqLLnaGXkwmuBEpLvCt4u9B/S0tCV/Uu/7wUAlHQsEPZ+utneX92m/eFYOgr\nPX0O5GTsZ3ulZY9936/bJWV/Jy1637v8Q8fzoU4Lb2zlVFg/8+fr1WwAx193VH6F5am8iFQG0XFl\nf0kcjuS2MOjuvcdq1PEOEj5ctRrCHeu8v6zKu+w970DkksKfl5/QwxUXsHDdNr5cnM72AuPFp77l\ntz2b8afBx1G/828hb3tZcQFo2NGbKdpfofqpcBWWvacr8crCHrs3wvZV3np7ZK6AVVMP7TMntS0r\nL4vegy//DSfdASff6Y2t/Q7GjTjARmzvcnPCTXD89d5La77xbiuRMgBOv88by90OH938KzNN+ylJ\ndVO8wgve72ndDIipBc16lsXYud4rUsGYsltb/NIXmSv1CmFym7L1N87zfp9Nupd9KW6a7826uVIo\nLT24L8fOF5Z9KS79GHath2PPgjrNvbHVX0P6rHLbO0DO0hLoPaqsUC56D36cDB3Og2MHe2PrZ8F3\nj3ufe7+lYD85e11ddu+yFV/Ap3+B1ifDmfeX/T5fOvPn29vf72LPdtufC8Ne8dbfsQYe7wZ1WsJN\nC8p+z090985OPFj7rv/Ghd76f1oN8fW8sS/+Dqu/PPhtAnT8Tdn/pzkvwapp0KhT2d9Lq6bBt4/+\nfL0GHVVeRKQSCQQgbj/3NCr/r75fYEDn3tCqoJgnp6wg+ptVvD0nnU8WbubGQVdxxakpRAfL7fo5\n5a6Dz+UclBZ7/wLd47ynvfKT0KhsrNvF0KLf/svPL5Wjeq3K1k9uB8edA/WPLRuLqenNeP3aNl2J\nd6HCPRcrLC4oWz83EzZ+D7Wblo0VZsPi9w/+84OXa095yd4Krw6B2s3glkVly4w5yXu/g1WnhTeb\ntcer50H+zr2/FD/7m/cldiiOPbPsS3HWGK9QJrUpKy8rv4BvHjn0be4pL5vmw/w3IemYsvKyeyMs\n+fDQt7lHYTZkLPX+UbCHK/WK16Fw5Yp/IFhWTMuLT/bGLOAVSAt4f/b2eh4se1678d7rN+8DhTl7\nX0OqWar3Oz+Y7e257Umg3PFp7YdAw06Q2LxsrPVJ3jFxFth7mzXrH9rvpILobCMRCbvV23K476PF\nTFnqXfOmTYNa3H1uBwa09ecvvrAqLdm73ETHle2Oy9sBmau853u+fAuyYfmnB55t+un1ImjcBfqM\n9tbfvQn+OxLik8r+lQ/w9PHeDFdxgfcF9bMvrnKPQBASGsMVE8vWf/V8rxBe+g7UqOuNffY3b0Zm\nry+/QNmusv19OZ75b4ir7a0/cwxs+xF6XQMNjvPGln8Oa7/Ze3u/+oUb8IrGnhmBjd979xRr3NWb\nKQDYtQHSZ+8n5698iSc2hdpNvPXzd3nbiKsNic28sZJiyNr4y5l+VgrKfQ45bDpVWkR8N2XpFu79\ncDFrMnMBOL1DQ+46uwMtkg7+DCcRqT78uKu0iMheTjmuIZ/efCJ/Hnwc8TFBJi/ewqBHvuThycvI\nKyw58AZERFB5EZGjLDYqyO8HHsPU2wbym+5NKSwu5fEpKzj1oWl8tGAjVWU2WETCR+VFRHzRsHYc\nj1zUjXeu7UfHJrXZuCuf6978nhHPz2Dp5t1+xxORSkzlRUR8lZpSj4nXncA/f9OZuvHRzFi1nbMe\n+5q7P1jIztzCA29ARKodlRcR8V0wYFzcpwXTbjuZK49Pwcx4ZfpaTn5wGm/MXEtJqXYliUgZlRcR\nqTQS46O5Z0hHJt1wAn1b12NHbhH/995Chjz5DWlrtvsdT0QqCZUXEal0jmtUm7dG9uWpi3vQJDGO\nRRt3c+Gz07lp3Pds3pXvdzwR8ZnKi4hUSmbG2V0a88WtA7nh1LbERAV4f95GTnloGk9PW0FBsU6t\nFqmuVF5EpFKrERPkltPa8cUtJzG4YyNyC0v4zyfLOOORr5iydIvf8UTEByovIhIRmteL59nLevLa\n1b1p06AWazJz+d3LaVz10ixWZWT7HU9EjiKVFxGJKAPa1ud/Nw7gr+d0ICE2iqnLMjjj0a/41/+W\nkF1Q7Hc8ETkKVF5EJOJEBwNcfUIrptw2kGGpzSgudTz35SpOeXAa732frqv0ilRxKi8iErHqJ8Ty\nnwu78v4f+tOteR22ZhVw8/j5DH3mO35I3+V3PBEJE5UXEYl4XZvX4b+/P54Hf9uV5FqxzF23kyFP\nfcOd/11AZnaB3/FEpIKpvIhIlRAIGBf2bMaU205i5IBWBM14a9Z6Tn5wGi9/u5riklK/I4pIBVF5\nEZEqpXZcNP93dgc+uelEBrRNZnd+Mfd8uJizH/+G71Zs8zueiFQAlRcRqZLaNKjFq7/rzfOXp9Ki\nXjzLtmRx8diZ/OGNOaTvyPU7nogcAZUXEamyzIzTOjRk8s0nctvp7agRHeTjHzYz6OEveezz5eQX\n6Sq9IpFI5UVEqry46CDXndKWL249iXO7NiG/qJRHPv+RUx/6kk8WbtKp1SIRRuVFRKqNJnVq8MSI\n7owf1ZfjGiWwYWce174+l0tfmMmPW7L8jiciB0nlRUSqnT6tk/jo+hO477yOJNaI5tsVmZz52Nfc\n++FiduUV+R1PRA5A5UVEqqWoYIDL+qUw7baBXNq3Bc45Xvx2Nac8OI3xs9dRWqpdSSKVlcqLiFRr\ndWvG8I/zO/Ph9SfQO6UemTmF/PndHzj/6W+Zu26H3/FEZD/CWl7MbLCZLTOzFWZ2x68sN9TMnJml\nlhu7M7TeMjM7I5w5RUQ6Nklk/Oi+PDa8G41qx7EgfRcXPP0dt06Yz9asfL/jiUg5YSsvZhYEngLO\nBDoAI8ysw36WSwBuBGaWG+sADAc6AoOBp0PbExEJGzPjvG5N+eLWk/jjyccQEwzw7tx0TnnwS8Z8\ntZLCYl2lV6QyCOfMS29ghXNulXOuEBgHnLef5e4D/g2U/6fNecA451yBc241sCK0PRGRsKsZG8Xt\nZxzHZ7ecyKD2DcguKOafHy9l8KNfMW3ZVr/jiVR74SwvTYH15Z6nh8Z+YmY9gObOuUmHum5o/VFm\nlmZmaRkZGRWTWkQkpGVSTcZe0YuXrupF6+SarNqWw5UvzeaaV9JYm5njdzyRasu3A3bNLAA8DNx6\nuNtwzo1xzqU651Lr169fceFERMo5+dgGfHLTifzlrOOoGRPk8yVbOO3hr3jg06XkFBT7HU+k2gln\nedkANC/3vFlobI8EoBMwzczWAH2BiaGDdg+0rojIURUTFWDUiccw9baBXNCjKYUlpTw1dSWnPvQl\nH8zboKv0ihxF4Swvs4G2ZtbKzGLwDsCduOdF59wu51yycy7FOZcCzACGOOfSQssNN7NYM2sFtAVm\nhTGriMhBaVA7joeHdePd3x9P56aJbN6dz43j5nHRczNYvHG33/FEqoWwlRfnXDFwHfApsASY4Jxb\nZGb3mtmQA6y7CJgALAY+Af7onNMd1ESk0ujZsi4f/LE//x7amaSaMcxas51znviau97/gR05hX7H\nE6nSrKpMdaamprq0tDS/Y4hINbQrr4hHP/+RV6evpaTUkVgjmttOb8fFfVoSDJjf8UQilpnNcc6l\n7juuK+yKiByhxBrR3H1uR/534wCOPyaJXXlF/PWDRZzzxDfMXJXpdzyRKkflRUSkgrRrmMAb1/Th\n2Ut70LRODZZs2s1FY2Zw/Vvfs3Fnnt/xRKoMlRcRkQpkZgzu1JjPbzmJmwa1JTYqwIfzN3LqQ1/y\n5JTl5Bfp8D2RI6XyIiISBjVigtw0qB1f3HoSZ3VuRF5RCQ9O/pHTH/mKzxZv0anVIkdA5UVEJIya\n1Y3n6Ut68uY1fWjXsBbrtucy8tU0rnhpNiu2ZvsdTyQiqbyIiBwFx7dJZtINA7j73A4kxEXx1Y8Z\nDH70K/7fpMVk5Rf5HU8koqi8iIgcJdHBAFf1b8W02wYyondzSpzj+a9Xc/KDX/LOnHRKS7UrSeRg\nqLyIiBxlSbVi+dcFXZj4xxPo0aIO27ILuO3t+VzwzHfMX7/T73gilZ7Ki4iITzo3S+Sda4/n4WFd\nqZ8Qy7z1Ozn/6W/58zsL2JZd4Hc8kUpL5UVExEeBgHFBj2ZMvW0go09qTVTAGJ+2npMfmMYL36ym\nqKTU74gilY7Ki4hIJVArNoo7z2zPpzedyMBj65NVUMx9Hy3mzMe+5pvl2/yOJ1KpqLyIiFQirevX\n4qUre/HCFam0TIpnxdZsLn1hJte+Nof123P9jidSKai8iIhUMmbGqe0bMvnmE/nT4GOJjwnyyaLN\nDHr4Sx7+7EfyCnWVXqneVF5ERCqp2KggfxjYhim3DuS8bk0oKC7l8S+Wc+pD05i0YJOu0ivVlsqL\niEgl1ygxjseGd+fta/vRoXFtNu7K549vzuXi52eydPNuv+OJHHUqLyIiEaJXSj0+vP4E/t9vOlE3\nPprpqzI5+/FvuGfiInbl6iq9Un2ovIiIRJBgwLikT0um3jaQy/u1xDnHy9+tYeCDU3lz5jpKdJVe\nqQZUXkREIlCd+BjuPa8Tk24YQJ9W9diRW8Rf3vuB8576hrQ12/2OJxJWKi8iIhGsfePajBvVlycv\n7k7jxDgWbtjNhc9O5+bx89iyO9/veCJhofIiIhLhzIxzujThi1tP4vpT2hATFeC97zdw8oPTeGba\nSvKLdGq1VC1WVU61S01NdWlpaX7HEBHx3brMXP4xaTGTF28BoE58NL/p3pQRvVvQrmGCz+lEDp6Z\nzXHOpf5sXOVFRKRq+urHDB74dBk/bNj101iPFnUY0bsF53RpQo2YoI/pRA5M5UVEpJpauGEXb81a\nxwfzNpJdUAxAQmwU53VvwojeLejYJNHnhCL7p/IiIlLN5RQUM2nBJt6avY7v1+38abxLs0SG92rB\nkG5NqBUb5WNCkb2pvIiIyE+Wbt7NuFnr+e/cdHbne7Mx8TFBzu3ShBF9WtC1WSJm5nNKqe5UXkRE\n5Gfyi0r438JNvDVzPbPKXR/muEYJjOjdgvO7NyWxRrSPCaU6U3kREZFftWJrNuNnr+PduRvYnlMI\nQFx0gLM6N2ZE7xaktqyr2Rg5qlReRETkoBQUlzB50RbGzV7Htysyfxpv06AWw3s1Z2iPZtStGeNj\nQqkuVF5EROSQrc3MYfzs9UxIS2dbdgEAMcEAZ3RqxIhezenbOolAQLMxEh4qLyIictiKSkr5YslW\nxs1ex5c/ZrDnqyMlKZ6LerXgwp7NqJ8Q629IqXJUXkREpEKk78hlQlo6b6etZ9Mu7/5JUQFjUPuG\njOjTggFtkjUbIxVC5UVERCpUSanjyx+38tas9UxZupWSUu/7pGmdGlzUqznDUpvTKDHO55QSyVRe\nREQkbLbszufttPWMm72e9B15wP9v786D46zvO46/vzpty/IpH7K0i7Exh43xqQVyh9OBYCdgQKKl\npNMMTQOTZjKdSZOSJmEm0/yVlKSZkkzKDEnpyoAhMQ4pSUjCkcZeCdvgC4xt4l3Jl3xJvnV9+8c+\nloViGSl499mVPq+ZnXn29/x296vfPPbz2d8+uz8oMLju8snU1kT52GWTKCrUWsAyOKGEFzNbAjwC\nFAI/dvdv99n/OeABoAs4Btzv7lvMbDqwFXgr6LrG3T93vtdSeBERCV93t/OHHQeIJ5L8avM+OoPZ\nmKljRnDn4mruWhwhMmFUyFVKvsh6eDGzQmAbcCPQBDQAde6+pVefMe7eFmwvBT7v7kuC8LLa3a8c\n6OspvIiI5JYDx06z8rUm6htSvHPgOABm8OFZk6iriXDD7CkUazZGzqO/8JLJRSxiwHZ33xkUUA8s\nA3rCy5ngEigDhsZnWCIiQsXoUv7+ozO5/yMzWLPzEPUNSX65aS8vb2vh5W0tVIwuZfmiamprIkyv\nKAu7XMkjmQwvVUCq1/0m4Oq+nczsAeBLQAlwXa9dF5vZeqANeMjdX8lgrSIikiFmxrUzJ3LtzIl8\n43g7z65vJp5I8vb+Yzz60g4efWkH186YSN3VUW6eM4XSosKwS5Ycl8mPjZYDS9z9s8H9e4Gr3f3B\nfvrfA9zs7veZWSkw2t0Pmtki4GfAnD4zNZjZ/cD9ANFodNGuXbsy8reIiMiF5e6sSx4mnkix+o3d\nnOroBmD8qGJuX1hNXSzCJZPLQ65SwhbGNS/XAt9w95uD+18BcPd/66d/AXDY3ceeY9/vgX9y934v\natE1LyIi+an1ZAerNjQTT6TYsufse9Sa6eOprYlyy9xKRpZoNmY4CiO8FJG+YPd6oJn0Bbv3uPvm\nXn1mufvbwfZtwNfdfbGZTQIOuXuXmc0AXgHmuvuhP3uhgMKLiEh+c3c2NrcSTyRZtWE3x9u7ACgf\nUcSnF1RRWxNl9rQxqlvM+AAADl1JREFUIVcp2RTWV6VvAf6d9FelH3P3b5nZw0Cju68ys0eAG4AO\n4DDwoLtvNrM7gIeD9m7Soea5872WwouIyNBx7HQnq1/fTbwhxeupIz3t8yLjqKuJcNu8aZSVZvKy\nTckF+pE6ERHJS1t2t1HfkOTZ9c0cPdUJQFlJIUvnV1EXizC3aixmWo5gKFJ4ERGRvHayvYvnN+4h\nnkjSuOtwT/vsyjHUXR1l2fxpjBlRHGKFcqEpvIiIyJDx9r6j1DekWLmuiSMnOgAYWVzIrVdVUheL\nsDA6XrMxQ4DCi4iIDDmnOrp4YfNe6hMp/rjzYE/7pVNGU1sT5faFVYwbVRJihfJ+KLyIiMiQ9s6B\n46xoSPH0aykOHGsHoKSogE9cOZW6WJSrL56g2Zg8o/AiIiLDQntnNy9u3Ue8IcUrb7dw5jQ3o6KM\nu2si3LGomorRpeEWKQOi8CIiIsNO6tAJnmxM8WRjin1tpwEoLjRumj2V2liED86soKBAszG5SuFF\nRESGrc6ubn7/VgvxRJLfvbWf7uDUF5kwkrsXR7hzcYQpY0aEW6T8GYUXERERYE/rSZ5qbGJFQ4rm\nIycBKCwwrrt8MnWxCB+9dDKFmo3JCQovIiIivXR1O69uP0B8bZLfbN1HZzAdUzl2BHctjnBXTYSq\ncSNDrnJ4U3gRERHpx/6jp1j5WjP1DUl2HTwBgBl89NJJ1MWiXHf5ZIoLC0KucvhReBEREXkP3d3O\nmp0HiTekeGHTXtq7ugGYVF7KnYuqqa2JEp04KuQqhw+FFxERkUE4dLydZ9Y1EU8k2dFyvKf9Q5dU\nUBuLcNPsqZQUaTYmkxReRERE/gLuTuOuw8TXJvnFxj2c7kzPxkwoK+GOhVXUxqLMnDQ65CqHJoUX\nERGR96n1RAc/29BMPJHkzb1He9pjF0+gLhbhE1dWMqK4MMQKhxaFFxERkQvE3Xm9qZX42iTPvbGb\nE+1dAIwdWcynF1RRF4ty2dTykKvMfwovIiIiGXD0VAfPvb6HeCLJxubWnvYF0XHUxaJ88qpKRpUU\nhVhh/lJ4ERERybBNza3UNyT5+frdHD3dCUB5aRFL50+jLhblyqqxIVeYXxReREREsuREeye/eCM9\nG7MueaSnfW7VWGpjEZbOm0b5iOIQK8wPCi8iIiIheGvvUeobkjyzrpnWkx0AjCwu5LZ5ldTFosyP\njMNMyxGci8KLiIhIiE51dPG/m/YSTyRZ+86hnvbLp5ZTWxPh0wuqGTtKszG9KbyIiIjkiB0tx1jR\nkOLp15o4dLwdgNKiAm6dW0ltLErN9PGajUHhRUREJOe0d3bz6y37qG9I8srbB3raZ04qoy4W5faF\n1UwoKwmxwnApvIiIiOSw5METrGhM8lRjE/uPngagpLCAm+ZMoS4W5doZEykoGF6zMQovIiIieaCz\nq5vfvrmfeCLJS9ta6A5O0xdNHMXdNRGWL6pmcvmIcIvMEoUXERGRPLP7yEmebEzxZEOK3a2nACgq\nMK6/YjJ1sSgfnjWJwiE8G6PwIiIikqe6up2Xt7UQTyR58c39dAXTMVXjRnLX4gh31VRTOXZkyFVe\neAovIiIiQ8D+tlM89VoT9Q1JUodOAlBg8PHLJlMbi/LxyyZRVFgQcpUXhsKLiIjIENLd7fzfjoPE\nG5L8avNeOrrS5/MpY0q5c1GEu2siRCaMCrnK90fhRUREZIg6eOw0K9c1UZ9IsfPAcQDM4EOXVFAX\ni3LDFVMoKcq/2RiFFxERkSHO3Um8c4h4Isnzm/bS3tkNQMXoEu5YVE1tTZSLK8pCrnLgFF5ERESG\nkSMn2nl2fTP1iRRv7Tva037NjAnUxaLcPGcqI4oLQ6zwvSm8iIiIDEPuzvrUEeJrk6x+Yw8nO7oA\nGDeqmNsXVFMXizBrSnnIVZ6bwouIiMgw13aqg1UbdhNPJNm8u62nffFF46mNRbl1biUjS3JnNkbh\nRURERHpsbGol3pBk1YbdHDvdCUD5iCI+Nb+K2liEOdPGhlxhSOHFzJYAjwCFwI/d/dt99n8OeADo\nAo4B97v7lmDfV4C/C/Z9wd1fON9rKbyIiIgM3vHTnax+YzfxRIoNqSM97fOqx1Ibi3LbvGmMLi0K\npbashxczKwS2ATcCTUADUHcmnAR9xrh7W7C9FPi8uy8xs9lAHIgB04DfAJe6e1d/r6fwIiIi8v5s\n3dNGfSLJs+ubaTuVno0pKylk6fxp1NZEuap6LGbZW46gv/CSyS99x4Dt7r7T3duBemBZ7w5ngkug\nDDiTpJYB9e5+2t3fAbYHzyciIiIZckXlGL657EoS/3ID37lrHrHpEzje3kU8kWLZD/7ALd97lZ/+\n8U+0neoItc5MzgNVAale95uAq/t2MrMHgC8BJcB1vR67ps9jqzJTpoiIiPQ2oriQ2xdWc/vCarbv\nP0p9IsXKdU1s3dPG136+mW89v5Vb506jLhZh0UXjszobA5mdeRkQd/+Bu88Evgw8NJjHmtn9ZtZo\nZo0tLS2ZKVBERGQYu2RyOQ99cjZrvno9369bwAdmTuRURzcr1zWx/NE/8olHXun5MbxsyeTMSzMQ\n6XW/OmjrTz3wn4N5rLv/CPgRpK95eT/FioiISP9Kiwq5bd40bps3jT8dOM6KxhRPNTZRPX5U1pce\nyGR4aQBmmdnFpINHLXBP7w5mNsvd3w7u3gqc2V4F/I+ZfYf0BbuzgEQGaxUREZEBml5RxpeXXM6X\nbryUwyfas/76GQsv7t5pZg8CL5D+qvRj7r7ZzB4GGt19FfCgmd0AdACHgfuCx242syeBLUAn8MD5\nvmkkIiIi2VdcWMDk8hFZf139SJ2IiIjkpDC+Ki0iIiJywSm8iIiISF5ReBEREZG8ovAiIiIieUXh\nRURERPKKwouIiIjkFYUXERERySsKLyIiIpJXFF5EREQkryi8iIiISF5ReBEREZG8MmTWNjKzFmBX\nhp6+AjiQoeceijReg6PxGjyN2eBovAZH4zU4mRyvi9x9Ut/GIRNeMsnMGs+1MJScm8ZrcDReg6cx\nGxyN1+BovAYnjPHSx0YiIiKSVxReREREJK8ovAzMj8IuIM9ovAZH4zV4GrPB0XgNjsZrcLI+Xrrm\nRURERPKKZl5EREQkryi89GJmS8zsLTPbbmb/fI79pWa2Iti/1symZ7/K3DGA8fqMmbWY2Ybg9tkw\n6swVZvaYme03s0397Dcz+14wnm+Y2cJs15hLBjBeHzOz1l7H179mu8ZcYWYRM/udmW0xs81m9o/n\n6KPjKzDA8dLx1YuZjTCzhJm9HozZN8/RJ3vnSHfXLf3RWSGwA5gBlACvA7P79Pk88GiwXQusCLvu\nHB+vzwD/EXatuXIDPgIsBDb1s/8W4JeAAdcAa8OuOcfH62PA6rDrzIUbUAksDLbLgW3n+Peo42tw\n46Xj693jYcDoYLsYWAtc06dP1s6Rmnk5KwZsd/ed7t4O1APL+vRZBjwebD8NXG9mlsUac8lAxkt6\ncfeXgUPn6bIM+ImnrQHGmVlldqrLPQMYLwm4+x53XxdsHwW2AlV9uun4CgxwvKSX4Lg5FtwtDm59\nL5rN2jlS4eWsKiDV634Tf34w9/Rx906gFZiYlepyz0DGC+COYIr6aTOLZKe0vDXQMZWzrg2msX9p\nZnPCLiYXBFP1C0i/M+5Nx9c5nGe8QMfXu5hZoZltAPYDv3b3fo+xTJ8jFV4kk54Dprv7VcCvOZvI\nRS6EdaR/Onwe8H3gZyHXEzozGw2sBL7o7m1h15Pr3mO8dHz14e5d7j4fqAZiZnZlWLUovJzVDPSe\nGagO2s7Zx8yKgLHAwaxUl3vec7zc/aC7nw7u/hhYlKXa8tVAjkEJuHvbmWlsd38eKDazipDLCo2Z\nFZM+ET/h7s+co4uOr17ea7x0fPXP3Y8AvwOW9NmVtXOkwstZDcAsM7vYzEpIX2y0qk+fVcB9wfZy\n4LceXJk0DL3nePX5PH0p6c+VpX+rgL8JvhVyDdDq7nvCLipXmdnUM5+nm1mM9P9nw/LNRDAO/wVs\ndffv9NNNx1dgIOOl4+vdzGySmY0LtkcCNwJv9umWtXNkUSaeNB+5e6eZPQi8QPqbNI+5+2Yzexho\ndPdVpA/2n5rZdtIXEtaGV3G4BjheXzCzpUAn6fH6TGgF5wAzi5P+BkOFmTUBXyd90Rvu/ijwPOlv\nhGwHTgB/G06luWEA47Uc+Acz6wROArXD+M3EB4F7gY3BNQkAXwWioOPrHAYyXjq+3q0SeNzMCkkH\nuSfdfXVY50j9wq6IiIjkFX1sJCIiInlF4UVERETyisKLiIiI5BWFFxEREckrCi8iIiKSVxReRGTI\nCVYEXh12HSKSGQovIiIiklcUXkQkNGb212aWMLMNZvbDYOG3Y2b2XTPbbGYvmtmkoO98M1sTLPT5\nrJmND9ovMbPfBAvorTOzmcHTjw4WBH3TzJ4YxivAiww5Ci8iEgozuwK4G/hgsNhbF/BXQBnpX+yc\nA7xE+pd1AX4CfDlY6HNjr/YngB8EC+h9ADjzk/cLgC8Cs4EZpH9VVUSGAC0PICJhuZ70Yp0NwaTI\nSGA/0A2sCPr8N/CMmY0Fxrn7S0H748BTZlYOVLn7swDufgogeL6EuzcF9zcA04FXM/9niUimKbyI\nSFgMeNzdv/KuRrOv9en3l65hcrrXdhf6/05kyNDHRiISlheB5WY2GcDMJpjZRaT/X1oe9LkHeNXd\nW4HDZvbhoP1e4CV3Pwo0mdmngucoNbNRWf0rRCTr9E5ERELh7lvM7CHgV2ZWAHQADwDHgViwbz/p\n62IA7gMeDcLJTs6uinwv8MNgddsO4M4s/hkiEgKtKi0iOcXMjrn76LDrEJHcpY+NREREJK9o5kVE\nRETyimZeREREJK8ovIiIiEheUXgRERGRvKLwIiIiInlF4UVERETyisKLiIiI5JX/B63/aP/9Nc2a\nAAAAAElFTkSuQmCC\n", 480 | "text/plain": [ 481 | "
" 482 | ] 483 | }, 484 | "metadata": { 485 | "tags": [] 486 | } 487 | } 488 | ] 489 | }, 490 | { 491 | "cell_type": "code", 492 | "metadata": { 493 | "id": "hAcgG2K70t5v", 494 | "colab_type": "code", 495 | "outputId": "b3a7b291-2822-4a21-e889-c4ff3a42ad63", 496 | "colab": { 497 | "base_uri": "https://localhost:8080/", 498 | "height": 404 499 | } 500 | }, 501 | "source": [ 502 | "plt.figure(figsize=(9,6))\n", 503 | "plt.plot(history.history['acc'], '-', linewidth=2)\n", 504 | "plt.plot(history.history['val_acc'], '-.', linewidth=2)\n", 505 | "plt.title('model accuracy')\n", 506 | "plt.ylabel('accuracy')\n", 507 | "plt.xlabel('epoch')\n", 508 | "plt.legend(['train', 'test'], loc='upper left')\n", 509 | "plt.show()" 510 | ], 511 | "execution_count": 0, 512 | "outputs": [ 513 | { 514 | "output_type": "display_data", 515 | "data": { 516 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAi8AAAGDCAYAAADj4vBMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nOzdeXxU5b3H8c+TfSUhC2sICQRklS2y\nBVqtWnHfWusCCvaK1mptr120tdba21tvF7u6tlUQBPeFVq1L1SqrhFVAlgAhCWsWkpCEbDPP/eNM\nkglrgEwmM/m+X695MfOcc2Z+QWS+nOd3nmOstYiIiIgEihB/FyAiIiJyKhReREREJKAovIiIiEhA\nUXgRERGRgKLwIiIiIgFF4UVEREQCisKLiLQ7Y8wcY8z/tHHffGPMBb6uSUSCh8KLiIiIBBSFFxGR\n4zDGhPm7BhE5msKLSBflma75gTFmvTGm2hjzd2NMT2PMO8aYQ8aYD4wx3b32v8IYs9EYU26M+dgY\nM9Rr2xhjzGrPcS8CUUd81mXGmLWeY5caY85uY42XGmPWGGMqjTGFxpiHjtg+xfN+5Z7tMz3j0caY\n3xljdhljKowxiz1j5xpjio7x+3CB5/lDxphXjDHzjTGVwExjzHhjzDLPZ+w1xvzFGBPhdfxwY8z7\nxpgyY8x+Y8yPjTG9jDE1xphkr/3GGmOKjTHhbfnZReT4FF5EurZrgQuBwcDlwDvAj4FUnL8fvgNg\njBkMLAS+69n2NvAPY0yE54v8DWAekAS87HlfPMeOAZ4BbgeSgaeARcaYyDbUVw3cDCQClwLfMsZc\n5Xnf/p56/+ypaTSw1nPcb4FxwGRPTT8E3G38PbkSeMXzmc8DLuB7QAowCTgfuNNTQzzwAfAvoA+Q\nBfzbWrsP+Bi4zut9ZwAvWGsb2liHiByHwotI1/Zna+1+a+1u4FNghbV2jbW2FngdGOPZ7xvAW9ba\n9z1fvr8FonHCwUQgHPiDtbbBWvsKsNLrM2YDT1lrV1hrXdbauUCd57gTstZ+bK393FrrttauxwlQ\nX/ZsvhH4wFq70PO5pdbatcaYEOBW4B5r7W7PZy611ta18fdkmbX2Dc9nHrbWrrLWLrfWNlpr83HC\nV1MNlwH7rLW/s9bWWmsPWWtXeLbNBaYDGGNCgRtwAp6InCGFF5Gubb/X88PHeB3ned4H2NW0wVrr\nBgqBvp5tu23ru7zu8nreH7jXM+1SbowpB/p5jjshY8wEY8xHnumWCuAOnDMgeN5j+zEOS8GZtjrW\ntrYoPKKGwcaYfxpj9nmmkv63DTUAvAkMM8Zk4pzdqrDWfnaaNYmIF4UXEWmLPTghBABjjMH54t4N\n7AX6esaapHs9LwR+aa1N9HrEWGsXtuFzFwCLgH7W2gTgSaDpcwqBgcc4pgSoPc62aiDG6+cIxZly\n8maPeP0EsBkYZK3thjOt5l3DgGMV7jl79RLO2ZcZ6KyLSLtReBGRtngJuNQYc76n4fRenKmfpcAy\noBH4jjEm3BhzDTDe69i/And4zqIYY0yspxE3vg2fGw+UWWtrjTHjcaaKmjwPXGCMuc4YE2aMSTbG\njPacFXoGeNQY08cYE2qMmeTpsdkKRHk+Pxx4ADhZ7008UAlUGWOGAN/y2vZPoLcx5rvGmEhjTLwx\nZoLX9ueAmcAVKLyItBuFFxE5KWvtFpwzCH/GObNxOXC5tbbeWlsPXIPzJV2G0x/zmtexucBtwF+A\ng0CeZ9+2uBN42BhzCHgQJ0Q1vW8BcAlOkCrDadYd5dn8feBznN6bMuD/gBBrbYXnPf+Gc9aoGmh1\n9dExfB8nNB3CCWIvetVwCGdK6HJgH7ANOM9r+xKcRuHV1lrvqTQROQOm9TS1iIi0J2PMh8ACa+3f\n/F2LSLBQeBER8RFjzDnA+zg9O4f8XY9IsNC0kYiIDxhj5uKsAfNdBReR9qUzLyIiIhJQdOZFRERE\nAorCi4iIiASUoLljakpKis3IyPB3GSIiItJOVq1aVWKtPXIhyeAJLxkZGeTm5vq7DBEREWknxphj\nro+kaSMREREJKAovIiIiElAUXkRERCSgBE3Py7E0NDRQVFREbW2tv0vxuaioKNLS0ggPD/d3KSIi\nIj4V1OGlqKiI+Ph4MjIyMMac/IAAZa2ltLSUoqIiMjMz/V2OiIiITwX1tFFtbS3JyclBHVwAjDEk\nJyd3iTNMIiIiQR1egKAPLk26ys8pIiIS9OHF38rLy3n88cdP+bhLLrmE8vJyH1QkIiIS2BRefOx4\n4aWxsfGEx7399tskJib6qiwREZGAFdQNu53Bfffdx/bt2xk9ejTh4eFERUXRvXt3Nm/ezNatW7nq\nqqsoLCyktraWe+65h9mzZwMtKwZXVVVx8cUXM2XKFJYuXUrfvn158803iY6O9vNPJiIi4h9dJrxk\n3PeWT943/5FLT7j9kUceYcOGDaxdu5aPP/6YSy+9lA0bNjRfFfTMM8+QlJTE4cOHOeecc7j22mtJ\nTk5u9R7btm1j4cKF/PWvf+W6667j1VdfZfr06T75eURERDq7LhNeOovx48e3upz5T3/6E6+//joA\nhYWFbNu27ajwkpmZyejRowEYN24c+fn5HVaviIjIkRpdbtbvrmDxthJiIkL5r6kDOvTzu0x4OdkZ\nko4SGxvb/Pzjjz/mgw8+YNmyZcTExHDuuece83LnyMjI5uehoaEcPny4Q2oVEREBZz2x/NIaFm8r\n5tNtJSzbUcqhWqd3s19StMJLsImPj+fQoUPH3FZRUUH37t2JiYlh8+bNLF++vIOrExERObay6nqW\n5JWweFsJi/NK2F3e+h/OGckxTBmUwpSsFNxuS0hIxy3ZofDiY8nJyeTk5DBixAiio6Pp2bNn87Zp\n06bx5JNPMnToUM466ywmTpzox0pFRKQrq21wkZt/kE/zilmSV8LGPZVY27K9e0w4k7OcsDIlK4V+\nSTF+q9VY78oCWHZ2ts3NzW019sUXXzB06FA/VdTxutrPKyIip8/ttmzaW+mcXckr4bOdZdQ1upu3\nR4SFcE5Gd3KyUpialcrwPt069OwKgDFmlbU2+8hxnXkRERHpIvaUH2bxthI+zSthaV4JpdX1rbYP\n7d2NqZ6poHMykoiOCPVTpSem8CIiIhKkDtU2sGx7KYs9vSs7Sqpbbe+dEOVMAw1KYfLAFFLjI4/z\nTp2LwouIiEiQaHC5WVtY3txku7awHJe7pT0kLjKMiQOSmToohZysFAamxgbkvfEUXkRERAKUtZbt\nxdUs3lbM4rwSlu8oo6qu5fYzoSGG7P6evpVBKYzql0h4aODfGUjhRUREJICUVNWxJK+ET7eVsCSv\nhL0VrdcHG5Aay9SsFKYMSmXigCTio8L9VKnvKLyIiIh0YofrXXyWX9a8QNzmfa3XDkuOjSDH07cy\nJSuFPonBf+87hRcfKy8vZ8GCBdx5552nfOwf/vAHZs+eTUyM/66lFxGRjuVyWzbuqWhuss3NP0i9\nq+US5siwEMZnJjX3rQzt1fGXMPubwouPlZeX8/jjj592eJk+fbrCi4hIkCssq2kOK0u2l1Be09C8\nzRgY2TehuW9lXP/uRIV3zkuYO4rCi4/dd999bN++ndGjR3PhhRfSo0cPXnrpJerq6rj66qv5+c9/\nTnV1Nddddx1FRUW4XC5++tOfsn//fvbs2cN5551HSkoKH330kb9/FBERaScVhxtYtt3pW1mcV8Ku\n0ppW2/smRjvrrXguYU6KjfBTpZ1T1wovDyWc2v69R8Htnxx9/EMVbX6LRx55hA0bNrB27Vree+89\nXnnlFT777DOstVxxxRV88sknFBcX06dPH9566y3AuedRQkICjz76KB999BEpKSmnVreIiHQq9Y1u\nVhccbL6EeX1ROV5XMBMfFcbkgclMGZTK1KwU+ifHBOQlzB2la4UXP3vvvfd47733GDNmDABVVVVs\n27aNqVOncu+99/KjH/2Iyy67jKlTp/q5UhERORPWWrbur/JMBRWzYmcZNfWu5u3hoYbs9O6eq4JS\nGNk3gbAguIS5o3St8HIKZ0x8cby1lvvvv5/bb7/9qG2rV6/m7bff5oEHHuD888/nwQcfPKPPEhGR\njnWgsra5b2VxXgkHDtW12j64Z1xz38qEzGRiI7vWV3B70u+cj8XHx3PokHNZ20UXXcRPf/pTbrrp\nJuLi4ti9ezfh4eE0NjaSlJTE9OnTSUxM5G9/+1urYzVtJCLS+dTUN7JiR1nzeitb9re+hDk1PrL5\nDsxTBqXQs1uUnyoNPgovPpacnExOTg4jRozg4osv5sYbb2TSpEkAxMXFMX/+fPLy8vjBD35ASEgI\n4eHhPPHEEwDMnj2badOm0adPHzXsioj4mcttWV/UsvT+6oKDNLhaGleiw0OZMCCJKVkpTB2UyuCe\ncepb8RFjrT35XgEgOzvb5ubmthr74osvGDp0qJ8q6nhd7ecVEfElay27SlsuYV66vYTK2pal90MM\njExLZGqWs97K2P6JRIZ17UuY25sxZpW1NvvIcZ15ERER8ThYXc/S7aUsznNWsy06eLjV9v7JMc1T\nQZMHppAQE3xL7wcChRcREemy6hpdrMo/yKd5Tt/K57sr8J6QSIgOJycrmSlZqUwdlEK/JC0a2hko\nvIiISJdhreWLvYecGxvmlfDZzlJqG1qW3o8IDWFc/+5MGeRcFTS8TwKhXWzp/UAQ9OHFWtslGqaC\npXdJRKS97a043NxkuySvhJKq+lbbh/SK96xmm8r4jCSiI9S30tkFdXiJioqitLSU5OTkoA4w1lpK\nS0uJitJleCIiVXWNLN9eyuK8Ej7dVsz24upW23t1i2q+A3NOVgqp8ZF+qlROl0/DizFmGvBHIBT4\nm7X2kSO2pwNzgUTPPvdZa982xmQAXwBbPLsut9becaqfn5aWRlFREcXFxaf/QwSIqKgo0tLS/F2G\niEiHa3S5WVdU3rzeypqCchq91t6PjQhl4oDk5qmggam6hDnQ+Sy8GGNCgceAC4EiYKUxZpG1dpPX\nbg8AL1lrnzDGDAPeBjI827Zba0efSQ3h4eFkZmaeyVuIiEgnY61lR0m107eyrYTl20s5VNdyCXNo\niGFseqJzn6BBKYzul0i4lt4PKr488zIeyLPW7gAwxrwAXAl4hxcLdPM8TwD2+LAeEREJUKVVdSzZ\nXsribcUs3lbCnoraVtsHpMQ2TwVNHJhMtyhdwhzMfBle+gKFXq+LgAlH7PMQ8J4x5m4gFrjAa1um\nMWYNUAk8YK399MgPMMbMBmYDpKent1/lIiLiV7UNLlbml7F4m3N2ZdPeylbbk2IjnPsEZaWQMyiF\nvonRfqpU/MHfDbs3AHOstb8zxkwC5hljRgB7gXRrbakxZhzwhjFmuLW21Z9ea+3TwNPgrLDb0cWL\niEj7cLstm/ZWNvetfJZfRn2j1yXMYSGMz0hqPrsyrHc3QnQJc5fly/CyG+jn9TrNM+btm8A0AGvt\nMmNMFJBirT0A1HnGVxljtgODgVxERCQo7C4/zOJtzkq2S7eXUlbd+hLm4X26OU22WalkZ3QnKlyX\nMIvDl+FlJTDIGJOJE1quB248Yp8C4HxgjjFmKBAFFBtjUoEya63LGDMAGATs8GGtIiLiY5W1DSzb\nXtq85srOktaXMPdNjG6+A/Pkgckkx+kSZjk2n4UXa22jMeYu4F2cy6CfsdZuNMY8DORaaxcB9wJ/\nNcZ8D6d5d6a11hpjvgQ8bIxpANzAHdbaMl/VKiIi7a/B5WZNQbnTZJtXwrqiClxelzDHR4YxaWAy\nUwc5661kpsTqEmZpk6C+q7SIiHQcay15B6qa78K8fEcp1fWu5u1hIYax6d3J8ZxdGZWWQJguYZYT\n0F2lRUSk3RUfqmteb2VJXgn7KltfwpzVI44pWc7icBMGJBMXqa8dOXP6UyQiIm12uN7Fip0tfSub\n9x1qtT0lLpIpWclMGZTKlKwUeiXotiXS/hReRETkuFxuy4bdFc1TQat2HaTe1XIJc1R4CBMyW/pW\nhvSKV9+K+JzCi4iItFJYVsOn20pYnFfM0u2llNc0NG8zBkalJTT3rYzr353IMF3CLB1L4UVEpIur\nqGlg6fYSPvWcXSkoq2m1vV9SNFOynPsETR6YTGJMhJ8qFXEovIiIdDF1jS5W7ypncZ5zn6DPd1fg\ndQUzCdHhTB6Y3LxAXHpyjP+KFTkGhRcRkSBnrWXL/kPNTbYrdpRxuKHlEubwUMP4/t2Z6mmyHdE3\ngVAtvS+dmMKLiEgQstaSu+sgCz8r4NNtJRQfqmu1fUiveKZ4bmo4ITOJmAh9HUjg0J9WEZEgUtvg\n4h/r9jBnaT4b97Tcy7Znt0jnLsyeq4J6xOsSZglcCi8iIkFgf2Ut85fvYsGKAko9NzhMjo3gxgnp\nXDGqD1k94nQJswQNhRcRkQC2puAgzy7J5+3P99Lo6bod3qcbs3Iyuezs3roTswQlhRcRkQBT3+jm\nnQ17eWZJPusKywEIDTFcMrIXs3Iyye7fXWdZJKgpvIiIBIiSqjoWrChg/vJdHPA04CbGhHP9OenM\nmNSfvonRfq5QpGMovIiIdHIbdlfw7JJ8/rFuT/PS/Gf1jGdmTgZXje5LdISmhqRrUXgREemEGl1u\n3t24nzlLd7Iy/yDgLM1/4bCezJqcwaSByZoaki5L4UVEpBM5WF3PwpUFzFu2i70VtQDER4Xxjex+\n3DwpQ6vdiqDwIiLSKWzeV8mcJfm8vmY3dY3O1NCA1FhmTc7gmrFpxEbqr2uRJvq/QUTET1xuy7+/\n2M+zS/JZtqO0efzcs1KZlZPJ1KwUQrRMv8hRFF5ERDpYxeEGXs4tZO6yfArLDgMQGxHK18alccvk\nDAakxvm3QJFOTuFFRKSD5B2oYu7SfF5dXURNvXNjxPSkGG6ZnMHXs9PoFhXu5wpFAoPCi4iID7nd\nlv9sLeaZJTv5dFtJ8/iUrBRmTs7gvCE9dAdnkVOk8CIi4gNVdY28klvI3GW72FlSDUBUeAjXjE1j\n5uQMBveM93OFIoFL4UVEpB3ll1Qzd1k+L+cWUVXXCEDfxGhuntSfb5zTj8SYCP8WKBIEFF5ERM6Q\ntZbFeSXMWZLPh1sOYJ37IzIhM4lZORlcMLQnYaEh/i1SJIgovIiInKaa+kZeW72buUvz2XagCoCI\nsBCuGt2HWyZnMLxPgp8rFAlOCi8iIqeosKyGect38cJnBVTWOlNDPbtFcvOkDK4/px/JcZF+rlAk\nuCm8iIi0gbWWFTvLeHbJTt7ftB+3Z2pobHois3IymTaiF+GaGhLpEAovIiInUNvgYtHaPTy7NJ8v\n9lYCEB5quPLsPsycnMGofol+rlCk61F4ERE5hr0Vh5m/fBcLVhRwsKYBgJS4CG6a0J+bJqbTIz7K\nzxWKdF0KLyIiHtZaVhcc5Nkl+byzYR8uz9zQyL4JzMrJ4NKzexMZFurnKkVE4UVEury6Rhdvrd/L\nnKX5rC+qACA0xHDZ2b2ZlZPB2PTuGKNVcEU6C4UXEemyDhyq5fnlBTy/ooCSqjoAuseEc+OEdKZP\n7E/vhGg/Vygix6LwIiJdzrrCcuYszeef6/fQ4HKmhob0iufWnEyuGN2HqHBNDYl0ZgovItIlNLjc\nvLNhH3OW7GR1QTkAIQYuGt6TWTmZTMhM0tSQSIBQeBGRoFZaVccLKwuZt2wX+yprAegWFcb149OZ\nMbE//ZJi/FyhiJwqhRcRCUqb9lQyZ+lO3li7h/pGNwBZPeKYOTmDa8b2JSZCf/2JBCr93ysiQaPR\n5eaDL/bz7JJ8VuwsA8AYOH9ID2bmZDAlK0VTQyJBQOFFRAJeeU09L64s5Lllu9hdfhiAuMgwvp6d\nxi2TMshIifVzhSLSnhReRCRgbd1/iDlL83ltdRG1Dc7UUGZKLLdM6s+149KIjwr3c4Ui4gsKLyIS\nUNxuy4ebDzBnaT6L80qax6cOSuHWnEy+PDiVkBBNDYkEM4UXEQkIlbUNvJxbxHPL8tlVWgNAdHgo\n147ry8zJGWT1iPdvgSLSYRReRKRT21Fcxdyl+byyqojqehcAad2juWVSBted04+EaE0NiXQ1Ci8i\n0um43ZZP80p4dslOPt5S3Dw+aUAyM3MyuGBoT0I1NSTSZSm8iEinUV3XyGuri5izNJ/txdUARIaF\ncPWYvtwyOYOhvbv5uUIR6QwUXkTE7wpKa3huWT4v5hZyqLYRgN4JUcyY1J/rz0knKTbCvwWKSKei\n8CIifmGtZdn2Up5dms8HX+zHOvdH5JyM7sycnMlXh/ckPDTEv0UGKlcD1JRCfK+WsX2fQ0MthEVA\naKTXr55HaCSEhjur+ol0cgovItKhDte7eGPtbuYsyWfL/kMARISGcPmoPsycnMHItAQ/V9hJuV1Q\nXQLVB6BqP1QdaHlExMJXfuLs52qE/+kB1sJPSyDU89f8W/dC4YqTf05ToAmNgO9tgPBoZ/zNu6B4\nC1z2e+g1whlbuwB2fOzs2xSAThSOwiIg88sQk+QcX14ANWWQ0A9ik52xxjrn0VSDwpQcg0/DizFm\nGvBHIBT4m7X2kSO2pwNzgUTPPvdZa9/2bLsf+CbgAr5jrX3Xl7WKiG/tKT/Mc8t28cLKAsprGgBI\njY9kxsT+3DA+ndT4SD9X6AfWOl/edRWQNKBl/N8PQ8VuT1DxhJWaUrDuY79PYnpLeAkNg9hU571r\nK1pCQeoQcDdCYz246qCx1uu551d3o/Orq87zXl7Tdfs3wp7VznFNinJh/Yun9jN/84OW8LLkj7Dy\nb3Dxb2DCbGfs85fhzW+37O8dpo75ayRMuhOGXOrsv2upE6r658DoG5yxqmJY89zx36s5XHmNxSRD\nXA/neLcb3A0KU52Iz8KLMSYUeAy4ECgCVhpjFllrN3nt9gDwkrX2CWPMMOBtIMPz/HpgONAH+MAY\nM9ha6/JVvSLS/qy15O46yLNLdvLuxv243M7c0Kh+idyak8HFI3oTERZkU0PWQm2515mR/VBd3HK2\nJCkTvvQDZ9/K3fD74RDfG+7d3PIeaxfCoT1HvLGBmBTnCzWuB8T2aHnerW/rXf97M4Qc8ft6xZ9O\nXrvbBa5658yHqx5CQlu2XfWEE4ZSz2oZG30TpJ3TOgA1HXuscNRY1xKmwPm5e46EuNTWP2dEnHP8\nkWHqeEZ+reV58WZYMw9MSEt4qSxyAuGpGDMDrvyL83zfenj6y9BrJNyx2BmzFv4wEkLCjg5Txzv7\n1D0DJt/d8hnLHnOOy/5my3+vXUuhoeYEoe2I9+2iYcqXZ17GA3nW2h0AxpgXgCsB7/BigabLBxKA\npv9brwResNbWATuNMXme91vmw3pFpJ3UNrj45/q9PLtkJxv3VAIQFmK4YlQfZuVkMCa9u58rPA2N\ndVBe6PwLvMdQZ6y+Bt75oVc4KXbOlrjqj/8+/Sa0hJfYVIhKhOgk58uw6YvoKz8BTOugEpvi9KS0\nxZHBpa1CQiEkumWqyFuPIUePpY1zHqfrS993Ht7G3OQ8wDnj0SoQef9a1zLFlDKo5fj+U+DyP0Ky\n11hsKuR899jHHvN961sHQrcLQsKdwNA81ggVhaf28/YZ2xJeXI3w7o8BA+f8V8s+b/8Q9n/e9vcM\njYQH9rf82VlwPZTtgG/Mawmay5+AHf85+ZSed2AaennLGbIDm50/4ymDIb6nM1ZbCXWVEB7Tsl8H\n8mV46Qt4/5ctAiYcsc9DwHvGmLuBWOACr2OXH3HsEf+0AGPMbGA2QHp6ersULSKnb39lLfOX72LB\nigJKq50v8OTYCG6ckM70if3p2S3KzxUeoa7Ka2rmyLMkxZCRA5M8Uxh718PfL4A+Y2D2x85YWCSs\nff7o6ZzIbkefHYnrAXE9nX99NwmLhPt2HV3XmOk++GEDUEjI8cPU8aQOdh7eEtLgwp+ffh1p4+DB\nEpq7ysE54/LdDUeHnqZfG2uPHmv1JW9h4p1OMPI+e5I+wTkT1Ryu6o5xBsvza1NI9j6+bAeUbHHe\nt8metbD1nVP8mbNbT++tWwBXPtbyZ3PdC/DODyDlLLjrs1N773bg74bdG4A51trfGWMmAfOMMSPa\nerC19mngaYDs7Gx7kt1FxEfWFBzk2SX5vP35Xho9U0PDendjVk4Gl4/qQ1R46EneoZ3VVkDJNicc\n9BrpjFXshn/9qHWja0P1id8nzKvnI94TPLz/RR4S6vyFHpXQOqicypetBA7vkGAMJPY7/fcKDYdp\nvzp6/NLftf093O6jz/JNfwXqq6F7ZsvYpG/DsCtan2HyPgN1rLNPMV7TeylZTg9RfO+WsbBIiO/T\n0hfUwXwZXnYD3v9l0zxj3r4JTAOw1i4zxkQBKW08VkT8qL7RzTsb9vLMknzWFZYDEGLgkpG9mDk5\nk3MyumPaaz6+sd7rKhvPmZFWZ0wOwKjrYdwtzv47PoaXboazLoUbFjhjxsAX/2j9vmFRR58d8X6d\nnNWyb2I63LPu6NpG39g+P6PIqQoJgZAjzmYmHmMWovfZzuN0Tb3XeXgbd0vL/29+4MvwshIYZIzJ\nxAke1wNH/l9eAJwPzDHGDAWigGJgEbDAGPMoTsPuIKDjz0uJyFFKqupYsKKA+ct3ceCQ00iZEB3O\nDePTmTGpP30TT/GsQ+Ue50qW+F4tZ0n2rIX3HmiZyqktP/n79BnT8jyhn/M62esKntge8PU5nnDS\n0zk1H9mtyzY8igQyn4UXa22jMeYu4F2cy6CfsdZuNMY8DORaaxcB9wJ/NcZ8D6d5d6a11gIbjTEv\n4TT3NgLf1pVGIv61YXcFzy7J5x/r9lDvcno8BveMY1ZOJleN7kt0RKgzz950ZuTIq2y8e0rOvQ+G\nXem88aZFznTOObfBpb91xqwL8j9t+XAT6jRdeveOxKZ6QohnzPtS475jW/pSmoSGwfCrffb7IyId\nx6c9L541W94+YuxBr+ebgJzjHPtL4Je+rE9ETqzR5ebdjfuZs3QnK/MPkmWKuDhkFwkDxzDtvPOY\nNDAZs+09eOZGJ5xUFx9/LRJv5QUtz1MGwYDzWl8xknIWzHijJZxEJ53+FTQiEnT83bArIv7QtIBZ\n1YFj9pI0VOzj4IHdULWfuXXfZqUdSnxkGA/3Xs/kffPhrJ9CVorzXq56Z+n5JtFJR5whOcYVN97z\n8lnnOw9vkXEw8Dzf/z6ISIaRUNIAACAASURBVEBSeBEJFk2XcTb1cOxa6vSSDPwKJA90xlb+HRb/\n3gktJ1j4KxxouoZgZLcaLjt3ONeOTSM2rwY2Vbc+S5IxxZmiievpLKLmfYWOiIgPKLyIdHb1Ncfu\nH/FeOr5p7I7FzmWNALnPwucvwVVPtoQX625ZWCsiDuJ6YGN7cMCdwNqDEWyoiKSYREpsAv3SM/jq\n+JH8ZOQwQsI9i3MNv/rovpHo7s5DRKSDKLyIdAYb34CSrc4CUN36OGPv/8y570t9Vdvfp/pAS3gZ\ncK4z/ZLktd7DiGsh6wKI60GFK4KXcwuZuyyfwrLDAMRGhPK1cWncPzmDgalx7fGTiYi0O4UXkY5m\nrTOd03N4yxTPyr85V9f0G98SXkyIE1xCI1oaV2N7nPiKmwivwOG9zHqTmCTyqiKY+/ZOXl1dRE29\ncxFfelIMt0zO4OvZaXSLauMS9CIifqLwItJRDpc7d8xdNde5d8ltHzmX9AKMuMa5wV18n5b9c+5x\nHlEJZ7wWidtt+c/WYp5dms8nW4tbPiIrmVmTMzlvSA9CQ7TeiYgEBoUXEV+yFgqWw+q5ztRQozM9\nQ3R3KN/VEl6ybz362OjEM/74qrpGXsktZO6yXewscZbCjwoP4eoxaczKyWBwz/gz/gwRkY6m8CLi\nC9WlsG4hrH7OuUlak8wvwdhbnDu2hkUe//gzlF9Szdxl+bycW0RVXSMAfROjmTGpP9ef04/EGF0R\nJCKBS+FFpL243ZD/iTMttPmfLTdMi+3h6T+Z0XLVjw9Ya1mcV8KcJfl8uOVA85XT4zOTmDU5gwuH\n9SQsVAu9iUjgU3gRaQ/Wwl/Phb1NN+4zkHWhc+OywdOcO8j6SE19I6+t3s3cpflsO+BcmRQRFsKV\no/pwy+QMRvRN8Nlni4j4g8KLyOlwuyDv384CbRExTkNt32xnumjMdOeR2O/k73MGig7WMG/ZLhZ+\nVkBlrTM11LNbJDMm9ueG8ekkx/luWkpExJ8UXkROx0s3O1NDVz0Boz03S7/gIbjkNxAS6rOPtday\nYmcZc5bk896mfbg9U0Nj0xOZmZPJxSN6Ea6pIREJcgovIifjaoAt7zjrsjT1rAy+yFmrJdSr8TWq\nm89KqG1wsWjtHp5dms8XeysBCA81XHl2H2ZOzmBUvzO/MklEJFAovIgcT+l252qhtQuclWvH3w6X\n/NrZNupGGD3d53c63ltxmPnLd7FgRQEHaxoASImL4KYJ/blpQjo9ukX59PNFRDojhRcRbw21znTQ\nqjnOirdNUgZDj6Etr0N997+OtZbVBQd5dkk+72zYh8szNzSybwKzcjK49OzeRIb5bmpKRKSzU3gR\nATiw2VlIbt1COHzQGQuLdm5COO4W6DfhjFe5PZm6Rhdvrd/LnKX5rC+qACA0xHDp2b25NSeDsend\nMT6uQUQkECi8SNdVXwMbX3dCS+GKlvFeI52F5EZ+vV1WuT2ZA4dqeX55Ac+vKKCkqg6A7jHh3DA+\nnekT+9MnMdrnNYiIBBKFF+m63vsJ5D7jPI+Ig5Ffc0JLnzE+P8vS5LXVRdz36ufUu9wADOkVz6yc\nDK4c3ZeocE0NiYgci8KLdA21lbDhFeieAQO/4oyNuhH2rnemhYZfA5FxJ3yL9ra64GBzcLlwWE9u\nzclk4oAkTQ2JiJyEwot0DetfhLe/79xbqCm89DsHbvu3X8o5UFnLHfNWUe9yM3NyBg9dMdwvdYiI\nBCKFFwk+NWVOWMHAxDucsZFfhy1vO2db/Ky+0c23nl/NgUN1TMhM4ieXDj35QSIi0kzhRYKDtZC/\n2Gm+3bQIXHUQkwLZt0JYhNN4O+N1f1cJwEP/2MiqXQfpkxDFYzeN1Yq4IiKnSOFFAlvVAWcRudXP\nQdl2z6BxpobG3gymcwWDhZ8VsGBFARFhITw5Yxwpuv+QiMgpU3iRwON2w44PYdVcZyrI7dyUkPje\nLTdF7J7h1xKPZdWuMh58cwMAv7p6JGenaUl/EZHTofAigePQfmdaaPU8qChwxkwIDL7YuWIo60Kf\nrnx7JvZX1nLH/NU0uCwzJ2dw7bg0f5ckIhKwOuff9CLHUrIVPvql8zwh3ZkWGnMTdOvj37pOoq7R\nxR3zV1GsBl0RkXah8CKdU00ZLH8cDu2FKx9zxjKmwPjZMHgaDDjP5zdFbC8PLdrImoJyNeiKiLQT\nhRfpPKxtWdnWWljyR3A1wLn3Q0Kas+2S3/i3xlP0/IpdLPyskMiwEJ6aka0GXRGRdqDwIv5Xss3p\nZdnxMdz2sdO3EpsMF/0v9BgG3fr6u8LTkptfxkOLNgLwyLUjGZmW4OeKRESCg8KL+EfDYdj0pnOJ\n864lLeP5n7SsgDv+Nv/U1g72V9byreedBt1bczK5eowadEVE2ovCi3SsfRucsyzrX4TaCmcsPBZG\nXAPjZkLfcX4trz14N+hOGpDMjy8Z4u+SRESCisKL+F5dFWx41Qktu1e1jPcZ49zFecS1ENXNf/W1\nI2stD77hNOj2TYzmLzeOIUwNuiIi7UrhRXynthLee8AJLvVVzlhkNzj7Oie09D7bv/X5wPwVBbyY\n29SgO45kNeiKiLQ7hRdpXw21EB7lPI+Ig+0fOcGl30RnIblhV0FEjH9r9JGV+WX83NOg+3/Xns2I\nvmrQFRHxBYUXaR/WwqK7nSbcu3IhvqezDssVf4T4PtAjuPs+9lYc5lvzV9PotnxzSiZXjQnMK6RE\nRAKBJuPl9FWXOvcZAmcNlsMHoa4Sdv6nZZ+BXwn64FLb4OKO+aspqapj8sBk7r84uH9eERF/U3iR\nU+N2O+uxvDwLHh0COz5q2Xb+g/CdNU5PSxdhreXBNzewrrCpQXesGnRFRHxM00bSNof2wZr5sGYe\nHMz3DBrYuxayzndepp7lr+r8Zv7yXbyUW0RUeAhP3zyOpNgIf5ckIhL0FF7k+NwuyPsAVs2Frf8C\n63LGu6XBmOnOI7Gff2v0o892lvHzf2wCnAbd4X3UoCsi0hEUXuRo5QWesyzzoXK3M2ZCYchlziXO\nWedDSKh/a/SzvRWHufP5VTS6LbdNzeTK0WrQFRHpKG0KL8aY14C/A+9Ya92+LUn86o1vw9rnAeu8\n7p4JY2+G0Tc5VxCJ06A7bxUlVfXkZCXzo2lq0BUR6UhtPfPyODAL+JMx5mXgWWvtFt+VJR2mdDvE\nJEN0ovM6vieEhsPQy52zLBlTnUueBXAadB94YwPriipI6x7NX25Qg66ISEdr09+61toPrLU3AWOB\nfOADY8xSY8wsY0y4LwsUH/rg5/DnsZ4zLR4Tvw3/vRm+9gwM+LKCyxGeW7aLV1Z5GnRnZNNdDboi\nIh2uzd9MxphkYCbwX8Aa4I84YeZ9n1Qm7e/AZijZ1vK671gIi3aW8W8Sm+w85CgrdpTyi386Dbq/\n/toohvUJjvsxiYgEmrb2vLwOnAXMAy631u71bHrRGJPrq+KkHdRXw8Y3nJsiFq6AkdfBtX91tg2e\nBt/fAlG6SuZk9pQf5s7nnRV0b//SAK4Y1cffJYmIdFlt7Xn5k7X2o2NtsNZmt2M90l72rIXVz8Hn\nLzur3oJzr6GYpJZ9QsMhVMHlZJwVdFdRWl3P1EEp/FANuiIiftXW8DLMGLPGWlsOYIzpDtxgrX38\nRAcZY6bhTC+FAn+z1j5yxPbfA+d5XsYAPay1iZ5tLuBzz7YCa+0Vbay166qtdMLK6rmwd13LeN9s\n56aIw6+ByDj/1ReArLX85PUNrC+qoF9SNH++YQyhIcbfZYmIdGltDS+3WWsfa3phrT1ojLkN5yqk\nYzLGhAKPARcCRcBKY8wia+0mr/f5ntf+dwNjvN7isLV2dBvr67qshaKVzkJyG1+DhhpnPCoBzr7e\nCS09h/u3xgA2d2k+r64uIjo8lKdnZJMYowZdERF/a2t4CTXGGGutheZgcrK/xccDedbaHZ5jXgCu\nBDYdZ/8bgJ+1sR5psuSP8IHXb1v/Kc66LMOugPBo/9UVBJZtL+UXb30BwG++fjZDe6tBV0SkM2hr\nePkXTnPuU57Xt3vGTqQvUOj1ugiYcKwdjTH9gUzgQ6/hKE8zcCPwiLX2jTbWGryshV1LwNUAAz2z\nbUMuhWV/gVE3OOuypGT5t8Ygsbv8MN9esBqX23L7lwdw2dlq0BUR6SzaGl5+hBNYvuV5/T7wt3as\n43rgFWubbp4DQH9r7W5jzADgQ2PM59ba7d4HGWNmA7MB0tPT27GcTmrzP+HF6dDrbBj4qTOWMshZ\nlyVUd3poL7UNLm6fl0tZU4PuRWrQFRHpTNq6SJ3bWvuEtfZrnsdTRwSNY9kNeN+1L80zdizXAwuP\n+Mzdnl93AB/Tuh+maZ+nrbXZ1trs1NTUtvwogcPtdm6KuPLvLWNZF0LPETD4IufsSxMFl3ZjreXH\nr33Oht2VpCfFqEFXRKQTaus6L4OAXwHDgKimcWvtgBMcthIYZIzJxAkt1wM3HuO9hwDdgWVeY92B\nGmttnTEmBcgBft2WWgNexW5nxdvV86CiAMJj4ezrIDIewqPgjsVg9GXqK88uyee1NbudBt2bx6lB\nV0SkE2rrP9mfxWmmbbq0eRYnOWtjrW00xtwFvItzqfQz1tqNxpiHgVxr7SLPrtcDLzQ1A3sMBZ4y\nxrg9n/OI91VKQcfVCNvecy5x3vYeNN37MjEdxtzc8hoUXHxo6fYSfvm206D726+PYkgvNeiKiHRG\npnVmOM5Oxqyy1o7z9J2M9B7zeYVtlJ2dbXNzA2yx34P5zkJya56Hqn3OWEg4DLnEab4dcJ7uLdRB\nig7WcMVfllBWXc+3zh2oO0WLiHQCnqxx1GK4bT3zUmeMCQG2ec6m7Aa02tnpaKx3Gm9Xz4UdH7eM\nJ2c5lziPuhHigqx/p5M7XO/i9nmrKKuu58uDU/n+V8/yd0kiInICbQ0v9+CsgPsd4Bc4U0e3+Kqo\noFayFV6Z5TwPjYRhVzoLyfXP0ZSQH1hruf+19WzcU0n/5Bj+dL0adEVEOruThhfPgnTfsNZ+H6jC\n6XeRtmisc26KWPQZXPo7Z6zXCGdNlt6jnUZc73sNSYf7++KdvLF2DzERzgq6CTHh/i5JRERO4qTh\nxVrrMsZM6Yhigo67Ed7+vnNjxOxbW5bpv/pJ/9YlACzNK+FX72wGnAbds3rF+7kiERFpi7ZOG60x\nxiwCXgaqmwatta/5pKpAVFcFG151Hje8ABExEBELX/6hczfnxC6wiF4AKSyraV5B985zB3LJyN7+\nLklERNqoreElCigFvuI1ZoGuHV6shT2rnZsibngV6quc8U1vwugbnOeT7/ZffXJMTQ26B2saOPes\nVO5Vg66ISEBpU3ix1qrPxdvhcvj8ZSe07P+8ZbzfRKf5dtiV/qtNTshay32vrWfT3koykmP44zfU\noCsiEmjausLuszhnWlqx1t7a7hV1VtZCwTInsGx6AxprnfHoJM9NEW+GHlobpLP7++KdvLl2D7ER\noTx9sxp0RUQCUVunjf7p9TwKuBrY0/7ldEKuRljxhLOYXMnWlvHMLzkLyQ29HMIi/VeftNmSvBL+\n17OC7u+uG8XgnmrQFREJRG2dNnrV+7UxZiGw2CcVdTYhobDuRSe4xPWE0TfB2BmQdKLbOklnU1hW\nw10LVuO2cNd5WUwboQZdEZFAdbq3Ix4E9GjPQjotY+D8n4KrHgZPg1BNMwSaw/UuZnsadM87K5Xv\nXTjY3yWJiMgZaGvPyyFa97zsA37kk4o6o8EX+bsCOU3WWn746nq+2FtJZkosf9AKuiIiAa+t00Zq\nDpCA9NdPd/CPdZ4G3RnjSIjWmTMRkUDXplsWG2OuNsYkeL1ONMZc5buyRM7cp9uKecSzgu7vrhvN\nIDXoiogEhTaFF+Bn1tqKphfW2nLgZ74pSeTMFZbVcPfCNbgtfOcrWUwb0cvfJYmISDtpa3g51n6n\n2+wr4lM19Y3c9lwu5TUNnD+kB9+9QA26IiLBpK3hJdcY86gxZqDn8SiwypeFiZwOay0/fGU9m/cd\nYkBKLL+/fjQhatAVEQkqbQ0vdwP1wIvAC0At8G1fFSVyup7+ZAf/XL+XuMgwnr55HN2i1KArIhJs\n2nq1UTVwn49rETkjn2wt5v/+5TToPnrdKLJ6qEFXRCQYtfVqo/eNMYler7sbY971XVkip2ZXaXVL\ng+75g/jqcDXoiogEq7ZOG6V4rjACwFp7kK6ywq50ejX1jdw+bxUVhxu4YGgPvnv+IH+XJCIiPtTW\n8OI2xqQ3vTDGZHCMu0yLdDRrLT9oatBNjeXRb6hBV0Qk2LX1cuefAIuNMf8BDDAVmO2zqkTa6Mn/\n7OCtpgbdGdlq0BUR6QLa2rD7L2NMNk5gWQO8ARz2ZWEiJ/OfrcX8+l2nQff33xhNVo84P1ckIiId\noa03Zvwv4B4gDVgLTASWAV/xXWkix7ertJq7F6zGWvjuBYO4cFhPf5ckIiIdpK09L/cA5wC7rLXn\nAWOA8hMfIuIb1XWNzH5uFZW1jVw4rCff+YoadEVEupK2hpdaa20tgDEm0lq7GTjLd2WJHJvToLuO\nLfsPMTA1lkevG6UGXRGRLqatDbtFnnVe3gDeN8YcBHb5riyRY3v84+28/fk+4iPDePrmbOLVoCsi\n0uW0tWH3as/Th4wxHwEJwL98VpXIMXy05QC/fW8L4DToDkxVg66ISFd0yneGttb+xxeFiJxIfkk1\n9yxcg7XwvQsGc4EadEVEuqy29ryI+E1VXSOz5+VSWdvIV4f15O6vZPm7JBER8SOFF+nUrLX84OV1\nbN1fRVaPOH6nBl0RkS5P4UU6tcc/3s47G/YRHxXG0zPGqUFXREQUXqTz+miz06BrDPzx+tEMUIOu\niIig8CKd1M6Sar7zgtOg+98XDOYrQ9SgKyIiDoUX6XSq6hqZ/Vwuh2obuWh4T759nhp0RUSkhcKL\ndCput+Xel9ay7UAVg3rE8bvrRqtBV0REWlF4kU7lsY/yeHfjfqdB9+Zs4iJPeSkiEREJcgov0ml8\nuHk/j36wFWPgT9ePITMl1t8liYhIJ6TwIp3CjuIq7lm4Fmvh3gsHc96QHv4uSUREOimFF/G7Q7UN\nzJ63ikN1jVw8opcadEVE5IQUXsSvnAbddeQdqGJwzzh++/VRGKMGXREROT6FF/GrP3+Yx3ub9tMt\nKoynZ2QTqwZdERE5CYUX8ZsPNu3n900NujeMIUMNuiIi0gYKL+IX24ur+N6LawH4/lfP4tyz1KAr\nIiJto/AiHe5QbYOzgm5dI5eM7MWd5w70d0kiIhJAFF6kQ7ndlu+9uI7txdWc1TOe33xNDboiInJq\nfBpejDHTjDFbjDF5xpj7jrH998aYtZ7HVmNMude2W4wx2zyPW3xZp3ScP324jQ++8DTo3jxODboi\nInLKfPbNYYwJBR4DLgSKgJXGmEXW2k1N+1hrv+e1/93AGM/zJOBnQDZggVWeYw/6ql7xvfc37ecP\nH2xrbtDtn6wGXREROXW+PPMyHsiz1u6w1tYDLwBXnmD/G4CFnucXAe9ba8s8geV9YJoPaxUfyzvQ\n0qD7w4uGqEFXREROmy/DS1+g0Ot1kWfsKMaY/kAm8OGpHGuMmW2MyTXG5BYXF7dL0dL+KmsbmD0v\nl6q6Ri49uzd3fHmAv0sSEZEA1lkadq8HXrHWuk7lIGvt09babGttdmpqqo9KkzPhdlu+98JadhRX\nM6RXPL/52tlq0BURkTPiy/CyG+jn9TrNM3Ys19MyZXSqx0on9od/b+Pfmw+QEB3O0zOyiYlQg66I\niJwZX4aXlcAgY0ymMSYCJ6AsOnInY8wQoDuwzGv4XeCrxpjuxpjuwFc9YxJA3t24jz/9exshBv58\nwxjSk2P8XZKIiAQBn/0z2FrbaIy5Cyd0hALPWGs3GmMeBnKttU1B5nrgBWut9Tq2zBjzC5wABPCw\ntbbMV7VK+8s7cIj/bmrQnTaELw3WtJ6IiLQP45UZAlp2drbNzc31dxmC06B71V+WsKOkmsvO7s2f\nbxijPhcRETllxphV1trsI8c7S8OuBInmBt0Sp0H312rQFRGRdqbwIu3qDx9s5d+bD5AYowZdERHx\nDYUXaTf/2rCPP32YR4iBv9wwVg26IiLiEwov0i627T/EvS85Dbr3XTyEKYNS/FyRiIgEK4UXOWMV\nhxuYPW8V1fUurhjVh9umagVdERHxHYUXOSMut+W7L6xhZ0k1Q3t34/+uVYOuiIj4lsKLnJHfv7+V\nj7YUexp0xxEdEervkkREJMgpvMhp+9eGvfzlo5YG3X5JatAVERHfU3iR07J1/yH++6V1ANx/8VA1\n6IqISIdReJFTVlHTwOzncqmpd3Hl6D7819RMf5ckIiJdiMKLnBKX23LPi2vIL61hWO9uPHKNGnRF\nRKRjKbzIKfnde1v4eEsx3WPCeUoNuiIi4gcKL9Jmb3++l8c/3k5oiOGxG9WgKyIi/qHwIm2yZd8h\nvv9yU4PuECZnqUFXRET8Q+FFTqqipoHZ85wG3atG9+GbU9SgKyIi/qPwIifkclvufmENu0prGN6n\nG79Sg66IiPiZwouc0G/f28InW4tJio1Qg66IiHQKCi9yXG+t38sTngbdv9w4hrTuatAVERH/U3iR\nY/pib2Vzg+5PLhnK5IFq0BURkc5B4UWOUl5Tz+x5uRxucHHN2L7Mysnwd0kiIiLNFF6kFZfbcvfC\nNRSWHWZk3wT+9+qRatAVEZFOReFFWvn1u5v5dFsJybERPDljHFHhatAVEZHOReFFmv1j3R6e+s8O\nT4PuWPomRvu7JBERkaMovAgAm/ZU8sNX1gPwwKVDmTQw2c8ViYiIHJvCi3Cwup7b57c06M6cnOHv\nkkRERI5L4aWLa3S5+c4LatAVEZHAofDSxf3m3S3NDbpPqUFXREQCgMJLF/bm2t089ckOwkIMj980\nlj5q0BURkQCg8NJFbdxTwY9edRp0f3rZMCYMUIOuiIgEBoWXLuhgdT23z1tFbYObr41L4+ZJ/f1d\nkoiISJspvHQxjS43dy1cTdHBw4xKS+B/rhqhBl0REQkoCi9dzP/9azNL8kpJidMKuiIiEpgUXrqQ\nN9fu5q+f7vQ06I6jd4IadEVEJPAovHQR3g26D14+jPGZSX6uSERE5PQovHQBZdX1zH7OadD9+rg0\nZkxUg66IiAQuhZcg1+hyc9eC1ewuP8yofon8Qg26IiIS4BRegtyv3tnM0u2lpMRF8tR0NeiKiEjg\nU3gJYq+vKeLvi50G3Semj6VXQpS/SxIRETljCi9BasPuCu579XMAfnbFcM7JUIOuiIgEB4WXIFRa\nVcft81ZR1+jmG9n9mD4h3d8liYiItBuFlyDjNOiuYXf5YUb3S+Thq4arQVdERIKKwkuQ+d+3N7Ns\nRymp8ZE8OX0ckWFq0BURkeCi8BJEXltdxDNLdhIeanjiJjXoiohIcFJ4CRKfF1Vw/2ueBt3Lh5Ot\nBl0REQlSCi9BoKSqjtvn5VLX6OaG8f24SQ26IiISxBReAlyDy823n1/NnopaxqYn8tAVatAVEZHg\n5tPwYoyZZozZYozJM8bcd5x9rjPGbDLGbDTGLPAadxlj1noei3xZZyD75VtfsGJnGanxkTyhBl0R\nEekCwnz1xsaYUOAx4EKgCFhpjFlkrd3ktc8g4H4gx1p70BjTw+stDltrR/uqvmDw6qoi5izNJzzU\n8OT0sfTspgZdEREJfr488zIeyLPW7rDW1gMvAFcesc9twGPW2oMA1toDPqwnqKwvKuf+150G3Z9f\nMYJx/dWgKyIiXYMvw0tfoNDrdZFnzNtgYLAxZokxZrkxZprXtihjTK5n/Cof1hlwSjwr6NY3urlh\nfDo3qkFXRES6EJ9NG53C5w8CzgXSgE+MMSOtteVAf2vtbmPMAOBDY8zn1trt3gcbY2YDswHS07vG\nF3iDy82dz69mb3OD7jB/lyQiItKhfHnmZTfQz+t1mmfMWxGwyFrbYK3dCWzFCTNYa3d7ft0BfAyM\nOfIDrLVPW2uzrbXZqamp7f8TdEK/fOsLPttZRg+toCsiIl2UL8PLSmCQMSbTGBMBXA8cedXQGzhn\nXTDGpOBMI+0wxnQ3xkR6jecAm+jiXs4tZM7SfCJCQ3hyxjh6qEFXRES6IJ9NG1lrG40xdwHvAqHA\nM9bajcaYh4Fca+0iz7avGmM2AS7gB9baUmPMZOApY4wbJ2A94n2VUle0rrCcn7yxAYCHrxzO2PTu\nfq5IRETEP4y11t81tIvs7Gybm5vr7zJ8ovhQHZf/eTH7Kmu5aUI6v7x6pL9LEhER8TljzCprbfaR\n41pht5Orb3RW0N1XWUt2/+787PLh/i5JRETErxReOrn/eWsTn+WX0bNbJI9PH0tEmP6TiYhI16Zv\nwk7spdxCnlu2y2nQnT6OHvFq0BUREVF46aTWFpbzwOtOg+4vrhrOGDXoioiIAAovndKBQ7XcMW8V\n9S430yem841zusYCfCIiIm2h8NLJ1De6uXO+06B7TkZ3HrxMDboiIiLeFF46mYf/uZHcXQfp1S2K\nx25Sg66IiMiR9M3Yiby4soD5ywuICPOsoKsGXRERkaMovHQSqwsO8tM3NgLwP1eNYHS/RD9XJCIi\n0jkpvHQCBw7V8q35ToPuzZP6c112v5MfJCIi0kUpvPhZU4Pu/so6xmck8dPLhvm7JBERkU5N4cXP\nfv4Pp0G3d4LToBseqv8kIiIiJ6JvSj9a+FkBz6/wNOhOH0dqfKS/SxIREen0FF78ZNWugzz4prOC\n7i+vGsEoNeiKiIi0icKLH+yvdBp0G1yWmZMz+LoadEVERNpM4aWD1TW6+Nb8VRw4VMeEzCR+culQ\nf5ckIiISUBReOthDizaxuqCcPmrQFREROS365uxAC1YUsPCzlhV0U+LUoCsiInKqFF46yKpdZfxs\nkdOg+6urR3J2mhp0RURETofCSwfYX1nLHfNXNzfoXjsuzd8liYiIBCyFFx+ra3Rxx/xVFKtBV0RE\npF0ovPiQtZafvbmRcND0hgAACehJREFUNWrQFRERaTf6JvWh51cU8MLKQiLDQnhqRrYadEVERNqB\nwouP5OaX8fN/bATgV9eMZGRagp8rEhERCQ4KLz6wr6KlQffWnEyuGasGXRERkfai8NLOmhp0S6rq\nmDQgmR9fMuT/27v/WLvr+o7jz5dt0WqdLbRmWJiAVCMaoXjTMBmGzKjd/qAmdg6cFRZ/RIRs/rMp\nxh+x+2tLtiUTEmQbCb+cTCbkSjSOOcPispZemjJo1a1WkctIWgtUcSBtee+P8+16er2l58A953vO\nvc9HcpLv+Z7POXmfdz73ft/n+32f82k7JEmS5hWLlzlUVXzurp3seORJVi9fyrXvX8tiG3QlSZpT\nHlnn0K1bf8LtU0cadN/KKTboSpI05yxe5si2Hz/OFyY7Dbp//t638ObVNuhKkjQIFi9z4LEDT3Pl\nrds59Fzxod86k/esXd12SJIkzVsWLy/SMwcP87FbOg26b3vdKVzzOzboSpI0SBYvL0JV8dm7HuKB\n6QNNg+75NuhKkjRgHmlfhFu2PMxX75/mZUs6Dbonv+KktkOSJGnes3h5gbbu2c/mr+8CbNCVJGmY\nLF5egP958mmu+nKnQfcjF53JhvNs0JUkaVgsXvr0zMHDXHnr/fz0qWe58OxT+OR6G3QlSRomi5c+\nVBWfaRp0T1uxlGsvs0FXkqRh88jbh5v/42HuaBp0b9g0wQobdCVJGjqLlx5t2bOfzXd3GnT/YuO5\nnPOaX2s5IkmSFiaLlx48+uTTXHXbdg4/V3z07WdxybmvaTskSZIWLIuXEzjyC7r7f/EsF61ZyZ++\n+w1thyRJ0oJm8fI8qopP3/kgDz56gNNPXsoXL1trg64kSS3zSPw8Dj1XULB0ySJu2DTB8pfboCtJ\nUtsWtx3AKFuy6CX85fvO5erfPpuzVi1rOxxJkoRnXk4oiYWLJEkjxOJFkiSNFYsXSZI0VgZavCRZ\nn+QHSXYn+dRxxrwvya4kO5N8uWv/5Un+u7ldPsg4JUnS+BhYw26SRcB1wDuBaWBbksmq2tU1Zg1w\nDXBhVT2R5NXN/pOBzwMTQAH3N899YlDxSpKk8TDIMy/rgN1VtaeqngW+AmyYMeYjwHVHipKq2tvs\nfzdwT1U93jx2D7B+gLFKkqQxMcjiZTXwSNf96WZft9cDr0/y70m2JFnfx3NJ8tEkU0mm9u3bN4eh\nS5KkUdV2w+5iYA1wMXAZ8LdJlvf65Kq6oaomqmpi1apVAwpRkiSNkkEWL48Cp3fdP63Z120amKyq\ng1X1I+C/6BQzvTxXkiQtQIMsXrYBa5KcmeQk4FJgcsaYu+icdSHJSjqXkfYA3wLelWRFkhXAu5p9\nkiRpgRvYt42q6lCSq+kUHYuAG6tqZ5LNwFRVTXK0SNkFHAb+pKr2AyT5MzoFEMDmqnp8ULFKkqTx\nkapqO4Y5MTExUVNTU22HIUmS5kiS+6tqYub+tht2JUmS+jJvzrwk2Qc8PKCXXwn8dECvPR+Zr/6Y\nr/6Zs/6Yr/6Yr/4MMl+vrapf+TrxvCleBinJ1GynrTQ789Uf89U/c9Yf89Uf89WfNvLlZSNJkjRW\nLF4kSdJYsXjpzQ1tBzBmzFd/zFf/zFl/zFd/zFd/hp4ve14kSdJY8cyLJEkaKxYvXZKsT/KDJLuT\nfGqWx1+a5Pbm8a1Jzhh+lKOjh3xdkWRfkh3N7cNtxDkqktyYZG+Sh47zeJL8TZPP/0xy/rBjHCU9\n5OviJAe65tfnhh3jqEhyepLvJNmVZGeSP55ljPOr0WO+nF9dkrwsyX1JHmhy9oVZxgzvGFlV3jqX\nzhYBPwTOAk4CHgDOmTHm48D1zfalwO1txz3i+boCuLbtWEflBrwdOB946DiP/y7wTSDABcDWtmMe\n8XxdDNzddpyjcANOBc5vtl9JZ5HbmX+Pzq/+8uX8OjYfAZY120uArcAFM8YM7RjpmZej1gG7q2pP\nVT0LfAXYMGPMBuCmZvsO4B1JMsQYR0kv+VKXqvo34PnW6NoA3FwdW4DlSU4dTnSjp4d8qVFVj1XV\n9mb758D3gNUzhjm/Gj3mS12aefNUc3dJc5vZNDu0Y6TFy1GrgUe67k/zq5P5/8dU1SHgAHDKUKIb\nPb3kC+C9zSnqO5KcPpzQxlavOdVRv9mcxv5mkje1HcwoaE7Vr6Xzybib82sWz5MvcH4dI8miJDuA\nvcA9VXXcOTboY6TFiwbp68AZVfUW4B6OVuTSXNhO56fDzwW+CNzVcjytS7IM+CfgE1X1s7bjGXUn\nyJfza4aqOlxV5wGnAeuSvLmtWCxejnoU6D4zcFqzb9YxSRYDrwL2DyW60XPCfFXV/qr6ZXP374C3\nDim2cdXLHFSjqn525DR2VX0DWJJkZcthtSbJEjoH4tuq6muzDHF+dTlRvpxfx1dVTwLfAdbPeGho\nx0iLl6O2AWuSnJnkJDrNRpMzxkwClzfbG4F/raYzaQE6Yb5mXE+/hM51ZR3fJPDB5lshFwAHquqx\ntoMaVUl+/cj19CTr6Pw/W5AfJpo8/D3wvar6q+MMc341esmX8+tYSVYlWd5sLwXeCXx/xrChHSMX\nD+JFx1FVHUpyNfAtOt+kubGqdibZDExV1SSdyX5Lkt10GgkvbS/idvWYrz9KcglwiE6+rmgt4BGQ\n5B/ofINhZZJp4PN0mt6oquuBb9D5Rshu4H+BP2wn0tHQQ742AlcmOQQ8DVy6gD9MXAhsAh5sehIA\nPg38Bji/ZtFLvpxfxzoVuCnJIjqF3D9W1d1tHSP9hV1JkjRWvGwkSZLGisWLJEkaKxYvkiRprFi8\nSJKksWLxIkmSxorFi6R5p1kR+O6245A0GBYvkiRprFi8SGpNkg8kuS/JjiRfahZ+eyrJXyfZmeTb\nSVY1Y89LsqVZ6PPOJCua/Wcn+ZdmAb3tSV7XvPyyZkHQ7ye5bQGvAC/NOxYvklqR5I3A7wMXNou9\nHQb+AHgFnV/sfBNwL51f1gW4Gfhks9Dng137bwOuaxbQextw5Cfv1wKfAM4BzqLzq6qS5gGXB5DU\nlnfQWaxzW3NSZCmwF3gOuL0ZcyvwtSSvApZX1b3N/puAryZ5JbC6qu4EqKpnAJrXu6+qppv7O4Az\ngO8O/m1JGjSLF0ltCXBTVV1zzM7kszPGvdA1TH7ZtX0Y/99J84aXjSS15dvAxiSvBkhycpLX0vm/\ntLEZ837gu1V1AHgiyUXN/k3AvVX1c2A6yXua13hpkpcP9V1IGjo/iUhqRVXtSvIZ4J+TvAQ4CFwF\n/AJY1zy2l05fDMDlwPVNcbKHo6sibwK+1KxuexD4vSG+DUktcFVpSSMlyVNVtaztOCSNLi8bSZKk\nseKZF0mSNFY88yJJksaKxYskSRorFi+SJGmsWLxIkqSxYvEiSZLGisWLJEkaK/8H5VEGUuuUYjIA\nAAAASUVORK5CYII=\n", 517 | "text/plain": [ 518 | "
" 519 | ] 520 | }, 521 | "metadata": { 522 | "tags": [] 523 | } 524 | } 525 | ] 526 | } 527 | ] 528 | } -------------------------------------------------------------------------------- /Chapter5/README.md: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /Data/README.md: -------------------------------------------------------------------------------- 1 | 2 | 3 | dog_n_cup.mp4: https://bjpublic-my.sharepoint.com/:v:/g/personal/bjpublic_bjpublic_co_kr/EYSXHq87jL9Gkj-QWArZs6EBr481ACJ7U7WM_r8VyOJdng?e=57pmHz 4 | 5 | oxford-flowers-17: http://www.robots.ox.ac.uk/~vgg/data/flowers/17/17flowers.tgz 6 | -------------------------------------------------------------------------------- /Data/dataset.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bjpublic/TMI-Deeplearning/f955fb015a3cecf9c3cde45f640b1353034cf2a5/Data/dataset.zip -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # 친절한 실전 딥러닝 수업 2 | 3 | ## 구매 링크 4 | 5 | - [yes24](http://www.yes24.com/Product/Goods/89649505?scode=032&OzSrank=1) 6 | - [교보문고](http://www.kyobobook.co.kr/product/detailViewKor.laf?ejkGb=KOR&mallGb=KOR&barcode=9791190014809&orderClick=LET&Kc=) 7 | - [알라딘](https://www.aladin.co.kr/shop/wproduct.aspx?ItemId=236205748) 8 | 9 | ![입체표지](https://user-images.githubusercontent.com/21074282/77288089-650ab800-6d1a-11ea-83e3-aea5886e9661.png) 10 | 11 | ## 책 소개 12 | 13 | >기초 딥러닝 모델을 기반으로 배우는 트랜스퍼 러닝, 14 | >한 권으로 끝내자 15 | 16 | 딥러닝의 학습 과정은 엄청난 규모의 데이터와 컴퓨팅 파워(돈과 시간)가 필요하다. 그래서 아무리 열심히 해도 세계 최고를 이기기는 힘들다. 하지만, 이것은 기초가 되는 모델에 대한 이야기이다. 세계 최고의 기업과 연구소에서 학습한 기초 모델(Base Model)을 기반으로 트랜스퍼 러닝(Transfer Learning)을 한다면, 누구나 그보다 뛰어난 결과를 낼 수 있다. 쉽게 말해, 구글이 학습한 모델을 '더' 학습하여 구글보다 똑똑한 인공지능을 만들 수 있다. 왜냐하면, 기본 모델은 범용적인 모델이기 때문이다. 특정한 문제를 풀기 위해서는 이 기본 모델에서 한 걸음 더 나아가야 하는데, 그 방법이 바로 트랜스퍼 러닝이다. 이 책에서는 기본적인 케라스 프레임워크와 딥러닝 이론으로 시작하여 트랜스퍼 러닝에 대해 집중해서 다루고자 한다. 17 | 18 | ## 이 책의 특징 19 | 20 | - 케라스 프레임워크를 통한 기초 딥러닝 모델 구현 및 문제를 해결한다. 21 | - 수학적인 이론 및 증명보다는 설명과 예제에 집중한다. 22 | - 제한된 자원 속에서 실전 개발 및 연구 문제 해결을 위한 트랜스퍼 러닝 활용한다. 23 | 24 | ## 이 책이 필요한 독자 25 | 26 | - 머신러닝/딥러닝을 실제로 구현하고 실전 문제에 적용시켜보고 싶은 분들 27 | - 기술적 트렌드에 뒤쳐지는 것이 싫은, 빠르게 현대 기술의 시작점에 서고 싶은 분들 28 | - 수학적 증명보다는 친절한 설명이 필요한 분들 29 | 30 | ## 목차 31 | 32 | Chapter 1. 시작하는 이들을 위한 잔소리 33 | - 1.1 딥러닝의 아버지 34 | - 1.2 뭐가 딥하고, 뭐가 힙한지? 35 | - 1.3 학회부터 깃허브까지 36 | - 1.4 이 책의 I/O 37 | 38 | Chapter 2. 현대적인 개발환경 구축하기 39 | - 2.1 윈도우 환경 40 | - 2.2 리눅스 환경 41 | - 2.3 기타 라이브러리 및 개발환경 42 | 43 | Chapter 3. 케라스로 시작하는 딥러닝 기초 44 | - 3.1 인공신경망의 구성요소 45 | - 3.2 범죄율로 집값 예측하기 (기초 회귀 문제) 46 | - 3.3 간단한 과일 분류 문제 비틀기 (이미지 분류 기초 문제) 47 | - 3.4 뉴스 기사(문서) 분류하기 48 | - 3.5 학습을 더 쉽고, 빠르고, 정확하게! 49 | 50 | Chapter 4. 계속 연구되고 있는 고급 딥러닝 51 | - 4.1 트랜스퍼 러닝 52 | - 4.2 자기지도학습과 준지도학습 (Self-supervised Learning and Semi-supervised Learning) 53 | - 4.3 당신에게 필요한 건 집중! 54 | 55 | Chapter 5. 실전 딥러닝 56 | - 5.1 딥러닝이 푼 첫 문제 이미지 분류 57 | - 5.2 사람도 찾고, 강아지도 찾는 객체 검출 58 | - 5.3 모든 것을 답해주는 자비스를 기대하며 59 | - 5.4 딥러닝과 마이크로 서비스 60 | - 5.5 인공지능을 위한 인공지능 61 | 62 | ## 출판사 리뷰 63 | 이 책은 딥러닝을 실전에 활용할 수 있도록 실습 위주의 내용으로 구성하였다. ‘딥러닝을 지금 시작하기에는 너무 늦었다’라는 말을 할 수도 있겠지만 이 책은 어려운 수학을 설명하는 데에 내용을 할애하는 것이 아닌 실전으로 영상과 텍스트에서 각각 어떻게 적용되는지에 대해 흥미롭게 설명하므로 포기하지 않고 따라갈 수 있을 것이다. 영상과 자연어 처리, 음성 처리 등 다양한 분야에서 딥러닝이 활용되고 있는데 이 책을 끝내면 딥러닝에 대해 한 걸음 더 나아가는 자기 자신을 발견할 것이다. 64 | --------------------------------------------------------------------------------