├── README.md └── RNN implementation with Numpy (binary addition).ipynb /README.md: -------------------------------------------------------------------------------- 1 | # RNN-implementation-using-Numpy-binary-digit-addition 2 | RNN implementation using only Numpy. Try to learn binary digit addition. 3 | ## [objective] 4 | * Understand RNN with a simple numpy implementation. 5 | * Train RNN for a binary opperation, e.g. addition. 6 | * Check if the trained RNN can be extended to the unseen data with longer digits (e.g. 8 bytes digits training -> 10 bytes digit test) 7 | ## Problem (binary operation, especially addition) 8 | ![binary addition](https://iamtrask.github.io/img/binary_addition.GIF "Logo Title Text 1") 9 | ## [Refercence] 10 | If you want more detailed background with explanations about RNN: 11 | * original post (Eng): https://iamtrask.github.io/2015/11/15/anyone-can-code-lstm/ 12 | * Korean translation : http://jaejunyoo.blogspot.com/2017/06/anyone-can-learn-to-code-LSTM-RNN-Python.html 13 | -------------------------------------------------------------------------------- /RNN implementation with Numpy (binary addition).ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "## Anyone can Learn To Code an LSTM-RNN in Python\n", 8 | "\n", 9 | "원문 링크 (Eng): https://iamtrask.github.io/2015/11/15/anyone-can-code-lstm/\n", 10 | "\n", 11 | "번역 링크 (Kor): http://jaejunyoo.blogspot.com/2017/06/anyone-can-learn-to-code-LSTM-RNN-Python.html\n", 12 | "\n", 13 | "### [목표]\n", 14 | "* 간단한 toy code로 RNN을 이해한다. \n", 15 | "* RNN을 사용하여 이진수 더하기 연산을 학습시킨다.\n", 16 | "\n", 17 | "### [objective]\n", 18 | "* Understand RNN with a simple numpy implementation.\n", 19 | "* Train RNN for a binary opperation, e.g. addition.\n", 20 | "* Check if the trained RNN can be extended for the unseen data with longer digits (e.g. 8 bytes digits training -> 10 bytes digit test)\n" 21 | ] 22 | }, 23 | { 24 | "cell_type": "code", 25 | "execution_count": 1, 26 | "metadata": { 27 | "collapsed": true 28 | }, 29 | "outputs": [], 30 | "source": [ 31 | "# Import libraries\n", 32 | "import copy, numpy as np\n", 33 | "import matplotlib.pyplot as plt\n", 34 | "np.random.seed(0)\n", 35 | "%matplotlib inline" 36 | ] 37 | }, 38 | { 39 | "cell_type": "code", 40 | "execution_count": 2, 41 | "metadata": { 42 | "collapsed": true 43 | }, 44 | "outputs": [], 45 | "source": [ 46 | "# Utility functions\n", 47 | "def sigmoid(x):\n", 48 | " output =1/(1+np.exp(-x))\n", 49 | " return output\n", 50 | "\n", 51 | "def sigmoid_output_to_derivative(output): \n", 52 | " return output*(1-output)" 53 | ] 54 | }, 55 | { 56 | "cell_type": "code", 57 | "execution_count": 3, 58 | "metadata": { 59 | "collapsed": false 60 | }, 61 | "outputs": [], 62 | "source": [ 63 | "# Decide the maximum binary dimension (최대 이진수의 범위를 8 byte로 한정)\n", 64 | "max_binary_dim = 8\n", 65 | "largest_number = pow(2,max_binary_dim)\n", 66 | "#print(2**8)\n", 67 | "#print(pow(2,max_binary_dim))\n", 68 | "#print(range(2**3))" 69 | ] 70 | }, 71 | { 72 | "cell_type": "markdown", 73 | "metadata": {}, 74 | "source": [ 75 | "## Create binary lookup table" 76 | ] 77 | }, 78 | { 79 | "cell_type": "code", 80 | "execution_count": 4, 81 | "metadata": { 82 | "collapsed": false 83 | }, 84 | "outputs": [ 85 | { 86 | "name": "stdout", 87 | "output_type": "stream", 88 | "text": [ 89 | "[0 0 0 0 1 0 0 0]\n", 90 | "====================\n", 91 | "(256, 8) [[0 0 0 ..., 0 0 0]\n", 92 | " [0 0 0 ..., 0 0 1]\n", 93 | " [0 0 0 ..., 0 1 0]\n", 94 | " ..., \n", 95 | " [1 1 1 ..., 1 0 1]\n", 96 | " [1 1 1 ..., 1 1 0]\n", 97 | " [1 1 1 ..., 1 1 1]]\n", 98 | "====================\n", 99 | "lookup table test\n", 100 | "[0 0 0 0 0 0 1 1] [0 0 0 0 0 0 1 1]\n" 101 | ] 102 | } 103 | ], 104 | "source": [ 105 | "# Create binary lookup table (그저 편의상 만들뿐 굳이 이런 식으로 하지 않아도 됨)\n", 106 | "# np.unpackbits e.g.\n", 107 | "print(np.unpackbits(np.array([8], dtype = np.uint8)))\n", 108 | "print(\"====================\")\n", 109 | "# 이진수로 만들 정수 값들 1~256을 list in list 형태로 만듬.\n", 110 | "# e.g\n", 111 | "# binary_gonna_be = np.array([range(largest_number)], dtype=np.uint8).T\n", 112 | "# print(binary_gonna_be)\n", 113 | "\n", 114 | "# 이런 식으로 binary lookup table 완성\n", 115 | "binary = np.unpackbits(np.array([range(largest_number)], dtype=np.uint8).T, axis = 1)\n", 116 | "print(binary.shape, binary)\n", 117 | "print(\"====================\")\n", 118 | "int2binary = {}\n", 119 | "for i in range(largest_number):\n", 120 | " int2binary[i] = binary[i]\n", 121 | "print(\"lookup table test\")\n", 122 | "print(binary[3], int2binary[3])\n", 123 | "#print(int2binary)" 124 | ] 125 | }, 126 | { 127 | "cell_type": "markdown", 128 | "metadata": {}, 129 | "source": [ 130 | "## Initial parameter setting" 131 | ] 132 | }, 133 | { 134 | "cell_type": "code", 135 | "execution_count": 5, 136 | "metadata": { 137 | "collapsed": false 138 | }, 139 | "outputs": [ 140 | { 141 | "name": "stdout", 142 | "output_type": "stream", 143 | "text": [ 144 | "(2, 16) (16, 1) (16, 16)\n" 145 | ] 146 | } 147 | ], 148 | "source": [ 149 | "alpha = 0.1 # learning rate\n", 150 | "input_dim = 2 # 각 자리수끼리 더할 것이므로 서로 더할 두 이진수의 n번째 자리에 해당하는 digit 두 개가 input이 됨\n", 151 | "hidden_dim = 16 # 바꾸면서 성능이 어떻게 변하는지 확인해보면서 놀아보자, You can vary this and see what happens\n", 152 | "output_dim = 1 # output은 결국 n번째 자리의 digit 두 개가 합해서 나올 값이므로 one dim이 된다. e.g. 1(2) + 1(2) = 0(2) with overflow 1\n", 153 | "\n", 154 | "# weight initialization\n", 155 | "synapse_0 = 2*np.random.random((input_dim,hidden_dim))-1\n", 156 | "synapse_1 = 2*np.random.random((hidden_dim,output_dim))-1\n", 157 | "synapse_h = 2*np.random.random((hidden_dim,hidden_dim))-1\n", 158 | "\n", 159 | "print(synapse_0.shape, synapse_1.shape, synapse_h.shape)" 160 | ] 161 | }, 162 | { 163 | "cell_type": "code", 164 | "execution_count": 6, 165 | "metadata": { 166 | "collapsed": true 167 | }, 168 | "outputs": [], 169 | "source": [ 170 | "# saving for updates and visualization\n", 171 | "s0_update = np.zeros(synapse_0.shape) # s0_update = np.zeros_like(synapse_0)\n", 172 | "s1_update = np.zeros(synapse_1.shape) \n", 173 | "sh_update = np.zeros(synapse_h.shape) \n", 174 | "\n", 175 | "overallError_history = list()\n", 176 | "accuracy = list()\n", 177 | "accuracy_history = list()\n", 178 | "accuracy_count = 0" 179 | ] 180 | }, 181 | { 182 | "cell_type": "markdown", 183 | "metadata": {}, 184 | "source": [ 185 | "## training!!" 186 | ] 187 | }, 188 | { 189 | "cell_type": "code", 190 | "execution_count": 7, 191 | "metadata": { 192 | "collapsed": false 193 | }, 194 | "outputs": [ 195 | { 196 | "name": "stdout", 197 | "output_type": "stream", 198 | "text": [ 199 | "Error:[ 3.97540736]\n", 200 | "Pred:[0 0 0 0 0 0 0 0]\n", 201 | "True:[0 1 0 0 0 1 1 1]\n", 202 | "False\n", 203 | "10 + 61 = 0\n", 204 | "------------\n", 205 | "Error:[ 4.0325467]\n", 206 | "Pred:[0 1 1 0 1 1 0 1]\n", 207 | "True:[1 0 0 1 0 1 0 1]\n", 208 | "False\n", 209 | "111 + 38 = 109\n", 210 | "------------\n", 211 | "Error:[ 3.79424599]\n", 212 | "Pred:[0 0 0 0 0 0 0 0]\n", 213 | "True:[0 0 1 1 1 0 0 0]\n", 214 | "False\n", 215 | "55 + 1 = 0\n", 216 | "------------\n", 217 | "Error:[ 3.87395152]\n", 218 | "Pred:[0 0 0 0 0 0 0 0]\n", 219 | "True:[0 0 0 1 1 0 0 1]\n", 220 | "False\n", 221 | "20 + 5 = 0\n", 222 | "------------\n", 223 | "Error:[ 4.01458433]\n", 224 | "Pred:[1 1 1 1 1 1 1 1]\n", 225 | "True:[0 1 0 1 0 1 1 0]\n", 226 | "False\n", 227 | "12 + 74 = 255\n", 228 | "------------\n", 229 | "Error:[ 3.99141219]\n", 230 | "Pred:[0 0 0 0 0 0 0 0]\n", 231 | "True:[0 1 1 0 0 0 1 0]\n", 232 | "False\n", 233 | "30 + 68 = 0\n", 234 | "------------\n", 235 | "Error:[ 3.97121353]\n", 236 | "Pred:[0 0 0 0 1 0 0 0]\n", 237 | "True:[0 0 1 0 0 0 0 1]\n", 238 | "False\n", 239 | "28 + 5 = 8\n", 240 | "------------\n", 241 | "Error:[ 4.02250446]\n", 242 | "Pred:[1 1 1 1 1 1 1 0]\n", 243 | "True:[0 1 0 1 1 0 1 1]\n", 244 | "False\n", 245 | "60 + 31 = 254\n", 246 | "------------\n", 247 | "Error:[ 4.04634296]\n", 248 | "Pred:[1 1 0 1 0 1 0 1]\n", 249 | "True:[0 1 1 1 1 1 1 1]\n", 250 | "False\n", 251 | "23 + 104 = 213\n", 252 | "------------\n", 253 | "Error:[ 3.87790083]\n", 254 | "Pred:[1 0 1 1 1 1 1 0]\n", 255 | "True:[1 1 0 1 0 1 1 0]\n", 256 | "False\n", 257 | "124 + 90 = 190\n", 258 | "------------\n", 259 | "Error:[ 3.8993887]\n", 260 | "Pred:[0 0 0 0 0 0 0 0]\n", 261 | "True:[0 1 0 1 0 1 0 1]\n", 262 | "False\n", 263 | "75 + 10 = 0\n", 264 | "------------\n", 265 | "Error:[ 4.00422894]\n", 266 | "Pred:[0 0 0 0 0 0 1 0]\n", 267 | "True:[0 0 0 0 0 1 0 1]\n", 268 | "False\n", 269 | "4 + 1 = 2\n", 270 | "------------\n", 271 | "Error:[ 3.97174173]\n", 272 | "Pred:[0 1 1 0 0 1 1 0]\n", 273 | "True:[1 0 0 0 0 0 1 0]\n", 274 | "False\n", 275 | "15 + 115 = 102\n", 276 | "------------\n", 277 | "Error:[ 4.08583973]\n", 278 | "Pred:[0 0 0 0 0 0 0 0]\n", 279 | "True:[0 1 1 0 1 1 0 0]\n", 280 | "False\n", 281 | "95 + 13 = 0\n", 282 | "------------\n", 283 | "Error:[ 4.20674656]\n", 284 | "Pred:[0 0 0 0 0 0 0 0]\n", 285 | "True:[0 1 1 1 0 1 1 1]\n", 286 | "False\n", 287 | "28 + 91 = 0\n", 288 | "------------\n", 289 | "Error:[ 4.25291382]\n", 290 | "Pred:[0 0 0 0 0 0 0 0]\n", 291 | "True:[0 1 1 1 1 1 1 0]\n", 292 | "False\n", 293 | "88 + 38 = 0\n", 294 | "------------\n", 295 | "Error:[ 3.53099877]\n", 296 | "Pred:[0 0 0 0 0 0 0 0]\n", 297 | "True:[0 1 0 0 1 0 0 0]\n", 298 | "False\n", 299 | "40 + 32 = 0\n", 300 | "------------\n", 301 | "Error:[ 3.89231997]\n", 302 | "Pred:[1 1 1 1 1 1 1 1]\n", 303 | "True:[1 0 1 1 0 1 0 1]\n", 304 | "False\n", 305 | "79 + 102 = 255\n", 306 | "------------\n", 307 | "Error:[ 4.02295019]\n", 308 | "Pred:[0 1 0 0 1 1 1 1]\n", 309 | "True:[0 1 1 1 0 0 1 0]\n", 310 | "False\n", 311 | "75 + 39 = 79\n", 312 | "------------\n", 313 | "Error:[ 3.880903]\n", 314 | "Pred:[1 1 1 1 1 1 0 1]\n", 315 | "True:[0 1 1 0 1 1 0 1]\n", 316 | "False\n", 317 | "105 + 4 = 253\n", 318 | "------------\n", 319 | "Error:[ 3.92571431]\n", 320 | "Pred:[1 1 1 1 1 1 1 1]\n", 321 | "True:[1 0 0 1 1 0 1 1]\n", 322 | "False\n", 323 | "32 + 123 = 255\n", 324 | "------------\n", 325 | "Error:[ 3.7589035]\n", 326 | "Pred:[0 1 0 0 1 0 1 0]\n", 327 | "True:[1 0 0 0 1 1 1 0]\n", 328 | "False\n", 329 | "39 + 103 = 74\n", 330 | "------------\n", 331 | "Error:[ 3.87579648]\n", 332 | "Pred:[1 1 1 1 1 0 1 0]\n", 333 | "True:[1 0 1 0 1 0 1 1]\n", 334 | "False\n", 335 | "58 + 113 = 250\n", 336 | "------------\n", 337 | "Error:[ 4.19142604]\n", 338 | "Pred:[0 0 0 0 1 1 0 0]\n", 339 | "True:[0 1 1 1 0 1 0 1]\n", 340 | "False\n", 341 | "31 + 86 = 12\n", 342 | "------------\n", 343 | "Error:[ 3.86476286]\n", 344 | "Pred:[0 0 1 1 0 0 1 0]\n", 345 | "True:[0 0 1 1 1 1 1 0]\n", 346 | "False\n", 347 | "5 + 57 = 50\n", 348 | "------------\n", 349 | "Error:[ 4.05545722]\n", 350 | "Pred:[1 1 0 1 1 1 1 1]\n", 351 | "True:[1 0 0 1 0 1 0 1]\n", 352 | "False\n", 353 | "38 + 111 = 223\n", 354 | "------------\n", 355 | "Error:[ 4.11675483]\n", 356 | "Pred:[1 1 1 1 1 1 1 1]\n", 357 | "True:[1 0 1 0 0 0 0 1]\n", 358 | "False\n", 359 | "95 + 66 = 255\n", 360 | "------------\n", 361 | "Error:[ 3.90005213]\n", 362 | "Pred:[1 0 0 0 0 0 1 1]\n", 363 | "True:[1 1 1 0 1 1 1 1]\n", 364 | "False\n", 365 | "120 + 119 = 131\n", 366 | "------------\n", 367 | "Error:[ 4.16466363]\n", 368 | "Pred:[0 0 1 1 0 0 1 1]\n", 369 | "True:[1 0 0 1 0 1 1 0]\n", 370 | "False\n", 371 | "121 + 29 = 51\n", 372 | "------------\n", 373 | "Error:[ 3.76799453]\n", 374 | "Pred:[0 0 0 1 1 0 0 0]\n", 375 | "True:[0 1 1 0 1 0 0 0]\n", 376 | "False\n", 377 | "12 + 92 = 24\n", 378 | "------------\n", 379 | "Error:[ 3.74916647]\n", 380 | "Pred:[1 1 1 1 0 1 1 0]\n", 381 | "True:[0 1 1 1 0 1 1 0]\n", 382 | "False\n", 383 | "10 + 108 = 246\n", 384 | "------------\n", 385 | "Error:[ 4.19330056]\n", 386 | "Pred:[0 1 1 1 1 1 1 0]\n", 387 | "True:[1 0 0 0 0 1 1 0]\n", 388 | "False\n", 389 | "72 + 62 = 126\n", 390 | "------------\n", 391 | "Error:[ 3.63455126]\n", 392 | "Pred:[1 1 0 1 1 1 1 0]\n", 393 | "True:[0 1 0 1 0 1 1 0]\n", 394 | "False\n", 395 | "72 + 14 = 222\n", 396 | "------------\n", 397 | "Error:[ 3.79994751]\n", 398 | "Pred:[1 1 0 0 1 1 1 0]\n", 399 | "True:[1 1 0 0 1 1 0 0]\n", 400 | "False\n", 401 | "97 + 107 = 206\n", 402 | "------------\n", 403 | "Error:[ 3.62919201]\n", 404 | "Pred:[1 0 0 0 1 0 1 0]\n", 405 | "True:[1 0 1 1 1 0 1 0]\n", 406 | "False\n", 407 | "112 + 74 = 138\n", 408 | "------------\n", 409 | "Error:[ 3.73175955]\n", 410 | "Pred:[0 0 1 0 0 0 0 0]\n", 411 | "True:[0 1 0 1 0 1 1 0]\n", 412 | "False\n", 413 | "28 + 58 = 32\n", 414 | "------------\n", 415 | "Error:[ 3.37270867]\n", 416 | "Pred:[0 1 0 1 1 0 1 1]\n", 417 | "True:[0 1 0 1 0 0 1 1]\n", 418 | "False\n", 419 | "12 + 71 = 91\n", 420 | "------------\n", 421 | "Error:[ 3.42578141]\n", 422 | "Pred:[1 0 0 1 1 1 1 1]\n", 423 | "True:[1 0 0 1 1 1 1 1]\n", 424 | "True\n", 425 | "93 + 66 = 159\n", 426 | "------------\n", 427 | "Error:[ 3.96962954]\n", 428 | "Pred:[0 0 1 1 0 1 1 0]\n", 429 | "True:[0 1 0 0 1 0 0 0]\n", 430 | "False\n", 431 | "63 + 9 = 54\n", 432 | "------------\n", 433 | "Error:[ 3.20748485]\n", 434 | "Pred:[1 1 1 0 1 1 1 1]\n", 435 | "True:[0 1 1 0 1 1 1 1]\n", 436 | "False\n", 437 | "15 + 96 = 239\n", 438 | "------------\n", 439 | "Error:[ 3.07873713]\n", 440 | "Pred:[0 0 1 1 1 1 1 1]\n", 441 | "True:[0 0 1 0 1 1 1 1]\n", 442 | "False\n", 443 | "33 + 14 = 63\n", 444 | "------------\n", 445 | "Error:[ 3.27911476]\n", 446 | "Pred:[0 1 1 0 0 0 1 1]\n", 447 | "True:[0 1 0 1 0 0 1 1]\n", 448 | "False\n", 449 | "24 + 59 = 99\n", 450 | "------------\n", 451 | "Error:[ 3.91718754]\n", 452 | "Pred:[1 1 0 0 1 0 1 1]\n", 453 | "True:[1 0 1 1 0 0 0 1]\n", 454 | "False\n", 455 | "52 + 125 = 203\n", 456 | "------------\n", 457 | "Error:[ 2.84676144]\n", 458 | "Pred:[1 1 0 1 1 0 1 0]\n", 459 | "True:[1 1 0 1 1 0 1 0]\n", 460 | "True\n", 461 | "98 + 120 = 218\n", 462 | "------------\n", 463 | "Error:[ 3.56648456]\n", 464 | "Pred:[0 1 0 0 1 1 1 0]\n", 465 | "True:[0 1 0 1 0 0 0 0]\n", 466 | "False\n", 467 | "37 + 43 = 78\n", 468 | "------------\n", 469 | "Error:[ 2.61810298]\n", 470 | "Pred:[1 1 1 0 0 1 1 1]\n", 471 | "True:[1 1 0 0 0 0 1 1]\n", 472 | "False\n", 473 | "97 + 98 = 231\n", 474 | "------------\n", 475 | "Error:[ 2.34280248]\n", 476 | "Pred:[1 0 0 1 1 0 0 0]\n", 477 | "True:[1 1 0 1 1 0 0 0]\n", 478 | "False\n", 479 | "112 + 104 = 152\n", 480 | "------------\n", 481 | "Error:[ 3.83818822]\n", 482 | "Pred:[0 1 1 1 1 0 1 0]\n", 483 | "True:[1 0 0 0 0 0 1 0]\n", 484 | "False\n", 485 | "120 + 10 = 122\n", 486 | "------------\n", 487 | "Error:[ 2.57289539]\n", 488 | "Pred:[0 1 1 0 0 1 1 0]\n", 489 | "True:[0 1 1 0 0 1 1 0]\n", 490 | "True\n", 491 | "16 + 86 = 102\n", 492 | "------------\n", 493 | "Error:[ 3.31327062]\n", 494 | "Pred:[1 1 0 0 1 0 0 0]\n", 495 | "True:[1 0 0 0 1 0 1 0]\n", 496 | "False\n", 497 | "19 + 119 = 200\n", 498 | "------------\n", 499 | "Error:[ 2.69438107]\n", 500 | "Pred:[1 0 0 1 1 0 0 1]\n", 501 | "True:[1 1 0 1 0 0 0 1]\n", 502 | "False\n", 503 | "100 + 109 = 153\n", 504 | "------------\n", 505 | "Error:[ 2.95509023]\n", 506 | "Pred:[1 0 1 1 1 0 0 1]\n", 507 | "True:[1 0 1 1 0 0 0 1]\n", 508 | "False\n", 509 | "111 + 66 = 185\n", 510 | "------------\n", 511 | "Error:[ 2.24368312]\n", 512 | "Pred:[0 1 0 1 1 1 0 1]\n", 513 | "True:[0 1 0 1 1 1 0 1]\n", 514 | "True\n", 515 | "24 + 69 = 93\n", 516 | "------------\n", 517 | "Error:[ 3.63229369]\n", 518 | "Pred:[1 0 1 1 0 1 0 0]\n", 519 | "True:[1 1 0 0 0 1 0 0]\n", 520 | "False\n", 521 | "78 + 118 = 180\n", 522 | "------------\n", 523 | "Error:[ 2.52654005]\n", 524 | "Pred:[1 1 0 0 1 1 1 1]\n", 525 | "True:[1 1 0 0 1 1 1 1]\n", 526 | "True\n", 527 | "110 + 97 = 207\n", 528 | "------------\n", 529 | "Error:[ 2.83996122]\n", 530 | "Pred:[1 1 1 0 1 0 0 1]\n", 531 | "True:[1 1 1 0 1 0 0 1]\n", 532 | "True\n", 533 | "111 + 122 = 233\n", 534 | "------------\n", 535 | "Error:[ 1.07711253]\n", 536 | "Pred:[0 1 1 0 0 0 1 0]\n", 537 | "True:[0 1 1 0 0 0 1 0]\n", 538 | "True\n", 539 | "97 + 1 = 98\n", 540 | "------------\n", 541 | "Error:[ 1.51067066]\n", 542 | "Pred:[1 0 0 1 0 0 0 0]\n", 543 | "True:[1 0 0 1 0 0 0 0]\n", 544 | "True\n", 545 | "72 + 72 = 144\n", 546 | "------------\n", 547 | "Error:[ 1.24360068]\n", 548 | "Pred:[0 1 1 0 1 1 1 0]\n", 549 | "True:[0 1 1 0 1 1 1 0]\n", 550 | "True\n", 551 | "85 + 25 = 110\n", 552 | "------------\n", 553 | "Error:[ 3.40019024]\n", 554 | "Pred:[1 0 1 1 1 1 1 1]\n", 555 | "True:[0 1 1 1 1 1 1 1]\n", 556 | "False\n", 557 | "66 + 61 = 191\n", 558 | "------------\n", 559 | "Error:[ 1.63590052]\n", 560 | "Pred:[1 0 0 0 0 0 1 1]\n", 561 | "True:[1 0 0 0 0 0 1 1]\n", 562 | "True\n", 563 | "98 + 33 = 131\n", 564 | "------------\n", 565 | "Error:[ 2.72216248]\n", 566 | "Pred:[1 0 0 0 0 0 0 0]\n", 567 | "True:[1 0 0 0 1 0 0 0]\n", 568 | "False\n", 569 | "107 + 29 = 128\n", 570 | "------------\n", 571 | "Error:[ 0.53601483]\n", 572 | "Pred:[0 1 0 1 0 1 0 1]\n", 573 | "True:[0 1 0 1 0 1 0 1]\n", 574 | "True\n", 575 | "16 + 69 = 85\n", 576 | "------------\n", 577 | "Error:[ 1.53384966]\n", 578 | "Pred:[1 0 1 0 1 1 1 0]\n", 579 | "True:[1 0 1 0 1 1 1 0]\n", 580 | "True\n", 581 | "89 + 85 = 174\n", 582 | "------------\n", 583 | "Error:[ 1.23589765]\n", 584 | "Pred:[0 1 0 0 1 1 1 0]\n", 585 | "True:[0 1 0 0 1 1 1 0]\n", 586 | "True\n", 587 | "32 + 46 = 78\n", 588 | "------------\n", 589 | "Error:[ 1.52746406]\n", 590 | "Pred:[1 1 0 1 1 0 0 1]\n", 591 | "True:[1 1 0 1 1 0 0 1]\n", 592 | "True\n", 593 | "122 + 95 = 217\n", 594 | "------------\n", 595 | "Error:[ 0.8104674]\n", 596 | "Pred:[0 1 1 1 1 1 1 0]\n", 597 | "True:[0 1 1 1 1 1 1 0]\n", 598 | "True\n", 599 | "62 + 64 = 126\n", 600 | "------------\n", 601 | "Error:[ 1.41268677]\n", 602 | "Pred:[1 0 1 1 1 0 0 0]\n", 603 | "True:[1 0 1 1 1 0 0 0]\n", 604 | "True\n", 605 | "106 + 78 = 184\n", 606 | "------------\n", 607 | "Error:[ 1.58945965]\n", 608 | "Pred:[0 1 1 1 1 0 1 1]\n", 609 | "True:[0 1 1 1 1 0 1 1]\n", 610 | "True\n", 611 | "95 + 28 = 123\n", 612 | "------------\n", 613 | "Error:[ 1.57149055]\n", 614 | "Pred:[1 0 1 1 0 1 1 1]\n", 615 | "True:[1 0 1 1 0 1 1 1]\n", 616 | "True\n", 617 | "109 + 74 = 183\n", 618 | "------------\n", 619 | "Error:[ 0.99503431]\n", 620 | "Pred:[1 1 1 1 0 1 1 1]\n", 621 | "True:[1 1 1 1 0 1 1 1]\n", 622 | "True\n", 623 | "122 + 125 = 247\n", 624 | "------------\n", 625 | "Error:[ 1.23742639]\n", 626 | "Pred:[1 1 0 0 0 1 1 1]\n", 627 | "True:[1 1 0 0 0 1 1 1]\n", 628 | "True\n", 629 | "79 + 120 = 199\n", 630 | "------------\n", 631 | "Error:[ 1.00171615]\n", 632 | "Pred:[1 0 1 0 0 1 1 1]\n", 633 | "True:[1 0 1 0 0 1 1 1]\n", 634 | "True\n", 635 | "42 + 125 = 167\n", 636 | "------------\n", 637 | "Error:[ 0.88834079]\n", 638 | "Pred:[1 0 0 1 0 1 1 0]\n", 639 | "True:[1 0 0 1 0 1 1 0]\n", 640 | "True\n", 641 | "126 + 24 = 150\n", 642 | "------------\n", 643 | "Error:[ 0.62501817]\n", 644 | "Pred:[1 1 0 0 0 0 0 1]\n", 645 | "True:[1 1 0 0 0 0 0 1]\n", 646 | "True\n", 647 | "112 + 81 = 193\n", 648 | "------------\n", 649 | "Error:[ 0.68401851]\n", 650 | "Pred:[0 1 0 1 0 1 0 1]\n", 651 | "True:[0 1 0 1 0 1 0 1]\n", 652 | "True\n", 653 | "71 + 14 = 85\n", 654 | "------------\n", 655 | "Error:[ 0.35307906]\n", 656 | "Pred:[0 1 0 1 0 1 1 0]\n", 657 | "True:[0 1 0 1 0 1 1 0]\n", 658 | "True\n", 659 | "82 + 4 = 86\n", 660 | "------------\n", 661 | "Error:[ 1.27283888]\n", 662 | "Pred:[1 0 0 0 0 0 0 1]\n", 663 | "True:[1 0 0 0 0 0 0 1]\n", 664 | "True\n", 665 | "30 + 99 = 129\n", 666 | "------------\n", 667 | "Error:[ 0.59094215]\n", 668 | "Pred:[0 1 0 1 0 0 1 0]\n", 669 | "True:[0 1 0 1 0 0 1 0]\n", 670 | "True\n", 671 | "78 + 4 = 82\n", 672 | "------------\n", 673 | "Error:[ 0.58736554]\n", 674 | "Pred:[1 0 0 1 1 1 0 1]\n", 675 | "True:[1 0 0 1 1 1 0 1]\n", 676 | "True\n", 677 | "82 + 75 = 157\n", 678 | "------------\n", 679 | "Error:[ 0.38656398]\n", 680 | "Pred:[1 0 0 0 1 1 0 1]\n", 681 | "True:[1 0 0 0 1 1 0 1]\n", 682 | "True\n", 683 | "77 + 64 = 141\n", 684 | "------------\n", 685 | "Error:[ 0.43420519]\n", 686 | "Pred:[0 0 1 1 0 0 1 1]\n", 687 | "True:[0 0 1 1 0 0 1 1]\n", 688 | "True\n", 689 | "26 + 25 = 51\n", 690 | "------------\n", 691 | "Error:[ 0.50252171]\n", 692 | "Pred:[1 0 1 0 0 1 1 0]\n", 693 | "True:[1 0 1 0 0 1 1 0]\n", 694 | "True\n", 695 | "80 + 86 = 166\n", 696 | "------------\n", 697 | "Error:[ 0.66292361]\n", 698 | "Pred:[1 0 0 1 1 0 0 1]\n", 699 | "True:[1 0 0 1 1 0 0 1]\n", 700 | "True\n", 701 | "42 + 111 = 153\n", 702 | "------------\n", 703 | "Error:[ 0.61874068]\n", 704 | "Pred:[1 1 0 0 1 1 0 0]\n", 705 | "True:[1 1 0 0 1 1 0 0]\n", 706 | "True\n", 707 | "117 + 87 = 204\n", 708 | "------------\n", 709 | "Error:[ 0.39324392]\n", 710 | "Pred:[1 0 0 0 1 1 1 0]\n", 711 | "True:[1 0 0 0 1 1 1 0]\n", 712 | "True\n", 713 | "65 + 77 = 142\n", 714 | "------------\n", 715 | "Error:[ 0.45513889]\n", 716 | "Pred:[0 1 0 1 0 0 1 1]\n", 717 | "True:[0 1 0 1 0 0 1 1]\n", 718 | "True\n", 719 | "70 + 13 = 83\n", 720 | "------------\n", 721 | "Error:[ 0.27921542]\n", 722 | "Pred:[0 1 0 1 1 0 1 1]\n", 723 | "True:[0 1 0 1 1 0 1 1]\n", 724 | "True\n", 725 | "90 + 1 = 91\n", 726 | "------------\n", 727 | "Error:[ 0.62198489]\n", 728 | "Pred:[1 1 1 1 0 1 0 1]\n", 729 | "True:[1 1 1 1 0 1 0 1]\n", 730 | "True\n", 731 | "127 + 118 = 245\n", 732 | "------------\n", 733 | "Error:[ 0.50229076]\n", 734 | "Pred:[1 0 0 1 0 1 0 1]\n", 735 | "True:[1 0 0 1 0 1 0 1]\n", 736 | "True\n", 737 | "57 + 92 = 149\n", 738 | "------------\n", 739 | "Error:[ 0.58217015]\n", 740 | "Pred:[0 1 1 1 0 0 0 0]\n", 741 | "True:[0 1 1 1 0 0 0 0]\n", 742 | "True\n", 743 | "21 + 91 = 112\n", 744 | "------------\n", 745 | "Error:[ 0.50710971]\n", 746 | "Pred:[1 0 1 1 1 0 1 0]\n", 747 | "True:[1 0 1 1 1 0 1 0]\n", 748 | "True\n", 749 | "115 + 71 = 186\n", 750 | "------------\n", 751 | "Error:[ 0.51719454]\n", 752 | "Pred:[0 1 1 1 0 0 0 1]\n", 753 | "True:[0 1 1 1 0 0 0 1]\n", 754 | "True\n", 755 | "18 + 95 = 113\n", 756 | "------------\n", 757 | "Error:[ 0.47046102]\n", 758 | "Pred:[1 1 1 0 1 1 1 0]\n", 759 | "True:[1 1 1 0 1 1 1 0]\n", 760 | "True\n", 761 | "116 + 122 = 238\n", 762 | "------------\n", 763 | "Error:[ 0.51538252]\n", 764 | "Pred:[0 1 0 1 0 0 0 1]\n", 765 | "True:[0 1 0 1 0 0 0 1]\n", 766 | "True\n", 767 | "47 + 34 = 81\n", 768 | "------------\n", 769 | "Error:[ 0.48485176]\n", 770 | "Pred:[1 0 1 1 0 0 0 1]\n", 771 | "True:[1 0 1 1 0 0 0 1]\n", 772 | "True\n", 773 | "73 + 104 = 177\n", 774 | "------------\n", 775 | "Error:[ 0.45145853]\n", 776 | "Pred:[1 0 1 0 1 0 1 0]\n", 777 | "True:[1 0 1 0 1 0 1 0]\n", 778 | "True\n", 779 | "77 + 93 = 170\n", 780 | "------------\n", 781 | "Error:[ 0.38580434]\n", 782 | "Pred:[1 0 0 1 1 1 0 1]\n", 783 | "True:[1 0 0 1 1 1 0 1]\n", 784 | "True\n", 785 | "116 + 41 = 157\n", 786 | "------------\n", 787 | "Error:[ 0.25024716]\n", 788 | "Pred:[1 0 0 1 0 0 0 1]\n", 789 | "True:[1 0 0 1 0 0 0 1]\n", 790 | "True\n", 791 | "80 + 65 = 145\n", 792 | "------------\n", 793 | "Error:[ 0.18765659]\n", 794 | "Pred:[0 0 1 0 1 0 1 1]\n", 795 | "True:[0 0 1 0 1 0 1 1]\n", 796 | "True\n", 797 | "2 + 41 = 43\n", 798 | "------------\n", 799 | "Error:[ 0.48164219]\n", 800 | "Pred:[1 0 1 1 1 0 1 1]\n", 801 | "True:[1 0 1 1 1 0 1 1]\n", 802 | "True\n", 803 | "124 + 63 = 187\n", 804 | "------------\n", 805 | "Error:[ 0.52533063]\n", 806 | "Pred:[1 0 0 1 1 0 1 1]\n", 807 | "True:[1 0 0 1 1 0 1 1]\n", 808 | "True\n", 809 | "52 + 103 = 155\n", 810 | "------------\n", 811 | "Error:[ 0.14392675]\n", 812 | "Pred:[0 1 0 1 0 0 1 0]\n", 813 | "True:[0 1 0 1 0 0 1 0]\n", 814 | "True\n", 815 | "81 + 1 = 82\n", 816 | "------------\n", 817 | "Error:[ 0.2536919]\n", 818 | "Pred:[0 0 0 1 1 0 0 1]\n", 819 | "True:[0 0 0 1 1 0 0 1]\n", 820 | "True\n", 821 | "20 + 5 = 25\n", 822 | "------------\n", 823 | "Error:[ 0.4577056]\n", 824 | "Pred:[1 0 1 0 0 0 0 1]\n", 825 | "True:[1 0 1 0 0 0 0 1]\n", 826 | "True\n", 827 | "90 + 71 = 161\n", 828 | "------------\n", 829 | "Error:[ 0.35454413]\n", 830 | "Pred:[0 1 1 0 0 0 0 1]\n", 831 | "True:[0 1 1 0 0 0 0 1]\n", 832 | "True\n", 833 | "72 + 25 = 97\n", 834 | "------------\n", 835 | "Error:[ 0.3027905]\n", 836 | "Pred:[0 1 0 1 0 0 0 0]\n", 837 | "True:[0 1 0 1 0 0 0 0]\n", 838 | "True\n", 839 | "11 + 69 = 80\n", 840 | "------------\n", 841 | "Error:[ 0.32028173]\n", 842 | "Pred:[0 1 0 0 0 1 0 0]\n", 843 | "True:[0 1 0 0 0 1 0 0]\n", 844 | "True\n", 845 | "49 + 19 = 68\n", 846 | "------------\n", 847 | "Error:[ 0.47741899]\n", 848 | "Pred:[1 0 1 1 0 0 1 1]\n", 849 | "True:[1 0 1 1 0 0 1 1]\n", 850 | "True\n", 851 | "70 + 109 = 179\n", 852 | "------------\n", 853 | "Error:[ 0.19099271]\n", 854 | "Pred:[0 1 0 1 1 0 1 1]\n", 855 | "True:[0 1 0 1 1 0 1 1]\n", 856 | "True\n", 857 | "18 + 73 = 91\n", 858 | "------------\n", 859 | "Error:[ 0.44434423]\n", 860 | "Pred:[1 1 0 1 0 0 1 1]\n", 861 | "True:[1 1 0 1 0 0 1 1]\n", 862 | "True\n", 863 | "111 + 100 = 211\n", 864 | "------------\n", 865 | "Error:[ 0.34579419]\n", 866 | "Pred:[1 0 0 1 1 0 0 0]\n", 867 | "True:[1 0 0 1 1 0 0 0]\n", 868 | "True\n", 869 | "73 + 79 = 152\n", 870 | "------------\n", 871 | "Error:[ 0.16166863]\n", 872 | "Pred:[0 1 1 1 0 1 1 0]\n", 873 | "True:[0 1 1 1 0 1 1 0]\n", 874 | "True\n", 875 | "113 + 5 = 118\n", 876 | "------------\n", 877 | "Error:[ 0.18488997]\n", 878 | "Pred:[0 0 1 1 0 1 1 1]\n", 879 | "True:[0 0 1 1 0 1 1 1]\n", 880 | "True\n", 881 | "2 + 53 = 55\n", 882 | "------------\n", 883 | "Error:[ 0.39762466]\n", 884 | "Pred:[1 0 0 1 0 1 1 0]\n", 885 | "True:[1 0 0 1 0 1 1 0]\n", 886 | "True\n", 887 | "127 + 23 = 150\n", 888 | "------------\n", 889 | "Error:[ 0.30393507]\n", 890 | "Pred:[0 1 1 0 1 1 1 1]\n", 891 | "True:[0 1 1 0 1 1 1 1]\n", 892 | "True\n", 893 | "97 + 14 = 111\n", 894 | "------------\n", 895 | "Error:[ 0.43896228]\n", 896 | "Pred:[1 0 1 0 1 1 0 0]\n", 897 | "True:[1 0 1 0 1 1 0 0]\n", 898 | "True\n", 899 | "78 + 94 = 172\n", 900 | "------------\n", 901 | "Error:[ 0.19477698]\n", 902 | "Pred:[0 1 1 0 0 1 1 0]\n", 903 | "True:[0 1 1 0 0 1 1 0]\n", 904 | "True\n", 905 | "36 + 66 = 102\n", 906 | "------------\n", 907 | "Error:[ 0.384797]\n", 908 | "Pred:[1 0 1 1 0 1 0 0]\n", 909 | "True:[1 0 1 1 0 1 0 0]\n", 910 | "True\n", 911 | "63 + 117 = 180\n", 912 | "------------\n", 913 | "Error:[ 0.24173037]\n", 914 | "Pred:[0 1 1 1 1 1 1 1]\n", 915 | "True:[0 1 1 1 1 1 1 1]\n", 916 | "True\n", 917 | "68 + 59 = 127\n", 918 | "------------\n", 919 | "Error:[ 0.40062057]\n", 920 | "Pred:[0 1 0 1 0 1 0 1]\n", 921 | "True:[0 1 0 1 0 1 0 1]\n", 922 | "True\n", 923 | "27 + 58 = 85\n", 924 | "------------\n", 925 | "Error:[ 0.4056001]\n", 926 | "Pred:[0 1 0 0 0 1 0 1]\n", 927 | "True:[0 1 0 0 0 1 0 1]\n", 928 | "True\n", 929 | "63 + 6 = 69\n", 930 | "------------\n", 931 | "Error:[ 0.41176447]\n", 932 | "Pred:[1 0 0 1 1 1 1 1]\n", 933 | "True:[1 0 0 1 1 1 1 1]\n", 934 | "True\n", 935 | "127 + 32 = 159\n", 936 | "------------\n", 937 | "Error:[ 0.45722834]\n", 938 | "Pred:[1 1 0 0 1 0 0 0]\n", 939 | "True:[1 1 0 0 1 0 0 0]\n", 940 | "True\n", 941 | "106 + 94 = 200\n", 942 | "------------\n", 943 | "Error:[ 0.26925754]\n", 944 | "Pred:[0 1 1 0 1 1 1 0]\n", 945 | "True:[0 1 1 0 1 1 1 0]\n", 946 | "True\n", 947 | "80 + 30 = 110\n", 948 | "------------\n", 949 | "Error:[ 0.18420419]\n", 950 | "Pred:[0 0 0 1 1 1 0 0]\n", 951 | "True:[0 0 0 1 1 1 0 0]\n", 952 | "True\n", 953 | "23 + 5 = 28\n", 954 | "------------\n", 955 | "Error:[ 0.42250863]\n", 956 | "Pred:[1 1 0 1 0 0 1 0]\n", 957 | "True:[1 1 0 1 0 0 1 0]\n", 958 | "True\n", 959 | "102 + 108 = 210\n", 960 | "------------\n", 961 | "Error:[ 0.15972739]\n", 962 | "Pred:[0 0 1 1 1 0 1 0]\n", 963 | "True:[0 0 1 1 1 0 1 0]\n", 964 | "True\n", 965 | "18 + 40 = 58\n", 966 | "------------\n", 967 | "Error:[ 0.36053754]\n", 968 | "Pred:[1 0 0 1 0 1 0 0]\n", 969 | "True:[1 0 0 1 0 1 0 0]\n", 970 | "True\n", 971 | "95 + 53 = 148\n", 972 | "------------\n", 973 | "Error:[ 0.12271426]\n", 974 | "Pred:[0 0 0 0 1 1 0 1]\n", 975 | "True:[0 0 0 0 1 1 0 1]\n", 976 | "True\n", 977 | "5 + 8 = 13\n", 978 | "------------\n", 979 | "Error:[ 0.29394636]\n", 980 | "Pred:[0 1 1 0 0 1 1 0]\n", 981 | "True:[0 1 1 0 0 1 1 0]\n", 982 | "True\n", 983 | "15 + 87 = 102\n", 984 | "------------\n", 985 | "Error:[ 0.43435847]\n", 986 | "Pred:[1 0 1 0 0 0 0 0]\n", 987 | "True:[1 0 1 0 0 0 0 0]\n", 988 | "True\n", 989 | "101 + 59 = 160\n", 990 | "------------\n", 991 | "Error:[ 0.25130702]\n", 992 | "Pred:[0 0 1 0 0 1 0 0]\n", 993 | "True:[0 0 1 0 0 1 0 0]\n", 994 | "True\n", 995 | "31 + 5 = 36\n", 996 | "------------\n", 997 | "Error:[ 0.37297311]\n", 998 | "Pred:[0 1 0 1 0 0 1 0]\n", 999 | "True:[0 1 0 1 0 0 1 0]\n", 1000 | "True\n", 1001 | "54 + 28 = 82\n", 1002 | "------------\n", 1003 | "Error:[ 0.23398697]\n", 1004 | "Pred:[0 0 0 1 1 0 1 0]\n", 1005 | "True:[0 0 0 1 1 0 1 0]\n", 1006 | "True\n", 1007 | "12 + 14 = 26\n", 1008 | "------------\n", 1009 | "Error:[ 0.33698497]\n", 1010 | "Pred:[0 1 1 1 1 0 1 0]\n", 1011 | "True:[0 1 1 1 1 0 1 0]\n", 1012 | "True\n", 1013 | "110 + 12 = 122\n", 1014 | "------------\n", 1015 | "Error:[ 0.32293045]\n", 1016 | "Pred:[0 1 0 1 0 0 0 0]\n", 1017 | "True:[0 1 0 1 0 0 0 0]\n", 1018 | "True\n", 1019 | "74 + 6 = 80\n", 1020 | "------------\n", 1021 | "Error:[ 0.29383664]\n", 1022 | "Pred:[0 1 0 0 1 0 0 1]\n", 1023 | "True:[0 1 0 0 1 0 0 1]\n", 1024 | "True\n", 1025 | "28 + 45 = 73\n", 1026 | "------------\n", 1027 | "Error:[ 0.29335243]\n", 1028 | "Pred:[1 0 0 1 1 0 1 0]\n", 1029 | "True:[1 0 0 1 1 0 1 0]\n", 1030 | "True\n", 1031 | "53 + 101 = 154\n", 1032 | "------------\n", 1033 | "Error:[ 0.37123225]\n", 1034 | "Pred:[1 1 1 0 1 1 0 0]\n", 1035 | "True:[1 1 1 0 1 1 0 0]\n", 1036 | "True\n", 1037 | "126 + 110 = 236\n", 1038 | "------------\n", 1039 | "Error:[ 0.26784038]\n", 1040 | "Pred:[1 1 0 1 1 1 1 0]\n", 1041 | "True:[1 1 0 1 1 1 1 0]\n", 1042 | "True\n", 1043 | "122 + 100 = 222\n", 1044 | "------------\n", 1045 | "Error:[ 0.34925971]\n", 1046 | "Pred:[0 0 1 1 0 1 1 1]\n", 1047 | "True:[0 0 1 1 0 1 1 1]\n", 1048 | "True\n", 1049 | "15 + 40 = 55\n", 1050 | "------------\n", 1051 | "Error:[ 0.2196544]\n", 1052 | "Pred:[0 1 1 0 1 1 1 0]\n", 1053 | "True:[0 1 1 0 1 1 1 0]\n", 1054 | "True\n", 1055 | "25 + 85 = 110\n", 1056 | "------------\n", 1057 | "Error:[ 0.14428875]\n", 1058 | "Pred:[0 1 0 0 1 0 1 0]\n", 1059 | "True:[0 1 0 0 1 0 1 0]\n", 1060 | "True\n", 1061 | "7 + 67 = 74\n", 1062 | "------------\n", 1063 | "Error:[ 0.30459088]\n", 1064 | "Pred:[0 1 0 0 0 0 0 0]\n", 1065 | "True:[0 1 0 0 0 0 0 0]\n", 1066 | "True\n", 1067 | "63 + 1 = 64\n", 1068 | "------------\n", 1069 | "Error:[ 0.1506828]\n", 1070 | "Pred:[0 1 1 1 1 0 1 1]\n", 1071 | "True:[0 1 1 1 1 0 1 1]\n", 1072 | "True\n", 1073 | "64 + 59 = 123\n", 1074 | "------------\n", 1075 | "Error:[ 0.27649987]\n", 1076 | "Pred:[0 1 0 1 1 1 0 0]\n", 1077 | "True:[0 1 0 1 1 1 0 0]\n", 1078 | "True\n", 1079 | "50 + 42 = 92\n", 1080 | "------------\n", 1081 | "Error:[ 0.27844325]\n", 1082 | "Pred:[1 1 0 0 0 0 1 0]\n", 1083 | "True:[1 1 0 0 0 0 1 0]\n", 1084 | "True\n", 1085 | "77 + 117 = 194\n", 1086 | "------------\n", 1087 | "Error:[ 0.29936468]\n", 1088 | "Pred:[0 1 0 1 0 0 0 0]\n", 1089 | "True:[0 1 0 1 0 0 0 0]\n", 1090 | "True\n", 1091 | "33 + 47 = 80\n", 1092 | "------------\n", 1093 | "Error:[ 0.17910203]\n", 1094 | "Pred:[1 0 0 1 1 0 1 1]\n", 1095 | "True:[1 0 0 1 1 0 1 1]\n", 1096 | "True\n", 1097 | "82 + 73 = 155\n", 1098 | "------------\n", 1099 | "Error:[ 0.19257083]\n", 1100 | "Pred:[1 0 1 1 1 1 0 1]\n", 1101 | "True:[1 0 1 1 1 1 0 1]\n", 1102 | "True\n", 1103 | "105 + 84 = 189\n", 1104 | "------------\n", 1105 | "Error:[ 0.12016148]\n", 1106 | "Pred:[0 1 0 0 1 1 1 0]\n", 1107 | "True:[0 1 0 0 1 1 1 0]\n", 1108 | "True\n", 1109 | "11 + 67 = 78\n", 1110 | "------------\n", 1111 | "Error:[ 0.26722366]\n", 1112 | "Pred:[1 0 1 0 1 1 1 0]\n", 1113 | "True:[1 0 1 0 1 1 1 0]\n", 1114 | "True\n", 1115 | "112 + 62 = 174\n", 1116 | "------------\n", 1117 | "Error:[ 0.28009711]\n", 1118 | "Pred:[1 0 1 0 0 1 0 1]\n", 1119 | "True:[1 0 1 0 0 1 0 1]\n", 1120 | "True\n", 1121 | "108 + 57 = 165\n", 1122 | "------------\n", 1123 | "Error:[ 0.22371447]\n", 1124 | "Pred:[0 1 1 1 0 0 1 0]\n", 1125 | "True:[0 1 1 1 0 0 1 0]\n", 1126 | "True\n", 1127 | "15 + 99 = 114\n", 1128 | "------------\n", 1129 | "Error:[ 0.27980071]\n", 1130 | "Pred:[1 0 1 0 0 1 1 1]\n", 1131 | "True:[1 0 1 0 0 1 1 1]\n", 1132 | "True\n", 1133 | "113 + 54 = 167\n", 1134 | "------------\n", 1135 | "Error:[ 0.22918159]\n", 1136 | "Pred:[0 0 0 1 1 1 1 1]\n", 1137 | "True:[0 0 0 1 1 1 1 1]\n", 1138 | "True\n", 1139 | "1 + 30 = 31\n", 1140 | "------------\n", 1141 | "Error:[ 0.31937504]\n", 1142 | "Pred:[1 0 1 1 0 0 1 1]\n", 1143 | "True:[1 0 1 1 0 0 1 1]\n", 1144 | "True\n", 1145 | "102 + 77 = 179\n", 1146 | "------------\n", 1147 | "Error:[ 0.22852688]\n", 1148 | "Pred:[1 1 1 0 0 1 1 0]\n", 1149 | "True:[1 1 1 0 0 1 1 0]\n", 1150 | "True\n", 1151 | "112 + 118 = 230\n", 1152 | "------------\n", 1153 | "Error:[ 0.19959135]\n", 1154 | "Pred:[1 1 0 1 0 0 1 0]\n", 1155 | "True:[1 1 0 1 0 0 1 0]\n", 1156 | "True\n", 1157 | "121 + 89 = 210\n", 1158 | "------------\n", 1159 | "Error:[ 0.12430546]\n", 1160 | "Pred:[0 1 0 0 1 1 0 0]\n", 1161 | "True:[0 1 0 0 1 1 0 0]\n", 1162 | "True\n", 1163 | "4 + 72 = 76\n", 1164 | "------------\n", 1165 | "Error:[ 0.22408818]\n", 1166 | "Pred:[0 1 0 0 0 1 1 0]\n", 1167 | "True:[0 1 0 0 0 1 1 0]\n", 1168 | "True\n", 1169 | "48 + 22 = 70\n", 1170 | "------------\n", 1171 | "Error:[ 0.20221729]\n", 1172 | "Pred:[1 0 1 0 0 1 1 0]\n", 1173 | "True:[1 0 1 0 0 1 1 0]\n", 1174 | "True\n", 1175 | "115 + 51 = 166\n", 1176 | "------------\n", 1177 | "Error:[ 0.17104793]\n", 1178 | "Pred:[1 0 0 0 1 0 1 0]\n", 1179 | "True:[1 0 0 0 1 0 1 0]\n", 1180 | "True\n", 1181 | "71 + 67 = 138\n", 1182 | "------------\n", 1183 | "Error:[ 0.29566632]\n", 1184 | "Pred:[1 0 0 0 0 1 1 1]\n", 1185 | "True:[1 0 0 0 0 1 1 1]\n", 1186 | "True\n", 1187 | "125 + 10 = 135\n", 1188 | "------------\n", 1189 | "Error:[ 0.26591579]\n", 1190 | "Pred:[1 0 1 0 1 0 0 1]\n", 1191 | "True:[1 0 1 0 1 0 0 1]\n", 1192 | "True\n", 1193 | "95 + 74 = 169\n", 1194 | "------------\n", 1195 | "Error:[ 0.23643373]\n", 1196 | "Pred:[0 0 1 0 0 1 1 0]\n", 1197 | "True:[0 0 1 0 0 1 1 0]\n", 1198 | "True\n", 1199 | "28 + 10 = 38\n", 1200 | "------------\n", 1201 | "Error:[ 0.29028257]\n", 1202 | "Pred:[1 0 1 1 0 0 1 0]\n", 1203 | "True:[1 0 1 1 0 0 1 0]\n", 1204 | "True\n", 1205 | "55 + 123 = 178\n", 1206 | "------------\n", 1207 | "Error:[ 0.23127843]\n", 1208 | "Pred:[1 1 0 0 1 1 0 1]\n", 1209 | "True:[1 1 0 0 1 1 0 1]\n", 1210 | "True\n", 1211 | "88 + 117 = 205\n", 1212 | "------------\n", 1213 | "Error:[ 0.21379392]\n", 1214 | "Pred:[1 0 0 0 0 1 1 1]\n", 1215 | "True:[1 0 0 0 0 1 1 1]\n", 1216 | "True\n", 1217 | "98 + 37 = 135\n", 1218 | "------------\n", 1219 | "Error:[ 0.28917566]\n", 1220 | "Pred:[1 0 1 0 1 0 0 1]\n", 1221 | "True:[1 0 1 0 1 0 0 1]\n", 1222 | "True\n", 1223 | "111 + 58 = 169\n", 1224 | "------------\n", 1225 | "Error:[ 0.25232534]\n", 1226 | "Pred:[1 1 0 1 1 1 0 0]\n", 1227 | "True:[1 1 0 1 1 1 0 0]\n", 1228 | "True\n", 1229 | "95 + 125 = 220\n", 1230 | "------------\n", 1231 | "Error:[ 0.23058793]\n", 1232 | "Pred:[1 0 1 0 1 0 1 0]\n", 1233 | "True:[1 0 1 0 1 0 1 0]\n", 1234 | "True\n", 1235 | "88 + 82 = 170\n", 1236 | "------------\n", 1237 | "Error:[ 0.30600955]\n", 1238 | "Pred:[1 0 0 0 0 0 0 1]\n", 1239 | "True:[1 0 0 0 0 0 0 1]\n", 1240 | "True\n", 1241 | "99 + 30 = 129\n", 1242 | "------------\n", 1243 | "Error:[ 0.24408132]\n", 1244 | "Pred:[0 1 0 1 1 0 0 0]\n", 1245 | "True:[0 1 0 1 1 0 0 0]\n", 1246 | "True\n", 1247 | "52 + 36 = 88\n", 1248 | "------------\n", 1249 | "Error:[ 0.26199122]\n", 1250 | "Pred:[1 0 0 1 1 0 1 0]\n", 1251 | "True:[1 0 0 1 1 0 1 0]\n", 1252 | "True\n", 1253 | "127 + 27 = 154\n", 1254 | "------------\n", 1255 | "Error:[ 0.2308597]\n", 1256 | "Pred:[1 1 0 1 1 1 0 0]\n", 1257 | "True:[1 1 0 1 1 1 0 0]\n", 1258 | "True\n", 1259 | "106 + 114 = 220\n", 1260 | "------------\n", 1261 | "Error:[ 0.26871153]\n", 1262 | "Pred:[1 0 0 0 1 0 0 0]\n", 1263 | "True:[1 0 0 0 1 0 0 0]\n", 1264 | "True\n", 1265 | "21 + 115 = 136\n", 1266 | "------------\n", 1267 | "Error:[ 0.14836389]\n", 1268 | "Pred:[0 1 1 1 0 1 1 1]\n", 1269 | "True:[0 1 1 1 0 1 1 1]\n", 1270 | "True\n", 1271 | "99 + 20 = 119\n", 1272 | "------------\n", 1273 | "Error:[ 0.26056678]\n", 1274 | "Pred:[1 1 1 1 0 1 1 1]\n", 1275 | "True:[1 1 1 1 0 1 1 1]\n", 1276 | "True\n", 1277 | "126 + 121 = 247\n", 1278 | "------------\n", 1279 | "Error:[ 0.21699123]\n", 1280 | "Pred:[0 1 0 0 0 1 1 1]\n", 1281 | "True:[0 1 0 0 0 1 1 1]\n", 1282 | "True\n", 1283 | "52 + 19 = 71\n", 1284 | "------------\n", 1285 | "Error:[ 0.15114712]\n", 1286 | "Pred:[0 1 1 1 0 0 1 1]\n", 1287 | "True:[0 1 1 1 0 0 1 1]\n", 1288 | "True\n", 1289 | "51 + 64 = 115\n", 1290 | "------------\n", 1291 | "Error:[ 0.23657825]\n", 1292 | "Pred:[1 0 0 1 1 1 1 1]\n", 1293 | "True:[1 0 0 1 1 1 1 1]\n", 1294 | "True\n", 1295 | "54 + 105 = 159\n", 1296 | "------------\n", 1297 | "Error:[ 0.21836464]\n", 1298 | "Pred:[1 1 1 1 1 0 1 0]\n", 1299 | "True:[1 1 1 1 1 0 1 0]\n", 1300 | "True\n", 1301 | "125 + 125 = 250\n", 1302 | "------------\n", 1303 | "Error:[ 0.11765047]\n", 1304 | "Pred:[0 1 1 0 1 0 1 1]\n", 1305 | "True:[0 1 1 0 1 0 1 1]\n", 1306 | "True\n", 1307 | "42 + 65 = 107\n", 1308 | "------------\n", 1309 | "Error:[ 0.22654752]\n", 1310 | "Pred:[0 1 1 1 1 1 1 1]\n", 1311 | "True:[0 1 1 1 1 1 1 1]\n", 1312 | "True\n", 1313 | "97 + 30 = 127\n", 1314 | "------------\n", 1315 | "Error:[ 0.09659835]\n", 1316 | "Pred:[0 1 1 1 0 1 0 1]\n", 1317 | "True:[0 1 1 1 0 1 0 1]\n", 1318 | "True\n", 1319 | "32 + 85 = 117\n", 1320 | "------------\n", 1321 | "Error:[ 0.20630273]\n", 1322 | "Pred:[1 1 1 1 0 0 1 1]\n", 1323 | "True:[1 1 1 1 0 0 1 1]\n", 1324 | "True\n", 1325 | "120 + 123 = 243\n", 1326 | "------------\n", 1327 | "Error:[ 0.19706751]\n", 1328 | "Pred:[0 1 1 1 0 1 0 0]\n", 1329 | "True:[0 1 1 1 0 1 0 0]\n", 1330 | "True\n", 1331 | "21 + 95 = 116\n", 1332 | "------------\n", 1333 | "Error:[ 0.22943276]\n", 1334 | "Pred:[1 0 1 0 0 0 1 0]\n", 1335 | "True:[1 0 1 0 0 0 1 0]\n", 1336 | "True\n", 1337 | "104 + 58 = 162\n", 1338 | "------------\n", 1339 | "Error:[ 0.23257017]\n", 1340 | "Pred:[0 1 1 0 0 0 1 1]\n", 1341 | "True:[0 1 1 0 0 0 1 1]\n", 1342 | "True\n", 1343 | "22 + 77 = 99\n", 1344 | "------------\n", 1345 | "Error:[ 0.15080149]\n", 1346 | "Pred:[0 1 0 1 1 1 1 0]\n", 1347 | "True:[0 1 0 1 1 1 1 0]\n", 1348 | "True\n", 1349 | "45 + 49 = 94\n", 1350 | "------------\n", 1351 | "Error:[ 0.12567795]\n", 1352 | "Pred:[0 0 1 1 0 1 1 1]\n", 1353 | "True:[0 0 1 1 0 1 1 1]\n", 1354 | "True\n", 1355 | "38 + 17 = 55\n", 1356 | "------------\n", 1357 | "Error:[ 0.28120981]\n", 1358 | "Pred:[1 1 0 0 0 1 1 0]\n", 1359 | "True:[1 1 0 0 0 1 1 0]\n", 1360 | "True\n", 1361 | "108 + 90 = 198\n", 1362 | "------------\n", 1363 | "Error:[ 0.32838608]\n", 1364 | "Pred:[1 0 0 0 1 0 0 0]\n", 1365 | "True:[1 0 0 0 1 0 0 0]\n", 1366 | "True\n", 1367 | "110 + 26 = 136\n", 1368 | "------------\n", 1369 | "Error:[ 0.22691487]\n", 1370 | "Pred:[1 1 0 0 0 1 1 1]\n", 1371 | "True:[1 1 0 0 0 1 1 1]\n", 1372 | "True\n", 1373 | "72 + 127 = 199\n", 1374 | "------------\n", 1375 | "Error:[ 0.09966136]\n", 1376 | "Pred:[0 0 1 0 0 1 0 0]\n", 1377 | "True:[0 0 1 0 0 1 0 0]\n", 1378 | "True\n", 1379 | "4 + 32 = 36\n", 1380 | "------------\n", 1381 | "Error:[ 0.23196088]\n", 1382 | "Pred:[0 1 0 1 0 1 1 1]\n", 1383 | "True:[0 1 0 1 0 1 1 1]\n", 1384 | "True\n", 1385 | "28 + 59 = 87\n", 1386 | "------------\n", 1387 | "Error:[ 0.14169074]\n", 1388 | "Pred:[0 0 0 1 1 0 1 0]\n", 1389 | "True:[0 0 0 1 1 0 1 0]\n", 1390 | "True\n", 1391 | "19 + 7 = 26\n", 1392 | "------------\n", 1393 | "Error:[ 0.11328021]\n", 1394 | "Pred:[0 1 1 0 1 1 0 0]\n", 1395 | "True:[0 1 1 0 1 1 0 0]\n", 1396 | "True\n", 1397 | "100 + 8 = 108\n", 1398 | "------------\n" 1399 | ] 1400 | } 1401 | ], 1402 | "source": [ 1403 | "max_iter = 20000\n", 1404 | "for j in range(max_iter):\n", 1405 | " # 랜덤하게 정수 두 개를 뽑은 후 binary lookup table에서 해당 이진수 가져오기.\n", 1406 | " # Randomly pick two integers and change it to the binary representation\n", 1407 | " a_int = np.random.randint(1,largest_number//2)\n", 1408 | " a = int2binary[a_int]\n", 1409 | " b_int = np.random.randint(1,largest_number//2)\n", 1410 | " b = int2binary[b_int]\n", 1411 | " # 실제 정답 계산 및 binary 벡터 저장.\n", 1412 | " # Calculate the answer and save it as a binary form\n", 1413 | " c_int = a_int + b_int\n", 1414 | " c = int2binary[c_int]\n", 1415 | " \n", 1416 | " # RNN이 예측한 binary 합의 값 저장할 변수 선언.\n", 1417 | " # Declare the variable for saving the prediction by RNN\n", 1418 | " pred = np.zeros_like(c)\n", 1419 | " \n", 1420 | " overallError = 0\n", 1421 | " \n", 1422 | " output_layer_deltas = list()\n", 1423 | " hidden_layer_values = list()\n", 1424 | " hidden_layer_values.append(np.zeros(hidden_dim)) # dim: (1, 16)\n", 1425 | "\n", 1426 | " # feed forward !\n", 1427 | " # 이진수의 가장 낮은 자리수부터 시작해야하므로 reversed로 for문 돌림.\n", 1428 | " # As you have to calculate from the \"first\" position of the binary number, which stands for the lowest value, loop backward.\n", 1429 | " # e.g. \n", 1430 | " # 10(2) + 11(2), for the first iteration: X = [[0,1]] y = [[1]]\n", 1431 | " for position in reversed(range(max_binary_dim)):\n", 1432 | " \n", 1433 | " # RNN에 들어갈 input과 output label 이진수 값 가져오기\n", 1434 | " # Take the input and output label binary values\n", 1435 | " X = np.array([[a[position],b[position]]]) # dim: (1, 2), e.g. [[1,0]]\n", 1436 | " y = np.array([[c[position]]]) # dim: (1, 1), e.g. [[1]]\n", 1437 | " \n", 1438 | " # hidden layer 계산하기 h_t = sigmoid(X*W_{hx} + h_{t-1}*W_{hh})\n", 1439 | " hidden_layer = sigmoid(np.dot(X,synapse_0) + np.dot(hidden_layer_values[-1],synapse_h)) # dim: (1, 16)\n", 1440 | " \n", 1441 | " # output_layer 계산하기 \n", 1442 | " output_layer = sigmoid(np.dot(hidden_layer,synapse_1)) # dim: (1, 1), e.g. [[0.47174173]]\n", 1443 | " \n", 1444 | " # error 계산\n", 1445 | " output_layer_error = y-output_layer # dim: (1, 1) \n", 1446 | " \n", 1447 | " # display를 위한 저장 (just for displying error curve)\n", 1448 | " overallError += np.abs(output_layer_error[0]) # dim: (1, ) \n", 1449 | " \n", 1450 | " # 이 후 backpropagation에서 사용될 delta 값 미리 계산하여 저장\n", 1451 | " # Save it for the later use in backpropagation step \n", 1452 | " output_layer_deltas.append((output_layer_error) * sigmoid_output_to_derivative(output_layer)) \n", 1453 | " \n", 1454 | " # 현재 자리수에 대한 예측값 저장\n", 1455 | " # save the prediction by my model on this position\n", 1456 | " pred[position] = np.round(output_layer[0][0])\n", 1457 | " \n", 1458 | " # 현재까지 계산된 hidden layer 저장\n", 1459 | " # save the hidden layer by appending the values to the list\n", 1460 | " hidden_layer_values.append(copy.deepcopy(hidden_layer)) \n", 1461 | " \n", 1462 | " if (j%100 == 0):\n", 1463 | " overallError_history.append(overallError[0])\n", 1464 | " \n", 1465 | " # 이제 backpropagation !\n", 1466 | " \n", 1467 | " # 맨 처음 시작할 때는 현재 시점보다 앞에 있는 hidden layer가 없으므로 delta 값이 0임. \n", 1468 | " # As RNN needs to consider the \"future\" hidden layer value to calculate the backpropagation and it does not have the \n", 1469 | " # value at the first time (at the end of the position where backpropagation starts), we have to initialize it with zeros\n", 1470 | " future_hidden_layer_delta = np.zeros(hidden_dim)\n", 1471 | " \n", 1472 | " # backpropagation을 할 때는 이진수의 가장 앞자리수 시점부터 돌아와야 하므로 정상적인 for문\n", 1473 | " # Now it should go \"backward\" which means an ordinary way in the for loop\n", 1474 | " for position in range(max_binary_dim):\n", 1475 | " \n", 1476 | " # 필요한 값들 다시 불러오고 \n", 1477 | " # bring what you needs for calculation\n", 1478 | " X = np.array([[a[position],b[position]]])\n", 1479 | " hidden_layer = hidden_layer_values[-position-1]\n", 1480 | " prev_hidden_layer = hidden_layer_values[-position-2]\n", 1481 | " \n", 1482 | " # 현재 시점에서 output layer error로부터 돌아오는 gradient 값\n", 1483 | " # Get the gradients flowing back from the error of my output at this position, or time step\n", 1484 | " output_layer_delta = output_layer_deltas[-position-1]\n", 1485 | " \n", 1486 | " # 현재 시점의 hidden layer에 더해진 gradient를 계산하기 위해서는\n", 1487 | " # 이전 시점의 hidden layer로부터 돌아오는 error gradient + 현재 시점 output layer로부터 돌아오는 error gradient\n", 1488 | " # 이 둘의 합에 sigmoid의 derivative 계산해줘야 함\n", 1489 | " # 이유: h_t = sigmoid(X*W_{hx} + h_{t-1}*W_{hh})를 역전파 하는 것을 생각하면 됨.\n", 1490 | " # Important part! (Backpropagation)\n", 1491 | " # Think about the feed forward step you have done before: h_t = sigmoid(X*W_{hx} + h_{t-1}*W_{hh})\n", 1492 | " hidden_layer_delta = (np.dot(future_hidden_layer_delta,synapse_h.T) + np.dot(output_layer_delta,synapse_1.T)) \\\n", 1493 | " * sigmoid_output_to_derivative(hidden_layer)\n", 1494 | " \n", 1495 | " # 8자리 모두를 다 계산한 후 gradient의 합을 한 번에 update 해준다. \n", 1496 | " # 이유: backprop이 아직 다 끝나지 않았는데 중간에 hidden layer의 value가 바뀌면 안됨\n", 1497 | " # Save the updates until the for loop finishes calculation for every position\n", 1498 | " # Hidden layer values must be changed ONLY AFTER backpropagation is fully done at every position.\n", 1499 | " s1_update += np.atleast_2d(hidden_layer).T.dot(output_layer_delta)\n", 1500 | " sh_update += np.atleast_2d(prev_hidden_layer).T.dot(hidden_layer_delta)\n", 1501 | " s0_update += X.T.dot(hidden_layer_delta)\n", 1502 | " \n", 1503 | " # 다음 position으로 넘어가면 현재 hidden_layer_delta가 future step이 되므로 이를 넣어준다.\n", 1504 | " # Preparation for the next step. Now the current hidden_layer_delta becomes the future hidden_layer_delta.\n", 1505 | " future_hidden_layer_delta = hidden_layer_delta\n", 1506 | "\n", 1507 | " # weight 값들 update (learning rate를 곱하여)\n", 1508 | " synapse_1 += s1_update*alpha\n", 1509 | " synapse_0 += s0_update*alpha\n", 1510 | " synapse_h += sh_update*alpha\n", 1511 | " \n", 1512 | " # update value initialization for the new training data (새로운 a,b training 이진수에 대해 계산을 해줘야하므로)\n", 1513 | " s1_update *= 0\n", 1514 | " s0_update *= 0 \n", 1515 | " sh_update *= 0\n", 1516 | " \n", 1517 | " # accuracy 계산\n", 1518 | " check = np.equal(pred,c)\n", 1519 | " if np.sum(check) == max_binary_dim:\n", 1520 | " accuracy_count += 1\n", 1521 | " if (j%100 == 0):\n", 1522 | " accuracy_history.append(accuracy_count)\n", 1523 | " accuracy_count = 0\n", 1524 | " \n", 1525 | " \n", 1526 | " if (j % 100 == 0):\n", 1527 | " print (\"Error:\" + str(overallError))\n", 1528 | " print (\"Pred:\" + str(pred)) # 예측값\n", 1529 | " print (\"True:\" + str(c)) # 실제값\n", 1530 | "\n", 1531 | " final_check = np.equal(pred,c)\n", 1532 | " print (np.sum(final_check) == max_binary_dim)\n", 1533 | "\n", 1534 | " out = 0\n", 1535 | "\n", 1536 | " for index, x in enumerate(reversed(pred)):\n", 1537 | " out += x * pow(2, index)\n", 1538 | " print (str(a_int) + \" + \" + str(b_int) + \" = \" + str(out))\n", 1539 | " print (\"------------\")" 1540 | ] 1541 | }, 1542 | { 1543 | "cell_type": "code", 1544 | "execution_count": 8, 1545 | "metadata": { 1546 | "collapsed": false 1547 | }, 1548 | "outputs": [ 1549 | { 1550 | "data": { 1551 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD8CAYAAAB0IB+mAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXl0VdX5/p+dEEgIMwQIY0AEBBxApNaBAoqi1gG1ilZb\nxVWk2tZ++7O1VqvWrjpjnesXFWfUOqP9qogDMgg1ICBlCiBjQCYZEgiZ3t8f7909596ce++5yR2T\n57PWXfsM+5yzc+7Ns9/z7H32NiICQgghjZ+sVBeAEEJIcqDgE0JIE4GCTwghTQQKPiGENBEo+IQQ\n0kSg4BNCSBOBgk8IIU0ECj4hhDQRKPiEENJEaJbqArjp1KmTFBUVpboYhBCSMSxatGiXiBT4yZtW\ngl9UVITi4uJUF4MQQjIGY8xGv3lp6RBCSBOBgk8IIU0ECj4hhDQRKPiEENJEoOATQkgTgYJPCCFN\nBAo+IYQ0ESj4kRABpk8Htm5NdUkIIaTBUPAj8fLLwE9/CkybluqSEEJIg6Hgh2PTJuD663V5//7U\nloUQQuIABT8c99wDVFUBLVsC5eX+j5s5E/jiC//5Fy0CHnoo9vIRQkiMUPC9EAE++AAYOxbo3Dk2\nwf/974Fbbw2/f9kyoKbGWZ82DbjxRr0mIYQkEAq+F+vWARs2AGecAeTnA2Vl/o8tLQW2bfPet3Ej\ncNxxwJtvOtv27NEK4NChBhWZEEKiQcH3YuZMTc84A2jVyn+Ef/gwsGuXCr47YrfLa9bo8kbX4Ha7\nd2u6b1/Dy00IIRGg4HsxcyZQVAT06xdbhL99u6bl5cCBA7r8zTdAQQFQXOwI/c6dzjF79mhKwSeE\nJBgKfihVVcCnn2p0b4wKvt8I391f39o6f/qTRvHz53sLPiN8QkiSSKsJUNKChQs1Oj/jDF2PxdIp\nLXWWt21TMX//fV1fs8bp3rlrl5PPRvjs+kkISTAJF3xjTDaAYgBbReTHCb2YiEblkaitBZ54AujY\nEbjssrr7Z84EsrKAMWN0PRZLJ1TwX39de/kUFACrVwOVlbrPRvhVVY7QM8InhCSYZFg6NwBYmfCr\nPPcc0KsXcPCgs6283BFZQCP3ceOAX/8auPlm7/PMnAmMGAG0b6/roZZOaSnw2GPe3ShLS50KZ9s2\n9e1PP1175qxZU9fS+f5751gKPiEkwSRU8I0xPQCcA+DpRF4He/dq//ctW4CSEmf76NHAwIHAggW6\n/qc/AbNmAaNGqfhaO+XwYd33xRfAV18BZ57pnKNVK43wrcA/8ohWGCtW1C1HaalWOrm5KvCbNwOD\nBwP9++ubu1u2aD5r6Vj/HqClQwhJOImO8B8C8AcAteEyGGMmGWOKjTHFO92NmbFw112OiK5fr+mu\nXSremzYBp5wCXH018PjjwdH9kiWafvQRcPfdauPU1jr+PaARfm2tVgoAMHu2pl9+WbccpaVA9+5A\nYaFWLIAj+ID2t+/dW8X98GGnwgEY4RNCEk7CBN8Y82MAO0RkUaR8IjJVRIaLyPCCgoLYL7R3r3ry\nF1yg6+vWaWoF+e23Veyfew7o1g3461/VYgEcwZ8xA2jdGujaVa2cESOc87dqpWlZmVo7xcXB53dT\nWqrXKCx0yjFkiCP4AHD88Zru2hUc4fsR/IsvBm67LXo+QgjxIJGNticDOM8YczaAXABtjDEvicgV\ncb1Ku3Yqwm3bqiVjI/z584FmzdRDP/dcYPJkFfU2bfTTrRvw9dcavb/3HnDOOTqmzZ49epwlP1/T\n8nLNX12tlcL8+XXLUlqqwzFY+ycvD+jTJ7gNYPhw4K231Me3Eb4x/iydhQudJw1CCImRhEX4InKz\niPQQkSIAEwB8GnextwwcqFF1375OZD1vHjBsmIouoJG1O9IeOlQFfOFCYMcO4LzzgC5dgKOOCj63\nW/Bnzways4Ff/hJYtQp49lm1bPbu1f379jkRPqDnysrSisZu84rwu3XzF+Hv3RvcKG1ZuBCYMyf6\n8YSQJk3jevHqiCM0wq+sVP/+5JPD5x06VEX78cc1oj/rLO98bktn9mwVbNuoe8012ni7cKHzolW3\nbvoBtDKw9O+vFUrPnrpuI/zsbKBHj+iCX1Pj2Eqh3HqrNloTQkgEkvLilYh8DuDzhF+ob18dmKy4\nGKioiC74NTU6yckvf6nWkBc2wi8r0/NOnqy2TLNmQIsWjtVjnyS6dXNGwxwyxDnP5MnaM8i2U+zc\nqRF+hw56bXcDrhfW8vES/H379O8lhJAINK43bY84Qj32xx5TK+WUU8LnHT1affvLL/d+ActiI/zS\nUhXVnj11jPy771bL5je/ARYvdiqGfv2cY48+2lmeMEHTmhr17G2E37Gjtj9s2BD5b7NPAF6Cf+CA\ntkUQQkgEGpfg9+2r6SuvOJ58ONq3d4Y9iIQVctsY3LmzpjfeqOmzz2qEf/Cgin3v3lopvPZacPdO\nS3a2irz18Dt0UMGPZulEE/zsbO/jamuBa6/VisldARFCmhyNz8O3XHttfM4ZKvihlcjQocDatTrg\nmvX2s7KASy4JL8IFBcERfps2/gXfq9H2wIHwls6uXcDTTzvvBRBCmiyNS/C7dweaN9e3Xd1vyzYE\na+mEE/xhwzQ9dMg7oveiU6dgD79tWz3+tdeAE09UWyqUvXs1LS+vO9Z+WVn47ppVVZqyOychTZ7G\nJfjZ2fom7b33ho+uY8VPhA9oI+6oUf7OWVCgY+fv3u14+ADw/PPa42fNmrrH2AhfJDiaP3hQbZtw\nEb4dS8g9phAhpEnSuDx8AHjggfier3lzFfOtW7WxtWPH4P1du2of+3791Jrxw7Bh+vIV4ET4gNOX\nfskSYNCg4GPclk95udMryI7kefiw92ihjPAJIQEaV4SfCOwkKIBaMc086shXX9XhHfxy8806efng\nwdp11FYUVryXLNH+/b/5jdP7JlTwLXZmLcBb1BnhE0ICUPD9YAU/XK+fkSOD+9xHIytLx/dZvlxt\nIBvhA1rBLFkCTJkCPPqoM8KmW/DdDbfRBJ8RPiEkAAXfD7bhNlI3z4bgFvzTT9dunjNm6Pp332nq\nJ8L38vEZ4RNCAlDw/WAjfNsHP95Ywe/USQd627XLGe65oYLPCJ8QEoCC74dER/jWwx82zOn1Y7GC\nv3ev0yBLD58QUg8o+H6I5uE3lHbtgJwcHaPnmGN0m51T1x3h2yeMcB5+QyL8Cy/UISkIIY2Wxtct\nMxEkWvCbN9c3dYcM0Wh/yhQdB2js2GDB795d1xPh4c+d679bKSEkI6Hg+yHRlg4QPNDb737nXM8t\n+EccoQO1xWLp+I3wKys54iYhjRxaOn5IdKNtOEIF346zHynCf+YZnZDd4jfCr6xkwy4hjRwKvh+S\nEeF7YQW/ulpFvmtX3e4WfPuyFqCCP2MG8M9/OtsY4RNCAlDw/dC5sw5lkOwIv3NnFXw7+Un79kBu\nbuQXr8rKdCA2i58Iv6ZGP4zwCWnUUPD9MHmyznbVokVyr9uliw6hvHOnrrdrp/ZSqKVjLaeKCt3n\nFnw/Eb7NwwifkEYNBd8P+fl1BzNLBtZCWrtW07ZtvQXfTptoBd8t3H4ifLuPET4hjRoKfjpjBd8O\nlxxN8N2Wjh0z30+ET8EnpElAwU9nvAS/Zcu6Hn5ohA844h1LhE9Lh5BGDQU/nbGCP3u2ph07ekf4\nnTrpckWF02vHincsHj4jfEIaNRT8dMYK/sqVwDnn6ATpXoLfoYOOs3PwoNNga9N4Rvh33ulM3k4I\nyTj4pm0606qVWji1tcAjjziTsWzYoPtrazWib91aexDt2eMcawU/nh7+nDnO3LqEkIyDgp/uXHkl\ncNxxQN++ut6ypRPh27R1a+2fv3u3c5yN1q2YV1V5T4HozhMtwq+udioQQkjGQcFPd558Mng9P99p\ntLUvXXkJfmiEb5ebN697DXelUFurM3J5UV2tH0JIRkIPP9Nwe/i2gdZaOl6C7/buDx8GrrlGR8Z0\n484TyetnhE9IRkPBzzTy81XMa2sjR/ihlg4AfP+9Tp7+4YfB53TniWTrUPAJyWgo+JlGy5aaHjyo\nAg7oOPZ+LB2b3z3+DlD3KSAcNTW0dAjJYCj4mYYdN+fgQeeFrH791NJxj6HjZenYXjx2MDYLI3xC\nmgQU/EzDCn55ufbPb91aZ8LKzQ3OF/riFdDwCJ+CT0hGQ8HPNKzgl5Wp4A8cqF0tQwU/UoQfSfCj\nRfi0dAjJWCj4mYbtj798uQr+UUfpeujQzV4evh/BZ4RPSKOFgp9pHHecNtK++y5QWuoIfiwRfiQP\nP1qjLQWfkIyFgp9pZGcDI0cCb72l63acfiv4NtJ3e/g5Obrsx8OnpUNIo4WCn4mMGuVE2qGWju2T\n747w7Zy88bB0RDTSJ4RkHBT8TGTUKE2bNwf69NFlG+Hn5+v8u24P3wq+jfD373cmSAFii/DtOQkh\nGQfH0slErI/fqxfQLPAVugW/qir4TdvQCL+mRvfn5Tl5LNEifHdKCMkoEib4xphcAF8AaBG4zhsi\ncnuirtekyM4Gfv97tW8sVvBbtVIxd0f4HTrosnv45AMHHMF3R+yM8AlptCQywj8MYIyIlBljcgDM\nNcZ8ICILEnjNpsOttwavWw/fDq4WycMH1Nbp3NnJY4nWSweg4BOSoSTMwxclMJwjcgIfiXAIaQju\nCD8vL7iXjhV8d2Ote5mWDiFNgoQ22hpjso0xSwDsAPCxiCxM5PWaNG4PP1wvHTehgm/toXCWjrt3\nDiN8QjKShAq+iNSIyHEAegAYYYwZEprHGDPJGFNsjCneuXNnIovTuLGWjo3wvXrpuHG/fOWuFMJF\n+O6umBR8QjKSpHTLFJG9AD4DMM5j31QRGS4iwwsKCpJRnMZJaLdMr146ANCxo6YHDuiE5J98onla\ntNBunuEifLeNQ0uHkIwkkb10CgBUicheY0wegLEA7k3U9Zo8bg/fbelUVekY+saoLdO1q46bv2MH\nMGWKin1lpYp9ixaM8AlpxCQywi8E8JkxZhmAr6Ae/vsJvF7Txt1Lx1o6NTU6M1bz5s5ctl27arp8\nuaZlZcGC7yfCp+ATkpEkLMIXkWUAhibq/CQEr146Vpjd0XuXLrrNCn55uSP4ubnhI3xaOoRkPHzT\ntrHg1UvHdrfMyXEi/NattVJwR/hVVdEtHUb4hGQ8HEunseDVSyc0wge0QmjdWoUeqBvhV1QAmzcH\nz48LUPAJaQRQ8BsLAwcCo0cDI0ao4FdXOw237gg/P1/H4bGEeviHDwPnnadDN7ihpUNIxkNLp7HQ\nti3w6ae6bO0d29feK8K3lJdr2r6902hbUgIUFQWfn710CMl4KPiNETsomhX80AjfLfhlZc7+3Fxg\n506tBEJFnZYOIRkPLZ3GSKjgR4vw3ZbO+vW6PdS2oaVDSMZDwW+MhFo67gi/VSvHw+/aNdjDz811\nxthhhE9IoyOq4AcGQHsgGYUhccJG+Pv2aRouwh8yRMW7vDw4DxBd8D/8EHj66cSUnxCSEKIKvojU\nADglCWUh8cKvhz94sKbff19X8ENtG3ejbXU18NRTwP33x7/shJCE4bfR9mtjzAwArwMotxtF5K2E\nlIo0jGi9dAYMUDtnwADdZl+8qq11zhEtwq+o4GTmhGQYfj38XAC7AYwBcG7g8+NEFYo0kGgR/tVX\nA5s2Ae3aOcfEaukcPszGW0IyDF8RvohcneiCkDji1UvHLfjGaCXgHja5eXPdbonWS6eigoJPSIbh\nK8I3xvQwxrxtjNkR+LxpjOmR6MKRemItHdtom5MTbOlY3MuxRvi0dAjJOPxaOs8CmAGgW+DzXmAb\nSUfsJCfbtmkaGuFb3BF+To5TUQC0dAhphPgV/AIReVZEqgOf5wBweqp0pWNHtXXWrtV1G+FnZwdH\n8aGWTiy9dGjpEJJx+O2ls9sYcwWAVwLrl0EbcUk6YgzQqxewerWuN28O/OhHOsuV26cPtXTsvg4d\nGOET0gjxG+FPBHAJgO0AtgG4GAAbctOZnj2d5ZwcYPx4YPr04DyhEb61dHr3podPSCMkaoRvjMkG\ncKGInJeE8pB40auXs2z9+1BCI/w+fbQSGDTIGVPHwl46hGQ8ft+0vSwJZSHxxC34OTneeayvD6jg\njx6tPXu6dqWlQ0gjxK+HP88Y8xiA1xD8pu3ihJSKNBw/Eb4xGuXv3+/kycrSCiJU8N32TWWlM31i\nba0eQwhJe/wK/nGB9E7XNoG+eUvSET8RPqAWjlvwAaBZs8gvXpWXB28PV6EQQtIKPx5+FoB/iMg/\nk1AeEi/cgt8swtdsfXy3aOfkACIa1VvLxwp+VpYzHy7AhltCMgg/Hn4tgD8koSwknvQIvAidkxPc\nFTMU21MnVPCBYFvHCn5enjNmvns7ISTt8Wu+zjLG3GiM6WmM6WA/CS0ZaRh5eUDnztHtFi/Bt08E\nXrNc5eUFR/gUfEIyBr8e/qWB9HrXNgHQN77FIXGlZ09g3brIecJZOkBwhG+tm9xcCj4hGYrf0TL7\nJLogJAH06gVs3hw5T6yWjnsaRIAePiEZRERLxxjzB9fyT0L23ZWoQpE4MWECcMUVkfN4Rfi0dAhp\nlETz8Ce4lm8O2TcuzmUh8eaSS4ApUyLnqU+ET8EnJCOJJvgmzLLXOslE/Hr41dXa26dFC+DQoeDt\nhJCMIJrgS5hlr3WSicTSSyc7u+5LXPTwCckYojXaHmuM2Q+N5vMCywis54Y/jGQMrVtr6h4LP1wv\nnWbN6r7ExQifkIwhouCLSHayCkJSxOWXA23bOrNkAeEtnWbN6kb4FHxCMoaIgh/t5SoR2RPf4pCk\n07kzcHXI1AZW1EMtHQo+IRlNNEtnEdSr92qg5YtXjRVr23hF+KGWDj18QjKGaJYOX7hqitDSIaRR\nEs3SGRZpP8fDb6R4WTp25EwKPiEZSzRLJ9JbOxwPv7ESi6VDwSckY4hm6YxOVkFIGhGLpUMPn5CM\nwe9omTDGDAEwCK7+9yLyQiIKRVIMe+kQ0ijxJfjGmNsBjIIK/v8BOAvAXABhBd8Y0zOwvwvU/pkq\nIg83sLwkGdDSIaRR4ncClIsBnAZgu4hcDeBYAG2jHFMN4P+JyCAAJwK43hgzqN4lJckjnKXDRltC\nMhq/gn8oMNVhtTGmDYAdAHpGOkBEttlePCJyAMBKAN0bUliSJML10qGlQ0hG49fDLzbGtAPwFPRl\nrDIAX/q9iDGmCMBQAAs99k0CMAkAerkn3iapI5KlYwW/eXOgspKNtoRkEFEF3xhjANwtInsBPGmM\n+RBAGxFZ5ucCxphWAN4E8FsR2R+6X0SmApgKAMOHD+cInOlApF46tjLIz1fBZ4RPSMYQ1dIREYE2\n1Nr1DTGIfQ5U7F8WkbfqXUqSXPx0y7Tj6FPwCckY/Hr4i40xJ8Ry4sCTwTMAVorIgzGXjKQOP+Ph\n23H0KfiEZAx+Bf8HAL40xqwzxiwzxnxjjIkW5Z8M4EoAY4wxSwKfsxtUWpIc/IyHbyN8eviEZAx+\nG23PjPXEIjIXnAYxM6GlQ0ijxFeELyIbod0wxwSWD/o9lmQg2YF5byK9aUvBJyTj8CXagTdtbwJw\nc2BTDoCXElUokmKMUXGP1EuHHj4hGYffKH08gPMAlAOAiJQCaJ2oQpE0ICcn8pu29PAJyTj8Cn5l\noHumAIAxJj9xRSJpQbNmkS0dRviEZBx+Bf+fxpj/BdDOGPMLALOgb92SxkpohB+ulw4Fn5CMwVcv\nHRF5wBgzFsB+AAMA3CYiHye0ZCS1eFk67gg/L0+9fgo+IRmD3+GRfwfgNYp8EyKapZObq+v08AnJ\nGPxaOq0BzDTGzDHG/MoY0yWRhSJpQLhGW2vptGhRt1IghKQ1fvvh/0VEBgO4HkAhgNnGmFkJLRlJ\nLdEsndxcrQAo+IRkDLG+PLUDwHYAuwF0jn9xSNoQGr2HjofPCJ+QjMPvi1fXGWM+B/AJgI4AfiEi\nxySyYCTFhIvwraVjPXwKPiEZg9+xdHoCuAHASGhf/JzI2UnGE07wbXfMVq3YaEtIhuHX0tkOHUqh\nE9TKeckY8+uElYqkHq9eOtnZwODBwOuvA+PG0cMnJMPwG+FfA+BEESkHAGPMvdApDh9NVMFIinFH\n+CKOh28McPHFut2vpfOLXwBt2wK33AK0b5+4MhNCIuJX8A0A97N7DTj0cePGLfi1tZo2C/m5+BH8\nXbuAp5/W5ddfB9as0QZfQkjS8WvpPAtgoTHmDmPMHQAWQGezIo2VnBxHzG3qJfg1NTq37f460xUr\nywLz5Jx7LrBpE/Ddd4kpLyEkKn774T8I4GoAewKfq0XkoUQWjKQY9/DINg0VfOvh33EHcNJJuu3t\nt4ErrnDyWME/MzCHTriKgRCScPxaOhCRxQAWJ7AsJJ2wls6llzo9cezEKBZr6WzeDKxYofnfew94\n9VXgxRfV71+6FOjSBTjySD2Ggk9IyvAt+KSJYS2dpUuBfft0WzgPv6JCG3a3bwe2bNEK4vBh7au/\nbBlwzDFAmzZ6jD0XISTpcJpC4o21dHbvViG320Lz1NSo4APA1q0a7QNAeblWBv/5D3DssY7gM8In\nJGUwwife5ORoY+yePc62cBG+9fg3b3YEv6xMG2gPH9YIv21b3U7BJyRlUPCJNzk52qXSdskEwjfa\nVlbq+ooVGtkDmi5dqsuM8AlJCyj4xJtmzRzxtoRrtLWWzsKFzr6yMqCkRJcHDNAKxBh6+ISkEHr4\nxJscj+GSonn4CxY4+8rKNJpv2VJftMrKAlq3ZoRPSAqh4BNv/Aq+O8L//ntnX3m5iru1cgD18Sn4\nhKQMCj7xxi3udiiEcB7+oUN1j7cRvlvw27ShpUNICqHgE2/cEf6xx2oaLcIH1LoBNMI/cEBtHEub\nNozwCUkhFHzijRX8rCztVgl4N9paD98+BfTtq2m4CJ+CT0jKoOATb2w036EDUFQUvM2dp6pKBf+I\nI3TbgAGaNtTDP3gQuPxyYNu2ev8JhJBgKPjEGxvhd+wI9O6ty14e/qFD2le/Xz/d1revHltW5m3p\nWA/f3b/fi+XLgVdeAebMafjfQggBQMEn4bCC36EDMHSorvfoEZynWTMVdgDo31/T3r11GkSvCN9a\nOrNm6fLOneGvbxuC7fkJIQ2Ggk+8sdF8x446rWF5OTBwYN08VpCLioA33wQmTtT5bsN5+OXlwLx5\nmn7zTfjrU/AJiTsUfOKN29Jxr7txj5mfmwtceKFOYZifr2PwVFUFWzp2PJ0lSzRdty789Sn4hMQd\nCj7xJlTwvXD32snLc5ZbtQJKS3U5NMIHKPiEpAgKPvHGbelEywNohG/Jz3d613gJ/oYNmlLwCUkq\nFHzijZ8IP5zgt2rljKEf2kvHjR/BP3AgelkJIb6g4BNvGiL4+fnOxOeh/fAt7dqp4It4n5sRPiFx\nh4JPvInV0gn18C1elg4AnHaa9uLZvdv73BR8QuJOwgTfGDPNGLPDGLM8UdcgCaR/f518PLQrpht3\no21ohG8JZ+mMHatpOFvHjs9DwSckbiQywn8OwLgEnp8kkqOPVh++sDB8nkiWjsUrws/NBU46SZfD\nCT4jfELiTsIEX0S+ALAnakaSuURqtLW4Bb9VK531qm9fZ+wdCj4hSYMePqk/0SJ8Y4KjfWO0Aujb\nV2fCKiwE1q71PjcFn5C4k/I5bY0xkwBMAoBevXqluDQkJiK9eAWof29M8DHnnAOMHKnLJ5wAzJ6t\nPXVC81HwCYk7KY/wRWSqiAwXkeEFBQWpLg6JhWgRfmi/ewB4+WXg2mt1+ayzgI0bgdWr6+azgl9e\nHn1kTUKIL1Iu+CSD8ZoGEQiO8CMxLtCm/+GHdfdZwRfxnkKREBIzieyW+QqALwEMMMZsMcZck6hr\nkRRhBb9Fi2BLJlKE76aoSLt9fvBB3X1ukaetQ0hcSJiHLyKXJercJE2wHr7bvwecCD+a4ANq6zzx\nhM5w1bKlsz1U8Lt0aVhZCSG0dEgDsBG+278HnAg/mqUDAGPGAIcPOyNoWg4dcs7PCJ+QuEDBJ/Un\nnODHEuHbSc83bgzefugQYBvxKfiExAUKPqk/0SJ8P4Jv58u1QyZbKPiExB0KPqk/0Tx8P5ZOfj7Q\nqRMjfEKSQMpfvCIZTLgIPzcX+POfgYsu8nee3r0p+IQkAQo+qT/hBN8Y4M47/Z+nd29gxQpnvbpa\nPxR8QuIKLR1Sf8IJfqwUFWmEbydDsV0yKfiExBUKPqk/VvBDPfxY6d1bRX7XLl23gt+hA5CVRcEn\nJE5Q8En9sY22DY3wbU8d6+Nbwc/L0wZgzmtLSFyg4JP6E09LBwDmzgUuuQTYvFnX8/K0pw8jfELi\nAhttSf2Jl+DbCP+WW3SIhcGDdd1G+OEEXwSYNAn4yU+AM85oWBkIaQIwwif1J14efrt2+pLWwYO6\nvn69c95Igv/tt8DTTwMTJgCbNjWsDIQ0ASj4pP7Ey8MH1Nbp0EG7dH77rW6LJviLFmm6fz9w5ZVO\nLx/LqlXABRfQEiIkAAWf1J94WToA8PDDwNtv61u3NsLPzVXB37YNWLasrqAXFwPNm+tLXl98Aezc\nGbx/1izg3XeBjz9uePkIaQRQ8En9iafgjxqlUx8WFgKlpbotLw/o0QNYswY49ljgjTeCj1m0CDj6\naGD4cF0PnR93+3ZNP/4Y+PprzRv6Ri8hTQgKPqk/8fLw3RQWOpF8Xh7wyCPAwoVATo5j4QCaZ9Ei\n4PjjgSOP1G0lJcHnsoI/axYwZQqwfLmej5AmCgWf1J94eviWwkJnOS9PLZsRI4B+/YCVK51969cD\ne/eq4BcV6Qta4SL8khLgtde0gnrmmfp7+tXVwJYt9TuWkDSAgk/qT+/ewFVX6SQm8SJU8C1HHaWN\nsBYb7Q8frpVCUZF3hG/7+FdXA489BuzbB1x4IXDKKVqBfPcd8LOfAYsXRy6XCPDzn2s5Kivr+9cR\nklIo+KT+5OQAzz4L9OkTv3NGEvx16xyx/ewznUt3yBBd79fPO8IfPVrbAX74Q+2zf9JJ2sC7ZAlw\nxRUq4i++CJx2WrBlFMpzzwHTp+vTgR0CgpAMg4JP0gu34LutoqOOAmpqNIrfuRN4/nng8ss1ugfU\nxy8pcfykVyTiAAAScklEQVT/2lqN3gsLgY8+UkvHGPXz9+xRkV+8WPfdequ+C3D66drzJ5TqauCG\nGzQPAOzYEf3vuOEGPS8haQQFn6QXVvBzc1WgLUcdpenKldrwWlEB/OEPzv4jj9T++NOna4+f0lIV\n6q5dgUGDgJ49NV9enk6WPn48cNNNwOTJOpTz558D7dur6LuHaga0W+iBA8B55+m6u/vn2LHA3/4W\nnF8EeOEF4F//0vUNG+raTaHU1vqziior9Wnkttui5yUkBAo+SS+6dtU0tOfPgAGazp2rXvz48cDA\ngc7+fv00vfZaYPZsp++9PZ8X99wD/OMfWrH07q2ib4wO8eDGNtQOG6apjfAPHQI++QSYOjX4HQHb\noGzHBJo8GTj77LrvEbi57z79GyPlAYA77gA+/dSpTAA9ZuHC8Mc+84y2U5AmDwWfpBc2wg8V/Px8\nFeWHHwbKy4G//CV4v+2aWV6u6cyZmkYS/FB69VIr5p139EUvixX8oUM1tRG+tZA2bdI2ActXX2m6\ne7cOF7FmjbYvLF0afL3qaqe8772nTwK7dwfnqajQ9w9uu00ruXvu0SeUFSvU4gK0EjzxRGDOHO+/\na9o04KWXnKErSJOFgk/Si5YtdVwdr7791ta54w6nsdZSVKTdRIcN07dz/UT4Xtxwg47Q6bZprOAP\nGaJdO22E7+4m+u67zrK7HWDjRmecn9AXx269FTjmGBV9W0msX69PBi++qOtTp+rgcH/7m17vqqtU\n9CsqnDeSrQW1enXdv+fwYS2PSF2rijQ5KPgk/Sgs9Bb8Sy8FLr442Lu3NG+uPYZefFHfqLWRcqyC\n3749MHGiRvkVFbptyxatiNq311m4rOCvWqUW0PDhmt/y1VfagwkAFizQSNwYFXy37bJwoYr2Aw8A\nVVW6bf164NFH1YLZulWP79FDK4VVqzRaP/FEzbt8uXMM4LxFPGuWE/0vWuS0DXzzjQ42d/LJka2j\nBQv0Gt9/H9u9I2kPBZ+kHz16aJQdylVXAa+/7rzhG8qVV2oD7bHH6nrLlhrtx8qYMSqS//63rm/Z\nomUyBujc2bF0Vq7UJ4vLLlO7ZvFiFdrFi7VhFXBslgsu0AjcHWXbiPzee51t337rCPmXX2p0fsIJ\nwT2WBg3S1OZbt07TjRs1/9ixTgU0f76mOTkq+C+8oNtKSjT6X7GirvjffbdWRu++q2UcM6buOEUk\nI6Hgk/TjoYe0Yba+HHOMpl27Bvf08cvJJ2s6d66mVvCBuhH+UUfpE0GXLto4u2iR9tW/4ALNYwX/\nuus0nTdP0wMHtPdPdrY2/g4ZoudYv94R8g8+UGG2YwVZ8vOBvn29I/xvvtFlW5nMm6cN2sccoxXI\nggXO9rvu0rkHhg/XdxPsOd5/X5ffeQd48EF95+GDD2K/j34pLo7eiymTWbkSOPVUbaNJMRR8kn4M\nGeL0iKkPNsKP1c6xdOyoUbQV682bHcG3EX5trYrqwIHaP//vf1cr56ST1I4aN07Ps3atVjqnnqpP\nLVaQ16zR9IorND31VBXxJUuc3j3Tp2saKviACvXy5Rqd2wh/wwbnCWL9et03f76W6eijVeytdTR3\nrj4t9e+vPYrGjlW7xz5tnHuuNnzbMnz+uXPt6dO1bSGUr7/WyiEWKiv1XvntRbRqVd3uq9F6Ni1f\nrg3ee/fGVrZ4UFWl3/HcuU67TCoRkbT5HH/88UJIg9m/XwQQufDC+p/j2mtF2rQROXxYJDtb5E9/\n0u2//a1I69Yi69frNaZO1e21tSK/+IXIlVfqPhGR447TPD166PoPfyjyox/p8ssv675Fi0QuuURk\n4UKRyy/XbYDIwIHO8q5ddct3880izZqJlJZqnrZtRbKyRM48U9dHjRJZt06Xn3xSZMoUXW7RQuT0\n00Xat9f1Rx4R2bNH5OSTneudf77Ixx876336iPTt61x78GCRrl31b16/XuTxx0UmT9br5+WJHDpU\nt7ybNoncd5/I0qW6/tZbujxjhnOdtWsjfyerVuk1xozR71hE5K9/1Xtl1ysr9T7fdJOWT0TvLyBy\nyy11z2nzJIo779Rrd+okMnRoQi4BoFh8amzKRd79oeCTuPHjH4s8/HD9j3/xRf33+Ne/NH3iCd3+\nt7/p+ltvaTpnTvhznHuu5jn1VF2fNEmkY0cVmdtvFzEmWBxvvdURPyvQRUXe57YVxhNPaHreeZq2\nbKlpr14ib7yhy199JTJzpi6fdprIPfc419m4Uc9XUaF5ZswQ2blThbNdO5FBg0T+/ncnb1mZii4g\nsnmzyFln6XJWllZmgMgnnwSX9Z13nGMGD9byGKMV4TnnaMUKqDhG4tFHNV92tsjw4SJffKGVHqD3\nU0Tkn/90/rYHHhDZskXz5+WJ5OeLTJ8uMnGiVkDvvy/SoYPIsmX6Pdx0k+aPJ337iowbJ3L//Vqm\nb7+N7/mFgk9Iw9m4Uf89rGjPmKHbn3pK16+9VsJG35brrtM8V16p6488ouulpSITJmjk7GbaNN2f\nl+dE7hdf7H3unTtFcnK0QgBEHnrIEbrcXBXU3/9exe7gQZHvvlNxvP9+raQAkWHDIt+Djz4SKS4W\n+fprzf/CCyJz5zrXefVVFdFJk7TC2L9fr/HHPwafZ8IEfSJ47DE9rn17fSIxRtevu05k5EiRAQMi\nR9zjx+s9e/99/RsBrZTOPFPLUVqqlWufPnrfAD2nMSIffOBUOoBI7956jK1obOV48skiVVV6vdWr\nRV55xVmPxN69IvPn68eya5ee85579OkFEHnwQd33/fciW7dGP68PKPiExIOzz3YEYvFi3fbOO47Q\nRBPMu+7SvLfdpuuffabrH32kj/fjxgXnt/vt/8GNN6q1Eo7x453yLV3qLNtyH3mkRtSWZcvUojp0\nSC2GKVP83YeaGhXpiRP1qQlQER05UpffeMPJe8opGn276dNH5Cc/UTEfNUqP+cc/RH79a12eP1/k\nf//Xqbgsd94pcsEFWvlWVztlENFKq0cPrSRLSrTy69xZ/vt0VFGhEXtWllpUIlpZP/qoyLx5eq7u\n3UX699dKYuJE52nBWj9jxsh/n0pKSvTeDRwo8txzur+2Vj9Ll+r9tPd/wQLd/+GHEvTEc/TRjqV3\nySX6FFZd7e87iAAFn5B4UFzs/BPv2KHb5s93tv35z5GPt7bQtGm6vnOnrt9/v0aXN9wQnN8+Vfzs\nZ/7KZ/3vwkIVOFsua/MAIj/9qfex5eUq5H65/HKNyi+6SKP14493ruF+yrnjDq0M5szRp4Ht2+W/\n9oqIWhr33qtRc0WFVnIiunz++Zr3vvtUXK3V06aNyN136/JLLznXcj8NzJ+v3n1BgbZJWNau1eg7\nlNJSfeq56SYV+s6dtVK68kqR5s2dJ5kLLlCbbPJkp13j7LNVqPv3F+nZ06k83nhDv9drrtFr/PWv\nmt9e/8YbtWIqK1Nrz8v+qgcUfELixfjx2khrxaWkpG4kF44lSzTCtE8HIiqWRx7pRLluqqs18n/l\nFX9lq6xUoTrlFF0vLFSx/fZbp4z33+/vXNGYN8855znnqAACIsceGz5fdrbju8+dG/0aVVXa9tKq\nlchrr+lxjz6q17DnjGSD1Nb6s1/cuBunn31WZMMGrQCsIG/YoPZQly7OE0mbNk6FcMIJ+qRTUqLn\nu+YaFf39+7VdZcAA51q2Pcg+JQHOE0sDoOATEi/27XN6lth1QCNJPxHy7t3B62PH6vFjx+q5Gsrc\nudoIKiJy4onaSFhb6zTeRrKEYqG2Vi0s20Bq2xv+53+C81VVqaf/5z+r4LdqpQJ68KC/6yxcqOft\n2FEr2ooKjdhPPFEj+Hhz6JDTHrBtm26bOFHXx4zR9enT5b9tK/a+nn22Vuah3++XXzqVeWGhyBVX\nOPv27dN70q2b5jn1VK08vHo1xQAFn5BEUVurIvbzn9fv+K++Enn++djsFL/MmOE8HQwZIkFWVDx4\n9lk95/vva3fM3FyR2bPD57/0Us0f6ulH4wc/0OMuu8zZVlurNk8iOPfc4Mpk7VqtbN56S9f37VOb\nx/r7Njq3T1ZuamvV7rL5Q3uKjRih27t3d3pO3X57g7qHUvAJSSTz5qn/m85cdJH6y/GkpkbF3lZW\n0ewTG61ff31s17FdTt98s37ljJUDB+r6/KGNqbYhfMMGx5K7+27v8+3Yod1fAX3Pws1NNzmVWU2N\n9mACRH71q3o34MYi+GEGJSGEhOWkk1Jdgujcf3/83yzNygLOOcdZDzemkWXECB3sbfTo2K5z2WVA\n9+7AyJGxl7E+eI23lJ0dvH777Vqe3r2BH/1Ih4Jw3ws3BQU6k1pJSfCcDYDei3vv1XNlZQEvv6xv\ncX/xhY5t1LJlfP6mMBitINKD4cOHS7HXFHOEEJIuLF8OvPmmzlEQ61hNNTXAU0/pQH/5+c72Q4e8\nR4j1gTFmkYh4jL9Rl4RG+MaYcQAeBpAN4GkRuSeR1yOEkIQzZEjd+Rj8kp2tg+yFUk+xj5WEDZ5m\njMkG8DiAswAMAnCZMWZQoq5HCCEkMokcLXMEgLUisl5EKgG8CuD8BF6PEEJIBBIp+N0BbHatbwls\nI4QQkgJSPh6+MWaSMabYGFO8k7PqEEJIwkik4G8F0NO13iOwLQgRmSoiw0VkeEFBQQKLQwghTZtE\nCv5XAI40xvQxxjQHMAHAjARejxBCSAQS1i1TRKqNMb8C8BG0W+Y0EflPoq5HCCEkMgnthy8i/wfg\n/xJ5DUIIIf5IqzdtjTE7AWys5+GdAOyKY3HiBcsVO+laNpYrNliu2KlP2XqLiK8G0LQS/IZgjCn2\n+3pxMmG5Yiddy8ZyxQbLFTuJLlvKu2USQghJDhR8QghpIjQmwZ+a6gKEgeWKnXQtG8sVGyxX7CS0\nbI3GwyeEEBKZxhThE0IIiUDGC74xZpwxZrUxZq0x5o8pLEdPY8xnxpgVxpj/GGNuCGy/wxiz1Riz\nJPA5O0Xl22CM+SZQhuLAtg7GmI+NMSWBtH2SyzTAdV+WGGP2G2N+m4p7ZoyZZozZYYxZ7toW9v4Y\nY24O/OZWG2POTEHZ7jfGrDLGLDPGvG2MaRfYXmSMOeS6d08muVxhv7tk3bMw5XrNVaYNxpglge3J\nvF/hNCJ5vzO/cyGm4wf6Bu86AH0BNAewFMCgFJWlEMCwwHJrAGug8wDcAeDGNLhXGwB0Ctl2H4A/\nBpb/CODeFH+X2wH0TsU9AzASwDAAy6Pdn8D3uhRACwB9Ar/B7CSX7QwAzQLL97rKVuTOl4J75vnd\nJfOeeZUrZP8UALel4H6F04ik/c4yPcJPmzH3RWSbiCwOLB8AsBLpPxz0+QCeDyw/D+CCFJblNADr\nRKS+L941CBH5AsCekM3h7s/5AF4VkcMi8i2AtdDfYtLKJiIzRaQ6sLoAOjhhUglzz8KRtHsWqVzG\nGAPgEgCvJOLakYigEUn7nWW64KflmPvGmCIAQwEsDGz6deDRe1qybRMXAmCWMWaRMWZSYFsXEdkW\nWN4OoEtqigZAB9dz/xOmwz0Ld3/S7Xc3EcAHrvU+AXtitjHm1BSUx+u7S5d7diqA70SkxLUt6fcr\nRCOS9jvLdMFPO4wxrQC8CeC3IrIfwD+gltNxALZBHydTwSkichx0ysnrjTEj3TtFnyFT0mXL6Giq\n5wF4PbApXe7Zf0nl/YmEMeYWANUAXg5s2gagV+C7/h2A6caYNkksUtp9dyFchuDAIun3y0Mj/kui\nf2eZLvi+xtxPFsaYHOgX+bKIvAUAIvKdiNSISC2Ap5DAR/9IiMjWQLoDwNuBcnxnjCkMlL0QwI5U\nlA1aCS0Wke8CZUyLe4bw9yctfnfGmKsA/BjATwNCgcDj/+7A8iKo79s/WWWK8N2l/J4ZY5oBuBDA\na3Zbsu+Xl0Ygib+zTBf8tBlzP+ANPgNgpYg86Npe6Mo2HsDy0GOTULZ8Y0xruwxt8FsOvVc/D2T7\nOYB3k122AEFRVzrcswDh7s8MABOMMS2MMX0AHAng38ksmDFmHIA/ADhPRA66thcYY7IDy30DZVuf\nxHKF++5Sfs8AnA5glYhssRuSeb/CaQSS+TtLRut0glu+z4a2dq8DcEsKy3EK9FFsGYAlgc/ZAF4E\n8E1g+wwAhSkoW19oa/9SAP+x9wlARwCfACgBMAtAhxSULR/AbgBtXduSfs+gFc42AFVQr/SaSPcH\nwC2B39xqAGeloGxrof6u/a09Gch7UeA7XgJgMYBzk1yusN9dsu6ZV7kC258DMDkkbzLvVziNSNrv\njG/aEkJIEyHTLR1CCCE+oeATQkgTgYJPCCFNBAo+IYQ0ESj4hBDSRKDgE0JIE4GCTwghTQQKPiGE\nNBH+P7Swty0nS9LmAAAAAElFTkSuQmCC\n", 1552 | "text/plain": [ 1553 | "" 1554 | ] 1555 | }, 1556 | "metadata": {}, 1557 | "output_type": "display_data" 1558 | }, 1559 | { 1560 | "data": { 1561 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAD8CAYAAABthzNFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHPtJREFUeJzt3XmUlPWd7/H3F5RF8LLbgCwNiiCaiWDH5Y4LLrlKMpGY\nGxHjwowLSW5M9EZnxJNMdOacUZOoJzdnEidejSHuxmggjlcHEY0Zo2NLQ3BpbLBBGhq6gYAosjT9\nvX/8nrILqIbqpp966qn6vM7p8yz1VNWXp4v69O/3exZzd0RERPbWLekCRESkOCkgREQkJwWEiIjk\npIAQEZGcFBAiIpKTAkJERHJSQIiISE4KCBERyUkBISIiOR2SdAEHY/DgwV5ZWZl0GSIiqfLmm29u\ncPchB9ou1QFRWVlJdXV10mWIiKSKma3KZzt1MYmISE4KCBERyUkBISIiOSkgREQkJwWEiIjkFFtA\nmNkvzazJzN7KWjfQzOabWV00HZD12M1mttzMlpnZeXHVJSIi+YmzBfEr4Py91s0GFrj7OGBBtIyZ\nTQRmAMdFz/m5mXWPsTYRETmA2M6DcPc/mFnlXqunAVOi+TnAS8BN0frH3H0HUG9my4GTgD/FVZ+k\n3wMPQH190lWIJOP442H69Hjfo9AnylW4e2M0vw6oiOaPBF7L2q4hWrcPM5sFzAIYNWpUTGVKsVu/\nHq68MsybJVuLSBIuvrj0AuJT7u5m5p143r3AvQBVVVUdfr4Uv5YW+PnP4YoroH//sG7JEqirg69+\nNSwvWhSmL70EZ56ZSJkiJa/QRzGtN7NhANG0KVq/BhiZtd2IaJ2UoT/8Aa67Di69FFpbw7rZs+Gi\ni2DevLCcCYgTTkimRpFyUOiAmAfMjOZnAnOz1s8ws55mNgYYB/xXgWuTIlFbG6bPPgt33AE7d8Ir\nr4R1V1wB778PNTVw9NHQr19ydYqUujgPc32UMMg83swazOwq4A7g82ZWB5wbLePubwNPAO8AzwHf\ncvfdcdUmxW3ZMujbF77yFbjtNli4ED7+GO66C3bsgDvvDC2ISZOSrlSktMV5FNMl7Tx0Tjvb/wvw\nL3HVI+lRWwvjx8MNN8BTT4XuJjOYOTMEw0MPwdatMGtW0pWKlDadSS1Fp7YWJkyAU08N02XLwljD\noEFw1VUhHEAtCJG4KSCkqGzbBh98EILBLAQCwNlnh+mZZ8JRR4V5BYRIvBQQUlTeey9Mx48P05kz\n4eST4WtfC8vdusE//mMYnzjiiGRqFCkXqb6jnJSOrVvh9tthxIiwPGFCmA4ZAq+9tue2M2eGHxGJ\nlwJCisL8+SEgevYMXUtHH510RSKiLiYpCplrKu3YAZWV0Lt3ouWICGpBSJGorw8nvV1zTTgHQkSS\np4CQolBfD2PGwI9/nHQlIpKhLiYpCpmAEJHioYCQxLnDypUKCJFio4CQxK1fD598ooAQKTYKCElc\n5ggmBYRIcVFASOIUECLFSQEhicsERGVlomWIyF4UEJK4+nqoqIDDDku6EhHJpoCQxOkQV5HipICQ\nxCkgRIqTAkIS1dIS7v+ggBApPgoISVRDA+zerYAQKUYKCEmUDnEVKV4KCEmUAkKkeCkgJFH19eE2\noiNHJl2JiOxNASGJqq8P4XDooUlXIiJ7U0BIQb3wApx7LmzbFpZ1iKtI8VJASMGsXAkXXwwLFsA7\n74R1CgiR4qWAkIKZORM+/DDM19eHS3w3NiogRIqVAkIKYvduePVVuPrqsFxfD6tWhfmxY5OrS0Ta\np4CQgmhqCmdNf+YzMHBgCIj33w+PqQUhUpwUEFIQDQ1hOmJECIT6eli6NKybMCG5ukSkfYckXYCU\nh9Wrw3TkyBAQf/4z9O8f7gExcGCipYlIOxJpQZjZ/zazt83sLTN71Mx6mdlAM5tvZnXRdEAStUk8\n9m5BrFwJ1dUwaVKiZYnIfhQ8IMzsSOA7QJW7Hw90B2YAs4EF7j4OWBAtS4loaICePWHw4BAQO3fC\nihUweXLSlYlIe5IagzgE6G1mhwCHAWuBacCc6PE5wJcTqk1i0NAQWg9mew5KqwUhUrwKHhDuvga4\nE/gAaAS2uPt/ABXu3hhttg6oKHRtEp/Vq0NAwJ4BoRaESPFKootpAKG1MAYYDvQxs8uyt3F3B7yd\n588ys2ozq25ubo69XukamRYEwOjRYVpRAcOGJVeTiOxfEl1M5wL17t7s7ruAp4D/Dqw3s2EA0bQp\n15Pd/V53r3L3qiFDhhSsaOm81lZYs6btiq29eoWwUOtBpLglcZjrB8ApZnYY8AlwDlANfAzMBO6I\npnMTqE1i0NQEu3a1tSAAfv1rOOKI5GoSkQMreEC4++tm9iSwCGgBaoB7gb7AE2Z2FbAKmF7o2iQe\n2Ye4Zpx1VjK1iEj+EjlRzt1vAW7Za/UOQmtCSkyugBCR4qdLbUjsFBAi6aSAkNht2RKmA3RuvEiq\nKCAkdlu3Qo8e4UdE0kMBIbH76CPo2zfpKkSkoxQQEjsFhEg6KSAkdlu3wuGHJ12FiHSUAkJipxaE\nSDopICR2akGIpJMCQmKnFoRIOikgJHYKCJF0UkBI7NTFJJJOCgiJnVoQIumkgJBYtbTA9u0KCJE0\nUkBIrD76KEzVxSSSPgoIiVUmINSCEEkfBYTEauvWMFULQiR9FBASK7UgRNJLASGxUkCIpJcCQmKl\nLiaR9FJASKzUghBJLwWExEoBIZJeCgiJlbqYRNJLASGxyrQg+vRJtg4R6TgFhMRq61Y47DDo3j3p\nSkSkoxQQEitdqE8kvRQQEisFhEh6KSAkVroXhEh6HZJ0AVKaNm+Gujq1IETSTC0IicWPfgSnnALv\nvaeAEEkrBYTE4o03oLUVGhrUxSSSVgoI6XLusGhR27JaECLplEhAmFl/M3vSzGrN7F0zO9XMBprZ\nfDOri6YDkqhNDt7q1bBpE5x+elhWC0IknZJqQfwf4Dl3nwB8FngXmA0scPdxwIJoWVIo03r453+G\n0aPhmGOSrUdEOqfgRzGZWT/gDOBvAdx9J7DTzKYBU6LN5gAvATcVuj45eDU10K0bnHQSrFihs6hF\n0iqvFoSZPWVmXzSzrmhxjAGagQfMrMbM7jOzPkCFuzdG26wDKtqpZZaZVZtZdXNzcxeUI11t0SKY\nMEGX2BBJu3y/8H8OfA2oM7M7zGz8QbznIcBk4B53nwR8zF7dSe7ugOd6srvf6+5V7l41ZMiQgyhD\n4lJTA5MnJ12FiBysvALC3V9w90sJX+wrgRfM7FUz+zszO7SD79kANLj769Hyk9HrrjezYQDRtKmD\nrytFoKkJ1qyBSZOSrkREDlbeXUZmNogwbnA1UEMYaJ4MzO/IG7r7OmB1VivkHOAdYB4wM1o3E5jb\nkdeV4lBTE6ZqQYikX16D1Gb2NDAeeBD4UtZYweNmVt2J9/028LCZ9QDeB/6OEFZPmNlVwCpgeide\nVxKWOYLphBOSrUNEDl6+RzH91N0X5nrA3as6+qbuvhjI9bxzOvpaUlxqamDsWOjfP+lKRORg5dvF\nNNHMPv0vb2YDzOx/xVSTpNiiRRp/ECkV+QbENe6+ObPg7n8BromnJEmrLVvCeQ8afxApDfkGRHcz\ns8yCmXUHesRTkqTV4sVhqhaESGnIdwziOcKA9C+i5a9H60Q+pSOYREpLvgFxEyEUvhktzwfui6Ui\nSa0//QlGjICKnOfAi0ja5BUQ7t4K3BP9iOyjtRUWLoSpU5OuRES6Sr7nQYwDbgcmAr0y6919bEx1\nScq89RY0N8PZZyddiYh0lXwHqR8gtB5agLOAXwMPxVWUpM+LL4bpWWclW4eIdJ18A6K3uy8AzN1X\nufutwBfjK0vS5sUX4aijYNSopCsRka6S7yD1juhS33Vmdi2wBtCNJAWAlhZ4+WW4+OKkKxGRrpRv\nC+I64DDgO8CJwGW0XVhPytzq1fDhh+EGQSJSOg7YgohOirvY3W8EPiJcWE/kU43RpRuPPDLZOkSk\nax2wBeHuu4HTClCLpNS6dWE6bFiydYhI18p3DKLGzOYBvyHcAQ4Ad38qlqokVTItCAWESGnJNyB6\nARuB7KPcHVBACI2N0K0bDB6cdCUi0pXyPZNa4w7SrsbGcHmN7t2TrkREulK+Z1I/QGgx7MHdr+zy\niiR11q1T95JIKcq3i+mZrPlewIXA2q4vR9KosRGGD0+6ChHpavl2Mf02e9nMHgX+GEtFkjqNjXDi\niUlXISJdLd8T5fY2DjiiKwuRdNq9G5qaYOjQpCsRka6W7xjEVvYcg1hHuEeElLmmpnCpb41BiJSe\nfLuYDo+7EEknnSQnUrry6mIyswvNrF/Wcn8z+3J8ZUla6CQ5kdKV7xjELe6+JbPg7puBW+IpSdJE\nASFSuvINiFzb5XuIrJSwTEBokFqk9OQbENVmdreZHRX93A28GWdhkg5r18KAAdCzZ9KViEhXyzcg\nvg3sBB4HHgO2A9+KqyhJh9274ZlndB8IkVKV71FMHwOzY65FUuaFF8LNgu66K+lKRCQO+R7FNN/M\n+mctDzCz5+MrS9Lg/vth0CC44IKkKxGROOTbxTQ4OnIJAHf/CzqTuqxt3gxz58Jll2n8QaRU5RsQ\nrWY2KrNgZpXkuLprR5hZdzOrMbNnouWBUUulLpoOOJjXl3itWAE7d8KUKUlXIiJxyTcgvgf80cwe\nNLOHgJeBmw/yva8D3s1ang0scPdxwAI05lHUNm4M0yFDkq1DROKTV0C4+3NAFbAMeBS4Afiks29q\nZiOALwL3Za2eBsyJ5ucAOlO7iG3YEKa6i5xI6cr3Yn1XE/7iHwEsBk4B/sSetyDtiJ8A/wBkX+Op\nwt2j065YB1R08rWlADIBMWhQsnWISHzy7WK6DvgcsMrdzwImAZv3/5TczOxvgCZ3b/dEO3d32hnj\nMLNZZlZtZtXNzc2dKUG6wIYNYBZOkhOR0pRvQGx39+0AZtbT3WuB8Z18z78GLjCzlYST7s6OxjXW\nm9mw6D2GAU25nuzu97p7lbtXDVEHeGI2boSBA3UfapFSlm9ANETnQfwOmG9mc4FVnXlDd7/Z3Ue4\neyUwA3jR3S8D5gEzo81mAnM78/pSGBs2aPxBpNTleyb1hdHsrWa2EOgHPNfFtdwBPGFmVxHCZ3oX\nv750IQWESOnr8BVZ3f3lrnpzd38JeCma3wic01WvLfHasAEqK5OuQkTi1Nl7UkuZUwtCpPQpIKTD\n3MMgtQJCpLQpICRva9bA6NHw2muwY4cCQqTUKSAkb4sXwwcfwMMPh2UFhEhpU0BI3hoawnT+/DDV\nWdQipU0BIXlbvTpM33svTNWCECltCgjJW6YFkaGAECltCgjJW0NDuP5ShgJCpLQpICRvq1fDKaeE\n+W7doH///W8vIummgJC8uIcWxMknh5sEDRwYQkJESpf+i0teNm+Gbdtg5EioqoJhw5KuSETi1uFr\nMUl5yhzBNHIk/Oxn8NFHydYjIvFTQEheMkcwjRgBY8YkW4uIFIa6mKRdu3ZBS0uYzw4IESkPCghp\n14UXwte/HuZXrw6D0hp7ECkf6mKSdi1ZAkceGeYbGkI4HKJPjEjZ0H93yam1FdavbwuEhgZ1L4mU\nG3UxCRDOc9i0qW1506YwBtHUFJYbG2H48GRqE5FkKCAEgN//PnQnNTeH5cbGMN22DT7+OCwPHZpc\nfSJSeAoIAWDFCti+HZYvD8uZgIAwQL1pkwaoRcqNAkIA2LIlTDMnxK1b1/bY0qVhqoAQKS8KCAHa\nAiJzvkN2C2LJkjBVQIiUFwWEAOFaS7D/gNAYhEh5UUAIsG8XU2NjuO4SqAUhUq4UEALs28W0bl24\n5tLhh4fQMIMjjkiuPhEpPAWEALlbEEOHhns/QAgHnUUtUl4UEAK0BURjY7hAX2Nj6FLKtBo0/iBS\nfhQQAoRB6h49wiU2li8P93vIDgiNP4iUHwWE4B5aEBMmhOU33ghTBYRIeVNACNu3h+suHXdcWM4E\nxNChCgiRcqaAkE/HHzIB8cQTYTp2rMYgRMpZwQPCzEaa2UIze8fM3jaz66L1A81svpnVRdMBha6t\nHGzZAtXVe67LnCQ3Zgz07h0u833jjXD00WpBiJSzJFoQLcAN7j4ROAX4lplNBGYDC9x9HLAgWpYu\ndvfdcNJJ8NxzbesyLYh+/eD44+HMM+G228K6Y4+F7t1h4sTC1yoiySr4ke3u3gg0RvNbzexd4Ehg\nGjAl2mwO8BJwU6HrK3VLl4ZB6UsvhTffhMrKtoDo3x8WLgxHMx16aFh3wgnwl7+EE+ZEpLwkOgZh\nZpXAJOB1oCIKD4B1QEU7z5llZtVmVt2cuXmB5G3ZMpg8GbZuhXvuCeuyWxB9+rSFQ4bCQaQ8JRYQ\nZtYX+C1wvbt/mP2YuzvguZ7n7ve6e5W7Vw3JnOYreWlpgbo6+PznQ8th5cqwPjMG0a9fUpWJSDFK\nJCDM7FBCODzs7k9Fq9eb2bDo8WFAUxK1lbL6+nA464QJ4f7SmesuZbcgREQykjiKyYD7gXfd/e6s\nh+YBM6P5mcDcQtdWqhob4dVXQ/cSwPjx+waEGfTtm1yNIlJ8kmhB/DVwOXC2mS2Ofr4A3AF83szq\ngHOjZekCt90GU6bAK6+E5fHjw6W8166F3btDQPTrB910VoyIZEniKKY/AtbOw+cUspZyUVcXupb+\n9V/D1VkHDgwtiJaWcM7D5s3qXhKRfelvxjJQXx+m27a1XW9pxIgwbWhoa0GIiGRTQJS41tZwtNLw\n4WE5ExCZu8UpIESkPQqIEtfYCDt3wvXXw+jRYSwC2loQq1eHgOjfP7ESRaRI6R5hJS7TvfSZz7Sd\n9wAwaBD06hVaEJs3h0tsiIhkUwuixGUCYsyYPdebhVZETU0IiUyXk4hIhloQJS4TEKNH7/vYiBGw\nYEGYv+yywtUkIumgFkSJq68PA9S9eu37WGYc4pRTdLVWEdmXAqLE1dfv272UkelWuuqqwtUjIumh\ngChx+wuI00+Hv/oruPjiwtYkIumggChhu3aFAej2AmLqVFiyRJfzFpHcFBAlbOXKcKLc2LFJVyIi\naaSAKGG1tWGaOXtaRKQjFBAlLPvy3iIiHaWAKGG1tXDEETBgQNKViEgaKSBK2LJl6l4Skc5TQJSw\n2lp1L4lI5ykgStTGjbBhg1oQItJ5CogSlRmgVkCISGcpIEpU5hBXdTGJSGcpIEpUbS306AGVlUlX\nIiJppYAoQe7wzDPwuc9B9+5JVyMiaaX7QaTUhg1w7bWwbRt86UtwzTXw+9+Hy2tUVcG778J99yVd\npYikmQIipZ59Fh5/HIYMgYULYfp0+MY3YO3aMO7Qp09YJyLSWepiSqmaGujdG37zG/joI7j66hAO\nw4eHI5imT9dVWkXk4CggUmrRIvjsZ+GMM0KL4cknQ2vi9dfh0kth9uykKxSRtFNApFBrKyxeDJMn\ngxlceWVYf8UV4TaiDz0ExxyTbI0ikn4ag0ih99+HDz8MAQHhlqE1NfCd7yRbl4iUFrUgErB9e/hr\nf/Hi/J/zT/8EjzwS5mtqwnTSpDAdNAgefRRGjeraOkWkvKkFkYCnn4YHH4RPPgmDzAeyalUIiOHD\nw/2jFy2CQw+F446Lv1YRKV9qQSTg/vvDdO5caG4O8wsWhLGE1tZ9t//Vr8LJb2vWwPPPw3/+ZwiH\nnj0LVrKIlKGiCwgzO9/MlpnZcjMruWNx6utDGFx0EezaBQ8/HNbfcgs88EA4pyFba2tYf+aZ4Sil\na66BV16BSy4pfO0iUl6KKiDMrDvwM2AqMBG4xMwmJlHL1q2hO+ff/z2/7d3hppvghz/M/fjatXDe\neTBlSjjy6M474aST4Be/gKVLQ6sA2loXGc88E7qYvvlNuPzy8DpTp8KNN3b6nyYikh93L5of4FTg\n+azlm4Gb29v+xBNP9Di0trpfdJE7uPft615be+Dn/PSnYXtwf+yxPR/budP9tNPcDzvM/YIL3G+7\nLaz/3e/C9hUV7t27h/fs2dN906bweEOD+5Ah7sce6759u/vate7f/a77xo1d++8VkfICVHse38kW\nti0OZvZV4Hx3vzpavhw42d2vzbV9VVWVV1dXd/h9li6FGTPaf3zXLqirgxtugDlzQjfP0KH7f833\n3gt/2W/cGAaRx45te2zbtnCNpEce2bdr6IYb4O67Ydo0uPXWcGTSqFHQt2+43tLHH8Mbb8Cxx3b4\nnykikpOZvenuVQfaLnVHMZnZLGAWwKhOHtfZuzdMPEDH1eWXw/e/H8YKfvITaGnZ//ZnnAG33x7C\n4Ac/gC1b9nz87/8+97jBHXeEMLjkknBzn+9/v+1eDmYwa5bCQUSSUWwtiFOBW939vGj5ZgB3vz3X\n9p1tQYiIlLN8WxBFNUgNvAGMM7MxZtYDmAHMS7gmEZGyVFRdTO7eYmbXAs8D3YFfuvvbCZclIlKW\niiogANz9WeDZpOsQESl3xdbFJCIiRUIBISIiOSkgREQkJwWEiIjkpIAQEZGciupEuY4ys2Zg1UG8\nxGBgQxeV05VUV8eoro4r1tpUV8d0tq7R7j7kQBulOiAOlplV53M2YaGpro5RXR1XrLWpro6Juy51\nMYmISE4KCBERyancA+LepAtoh+rqGNXVccVam+rqmFjrKusxCBERaV+5tyBERKQdZRkQZna+mS0z\ns+VmNjvBOkaa2UIze8fM3jaz66L1t5rZGjNbHP18IYHaVprZ0uj9q6N1A81svpnVRdMBCdQ1Pmu/\nLDazD83s+iT2mZn90syazOytrHXt7iMzuzn6zC0zs/MKXNePzazWzP5sZk+bWf9ofaWZfZK13/4t\nrrr2U1u7v7uE99njWTWtNLPF0fqC7bP9fEcU5nOWz31JS+mHcBnxFcBYoAewBJiYUC3DgMnR/OHA\ne8BE4FbgxoT300pg8F7rfgTMjuZnAz8sgt/lOmB0EvsMOAOYDLx1oH0U/V6XAD2BMdFnsHsB6/of\nwCHR/A+z6qrM3i6hfZbzd5f0Ptvr8buAHxR6n+3nO6Ign7NybEGcBCx39/fdfSfwGDAtiULcvdHd\nF0XzW4F3gSOTqCVP04A50fwc4MsJ1gJwDrDC3Q/mZMlOc/c/AJv2Wt3ePpoGPObuO9y9HlhO+CwW\npC53/w93z9w49zVgRBzvfSDt7LP2JLrPMszMgOnAo3G89/7s5zuiIJ+zcgyII4HVWcsNFMGXsplV\nApOA16NV3466A36ZRFcO4MALZvZmdB9wgAp3b4zm1wEVCdSVbQZ7/qdNep9B+/uomD53VwL/L2t5\nTNRV8rKZnZ5QTbl+d8Wyz04H1rt7Xda6gu+zvb4jCvI5K8eAKDpm1hf4LXC9u38I3EPoAjsBaCQ0\nbwvtNHc/AZgKfMvMzsh+0EN7NrFD4CzckvYC4DfRqmLYZ3tIeh/lYmbfA1qAh6NVjcCo6Hf9XeAR\nM/tvBS6r6H53e7mEPf8QKfg+y/Ed8ak4P2flGBBrgJFZyyOidYkws0MJv/iH3f0pAHdf7+673b0V\n+L/E1KzeH3dfE02bgKejGtab2bCo7mFAU6HryjIVWOTu66E49lmkvX2U+OfOzP4W+Bvg0uhLhagr\nYmM0/yahz/qYQta1n99dMeyzQ4CvAI9n1hV6n+X6jqBAn7NyDIg3gHFmNib6K3QGMC+JQqK+zfuB\nd9397qz1w7I2uxB4a+/nxlxXHzM7PDNPGOB8i7CfZkabzQTmFrKuvezxV13S+yxLe/toHjDDzHqa\n2RhgHPBfhSrKzM4H/gG4wN23Za0fYmbdo/mxUV3vF6qu6H3b+90lus8i5wK17t6QWVHIfdbedwSF\n+pwVYiS+2H6ALxCOBlgBfC/BOk4jNA3/DCyOfr4APAgsjdbPA4YVuK6xhCMhlgBvZ/YRMAhYANQB\nLwADE9pvfYCNQL+sdQXfZ4SAagR2Efp6r9rfPgK+F33mlgFTC1zXckLfdOZz9m/Rtv8z+h0vBhYB\nX0pgn7X7u0tyn0XrfwV8Y69tC7bP9vMdUZDPmc6kFhGRnMqxi0lERPKggBARkZwUECIikpMCQkRE\nclJAiIhITgoIERHJSQEhIiI5KSBERCSn/w9VvWVA2xS/eQAAAABJRU5ErkJggg==\n", 1562 | "text/plain": [ 1563 | "" 1564 | ] 1565 | }, 1566 | "metadata": {}, 1567 | "output_type": "display_data" 1568 | } 1569 | ], 1570 | "source": [ 1571 | "#print(overallError_history)\n", 1572 | "x_range = range(max_iter//100)\n", 1573 | "plt.plot(x_range,overallError_history,'r-')\n", 1574 | "plt.ylabel('overallError')\n", 1575 | "plt.show()\n", 1576 | "\n", 1577 | "plt.plot(x_range,accuracy_history,'b-')\n", 1578 | "plt.ylabel('accuracy')\n", 1579 | "plt.show()" 1580 | ] 1581 | }, 1582 | { 1583 | "cell_type": "markdown", 1584 | "metadata": {}, 1585 | "source": [ 1586 | "## max binary dimension 이상의 수에서도 될까?\n", 1587 | "#### Is it possible to use RNN trained on the 8 bytes digits to 10 bytes or more?\n", 1588 | "\n", 1589 | "* 8 bytes에서 학습된 RNN이 10bytes와 같이 그 이상의 자릿수를 가진 이진수에서도 잘 동작하는지 확인하자!" 1590 | ] 1591 | }, 1592 | { 1593 | "cell_type": "code", 1594 | "execution_count": 9, 1595 | "metadata": { 1596 | "collapsed": false 1597 | }, 1598 | "outputs": [ 1599 | { 1600 | "data": { 1601 | "text/plain": [ 1602 | "array([0, 0, 0, 0, 0, 0, 0, 1, 1, 0])" 1603 | ] 1604 | }, 1605 | "execution_count": 9, 1606 | "metadata": {}, 1607 | "output_type": "execute_result" 1608 | } 1609 | ], 1610 | "source": [ 1611 | "# Test for the codes (garbage)\n", 1612 | "# ====================================================== #\n", 1613 | "#int('{0:09b}'.format(6))\n", 1614 | "#list(format(6, \"08b\"))\n", 1615 | "#results = list(map(int, list(format(6, \"08b\"))))\n", 1616 | "#results\n", 1617 | "#str(10)+\"b\"\n", 1618 | "# ====================================================== #\n", 1619 | "\n", 1620 | "# create a binary digit over 8 bytes\n", 1621 | "max_binary_dim = 10\n", 1622 | "largest_number = pow(2,max_binary_dim)\n", 1623 | "digit_key = \"0\"+str(max_binary_dim)+\"b\"\n", 1624 | "np.array(list(map(int,list(format(6, digit_key)))))" 1625 | ] 1626 | }, 1627 | { 1628 | "cell_type": "code", 1629 | "execution_count": 10, 1630 | "metadata": { 1631 | "collapsed": true 1632 | }, 1633 | "outputs": [], 1634 | "source": [ 1635 | "# initialization\n", 1636 | "overallError_history = list()\n", 1637 | "accuracy = list()\n", 1638 | "accuracy_history = list()\n", 1639 | "accuracy_count = 0" 1640 | ] 1641 | }, 1642 | { 1643 | "cell_type": "markdown", 1644 | "metadata": {}, 1645 | "source": [ 1646 | "## Everything is exactly the same except the digit length tested\n", 1647 | "\n", 1648 | "* This time, we only need to calculate the feed-forward step to see the prediction." 1649 | ] 1650 | }, 1651 | { 1652 | "cell_type": "code", 1653 | "execution_count": 11, 1654 | "metadata": { 1655 | "collapsed": false, 1656 | "scrolled": false 1657 | }, 1658 | "outputs": [ 1659 | { 1660 | "name": "stdout", 1661 | "output_type": "stream", 1662 | "text": [ 1663 | "Error:[ 0.17447237]\n", 1664 | "Pred:[1 1 0 0 0 1 1 1 1 0]\n", 1665 | "True:[1 1 0 0 0 1 1 1 1 0]\n", 1666 | "True\n", 1667 | "384 + 414 = 798\n", 1668 | "------------\n", 1669 | "Error:[ 0.27651708]\n", 1670 | "Pred:[0 1 1 0 1 1 0 1 0 1]\n", 1671 | "True:[0 1 1 0 1 1 0 1 0 1]\n", 1672 | "True\n", 1673 | "246 + 191 = 437\n", 1674 | "------------\n", 1675 | "Error:[ 0.18742734]\n", 1676 | "Pred:[1 0 1 1 1 0 0 1 1 1]\n", 1677 | "True:[1 0 1 1 1 0 0 1 1 1]\n", 1678 | "True\n", 1679 | "419 + 324 = 743\n", 1680 | "------------\n", 1681 | "Error:[ 0.26819747]\n", 1682 | "Pred:[1 0 1 1 1 0 1 1 1 1]\n", 1683 | "True:[1 0 1 1 1 0 1 1 1 1]\n", 1684 | "True\n", 1685 | "312 + 439 = 751\n", 1686 | "------------\n", 1687 | "Error:[ 0.15437666]\n", 1688 | "Pred:[1 1 0 1 1 0 1 0 0 1]\n", 1689 | "True:[1 1 0 1 1 0 1 0 0 1]\n", 1690 | "True\n", 1691 | "416 + 457 = 873\n", 1692 | "------------\n", 1693 | "Error:[ 0.25902133]\n", 1694 | "Pred:[1 1 0 0 1 1 1 0 1 1]\n", 1695 | "True:[1 1 0 0 1 1 1 0 1 1]\n", 1696 | "True\n", 1697 | "333 + 494 = 827\n", 1698 | "------------\n", 1699 | "Error:[ 0.21948908]\n", 1700 | "Pred:[1 0 1 0 0 1 1 0 0 1]\n", 1701 | "True:[1 0 1 0 0 1 1 0 0 1]\n", 1702 | "True\n", 1703 | "312 + 353 = 665\n", 1704 | "------------\n", 1705 | "Error:[ 0.30909945]\n", 1706 | "Pred:[1 0 0 1 1 0 1 1 1 1]\n", 1707 | "True:[1 0 0 1 1 0 1 1 1 1]\n", 1708 | "True\n", 1709 | "503 + 120 = 623\n", 1710 | "------------\n", 1711 | "Error:[ 0.29611804]\n", 1712 | "Pred:[1 1 1 0 1 1 0 0 0 1]\n", 1713 | "True:[1 1 1 0 1 1 0 0 0 1]\n", 1714 | "True\n", 1715 | "486 + 459 = 945\n", 1716 | "------------\n", 1717 | "Error:[ 0.3206195]\n", 1718 | "Pred:[1 0 0 0 0 0 1 0 0 1]\n", 1719 | "True:[1 0 0 0 0 0 1 0 0 1]\n", 1720 | "True\n", 1721 | "397 + 124 = 521\n", 1722 | "------------\n", 1723 | "Error:[ 0.3040861]\n", 1724 | "Pred:[1 0 1 0 0 1 0 0 0 1]\n", 1725 | "True:[1 0 1 0 0 1 0 0 0 1]\n", 1726 | "True\n", 1727 | "278 + 379 = 657\n", 1728 | "------------\n", 1729 | "Error:[ 0.20313793]\n", 1730 | "Pred:[1 0 0 1 1 0 1 1 0 0]\n", 1731 | "True:[1 0 0 1 1 0 1 1 0 0]\n", 1732 | "True\n", 1733 | "171 + 449 = 620\n", 1734 | "------------\n", 1735 | "Error:[ 0.32864245]\n", 1736 | "Pred:[1 0 1 0 0 1 0 0 1 0]\n", 1737 | "True:[1 0 1 0 0 1 0 0 1 0]\n", 1738 | "True\n", 1739 | "499 + 159 = 658\n", 1740 | "------------\n", 1741 | "Error:[ 0.2906054]\n", 1742 | "Pred:[0 1 0 1 1 0 0 0 0 0]\n", 1743 | "True:[0 1 0 1 1 0 0 0 0 0]\n", 1744 | "True\n", 1745 | "211 + 141 = 352\n", 1746 | "------------\n", 1747 | "Error:[ 0.15756375]\n", 1748 | "Pred:[1 0 0 0 1 0 1 1 1 0]\n", 1749 | "True:[1 0 0 0 1 0 1 1 1 0]\n", 1750 | "True\n", 1751 | "292 + 266 = 558\n", 1752 | "------------\n", 1753 | "Error:[ 0.30643925]\n", 1754 | "Pred:[0 1 1 0 1 0 0 0 1 0]\n", 1755 | "True:[0 1 1 0 1 0 0 0 1 0]\n", 1756 | "True\n", 1757 | "316 + 102 = 418\n", 1758 | "------------\n", 1759 | "Error:[ 0.24416273]\n", 1760 | "Pred:[0 0 1 1 0 0 0 1 1 1]\n", 1761 | "True:[0 0 1 1 0 0 0 1 1 1]\n", 1762 | "True\n", 1763 | "125 + 74 = 199\n", 1764 | "------------\n", 1765 | "Error:[ 0.17996497]\n", 1766 | "Pred:[0 1 1 0 1 0 0 0 0 1]\n", 1767 | "True:[0 1 1 0 1 0 0 0 0 1]\n", 1768 | "True\n", 1769 | "240 + 177 = 417\n", 1770 | "------------\n", 1771 | "Error:[ 0.14176455]\n", 1772 | "Pred:[0 0 1 0 1 1 1 0 0 1]\n", 1773 | "True:[0 0 1 0 1 1 1 0 0 1]\n", 1774 | "True\n", 1775 | "72 + 113 = 185\n", 1776 | "------------\n", 1777 | "Error:[ 0.21626245]\n", 1778 | "Pred:[0 1 0 1 0 1 0 1 1 1]\n", 1779 | "True:[0 1 0 1 0 1 0 1 1 1]\n", 1780 | "True\n", 1781 | "98 + 245 = 343\n", 1782 | "------------\n", 1783 | "Error:[ 0.2611522]\n", 1784 | "Pred:[1 0 1 0 1 1 0 0 0 0]\n", 1785 | "True:[1 0 1 0 1 1 0 0 0 0]\n", 1786 | "True\n", 1787 | "440 + 248 = 688\n", 1788 | "------------\n", 1789 | "Error:[ 0.16428819]\n", 1790 | "Pred:[0 0 1 0 1 0 1 1 0 1]\n", 1791 | "True:[0 0 1 0 1 0 1 1 0 1]\n", 1792 | "True\n", 1793 | "10 + 163 = 173\n", 1794 | "------------\n", 1795 | "Error:[ 0.20849092]\n", 1796 | "Pred:[1 0 1 0 1 0 1 1 0 1]\n", 1797 | "True:[1 0 1 0 1 0 1 1 0 1]\n", 1798 | "True\n", 1799 | "266 + 419 = 685\n", 1800 | "------------\n", 1801 | "Error:[ 0.2461282]\n", 1802 | "Pred:[0 1 1 0 1 1 1 1 1 1]\n", 1803 | "True:[0 1 1 0 1 1 1 1 1 1]\n", 1804 | "True\n", 1805 | "345 + 102 = 447\n", 1806 | "------------\n", 1807 | "Error:[ 0.31476154]\n", 1808 | "Pred:[1 0 0 0 0 0 0 0 0 1]\n", 1809 | "True:[1 0 0 0 0 0 0 0 0 1]\n", 1810 | "True\n", 1811 | "181 + 332 = 513\n", 1812 | "------------\n", 1813 | "Error:[ 0.16605737]\n", 1814 | "Pred:[0 1 0 0 1 1 1 1 0 1]\n", 1815 | "True:[0 1 0 0 1 1 1 1 0 1]\n", 1816 | "True\n", 1817 | "172 + 145 = 317\n", 1818 | "------------\n", 1819 | "Error:[ 0.27297363]\n", 1820 | "Pred:[0 1 1 0 0 1 0 0 0 1]\n", 1821 | "True:[0 1 1 0 0 1 0 0 0 1]\n", 1822 | "True\n", 1823 | "221 + 180 = 401\n", 1824 | "------------\n", 1825 | "Error:[ 0.2958943]\n", 1826 | "Pred:[1 1 0 0 1 1 0 1 1 0]\n", 1827 | "True:[1 1 0 0 1 1 0 1 1 0]\n", 1828 | "True\n", 1829 | "447 + 375 = 822\n", 1830 | "------------\n", 1831 | "Error:[ 0.31495512]\n", 1832 | "Pred:[1 0 0 0 0 1 0 0 0 1]\n", 1833 | "True:[1 0 0 0 0 1 0 0 0 1]\n", 1834 | "True\n", 1835 | "43 + 486 = 529\n", 1836 | "------------\n", 1837 | "Error:[ 0.3297667]\n", 1838 | "Pred:[1 1 0 0 0 1 1 1 0 0]\n", 1839 | "True:[1 1 0 0 0 1 1 1 0 0]\n", 1840 | "True\n", 1841 | "477 + 319 = 796\n", 1842 | "------------\n", 1843 | "Error:[ 0.26117005]\n", 1844 | "Pred:[0 1 1 0 0 1 1 0 0 0]\n", 1845 | "True:[0 1 1 0 0 1 1 0 0 0]\n", 1846 | "True\n", 1847 | "359 + 49 = 408\n", 1848 | "------------\n", 1849 | "Error:[ 0.35877471]\n", 1850 | "Pred:[1 1 0 0 0 0 1 1 0 1]\n", 1851 | "True:[1 1 0 0 0 0 1 1 0 1]\n", 1852 | "True\n", 1853 | "318 + 463 = 781\n", 1854 | "------------\n", 1855 | "Error:[ 0.21790624]\n", 1856 | "Pred:[0 1 0 1 1 1 0 1 0 1]\n", 1857 | "True:[0 1 0 1 1 1 0 1 0 1]\n", 1858 | "True\n", 1859 | "263 + 110 = 373\n", 1860 | "------------\n", 1861 | "Error:[ 0.30378748]\n", 1862 | "Pred:[1 0 0 1 1 0 1 1 0 1]\n", 1863 | "True:[1 0 0 1 1 0 1 1 0 1]\n", 1864 | "True\n", 1865 | "430 + 191 = 621\n", 1866 | "------------\n", 1867 | "Error:[ 0.28304278]\n", 1868 | "Pred:[1 0 0 1 1 1 0 0 1 1]\n", 1869 | "True:[1 0 0 1 1 1 0 0 1 1]\n", 1870 | "True\n", 1871 | "202 + 425 = 627\n", 1872 | "------------\n", 1873 | "Error:[ 0.19708022]\n", 1874 | "Pred:[1 0 0 1 0 0 1 1 0 0]\n", 1875 | "True:[1 0 0 1 0 0 1 1 0 0]\n", 1876 | "True\n", 1877 | "262 + 326 = 588\n", 1878 | "------------\n", 1879 | "Error:[ 0.15685053]\n", 1880 | "Pred:[1 1 0 1 0 0 1 0 1 1]\n", 1881 | "True:[1 1 0 1 0 0 1 0 1 1]\n", 1882 | "True\n", 1883 | "384 + 459 = 843\n", 1884 | "------------\n", 1885 | "Error:[ 0.12434756]\n", 1886 | "Pred:[0 1 0 0 1 0 0 1 1 0]\n", 1887 | "True:[0 1 0 0 1 0 0 1 1 0]\n", 1888 | "True\n", 1889 | "256 + 38 = 294\n", 1890 | "------------\n", 1891 | "Error:[ 0.20826115]\n", 1892 | "Pred:[0 1 1 1 1 1 0 1 1 0]\n", 1893 | "True:[0 1 1 1 1 1 0 1 1 0]\n", 1894 | "True\n", 1895 | "191 + 311 = 502\n", 1896 | "------------\n", 1897 | "Error:[ 0.166506]\n", 1898 | "Pred:[0 1 1 1 1 1 1 1 0 0]\n", 1899 | "True:[0 1 1 1 1 1 1 1 0 0]\n", 1900 | "True\n", 1901 | "431 + 77 = 508\n", 1902 | "------------\n", 1903 | "Error:[ 0.18177219]\n", 1904 | "Pred:[0 1 0 1 0 0 1 0 1 0]\n", 1905 | "True:[0 1 0 1 0 0 1 0 1 0]\n", 1906 | "True\n", 1907 | "169 + 161 = 330\n", 1908 | "------------\n", 1909 | "Error:[ 0.29100458]\n", 1910 | "Pred:[0 0 1 1 0 1 1 0 0 0]\n", 1911 | "True:[0 0 1 1 0 1 1 0 0 0]\n", 1912 | "True\n", 1913 | "158 + 58 = 216\n", 1914 | "------------\n", 1915 | "Error:[ 0.10837928]\n", 1916 | "Pred:[0 0 0 0 1 1 1 0 0 1]\n", 1917 | "True:[0 0 0 0 1 1 1 0 0 1]\n", 1918 | "True\n", 1919 | "56 + 1 = 57\n", 1920 | "------------\n", 1921 | "Error:[ 0.28396873]\n", 1922 | "Pred:[0 1 1 0 0 1 0 1 1 1]\n", 1923 | "True:[0 1 1 0 0 1 0 1 1 1]\n", 1924 | "True\n", 1925 | "125 + 282 = 407\n", 1926 | "------------\n", 1927 | "Error:[ 0.1608341]\n", 1928 | "Pred:[1 1 0 0 0 1 0 1 0 1]\n", 1929 | "True:[1 1 0 0 0 1 0 1 0 1]\n", 1930 | "True\n", 1931 | "324 + 465 = 789\n", 1932 | "------------\n", 1933 | "Error:[ 0.28731671]\n", 1934 | "Pred:[1 0 0 0 0 1 1 1 1 1]\n", 1935 | "True:[1 0 0 0 0 1 1 1 1 1]\n", 1936 | "True\n", 1937 | "288 + 255 = 543\n", 1938 | "------------\n", 1939 | "Error:[ 0.15308156]\n", 1940 | "Pred:[0 1 0 1 0 0 0 0 1 1]\n", 1941 | "True:[0 1 0 1 0 0 0 0 1 1]\n", 1942 | "True\n", 1943 | "128 + 195 = 323\n", 1944 | "------------\n", 1945 | "Error:[ 0.15743634]\n", 1946 | "Pred:[0 0 0 1 0 1 1 1 0 1]\n", 1947 | "True:[0 0 0 1 0 1 1 1 0 1]\n", 1948 | "True\n", 1949 | "71 + 22 = 93\n", 1950 | "------------\n", 1951 | "Error:[ 0.21407874]\n", 1952 | "Pred:[1 0 0 0 0 0 1 1 1 0]\n", 1953 | "True:[1 0 0 0 0 0 1 1 1 0]\n", 1954 | "True\n", 1955 | "365 + 161 = 526\n", 1956 | "------------\n", 1957 | "Error:[ 0.2708268]\n", 1958 | "Pred:[0 1 0 1 0 0 0 1 0 1]\n", 1959 | "True:[0 1 0 1 0 0 0 1 0 1]\n", 1960 | "True\n", 1961 | "287 + 38 = 325\n", 1962 | "------------\n", 1963 | "Error:[ 0.22848946]\n", 1964 | "Pred:[1 0 0 1 1 0 0 1 0 0]\n", 1965 | "True:[1 0 0 1 1 0 0 1 0 0]\n", 1966 | "True\n", 1967 | "176 + 436 = 612\n", 1968 | "------------\n", 1969 | "Error:[ 0.23140022]\n", 1970 | "Pred:[1 1 0 1 0 1 0 1 1 0]\n", 1971 | "True:[1 1 0 1 0 1 0 1 1 0]\n", 1972 | "True\n", 1973 | "418 + 436 = 854\n", 1974 | "------------\n", 1975 | "Error:[ 0.27559053]\n", 1976 | "Pred:[1 1 1 0 0 1 1 0 1 0]\n", 1977 | "True:[1 1 1 0 0 1 1 0 1 0]\n", 1978 | "True\n", 1979 | "413 + 509 = 922\n", 1980 | "------------\n", 1981 | "Error:[ 0.20241596]\n", 1982 | "Pred:[0 0 1 1 0 0 0 0 1 1]\n", 1983 | "True:[0 0 1 1 0 0 0 0 1 1]\n", 1984 | "True\n", 1985 | "48 + 147 = 195\n", 1986 | "------------\n", 1987 | "Error:[ 0.16887661]\n", 1988 | "Pred:[0 0 1 1 0 1 1 1 1 1]\n", 1989 | "True:[0 0 1 1 0 1 1 1 1 1]\n", 1990 | "True\n", 1991 | "81 + 142 = 223\n", 1992 | "------------\n", 1993 | "Error:[ 0.26463917]\n", 1994 | "Pred:[0 1 1 1 1 0 0 0 1 1]\n", 1995 | "True:[0 1 1 1 1 0 0 0 1 1]\n", 1996 | "True\n", 1997 | "118 + 365 = 483\n", 1998 | "------------\n", 1999 | "Error:[ 0.28126299]\n", 2000 | "Pred:[1 0 1 1 0 0 0 0 0 1]\n", 2001 | "True:[1 0 1 1 0 0 0 0 0 1]\n", 2002 | "True\n", 2003 | "196 + 509 = 705\n", 2004 | "------------\n", 2005 | "Error:[ 0.24976575]\n", 2006 | "Pred:[0 1 1 1 0 1 1 1 1 1]\n", 2007 | "True:[0 1 1 1 0 1 1 1 1 1]\n", 2008 | "True\n", 2009 | "45 + 434 = 479\n", 2010 | "------------\n", 2011 | "Error:[ 0.2716244]\n", 2012 | "Pred:[0 1 0 0 0 1 0 0 1 1]\n", 2013 | "True:[0 1 0 0 0 1 0 0 1 1]\n", 2014 | "True\n", 2015 | "89 + 186 = 275\n", 2016 | "------------\n", 2017 | "Error:[ 0.21204717]\n", 2018 | "Pred:[0 1 0 1 1 1 0 0 0 1]\n", 2019 | "True:[0 1 0 1 1 1 0 0 0 1]\n", 2020 | "True\n", 2021 | "330 + 39 = 369\n", 2022 | "------------\n", 2023 | "Error:[ 0.1601862]\n", 2024 | "Pred:[0 0 1 1 0 1 1 1 0 0]\n", 2025 | "True:[0 0 1 1 0 1 1 1 0 0]\n", 2026 | "True\n", 2027 | "18 + 202 = 220\n", 2028 | "------------\n", 2029 | "Error:[ 0.11763846]\n", 2030 | "Pred:[0 1 1 1 1 1 1 1 0 0]\n", 2031 | "True:[0 1 1 1 1 1 1 1 0 0]\n", 2032 | "True\n", 2033 | "276 + 232 = 508\n", 2034 | "------------\n", 2035 | "Error:[ 0.3060972]\n", 2036 | "Pred:[0 1 1 0 0 0 0 1 1 0]\n", 2037 | "True:[0 1 1 0 0 0 0 1 1 0]\n", 2038 | "True\n", 2039 | "92 + 298 = 390\n", 2040 | "------------\n", 2041 | "Error:[ 0.28846954]\n", 2042 | "Pred:[0 1 1 0 1 1 1 0 1 1]\n", 2043 | "True:[0 1 1 0 1 1 1 0 1 1]\n", 2044 | "True\n", 2045 | "102 + 341 = 443\n", 2046 | "------------\n", 2047 | "Error:[ 0.36933459]\n", 2048 | "Pred:[1 1 1 1 0 0 0 1 0 0]\n", 2049 | "True:[1 1 1 1 0 0 0 1 0 0]\n", 2050 | "True\n", 2051 | "494 + 470 = 964\n", 2052 | "------------\n", 2053 | "Error:[ 0.31882406]\n", 2054 | "Pred:[1 0 0 1 0 1 0 0 0 0]\n", 2055 | "True:[1 0 0 1 0 1 0 0 0 0]\n", 2056 | "True\n", 2057 | "244 + 348 = 592\n", 2058 | "------------\n", 2059 | "Error:[ 0.19395465]\n", 2060 | "Pred:[0 0 0 0 1 1 0 0 0 0]\n", 2061 | "True:[0 0 0 0 1 1 0 0 0 0]\n", 2062 | "True\n", 2063 | "45 + 3 = 48\n", 2064 | "------------\n", 2065 | "Error:[ 0.16826027]\n", 2066 | "Pred:[0 1 0 1 0 1 1 0 1 1]\n", 2067 | "True:[0 1 0 1 0 1 1 0 1 1]\n", 2068 | "True\n", 2069 | "129 + 218 = 347\n", 2070 | "------------\n", 2071 | "Error:[ 0.28807155]\n", 2072 | "Pred:[0 1 0 0 1 0 1 0 1 1]\n", 2073 | "True:[0 1 0 0 1 0 1 0 1 1]\n", 2074 | "True\n", 2075 | "94 + 205 = 299\n", 2076 | "------------\n", 2077 | "Error:[ 0.14074882]\n", 2078 | "Pred:[0 0 0 1 0 0 1 1 1 1]\n", 2079 | "True:[0 0 0 1 0 0 1 1 1 1]\n", 2080 | "True\n", 2081 | "8 + 71 = 79\n", 2082 | "------------\n", 2083 | "Error:[ 0.33998782]\n", 2084 | "Pred:[1 0 0 0 1 1 1 1 0 1]\n", 2085 | "True:[1 0 0 0 1 1 1 1 0 1]\n", 2086 | "True\n", 2087 | "99 + 474 = 573\n", 2088 | "------------\n", 2089 | "Error:[ 0.20388869]\n", 2090 | "Pred:[0 1 1 1 1 1 1 0 0 0]\n", 2091 | "True:[0 1 1 1 1 1 1 0 0 0]\n", 2092 | "True\n", 2093 | "409 + 95 = 504\n", 2094 | "------------\n", 2095 | "Error:[ 0.27113412]\n", 2096 | "Pred:[0 1 1 0 0 0 0 1 1 1]\n", 2097 | "True:[0 1 1 0 0 0 0 1 1 1]\n", 2098 | "True\n", 2099 | "60 + 331 = 391\n", 2100 | "------------\n", 2101 | "Error:[ 0.27800925]\n", 2102 | "Pred:[1 0 0 0 0 1 0 0 0 1]\n", 2103 | "True:[1 0 0 0 0 1 0 0 0 1]\n", 2104 | "True\n", 2105 | "356 + 173 = 529\n", 2106 | "------------\n", 2107 | "Error:[ 0.22105296]\n", 2108 | "Pred:[1 1 1 0 0 0 0 1 0 1]\n", 2109 | "True:[1 1 1 0 0 0 0 1 0 1]\n", 2110 | "True\n", 2111 | "437 + 464 = 901\n", 2112 | "------------\n", 2113 | "Error:[ 0.2230257]\n", 2114 | "Pred:[1 1 1 0 1 0 0 0 0 1]\n", 2115 | "True:[1 1 1 0 1 0 0 0 0 1]\n", 2116 | "True\n", 2117 | "464 + 465 = 929\n", 2118 | "------------\n", 2119 | "Error:[ 0.18593703]\n", 2120 | "Pred:[0 1 0 1 0 1 1 1 1 1]\n", 2121 | "True:[0 1 0 1 0 1 1 1 1 1]\n", 2122 | "True\n", 2123 | "52 + 299 = 351\n", 2124 | "------------\n", 2125 | "Error:[ 0.27264573]\n", 2126 | "Pred:[0 1 1 1 0 1 0 0 0 0]\n", 2127 | "True:[0 1 1 1 0 1 0 0 0 0]\n", 2128 | "True\n", 2129 | "429 + 35 = 464\n", 2130 | "------------\n", 2131 | "Error:[ 0.2599256]\n", 2132 | "Pred:[1 0 0 1 0 1 1 1 1 1]\n", 2133 | "True:[1 0 0 1 0 1 1 1 1 1]\n", 2134 | "True\n", 2135 | "430 + 177 = 607\n", 2136 | "------------\n", 2137 | "Error:[ 0.20877375]\n", 2138 | "Pred:[1 0 1 0 1 0 0 1 1 0]\n", 2139 | "True:[1 0 1 0 1 0 0 1 1 0]\n", 2140 | "True\n", 2141 | "393 + 285 = 678\n", 2142 | "------------\n", 2143 | "Error:[ 0.32661152]\n", 2144 | "Pred:[1 0 0 0 0 0 0 1 1 1]\n", 2145 | "True:[1 0 0 0 0 0 0 1 1 1]\n", 2146 | "True\n", 2147 | "317 + 202 = 519\n", 2148 | "------------\n", 2149 | "Error:[ 0.28557791]\n", 2150 | "Pred:[1 0 1 0 0 1 0 0 0 1]\n", 2151 | "True:[1 0 1 0 0 1 0 0 0 1]\n", 2152 | "True\n", 2153 | "504 + 153 = 657\n", 2154 | "------------\n", 2155 | "Error:[ 0.21642076]\n", 2156 | "Pred:[1 0 0 0 1 1 0 1 0 1]\n", 2157 | "True:[1 0 0 0 1 1 0 1 0 1]\n", 2158 | "True\n", 2159 | "402 + 163 = 565\n", 2160 | "------------\n", 2161 | "Error:[ 0.20254958]\n", 2162 | "Pred:[0 1 1 1 0 1 0 1 1 0]\n", 2163 | "True:[0 1 1 1 0 1 0 1 1 0]\n", 2164 | "True\n", 2165 | "432 + 38 = 470\n", 2166 | "------------\n", 2167 | "Error:[ 0.15904447]\n", 2168 | "Pred:[1 0 0 1 1 1 1 0 1 1]\n", 2169 | "True:[1 0 0 1 1 1 1 0 1 1]\n", 2170 | "True\n", 2171 | "282 + 353 = 635\n", 2172 | "------------\n", 2173 | "Error:[ 0.20761984]\n", 2174 | "Pred:[1 0 1 0 1 1 0 1 0 1]\n", 2175 | "True:[1 0 1 0 1 1 0 1 0 1]\n", 2176 | "True\n", 2177 | "387 + 306 = 693\n", 2178 | "------------\n", 2179 | "Error:[ 0.29115035]\n", 2180 | "Pred:[1 0 0 1 1 0 0 1 1 0]\n", 2181 | "True:[1 0 0 1 1 0 0 1 1 0]\n", 2182 | "True\n", 2183 | "463 + 151 = 614\n", 2184 | "------------\n", 2185 | "Error:[ 0.27264868]\n", 2186 | "Pred:[1 1 0 0 0 1 0 0 0 0]\n", 2187 | "True:[1 1 0 0 0 1 0 0 0 0]\n", 2188 | "True\n", 2189 | "408 + 376 = 784\n", 2190 | "------------\n", 2191 | "Error:[ 0.30618218]\n", 2192 | "Pred:[1 0 1 0 1 1 0 1 0 1]\n", 2193 | "True:[1 0 1 0 1 1 0 1 0 1]\n", 2194 | "True\n", 2195 | "471 + 222 = 693\n", 2196 | "------------\n", 2197 | "Error:[ 0.29745122]\n", 2198 | "Pred:[0 1 1 1 0 0 0 1 1 0]\n", 2199 | "True:[0 1 1 1 0 0 0 1 1 0]\n", 2200 | "True\n", 2201 | "254 + 200 = 454\n", 2202 | "------------\n", 2203 | "Error:[ 0.21469789]\n", 2204 | "Pred:[0 1 0 1 1 1 1 1 1 1]\n", 2205 | "True:[0 1 0 1 1 1 1 1 1 1]\n", 2206 | "True\n", 2207 | "188 + 195 = 383\n", 2208 | "------------\n", 2209 | "Error:[ 0.28374133]\n", 2210 | "Pred:[1 0 1 0 1 0 0 1 0 1]\n", 2211 | "True:[1 0 1 0 1 0 0 1 0 1]\n", 2212 | "True\n", 2213 | "237 + 440 = 677\n", 2214 | "------------\n", 2215 | "Error:[ 0.28252835]\n", 2216 | "Pred:[0 1 0 0 0 1 0 1 0 1]\n", 2217 | "True:[0 1 0 0 0 1 0 1 0 1]\n", 2218 | "True\n", 2219 | "111 + 166 = 277\n", 2220 | "------------\n", 2221 | "Error:[ 0.09764052]\n", 2222 | "Pred:[0 1 1 0 1 1 0 1 1 0]\n", 2223 | "True:[0 1 1 0 1 1 0 1 1 0]\n", 2224 | "True\n", 2225 | "3 + 435 = 438\n", 2226 | "------------\n", 2227 | "Error:[ 0.20357224]\n", 2228 | "Pred:[0 1 1 0 1 1 0 0 1 1]\n", 2229 | "True:[0 1 1 0 1 1 0 0 1 1]\n", 2230 | "True\n", 2231 | "267 + 168 = 435\n", 2232 | "------------\n", 2233 | "Error:[ 0.18996958]\n", 2234 | "Pred:[1 1 0 1 1 1 1 0 1 0]\n", 2235 | "True:[1 1 0 1 1 1 1 0 1 0]\n", 2236 | "True\n", 2237 | "418 + 472 = 890\n", 2238 | "------------\n", 2239 | "Error:[ 0.15201967]\n", 2240 | "Pred:[0 0 1 1 1 0 1 1 1 1]\n", 2241 | "True:[0 0 1 1 1 0 1 1 1 1]\n", 2242 | "True\n", 2243 | "64 + 175 = 239\n", 2244 | "------------\n", 2245 | "Error:[ 0.16760019]\n", 2246 | "Pred:[1 0 1 0 0 0 0 0 1 1]\n", 2247 | "True:[1 0 1 0 0 0 0 0 1 1]\n", 2248 | "True\n", 2249 | "387 + 256 = 643\n", 2250 | "------------\n", 2251 | "Error:[ 0.25905665]\n", 2252 | "Pred:[0 1 1 0 0 0 0 1 1 1]\n", 2253 | "True:[0 1 1 0 0 0 0 1 1 1]\n", 2254 | "True\n", 2255 | "251 + 140 = 391\n", 2256 | "------------\n", 2257 | "Error:[ 0.24378146]\n", 2258 | "Pred:[1 0 0 0 1 0 0 1 1 0]\n", 2259 | "True:[1 0 0 0 1 0 0 1 1 0]\n", 2260 | "True\n", 2261 | "497 + 53 = 550\n", 2262 | "------------\n" 2263 | ] 2264 | } 2265 | ], 2266 | "source": [ 2267 | "max_iter = 10000\n", 2268 | "for j in range(max_iter):\n", 2269 | " # 랜덤하게 정수 두 개를 뽑은 후 binary lookup table에서 해당 이진수 가져오기.\n", 2270 | " a_int = np.random.randint(1,largest_number//2)\n", 2271 | " a = np.array(list(map(int, list(format(a_int, digit_key)))))\n", 2272 | " b_int = np.random.randint(1,largest_number//2)\n", 2273 | " b = np.array(list(map(int, list(format(b_int, digit_key)))))\n", 2274 | " # 실제 정답 계산 및 binary 벡터 저장.\n", 2275 | " c_int = a_int + b_int\n", 2276 | " c = np.array(list(map(int, list(format(c_int, digit_key)))))\n", 2277 | " \n", 2278 | " # RNN이 예측한 binary 합의 값 저장할 변수 선언.\n", 2279 | " pred = np.zeros_like(c)\n", 2280 | " \n", 2281 | " overallError = 0\n", 2282 | " \n", 2283 | " output_layer_deltas = list()\n", 2284 | " hidden_layer_values = list()\n", 2285 | " hidden_layer_values.append(np.zeros(hidden_dim)) # dim: (1, 16)\n", 2286 | "\n", 2287 | " # feed forward !\n", 2288 | " # 이진수의 가장 낮은 자리수부터 시작해야하므로 reversed로 for문 돌림.\n", 2289 | " for position in reversed(range(max_binary_dim)):\n", 2290 | " \n", 2291 | " # RNN에 들어갈 input과 output label 이진수 값 가져오기\n", 2292 | " X = np.array([[a[position],b[position]]]) # dim: (1, 2), e.g. [[1,0]]\n", 2293 | " y = np.array([[c[position]]]) # dim: (1, 1), e.g. [[1]]\n", 2294 | " \n", 2295 | " # hidden layer 계산하기 h_t = sigmoid(X*W_{hx} + h_{t-1}*W_{hh})\n", 2296 | " hidden_layer = sigmoid(np.dot(X,synapse_0) + np.dot(hidden_layer_values[-1],synapse_h)) # dim: (1, 16)\n", 2297 | " \n", 2298 | " # output_layer 계산하기 \n", 2299 | " output_layer = sigmoid(np.dot(hidden_layer,synapse_1)) # dim: (1, 1), e.g. [[0.47174173]]\n", 2300 | " \n", 2301 | " # error 값 계산\n", 2302 | " output_layer_error = y-output_layer # dim: (1, 1) \n", 2303 | " \n", 2304 | " # display를 위한 저장\n", 2305 | " overallError += np.abs(output_layer_error[0]) # dim: (1, ) \n", 2306 | " \n", 2307 | " # 이 후 backpropagation에서 사용될 delta 값 미리 계산하여 저장\n", 2308 | " output_layer_deltas.append((output_layer_error) * sigmoid_output_to_derivative(output_layer)) \n", 2309 | " \n", 2310 | " # 현재 자리수에 대한 예측값 저장\n", 2311 | " pred[position] = np.round(output_layer[0][0])\n", 2312 | " \n", 2313 | " # 현재까지 계산된 hidden layer 저장\n", 2314 | " hidden_layer_values.append(copy.deepcopy(hidden_layer)) \n", 2315 | " \n", 2316 | " if (j%100 == 0):\n", 2317 | " overallError_history.append(overallError[0])\n", 2318 | " \n", 2319 | " \n", 2320 | " # accuracy 계산\n", 2321 | " check = np.equal(pred,c)\n", 2322 | " if np.sum(check) == max_binary_dim:\n", 2323 | " accuracy_count += 1\n", 2324 | " if (j%100 == 0):\n", 2325 | " accuracy_history.append(accuracy_count)\n", 2326 | " accuracy_count = 0\n", 2327 | " \n", 2328 | " \n", 2329 | " if (j % 100 == 0):\n", 2330 | " print (\"Error:\" + str(overallError))\n", 2331 | " print (\"Pred:\" + str(pred)) # 예측값\n", 2332 | " print (\"True:\" + str(c)) # 실제값\n", 2333 | "\n", 2334 | " final_check = np.equal(pred,c)\n", 2335 | " print (np.sum(final_check) == max_binary_dim)\n", 2336 | "\n", 2337 | " out = 0\n", 2338 | "\n", 2339 | " for index, x in enumerate(reversed(pred)):\n", 2340 | " out += x * pow(2, index)\n", 2341 | " print (str(a_int) + \" + \" + str(b_int) + \" = \" + str(out))\n", 2342 | " print (\"------------\")" 2343 | ] 2344 | }, 2345 | { 2346 | "cell_type": "markdown", 2347 | "metadata": {}, 2348 | "source": [ 2349 | "### error와 accuracy가 유지되는 것을 확인\n", 2350 | "\n", 2351 | "* Check if the error and accuracy are maintained low and high" 2352 | ] 2353 | }, 2354 | { 2355 | "cell_type": "code", 2356 | "execution_count": 12, 2357 | "metadata": { 2358 | "collapsed": false 2359 | }, 2360 | "outputs": [ 2361 | { 2362 | "data": { 2363 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAD8CAYAAABkbJM/AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXm0ZFV977+/e2/37du3J7q53X3pbuhuBhFEhrQ4oYLM\nRASjPtE4PNT4eJEYiT6FvLdMonm64kqyXswzEjSYJzGyjIpiGEWJoAyhgWaUoYFuupueafoOPdxp\nvz9+9fPs2nWGfU6doaru77NWrZrOqdqn6pz93d/fbw9kjIGiKIqiJNFVdQEURVGU9kAFQ1EURfFC\nBUNRFEXxQgVDURRF8UIFQ1EURfFCBUNRFEXxQgVDURRF8UIFQ1EURfFCBUNRFEXxoqfqAuTJoYce\nalauXFl1MRRFUdqGBx98cJcxZsBn244SjJUrV2Lt2rVVF0NRFKVtIKKNvttqSEpRFEXxQgVDURRF\n8UIFQ1EURfFCBUNRFEXxQgVDURRF8UIFQ1EURfFCBUNRFEXxQgVDUZR6pqaAa68FDh6suiRKi6GC\noShKPQ89BHzsY8BNN1VdEqXFUMFQFKWeV17h++3bqy2H0nKoYCiKUs/QEN/v3FltOZSWQwVDUZR6\nhof5fseOasuhtBwqGIqi1COCoQ5DcVDBUBSlHglJqcNQHFQwFEWpRx2GEoEKhlIt3/0u8Od/XnUp\nFBsVDCUCFQylWn74Q+C666ouhWIjIaldu3gQn6LUUMFQqmV0FDhwoOpSKDbiMKamgJdfrrYsSkuh\ngqFUiwpG6yEOA9DEt1JHoYJBROcR0dNEtJ6Irgx5/yIiepSI1hHRWiI6zXpvAxE9Ju8VWU6lQkZH\ndc6iVmN4GOjt5ceax1Aseor6YCLqBvB1AGcD2AzgASK60RjzpLXZzwHcaIwxRPRaAN8HcKz1/hnG\nmF1FlVFpAdRhtB7Dw8CqVcBTT6lgKHUU6TBOBbDeGPO8MWYMwPUALrI3MMaMGGNM7Wk/AANlejE6\nCkxOAhMTVZdEEYaGgCOP5McaklIsihSMZQA2Wc83116rg4jeRURPAbgJwEettwyAO4joQSL6RNSX\nENEnauGstTu1NdR+jI7yvYalWgdxGIA6DKWOypPexpgbjDHHArgYwJest04zxpwE4HwAnySit0bs\nf40xZo0xZs3AwEAJJVZyRQRDw1KtwdQUMDICHHIIsHChOgyljiIFYwuAFdbz5bXXQjHG3AVgNREd\nWnu+pXa/A8AN4BCX0kmMjQWhKBWM1mB0FDAGmDcPGBhQh6HUUaRgPADgaCJaRUQzAVwC4EZ7AyI6\nioio9vgUAL0AdhNRPxHNrb3eD+AcAI8XWFalCsRdABqSahVkDMbcucDixeowlDoK6yVljJkgossB\n3AagG8C1xpgniOiy2vtXA3g3gA8T0TiA/QDeV+sxtQTADTUt6QHwr8aYW4sqq1IRIyPBY3UYrYGM\nwZg7lx3GU09VWx6lpShMMADAGHMzgJud1662Hv8VgL8K2e95ACcWWTalBbAdhgpGayAOY948dhh3\n311teZSWovKktzKN0ZBU62GHpAYGeD6pycnsnzc2xjelI1DBUKpDHUbrISEpSXob09x8Uh/+MN+U\njqDQkJSixKKC0Xq4SW+AE99Zu6y/+CKLjtIRqGAo1aEhqdbDTXoDzXWtHR/XUfwdhAqGUh3qMFoP\nN+kNNCcYmsPoKDSHoVSHCkbrMTwMdHcDs2YFDqOZsRjj4/rfdhAqGEp1qGC0HkND7C6IgEWL+L5Z\nh6H/bceggqFUh+YwWo/hYc5fAEBPT/PzSanD6ChUMJRyuPpq4Ac/qH9tdBToqp2CWqm0BrZgAM3P\nJ6WC0VGoYLQiExPAOed0zijbsTHgc58D/vEf618fGQEWLODHWqm0BhKSEhYvzickpV1rOwIVjFZk\n927gZz8Dbr+96pLkwz33cMv1lVfqXx8d5dbsjBkakmoVwhxGsyEp+75M7rqruUGHSgMqGK2ItLY3\nb662HHlxyy18v3dv/eujo0B/P/fIUYfRGgwN1QtGHg4DKP//nZgAzjoL+Pu/L/d7OxwVjFZk/36+\n37Qpfrt2QQQjzGGoYLQWw8P1IamBAXa8WeeTEmdR9v+7bx9/98aN5X5vh6OC0Yp0ksPYtAl47DFg\nzhwWDDuWPTrKr/f2akiqVQgLSRnDopGWqalAaMoWDGl0vfRSud/b4ahgtCJysm/e3P7Jwltry5hc\nfDG3+OTYAHUYrYYxjQ6jmdHedt5CBaMjUMEom4ce4hbcM89EbyMn++hoYxin3bjlFmD5cuDNb+bn\ndh5DBaO12LePXYHrMIBsie9WEIwtkatCKxlQwSibX/6Su5PedFP0NvbF1c5hqbEx4I47gAsuAA45\nhF+zBVAEQ0NSrYE9U61gz1ibFnsOqSpyGAD3ktLGSG6oYJTN47WlyX/xi+ht7LBNOye+pTvt+ecD\n8+fza2GCoQ6jNbDXwhDkf5P30tAKDgPQsFSOqGCUjQjGXXdFT/tsn+zt7DBuuYXHWJx5ZjBATwWj\ndQlzGH19fJ/l/6nSYahgFIIKRplMTQFPPMEx/aEh4OGHw7ezL65WcRijo8B738sL4vhy663Aaadx\nBSSCITmMsTFugWpIqnUIcxgiGHYF7Is6jI5DBaNMNmzgiveyy/h5VFhKTva+vtZxGE88wXNB3XOP\n3/Z793J32jPO4Oeuw5CJB9VhtA5hDmPWLL7PIhit4jA08Z0bKhhlIuGoM88Ejj8euPPO8O3k4jrq\nqNZxGFLR+zqB//xP7qb5xjfyczeHoYLReoQJRlcXMHNmtv9HHUbHoYJRJiIYxx3HLe+77w5fjUxO\n9qOPbh2HIRW974V/7728lsKpp/LzWbO44gkTDA1JtQZhISmAnW67haSkl1R/vwpGjqhgFMUzzwB/\n+qf1Uyo8/jhwxBF8QZ5xBp/UDzzQuO/+/ZwsPuKI1hm8J7kH34r93nvZRUnlQ8RhKVcw5sxRh9Eq\nhDkMgP+fdg1JHXmkhqRyRAWjKH7yE+ArXwF+/evgtccfB17zGn78trdxJRqWxzhwgFt1y5e3zuC9\nNCGpqSngvvuCcJSwYEEgPBqSaj2GhjgENXt2/et9fe0bkjrqKHUYOaKCURQjI3z/wx/y/fg48NRT\ngWAsWgSceGJ4HmP/fq5EV6zg51Fhqa99Dfjxj/MtdxRpBOOZZ3h7VzDmz48OSalglM/oKHDddYGD\nlXmkiOq3yxqSqtphdHezS9+ypRiXvm0bz4Y7NZX/Z7coKhhFYQvG1BTw7LMsGiIYAPD2t3OvI/di\n2r8/cBhAdOL7a18Dvv3t/MseRpocxr338v0b3lD/elhIShzG+Pi0uvBagn//d+DDHw5csDvxoJA1\nJFW1w+jrA5Yt49BvloGHSVx3HfCpT5XXaGsBChUMIjqPiJ4movVEdGXI+xcR0aNEtI6I1hLRab77\ntjwiGFu2cI8hSXjbgnH66dxid/MYEpJKchgHD5a3QEyaHMa997I4vOpV9a/HCYb72f/xH8Cll7ZG\n/qZTERGQlR3d1faErCGpqh1GXx9w2GH8vIiw1Asv8P1f/uW0OU8LEwwi6gbwdQDnAzgOwPuJ6Dhn\ns58DONEYcxKAjwL4Vop9W5vhYZ6HZ8YMdhmPPcYW+dhjg21EENyZQCUktXQpx5SjHMbBg8CePcWU\n3yVNSOree9lddDmnV1QOo7eXH9uVyq23Av/8zzx2RSkGqdDvuovvoxxGu/aSsgWjiMT3hg18jj/8\ncLDmS4dTpMM4FcB6Y8zzxpgxANcDuMjewBgzYsxvpbkfgPHdt+UZGeGT9ayzeMDbY49xN1lpTQNc\nWQJBF0BBHEZPD39GnMNoNcEYGuJBfm7+AkjnMKTHztq12cusxCMV+j33cG++IkNSZXebtkNSQDEO\nY8MG4Hd/Fzj8cOBLX5oWLqNIwVgGwG4ab669VgcRvYuIngJwE9hleO/b0oyMcJfR97yHT6zbb68P\nRwGBYEjlKcjJDnAeI8phHDhQXkjKN4chA/bc/AXASe/9+7nykGOePTsQDPuzVTCKRxzG0BDw6KPF\nhaS6uqoJSc2eDQwO8vO8HYYxfF0ffTRw5ZXcKzBqIG4HUXnS2xhzgzHmWAAXA/hS2v2J6BO1/Mfa\nnc2sPZw3IhgXXcShqP370wmGVKLLl4c7DGP4gjxwIFvrLy2+OQwZsPf61ze+Z88nNTrKx9jdHR6S\nkhyQCkZx2DmGu+4qzmHMnVtdDqO/nxsqeTuMHTv4O1at4lzb4CDnMjqcIgVjC4AV1vPltddCMcbc\nBWA1ER2aZl9jzDXGmDXGmDUDsthLKyCCsWgRJ7eBRsGQ/u6uYEhICuA8R9jgPftiLyMs5RuSuvde\nHskuU4HY2PNJjYwEgpkUktLeU8Ug59CKFZz4jnMYzXSrrVIwAA5L5S0YkltbuZLP309/mh3G88/n\n+z0tRpGC8QCAo4loFRHNBHAJgBvtDYjoKCLu9E1EpwDoBbDbZ9+WRwQDAN7/fm51n3xy/TYzZvDN\nzWG4DiNs8J59ARYtGBMTQYs/7sI3Brj//vBwFNDoMFzBCHMYQ0PA+vXZy65EMz7O4aLTT2fBiEt6\nNzNwb968agXjsMPyD0nZggHwrAYAsGtXvt/TYhQmGMaYCQCXA7gNwG8AfN8Y8wQRXUZEtela8W4A\njxPROnCvqPcZJnTfospaCCMjwcX30Y/yoL3Vqxu36++Pz2FEda21W+NFC4bdhz3OYWzaxDmV3/md\n8PftCQhHRwNBDQtJDQ/ztA6AhqWKYmyMGyxveQuHWCYn8w1JVekwpJcUUIzDkC61Ihjyu4kz7lB6\nivxwY8zNAG52XrvaevxXAP7Kd9+2wnYYRMAxx4RvN3t2fEjKHrx3wgnBNnbFXXTi23Y3cYLxyCN8\nf+KJ4e/bIakwh+GGpN7yFm4Zrl0LfOAD2cquRDM2xhNCvvWtwWtRIamDBzk06HaVjsPOYZTVOUOQ\npDfADmPr1vTlj2PDBuDQQ4NrfJoIRuVJ745EFgeSkymO/v74kFQrOIy0gmELm02SYLghqUMOAU46\nqTiHcdttPAJ/uiKCccwxwdrdUSEpIH3X2FZIegMsGBMTjeOdfFm3DrjmmvrXNmwI3AWggqE0gcTf\nfQXDdhjG1DuMpUvZobiWuswchgjGwoXxF/4jj/Bkb2GVDhCdw4gKSc2ZA7zudcBDD9XP+psXH/kI\n8Ld/m//ntgvj4ywYROzmgHCHkXURpbExbtH391ef9Aayh6W+9S1e9MwOzapgKLnRjGBIK05O9p4e\nvmBdF1JmSEq61C5dGt/KXLcuOhwF8LF2dyeHpMShzZ0LrFnD2z79dPPH4TI8PL0nPZQcBhAIRpzD\niBOMxx4Dtm+vf00EqYrZiF2HAWRPfA8NcUPuvvv4+dSUCoaSI2kEw81hyEVpjwgPu+CqCEktWRIt\nGCMjwHPPxQsGUTBjbVxISi66OXNYMIDwdUOawRgW4bAFrKL4t38DPve5fMtRJRKSAoB3vxs4//zw\n/08q3rhK/8ILG8chiCCVLRjj4+xIXcHI6jDkfJTlibdv5+tg1apgm74+dlMqGEpq0joM2z3IhSUn\nO9AegvHYY1wJxwkGEEwPEheSshfyedWreLu88xjyXWkE46c/Bb7znXzLUSW2YCxfDtx8M48bcvEJ\nSe3ZEzhRQRxG2dPXy/WUFNb1Rc5HmdXX7VIL8OfPm1fMrLgthApGEdgt5CTckJSvw7Cfl9FLiggY\nGIi+8JN6SAkyAWFcSEoEd+5cDmGdckr+giGVSphgPPssuyWX/fs7K4QlFXoSPiGpsN/GdRhlzbUk\n5ZReUjNmsBC6ITNf5Hq+7z52LmGCAfD5GuYw1q8vZzaGElDBKIJmchhRDsM94aRyHRgo3mHs3cut\nJ+leGca6dSwGhx8e/1nz53NvlbExv5AUwGGpdevqJ7NrljjB+O//HfjkJxtfL2salrKwcxhxJIWk\nJib4v3HPjfHxQDCmpni7MpD/yL6G7JmS0zI8zMcxMsLLFMgYjCOOqN8uTDDWr+f5phYuBC64APj6\n19v6HFLBKII8chj2yR420lYuzsHBchzGggUcWjh4MLyl+Mgj7C7c1dpcFiwIko8iGD099RPU2Q4D\n4AvuwAFg9+7mj0WIE4yhofBlcffv5+2rnqrkkUfy6QRgh6TiSApJyethgiFJb6A8dxYlGFmXOh4e\nBk6rLdXz61+zw1i8ODh/hTDBkIlDzz6bnevll3MurE1RwSiCLDkMqYTTJr2XLi0nhzF/fpBrcFv6\nU1Ocw0gKRwF84W7dyo/tC84+RtdhFNEDJU4wxsbCK0cpX9lTdbu8+93AH/1R85/jKxhJISl5PS4k\nFfZ+UYQJhr08cFqGh3ls0eAgJ77dHlJCmGCIq/mLvwB+9St+7A7UbSNUMIogrWBMTQWVkG/SW56L\nYBQZHxaHEXXhP/ccXwS+giFjKuzfZ9as4Dewk972dvK75kEWwYiqGMtk40b+vZ95pvnPkpBREkkh\nqTwcRp6uLc+QlDHBHFtvelPgMHwFQ0RqwYLgt07T0aLFUMEoAqnYXMsahjtjbdjJHucwBgfrJwcs\ngr17g5CU/d2Cb8IbCAbvAfW/j92Txg1JtZrDqFIwZM2FTZuar3iKDkn5OowvfKFxOd9mkP9Wri0g\ne0hq/34Ws7lzgTe/mcXiuefSC8b8+cFvrYLRQbzvfcDXvtbcZ4yM8Mna3Z28rbvqXtqQlCwQU2RY\nys5h2N8trFvHxyozdsZhT3vuG5KS+zwFQ37nsES6rDMStU/egvHkk/4JfRGMqSngxReb+96iQ1I+\nDuPRR4EvfznfnkRRIaksDsN2u296Ez+emqofgyHEhaTmzVPB6Eh+8Qte06EZ7IkHk3AXUUobkipL\nMOwchluWRx7htcptkYsiymHYIamRERYg+TxxGGWFpA4eLM9h7NzJzuxf/zV5W2NYMGQgWljX3zSk\nFYy0ISnXYbjvT03xlBsSosyrU0NUSGp0NH1PO1swTj45OJY0DmPuXO7YoSGpDmTfvuYrpmYEI43D\nkLERQHE9paamgpBU1IUvPaR88AlJyTxS0uOqqpCUmxcqwmFs2sQhRZ9BZc8/z9v/1/8aPG8G3xxG\nM72k4kJS117LjbMLL+TnaQQjzi1ECUbSfmHYgjFzJs9tBkQLhtu9WBpbAJ/PM2bk2z28ZFQwbKam\nyhcMN4cR5jDCVjw7eJAvxIUL+XlRDmNkhCvOqJDUgQNcib361X6fF+cwbMGw5zSqIuk9NdV4YRch\nGNJjzKcik3DU7/8+/xeuYGzcCDz8sP93+zoMmaAwz5DUzp3A5z/Pc1hdcQW/5isYTz3F5/0vfxlf\nHjckBTQnGADwtrfxb++OwbC3sRs20tgSZs5Uh9ExyIlWhcNwcxg+Se/eXp4CHCjOYdi9PMIEQ8od\nNstpGHE5DDskFSYYZToMoP43N6axJ1sSGzfyYK24SmrbNr73FYylS1mcV69uDEldcQXn4HzxFQyi\n+FX3siS9/+ZveLzLN77B60oA/oKxYQML+nXXxZcnzGGkTXy7gnHllbyqpP3ZQphgSP5P6HTBIKJu\nIvrrMgpTOVKJtEJISipnIDqHYQtGUQ7D7uURduFLue0eKXGkCUkJM2bw+2UIhj0i2W5R28fsKxh3\n3w3cckt8q18cRtIcRJK/OP10rsBXr250GGvXppv+wlcwgPhV97J0q92wgY/h+OOD+at8BUOuzxtu\nCA/vuHNJAfWrPabB7YDR3x8dfhXBsP9LVzBmzOhswTDGTAI4rYSyVI9UfmEV0xe/CPyv/+X3OW6F\nF0dY0nvWrPoR07NmcWLQnlpBHMacOZxQK1owkhxGs4Jhi6LrMAB+XkZIyq6AmhUMqQBFFMLwdRjP\nPMOfc8YZ/FwEQ/Isu3ZxaHBoyD9G7pvDAMLDooLd48weTxHnMOyKNKtgvPxyEKZzy2MnmYF8chhJ\nRIWkbFfd6Q6jxsNEdCMRfYiIfk9uhZasCuIcxq23Arff7vc5zeQw7Hn8hbALTnIYROwyigpJyQWW\nJBg+Y04ADl0R8WfZ3Y7dgXvu7zdnTnEOw05u2xezXUFGiUccUgHGJbR9cxhSMYpgHHkk/x67dvFz\n28X4tqLTOAyfkBRQf27EOQxbMHp7+fzxFQw5D2bMCJ9mI+wayhqScscExSFh2ekckqoxC8BuAG8H\ncGHt9o6iClUZUmlLotdmaKhxEaMoms1huN1TowRDKvCFC9vHYXR18cXn/j5uSKosh2FM/Wp+bkI/\n6XEceTqMO+/kleOOOoqfr17N9xKWeuihYFufxoOE3vIMSQH1v1+cw9izp74iXbQovcN45zvDw1Jh\ngpFXSCoO12EYE570buNeUj0+GxljLi26IC2BCMbkZNCCF4aH2eb60GwvKR+HITkMgB1GGTkM+7uF\ntIIB1F9Agk9IqgiHAXDFJv9t2Q7DVzDWruWRxhKqtAXj9a+vdxg+giGVVhqH4SMY9m8T163WbXmn\nEQy5Fj/4QeCHP2QxPeec+vK415A42ywhqb4+v2vfFYzRUa5LpltIioiWE9ENRLSjdvshES0vunCl\nY1cibmt2eNjPYUg/bF/B6Onhk8gOSUU5DLclZwtG0b2k7IF7zTgMgCsKN4TlE5IqwmEA9RdwlGAU\n4TCM8U96Dw/XL24kI42lp9RDDwVrV/tUvCIYaXIYeYWkjGlOMKRBdt55XEm7Yal9+xrPx64uFo0s\nDsMnHAU0CobtzoXpIBgAvg3gRgCH1W4/rb3WWdizSNqVkzF8QftMXSCf4SsYQP2qe2EOI2ykre2A\nsoSk7r8f+PjHkyct3LuXyyc9leS7hbwEQ0JSk5P8OxftMOz/0kcwoh7HIfmFKIcxPBy0hmXd6ChG\nRup/s9mzeZT/88/zvs8+C5x1Fr/n03iQ4ywjJNXTw/kqe6T82FhzDmPOHC7ThRc2hqXCHAaQbcba\nvAWj03tJ1RgwxnzbGDNRu/0zgIECy1UNUQ5DKrIwh7FvXxBWsPfzPcmA+kWU0iS9m3EYP/kJ8E//\nlFwJ2y3BsHJkEYwrrgA+85n61yQkFTXTb1FJbyC9YOQVkhJ3ccwxnFOIclBTU/z97m8iPaVk8sci\nBSNrSEo+3w45hlWkaR2GXF/vfS/vd/fd9eUJE4wsM9amEYyeHj5OOU/lu4oISb3ySnxurCB8BWM3\nEX2wNiajm4g+CE6Cdxa2w7ArJwkXjI83rhr2pS8Fk5IB6aY2F1zB8El6uzmMvXvTTREtJ1tSmMee\n2iDMYaQdhwEAF18MXHJJ/WtyjFJplJX0Bupbp/bFnEfSu7ubyx0mdtLQOPZYvo+qzKJ6oh15JIek\nJH9xxhkcp08jGEWFpIzha0U+3+7UECUYe/bUd0CIwg5ZylQdzz5bX54owSjSYQD1TrjIkNQVV/D/\nXdaytzV8BeOjAP4LgG0AtgJ4D4DOS4RHhaTsi91tZb30Ei/Z6E7NnUYw7FX3fJPebi8p6ZHhi1RW\nSYu52A5DLv5mQ1JhyLFIGMf9/eRCzOsCiXIY9rE14zAOHODvOOYYfh7WGpTXZGrvqP8varr81at5\n9cJ77wWWLOEchq/bTJv0ThuScj8/zGHIoFOABUNyG0nYDkNGie/cWV+eqJBUkQ4DKE8wtm7lVReb\nnYAyJV4jvQH8njHmncaYAWPMYmPMxcaYJudWbkGiQlJxgiEVroQdsjoMu1ttmnEYQLbpQXwFw+4W\nKOMnXMEg8pupNg7ZXwTDvUjnzOEWa17x3337gn7zRSS9xSm95jV8HyYYrsOISnxH5cVWr+ZK9qab\ngFNO4dcWLmyNkJTrYHxCUoBfWMp2GL29/D/6CEbZDiMqJJVHt1qpZ267rfnPSoHvSO/3Z/lwIjqP\niJ4movVEdGXI+79PRI8S0WNEdA8RnWi9t6H2+joiWpvl+1MT5TDsC9nNY8hzWae6ipBUlgkIszgM\nKYubw5g9O3kt7ySSBCPvGWv37QuOK00OY+bMdIJxwgl8H5bH2LqVP08msotq/cp/FBaSAvg3Oflk\nflykYMSFpMRhug6jCMFwu10PDAA7dgTPw3pJyfdVEZIqIofRqoJR49dE9H+J6C1EdIrc4naoOZOv\nAzgfwHEA3k9ExzmbvQDgbcaYEwB8CcA1zvtnGGNOMsas8Sxnc2RxGHkLRlxIKq5bLeAvGJOTwQWW\nVjDCHEaz4Sj5XCBoKYYlvYFyBSPMVRxySDbBiHIYS5cmz6QaF5ISbIfhU+mmzWEkhaTkt5RzwxWk\nohwGwILhG5IaGkqX62tWMGbNqm8A5i0Yd95Zaq8rX8E4CcDxAL4I4G9qt6QJCU8FsN4Y87wxZgzA\n9QAusjcwxtxjjJFa7j4A1Y7tGB0NWutRguE6DKlwN2+u3y9rDiPMYSR1q00bktq1K7ho4gRDYsp2\nC6kowZBjkQs/ymHkkfiemuLfMovDWLAgnWCsXs3/X5TDGBxMFowoh7FkSfDbi8NYtKi4gXuTk+Hh\nFFsw5Lcp22H4hqTieqO5yLZpBUMiEu48UkB+3WpHR7mxMTIC3HNP85/niU8OowvAN4wxZzi3tyfs\nugzAJuv55tprUXwMwC3WcwPgDiJ6kIg+kVTOXBgd5SQaUXgvKSDZYaSZSkBIGocRNvDJTXoD/g4j\nrBtwGPv3c96gDIfhhqTCkt5APg7DrvyBdDmMtIKxaBGLQjMOIyqHIbPWzp8fDOQrMiQFRC9dK40W\n36R3b299w8hXMCYmGrsYL14cCIYx8YIh3++D/O7NOAx3VoM8HcY73sFdeX3nuMsBnxzGFIDPFVkI\nIjoDLBift14+zRhzEjik9UkiemvEvp8gorVEtHan3crIwr59fCL29/s7DHnuOgzfyfhkW5lGYGws\nWTDGx/nCyBqSsgUjzmGEtQSjchjN4vaSCkt6A/k4DPnPwgQjrpdUb298LN/GFozDDot3GHPm8Ejk\nqKR33Dl1/vk8FkFySAsX8v+W1D01y8A9IDwsFReSsh2GvBdWkc6fz12QkwQjrBIfGODzxphgxtyo\nkBTg31MqzUy1QhmCMTXFv8PgIPDGN5aax/ANSd1BRJ8lohVEtFBuCftsAbDCer689lodRPRaAN8C\ncJEx5rePp814AAAgAElEQVRnizFmS+1+B4AbwCGuBowx1xhj1hhj1gzIcqVZGR3li9KdhiLOYcgJ\nbOcwenv9Y8MAV7j79oUvzwo0rqUtF568LnFS35CU3dpNKxiuwxgdLcZhuJVj2Q5jxoxGwejrC1+b\nJIzdu/l3mTWLL2xXMMbH+ViXLuXKft689CEpAPjqV4FvfjN4Lm4zqRWdZWoQIFkwfENSbkVK5Jd/\nCXPwAwP8fXv3BuWLSnrL9/uQRTDmzeM6QJY2dkNSefSSkuWD58wBzj2Xp4Wxk/4F4isY7wPwSQB3\nAXiwdkvqufQAgKOJaBURzQRwCXh6kd9CRIcD+BGADxljnrFe7yeiufIYwDkAHvcsa3aktewKho/D\nsAUjTTgK4IrA7oPuto66u/nCcwXDFpY004OkdRhl5jB27eLfo8s5NfNMest/JscVJhjz5zcmvUWY\nfQVDwiyHHdYYkpILfHCQ730Ew+e8EsFIajwUHZKKS3q7M9UKPqO9w2ZSkIbizp3hiycJaWeszeow\nAP7PinIYdp703HP58c9+1txneuIlGMaYVSG31Qn7TAC4HMBtAH4D4PvGmCeI6DIiuqy22RcALALw\nD0732SUAfkVEjwD4TwA3GWNuzXB86YhyGFG9pIwJTtCXXgqSZFkEAwgulrCT3b7gXIcBpJseZNs2\nPrF7e+MFQyq5JUvqy1F0L6mwCzTPpHdcSMoWjDCHkSYkJYIxOMjnkF12+W2XLg2+L4vDcBHBSKp4\n8wpJGcO/h1TGvt1qswpGlMMAWITDlmcV0i6i1IxgDA9HC8bERLqeWi62YJxyCv9uJYWlYgWDiD5n\nPX6v896Xkz7cGHOzMeYYY8yRxpj/XXvtamPM1bXHHzfGHFLrOvvb7rO1nlUn1m7Hy76FExeSkovf\ndhgHDvAFs3w5nwQ7djQnGBKOCRsEZ19wcu8KRhqHsXRpfXfeMCQvs8KKLNpTPAD5O4w9e8J/vzxD\nUlkEI4vDkFHIhx3G97bLEJcnDkO6fIYhYU57wakoinYYrmDIbyErP7oD96KS3kU5DB/BKMNhDA9H\nh6SA5sJStuPs6gLOPpsT3yVME5LkMOwJf65y3jsv57JUT1xISlrZ9gUjFY9M/7BlS/pueEBQ4SY5\nDHft5KIFY9Mm/t6FVroqLCSVJsEfhQiGMeG/n1SYVTuMLCEpEQU7j5HWYfg2QuQ7kwQjaw7DPXa7\ngrbPjSocRpJgZA1JpWkAyrm7a1d9121Bfo9mwlJu1/0/+RPgO99pCcGgiMdhz9sfcRjuVNrDw9x1\nD6h3GPL46KP5fvPm4hyGHQoJy2FkEYykNSY2bWJ3YY/iLnrgHhAuGET5TXGeJBjyXW4OI6tgxDkM\naYgk5TB8RbkohxEVkkoSDNdhSK7OnkdKyNNhhJ2TM2dyOcsISYk7DwtJAfkIhpwTr3sdLyDl5v0K\nIOkbTMTjsOftjeQjwhzG0FCwgJB9wUjr3HUYVeUwinAYdjjKLQeQf0gKiP798lpEyU162+GBgweD\nisV1GL4hqclJrrBdwXAdxqJFQQUS5zDctTDikAqqrJCULRj2bxPWrXZigivh8fFohyGTNkYRNjC2\nr4+fJzkMIN30IM0IxqbaELSokFSeDqNEkgTjRCIaIqJhAK+tPZbnJ5RQvvKQ1k9U0nvevKD7qyCP\nV67k+G1ZghGVwxgebpx+3WX/fq6YfHMYrmC4rciJifwFI+oCLcthyCpxbg5DKsXx8fhxDq+8wueS\nCMb8+byf6zAkVCXb7N0bHlZI4zC6u/m4fAWjzJAUAGzfzvdRggHEu4yoSlxGe8f1kgLSLaIk35Um\n5OoKRpTDaCaH0aqCYYzpNsbMM8bMNcb01B7L8xQDDdoAuyeKCIZcvENDfCK4rU45OefO5Ys/a0hK\nKlzfpHeUwwCS7bZcsEmCMTnJLeLlzmwtdqWQZS2MKOyWbpzDKKNbbZLDAOJdhj1oD+AQlzt4b+vW\nIH8hZZHlfV3S5DAAv/EMWaY3B6JnOhAxjetWCwShuKyCETWIUQTDx2H4hqTkWk4T6vEVjE50GPYg\nvbBbWYUsBXtdB3cqbZmAzHUYdoW5bFn5ISlbWOTETApLSSt3cDBeMLZuZdGIcxh5rYUBBFOnA/EO\nI8+QlPTscQVDRnRHJb2BdIIBNA7eC3MYQHhlliYkBfhND5LVYSTlMOIG7gHNC8bwMJ9vbo8xmbE2\n75BU2g4sZYSksiwDnRM9Ce8/CM5VhCW4DYDYsRhtheswgKByGh/nkFSUw+jv55b4I48E04ukwVcw\nZLBXVEgKSBYMuWCTHIac8FE5DHsMSh6CIZ998GC8YIgLa4b9+wOBcgdS2Q4jauCePI8iTDAOOwx4\n9FF+bEyjw5C1OfburR/3AqQLScn3+ghGT49/6zltSKpIhxF2fQ0MAOvWJQvG/Pm8rK0PWQSjv5/P\nrTIcRtQxFkisYBhjVpVVkMpxHQYQDPEHwh2Gvc+yZbwYPVBcSCqpWy2QTjDikshhYzDkO2X5zbwF\nQ46njKR3Xx9f2HGCMTHBt56efBzGrbWxp3v38n/o6zCyhKSSVmIbG0s3fY1PLyk7JOU6DPlv83AY\nUYKR1EtKvrdIh0HE5YvK1eTVrTZsNoQSiBWMpDUvjDEP5VucCrEdhpz0w8NBcjPMYbghKdk2q2DE\nOYywbrVZBYOILzAfhxGWw5AyFOEwgHKS3lJmVzCkl5RdQUoX22YdxvAwV2p//uf82uGHB++XHZIa\nH/fPXwDB9DRJISkpf1i3WiAfhxF2fixezP+jVNRVhaSA4Dzt7m783/JyGBWEo4DkkNTfxLxnACRN\ncd4+2IIhf+bISCAC4jDclb1kH7tiTftndnfzBSUjfX2T3u44DCD5Yti2jcWipyc4VmlF22zaxO+7\nF7Z8py0YeQzcsz+7jKR3lGDYDgPgClEaEWkcRnd3ffxa3MTJJ3Ou61OfAt75zuB92TZstHfakJTM\nKzY1Fd0KleNMQ9i0KEkhqagchhvbB7g8c+ZkdxgA8OKL/J1Ro+Lnz+eySQMgjuHhRoftg4jM/PmN\nK1F2smAYY84oqyCVY7eWpXU0MhI8juslJQ5DyPJn9vfzSdzVFR4q8OlWC/g5DImdSyU0Otp4AYcN\n2rO/0+4vn3dIKs5hHDgQLnBpSCMYBw7UC7SvYCxcWP/bScVDBNxxB3DmmfX72DkMm/FxvqUVDGP4\ns8IGyAHZBCNs1b2ocRhRSe+tWxtXobNJGrw3MlLv3ARbMOJi+/b0IHYOKYysDkP+yzAXldfUIK0o\nGDZE9BrwUqu//aeNMd8polCVYDsMcRUjI0FlEjYOY3SUK/iZM5sXjNmz+UKR2LpLUrdaaeHlJRib\nNzeGo+zvrCokBfD/EnYx+pIkGNJLCuAK0a0UgWTBcCu1t70N+N73gPPOCy97VEgqS48YewLCOMFI\nk8MAGhtMQLakd9x/lyQYw8PBGug2IhgbN8YLhj09SFGCIfvECUabOgyvrAkR/RmAv6/dzgDwVQDv\njN2p3YhKetsDhcIchvSKyMNhANEtL0koymp7QGMLccGC7ILhEjbKGyhHMOJCUkDziW8fh2HnMEQc\nmhGMri7gkkuiK8soh5FlQS6f+aTS5jCAdCGp8XG+LiQ0JL/bjh3RIiZlz5LDEMHYti3+fEwzY22z\nghEVdgPySXpXgG+a/T0AzgSwzRhzKYATAYT8Gm1MVLdaiSlHjfSWk3PWrOBCzXKSyQkQ1Tqycwey\nPKvrRJKmBzGmvv9/lGCMj3PoIEww7Aozz4F7gF9ICmg+j5E2h2EvbJVVMJKQnFKUw0gbkgLiBSPP\nkNTMmUEezp4axP58+d0mJ5t3GHE5DGP8Q1JxyFKweTuMvHpJtbLDALC/tlTrBBHNA7AD9avptT92\n5Wcv1hPnMNzV5sRlNOMwkgRDWrx2OEpIEow9e/hEFYcR1WJ/6SW+8Kajw7DnkgIaQ1JxCwkJWQQD\nCJ/ivJmQVN6CERWSkt/EdRh2yCtskGkYIhh33MHTdh96aHBOGxPtMPr7g3L4hqTiCJvk0JfpHpIC\nsJaIFgD4Jngw30MA7i2sVFWwbx//mdLSA+odxpw5QUJc5mtyp/aWmH/WHAYQH5ICgiRsFsGwx2AA\n0Q5DxmD45jDyGkDkm8PIw2FImd0lM8OS3iIORToMIHzG2iwhKV/ByJLDCAtJRQlGmMMAkgVjzx4W\ni1//mn/Lp5/m9w4cYIcSdX3JjNI+DiMpJJVl4kGhjJBUqwoGERGArxhjXqktfHQ2gI/UQlOdg911\nsauLH0sOo7+fLbc7PYI7U2uRDsPttRMlGHEtJ1/BiBrlDTR2q/Vd2MeHpIF7eQnG/v3ZQlI+OYx9\n+/i9rA4jj5CU5AjyzmFEhaTkt5IZaScnGwXJVzDOPBM47TRep/yXv+TX5HxMavVLWCqPkFQegjEd\ne0kZYwwR3Yza7LTGmA1FF6oS3MrfnoBQTgB5f98+fs3tH/+Wt/CC7GkvRMAv6Q0EghG2XV4OI04w\n3G61eYWjAD6m3t7olm8RIakZM8J7SYUlvX0cRtigPV/CZlLNEpLq6WG3EpcLGBtLHofg4hOSAvj8\nzBqSeutbgbvv5sdyLsv5mLSgkY9gyDxUVQtGVocxMcHnXqs6jBoPEdHrCi1J1biVvwjG0FDQgyXJ\nYXzoQ8DatciEfE5SDkNCJGEOQ0axRq0XnEYw5s0LjtvGDUnl2VvjiCOA1THTk1Wd9PZxGDK9S145\njCwhKfn+KkJSAJ8bbo5kxoygk4Zvl+gFC/i40zqMuEYMkd+Mta0aksriOHPEVzBeD+BeInqOiB4l\noseI6NEiC1Y6bgJbBMPuWmc7DLnPq4Xtm/ROCkkZE7029PbtfMLKiRyXwwjLXwCNgpGnw7jqKuCB\nB6Lfz8NhTE5y2dMM3LMdhkzY57a0BWnVy3reacgrJAUkTw+SZy8pOyQF8O/lOgyi4H1fwSBil/vi\ni/w8D4cB8H8jIhRFluVZhSIdRoVTmwP+A/fOLbQUrYDbWhbBGB8PTgDXYaSdsiGOtCGpKMEA2MqH\nnayyNKa09Ozkvk3UGAy7HEUIRk9P/AhuKW8zDsOdnC6tw5CKr4iQVFjSuyjByDoOI0ww5PqIcxhA\nIDhpBl0efni+OQwAePvbeQ3suOlBmnEYch0uDFkBQvJ9bSoYXg7DGLMR3I327bXH+3z3bRvCQlLD\nw/UhqVZxGFEnetL0IHv31ttk6TsfFpKKEgw7h+G6sqLp6mp+Pim3K7AtGDIoMmrgnrxWlGDMn8/l\nsxOiIyMsUml7ohXhMNKEpFyHAaR3GACfh745DJ9eUgDP4TU6Ctx5Z/Q2zQjG6acD114LvPnNje+F\nzZCchnYQjNpI788DuKr20gwA/1JUoSohKulth6SSchjN4JvD2L/fz2GEsXdvY17CnbH24EEOXVUR\nkvKh2SnO4wRjcpJFY+bMYMoX12EAfoIR1rpMImwCQhHltFNZJ626lyWHYa+FIsSFpMIcBpBeMLZv\n5/Lm5TDOOIPPoxtvjN6mGcHo6QEuvTS692AzglHh4kmAv0t4F3gqkFEAMMa8BCDDL9nCuA5DVncL\nS3rv28eJZZnJNA+SQlK+3WqB6B4gQ0ONiThXMGRVuCSHUZVgNDvFeZxgyL0co4Rg0grG3LnZespF\nCUaWc+zQQ9lhRK09ntVhAPXHHpf0zsthGBOsZgkk5zCSzsneXuDcc1kwwtZQB/gc6+oqZpEid+xP\nGrJ2gsgJX8EYM8YY8JTmIKJqSlskaZLediVSVdI7TFiSlmn1cRhxXWqB6gXD12E88kh4ZekONgwT\nDKlIJQTjzg6cJBhZwlFA+HxSWfvcDwxwZRgVlsqawwD8BCMuJBU3l5SLnIebNiW3+n0dBgBceCE3\njh6KWNJHrvuwiUCbpdNDUgC+T0T/CGABEf0BgDvAo747h7Ckt6yM5oak9u3Lf1qMNEnvuKlBgGjB\n8HEYW7bwfVRISmKwRYzD8MHHYbz0Eq878W//1viej8OwBUMaB7NmBZVHUYIRNmNt1onmpPLcuTP8\n/ay9pIDwtc7t9w8ciE56A+HdTaOwBWNkhMM8Yec+wAnyD3yAQ05JXHABO4iosFTWiQd96HTBMMb8\nNYAfAPghgFcB+IIx5u+LLFipGBOe9JbxDG7Se//+/PtDpxmHERWSmjOHL6g0DmPOnHrB2LqV7+3l\nQ11kCoiqHEaSYOzYwf9p2DKlYb2kpqaC0cnyGhD06nE7GZQpGFlDUj6CkWUcBhD8hsakcxi9vfy7\npxEqEYwXXwwmHoxq9ff0AN/9LnDiicmfOzAAvOlN0YLx8svZ/8ckOl0wiOhPADxpjPkfxpjPGmN+\nVnC5ymVsjCsNNyQllOkwmhEMoujR3lNTfMGFOQw7xLNtG392XCtQplrPe+CeD5JbikMERfIxNmEO\nA+BzwJ023nYY9v/SDoIh40DydBhuSGp8nM8rn4F7AP9uadcx6e/nBL44jDxb/e98J7BuXTDOw2bH\njqDXVd64swukoR0EA5zgvp2I7iaiy4loic9ORHQeET1NROuJ6MqQ93/fGgh4DxGd6LtvroS5BfsP\nCRvpXXZISkbKxuUwgGjBkGlOkkJSW7eyu4iL3fb28vHbA+DKwicklUUwxsfjcxhpHEaWQXtAeNI7\n60RzcQ7D7g2WBjckFdYZAAgfuAdwxZ+0aFEY0rU2amrzrMgSuT/9aeN727cXJxhhDuP3fg+4/vrk\nfUdH2Ull6VSRA74hqb8wxhwP4JMABgH8kojuiNuHiLoBfB3A+eCV+t5PRMc5m70A4G3GmBMAfAnA\nNSn2zY+wdR3CHEZXV1BZ5h2SWrmS+22/LmIGFhkwFje9ORAtGPa6HjauYNgLLEVhr+zXiknvrA4j\nrpeUj8OYmOAeas06DPv/a9ZhyFQlNq4w+uKGpFzBcENS7ud/9at+laKLCEbeDuNVrwKOOgq4/fbG\n94p0GK5gGAP8+MfA176WvK/ktIpIxnuQdvDdDgDbAOwGkPRrngpgvTHmeWPMGIDrAVxkb2CMuccY\nI1fHfQCW++6bK1KJuN1qwx7Pnl2cw/jVr4ATTojeZtYsPmGmpuIFI6xbrYQ5fByGj2BI75sqHIa4\npSiaFQw3hyFJbyFsABsQ/CZZBaO3l/eV6eWB7IIhU8CEOQw5zqw5DDn2OMEIy5EsWwYcfXS67wSK\ncxgAi4Ybkhod5dsSr0BKetxutTK25b77gvneoqhwanPAP4fxh0T0HwB+DmARgD8wxrw2YbdlAOwJ\nWzbXXoviYwBuSbsvEX2CiNYS0dqdUfHaJHxDUgBfHEXkMHzo6wsq/ijBiFqmNY3DiEt4A1x5Vukw\njKlf+dBFBGPr1saJGN3/zV4BLaqX1IEDfg6jmVHewqpVwIYNwfNmKoiBgXjBKDsklZUVK1iMd+zI\nv+fS4GDQ0UPYsYPvy3IYck4aEx4es2kHwQBPC/LHAK4GsAs80js3iOgMsGB8Pu2+xphrjDFrjDFr\nBiRum5awyj8sJCXbFNFLyodZswLBSJvDiHIYc+bwySsx/N27W99hAPF5DHlvcrKxwty3L+gaDMQ7\njLQhqTwEY+XKQDDCeu+lIUowpHVbZEgqS1I9Cukp9dxz+VeWg4MsELIoGlCdYADAT34Sv2+bCMY2\n8FQgh4JDUf9CRH+UsM8W1C/jurz2Wh1E9FoA3wJwkTFmd5p9c6NdHMasWUG4KSmH4YZs4hwGwL/B\n9u38OMlhVJnDCBvc5mLnONzWo3QFlhhwUi+pNEnvPAVjaioIVeQtGGWEpPJ2GAD/JkU4DGMCkQCq\nE4zBQV6aNi5H1yaC8TEAbzDG/Jkx5gsA3gDgDxL2eQDA0US0iohmArgEQF2nZyI6HMCPAHzIGPNM\nmn1zJSnpbT8uKofhg+0w4gRjcrLxpIvLYQD8G0jlmuQw7AqzbMGQsrlCYGO7DzeP4Y4daUWHIfN5\nNduF8tBDyw1JdXWxSEQN3MvK4YcHj4twGED9+SSCUVQOw+1WK/XPJZfwf3/bbdH75jlDdgZ8BYMA\n2PMsTNZei8QYMwHgcgC3AfgNgO8bY54gosuI6LLaZl8A50T+gYjWEdHauH09y5qesKS3nJh9ffVT\nbovDCBOZovEVDKAxLOXjMCTh5uMwhLIF47DD+D4soS0MDweVVRbBsKcAiRq4NzlZH8YA8sthAOwy\nmg17DgxwLynXbTbbS0quF1cwgGBQ58REfg5j2bLAEebtMMIaIOK0s4a4k4hyGOecw12Pf/zj6H0r\ndhi+62F8G8D9RHRD7fnFAP4paSdjzM0AbnZeu9p6/HEAH/fdtzDCLkx57Faws2dzC2TfPr4g8roo\nfLBDUnE5DIAFw26Z7d3LF517smVxGLZglN3akXXTt8REKIeHgSOPBH7zm0bBsNfzBpIdxvg4X6Su\nw5DPsiuw3bt532Yu6JUr+X7DhuDca0Ywxscbp4TJmsOYP5+PbeNGfh4mGNKTL8vnRzFzJrf2t20r\nzmHYvZMkuV7ExINAYy8pEYx583iOqxtvjA7ptUNIyhjztwAuBfBy7XapMeb/FFmwUglzC93d/Nxt\n0dg5jLJb1/bFGNdLCmjsWjs0xMfiTpMd5jCSrHiVDmPePL5gkgRj0SKuMJsNSQH8W7oOA2gMS8ko\n72b6yB9xBN+/8EI+DgNoDEtlzWEQAa9+NfDkk/w8ymFISDDPxpTkMcpwGEWOwQCiHcbs2cBFF3Fj\nT9Y1d2kHwQAAY8xDxpiv1W4PF1mo0gkLSQH8x4Q5DOklVXbr2r0ww4gKSbmLJwmuwzj00OQL3a48\nyxYMgF1GkmDMncvhqzwEY2Ii3GFECUYz9PdzRb9hQ/M5jCTByOIAjjsOeKIWHa5CMPKuLHt7OQzk\nCkZR+QsgXjDOOYfPr5tuatzPmPYRjI5Ghtu7J/icOa3nMIQsOQxX/IDg5BOHkZS/cL+7VQVjzpzm\nBSPMVdiPXcHYtSufCetkLEbRDiOrYGzdyueXO1U8wL+NCEae01cU5TCAxrEYRU4LAjQKhh3h6O/n\nwYTPPNO439gY585UMComahK9xYsbWxp2L6kqBcMnh2GT5DBGRvxGeQPtIRhZHEZYt1qhLIcBBF1r\nixKMrDkMgAUD4PxQJzgMoFEwig5Jub2k3AjH8uX1o/2FihdPAlQwmKjw0ve+B/zd39W/ZveSKvuP\n83EY8+ZxrNnXYbg5jDQOI8yVlcGyZSwE7ihuwRaM7dvrezPt21dfwfmEpIB0OYxmWbmSE8vSs62Z\nbrVAfjkMIBCMJ59kwZCutEJvb/5JbwBYvZrvsyx9m4QtGJOT7BSrymEA9euY21Q8Uy2ggsG4q+0J\nK1c2trhnz+YKaO/e1gxJdXWFTw/i4zB8Jh60y1GFuwBYMCYmwscYSJx37lyuCKam6gdlRTmMqNlq\nhSSHYUx+grFqFZdF1vPI2jDp7+dy5xmSWrmSP1MEo6+vPslvh6TybExceCGPgn5t0oxEGVi6lM99\nWaFwaqr4HIaswQLwOSlryAPsMHbvrl+oClDBaBnSrOsgFcfu3a0pGED49CBRDkOOYfNmrkjSOIwq\nBQOInlzQmMBhuNsl5TB6eoKeZGkEY3iYRSwvhwEAjz/O98042bDR3s0IRldX0FPKHdAI1Iek8nQY\nPT08HXkRs7QODvJvsmdPMAajaIcBBKFBabDKsUn4zQ1LqWC0CFEOIwzZbteuakNSUTkMgB2G2602\nymF0d/NnSWs2TQ6jasEIy2PY6z6LYEi4YWKiMZToCoZdyaVJeucxaE+wBaO7u7mKVwbv2TSTwwA4\nLBUnGJJ7qSJcmQV7tHfR04IA9ecc0NiIkeWRVTBalCwOY2iofRzG+Dhf3GEOA+ATcP16fuzjMKQc\nVSXfRAh8BUMcxv33s2icckqwvTtbrV2JpnEYIhhZF0+ykbEYL74YvySpD3EOI2uFftxxHGPfsaNR\nMOxztJ0Fo+iQFFAvGPa1ZK9jbiNCrIJRMWkS2LZIlC0YPuMwAG4dibUGguRp1LKr/f3A88/z43Zw\nGEuXcmgkSTCWLOHKVgTjttt4vzPPDLZ3e0lFCUaZDqOvL/gfmhXlvENSQJD4fvDBcIchVLQqXGps\nwSgzJBXlMMRBRzkM7SVVMWlCUvYF0oq9pICgW57MIRQ1j5TQ3x+EKdohh9HTw2KQJBiynS0Yr399\n0PUYyOYw3FlbgXwFAwjCUs2eY2ETEOYlGGEOwz4v29VhdHfXnyN5Y59zQKNgzJ7N55HrMDQk1SKk\nCUlV6TBEMGbMaJziw2bFCj4ZpaKImqlWkGPv6/MbGFW1YADRYzFswQCCsRi7dwMPPACce2799t3d\nfBPBsCu8qnIYQCAYzVYOAwP10/EDQeMga4W+alX9MrY29u/ULg5j7ly+BkQwBgbir69mcR1GWIM1\nbCyGCkaLkNVhVCUYce4CaEya+YSkAG5p+cTLq+5WCyQLhlxUIhh33MGOyxUMIOgX30wOY9cu/u3y\napnm5TDCBu816zB6eng0MtAZDgMIutYWPS0I0NhLKmwQcNhYDA1JtQhr1gDHHOO3rf3HVhWSSisY\n4jDiQlKAX/7C/v5WFgzXYdx2G/ceW7OmcR8fwQgLB7oOY8ECdit5INOcFykYPb6TVYcgYalOyGEA\nweC9oqcFAZKT3kC0w5g1q7n/rUmq++ZW4uc/99+2HRyG28siyWFIa7zdBGPPnsaunWGCsWMHcMst\nwFlnhV9sUYLR3c2t5PHx+u/o6eGbKxh5haOAfENSQH3XWjnOZnpfRQlGO/aSAlgwHnmE/+ujjir2\nu5KS3gBfw7t3179X8eJJgDqM9LRCDiNuDAbALaSenvQOwyfhbX9/1YIBNA7ec+O80rV227bwcBQQ\nCIbbSwoIKkT3N5fFlYSiBKMIh5HH8qnHH8/3nRKSEodR9DxSgJ9gSJTAdtEVz1QLqGCkp8peUu7a\nyVF0dXGFmjaH0W4OA2gMSw0Pc7kkNCSCASQLRtiyoiIUYS1p12HkMQZDkMWvigpJNRsu6sSQ1PAw\nt46zsrQAAA7FSURBVOLLymHEJb3DxmKoYLQhreAwkgQDqE+a7d3Lrb2o/dI6DPmcKu1x1OA9mXjQ\n3e7Vrw4uQhdbMNzfKEqkwwQjT4cxaxZw2WXA+ec39znz57PbzFswjjySHavbGm/nkJRQtMOwu9Ua\nE+8w7DxGCwiG5jDSYl8QrZr0BviEe+ABfizTgkTFrDvNYYQJRpS7AIIlM8Mq0r4+/t3d365owQCA\nb3yj+c8gahyLkYdgzJgBrFvXWLm2s8MQygxJHTzIohGW9AaCRp8xwKOPAqefXmzZElCHkZaurupi\n+L45DKB+8F7UxINCWoexdClw9tnAG9/ot30RzJvH5U4SjCVLgKuvBj772ejPigtJ9fWFr+1sC8bY\nGLf+8haMvHBHe+eRwwC4J5db0bVrDsNuLJUlGOPjjVObC319fD6Jw3j8cc7DnX12sWVLQB1GFvr6\nuLJo9ZDUwYPcOyZq4kFBpqyWRGsSvb3A7bf7bVsUROFda13BAID/9t/iPysphxEm0LZg5D1oL29c\nwcjDYUTRjgP3gPrGUpk5jCjBAOrDyj/7Gd9XLBjqMLIgf26rh6QAbqEkOYyLL+beRkUsTlMkvoKR\nRLMOox0EI6xbbRG0q8NYtCjoci0dBYrCFgx7eVYXeyzG7bcDxx4bnYcrCRWMLEgFUrbDkIvRNyQF\n8AmX5DCIeNBZuxElGGkTg7JkZlS32iSHIZVxKwtGWQ7DFoy8BjGWQVcXh6Xmzg1vIORJWodx4ABw\n113AOecUWy4PNCSVBflziz6xXGQgmW9ICvBzGO2KLNVqTJCUzuow9uwJ7yX1h3/YuJ4EwIIhzqLV\nHcbixbw+ighFXjmMMERcmx0YWAWDg37XVrOECUZYtGL5cl4B8I47eMxPxeEoQAUjG9LqrKIF1dfn\nd1LL4L1Nm5IdRruybBlXfrt2BWGEvENSF1wQvk9YSCrPcRh5Igndbdt4fEcZDqOdwlHCmWcG/2WR\n2N1qkxwGAHz727xPxT2kABWMbMyeXV2X0uOP51hmEt3d3KV006bOdhgAu6iBAV4neXQ0X8GIQgRj\n717gm9/k86FVBcNeSEoEI+1v5IsIRjslvIWvfKWc70mTwwCAG28E3vSmysdgACoY2ejrq04w7rnH\nf9sVK4Bnn+VV5jrRYcho6I0bgZNPDi6+LIJx8CC7Fd+Krq+PxeKCC3g8wo9+VE44Iwv2eg9AOSGp\ndnQYZeHTrRYIHMbEREvkL4CCk95EdB4RPU1E64noypD3jyWie4noIBF91nlvAxE9RkTriGhtkeVM\nzcKF7dGjaPlyXnsZ6EyHsXo137/wAt+7Ew/6MnNmIDZpHMbLL/Oyr9dfD1x4YbrvLBN3bfMyQlLt\n6DDKorubk+xJOQxx0EBL5C+AAh0GEXUD+DqAswFsBvAAEd1ojHnS2uxlAJ8CcHHEx5xhjAnJOFbM\nV74SVE6tzPLlQTk70WEccggLYR6CIfv6VnQLFnBS97rrgHe/O933lY0sCCQTNZYxDkMdRjwSBo1z\nGH19HOacnAR+53fKLV8ERYakTgWw3hjzPAAQ0fUALgLwW8EwxuwAsIOIfrfAcuSPxBZbHbvPdic6\nDCIebSzrkTcjGJLA9g0rfeYzLBQnnpjuu6qgu5sT32U6DBWMeHwEAwBOOYVDry3SRblIwVgGwF4y\najOA16fY3wC4g4gmAfyjMeaaPAs3LbCFrRMdBsCC8cwz/LgZwQh7HMf8+e0hFsLgYOAwisxhaEjK\nDxGM0VFu+EQ1VG65hbuNtwitPHDvNGPMSQDOB/BJInpr2EZE9AkiWktEa3e6i91Pd2zB6ESHAXAe\n44UX+KIqUzDaDVnvAVCH0QrIYFGZqTZqzEpXV8u4C6BYwdgCwB7Hvrz2mhfGmC21+x0AbgCHuMK2\nu8YYs8YYs2ag6CH97YYdkupkh7F/Py+t6a7n7ct0EIzDDitHMIj4szv1d8wLGUAZtjxrC1OkYDwA\n4GgiWkVEMwFcAuBGnx2JqJ+I5spjAOcAeLywknYqS5YErZNOFgyAXYY6jGgGB3k1uahp3POkt1cd\nRhJ2DqPKZQJSUlgOwxgzQUSXA7gNQDeAa40xTxDRZbX3ryaipQDWApgHYIqIPg3gOACHAriB2Kb1\nAPhXY8ytRZW1Y7EH7xU1UKtqbMGQ5VlVMBqRrrXbtxebwwC4p1Sn/o55IYIxPq6CIRhjbgZws/Pa\n1dbjbeBQlcsQgDbKKLYwK1bweIGeDh2jKVOyv/BCsORlWotvV26tOviuWWTw3ksvpRugmAV1GMmI\nYExMtJVgtHLSW8mDFSs6NxwF8MW2dCl3rZWZartSntZ25dapLWNxGBs38r0KRrW0aUhKBaPTueqq\nfJb6bGVWrQpyGFlCb9MhJCUOowzBGBho3Zl7WwW7l1QbJb07NE6h/JYTT2yv8QJZWLWK59havFgF\nI4rFi9l5iWAU6QB+8AO/NVumMzJ/mToMRSmZ1auBF1/kNS1UMMLp6WHRKMNhLFumDiMJu1utCoai\nlMiqVTy1+ZNPqmDEMThYjmAoydgjvVUwFKVEpGvt5s3NC0an9pICOPGtgtEaaNJbUSpCpjkHsi0y\nM50cxt69/Fh7MVWLncNoo6S3CobS/ixfHowz0ZBUNNK1Fujs42wHZs7kgaZTU+owFKVUuruD1fdU\nMKKRrrVAZx9nOzBjBnfSAFQwFKV0JI+hghGN7TA0JFUtksMAVDAUpXQkj6GCEY06jNbB/v1VMBSl\nZPJyGJ065xagOYxWwv79NemtKCWTh2D09kYvZNMJLFkSHJ8KRrWow1CUCjnhBL63F43yRS7eTq9E\ne3p4nidAcxhV06aC0cH+W5lWHH88sGFD0FsqDVJ5drpgAByW2rFjehxrK9OmgqEOQ+kcjjgiW0hp\nujgMIEh8T4djbWVsh6eCoShtxHQSDEl8T4djbWU06a0obcp0EgxxGJrDqBYNSSlKm9LdzWtFdPLE\ng8LJJ/MKjAsWVF2S6Y0KhqK0MTNnTg+H8a53Abt2tVUl1ZHY51obLTalgqEowPQRDKLOHpzYLsi5\nNnt2W439UcFQFGD6CIbSGsi51kYJb0AFQ1EYFQylTKTTQZuFBlUwFAVQwVDKxQ5JtREqGIoC8AU8\nHXpJKa1BmwqGZr8UBQDe9z4eKa4oZaCCoShtzBe/WHUJlOmEJr0VRVEUL9rUYRQqGER0HhE9TUTr\niejKkPePJaJ7ieggEX02zb6Koihti/aSqoeIugF8HcD5AI4D8H4iOs7Z7GUAnwLw1xn2VRRFaU/U\nYTRwKoD1xpjnjTFjAK4HcJG9gTFmhzHmAQDjafdVFEVpW1QwGlgGYJP1fHPttVz3JaJPENFaIlq7\nc+fOTAVVFEUpFU16V4Mx5hpjzBpjzJoBWX5SURSllRHB6OurthwpKbJb7RYA9gLLy2uvFb2voihK\nazN3LvDlLwPveU/VJUlFkYLxAICjiWgVuLK/BMAHSthXURSl9bnqqqpLkJrCBMMYM0FElwO4DUA3\ngGuNMU8Q0WW1968moqUA1gKYB2CKiD4N4DhjzFDYvkWVVVEURUmGjDFVlyE31qxZY9auXVt1MRRF\nUdoGInrQGLPGZ9u2T3oriqIo5aCCoSiKonihgqEoiqJ4oYKhKIqieKGCoSiKonihgqEoiqJ40VHd\naoloJ4CNGXc/FMCuHIvTDkzHYwam53FPx2MGpudxpz3mI4wxXvMqdZRgNAMRrfXti9wpTMdjBqbn\ncU/HYwam53EXecwaklIURVG8UMFQFEVRvFDBCLim6gJUwHQ8ZmB6Hvd0PGZgeh53YcesOQxFURTF\nC3UYiqIoihfTXjCI6DwiepqI1hPRlVWXpyiIaAUR3UlETxLRE0T0x7XXFxLRz4jo2dr9IVWXNW+I\nqJuIHiaif689nw7HvICIfkBETxHRb4jojZ1+3ER0Re3cfpyIvkdEszrxmInoWiLaQUSPW69FHicR\nXVWr354monOb+e5pLRhE1A3g6wDOB3AcgPcT0XHVlqowJgB8xhhzHIA3APhk7VivBPBzY8zRAH5e\ne95p/DGA31jPp8Mx/x2AW40xxwI4EXz8HXvcRLQMwKcArDHGvAa8js4l6Mxj/mcA5zmvhR5n7Rq/\nBMDxtX3+oVbvZWJaCwaAUwGsN8Y8b4wZA3A9gIsqLlMhGGO2GmMeqj0eBlcgy8DH+/9qm/0/ABdX\nU8JiIKLlAH4XwLeslzv9mOcDeCuAfwIAY8yYMeYVdPhxgxeE6yOiHgCzAbyEDjxmY8xdAF52Xo46\nzosAXG+MOWiMeQHAenC9l4npLhjLAGyynm+uvdbRENFKACcDuB/AEmPM1tpb2wAsqahYRfF/AHwO\nwJT1Wqcf8yoAOwF8uxaK+xYR9aODj9sYswXAXwN4EcBWAHuNMbejg4/ZIeo4c63jprtgTDuIaA6A\nHwL4tDFmyH7PcJe5juk2R0TvALDDGPNg1Daddsw1egCcAuAbxpiTAYzCCcV02nHXYvYXgcXyMAD9\nRPRBe5tOO+YoijzO6S4YWwCssJ4vr73WkRDRDLBYfNcY86Pay9uJaLD2/iCAHVWVrwDeDOCdRLQB\nHG58OxH9Czr7mAFuRW42xtxfe/4DsIB08nGfBeAFY8xOY8w4gB8BeBM6+5htoo4z1zpuugvGAwCO\nJqJVRDQTnBy6seIyFQIRETim/RtjzN9ab90I4CO1xx8B8JOyy1YUxpirjDHLjTErwf/tL4wxH0QH\nHzMAGGO2AdhERK+qvXQmgCfR2cf9IoA3ENHs2rl+JjhP18nHbBN1nDcCuISIeoloFYCjAfxn1i+Z\n9gP3iOgCcJy7G8C1xpj/XXGRCoGITgNwN4DHEMTz/xScx/g+gMPBM/3+F2OMm1Bre4jodACfNca8\ng4gWocOPmYhOAif6ZwJ4HsCl4AZixx43Ef0FgPeBewQ+DODjAOagw46ZiL4H4HTwrLTbAfwZgB8j\n4jiJ6H8C+Cj4d/m0MeaWzN893QVDURRF8WO6h6QURVEUT1QwFEVRFC9UMBRFURQvVDAURVEUL1Qw\nFEVRFC9UMBRFURQvVDAURVEUL1QwFEVRFC/+P4Tik4xuToiEAAAAAElFTkSuQmCC\n", 2364 | "text/plain": [ 2365 | "" 2366 | ] 2367 | }, 2368 | "metadata": {}, 2369 | "output_type": "display_data" 2370 | }, 2371 | { 2372 | "data": { 2373 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAD8CAYAAABthzNFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEcBJREFUeJzt3X2sZHV9x/H3h0VBoIVFtpvlqUubDQZJLPbW4kONiklV\nrOAf4trSbC1K0yKiwSjUNNg/TPnDEG2iphsUt2pAilQ2xqDr+hTTFnsR0wILhQqL4C57taLWVPAu\n3/4xZ/Wy/Obu7N2dmbsz71dyM3POPJzfD/G8OefcuZOqQpKkvR027gFIkpYnAyFJajIQkqQmAyFJ\najIQkqQmAyFJajIQkqQmAyFJajIQkqSmw8c9gANxwgkn1Nq1a8c9DEk6pNx+++0/qKpV+3reIR2I\ntWvXMjs7O+5hSNIhJcn2QZ7nKSZJUpOBkCQ1GQhJUpOBkCQ1GQhJUtPQApHk40l2Jblzwbrjk2xJ\ncl93u3LBY1cmuT/JvUn+cFjjkiQNZphHEJ8AXrXXuiuArVW1DtjaLZPkDGA98NzuNR9JsmKIY5Mk\n7cPQPgdRVd9Isnav1ecBL+vubwK+BrynW39DVT0OPJDkfuAFwL8Oa3y/Gid8+MOwa9ewtyRJB8+Z\nZ8IFFwx3G6P+oNzqqtrR3d8JrO7unwT824LnPdyte5okFwMXA5x66qkHPKAHH4RLL93z3gf8dpI0\nEm984+QF4peqqpLUEl63EdgIMDMzs9+v39vjj/dur78e1q8/0HeTpMkx6t9iejTJGoDuds+JnUeA\nUxY87+Ru3dDNz/duV3jFQ5KeYtSB2Axs6O5vAG5ZsH59kiOSnAasA741igHtCcThh/RfpZKkg29o\nu8Uk19O7IH1CkoeBq4CrgRuTXARsBy4AqKq7ktwI3A3MA5dU1e5hjW0hAyFJbcP8LaY39XnonD7P\nfz/w/mGNp5/dXYYMhCQ91dR/ktojCElqMxAGQpKaDISBkKQmA2EgJKnJQBgISWoyEAZCkpoMhIGQ\npCYDYSAkqclAGAhJajIQBkKSmgyEgZCkJgNhICSpyUAYCElqMhAGQpKaDISBkKQmA2EgJKnJQBgI\nSWoyEF0gVqwY7zgkabkxEPNw2GG9H0nSr0z9bnF+3tNLktRiIAyEJDUZCAMhSU0GwkBIUpOBMBCS\n1GQgDIQkNRkIAyFJTQbCQEhSk4EwEJLUZCAMhCQ1GQgDIUlNBsJASFLTWAKR5J1J7kpyZ5LrkxyZ\n5PgkW5Lc192uHMVYDIQktY08EElOAt4OzFTVmcAKYD1wBbC1qtYBW7vloTMQktQ2rlNMhwPPSnI4\ncBTwfeA8YFP3+Cbg/FEMxEBIUtvIA1FVjwAfAB4CdgA/rqovAaurakf3tJ3A6tbrk1ycZDbJ7Nzc\n3AGPx0BIUts4TjGtpHe0cBpwInB0kgsXPqeqCqjW66tqY1XNVNXMqlWrDng8BkKS2sZxiumVwANV\nNVdVvwBuBl4EPJpkDUB3u2sUgzEQktQ2jkA8BJyd5KgkAc4BtgGbgQ3dczYAt4xiMAZCktpGvmus\nqtuS3AR8G5gH7gA2AscANya5CNgOXDCK8RgISWoby66xqq4Crtpr9eP0jiZGykBIUpufpDYQktRk\nIAyEJDUZCAMhSU0GwkBIUpOBMBCS1GQgDIQkNRkIAyFJTQbCQEhSk4EwEJLUNNWBqOoFYsWKcY9E\nkpafqQ7Ek0/2bj2CkKSnm+pAzM/3bg2EJD3dVAdi9+7erYGQpKeb6kB4BCFJ/RkIDIQktRgIDIQk\ntRgIDIQktRgIDIQktRgIDIQktRgIDIQktRgIDIQktRgIDIQktRgIDIQktRgIDIQktRgIDIQktRgI\nDIQktRgIDIQktRgIDIQktRgIDIQktRgIDIQktYwlEEmOS3JTknuSbEvywiTHJ9mS5L7uduWwx2Eg\nJKm/cR1BfAi4taqeAzwP2AZcAWytqnXA1m55qAyEJPU38kAkORZ4KfAxgKp6oqoeA84DNnVP2wSc\nP+yxGAhJ6m+gQCS5Ocm5SQ5GUE4D5oDrktyR5NokRwOrq2pH95ydwOqDsK1FGQhJ6m/QHf5HgD8G\n7ktydZLTD2CbhwPPBz5aVWcBP2Ov00lVVUC1Xpzk4iSzSWbn5uYOYBgGQpIWM1AgqurLVfUn9Hbs\nDwJfTvIvSd6c5Bn7uc2HgYer6rZu+abufR9Nsgagu93VZywbq2qmqmZWrVq1n5t+KgMhSf0NfMoo\nybOBPwPeAtxB70Lz84Et+7PBqtoJfG/BUcg5wN3AZmBDt24DcMv+vO9SGAhJ6m+gXWOSfwZOBz4J\n/NGCawWfSTK7hO1eCnw6yTOB7wJvpherG5NcBGwHLljC++4XAyFJ/Q26a/z7qvpq64GqmtnfjVbV\nd4DW687Z3/c6EAZCkvob9BTTGUmO27OQZGWSvxrSmEbGQEhSf4MG4q3dZxUAqKofAW8dzpBGx0BI\nUn+DBmJFkuxZSLICeOZwhjQ6BkKS+ht013grvQvS/9At/0W37pA2Pw8JHDbVf7JQktoGDcR76EXh\nL7vlLcC1QxnRCM3Pe/QgSf0MtHusqieBj3Y/E8NASFJ/g34OYh3wd8AZwJF71lfVbw1pXCNhICSp\nv0HPvl9H7+hhHng58I/Ap4Y1qFExEJLU36CBeFZVbQVSVdur6n3AucMb1mgYCEnqb9Dd4+Pdn/q+\nL8nbgEeAY4Y3rNEwEJLU36BHEJcBRwFvB34XuJBf/WG9Q5aBkKT+9rl77D4U98aqehfwv/T+sN5E\nMBCS1N8+jyCqajfwkhGMZeQMhCT1N+ju8Y4km4F/ovcNcABU1c1DGdWIGAhJ6m/Q3eORwA+BVyxY\nV4CBkKQJNegnqSfmusNCBkKS+hv0k9TX0TtieIqq+vODPqIRMhCS1N+gu8fPL7h/JPB64PsHfzij\nZSAkqb9BTzF9duFykuuBbw5lRCNkICSpv6V+E8I64DcO5kDGwUBIUn+DXoP4KU+9BrGT3ndEHNLm\n5+HII/f9PEmaRoOeYvq1YQ9kHDyCkKT+BjrFlOT1SY5dsHxckvOHN6zRMBCS1N+g1yCuqqof71mo\nqseAq4YzpNGZn4cVK8Y9CklangYNROt5h/x/e3sEIUn9DRqI2STXJPnt7uca4PZhDmwUDIQk9Tdo\nIC4FngA+A9wA/By4ZFiDGhUDIUn9DfpbTD8DrhjyWEZu924DIUn9DPpbTFuSHLdgeWWSLw5vWKPh\nEYQk9TfoKaYTut9cAqCqfoSfpJakiTZoIJ5McuqehSRrafx110ONgZCk/gbdPb4X+GaSrwMB/gC4\n+EA23H3X9SzwSFW9Nsnx9C6CrwUeBC7ojlSGxkBIUn8DHUFU1a3ADHAvcD1wOfB/B7jty4BtC5av\nALZW1TpgKyO4KG4gJKm/QS9Sv4XeTvty4F3AJ4H3LXWjSU4GzgWuXbD6PGBTd38TMPQ/5WEgJKm/\nQa9BXAb8HrC9ql4OnAU8tvhLFvVB4N3AkwvWra6qHd39ncDqA3j/gRgISepv0ED8vKp+DpDkiKq6\nBzh9KRtM8lpgV1X1/SR2VRV9LoInuTjJbJLZubm5pQwBgCef7P0YCElqG3T3+HD3OYjPAVuS/AjY\nvsRtvhh4XZLX0Pv60l9P8ing0SRrqmpHkjXArtaLq2ojsBFgZmZmyb9JtXt379ZASFLboBepX19V\nj1XV+4C/AT7GEq8RVNWVVXVyVa0F1gNfqaoLgc3Ahu5pG4BblvL+g5qf790aCElq2+/dY1V9fRgD\nAa4GbkxyEb2jkwuGtB3AQEjSvox191hVXwO+1t3/IXDOqLZtICRpcYNepJ44BkKSFmcgDIQkNRkI\nAyFJTQbCQEhSk4EwEJLUZCAMhCQ1GQgDIUlNBsJASFKTgTAQktRkIAyEJDUZCAMhSU0GwkBIUpOB\nMBCS1GQgDIQkNRkIAyFJTQbCQEhSk4EwEJLUZCAMhCQ1GQgDIUlNBsJASFKTgTAQktRkIAyEJDUZ\nCAMhSU0GwkBIUpOBMBCS1GQgDIQkNRkIAyFJTVMfiMOm9p+AJC1uaneP8/O9o4dk3CORpOVp6gMh\nSWozEJKkppEHIskpSb6a5O4kdyW5rFt/fJItSe7rblcOcxwGQpIWN44jiHng8qo6AzgbuCTJGcAV\nwNaqWgds7ZaHNwgDIUmLGnkgqmpHVX27u/9TYBtwEnAesKl72ibg/GGOw0BI0uLGeg0iyVrgLOA2\nYHVV7ege2gmsHua2DYQkLW5sgUhyDPBZ4B1V9ZOFj1VVAdXndRcnmU0yOzc3t+Ttz8/DihVLfrkk\nTbyxBCLJM+jF4dNVdXO3+tEka7rH1wC7Wq+tqo1VNVNVM6tWrVryGDyCkKTFjeO3mAJ8DNhWVdcs\neGgzsKG7vwG4ZZjjMBCStLhx7CJfDPwp8J9JvtOt+2vgauDGJBcB24ELhjkIAyFJixv5LrKqvgn0\n+wMX54xqHAZCkhbnJ6klSU1TG4jduw2EJC1magPhEYQkLc5ASJKaDIQkqclASJKaDIQkqclASJKa\nDIQkqclASJKaDIQkqclASJKaDIQkqclASJKaDIQkqclASJKaDIQkqclASJKaDIQkqWkqA1HlN8pJ\n0r5MZSB27+7dGghJ6m8qAzE/37s1EJLUn4GQJDUZCElSk4GQJDUZCElSk4GQJDUZCElSk4GQJDUZ\nCElSk4GQJDUtu0AkeVWSe5Pcn+SKYWzDQEjSvi2rQCRZAXwYeDVwBvCmJGcc7O0YCEnat2UVCOAF\nwP1V9d2qegK4ATjvYG/k2GPhDW+AE0882O8sSZNjuf039EnA9xYsPwz8/sHeyLp1cOONB/tdJWmy\nLLcjiH1KcnGS2SSzc3Nz4x6OJE2s5RaIR4BTFiyf3K37paraWFUzVTWzatWqkQ5OkqbJcgvEvwPr\nkpyW5JnAemDzmMckSVNpWV2DqKr5JG8DvgisAD5eVXeNeViSNJWWVSAAquoLwBfGPQ5JmnbL7RST\nJGmZMBCSpCYDIUlqSlWNewxLlmQO2H4Ab3EC8IODNJxDxTTOGaZz3s55euzvvH+zqvb5OYFDOhAH\nKslsVc2MexyjNI1zhumct3OeHsOat6eYJElNBkKS1DTtgdg47gGMwTTOGaZz3s55egxl3lN9DUKS\n1N+0H0FIkvqYykCM4mtNxy3JKUm+muTuJHcluaxbf3ySLUnu625Xjnusw5BkRZI7kny+W57oeSc5\nLslNSe5Jsi3JCyd9zgBJ3tn9+31nkuuTHDmJ807y8SS7kty5YF3feSa5stu/3ZvkD5e63akLxKi+\n1nQZmAcur6ozgLOBS7p5XgFsrap1wNZueRJdBmxbsDzp8/4QcGtVPQd4Hr25T/Sck5wEvB2Yqaoz\n6f2Bz/VM5rw/Abxqr3XNeXb/P18PPLd7zUe6/d5+m7pAMKKvNR23qtpRVd/u7v+U3g7jJHpz3dQ9\nbRNw/nhGODxJTgbOBa5dsHpi553kWOClwMcAquqJqnqMCZ7zAocDz0pyOHAU8H0mcN5V9Q3gf/Za\n3W+e5wE3VNXjVfUAcD+9/d5+m8ZAtL7W9KQxjWUkkqwFzgJuA1ZX1Y7uoZ3A6jENa5g+CLwbeHLB\nukme92nAHHBdd1rt2iRHM9lzpqoeAT4APATsAH5cVV9iwue9QL95HrR93DQGYqokOQb4LPCOqvrJ\nwseq9ytsE/VrbEleC+yqqtv7PWcC53048Hzgo1V1FvAz9jqtMoFzpjvnfh69QJ4IHJ3kwoXPmcR5\ntwxrntMYiH1+remkSPIMenH4dFXd3K1+NMma7vE1wK5xjW9IXgy8LsmD9E4fviLJp5jseT8MPFxV\nt3XLN9ELxiTPGeCVwANVNVdVvwBuBl7E5M97j37zPGj7uGkMxFR8rWmS0Dsnva2qrlnw0GZgQ3d/\nA3DLqMc2TFV1ZVWdXFVr6f1v+5WqupAJnndV7QS+l+T0btU5wN1M8Jw7DwFnJzmq+/f9HHrX2iZ9\n3nv0m+dmYH2SI5KcBqwDvrWkLVTV1P0ArwH+C/hv4L3jHs+Q5vgSeoec/wF8p/t5DfBser/xcB/w\nZeD4cY91iP8MXgZ8vrs/0fMGfgeY7f73/hywctLn3M37b4F7gDuBTwJHTOK8gevpXWf5Bb0jxosW\nmyfw3m7/di/w6qVu109SS5KapvEUkyRpAAZCktRkICRJTQZCktRkICRJTQZCktRkICRJTQZCktT0\n/7cKuqD7I8m7AAAAAElFTkSuQmCC\n", 2374 | "text/plain": [ 2375 | "" 2376 | ] 2377 | }, 2378 | "metadata": {}, 2379 | "output_type": "display_data" 2380 | } 2381 | ], 2382 | "source": [ 2383 | "#print(overallError_history)\n", 2384 | "x_range = range(max_iter//100)\n", 2385 | "plt.plot(x_range,overallError_history,'r-')\n", 2386 | "plt.ylabel('overallError')\n", 2387 | "plt.show()\n", 2388 | "\n", 2389 | "plt.plot(x_range,accuracy_history,'b-')\n", 2390 | "plt.ylabel('accuracy')\n", 2391 | "plt.show()" 2392 | ] 2393 | } 2394 | ], 2395 | "metadata": { 2396 | "anaconda-cloud": {}, 2397 | "kernelspec": { 2398 | "display_name": "Python [conda env:tensorflow35]", 2399 | "language": "python", 2400 | "name": "conda-env-tensorflow35-py" 2401 | }, 2402 | "language_info": { 2403 | "codemirror_mode": { 2404 | "name": "ipython", 2405 | "version": 3 2406 | }, 2407 | "file_extension": ".py", 2408 | "mimetype": "text/x-python", 2409 | "name": "python", 2410 | "nbconvert_exporter": "python", 2411 | "pygments_lexer": "ipython3", 2412 | "version": "3.5.2" 2413 | } 2414 | }, 2415 | "nbformat": 4, 2416 | "nbformat_minor": 1 2417 | } 2418 | --------------------------------------------------------------------------------