├── PCA and autoencoders.ipynb ├── README.md ├── autoencoder.png └── deep_autoencoder.png /PCA and autoencoders.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# PCA and autoencoders" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "This notebook explores dimensionality reduction through autoencoding of a classic dataset.\n", 15 | "\n", 16 | "Let's start be loading our dependencies:" 17 | ] 18 | }, 19 | { 20 | "cell_type": "code", 21 | "execution_count": 83, 22 | "metadata": { 23 | "collapsed": true 24 | }, 25 | "outputs": [], 26 | "source": [ 27 | "# Numpy is our primary dependency\n", 28 | "import numpy as np\n", 29 | "\n", 30 | "# Import datasets from scikit-learn only to get the iris data set\n", 31 | "from sklearn import datasets\n", 32 | "\n", 33 | "# We will need some plotting too\n", 34 | "import matplotlib.pyplot as plt" 35 | ] 36 | }, 37 | { 38 | "cell_type": "markdown", 39 | "metadata": {}, 40 | "source": [ 41 | "## The Dataset" 42 | ] 43 | }, 44 | { 45 | "cell_type": "markdown", 46 | "metadata": {}, 47 | "source": [ 48 | "We will use the iris data set for this exercise:" 49 | ] 50 | }, 51 | { 52 | "cell_type": "code", 53 | "execution_count": 2, 54 | "metadata": { 55 | "collapsed": false 56 | }, 57 | "outputs": [], 58 | "source": [ 59 | "# Load iris data set\n", 60 | "iris = datasets.load_iris()\n", 61 | "\n", 62 | "# Extract data and feature names\n", 63 | "data = iris.data\n", 64 | "feature_names = iris.feature_names\n", 65 | "\n", 66 | "# Similarly for labels and label names\n", 67 | "labels = iris.target\n", 68 | "label_names = iris.target_names" 69 | ] 70 | }, 71 | { 72 | "cell_type": "markdown", 73 | "metadata": {}, 74 | "source": [ 75 | "The dataset describes four characteristics of a sample of three different species of iris flowers. Our aim is to reduce the four dimension to two, so that we can make a scatter plot of the data set.\n", 76 | "\n", 77 | "The four features are:" 78 | ] 79 | }, 80 | { 81 | "cell_type": "code", 82 | "execution_count": 41, 83 | "metadata": { 84 | "collapsed": false 85 | }, 86 | "outputs": [ 87 | { 88 | "data": { 89 | "text/plain": [ 90 | "['sepal length (cm)',\n", 91 | " 'sepal width (cm)',\n", 92 | " 'petal length (cm)',\n", 93 | " 'petal width (cm)']" 94 | ] 95 | }, 96 | "execution_count": 41, 97 | "metadata": {}, 98 | "output_type": "execute_result" 99 | } 100 | ], 101 | "source": [ 102 | "feature_names" 103 | ] 104 | }, 105 | { 106 | "cell_type": "markdown", 107 | "metadata": {}, 108 | "source": [ 109 | "The three species labels are:" 110 | ] 111 | }, 112 | { 113 | "cell_type": "code", 114 | "execution_count": 43, 115 | "metadata": { 116 | "collapsed": false 117 | }, 118 | "outputs": [ 119 | { 120 | "data": { 121 | "text/plain": [ 122 | "array(['setosa', 'versicolor', 'virginica'], \n", 123 | " dtype='" 285 | ] 286 | }, 287 | "metadata": {}, 288 | "output_type": "display_data" 289 | } 290 | ], 291 | "source": [ 292 | "# Grab each component\n", 293 | "pc1 = np.array(pc2d[:, 0].transpose())\n", 294 | "pc2 = np.array(pc2d[:, 1].transpose())\n", 295 | "\n", 296 | "# Plot\n", 297 | "plt.scatter(pc1, pc2, c=labels)\n", 298 | "plt.show()" 299 | ] 300 | }, 301 | { 302 | "cell_type": "markdown", 303 | "metadata": {}, 304 | "source": [ 305 | "Each color corresponds to a species of iris." 306 | ] 307 | }, 308 | { 309 | "cell_type": "markdown", 310 | "metadata": {}, 311 | "source": [ 312 | "## Autoencoders" 313 | ] 314 | }, 315 | { 316 | "cell_type": "markdown", 317 | "metadata": {}, 318 | "source": [ 319 | "PCA is an example of an *autoencoder*.\n", 320 | "\n", 321 | "An autoencoder consists of two parts: A *coder* and a *decoder*. The coder transforms the origin data points into a feature space of lower dimension than the original dataset. The decoder then transforms from the low dimension space back into the origin set of dimensions. The aim is for the decoder to get as close to the original data as possible.\n", 322 | "\n", 323 | "When the coder and decoder are *linear transformations*, the resulting optimal autoencoder is equivalent to PCA.\n", 324 | "\n", 325 | "But what if we add non-linearities? The situation is similar to adding an activation function to a neural net, and indeed that is what we will do here. The resulting neural net is shown in the figure below. This has a minimum number of layers." 326 | ] 327 | }, 328 | { 329 | "cell_type": "markdown", 330 | "metadata": {}, 331 | "source": [ 332 | "![Structure of a shallow neural net autoencoder](autoencoder.png)" 333 | ] 334 | }, 335 | { 336 | "cell_type": "markdown", 337 | "metadata": {}, 338 | "source": [ 339 | "## The neural net model" 340 | ] 341 | }, 342 | { 343 | "cell_type": "markdown", 344 | "metadata": {}, 345 | "source": [ 346 | "But we could add several additional layers in the coder and decoder. In this case we get a *deep autoencoder*. Here, we will make each two layers deep, such that:\n", 347 | "\n", 348 | "- The coder transforms from 4 to 3 to 2 dimensions.\n", 349 | "- The decoder transforms from 2 to 3 to 4 dimensions.\n", 350 | "\n", 351 | "I.e. our model will be the one depicted below:" 352 | ] 353 | }, 354 | { 355 | "cell_type": "markdown", 356 | "metadata": {}, 357 | "source": [ 358 | "![The autoencoder model we will use](deep_autoencoder.png)" 359 | ] 360 | }, 361 | { 362 | "cell_type": "markdown", 363 | "metadata": {}, 364 | "source": [ 365 | "### Activation functions" 366 | ] 367 | }, 368 | { 369 | "cell_type": "markdown", 370 | "metadata": {}, 371 | "source": [ 372 | "We will use a number of activation functions to study their differences. Each function will return the activation values as well as the derivatives." 373 | ] 374 | }, 375 | { 376 | "cell_type": "code", 377 | "execution_count": 190, 378 | "metadata": { 379 | "collapsed": false 380 | }, 381 | "outputs": [], 382 | "source": [ 383 | "# No activation function\n", 384 | "def linear(x):\n", 385 | " linear_grad = np.ones(x.shape)\n", 386 | " return x, linear_grad\n", 387 | "\n", 388 | "# Sigmoid\n", 389 | "def sigmoid(x):\n", 390 | " sigmoid = 1 / (1 + np.exp(-x))\n", 391 | " sigmoid_grad = sigmoid * (1 - sigmoid)\n", 392 | " return sigmoid, sigmoid_grad\n", 393 | "\n", 394 | "# ReLU\n", 395 | "def relu(x):\n", 396 | " relu = np.maximum(x, np.zeros(x.shape))\n", 397 | " relu_grad = np.ones(x.shape) * (x > 0)\n", 398 | " return relu, relu_grad" 399 | ] 400 | }, 401 | { 402 | "cell_type": "markdown", 403 | "metadata": {}, 404 | "source": [ 405 | "### Neural network math" 406 | ] 407 | }, 408 | { 409 | "cell_type": "markdown", 410 | "metadata": {}, 411 | "source": [ 412 | "We will represent our neural network as a list of dictionaries, each corresponding to a layer of neurons:" 413 | ] 414 | }, 415 | { 416 | "cell_type": "code", 417 | "execution_count": 78, 418 | "metadata": { 419 | "collapsed": false 420 | }, 421 | "outputs": [], 422 | "source": [ 423 | "# Make a new layer\n", 424 | "def new_layer(dim_in, dim_out):\n", 425 | " weights = np.random.rand(dim_out, dim_in) - 0.5\n", 426 | " bias = np.random.rand(dim_out, 1) - 0.5\n", 427 | " \n", 428 | " return {\n", 429 | " 'weights': weights,\n", 430 | " 'bias': bias,\n", 431 | " 'activations': np.zeros((dim_out, AEdata.shape[1])),\n", 432 | " 'act_grad': np.zeros((dim_out, AEdata.shape[1])),\n", 433 | " 'errors': np.zeros((dim_out, AEdata.shape[1])),\n", 434 | " 'weights_grad': np.zeros(weights.shape),\n", 435 | " 'bias_grad': np.zeros(bias.shape)\n", 436 | " }\n", 437 | "\n", 438 | "# Our specific models hyperparameters\n", 439 | "def initialize_model():\n", 440 | " return [new_layer(4, 3), new_layer(3, 2), new_layer(2, 3), new_layer(3, 4)]" 441 | ] 442 | }, 443 | { 444 | "cell_type": "markdown", 445 | "metadata": {}, 446 | "source": [ 447 | "We need to be able to forward propagate values ..." 448 | ] 449 | }, 450 | { 451 | "cell_type": "code", 452 | "execution_count": 169, 453 | "metadata": { 454 | "collapsed": false 455 | }, 456 | "outputs": [], 457 | "source": [ 458 | "def forward_propagate(layers, inputs, activation_function):\n", 459 | " for layer in layers:\n", 460 | " zs = np.matmul(layer['weights'], inputs) + layer['bias']\n", 461 | " activations, act_grad = activation_function(zs)\n", 462 | " layer['activations'] = activations\n", 463 | " layer['act_grad'] = act_grad\n", 464 | " inputs = activations" 465 | ] 466 | }, 467 | { 468 | "cell_type": "markdown", 469 | "metadata": {}, 470 | "source": [ 471 | "... and backpropagate values:" 472 | ] 473 | }, 474 | { 475 | "cell_type": "code", 476 | "execution_count": 170, 477 | "metadata": { 478 | "collapsed": true 479 | }, 480 | "outputs": [], 481 | "source": [ 482 | "def backward_propagate(layers, inputs, outputs): \n", 483 | " # Initiate errors for the last layer (normalized)\n", 484 | " errors = (layers[-1]['activations'] - outputs)\n", 485 | " \n", 486 | " # Go backwards through the layers\n", 487 | " for layer_number, layer in reversed(list(enumerate(layers))):\n", 488 | " layer['errors'] = errors\n", 489 | " \n", 490 | " # Calculate Hadamard product between errors and activation gradients\n", 491 | " hadamard = errors * layer['act_grad']\n", 492 | " \n", 493 | " # Get activations of previous layer - if at the first layer use input\n", 494 | " if layer_number == 0:\n", 495 | " last_activations = inputs\n", 496 | " else:\n", 497 | " last_activations = layers[layer_number - 1]['activations']\n", 498 | " \n", 499 | " # Calculate derivatives of weights\n", 500 | " weights_grad = np.matmul(hadamard, last_activations.transpose()) / inputs.shape[1]\n", 501 | " layer['weights_grad'] = weights_grad\n", 502 | " \n", 503 | " # Calculate derivatives of biases\n", 504 | " bias_grad = hadamard.sum(axis=1).reshape(layer['bias'].shape) / inputs.shape[1]\n", 505 | " layer['bias_grad'] = bias_grad\n", 506 | " \n", 507 | " # Backpropagate errors, unless we're at the first layer\n", 508 | " if layer_number != 0:\n", 509 | " errors = np.matmul(layer['weights'].transpose(), hadamard)" 510 | ] 511 | }, 512 | { 513 | "cell_type": "markdown", 514 | "metadata": {}, 515 | "source": [ 516 | "And we need to be able to update weights and biases based on the results of the backpropagation:" 517 | ] 518 | }, 519 | { 520 | "cell_type": "code", 521 | "execution_count": 171, 522 | "metadata": { 523 | "collapsed": false 524 | }, 525 | "outputs": [], 526 | "source": [ 527 | "def gradient_descent(layers, learning_rate):\n", 528 | " for layer in layers:\n", 529 | " layer['weights'] = layer['weights'] - layer['weights_grad'] * learning_rate\n", 530 | " layer['bias'] = layer['bias'] - layer['bias_grad'] * learning_rate" 531 | ] 532 | }, 533 | { 534 | "cell_type": "markdown", 535 | "metadata": {}, 536 | "source": [ 537 | "Finally, we need the value of the error function to visualize training:" 538 | ] 539 | }, 540 | { 541 | "cell_type": "code", 542 | "execution_count": 172, 543 | "metadata": { 544 | "collapsed": false 545 | }, 546 | "outputs": [], 547 | "source": [ 548 | "def error_function(layers):\n", 549 | " errors = layers[-1]['errors']\n", 550 | " e_squared = errors ** 2\n", 551 | " return e_squared.sum() / errors.shape[1]" 552 | ] 553 | }, 554 | { 555 | "cell_type": "markdown", 556 | "metadata": {}, 557 | "source": [ 558 | "## Training the autoencoder" 559 | ] 560 | }, 561 | { 562 | "cell_type": "markdown", 563 | "metadata": {}, 564 | "source": [ 565 | "The following function will initialize and train our autoencoder:" 566 | ] 567 | }, 568 | { 569 | "cell_type": "code", 570 | "execution_count": 178, 571 | "metadata": { 572 | "collapsed": false 573 | }, 574 | "outputs": [], 575 | "source": [ 576 | "def trainAE(activation_function, learning_rate, epochs):\n", 577 | " # Initialize model\n", 578 | " layers = initialize_model()\n", 579 | " \n", 580 | " # Array of errors\n", 581 | " error_history = []\n", 582 | " \n", 583 | " # Run through the epochs\n", 584 | " for epoch in range(epochs):\n", 585 | " forward_propagate(layers, AEdata, activation_function)\n", 586 | " backward_propagate(layers, AEdata, AEdata)\n", 587 | " gradient_descent(layers, learning_rate)\n", 588 | " error_history.append(error_function(layers))\n", 589 | " \n", 590 | " # Return trained model, and error_history for plotting\n", 591 | " return layers, error_history" 592 | ] 593 | }, 594 | { 595 | "cell_type": "markdown", 596 | "metadata": {}, 597 | "source": [ 598 | "### Linear autoencoder" 599 | ] 600 | }, 601 | { 602 | "cell_type": "markdown", 603 | "metadata": {}, 604 | "source": [ 605 | "First, let's train a linear autoencoder.\n", 606 | "\n", 607 | "The error history is plotted below." 608 | ] 609 | }, 610 | { 611 | "cell_type": "code", 612 | "execution_count": 193, 613 | "metadata": { 614 | "collapsed": false 615 | }, 616 | "outputs": [ 617 | { 618 | "data": { 619 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEACAYAAABfxaZOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGH5JREFUeJzt3XuQFOW5x/HvwyIKqAjeRUEiUbyhhRGJmjheoqgpLydl\noibxkjoJVYk5alkJGsu4FZM6sUo9xpBzDInR3LxrCJ6oIZpsTsUgXlBRBMFLEBbECyKiCLvLc/54\nZ5xh2WVnd3vnss/vUzW13T093e801K/fft+3e8zdERGRGAZUuwAiIlI5Cn0RkUAU+iIigSj0RUQC\nUeiLiASi0BcRCaTL0DezW8xspZnN28I6N5nZYjN71swOzbaIIiKSlXJq+rcCJ3X2ppmdDOzj7p8E\npgA3Z1Q2ERHJWJeh7+7/AN7dwiqnA7/JrzsHGGZmu2ZTPBERyVIWbfojgaUl8835ZSIiUmPUkSsi\nEsjADLbRDOxVMr9nftlmzEwP+hER6QF3tyy2U25N3/KvjswEzgMws0nAandf2dmG3F0vd66++uqq\nl6FWXjoWOhY6Flt+ZanLmr6Z3Q7kgB3N7HXgamBQym+f7u4PmtkpZvYy8AFwYaYlFBGRzHQZ+u5+\nbhnrXJRNcUREpC+pI7dKcrlctYtQM3QsinQsinQs+oZl3V60xZ2ZeSX3JyLSH5gZXuGOXBER6QcU\n+iIigSj0RUQCCRX6y5bBkUdWuxQiItUTKvSfeAJmz652KUREqidU6GvgkIhEp9AXEQkkVOiLiEQX\nKvRV0xeR6BT6IiKBKPRFRAJR6IuIBKLQFxEJRKEvIhKIQl9EJBCFvohIIAp9EZFAFPoiIoEo9EVE\nAgkV+iIi0YUKfdX0RSQ6hb6ISCAKfRGRQEKF/saN1S6BiEh1hQp9EZHoQoW+mndEJDqFvohIIAp9\nEZFAFPoiIoEo9EVEAlHoi4gEotAXEQlEoS8iEohCX0QkkLJC38wmm9lCM1tkZlM7eH97M5tpZs+a\n2fNmdkHmJRURkV7rMvTNbAAwDTgJOBA4x8zGtVvtW8B8dz8UOBa43swGZl3Y3lJNX0SiK6emPxFY\n7O5L3L0FuBM4vd06DmyXn94OeMfdW7MrZjYU+iISXTmhPxJYWjK/LL+s1DTgADNbDjwHXJxN8bKl\n0BeR6LJqgjkJeMbdjzOzfYC/mNl4d1/bfsXGxsaPp3O5HLlcLqMidE2hLyL1oKmpiaampj7ZtnkX\nSWhmk4BGd5+cn78ccHe/tmSd/wX+090fy88/Ckx196fabcu72l9fuuEGuOwyhb+I1Bczw90ti22V\n07zzJDDWzEab2SDgbGBmu3WWACfkC7crsC/wahYFzJLCXkSi67J5x93bzOwiYBbpJHGLuy8wsynp\nbZ8O/BC4zczm5T/2XXdf1Wel7iGFvohEV1abvrs/DOzXbtnPS6ZXkNr1a5pCX0Si0x25IiKBhAp9\nEZHoQoW+avoiEp1CX0QkEIW+iEggCn0RkUAU+iIigSj0RUQCUeiLiASi0BcRCUShLyISSKjQFxGJ\nLlToq6YvItEp9EVEAlHoi4gEotAXEQlEoS8iEohCX0QkEIW+iEggCn0RkUAU+iIigSj0RUQCUeiL\niAQSKvQ3bqx2CUREqitU6KumLyLRhQp91fRFJLpQoa+avohEFyr0VdMXkehChb5q+iISXajQV01f\nRKILFfqq6YtIdKFCXzV9EYkuVOirpi8i0Sn0RUQCUeiLiARSVuib2WQzW2hmi8xsaifr5MzsGTN7\nwcz+lm0xs6E2fRGJbmBXK5jZAGAacDywHHjSzP7o7gtL1hkG/Aw40d2bzWynvipwb6imLyLRlVPT\nnwgsdvcl7t4C3Amc3m6dc4H73L0ZwN3fzraY2VBNX0SiKyf0RwJLS+aX5ZeV2hcYYWZ/M7Mnzeyr\nWRUwS6rpi0h0XTbvdGM7E4DjgKHAbDOb7e4vZ7T9TKimLyLRlRP6zcCokvk988tKLQPedvePgI/M\n7P+AQ4DNQr+xsfHj6VwuRy6X616Je0E1fRGpB01NTTQ1NfXJts27SEIzawBeInXkrgCeAM5x9wUl\n64wDfgpMBrYG5gBfcvcX223Lu9pfXzr3XLjjDoW/iNQXM8PdLYttdVnTd/c2M7sImEXqA7jF3ReY\n2ZT0tk9394Vm9mdgHtAGTG8f+LVAYS8i0XVZ0890Z1Wu6X/pS3D33alt3zI5Z4qI9L0sa/qh7shV\nR66IRBcy9NXMIyJRhQx9EZGoQoa+avoiElXI0BcRiSpk6KumLyJRhQx9EZGoQoa+avoiElXI0BcR\niUqhLyISSMjQV/OOiEQVMvRFRKIKGfqq6YtIVCFDX0QkqpChr5q+iEQVMvRFRKIKGfqq6YtIVCFD\nX0QkqpChr5q+iEQVMvRFRKIKGfqq6YtIVCFDX0QkqlChX6jhq6YvIlGFDH0RkahChr7CX0SiChn6\nIiJRhQp9jd4RkehChb7CXkSiq3joVzN41aYvItGppi8iEohq+iIigYQMfRGRqEI172j0johEp5q+\niEggIUNf4S8iUYUMfRGRqMoKfTObbGYLzWyRmU3dwnqHm1mLmf1bdkXMjmr6IhJdl6FvZgOAacBJ\nwIHAOWY2rpP1fgz8eUvbU01fRKR6yqnpTwQWu/sSd28B7gRO72C9bwP3Am9uaWPVDF6N3hGR6MoJ\n/ZHA0pL5ZfllHzOzPYAz3P1/AMuueNlS2ItIdFl15N4IlLb1dxr81fzJQrXpi0h0A8tYpxkYVTK/\nZ35ZqU8Bd5qZATsBJ5tZi7vPbL+xa65pZOut03QulyOXy/Wg2D2jsBeRetDU1ERTU1OfbNu8iyQ0\nswbgJeB4YAXwBHCOuy/oZP1bgQfc/f4O3vNVq5zhw3td7h7ZcUd491144w3YZZfqlEFEpLvMDHfP\npOm8y5q+u7eZ2UXALFJz0C3uvsDMpqS3fXr7j2xpe62tPS5rr7mD1WyPg4hI3+uypp/pzsx8+XJn\n990rtstNDB8Oa9fCsmWw667VKYOISHdlWdOv+B251azpb9wIA0I9Yk5EZFOhQt89hb46dEUkqnCh\nrzZ9EYksXOirpi8ikYULfdX0RSSycKGvmr6IRBYq9DV6R0SiCxX6qumLSHThQl9t+iISWbjQV01f\nRCILF/qq6YtIZOFCXzV9EYksVOhr9I6IRBcq9EE1fRGJLUzoF4JebfoiElm40G8/LSISScVDv6Wl\n0ntMCiN3VNMXkcgqHvqrV1d6j0lh5E5hWkQkooqH/vLlld5jsnGjavoiIhUP/ebmSu8xKb0xSzV9\nEYmq4qH/z39WJ3TVpi8iUoXQHzwYmpoqvVfV9EVEoAqhP3UqXHZZ5YduqqYvIlKF0P/KV2CnneD7\n36/sfjV6R0SkCqFvBr//Pdx9N/zyl5Xbr0bviIjAwGrsdOed4cEHIZeDYcPgrLP6fp9q0xcRqVLo\nA+y7Lzz8MHzuczBkCJx6at/uT236IiJVaN4pNX48PPAAXHghPPpo3+5LNX0RkSqHPsDEiXDPPXDu\nufD66323H9X0RURqIPQBjjkmDeM8//y+q4Vr9I6ISI2EPqTQX7UK7ruvb7av0TsiIjUU+g0NcMMN\n8L3vpYDOmtr0RURqKPQBjjsOttsuDefMmtr0RURqLPTN4JJLYNq07Letmr6ISI2FPsAXvgCzZ8Pb\nb2e73dLQFxGJquZCf8gQOPFEmDEj2+0WRu8o+EUksrJC38wmm9lCM1tkZlM7eP9cM3su//qHmR3c\nm0KddVb2o3gKo3dAzTsiEleXoW9mA4BpwEnAgcA5Zjau3WqvAp9190OAHwK/6E2hTjgBHnss2x9R\nV0euiEh5Nf2JwGJ3X+LuLcCdwOmlK7j74+7+Xn72cWBkbwo1YgTssw889VRvtrIpdeSKiJQX+iOB\npSXzy9hyqP878FBvCgXpCZxZ/sKWavoiIhk/ZdPMjgUuBI7ubJ3GxsaPp3O5HLlcrsP1jjkGbr4Z\nrrgim7Kppi8i9aKpqYmmPvpdWfMuEtDMJgGN7j45P3854O5+bbv1xgP3AZPd/ZVOtuVd7a9gxQo4\n+GB4661sauf/+le6ehg6NP2Ay4EH9n6bIiKVYGa4eybtFOU07zwJjDWz0WY2CDgbmNmuQKNIgf/V\nzgK/u3bfHbbaKrsnb2r0johIGc077t5mZhcBs0gniVvcfYGZTUlv+3TgKmAE8N9mZkCLu0/sbeEm\nTIC5c2H06N5uSW36IiJQZpu+uz8M7Ndu2c9Lpr8OfD3bosFhh6XQP/PM3m9LbfoiIjV4R26pCRPg\n6aez2ZZq+iIiNR76hxwC8+Zls61C6A8cmO1NXyIi9aSmQ3/0aFizJv24Sm8Vnr0zdCh88EHvtyci\nUo9qOvQHDICDDoLnn+/9tgqjdxT6IhJZTYc+pLH6WYR+oXlHoS8ikdV86I8fn03ot7am9nyFvohE\nVvOhf/DB2XTmtrSkm70U+iISWV2E/vz5vf+x9ELob7utQl9E4qr50B8+HIYNgyVLered1tZiTX/t\n2mzKJiJSb2o+9CGbJp5CTX+XXWDlymzKJSJSb+om9HvbmdvSkjpyR43K7iFuIiL1pi5CP4sRPIWa\n/qhRsHRp1+uLiPRHdRH6WTTvFNr099pLNX0RiasuQn/cuPQjKB991PNtFGr6w4enE8CaNZkVT0Sk\nbtRF6A8aBGPHwoIFPd9GIfTN1MQjInHVRehD75t4Ch25oM5cEYmrrkK/N525hTZ9ULu+iMRVN6E/\nfjw891zPP19o3gHV9EUkrroJ/UmTYM4c2LChZ59vH/pq0xeRiOom9HfcEfbbD2bP7tnnS0NfzTsi\nElXdhD7AKafAjBk9+2zh0coAe+8Nr72WWbFEROpGXYX+l78Md9zRs9+4La3pjx4Nb73V/x+8Nndu\n+uUxPVVURArqKvT33RcOOAB+97vuf3b9+jTeH6ChITUVLVyYbflqzfHHp8dSn3VWtUsiIrWirkIf\n4Jpr4KqrUk29O1avTnfjFhxwALz4YrZlqyVtbek7Azz0UDrpiYjUXegfdVRq5jn77O41W7zzDowY\nUZw/6CB49tnsy1crXnhh0/lnnqlOOUSkttRd6AP86EdpBM5nPpParcuxatWmoX/ssfDII31Tvlpw\nySUweDCsWJHmP/3p6pZHRGpDXYb+wIFw663w7W+nET2nnAL33ltszuhI+9D/1Kdg2bL06o/efz+d\n1HbbDS6/PC37+9+rWyYRqT5z98rtzMyz3t+HH6bAv/12eOyx1Nk7fnx6bMP++6fhmTvvnJbPnw+7\n71787KWXphPFr36VHsTWXzz2GHzxi7BoUfp5SCh+vxdfTMelL9x/P9x8c7oSO/zwvtmHSERmhrtn\nklJ1H/ql1q9PzT3PP5/atBcsSDdhLV0KZ5yRTgylVq+GE09Mj1k+5JB0A9iOO6bf5N122/TabrvO\npwtDQGvNBReku45/8IPislWr0ncD+MY34NprYYcdstlfWxscfTQ8/nhx2Y03wsUXZ7N9kegU+hlq\na0sduosWpc7ed95JTSNr1xb/lk6X/m1ogCFD0mvw4J5Nl7PuoEHlX4nMmAHf/Gaq0bcP9fXr4Tvf\ngZ/+NM0fdli6IjjttDSEtSdXO2vWwB57dNyp/sAD8PnPd/15gO237/6+RaJQ6NcA9xSiH36YXuvW\n9Wy6nHVbW7s+QbjDyy+n9e+6C444ovOyt7TAo4+mE8ScOcVRTEcemZrG9t8fPvlJ+MQnUvPY1ltv\nvo033oDrroPrr0/z558Pv/hFuvpZvDg1p0Fa57LLNv/8a6+l7RccdxzMmpVOpFs65h9+mPp0OiqT\nSH+l0A+mtbXrk4RZGtF06KFbDs6OtLWlG9Xmzk1XCIsWwSuvpGBesyadVIYPT39bWjZ9btGkSeku\n6b333nSbGzakJrOFC9OJYNq01ATU3Jzus5gzJ623zz5pXwUvvAAHHrh5+a67rtghDWn01YwZukKQ\nGBT6UjEffQTLl8Obb6YmrdbW1KcxalR6ddUk9MgjcN55xaGjBVddBVdeWayxz56drjQgdbafdlo6\nif3hD/D008XPDR8O775bnL/4Ymhs3LQpa+1aePhhuO02+NOf0q+uTZkCX/vapiO4ROqFQl/qTltb\navcfODA1SXW2zj33wNSpm15NHHMM3HRTGpFVOMksXJg64UsfkT1gAGzc2HVZrrwSzjwzXZ0MHpxO\nIs3NqeN//vx0ottppzSsd+LENC1STQp9kbw1a1KH8UMPpWaiMWPgs5+FyZPTg/UKJ4mlS1Ofw003\nwXvvdX8/e++d7uIeMyYNAd5mm7Tt1tbU5FV4CODgwelqYswYGDcuXbXU6igvqR8VD30zmwzcSLqZ\n6xZ3v7aDdW4CTgY+AC5w980ecqDQl1qxahW8/XbqD9l++9RsNGxYulqAdMWwbl0a/vuXv6TfZ25u\nTsN816xJId/WltYvfKbQuf/++5tfceyyS9rHiBHpSmfo0DQqa9Cg1AczaFCxg3rQoHTyKO2sHzo0\nvQrLttmmOLJrm23SZwt/t966WK7+dP9JZBUNfTMbACwCjgeWA08CZ7v7wpJ1TgYucvdTzewI4Cfu\nPqmDbSn085qamsjlctUuRk3oz8figw9Sf0Zzc7rCeP/91Hy0bl26StiwIZ0gNmxIJ5HFi5vYY48c\n69cX1yt9FZYV3t+wIU23thb/tramfQ8YkE4opSeB7bdP+xk6NJ2khgxJfwcPTp8pjAQr/C30uRRO\nKIWTyFZbFX+foqEhzZul18CBm65bKEdDQ3GdQnkaGoonp9LXgAHwyitN7LdfbrP3Stfp7L2OXtD5\nfEfT7f9u6b2CcqZ78pkTTsgu9AeWsc5EYLG7L0kFsTuB04HSBxOfDvwGwN3nmNkwM9vV3VdmUcj+\nqD8HXXf152MxdGjqSB47trz1GxubaGzM9WqfGzemwC6cSDZsSPNm6SqloSGdjMyKI7/WrUufXbeu\nOG+WTiyFv4WTiXuxWcusuI/Cvtvais1d7sVlbW3Fsm3cWFxeWFaoDxbmH3+8iba23Cbvl746W97R\nq1CWjuY7mm7/d0vvFZQz3ZPPZF1PLif0RwKlvyi7jHQi2NI6zfllCn2RCis0NxVq76XqafRSY2N6\nSbbNdHX5wDUREemZctr0JwGN7j45P3854KWduWZ2M/A3d78rP78QOKZ9846ZqUFfRKQHKtmm/yQw\n1sxGAyuAs4Fz2q0zE/gWcFf+JLG6o/b8rAotIiI902Xou3ubmV0EzKI4ZHOBmU1Jb/t0d3/QzE4x\ns5dJQzYv7Ntii4hIT1T05iwREamuinXkmtlkM1toZovMbGql9lsNZranmf3VzOab2fNm9h/55cPN\nbJaZvWRmfzazYSWfucLMFpvZAjM7sXql7xtmNsDM5prZzPx8yGORH858T/67zTezIwIfi0vN7AUz\nm2dmvzezQVGOhZndYmYrzWxeybJuf3czm5A/fovM7Maydu7uff4inVxeBkYDWwHPAuMqse9qvIDd\ngEPz09sCLwHjgGuB7+aXTwV+nJ8+AHiG1Ny2d/5YWbW/R8bH5FLgd8DM/HzIYwHcBlyYnx4IDIt4\nLIA9gFeBQfn5u4DzoxwL4GjgUGBeybJuf3dgDnB4fvpB4KSu9l2pmv7HN3i5ewtQuMGrX3L3Nzz/\nGAp3XwssAPYkfedf51f7NXBGfvo04E53b3X3fwGL2fxeiLplZnsCpwC/LFkc7liY2fbAZ9z9VoD8\nd3yPgMcirwEYamYDgcGk+3tCHAt3/wfwbrvF3fruZrYbsJ27P5lf7zcln+lUpUK/oxu8RlZo31Vl\nZnuTzuiPAx/fpezubwC75Ffr7Oa2/uK/gO8ApR1IEY/FGOBtM7s139Q13cyGEPBYuPty4HrgddL3\nes/dHyHgsSixSze/+0hSlhaUlau6OasPmdm2wL3Axfkaf/te837fi25mpwIr81c+Wxqy2++PBeny\nfALwM3efQBrpdjkx/1/sQKrZjiY19Qw1sy8T8FhsQZ9890qFfjMwqmR+z/yyfit/yXov8Ft3/2N+\n8Uoz2zX//m7Am/nlzcBeJR/vT8fnKOA0M3sVuAM4zsx+C7wR8FgsA5a6+1P5+ftIJ4GI/y9OAF51\n91Xu3gb8ATiSmMeioLvfvUfHpFKh//ENXmY2iHSD18wK7btafgW86O4/KVk2E7ggP30+8MeS5Wfn\nRy+MAcYCT1SqoH3J3b/n7qPc/ROkf/e/uvtXgQeIdyxWAkvNLP8LwhwPzCfg/wtSs84kM9vGzIx0\nLF4k1rEwNr367dZ3zzcBvWdmE/PH8LySz3Sugr3Vk0mjWBYDl1e797yPv+tRQBtplNIzwNz89x8B\nPJI/DrOAHUo+cwWpV34BcGK1v0MfHZdjKI7eCXksgENIlaBngftJo3eiHour899rHqnjcqsoxwK4\nnfSo+vWkE+CFwPDufnfgMOD5fK7+pJx96+YsEZFA1JErIhKIQl9EJBCFvohIIAp9EZFAFPoiIoEo\n9EVEAlHoi4gEotAXEQnk/wE+jNXVqyS63wAAAABJRU5ErkJggg==\n", 620 | "text/plain": [ 621 | "" 622 | ] 623 | }, 624 | "metadata": {}, 625 | "output_type": "display_data" 626 | } 627 | ], 628 | "source": [ 629 | "layers, error_history = trainAE(linear, .4, 1000)\n", 630 | "\n", 631 | "# Plot error history\n", 632 | "plt.plot(range(1000), error_history)\n", 633 | "plt.show()" 634 | ] 635 | }, 636 | { 637 | "cell_type": "markdown", 638 | "metadata": {}, 639 | "source": [ 640 | "Let's look at the encoded coordinates:" 641 | ] 642 | }, 643 | { 644 | "cell_type": "code", 645 | "execution_count": 194, 646 | "metadata": { 647 | "collapsed": false 648 | }, 649 | "outputs": [ 650 | { 651 | "data": { 652 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYEAAAEACAYAAABVtcpZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xdc1dX/wPHX517gci/IkiUoDsCBM/dKcc8cubU0zZ/l\nKM32MM0c2bAyM1OzUvumkiMVd4p7z1woDgQUlD0ucNf5/XGJNCVRrqJxno8Hjwf33vM5n/cHvfd9\nP2cqQggkSZKkkklV3AFIkiRJxUcmAUmSpBJMJgFJkqQSTCYBSZKkEkwmAUmSpBJMJgFJkqQSrMhJ\nQFGUsoqibFMU5bSiKH8qivJqAeVmKYpyQVGU44qi1CnqeSVJkqSis7NBHSZgvBDiuKIozsARRVE2\nCyHO/VVAUZROQKAQIlhRlEbAXKCxDc4tSZIkFUGR7wSEEPFCiON5v2cCZwH/fxTrDizKK3MAcFUU\nxaeo55YkSZKKxqZ9AoqiVADqAAf+8ZI/EHPL4zjuTBSSJEnSI2azJJDXFPQbMDbvjkCSJEl6zNmi\nTwBFUeywJoDFQojf71IkDih3y+Oyec/drS65mJEkSdJ9EkIoD3Kcre4EFgJnhBBfF/D6GmAwgKIo\njYFUIURCQZUJIf6TPxMnTiz2GOT1yeuT1/ff+ymKIt8JKIrSDBgE/KkoyjFAAO8B5a2f52KeEGK9\noiidFUWJArKAoUU9ryRJklR0RU4CQog9gLoQ5cYU9VySJEmSbckZw49QaGhocYfwUMnre7LJ6yuZ\nlKK2J9maoijicYtJkiTpcaYoCqKYO4YlSZKkJ5BMApIkSSWYTAKSJEklmEwCkiRJJZhMApIkSSWY\nTAKSJEklmEwCkiRJJZhMApIkSSWYTAKSJEklmEwCkiRJJZhMApIkSSWYTAKSJEklmEwCkiRJJZhM\nApIkSSWYTAKSJEklmEwCkiRJJZhMApIkSSWYTAKSJEklmEwCkiRJJZhMApIkSSWYTAKSJEklmEwC\nkiRJJZhMApIkSSWYTAKSJEklmEwCkiRJJZhMApIkSSWYTAKSJEklmEwCkiRJJZhMApIkSSWYTAKS\nJEklmEwCkiRJJZhMApIkSSWYTAKSJEklmEwCkiRJJZhNkoCiKD8oipKgKMrJAl5vqShKqqIoR/N+\nPrDFeSVJkqSisbNRPT8C3wCL/qXMTiFENxudT5IkSbIBm9wJCCF2Ayn3KKbY4lySJEmS7TzKPoEm\niqIcVxQlXFGUkEd4XkmSJKkAtmoOupcjQIAQQq8oSidgNVC5oMKTJk3K/z00NJTQ0NCHHZ8kSdIT\nIyIigoiICJvUpQghbFORopQH1gohahWi7GWgnhAi+S6vCVvFJEmSVBIoioIQ4oGa3G3ZHKRQQLu/\noig+t/zeEGvyuSMBSJL0cAgh+GjiRPw8PfH38uKzGTMo6MvWzp07+fLLL1m9enWBZaT/Dps0BymK\n8j8gFCitKMpVYCLgAAghxDygt6IoIwEjkA30s8V5JUkqnFlffcXCzz/nWb0eC/DV5MmU9vJi2LBh\nt5WbMX06X0yZQrDJRKy9PWFdurBk6VIURY7r+K+yWXOQrcjmIEmyvZaNG+N/4ABV8h7/CeS2a8fa\nzZvzy6Snp+Pj6ckooxEXrN/YFjg5sWbbNho2bFgMUUuF9bg0B0mS9Jhyc3cn9ZbHaSoV7qVL31Ym\nNTUVrb09LnmP7YHSdnYkJiY+qjClYiDvBCSpBDhx4gShzZtTLTsbi6IQpdOx99AhKlf+e5Ce2Wym\nSqVKBMXEUF8ILgKbSpXi7IUL+Pj4FFy5VOyKcicgk4AklRBRUVEsW7YMlUrFwIEDKV++/B1lLl68\nSN8ePTh59izlfH1ZvGwZzZo1K4Zopfshk4AkSVIJJvsEJEmSpAcik4AkSVIJJpOAJElSCSaTgCRJ\nUgkmk4AkSVIJJpOAJElSCSaTgCRJUgn2qPYTkCTpCWOxWNi0aRPXrl2jUaNG1KhRo7hDkh4COVlM\nkqQ7WCwW+vbsycFt2/AVggsWC3MWLGDAwIHFHZp0F3LGsCRJNrVp0yb+r3dvhmRmYgfEA0u0WtKz\nsuSy0o8hOWNYkiSbio+Px5u/24t9gFyDgezs7GKMSnoYZBKQJOkOjRo1IspiIQ6wAHtVKqoGB6PT\n6Yo7NMnGZBKQJOkOVatW5YdFi1ju5MRUlYobVaqwZsOG4g5Leghkn4AkSQUSQmAwGNBoNMUdivQv\nZMewJElSCSY7hiVJkqQHIpOAJElSCSaTgCRJUgkmk4AkSVIJJpOAJElSCSaTgCRJUgkmk4AkSVIJ\nJpPAXURHRzNu3HiGDRvB5s2bizscSZKkh0ZOFvuHmJgYatWqR3p6FSwWZ3S6g8yf/w0D5RK6kiQ9\npuRkMRuaP38BGRlBWCxtgcbo9c/w/vuTizssSZKkh0ImgX/Izs7BbL51nRQtBkNuscUjSZL0MMkk\n8A/9+vVBpzsGnAFi0Ok2MWTIc8UdliRJ0kMh+wTuYsuWLbz11gQyMjJ57rm+TJjwPmq1ulhjkiRJ\nKohcRVSSJJtISEggMjKSgIAAKlSoUOT6rly5wpkzZ6hYsSLVqlUreoDSXcmOYUmSimzNmjVUqVSJ\nF7t1o05ICF989lmR6luyZAl1QkJ4c+BAmtarx/QpU2wUqWRL8k5AkiRycnLwKV2avno9ZYE04Eet\nln1Hj1K1atX7ri8jIwM/b28G5+TgDWQAP2i17D92jCpVqtg4eqnY7wQURflBUZQERVFO/kuZWYqi\nXFAU5biiKHVscV5Jku4khGDevHkM6NWLN157jcTExHseEx8fjz1QNu+xK+Dv4MDFixfv+/znzp3j\nldGjUZtMeOc9VwrwdXAgOjr6vuuTHi5bNQf9CHQo6EVFUToBgUKIYOAlYK6NzitJ0i0iIyPp2qkT\nE0eNQr9yJbtmz6ZR3bpkZGQUeMzWrVsZ0r8/mTk5XMp7LhGINRju+y7g9OnTNG3QgEuLF5NrMnEh\n7/nrwDWDgZCQkAe5LOkhsrNFJUKI3YqilP+XIt2BRXllDyiK4qooio8QIsEW55ekkk4IwaujRvHz\nwoVkGQyMB5wBTCZ+vXmTdevWMWDAgDuO27dvH326d6eVXk954FfAWa3GYGfH17NmERgYeF9xfPnZ\nZ9TNyuJpIAhYCqhUKlQaDQt//pmyZcveowbpUXtUHcP+QMwtj+PynpMk6T5FRkbSsnFjyvn40K1j\nRxISEti8eTMrFy8m0GAAwOHWAwwGjEbjXev6acEC6uv1XACMQBvAyWymdq1aDB027L5j02dloc3r\n0ysP9ASCqlQhISmJ3n363Hd90sMnRwdJ0hMkLS2N0GbNcDp4kB43bpCybRsdWre2Dus0m1EAP+A3\nIBrYB1wB2rdvf9f6HDQasoBLwECgMTAUOHP0KMePH7/v+J4fNox9Oh1ReeffodPxf6NGodVq7/9i\npUfCJs1BhRAHlLvlcdm85+5q0qRJ+b+HhoYSGhpq84BMJhOffPIpGzf+Qbly/syYMYWAgACbn0eS\nbOngwYO4GI00yvu23cZoZNbly/j6+nJJpaIlcB5rR2w4oFepeOeDD/D19b1rfSPHjKH+ggWojcb8\nDwM1oDKb0ev19x1fp06dmP3DD8z46CNMJhNvjBrFqNGjH+BKpX8TERFBRESEbSoTQtjkB6gA/FnA\na52B8LzfGwP7/6Ue8SgMGzZC6HRBAvoLtTpUeHmVEYmJiY/k3JL0oPbs2SP8nJ3FBBCTQLwNQufg\nIBITE8XEDz4QThqNcNdqhbNaLaoHBopvZ88WFosl//iIiAjxxRdfiJUrV+Y/P3XqVOEIogGI/wPR\nAoQDiKysrOK6TOk+5X1uPthn94MeKG7/4P4fcA3IBa5ivaN8CRhxS5nZQBRwAqj7L3U9vL9UHpPJ\nJNRqewFvC5gkYJJwcqolFi1a9NDPLUlFYTKZRNsWLURVrVa0BVHeyUmMGjEi//X4+Hhx+vRpkZ2d\nfcex06dOFV46nWjq4CACnJzEgN69hcViEZmZmSKoQgXhpSjCDYROpRLPDRwoDAbDo7w0qQiKkgRK\n5GQxs9mMRqPFbB4PWNsqnZxWMnfumzz3nFwsTnq8GQwG5s+fz/mzZ2nQuDGDBg1CUe6cJ5SYmMiw\n555j7759+Hh7c/HKFUabTLhg7QRe4OTE73/8QaNGjUhLS2Pa1KnMnzMHZ7MZi0qFd6VK7Ny3D2dn\n5yLHfPjwYY4cOUL58uXp0KHDXeOVHlxRJos9qj6Bx4pareaFF4bx668r0OvroVYnoNMl0rlz59vK\nWSwWFEWR/2Glx4qDgwOjC2hnT0lJYfSIERzcv5+0tDSCsrMZbDJxPj2dS4BLXjl7oLSdHUlJSQC4\nurpyJSqKGtnZtLFYsAC/nTrFmJEj+Wnx4iLFO2f2bCa8/TZBQJyi0LZ7d35cskS+rx4TJXZ00Pff\nf8v77w/n6aeT6N+/PEeO7MfDwwOAzMxMunZ9FgcHR5yd3Zg165tijlZ6HAkh+Obbb+jYvQPPDX2O\nqKioh3KOwt4ZCyHo2qEDl9asITQ2lpSMDNqbTLgC9QC1orBXUTAA54DrZjP16tXLP/7UiRMEWiyA\n9YOhMrBy2bJCn/9usrOzeeP113lOr6ezXs+QrCw2/v47Bw4ceOA6JdsqsUlArVbz3ntvs3PnFpYs\n+Yly5f4evDRixGi2br2M2fwGev1g3n33YzZs2FCM0UqPowmTJvD5ws8oNdiZ68FxNHm6CXFxBQ56\nuy9Go5ERw4ah1Whw1mp5/5137vlhfOPGDU6ePEkHgwE/QMG6Zs9f3LVaLgcE8JlazYGyZVm3aRM+\nPj75r5cpW5YjgAVrc9FJINtkIj09/YGvIzU1FQeVCo+8xw6At50dCQlynujjokQ2B93L1q1/kJvb\nE9AAGvT6mmzduo1OnToVd2jSY+TbOd/y3MGBuFd0AyAtKp0VK1bw6quvFrnuyRMnsnPpUsYajRiB\nxd98Q/lKlRgxYsRt5SIjIwkLC8Pe3p5u3bphsljIxdrT1QpYANQFbuh0lK9Rg+27d2Nvb3/Xc74/\naRKd27ThMyGwYJ1v4ObqiouLy13LF4aPjw8enp4cjIujvhBEA1dNptvuQKTiJZPAXXh6enLzZgLg\nCQg0miR8fb3vdZhUwgghuLVZW1EpRWo6udWm8HAaZWejy3tcT69nS3j4bUng8OHDtAsNJSQnB7NK\nxefTp9OrVy+WrVlDVb2eeEdH/P39ada3L+UCAujVqxenTp2iTJkyxMTEkJWVRb169ShVqhQAG9at\nw0Gtxs1kIh64odGwYc2aIrXdq1QqNv7xB8927crGqCi83N0JW7pULh/xGClxo4PS09NxdHTEwcGh\nwDK7d++mY8euWCxVUKky8PNTOHJkf/6bRfpvMZvNLFy4kFNnT1GjWg2GDRt2153kTp48ychxI4mL\njaVJk6b4ePmweudqGr3fgKRzyRz/8gTHDx+/4wMuMTGRjIwMAgICCr1D3TMdOmDasoXGee+FrXZ2\n1B4+nNnffZdfpmOrVthHRFA/73GESkXQ4ME0bdGC/bt3Uyk4mFfHjkWr1bJ9+3Z6de+OE5CYmYmT\nvT3ujo7oNRoi9uwhOzub1k2aMFyvR4t1AbmFGg03k5PR6XSkpqaiUqmKdFdgMpmws7PN987IyEhe\nGDiQC1FRVA8J4edff7XJJjhPqqKMDrLZZDFb/fCQ5gncuHFD1KvXWNjZaYSdnYOYPHnKv5a/cOGC\nmDNnjli0aJHIzMx8KDFJxc9isYi+g/qKwKcDRZtPW4vApwNFn4F9bptgJYR1/H1p39Ki6/zO4uUz\nI0T94fVEaLtQMfPrmaJ159ai73N9RWRk5B11vzr+VaFz0QkPfw8RUidExMXFFSqu06dPi9IuLqKe\nVitq63TC39tbXLt27bYyjerUEc/nTRqbBKIHiN7dut1Rl8FgEB4uLmIwiC4gKkL+ZLNOKpVo2bix\nWLdunaju6ppf1yQQpXU6ce7cOdG9c2fhaG8vNPb2YmDfvsJoNN7nX9m2MjIyRBkvL9FZUcRrINqq\nVKJSuXIiNze3WOMqThRhnkCJaQ4aNGgoJ0+qMZneBjKZMWM29eo9dcew0L8EBQURFBT0aIOUHrmo\nqCi2bNvCSxeHY6+1p8GYeswLWsD58+dv2/xkx44d+DUqw1PDrVthdPiuHZ+7fsmaFWt47dXX7lp3\nWFgYK/9YyagrL+Ho5sjOD3cz9KWhbFq76Z5xhYSEcOLMGcLDw7Gzs6NHjx75o9f+0qtfP+acP4+L\nXo8J2K/TMbNfvzvqunHjBhajkUrABSAQ69IQAEEWC8sjI6lZsyaxRiOxWNd0OQXY63TMnzuXi9u3\n84bRiAVYsXYtn06fznsTJtzzGh6WP//8E8fcXBrm3SU1t1g4nZLChQsXqF69erHF9aQqMaODDhzY\nj9HYEOslu5CVVYU9e/barH6z2czixYuZNGkSa9assVm90sOVlZWFzl2HvdbaWWqvtUfn4URWVtZt\n5XQ6HZnxWflt/tnJ2QiLQKPRFFj3kWNHCOpdCa27FkVRqD28FsePFX5RNn9/f0aMGMGwYcPuSAAA\nr7/1FoNefZUVpUsT7u3NO1OnMmDgwDvKeXl5IdRqogFf4AyQAwjgCJCZlsb7b7/NT7/8wnKdjs80\nGvZ6eRG+aRP7d+2iTnY2dlhH9tTKzmbPjh2FvoaHwcXFhXSTib/WRc0BskymIjVVlWQlJgmUKXPr\natYWdLoEAgLK/dshd5Wbm8v//d8ovLz8qVSpKmvXrkUIQffuvRk5chKTJ29nwICX6dLlGaZMmcKm\nTff+1icVn2rVquFocWTPlL0kRiaxZ+peHIwOt21+smrVKsJWh5F7I5dVvdaw7/P9LGvzG+PfGJ/f\nt5SQkECn7p3wLONJnUZ1OHLkCEGVgoj94xpmgxmASxsvUTGwYqHiCgsLo23z5nQIDWXz5s13LaNS\nqZgyfTrXExOJSUhg7Lhxdy3n4ODAst9+Y5WTE8ddXEhRqfgC+ALr6qEvWizsWrsWk8lEUloal2Ni\niImPp27dulQMCiImrx1fADH29lQMDi7UNTwsISEhtOvUif85ObEd+J+TEwMGDrxtmLd0Hx60Helh\n/fCQ+gQOHDggSpVyF6VK1RbOzuVF48ZPP1Ab4rBhLwmtNkTAKwKeFzqdm/jxxx+Fk5OvgA8EfCgg\nWICvUKmeFk5OPmLSpI8fwhVJthIdHS3ad20vygWWE+27thfR0dH5r339zdfCO9BbtP+6rajUrqLQ\naDWiW49uYunSpfn9BhaLRdRrUk80e6OJGBszRvRY0k14+HiI2NhY0aVHZ+Eb7CsqtwgWPmV9xOnT\np+8Zz7Jly0RpnU70AfEsCDetVmzbtu22MkajUWzevFn89ttv4vr164W6zsTERLF3715x9epVoXVw\nECNAfJjX/t9crRbTpk2745i4uDhR3s9PVClVSgSXKiWqVKz4WCy0aDabxaJFi8R77757279FSYVc\nO6hwrl27xu7du3FxcaFt27YPNFLBw8OHlJQBgDsAKtU2Bg4MYM2aw6Sn98e6ivpaYCTWltcM7O2/\nJSUlEScnJ9tdjPRI+JT1oXt4VyLe30l2Sg7eNb24sDKKmZ/MZPiw4YB19E+FoAq8lvJq/nDKVc+s\nYfKwyXTv3p3Dhw+Tnp5O/fr1cXNzu+McBoOB0eNGs/R/S3HQOOCpdeap6KtUy3v9EODcsydLV67M\nL98uNJToP//ERVG4pihs2rbtvsbeN6xTh9J//kljiwU9sMTJifnLl9+1jywjI4MdO3agUqkIDQ1F\np9PdWaFUrOTaQYXk5+dH3759i1SHTudMSkoafyUBe/tMKleujKJsBI5jvWkuxd9db86o1Q5kZmbK\nJPAEys3JJf74DTLjMxm2/wVUdioSxyUytsFYhr0wDJVKhZOTEyaDiayELJx9nbGYLaRdTcXFxQWV\nSkXDhg3/9RzvT3yfXRd38n+Rw8hJyWHxUz9S65bXLUBaejqtmjUjOTER34AAbpw4wRC9HhXWZXlH\nDBnCkVOnCn1dv65YQduWLTmWlkam0cjol18ucJBEqVKl6Nq1a6Hrlp4sJepOwBaWL1/OCy+8TE5O\nbeztM/D0TOTPP49y9epVBg58gejoS+TkGLBYngEqoFYfJigokbNnT8oFs55Ao14dxcqtK/GsXZpn\nf+0BgLAIZmg/Jy01LX/HrHHjx7Fo1SJqPB9C7I44KjhWZMv6LYWaF1C9XnUaz2mAfyPrjqvrR27g\n3PyTtDGbMQG7HR0RikKr7Gy8gNV2dtQymWiZd3wKsNTdnfjk5Pu6NqPRyKVLl3B3d8fbW06GfJLJ\nO4FHqG/fvvj5+bF27Trc3d0YMWIEHh4eeHh4cOaMdeTHwYMHGTRoKNeubaZOnbosX75RJoAn1Ndf\nfI3xVSOLliziyvYr+DX0Y+/UfTzV8Kn8BJCYmMjSsKWUC/Un+WIK2YnZlGlQptATwzxLe5J4Nik/\nCdjZ2dOpV08sWdlo7OwYEhDAwXnzqJtXvrnJxHasy0E4AYft7Wlwj7uNu7G3t79tGKxUMsk7AUkq\nhI0bNzJ85HBuXr9Jo+aNWLZ4GWXKlAHgl19+YcZvn9BjVTcADJkGZpb+mqzMrALX6bnVoUOHaN+5\nPZV7BZObmkvSwWSO7D+S/+38u+++Y97rr9MjOxuwzub90cEBo8WCWqWiVvXqrN20CS8vr4dz8dJj\nryh3AiVmiOijZLFYsOQtySv9N3Ts2JHYy7H8sfUP6tSsw6zZs7h69SpgXZHWbPj739tsNINCoe/+\nGjRowKF9hxhYbRAvh47kxOETtzXP9O3bl5ulSrHZzo7DwAqdjomTJ5OWkUFcfDz7jx6VCUB6YPJO\nwIYsFgtjx47n++/nIoRg8OAX+P77b222XopUvNasWcOQEUOoO64O2TdziPo1isP7j+Dq6krt+rXx\n7+aHTwNvTnxzko71O/Ht19/a7Nzx8fF8NmMGN65fp2vPnvS7y8xgqeQqyp2ATAKFZLFY2LhxI9ev\nX6dRo0bUqFHjjjIzZ37FhAmz0Ot7Awo63Sreeut5Jk4svin2ku081bgOVT+sQnBn63IiW1/fRgtN\nSz6Z9gkJCQl8NPUjYq/H0qp5K8a+MhaV6u432kIITCZToZqKHraUlBQmTZjAxchIGjZrxjvvvfev\niytKjyfZMfwQrV69moiInWzZspXo6GTAD4vlDcaMGUG1alVp0qQJVatWBWD9+i3o9fWxdteBXt+A\n9eu3yCTwH6HXZ+Pk/fcwX52PFv11PWBdN3/OrDn3rCMsLIwRw4aRnpXFUzVqsCo8/L5muq5bt47f\nfv0VV3d3xr/5JuXLl7+va0hISGDSBx8QGx1Ns9BQlvz8M05XrhBgMLB0zx5OHjvGb7//fl91Sk+4\nB51l9rB+eEgzhh/EpEmThU5XRkAbAUECyubNCn5ZgJ3Q6eoJrdZVrF69WgghxODBw4Ra3ULAJAGT\nhErVRvTs2beYr0KylYmTJ4qKjSuIFw8NFQPW9xPuvu5i586dhTp2z5494sMPPxROt8zUba1Wi3o1\naxb6/AsWLBBeOp3oDOJplUp4ubmJmJiYQh+flpYmyvv5iWZ2dqI3CD+NRpSxtxcT82YNvw/CycFB\n3Lhxo9B1So8H5Ixh2zOZTGi1TphMY7Buzy2AH/J+j8G6bFVroCyurqtJTU0kNjaWunUbodd7Aioc\nHK5x+PA+KlWqVGzXIdmO2Wzm42kf82vY/zCZzDSp24SePXvy7LPP/msn8JSPPmLWp59S1mAgymSi\nHNaVPH2BhSoV+uzsQjXBBJYtS6u4OP66b1hvZ8czEyfywQcfFCr+sLAwJg0bRt/MTADOAtuBUX9d\nH/ClRkNUdPRt205Kjz/ZHPQQGI3GvDuTv27/FcAApANDgVwgDHAmPT0ZDw8f0tKSqFWrLoMHD8DN\nzY0uXbrISTj/IWq1mkkTJpGckszKLSu54n+J8VNfI3xzOAu/X3jXY27cuMEn06bxssGAM7ACiAXs\ngW2Ag51dofsGcg0Gbl2zVGM2Y8jNLXT8ZrOZW2cuBGCdaPaHvT3ljUb+dHSkWZMm8v9sCSPvBP7F\n00+35uDBLAyGRljfuhuA/kCFvBKHgcMoSgpCDAL8UKt3UbNmNseOHSieoCWb2rZtG2vXr8XNxY1R\nI0eRm5tLSO0QXr74fzi6OWLINDCv8g/s3bY3v2/oVqdPn6Zl3bq8YjAQDawBXsaaBBKB+XZ2ZOj1\nhUoE7739Nstmz6alXk8a8IdOx469e6ldu3ahriU5OZkaVapQLSUFP7OZI1otVdu1o5SzM1GRkTRq\n1owp06fLtYGeQPJO4CFZu3Ylw4a9xO7dKylTxg+oycmTKfydBJJxcspGiBro9QEAmM0tOXlyGrm5\nuf+61rz0+Fu8ZBFj3x7HU6/UJv1yBgsaLeCXn36hlHcpHN0cAXBwdsDN342UlJS71hEYGEgO1vV9\n1IAH1gQA1h2s7dVq0tLS8PT0vGc8H0+bhlan47dff8XFxYU1n39e6AQA4OHhwb7Dh3l7/HguX73K\ns23b8uFHH8nRQCWcvBO4DwcOHKBNm47k5ISgVhtwcrrK559PZ+zYyWRmDsb6Nk9Aq11MVla6XCri\nCWU0Gjly5AitOrai//q+lGtq3TM4/IUN9Knalznz51B9XAjVB1YjctV5Dk8+yvnT5+/YgzouLo6Z\nX89k85bNXDxzlmyDGXvgOaAccFBRuBgQQOTlyw/9/0pGRgYpKSn4+/sXejkL6ckhZww/Io0aNeLw\n4X189FFnpk7tx+nTxxkyZAgNGlTFyekXHB03otMtZe7cOTIBPIEOHz5MheAKaDQaWrZtiUqjwrmM\nc/7rWl8tuYZctm7YStKyJL4PXEDs/Di2bth6RwJISEigfpP67DPtxX9YGVQujtg72qNy0vCrnR1T\nFIXY4GDWb92KoijEx8czbdo0JnzwAceOHbvv2IUQfDZjBjUrV6ZhrVqsXbs2/7XPPvkEH09P6lSr\nRnCFCly4cOHB/0jSf468E7CB3Nxcnn9+CAcOHCUwsCILFnwnRwQ9YTIzM6lUuSLNv2iGo5uGdSPW\nU+3ZqiQzq4HzAAAgAElEQVSeS6LdF21IvZzG+qEb2bp+K1lZWej1epo2bYq7u/td65s5cyb/O/UL\nnRd2BOD60XjWdg9n5oyZXL9+nSpVquQvzxwXF0f92rUpl56O1mTipFbLb2vW0KZNm0LH/+n06Xw7\nZQpt9HqygY1aLb9vtC5c+GzHjjyv1+MKHFAUrlWtyvEzZ4r6J5MeI7JPoJi9/PIYwsOPodc3JDY2\nngYNmnLu3J9yPZcnSGRkJPZuDuyasgeT3kh2Ug45abl41/Qi7NkVZCdm88XUmYwYM4IrsZetexLr\nFfbu3EvwXbZbzMnJwcHt77Z2R3dHMpNTeWvECMobjVy2t2fg8OF8/tVXfPP111RMTaWD2boNpa9e\nz7vjx3PwxIlCx7/ohx9oq9cTkPc4KTubXxcvplrNmgSbzbjmPV9PCDZHRlrHh8u7VQnZHFRkZrOZ\nJUsWodf3BIKxWJ4mN7cM4eHhxR2adB+8vb1Jvp5MjQEhvHJ5NK9de5WYPbHERVwDvcKbr73F+fPn\nOXvuLNUGViO4dzAZ2Rk8N/S5u9bXo0cPziw+y8klp4jdH8ea/usQBhPPZ2XR3mBgcFYW87//npiY\nGFKTk3HJSwAArljb8IUQzP7mG1o2bkz3Tp04evRogfE7OjqSfcvjHEXBUaejYsWKxNrZYch7/iLg\n7+MjE4CUTyaBQggPD8fbuyz29hqaN2/NjRs38l/7+81066qhlkK9ycLDw6lSpRbe3mXp3XsAaWlp\ntg1cKrRy5cqhEipqv2Dd08vRzZEaA6vToEJDtq7dykcTPiJ8azgtPmxOu8/b0HZGa0Int+D8xfN3\nrS8kJIT1v68n6adkDo45TKsarfDWOeGY97oWcHVwICkpiZ59+nBYp+Mq1mGjETodPfv0YfqUKXz6\n7rv4HziAaeNG2rRoQWRk5F3P9+G0aWzU6dgLbFMUTjs7M/qVV+jatSuh3bqxwMmJ5a6ubHR2Zsmy\nZfnHGY1Gzp07x7Vr12z2t5SeMA861fhh/fAYLRshhBBnzpwRWq2rgBcEvCvs7JqJhg2b57+ekpIi\nateuL9RqPwF9hJ1dC+Ht7SeSkpIKrNNisYiBAwcLsBfQU8AY4eBQX7Rs2fZRXNJ/nslkEvHx8cJo\nNBb6GKPRKHTuOtFpTkcxQbwn3s1+S/jU8RGff/55fpnajWqLZ5f1EBPEe2KCeE/0WdVLlA32L1T9\n6enpwtvDQ/QA8R6IbiDKeHqKrKwsIYQQP//0k6jk7y/KeHqK8WPHCqPRKMr5+IiReUs6TALRTKUS\nEz/8sMBzbN++XYwYNkyMHTNGXLhwIf95i8UiDh06JNavXy/i4+Pzn798+bIICggQPs7OwlmjEaNf\neqnEb9j+pKIIy0YU+4f+HQE9Zklg3rx5QqdrkL8eEEwQKpVaGI1GkZKSIoKDqwt7+3oCagooJRRF\nI+bOnXtbHadOnRIzZswQ33zzjUhOThbLli0TGk1pASG31atW24vs7OxiutL/hl27dglPX0/h7OEs\ndC468corr4i0tLR/PcZkMomdO3cKZw9n4VLORZRt4i/cKrgKj/Lu+etCCSHEjz/9KDwqeYgXDw0V\nw48MEx6B7uLLr74sdGzHjx8X1QIDhYOdnahRubI4derUv5Yv7+srRtySBJqo1eKjjz4q9PnupUXj\nxqKtSiUmgXgHRDknJxEWFmaz+qVHpyhJQHYM30Pp0qVRqZKwNveogEQ0Gi3BwdW5cuUC4Ay0Bayz\nRYX4jHHj3qJFixZUq1aNiIgIunTpgcFQHXt7PdOnf06/fr3IzfUFMrCuSaQAWahUqsdieeEnVWZm\nJt17dafDT+0I6hRI9M6rfN/le8I3hXP04FFcXV3vOOb69eu07dyWhKQEcrJzCGodSIMx9VEUWDtg\n/W2rdA4ZPITU9FS+GvQVFouFd0a+y9hXxxY6vtq1a3MmKqrA10+dOsXw55/nSnQ0derUYdjLLzP3\n009poteTrihE6nT88vzz9/dH+Renz5xhSN7mR45ApawsTpw4Qe/evW12DunxJ/sE7qFbt27UqhWA\nk9OvODhsRqtdioODhitXfLC+dcoBp4D5WNcWUqFSVWL//v0AjBnzOnp9B0ym9mRn9+DmTW8uX76E\nTqfHumTXMmAXirKQCRM+lBN5iuDSpUtoPbUEdQoEoHyLALxqeKEEKMyfP/+uxwwfNRyPju6MjB7B\n+PixJJ1JYvv4CMKf20hQhSA6PtOR8sHlWbBwAYqiMO6VcVyJvMLVC1d58/U3bdbBmpKSQpsWLfA+\ncYL+KSlk79rFb0uX8smcORg7dKBMv37sOXiQihUr2uR8AIEVK3I+L34jEOPkROXKlW1Wv/RkkHcC\n92BnZ0dExGaWL19OQkIClStXpm/f54E4rHcA9fJKrgZ+BgJQqZLw9fUFIDU1BSidX5/R6IafX1k6\ndHBg48Y/ECIHiOWzz6YzZsyYR3lp/zllypQh7VoaqVdScavgRsb1TFKikqn5XE0SkxPR6/VMnjaZ\nk6dPUL1KDSZ+MJHDhw/T+1PrKqAaFw01B9ckbtE12nVox64Lu+i9vSfZyTm82+dd/Hz96Ny5s01i\nTUpKYuonU7kaF02Lpi0JDgzGw2KhXt4cmVYmE19fucLKFSs4sHcvGq2WLj163HV9ogf14//+R5sW\nLThrNJJmMtG6fXsGDRpks/qlJ4NNJospitIR+ArrncUPQogZ/3i9JfA7cCnvqZVCiCkF1CVsEdO9\npKamsmjRItLS0ujcuTP16tW790GAXq/H3d0Tg8EZ6AH4571yENgCWAgIqMilS+dQq9WMHPkKP/+8\njezslsBJ7O2PM3fuLIYOHcqpU6dIS0ujdu3ad8w4lR7M7O9m8/b7b+PbwIcbf96kWp+qXFxxibBF\nYXw842OSXZMI7hvExdWXcLzqSOy1OKqNrkLj1xphNpj5peOv5JzPobR3aRrPbYR/Qz8ADs46RLlz\n5Zk/5+53FPcjKyuLpxo+hXsLN8o09eXPuaeo7VeH/Rs2MzwrCzWgB2YqCqWEoD1YF4wDfl+/nk6d\nOhVYt16vx2Aw4ObmVqhY0tPTOXHiBK6urtSsWVMOHX1CFWWymC06clVAFFAe69pYx4Gq/yjTElhT\nyPps3mnyT8nJyaJcuUrC0bGOUKubC53OTaxbt67Qx3/77RyhVjsJCBbwroDXBLgLKC2ggtBovMX8\n+fOFEELk5OSIPn36C0VxEFBN2Nk1EE5ObuLAgQMP6/JKpGvXrom1a9eKAwcOiIMHD4rmLZsLp1I6\n4VvOV/z484/i3LlzwrNcafG+8R0xQbwnPjC/K3yDfUXPXj2FtrRWlG3iL9wD3UXZZmVF7Ya1ROOW\njcWzS/8eCdRkXCPx5jtv2iTWVatWiSqtK+fX/WbKeGHnYCc6tGolKut0olVeJ60GxEu3dgyDaPH0\n03et02KxiNdefVU42NkJrb29aNGkiUhJSbFJvNLjj2LuGG4IXBBCRAMoirIU6A6c+0e5x+Yrxg8/\n/EBCgisGQ3cA9PoKjBnzOl26dCnU8aNGjaROndq88sprHD/+GRaLADRAT8BAbu46li9fwfDhw9Fo\nNJQrVxa1uh4mUwdMJjCZjvPqq2+wf//OO+oOCwtj4cIlODs788EHb93XKpEl1Y4dO+jRpwd+dcuQ\neD6RDq06snP7ztu+1Z47dw5FpUJR5T2ngEqt4sWhL7Jj1w5cA1wp06AMF5ZF8c3cb3B3d6d77+4k\nHE0gJymXuE1x/HZgpU3iNRqN2Gn/fuvZOdqhKArLV69m2bJlRF24wMgGDRjSvz/C8vf8EwEoBexb\nvGTJElb88APjTCYcgQ1HjjDmpZdumxMgSXdji45hf6xbbf0llr/bSG7VRFGU44qihCuKEmKD8z6w\npKRkDIZbR4p4kJFxfxO1mjZtypEjB/j229lYb4B6AcFAdSCU+Pi/J5QlJCRiMt26xowniYnJd9S5\ncOGPvPDCaDZuVPPbb4k0axbKGbnGyz0NemEQnRZ1oNfGngw79QLbD21n/fr1t5UJDg4msHwgG17c\nSNTGi6wbvB53R3fatWvH0YNH6ViuE5aDAn8/f5auXErFihXZ+cdO2jl1oG/lfhw7dBw/Pz+bxNum\nTRuSTiazd/o+rkRE83v/ddRrUJfZs2fj4eHBJzNm0KdPH9p37swyrMMO9gBHgAkT7r5f9Z4dOwjJ\nykKH9U1d32Bg3549NolX+m97VKODjgABQog6wGysvajFpkuXzuh0J4CrQBpa7Ta6di3cXcCtUlJS\neO21NwB3yJ+YD5BLnTp/f4Pv2bMrOt0R4AaQjla7ix497jzf9Okz0es7A7WBpmRl1Wb+/B/uO66S\nxGKxcP3qdSq2qQCAvc4e/6ZluHz58m3l1Go1m9Zuor5zQ/54aTtnV5/jasxVuvTsgpeXFzHXY9CX\n1lP7k5okBMbTpEUTypUrx8QPJ/LWW2/ld/TbgoeHB3t37MXlhBtnJ0TieMWOmGOn2PDhh4wfMoTh\nL7wAwMrff6fX0KHsdXcnys+PsNWrC1xUrkJgIHGOjvzVm3ZVUQgICLhrWUm6lS2ag+KAW/+3lc17\nLp8QIvOW3zcoijJHURQPIcSdX4eBSZMm5f8eGhpKaGioDcL8W/PmzVmw4FvGj3+HrKxMevToznff\nfXPf9Zw9exaLRQHcgLVYx/3nAjvZubMC58+fp3LlyvTq1YsrV67y8cfTMBoNDBgwgOnT7+wXt1gs\ncNsGgKq856SCqFQqqj9VnaNzj9PglXqkXU0jav0l6g6ue0dZFxcXyviUoXQND15c8QKKWiH8+Q28\n/f7brPptFa8nj8NeZ09g+0rcPHiTrVu30qdPn4cSd8WKFVm5dCUxMTGEBAUx2mBAC+RmZfHd8uW8\n9d57VKlShR8WLoSFd9+68lavjh3LymXLWHTpEk6KQoJaTcSCBQ8ldqn4RUREEBERYZO6ijw6SFEU\nNRAJtAGuYx0mM0AIcfaWMj5CiIS83xsCy4UQFQqoTxQ1pkchOTmZ6tXrEB/vhnUI6L68V8oDoShK\nLH5+Z4iOjir02P9Zs77h3Xeno9e3BLLQ6Xaye/d2nnrqqYdzEf8RFy5coEPXDqSmp5CdkcPUqVMZ\nP3b8Xct27dUVTT97qve1tkhe3HyJy9OvcHjfEV678SoaF+tucGHtVjB15DSeffbZhxr7zp076Rba\nktdu+S8/W6Vi9c6dNGvW7L7qMhgMbN++Hb1ez9NPP12o3cqk/4ZiXUpaCGFWFGUMsJm/h4ieVRTl\nJevLYh7QW1GUkVjnpGQD/Yp63uI2a9YskpJKYx0mCpCKdY1GE5CIEA1JSdlHXFxcoW/LX3llDFqt\nNq9j2ImPPlovE0AhBAcHc+HMBa5fv46bmxvOzs4Flg2qGMSeLbsJ6VMNRVGI3hxN5aAqhFStzopn\nVlN7VE2u77uOIcZIu3btHnrs6enp5NqrOWA0U1PAWQXSEYUe4nkrBwcHOnTo8BCilP7L5KYyDyAi\nIoL27TthNNYD2mGd/vAb0BlrXt0INMbBYQeJiQlyDsBj4Ny5c7w46kWizl/AYDLi4u+CnYMd6nQ1\nu7btwtPTk5lfz2Tn3p0E+Acw6YNJj2Q/iEOHDtGpVye0ripuXkzBo5wLyTFZXI+7XuCGNX/Jyspi\n3759qNVqmjVrJvcKLsGKcicgk8AD8PEpy40btYCdWIeFHgEqAQ3ySkSiKGv49NOPeOON14srzBJL\nCMGlS5fIycmhSpUqZGVlUaVGFeq+U4fAzoEc//44Mb/FMW/OPCpXrsy2bdtQqVQ888wzlC5d+t4n\nsHGsfQb25uiVY/iHluHiqku80Hco0yZP+9fjrl27RvNGjVCnp2MSAteyZdm5fz8uLi6PKHLpcSKT\nwCNksViws7NHiPexNv9sw9oU1BJoklfqNCEhUZw+XfAmINLDYTQa6TOwDzv37MRB54CXqxeTP5jM\n2zPfov+uvoD1g/e7gHmELQqj/3P98HvaD4vJws1DiRzcc5CyZcs+0pgtFgtLly7l4sWL1K1bt1Dz\nVQb26cO11atpZTIhgHUaDa1Hj+bTL754+AFLjx25veQjpFKpqFy5OufPH0aIRoA3Gs2PKMpecnIA\n7NFq9/Dpp4uLOdKSadbsWZxNO8vLl/8PtYOaba9v59t535Ien4HZaEZtryY3LZecjBy+mvMVNUbX\noNl71uQd8f5OJk6ZyA9zH+2wXJVKxcCBA+/rmKjz5wkxmQDrLMzyublcOPfP+ZmSdG9yFdEHsHbt\nCvz9I9Fqv8LB4XumTZvI4MEDUZTtwBYCA8vf98gOyTZOnD5BUK9K2Gmss3CrDahK3PVY6teoR1iH\nFeyetoelrcMY8sIQoi5F4fOUd/6xXrU9ib8RX4zRF17DJk046eiIGetoi9M6HY2bNy/usKQnkEwC\nDyA4OJgrV85z/vyfJCffpFKlivzyyxqEGAO8xfnz9gwb9lJxh1kiVa9SnctrrmA2mhFCcH5VFNWq\nhrA67Hfe6P8m9dIaMOPNGZQvW57Ya7HsnraH7ORssm5mcXTmMdqGti3uSyiUGV98gUeDBnyl0fCV\ngwO1O3bk9TffLO6wpCeQ7BOwgddee52vvjoBPJ33TBJeXiu5cSO2OMMqkQwGA88825Vjp4/jWMoR\nrUVLxJYIypQpQ25uLqmpqXh5eeHt703vLT05MvcYx384gbAIWrRswdaNW1EVsD7P40YIQUJCAmq1\n+pGMZJIeX7JPoJgFBJTF0XELOTl/7RIWi7u7R3GHVSI5ODiwYc1GTp8+TW5uLjVr1kSj0bDwp4WM\neWUMdho7SnuUxpBrwNFdS6fZHeg4qz2bRm+ha3DXJyYBgPWNb8vlLKSSSd4JFEFMTAwjRozh3Lnz\npKQkkZvrRE6OHXAVjUbD6NEv88UXnxZ3mP9JBoOBNWvWkJqaSmhoKEFBQQWWPXnyJC3bt2TAjn54\nVinN0e+Pse+jA3hU8aDZ5MYknk1i82tb6dWrFz/O+xFHR0ebxanX69myZQsGg4FWrVrJWbzSQyGH\niBaDrKwsKleuzvXrzgjhiEqVisVyBWiKdb6AHU5OP7J582qaNm1avMH+x+Tk5NCibQtSVMm4VnQl\nav1FVi1fRatWre5a/qeffmL2H9/QeXFHwNqM8onmM4a+OJQly5fgHuhG49cbEfnreZ7yqMvihbYZ\n2ZWamkqz0KYY3UxoXDTcOHqD3dt3ExwcbJP6JekvsjmoGBw6dIgbN9IQwggEYrHE8PeWCYsBNSaT\nK6dOnSI5ORk7OztatmyJVqstvqD/I3766ScyXTLoH94XRVG4EB7FyLEjOXfy7kMkAwICuHboOoYs\nAw5ODsTtj8PZ1ZlqlatRq19N2s+xdgZXbFOBuYHz4d7rtRXKjM9noK2r5dkfOqAoCgdmHmTcW+MI\nXxVumxNIkg3IJPCArl27hslkAEYBDkAz4AvgMNa19NLJzd3L+PFvoVaXQQgTPj4OHDy4+57LAUj/\nLj4hHs+6pfM3jfGt68uN+C0Flm/VqhWdQjvxY61F+FT35uq+GJb8uIRr166hT9Dnl8uMz8RRZ7um\noKtxVynTzDc/Tv8mfhxeesxm9UuSLTw5vWCPAbPZzKpVq5g7dy45OTmo1c5YEwCAI9bdxToA9YHW\nQH2ysiA9vT8ZGYO4etWZjz6aQnJyMj179qNMmQo0atSC06dPF9MVPZlatmjJmUXnSDqfhNlgZu/k\nfTzd8ukCyyuKwleffcXsT2bz7oD3OHH4BF27dqVfv35kndYTPnQDez/bzy9tl1K6tAdnz54tsK77\n0aJpC07NP012SjZmg5kjXx2jeZOCx/JfunSJl8e8zIAhA1ixcoVNYpCke3rQfSkf1g+PYI/hB2Ey\nmUTbtp2Es3NFodU2Eo6OrsLZ2V1AVwFvCEVpL8BewHABk/J+WgjwuuXxs6JTpx6iQYOmwsGhkYDR\nQlG6Cnd3L3Hz5s3ivsQnyrfffSucXJyE2k4t2nRqI5KTkwssu3T5UuHs5iy8K3oLdy93sW3btvzX\nkpOTRfmg8sK3to/oOLuD6PRNB+HlZ5t/D4vFIl557RVhr7EXDo4OomvPriIrK+uuZaOjo4Wnr6do\nMeFp0XV+Z+FdyVvMnTe3yDFIJQNF2GNYdgwX0po1axg06FUyMwdj3fjlOlrtLwQGVuHy5SiqVg0h\nNjaahAQD1tVEM4B1qNU+mM0vAGZgERpNKgaDESFqYN2KMhAXlzAWL55Gt27diuvynkhCCMxmM3Z2\nBbdqxsbGUr1Odfr90Qff2j5c3naF8P4biL0Si06nIyUlBf/y/oxPGYtKbb0xXtl5NVNGTKVHjx4F\n1ns/cnNzMZlMODk5FVjm4ykfE56wlvbfWJevjjsQx7bBEcz4+FOioqKoVasWXbp0uW3fZEn6i+wY\nfgRu3LiBxeLN3zt/+ZCbm83Ro/uwt7cHoHr1uiQkXALCAFCrNQQFuREZOR1rp3FlcnOTgIZAKay7\nbHbAYslEp9M94it68imK8q8JACAyMhLfGr741vYBoGLrCji4OBATE0OVKlXQaDRYTBZy03LRemgR\nFkFWot6mHfgajQaNRvOvZYxGI3bO9vmP7XR2JMfd4K1hwyiXnc23Wi0DR4zgs5kzbRaXJIHsEyi0\npk2bIsR5IBYwo1bvoGbNuvkJAODq1ctAF6zf8GtgNlemfv3auLgEAm9i3Yu4HtAWaAR0R1E2UbWq\nPy1btnzUl1QiVKhQgYQzCaTHpgNw49QNspKy8jeN1+l0jBw9kuXtfuPA1wf5vd9aPO09Cxxu+k/Z\n2dmMfvllQipVonWzZpw8efKB4uzbpy+nFpzmxE8nubztCmsHrEcxmBiUlUVbi4XnsrL4bs4cEhIS\nHqh+SSqITAKFFBISwpIlC3F1XYVKNZVatXIID191WxnrEsQrsX7YuwCn8Pb2xmS6ibU5yIy1A/kv\njri7O7Nr1x+3JRPJdgIDA5nw3gR+qruY5a1XsLRVGN9/9/1tG/3M/HQmH4+dQsULgQysP4gdW3YU\neoOWwQMGsOvnn2l++TJOe/fSqnlz4uLi7n3gP9SoUYMNazaQuVzPuYnneebpZ/DS6fKHHegAZ3t7\nUlNT77tuSfo3sk/gAZjN5rvuG9y8eWv27CkF/LXJ+QG6d3emSpUgZs+eD7ij10cD3QEndLptvPfe\nSN5//51HF3wJdebMGY4cOULjxo1tNlnLZDKhc3TkTbM5/8N6jU7HmNmzGTp0aJHqTk1NpXKlSjRL\nSaEycEKl4ryfH5GXLskvDNIdZJ/AI1bwxvEK//ymn5uby4wZ0+jT51kuXbpEeno63323EL1ez9Ch\nY3jzzTceQcQl2969e+nRuwcWtYXcjFzmz5tP/779i1yvSqVCpVKRc0sSyFGp7tn+Xxhubm5sjYjg\n+X792BodTfWqVdkSFiYTgGR7Dzqs6GH98JgOES2MxYuXCJ3OW8DzAgYJna60WLVqVXGHVaJ8+vmn\nws3TTWidtWLoiKEiIyNDeJXxEv3W9hETxHvi/46/KFw9XcWVK1cKXWdOTk6Br334wQfCX6cTnUHU\nd3AQwRUqiIyMDFtciiQVGkUYIirvBGzg8uXLfPnlLNLTM3j55efZtCkCRVF4991ZNhtmWNJER0ez\nbt06HBwc6NWrFx4e916VNSwsjJnzZzJwb3+07o6ED97IuDfGIewElbtam4B8a/vgX9eP06dPU758\n+TvqEEIweepkZs2ehdFgBAUyUzOpEFSBlctWUqdOndvKT5o8mcpVq/LHxo08Va4cr7/5Js7Ozrb5\nI0jSIyD7BIooOjqa2rXrk5FRDYvFCZ3uED/8MJv+/Yve3FBSHT9+nNbtWxPYrRKGdCPJh5M5vO8w\nPj4+/3rc0BFDSahznfqj6gEQd/Aa+14+QMylGAbu6o9PTW8yEzL5qc5idv+xm5CQkDvqmLdgHh9/\n8zHdV3RFZa/mt94rqd6vGs5lnNn3zgGuXLhi01VGJckWitInIEcHFdH3388jM7MyFksboDF6fRcm\nTJhS3GE90d54/w2aTmlMpwUd6L68K2W7+TPj8xn3PM7b05uk08n5j2+evomXpxfzvp/HstZhrOi4\nmp/qLGLcK+PumgAA1m5cS4N36+ER5IFbeVdCJ7fg4qZL1BxUA5VW4fLlyza7Tkl6HMjmoHuwWCxM\nmTKNn376BUdHR6ZNm3hbE092dg5m860dgY7kWHeclx7QzcSb1K1eO/9x6eoeJOy99/j4N157g6VN\nlvJ777U4emg4vzqKrRu2Uq9ePZo0asLp06epWLFigQkAoLR7aaIj//6gTzyXiNZDS3pcBmkJ6XIH\nL+k/RzYH3cPHH0/lk0/mo9e3A7LRatezfv1KQkNDAThw4ACtW3dEr28POKPTbfv/9u49turyjuP4\n+1vaQoXQwiJ4A6SCOEkNMC2daCQzjItcdJuTZN4RbyM1DlyZbI75h8Ki84JTw9REpkQnUWQCEyqW\njKEdActtIJWCQ24j3ATBtrTf/XEO2EBP28Np+zunv88raTiX55zzffJtz5fn+T3P78eUKXcyevQo\ntmzZQv/+/c+YR5aGFU0rYv6q+Yx+cyRVR6p4d8z7zHx0Jrf+4tZGX3vw4EHmzZtHZWUlo0aNIjc3\nN67PrqiooOCaAnoOv4jaNGfDWxu5dFhf9q7Zy5TCRyiaUnS23RJpMbqoTAvKzb2cbduuBnpEH1nJ\nxIm5zJ790qk2S5YsoajoMY4e/YbbbruFysoqnn32Jdq160lNzTYef/x3TJ78cCDxp6Lq6moeKHyA\nuW/MJSMzg0emPMK0qdNa7bw5u3fv5p133qGmpoYLLriAI0eOkJeXx+DBg1vl80XipSLQgvLyrmTD\nhkuBfgCkpRVTWJjPM888XW/7rVu3kpf3A44fvxfoCBymffvZ7NixTVMJItIidGC4BT355HSyshYD\nK0hLK6ZTp00UFk6K2X7Xrl1kZp5LpAAAZJOZmaNzvohIUtJIoAlWrFjB3Llv07HjOTz44P307t07\nZukFC4kAAAjjSURBVNv9+/dz8cV9OXp0DJALbCYn5yN27tyuM4XGYd++fbzw4gscOHiA0SNHM3z4\n8KBDEklamg5KMsuWLeOmm27m+PHjdO6czcKF8zWfHIcDBw4wMH8g3a4/l5y+2ZTNWscTjz3BxAkT\ngw5NJCmpCCQhd+fw4cNkZ2frQiBxmjVrFq99+ipj3rwBgN1r9rDoJ4vZuX1XwJGJJCedQC4JmRk5\nOTlBh5GSjh07Rla373blduzekWPfHGfGH2ewduNa+vfrz5RfTdHOXZFmoJGAJJ2NGzcyZOgQhr18\nPV37dmV50T+prPiWrD5Z9PnpJVQs2Ea3b7tTvKiYtDStbRDRdJC0OSUlJUx+dDKHDh1iSMEQPvjw\nAx7Ydi/tMttRe6KWV/q9xpL3lnLFFVcEHapI4DQdJG3O0KFDWb1yNRAZGSxZ/iFpGZH/9Vs7IyMr\ng6qqqiBDFGkTNBKQpHfixAmuvPpKsgo6cNn4fpS/W87+pQdZu2ptky8DKdKWBb5ZzMxGmNlmM9ti\nZvWeXMXMnjezcjMrMzOdTEeaLD09nY8Wf0Sfo31Z83AZPfb1ius6wCISW8IjATNLA7YA1wO7gFXA\neHffXKfNSGCSu99gZoOB59y9IMb7aSQgIhKHoEcC+UC5u3/p7tXAW0SupF7XOGAOgLuXAtlm1vAV\nQkREpMU1RxG4ENhR5/5X0ccaarOznjYiItLKknJ10PTp00/dHjp06Klz94uISGQJdUlJSbO8V3Mc\nEygAprv7iOj9qYC7+8w6bV4GPnb3t6P3NwPXufsZp9bUMQERkfgEfUxgFdDHzHqZWSYwHlhwWpsF\nwO1wqmgcqq8AiIhI60p4Osjda8xsErCESFF51d03mdl9kad9trsvMrNRZvYF8A1wV6KfKyIiidNm\nMWmTli9fzsxnZ1JVXcU9t93D+FvGBx2SSIsJejpIJKl88sknjLt5HOlj08i5szOFUwuZ88acoMMS\nSUoaCUibc/d9d/PVZTsoeDgfgC8Wb2XrjG2ULi8NODKRlqGRgEgdZobX1J66X3uiVhf2EYkhKfcJ\nSP2qq6spKyvD3Rk4cCAZGRlBh5SU7p9wP8NGDSM9K532nduz4tF/8eenXww6LJGkpOmgFPH1119z\n7bU/oqJiD2ZGjx7fY8WKj+nSpUvQoSWl0tJSnnr+Kaqqq5hw6wTGjh0bdEgiLUYXlQmBSZMe4pVX\nVlJZOQowMjP/wR13DGL2bP0PVyTsdFGZEFi/fjOVlZdw8jBOVdUlrFu3MdigRCTl6cBwisjPH0CH\nDpuAGqCWDh02cdVVg4IOS0RSnKaDUsSxY8cYPnwMq1eXYWbk5V1OcfEiOnXqFHRoIhIwHRMICXen\noqICdyc3N5e0NA3kRERFQEQk1LRZTEREzoqKgIhIiKkIiIiEmIqAiEiIqQiIiISYioCISIipCIiI\nhJiKgIhIiKkIiIiEmIqAiEiIqQiIiISYioCISIipCIiIhJiKgIhIiKkIiIiEmIqAiEiIqQiIiISY\nioCISIipCIiIhJiKgIhIiKkIiIiEmIqAiEiIqQiIiISYioCISIilJ/JiM+sCvA30ArYDP3f3w/W0\n2w4cBmqBanfPT+RzRUSkeSQ6EpgKFLt7P2AZ8JsY7WqBoe4+MMwFoKSkJOgQWpT6l9rUv3BKtAiM\nA16P3n4duDFGO2uGz0p5bf2XUP1LbepfOCX6xdzN3fcCuPseoFuMdg4sNbNVZjYxwc8UEZFm0ugx\nATNbCnSv+xCRL/Xf1tPcY7zNEHffbWbnEikGm9x9RdzRiohIszL3WN/bTXix2SYic/17zew84GN3\n/34jr/k9cMTd/xTj+bMPSEQkpNzdzuZ1Ca0OAhYAdwIzgTuA909vYGbnAGnuftTMOgI/Bv4Q6w3P\ntiMiIhK/REcCXYG/AT2AL4ksET1kZucDf3H30WbWG3iPyFRROvCmu89IPHQREUlUQkVARERSW6DL\nNs3sZ2a2wcxqzGxQA+1GmNlmM9tiZkWtGWMizKyLmS0xs8/N7EMzy47RbruZrTWzz8zs360dZ7ya\nkg8ze97Mys2szMwGtHaMiWisf2Z2nZkdMrM10Z/6FkkkJTN71cz2mtm6Btqkcu4a7F+K5+4iM1tm\nZhvNbL2ZFcZoF1/+3D2wH6Af0JfIRrNBMdqkAV8Q2ZWcAZQBlwUZdxz9mwn8Onq7CJgRo10F0CXo\neJvYp0bzAYwEFkZvDwY+DTruZu7fdcCCoGM9y/5dAwwA1sV4PmVz18T+pXLuzgMGRG93Aj5vjr+9\nQEcC7v65u5cTWXYaSz5Q7u5funs18BaRTWqpoC1upmtKPsYBcwDcvRTINrPupIam/r6l5AIGjyzN\nPthAk1TOXVP6B6mbuz3uXha9fRTYBFx4WrO485cKXzwXAjvq3P+KMzuerNriZrqm5OP0NjvraZOs\nmvr79sPocHuhmV3eOqG1ilTOXVOlfO7M7GIiI57S056KO3+JLhFtVAObzaa5+99b+vNbmjbThdJq\noKe7HzOzkcB84NKAY5KmSfncmVknYB7wUHREkJAWLwLuPizBt9gJ9Kxz/6LoY0mhof5FD1B19+82\n0/0vxnvsjv67z8zeIzIlkaxFoCn52Elk2XBDbZJVo/2r+4fn7ovN7EUz6+ruB1opxpaUyrlrVKrn\nzszSiRSAv7r7GfuyOIv8JdN0UKx5ulVAHzPrZWaZwHgim9RSwcnNdNDAZrpoZafOZroNrRXgWWhK\nPhYAtwOYWQFw6OS0WApotH9151jNLJ/IUuuU+BKJMmL/vaVy7k6K2b82kLvXgP+4+3Mxno8/fwEf\n7b6RyPzVcWA3sDj6+PnAB3XajSByJLwcmBr0Ufo4+tcVKI7GvgTIOb1/QG8iK1A+A9anQv/qywdw\nH3BvnTYvEFlls5YYK7+S9aex/gG/JFKoPwNWAoODjjmOvs0FdgGVwH+Bu9pY7hrsX4rnbghQU+f7\nYk30dzWh/GmzmIhIiCXTdJCIiLQyFQERkRBTERARCTEVARGREFMREBEJMRUBEZEQUxEQEQkxFQER\nkRD7P344UahEr4zRAAAAAElFTkSuQmCC\n", 653 | "text/plain": [ 654 | "" 655 | ] 656 | }, 657 | "metadata": {}, 658 | "output_type": "display_data" 659 | } 660 | ], 661 | "source": [ 662 | "# Get encoded coordinates\n", 663 | "ec1 = np.array(layers[1]['activations'][0, :].transpose())\n", 664 | "ec2 = np.array(layers[1]['activations'][1, :].transpose())\n", 665 | "\n", 666 | "# Plot encoded data\n", 667 | "plt.scatter(ec1, ec2, c=labels)\n", 668 | "plt.show()" 669 | ] 670 | }, 671 | { 672 | "cell_type": "markdown", 673 | "metadata": {}, 674 | "source": [ 675 | "Since a linear autoencoder is equivalent to PCA, it is not surprising, that this looks similar (but may be rotated/inverted)." 676 | ] 677 | }, 678 | { 679 | "cell_type": "markdown", 680 | "metadata": {}, 681 | "source": [ 682 | "### Sigmoid autoencoder" 683 | ] 684 | }, 685 | { 686 | "cell_type": "markdown", 687 | "metadata": {}, 688 | "source": [ 689 | "Same procedure for the sigmoid activation function:" 690 | ] 691 | }, 692 | { 693 | "cell_type": "code", 694 | "execution_count": 203, 695 | "metadata": { 696 | "collapsed": false 697 | }, 698 | "outputs": [ 699 | { 700 | "data": { 701 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEACAYAAACznAEdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEgVJREFUeJzt3X2s3mV9x/H3p60VKxPFgMQiz2EwwsOY1jqQHYVJ45ZC\ncHEFA4pjITGo8Y8NNFH6B0vExGwYZ7JuyNDpmAPHQzIZOnOysFjLRnm0pRWUZ9CJVnku9bs/7uvY\nm5PTnvvA3fO76f1+JXf4PVzX73ddFyf3p9fv4ZxUFZIkLei6AZKk0WAgSJIAA0GS1BgIkiTAQJAk\nNQaCJAkYMBCSrEiyMcmmJBfOsH9lktuTrE+yLskJbfv+Sb6b5O4kdyb52LA7IEkajsz2HkKSBcAm\n4GTgEeAWYFVVbewrs6Sqnm7LRwPfqKojk+wH7FdVtyXZE/hf4LT+upKk0TDIDGEZsLmq7q+qrcBV\nwGn9BabCoNkT+HXb/lhV3daWnwQ2AEuH0XBJ0nANEghLgQf71h9ihi/1JKcn2QDcAHx4hv0HAccB\n338pDZUk7VpDu6lcVddW1ZHA6cAl/fva5aKrgY+3mYIkacQsGqDMw8ABfev7t20zqqqbkxySZO+q\neiLJInph8NWqum5H9ZL4S5UkaY6qKsM61iAzhFuAw5IcmGQxsAq4vr9AkkP7lo8HFlfVE23Tl4Ef\nVNVls52oqvxUcfHFF3fehlH4OA6OhWOx88+wzTpDqKptSS4AbqIXIJdX1YYk5/d21xrgfUnOAZ4H\nngHeD9AeP/0AcGeS9UABn6qqG4feE0nSyzLIJSPaF/hvT9v2d33LnwM+N0O9/wYWvsw2SpLmgW8q\nj6CJiYmumzASHIftHIvtHItdZ9YX0+ZLkhqVtkjSK0ESap5vKkuSxoCBIEkCDARJUmMgSJIAA0GS\n1BgIkiTAQJAkNQaCJAkwECRJjYEgSQIMBElSYyBIkgADQZLUGAiSJMBAkCQ1BoIkCTAQJEmNgSBJ\nAgwESVJjIEiSAANBktQYCJIkwECQJDUGgiQJMBAkSY2BIEkCDARJUmMgSJIAA0GS1BgIkiTAQJAk\nNQaCJAkwECRJjYEgSQIGDIQkK5JsTLIpyYUz7F+Z5PYk65OsS3JC377Lkzye5I5hNlySNFypqp0X\nSBYAm4CTgUeAW4BVVbWxr8ySqnq6LR8NfKOqjmzrJwJPAl+pqmN2cp6arS2SpO2SUFUZ1vEGmSEs\nAzZX1f1VtRW4Cjitv8BUGDR7Ar/u23cz8PMhtFWStAsNEghLgQf71h9q214kyelJNgA3AB8eTvMk\nSfNl0bAOVFXXAte2S0SXAH8412OsXr36N8sTExNMTEwMq3mS9Io3OTnJ5OTkLjv+IPcQlgOrq2pF\nW78IqKq6dCd17gXeVlVPtPUDgRu8hyBJw9PFPYRbgMOSHJhkMbAKuH5aow7tWz4eWDwVBlOb20eS\nNKJmvWRUVduSXADcRC9ALq+qDUnO7+2uNcD7kpwDPA88A7x/qn6SrwMTwBuTPABcXFVXDL8rkqSX\nY9ZLRvPFS0aSNDddXDKSJI0BA0GSBBgIkqTGQJAkAQaCJKkxECRJgIEgSWoMBEkSYCBIkhoDQZIE\nGAiSpMZAkCQBBoIkqTEQJEmAgSBJagwESRJgIEiSGgNBkgQYCJKkxkCQJAEGgiSpMRAkSYCBIElq\nDARJEmAgSJIaA0GSBBgIkqTGQJAkAQaCJKkxECRJgIEgSWoMBEkSYCBIkhoDQZIEGAiSpGagQEiy\nIsnGJJuSXDjD/pVJbk+yPsm6JCcMWleSNBpSVTsvkCwANgEnA48AtwCrqmpjX5klVfV0Wz4a+EZV\nHTlI3b5j1GxtkSRtl4SqyrCON8gMYRmwuarur6qtwFXAaf0FpsKg2RP49aB1JUmjYZBAWAo82Lf+\nUNv2IklOT7IBuAH48FzqSpK6N7SbylV1bVUdCZwOXDKs40qS5seiAco8DBzQt75/2zajqro5ySFJ\n9p5r3dWrV/9meWJigomJiQGaJ0njYXJyksnJyV12/EFuKi8E7qF3Y/hRYB1wZlVt6CtzaFXd25aP\nB66rqrcMUrfvGN5UlqQ5GPZN5VlnCFW1LckFwE30LjFdXlUbkpzf211rgPclOQd4HngGeP/O6g6r\n8ZKk4Zl1hjBfnCFI0tx08dipJGkMjFQgOEGQpO6MVCBs29Z1CyRpfI1UILzwQtctkKTxZSBIkoAR\nCwQvGUlSd0YqEJwhSFJ3DARJEmAgSJIaA0GSBBgIkqTGQJAkAQaCJKkxECRJwIgFgi+mSVJ3RioQ\nnCFIUncMBEkSYCBIkhoDQZIEGAiSpMZAkCQBBoIkqRmpQPA9BEnqzkgFgjMESeqOgSBJAgwESVJj\nIEiSAANBktSMVCBs3dp1CyRpfI1UIDhDkKTujFQgPP981y2QpPFlIEiSAANBktQYCJIkwECQJDUG\ngiQJGDAQkqxIsjHJpiQXzrD/rCS3t8/NSY7p2/fxJHe2z8d2dh4DQZK6M2sgJFkAfBE4FTgKODPJ\nEdOK3QecVFXHApcAa1rdo4A/A94KHAf8cZJDdnQuX0yTpO4MMkNYBmyuqvuraitwFXBaf4GqWltV\nW9rqWmBpWz4S+H5VPVdV24D/As7Y0YmcIUhSdwYJhKXAg33rD7H9C38m5wHfast3Ae9M8oYkS4D3\nAm/ZUUUDQZK6s2iYB0vyLuBc4ESAqtqY5FLg28CTwHpgh38XzUCQpO4MEggPAwf0re/ftr1Iu5G8\nBlhRVT+f2l5VVwBXtDJ/xYtnGy9y112rWb26tzwxMcHExMQAzZOk8TA5Ocnk5OQuO36qaucFkoXA\nPcDJwKPAOuDMqtrQV+YA4D+Bs6tq7bT6+1TVT1uZG4HlVfXLGc5Tp5xSfPvbL7dLkjQeklBVGdbx\nZp0hVNW2JBcAN9G753B5VW1Icn5vd60BPg3sDXwpSYCtVbWsHeKaJHsDW4GPzBQGU7xkJEndmXWG\nMF+S1PLlxfe+13VLJOmVYdgzBN9UliQBBoIkqTEQJEmAgSBJakYqEPxdRpLUnZEKBGcIktQdA0GS\nBBgIkqTGQJAkASMWCFWwbYe/C1WStCuNVCAsXuyTRpLUlZELBC8bSVI3Ri4Qnnuu61ZI0ngaqUDY\nYw949tmuWyFJ42mkAuE1r4Fnnum6FZI0ngwESRJgIEiSGgNBkgSMYCA8/XTXrZCk8TRygeAMQZK6\nYSBIkgADQZLUGAiSJGDEAmHJEgNBkroyUoHgDEGSujNygeBjp5LUjZELBGcIktQNA0GSBBgIkqTG\nQJAkAQaCJKkZqUBYssSnjCSpKyMVCK97HfzqV123QpLG08gFwpYtXbdCksbTyAXCL3/ZdSskaTyN\nZCBUdd0SSRo/AwVCkhVJNibZlOTCGfafleT29rk5yTF9+z6R5K4kdyT5WpLFOzrPq18NCTz33Evr\njCTppZs1EJIsAL4InAocBZyZ5Ihpxe4DTqqqY4FLgDWt7puBjwLHV9UxwCJg1c7O52UjSerGIDOE\nZcDmqrq/qrYCVwGn9ReoqrVVNXU7eC2wtG/3QuC1SRYBS4BHdnYyA0GSujFIICwFHuxbf4gXf+FP\ndx7wLYCqegT4PPAA8DDwi6r6zs5OttdeBoIkdWHRMA+W5F3AucCJbf319GYTBwJbgKuTnFVVX5+p\n/urVq3niCfjCF+BDH5pgYmJimM2TpFe0yclJJicnd9nxU7M80pNkObC6qla09YuAqqpLp5U7BrgG\nWFFV97ZtfwKcWlV/3tbPBt5eVRfMcJ6qKlauhPPOg5Urh9A7SdqNJaGqMqzjDXLJ6BbgsCQHtieE\nVgHXT2vUAfTC4OypMGgeAJYn2SNJgJOBDTs7mfcQJKkbs14yqqptSS4AbqIXIJdX1YYk5/d21xrg\n08DewJfaF//WqlpWVeuSXA2sB7a2/67Z2fl8W1mSujHrJaP5MnXJ6DOfgYUL4eKLu26RJI22Li4Z\nzat994Wf/KTrVkjS+DEQJEnACAbCPvvAT3/adSskafyMXCA4Q5CkboxcIOyzj4EgSV0YuaeMtm3r\n/dbTZ5+FRUN9j1qSdi+7/VNGCxfCG9/oLEGS5tvIBQLAQQfBj3/cdSskabyMZCAccgj86Eddt0KS\nxsvIBsJ993XdCkkaLyMZCAcfbCBI0nwbyUA49FD44Q+7boUkjZeRe+wU4IknejeWf/ELWDCSkSVJ\n3dvtHzsF2Hvv3gtq99zTdUskaXyMZCAAvPWtsG5d162QpPExsoHw7nfDjTd23QpJGh8jeQ8B4LHH\n4Mgj4dFHYY89OmyYJI2osbiHALDffvCOd8CVV3bdEkkaDyM7QwBYuxbOOANuvbUXEJKk7cZmhgCw\nfDlccAGcdFLvfsILL3TdIknafY30DGHKN78Jl1zSe1nt8MN7f0Rnzz17vx570aLeb0hduBCyk5x8\nqfteTt35Pq51rGOd8arz0Y8Od4bwigiEKT/7We9XWjz+ODz1FGzb1ps1vPBCb3lHdnbY2br/UuvO\n93GtYx3rjFedKvjSl8Y4ECRJ243VPQRJ0vwxECRJgIEgSWoMBEkSYCBIkhoDQZIEGAiSpMZAkCQB\nBoIkqTEQJEmAgSBJagwESRIwYCAkWZFkY5JNSS6cYf9ZSW5vn5uTHN22H55kfZJb23+3JPnYsDsh\nSXr5Zg2EJAuALwKnAkcBZyY5Ylqx+4CTqupY4BLg7wGqalNV/W5VHQ/8HvAU8G9DbP9uaXJysusm\njATHYTvHYjvHYtcZZIawDNhcVfdX1VbgKuC0/gJVtbaqtrTVtcDSGY5zCnBvVT34cho8DvyB73Ec\ntnMstnMsdp1BAmEp0P8l/hAzf+FPOQ/41gzb/xT458GbJkmaT4uGebAk7wLOBU6ctv1VwErgomGe\nT5I0PLP+xbQky4HVVbWirV8EVFVdOq3cMcA1wIqqunfavpXAR6aOsYPz+OfSJGmOhvkX0waZIdwC\nHJbkQOBRYBVwZn+BJAfQC4Ozp4dBcyazXC4aZqckSXM30N9UTrICuIzePYfLq+qzSc6nN1NYk+Tv\ngTOA+4EAW6tqWau7pG0/pKp+tYv6IUl6mQYKBEnS7q/zN5Vne+ltd5Nk/yTfTXJ3kjunXtRL8oYk\nNyW5J8l/JNmrr84nk2xOsiHJe7pr/fAlWdBeXLy+rY/lOAAk2SvJv7b+3Z3k7eM4Hkk+keSuJHck\n+VqSxeM0DkkuT/J4kjv6ts25/0mOb2O4KcnfDHTyqursQy+QfggcCLwKuA04oss2zUOf9wOOa8t7\nAvcARwCXAn/Ztl8IfLYt/w6wnt79noPaeKXrfgxxPD4B/BNwfVsfy3FoffxH4Ny2vAjYa9zGA3gz\nvRddF7f1fwE+OE7jQO8pzeOAO/q2zbn/wPeBt7XlfwdOne3cXc8QZn3pbXdTVY9V1W1t+UlgA7A/\nvX5f2YpdCZzellcCV1XVC1X1Y2AzvXF7xUuyP/Be4B/6No/dOAAkeR3wzqq6AqD1cwvjOR4Lgdcm\nWQS8BniYMRqHqroZ+Pm0zXPqf5L9gN+qqltaua/01dmhrgNhri+97VaSHETvXwJrgTdV1ePQCw1g\n31Zs+hg9zO4zRn8N/AXQfyNrHMcB4GDg/5Jc0S6hrWkPZIzVeFTVI8DngQfo9WlLVX2HMRuHGew7\nx/4vpfd9OmWg79auA2FsJdkTuBr4eJspTL+7v1vf7U/yR8Djbba0s0eOd+tx6LMIOB742+r97q+n\n6L3IOW4/F6+n96/hA+ldPnptkg8wZuMwgF3S/64D4WHggL71/du23VqbCl8NfLWqrmubH0/yprZ/\nP+AnbfvDwFv6qu8uY3QCsDLJffTeUXl3kq8Cj43ZOEx5CHiwqv6nrV9DLyDG7efiFOC+qnqiqrbR\n+2WYv8/4jcN0c+3/SxqXrgPhNy+9JVlM76W36ztu03z4MvCDqrqsb9v1wIfa8geB6/q2r2pPWhwM\nHAasm6+G7ipV9amqOqCqDqH3//27VXU2cANjNA5T2uWAB5Mc3jadDNzNmP1c0LtUtDzJHklCbxx+\nwPiNQ3jxzHlO/W+XlbYkWdbG8Zy+Ojs2AnfUV9B70mYzcFHX7ZmH/p4AbKP3RNV64NY2BnsD32lj\ncRPw+r46n6T39MAG4D1d92EXjMkfsP0po3Eeh2Pp/SPpNuCb9J4yGrvxAC5ufbqD3g3UV43TOABf\nBx4BnqMXkOcCb5hr/+n9yYE723frZYOc2xfTJElA95eMJEkjwkCQJAEGgiSpMRAkSYCBIElqDARJ\nEmAgSJIaA0GSBMD/A0VG7tc6jutWAAAAAElFTkSuQmCC\n", 702 | "text/plain": [ 703 | "" 704 | ] 705 | }, 706 | "metadata": {}, 707 | "output_type": "display_data" 708 | } 709 | ], 710 | "source": [ 711 | "layers, error_history = trainAE(sigmoid, 1, 1000)\n", 712 | "\n", 713 | "# Plot error history\n", 714 | "plt.plot(range(1000), error_history)\n", 715 | "plt.show()" 716 | ] 717 | }, 718 | { 719 | "cell_type": "code", 720 | "execution_count": 204, 721 | "metadata": { 722 | "collapsed": false 723 | }, 724 | "outputs": [ 725 | { 726 | "data": { 727 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEACAYAAACgS0HpAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd4VVXa9/HvnUpCINRQpYRmV4oUBY10UBEZC4yOjoyj\n4yj6zDuPyjg6oOOjgmUY6wxjL8iIIGCjEwtIBxWklxAICSWQAElIOev9Yx9iCCWbkJAov891nYuz\n1157n3svktxnrbWLOecQEREpSUhFByAiIj8PShgiIuKLEoaIiPiihCEiIr4oYYiIiC9KGCIi4ouv\nhGFmfc1sjZmtM7OHTlDvEjPLM7NBRcr+ZGYrzex7M3vfzCKC5SPMbJuZLQu++p764YiISHkpMWGY\nWQjwEtAHOA8YYmZnH6fe08D0ImUNgWFAO+fchUAYMLjIZs8759oFX9NO6UhERKRc+elhdATWO+eS\nnHN5wHjg2mPUGwZ8BOwsVh4KVDWzMCAaSCmyzk4+ZBERqQh+EkYjILnI8rZgWaFgT2Kgc+5ViiQB\n51wK8BywFdgO7HPOzSqy6b1mtsLMXjOz2FIeg4iInAZlNek9Big6t2EAZlYDrzfSFGgIxJjZr4N1\nXgHinXMXA6nA82UUi4iIlIMwH3W2A02KLDcOlhXVARhvZgbUAfqZWR4QAWxyzqUDmNkk4FJgnHNu\nV5Ht/wN8cqwPNzPd7EpEpBScc2U67O+nh7EYaGlmTYNnOA0GphYLKj74ao43j/FH59xUvKGozmZW\nJZhMegCrAcysfpFdDAJWHi8A51ylf40YMaLCY1CcilFxKs7Dr/JQYg/DOVdgZvcCM/ASzOvOudVm\ndpe32o0tvkmRbReZ2UfAciAv+O/h+qPN7GIgAGwB7jrVgxERkfLjZ0gK553y2qZY2b+PU3doseXH\ngMeOUe9W/2GKiEhF05XeZSQhIaGiQ/BFcZadn0OMoDjL2s8lzvJg5TXWVVbMzFX2GEVEKhszw1XA\npLeIiIgShoiI+KOEISIivihhiIiIL0oYIiLiixKGiIj4ooQhIiK+KGGIiIgvShgiIuKLEoaIiPii\nhCEiIr4oYYiIiC9KGCIi4osShoiI+KKEISIivihhiIiIL0oYIiLiixKGiIj4ooQhIiK+KGGIiIgv\nShgiIuKLEoaIiPiihCEiIr4oYYiIiC9KGCIi4osShoiI+OIrYZhZXzNbY2brzOyhE9S7xMzyzGxQ\nkbI/mdlKM/vezN43s4hgeU0zm2Fma81supnFnvrhiIhIeSkxYZhZCPAS0Ac4DxhiZmcfp97TwPQi\nZQ2BYUA759yFQBgwOLh6ODDLOdcGmAP85dQORUREypOfHkZHYL1zLsk5lweMB649Rr1hwEfAzmLl\noUBVMwsDooHtwfJrgbeD798GBp5k7CIichr5SRiNgOQiy9uCZYWCPYmBzrlXATtc7pxLAZ4DtuIl\nin3OudnB1XHOubRgvVQgrrQHISIi5a+sJr3HAEXnNgzAzGrg9SSaAg2BGDP79XH24cooFhERKQdh\nPupsB5oUWW7MT8NKh3UAxpuZAXWAfmaWB0QAm5xz6QBmNgm4FBgHpJlZPedcmpnV5+ihrEIjR44s\nfJ+QkEBCQoKPsEVEzhyJiYkkJiaW62eYcyf+Ym9mocBaoAewA1gEDHHOrT5O/TeBT5xzk8ysI/A6\ncAlwCHgTWOyce9nMRgHpzrlRwTOvajrnhh9jf66kGEVE5EhmhnPOSq7pX4k9DOdcgZndC8zAG8J6\n3Tm32szu8la7scU3KbLtIjP7CFgO5AX/PVx/FPChmQ0FkoAbT/loRESk3JTYw6ho6mGIiJy88uhh\n6EpvERHxRQlDRER8UcIQERFflDBERMQXJQwREfFFCUNERHxRwhAREV+UMERExBclDBER8UUJQ0RE\nfFHCEBERX5QwRETEFyUMERHxRQlDRER8UcIQERFflDBERMQXJQwREfFFCUNERHxRwhAREV+UMERE\nxBclDBER8UUJQ0REfFHCEBERX5QwRETEFyUMERHxRQlDRER8UcIQERFflDBERMQXJQwREfHFV8Iw\ns75mtsbM1pnZQyeod4mZ5ZnZoOByazNbbmbLgv9mmNl9wXUjzGxbcN0yM+tbNockIiLlwZxzJ65g\nFgKsA3oAKcBiYLBzbs0x6s0EsoE3nHOTjrF+G9DRObfNzEYA+51zz5fw+a6kGEVE5EhmhnPOynKf\nfnoYHYH1zrkk51weMB649hj1hgEfATuPs5+ewEbn3LYiZWV6MCIiUn78JIxGQHKR5W3BskJm1hAY\n6Jx7leMngZuAD4qV3WtmK8zsNTOL9RmziIhUgLAy2s8YoOjcxhFJw8zCgQHA8CLFrwCPO+ecmT0B\nPA/87lg7HzlyZOH7hIQEEhISyiRoEZFfisTERBITE8v1M/zMYXQGRjrn+gaXhwPOOTeqSJ1Nh98C\ndYCDwJ3OuanB9QOAPx7exzE+oynwiXPuwmOs0xyGiMhJKo85DD89jMVAy+Af9R3AYGBI0QrOufgi\nQb6J98d/apEqQyg2HGVm9Z1zqcHFQcDKkw9fREROlxIThnOuwMzuBWbgzXm87pxbbWZ3eavd2OKb\nFF0ws2i8Ce87i9UbbWYXAwFgC3BX6Q5BREROhxKHpCqahqRERE5eRZ1WKyIiooQhIiL+KGGIiIgv\nZXUdhojIEQ4ePMiUKVPIysqiV69eNG3atKJDklOkSW8RKXMZGRl0ad8eUlOp6hwbQ0KYMXcuHTp0\nqOjQzhgVdR2GiMhJeenFF6mSnMyA3FwMWAEMu+suvl261Pc+0tLSmDlzJuHh4Vx11VXExMSUW7zi\njxKGiJS57cnJ1A0mC4AGwPdpab63X7t2Ld06d6Zhfj55wCO1a7Ng6VJq165dHuGKT5r0FpEylZub\nS4/evfkhOpp9QC7wbZUqXNmzp+99/L9776V9RgbXHTjAjQcOUDslhaeffLLcYhZ/1MMQOYPMmTOH\nr7/+mgYNGnDbbbcRGRlZ4jbOOWbNmsWWLVto164d7du3P2a93bt386trrmH+okVEhIfTq1cvxs6c\nSV5+Plf37Mk/X37Zd5wp27fTtsjcZb28PLYlJfneXsqHEobIGeLFF17g73/5C+dkZ7MrKoq3//Mf\nEufPJzw8/LjbOOf43W23MX3SJBo7xwbgidGjufuee46qe+vgweQtXcpfAgH2HTrEB3Pm8OkXX3D5\n5ZcTGhp6UrEm9OjBrC1baJCdTR7wXXQ0D/fufZJHLGVNZ0mJnAECgQAxUVH8PjeXWng3cHs/JoZn\n332XgQMHHne7xYsXc/WVV3LHwYNE4N2JdAZAWBiXd+nC+EmTqFOnDgCxVatyZ1YWh6emZ4eE0Pux\nx3jkkUdOOt6cnBxuHTKEyZ98gplx3733Mvr55zHTM9f80llSIlIqe/bs4VBuLoefUhYC1HCOffv2\nHVV3+fLlfPLJJ8TExFCvXj3iQkOJANKAucBvgHr5+SQuWMBN113H7K+/BiCuTh1Stm6lNV5C2lWl\nCg0aNChVvFWqVOHDjz8mLy+PkJCQk+6hSPlQD0PkDDDk+uuZNmkSrZzjciAF+CQyku9Xr6Z58+aF\n9aZPn85NgwZxQU4OWeHhpMXGknngAIOyskjDSxqHn8+cB4wKDSU3Lw8zY86cOQy65hpamrHPjHrn\nnMPcb74hIiLitB+vlE8PQwlD5AzQoE4drtmzh6+BrXhDC80vuIDeffrQt18/unfvDsCFbdpw/rp1\ntAlu92l4OOcNGcIXn3zCrr17aWDGHc4Rgpd0JsbGsjvYS3HOMXHiRL788ksuuOACbr/99hPOj0j5\n0pCUiJRKvbg49u3Zw2C8B9aMA3b/+CNLfviBN155hWdefJHbhw4lMzOTmkW2q56XR+1atUhLT+fg\nwYP07d6dD1atok5+PqtDQnjl1VcBL1ncOXQon0yYQL3QUN7Nz6dhw4ZcffXVFXC0Ul7UwxA5A8yb\nN4+r+/ShJbAnP5/dubkMc44IvJ7ClFq1SN2zh2F3383st9+mT3Y2+4HJ0dFM+PRTrrzySsC7xmLC\nhAns2rWLbt26FZ5iO3fuXG6+5hp+e/AgkUAyMDEmhvTMTE1UVxD1MESkVC677DKW/fADc+bMYe7c\nuWz473+JyM8HoDqQlZ0NwLNjxnB/bi7vTZxI1ehoxowaVZgsACIiIrj55puP2v/WrVtpaMbhqzoa\nAwezs8nKyqJq1arlfHRyuqiHIXKGWblyJV07daJ/VhZ1gMQqVbhwwADe++9/T7hdXl4ejz78MJ9N\nnkytOnUYPWYMnTp1AuCHH37g8k6duCU7mzrAEmBt06as27KlvA9HjkOT3iICeNdVhISU/s4+s2bN\n4k/33MPevXvp278/L776KlFRUSfc5s6hQ/ly/Hi6ZWezB0isWpVFy5fTqlUrAN54/XWG3XMPYWbU\nql2bz2fO5Jxzzil1jHJqlDBEfqEWLlzIO+PeISI8gj/8/g+0adPmmPXWrl3L9b++nh9X/EjDpg0Z\n9/Y4unXr5uszDh06RGpqKvXr1/d1S5DDnHM8+vDDjHr6ae7DG8ICmBYezvVPPcWf//znIz5j3759\n1K1b95QSmpw6PdNb5Bdo9uzZ9LmmD2sa/MjSqCV06daF1atXF65PSUmhR78eVK9Vnbad2tL49oYM\nz3mAri9dyrW/GkCaj7vATp8+nXp16tD+3HOpX6cOM2bM8B3f5MmTefPFF6kC5BQpzw0NPSrxREZG\nUq9ePSWLXyj1MEQq2BW9L6f20FqcP/g8AKYPm0GNjbV45cVXiI+Pp22ntsT2rkarga3474AP+VPK\n/YXbTug5iTEPjKFPnz7H3X96ejotmjThuoMHaQokAR9Xrcqm5GRq1qx53O0OG/7QQywaPZpIYBHQ\nBdgJbI+L47tVqwpvDSKVi3oYIr9AWdnZRNeJBmDe0/NZOX4V67PW075Le15+5WU2rN/A5X/vRu3W\ntcjdn0fmtkwAcg/msmfDburWrXvC/W/YsIEaoaEcfkBqU6BGaCgbNmzwFV+z5s3ZER3NJUBP4Edg\nW+3aLP3uOyWLM4xOqxWpYLcOvpVRf3qarL9m8+1zC/nDD78npn4MKctSeeCKByjILeDAjgNUa1iN\nbn+7jP+0e4Nzrz2HHQtS6XNlX9LS0pg9ezbt27dnz549NGjQgOhoLwHl5uZSvXp19uTmsheoCewF\n9uTm0rhxY1/xDR06lAnvv89bK1YQGxLCfjNmzppF/fr1y61NpHLSkJRIBUpJSWHcuHHM/XIuy79f\nhsWFcOWTV7B59ma+e/sHwqPCObT3EGERoVw49ELS5qdRu6A2N990C9WrV+dvf/8bkY0iyUjJIH3j\nTiKAPIyRTzyBC3H8/fG/YyFG/bg40nfspHZ+Pin5+cTVr8/EqVO55JJLfMVZUFDAV199RWZmJl26\ndCEuLq58G0ZOmc6SEvkF2bx5M50u60TTq5pw6EAOG6Zt8k6XDTPCosLp+shldPhDO3Iycni3yzi6\nt+3O5Zdfzuatm1m/eT0b1m+gZv8aXPI/7Xn5rJcYcjCPJnhXWb8NVGtUk9sW3UJU7Sim/uZTkj7d\nRINDuXQJBDgAfB4eTkhBAfmBALVq16ZXr17cdMstLFm0iLkzZtC4SROefOYZGjduzLOjRjHunXeo\nWrUqj48aRY8ePSq28aREutJb5BfkqWeeonH/RqT8kMKulbvp88/eNOrUgHlPfcvqiWs4f/C5AFSJ\nrUJ8/+acV+s8Xn/nNfKa5dG0XxM2f7mZi3tcy96N+4gNDaFJcL9nATXCQ4g8K4rI2Eje6/w2uevT\nqZqdx26gFhANuLw8zgF+AFru2cPu8eMZPGECMWZ0z89n++LFdJ4zh25XXsmsyZOpm5dHKjCgd2/+\nO2WK7hN1BtKkt0gFSd6ezOqJq6l3cX1a9m9B299dRNz5cVzzxlWERoTy4wTv1NpDmYdImr6V0NBQ\ndmTu4Op3+3Phby6g9bWtWPTCEmIaxLAvN5/04H7TgYwCR256Lt8+PZ/qq3dz18E87gYuwnsA0gag\nAd4Edju8yexOwKCCAlx+Pq2BbgUFZKenM2fCBFxeHgVAH6BDIMAtN91ERkbGaW0vqXi+EoaZ9TWz\nNWa2zsweOkG9S8wsz8wGBZdbm9lyM1sW/DfDzO4LrqtpZjPMbK2ZTTez2OPtV+TnLi0tjaSkJAKB\nQGFZo7hG1G9bn8ytGaQs2sG2BdsAyN6TTUFuAYmPfsWL8a8wpvGL7EvZR+rOVMKjI7AQb5Sh+1MJ\nbJm1hdfOf538AvgX8EZkKP8ONeo2rEeTWk1Z/MxC4g8VFP6itwB2AN+ZsRNoCRS9vrsKUBB8vwqI\nDgQYCuwHBgPxQHcgNieHuXPnlk9jSaVVYsIwsxDgJbwvF+cBQ8zs7OPUexqYfrjMObfOOdfWOdcO\naA8cBCYFVw8HZjnn2gBzgL+c4rGIVDoFBQXc+rtbaXF2C9p2vpjOl3cmPd3rCzQ5qwkpi3fQLKEp\nlwzrwPt9xjPh+omMveg1wqLCOJR5iCqxVfjNnJu5ee4Q3v3oXXJ25JD4ly9J+jKJOf8zl3Zt27Fp\n7WZ2pu3i9Xff5dJBN/DAX/7K/G8WsG3bNhp1b8aKyFBy8Z6Ct9jgANC0bVsCeA9BWoCXHLYAHwBZ\neMnna6BRsDzAT4nEAbmBAC+88AI7duw4XU0plUCJk95m1hkY4ZzrF1weDjjn3Khi9e4HcoFLgE+d\nc5OKre8NPOqc6xZcXgNc4ZxLM7P6QKJz7liJSJPe8rP18qsv8/y457lh2iDCo8KZOWw28QdbMO6t\ncQwaPIjsbge55J4OACx/fQUz/zybXs/2IKZhDDPun0mb61rTc7Q3wTzv6fm0TGpNxsEM1m9cT/uL\n2/PsU89SvXr1oz73gw8+4MnxT3LtpKv57DdT+XHCGlx+gFggFIjAG4pai3cRXiiwD4gBrsG7onsq\nEBYRQcvcXPYE67QDNgNrgOZm7IqNZcHSpcTHx5dnM0opVNSFe43wTrw4bFuwrGhgDYGBzrlXgeMF\neBPeF5jD4pxzaQDOuVRA5+nJL86S5Uto8+tWRFT1hpIu+N35LFu+lLS0NJYuX0pkbJXCupGxkVRv\nXI22d1xMq/4tue6DgayesKZw/d7V+2japCnj3xnP0nlLGfvy2GMmC4CwsDAO7DrAv1q8wsqP1hAe\n5g0l3QpkBP9tC9wIEBLCWeefTxgwAG/Y6VzgCqBxkyasNCMWbzL9c7weR11gl3OcnZHBiIcfLuNW\nk8qqrM6SGgMUnds4ImmYWTjez+LwE+zjuN2IkSNHFr5PSEggISGhNDGKnHat4luxeMZi2t3ZlpDQ\nEDZN20zzZvFc0fNyos6OYtYDs4mqVYWQ0BCmD5tB/Xb1CrctOJRPdno2Yxq9QCDfER0azd1j7vb1\nuR07dmTHt1uIBuoDO/O8X/YAXk8hNFgvBKgRE0Pzli3ZvHIlh4rsIwdIT0oipkoVtoeFsXX/furi\nXfwXj/ctcpdzRKWmnlojSZlITEwkMTGxXD/D75DUSOdc3+DyUUNSZrbp8FugDt5cxZ3OuanB9QOA\nPx7eR7BsNZBQZEhqrnPuqHsha0hKfk4CgQA7duygRo0aVK1alZycHLp178bGpA1ExkRyaG8ubdq0\nYeWGlTTr0ZQtc5Oo1iAGB2Qk7SPvQB5dH+lK9cbVmPNwIjn7crh52mDCosL54rfTeWTYI9x917GT\nxrJly7jlxhvZvHUrsdWqUZCezhCgNjAfb07iBryzpOKADsDGsDCSGzTggvPPJ+2LL1gJdAOygYXA\nULzk8lpEBPm5ubQPbvst3i/69pAQHnv2We7/05/Ks1mlFCpqSGox0NLMmppZBN7JElOLVnDOxQdf\nzYGP8JJD0TpDOHI4iuA+fht8fxswpRTxi1QaGzdupM35rTmv3XnUqVeH58Y8x7x581i/fj2NuzQm\njzyyDh0k0D6f7qMS2DhtI+cNOZcbPv4Vv18ylL4v9sHCQkhZksLGaZvo/XxP6rSpTfbeHBp2aEC3\npy9j/KTxx/zsffv20fvKK4nfuJGb8vLITE+nAHgNmA20IjjpHR9PXOvW1O7alaVnn03dq65iZmIi\n3fv0YWd0NFcD64FlwO14Q0+1gEgzmoaE0AdvKGtwsF6D+HguvPji8m5aqSRKHJJyzhWY2b14X0xC\ngNedc6vN7C5vtRtbfJOiC2YWjXea953F6o0CPjSzoXg30LyxlMcgUinccPMNxN/RnCH/7yYykjN5\n6rKniAiJ4Kr3+9GidzyB/ABvdHmLOufWZcGzCwmvGsHiF5aw5MWl1L2gLpnJmYRHhbNldhJdHujM\neTedy+qP1pCz17up+IGUAxw8cJD8/HzCwn761c3Pz+eG664jIzOTWXi/pFcB5+Od8fQfYDfQpEkT\nfty4sXC7r776iqt79eKjKVMIB6pVr86HwXURoaFkF3jnRa0EAqGh3nMwDnmDVuF4CahGcjI3XX01\n9z34II+MGFFubSuVg24NIlIGnHOER4Tz4IE/Exbp/TGfec9slr25nPt33EuV4OT2F/dOZ/ePuwmN\nCGXvpr3cmngLMfVj+OKeaaQs2cEdi4dyIO0Ab3Z5h7a/v5hvnpjH+UPOIzoumiUvL6FO87q0qteK\nzz7+DIDo6GhGPfUUrz/+ODfk5GDA/wF/46fhg0nA9thYVq5ZU3jDwEAgQLXISC7Nz+dSvPmIcUA/\nYHlICPurVyc/N5e8vDxq1azJv994g1uHDKHzgQPUdY5ZeHMj1+Bdo/FqZCRbt2+ndu3ap6W9pWS6\nvblIJWVmNGzSkC1zkgDIy85j+/wU2pzbhgVPL8QFHOkb0lk5bhU7V+4ktEooF9xyPtUaVsNCjEuH\nX8qBHQcAiKkXQ3yvZsx7ch6hIaGsGvcjB9MO8tt5t3Lbklv4bs13xNaMpWbtmlw1sD/fzJ3LBTk5\nROB986+Bd10FeD2M1Kgo/jtx4hF3l01KSuJQfj7ReKfJNsGbyJ4FVA0EaLFvH4dycujRuzdzv/mG\nq666isR583Ddu7OoeXMORUbSP7ivakC18HD27NlT7u0sFUv3khIpI+++8S4DbxhIow4N2b1uN90v\n68GzTz1Lz/49efK5UYSGh9Lz2e6sHLeKtBU7KcgpwAUcFmJsm7+NiGre0+tyD+aybUEKA9+/lk3T\nN7PsteX0/1dfQsNDWf76CsJjw/mf5cOIiIng899+gdscICIiggtyczGgOfB5aChLqlYlPTeXe+69\n96ibBc6ePZtQvGSRCjTEuwK8Hj+NDbcIBJj22Wd0/uorFi1fzgUXXMDns2axZ88e2sTHs+7QIVrh\nXTUeXrUqzZo1Oy3tLBVHQ1IiZSglJYWlS5dSt25dOnXqhJnx5JNP8smuqfR49kpCQkPYn7qfl1q8\nSkiIUa1xdWo0jWXrvGTMjJrxNcjanU2rq1rQ/1/9WPTPxXzz+HwuHnoRlz/elYmDJ9Pqqha0v6sd\nANsXpTDv998SnuPI2rGDMOBQtWrMmDuXzMxM4uLiaNKkyREx5uXlUaNaNX576BBxeFd7vwgcAi7F\nu/4CYBfeMNX5ZrQfNox//POfhftYsGABv77+erbu2MHZLVowYcoUzjnnqJMcpQLpbrUilVzDhg1p\n2LAh8+bNo12XduzauYsmjZqwJzwdM+93N3VZGg0aNCBzXybRsdHsXuMN5TTq1JCt3yRTu1UtujzY\nmZTFO/jm/+ZzVoOziPghkudqjiEyOpLo6lG0u7MtZsb2b1No1rQpkz+cwldffUVBQQFdu3alWrVq\nx40xMzMTV1BQeKVsONAoIoK4Ll1Y8s03tCoooBre8FRLIMo5sg8ePGIfnTt3ZtO2bTjnCo9LfvnU\nwxDfnHN8+umnJCUl0aFDBzp37ly4Ljk5ma+++orq1avTp08fIiIiKjDSirVx40bad25Pz1e6Exkb\nyfy/f0vmhv3ENo2lZsuabJy+kUn/nUSbNm0YPnw4i3YvpOeY7uxctYsv7plGy74t2DRzC+FRYVRv\nFkv/1v15+eWXcc6xf/9+Lr2iC4eq5VKlRhXSlqbx9ZyvadOmja/YAoEANw0axKdTp5LgHJ2A7cCE\n6GiWfv89iYmJ/Pm++ziYlUU83t1tZ0ZFMWXaNC6//PJybDUpa+XRw8A5V6lfXohS0QKBgPvVrwa7\nmJgmrkqVzi46urb7xz/+6ZxzbsGCBS4mpqaLiWnrYmLiXbt2nV12dnYFR1xx/vWvf7kOt7d3V/27\nn6saF+1a9I13EdUj3O133O7eeustt3HjxsK6U6dOdY0vbOyGH3zAPeoedtUaVXO/X/4796h72D3q\nHnYd/3iJe/LJJ4/Yf1ZWlpsyZYr78MMP3c6dO08qtqlTp7qzqlZ1fwRXD1wIuDBwU6ZMKawTCATc\nqKeecue3bOk6XHCBmzp16hHr9u/f7wKBQClbR06X4N/OMv17rB7GGWjnzp08+ODDbNyYxJVXXsaj\nj/6V8PDwE24zf/58eva8luzsXngnVAaIiBhLRkY67dp1ZvXqVnhn/geIivqIZ575I/fcc89pOJrK\n5/7772fSwonsXruHO5beTs34mmQkZ/LmxW/z43c/HvEsbecc3Xp0Y8WqFcQ2i2Xn9zupeVZNLhvR\nhYzNGXz/0kqWLVp21DxEab3yyiu887//S7/sbMC7/ccoIC8/n9DQ0BNu++233zJowADS9+2jVmws\nE6dO5dJLLy2TuKTsaQ5DTtnBgwfp0KELqakNyMs7i6VLP2TVqtVMnPjfY9afM2cOo0ePYcWKZWRn\nH8C7GXYqMICQkHAyMjJITU0FEoJbhJCdHUdy8raj9pWRkUFUVNQverhq8uTJvPfRe+SF5hJVJ4qa\n8TUBiD2rOnVa1CE5OfmIhLF8+XJW/biKPi/2IiImgq2JW9kxJQ031WhSvRn//vo/ZZYswLvH1CNm\ndMS7gntZSAgXnX12ickiMzOTa/r2pXdmJm2AtXv2MKBfPzYlJx/3Bojyy6PrMM4ws2fPZu/eUPLy\negFnk509iE8+mUJmZuZRdRMTE+nXbwDTpy8jLW033p2IbgN+A0wiKiqCQCDApZdeSnj4ArwnLXxB\nSMgC6tf1eh2wAAATDElEQVSvR3bwW2xqaioXXdSBunXrExNTndGjnzltx3s6BAIBnnjqCVqe15Lb\n776d+OuaMfTb35K1K4stc7cAkDwvmfRN6bRu3fqIbRcuXEjrAa0478ZzadW/JQn/dwVJG5OY9MEk\n3vrPW7Rq1apUMS1btoy/DB/OY489xrZtPyXvDh068PTzz/NaRATPRESwtVkzJn36aYn7W7duHTHA\n4ZmSNnjXX6xdu7ZU8cnPk4akzhD79+/nxhtvZvr0z/F6qV2BLkAI4eHPsnv3zqO+KXbu3JWFC5fh\n9R5y8W459xu8h3s+jfdInVDq1atHlSphJCUl431vjQB2YBagR48+ZGdnsXBhAfn53YFMoqPfZ8qU\n9+nZs+dpOvry9czzz/DS+y/Se2xPDu3P5eNfT+Hq//QnrEooE66fRGhoKOGE88G7H9CvX78jtp08\neTL/88T93Dx/CKERoSTPS+azG6exc/vOUsczd+5cBl19NRdlZXEoNJSN1aqxeMUKmjZtWlgnLy+P\nzMxMatWq5essp23btnFuq1bcmZNDNbyvBv+uUoVV69Zx1llnlTpWKT8akpJSGzr0TmbM2IxzDwGZ\nwBvAl5hFc8UV3Y9KFtnZ2SxfvhK4Gu9Bi4ctwft+WQBEA1eSlrYNWIF3+dfteDct/hrnljF79hac\nWw/8OlgeS05OGxYuXPiLSRgffPQBVzx3OQ3aNwCg26OXMX/0t5x307lEhkXy2iuvMWDAACIjI4/a\ndsCAAbz53hu8e8n71D2vLhtnbuK9N987pXgeeeABemZlcT5AQQGzMjMZ89xz/OOFFwrrhIeHn9Rt\nPBo3bsyDw4fzz9GjaRoSQlIgwAMPPKBkcYZRwjhDTJs2g0DgFrxv/3WAjng9hi7HnG+47robyc0l\nWP+wKng3nfge7ybXbfHO1m+Ad2OJzXiXe8UBTYEFONcV705FHwK/ApoTFZVKREQE+/bto0aNGuVw\ntKdXTNWYwtt6ABxIOUjknkiqfRvLZx9/RteuXY+7bUhICB9/OJkZM2awc+dOujzWpdTDUIWfv38/\nRdN/tUCAzIyMU9onwCMjRtC7Xz9Wr17N2WefTadOnU55n/LzoiGpM0S9emexc+dlwDl4NxT+EFgN\nPILZk+TkZBdORqenp1O/fmPy8nrgJZV+eNcDT8ZLGgZE4X3faAL0Cn7Kl3jJ5G68u9xn4CWVdXgJ\najJRUTXIz88kPLwK+flZPP74SB566IHT0ALlJzExkYE3DuTiey4kd38e695bx4JvFtKyZcsKiefv\njz3Gm6NH0zcrixzgk+ho3vnoo6OGw+SXrTyGpJQwzhD//ve/+cMfhuGd+poBpOP98b+aatUmkpGx\np3Ase9++fdSqFYdz/4t3c+vvgvVzgRZ4T+k9gNf7GMBPQ1brgIl4ySUC7/Tb3cDNeMlkXbBeY7zJ\n8wNER7/LrFlT6dKlywnjz8zMJD09ncaNGx9xa+/KYunSpXzw4QdERkRyx9A7aN68eYXFUlBQwIhH\nHuG9t94iIjKSRx57jFtvu63C4pGKoYQhp+Sxxx7n739/gkAggHNhVK3akkBgG+PHv8OAAQMK66Wk\npHDWWS0IBBoA7fGe4rwV7yypV/FOrjsHbwgqBrglWDYO7xZ2NwBpeL2TXwXrpeDNYwCMB5oBlxMV\n9QXPPXcbd999/EePjh79LI8++jfCwqKpUSOGOXOm+76yWeRMpYQhpyw3N5eMjAy+++470tLS6Nix\n41Fj5hs2bODCCzuRnR2HNyfRCrgcLyk8jzdE1RbvQZ5jgHy8Ya44vB7IMrxnsqUD0/CGsK7jp5My\nVwPLgV9RterbTJr0Jr179z5mvPPnz6dXr2vJyvoNEIvZYlq23MK6dSvLqklEfpF0lpScsoiICOrW\nrXvCM5SaN29Os2ZnsX59GPn5SXiJYC8wm5CQMAKBesGaUXhPgJ6Ld5/Tw/uMw5vDMKA63iPek/gp\nYWwhPHw34eFjGTz4Bnr1OjwHcrTly5cTCLQEYgFwrh0bNnxBIBAgJESXEYmcTkoYcpTQ0FC+/HIm\nd9xxNwsWpJGd/S1VqlQhIaErsbGxvP76LJz7FV4PYwkQCtQssofqeD2Ozng9iVy8uZBUvNNxUxg7\ndizt2rXjwgsvPGEs8fHxhIYmB/cRAWwiLq6hkoVIBdCQlBxh165dJCUl0axZM+rUqXPU+pycHIYM\nuZUpUz7G+28JwXvGWw5wPd4f9Yl49zntCmwCPsV7pPtmAMxmsnjxDNq3b19iPM45brvtd0yc+Cnh\n4XUpKNjBZ59N1p1TRUqgOQwpV+PGjeOOO/5AeHgt8vL28tZbr3HjjTccs65zjsaNW5CSkgQ8hHd9\nxrfAAcLCwsjP/xNeMtkHvAwMw+t5ZBIW9ipJSRto2LChr7iccyxbtoydO3fStm3bIx41KiLHpoQh\n5SYtLY3mzVuTnX0z3oM6dxAVNY5t27ZQq1atY27TtGlrtm7dCDyId4ouwBvExh5g//4OBAKtCAv7\ngaioVRw8mA00Jiwslcce+yvDhz94Wo5L5ExVHglDA8ECwObNm4mIqI2XLAAaEB5eky1bthx3mz//\neRhmUXin0/4ITCM2NpvExNl06ZJPvXpTufLKaqxd+wNLlnzD++8/xsKFXypZiPxMqYchQNEexi14\nZzmlEhX1PsnJm497zyHnHM899w+efHI0OTl5dOzYlvHj39OQkUgloCEpKVfvvfced975R8LDa5OX\nl84bb4xl8OCbKjosESkFJQwpd2lpaWzZsoX4+Hjq1q1b0eGISCkpYYiIiC+a9BYRkQqjhCEiIr74\nShhm1tfM1pjZOjN76AT1LjGzPDMbVKQs1swmmNlqM1tlZp2C5SPMbJuZLQu++p764YiISHkp8V5S\nZhYCvAT0wLtH9WIzm+KcW3OMek8D04vt4p/A5865G8wsDO+5noc975x7/lQOQERETg8/PYyOwHrn\nXJJzLg/vYQbXHqPeMLxblBY+vd7MqgPdnHNvAjjn8p1zmUW2KdMJGRERKT9+EkYjvIcyH7YtWFbI\nzBoCA51zr3JkEmgO7DazN4PDTmPNuzT4sHvNbIWZvWZmsaU8BhEROQ3K6vbmY/DuQHes/bcD7nHO\nLTGzMcBwYATwCvC4c86Z2RN4T+b53bF2PnLkyML3CQkJJCQklFHYIiK/DImJiSQmJpbrZ5R4HYaZ\ndQZGOuf6BpeHA845N6pInU2H3wJ18J6YcyewEPjWORcfrNcVeMg5d02xz2gKfOKcO+rhCLoOQ0Tk\n5FXUE/cWAy2Df9R34D17c0jRCocTQjDIN/H++E8NLiebWWvn3Dq8ifMfg+X1nXOpwc0G4T1hR0RE\nKqkSE4ZzrsDM7gVm4M15vO6cW21md3mr3djimxRbvg9438zC8Z6mc3uwfLSZXQwEgC3AXaU/DBER\nKW+6NYiIyC+Qbg0iIiIVRglDRER8UcIQERFflDBERMQXJQwREfFFCUNERHxRwhAREV+UMERExBcl\nDBER8UUJQ0REfFHCEBERX5QwRETEFyUMERHxRQlDRER8UcIQERFflDBERMQXJQwREfFFCUNERHxR\nwhAREV+UMERExBclDBER8UUJQ0REfFHCEBERX5QwRETEFyUMERHxRQlDRER8UcIQERFflDBERMQX\nXwnDzPqa2RozW2dmD52g3iVmlmdmg4qUxZrZBDNbbWarzKxTsLymmc0ws7VmNt3MYk/9cEREpLyU\nmDDMLAR4CegDnAcMMbOzj1PvaWB6sVX/BD53zp0DXASsDpYPB2Y559oAc4C/lPYgRESk/PnpYXQE\n1jvnkpxzecB44Npj1BsGfATsPFxgZtWBbs65NwGcc/nOuczg6muBt4Pv3wYGlu4QRETkdPCTMBoB\nyUWWtwXLCplZQ2Cgc+5VwIqsag7sNrM3zWyZmY01s6jgujjnXBqAcy4ViCvtQYiISPkrq0nvMcCx\n5jbCgHbAy865dkAW3lAUHJlYAFwZxSIiIuUgzEed7UCTIsuNg2VFdQDGm5kBdYB+ZpYPLASSnXNL\ngvU+4qfEkmpm9ZxzaWZWnyJDWcWNHDmy8H1CQgIJCQk+whYROXMkJiaSmJhYrp9hzp34i72ZhQJr\ngR7ADmARMMQ5t/o49d8EPnHOTQoufwn83jm3zsxGANHOuYfMbBSQ7pwbFTzzqqZzbvgx9udKilFE\nRI5kZjjnio/knJISexjOuQIzuxeYgTeE9bpzbrWZ3eWtdmOLb1Js+T7gfTMLBzYBtwfLRwEfmtlQ\nIAm48RSOQ0REylmJPYyKph6GiMjJK48ehq70FhERX5QwRETEFyUMERHxRQlDRER8UcIQERFflDBE\nRMQXJQwREfFFCUNERHxRwhAREV+UMERExBclDBER8UUJQ0REfFHCEBERX5QwRETEFyUMERHxRQlD\nRER8UcIQERFflDBERMQXJQwREfFFCUNERHxRwhAREV+UMERExBclDBER8UUJQ0REfFHCEBERX5Qw\nRETEFyUMERHxRQlDRER88ZUwzKyvma0xs3Vm9tAJ6l1iZnlmNqhI2RYz+87MlpvZoiLlI8xsm5kt\nC776ntqhiIhIeSoxYZhZCPAS0Ac4DxhiZmcfp97TwPRiqwJAgnOurXOuY7F1zzvn2gVf00p1BJVE\nYmJiRYfgi+IsOz+HGEFxlrWfS5zlwU8PoyOw3jmX5JzLA8YD1x6j3jDgI2BnsXI7weeY30Aru5/L\nD5HiLDs/hxhBcZa1n0uc5cFPwmgEJBdZ3hYsK2RmDYGBzrlXOToJOGCmmS02s98XW3evma0ws9fM\nLPYkYxcRkdOorCa9xwBF5zaKJo3LnHPtgP7APWbWNVj+ChDvnLsYSAWeL6NYRESkHJhz7sQVzDoD\nI51zfYPLwwHnnBtVpM6mw2+BOsBB4E7n3NRi+xoB7HfOPV+svCnwiXPuwmN8/okDFBGRY3LOlemw\nf5iPOouBlsE/6juAwcCQYkHFH35vZm/i/fGfambRQIhz7oCZVQV6A48F69V3zqUGNxsErDzWh5f1\nAYuISOmUmDCccwVmdi8wA28I63Xn3Gozu8tb7cYW36TI+3rAx8FeQhjwvnNuRnDdaDO7GO8sqi3A\nXad2KCIiUp5KHJISERGB03yld0kXAJrZFWa2r8jFfI8UWXe/mf0QfN1XpLymmc0ws7VmNr0szrYq\nwzjvL1Je5hcq+rmg0swSghdNrjSzuSVtWxHtWco4K1t7vm5maWb2fbH6Zdqe5RRjpWlLM2tsZnPM\nbFVl+F0vZZyVqT0jzWxhsPwH8+aRD9c/+fZ0zp2WF15y2gA0BcKBFcDZxepcAUw9xrbnAd8DkUAo\nMBPvDCuAUcCDwfcPAU9X0jhHAP/vNLdnLLAKaBRcrlPSthXUnqWJs9K0Z/B9V+Bi4Pti25RZe5Zj\njJWmLYH6wMXB9zHA2kr6s3miOCtNewbfRwf/DQUWAB1L256ns4fh9wLAY01ynwMsdM4dcs4VAF/i\nTZQT3MfbwfdvAwMraZzH26Y84/w1MNE5tx3AObfbx7YV0Z6liRMqT3vinPsG2HuM/ZZle5ZXjFBJ\n2tI5l+qcWxF8fwBYzU/XfVWan80S4oRK0p7B91nBt5F4c8mH5yFOuj1PZ8Io8QLAoC7mXcz3mZmd\nGyxbCXQLdqGi8a7pOCu4rp5zLg28/0QgrpLGCWV7oaKfOFsDtcxsrnkXTv7Gx7YV0Z6liRMqT3ue\nSFwZtmd5xQiVsC3NrBlej2hBsKgs27I84lxYpLjStKeZhZjZcrzr3WY65xYHV510e1a2u9UuBZo4\n72K+l4DJAM65NXjdp5nA58ByoOA4+zgds/ilibMiLlQMA9oB/YC+wKNm1vIk93E62rM0cao9j+0X\n0ZZmFoN3q6H7nXMHj7OPCv/ZLBbngWBxpWpP51zAOdcWaAx0KvIFt7gS2/N0JoztQJMiy42DZYWc\ncwcOd5+cc18A4WZWK7j8pnOug3MuAdgHrAtulmpm9cC7toOj72VVKeJ0zu1ywcFC4D/AJeUdJ943\nkenOuRzn3B7gK+CiErY97e1ZmjgrWXueSFoZtme5xFjZ2tLMwvD+CL/rnJtSZJuybMtyi7OytWeR\nuDKBuXgJBUrTniVNcpTVC2/C5fDETQTexM05xerUK/K+I7ClyHLd4L9NgB+B6u6niZuHXNlNhJVX\nnPWL1PkTMO40xHk2Xm8nFIgGfgDOPdG2FdSepYmz0rRnkfXNgB+KbVNm7VmOMVaqtgTewbuTdfH9\nVpqfzRLirDTtiXfnjdhgnSi8RNK/tO1Z6oMo5YH3xTubYD0wPFh2F95tRADuwZsHWA7MBzoV2far\nIusSipTXAmYF9zsDqFFJ43wH7wyqFXhDWPXKO87g8v/inT3xPTDsRNtWVHuWMs7K1p7jgBTgELAV\nuL082rOcYqw0bQlchjeMuyL4O7QM6FvZfjZLiLMytecFwdhWBMv/WqT+SbenLtwTERFfKtukt4iI\nVFJKGCIi4osShoiI+KKEISIivihhiIiIL0oYIiLiixKGiIj4ooQhIiK+/H83k4ngRkSJwQAAAABJ\nRU5ErkJggg==\n", 728 | "text/plain": [ 729 | "" 730 | ] 731 | }, 732 | "metadata": {}, 733 | "output_type": "display_data" 734 | } 735 | ], 736 | "source": [ 737 | "# Get encoded coordinates\n", 738 | "ec1 = np.array(layers[1]['activations'][0, :].transpose())\n", 739 | "ec2 = np.array(layers[1]['activations'][1, :].transpose())\n", 740 | "\n", 741 | "# Plot encoded data\n", 742 | "plt.scatter(ec1, ec2, c=labels)\n", 743 | "plt.show()" 744 | ] 745 | }, 746 | { 747 | "cell_type": "markdown", 748 | "metadata": {}, 749 | "source": [ 750 | "Perhaps somewhat surprisingly, this generally seems to perform a bit worse than the linear autoencoder." 751 | ] 752 | }, 753 | { 754 | "cell_type": "markdown", 755 | "metadata": {}, 756 | "source": [ 757 | "### ReLU autoencoder" 758 | ] 759 | }, 760 | { 761 | "cell_type": "markdown", 762 | "metadata": {}, 763 | "source": [ 764 | "And finally the ReLU activation function:" 765 | ] 766 | }, 767 | { 768 | "cell_type": "code", 769 | "execution_count": 220, 770 | "metadata": { 771 | "collapsed": false 772 | }, 773 | "outputs": [ 774 | { 775 | "data": { 776 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYEAAAEACAYAAABVtcpZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGHtJREFUeJzt3X2QHPV95/H3Z/WAHhCSwEKCxYCDSsgIk0OKZQFxNFgE\n5AcschXHUohDbB9WnFMO313uJFKpaKlKxaHKxyUx3MWKCbZTHCoSMBLmAsJlJo4cU6yNEALvSuJB\nQk8W4YRsYR4sVt/7o3ul8bCr6d2dnemZ/ryquma6+9fdv2ltzUe/36+7RxGBmZkVU0ezK2BmZs3j\nEDAzKzCHgJlZgTkEzMwKzCFgZlZgDgEzswLLFAKSlkrqlbRD0uoB1p8maaOkpyRtk/R7Wbc1M7Pm\nUa37BCR1ADuAJcB+oBtYHhG9FWVuBk6LiJslvQvYDswEjtXa1szMmidLS2AhsDMidkfEUWA9sKyq\nTABT0vdTgP8XEW9n3NbMzJokSwh0Ansq5vemyyrdDlwkaT+wFbhpCNuamVmT1Gtg+BpgS0ScDVwK\n3CHp1Drt28zMRsnYDGX2AedWzJ+TLqv0aeCLABHxvKQXgbkZtwVAkh9iZGY2RBGhkWyfpSXQDcyW\ndJ6k8cByYGNVmd3AVQCSZgJzgBcybntcRHiKYO3atU2vQx4mnwefC5+Lk0/1ULMlEBF9klYBm0hC\n486I6JG0Mlkd64A/A74m6el0s/8eEYcABtq2LjU3M7MRy9IdREQ8DFxYtewrFe8PkIwLZNrWzMzy\nwXcM51CpVGp2FXLB5+EEn4sTfC7qq+bNYo0iKfJSFzOzViCJaMDAsJmZtSmHgJlZgTkEzMwKzCFg\nZlZgDgEzswJzCJiZFZhDwMyswBwCZmYF5hAwMyswh4CZWYE5BMzMCswhYGZWYA4BM7MCcwiYmRWY\nQ8DMrMAcAmZmBeYQMDMrMIeAmVmBOQTMzArMIWBmVmAOATOzAssUApKWSuqVtEPS6gHW/5GkLZKe\nlLRN0tuSpqXrdknamq5/4mTHiRjehzAzs+FR1PjmldQB7ACWAPuBbmB5RPQOUv5jwBci4qp0/gVg\nQUS8WuM4cehQMH360D+EmVkRSSIiNJJ9ZGkJLAR2RsTuiDgKrAeWnaT8CuCeinllPA4vv5yllJmZ\n1UuWL+dOYE/F/N502TtImggsBe6rWBzAo5K6Jd14sgP9279lqI2ZmdXN2Drv71pgc0Qcrlh2RUQc\nkDSDJAx6ImLzQBt/+ctdfPvbyftSqUSpVKpz9czMWle5XKZcLtd1n1nGBBYBXRGxNJ1fA0RE3DpA\n2fuBeyNi/SD7WgsciYjbBlgXf/M3wcqVw/gUZmYF1KgxgW5gtqTzJI0HlgMbB6jMVGAxsKFi2SRJ\np6bvJwNXA88MdiB3B5mZNVbN7qCI6JO0CthEEhp3RkSPpJXJ6liXFr0OeCQi3qjYfCbwTUmRHuvu\niNg02LE8MGxm1lg1u4MaRVIsXx7cc0/tsmZm1rjuoIZxd5CZWWPlKgTcHWRm1li5CgG3BMzMGitX\nYwJjxwZvvQUduYomM7N8arsxgcmT4fDh2uXMzKw+chUCM2a4S8jMrJFyFQJnnukQMDNrpFyFwIwZ\nvkLIzKyRchUCbgmYmTVWrkLALQEzs8bKXQi4JWBm1ji5CgF3B5mZNVauQsDdQWZmjZW7EHBLwMys\ncXIVAu4OMjNrrFw9O+itt4LJk/Hzg8zMMmi7ZweNH+/nB5mZNVKuQgDcJWRm1ki5CwFfIWRm1ji5\nCwG3BMzMGid3IeCWgJlZ4+QuBM4+G/bvb3YtzMyKIXch0NkJ+/Y1uxZmZsWQKQQkLZXUK2mHpNUD\nrP8jSVskPSlpm6S3JU3Lsm21c86BvXuH/kHMzGzoaoaApA7gduAaYB6wQtLcyjIR8aWIuDQi5gM3\nA+WIOJxl22puCZiZNU6WlsBCYGdE7I6Io8B6YNlJyq8A7hnmtm4JmJk1UJYQ6AT2VMzvTZe9g6SJ\nwFLgvqFu22/69OSxEa+9lqFmZmY2ImPrvL9rgc0RMawHP3R1dQEwcSLcf3+J3/3dUv1qZmbW4srl\nMuVyua77zBIC+4BzK+bPSZcNZDknuoKGuu3xECiXk24hMzM7oVQqUSqVjs/fcsstI95nlu6gbmC2\npPMkjSf5ot9YXUjSVGAxsGGo21bzuICZWWPUbAlERJ+kVcAmktC4MyJ6JK1MVse6tOh1wCMR8Uat\nbWsd01cImZk1Rq5+T6C/Ll/+MvT2wh13NLlSZmY51na/J9DPLQEzs8bIZQh4TMDMrDFyGQJuCZiZ\nNUYuxwT6+pJ7BX72Mxg3rskVMzPLqbYdExgzJvlxmQMHml0TM7P2lssQAI8LmJk1Qm5DwOMCZmaj\nL7ch4JaAmdnoy20IuCVgZjb6chsCbgmYmY2+3IaAWwJmZqMvtyHgloCZ2ejL5c1iAG++CVOnwhtv\nQEduo8rMrHna9mYxgAkTYMoUeOWVZtfEzKx95TYEIOkS8riAmdnoyXUIdHZ6XMDMbDTlOgTcEjAz\nG125DgG3BMzMRleuQ8AtATOz0ZXrEOjshD17ml0LM7P2lesQOP982LWr2bUwM2tfub1ZDOCtt5Ib\nxo4c8S+MmZlVa+ubxQBOOQVmzYKXXmp2TczM2lOmEJC0VFKvpB2SVg9SpiRpi6RnJD1WsXyXpK3p\nuieGWsELLoDnnx/qVmZmlsXYWgUkdQC3A0uA/UC3pA0R0VtRZipwB3B1ROyT9K6KXRwDShHx6nAq\nOHs2PPccXH31cLY2M7OTydISWAjsjIjdEXEUWA8sqyrz28B9EbEPICIqn/ijjMcZkFsCZmajJ8uX\ncydQeaHm3nRZpTnA6ZIek9Qt6VMV6wJ4NF1+41Ar2N8SMDOz+qvZHTSE/cwHPgRMBr4v6fsR8Rxw\nRUQckDSDJAx6ImLzQDvp6uo6/r5UKlEqldwSMDNLlctlyuVyXfdZ8xJRSYuArohYms6vASIibq0o\nsxqYEBG3pPNfBf4pIu6r2tda4EhE3DbAcd5xiSjAa6/BmWcmr/5dATOzExp1iWg3MFvSeZLGA8uB\njVVlNgC/KmmMpEnAB4AeSZMknZpWdjJwNfDMUCp46qlw2mlw4MBQtjIzsyxqdgdFRJ+kVcAmktC4\nMyJ6JK1MVse6iOiV9AjwNNAHrIuIH0l6D/BNSZEe6+6I2DTUSvaPC3RWj0SYmdmI5PqO4X433ACL\nF8NnPtPgSpmZ5Vjb3zHcz1cImZmNjpYIAV8hZGY2OhwCZmYF1hIh0N8dlJPhCzOzttESIXD66cnr\noUPNrYeZWbtpiRCQPDhsZjYaWiIEwOMCZmajoWVCwC0BM7P6a5kQcEvAzKz+WiYE3BIwM6u/lgkB\ntwTMzOqvZULgrLPgpz+FI0eaXRMzs/bRMiHQ0QFz5sD27c2uiZlZ+2iZEAC45BJ4+ulm18LMrH20\nVAi8730OATOzemqpELjkEti2rdm1MDNrHy0XAlu3+kFyZmb10lIhMGtW8vrjHze3HmZm7aKlQkBy\nl5CZWT21VAiArxAyM6unlgsBXyFkZlY/LRcC7g4yM6sfRU4utZEUWery+utwxhnJIyTGjWtAxczM\nckoSEaGR7CNTS0DSUkm9knZIWj1ImZKkLZKekfTYULYdikmT4N3vhh07RronMzOrGQKSOoDbgWuA\necAKSXOrykwF7gA+FhEXA5/Iuu1wuEvIzKw+srQEFgI7I2J3RBwF1gPLqsr8NnBfROwDiIhXhrDt\nkHlw2MysPrKEQCewp2J+b7qs0hzgdEmPSeqW9KkhbDtkvkzUzKw+xtZxP/OBDwGTge9L+v5Qd9LV\n1XX8falUolQqDVjO3UFmVkTlcplyuVzXfda8OkjSIqArIpam82uAiIhbK8qsBiZExC3p/FeBfwL2\n1dq2Yh+Zrg4COHYMTjsN9u6FadMybWJm1nYadXVQNzBb0nmSxgPLgY1VZTYAvyppjKRJwAeAnozb\nDr3SHXDxxW4NmJmNVM0QiIg+YBWwCXgWWB8RPZJWSvpcWqYXeAR4GngcWBcRPxps23pU3OMCZmYj\n13I3i/X727+Ff/kX+MY3RrFSZmY51rCbxfLoiivge99rdi3MzFpby4bA3Lnw6qv+bQEzs5Fo2RDo\n6IDLLnNrwMxsJFo2BMBdQmZmI+UQMDMrsJa9OgiSx0rPmAGvvAITJ45SxczMcqrQVwdB8ljpiy+G\n7u5m18TMrDW1dAiAu4TMzEbCIWBmVmAtPSYAyX0CF12UjAt0tHykmZllV/gxAYBZs2D6dOjtbXZN\nzMxaT8uHACRdQv/6r82uhZlZ62mLELj8co8LmJkNR1uEwK/9GpTLkJPhDTOzltEWIfDe9yYB0FOX\nXyowMyuOtggBCT72MfjWt5pdEzOz1tIWIQBJCDz4YLNrYWbWWlr+PoF+b74JM2fCCy/AGWfUsWJm\nZjnl+wQqTJgAV14JDz/c7JqYmbWOtgkBgGuv9biAmdlQtE13EMCBAzBvHhw8COPG1aliZmY55e6g\nKmedBRdc4BvHzMyyaqsQAF8qamY2FJlCQNJSSb2SdkhaPcD6xZIOS3oynf6kYt0uSVslbZH0RD0r\nPxCHgJlZdmNrFZDUAdwOLAH2A92SNkRE9XM7vxsRHx9gF8eAUkS8OuLaZnDppXDkSPJU0blzG3FE\nM7PWlaUlsBDYGRG7I+IosB5YNkC5wQYnlPE4ddHRAddfD3fd1agjmpm1rixfzp3Anor5vemyapdJ\nekrSQ5IuqlgewKOSuiXdOIK6ZvaZz8A3vgFHjzbiaGZmratmd1BGPwTOjYjXJX0YeACYk667IiIO\nSJpBEgY9EbF5oJ10dXUdf18qlSiVSsOqzNy5MHs2PPQQXHfdsHZhZpY75XKZcrlc133WvE9A0iKg\nKyKWpvNrgIiIW0+yzYvAgog4VLV8LXAkIm4bYJsR3ydQ6Wtfg/vu8/OEzKx9Neo+gW5gtqTzJI0H\nlgMbqyoys+L9QpJwOSRpkqRT0+WTgauBZ0ZS4aw+8YnkfoF9+xpxNDOz1lQzBCKiD1gFbAKeBdZH\nRI+klZI+lxb7TUnPSNoC/CXwyXT5TGBzuvxx4MGI2FT3TzGAyZOTIPj61xtxNDOz1tRWj42o1t0N\ny5fDzp3JVUNmZu3Ej42o4Vd+JWkR/PM/N7smZmb51NYhIMHv/z7c9o5haDMzgzbvDoLkx2Zmz4YN\nG2DBgrrv3sysadwdlMGECbBmDVTcgmBmZqm2bwnAidbAAw8k4wRmZu3ALYGMJkyAm292a8DMrFoh\nWgIAb72VtAbuvx/e//5RO4yZWcO4JTAEp5yStAb+9E+bXRMzs/woTAgAfPazsGtX0howM7MCdQf1\n27wZPvlJePZZmDZt1A9nZjZq6tEdVLgQAPj856GvD9ata8jhzMxGhUNgmH7yE5g3D+6+GxYvbsgh\nzczqzgPDwzR1KtxxB9x4I7zxRrNrY2bWPIVsCfS7/noYNy75PWKNKEvNzBrPLYERWrcOnnoK/vqv\nm10TM7PmKHRLAJJLRhctSsYHlixp+OHNzIbNLYE6OP98uOeepGvoxRebXRszs8YqfAgAXHllcifx\nVVclLQMzs6IY2+wK5MUf/AEcO5ZcMvroozBnTrNrZGY2+hwCFVatSn6O8sor4eGH4X3va3aNzMxG\nl0Ogyqc/DZMmJYPEX/kK/MZvNLtGZmajp/BXBw3m8cdhxQr46EfhS19KfpPAzCxPfHXQKFq0CLZs\ngYMHk/c//GGza2RmVn+ZQkDSUkm9knZIWj3A+sWSDkt6Mp3+JOu2eTZtGtx7L9x0E1x7LfzO78Du\n3c2ulZlZ/dQMAUkdwO3ANcA8YIWkuQMU/W5EzE+nPxvitrklJeME27fDBRfA/PnwhS/Ac881u2Zm\nZiOXpSWwENgZEbsj4iiwHlg2QLmB+qWybpt7U6bALbfAtm0wcSJcfjl85CPwrW/Bz3/e7NqZmQ1P\nlhDoBPZUzO9Nl1W7TNJTkh6SdNEQt20ZZ58NX/wivPRS8uM0f/7ncNZZSWvhoYf8VFIzay31ukT0\nh8C5EfG6pA8DDwBDvt2qq6vr+PtSqUSpVKpT9epvwgS44YZk2rs3+cnKW29NgmHBguRegw9+MOk+\nmj692bU1s3ZQLpcpl8t13WfNS0QlLQK6ImJpOr8GiIi49STbvAgsIAmCTNvm7RLR4XrtteQnLB97\nDL73Pdi6FWbMgEsvhQsvTO5EnjMneWbRrFnQ4euzzGyYGvLLYpLGANuBJcAB4AlgRUT0VJSZGREH\n0/cLgXsj4vws21bsoy1CoNqxY7BzZ/LI6h07kmn79uQqo8OHk+6lzk6YORPOPDOZTj89maZPT34A\nZ8qUE9OkScmYxJgxzf5kZtZs9QiBmt1BEdEnaRWwiWQM4c6I6JG0Mlkd64DflPR54CjwBvDJk207\nkgq3mo6OpAVw4YXvXPfmm7BvH+zfn9yP8PLLyevzz8MPfgCHDsFPf5pMR44k0+uvJ+MO48cnXVL9\n0ymnJMvGj09+KGfcOBg7NnkdMyZ5P3Zs8r6jI3ntf189SSdeK6fKZfCLr5VTv4HKVa+rfj+UcoNt\nk3XdcPfXSvuoRx2ylmvksbKWa8c6LVgAl12WbR+ZjpOX/323a0tgNEQkAVI5/fznvzi9/TYcPZpM\nfX3JdPRo0jLpn+/rS/bVv6z/ff8U8c7p2LETdeh/rZwq6zjQ68neD6XcYNtkXTfc/bXSPupRh6zl\nGnmsrOXatU6//uuwLL3G0j80b2ZWYH5shJmZjYhDwMyswBwCZmYF5hAwMyswh4CZWYE5BMzMCswh\nYGZWYA4BM7MCcwiYmRWYQ8DMrMAcAmZmBeYQMDMrMIeAmVmBOQTMzArMIWBmVmAOATOzAnMImJkV\nmEPAzKzAHAJmZgXmEDAzKzCHgJlZgWUKAUlLJfVK2iFp9UnKvV/SUUn/vmLZLklbJW2R9EQ9Km1m\nZvVRMwQkdQC3A9cA84AVkuYOUu4vgEeqVh0DShFxaUQsHHmV21+5XG52FXLB5+EEn4sTfC7qK0tL\nYCGwMyJ2R8RRYD2wbIByfwj8I/By1XJlPI6l/Eee8Hk4wefiBJ+L+sry5dwJ7KmY35suO07S2cB1\nEfG/Sb70KwXwqKRuSTeOpLJmZlZfY+u0n78EKscKKoPgiog4IGkGSRj0RMTmOh3XzMxGQBFx8gLS\nIqArIpam82uAiIhbK8q80P8WeBfwM+BzEbGxal9rgSMRcdsAxzl5RczM7B0iorr3ZUiytAS6gdmS\nzgMOAMuBFVWV+KX+95LuAh6MiI2SJgEdEfGapMnA1cAtAx1kpB/EzMyGrmYIRESfpFXAJpIxhDsj\nokfSymR1rKvepOL9TOCb6f/yxwJ3R8SmOtXdzMxGqGZ3kJmZta+mX7qZ9Ua0diTpHEnfkfSspG2S\n/lO6fLqkTZK2S3pE0tRm17VRJHVIelLSxnS+kOdC0lRJ/yCpJ/37+ECBz8V/lvSMpKcl3S1pfFHO\nhaQ7JR2U9HTFskE/u6SbJe1M/26uznKMpoZA1hvR2tjbwH+JiHnAZcB/TD//GuDbEXEh8B3g5ibW\nsdFuAn5UMV/Uc/FXwP+NiPcCvwz0UsBzkV5+/ofA/Ii4hKRbeQXFORd3kXw/Vhrws0u6CPgt4L3A\nh4H/JanmWGuzWwJZb0RrSxHx44h4Kn3/GtADnENyDr6eFvs6cF1zathYks4BPgJ8tWJx4c6FpNOA\nD0bEXQAR8XZE/IQCnovUGGCypLHARGAfBTkX6eX0r1YtHuyzfxxYn/697AJ2knzHnlSzQ6DmjWhF\nIel84N8BjwMzI+IgJEEBnNm8mjXU/wT+G1UXFxTwXLwHeEXSXWnX2Lr0SrvCnYuI2A/8D+Alki//\nn0TEtynguahw5iCfvfr7dB8Zvk+bHQIGSDqV5JEbN6UtgurR+rYfvZf0UeBg2jI6WRO27c8FSZfH\nfOCOiJhPct/NGor5dzGN5H++5wFnk7QIrqeA5+IkRvTZmx0C+4BzK+bPSZcVRtrE/Ufg7yNiQ7r4\noKSZ6fpZvPN5TO3oCuDj6Y2H9wAfkvT3wI8LeC72Ansi4gfp/H0koVDEv4urgBci4lBE9AHfBC6n\nmOei32CffR/w7opymb5Pmx0Cx29EkzSe5Ea0jTW2aTd/B/woIv6qYtlG4PfS9zcAG6o3ajcR8ccR\ncW564+Fy4DsR8SngQYp3Lg4CeyTNSRctAZ6lgH8XJN1AiyRNSAc5l5BcOFCkcyF+sXU82GffCCxP\nr556DzAbqPn4/qbfJyBpKcmVEP03ov1FUyvUQJKuAL4LbCNp0gXwxyT/cPeSpPpu4Lci4nCz6tlo\nkhYD/zUiPi7pdAp4LiT9MskA+TjgBeDTJAOkRTwXa0n+Y3AU2AL8B2AKBTgXkv4PUALOAA4Ca4EH\ngH9ggM8u6WbgsyTn6qYsN+c2PQTMzKx5mt0dZGZmTeQQMDMrMIeAmVmBOQTMzArMIWBmVmAOATOz\nAnMImJkVmEPAzKzA/j/d9/RTnWrIuAAAAABJRU5ErkJggg==\n", 777 | "text/plain": [ 778 | "" 779 | ] 780 | }, 781 | "metadata": {}, 782 | "output_type": "display_data" 783 | } 784 | ], 785 | "source": [ 786 | "layers, error_history = trainAE(relu, .1, 100)\n", 787 | "\n", 788 | "# Plot error history\n", 789 | "plt.plot(range(100), error_history)\n", 790 | "plt.show()" 791 | ] 792 | }, 793 | { 794 | "cell_type": "code", 795 | "execution_count": 221, 796 | "metadata": { 797 | "collapsed": false 798 | }, 799 | "outputs": [ 800 | { 801 | "data": { 802 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEACAYAAABcXmojAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X2UHHWd7/H3p3tmMnmAhAchEAIICATkwUBCdgOm5XFE\nL8kRjwu4slevyvEe1KOuG/bpMHuXc1a861H3uO69WRGXVS6y0UVwBRHDIMFAAiQBQkIikJBHniRA\nkknmob/3j6p0OmNlZpLpnmqSz+ucPqlfV9XUZzrd8+2qX/2qFBGYmZkBFPIOYGZmjcNFwczMKlwU\nzMyswkXBzMwqXBTMzKzCRcHMzCoGVRQktUlaIWmlpNn9LDdFUrekj/R5viDpSUl3DzWwmZnVz4BF\nQVIB+A5wGXA6cLWkU/ew3NeAX2b8mC8Czw4tqpmZ1dtg9hSmAqsiYk1EdAN3ADMzlvs8MBd4pfpJ\nSccAlwPfG2JWMzOrs8EUhQnA2qr2uvS5CklHA7Mi4l8A9Vn/m8BXAQ+dNjNrcLXqaP4W8Ad9DZI+\nBLwcEUtIikXfgmFmZg2kaRDLrAeOrWofkz5X7VzgDkkCDgc+KKkHmAZcIelyYCRwkKTbIuLavhuR\n5D0JM7O9FBG1/bIdEf0+gCLwO+A4oAVYAkzqZ/lbgY9kPD8DuLuf9aLR3XjjjXlHGBTnrC3nrC3n\nrJ307+aAf8f35jHgnkJE9Eq6Hrif5HDTLRGxXNJ1aaA5fVepQa0yM7McDObwERFxH3BKn+f+7x6W\n/dQenn8IeGhvA5qZ2fDxiOa9UCqV8o4wKM5ZW85ZW87Z2BQNcpMdSdEoWczM3gkk1byj2XsKZmZW\n4aJgZmYVLgpmZlbhomBmZhUuCmZmVuGiYGZmFS4KZmZW4aJgZmYVLgpmZlbhomBmZhUuCmZmVuGi\nYGZmFS4KZmZW4aJgZmYVgyoKktokrZC0UtLsfpabIqlb0kfS9jGS5klaJulpSV+oVXCzeuvq6mLB\nggU89thjdHd35x3HbFgMeD8FSQVgJXARsAFYBFwVESsylvsV0Al8PyJ+Kmk8MD4ilkgaAzwBzOy7\nbrq+76dgDeP111+ndMkM3ux9i3JvmfEHj2feffM4+OCD845mVpHX/RSmAqsiYk1EdAN3ADMzlvs8\nMBd4ZecTEbEpIpak01uA5cCEIac2q7PZfzOb0X88mj9b8qd88qlriZPLtN/Unncss7obTFGYAKyt\naq+jzx92SUcDsyLiX4DMqiXpeOBs4LF9CWo2nFasWsGJ/+0EJKGCePeHj2fFquV5xzKru6Ya/Zxv\nAdV9DbsVhvTQ0Vzgi+keQ6b29vbKdKlUOmDvkWr5m3zmZOb/8GHeffHxRDl47vaVzDxzVt6x7ADX\n0dFBR0dHXbcxmD6FaUB7RLSl7RuAiIibq5Z5YeckcDiwFfhsRNwtqQn4OXBvRHy7n+24T8EaxpYt\nW7h85gdZtuJZolxmyjlT+Nncu2ltbc07mllFPfoUBlMUisBzJB3NG4GFwNURkbkvLelW4J6I+Gna\nvg14LSK+PMB2XBSsoUQEL774IpI4/vjjkWr62TMbsnoUhQEPH0VEr6TrgftJ+iBuiYjlkq5LZsec\nvqtUBZ4OfBx4WtLidN5fRcR9NfsNzOrk9ddfZ8GCBUjikEMOYdy4cXlHMqu7AfcUhov3FKyRrF69\nmmkXTOOIc99FuafMm8veYuEjCznqqKPyjmZWkcvho+HiomCN5E8+8Se8evLLnP+30wF4cHYHp759\nGv/63X/NOZnZLnmNUzA74GzYtIHxU8ZX2keeeyQbNm3IMZHZ8HBRMMtw4QUX8sQ3nmTHWzvofKOT\nxd9ewgcu+EDesczqzoePzDJ0d3fzmf/5GW6/7XYQfPqzn+Y73/oOhYK/R1njcJ+C2TDr6ekBoKmp\nVuM8zWonl1NSzQ5kLgZ2oPG+sJmZVfhrkNke9PT08OyzzyKJ0047jWKxmHcks7pzUTDLsHnzZi5+\n//tZ/+KLRAQnnX46v5w3j9GjR+cdzayufPjILMNf/vmfU1i5ks9u2cJ1W7ey7amn+F833ph3LLO6\n856CWYanly7llB07Kt+aTtq+nacXL841k9lw8J6CWYYzzj6b5c3NlIEy8FxLC2dOnpx3LLO6c1Ew\ny3D5FVewrNzLt1uKfLOlwMpyLx/88IfzjmVWdx68ZpahbWYbLbOaOHrKUSCx+terGbvoEH787z/O\nO5pZhQevmQ2Tru4u3n7uTVb850okOPQ9h9DaNSrvWGZ158NHZhnOnnQ2j3/3SXre6KT79U6e+D+L\nOee95+Qdy6zuBlUUJLVJWiFppaTZ/Sw3RVK3pI/s7bpmjeShRzoo7ujl5PnrOOmRdRS7y/z64V/n\nHcus7gZzj+YCsJLkHs0bgEXAVRGxImO5XwGdwPcj4qeDXTdd330K1jCOOvwQJr2+mTfS9sHAmglH\ns3rd+jxjme0mr5vsTAVWRcSaiOgG7gBmZiz3eWAu8Mo+rGvWULq7epkPbAO2AguAnq7ufEOZDYPB\nFIUJwNqq9rr0uQpJRwOzIuJfAO3NumaNaMvbb3M+cA3wcWAK8Pprr+UbymwY1Orso28BQ+4vaG9v\nr0yXSiVKpdJQf6TZPikAR1e1JwCLfXjTctbR0UFHR0ddtzGYPoVpQHtEtKXtG4CIiJurlnlh5yRw\nOMke92dJDiX1u27Vz3CfgjWMUc3NHNbTw8dJRjTfBmxtbeXtzs6ck5ntktc4hUXASZKOAzYCVwFX\nVy8QESdUhbwVuCci7pZUHGhds0Y08YTjWb3yd3yd5JtOE3D6aZNyTmVWfwMWhYjolXQ9cD/JXvUt\nEbFc0nXJ7JjTd5WB1q1dfLP62PDKJpoLMK2ctBcWYPW6NfmGMhsGvsyFWYaRxQIjysFBJIePtgHb\niwU6e3pzTma2iy9zYTZMIoJJQBvJru89wLIo5xvKbBj4MhdmGQoSJ6XTAk4EigXfjtP2fy4KZhnG\nHzuRx4BeoJvkbIsTTjkl31Bmw8BFwSzDBReWeKm5wNcENwMbWwpcfNmleccyqzt3NJtlOPK4Izn6\n0qMo/f0MBNz/5Qd4e9EW1qzyGUjWOPK69pHZAWfrtq2c9tFJHDR+DGPGj+HUj5zC5rfeGHhFs3c4\nFwWzDFEOlv7gKaIclHvKPPVvT+OTj+xA4MNHZhkmnTWJNetXU2xpIiIod/Vyxsln8NiChXlHM6vw\n4SOzYTL7S7PZ/vYOOt/oZPvm7XS+tZ3ZX70h71hmdeeiYJbh+z/8Pk3dwQnbezl+ey/NZfjBj27N\nO5ZZ3fnwkVmG0a0tnLejmxlp+37g6dEjeWvLtjxjme3Gh4/Mhkn09vzB/RR6unbkFcds2LgomGXo\nbSrwcEuRTpKbg8xvKdBTrOkXMrOG5MNHZhmaRjVR7A56essIKBQFLU10be3KO5pZha+SajZMis1F\naIIJZx1BuafMa8tfx19a7EDgomCWIcrBGR8/nTFHjAaJQ08+lOV3rsg7llndDerwkaQ24Fvsunva\nzX3mXwH8Pcn9SLqBL0XEI+m8LwH/I533NPDJiPiDfXAfPrJGMmLsCApdvUzu6qUssbSlSG9zgR1v\nurPZGkc9Dh8NWBQkFYCVwEXABpKrCF8VESuqlhkVEdvS6TOAOyNikqSjgfnAqRHRJenHwH9FxG0Z\n23FRsIYxoljg4nIwNW0/DMwvFtjuO69ZA8nrlNSpwKqIWBMR3cAdwMzqBXYWhNQYkr2CnYrAaElN\nwCiSwmLW0AoRjK1qjwPwlxY7AAymKEwA1la116XP7UbSLEnLSe5c+CmAiNgAfAN4CVgPbI6IB4Ya\n2qzeelubeKClyEaSN+68lgLlEb7zmu3/atbRHBF3AXdJOh+4CbhE0jiSvYrjgDeBuZKuiYjbs35G\ne3t7ZbpUKlEqlWoVz2zvNInNXb38MG12lwON8LAey1dHRwcdHR113cZg+hSmAe0R0Za2bwCib2dz\nn3WeB6YAFwKXRcRn0uc/AZwXEddnrOM+BWsYo1qaOb+7h+lpex7weGsLWzvd0WyNI68+hUXASZKO\nk9QCXAXc3SfYiVXTk4GWiPg9yWGjaZJaJYmks3p5zdKb1cn4dx3Bu6raRwAnHnt8TmnMhs+Ah48i\nolfS9STXBNt5SupySdcls2MOcKWka4EuoBP4WLruQklzgcUkp6ouBubU51cxq51Rh41l3qaNHFEO\nykBHUZx42Li8Y5nVnS9zYZbhlDNOYfWq5yn3Ju9JFeHM972Pxxc8nnMys11yGacwXFwUrJGMOWwM\n9HRzdEA54JUmUWhuZfMrm/OOZlbhax+ZDZOetzs5r7fMhemIm/uKYnGxJ99QZsPA59iZZVBPmYlV\nQzAn9gbqKe95BbP9hPcUzDLEiCbmd/VwUBkC+G0B1NqSdyyzunOfglmGkWNGU966jSJJUSgDI8Ye\nzFub38w5mdku7lMwGybRtYP3AFeSFIU7gJc6fX9m2/+5T8Esg3p6eR/JB6QInA3Q445m2/+5KJhl\niOYCy5sKlUNHK5oKRLMviGf7Px8+MstQHlHkme29PE9SFLoKotzk71C2/3NRMMtQ3hHEiAJjzz8G\ngPULNyQXajHbz7komGUothZgSzctD75EGSgUBaOb845lVnc+JdUsw4iCmB4wI23fDywqQFev36PW\nOPK6dLbZAacYcHRVewJQ9IBmOwC4KJhl6AF+Q3Id+K3Aw7hLwQ4MPnxklqHYVKCpN9g5MqEJ6C5C\nucfvUWscuR0+ktQmaYWklZJmZ8y/QtJSSYslLZQ0vWreWEn/IWm5pGWSzqvlL2BWD8WRTYw4YiTN\nQDPQcsQoii3uaLb932Du0VwAVpLcSnMDye05r4qIFVXLjIqIben0GcCdETEpbf8AeCgibpXUBIyK\niLcytuM9BWsYzWOaKXb2cHDaj7C5ALQ207W1K9dcZtXy2lOYCqyKiDUR0U1yGZiZ1QvsLAipMSTj\nfZB0MHBBRNyaLteTVRDMGo229tBahjbgUmBEGaLTvQq2/xtMUZgArK1qr0uf242kWZKWA/cAn0qf\nfjfwmqRbJT0paY6kkUMNbVZvTSQF4UTgPcDFQJN3ZO0AULOzjyLirvSQ0SzgpvTpJmAy8M8RMRnY\nBtxQq22a1UsZ2F7V3k5ytVSz/d1gRjSvB46tah+TPpcpIuZLOkHSoSR7FWsjYufdzucCf9BRvVN7\ne3tlulQqUSqVBhHPrPZ6Wpq4t6uHbUAvySmpMcIdzZavjo4OOjo66rqNwXQ0F4HnSDqaNwILgasj\nYnnVMidGxPPp9GTgZxExMW0/BHwmIlZKupGkoznrDCZ3NFvDkFopageF9C3ZK1BhJD09vqeCNY5c\nbrITEb2SricZ6V8AbomI5ZKuS2bHHOBKSdcCXSTjfT5W9SO+APxIUjPwAvDJWv4CZvVw0KgWRm7b\nwTUkh5J+GKCxo/OOZVZ3HrxmluHkY49lytq1nJy2nwGeP+00nly2LM9YZrvxtY/MhsmGTS/zalX7\nVeD51WvyimM2bHzpbLMMPa1FHqLAup4yIXixqUDBnxY7APhtbpYlylAOXgAigN5ANd1JN2tMLgpm\nGcrbuzmhN/gYyfiE23uDtdu2D7Sa2Tue+xTMMjT1lJkCFEm+OZ0DFLt78w1lNgy8p2CWoQwsJ7mH\ngoAV6XNm+zsXBbMM5RFFntrRy8skxeBVoDhqRM6pzOrPRcEsw4gocBa9XJK2/wtY6aOtdgDwu9ws\nw4hiE8dXtY8HRrV4T8H2f95TMMswcuxYftvZWfmALAAOOuywPCOZDQsXBbMMb5d3sL0gflYOAthR\nFGN3+GJ4tv9zUTDL0PXWFs4owIfKyTiFuySef+ONvGOZ1Z37FMwyNIc4tScQyYfklJ4yLf642AHA\n73KzDAcffihPkpyO2gssBo485g/uQmu23/Gls80yjJAYQzJ4LYAWYGuhwPZej2q2xpHLTXbMDkRF\noA04nGRE83rg52WPabb936AOH0lqk7RC0kpJWbfSvELSUkmLJS2UNL3P/IKkJyXdXavgZvVUJrnv\n7EHAKGARyWEks/3dYO7RXABWktyjeQPJ5+OqiFhRtcyoiNiWTp8B3BkRk6rmf4nkmmIHR8QVe9iO\nDx9ZwxjZ0kKhu5uutN0MNI0ezVtbtuQZy2w3ed15bSqwKiLWREQ3cAcws3qBnQUhNYaqa4dJOga4\nHPje0OOaDY+yRtJJchipCOwAQq35hjIbBoMpChOAtVXtdelzu5E0S9Jy4B7gU1Wzvgl8laS/zuwd\noatrGyOBC4ESyZ7Ctm1v55rJbDjU7JTUiLgrPWQ0C7gJQNKHgJcjYglJf53vXWXvCC30Mhp4LH2M\nBZrK3fmGMhsGgzn7aD1wbFX7mPS5TBExX9IJkg4FpgNXSLocGAkcJOm2iLg2a9329vbKdKlUolQq\nDSKeWT0EE4CdHWBzgc05pjED6OjooKOjo67bGExHcxF4jqSjeSPJSRlXR8TyqmVOjIjn0+nJwM8i\nYmKfnzMD+Io7mu2dYKTER4GT0vYykuOinX6PWgPJZZxCRPRKuh64n+Rw0y0RsVzSdcnsmANcKela\noAvoBD5Wy5Bmw60MPAOckLafwaek2oHBI5rNMhRaCjR1ByNIzpDoAsojmunZ3jXAmmbDxyOazYZJ\nc1kUSYoCQA/Q5EuF2QHA73KzDNFbZgTJAJs2kmsf9e7YkW8os2HgomCWoQm4DDgReA/JWRb+sNiB\nwO9zsz3Y3mdaBX9cbP/nPgWzDM0jR3JfZyfbSM46mg8cdNBBOacyqz8XBbMM48aM4bTOTl4jGYY/\nA3hp3LicU5nVn/eHzTJML5VYx66O5jXAhZddlm8os2HgPQWzDF3FMhuOGsO3N2+nDDQf0kqXPHzN\n9n8uCmYZXn7tZS793uUccsI4VBCblrzMxjs25h3LrO58+Mgsw4zpM1jyT0s5eOLBjD5yNE/989OU\nppfyjmVWd77MhVmGrq4uLmq7kN8+vACAiy+9mF/c/QuKxWLOycx2yevOa2YHnJ/89CcsfuJJWluK\ntLYUeeTh33DvvffmHcus7rynYJbh5FNP5M3fvcif9AYB/L+imHj2mSx+fEne0cwqvKdgNkxeXbOB\ni3uDw4DDgQ/0ButXvZh3LLO6c1EwyxARvF7Vfh0ol8t5xTEbNj4l1SxDsXkED+3Ywe9JLnPxLDCu\ndXTOqczqb1B7CpLaJK2QtFLS7Iz5V0haKmmxpIWSpqfPHyNpnqRlkp6W9IVa/wJm9XDSaWfQTYGN\nwMuIboqc8b5z8o5lVneDuUdzAVhJcvXgDcAi4KqIWFG1zKiI2JZOnwHcGRGTJI0HxkfEEkljgCeA\nmdXrVv0MdzRbw5g/fz6XXPJBtm8/AoCRI19l/vwOJk+enHMys13yuvPaVGBVRKxJQ9wBzAQqf9h3\nFoTUGJJb3BIRm4BN6fQWScuBCdXrmjWi888/n9/8Zh7f+96tFAoFPve5z3LmmWfmHcus7gZTFCYA\na6va60gKxW4kzQL+AXgX8KGM+ccDZwOP7UNOs2E3ZcoUpkyZkncMs2FVs47miLgLuEvS+cBNwCU7\n56WHjuYCX4yILXv6Ge3t7ZXpUqlEqVSqVTwzs3e8jo4OOjo66rqNwfQpTAPaI6Itbd8ARETc3M86\nzwNTIuL3kpqAnwP3RsS3+1nHfQpmZnshr8Fri4CTJB0nqQW4Cri7T7ATq6YnAy0R8fv0qe8Dz/ZX\nEMzMrDEMePgoInolXQ/cT1JEbomI5ZKuS2bHHOBKSdcCXUAn8DGA9NTUjwNPS1oMBPBXEXFffX4d\nMzMbCl/7yGwPtmzZwsMPP4wkZsyYwciRI/OOZLabvE5JNTvgbNy4kT+eMoWmN9+kHEHzEUfwyKJF\nHHbYYXlHM6srX/vILMNXvvAFjl6/nj/dsoVPbN3KQatX8zc33JB3LLO6c1Ewy/DIQw9xQjot4N0R\ndDzwQJ6RzIaFi4JZhigUWAj0kJw98ThQVk0P3Zo1JBcFswwzLryQNyS+DvxvYEuhwKVtbXnHMqs7\nn31kluHVV1/lgvPOY9srrxDAYccey0MLFjB27Ni8o5lV+M5rZsNk3LhxnDtlChs7O9nU2cnUP/oj\nxowZk3css7pzUTDL8Pft7Tx+zz18tVzmK+Uy8+64g29+4xt5xzKrOxcFswwP/upXnNvZyQhgJDB5\n2zYevP/+vGOZ1Z2LglmGdx15JOuq2uuAI446Kq84ZsPGI5rNMjS1tLCQ5A5RZZKi8N6WlnxDmQ0D\nFwWzDBteeokPk95CEDgdWLd6dX6BzIaJDx+ZZThn6lR+19rKacAk4HcjR3LOeeflHcus7jxOwSzD\nli1b+PBll/HU4sWUI/ij6dP5z5//nNbW1ryjmVXUY5yCi4LZHkQEL730EpKYOHEi8mUurMG4KJiZ\nWUVuI5oltUlaIWmlpNkZ86+QtFTSYkkL0zuuDWpdMzNrHAPuKUgqACuBi4ANJPdsvioiVlQtMyoi\ntqXTZwB3RsSkwaxb9TO8p2Bmthfy2lOYCqyKiDUR0Q3cAcysXmBnQUiNYdeZfAOua2ZmjWMwRWEC\nsLaqvS59bjeSZklaDtwDfGpv1jVrVOVymXK5PPCCZvuJmg1ei4i7gLsknQ/cBFyytz+jvb29Ml0q\nlSiVSrWKZ7ZXurq6mDp5Mk8vWwbA1HPPZf6jj1IsFnNOZgeyjo4OOjo66rqNwfQpTAPaI6Itbd8A\nRETc3M86zwNTgJMHu677FKyRtF10Ec/Om8c1JMdC/x244MoruXPu3JyTme2SV5/CIuAkScdJagGu\nAu7uE+zEqunJQEtE/H4w65o1oicee4z3k1whdTRwAfDIvHn5hjIbBgMePoqIXknXA/eTFJFbImK5\npOuS2TEHuFLStSS3s+0EPtbfunX6Xcxqpkxyutx70vZ6oFzwVWFs/+fBa2YZzjzlFF5YuZLj2HWV\n1LPOPZf5ixblnMxsl3ocPvJVUs0ynDJpEoesWsUhEQg4tFDglNNOyzuWWd15T8Esw6pVq5g+dSoT\nd+ygLLFp1CgWPvkkEydOzDuaWYWvfWQ2jDZt2sQ999yDJGbNmsXhhx+edySz3bgomJlZRW4XxDMz\nswODi4KZmVW4KJiZWYWLgpmZVbgomJlZhYuCmZlVuCiYmVmFi4KZmVW4KJiZWYWLgpmZVbgomJlZ\nhYuCmZlVDKooSGqTtELSSkmzM+ZfI2lp+pgv6cyqeV+S9IykpyT9KL0tp5mZNaABi4KkAvAd4DLg\ndOBqSaf2WewF4P0RcRZwEzAnXfdo4PPA5Ig4k+SmPlfVLr6ZmdXSYPYUpgKrImJNRHQDdwAzqxeI\niEcj4s20+SgwoWp2ERgtqQkYRXLrWzMza0CDKQoTgLVV7XXs/ke/r08D9wJExAbgG8BLJPc+3xwR\nD+xbVDMzq7ea3qNZ0geATwLnp+1xJHsVxwFvAnMlXRMRt2et397eXpkulUqUSqVaxjMze0fr6Oig\no6OjrtsY8M5rkqYB7RHRlrZvACIibu6z3JnAT4C2iHg+fe6jwGUR8Zm0/QngvIi4PmM7vvOamdle\nyOvOa4uAkyQdl545dBVwd59gx5IUhE/sLAipl4BpklolCbgIWF6b6GZmVmsDHj6KiF5J1wP3kxSR\nWyJiuaTrktkxB/hb4FDgu+kf/+6ImBoRCyXNBRYD3em/c+r1y5iZ2dAMePhouPjwkZnZ3snr8JGZ\nmR0gXBTMzKzCRcHMzCpcFMzMrMJFwczMKlwUzMyswkXBzMwqXBTMzKzCRcHMzCpcFMzMrMJFwczM\nKlwUzMyswkXBzMwqXBTMzKzCRcHMzCoGVRQktUlaIWmlpNkZ86+RtDR9zE9vzblz3lhJ/yFpuaRl\nks6r5S9gZma1M2BRkFQAvgNcBpwOXC3p1D6LvQC8PyLOAm5i97urfRv4RURMAs7iHXw7znrfMLtW\nnLO2nLO2nLOxDWZPYSqwKiLWREQ3cAcws3qBiHg0It5Mm48CEwAkHQxcEBG3psv1RMRbNUs/zN4p\nbxLnrC3nrC3nbGyDKQoTgLVV7XXpc3vyaeDedPrdwGuSbpX0pKQ5kkbuW1QzM6u3mnY0S/oA8Elg\nZ79DEzAZ+OeImAxsA26o5TbNzKyGIqLfBzANuK+qfQMwO2O5M4FVwIlVzx0JvFDVPh+4Zw/bCT/8\n8MMPP/buMdDf8L19NDGwRcBJko4DNgJXAVdXLyDpWOAnwCci4vmdz0fEy5LWSjo5IlYCFwHPZm0k\nIjSILGZmVkcDFoWI6JV0PXA/yeGmWyJiuaTrktkxB/hb4FDgu5IEdEfE1PRHfAH4kaRmkrOUPlmP\nX8TMzIZO6aEbMzOz+o5olnSIpPslPSfpl5LG7mG5zMFxe1pfUpOkH0h6Kh0QN6TO63rlTOedKem3\nkp5JB/e1NGLOdP6xkt6W9OV9zVjPnJIulvR4+jouSk9s2Nts/Q7ETJf5J0mrJC2RdPa+5h2KOuX8\nejqIdImkn6SnjDdczqr5X5FUlnRoo+aU9Pn0NX1a0tcaMaeksyQtkLRY0kJJ5/YbotadFH06j28G\n/iKdng18LWOZAvA74DigGVgCnNrf+iR9Gren0yOBF4FjGzBnEVgKvDdtH0K6d9ZIOavW/Q/gx8CX\nG/T//SxgfDp9OrBuL3PtcZtVy3wQ+K90+jzg0aG+rvvw+tUr58VAIZ3+GvAPjZgznX8McB/JZ/vQ\nRswJlEgOqzel7cMbNOcvgUur1n+wvxz1vvbRTODf0ul/A2ZlLNPf4Lg9rR/AaElFYBSwAxjKoLh6\n5bwUWBoRzwBExBuR/s80WE4kzSTp81k2hHx1zRkRSyNiUzq9DGhV0lc1WAMOxEzbt6XbeAwYK+nI\nfck7BHXJGREPREQ5Xf9Rkj+8DZcz9U3gq0PMV++cnyP5AtCTrvdag+YsAzv3XscB6/sLUe+icERE\nvAyQfpiPyFimv8FxR/ZZ/8j0+bkkYx42AquBf4yIzQ2Uc+f6JwNIui897DHUN3ldXk9JY4C/AP4O\nqMVZYPV6PSskfRR4Mv0ADNZgBmLuaZkh5d1L9cpZ7VPsGmTaUDklXQGsjYinh5ivrjlJPt/vl/So\npAcHPCwheaANAAACv0lEQVSTX84vAf8o6SXg68Bf9hdiMKek9kvSr9j1xxqSPyoB/E3G4kPt1d75\nLec8oAcYDxwGPCzpgYhY3SA5d67fBEwHzgW2A7+W9HhEPNggOXe+njcC34yIbZJ2brNfOb2eO7d9\nOvAPwCVD/LmDsS9FMo+zNwadU9Jfk5wheHsd8+xx8/3OTK548Ffs/n+bx+nqg9lmE3BIREyTNAW4\nEzihvrH+wGByfg74YkTclX6Z+j79fHaGXBQiYo8/XNLLko6MZLzCeOCVjMXWA8dWtY9h1+7Npj2s\nfzXJgLoy8KqkR0j+8K5usJzrgN9ExBvpdn5BMsJ7j0Uhp5znAVdK+jpJv0evpM6I+G6D5UTSMcBP\nScbErN5Thj3ob5vVy0zMWKZlX/Luo3rlRNJ/By4HLhxixnrlPBE4Hliq5BvKMcATkqZGxL6+rvV6\nPdeRvBeJiEVpp/hhEfF6g+X8s4j4YppzrqRb+k0xlI6RgR4kHXCzo58OOJLO2J0dJC0kHSSTMta/\ngV0djn9BMl4CYDTJcfD3NkjO6o7RccDjQCtJAf4V8MFGy9ln/RupTUdzvV7PJcCsfcy1x21WLXM5\nuzryprGrI29Ir2uD5GxLPyuHDSVfvXP2Wf9Fkm/jDZcTuA74u3T6ZGBNg+Xc2dG8DJiRTl8ELOo3\nRy3eHP38kocCDwDPkfTSj0ufPwr4edVybekyq4AbBrH+aJJdtWfSx1D/iNUlZzrvmjTjUwz9bI+6\n5axaphZFoV7/738NvA08CSxO/92rMz6ytkny4f5s1TLfST9gS4HJtXhd9+E1rEfOVcCa9HV7Evhu\nI+bs8/NfYIhnH9Xx9WwG/h14muTL34wGzfnHab7FwALgff1l8OA1MzOr8O04zcyswkXBzMwqXBTM\nzKzCRcHMzCpcFMzMrMJFwczMKlwUzMyswkXBzMwq/j8/mJKYsmji7QAAAABJRU5ErkJggg==\n", 803 | "text/plain": [ 804 | "" 805 | ] 806 | }, 807 | "metadata": {}, 808 | "output_type": "display_data" 809 | } 810 | ], 811 | "source": [ 812 | "# Get encoded coordinates\n", 813 | "ec1 = np.array(layers[1]['activations'][0, :].transpose())\n", 814 | "ec2 = np.array(layers[1]['activations'][1, :].transpose())\n", 815 | "\n", 816 | "# Plot encoded data\n", 817 | "plt.scatter(ec1, ec2, c=labels)\n", 818 | "plt.show()" 819 | ] 820 | }, 821 | { 822 | "cell_type": "markdown", 823 | "metadata": {}, 824 | "source": [ 825 | "This has a tendency to squash the dimensions :-/ So probably not a good pick." 826 | ] 827 | } 828 | ], 829 | "metadata": { 830 | "anaconda-cloud": {}, 831 | "kernelspec": { 832 | "display_name": "Python [default]", 833 | "language": "python", 834 | "name": "python3" 835 | }, 836 | "language_info": { 837 | "codemirror_mode": { 838 | "name": "ipython", 839 | "version": 3 840 | }, 841 | "file_extension": ".py", 842 | "mimetype": "text/x-python", 843 | "name": "python", 844 | "nbconvert_exporter": "python", 845 | "pygments_lexer": "ipython3", 846 | "version": "3.5.2" 847 | } 848 | }, 849 | "nbformat": 4, 850 | "nbformat_minor": 2 851 | } 852 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # PCA and autoencoders 2 | 3 | Principal component analysis (PCA) is an example of dimensionality reduction. 4 | 5 | Autoencoders generalize the idea to non-linear transformations. 6 | 7 | The interactive Jupyter notebook let's you train such autoencoders yourself, and see the results. 8 | -------------------------------------------------------------------------------- /autoencoder.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kwichmann/PCA_and_autoencoders/604669255671dfd3d2ebd0331bf06ae4ccaa9a1c/autoencoder.png -------------------------------------------------------------------------------- /deep_autoencoder.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/kwichmann/PCA_and_autoencoders/604669255671dfd3d2ebd0331bf06ae4ccaa9a1c/deep_autoencoder.png --------------------------------------------------------------------------------