├── OneMax.ipynb ├── README.md ├── SIGEvolution.ipynb ├── images └── deap_highlights.png └── styles └── custom.css /OneMax.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "metadata": { 3 | "name": "" 4 | }, 5 | "nbformat": 3, 6 | "nbformat_minor": 0, 7 | "worksheets": [ 8 | { 9 | "cells": [ 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "# OneMax Problem\n", 15 | "The problem is very simple, we search for a 1 filled solution. This notebook will cover creating a program to evolve a solution to this problem as well as go a little more in detail on the step of the algorithm.\n", 16 | "\n", 17 | "## Imports\n", 18 | "We first get the required modules for our evolution.\n", 19 | "\n", 20 | "- `random` gives us a way to generate random bits;\n", 21 | "- `base` gives us access to the Toolbox and base Fitness;\n", 22 | "- `creator` allows us to create our types;\n", 23 | "- `tools` grants us access to the operators bank;\n", 24 | "- `algorithms` enables us some ready generic evolutionary loops." 25 | ] 26 | }, 27 | { 28 | "cell_type": "code", 29 | "collapsed": false, 30 | "input": [ 31 | "import random\n", 32 | "from deap import base, creator, tools, algorithms" 33 | ], 34 | "language": "python", 35 | "metadata": {}, 36 | "outputs": [], 37 | "prompt_number": 12 38 | }, 39 | { 40 | "cell_type": "markdown", 41 | "metadata": {}, 42 | "source": [ 43 | "## Type Creation\n", 44 | "First step with DEAP is to create the required types. Usually the types created are the fitness and the individual. For the OneMax problem, we want to have a solution with as many ones as possible. Thus we need a maximizing fitness and a individual that is a sorted container, we'll choose the standard `list`.\n", 45 | "\n", 46 | "Type creation is done by calling the function `create` in the creator module. This function takes two mandatory arguments and additional optional arguments. The first argument is the actual name of the type that we want to create. The second argument is the base classe that the new type created should inherit from. Finally, the optional arguments are members to add to the new type." 47 | ] 48 | }, 49 | { 50 | "cell_type": "code", 51 | "collapsed": false, 52 | "input": [ 53 | "creator.create(\"FitnessMax\", base.Fitness, weights=(1.0,))\n", 54 | "creator.create(\"Individual\", list, fitness=creator.FitnessMax)" 55 | ], 56 | "language": "python", 57 | "metadata": {}, 58 | "outputs": [], 59 | "prompt_number": 13 60 | }, 61 | { 62 | "cell_type": "markdown", 63 | "metadata": {}, 64 | "source": [ 65 | "The first line creates a maximizing fitness by replacing, in the base type Fitness, the pure virtual weights attribute by (1.0,) that means to maximize a single objective fitness. The second line creates an Individual class that inherits the properties of list and has a fitness attribute of the type FitnessMax that was just created.\n", 66 | "\n", 67 | "Single objective is considered by DEAP the same way a multi objective function would but with a single value. The weights (as well as the returned value from the evaluation) are **always required** to be iterable.\n", 68 | "\n", 69 | "The created classes are made available in the creator module. We can instantiate directly objects of the created class like follow. This step is not required in an algorithm, the instatiation will be automated later." 70 | ] 71 | }, 72 | { 73 | "cell_type": "code", 74 | "collapsed": false, 75 | "input": [ 76 | "ind = creator.Individual([1, 0, 1, 1, 0])\n", 77 | "\n", 78 | "print(ind)\n", 79 | "print(type(ind))\n", 80 | "print(type(ind.fitness))" 81 | ], 82 | "language": "python", 83 | "metadata": {}, 84 | "outputs": [ 85 | { 86 | "output_type": "stream", 87 | "stream": "stdout", 88 | "text": [ 89 | "[1, 0, 1, 1, 0]\n", 90 | "\n", 91 | "\n" 92 | ] 93 | } 94 | ], 95 | "prompt_number": 14 96 | }, 97 | { 98 | "cell_type": "markdown", 99 | "metadata": {}, 100 | "source": [ 101 | "## Toolbox\n", 102 | "The toolbox is intended to store functions with their arguments under standard aliases for uses in algorithms. Functions are registered by a function call with two mandatory arguments, the alias to give to the function and the function it will be associate with. Any additional argument will be given as argument when the alias is called." 103 | ] 104 | }, 105 | { 106 | "cell_type": "code", 107 | "collapsed": false, 108 | "input": [ 109 | "toolbox = base.Toolbox()\n", 110 | "toolbox.register(\"attr_bool\", random.randint, 0, 1)\n", 111 | "toolbox.register(\"individual\", tools.initRepeat, creator.Individual, toolbox.attr_bool, n=10)\n", 112 | "toolbox.register(\"population\", tools.initRepeat, list, toolbox.individual)" 113 | ], 114 | "language": "python", 115 | "metadata": {}, 116 | "outputs": [], 117 | "prompt_number": 15 118 | }, 119 | { 120 | "cell_type": "markdown", 121 | "metadata": {}, 122 | "source": [ 123 | "In the last block of code we created a toolbox object and registered three functions. This first one, `attr_bool`, calls randint from the random module with arguments (0, 1) to create an integer in the interval $[0, 1]$. The second function, `individual`, when called, will use the initRepeat function made available in the tools module to fill an `Individual` class with what is produced by 10 calls to the previously defined `attr_bool` function. The same thing is done for the `population` function.\n", 124 | "\n", 125 | "For example, calling every function individually shows how it proceeds." 126 | ] 127 | }, 128 | { 129 | "cell_type": "code", 130 | "collapsed": false, 131 | "input": [ 132 | "bit = toolbox.attr_bool()\n", 133 | "ind = toolbox.individual()\n", 134 | "pop = toolbox.population(n=3)\n", 135 | "\n", 136 | "print(\"bit is of type %s and has value\\n%s\" % (type(bit), bit))\n", 137 | "print(\"ind is of type %s and contains %d bits\\n%s\" % (type(ind), len(ind), ind))\n", 138 | "print(\"pop is of type %s and contains %d individuals\\n%s\" % (type(pop), len(pop), pop))" 139 | ], 140 | "language": "python", 141 | "metadata": {}, 142 | "outputs": [ 143 | { 144 | "output_type": "stream", 145 | "stream": "stdout", 146 | "text": [ 147 | "bit is of type and has value\n", 148 | "0\n", 149 | "ind is of type and contains 10 bits\n", 150 | "[0, 0, 0, 0, 0, 0, 0, 1, 0, 0]\n", 151 | "pop is of type and contains 3 individuals\n", 152 | "[[1, 1, 0, 0, 0, 1, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 1, 0, 0], [1, 0, 0, 1, 0, 1, 1, 0, 0, 1]]\n" 153 | ] 154 | } 155 | ], 156 | "prompt_number": 16 157 | }, 158 | { 159 | "cell_type": "markdown", 160 | "metadata": {}, 161 | "source": [ 162 | "## Evaluation Function\n", 163 | "The evaluation function is pretty simple for the OneMax problem, we need to count the number of ones in an individual. We recall here that the returned value must be an iterable of length equal to the number of objectives (weights)." 164 | ] 165 | }, 166 | { 167 | "cell_type": "code", 168 | "collapsed": false, 169 | "input": [ 170 | "def evalOneMax(individual):\n", 171 | " return sum(individual)," 172 | ], 173 | "language": "python", 174 | "metadata": {}, 175 | "outputs": [], 176 | "prompt_number": 17 177 | }, 178 | { 179 | "cell_type": "markdown", 180 | "metadata": {}, 181 | "source": [ 182 | "## Genetic Operators\n", 183 | "Registering the operators and their default arguments in the toolbox is done as follow." 184 | ] 185 | }, 186 | { 187 | "cell_type": "code", 188 | "collapsed": false, 189 | "input": [ 190 | "toolbox.register(\"evaluate\", evalOneMax)\n", 191 | "toolbox.register(\"mate\", tools.cxTwoPoint)\n", 192 | "toolbox.register(\"mutate\", tools.mutFlipBit, indpb=0.10)\n", 193 | "toolbox.register(\"select\", tools.selTournament, tournsize=3)" 194 | ], 195 | "language": "python", 196 | "metadata": {}, 197 | "outputs": [], 198 | "prompt_number": 18 199 | }, 200 | { 201 | "cell_type": "markdown", 202 | "metadata": {}, 203 | "source": [ 204 | "The evaluation is given the alias `evaluate`. Having a single argument being the individual to evaluate we don\u2019t need to fix any, the individual will be given later in the algorithm. The two points crossover function is registered the same way under the alias `mate`. The mutation, for its part, needs an argument to be fixed (the independent probability of each attribute to be mutated `indpb`). In the algorithms the `mutate()` function is called with the signature `toolbox.mutate(mutant)`. This is the most convenient way because each mutation takes a different number of arguments, having those arguments fixed in the toolbox leave open most of the possibilities to change the mutation (crossover, selection, or evaluation) operator later in your researches. Finally, the selection operator is registered under the name `select` and the size of the tournament set to 3.\n", 205 | "\n", 206 | "We can for example mutate an individual and expect 10% of its attributes to be flipped." 207 | ] 208 | }, 209 | { 210 | "cell_type": "code", 211 | "collapsed": false, 212 | "input": [ 213 | "ind = toolbox.individual()\n", 214 | "print(ind)\n", 215 | "toolbox.mutate(ind)\n", 216 | "print(ind)" 217 | ], 218 | "language": "python", 219 | "metadata": {}, 220 | "outputs": [ 221 | { 222 | "output_type": "stream", 223 | "stream": "stdout", 224 | "text": [ 225 | "[0, 1, 1, 1, 0, 0, 1, 0, 0, 1]\n", 226 | "[0, 1, 1, 1, 0, 0, 0, 0, 0, 1]\n" 227 | ] 228 | } 229 | ], 230 | "prompt_number": 19 231 | }, 232 | { 233 | "cell_type": "markdown", 234 | "metadata": {}, 235 | "source": [ 236 | "The operators execute their behaviour on the individuals in place, meaning that if the individual is not copied before modified the old individual is lost. Copying an individual is done with the `clone` function available in every toolbox." 237 | ] 238 | }, 239 | { 240 | "cell_type": "code", 241 | "collapsed": false, 242 | "input": [ 243 | "mutant = toolbox.clone(ind)\n", 244 | "print(mutant is ind)\n", 245 | "print(mutant == ind)" 246 | ], 247 | "language": "python", 248 | "metadata": {}, 249 | "outputs": [ 250 | { 251 | "output_type": "stream", 252 | "stream": "stdout", 253 | "text": [ 254 | "False\n", 255 | "True\n" 256 | ] 257 | } 258 | ], 259 | "prompt_number": 20 260 | }, 261 | { 262 | "cell_type": "markdown", 263 | "metadata": {}, 264 | "source": [ 265 | "## Evolving the Population\n", 266 | "The main program shall be defined in a main function. All previous declarations are put in the global scope of the module to make created objects available for further use in other user experiments.\n", 267 | "\n", 268 | "The main program is very simple. It consists of generating a population, and giving it to the algorithm for it to evolve a solution. Here we will employ some helpful introspection tools such as Statistics and a Hall of Fame. The statistics are computed using `numpy` functions on the population, and the hall of fame keeps track of the best individuals that ever appeared during the evolution. The algorithm take as arguments, among other, the population and the toolbox. Once the evolution is finished the population contains the individuals from the last generation." 269 | ] 270 | }, 271 | { 272 | "cell_type": "code", 273 | "collapsed": false, 274 | "input": [ 275 | "def main():\n", 276 | " import numpy\n", 277 | " \n", 278 | " pop = toolbox.population(n=50)\n", 279 | " hof = tools.HallOfFame(1)\n", 280 | " stats = tools.Statistics(lambda ind: ind.fitness.values)\n", 281 | " stats.register(\"avg\", numpy.mean)\n", 282 | " stats.register(\"min\", numpy.min)\n", 283 | " stats.register(\"max\", numpy.max)\n", 284 | " \n", 285 | " pop, logbook = algorithms.eaSimple(pop, toolbox, cxpb=0.5, mutpb=0.2, ngen=10, stats=stats, halloffame=hof, verbose=True)\n", 286 | " \n", 287 | " return pop, logbook, hof" 288 | ], 289 | "language": "python", 290 | "metadata": {}, 291 | "outputs": [], 292 | "prompt_number": 21 293 | }, 294 | { 295 | "cell_type": "markdown", 296 | "metadata": {}, 297 | "source": [ 298 | "Next, we protect our call to the main function and launch the evolution, the verbose argument tell to output the stats on every generations. We can print and plot the data returned. " 299 | ] 300 | }, 301 | { 302 | "cell_type": "code", 303 | "collapsed": false, 304 | "input": [ 305 | "if __name__ == \"__main__\":\n", 306 | " pop, log, hof = main()\n", 307 | " print(\"Best individual is: %s\\nwith fitness: %s\" % (hof[0], hof[0].fitness))\n", 308 | " \n", 309 | " import matplotlib.pyplot as plt\n", 310 | " gen, avg, min_, max_ = log.select(\"gen\", \"avg\", \"min\", \"max\")\n", 311 | " plt.plot(gen, avg, label=\"average\")\n", 312 | " plt.plot(gen, min_, label=\"minimum\")\n", 313 | " plt.plot(gen, max_, label=\"maximum\")\n", 314 | " plt.xlabel(\"Generation\")\n", 315 | " plt.ylabel(\"Fitness\")\n", 316 | " plt.legend(loc=\"lower right\")\n", 317 | " plt.show()" 318 | ], 319 | "language": "python", 320 | "metadata": {}, 321 | "outputs": [ 322 | { 323 | "output_type": "stream", 324 | "stream": "stdout", 325 | "text": [ 326 | "gen\tnevals\tavg\tmin\tmax\n", 327 | "0 \t50 \t5 \t1 \t8 \n", 328 | "1 \t26 \t6.14\t3 \t8 \n", 329 | "2 \t33 \t7.16\t5 \t9 \n", 330 | "3 \t39 \t7.88\t6 \t10 \n", 331 | "4 \t37 \t8.4 \t6 \t10 \n", 332 | "5 \t31 \t8.9 \t6 \t10 " 333 | ] 334 | }, 335 | { 336 | "output_type": "stream", 337 | "stream": "stdout", 338 | "text": [ 339 | "\n", 340 | "6 \t30 \t9.28\t7 \t10 \n", 341 | "7 \t27 \t9.58\t7 \t10 \n", 342 | "8 \t30 \t9.74\t7 \t10 \n", 343 | "9 \t27 \t9.86\t9 \t10 " 344 | ] 345 | }, 346 | { 347 | "output_type": "stream", 348 | "stream": "stdout", 349 | "text": [ 350 | "\n", 351 | "10 \t31 \t9.72\t8 \t10 \n", 352 | "Best individual is: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1]\n", 353 | "with fitness: (10.0,)\n" 354 | ] 355 | }, 356 | { 357 | "metadata": {}, 358 | "output_type": "display_data", 359 | "png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEKCAYAAAAVaT4rAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XdYVNfWwOEfIqIiIiJgwV6i2MDexV4ilogNo0hsmJhr\ny40mxlxMU9OMiS1WUCNoNEH0qrlqxFiCqKBYosSOIhpBFAGp5/vjfI6YoPQ5A7Pe5+FxGGbOXo6y\nZs86++xloiiKghBCCKNRQusAhBBC6JckfiGEMDKS+IUQwshI4hdCCCMjiV8IIYyMJH4hhDAyBZ74\n33jjDezt7WnatKnuvtjYWHr16kWDBg3o3bs3cXFxBT2sEEKIHCrwxO/p6cnevXufu2/hwoX06tWL\niIgIevTowcKFCwt6WCGEEDlkUhgXcF2/fh1XV1fOnj0LQMOGDTl06BD29vZER0fj4uLCxYsXC3pY\nIYQQOVBSH4PcvXsXe3t7AOzt7bl79+4/HmNiYqKPUIQQotjJ7fxdL4k/MxMTkxcmedk9QuXt7Y23\nt7c2g6emwrx58MMPsHYtNGmiTRz/z/urr/CeNUvTGAyFvBbPyGvxjEm1arl+jl4S/9MST+XKlblz\n5w52dnb6GFbk1o0bMGoUVKgAoaFga6t1RGBpCVWrah2FYZDX4hl5LfJFL4l/4MCB+Pr6Mnv2bHx9\nfRk8eLA+hhW5ERAAkyfDv/8NM2dCCVnpK0Rm6ekQEwN37z77io8HOzuoVk19H6pcGczMtI40ewWe\n+EeNGsWhQ4e4f/8+1atX56OPPmLOnDkMHz6ctWvXUqtWLbZu3VrQwxYrLi4u+hssOVlN9jt3wo4d\n0K6d/sbOAb2+FgZOXotnCuq1SEmBe/fUJP70z8xfme+LjVU/DNvZgb29+mVpqT7m9m2IilJv29io\nbwRP3wyy+rNiRdDytGahrOrJCxMTE6nx69vlyzBiBNSqBWvWgLW11hEJkW8JCVkn8qy+j49XK5pP\nE3nmpP73+2xtoWQ2U+W0NPXYUVHP3gyy+jMpSX0T+Psbwt/fJMqWzf7vm5fcKYnfWPn5wb/+BfPn\nw5Qp2k4/hMiBR4/g5k3168YNuHMn68Senp59En/6ZW2tTVUzMTH7N4eoKChT5sWfGp7+6eAgiV9k\nJzERpk2DQ4dgyxZwdtY6IiFIT4fo6GdJPXOCf3o7JQVq1oQaNdSvqlWzTuqWlsVjHqMoannpZW8O\nt2/D3buS+MXLXLgAw4eDkxOsWKH+hgihBwkJEBn54qR++7Za936a1DMn+Ke3ta6LGyop9YisKQqs\nXw+zZ8Pnn8O4cfIbJAqMoqhllhcl9Rs31MRfvXrWCb1GDfVn5uZa/02KJkn84p/i48HLC8LD1dKO\no6PWEYkiKiYGgoPh1Cm4fv1ZUo+MVD88viip16ypnhiVuUbhkMQvnhcWpq7acXGBb77J2RIBIVBr\n7ufPw++/P/uKjobWraFNG6hT51lSr15d/mtpSRK/UCkKLF+urtj59lsYOVLriISBezqbf5rkT5xQ\nT562awft26tfjRuDqanWkYq/k8Qv4MEDGD9e/Qy+ZQvUq6d1RMLAvGw2/zTJt22rXogkDJ8kfmMX\nHKzO7gcPhkWL5GyZAJ6fzQcHq7P5ypWfJXmZzRdtkviNVUYGfPUVfPklfP+9mviFUcpqNn/njlqX\nl9l88SSJ3xj99Rd4eMDDh+rVuDVqaB2R0COZzQtJ/Mbm0CEYPRrGjIGPPioa2wKKPJPZvMiKJH5j\nkZ4On3wCK1eCjw/06aN1RKKQ3LkDv/wCe/fCvn1QqZLM5sXz8pI79d6BS+RTVBS8/rp6NUxoKFSp\nonVEogClpMCxY2qi37tXvUiqZ0/o21c9jZOHZktC/IPM+IuSX35Rt1t48014/32Z6hUT168/m9Uf\nPAgNGqiJvm9ftYyT3VbAwrgZfKlnyZIlrFmzBkVRmDhxItOmTXsWiCT+F8vcB3fTJujaVeuIRD4k\nJcFvvz2b1cfGqtW6vn2hVy/D6Hgpig6DLvWcO3eONWvWcOLECczMzOjbty8DBgygbt26+gqhaDLE\nPrgiVxQFIiKeJfojR9QNUvv2Vd/HnZ2l06XQL739d7t48SJt27aldOnSmJqa0rVrV3766Sd9DV80\nBQSon/Vfew127ZKkX4TEx6udLKdMUfe16dkTzp2DCRPUTc0OH4a5c6FlS0n6Qv/0NuNv0qQJc+fO\nJTY2ltKlS/Pf//6XNm3aPPcYb29v3W0XFxfj7TGanAzvvguBgepX27ZaRySyoSjqBqhPZ/UnT6r7\n3PTtq75nOzrK7pTFUXJaMuYl9XuFfFBQEEFBQfk6hl5r/OvWrWP58uVYWFjQuHFjzM3NWbx4sRqI\n1PhVmfvgrl2rlniEQYqNVZdY7t2rnpy1sHh2UtbFRf1eFF97L+9l6NahfN37aya1nISJRu/sBn9y\nN7P333+fGjVq4OXlpQYiiV+98nbaNPD2lj64Big9XZ3JP53Vnz+vnmfv21c9OSv74RmPuCdxNF3R\nlHld5rH8xHJeqfQKqwaswqq0ld5jMfjEf+/ePezs7Lh58yZ9+vTh+PHjlC9fXg3EmBP/0z64v/2m\n7qjp5KR1ROL/paaqs3k/P/XPypXVRN+vH3TqJPvgGSuPAA8sS1mytP9SnqQ9Ydb/ZrH38l78h/rT\nulprvcZi0Kt6ANzc3IiJicHMzIzly5frkr5Ry9wH9+RJ6YNrIE6fBl9f2LwZ6tZVd8VYuFBtOiKM\nW+ClQI7ePMoZrzMAlC5ZmmX9l7HtwjZe3fwq73V6j+ntpmtW+skJuYBLK9IH1+DcvateKuHrC3Fx\nMHasmvAbNNA6MmEoYhJjaLqiKVvcttC5Zud//Pzag2uM3D4Sewt71g9aj03Zwt84yeBLPS9jVIlf\n+uAajCdP1IVTvr5w9Ki6o7WHh1q7l2WW4u9GbR9FlXJV+LrP1y98TEp6Cu8feJ+t57eyeehmOtXo\nVKgx5SV3yn9tfQsLUxdvW1jA8eOS9DWgKOrOll5e6t43q1ap/Wtu31b3vOvWTZK++KdtF7YRdieM\nT7t/+tLHlTItxZe9v2TFqytw2+rGZ4c/I0PJ0FOUOSMzfn1RFFi2TN0+WfrgauLGDdi4ETZsUKtq\nHh7qfnfSwkBk517CPZqtaEbAyADaObTL8fNuPbqF+3Z3zEuas3HIRiqXq1zgsUmpx1A97YN78yb4\n+8u6Pz16/Bi2b1dLOeHh6nl0Dw/1gmg5pSJyQlEUhm4dSgObBizsuTDXz0/LSOOjQx+xJnQNG4Zs\noGedngUan5R6DFFwMLRooU4rjx6VpK8HGRnw669qgndwgG3b1A1Nb9+G5cvVC6El6Yuc2nx2MxEx\nEcx3mZ+n55csUZKPun3ExiEb8Qjw4INfPyAtI62Ao8wdmfEXlowMtQfuV1+pReRBg7SOqNi7dEkt\n42zcqHah8vAAd3ews9M6MlFURcVH4bTSiT2j99Cyast8H+/u47uM+XkMSWlJ+A31w6G8Q76PKTN+\nQ/HXX/Dqq+ouXSdOSNIvRA8ewIoVajeqrl3VVTq7dqnn0KdPl6Qv8k5RFCbtnMSU1lMKJOkD2Jez\nZ+/re+lfrz8tV7VkV8SuAjlubsmMv6AFBalnDKUPbqF5ejWtr6+6V06fPursvndvaVoiCs76sPV8\nG/Itxyccp5RpqQI//tGbR3H/yR03RzcW9FiQ5zHk5K6WpA9uoTt9Wi3lPL2aduxY9WSttbXWkYni\nJvJhJC1WteDA2AM0s29WaOPEJMbgucOT6MfR+Lv5U8e6Tq6PIYlfK5n74G7aJH1wC1BWV9OOHQv1\n62sdmSiuFEWhz6Y+dK3Zlbld5uplvCXHl/DZ4c9Y1n8ZwxoPy9XzJfFrYe9e8PSUPrgFKC0N9uyB\ndevUytmgQXI1rdCf709+z9qwtRwbf4ySJfRXOzwZdZIR20bQu25vvu79NWXMyuToeZL49Un64Ba4\niAh1+yJfX7UdwfjxailH9q0T+nLtwTXarGnDb+N+o5FtI72P//DJQybtmsTF+xfZ4raFhpUaZvsc\nWdWjLzduqIk+PFztgytJP88SEtRE36ULdO6szvb374djx9TEL0lf6EuGkoHnDk9md5ytSdIHsCpt\nhf9Qf95q/Rad13dmw5kNhTKOzPhzKyAAJk+Gf/8bZs6U2kMeKIq6TdG6derFVR07whtvwIABsghK\naOfb49+y5fwWfhv3G6YltC/Znr17luHbhtOmWhuW9V9GuVLlsnyclHoKU3Kymux37lS3XZA+uLl2\n755aFVu7FlJS1GTv4QFVq2odmTB2ETERdFjbgd/H/059G8NZOZCQksDUPVP5PfJ3tg7bmuUKIyn1\nFJY//4QOHdRr/sPCJOnnQloa7N4NQ4eq+9qHh6sXXEVEwHvvSdIX2kvPSGdcwDg+7PqhQSV9AItS\nFqwftJ65nefSY0MPVp5cWSATZL0m/gULFtC4cWOaNm2Ku7s7ycnJ+hw+bzZvVpP++PFqXUKan+fI\n5cswd656kvajj9TLGm7eVC9x6NJF9soRhmNx8GLMS5oztc1UrUN5oTHNx3DE8wgrTq5gxLYRPHzy\nMF/H01viv379OqtXryY0NJSzZ8+Snp6Ov7+/vobPvcREmDBBbXy+b5+6XFOy1UslJqr75Li4qO+V\nT56oq12Dg2HSJJBOm8LQXPjrAouOLmLdwHWUMDHsAsgrlV7h+ITj2FrY0mJVC07cPpHnY+ltkWr5\n8uUxMzMjMTERU1NTEhMTqVatmr6Gz53z52HECLUP7qlTsrTkJRRF3Y5o3TrYulXdM+ftt8HVFUoV\n/FXuQhSYtIw0xgWM4+NuH1PburbW4eRIVv1980Jvib9ixYrMmjWLGjVqUKZMGfr06UPPns/vS+3t\n7q677dKkCS5Nm+orvGcuX4bPPpM+uNm4f//ZidrERPVEbXi4ug2yEEXBoiOLqFC6ApNbTtY6lFwJ\nCgriXNA53J+489WCr/J0DL2t6rly5Qqurq4cPnwYKysrhg0bhpubG6NHj1YDMTFBGTBAH6G8XJky\nanlHWiL+Q3q6WvVau1b909VVPfXRpYusahVFy5noM/Tc2JPQSaFUt6qudTh5lpKegnlJ81yf8NXb\njP/kyZN06NABGxu16/xrr73GsWPHdIkfUJdKCoNz9ap6Ra2Pj7oN0RtvwJo1YGWldWRC5F5Kegrj\ndozj856fF+mkD+R5R0+9zdMaNmxIcHAwSUlJKIrC/v37cZRZtcFKSwM/P+jeXV29Gh+vLssMCVGb\nlEvSF0XVp4c/pZplNcY5jdM6FM3obcbfvHlzxo4dS6tWrShRogQtWrRg0qRJ+hpe5JCiwM8/q0sx\nbW3VE7UDB4K5udaRCZF/p6JOseLECk57ncbEiM/fyZW7QufgQZgzR71IeeFCde29Ef9uiGImOS2Z\nlqta8l6n9xjdbHT2Tygi8pI7pV+RICxMvYr2zz/VXjIjRsjJWlH8/CfoPzSwaYB7U/fsH1zMya+3\nEbt8GUaNgv791XLOH3+o30vSF8VN8K1gfE77sHLASqMu8Twlv+JG6M4d9ULkdu2gcWN1pv/mm3LB\nlSieklKTGBcwju/6fYedhZ3W4RgESfxG5OFD9aRtkybq5QoXL8IHH0C5rHd7FaJYmPvrXJwqO+W6\npWFxJjV+I5CUBMuWqRcjDxig1vRr1NA6KiEK3+Ebh/E/58/ZKWe1DsWgSOIvxtLS1O5W3t7QqpXa\nv1YunRDG4nHKY8btGMfKASuxKWujdTgGRRJ/MZR5Lb69Pfz4o1rPF8KYzNk/h041OjHwlYFah2Jw\nJPEXM5nX4i9eLGvxhXE6cPUAOy7tkBLPC0jiLyZkLb4QqkfJjxgfOJ7VrqupUFoaJ2VFUkMR93Qt\n/quvylp8IQBm/W8Wver2om+9vlqHYrAkPRRRmdfiN2kia/GFANjz5x72XdnHV73ztk+9sZDEX8Rk\nXotftixcuqR+b2GhdWRCaOtB0gMm7ZrEukHrKG8ufT5fRhJ/EZGUBF9+CfXrQ3S0WtP/8kuwkVVq\nQgAw/ZfpDHplEN1rd9c6FIMnJ3cNXOa1+K1by1p8IbISeCmQozePctrrtNahFAmS+A2UrMUXImdi\nEmPw2uXFFrctlCsl+4/khOzHb4BCQtQGKCkpsGCBrMUX4mVGbhtJVcuqfN3na61D0URecqfeavyX\nLl3C2dlZ92VlZcW3336rr+GLBEVRL7pydVUT/6lT0LevJH0hXuTH8z9yOvo0n3b/VOtQihRNZvwZ\nGRlUq1aNkJAQqldXmx0b+4w/Lk5tYn7rFmzZArVrax2REIbtXsI9mq1oRsDIANo5GG8d1KBn/Jnt\n37+funXr6pK+sTt1Clq0AAcHOHxYkr4Q2VEUBa9dXoxzGmfUST+vNDm56+/vj7v7P9ufeXt76267\nuLjg4uKiv6A0oCiwfDnMn69umzxMtgsXL7Hvyj6ORh7VOgyDEP04moiYCPyG+mkdit4FBQURFBSU\nr2PovdSTkpJCtWrVuHDhAra2ts8CMbJSz6NHMHEiRESoK3bq1dM6ImGoUtJTeO/Ae/x4/kfGOY2j\nhIlcfgPg3tSdBjYNtA5Dc0Wi2fqePXto2bLlc0nf2Jw5o87uu3eH33+H0qW1jkgYqqsPrjJy20js\ny9kTNjlM9pUXBULvUwc/Pz9GjRql72ENgqLA6tXQs6da3lm5UpK+eLFtF7bRbk073Ju6EzgyUJK+\nKDB6LfUkJCRQs2ZNrl27hqWl5fOBFPNSz+PH4OWlzvZ//BEaNtQ6ImGonqQ9YeYvM/nlyi/4D/Wn\ndbXWWockDJjBr+qxsLDg/v37/0j6xd358+p2C6VKwfHjkvTFi126f4m2a9oSkxRD6KRQSfqiUMhZ\nokLm6wsuLmpXrHXr1B01hcjKxjMb6bS+E2+2ehP/of5YlbbSOiRRTMlePYUkMRGmTlVP3h48qG6j\nLERWHqc8ZuruqRy/fZwDYw/QzL6Z1iGJYk5m/IXg4kVo2xZSU+HECUn64sXC74bTerVazjkx8YQk\nfaEXkvgL2ObN0LkzTJsGGzZAOdksUGRBURS+P/k9PTb04L1O7+Ez2Ed2lhR6I6WeAvLkiZrsf/0V\n9u0DJyetIxKG6uGTh0zaNYmL9y9y2PMwDSvJ2X6hXzLjLwCXL0P79upGa6dOSdIXL3Yy6iQtVrXA\npowNweODJekLTUjiz6dt26BDB3X7BX9/KC+tPkUWFEVh8e+L6f9Dfxb2WMjyV5dTxqyM1mEJIyWl\nnjxKToZ33oHdu2HPHmjZUuuIhKGKSYzBc4cn0Y+jCZ4QTB3rOlqHJIyczPjz4No16NRJ3Tv/1ClJ\n+uLFjtw8gvP3ztS3qc+RN45I0hcGQRJ/Lu3YoS7VHD0afvoJKlTQOiJhiDKUDBYcXsDQrUNZ/upy\nvur9FaVMS2kdlhCAlHpyLDVVvfp2+3YIDJTG5+LF7j6+y5ifx5CUlsTJiSepbiUNh4RhkRl/Dty8\nCV26qHvnh4ZK0hcvduDqAVqsakFbh7Yc9DgoSV8YJEn82di9G9q0gSFD1DJPxYpaRyQMUVpGGvMO\nzmPMz2PwHezLx90+pmQJ+UAtDJP8z3yBtDSYNw82bVKXbHbqpHVEwlDdenQL9+3umJc0J3RyKJXL\nVdY6JCFeSmb8Wbh9W+2OFRqqfknSFy/y34j/0mpVK/rW68svr/8iSV8UCTLj/5t9+2DsWHjrLXj/\nfSghb40iC5n74G4bvo1ONWR2IIqOXKW12NhYwsPD8zxYXFwcbm5uNGrUCEdHR4KDg/N8rIKmKLB4\nMXh4qButffCBJH2RtWsPrtFpXSciYiIImxwmSV8UOdmmtq5du/Lo0SNiY2Np2bIlEyZMYMaMGXka\nbNq0afTv358//viD8PBwGjVqlKfjFLTUVLUt4vr16v753bppHZEwVNsubKPtmrbSB1cUadn23HVy\ncuL06dOsWbOGyMhI5s+fT9OmTTl79myuBnr48CHOzs5cvXo160A06rn74AEMGwbm5uDnJ3vtiKxJ\nH1xhqPKSO7Ot8aenp3Pnzh22bt3KJ598ohsot65du4atrS2enp6cOXOGli1bsmTJEspm6kXo7e2t\nu+3i4oKLi0uux8mNy5dhwADo1w++/BJMTQt1uCLl6oOrfPDrB8SnxGsdikGIiImguX1zQieFSktE\noamgoCCCgoLydYxsZ/w//vgjH3/8MR07dmTFihVcuXKFd999l+3bt+dqoJMnT9K+fXuOHTtG69at\nmT59OuXLl+ejjz5SA9HzjP+332D4cPD2Vss84pmt57cydfdUZrSbQRM7aR8GUN68PF1qdsnTpEeI\nwpSX3Jlt4i8o0dHRtG/fnmvXrgFw5MgRFi5cyK5du9RA9Jj4fXxg9mx1jX6vXnoZskhISk1ixi8z\n2H91P/5u/rSq2krrkIQQ2chL7sz25O67777Lo0ePSE1NpUePHlSqVImNGzfmOrjKlStTvXp1IiIi\nANi/fz+NGzfO9XHyIyMD3nsPPvkEDh2SpJ/ZxfsXabumLXFP4gidHCpJX4hiLNvE/8svv1C+fHl2\n7dpFrVq1uHLlCl988UWeBvvuu+8YPXo0zZs3Jzw8nPfffz9Px8mLhARwc4OjRyE4GBpK4yMd39O+\ndF7fmbfbvI3fUD/Km8sZbiGKs2xP7qalpQGwa9cu3NzcsLKyynOds3nz5pw4cSJPz82P27dh4EBo\n2lRduWNurvcQDNLjlMe8tfstTtw+wa9jf6WpfVOtQxJC6EG2M35XV1caNmzIqVOn6NGjB/fu3aN0\n6dL6iK1APN1Nc9gwdZ2+JH1V+N1wWq1qhamJKScmnpCkL4QRydHJ3djYWKysrDA1NSUhIYH4+Hgq\nVy7YPUkK4+RuQIDaC3flShg6tEAPXWQpisL3p75n3sF5LO6zmNebva51SEKIfCiUdfwJCQksW7aM\nmzdvsnr1aqKiorh06RIDBgzIc6CFTVHgiy/g22/Vfrit5DwlAA+fPGTizon8GfsnR984SgObBlqH\nJITQQLalHk9PT0qVKsWxY8cAqFq1KnPnzi30wPIqJQXGj1dr+cHBkvSfOnH7BM7fO2NnYcfv43+X\npC+EEcs28V+5coXZs2dTqpTaL9TCwqLQg8qrmBh1iWZsLBw+DA4OWkekPUVRWPz7Yl7d/Cpf9PqC\npf2XUrpk0TlHI4QoeNmWeszNzUlKStJ9f+XKFcwN8AzppUvq9guvvQYLFsjOmgAxiTGM2zGOewn3\nOD7hOLWta2sdkhDCAGSbHr29venbty+3bt3C3d2d7t27s2jRIn3ElmMHDqg9cefMgUWLJOkDHL5x\nGOfvnWlYqSGHPQ9L0hdC6ORoVc/9+/d1e+e3a9eOSpUqFXwgeVzVs3q1unf+li1QyHu6FQnpGeks\nPLKQ70K+Y92gdfSv31/rkIQQhahQVvUAJCcnY21tTVpaGhcuXACgS5cuuY+wAKWnw7vvws6dcOQI\n1K+vaTgGIfpxNGN+HkNKegqnJp2iWvlqWockhDBA2Sb+2bNns2XLFhwdHTHNtG+xlok/Ph7c3dVt\nGIKDoWJFzUIxGPuv7mfsz2OZ2HIi87rMo2QJ6aophMhatqWeBg0acPbs2UI/oZvTjys3b4KrK7Rp\nA8uXg5lZoYZl8NIy0vAO8mb96fVsHLKR7rW7ax2SEEKPCqXUU7duXVJSUgxiJU9ICAwZAjNnql/G\nvjV65MNI3H9yp6xZWUInhWJfzl7rkIQQRUC2ib9MmTI4OTnRo0cPXfI3MTHh22+/LfTgMtu6Fd56\nC9auVTdcM3Y7L+1k4s6JTG83nXc7vksJE1nKJITImWwT/6BBgxg4cKBuR05FUfTahUhR4NNPYdUq\n2LcPnJz0NrRBSklPYc7+OWz/Yzs/jfiJDtU7aB2SEKKIyTbxP3jwgOnTpz933zfffFNoAWWWnAwT\nJsDFi3D8OFSpopdhDdbVB1cZsW0EVS2rEjY5jIpl5Ky2ECL3sq0P+Pr6/uM+Hx+fwojlOffuQffu\n8OSJ2i3L2JP+1vNbabemHWOajSFgRIAkfSFEnr1wxu/n58fmzZu5du0arq6uuvvj4+OxsbHJ02C1\natWifPnymJqaYmZmRkhISJaPO39eXbkzejTMn2/cV+ImpSYx838z2XdlH3tG76Fl1ZZahySEKOJe\nmPg7dOhAlSpV+Ouvv3jnnXd0y4UsLS1p3rx5ngYzMTEhKCiIii9ZeP/LLzBmDHz1lfqnMbt4/yIj\nto3A0daR0Mmh0hJRCFEgcrRlQ0GpXbs2J0+ezPITg4mJCUuXKnzyCfz4I3TqpK+oDJPvaV/e2fcO\nC3osYLzzeL2eUBdCFB0Fuo6/Y8eOHD16lHLlyv0j6ZiYmPDo0aM8BdizZ09MTU2ZPHkyEydOfO7n\n3t7euLvD/v2QluaCi5FuvvPpb5/yw9kfpA+uEOIfgoKCCAoKytcxXjjjv3HjBjVr1szXwf/uzp07\nuvJRr169+O677+jcubMaiIkJDx4oVKhQoEMWOaeiTtHvh36c9jpNVcuqWocjhDBweZnxv/C06ZAh\nQ3S3hxZQw9oq/780x9bWliFDhvzj5K6xJ/3ktGQ8AjxY3GexJH0hRKF5YeLP/A5y9erVfA+UmJhI\nfHw8oPbx/d///kfTplLGyMz7kDcNbBrg3tRd61CEEMWY3rZwvHv3ru5TRFpaGqNHj6Z37976Gt7g\nBd8KZn3YesKnhMuJXCFEoXphjd/U1JSyZcsCkJSURJkyZZ49KY8nd18aSB4bsRQHSalJOH/vzMfd\nPmZY42FahyOEKELykjv1upzzZYw58c/8ZSZR8VH4u/lrHYoQoogptA5covAcvnEY/3P+nJ1yVutQ\nhBBGwog3Q9BeQkoCnjs8WTlgJTZl87YNhhBC5JaUejQ0dfdU4lPi8R38z43whBAiJ6TUU4QcuHqA\nHZd2EO4VrnUoQggjI6UeDTxKfsT4wPGsdl2NdRlrrcMRQhgZKfVoYOJOdY+i1a6rNY5ECFHUSamn\nCNh7eS/7ruwjfIqUeIQQ2pDEr0dxT+KYuHMiPoN8ZG99IYRmpNSjRx4BHpQrVY5l/ZdpHYoQopiQ\nUo8BC7w7NJ/SAAAaEklEQVQUyNGbRzntdVrrUIQQRk4Svx7EJMbgtcuLLW5bKFeqnNbhCCGMnJR6\n9GDU9lFUKVeFr/t8rXUoQohiRko9BmjbhW2E3Qlj3eR1WocihBCAJP5CdS/hHlN3TyVgZABlzMpk\n/wQhhNADvV65m56ejrOzM66urvocVhOKouC1y4txTuNo59BO63CEEEJHrzP+JUuW4OjoqGvBWJz5\nnfMjIiYCv6F+WocihBDP0duM/9atW+zevZsJEyYU25O4T0XFRzHjlxn4DvbFvKS51uEIIcRz9Dbj\nnzFjBl988cVLWzZ6e3vrbru4uODi4lL4gRUwRVGYtHMSXq28aFm1pdbhCCGKmaCgIIKCgvJ1DL0s\n59y1axd79uxh2bJlBAUF8dVXX7Fz587nAykmyznXh63n25BvOT7hOKVMS2kdjhCimDPY5ZzHjh0j\nMDCQ3bt38+TJEx49esTYsWPZsGGDPobXm8iHkby7/10OjD0gSV8IYbD0fgHXoUOH+PLLL4vdjF9R\nFPps6kPXml2Z22Wu1uEIIYxEXnKnJo1YTExMtBi2UK06tYq4J3HM7jRb61CEEOKlZMuGAnDtwTXa\nrGnDoXGHcLR11DocIYQRKTIz/uIkQ8nAc4cn73Z4V5K+EKJIkMSfT0tDlpKakcrM9jO1DkUIIXJE\nSj358GfMn7Rf257fx/9OfZv6WocjhDBCUurRo/SMdMbtGMeHXT+UpC+EKFIk8efR4uDFmJUwY2qb\nqVqHIoQQuSLbMufBhb8usPDIQkImhlDCRN47RfFSsWJFHjx4oHUY4m+sra2JjY0tkGNJ4s+ltIw0\nxgWM45Pun1DHuo7W4QhR4B48eFDkzrcZg4K8/kmmq7n0+dHPqVC6ApNbTtY6FCGEyBOZ8edC+N1w\nvgn+hlOTThXLq4+FEMZBZvw5lJKegkeAB4t6LqK6VXWtwxFCiDyTxJ9Dnx7+lKqWVRnnNE7rUIQQ\nIl+k1JMDp6JOseLECk57nZYSjxCiyJMZfzaS05LxCPBgcZ/FVLWsqnU4QgiRb5L4s+F9yJsGNg1w\nb+qudShCCD1IS0vTOoRCJ4n/JYJvBbM+bD0rB6yUEo8QBmLhwoXUq1eP8uXL07hxYwICAkhOTqZC\nhQqcP39e97i//vqLsmXLcv/+fUBtAevk5IS1tTUdO3bk7NmzusfWqlWLzz//nGbNmmFpaUl6enqW\n4zyVkZHBrFmzsLW1pU6dOixdupQSJUqQkZEBwMOHDxk/fjxVq1bFwcGBefPm6X5mEBQDYUChKIqi\nKIkpicor372ibD23VetQhNArQ/td/Lsff/xRuXPnjqIoirJlyxbFwsJCuXPnjvLGG28oc+fO1T1u\n6dKlSr9+/RRFUZTQ0FDFzs5OCQkJUTIyMhRfX1+lVq1aSkpKiqIoilKzZk3F2dlZuXXrlvLkyZMX\njhMdHa0oiqKsWLFCcXR0VG7fvq08ePBA6dGjh1KiRAklPT1dURRFGTx4sOLl5aUkJiYq9+7dU9q0\naaN8//33+fp7v+jfJS//Xnr7F05KSlLatGmjNG/eXGnUqJEyZ86c5wMxsP9sM/bOUEb8OELrMITQ\nu+x+F6FgvgqKk5OTsmPHDmX//v1K3bp1dfd36NBB2bhxo6IoiuLl5aXMmzfvuee98sorym+//aYo\niqLUqlVLWb9+fbbjBAYGKoqiKN26dVNWrVql+9n+/fsVExMTJT09XYmOjlbMzc2VpKQk3c83b96s\ndOvWLV9/z4JM/Hpb1VO6dGkOHjxI2bJlSUtLo1OnThw5coROnTrpK4QcO3zjMP7n/Dk75Wz2DxbC\nyGi9m8OGDRtYvHgx169fB+Dx48fExMQwYMAAEhMTCQkJwc7OjjNnzjBkyBAAbty4wYYNG/juu+90\nx0lNTSUqKkr3ffXq1bMd52nZ6M6dO8893sHBQXf7xo0bpKamUqVKFd19GRkZ1KhRo2BegAKg1+Wc\nZcuWBSAlJYX09HQqVqyoz+FzJCElAc8dnqwcsBKbsjZahyOEyOTGjRtMmjSJX3/9lfbt22NiYoKz\nszOKolCiRAmGDx+On58fdnZ2uLq6YmFhAUCNGjWYO3cu77///guPnfk83svGAahSpQqRkZG6x2e+\nXb16dczNzYmJiaFECcM8jarXxJ+RkUGLFi24cuUKU6ZMwdHx+VaF3t7eutsuLi64uLjoMzwAZu+f\nTccaHRn4ykC9jy2EeLmEhARMTEyoVKkSGRkZbNiwgXPnzul+7u7uzqBBg6hUqRKfffaZ7v6JEycy\nZMgQevbsSevWrUlMTCQoKIiuXbtSrly5XI8zfPhwlixZwquvvkrZsmVZtGiR7o2jSpUq9O7dm5kz\nZ/Lxxx9jYWHBtWvXuH37Nl26dMn3axAUFERQUFD+DpKvolMexcXFKW3btlUOHjyou0+jUJ6z/8p+\nxeFrByU2MVbrUITQjCH8Lr7M3LlzlYoVKyqVKlVSZs6cqbi4uChr167V/bxevXqKjY2Nkpqa+tzz\n9u7dq7Ru3VqpUKGCUqVKFWX48OHK48ePFUVRa/wHDhzI8ThpaWnKjBkzFBsbG6VOnTrK4sWLFTMz\nM91zHz58qEyZMkVxcHBQrKysFGdnZ2XLli35+nu/6N8lL/9emrVe/PjjjylTpgzvvPMOoH3rxUfJ\nj2i2ohkrB6ykb72+msUhhNa0/l0sivbs2cOUKVN05wMKw4v+XQy69eL9+/eJi4sDICkpiX379uHs\n7Kyv4bP1zv/eoVfdXpL0hRDZevLkCbt37yYtLY3bt28zf/58XnvtNa3DyjG91fjv3LmDh4cHGRkZ\nZGRkMGbMGHr06KGv4V9q7+W9/O/K/wifEq51KEKIIkBRFLy9vRk5ciRlypRhwIABfPTRR1qHlWOa\nlXr+TquPl3FP4mi6oik+g3zoUccw3oiE0JKUegxTQZZ6jD7xewR4UK5UOZb1X6b3sYUwRJL4DVNB\nJn6j3pY58FIgR28e5bTXaa1DEUIIvTHaxB+TGIPXLi+2uG2hXKl/ruMVQojiymhLPaO2j6JKuSp8\n3edrvY0pRFEgpR7DJKWefNp2YRthd8JYN3md1qEIIYTeGeZGEoXoXsI9pu6eis9gH8qYldE6HCFE\nIbh58yaWlpY5mgnn5rHFhVGVehRFYejWoTSwacDCngsLdSwhiiop9RgmKfXkkd85PyJiIvAb6qd1\nKEIIoRmjKfVExUcx45cZ+A72xbykudbhCCHyoFatWnz55Ze6Fonjx4/n7t279OvXDysrK3r16kVc\nXBzXr19/rhWii4sLH374IZ06daJ8+fL06dOHmJgYgCwfO2/ePDp27IilpSUDBw7k/v37jB49Gisr\nK9q0acONGzeyfO7T569duxYAHx8fOnbsyMyZM7G2tqZevXocO3aM9evXU6NGDezt7dmwYYM+X0LA\nSBK/oihM2jkJr1ZetKzaUutwhBB5ZGJiwk8//cSBAwe4dOkSu3btol+/fixcuJB79+6RkZHBt99+\nm+Vz/fz88PHx4d69e6SkpPDll1++cJwtW7awadMmbt++zZUrV2jfvj3jx48nNjaWRo0aMX/+/JfG\nmHlv/5CQEJo3b05sbCyjRo1i+PDhhIaGcuXKFTZt2sTUqVNJTEzM+4uSB0ZR6vE57cPt+Nv81Pkn\nrUMRosgzmW+S/YNyQPlP3s4jvP3229ja2gLQuXNn7O3tad68OQBDhgzhwIEDeHh4PPccExMTPD09\nqVevHqDupx8YGJjl8Z8+tnbt2gD069ePP/74g+7duwMwbNgw5s2bl+N4a9eurYtn+PDhfPrpp3z4\n4YeYmZnRq1cvSpUqxeXLl2nWrFkuXoX8KfaJP/JhJO/uf5cDYw9QyrSU1uEIUeTlNWEXFHt7e93t\nMmXKPPd96dKlefz4cZbPq1y58nPPe9Hj/j5G6dKlsbOzy9EYOYkX0L1x5SSWwlCsSz2KojA+cDzT\n206nmb3+3k2FEPpT2CuQMpdt/u5pa8fMpZro6OhCjacgFOvEv+rUKuKexDG702ytQxFCaCw3bxCZ\nH/uy59na2lKtWjU2btxIeno669at48qVK/mKUx+KbeK/9uAaHxz8AJ/BPpQsUewrWkIYrcwz8swn\nVv8+U3/R43L72L//fPXq1XzxxRdUqlSJCxcu0LFjxxw/VyvF8gKuDCWD7r7debX+q/y7478L5JhC\nGAu5gMswFcnWi5GRkXTr1o3GjRvTpEmTFy65KghLQ5aSmpHKzPYzC20MIYQoqvQ244+OjiY6Ohon\nJyceP35My5YtCQgIoFGjRmogBTTLiIiJoMPaDvw+/nfq29TP9/GEMDYy4zdMRXLGX7lyZZycnAAo\nV64cjRo1IioqqkDHSM9Ix3OHJx92/VCSvhBCvIAmZz2vX79OWFgYbdu2fe5+b29v3W0XFxdcXFxy\nddzFwYsxK2HG1DZTCyBKIYQwPEFBQQQFBeXrGHo/ufv48WNcXFz44IMPGDx48LNA8vnx8sJfF+iy\nvgshE0OoY12nIEIVwihJqccwFclSD0BqaipDhw7l9ddffy7p51daRhrjAsbxSfdPJOkLIUQ29Jb4\nFUVh/PjxODo6Mn369AI99qIji6hQugKTW04u0OMKIURxpLdSz5EjR+jSpQvNmjXTXcCwYMEC+vbt\nqwaSx4+X4XfD6bGhB6GTQqluVb1AYxbCGEmpxzAVZKmnSF/AlZKeQts1bflXm3/h6exZSJEJYVyM\nPfEvWLCAq1evsnr1aq1DeY4k/v/3n6D/cDLqJLtG7TKIy6CFKA6MPfEbKmm9CJyKOsWKEys47XVa\nkr4QQuRCkdykLTktGY8ADxb3WUxVy6pahyOE0JOctl4EtWFKlSpVqFChAl27duXChQsApKSk4Ozs\nzNKlSwFIT0+nY8eOfPLJJ4B6PdGYMWOAZ60VfXx8qFGjBjY2NqxcuZITJ07QrFkzrK2tefvtt3Xx\nZX5u5ufnpa1jYSqSid/7kDcNbBrg3tRd61CEEHqUm9aL/fv35/Lly/z111+0aNGC0aNHA1CqVCk2\nbdrEhx9+yMWLF1m4cCGKojB37lzdGH8XEhLC5cuX8ff3Z9q0aXz22Wf8+uuvnD9/nq1bt/Lbb7+9\n8Ll/l5+2jgWlyJV6gm8Fsz5sPeFTwqXEI4QWCur3Lo/nEXLSehHA0/PZgo///Oc/LFmyhPj4eCwt\nLWncuDEffPABgwYN4v79+4SEhOjySVb18nnz5lGqVCl69eqFpaUl7u7uVKpUSRdDWFgYXbp0ybbW\nXtBtHfOqSM34k1KTGBcwju/6fYedhV32TxBCFDxFKZivPMpJ68WMjAzmzJlDvXr1sLKyonbt2piY\nmHD//n3dY8eOHcvNmzfp378/devWzfOYZcqUISEhIU/x57etY14VqcQ/99e5OFV2YljjYVqHIoQw\nEFnNsjdv3kxgYCAHDhzg4cOHXLt2DUVRnnvsm2++yYABA9i7dy9Hjx7V3Z+fSkK5cuVy1YZRq6pF\nkSn1HL5xGP9z/pydclbrUIQQBi4+Ph5zc3MqVqxIQkIC77///nM/37hxI2FhYZw5c4YdO3bg4eHB\nmTNnsLCwyNNS1qfPcXJyYtGiRURGRlK+fHkWLFjwwsf+/bY+FYkZf0JKAp47PFk5YCU2ZW20DkcI\nYUCyapM4duxYatasSbVq1WjSpAnt27fXPe7mzZvMmDGDDRs2ULZsWUaNGkWrVq2YOXPmc8fI6vjZ\nxdCzZ09GjBhBs2bNaN26Na6urvlq61hYisQFXFN3TyU+JR7fwb56jkoI4yMXcBkmo7qA68DVA+y4\ntINwr3CtQxFCiGLBoEs9j5IfMT5wPKtdV2NdxlrrcIQQolgw6FLPxJ0TAVjtalibJQlRnEmpxzAZ\nRaln7+W97Luyj/ApUuIRQp+sra3l4kgDZG1dcFUPg0z8cU/imLhzIj6DfChvXl7rcIQwKrGxsVqH\nIAqZQdb4p+2dxsBXBtKjTg+tQ9FEfhspFyfyWjwjr8Uz8lrkj94S/xtvvIG9vT1NmzZ96eMCLwVy\n9OZRFvVcpKfIDI/8p35GXotn5LV4Rl6L/NFb4vf09GTv3r0vfUxMYgxeu7xYP2g95UqV01NkQghh\nXPSW+Dt37pztyYmpe6YysslIOtfsrKeohBDC+Oh1Oef169dxdXXl7Nl/7rcjqwiEECJviuxyTlk3\nLIQQ+mGQq3qEEEIUHkn8QghhZPSW+EeNGkWHDh2IiIigevXqrF+/Xl9DCyGEyERvid/Pz4+oqCiS\nk5OJjIx8rh/m3r17adiwIfXr12fRIuNdvx8ZGUm3bt1o3LgxTZo00TWNNmbp6ek4Ozvj6uqqdSia\niouLw83NjUaNGuHo6EhwcLDWIWlmwYIFNG7cmKZNm+Lu7k5ycrLWIelNVtdDxcbG0qtXLxo0aEDv\n3r2Ji4vL9jial3rS09OZOnUqe/fu5cKFC/j5+fHHH39oHZYmzMzMWLx4MefPnyc4OJhly5YZ7Wvx\n1JIlS3B0dDT6VV/Tpk2jf//+/PHHH4SHh9OoUSOtQ9LE9evXWb16NaGhoZw9e5b09HT8/f21Dktv\nsroeauHChfTq1YuIiAh69OjBwoULsz2O5ok/JCSEevXqUatWLczMzBg5ciQ7duzQOixNVK5cGScn\nJ0Dt3dmoUSOioqI0jko7t27dYvfu3UyYMMGoV309fPiQw4cP88YbbwBQsmRJrKysNI5KG+XLl8fM\nzIzExETS0tJITEykWrVqWoelN1ldDxUYGIiHhwcAHh4eBAQEZHsczRP/7du3qV69uu57BwcHbt++\nrWFEhuH69euEhYXRtm1brUPRzIwZM/jiiy8oUULz/6aaunbtGra2tnh6etKiRQsmTpz4XENvY1Kx\nYkVmzZpFjRo1qFq1KhUqVKBnz55ah6Wpu3fvYm9vD4C9vT13797N9jma/0YZ+0f4rDx+/Bg3NzeW\nLFlCuXLGuXXFrl27sLOzw9nZ2ahn+wBpaWmEhoby5ptvEhoaioWFRY4+zhdHV65c4ZtvvuH69etE\nRUXx+PFjfvjhB63DMhhZ9fDNiuaJv1q1akRGRuq+j4yMxMHBQcOItJWamsrQoUN5/fXXGTx4sNbh\naObYsWMEBgZSu3ZtRo0axa+//srYsWO1DksTDg4OODg40Lp1awDc3NwIDQ3VOCptnDx5kg4dOmBj\nY0PJkiV57bXXOHbsmNZhacre3p7o6GgA7ty5g52dXbbP0Tzxt2rVij///JPr16+TkpLCli1bGDhw\noNZhaUJRFMaPH4+joyPTp0/XOhxNffbZZ0RGRnLt2jX8/f3p3r07GzZs0DosTVSuXJnq1asTEREB\nwP79+2ncuLHGUWmjYcOGBAcHk5SUhKIo7N+/H0dHR63D0tTAgQPx9fUFwNfXN2cTRsUA7N69W2nQ\noIFSt25d5bPPPtM6HM0cPnxYMTExUZo3b644OTkpTk5Oyp49e7QOS3NBQUGKq6ur1mFo6vTp00qr\nVq2UZs2aKUOGDFHi4uK0DkkzixYtUhwdHZUmTZooY8eOVVJSUrQOSW9GjhypVKlSRTEzM1McHByU\ndevWKTExMUqPHj2U+vXrK7169VIePHiQ7XEMpueuEEII/dC81COEEEK/JPELIYSRkcQvhBBGRhK/\nEEIYGUn8oki7e/cu7u7u1K1bl1atWtGhQ4ccXbJeGA4dOsTvv/+u+/77779n48aNmsQixMsYTAcu\nIXJLURQGDx6Mp6cnmzdvBuDmzZsEBgYW2pjp6emYmppm+bODBw9iaWlJ+/btAZg8eXKhxSFEfshy\nTlFkHThwgI8//pigoKB//Cw9PZ05c+Zw6NAhkpOTeeutt5g0aRJBQUF4e3tja2vLuXPnaNmyJZs2\nbQLg1KlTzJo1i8ePH1OpUiV8fHyoXLkyLi4uODs7c+TIEUaNGkWDBg345JNPSElJwcbGhh9++IHE\nxETat2+Pqakptra2fPfdd+zfvx9LS0tmzZrF6dOn8fLyIikpibp167Ju3ToqVKiAi4sL7dq14+DB\ng8TFxbF27Vo6deqk51dSGBsp9Ygi6/z587Ro0SLLn61du5YKFSoQEhJCSEgIq1ev5vr16wCcPn2a\nJUuWcOHCBa5evcrRo0dJTU3l7bffZvv27Zw8eRJPT0/mzp0LqPufpKamcuLECWbOnEmnTp0IDg4m\nNDSUESNG8Pnnn1OrVi28vLyYOXMmYWFhdOrU6bl9U8aOHcsXX3zBmTNnaNq0KfPnz9cdOz09nePH\nj/PNN9/o7heiMEmpRxRZf9+M6q233uLo0aOUKlWKmjVrEh4ezrZt2wB49OgRly9fxszMjDZt2lC1\nalUAnJycuH79OlZWVpw/f16302N6erruMQAjRozQ3Y6MjGT48OFER0eTkpJCnTp1dD/L6gP0o0eP\nePjwIZ07dwbUrXOHDRum+/lrr70GQIsWLXRvTkIUJkn8oshq3Lgx27dv132/bNkyYmJiaNWqFTVr\n1mTp0qX06tXruecEBQVhbm6u+97U1JS0tDTd8V604ZeFhYXu9ttvv80777zDgAEDOHToEN7e3rmK\n++9vDk/jyRyLEIVJSj2iyOrevTtPnjxh5cqVuvsSEhIA6NOnD8uXL9cl0oiIiBfuYW9iYsIrr7zC\nX3/9pWtpmJqayoULF3SPyZysHz16pPs04OPjo7vf0tKS+Pj4546tKArly5fH2tqaI0eOALBx40Zc\nXFzy+LcWIv9kxi+KtICAAGbMmMHnn3+Ora0tFhYWfP7557i5uXHt2jVatGiBoijY2dnx888/v3C/\ncjMzM7Zt28a//vUvHj58SFpaGjNmzNDt/Jj5Od7e3gwbNgxra2u6d+/OjRs3AHB1dcXNzY3AwEBd\nv+Snz/P19cXLy4vExETq1q3L+vXrs/z7SH8KoQ+yqkcIIYyMlHqEEMLISOIXQggjI4lfCCGMjCR+\nIYQwMpL4hRDCyEjiF0III/N/vALe4Mjf4GoAAAAASUVORK5CYII=\n", 360 | "text": [ 361 | "" 362 | ] 363 | } 364 | ], 365 | "prompt_number": 22 366 | }, 367 | { 368 | "cell_type": "code", 369 | "collapsed": false, 370 | "input": [], 371 | "language": "python", 372 | "metadata": {}, 373 | "outputs": [], 374 | "prompt_number": 22 375 | } 376 | ], 377 | "metadata": {} 378 | } 379 | ] 380 | } 381 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | notebooks 2 | ========= 3 | 4 | Notebooks on how to use Distributed Evolutionary Algorithm in Python (DEAP) 5 | 6 | * [**DEAP: Enabling Nimbler Evolutions**](SIGEvolution.ipynb) (published in [SIGEvolution, volume 6 issue 2](http://sigevolution.org)) 7 | * [**Genetic algorithm : OneMax problem**](OneMax.ipynb) 8 | 9 | Notebooks by [Luis Martí](http://lmarti.com) for the graduate course [Advanced Evolutionary Computation: Theory and Practice](http://lmarti.com/aec-2014). 10 | * [Elements of Evolutionary Algorithms](https://github.com/lmarti/evolutionary-computation-course/blob/master/AEC.02%20-%20Elements%20of%20Evolutionary%20Algorithms.ipynb) 11 | * [Using genetic algorithms to solve the traveling salesperson problem](https://github.com/lmarti/evolutionary-computation-course/blob/master/AEC.03%20-%20Solving%20the%20TSP%20with%20GAs.ipynb) 12 | * [Understanding evolutionary strategies and covariance matrix adaptation](https://github.com/lmarti/evolutionary-computation-course/blob/master/AEC.04%20-%20Evolutionary%20Strategies%20and%20Covariance%20Matrix%20Adaptation.ipynb) 13 | * [Evolutionary Multi-Objective Optimization](https://github.com/lmarti/evolutionary-computation-course/blob/master/AEC.06%20-%20Evolutionary%20Multi-Objective%20Optimization.ipynb) 14 | -------------------------------------------------------------------------------- /SIGEvolution.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "metadata": { 3 | "name": "" 4 | }, 5 | "nbformat": 3, 6 | "nbformat_minor": 0, 7 | "worksheets": [ 8 | { 9 | "cells": [ 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "#DEAP - Enabling Nimbler Evolutions\n", 15 | "by Fran\u00e7ois-Michel De Rainville, F\u00e9lix-Antoine Fortin, Marc-Andr\u00e9 Gardner, Marc Parizeau, and Christian Gagn\u00e9\n", 16 | "\n", 17 | "\n", 18 | "\n", 19 | "DEAP is a Distributed Evolutionary Algorithm (EA) framework written in Python and designed to help researchers developing custom evolutionary algorithms. Its design philosophy promotes explicit algorithms and transparent data structures, in contrast with most other evolutionary computation softwares that tend to encapsulate standardized algorithms using the black-box approach. This philosophy sets it apart as a rapid prototyping framework for testing of new ideas in EA research.\n", 20 | "\n", 21 | "##Introduction\n", 22 | "\n", 23 | "The [DEAP framework](#deap-jmlr) is designed over the three following founding principles:\n", 24 | "\n", 25 | "1. Data structures are key to evolutionary computation. They must facilitate the implementation of algorithms and be easy to customize.\n", 26 | "2. Operator selection and algorithm parameters have strong influences on evolutions, while often being problem dependent. Users should be able to parametrize every aspect of the algorithms with minimal complexity.\n", 27 | "3. EAs are usually embarrassingly parallel. Therefore, mechanisms that implement distribution paradigms should be trivial to use.\n", 28 | "\n", 29 | "With the help of its sister project [SCOOP](#scoop) and the power of the Python programming language, DEAP implements these three principles in a simple and elegant design.\n", 30 | "\n", 31 | "###Data Structures\n", 32 | "\n", 33 | "A very important part of the success for designing any algorithm - if not the most important - is choosing the appropriate data structures. Freedom in type creation is fundamental in the process of designing evolutionary algorithms that solve real world problems. DEAP's *creator* module allows users to:\n", 34 | "\n", 35 | "- create classes with a single line of code (inheritance);\n", 36 | "- add attributes (composition);\n", 37 | "- group classes in a single module (sandboxing).\n", 38 | "\n", 39 | "In the following listing, we create a minimizing fitness.\n" 40 | ] 41 | }, 42 | { 43 | "cell_type": "code", 44 | "collapsed": false, 45 | "input": [ 46 | "from deap import base, creator\n", 47 | "creator.create(\"FitnessMin\", base.Fitness, weights=(-1.0,))" 48 | ], 49 | "language": "python", 50 | "metadata": {}, 51 | "outputs": [], 52 | "prompt_number": 1 53 | }, 54 | { 55 | "cell_type": "markdown", 56 | "metadata": {}, 57 | "source": [ 58 | "The `create` function expects at least two arguments; the name of the class to be created and the base class it inherits from. The next arguments are used as class attributes.\n", 59 | "Thus, the class just created is a `FitnessMin` inheriting from the base class `Fitness` and having a `weights` attribute set to the one element tuple `(-1.0,)`, indicating minimization of a single objective. A multi-objective fitness would be created using a multi-element tuple.\n", 60 | "\n", 61 | "Next, we define with the same mechanism an `Individual` class inheriting from a `list` and composed with a `fitness` attribute." 62 | ] 63 | }, 64 | { 65 | "cell_type": "code", 66 | "collapsed": false, 67 | "input": [ 68 | "creator.create(\"Individual\", list, fitness=creator.FitnessMin)" 69 | ], 70 | "language": "python", 71 | "metadata": {}, 72 | "outputs": [], 73 | "prompt_number": 2 74 | }, 75 | { 76 | "cell_type": "markdown", 77 | "metadata": {}, 78 | "source": [ 79 | "When an `Individual` is instantiated, its `fitness` is initialized with an instance of the previously defined `FitnessMin` class. This is illustrated in the following example," 80 | ] 81 | }, 82 | { 83 | "cell_type": "code", 84 | "collapsed": false, 85 | "input": [ 86 | "ind = creator.Individual([1,0,1,0,1])\n", 87 | "ind.fitness.values = (sum(ind),)" 88 | ], 89 | "language": "python", 90 | "metadata": {}, 91 | "outputs": [], 92 | "prompt_number": 3 93 | }, 94 | { 95 | "cell_type": "markdown", 96 | "metadata": {}, 97 | "source": [ 98 | "where an individual is created from a list of binary values and the value of its fitness is set to the sum of its elements.\n", 99 | "In DEAP, the fitness value is always multi-objective with the single objective case being a tuple of one element. This tuple is instantiated by the comma following the sum operation.\n", 100 | "\n", 101 | "###Operators\n", 102 | "\n", 103 | "Operator selection is another crucial part of evolutionary algorithms. It must be straightforward and its parametrization intuitive. DEAP's *Toolbox* enables users to:\n", 104 | "\n", 105 | "- create aliases for operators;\n", 106 | "- register operators' parameters;\n", 107 | "- interchange operators efficiently;\n", 108 | "- regroup all operators in a single structure.\n", 109 | "\n", 110 | "The next example presents the construction of a toolbox and how operators and their parameters are registered." 111 | ] 112 | }, 113 | { 114 | "cell_type": "code", 115 | "collapsed": false, 116 | "input": [ 117 | "from deap import tools\n", 118 | "toolbox = base.Toolbox()\n", 119 | "toolbox.register(\"mate\", tools.cxOnePoint)\n", 120 | "toolbox.register(\"mutate\", tools.mutGaussian, mu=0.0, std=1.0)" 121 | ], 122 | "language": "python", 123 | "metadata": {}, 124 | "outputs": [], 125 | "prompt_number": 4 126 | }, 127 | { 128 | "cell_type": "markdown", 129 | "metadata": {}, 130 | "source": [ 131 | "The `register` function expects at least two arguments; the alias of the function and the function itself. The next arguments are passed to the function when called, similarly to the `partial` function from the standard *functools* module.\n", 132 | "Thus, the first operator is a one point crossover registered under the alias `mate`. The second operator, a gaussian mutation, is registered with its parameters under the generic name `mutate`. Both operators are available from the *tools* module along with many more instruments to support evolution presented at the end of this paper.\n", 133 | "\n", 134 | "During subsequent experiments, replacing the one point crossover by a two point crossover is as easy as substituting the third line of the previous listing by the following one." 135 | ] 136 | }, 137 | { 138 | "cell_type": "code", 139 | "collapsed": false, 140 | "input": [ 141 | "toolbox.register(\"mate\", tools.cxTwoPoint)" 142 | ], 143 | "language": "python", 144 | "metadata": {}, 145 | "outputs": [], 146 | "prompt_number": 5 147 | }, 148 | { 149 | "cell_type": "markdown", 150 | "metadata": {}, 151 | "source": [ 152 | "Wherever the generic function `mate` is used, the new two point crossover will be used.\n", 153 | "\n", 154 | "###Parallelization\n", 155 | "DEAP is parallel ready. The idea is to use a mapping operation that applies a function to every item of a sequence, for instance to evaluate the individuals fitness. By default, every toolbox is registered with the standard `map` function of Python. For algorithms to evaluate individuals in parallel, one only needs to replace this alias by a parallel map such as the one provided by [SCOOP](#scoop), a library capable of distributing tasks over a cluster of computers." 156 | ] 157 | }, 158 | { 159 | "cell_type": "code", 160 | "collapsed": false, 161 | "input": [ 162 | "from scoop import futures\n", 163 | "toolbox.register(\"map\", futures.map)" 164 | ], 165 | "language": "python", 166 | "metadata": {}, 167 | "outputs": [], 168 | "prompt_number": 6 169 | }, 170 | { 171 | "cell_type": "markdown", 172 | "metadata": {}, 173 | "source": [ 174 | "DEAP is also compatible with the *multiprocessing* standard module, if the user only cares to run on a single multicore machine." 175 | ] 176 | }, 177 | { 178 | "cell_type": "code", 179 | "collapsed": false, 180 | "input": [ 181 | "import multiprocessing\n", 182 | "pool = multiprocessing.Pool()\n", 183 | "toolbox.register(\"map\", pool.map)" 184 | ], 185 | "language": "python", 186 | "metadata": {}, 187 | "outputs": [], 188 | "prompt_number": 7 189 | }, 190 | { 191 | "cell_type": "markdown", 192 | "metadata": {}, 193 | "source": [ 194 | "With these powerful tools, DEAP allows scientists and researchers with little programming knowledge to easily implement distributed and parallel EAs." 195 | ] 196 | }, 197 | { 198 | "cell_type": "markdown", 199 | "metadata": {}, 200 | "source": [ 201 | "##Preaching by Example\n", 202 | "\n", 203 | "The best introduction to evolutionary computation with DEAP is to present simple, yet compelling examples. The following sections set forth how algorithms are easy to implement while keeping a strong grip on how they behave. The first section introduces a classical genetic algorithm and exposes different levels of explicitness. The second section presents how genetic programming is implemented in DEAP and the versatility of the GP module. The final example demonstrate how easy it is to implement a generic distributed island model with SCOOP.\n", 204 | "\n", 205 | "###A Simple Genetic Algorithm\n", 206 | "A commonly used example in evolutionary computation is the OneMax problem which consists in maximizing the number of ones in a binary solution. The more ones an individual contains, the higher its fitness value is. Using a genetic algorithm to find such an individual is relatively straightforward. Applying crossovers and mutations on a population of randomly generated binary individuals and selecting the fittest ones at each generation usually lead quickly to a perfect (all ones) solution. A problem of this simplicity should be solved with a very simple program." 207 | ] 208 | }, 209 | { 210 | "cell_type": "code", 211 | "collapsed": false, 212 | "input": [ 213 | "import random\n", 214 | "from deap import algorithms, base, creator, tools\n", 215 | "\n", 216 | "creator.create(\"FitnessMax\", base.Fitness, weights=(1.0,))\n", 217 | "creator.create(\"Individual\", list, fitness=creator.FitnessMax)\n", 218 | "\n", 219 | "def evalOneMax(individual):\n", 220 | " return (sum(individual),)\n", 221 | "\n", 222 | "toolbox = base.Toolbox()\n", 223 | "toolbox.register(\"attr_bool\", random.randint, 0, 1)\n", 224 | "toolbox.register(\"individual\", tools.initRepeat, creator.Individual, toolbox.attr_bool, n=100)\n", 225 | "toolbox.register(\"population\", tools.initRepeat, list, toolbox.individual)\n", 226 | "toolbox.register(\"evaluate\", evalOneMax)\n", 227 | "toolbox.register(\"mate\", tools.cxTwoPoint)\n", 228 | "toolbox.register(\"mutate\", tools.mutFlipBit, indpb=0.05)\n", 229 | "toolbox.register(\"select\", tools.selTournament, tournsize=3)\n", 230 | "\n", 231 | "if __name__ == \"__main__\":\n", 232 | " pop = toolbox.population(n=300)\n", 233 | " algorithms.eaSimple(pop, toolbox, cxpb=0.5, mutpb=0.2, ngen=40, verbose=False)\n", 234 | " print(tools.selBest(pop, k=1))" 235 | ], 236 | "language": "python", 237 | "metadata": {}, 238 | "outputs": [ 239 | { 240 | "output_type": "stream", 241 | "stream": "stdout", 242 | "text": [ 243 | "[[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]]\n" 244 | ] 245 | } 246 | ], 247 | "prompt_number": 8 248 | }, 249 | { 250 | "cell_type": "markdown", 251 | "metadata": {}, 252 | "source": [ 253 | "The preceeding code presents all that is needed to solve the OneMax problem with DEAP. The first two lines import the necessary modules. Next, on lines 3 and 4, two types are created; a maximizing fitness (note the positive weights), and a list individual composed with an instance of this maximizing fitness. Then, on lines 5 and 6, the evaluation function is defined. It counts the number of ones in a binary list by summing its elements (note again the one element returned tuple corresponding to a single objective fitness). Subsequently, a `Toolbox` is instantiated in which the necessary operators are registered. The first operator, on line 8, produces binary values, in this case integers in [0, 1]$, using the standard *random* module. The alias `individual`, on line 9, is assigned to the helper function `initRepeat`, which takes a container as the first argument, a function that generates content as the second argument and the number of repetitions as the last argument.\n", 254 | "Thus, calling the individual function instantiates an `Individual` of `n=100` bits by calling repeatedly the registered `attr_bool` function. The same repetition initializer is used on the next line to produce a population as a list of individuals. The missing number of repetition `n` will be given later in the program. Subsequently, on lines 11 to 14, the evaluation, crossover, mutation and selection operators are registered with all of their parameters.\n", 255 | "\n", 256 | "The main program starts at line 16. First, a population of `n=300` individuals is instantiated. Then, the algorithm, provided with the population and the toolbox, is run for `ngen=40` generations with `cxpb=0.5` probability of mating and `mutpb=0.2` probability of mutating an individual for each generation. Finally, on the last line, the best individual of the resulting population is selected and displayed on screen.\n", 257 | "\n", 258 | "###Controlling Everything\n", 259 | "When developing, researching or using EAs, pre-implemented canned algorithms seldom do everything that is needed. Usually, developers/researchers/users have to dig into the framework to tune, add or replace a part of the original algorithm. DEAP breaks with the traditional black-box approach on that precise point; it encourages users to rapidly build their own algorithms. With the different tools provided by DEAP, it is possible to design a nimble algorithm that tackles most problems at hand." 260 | ] 261 | }, 262 | { 263 | "cell_type": "code", 264 | "collapsed": false, 265 | "input": [ 266 | "import random\n", 267 | "from deap import algorithms, base, creator, tools\n", 268 | "\n", 269 | "creator.create(\"FitnessMax\", base.Fitness, weights=(1.0,))\n", 270 | "creator.create(\"Individual\", list, fitness=creator.FitnessMax)\n", 271 | "\n", 272 | "def evalOneMax(individual):\n", 273 | " return (sum(individual),)\n", 274 | "\n", 275 | "toolbox = base.Toolbox()\n", 276 | "toolbox.register(\"attr_bool\", random.randint, 0, 1)\n", 277 | "toolbox.register(\"individual\", tools.initRepeat, creator.Individual, toolbox.attr_bool, n=100)\n", 278 | "toolbox.register(\"population\", tools.initRepeat, list, toolbox.individual)\n", 279 | "toolbox.register(\"evaluate\", evalOneMax)\n", 280 | "toolbox.register(\"mate\", tools.cxTwoPoint)\n", 281 | "toolbox.register(\"mutate\", tools.mutFlipBit, indpb=0.05)\n", 282 | "toolbox.register(\"select\", tools.selTournament, tournsize=3)\n", 283 | "\n", 284 | "if __name__ == \"__main__\":\n", 285 | " pop = toolbox.population(n=300)\n", 286 | " \n", 287 | " ngen, cxpb, mutpb = 40, 0.5, 0.2\n", 288 | " fitnesses = toolbox.map(toolbox.evaluate, pop)\n", 289 | " for ind, fit in zip(pop, fitnesses):\n", 290 | " ind.fitness.values = fit\n", 291 | "\n", 292 | " for g in range(ngen):\n", 293 | " pop = toolbox.select(pop, k=len(pop))\n", 294 | " pop = algorithms.varAnd(pop, toolbox, cxpb, mutpb)\n", 295 | " \n", 296 | " invalids = [ind for ind in pop if not ind.fitness.valid]\n", 297 | " fitnesses = toolbox.map(toolbox.evaluate, invalids)\n", 298 | " for ind, fit in zip(invalids, fitnesses):\n", 299 | " ind.fitness.values = fit\n", 300 | " \n", 301 | " print(tools.selBest(pop, k=1))" 302 | ], 303 | "language": "python", 304 | "metadata": {}, 305 | "outputs": [ 306 | { 307 | "output_type": "stream", 308 | "stream": "stdout", 309 | "text": [ 310 | "[[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]]\n" 311 | ] 312 | } 313 | ], 314 | "prompt_number": 9 315 | }, 316 | { 317 | "cell_type": "markdown", 318 | "metadata": {}, 319 | "source": [ 320 | "Starting from the previous OneMax solution, a first decomposition of the algorithm replaces the canned `eaSimple` function by a generational loop. Again, this example is exhaustive but still very simple. On the first 3 lines, the evaluation is applied to every individual in the population by the `map` function contained in every toolbox. Next, a loop over both the population and the evaluated fitnesses sets each individual's fitness value. Thereafter, the generational loop begins. It starts by selecting `k` individuals from the population. Then, the selected individuals are varied by crossover **and** mutation by the `varAnd` function. A second variation scheme `varOr` can also be used, where the individuals are produced by crossover **or** mutation. Once modified, the individuals are evaluated for the next iteration. Only freshly produced individuals have to be evaluated; they are filtered by their fitness validity. This version of the program provides the possibility to change the stopping criterion and add components to the evolution." 321 | ] 322 | }, 323 | { 324 | "cell_type": "code", 325 | "collapsed": false, 326 | "input": [ 327 | "import random, math\n", 328 | "from deap import algorithms, base, creator, tools\n", 329 | "\n", 330 | "creator.create(\"FitnessMax\", base.Fitness, weights=(1.0,))\n", 331 | "creator.create(\"Individual\", list, fitness=creator.FitnessMax)\n", 332 | "\n", 333 | "def evalOneMax(individual):\n", 334 | " return (sum(individual),)\n", 335 | "\n", 336 | "toolbox = base.Toolbox()\n", 337 | "toolbox.register(\"attr_bool\", random.randint, 0, 1)\n", 338 | "toolbox.register(\"individual\", tools.initRepeat, creator.Individual, toolbox.attr_bool, n=100)\n", 339 | "toolbox.register(\"population\", tools.initRepeat, list, toolbox.individual)\n", 340 | "toolbox.register(\"evaluate\", evalOneMax)\n", 341 | "toolbox.register(\"mate\", tools.cxTwoPoint)\n", 342 | "toolbox.register(\"mutate\", tools.mutFlipBit, indpb=0.05)\n", 343 | "toolbox.register(\"select\", tools.selTournament, tournsize=3)\n", 344 | "\n", 345 | "if __name__ == \"__main__\":\n", 346 | " pop = toolbox.population(n=300)\n", 347 | " \n", 348 | " ngen, cxpb, mutpb = 40, 0.5, 0.2\n", 349 | " fitnesses = toolbox.map(toolbox.evaluate, pop)\n", 350 | " for ind, fit in zip(pop, fitnesses):\n", 351 | " ind.fitness.values = fit\n", 352 | "\n", 353 | " for g in range(ngen):\n", 354 | " pop = toolbox.select(pop, k=len(pop))\n", 355 | " pop = [toolbox.clone(ind) for ind in pop]\n", 356 | "\n", 357 | " for child1, child2 in zip(pop[::2], pop[1::2]):\n", 358 | " if random.random() < cxpb:\n", 359 | " toolbox.mate(child1, child2)\n", 360 | " del child1.fitness.values, child2.fitness.values\n", 361 | "\n", 362 | " for mutant in pop:\n", 363 | " if random.random() < mutpb:\n", 364 | " toolbox.mutate(mutant)\n", 365 | " del mutant.fitness.values\n", 366 | " \n", 367 | " invalids = [ind for ind in pop if not ind.fitness.valid]\n", 368 | " fitnesses = toolbox.map(toolbox.evaluate, invalids)\n", 369 | " for ind, fit in zip(invalids, fitnesses):\n", 370 | " ind.fitness.values = fit\n", 371 | " \n", 372 | " print(tools.selBest(pop, k=1))" 373 | ], 374 | "language": "python", 375 | "metadata": {}, 376 | "outputs": [ 377 | { 378 | "output_type": "stream", 379 | "stream": "stdout", 380 | "text": [ 381 | "[[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]]\n" 382 | ] 383 | } 384 | ], 385 | "prompt_number": 10 386 | }, 387 | { 388 | "cell_type": "markdown", 389 | "metadata": {}, 390 | "source": [ 391 | "An even greater level of detail can be obtained by substituting the `varAnd` function by its full content, presented in last code example. This listing starts with the duplication of the population by the `clone` tool available in every toolbox. Then, the crossover is applied to a portion of consecutive individuals . Each modified individual sees its fitness invalidated by the deletion of its value. Finally, a percentage of the population is mutated and their fitness values is also deleted. This variant of the algorithm provides control over the application order and the number of operators, among many other aspects.\n", 392 | "\n", 393 | "The explicitness in which algorithm are written with DEAP clarifies the experiments. This eliminates any ambiguity on the different aspects of the algorithm that could, when overlooked, jeopardize the reproducibility and interpretation of results.\n", 394 | "\n", 395 | "###Genetic Programming\n", 396 | "DEAP also includes every component necessary to design genetic programming algorithms with the same ease as for genetic algorithms. For example, the most commonly used tree individual can be created as following." 397 | ] 398 | }, 399 | { 400 | "cell_type": "code", 401 | "collapsed": false, 402 | "input": [ 403 | "import operator, random\n", 404 | "from deap import algorithms, base, creator, tools, gp\n", 405 | "\n", 406 | "creator.create(\"FitnessMin\", base.Fitness, weights=(-1.0,))\n", 407 | "creator.create(\"Individual\", gp.PrimitiveTree, fitness=creator.FitnessMin)" 408 | ], 409 | "language": "python", 410 | "metadata": {}, 411 | "outputs": [], 412 | "prompt_number": 11 413 | }, 414 | { 415 | "cell_type": "markdown", 416 | "metadata": {}, 417 | "source": [ 418 | "The primitive tree is provided in the *gp* module since it is one of the few data types the Python standard library does not provide. The primitives and terminals that will populate the trees are regrouped in a primitive set. The following listing presents a primitive set instantiation with basic operators provided by the standard library *operator* module. The `arity` of a primitive is its number of operands." 419 | ] 420 | }, 421 | { 422 | "cell_type": "code", 423 | "collapsed": false, 424 | "input": [ 425 | "pset = gp.PrimitiveSet(name=\"MAIN\", arity=1)\n", 426 | "pset.addPrimitive(operator.add, arity=2)\n", 427 | "pset.addPrimitive(operator.sub, arity=2)\n", 428 | "pset.addPrimitive(operator.mul, arity=2)\n", 429 | "pset.addPrimitive(operator.neg, arity=1)\n", 430 | "pset.renameArguments(ARG0=\"x\")" 431 | ], 432 | "language": "python", 433 | "metadata": {}, 434 | "outputs": [], 435 | "prompt_number": 12 436 | }, 437 | { 438 | "cell_type": "markdown", 439 | "metadata": {}, 440 | "source": [ 441 | "Functions that initialize individuals and populations are registered in a toolbox just as in the preceding genetic algorithm example. DEAP implements the three initialization methods proposed by [Koza](#koza) to generate trees: full, grow, and half-and-half." 442 | ] 443 | }, 444 | { 445 | "cell_type": "code", 446 | "collapsed": false, 447 | "input": [ 448 | "toolbox = base.Toolbox()\n", 449 | "toolbox.register(\"expr\", gp.genFull, pset=pset, min_=1, max_=3)\n", 450 | "toolbox.register(\"individual\", tools.initIterate, creator.Individual, toolbox.expr)\n", 451 | "toolbox.register(\"population\", tools.initRepeat, list, toolbox.individual)" 452 | ], 453 | "language": "python", 454 | "metadata": {}, 455 | "outputs": [], 456 | "prompt_number": 13 457 | }, 458 | { 459 | "cell_type": "markdown", 460 | "metadata": {}, 461 | "source": [ 462 | "We may now introduce an example of a symbolic regression evaluation function. First, the `gp.compile` function transforms the primitive tree into its executable form, a Python function, using a primitive set `pset` given as the evaluation function's third argument. Then, the rest is simple maths: we compute the root mean squared error between the individual's program and the target x^4+x^3+x^2+x on a set of points, the evaluation function's second arguent." 463 | ] 464 | }, 465 | { 466 | "cell_type": "code", 467 | "collapsed": false, 468 | "input": [ 469 | "def evaluateRegression(individual, points, pset):\n", 470 | " func = gp.compile(expr=individual, pset=pset)\n", 471 | " sqerrors = ((func(x) - x**4 - x**3 - x**2 - x)**2 for x in points)\n", 472 | " return math.sqrt(sum(sqerrors) / len(points))," 473 | ], 474 | "language": "python", 475 | "metadata": {}, 476 | "outputs": [], 477 | "prompt_number": 14 478 | }, 479 | { 480 | "cell_type": "markdown", 481 | "metadata": {}, 482 | "source": [ 483 | "Next, the evaluation function and the variation operators are registered similarly to the onemax example, while the other operators remain the same." 484 | ] 485 | }, 486 | { 487 | "cell_type": "code", 488 | "collapsed": false, 489 | "input": [ 490 | "toolbox.register(\"evaluate\", evaluateRegression, points=[x/10. for x in range(-10, 10)],\n", 491 | " pset=pset)\n", 492 | "toolbox.register(\"mate\", gp.cxOnePoint)\n", 493 | "toolbox.register(\"expr_mut\", gp.genFull, min_=0, max_=2)\n", 494 | "toolbox.register(\"mutate\", gp.mutUniform, expr=toolbox.expr_mut, pset=pset)\n", 495 | "toolbox.register(\"select\", tools.selTournament, tournsize=3)\n", 496 | "\n", 497 | "if __name__ == \"__main__\":\n", 498 | " pop = toolbox.population(n=300)\n", 499 | " algorithms.eaSimple(pop, toolbox, cxpb=0.5, mutpb=0.2, ngen=40, verbose=False)\n", 500 | " bests = tools.selBest(pop, k=1)\n", 501 | " print(bests[0])\n", 502 | " print(bests[0].fitness)" 503 | ], 504 | "language": "python", 505 | "metadata": {}, 506 | "outputs": [ 507 | { 508 | "output_type": "stream", 509 | "stream": "stdout", 510 | "text": [ 511 | "sub(sub(mul(x, x), neg(x)), sub(sub(x, x), mul(add(add(x, x), neg(x)), mul(x, add(x, mul(x, x))))))\n", 512 | "(6.713178136967714e-17,)\n" 513 | ] 514 | } 515 | ], 516 | "prompt_number": 15 517 | }, 518 | { 519 | "cell_type": "markdown", 520 | "metadata": {}, 521 | "source": [ 522 | "Furthermore, using external libraries such as [NetworkX](#networkx) and [PyGraphviz](#pygraphviz), the best primitive trees can be visualized as follow." 523 | ] 524 | }, 525 | { 526 | "cell_type": "code", 527 | "collapsed": false, 528 | "input": [ 529 | "import matplotlib.pyplot as plt\n", 530 | "import networkx\n", 531 | "\n", 532 | "nodes, edges, labels = gp.graph(bests[0])\n", 533 | "graph = networkx.Graph()\n", 534 | "graph.add_nodes_from(nodes)\n", 535 | "graph.add_edges_from(edges)\n", 536 | "pos = networkx.graphviz_layout(graph, prog=\"dot\")\n", 537 | "\n", 538 | "plt.figure(figsize=(7,7))\n", 539 | "networkx.draw_networkx_nodes(graph, pos, node_size=900, node_color=\"w\")\n", 540 | "networkx.draw_networkx_edges(graph, pos)\n", 541 | "networkx.draw_networkx_labels(graph, pos, labels)\n", 542 | "plt.axis(\"off\")\n", 543 | "plt.show()" 544 | ], 545 | "language": "python", 546 | "metadata": {}, 547 | "outputs": [ 548 | { 549 | "metadata": {}, 550 | "output_type": "display_data", 551 | "png": "iVBORw0KGgoAAAANSUhEUgAAAbsAAAGlCAYAAABjtVj1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XdYFNf3P/D3iohRVBAQKSKKRqXDYomKGruxkqgBwRIF\nNZbEWONHjSUaNdHEaLCExF5iLLEnoqKiRkUWC1jAAiJNpElf2N3z+yM/+YqALDCz6Hpez5PnIbsz\n554ZcM/euTP3SoiIwBhjjGmxGtWdAGOMMSY2LnaMMca0Hhc7xhhjWo+LHWOMMa3HxY4xxpjW42LH\nGGNM63GxY4wxpvW42DHGGNN6XOwYY4xpPS52jDHGtB4XO8YYY1qPix1jjDGtx8WOMcaY1uNixxhj\nTOtxsWOMMab1uNgxxhjTelzsGGOMaT0udowxxrQeFzvGGGNaj4sdY4wxrcfFjjHGmNbjYscYY0zr\ncbFjjDGm9bjYMcYY03pc7BhjjGk9LnaMMca0Hhc7xhhjWq9mdSfAWHVQKBS4fv06ZDIZZDIZnjx5\ngoKCAtSqVQvm5uZwc3ODVCqFq6srdHV1qztdxlgVSYiIqjsJxjQlKSkJv/32GzZt2gQDAwO0a9cO\nUqkUzZo1g56eHuRyOR4/fgyZTIZr167h2bNn8PPzg5+fHywsLKo7fcZYJXHPjr0TFAoFvv/+e6xa\ntQrDhg3D0aNH4ezsXO5+ERER2LBhAxwcHDBlyhTMnz8ftWrV0kDGjDEhcc+Oab179+7Bx8cHDRs2\nxG+//QYrK6sKx0hISMD48ePx5MkT7Ny5Ew4ODiJkyhgTC9+gwrTatWvX0K1bN/j6+uLkyZOVKnQA\nYG5ujqNHj+Krr75Cjx49cPHiRYEzZYyJiXt2TGuFh4ejZ8+eCAgIwKBBgwSLGxgYCB8fH5w4cQJu\nbm6CxWWMiYeLHdNKeXl5cHJywoIFCzBy5EjB4x84cAAzZsxAeHg46tWrJ3h8xpiwuNgxrTRjxgwk\nJCRgz549orUxbtw46OrqYuPGjaK1wRgTBhc7pnVu3LiBfv36ITw8HMbGxqK1k5GRAQcHB+zduxcd\nO3YUrR3GWNXxDSpM66xduxbTpk0TtdABgIGBAWbPno21a9eK2g5jrOq4Z8e0SlpaGmxsbBAVFQUT\nE5MqxbK2tsbvv/+OHj16lLnN8+fPYW1tjTt37sDMzKxK7THGxMM9O6ZVDh06hF69elW50AGARCKB\nRCJ57TYNGjTA4MGDsX///iq3xxgTDxc7plWuXbuGTp06abTNTp06ITQ0VKNtMsYqhosd0yoymQxS\nqbTE6ytXroSlpSXq16+P1q1bIygoCGPGjMGCBQuKtjl37hyaNGlSbL+QkBDY2dmhYcOGGDt2LORy\neYnYUqkUMplM+INhjAmGix3TKlFRUWjTpk2x1yIjI+Hv74/Q0FBkZmYiMDAQ1tbW5V6mJCLs3r0b\ngYGBePjwIaKiorB06dIS29na2iIqKkrwY2GMCYeLHdMq+fn5qFOnTrHXdHR0IJfLcfv2bRQWFsLK\nygrNmzcH8F9BK4tEIsGUKVNgYWEBQ0NDzJs3r9Tn9vT09KBUKqFQKIQ9GMaYYLjYMa2io6MDpVJZ\n7LUWLVpgzZo1WLRoEUxNTeHl5YXExES14r18WdPKygoJCQkltlGpVCAi6OjoVC15xphouNgxrWJk\nZISkpKQSr3t5eeHChQt4/PgxJBIJ5syZg7p16yI3N7dom9L2i42NLfazubl5iW2Sk5PRsGHDcu/c\nZIxVHy52TKu4uLggLCys2GtRUVEICgqCXC6Hnp4eateuDR0dHTg7O+PEiRNIT09HUlIS1qxZU2w/\nIoK/vz/i4+ORlpaGZcuWwdPTs0SbYWFhcHFxEfW4GGNVw8WOaZXS7oyUy+WYO3cuTExMYGZmhpSU\nFCxfvhwjR46Ek5MTrK2t0bdvX3h6ehbrnUkkEnh7e6N3796wsbFBy5YtMX/+/BJthoaGlnoHKGPs\nzcEzqDCtcunSJYwbNw53797VyGVFIoKrqytWrlyJ3r17i94eY6xyuGfHtErHjh2hq6uLs2fPaqS9\nq1evIisrCz179tRIe4yxyuFix7SKRCLBpEmT8NNPP2mkvTVr1uDzzz9HjRr8T4mxNxn/C2VaZ8yY\nMYiMjMTBgwdFbefvv//GlStXMH78eFHbYYxVHY/ZMa106dIlDB06VLQ17V6sZbd169bXrorAGHsz\ncM+OaaVOnTphzJgx8PDwKPYsnRDy8/MxdOhQeHh4cKFj7C3BPTumtVQqFcaOHYuYmBgcOXIE9evX\nr3LMnJwcfPLJJ2jQoAF2797Ns6Yw9pbgnh3TWjVq1MDmzZthb28PqVSKCxcuVCne1atX4ebmBgsL\nC+zatYsLHWNvEe7ZsXfC4cOHMX78eAwePBjz58+HlZWV2vsmJCRg+fLl2LNnD9avX4/hw4eLmClj\nTAxc7Ng7QaVSoW3btjA1NcXVq1fh7u4OHx8ftGvXDk2aNCn2ADoRIT4+HteuXcOuXbsQFBSEDz74\nANHR0QgPD+ceHWNvoZrVnQBjmvDHH3+gRo0aOHbsGPLy8rBr1y5s3boVU6dOhUKhQLNmzaCnp4eC\nggLExMQA+G/qsQEDBmDLli3Q19dH165dsWXLFvj6+lbvwTDGKox7dkzr5eXloXXr1ti5cyfc3d2L\nvUdESEhIQFxcXNFE0ebm5rC0tCwx3VhoaCgGDRqEyMhI1KtXT5OHwBirIi52TOt99913CAsLw/79\n+6sca9SoUbCysip1xXLG2JuLix3TaklJSbC3t8fVq1dhY2NT5XhPnjyBs7Mzrl+/XqGbXBhj1YuL\nHdNq48ePR/369bFq1SrBYn7zzTd49OgRdu7cKVhMxpi4uNgxrXXr1i306tULkZGRMDAwECxudnY2\nWrVqhb/++gvt2rUTLC5jTDz8UDnTSkSEGTNmYMGCBYIWOgDQ19fHt99+i+nTp4O/KzL2duBix7TS\n33//jSdPnmDChAmixB89ejSys7Nx4MABUeIzxoTFlzGZ1iksLISjoyN++OEHDBgwQLR2zpw5Az8/\nP9y9exd6enqitcMYqzru2TGtExAQAAsLC/Tv31/Udnr06AF7e3usW7dO1HYYY1XHPTumVTIyMtCq\nVSsEBgbCyclJ9PYiIyPRqVMn3L17FyYmJqK3xxirHC52TKvMnj0baWlp+O233zTW5hdffAGVSoVf\nfvlFY20yxiqGix3TGo8ePULbtm0REREBMzMzjbWbmpqK1q1bIzg4GG3atNFYu4wx9XGxY1pj+PDh\ncHR0xPz58zXe9o8//oigoCAcO3ZM420zxsrHxY5phUuXLsHLywv37t1DnTp1NN6+XC6HnZ0dNmzY\ngF69emm8fcbY6/HdmOytp1KpMH36dHz33XfVUugAQE9PD99//z1mzJgBpVJZLTkwxsrGxY699f74\n4w+oVCqMGDGiWvPw8PCAgYEBtmzZUq15MMZK4suY7I1y584dnD9/HjKZDDdu3EBGRgZUKhXq1KmD\n1q1bQyqVon379ujatSt0dHReu1ZddXh1zTulUomLFy/i6tWrCA0Nxd27d5GTkwOJRAIDAwM4OTlB\nKpXC3d0djo6O1Z0+Y1qLix2rdgUFBThw4AD8/f0RExODPn36QCqVwsXFBSYmJpBIJMjJyUFERARk\nMhmCg4ORnJyMCRMmIDs7G1FRUYKsVSeUUaNGwdjYGI0bN8bGjRvRoEEDdOvWDVKpFA4ODtDX1wcR\nITU1FdevX4dMJkNgYCDMzMwwadIkfPrppzwjC2NCI8aqUWhoKNnb21PXrl3pwIEDVFhYqNZ+MpmM\nxo4dS3Xq1KHvvvuOVCqVyJmqR6VS0U8//UR16tShUaNGUUhIiFr7KRQKOnLkCPXq1YtatWpFly9f\nFjlTxt4t3LNj1UKlUmHJkiXYsGEDfvzxR4wYMQISiaTCcW7duoXPPvsMJiYm2LFjR7XOYpKeno4x\nY8YgOjoaW7duhaura6Xi7N+/H1OnTsXo0aOxbNky6OjoCJwpY+8evkGFaZxSqcS4ceNw6tQp3Lhx\nA97e3pUqdADg6OiIK1euwNHREV26dEFCQoLA2aonOTkZ3bp1g5WVFUJDQytd6ABg6NChuHXrFkJC\nQuDt7Q2FQiFgpoy9m7hnxzSKiDB58mTcuXMHx48fR926dQWLvWzZMuzevRsXLlxAw4YNBYtbnqys\nLHTt2hV9+/bFsmXLKl24X5Wfn48hQ4agcePG2LJli2BxGXsXcbFjGvXnn39i4cKFuHr1KurXry9o\nbCLClClTkJGRgV27dgka+3X8/PygUCiwefNmwQtSTk4OOnXqhGnTpmHMmDGCxmbsXcLFjmlMcnIy\nHB0dcfjwYbRv316UNnJzc+Hk5IQffvgBQ4YMEaWNl508eRLjx49HeHi44MX7hZs3b6JXr164fv06\nLCwsRGmDMW3HxY5pjK+vLwwNDfHDDz+I2s7Fixfx6aefIjo6GrVq1RKtHaVSiZYtW2Ljxo3o3bu3\naO0AwKJFixAZGYk9e/aI2g5j2oqLHdOI1NRUtGjRAvfv34exsbHo7fXo0QN+fn7w9PQUrY0jR47g\nu+++w5UrV0Rr44Xnz5/D2toad+7c0eiKDoxpC74bk2nE1q1bMXDgQEEKnbW1Nc6cOfPabSZNmoT1\n69dXua3XWb9+PSZPnlzlOOocT4MGDfDpp59qdJ0+xrQJFzumEfv378fo0aMFiSWRSMq9EWTQoEEI\nDw/H06dPBWnzVZmZmbh48SKGDRtW5VjqHA8AjBkzBvv27atye4y9i7jYMdEpFArcunUL7dq101ib\nurq6cHNzg0wmEyV+WFgYHB0dUbt2bVHil8bFxQUPHjxAXl6extpkTFtwsWOiu3PnDqysrFCvXr0S\n761cuRKWlpaoX78+WrdujaCgIIwZMwYLFiwo2ubcuXNo0qRJsf1CQkJgZ2eHhg0bYuzYsZDL5SVi\nS6VS0YqdTCaDm5tbidfFPB49PT20adMGN2/eFP6AGNNyXOyY6KKjo9GiRYsSr0dGRsLf3x+hoaHI\nzMxEYGAgrK2ty72sR0TYvXs3AgMD8fDhQ0RFRWHp0qUltmvZsiWio6MFPZYXSjsmsY8HEPeYGNNm\nXOyY6ORyeamX+3R0dCCXy3H79m0UFhbCysoKzZs3B/BfASiLRCLBlClTYGFhAUNDQ8ybN6/UW/Jr\n166N/Px84Q7kJaUdk9jHA4h7TIxpMy52THS6urqlzu/YokULrFmzBosWLYKpqSm8vLyQmJioVsyX\nLwNaWVmVOidmYWGhaM/ZlXZMYh8PIO4xMabNuNgx0ZmamiIuLq7U97y8vHDhwgU8fvwYEokEc+bM\nQd26dZGbm1u0TVJSUon9YmNji/1sbm5eYpu4uDg0atRIgCMoqaxjEvN4AHGPiTFtxsWOic7JyQl3\n7txBQUFBsdejoqIQFBQEuVwOPT091K5dGzo6OnB2dsaJEyeQnp6OpKQkrFmzpth+RAR/f3/Ex8cj\nLS0Ny5YtK/Xh8bJuIhFCaTe/iH08KpUK169fh1QqFeWYGNNmXOyY6OrWrYtmzZohIiKi2OtyuRxz\n586FiYkJzMzMkJKSguXLl2PkyJFwcnKCtbU1+vbtC09Pz2I3eEgkEnh7e6N3796wsbFBy5YtMX/+\n/GKxiQihoaGiFYYXxe7lsTgxjwf4r5gaGxtrdEUHxrQFTxfGNGLGjBnQ1dXFihUrNNLe5cuXMWrU\nKERFRYm2NI6joyN++ukn9OjRQ5T4r1qyZAkSExOxYcMGjbTHmDbhYsc04v79++jUqRNiY2M18iD2\nyJEj4eLigunTp4vWxsaNG3Hq1CkcOHBAtDZeKCwsRLNmzXDixAk4OjqK3h5j2oYvYzKNaNmyJVxc\nXLB9+3bR24qNjcWxY8dEX//N29sb586dQ1RUlKjtAMDevXvRrFkzLnSMVRL37JjG3LhxA7179xZ1\nXTYiQp8+fdCtWzf873//E6WNl/3888/Yv38/zp8/jxo1xPnu+OzZMzg4OODQoUPo0KGDKG0wpu24\nZ8c0xtnZGZMmTcKECRNe+5B1VQQEBCA9PR2zZ88WJf6rpk6dColEgp9//lmU+ESESZMmYdSoUVzo\nGKsC7tkxjSooKEC3bt3QoUMHrF69WtCbR06dOlV0adHW1lawuOV5+PAhOnXqhF9//RWDBg0SNPbc\nuXOxefNm3L59WyPrADKmrbhnxzSqVq1aOH78OM6ePYsvv/wSSqVSkLhHjhyBt7c3Dh48qNFCBwA2\nNjY4evQo/Pz88OeffwoSU6VS4euvv8aRI0fQo0cP9OvXD8nJyYLEZuxdxMWOaZyhoSHOnj2Lc+fO\nwc3NDZGRkZWOlZ2djalTp2LSpEk4duwYOnfuLGCm6mvbti1OnjyJWbNmYfz48cjMzKx0rEePHqFH\njx64dOkSgoODsWvXLnz00Ufo2LEjHjx4IGDWjL07uNixahEXF4eEhAT0798fnTp1wjfffFPmfJCl\nyc/Px/bt2+Hk5ITMzEyEh4drdL280jg7OyM8PBwA4ODggM2bNxebJqw8T58+xdKlS9G+fXsMGDAA\n586dg5GRESQSCRYvXow5c+bA3d0dISEhYh0CY9qLGNOwrKwsat26NW3dupWIiB4+fEgTJ04kAwMD\nGjp0KG3ZsoVu3bpFhYWFRfuoVCqKjY2lv/76i6ZPn04mJibUp08fOnnyZHUdxmsFBQVR//79ycjI\niL744gs6cOAAxcTEkEqlKtpGoVDQ7du3adu2beTp6UkGBgY0btw4ioyMLDPu0aNHycTEhI4ePaqJ\nw2BMa/ANKkyjiAijR4+Gjo4OtmzZUuy9zMxM7N69G8HBwZDJZHjy5An09fWhUqlQWFiI2rVrQyqV\nol27dvDx8Sl1jbw3TXR0NHbu3ImrV69CJpMhJycHenp6AIDc3FyYmZlBKpXC3d0d3t7eMDQ0LDfm\n1atXMWTIECxZsgR+fn5iHwJjWoGLHdOoLVu2YNWqVQgJCUHdunVfu21OTg62b9+Ov//+G1u3boWh\noaFoU39pSnp6Oj7//HO0bdsWEyZMgL6+fqXi3L9/H/369YO3tzcWLVr01p8XxsTGY3ZMYyIiIjB7\n9mzs27ev3EIH/DeBtKGhIerUqYOGDRtqxQe6oaEh6tatCwMDg0oXOuC/GWkuXbqEEydOwNfXF4WF\nhQJmyZj24WLHNCI7OxvDhg3DqlWrNP5ogLYyNTXF2bNnkZiYiMGDByM7O7u6U2LsjcXFjomO/v8s\nIB06dMDo0aOrOx2toq+vj8OHD8PMzAwffvghP4vHWBm42DHRbd26FTKZDL/88kt1p6KVdHV18dtv\nv/GzeIy9Rs3qToBptxfjdOfPn1drnI5Vzotn8SwtLeHu7o7Dhw9X+3OHjL1JuGfHRMPjdJrn5+eH\ngIAADBgwAMeOHavudBh7Y3CxY6LgcbrqM2DAgKK5OgMCAqo7HcbeCHwZk4nixTgdT21VPdq3b4/g\n4GD069cPcXFx/Cwee+dxz44JrqLP0zFx8LN4jP0fLnZMUDxO92bhZ/EY+w8XOyYYHqd7M/GzeIxx\nsWMC4ufp3lz8LB571/ENKkwQ/Dzdm4+fxWPvMu7ZsSrjcbq3Cz+Lx95FXOxYCUQEdVd+EnucriK5\nvC3ehGMS+lm86j4exsrD69m94/Ly8nD48GFcvnwZMpkM4eHhyMrKAgDUq1cPDg4OkEql+OCDDzB4\n8GC89957xfavyPp05VEoFDh58mTR4q03btxAeno6VCoV6tSpgzZt2hQt3vrxxx+rtdBpdcvMzMTB\ngweLFm+9c+cOcnJyUKNGDTRo0ABOTk5Fi7d+9NFH0NXV1Wh+lVkXLyYmBn/99RdCQ0Mhk8kQHR2N\ngoIC1KxZE40bN4arqyukUin69u2Ltm3b8vN97I3APbt31JMnTzBr1ixYWVlhy5YtsLS0xJIlS/Do\n0SMUFBSgoKAAjx49wpIlS2BpaYktW7bAysoKs2bNQmxsLADhnqdLS0vDd999h+bNm+Pbb7+Fvr4+\npk+fjvDwcMjlcigUCiQmJmLt2rWwt7fHP//8g+bNm8PPzw8RERFCnRJB3bt3D59//jmaNm2Kw4cP\no3Xr1vjxxx8RHx8PhUIBuVyOu3fvYs6cOTA0NMQPP/wAa2trLF68GM+ePdNYnhV5Fi8wMBADBw6E\nm5sb7t27h549e+LPP/9ERkYGlEolcnJycP78efj4+CAnJweenp5o27YtNm/ezM/4sepH7J2iVCrJ\n39+fjI2Nafr06fTgwQO1933w4AHNmDGDjIyM6Mcff6RWrVrR1q1bq5TPoUOHyMzMjMaMGUMymUzt\n/ZKSkmjp0qXUqFEjmjdvHuXn51cpD6EUFBTQkiVLyNjYmBYuXEjx8fFq73vz5k3y8/MjU1NT2rt3\nr4hZlpSVlUX9+vWjfv36UVZWVrH3nj59SkOHDqX333+ffv/9d8rJyVErplKppBMnTlD37t3JxcWF\nbty4IUbqjKmFi907JDU1lbp3707t27enO3fuVDrOnTt3yNnZmZo2bUopKSmVipGfn0+jRo0iGxsb\nCg4OrnQuCQkJNGjQILK3t6f79+9XOo4QYmJiyMXFhfr27UuxsbGVjnPlyhVq3bo1DR8+XO3CIoSC\nggIaO3Ysubm50dOnT4mI6OTJk2RqakqzZ8+mvLy8SsVVqVS0efNmMjY2plWrVpFKpRIybcbUwsXu\nHfH06VOyt7enGTNmkEKhqHI8hUJBM2bMIDs7u6IPRnXl5eVR79696eOPP6bs7Owq56JSqcjf35/M\nzc3p9u3bVY5XGffv36cmTZrQ6tWrBfkwz8vLI29vb+rSpYsg50hdKpWKvvnmG7KxsaFffvmFTExM\n6Pz584LEjo2NJTs7O/r666+54DGN42L3DsjKyiIXFxeaP3++oB8yKpWK5s+fT87OziUufZVFqVTS\noEGDyNPTkwoLCwXLhYho+/btZGlpWaVeVWUkJiaStbU1bdq0SdC4SqWSPvvsM+rVq5fg56o8X331\nFdWrV4/CwsIEjZuSkkL29va0fPlyQeMyVh6+G/MdMGXKFGRkZGDHjh2C3xlHRBg1ahQaNGig1swp\n69atw549e3D+/HlR7jxcunQpzp8/j8DAQI3cBUhEGDJkCGxtbbF8+XLB4ysUCvTu3Rt9+vTBnDlz\nBI9fmvT0dDg4OGDbtm3o0aOH4PETEhLg4uKCo0eP8kPtTHOqt9YysZ09e5YsLCwoLS1NtDbS0tLI\nwsKCgoKCXrvdgwcPyMjIiO7duydaLoWFheTm5iZ4L6ssO3bsIHt7e1FvkImOjiYjIyONXaIdNWoU\nTZ48WdQ29uzZQ7a2tpUeB2SsorjYaTGVSkX29vb0119/id7WoUOHyN7e/rWXSYcOHUorVqwQPZeI\niAgyMjISfawrPz+fTE1N6dq1a6K2Q0S0bt066tevn+jtXL16lZo0aaL2ZenKUqlUNGjQIFq9erWo\n7TD2Aj9np8UuXLgAhUKBwYMHi97WoEGDoFAocOHChVLfj4+Px5kzZzBp0iTRc7Gzs4O7uzt2794t\najv79++Ho6Mj3NzcRG0HAHx9fXHt2jU8fPhQ1HbWr1+PqVOnQl9fX9R2JBIJvv76a2zYsAEqlUrU\nthgD+KFyrbZ+/XpMmjRJ8LGrRYsWYeTIkcVek0gkmDRpEvz9/UvdJyAgACNGjEC9evUAANbW1li9\nejWcnJxgYGAAT09PyOVyAMCxY8fg7OwMQ0NDdOrUCeHh4UVxwsLC4OLigvr162P48OH49NNPsWDB\nghLtvciFRBySfnF+q8ra2hpnzpx57Ta1a9fGZ599ho0bN1a5vbKkpqbi8OHD+OyzzwSPXdrfTIcO\nHaCvr4/Tp08L3h5jr+Jip6WUSiWOHz+OESNGCB67rOI5YsQInDhxAkqlssR7R44cgbe3d7EY+/bt\nw8mTJxEdHY1bt25h69atuH79OsaNG4eAgACkpaVhwoQJGDRoEAoLC1FQUAAPDw+MHTsW6enp8PLy\nwqFDh0rNp0ePHkhMTERcXJxwB/6S1NRUREREYMCAAVWOJZFI1PpC4u3tjaNHj1a5vbKcOnUKXbt2\nhbGxseCxSzs+iUQCb29vHDlyRPD2GHsVFzstFRUVhUaNGsHIyEjw2GX1loyMjNCoUSNERUUVez0/\nPx/37t2Ds7Nzsde/+OILNG7cGIaGhhg4cCBu3LiBgIAATJgwoWhOxVGjRkFPTw+XL1/GlStXoFQq\nMXXqVOjo6MDDw6PMu/lq1KgBNzc3yGQyYQ76FTKZDK6urqhZU3OrZNnZ2SEuLg6ZmZmixJfJZGjb\ntq0oscv6m2nbtq1ovyPGXsbFTkvJZDJIpdJir1lbW2PVqlVwdHREvXr1MG7cODx9+hT9+vVDgwYN\n0KtXL2RkZODcuXNo0qRJiX2DgoLKbVcqlZb48AoPD0fLli1LTCLduHHjop/r1KmD7OxsPH78GKtX\nr4ahoWHRf3FxcUhMTERCQgIsLCyKxWjSpEmZH6RSqRShoaHl5lwZoaGhJc4vAKxcuRKWlpaoX78+\nWrdujaCgIIwZM6bYpdbSzm9ISAjs7OzQsGFDjB07tuiS7stq1qwJBwcHXL9+XfgDQunHJPbfjIuL\nC27dugWFQiH48TD2Mi52WiouLg7W1tbFXpNIJDh48CDOnDmDyMhIHDt2DP369cOKFSuQnJwMlUqF\ntWvXlnnJSR3NmjUrcekwPj6+RC5ladKkCebNm4f09PSi/7Kzs/Hpp5/CzMwM8fHxxbaPjY0tM7dm\nzZqV2F4opR1TZGQk/P39ERoaiszMTAQGBsLa2rrcy5REhN27dyMwMBAPHz5EVFQUli5dWuq2Yh9T\ns2bNir0m9t9M/fr1UadOHaSlpQl2HIyVhoudliooKICenl6J16dOnQoTExOYm5vD3d0dH3zwAZyc\nnKCnpwcPD48q9xr09PRK9ErKyuVlL3pnfn5+2LhxI0JCQkBEyMnJwfHjx5GdnY2OHTtCR0cHv/zy\nCxQKBQ4fPoxr166VGbNWrVql9pCEUNox6ejoQC6X4/bt2ygsLISVlRWaN29e7PhKI5FIMGXKFFhY\nWMDQ0BATH/nHAAAgAElEQVTz5s3Dnj17St1W08cEiP83U6tWLRQUFFQpBmPl4WKnpUorOgBgampa\n9PN7771X7P9r166N7OzsKrUrl8tLfGCq8wH9ovcjlUoREBCAKVOmoGHDhmjZsiW2b98OANDV1cXB\ngwfx+++/w9DQELt27cKAAQNQq1atUmOqU2Qrq7RjatGiBdasWYNFixbB1NQUXl5eSExMVCvey5cA\nrayskJCQUOp2mj4mQPy/mYKCgjJ/h4wJRXOj60yjLC0t1Rr4L63HUbduXeTm5hb9v1KpVHuNtejo\naDg6OpbIJSYmpsR2L1u4cGHRz3369EGfPn1KjS+VSov1JNq3b49BgwaVmYulpaVaeVdUaccEAF5e\nXvDy8kJWVhYmTJiAOXPmoH79+sXOZ1JSUon9XqwR+OJnc3PzUtsV+5iio6PRqlWr124n5N9MZmYm\n8vLy0LBhw4onzFgFcM9OS1Xl5oz3338f+fn5OHHiBAoLC7F06VK1L52VdpODg4MD7t+/j7y8vErl\n87Lg4GAkJSVBoVBg27ZtiIiIQN++fdXORSilnd+oqCgEBQUV9W5r164NHR0dODs748SJE0hPT0dS\nUhLWrFlTbD8igr+/P+Lj45GWloZly5bB09OzRJsKhQK3bt2Ci4uLxo5JXZX9mwkLC4Ojo6NG72pl\n7yYudlrq/fffR0pKClJSUl673cs3Eby4lFi/fn2sX78evr6+sLS0hL6+frHLbGXdcJGSkoJnz56h\nZcuWxV7X09NDmzZtBLmLMDIysuiB859++gn79+8vdlntBZVKVeodqUJ50cN8+S5CuVyOuXPnwsTE\nBGZmZkhJScHy5csxcuRIODk5wdraGn379oWnp2eJ8+7t7Y3evXvDxsYGLVu2xPz580u0GRERgSZN\nmhQ9mC80Nze3146BviDk38y1a9fg6uoqzAEw9hq86oEWGzFiBDp06IAvvvhCI+2tXbsW//77L/74\n448S7y1evBjJycllzrAitMDAQMyZMwdhYWGirX7QuXNnzJgxAx4eHqLEf9WsWbMgkUjw/fffixI/\nLS0NNjY2uH//vigPlr+KiODs7IxVq1ahV69eorfH3nHVMSEn04zg4GBq1aqVRhbKVKlU1KpVqzJX\nHY+PjydDQ0PKzMwUPRciosGDB1NAQICobezatYt69uwpahsv5ObmkrGxMT18+FDUdsaMGUMrV64U\ntY0XLl26RC1btiSlUqmR9ti7jS9jarHOnTtDT08Pf/31l+htHTp0CLVq1ULnzp1Lfd/c3Bw9e/ZU\na827qgoPD8fFixfh5eUlajuffPIJIiIi1Lr0V1UBAQFo165d0aMMYpk8eTLWrVuHrKwsUdshIixf\nvhyTJk1CjRr8McQ0oLqrLRPX+fPnydzcnFJTU0VrIzU1lczNzen8+fOv3e7Ro0dkZGREd+/eFS2X\ngoICcnV1Fb1X98Lu3bvJzs5O1PXsHj58KPp5e9lnn31Gn3/+uaht7Nq1S/R1ABl7GY/ZvQO+/PJL\nPHv2DLt27RJlpXJvb28YGxtj7dq15W6/fv16bN++HcHBwaI8W7VkyRJcunQJ//zzj8ZWKv/444/x\n/vvvY+XKlYLHVygU6NWrF/r374+ZM2cKHr80GRkZcHBwwObNm0UZS4uPj4erqyuOHz+ukeWRGAPA\nPbt3QXZ2NkmlUpo7d66g43cqlYrmzp1Lrq6uai+UqlQqycPDg4YNG0aFhYWC5UJEtGXLFmrSpAk9\nefJE0LjlSUpKoubNm9P69esFjatQKGjUqFHUt29fUigUgsYuT1BQEJmYmFBoaKigcZ89e0a2trb0\n/fffCxqXsfJwsXtHPHv2jBwdHWnatGmCFJnCwkKaNm0aOTg4UHJycoX2zcvLo759+9KQIUMEWRFb\npVLR2rVrycLCQmOX+l718OFDatq0Ka1cuVKQLxS5ubnk5eVF3bp1o5ycHAEyrLhDhw6RiYkJBQUF\nCRIvJiaG2rRpQ/PmzRMkHmMVwcXuHZKamkrW1tbk4OBAERERlY4TERFBbm5u1KtXL0pLS6tUDLlc\nTmPHjqVmzZrR2bNnK51LXFwc9e/fn5ycnES/U7E8sbGx5ObmRj179qSYmJhKx7l06RK9//77NGLE\nCMrNzRUww4o7ffo0mZmZ0fTp0ytddFUqFW3atIn09fVp0qRJAmfImHq42L1DNm/eTG3atKF169aR\nsbExffnllxQZGan2/pGRkTRt2jQyNjamTZs2CdKDOXr0KFlYWJCPjw+FhISovV98fDwtXryYTExM\naOHChSSXy6ucixAKCwvpu+++I2NjY5o/fz7FxsaqvW9YWBiNHTuWGjduTAcOHBAxy4p59uwZeXp6\nUosWLWjTpk1q98YVCgUdOXKEunbtSlKplH799VcyMzOjxMREkTNmrCS+QeUdERERgQ8//BDnz5+H\nra0t4uPj4e/vj99//x329vbo2bMnpFIpnJ2d0aBBAwDA8+fPcePGDchkMpw5cwa3bt3CuHHjMHny\nZEHnZ8zIyMCvv/6KDRs2wMjICP3794dUKoWrqyuMjY1Ro0YNZGdnIyIiAjKZDBcuXMDZs2fh6emJ\nqVOnwtbWVrBchBIVFYW1a9fi999/R48ePfDhhx9CKpXCwcEB9erVg0qlQlpaGq5fv47Q0FAcOXIE\njx49wsyZMzFx4kRRFt2tqjNnzmDdunUIDg7Gxx9/jPbt20MqlaJly5aoXbs2CgsLERcXB5lMhtDQ\nUOzfvx9mZmb4/PPP4e3tjZo1a2LhwoW4ePEiAgMDoaOjU92HxN4hXOzeAdnZ2Wjbti2+/vprjB49\nuth7crkcR48exeXLlyGTyXDr1i1kZWWBiFC/fn04OjpCKpXigw8+wMCBA0WbcR/4b/Lg06dPIzg4\nGDKZDNevX0d6ejqUSiX09fXRpk0bSKVStG3bFh4eHkVF+U118uRJzJ07F1999RVCQkIgk8lw+/Zt\n5OTkoEaNGjAwMICTkxOkUilcXFzg5+eH6OjoN7LQvSw2NhaHDh2CTCaDTCZDdHQ08vLyoKenB1NT\nU0ilUkilUvTp06fEdG1KpRI9e/ZEt27dik3+zZjYuNhpOSLC6NGjoaOjgy1btqi1z+LFi6FSqbB4\n8WKRsyvfH3/8gUOHDpU6BdmbzsvLC507d8bkyZPV2n7EiBHo2LEjpkyZInJmwrOyssLFixdhZWVV\n7raJiYmQSqXYuXMnunfvroHsGOOJoLXe1q1bIZPJNDJzCfs/GRkZOHHiRKmrF5RlzJgx2Lp1q3hJ\nvSHMzMywfft2+Pj4lLrcEWNi4GKnxSIiIjB79mzs27cPdevWre503il79+5F7969K3RJskePHkhK\nSkJ4eLiImb0ZevbsCT8/P3h7e0OpVFZ3OuwdwMVOS2VnZ2PYsGFYtWrVG3kDh7bbunUrxowZU6F9\ndHR0MGrUKGzbtk2cpN4w33zzDVQqFZYuXVrdqbB3ABc7LUREmDRpEjp06FDihhQmvsjISMTExJS5\n2vrrjB49Gjt37kRhYaEImb1ZdHR0sHv3bmzatAlBQUHVnQ7TclzstBCP01Wvbdu2wcfHp1Krb7dq\n1QrNmzfHyZMnRcjszcPjd0xTuNhpGR6nq15KpRLbt2+vUo/6XblR5QUev2OawMVOi/A4XfU7c+YM\nzMzMYG9vX+kYw4cPx6lTp5CamipgZm82Hr9jYuNipyV4nO7NUJkbU15lYGCA/v37Y8+ePcIk9Rbg\n8TsmNi52WoLH6apfZZ6tK8u7dikT4PE7Ji4udlqAx+neDH/++Sd69eolyHRf79Izdy/j8TsmFi52\nbzkep3tzCHEJ84V37Zm7l/H4HRMDF7u3GI/TvTkiIyMRHR1dqWfryvIuPXP3Mh6/Y2LgYvcW43G6\nN0dVnq0ry7v2zN3LePyOCY2L3VuKx+neHEI8W1eWd/FGlRd4/I4JiYtdNSMiJCcnIyYmBk+ePEFm\nZma5+4g5TpeWlobnz58jIyMDaWlpgsauqMzMTKSmpiInJwfJycmoztWocnNzERcXh5iYGCQmJkKl\nUhW9J8SzdWUp7Zk7uVyO+Ph4REdHIyEhoVovcyoUCiQkJEChUCApKQn5+fmCxq/I+B0R4enTp0X/\nlrKzswXNhb3deD27ahAZGYkdO3YULegJAHXr1gURIT09HY0aNYJUKkXXrl3h4+MDAwODon0rsz7d\n6yQkJGD79u34999/IZPJkJ2djTp16gD47wNeX18fUqkUHTt2xKhRo2Bubl7lNsuSkZGBXbt24fz5\n85DJZHj69GnRqt4KhQIAIJVK0a5dO4wcORKtWrUSLZf8/Hzs27cPp06dQmhoKGJiYmBoaIiaNWsi\nLy8P+fn5cHZ2hpubG27fvo2BAweKtg6dp6cnGjRoAJVKBZlMhnv37sHAwAC1atWCXC5HZmYm7Ozs\nIJVK4eHhgZ49e6JGDXG+xxIRLly4gP379yM0NBS3bt2Cvr4+lEoldHV1kZGRgRYtWkAqlaJv377w\n8PBArVq1qtTm69a/u3XrFnbt2oVr164hLCwMurq6qFOnDpRKJdLT02FhYQGpVIru3bvDy8sL+vr6\nVcqFvcWIacyJEyeoR48e1KhRI5o1axYdPXqUEhISim2jUCjozp07tGPHDvL09CQDAwPy9fWl+/fv\nExHR5s2bydbWlrKzs6uUS0hICA0dOpQMDQ1pwoQJtG/fPnr06BGpVKqibVQqFT169Ij27dtHEydO\nJENDQ/rkk0/o6tWrVWr7Vffv3ydfX18yMDAgT09P2rFjB925c4cUCkWx7RISEujo0aM0a9YsatSo\nEfXo0YNOnDghaC5Pnz6lmTNnkrGxMfXt25cCAgIoLCyM5HJ5se1SU1Pp1KlTtHjxYjI1NSVnZ2fa\ntm0bKZVKwXLJzs6mZcuWkZmZGTk6OtLPP/9M//77L+Xk5BTbLisriy5cuECrV68mZ2dnsrGxoR9/\n/JHy8/MFy6WwsJA2btxIdnZ21Lp1a1q+fDmdPXuWMjIyim2Xn59P165dow0bNtCHH35IjRs3pvnz\n51N6enqV2j916hSZmZlRYmIiqVQq2r9/P3Xq1IksLCxo3rx59Pfff1NycnKJnMPDw2nLli3k4eFB\nhoaGNHnyZIqNja1SLuztxMVOA1JSUmjEiBFkY2NDu3btqtCHUFJSEi1evJiMjIxo9uzZZGRkRLdv\n3650Lnl5eTRr1iwyNTWldevW0fPnz9XeNzMzk3755RcyNTWlmTNnUm5ubqXzICJSKpX0008/kZGR\nES1evJiSkpLU3jc/P5927dpFLVq0oBEjRlBKSkqVciEi2rt3L5mamtLUqVPpwYMHau+nVCrpxIkT\n1K5dO/rwww/p0aNHVc4lODiYbGxs6NNPPyWZTKb2fiqViv7991/q378/2dvbU2hoaJVzuXPnDrVr\n1466detGQUFBxb4Qlef27ds0duxYsrS0pOPHj1cpj2+++YY6depEAwcOJHt7ezp48CAVFhaqvX9s\nbCzNmTOHjI2N6bfffqvQcbC3Hxc7kV25coXMzc3pyy+/LPGNvCLu379PUqmU7O3tKS0trVIxHj16\nRG3atKFhw4aV+BZcEcnJyTRs2DBq3bo1PXz4sFIx0tLSqFu3btS5c2eKioqqdC45OTn05ZdfkpmZ\nGV2+fLlSMeRyOfn4+FDr1q0rHYPov175999/T0ZGRrR///5KxVCpVDR//nwyNzenw4cPVzoXlUpF\nO3fupEaNGtGqVasqHef3338nIyMjWr9+fZV6rWfOnCFra2uaPHlyiR67uk6ePEn6+vo0d+7cEj3t\nirh58ya5urrSgAEDqnyFhL09uNiJ6Pz582RiYkLHjh0TJJ5SqaTp06eTo6NjhXsyUVFRZGlpSb/8\n8osguRAR+fv7k6WlZYWLVWpqKjk5OdG0adMEu+x37NgxMjExofPnz1doP7lcTh999BENGTKkyj3V\nF8LCwsjMzIx27NhRof1UKhVNmTKF2rZtW6UvIy978uQJtWnThr755psK77t27Vpq2rQpRUZGCpLL\n8+fPqXv37uTl5VXhgnf06FFq1KgRBQcHC5JLQUEBjRkzhjp27EhZWVmCxGRvNi52IomIiCATExM6\nffq0oHFVKhXNmjWL2rdvr/a32+TkZLK2tqaAgABBcyEiCggIoKZNm6r94SyXy6l9+/Y0c+ZMwS8j\nnT59mkxMTCg8PFyt7VUqFXl5edGQIUMqdDlMHXfu3KHGjRvT33//rfY+ixcvJqlUWqFLy+pITk6m\n1q1b07p169TeZ/fu3WRlZUUxMTGC5pKXl0fdu3enKVOmqL3P5cuXycTERPCxYqVSSWPHjqXevXsL\nOtbK3kxc7ERQUFBArq6u9Ouvv4oSX6VS0cCBA2nBggVqbT98+HCaMWOGKLkQEc2YMYOGDRum1rYL\nFiygAQMGiDZeEhAQQK6urlRQUFDutjt37iQ7OzvKy8sTJZdz586Rubk5paamlrvtlStXyNTUtELj\nlhXx6NEjMjY2pjt37pS77ZMnT8jY2Jhu3LghSi7Pnz+npk2bqvVFICcnh1q0aEEHDx4UJReFQkGd\nO3emNWvWiBKfvTm42Ilg6dKl1KdPH1EHwBMSEsjExKTcmxf27dtHrVq1EuwSXWlyc3OpVatW9Oef\nf752O5lMRiYmJiXuQBWSSqWiPn360Lfffvva7RITE6lRo0Z07do10XIhIvriiy/Ix8fntdvk5eVR\n69atae/evaLm4u/vT+3bt3/tJUSVSkX9+vWjxYsXi5rL6dOnqUmTJiXu5nzVtGnTaMSIEaLmEhUV\nRUZGRkV3PDPtxMVOYCkpKWRgYECPHz8Wva2AgADq2bNnme8rFApq2rSpYOMcrxMcHExNmzZ97Qdp\nr169RLmU+qrY2FgyMDB47bjmlClTRO3tvpCdnU2Wlpav/VLyyy+/iNrbfUGpVFLHjh1pz549ZW5z\n6tQpatOmjVo946oaM2YMLVy4sMz3o6OjycjISJA7bcuzYsUK8vT0FL0dVn242Als1apVNHLkSI20\nlZ+fT40aNSrzBoKjR49Su3btNJILEVG7du3o6NGjpb4XGRlJjRo1EvTZr9cZNWpUmXchZmVlkaGh\nIT158kQjuSxbtox8fX1LfU+lUpGtrS2dO3dOI7ns37+f3N3dy3zfw8ODNm7cqJFcwsPDydzcvMzC\n+vXXX9NXX32lkVzS09PJwMCAEhMTNdIe0zwudgJSqVRkY2NTpdvXK2ru3LllfiB89NFHtHXrVo3l\nsnXrVurXr1+p73311Vc0d+5cjeVy5coVsrGxKbW3tHHjRvLw8NBYLklJSWRgYFDqg9Xnzp0jW1tb\njT3zVVBQQBYWFnTr1q0S7z158oQMDQ0rfXdidHQ0SSSSMm/2WLhwYYlLul26dCn1MQ25XP7aL3JV\nUVoeRER+fn60dOlSwdtjbwaeG1NADx8+hFwuR/v27dXe3sjICNevXwfw39RdJiYmCA4OVrvNYcOG\n4Z9//inxemFhIc6ePYuPP/5YY7l8/PHHOHfuXKlzNZ48eRJDhw7VWC7t2rVDQUEBHj58WKVchMjH\n1NQUbm5uuHjxYpm5SCQSjeSiq6sLDw8PBAYGlnjvzJkz6NWrl2hTapV2jGX9/YaFhcHMzAzvv/++\nRvJ4kcu7uMLEu4KLnYBkMhnatm2r9geXjY0NVq5cCR8fH+Tl5eGzzz7DZ599hi5duqjdpp2dHWJi\nYkpMenv79m1YW1ujXr16GsulXr16sLa2xu3bt4u9npOTg5iYGLUnShYiF4lEAjc3t6K5R1/24vek\nLiHy0cZchFAduVAZ0wFLpVLcuHGj2CTfTHtwsRNQWFgYpFJphfbx9fVFixYt0K5dOzx9+hTLli2r\n0P61atWCvb09bt68We25AP99YISFhRV77ebNm7Czs6vQhMBC5fLqB2lqaioyMjJgY2NToVhVzae0\n8wJUz++ptPPyulxWrFiBFi1aoH79+rCzs8OhQ4cA/Le00cyZM2FiYgIbGxscP3682H7R0dHo2rUr\n6tevj969eyMlJaVEbCcnJ9y7dw9yubzcXKytrbFq1So4OjqiXr16GDduHJ4+fYp+/fqhQYMG6NWr\nFzIyMnDu3Dk0adKkxL7lLQTbsGFDmJiYICoq6rXbsbcTFzsBPX36FBYWFhXez9fXF7dv38bUqVOh\nq6tb4f0tLCyQnJz8xuTy9OnTNyaX0s6Lubl5pVYFqEo+pZ0XpVKJ1NRUNG7cWOO5vHpegLJ/Ty1a\ntMDFixeRmZmJhQsXFi2oGhAQgOPHj+PGjRsIDQ3F/v37i13VGDFiBNq2bYvU1FQsWLAA27ZtK3HV\n47333oO+vj4yMjLKzUUikeDgwYM4c+YMIiMjcezYMfTr1w8rVqxAcnIyVCoV1q5dW+qVFXWvtpR1\nbtjbj4udgAoLCyu8UnV2djamTZsGX19fLFy4EOnp6RVuV1dXFwUFBW9ELrVq1XpjchHqvAiRT2m5\nKBQK1KxZU+0PYjFzAf47N6UVzqFDhxYV5OHDh6Nly5YICQnBvn378NVXX8HCwgKGhob43//+V3SJ\nMDY2FqGhofj222+hq6sLd3d3DBw4sNRLiLq6uiXGecv6PU2dOhUmJiYwNzeHu7s7PvjgAzg5OUFP\nTw8eHh5FY5mVVda5YW8/LnYC0tPTq/DilV9++SXatWuHX3/9Ff3798fEiRMr3G5+fj5q165d7LXa\ntWtXSy55eXklcnmTzktlchEin9JyqVWrFhQKRYVX4RYjF6Dsc7N9+3a4uLjA0NAQhoaGiIiIQEpK\nChISEopdLrSysir6OSEhAYaGhnjvvfeKXmvatGmZ+ejp6RV7ray/X1NT06Kf33vvvWL/X7t27Sov\n2FrWuWFvPy52AmrWrBnu37+v9vaHDx9GYGAgNmzYAAD48ccfERYWhj179lSo3aioKDRr1uyNyaV5\n8+ZvTC6vnhcrKyvEx8eXGCMSO5/SzotEIkHTpk3x4MEDjefy6nkB/vs9vTpe9fjxY4wfPx7+/v5I\nS0tDeno67O3tQUQwMzNDbGxs0bYv/2xmZob09HTk5uYWi/VqLzY1NRUKhQJGRkYlclHnb6a0nmLd\nunWLtatUKvHs2TO1YpV1btjbj4udgMq6s6wsgwcPxpMnT4pWIq9bty7u378PLy8vtWNkZWXhyZMn\nsLW1LfZ6WTchiJkL8N9ddK/eWGBra4u4uDhkZWVpPBc3N7dir9WpUwc2NjaIiIhQO44Q+ZR2XoDq\n+Zsp7byUlUtOTg4kEgmMjY2hUqmwZcuWonM3fPhwrF27FvHx8UhPT8eKFSuK9mvatCnc3NywcOFC\nFBYW4uLFizh27Fipubi4uJQYQ63o3+/L3n//feTn5+PEiRMoLCzE0qVL1fpyExsbi5o1a8Lc3LxS\n7bI3Gxc7Ab34B1rac2ZiCQkJgYODQ4nxjWbNmiEvLw9xcXEayyUuLg65ubklvhnXrFkTjo6OCAkJ\n0VguhYWFry0wV65c0VguAHDlypW3MhdbW1vMmDEDH3zwARo3boyIiAh07twZEokEfn5+6NOnD5yc\nnODm5oZPPvmkWM9t9+7duHr1Kho2bIglS5Zg9OjRFc6lrMcEXni5PYlEAolEgvr162P9+vXw9fWF\npaUl9PX1i11ufbFdWblUdAyVvSWq8YF2rdSlS5dyJ0QWko+PD61evbrU98aPH1/uhMhC+vbbb8nP\nz6/U91avXl3uhMhC2rdvH3Xp0qXU944dO0Zubm4ay+XmzZtlTosVFRVFJiYmoq288Kq0tDQyMDAo\ndXWFF9OoaWJeV6L/5ups3rw5XblypcR7KpWK7OzsKCgoSCO5EBH17dtXozMOMc3iYiewP/74g7p1\n66aRtpKTk8nAwKDMJWSuX79OlpaWgq/VVprCwkKytLSk69evl/p+amoqGRgYCLYoaXk+/PBD+uOP\nP0p9T6FQkLW1NYWEhGgkl4kTJ9KiRYvKfL93794VXui1sn766Sfy8vIq8/2pU6fSvHnzNJLL33//\nTa6urmVOlebv709Dhw7VSC4PHjwgY2NjUVcHYdWLi53A5HI5WVpaamRi3xkzZpQ5wfAL7u7uoq2r\n97Jff/2VOnfu/NptfH19NbLSwPnz58nCwuK1i9v+8MMP1L9/f9HnpHz06BE1bNiQ4uPjy9zm6NGj\nZGtrK/ok2ZmZmWRlZUWXLl0qc5u7d++SiYkJPX36VNRcFAoFdezYkTZv3lzmNs+fPxd1Xb2XjR49\nmubMmSN6O6z6cLETweHDh8nGxoays7NFa+Pff/+lxo0bl9tTunXrFhkbG4t6aerx48dkbGxc6uTC\nL0tOTqbGjRvTv//+K1ou2dnZZGNjQ4cPH37tdnK5nBwcHGj79u2i5aJUKqlbt270/fffv3Y7lUpF\ngwcPFr1HNWHCBBo7dmy5282ePZs++eQTUb8I/PTTT+Tu7l7uCuGbN28mZ2dnUZccOn78OFlbW1Nm\nZqZobbDqx8VOJD4+PjRhwgRRPjAyMjKoVatWtG/fPrW2X7p0KfXo0UOUD4yCggLq2bOn2mODLxaT\nLW/RzspQqVQ0ceJEtccGXywmGx0dLXguRP99oHfo0OG1a/y98GIx2df1uqri+PHjai2WSvTfYrJt\n2rQR7YtAeHi42oulvlhM9n//+58ouTx9+pQsLS3pzJkzosRnbw4udiJJT08nBweH147VVEZWVha5\nu7vT5MmT1d6noKCA+vXrR97e3mp98KpLoVCQj48P9e3bt0KFdMqUKeTu7l7ppWTKsnjxYrK3t6e0\ntDS191m3bh3Z2NhQXFycoLns2rWLzM3N6cGDB2rvc+LECWrUqJHgl+0uXrxIxsbGdOHCBbX3uXnz\nJpmYmNCxY8cEzeX+/ftkaWlJu3btUnufxMREat68Oa1bt07QXFJTU8nZ2Zm++eYbQeOyNxMXOxEl\nJCSQra0tffXVV68dP1JXfHw8dejQgcaOHVvu5Z9X5eTkUM+ePemTTz6h58+fVzmX58+f0yeffEI9\neuDTtdcAACAASURBVPSgnJycCu2rVCpp3Lhx1L59e0GKjFwup+nTp5OtrS0lJCRUeP8VK1ZQs2bN\nKCIiosq5qFQqWrt2LZmbm5d7Wbc0f/75JzVq1Eiwnsbhw4fJ2NiY/vnnnwrve/nyZTIxMaGtW7cK\ncoUiJCSEzM3NadOmTRXe99GjR9SsWTNavHixIF/YHj58SPb29jRz5kyNrSXIqhcXO5GlpKTQRx99\nRE5OTmXeqVgelUpF27ZtIxMTE1qyZEml/3Hm5+eTn58fWVlZUWBgYKViEBEFBgaSlZUV+fr6VvqW\neZVKRUuWLKnyh+n169fJycmJPvroI3r27FmlYhD9t/CssbExfffdd5W+e/Xx48fUq1cvkkqlal2i\nK8upU6fIwsKCpkyZUuneb1paGo0ePZqaNWtWpTHSQ4cOUb169WjIkCGlPq6gjvz8fJo7dy41atSI\nDhw4UOlc4uLiqEuXLtSxY0e6d+9epWIolUpau3Yt6evr00cffcSF7h3CxU4DVCoVbd26lUxMTGj0\n6NF09epVtfaTy+W0d+9e6ty5c5WK5atOnjxJVlZWNGjQIDp58qRavUSlUkknT56kQYMGkZWVFZ08\neVKQXF4Uq86dO9PevXvV7gGHhITQmDFjBO15xMTEUM+ePcnOzo42btyodqGJjIykadOmkZGRUZWK\n5cvS0tJo1KhR1KRJE1qxYoXaj2zEx8fTokWLqHHjxjR58uQqXSrOyckhe3t7Wr9+PX399ddkYmJC\nX3/9tdpjnOnp6fTzzz9TixYtaMiQIZSYmFjpXF5QKpW0bt06MjIyookTJ9LNmzfV2i8vL4+2bdtG\nUqmUPvjgAwoODiYzMzMeq3uHSIjKmaKACSYlJQWbN2/Ghg0b0KBBA3Tp0gVSqRT29vaoV68elEol\nUlJScP36dYSGhuLUqVNo1aoVJk2aBA8Pj0otc1OWnJwc7Nq1C/7+/sjJyUH37t0hlUrh7OyMBg0a\nQCKRICMjAzdu3IBMJkNQUBDq1q2LyZMnw9vbG3Xr1hUsl8LCQhw6dAj+/v6IjIxEr169IJVK4erq\nCmNjY+jo6CArKwsRERGQyWS4cOECMjIy8Pnnn2Ps2LEwNjYWLBciwunTp7F+/XqcP38ePXv2hFQq\nhVQqhbm5OXR1dZGbm4t79+5BJpPh8uXLiIqKwtixYzFx4sQyJzuurGvXrmH9+vX466+/4O7ujrZt\n20IqlaJp06bQ09ODXC7Hw4cPIZPJEBISgqtXr8LT0xOTJk2Cg4NDldr29fVFfn4+duzYAYlEgqio\nKGzcuBHbt2+Hk5MT2rdvD6lUipYtW+K9995DQUEB4uLiIJPJEBoairNnz6Jv3774/PPP4e7uLujM\nJImJiQgICMCmTZtgaWmJTp06QSqVwtbWFnXr1oVCoUBycjLCwsIgk8kQGBgIqVSKSZMmoX///tDR\n0cHp06cxatQohIWFVWqZJfZ24WJXDZRKJS5duoSQkBCEhobi7t27yM3NhY6ODgwMDODs7AypVAp3\nd3e0bt1a1FyICGFhYbh8+TJCQ0MRHh6O7OxsEBHq1asHBwcHSKVSdOzYEa6urqJPpRQZGYng4GDI\nZDLcuHEDGRkZUCqVqFOnDtq0aQOpVIp27dqhc+fO0NHRETWXuLg4nD17FjKZDGFhYXj27BkKCwtR\nu3ZttGjRAm5ubpBKpejevXuJWfuFlpqaiqCgIMhkMshkMiQkJEAul0NPTw9NmjQplkuDBg2q3N6O\nHTuwbNkyhIaGQl9fv9h7ubm5OHPmTFEu0dHRiIuLg6mpKczMzIpy+fDDD0UvIoWFhQgODsa1a9cQ\nGhqKqKgoxMfHo379+jA3N4eLiwukUim6du1aYiJuAFi4cCEuXryIwMBA0f+eWPXiYscYK+bu3bvo\n0qULzpw5A0dHR7X2sbKywsWLF4st81NdevfujZkzZ6J3797lbqtUKtGzZ09069YNCxcu1EB2rLrw\nRNCMsSK5ubkYPnw4li9frnahe5vp6Ohg9+7d2LRpE4KCgqo7HSYiLnaMsSJffPEFnJycMG7cuOpO\nRWPMzMywfft2+Pj4ICkpqbrTYSLhYvf/2rvzsKjL/X/8T1ARN2SZYYldRNxYhkFwJ9NEj5Gap8wV\nUDA1TbOyxY8fbfGkZXY+nQKUPOKSu+aaS5GGKC4Mi4CIJggioiD7Psvr+0c/+WlsM8wM4PB6XFfX\nVTP38pp5d82L+33f9/tmjAH4a54uJiYG4eHhHe6Ym3HjxiEkJASzZs1S+eR49nzgZMcYQ1paGlas\nWIH9+/fXW5DSUfzv//4vFAoFvvjii7YOhWkBJzvGOriONk/XGJ6/022c7Bjr4DriPF1jeP5Od3Gy\nY6wD68jzdI3h+TvdxMmOsQ6K5+kax/N3uoeTHWMdEM/TNY3n73QPJzvGOiCep2sez9/pFk52jHUw\nPE+nPJ6/0x2c7BjrQHieTnU8f6cbONkx1kHwPF3L8PydbuBkx1gHwfN0Lcfzd8+/zm0dAGNMeUSE\nu3fvIjc3F7W1tTAwMICtrS1sbW2bnH97Mk8XFxen0Xm63Nxc3L17F9XV1bh27Rpqamrg5OQEff3W\n/zu6sLAQt27dQmFhIZKTk2FtbQ0XFxd07qyZn7mn5++aO/+uvLwcN2/eRHl5OfT09NC7d2/0798f\nhoaGGomFqY7Ps2OsnSsuLsb27dtx4sQJxMfHo1u3brC3t4eBgQFqamqQkZEBhUIBsViMKVOmYNas\nWc/Mx7XkfLrGVFdXY//+/Th48CDi4uJQW1sLZ2dn1NTUwNDQELm5uSguLoZIJMKECRMwb948CIVC\ndb+CBsnlcpw8eRI//fQTrl27hoKCAvTv3x9EhM6dO6OwsBD379+Hq6srxowZg5CQEDg6OqrdZ0Pn\n3xERLly4gMjISFy+fBlZWVlwcXFBr169QEQoLCxERkYGXFxcMHLkSISEhPCt5NZGjLF2KTc3l0JC\nQsjY2JhmzJhBx44dowcPHtQrp1AoKCcnhw4fPkxTp04lExMTWrp0KRUUFFBFRQUNHjyYIiIi1Iql\ntLSUPvroIxIIBDRx4kTau3cv3b17lxQKRb2y+fn5dPr0aZo3bx4ZGxvT7NmzKSMjQ63+n1ZbW0sb\nN24kOzs7GjZsGP3444908+ZNksvl9cqWlJTQ+fPnacWKFWRmZkaTJk2iuLg4tfrPzc0lKysrioqK\nIoVCQdu3b6dBgwZR//796dtvv6XExESqra2tV6+yspIuX75Mn376KVlbW9OIESPo1KlTasXClMfJ\njrF2RqFQ0I4dO0goFNJHH31EeXl5KtXPzs6mpUuXkpWVFY0bN45mzZrVYFJS1q+//kr29vYUGBhI\nf/75p0p1Hz9+TF988QUJBAL64YcfGkxIqkhKSiKRSEQTJkyg+Ph4lepWVFRQeHg4mZub0yeffELV\n1dUtjuPXX38lc3NzGjNmDHl6etYlPmVJpVI6ePAgOTg4UGBgIBUVFbU4FqYcTnaMtSO1tbU0a9Ys\ncnV1JYlEolZb0dHRZG1tTQsWLGhRklEoFLR69WqytbVVewSSlpZGQ4cOpUmTJlFFRUWL2ti2bRsJ\nhUL673//q1byfvDgAU2ePJlEIpHKf0g88dtvv1Hv3r1pzZo1DY7ilFVWVkaLFy8mOzs7Sk1NbXE7\nrHmc7BhrJ2pra2ny5Mnk7+9PlZWVGmmzpKSEfH19KSAgQKWEp1AoaMWKFSQSiejRo0caiUUqldKc\nOXPI19dX5c+3efNmsrOzo5s3b2okFoVCQZ9++in169evwVvDTTlz5gwJhUI6f/68RmIhItq1axdZ\nWlpScnKyxtpkz+Jkx1g78dZbb9E//vEPqqmp0Wi7FRUVNHLkSPr444+VrrNp0yZyc3OjwsJCjcYi\nl8tp5syZ9Nprryk9Ojt+/DhZW1urfAtVGWvXriWRSKT0Lc2kpCQSCAQUExOj8Vh2795N1tbW9PDh\nQ423zTjZMdYunDx5khwcHKi0tFQr7T969IgsLS3p0qVLzZZNTU0lgUBAmZmZWomlpqaG3NzcaMeO\nHc2WLSgoICsrK4qOjtZKLAqFgqZMmUKrVq1qtmxtbS2JRCLaunWrVmIhIvrwww9p2rRpat2mZQ3j\nZMdYGysqKiIbGxuKiorSaj8HDhwgFxeXJm8hymQy8vb2prCwMK3GIpFISCgU0v3795ssN2vWLFq2\nbJlWY3nw4AGZm5s3u0rz888/pwkTJmg1EVVVVdGAAQNo7969Wuujo+J9doy1sS+//BJpaWnYsWOH\n1vv6xz/+gSlTpmDBggUNvn/48GFs2LABsbGxWt8YvmLFCujp6eGbb75p8P3U1FSMGzcOd+7cQffu\n3bUay+bNm3Hs2DGcPHmywfdLSkrg4OCApKQk2NnZaTWWP/74A8HBwUhPT2+Tzfm6ir9JxtqQXC5H\neHg4li9f3ir9LV++HD/88AMa+xs3NDQUy5Yta5Uf2aVLl2L79u2orKxs8P2wsDAsWLBA64kOAObM\nmYMrV64gIyOjwfd37NiB8ePHaz3RAcDo0aPRs2dP/Pbbb1rvqyPhZMdYGzp9+jSsrKzg6enZovp3\n796Fvr4+FApFg++vXbsWc+bMqfvvcePGobKyEleuXKlX9vbt20hOTsa0adPqXnNwcMA333wDd3d3\nGBsb480330RNTQ0A4MSJE/Dw8ICJiQlGjBiB5OTkunrx8fEQiUQwMjLCG2+8genTp2P16tXP9Ofo\n6IihQ4di//799WKpqKjA7t27ERISotoX0kLdu3dHQEAAIiIiGnw/PDwcixcvblHbql4jPT09LF68\nGOHh4S3qjzWMkx1jbejcuXOYPHmy1tr/+3Mw9fX1MXnyZJw7d65e2fPnz2PChAno2rXrM/UPHDiA\nM2fOIDMzE9evX0dkZCQSEhIwf/58REREoLCwEG+99RZeffVVSKVS1NbWYurUqZg3bx6KioowY8YM\nHDlypMFncjYWi0QiQb9+/WBjY6PU57xz5w7MzMyQkJAA4K9ndgqFQkRHRytVv6lYHj58iAcPHmD0\n6NFKt6WKpr4XnmXSHE52jLWhuLg4iMXiVu1TLBYjLi5O6VjeeecdWFpawsTEBP7+/khMTERERATe\neustDBkyBHp6epg7dy66du2K2NhYXL58GXK5HEuXLkWnTp0wdepUeHt7NxqLRCKp97pEIlHpe3Fy\ncsKGDRswe/ZsVFVVISgoCEFBQSolKJFIhOTkZEil0nqxeHp6tupBt+bm5ujZs2ejt1WZ6jjZMdaG\nEhISGryFuX79evTt2xdGRkYYNGgQjhw5AuCvOb73338fQqEQTk5O9RZUZGZmwtfXF0ZGRhg/fjwK\nCgrqtS0Wi+tGQMrEYmlpWffv3bt3R3l5ObKysvDNN9/AxMSk7p+cnBw8ePAAubm5sLa2fqYNW1vb\nBkcpgwcPRkZGBqqqqpSKpSnBwcHo27cvvL298fDhQ6xbt06l+r169YKdnR1u3rypVCzavEZA49eJ\ntQwnO8baiEwmQ3l5OczMzOq917dvX8TExKC0tBRr1qypO0ctIiICJ0+eRGJiIuLi4nDw4MFnRhwz\nZ87EkCFD8PjxY6xevRrbt2+vNyIxNzdHYWFhvT4LCwthYWGhVOy2trZYtWoVioqK6v4pLy/H9OnT\nYWVlhfv37z9TPjs7u8GRkYGBAXr16oWSkpJnXn/8+LHSsTwtODgYqampWLp0Kbp06aJy/Ya+m8a+\nF21eo8ZiYS3HyY6xNiKXy9GpU6cGf+j++c9/1o2o3njjDTg7O+Pq1as4cOAA3n33XVhbW8PExASf\nfPJJ3YgpOzsbcXFx+Pzzz9GlSxeMGjUK/v7+9UZUnTt3hkwmq9enTCZr9uy3J22FhIQgPDwcV69e\nBRGhoqICJ0+eRHl5OYYPH45OnTrh+++/h0wmw9GjR3Ht2rVG22woniffjSrKy8uxfPlyBAcHY82a\nNSgqKlKpfmOxyGSyBmPR5jVqLBbWcpzsGGsjBgYGkMvl9eaIgL+WuotEorpbhCkpKSgoKEBubi5s\nbW3ryj29FD43NxcmJibo1q1b3Wv29vb12q6oqHimzBPdunVDRUVFkzHr6elBT08PYrEYERERWLJk\nCUxNTeHs7Fy3T7BLly44fPgwtm7dChMTE/z000945ZVXYGBg0GCbDcXTrVu3RrckNGbZsmXw9vbG\nli1bMGnSJCxcuFCl+qrGos1r1FgsrOX4pHLG2oienh6cnZ2Rlpb2zEGeWVlZWLBgAX7//XcMGzYM\nenp6EIlEICJYWVkhOzu7ruzT/25lZYWioiJUVlbW7U3LysqqNypJS0tDv3796sXTr18/3LhxA4MH\nD657LTMz85kyTx9Y6ufnBz8/vwY/29/nm3x8fPDqq6/WK5ebm4tOnTrB1NT0mdednZ1x48aNZ7ZB\nNOXo0aM4e/Zs3faHTZs2wcPDA3v27MGMGTOUakOhUDT43fTr1w+///77M69p+xoBf12nxjb/M9Xx\nyI6xNtTQasSKigro6elBIBBAoVBg27ZtSElJAfDX7bLvvvsO9+/fR1FREdavX19Xz97eHl5eXliz\nZg2kUiliYmJw4sSJen02tuqysZWRLREdHY28vDzIZDJs374dKSkpmDBhQr1yT1Zd/v1WrqqxTJ48\nGffu3YOxsTEAoEePHrh9+7bSiQ74a/uCsbExBAJBs7Fo+xrV1tYiJSUFHh4eSsfPmsbJjrE2NGTI\nEFy8ePGZ1wYOHIj33nsPw4YNg6WlJVJSUjBy5Ejo6ekhJCQEfn5+cHd3h5eXF6ZNm/ZMoti9ezeu\nXLkCU1NTfPbZZwgICKjX58WLFzFkyBClYmmp9PT0ug3n3377LQ4ePNjgIo+mYrl8+XKrzlnFxMQ0\nGMvAgQORk5OD/Pz8Z17T5jWSSCRwcnJCjx49tPNhO6I2eSInY4yIiO7fv08mJiZUUlLSKv3l5uaS\nsbExFRcX13uvpqaGLCws6MaNG60SS01NDVlaWlJKSkqD7w8dOpSOHDnSKrEQEQ0fPpwOHz7c4HuB\ngYG0YcOGVotl3rx59OWXX7Zafx0Bj+wYa0MvvPACxo4di127drVKfxEREZg+fTp69+5d7z0DAwME\nBwcjLCysVWI5cuQIXFxcMGjQoAbfX7x4MUJDQ1sllsTERGRlZcHf37/RWMLCwiCXy7UeS1FREQ4f\nPox58+Zpva8Opa2zLWMd3YULF8jW1rbB0ZYm5eXlkbm5eZOnYWdnZ5OZmZlWDkp9WlVVFQ0cOJB+\n/vnnJsvY2NjQ77//rtVYFAoFTZo0idavX99kueHDh1N4eLhWYyEiWrFiBQUFBWm9n46Gkx1j7UBI\nSAiFhIRorX2FQkFTp06ljz76qNmyGzdupNGjR5NcLtdaPB9//DFNnTq12bPhTpw4QY6OjlRWVqa1\nWCIjI8nd3b3ZE+JTUlJIIBDQ3bt3tRbLxYsXydLSkvLz87XWR0fFyY6xdqCkpITs7OwanTNS148/\n/kgDBw6k6urqZsvKZDIaPnx4syOdljp37hyZm5tTXl6eUuUDAgIoMDBQK4em3rp1i4RCISUkJChV\n/l//+hf5+voq9T2qqqCggJydnenQoUMab5txsmOs3bh69SoJhUI6e/asRts9cOAAWVpaqrTwJCsr\ni+zs7Gjz5s0ajeXy5cskFArpt99+U7pOaWkpeXl50fvvv6/RhJeRkUH29vYUERGhdB2pVEpTp06l\n1157rdmRoCqKiorI29ubPvjgA421yZ7FyY6xduTChQtkbm5OW7duVfuHXS6X07///W+ysrJSeuTy\ntFu3bpG9vT2tXbuWpFKpWrEQER06dIiEQiGdPHlS5boFBQXk5eVF8+bNo/LycrVjuXTpEtnY2ND3\n33+vct3q6mqaPHkyvfzyy0qPTpuSnp5Obm5utHz5cq2MXtlfONkx1s6sW7eOjI2N6ZVXXqH79++3\nqI07d+7Qiy++SMOGDaPbt2+3OJacnBwaN24cDRkypNEtAs0pKCigGTNmkLOzM8XGxrY4lpKSEpo7\ndy716dOHzp8/36I2Kisr6b333iNLS8smF8c0RyqV0scff0wWFha0Z8+eFiUpmUxGmzZtIoFAQN9/\n/z0nOi3jZMdYO3Ljxg0SCAQUFxdHa9euJRMTE1q4cCFdv35dqfpXr16lwMBAMjExoY0bN5JMJlM7\nJoVCQVu2bCEzMzN68803KTo6Wqkf5vT0dHr33XfJzMyMVqxYQRUVFWrHQkR0/PhxsrGxoQkTJtCx\nY8eU+oy5ubn06aefkrW1NU2fPl1jC0CuXr1KgwYNIh8fH9q+fTtVVVU1W6eoqIj+/e9/U//+/cnX\n11frK1/ZX/SI+ChcxtqDyspK+Pj4YNmyZQgODgbw17Mjf/zxR2zZsgUmJiYYMmQIxGIx7O3tYWBg\ngJqaGmRkZEAikeDq1auoqanBokWLEBQUBKFQqNH4iouLsWPHDoSFhaG2thY+Pj4Qi8VwdnaGoaEh\npFIpcnJyIJFIEBcXh/v372P+/PlYsGABHBwcNBpLdXU1Dhw4gNDQUGRlZWHo0KEQi8UYOHAgevTo\nAZlMhkePHiE+Ph4SiQRpaWmYPn06Fi1a9MxzSDVBLpfjyJEjCAkJAYC6WNzc3GBkZASFQoGioqK6\nI38kEgl69OiBffv21T11hWkfJzvG2ong4GBUV1dj586d9X4AZTIZEhMTIZFIIJFIkJOTg9jYWIwY\nMQK2trbw8vKCWCyGq6urykfjqIqIkJqaWhdLZmYmrly5Ajc3t2diEYlE6Nq1q1ZjAf56pmVcXBzi\n4uJw69YtVFZWonPnzjAzM4NIJIJYLIaXlxd69uyptRgOHDiAzZs3Y8eOHbh27Rri4uKQmppa9wxN\nIyMjuLu718Xi6emJY8eO8bMvWxEnO8bagZ07d2LdunWIi4tT6ke5qqoKpqam9U74bit2dnaIiYl5\n5jibjmTSpEmYMWMGZs+erVT51atXo7y8HN9++62WI2NPcLJjrI2lpaVh9OjRiIqKUvoWGye79uPB\ngwd1D4tW9sHNf/75J4YPH46cnJxGz/ljmsXPxmSsDVVWVuKNN97Al19+qfG5JNY6du3ahWnTpql0\nQkHfvn3h4uKCU6dOaTEy9jROdoy1oXfeeQfu7u6YP39+W4fCWoCIEBkZicDAQJXrBgYGIjIyUuMx\nsYZxsmOsjezcuRMxMTEIDw/nFXnPqbi4ONTU1GDEiBEq13399ddx7ty5Z87JY9rDyY6xNpCWloYV\nK1Zg//79Wl0lyLTryaiuJX+sGBkZwd/fH7t379ZCZOzvONkx1sp4nk43VFdXY9++fZg7d26L2+Bb\nma2Hkx1jrYzn6XTD8ePH4eHhodYK1DFjxuDx48dITEzUYGSsIZ3bOgDGOpIn83RxcXE8T/eca+nC\nlKfp6+sjICAA27dv5w3mWsb77BhrJS3ZT9cY3mfXtlqyt64xvOeudfBtTMZaAc/T6ZaW7K1rDO+5\nax2c7BhrBTxPpzvU2VvXGF6oon2c7BjTMt5Pp1vU2VvXGN5zp32c7BjTIt5Pp3vU2VvXGN5zp32c\n7BjTEp6n0z2a2FvXGL6VqV289YCxZjx8+BDx8fHIz8+HTCaDoaEhnJyc4O7uDkNDw0braWOerri4\nGPHx8bh79y5kMhl27doFW1tbiEQiGBkZaawfZVRWViIxMRGZmZkoLy/H4cOHMXDgQHh6ekIgELRq\nLJqSn5+P+Ph4PHr0CFKpFF27dkWfPn3g7u6O7t27a2RvXWOe3nPn4eGBkpISJCQkICcnB7W1tTAw\nMIC1tTU8PT3Ru3dvjfev63jrAWN/Q0S4du0awsLC8Ouvv6KyshKenp6wsrJC586dUVVVhfT0dKSn\np8PFxQXTp0/H/PnznzkZXNXz6ZqSnp6OsLAwnDhxAg8fPqz7sZXJZOjcuTMyMjKQnJwMGxsbTJky\nBW+99RYcHR3V/RoalJubi4iICBw8eBAZGRkYOHAgnJ2dIZfL0blzZ+Tm5iI+Ph4mJiaYMGECFi1a\nBHd3d63Eoinx8fEICwvDmTNnUFpaCk9PT1hbW6NLly6oqqrCrVu3kJaWBmdnZ9TW1uLtt9/GkiVL\ntBLL22+/jWvXrqG4uBi5ublwd3eHg4MDunbtipqaGty9exdJSUl44YUX8Oqrr2LhwoXo27evVmLR\nOcQYq/Pbb7+RWCwmR0dH+uqrr+jOnTukUCgaLFtVVUUxMTEUFBRExsbGFBgYSA8fPqQbN26QQCCg\npKQktWJJSkqicePGkbm5OX3yySeUnJxMMpmswbJSqZQSEhJoxYoVZGZmRv7+/nTr1i21+n9adnY2\nTZ8+nYyNjWnRokV05coVqqmpabCsXC6n9PR0+uyzz8ja2ppGjBhBsbGxGotFU6Kjo8nHx4fs7Ozo\nX//6F926davRa11dXU2xsbE0d+5cMjY2plmzZlFubq7GYrlx4wZNnDiRBAIBLV++nJKSkkgqlTZY\nViaTUVJSEq1cuZIEAgFNmDCBUlNTNRaLruJkxxgRlZaW0sKFC8nGxoYOHz5McrlcpfqPHz+mDz74\ngCwsLMjW1pYiIiJaHEttbS199tlnJBAIKDw8nKqrq1WqX1FRQd988w2ZmZnRt99+q/JneZpCoaAf\nf/yRBAIBrV27lkpKSlSqL5VKadeuXWRhYUHvv/8+VVZWtjgWTSkvL6d33nmHrKysaO/evY3+AdGY\noqIiWrVqFQmFQtqxY0ejCVIZMpmMNmzYQAKBgL777juqqqpSqX5VVRX95z//ITMzM1q/fn2jCZJx\nsmOMsrKyyMXFhebNm0fFxcVqtXX58mVydHSkpUuXtijJlJSUkK+vL/n5+dG9e/fUiuX27ds0cuRI\neuWVV1qUZGpra2nOnDkkEonUHqU+evSIXn/9dRKJRPTw4UO12lJHbm4uDR48mGbOnEmPHz9Wqy2J\nREKDBw+m4OBglRMm0V9Jd/z48TRmzBjKzMxUK5bMzEx66aWX6OWXX6aysjK12tJVnOxYh3bvn+oN\nXAAAG35JREFU3j1ycHCgTZs2aazN4uJiGjlyJC1YsEClv/pLS0vJx8eHFi1apNZo7Gm1tbU0Y8YM\nGjdunEojRJlMRq+//jpNnDiRKioqNBKLQqGg//mf/6GBAwdSfn6+RtpURV5eHjk7O9O6devUGo09\nraysjMaOHUtz5sxR6ZpVVFTQ6NGjKSgoSGOjMalUSvPmzaNRo0Zp7JrpEl6gwjqs6upqeHt7Y/bs\n2Vi5cqVG2y4rK8PYsWMxbdo0fPjhh82WJyJMmTIFQqEQERERGt3DJZfLMX36dPTq1Qvbtm1Tqs7K\nlSshkUhw8uTJJlectsRHH32Eixcv4vz58+jUqZNG226MVCrF8OHDMWnSJKxdu1ajbVdWVsLPzw8v\nvfQSPv30U6XqzJgxA/r6+ti5cyf09TW3A0yhUCAgIAC1tbXYt2+fxtrVCW2cbBlrMx9//DFNmTJF\nY3/l/93du3dJIBBQcnJys2UjIyPJ3d290UUf6iorK6M+ffrQ8ePHmy178eJFsrS01NroSy6Xk6+v\nL23cuFEr7Tfk888/Jz8/P61d69zcXBIKhRQXF9ds2f3795OLi4vW5i+rqqqof//+tH//fq20/7zi\nZMc6pLi4ODI3N6cHDx5otZ/NmzeTl5dXk3M6T34oExIStBrLuXPnyNramoqKihotU1VVRf369aND\nhw5pNZY7d+6QmZkZpaena7UfIqLk5GQSCASUnZ2t1X527txJgwcPptra2kbL5Ofnk4WFhdZXp8bG\nxpKFhUWb3C5ur/gJKqxDWrduHdasWQNLS0ut9hMSEgIATT7R/j//+Q9mzJih9fPMXnzxRfj6+mLr\n1q2Nltm3bx8cHBzw2muvaTWWPn36YNmyZfj666+12g8ArF+/Hh9++CFsbW212s+sWbNgYmKCn3/+\nudEy4eHh8Pf3x9ChQ7Uay9ChQ+Hv74+wsDCt9vNcaetsy1hru3fvHpmYmLTaqrXIyEiaOHFig+9V\nV1eTubl5q4xwiP76i9/JyanRxRTe3t5K3erUhLy8PDI2Nm5ypKmuhw8fkrGxMRUWFmqtj6ft37+f\nfH19G3xPKpWSjY0NJSYmtkosiYmJZG1tzdsR/j88smMdzn//+1/MnDmz1R7M/MYbb+Dq1avIysqq\n997Ro0fh5uaGfv36tUosPj4+MDIywrlz5+q9l5SUhLy8PEycOLFVYrGwsMCECRPw008/aa2PHTt2\nYOrUqTAxMdFaH0+bMmUKbt26hfT09HrvnT59GnZ2dlp5oszatWsxZ86cZ1578vQVPifvL5zsWIdz\n4cIFTJgwQenyd+7cgZmZGRISEgD89cgsoVCI6Ohopep369YNvr6+uHTpUpvHoqenh4kTJ+LChQsN\nxjJ+/HilV0iqGwuARmPRlNb+frt06YKxY8ciJiZG7VhU0djqXW1/v88TTnasQyEiSCQSiMVipes4\nOTlhw4YNmD17NqqqqhAUFISgoCCMHj1a6TbEYjEkEkm91yUSCby8vDgWLWlPn0nVWFRBjewg0/b3\n+1xp6/uojLWm7OxssrCwaFHdV199lQYPHkzu7u5NrrhryC+//ELjxo2r93r37t1b9NQWdWLJyMgg\nGxubeq+LRCK6fPlyq8Yik8moW7duWpk/LSgoICMjoxZtN1DnM50/f56GDx9e73WBQFDveZr29vb0\n9ddfk6urK/Xs2ZPmzZtHeXl5NGHCBDIyMqJx48ZRUVERnTt3rt41s7e3p6ioKCIiWrNmDc2ePbte\nn3l5eWRqaqpS/LqKR3asQykuLoaZmVmL6gYHByM1NRVLly5Fly5dVKpramqK4uLiZ16TSqWoqalp\n0XEt6sZSUlJS7/Xi4uIWHc2jTiydOnWCkZERysrKVO63OcXFxTA1NW3RBn1NX+sn8fz9/z09PT0c\nPnwYUVFRSE9Px4kTJzBx4kSsX78ejx49gkKhwHfffdfgZ1DmczV2rTsiTnasQyGiFv34lZeXY/ny\n5QgODsaaNWtQVFSkUn19fX0oFIp2G0tL41E3lqbiUdfz8v0uXboUQqEQL7zwAkaNGoVhw4bB3d0d\nXbt2xdSpU+vmD1viSSzED8riZMc6lh49eqC8vFzlesuWLYO3tze2bNmCSZMmYeHChSrVLysrQ48e\nPZ557cmIobq6us1jAf76blQdYakbCxE1Go+62tO1bioeCwuLun/v1q3bM/9taGjYos/w91g0+fi5\n5xUnO9ah2Nvbo6CgQKVbO0ePHsXZs2frNuhu2rQJ8fHx2LNnj9JtJCcnY9CgQc+8pqenhwEDBiAl\nJaXVYxk4cGC91wcMGIDk5ORWjSUrKwu9evWCsbGx0nWUZWlpCZlMhocPHypdR1vXGlD++21oFNaj\nRw9UVlbW/bdcLkd+fr5SsTR0rTsiTnasQ+ncuTPc3NxUujU0efJk3Lt3r+4HuUePHrh9+zZmzJih\ndBuNrQD18vJSabWcpmJpaFVgW8WiyspYVejp6cHT07PdfCZVv9+n9evXD9XV1fjll18glUrxxRdf\noKamRqlYtLUC9HnDyY51OD4+Pjh//nyr9adQKBAdHQ1vb+82jwUA/vjjj3YVi4+Pj9bab+3PRERq\nf79P33LU09ODnp4ejIyMEBoaiuDgYNjY2KBnz57PPP7sSbm/ayyWDqnN1oEy1kYSEhLIxsam1R6j\ndOrUKRKJRA0ugX/8+DEZGxvTo0ePWiWWP//8kwQCQYNP3JfJZOTg4EBXr15tlVjKy8vJ1NSU7t69\nq7U+bt68Sebm5iqf9t5Sf/zxB/Xv37/Ba11aWkomJiZqH8qrrCePxSstLW2V/to7HtmxDsfDwwN2\ndnY4fvx4q/QXGhqKt99+u8G/vE1NTfHaa681+XBmTQoPD0dQUBC6detW771OnTph0aJF+OGHH1ol\nlr1792LEiBGwt7fXWh8uLi5wc3PDwYMHtdbH00JDQ7F48eIGr3WvXr0wa9YsbNmypVViiYiIwMyZ\nM9GrV69W6a/da+tsy1hb+Pnnn6l///5UVVWl1X6ioqLIxsaGysvLGy1z/fr1Vjlu6Pbt22RmZkaZ\nmZmNlikoKCBzc3OSSCRajaW4uJhsbW3p/PnzWu2HiOj06dPUp0+fJq+BJly6dIksLS2bfEhAeno6\nCQQCysrK0mosWVlZJBAI6ObNm1rt53nCyY51SAqFgqZNm0YrV67UWh9lZWXk4OBAv/zyS7NlV61a\nRZMnT9ba4aJyuZxGjRpFmzZtarbsjh07yNXVVWsHyRIRzZ8/n9566y2ttf93c+bMoSVLlmit/crK\nSnJxcaGDBw82W3bdunX08ssva+1aKxQKGj9+PH3xxRdaaf95xcmOdVgPHz4kCwsLOn36tMbblsvl\nNGvWLAoKClKqfHV1Nbm6utL333+v8ViIiD777DMaMWJEk4fIPqFQKMjf35+WLl2qlR/kvXv3kr29\nfavOJRUWFpK1tTUdOXJE420rFAoKDg6m6dOnK1VeKpWSl5cXffXVVxqPhYho48aNJBaL+Wifv+Fk\nxzq0mJgYEgqFdc8Y1AS5XE4LFy6kkSNHUkVFhdL17ty5Q9bW1rRt2zaNxUJEtGnTJnJycqr3XMam\nFBYWkpubG3388ccaTXg///wzmZubU1JSksbaVNa1a9dIKBQqNdJWlkKhoHfffZeGDBmiUvLOzs4m\ne3t7Cg0N1VgsRERhYWFkZ2en9dukzyNOdqzDO3/+PAmFQtq8ebPaP+z5+fn02muv0ejRo1v0gOe0\ntDSytbWl1atXq30bsbKykpYtW0b9+vVr0Y/fo0ePSCQSUUBAAJWUlKgVi0wmo40bN5KlpSXFxcWp\n1ZY6Ll26RObm5vTdd981eoCtsgoLC2nGjBnk4+NDjx8/Vrn+nTt3qE+fPrRy5Uq1V4tWV1fTypUr\nydHRkf7880+12tJVvBqTdXi+vr44d+4ctmzZAj8/vwYPWW0OEeHw4cNwc3ODvb09Tp8+3aIHPPfv\n3x+XL19GfHw8vL29kZiYqHIbAHDp0iV4eHggLy8PFy9ehJ2dncptCIVC/PHHHzAwMICrqyvOnj3b\noljS09MxevRoHDt2DBcvXtTaJnJlDBs2DBcuXMCePXvw0ksv4c6dOyq3QUQ4ceIEXF1dYWJigqio\nKJiamqrcTp8+fRAbG4vbt2/D09MTV69eVbkNALh27RrEYjFu3bqF2NhYODk5tagdndfW2Zax9kIq\nldK6devI2NiY3nzzTYqOjm52pFdWVkabN28md3d3GjBgAF24cEEjsSgUCoqMjCRzc3Py8/Ojo0eP\nNjvfVlNTQ/v27SNfX1964YUXlFosoawzZ86Qo6Mj+fj40Pbt25tdxSqXy+ns2bM0ZcoUMjU11chI\nSpNkMhl9/fXXZGJiQtOmTaOoqKhmr3VFRQVt3bqVxGIxOTs70++//66RWBQKBe3evZssLS3ppZde\nokOHDjU73yaVSunQoUP00ksvkYWFBf30009aW/CiK/SI+HHYjD2tpKQEO3bsQGhoKAoKCiAWiyEW\ni2FlZYUuXbqgqqoKN2/ehEQiwY0bNzB+/HgsXrwYY8eOhb6+Zm+WVFdX48CBA/jhhx+QlpYGT09P\niMVi2Nvbw8DAADU1NcjIyIBEIkFiYiLEYjHefvttTJkyReWjaZojl8tx6tQphIaGIjo6Gq6urhCL\nxXB2doahoSGkUilycnIgkUgQHx8PW1tbvP3225g5c6ZWHvSsCWVlZdi1axdCQ0Px4MGDuu/X2tq6\n7lrfunULEokEqampGDNmDBYvXgw/Pz+NX+uamhosWLAA586dQ3FxMUQiEcRiMRwdHWFgYIDa2lpk\nZmZCIpEgISEBJiYm8PX1RUREBLp27arRWHQRJzvGGkFEyM3Nrfvxzs/Ph1QqhaGhIfr27QsvLy+4\nu7u32g95QUEB4uPjIZFIkJubi5qaGnTt2hW2trbw8vKCp6enVh6o3JCysjIkJCRAIpEgMzMT1dXV\nMDAwgLm5Oby8vCAWi595ev/z4MGDB5BIJJBIJMjPz0dtbS0MDQ3Rp08feHl5wcPDAz179tRqDJ6e\nnvjqq6/g6emJ+Ph4xMXF4f79+3XX2trauu5aJyQk4P3331frCKCOhJMdY4y1A0lJSfD390dmZiY6\nderUbHmFQgEHBwccO3YMHh4erRDh840XqDDGWDuwfft2zJ07V6lEB/x1MGtAQAC2b9+u5ch0A4/s\nGGOsjUmlUtjY2CAmJgbOzs5K1/vzzz8xfPhw5OTkwMDAQIsRPv94ZMcYY23s1KlTcHZ2VinRAUDf\nvn3h4uKCU6dOaSky3cHJjjHG2lhkZCQCAwNbVDcwMBCRkZEajUcX8W1MxhhrQ/n5+XB2dkZWVlaL\nHkRQWloKOzs73L59G0KhUAsR6gYe2THGWBvas2cPXnnllRYlOgAwMjKCv78/du/ereHIdAsnO8YY\na0Pq3MJ8gm9lNo+THWOMtZGkpCQUFBRgzJgxarUzZswYPH78uMXPUu0IONkxxlgbUXVvXWN4z13z\neIEKY4y1gZburWsM77lrGo/sGGOsDbR0b11jeM9d0zjZMcZYG9DEwpS/44UqjePbmIwx1srU3VvX\nGN5z1zge2THGWCtTd29dY57suduzZ49G29UFPLJjjDENysvLqzvstby8HHp6eujduzfc3NwgFoth\nYmJSd27duHHjNN5/VFQUPvjgA8THx6OoqAjx8fFISkpCSUkJiAg9e/bEoEGDIBaLYWlpqfH+2ytO\ndowxpqb79+8jIiICkZGRKC0thVgshpubG4yMjEBEKCwsREJCAhITE2FlZYWCggLcunULAoFA47EU\nFhaib9++EAgEePDgATw8PODh4QEzMzPo6emhtLQU169fh0QigZGREQIDAxESEgJra2uNx9KuEGOM\nsRYpKCiggIAAMjY2pkWLFlFiYiIpFIpGy8tkMoqOjqZ//vOf1Lt3b1q5ciVVVlZqJJaqqir68MMP\nydjYmKZNm0bR0dEkk8kaLa9QKCgpKYkWL15MJiYmFBAQQAUFBRqJpT3iOTvGGGuBo0ePwtXVFb17\n90ZWVhZCQ0Ph7u4OPT29Rut06tQJo0aNwoEDB5CWlobMzEyIRCLExsaqFcvly5chEolw584d3Lhx\nAwcPHsSoUaOa3Kyup6cHNzc3/PDDD7h79y6MjY3h6uqKI0eOqBVLu9XW2ZYxxp4nCoWCPv/8c3Jw\ncKA//vhD7fb2799P5ubm9NNPP7Wo/u7du8nc3Jz279+vdizR0dHk4OBAn3/+eZMj1OdR57ZOtowx\n9jxZt24d9u7di0uXLsHKykrt9l5//XUMGDAAfn5+0NfXx5tvvql03X379uH9999HVFQUBg8erHYs\no0aNwqVLl/Dyyy+DiLB69Wq122wveIEKY4wp6dChQ1i5ciUuXryo8ZWMycnJGDt2LE6fPg1PT89m\nyyckJMDPzw+//fYb3NzcNBpLXl4eRowYgfXr1+P111/XaNtthZMdY4wpIT8/H25ubjh8+DCGDRum\nlT527tyJr7/+GnFxcU0+37K2thZDhgzBe++9h7lz52olltjYWEydOhXJyck6sUGdkx1jjClh1qxZ\nsLKywsaNG7XWBxFh8uTJEIvFWLNmTaPlPvvsM1y7dg3Hjh1rckGMuj744APk5OToxCZ1TnaMMdaM\nzMxMDBkyBFlZWejRo0er9JWdnY3u3bvXe7+qqgq2tra4cuUKnJyctBpLZWUl7OzscPXqVfTp00er\nfWkbbz1gjLFmbN68GQEBAVpPdADg6OiIoUOHYt++fQ2+v2/fPvj4+Gg90QFA9+7dERAQgM2bN2u9\nL23jkR1jjDWBiGBpaamxc+eUceLECaxfvx4xMTH13hs1ahRWrlwJf3//Vonl9u3bGDFiBPLy8qCv\n//yOj57fyBljrBVkZGSga9euSie6O3fuwMzMDAkJCQCA3NxcCIVCREdHK93niy++iISEBEil0mde\nl0qliI+Px5gxY1otFmdnZxgaGiIjI0PpOu0RJzvGGGtCXFwcxGKx0uWdnJywYcMGzJ49G1VVVQgK\nCkJQUBBGjx6tdBs9e/aEvb09UlNTn3n9xo0bsLOzQ8+ePVstFgAQi8WQSCQq1WlvONkxxlgTUlJS\n4O7urlKd4OBg9O3bF97e3nj48CHWrVuncr/u7u5ISUlpN7EkJyerXK894WTHGGNNKCsra9G5c8HB\nwUhNTcXSpUvRpUsXlev37t0bZWVl7SIWY2PjerE8bzjZMcZYE1qyj628vBzLly9HcHAw1qxZg6Ki\nIo303VaxEJFW9/O1Bk52jDHWhF69eqmcIJYtWwZvb29s2bIFkyZNwsKFC1Xut7i4uN7cXFvG0qtX\nL5XrtSec7BhjrAmurq5ISkpSuvzRo0dx9uxZhIWFAQA2bdqE+Ph4lZ9CkpiYWO+Zl+0plucN77Nj\njLEmZGRkYPTo0cjJyWm1PsvKymBpaYmSkhJ07vz/H04jk8nQu3dvPHjwAEZGRq0Wj42NDaKjo5/r\np6jwyI4xxprg6OgIuVyOtLS0Vuvz999/h5eX1zOJDgA6d+6MIUOGICoqqtViuXnzJmQyGRwdHVut\nT23gZMcYY03Q09PD/PnzER4e3mp9hoWFITg4uMH3goODWz2W+fPnP/cLVPg2JmOMNSM7OxsikQh3\n797V+kKNJ4/nys7OhqGhYb33q6urYWdnh5iYGPTr10+rsZSXl8Pe3h7x8fGwt7fXal/axiM7xhhr\nhp2dHfz9/bFq1Sqt9kNEWLp0KZYvX95gogMAQ0NDrFixAu+88w60PVZZtWoVXnnllec+0QE8smOM\nMaUUFhbC1dUVu3fvhq+vr1b62Lp1K0JDQ3H58uUmN3/LZDIMHToUCxcubPR2p7qio6Px5ptvIiUl\nBaamplrpozVxsmOMMSUdP34cS5YswcWLF2FjY6PRtuPj4+Hn54eoqCillvmnpKRgzJgxOH36tErP\n7lRGTk4ORo4cif/7v//D5MmTNdp2W+HbmIwxpiR/f3+88847GDt2LLKzszXWbnx8PCZNmoQtW7Yo\nvZ9t8ODB+PHHHzFp0iTEx8drLJZ79+5h7NixWLJkic4kOgDotHbt2rVtHQRjjD0vhg8fDqlUipCQ\nEAwePFitQ1SJCDt37sTcuXMRHh6OqVOnqlS/f//+6Nu3L6ZPnw5zc3O4u7urtWry119/xauvvool\nS5bgvffea3E77RHfxmSMsRY4e/YsQkJC4Ofnhy+//BJmZmYq1c/KysKSJUuQnZ2NyMhIiESiFseS\nmJiIwMBA2NjY4Pvvv4eDg4NK9R8/foxPPvkEp06dQkREBPz8/FocS3vFtzEZY6wFxo8fj+vXr6Nz\n585wcnJCYGAgYmNjIZfLG61TW1uLM2fOYMqUKRCJRPDy8sK1a9fUSnQA4OHhgatXr8Lb2xtisRhT\npkzBmTNnUFNT02gduVyOy5cvIzAwEE5OTtDX10dycrJOJjqAR3aMMaa2goICbNu2Ddu2bUN2djbc\n3d3h5uYGIyMjKBQKFBUVITExETdu3MCgQYOwYMECzJgxQ+lDWFVRUVGB3bt3IyIiAikpKRgwYABE\nIhFMTEygr6+P0tJSXL9+HUlJSbC1ta070FUoFGo8lvaEkx1jjGlQcXExEhISkJqaivLycujr68PI\nyAhubm5wd3dHjx49Wi2WyspKJCUlISkpCaWlpVAoFOjRowcGDx4MkUgEY2PjVoulrXGyY4wxpvN4\nzo4xxpjO42THGGNM53GyY4wxpvM42THGGNN5nOwYY4zpPE52jDHGdB4nO8YYYzqPkx1jjDGdx8mO\nMcaYzuNkxxhjTOdxsmOMMabzONkxxhjTeZzsGGOM6TxOdowxxnQeJzvGGGM6j5MdY4wxncfJjjHG\nmM7jZMcYY0zncbJjjDGm8zjZMcYY03mc7BhjjOk8TnaMMcZ0Hic7xhhjOo+THWOMMZ3HyY4xxpjO\n42THGGNM53GyY4wxpvM42THGGNN5nOwYY4zpPE52jDHGdB4nO8YYYzqPkx1jjDGdx8mOMcaYzuNk\nxxhjTOdxsmOMMabzONkxxhjTeZzsGGOM6TxOdowxxnQeJzvGGGM6j5MdY4wxncfJjjHGmM7jZMcY\nY0zncbJjjDGm8zjZMcYY03mc7BhjjOk8TnaMMcZ0Hic7xhhjOo+THWOMMZ3HyY4xxpjO42THGGNM\n53GyY4wxpvM42THGGNN5nOwYY4zpPE52jDHGdB4nO8YYYzqPkx1jjDGdx8mOMcaYzuNkxxhjTOdx\nsmOMMabzONkxxhjTeZzsGGOM6TxOdowxxnQeJzvGGGM6j5MdY4wxncfJjjHGmM7jZMcYY0zncbJj\njDGm8zjZMcYY03mc7BhjjOk8TnaMMcZ0Hic7xhhjOo+THWOMMZ3HyY4xxpjO42THGGNM53GyY4wx\npvM42THGGNN5nOwYY4zpPE52jDHGdB4nO8YYYzrv/wFq3k3+iPJu4gAAAABJRU5ErkJggg==\n", 552 | "text": [ 553 | "" 554 | ] 555 | } 556 | ], 557 | "prompt_number": 16 558 | }, 559 | { 560 | "cell_type": "markdown", 561 | "metadata": {}, 562 | "source": [ 563 | "The primitives are not limited to standard library operators, any function or instance method can be added to a primitive set. Terminals can be any type of objects and even functions without argument. The next example takes advantage of this flexibility and reduces the runtime of the previous example by vectorizing the evaluation using [Numpy](#numpy), a library of high-level mathematical functions operating on multi-dimensional arrays." 564 | ] 565 | }, 566 | { 567 | "cell_type": "code", 568 | "collapsed": false, 569 | "input": [ 570 | "import numpy, random\n", 571 | "from deap import algorithms, base, creator, tools, gp\n", 572 | "\n", 573 | "creator.create(\"FitnessMin\", base.Fitness, weights=(-1.0,))\n", 574 | "creator.create(\"Tree\", gp.PrimitiveTree, fitness=creator.FitnessMin)\n", 575 | "\n", 576 | "pset = gp.PrimitiveSet(name=\"MAIN\", arity=1)\n", 577 | "pset.addPrimitive(numpy.add, arity=2)\n", 578 | "pset.addPrimitive(numpy.subtract, arity=2)\n", 579 | "pset.addPrimitive(numpy.multiply, arity=2)\n", 580 | "pset.addPrimitive(numpy.negative, arity=1)\n", 581 | "\n", 582 | "def evaluateRegression(individual, points, pset):\n", 583 | " func = gp.compile(expr=individual, pset=pset)\n", 584 | " sqerrors = (func(points)-(points**4 + points**3 + points**2 + points))**2\n", 585 | " return (numpy.sqrt(numpy.sum(sqerrors) / len(points)),)\n", 586 | "\n", 587 | "toolbox = base.Toolbox()\n", 588 | "toolbox.register(\"expr\", gp.genFull, pset=pset, min_=1, max_=3)\n", 589 | "toolbox.register(\"individual\", tools.initIterate, creator.Tree, toolbox.expr)\n", 590 | "toolbox.register(\"population\", tools.initRepeat, list, toolbox.individual)\n", 591 | "toolbox.register(\"evaluate\", evaluateRegression, points=numpy.linspace(-1, 1, 1000), pset=pset)\n", 592 | "toolbox.register(\"mate\", gp.cxOnePoint)\n", 593 | "toolbox.register(\"expr_mut\", gp.genFull, min_=0, max_=2)\n", 594 | "toolbox.register(\"mutate\", gp.mutUniform, expr=toolbox.expr_mut, pset=pset)\n", 595 | "toolbox.register(\"select\", tools.selTournament, tournsize=3)\n", 596 | "\n", 597 | "if __name__ == \"__main__\":\n", 598 | " pop = toolbox.population(n=300)\n", 599 | " algorithms.eaSimple(pop, toolbox, cxpb=0.5, mutpb=0.2, ngen=40, verbose=False)\n", 600 | " print(tools.selBest(pop, k=1)[0])" 601 | ], 602 | "language": "python", 603 | "metadata": {}, 604 | "outputs": [ 605 | { 606 | "output_type": "stream", 607 | "stream": "stdout", 608 | "text": [ 609 | "add(ARG0, multiply(ARG0, add(multiply(ARG0, ARG0), add(negative(multiply(multiply(ARG0, ARG0), negative(ARG0))), ARG0))))\n" 610 | ] 611 | } 612 | ], 613 | "prompt_number": 17 614 | }, 615 | { 616 | "cell_type": "markdown", 617 | "metadata": {}, 618 | "source": [ 619 | "The idea is to evolve a program whose argument is a vector instead of a scalar. Most of the code remains identical; only minor modifications are required. First, we replace the operators in the primitive set by Numpy operators that work on vectors. Then, we remove the loop from the evaluation function, since it is implicit in the operators. Finally, we replace the `sum` and `sqrt` functions by their faster Numpy equivalent and our regression problem is now vectorized. The execution is thereby significantly improved as the scalar example runs in around 3 seconds to optimize the regression on 20 points, while the vectorial runtime is identical but for a regression on 1000 points. By modifying only 6 lines of code, not only are we able to vectorize our problem, but the runtime is reduced by a **factor 50**!\n", 620 | "\n", 621 | "In addition to the wide support of function and object types, DEAP's *gp* module also supports automatically defined functions (ADF), strongly typed genetic programming (STGP), and object-oriented genetic programming (OOGP), for which examples are provided in the library documentation." 622 | ] 623 | }, 624 | { 625 | "cell_type": "markdown", 626 | "metadata": {}, 627 | "source": [ 628 | "##Distributed Island Model\n", 629 | "*This section is skipped because SCOOP cannot run inside a notebook*" 630 | ] 631 | }, 632 | { 633 | "cell_type": "markdown", 634 | "metadata": {}, 635 | "source": [ 636 | "##Evolution Support\n", 637 | "DEAP comes with several supporting tools that can be easily integrated into any algorithm. This section presents some of them in the context of the OneMax example.\n", 638 | "\n", 639 | "The first tool, *Statistics*, computes statistics on arbitrary attributes of designated objects, usually the fitness individuals. The attribute is specified by a key function at the statistics object instantiation before starting the algorithm." 640 | ] 641 | }, 642 | { 643 | "cell_type": "code", 644 | "collapsed": false, 645 | "input": [ 646 | "stats = tools.Statistics(key=operator.attrgetter(\"fitness.values\"))" 647 | ], 648 | "language": "python", 649 | "metadata": {}, 650 | "outputs": [], 651 | "prompt_number": 18 652 | }, 653 | { 654 | "cell_type": "markdown", 655 | "metadata": {}, 656 | "source": [ 657 | "This is followed by the registration of the statistical functions as for a toolbox." 658 | ] 659 | }, 660 | { 661 | "cell_type": "code", 662 | "collapsed": false, 663 | "input": [ 664 | "stats.register(\"max\", numpy.max)\n", 665 | "stats.register(\"mean\", numpy.mean)\n", 666 | "stats.register(\"min\", numpy.min)" 667 | ], 668 | "language": "python", 669 | "metadata": {}, 670 | "outputs": [], 671 | "prompt_number": 19 672 | }, 673 | { 674 | "cell_type": "markdown", 675 | "metadata": {}, 676 | "source": [ 677 | "Ultimately, at every generation, a statistical record of the population is compiled using the registered functions." 678 | ] 679 | }, 680 | { 681 | "cell_type": "code", 682 | "collapsed": false, 683 | "input": [ 684 | "record = stats.compile(pop)\n", 685 | "print(record)" 686 | ], 687 | "language": "python", 688 | "metadata": {}, 689 | "outputs": [ 690 | { 691 | "output_type": "stream", 692 | "stream": "stdout", 693 | "text": [ 694 | "{'max': 2.7738735050143437, 'mean': 0.31086609078498684, 'min': 1.1588576056708691e-16}\n" 695 | ] 696 | } 697 | ], 698 | "prompt_number": 20 699 | }, 700 | { 701 | "cell_type": "markdown", 702 | "metadata": {}, 703 | "source": [ 704 | "The statistics compilation produced a dictionary containing the statistical keywords and their respective value. These last lines, added after the evaluation part, will produce a screen log of the evolution statistics.\n", 705 | "\n", 706 | "For posterity and better readability, statistics can also be logged in a \\textit{Logbook}, which is simply a list of recorded dictionaries that can be printed with an elegant layout. For example, the following lines create a new logbook, then record the previously computed statistics and print it to screen." 707 | ] 708 | }, 709 | { 710 | "cell_type": "code", 711 | "collapsed": false, 712 | "input": [ 713 | "logbook = tools.Logbook()\n", 714 | "logbook.record(gen=0, nevals=300, fitness=record)\n", 715 | "print(logbook)" 716 | ], 717 | "language": "python", 718 | "metadata": {}, 719 | "outputs": [ 720 | { 721 | "output_type": "stream", 722 | "stream": "stdout", 723 | "text": [ 724 | " \t \t fitness \n", 725 | " \t \t-----------------------------------\n", 726 | "gen\tnevals\tmax \tmean \tmin \n", 727 | "0 \t300 \t2.77387\t0.310866\t1.15886e-16\n" 728 | ] 729 | } 730 | ], 731 | "prompt_number": 21 732 | }, 733 | { 734 | "cell_type": "markdown", 735 | "metadata": {}, 736 | "source": [ 737 | "The next tool, named *Hall of Fame*, preserves the best individuals that appeared during an evolution. At every generation, it scans the population and saves the individuals in a separate archive that does not interact with the population. If the best solution disappears during the evolution, it will still be available in the hall of fame. The hall of fame can be provided as an argument to the algorithms as follows:" 738 | ] 739 | }, 740 | { 741 | "cell_type": "code", 742 | "collapsed": false, 743 | "input": [ 744 | "halloffame = tools.HallOfFame(maxsize=10)\n", 745 | "pop = toolbox.population(n=300)\n", 746 | "pop, logbook = algorithms.eaSimple(pop, toolbox, cxpb=0.5, mutpb=0.2,\n", 747 | " ngen=40, halloffame=halloffame, verbose=False)" 748 | ], 749 | "language": "python", 750 | "metadata": {}, 751 | "outputs": [], 752 | "prompt_number": 22 753 | }, 754 | { 755 | "cell_type": "markdown", 756 | "metadata": {}, 757 | "source": [ 758 | "Moreover, the hall of fame can be updated manually right after the population is evaluated with the following line of code." 759 | ] 760 | }, 761 | { 762 | "cell_type": "code", 763 | "collapsed": false, 764 | "input": [ 765 | "halloffame.update(pop)" 766 | ], 767 | "language": "python", 768 | "metadata": {}, 769 | "outputs": [], 770 | "prompt_number": 23 771 | }, 772 | { 773 | "cell_type": "markdown", 774 | "metadata": {}, 775 | "source": [ 776 | "The hall of fame proposes a list interface where the individuals are sorted in descending order of fitness. Thus, the fittest solution can be retrieved by accessing the list's first element." 777 | ] 778 | }, 779 | { 780 | "cell_type": "code", 781 | "collapsed": false, 782 | "input": [ 783 | "best = halloffame[0]" 784 | ], 785 | "language": "python", 786 | "metadata": {}, 787 | "outputs": [], 788 | "prompt_number": 24 789 | }, 790 | { 791 | "cell_type": "markdown", 792 | "metadata": {}, 793 | "source": [ 794 | "A Pareto dominance version of the hall of fame is also available. The *Pareto Front* maintains an archive of non-dominated individuals along the evolution. Its interface is the same than the standard hall of fame.\n", 795 | "\n", 796 | "Another tool, called the *History*, tracks the genealogy of the individuals in a population. By wrapping the variation operators, the history saves the parents of each individual. This feature is added to the variation operators of the toolbox with the following lines." 797 | ] 798 | }, 799 | { 800 | "cell_type": "code", 801 | "collapsed": false, 802 | "input": [ 803 | "history = tools.History()\n", 804 | "toolbox.decorate(\"mate\", history.decorator)\n", 805 | "toolbox.decorate(\"mutate\", history.decorator)" 806 | ], 807 | "language": "python", 808 | "metadata": {}, 809 | "outputs": [], 810 | "prompt_number": 25 811 | }, 812 | { 813 | "cell_type": "markdown", 814 | "metadata": {}, 815 | "source": [ 816 | "It is therefore possible to determine the genesis of individuals. The next code presents the genealogy of the best individual in the OneMax example for the last 5 generations of the evolution. The graph is produced by the NetworkX library and the following listing." 817 | ] 818 | }, 819 | { 820 | "cell_type": "code", 821 | "collapsed": false, 822 | "input": [ 823 | "halloffame = tools.HallOfFame(maxsize=1)\n", 824 | "pop = toolbox.population(n=300)\n", 825 | "pop, logbook = algorithms.eaSimple(pop, toolbox, cxpb=0.5, mutpb=0.2,\n", 826 | " ngen=40, halloffame=halloffame, verbose=False)\n", 827 | "\n", 828 | "h = history.getGenealogy(halloffame[0], max_depth=5)\n", 829 | "graph = networkx.DiGraph(h)\n", 830 | "graph = graph.reverse() # Make the grah top-down\n", 831 | "colors = [toolbox.evaluate(history.genealogy_history[i])[0] for i in graph]\n", 832 | "pos = networkx.graphviz_layout(graph, prog=\"dot\")\n", 833 | "networkx.draw(graph, pos, node_color=colors)\n", 834 | "cb = plt.colorbar()\n", 835 | "cb.set_label(\"Error\")\n", 836 | "plt.show()" 837 | ], 838 | "language": "python", 839 | "metadata": {}, 840 | "outputs": [ 841 | { 842 | "metadata": {}, 843 | "output_type": "display_data", 844 | "png": "iVBORw0KGgoAAAANSUhEUgAAAc8AAAE+CAYAAADrtpuXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xl4TNcbwPHvnSXJzGQTgkjsFCG2UnsTW0UstTeqSlFK\nqVar2qKlVdpfN1W1tbXXvkXtte9LK9ROKUWJWLJOkpnMnN8fEyEImWQiRs7nee4TM3PPnffeuead\ns9xzFSGEQJIkSZKkLFPldQCSJEmS5Gxk8pQkSZIkO8nkKUmSJEl2kslTkiRJkuwkk6ckSZIk2Ukm\nT0mSJEmyk0yekiRJkmQnmTwlSZIkyU4yeUqSJEmSnWTylCRJknLMx8cHRVHsXnx8fPI69GxR5PR8\nkiRJUk4pisKobJQbBThjGtLkdQCSJEnS0yE/JRTZbCtJkiRJdspPPxQkSZKkXKTN6wAeI5k8JUmS\nJIfITwklP+2rJEmSlItkzVOSJEmS7JSfEkp+2ldJkiQpF8mapyRJkiTZKT8llPy0r5IkSVIukjVP\nSZIkSbJTfkoocpIESZIkSbJTfvqhIEmSJOUi2WwrSZIkSXaSyVOSJEmS7JSfEkp+2ldJkiQpF8ma\npyRJkiTZKT8llPy0r5IkSVIukjVPSZIkSbJTfkoo+WlfJUmSpFyUn2qecpIESZIkSbKTrHlKkiRJ\nDpGfEkp+2ldJkiQpF+WnZluZPCVJkiSHyE8JJT/tqyRJkpSLZM1TkiRJkuwkk6ckSZIk2Sk/JZT8\ntK+SJElSLtJmJ6OkOjyMx0ImT0mSJMkhNPkoecpJEiRJkiTJTrLmKUmSJDmEVp3XETw+MnlKkiRJ\nDpGtZlsnJZttJUmSJIfQauxfMrNu3ToqVqxI+fLl+fLLL+97/datW7Rv355q1apRp04djh07luWy\njqAIIUSubFmSJEnKNxRFQRTORrlrcG8aslgsVKhQgY0bN+Lv70/t2rWZP38+lSpVSl9n6NCheHp6\nMnLkSE6dOsWbb77Jxo0bs1TWEWTNU5IkSXIMTTaWB9i/fz/lypWjVKlSaLVawsPDiYiIyLDOiRMn\naNy4MQAVKlTg/PnzXLt2LUtlHbWrkiRJkpRzWcgoW5Nha8rD17l8+TLFixdPfxwQEMC+ffsyrFOt\nWjWWLVtGw4YN2b9/PxcuXODSpUtZKusIMnlKkiRJjpGFjBLibltuGx17/zqKojxyOx988AGDBw+m\nRo0aBAUFUaNGDdRqdZbKOoJMnpIkSZJjOOhSFX9/fy5evJj++OLFiwQEBGRYx8PDg+nTp6c/Ll26\nNGXLliUpKemRZR1B9nlKkiRJT5RatWpx5swZzp8/j8lkYuHChbRt2zbDOrGxsZhMJgB++ukngoOD\ncXd3z1JZR5A1T0mSJMkxHJRRNBoNEydOpEWLFlgsFnr37k2lSpWYOnUqAP369eP48eP07NkTRVGo\nUqUKv/zyy0PLOpq8VEWSJEnKMUVREEHZKHfk/ktVnIGseUqSJEmOIafnkyRJkiQ75aOMko92VZIk\nScpV+Sij5KNdlSRJknKVbLaVJEmSJDvlo4ySj3ZVkiRJylX5KKPISRIkSZIkyU756HeCJEmSlKvy\nUUbJR7uad44ePcoPU6Zx8Ohx4uLicHd3p2LZMgzs25u6des+lomMLRYL69atY8rUuVy8+B/JKcl4\ne3nTOOQ5Bgzom+EuBLlJCMGOHTv4cdIMzpw5T6IxES9PL2rXDmLQwL5UrFjxscQBcOjQISb8MI2/\njpwmPj4OD3cPAgPL8dag16lVq9Zji+Pvv//mh8lT2RN5mNiYGPQGA2VKFOfN3j1p3LjxY5vo+r//\n/mPy5J/YuGkPMbExuGhdCAjwo+/rL9OqVSs0msfzdXHr1i2mT5/J8oiN3LxxE7VaTZEivvR4tQOd\nO3fGzc3tscSRmJjI/Pnz+XX5b0Rfv44QgoIFC/JS65Z07/4Knp6ejyUOp5KPBgzJGYZy0erVqxk5\n7n+cPH0GU8vXsVRpCDoPSE5E+TsS/dqpFPVyZ8SQt+nR49Vc+ZI0m8188814vv1uEslmX+LpA5pn\nQHED601crWsh6VcaNQpm7JiPqF27tsNjAFvS/Omnnxk77nuu30rFqOqH0FQDRQ/WWDTW7WiSfyYo\nqAqff/YBzZs3z5U4AJYtW8Yno77m3D8XSdH2xaKpC4oHiARUlj9wM0+hREBhPvl4COHh4bkWx9at\nWxn++RccPHgQS4vXMFdtDO7ekGKE88dwXzsVb5WFDwYPon//N1CpcqeX5dChQ3w0/HM2b94I+nBS\nVK1B5QPCBKl/46H8govqEoPfeoP3338XV1fXXInj3LlzjBj5OcuXL0Olb4WRTqAuDFgg9SLuyiww\nR9Kn92t88vGHeHt750oc165dY9SYscyaMwelSgMSQ7pBIX9QFLh5FcP2BVj/3MhL4eGM+XgE/v7+\nuRKHs1EUBZGNKWSVlc45w5BMnrlACMHoMWP5aspPGHv/Dxq2B432/hWtVji4Ef3PQ+n4fB2mT5nk\n0F/3cXFxhLXqxMGjCkkuY8A1k8RoTQDjHPSmT/jl5x8ID3/JYTGALYG/3K03azacwOjyJbg2tn0R\n3UukgHEpOtP7fPrJu7z33jsOjUMIwdD3hzN56mKMLl+BrjUoDzjewgLJ69Cb3qNn95b8MOFrhyeu\niZMm8/4nn5LU+0sI6QIuD6hNCQFHdqCf/gFNygew+NfZDq91RURE8HK3PhhdRoK+J6gyqU2ZItGZ\nPiGwXCy/r19BgQIFHBrH3r17CW3ZnnilH1bdgLSk+QDmM7ikjKOY9z62bV1LiRIlHBrH6dOnCW7R\nkhs1wjB3eheKlnrwitcvo14+Aa/t89i8ZhXVqlVzaBzOSFEURIdslFsmk6eU5suvv+HTKTMwjt0A\nhYo9uoAxHv3odrz0bAV+mfyjQ2qgJpOJ4JAwIk+WIsUw5cFJ4r5Cf6FPCGXhgmm0bt06xzGA7T9F\neHhPfvs9miT9ElDpH10o9V/0Cc34cuzbDBw4wCFxAAwfMYrxE1dhNKwDdaFHF7DeQp/Yir49G/Hd\nd186LI7pM2YyaMRojF9ugmJlHl3AlIxu3Ms089WyYtF8hyXyjRs30rZdN5LcV4NLFpqphRUX42AC\nS0ayZ/dGhyXyI0eOUL9BUxJcZ4CuVZbKqI1fU1Q3jUORuylUKAufZRZcvnyZ6nXqcSN8JCLs9awV\n2rIQr5/e4c9dOyhbtqxD4nBWiqIgOmej3GKZPCXgwIEDhLRuh3HCPvC14x5yiXEYhjRg+ucj6dKl\nS47j+Gj4J4yffJAkwwpQ7OiISNmPwRjGP+dO4Ovrm+M4ZsyYyaB3JpLovj1rifO21HPo4uqxf+8m\nqlSpkuM4tm7dSqu2PTF67AN1kawXtNzEkFCHRfO/JywsLMdx/P3331R9rh5J3+6AEnb075qS0Q9r\nyv/6duPNN3P+gyI2NpaA4uVI0C0D10ZZLyis6BLD6dO9JBO+/yrHcVgsFkqUrMh/yZ+CvqtdZbWJ\nQ2la5x/WrlmS4zgA6oQ05WC5pqS+/JFd5VQrJ/HMtukcP3jgsfVPP4kURUHY9xHays13zuSJkLKt\nW7duomjRosLDw0OULl1ajBkzRnTs1l0ofb4QPN9RULSUQFEE324VbBJ3lm82C6qFCAxetnVuPz9q\nqahat4EICQkRvr6+wsPDQ1SsWFFMmzbNrhiSk5OFh2dhgesLAnUpAYrAd6uguLizeH4iQCNQ3NMW\nD4HfP4LiQugK9hBvv/2OCA4OFl5eXiIgIEB89tlnmcaQkpIievXqJUqWLCk8PDxE9erVxdq1a4XV\nahVly1cXeI4TaCoIFL3AtbHA78KdONxC74rBXYCLQBskKC6E2meU6PnaG2L8+PGidOnSwmAwiEqV\nKonTp0/bHcsLLdoL9K9lfC9Fbzs2RQ7aYvHdLHANEShetuN2O0afWaJBo1C7PpfMPpuBbw8Rmpfe\nffj5UTtUoHO/s2hdBKWDBN9tF/7lKohz586JkJAQodfrRcWKFcXGjRvtPh4TJvwgdD6dBLqOmZ8j\nAWaB+0CBqqhA5SNwayModlng94/QG3xEo0aNcnw8Vq1aJfQelQSuzWzvofIV6DoLil25E4fX/wTa\nKrZzVF1a4PWV7Xn/eOGqKyDq1auX4ziOHDkidL5+gi/WCYpXELjpBdUbC+ZduPO5RNwSvPCqwLuw\nbekxyvb87xah8y8l9Hq9cHd3T18URRHffvvtQ2N5mgBCdLd/cdY05JxRPyGOHj0qkpKShBBCnDx5\nUhQuXFhodXrBoiuCN8cLvt8pKOgn+G5bxi/HH/cLPpwrGDItY/LcYBb6Iv5i8eLFwmQyCSGE2Ldv\nn3B1dRUnT57MUgxFihQRw4YNE+4+jQXe4wWFdwpUfgLfbfckz1ECffeMz91eCu8TGo2LGD58uLBa\nreLs2bPCz89PrFy58oExJCYmilGjRokLFy4IIYRYtWqV8PDwEMuWLRN691ICxVNQcIkgIEXgMVTg\nUvfB71tc2JKX52e2fxe7LLRanahSpYo4ceKEEEKIc+fOiZs3b2b6mTwoFnd3d+Hq6inwj8v4Xj4z\nBZpyd+33foHPXEGBaRmTZ0CScNMVEqtXr87y55LZ+eHq7iGYcfLh58e9S7UQwWufCTZahXu5KiIw\nMFC8++67Ijk5WSxdulR4e3uL6Ohouz4bv2LlBIU2Cry/z/wc8f5eoK0mKHZNEJAs0L8q0HUQFBfC\nULC1GDVqVI6OR5EiRUSVoNoC9yG288M/XuBvFBh62X5U3Z08i0QKAiyCoqcE6pKCggsExYVw8Rkk\nXuv1eo7jaNGqtVB1eU9g8BR8skSwLkXw0lBBYN07n0OLnoKQLoK1SYJ55wXFygrenyHYJIQy4FvR\nPrxb+nv8888/Qq1Wpx/3/AAQoqf9i7MmTzlJQg5Urlw5Q7+PyWRCVS0EChaFDoOhSgNQPaDJtGJt\naNYNipbO+LxaQ0qL3mzesQut9s4AI3d390yHxd8bg0ajYc2abSQo/cFjMLg2yKTZVqQtD+BSm9RU\nM+XLl0dRFMqUKUPDhg05fvz4A1fX6/V88skn6YM3WrVqRenSpfn++8kYrTVBGwT6jqC4gOcoMB8G\n8+n7N5R6HlJ2gOFV22NVUVIt0LJly/RLWEqXLv3QwSoPisXd3R2Lth6oPDKunDgT9K/eeexaGwzd\nQHPP56K4YXF7lZ0792b5c4H7P5vU1FRUpatAiQoPPz/udvU8HNkBzV8FRSGhbntOnjzJ6NGjcXV1\npUOHDlStWpWlS5dm+XgUKVKEWzFGcGsCHm9lfo6Yj4FbC1D7guIK+i6254BE5Q2WLv89R8dDpVJx\n6vRJ8BpjOz9U7qDSgfubkLLrTkHPoeBSHRQVaJ8B3Yvpr5tc+rFixeocxaHVatmyZQtWr0JQOgie\n7whaF3h1FJw9DBfTztW9q6DLUNvgriIloWVvWDsdAPFCT9asWklCQgIAs2bNIjg42OEDmqQnh0ye\nOTRgwAAMBgOVK1fm2Vq1SaliR//RA1hKVubvCxdp3bo1Op2OkJAQpk+fjp+fX5ZiGDFiBDdu3gJt\n5Ue8kwJJv8HlgnClCiRMueslBbVLMZYsWUJqaionT55kz549NGvWLEv7EBUVxenTp4mNTwZrKrjc\nNRJRpQdNOTAfvb9g4mxwfR40aV84lksIaxJHjhylRIkSlClThlGjRtnVPxIVFUV0dDSpynMZX0i9\nkDFRP4KZypz5+1+7PhfI+NkEBwdjDqyf5dgB2DAbqj4PRdKOiUaD1tUNg8GQvkq1atU4duxYljYX\nFRXFhQsXULtVfvCI57u5vQDJa8FyBaxGMP4Kbmn9vtrK/HfZvvMUMh6PHj16oDeUsyXMu6VsB20m\n/dxCZHxdU4mYW1dp1apVtuN4++23UTRauP4flLnrXHXTQ7FycOGuY3v3uSescD7tPPYogNarIFFR\nUQghmD17Nj169HhoDE8lTTYWJyWTZw5NmjSJhIQENm7cyM6dOyAmOmcbdDOQaDSyatUqEhISmD17\nNj179uTff//NUgwjRowgIf6W7frJh9F3Ab+TUOw6+PwEsZ+CcUH6yxpDI/bu3YtOpyMwMJA+ffrw\n7LPPPjJ8s9lMt27d6NmzJ8KqAKmg3FMLUDxBJNxf2DgbDD3vPLZcAhROnz7N0aNH2bJlC/Pnz+eX\nX355ZBx3x1KyZFnQlMr4YnqiLpmlbaEyEJ9g3+cCGT+bdevWkRp3K2vvd9vvs6FFzzuPrdb7BqV4\nenoSHx//yE3dPh4NGzZEycqgKX1H0NaA//zhsheYT4HXSNtrip7k5MQcHY/JkyeTarFkXMH0F8R9\nBt6ZDEaKG2X7a3gtLQ4Vao0bCxcuzHYcn3/+Oag1kJwIhnvOVYMnGNOObe1QWPglJCXA5b9ttc6U\npPRVVToDRqORnTt3cu3aNTp16vTQGJ5KMnlK9lAUhZCQEAIrBcLxPTnbWGIsBby9AFCr1XTq1Ik6\ndeqwfPnyLMXQuXNnhLCANe7h76OtBOqittqHaz1bE68xbdSi1YgpdiVdunQhJSWFixcvsm7dOiZP\nnvzQTVqtVrp3746bmxsTJ07Ey9sLUIO4JxYRa5uU4G4pO8ESBbq7vnAUHSBo2rQJnp6elCxZkn79\n+rFmzZqH79s9sTRt+sL9MRhng8GOmoE1loI+9n8ucOezqVGjBqozB7P+nkd2wq0oeP7hX8IxMTGP\nnO3m7uPx5ptvolIecX4AxLwHIh78b0JAIujaQ3TLtA3GYnC3TVKQ3ePxwgsvYE6+fOcF899wPQy8\nJ9iaku8VPxES54LvalDSmmmFGaslBYPBkO042rdvjzk+FnTukHjPcUmMBX3auTpwAmjd4NXy8El7\naPqybfKENJaEWLy8vJg1axadOnVCr7djdPnTQiZPKTsKFPDG5dZ/OdqG7q8t1K1eNcNzZrM5QzPd\nw5jNZooV80Mxb8l+EObDCGsSAwYMQKVS4e/vz0svvfTQpCWEoHfv3kRHR7N06VLUajX161ZDozaC\n6fCdFa2JkHr2/mblxFmg65jxchZNBUC57/q5R10OcG8sdevWwF191/FI2WVritRlvWagV22hbp2M\nF8Lb87kAeHt7o75xOWPT38NsmAWNOtqaD9Nobl3FnJKS3rcGcPjwYSpXzryZ/t7jUaNGDUwJe0Ak\nP/z9k9bZangqb1t/tcdAMO0Hy00wbaFa1eyfpwA6nQ6rJdH2WaRegOjm4Pmxrd/5XgnTIf5/UHgT\nqO+6djplK6XLBmU4J+yNQ1EU3PR6cDPAubvO1aRE+O8slEw7th4F4KO5sPgK/HwErBaoVMf22sXT\nqFJNeHt7s2TJkvzZZAu26fnsXZyUTJ7ZFB0dzYIFC0hMTMRisbB+/Xr++OMPtEnxcOEEmFLAlPbl\nZL7r32D78jQlg8Wc9u8UMJsgIQbLloWUKVWSpKQkzGYzc+fO5Y8//uCFF17IUgyLFy/mvfcGo0+d\nBNbku74gUzJ+WSZFgPVWWh/SfkiYYBuIAWA+jUql8Ndff2G1Wrl69SoLFy586Cwq/fv35+TJk6xc\nuTJ9+rY33uiD2vKnrX/TuMz2/nGjQVvdNvDjNmsSGBdnbLIFSD2Bq6ueHTt2kJCQwKVLl/jpp58e\nOYHDvbG89NJLWJN32r6gIS1RdwLVPV+wQthiFGZA2GY8EiawXCU1YTV+fkWz9Llk9tns3LkTb4MO\nju56+PkBtubAbYszNtmaUtDuXk5QUBVGjx5NcnIyy5Yt4+jRo3Ts2DHLx6NMmTLUrFnT1tIgUjI/\nR1yq2o6VNc52TBImgdofVAXQm78h+Plnc3Q8VqxYQWhoGCrjt3CtCbgPBPe+9xdO/BVih4Pvhvua\n3/XmL2nZolGO4li8eDE9ur+C27V/4J+jsGOZ7fOYMxrKVofiaefqf+cg9gZYLLB/Laz+CbqNsB2q\n1VPo16c3q1atwsfHh5CQkEw/j6daPqp5OucY4SdAdHS0CA4OFt7e3sLLy0vUrl1bREREiGEfDRcu\nHQYJipS0XcOnUt35e/uasW+22J67+/XqjYUy4DvRJLSVqFOnjvDw8BA+Pj4iODhY7Ny5064YrFar\nKF22qkBVxHb9Hqo7f29fY6nvKlAVtF3vqKko8P4h/dIAj4KNxbBhw0SNGjWEp6enKFq0qOjbt2/6\n8P57nT9/XiiKInQ6XYbr3ObNmyeaNGsr8HjH9h6K7v7rPIsLgc+8jJeGpC1uBXuJESM+EeHh4cLD\nw0MUL178odebPiyWZs1ChcbnQ0FAkkDxtl3Tee9lMr5b0o7TXcfMtbFQ+3wmXnyxS5Y/l4d9Nt9+\nN17om4U//PzYJATD52W8jGmTEHw4V9Rt0lycP39ehISECJ1OJypWrCg2bdpk9/F49913hYdPXdtl\nH5mdI8Wu2q65VBWyHTPXRoIiBwSFdwrfwiUdcjwOHz4stC4etvfOcA2ux53PRV3adv3v3a+79xf4\nXRBuOtu2chrHtWvXhKuHl2DUMkGJigJX3f3XeX68SFCwmO0a0HI1BP/bYHt+daJw8y4ozp8/L1q0\naCE+/vjjh56jTytAiGH2L86ahuQMQw52+fJlKlWvSfzQOVDrwb9+H+jcEfTDmrJ9w9osDcx5lIUL\nF9Hr9Y8weuzOfJ7QB1AZJ+JvmMjfZ/7CxcUlx3Hs3r2bZi90IMljG2grZL1g0kq8LW9w+tRhh8x0\ndPbsWapVr0uifjm4Nsx6wZQD6BPD+OPAdipVqpTjOGJiYihXOYgbvb+xzWmbVVf+QfdOA1YtmEuT\nJk1yHIfFYqFi4LP8c6MbFv1QOwreRJ/QgAnfvk/v3q/lOA6Aps3asisygBT9j48eAXybSEGf8AKD\n+gXzxRefOiSOvgPfYm7kPySNWm4bQJQVViu6L18hrIiGJXNnOyQOZ6UoCsK+yZls5cY65wxDstnW\nwfz9/Vm9bAn6L7vBgfVZK3QmEoYEM7DPaw5JnAAvvdSFt958GX1CM0i9/OgCgMo4BS8xjq1b1jgk\ncQLUr1+fCePHok94If0awUdKisCQ3Jv161Y4JHEClC1blqVL5qKNaQvJ27NWKGUPusQ2zPv1Z4ck\nTrD1e25aswr3SQNh66KsFbp4Gv0HzRg38iOHJE6wDfLZsmkVBVQ/oDZ+l7VClij0iS147dUwhyVO\ngOXL5lK80G5ckt61Xf7xKNYE9IntadywMGPHjnJYHN9+MRa3c5GoP+18fzP6g6SacRv/Os/EX2DO\nT1MdFodTy0fNtjJ55oJGjRqxYeUKPL/pgdu3veH0nw9e8eJpXKYMQf/RC4wZ9h4zZsxg06ZNDotj\n7NjRDB/2Crq4WqgTPwPL1ftXElZI3ojB2A4/3bcc2L+dMmWyMFm5Hfr06cXUyWPRx4egTXwfUs89\nIA4BKfvQG3vgbenP1i1ree655+5fLweSkpLw9FBhSOqIm/EN22URD2I+jqtxEAZjW5Ysms6LL77o\n0DiqVavGzk2/U/CXd9GPe9nWB/qgX95RF1CmvY/unQaM/+QjBg8a6NA4AgIC+POPnRT3+gXX2Bcg\nae2Dk5flGqrEL9DH1WTIoDb8MOFrh8bh6enJ/r1bqFr2IO6JjcC4yNbXfC9rHErijxjia9K+VTGW\nL5vnsEnyU1JSCA8PJ6Tec4QW1uI+qDas+dk2aOhepmT4fQ7ug+tSN/UqOzeuR6fT3b+e9FSTzba5\nKDo6mmk//8L4yVNI8SpCcqUGmHWeaFIS0J07BP8coV+f3gx8ox8lS5Zk+/btdOzYkSVLlhAcHOyw\nOA4fPsx3301m4aKFaAxNMJrLYhVuuGhicLGsx7eQG8OGvskrr3Sza5Sivc6ePcuEH6YwffpMFNdn\nSbIEkWo1gCUaD+1e9K4xDHmnP717v0bBggUd+t4HDhwgLCyMtWvX4u/vz5QpP/PDxKmkUopkUQez\nxQOtOh431Z+oracZ0P91+vd/nYAAOyb3t1NMTAwzZs7i64mTiFO7YaraBJPeC43JiNu/x7Ce2Iew\nWFgwZxZt22bjRolZFB8fT/ny5dG4FCAm1kSqJowUiw9qJQWd5hypib/ToUNH3h0ywDbQKJeYzWYi\nIiL48n+TOHb8BMLtRZLNhVCwoNNexpL4G02bNue9d/sTEhLisEnYTSYTnTt3RqPRsGDBAjQaDevX\nr+eriZPZvWsnSv0XSS7gB4qCW0wUYncEz9asyfuDBtCqVSvUaiceMupAiqIgstGCrnzsnM22Mnk+\nBrdH9Z04cYK4uDgMBgNlypShTZs2991YeNOmTYSHh7NixQoaNHjAtW45EBsby4oVK9i9ezdr1qxl\nyJB3qFOnDvXq1Xusd4NISkpi5cqVXLhwgYSERMaNG8vSpUtp3bp1rtzw+fz58zRo0IBJkyZlqEWm\npqaydu1aTp06RVxcHB4eHpQrV47WrVtnmO4ttwkh2LJlC4cOHSI2Nha9Xk/x4sVp164dP/74I3/8\n8QcLFy7Mtff/9ddfmTJlCtu2beOPP/5g9+7dxMTE4OrqStGiRWnXrp3D79/5KMePH2fjxo3Mnj0b\nf39/wsLCaNOmDcWKZeEWf3Ywm8220dhWK4sWLbqvu+LChQusXr2aGzduIITAx8eH0NBQypUr59A4\nngaKoiDGZqPcRzJ5Sg6yfv16unfvzqpVqxzedAmwc+dOPvjgA3bu3OnwbWdHxYoVWbZsGYGBgQ7f\ndkxMDA0aNKBv374MHjzY4dvPbQkJCZQpU4Zt27Y5rN/1bhaLhcDAQCZNmkTTpk0dvv2ceuONN6he\nvTpvvPGGw7dtNpvp2rUrJpOJJUuWOKyfP79SFAWRjdveKsOcM3nKPs8nUIsWLZg+fTpt2rTh4EE7\nZqRxUqVKleL8+fMO367ZbKZz5840bdrUKRMn2CY5Hzx4MGPHZuMnfRYsWLCAwoULO2wgkrNITU2l\nW7duJCUlsXjxYpk4HUUOGJLyWuvWrZkyZQphYWH89VcmA1ueErmRPIUQvPHGG7i5ufHdd1kcTfqE\nGjhwIGvXruXs2bMO3a7FYuHTTz9l1KhR+eomzqmpqXTv3p34+HiWLl16X9eJlAMyeUpPgvbt2zNh\nwgRCQ0Nf5ebgAAAgAElEQVQzvR3Y0yA3kue4ceOIjIxk/vz5Tj+gw8vLizfffJNx48Y5dLv5sdZp\nsVjo2bMnN27cYPny5RluTSY5QD6ans+J837+0KVLF8xmM82bN2fz5s1UqGDHRANOolSpUg5tnp4/\nfz5Tp05lz549uLu7O2y7eWnw4MGUL1+ekSNHUrJkFu8E8xC3a52TJk3KN7VOi8VCr169uHr1Kr/9\n9ptMnLkhH2UUWfN0At26dWPMmDE0a9bM4U13TwJH1jx37tzJ4MGDWbVqlcNHZuYlHx8fXn/9db78\nMhsjMh4gv9U6rVYrffr04d9//2XlypXyuszcIpttpSfNa6+9xogRI2jatGmuDK7JS45Knn///Ted\nO3dm7ty5BAUF5TywJ8yQIUNYsGABly9nbcaozOS3vk6r1Uq/fv04d+4cq1atyp+3CnNC69ato2LF\nipQvX/6BPxq//vpratSoQY0aNQgKCkKj0RATEwPYvlOqVq1KjRo1cuWKBZDJ06n069ePd999lyZN\nmnDx4sW8DsdhihQpQnx8PImJD5jNJYtu3LhBWFgYn376aaZ31HB2hQsXpmfPnnz9dc5m+FmwYAG+\nvr75otZptVoZMGAAJ0+eZPXq1bk6CYiEw/o8LRYLAwcOZN26dRw/fpz58+dz4sSJDOu89957REZG\nEhkZybhx4wgJCcHb23aPWUVR2Lp1K5GRkezfvz839lQmT2czaNAgBg4cSNOmTfnvv5zdO/RJoSgK\nJUuW5N9//81W+eTkZNq1a0f79u15/fXXHRzdk+W9995j1qxZXLt2LVvl81OtUwjBwIEDOXLkCGvW\nrHlq+r+faA5qtt2/fz/lypWjVKlSaLVawsPDiYiIyPRt582bR9euXTM8l9vXjsrk6YSGDBnCa6+9\nRtOmTYmKisrrcBwiu023Qgh69eqFn5+fw0ejPomKFStG165d+eabb7JV/nat80mcEMGRhBC89dZb\nREZGsnbtWjw8PPI6pPwhC8ly62kYteLO8iCXL1+mePHi6Y8DAgIy7a4wGo2sX78+wz1tFUWhWbNm\n1KpVi59++skhu3YvJ+6uzd8+/PBDTCYTzZo1Y8uWLRQqVCivQ8qRkiVLZit5fvzxx5w/f55Nmzbl\nytR+T6Jhw4ZRo0YN3n//fbvmAL5d6/zxxx+f6lqnEIJ33nmHffv28fvvv+Pp6ZnXIeUfWcgoIVVt\ny22jl96/jj3n52+//UbDhg3Tm2wBdu3ahZ+fH9HR0TRv3pyKFSvSqFGjLG8zK/LHt81T6uOPP6ZN\nmzY0b96cmzdv5nU4OZKdmueMGTOYN28eERER+Wr0ZIkSJejQoQPjx4+3q1x+qHUKIRg6dCg7d+5k\nw4YNeHl55XVI+YuD+jz9/f0zjOu4ePFipjdpWLBgwX1Ntn5+fgD4+vrSvn37XOn3lMnTiSmKwuef\nf07Tpk1p0aIFsbGxeR1SttmbPDdt2sQHH3zAmjVrHHbPT2fy4YcfMnny5PTRhY+SH/o6hRB88MEH\nbN68mQ0bNmSoiUiPiYP6PGvVqsWZM2c4f/48JpOJhQsXPvDOQrGxsWzfvj3DDR+MRiPx8fEAJCYm\nsmHDhlwZfS+Tp5NTFIWvvvqKevXqERoamn7SOBt7kufx48fp2rUrixYteionjciKMmXKEBYWxsSJ\nE7O0/tNe6xRCMHz4cNavX8/vv/+Oj49PXoeUPzkoeWo0GiZOnEiLFi0IDAzkpZdeolKlSkydOpWp\nU+/ceHzFihW0aNEiQ8tTVFQUjRo1onr16tSpU4fWrVvnygh8eVeVp4QQgv79+3Ps2DHWrVuHwWAg\nNTUVjeb+s/NJu6sKwJUrV6hevXqmA6CEECiKQlRUFHXr1uWzzz7jlVdeecxRPllOnjzJ888/z9mz\nZx86IOb2nVN+/PFHmjVr9hgjzLmH3VXFarUihEClUvHxxx8TERHB5s2bnb7/31kpioJYn41yLZzz\nripywNBTQlEUJk2aRJ8+fWjTpg3h4eFMmzaNDRs2ZPgVfuXKlfT7Vx4/fpwyZco8EdOUeXl5ERMT\nw5YtW/D29sbPz4+iRYsCtvt/tmnThr59+/L111/Ts2fPfJ84wXYrtyZNmvDjjz9SoUIFSpUqhVqt\nJiEhAXd3d/R6PTt27EClUjldrdNisXD27FmioqK4cOEC58+fp2TJkulNzrcnPkhOTqZMmTIsX778\nqRg4JzkRIT1VUlNTRZ06dQQgAFGrVi1x/fp1sXTpUvFc3aZCrfEQapdyAm0loXYpLnR6H/H220PF\n2bNn8yTeM2fOiJdf7ik0Wg+BuphQXAKFVveMcHUrIOo1eEEsW7ZMdOjQIX1/ateuLaxWa57E+qRJ\nTU0VX3zxhVCr1QJFJ9RqvfAsUFl4FaorPAtUFmqNIe15tXj33XeFyWTK65Af6dq1a2LMmHGiYKES\nQq0tKhRtRaFyeUaoNQVEuWdqiJ9++knEx8eLfv36pZ8Tnp6e4tKlS3kder4HCLHJ/sVZ05BzRi1l\navny5elfKrcXjdZDuPs0FPjMEwQkC4qLO0vRM0Lr865w0xcSXV/uJZKTkx9LnElJSaJjp+7CTe8r\nNAWGCvzOZowrIEngM0e4GGoKFF2G/Zk8efJjifFJ9/PPPwsUg0DlJ/D6SlDsRsZjWOy6wOtLgaqo\nQDGIL774Iq9DzpTVahUjRowWbjpvoSvYS1DkwD3ng0VQaK1wL9jG9kPrnnN84MCBeb0L+R4gxDb7\nF2dNnrLP8yljNBoJCwtj27ZtticUPRRcBLpWDy9oTUSX9CrVKsSwZfPqXG3KNRqNPB/ckuPnipKk\nmwGqR8w1alwBN7uBMBIYGMiuXbvy/UjKf/75h+fqhHDd1Bs8RoDykLF/wgpxo/F1m8X+fVspVarU\nY4szK4QQ9Oz5BksiDmI0rAS138MLmA5BdDOw3gKsFC1alK1bt+bbwWNPCkVRENkYRqE0dM4+T5k8\nn0IJCQk0bNiQw3+dAd8N4NogawWFBZ3xZZo1hIgVC3LlkgYhBKEtO7D9gDvJ+lkP/9K/W/I2lBth\nrF+3gubNmzs8LmcSExND1Wp1uZz4Jlb9oCyXUxm/p7jHVA4f2vNEXf84cuSnfPvDKozum0GVxSn0\nUs/D1Zp4eVrZs2cPlSpVytUYpUdTFAWxNxvl6jpn8pSXqjwFXnnlFfz8/PD09KRMmTJ8//33uHsW\nBe+v7iTO2E/hogqSN98pGP8d/FcWLnnC5SJwsw9JbhPZvPUgq1evpmvXrvj7++Pt7U3Dhg3tvtD4\n3rg+//xztmzZwu69p0hWdYOrgXDJANeaQOpd89rGfQVXg2xx/VcG4r4Gt2DwGsO4L77PcVzO5EHH\ncOLEyUQnPIvVrTfcGgCXfeGSN1wLvlMwdhRc1MIlj7TFE6vLi0TFVmPy5KmMHDmSoKAgtFoto0eP\nzrN9iYqK4n9ffY3RmAJXStj2I6oBpNxVhbn3PL3xGqgKQqHlqLVe3Lhxg+eeew5PT0+qVavGrl27\nHsv+SA+Qj25J5pyNzVIGR48eFUlJSUIIIU6ePCkKFSoktC4eAv9EW3+R398CbZBA7S/w3XSnH8nv\nrMD/pu3f/jcFrk0EHsOEUuBb0fyFtuK7774TV69eFVarVUybNk0UKlRIJCQkZDuuIkWKiFq1Ggi8\nvxQonoKCSwQBKQKPoQKXunfi8vqfoEikrZ+r6CmBuqSg4AKBf5xwcfUUn3zySY7iciYPOoaeXr6C\nIn8I9N0E+q62vs0Aq6DIwTvH0HOUQN89Y79hcSEovF8ULlJazJgxQ6xdu1a8+OKLYvTo0Xm2L927\n9xBuBV4V+J2z7UOAVeA9QaAq8sjzlACrcPeqKjw8PMSSJUuE1WoVc+fOFQUKFBC3bt16LPsk3QEI\nccj+xVnTkKx5PgUqV66coY8yOTkF4db2Tl/irYHg9SWgzVhQUwZUBWz/FlZABWo/hL4n27dv4+WX\nX6ZIkSIoisLrr7+OyWTi9OnT2Y5LpVJx+K/DgCtog0DfERQX8BwF5sNgTtu251BwqW5r0tU+A7oX\nIWUXqDxQDN0xm0WO4nIm9x7D1NRUUq2FQTFA0m9QYBqoC4KigEuNu0reHktzD9faJJkKUrhwYUJD\nQ/Hw8HhsTWb37otGo+G339aT7DIYNKVt+4CF2+fhnRUffJ6iKCSkBmOxCDp27IiiKHTr1g1fX1+W\nLVv2WPZJuoeDpudzBjJ5PiUGDBiAwWCwfUHpvEl1G2h7wbgYFDfQtXxwwcR5cMkL/vMFtS94DAZV\nAVw8gtm+fXv6aocOHcJkMlGuXLlsxxUaGoqrVyikngOXandWUulBUw7MR+/fgBCQsh20VQBIUXdh\necSGHMflTO4+hpUCgzCqeoJpP2hKQuzHtmbbq1XBeHfCUGzJ9XJBuFIFEqakvxJv7cKqVRvue5/H\n4e596dOnD6nCAC41bS9e8oZLOoj/HxRakrHgg85TANdGGI0Z7wNrtVo5duzYY9gb6T75qNlWJs+n\nxKRJk0hISGDjxo3cuH4ZLFfAGg+xw8H7+8wLGl6GgFgoehrMJ2z9S4BF+HLr1i0A4uLi6N69O6NG\njbL71k53x7Vo0SJMKQJEIij33OlC8QSRcP8G4kalxfma7a/al9iYnMflTO4+hvv37bEdJ8sl248N\nlTcUuwLeE+FmDzCftBXSdwG/k1DsOvj8ZOvzNi6wvab2JerarTzfl/HjxyO4a0L/gBjwjwV9OFzv\nbPvhdFsm5yluTQDBnDlzMJvNzJo1i3PnzmE0Gh/rfkn5j0yeTxFFUQgJCcHNzQBJK22JR98dNCXu\nWiuTJjptOfD8ABJnpz1hQaVSpc/uU79+fYYNG5ajuGrWrInFdBoUdxBxGVcSsaDckwDjJ0LiXPBd\nDcrtJmfHxeVMbh9D/4CSkLLHdgkSWvAcAYoG3J4H18aQnFaj1FYCdVFbU6hrPVtNzZhWmxMW1Oq8\n+69/e1+aNGmCOSU644sqPXh9AamnwXzk/sL3nqcqHxRFzYQJEyhatCjr16+nWbNmmd6BQ8pl+ajm\n6cShS5lxcXUjyWS2jay1XIKESbYXrNFwowt4fGDrV7yXMKd9KYOGy3h5edGuXTtKlCiRYTLm7NJq\ntaiVJCzaypA4684L1kRIPQvayneeS5hua74rvB3Uxe48n3oZ78I+Do3LmejcXIAU0N6+IeK9P4Ye\nfXmRIi5TrNidaezy6i4rGo0Gq8Voq2FmiMECWNPPxfvcdZ5ijcbFRceBAwcAW59w2bJlee+993I1\ndikTTtyHaS9Z83Ry0dHRLFiwgMTERCwWC+vXryc5KQ5XbSwU3gxFj0HRw1D0kC0JFZgG7m/aCif8\nDJa0X/7m4xD3hW0QT+olTIkHmD59Onq9npkzZzokrj/++AOsV8Glvq3J0bgMRDLEjQZtddvgIIDE\nX23Nzb4bQFMqw3Z11plYU2OyHZczedAxvHDhH9w058Cloa1FIW4ciFTbgKqUreDWwlY4KcI2iYAQ\nkLIfEibYBl4JK3rrHDp2aEtycjIWiwWz2UxycjJWq/Wx7sv69ettt5OL/xpMkSAsYI2DmCGgqWCr\nZULm5ymgSp5Nw0bBmM1m4uLieO+99yhRokS+vxY4z+SjmqdzjhGW0kVHR4vg4GDh7e0tvLy8RO3a\ntcXs2bOFm877/una1KUyXqpieM12SYDiLtA8Y5vKLcAqND4jRVirdkJRFGEwGIS7u3v6snPnzmzH\nFRERIV7p3keofcYIfDcKNBVtU++5Nhb4XbgrztICXGxx3V7c+wuKRQkXF/ccxeVMHnQMV6xYIUqX\nCRL4/i4oekzgUs82RZ+msqDQijvHUN9VoCqY9tlWFHj/YHved70oW7666NGjh1AUJcMya9asx7ov\nERER4scfJwlX9zpp54K7bSpBfbjA799HnqcEWITBs4xo3ry58PLyEl5eXiI8PFxER0fn2n5ImQOE\n+M/+xVnTkJxh6CnVsVN3IjaXwuL+mX0FLdfRxVbhwL5NVK5c+dHr2ykyMpKGz7fB6HXkzuUHWaRJ\nGEbnlteZ9+svDo/LmUyePIX3hi/BaFgPih3tZMKCPrE5344Lp1+/vrkXoB1OnTpFYOWaWH33Z2y2\nzwrjAsoX/B+nTv751N7g25koioK4lo1yheUMQ9IT5Pvx4/BWzwTjvKwXsibA9WaEtWySK4kToEaN\nGvTs0QV9YjuwZn1EpGKcSQHtQr75ekyuxOVMevfuRZVnLLgY3844IvVhhMDV+BZVKyq89lrPXI0v\na+HYRsg2atSItm1aoEtoBamXsr6BlL3okwcxd84UmTifIEJt/+KsZPJ8SgUEBLBl8xoKiKGoEr8C\nYXp4gdSzGBKCadGkFNu3beTXX3/Ntdh+mPA1bVqURp8QAqn/PHxlkYI6cRw+yki2bV2Ln98jJg3P\nB1xcXFi/bjnli+3HzdgjbYL0h7Dews3YnWcC/mTd2mW4uLg8nkAzcfXqVdq1a8dXX33FunXrWL58\nGaNGvoUuvr5tJPHDCCsYF6JPbMPiRbN47rnnHk/QUpZYNPYvzkomz6dYUFAQB//cRa1n1qG7VRJN\n4kjbHLK3aysiFZJW424Mwz2hLsM/6MraNbabCn/44Yd88803uRKXSqVi/vwZDBvSHkN8bdyNbSBp\nrS0esMWXeh5t4kfobpagTuBWIg/ulpN/38Xb25v9+7bQPtQFt5tlcDP2AdOfd322Akx/ojP2xu1m\nGTq21LF3z+Y8nRBeCMH8+fOpVq0aQUFBHDhwgJo1bRMkvP/+EGb8/DWFRDgeifUgcQ5Yk+4UttxA\nSfgWQ1wFyhYYx+ZNqwgLC8ujPZEyk5+Sp3P21Ep2O378uOjbd5DQGwoIlVorXFw9hKKoRMVKtcWM\nGTOE0WjMsP6///4rAgMDxTvvvCMsFkuuxZWYmCh+/vlnUb5CTaEoKqGodEKl0gqDu4/oP+BtcfLk\nyVx776dFVFSU+OyzscK3SCmhqNQC3ISiUovCRUqLMWPGiaioqLwOUURFRYkOHTqIwMBAsX///kzX\nS01NFREREaJBwxZCpdIIrYtBaDRuQuuiE+07dBO7du2SN0N/QgEiOdH+xVnTkBwwlA8lJyeTnJyM\np6cnKlXmjQ83b97kxRdfpHjx4sycOTPXm/ssFgu1a9dmwoQJNGzYMFff62llMplwdXXFZDKh1Wof\nXeAxWLx4MYMGDaJnz56MGjUqy/eKFUIQHx+PRqNBp9PJvs0nnKIoJCTb35jp7mZ1ygFDzlxplrLJ\nzc0tS19gPj4+bNiwgZdffplWrVqxdOlSPD09H1kuu9RqNRqNJs/75JzZ7YT5JCTO69evM3DgQA4d\nOsSKFSuoW7euXeUVRcnV801yPIsmOynlEeMxnlCyz1N6KJ1Ox5IlSyhbtiwhISFcvXo1r0OSnMDy\n5csJCgrC39+fyMhIuxOn5JwsarXdi7OSNU/pkdRqNZMnT2bMmDE0aNCAdevWUb58+bwOS3oC3bx5\nk7feeot9+/axePFi2fyez1jy0fx8suYpZYmiKIwcOZIPP/yQ559/Pn0uUUm6bdWqVQQFBVGwYEEO\nHTokE2c+lIra7sVZyZqnZJc+ffpQuHBhwsLCmDNnDqGhoXkdkpTHYmJiePvtt9m+fTvz5s0jODg4\nr0OS8oglH6UUWfOU7Na2bVsiIiLo0aMHc+bMyetwpDy0du1agoKCMBgM/PXXXzJx5nMW1HYvzir/\n/EyQHKp+/fps3bqV0NBQrly5wtChQ+WlBPlIXFwcQ4YMYePGjcycOZOmTZvmdUjSE8CZk6G9ZM1T\nyrZKlSqxe/du5syZwzvvvJOrt7SSnhwbN24kKCgItVrNX3/9JROnlC/J5CnliL+/Pzt27CAyMpKu\nXbuSkpKS1yFJuSQ+Pp7+/fvTq1cvpk2bxtSpU+V1mFIG+anZViZPKce8vb1Zv349qamptGzZktjY\n2LwOSXKwLVu2ULVqVUwmE0eOHKFFixZ5HZL0BMpPo21l8pQcws3NjUWLFlGpUiWCg4O5cuVKXock\nOUBiYiKDBg2ie/fuTJw4kV9++SVPJ5eXnmwWNHYvzkomT8lh1Go1EydOpHPnztSvX59Tp07ldUhS\nDuzYsYNq1aoRGxvLkSNHaNWqVV6HJD3h8lOzrfOmfemJpCgKw4cPx8/Pj+DgYCIiIqhTp05ehyXZ\nwWg0MmLECBYsWMDkyZN58cUX8zokyUk4czK0l6x5SrmiV69e/PLLL7Rp04Y1a9bkdThSFu3Zs4ca\nNWpw9epVjhw5IhOnZBdH9nmuW7eOihUrUr58eb788ssHrrN161Zq1KhBlSpVCAkJsatsTsnkKeWa\nVq1asXLlSnr16sXMmTNJTU1l6tSpWCyWvA5NSrN48WKmTp1KcnIy77//Ph06dGDs2LHMmzePggUL\n5nV4kpNxVJ+nxWJh4MCBrFu3juPHjzN//nxOnDiRYZ2YmBjefPNNfvvtN44ePcqSJUuyXNYRZPKU\nclXdunXZtm0bo0aNon79+rzxxht06tSJpKSk9HUSEhKYMGEC7du04dyJEwwZOJD+/fpx7NixPIzc\nuVy5coUPPxxBy7BOoHgQGtaRjz4amenArevXrxMeHk6XLl146623qFy5Mv/88w9//fUXHTt2fMzR\nS08LR/V57t+/n3LlylGqVCm0Wi3h4eFERERkWGfevHl07NiRgIAAAAoVKpTlso4gk6eU6ypUqECn\nTp3SJ5NfsWIFzZs3588//6R/374UK1yYacOGkbpqFY0TEvA9cIAj06bRqHZt6j/7LEuWLHHKm+U+\nDnv27KFV6y6ULhPI15Ousn5fGPj8zPq9YXz143+ULhNIm7bh7N27N73M8uXLqVy5MgsXLgRsN9BW\nqVTMmzcPX1/fvNoVKZ+I3BrP9FH/pS8PcvnyZYoXL57+OCAggMuXL2dY58yZM9y8eZPGjRtTq1at\n9KlCs1LWEeSAISnXxcbGsmjRogzP7dq1i/q1a1NPpaKPxcKDLn5okpTEyYMHeadnT9asXMm06dPR\nZOtmu0+n77+fyEcjxpLkMhxR8GdQZZywIJXepFq/ZfXOOWxu3oGRw9/h6NHD/Prrr/dt68KFC/zx\nxx/Uq1fvcYUvPYWyMmCoaog3VUO80x9PH31/60hWpvo0m80cPHiQTZs2YTQaqVevHnXr1n1s04TK\nbyIp13l5ebF7925CQ0PTm2K1wMtCUOoh/Z9qoDJQLjGRZUuX0tNkYs78+XIOXdIS58gJGD12gaZ0\n5iuqvBDuAzGmhvHRiOcRlvt/6desWZOZM2cSFBSUixFL+YGjRtv6+/tz8eLF9McXL15Mb569rXjx\n4hQqVAidTodOp+P555/n8OHDBAQEPLKsI8hmW+mxCAgIYMeOHdStWxcNEA6UymJZV6Cj0cjO335j\n8qRJuRajs9i7dy8fjRiL0X39wxPn3TRlEIW3g+Jx5ymNhtGjR7N3716ZOCWHcNRo21q1anHmzBnO\nnz+PyWRi4cKFtG3bNsM6L774Ijt37sRisWA0Gtm3bx+BgYFZKusIMnlKueaVV17Bz88PT09PypQp\nw6RJk2jfrh0V1GpKAKuB/wFfADPuKrcH+B4YB3wFrAAE0Mxo5PNPP8XDwyPDolKp+O677x7z3j0e\n9x7Dzz//nLHjvidJ+wGYDsOVynDJ0/Y36Z5BEaaDcO15uOQBl4tC0irw/AQUPVWrVuXAgQMEBwfj\n6urKyJEj82YHpaeKo0bbajQaJk6cSIsWLQgMDOSll16iUqVKTJ06lalTpwJQsWJFQkNDqVq1KnXq\n1OH1118nMDAw07KOpgg5EkPKJceOHaNs2bK4ublx6tQpgoODEWYzrW/e5A9sCbEloAOuAn5p5W4B\nbmnPJwGLAH+gKTDTw4NpS5bwwgsvAHD+/HnKlSvHuXPnKFGixOPdwcfg3mPYqFEjYmKMmAtGQlR1\nKLgMdC0gaQ3c6Ax+50HtC5brcLUyeI8HfScQJrBcBHUR1FElOP/PCYoUKULt2rXR6/U0a9aMTz/9\nNK93V3JiiqKwSth/h53WyianHBAo+zylXFO5cuUMj61WK6rkZHTAKeBdwCXtNb+71itw178FoADu\naX+rxsfz43ffpSfPWbNmERwc/FQmTrj/GKakpKDomoM1GhR3W+IE0IWBYoDUc7bkGf8tuIWCoavt\ndUULqooAaL26MG/eAqzWVEJDQ4mKinLKLy/pySNnGJIkBxkwYAAGg4HKlSvTokULSgD/Ad7AFmzN\ntpOBey9hPsKdZlsDUDfteT/gzOnTAAghmD17Nj169Mj9HclDdx/DihWDMKlDQVsVFI2tKVZYwLgC\nFDfb8wCmfaAqAFEN4HIRiG4LqbZBFMmWZ9m3P5IZM2YwcuRImTglh5F3VZEkB5k0aRIJCQls3LiR\n5cuXk2w2Ewdcw9Y0+x4QBiwHrt9VLgj4EBgERGPrBwXb4KGExEQAdu7cybVr1+jUqdPj2Zk8cvcx\njIw8AJYoULlDgalw4yW45AY3u0GBKaDS2QpZLkLiLCgwAYr9axtYdCOtFqryYPeubYwZMwaDwYCi\nKHIEs+QQ8q4qkuRAiqIQEhJC7dq1uSYEWmwn3vNpf0sCpYGzDyjrAzQEDqc9TgE8DAbA1mTbqVMn\n9Hp9bu9Cnrt9DEuVLgsp22yDgW72hcI7oLgZCm+Dm31sg4gAFD3oO4DLs6C42gYKmXaDNR5SdoMC\nnTt3Bmw1eFn7lCT7OG/al5yOu7s7RkWhcNrj2/2Zj2LFdl0owGWgQmAgSUlJLFmyhBUrVuRGqE8s\nby8PNKpoUpM3g2tdcKlpe8GlFrjWgZRN4FLtTvPtfQRqyzZibt3Az8/W0xwbG4tarebo0aMsX778\n8eyI9FSSfZ6SlEPR0dEsWLCAxMRELBYL69evZ/v27XgWsA0H8gJ2YkuM/wLngXJpZQ8Cibe3k7Ze\nILZke9jdnYHvvMPy5cvx8fHJcCeFp82DjuHJkydQLOdAXQpSdtypaZoibY9vJ03Da5C03Pa6MEPc\nZ/oCidcAACAASURBVODaCIQJDf8RGRnJ4cOHOXToEG3btqVv377MmDEj01gkKSvk/TwlKYcURWHK\nlCn0798fIQTPPPMMc+bM4cKFC0z/8EO6JiayElti9AbaA7fv4XER2AyYAA+gJrYBQ/8CLt7eNG7c\nmJYtW9K9e/fHv2OP0YOO4dy5c5k1ezHLN19AeLwP1zuA9RqoCoPncHBrZivs1hi8xsL1ViCM4NII\nfOahSppB2xfbUbFixfT30el0GAwGvL29M4lEkrLGmQcA2Ute5yk9VrGxsZQvVYpmMTFUsKNcCjDX\nYODj8ePp06dPboXnFP78808aBYeR5LEDtM9kvaD5FLr459m1Yx01atTIvQClfElRFH4RL9tdrrcy\nL8/63K1WK3v37qV+/fp2l5XNttJj5eXlxar161mj1/N3FsskAYv1elp07kzv3r1zMzyn8Oyzz/L9\nd2PRJ7QA86msFTKfRJ/w//buPz7nev/j+OOzX/aT+R0bhpGF2GISi05+lSyi7FSigxBO+Z7TD52c\nnNs5CpVFO0JOBxUjOpawhONnaY7MUQgxZ1YyP2Y/ubbrur5/XDUW2a51zXVdu5732+1zs891vd/X\n9Xr/Ma+9f3768ubs6UqcUmXcbdjWy8uLJ598snJ1HRyLSLliY2NZu2ED62vW5FM/P87+QrliYC+w\nOCiIe4YPZ97ChdpS8aPRo0cyO3EKfufvgNyXwZx97YLmbLwKphOY14M3Z09l5MjHb2yg4lHcLXkC\n9OrVq1KPPdSwrTjNyZMnmT1rFv9YuJCbrFaa5OcTgC1pZgMH/fy4/fbb+b/nn+eee+5xcrSuJzs7\nm9atWxPbpSfbtv4bI+g+isyx4BUCljwCvdOwFK5l4MBBPPfs7+nYsaOzQ5ZqzDAMkqz2jwxNMP7h\n1K1SwcHBFBYW4u3tjb+/P2BrS25u7nXrKXmK0128eJGVK1eS9tlnnM3OJjAoiAOHD3N71668/vrr\nzg7PZf3ud7+jVq1aJCYmcvbsWd57732++voIOTl5hIaG0K5tK4YNe5Q6deo4O1TxAIZh8Ib1Cbvr\nPW0scMt9xkqe4pLWrFnDnDlz+PTTT50dikvasWMHCQkJHDx4kJCQkPIriFQxd06eKSkpbNu2DcMw\n6NGjBwMGDCi3juY8xSXFxcWxa9cuTCaTs0NxOcXFxYwbN47ExEQlTnEp7ng83/PPP8+cOXNo27Yt\nUVFRzJkzh8mTJ5dbz/mRi1xDaGgorVu3Zvfu3XTr1s3Z4biUN954g7CwsGp/pq+4H1dYAGSvtWvX\nkp6ejre3LfYRI0bQsWNHXnnllevWU89TXFbPnj3ZsmWLs8NwKZmZmcyYMYOkpCStPBaX446rbQ3D\nICcnp/Q+JyenQr9bSp7ispQ8r/bUU08xceJEIiMjyy8scoO54yPJJk+eTExMDCNGjGD48OHcdttt\nvPDCC+XW07CtuKy4uDgefvhhTCYTfn5+5Veo5tauXcv+/ftZunSps0MRuSZXmMO0h8ViwcvLi88/\n/5zdu3djGAbTp08vfWjC9bhXS8WjaN7zssLCQiZMmMCCBQtK96KJuBpXGIa1h5eXFzNnzmTo0KHc\nf//99tWtophEHEJDtzbTpk2jS5cu9O7d29mhiPwid5zz7N27N6+99hqZmZmcO3eu9CqP9nmKS9N+\nTzh48CBxcXH897//pXHjxs4OR+SaDMPgOetUu+vNMKY6dZ9nRETEVQuEDMPg2LFj162nYVtxaZ4+\n72m1Whk/fjxTpkxR4hRxMIvFwowZMxg6dKjddTVsKy7tynlPT7R06VLOnz/P+PHjnR2KSLncbbXt\nT3Oelarr4FhEHM5T5z1zcnJ45plnmDdvHj4+GiQS1+eOJwxpzlOqLU+d9xw/fjxms5l58+Y5OxSR\nchmGwUSr/b24N41nXW7OE+D48ePXref8tC9SDk+c99y9ezerVq3i4MGDzg5FpMJcYfWsvTIyMipV\nT8O24vI8bd7TbDYzbtw4ZsyYQe3atZ0djkiFudOc55VznR988EGZ9ypywpCSp7iFnj17snnzZs6e\nPcuJEyfIz893+mOMfg2z2cy5c+fIysoqbcu+fftYt24d8+bNIygoiMcee8zZYYrYxZ3mPJctW1b6\n88svv1zmvfXr15dbX8O24vLWrFnD2nUb+eabb/jzS69Qwz8Ec0ke9eo1ZtLTYxk58nHq1q3r7DAr\nZP/+/SS+8RbLlr2P1eqFt08Apks51Ksfhq93MZmZJ/Dz82PdunU6+F3cjjsO21aWep7isrKysuh6\nRy8GDX6cb7LuhZuOQnghl+r9QEnDAk6Z32fqzP2Eh7dk3JOTKC4udnbIvygzM5POXe6iS9d+vLu6\nIRdDD3Cp/nkKa39na0vJYjKzY4AamEwQHx/PiRMnnB22iF3c8YShytJqW3FJhw8fplv33uSYn6Ak\n8BkwrrNQyJxNYNFjdO5g8Enqv6hRo8aNC7QCDhw4QFyPvlywTsQcMAkM318ubD4NZwZSP/Q7TmYe\n9pgFUuL+DMNgmHWB3fXeNZ5wyhSMt7c3gYGBABQVFREQEFD6XlFRESUlJdetr56nuJzTp0/To+c9\nnLVMoSToT9dPnADe9SkM+oi0fQH89uHfudRc6KlTp+h5172ct/4Nc+Cz10+cAN4NoMEW8k3teewx\n5/ynIlJZ7tTzNJvN5OXlkZeXR0lJSenPP92XR8lTnOrRRx+lUaNG1KxZkxYtWjBt2jT+/NI0si+0\nxHpuEpwM+fEKgkwvMO21VcxLhO9awsmakNUQzj1Bkf9bbNi0l6VLlxISElLm8vLyIjExscrbc+TI\nEfz9/Rk2bBgA4558iuyzF7GenwQnQ+GHbnBpx+UKua/Cqfa2dnzXAnJfA8OPoqBkPl7/GcOHD6d9\n+/b4+vryl7/8pcrjF/k13Gm17a+l5ClONXnyZI4fP05ubi7r169nzpw5LPrnO5hrvQ3heZev2nPB\npyX4RdsqBtwPN/0HwnOh0SEw/w/yZ1Hg9RSL311V5q/I/fv34+XlxeDBg6u8PePHjyc2NhbDMMjN\nzSV1/SdQdzU0Pgth5yEwAc4MKVupzrsQlgP1UyE/CQqXg1cQhd4T2b//MK+++ir9+/fXAiJxeY5c\nbZuamkqbNm1o1aoVM2bM+MVyu3fvxsfHh1WrVpW+FhERwa233kp0dDSxsbEObeNPtNpWnKpt27Zl\n7k0mE4Z/DPg0K1uwYBEEXrF1w6fF5Z+tFsALvBtB4CNs2zaZkydPEh4eDsDixYvp0aMHTZs2rZpG\n/Cg5OZnatWtzyy23cPToUZYseRevoN5Q4/Yf4yy5HOdPaj5z+Wff1rY/Ci7thMChWAMf4+ChP9Ox\nY0dCQkI0hCsuz1HDsGazmQkTJrBx40bCwsLo3Lkz8fHxREVFXVXuueeeo1+/fmVeNwyDLVu2UKdO\nHYfEcy3qeYrTPfnkkwQFBdG2bVvq1W/CRZ8JZQuUnIBL2yHoZ/seC5bCyVrwXX3wrg8hT4FXMN5B\nA1i7di1geyrJkiVLGD58eJW2ITc3l5deeonExMTSJPfe0o8oNB61FTgZCicDIG8m1Ft57Q+xWuHS\nNvBtZ7v3qoVPcL8K7TkTcQWOmvNMS0sjMjKSiIgIfH19SUhIICUl5apyb775JkOGDKF+/fpXvVfV\nf2wqeYrTzZ07l/z8fDZu3MjxYwfBklO2QMESqHHn1b3RoIch/ALcdBiKD9rmQYGLJY05e/YsADt2\n7OD06dMMGfKzoVIHmzJlCqNGjaJx48alw6tnzpwB7zBbgfAcCLvw47Dtg7ZE+XO5U39s1+OlL5nM\nl9si4uoqMsf5w5aDHJy6svS6lqysLJo0aVJ6Hx4eTlZW1lVlUlJSGDduHECZaQ3DMOjVqxedOnXi\n7bffroKWathWXIRhGPTs2ZOQmnXIubgRgkdffrNwCdR88Zcr+0ZCzechdzqETAIMLBYLYBuyHTJk\nSOmS9KqQnp7Opk2b2LvXtpjpp794bf9eMU/pFQi1pkP+36F4P/jdevm9vCQoeA8abi+zItd6RVtE\nXF1FTgwK7dmR0J4dS++P/mXFVWUqMr//9NNPM336dAzDwGq1lulp7ty5k0aNGpGdnU3v3r1p06YN\ncXFxFWxFxSh5ikvx8/ODi1cki0s7wfw9BJTTc7QWg2FLkDW8T1OnTgxFRUWsXLmS1atXV2HEsHXr\nVjIyMkrnVPPz8zGbzRiGDwSe+llpM2ApjdVW4R3bcG6DbeBd9oHXtra0ASr2H4qIMzlqzjMsLIzM\nzMzS+8zMzNI1DD/Zs2cPCQkJgG2UZ/369fj6+hIfH0+jRrZ1BfXr12fQoEGkpaU5PHlq2FacJjs7\nm+TkZAoKCjCbzXzyySfkXjhDDb8rhm0LFtsSp1dQ2cr5C8Gcbfu5+ICt1xk4GCxFWItS6Nu3L//6\n17+oU6cOPXv2rNJ2PPHEExw7dox9+/aRnp7O2LFjue+++3jmmT/id+kN2/YaqxksuZDzf+Bzs623\nDFDwPlz4E9TfAD4RZT/YUkBJ/sfceeedmM1miouLuXjxonqiUu116tSJI0eOkJGRgclkYvny5cTH\nx5cpc+zYMY4fP87x48cZMmQIb731FvHx8RQWFpKXlwdAQUEBGzZsoH379g6PUT1PcRrDMJg3bx7j\nxo3DarXSunVr3nnnHX438kkwnwKvUCj8AOp9eHVl02dw4UWwFth6a0EjIXgSFL5Lp06dadmyJePH\njy/db1mVAgICypxOEhwcjL+/PxMnjuflV2ZA0VBb79kIBv+eUO+jy5UvTAHLOfih8+XXgobZtuYU\nLaNO7WDatGlT+ta0adNYtGiRDo0Xl+SonqePjw9JSUn07dsXs9nMyJEjiYqKYv78+QCMGTPmF+ue\nOnWKBx54AICSkhIeeeQR+vTp45C4rqTj+cTlPDb8CZauqYc5+OXyC1/JWkxw/u0sXTKVAQMGVE1w\ndhqaMIKVGyKwBE+1r6LVRHBeJ1Yuf5W+fftWSWwijmQYBt2tG+yut8Po45bbsJQ8xeVkZmbSoePt\nnPeaXf5c50+sFvwLR3N7h1Ns/PQjvL1d4+SS48ePE3NbN3K837Lt4awIqwX/whHEdcoldf2HeHlp\ndkVcn2EYdLVutrve58Zv3DJ56rdSXE6TJk3YtPFjapVMxChccO1tHVeyFOJ1/iFubnqINR8td5nE\nCdC8eXM2fJJC0MXRtnnacttSQEBhAm1bnmD1v5YqcYpbcaezbX8t/WaKS4qOjmZ32jYi675FcN4t\nGPlzrt7/WXwEv4I/4H+uKbUDdvPAoL4EBwc7J+Dr6NixIy1bNKKu918JzmuHUfB32+KhKxV/g1/B\n0/ifa8r9fYLZse2TKt1eI1IVlDxFXECrVq345tCXrFuzgPviPqfG2WYE50ZSs+BWgnKaEVLYnYmP\n+3Hg692k793J3Llz2blzp7PDvsqLL75IeHg4P5w6xscpc7n3jm34nQm3tSW/PUE5TalZ1IOnRwfz\nzaG9LFv2Dv7+/s4OW8RunpQ8NecpbiM3N5cffviBwsJCatWqRePGjcs873LNmjVMmDCBvXv3VumZ\nlvZITU1l9OjR7N27l3r16pW+/vO2hIWF4etbzuPKRFyYYRi0s6bZXe8rI9Yt5zyVPKVamTRpEhkZ\nGXz44YdOP1Tgu+++47bbbiM5OZkePXo4NRaRqmYYBlHWL+2ud9CIccvkqWFbqVamT59OZmYmSUlJ\nTo3DbDbz6KOPMnbsWCVOkWpIhyRItVKjRg2Sk5Pp2rUr3bt3Jzo62ilxvPLKK1gsFl588Tpn8opU\nM+48h2kvJU+pdiIjI5kzZw5Dhw5lz549hISE3NDv3759O0lJSezZs8elts2IVDVPSp6a85Rqa9So\nUVy6dIklS5bcsPnPs2fPEh0dzbx587j33ntvyHeKuALDMGhmPWh3vRNGlOY8RVzJnDlz+PLLL1m8\nePEN+T6r1cqIESN46KGHlDjFI5nxsftyV+4buUg5AgMDWb58OXfddRddunQhKiqqSr9v9uzZ/PDD\nD6xatapKv0fEVWnYVqQaWbBgAX//+9/ZtWtXmaefONKePXvo168fX3zxBS1atKiS7xBxZYZh0MB6\nwu56p41mbjlsq+Qp1Z7VaiUhIYG6desyd+5ch39+bm4uMTExvPzyyzz00EMO/3wRd2AYBnXNJ+2u\nd9Y7XMlTxFVduHCBmJgYZs6cyeDBgx32uVarlUceeYSQkJDSZw2KeCLDMKh16Xu7612o0cgtk6fm\nPMUj1KpVi+TkZPr3709MTAzNmzd3yOe+88477N+/n7Q0+48lExH3pZ6neJRZs2axYsUKtm/f/qvP\nkv3666/p2bMnW7du5ZZbbnFQhCLuyTAMgguy7a6XH1TfLXueSp7iUSwWCwMGDKBdu3bMmDGj0p9T\nVFRE586dmTRpEiNHjnRghCLuyTAMAi6cs7teUa06Sp4i7uDMmTNER0fz9ttv069fv0p9xpgxY8jL\ny+P99993+gH0Iq7AMAz8zl6wu56pbi23TJ6a8xSPU69ePd577z0SEhLYs2cP9erVw2w2l7uNJTc3\nl5o1a7JixQo2b97Mnj17lDhFrlBS7Dn7PJU8xSP16NGDsWPHMnjwYCwWC+3atWPhwoV8/vnnbN26\nlbPnz+Ln60fDBg0ZPHgwQUFBxMTE0KNHD9auXUtqaio1a9Z0djNEXIrF7DkpRcO24rFWrFjBww8/\njNlsBiC8ZROKvUpodn8kNer6YymxUHQinyMfHqBmaC2yjtn2sIWHh7N582ZatWrlzPBFXIphGHCi\n2P6KzXzdcthWyVM8kslkokOHDhw6dAifQF8adQmnyws9aPqb5hheZY98NuVf4uB7+9j1ty0UnSnE\nfMnMM888w8yZM50UvYjrMQwDvrXYX7Gll5KniDvZtGkT98TfQ5cXexD7/J3lzl+a8i6x6t4lXDpa\nSOax/1XZUX8i7sgwDPimEunkZsMtk6eeqiIeyWq18scXniH22TvpMrlHhRb++IXUYMiGEQRFhJA4\nJ/EGRCkirspzZnfF4x05coT27dvz4IMPMmLECLILzxD3x/5sfHINhz/4CkuxhfodbmLoVtu+zT2J\nn7E3aRdF2YX4BPjQ/N7W/GZOf+5eFM/M7q+y78t97Nixg4KCAtq1a8esWbOIjY11citFnKjE2QHc\nOBq2FY/Rp08fLl68SEREBOcLcrjYC7J2/g+rxcpv3uyPf50ATqd/T8PoxgDkHDuHf+0A/GsHcPF8\nEWuGJNOwcxh3Tu/Dh90X0a15V1577TUaNGjAwoULeeGFF8jIyCAoKMjJLRW58QzDgH2VSCcdNGwr\n4rKSk5OpXbs2d999N4WFhWz59xYaxobx7ZpD9F4QT0DdQAzDKE2cAKEt6uBf2zavabVYwcsguFEI\nAO3/EMvhE4dp2LAhhmEwevRoTCYThw8fdkr7RFxCSSUuN6XkKdVebm4uL730EomJiVitVnJzc7mp\nXWPOfn2ams1C+ezPm5lb/xUW35rEkQ8PlKl7cOk+3qz1N+bWn05g/SBinuoKQHhcMw5+dai0XHp6\nOiaTicjIyBvaNhGXUlyJy00peUq1N2XKFEaNGkXjxo0xDIPi4mL8atUg/2QuZ746TY1Qf8Z+/yx3\nJ93H+uGrOHfo8uHWUQ93YOKFF/nd4ac4dzCbPYmfAeBXy5+C3ALAlpyHDRvG1KlTCQkJcUobRVyC\nuRKXm1LylGotPT2dTZs28fTTTwO2Vbbe3t6UFBTjE+CLt68Xt7/YAy8fb8LvjKDpXc3J2HD0qs+p\nHVmXzs/HcWBJOgDFBSYCgvwpKipiwIAB3HHHHTz33HM3tG0iLseDhm212laqta1bt5KRkUHTpk0B\nyM/Pp6SkhOKSYto/b1sZa7XClRtVfmnbiqXYjE+g7TFmZ/77A42bhjFw4ECaNm2qB2GLgFsnQ3up\n5ynV2hNPPMGxY8fYt28f6enpjB07lvvuu49bO93KxfNFhDStRdor27CUmMnaeYLMLRk062ubt9y/\n8D8UZtuGZs8eOE3a9O20Gmx7bueBuXvxs/oSGBjIokWLnNU8EdfiQT1PJU+p1gICAmjQoAENGjSg\nYcOGBAcHExAQwLNPPcs38/cxMOURjq87TFLoy3w65iPueXcwdVrXA+C7zzJZ3D6JOSF/JWXQUm55\nrCO3TbqDgh/yObr2IAcOHODTTz8lNDSUkJAQQkJC2Llzp5NbLFI9pKam0qZNG1q1anXNZ++mpKTQ\noUMHoqOjue2229i8eXOF6zqC9nmKRzKZTDSLjKBzYg9aD25b4XpWq5WNj6UQGxLD23MXVGGEIu7F\nMAxYW4l00v/qfZ5ms5mbb76ZjRs3EhYWRufOnVm2bBlRUVGlZQoKCkr3VO/fv59BgwZx9OjRCtV1\nBPU8xSP5+fmxdvXHbBubes0FQtditVjY8YdP8TpsYfZrb1RxhCJuyEHDtmlpaURGRhIREYGvry8J\nCQmkpKSUKXPlYST5+fnUq1evwnUdQQuGxGPFxMTw8b/WcP+QgUSN6UD7cZ0IuunaW01O7T7Jf/66\nk5rnAtm47lMCAwNvcLQibqAic5hfbYGvt1y3SFZWFk2aNCm9Dw8P54svvriq3OrVq5k8eTLff/89\nGzZssKvur6XkKR6te/fupO38gmkzX+a9qLdo3ieSpoMiCagbiLnYQu6JHI6+8xWm7EtMHDeBSU9N\nwt/f39lhi7imihx6cHNP2/WTFX+5qkhFHtQAMHDgQAYOHMj27dsZNmwYhw4dKr+Sgyh5isdr2bIl\n78z/B4kzZ7Fo8SI2rtzIjj07qFe/Hm3btGXS1Dfp168f3t7ezg5VxLU56NCDsLAwMjMzS+8zMzMJ\nDw//xfJxcXGUlJRw7tw5wsPD7apbWVowJHINQ4cO5YEHHmDo0KHODkXELRiGAYsrkU6GX71gqKSk\nhJtvvplNmzbRuHFjYmNjr1r08+2339KiRQsMw+DLL7/kwQcf5Ntvv61QXUdQz1NERBzDQfs2fXx8\nSEpKom/fvpjNZkaOHElUVFTpYSRjxoxh1apVLFmyBF9fX4KDg0lOTr5uXUdTz1PkGtTzFLGPYRgw\nvxLpZIweSSYiIuIRNGwrIiKO4cbH7dlLyVNERBxDyVNERMROSp4iIiJ2qsghCdWEkqeIiDiGgw5J\ncAdKniIi4hgathUREbGTkqeIiIidPCh56pAEERERO6nnKSIijqHVtiIiInbSalsRERE7ac5TxLNY\nrVb+/e9/06vXvdSoEcyKFR/w298+SkhIHSZN+iPHjh1zdogirq+kEpeb0iPJxOOtXr2aiRP/QE6O\nifz8DkAbIACwABfw9f0v3t77iI2NZdGiBTRv3ty5AYu4IMMwYFQl0slC93wkmZKneLTExDf405/+\nSlHRfUBzwPiFksV4ee2mZs29bN78CdHR0TcwShHXZxgGPF6JdPJP90yeGrYVj/Xuu+/y4osvU1Q0\nHGjBLydOAF8sljvIybmL3/ymL8ePH79BUYq4EQ3bilRv58+fJyysGUVFw4AGdtX18vqMuDgrW7Zs\nqJrgRNyQYRjw20qkk2Xu2fPUalvxGEeOHKF9+/Y8+OCDdOwYjdlcF1h4RQkrtj+FnwAaAZ8DaUAh\ntl+VVsA9WCyd2bVrNvHx8ezZs4eCggLatWvHrFmziI2NvcGtEhFnUPIUjzF+/PjS5Pb663Mwme4G\nml5RIh3Yhi1xgm3hUEdsi4eKgBXAdqAXFksbcnPz+fLLL2nQoAELFy6kf//+ZGRkEBQUdKOaJOJa\nPOiQBM15ikdITk6mdu3a3H333Zw7d468PBPQ5Gel0oEOV9zXxpY4wdYrNYBgAIqLYzlw4DANGzbE\nMAxGjx6NyWTi8OHDVdwSERdmrsTlppQ8pdrLzc3lpZdeIjExEavVysWLF/H2rkPZBUI5wP8omzwB\n9gOvAK8CQcDtP74eSk7O2dJS6enpmEwmIiMjq6wdIi7PgxYMadhWqr0pU6YwatQoGjdujGEYWCwW\nrv67cR/QDAj92evtf7zOYRu2/RzoCnhhNtvGqHJzcxk2bBhTp04lJCSkKpsi4trcOBnaS8lTqrX0\n9HQ2bdrE3r17AdtJQjVq1MC2COhK+4A7r/NJdYDuwA5sybOIwMCaFBUVMWDAAO644w6ee+65KmiB\niBvxoDlPJU+p1rZu3UpGRgZNm9oWBuXn52M2m7l48RJwAaiFbbg2H7ilnE+zAL4AGMYhunXrxsCB\nA2natCnz58+vsjaIuA03nsO0l/Z5SrVWVFREXl4eYOt1vvbaa2RkZODvH0xycgYlJT2BNdh+6wf+\nrPaXwM3Y5jqzgQ+AaOB2goLe5tZbm9GwYUNWrlyJt7f3jWqSiEsyDAPiKpFOtmufp4jLCQgIICAg\noPQ+ODiYgIAAnn/+GVat6k5JSRfga2DoNWpnApsBExACxGBbMHScwEDYtWsXgYGBhIZenidNTU2l\nW7duVdgiERfmQXOe6nmKx0pIGMaaNfsoLBxExRee5xAQsISlS//BwIE/76mKeC7DMKBrJdLJ5+7Z\n89RWFfFYS5b8g5iYegQEfEjFVjqcJjDwfaZN+7MSp8i1FFficlPqeYpHM5lMDB8+ipSUjyku7kBJ\nSTS2RUQ/sQKZBAbuxWo9yvz5f2fYsGFOilbEdRmGAdGVSCd73bPnqeQpAhw4cIDZs5N499338PWt\nj2EEAWbM5vOEhPjxxz8+xeOPj6B27drODlXEJRmGAe0rkU72K3mKuL38/HzS09M5f/48fn5+NGjQ\ngA4dOuDlpRkOkesxDAOiKpFODip5ioiIhzIMAyIrkU6Oumfy1J/TIiLiGA48GD41NZU2bdrQqlUr\nZsyYcdX7hw4domvXrvj7+/P666+XeS8iIoJbb72V6OjoKntMoPZ5ioiIYzhon6fZbGbChAls3LiR\nsLAwOnfuTHx8PFFRUaVl6taty5tvvsnq1auvqm8YBlu2bKFOnTqOCega1PMUERHHcNBTVdLSXQOm\nKgAABCdJREFU0oiMjCQiIgJfX18SEhJISUkpU6Z+/fp06tQJX1/fa35GVQ8Fq+cpIiI3zqUtYNpy\n3SJZWVk0aXL5ebvh4eF88cUXFf4KwzDo1asX3t7ejBkzhtGjR1cy2F+m5CkiIo5RkUMPvHqCf8/L\n9/l/uaqIYRhXvWaPnTt30qhRI7Kzs+nduzdt2rQhLi7uV33mz2nYVkREHMNBC4bCwsLIzMwsvc/M\nzCQ8PLzCYTRq1AiwDe0OGjSItLQ0u5tSHiVPERFxDAfNeXbq1IkjR46QkZGByWRi+fLlxMfHX7Ps\nz+c2CwsLS5+kVFBQwIYNG2jfvv2vbtrPadhWREQcw0GrbX18fEhKSqJv376YzWZGjhxJVFRU6XNz\nx4wZw6lTp+jcuTO5ubl4eXkxe/ZsDhw4wOnTp3nggQds4ZSU8Mgjj9CnTx/HBHYFHZIgIiK/mmEY\n4FOJdFLinockqOcpIiKOcZ1DD6obJU8REXEM9+tAVpoWDImIiNhJyVNERMROSp4iIiJ2UvIUERGx\nkxYMiYiIg1TkfL7qQclTREQcxEGnJLgBJU8REXEQ9TxFRETspJ6niIiIndTzFBERsZPnJE9tVRER\nEbGTep4iIuIgmvMUERGxk+cM2yp5ioiIg6jnKSIiYif1PEVEROyknqeIiIid1PMUERGxk3qeIiIi\ndvKcnqcOSRAREbGTep4iIuIgGrYVERGxk+cM2yp5ioiIg6jnKSIiYif1PEVEROyk5CkiImInDduK\niIjYST1PERERO3lOz1OHJIiIiNhJPU8REXEQDduKiIjYScO2IiIidiquxHVtqamptGnThlatWjFj\nxoxrlvn9739Pq1at6NChA3v37rWr7q+l5CkiIg5SUonramazmQkTJpCamsqBAwdYtmwZBw8eLFNm\n3bp1HD16lCNHjrBgwQLGjRtX4bqOoOQpIiIO4pieZ1paGpGRkURERODr60tCQgIpKSllynz00UcM\nHz4cgC5dupCTk8OpU6cqVNcRNOcpIiIO8qLdNYKDg696LSsriyZNmpTeh4eH88UXX5RbJisri+++\n+67cuo6g5CkiIr+a1Wp12GcZhnHDv9NeSp4iIuJSwsLCyMzMLL3PzMwkPDz8umVOnjxJeHg4xcXF\n5dZ1BM15ioiIS+nUqRNHjhwhIyMDk8nE8uXLiY+PL1MmPj6eJUuWALBr1y5CQ0Np2LBhheo6gnqe\nIiLiUnx8fEhKSqJv376YzWZGjhxJVFQU8+fPB2DMmDHce++9rFu3jsjISIKCgvjnP/953bqOZlid\nOWgsIiLihjRsKyIiYiclTxERETspeYqIiNhJyVNERMROSp4iIiJ2UvIUERGxk5KniIiInZQ8RURE\n7KTkKSIiYiclTxERETspeYqIiNhJyVNERMROSp4iIiJ2UvIUERGx0/8D+ijzD08pxqsAAAAASUVO\nRK5CYII=\n", 845 | "text": [ 846 | "" 847 | ] 848 | } 849 | ], 850 | "prompt_number": 26 851 | }, 852 | { 853 | "cell_type": "markdown", 854 | "metadata": {}, 855 | "source": [ 856 | "The last presented tool is a checkpointing facility. Rather than a DEAP object, checkpointing is ensured by the powerful pickle module available in the standard library. Pickle is a module that can serialize almost any Python object. Checkpointing only requires selecting objects that shall be preserved and the write frequency. This is exactly what is done in the following lines that can be added at the end of the generational loop" 857 | ] 858 | }, 859 | { 860 | "cell_type": "code", 861 | "collapsed": false, 862 | "input": [ 863 | "import pickle\n", 864 | "\n", 865 | "freq = 5\n", 866 | "g = 0\n", 867 | "if g % freq == 0:\n", 868 | " cp = dict(population=pop, generation=g, rndstate=random.getstate())\n", 869 | " pickle.dump(cp, open(\"checkpoint.pkl\", \"w\"), 2)" 870 | ], 871 | "language": "python", 872 | "metadata": {}, 873 | "outputs": [], 874 | "prompt_number": 27 875 | }, 876 | { 877 | "cell_type": "markdown", 878 | "metadata": {}, 879 | "source": [ 880 | "These last lines write into a file the population, the generation number, and the random number generator state so that this information can be used latter to restart an evolution from this exact point in time.\n", 881 | "Reloading the data is as simple as reading the pickled dictionary and accessing its attributes." 882 | ] 883 | }, 884 | { 885 | "cell_type": "code", 886 | "collapsed": true, 887 | "input": [ 888 | "cp = pickle.load(open(\"checkpoint.pkl\", \"r\"))\n", 889 | "pop = cp[\"population\"]\n", 890 | "g = cp[\"generation\"]\n", 891 | "random.setstate(cp[\"rndstate\"])" 892 | ], 893 | "language": "python", 894 | "metadata": {}, 895 | "outputs": [], 896 | "prompt_number": 28 897 | }, 898 | { 899 | "cell_type": "markdown", 900 | "metadata": {}, 901 | "source": [ 902 | "This simple mechanism provides fault tolerance to any sort of evolutionary algorithms implemented with DEAP. This happens to be critical when exploiting large computational resources where chances of failure grow quickly with the number of computing nodes. Even in very stable execution environments, checkpoints can significantly reduce the amount of time spent on experimentation by allowing evolutions to restart and continue beyond the original stopping criteria.\n", 903 | "\n", 904 | "##Conclusion\n", 905 | "DEAP proposes an agile framework to easily prototype and execute explicit evolutionary algorithms. Its creator module is instrumental for building custom transparent data structures for the problem at hand. Its toolbox gathers all necessary operators and their arguments in a single handy structure. Its design provides straightforward distributed execution with multiple distribution libraries. The presented examples only cover a small part of DEAP's capabilities that include evolution strategies (including CMA-ES), multi-objective optimization (NSGA-II and SPEA-II), co-evolution, particle swarm optimization, as well as many benchmarks (continuous, binary, regression, and moving peaks), and examples (more than 40).\n", 906 | "\n", 907 | "After more than 4 years of development, DEAP version 1.0 has been released in February 2014.\n", 908 | "DEAP is an open source software, licensed under LGPL, developed primarily at the Computer Vision and Systems Laboratory of Universit\u00e9 Laval, Qu\u00e9bec, Canada. DEAP is compatible with Python 2 and 3. It has a single dependency on Numpy for computing statistics and running CMA-ES. Try it out and become nimbler too: ." 909 | ] 910 | }, 911 | { 912 | "cell_type": "markdown", 913 | "metadata": {}, 914 | "source": [ 915 | "##References\n", 916 | "\n", 917 | "- [1] F.-A. Fortin, F.-M. De Rainville, M.-A. Gardner, M. Parizeau, and C. Gagn\u00e9.\n", 918 | "DEAP: Evolutionary Algorithms Made Easy.\n", 919 | "*Journal of Machine Learning Research*,\n", 920 | "13:2171--2175, 2012.\n", 921 | "\n", 922 | "- [2] F.-M. De Rainville, F.-A. Fortin, M.-A. Gardner, M. Parizeau, and C. Gagn\u00e9.\n", 923 | "DEAP: A Python Framework for Evolutionary Algorithms. \n", 924 | "*In Companion Proceedings of the Genetic and Evolutionary Computation Conference*,\n", 925 | "pages 85--92, 2012.\n", 926 | "\n", 927 | "- [3] Y. Hold-Geoffroy, O. Gagnon, and M. Parizeau.\n", 928 | "SCOOP: Scalable COncurrent Operations in Python.\n", 929 | "\n", 930 | "\n", 931 | "- [4] J. R. Koza.\n", 932 | "Genetic Programming - On the Programming of Computers by Means of Natural Selection.\n", 933 | "MIT Press, 1992.\n", 934 | "\n", 935 | "- [5] A. A. Hagberg, D. A. Schult, and P. J. Swart. \n", 936 | "Exploring network structure, dynamics, and function using NetworkX.\n", 937 | "*In Proceedings of the Python in Science Conference*,\n", 938 | "pages 11-15, 2008.\n", 939 | "\n", 940 | "\n", 941 | "- [6] A. A. Hagberg, D. A. Schult, and M. Renieris.\n", 942 | "PyGraphviz a Python interface to the Graphviz graph layout and visualization package.\n", 943 | "\n", 944 | "\n", 945 | "- [7] E. Jones and T. Oliphant and P. Peterson and others.\n", 946 | "SciPy: Open source scientific tools for Python.\n", 947 | "" 948 | ] 949 | } 950 | ], 951 | "metadata": {} 952 | } 953 | ] 954 | } -------------------------------------------------------------------------------- /images/deap_highlights.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DEAP/notebooks/6ce40541b989c9b07275f394ab03ef0d8561ccc9/images/deap_highlights.png -------------------------------------------------------------------------------- /styles/custom.css: -------------------------------------------------------------------------------- 1 | 46 | --------------------------------------------------------------------------------