├── .gitattributes ├── 000-introduction.ipynb ├── 003-cameras-and-eyes.ipynb ├── 005-images-as-arrays.ipynb ├── 010-linear-filters.ipynb ├── 012-gradient-filters.ipynb ├── 014-anisotropic-filters.ipynb ├── 020-frequency-domain.ipynb ├── 025-fourier-transform.ipynb ├── 027-image-compression.ipynb ├── 030-nonlinear-filters.ipynb ├── 033-document-normalization.ipynb ├── 035-binary-morphology.ipynb ├── 037-grayscale-morphology.ipynb ├── 038-1d-grayscale-morphology.ipynb ├── 042-distance-transform.ipynb ├── 045-labeling.ipynb ├── 070-template-matching.ipynb ├── 101-intro-classification.ipynb ├── 102-nearest-neighbor-and-preproc.ipynb ├── 105-simple-probability.ipynb ├── 107-bayesian-decision-theory.ipynb ├── 118-parametric-gaussian-classifier.ipynb ├── 125-perceptrons.ipynb ├── 126-linear-lsq.ipynb ├── 127-logistic-regression-grad.ipynb ├── 128-linear-to-nonlinear.ipynb ├── 130-kmeans.ipynb ├── 135-simple-pca.ipynb ├── 140-summary-classifiers.ipynb ├── 150-speech-worksheet.ipynb ├── 160-fourier-transform.ipynb ├── 170-dtw.ipynb ├── 180-markov-models.ipynb ├── 185-hmm-ocr.ipynb ├── 200-intro-python.ipynb ├── 205-quadratics.ipynb ├── 210-normal-distributions.ipynb ├── 220-bayesian-parameter-estimation.ipynb ├── Figures-intro ├── bayer-pattern.png ├── bear-illum-1.png ├── bear-illum-2.png ├── bear-illum-3.png ├── bear-illum-4.png ├── camera-model-2.png ├── camera-model.png ├── ccd-sensor.png ├── cie-colorspace.png ├── cie-sunlight.png ├── cmyk-equations.png ├── cmyk-formulas.png ├── cmyk.png ├── color-channels-historical.png ├── color-circle.png ├── color-cube-schematic.png ├── color-pyramid.png ├── color-space-cones.png ├── colorcube.png ├── common-rgb.png ├── cube-color-1-revealed.png ├── cube-color-1.png ├── cube-color-2-revealed.png ├── cube-color-2.png ├── eye.png ├── gamut.png ├── gimp-a-channel.png ├── gimp-channel-decompose.png ├── gimp-hsv.png ├── grey-quant-skull-2.png ├── grey-quant-skull.png ├── human-cones.png ├── illustration-level-quant.png ├── image-formation.png ├── intensity-quantization.png ├── lcd-spectra.png ├── lenslet.png ├── metameric-patch.png ├── metamerism.png ├── newton-purple.png ├── prism.png ├── reflected-light.png ├── rgb-color-display.png ├── rgb-pixel-lcd.png ├── rose-spatial-quant.png ├── sensitivity-curves.png ├── sensitivity.png ├── shadow-illusion.png ├── spatial-quantization.png ├── spectrum.png ├── transformation-imaging.png └── visible-spectrum.png ├── Figures ├── states-1.png ├── states-1.xcf ├── states-2.png ├── states-3.png └── states-4.png ├── Frequency Modulation and Encoding.ipynb ├── HG_HISTORY ├── Initialen.jpg ├── Makefile ├── Professor_X_03.jpg ├── README.md ├── bookpage.jpg ├── bridge.jpg ├── floorplan.png ├── hw-edge-detection.ipynb ├── hw-frequency-modulation.ipynb ├── imagenoise.jpg ├── lexx.jpg ├── lexx.png ├── lexx2.jpg ├── lexx2.png ├── lw.tex ├── misc ├── 049-thresholding.ipynb ├── cooley-tukey.ipynb ├── dct-example.ipynb ├── dtw-synchronous.ipynb ├── intro-images.odp ├── intro-mlp.ipynb ├── noise-generation.ipynb ├── openfst-misc.odp ├── pencil-sketch.ipynb ├── temp-speech-hacks.ipynb └── temp-speech-worksheet.ipynb ├── mkindex ├── msak0_306.ema ├── msak0_306.epg ├── msak0_306.lab ├── msak0_306.lar ├── msak0_306.wav ├── ocr-hw ├── HOMEWORK ├── c.png ├── charread.ipynb ├── charread.py ├── charread.pyc ├── t.png ├── w.png └── w.txt ├── river.jpg ├── simple-ocr ├── small1.png ├── template-A.png ├── template-E.png └── template-I.png ├── test.wav ├── test2.wav ├── testimage.jpg ├── testimage.png ├── testpage.png ├── testpar.png └── tmp.png /.gitattributes: -------------------------------------------------------------------------------- 1 | *.ipynb linguist-language=Python 2 | *.tex linguist-detectable=false 3 | *.html linguist-detectable=false 4 | -------------------------------------------------------------------------------- /101-intro-classification.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "id": "d642ce77", 6 | "metadata": {}, 7 | "source": [ 8 | "Title: Introduction to Classification\n", 9 | "Author: Thomas Breuel\n", 10 | "Institution: UniKL" 11 | ] 12 | }, 13 | { 14 | "cell_type": "code", 15 | "execution_count": 18, 16 | "id": "a269b49c", 17 | "metadata": { 18 | "collapsed": false 19 | }, 20 | "outputs": [], 21 | "source": [ 22 | "\n", 23 | "import numpy,scipy,scipy.ndimage,zlib\n", 24 | "import random as pyrandom\n", 25 | "\n", 26 | "from pylab import *\n", 27 | "from pylab import random as arandom\n", 28 | "\n", 29 | "def method(cls):\n", 30 | " import new\n", 31 | " def _wrap(f):\n", 32 | " cls.__dict__[f.func_name] = new.instancemethod(f,None,cls)\n", 33 | " return None\n", 34 | " return _wrap" 35 | ] 36 | }, 37 | { 38 | "cell_type": "markdown", 39 | "id": "b79c0edb", 40 | "metadata": {}, 41 | "source": [ 42 | "# An Object-Oriented View of Classification" 43 | ] 44 | }, 45 | { 46 | "cell_type": "markdown", 47 | "id": "564455b7", 48 | "metadata": {}, 49 | "source": [ 50 | "In regular software engineering, people think of software \n", 51 | "development as starting from a specification. In pattern \n", 52 | "recognition, however, you don't know the \"specification\". Instead, \n", 53 | "you have a source of data that generates instances. Let's \n", 54 | "look at how this works.\n", 55 | "\n", 56 | "First, we generate a problem instance. In this case, the problem instance is actually the \n", 57 | "instantiation of a class, but in the real world, the problem instance \n", 58 | "is usually some particular physical instance: a device, \n", 59 | "a web site, a person, a book, etc.\n", 60 | "\n", 61 | "We may call the problem instance nature, \n", 62 | "in order to emphasize that it is usually \n", 63 | "given by physics and not another software system.\n", 64 | "However, \"nature\" may well be another software system." 65 | ] 66 | }, 67 | { 68 | "cell_type": "code", 69 | "execution_count": 19, 70 | "id": "e96f3c4d", 71 | "metadata": { 72 | "collapsed": false 73 | }, 74 | "outputs": [], 75 | "source": [ 76 | "# Nature\n", 77 | "class Nature:\n", 78 | " def training_sample(self):\n", 79 | " return (c,r)\n", 80 | " def challenge(self):\n", 81 | " return c\n", 82 | " def response(self,r):\n", 83 | " return reward" 84 | ] 85 | }, 86 | { 87 | "cell_type": "code", 88 | "execution_count": 20, 89 | "id": "ebe9e99c", 90 | "metadata": { 91 | "collapsed": true 92 | }, 93 | "outputs": [], 94 | "source": [ 95 | "# An instance of \"nature\"\n", 96 | "class SevenSegments(Nature):\n", 97 | " def __init__(self):\n", 98 | " self.vs = [None] * 10\n", 99 | " self.vs[0] = array((1,1,1,1,1,1,0))\n", 100 | " self.vs[1] = array((0,1,1,0,0,0,0))\n", 101 | " self.vs[2] = array((1,1,0,1,1,0,1))\n", 102 | " self.vs[3] = array((1,1,1,1,0,0,1))\n", 103 | " self.vs[4] = array((0,1,1,0,0,1,1))\n", 104 | " self.vs[5] = array((1,0,1,1,0,1,1))\n", 105 | " self.vs[6] = array((1,0,1,1,1,1,1))\n", 106 | " self.vs[7] = array((1,1,1,0,0,0,0))\n", 107 | " self.vs[8] = array((1,1,1,1,1,1,1))\n", 108 | " self.vs[9] = array((1,1,1,1,0,1,1))" 109 | ] 110 | }, 111 | { 112 | "cell_type": "markdown", 113 | "id": "712a8381", 114 | "metadata": {}, 115 | "source": [ 116 | "Nature gives us _training samples_ consisting of measurements and correct responses.\n" 117 | ] 118 | }, 119 | { 120 | "cell_type": "code", 121 | "execution_count": 21, 122 | "id": "3855135d", 123 | "metadata": { 124 | "collapsed": false 125 | }, 126 | "outputs": [], 127 | "source": [ 128 | "@method(SevenSegments)\n", 129 | "def training_sample(self):\n", 130 | " c = pyrandom.randint(0,9)\n", 131 | " v = self.vs[c]\n", 132 | " return (v,c)" 133 | ] 134 | }, 135 | { 136 | "cell_type": "markdown", 137 | "id": "4301aa58", 138 | "metadata": {}, 139 | "source": [ 140 | "Based on these training samples, we need to build a model that then returns correct classifications. These are training samples without the classification.\n", 141 | "\n" 142 | ] 143 | }, 144 | { 145 | "cell_type": "code", 146 | "execution_count": 22, 147 | "id": "6370e213", 148 | "metadata": { 149 | "collapsed": false 150 | }, 151 | "outputs": [], 152 | "source": [ 153 | "@method(SevenSegments)\n", 154 | "def challenge(self):\n", 155 | " v,self.c = self.training_sample()\n", 156 | " return v" 157 | ] 158 | }, 159 | { 160 | "cell_type": "markdown", 161 | "id": "48508c74", 162 | "metadata": {}, 163 | "source": [ 164 | "Our classifier needs to respond with a class, and nature evaluates our response and gives us feedback.\n", 165 | "\n" 166 | ] 167 | }, 168 | { 169 | "cell_type": "code", 170 | "execution_count": null, 171 | "id": "9518a38e", 172 | "metadata": { 173 | "collapsed": false 174 | }, 175 | "outputs": [], 176 | "source": [ 177 | "@method(SevenSegments)\n", 178 | "def response(self,c):\n", 179 | " return (c==self.c)" 180 | ] 181 | }, 182 | { 183 | "cell_type": "code", 184 | "execution_count": 23, 185 | "id": "5aa34c99", 186 | "metadata": { 187 | "collapsed": false 188 | }, 189 | "outputs": [ 190 | { 191 | "data": { 192 | "text/plain": [ 193 | "(array([1, 1, 1, 1, 1, 1, 0]), 0)" 194 | ] 195 | }, 196 | "execution_count": 23, 197 | "metadata": {}, 198 | "output_type": "execute_result" 199 | } 200 | ], 201 | "source": [ 202 | "nature = SevenSegments()\n", 203 | "nature.training_sample()" 204 | ] 205 | }, 206 | { 207 | "cell_type": "markdown", 208 | "id": "dabf0858", 209 | "metadata": {}, 210 | "source": [ 211 | "We are trying to build a _model_ of nature.\n", 212 | "These respond to challenges by predicting a response.\n", 213 | "\n" 214 | ] 215 | }, 216 | { 217 | "cell_type": "code", 218 | "execution_count": 24, 219 | "id": "d46afd31", 220 | "metadata": { 221 | "collapsed": false 222 | }, 223 | "outputs": [], 224 | "source": [ 225 | "# Models\n", 226 | "class Model:\n", 227 | " def __init__(self,dataset):\n", 228 | " self.dataset = dataset\n", 229 | " def predict(self,v):\n", 230 | " return 0" 231 | ] 232 | }, 233 | { 234 | "cell_type": "markdown", 235 | "id": "f69daeb3", 236 | "metadata": {}, 237 | "source": [ 238 | "Nature gives us training examples. These training examples consist of some kind of measurement vector v and a corresponding classification c.\n", 239 | "\n" 240 | ] 241 | }, 242 | { 243 | "cell_type": "code", 244 | "execution_count": 25, 245 | "id": "b7fc36e0", 246 | "metadata": { 247 | "collapsed": false 248 | }, 249 | "outputs": [ 250 | { 251 | "name": "stdout", 252 | "output_type": "stream", 253 | "text": [ 254 | "measurement vector [1 0 1 1 0 1 1]\n", 255 | "class 5\n" 256 | ] 257 | } 258 | ], 259 | "source": [ 260 | "v,c = nature.training_sample()\n", 261 | "print \"measurement vector\",v\n", 262 | "print \"class\",c" 263 | ] 264 | }, 265 | { 266 | "cell_type": "markdown", 267 | "id": "f869f2c8", 268 | "metadata": {}, 269 | "source": [ 270 | "At training time, we can request training samples.\n", 271 | "Usually, generating labeled training samples costs money, so we only obtain a limited number of them. \n", 272 | "These are usually collected in a dataset.\n", 273 | "\n" 274 | ] 275 | }, 276 | { 277 | "cell_type": "code", 278 | "execution_count": 26, 279 | "id": "da1fb60d", 280 | "metadata": { 281 | "collapsed": false 282 | }, 283 | "outputs": [ 284 | { 285 | "data": { 286 | "text/plain": [ 287 | "[(array([1, 1, 0, 1, 1, 0, 1]), 2),\n", 288 | " (array([1, 0, 1, 1, 1, 1, 1]), 6),\n", 289 | " (array([1, 1, 1, 1, 1, 1, 0]), 0),\n", 290 | " (array([1, 1, 1, 1, 1, 1, 0]), 0),\n", 291 | " (array([1, 1, 1, 1, 0, 0, 1]), 3)]" 292 | ] 293 | }, 294 | "execution_count": 26, 295 | "metadata": {}, 296 | "output_type": "execute_result" 297 | } 298 | ], 299 | "source": [ 300 | "trainingset = [nature.training_sample() for i in range(100)]\n", 301 | "trainingset[:5]" 302 | ] 303 | }, 304 | { 305 | "cell_type": "markdown", 306 | "id": "04b3580d", 307 | "metadata": {}, 308 | "source": [ 309 | "We usually use the dataset to create a model. \n", 310 | "\n", 311 | "\n" 312 | ] 313 | }, 314 | { 315 | "cell_type": "code", 316 | "execution_count": 27, 317 | "id": "3b31eed4", 318 | "metadata": { 319 | "collapsed": true 320 | }, 321 | "outputs": [], 322 | "source": [ 323 | "model = Model(trainingset)" 324 | ] 325 | }, 326 | { 327 | "cell_type": "markdown", 328 | "id": "bd43aaa3", 329 | "metadata": {}, 330 | "source": [ 331 | "After training, nature presents us with challenges or test samples.\n", 332 | "\n" 333 | ] 334 | }, 335 | { 336 | "cell_type": "code", 337 | "execution_count": 28, 338 | "id": "37ca2c8e", 339 | "metadata": { 340 | "collapsed": false 341 | }, 342 | "outputs": [ 343 | { 344 | "data": { 345 | "text/plain": [ 346 | "array([1, 1, 1, 1, 0, 1, 1])" 347 | ] 348 | }, 349 | "execution_count": 28, 350 | "metadata": {}, 351 | "output_type": "execute_result" 352 | } 353 | ], 354 | "source": [ 355 | "nature.challenge()" 356 | ] 357 | }, 358 | { 359 | "cell_type": "markdown", 360 | "id": "afaab8b6", 361 | "metadata": {}, 362 | "source": [ 363 | "The task of the model is to predict a value based on the challenge.\n", 364 | "\n" 365 | ] 366 | }, 367 | { 368 | "cell_type": "code", 369 | "execution_count": 29, 370 | "id": "d201f147", 371 | "metadata": { 372 | "collapsed": false 373 | }, 374 | "outputs": [ 375 | { 376 | "name": "stdout", 377 | "output_type": "stream", 378 | "text": [ 379 | "0\n" 380 | ] 381 | } 382 | ], 383 | "source": [ 384 | "prediction = model.predict(nature.challenge())\n", 385 | "print prediction" 386 | ] 387 | }, 388 | { 389 | "cell_type": "markdown", 390 | "id": "84b57735", 391 | "metadata": {}, 392 | "source": [ 393 | "Our prediction is then handed back to nature for evaluation. We may or may not see this evaluation, but eventually, our model will be judged on the quality of its predictions.\n", 394 | "\n" 395 | ] 396 | }, 397 | { 398 | "cell_type": "code", 399 | "execution_count": 30, 400 | "id": "a4a665fb", 401 | "metadata": { 402 | "collapsed": false 403 | }, 404 | "outputs": [ 405 | { 406 | "data": { 407 | "text/plain": [ 408 | "False" 409 | ] 410 | }, 411 | "execution_count": 30, 412 | "metadata": {}, 413 | "output_type": "execute_result" 414 | } 415 | ], 416 | "source": [ 417 | "nature.response(prediction)" 418 | ] 419 | }, 420 | { 421 | "cell_type": "markdown", 422 | "id": "9494acc4", 423 | "metadata": {}, 424 | "source": [ 425 | "A second important question is how we validate model. In standard software engineering, \n", 426 | "the assumption is (rightly or wrongly) that all we need to show is \n", 427 | "conformance to the specification. However, pattern recognition and \n", 428 | "machine learning methods do not have a specification. They do, however, \n", 429 | "have lots of data, and that lets us make good empirical predictions about performance." 430 | ] 431 | }, 432 | { 433 | "cell_type": "code", 434 | "execution_count": 31, 435 | "id": "356b6722", 436 | "metadata": { 437 | "collapsed": false 438 | }, 439 | "outputs": [ 440 | { 441 | "data": { 442 | "text/plain": [ 443 | "93" 444 | ] 445 | }, 446 | "execution_count": 31, 447 | "metadata": {}, 448 | "output_type": "execute_result" 449 | } 450 | ], 451 | "source": [ 452 | "# evaluating the model\n", 453 | "def count_errors(nature,model,trials=100):\n", 454 | " errors = 0\n", 455 | " for i in range(trials):\n", 456 | " v = nature.challenge()\n", 457 | " c = model.predict(v)\n", 458 | " if nature.response(c)!=True:\n", 459 | " errors += 1\n", 460 | " return errors\n", 461 | "count_errors(nature,model,trials=100)" 462 | ] 463 | }, 464 | { 465 | "cell_type": "markdown", 466 | "id": "0fc4217f", 467 | "metadata": {}, 468 | "source": [ 469 | "OK, that's not very good... the model is wrong about 90 percent of the time--chance level. Let's try for something better.\n", 470 | "\n" 471 | ] 472 | }, 473 | { 474 | "cell_type": "code", 475 | "execution_count": 32, 476 | "id": "83e261ab", 477 | "metadata": { 478 | "collapsed": true 479 | }, 480 | "outputs": [], 481 | "source": [ 482 | "class MemoryModel:\n", 483 | " def __init__(self,dataset):\n", 484 | " self.memory = {}\n", 485 | " for v,c in dataset:\n", 486 | " self.memory[tuple(v)] = c\n", 487 | " def predict(self,v):\n", 488 | " return self.memory.get(tuple(v),0)\n", 489 | " " 490 | ] 491 | }, 492 | { 493 | "cell_type": "code", 494 | "execution_count": 33, 495 | "id": "e8e07326", 496 | "metadata": { 497 | "collapsed": true 498 | }, 499 | "outputs": [], 500 | "source": [ 501 | "model = MemoryModel(trainingset)\n" 502 | ] 503 | }, 504 | { 505 | "cell_type": "code", 506 | "execution_count": 34, 507 | "id": "f68736d8", 508 | "metadata": { 509 | "collapsed": false 510 | }, 511 | "outputs": [ 512 | { 513 | "data": { 514 | "text/plain": [ 515 | "0" 516 | ] 517 | }, 518 | "execution_count": 34, 519 | "metadata": {}, 520 | "output_type": "execute_result" 521 | } 522 | ], 523 | "source": [ 524 | "count_errors(nature,model,trials=100)\n" 525 | ] 526 | }, 527 | { 528 | "cell_type": "markdown", 529 | "id": "3247437f", 530 | "metadata": {}, 531 | "source": [ 532 | "In the noise free case, memorization of samples may give good response.\n", 533 | "\n", 534 | "It still lacks _generalization_: that is, it can't make good predictions for previously unseen samples that are similar to, but different from, existing samples." 535 | ] 536 | }, 537 | { 538 | "cell_type": "markdown", 539 | "id": "fb24cbad", 540 | "metadata": {}, 541 | "source": [ 542 | "# Noisy Samples" 543 | ] 544 | }, 545 | { 546 | "cell_type": "markdown", 547 | "id": "95b18e36", 548 | "metadata": {}, 549 | "source": [ 550 | "In the above example, the model just generated fixed patterns\n", 551 | "with a one-to-one correspondence to classes.\n", 552 | "\n", 553 | "In practice, there is usually noise.\n", 554 | "\n", 555 | "Noise means:\n", 556 | "\n", 557 | "- multiple patterns correspond to each class\n", 558 | "- each pattern may correspond to multiple classes\n", 559 | "- the correspondences are non-deterministic\n", 560 | "\n", 561 | "The overall goal is to minimize the *error rate*." 562 | ] 563 | }, 564 | { 565 | "cell_type": "code", 566 | "execution_count": 62, 567 | "id": "ae9e20ee", 568 | "metadata": { 569 | "collapsed": true 570 | }, 571 | "outputs": [], 572 | "source": [ 573 | "from pylab import random as arandom\n", 574 | "class NoisySevenSegments(SevenSegments):\n", 575 | " def __init__(self):\n", 576 | " self.p_noise = 0.07\n", 577 | " self.vs = [None] * 10\n", 578 | " self.vs[0] = array((1,1,1,1,1,1,0))\n", 579 | " self.vs[1] = array((0,1,1,0,0,0,0))\n", 580 | " self.vs[2] = array((1,1,0,1,1,0,1))\n", 581 | " self.vs[3] = array((1,1,1,1,0,0,1))\n", 582 | " self.vs[4] = array((0,1,1,0,0,1,1))\n", 583 | " self.vs[5] = array((1,0,1,1,0,1,1))\n", 584 | " self.vs[6] = array((1,0,1,1,1,1,1))\n", 585 | " self.vs[7] = array((1,1,1,0,0,0,0))\n", 586 | " self.vs[8] = array((1,1,1,1,1,1,1))\n", 587 | " self.vs[9] = array((1,1,1,1,0,1,1))" 588 | ] 589 | }, 590 | { 591 | "cell_type": "markdown", 592 | "id": "0dc948f1", 593 | "metadata": {}, 594 | "source": [ 595 | "Now the training samples consist of the true targets, but with some of the segments flipped.\n", 596 | "\n" 597 | ] 598 | }, 599 | { 600 | "cell_type": "code", 601 | "execution_count": 63, 602 | "id": "8b7113e4", 603 | "metadata": { 604 | "collapsed": false 605 | }, 606 | "outputs": [], 607 | "source": [ 608 | "@method(NoisySevenSegments)\n", 609 | "def training_sample(self):\n", 610 | " c = pyrandom.randint(0,len(self.vs)-1)\n", 611 | " v = self.vs[c]\n", 612 | " flip = 1.0*(arandom(size=7)maxs)*1.0/len(maxs)" 328 | ] 329 | }, 330 | { 331 | "cell_type": "markdown", 332 | "id": "a55ccc58", 333 | "metadata": {}, 334 | "source": [ 335 | "Interestingly, you can get much greater extreme value differences without any bias at all, simply by having more variance in one population than in another.\n", 336 | "\n" 337 | ] 338 | }, 339 | { 340 | "cell_type": "code", 341 | "execution_count": 104, 342 | "id": "d4d44a1c", 343 | "metadata": { 344 | "collapsed": false 345 | }, 346 | "outputs": [ 347 | { 348 | "data": { 349 | "text/plain": [ 350 | "0.82399999999999995" 351 | ] 352 | }, 353 | "execution_count": 104, 354 | "metadata": {}, 355 | "output_type": "execute_result" 356 | }, 357 | { 358 | "data": { 359 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD9CAYAAACsq4z3AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHWRJREFUeJzt3X9sU+f9L/D3SQjxKIwfFXG4Tq4Src1KaCD2WDNNcGua\nJgzzTZSMqBpjYAGZENM2tfAP/AFLriaWqUIVZdWEqqwKQ4JWm2Bpl2WFC55pEIm6xFRdf6Rx4TaJ\nYt9SCGqc+Pe5f+R7XCexY/vYjn1O3i8JYY7Pj8/RA58czvN5nkcQRVEEEREpXk6mAyAiotRgQici\nUgkmdCIilWBCJyJSCSZ0IiKVYEInIlKJuBJ6IBCAXq9HXV0dAODBgweoqalBWVkZamtrMT4+ntYg\niYgotrgS+pkzZ1BeXg5BEAAAbW1tqKmpweDgIKqrq9HW1pbWIImIKLaYCX1kZARdXV1obm6GNAap\ns7MTZrMZAGA2m3HlypX0RklERDHFTOgvvfQSXn75ZeTkfLOr0+mEVqsFAGi1WjidzvRFSEREcVky\n35fvvPMOCgoKoNfrYbFYIu4jCELoVUyk74iIKHFyZmWZ9wn91q1b6OzsRGlpKXbv3o3r169j7969\n0Gq1cDgcAICxsTEUFBTMG5Raf/3mN7/JeAy8N94f7099v+SaN6GfOnUKw8PDuHv3Li5duoTnnnsO\nf/7zn1FfX4+Ojg4AQEdHBxoaGmQHQEREqZFQHbr0CuXYsWO4evUqysrKcP36dRw7diwtwRERUfzm\nfYce7tlnn8Wzzz4LAFizZg2uXbuWtqCUwmg0ZjqEtFHzvQG8P6VT+/3JJYjJvLCJdXJBSOp9EBHR\nYiQ3d3LoPxGRSjChExGpBBM6EZFKMKETEakEEzoRkUrEXbZIlAxTowmOhw4Uri5E1+WuTIdDpEp8\nQqcF4XjogO6QDo6HjkyHQqRaTOhERCrBhE5EpBJM6EREKsGETkSkEkzoREQqwYRORKQSTOhERCrB\nhE5EpBJM6EREKsGh/wpgMu2GwzEBACgsXI6urosZjoiIshETugI4HBPQ6d4GAIyO1mU4GiLKVjFf\nubjdblRVVaGyshLl5eU4fvw4AKClpQVFRUXQ6/XQ6/Xo7u5Oe7BERBRdzCd0jUaDGzduYNmyZfD7\n/diyZQvee+89CIKAI0eO4MiRIwsRJxERxRBXp+iyZcsAAF6vF4FAAKtXrwYALgBNRJRF4krowWAQ\nlZWV0Gq12LZtGzZs2AAAOHv2LDZt2oSDBw9ifHw8rYESEdH84uoUzcnJgc1mw6NHj7B9+3ZYLBYc\nPnwYJ0+eBACcOHECR48eRXt7+5xjW1paQp+NRiOMRmNKAiciUguLxQKLxZL0eRKqclm5ciV27tyJ\n999/f0Zibm5uRl1d5OqL8IRORERzzX7YbW1tlXWemK9c7t+/H3qdMjU1hatXr0Kv18Ph+GblmcuX\nL6OiokJWAERElBoxn9DHxsZgNpsRDAYRDAaxd+9eVFdXY9++fbDZbBAEAaWlpTh37txCxEtERFHE\nTOgVFRXo7++fs/38+fNpCYiIiOThSFGKytRoguOhA4WrC9F1uSvT4RBRDJyci6JyPHRAd0gHx0NH\n7J2JKOOY0ImIVIIJnYhIJZjQiYhUggmdiEglWOWyQOQsUiEdY7ffhU6X2HVSuRDG0OAQDEYDALDi\nhSiL8Ql9gUiLVOh0b4cSe7zH+HzBhK8T7zXi4YcfukM6VrwQZTkmdCIilWBCJyJSCSZ0IiKVYKco\nZRVpugGAHbBEiWJCp6wiTTcAAKPnRjMcDZGy8JULEZFKMKETEakEEzoRkUowoRMRqQQ7RSmlEqlS\nkVPRwioYouiY0CmlEqlSkVPRwioYoujmfeXidrtRVVWFyspKlJeX4/jx4wCABw8eoKamBmVlZait\nrcX4+PiCBEtERNHNm9A1Gg1u3LgBm82GDz74ADdu3MB7772HtrY21NTUYHBwENXV1Whra1uoeImI\nKIqYnaLLli0DAHi9XgQCAaxevRqdnZ0wm80AALPZjCtXrqQ3SiIiiinmO/RgMAiDwQC73Y7Dhw9j\nw4YNcDqd0Gq1AACtVgun0xn1+JaWltBno9EIo9GYdNCUWumYQ52I4mexWGCxWJI+T8yEnpOTA5vN\nhkePHmH79u24cePGjO8FQYAgCFGPD0/olJ2kOdRHR+syHQrRojT7Ybe1tVXWeeKuQ1+5ciV27tyJ\nf//739BqtXA4pkvHxsbGUFBQIOviRESUOvMm9Pv374cqWKampnD16lXo9XrU19ejo6MDANDR0YGG\nhob0R0pERPOa95XL2NgYzGYzgsEggsEg9u7di+rqauj1erzwwgtob29HSUkJ3nrrrYWKl4iIopg3\noVdUVKC/v3/O9jVr1uDatWtpC4qIiBLHkaJZSKo6sdvvQqfLdDSZMzQ4BIPRAIDD/Iniwcm5spBU\ndeLzBTMdSkb54YfukA66Q7rQ/C1EFB0TOhGRSjChExGpBBM6EZFKMKETEakEEzphaKQXVpsBn4xa\nYWo0ZTocIpKJZYsEv+DBiiYd4B6F42NWkxApFZ/QiYhUggmdiEglmNCJiFSC79AzYGjoUxgMdUkt\nKDF7UYpMTBdgajSFRnDGOzQ/fDi/3W6HDroZ28O3EVFi+ISeAX7/Euh0b8PhmJB9Dml6AOkcmZgu\nwPHQkfDQ/PDh/L6Ab8728G1ElBgmdCIilWBCJyJSCSZ0IiKVYEInIlIJVrlkETmVKlLFTCqrW0Jx\n/L97cypOolWpRIwtgX1TSaq+4aIYtNjwCT2LyKlUkSpmUlnd8k0cgbnXi1KlEjG2BPZNJan6hoti\n0GITM6EPDw9j27Zt2LBhA55++mm8+uqrAICWlhYUFRVBr9dDr9eju7s77cESEVF0MV+55OXl4ZVX\nXkFlZSUmJibwve99DzU1NRAEAUeOHMGRI0cWIk4iIoohZkIvLCxEYWEhAGD58uVYv349RkdHAQCi\nKKY3OiIiiltCnaL37t3DwMAAfvCDH6Cnpwdnz57F+fPnsXnzZpw+fRqrVq2ac0xLS0vos9FohNFo\nTDZm1UjFFABEpHwWiwUWiyXp88Sd0CcmJtDU1IQzZ85g+fLlOHz4ME6ePAkAOHHiBI4ePYr29vY5\nx4UndJpJ6tAcHa3LdChElEGzH3ZbW1tlnSeuKhefz4ddu3bhZz/7GRoaGgAABQUFEAQBgiCgubkZ\nfX19sgIgIqLUiJnQRVHEwYMHUV5ejhdffDG0fWxsLPT58uXLqKioSE+EREQUl5ivXHp6enDhwgVs\n3LgRer0eAHDq1ClcvHgRNpsNgiCgtLQU586dS3uwREQUXcyEvmXLFgSDcwet7NixIy0BERGRPBz6\nrzCJDvWXhvEDmFNNs7b4f8DldWHKM4EV6Qo4jcKnFuAwfyIO/VecRIf6S8P4Iy2o4fK6sOKn/wsQ\nlDmeIHxqAQ7zJ2JCJyJSDSZ0IiKVYEInIlIJdoqqmKnRhE9Ge/Dh52vxrRXF8HntMDWaAM9KOBwT\n8Hi9czpDhwaH4A/YYf/SAI/PFfXcvb39cLs96O3tR1WVIb03gm86QBdyXnUipeETuoo5HjqQ92MN\nArkuaJp0yPuxBo6HjlBHaaTJ1fzwI+/HGmiadBARvbPU4/FDEPLh8fjTeQsz4lroedWJlIYJnYhI\nJZjQiYhUggmdiEglmNCJiFSCVS5pIg25XyyLV7AKhSjz+ISeJlIlyezh9mrFKhSizGNCJyJSCSZ0\nIiKVYEInIlIJJnQiIpVglUuSlFTN4nK5cKf/AyD3Y9i/NECEb873QbcHPlcuNJoMBUlEsvEJPUlK\nqmYRRQHIWwr81xJomnRzFrYQRQGCkI+gMte7IFr0Yib04eFhbNu2DRs2bMDTTz+NV199FQDw4MED\n1NTUoKysDLW1tRgfH097sEREFF3MhJ6Xl4dXXnkF//nPf3D79m289tpr+Pjjj9HW1oaamhoMDg6i\nuroabW1tCxEvERFFETOhFxYWorKyEgCwfPlyrF+/HqOjo+js7ITZbAYAmM1mXLlyJb2REhHRvBLq\nFL137x4GBgZQVVUFp9MJrVYLANBqtXA6nRGPaWlpCX02Go0wGo2yg6XM83i8sFr7MOmazHQoCTE1\nmkILSReuLkTX5a4MR0T0DYvFAovFkvR54k7oExMT2LVrF86cOYMVK2aucyMIAgRBiHhceEIn5RNF\nERrNM5iYsGQ6lIQ4HjqgOzQ9x8zoudEMR0M00+yH3dbWVlnniavKxefzYdeuXdi7dy8aGhoATD+V\nOxzTTzxjY2MoKCiQFQAREaVGzIQuiiIOHjyI8vJyvPjii6Ht9fX16OjoAAB0dHSEEj0REWVGzFcu\nPT09uHDhAjZu3Ai9Xg8A+N3vfodjx47hhRdeQHt7O0pKSvDWW2+lPVgiIoouZkLfsmULgsFgxO+u\nXbuW8oCIiEgeDv1PkaGhT2Ew1MmaAkA61m6/Cx3Xhkg7aTGOWNUurIwhpeHQ/xTx+5fIngJAOtbn\ni/w/IUotaTEOKVlHI1XGxLMvUTZgQiciUgkmdCIilWBCJyJSCXaKkipIHZ0AYLfboQN7l2nx4RM6\nqYLU0ak7pIMv4It9AJEKMaETEakEEzoRkUowoRMRqQQTOhGRSrDKRSaTaTccjomsHa4/NPQp/N9y\nwvc/OfqUaLHgE7pMDsdEVg/X9/uXIG/pdxAUMx0JES0UJnQiIpVgQiciUgkmdCIilWBCJyJSCSb0\nFAtfrCLSn1PFI3wBq80A99LP8c+ba+Fe+jmsNgM8whcpu4YoBmG19sHt9qC3tz9l5yWi9GBCT7HZ\ni1Wka/EKMc8PTZMOQl0eArkuCHV50DTpIOb5U3gVARrNMxCEfHg8qTwvEaVDzIR+4MABaLVaVFRU\nhLa1tLSgqKgIer0eer0e3d3daQ2SiIhii5nQ9+/fPydhC4KAI0eOYGBgAAMDA/jRj36UtgCJiCg+\nMRP61q1bsXr16jnbRZEjVoiIsonsof9nz57F+fPnsXnzZpw+fRqrVq2KuF9LS0vos9FohNFolHtJ\nioPo94Y6S72T+ZkOh4jiYLFYYLFYkj6PrIR++PBhnDx5EgBw4sQJHD16FO3t7RH3DU/otADyAE2T\nDhP3P4PY5QOQm+mIiCiG2Q+7ra2tss4jq8qloKAAgiBAEAQ0Nzejr69P1sWJiCh1ZCX0sbGx0OfL\nly/PqIAhIqLMiPnKZffu3fjXv/6F+/fvo7i4GK2trbBYLLDZbBAEAaWlpTh37txCxEpERPOImdAv\nXrw4Z9uBAwfSEgwREcnHBS7CSItWFBYuR1fXxZjbFxOXywWrtU9R5apDg0MwGA0AALvdDh3iW4nE\n1GiC46FjxjHh5ypcXYiuy13pCZooCRz6H0ZatMLhmIhr+2IiitPTACiJH37oDumgO6SDL+CL+zjH\nQ8ecY8LP5XjoSEe4REljQiciUgkmdCIilWBCJyJSCXaKUtbp7e2Hx+MPzcNeVWVIy3XkdnSyg5Sy\nFRM6ZR2Pxw+N5hlMCP8nrfOwSx2dADB6bjTtxxGlG1+5EBGpBBM6EZFKMKETEakEEzoRkUqwUzRL\n9d4xwb30c1htBniELzIdTkwejxdWax88Hu+c73p7+0MVK4uVNJ0AwMoYSh8+oWcpj+iAUJcHTZMO\nYl76Kj1SRRRFaDTPRJzrxePxQxDy01qxku2k6QQ4dQClExM6EZFKMKETEakEEzoRkUqwUzQBQ0Of\nwmCoAwDY7Xehi2967aSJfi+stumh5kroII3G5XIhmObh/AtNmgaAHZ2UDfiEngC/fwl0ureh070N\nny+4cBfOAzRNOsV0kEYjioLqOkelaQDY0UnZgAmdiEglYib0AwcOQKvVoqKiIrTtwYMHqKmpQVlZ\nGWprazE+Pp7WIImIKLaYCX3//v3o7u6esa2trQ01NTUYHBxEdXU12tra0hYgERHFJ2ZC37p1K1av\nXj1jW2dnJ8xmMwDAbDbjypUr6YmOiIjiJqvKxel0QqvVAgC0Wi2cTmfUfVtaWkKfjUYjjEajnEuS\nQohiEFZrH9xTk/C5PbBa+zDpmsx0WERZzWKxwGKxJH2epMsWBUGAIAhRvw9P6LQYCNOLU0xYIAj5\noc9EFN3sh93W1lZZ55FV5aLVauFwTJdpjY2NoaCgQNbFiYgodWQl9Pr6enR0dAAAOjo60NDQkNKg\niIgocTET+u7du/HDH/4Qn376KYqLi/HGG2/g2LFjuHr1KsrKynD9+nUcO3ZsIWIlIqJ5xHyHfvHi\nxYjbr127lvJgiIhIPs7lQklxuVywWvsizoOeKGkhDJ8rFxrNzPNHWjhjMeDCGJQIDv2npIjidFVL\nKkgLYQTDfjZI50/FDwwl4sIYlAgmdCIilWBCJyJSCSZ0IiKVYEKPQFrIwmTanelQ6L9JUwoorXPU\n1GiCwWiA3W7PdCi0CDChRyAtZOFwTGQ6FApRZueo1KnpC/gyHQotAkzoREQqwYRORKQSTOhERCrB\nhE5EpBIc+p9leu+Y4F76ObyT+RG/F/1eWG0GiFBuJ5s0nH9qyg2rtQ/5+Zn/azg0OASD0QAAsNvt\n0EEn+3gO0adM4RN6lvGIDgh1edETdh6gadIBgrKqPcKFTxeg0TwDj8ef4YgAP/yhIfZyKlLCj+cQ\nfcoUJnQiIpVgQiciUgkmdCIilch8b1QWMJl2w+GYgN1+F7qwvjBpCoDZ2ynzpLnTe3v7UVVlSOhY\nj8eb1mkEku1gDZ8DPdnj2UG7uPAJHYDDMQGd7m34fMEZ26UpAGZvp8yT5k6X06EqimJapxFItoM1\nfA70ZI9nB+3iktQTeklJCb797W8jNzcXeXl56OvrS1VcRESUoKQSuiAIsFgsWLNmTariISIimZJ+\n5aK02e+IiNQqqYQuCAKef/55bN68Ga+//nqqYiIiIhmSeuXS09ODdevW4csvv0RNTQ2eeuopbN26\ndcY+LS0toc9GoxFGozGZSyZNqmgBgOHhIRQXP5HxKhaP8AWsNsO8Q/7VzOVyIej2wOfKjbmvtNDF\npGtyxnapciWRaQSkcy301APhVTDDd4dRXFoc+h2QV9lCymaxWGCxWJI+T1J/k9etWwcAWLt2LRob\nG9HX1zdvQs8GUkULAHzySTl0urfxySflGY1JzPND06TDxP3PIHb5AMRObGoiigIEIR/BuN7eTU8b\nMDFhmXWO6coVtzuRjnlBxjHJk6pgAOCTo59Ad0gX+l3aRovL7Ifd1tZWWeeR/cplcnISX3/9NYDp\nJ6x3330XFRUVck9HRERJkv2E7nQ60djYCADw+/3Ys2cPamtrUxYYERElRnZCLy0thc1mS2UsRESU\nBA79X2DSfOe9d0yZDkWVpA7WdA7tzxSpMzXacH5pyP9CdKpK1+LUAtmFQ/8XmDTfuUfkkOx0kDpY\n0zm0P1OkztRow/mlIf9ypgtIlHQtTi2QXZjQiYhUggmdiEglmNCJiFSCCZ2ISCUWTZVLtEUsMsX1\n9RCCS72w2gzRF4SmmFwuF6zWPtV1gM4n0tQBQPxTBnABDPVaNE/o0RaxyBQx1w+hLg+aJh0gLJ5k\nlGqiOD18fzEJX0DD5XMlvBgGF8BQr0WT0ImI1I4JnYhIJZjQiYhUggmdiEglVFPlIlWxFBYuR1fX\nxTnbs6W6hRaOtICFe2oSPrcH7/7TAs23ls2piJHmf+nt7UdVlWHGd729/fB4/FHnhZEW1nCHHR/a\nFnZdX5Tzp1ukipjwaphoFTPS50xVwbASRx7VPKFLVSzSakSzt2dLdQstpOkKGH8AEIR8+AOIWBEj\nzf/i8fjnfOfx+OedF0ZaWCP8eGlb+HWjnT/dIlXEhFfDRKuYkT5nqgqGlTjyqCahExEtdkzoREQq\nobh36MFgEB9++CF8Ph+WL1+O7373uwtyXVEUEQy6MT7+7//+88L/95mIaD6KS+hjY2Nobv7fyM0t\nR07OB+jpuTLj+6GhT2Ew1M3pHE2W3/8I3vwxfDj1KwTdU/Dk/F9YbYaoi1V4hC9C3//z5lr4l34N\nq80A16Q9ZTFRaknTCLjDOjJ9rlxoNN/s09vbH/o+UgdrPOefb+GN2eeX9g3fHq2DVeqMnZpyw2rt\nQ35+7H/esztwZ583kWkGInVkhm/Lpk7XdMtUp67iXrmIoojc3LV4/PEWuN1zOzr9/iURO0dTcV3k\nA0u3Pg7hyaUQ8wLQNOmiLlYh5vlD3wdyXaFh/py3JXtJ0wiEd2QGZ+Vrj8c/bwdrPOef74fA7PNL\n+4Zvj9bBKnXGAtPHxtMJO7sDd/YxiUwzEKkjM3xbNnW6plumOnWTSujd3d146qmn8OSTT+L3v/99\nqmJSDO/IV5kOIW3UfG8A70/pLBZLpkPISrITeiAQwC9/+Ut0d3fjo48+wsWLF/Hxxx+nMras51Px\nPxo13xvA+1M6JvTIZCf0vr4+PPHEEygpKUFeXh5+8pOf4G9/+1sqYyMiogTITuijo6MoLi4O/bmo\nqAijo6MpCWo+giAgELiPr776LfLzF64LQBAEwAt4e+5DHPIAEBbs2kRE8RBEmSsD/PWvf0V3dzde\nf/11AMCFCxfQ29uLs2fPfnNygUmPiEgOOalZdtmiTqfD8PBw6M/Dw8MoKipKOiAiIpJH9juLzZs3\n47PPPsO9e/fg9Xrx5ptvor6+PpWxERFRAmQ/oS9ZsgR/+MMfsH37dgQCARw8eBDr169PZWxERJSA\npHoVd+zYgTt37uDxxx/Hm2++ifLychw/fjzivr/+9a/x5JNPYtOmTRgYGEjmsgvG7XajqqoKlZWV\nUe/NYrFg5cqV0Ov10Ov1+O1vf5uBSJMTCASg1+tRV1cX8Xsltl24+e5P6e1XUlKCjRs3Qq/X45ln\nIg90UnL7xbo/Jbff+Pg4mpqasH79epSXl+P27dtz9km47cQUcLlcoiiKos/nE6uqqsSbN2/O+P7v\nf/+7uGPHDlEURfH27dtiVVVVKi67IGLd240bN8S6urpMhJYyp0+fFn/6059GvA8lt51kvvtTevuV\nlJSIX331VdTvld5+se5Pye23b98+sb29XRTF6fwyPj4+43s5bZeSur9ly5YBALxeLwKBANasWTPj\n+87OTpjNZgBAVVUVxsfH4XQ6U3HptIt1b4CyO39HRkbQ1dWF5ubmiPeh5LYDYt8foOz2A+aPX+nt\nB8RuHyW236NHj3Dz5k0cOHAAwPQr7JUrV87YR07bpSShB4NBVFZWQqvVYtu2bSgvL5/xfaSa9ZGR\nkVRcOu1i3ZsgCLh16xY2bdoEk8mEjz76KEORyvPSSy/h5ZdfRk5O5L8KSm47IPb9Kb39BEHA888/\nj82bN4dKiMMpvf1i3Z9S2+/u3btYu3Yt9u/fD4PBgJ///OeYnJycsY+ctktJQs/JyYHNZsPIyAis\nVmvEYbmzf4oqpUY91r0ZDAYMDw/jzp07+NWvfoWGhobMBCrDO++8g4KCAuj1+nmfcpTadvHcn5Lb\nDwB6enowMDCAf/zjH3jttddw8+bNOfsotf2A2Pen1Pbz+/3o7+/HL37xC/T39+Oxxx5DW1vbnP0S\nbbuUDrVcuXIldu7ciffff3/G9tk16yMjI9ApbIHPaPe2YsWK0GuZHTt2wOfz4cGDB5kIMWG3bt1C\nZ2cnSktLsXv3bly/fh379u2bsY+S2y6e+1Ny+wHAunXrAABr165FY2Mj+vr6Znyv5PYDYt+fUtuv\nqKgIRUVF+P73vw8AaGpqQn9//4x95LRd0gn9/v37GB8fBwBMTU3h6tWr0Ov1M/apr6/H+fPnAQC3\nb9/GqlWroNVqk7102sVzb06nM/RTtK+vD6IoRnzPno1OnTqF4eFh3L17F5cuXcJzzz0XaieJUtsO\niO/+lNx+k5OT+PrrrwFMz7X+7rvvoqKiYsY+Sm6/eO5Pqe1XWFiI4uJiDA4OAgCuXbuGDRs2zNhH\nTtslvcDF2NgYzGYzgsEggsEg9u7di+rqapw7dw4AcOjQIZhMJnR1deGJJ57AY489hjfeeCPZyy6I\neO7tL3/5C/74xz9iyZIlWLZsGS5dupThqOWT/junhraLJNL9Kbn9nE4nGhsbAUz/F37Pnj2ora1V\nTfvFc39Kbr+zZ89iz5498Hq9+M53voM//elPSbed7LlciIgouyhuxSIiIoqMCZ2ISCWY0ImIVIIJ\nnYhIJZjQiYhUggmdiEgl/j9qBdnj41BqNwAAAABJRU5ErkJggg==\n" 360 | }, 361 | "metadata": {}, 362 | "output_type": "display_data" 363 | } 364 | ], 365 | "source": [ 366 | "maxs = array([amax(randn(10000)) for i in range(1000)])\n", 367 | "max2s = array([amax(1.1*randn(10000)) for i in range(1000)])\n", 368 | "_=hist(maxs,bins=100,alpha=0.7)\n", 369 | "_=hist(max2s,bins=100,alpha=0.7)\n", 370 | "sum(max2s>maxs)*1.0/len(maxs)" 371 | ] 372 | }, 373 | { 374 | "cell_type": "markdown", 375 | "id": "7de893f1", 376 | "metadata": {}, 377 | "source": [ 378 | "# Conditional Distributions" 379 | ] 380 | }, 381 | { 382 | "cell_type": "markdown", 383 | "id": "641f47be", 384 | "metadata": {}, 385 | "source": [ 386 | "Let's look at another important example from statistics. Consider a population of size $N = 1000000$:\n", 387 | "\n" 388 | ] 389 | }, 390 | { 391 | "cell_type": "code", 392 | "execution_count": 105, 393 | "id": "072924e0", 394 | "metadata": { 395 | "collapsed": false 396 | }, 397 | "outputs": [], 398 | "source": [ 399 | "N = 1000000" 400 | ] 401 | }, 402 | { 403 | "cell_type": "markdown", 404 | "id": "994cab7b", 405 | "metadata": {}, 406 | "source": [ 407 | "Now assume that about 1:1000 of the population are infected with some disease.\n", 408 | "\n" 409 | ] 410 | }, 411 | { 412 | "cell_type": "code", 413 | "execution_count": 106, 414 | "id": "86e52b79", 415 | "metadata": { 416 | "collapsed": false 417 | }, 418 | "outputs": [ 419 | { 420 | "data": { 421 | "text/plain": [ 422 | "1045" 423 | ] 424 | }, 425 | "execution_count": 106, 426 | "metadata": {}, 427 | "output_type": "execute_result" 428 | } 429 | ], 430 | "source": [ 431 | "infected = (rand(N)<1e-3)\n", 432 | "sum(infected)" 433 | ] 434 | }, 435 | { 436 | "cell_type": "markdown", 437 | "id": "a2d3b004", 438 | "metadata": {}, 439 | "source": [ 440 | "Also, assume we have a test that is 99 percent accurate. We give the test to the entire population and measure the results.\n", 441 | "\n" 442 | ] 443 | }, 444 | { 445 | "cell_type": "code", 446 | "execution_count": 107, 447 | "id": "79816e05", 448 | "metadata": { 449 | "collapsed": false 450 | }, 451 | "outputs": [ 452 | { 453 | "data": { 454 | "text/plain": [ 455 | "11120" 456 | ] 457 | }, 458 | "execution_count": 107, 459 | "metadata": {}, 460 | "output_type": "execute_result" 461 | } 462 | ], 463 | "source": [ 464 | "testresult = where(rand(N)<1e-2,1-infected,infected)\n", 465 | "sum(testresult)" 466 | ] 467 | }, 468 | { 469 | "cell_type": "markdown", 470 | "id": "03291ca7", 471 | "metadata": {}, 472 | "source": [ 473 | "Now, let's look at the population of people whose test result is positive.\n", 474 | "What fraction of those people are infected?\n", 475 | "\n" 476 | ] 477 | }, 478 | { 479 | "cell_type": "code", 480 | "execution_count": 108, 481 | "id": "371eb20b", 482 | "metadata": { 483 | "collapsed": false 484 | }, 485 | "outputs": [ 486 | { 487 | "data": { 488 | "text/plain": [ 489 | "0.092985611510791363" 490 | ] 491 | }, 492 | "execution_count": 108, 493 | "metadata": {}, 494 | "output_type": "execute_result" 495 | } 496 | ], 497 | "source": [ 498 | "sum(infected[testresult==True])*1.0/sum(testresult==True)" 499 | ] 500 | }, 501 | { 502 | "cell_type": "markdown", 503 | "id": "c201f550", 504 | "metadata": {}, 505 | "source": [ 506 | "(Implications)\n", 507 | "\n", 508 | "That's not good: less than 10 percent of the population that tests positive is actually infected. Ninety percent of those who test positive are _false positives_ even though the test is 99 percent accurate.\n", 509 | "\n", 510 | "This is a common issue in medicine, anti-terrorism, and many other problems:\n", 511 | "\n", 512 | "- testing everybody for HIV infection\n", 513 | "- testing everybody for prostate cancer / breast cancer\n", 514 | "- widespread Internet surveillance for \"terrorism\"\n", 515 | "\n", 516 | "All of these end up generating enormous numbers of false positives." 517 | ] 518 | }, 519 | { 520 | "cell_type": "markdown", 521 | "id": "4b372949", 522 | "metadata": {}, 523 | "source": [ 524 | "(Bayesian Calculation)\n", 525 | "\n", 526 | "Mathematically, we write this in terms of _conditional probabilities_:\n", 527 | "\n", 528 | "prior probability: \n", 529 | "\n", 530 | "$$P(i)$$\n", 531 | "\n", 532 | "probability of test: \n", 533 | "\n", 534 | "$$P(t=0|i=0) = 0.99$$\n", 535 | "$$P(t=1|i=1) = 0.99$$\n", 536 | "\n", 537 | "Bayes formula:\n", 538 | "\n", 539 | "$$ P(i=1|t=1) = \\frac{P(t=1|i=1) P(i=1)}{P(t=1)} = \\frac{P(t=1|i=1) P(i=1)}{P(t=1|i=0) P(i=0) + P(t=1|i=1) P(i=1) } $$\n", 540 | "\n", 541 | "$$ ~~~~~~ = \\frac{0.99 \\cdot 0.001}{ 0.01 \\cdot 0.999 + 0.99 \\cdot 0.001 } \\approx \\frac{0.001}{0.01} = 0.1 $$\n", 542 | "\n" 543 | ] 544 | }, 545 | { 546 | "cell_type": "markdown", 547 | "id": "75b760ee", 548 | "metadata": {}, 549 | "source": [ 550 | "(Derivation of Bayes Formula)\n", 551 | "\n", 552 | "Joint density ($c$ and $x$ could be any variables):\n", 553 | "\n", 554 | "$$ p(c,x) $$\n", 555 | "\n", 556 | "Conditional density:\n", 557 | "\n", 558 | "$$ p(c,x) = P(c|x) ~ p(x) $$\n", 559 | "\n", 560 | "Bayes formula:\n", 561 | "\n", 562 | "$$ P(c|x) ~ p(x) ~ = ~ p(x|c) ~ P(c) $$\n", 563 | "\n", 564 | "$$ P(c|x) = \\frac{p(x|c) P(c)}{p(x)} $$" 565 | ] 566 | }, 567 | { 568 | "cell_type": "code", 569 | "execution_count": null, 570 | "id": "275dfb26", 571 | "metadata": { 572 | "collapsed": false 573 | }, 574 | "outputs": [], 575 | "source": [] 576 | } 577 | ], 578 | "metadata": {}, 579 | "nbformat": 4, 580 | "nbformat_minor": 5 581 | } 582 | -------------------------------------------------------------------------------- /180-markov-models.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "id": "60318732", 7 | "metadata": { 8 | "collapsed": true 9 | }, 10 | "outputs": [], 11 | "source": [ 12 | "from pylab import *" 13 | ] 14 | }, 15 | { 16 | "cell_type": "markdown", 17 | "id": "6ba6bd25", 18 | "metadata": {}, 19 | "source": [ 20 | "Markov Chains\n", 21 | "=============" 22 | ] 23 | }, 24 | { 25 | "cell_type": "markdown", 26 | "id": "633f59d2", 27 | "metadata": {}, 28 | "source": [ 29 | "Assume we have a state space consisting of $N$ states $0...N-1$.\n", 30 | "\n", 31 | "A _Markov chain_ is an infinite sequence of random states $X_i$ such that\n", 32 | "\n", 33 | "$$ P(X_{n+1}= x | X_1=x_1 ... X_n=x_n) = P(X_{n+1} =x | X_n=x_n) $$\n", 34 | "\n", 35 | "That is, the probability of each subsequent state only depends on the previous state." 36 | ] 37 | }, 38 | { 39 | "cell_type": "markdown", 40 | "id": "1f86e523", 41 | "metadata": {}, 42 | "source": [ 43 | "There are many variations:\n", 44 | "\n", 45 | "- usually, the probability is independent of $n$ (_time homogeneous_)\n", 46 | "- the probability can depend on a bounded number of previous states giving a higher order Markov chain\n", 47 | "- the set of states can be countably infinite\n", 48 | "- the state index $n$ can be continuous (continuous time Markov process)" 49 | ] 50 | }, 51 | { 52 | "cell_type": "markdown", 53 | "id": "bf0f8e97", 54 | "metadata": {}, 55 | "source": [ 56 | "We represent time homogeneous Markov chains with a finite state space as a matrix of transition probabilities $M_{ij}$.\n", 57 | "\n", 58 | "Here, $P(j\\rightarrow i) = M_{ij}$ and $\\sum_i M_{ij} = 1$." 59 | ] 60 | }, 61 | { 62 | "cell_type": "code", 63 | "execution_count": 50, 64 | "id": "561d1db0", 65 | "metadata": { 66 | "collapsed": true 67 | }, 68 | "outputs": [], 69 | "source": [ 70 | "M = rand(5,5)" 71 | ] 72 | }, 73 | { 74 | "cell_type": "code", 75 | "execution_count": 51, 76 | "id": "9e5b6238", 77 | "metadata": { 78 | "collapsed": false 79 | }, 80 | "outputs": [ 81 | { 82 | "data": { 83 | "text/plain": [ 84 | "array([ 1.85666173, 0.66606344, 2.39313239, 2.00524954, 3.69764657])" 85 | ] 86 | }, 87 | "execution_count": 51, 88 | "metadata": {}, 89 | "output_type": "execute_result" 90 | } 91 | ], 92 | "source": [ 93 | "sum(M,axis=0)" 94 | ] 95 | }, 96 | { 97 | "cell_type": "code", 98 | "execution_count": 82, 99 | "id": "646f87a0", 100 | "metadata": { 101 | "collapsed": false 102 | }, 103 | "outputs": [ 104 | { 105 | "data": { 106 | "text/plain": [ 107 | "array([ 1., 1., 1., 1., 1.])" 108 | ] 109 | }, 110 | "execution_count": 82, 111 | "metadata": {}, 112 | "output_type": "execute_result" 113 | } 114 | ], 115 | "source": [ 116 | "M /= sum(M,axis=0)[newaxis,:]\n", 117 | "sum(M,axis=0)" 118 | ] 119 | }, 120 | { 121 | "cell_type": "markdown", 122 | "id": "21371379", 123 | "metadata": {}, 124 | "source": [ 125 | "We call $M$ a _transition matrix_ or a _stochastic matrix_.\n", 126 | "\n", 127 | "(Note that the matrix is often written the other way around in the literature.)" 128 | ] 129 | }, 130 | { 131 | "cell_type": "markdown", 132 | "id": "7dbc4dcd", 133 | "metadata": {}, 134 | "source": [ 135 | "We often draw a transition diagram; this shows the state and the probabilities of transitions between them:\n", 136 | "\n", 137 | "![transitions](http://upload.wikimedia.org/wikipedia/commons/thumb/2/2b/Markovkate_01.svg/220px-Markovkate_01.svg.png)\n", 138 | "\n", 139 | "![transitions](http://upload.wikimedia.org/wikipedia/commons/thumb/4/47/MarkovChain1.png/200px-MarkovChain1.png)" 140 | ] 141 | }, 142 | { 143 | "cell_type": "markdown", 144 | "id": "c41e181b", 145 | "metadata": {}, 146 | "source": [ 147 | "We can compute the probability of being in the next state given that we are in state $0$ with a matrix multiplication." 148 | ] 149 | }, 150 | { 151 | "cell_type": "code", 152 | "execution_count": 83, 153 | "id": "46d4b479", 154 | "metadata": { 155 | "collapsed": false 156 | }, 157 | "outputs": [ 158 | { 159 | "data": { 160 | "text/plain": [ 161 | "array([ 0.01215787, 0.40654973, 0.16527515, 0.36084789, 0.05516936])" 162 | ] 163 | }, 164 | "execution_count": 83, 165 | "metadata": {}, 166 | "output_type": "execute_result" 167 | } 168 | ], 169 | "source": [ 170 | "state = zeros(5)\n", 171 | "state[0] = 1\n", 172 | "dot (M,state)" 173 | ] 174 | }, 175 | { 176 | "cell_type": "markdown", 177 | "id": "cfd0bc71", 178 | "metadata": {}, 179 | "source": [ 180 | "Sampling from a Markov Chain\n", 181 | "=============================" 182 | ] 183 | }, 184 | { 185 | "cell_type": "code", 186 | "execution_count": 53, 187 | "id": "009e73da", 188 | "metadata": { 189 | "collapsed": true 190 | }, 191 | "outputs": [], 192 | "source": [ 193 | "v = add.accumulate(M[:,0])" 194 | ] 195 | }, 196 | { 197 | "cell_type": "code", 198 | "execution_count": 54, 199 | "id": "e5d48573", 200 | "metadata": { 201 | "collapsed": true 202 | }, 203 | "outputs": [], 204 | "source": [ 205 | "def binsearch(a,x,lo,hi):\n", 206 | " while lox: hi = mid\n", 211 | " else: return mid\n", 212 | " assert lo==hi\n", 213 | " return lo" 214 | ] 215 | }, 216 | { 217 | "cell_type": "code", 218 | "execution_count": 55, 219 | "id": "dbd8fd85", 220 | "metadata": { 221 | "collapsed": false 222 | }, 223 | "outputs": [ 224 | { 225 | "data": { 226 | "text/plain": [ 227 | "3" 228 | ] 229 | }, 230 | "execution_count": 55, 231 | "metadata": {}, 232 | "output_type": "execute_result" 233 | } 234 | ], 235 | "source": [ 236 | "binsearch(v,0.9,0,len(M))" 237 | ] 238 | }, 239 | { 240 | "cell_type": "code", 241 | "execution_count": 75, 242 | "id": "d0572dc1", 243 | "metadata": { 244 | "collapsed": true 245 | }, 246 | "outputs": [], 247 | "source": [ 248 | "def rsample(dist):\n", 249 | " assert amin(dist)>=0.0 and amax(dist)<=1.0\n", 250 | " v = add.accumulate(dist)\n", 251 | " assert abs(v[-1]-1.0)<1e-6\n", 252 | " val = rand()\n", 253 | " result = binsearch(v,val,0,len(v))\n", 254 | " return result\n", 255 | " " 256 | ] 257 | }, 258 | { 259 | "cell_type": "code", 260 | "execution_count": 76, 261 | "id": "b41e66a6", 262 | "metadata": { 263 | "collapsed": true 264 | }, 265 | "outputs": [], 266 | "source": [ 267 | "def sample_chain(M,state,N):\n", 268 | " result = [state]\n", 269 | " for i in range(N):\n", 270 | " state = rsample(M[:,state])\n", 271 | " result.append(state)\n", 272 | " return result" 273 | ] 274 | }, 275 | { 276 | "cell_type": "code", 277 | "execution_count": 79, 278 | "id": "661b250c", 279 | "metadata": { 280 | "collapsed": false 281 | }, 282 | "outputs": [ 283 | { 284 | "data": { 285 | "text/plain": [ 286 | "[0, 4, 3, 4, 3, 2, 0, 2, 2, 4, 1]" 287 | ] 288 | }, 289 | "execution_count": 79, 290 | "metadata": {}, 291 | "output_type": "execute_result" 292 | } 293 | ], 294 | "source": [ 295 | "sample_chain(M,0,10)" 296 | ] 297 | }, 298 | { 299 | "cell_type": "code", 300 | "execution_count": 59, 301 | "id": "70c968d3", 302 | "metadata": { 303 | "collapsed": false 304 | }, 305 | "outputs": [ 306 | { 307 | "data": { 308 | "text/plain": [ 309 | "(5, 5)" 310 | ] 311 | }, 312 | "execution_count": 59, 313 | "metadata": {}, 314 | "output_type": "execute_result" 315 | } 316 | ], 317 | "source": [ 318 | "M.shape" 319 | ] 320 | }, 321 | { 322 | "cell_type": "markdown", 323 | "id": "9e5db3ce", 324 | "metadata": {}, 325 | "source": [ 326 | "n-grams and Markov Chains\n", 327 | "==========================\n", 328 | "\n", 329 | "An $n$-gram is a sequence of $n$ letters, syllables, words or other linguistic entity (usually words).\n", 330 | "\n", 331 | "An $n$-gram model is a model of the probability that $n$ words occur.\n", 332 | "\n", 333 | "Usually, $n$-gram models are expressed as a conditional probability, given a very long sequence\n", 334 | "of words $w_1 ... w_L$:\n", 335 | "\n", 336 | "$$P(w_i | w_(i-1)... w_i-(n-1)$$\n", 337 | "\n", 338 | "Sometimes, it might alwo be expressed as a joint probability:\n", 339 | "\n", 340 | "$$P(w_i ... w_i-(n-1)$$\n", 341 | "\n", 342 | "(Boundary conditions are handled by adding special symbols to \"fill things up\".)\n", 343 | "\n", 344 | "An $n$-gram model defines a Markov chain.\n", 345 | "\n", 346 | "Google has released a large $n$-gram model for English and other languages, and you can experiment with it here:\n", 347 | "\n", 348 | "http://books.google.com/ngrams/" 349 | ] 350 | }, 351 | { 352 | "cell_type": "code", 353 | "execution_count": 98, 354 | "id": "17020959", 355 | "metadata": { 356 | "collapsed": true 357 | }, 358 | "outputs": [], 359 | "source": [ 360 | "s = \"\"\"Humpty Dumpty sat on a wall,\n", 361 | "Humpty Dumpty had a great fall;\n", 362 | "All the King's horses and all the King's men\n", 363 | "Couldn't put Humpty together again\"\"\".lower()" 364 | ] 365 | }, 366 | { 367 | "cell_type": "code", 368 | "execution_count": 99, 369 | "id": "1fc7c9a7", 370 | "metadata": { 371 | "collapsed": false 372 | }, 373 | "outputs": [ 374 | { 375 | "data": { 376 | "text/plain": [ 377 | "['humpty', 'dumpty', 'sat', 'on', 'a', 'wall', 'humpty', 'dumpty', 'had', 'a']" 378 | ] 379 | }, 380 | "execution_count": 99, 381 | "metadata": {}, 382 | "output_type": "execute_result" 383 | } 384 | ], 385 | "source": [ 386 | "import re\n", 387 | "words = re.split(r'\\W+',s)\n", 388 | "words[:10]" 389 | ] 390 | }, 391 | { 392 | "cell_type": "code", 393 | "execution_count": 100, 394 | "id": "1818d9ce", 395 | "metadata": { 396 | "collapsed": true 397 | }, 398 | "outputs": [], 399 | "source": [ 400 | "from collections import Counter\n", 401 | "bigrams = Counter()\n", 402 | "for i in range(len(words)-1):\n", 403 | " bigrams[tuple(words[i:i+2])] += 1" 404 | ] 405 | }, 406 | { 407 | "cell_type": "code", 408 | "execution_count": 101, 409 | "id": "d05a1b40", 410 | "metadata": { 411 | "collapsed": false 412 | }, 413 | "outputs": [ 414 | { 415 | "data": { 416 | "text/plain": [ 417 | "Counter({('the', 'king'): 2, ('king', 's'): 2, ('all', 'the'): 2, ('humpty', 'dumpty'): 2, ('horses', 'and'): 1, ('humpty', 'together'): 1, ('couldn', 't'): 1, ('on', 'a'): 1, ('s', 'men'): 1, ('t', 'put'): 1, ('s', 'horses'): 1, ('had', 'a'): 1, ('wall', 'humpty'): 1, ('together', 'again'): 1, ('a', 'wall'): 1, ('a', 'great'): 1, ('men', 'couldn'): 1, ('great', 'fall'): 1, ('put', 'humpty'): 1, ('fall', 'all'): 1, ('dumpty', 'sat'): 1, ('sat', 'on'): 1, ('and', 'all'): 1, ('dumpty', 'had'): 1})" 418 | ] 419 | }, 420 | "execution_count": 101, 421 | "metadata": {}, 422 | "output_type": "execute_result" 423 | } 424 | ], 425 | "source": [ 426 | "bigrams" 427 | ] 428 | }, 429 | { 430 | "cell_type": "code", 431 | "execution_count": 106, 432 | "id": "7dffb429", 433 | "metadata": { 434 | "collapsed": false 435 | }, 436 | "outputs": [ 437 | { 438 | "data": { 439 | "text/plain": [ 440 | "[(('a', 'great'), 1),\n", 441 | " (('a', 'wall'), 1),\n", 442 | " (('all', 'the'), 2),\n", 443 | " (('and', 'all'), 1),\n", 444 | " (('couldn', 't'), 1),\n", 445 | " (('dumpty', 'had'), 1),\n", 446 | " (('dumpty', 'sat'), 1),\n", 447 | " (('fall', 'all'), 1),\n", 448 | " (('great', 'fall'), 1),\n", 449 | " (('had', 'a'), 1),\n", 450 | " (('horses', 'and'), 1),\n", 451 | " (('humpty', 'dumpty'), 2),\n", 452 | " (('humpty', 'together'), 1),\n", 453 | " (('king', 's'), 2),\n", 454 | " (('men', 'couldn'), 1),\n", 455 | " (('on', 'a'), 1),\n", 456 | " (('put', 'humpty'), 1),\n", 457 | " (('s', 'horses'), 1),\n", 458 | " (('s', 'men'), 1),\n", 459 | " (('sat', 'on'), 1),\n", 460 | " (('t', 'put'), 1),\n", 461 | " (('the', 'king'), 2),\n", 462 | " (('together', 'again'), 1),\n", 463 | " (('wall', 'humpty'), 1)]" 464 | ] 465 | }, 466 | "execution_count": 106, 467 | "metadata": {}, 468 | "output_type": "execute_result" 469 | } 470 | ], 471 | "source": [ 472 | "sorted(bigrams.items())" 473 | ] 474 | }, 475 | { 476 | "cell_type": "markdown", 477 | "id": "855a5a9a", 478 | "metadata": {}, 479 | "source": [ 480 | "So, we see that:\n", 481 | "\n", 482 | "$P( \\hbox{great} | \\hbox{a}) = 0.5$\n", 483 | "\n", 484 | "$P( \\hbox{wall} | \\hbox{a}) = 0.5$\n", 485 | "\n", 486 | "$P( \\hbox{(anything else)} | \\hbox{a}) = 0.0$" 487 | ] 488 | }, 489 | { 490 | "cell_type": "markdown", 491 | "id": "745f1fb9", 492 | "metadata": {}, 493 | "source": [ 494 | "Note that there are many pairs of words (even in this restricted vocabulary) that we don't see, even though\n", 495 | "their probability isn't zero.\n", 496 | "\n", 497 | "The process of fixing this is called _smoothing_.\n", 498 | "\n", 499 | "A simple mechanism is _pseudocounts_: here, we assume that anything that hasn't occurred has actually occurred\n", 500 | "some small number of times (1, 0.5, or smaller).\n", 501 | "\n", 502 | "Alternatively, we can compute $n$-grams for different $n$ and linearly interpolate." 503 | ] 504 | }, 505 | { 506 | "cell_type": "markdown", 507 | "id": "a8ff3063", 508 | "metadata": {}, 509 | "source": [ 510 | "Note that $n$-grams for $n>2$ give rise to higher order Markov chains.\n", 511 | "\n", 512 | "We can transform higher order Markov chains into first order Markov chains by associating states\n", 513 | "with each context of $n-1$ words. However, then the emitted symbol differs from the state label." 514 | ] 515 | }, 516 | { 517 | "cell_type": "markdown", 518 | "id": "f97d82dc", 519 | "metadata": {}, 520 | "source": [ 521 | "State Distributions and Steady State\n", 522 | "==============" 523 | ] 524 | }, 525 | { 526 | "cell_type": "markdown", 527 | "id": "9af72a9c", 528 | "metadata": {}, 529 | "source": [ 530 | "Assume we are given a vector of probabilities $p$ of states.\n", 531 | "\n", 532 | "The probability distribution after one state transition is just the product $M\\cdot p$." 533 | ] 534 | }, 535 | { 536 | "cell_type": "code", 537 | "execution_count": 80, 538 | "id": "945c1e5d", 539 | "metadata": { 540 | "collapsed": true 541 | }, 542 | "outputs": [], 543 | "source": [ 544 | "p = zeros(5)\n", 545 | "p[0] = 1.0" 546 | ] 547 | }, 548 | { 549 | "cell_type": "code", 550 | "execution_count": 81, 551 | "id": "069d11ad", 552 | "metadata": { 553 | "collapsed": false 554 | }, 555 | "outputs": [ 556 | { 557 | "name": "stdout", 558 | "output_type": "stream", 559 | "text": [ 560 | "[ 1. 0. 0. 0. 0.]\n", 561 | "[ 0.01215787 0.40654973 0.16527515 0.36084789 0.05516936]\n", 562 | "[ 0.06709196 0.0425824 0.21430012 0.36536528 0.31066025]\n", 563 | "[ 0.11259592 0.11789572 0.24105906 0.240486 0.28796329]\n", 564 | "[ 0.10613046 0.13175719 0.23142557 0.25488324 0.27580354]\n", 565 | "[ 0.10373109 0.12629671 0.23029253 0.26201374 0.27766592]\n", 566 | "[ 0.10424015 0.12572506 0.23096495 0.26074477 0.27832507]\n", 567 | "[ 0.1043648 0.12608833 0.23100254 0.26036384 0.27818048]\n", 568 | "[ 0.1043276 0.12610693 0.23095915 0.26046004 0.27814629]\n", 569 | "[ 0.10432159 0.12608359 0.23095882 0.26047919 0.27815681]" 570 | ] 571 | } 572 | ], 573 | "source": [ 574 | "for i in range(10):\n", 575 | " print p\n", 576 | " p = dot(M,p)" 577 | ] 578 | }, 579 | { 580 | "cell_type": "markdown", 581 | "id": "e6f80e92", 582 | "metadata": {}, 583 | "source": [ 584 | "This should look familiar: after a lot of applications of the matrix, it converges to a _steady state_ distribution.\n", 585 | "\n", 586 | "This steady state distribution is an _eigenvector_ of the matrix." 587 | ] 588 | }, 589 | { 590 | "cell_type": "markdown", 591 | "id": "5aea0eed", 592 | "metadata": {}, 593 | "source": [ 594 | "There are a number of concepts about Markov chains that are important:\n", 595 | "\n", 596 | "- a state $j$ is _accessible_ from a state $i$ if there is some path connecting the two states with non-zero probability\n", 597 | "- two states _communicate_ if they are accessible from each other\n", 598 | "- a Markov chain is _irreducible_ if all states are accessible to each other\n", 599 | "- a state is _transient_ if there is a non-zero probability that we never return to it\n", 600 | "- a state is _recurrent_ if it is not transient\n", 601 | "- a state is _positive recurrent_ if the expected recurrence time is finite\n", 602 | "- a state is _periodic_ with period $k$ if we are guaranteed to return to it exactly every $k$ steps\n", 603 | "- a state is _absorbing_ if there are no transitions out of it\n", 604 | "- a Markov chain is _absorbing_ if every state can reach an absorbing state\n", 605 | "- a state is _ergodic_ if it is positive recurrent and not periodic\n", 606 | "\n", 607 | "Can we construct matrices illustrating these concepts?" 608 | ] 609 | }, 610 | { 611 | "cell_type": "markdown", 612 | "id": "629db931", 613 | "metadata": {}, 614 | "source": [ 615 | "Reversible Markov Chains\n", 616 | "========================\n", 617 | "\n", 618 | "A Markov chain is _reversible_ if there is some probability distribution $p$ such that \n", 619 | "\n", 620 | "$$p \\cdot M = M \\cdot p$$\n", 621 | "\n", 622 | "Note that $p \\cdot M$ is the Markov chain running in reverse.\n", 623 | "\n", 624 | "This means that if $p$ is the _steady state distribution_, \n", 625 | "running the Markov chain forwards or backwards gives the same result.\n", 626 | "\n", 627 | "Reversible Markov chains are very important for modern pattern recognition, statistics,\n", 628 | "physics, and simulations.\n", 629 | "They are used in Gibbs samplers and the Metropolis algorithm for probabilistic decision making." 630 | ] 631 | }, 632 | { 633 | "cell_type": "markdown", 634 | "id": "18ddbefe", 635 | "metadata": {}, 636 | "source": [ 637 | "Hidden Markov Models\n", 638 | "======================\n", 639 | "\n", 640 | "In Markov chains, \n", 641 | "\n", 642 | "- the state sequence is what we are directly interested in\n", 643 | "- we can \"observe\" the state sequence directly\n", 644 | "\n", 645 | "In _Hidden Markov Models_...\n", 646 | "\n", 647 | "- states are associated with actions or observations\n", 648 | "- often we can only observe the actions or observations and want to infer the state\n", 649 | "\n", 650 | "That is, each state $x$ _emits_ one symbol $y$ from a set of $k$ symbols.\n", 651 | "\n", 652 | "Extremely widely used:\n", 653 | "\n", 654 | "- speech recognition: hidden state is part-of-phoneme, observation is acoustic signal\n", 655 | "- handwriting recognition: hidden state is part-of-handwriting, observation is pen position\n", 656 | "- bioinformatics: hidden state is part-of-gene, observation is DNA \"letter\"\n", 657 | "- natural language processing: hidden state is part-of-speech, observation is word\n", 658 | "\n", 659 | "![hmm-example](http://upload.wikimedia.org/wikipedia/commons/thumb/4/43/HMMGraph.svg/400px-HMMGraph.svg.png)" 660 | ] 661 | }, 662 | { 663 | "cell_type": "markdown", 664 | "id": "85d62bd1", 665 | "metadata": {}, 666 | "source": [ 667 | "To specify an HMM, we need:\n", 668 | "\n", 669 | "- sets of states, a symbol alphabet\n", 670 | "- a matrix of transition probabilities between states\n", 671 | "- a matrix of emission probabilities, one for each state" 672 | ] 673 | }, 674 | { 675 | "cell_type": "markdown", 676 | "id": "d2d98b9f", 677 | "metadata": {}, 678 | "source": [ 679 | "HMM Algorithms\n", 680 | "===============\n", 681 | "\n", 682 | "Usually, we are only given a training set of observations (outputs) $Y_1...Y_M$, \n", 683 | "but may also be given a model (state transition matrix, emission probabilities).\n", 684 | "\n", 685 | "There are several different things we may want to compute:\n", 686 | "\n", 687 | "- given a model $M$ and a sequence of observations $Y$\n", 688 | " - determine $P_M(Y)$\n", 689 | " - determine the most likely state sequence $X$ (most likely explanation)\n", 690 | " - determine the probability distribution of the next state (filtering)\n", 691 | " - determine the probability distribution for state $X_k$ (smoothing)\n", 692 | "\n", 693 | "- given a set of training observations $Y$ and constraints on the model\n", 694 | " - find a model that assigns the overall highest likelihood to the training data\n", 695 | "\n", 696 | "Why do we need constraints? Is there a trivial example?" 697 | ] 698 | } 699 | ], 700 | "metadata": {}, 701 | "nbformat": 4, 702 | "nbformat_minor": 5 703 | } 704 | -------------------------------------------------------------------------------- /Figures-intro/bayer-pattern.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/bayer-pattern.png -------------------------------------------------------------------------------- /Figures-intro/bear-illum-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/bear-illum-1.png -------------------------------------------------------------------------------- /Figures-intro/bear-illum-2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/bear-illum-2.png -------------------------------------------------------------------------------- /Figures-intro/bear-illum-3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/bear-illum-3.png -------------------------------------------------------------------------------- /Figures-intro/bear-illum-4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/bear-illum-4.png -------------------------------------------------------------------------------- /Figures-intro/camera-model-2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/camera-model-2.png -------------------------------------------------------------------------------- /Figures-intro/camera-model.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/camera-model.png -------------------------------------------------------------------------------- /Figures-intro/ccd-sensor.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/ccd-sensor.png -------------------------------------------------------------------------------- /Figures-intro/cie-colorspace.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/cie-colorspace.png -------------------------------------------------------------------------------- /Figures-intro/cie-sunlight.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/cie-sunlight.png -------------------------------------------------------------------------------- /Figures-intro/cmyk-equations.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/cmyk-equations.png -------------------------------------------------------------------------------- /Figures-intro/cmyk-formulas.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/cmyk-formulas.png -------------------------------------------------------------------------------- /Figures-intro/cmyk.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/cmyk.png -------------------------------------------------------------------------------- /Figures-intro/color-channels-historical.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/color-channels-historical.png -------------------------------------------------------------------------------- /Figures-intro/color-circle.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/color-circle.png -------------------------------------------------------------------------------- /Figures-intro/color-cube-schematic.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/color-cube-schematic.png -------------------------------------------------------------------------------- /Figures-intro/color-pyramid.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/color-pyramid.png -------------------------------------------------------------------------------- /Figures-intro/color-space-cones.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/color-space-cones.png -------------------------------------------------------------------------------- /Figures-intro/colorcube.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/colorcube.png -------------------------------------------------------------------------------- /Figures-intro/common-rgb.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/common-rgb.png -------------------------------------------------------------------------------- /Figures-intro/cube-color-1-revealed.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/cube-color-1-revealed.png -------------------------------------------------------------------------------- /Figures-intro/cube-color-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/cube-color-1.png -------------------------------------------------------------------------------- /Figures-intro/cube-color-2-revealed.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/cube-color-2-revealed.png -------------------------------------------------------------------------------- /Figures-intro/cube-color-2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/cube-color-2.png -------------------------------------------------------------------------------- /Figures-intro/eye.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/eye.png -------------------------------------------------------------------------------- /Figures-intro/gamut.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/gamut.png -------------------------------------------------------------------------------- /Figures-intro/gimp-a-channel.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/gimp-a-channel.png -------------------------------------------------------------------------------- /Figures-intro/gimp-channel-decompose.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/gimp-channel-decompose.png -------------------------------------------------------------------------------- /Figures-intro/gimp-hsv.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/gimp-hsv.png -------------------------------------------------------------------------------- /Figures-intro/grey-quant-skull-2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/grey-quant-skull-2.png -------------------------------------------------------------------------------- /Figures-intro/grey-quant-skull.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/grey-quant-skull.png -------------------------------------------------------------------------------- /Figures-intro/human-cones.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/human-cones.png -------------------------------------------------------------------------------- /Figures-intro/illustration-level-quant.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/illustration-level-quant.png -------------------------------------------------------------------------------- /Figures-intro/image-formation.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/image-formation.png -------------------------------------------------------------------------------- /Figures-intro/intensity-quantization.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/intensity-quantization.png -------------------------------------------------------------------------------- /Figures-intro/lcd-spectra.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/lcd-spectra.png -------------------------------------------------------------------------------- /Figures-intro/lenslet.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/lenslet.png -------------------------------------------------------------------------------- /Figures-intro/metameric-patch.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/metameric-patch.png -------------------------------------------------------------------------------- /Figures-intro/metamerism.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/metamerism.png -------------------------------------------------------------------------------- /Figures-intro/newton-purple.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/newton-purple.png -------------------------------------------------------------------------------- /Figures-intro/prism.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/prism.png -------------------------------------------------------------------------------- /Figures-intro/reflected-light.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/reflected-light.png -------------------------------------------------------------------------------- /Figures-intro/rgb-color-display.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/rgb-color-display.png -------------------------------------------------------------------------------- /Figures-intro/rgb-pixel-lcd.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/rgb-pixel-lcd.png -------------------------------------------------------------------------------- /Figures-intro/rose-spatial-quant.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/rose-spatial-quant.png -------------------------------------------------------------------------------- /Figures-intro/sensitivity-curves.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/sensitivity-curves.png -------------------------------------------------------------------------------- /Figures-intro/sensitivity.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/sensitivity.png -------------------------------------------------------------------------------- /Figures-intro/shadow-illusion.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/shadow-illusion.png -------------------------------------------------------------------------------- /Figures-intro/spatial-quantization.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/spatial-quantization.png -------------------------------------------------------------------------------- /Figures-intro/spectrum.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/spectrum.png -------------------------------------------------------------------------------- /Figures-intro/transformation-imaging.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/transformation-imaging.png -------------------------------------------------------------------------------- /Figures-intro/visible-spectrum.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures-intro/visible-spectrum.png -------------------------------------------------------------------------------- /Figures/states-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures/states-1.png -------------------------------------------------------------------------------- /Figures/states-1.xcf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures/states-1.xcf -------------------------------------------------------------------------------- /Figures/states-2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures/states-2.png -------------------------------------------------------------------------------- /Figures/states-3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures/states-3.png -------------------------------------------------------------------------------- /Figures/states-4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Figures/states-4.png -------------------------------------------------------------------------------- /HG_HISTORY: -------------------------------------------------------------------------------- 1 | changeset: 24:fe6a06e781af 2 | tag: tip 3 | user: Tom 4 | date: Fri Oct 24 01:22:31 2014 -0700 5 | files: README.md 6 | description: 7 | added README 8 | 9 | 10 | changeset: 23:a0448db6155a 11 | user: Tom 12 | date: Fri Jul 19 14:48:19 2013 +0200 13 | files: 108-bayesian-parameter-estimation.ipynb 110-parametric-gaussian-classifier.ipynb 111-normal-decision-regions.ipynb 118-parametric-gaussian-classifier.ipynb 205-quadratics.ipynb 220-bayesian-parameter-estimation.ipynb lw.tex 14 | description: 15 | bug fixes 16 | 17 | 18 | changeset: 22:d7f13694a2e9 19 | parent: 21:170a07d8441a 20 | parent: 20:12e8f3f69da4 21 | user: Tom 22 | date: Fri Jul 19 12:53:23 2013 +0200 23 | description: 24 | merged 25 | 26 | 27 | changeset: 21:170a07d8441a 28 | parent: 18:3b18cade6654 29 | user: Tom 30 | date: Fri Jul 19 12:53:14 2013 +0200 31 | files: 111-normal-decision-regions.ipynb 125-perceptrons.ipynb 128-linear-to-nonlinear.ipynb tmp.png 32 | description: 33 | updated 34 | 35 | 36 | changeset: 20:12e8f3f69da4 37 | user: tmb@localhost 38 | date: Fri Jul 12 05:38:34 2013 +0200 39 | files: 205-quadratics.ipynb 210-normal-distributions.ipynb 40 | description: 41 | fixes 42 | 43 | 44 | changeset: 19:93ff722b5ba2 45 | user: Tom 46 | date: Thu Jul 11 16:25:58 2013 +0200 47 | files: 205-quadratics.ipynb 210-normal-distributions.ipynb 48 | description: 49 | updated 50 | 51 | 52 | changeset: 18:3b18cade6654 53 | user: Tom 54 | date: Thu Jul 11 10:42:13 2013 +0200 55 | files: 105-simple-probability.ipynb 107-bayesian-decision-theory.ipynb 108-bayesian-parameter-estimation.ipynb 110-parametric-gaussian-classifier.ipynb 111-normal-decision-regions.ipynb 205-quadratics.ipynb 210-normal-distributions.ipynb lw.tex 56 | description: 57 | added notebooks on Gaussian 58 | 59 | 60 | changeset: 17:fb35801ff2df 61 | user: Tom 62 | date: Mon Jul 08 09:43:33 2013 +0200 63 | files: 033-document-normalization.ipynb 035-binary-morphology.ipynb 037-grayscale-morphology.ipynb 042-distance-transform.ipynb 045-labeling.ipynb 070-template-matching.ipynb 101-intro-classification.ipynb 102-nearest-neighbor-and-preproc.ipynb 110-parametric-gaussian-classifier.ipynb 110-perceptrons.ipynb 111-linear-lsq.ipynb 111-normal-decision-regions.ipynb 112-logistic-regression-grad.ipynb 115-linear-to-nonlinear.ipynb 120-parametric-gaussian-classifier.ipynb 121-normal-decision-regions.ipynb 125-perceptrons.ipynb 126-linear-lsq.ipynb 127-logistic-regression-grad.ipynb 128-linear-to-nonlinear.ipynb 64 | description: 65 | cleanup 66 | 67 | 68 | changeset: 16:dff50e2f4d8b 69 | user: Tom 70 | date: Mon Jun 10 00:48:59 2013 +0200 71 | files: 037-grayscale-morphology.ipynb 038-1d-grayscale-morphology.ipynb 042-distance-transform.ipynb 045-labeling.ipynb 049-thresholding.ipynb lw.tex misc/049-thresholding.ipynb 72 | description: 73 | lecture 74 | 75 | 76 | changeset: 15:1f4ce6a32c3c 77 | user: Tom 78 | date: Mon Jun 03 11:51:40 2013 +0200 79 | files: 033-document-normalization.ipynb 035-binary-morphology.ipynb 037-grayscale-morphology.ipynb 045-labeling.ipynb lw.tex 80 | description: 81 | cleanup 82 | 83 | 84 | changeset: 14:d9d1b2e5a56c 85 | user: Tom 86 | date: Mon Jun 03 03:02:29 2013 +0200 87 | files: 030-nonlinear-filters.ipynb 033-document-normalization.ipynb 035-binary-morphology.ipynb 037-grayscale-morphology.ipynb 040-nonlinear-filters.ipynb 042-distance-transform.ipynb Untitled0.ipynb Untitled1.ipynb 88 | description: 89 | cleanup 90 | 91 | 92 | changeset: 13:27a7f82d1bed 93 | user: Tom 94 | date: Mon Jun 03 02:39:32 2013 +0200 95 | files: HW - Frequency Modulation and Encoding.ipynb hw-frequency-modulation.ipynb 96 | description: 97 | cleanup 98 | 99 | 100 | changeset: 12:aa841c9fc6d2 101 | user: Tom 102 | date: Mon Jun 03 02:39:09 2013 +0200 103 | files: 027-image-compression.ipynb 160-fourier-transform.ipynb Frequency Modulation and Encoding.ipynb HW - Frequency Modulation and Encoding.ipynb Untitled0.ipynb Untitled1.ipynb 104 | description: 105 | added JPEG 106 | 107 | 108 | changeset: 11:e41225ce8f98 109 | user: Tom 110 | date: Mon May 27 13:09:14 2013 +0200 111 | files: 020-frequency-domain.ipynb 025-fourier-transform.ipynb 030-template-matching.ipynb 040-nonlinear-filters.ipynb 070-template-matching.ipynb lw.tex 112 | description: 113 | updated for lecture 114 | 115 | 116 | changeset: 10:8ac8f5ba4396 117 | user: Tom 118 | date: Mon May 13 12:54:26 2013 +0200 119 | files: 020-frequency-domain.ipynb 025-fourier-transform.ipynb 160-fourier-transform.ipynb 120 | description: 121 | updated for fourier 122 | 123 | 124 | changeset: 9:830cd5ebeacc 125 | user: Tom 126 | date: Mon May 06 11:44:58 2013 +0200 127 | files: 003-cameras-and-eyes.ipynb 010-linear-filters.ipynb 012-gradient-filters.ipynb 014-anisotropic-filters.ipynb 020-frequency-domain.ipynb 030-template-matching.ipynb hw-edge-detection.ipynb 128 | description: 129 | added more notebooks 130 | 131 | 132 | changeset: 8:184d3e30b14a 133 | user: Tom 134 | date: Mon Apr 15 13:14:51 2013 +0200 135 | files: 000-introduction.ipynb Makefile 136 | description: 137 | small fixes 138 | 139 | 140 | changeset: 7:cf2da5a773ff 141 | user: Tom 142 | date: Mon Apr 15 12:31:02 2013 +0200 143 | files: 000-introduction.ipynb 003-cameras-and-eyes.ipynb 005-images-as-arrays.ipynb 250-color.ipynb Makefile lw.tex 144 | description: 145 | updated for lecture 146 | 147 | 148 | changeset: 6:08b7ba2d39ff 149 | user: Tom 150 | date: Mon Apr 15 11:58:26 2013 +0200 151 | files: 010-linear-filters.ipynb 250-color.ipynb Makefile mkindex 152 | description: 153 | smallish changes 154 | 155 | 156 | changeset: 5:ab433be3c140 157 | user: Tom 158 | date: Wed Apr 10 04:06:31 2013 +0200 159 | files: Makefile lw.pdf lw.tex lw.toc temp.png 160 | description: 161 | cleanup 162 | 163 | 164 | changeset: 4:fd3418e33149 165 | user: Tom 166 | date: Tue Apr 09 03:54:23 2013 +0200 167 | files: Makefile lw.pdf lw.toc temp.png 168 | description: 169 | cleanup 170 | 171 | 172 | changeset: 3:5998c1512a95 173 | user: Tom 174 | date: Tue Apr 09 03:25:54 2013 +0200 175 | files: 200-intro-python.ipynb 250-color.ipynb lw.tex 176 | description: 177 | added color etc 178 | 179 | 180 | changeset: 2:12a50876fb12 181 | user: Tom 182 | date: Tue Apr 09 03:24:41 2013 +0200 183 | files: 010-linear-filters.ipynb 012-gradient-filters.ipynb 014-anisotropic-filters.ipynb 02-images.ppt 020-frequency-domain.ipynb 030-template-matching.ipynb 040-nonlinear-filters.ipynb 042-distance-transform.ipynb 045-labeling.ipynb 049-thresholding.ipynb 101-intro-classification.ipynb 102-nearest-neighbor-and-preproc.ipynb 110-perceptrons.ipynb 111-linear-lsq.ipynb 112-logistic-regression-grad.ipynb 115-linear-to-nonlinear.ipynb 120-parametric-gaussian-classifier.ipynb 121-normal-decision-regions.ipynb 130-kmeans.ipynb 135-simple-pca.ipynb 140-summary-classifiers.ipynb 150-speech-worksheet.ipynb 160-fourier-transform.ipynb 170-dtw.ipynb 180-markov-models.ipynb 185-hmm-ocr.ipynb Figures-intro/bayer-pattern.png Figures-intro/bear-illum-1.png Figures-intro/bear-illum-2.png Figures-intro/bear-illum-3.png Figures-intro/bear-illum-4.png Figures-intro/camera-model-2.png Figures-intro/camera-model.png Figures-intro/ccd-sensor.png Figures-intro/cie-colorspace.png Figures-intro/cie-sunlight.png Figures-intro/cmyk-equations.png Figures-intro/cmyk-formulas.png Figures-intro/cmyk.png Figures-intro/color-channels-historical.png Figures-intro/color-circle.png Figures-intro/color-cube-schematic.png Figures-intro/color-pyramid.png Figures-intro/color-space-cones.png Figures-intro/colorcube.png Figures-intro/common-rgb.png Figures-intro/cube-color-1-revealed.png Figures-intro/cube-color-1.png Figures-intro/cube-color-2-revealed.png Figures-intro/cube-color-2.png Figures-intro/eye.png Figures-intro/gamut.png Figures-intro/gimp-a-channel.png Figures-intro/gimp-channel-decompose.png Figures-intro/gimp-hsv.png Figures-intro/grey-quant-skull-2.png Figures-intro/grey-quant-skull.png Figures-intro/human-cones.png Figures-intro/illustration-level-quant.png Figures-intro/image-formation.png Figures-intro/intensity-quantization.png Figures-intro/lcd-spectra.png Figures-intro/lenslet.png Figures-intro/metameric-patch.png Figures-intro/metamerism.png Figures-intro/newton-purple.png Figures-intro/prism.png Figures-intro/reflected-light.png Figures-intro/rgb-color-display.png Figures-intro/rgb-pixel-lcd.png Figures-intro/rose-spatial-quant.png Figures-intro/sensitivity-curves.png Figures-intro/sensitivity.png Figures-intro/shadow-illusion.png Figures-intro/spatial-quantization.png Figures-intro/spectrum.png Figures-intro/transformation-imaging.png Figures-intro/visible-spectrum.png Makefile im-10-linear-filters.ipynb im-12-gradient-filters.ipynb im-14-anisotropic-filters.ipynb im-20-frequency-domain.ipynb im-30-template-matching.ipynb im-40-nonlinear-filters.ipynb im-42-distance-transform.ipynb im-45-labeling.ipynb im-49-thresholding.ipynb improc.pdf intro-images.odp lw.pdf lw.tex lw.toc markov-models.odp misc/intro-images.odp misc/openfst-misc.odp pr-01-intro-classification.ipynb pr-02-nearest-neighbor-and-preproc.ipynb pr-10-perceptrons.ipynb pr-11-linear-lsq.ipynb pr-12-logistic-regression-grad.ipynb pr-15-linear-to-nonlinear.ipynb pr-20-parametric-gaussian-classifier.ipynb pr-21-normal-decision-regions.ipynb pr-30-kmeans.ipynb pr-35-simple-pca.ipynb pr-40-summary-classifiers.ipynb pr-50-speech-worksheet.ipynb pr-60-fourier-transform.ipynb pr-70-dtw.ipynb pr-80-markov-models.ipynb pr-85-hmm-ocr.ipynb temp.png 184 | description: 185 | cleanup 186 | 187 | 188 | changeset: 1:04bdeb513e6d 189 | user: Tom 190 | date: Tue Apr 09 00:19:55 2013 +0200 191 | files: 02-images.odp Makefile cooley-tukey.ipynb dct-example.ipynb dtw-synchronous.ipynb dtw.ipynb fourier-transform.ipynb hmm-ocr.ipynb im-49-thresholding.ipynb intro-images.odp intro-mlp.ipynb lw.pdf lw.tex markov-models.ipynb misc/cooley-tukey.ipynb misc/dct-example.ipynb misc/dtw-synchronous.ipynb misc/intro-mlp.ipynb misc/noise-generation.ipynb misc/pencil-sketch.ipynb misc/temp-speech-hacks.ipynb misc/temp-speech-worksheet.ipynb noise-generation.ipynb pencil-sketch.ipynb pr-40-summary-classifiers.ipynb pr-50-speech-worksheet.ipynb pr-60-fourier-transform.ipynb pr-70-dtw.ipynb pr-80-markov-models.ipynb pr-85-hmm-ocr.ipynb pr-90-summary-classifiers.ipynb speech-worksheet.ipynb temp-speech-hacks.ipynb temp-speech-worksheet.ipynb thresholding.ipynb 192 | description: 193 | cleanup 194 | 195 | 196 | changeset: 0:0a640016a839 197 | user: Tom 198 | date: Sun Apr 07 10:30:07 2013 +0200 199 | files: 02-images.odp 02-images.ppt Figures/states-1.png Figures/states-1.xcf Figures/states-2.png Figures/states-3.png Figures/states-4.png Initialen.jpg Makefile Professor_X_03.jpg bookpage.jpg bridge.jpg cooley-tukey.ipynb dct-example.ipynb dtw-synchronous.ipynb dtw.ipynb floorplan.png fourier-transform.ipynb hmm-ocr.ipynb im-10-linear-filters.ipynb im-12-gradient-filters.ipynb im-14-anisotropic-filters.ipynb im-20-frequency-domain.ipynb im-30-template-matching.ipynb im-40-nonlinear-filters.ipynb im-42-distance-transform.ipynb im-45-labeling.ipynb imagenoise.jpg improc.pdf intro-mlp.ipynb lexx.jpg lexx.png lexx2.jpg lexx2.png lw.pdf lw.tex lw.toc markov-models.ipynb markov-models.odp mnist.h5 msak0_306.ema msak0_306.epg msak0_306.lab msak0_306.lar msak0_306.wav noise-generation.ipynb ocr-hw/HOMEWORK ocr-hw/c.png ocr-hw/charread.ipynb ocr-hw/charread.py ocr-hw/charread.pyc ocr-hw/t.png ocr-hw/w.png ocr-hw/w.txt pencil-sketch.ipynb pr-01-intro-classification.ipynb pr-02-nearest-neighbor-and-preproc.ipynb pr-10-perceptrons.ipynb pr-11-linear-lsq.ipynb pr-12-logistic-regression-grad.ipynb pr-15-linear-to-nonlinear.ipynb pr-20-parametric-gaussian-classifier.ipynb pr-21-normal-decision-regions.ipynb pr-30-kmeans.ipynb pr-35-simple-pca.ipynb pr-90-summary-classifiers.ipynb river.jpg simple-ocr/small1.png simple-ocr/template-A.png simple-ocr/template-E.png simple-ocr/template-I.png speech-worksheet.ipynb temp-speech-hacks.ipynb temp-speech-worksheet.ipynb temp.png test.wav test2.wav testimage.jpg testimage.png testpage.png testpar.png thresholding.ipynb tmp.png 200 | description: 201 | initial 202 | 203 | 204 | -------------------------------------------------------------------------------- /Initialen.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Initialen.jpg -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | NBS := $(wildcard [012]??-*.ipynb) 2 | 3 | slides: $(NBS:%.ipynb=%-slides.pdf) 4 | 5 | all: $(NBS:%.ipynb=%-slides.pdf) lw.pdf 6 | 7 | public: all 8 | mkindex > index.html 9 | rsync -v index.html lw.pdf $(NBS) $(NBS:%.ipynb=%-slides.pdf) iupr1.cs.uni-kl.de:public_html/lw/. 10 | 11 | 12 | %.tex: %.ipynb 13 | nb2tex -n $^ 14 | 15 | %-slides.pdf: %.ipynb 16 | nb2tex -b $^ 17 | pdflatex -file-line-error -interaction=batchmode $(^:%.ipynb=%-slides.tex) || true 18 | egrep ':[0-9]+:' $(^:%.ipynb=%-slides.log) | uniq 19 | 20 | lw.pdf: lw.tex $(NBS:%.ipynb=%.tex) 21 | pdflatex -file-line-error -interaction=batchmode lw.tex || true 22 | pdflatex -file-line-error -interaction=batchmode lw.tex || true 23 | egrep ':[0-9]+:' $(^:%.ipynb=%.log) | uniq 24 | 25 | clean: 26 | rm -f $(NBS:%.ipynb=%.tex) 27 | rm -f $(NBS:%.ipynb=%-slides.tex) 28 | rm -f $(NBS:%.ipynb=%-slides.pdf) 29 | rm -f *.log *.nav *.snm *.toc *.vrb *.aux *.out 30 | rm -rf _figs 31 | -------------------------------------------------------------------------------- /Professor_X_03.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/Professor_X_03.jpg -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Introduction to Learning and Perception 2 | 3 | ## Introduction 4 | 5 | [Introduction](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/000-introduction.ipynb) 6 | 7 | [Cameras and Eyes](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/003-cameras-and-eyes.ipynb) 8 | 9 | [Images and Arrays](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/005-images-as-arrays.ipynb) 10 | 11 | ## Image Processing 12 | 13 | [Linear Filters](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/010-linear-filters.ipynb) 14 | 15 | [Gradient Filters](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/012-gradient-filters.ipynb) 16 | 17 | [Anisotropic Filters](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/014-anisotropic-filters.ipynb) 18 | 19 | [Frequency Domain](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/020-frequency-domain.ipynb) 20 | 21 | [Fourier Transform](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/025-fourier-transform.ipynb) 22 | 23 | [Image Compression](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/027-image-compression.ipynb) 24 | 25 | [Nonlinear FIlters](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/030-nonlinear-filters.ipynb) 26 | 27 | [Document Normalization](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/033-document-normalization.ipynb) 28 | 29 | [Binary Morphology](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/035-binary-morphology.ipynb) 30 | 31 | [Grayscale Morphology](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/037-grayscale-morphology.ipynb) 32 | 33 | [Grayscale Morphology in 1D](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/038-1d-grayscale-morphology.ipynb) 34 | 35 | [Distance Transform](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/042-distance-transform.ipynb) 36 | 37 | [Labeling](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/045-labeling.ipynb) 38 | 39 | [Template Matching](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/070-template-matching.ipynb) 40 | 41 | ## Learning 42 | 43 | [Classification](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/101-intro-classification.ipynb) 44 | 45 | [Nearest Neighbor](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/102-nearest-neighbor-and-preproc.ipynb) 46 | 47 | [Simple Probabilities](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/105-simple-probability.ipynb) 48 | 49 | [Bayesian Decision Theory](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/107-bayesian-decision-theory.ipynb) 50 | 51 | [Parametric Gaussian Classifier](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/118-parametric-gaussian-classifier.ipynb) 52 | 53 | [Perceptrons](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/125-perceptrons.ipynb) 54 | 55 | [Linear Least Squares](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/126-linear-lsq.ipynb) 56 | 57 | [Logistic Regression](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/127-logistic-regression-grad.ipynb) 58 | 59 | [Using Linear Classifiers for Nonlinear Classification](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/128-linear-to-nonlinear.ipynb) 60 | 61 | [k-Means](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/130-kmeans.ipynb) 62 | 63 | [Simple PCA](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/135-simple-pca.ipynb) 64 | 65 | [Summary of Classifiers](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/140-summary-classifiers.ipynb) 66 | 67 | ## Applications 68 | 69 | [Speech](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/150-speech-worksheet.ipynb) 70 | 71 | [Spectrum](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/160-fourier-transform.ipynb) 72 | 73 | [Dynamic Time Warping](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/170-dtw.ipynb) 74 | 75 | [Markov Models](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/180-markov-models.ipynb) 76 | 77 | [OCR by HMM](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/185-hmm-ocr.ipynb) 78 | 79 | ## Appendix 80 | 81 | [Introduction to Python](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/200-intro-python.ipynb) 82 | 83 | [Quadratics](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/205-quadratics.ipynb) 84 | 85 | [Normal Distributions](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/210-normal-distributions.ipynb) 86 | 87 | [Bayesian Parameter Estimation](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/220-bayesian-parameter-estimation.ipynb) 88 | 89 | [Frequency Modulation](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/Frequency Modulation and Encoding.ipynb) 90 | 91 | [Homework Edge Detection](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/hw-edge-detection.ipynb) 92 | 93 | [Homework Ferquency Modulation](http://nbviewer.ipython.org/urls/bitbucket.org/tmbdev/teaching-lw/raw/tip/hw-frequency-modulation.ipynb) 94 | 95 | -------------------------------------------------------------------------------- /bookpage.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/bookpage.jpg -------------------------------------------------------------------------------- /bridge.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/bridge.jpg -------------------------------------------------------------------------------- /floorplan.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/floorplan.png -------------------------------------------------------------------------------- /imagenoise.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/imagenoise.jpg -------------------------------------------------------------------------------- /lexx.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/lexx.jpg -------------------------------------------------------------------------------- /lexx.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/lexx.png -------------------------------------------------------------------------------- /lexx2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/lexx2.jpg -------------------------------------------------------------------------------- /lexx2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/lexx2.png -------------------------------------------------------------------------------- /lw.tex: -------------------------------------------------------------------------------- 1 | \documentclass{book} 2 | \usepackage{graphicx} 3 | \usepackage{appendix} 4 | \usepackage{hyperref} 5 | \usepackage{listings} 6 | \usepackage{color} 7 | \definecolor{mylight}{rgb}{0.9,0.9,0.9} 8 | \definecolor{mygreen}{rgb}{0,0.6,0} 9 | \definecolor{myblue}{rgb}{0,0,0.6} 10 | \definecolor{mygray}{rgb}{0.6,0.6,0.6} 11 | \definecolor{myviolet}{rgb}{0.6,0,0.6} 12 | \lstset{ % 13 | backgroundcolor=\color{mylight}, % 14 | basicstyle=\ttfamily\fontsize{8pt}{8pt}, % 15 | breakatwhitespace=false, % 16 | breaklines=true, % 17 | captionpos=b, % 18 | commentstyle=\color{mygreen}, % 19 | frame=single, % 20 | keywordstyle=\color{myblue}, % 21 | language=Python, % 22 | numbers=left, % 23 | numbersep=5pt, % 24 | numberstyle=\tiny\color{mygray}, % 25 | showspaces=false, % 26 | tabsize=2, % 27 | stringstyle=\color{myviolet} % 28 | } 29 | \parindent0ex 30 | \parsep1ex 31 | \parskip1ex 32 | \title{Learning and Perception \\ Lecture Notes} 33 | \author{Thomas M. Breuel} 34 | \begin{document} 35 | \maketitle 36 | For teaching only. 37 | Please do not distribute outside the university. 38 | \tableofcontents 39 | 40 | \part{Image Processing} 41 | 42 | \chapter{Cameras and Eyes} 43 | \include{003-cameras-and-eyes} 44 | 45 | \chapter{Images} 46 | \include{005-images-as-arrays} 47 | 48 | \chapter{Linear Filters} 49 | \include{010-linear-filters} 50 | 51 | \chapter{Gradient Filters} 52 | \include{012-gradient-filters} 53 | 54 | \chapter{Anisotropic Filters} 55 | \include{014-anisotropic-filters} 56 | 57 | \chapter{Frequency Domain} 58 | \include{020-frequency-domain} 59 | 60 | \chapter{The Fourier Transform} 61 | \include{025-fourier-transform} 62 | 63 | \chapter{Image Compression} 64 | \include{027-image-compression} 65 | 66 | \chapter{Nonlinear Filters} 67 | \include{030-nonlinear-filters} 68 | \include{033-document-normalization} 69 | 70 | \chapter{Mathematical Morphology} 71 | \include{035-binary-morphology} 72 | 73 | \chapter{Grayscale Morphology} 74 | \include{037-grayscale-morphology} 75 | 76 | \chapter{Grayscale Morphology in 1D} 77 | \include{038-1d-grayscale-morphology} 78 | 79 | \chapter{Distance Transform} 80 | \include{042-distance-transform} 81 | 82 | \chapter{Labeling} 83 | \include{045-labeling} 84 | 85 | \chapter{Template Matching} 86 | \include{070-template-matching} 87 | 88 | \part{Pattern Recognition} 89 | 90 | \chapter{Classification} 91 | \include{101-intro-classification} 92 | 93 | \chapter{Nearest Neighbor} 94 | \include{102-nearest-neighbor-and-preproc} 95 | 96 | \chapter{Probability Review} 97 | \include{105-simple-probability} 98 | 99 | \chapter{Bayesian Decision Theory} 100 | \include{107-bayesian-decision-theory} 101 | 102 | \chapter{Parametric Gaussian Classifier} 103 | \include{118-parametric-gaussian-classifier} 104 | 105 | \chapter{Decision Regions for Normal Densities} 106 | \include{120-normal-decision-regions} 107 | 108 | \chapter{Perceptrons} 109 | \include{125-perceptrons} 110 | 111 | \chapter{Linear Least Squares} 112 | \include{126-linear-lsq} 113 | 114 | \chapter{Logistic Regression and Gradient Descent} 115 | \include{127-logistic-regression-grad} 116 | 117 | \chapter{Nonlinear Classifiers} 118 | \include{128-linear-to-nonlinear} 119 | 120 | \chapter{$k$-Means} 121 | \include{130-kmeans} 122 | 123 | \chapter{Principal Component Analysis} 124 | \include{135-simple-pca} 125 | 126 | \part{Applications} 127 | 128 | \chapter{Classification} 129 | \include{140-summary-classifiers} 130 | 131 | \chapter{Speech} 132 | \include{150-speech-worksheet} 133 | 134 | \chapter{Fourier Transforms} 135 | \include{160-fourier-transform} 136 | 137 | \chapter{Dynamic Time Warping} 138 | \include{170-dtw} 139 | 140 | \chapter{Markov Models} 141 | \include{180-markov-models} 142 | 143 | \chapter{Hidden Markov Models} 144 | \include{185-hmm-ocr} 145 | 146 | \appendix 147 | 148 | \part{Appendix} 149 | 150 | \chapter{Python} 151 | \include{200-intro-python} 152 | 153 | \chapter{Linear Symmetric Systems} 154 | \include{205-quadratics} 155 | 156 | \chapter{Normal Distributions} 157 | \include{210-normal-distributions} 158 | 159 | \end{document} 160 | -------------------------------------------------------------------------------- /misc/cooley-tukey.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "metadata": { 3 | "name": "cooley-tukey" 4 | }, 5 | "nbformat": 2, 6 | "worksheets": [ 7 | { 8 | "cells": [ 9 | { 10 | "cell_type": "code", 11 | "collapsed": true, 12 | "input": [ 13 | "from pylab import *", 14 | "from scipy.ndimage import filters" 15 | ], 16 | "language": "python", 17 | "outputs": [], 18 | "prompt_number": 2 19 | }, 20 | { 21 | "cell_type": "code", 22 | "collapsed": false, 23 | "input": [ 24 | "s = filters.gaussian_filter(randn(1024),2.0)", 25 | "plot(s)" 26 | ], 27 | "language": "python", 28 | "outputs": [ 29 | { 30 | "output_type": "pyout", 31 | "prompt_number": 3, 32 | "text": [ 33 | "[]" 34 | ] 35 | }, 36 | { 37 | "output_type": "display_data", 38 | "png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAD9CAYAAABeOxsXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXt4ldWd7787yQ65AIFwCZCEgtwCigiDxXaGGrXogyjV\n2lO1Fx11PNRanXbaOdrTzow8UylWe9rOYeqxrW2101FbZxh5WkVwPKlWi0wP2HMsjqKChkDCLRdy\nv73nj+Uv7+9de73X/e5k753f53ny7GTn3e9e72191++yfithWZYFQRAEYVxTMNYNEARBEMYeEQNB\nEARBxEAQBEEQMRAEQRAgYiAIgiBAxEAQBEFAmmJw8803o6qqCsuXLzf+v6GhARUVFVi5ciVWrlyJ\nb3zjG+l8nSAIgpAhitL58E033YQ77rgDN9xwg+s2F154IXbs2JHO1wiCIAgZJi3LYO3atZg6darn\nNjKnTRAEIftJyzLwI5FI4OWXX8aKFStQXV2NBx54AMuWLTNuJwiCIIQnrgF3RgPIq1atQmNjI/7w\nhz/gjjvuwFVXXeW6rWVZefvzd3/3d2PeBjk2OT45vvz7iZOMisGkSZNQVlYGAFi/fj0GBgZw+vTp\nTH6lIAiCEIGMikFLS8uIeu3duxeWZaGysjKTXykIgiBEIK2YwfXXX4/f/OY3OHnyJGpra7F582YM\nDAwAADZt2oQnn3wSDz74IIqKilBWVobHH388lkbnGvX19WPdhIyRz8cGyPHlOvl+fHGSsOJ2PEVp\nRCIRu/9LEAQh34mz75QZyIIgCIKIgSAIgiBiIAiCIEDEQBAEQYCIgSAIggARA0EQBAEiBoIgCAJE\nDARBEASIGAiCIAgQMRAEQRAgYiAIgiBAxEAQBEGAiIEgCIIAEQNBEAQBIgaCIAgCRAwEQRAEiBgI\ngiAIEDEQBEEQIGIgxMDf/A2wc+dYt0IQhHSQNZCFtEkkgIsvBv7938e6JYIwvpA1kIWso7BwrFsg\nCEI6iBgIsSBiIAi5jYiBEAsiBoKQ24gYCLFQVDTWLRAEIR1EDIRYEMtAEHIbEQMhFgrkThKEnEYe\nYSEWJDNYEHIbEQMhFgYHx7oFgiCkQ1picPPNN6OqqgrLly933ebOO+/EokWLsGLFCuzfvz+drxOy\nGBEDQcht0hKDm266CTs96hA8/fTTeOutt3Dw4EH84Ac/wG233ZbO1wmCIAgZIi0xWLt2LaZOner6\n/x07duDGG28EAKxZswZtbW1oaWlJ5ysFQRCEDJDR7PCmpibU1taO/F1TU4MjR46gqqoqZdt77rln\n5Pf6+nrU19dnsmmCIAg5R0NDAxoaGjKy74xPFdKLKCUSCeN2XAyE3IEur2QTCULm0QfKmzdvjm3f\nGc0mqq6uRmNj48jfR44cQXV1dSa/Uhhlhoacr4Ig5CYZFYONGzfi0UcfBQDs2bMHU6ZMMbqIhNyF\nsogkm0gQcpu03ETXX389fvOb3+DkyZOora3F5s2bMTAwAADYtGkTLr/8cjz99NNYuHAhysvL8ZOf\n/CSWRgvZA4mAWAb5z2uvAeecM9atEDKFLG4jpEV7OzBlCvBnfwa8+OJYt0bIFJalSo4cOAAsXTrW\nrREIWdxGyBrEMhgf0HV+442xbYeQOUQMhLQgEZCYQX7T1+d8FfIPEQMhLSSAPD7o71evIgb5i4iB\nkBaSWjo+IBHo7R3bdgiZY9yIwWc+A7yf6CTEiFgG4wNxE+U/40YMfv5z4OTJsW5F/iGWwfhAxCD/\nGRdiQKNW8nsK8TE4qFIOxTLIb0QM8p9xIQbk5+zqGtt25CNDQ8CECWIZ5Ds0kJKYQf4yrsSgs3Ns\n25GPDA4qMRDLIL8RyyD/GRdi0NOjXsUyiB8SA7EM8hsRg/xHxEBIC3IThbEMenpUCYt3381cu4R4\noesrbqL8ZVyIAd3AJApCfERxE3V0qJpGx45lrl1CvND1FcsgfxkXYkAiIH7t+BkaAkpKwrmJaL6H\nzPvIHmpqgG99y/3/dH1FDPKXcSEGlAkhnU/8RLEM6DqIOGcPTU3A66+7/5/EQNxE+cu4EAOZJZs5\noqSWihhkJzNnuv9vcFBZgGIZ5C/jQgyooxLLIH6iWAYiztmJl6APDQFlZSIG+YyIgZAWQ0NAcbF6\nDbrGhsQMshOv6zE4CJSXi5sonxlXYiAj0fgZHASSSSCRAIaHg31G3ETZiVe5lqEhJQZiGeQv40IM\nqNOJayR6773Ajh3x7CvXGRoCiorUT9C4gYhBdhLEMhAxyF/GhRjEaRlYFvD1rwP/43+kv698YHAQ\nKCxUP0HPLxfnvj5lVcgS2GMHnXs/y0BiBvlNXopBT4+a2ETEGTNoa1Ovp06lv698gFsGQcWAWwY0\nC1kmBIYnLks3yOxichNJzCB/yUsxWLoU+OQn7b/jzF45fhyYOFHWRiAGB9NzE5EYdHdnpn35THEx\n8P/+X/r7oevhNeoXN1H+k5di8O67wP/9v/bfcVoGx48Dixapcgq5imUFD/b6obuJ+vv9g8k8m0jq\nRqXHgQPp7yNIqQlxE+U/eSkGgLMzijNm0NmpJuf094cTl//4D+C///f0vz8ObrkFWLEinn3pAeRD\nh9T7Xp07twzod7EMwkH3dxzuyjCWgbiJ8pe8FQPusogzm6ivDygtBSZPDmcd/K//BXzzm+l/fxz8\n9rfAa6/Fsy/dMiC3j9faEdxtR9dELINw0PmK47yFsQz6+0cn2C8xpNEnb8XAZBnEIQa9vWrGbUWF\nUwyGhoCdO90/V1yc/nfHRUlJfPvSLQMSAS8x4G4isQyiceaMeuWJElEJsiwsTS4sLMz8ZMHhYSU8\nR49m9nsEJ3krBtwyiNNN1NurOlNdDN56C1i/3n2klk1iEOfDrFsG1KkHEQOxDKJDYkCv6RDUTVRY\nqAZCmXYVvfOOepX1LkaXnBKDj34UeOmlYNvqlsGECfFZBiQGhw4B27ap9xMJ9ep2A2dTHn0co0lC\ntwyoU/fqpExiIIHJcND5itMy8HMTFRXFX6zONEA7cUK9Hj4c3/cI/qQtBjt37kRdXR0WLVqE++67\nL+X/DQ0NqKiowMqVK7Fy5Up84xvfiPxd//7vwPbtwbblYkAVF/0sgyAdNheDf/xH4I477O/gr6bP\n6e0aK6JkQn3uc8Ajj6S+72YZeI306RwNDYkYRIXOW1yWgV8nT9c5mQROn07/OwHg979X+9OhZ2XP\nHlVaWxgd0hKDoaEhfOELX8DOnTtx4MABPPbYY3jdUBT9wgsvxP79+7F//358/etfj/RddIMEzWXX\n3USlpd6WwU9+AhQEOBt9fXbMgDJnAP/ia9nU6UVxyTz0EPDDH6a+z+cZcDHwOtf0v+Fh+3fJUgkH\nCWpcloHfHAKyDJqbgbq69L8TsJMYdJci3Qv/8A/AjTfG812CP2mJwd69e7Fw4ULMmzcPyWQS1113\nHZ566qmU7aw0fSQ9PcCXvqR+D9ppmNxEXpYBmaZ+cMugudl+388yoODcWIsBfX+UIHJpaep7/f0q\nHkJuojBiIJZBdOK2DPzEgCyD6dPT/z6CBiUtLc73+TM+Y0bq5955R2XECfFSlM6Hm5qaUFtbO/J3\nTU0NXnnlFcc2iUQCL7/8MlasWIHq6mo88MADWLZsWcq+7rnnnpHf6+vrUV9fP/L3//k/KjUTCD6q\nNbmJeAd17JhyC82Zo/4mc7W7W2UyuNHbq9JKKyrsm5bPOXDrBLNFDE6cUA90a2v4z5pM+oEBJRK6\nm8grM8UkBrluGWzdCnzxi/FmankxMABMmxafZTBxoveAaGhIXeOWFiX8w8PBLGkvSMhOnAAWLLDf\n7+0Fzj1XTRydPDn1c6tXq/s3m+Jwo0VDQwMaGhoysu+0xCBBUVMPVq1ahcbGRpSVleGZZ57BVVdd\nhTfffDNlOy4GOm++CcydC9x8s3NmcVBonV4+al+9Wt10NGmHag6dOAF84APu++rtVZPOitiZ6+jI\nHcvgxAklgKdO2Q94UEzpnwMD6oEly4AvMWpZasZ2VVXqZ2j7fBGDr34VWLsW+NM/HZ3vGxgAKivj\nEYMglgG5iQoK1LadneaOOgxuaci9vcDKlcDttwOmfq+11Wyljgf0gfLmzZtj23da2l5dXY3GxsaR\nvxsbG1FTU+PYZtKkSSh7f6i9fv16DAwM4HTICNQbbwCbNgEf/KB3yiKHjxpIDHTLgDeDi4EXNM9g\nyhT7vb6+3IkZnDihTO/iYu/RuwmT0A0MKIuBl6Og9598Epg1y7yfCROcMYOxPi/pQPfaaM6VGByM\nTwzIcqbf3bahgcOkSfG4p2gfurVPrtiqKvfnPZtStfOFtMRg9erVOHjwIA4fPoz+/n488cQT2Lhx\no2OblpaWkZjB3r17YVkWKisrQ33Pm28CixerEUmU4KdpaUbdxKQZj35i0NdnxwyI/v7csgxmzFDn\nI2xbvMSAWwYkvG6aT9krZBlMnJjblgG1PYrrLSoDA+oejGOmLi1Q5HVPkGUAxCcGXpZBaan395hc\nlkJ6pOUmKioqwrZt23DZZZdhaGgIt9xyC5YuXYqHHnoIALBp0yY8+eSTePDBB1FUVISysjI8/vjj\nob/n8GHgrLNUBx5FDMgy8HpwqLP2q0ZKo5a5c9XfpaVOMQgbM2htVVZGAI9bLKRjGZjSYimAzC2D\n8nL1OnGieT9cDMhfPdoi+etfA//zf3rPGg8KdVijWbyQRHRgILy7z7SvoiJbDMrLU7fJhGVAAwc3\ny6C42P15EssgftISA0C5ftavX+94b9OmTSO/33777bj99tvT+o7Dh5Ufv6Ul2giSxMDLpO7vV51k\nEDdRSQlAMfB585xlFcJaBpWVwN//vVowZzSIYhlQmq5JPEyWQXm53cGYGBiw3UQkBqNtGfz618Cz\nz/pvd/So+lm92n0bGtmOpqDReaf5AV5JD36QZdDTA3z/++Z7kQvOxInB3bVeDAwAU6eaLYOSEnX/\niBiMHlk/A7m93faPBunAaPTqN+lMH4n39wO1te71UFpb1VwEihlMnqy+I4xl4DYC3rXL+5jipLVV\nPYBhLAPazi2ArMcM/MSA3HZDQ2NXGjnofJV/+Afg/PO9t6GRbZRj6O2NNomLi0G6riK6Vn19wN/8\njXnUT/NJAHXPxyHe/f3+YuA2uBI3UfxkvRi8+64afScS6gbxuwnpIQHsB94UQNbT4vr7gQsuUKWm\nTTzzjMpm6umxg22JhPquIKmlAwPKvB7rmMGZM6odYSwDEgNTp8MtAxIDcl/QOdY7Xu4moqJko20Z\nUAfk1pF+7WvKhRRkBJpOPOiTnwS0nItAUOdcWuovBpYF3Hmnu9XLO3rAvD9uGQR5DoNAmWj6vkgM\nkslUMaBBXjKpfo+SXSiYyXoxOH5cpXICwTow6pz4yNeUWmqyDM49F3j7bfN+qWM7csSZS07fEySA\n7CYG6fh7w0JiENYySCS8LQPdTdTfb+9f9wlzNxFZBqMtBjQa14O+w8OqBtaWLcDDD9v3iVdOe5Da\nPm4891y0kT23DPzO3ZkzKj7ym99474twEwMSjDisEcB+JtzEwGQZ8EmNf/iDWpcjbOxLMJP1YkDr\nBwDBimTRjZ1MOic36ZaBSQw+8AElPiYXAgWWT53yFgMvN5FbLne6k3fCwC0Dr4do+3b7HPX1qSC3\nqQNwCyAPDNj717+HxIDcRGOxnCJZBrpQtbaqGliAGrG7HQMnHTEI6q7Sofs8iGVAwpeOZcADyHFb\nBvp584oZkAurt9d+blh2u5AGWS8G5KMHgpXPNYmBKbXUJAbl5arTMz009F5ra6oYmALIHR2pcx3G\nQgz0B5tbBl6d144d9u/9/bY5r4+QvQLIfM6B/pnS0uxwE+mdPL/2xcW2WAQpyR1FDOjeDksYy4DE\n4Phx8/8pgHzihMqSM+2PWwZxxgzCWgZnzqj7Syrexk9OiAF1vjQN3qvGUDqWQXGx6vRMDz7vJPgD\nTDED3TKoqHAWdjMFSsn/malp9f396ju5XzWoZcCDiJQCaBJjPYDc12fHDOhY9etlchON9gPd1aU6\nNf17+XUeHAwmBulYBlHFIEzMgMTAbYlMCiBPnw7Mnh3MMojDTURxtDAxgzNnVNCZT3DM5Tkq2UTW\niwFVCQXsILLXQ+clBvzGMgWQi4vd0+Z4J2FyE5lSS8ndQG3QO71M38zUoXMzurMzmGXAA6d0bkwd\nD4mLV8xgcFD9j0oL8AAyWUxjYRlUVqYKIs1Ep22CLNYzFmIQxjKgtrvNDSDLAHAXl0wEkKNaBlOm\nOC0DEYN4yHox4JYB4B9ENgWQTYXq3CwDLzGgXO6gMQMeGPYSg0yt90rHwQO/QS0DLgYkyKWlzn19\n7nOqJv3kyd4xg4EBZZ1cdJE963iss4m6upQYmPzVRGen/f8ghfeiiEHUFMkwMYPubnV93MSApwF7\niUHcbiI/y8AUM+jsFDHIFFkvBtwyAPxHJW6WgdtKZ+SqCSIG8+bZbSAoZmDKJuLtHhxMFYNM38x6\n+qRlpWcZlJU5O4r3J5qjosI7ZsADlO+9l+ommjjRPLO8rExtHzc0k91PDHp7vdNG//hH9UrXPMh1\nHB5Wdbbc/PdBCWMZdHerjLwgloHb/jLhJqJYlFcA2c1NxIsiBjnvozk7PFfJejGIahn4uYnofzxb\nJKgYmGIGfJ4BxQD4qI/cIXyE6RZgjQtdDHp67GCvn2VAnbdlOd1EZBnwOIduGZhiBlz49GyiadNS\nH1bLUu3NxDq4fX3KTThxYuo5oDafc44zMUDf7t131TY9PWobP3ElOjuBH/wAePFF9XfUMiTUgQex\nDHp6VNG3IJYBf74+/nH7uuippaNhGfjFDIIOprq7lTVB4i2YyVoxsCw1iqLCcESUmMHgoHOlM8tS\nv5eVBROD4WEVhKuuVn/zNDzuJkok7CCqjpebyG85zqjoYkAuImq313kki6mnR22nxwy4u6iszDnp\nTI8Z6J2qHjMgMSCBsSw1nwOInnrpRVeXus6mc9DbC3z+88C3vmW3NZFIFQOaj0LXPmh9JeqQjx1L\n7xioAw9qGXiJAbfcuBhs364qBtM2ZBnEmU3kNeksrpgBraXM40FCKlkrBtddB3z4w87UUsA/vdTL\nTcTX3i0sdGaTeIlBW5t6n888JngAmeIS9NDxDsQkBtTWTFkG5HrhHTgVIfOzsLjro79fbV9WZotA\na6taF6GvT50PPzcRd7dwNxGV9Cgqstv5wx/ahQAz8QB3dak2ms4BDT7I/edWRoRbgkGWjSTo3qCy\nJ1Etg7Axg2nT3LfjbiL9nJAAZGoGcpSYQUWFPVCk7b2gLKo4yn3nM1krBm+8AbzySjwB5KEh9Td1\nPv39dsneIJZBW5sajZhmCnPLgKwPUw67m2VAHWcmoIeEOnBeSiOsGOiWQVeXOh6KLQR1E9G54pZB\nQYHy39NDy9eWzoQYdHaq826ahU2DD+7+0917dEyAbfVEtQzIAuMWUF+fvcyrG2FiBj093mLA3UR0\nTqhdlHXHrYfRnIE8NOR0SfLsNfpcUDGQuIE3WSsGU6eq17gCyEVFtg+SC0YQy6C9XYmBycdO30Ni\nMDgY3DIIKgZ33w185zve25ig76IHl+rEA9HEgAeQyU9OFBXZnQiJoskyMLmJCguB+fNt14v+8McN\nuYlM50AvnxxEDKK4icgyMNW0OnoU+O53nenMNTXOchKZtgz4kq6A0zKIO5vI7RokEuo7uVByMeD3\ntRdiGQQja8WAiCOATP5OMjtpG9oXxRCSSXfLoKLCHFjmbiLqBOmBp3ZalvrRJzlR3MIvZnDffXbm\nThj0BzqKZdDX54wZ8NowPEBeWKj2X1xsnxOvmAF3ExUWqsWLfvtboLnZKQaZqDtDloGXm4hbBqZA\nc1Q3UUeHSkSgmIgpiYDe4xZSUxOwd6/9N43Ug8YMgloGdE7oOtO+MzXPwCtmAKTGDbgY6G10g+pP\nZWJgkU9krRjwujheAeT2dmeanptlUFhoWwY00uUdOa3v6mUZfOc7dkCN0N1Eg4OpbiJyheidD80Q\nDuImIkspDH196ri5bzWqm4jmGXDLgIsBlW4oLrbPfVDLoKAAWLJElU9evtzZ8WZiZvLp06pz9HIT\n+cUM0nETrVoFvPOObaXqAWoe8AfskTEvyx7GMujpUfdPX5+aCKnHKUyWgV6WW08/TddNZFnmtSxo\nKVSyOvW4wZkz6jM0+OBtdINcjVK2wpusFwO/APKaNcCHPmT/7eUmcrMMSBwAb8ugslKNYDlulgF3\nK5EYmcQgaMxAf/gOHQLuusv7M729qt3cXZSum4hGY/ycAer3M2ecYuAWMzC5iZYuVf8/edLZrrgf\nYMsCHnxQlV7wchOFiRmEDSDPnKnEqKnJvH9dDOiV35dh5xlQ9pSpcqmXZcCvYZzzDNzaT25hev71\n9FKaJxPGTdTaqhZ0kslp3uSEGOiWAb+ob7zh9AV6uYnoPT3I7CcGZBmYMMUMTp9WC8H7iQF3E/nV\nJ9LLR//oRyr90YveXueknjCWAY1Ge3vNqaW6ZTBhgjpvZHHRqJosH90y4JPOCguVqAOqNo5pLkZc\nNDaq0fG775rnWujZREHcRGEtg0mTlBi0toYTA+7mCBszKCtT25rqE5lSS/XJdnoAOV2R5s8pLX8K\nOONaQKqbiDLBSAzcZq8/+yxAK+y2tqrnUSwDb7JWDKgz6upyWgb6LFjAnK6pZxNRzOCv/sru/N0s\ng44OVf+d2tDerkbYJkzZRKdOqZtPd1NNmKAWTFm+XL1P7hc9SGaipwc4cMD+O8gM1r4+JQZ8pmYU\ny4A+5xUz0C0DElm9wqRuGVDMYOZM4M03U9sV5wPc3GzfOz/6kfs8A55NxOdNmM4PXfug9ZVIDCor\n1aDBtH86x16WAZ90FiSbiK4f7YsPPkxuIr3eEp90FmZhJDfomdPrjemDP10M6HMkBqZsJAC45Rbg\n+uvV721tIgZByFoxoIdWLxltGrlzcfCKGXR3q9FCW5sztVQXgxdfVCtDNTer9yi11ITuJhocVGIw\nY4ZzpTUSAwB47TX1SimuXmu9EkeOAGefbbcpyGQsk5soTMyARl30gPKOx2QZ6G6iIGJAMQNACVdX\nl2rXvfcC998f/AGmIL0bp07ZFTnPPVctcRo0m4iuq35+6DyEdRNNmqR8+CdO2PcFv/50P+u1pXQ3\nEQWQw1gGtA/TvoDUWlsmyyAOMeD3j58Y6MF1Wj/DSwxoPg0glkFQskYMvvUt4KWX7L+DikFBQWqZ\nCV0M6D3KKmhpcaaW6mJAUPpfWMvg9Gk10nUTA95W3nkG4c037eMGvDtAsgyiBpApuBdEDIqLbTcR\njxmQG8xt0hnPUqGYRF8fsGyZ06rx46qr1E9TE/CZz5jPBeBcnMjNTaTPM+Cz1wndTUTuMB7kNUFi\nUFGhxEC3YgH7HPOFXPgx0PcGtQxoTkhpqV3OmrsdTZYBd7HSayYsA8Dp+tXFQI8Z0DND2URBxaCq\nSsTAj6wRg7vuUmUACBKD06ednaguBty/CNijHJMYEM3N3m4igiYGRbEMuBjQg0TLdxL0vWHE4Pvf\nV6mm+oQyE+kGkMkPTg8oH4XqAWRuGfCYAQXI/dxEgFMM+PUJwo4dqoLqq68CP/956nmh8/Xee/Y5\ncHMT6TOQTem/umXAs9O8IDEgl41JDOhcBRGDIJYB/04SA92a1jt6vfAiDyDHbRnwpJC43ES8XyA3\nkQSQvckaMQCco1xS/e7uVMuAB9JMYmAKIHMxIMuAOh/esU2frl7nzrVFx8sy0APIFDMwuYnmzLE/\nR6NlaqvXXANeC+mJJ9QkNL8F3YH0AsjpWgZ6zIBbBoODzkJ1ZOUUFqrPd3So/4ftdHi6oV77h95v\nbPQ+B3yeAR/1e4kBdaZB2ksdc0mJuxhQnSs+X0UXrqCWwYkTaj+TJ7uLgVfMQB/QAPHMMwhqGUQV\nA17nStxEwcgqMSgvVx3pXXfZU+iBVMuAiwHdUHwEoz9gfOQDqM49mVQ3Umdn6o3Z2akWRafRZRDL\ngGcTkWXASw1QNtM116gHvavLafK6WQaUj61DYuB1g+sB5CiWARcDPgoNEjPgbiI6PqqtT6vWcTcR\noNrX2hpdDChbhjo9gtp96pTTMvArR0HH5uUmoo4yjBjQ+dIHLoDa39SpapJZV1eqqNM2QWIGM2fa\nk+zo3PLzQcfgZxnwADJZTX4uMS/0mIGXGOgxA4qzeYkBL7RYUOC0kAUzWSUGxcXAc8+p+AEXA35z\nzJ7tHPXRA0t1R7xiBgSNYElYdJdHebkzaylIzID7lltblYWhWwYA8OST6gHt6grmJqKH/p//Gdi6\n1X6fShV43eC6myisZUAZMkEtA9OkM160bsIEtU0yqR5QPs+AKCmxLQM/i0mnoMDu7HQxoHa3tXkL\nIokBWSsFBeZ28E6SBiBBUi5plE6Wgcm9NDCgxOC114DbbrPFQN8maMyAjoNng+mWgS4GXjGDRCKY\nS8wL6tQB7wCyfu71ALJpBjNgP3utrWogF4drK9/JKjEYHHT6tysr1e/85pg3zzlNv69PdThPPaX+\n9nIT3Xyz+ruz090yIPiDc+KE7T7ScStUV1FhFgPAXsyFu4lMYvDb36qbubhYpcl97nPA7t2qTg2d\ngyCWQVxi4GcZ0PnQYwZkGZSX22JA6bQ8ZkD76eiwLaawYsAtAA4Xdq9zYLICTe3gYhDGTdTe7hQD\nNzfRsmX29n19qSPbIDEDPeOMjpufJ74vIJibiG8XFbKKqV3UofOMNyD13HNrmiwDUzuo3W1tSlhF\nDPzJKjHo7naO6CiYy91E8+apB52PjNetU3nFgLtlUFQEPPwwcMMN9kPoZhkAtmVw9Kh6IN3KQXDf\nckmJ2ld3t3rg3cSAOkWeTWTq9NauVcFialtFhXJfTZ2q2l1R4T0qjCOA3NurHijyOfN6R/oMZHql\nB1jPJiIRJDEgNxFfj5omr02YEFwMaBvLso+JyikMDam2c5cfPwe8E370UZVcMGOG/Z5lmd14PG2Z\nBhtBOhwqheEnBkuXAt/7nvqffh3p+8kyaGlRJS50qI0XX6xe6bgrK90tAz21lCZEmiy4MHGD48eB\n/fvtv7kR58tBAAAgAElEQVRlwAPINKmM4PcALbREbiKvbCJyE1H1AL/S90KWiUFXlzMLhEYIfASa\nTAIf+Qjw/PP2zfHLX6obnTpYLzcRD3ROnqxGoW6WQU+PWlehoMC97rxuGbS3q1eaWQmYxYBbJNTZ\n9PQ4K1UCSvj0ttHIddo0fzeRPukszAxkEoOWFhWAC2oZ0PoGlN9OYsktAzc3Ee0njBjwKrG6GJw8\nqf7f1KT+bmuzz4EelL3xRvt9jpebaHDQGTPw6nD6+9X/aW0MHjPQxaCoSNVr6ugwiwGPGQDOjpZ/\n36RJasY14C0GbqmlPMjPn4GwI+2773YKlptl0NnpzOjjQjw0ZFcyJcvAzU1EbaVSHGIZ+JO2GOzc\nuRN1dXVYtGgR7rvvPuM2d955JxYtWoQVK1Zgv+muhbrZu7qcNyl1EnpHvG6d6qQPHrRrDi1YoHLw\nTTOQ9Zud3ETTpqnO1stN1N+v3DVu6GJAC+HQyBdwtwx0N9GNN6amn1Kwm0NiMH16ODdRFMvg9deB\nt95SYuAXM+CvfPF0chPploHJTcTFKowYUMdOx0xiQJP03nlHvXrFDM4+217NjmOyDKK4iU6fVh0x\nzbrVU3H5vouK7MGKKYBM55/E0zRY0e9rNzHwCyDz4DGhH+uf/zlwwQXux667WfWYgZsYcCHmx+M3\n6YyLQVlZPCU08p20xGBoaAhf+MIXsHPnThw4cACPPfYYXn/9dcc2Tz/9NN566y0cPHgQP/jBD3Db\nbbcZ90UdG79JC1xa94lP0L7th2HJElWnSLcMzpxJvdlJDGbMUCNxLzcRXyrShB5Abm1VNzONfNV5\n8ncTDQw4y01QahwtzMOhh8UvQyLdAPK0acCuXcD69XY6ZBDLQD8/bm4iPbWU7yesGJCV1Nen3Ghk\nYVLZjsOH1auXGJSXA//yL6n7N1kGXm4it4mA5CKi7+7ocHcT6WIwZYpZDKjT0yc0AsHFgA+W6Brr\nAXI/Mdi9Wy1G5Ya+MJRuGdC+KPOJ4PcA/4xfNhGdF5pwR+1taQEaGtzbOZ5JSwz27t2LhQsXYt68\neUgmk7juuuvwFEVy32fHjh248X37e82aNWhra0NLS0vKvqZPVxc1SDXEOXOAr31NBVG5GPznfzrF\nYOdO9UDpATJyE82YoToLL8vATwzIxCfLALAtgzAB5MFB52Q6+r23N7VtvMP0ckv09dkBNssKX5vo\nAx9Qv3/96+rhCmIZUNuoQ6SHmSwDEuKCAnNqKReVMGIwfbrtJpo61bYMKMvs5En7b7cZyDSK1AkT\nQN62zdwxA8oK5UkRdC7cxIBiWj097jEDoqpKvR46ZJ/7KJZBebk6DyY3GGfCBDWB78EH7XPkhW5Z\nBbUM+Lnnx0Pf5xcz6OiwxaC3V8UML7rIu63jFZ9L6E1TUxNqa2tH/q6pqcEr2vDAtM2RI0dQRXfv\n+5w+fQ+6u4Hf/Q4A6gHUe3ZWc+aozp4evBUrVPrl9On2yJP8xIA9+qSYQTKpRlvt7d4xgyCWAfnE\ng4oBWQY0wYncENSBAXaKJGXWcGjU69eh9/aq76IlKfXaRF5CMjiogvUAsHq1etVnIPPzQteBzgE9\njHRslFnU1aW2ofNjWfHEDKZNA/74R/U9lZWp9Xx4YoKbIHqJgclNRFYPjxkcPKi2PXky1TVCbiLA\nO2ZAnTNZp729qeUU+Gj+9deBj31M/X7WWWoN6b/4i2iWAbdaAdsy0Ef2JSXAM88o8fvzPzcvCcsx\nBeBNMYMzZ4BFi+zt+LnnAkJiQBVlLcvpKqP7pq3NaRmQhZirNDQ0oCFDpk1aYpAIuJq3pdnNps+t\nWXMP3n5b3QgUVvDqrCZPVg8ctwzefluNoEgMTB0ldxPRSNXNTRTEMqBRXXFxODHo7LQ7Z3ITcXcJ\nX51Jn2X9xS8C55yjLCE/MeCTt7ibiDKf9IeIGBxU5/TkSfvcBLEMqDOlS05Wz8CAHReaPNl5fvhx\n0z4pY8RPDPr6VBG/KVPUcXR3Oy0DEoNTp2xrxM1V1t2deq4ty91NRLOouZuosVH9f88e4IornJ/h\nbqKSEltQTJPO+Ax5ihkMDqr2FxSkjua5Vfmf/6le9fuajnvqVHfLoLhYfQf938syoO987z3/OQd0\n39Cz4JZNpIuoV8yAPktLrnKLjD7T3u4UAzfXc65QX1+P+vr6kb83b94c277TOjXV1dVopLsfQGNj\nI2pqajy3OXLkCKoNUTrySbe2Ao89ph4qr46uvFw9XHRzTJum/uYBZBP0AFPgzrJstxGH/P9U58gN\nPYAMmMWAP0w0+iK3DXUG/Hu4GOht+9CH1KpgfpYBWR60HQ8g02Qqt4eYOgDqvAD7wSTXiClmoHem\n1KGTZdDRYVsGJjcRPayUkeQnBrffruaP0Kzejg5nyi0XAwq8u9UmMlkGbqmlg4N2fSXuJmprA/7k\nT8yLyOhuIsBcjoTOPVmnJFL6rHrTpC3Avu90MaDzXF7unlqaSKj/t7erz1JcxyQGVMjxvfe8a2QB\ntjjz9pssgxMnnKm9/NzrMQPAvYYVfaa11SkGfu6s8UxaYrB69WocPHgQhw8fRn9/P5544gls3LjR\nsc3GjRvx6KOPAgD27NmDKVOmpLiIAKcYVFSoiVUXXqiyhEyQGFAnRJlBPGZggran4NvEiU5RIcrK\nVLDJyyqg/fAAMpCaTaSb2VwMyDLQR19cDNyOxZQhwcMxZBnQw+a3UBDH5BrgcQM3y4DOAY8Z8NRS\nsobcUkv5PoOIAVVxnTTJrmtkEgOKJfA26jED/fzQMbvFDEyWAaD80rt3p7aVu4n4fajvn+6FggJ1\nTJQOywPU/H7RLQs3MaD39QVxuJsIsMVgwgRvy4DcsO+95x/r08XAbQby8ePOjDo3y4Afu0kMdDdR\nUZH/AlLjnbTEoKioCNu2bcNll12GZcuW4dprr8XSpUvx0EMP4aH3V3C//PLLcdZZZ2HhwoXYtGkT\nvv/97xv3RSM7nu1x990qrdFEebldwwZQnykosFMx3YJ4/CEE3MWgtFTdmH5iQDED3TLwyiaiADJ1\nPtRh6pbBzJlmy4Afi96Zz5ql3GyDg3ZHRuLDLQPAXwxMoyjuU9Y7bjofgDNmwLOJ6HvdUkv5sQYR\nA/qsl2VApUTIMqAOnzpF6iT0ThHwdhORZaDn/G/cCPzhD6qoIEd3EwG2GJhSSwHVkZ0+rdpKnR4V\nriMrKqgY3HqrXZvJzU0E2GKgHx+npESJwfTpatW4IGW0qU30nWEtAzcxMN3H1GZyEyUStrgJZtI2\nmtavX4/169c73tu0aZPj723btvnuZ+pUdVG5GHgxcaI9OiMmT1Y3ezLpXUuIv5IYUOYMQSNgPzEg\nC6Cvz273pEnBAsjUOdPDzLehsrvHj3uLgclNtHevir1Qh0OxkTCWgck1AKgOtbXVHGfhmGIGlDJI\nYkCCwf24vIRCEDGYPFm9UmxEn5nd2alElS9dSteJXIBkQZE/XsfLTUTHRvNdADVXYf58VZL92mvt\nz5jcRBQzME0oA1RHRmswmNx9gH3/cBGgV36Nkkn1/UEsA7JGyDIwBZDb2oBLL1UxCrKy3M6hn2VA\n1mZHh32OqM3cMtADyG5uIj7vh1x/EybYx+3WzvFM1pyOKVNsN1EQMaCOhYsBjWgoU8iEbhmUlakb\nxuQmAvzFgIp28eAjdxOZpvJTwI+7iUwB5Fmz1O9+YqCbv5SBQsfqJQYm83542P1hmTrVXrvX5L7S\nK1nqqaX0vW5uIu57DiIG1AHOmOFuGZDbQRcDwD431CGagulBA8iUxppMqkwb3qkBqdlEtK2p/o5u\nGehiYCrmxuv7AO6C7WcZlJX5WwZ0LVetUpbo9OnegwtdDPr6UlOkT55UlhO/79wsA7pn3NxEAwP2\ns83FwFS1VVBkjRiQZTA0FF0M6OJzy4BGjoQuBlRCwuQmAvzFgPbFs1TIRZNImIOk5eXACy8Ax46l\nLwbt7epz/GGgfHtuGdBDwDtwt4qXZBWYOkYvMejoUAFdjp5aCni7iXh6bdBsor/9W+C//ld7dOom\nBhQz0NfH6OxM7RD1YzDFDPQAMvd160FaID43EZ8vAtglGujc8Y7XdO/wQQAXQt52HjMwWYn0XKxa\npVy5JouD4yUGJCJcLAkuxKYAspebiESNi0FHh/rdL+A9HskqMXDLSDFBHQvP/uCWAYnBlVc6P6eL\nQUlJ+mJAnTjdoHTD81m2vMOrrFS+0cZG1WbqbLgYnDypgui8rToTJgD/8R/q96Yme5RMlgF1OJMm\nqf3pwVG3kZxbvADwFgNyj3G4m0i3DMhq4sf91a8CW7ao34OKwcUX29lhgLIAuBiQD9rPMtCP+S//\nUqXx+mUTUWd6770q5x+w00I5YbKJuOVqWd5uItqPvsaFl2XQ26vunUmTzDEDchP5WQYf/KB6pTa5\ndbJdXer803k0iUFXl3PCGRAsZuDmJtItAzrnpmsjZJEYrFrlzIH3g8SAPxRUITOZVDf5s8+qSqVU\nkgBIzXopLTVP7NK384JcIzSSPucc9eqWPnnBBbZI1dbalgG5e4aHVZtJDNyC4SUldjZNY6P90HR2\nOh+2iROV+OjHErcY6PBMHD2ATOfKspxicPHFShCAYGLAOwg6Xj83kW4ZnDljDh5/97vAd77j7Sbi\n2TbJJFBXp/5v6nBM2UQUa/CyDABnaqmbGFDBPtMkLQ6N4Om50K8BDyC7ZRPRIGnGDHvwxdcA4QwP\nKyGcPdvdMujrs0tHcPQZyGFiBtQebhkAyjoTN1EqWSMGlAIJBOuAqXPVb2LA9v1eeqnaL89OoBuC\ni4nJMqDOKkheMg96WhbwqU/ZbTNZBomE6mSoPSQG/CE+cUIJBeAujiUldpXTtjb78zxTCbDFIE7L\nwC+ATMfJU0tJDOjhpHPsNnfRNCLX0X3PgFMMzpyxxYA6Yn5/TZumrCY/N1GQeQYcclnRvdHX5xRE\n3S9uSi3lbS0psTPX3MTAFKT1ihnwFQP5NaD5INwy0C0+LlYHD6oYiZubqLlZXRNeQdfNMuB1iei4\n/FJLvdxE1EbA/iyfoS7YZNUUDH5zBIUHLPWL7vUdfMTl1bH5TbMHUhcR4Z81iQGg5k/wjBtKTwXU\ng8LFwM0y4C4sqrAK2MFibhk0NcUnBseOBbMMysrsh5nHDNyC+zpB3UR+YkCxlz/7M+D++51rU8yZ\noyZPmSwDwmQZmOYZcGhORk+PnbFGFUs5w8Puk84Ap4uDLANTXK242FlEEPAXA71cOlFerr7Da57B\nVVfZbaNMPDcx+MUvVEXYoSHvALJJDNwmnXF3r24ZUAIEdwvR+4C6HiIGqWSNZQDYFytIB0zwTBqT\n60jHZBkA7mIQxDJwWwvWSww4umXQ26vMapqo7SYGPH22uzvVTaRbBvp5cXt4vcSgstI509uLSZPM\n2URuab865GbzWmuXd3j0SjXuLUuNcGmO45QpwFe+4rwWJAZhA8g8m8jts9xtYgqOAnZwNg43UVgx\ncOsQubvWbQZydTVw002p++3uVmVheC2od95RtZP4DOp0LQMS1YICpxisWaPEh1choHNIgzaJGZjJ\nKjGIsqYqv6j8wXGDRqVBxGDKFDUC8iOKZcDRxYBmPptWeuNQptSsWU4x0N1EUQLIbu3VZ3p7MXmy\nM5uIjifoilPkZuLnN5FQawMTvFPh9Woow6qnRwU5H33UjsFwZsxQ58ZLAHlnzZeB9PKpA86AqpsY\n8BnohJ5aSsdEs939xCCom8itQ+SuKa/j0yHxO/98VT2AaGlRghxFDNwK1fFBIL+P9+4FfvUr9Tk6\ndtonCYiIgZmschP5+Yd1Zs1ymvymDCOd2bPVq+6SMgkIpWP64ebbDioGRUV2DfnycntmJ7XRzWqh\nEfb8+erzfgHkuXOdn4/iJpo+3faxe7njZs0C6uudlgEvTBYU+vzbbwNbt6r3+HXhHR63ICieQmVO\nPvtZ8/7J0gniJqLF7IeHbTGg62b6LO/8Tp1y1nkiqFSCm5uI7k8qTUExA13YafIcrcOtnxsOiYFb\nEJVbBmHEgPbb2qruQercg4hBX5/qoINaBvxa624iGkTwjCzAmRQiYpBKVolB2Kni777rDMQFsQz0\nDomPgqLCH0AOzybyepiSSeXOoIkyJAZ+AVbyha9aleomMgWQFy92ft5NDLzaS2JAhe7caGxU21Bp\nDBKPpibbbRME6igfeUT90PERvFPh14CLgRfc7eVnGdCqaSdO2G4irzkKvEyEm2VAYmCqWkrHR1Bn\nqs8zoO8ii8FPDHj2jgndTeRlKXJ4J0tuwd/+Nj7LgA9AuGVgyiai2k5A6rkUy8BMVrmJwnQSgL0A\nCsFLE3vR3m53jDzFNCo8K4ND2UR+DxN3E02cqAK0ei18E5WV6qGorbUDyBMmmLOJaB0BThTLoLIy\nmJuIHkZ6mGmfc+aEiwmRGPCBQlub/TvvVPg2JSUqdVKfdKgzdaq/ZUBtOHZM/d3YaJ5noBNEDBYu\n9A4gX3CBfS8EcRNRQgTgLgZkZbhZvlHdRKWlaqBAny8rA66/3l8MvALIbpaB7ibiM/F7epxiwJfA\npHZJamkqWWUZPPCAvUhHFNzWTNbhHUQclsFtt5kfurAxA3ITHT3qdClQmQM3yDwfGFBxDpObCIgn\nm2jSJHtpQr+YAaD2091tLvUQpIokdcT84XVzE/FBQEmJ6oT8xIA6Bi/LgDoknsbLYwZu15d3flRq\ngdPRoa7N7t3uAeTPfEb98P35iQFPMeVuVE5pqTPIywkSQDZRVqaEcvp0VcJ7yhSVaXTmjBLCOGIG\npmtNYsJjZlQVgPPTnyrrbu9esQxMZJUYzJ0LfPrT0T9v8sn6EYdl8I//aH4/SgC5rEyJAVku1dV2\nimmQz0+dqh5I3TIAUsWARnI6XmJABd3ogfMjmbTFIAokBtw1xBdeoW2A1LpLQSwD6oj8LIOBAbsD\nIZccuScKC91rGlEHdfSoWiyIQy4sr5gBx2+eAb1PYumVMl1aqiy8+npg377U/wHeqaVu+zx4UD2H\n8+er9yZPVmJA1oifGOjxPrdCdXffbQ8ceTVXQO1HTzwAgD/9U/X62mvuLrLxTFaJQbrccIO9VGNQ\n4hADN8KIQV+f6mRJDD70IfW/I0f8v4cvvTllCvDGG6pjoIeNOp043ERea0CYoOC4adtEwt864GLw\n+OPAq6863SB61Voiihi4HTNdOxKkri7bMujt9Q48kxg0NdmpwqbtgooBWQb6XI0wMQPAtgy+8Q3g\nssuc/+P5+W7lKNz2efiw093Lv5/aT7Wz+CzsREJZwH6WAf2/vNxejpViNyQGnZ3ec1SmTrVn7gs2\nWRUzSJfCwvCLXYcpgRGlPUHFgPyctIJUkJgB/zwtsFNSYld/9bMMoogBYItLWDeRThg3UWen6tjp\nWAG7rj9x//20hnZ4MfByE1E7yF1HC8b7rZ5FYnDjjcDzzwNLl7rvm7uJvALSbm4imnRWVhZcDGht\nAx26X6ZODZ9aeuiQndgAAMuXO9v/yiv2edPdeqdOmcXAVKiOQ9dQtwxWr07dH6CeLbcJd+OZvBKD\nKIQpjhcWt9pEOjR6pokyeszADz7iSibVA0A18AE7BXUsxIA6sDjcRBMnursaAOWXvuAC9XucbiJq\nB694ScLb2+svBocPq2qubu6+KJaByU1EWUZ+AWTALsNiSsPmiwF5Taoz7fP0aadl8M//bK/JXFxs\nu6SCigG3rtyOR3cT0Rrnd9/tdC8SNLdEcDLuxeBP/kSZypmwDNxqE+mQZUBT64HwlgG5iZJJ1Wme\nOpUqdLpfWxeDd94B/vf/Di4GQeYL8IJiUSAxMK0D7NXZBU0tpUwUvw6P0n8B2zLwcxNRW9va1Cpj\nfsdIeM1b8JpnQO9TaXA/MeCvHBJQsgxMqaxe++SWQXm5HSspLraXZeVL1gLqdz/LwG1Q4RYzcGPO\nHBVXE5yMezGYNQv42tcys+8wbqLubuesySiWARcD04xjUzkKLgZXXqmqhvqlwtKDFiRFlBcUM+GX\n+cUnrVGZalMZZJ2glgG1z6tTp+3a2+1squFh1ZYgbqLjx53FEk37dssm4gTJJqLJVjTr208MglgG\nprWhTdC+uBjo7eeLywS1DHjNrjBuIjcWLlTrNkepeJDPjHsxyCRhYwa8nkoYy4AHkIuLVcfz3nup\nHaVejoEvcgLYJrWfZRBmuUA/yyCMGND5cQsgc4KKAW175kwwy2D6dPVaVKSuaZAAsqlOv75dEDcR\nudzcJp2RyyqMGJhG/CQGFRW2ZRBEDChA7jZfSG+LLgaW5R1A5uWo9f3Qoj90v3hdywkTlMvu7be9\nj2e8IWKQQcJaBsmk3bGYJih5fZ5bBjU1Sgz4A3z4sApkcnQ3EV+zOS4xoONJN2ZAHZseM/Dq7Pr6\ngosBZaB4tYOLQTKprmkQy8DLgqF9x2kZ0OAgqmWQTKoJdlSIz+SWMrFwoXpdudL8f7c1QwDbnedl\nGZiOG3C6ieh6+8U4li61YxmCQsQgg0RxE1Eue5CAHcEffhIDwPkAf+ADqZ24LgaU3eM3sgo7g5ja\naMJPWHTLgGcT+cUMAPdJV/q2FHT0akd7u9MyKCryDiBz951XzCTOeQaFhfY5CnJ+3GIBs2apfYWx\nDGbPVktgup1zvS382lMn7mUZdHebxYu7iWg/foOPujp7VTpBIWKQQXgJZq9Oj7uJovgxTZYBYE6r\n4+hiwH2zfmZ2UEg4vOYteJGOmwgIVuIkrJuovT24m4gm53kdp1fVUk6cMQPqYP0EMEzMAFDrdLjh\nJ4hA6vUMaxmQi8tvMFVXJ3MNdEQMMkjQbCKeWhpkJKujxwxIDPw6Qj2AzBdU93qYaFnPIPgV2wsi\nBjxTKmg2EbnZgogBTVoKahnQOttB3ESdnf7iyUe/tC501HkGQd1EQWrzhLUM/PASA7eKxWFiBmHc\nRJ/5jFoSV7DJqxnI2UaUSWdf+Yr36Mrt89wyoFIWtNyjG9wysCxlhlO2jNfDtHmzKkAWto0mgogB\n5bqTGASZVEUBW79zANiWgZclxWMGp0+rzjiIZRBEDPjol+4V03mJ000UZLU5sjCCxgz8oLZMmpRa\n3NFtAiI/N5RerBPFTRQ11TmfETHIIGHEgJY/XLQI+G//Ldz36DGDMGJAI0QqGVFe7i8GpaXuQUI3\n3PbnFzOg0WlYy2DDBhVEDxLfoJiB1wps1MFOn65SIKuq1DEND8djGVCH5xWv4W4it3kGQd1Et9/u\nPwOc7o/mZu/U2KBQW265BXjsMef/vvxl872gWwZebqLeXvcJloI/4ibKIGHEgL+GRbcMCgrUg+43\n+uGWAVWMLCnxdxNFIZ2YgW4ZcDFwO2fz5wPf+lawtgUNIANKDNra1Gf84iHFxWoEHMZN5CUGXm4i\nnlrKZ6S73QOrVgE//rF/u4qKVNZNWGvVBLXl9tvttSGI9euBXbtSP0OWgWWZK78CTjcRZSXJyD88\nIgYZJKwYRL2B9ZhBUNzEwM8yiEI6qaW0CDyNeoNYBmEIcszUfpr/QW4i/j/TZ8K6ifwsA+r00rUM\ngjIwoNxjYQtAmuBuoqDQ9W5rsydU6nA3EV8ZTgiHnLIMErQ2EXUmYbJ09M9zyyAo1OkMDqaKQZj0\n0TDfpRMktZSC64mE0zIIK35uhLEMyF0SxDIIKgY0aKDlNINYGvp5Sybt+yzq4MCNurp4BgckAmHE\ngMqFHDniXvWVWwZ0rv0sTiGVyJf49OnTuPbaa/Huu+9i3rx5+MUvfoEphqjUvHnzMHnyZBQWFiKZ\nTGLv3r1pNTiXCJNNBKRnGUQRA8DOKCIxoA4sSOA1DFEDyIWFthgAwWMGYQiSWkr/IzdFSYn9Xrpi\nkEiobegaeolBe7vZb07nh1tPcZ2fIMHmIKxZo2oTea1RrkOj/jfeSF22leBiILGC6ES2DLZu3Yp1\n69bhzTffxCWXXIKttFq5RiKRQENDA/bv3z+uhAAYfTdRlOqgZWXOxcspzTKbYgY8YydoNlEYghwz\nrZpF2SrcMnBrQzKpRCZIG4NUT00mg4mBPms7XaKkO5tIJMIPMui8vP22PcNZh4S0t1csg3SILAY7\nduzAje/XN7jxxhvxb//2b67bWkEK1+choyUGUd1EgBrpnjrlFAOaDR0ncbiJgMxYBiQGQfZFnU0y\n6T5RiiguDmYZAMEW2SkuVv57LzGg1NI4xSCOeEFUaNR/6pR78UZaa5sH6/UlLwV/Ij/yLS0tqHp/\nRk9VVRVaqDatRiKRwEc/+lEUFhZi06ZNuNWllu8999wz8nt9fT3q6+ujNi1rCCoGZNpGjRlQxk2U\nh3/6dJWlQWLQ22vPsI0TN39vEMtgNMSguzvYDFlqb0mJvb2XZRC3GLilV1IbeJkMIP3Yzw9/qKrZ\njhXkQmtpAc46y327CROUUNK5Pn58dNo32jQ0NKChoSEj+/Z85NetW4dmPQcMwL333uv4O5FIIOHy\nVL/00kuYPXs2Tpw4gXXr1qGurg5r165N2Y6LQb4QVgyidsCJhLPYXRh0MaDJZ3GKQTIJbNxo/l+Q\nZS+5GGQim4g6EK996RYMBWq9PhdFDCzLWwyAYG4iP3ELyl/8Rfr7SJeSElU4zyt2QWs/T5ig7uM4\n5kVkI/pAefPmzbHt2/OR3717t+v/qqqq0NzcjFmzZuHYsWOY6eIMnD17NgBgxowZuPrqq7F3716j\nGOQjQcWAdDQd07a4OPgi9RxyE/X1qYeovz9+MXCrt/TlL/tXFaUAMu94uRjEsUJdEDHQF1CnOQ/8\n8zok0GHEoLDQOyBN27r9jwLIcYlBNlBaquYleIlBRYVazGjCBGUVxG3Zjgcixww2btyIRx55BADw\nyCOP4KqrrkrZpru7G2fen3fe1dWFXbt2YTlfFDXPCSoGBJWQjgIvihYGsgza29UDVVycmZiBiQce\nAP72b723GS03Ee3bDVqUBVArwn372/6fC5MyHGQt5iCWAYlJV1f+iEFJiRIDr0B2RYUSgZISlRSR\nL6K5jcEAAA/SSURBVMc+mkQWg7vvvhu7d+/G4sWL8fzzz+Puu+8GABw9ehQbNmwAADQ3N2Pt2rU4\n77zzsGbNGlxxxRW49NJL42l5DhBWDHjlyrCQZRA1ZtDRMfpiEASTGFA2UVx59EHE4Ec/Av7lX9Tv\n8+erXHm/wL+f5cAJGjMAgqWW5pNlUFamRv1+lsHx49HjbkIaAeTKyko899xzKe/PmTMHv/71rwEA\nZ511Fl599dXorctxRlMMosYMZswA9u9XLqL587NfDChmYFnxzjMAvPf1oQ+lvkfX1C0jKoplUFTk\nnZ0EjG7MIBsgV6KfGHR1iRikg8xAziBhxSBIuWU3osYM5s4F3n3X6SYaGIh/BnJUdDGgNMI4Uyd5\numicRBEDr2Oi/QURg3xyE9GMZT8xAEQM0iFLxn/5CaV8BhGDDRtUNceo0ASnsB3aWWep1akKCpSV\nwFMUswFTmei4Z9gGcRNFIYoY8JRVt/2ZSm3r8wzyyTIgy8vrPIoYpI9YBhmEyi8HEYNf/Qo477zo\n3xXVTVRbqyyBF14Azj3XfpiyRQx0ywAIVpUzDNS5xt15RokZeK3rTFlnbusWA/npJuILMLkhpavT\nJ0se+fwkrJsoHSibKGwHkEgABw6oST1VVdlnGVDHxlNQ4xaDdMseu6XHxu0mItyWxKT/5ZubKMhx\nkBh4rUkheJMlj3x+MppiUFzsXdfGi5kz7Zox2SgGJssgai0mE+mKgSm4DEQXA7/t9VXC+Hflo5vo\n7//efzEliifQcqdCeLLkkc9PRtsy4K9R8ZtVO9roVUuBzFkGUSq1dnS4l2TOlGVgusamAHIcE/Ky\ngTVr1I8Xc+aoV7EMoiNikEFEDNJHr1oKqLY9/bR6P452Uqf5/mT5UHjV5o8iBl4xA0BNeDNlnfHZ\nyeRaG08dI9UtkkVtoiNikEFG203EX9PdT7ZkZZjcRMkkcMcd6vc4xGDGDDWyjDu1NGwAua3N3zKY\nP9/8PheefHMTBWHxYucscSE8oqMZJBctg0ylWUbFJAa8JqIpzTIskycDTU3p70cnrGXQ0xMsZuD1\nXWQZ5FMAOShxLcIzXhExyCC5KAbZ6CbSA+MdHfbvYZZQHG2iikGUa2hyE2XLNRRyAxGDDCJikD6m\nRefb2+3f80UMysqUGNDEs6jfVVIyPt1EQvqIGGSQsYgZ5JsYmNaH5qmV2SwGFJgOsmxkaakSA7fF\na/zgwkNB92y5hkJuIAHkDDIWlkFcAeRs6UhIDPRsop4e9Xs2zzgtKVGL2wSJa6QrBnS9yDLg7wlC\nEMQyyCBhahOlS75bBvy4zj1XvV50UfYvfB40wB23ZQBkzzUUcgOxDDII1SYaHs58/jP5pdMVnWzL\nJjLFDHbtUoHW6dPHpk2ZIF0xIFEsLhYxEKIhYpBBxsJNlC65YBn4LZWZi6QrBoBKjy0rEzeREA1x\nE2WQ0RQDv4Xlg7J8uVrScdasePaXLiYxyEfiEAMqySCWgRAFsQwyyGiKAS0FmS5FRcBf/VU8+4oD\nEYPwiBgIURDLIIPkohhkG+OlY+NiYFqvIAziJhKiIGKQQUZTDPr7M7v/scIUQM5HSAy6utIvsTFe\nBFSIFxGDDDKaYkDplvnGeHMTdXWlbxmMl3MmxIuIQQYZTTH4y79UKaz5BnVs2Ty5LA6KilT6cXt7\n+pYBiUC+rGcgjA4SQM4goykG2T75KiokBumOlnOB0lJVxjouy2A8nDMhPsQyyCCjKQb5Co1ux0PH\nRtZPXAHk8XDOhPgQMcggNAN5cDB71hTONWiC2Xjo2Mi6S3e2+ngSUCE+RAwySFGRyvIpKJDl+KIy\nnsQgrpgPLQo/Hs6ZEB/SRWWQwsLU9XuFcEycqF6zZRnOTBLXLHIqmS0BZCEMIgYZRMQgfciiGhoa\n23aMBnFZBiSc+ZhdJmQOEYMMImIQD1u2qAXP8524LAMA2LYNWLQovv0J+U9kMfjlL3+Js88+G4WF\nhdi3b5/rdjt37kRdXR0WLVqE++67L+rX5SQiBvHw1a+Oj9m0cY7kb79d7jshHJHFYPny5di+fTs+\n8pGPuG4zNDSEL3zhC9i5cycOHDiAxx57DK+//nrUr8w5SAwkk0gIQpyWgSCEJbIY1NXVYbGP7b53\n714sXLgQ8+bNQzKZxHXXXYennnoq6lfmHGIZCGHgazsLwmiT0TFrU1MTamtrR/6uqanBK6+8Ytz2\nnnvuGfm9vr4e9fX1mWzaqFBYCPT1AVOmjHVLhFzgF7/Ir9XbhPhpaGhAQ0NDRvbtKQbr1q1Dc3Nz\nyvtbtmzBlVde6bvzRIgaCVwM8gWxDIQw/Jf/MtYtELIdfaC8efPm2PbtKQa7d+9Oa+fV1dVobGwc\n+buxsRE1NTVp7TOXKCxU6wyIGAiCkO3EklpquUS+Vq9ejYMHD+Lw4cPo7+/HE088gY0bN8bxlTnB\neKnFLwhC7hNZDLZv347a2lrs2bMHGzZswPr16wEAR48exYYNGwAARUVF2LZtGy677DIsW7YM1157\nLZYuXRpPy3MAEQNBEHKFhOU2rB/NRiQSrtZFLvPWW2riz9q1wAsvjHVrBEHIN+LsO2UGcgYRy0AQ\nhFxBxCCDkBhIwTBBELIdEYMMMl6WbBQEIfcRMcggVD1SLANBELIdEYMMQiIgYiAIQrYjYpBByD00\nHhZmEQQhtxExyCCy1KUgCLmCdFeCIAiCiMFokIfz6QRByDNEDEaBGTPGugWCIAjeiBiMAhdeONYt\nEARB8EYWZMwww8NAiGUdBEEQxgSxDDKMCIEgCLmAiIEgCIIgYiAIgiCIGAiCIAgQMRAEQRAgYiAI\ngiBAxEAQBEGAiIEgCIIAEQNBEAQBIgaCIAgCRAwEQRAEiBgIgiAIEDEQBEEQIGIgCIIgQMRAEARB\ngIjBqNDQ0DDWTcgY+XxsgBxfrpPvxxcnkcXgl7/8Jc4++2wUFhZi3759rtvNmzcP5557LlauXIkP\nfvCDUb8up8nnGzKfjw2Q48t18v344iTySmfLly/H9u3bsWnTJs/tEokEGhoaUFlZGfWrBEEQhAwT\nWQzq6uoCb2tZVtSvEQRBEEaBhJVmT33RRRfh29/+NlatWmX8/1lnnYWKigoUFhZi06ZNuPXWW1Mb\nIWtDCoIgRCKuwbanZbBu3To0NzenvL9lyxZceeWVgb7gpZdewuzZs3HixAmsW7cOdXV1WLt2rWMb\nsRwEQRDGFk8x2L17d9pfMHv2bADAjBkzcPXVV2Pv3r0pYiAIgiCMLbGklrqN7Lu7u3HmzBkAQFdX\nF3bt2oXly5fH8ZWCIAhCjEQWg+3bt6O2thZ79uzBhg0bsH79egDA0aNHsWHDBgBAc3Mz1q5di/PO\nOw9r1qzBFVdcgUsvvTSelguCIAjxYY0xzzzzjLVkyRJr4cKF1tatW8e6OaF57733rPr6emvZsmXW\n2WefbX3ve9+zLMuyTp06ZX30ox+1Fi1aZK1bt85qbW0d+cyWLVushQsXWkuWLLGeffbZsWp6KAYH\nB63zzjvPuuKKKyzLyq/ja21tta655hqrrq7OWrp0qbVnz568Or4tW7ZYy5Yts8455xzr+uuvt3p7\ne3P2+G666SZr5syZ1jnnnDPyXpRj+f3vf2+dc8451sKFC60777xzVI/BC9PxfeUrX7Hq6uqsc889\n17r66quttra2kf/FeXxjKgaDg4PWggULrEOHDln9/f3WihUrrAMHDoxlk0Jz7Ngxa//+/ZZlWdaZ\nM2esxYsXWwcOHLD++q//2rrvvvssy7KsrVu3WnfddZdlWZb1xz/+0VqxYoXV399vHTp0yFqwYIE1\nNDQ0Zu0Pyre//W3rU5/6lHXllVdalmXl1fHdcMMN1sMPP2xZlmUNDAxYbW1teXN8hw4dsubPn2/1\n9vZalmVZn/zkJ62f/vSnOXt8L7zwgrVv3z5HZxnmWIaHhy3Lsqzzzz/feuWVVyzLsqz169dbzzzz\nzCgfiRnT8e3atWvkGtx1110ZO74xFYOXX37Zuuyyy0b+/uY3v2l985vfHMMWpc/HPvYxa/fu3daS\nJUus5uZmy7KUYCxZssSyLKXk3AK67LLLrN/97ndj0tagNDY2Wpdccon1/PPPj1gG+XJ8bW1t1vz5\n81Pez5fjO3XqlLV48WLr9OnT1sDAgHXFFVdYu3btyunjO3TokKOzDHssR48eterq6kbef+yxx6xN\nmzaNUuv90Y+P86//+q/Wpz/9acuy4j++Ma1N1NTUhNra2pG/a2pq0NTUNIYtSo/Dhw9j//79WLNm\nDVpaWlBVVQUAqKqqQktLCwAVU6mpqRn5TC4c85e+9CXcf//9KCiwb5d8Ob5Dhw5hxowZuOmmm7Bq\n1Srceuut6Orqypvjq6ysxJe//GXMnTsXc+bMwZQpU7Bu3bq8OT4g/L2ov19dXZ31x0j8+Mc/xuWX\nXw4g/uMbUzHIp8lmnZ2duOaaa/C9730PkyZNcvwvkUh4Hms2n4df/epXmDlzJlauXOmaNZbLxzc4\nOIh9+/bh85//PPbt24fy8nJs3brVsU0uH9/bb7+N7373uzh8+DCOHj2Kzs5O/NM//ZNjm1w+Ph2/\nY8ll7r33XhQXF+NTn/pURvY/pmJQXV2NxsbGkb8bGxsdipYrDAwM4JprrsFnP/tZXHXVVQDUCIUm\n7B07dgwzZ84EkHrMR44cQXV19eg3OiAvv/wyduzYgfnz5+P666/H888/j89+9rN5c3w1NTWoqanB\n+eefDwD4xCc+gX379mHWrFl5cXy///3v8eEPfxjTpk1DUVERPv7xj+N3v/td3hwfEO5Zq6mpQXV1\nNY4cOeJ4P9uP8ac//Smefvpp/PznPx95L+7jG1MxWL16NQ4ePIjDhw+jv78fTzzxBDZu3DiWTQqN\nZVm45ZZbsGzZMnzxi18ceX/jxo145JFHAACPPPLIiEhs3LgRjz/+OPr7+3Ho0CEcPHgwq6u5btmy\nBY2NjTh06BAef/xxXHzxxfjZz36WN8c3a9Ys1NbW4s033wQAPPfcczj77LNx5ZVX5sXx1dXVYc+e\nPejp6YFlWXjuueewbNmyvDk+IPyzNmvWLEyePBmvvPIKLMvCz372s5HPZCM7d+7E/fffj6eeegol\nJSUj78d+fNHDHPHw9NNPW4sXL7YWLFhgbdmyZaybE5oXX3zRSiQS1ooVK6zzzjvPOu+886xnnnnG\nOnXqlHXJJZcY093uvfdea8GCBdaSJUusnTt3jmHrw9HQ0DCSTZRPx/fqq69aq1evdqTu5dPx3Xff\nfSOppTfccIPV39+fs8d33XXXWbNnz7aSyaRVU1Nj/fjHP450LJR6uWDBAuuOO+4Yi0Mxoh/fww8/\nbC1cuNCaO3fuSP9y2223jWwf5/GlXahOEARByH1kpTNBEARBxEAQBEEQMRAEQRAgYiAIgiBAxEAQ\nBEGAiIEgCIIA4P8DQWYb5ZVmE00AAAAASUVORK5CYII=\n" 39 | } 40 | ], 41 | "prompt_number": 3 42 | }, 43 | { 44 | "cell_type": "code", 45 | "collapsed": true, 46 | "input": [ 47 | "X = fft(s)", 48 | "E = fft(s[::2])", 49 | "O = fft(s[1::2])" 50 | ], 51 | "language": "python", 52 | "outputs": [], 53 | "prompt_number": 9 54 | }, 55 | { 56 | "cell_type": "code", 57 | "collapsed": false, 58 | "input": [ 59 | "plot(abs((X[:512]-E)/O))", 60 | "plot(angle((X[:512]-E)/O))" 61 | ], 62 | "language": "python", 63 | "outputs": [ 64 | { 65 | "output_type": "pyout", 66 | "prompt_number": 12, 67 | "text": [ 68 | "[]" 69 | ] 70 | }, 71 | { 72 | "output_type": "display_data", 73 | "png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD9CAYAAABHnDf0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFM5JREFUeJzt3XtwlPW9x/HPJsELYDlaICiJwISQkBCSkGBOvXVpGqhg\nuAxxEEc8AxRI/MPSdpxeHekfhGCHcaS2G0EuAo5QC6kOSgoKKxytQC6oXAQqiQQwWJCcIRGbEH/n\nD5ttApFL9tns7m/frxlmyGb3eX4/Jr5Zn90v6zLGGAEArBIV7AUAAJxH3AHAQsQdACxE3AHAQsQd\nACxE3AHAQn7Fva6uTmPGjFFqaqpGjBihpUuXOrUuAIAfXP68z72+vl719fXKyMhQY2OjsrKy9Ne/\n/lXDhw93co0AgOvk1zP3AQMGKCMjQ5LUu3dvDR8+XKdOnXJkYQCArotx6kC1tbWqrq5WTk6O7zaX\ny+XU4QEgovj7jwc48oJqY2OjCgoK9Nxzz6l3794dvmeMsfbX008/HfQ1sD/2F2l7i4T9OcHvuLe0\ntGjq1Kl69NFHNXnyZCfWBADwk19xN8Zo9uzZSklJ0fz5851aEwDAT37F/d1339W6deu0Y8cOZWZm\nKjMzU+Xl5U6tLeS53e5gLyGg2F/4snlvkv37c4Jfb4W86sFdLseuHwFApHCinUyoAoCFiDsAWIi4\nA4CFiDsAWIi4A4CFiDsAWIi4A4CFiDsAWIi4A4CFiDsAWIi4A4CFiDsAWIi4A4CFiDsAWIi4A4CF\niDsAWIi4A4CFiDsAWIi4A4CFiDsAWIi4A4CFiDsAWIi4A4CFiDsAWIi4A4CFiDsAWIi4A4CFiDsA\nWMivuM+aNUuxsbFKS0tzaj0AAAf4FfeZM2eqvLzcqbUAABziV9zvu+8+3XrrrU6tBQDgkJhAn8Dl\nWtDuK/e/fwFAYBgT7BVcP6/XK6/X6+gxXcb490dRW1ur/Px8ffTRR5cf3OWSn4cHgIjjRDt5twwA\nWIi4A4CF/Ir79OnTdffdd+vIkSOKj4/XqlWrnFoXAMAPfl9zv+LBueYOANeNa+4AgE4RdwCwEHEH\nAAsRdwCwEHEHAAsRdwCwEHEHAAsRdwCwEHEHAAsRdwCwEHEHAAsRdwCwEHEHAAsRdwCwEHEHAAsR\ndwCwEHEHAAsRdwCwEHEHAAsRdwCwEHEHAAsRdwCwEHEHAAsRdwCwEHEHAAsRdwCwEHEHAAsRdwCw\nkN9xLy8vV3JyshITE7V48WIn1gQA8JPLGGO6+uDW1lYlJSXprbfe0sCBAzV69Gi98sorGj58+DcH\nd7nkx+EBICI50U6/nrnv2bNHQ4cO1eDBg9WjRw89/PDDeu211/xaEADAfzH+PPjkyZOKj4/3fR0X\nF6fdu3d3uM/4OeOVHpuuG2NulNvtltvt9ueUAGAdr9crr9fr6DH9irvL5brqfXqP7a3SY6V6KOUh\nfSfpO/6cDgCsdOkT39/97nd+H9OvyzIDBw5UXV2d7+u6ujrFxcV1uM+fH/qzDj5+UIP6DNKUDVOU\n82KOVu9brQstF/w5NQDgCvx6QfXixYtKSkrS22+/rTvuuEN33XXXFV9Qbf26VW8efVOllaXafWK3\nHkt/TPOy5impb5L/OwEASwT9BdWYmBg9//zzGjdunFJSUjRt2jRf2DsTHRWt/KR8vfHIG9o7Z69u\njLlR96++X7lrcvWXg39RS2uLP8sBAPybX8/cr3rwa/jb518X/6Wyj8vkqfDo6Nmjmj1qtuaOmqv4\nPvFXfBwA2MqJZ+5Bj3t7Bz4/oNLKUr384cu69857VZRdpHFDxynKxSAtgMhhXdzbNDU36ZX9r8hT\n4dG5C+c0L2ueZmXOUr9e/QKwSgAILdbGvY0xRhWnKuSp8Kjs4zI9MPQBFWUX6d47772mt2ECQDiy\nPu7tnbtwTms+WCNPhUcxUTEqzC7UjJEz1OemPo4cHwBCRUTFvY0xRt5arzwVHm07tk0PpTykwuxC\njbp9lKPnAYBgici4t1ffWK8VVSu0rGqZBvQeoKLsIk1Lnaabe9wcsHMCQKBFfNzbXDocNSN9hgqz\nChmOAhCWiHsnas7VaFnVMq2sXqkR/UeoKLtIk5ImqUd0j25dBwB0FXG/AoajAIQr4n6NDnx+QC9U\nvqCXP3pZ98Tfw3AUgJBG3K9TZ8NRMzNnqn+v/sFeGgD4EPcuaj8ctenQJo1PHM9wFICQQdwdwHAU\ngFBD3B3EcBSAUEHcA4ThKADBRNwDrPXrVm35xxZ5KjwMRwHoNsS9G106HFWYVajJyZMZjgLgOOIe\nBAxHAQg04h5kDEcBCATiHiIYjgLgJOIegvae3MsnRwHwC3EPYQxHAegq4h4GLh2OKkgpUFF2EcNR\nAL4VcQ8zDEcBuBbEPUwxHAXgSoi7BWrO1Wh51XKtrF6p1P6pDEcBIO42aW5t1qZDm1RaUaojZ49o\n9qjZmjNqju7sc2ewlwagmxF3Sx3850GVVpQyHAVEKCfa2eVavPrqq0pNTVV0dLSqqqr8WgQ6SumX\noqUPLNXx+cc1MWmifrvjtxq6dKgW/+9ifd70ebCXByAMdDnuaWlpKisr0/333+/ketBOrxt66cej\nfqzKuZXaULBBh88eVtLzSXpk4yPa9eku/q8IwLeK6eoDk5OTnVwHrmL0wNEaPXC0loxdojUfrNHc\nzXMV7YpmOApAp7oc92u1YMEC3+/dbrfcbnegT2m1W2++VT/575/oiZwn9M6n78hT4dFTO55iOAoI\nY16vV16v19FjXvEF1by8PNXX1192e3FxsfLz8yVJY8aM0ZIlSzRq1OVR4QXV7nHpcFRhVqGmjZim\nnj16BntpALogJN4tQ9xDB8NRgB2C+m6Z9gh4aIiOitaDwx7UG4+8ob1z9urmmJv1/dXfV+6aXL16\n4FW1tLYEe4kAukmXn7mXlZXpiSee0JkzZ9SnTx9lZmZqy5YtHQ/OM/egYzgKCD8hcVnmigcn7iGl\ns+GosQljFR0VHeylAWiHuKNLmpqbtH7/enkqPDp74azmZc3TrMxZfHIUECKIO/zW9slRmw5t0vjE\n8SrMLtR9d97HJ0cBQUTc4Zi2T44qrSxlOAoIMuIOxxljfMNRWz/ZynAUEATEHQHFcBQQHMQd3YLh\nKKB7EXd0u9qGWi2rXKaV1SuV0i9FRdlFfHIU4DDijqBhOAoIHOKOkMBwFOAs4o6QwnAU4AzijpC1\n9+RelVaWatOhTXpg6AMMRwHXgbgj5DEcBVw/4o6wwXAUcO2IO8ISw1HAlRF3hDWGo4DOEXdYg+Eo\n4D+IO6zT3NqsskNl8lR4dPjsYc3OnK25WXMZjkJEIe6wGsNRiFTEHRGB4ShEGuKOiMNwFCIBcUfE\nYjgKNiPuiHgMR8FGxB1op76xXiurV+qFyhcU2ytWRdlFDEchLBF3oBMMRyHcEXfgKhiOQjgi7sA1\nYjgK4YS4A11w8J8H9ULlC1r34TrdE3+PCrMLNS5hHMNRCBnEHfADw1EIVUGN+5NPPqnNmzfrhhtu\nUEJCglatWqU+fTq+x5i4I1xUnKqQp8LDcBRCQlDjvm3bNuXm5ioqKkq//OUvJUklJSWOLxDoTgxH\nIRSEzGWZsrIybdy4UevWret4cOKOMMVwFILJiXbGOLGQlStXavr06Z1+b8GCBb7fu91uud1uJ04J\nBJTL5ZJ7sFvuwW7fcNSUDVMYjkJAeL1eeb1eR495xWfueXl5qq+vv+z24uJi5efnS5IWLlyoqqoq\nbdy48fKD88wdFmkbjiqtKNX7J97XjPQZmpc1T8l9k4O9NFgm6JdlVq9ereXLl+vtt9/WTTfdFJAF\nAqGI4SgEUlDjXl5erp///Od655131Ldv34AtEAhlDEchEIIa98TERDU3N+u2226TJH3ve9/Tn/70\nJ8cXCIQLhqPglKBflrnqwYk7IhDDUfAXcQdCXPvhqB8N/ZGKsosYjsJVEXcgTJy7cE5rP1wrT4VH\nUa4oFWYV6rH0xxiOQqeIOxBm2oajSitK9bdP/sZwFDpF3IEwdrrxtFZUr+CTo3AZ4g5Y4NLhqEdH\nPqrC7EKGoyIYcQcs09lw1KTkSboh+oZgLw3diLgDlmI4KrIRdyACHPrnIZVWljIcFUGIOxBBGI6K\nHMQdiFAMR9mNuAMRjuEoOxF3AJIYjrINcQdwmbbhqGWVy9S/V3+Go8IQcQfwrVq/blX5P8rlqfAw\nHBVmiDuAa1LbUKvlVcu1omoFw1FhgLgDuC4MR4UH4g6gy9oPR90df7eKsosYjgoRxB2A35qam7Th\nwAZ5Kjw68+UZzR01V7NHzWY4KoiIOwBHVZyqUGlFqTYe2shwVBARdwAB0fBVg9Z8sIbhqCAh7gAC\nqrPhqMKsQmXdkRXspVmNuAPoNgxHdR/iDqDbMRwVeMQdQFB92vCpllUtYzjKYcQdQEhoG44qrSzV\nx2c+ZjjKT8QdQMhhOMp/xB1AyGI4quuIO4CwwHDU9SHuAMJK23BUaUWpXC4Xw1HfIqhxf+qpp/T6\n66/L5XLpu9/9rlavXq34+HjHFwjAPsYY7fx0pzwVHoajOhHUuJ8/f1633HKLJOkPf/iDPvjgA734\n4ouOLxCA3RiOupwT7Yzq6gPbwi5JjY2N6tu3r18LARCZYnvH6tf3/VqfPPGJnv7+09p4aKPufPZO\nzS+fr4/PfBzs5YWtGH8e/Jvf/EZr165Vz5499f7773d6nwULFvh+73a75Xa7/TklAEtFR0VrwrAJ\nmjBsgm84yr3areH9hqsou0iTkydbOxzl9Xrl9XodPeYVL8vk5eWpvr7+stuLi4uVn5/v+7qkpESH\nDx/WqlWrOh6cyzIA/BCpw1Eh826Z48ePa/z48dq/f3/HgxN3AA6JpOGooMb96NGjSkxMlPTNC6p7\n9uzR2rVrHV8gALT3ZcuXWr9/vdXDUUGNe0FBgQ4fPqzo6GglJCTI4/Gof/+Of7jEHUAgXTocVZhV\nqPsH3R/2w1Ehc1nmWw9O3AF0g/bDUQN6D9D2/9ke7CX5hbgDQDvGGB3/v+Ma9F+Dgr0UvxB3ALBQ\nUIeYAAChi7gDgIWIOwBYiLgDgIWIOwBYiLgDgIWIOwBYiLgDgIWIOwBYiLgDgIWIOwBYiLgDgIWI\nOwBYiLgDgIWIOwBYiLgDgIWIOwBYiLgDgIWIOwBYiLgDgIWIOwBYiLgDgIWIOwBYiLgDgIWIOwBY\niLj7wev1BnsJAcX+wpfNe5Ps358T/I77kiVLFBUVpS+++MKJ9YQV23/A2F/4snlvkv37c4Jfca+r\nq9O2bds0aNAgp9YDAHCAX3H/2c9+pmeeecaptQAAHOIyxpiuPPC1116T1+vVs88+qyFDhqiyslK3\n3XZbx4O7XI4sEgAiTRfT7BNzpW/m5eWpvr7+stsXLlyoRYsWaevWrVdciL+LAwB0TZeeue/fv1+5\nubnq2bOnJOnEiRMaOHCg9uzZo/79+zu+SADA9enyZZn2vu2yDAAgOBx5nzvX1gEgtDgS92PHjl32\nrL28vFzJyclKTEzU4sWLnThNt5s1a5ZiY2OVlpbmu+2LL75QXl6ehg0bprFjx6qhocH3vUWLFikx\nMVHJyckdXo8IRXV1dRozZoxSU1M1YsQILV26VJI9+/vqq6+Uk5OjjIwMpaSk6Fe/+pUke/bXprW1\nVZmZmcrPz5dk1/4GDx6skSNHKjMzU3fddZcku/bX0NCggoICDR8+XCkpKdq9e7ez+zMBcPHiRZOQ\nkGBqampMc3OzSU9PNwcPHgzEqQJq586dpqqqyowYMcJ325NPPmkWL15sjDGmpKTE/OIXvzDGGHPg\nwAGTnp5umpubTU1NjUlISDCtra1BWfe1+Oyzz0x1dbUxxpjz58+bYcOGmYMHD1qzP2OMaWpqMsYY\n09LSYnJycsyuXbus2p8xxixZssQ88sgjJj8/3xhjz8+nMcYMHjzYnD17tsNtNu3vscceMytWrDDG\nfPMz2tDQ4Oj+AhL39957z4wbN8739aJFi8yiRYsCcaqAq6mp6RD3pKQkU19fb4z5JpBJSUnGGGOK\ni4tNSUmJ737jxo0zf//737t3sX6YNGmS2bZtm5X7a2pqMtnZ2Wb//v1W7a+urs7k5uaa7du3mwcf\nfNAYY9fP5+DBg82ZM2c63GbL/hoaGsyQIUMuu93J/QXk35Y5efKk4uPjfV/HxcXp5MmTgThVtzt9\n+rRiY2MlSbGxsTp9+rQk6dSpU4qLi/PdL5z2XFtbq+rqauXk5Fi1v6+//loZGRmKjY31XYKyaX8/\n/elP9fvf/15RUf/5z9im/blcLv3whz9Udna2li9fLsme/dXU1Khfv36aOXOmRo0apTlz5qipqcnR\n/QUk7pHyAqvL5briXsPhz6GxsVFTp07Vc889p1tuuaXD98J9f1FRUdq3b59OnDihnTt3aseOHR2+\nH87727x5s/r376/MzMxvnScJ5/1J0rvvvqvq6mpt2bJFf/zjH7Vr164O3w/n/V28eFFVVVV6/PHH\nVVVVpV69eqmkpKTDffzdX0DiPnDgQNXV1fm+rqur6/C3TjiLjY31DXZ99tlnvvf1X7rntvf+h7KW\nlhZNnTpVM2bM0OTJkyXZtb82ffr00YQJE1RZWWnN/t577z29/vrrGjJkiKZPn67t27drxowZ1uxP\nkm6//XZJUr9+/TRlyhTt2bPHmv3FxcUpLi5Oo0ePliQVFBSoqqpKAwYMcGx/AYl7dna2jh49qtra\nWjU3N2vDhg2aOHFiIE7V7SZOnKiXXnpJkvTSSy/5ojhx4kStX79ezc3Nqqmp0dGjR32v8IciY4xm\nz56tlJQUzZ8/33e7Lfs7c+aM750GFy5c0LZt25SZmWnN/oqLi1VXV6eamhqtX79eP/jBD7R27Vpr\n9vfll1/q/PnzkqSmpiZt3bpVaWlp1uxvwIABio+P15EjRyRJb731llJTU5Wfn+/c/hx7heASb775\nphk2bJhJSEgwxcXFgTpNQD388MPm9ttvNz169DBxcXFm5cqV5uzZsyY3N9ckJiaavLw8c+7cOd/9\nFy5caBISEkxSUpIpLy8P4sqvbteuXcblcpn09HSTkZFhMjIyzJYtW6zZ34cffmgyMzNNenq6SUtL\nM88884wxxlizv/a8Xq/v3TK27O/YsWMmPT3dpKenm9TUVF9DbNmfMcbs27fPZGdnm5EjR5opU6aY\nhoYGR/fnyIQqACC08ElMAGAh4g4AFiLuAGAh4g4AFiLuAGAh4g4AFvp/tNfW/7GWoj8AAAAASUVO\nRK5CYII=\n" 74 | } 75 | ], 76 | "prompt_number": 12 77 | }, 78 | { 79 | "cell_type": "code", 80 | "collapsed": false, 81 | "input": [ 82 | "plot(abs((X[512:]-E)/O))", 83 | "plot(angle((X[512:]-E)/O))" 84 | ], 85 | "language": "python", 86 | "outputs": [ 87 | { 88 | "output_type": "pyout", 89 | "prompt_number": 13, 90 | "text": [ 91 | "[]" 92 | ] 93 | }, 94 | { 95 | "output_type": "display_data", 96 | "png": "iVBORw0KGgoAAAANSUhEUgAAAXgAAAD9CAYAAAC2l2x5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XlYlOXeB/DvCLiMIIgL6ICiggioM8M2maiDoOYCrnVc\nQMINLBM75+2Yb50365hpebzSNEFMAyytDBQNKLdxIyBlUbOTKwkqlAsmLgeE+/2j9yVxQRSGh3n4\nfq6L62LmuXme7+1Vv/lxcz8zCiGEABERyU4zqQMQEZFxsMATEckUCzwRkUyxwBMRyRQLPBGRTLHA\nExHJVI0F/s6dO9DpdNBoNHB3d8eCBQseGGMwGGBtbQ2tVgutVotFixYZLSwREdWeeU0HW7Zsib17\n90KpVOLu3bvw8/PDwYMH4efnV23coEGDkJycbNSgRET0ZB67RKNUKgEAZWVlqKiogK2t7QNjeK8U\nEVHjU2MHDwCVlZXw9PTEmTNnMHv2bLi7u1c7rlAokJ6eDrVaDZVKhWXLlj10DBERPbk6NdCilkpK\nSoROpxN79+6t9vzvv/8ubt68KYQQIiUlRbi4uDzws09wGZP01ltvSR3BaOQ8NyE4P1Mn9/nVtXbW\neheNtbU1Ro4cicOHD1d73srKqmoZZ/jw4SgvL8fVq1ef/hWHiIjqRY0F/vLlyygpKQEA3L59Gzt3\n7oRWq602pri4uOpXiKysLAghHrpOT0REDavGNfhLly4hLCwMlZWVqKysRGhoKAICAhATEwMAiIiI\nwJYtW7BmzRqYm5tDqVRi8+bNDRK8MdHr9VJHMBo5zw3g/Eyd3OdXVwohjL8FRqFQcKcNEdETqmvt\n5J2sREQyxQJPRCRTLPBERDLFAk9EJFMs8EREMsUCT0QkUyzwREQyxQJPRCRTLPBERDLFAk9EJFMs\n8EREMsUCT0QkUyzwREQyxQJPRCRTLPBERDLFAk9EJFMNVuD5gR9ERA2rwQr8uC/H4bebvzXU5YiI\nmrwGK/Auti7QxGiQeiq1oS5JRNSkNehnshryDQjbGoZRPUfhgyEfQGmhNPaliYhMlkl9JqveSY+8\nyDyU3CmBZ4wnjlw80pCXJyJqUhq0g7/XpmObEJUWhXnPzMP8/vNh1szM2DGIiExKXTt4yQo8ABRc\nL0DY1jCUVZQhYWwCurXtZuwoREQmw6hLNHfu3IFOp4NGo4G7uzsWLFjw0HFz586Fi4sL1Go1cnJy\nan1xR2tH7Jq6C2N7jYXvOl/E5cZxOyURUT15bAd/69YtKJVK3L17F35+fli2bBn8/PyqjqekpGDV\nqlVISUlBZmYmoqKikJGRUf0itXgVOlp8FFMSp8C1nStiRsWgnbJdHaZFRGT66trBmz9ugFL5x06X\nsrIyVFRUwNbWttrx5ORkhIWFAQB0Oh1KSkpQXFwMOzu7auMWLlxY9b1er4der692vK9dX/ww8we8\nsecNqKPVWD96PYb2GPo0cyIiMkkGgwEGg6HezvfYDr6yshKenp44c+YMZs+ejffff7/a8aCgICxY\nsADPPvssACAwMBBLly6Fl5fXnxd5wleh3Wd348VtL2Kc2zgsCViCVhatnmRORESyYPRtks2aNUNu\nbi4KCwuxf//+h7663B9AoVA8dSAACOgegLzIPBSVFsE71hs5l2q/rk9ERH+o9T54a2trjBw5EocP\nH672vEqlQkFBQdXjwsJCqFSqOgezbWWLzeM3Y4HfAgzdOBRLDy5FRWVFnc9LRNRU1FjgL1++jJKS\nEgDA7du3sXPnTmi12mpjgoODER8fDwDIyMiAjY3NA+vvT0uhUCCkbwgOzzyMlNMp8I/zR35Jfr2c\nm4hI7mos8JcuXcLgwYOh0Wig0+kQFBSEgIAAxMTEICYmBgAwYsQIdO/eHc7OzoiIiMDHH39c7yG7\n2nTFnql7MKrnKPjE+iAhL4HbKYmIHkPSG52eRs6lHIQkhaB3x95YM3INbFvZPv6HiIhMkEm9F019\n0HbS4vDMw7C3tIc6Wo1dZ3dJHYmIqFEyuQ7+Xt+d+Q7Ttk3DCx4vYHHAYrQ0b1nv1yAikkqT6+Dv\nNbTHUORF5qHg9wL4xPogryhP6khERI2GSRd4AGinbIcvJ3yJ1559DYEJgViWvgyVolLqWEREkjPp\nJZr7nbt2DqFJoWhu1hxxY+LgaO1o9GsSERlLk16iuV+3tt2w78V9GNJ9CLzWemHTsU1SRyIikoys\nOvh7Hbl4BCFJIdDaa/HxyI9h09KmQa9PRFRX7OAfwauzF47MOgLbVrZQR6ux99xeqSMRETUo2Xbw\n90o9lYoZ22dgcp/JWOS/CC3MW0iWhYiottjB18Jwl+HIi8zD6aun4bvOF8d/PS51JCIio2sSBR4A\n2ivbI/GFRETpouAf548PMz7kdkoikrUmsURzvzNXzyA0KRRKCyU+HfMpHNo4SB2JiOgBXKJ5Cj1s\ne2B/+H4M6joIXmu98OWPX0odiYio3jXJDv5eWReyEJIYAp2DDquGr4J1S2upIxERAWAHX2e+Kl/k\nROTAsrkl1NFq7P9lv9SRiIjqRZPv4O+14+QOzNw+E2HqMLzj/w6amzWXOhIRNWF1rZ0s8Pf59eav\nmJE8AwW/F+CzcZ/BvYO71JGIqIniEk0969i6I7ZN3IaXvF/CwA0D8VHmR9xOSUQmiR18DU5dOYWQ\npBDYtLTBhtEb0Nmqs9SRiKgJYQdvRC7tXHAw/CD6OfSDNkaLr098LXUkIqJaYwdfS5mFmQhJCoFf\nFz+seG4F2rRoI3UkIpI5dvANROegQ05EDiyaWUATrcHB8weljkREVKMaC3xBQQH8/f3h4eGB3r17\nY+XKlQ+MMRgMsLa2hlarhVarxaJFi4wWVmqWzS2xNmgtPnzuQzz/1fN4Y88bKKsokzoWEdFD1bhE\nU1RUhKKiImg0GpSWlsLLywtbt26Fm5tb1RiDwYDly5cjOTn50ReRwRLN/YpKizA9eTqKS4uxcdxG\n9GrfS+pIRCQzRl2isbe3h0ajAQBYWlrCzc0NFy9efGCc3Ip3bdhb2mPHpB2Y4TkDAzYMwMc/fNwk\n/x2IqPEyr+3A/Px85OTkQKfTVXteoVAgPT0darUaKpUKy5Ytg7v7gzcHLVy4sOp7vV4PvV7/1KEb\nC4VCgUjvSPg7+SMkKQQ7Tu7A+tHrYW9pL3U0IjJBBoMBBoOh3s5Xq100paWl0Ov1ePPNNzFmzJhq\nx27cuAEzMzMolUqkpqYiKioKJ0+erH4RGS7R3K+8ohzv7H8HsUdiET0qGmN6jXn8DxER1cDob1VQ\nXl6OUaNGYfjw4Zg3b95jT9itWzccOXIEtra29RbSlKQXpCM0KRT+Tv748LkPYdncUupIRGSijLoG\nL4TA9OnT4e7u/sjiXlxcXBUgKysLQohqxb2pedbxWeRG5KJSVEITrcH3Bd9LHYmImqgaO/iDBw9i\n4MCB6Nu3LxQKBQBg8eLFOH/+PAAgIiICq1evxpo1a2Bubg6lUonly5fjmWeeqX6RJtTB3yvxp0TM\n/mY2Irwi8I+B/4CFmYXUkYjIhPDdJBu5SzcuIXxbOK7evoqN4zaiZ7ueUkciIhPBO1kbuU5WnZA6\nJRVT1VPx7CfPIuZwTJN9sSOihsUOvgH99NtPmJI4Bao2KqwLWgc7SzupIxFRI8YO3oS4dXBDxowM\n9OnYB5oYDbb/vF3qSEQkY+zgJXLglwOYunUqhvYYiuVDl6N189ZSRyKiRoYdvIka0HUAciNycefu\nHWhjtMi6kCV1JCKSGXbwjcCWE1vwcsrLeNnnZfz3gP+GebNav4MEEckYt0nKxIXfLyB8WzhulN1A\nwtgEONs6Sx2JiCTGJRqZULVRIS0kDRM9JqLfJ/2wLnsdXxSJqE7YwTdCP/76I6YkToGTjRNig2LR\noXUHqSMRkQTYwcuQR0cPZM7IhGt7V6ij1Ug5lSJ1JCIyQezgG7l9+fswdetUjOo5Ch8M+QBKC6XU\nkYiogbCDl7lBToOQF5mH63euwzPGE4cvHpY6EhGZCHbwJmTz8c2YmzoXUboovO73OsyamUkdiYiM\niNskm5iC6wV4cduL+M/d/yBhbAK6te0mdSQiMhIu0TQxjtaO2Bm6E+PcxsF3nS8+zf2UL55E9FDs\n4E3Y0eKjCEkMgUs7F8SMikF7ZXupIxFRPWIH34T1teuLrJlZcLJxgjpajW9Pfyt1JCJqRNjBy8Tu\ns7sRvi0co3uNxvuB76OVRSupIxFRHbGDJwBAQPcA5EXm4bebv8FrrReyL2VLHYmIJMYOXmaEEPj8\n2OeY9+08/K3f3/Das69xOyWRieI2SXqoX0p+QdjWMFSKSsSPjYeTjZPUkYjoCXGJhh6qq01X7J66\nG0E9g+AT64OEvAS+yBI1Mezgm4DcolyEJIbAvYM7okdFw7aVrdSRiKgWjNrBFxQUwN/fHx4eHujd\nuzdWrlz50HFz586Fi4sL1Go1cnJynjoMGYfGXoMfZv6AzladoY5WY9fZXVJHIqIGUGMHX1RUhKKi\nImg0GpSWlsLLywtbt26Fm5tb1ZiUlBSsWrUKKSkpyMzMRFRUFDIyMqpfhB18o7HzzE6EbwvH8x7P\n472A99DSvKXUkYjoEYzawdvb20Oj0QAALC0t4ebmhosXL1Ybk5ycjLCwMACATqdDSUkJiouLnzoQ\nGdeQHkNwdPZRFP5eCO+13sgrypM6EhEZSa0/3Tk/Px85OTnQ6XTVnr9w4QIcHR2rHjs4OKCwsBB2\ndnbVxi1cuLDqe71eD71e/3SJqc5sW9niywlfIuFoAgITAjG//3z8td9f0UzBv7kTSclgMMBgMNTb\n+WpV4EtLSzFhwgSsWLEClpaWDxy//1cIhULxwJh7CzxJT6FQYKp6KgZ2HYjQpFB8c+obxI2JQxfr\nLlJHI2qy7m9+33777Tqd77EtW3l5OcaPH4+QkBCMGTPmgeMqlQoFBQVVjwsLC6FSqeoUihqOk40T\nDGEGDOsxDN5rvfH5sc+ljkRE9aTGAi+EwPTp0+Hu7o558+Y9dExwcDDi4+MBABkZGbCxsXlgeYYa\nN7NmZnjd73WkhaThn/v/iclfT8a129ekjkVEdVTjLpqDBw9i4MCB6Nu3b9Wyy+LFi3H+/HkAQERE\nBABgzpw5SEtLQ+vWrbFhwwZ4enpWvwh30ZiMW+W3MH/XfGz79zbEjYmDfzd/qSMRNVl8qwIyirTT\naZiePB2Tek/Cu4PfRQvzFlJHImpy+FYFZBTPOT+HvMg8nL12Fr7rfHGs+JjUkYjoCbHA0yO1V7bH\n1y98jXm6eRgcPxjLv1+OSlEpdSwiqiUu0VCtnL12FqFJoWhp3hJxY+Lg0MZB6khEssclGmoQ3dt2\nx74X98HfyR+eMZ744vgXUkciosdgB09P7IcLPyAkKQS+Kl+sGr4K1i2tpY5EJEvs4KnB+ah8kD0r\nG1bNraCOVmNf/j6pIxHRQ7CDpzr55uQ3mLl9JkLVoXhH/w63UxLVI+6DJ8n9dvM3zNw+E79c/wUb\nx26ER0cPqSMRyQKXaEhyHVp3QNJfkvCyz8sY9OkgrMxcye2URI0AO3iqV6evnkZIYgjatGiDT8d8\nis5WnaWORGSy2MFTo+Js64yD0w6iv2N/aGO02HJii9SRiJosdvBkNJmFmQhJCkF/x/5YOXwl2rRo\nI3UkIpPCDp4aLZ2DDjkROWhh3gKaaA0Onj8odSSiJoUdPDWI5J+TEbEjAtO00/DWoLfQ3Ky51JGI\nGj1ukySTUVxajOnJ03Gp9BI+G/cZerXvJXUkokaNSzRkMuws7bB90nbM9JyJARsGYHXWar7wExkR\nO3iSxM+Xf0ZIUgg6KDtg/ej1sLe0lzoSUaPDDp5Mkmt7V6RPS4d3Z29oojVI+ilJ6khEssMOniSX\nXpCO0KRQ6J30+HDYh7BqYSV1JKJGgR08mbxnHZ9FbkQuAEATo0F6QbrEiYjkgR08NSpJPyVh9jez\nMctrFv4x8B+wMLOQOhKRZLhNkmTn0o1LmJY8DVduXcHGcRvRs11PqSMRSYJLNCQ7naw6IWVyCsLU\nYei/vj+iD0ezQSB6CjUW+GnTpsHOzg59+vR56HGDwQBra2totVpotVosWrTIKCGp6VEoFHjZ92Uc\nCD+A2OxYBG0KQnFpsdSxiExKjQU+PDwcaWlpNZ5g0KBByMnJQU5ODt588816DUfUq30vfD/9e6jt\n1dDEaJD8c7LUkYhMRo0FfsCAAWjbtm2NJ+CvzmRszc2a493B7+Kr579CVFoUZm2fhdKyUqljETV6\n5nX5YYVCgfT0dKjVaqhUKixbtgzu7u4PHbtw4cKq7/V6PfR6fV0uTU2QXxc/5EXmYW7qXGhjtNg4\ndiN0DjqpYxHVG4PBAIPBUG/ne+wumvz8fAQFBeHYsWMPHLtx4wbMzMygVCqRmpqKqKgonDx58sGL\ncBcN1bMtJ7bg5ZSX8ZL3S3hj4Bswb1anXoWoUZJ0F42VlRWUSiUAYPjw4SgvL8fVq1frckqiWpng\nPgE5ETk4VHAIfuv9cPrqaakjETU6dSrwxcXFVa8uWVlZEELA1ta2XoIRPU5nq85IC0nD5D6T0e+T\nfliXvY6/KRLdo8YlmkmTJmHfvn24fPky7Ozs8Pbbb6O8vBwAEBERgdWrV2PNmjUwNzeHUqnE8uXL\n8cwzzzx4ES7RkJH9+OuPCEkKQRfrLlgXtA4dWneQOhJRnfFOVqL/85+7/8FbhrcQnxePdcHrMMJl\nhNSRiOqEBZ7oPvvy9yFsaxhGuIzAsqHLoLRQSh2J6KnwrQqI7jPIaRDyIvNwo+wGtDFaHL54WOpI\nRJJgB0+y9sXxL/BK6iuI0kVhvt98bqckk8IlGqLHKPy9EGFbw3Dn7h0kjE1A97bdpY5EVCtcoiF6\nDIc2DtgZuhMT3CZAt06H9Tnr2XBQk8AOnpqUY8XHMCVxCpxtnbE2aC3aK9tLHYnokdjBEz2BPnZ9\nkDUzC93bdoc6Wo200zW/WyqRKWMHT03WnnN78OLWFzG612gsDVzK7ZTU6LCDJ3pKg7sNRl5kHq7c\nugKvtV7IvpQtdSSiesUOngjA58c+x7y0eXj1mVfx9/5/h1kzM6kjEXGbJFF9OX/9PMK2huFu5V0k\njE2Ak42T1JGoieMSDVE96WLdBbun7sZo19HwifVBfF48GxMyaezgiR4irygPUxKnwL2DO6JHRcO2\nFd8GmxoeO3giI1Dbq3F41mGo2qigjlZj55mdUkciemLs4IkeY9fZXQjfFo7xbuPxXsB7aGXRSupI\n1ESwgycyssDugciLzMPFGxfhE+uDvKI8qSMR1QoLPFEt2LayxRcTvsD8/vMRmBCIDw59gIrKCqlj\nEdWISzRETyi/JB9Tk6aimaIZ4sfGo4t1F6kjkUxxiYaogTnZOGFv2F485/wcvNd64/Njn0sdieih\n2MET1UH2pWyEJIZAba/GxyM+RttWbaWORDLCDp5IQp6dPHFk1hF0UHaAOlqNPef2SB2JqAo7eKJ6\n8u3pbzEteRom9p6Idwe/i5bmLaWORCaOHTxRIzHMeRjyIvOQX5IP31hfHCs+JnUkauJqLPDTpk2D\nnZ0d+vTp88gxc+fOhYuLC9RqNXJycuo9IJEpaa9sjy3Pb8Ff+/0Vg+MHY/n3y1EpKqWORU1UjQU+\nPDwcaWmP/sSblJQUnD59GqdOncLatWsxe/bseg9IZGoUCgVe1LyIzBmZSPwpEUMShqDgeoHUsagJ\nqrHADxgwAG3bPnpXQHJyMsLCwgAAOp0OJSUlKC4urt+ERCaqe9vu2PfiPgR0C4DXWi9sPr5Z6kjU\nxJjX5YcvXLgAR0fHqscODg4oLCyEnZ3dA2MVioX3PNL/3xeR3JkB+G+g81BMOh+CSRd3ACmrgDs2\nUgeTPVPc12EwGGAwGOrtfHUq8AAe+AuvQqF4xLiFdb0UkQnzxq3ybLy28zXsGKBG3Jg46J30Uoei\nRkav10Ov11c9fvvtt+t0vjrtolGpVCgo+HNtsbCwECqVqk6BiORKaaHE6hGrsWbkGkz+ejLm75qP\n/9z9j9SxSMbqVOCDg4MRHx8PAMjIyICNjc1Dl2eI6E8jXEYgLzIPP1/+Gbp1Ovz4649SRyKZqvFG\np0mTJmHfvn24fPky7Ozs8Pbbb6O8vBwAEBERAQCYM2cO0tLS0Lp1a2zYsAGenp4PXoQ3OhE9QAiB\n9Tnr8fru1/HmgDfxiu4VNFPw1hT6Ez90m8jEnb56GqFJobBqboUNozdA1YbLnPQH3slKZOKcbZ1x\nIPwA/Lr4wXOtJ7ac2CJ1JJIJdvBEjUhmYSZCk0LRz7EfPhr+Edq0aCN1JJIQO3giGdE56JAdkY2W\n5i2hjlbjwC8HpI5EJowdPFEjtf3n7Zi1YxbCNeFYqF+I5mbNpY5EDYx/ZCWSseLSYszYPgMXb1zE\nxrEb4dbBTepI1IC4REMkY3aWdkiemIxZnrMw8NOBWJ21ms0S1Ro7eCITcfLKSYQkhqCdsh3WB69H\nJ6tOUkciI2MHT9RE9GzXE4emHYKvyhfaGC0Sf0qUOhI1cuzgiUzQ9wXfIzQpFAO7DsSK51bAqoWV\n1JHICNjBEzVB/Rz7ITcyF2bNzKCJ0SC9IF3qSNQIsYMnMnFb/70VkTsiMdNrJv5n4P/AwsxC6khU\nT7hNkohQVFqEadum4bdbv2Hj2I1wbe8qdSSqB1yiISLYW9rjm8nfIFwTDr8Nfljzwxo2VcQOnkhu\n/n353whJDIG9pT0+Cf4Edpb8jAZTxQ6eiKrp1b4X0qenQ2OvgSZGg+Sfk6WORBJhB08kY4fOH0Jo\nUigCuwdi+bDlsGxuKXUkegLs4Inokfp36Y/cyFyUV5ZDG6NFRmGG1JGoAbGDJ2oivj7xNV5KeQmz\nvWfjzYFvwryZudSR6DG4TZKIau3ijYsI3xaO63euI2FsAlzauUgdiWrAJRoiqrXOVp2ROiUVU/pM\nQb9P+iH2SCybLxljB0/URJ347QSmJE5BF+suiA2KRcfWHaWORPdhB09ET8W9gzsyZ2TCrb0bNNEa\nfHPyG6kjUT1jB09E2P/LfkxNmorhLsOxbMgytG7eWupIhAbo4NPS0tCrVy+4uLhg6dKlDxw3GAyw\ntraGVquFVqvFokWLnjoMEUljYNeByIvMw82ym/Bc64kfLvwgdSSqBzV28BUVFXB1dcWuXbugUqng\n4+ODTZs2wc3tz8+FNBgMWL58OZKTH323HDt4ItPx5Y9f4pXUVzDHZw4WDFjA7ZQSMmoHn5WVBWdn\nZzg5OcHCwgITJ07Etm3bHhjH4k0kHy94vIDsWdnYf34/Bm4YiDNXz0gdiZ5SjS/NFy5cgKOjY9Vj\nBwcHZGZmVhujUCiQnp4OtVoNlUqFZcuWwd3d/YFzLVy4sOp7vV4PvV5ft+REZDSqNip8G/ItVmau\nxDOfPIOlgUsRrgmHQqGQOpqsGQwGGAyGejtfjUs0X3/9NdLS0hAbGwsA2LhxIzIzM/HRRx9Vjblx\n4wbMzMygVCqRmpqKqKgonDx5svpFuERDZLKO/3ocUxKnoHvb7ogNikV7ZXupIzUZRl2iUalUKCgo\nqHpcUFAABweHamOsrKygVCoBAMOHD0d5eTmuXr361IGIqHHp3bE3smZkwdnWGepoNVJPpUodiWqp\nxgLv7e2NU6dOIT8/H2VlZfjiiy8QHBxcbUxxcXHVK0xWVhaEELC1tTVeYiJqcC3MW+CDIR9g49iN\niPwmEnNS5uBW+S2pY9Fj1Fjgzc3NsWrVKgwbNgzu7u74y1/+Ajc3N8TExCAmJgYAsGXLFvTp0wca\njQbz5s3D5s2bGyQ4ETU8/27+yIvMw7U71+C11gvZl7KljkQ14I1ORPRUNh3bhKi0KLz6zKv4e/+/\nw6yZmdSRZIfvJklEkjl//TzCtobhbuVdxI+JR7e23aSOJCt8LxoikkwX6y7YPXU3xriOge86X8Tl\nxrGZa0TYwRNRvcgrysOUxClw6+CG6JHRaKdsJ3Ukk8cOnogaBbW9GodnHYZjG0eoo9X47sx3Ukdq\n8tjBE1G923V2F8K3hWOc2zgsCViCVhatpI5kktjBE1GjE9g9EHmReSgqLYJ3rDdyi3KljtQksYMn\nIqMRQuCzY5/h1W9fxWvPvoa/9fsbt1M+AW6TJKJG75eSXzB161QooEDcmDh0tekqdSSTwCUaImr0\nutp0xZ6pezDceTi8Y72x8ehGNn0NgB08ETWonEs5mJI4BX3t+mLNyDVo26qt1JEaLXbwRGRStJ20\nODLrCOws7dA3ui92n90tdSTZYgdPRJL57sx3mLZtGl7weAGLAxajpXlLqSM1KuzgichkDe0xFHmR\neTh//Tx8Yn1wtPio1JFkhQWeiCTVTtkOXz3/Ff6r338hID4A/0r/FypFpdSxZIFLNETUaJy7dg6h\nSaFobtYccWPi4Gjt+PgfkjEu0RCRbHRr2w37XtyHwO6B8Frrhc3H+QFCdcEOnogapSMXj2BK4hR4\ndfbC6hGrYdPSRupIDY4dPBHJkldnL2RHZKNty7ZQR6thyDdIHcnksIMnokYv5VQKZiTPQEjfEPzT\n/59oYd5C6kgNgh08EcneCJcRyIvMw6mrp6Bbp8PxX49LHckksMATkUno0LoDEl9IxCu+r8A/zh8f\nZnzI7ZSPwSUaIjI5Z66eQUhSCCybW+LT0Z9C1UYldSSj4BJNI2AwGKSOYDRynhvA+ZmqHrY9cCD8\nAByvOsJzrSe++vErqSM1So8t8GlpaejVqxdcXFywdOnSh46ZO3cuXFxcoFarkZOTU+8hGzu5/k8E\nyHtuAOdnysybmaPLtS7YPmk73tjzBsK2huH6netSx2pUaizwFRUVmDNnDtLS0nDixAls2rQJP/30\nU7UxKSkpOH36NE6dOoW1a9di9uzZRg1MRHQvX5UvciJy0Mq8FTQxGhz45YDUkRqNGgt8VlYWnJ2d\n4eTkBAsLC0ycOBHbtm2rNiY5ORlhYWEAAJ1Oh5KSEhQXFxsvMRHRfVo3b43oUdFY+dxKvLDlBSzY\nvQBlFWVSx5KeqMFXX30lZsyYUfU4ISFBzJkzp9qYUaNGiUOHDlU9DggIEIcPH642BgC/+MUvfvHr\nKb7qwhztrn/cAAAFjklEQVQ1UCgUNR2ucv9fee//Oe6gISJqeDUu0ahUKhQUFFQ9LigogIODQ41j\nCgsLoVLJc8sSEZEpqbHAe3t749SpU8jPz0dZWRm++OILBAcHVxsTHByM+Ph4AEBGRgZsbGxgZ2dn\nvMRERFQrNS7RmJubY9WqVRg2bBgqKiowffp0uLm5ISYmBgAQERGBESNGICUlBc7OzmjdujU2bNjQ\nIMGJiOgx6rSCXwupqanC1dVVODs7iyVLlhj7ckYRHh4uOnbsKHr37l313JUrV0RgYKBwcXERQ4YM\nEdeuXas6tnjxYuHs7CxcXV3Ft99+K0XkWjt//rzQ6/XC3d1deHh4iBUrVggh5DO/27dvC19fX6FW\nq4Wbm5t4/fXXhRDymd//u3v3rtBoNGLUqFFCCHnNr2vXrqJPnz5Co9EIHx8fIYR85nft2jUxfvx4\n0atXL+Hm5iYyMjLqdW5GLfB3794VPXr0EOfOnRNlZWVCrVaLEydOGPOSRrF//36RnZ1drcC/9tpr\nYunSpUIIIZYsWSLmz58vhBDixx9/FGq1WpSVlYlz586JHj16iIqKCkly18alS5dETk6OEEKIGzdu\niJ49e4oTJ07IZn5CCHHz5k0hhBDl5eVCp9OJAwcOyGp+Qgjxr3/9S0yePFkEBQUJIeTz36cQQjg5\nOYkrV65Ue04u85s6dar45JNPhBB//PdZUlJSr3MzaoFPT08Xw4YNq3r83nvviffee8+YlzSac+fO\nVSvwrq6uoqioSAjxR5F0dXUVQvzxCnvvbyrDhg0T33//fcOGrYPRo0eLnTt3ynJ+N2/eFN7e3uL4\n8eOyml9BQYEICAgQe/bsqerg5TQ/Jycncfny5WrPyWF+JSUlolu3bg88X59zM+p70Vy4cAGOjn9+\npqKDgwMuXLhgzEs2mOLi4qo/JtvZ2VXd3HXx4sVqO41Mac75+fnIycmBTqeT1fwqKyuh0WhgZ2cH\nf39/eHh4yGp+r776Kj744AM0a/bn/85ymp9CoUBgYCC8vb0RGxsLQB7zO3fuHDp06IDw8HB4enpi\n5syZuHnzZr3OzagFvrb76E2dQqGoca6m8O9QWlqK8ePHY8WKFbCysqp2zNTn16xZM+Tm5qKwsBD7\n9+/H3r17qx035fnt2LEDHTt2hFarfeT9JqY8PwA4dOgQcnJykJqaitWrV+PAgepvRWCq87t79y6y\ns7Px0ksvITs7G61bt8aSJUuqjanr3Ixa4Guzj95U2dnZoaioCABw6dIldOzYEYBp3hdQXl6O8ePH\nIzQ0FGPGjAEgr/n9P2tra4wcORJHjhyRzfzS09ORnJyMbt26YdKkSdizZw9CQ0NlMz8A6NSpEwCg\nQ4cOGDt2LLKysmQxPwcHBzg4OMDHxwcAMGHCBGRnZ8Pe3r7e5mbUAl+bffSmKjg4GHFxcQCAuLi4\nqsIYHByMzZs3o6ysDOfOncOpU6fg6+srZdQaCSEwffp0uLu7Y968eVXPy2V+ly9fRklJCQDg9u3b\n2LlzJ7RarWzmt3jxYhQUFODcuXPYvHkzBg8ejISEBNnM79atW7hx4wYA4ObNm/juu+/Qp08fWczP\n3t4ejo6OOHnyJABg165d8PDwQFBQUP3Nrd7+YvAIKSkpomfPnqJHjx5i8eLFxr6cUUycOFF06tRJ\nWFhYCAcHB7F+/Xpx5coVERAQ8NCtTO+++67o0aOHcHV1FWlpaRImf7wDBw4IhUIh1Gq10Gg0QqPR\niNTUVNnM7+jRo0Kr1Qq1Wi369Okj3n//fSGEkM387mUwGKp20chlfmfPnhVqtVqo1Wrh4eFRVUPk\nMr/c3Fzh7e0t+vbtK8aOHStKSkrqdW4N8olORETU8PiJTkREMsUCT0QkUyzwREQyxQJPRCRTLPBE\nRDLFAk9EJFP/C8qBlegeqe2HAAAAAElFTkSuQmCC\n" 97 | } 98 | ], 99 | "prompt_number": 13 100 | }, 101 | { 102 | "cell_type": "code", 103 | "collapsed": true, 104 | "input": [], 105 | "language": "python", 106 | "outputs": [] 107 | } 108 | ] 109 | } 110 | ] 111 | } -------------------------------------------------------------------------------- /misc/intro-images.odp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/misc/intro-images.odp -------------------------------------------------------------------------------- /misc/openfst-misc.odp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/misc/openfst-misc.odp -------------------------------------------------------------------------------- /mkindex: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | 3 | import os,sys,os.path,glob,re,markdown 4 | 5 | print markdown.markdown(""" 6 | Lecture Notes: Learning and Perception 7 | ----------------------- 8 | 9 | This is the lecture note and slide archive for the Learning and Perception course. 10 | For the course pages themselves, please go to [lw.iupr.com](http://lw.iupr.com/). 11 | 12 | [All the lecture notes are available in PDF Format](lw.pdf). This is a single, large 13 | PDF file that contains the content from the slides/worksheets, plus some additional 14 | text. 15 | 16 | Below are all the notebooks and corresponding slides from the lecture. 17 | 18 | _Note that slides and notes will get updated throughout the class_ 19 | 20 | """) 21 | 22 | print "" 23 | for nb in sorted(glob.glob("[0-9]*.ipynb")): 24 | with open(nb) as stream: text = stream.read() 25 | title = re.findall(r"Title:\s+([^\\\"]*)",text) 26 | base,_ = os.path.splitext(nb) 27 | if len(title)>0: title = title[0] 28 | else: 29 | title = base 30 | title = re.sub('^\d+-','',title) 31 | title = re.sub('[_-]+',' ',title) 32 | print "" 33 | print "".format(title) 34 | print "".format(nb) 35 | print "".format(base+"-slides.pdf") 36 | print "" 37 | print "
{}notebookslides
" 38 | 39 | print """ 40 | 41 | """ 42 | -------------------------------------------------------------------------------- /msak0_306.ema: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/msak0_306.ema -------------------------------------------------------------------------------- /msak0_306.epg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/msak0_306.epg -------------------------------------------------------------------------------- /msak0_306.lab: -------------------------------------------------------------------------------- 1 | 0 0.2 sil 2 | 0.2 0.49 breath 3 | 0.49 0.69 sil 4 | 0.69 0.76 w 5 | 0.76 0.89 ii 6 | 0.89 0.94 l 7 | 0.94 1.04 ai 8 | 1.04 1.07 k 9 | 1.07 1.17 b 10 | 1.17 1.22 l 11 | 1.22 1.33 uu 12 | 1.33 1.45 ch 13 | 1.45 1.59 ii 14 | 1.59 1.68 z 15 | 1.68 1.74 b 16 | 1.74 1.82 @ 17 | 1.82 1.85 t 18 | 1.85 1.92 v 19 | 1.92 2 i 20 | 2 2.04 k 21 | 2.04 2.12 t 22 | 2.12 2.16 @ 23 | 2.16 2.21 l 24 | 2.21 2.31 ai 25 | 2.31 2.34 k 26 | 2.34 2.4 s 27 | 2.4 2.49 b 28 | 2.49 2.58 r 29 | 2.58 2.74 ii 30 | 2.74 3.7 sil 31 | -------------------------------------------------------------------------------- /msak0_306.lar: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/msak0_306.lar -------------------------------------------------------------------------------- /msak0_306.wav: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/msak0_306.wav -------------------------------------------------------------------------------- /ocr-hw/HOMEWORK: -------------------------------------------------------------------------------- 1 | HOMEWORK 2 | ======== 3 | 4 | Most of the code for the following functions is already in the lecture notes. You 5 | need to pull it out and put it together in some functions. You need no extra libraries 6 | beyond what was covered in lecture. 7 | 8 | 1. Implement convolution using the fft2/ifft2 functions. Your function should look like 9 | 10 | def fftconvolve(image,mask): 11 | ... 12 | 13 | Image and mask are allowed to be different sizes, so you need to do something about the 14 | mask. The easiest is to pad the mask so that it is the same size as the image. 15 | 16 | 2. Implement an image standardization function. That is, a function that, given 17 | a rectangle size b=(bw,bh) replaces each pixel by subtracting the mean over that 18 | rectangle from the pixel and then divides the result by the norm of the patch. 19 | 20 | def standardize(image,b): 21 | ... 22 | 23 | Implement the convolutions necessary for standardization using fftconvolve above. 24 | 25 | 3. Implement normalized cross correlation using your fftconvolve function and the 26 | standardize function you defined above. 27 | 28 | def ncc(image,template): 29 | ... 30 | 31 | Output the image. Demonstrate your function on the w.png image and letter 32 | templates you obtain using the charread.py file (you may want to look at it). 33 | 34 | import charread 35 | chars = charread.charread() 36 | imshow(chars[17][0]) 37 | imshow(chars[17][5]) 38 | 39 | Note that you need to run ncc on page images with black background (why?). 40 | 41 | OPTIONAL 42 | ========= 43 | 44 | The following two problems are a lot more work, but quite rewarding. 45 | 46 | 4. Implement normalized cross correlation using multiple templates. 47 | 48 | def nccs(image,templates): 49 | ... 50 | 51 | This works like the function above but returns a list of normalized cross correlations, 52 | one for each template. This makes the simplifying assumption that it is sufficient 53 | to standardize the input image only once with parameters that are about right for all the 54 | templates (use the mean width and mean height). Furthermore, the Fourier transform 55 | of the standardized image for the convolution step needs to be carried out only once. 56 | 57 | 5. Compute normalized cross-correlations for all the templates and try to 58 | label all the characters in the input image uniquely (i.e., so that there is only 59 | one peak/output per input character). Remember that you know the approximate size 60 | of all the characters, and they are all about the same size.. 61 | 62 | 6. Try to find a way of outputting the characters in reading order (hard!). 63 | -------------------------------------------------------------------------------- /ocr-hw/c.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tmbdev-teaching/teaching-lw/36924cddf1ed5dae69d3968745c0350434c3da16/ocr-hw/c.png -------------------------------------------------------------------------------- /ocr-hw/charread.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | from pylab import * 4 | from scipy.ndimage import measurements,filters 5 | 6 | ion() 7 | gray() 8 | 9 | def crop(c,threshold=0.6,r=5): 10 | b = filters.maximum_filter(c