├── Basic-LRP
├── README.md
├── Taylor decomposition for the explanation of non-linear classification desition.ipynb
├── Taylor+decomposition++for+the+explanation+of+non-linear+classification+desition.py
└── result.JPG
├── LICENSE
├── LRP-Time-Series
├── AUTHOR.txt
├── LICENSE
├── LRP_tutorial.ipynb
├── LRP_tutorial.py
├── README.md
├── checkpoints-cnn
│ ├── checkpoint
│ ├── har.ckpt.data-00000-of-00001
│ ├── har.ckpt.index
│ └── har.ckpt.meta
├── howtorun.gif
├── model.jpg
├── result.jpg
└── utilities.py
├── README.md
└── Visual-Explanation-of-Atari
├── README.md
├── assets
├── actor_breakout.gif
├── actor_pong.gif
├── critic_breakout.gif
└── critic_pong.gif
├── checkpoints
├── breakout
│ ├── network_00097000.data-00000-of-00001
│ ├── network_00097000.index
│ └── network_00097000.meta
└── pong
│ ├── network_00029000.data-00000-of-00001
│ ├── network_00029000.index
│ └── network_00029000.meta
├── config.py
├── env.py
├── main.py
├── network.py
├── notebook.ipynb
└── sailency.py
/Basic-LRP/README.md:
--------------------------------------------------------------------------------
1 | LRP method for the explanation of non-linear classification
2 | ==
3 |
4 | Python implementation of Taylor decomposition and simple LRP method for the explanation of non-linear classification decision.
5 |
6 | ## Reference Code
7 | Based on code by [Denny Britz](https://github.com/dennybritz/nn-from-scratch/blob/master/nn-from-scratch.ipynb)
8 |
9 | ## Reference Paper
10 | **"Explaining nonlinear classification decisions with deep taylor decomposition"**. Gregoire Montavon, Sebastian Bach, Alexander Binder, Wojciech Samek, and Klaus-Robert Muller (https://arxiv.org/abs/1512.02479)
11 |
12 | ## Result
13 | This is a deep learning method to classify binary dataset. Our goal is to test how the LRP (more specifically deep Taylor Decomposition) can perform to depict the important time epochs and features from raw data.
14 |
15 | 
16 |
17 |
18 | ## Dataset
19 | We will use the generated dataset.
20 |
21 | ## Installation
22 |
23 | **1. Fork & Clone** : Fork this project to your repository and clone to your work directory.
24 |
25 | ``` $ git clone https://github.com/OpenXAIProject/Basic-LRP.git ```
26 |
27 | **2. Run** : Run "Taylor decomposition for the explanation of non-linear classification desition.ipynb" or "Taylor decomposition for the explanation of non-linear classification desition.py"
28 |
29 | ## Requirements
30 | + tensorflow (1.9.0)
31 | + numpy (1.15.0)
32 | + matplotlib (2.2.2)
33 | + scikit-learn (0.19.1)
34 |
35 | ## License
36 | [Apache License 2.0](https://github.com/OpenXAIProject/tutorials/blob/master/LICENSE "Apache")
37 |
38 | ## Contacts
39 | If you have any question, please contact Xie Qin(xieqin856@unist.ac.kr).
40 |
41 |
42 |
43 |
44 | # XAI Project
45 |
46 | **This work was supported by Institute for Information & Communications Technology Promotion(IITP) grant funded by the Korea government(MSIT) (No.2017-0-01779, A machine learning and statistical inference framework for explainable artificial intelligence)**
47 |
48 | + Project Name : A machine learning and statistical inference framework for explainable artificial intelligence(의사결정 이유를 설명할 수 있는 인간 수준의 학습·추론 프레임워크 개발)
49 |
50 | + Managed by Ministry of Science and ICT/XAIC 
51 |
52 | + Participated Affiliation : UNIST, Korea Univ., Yonsei Univ., KAIST, AItrics
53 |
54 | + Web Site :
55 |
56 |
--------------------------------------------------------------------------------
/Basic-LRP/Taylor decomposition for the explanation of non-linear classification desition.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "code",
5 | "execution_count": 1,
6 | "metadata": {},
7 | "outputs": [],
8 | "source": [
9 | "#Reference: https://github.com/dennybritz/nn-from-scratch/blob/master/nn-from-scratch.ipynb\n",
10 | "\n",
11 | "import matplotlib.pyplot as plt\n",
12 | "import numpy as np\n",
13 | "import sklearn\n",
14 | "import sklearn.datasets\n",
15 | "import sklearn.linear_model\n",
16 | "import matplotlib\n",
17 | "# Display plots inline and change default figure size\n",
18 | "%matplotlib inline\n",
19 | "plt.rcParams['figure.figsize'] = (10.0, 8.0)"
20 | ]
21 | },
22 | {
23 | "cell_type": "code",
24 | "execution_count": 2,
25 | "metadata": {},
26 | "outputs": [
27 | {
28 | "data": {
29 | "text/plain": [
30 | ""
31 | ]
32 | },
33 | "execution_count": 2,
34 | "metadata": {},
35 | "output_type": "execute_result"
36 | },
37 | {
38 | "name": "stderr",
39 | "output_type": "stream",
40 | "text": [
41 | "/usr/lib/pymodules/python2.7/matplotlib/collections.py:548: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison\n",
42 | " if self._edgecolors == 'face':\n"
43 | ]
44 | },
45 | {
46 | "data": {
47 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAl4AAAHfCAYAAACBJdZmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xd4k2X3wPFvkqZt0sGmbBBBhmyQoSBFVARlqaDiYDhQ8fWn6KsIqKjgeMUXF6Doi4oDByAOZCM4kK3I3nvJhjZdGb8/Dm2atkBHmidJz+e6etHnafLktLTJyX2f+9yglFJKKaWUUkoppZRSSimllFJKKaWUUkoppZRSSimllFJKKaWUUkoppZSPycARYN15vp4InAb+PPcxMjBhKaWUUkqFnw5Acy6ceH0fsGiUUkoppYKU2Q/X+BU4eZHbmPzwOEoppZRSIc0fidfFeIArgbXAT0DDADymUkoppVTQiQjAY6wBqgMOoCswE7gs540uvfRSz44dOwIQjlJKKaVUke0A6hT0ToEY8TqLJF0AswErUDbnjXbs2IHH4ylxH88//7zhMej3rd+3ft/6fev3rd+3ft8F+wAuLUxSFIjEKwFvjVfrc5+fCMDjKqWUUkoFFX9MNU4FOgLlgX3A88ioFsD7wK3AQ4ATGfm63Q+PqZRSSikVcvyReN1xka+PP/eh8pCYmGh0CIbQ77tk0e+7ZNHvu2Qpqd93YQVTmwfPuTlTpZRSSqmgZjKZoBB5VCBqvJRSSimlFJp4KaWUUkoFjCZeSimllFIBoomXUkoppVSAaOKllFJKKRUgmngppZRSSgWIJl5KKaWUUgGiiZdSSimlVIBo4qWUUkopFSCaeCmllFJKBYgmXkoppZRSAaKJl1JKKaVUgGjipZRSSikVIJp4KaWUUkoFiCZeSimllFIBoomXUuHm2DG45x5o2RIeegjOnjU6IqWUUueYjA4gG4/H4zE6BqVCW1oaNGoEe/ZARgZER0PTpvDHH2AKpj93pZQKbSZ5Ti3wE6uOeCkVTlavhiNHJOkCSE2Fdetg1y5j41JKKQVo4qVUeDGbIefIsdsNFosx8RSXU6dgxQrYv9/oSJRSqkA08VIqnLRsCbVrQ1SUHNts0L491KhhbFz+tGQJVK8O110HdevCq68aHZFSSuVbMBV9aI2XUv5w9iw89xysXw9t2sCzz3oTsVDnckG5cnD6tPec3Q5Ll0otm1JKBUhha7wi/B+KUspQcXEwbpzRURSPkyelbi07iwU2b9bESykVEnSqUSkVOsqWlZWa2blcUL++MfEopVQBaeKlVEmTkQFHj+Yuwg8FZjN8952M6sXHSxL23HM62qWUChla46VUSfLllzBwoCRdZcrAvHnQuLHRURXc6dOwbRtUrgxVqxodjVKqBCpsjZcmXkqVFNu2ychQSor3XKVKcOCAjCQppZTKNy2uV0pd2F9/QUSOP/mTJ+H4cahQIfftT56Et9+Whqw33igfSimlikQTL6VKiurVpRA9O5MJSpfOfdszZ2R07MgRSE+HTz6BV16BRx8NTKxKKRWmdH5BqZKibVvZPDsmRgrT7Xb46COwWnPf9uuvZSQsPV2OHQ4YOTKw8SqlVBjSES+lSpKJE2HAANi3D5o1gzp18r6dw5F7dCwzCVNKKVVoWlyvlMpt2zZo3hySk+XYZoPu3eGrr4yNSymlgkRhi+t1qlEplVvdujB3LjRpIu0a7rwTPv7Y6KiUUirk6YiXUkoppVQB6YiXUkoppVSQ08RLKaWUUipANPFSSimllAoQTbyUUkoppQJEEy+llFJKqQDRxEsppZRSKkA08VJKKaWUChBNvJRSSimlAkQTL6VUyfPHH9ClC3ToAFOmGB2NUqoE0U2ylfKHffvg558hNhZuvBGiooyOSJ3PmjVw7bWyEXjmcWoqPPCAsXEppUoE3TJIBc7JkzBpEhw/Dt26QWKi0RH5x/Ll8kKe6ZJLYNkysNuNi0md38MPw8SJvucuuwy2bDEmnoJKTYXFiyE9Ha6+GkqXNjoipUqkwm4ZpCNeKjBOn4ZmzeDwYXnBGD8eJkyA/v2Njqzo7r0XkpK8x9u2SYL52GPGxaTOz5TH82Re54LR6dPQpg0cPCjH0dGS+F9yibFxKaXyTWu8VGB89hkcPSpJF8g0z5NPGhuTvxw54nucmgr79xsTi7q4wYMhJsZ7bLfDsGHGxVMQr7wCu3fD2bPycfy4jOAppUKGJl4qMJKSICPD91xKijGx+FuHDhAZ6T2226FjR+PiURfWpAksWQK9esF118HHH8OAAUZHlT/btkFamvfY7YZdu4yLRylVYJp4qcDo2tU3OYmOhp49jYvHnyZPhiuvBIsFrFYYMQK6dzc6KnUhLVvCt9/CvHnQp4/R0eRfp06+tYPR0ZL4K6VCRjAVNmhxfbibNw+GDJE6lRtvlBovm83oqPwnLU0SL7O+n1HFxO2G+++XFhgmE7RvDz/84Dt1qpQKiMIW12vipZRSoSY5WabudUWjUobRxEsppZRSKkAKm3jpnIhSShVWUhKsXy896pRSKh808VIq3B0/Dv/9L7z4Ivz1l/fc11/DjBm+PciMtncv3HQT1K8PAwdKy4RgtXAhVK4sCyuqVIGPPjI6IqVUCNCpRqXC2dGj0j7h5EmpCYqOlkUNTzzh7alWtiysXg3lyhkb69mz0kH+6FFwuWTbpZYt4bffAtfgdPx4GDNGflb33w+jR+e9WCI1FSpW9E0MbTbYuBFq1QpMrEopQ+lUo1IqtwkTZHQrLU1WxDkcsrL05ElvE86DB2U0zGhLl0p8Lpccp6VJQvjPP4F5/GnT4Kmn4NAhOHYM3noLXnst79seOCA/z+wiI2Hz5uKPUykV0jTxUiqcnTiRu3FtZhKWKSMjOJpwWq2Qc9Tb45HzgTB1qnfjbJDPv/wy79tWrpw71vR0uPTS4otPKRUWNPFSKpz17OnbcNNmgwYNfPun2e2+m3wbpX17maaLipJjux1uuUWmQgOhbNnc04qlSuV9W7tdtsGy2+U20dHw0ktQt27xx6mUCmla46VUuJsyBZ5+WrZo6tMH3ngD7rkHfvxRaqcGDYKJE4Oj8WtSkuxHuHmzJGKPPio7AgTCzp3QooX0yHK7JZlauBDatj3/fQ4fhi1bJGGsWTMwcSqlgoL28VJKFUxqqiRb2bdyKun27ZNE1emUJLVhQ6MjUkoFKU28lFIqL06njJoFamWkUqpE0FWNSqn88Xhgzx7Yvz93gXg4SUqSzdmjo+Xj5ZeNjkgppTTxUqpEOXsW2rWTAvu6dWWz8sx+XuHmwQfh55+lPUV6uvTnmjnT6KiUUiWcJl5KlSRPPCHd61NSpMZr8WJ49VWjoyoeixZJ64xMDgfMn29cPEopBUQYHYBShnA64fPPZYua1q2hSxejIwqMFSt8k5GUFFi2zLh4ilNCgjRDzRQVBdWq5e++mzfD9Omy8ODOO2VLIKWU8gNNvFTJ43LB9ddLEpKSIvU/w4fDiBFGR1b86teXbW0ym6pGRcHllxsbU3GZNAmuucZbx1alCjzyyMXvt3w5dO7sXfX5yivw55/aLkIp5Rf+WOYzGbgR+AdofJ7bvA10BRzAAODPPG6jqxpVYPz8M/To4bs5tNUKZ85IEhbOjhyRvlTHj0tCUru27IUYF2d0ZMVj716ZXoyJkf/z7M1kz6dDB/mZZLJY4L774L33ct82JQW2bYPy5XVUTKkSprCrGv0x4vUR8A4w5Txf7wbUAeoCbYCJwAU6EipVzE6ezN0s1GSSxpnhnnglJMCmTTLaFxEBV1wRuC15jFCjBtx7b8Huc/Kk77HLJYlqTuvWyYhaWpoU7z/2WPjWyyml/MYfxfW/Aicv8PUewCfnPl8OlAYS/PC4ShVOu3a+exVaLFCnTuC2psmUlCQjJoEWHQ1XXw1XXhm8SdepUzB0KNx0E/znP96NswOhTx/fkTG7Hfr2zX27Xr1kM+2zZyX5evdd6XSvlFIXEIhVjVWBfdmO9wP5rHBVqhhUrizTT5deKi+q7drBggWBa7CZnCw1ZmXKQHw8DB7smwiWdKmp0KYNTJgAs2bBCy/A3XcH7vFHjoSHHpL/nwoVpMarTx/f23g8sHu37zmnEzZsCFiYSqnQFKji+pyvaHkWc40aNSrr88TERBITE4svIlWytW0L27cb89hDh8Kvv8oLNchmy82ayYu9gl9+kdWImasvHQ6YNk32kzzfptX+ZLHA2LHycT4mE1SvLo1oM0VEQL16xR+fUsoQixcvZvHixUW+jr/e4tcCfiDv4vr3gMXAl+eONwMdgSM5bqfF9apkqFcPtm71PXfrrfDNN8bEE2x++gnuuEMWO2SKjJRkLNDTwReyejVce62MVmZkwMCBMt2oWxMpVSIE85ZB3wP3nPu8LXCK3EmXUiVHzZq+xf2RkbK6UIn27WUK2GKR4+hoOVemjLFx5dSypUw3zpkDa9fC+PH+S7o8Hhlxq1FDfl8mTvTPdZVShvPHs8RUZASrPJJQPQ9kVuy+f+7fd4EbgGRgILAmj+voiJcqGbZvl6nOzK16KlWClSsDM40WKvbvh3/9C3bskKRr7Nj8tYIIF++9B08+KfWAIN/7Bx9Av37GxqWUylLYEa9gGhPXxEuVHCdOyHY9kZHSrNNmMzoiFUzatcu9o0CXLjK6ppQKCpp4KaVUuLj++tz7Sloskqjfey+8+aZ3KlYpZYhgrvFSSilVEKNHS7f97Fwu6fs2ebK0uFBKhSRNvJRSKti0bi1TjU8+CZdc4vs1hwNmzjQmLqVUkekm2UoFE7cbZs+Gf/6ROp/69Y2OSBmlUSN4/XU4elT2nMzs3m8yQcWKxsamlCo0rfFSKli4XNCtGyxdKu0EPB744gvo2dPoyELb1q3S9qFBA2l6Gmr27oXmzWWa0e2WOq9ly6BhQ6MjU6pE0+J6pULdzJmyNU5SkvdcqVKyb6EqnNGj4eWXJVlJT4ePP85738Vgd+QITJ8uyXmvXqGZQCoVZjTxUirUvfeebCeUfeNss1kShpKygs3tlk2xp0yR4vJXX5V2G4WxeTO0aOH787TZZGPrktQTTClVLHRVo1Khrm1b32OLBZo0KTlJF8CYMfDSS7BpE6xaBT16SHPZwti1C6xW33Nms4weKaWUQTTxUqqonE4YPly2/WnWDBYuLNx1mjWD99+XURmzWWqSfvgh79suXgx33gmDBsHffxc69KDz4Yeyai+TwwGff164azVoIHsoZhcRAVWqFD4+pZQqIp1qVKqonnhCpgkzEwa7HX79Vaa5CsPjgbQ02aMwPV16Ni1dKsXUL7wAv/0mm2pnTqHFxMjXmzTxz/djpLp1ZUulTGYz/PvfMuVYGJ99BvffL6OGERGSyHbo4J9YlVIlmtZ4qfCUkQHvvCObELdsCQ8/LC+gwaRiRVnyn8lkgmeekWmzovB44Kab4OefJcmKioI6deTfNTm2Ox0wAD76qGiPFwy++EISJYdDfo5xcfDnn0XbRPzsWTh8WArSo6P9F6tSqkQrbOIVZK9gSmXjdkt7hd9/l8Rj2jRYsAC++05elLPbuRM2bIBataBx48DGGRXle2yx+Kd4+9AhmbZMS5PjtDRpLZBXD6fsBeShrF8/KFsWPv1Ukq4nnyxa0gVynbi43OdXroQ+feDAAbjsMvj2W/lXKaWKkY54qeC1bp0UnGev+bHZ5Pyll3rPffEF3HefFFJnZMjU30svBS7OKVPgoYckTotFWkCsXw+VKxftuvv2SSKQmuo9Fxcn3+v773t/LjYb/PgjXHNN0R6vJDl+XBK6M2fk2GSS/6/du3MX5F/M+vUyFXr2LAwcKO0elFJhT6caVfhZuVJaCZw96z0XGwt//CFdvUGSj3LlfJMTux1WrIDLLy/8Y3s8ck2bLX+3nzsXvvkGSpeGxx6DatUK/9jZY+jQAVavllisVqhaVUb2pkyBCROkP9WoUTIlqfJv0SK4+WY4fdp7LiYG/vpLpnPza/NmuOIKSE6W/y+7Xf5f+vf3f8wFkflcmnNkWCnlN9pOQoWfxo1l2imznYLVCpUqQb163tv8848UYGdntcKePYV/3MWLJZmLjZVE588/L36fLl1kRd7Ysf5JukBeNOfMkVGU5s2loH7ZMnlxf/BBWc24apUmXYVRrlzuFY8ZGVCmTMGuM2mSN+kCeSNQ1Nq+ovB44MUX5XckMlKmbjOnqpVSQUFrvFTwio6W+q777oONG2XV3gcf+E4FVakiLzDZpyMzMgo/2nX0KHTv7u0ef/AgXHcd7N9vTGF2bKyMoBSF0ykvyAWdQgtnTZpA796yW0BGhvxsHn9cErKCyPzZ5jxnlKlTpQFt5gjwzJlQvjy8/bZxMSmlfOiIlwpuVavKptF79kgrgEqVfL8eGSlfL1NG3uXbbDB5MtSsWbjHW78+d8PStDSp/Qk1bjf861/yM7HZZKscHf0QJpMU8H/2mWwpNHNm4eoC+/f3XUhht8Mjj/gvzoKaNUtG4DKlpMjfh1IqaOiIlwp9bdtKN/JDh6BChfzXZeWlUiXpnZVdRoZcN9RMmCBJaOYIzI8/SqPXN94wNq5gYTIVvRC+ZUuZDh45UhKeQYNkoYVRqlTxLjLJlJBgXDxKqVyCqfJSi+tVcHjsManXAplGGjVKmniGmu7dJdnKrnHj8Op0XxgrV0rD2UqV4JZbgq8vXFEcOyY7IJw8Kb+7Fos0823WzOjIlAo72sdLKX958015Qd6+XRKVVq2Mjqhwatb0Hf0wm/1X+B+qPvpIpgJdLkm43ntPesOFy36Y5cvLqtcZM2RauVs3qFHD6KiUUtnoiJdS4eroUdm26NQpObZapc1GQdolhJPMdg/ZW4/ExkofuO7djYurKH78UQrqS5eWZrOXXGJ0REqVGDripZTyVaGCjH7Mni11XtdfH5q1av6Snp67fs/jkem5UJQ5eudwyGjm55/LNLKOcCkV1HTESylVcrRoIcmJyyXHdrv0aQvFrYJq1JDdDTJZLPDss/D888bFpFQJog1UlVIqO7cbTpyQfzPNmiU1exaL1EN9801oJl2Qe/TO7fadRj2f/ftlscgDD0h9m1IqoHTESxXdkSPw88/SYLRLl6K1c1DKH5YskVYRDoeMan3/vWy/FOocDti0SWq6Jk+WhSCZzYPtdmk4fKEVjAcOSPPY06dl1M9ul6bE/foFJn6lwoju1aiMsWEDXHWVd+qmalUp4I6PNzaugti8Gd55R5pNDhwYHi/QJdmpUzINl32Pz7g4GekJpd/LnLZsgauvllGtjAxpiHvppVJcHxcnHes7drzwNV58URrFZu+uX6sW7NpVrKErFY60uF4ZY/BgOHPGu23K7t3SoPOFFwwNK982bYLWrb377X31FUybBl27nv8+GRnwv//B1q2yQfLtt+tmxMFk69bc/x8mE2zbJg1PQ1XfvrJSNfNvbdo0+Phj2U4rvxyO3FsapaT4LUSl1MVpjZcqmgMHfPeqC7Xtdd58M/cmxyNHnv/2bjfccAM88QSMGwf33w9DhgQmVpU/Varkrn9KT4fKleXzn36SBKxRIxnpDJWR9u3bfWN1OM6fdJ08CfPnw/LlvjVut96ae4uju+8unniVUnnSxEsVTceOEBXlPY6JgWuuMS6egsqedGXKuZ+hx+M9t2KFfGTW1SQny+hXqLYkCEfVqknybLfLFJzdLrsPVKkCv/wiyceaNTJNPmyYJF+hoE4d35E8ux0aNsx9u3XrZAqyTx/o3BluvNFbCtCqFXz7rdR5XXKJ7NLw6quBiV8pBWjipYrq3XelJspikU7gDz4I99xjdFT5N3Bg7hGA++/3Hk+f7n3xbthQRh3MOf5sLBbfjYmV8UaMgN9+g0mTpOD86afl/OTJvlNrDod0rw8FX38tfdji472bnt9yS+7b9esnI16nT8vv5a+/wpQp3q9ffz2sXQs7d8KYMeHTtV+pEKE1XqpoYmNlSsPhkM7oVqvRERVM587SufzZZ2U66sEH4dFH5WubN0sSmTm6tWWLFCdbrTLy4PFIslmzJlSvbtz3oPLWvLl8ZBcd7f2/yxQZGdi4CqtePSmC37xZVjXWrp337fbu9T1OToYdO4o/PqVUvuiIl/IPu90/SVdKiiRy8+Z5E57i1rOnNNXcvFmmXjKnc5Yv9x3dcrtllGDePKkRKl8eEhOllUbOUTAVnB59VH5XM/+PbbbQWQgCEnuLFudPukCmEbOPYsXEyH2UUkEhmJZiaTuJku7YMVlhmFkvVbYsrFxp3DY3c+dKPVBSkveczSYjCLqKMXRt3Aj//a8k+ffeG1o1ifmxfz906gQHD8oKxocflu9Xf2eV8ivt46VC3733wqefSrsGkBG0fv1kybwR3G7o3RsWLZKpKbcbPvxQm02q4OdySQIWFydvYJRSfqd9vFTo27rVm3SBfL5tm3HxmM2yAmzuXBk9aNNGWhAoFewsFqk9VEoFHU28VPC4+mpYvdq76sxmg/btjY3JbL5wM1WllFKqAHSqUQWPtDRZHj9/vhx37iwjTtn7hCmllFJBQGu8VPg4flxqqsqXNzqSQnO5Xaw9shan20nThKZERWjyWChOp2yNc/iw7Al6xRVGR6RKII/HQ1J6ErGRsZkvtkpp4qVUsHBkOEj8OJGNRzdiNplJiE1g6aClVIgxaHVmqHI6ZdRz9Wr53GKB8eNhwACjI1MlyNJ9S+kxtQen004THxXPd7d/R/saBpdAqKCgiZdSQWL4wuGMWzaOVGcqAFazlVsa3MLUW6caHFmI+e47uOsubeehDHM27SzVxlXjTNqZrHPxUfHsfWwvpaJLGRiZCgaFTby066NSfrb2yNqspAsgw53BhqMbDIwoRC1YkHsrprQ035WvShWjbSfyXlW95fiWAEeiwokmXkr5WesqrbFF2LKOoyxRtKisncMLZPFi6ZmWfRTcZJItgEJlix8V8irFViLdme5zLt2VTuXYygZFpMKBJl5K+dmw9sO4svqV2CJs2K126pevz5s3vGl0WKHlo48gNdX3XFQU/PijMfGoEqlKXBVGXD0Cu9VObGQsdqudYVcNo3op3Zv1fFxuF88seIYqb1Sh9lu1+XrD10aHFHSCqVBCa7xU2PB4POw8uROn20mdsnWwmC0Xv5PyevBBmDTJd8SraVP46y/jYlIl1qqDq9h0dBP1y9fniqq6svZCRi4aybhl43BkyF67dqudH+74gWsuCbOtudDieqVUONmyBVq1khovj0c2h/7qK7jpJt/bjBkDp0/DnXdC377GxVtYR47AP//ApZfK96hUiKv9Vm12ndrlc+6Blg/w/k3vGxRR8dEtg5S6GI9HuuLbbLoqLtjVqycbpL/9tkw59u8PHTt6v75zp/T0SkqS/9cFC+DECRkpCxUvvwwvvig1axERMG+eJJtKhbCYyBifY4vJQqkoXQGaXTC9+uiIlyo+v/0GvXrBqVNQrpzUCmkzTmNkZEhPLnMRSkyffVYSF7fbe65aNdi3r+jxBcKKFdCpEzgc3nOVKsGhQ8bFlF9JSbIBd9Wqsgm3CmtOtxOzyYzZlL+/1znb53DL17fgyHBgMVmIj4pn7YNrw7IuTttJqNDh8fjW7hS3kyehWzfpiO9yydROly65WxWo4pWcLP8PNpt8vPBC4a/ldOb+HXK5ihZfIG3YkHvU9ehR30QsGM2aBQkJ8qalUiXZ0kuFpZSMFHp/1Zvo0dFEj47mmQXPkJ/BkRvq3MCiexbxZLsnGdFhBH8/9HdYJl1FoYmXChynE+67T1an2Wzw1FOBScA2bco9uuJ0wvbtxf/YymvIEFi0SBKk9HT4z3/gm28Kd60775TfoUwxMfDQQ/6JMxAuuyz3737p0r7fU7A5dQpuu02Sw6Qk+feuuyRhVGFn6NyhzNk+B5fHRYY7g3dWvMOnf3+ar/u2qdaG169/nRc6vUC1+GrFHGno0cRLBc4LL8DUqTLVlJYm279MnFj8j1upkrzQZ5eeLu/cVeAsXCj/75kcDu+G6AXVqJFcr1MnaNECXnoJRo70T5yBcNVVkohGR0OpUjJlN3NmcNce7tolU8TZWa36BiZMzd8536cRdHJGMnO3zzUwovChiZcKnFmzfKdSHA45V9xq14b/+z9ZNRYTI/8++6wkZCpwKudoOhkVBdWLMAXRtq2MoK1eDY8/HtxJS17+8x9Yv17qDffsgfZBvv9f9ep5v4GpWdOYeFSxqhxXGVO28qVISyQ1StUwMKLwEUzPVFpcH+5uvBFmz/ZOsUREyIbHH3wQmMf/4w9pQdCwIbRuHZjHVF5r1kBiohTEm0yS+K5eDfHxBbrMvtP7eGv5W5xJO0Pfy/tybe1riydeldvkyfDII7ISMz0dxo2DwYONjkoVg3VH1tH+o/a43C5MmChnL8efg/+kjK2M0aEFDe3jpYLfli3Qpo13r73YWHkxrlrV2LhU4OzfL60fbDbo3r3Avav2n9lPk4lNOJN2BpfHhS3Cxv96/I87Gt9RTAGrXPbtg61boU4dHe0KcwfPHmTejnlEWaLoXq87sZGxRocUVDTxUqHh4EGZWrFYpL1DuXJGR6RCyPM/P8/Lv72M0+3MOlerdC12/d+uC9xLKaX8TxuoqtBQpQo88IDRUagQ5XA6fJIuwKcAWCmlgp0W16vgcfy4FNsvWRJaPZlKuANnDvDV+q+YvW12rqTI3/o27IstwttywW61c0+Te4r1MZVSyp90qlH5T2btltVa8PuuWwdXXy2F1243NGsm7QIiI/0bo/Kr5fuXc+2n12LChAcPjSo2YsmAJURaiu//be72ufx7/r9JSk+iX+N+vJD4gm5CrpQKOK3xUsZxOmHQIPjiCzm++2748MPcPX8upEUL+PNP77HNBmPHwsMP+zdW5Vf1363PluNbso7tEXbe6PIGD7YKoT0TSzq3W940RUUZHYlSIUW3DFLGGTMGpk+X6UGXC77+Gl59tWDX2LvX9zglBXbs8F+MRtq1Szq0//ZbYLdKCoDDSYd9jh1OB/tOh8h+iQpef13e5Njtsgn5yZNGR6RU2NPESxXdvHm5G6POm1ewa7RqJX29MsXESOuJ4rZihUxxNmkCo0f7brrsDz/QzbKvAAAgAElEQVT8IF3W77sPbrhBtlgJo+SrTdU2WM3eqWW71c5VNa4yMCKVb7Nnw6hR0o/L7YZly6SvnlKqWGnipYquenXfacWIiIL395kyBRo0kC1UrFZZ+dinj3/jzGnzZrjmGvj1V6kxe+UVGDbMf9f3eGRPQYcDzpyRTaK/+05q18LEZzd/RtOEpkSYI4gwR/BM+2foVreb0WGp/PjlF983TOnp8reglCpW2k5CFd3rr8PPP3ufxGNiJIkpiIoVYe1aOHJE7h8X5/84c5o2DVKztSJwOOB//5OtXPwhNVWSrZz2hc9UXIWYCqx8YCVn0s5gi7BhtRRiYYXyj08/lWn+8uVlS6zatS98+ypVZJoxJcV7TvcvVarYaeKliq56dRk9mjNHjrt2hdKlC36dzG1kAiUyEsxm39YVBVkQcDE2G9SqJTVemdOLbje0bOm/xwgS8VEF2/ZH+dkbb8Bzz8mbB7MZvv1W9oGsVu3897nvPnmjsWOH/H6aTHKslCpWuqpRlVwHDkj91ZkzkhDZ7fDyy7Khtr9s3QrXXScjeQATJ8LAgf67vlIAFSrAsWPeY6sVXnoJnn76wvdLT5feeWfOyD6axbkFUEqK1FRaLLJXqraKUSFOO9crMWeOtHKw2+Hf/4bGjY2OKHhVrSotLF55RV60brsN+vb172Ncdhns3i3NYUuVKlyPM6UuJmfDYbdb2rxcTGQk9O5dPDFld+QItG0LJ05IbLVqwe+/F3iDdBV6TqScYO72uVjMFm6oc4OOjuOfEa8bgDcBC/Ah8FqOrycC3wE7zx1PB0bncR0d8Sqqb76RVUkOh0wb2O2wfDlcfrnRkanCGj8eXntNXlj/9S8ZwTAF00C1CgrDhsE77/jWWa5ZI4l/MLj9dpgxw9tkOSoKhgyRKVIVtnaf2k3rD1qT4pQ6wvioeFY/sJpKsQEsKSlGRvXxsgDvIslXQ+AOoEEet1sCND/3kVfSpfzhxRe9T7wej3z+7rvGxqQK74sv4KmnpBj/4EGZOpowwS+Xdrl1Sya/SEqCwYOlHcmtt8KhQ8bE8fLLUlDfvDl07iwrFoMl6QLYssWbdAGkpcGGDcbFowJi6NyhnEg5QVJ6EknpSfyT/A8jF400OizDFTXxag1sB3YDGcCXQM88bqdv0QMh+xMbSPKVnm5MLKEgLQ3694fYWFkJNmmS0RH5+uyz3P3Rpkwp0iXXHVnHJW9dgvUlK5XHVmbpvqVFDLIE83ikN9snn0g7ku++k95z2f/PAsVsllGvNWtgwQLZCSKYtGnj2xnfZoMrrzQuHhUQ+07vw+Xxvslzup3sObXHwIiCQ1ETr6pA9rXx+8+dy84DXAmsBX5CRsZUcXjkEZlezGS3w733GhdPsHvsMZmeTU6WGqzHH/euzAwG8fG5pxVLlSr05dKcaVwz5Rp2n9qNBw+Hkw9zw2c3cNxxvIiBllCHDsHq1ZLAg9RUnT4tjUiVr7FjJRmMjpaPxET/9sxTAeV0Oxk6dygJYxOo9WYtvlz/ZZ63u7b2tbk2tb++zvWBCjNoFbW4Pj9FWWuA6oAD6ArMBPIcAx81alTW54mJiSQmJhYxvBJmyBBZMfTBB/Lk9sIL+q7yQn74wbeHkcMh5264wbiYsnvuOVlx5nDI6IrdLtszFdLOkztJdab6nDObzKz7Zx2JtRKLGGwJFBGRexcCj8d3BwYlYmOlmH7/fnmOqlxZaxVD2PCFw3l/9fs4MmR0997v76ViTEWuueQan9u90OkFdp3axbSN0wC4o9EdDG07NODx+svixYtZvHhxka9T1N/8tsAopMYL4BnATe4C++x2AS2BEznOa3G9CqxGjXzrTKxWGD5ctlHJzu2WqRwjbN8uU1lOp2w3VISFEsccx6j232qkudKyztkibKx+YDUNKuRVmqkuqlcvmD9fkuOoKKhTR6b7tFWCCmM136zJ3tO+++s+1OohJtyYdw1quisdE6awa7BsVHH9KqAuUAuIBG4Dvs9xm4RsgbU+93nOpEupwHv3XRlFslplhLBCBVk5mOnPP6WvUUSENIldtSrwMdapI0X1r7xS5NWp5e3lGd5hOHarHVuEjRhrDAOaDdCkqyi++QZGjIBu3eR3548/jE26UlJkZGnVqtwtJkLB99/Lz/ODD3LXrKqgERsZ63NsMVkoFXX+MohIS2TYJV1F4Y+x3q5420n8D3gFGHzua+8DQ4CHACcy3TgUyKsIQke8VOBt2AA//SQJWL9+UKaMnE9Ohho1pO9QptKlYe/ewGxnVIx+2/sb646so265unS+pHPmuzYV6g4ckNKCU6ck6WrWTArto6ONjix/RoyAt96Svz27HVq1gkWL/LubhPKLOdvncPNXN5PiTCHCHEGpqFKsfXAtVeNzlniHt8KOeAXTM64mXip4rFkDnTpJR+9M8fEyrdS6tXFxKXU+N90ki0MyR7psNklmRowwNq78cDhk4Uj2pq+xsTIC1qmTcXGp81pxYAUzNs0gxhrDfS3uo3JcZaNDCjjtXK+UP1WokLsVR3q6bOatVDDatMl3ejElRdpchILkZBnZyp54mc2+b3xUUGldtTWtq+qb0MIwqGJYKQOtWSMbVVetKtOLZ8/mvk316rJnY0yMjBzExMiq0Vq1ivbYbre0ITCi15MKby1a+G5JZbdL/6xQUL481K2be0Vou3bGxKNUMdKpRhU8UlLg4Ydh9myptRo/Hq655uL3K4gDB6BBA2+yFRUFHTrIFGJeFi+GjRuhfv2ix7J7t1zj0CFJwF588eKbGCuVX8ePQ8eO8nvmckGXLjBtWui0tzhyRN4IrV4tb4o+/TT4GsEqlY3WeKnQd9ttUtOReq7XlN0OK1dCQz/23P30U0nukpK85ywWmerI3lm7ODRvDn//LUkXyPf300/yYqmUP7hcsHOn/C5Xr669spQqRka1k1DKf374wZt0gSwn93cn+eyd/TOZTIEZFVi/3pt0gdSzrF5d/I+rSg6LRabsatQwNunyeGDcONkv8vLLpe2GUgrQ4npV3JxOaQC6a5csD+/Z8/wvCFFRvp3krVaprfKnbt2gWjWJJy1NErEnnwzMkvWEBJnqzGS1Sp+wC/F4JFnTJfUqlIwfDyNHemsZ+/eXVcFduhgbl1JBIJjGoXWqMVjNmiXNRC+9VKYD89vF3e2W7Xd+/12egGNi4KGH4PXX8779pEmyX6LDIU0oK1WSVVnx8f77XkCmGd95R3pyde4Mt97q3+ufz2+/Qdeu8vNzueDaa2HGjPP/PN99F556SlZTJibC9OlF2qtRqYBp2lSm1bPr2xe++sqYeFSBnEo9xf0/3M/SfUupHl+dD3t8SKOKjYwOK+hojZcqHk89BRMmyEiUzQbXXSfJQn6mMX7/XRKv7PVUViscPXr+BGL+fCmur1BBkrTSpf3zfQSLQ4ekbq1cOWl2eb6f44IFMjqYOWIQGSlJ28yZgYtVqcJq2xaWL/cem0wwYABMnmxYSCr/rvzflaw+tDprq59S0aXY+shWKsRUMDq0oKKJl/K/48ehShXfflYxMbBkibRjuJjZs+H223178dhssHWrTPep8xsxAl5+2fdc6dJw8qQx8ShVEPPnyxuHlBRJumJiJBHz50IZVSxOpZ6i4usVyXB7t2yKi4zj414fc3ODmw2MLPhoA1Xlf6dPywhV9sQrIkK2JMmP1q19p9EsFllpVaWKf+MMR5UqSZKaveatfHnj4lGqIK67TkZtP/5YajeHDJGWLCroRVmi8OA7COLBg92ax8IkVSg64qXOz+mUTZr37fOuxitdGnbsgLJl83eNtWvhrruknqp5c/jiC0288sPhkOaXu3d7f/baekIpFQBD5w5l0upJJGckEx0RTf3y9Vl+33IiLQZuAB+EdKpRFY/du6FPH9lMukYNmDpVEihV/FJTpabrzBlpvFqnjtERKaVKAI/Hw9T1U/llzy/ULlObR1o/oiNeedDESxnrl1/g/fdlKvKxxzQ5UyocLV8ub75sNnjwwYu3Q1EqjGnipYwzbx706uWtR7LbJRHLTwG+Uio0zJ0LN98s0+BmM8TFSZuZSy4xOjKlDKGd65VxXnzRtwjc4YCxY42LRymVN5cLli2TlcnJyQW779NPe9ubuN2y3+m4cb632bdPRsXyuwBHqRJIEy9VdNlXPWbKvvWPUsp4KSlw1VWy4rBHD1lluG9f/u+fM1Fzu3179I0ZI1sEXX+9rF5essQ/cSsVZjTxUkX38MO+eyBm1n8opYLH2LGyyjgpSRZsHDpUsL/Te+7J/Xfer598/uef0ncuNVWunZQkfbyy702qlAK0j5fyhwED5An27belV9dzz+mebEoFmw0bfEeiXS7YsiX/9x8xQlrMfPSR9OYaPVq2vQK5Ts79RFNSpOFvuXJFj12pMKLF9UqpEm/HiR3M2jaL6Iho+l7el9LRYbZVFcCbb0rylH0bqt694csvi37tv/6SLbCy13qWKQPHjuV/b1elQoyualRKqUJYvn85nad0xul2YjFZKGMrw9oH11LOHmYjNU6nbHI/a5aMTl12GSxcmP9myBfz2mswapQkdB6PPE6HDv65tlJBSBMvpZQqhFaTWrH60OqsY6vZyr+v/DdjOo8xMKpidPCgLIipUcP/o1EHD0rtWN26EB/v32srFWR0r0alCuOff+Df/4Zt2+Td+YsvSv2KKhKX28WsbbM4knSEK6tfyeUVLzc6pPM67jjuc5zhzuBw0mGDogkAf2zZ5XTCZ5/JzhatW0O3bt5r65ZgSl2QJl6q5HI45EXj4EHIyJA6lXXrZE9EVWgut4sbPruBZQeW4fa48Xg8fNr7U25peIvRoeWpW91ufPTXR6Q4pT7JbrXTvV53g6MKYm43dO0Kf/whf0M2GzzxhLxpUUpdlFY9qpLr11/hxAlJukAKgxcuhOPHL3y/7L78UqZsKlSAf/3Le60S7IetP7DswDKS0pNwZDhIcaYw6PtBRod1Xv/t8l961+9NlCWKuMg4Xuz0Ir3q9zI6LOMcOSJvQs6ezfvrv/0mTViTk6WWy+GAV18teENWpUooHfEKdXPnyghNQoL00yodhquxiovpPFPz5zuf0+LFcO+93lVikydL0fKbb/olvFB1JOkILrfL59zZtLO43C4sZst57mWcqIgoPr/lcz7nc6NDKX7Hj8Ps2fI73q2brDzMbtw4GD78wgXyp07lrg2zWCRRi4kp3vhVvng8HtYeWcuZtDM0q9SM+CittwsmmniFsgkTpD7J4ZAnykmT4O+/tag1v9q3h/LlpbdRRoZMmXTunP9VXjNmeJMukM+nTSsRidePW3/kpV9ewuV28WibR7mn6T1ZX2tXvZ3PbS0mC00SmgRl0lWi7NkDrVp5e3nZ7bBmDVStKsfr18PIkfL1zNv06CEtIbL36GrTRpKyTBYL1Kolb/6U4VxuF72/6s2iXYuwmC1EmiP5ddCv1C9f3+jQ1Dk61RjKnnnG+8Kfng5Hj8I33xgbUyix22HFCunI3aEDPP44TJ+e//uXLg0ROd67xMb6N8YgtGDnAvp+05cVB1aw+tBqHpr1EFPWTsn6epOEJnzU8yNiI2MxYaJxQmN+uOMHAyNWgLxJO3lSusonJck0+/Dh3q9v2pT79zklJffUe0ICLFok7ShiY6FdO5miz+9IsSpWU9ZOYeGuhSRnJHMm7QzHU47Tb3o/o8NS2eiIVyhLS/M9drl8905TF1e+PHz4YeHuO2QIvPeeTL04nTJilnPT4DA0YeWErEJ0AEeGg3dXvOsz6nVbo9voe3lfnG4nVovViDBVTvv2yXNEJqfTd6/Gyy6Tc9lFRubdeb5Vq4J1vVcBs/X4VhwZ3pF4Dx52ntzpl2v/tO0nVh1cRc1SNbmryV06il1IOuIVyrp3h+ho77HFAjfcYFw8JU1CgqyCfOEFGX1cvFhWe4Uwj8fD9hPb2Xp8K25P3vvsRVoic52zmnMnVyaTSZOunE6dkk2krVYZMf3008A9dpcuvnst2u2+W3s1bSojYNHRUKqUjGZ9+23urYBUUGtWqRkxVm+tncVkoVHFRkW+7nM/P0efb/owavEohvw0hG6fdzvvc4S6sGAaG9YGqgWVkiIF9bNny7vSiRPh6quNjkqFqJSMFLp81iWrmWjDCg1ZdM8i4qLifG638sBKEj9JzHpXbYuwMb3vdLrWDe2kMyC6dZNpufR0Obbb5bht2+J/bKdTFoN8fm4RwaBB8pyRM7HauxcOHIB69fzX1V4FjMfj4cFZD/LJX58QYY6gQkwFlgxYQo1SNc57H6fbyaOzH+XzdZ8TZYliVOIoHr7i4ayvJ6UnUfa1smS4vau2YyNjmX3nbNrXaA9AhitDpjjTk2lfoz0JseFf86ed65VSRfL0/Kd5e8XbpDqlsDrKEsWAZgN476b3ct121cFVvLH0DZxuJw9f8TCdLukU6HBDk93uu5+hxSKbTQ8bFrgYMqcTc9ZzqbBy6OwhzqSdoXaZ2hcdeX5q/lOMXzEeh1PeTNmtdqbeMpUe9XpkXav227WznhsA4qPi+fzmz7npsptIyUih/Uft2Xp8Kx6Ph3RXOldWv5Kh7YZmXSMcFTbx0qlGpRQgyVT2J9Y0VxqrD67O87atqrRi6q1T+abvN5p0FUTOdi9RUVJnGEgRERdOuhYulCnIa6+VdhIqJFWOq0y98vXyNd0/fdP0rKQLpG5zxqYZWccJsQnULFUTi8l3dLR11dYAvLfqPTYe3UhSehLJGclkuDNYsmcJd0y/w2fhjRKaeCmlAGic0Jgoi3e7pEhLpF9qQ1Q2kybJqFdUlPS8uvRSuOsuo6Py+vlnqR2dN08SsL594QddkRruykT79nOLMEdQwV4h69hsMrPwnoW0rdaWGGsMl5W9jIX3LKRiTEUAdpzc4fOmLZMjw8HoX0YXb/AhSKcalVKANDm9+uOr2XZ8G+mudKwWKw+3epjR14wmKkL3r/SbdeukHUPZstCnj+8CGaP16JE70WrfXnZ5UGHr972/c/1n15PmTCPCHEF8VDxrH1xL5bjK+br/1xu+ZtB3g0jOyL17Qe0ytdnx6A5/hxwUtMZLKVVkjgwHTSY2Ye/pvWS4M7BF2OhYsyM/3flT5pNMgaS70jmTdoZytnKFur8KsJ494fvvfc9p4lUibDy6ke+3fE+UJYo7m9yZNZqVHx6Ph2ELhvHGH2/g8nhbltitdsZcM4bH2j5WHCEbThMvpVSR/b73d7p+3pWz6d59+qIt0Wx/dDtV46sW6FrvrHiHJ+Y+gclkokapGsy/ez61Stfyc8TKrxYvlpWXmQsA7HbZj7S7bhquLi7VmcrCnQt57ffXcGQ4GNR8EA+1eihs33Rp4qWUKrJf9vzCTV/c5Jt4RUSz5ZEtF1yOntOy/cvoPKVzVssJs8lMo4qNWPvgWr/HrPxs0SL4z3+k2epjj8GNNxodkVJBqbCJl64nVkplaV21NeXt5Ul1ppLhziDKEkXzSs2pHl+9QNdZcWCFz0bZbo+b9f+sx+Px+P3dr9vjZs+pPVgtVqrGVQ3bd9cBc8018qGUKha6qlGpMOfxeEhKz99WUtER0Sy/bzl9Lu9D80rNGdR8EPPunlfgZKZ6fPVcy9jL28v7PSk6lXqKKz64gkYTG1H37br0/LInTrfz4ndUSimDBNNbQ51qVMrPftr2E7dNu41UZyqVYisx5845XF7x8mJ/XLfHTe8ve7No9yLMmHF5XHx3+3d0rt3Zr49z97d38/WGr0l3SSd4e4Sd5xOf56mrnvLr4yilVE5a46WU8rHv9D7qj6/vs2FupdhK7H98f0A2t/V4PCzZs4RjjmO0rtq6QDVi+dVgfAM2H9vsc653/d7MuG3Gee6hlFL+oZ3rlVI+/jz8JxFm3zLO06mnOZx0OCCPbzKZSKyVyK0Nby2WpAugXrl6RJi832N0RLQ2fTXKkiXQsCFUqgQDBoDD4fv1U6dkf8jmzaF/fzhxwpAwlTKaFtcrFaaqxFXJVe/k8rgoawufjY8n3DiBdofbcSr1FG6Pm3rl6jGsfQD3PVRiyxZpQ5GZbH31FSQlwbRpcuxyQceOsHmzbBC+cSOsXAlr14L14lvaKBVONPFSKky1qtKKfo37MXXdVEwmEy63i7e6voXNajM6NL+pEleFzUM2s+rgKiItkbSs0jLXKJ8KgLlzJbnKlJrq2wF/82bYsUOSLpB/9+2D9etlBEypEkSfoZQKY5NumsTdTe5mz6k9NKvUjMYJjQt0/yNJR+g/sz9/Hv6TS0pfwie9PqFe+XrFFG3h2Kw2OtTsYHQYJVtMDFhy1A1m3wrJYoGcNbweT+77KFUCaHG9UipPbo+bRhMase3ENpxuJyZMlLOVY/uj2ykVXcro8FQwOXsWmjSBgwdlNMtuh9dfh4cflq+73TLVuGqVjIZFR0PTpvD775p8qZClxfVKKb/ad3ofu0/tzqoT8+Ah3Z3OqoOrDI7MfzweDy8teYn4V+KJfTmWR2c/6tP4VeVTXBz89ReMGiXd7mfM8CZdAGYzzJsHQ4fC9dfDo49Kh/wSmnS5PW7GLRtHp0860W96P3ad3GV0SCqAdMRLKZWnY45jVP1v1aweWQCxkbEsuHsBbaq1MTAy/5n852T+NftfWS037FY7w64axrMdnzU4MhWsMlwZALkaBBfEk/OeZOKqiTgyHFhMFuKj4tk0ZBMJsQn+ClMFgI54KaX8qry9PP2b9sdutQNgi7DRsnJLrqh6hcGR+c+MTTN8+pw5Mhx8u/lbAyNSwcrpdnLPt/dgG2PDNsbGoO8GFXp0dPzK8Vm/dy6Pi1Rnqv7elSBaXK+UOq/3b3qfq2tezbL9y2hQvgEPtHwAsyl83q8lxCRgMVlweeQF1ISJCvYKBkelgtGYX8YwfeP0rN+VrzZ8xaVlL2VEhxF+ub7O+JQc4fMMqi7M7S76NfbsgcGD4eab4csvi349FfRMJhN3NbmLd7u9y5DWQ4o0vRIoLreLg2cPkpKRctHbPp/4PKWiSxEdEU2kJZKYyBjGXj82AFGqUDNv5zwcTt/R0fk75mcdH00+yrAFwxgwcwAzNl1454T7W9yfNZJsNpmJioiiV/1exRN4EPvr8F90+qQTjSc0ZuSikSVmn1Ud8Qp306fDvffKqqMWLeD776Fy5YJf59Ah6bdz5oz065k7V1YwDR3q/5iVKqT1/6zn2inXcibtDG6Pm/E3jufe5vee9/Y1StVg48Mb+XrD17g8LnrV70Wt0rUCF7AKGdXjq/uMjkaYIrJ2ZDiZcpKm7zXlmOMYGe4Mvtn4DbtO7eKJdk/kea1xXcZRJa4K32/5nspxlXnt2teoHFeI5+UQtuvkLjp81IGk9CQAdp7cyXHHcSbeNNHgyIqfFteHs/XroXVrSDn3zj8iQpZwryrEqrQ33oDhw70NEAHKloXjx/0Tq1JF5PF4qD6uOgfOHsg6Z4+ws/z+5bqNkCqyfaf30WpSq6xRrxhrDKsfWE3V+Kq8v+p9Hp/7OClO7yhrXGQcZ545Y1S4Qe/NZW/y9IKnfRbv2K12kocnGxhVwRS2uF5HvMLZ77+DKdvvhNMJa9bIvxEF/K/PyMg9XenSZffKf447jrNg5wKsFitdLu1CTGRMge5/Nv0sR5KP+Jwzm838dfgvTbxUkVUvVZ1Nj2xizvY5mDDRtW5XSkeXBiDVmYrb4/v8mOHOMCLMkGE1W3PVi1pMJaO9iNZ4hbOKFaV/TnZ5dZjOj5tvhqgo77HdLhvhKnXO8v3LGTJrCEPnDmXr8a0Fuu+OEzuo92497vvhPvrP7E+jCY047ijYaGpsZCxRliifcx6Ph5qlahboOkqdT1lbWfo17scdje/ISroAbrzsRp+tqmwRNvo07GNEiCGj7+V9iY2MzUq27FZ7idlnVacaw5nTCddeC6tXe0enJk+G228v3PVWroQnn4QTJ+DWW2HkyBLbAFH5WrRrEd2/6I7D6cCEiZjIGFbct4IGFRrk6/5dP+/KvB3zskYNIs2RPHTFQ7x5w5sFiuPHrT9y27TbiDBH4HQ7uavxXbx303uZUwJKFZsVB1Yw5KchHHcc56bLbuL1614nKiLq4ncswQ6cOcDoX0ZzJPkIver34u4md4fU32phpxqD6TvUxKs4OJ3w3Xdw5AhcdZXUeCnlZ+0+bMeyA8uyjk2YGNBsAJN7Ts7X/RtNaMSGoxt8zvWs15OZt88scCx7Tu3hr8N/UTW+Kq2qtCrw/ZVSKj+0xkvlLSICbrnF6ChUmEvO8C2I9eDJWq2UH50u6cSOkztIdaYCMu1wbe1rCxVLzdI1qVlapxeVUsFJa7yUUkU2qPmgrL5EIInTgGYD8n3/1697netqX4fFZMFistC/aX8evuLhi99RKaVCjE41KqWKzOPxMPaPsby36j2sZivPd3yeOxrfUeDrpDpTMZvMRFoiiyFKpZTyH63xUkoppZQKEN0kW4WeBQtg9Gj45BNZBBDMZs2Cyy6Trv+PPOLbSFYppZTKJx3xUsZ47TV48UVITQWbDVq1goULg7M9xcqVkJgIjnP7tNls0L8/TAz/rS2UUioYeTweluxZwt7Te2lRuYUhTZJ1qlGFjvR0iI2VbviZYmNhxgy47jrj4jqfZ5+FMWMg++9nuXJw7JhxMSmllEFOppxkwHcDWLp3KQmxCUzuOZnWVVsH7PE9Hg8DvxvItI3TMJlMuNwuJt44kf7N+gcsBtCpRhVKMkeOsjOb4dSpwMeSH3FxYLX6nrPb876tUqHK44FPP4Xu3WHgQNixw+iIIC0N5syBmTOlcbMKCj2+7MGc7XM4lnKMDUc3cO2Ua9l/Zn/AHn/Z/mVM2ziN5IxkktKTSHGmMPjHwT77PgYzfyReNwCbgW3A0+e5zdvnvr4WaO6Hx1ShrHRpqF/fd79ItxvatTMupgsZOBDKlPEmX3Y7vP66sTGpAnlz2ZtUfL0iZV8rywid8vAAACAASURBVLAFw3Ltq6eA//4XHnwQfvwRpkyBli1hf+BeTHNJSpIY+vaFe+6RGstt24yLRwGQkpHCH/v+8ElyPHhYsnvJee/jcrvYcmwLe07twR8zWwfPHsRizl2Wcio1SN+851DUxMsCvIskXw2BO4Cce4R0A+oAdYEHAC2MUTBvHlx5pSQxtWvLu9pq1YyOKm8VKsC6dTLlOHQozJ4Nt91mdFQqn6aum8qIRSM46jjKydSTvLPiHV797VWjwwo+r77qHY12u+Xzzz83Lp6xY2H7djh7Vj5OnoTBg42LRwFgtVjz3NYnNjI2z9sfTT5K44mNaTmpJQ3GN6D3V71xuou2mKpF5RY+1zBhooK9AhXsFYp03UApauLVGtgO7AYygC+Bnjlu0wP45Nzny4HSQEIRH1eFukqVYMkSSE6WKY2rrjI6ogurUEESrzfegKuvNjoalU9Hk48y+a/JODK809uODAdfb/jawKiCVOZ+rpncbmNXG2/fLlON2ePZvduwcJSIMEfwfMfnsxom2yJs1Clbh651u+Z5+8E/Dmb7ie0kZyST4kxh/s75TFg5oUgxXFLmEqbeMpUYawwR5giql6rO/Hvmh8w+j0XdMqgqsC/b8X6gTT5uUw04UsTHVkqpPHk8HobOHcqEVRNwu3NPK5aJLmNAVEFu8GB4+23vqFd0NPTpY1w8iYnw7bfeeKKioEMH4+JRWUZePZKmCU35Zc8vVC9Vnftb3H/epsd/Hv6TDLd3IZUjw8GKAyuKHEOPej04+8xZkjOSzzvaFqyKmnjld7I2ZxqqyxeVUsXm+y3f88GaD3IV21pMFqIjovnPdf8xKLIgNmaM1F9+/bXUNP7nP1JXZZR774XVq+HDD8Fkgtat4d13jYtH+eherzvd63W/6O3ql6/PvtP7cHlkRNUWYaNJQhO/xGAymUIu6YKiJ14HgOrZjqsjI1oXuk21c+dyGTVqVNbniYmJJCYmFjE8pVRJtPbIWp/pRQCr2cpzHZ+jT8M+1CtfL+u8y+1i9K+jmbpuKnFRcYy9biwda3UMdMiB4/HAe+9JEX3VqjBqFFSpIiuLn35aPoKBySS98t54Q1rQlC5tdESqED7o/gFX/u9KTqedxuV20bJKS/6vzf8ZHVahLF68mMWLFxf5OkWdEI0AtgCdgYPACqTAflO223QDHjn3b1vgzXP/5qR9vJQKMb/s+YURi0aQnJ7MwGYDeaT1I8VWZ7Ht+Db6zejHjhM7aFihIZ/f/Dk1S9fM87af//05g38cTHJGcta5BuUbsHHIxly3HbZgGO+seCcrUbNb7fw+6HeaVWpWLN8HwP4z+/kn+R/qlatHTGRMsT1OnoYNg3fekSm8iAgoWxY2bpTedCospTpT+d+a/3Hw7EGurnk1Xep0Cejjp2SksPbIWqIsUTSt1BSzKTw6WRnZQLUrkkxZgP8BrwCZS0/eP/dv5srHZGAgsCaP62jipVQIWX1wNVd/dDUOpzdheSHxBZ688km/P1ZyejK1367N0eSjePBgMVmoGl+Vbf/almdtidvj5pavb2H+jvlEmCMwm8wsHrA4zymOhLEJ/JP8T9axCRPDOwxn9DWj/f59ADyz4BneXP4mkeZIIswRzL9nPi0qtyiWx8rF45GdF7IXrdvt8NZbcN99gYlBBVS6K522H7Zl87HNpDhTivXvtKQpbOJV1KlGgNnnPrJ7P8fxI354HKVUEJmydkpW0gVSNDt+xfhieUJfe2Qtqc5UPOfKQ10eFydSTrD9xHYaVmiY6/Zmk5kZfWfw1+G/OJV6iuaVm1M6Ou+pqpyJm8VsIcoS5ffvAWSE8O0Vb5PqTCWVVAB6ftmTfY/vu8g9/SjnYgOPJ/eKRhU2Zm2dxbYT20hxpgDydzpi0QiGthsaNiNPoUZ/6kqpQrFarJhyvNmLMPvjvVxucZFxuXr/ON1O4iLjznsfk8lE88rN6XRJp/MmXQAvJL6QtTTebDITGxnLwOYD/RN4DhuPbszVQPLAmQO43AFKfEwmuPtu784LJpM0Br7ppsA8vgq4M2lncp1zup1kuDLyuLUKhOJ5llRKhb0HWj7A+6vfJzk9GQ8e7FY7IzuOLJbHalSxEddfej3zd8wnOSOZGGsMfRr2oXqp6he/80UMaj6IijEV+XL9l5SOLs2/r/w31eKLp5lv/fL1c40yVI6rnGcX7mLz/vtQuTLMmiX99MaNkyJ7FZYSayX6JPuR5kjaVGtDVETxjOqqiwumbmNa46VUiNl4dCOv/vYqZ9POMqDZAHrWz9k/2X9cbhdT1k5h07FNNE1oSr/G/UKmYWJ2T857kvErxxNlicKEiXl3z+OKqlcYHZYKY0v3LWXgdwP5J/kf2tdoz5ReUyhj0152RWVkcb2/aOKlVAnh9rgxYQrJxMkfdp3cxT/J/9CgQgPio+KNDkcpVQiaeCmlgl5KRgr9ZvTjhy0/EGGOYHiH4TzX8Tmjw1JKqQIrbOKlxfVKqYB5dM6jzNk+B5fHRZorjdd+f033TVRKlSiaeCmlAmb+jvmkOlOzjh0ZDuZsn2NgREopFViaeCmlAiYhNsHnONISWWwrCEsCj8fDpNWTuPGLG7nv+/vYfybnjm0hwuOR1ZWVK8tKy9Gj5ZxSYUhrvJRSAbPm0Bo6ftwxq7i+QkwF1jywRldYFdLwhcN5e/nbJGckYzFZKGMrw6YhmyhvL290aAXzyScwZAgkn9viyW6XTbqHDDE2rhDz297fWLpvKVXiqnB7o9uLra+eElpcr5QKCXtP72XejnnYImz0rN+T2MhYo0MKWfYx9qyO5AC2CBvjuoxjcKvBF7hXELr+epg/3/dcu3awdKkx8YSg8SvG89SCp0h3pRNliaJllZYsumdRsfSI+/vI3yzdt5SEmAR61OsR2D50QcTILYOUUirfapSqwX0tdF/AIjl9Gj75hKcXpTOrNqw8N1vrwYPLE4Lb/5QpI130s7/5LlWqUJdafXA10zdNJzYylkHNB1EptpKfggxebo+bofOGku5KB6Qz/ZpDa5izfQ43XnajXx/r6w1fM3DmQDx4MJvMtK7amvl3zy+xyVdh6IiXUkqFktOnoVkzOHwYd1oqqRFwV2+Y2dBEXFQcGx7eEHp1c5s2QZs24HB4N/L+9Vdo3rxAl5m7fS69v+pNqjOVCHMEpaJLsfbBtVSJq1JMgfvyeDz8vPtn9p7eS4vKLfLclL04ODIcxL8S75N0x0bGMqHbBO5uerdfHsPj8eDxeCj9WmnOpp/1eZxPe39Kr/q9/PI4oURHvJRSqiT45BM4fBhSUzED9gyYOM9KSo/OvHH9G6GXdP1/e3ceH1V1/3/8NVuWCRFQQlgFUdmUVQUF1CCIKCClVavigvar0oLaryhW7a9VW79abLVa1IqKUtwFVxYF1ChIEUVCqYhssicYQsg2SWa7vz8mDA5JIDCTe2cm7+fjwYO5d87c87k5gXxyzrnnAPToAXl5MHt2aBPvq6+Gbt2O+jJ3LLojPPTqC/rYX7Wf6Sun83/D/i/WEddiGAYT3pvA3HVzgVAv1PRLpnNjvxsbvW63y03v7N6s3bMWv+EP1z/4xMFRX9swDKYunso/Vv6DoBHEF4zc4zFoBNlTvifqepoSJV4iIolk/37weiNOZQfTWTh+oUUBxUiXLvDHP0Z1ibLqsohjf9DP/qr9UV2zoVbsXMHcdXOp8FWEz/1m/m+4pvc1pDhSGr3+BeMXcNmbl/H17q9p5W7FrJ/NokvLLlFfd/rK6Tz99dNUB6oBsNV08BiERqgMw2BQx0FR19OUaDkJEZFEMnIkpP5kg+O0NBgV23k8ieqK067A7XKHj90uN5f1vMyUuvPL8+uc51RcWWxK/W2atWHZjcuo+n0VO+/YybAuw2Jy3Q82fIDH5wkfGxiku9KxYyczJZNZP5tFr+xeMamrqVCPl4hIIhkwAF55BW69FcrKYPRoeO45q6OKC48MfwR/0M+ra18lzZnGI8Mf4YKTLjCl7v5t++MP+sPHNmxkubNondHalPobS/vM9jhsjvD8MbvNzvCThvP2L9/WhPpjpMn1IiIih7GxaCNPrnwSj9fDdX2u4/zO59dZbt6GeVw992o8Pg8dm3dk4fiFdG/V3eRoY2tn6U76PdsPj8+DYRikOFJYedNKup7Q1erQLKd1vEQk4Wwo2sCMVTPwB/1c1+c6+rftb3VIIhE2Fm3kjBlnUO4tDw2zOdN547I3GNNtTJ3lDcOg0l8ZMeSZ6PZ69vLu+ncJBANc2u1S2ma2tTqkuKDES0QSyrrCdQx8fiAV3goMDNwuNx9d8xFDThxidWgiYZMXTObpr54OTyYH6N26N2t+vcbCqCQeHGvipcn1ImKJh5c+HE66ILQW0X2f3GdxVBIL3oCXDzd9yLvr36XIU2R1OLX8Z89/eOe7d9hQtOGIZT0+T0TSBURs9C5ytDS5XkQsUeYtq/UDrcJbUU9pSRQV3grOfuFstu3fhg0bToeT5Tcup1uro1+XqzHcn3s/076Yhsvhwhfw8eTFTx52J4Xr+1zPG/99A48/9GSf2+XmV/1/ZVa4koTU4yUilri+z/W1Hv2f0HeCdQFJTPzt339jY9FGyrxllHpLKa4s5qYPbrI6LCA0p3DaF9Oo9FdSWl1Kpb+SyQsmU1JVUu9nzu98Pq9f9jq9W/em6/Fd+eP5f+SuQXeZGLUkG/V4iYglxvUYx1PVT/Gnz/9EIBhg0oBJTDprktVhSZQ2F28OL7YJoXWftpdstzCig3aU7CDFkRKxsbjL4aKgvIDmafXvDTmm25h6J9OLHC0lXiJimQl9J6iXK8mc3+l85qybE150M9WRyuCO0W9dEws9s3rW2vLGYXPQqUUniyKSpkhDjSIiMVJaXUpeQR6FFYVWh2KZG/rewA19b8Bpd+Kyuzir/Vk8M/oZq8MCoG1mW9647A3cLjdpzjRaprVkwfgFpDnT6v3M/A3zOffFcxk8czDvfPeOidFKstJyEiIiMfDxlo/52Rs/w26z4w14eXzE40w8a6LVYVnG4/PgDXhpkdbC6lBq8Qa87PXspXVGa5z2+gd+Ptr0EePeGBcemnS73Lw87mXG9RhnVqgSx7SOl4iIRbwBL62mtaLMe3CT5nRnOmsmruHUE061MDKJxiWvXMLCTZGbjw85cQhLb1hqUUQST7SOl4iIRfLL8sN72R2Q4kjh+6LvLYpIYqGu3rDD9ZCJNIQSLxGRKLVp1gbbIb/4egNeTj1evV2J7K5Bd+F2HlzyJN2Zzj1D7rEwIkkGGmoUEYmBBRsXcPlbl+Oyu6gOVPPQBQ9xxzl3WB2WRGnptqX87d9/w8DgtgG3MazLMKtDkjihOV4iIhYrrChkQ9EGTmx+Ih2bd7Q6HGlCqv3VfJP/DQ67g/5t+2tI1ARKvEQk6VT6Klm4aSGVvkqGdRlGm2ZtorqeYRj8e+e/Ka4s5qz2Z9E6o3WMIo1f8zbMY+bqmWS4Mrh7yN2c3vp0q0OSGCusKOScF87hx4ofMTDoekJXPpvwGc1SmlkdWlJT4iUiSaWsuowznzuT3WW7AbDb7Cy7YRm9snsd0/UCwQBjXx9L7tZcHHYHhmGw5LolDGg/IJZhx5XX1r7G/3zwP3h8HmzYcLvcfHXTV/TI6mF1aBJDV8+9mjnr5oQXh01zpHHrwFuZduE0iyNLbnqqUUSSymP/foxt+7dR7i2n3FtOWXUZt8y75Ziv9+a3b5K7NZcKXwWl1aWUecu4au5VMYy4ftv2b+OD7z8gryDPlPoOePDzB8MryBsYeHwenv76aVNjkMa3rnBdxIr8VYEq1u5Za2FEcjgaBBaRuLStZFutPf92le065utt3b+VKn9VxLn8svxjvl5Dvf3d21z7zrU47U78QT8Tz5jI3y76W6PXC+AP+iOODQx8AV89pSVRndHuDNbvXR/+95LuTE/qntxEpx4vEYlLw7sMx+06+Ch/qiOVnE45x3y9M9qdQaozNXzssDno1frYhi0byhfwcc3b1+DxeSitLsXj8/DPVf/k691fN2q9B0w+a3LE19DtcnND3xtMqVvM8/hFj3N669Nxu9ykO9MZ3HEw9557r9VhST3U4yUicemq069iTcEaHlvxGIZhcF6n83hq1FPHfL0RJ49g6qCpPLT0IZx2J+0y2zHnijkxjLi2fZX7MIicu+q0O9m6fytntjuzUesGuG3gbbgcLp7/5nncLjd/GvonBnYY2Oj1SsMEjSB5BXmUe8vp16YfmamZx3Sd41KPY+VNK9lSvAWHzUHnFp0PzD+SOBRPLaPJ9SJSiz/oxx/0H3Yj46NRVl1GaXUpbTPbYrc1bqd/0AjS5q9tKPQc3DQ73ZlO3sQ8up7QtVHrjpWVu1Zy+VuXs6t0F6ccfwrvXvku3Vt1tzqshOcP+hn96miWbV+Gw+4g1ZHKFzd+oS2mEogm14tIUnLanRFJly/gY0vxFkqqSo7pepmpmbQ/rn2jJ10QehLzw2s+pJW7FenOdNKcafxz1D/jKulaum0pI18eydCXhjJnXWQP4L7KfYyYPYLtJdsJGAE2FG1g6KyheANei6JNHi988wJLty8NP+xRVFnEte9ca3VYYgINNYpI3PAFfFQHqutdf2hd4TqG/WsYZdVl+II+/nzBn7lr0F0mR3l0+rftT/6UfPLL8kMJmCv9qK+RuzWXj7d8TOuM1tzY70YyUjJiEtuKnSsY+crI8JOPK3evxBvwcnWvqwFYU7AmoryBQbm3nB+Kf6Bbq24xiaGpWr93ffjrDqHe0c3Fmy2MSMyiHi8RiQt//vzPuP/PTcu/tGTgcwMp8hTVKjPmtTEUlBdQ4avAG/Byf+79fLnzSwuiPTpOu5OOzTseU9L13KrnGPXqKB5a+hBTl0zlzOfOpNJXGZO4pq+cHvHD3+Pz8OjyR8PHWRlZEcsUQGgPyuPTj49J/U1Z/7b9Ix58cNgc9M7uHbPrl1SVsHnfZvVOxiElXiJiufkb5vPwsofD87lWF6yuNeziD/r5ofiHWp81e20ss01ZNAWPz4OBQZW/ih0lO3hr3VsxufahG3sfeu701qdzRc8ryHBlkOpIJcOVwdRBU8nKyIpJ/U3ZNb2v4Zen/TL8dT2p5UnMHjc7Jtd+4ssnyP5rNn3+2Yf2j7Wv1XMp1tJQo4hYbtn2ZRE9L76gjxU7V0SUcdqdnOA+gb2eveFzNmx0adnFtDjNZhgGlf7I3q2AEaC0ujQm1588YDJvf/c2Hn/oa+92uZk6eGpEmZljZ/KLnr9g075N9M7uzQUnXRCTups6m83GzLEzeeiChyj3lnNSy5Nisr/i6vzV3LvkXqoD1VQHqqnwVTDq1VHsvGNnDKKWWFDiJSKW69i8I+nO9Igko659GedcPofRr43GYXPgC/q48vQrGd5luJmhmspms3Fhlwv55IdPwotj2m32mN3zwA4DWXTtIh5e9jBV/iomnTWJcT3G1YphdNfRMalPamub2TZm1/r2x2955utnCBrBiPMF5QVUeCtiNjdQoqPlJETEctX+as578TzW7V2HDRsGBh9f93Gdq2/vKd9DXkEe2c2y6dumrwXRmqu0upQb3ruBT374hJZpLZkxZkZSJ5tybGavmR3eUuvQXtLmqc0pvrtYa3vFmDbJFpGE5gv4WLxlMWXVZQw5cQjtj2tvdUgijaKkqoSbP7iZz7d/TttmbXnh0hfo17bfMV/PF/CR+XBmxBZbABmuUA/Xe1e+x7Auw6KKWWpT4iUikmSKPEV8X/Q9HY7rwInNTzSlzo1FG7lz0Z3kl+czuuto7jv3Phx2hyl1J5IlW5Yw4d0JFFUWMbD9QN68/E1aZ7QGQktD7CrdRWZqJi3SWtT6bM5LOazYuSKcKGWmZLJ+8nraZbY7plj2evbS4bEOEYlXhiuD/z37f5k0YFKdw/YSPS2gKiKSRD7a9BGd/t6Ji1+5mG7TuzHti2mNXmd+WT4Dnh/AvI3z+Gr3V/zli78wacGkRq830Wzat4mxr49lV9kuqvxVfLHjC0a/GpoHt6t0F92nd6fb9G5kP5rNlEVT+GmnQqWvkmXbl9XaAD53a+4xx3NC+glkubMinkg1DIMb+t2gpCsOKfESEYkzvoCPy968LLyqeZW/igdyH+DbH79t1Ho/2PAB1f7q8ORsj8/Di3kvotGISEu3LY1IcvxBP6vyV1Htr+bquVezpXgLlf5KvEEvz379LO+ufzdc1uVw1TnX6sCw4LGw2Wwsvm4xHZt3xGl34na5efnnLyf1E7+JTImXiEic+bHiRwJGIOKc0+Fk476NjVqvDVutpKCutb6aupbpLWt9nVx2FymOFPL25EW0XYWvglX5q8LHTruTnM45EZ/Nzshm5Ckjo4qpe6vubPvtNorvLqb8nvJaT6dK/FDiJSISZ1pntMblcEWc8wV8jb459bge43C73DhsoTldbpebyQMmx+xpuG0l27hnyT384dM/kF+WH5NrWmF019H0at2LDFcGTluoh+nxkY9js9lqzcXLcGVwcsuTw8df7/6a5TuWR5Tx+DykOlNjEluzlGZ6ejHOxVPraHJ9IvN4IC0N7MrlRWIhd2sul752KTabDW/Ay6MXPsrkAZMbvd4dJTv4f5/+P3aX7WZM1zExS7w+2/oZQ2cNxSD0/7zT7iTvljxOa31a1Ne2gi/g49W1r5Jfns+QE4cw5MQhAPxnz384/6XzMQwDf9DP4BMHM//q+eHFUV9c/SK3LryVCl9F+FoOm4PSe0ojthCS+KenGsUaO3bAxRfD+vXgcsEzz8CECVZHJZIUyqrL2FK8hXaZ7RJ+m55W01pRVBm5/2bPVj35dlLjzluzwr7KfXy16yuapzVnQPsB2G0HfyH9fNvnXPLKJRGJV8u0lhRNLVJPVYJR4iXW6NcP1q6FQM2cBrcbli6F/v2tjUskCfiDfp79+llWF6ymT3Yffn3Wr2OyrYwVXH9y4Q/6I86l2FOo+n1Vk0s4bv/wdp5b9RwpjhQCRoD3r3yfoScNtTosOUpKvMR8wWColyv4k+0p0tPhr3+F3/zGurhEkoBhGIx9fSwf//AxHp8Ht8vNeSeex4LxCxIyUTn5yZPZUrwl4pwdO8tuXMY5Hc+xKCrrrCtcR35ZPr2ye4XX/5LEonW8xHx2OzRvHnnO4YC2sdt7TKSp2ly8mSVbloQ3D/f4PHy+/XO+2/udxZEdm7cue6vWObfLTX554k6yj0bPrJ4M6zJMSVcTpMRLojNrVmh4sVmz0J8hQ2DsWKujEkl4Hp+n1orxDpuDSl9lPZ+Ib33b9qV9s8htoIIE6d9W0xKkaVHiJdEZMwby8uCpp+Ctt2D+fD3ZKE1SkaeIn7/xczo+3pGcl3LYtG9TVNfr3qo7We4snLbQnC6HzUHL9JaN+hSgL+Djh+IfKK0ujfm17TY7S65fQucWnXHYHGSmZPLmZW/SuUXnmNclEs/iaaKA5niJSEIyDIP+M/rz7Y/f4gv6sNvsnJB+Ahtv3UjztOZHvkA9dpft5sb3buS/P/6Xnlk9mTl2Jh2O6xDDyA/6rvA7LvjXBZRVl+EL+njogoe4c9CdjVKXx+ch3ZmekHPVRA7Q5HoREYvsKt3FKf84hSp/VfjccanH8dblbzHi5BEWRtZwJz95Mj8U/xBeZ8vtcvPJdZ8wsMNAiyNLLJW+SlKdqRFLSEhy0uR6ERGLpLvSw/sbHhA0gqQ70y2K6Oj4g/6IpAtCvXirC1ZbGFVi+bHiR86ccSaZD2eS/lA6T375pNUhSZxS4iUiEqXj049nfK/x4ZXH05xp9MzqmTDLJDjtTk5wnxBxzm6zR2x1I4f3yzm/ZM2eNQSMAN6Al3s+vofcrblWh2W6dYXruGfJPdyz5B42FG2wOpy4pKFGEZEYCBpBXlz9Ist3LKd7q+7cOvBW0pxpVofVYLlbcxnz2hgcNge+oI8rT7+S58c8r3lYDeR+yE2l/+ATpw6bgweHPsi9595rYVTmWrV7Fee/dH54CRS3y83yXy2nd3ZviyNrHJrjJSIiUSkoLyCvII/y6nIyUjI4vfXpdGze0eqwEkKnv3die8n28HGGK4Ppl0xnQt8J1gVlslGvjGLBpgXhYxs2xvUYx9wr5loYVeM51sQrMfeeEBGRmGvTrA2fbf2MJ798EpfDhS/o4+VxLzOuxzirQ4uJ4spi/rr8r+ws3clFp1zEVadfFbMevVk/m8XoV0eHr9c7uzfje42PybUTRUl1ScSxgUFJVUk9pZsu9XiJiAgAeQV5DJ45ODxUBJDuTKfkdyW4HC4LI4teubecXs/0YnfZbrwBL26XmynnTOHBoQ/GrI6t+7eybPsyWqS1YOQpIxN2X81jNWPVDO746I7wBuBul5tnRj3DdX2usziyxmFFj9fxwBtAJ2ArcAWwv45yW4FSIAD4gAFR1CkiIo1kS/GWWsmCgcFez17aZib2VmDvf/8+hRWFeANeILSW2F+W/YUHch6IWa9X5xadm/SCsDf1v4nS6lKeWPEENpuNuwbflbRJVzSiSbx+BywGpgF31xz/ro5yBpAD7IuiLhERaWS9WvfCF/BFnMtwZcTtfoKb9m1i9prZGBiM7zWebq261Vu2yl8VsVwGQMAIEDAC4d0BJDo2m407B93ZaAvvJotovtsuBc6veT0LyKXuxAvia0hTRETqcOoJp/LMqGeYOH8idpudVEcqC8cvrLVnZDz49sdvOfuFs6n0VWJg8Ni/H2PZjcvo26ZvneVHnDwCh+3gfaQ507jo5Iua3HCgWC+ahKgYaPmT6+z7yfFPbQFKCA01Pgs8V8/1NMdLRCQOVHgrKPQU0i6zHSmOFKvDqdMv3vwF73z3TkQvVnZGNi+OfZGLT724zs+sKVjDr+f/moLyAi7sciF/H/l30l2JscitxJ/GmuO1GGhTx/n7Djk2av7UZTCQD2TVXG89sLSugvfff3/4dU5ODjk5OUcIvsRjcwAAEbBJREFUT0REYi0jJYOMlAyrwzis/VX7aw0d7qnYwy/e/AUv//xlft7j57U+06dNH5b/arlZIUqSyc3NJTc3N+rrRNPjtZ7Q3K0CoC3wKdD9CJ/5I1AO/K2O99TjJSJN3to9a3n9v6+T4khhQt8JdGrRyeqQ4tKLq19k8sLJEU9gHtAnuw95E/PCx2v3rGXTvk30yOpB91ZH+jEl0jBWPNX4PnA98Jeav9+to4wbcABlQAYwAnggijpFRJLW8h3LuXD2hVT6KrHb7Dy24jFW3byKU44/xerQ4s6EvhPYV7WP33/y+4jNyQECwUD49Z8//zMPL30Yp8OJL+DjsRGPMfGsiWaHKxIWzV6NjwAXAhuAC2qOAdoB82tetyE0rJgHfAnMAxZFUaeISNKaungqHp8HA4OAEaC8upxHlj1y5A82QTabjSnnTGHJtUvCe2RCaO2o28++HQgtj/HQ0ofw+D2UVpdS6a/ktx/9ln2VesherBNNj9c+YHgd53cDo2pebwHqfsREREQilFWXRRwHCVJcVWxRNIlh8ImDee/K93jgsweo9lcz8cyJ3NjvRgB2lOwg1ZEa0SOW4kghvyyf49OPtypkaeL0HK2ISJy4qtdVbPp8U3jeksvuok1GG6r91aQ6Uy2OLn4N7zKc4V1q9wP0yOqBP+iPOGez2Tip5UlmhSZSSzRDjSIiEkNTB0/lznPu5LjU4wDwB/28tOYlBs0cFF5xXRqudUZr3rr8LTJcGaQ502iR1oL5V8+PGJoUMVs8LWyqpxpFRIDjHj6OMu/BYcdmKc2YMXoGV/W6ysKo6lZcWcw/Vv6D3WW7ueTUS7i026VWh1SLL+Bjr2cvWRlZWjBVYsaKpxpFRCTGDMOotUSCP+iPywnhZdVl9Hu2H/nl+XgDXmb/ZzYPDn2QKedMsTq0CC6HK+H3mpTkoaFGEZE4YrPZOPfEc3HZXQfPYSOnc451QdVjzro5FHoiN57+46d/tDgqkfimxEtEJM7M/eVccjrnkOZMo02zNrx1+Vuc1vo0q8OqxePzEDSCEeeqA9Vo2ohI/TTHS0Qa3brCdRRWFNI7uzct0+va0lUS0eZ9m+nzzz5U+CqA0MbTo04dxZwr5lgcmUjjO9Y5Xkq8RKTRGIbBzfNu5tW1r+KyuzAwWHTNIgZ2GGh1aBIjK3au4Nfzfk2hp5CLTrmI6RdP18bT0iQo8RKRuPPhpg+57M3Lwj0iAB2O68CO/91hYVQiItE71sRLc7xEpNFsLNpIwAhEnNtdtltzgESkyVLiJSKNpld2L+y2g//N2LDRpWWXA78piog0OUq8RKTR5HTO4c5z7iTVkUqzlGZkZWTx3pXvWR2WSFzzB/2UVpdaHYY0knj6tVNzvESSVGFFIUWVRZzU4iTtOShyGP/48h/cuehOggTpdkI3Fl27iHaZ7awOS+qgyfUiIiIJbNn2ZVz08kXhnQscNgdntjuTFf+zwuLIpC6aXC8iIk3KvA3zGP6v4Yx8eSSf/vCp1eFEbcXOFfgCvvBxwAjwTf43FkYkjUF7NYqISMJ5b/17XDX3Kir9lQAs3baUBeMXcH7n8y2O7Ni1z2xPiiMFX/Bg8pWVkWVhRNIY1OMlIiIJZ9ryaeGkC8Dj9/D4isctjCh6V5x2Bed0OIdmKc3ITMkkw5XB7HGzrQ5LYkw9XiIiInHAYXfw0bUfsWTLEoo8RQzqOIhOLTpZHZbEmCbXi4hIwjl0qNHtdCf8UKMkFj3VKCIiTcoH33/AE18+gdPu5O7BdzP0pKFWhyRNiBIvERGRJGIYBo8uf5Snv3oap93J78/7PRP6TrA6LKmh5SRERJLA7P/M5vSnT6fnUz154ZsXrA5HLDR95XQe+OwBtpVsY3PxZiYtmMR767XzQ6LT5HoRkTgxd91cJs6bGF5A87YPbyPVmco1va+xOLL6lVSVsLtsN51adMLtclsdTsIzDIPFWxbz/d7vefLLJ8PfCwAen4dZa2YxtvtYCyOUaCnxEhExSbW/mruX3M1Hmz+ibbO2TL9kOj2zeobfn7FqRq0ftDNWzYjbxGvm6plMWjAJl92FDRvzrp7HuZ3OtTqshHbrwlt5Ke8lAkYAb8Ab8Z4NG5mpmRZFJrGioUYREZNc/+71zFg1g/V715O7NZdzXjiH3WW7w++nu9Jrfaauc/Fg075NTF4wmSp/FWXeMkq9pYx5bUzEyutydDYWbWTm6plU+Cqo8lcRNILh92zYyEjJ4HeDf2dhhBILSrxEREwQCAaYs25OePkDAwN/0M+Hmz4Ml7nv3PsihuvcLjd/OO8PpsfaEN8VfofL4Yo45wv4KCgvsCiixFfoKaz1NXU73YzvNZ4pg6bwzc3f0COrh0XRSaxoqFFExAR2mx2HzUHACITP2bCR4kgJH5/V/iyW3bCMZ75+hqAR5JYzbuGs9mdZEe4RnXz8ybV7t2zQOqO1NQElgdOyTsN+SH+I2+XmuTHPxW3Ppxw99XiJiJjAZrPx27N/G+7RctldtEhrwZiuYyLK9WvbjxljZvD8pc/HbdIF0DOrJ/eddx/pznSapzbH7XLz6s9fJdWZanVoCat5WnMWX7eYjsd1xIaNLi268Mn1nyjpSjJax0tExCSGYfBi3oss3LiQDs07cN+599HK3crqsKKyad8mtpdsp0erHrTNbGt1OEnDMIwD60RJnNICqiIiIiIm0QKqIiIiInFOiZeIiIiISZR4iYiIiJhEiZeIiIiISZR4iYiIiJhEiZeIiIiISZR4iYiIiJhEiZeIiIiISbRXo4iIRM0b8PLEiif4puAb+rfpz+1n3x6xD6WIhGjlehERiYphGFz08kUs276MSn8l6c50BncczKJrF2nbG0laWrleREQssX7ver7Y8QWV/koAKv2VLN+5nO+Lvrc4MpH4o8RLRESiUh2oxmFzRJyz2+xU+6stikgkfinxEhGRqPTM6kl2s2xcdhcALruLNs3a0COrh8WRicSfeBp81xwvEZEEtad8D7fMu4W1P66lV+tePDv6WbKbZVsdlkijOdY5Xkq8RERERI6SJteLiIiIxDklXiIiIiImUeIlIiIiYhIlXiIiIiImUeIlIiIiYhIlXiIiIiImUeIlIiIiYhIlXiIiIiImUeIlIiIiYhIlXiIiIiImUeIlIiIiYhKn1QGIiIjEky3FW1i8eTEZKRmM6z6OjJQMq0OSJKJNskVERGos37GcEbNHEDSC2G122mW2Y9XNq8hMzbQ6NIkz2iRbREQkSrd8cAsVvgoq/ZVU+CrYXrKdp7962uqwJIko8RIREalR6CmMOK4OVLO7fLdF0UgyUuIlIiJSY1iXYaQ6UsPHbpebEV1GWBiRJBslXiIiIjWeHf0sw7sMx2FzkOZM48GhDzKq6yirw5Ikosn1IiIihwgaQWzYDkygFqnlWCfXazkJERGRQ9htGhCSxqHvLBERERGTRJN4XQ58CwSA/ocpNxJYD2wE7o6iPhEREZGEFk3itRYYB3x+mDIOYDqh5KsncBXQI4o6RURERBJWNHO81jegzABgE7C15vh1YCzwXRT1ioiIiCSkxp7j1R7Y8ZPjnTXnRERERJqcI/V4LQba1HH+XuCDBlz/qNaHuP/++8Ovc3JyyMnJOZqPi4iIiDSK3NxccnNzo75OLBYo+RSYAnxTx3tnA/cTmuMFcA8QBP5SR1mt4yUiIiIJwepNsuur+GvgVKAzkAL8Eng/RnWKiIiIJJRoEq9xhOZvnQ3MBxbWnG9XcwzgByYDHwHrgDfQxHoRERFpouJpLwQNNYqIiEhCsHqoUURERESOQImXiIiIiEmUeImIiIiYRImXiIiIiEmUeImIiIiYRImXiIiIiEmUeImIiIiYRImXiIiIiEmUeImIiIiYRImXiIiIiEmUeImIiIiYRImXiIiIiEmUeImIiIiYRImXiIiIiEmUeImIiIiYRImXiIiIiEmUeImIiIiYRImXiIiIiEmUeImIiIiYRImXiIiIiEmUeImIiIiYRImXiIiIiEmUeImIiIiYRImXiIiIiEmUeImIiIiYRImXiIiIiEmUeImIiIiYRImXiIiIiEmUeImIiIiYRImXiIiIiEmUeImIiIiYRImXiIiIiEmUeImIiIiYRImXiIiIiEmUeImIiIiYRImXiIiIiEmUeImIiIiYRImXiIiIiEmUeImIiIiYRImXiIiIiEmUeImIiIiYRImXiIiIiEmUeImIiIiYRImXiIiIiEmUeImIiIiYRImXiIiIiEmUeImIiIiYRImXiIiIiEmUeImIiIiYRImXiIiIiEmUeImIiIiYRImXiIiIiEmUeImIiIiYRImXiIiIiEmUeImIiIiYRImXiIiIiEmUeImIiIiYRImXiIiIiEmUeImIiIiYRImXiIiIiEmUeImIiIiYRImXiIiIiEmUeImIiIiYRImXiIiIiEmUeImIiIiYRImXiIiIiEmiSbwuB74FAkD/w5TbCvwHWA2sjKK+pJSbm2t1CJbQfTctuu+mRffdtDTV+z5W0SRea4FxwOdHKGcAOUA/YEAU9SWlpvoNq/tuWnTfTYvuu2lpqvd9rJxRfHb9UZS1RVGPiIiISFIwY46XASwBvgZuMqE+ERERkbh0pJ6oxUCbOs7fC3xQ8/pTYArwTT3XaAvkA1k117sVWFpHuU3AyUeIR0RERCQebAZOOdoPHWmo8cJjiyVCfs3fhcA7hOZ51ZV4HXXwIiIiIokkVkON9fWcuYHMmtcZwAhCk/JFRERE5CiMA3YAlUABsLDmfDtgfs3rLkBezZ//AveYHKOIiIiIiIiIiIg5HgW+A9YAbwPN6yk3ktCyFRuBu80JrdE1xYVnG3rPydjexxN6qGQDsAhoUU+5rSR+ezek/Z6seX8NobX9ksGR7jsHKCHUtquB35sWWeOaCezh8NNHkrG9j3TfOSRfe3ck9CDdt4RGr26rp1wytndD7j2HBGjzCzk4v+yRmj+HchB60rEz4CI0XNnDjOAaWXegK6GGPFwS8gOhH9rJoCH3nKztPQ2YWvP6bur+XofEb++GtN8lwIKa1wOBFWYF14gact85wPumRmWOcwn9cK0vAUnG9oYj33cOydfebYC+Na+bAd/TNP59Q8PuPYejaHOr9mpcDARrXn8JdKijzABC/6FtBXzA68BYM4JrZOsJ9X40RLIsPNuQe07W9r4UmFXzehbws8OUTeT2bkj7/fRr8SWh3r9sk+JrLA39vk3ktq3PUqD4MO8nY3vDke8bkq+9Cwj9UgFQTmjEqt0hZZK1vRty73AUbR4Pm2TfyMEs+afaE5q8f8DOmnNNRVNbeDZZ2zub0LAENX/X9x9Rord3Q9qvrjJ1/dKVSBpy3wYwiNDwywKgpzmhWS4Z27shkr29OxPq8fvykPNNob07U/e9H1WbR7Nl0JE0ZPHV+wAv8God5YxGissMDbn3IxlM5MKz66l7/bN4Ee09J2N733fIsUH995lo7X2ohrbfob8VJnK7Q8Pi/4bQPBEPcDHwLqGh96Yg2dq7IZK5vZsBc4DbCfX+HCqZ2/tw935Ubd6YideRFl+dQGhMeFg97+8idCMHdCSUQScCMxeejRfR3nOytvceQklZAaFdHH6sp1yitfehGtJ+h5bpUHMukTXkvst+8noh8DSh+Xz7Gjc0yyVjezdEsra3C5gLvEwosThUMrf3ke49Idp8JKEnBFodpoyT0HL8nYEUkmey9QGfAmfU896hC89+QWjx2UR3uHtO1vaexsEn3X5H3ZPrk6G9G9J+P518ezbJMfm2IfedzcGegAGE5oMli840bHJ9srT3AZ2p/76Tsb1twL+Axw9TJlnbuyH3nhBtvhHYxsFHL5+uOf/TxVch1GX3PaHJq8my+GpTXHi2IfcMydnexxOau3XochLJ2N51td8tNX8OmF7z/hoO/1RvIjnSfU8i1K55wHJCP5SSwWvAbkLTRXYQmq/bFNr7SPedjO09hNADcXkc/Ll9MU2jvRty78nY5iIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiJx5f8DQ6yWnjMkMiIAAAAASUVORK5CYII=\n",
48 | "text/plain": [
49 | ""
50 | ]
51 | },
52 | "metadata": {},
53 | "output_type": "display_data"
54 | }
55 | ],
56 | "source": [
57 | "# Generate a dataset and plot it\n",
58 | "np.random.seed(0)\n",
59 | "X, y = sklearn.datasets.make_moons(200, noise=0.20)\n",
60 | "color0 = ['red' if l == 0 else 'green' for l in y]\n",
61 | "plt.scatter(X[:,0], X[:,1], color=color0)"
62 | ]
63 | },
64 | {
65 | "cell_type": "code",
66 | "execution_count": 3,
67 | "metadata": {},
68 | "outputs": [],
69 | "source": [
70 | "# ipdb.set_trace()\n",
71 | "num_examples = len(X) # training set size\n",
72 | "nn_input_dim = 2 # input layer dimensionality\n",
73 | "nn_output_dim = 2 # output layer dimensionality\n",
74 | " \n",
75 | "# Gradient descent parameters (I picked these by hand)\n",
76 | "epsilon = 0.01 # learning rate for gradient descent\n",
77 | "reg_lambda = 0.01 # regularization strength"
78 | ]
79 | },
80 | {
81 | "cell_type": "code",
82 | "execution_count": 4,
83 | "metadata": {},
84 | "outputs": [],
85 | "source": [
86 | "# Helper function to evaluate the total loss on the dataset\n",
87 | "def calculate_loss(model):\n",
88 | " W1, b1, W2, b2 = model['W1'], model['b1'], model['W2'], model['b2']\n",
89 | " # Forward propagation to calculate our predictions\n",
90 | " z1 = X.dot(W1) + b1\n",
91 | " a1 = np.tanh(z1)\n",
92 | " z2 = a1.dot(W2) + b2\n",
93 | " exp_scores = np.exp(z2)\n",
94 | " probs = exp_scores / np.sum(exp_scores, axis=1, keepdims=True)\n",
95 | "\n",
96 | " probs_new = probs[range(num_examples), y]\n",
97 | "\n",
98 | " corect_logprobs = -np.log(probs[range(num_examples), y])\n",
99 | "\n",
100 | " data_loss = np.sum(corect_logprobs)\n",
101 | "\n",
102 | " data_loss += reg_lambda/2 * (np.sum(np.square(W1)) + np.sum(np.square(W2)))\n",
103 | " return 1./num_examples * data_loss"
104 | ]
105 | },
106 | {
107 | "cell_type": "code",
108 | "execution_count": 5,
109 | "metadata": {},
110 | "outputs": [],
111 | "source": [
112 | "# Helper function to predict an output (0 or 1)\n",
113 | "def predict(model, x):\n",
114 | " model_rel = {}\n",
115 | " W1, b1, W2, b2 = model['W1'], model['b1'], model['W2'], model['b2']\n",
116 | " # Forward propagation\n",
117 | " z1 = x.dot(W1) + b1\n",
118 | " a1 = np.tanh(z1)\n",
119 | " z2 = a1.dot(W2) + b2\n",
120 | " exp_scores = np.exp(z2)\n",
121 | " probs = exp_scores / np.sum(exp_scores, axis=1, keepdims=True)\n",
122 | " model_rel = {'W1': W1, 'b1': b1, 'W2': W2, 'b2': b2, 'z1':z1, 'a1': a1, 'z2': z2}\n",
123 | " return probs, model_rel"
124 | ]
125 | },
126 | {
127 | "cell_type": "code",
128 | "execution_count": 6,
129 | "metadata": {},
130 | "outputs": [],
131 | "source": [
132 | "# This function learns parameters for the neural network and returns the model.\n",
133 | "# - nn_hdim: Number of nodes in the hidden layer\n",
134 | "# - num_passes: Number of passes through the training data for gradient descent\n",
135 | "# - print_loss: If True, print the loss every 1000 iterations\n",
136 | "def build_model(nn_hdim, num_passes=20000, print_loss=False):\n",
137 | " \n",
138 | " # Initialize the parameters to random values. We need to learn these.\n",
139 | " np.random.seed(0)\n",
140 | " W1 = np.random.randn(nn_input_dim, nn_hdim) / np.sqrt(nn_input_dim)\n",
141 | " b1 = np.zeros((1, nn_hdim))\n",
142 | " W2 = np.random.randn(nn_hdim, nn_output_dim) / np.sqrt(nn_hdim)\n",
143 | " b2 = np.zeros((1, nn_output_dim))\n",
144 | " \n",
145 | " # This is what we return at the end\n",
146 | " model = {}\n",
147 | " \n",
148 | " # Gradient descent. For each batch...\n",
149 | " for i in range(0, num_passes):\n",
150 | " \n",
151 | " # Forward propagation\n",
152 | " z1 = X.dot(W1) + b1\n",
153 | " a1 = np.tanh(z1)\n",
154 | " z2 = a1.dot(W2) + b2\n",
155 | " exp_scores = np.exp(z2)\n",
156 | " probs = exp_scores / np.sum(exp_scores, axis=1, keepdims=True)\n",
157 | " \n",
158 | " # Backpropagation\n",
159 | " delta3 = probs\n",
160 | " delta3[range(num_examples), y] -= 1\n",
161 | " \n",
162 | " dW2 = (a1.T).dot(delta3)\n",
163 | " db2 = np.sum(delta3, axis=0, keepdims=True)\n",
164 | " delta2 = delta3.dot(W2.T) * (1 - np.power(a1, 2))\n",
165 | " dW1 = np.dot(X.T, delta2)\n",
166 | " db1 = np.sum(delta2, axis=0)\n",
167 | " \n",
168 | " # Add regularization terms (b1 and b2 don't have regularization terms)\n",
169 | " dW2 += reg_lambda * W2\n",
170 | " dW1 += reg_lambda * W1\n",
171 | " \n",
172 | " # Gradient descent parameter update\n",
173 | " W1 += -epsilon * dW1\n",
174 | " b1 += -epsilon * db1\n",
175 | " W2 += -epsilon * dW2\n",
176 | " b2 += -epsilon * db2\n",
177 | " \n",
178 | " # Assign new parameters to the model\n",
179 | " model = { 'W1': W1, 'b1': b1, 'W2': W2, 'b2': b2}\n",
180 | " \n",
181 | " # Optionally print the loss.\n",
182 | " # This is expensive because it uses the whole dataset, so we don't want to do it too often.\n",
183 | " if print_loss and i % 1000 == 0:\n",
184 | " print (\"Loss after iteration %i: %f\" %(i, calculate_loss(model)))\n",
185 | " \n",
186 | " return model"
187 | ]
188 | },
189 | {
190 | "cell_type": "code",
191 | "execution_count": 7,
192 | "metadata": {},
193 | "outputs": [
194 | {
195 | "name": "stdout",
196 | "output_type": "stream",
197 | "text": [
198 | "Loss after iteration 0: 0.432387\n",
199 | "Loss after iteration 1000: 0.068947\n",
200 | "Loss after iteration 2000: 0.068926\n",
201 | "Loss after iteration 3000: 0.071218\n",
202 | "Loss after iteration 4000: 0.071253\n",
203 | "Loss after iteration 5000: 0.071278\n",
204 | "Loss after iteration 6000: 0.071293\n",
205 | "Loss after iteration 7000: 0.071303\n",
206 | "Loss after iteration 8000: 0.071308\n",
207 | "Loss after iteration 9000: 0.071312\n",
208 | "Loss after iteration 10000: 0.071314\n",
209 | "Loss after iteration 11000: 0.071315\n",
210 | "Loss after iteration 12000: 0.071315\n",
211 | "Loss after iteration 13000: 0.071316\n",
212 | "Loss after iteration 14000: 0.071316\n",
213 | "Loss after iteration 15000: 0.071316\n",
214 | "Loss after iteration 16000: 0.071316\n",
215 | "Loss after iteration 17000: 0.071316\n",
216 | "Loss after iteration 18000: 0.071316\n",
217 | "Loss after iteration 19000: 0.071316\n"
218 | ]
219 | }
220 | ],
221 | "source": [
222 | "# Build a model with a 3-dimensional hidden layer\n",
223 | "model = build_model(3, print_loss=True)\n",
224 | "\n",
225 | "prediction, model_rel = predict(model, X)"
226 | ]
227 | },
228 | {
229 | "cell_type": "code",
230 | "execution_count": 8,
231 | "metadata": {},
232 | "outputs": [],
233 | "source": [
234 | "def backward_simpleLRP_rel(top_rel, inputs, weights, outputs, epsilon=1e-4):\n",
235 | " return np.sum((inputs.T.dot(top_rel) * weights) / (outputs + epsilon),\n",
236 | " axis=1,\n",
237 | " keepdims=True).T"
238 | ]
239 | },
240 | {
241 | "cell_type": "code",
242 | "execution_count": 9,
243 | "metadata": {},
244 | "outputs": [],
245 | "source": [
246 | "def backprop_taylor_rel(inputs, weights, top_rel, lowest=-1.5, highest=2.5):\n",
247 | " w_p = np.maximum(np.zeros_like(weights), weights)\n",
248 | " w_n = np.minimum(np.zeros_like(weights), weights)\n",
249 | " \n",
250 | " L = np.ones_like(inputs) * lowest\n",
251 | " H = np.ones_like(inputs) * highest\n",
252 | " \n",
253 | " z_o = inputs.dot(weights)\n",
254 | " z_p = L.dot(w_p)\n",
255 | " z_n = H.dot(w_n)\n",
256 | " \n",
257 | " z = z_o - z_p - z_n + 1e-10\n",
258 | " s = top_rel / z\n",
259 | " \n",
260 | " c_o = s.dot(weights.T)\n",
261 | " c_p = s.dot(w_p.T)\n",
262 | " c_n = s.dot(w_n.T)\n",
263 | " \n",
264 | " return inputs * c_o - L * c_p + H * c_n\n",
265 | " "
266 | ]
267 | },
268 | {
269 | "cell_type": "code",
270 | "execution_count": 10,
271 | "metadata": {},
272 | "outputs": [],
273 | "source": [
274 | "output, model_rel = predict(model, X)\n",
275 | "output_ = np.round(np.argmax(output, axis=1))"
276 | ]
277 | },
278 | {
279 | "cell_type": "code",
280 | "execution_count": 11,
281 | "metadata": {},
282 | "outputs": [],
283 | "source": [
284 | "maps1_x = []\n",
285 | "maps1_y = []\n",
286 | "for i in range(200):\n",
287 | " output, model_rel = predict(model, X[i,:])\n",
288 | " y_rel = np.round(np.argmax(output, axis=1)) \n",
289 | " temp = np.asarray([1.]).reshape((1,1))\n",
290 | " x = np.expand_dims(X[i,:], axis=0)\n",
291 | " temp1 = backprop_taylor_rel(model_rel['a1'], model_rel['W2'], temp)\n",
292 | " temp1 = backprop_taylor_rel(x, model_rel['W1'], temp1)\n",
293 | " maps1_x.append(temp1)\n",
294 | " maps1_y.append(y_rel)"
295 | ]
296 | },
297 | {
298 | "cell_type": "code",
299 | "execution_count": 12,
300 | "metadata": {},
301 | "outputs": [],
302 | "source": [
303 | "X_rel1 = np.squeeze(maps1_x, axis=1)\n",
304 | "y_rel1 = np.squeeze(maps1_y, axis=1)"
305 | ]
306 | },
307 | {
308 | "cell_type": "code",
309 | "execution_count": 13,
310 | "metadata": {},
311 | "outputs": [
312 | {
313 | "data": {
314 | "text/plain": [
315 | ""
316 | ]
317 | },
318 | "execution_count": 13,
319 | "metadata": {},
320 | "output_type": "execute_result"
321 | },
322 | {
323 | "data": {
324 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAl4AAAHfCAYAAACBJdZmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xd8VFX6x/HP1CSTCqEjRanSFUSqRlQEK/ZeENkVe3dXd13Wn+vqWnaxYEcsgF1BERVUQLqhSRFpSg8EAqRM2pTfH4eUIQmEZDKTZL7v1ysvcu/cufcZdwlPznnOc0BERERERERERERERERERERERERERERERERERERERERERCTAMGAdsAF4uJzXrwVWAr8A84EeoQtNREREpP6wARuBtoADWAGceNg1/YHEQ98PAxaFKjgRERGR+qQ/8E2p478c+qpIA2B7jUYkIiIiUktZq/n+lsC2UsfbD52ryCjg62o+U0RERKROslfz/f5juPYM4GZgYHkvtmvXzr9p06ZqhiMiIiISEpuA9sf6puqOeO0AWpU6bkX5U4k9gDeAC4H95d1o06ZN+P3+iPv6xz/+EfYY9Ln1ufW59bn1ufW59bmP7QtoV5XEqbqJVyrQAVNc7wSuBKYddk1r4DPgOkwhvoiIiEhEqu5Uowe4A/gWs8LxLeBX4M+HXn8NeAxTVP/KoXOFQN9qPldERESkzqlu4gUw49BXaa+V+v6WQ19SjpSUlHCHEBb63JFFnzuy6HNHlkj93FVlCXcApfgPzZmKiIiI1GoWiwWqkEdVt8ZLRERERCpJiZeIiIhIiCjxEhEREQkRJV4iIiIiIaLES0RERCRElHiJiIiIhIgSLxEREZEQUeIlIiIiEiJKvERERERCRImXiIiISIgo8RIREREJESVeIiIiIiGixEtEREQkRJR4iYiIiISIEi8RERGREFHiJSIiIhIiSrxEREREQkSJl4iIiEiIKPESERERCRElXiIiIiIhosRLREREJESUeImIiIiEiBIvERERkRBR4iUiIiISIkq8REREREJEiZeIiIhIiCjxEhEREQkRJV4iIiIiIaLES0RERCRElHiJiIiIhIgSLxEREZEQUeIlIiIiEiJKvERERERCRImXiIiISIgo8RIREREJESVeIiIiIiGixEtEREQkRJR4iYiIiISIEi8RERGREFHiJSIiIhIiSrxEREREQkSJl4iIiEiIKPESERERCRElXiIiIiIhosRLREREJESUeImIiIiEiBIvERERkRBR4iUiIiISIkq8REREREJEiZeIiIhIiCjxEhEREQkRJV4iIiIiIaLES0RERCRElHiJiIiIhIgSLxEREZEQUeIlIiIiEiJKvERERERCRImXiIiISIgo8RIREREJESVeIiIiIiGixEtEREQkRJR4iYiIiISIEi8RERGREFHiJSIiIhIiSrxEREREQiQYidcwYB2wAXi4nNc7AwuBPOD+IDxPREREpE6yV/P9NuAl4CxgB/AzMA34tdQ1+4A7gRHVfJaIiIhInVbdEa++wEbgD6AQ+AC46LBr0oHUQ6+LiIiIRKzqJl4tgW2ljrcfOiciIiIih6nuVKM/KFEcMnbs2OLvU1JSSElJCebtRURERKpk9uzZzJ49u9r3sVTz/f2AsZgCe4C/Aj7g6XKu/QeQDTxXwb38fn9Q8zgRERGRGmGxWKAKeVR1pxpTgQ5AW8AJXIkpri9PdZM8ERERkTotGMnQcOB/mBWObwH/Bv586LXXgGaY1Y4JmNGwLKALZvSrNI14iYiISJ1Q1RGv2jQKpcRLRERE6oRwTTVKkO1172XY+8NIeiqJTi91YtH2ReEOSURERIJEI161TN83+rIibQWFPtP2LM4Zx6+3/8pxCccF7RkLty1kY8ZGujXpxknNTwrafUVERCKFRrzqgeyCbJanLS9OugAsWPhpy09Be8ZDMx/irPfO4ravb2PQ24N4fuHzQbu3iIiIHJkSr1okyhZV5pwfP4nRiUG5//p963lpyUu4C91kF2TjLnTzyPePsM+9Lyj3FxERkSNT4lWLOGwOxqaMJdYRC4DL4aJL4y4MbTc0KPffmbUTp80ZcM5pc7InZ09Q7i8iIiJHVt3O9RJkjw5+lJOancT8rfNpldiKkb1GYrcG53+mro274vV7A845bA7aJrUNyv1FRETkyFRcH2F++P0HLvnwEnIKc2gY05Dp10ynT4s+4Q6rzsspyOH+7+5n/tb5tE9uz4vDXwzqgggREald1MdLKs3v95NdkE2cM67o/zhSDX6/n5R3UliyfQl53jxsFhtN45qy7vZ1xEfFhzs8ERGpAVrVKJVmsViIj4qPuKTL6/MyYfkEHpz5IO//8j7BSvTT3eks3r6YPG+eeY7fS3ZBNgu2LQjK/UVEpP5QjZdEBL/fzyUfXcKszbNwF7qJdcQyc/NM3hnxTrXvbbPY8BOYxPn9/qDV5omISP2hES+p1/x+P28te4tT3zyV6eun4y50A5BTmMNHaz5i68Gt1X5GsiuZizpdhMvuAkxbkOMSjmNQ60HVvvfhsvKzuGDKBUQ9EUWDpxswccXEoD9DRERqTm2aa1KNlwTdMwueYezsscUJV2lxzjgW37KYLo27VPs5Hp+H5xY+x7yt8+iU3InHTn+MhKiEat/3cJd9dBlfrf+KfG8+YFqOfHPtNwxuMzjozxIRkYqpuF6kHM2fbU5aTlqZ8zaLjZYJLdlw54Yyvc1qs8SnEsnMzyw+tmDhsdMfY2zK2PAFJSISgVRcL1KOw2uvAKLt0QxoNYCfRv5Up5IugKTopIDjaHs0jVyNwhSNiIgcKyVeUq/dferduByu4uNYRyy/3PoLc0fOpXVi6zBGVjWvnf8aLoeLKFsUsY7Y4ia7IiJSN2iqUeo1v9/Pq0tfZdIvk2gQ04AnzniCns16hjusalm9ZzWzNs8iMSqRK7tdGZBYiohIaKjGS0RERCREVOMlIiIiUssp8RIREREJESVeIlJvrNmzhus/u56LPriIL9Z9Ee5wRETKUI2XiNQLy3YuY+DbA8nzmD0zXQ4X488bz409bwxzZCJSH6m4XkQiVnpOOu1eaEdWQVbA+fYN27Phzg1hiqrmeH1erBZrxG10L1KbqLheRCLWP+f8k5yCnDLnvT5vude/tOQlmjzThJgnYhj41kC2Z24v9zqvz8vnv37Oa6mvsWr3qqDGDLA7ezdvL3+bd1e+y/7c/Ue9Pj0nnQFvDcD5hJOEpxJ4d+W7QY9JRGpWbfp1SSNeIlIl508+n+kbpgecs2DhmaHPcH//+wPOP/DdAzy38LmAcyckncD6O9djs9qKz3l9Xs567yxSd6Ti9XuxWCxMvGgil3e9PCgxb8zYSN83+lLgLQDM3qHL/ryMFvEtKnzPaW+fxqLtiyj0FQIQY49hzk1zOKXlKUGJSUQqTyNeIhKxzjz+zIBGshYsnNH2DO7rd1/AddkF2fxv0f/KvP+PA39w37f3kZ6TXnxu6m9Tmb91PtmF2eR6cnEXurl52s38c/Y/ueyjy/jrrL+ybu+6Ksd837f3cTD/IDmFOeQU5rDPvY/HfnzsiO8pnXQBeP1e5m6ZW+UYRCT07OEOQESkqt5c9ib3fHMPuYW5NItrRr4nH4ALO13IlEunlKmBysrPwmqx4vUHTkH68DE+dTwfrvmQNbetIdmVzJe/fRmQ5IBJ3MbOGVt8/OzCZ+ncqDM5BTl0b9Kd1y94naZxTSsV+47MHfj8vuJjj9/DjswdR3xPUnQS6e6S5NBhdRQ/74fff+DWr25lf+5+hrYbyusXvE6sM7ZSsYhI6GjEC9ieuZ3F2xdXqsZCRGqHuVvmcvc3d5NTmIMPH+nudOKd8bRKaEXDmIZlkiuAtOy0cs8DeHweDuQd4L1f3gNgV/auo8bg8XlYvWc1vx/4nRkbZzD47cEUeguP+j6AYe2HBYzSuRwuhncYfsT3TLhoAi6Hixh7DHHOOLo16caVXa/k1/RfOX/y+WzI2MDe3L18vPZjrvvsugrvk+/J5+cdP/PL7l8Ckr+M3Ay2HtwacE5EgiviR7yenvc0Y+eMxWlz4vP7mHbVNM44/oxwhyUiRzHnjznkFeYVHxf6CjmQf4AD+QeYtGoSadlpfHXNVwHvueqTq46YVHh8nuIi/W5NujFz00x8VC4JKfQVsit7F7/t+41uTbod9fqxKWPZnrWdyasmA3DLybdwR987ylzn8/vYlLEJm9XGeR3O4+fRPzN3y1ySY5IZ0XkEDpuDL9Z9Qa4nNyCWL9d/We5zd2XtYuCEgex178Xn93FKy1P45tpveGjmQ7yS+goOq4PjEo/jxxt/PGK9mYhUTUQX1/+y+xf6v9kft8ddfC4hKoGMhzICimxFpPZ5LfU17vv2voC/v6XZLDbcj7px2pzF56KfiCbfmx9wnd1qx+PzAOCyu1h4y0J6NO3BXvdeer3ai3R3enEB/NHE2GNYeetKOiR3qPTn8PpM4b7VUnYCIjM/kzPfPZO16Wvx+/30O64fX1/7NdH2aMAkiuN/Hs+Li19k4/6NZd7ve8xXZrr1wikXMmPjjOLPHGOP4eLOFzP1t6nkFJqk026xM6jNIH688cdKfw6RSKPi+ir4be9v2G2Bg375nnz25e4LU0QiUlk39LyB9sntiXXEBiRXRSwWCzZL4C9QnRt1DkhwXHYXw9oNo2V8S7o07sK0q6fRo2kPABq5GrHmtjW8NPwljos/DpfDhcPqwHLo56zNYsNutRNliyq+V0rbFNo3bH9Mn8NmtZWbdAE8+N2DrNq9Cnehm1xPLgu2LWDEByOYuGIimXmZXDjlQv76/V/LTbosFfx7sCZ9TXHSBZDryWXa+mnFSReYerMVaSuO6XOISOVE9IjXqt2r6PdWP9yFJb8xJ0Ylsu+hfRrxEqkD8j35fPbrZ+zJ2cNzC59jd85uCrwFxNhjGNF5BPf1v4+Tm59cnNhszNhIysQUDuYfpNBbyO19b+e5oc8d5SmQ58nj4zUfk5GbwRnHn0GXxl3Y595HQlQCLy15iRVpK+jdojd3nXoXdmvwKjh6v9abZWnLAs5ZsOByuIh1xLI/b3+ZBQAATquTS068hCmXTSnz2iUfXsJX678qfp/VYsXv9+Mn8Odvz6Y9WXHr0ZOv3/b+xvK05bRKaMWAVgPU1FUihjrXV9EzC57hsR8fw2lz4vf7+fLqLzm97elBfcbv+3/n8o8vZ236WlontuaDyz6gV7NeQX2GSKQ7mHeQZxc8y6o9q/jh9x+wWCx4fV76H9efr6/9GofNAUCht5DN+zfTIKYBTWKbhDnqI7vh8xv4YPUH5SZXFYmyRTGi8wjeGfEOUfaoMq+n56Qz+O3B7MjagdfnDagNK+Kyu1g8evFRa9UmrZrE6GmjsVvt+Pw+ru5+NW9c8EalYxWpy5R4VcOurF3szNpJ+4btSYxODOq9C72FtHuhHTuySpaOJ0Un8fvdv5MUnRTUZ4nUVT/8/gNTf5tKw+iG3HbKbTSObVzlew14awCLdywu/vvmcrh45uxnuO2U24IVbsjsc+9jwIQB7MraRXZBdplRqYrEOmJZMnoJXRp3Kfd1j8/Db3t/w2F10OPVHgF1b9H2aF4c9iK39L7liM8o9BaS8FRC8d6YYNpbfH/D9wxuM7hScYrUZarxqobm8c3p3aJ30JMuMI0ZM3IzAlZS+f3+kNZPbMzYyLyt88jIzQjZM0Uq692V73LBlAt4YfEL/Ounf9Hj1R7sc1e9znLT/k0Bf9/chW7Wpq8NRqghl+xKZtWYVcy8fibndzy/uJ7saNyF7uLVkuWxW+10bdKVjo068tRZTxW3tYi2R9M2qS3X9rj2qM84kHegzArRQl8hD816iNun307TZ5tywrgT+PzXzysVs0ikUOJVw5Kik8pMExT6CkM22vXQzIfo8UoPzp98Pm3/15Z5W+eF5LkilfWXWX8prrMs9BWyP3c/76x8p8r369G0B3ZLSZ1VrCOWU1rU3S11nDYnpx53Kh9e9iHndzy/zIKB8lgt1krXmt3T7x4+veJTHhjwAE8OeZLU0anEOGKO+r5GrkYBfciKLNq+iNdSX2NPzh5+P/A71352LQu2LahULCKRQIlXDWsc25g7+t5BrCMWq8VKrCOW4e2H07Npzxp/9vyt8xn/83hyPbkczD9IVkEWIz4YQSTtienxeVi1exVr09eqKWQtdXiNUaGvkOyC7Crf790R73JCwxNwOVxE2aK4vMvl3NDzhuqGGXYxjhg+ueITCv9eyKoxq2gR3wKnzYnL4WLUSaOKk6Ci4vsbe95Y6XsPaz+MZ85+hnv731vpbvcWi6XChNZLSZPaXE8uLyx+IaJ+7ogciWq8QsDv9zN9w3RWpq2kfcP2XN718gqXjwfT28vf5s4ZdwYsE7darGT/NbtSv9HWdQfyDnDa26exef9mAHo168XM62dGxGevS0ZPG82kVZOKEzCXw8W8kfM4qflJVb6n1+fljwN/EOuMpVlcs2CFWqv4/X725+0nMSoRm9XG5FWTmbxqMknRSfz9tL/TqVGnGn1+WnYabf7bhgLf0XucOawOLj7xYj649AOtepR6Q8X1UsaSHUs4450zAtplNItrxq77j74VSn0w8ouRTF49ubj5ZbQ9mnv73cuTZz6Ju8BNRl4GzeKaBXX5vxy7Am8B935zL5+v+5yEqAReGP4CQ9sNDXdYchQr01Zy2sTTyMzPrNT1cc44Pr/yc8464awajkwkNKqaeOlfnHqsb8u+PDr4UR6f8zhOmxO71c5XV3919DfWEyvSVgR0HM/z5LFw+0I6vtiRDRkbAEhwJjB35Fx6Nqv5qV8pn9Pm5OXzXubl814OdyhyDNo3bF9uvVmcI46E6AR2Zu0s89rRNgE/Fj6/j6U7l5JdkM3JzU+ukcVRIjVBI14RYE/OHnZn76Zdw3blFsPWVzd9cRNTVk8JGPFKjEpkd87ugOuSY5JJfzBdUyAixyh1ZyoXTLmAPTl7aBDdgLtPvZvuTbsztN1QerzSg837Nxe3wHDZXSwZvYSuTbpW+7ken4dzJ53Lgm0LsFvt2K125t08j86NOlf73iKVpalGkcNk5GYw+O3BbD24Fb/fT7cm3fh5589liuxtFhvpD6bTIKZBmCIVqdvyPHnF+0cW2ZixkaHvDWV75nZsVhtvXvgm13Y/epuKynh96evc++29xWUUFiz0btGbn0f/XO717kI39317H7P/mE3rxNa8fO7Lx7Sfpkh5lHiJlMPj87B6z2psFhtdm3Ql6akksgqyAq5x2py4H3FrmyiRGnAw7yBxzrig/v26/7v7eX7h8wHnkmOS2fvQ3nKvH/b+MOZsmUOeJw+rxUpSdBLr71hPsis5aDFJ5FEDVZFy2K12ejXrRfem3bFarEy4aEJAMb3VYmXSJZOUdInUkMToxKD//erdvDexjpK2F3aLvcI6zZyCHL7//fviDvs+v48CTwFTVk9h/b71ajMjIacRL4k4K9JWMHnVZHx+H7f2vpX2ye3DHZKIHAO/38+t02/lnRXvYLPaaBnfktk3zaZFfIsy1+Z78ol9Mhavv6S3mM1iw2a1Ybfa6ZTciR9v/FHF+XLMNNUoIiIRJT0nneyCbFontj7iqNqYr8bw7i/v4i50Y7VY8fv9xUX/UbYorul+DRMumhCqsKWeUOIltdLBvIN8se4LCn2FDG8/nJYJLcMdkohEGJ/fx2upr/HD7z+QuiuVPw78EfB6z6Y9WXFr6PbPlfpBiZfUOuk56fR6rRcH8w7ix4/damfBzQuCspxcRKQqHp75MOMWjyPfmw+YxTWXdbmMSZdMCnNkUtco8ZJa5+5v7uaVn18p3iTcgoWUtin8cOMPZa71+X1k5GbQILpBtQpxZ22eRerOVNoktuHKblcWb820z72PlbtX0tjVmO5Nu1f5/hJcfr+ft1e8zZwtc2jXoB3397+/0nsFilRFTkEOKe+ksG7vOixYaB7fnPk3z6eRq1G4Q5M6Rp3rpdbZnrm9OOkC8ONnV3bZ7YpSd6YyfNJwsvKzsFvtfHjZh5zX8bxjft4j3z/Cf+b/B6/fiw0bLy55kbk3zeWzdZ9x89SbsWLF4/dwZdcrmXDRBDVMrQXu+PoOJq6ciLvQTZQtik/WfkLqn1Jx2pzhDk3qqVhnLItGLWJF2gq8fi+9mvXS/98kpGrTvzwa8apnJiyfwF0z7irepDvGHsOtfW7l+XNK+u8UeAto/lxzMnIzis+5HC423LkhYIXSHwf+4E9f/omNGRvp27Ivl514GZNXT2Zn1k5+2/cbeYV55HnzysRgt9rx+DwB52wWG59e8SkXdb4IgEKvSQ4dNkfwPrwcVU5BDklPJwX87xPvjOejyz9iWPthYYxMROToNOIltYrf7+emnjexYd8Gnl/0PF6fl4GtB3Jbn9vY695LckwyFouFbQe3ke/JD3iv3WJn9Z7VxYlXVn4W/d7sx173Xrx+L9syt/HRmo+KVyUdyeFJF4DX72Xc4nGc2+FcRk4dyQerPwDghp438NLwl/j9wO8kRSdpIUANK/AWYCnnZ1ZRvyURCZ5CbyH/nvdvZv8xm/YN2/PkmU9qejVMNOIlQeXz+7h7xt28sewNAG7tcys7M3fyxW9f4PV78fl9OKwOBrUexFfXfIXX56XRfxpR4CsIuM/3139Pr+a9cBe6WZu+los/vLh4e5BgaNegHdd2v5ZnFjxDricXABs2bDYbDqsDj8/DqJNG8dK5L2lKsgYNmjCI1J2p5HvzsWChQUwDNty5gYYxDcMdmki9csXHVzB9/XTcHjcOq4OW8S1Zfdtq1VRWg4rrpVZ4at5T/N/c/ytOkhxWR0CdV5FoezQ397qZl897mZGfj2TiLxMDXo9zxlHgLcBmsRHriGVvbvlbgVSFBQvndzyfdHc6i7YvqvC6WEcs71/yPmcefyaLti8iyh7FgFYDAjrfS/Vk5mcyZvoYFmxdQJukNrx+wet0TO4Y7rBE6pWs/CyS/5Mc8LM43hnPlEunVKmeVgxNNUqt8OX6LwNGpspLusBMJ83fNh+A9Nz0Mq9nF2QXf180IhUMLocLl93FuGHjeHjWwyxhCT7K3zIk15PLvK3zGDN9DO5CNz6/j47JHflp5E+4HK6gxRTJEqIStIxfpIZVVJZRmXINCT7t1ShB1SKuRXELhyOxWWy0a9gOMJvbVkdKm5RKPTPaHs2dfe9k3R3rOL7B8Tw79Fmi7dEVXu/3+/li3Rek56STmZ9JdkE2a9PX8tyC56oVr9SctOw0nl3wLP+a+y9+Tf813OGI1AoJUQmc0+4cYuwxgFl0FB8Vz+ltTg9zZJFJiZcE1dNnP01CVAIx9hhcdhcOq6NMUhRli6JpXFNeGPYCAGNTxmKzBPbuOrzourwibDAJ3KRLJ/F4yuM4rEdelWi32jn7hLNJdplEr3Viax4c+CDWCv4a+PGzef/mgD3e8jx5/LpX/6DXRtszt9NtfDce/eFR/jH7H/R5ow8Lti0Id1gitcInV3zCHX3voG+Lvlze5XJSR6cSHxUf7rAikmq8JOjSstOYum4qFouFM9qewcipI1myYwk+v49zO5zLmD5jGNxmMHHOuOL3LN+1nJFfjCQ9N51h7Ybx277fWLl7JVaLFbvFzqDWg5i9ZTZZ+VkBw+NOm5ODfznIXvdeuo3vRlZBFj6/D5fDRceGHVmfsb64R1TH5I5lekQV/WOdmZ9Z7rC73WrH7/cXJ18uh4uxp48l15PLlgNbOLvd2VzZ9UoV4NcCd824i/E/jw9IlPu17MfCWxaGMSoRqa9UXC+12oG8A0TZoohxxFTqeq/Py+Idi8kuyKZvy74kRScBMPrL0UxZNQULFtMWYvg4Rp88GoCNGRv5x+x/kJ6TzhVdr2Bkr5G8mvoqP239iY7JHXl44MPlruDZlLGJJ+Y+wfe/f8+2zG0Br7nsLlrGt2Rb1jZ8fh8jOo1gWdoyth7cSoG3AJfDxZ1972RQ60HkefJIaZuiJdphcs2n1zBl9ZSAc52SO7HujnVhikhE6jMlXhIR/H4/P239iS0HttCrWa+gbf+z7eA2Or7UsUwPqdaJrVl/x3r2uvcSZY9i9h+zufHzG3F7AltbxDnisFgs2K125t88nxMbnxiUuKTyPv/1c677/LrixR0uh4t7+93LE0OeCHNkIlIfVTXxUo2X1CkWi4XT2pzG9T2vD+qei5n5meXWiO3O3s1NX9xEi/gWNHI1Ij0nvUzSBZBdmE1WQRYH8g7w56/+HLS4pPIuPvFi/nP2f2jsakxSdBK3nHQLY1PGhjssEZEAaichAnRI7kBCVAI5hTn4/CXtJfK9+Uz9bSqb92+mXcN2R63l8uMvM10poXP7Kbdz+ym3hzsMEZEKacRLBFOkP3fkXDo36lzmNbvVTlZBFnty9vDX7/96xPtE2aI4rfVpNRWmiIjUcUq8RA45ocEJpI5OpXlc8+IWGFaLlYSoBE5sdCITV0wkOz+7zPtOanYSdosdm8WG3Wpn0qpJtPlvGxZu02o6EREJpMRLpJQYRwzzbp5H35Z9aRDdgH4t+/HTyJ+IskeRU5AT0KoAzNZGy/68jIN/OUizuGbkenLx+r1szdzKOe+fw56cPWH6JCIiUhtpVaNIJa1IW8HACQMDVs3d3OtmXjz3RbYc2EKX8V0CtkuyWqw4rA46Jnfk48s/plOjTuEKXUREgiycqxqHAeuADcDDFVzzwqHXVwInBeGZIiHXq1kvpl01je5NutMmsQ1j+ozh+XOeByApOgmPzxNwvc/vI9+bz6o9q+gyvgsr0laEI2wREalFqjviZQN+A84CdgA/A1cDpfdUORe449CfpwLjgH7l3EsjXlKnPT7ncZ6e/zQF3oIySRiYnmBb7tkShshERCTYwjXi1RfYCPwBFAIfABcdds2FwDuHvl8MJAFNq/lckVrnsdMf48urv+SOvneU2XsSzPZE+uVCRCSyVTfxagmUblq0/dC5o11zXDWfK1IrDTl+CM8PfZ5TWp5S5rXWCa2P2Ads8Y7FvLPiHTZlbKrJEEVEJIyq20C1sr++H/6vTbnvGzt2bPH3KSkppKSkVCkokXCyWCz8dNNPnPP+OczdOhdiBIiFAAAgAElEQVSn1UmUPYpPr/y0wvdcOOVCvlz/ZfHx/53xf/zttL+FIlwREamE2bNnM3v27Grfp7o1Xv2AsZgCe4C/Aj7g6VLXvArMxkxDginEPx3Yfdi9VOMl9c7GjI3sde+lS+MuJEQllHvNNxu+Yfjk4WXOp92Xxh73HhKjE2md2LqmQxURkWNQ1Rqv6o54pQIdgLbATuBKTHF9adMwxfUfYBK1A5RNukTqpfYN29O+YfsjXrNk55Jyz3d7tRv5nnwKfYVc3e1q3rrwraNuWSQiIrVbdWu8PJik6ltgLfAhZkXjnw99AXwNbMYU4b8G3FbNZ4rUK4NaDyr3fEZuBlkFWeR58vhozUd8uObDEEcmIiLBFoxNsmcc+irttcOO7wjCc0TqpSHHD+HyLpfz8dqPi885bU4KvAXFxzmFOazeszoc4YmISBAFI/ESkWr66PKPSN2RyvK05Zza8lRGTh3J8rTl+A+tQ4l1xJa7gbeIiNQt2qtRpBZ4fenrnP3e2dw5404en/s4b130Fo1cjUiISsDlcHFeh/O4pvs14Q4zwMxNMznu+eNw/cvF0PeGss+9L9whiYjUerWpUlerGiUizdw0kxEfjije5zHaHs0VXa5g/HnjWbVnFYlRiXRu1DmgsH5l2kre/eVd7BY7o04eRcfkjiGNef2+9Zz02knFMTusDk5teSo/3fxTSOMQEQmXcK1qFJFq+mbTNwGba+d58pixcQaxzlj6HVd2d62F2xZy1ntn4S50Y8HC+NTxLBq1iK5NuoYs5jl/zAk4LvQVsmD7Ajw+D3arfqyIiFREU40iYdbY1ZgoW1TAuYYxDSu8/tEfHi1O1Pz4ySnI4cmfnqzRGA/XIKYBVkvgj48oW1S5WyWJiEgJJV4iYTamzxiaxzfHZXfhtDlxOVyMP298hddnF2QHHPvxczD/YE2HGeDCThfSKbkTLocLm8WGy+Fi3LBxNdpnzOf3sfXgVvbn7q+xZ4iI1DTNCYiEWWJ0Ir/c+gsfrvmQrPwshrUfxomNT6zw+ht73sja9LXkFOYA4HK4uLHnjaEKFzDtLubfPJ/3f3mf3Tm7Oa3NaRX2IwuGHZk7OOOdM9ieuR2vz8ttp9zG8+c8r4ayIlLn1KafWiquFzkCr8/Lt5u+ZZ97H7/u/ZX3f3kfm9XGo4Mf5ZaTbwl3eDXqtLdPY8G2BXj9XsC013j34ne55MRLwhyZiEQqFdeL1GMen4cz3z2TZbuWAWba7curv2TI8UPCHFlorNy9sjjpAtNQNnVnqhIvEalzVOMlUgd8sPoDlu5cSnZBNtkF2bgL3dzw+Q3hDitkDt8k3OVwHXUPTBGR2kiJl0gdsDNrJ/ne/IBze917wxRN6E2+ZDINohuQEJVArCOWwa0Hc0PPyEk8RaT+0FSjSB3Q/7j+OG1OPD4PAHaLnT4t+oQ5qtDp3rQ7G+/aSOrOVBKiEujbsm+ZdhYiInWBiutF6oiXl7zMfd/dh8/vo1vjbsy4bgbN4pqFOywRkYhU1eJ6JV4idYjX5yXPk0esMzbcoYiIRLSqJl4aqxepQ2xWm5KuqsrJgXvvhdNPh7vvhuzso79HRCTINOIlEql27wavF5o3h/reiNTng/794ZdfIC8PoqKgWzdYvBhs2uZIRI6dRrxEpHIKCuD886FNG2jXDlJSwO0+6tvqtHXrYM0ak3QB5OeXnBMRCSElXiKR5l//gh9+MMlHXh4sWQIPPxzuqGpWeaPpFkv550VEapASL5FIs2AB5OaWHOflwcKF4YsnFDp3ho4dzRQjmD/btYOuXcMbl4hEHCVeIpGmc2dwOkuO7Xbo1Cl88YSCzQazZ8PIkXDqqXDTTTBnjvnsIiIhVJsqalVcLxIKBw6YQvMdO8xxw4ZmurFJk5p/9pQp8NFHkJwMjz4Kxx9f888UEakB6uMlIpWXnw+LFplVjf37Q0xMzT/zv/+Fv/3NFPJbrZCQAKtXQ8uWNf9sEZEgU+IlIrVbkyaQnl5y7HDAE0/AQw+FLyYRkSpSOwkRCY78fNi4ETIzg3tfjyfw2OeDwsLgPkNEpJZT4iUiJX7+2TRU7dXLjFC98Ubw7j16NLhcJcfR0XD55cG7v4hIHaCpRhExfD6TbO3bV3IuJgaWLoUTTwzO/Z96yhTXN2gAzzwDffpU/74iImGgGi8RqZ49e6B1azPVWCQhAd58UyNTIiKHUY2XiFRPw4Zl+1p5PHDCCVW7n99ftq5LRCTCKfESEcNuhw8+MHVYiYlmmvGuu6B372O/1/vvm9Eyp9M0LN29O/jxiojUQZpqFIk0fr/Zq3HPHujb12ydU9rOnWbz6OOOq1pt19KlcNppJRtv2+3mOfPnVz92EZFaoqpTjdovQySS+Hxw8cUm8bJYTAPVjz+Gc88tuaZFC/NVmscDEyaYhKxXL7jyysAViqXNm2fuW/q9ixcH/7OIiNRBSrxEIsn06Sbpys4uOXfddZCRUfF7/H646CKz12HRKNaoUWY0bMYMU5BfWtOmZpSrdJF+UlLQPoKISF2mGi+RSLJ9e+BoFMD+/XDLLYFtJEpbtSow6QKTjK1bB+edV/b6Sy+Fnj0hLs6MirlcZrRMRERU4yUSUZYuhcGDITc38LzDAa1amb0TD9+3ceFCGDas/E72FosZ2XI4As97PDBtmtkiaPBg6NIluJ9DRCTM1MdLRCpnwgQYMwYKCgLPx8fDJ5/A0KGB53NyoH17szLx8L+j8fFw8KBJwEREIoj6eIlI5dx8M2zdWnaUCsomVgCxsWZFYr9+5j1Wa8kU4sSJtT/p8vvNogIRkVpAiZdIJGraFIYMKZlWdDhMA9VBg8q//oQTYMECyMuDr76Cl18205aXXBK6mI+V3w9/+YvZEzIqyiwiOHyUT0QkxGrTr6qaahQJpbw8+PvfTfuHDh3g2WfNXo31xRtvwD33lCwKiImB2283e0SKiFSTarxEpPbasQNmzTLTk+efX7aAvyZcfDF88UXgua5dzQICEZFqUgNVEQm9bdvgnXegsNA0VS1v9eKKFaaTvc9n6sGaN4fUVLOlUE067jjTT6xov8iiZ4uIhJFGvESkrHXrYMMG6NzZTEOWZ9Mms49jTo5JqqKj4fvvTRF+aaeeCkuWlBxHRcEjj8Bjj9Vc/GBWYZ50kmmD4febJGzRoqptgyQichiNeIlIcDz3nKn9cjjMSNZzz8FNN5lu940alaxi/Pe/ISurZMWg2w0PPwxz5gTeb+fOwOP8fDNSVtOaNoW1a2HqVDPqNXx42a2QRERCTKsaRaTEtm3wt7+ZBquZmebPu+4y04KtWkG3brBrl7k2I6Nsm4YDB8rec8gQMxpWxOWCs86quc9QWlIS3Hij2eJISZeI1AJKvESkxLZtZiqwNI/HfOXnw/r1cNVV5vzVVwdulO1ylbxW2ssvw+mng81mpvvuvx+uuKLmPoOISC2mGi8RKbF3L7RpE7gv4+Hi4swUI5ik6oknzEiXx2NWKz7yiOmfdbiCApN4WfX7nojUfepcLyLV16iR2Tbo8FGv0kpP2d1+O4wcaZIpj8ckZP/3fzB5ctn3OZ1KukQk4umnoIgEGj7cJE/2CtbejB8fePzFF4EjZG43fP55xff3esvfmkhEJAIo8RKRsoYMqfi1w1cpNmoUeGy3mxWFh8vNNU1No6LMlOTjj1c/ThGROkaJl4iU1bu32Uy7PF5v4PH//mfqvpxOs3qxQQNT53W4u+6Cb74x78/Ph6efho8+Cn7sIiK1mBIvESnfuHGmHUNpUVFm5eM335jpwuxsk2j9/HNJUf3q1eW3bpg50+wPWcTthhkzavYziIjUMmqgKiLli46GrVtNi4gVK8y5jAyzitFmMx3q588354u2A3I44LPP4McfoWHDwPs1bQpbtpQcO51mWx8RkQiidhIicnTbtpmtg/LzK3d9YiI8/3zgdOXy5YF7NjZubM4dPqomIlIHaMsgEak5e/eW7VJ/JAcPwp13mpGxG2805046Cdasge++M8X1F11kasNERCKIRrxE5OjcbpMkHevf0VNPNRtTi4jUM2qgKiI1x+WC448/9vcdqRGriEgEUuIlIpXz6qvHdn1MDPzjH6Z/15o1kJ5eM3GJiNQhSrxEpHLOPhv696/ctTabaR+RmAgtW5r3tWpleneJiEQw1XiJSOV5vaYNxNEK7aOjYdky0wE/La3kvMsFc+ZAnz41G6eISA1TjZeI1DybzWyGXdE+jkXy880I2eHTixaLabAqIhKhlHiJyLGxWI7ecd7vhz17ID6+7PkOHWouNhGRWk6Jl4gcu7POKn9boNJsNvj0U9OGIjHRFNvffjsMHBiaGEVEaiHVeIlI1bz3HowcWXbT7Lg4UwP2+utw7bWwb59Z1di8uUa7RKTeqGqNlxIvEakavx8efBD+9z/z/QknQHKyKaZPSTHtJ6Kjwx2liEiNUOIlIuFRUGDquXr3NqNbXq+ZVjzrLJg2LXRxzJsHjz1muuzfcguMGmXq0eoyvx9mzTKbi598svkSkVpBezWKSHg4nbBkiWmUWjTtmJtrCvBzciA2tuZjSE2Fc84xSRfAqlWQlwd33FHzz64pfj/ccAN8/nnJVk3PPQe33hreuESkWqpTXN8QmAmsB74Dkiq4bgKwG1hVjWeJSG1ms5U95/eHbsRpwoSSpAvM9+PGhebZNWXxYpN05eSYz+N2w913m4RSROqs6iRef8EkXh2B7w8dl+dtYFg1niMitd1ZZ0HDhuBwmGOXC665xvwZCuUlftY6vmg7La3s57JaYf/+8MQjIkFRnZ9MFwLvHPr+HWBEBdf9BOgnhUh9FhtrpvtGj4Zhw8wejW+/Hbrn33pr4JSmywWPPBK659eEk082zWqLWCzQqBE0bRq+mESk2qozD7AfaFDqPhmljg/XFvgS6H6E+6m4XkSqbuVKePJJyM42hfWXXBLuiKrv22/hqqvg4EGzanT6dOjUKdxRiQg1t6pxJtCsnPOPYka5SidaGZi6r/K0RYmXiEjVFBSYRQwiUmvU1KrGs4/w2m5MUpYGNAf2HOvDDzd27Nji71NSUkhJSanuLUVE6j4lXSJhN3v2bGbPnl3t+1RnqvE/wD7gaUxhfRIVF9i3RSNeIiIiUk9UdcSrOsX1T2FGxNYDQw4dA7QAppe6bgqwALP6cRswshrPFJG6xOuFrVshKyvckYiI1Aq1qa2zRrxE6pP162HIENP+wOOBf/0LHngg3FHVD5s2wV/+Art2wUUXwf331/32GSJ1jLYMEpHapVMn2LChpOu6ywUzZ8KAAeGNq65LS4MTT4TMTLMZuctltkiq6w1jReqYcEw1ioiUz+cLTLqKzi1fHr6Y6ovPP4f8fPPfE0xH+9deC29MIlJpSrxEJPisVmjSJPCczQbHHx+eeOqT8rZhquubgYtEECVeIlIzPvoI4uIgMdF0lR8xAoYPD3dUdd/FF0N0dElNl8sFY8aENyYRqbTa9GuSarxE6pu0NFi2zIx+9e6tkZlg2bzZbIlUVFx/zz0qrhcJMRXXi4iIiISIiutFREREajklXiIiIiIhosRLRKQ+27wZ5syBPdXeTldEgkCJl4hIffXEE9C1qynAP+EEmDEj3BGJRDwV14uI1EerVkG/fqbBapHYWLOFk8MRvrhE6gkV14uIRJoVK0wn+w0byr62cSPY7YHnvF5ITw9NbCJSLiVeIiJ10d/+BgMHwk03Qc+e8Pbbga936QKFhYHnoqPL7iggIiGlxEtE6p9Zs+BPf4IHH4Tt28MdTdWtWgW9ekHDhnDmmaYhLcC6dfD882YaMTMTcnNN9/qsrJL3dupkromOhvh4SEiAL78sOwomIiGlGi8RqT02boRnnzUJxA03wDnnHPs9Jk+G0aNNUmKzmYRj1Spo2TL48dakjAxo3x4OHDCbjdvt0KEDrF4NM2fClVfCwYMl18fGwsqV0K5d2fukpUHbtmZ7IREJCnWuF5G6bfNmM7qTnW0SDZcL3noLrrrq2O7Tti1s2VJybLfD3/8Ojz0W1HBr3DffmM9eOrmKiSmp5+rYMbBwPinJJFhRUaGNUyRCqbheROq2V1+FnByTdIFJKiqTLKWlmam3ggJznJcX+LrHY+5b18THm2L40rxeM7LVsiVMmmSS0+hoaNDAJGpKukRqPSVeIlI75OWBzxd4riiZKo/fD/fea0a4TjnF9KnatAmuvz5wSs3lgssuq5GQa1T//uZzFX2W2Fi47TYzsgUwYoRpDbFpk1mpeOqp4YtVRCpNVZYiUjtcdx28+aYpFAeTcNxyS8XXT58Ob7wB+fnmy+02CVZqKjidMGUKxMWZmrFTTgnNZwgmqxW+/dasVty4Efr2hUsvDbzG6YQWLcITn4hUiWq8RKT2mDUL/vpXMzV4001mVaKlgh9TTz5ppiJLT8dFR5ckbiIiNaiqNV4a8RKR2uOss8xXZbRvbxKt0vVbbdoc/X35+fDFF2aaLiUFOneuUqi1mtcLP/5oWk0MGADNmoU7IhE5RCNeIlI3+f1wzTUwbZrZAsdqNclGz54VvycvzyQiGzaY5MRiMZ3fhw4NXdw1raDA9PxascL8N/H7zX+X3r3DHZlIvaJ2EiISefx+06Nr/36TcBUVnlfkjTfgnnsC2zC0bFm3m6we7rXX4L77Aj/jiSfC2rXhi0mkHlI7CRGJPBYL9OgBp59+9KQLYM8eM9VY2v79NRNbuGzZEph0AezcWfY6j8dsO9SzpxnxW706NPGJRDjVeIlI5Dj9dNPrqigxcTph8ODwxhRs/fub1hNFtW92e/mrOseMMV3+3W6TwA4YAGvWQKtWoY1XJMJoxEtEIsegQTBunElMbDaTpEyeHO6oguuCC+D++03dm9NpNsueNKnsde+9V5KA+v1mQ+1PPjEtLL7/vuzIoIgEhWq8RCQy+Xym+Lw+yMw0I1jz5kHr1vD663D88WbUq2HD8ltyuFyBrTeio81ooN9vvlq2hEWLIDExdJ9DpA5RjZeIyLGoL0kXwPnnw6efwtatMH8+DBxo9rxMTq64D9r995d0xbfbTSKalWWSuKwss3fmP/8Zus8gEiFU4yUiUpcdPAgLF5pieTCjVR4PzJlTttN9aY8/brZbmjYNmjc3169bV/J6QQGsX1+joYtEonr0K5+I1GtFWwNJIKez7Dm/H2Jijvw+iwVGjYKpU80G5SkpZrqxiMtl9n9cvhy2bQtqyCKRTImXiNRuBQVw1VWmID421mwlVHqboEgXE2Pqu4qmDaOiTAf/M888tvs8+6xJtKKiTDJXtBDh9NOhQwezIbmIVJuK60Wkdnv0UfjvfwM3z/7b38yejmL4/TBxopkubNfOJElxcVW7T3q6qfnq1890+C8SG2vqyM45J2hhi9Rl6lwvIvVTv36weHHguSFDTMsDCT6fDzZtMt3uS48sOp3w73+brvgiolWNIlJPtWljem4VsdtNUbgE3/79Zk/HXr3KTudaLPDLL/D11+GJTaSe0IiXiNRu27ZBnz4lzT4TEmDpUmjWLLxx1UfXXQcff2zq6oo4nSUbivv9pgB/zBh45pnwxSlSC2iqUUTqr/37YeZM84//OeeYOq/9+02fqmD34/L7TT8st9sUldsjqOtOly7w66+B504+2Zwr3WzV6TQJcZMmoY1PpBbRVKOI1F8NGsAVV8Dll8OPP5pu6q1amVGvpUuD9xyvFy67DDp3NvsbdukCaWnBu39td+KJgYlmdDR07Vq2ZYXTCQcOhDY2kXpCI14iUnds3w6dOpVMO4IZ9dq1y+xNWF2vvAIPPFByf7sdhg6F6dOrf++6IC3N7F+ZkWGK7Lt0gS++MH8WJVpWK7RoYTrbB+O/uUgdpREvEan/fvml7NRfbi7s2BGc+6emBiZ1Hg+sXBmce4dKdX6BbdbMTCvOmGFGFhcsMF3tf/jBTLtGRUGPHjB7tpIukSpS4iUidUfr1iVb4xTx+aBx4+Dcv0ePwI7vNpuZdqwLxo+H+HgzDXjhhWavxqqIjoYBA8yChqLVpCedZLYPyssznezbtQte3CIRRomXiNQd3bqVdGlPSDB/vvyyae4ZDLfdZpKO2Fhz/+bNYcKE4Ny7Jn33HTz4oEm2PB5z/Kc/hTsqESmHarxEpO5JTYXff4fu3YM/IuXzwapVZgqzZ8+j73lYGzz4oNnyp7RGjUwXehGpEVWt8YqgddIiUm/06WO+aoLVahKuuqRZM1N/VXoT8eTk8MUjIhXSiJeISF2XlWX6be3aZVpiWK1mJWZKSrgjE6m31EBVRCSS5eTAJ5+YJGzoUOjYMdwRidRrSrxEREREQkR9vERERERqOSVeIiIiIiGixEtEIpfHA+vWmQ2fawufDxYtglmztB+iSD2kxEtEItPOnaYHWJ8+phD9uutM0hNOhYVw9tnm69JLoX17s4VPVfj9MG4cHH+86TT/xhvBjVVEqkSJl4hEphtvhD/+MKsB8/Lg88/hnXfCG9Nbb5nRruxsyMw0m1XfcEPV7vXGG/DII+Yzbt4M99wDH34Y1HBF5Ngp8RKRyLRqlel5VcTthqVLwxcPmP0QS2/S7febDv1V8fbbgfdyu2HixGqFJyLVp8RLRCJThw6m0WgRlwu6dj3yew4eLOkOn5sLP/wAP/5oRsyC4ZRTAvedtNvNBtVVERcXeGyxlD0nIiGnPl4iEpk2b4aBA81IkNcLgwbBV1+ZZOdw+/bB8OGwYoUZhbr9dpg61ZwHs2XP4sXQoEH1Yiq691tvgc0GbdqYxK5Zs2O/16JFcOaZ5vNZLCaxXLAAevSoXowiAqiBqojIscvJMclUbKzZn9FSwY/ECy6Ab781xe9gkjO/v2Sq0umEm2+GV14JTlz79pk6r1atAkfljtXKlaZuzWqFUaPgxBODE5+IKPESEakxjRvD3r1HviYlxYxOiUhEUOd6EZGa0rJl4LHNFjglGRNjpi1FRI5CiZeIyNFMnAgJCRAfbwrUe/WCwYMhKsp8paTA3/8e7ihrxiefmF5gTZrAHXeUTLeKSJVoqlFEpDJ274Z580ziNWSIGfFKSzN1YVUpfq8L5s2DoUPNCk4wI3ujRsGLL4Y3LpFaQDVeIiISXA88AM89F3iuWTPYtSs88YjUIlVNvMpZNy0iIgIkJoLDETi9eKReYHv2wEcfmetHjDBTlCISQCNeIiJSvj17oHt32L/fJFMuF3zwgWmvcbjt202z1+xss+el0wk//WTq4UTqIU01iohI8O3ZA2++afaOHDEC+vUr/7pbbzXXld6G6YwzTHd/kXpIU40iIhJ8TZqYzbaPZvfuwKQLjt77TCQCqZ2EiIhU34gRZiqyiMsFF10UvnhEailNNYqISPX5/fDkk/Cf/4DHAzfdBOPGlb/3pUg9EK4ar4bAh0Ab4A/gCuDAYde0At4FmgB+4HXghXLupcRLRERE6oRwbRn0F2Am0BH4/tDx4QqBe4GuQD/gdkA7tYqIiEjEqW7idSHwzqHv3wFGlHNNGrDi0PfZwK9Ai2o+V0RERKTOqe5U436gQal7ZZQ6Lk9bYA5m9Cv7sNc01SgiIiJ1Qk22k5gJlLcR2aOHHfsPfVUkDvgEuJuySRcAY8eOLf4+JSWFlJSUSoQnIlIPTJ0Kf/6z6Zd1+ukwZQokJYU7KhE5ZPbs2cyePbva96nuiNc6IAUzndgc+BHoXM51DuArYAbwvwrupREvEYlMK1dC//4lm1E7nab56DffhDcuEalQuIrrpwE3Hvr+RuCLcq6xAG8Ba6k46RIRiVw//BDYfLSgQB3fReqp6iZeTwFnA+uBIYeOwRTPTz/0/UDgOuAMYPmhr2HVfK6ISP3RoIHZjLq0+PjwxCIiNUoNVEVEwi0vD045BTZvNqNdDgdMmABXXRXc5/j9MGsWbNgAPXrAoEFHvtbrVQNUkQpok2wRkbosLw/efx/27YMhQ0wiFmxjxsB774HPBxYLPPwwPPZY2eueegrGjjUd6M8+Gz7+GOLigh+PSB2mxEtERCq2di306VNSwA8QFQXbtkHjxiXnpk6Fa64Bt7vkmssuM0mhiBQLV3G9iIjUBXv2mNWSpTkcsHdv4LnvvitJugDy8+H770uOMzNh1Cjo2ROuvtrcV0QqTZP3IiKVsXQpzJwJiYlw/fV1b+qte3czxVhaTAzk5ECvXrB1K5x8Mpx6qhnlys8vua5pU/Onz2emQVevNq//+issWWJG06KiQvdZROowTTWKiBzNtGlmdKeo8L1FC1i+vOKVh243/P47NGsGycmhjfVIFi6ESy+FtDRo2xbefRcuuAAOHDCv2+3Qrp0pqk9LM4mW1WpGvPr2hU2bTFF+6RGx+HjTb2zAgLB8JJFw0VSjiEhNue02k2x4PKZGascOmDix/GsXLTKJWf/+0LIljBsX0lCPqH9/2LkTCgvNCsrMzMBRMI8HtmwxidTrr5vYV60ySReAzWZWO5bm92vlo8gxUOIlInI0mZmBxwUFsH9/2et8PjjvPDh4ELKyzHTcI4+Y5KU2sdnMn/HxZacfvV5TbH/11XDOOeZzFo1wtWkDgwebKUqA6Gjo2NFMUYpIpSjxEhE5muHDA2uYoqNh6NCy1+3fD9mHbUVrs8GaNTUbX1UNHAi9e4PLZY5jY+HOOyEhAe691yRVKSkm4Vq92rSg+PJL04Zi+HBz7Zw5GvESOQaq8RIROZqcHLjpJpgxwxTVjxsHV15Z9jqv13Shz8oqOedywdy5JsGpjQoLTbPWTZtMYf0ll5iVjZdeaj53kQ4dYP368MUpUstUtcZLv6aIiBxNbKxpIno0Nht89hmMGGFGgQoK4P77K066li0zrRl27TKjT2+9BUlJwY39aBwO+POfA8+tWWMSstI2bw5dTCL1mEa8RESCLT3dtFpo2dKsEizPrl3QqVPJ6JjTaYrYf/opdHFW5Ouv4YorAke8TjjBjIqJCKBVjSIitUfjxnDaaRUnXWBqow439+YAAA7YSURBVEorKDArIksnO+vWmSm/006Dl14qu6KwpgwfbnqVxcSYeq+GDeHTT0PzbJF6TlONIiLhEBtbfiJVVMS/ZYsZAcvONtctXWpG0v75z+o/OzcXnnvOJHYDBsCtt5p+XUUsFnjlFTNNum8fdOlScc8yETkmmmoUEQmHggKTWP32m9kg2+WCBx4oSayeeQYefTSw1iopqfw2FpW1cKGpI5s2zbS8KCgwz730UtNMVUQqTcX1IiJ1idNpEqFXXjGjW6efblYU1pRZs+DCCwM3yQbTo+vDD+G//61dXfZF6imNeImI1EZbtpj9FYumGl0uM/X3+ONVu1///qaGrDzR0bBxo1kMICKVohEvEZH6pE0bswH1o4/C3r1w+eVw++1Vv9/hI11FnE5Tw9WiRdXvLSKVplWNIiLhlJcHN99spvnatIGpU0te69zZrCacMwfuuMMUvVfVLbeUdKgHU0zfpInpOTZrVvXuLSKVVpv+pmmqUUQiz8iR8MEHJgED08Jh9uySjamDxe+HF16Al182zV3HjjW9ukSkSqo61ajES0QknBo2DFypaLHAY4+ZxEhEai01UBURqYtiYwOPnU7TtLS0ffvMZtUOh0nUPvwwZOGJSHBpxEtEJJw++QRuuMFMNTqd0KgRrFplNtsucsYZMH9+SU8vl8tsLXTyyeGJWUS0qlFEpE667DLTxmH6dNMgddSowKQLYN488HhKjj0emDtXiZdIHaTES0Qk3Pr3N18VSUiAjIySY4fDjIyJSJ2jGi8Rkdru9dfN9GJUlKkJ69w5uCsSd+6EgQPNM44/3kxrikiNUI2XiEi4+P0waZKZNmzXDu68M7DXVmkrV5p+XsnJppmq0xm8GLp1M3tGer3mXFyc2UBbnexFKqR2EiIidc0998Cbb0JOjtm2p3NnWLw4eElVZezbZ7rWFxSUnEtIMJtpX3ZZ6OIQqWPUTkJEpC5xu00z05wcc5yXB5s2wY8/hjaOuDgz6lWaz2faVohI0CnxEhEJh/z8stv0WCwmIQulqCiz8bbLBTabqSHr18/0DRORoNNUo4hIuPTvD8uWmWk+iwUSE2HDhvCsWPz+e7Mp93HHwdVXm22FRKRCqvESEalrDhyA0aNh4UKT8Lz1FnTtGu6oRKQSlHiJiIiIhIiK60VERERqOSVeIiIiIiGixEtEREQkRJR4iYiIiISIEi8RERGREFHiJSIiIhIiSrxERET+v727j5GjruM4/i5tKb1r5NoUaEuBg0otNND6ABZUWIIkFGslNcFQQUglIUEjRrG0EvVi4lN9wBiNJm3AgijUKgjaqK1yNuITplyhVB7KQywqh4hIOa6A9PzjN+ddt3u92Zud3+7svl/JpTN7w+73y7azn535zW+kSAxekiRJkRi8JEmSIjF4SZIkRWLwkiRJisTgJUmSFInBS5IkKRKDlyRJUiQT6l2AJDWkffvgpptg2zY4+WS44gqY4C5TUjbj6l3AMAMDAwP1rkGSgksvhdtvh74+aGuDs86CTZtgXCPtNiXVy7iwL6h6h9BIexCDl6TGsHs3zJ0Le/cOPdbeDvfcAwsW1K8uSQ1jrMHLMV6SVK6v78DTiuPHw4sv1u419u6Fnh544onaPaekhmfwkqRyc+bAkUeGsAVwyCEweTIsXFib59+1C044IZy+nD8fLr8cPOIvtQSDlySVmzgRtm6FUgmOOAIWLQqnGdvba/P8F18Mvb2wZw/098PGjbBhQ22eW1JD8xIdSark6KNhy5bKv3vhBbj/fpg6NVzxWO2A+4cfDldNDurrgwcfHHutkgrDI16SVI0dO+D442HJEjj9dFi+vPrThCeeuH9Ya2+Hk06qbZ2SGpJXNUpSNebPh507h9bb2+GGG+Cii9I/xyOPhPFd/f3w6quwbBncfLNTVUgFMtarGj3VKEnVePLJ/df7+0OQqsbcufD44+H0YkdHOAImqSV4qlGSqjFvXrjKcdDkyXDKKdU/T1sbnHaaoUtqMQYvSarGhg0wcyZMmQKTJoWpIJYurXdVkgqikQYUOMZLUjG88kqYi6ujA2bNqnc1kurAWwZJkiRF4uB6Saq1nh649dZwSnHFCjjuuHpXJKngPOIlSZVs3QqLF8NLL4VbB02ZAtu2hVv9SGp53iRbkmpp5coQugBeey3c3mfNmvrWJKnwDF6SVMmePfuv79sHzz9fn1okNQ2DlyRVsnx5mGtrUFsbXHJJ/eqR1BSyDK6fBtwGHAc8CVwElH8dPAz4DTAJOBT4CbA6w2tKUhyrV8PLL8O6dTBxInR1hfszSlIGWQbXrwGeTf68FpgKrKqwXRvwEiHk/Ra4JvmznIPrJUlSIdRjcP1SYH2yvB64cITtktGpHAqMB57L8JqSJEmFlSV4HQX0Jsu9yfpIr9GTbHM3sDPDa0qSJBXWaGO8NgMzKjx+Xdn6QPJTyT5gIXA48AugBHRX2rCrq+v/y6VSiVKpNEp5kiRJ+evu7qa7uzvz82QZ4/UQIUQ9DcwkHM2aN8p/8ymgH/hKhd85xkuSJBVCPcZ43QlclixfBtxRYZvpQEeyPBk4D7gvw2tKkiQVVpYjXtOADcCx7D+dxCxgLfAu4FTgu4SAdwhwM/DlEZ7PI16SJKkQxnrEy3s1SpIkVcl7NUqSJDU4g5ckSVIkBi9JkqRIDF6SJEmRGLwkSZIiMXhJkiRFYvCSJEmKxOAlSZIUicFLkiQpEoOXJElSJAYvSZKkSAxekiRJkRi8JEmSIjF4SZIkRWLwkiRJisTgJUmSFInBS5IkKRKDlyRJUiQGL0mSpEgMXpIkSZEYvCRJkiIxeEmSJEVi8JIkSYrE4CVJkhSJwUuSJCkSg5ckSVIkBi9JkqRIDF6SJEmRGLwkSZIiMXhJkiRFYvCSJEmKxOAlSZIUicFLkiQpEoOXJElSJAYvSZKkSAxekiRJkRi8JEmSIjF4SZIkRWLwkiRJisTgJUmSFInBS5IkKRKDlyRJUiQGL0mSpEgMXpIkSZEYvCRJkiIxeEmSJEVi8JIkSYrE4CVJkhSJwUuSJCkSg5ckSVIkBi9JkqRIDF6SJEmRGLwkSZIiMXhJkiRFYvCSJEmKxOAlSZIUicFLkiQpEoOXJElSJAYvSZKkSAxekiRJkRi8JEmSIjF4SZIkRWLwkiRJisTgJUmSFInBS5IkKRKDlyRJUiRZgtc0YDPwCPBLoOMg244H7gPuyvB6Tam7u7veJdSFfbcW+24t9t1aWrXvscoSvFYRgtdc4FfJ+kiuBnYCAxlerym16l9Y+24t9t1a7Lu1tGrfY5UleC0F1ifL64ELR9huNnABsA4Yl+H1JEmSCi1L8DoK6E2We5P1Sq4HPgHsy/BakiRJhTfaEajNwIwKj19HOMo1ddhjzxHGfQ23BFgMfAgoAR8H3j3Ca+0C5oxSjyRJUiN4DHh9zBd8iKFQNjNZL/d5YDfwBPAPoA+4KUp1kiRJTWQNcG2yvAr44ijbn41XNUqSJI3JNGALB04nMQv4WYXtzwbujFOaJEmSJEmSVAdpJ2BdDTwIPAB8H5gUpbr8pO27A9gI/IUwB9qiKNXlp1Un3E3T9zHA3YS/5zuAj0SrrvbOJ4z3fJShoQjlvpH8fjvwxkh15W20vt9P6Pd+4B7g1Hil5SbNew1wGvBfYFmMoiJI03eJsA/bAXRHqSqO0XqfDvwc6CH0fnm0yvJzA2HmhgcOsk1h9mlrgJXJ8rVUHiPWCTzOUNi6Dbgs98rylaZvCFeNrkiWJwCH51xX3tL2DfAx4Baa49R0mr5nAAuT5SnAw8BJ+ZdWc+MJVyd3AhMJO9/yPi4ANiXLbwX+EKu4HKXp+wyG/g2fT/H7TtPz4Ha/Bn4KvDdWcTlK03cH4UvU7GR9eqzicpam9y7gC8nydOBfhM+vInsHIUyNFLwKtU97iKG5v2ZQ+arIaYQPoamEN+8u4J1RqstPmr4PJwTOZpKmbwg7qy3AOTTHEa+0fQ93B3BubhXl5wzCt91BqzjwjhbfAd43bH34/5+iStP3cFOBp3KtKH9pe/4ocBVwI80RvNL0fRXw2WgVxZOm9yuBbyXLJxCO9DeDTkYOXlXv0+p5k+w0E7A+B3wV+Cvwd+B5wodykaXp+3jgn4Sd1TZgLdAWpbr8tOqEu2n7HtRJ+Hb1xxxrysvRhOljBj2VPDbaNrMptjR9D/dBhr4hF1Xa9/o9wLeT9Wa4ZVyavk8kHDS4G/gzcGmc0nKXpve1wHzC5/V2wu0Cm13V+7S8DwEebALW4Qao/I9yDuEbUyfwH+CHhLESt9SuxFxk7XsC8Cbgw8C9wNcJ3yw+XcMa85C17yXAM4SxEaWaVpavrH0PmkIY13c18GJtSosq7Qdr+cTNRf9Arqb+cwhDCN6WUy2xpOl5cL81QHjPm+GWcWn6nkjYf59L+ML8e8Lpp0dzrCuGNL1/knAKskT4/N4MLAD25FdWQ6hqn5Z38DrvIL/rJXxYPU2YgPWZCtu8Bfgd4TwxwI+BM2n84JW176eSn3uT9Y0c/NRFo8ja95mEe4BeABwGvI4w4e4HaltmzWXtG8LO+kfA9winGovob4QLBQYdw4Gn1Mq3mZ08VmRp+oYwoH4tYYzXvyPUlac0Pb8ZuDVZnk64i8mrFHvsZpq+dwPPAv3Jz1ZC+Ch68ErT+5nA55LlxwiTp7+BcOSvWRVqn5ZmAtYFhCsjJhMS5XrC7YeKLO3Es1uBuclyF/ClfMvKXatOuJum73GEgHl9rKJyMoGws+0EDmX0wfWLaPCBqCml6ftYwsDkol+dPChNz8PdSHNc1Zim73mEITHjCUe8HgBOjldibtL0/jXgM8nyUYRgVn4rwSLqJN3g+obfp6WdgHUlQ9NJrCccGSiytH0vIBzx2k440lf0qxpbdcLdNH2/nTCmrYdwmvU+wlGRIlpMuCBmF2EqGAgDbq8cts03k99vJ5ySaQaj9b2OcOR+8P39U+wCc5DmvR7ULMEL0vV9DUOfW0WeHqbcaL1PJ3xh3k7ofXnsAnPwA8KYtVcIRzNX0Br7NEmSJEmSJEmSJEmSJEmSJEmSJEmSJEmSJEmSJKmh/A9yuKUwUDQm9QAAAABJRU5ErkJggg==\n",
325 | "text/plain": [
326 | ""
327 | ]
328 | },
329 | "metadata": {},
330 | "output_type": "display_data"
331 | }
332 | ],
333 | "source": [
334 | "#Taylor decomposition explanation result\n",
335 | "color = ['red' if l == 0 else 'green' for l in y_rel1]\n",
336 | "\n",
337 | "plt.scatter(X_rel1[:,0], X_rel1[:,1], color=color)"
338 | ]
339 | },
340 | {
341 | "cell_type": "code",
342 | "execution_count": 14,
343 | "metadata": {},
344 | "outputs": [],
345 | "source": [
346 | "maps_x = []\n",
347 | "maps_y = []\n",
348 | "for i in range(200):\n",
349 | " output, model_rel = predict(model, X[i,:])\n",
350 | " y_rel = np.round(np.argmax(output, axis=1))\n",
351 | "# temp = np.asarray(np.max(output, axis=1)).reshape((1,1))\n",
352 | " temp = np.asarray([1.]).reshape((1,1))\n",
353 | " x = np.expand_dims(X[i,:], axis=0)\n",
354 | " temp = backward_simpleLRP_rel(temp,model_rel['a1'], model_rel['W2'], model_rel['z2'], epsilon=1e-4)\n",
355 | " temp = backward_simpleLRP_rel(temp,x, model_rel['W1'], model_rel['z1'], epsilon=1e-4)\n",
356 | " maps_x.append(temp)\n",
357 | " maps_y.append(y_rel)"
358 | ]
359 | },
360 | {
361 | "cell_type": "code",
362 | "execution_count": 15,
363 | "metadata": {},
364 | "outputs": [],
365 | "source": [
366 | "X_rel = np.squeeze(maps_x, axis=1)\n",
367 | "y_rel = np.squeeze(maps_y, axis=1)"
368 | ]
369 | },
370 | {
371 | "cell_type": "code",
372 | "execution_count": 16,
373 | "metadata": {},
374 | "outputs": [
375 | {
376 | "data": {
377 | "text/plain": [
378 | ""
379 | ]
380 | },
381 | "execution_count": 16,
382 | "metadata": {},
383 | "output_type": "execute_result"
384 | },
385 | {
386 | "data": {
387 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlkAAAHfCAYAAABj+c0fAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmYVNWd//F3dVXvIEtkURZBNC6IxgjigtqCIq4hceI+bhOjSUZNjEaIPxVN4sQtahZNoo7BdVyj4hgHUFsSZiJJFIOCiuKGyiY7Te/1++NW0wvd9FJ9urrp9+t56um6p6ru/XKU5lPnnnsuSJIkSZIkSZIkSZIkSZIkSZIkSZIkSZIkqRPoDTwBLAIWAmOBvsAs4F1gZuo9kiRJaoXpwPmp5wmgF3AT8KNU25XAzzNQlyRJUpfVC1jSSPvbwIDU84GpbUmSJLXQV4BXgfuA14C7gUJgTZ33xBpsS5IkbfcS7fD5rwL/DvwNuB2Y0uA9ydSjnhEjRiTff//9NA8vSZLUId4HdmvNB7LSPODS1ONvqe0niELXMqLThAA7ASsafvD9998nmUz6aOPj2muvzXgNXflh/9l/9l3XfNh/9l+mHsCI1oakdEPWMuAT4Mup7aOAt4AZwDmptnOAp9M8jiRJUpeS7ulCgIuBh4AcoqG084A48Bjwb8CHwCntcBxJkqQuoz1C1hvAmEbaj2qHfasJRUVFmS6hS7P/0mP/tZ19lx77Lz32X8eKZfDYydQ5TkmSpE4tFotBK3NTunOyJEmS1AhDliRJUgCGLEmSpAAMWZIkSQEYsiRJkgIwZEmSJAVgyJIkSQrAkCVJkhSAIUuSJCkAQ5YkSVIAhixJkqQADFmSJEkBGLIkSZICMGRJkiQFYMiSJEkKwJAlSZIUgCFLkiQpAEOWJElSAIYsSZKkAAxZkiRJARiyJEmSAjBkSZIkBWDIkiRJCsCQJUmSFIAhS5IkKQBDliRJUgCGLEmSpAAMWZIkSQEYsiRJkgIwZEmSJAVgyJIkSQrAkCVJkhSAIUuSJCkAQ5YkSVIAhixJkqQADFmSJEkBGLIkSZICMGRJkiQFkMh0AZKkOtasgTvvhBUr4Ljj4JhjMl2RpDaKZfDYyWQymcHDS1Ins24djBoFy5dDeTkUFMAvfgEXXpjpyqRuLxaLQStzk6cLJamz+K//gi++iAIWQEkJTJmS2ZoktZkhS5I6i02boLKyfltZWWZqkZQ2Q5YkdRbHHguJOlNl8/Jg8uTM1SMpLYYsSeos9toLnn8e9t4bBg6EM8+Ee+/NdFWS2siJ75IkSc1w4rskSVInYciSJEkKwJAlSZIUgCFLkiQpAEOWJElSAIYsSZKkAAxZkiRJARiyJEmSAjBkSZIkBWDIkiRJCsCQJUmSFIAhS5IkKQBDliRJUgCGLEmSpAAMWZIkSQEYsiRJkgIwZEmSJAVgyJIkSQrAkCVJkhSAIUuSJCmA9gpZceB1YEZquy8wC3gXmAn0bqfjSJIkdQntFbIuBRYCydT2FKKQ9WXgxdS2JElSt9EeIWswcBxwDxBLtZ0ETE89nw5MbofjSJIkdRntEbJuA64Aquu0DQCWp54vT21LkiR1G4k0P38CsIJoPlZRE+9JUnsasZ5p06ZteV5UVERRUVO7kCRJ6jjFxcUUFxentY9Y82/ZphuAfwUqgTxgB+ApYAxR6FoG7AS8DOzZ4LPJZLLR7CVJktSpxGIxaGVuSjdk1XUEcDlwInAT8AVwI9Gk995sPfndkCVJkrqEtoSs9l4nqyY1/Rw4mmgJh/GpbUmSpG6jPUeyWsuRLEmS1CV0hpEsSZIkYciSJEkKwpAlSZIUgCFLkiQpAEOWJElSAIYsSZKkAAxZkiRJARiyJEmSAjBkSZIkBWDIkiRJCsCQJUmSFIAhS5IkKQBDliRJUgCGLEmSpAAMWZIkSQEYsiRJkgIwZEmSJAVgyJIkSQrAkCVJkhSAIUuSJCkAQ5YkSVIAhixJkqQADFmSJEkBGLIkSZICMGRJkiQFYMiSJEkKwJAlSZIUgCFLkiQpAEOWJElSAIYsSZKkAAxZkiRJARiyJEmSAjBkSZIkBWDIkiRJCsCQJUmSFIAhS5IkKQBDliRJUgCGLEmSpAAMWZIkSQEYsiRJkgIwZEmSJAVgyJIkSQrAkCVJkhSAIUuSJCkAQ5YkSVIAhixJkqQADFmSJEkBGLIkqaM9/jgMHAiFhfCNb8DGjZmuSFIAsQweO5lMJjN4eEnKgHnzoKgINm+OtnNz4fjj4cknM1qWpG2LxWLQytzkSJYkdaRZs6C8vHa7rAxmzsxcPZKCMWRJUkfq0wdycuq39eiRmVokBWXIkqSOdPbZMHgw5OdDIhH9vPPOTFclKQDnZElSR9u4ER58ENauhaOOgtGjM12RpGa0ZU6WIUuSJKkZTnyXJEnqJAxZkiRJARiyJEmSAjBkSZIkBWDIkiRJCsCQJUmSFIAhS5IkKQBDliRJUgCGLEmSpAAMWZIkSQEYsiRJkgJIN2QNAV4G3gLeBC5JtfcFZgHvAjOB3mkeR5IkqUtJ9wbRA1OP+UAP4B/AZOA8YBVwE3Al0AeY0uCz3iBakiR1CZm4QfQyooAFsBFYBAwCTgKmp9qnEwUvSZKkbqM952QNA/YHXgUGAMtT7ctT25IkSd1Gop320wN4ErgU2NDgtWTqsZVp06ZteV5UVERRUVE7lSNJktR2xcXFFBcXp7WPdOdkAWQDzwF/Am5Ptb0NFBGdTtyJaHL8ng0+55wsSZLUJWRiTlYMuBdYSG3AAngWOCf1/Bzg6TSPI0mS1KWkO5I1DpgD/JPaU4JTgXnAY8BQ4EPgFGBtg886kiWpe1qzBhYuhIEDYcSITFcjqQXaMpLVHqcL28qQJan7mTsXjj0WsrKgrAwuuQRuvDHTVUlqhiFLkjqzZBL694dVq2rbCgvhhRdg3LjM1SWpWZmYkyVJaqmyMli9euv2d9/t+FokBWfIkqSOkpcXjWTVlUzCyJGZqUdSUIYsSepIM2ZAnz7Qsyfk5sLUqTB2bKarkhSAc7IkqaOVlMB770WjWgMHZroaSS3gxHdJ6io2bYLHH4f162HiRNiz4XrNkjoTQ5YkdQUbNsABB8Bnn0FVVbScw4wZMH58piuT1ASvLpSkruDuu+GTT6LRrNLS6PThRRdluipJ7cyQJUkdbfnyKFzV1djSDpK6NEOWJHW0iROhoKB2OzcXjjoqc/VICsKQJUkdbcIE+MUvomUcsrPhmGPgnnsyXZWkdubEd0mSuokla5bw4pIX6Znbk8l7TiYvkZfpkroMry6UpK5g8WKYNi26h+Fpp8G550Isk7+O1R385eO/MOnBSSRJkkUWQ3sP5W8X/I2C7ILmP6w2haxEmFIkSY36+GMYPRo2boTqapg7F1asgCuvzHRl2s5969lvsali05btJWuWcPc/7ubSgy7NYFXbN+dkSVJHeuSRaMmG6upoe9MmuOWWzNakbmFlycp626WVpXy24bMMVdM9GLIkqSNVVkY3ha6rqioztahbKdqliNx47pbtguwCxg93AdyQDFmS1JFOPRXy82u3Cwrge9/LXD3qNu6bfB+HDT2MeCxOXjyPn43/Gcfsdkymy9quOfFdkjraG2/AlCnRAqSnngo/+IET39VhKqsricfiNRO51UJeXShJkhSA9y6UJEnqJAxZkiRJARiyJEmSAnAxUklSrdWr4fe/hzVr4Pjj4fDDM12R1GU58V2StjfPPw8XXQTr1sHRR8Mf/gA9ejT/uTVrYNQoWLkSysuj5SXuvhvOOCN4yVJn59WFktTdvfEGHHJItKo8QG4uHHMMPPNM85+9/fZoaYmystq2nXaCz1wVXPLqQknq7mbPjlaVr1FWBjNntuyzGzZARUX9tpqwJqnVDFmStD3p1QsSDabbFha27LPHHx+NfNXIz4evf739apO6GUOWJHWkZDI6dffTn8Ljj299H8N0nX46DBoEeXnRKvL5+XDbbS377Fe/Ck88ASNGQL9+0Vysu+5q3/qkbsQ5WZLUkb73PZg+HTZvjgLQ5MnwwAPte1udjRujye6rVkUT3w89tP32LXVTTnyXpM5s6VLYfXcoLa1ty8+Hf/wD9torc3VJapYT3yWpM1u7FrKz67clElG7pO2OIUuSOsruu0drT9VVUgKDB2emHklBGbIkqaPk5m69gnpWFtxwQ2bqkRSUIUuSOtInn9TfrqiAt97KTC2SgjJkSVJH2bQJFi+u35abCwcdlJl6JAXl1YWS1BHWroWzz47uK1hVVdveqxd8/nl0laGkTqstVxcmmn+LJCktX3wR3Xj588+3fq1XLwOWtJ3ydKEkhfLFF3DWWTByZOMBKzvbhUKl7ZinCyUphPJy2HdfWLJk65su1xg+HF5/PRrNktSpuRipJHUWCxbAZ581HbASCbj2WgOWtB0zZElSCPE4VFdv+z0nn9wxtUjKCEOWJIUwalQ0Fysvr/HXKyujpRt++tP6VxvWWLoUTjkFRo+GK66AsrKw9Upqd87JkqRQSkrgJz+B225rOiQVFMAZZ8Ddd9e2rVsHe+wBq1ZFASw/HyZOhKef7pi6JW2lLXOyDFmSFNoDD0RrZDUlkYhCWFbq5MIf/wjnnAMbNtR/z9q1UFgYtlZJjXLiuyR1NqWl8PLLrftMVhO/mmOZ/F4sqbVcjFSSQjr5ZJg9u+nX8/KiUa7qarj0UrjvvihkZWVF62hVVESnFL/+9einpC7D04WSFMr69fClL0WT3LelX78obH3+ee178/Nh7NjoNOH48dHk94Tfi6VM8XShJHUm8XjL3rdyJXzySf0wtnlztIbWr38d3UT6nnuiOVmSugxHsiQppLPOgscea3pR0qbE43D88dGpxsrKaLtvX3jjjWh0bMGCaL95eXDuuTBoUJDyJUW8ulCSOpuqquh04Jo1Lf9MdjbssAP07w+LFtVvnzoVjjoKJk2KRrvicejZM7o9zy67tH/9kgBPF0pS5xOPw8aNrfvMSSfBW2/VX8IBotGwZcui+VklJZBMUlldyZKsday55SftV7OkdmHIkqTQdtih5e+NxeCww2DAADjxxGgCfI2Cgqht3ToA3u8Dwy+FfS+sZmDf+7j6pavbuXBJ6TBkSVJoo0e3/L2xGEyeHD2/7bZoCYj8fOjdG268EU44AU47DQoK+Map8FlP2JQD5VnV3PbX25j1/qwwfwZJrWbIkqSQSkthVguDT1YWXH117dyq3Fx44AFefuu/2f/nwxhY+lP2uXMfLhuzmo3fPpdF/aC6zm/x8qpy5i+b3/5/Bklt4qIrkhTStddGC402p1cv+OpX4aWXSH78Eeuuv4odBu3KguULOOHhEyipLAFg+ablvLXyLR7auT8DE0P4ZP0nW3aRE89hRN8Rof4kklrJqwslKaT8/Gg0q6HDD4e5c6OrD2Ox6FFdzUc7wIEXwuo8yEvkMXmfk3lowcMkaf735aCeg/jkB5/UXAUlqR15daEkdRJLVi/hgruOo6yikYAFcM018P/+X/Q8mYTqatbnwl1jYH0OJKrg6hdK+dl5D7H4jiTnvN78MT/d8ClrS12wVOosPF0oafsxb160QGdBAVxwAQwZ0vE1JJMsv2ka5Tdfz23rIKeqiffdfz9lTz9JDtFX480JGHMBfNgbDvkEnn8A8qprvzbf/gI8syeszW9ifylL1y+lT36f9vvzSGozR7IkbR9mzoSiIrj1VrjhBth3X/joo9btY/Vq+OKLttfw6acweTI7TrmePb+AHpWNn1tY3AdG7PQYO124iZLsqO36I2DxlyBeDc88AvnV9T87a1cobfC1OKcSvvdq9BMgHouz1457tb1+Se3KkCVp+3DFFdEK6BDNc9qwAe64o2WfLSuL1p/aaSfYeedomYTly6Nb2DS2Uvv69XDeebDPPvAv/xItEPrOO7DHHvDsszR2x8Jk6vFxT9jjYliSV8qaQuj9I+h1Jdx8CEydAy9Oh/xG7sCz6xqobpDYKrPgP2bDS3+AnOosZp89m0TcExRSZ+HfRknbh4arqldVbVm0s1nTpsGLL0J5ebQ9axYMHhxNWq+ogH/7t2hB0VGj4JRTYMKEKIBVVES3vZk7F0aMgE2bmjxEDKgEdvkB9b7eVmbD+myY/hScvAgKK6AsC6qAeJ257vsvg4o6n8uugjGfQs8KOGhNAWXjn4dhR7TszyupQxiyJG0fzjoLbrklut0MRAHp9NNrX1+/Hl56KbqKb8IE6NGj9rU//7l2FAxqw1bNbW1+85voZ2FhNDr297/Xvre6OhrJauY0Y81IVmPnD3Mq4YwFkEhCWSwKV1mNXEyYVwFl2ZBdDYd8DI89EbXH44kW3bqnvKqcnHhOs+9rrZKKEuZ+PJdYLMa4oePIS+S1+zGkrsiQJWn7cM010cjS9OnRIp433BDdSBmiuVJjxtQGkd69o6DUv3+0/eUvR5PmKxo5T1fXpk3w6quNv9bcZ4EXd6XRkFWZBZ/3gCEbIE4Utuoqz4qC1Z3Pw+Mj4a7/STBkPcQqKknGYpRTxctfWsuEqgqy49lb7f+1z1/jpEdO4rMNn7FjwY48fdrTHDLkkGbrbYnlG5cz9p6xrNm8hiRJBvQYwKvfepW++X3bZf9SV+Y6Wa1RWgqvvAKVldG9xVpzPzJJmXPGGfD449HfXYDsbDj7bLjnnmh75Uo48MBoNCqZjMJUO/9+qozBvt+BRf0bf73/Rlj0S+hTXv8XcxJYuCOMXFWnsV8/OPBAKv76v/wzbx0X/Usubw+Is3vf3bn56JspzClk9M6jSWQlKKkoYchtQ1i9efWWj++QuwMfXvphu1yFeOZTZ/LYW49RWR31bU5WDufvfz53nXBX2vuWOpO2rJPlSFZLrV0b/RJetizaLiyMvvlm4hJxSa2zZEltwIJo1GnJktrtfv3grbei04bJJLz5ZjSRvqGaRT4bBrB4PJoD1pSsLBYctQ+L+v+zybesKIRn94BzFmz92p+H1glZWVmw557w3HPs95u9WbRqDbAZyuH1Za9z/MPHkx3PZve+uzPnvDl8sOYDKqrqj7LFiLFw5UIOHXpo0zW30OIvFm8JWADl1eW888U7ae9X2h6EvLpwEvA2sBi4MuBxOsZ118HHH0dzNDZsiL75XnxxpquS1BJHHhnN0apRUBC11VVQAMccA5MmwTnnRKNdDe25ZzT3q7Cwdn85OTBxIgwdWv+9hx0W3eD54oupevghfvHtUdssMSsJo1ZEX5NrIlwSKEnAQUuhPCdBZc9Ckv36wX33AfD5hs+33s/mMnLWbmTRykX8bM7P6FfYj/Kq8nrvKa8qZ0CPAdusp6UOHXpovTlY+Yl8Dht6WLvsW+rqQoWsOPBroqC1N3A60LUXb1m8OLrMu0ZVFbz/fubqkdRy110XBahEInqceCJMndr0+/v1g298IwpeAHl5sP/+0RWF998Ps2fD738ftSUS0QhYz57wgx9Eyz/ccUc0teD734df/pJzsp/jwTcfavJwWbEsqrPgqgnw37vB+72j2+oAFFTCfisgXl7JX/ts4twTq1i9U3Sar19hv/o7SkaB7LNb4f5HSlm47J8M7DGQqYdNpSC7gILsAgqzC/numO+yW9/d0unRLW4YfwPjhowjN55LTjyHo3Y9iqsOv6pd9i11daFOFx4IvAd8mNr+L+BrwKJAxwvv8MPh5Zdrr1zKy4u+qUrq/HJy4I9/jCa+x2LRSFRzHnoIfvUr+N//hb32giuvrB3dOuggePrpaPmGmvsSLl4cTSmYMaPeblZuWslDC5oOWEN3GMrFYy/mvdXv8Tt+xyvD4Ad/hWteqT/5Iw6M+xj2n76KK0tGcMcDqxjeeziLVy+ufVMMVhZCbhUctxj6/rkCzoJrj7iWibtO5M0Vb7LHjntw+C6Ht6TXWiQ/O5+Z/zqTVSWriMVi7FiwY7vtW+rqQoWsQcAndbaXAmMDHatj/PCHMH8+PPFE9Ev60EOjy8UldR11l21oTjwejUR9//uNv/7aa/Vv/FxeHv2OaODJRU9u8zD7DdiPyw+5HIDvjfkef509nW/eciu5jUzxSgJzh0DRa2uZPn86owaMYs7HcyitjOrIroSRK1PlxGHNpi+Y8c4MJuw6gYOHHMzBQw6mOlnNyx+8zKqSVYwdPJahvYZufaBWisViW4+qSQoWsrrYZYMtEI/Dww/DXXdFpwr79KmdBCup+/nKV6LThDVBKycnupVPA5XVldHpwGR1o7v5YO0HW56PGjCKUWfewuDXbuXdX0anCmskgQ96wzdPiZZ0KJ3xbwAkshLkxnNJlJUzYH2SX/4pet+BF0BZwQJ46kz6Ffbj7xf8nV55vTjpkZN45aNXyCKLqmQVM06fwZHD689Pe3HJi9z7+r3kZ+dz2UGXMbL/yPT6SuqmQoWsT4G6l90NIRrNqmfatGlbnhcVFVFUVBSonHbUq1emK5CUSclkNKH90UejhUhzcqLHLrtE7Q2M2XlMkwEL4MQ9Ttz6EIN25uizP2P2/ZCXClobcmD8ObC+wTqfldWVVFJJNfD8g5BXGeP8b2SxJr+KqlgFlFdQVlXG9XOu59Ahh1L8YTGbKmpXpj/jqTP4/Ie1E+ifefsZTn/ydDZXbiZGjEfffJR5F8xj7357t66fpC6uuLiY4uLitPYRaigmAbwDTAA+A+YRTX6vOyer662TJUm/+x1cdln9+ZnXXBMt+ZCo/731mbef4YynzqCkoqTJ3VVcXUEiq/7nZrwzg1MeP4W8jaWMWg4jV8CMPeDT5r7jVcMuG2KU9u7J8uT6ei+d9OWTKBpWxJQXp9S72jAei1N5Te2Q2Vd++xXeWP7Glu0YMb59wLf57Qm/bebg0vatLetkhbq6sBL4d+B/gIXAo3TlSe+SVOMPf6gNWFC7SHGDgFVZXdlswIrH4lsFLIhGt/58/p/518Mv5o09duB3B8aaD1gAWfBRryQrkxvJjeduaS7ILuDI4Udy4KAD6x0vHouz74D6pzgbLveQJLllzpek1gm5TtafgD2A3YD/CHgcSeo4DSfPx2LR8g0NrN68ut4inY05eMjBTb42eufR/PK4X7L2yrWsuGIFM8+aSSIWr3MTxKZVU81OPXcikZUgkZXgtJGnccnYSzh06KHcMOEGsrOyyYnnsGufXXn6tKfrffai0RdRkF2wZTs/kc/5+5+/7QNKapS31ZGk1pg3L1rItKSEDbmQlV9A4Zy/wqj6i43eNPcmrpy97XWYb514K5cdfFmLD129dCl/OWIYJ51Sxbpctvkb/MIDLuSWibcQj8XJz86v91p5VTkbyjbQN79vzSmQLZLJJL+e92t+/4/fk5vI5fojr+e43Y9rcY3S9qotpwsNWZLUSqXz/87JT57CrPhHJLNinDbqdP7wtT8Qz4oD0SjWgFsGNDuSdfnBl3PzxJtbd/Azz2TZrD9y/ZjN3PcVKM1mq9/k+Ylo7apxQ8e1bt+SmuS9CyWpA0xZ9iAv5X1ORWU1JOGpRU8xqv8ofnTojwC4+7W7mw1YWWSx/077t/7gDzzAwHvv5c5XX+WOL+3J3QfnsHD1u6zYtII3lr9BfnY+N0y4wYAldQKOZElSK+3/2/2Zv7z+wqPH7nYsz5/5fJOvNzSgYACfX/75VqfrJHVOnenqQknabg3vM5x4LL5lOycrhxF9RmzZzk3kNvaxesqqywxY0nbOkCVJrXTHpDvYsWBHeub0pGdOTwbtMIjrjrxuy+vXHHENBYmCbewBdum1S+gyJWWYpwslqQ3Wla6j+MNi4llxxg8fX2/ZA4huTfO7f/yO7Hg239zrm0x5cQofrPmA3EQuiawEc86bwz7998lQ9ZJay6sLJamTqk5WM+/TeWwo28DonUfTJ79PpkuS1AqGLEmSpACc+C5JktRJGLIkolWu73v9Pg659xCOuv8o5n48N9MlSZK6OE8XSsBdf7+Ly2devuVmvgXZBcw5dw4H7HxAhiuTJHUGni6U2ui2/7ttS8ACKKko4d7X781gRZKkrs6QJQFZsfp/FWLEtmqTJKk1/FdEAqYeNrXeOkcF2QVcNPqiDFYkSerqnJMlpTy58Enuff1eCnMK+fG4H7ft5r2SpO2S62RJktRJvfzByzz9ztP0yevDd8d8l/6F/TNdklrBkCVJUif0yIJH+NaMb1FSUUIiK0Hf/L68+Z036VfYL9OlqYW8ulCSpE7oillXbLmCubK6krWla7lv/n0ZrkqhGbIkSQpsc+XmetsVVRVsKNuQoWrUUQxZkiQFdurIU8lP5G/Zzs/OZ/KekzNYkTpCItMFSJK0vbt90u0kshI8uehJeub05PZJt3tHiW7Aie+SJEnNcOK7JElSJ2HIkiRJCsCQJUmSFIAhS5IkKQBDliRJUgCGLEmSpAAMWZIkSQEYsiRJkgJwxXdJagfPvfscLy55kZ177sx3xnyHHjk9Ml2SpAxzxXdJStNNc2/iuleuo6SihLxEHsN6D+O1b79GfnZ+8x+W1CW44rskdbBkMsnVL19NSUUJAKWVpSxdv5Rn33k2w5VJyjRDliSloSpZRVV1Vb226mQ1G8s3ZqgiSZ2FIUuS0pDISjB++Hhy47lb2mLEmLDrhAxWJakzMGRJUpqePOVJJu85mf6F/dmn3z7MPns2w3oPy3RZkjLMie+SJEnNcOK7JHVC76x6h/1+ux+FNxSy71378vaqtzNdkqQO4EiWJAW0uWIzw+4YxspNK0mSJEaMHQt25MPvf0hBdkGmy5PUQo5kSVIns2jVIkorSkkSfalMkqSsqoxFKxdluDJJoRmyJCmgPnl9KK8ur9dWXlVO77zeGapIUkcxZElSQMP7DOeMUWdQmF1IFlkUZhdy2sjTGNF3RKZLkxSYc7IkqRX+tPhPXPjchawpXcP44eO5f/L99Mrrtc3PJJNJnlr0FAtXLmSvfntx8l4n18zvkNRFtGVOliFLklrorRVvMebuMWyu3AxAbjyX8cPH8/yZz2e4MkmhOfFdkgKavWQ21cnqLdtlVWXMWjIrgxVJ6swMWZLUQr3zepPIStRrK8wuzFA1kjo7Q5YktdApI09hl967kJ/IJ4ssCrILuOPYOzJdlqROyjlZktQKJRUlTJ9IzVt3AAAK30lEQVQ/nZUlKxk/fDzjho7LdEmSOoAT3yVJkgJw4rskSVInYciSJEkKwJAlSZIUgCFLkiQpAEOWJElSAIYsSZKkAAxZkiRJARiyJEmSAjBkSZIkBWDIkiRJCsCQJUmSFIAhS5IkKQBDliRJUgCGLEmSpAAMWZIkSQEYsiRJkgJIJ2TdDCwC3gCeAnrVeW0qsBh4G5iYxjEkSZK6pHRC1kxgJLAf8C5RsALYGzg19XMScGeax5EkSepy0gk/s4Dq1PNXgcGp518DHgEqgA+B94AD0ziOJElSl9NeI0znA8+nnu8MLK3z2lJgUDsdR5IkqUtINPP6LGBgI+0/Bmaknl8FlAMPb2M/ycYap02btuV5UVERRUVFzZQjSZIUXnFxMcXFxWntI5ZmDecCFwATgNJU25TUz5+nfr4AXEt0SrGuZDLZaPaSJEnqVGKxGLQyN6VzunAScAXRHKzSOu3PAqcBOcBwYHdgXhrHkSRJ6nKaO124Lb8iClKzUtv/B3wXWAg8lvpZmWpzyEqSJHUr6Z4uTIenCyVJUpfQ0acLJUmS1ARDliRJUgCGLEmSpAAMWZIkSQEYsiRJkgIwZEmSJAVgyJIkSQrAkCVJkhSAIUuSJCkAQ5YkSVIAhixJkqQADFmSJEkBGLIkSZICMGRJkiQFYMiSJEkKwJAlSZIUgCFLkiQpAEOWJElSAIYsSZKkAAxZkiRJARiyJEmSAjBkSZIkBWDIkiRJCsCQJUmSFIAhS5IkKQBDliRJUgCGLEmSpAAMWZIkSQEYsiRJkgIwZEmSJAVgyJIkSQrAkCVJkhSAIUuSJCkAQ5YkSVIAhixJkqQADFmSJEkBGLIkSZICMGRJkiQFYMiSJEkKwJAlSZIUgCFLkiQpAEOWJElSAIYsSZKkAAxZkiRJARiyJEmSAjBkSZIkBWDIkiRJCsCQJUmSFIAhS5IkKQBDliRJUgCGLEmSpAAMWZIkSQEYsiRJkgIwZEmSJAVgyJIkSQrAkCVJkhSAIUuSJCkAQ5YkSVIAiUwXIEndwYLlC/jN335DRVUFFxxwAQcNPijTJUkKLJbBYyeTyWQGDy9JHWP+svmM+89xbKrYBEBBooDnzniOI4cfmeHKJLVULBaDVuYmTxdKUmA3/uXGLQELoKSyhGmvTMtcQZI6hCFLkgIrqSzZqq20ojQDlUjqSIYsSQrsgq9eQEF2wZbtguwCvn3AtzNYkaSO4JwsSeoAj775KD+Z8xOqklVcMvYSLjrgopo5HpK6gLbMyTJkSZIkNSNTE99/CFQDfeu0TQUWA28DE9vhGJIkSV1KuutkDQGOBj6q07Y3cGrq5yBgNvBloiAmSZLULaQ7kvUL4EcN2r4GPAJUAB8C7wEHpnkcSZKkLiWdkPU1YCnwzwbtO6faaywlGtGSJEnqNpo7XTgLGNhI+1VE867qzrfa1mSwRme4T5s2bcvzoqIiioqKmilHkiQpvOLiYoqLi9PaR1uvLtwHeBGoWWFvMPApMBY4L9X289TPF4BrgVcb7MOrCyVJUpeQySUcPgAOAFYTTXh/mGgeVs3E993YejTLkCVJkrqEtoSsdK8urFE3LS0EHkv9rAS+SxOnCyVJkrZXLkYqSZLUjEwtRipJkqQGDFmSJEkBGLIkSZICMGRJkiQFYMiSJEkKwJAlSZIUgCFLkiQpAEOWJElSAIYsSZKkAAxZkiRJARiyJEmSAjBkSZIkBWDIkiRJCsCQJUmSFIAhS5IkKQBDliRJUgCGLEmSpAAMWZIkSQEYsiRJkgIwZEmSJAVgyJIkSQrAkCVJkhSAIUuSJCkAQ5YkSVIAhixJkqQADFmSJEkBGLIkSZICMGRJkiQFYMiSJEkKwJAlSZIUgCFLkiQpAEOWJElSAIYsSZKkAAxZkiRJARiyJEmSAjBkSZIkBWDIkiRJCsCQJUmSFIAhS5IkKQBDliRJUgCGLEmSpAAMWZIkSQEYsiRJkgIwZEmSJAVgyJIkSQrAkCVJkhSAIUuSJCkAQ5YkSVIAhixJkqQADFmSJEkBGLIkSZICMGRJkiQFYMiSJEkKwJAlSZIUgCFLkiQpAEOWJElSAIYsSZKkAAxZkiRJARiyJEmSAjBkSZIkBWDIkiRJCsCQJUmSFIAhS5IkKQBDliRJUgCGLEmSpAAMWZIkSQGkG7IuBhYBbwI31mmfCiwG3gYmpnkMNaK4uDjTJXRp9l967L+2s+/SY/+lx/7rWOmErCOBk4B9gX2AW1LtewOnpn5OAu5M8zhqhH9R0mP/pcf+azv7Lj32X3rsv46VTvj5DvAfQEVqe2Xq59eAR1LtHwLvAQemcRxJkqQuJ52QtTtwOPBXoBgYnWrfGVha531LgUFpHEeSJKnLiTXz+ixgYCPtVwE/A14CLgXGAI8CuwK/IgpeD6Xeew/wPPBUg328B4xoU9WSJEkd631gt9Z8INHM60dv47XvUBuc/gZUAzsCnwJD6rxvcKqtoVYVKkmS1F1cCFyXev5l4OPU872B+UAOMJwo+TU3YiZJkqSUbOABYAHwD6Cozms/Jjod+DZwTIdXJkmSJEmSJLWHA4F5wOtEc7nG1HnNRUxbxkVg0/NDojmEfeu02XfNu5no/7s3iOZj9qrzmv3XMpOI+mgxcGWGa+nshgAvA28R/a67JNXel+iirHeBmUDvjFTXdcSJ/r2dkdq2/1quN/AE0e+9hcBYukD/FVN7CvFYor9EUDuXKxsYRnS60UVMt3Yk0X/g7NR2v9RP+69lhgAvAB9QG7Lsu5Y5mtp++XnqAfZfS8WJ+mYYUV/NB/bKZEGd3EDgK6nnPYB3iPrrJuBHqfYrqf3/UI27jOhq/2dT2/Zfy00Hzk89TxB9sez0/fcIcErq+enAg6nnU6n/ze4F4KAOrKureAwY30i7/dcyjxPdpaBuyLLvWu/r+He3tQ4m6psaU1IPtczTwFFEI4EDUm0DU9tq3GBgNtGX85qRLPuvZXoBSxppb1X/ZeLb5hTgVqKrEW8m+gUNLmLaUi4C23ZfI+qXfzZot+9a73yi9e/A/mupQcAndbbtp5YbBuwPvEr0D9zyVPtyav/B09ZuA64gmh5Rw/5rmeFEd7K5D3gNuBsopJX919w6WW21rUVML0k9/gh8E/hPml6PKxmkus5vW/2XAPoQjRSMIRrZ2rWJ/XTH/ttW302l/nyhbS0t0h37Dpruvx9T+034KqAceHgb++mu/bct9knb9ACeJFr4ekOD15LYr005AVhBNB+rqIn32H9NSwBfBf6daP747Ww98txs/4UKWdtaxPRBoiFfiCaU3ZN63tJFTLuDkIvAbu+a6rt9iL6ZvJHaHky09MhY7Lu6tvX/HsC5wHHAhDpt9l/LNOynIdQfAdTWsokC1gNEpwshGj0YCCwDdiIKEtraIcBJRH9f84AdiPrR/muZpanH31LbTxB9UV9GJ++/14AjUs8nUPsHcBHTlnER2PbR2MR3+27bJhFd6bVjg3b7r2USRH0zjKivnPi+bTHgfqJTXnXdRO0cwCl0wonHndAR1I5E238tN4fo31mAaUR91+n7bzTRefX5wP8RnWev4SKmzXMR2PaxhPpLONh3zVsMfER0+uF14M46r9l/LXMs0VVy71E7H1WNG0c0Uj+f2v/nJhH9vZ1NJ76EvhM6gtqrC+2/ltuPaCCo7rI19p8kSZIkSZIkSZIkSZIkSZIkSZIkSZIkSZIkSZK6vP8PLU/GhHCW7JkAAAAASUVORK5CYII=\n",
388 | "text/plain": [
389 | ""
390 | ]
391 | },
392 | "metadata": {},
393 | "output_type": "display_data"
394 | }
395 | ],
396 | "source": [
397 | "#Simple LRP explanation result\n",
398 | "\n",
399 | "color1 = ['red' if l == 0 else 'green' for l in y_rel]\n",
400 | "\n",
401 | "plt.scatter(X_rel[:,0], X_rel[:,1], color=color1)"
402 | ]
403 | },
404 | {
405 | "cell_type": "markdown",
406 | "metadata": {},
407 | "source": [
408 | "Reference:\n",
409 | "https://github.com/dennybritz/nn-from-scratch/blob/master/nn-from-scratch.ipynb"
410 | ]
411 | },
412 | {
413 | "cell_type": "code",
414 | "execution_count": null,
415 | "metadata": {},
416 | "outputs": [],
417 | "source": []
418 | },
419 | {
420 | "cell_type": "code",
421 | "execution_count": null,
422 | "metadata": {},
423 | "outputs": [],
424 | "source": []
425 | }
426 | ],
427 | "metadata": {
428 | "kernelspec": {
429 | "display_name": "Python 3",
430 | "language": "python",
431 | "name": "python3"
432 | },
433 | "language_info": {
434 | "codemirror_mode": {
435 | "name": "ipython",
436 | "version": 2
437 | },
438 | "file_extension": ".py",
439 | "mimetype": "text/x-python",
440 | "name": "python",
441 | "nbconvert_exporter": "python",
442 | "pygments_lexer": "ipython2",
443 | "version": "2.7.6"
444 | }
445 | },
446 | "nbformat": 4,
447 | "nbformat_minor": 2
448 | }
449 |
--------------------------------------------------------------------------------
/Basic-LRP/Taylor+decomposition++for+the+explanation+of+non-linear+classification+desition.py:
--------------------------------------------------------------------------------
1 |
2 | # coding: utf-8
3 |
4 | # In[1]:
5 |
6 |
7 | #Reference: https://github.com/dennybritz/nn-from-scratch/blob/master/nn-from-scratch.ipynb
8 |
9 | import matplotlib.pyplot as plt
10 | import numpy as np
11 | import sklearn
12 | import sklearn.datasets
13 | import sklearn.linear_model
14 | import matplotlib
15 | # Display plots inline and change default figure size
16 | # get_ipython().magic('matplotlib inline')
17 | plt.rcParams['figure.figsize'] = (10.0, 8.0)
18 |
19 |
20 | # In[2]:
21 |
22 |
23 | # Generate a dataset and plot it
24 | np.random.seed(0)
25 | X, y = sklearn.datasets.make_moons(200, noise=0.20)
26 | color0 = ['red' if l == 0 else 'green' for l in y]
27 | plt.scatter(X[:,0], X[:,1], color=color0)
28 | plt.savefig('graph1.png')
29 | plt.close()
30 | # plt.show()
31 |
32 | # In[3]:
33 |
34 |
35 | # ipdb.set_trace()
36 | num_examples = len(X) # training set size
37 | nn_input_dim = 2 # input layer dimensionality
38 | nn_output_dim = 2 # output layer dimensionality
39 |
40 | # Gradient descent parameters (I picked these by hand)
41 | epsilon = 0.01 # learning rate for gradient descent
42 | reg_lambda = 0.01 # regularization strength
43 |
44 |
45 | # In[4]:
46 |
47 |
48 | # Helper function to evaluate the total loss on the dataset
49 | def calculate_loss(model):
50 | W1, b1, W2, b2 = model['W1'], model['b1'], model['W2'], model['b2']
51 | # Forward propagation to calculate our predictions
52 | z1 = X.dot(W1) + b1
53 | a1 = np.tanh(z1)
54 | z2 = a1.dot(W2) + b2
55 | exp_scores = np.exp(z2)
56 | probs = exp_scores / np.sum(exp_scores, axis=1, keepdims=True)
57 |
58 | probs_new = probs[range(num_examples), y]
59 |
60 | corect_logprobs = -np.log(probs[range(num_examples), y])
61 |
62 | data_loss = np.sum(corect_logprobs)
63 |
64 | data_loss += reg_lambda/2 * (np.sum(np.square(W1)) + np.sum(np.square(W2)))
65 | return 1./num_examples * data_loss
66 |
67 |
68 | # In[5]:
69 |
70 |
71 | # Helper function to predict an output (0 or 1)
72 | def predict(model, x):
73 | model_rel = {}
74 | W1, b1, W2, b2 = model['W1'], model['b1'], model['W2'], model['b2']
75 | # Forward propagation
76 | z1 = x.dot(W1) + b1
77 | a1 = np.tanh(z1)
78 | z2 = a1.dot(W2) + b2
79 | exp_scores = np.exp(z2)
80 | probs = exp_scores / np.sum(exp_scores, axis=1, keepdims=True)
81 | model_rel = {'W1': W1, 'b1': b1, 'W2': W2, 'b2': b2, 'z1':z1, 'a1': a1, 'z2': z2}
82 | return probs, model_rel
83 |
84 |
85 | # In[6]:
86 |
87 |
88 | # This function learns parameters for the neural network and returns the model.
89 | # - nn_hdim: Number of nodes in the hidden layer
90 | # - num_passes: Number of passes through the training data for gradient descent
91 | # - print_loss: If True, print the loss every 1000 iterations
92 | def build_model(nn_hdim, num_passes=20000, print_loss=False):
93 |
94 | # Initialize the parameters to random values. We need to learn these.
95 | np.random.seed(0)
96 | W1 = np.random.randn(nn_input_dim, nn_hdim) / np.sqrt(nn_input_dim)
97 | b1 = np.zeros((1, nn_hdim))
98 | W2 = np.random.randn(nn_hdim, nn_output_dim) / np.sqrt(nn_hdim)
99 | b2 = np.zeros((1, nn_output_dim))
100 |
101 | # This is what we return at the end
102 | model = {}
103 |
104 | # Gradient descent. For each batch...
105 | for i in range(0, num_passes):
106 |
107 | # Forward propagation
108 | z1 = X.dot(W1) + b1
109 | a1 = np.tanh(z1)
110 | z2 = a1.dot(W2) + b2
111 | exp_scores = np.exp(z2)
112 | probs = exp_scores / np.sum(exp_scores, axis=1, keepdims=True)
113 |
114 | # Backpropagation
115 | delta3 = probs
116 | delta3[range(num_examples), y] -= 1
117 |
118 | dW2 = (a1.T).dot(delta3)
119 | db2 = np.sum(delta3, axis=0, keepdims=True)
120 | delta2 = delta3.dot(W2.T) * (1 - np.power(a1, 2))
121 | dW1 = np.dot(X.T, delta2)
122 | db1 = np.sum(delta2, axis=0)
123 |
124 | # Add regularization terms (b1 and b2 don't have regularization terms)
125 | dW2 += reg_lambda * W2
126 | dW1 += reg_lambda * W1
127 |
128 | # Gradient descent parameter update
129 | W1 += -epsilon * dW1
130 | b1 += -epsilon * db1
131 | W2 += -epsilon * dW2
132 | b2 += -epsilon * db2
133 |
134 | # Assign new parameters to the model
135 | model = { 'W1': W1, 'b1': b1, 'W2': W2, 'b2': b2}
136 |
137 | # Optionally print the loss.
138 | # This is expensive because it uses the whole dataset, so we don't want to do it too often.
139 | if print_loss and i % 1000 == 0:
140 | print ("Loss after iteration %i: %f" %(i, calculate_loss(model)))
141 |
142 | return model
143 |
144 |
145 | # In[7]:
146 |
147 |
148 | # Build a model with a 3-dimensional hidden layer
149 | model = build_model(3, print_loss=True)
150 |
151 | prediction, model_rel = predict(model, X)
152 |
153 |
154 | # In[8]:
155 |
156 |
157 | def backward_simpleLRP_rel(top_rel, inputs, weights, outputs, epsilon=1e-4):
158 | return np.sum((inputs.T.dot(top_rel) * weights) / (outputs + epsilon),
159 | axis=1,
160 | keepdims=True).T
161 |
162 |
163 | # In[9]:
164 |
165 |
166 | def backprop_taylor_rel(inputs, weights, top_rel, lowest=-1.5, highest=2.5):
167 | w_p = np.maximum(np.zeros_like(weights), weights)
168 | w_n = np.minimum(np.zeros_like(weights), weights)
169 |
170 | L = np.ones_like(inputs) * lowest
171 | H = np.ones_like(inputs) * highest
172 |
173 | z_o = inputs.dot(weights)
174 | z_p = L.dot(w_p)
175 | z_n = H.dot(w_n)
176 |
177 | z = z_o - z_p - z_n + 1e-10
178 | s = top_rel / z
179 |
180 | c_o = s.dot(weights.T)
181 | c_p = s.dot(w_p.T)
182 | c_n = s.dot(w_n.T)
183 |
184 | return inputs * c_o - L * c_p + H * c_n
185 |
186 |
187 |
188 | # In[10]:
189 |
190 |
191 | output, model_rel = predict(model, X)
192 | output_ = np.round(np.argmax(output, axis=1))
193 |
194 |
195 | # In[11]:
196 |
197 |
198 | maps1_x = []
199 | maps1_y = []
200 | for i in range(200):
201 | output, model_rel = predict(model, X[i,:])
202 | y_rel = np.round(np.argmax(output, axis=1))
203 | temp = np.asarray([1.]).reshape((1,1))
204 | x = np.expand_dims(X[i,:], axis=0)
205 | temp1 = backprop_taylor_rel(model_rel['a1'], model_rel['W2'], temp)
206 | temp1 = backprop_taylor_rel(x, model_rel['W1'], temp1)
207 | maps1_x.append(temp1)
208 | maps1_y.append(y_rel)
209 |
210 |
211 | # In[12]:
212 |
213 |
214 | X_rel1 = np.squeeze(maps1_x, axis=1)
215 | y_rel1 = np.squeeze(maps1_y, axis=1)
216 |
217 |
218 | # In[13]:
219 |
220 |
221 | #Taylor decomposition explanation result
222 | color = ['red' if l == 0 else 'green' for l in y_rel1]
223 |
224 | plt.scatter(X_rel1[:,0], X_rel1[:,1], color=color)
225 | plt.savefig('graph2.png')
226 | plt.close()
227 |
228 | # plt.show()
229 |
230 | # In[14]:
231 |
232 |
233 | maps_x = []
234 | maps_y = []
235 | for i in range(200):
236 | output, model_rel = predict(model, X[i,:])
237 | y_rel = np.round(np.argmax(output, axis=1))
238 | # temp = np.asarray(np.max(output, axis=1)).reshape((1,1))
239 | temp = np.asarray([1.]).reshape((1,1))
240 | x = np.expand_dims(X[i,:], axis=0)
241 | temp = backward_simpleLRP_rel(temp,model_rel['a1'], model_rel['W2'], model_rel['z2'], epsilon=1e-4)
242 | temp = backward_simpleLRP_rel(temp,x, model_rel['W1'], model_rel['z1'], epsilon=1e-4)
243 | maps_x.append(temp)
244 | maps_y.append(y_rel)
245 |
246 |
247 | # In[15]:
248 |
249 |
250 | X_rel = np.squeeze(maps_x, axis=1)
251 | y_rel = np.squeeze(maps_y, axis=1)
252 |
253 |
254 | # In[16]:
255 |
256 |
257 | #Simple LRP explanation result
258 |
259 | color1 = ['red' if l == 0 else 'green' for l in y_rel]
260 |
261 | plt.scatter(X_rel[:,0], X_rel[:,1], color=color1)
262 | plt.savefig('graph3.png')
263 | plt.close()
264 |
265 | # plt.show()
266 |
267 | # Reference:
268 | # https://github.com/dennybritz/nn-from-scratch/blob/master/nn-from-scratch.ipynb
269 |
--------------------------------------------------------------------------------
/Basic-LRP/result.JPG:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/OpenXAIProject/Tutorials/58dbcb5650a44c2ef7f9557dea098fb5708fde3b/Basic-LRP/result.JPG
--------------------------------------------------------------------------------
/LICENSE:
--------------------------------------------------------------------------------
1 | Apache License
2 | Version 2.0, January 2004
3 | http://www.apache.org/licenses/
4 |
5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
6 |
7 | 1. Definitions.
8 |
9 | "License" shall mean the terms and conditions for use, reproduction,
10 | and distribution as defined by Sections 1 through 9 of this document.
11 |
12 | "Licensor" shall mean the copyright owner or entity authorized by
13 | the copyright owner that is granting the License.
14 |
15 | "Legal Entity" shall mean the union of the acting entity and all
16 | other entities that control, are controlled by, or are under common
17 | control with that entity. For the purposes of this definition,
18 | "control" means (i) the power, direct or indirect, to cause the
19 | direction or management of such entity, whether by contract or
20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the
21 | outstanding shares, or (iii) beneficial ownership of such entity.
22 |
23 | "You" (or "Your") shall mean an individual or Legal Entity
24 | exercising permissions granted by this License.
25 |
26 | "Source" form shall mean the preferred form for making modifications,
27 | including but not limited to software source code, documentation
28 | source, and configuration files.
29 |
30 | "Object" form shall mean any form resulting from mechanical
31 | transformation or translation of a Source form, including but
32 | not limited to compiled object code, generated documentation,
33 | and conversions to other media types.
34 |
35 | "Work" shall mean the work of authorship, whether in Source or
36 | Object form, made available under the License, as indicated by a
37 | copyright notice that is included in or attached to the work
38 | (an example is provided in the Appendix below).
39 |
40 | "Derivative Works" shall mean any work, whether in Source or Object
41 | form, that is based on (or derived from) the Work and for which the
42 | editorial revisions, annotations, elaborations, or other modifications
43 | represent, as a whole, an original work of authorship. For the purposes
44 | of this License, Derivative Works shall not include works that remain
45 | separable from, or merely link (or bind by name) to the interfaces of,
46 | the Work and Derivative Works thereof.
47 |
48 | "Contribution" shall mean any work of authorship, including
49 | the original version of the Work and any modifications or additions
50 | to that Work or Derivative Works thereof, that is intentionally
51 | submitted to Licensor for inclusion in the Work by the copyright owner
52 | or by an individual or Legal Entity authorized to submit on behalf of
53 | the copyright owner. For the purposes of this definition, "submitted"
54 | means any form of electronic, verbal, or written communication sent
55 | to the Licensor or its representatives, including but not limited to
56 | communication on electronic mailing lists, source code control systems,
57 | and issue tracking systems that are managed by, or on behalf of, the
58 | Licensor for the purpose of discussing and improving the Work, but
59 | excluding communication that is conspicuously marked or otherwise
60 | designated in writing by the copyright owner as "Not a Contribution."
61 |
62 | "Contributor" shall mean Licensor and any individual or Legal Entity
63 | on behalf of whom a Contribution has been received by Licensor and
64 | subsequently incorporated within the Work.
65 |
66 | 2. Grant of Copyright License. Subject to the terms and conditions of
67 | this License, each Contributor hereby grants to You a perpetual,
68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable
69 | copyright license to reproduce, prepare Derivative Works of,
70 | publicly display, publicly perform, sublicense, and distribute the
71 | Work and such Derivative Works in Source or Object form.
72 |
73 | 3. Grant of Patent License. Subject to the terms and conditions of
74 | this License, each Contributor hereby grants to You a perpetual,
75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable
76 | (except as stated in this section) patent license to make, have made,
77 | use, offer to sell, sell, import, and otherwise transfer the Work,
78 | where such license applies only to those patent claims licensable
79 | by such Contributor that are necessarily infringed by their
80 | Contribution(s) alone or by combination of their Contribution(s)
81 | with the Work to which such Contribution(s) was submitted. If You
82 | institute patent litigation against any entity (including a
83 | cross-claim or counterclaim in a lawsuit) alleging that the Work
84 | or a Contribution incorporated within the Work constitutes direct
85 | or contributory patent infringement, then any patent licenses
86 | granted to You under this License for that Work shall terminate
87 | as of the date such litigation is filed.
88 |
89 | 4. Redistribution. You may reproduce and distribute copies of the
90 | Work or Derivative Works thereof in any medium, with or without
91 | modifications, and in Source or Object form, provided that You
92 | meet the following conditions:
93 |
94 | (a) You must give any other recipients of the Work or
95 | Derivative Works a copy of this License; and
96 |
97 | (b) You must cause any modified files to carry prominent notices
98 | stating that You changed the files; and
99 |
100 | (c) You must retain, in the Source form of any Derivative Works
101 | that You distribute, all copyright, patent, trademark, and
102 | attribution notices from the Source form of the Work,
103 | excluding those notices that do not pertain to any part of
104 | the Derivative Works; and
105 |
106 | (d) If the Work includes a "NOTICE" text file as part of its
107 | distribution, then any Derivative Works that You distribute must
108 | include a readable copy of the attribution notices contained
109 | within such NOTICE file, excluding those notices that do not
110 | pertain to any part of the Derivative Works, in at least one
111 | of the following places: within a NOTICE text file distributed
112 | as part of the Derivative Works; within the Source form or
113 | documentation, if provided along with the Derivative Works; or,
114 | within a display generated by the Derivative Works, if and
115 | wherever such third-party notices normally appear. The contents
116 | of the NOTICE file are for informational purposes only and
117 | do not modify the License. You may add Your own attribution
118 | notices within Derivative Works that You distribute, alongside
119 | or as an addendum to the NOTICE text from the Work, provided
120 | that such additional attribution notices cannot be construed
121 | as modifying the License.
122 |
123 | You may add Your own copyright statement to Your modifications and
124 | may provide additional or different license terms and conditions
125 | for use, reproduction, or distribution of Your modifications, or
126 | for any such Derivative Works as a whole, provided Your use,
127 | reproduction, and distribution of the Work otherwise complies with
128 | the conditions stated in this License.
129 |
130 | 5. Submission of Contributions. Unless You explicitly state otherwise,
131 | any Contribution intentionally submitted for inclusion in the Work
132 | by You to the Licensor shall be under the terms and conditions of
133 | this License, without any additional terms or conditions.
134 | Notwithstanding the above, nothing herein shall supersede or modify
135 | the terms of any separate license agreement you may have executed
136 | with Licensor regarding such Contributions.
137 |
138 | 6. Trademarks. This License does not grant permission to use the trade
139 | names, trademarks, service marks, or product names of the Licensor,
140 | except as required for reasonable and customary use in describing the
141 | origin of the Work and reproducing the content of the NOTICE file.
142 |
143 | 7. Disclaimer of Warranty. Unless required by applicable law or
144 | agreed to in writing, Licensor provides the Work (and each
145 | Contributor provides its Contributions) on an "AS IS" BASIS,
146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
147 | implied, including, without limitation, any warranties or conditions
148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
149 | PARTICULAR PURPOSE. You are solely responsible for determining the
150 | appropriateness of using or redistributing the Work and assume any
151 | risks associated with Your exercise of permissions under this License.
152 |
153 | 8. Limitation of Liability. In no event and under no legal theory,
154 | whether in tort (including negligence), contract, or otherwise,
155 | unless required by applicable law (such as deliberate and grossly
156 | negligent acts) or agreed to in writing, shall any Contributor be
157 | liable to You for damages, including any direct, indirect, special,
158 | incidental, or consequential damages of any character arising as a
159 | result of this License or out of the use or inability to use the
160 | Work (including but not limited to damages for loss of goodwill,
161 | work stoppage, computer failure or malfunction, or any and all
162 | other commercial damages or losses), even if such Contributor
163 | has been advised of the possibility of such damages.
164 |
165 | 9. Accepting Warranty or Additional Liability. While redistributing
166 | the Work or Derivative Works thereof, You may choose to offer,
167 | and charge a fee for, acceptance of support, warranty, indemnity,
168 | or other liability obligations and/or rights consistent with this
169 | License. However, in accepting such obligations, You may act only
170 | on Your own behalf and on Your sole responsibility, not on behalf
171 | of any other Contributor, and only if You agree to indemnify,
172 | defend, and hold each Contributor harmless for any liability
173 | incurred by, or claims asserted against, such Contributor by reason
174 | of your accepting any such warranty or additional liability.
175 |
176 | END OF TERMS AND CONDITIONS
177 |
178 | APPENDIX: How to apply the Apache License to your work.
179 |
180 | To apply the Apache License to your work, attach the following
181 | boilerplate notice, with the fields enclosed by brackets "[]"
182 | replaced with your own identifying information. (Don't include
183 | the brackets!) The text should be enclosed in the appropriate
184 | comment syntax for the file format. We also recommend that a
185 | file or class name and description of purpose be included on the
186 | same "printed page" as the copyright notice for easier
187 | identification within third-party archives.
188 |
189 | Copyright [yyyy] [name of copyright owner]
190 |
191 | Licensed under the Apache License, Version 2.0 (the "License");
192 | you may not use this file except in compliance with the License.
193 | You may obtain a copy of the License at
194 |
195 | http://www.apache.org/licenses/LICENSE-2.0
196 |
197 | Unless required by applicable law or agreed to in writing, software
198 | distributed under the License is distributed on an "AS IS" BASIS,
199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
200 | See the License for the specific language governing permissions and
201 | limitations under the License.
202 |
--------------------------------------------------------------------------------
/LRP-Time-Series/AUTHOR.txt:
--------------------------------------------------------------------------------
1 | Copyright 2018 UNIST under XAI Project supported by Ministry of Science and ICT, Korea
2 |
3 | # This is the list of UNIST Xie Qin Cho for copyright purposes.
4 | # This does not necessarily list everyone who has contributed code, since in
5 | # some cases, their employer may be the copyright holder. To see the full list
6 | # of contributors, see the revision history in source control
7 |
--------------------------------------------------------------------------------
/LRP-Time-Series/LICENSE:
--------------------------------------------------------------------------------
1 | Copyright 2018 UNIST under XAI Project supported by Ministry of Science and ICT, Korea
2 |
3 | Apache License
4 | Version 2.0, January 2004
5 | http://www.apache.org/licenses/
6 |
7 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
8 |
9 | 1. Definitions.
10 |
11 | "License" shall mean the terms and conditions for use, reproduction,
12 | and distribution as defined by Sections 1 through 9 of this document.
13 |
14 | "Licensor" shall mean the copyright owner or entity authorized by
15 | the copyright owner that is granting the License.
16 |
17 | "Legal Entity" shall mean the union of the acting entity and all
18 | other entities that control, are controlled by, or are under common
19 | control with that entity. For the purposes of this definition,
20 | "control" means (i) the power, direct or indirect, to cause the
21 | direction or management of such entity, whether by contract or
22 | otherwise, or (ii) ownership of fifty percent (50%) or more of the
23 | outstanding shares, or (iii) beneficial ownership of such entity.
24 |
25 | "You" (or "Your") shall mean an individual or Legal Entity
26 | exercising permissions granted by this License.
27 |
28 | "Source" form shall mean the preferred form for making modifications,
29 | including but not limited to software source code, documentation
30 | source, and configuration files.
31 |
32 | "Object" form shall mean any form resulting from mechanical
33 | transformation or translation of a Source form, including but
34 | not limited to compiled object code, generated documentation,
35 | and conversions to other media types.
36 |
37 | "Work" shall mean the work of authorship, whether in Source or
38 | Object form, made available under the License, as indicated by a
39 | copyright notice that is included in or attached to the work
40 | (an example is provided in the Appendix below).
41 |
42 | "Derivative Works" shall mean any work, whether in Source or Object
43 | form, that is based on (or derived from) the Work and for which the
44 | editorial revisions, annotations, elaborations, or other modifications
45 | represent, as a whole, an original work of authorship. For the purposes
46 | of this License, Derivative Works shall not include works that remain
47 | separable from, or merely link (or bind by name) to the interfaces of,
48 | the Work and Derivative Works thereof.
49 |
50 | "Contribution" shall mean any work of authorship, including
51 | the original version of the Work and any modifications or additions
52 | to that Work or Derivative Works thereof, that is intentionally
53 | submitted to Licensor for inclusion in the Work by the copyright owner
54 | or by an individual or Legal Entity authorized to submit on behalf of
55 | the copyright owner. For the purposes of this definition, "submitted"
56 | means any form of electronic, verbal, or written communication sent
57 | to the Licensor or its representatives, including but not limited to
58 | communication on electronic mailing lists, source code control systems,
59 | and issue tracking systems that are managed by, or on behalf of, the
60 | Licensor for the purpose of discussing and improving the Work, but
61 | excluding communication that is conspicuously marked or otherwise
62 | designated in writing by the copyright owner as "Not a Contribution."
63 |
64 | "Contributor" shall mean Licensor and any individual or Legal Entity
65 | on behalf of whom a Contribution has been received by Licensor and
66 | subsequently incorporated within the Work.
67 |
68 | 2. Grant of Copyright License. Subject to the terms and conditions of
69 | this License, each Contributor hereby grants to You a perpetual,
70 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable
71 | copyright license to reproduce, prepare Derivative Works of,
72 | publicly display, publicly perform, sublicense, and distribute the
73 | Work and such Derivative Works in Source or Object form.
74 |
75 | 3. Grant of Patent License. Subject to the terms and conditions of
76 | this License, each Contributor hereby grants to You a perpetual,
77 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable
78 | (except as stated in this section) patent license to make, have made,
79 | use, offer to sell, sell, import, and otherwise transfer the Work,
80 | where such license applies only to those patent claims licensable
81 | by such Contributor that are necessarily infringed by their
82 | Contribution(s) alone or by combination of their Contribution(s)
83 | with the Work to which such Contribution(s) was submitted. If You
84 | institute patent litigation against any entity (including a
85 | cross-claim or counterclaim in a lawsuit) alleging that the Work
86 | or a Contribution incorporated within the Work constitutes direct
87 | or contributory patent infringement, then any patent licenses
88 | granted to You under this License for that Work shall terminate
89 | as of the date such litigation is filed.
90 |
91 | 4. Redistribution. You may reproduce and distribute copies of the
92 | Work or Derivative Works thereof in any medium, with or without
93 | modifications, and in Source or Object form, provided that You
94 | meet the following conditions:
95 |
96 | (a) You must give any other recipients of the Work or
97 | Derivative Works a copy of this License; and
98 |
99 | (b) You must cause any modified files to carry prominent notices
100 | stating that You changed the files; and
101 |
102 | (c) You must retain, in the Source form of any Derivative Works
103 | that You distribute, all copyright, patent, trademark, and
104 | attribution notices from the Source form of the Work,
105 | excluding those notices that do not pertain to any part of
106 | the Derivative Works; and
107 |
108 | (d) If the Work includes a "NOTICE" text file as part of its
109 | distribution, then any Derivative Works that You distribute must
110 | include a readable copy of the attribution notices contained
111 | within such NOTICE file, excluding those notices that do not
112 | pertain to any part of the Derivative Works, in at least one
113 | of the following places: within a NOTICE text file distributed
114 | as part of the Derivative Works; within the Source form or
115 | documentation, if provided along with the Derivative Works; or,
116 | within a display generated by the Derivative Works, if and
117 | wherever such third-party notices normally appear. The contents
118 | of the NOTICE file are for informational purposes only and
119 | do not modify the License. You may add Your own attribution
120 | notices within Derivative Works that You distribute, alongside
121 | or as an addendum to the NOTICE text from the Work, provided
122 | that such additional attribution notices cannot be construed
123 | as modifying the License.
124 |
125 | You may add Your own copyright statement to Your modifications and
126 | may provide additional or different license terms and conditions
127 | for use, reproduction, or distribution of Your modifications, or
128 | for any such Derivative Works as a whole, provided Your use,
129 | reproduction, and distribution of the Work otherwise complies with
130 | the conditions stated in this License.
131 |
132 | 5. Submission of Contributions. Unless You explicitly state otherwise,
133 | any Contribution intentionally submitted for inclusion in the Work
134 | by You to the Licensor shall be under the terms and conditions of
135 | this License, without any additional terms or conditions.
136 | Notwithstanding the above, nothing herein shall supersede or modify
137 | the terms of any separate license agreement you may have executed
138 | with Licensor regarding such Contributions.
139 |
140 | 6. Trademarks. This License does not grant permission to use the trade
141 | names, trademarks, service marks, or product names of the Licensor,
142 | except as required for reasonable and customary use in describing the
143 | origin of the Work and reproducing the content of the NOTICE file.
144 |
145 | 7. Disclaimer of Warranty. Unless required by applicable law or
146 | agreed to in writing, Licensor provides the Work (and each
147 | Contributor provides its Contributions) on an "AS IS" BASIS,
148 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
149 | implied, including, without limitation, any warranties or conditions
150 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
151 | PARTICULAR PURPOSE. You are solely responsible for determining the
152 | appropriateness of using or redistributing the Work and assume any
153 | risks associated with Your exercise of permissions under this License.
154 |
155 | 8. Limitation of Liability. In no event and under no legal theory,
156 | whether in tort (including negligence), contract, or otherwise,
157 | unless required by applicable law (such as deliberate and grossly
158 | negligent acts) or agreed to in writing, shall any Contributor be
159 | liable to You for damages, including any direct, indirect, special,
160 | incidental, or consequential damages of any character arising as a
161 | result of this License or out of the use or inability to use the
162 | Work (including but not limited to damages for loss of goodwill,
163 | work stoppage, computer failure or malfunction, or any and all
164 | other commercial damages or losses), even if such Contributor
165 | has been advised of the possibility of such damages.
166 |
167 | 9. Accepting Warranty or Additional Liability. While redistributing
168 | the Work or Derivative Works thereof, You may choose to offer,
169 | and charge a fee for, acceptance of support, warranty, indemnity,
170 | or other liability obligations and/or rights consistent with this
171 | License. However, in accepting such obligations, You may act only
172 | on Your own behalf and on Your sole responsibility, not on behalf
173 | of any other Contributor, and only if You agree to indemnify,
174 | defend, and hold each Contributor harmless for any liability
175 | incurred by, or claims asserted against, such Contributor by reason
176 | of your accepting any such warranty or additional liability.
177 |
178 | END OF TERMS AND CONDITIONS
179 |
180 | APPENDIX: How to apply the Apache License to your work.
181 |
182 | To apply the Apache License to your work, attach the following
183 | boilerplate notice, with the fields enclosed by brackets "[]"
184 | replaced with your own identifying information. (Don't include
185 | the brackets!) The text should be enclosed in the appropriate
186 | comment syntax for the file format. We also recommend that a
187 | file or class name and description of purpose be included on the
188 | same "printed page" as the copyright notice for easier
189 | identification within third-party archives.
190 |
191 | Copyright [yyyy] [name of copyright owner]
192 |
193 | Licensed under the Apache License, Version 2.0 (the "License");
194 | you may not use this file except in compliance with the License.
195 | You may obtain a copy of the License at
196 |
197 | http://www.apache.org/licenses/LICENSE-2.0
198 |
199 | Unless required by applicable law or agreed to in writing, software
200 | distributed under the License is distributed on an "AS IS" BASIS,
201 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
202 | See the License for the specific language governing permissions and
203 | limitations under the License.
204 |
--------------------------------------------------------------------------------
/LRP-Time-Series/LRP_tutorial.py:
--------------------------------------------------------------------------------
1 |
2 | # coding: utf-8
3 |
4 | # In[1]:
5 |
6 | # Imports
7 | import numpy as np
8 | import os
9 | from utilities import *
10 | from sklearn.model_selection import train_test_split
11 | import matplotlib.pyplot as plt
12 |
13 | # get_ipython().magic(u'matplotlib inline')
14 |
15 |
16 | # In[2]:
17 |
18 | # 각자의 데이터 경로로 수정
19 | X_train, labels_train, list_ch_train = read_data(data_path="UCI HAR Dataset/", split="train") # train
20 | X_test, labels_test, list_ch_test = read_data(data_path="UCI HAR Dataset/", split="test") # test
21 |
22 | assert list_ch_train == list_ch_test, "Mistmatch in channels!"
23 |
24 |
25 | # In[3]:
26 |
27 | # Normalize?
28 | X_train, X_test = standardize(X_train, X_test)
29 |
30 |
31 | # In[4]:
32 |
33 | X_tr, X_vld, lab_tr, lab_vld = train_test_split(X_train, labels_train,
34 | stratify = labels_train, random_state = 123)
35 |
36 |
37 | # In[5]:
38 |
39 | y_tr = one_hot(lab_tr)
40 | y_vld = one_hot(lab_vld)
41 | y_test = one_hot(labels_test)
42 |
43 |
44 | # In[6]:
45 |
46 | # Imports
47 | import tensorflow as tf
48 |
49 |
50 | # In[7]:
51 |
52 | batch_size = 600 # Batch size
53 | seq_len = 128 # Number of steps
54 | learning_rate = 0.0001
55 | epochs = 1000
56 |
57 | n_classes = 6
58 | n_channels = 9
59 |
60 |
61 | # As in many CNN architectures, the deeper the layers get, the higher the number of filters become.
62 |
63 | # In[8]:
64 |
65 | class New_CNN:
66 |
67 | def __init__(self, name):
68 | self.name = name
69 |
70 | def __call__(self, X, reuse=False):
71 |
72 | with tf.variable_scope(self.name) as scope:
73 |
74 | if reuse:
75 | scope.reuse_variables()
76 |
77 | with tf.variable_scope('layer0'):
78 | X_img = X
79 |
80 | # Convolutional Layer #1
81 | with tf.variable_scope('layer1'):
82 | # (batch, 128, 9) --> (batch, 128, 18)
83 | conv1 = tf.layers.conv1d(inputs=X_img, filters=18, kernel_size=2,
84 | padding='same', activation = tf.nn.relu, use_bias=False)
85 |
86 | # Convolutional Layer #2
87 | with tf.variable_scope('layer2'):
88 | # (batch, 64, 18) --> (batch, 128, 36)
89 | conv2 = tf.layers.conv1d(inputs=conv1, filters=36, kernel_size=2,
90 | padding='same', activation = tf.nn.relu, use_bias=False)
91 |
92 | # Convolutional Layer #3
93 | with tf.variable_scope('layer3'):
94 | # (batch, 32, 36) --> (batch, 128, 72)
95 | conv3 = tf.layers.conv1d(inputs=conv2, filters=72, kernel_size=2,
96 | padding='same', activation = tf.nn.relu, use_bias=False)
97 |
98 |
99 | # Dense Layer with Relu
100 | with tf.variable_scope('layer4'):
101 | # (batch, 16, 72) --> (batch, 128, 144)
102 | conv4 = tf.layers.conv1d(inputs=conv3, filters=144, kernel_size=2,
103 | padding='same', activation = tf.nn.relu, use_bias=False)
104 |
105 | # Logits (no activation) Layer: L5 Final FC 625 inputs -> 10 outputs
106 | with tf.variable_scope('layer5'):
107 | # Flatten and add dropout
108 | flat = tf.reshape(conv4, (-1, 128*144))
109 |
110 | # Predictions
111 | logits = tf.layers.dense(flat, n_classes, use_bias=False)
112 | prediction = tf.nn.softmax(logits)
113 |
114 | return [X_img, conv1, conv2, conv3,conv4, flat, prediction], logits
115 |
116 | @property
117 | def vars(self):
118 | return tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES, scope=self.name)
119 |
120 |
121 |
122 |
123 |
124 | # In[9]:
125 |
126 | graph = tf.Graph()
127 |
128 | # Construct placeholders
129 | with graph.as_default():
130 |
131 | new_CNN = New_CNN('CNN')
132 |
133 | inputs_ = tf.placeholder(tf.float32, [None, seq_len, n_channels], name = 'inputs')
134 | labels_ = tf.placeholder(tf.float32, [None, n_classes], name = 'labels')
135 | keep_prob_ = tf.placeholder(tf.float32, name = 'keep')
136 | learning_rate_ = tf.placeholder(tf.float32, name = 'learning_rate')
137 |
138 |
139 | # In[10]:
140 |
141 | with graph.as_default():
142 |
143 | activations, logits = new_CNN(inputs_)
144 |
145 | tf.add_to_collection('DTD', inputs_)
146 |
147 | for activation in activations:
148 | tf.add_to_collection('DTD', activation)
149 |
150 | # Cost function and optimizer
151 | cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=logits, labels=labels_))
152 | optimizer = tf.train.AdamOptimizer(learning_rate_).minimize(cost)
153 |
154 | # Accuracy
155 | correct_pred = tf.equal(tf.argmax(logits, 1), tf.argmax(labels_, 1))
156 | accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32), name='accuracy')
157 |
158 |
159 | # In[11]:
160 |
161 | # if (os.path.exists('checkpoints-cnn') == False):
162 | # get_ipython().system(u'mkdir checkpoints-cnn')
163 |
164 |
165 | # In[12]:
166 |
167 | validation_acc = []
168 | validation_loss = []
169 |
170 | train_acc = []
171 | train_loss = []
172 |
173 | with graph.as_default():
174 | saver = tf.train.Saver()
175 |
176 | config = tf.ConfigProto()
177 | config.gpu_options.allow_growth = True
178 |
179 | with tf.Session(config=config,graph=graph) as sess:
180 | sess.run(tf.global_variables_initializer())
181 | iteration = 1
182 |
183 | # Loop over epochs
184 | for e in range(epochs):
185 |
186 | # Loop over batches
187 | for x,y in get_batches(X_tr, y_tr, batch_size):
188 |
189 | # Feed dictionary
190 | feed = {inputs_ : x, labels_ : y, keep_prob_ : 0.5, learning_rate_ : learning_rate}
191 |
192 | # Loss
193 | loss, _ , acc = sess.run([cost, optimizer, accuracy], feed_dict = feed)
194 | train_acc.append(acc)
195 | train_loss.append(loss)
196 |
197 | # Print at each 5 iters
198 | if (iteration % 100 == 0):
199 | print("Epoch: {}/{}".format(e, epochs),
200 | "Iteration: {:d}".format(iteration),
201 | "Train loss: {:6f}".format(loss),
202 | "Train acc: {:.6f}".format(acc))
203 |
204 | # Compute validation loss at every 10 iterations
205 | if (iteration%100 == 0):
206 | val_acc_ = []
207 | val_loss_ = []
208 |
209 | for x_v, y_v in get_batches(X_vld, y_vld, batch_size):
210 | # Feed
211 | feed = {inputs_ : x_v, labels_ : y_v, keep_prob_ : 1.0}
212 |
213 | # Loss
214 | loss_v, acc_v = sess.run([cost, accuracy], feed_dict = feed)
215 | val_acc_.append(acc_v)
216 | val_loss_.append(loss_v)
217 |
218 | # Print info
219 | print("Epoch: {}/{}".format(e, epochs),
220 | "Iteration: {:d}".format(iteration),
221 | "Validation loss: {:6f}".format(np.mean(val_loss_)),
222 | "Validation acc: {:.6f}".format(np.mean(val_acc_)))
223 |
224 | # Store
225 | validation_acc.append(np.mean(val_acc_))
226 | validation_loss.append(np.mean(val_loss_))
227 |
228 | # Iterate
229 | iteration += 1
230 |
231 | saver.save(sess,"checkpoints-cnn/har.ckpt")
232 |
233 |
234 | # In[13]:
235 |
236 | # Plot training and test loss
237 | t = np.arange(iteration-1)
238 |
239 | fig = plt.figure(figsize = (6,6))
240 | plt.plot(t, np.array(train_loss), 'r-')
241 | plt.plot(t[t % 100 == 0], np.array(validation_loss), 'b*')
242 | plt.xlabel("iteration")
243 | plt.ylabel("Loss")
244 | plt.legend(['train', 'validation'], loc='upper right')
245 | plt.show()
246 | fig.savefig('checkpoints-cnn/loss_graph.png', dpi=fig.dpi)
247 |
248 |
249 | # In[14]:
250 |
251 |
252 | # Plot Accuracies
253 | fig = plt.figure(figsize = (6,6))
254 |
255 | plt.plot(t, np.array(train_acc), 'r-', t[t % 100 == 0], validation_acc, 'b*')
256 | plt.xlabel("iteration")
257 | plt.ylabel("Accuray")
258 | plt.legend(['train', 'validation'], loc='upper right')
259 | plt.show()
260 | fig.savefig('checkpoints-cnn/accuray_graph.png', dpi=fig.dpi)
261 |
262 |
263 | # In[15]:
264 |
265 | test_acc = []
266 |
267 | # config = tf.ConfigProto(device_count={'GPU': 0})
268 | config = tf.ConfigProto()
269 | # config.gpu_options.visible_device_list= '0' #only see the gpu 1
270 | config.gpu_options.allow_growth = True
271 | with tf.Session(config=config,graph=graph) as sess:
272 | # Restore
273 | saver.restore(sess, tf.train.latest_checkpoint('checkpoints-cnn'))
274 |
275 | for x_t, y_t in get_batches(X_test, y_test, batch_size):
276 | feed = {inputs_: x_t,
277 | labels_: y_t,
278 | keep_prob_: 1}
279 |
280 | batch_acc = sess.run(accuracy, feed_dict=feed)
281 | test_acc.append(batch_acc)
282 | print("Test accuracy: {:.6f}".format(np.mean(test_acc)))
283 |
284 |
285 | # In[16]:
286 |
287 | tf.reset_default_graph()
288 | # config = tf.ConfigProto(device_count={'GPU': 0})
289 | config = tf.ConfigProto()
290 | # config.gpu_options.visible_device_list= '0' #only see the gpu 1
291 | config.gpu_options.allow_growth = True
292 | sess = tf.InteractiveSession(config=config)
293 |
294 | new_saver = tf.train.import_meta_graph('checkpoints-cnn/har.ckpt.meta')
295 | new_saver.restore(sess, tf.train.latest_checkpoint('./checkpoints-cnn'))
296 | # weights = tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES, scope='.*kernel.*')
297 | # biases = tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES, scope='.*bias.*')
298 |
299 | weights = tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES, scope='.*kernel.*')
300 | activations = tf.get_collection('DTD')
301 | X = activations[0]
302 |
303 |
304 | # In[17]:
305 |
306 | X
307 |
308 |
309 | # In[18]:
310 |
311 | activations
312 |
313 |
314 | # In[19]:
315 |
316 | weights
317 |
318 |
319 | # In[20]:
320 |
321 | from tensorflow.python.ops import nn_ops, gen_nn_ops
322 | from tensorflow.python.layers import pooling
323 | class Taylor:
324 |
325 | def __init__(self, activations, weights, conv_ksize, pool_ksize, conv_strides, pool_strides, name):
326 |
327 | self.last_ind = len(activations)
328 | for op in activations[::-1]:
329 | self.last_ind -= 1
330 | if any([word in op.name for word in ['conv', 'pooling', 'dense']]):
331 | break
332 |
333 | self.activations = activations
334 | self.weights = weights
335 | self.conv_ksize = conv_ksize
336 | self.pool_ksize = pool_ksize
337 | self.conv_strides = conv_strides
338 | self.pool_strides = pool_strides
339 | self.name = name
340 |
341 | def __call__(self, logit):
342 | with tf.name_scope(self.name):
343 | Rs = []
344 | j = 0
345 |
346 | for i in range(len(self.activations) - 2):
347 |
348 | if i is self.last_ind:
349 |
350 | if 'conv' in self.activations[i].name.lower():
351 | Rs.append(self.backprop_conv_input(self.activations[i + 1], self.weights[j], Rs[-1], self.conv_strides))
352 | else:
353 | Rs.append(self.backprop_dense_input(self.activations[i + 1], self.weights[j], Rs[-1]))
354 |
355 | continue
356 |
357 | if i is 0:
358 | Rs.append(self.activations[i][:,logit,None])
359 | Rs.append(self.backprop_dense(self.activations[i + 1], self.weights[j][:,logit,None], Rs[-1]))
360 | j += 1
361 |
362 | continue
363 |
364 | elif 'dense' in self.activations[i].name.lower():
365 | Rs.append(self.backprop_dense(self.activations[i + 1], self.weights[j], Rs[-1]))
366 | j += 1
367 | elif 'reshape' in self.activations[i].name.lower():
368 | shape = self.activations[i + 1].get_shape().as_list()
369 | shape[0] = -1
370 | Rs.append(tf.reshape(Rs[-1], shape))
371 | elif 'conv' in self.activations[i].name.lower():
372 | Rs.append(self.backprop_conv(self.activations[i + 1], self.weights[j], Rs[-1], self.conv_strides))
373 | j += 1
374 | else:
375 | raise Exception('Unknown operation.')
376 |
377 | return Rs[-1]
378 |
379 | def backprop_conv(self, activation, kernel, relevance, strides, padding='SAME'):
380 | W_p = tf.maximum(0., kernel)
381 | z = nn_ops.conv1d(activation, W_p, strides, padding) + 1e-10
382 | s = relevance / z
383 | print(tf.shape(s))
384 | c = nn_ops.conv1d_transpose(s, W_p, tf.shape(activation), strides, padding)
385 | return activation * c
386 |
387 | def backprop_dense(self, activation, kernel, relevance):
388 | W_p = tf.maximum(0., kernel)
389 | z = tf.matmul(activation, W_p) + 1e-10
390 | s = relevance / z
391 | c = tf.matmul(s, tf.transpose(W_p))
392 | return activation * c
393 |
394 | def backprop_conv_input(self, X, kernel, relevance, strides, padding='SAME', lowest=0., highest=1.):
395 | W_p = tf.maximum(0., kernel)
396 | W_n = tf.minimum(0., kernel)
397 |
398 | L = tf.ones_like(X, tf.float32) * lowest
399 | H = tf.ones_like(X, tf.float32) * highest
400 |
401 | z_o = nn_ops.conv1d(X, kernel, strides, padding)
402 | z_p = nn_ops.conv1d(L, W_p, strides, padding)
403 | z_n = nn_ops.conv1d(H, W_n, strides, padding)
404 |
405 | z = z_o - z_p - z_n + 1e-10
406 | s = relevance / z
407 |
408 | c_o = nn_ops.conv1d_transpose(s, kernel, tf.shape(X), strides, padding)
409 | c_p = nn_ops.conv1d_transpose(s, W_p, tf.shape(X), strides, padding)
410 | c_n = nn_ops.conv1d_transpose(s, W_n, tf.shape(X), strides, padding)
411 |
412 | return X * c_o - L * c_p - H * c_n
413 |
414 | def backprop_dense_input(self, X, kernel, relevance, lowest=0., highest=1.):
415 | W_p = tf.maximum(0., kernel)
416 | W_n = tf.minimum(0., kernel)
417 |
418 | L = tf.ones_like(X, tf.float32) * lowest
419 | H = tf.ones_like(X, tf.float32) * highest
420 |
421 | z_o = tf.matmul(X, kernel)
422 | z_p = tf.matmul(L, W_p)
423 | z_n = tf.matmul(H, W_n)
424 |
425 | z = z_o - z_p - z_n + 1e-10
426 | s = relevance / z
427 |
428 | c_o = tf.matmul(s, tf.transpose(kernel))
429 | c_p = tf.matmul(s, tf.transpose(W_p))
430 | c_n = tf.matmul(s, tf.transpose(W_n))
431 |
432 | return X * c_o - L * c_p - H * c_n
433 |
434 |
435 | # In[21]:
436 |
437 | conv_ksize = 2
438 | pool_ksize = 2
439 | conv_strides = 1
440 | pool_strides = 2
441 |
442 | weights.reverse()
443 | activations.reverse()
444 |
445 |
446 | # In[22]:
447 |
448 | taylor = Taylor(activations, weights, conv_ksize, pool_ksize, conv_strides, pool_strides, 'Taylor')
449 |
450 |
451 | # In[23]:
452 |
453 | Rs = []
454 | for i in range(6):
455 | Rs.append(taylor(i))
456 |
457 |
458 | # In[24]:
459 |
460 | sample_imgs = []
461 | for i in range(6):
462 | sample_imgs.append(X_tr[np.argmax(y_tr, axis=1) == i][10])
463 |
464 |
465 | # In[25]:
466 |
467 | imgs = []
468 | for i in range(6):
469 | imgs.append(sess.run(Rs[i], feed_dict={X: sample_imgs[i][None,:]}))
470 |
471 |
472 | # In[26]:
473 |
474 | imgs = np.squeeze(imgs)
475 | sample_imgs = np.squeeze(sample_imgs)
476 |
477 |
478 | # 1 WALKING
479 | # 2 WALKING_UPSTAIRS
480 | # 3 WALKING_DOWNSTAIRS
481 | # 4 SITTING
482 | # 5 STANDING
483 | # 6 LAYING
484 |
485 | # In[27]:
486 |
487 | for i in range(6):
488 | plt.figure(figsize=(150,150))
489 | plt.subplot(1, 2, 1)
490 | plt.imshow(np.transpose(imgs[i]), cmap='hot_r')
491 |
492 | plt.subplot(1, 2, 2)
493 | plt.imshow(np.transpose(sample_imgs[i]), cmap='hot_r')
494 | plt.savefig(f'test{i}.png')
495 |
496 |
497 | # In[ ]:
498 |
499 |
500 |
501 |
--------------------------------------------------------------------------------
/LRP-Time-Series/README.md:
--------------------------------------------------------------------------------
1 |
2 | LRP-Time-Series
3 | ==
4 |
5 | Python implementation of the LRP method that is a novel methodology for interpreting generic multilayer neural networks by decomposing the network classification decision into contributions of its input elements.
6 |
7 | ## Reference Code
8 | Based on code by [Eric (Beomsu) Kim](https://github.com/1202kbs/Understanding-NN), [Chintan Zaveri](https://github.com/zaverichintan/HAR_prediction)
9 |
10 | ## Reference Paper
11 | **"Explaining nonlinear classification decisions with deep taylor decomposition"**. Gregoire Montavon, Sebastian Bach, Alexander Binder, Wojciech Samek, and Klaus-Robert Muller (https://arxiv.org/abs/1512.02479)
12 |
13 | ## Example Setup
14 | This is a deep learning method to classify time- series dataset. Our goal is to test how the LRP (more specifically deep Taylor Decomposition) can perform to depict the important time epochs and features from raw time series data.
15 |
16 | 
17 |
18 |
19 | ## Dataset
20 | We will use the classic Human Activity Recognition (HAR) dataset from the UCI repository. The dataset contains the raw time-series data on human activity.
21 | https://archive.ics.uci.edu/ml/datasets/human+activity+recognition+using+smartphones
22 |
23 | ## Details of Dataset and Models
24 | + We use deep neural network with four 1D convolution layers and 1 fully connected layer.
25 | + In the code, cast the data set in a numpy array with shape (batch-size, sequence-len, n-channels)
26 | + Batch-size: the # of examples training together
27 | + Sequence-len: the length of sequence in time (128 steps here)
28 | + N-channels: the # of channels in the layer (# of channels in input is the # of measurements) ties:
29 | + There are 6 classes of activities: walking, walking upstairs, walking downstairs, sitting standing, laying
30 |
31 | 
32 |
33 |
34 | ## Installation
35 | 
36 |
37 | **1. Fork & Clone** : Fork this project to your repository and clone to your work directory.
38 |
39 | ``` $ git clone https://github.com/OpenXAIProject/LRP-Time-Series.git ```
40 |
41 | **2. Download Dataset** : Go to the `UCI` repository site and download the "UCI HAR Dataset"
42 |
43 | **3. Change Directory** : Move the "UCI HAR Dataset" to your work directory. It must be in the same folder as `LRP_tutorial.ipynb`.
44 |
45 | **4. Run** : Run `LRP_tutorial.ipynb` or `LRP_tutorial.py`
46 |
47 | ## Requirements
48 | + tensorflow (1.9.0)
49 | + numpy (1.15.0)
50 | + matplotlib (2.2.2)
51 | + scikit-learn (0.19.1)
52 |
53 | ## License
54 | [Apache License 2.0](https://github.com/OpenXAIProject/tutorials/blob/master/LICENSE "Apache")
55 |
56 | ## Contacts
57 | If you have any question, please contact Xie Qin (xieqin856@unist.ac.kr) and/or Sohee Cho (shcho@unist.ac.kr).
58 |
59 |
60 |
61 |
62 | # XAI Project
63 |
64 | **This work was supported by Institute for Information & Communications Technology Promotion (IITP) grant funded by the Korea government (MSIT) (No.2017-0-01779, A machine learning and statistical inference framework for explainable artificial intelligence)**
65 |
66 | + Project Name : A machine learning and statistical inference framework for explainable artificial intelligence (의사결정 이유를 설명할 수 있는 인간 수준의 학습·추론 프레임워크 개발)
67 |
68 | + Managed by Ministry of Science and ICT/XAIC
69 |
70 | + Participated Affiliation : UNIST, Korea Univ., Yonsei Univ., KAIST, AItrics
71 |
72 | + Web Site :
73 |
74 |
--------------------------------------------------------------------------------
/LRP-Time-Series/checkpoints-cnn/checkpoint:
--------------------------------------------------------------------------------
1 | model_checkpoint_path: "har.ckpt"
2 | all_model_checkpoint_paths: "har.ckpt"
3 |
--------------------------------------------------------------------------------
/LRP-Time-Series/checkpoints-cnn/har.ckpt.data-00000-of-00001:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/OpenXAIProject/Tutorials/58dbcb5650a44c2ef7f9557dea098fb5708fde3b/LRP-Time-Series/checkpoints-cnn/har.ckpt.data-00000-of-00001
--------------------------------------------------------------------------------
/LRP-Time-Series/checkpoints-cnn/har.ckpt.index:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/OpenXAIProject/Tutorials/58dbcb5650a44c2ef7f9557dea098fb5708fde3b/LRP-Time-Series/checkpoints-cnn/har.ckpt.index
--------------------------------------------------------------------------------
/LRP-Time-Series/checkpoints-cnn/har.ckpt.meta:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/OpenXAIProject/Tutorials/58dbcb5650a44c2ef7f9557dea098fb5708fde3b/LRP-Time-Series/checkpoints-cnn/har.ckpt.meta
--------------------------------------------------------------------------------
/LRP-Time-Series/howtorun.gif:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/OpenXAIProject/Tutorials/58dbcb5650a44c2ef7f9557dea098fb5708fde3b/LRP-Time-Series/howtorun.gif
--------------------------------------------------------------------------------
/LRP-Time-Series/model.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/OpenXAIProject/Tutorials/58dbcb5650a44c2ef7f9557dea098fb5708fde3b/LRP-Time-Series/model.jpg
--------------------------------------------------------------------------------
/LRP-Time-Series/result.jpg:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/OpenXAIProject/Tutorials/58dbcb5650a44c2ef7f9557dea098fb5708fde3b/LRP-Time-Series/result.jpg
--------------------------------------------------------------------------------
/LRP-Time-Series/utilities.py:
--------------------------------------------------------------------------------
1 | # HAR classification
2 | # Author: Burak Himmetoglu
3 | # 8/15/2017
4 |
5 | import pandas as pd
6 | import numpy as np
7 | import os
8 |
9 | def read_data(data_path, split = "train"):
10 | """ Read data """
11 |
12 | # Fixed params
13 | n_class = 6
14 | n_steps = 128
15 |
16 | # Paths
17 | path_ = os.path.join(data_path, split)
18 | path_signals = os.path.join(path_, "Inertial Signals")
19 |
20 | # Read labels and one-hot encode
21 | label_path = os.path.join(path_, "y_" + split + ".txt")
22 | labels = pd.read_csv(label_path, header = None)
23 |
24 | # Read time-series data
25 | channel_files = os.listdir(path_signals)
26 | channel_files.sort()
27 | n_channels = len(channel_files)
28 | posix = len(split) + 5
29 |
30 | # Initiate array
31 | list_of_channels = []
32 | X = np.zeros((len(labels), n_steps, n_channels))
33 | i_ch = 0
34 | for fil_ch in channel_files:
35 | channel_name = fil_ch[:-posix]
36 | dat_ = pd.read_csv(os.path.join(path_signals,fil_ch), delim_whitespace = True, header = None)
37 | X[:,:,i_ch] = dat_.as_matrix()
38 |
39 | # Record names
40 | list_of_channels.append(channel_name)
41 |
42 | # iterate
43 | i_ch += 1
44 |
45 | # Return
46 | return X, labels[0].values, list_of_channels
47 |
48 | def standardize(train, test):
49 | """ Standardize data """
50 |
51 | # Standardize train and test
52 | X_train = (train - np.mean(train, axis=0)[None,:,:]) / np.std(train, axis=0)[None,:,:]
53 | X_test = (test - np.mean(test, axis=0)[None,:,:]) / np.std(test, axis=0)[None,:,:]
54 |
55 | return X_train, X_test
56 |
57 | def one_hot(labels, n_class = 6):
58 | """ One-hot encoding """
59 | expansion = np.eye(n_class)
60 | y = expansion[:, labels-1].T
61 | assert y.shape[1] == n_class, "Wrong number of labels!"
62 |
63 | return y
64 |
65 | def get_batches(X, y, batch_size = 100):
66 | """ Return a generator for batches """
67 | n_batches = len(X) // batch_size
68 | X, y = X[:n_batches*batch_size], y[:n_batches*batch_size]
69 |
70 | # Loop over batches and yield
71 | for b in range(0, len(X), batch_size):
72 | yield X[b:b+batch_size], y[b:b+batch_size]
73 |
74 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | # tutorials
2 |
3 | This repository contains a number of different tutorials by XAI Proeject.
4 |
5 | 1. [Basic-LRP](https://github.com/OpenXAIProject/tutorials/tree/master/Basic-LRP) : Taylor decomposition and simple LRP method for the explanation of non-linear classification decision.
6 |
7 | 2. [LRP-Time-Series Tutorial](https://github.com/OpenXAIProject/tutorials/tree/master/LRP-Time-Series) : Explaining Time Series Deep Learning Models with Layer-wise Relevance Propagation
8 |
9 | 2. [Visual-Explanation-of-Atari](https://github.com/OpenXAIProject/tutorials/tree/master/Visual-Explanation-of-Atari) : Visual Interpretation of Deep Reinforcement Learning for Atari Games
10 |
11 |
12 | ## License
13 | [Apache License 2.0](https://github.com/OpenXAIProject/tutorials/blob/master/LICENSE "Apache")
14 |
15 |
16 |
17 |
18 |
19 | # XAI Project
20 |
21 | **These works were supported by Institute for Information & Communications Technology Promotion (IITP) grant funded by the Korea government (MSIT) (No.2017-0-01779, A machine learning and statistical inference framework for explainable artificial intelligence)**
22 |
23 | + Project Name : A machine learning and statistical inference framework for explainable artificial intelligence(의사결정 이유를 설명할 수 있는 인간 수준의 학습·추론 프레임워크 개발)
24 |
25 | + Managed by Ministry of Science and ICT/XAIC
26 |
27 | + Participated Affiliation : UNIST, Korea Univ., Yonsei Univ., KAIST, AItrics
28 |
29 | + Web Site :
30 |
31 |
--------------------------------------------------------------------------------
/Visual-Explanation-of-Atari/README.md:
--------------------------------------------------------------------------------
1 | Visual-Explanation-of-Atari
2 | ==
3 |
4 | Tensorflow Implementation of Visualizing and Understanding Atari Agents that is a novel methodology for interpreting decision of agent trained under the reinforcement learning framework. The original pytorch implemetation is in https://github.com/greydanus/visualize_atari.
5 |
6 | ## Reference paper
7 | **"Visualizing and Understanding Atari Agents"**. Sam Greydanus, Anurag Koul, Jonathan Dodge and Alan Fern (https://arxiv.org/abs/1711.00138)
8 |
9 | ## Running Examples
10 | 
11 | 
12 |
13 | ## Environment
14 | We will use the OpenAI Gym environment. OpenAI Gym is a toolkit for developing and comparing reinforcement learning algorithms. https://gym.openai.com/envs/#atari
15 |
16 | ## Pretrained Models
17 | We provided pretrained models for the game of "breakout" and "pong".
18 | These models were obtained using [this repo](https://github.com/NVlabs/GA3C) (default hyperparameters).
19 |
20 | ## How to Use
21 |
22 | ```bash
23 | $ cd Visual-Explanation-of-Atari
24 | $ python main.py -m critic -e BreakoutDeterministic-v0 --first_frame 350 --num_frames 100
25 | ```
26 |
27 | ## Requirements
28 | + tensorflow (1.4.0)
29 | + numpy (1.15.0)
30 | + matplotlib (2.2.2)
31 |
32 | ## License
33 | [Apache License 2.0](https://github.com/OpenXAIProject/LRP-Time-Series/blob/master/LICENSE "Apache")
34 |
35 | ## Contacts
36 | If you have any question, please contact Kyowoon Lee(leekwoon@unist.ac.kr) and/or Sohee Cho(shcho@unist.ac.kr).
37 |
38 |
39 |
40 |
41 | # XAI Project
42 |
43 | **This work was supported by Institute for Information & Communications Technology Promotion(IITP) grant funded by the Korea government(MSIT) (No.2017-0-01779, A machine learning and statistical inference framework for explainable artificial intelligence)**
44 |
45 | + Project Name : A machine learning and statistical inference framework for explainable artificial intelligence(의사결정 이유를 설명할 수 있는 인간 수준의 학습·추론 프레임워크 개발)
46 |
47 | + Managed by Ministry of Science and ICT/XAIC
48 |
49 | + Participated Affiliation : UNIST, Korea Univ., Yonsei Univ., KAIST, AItrics
50 |
51 | + Web Site :
52 |
--------------------------------------------------------------------------------
/Visual-Explanation-of-Atari/assets/actor_breakout.gif:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/OpenXAIProject/Tutorials/58dbcb5650a44c2ef7f9557dea098fb5708fde3b/Visual-Explanation-of-Atari/assets/actor_breakout.gif
--------------------------------------------------------------------------------
/Visual-Explanation-of-Atari/assets/actor_pong.gif:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/OpenXAIProject/Tutorials/58dbcb5650a44c2ef7f9557dea098fb5708fde3b/Visual-Explanation-of-Atari/assets/actor_pong.gif
--------------------------------------------------------------------------------
/Visual-Explanation-of-Atari/assets/critic_breakout.gif:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/OpenXAIProject/Tutorials/58dbcb5650a44c2ef7f9557dea098fb5708fde3b/Visual-Explanation-of-Atari/assets/critic_breakout.gif
--------------------------------------------------------------------------------
/Visual-Explanation-of-Atari/assets/critic_pong.gif:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/OpenXAIProject/Tutorials/58dbcb5650a44c2ef7f9557dea098fb5708fde3b/Visual-Explanation-of-Atari/assets/critic_pong.gif
--------------------------------------------------------------------------------
/Visual-Explanation-of-Atari/checkpoints/breakout/network_00097000.data-00000-of-00001:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/OpenXAIProject/Tutorials/58dbcb5650a44c2ef7f9557dea098fb5708fde3b/Visual-Explanation-of-Atari/checkpoints/breakout/network_00097000.data-00000-of-00001
--------------------------------------------------------------------------------
/Visual-Explanation-of-Atari/checkpoints/breakout/network_00097000.index:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/OpenXAIProject/Tutorials/58dbcb5650a44c2ef7f9557dea098fb5708fde3b/Visual-Explanation-of-Atari/checkpoints/breakout/network_00097000.index
--------------------------------------------------------------------------------
/Visual-Explanation-of-Atari/checkpoints/breakout/network_00097000.meta:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/OpenXAIProject/Tutorials/58dbcb5650a44c2ef7f9557dea098fb5708fde3b/Visual-Explanation-of-Atari/checkpoints/breakout/network_00097000.meta
--------------------------------------------------------------------------------
/Visual-Explanation-of-Atari/checkpoints/pong/network_00029000.data-00000-of-00001:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/OpenXAIProject/Tutorials/58dbcb5650a44c2ef7f9557dea098fb5708fde3b/Visual-Explanation-of-Atari/checkpoints/pong/network_00029000.data-00000-of-00001
--------------------------------------------------------------------------------
/Visual-Explanation-of-Atari/checkpoints/pong/network_00029000.index:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/OpenXAIProject/Tutorials/58dbcb5650a44c2ef7f9557dea098fb5708fde3b/Visual-Explanation-of-Atari/checkpoints/pong/network_00029000.index
--------------------------------------------------------------------------------
/Visual-Explanation-of-Atari/checkpoints/pong/network_00029000.meta:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/OpenXAIProject/Tutorials/58dbcb5650a44c2ef7f9557dea098fb5708fde3b/Visual-Explanation-of-Atari/checkpoints/pong/network_00029000.meta
--------------------------------------------------------------------------------
/Visual-Explanation-of-Atari/config.py:
--------------------------------------------------------------------------------
1 | class Config:
2 |
3 | #########################################################################
4 | # Game configuration
5 |
6 | # Name of the game, with version (e.g. PongDeterministic-v0)
7 | ATARI_GAME = 'PongDeterministic-v0'
8 |
9 | # Enable to see the trained agent in action
10 | PLAY_MODE = False
11 |
12 | # Input of the DNN
13 | STACKED_FRAMES = 4
14 | IMAGE_WIDTH = 84
15 | IMAGE_HEIGHT = 84
--------------------------------------------------------------------------------
/Visual-Explanation-of-Atari/env.py:
--------------------------------------------------------------------------------
1 | # Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved.
2 | #
3 | # Redistribution and use in source and binary forms, with or without
4 | # modification, are permitted provided that the following conditions
5 | # are met:
6 | # * Redistributions of source code must retain the above copyright
7 | # notice, this list of conditions and the following disclaimer.
8 | # * Redistributions in binary form must reproduce the above copyright
9 | # notice, this list of conditions and the following disclaimer in the
10 | # documentation and/or other materials provided with the distribution.
11 | # * Neither the name of NVIDIA CORPORATION nor the names of its
12 | # contributors may be used to endorse or promote products derived
13 | # from this software without specific prior written permission.
14 | #
15 | # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
16 | # EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
17 | # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
18 | # PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
19 | # CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
20 | # EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
21 | # PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
22 | # PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
23 | # OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
24 | # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
25 | # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
26 |
27 | import sys
28 | if sys.version_info >= (3,0):
29 | from queue import Queue
30 | else:
31 | from Queue import Queue
32 |
33 | import gym
34 | import numpy as np
35 | import scipy.misc as misc
36 |
37 | from config import Config
38 |
39 |
40 |
41 |
42 | class GameManager:
43 | def __init__(self, game_name, display):
44 | self.game_name = game_name
45 | self.display = display
46 |
47 | self.env = gym.make(game_name)
48 | self.reset()
49 |
50 | def reset(self):
51 | observation = self.env.reset()
52 | return observation
53 |
54 | def step(self, action):
55 | self._update_display()
56 | observation, reward, done, info = self.env.step(action)
57 | return observation, reward, done, info
58 |
59 | def _update_display(self):
60 | if self.display:
61 | self.env.render()
62 |
63 |
64 | class Environment:
65 | def __init__(self):
66 | self.game = GameManager(Config.ATARI_GAME, display=Config.PLAY_MODE)
67 | self.nb_frames = Config.STACKED_FRAMES
68 | self.frame_q = Queue(maxsize=self.nb_frames)
69 | self.previous_state = None
70 | self.current_state = None
71 | self.total_reward = 0
72 |
73 | self.reset()
74 |
75 | @staticmethod
76 | def _rgb2gray(rgb):
77 | return np.dot(rgb[..., :3], [0.299, 0.587, 0.114])
78 |
79 | @staticmethod
80 | def _preprocess(image):
81 | image = Environment._rgb2gray(image)
82 | image = misc.imresize(image, [Config.IMAGE_HEIGHT, Config.IMAGE_WIDTH], 'bilinear')
83 | image = image.astype(np.float32) / 128.0 - 1.0
84 | return image
85 |
86 | def _get_current_state(self):
87 | if not self.frame_q.full():
88 | return None # frame queue is not full yet.
89 | x_ = np.array(self.frame_q.queue)
90 | x_ = np.transpose(x_, [1, 2, 0]) # move channels
91 | return x_
92 |
93 | def _update_frame_q(self, frame):
94 | if self.frame_q.full():
95 | self.frame_q.get()
96 | image = Environment._preprocess(frame)
97 | self.frame_q.put(image)
98 |
99 | def get_num_actions(self):
100 | return self.game.env.action_space.n
101 |
102 | def reset(self):
103 | self.total_reward = 0
104 | self.frame_q.queue.clear()
105 | self._update_frame_q(self.game.reset())
106 | self.previous_state = self.current_state = None
107 |
108 | def step(self, action):
109 | observation, reward, done, _ = self.game.step(action)
110 |
111 | self.total_reward += reward
112 | self._update_frame_q(observation)
113 |
114 | self.previous_state = self.current_state
115 | self.current_state = self._get_current_state()
116 | return reward, done
117 |
--------------------------------------------------------------------------------
/Visual-Explanation-of-Atari/main.py:
--------------------------------------------------------------------------------
1 | import matplotlib
2 | matplotlib.use("TkAgg")
3 | from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg
4 | import matplotlib.pyplot as plt
5 | import matplotlib.animation as animation
6 | import tkinter as tk
7 |
8 | import numpy as np
9 | import argparse
10 |
11 | from env import Environment
12 | from network import Network
13 | from sailency import score_frame
14 |
15 | from config import Config
16 |
17 | FUDGE_FACTOR = 50
18 |
19 | class Experience(object):
20 | def __init__(self, state, action, prediction, reward, done):
21 | self.state = state
22 | self.action = action
23 | self.prediction = prediction
24 | self.reward = reward
25 | self.done = done
26 |
27 | if __name__ == "__main__":
28 | parser = argparse.ArgumentParser(description=None)
29 | parser.add_argument('-e', '--env', default='PongDeterministic-v0', type=str, help='gym environment')
30 | parser.add_argument('-m', '--mode', default='actor', type=str, help='mode of sailency')
31 | parser.add_argument('-d', '--density', default=5, type=int, help='density of grid of gaussian blurs')
32 | parser.add_argument('-r', '--radius', default=5, type=int, help='radius of gaussian blur')
33 | parser.add_argument('-f', '--num_frames', default=100, type=int, help='number of frames in movie')
34 | parser.add_argument('-i', '--first_frame', default=350, type=int, help='index of first frame')
35 | args = parser.parse_args()
36 |
37 | Config.ATARI_GAME = args.env
38 |
39 | env = Environment()
40 | network = Network("cpu:0", "network", env.get_num_actions())
41 | if args.env == 'PongDeterministic-v0':
42 | network.saver.restore(network.sess, './checkpoints/pong/network_00029000')
43 | elif args.env == 'BreakoutDeterministic-v0':
44 | network.saver.restore(network.sess, './checkpoints/breakout/network_00097000')
45 | else:
46 | raise NotImplementedError
47 |
48 | env.reset()
49 | done = False
50 | experiences = []
51 |
52 | while not done:
53 | # very first few frames
54 | if env.current_state is None:
55 | env.step(0) # 0 == NOOP
56 | continue
57 |
58 | prediction, value = network.predict_p_and_v_single(env.current_state)
59 | action = np.argmax(prediction)
60 | reward, done = env.step(action)
61 | exp = Experience(env.previous_state, action, prediction, reward, done)
62 | experiences.append(exp)
63 |
64 | frames = []
65 | perturbation_maps = []
66 | for frame_id in range(args.first_frame, args.first_frame + args.num_frames):
67 | sailency = score_frame(network, experiences, frame_id, args.radius, args.density, mode=args.mode)
68 | pmax = sailency.max()
69 |
70 | sailency -= sailency.min() ; sailency = FUDGE_FACTOR * pmax * sailency / sailency.max()
71 | frames.append(experiences[frame_id].state[:, :, 3])
72 | perturbation_maps.append(experiences[frame_id].state[:, :, 3] + sailency)
73 | print(' [ %d / %d ] processing perturbation_map ... ' % (frame_id - args.first_frame, args.num_frames))
74 |
75 | # Visualize
76 | fig = plt.Figure()
77 |
78 | root = tk.Tk()
79 |
80 | label = tk.Label(root, text="Video")
81 | label.grid(column=0, row=0)
82 |
83 | canvas = FigureCanvasTkAgg(fig, master=root)
84 | canvas.get_tk_widget().grid(column=0, row=1)
85 |
86 | ax_1 = fig.add_subplot(121)
87 | ax_2 = fig.add_subplot(122)
88 |
89 |
90 | def vedio(i):
91 | frame = frames.pop(0)
92 | frames.append(frame)
93 | ax_1.clear()
94 | ax_1.imshow(frame, vmin=0, vmax=1, cmap='gray')
95 | p_map = perturbation_maps.pop(0)
96 | perturbation_maps.append(p_map)
97 | ax_2.clear()
98 | ax_2.imshow(p_map, vmin=0, vmax=1, cmap='gray') #actor_sailency)
99 |
100 | ani = animation.FuncAnimation(fig, vedio, 1, interval=200)
101 | tk.mainloop()
102 |
103 |
104 |
--------------------------------------------------------------------------------
/Visual-Explanation-of-Atari/network.py:
--------------------------------------------------------------------------------
1 | import os
2 | import re
3 | import numpy as np
4 | import tensorflow as tf
5 |
6 | from config import Config
7 |
8 | class Network(object):
9 | def __init__(self, device, model_name, num_actions):
10 | self.device = device
11 | self.model_name = model_name
12 | self.num_actions = num_actions
13 |
14 | self.img_width = Config.IMAGE_WIDTH
15 | self.img_height = Config.IMAGE_HEIGHT
16 | self.img_channels = Config.STACKED_FRAMES
17 |
18 | self.graph = tf.Graph()
19 | with self.graph.as_default() as g:
20 | with tf.device(self.device):
21 | self.create_placeholder()
22 | self.create_network()
23 | # self.create_train_op()
24 | self.sess = tf.Session(
25 | graph=self.graph,
26 | config=tf.ConfigProto(
27 | allow_soft_placement=True,
28 | log_device_placement=False,
29 | gpu_options=tf.GPUOptions(allow_growth=True)))
30 | self.sess.run(tf.global_variables_initializer())
31 |
32 | vars = tf.trainable_variables()
33 | self.saver = tf.train.Saver({var.name: var for var in vars}, max_to_keep=0)
34 |
35 | def create_placeholder(self):
36 | self.x = tf.placeholder(
37 | tf.float32, [None, self.img_height, self.img_width, self.img_channels], name='X')
38 |
39 | def create_network(self):
40 | # As implemented in A3C paper
41 | self.n1 = self.conv2d_layer(self.x, 8, 16, 'conv11', strides=[1, 4, 4, 1])
42 | self.n2 = self.conv2d_layer(self.n1, 4, 32, 'conv12', strides=[1, 2, 2, 1])
43 | self.action_index = tf.placeholder(tf.float32, [None, self.num_actions])
44 | _input = self.n2
45 |
46 | flatten_input_shape = _input.get_shape()
47 | nb_elements = flatten_input_shape[1] * flatten_input_shape[2] * flatten_input_shape[3]
48 |
49 | self.flat = tf.reshape(_input, shape=[-1, nb_elements._value])
50 | self.d1 = self.dense_layer(self.flat, 256, 'dense1')
51 |
52 | self.logits_v = tf.squeeze(self.dense_layer(self.d1, 1, 'logits_v', func=None), axis=[1])
53 | self.logits_p = self.dense_layer(self.d1, self.num_actions, 'logits_p', func=None)
54 | self.softmax_p = tf.nn.softmax(self.logits_p)
55 |
56 | def dense_layer(self, input, out_dim, name, func=tf.nn.relu):
57 | in_dim = input.get_shape().as_list()[-1]
58 | d = 1.0 / np.sqrt(in_dim)
59 | with tf.variable_scope(name):
60 | w_init = tf.random_uniform_initializer(-d, d)
61 | b_init = tf.random_uniform_initializer(-d, d)
62 | w = tf.get_variable('w', dtype=tf.float32, shape=[in_dim, out_dim], initializer=w_init)
63 | b = tf.get_variable('b', shape=[out_dim], initializer=b_init)
64 |
65 | output = tf.matmul(input, w) + b
66 | if func is not None:
67 | output = func(output)
68 |
69 | return output
70 |
71 | def conv2d_layer(self, input, filter_size, out_dim, name, strides, func=tf.nn.relu):
72 | in_dim = input.get_shape().as_list()[-1]
73 | d = 1.0 / np.sqrt(filter_size * filter_size * in_dim)
74 | with tf.variable_scope(name):
75 | w_init = tf.random_uniform_initializer(-d, d)
76 | b_init = tf.random_uniform_initializer(-d, d)
77 | w = tf.get_variable('w',
78 | shape=[filter_size, filter_size, in_dim, out_dim],
79 | dtype=tf.float32,
80 | initializer=w_init)
81 | b = tf.get_variable('b', shape=[out_dim], initializer=b_init)
82 |
83 | output = tf.nn.conv2d(input, w, strides=strides, padding='SAME') + b
84 | if func is not None:
85 | output = func(output)
86 |
87 | return output
88 |
89 | def predict_p_and_v_single(self, x):
90 | p, v = self.sess.run([self.softmax_p, self.logits_v], feed_dict={self.x: x[np.newaxis, :]})
91 | return p[0], v[0]
92 |
93 | def _checkpoint_filename(self, episode):
94 | return 'checkpoints/%s_%08d' % (self.model_name, episode)
95 |
96 | def _get_episode_from_filename(self, filename):
97 | # TODO: hacky way of getting the episode. ideally episode should be stored as a TF variable
98 | return int(re.split('/|_|\.', filename)[2])
99 |
100 | # def load(self):
101 | # filename = tf.train.latest_checkpoint(os.path.dirname(self._checkpoint_filename(episode=0)))
102 | # print(filename)
103 | # self.saver.restore(self.sess, filename)
104 | # return self._get_episode_from_filename(filename)
--------------------------------------------------------------------------------
/Visual-Explanation-of-Atari/notebook.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "markdown",
5 | "metadata": {},
6 | "source": [
7 | "# Import"
8 | ]
9 | },
10 | {
11 | "cell_type": "code",
12 | "execution_count": 1,
13 | "metadata": {},
14 | "outputs": [],
15 | "source": [
16 | "import matplotlib\n",
17 | "matplotlib.use(\"TkAgg\")\n",
18 | "from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg\n",
19 | "import matplotlib.pyplot as plt\n",
20 | "import matplotlib.animation as animation\n",
21 | "import tkinter as tk\n",
22 | "\n",
23 | "import sys\n",
24 | "if sys.version_info >= (3,0):\n",
25 | " from queue import Queue\n",
26 | "else:\n",
27 | " from Queue import Queue\n",
28 | "\n",
29 | "import gym\n",
30 | "import numpy as np\n",
31 | "import scipy.misc as misc\n",
32 | "from scipy.ndimage.filters import gaussian_filter\n",
33 | "from scipy.misc import imresize\n",
34 | "import os\n",
35 | "import re\n",
36 | "import numpy as np\n",
37 | "import tensorflow as tf"
38 | ]
39 | },
40 | {
41 | "cell_type": "markdown",
42 | "metadata": {},
43 | "source": [
44 | "# Config"
45 | ]
46 | },
47 | {
48 | "cell_type": "code",
49 | "execution_count": 2,
50 | "metadata": {},
51 | "outputs": [],
52 | "source": [
53 | "class Config:\n",
54 | "\n",
55 | " #########################################################################\n",
56 | " # Game configuration\n",
57 | "\n",
58 | " # Name of the game, with version (e.g. PongDeterministic-v0)\n",
59 | " ATARI_GAME = 'PongDeterministic-v0'\n",
60 | "\n",
61 | " # Enable to see the trained agent in action\n",
62 | " PLAY_MODE = False\n",
63 | "\n",
64 | " # Input of the DNN\n",
65 | " STACKED_FRAMES = 4\n",
66 | " IMAGE_WIDTH = 84\n",
67 | " IMAGE_HEIGHT = 84\n",
68 | " \n",
69 | "MODE = 'actor'\n",
70 | " \n",
71 | "FIRST_FRAME = 350\n",
72 | "NUM_FRAMES = 100\n",
73 | "\n",
74 | "DENSITY = 5\n",
75 | "RADIUS = 5\n",
76 | "FUDGE_FACTOR = 50\n"
77 | ]
78 | },
79 | {
80 | "cell_type": "markdown",
81 | "metadata": {},
82 | "source": [
83 | "# Environment"
84 | ]
85 | },
86 | {
87 | "cell_type": "code",
88 | "execution_count": 3,
89 | "metadata": {},
90 | "outputs": [],
91 | "source": [
92 | "class GameManager:\n",
93 | " def __init__(self, game_name, display):\n",
94 | " self.game_name = game_name\n",
95 | " self.display = display\n",
96 | "\n",
97 | " self.env = gym.make(game_name)\n",
98 | " self.reset()\n",
99 | "\n",
100 | " def reset(self):\n",
101 | " observation = self.env.reset()\n",
102 | " return observation\n",
103 | "\n",
104 | " def step(self, action):\n",
105 | " self._update_display()\n",
106 | " observation, reward, done, info = self.env.step(action)\n",
107 | " return observation, reward, done, info\n",
108 | "\n",
109 | " def _update_display(self):\n",
110 | " if self.display:\n",
111 | " self.env.render()\n",
112 | "\n",
113 | "\n",
114 | "class Environment:\n",
115 | " def __init__(self):\n",
116 | " self.game = GameManager(Config.ATARI_GAME, display=Config.PLAY_MODE)\n",
117 | " self.nb_frames = Config.STACKED_FRAMES\n",
118 | " self.frame_q = Queue(maxsize=self.nb_frames)\n",
119 | " self.previous_state = None\n",
120 | " self.current_state = None\n",
121 | " self.total_reward = 0\n",
122 | "\n",
123 | " self.reset()\n",
124 | "\n",
125 | " @staticmethod\n",
126 | " def _rgb2gray(rgb):\n",
127 | " return np.dot(rgb[..., :3], [0.299, 0.587, 0.114])\n",
128 | "\n",
129 | " @staticmethod\n",
130 | " def _preprocess(image):\n",
131 | " image = Environment._rgb2gray(image)\n",
132 | " image = misc.imresize(image, [Config.IMAGE_HEIGHT, Config.IMAGE_WIDTH], 'bilinear')\n",
133 | " image = image.astype(np.float32) / 128.0 - 1.0\n",
134 | " return image\n",
135 | "\n",
136 | " def _get_current_state(self):\n",
137 | " if not self.frame_q.full():\n",
138 | " return None # frame queue is not full yet.\n",
139 | " x_ = np.array(self.frame_q.queue)\n",
140 | " x_ = np.transpose(x_, [1, 2, 0]) # move channels\n",
141 | " return x_\n",
142 | "\n",
143 | " def _update_frame_q(self, frame):\n",
144 | " if self.frame_q.full():\n",
145 | " self.frame_q.get()\n",
146 | " image = Environment._preprocess(frame)\n",
147 | " self.frame_q.put(image)\n",
148 | "\n",
149 | " def get_num_actions(self):\n",
150 | " return self.game.env.action_space.n\n",
151 | "\n",
152 | " def reset(self):\n",
153 | " self.total_reward = 0\n",
154 | " self.frame_q.queue.clear()\n",
155 | " self._update_frame_q(self.game.reset())\n",
156 | " self.previous_state = self.current_state = None\n",
157 | "\n",
158 | " def step(self, action):\n",
159 | " observation, reward, done, _ = self.game.step(action)\n",
160 | "\n",
161 | " self.total_reward += reward\n",
162 | " self._update_frame_q(observation)\n",
163 | "\n",
164 | " self.previous_state = self.current_state\n",
165 | " self.current_state = self._get_current_state()\n",
166 | " return reward, done\n"
167 | ]
168 | },
169 | {
170 | "cell_type": "markdown",
171 | "metadata": {},
172 | "source": [
173 | "# Network"
174 | ]
175 | },
176 | {
177 | "cell_type": "code",
178 | "execution_count": 4,
179 | "metadata": {},
180 | "outputs": [],
181 | "source": [
182 | "class Network(object):\n",
183 | " def __init__(self, device, model_name, num_actions):\n",
184 | " self.device = device \n",
185 | " self.model_name = model_name\n",
186 | " self.num_actions = num_actions\n",
187 | "\n",
188 | " self.img_width = Config.IMAGE_WIDTH\n",
189 | " self.img_height = Config.IMAGE_HEIGHT\n",
190 | " self.img_channels = Config.STACKED_FRAMES\n",
191 | "\n",
192 | " self.graph = tf.Graph()\n",
193 | " with self.graph.as_default() as g:\n",
194 | " with tf.device(self.device):\n",
195 | " self.create_placeholder()\n",
196 | " self.create_network()\n",
197 | " # self.create_train_op()\n",
198 | " self.sess = tf.Session(\n",
199 | " graph=self.graph,\n",
200 | " config=tf.ConfigProto(\n",
201 | " allow_soft_placement=True,\n",
202 | " log_device_placement=False,\n",
203 | " gpu_options=tf.GPUOptions(allow_growth=True)))\n",
204 | " self.sess.run(tf.global_variables_initializer())\n",
205 | "\n",
206 | " vars = tf.trainable_variables()\n",
207 | " self.saver = tf.train.Saver({var.name: var for var in vars}, max_to_keep=0)\n",
208 | "\n",
209 | " def create_placeholder(self):\n",
210 | " self.x = tf.placeholder(\n",
211 | " tf.float32, [None, self.img_height, self.img_width, self.img_channels], name='X')\n",
212 | "\n",
213 | " def create_network(self):\n",
214 | " # As implemented in A3C paper\n",
215 | " self.n1 = self.conv2d_layer(self.x, 8, 16, 'conv11', strides=[1, 4, 4, 1])\n",
216 | " self.n2 = self.conv2d_layer(self.n1, 4, 32, 'conv12', strides=[1, 2, 2, 1])\n",
217 | " self.action_index = tf.placeholder(tf.float32, [None, self.num_actions])\n",
218 | " _input = self.n2\n",
219 | "\n",
220 | " flatten_input_shape = _input.get_shape()\n",
221 | " nb_elements = flatten_input_shape[1] * flatten_input_shape[2] * flatten_input_shape[3]\n",
222 | "\n",
223 | " self.flat = tf.reshape(_input, shape=[-1, nb_elements._value])\n",
224 | " self.d1 = self.dense_layer(self.flat, 256, 'dense1')\n",
225 | "\n",
226 | " self.logits_v = tf.squeeze(self.dense_layer(self.d1, 1, 'logits_v', func=None), axis=[1])\n",
227 | " self.logits_p = self.dense_layer(self.d1, self.num_actions, 'logits_p', func=None)\n",
228 | " self.softmax_p = tf.nn.softmax(self.logits_p)\n",
229 | "\n",
230 | " def dense_layer(self, input, out_dim, name, func=tf.nn.relu):\n",
231 | " in_dim = input.get_shape().as_list()[-1]\n",
232 | " d = 1.0 / np.sqrt(in_dim)\n",
233 | " with tf.variable_scope(name):\n",
234 | " w_init = tf.random_uniform_initializer(-d, d)\n",
235 | " b_init = tf.random_uniform_initializer(-d, d)\n",
236 | " w = tf.get_variable('w', dtype=tf.float32, shape=[in_dim, out_dim], initializer=w_init)\n",
237 | " b = tf.get_variable('b', shape=[out_dim], initializer=b_init)\n",
238 | "\n",
239 | " output = tf.matmul(input, w) + b\n",
240 | " if func is not None:\n",
241 | " output = func(output)\n",
242 | "\n",
243 | " return output\n",
244 | "\n",
245 | " def conv2d_layer(self, input, filter_size, out_dim, name, strides, func=tf.nn.relu):\n",
246 | " in_dim = input.get_shape().as_list()[-1]\n",
247 | " d = 1.0 / np.sqrt(filter_size * filter_size * in_dim)\n",
248 | " with tf.variable_scope(name):\n",
249 | " w_init = tf.random_uniform_initializer(-d, d)\n",
250 | " b_init = tf.random_uniform_initializer(-d, d)\n",
251 | " w = tf.get_variable('w',\n",
252 | " shape=[filter_size, filter_size, in_dim, out_dim],\n",
253 | " dtype=tf.float32,\n",
254 | " initializer=w_init)\n",
255 | " b = tf.get_variable('b', shape=[out_dim], initializer=b_init)\n",
256 | "\n",
257 | " output = tf.nn.conv2d(input, w, strides=strides, padding='SAME') + b\n",
258 | " if func is not None:\n",
259 | " output = func(output)\n",
260 | "\n",
261 | " return output\n",
262 | "\n",
263 | " def predict_p_and_v_single(self, x):\n",
264 | " p, v = self.sess.run([self.softmax_p, self.logits_v], feed_dict={self.x: x[np.newaxis, :]})\n",
265 | " return p[0], v[0]\n",
266 | "\n",
267 | " def _checkpoint_filename(self, episode):\n",
268 | " return 'checkpoints/%s_%08d' % (self.model_name, episode)\n",
269 | "\n",
270 | " def _get_episode_from_filename(self, filename):\n",
271 | " # TODO: hacky way of getting the episode. ideally episode should be stored as a TF variable\n",
272 | " return int(re.split('/|_|\\.', filename)[2])"
273 | ]
274 | },
275 | {
276 | "cell_type": "markdown",
277 | "metadata": {},
278 | "source": [
279 | "# Sailency"
280 | ]
281 | },
282 | {
283 | "cell_type": "code",
284 | "execution_count": 5,
285 | "metadata": {},
286 | "outputs": [],
287 | "source": [
288 | "def occlude(img, mask):\n",
289 | " ret = np.zeros_like(img)\n",
290 | " for d in range(img.shape[2]):\n",
291 | " ret[:, :, d] = img[:, :, d] * (1 - mask) + gaussian_filter(img[:, :, d], sigma=3) * mask\n",
292 | " return ret\n",
293 | "\n",
294 | "def get_mask(center, size, r):\n",
295 | " y,x = np.ogrid[-center[0]:size[0]-center[0], -center[1]:size[1]-center[1]]\n",
296 | " keep = x*x + y*y <= 1\n",
297 | " mask = np.zeros(size) ; mask[keep] = 1 # select a circle of pixels\n",
298 | " mask = gaussian_filter(mask, sigma=r) # blur the circle of pixels. this is a 2D Gaussian for r=r^2=1\n",
299 | " return mask/mask.max()\n",
300 | "\n",
301 | "def score_frame(network, experiences, frame_id, radius, density, mode='actor'):\n",
302 | " # with original state\n",
303 | " if mode == 'actor':\n",
304 | " L, _ = network.predict_p_and_v_single(experiences[frame_id].state)\n",
305 | " elif mode == 'critic':\n",
306 | " _, L = network.predict_p_and_v_single(experiences[frame_id].state)\n",
307 | " scores = np.zeros((int(Config.IMAGE_HEIGHT / density) + 1, int(Config.IMAGE_WIDTH / density) + 1))\n",
308 | " for i in range(0, Config.IMAGE_HEIGHT, density):\n",
309 | " for j in range(0, Config.IMAGE_WIDTH, density):\n",
310 | " mask = get_mask(center=[i,j], size=[Config.IMAGE_HEIGHT, Config.IMAGE_WIDTH], r=radius)\n",
311 | " # with occluded state\n",
312 | " if mode == 'actor':\n",
313 | " l, _ = network.predict_p_and_v_single(occlude(experiences[frame_id].state, mask))\n",
314 | " elif mode == 'critic':\n",
315 | " _, l = network.predict_p_and_v_single(occlude(experiences[frame_id].state, mask))\n",
316 | " scores[int(i / density), int(j / density)] = np.square(L - l).sum() * 0.5\n",
317 | "\n",
318 | " pmax = scores.max()\n",
319 | " scores = imresize(scores, size=[Config.IMAGE_HEIGHT, Config.IMAGE_WIDTH], interp='bilinear').astype(np.float32)\n",
320 | " return pmax * scores / scores.max()"
321 | ]
322 | },
323 | {
324 | "cell_type": "markdown",
325 | "metadata": {},
326 | "source": [
327 | "# Visualize"
328 | ]
329 | },
330 | {
331 | "cell_type": "code",
332 | "execution_count": null,
333 | "metadata": {},
334 | "outputs": [
335 | {
336 | "name": "stdout",
337 | "output_type": "stream",
338 | "text": [
339 | "INFO:tensorflow:Restoring parameters from ./checkpoints/pong/network_00029000\n"
340 | ]
341 | },
342 | {
343 | "name": "stderr",
344 | "output_type": "stream",
345 | "text": [
346 | "/anaconda3/lib/python3.6/site-packages/ipykernel_launcher.py:41: DeprecationWarning: `imresize` is deprecated!\n",
347 | "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
348 | "Use ``skimage.transform.resize`` instead.\n",
349 | "/anaconda3/lib/python3.6/site-packages/ipykernel_launcher.py:32: DeprecationWarning: `imresize` is deprecated!\n",
350 | "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
351 | "Use ``skimage.transform.resize`` instead.\n"
352 | ]
353 | },
354 | {
355 | "name": "stdout",
356 | "output_type": "stream",
357 | "text": [
358 | " [ 0 / 100 ] processing perturbation_map ... \n",
359 | " [ 1 / 100 ] processing perturbation_map ... \n",
360 | " [ 2 / 100 ] processing perturbation_map ... \n",
361 | " [ 3 / 100 ] processing perturbation_map ... \n",
362 | " [ 4 / 100 ] processing perturbation_map ... \n",
363 | " [ 5 / 100 ] processing perturbation_map ... \n",
364 | " [ 6 / 100 ] processing perturbation_map ... \n",
365 | " [ 7 / 100 ] processing perturbation_map ... \n",
366 | " [ 8 / 100 ] processing perturbation_map ... \n",
367 | " [ 9 / 100 ] processing perturbation_map ... \n",
368 | " [ 10 / 100 ] processing perturbation_map ... \n",
369 | " [ 11 / 100 ] processing perturbation_map ... \n",
370 | " [ 12 / 100 ] processing perturbation_map ... \n",
371 | " [ 13 / 100 ] processing perturbation_map ... \n",
372 | " [ 14 / 100 ] processing perturbation_map ... \n",
373 | " [ 15 / 100 ] processing perturbation_map ... \n",
374 | " [ 16 / 100 ] processing perturbation_map ... \n",
375 | " [ 17 / 100 ] processing perturbation_map ... \n",
376 | " [ 18 / 100 ] processing perturbation_map ... \n",
377 | " [ 19 / 100 ] processing perturbation_map ... \n",
378 | " [ 20 / 100 ] processing perturbation_map ... \n",
379 | " [ 21 / 100 ] processing perturbation_map ... \n",
380 | " [ 22 / 100 ] processing perturbation_map ... \n",
381 | " [ 23 / 100 ] processing perturbation_map ... \n",
382 | " [ 24 / 100 ] processing perturbation_map ... \n",
383 | " [ 25 / 100 ] processing perturbation_map ... \n",
384 | " [ 26 / 100 ] processing perturbation_map ... \n",
385 | " [ 27 / 100 ] processing perturbation_map ... \n",
386 | " [ 28 / 100 ] processing perturbation_map ... \n",
387 | " [ 29 / 100 ] processing perturbation_map ... \n"
388 | ]
389 | }
390 | ],
391 | "source": [
392 | "class Experience(object):\n",
393 | " def __init__(self, state, action, prediction, reward, done):\n",
394 | " self.state = state\n",
395 | " self.action = action\n",
396 | " self.prediction = prediction\n",
397 | " self.reward = reward\n",
398 | " self.done = done\n",
399 | "\n",
400 | "env = Environment()\n",
401 | "network = Network(\"cpu:0\", \"network\", env.get_num_actions())\n",
402 | "if Config.ATARI_GAME == 'PongDeterministic-v0':\n",
403 | " network.saver.restore(network.sess, './checkpoints/pong/network_00029000')\n",
404 | "elif Config.ATARI_GAME == 'BreakoutDeterministic-v0':\n",
405 | " network.saver.restore(network.sess, './checkpoints/breakout/network_00097000')\n",
406 | "else:\n",
407 | " raise NotImplementedError\n",
408 | "\n",
409 | "env.reset()\n",
410 | "done = False\n",
411 | "experiences = []\n",
412 | "\n",
413 | "while not done:\n",
414 | " # very first few frames \n",
415 | " if env.current_state is None:\n",
416 | " env.step(0) # 0 == NOOP\n",
417 | " continue\n",
418 | "\n",
419 | " prediction, value = network.predict_p_and_v_single(env.current_state)\n",
420 | " action = np.argmax(prediction)\n",
421 | " reward, done = env.step(action)\n",
422 | " exp = Experience(env.previous_state, action, prediction, reward, done)\n",
423 | " experiences.append(exp)\n",
424 | "\n",
425 | "frames = []\n",
426 | "perturbation_maps = []\n",
427 | "for frame_id in range(FIRST_FRAME, FIRST_FRAME + NUM_FRAMES):\n",
428 | " sailency = score_frame(network, experiences, frame_id, RADIUS, DENSITY, mode=MODE)\n",
429 | " pmax = sailency.max()\n",
430 | "\n",
431 | " sailency -= sailency.min() ; sailency = FUDGE_FACTOR * pmax * sailency / sailency.max()\n",
432 | " frames.append(experiences[frame_id].state[:, :, 3])\n",
433 | " perturbation_maps.append(experiences[frame_id].state[:, :, 3] + sailency)\n",
434 | " print(' [ %d / %d ] processing perturbation_map ... ' % (frame_id - FIRST_FRAME, NUM_FRAMES))\n",
435 | "\n",
436 | "# Visualize\n",
437 | "fig = plt.Figure()\n",
438 | "\n",
439 | "root = tk.Tk()\n",
440 | "\n",
441 | "label = tk.Label(root, text=\"Video\")\n",
442 | "label.grid(column=0, row=0)\n",
443 | "\n",
444 | "canvas = FigureCanvasTkAgg(fig, master=root)\n",
445 | "canvas.get_tk_widget().grid(column=0, row=1)\n",
446 | "\n",
447 | "ax_1 = fig.add_subplot(121)\n",
448 | "ax_2 = fig.add_subplot(122)\n",
449 | "\n",
450 | "\n",
451 | "def vedio(i):\n",
452 | " frame = frames.pop(0)\n",
453 | " frames.append(frame)\n",
454 | " ax_1.clear()\n",
455 | " ax_1.imshow(frame, vmin=0, vmax=1, cmap='gray')\n",
456 | " p_map = perturbation_maps.pop(0)\n",
457 | " perturbation_maps.append(p_map)\n",
458 | " ax_2.clear()\n",
459 | " ax_2.imshow(p_map, vmin=0, vmax=1, cmap='gray') #actor_sailency)\n",
460 | "\n",
461 | "ani = animation.FuncAnimation(fig, vedio, 1, interval=200)\n",
462 | "tk.mainloop()\n",
463 | "\n",
464 | "\n"
465 | ]
466 | },
467 | {
468 | "cell_type": "code",
469 | "execution_count": null,
470 | "metadata": {},
471 | "outputs": [],
472 | "source": []
473 | }
474 | ],
475 | "metadata": {
476 | "kernelspec": {
477 | "display_name": "Python 3",
478 | "language": "python",
479 | "name": "python3"
480 | },
481 | "language_info": {
482 | "codemirror_mode": {
483 | "name": "ipython",
484 | "version": 3
485 | },
486 | "file_extension": ".py",
487 | "mimetype": "text/x-python",
488 | "name": "python",
489 | "nbconvert_exporter": "python",
490 | "pygments_lexer": "ipython3",
491 | "version": "3.6.5"
492 | }
493 | },
494 | "nbformat": 4,
495 | "nbformat_minor": 2
496 | }
497 |
--------------------------------------------------------------------------------
/Visual-Explanation-of-Atari/sailency.py:
--------------------------------------------------------------------------------
1 | import numpy as np
2 | from scipy.ndimage.filters import gaussian_filter
3 | from scipy.misc import imresize
4 |
5 | from config import Config
6 |
7 | def occlude(img, mask):
8 | ret = np.zeros_like(img)
9 | for d in range(img.shape[2]):
10 | ret[:, :, d] = img[:, :, d] * (1 - mask) + gaussian_filter(img[:, :, d], sigma=3) * mask
11 | return ret
12 |
13 | def get_mask(center, size, r):
14 | y,x = np.ogrid[-center[0]:size[0]-center[0], -center[1]:size[1]-center[1]]
15 | keep = x*x + y*y <= 1
16 | mask = np.zeros(size) ; mask[keep] = 1 # select a circle of pixels
17 | mask = gaussian_filter(mask, sigma=r) # blur the circle of pixels. this is a 2D Gaussian for r=r^2=1
18 | return mask/mask.max()
19 |
20 | def score_frame(network, experiences, frame_id, radius, density, mode='actor'):
21 | # with original state
22 | if mode == 'actor':
23 | L, _ = network.predict_p_and_v_single(experiences[frame_id].state)
24 | elif mode == 'critic':
25 | _, L = network.predict_p_and_v_single(experiences[frame_id].state)
26 | scores = np.zeros((int(Config.IMAGE_HEIGHT / density) + 1, int(Config.IMAGE_WIDTH / density) + 1))
27 | for i in range(0, Config.IMAGE_HEIGHT, density):
28 | for j in range(0, Config.IMAGE_WIDTH, density):
29 | mask = get_mask(center=[i,j], size=[Config.IMAGE_HEIGHT, Config.IMAGE_WIDTH], r=radius)
30 | # with occluded state
31 | if mode == 'actor':
32 | l, _ = network.predict_p_and_v_single(occlude(experiences[frame_id].state, mask))
33 | elif mode == 'critic':
34 | _, l = network.predict_p_and_v_single(occlude(experiences[frame_id].state, mask))
35 | scores[int(i / density), int(j / density)] = np.square(L - l).sum() * 0.5
36 |
37 | pmax = scores.max()
38 | scores = imresize(scores, size=[Config.IMAGE_HEIGHT, Config.IMAGE_WIDTH], interp='bilinear').astype(np.float32)
39 | return pmax * scores / scores.max()
--------------------------------------------------------------------------------