├── 01_linear_regression.ipynb ├── 02_MNIST_classification.ipynb ├── 03_cnn_image_classification.ipynb ├── 04_LSTM_sentiment_analysis.ipynb ├── 06_GAN_image_generation.ipynb ├── 08_few_shot_learning.ipynb ├── 101_mosaic_video.ipynb ├── 102_Wav2Lip_Inference.ipynb ├── 201_native_rag.ipynb ├── README.en.md ├── README.md └── asserts ├── images ├── 01_01.png ├── 01_02.png ├── 06_01.png ├── 101_01.png ├── 101_02.png ├── 101_03.png ├── 101_04.png ├── 101_05.png └── 101_06.png └── mp4 └── kunkun.mp4 /01_linear_regression.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "source": [ 6 | "Pytorch入门实战（1） - 实现线性回归" 7 | ], 8 | "metadata": { 9 | "collapsed": false, 10 | "pycharm": { 11 | "name": "#%% md\n" 12 | } 13 | } 14 | }, 15 | { 16 | "cell_type": "markdown", 17 | "metadata": { 18 | "collapsed": true, 19 | "pycharm": { 20 | "name": "#%% md\n" 21 | } 22 | }, 23 | "source": [ 24 | "# 涉及知识点\n", 25 | "[Pytorch nn.Module的基本使用](https://blog.csdn.net/zhaohongfei_358/article/details/122797244)\n", 26 | "\n", 27 | "[Pytorch nn.Linear的基本用法](https://blog.csdn.net/zhaohongfei_358/article/details/122797190)" 28 | ] 29 | }, 30 | { 31 | "cell_type": "markdown", 32 | "source": [ 33 | "# 将线性回归神经网络化\n", 34 | "\n", 35 | "线性回归也可以看作一个简单的神经网络。以一个特征的一元线性回归为例：\n", 36 | "\n", 37 | "$$\n", 38 | "y = w \\cdot x + b\n", 39 | "$$\n", 40 | "\n", 41 | "可以改造下图神经网络：\n", 42 | "\n", 43 | "\n", 44 | "\n", 45 | "若将x泛化为向量，即 $x=(x_1, x_2, ... , x_n)$，则对应神经网络为：\n", 46 | "\n", 47 | "" 48 | ], 49 | "metadata": { 50 | "collapsed": false, 51 | "pycharm": { 52 | "name": "#%% md\n" 53 | } 54 | } 55 | }, 56 | { 57 | "cell_type": "markdown", 58 | "source": [ 59 | "# Pytorch 代码实现\n", 60 | "\n", 61 | "## 一元线性回归Pytorch方式实现" 62 | ], 63 | "metadata": { 64 | "collapsed": false, 65 | "pycharm": { 66 | "name": "#%% md\n" 67 | } 68 | } 69 | }, 70 | { 71 | "cell_type": "code", 72 | "execution_count": 2, 73 | "outputs": [], 74 | "source": [ 75 | "import torch\n", 76 | "import matplotlib.pyplot as plt" 77 | ], 78 | "metadata": { 79 | "collapsed": false, 80 | "pycharm": { 81 | "name": "#%%\n" 82 | } 83 | } 84 | }, 85 | { 86 | "cell_type": "markdown", 87 | "source": [ 88 | "首先生成测试数据：" 89 | ], 90 | "metadata": { 91 | "collapsed": false, 92 | "pycharm": { 93 | "name": "#%% md\n" 94 | } 95 | } 96 | }, 97 | { 98 | "cell_type": "code", 99 | "execution_count": 3, 100 | "outputs": [ 101 | { 102 | "data": { 103 | "text/plain": "tensor([[7.2543],\n [0.8824],\n [8.1629]])" 104 | }, 105 | "execution_count": 3, 106 | "metadata": {}, 107 | "output_type": "execute_result" 108 | } 109 | ], 110 | "source": [ 111 | "X = torch.rand(100, 1) * 10 # 生成一个100行一列的数据；该数据服从[0,10]的uniform分布\n", 112 | "X[:3]" 113 | ], 114 | "metadata": { 115 | "collapsed": false, 116 | "pycharm": { 117 | "name": "#%%\n" 118 | } 119 | } 120 | }, 121 | { 122 | "cell_type": "code", 123 | "execution_count": 5, 124 | "outputs": [ 125 | { 126 | "data": { 127 | "text/plain": "tensor([[33.6710],\n [13.3356],\n [36.9041]])" 128 | }, 129 | "execution_count": 5, 130 | "metadata": {}, 131 | "output_type": "execute_result" 132 | } 133 | ], 134 | "source": [ 135 | "y = 3 * X + 10 + torch.randn(100, 1) * 3 # 计算其对应的y值；y也是100行1列的\n", 136 | "y[:3]" 137 | ], 138 | "metadata": { 139 | "collapsed": false, 140 | "pycharm": { 141 | "name": "#%%\n" 142 | } 143 | } 144 | }, 145 | { 146 | "cell_type": "markdown", 147 | "source": [ 148 | "将生成的数据绘制成散点图，看下效果：" 149 | ], 150 | "metadata": { 151 | "collapsed": false, 152 | "pycharm": { 153 | "name": "#%% md\n" 154 | } 155 | } 156 | }, 157 | { 158 | "cell_type": "code", 159 | "execution_count": 6, 160 | "outputs": [ 161 | { 162 | "data": { 163 | "text/plain": "
", 164 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAD4CAYAAAD1jb0+AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAcF0lEQVR4nO3dfYyd5Znf8e/Pw+wybDa1EQN1xrimETKbhcbujlJaSxWYbKGBBocq0UYCWW20zh9LS2hKdsgfhVW7YrqQl5VaRSIJjbeg1CggoJCWRTZsFLShHccuLzUoqw2wTKZ40jDdZGNljbn6x5xjzhw/Z87znPO8nvP7SNac85y3+5yQ67mf677u+1ZEYGZmzbOh6gaYmdlgHMDNzBrKAdzMrKEcwM3MGsoB3Mysoc4q88POO++82LZtW5kfaWbWeIcPH/5xREx3H08dwCVNAAvAYkRcJ+lO4LeB5dZTPh8R317vPbZt28bCwkL6VpuZGZJeSzqepQd+C3AMeG/HsS9FxD3DNMzMzAaTKgcuaQtwLfC1YptjZmZppR3E/DLwOeCdruM3S3pe0n2SNiW9UNI+SQuSFpaXl5OeYmZmA+gbwCVdBxyPiMNdD30FeD+wA1gCvpD0+oi4NyJmI2J2evqMHLyZmQ0oTQ58F/BRSR8BzgbeK+n+iLix/QRJXwUeL6iNZmaWoG8Aj4jbgdsBJF0B/OuIuFHS5ohYaj3tY8CLRTXSzKwMjxxZ5O4nX+FHKyd438Ypbrt6O3t2zlTdrJ6GqQP/A0k7gABeBT6dR4PMzKrwyJFFbn/4BU6cPAXA4soJbn/4BYDaBvFMATwingGead2+qYD2mJmllmeP+e4nXzkdvNtOnDzF3U++MhoB3MysLvLuMf9o5USm42kVmZbxWihm1kjr9ZgH8b6NU5mOp9E+ySyunCB49yTzyJHFgd+zkwO4mTVS3j3m267eztTkxJpjU5MT3Hb19oHeD/I/yXRzADezRsq7x7xn5wx33XAZMxunEDCzcYq7brhsqHRHUWmZNufAzayRbrt6+5ocOAzfY96zcybXAcv3bZxiMSFYD5OW6eQeuJk1UhE95rwVkZbp5B64mTVW3j3mvLXbVlQVigO4mVmBijzJOIViZtZQDuBmZg3lAG5m1lAO4GZmDeUAbmbWUA7gZmYN5QBuZtZQrgM3M8ugTrv2OICbmaVUt117nEIxM0up6OVhs3IANzNLqejlYbNKHcAlTUg6Iunx1v1zJT0l6Qetv5uKa6aZWfWK2LVnGFl64LcAxzruzwEHI+Ji4GDrvplZozxyZJFd84e4aO4Jds0fWne7s6KXh80q1SCmpC3AtcDvA/+qdfh64IrW7f2s7lb/u/k2z8xsOOtVjWQdlCx6edis0lahfBn4HPCrHccuiIglgIhYknR+0gsl7QP2AWzdunXwlpqZZdQvQK83KNkrKNdpDfK+KRRJ1wHHI+LwIB8QEfdGxGxEzE5PTw/yFmZmA+lXNVK3Qcms0vTAdwEflfQR4GzgvZLuB96UtLnV+94MHC+yoWZmWfUL0EXvWVm0vj3wiLg9IrZExDbgt4BDEXEj8Biwt/W0vcCjhbXSzGwA/apG6jYomdUwdeDzwG9K+gHwm637ZmalSFM90i9AN2Fj5PUoIkr7sNnZ2VhYWCjt88xsNHUPTsJqYE4KvnVau2RQkg5HxGz3ca+FYmaNk6V6pE5VI3nzVHoza5ymV4/kxQHczBqnblPaq+IAbmaN0/Tqkbw4B25mjVO3Ke1VcQA3s0Ya5cHJtJxCMTNrKPfAzUbcKNRBD6PX9x+F38UB3GyE1W0PxyRJgRTyyW/3+v4Lr/2Ehw4v1vp3ScMzMc1G2K75Q4mLNc1snOLZud0VtGitpBmVkxsEgpOn3o1NvWZZ9tPr+09InEqIfXX5Xbr1monpHLjZCKv7hJekGZUn34k1wRsG3zi41/dMCt7rPb+unEIxG2FVL5faL8+cJWAOElx7ff9ePfCmTQRyD9xshFU54aWdHllcOUHwbp65c9XALAGz+7nDrEb4yb934UhMBHIANxthVS6X2m83HEgOsJMbxOSE1hzrDq5pTg7Q+/v/uz2XNXoZ2TYPYppZIS6ae4Kk6CLgh/PXnr4/SBVK3Qdn8+blZM2sVGnz771mVK7XG6774GxZnEIxs0IUmX/3aoSr3AM3s0LkteBUO8WyuHLidPXIxqlJJid0Rq140wYhh+UAbmaFGXbBqe6JPu3Sv5UTJ5ncIDadM8nKz082dir8sPoGcElnA98Bfrn1/G9FxB2S7gR+G1huPfXzEfHtohpqZuMnqZKl7eQ7wTm/dBZH/s0/KrlV9ZGmB/4LYHdE/EzSJPBdSf+t9diXIuKe4ppnZnVX5KJQ/QYlx23QslvfQcxY9bPW3cnWv/JqD82sttLWYw+q36DkuA1adktVhSJpQtJR4DjwVEQ813roZknPS7pP0qYer90naUHSwvLyctJTzKyh0kzW6ZZmBmVbUiVL2zgOWnZLFcAj4lRE7AC2AB+SdCnwFeD9wA5gCfhCj9feGxGzETE7PT2dS6PNrB6y1mNn6bG3UzMnTp5iQqszM9t/i5o5meXkUgeZqlAiYkXSM8A1nblvSV8FHs+5bWZWc1kXy1qvx94ZjJOqTwZdUjatJqyd3q1vD1zStKSNrdtTwIeBlyVt7njax4AXC2mhmVVimMWiutctab9PUrCHM3vsg6RmhlXFZw4rTQ98M7Bf0gSrAf/BiHhc0n+WtIPVAc1XgU8X1kozK80jRxa587GXWDlx8vSxXr3RfpN1kjZsSBKsrm9y5SXTPP3ycupAn6cmTs/vG8Aj4nlgZ8LxmwppkZlVZr2Am5TqgPUn66xXx91tceUE93/v9XWfU2TVSdVrpw/Ca6GYjYC8Bt/6BdysvdE8e69FV51UuXb6oBzAzRouz1rsfgE3a2+01/Pb63CnVcZ63VWunT4or4Vi1nBpKzvS6JVGgMF6o7ddvf3MTYsnxF/94u3UswHLXON72LVbyuYeuFnD5Tn41mvizKZzJgfqjXb3ajedMwnBmgHS9dQ9hVE198DNGi7Pwbe8loDtfs/263fNH+KtnycH75mNU6erUIpYV2VYRa75MigHcLOGS0pTDNNzLTKN0OuqQFDrrdDqOsnHKRSzhstj8K2sKeRN3UmnrpN83AM3GwHD9JrL7F3mfbVQlrpO8nEP3GzMDdq7HKTX3sRSPajvlYN74GZjbpDe5TC99qaV6kF9rxzcAzdrkCJy1YP0Lnv12u987KWh21NHdb1ycA/crCGKylUP0rvs1TtfOXGSR44sVh7YilDHKwf3wM0aoqhKiEF6l/1651YO98DNGqLISoisvcvbrt7OZw4cLaw9lo574GYNUadKiD07Z1anxSeoujJjnDiAmzVE3ZY7veOf/Hqt2jOOnEIxa4gi1ikZpfaMI0WkXdRxeLOzs7GwsFDa55kNo46LF9l4knQ4Ima7j6fZ1PhsSf9D0v+S9JKk32sdP1fSU5J+0Pq7qYiGm1Uhz00SzIqSJgf+C2B3RHwQ2AFcI+lyYA44GBEXAwdb981GQl0XLzLr1DeAx6qfte5Otv4FcD2wv3V8P7CniAaaVaGuixeZdUpVhSJpQtJR4DjwVEQ8B1wQEUsArb/n93jtPkkLkhaWl5dzarZZsepUsmfWS6oAHhGnImIHsAX4kKRL035ARNwbEbMRMTs9PT1gM83KVbeSvW5lrd9t9ZapjDAiViQ9A1wDvClpc0QsSdrMau/cbCTUuUSurrvDWPn6BnBJ08DJVvCeAj4M/HvgMWAvMN/6+2iRDTUrWx0XL4J8d6G3ZkvTA98M7Jc0wWrK5cGIeFzSnwIPSvoU8Drw8QLbaWYtHmC1tr4BPCKeB3YmHP+/wFVFNMosiSfWrMpzF3prNq+FYo3giTXvqvsAq5XHa6FYI9Qx71vVFUGdB1itXA7g1gh1y/tWWQlSdSqp6s+3dzmAWyMMkvfNO9B0vt8GiVNdC8GVcUVQdQlh1Z9vazkHbo2QNe+bd868+/26g3db0VcEVa/RUvXn21rugVsjZM375p0zT3q/JN1XBHlfBVSdSqr6820tB3BrjCwTa/IONGle131FUES6oeoSwqo/39ZyCsVqbdA1P/JejKrX6yaknju5F5FuqLqEsOrPt7XcA7faGqYHe9vV29e8FoYLNL3erztodyoi3VB1CWHVn29rOYBbbQ2Tx8470AzyfkWlG6peo6Xqz7d3OYBbbQ3bg8070PR7v+4Byysvmeahw4u5XQWYdXMAt9pq0oBZUrrnocOL/NPfmOHpl5dzuQrwBBrr5gButZV3HrtIvdI9T7+8zLNzu4d+f0+gsSSuQrHa2rNzhrtuuIyZjVM9Kz3qouj6aE+gsSTugVutNWXArOh0jyfQWBL3wM1yUHR9tDdZtiQO4GY5KDrd4wk0lsQpFLOcFJnu8QQaS+IAbtYQTRkPsPL0TaFIulDS05KOSXpJ0i2t43dKWpR0tPXvI8U318zM2tL0wN8GPhsR35f0q8BhSU+1HvtSRNxTXPPMzKyXNLvSLwFLrds/lXQM8HWcjSTPdrQmyVSFImkbsBN4rnXoZknPS7pP0qYer9knaUHSwvLy8nCtNStQ3rv4mBUtdQCX9B7gIeAzEfGXwFeA9wM7WO2hfyHpdRFxb0TMRsTs9PT08C02K4hnO1rTpArgkiZZDd4PRMTDABHxZkScioh3gK8CHyqumWbF82xHa5o0VSgCvg4ci4gvdhzf3PG0jwEv5t88s/J4tqM1TZoe+C7gJmB3V8ngH0h6QdLzwJXArUU21Kxonu1oTZOmCuW7gBIe+nb+zTGrThmzHV3lYnnyTEyzDkXOdvSa3pY3B/CGcQ+uuYbZ49MsiQN4g+TZg/OJoHyucrG8eTnZBsmrTrluE1YeObLIrvlDXDT3BLvmD43sxBlXuVjeHMAbJK8eXNYTQZEBtm4nkyK5ysXy5hRKg+S1bVeWE8EgaZss6Zk65IXLSid5TW/LmwN4g+S1S3uWE0HWAJsU8G89cJTPHDjKTELAqjovXHZliNf0tjw5gDdIrx4cwK75Q6l7dVlOBFkDbFLAj9bfpOBY9GbASTp73BskTkWsedyVIdYUDuAN092DG6QHmeVSPmuA7ddz7g6OeV1VpE2DdP9e3cE77fcwqwMH8IYbNIec9lI+a4DtFfA7dQbH7pPJ35iaRIJbDxzl7idfSZUjznISS/q9en0Ps7pzAG+4onPIWQfekgJ+t+7g2D6ZDJqP7ldV09n2ficXcGWINYcDeMOVkUPOMvDWGfAXV04g3s2Bw/rBcdCriV4nq/YJoPOE0N2etgmJdyJcGWKN4gDecHnlkNu6c8lXXjLN0y8vZyp76wz4WUr0Br2a6HUSm5ASB1STTip33XDZ6auAu598hVsPHHUwt9pzAG+4PGuLk1IY93/v9dOPD1Jil6X3PujVRK+TWK80TgAzG6fO+L282JQ1jQP4CEgTJNP0hNMM8BVZYjfo1USvk1g7jdNtZuMUz87tPuN4HSYVmWXhAD4G0vYs0w58FlViN8zVRK+TWJYTQtWTisyycgAfA2l7lmmrNIosscuScul3VZH1hFDFpCKzYTiAj4G0Pcs0JYB1KbFLe1WR5YSQ94CwWdG8GuEYSLuM6Z6dM9x1w2XMbJxCrOaKb7x865r77WqNquW1tG6npO9fl+9rlqRvD1zShcAfAX8TeAe4NyL+UNK5wAFgG/Aq8ImIeKu4ptqgsvQsm7LYUlH56qZ8fzNI1wN/G/hsRPwacDnwO5I+AMwBByPiYuBg677V0Cj2LNNeVYzLZhE2ntLsSr8ELLVu/1TSMWAGuB64ovW0/cAzwO8W0kob2qj1LNNcVbiu20ZdpkFMSduAncBzwAWt4E5ELEk6v8dr9gH7ALZu3TpUY22tcd7XMk2Fieu6bdQpeiynecYTpfcAfwL8fkQ8LGklIjZ2PP5WRGxa7z1mZ2djYWFhmPZaS3fvEtZOCTe4aO6JxHVPIHkmplldSTocEbPdx1NVoUiaBB4CHoiIh1uH35S0ufX4ZuB4Xo21/oqowhg1vfLkgrHYg9NGX98ALknA14FjEfHFjoceA/a2bu8FHs2/edaLZw32l7SJcNJqhD7xWVOlyYHvAm4CXpB0tHXs88A88KCkTwGvAx8vpIVjql9+e1xmDQ6T50/Kk/eaaeoTnzVRmiqU77LacUlyVb7NMUhXPTEOswbzqCLprr7ZNX9oLE58Nh48E7OG0uS3y6ztrqqWuog8f1JaZdROfDY+vBZKDaXNb5dR211lLXURef481083q5oDeA3lmd8eJIfc+ZoN0hk7t5dVS11Unn/UJjXZ+HIKpYayXOavl95o956zlMx1v6Y7eLeVMejndIfZ+hzAayhtfrtfgB4kh5xmVx4oZ9BvFNdwMcuTUyg1leYyv99U8UFyyGl61mX2gp3uMOvNPfAG6xeg067Yl+axCcm9YLOacQBvsH4BepAccq/XfOETH+SH89fy7NxuB2+zmnAAb7B+AXqQHLLzzmbN4Rx4gxVV0+y8s1kzOIA3XK9g+8iRRX7vv77EWz8/efqYNzQwGy0O4BUrYlOGpLXC27yhgdnocAAvSJrAXNQ09X613MNMwhnnXYDM6sYBvABpA3NRW371C9BZJ+G0g/biyok162k7JWNWLVehFCDtDMiiNmVYL0BnnYTTOdsTvBmCWZ00LoBXtbRpFmkD8yATbdJIKi8EOGdyA7981gZuPXA09W+XZmq9N0Mwq0ajAvggizNVIW1gvu3q7UxuWLtXxuQGDT1NPamW+8bLtxKIlRMnM/12aYKzN0Mwq0ajAnhTNvLNNAOye6+jXnsfZbRn5wzPzu0+PXvy6ZeXB/rt+gVnrw5oVp00mxrfJ+m4pBc7jt0paVHS0da/jxTbzFVN2cg37WzGu598hZOn1maVT56KQk5Ig/52vTYGBs/SNKtamiqUbwD/AfijruNfioh7cm/ROpq0kW+a2YxlnpAG/e28g41ZfaXZ1Pg7kraV0Ja+Rm0j3zJPSMP8dp5ab1ZPw+TAb5b0fCvFsim3Fq2jTgst5VENU+aOM3X67cwsH4oeW2atedJqD/zxiLi0df8C4MeslgX/W2BzRPzzHq/dB+wD2Lp162+89tprmRo47My/sqaqT01ODBQQy5jZ2NTZk01tt1neJB2OiNkzjg8SwNM+1m12djYWFhZSNRiGD5RJr2/PJJwZIiDsmj+UmPqY2TjFs3O7M79fkfI82bTfr4ygmne7zZqsVwAfKIUiaXPH3Y8BL/Z67jCGLRtMen33NPBBUh95Dz4WOTkpz9LLMuvwm1IyalalvoOYkr4JXAGcJ+kN4A7gCkk7WI2HrwKfLqJxwwbKfs8bdN2RPAYfy1pfJM+TTVFrtyRpSsmoWZX69sAj4pMRsTkiJiNiS0R8PSJuiojLIuLvRMRHI2KpiMYNO9U8zfMGCQjrDT6m6U2Xub5IntP1yy57zHLcbBzVeibmsFUavdYE6ZQmIHQHZSCxogNIlWIoc32RPCtdygyqZVbomDVVrZeTHXYSSefru1MVsBoQrrxkml3zh3q+f6+lYe+64bIzBix3zR9KlWIoc32RPCfilFmH7wlEZv2lqkLJS9YqlLx1V1Bceck0Dx1eXLfSIUvFyUVzT5yRDoHVypcfzl97+n6v9+zVhjpxaZ9Z+XpVodS6B76eQQJJ94zCND3mLHnftIObST3ZPMoby+BZmWb10cgAntdWZGmCc5aKk7QpBqcHzCwPjQzgeZWzpQnOWfK+SYH5ykumufvJV7j1wNE1gdo9WTMbVq2rUHrJq5wtTaVDljVEeuXY674BhZk1UyN74Hmt4pc2lZGmt5yU1nnge6/3rPF279vMhtXIAJ5nOVteqYz1pu1382xCM8tDIwN4HQcBswRlzyY0szw0MoBD/crZeqV1uk1ODL9psZkZNHQQs46SBkQnN4gN3ZsUlzdvysxGnAN4TpKqVd5z9lm80xWwT75TzKbFZjZ+GptCqaPutM5Fc08kPs+DmGaWB/fAC+QlUc2sSA7gBfKSqGZWJKdQClTHckczGx0O4AWrW7mjmY0Op1DMzBoqzabG9wHXAccj4tLWsXOBA8A2Vjc1/kREvFVcM9fnTQbMbByl6YF/A7im69gccDAiLgYOtu5XonODYK/4Z2bjJM2u9N8BftJ1+Hpgf+v2fmBPvs1Kb721wc3MRtmgg5gXRMQSQEQsSTq/1xMl7QP2AWzdunXAj+str7XBh+U0jpmVrfBBzIi4NyJmI2J2eno69/evw2QZp3HMrAqDBvA3JW0GaP09nl+TsqnDZBmnccysCoMG8MeAva3be4FH82lOdlm2PCtKXdI4ZjZe0pQRfhO4AjhP0hvAHcA88KCkTwGvAx8vspH9VD1ZJq8t3szMsugbwCPikz0euirntjRWnlu8mZml5an0OfCaJ2ZWhZEO4GWW9lWdxjGz8TOyAbxd2tdOa7RL+wAHWjMbCSO7mJVL+8xs1I1sAHdpn5mNupEN4HWYoWlmVqSRDeB1mKFpZlakkR3EdGmfmY26kQ3g4NI+MxttI5tCMTMbdQ7gZmYN5QBuZtZQDuBmZg3lAG5m1lCKiPI+TFoGXsvwkvOAHxfUnDrz9x4f4/idwd87q78VEWfsSVlqAM9K0kJEzFbdjrL5e4+PcfzO4O+d1/s5hWJm1lAO4GZmDVX3AH5v1Q2oiL/3+BjH7wz+3rmodQ7czMx6q3sP3MzMenAANzNrqNoGcEnXSHpF0p9Jmqu6PWWQdKGkpyUdk/SSpFuqblNZJE1IOiLp8arbUhZJGyV9S9LLrf/N/37VbSqDpFtb/32/KOmbks6uuk1FkHSfpOOSXuw4dq6kpyT9oPV30zCfUcsALmkC+I/APwY+AHxS0geqbVUp3gY+GxG/BlwO/M6YfG+AW4BjVTeiZH8I/PeIuAT4IGPw/SXNAP8SmI2IS4EJ4LeqbVVhvgFc03VsDjgYERcDB1v3B1bLAA58CPiziPjziPhr4L8A11fcpsJFxFJEfL91+6es/h965Bc0l7QFuBb4WtVtKYuk9wL/EPg6QET8dUSsVNqo8pwFTEk6CzgH+FHF7SlERHwH+EnX4euB/a3b+4E9w3xGXQP4DPAXHfffYAwCWSdJ24CdwHMVN6UMXwY+B7xTcTvK9LeBZeA/tVJHX5P0K1U3qmgRsQjcA7wOLAH/LyL+uNpWleqCiFiC1Q4bcP4wb1bXAK6EY2NT7yjpPcBDwGci4i+rbk+RJF0HHI+Iw1W3pWRnAX8X+EpE7AT+iiEvp5uglfO9HrgIeB/wK5JurLZVzVXXAP4GcGHH/S2M6GVWN0mTrAbvByLi4arbU4JdwEclvcpqqmy3pPurbVIp3gDeiIj2Fda3WA3oo+7DwA8jYjkiTgIPA/+g4jaV6U1JmwFaf48P82Z1DeD/E7hY0kWSfonVQY7HKm5T4SSJ1ZzosYj4YtXtKUNE3B4RWyJiG6v/Ox+KiJHvkUXE/wH+QtL21qGrgP9dYZPK8jpwuaRzWv+9X8UYDN52eAzY27q9F3h0mDer5abGEfG2pJuBJ1kdpb4vIl6quFll2AXcBLwg6Wjr2Ocj4tvVNckK9C+AB1qdlD8H/lnF7SlcRDwn6VvA91mtujrCiE6rl/RN4ArgPElvAHcA88CDkj7F6sns40N9hqfSm5k1U11TKGZm1ocDuJlZQzmAm5k1lAO4mVlDOYCbmTWUA7iZWUM5gJuZNdT/BzCmjV6J/KLLAAAAAElFTkSuQmCC\n" 165 | }, 166 | "metadata": { 167 | "needs_background": "light" 168 | }, 169 | "output_type": "display_data" 170 | } 171 | ], 172 | "source": [ 173 | "plt.scatter(X.numpy(), y.numpy())\n", 174 | "plt.show()" 175 | ], 176 | "metadata": { 177 | "collapsed": false, 178 | "pycharm": { 179 | "name": "#%%\n" 180 | } 181 | } 182 | }, 183 | { 184 | "cell_type": "markdown", 185 | "source": [ 186 | "接下来定义线性回归预训练模型：" 187 | ], 188 | "metadata": { 189 | "collapsed": false, 190 | "pycharm": { 191 | "name": "#%% md\n" 192 | } 193 | } 194 | }, 195 | { 196 | "cell_type": "code", 197 | "execution_count": 9, 198 | "outputs": [], 199 | "source": [ 200 | "class LinearRegression(torch.nn.Module):\n", 201 | " \"\"\"\n", 202 | " 模型需要继承 `torch.nn.Module`，在Pytorch中，模型都需要继承该类\n", 203 | " \"\"\"\n", 204 | "\n", 205 | " def __init__(self):\n", 206 | " super().__init__() # 初始化Module类\n", 207 | "\n", 208 | " \"\"\"\n", 209 | " 定义我们神经网络的第一层（线性层）。其接受的重要三个参数：\n", 210 | " in_features: 输入神经元的个数\n", 211 | " out_features：输出神经元的个数\n", 212 | " bias：是否包含偏置\n", 213 | "\n", 214 | "\t\t更多，关于torch.nn.Linear，可以参考：https://pytorch.org/docs/stable/nn.html#linear-layers\n", 215 | " \"\"\"\n", 216 | " self.linear = torch.nn.Linear(in_features=1, out_features=1, bias=True)\n", 217 | "\n", 218 | " def forward(self, x):\n", 219 | " \"\"\"\n", 220 | " 前向传播计算神经网络的输出\n", 221 | " \"\"\"\n", 222 | " predict = self.linear(x)\n", 223 | " return predict" 224 | ], 225 | "metadata": { 226 | "collapsed": false, 227 | "pycharm": { 228 | "name": "#%%\n" 229 | } 230 | } 231 | }, 232 | { 233 | "cell_type": "markdown", 234 | "source": [ 235 | "到这里预训练模型已经构建完毕。初始化预训练模型：" 236 | ], 237 | "metadata": { 238 | "collapsed": false, 239 | "pycharm": { 240 | "name": "#%% md\n" 241 | } 242 | } 243 | }, 244 | { 245 | "cell_type": "code", 246 | "execution_count": 10, 247 | "outputs": [], 248 | "source": [ 249 | "model = LinearRegression() # 初始化模型" 250 | ], 251 | "metadata": { 252 | "collapsed": false, 253 | "pycharm": { 254 | "name": "#%%\n" 255 | } 256 | } 257 | }, 258 | { 259 | "cell_type": "markdown", 260 | "source": [ 261 | "定义梯度下降器，这里选择随机梯度下降法：" 262 | ], 263 | "metadata": { 264 | "collapsed": false, 265 | "pycharm": { 266 | "name": "#%% md\n" 267 | } 268 | } 269 | }, 270 | { 271 | "cell_type": "code", 272 | "execution_count": 11, 273 | "outputs": [ 274 | { 275 | "name": "stdout", 276 | "output_type": "stream", 277 | "text": [ 278 | "Parameter containing:\n", 279 | "tensor([[-0.7801]], requires_grad=True)\n", 280 | "Parameter containing:\n", 281 | "tensor([0.3026], requires_grad=True)\n" 282 | ] 283 | } 284 | ], 285 | "source": [ 286 | "\"\"\"\n", 287 | "torch.optim.SGD 接受几个重要的参数：\n", 288 | "- params: 模型参数\n", 289 | "- lr: 学习率\n", 290 | "\"\"\"\n", 291 | "optimizer = torch.optim.SGD(model.parameters(), lr=1e-3)\n", 292 | "\n", 293 | "# 这里可以看下模型参数\n", 294 | "for param in model.parameters(): # 因为模型有多个参数，所以model.parameters会返回一个可迭代的对象\n", 295 | " print(param)" 296 | ], 297 | "metadata": { 298 | "collapsed": false, 299 | "pycharm": { 300 | "name": "#%%\n" 301 | } 302 | } 303 | }, 304 | { 305 | "cell_type": "markdown", 306 | "source": [ 307 | "定义损失函数，这里使用MSE：" 308 | ], 309 | "metadata": { 310 | "collapsed": false, 311 | "pycharm": { 312 | "name": "#%% md\n" 313 | } 314 | } 315 | }, 316 | { 317 | "cell_type": "code", 318 | "execution_count": 12, 319 | "outputs": [], 320 | "source": [ 321 | "loss_function = torch.nn.MSELoss()" 322 | ], 323 | "metadata": { 324 | "collapsed": false, 325 | "pycharm": { 326 | "name": "#%%\n" 327 | } 328 | } 329 | }, 330 | { 331 | "cell_type": "markdown", 332 | "source": [ 333 | "此时就可以训练模型了：" 334 | ], 335 | "metadata": { 336 | "collapsed": false, 337 | "pycharm": { 338 | "name": "#%% md\n" 339 | } 340 | } 341 | }, 342 | { 343 | "cell_type": "code", 344 | "execution_count": 14, 345 | "outputs": [], 346 | "source": [ 347 | "for epoch in range(10000): # 训练10000次\n", 348 | " \"\"\"\n", 349 | " 1. 将X带入模型，其会自动调用前向传递，计算出每个x对应的y值\n", 350 | " X.shape 和 predict_y.shape 都为(100,1),\n", 351 | " \"\"\"\n", 352 | " predict_y = model(X)\n", 353 | "\n", 354 | " \"\"\"\n", 355 | " 2. 通过损失函数计算损失\n", 356 | " \"\"\"\n", 357 | " loss = loss_function(predict_y, y)\n", 358 | "\n", 359 | " \"\"\"\n", 360 | " 3. 进行反向传播\n", 361 | " \"\"\"\n", 362 | " loss.backward()\n", 363 | "\n", 364 | " \"\"\"\n", 365 | " 4. 更新权重\n", 366 | " \"\"\"\n", 367 | " optimizer.step()\n", 368 | "\n", 369 | " \"\"\"\n", 370 | " 5.清空optimizer的梯度，否则会影响下次迭代\n", 371 | " \"\"\"\n", 372 | " optimizer.zero_grad()" 373 | ], 374 | "metadata": { 375 | "collapsed": false, 376 | "pycharm": { 377 | "name": "#%%\n" 378 | } 379 | } 380 | }, 381 | { 382 | "cell_type": "markdown", 383 | "source": [ 384 | "看下最后的参数，结果符合预期：" 385 | ], 386 | "metadata": { 387 | "collapsed": false, 388 | "pycharm": { 389 | "name": "#%% md\n" 390 | } 391 | } 392 | }, 393 | { 394 | "cell_type": "code", 395 | "execution_count": 16, 396 | "outputs": [ 397 | { 398 | "name": "stdout", 399 | "output_type": "stream", 400 | "text": [ 401 | "Parameter containing:\n", 402 | "tensor([[3.0832]], requires_grad=True)\n", 403 | "Parameter containing:\n", 404 | "tensor([9.7287], requires_grad=True)\n" 405 | ] 406 | } 407 | ], 408 | "source": [ 409 | "for param in model.parameters(): # 因为模型有多个参数，所以model.parameters会返回一个可迭代的对象\n", 410 | " print(param)" 411 | ], 412 | "metadata": { 413 | "collapsed": false, 414 | "pycharm": { 415 | "name": "#%%\n" 416 | } 417 | } 418 | }, 419 | { 420 | "cell_type": "markdown", 421 | "source": [ 422 | "再重新绘制一下图，看下最终效果：" 423 | ], 424 | "metadata": { 425 | "collapsed": false, 426 | "pycharm": { 427 | "name": "#%% md\n" 428 | } 429 | } 430 | }, 431 | { 432 | "cell_type": "code", 433 | "execution_count": 17, 434 | "outputs": [ 435 | { 436 | "data": { 437 | "text/plain": "
", 438 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAD4CAYAAAD1jb0+AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAixklEQVR4nO3df5RcZZ3n8fc3TWs6/AocGg0dYmfVDbpkJNnWVdudJQEFJUibObh6DojKMaxjXOAopmEdYY8yaX7rOsrZiIxxZBwYxSY2KjJpWI5x4ND5cfhhyFEhQJpAWiFqoMcJ6e/+UVVJddWtqnur7q26t/rzOicnXberbj1Vge997vf5Ps9j7o6IiGTPrFY3QERE6qMALiKSUQrgIiIZpQAuIpJRCuAiIhl1WDPf7LjjjvPe3t5mvqWISOZt3rz5d+7eXXo8dAA3sw5gDBh39xVmdhXwKWAi/5Qr3P0n1c7R29vL2NhY+FaLiAhm9nTQ8Sg98IuB7cBRRcducvfrG2mYiIjUJ1QO3MzmA2cBtyTbHBERCSvsIOZXgS8AUyXHV5vZI2Z2q5kdE/RCM1tlZmNmNjYxMRH0FBERqUPNAG5mK4A97r655Fc3A28ETgF2AzcEvd7d17l7n7v3dXeX5eBFRKROYXLg/cAHzewDwGzgKDP7nrufV3iCmX0LGEmojSIiEqBmAHf3y4HLAczsVODz7n6emc1z9935p30IeCypRoqINMPw1nGuu2cHz+2d5IS5XVx2xiIGlvS0ulkVNVIHfq2ZnQI4sBO4KI4GiYi0wvDWcS6/81Em9x8AYHzvJJff+ShAaoN4pADu7vcD9+d/Pj+B9oiIhBZnj/m6e3YcDN4Fk/sPcN09O9ojgIuIpEXcPebn9k5GOh5WkmkZrYUiIplUrcdcjxPmdkU6HkbhIjO+dxLn0EVmeOt43ecspgAuIpkUd4/5sjMW0dXZMe1YV2cHl52xqK7zQfwXmVIK4CKSSXH3mAeW9LB25WJ65nZhQM/cLtauXNxQuiOptEyBcuAikkmXnbFoWg4cGu8xDyzpiXXA8oS5XYwHBOtG0jLF1AMXkUxKoscctyTSMsXUAxeRzIq7xxy3QtuSqkJRABcRSVCSFxmlUEREMkoBXEQkoxTARUQySgFcRCSjFMBFRDJKAVxEJKMUwEVEMkp14CIiEaRp1x4FcBGRkNK2a49SKCIiISW9PGxUCuAiIiElvTxsVKEDuJl1mNlWMxvJPz7WzO41s1/n/z4muWaKiLRe5DXIN20Cs9yfJ5+MvT1ReuAXA9uLHg8CG939zcDG/GMRkUwZ3jpO/9AoCwfvpn9otOp2Z6GXh3322VzQfs97Dh3r7Y2x1TmhAriZzQfOAm4pOnwOsD7/83pgINaWiYjEoFqAjrpnZc01yF95Bd76Vliw4NCLHngA3GFW/BnrsFUoXwW+ABxZdOx17r4bwN13m9nxQS80s1XAKoAFxR9KRCRhtapGqg1KVqoqCVwe1h0+8QlYv/7QsXXr4FOfiu/DBKh5STCzFcAed99czxu4+zp373P3vu7u7npOISJSl1pVI7EMSn7jG7nedSF4X3QRTE0lHrwhXA+8H/igmX0AmA0cZWbfA14ws3n53vc8YE+SDRURiapWgG5oz8r77oPlyw89XrIEfvlLmD27rrbWo2YP3N0vd/f57t4LfAQYdffzgA3ABfmnXQDclVgrRUTqUKtqpK49K0dHcwOUxcF7fBy2bGlq8IbG6sCHgPea2a+B9+Yfi4g0RZjqkVoBOtLGyL/5TS5wn3baoWMPPpjLf59wQpwfLTRz96a9WV9fn4+NjTXt/USkPZUOTkIuMAcF34bXLnn1VejsnH7s9NPh3nsb+QiRmNlmd+8rPa61UEQkc6JUjzS0qbBZ+bGpqeDjLaCp9CKSOYlPaV+8uDxIv/hiLl2SkuANCuAikkGRp7SH9b735QL0Y48dOjY6mgvcx6RvtRAFcBHJnLqqR6r52c9ygbs4r716dS5wL1vWQEuTpRy4iGROIafd8MYK+/bBkUeWH29icUcjFMBFJJMaGpyE4Fx2RgJ3gVIoIjKzFJZ3LTYxkbngDQrgIm0vynKp7ajw+V+d1VEWuG9a8RkWrhmh/5ZHMvm9KIUi0sbStodjkKCJNhBDfjt/7t//9cVsevCHZb97yxd/murvJQzNxBRpY/1Do4GLNfXM7WLT4PKAVzRX0IzKzlkGBvsPHIpNlWZZVrVrF5x4Ytnh3jUjdJhxICD2peV7KaWZmCIzUNr2cCwVNKNy/1R5YK21RneZgAHK3jUjB38OCt6Qnu8lLAVwkTbW0HKpMai1DkmUgBnquQGBe+lnb+PFOUdPO1apB96s7yUuGsQUaWOxT3iJIMx2ZVECZulziwdnAytLrr6a4S27mDz62GmHuzo7+Oh/ObFl30uc1AMXaWOxTXipQ5gFpy47Y1HoHHhxcC1cHP75W6s5+YXflr95vnc9UNSW0s/f94ZjW/K9xEmDmCKSiIWDdxMUXQx4auisg4/rqUJZ9dd/x7qbP1t27v61G1M5CNkoDWKKSFOFzb9XmlFZsTdsxrqSQ4UBSsvYIGSjlAMXkUTEnn8PyHP/xcX/NK26JGuDkI1SD1xEEhFb/j2gsuQ77z6Xq0/9eNU8+UygAC4iiYl9NxwOpUs6HY6Z08neV/ZndhCyUTUDuJnNBh4AXpt//g/c/Uozuwr4FDCRf+oV7v6TpBoqIjPEunVw0UVlh4tTJZCb8DPnNYex9Uvva1bLUidMD/zPwHJ332dmncAvzOyn+d/d5O7XJ9c8EUm7hjcNLpiago6OssOlgbtY1mZOxq1mAPdcneG+/MPO/J/srbsoIrGLbbGsoHTJvn30f/0hqBKkZ9qgZalQVShm1mFm24A9wL3u/lD+V6vN7BEzu9XMAjeMM7NVZjZmZmMTExNBTxGRjKo2WaeSmjMoP/3p3EScww8PrGQpmImDlqVCDWK6+wHgFDObC/zIzE4Gbga+TK43/mXgBuCTAa9dB7myzb6+PvXcRdpI1MWyCj327V95f/AJiyYWFlIzk/sPHFy7pPB3T0KDlrGlg5okUhWKu+81s/uBM4tz32b2LaByokpE2lLUxbI6P3Uh2zffU3a8dAZlaWrmgHt9S8pGkIW100vVTKGYWXe+542ZdQGnA0+Y2byip30IeCyRFopIS4TZySfMZJ3hreMs+3Ju1/ezSoJ375oReteMlPXY60nNNKoV79moMD3wecB6M+sgF/DvcPcRM/sHMzuFXAplJ1Be9yMimTO8dZyrNjzO3sn9B49V6o3WmqwzvHWcgaXzDy4qVfCmzw/zaseh8OPkNp9YdlI39z0xEdirh2SrTtK+dnqQMFUojwBLAo6fn0iLRKRlgnbIKai0qULFyTpmZYH7/75jJWuXlQ2VAbmLxPcefKZq+5KsOmn12un10ExMkTYQ1+BbUBqhWL2bKkD1eu4wkq46CVraNu2VLgrgIhkX5+BbrQBdtTf6hjfAM+U96KDA3TO3i+fyGz2EkVTVSbFWrp1eLwVwkYwLs3FCWJXSCFClN/r88zBvXvlxd4a3jtNVumFDh/Hyn1+NFLybtcZ3Q2u3tICWkxXJuDgH3ypNnDlmTmdwCZ9ZefCemjq0I86SHtauXEzP3C4sfx6caQOk1aQ9hdFq6oGLZFycg2+h0whBee6rr4Yrrgg8Z+H1/UOjvPRKcPDumdt1sAoljSmMNE7yUQAXybi4B9+qphEqDFAScmvGSncFBqneCi2tk3wUwEUyLo7Bt5q9ywYDd0EWS/Ug3nGGOCmAi7SBRgbfqvYu2QNLl5a/qM7N0LNYqgfpneSjAC4yw1XqXQ4snV/+5IDFpqL0+rNYqgfpvXNQABeZ4Up7kTuvWVH+pFtugQsvPPiwkZxw1kr1IL13DgrgIhmSRCVEoXcZGLghMF1Sqdd+1YbHMxecw0jrnYMCuEhGJFUJseny04Lfb8uuiuetlPvdO7k/t4BVmwbxtH0uTeQRyYjYlzsdHg6sLulfu7Fq8Ibqud80L7/abtQDF8mIWCshgsoC86mSTSFeftkZi7jk9m3xtUfqoh64SEZU6vVGqoQI2oPyX/4lclngwJKe3LT4RtsjDVEAF8mIMLvfVBQUuCEXuE8LzoHXcuXZ/6n+9kgslEIRyYi6KiFimkEZW3skVuYx/EOG1dfX52NjY017P5FGpHHxotBuuAE+//ny4038/13iY2ab3b2v9HjNHriZzQYeAF6bf/4P3P1KMzsWuB3oJbcn5ofd/aU4Gy3SKmldvCiUKgOU0l7C5MD/DCx397cBpwBnmtk7gUFgo7u/GdiYfyzSFrK4Q3lgnvtXv1LwbmNhNjV2YF/+YWf+jwPnAKfmj68H7gfWxN5CkRZI6+JFgRLMc0u6hapCMbMOM9sG7AHudfeHgNe5+26A/N/HV3jtKjMbM7OxiYmJmJotkqxYSvaSVq2yRMF7RggVwN39gLufAswH3mFmJ4d9A3df5+597t7X3d1dZzNFmquhkr2knXuuArcAEcsI3X2vmd0PnAm8YGbz3H23mc0j1zsXaQupLJGbmoKO8v0qe9eM0NXZwdo2XYNEKgtThdIN7M8H7y7gdOAaYANwATCU//uuJBsq0mypWrwooMfdt/of+N3hxwDp2B1Gmi9MD3wesN7MOsilXO5w9xEz+1fgDjO7EHgGODfBdorMTBUGKHvXjJQdS+UAqyQqTBXKI8CSgOO/B+qbgytSh0xPrImqSmVJ/9AopHB3GGk+rYUimVCYWDO+dxLn0MSa4a3jrW5avBYsqDlAmeoBVmkqrYUimZDGXcFjvSN4+WU44ojy4wFVJakcYJWWUACXTEjbxJpYp9oH9bgnJ2H27Irv3crg3er3l0MUwCUT6tkVPO5AU3y+WWYcKOkdR74jCArcixbBE09UbUMr12hp9fvLdMqBSyZEzfvGnTMvPV9p8C4IdUdQbQZlleANrV+jpdXvL9MpgEsmDCzpYe3KxfTM7cKAnrldrF25uGKvL+5AE3S+IKV3BMNbx+kfGmXh4N2xTH1vdSqp1e8v0ymFIpkRZWJN3IEmzOtK7wgKvfajXnyBp7758fIX1DHtvZ5UUpxa/f4ynXrgkmrFPdj+odHQKZC4F6Oq9LoOs4p3BNfds4PtX3k/D5UE7/6/jb4HZUGrSwhb/f4ynXrgklqNDJhddsaiaa+FxgJNpfNVTOOYle3u/uOT/iufPWcN9od/q6sN0PoSwla/v0ynLdUktfqHRgNv13vmdrFpcHnN1ydZhVLxfCGmvodtv0hB3VuqibRKo3nsuBejqnq+CoH7LV/8aWx3ASKlFMAltTIxYDY2Bm9/e9nhwhKvf/Wfe7jviYlY7gI0gUZKKYBLasWdx45dQK+7OFUyuf8A9z0xEUu6RBNoJIiqUCS1otZ+N01APfe1f/mxRJd41QQaCaIeuKRa2jdVAMCduxJe4lUTaCSIeuAitYSYQZl0fXQmNlmWplMAF6nkxz8OPfU96XSPJtBIEKVQRIJUCtxVJJnu0QQaCaIALlIsKHBv2ABnn938tpRI1XiApELNFIqZnWhm95nZdjN73Mwuzh+/yszGzWxb/s8Hkm+uSEKq5blTELxFgoTpgb8KfM7dt5jZkcBmM7s3/7ub3P365JonkrAqlSUiaVezB+7uu919S/7nPwHbAd3HSbatXRsYvIe37KJ/7cbIqx+KtEKkHLiZ9QJLgIeAfmC1mX0MGCPXS38p4DWrgFUACxYsaLS9Io2rkCrRbEfJmtBlhGZ2BPBD4BJ3/yNwM/BG4BRgN3BD0OvcfZ2797l7X3d3d+MtFqlXUJ57bOxgukSzHSVrQvXAzayTXPC+zd3vBHD3F4p+/y2gfB6xSBqEzHNrtqNkTZgqFAO+DWx39xuLjs8retqHgMfib55IAyLuQanZjpI1YVIo/cD5wPKSksFrzexRM3sEWAZcmmRDRUL78Ifr2jxYsx0la2qmUNz9F0DQPehP4m+OSAPcYVZAnyRkSWAzZjtqTW+Jk2ZiSnsI6nHv2gU90YJjkrMdVeUicVMAzxj14EpkaCJOtSqXGf1vKHVTAM+QOHtwmb8QZChwF6jKReKm5WQzJK465cKFYHzvJM6hC0GrZh0Obx2nf2g03OzHefPqGqBMA1W5SNwUwDMkrh5c1AtBpAAbUeiLyeRkLnA///z04xkI3AWqcpG4KYWSIXHt0h7lQlBP2iZKeiZUXjiox71vHxx+eOA5o2pWOklrekvcFMAzJK5d2qNcCKIOvAUF/Etv38Ylt2+jJyBgVb2YBAXuo46CP/yh6ueLotmVIVrTW+KkAJ4hlXpwAP1Do6F7dVEuBFHTNkEBv5DgCAqOQReTndesCG54TKmS4h73LDMOlJxXlSGSFQrgGVPag6unBxnlVj5q2qZWPr40OBZfTBoJ3GHTIKXfV2nwDvs5RNJAATzj6q0tDnsrHzVtUyngFysOjgNLenjt7yd4/3uXlD2vf+3GXCCu0cYoF7Gg76vS5xBJOwXwjEu6tjjqwFtQwC81LTia8f6S3y/8wgbcZkHIfHStqpritte6uIAqQyQ7FMAzLq7KlGqiDLwVB/zxvZMYh3LgUBQcAwYof/6201h15vQ10cLcTVS6WBV64sU989L2FHSYMeWuyhDJFAXwjIurMqWgNJe87KRu7ntiIlLZW3HALz3fpstPg68EvMidiwbvDjxfrbuJShexDrPAAdWgi8ralYsZWNJzsL2X3r5NwVxSTwE84+KsLQ7KJX/vwWcO/r6eEruDwTzE1Pd67yYqXcQqpXEc6JnbVfZ9abEpyRoF8DYQJsURpkojzABf5BK7xx+Hk08uPx5Q/VHv3USli1ghjVOqZ24XmwaXlx3XYlOSNQrgM0DYnmXYgc/QA6SV1iypoJG7iUoXsSgXBC02JVmjAD4DhO1Zhq3SqDlAGhS4166FwcGa544yYFrrriLqBaEZA8IicVIAnwHC9izDlABWTWk0cYnXsHcVUS4IcQ8IiyRNqxHOAGGXMR1Y0sPalYvpmduFkcsVn/fOBdMeF6o1pom4eXAc4lpat1jQ5w/8vCIpUbMHbmYnAt8FXg9MAevc/WtmdixwO9AL7AQ+7O4vJddUqVeUnmWkxZbuvRfe977y401Y3jWpfLUWm5IsCZNCeRX4nLtvMbMjgc1mdi/wcWCjuw+Z2SAwCKxJrqlSr0SWMY04QBm3sPnqzO88JFJFmF3pdwO78z//ycy2Az3AOcCp+aetB+5HATy1YutZBgXu4WE455zGzx1BmLsK1XVLu4s0iGlmvcAS4CHgdfngjrvvNrPjK7xmFbAKYMGCBQ01VqZrau8yZXtQhrmrUF23tDvzkP8DmtkRwP8Drnb3O81sr7vPLfr9S+5+TLVz9PX1+djYWCPtlbzS3iVMnxIem5QF7igWDt4duO4JBM/EFEkrM9vs7n2lx0NVoZhZJ/BD4DZ3vzN/+AUzm5f//TxgT1yNldqSqMKY5pvfzOzmwQWVqm8MUrOhs0gjagZwMzPg28B2d7+x6FcbgAvyP18A3BV/86SSRGcNmsFnPjP9WIYCd0HQJsJBqxHGeuETaaIwOfB+4HzgUTPblj92BTAE3GFmFwLPAOcm0sIZqlZ+O5FZg0E97ocfhr6yO7emaSTPH5QnrzTTVNPlJYvCVKH8glzHJchp8TZHIFz1RKyzBlOa546jiqS0+qZ/aFTT5aVtaCZmCoXJb8cyazDkDMrhreP0D42ycPBu+odGm5YvTiLPH5RW0XR5ySqthZJCYfPbddd2X3opfPWr5ccDetytrKVOIs+fyKQmkRZRAE+hOPPb03LIR89m0xWnlz+pJHAXv2aWWdnO7c2qpU5qdUBNl5d2oRRKCkW5za+W3ij0nsf3TvLUNSvKg/euXYHBu/Aah7LgXdCMQT+lO0SqUw88hcLe5tdKb1x3zw62f6V0z/e8CoE5zK480JxBP6U7RKpTAE+pMLf5VaeKL53PpoDX9K4ZwYCnKpwzTM+6mb1gpTtEKlMAz7CgYHvD3TfyV4+Nlh3vXTNy8OdqvedqO7xPuasXLJIiCuAZVhxsDzvwKr+5fqDsOW/54k8j1YpXqi/XxgYi6aNBzAwrDPLtvGZFefB++WVwj1wrrl1pRLJDPfAMG1g6n4GSY8/3L+P1vyhPoUQ6r/LOIpmgAJ5FVaa+vz7/4/DWcf73jx/npVf2H/y1NjQQaS9KobRYpGnqy5aFnvp++Z2PTgveBVp5T6R9qAeekDCr6IWepv7HP8LRR5e/SZ213I1MwtEekyLpoQCegLCBOdSWX0E97gMHYFblm6daATrqJJxC0B7fOzltPW2lZERaSymUBIRdRa/qYk1BKwV+6Uu5XneV4A3VA3TUSTjFU+tBmyGIpEnmeuBZuIUPu4pe0KSZndesCD5phLW5g2q5AeZ0zuI1h83i0tu3cd09O0J9d2Gm1mszBJHWyFQPvHShpbTuZ1ipB1x6/LIzFtE5K9fL3vit/xEcvOvYyiyolvu8dy7AMfZO7o/03YUJztoMQaQ1MhXAE9/INyZRVtHrfvlFdl6zgje+uGv6Lxrcg3JgSQ+bBpfz1NBZbBpczn1PTNT13dUKzlodUKR1wmxqfKuZ7TGzx4qOXWVm42a2Lf/nA8k2MyfRjXxjFHY248DS+fzy/5w/7VjvmhH6126MvU31fneVNgYGzdIUabUwOfDvAH8HfLfk+E3ufn3sLaoiqQX+k1B1NmNAZcmqD/0vfv4f3wUkc0Gq97vTkq4i6RVmU+MHzKy3CW2pKdaNfFuhwgzK4pUCIZkLUiPfnabWi6RTI1Uoq83sY8AY8Dl3fymmNlWUpt5gpGqYww+HV14pP8eWXbk66iZckNL03YlIPOoN4DcDXyZXFvxl4Abgk0FPNLNVwCqABQsWRH6joEC5aXB5Q69vNGiFnkH529/Cm95UfoL84ORA/mHSQbX0O7jpv5+SicCdhZJRkVYyD1HpkE+hjLj7yVF+V6qvr8/HxsZCN640UEK0tamDXl+YSdjTQEDoHxoNzCf3zO06dHGptGZJkzX6HQadrxlBNe52i2SZmW12977S43WVEZrZvKKHHwIeq/TcRjRaNhj0+tJp4PXUkEeeQbllS9XgHWlBq4jiLL1sZh1+VkpGRVqpZgrFzL4PnAocZ2a7gCuBU83sFHLxcCdwURKNa7RssNbzytYdCSn0DMoFC+DppwPP0az1ReIsvQy1dktMslIyKtJKNXvg7v5Rd5/n7p3uPt/dv+3u57v7Ynf/C3f/oLvvTqJxYWc0Rn19sXoCQnFt9Po7vhQYvBeuGaH/038f2Dtt5voijX6HxZoZVONst0i7SvVMzCgzGsO+vlSYgFCa4gD42tuPZOc1K/hvT22Z9ty3fPGn9K4ZqZpiaOb6Io1+h8WaGVTjbLdIu0r1YlaNlr4Vv740VQG5gLDspG76h0Yrnj+o4mRg6fzyN3Onf2iUyZLAG5RiaOb6InGWDzazDl9ljyK1hapCiUvUKpS4lVZQLDupmx9uHq9a6VBccRKY556YgOOOA2Dh4N1l6RDIVb48NXTWwceVqlgqtSFNVNon0nyVqlBS3QOvpp5AUjqjsH9otOag3HN7JwMD99fe/VEu3vSP046Fna4e1JONo7yxGTQrUyQ9MhnAQ0+kqaHmoNzKlTz1ox+V/b53zQg9c7u4uOR42BSD0gMiEodMBvC4ytkq9Zjf82+7AyfiFNYsqZT3DQrMy07q5rp7dnDp7dumBWr1ZEWkUZkM4HGVs5X1mN3Zee3ZZc8b3rKL6+7ZgdXoLdfKsWsPSRGJUyYDeFzLyhb3mDddflr5E15+GebMYYDaATcorXPbg89UrPFWABeRRqW6DrySOGuEB5bOLw/e69fnpr7PmRP6PNWm7ZfSbEIRiUMme+CxDAKefTaMjJQfr7OsMkpQ1mxCEYlDJgM4NFDO9vDD8I53lB9vsB6+UlqnVGeHaTahiMQikymUukxN5SpLSoN3g5sHFwSldTpnGbNKi1mav6KsiLSpmRHAzaCjZE2U/ftjXZ87aCPjI2YfxlTJW+yfci2JKiKxyGwKJZRjjoG9e6cf++Uv4V3vSuTtStM6CwfvDnyeBjFFJA7t2QO/9tpcr7s4eF9ySa7HnVDwDqIlUUUkSe3VA9+xA046afqx7m7Ys6clzWnm6n0iMvO0RwCfmirPcUNL9qAspjVPRCRJ2Q/gQZsHFypOUkBrnohIUrKbA/+bvykP0i+8kOt1pyR4i4gkKcymxrcCK4A97n5y/tixwO1AL7lNjT/s7i8l18wiTz4Jb3zjtEP/+rX1fP6V+Tx348NKU4jIjBGmB/4d4MySY4PARnd/M7Ax/zhZ+/ZBb+/04P31rzO8ZRefnDie8b2TVfehFBFpN2F2pX8AeLHk8DnA+vzP64GBeJtVYtMmOPJIePrp3OPvfjeXKlm9uura4CIi7azeQczXuftuAHffbWbHV3qima0CVgEsWLCgvnd79tnc35deCjfeOO1Xca0N3ijtFSkizZb4IKa7r3P3Pnfv6+7uru8kH/lIrsddErwhHZNlCmuBK40jIs1UbwB/wczmAeT/bs1MGeJdG7xeSuOISCvUG8A3ABfkf74AuCue5kQXtIjU2pWLm5q+SEsaR0RmljBlhN8HTgWOM7NdwJXAEHCHmV0IPAOcm2Qja2n1ZJm4tngTEYmiZgB3949W+FXAJpIzk9Y8EZFWyP5U+hTQmici0gptHcCbWdrX6jSOiMw8bRvAC6V9hbRGobQPUKAVkbaQ3cWsalBpn4i0u7YN4CrtE5F217YBPA0zNEVEktS2ATwNMzRFRJLUtoOYKu0TkXbXtgEcVNonIu2tbVMoIiLtTgFcRCSjFMBFRDJKAVxEJKMUwEVEMsrcvXlvZjYBPB3hJccBv0uoOWmmzz1zzMTPDPrcUb3B3cv2pGxqAI/KzMbcva/V7Wg2fe6ZYyZ+ZtDnjut8SqGIiGSUAriISEalPYCva3UDWkSfe+aYiZ8Z9LljkeocuIiIVJb2HriIiFSgAC4iklGpDeBmdqaZ7TCz35jZYKvb0wxmdqKZ3Wdm283scTO7uNVtahYz6zCzrWY20uq2NIuZzTWzH5jZE/l/83e1uk3NYGaX5v/7fszMvm9ms1vdpiSY2a1mtsfMHis6dqyZ3Wtmv87/fUwj75HKAG5mHcA3gPcDbwU+amZvbW2rmuJV4HPu/hbgncBnZsjnBrgY2N7qRjTZ14CfuftJwNuYAZ/fzHqA/wn0ufvJQAfwkda2KjHfAc4sOTYIbHT3NwMb84/rlsoADrwD+I27P+nu/w78E3BOi9uUOHff7e5b8j//idz/0G2/oLmZzQfOAm5pdVuaxcyOAv4S+DaAu/+7u+9taaOa5zCgy8wOA+YAz7W4PYlw9weAF0sOnwOsz/+8Hhho5D3SGsB7gGeLHu9iBgSyYmbWCywBHmpxU5rhq8AXgKkWt6OZ/gMwAfx9PnV0i5kd3upGJc3dx4HrgWeA3cAf3P3nrW1VU73O3XdDrsMGHN/IydIawC3g2IypdzSzI4AfApe4+x9b3Z4kmdkKYI+7b251W5rsMGApcLO7LwFepsHb6SzI53zPARYCJwCHm9l5rW1VdqU1gO8CTix6PJ82vc0qZWad5IL3be5+Z6vb0wT9wAfNbCe5VNlyM/tea5vUFLuAXe5euMP6AbmA3u5OB55y9wl33w/cCby7xW1qphfMbB5A/u89jZwsrQH8YeDNZrbQzF5DbpBjQ4vblDgzM3I50e3ufmOr29MM7n65u893915y/86j7t72PTJ3fx541swW5Q+dBvyqhU1qlmeAd5rZnPx/76cxAwZvi2wALsj/fAFwVyMnS+Wmxu7+qpmtBu4hN0p9q7s/3uJmNUM/cD7wqJltyx+7wt1/0romSYI+C9yW76Q8CXyixe1JnLs/ZGY/ALaQq7raSptOqzez7wOnAseZ2S7gSmAIuMPMLiR3MTu3offQVHoRkWxKawpFRERqUAAXEckoBXARkYxSABcRySgFcBGRjFIAFxHJKAVwEZGM+v+gzt68hftpiwAAAABJRU5ErkJggg==\n" 439 | }, 440 | "metadata": { 441 | "needs_background": "light" 442 | }, 443 | "output_type": "display_data" 444 | } 445 | ], 446 | "source": [ 447 | "plt.scatter(X, y)\n", 448 | "plt.plot(X, model(X).detach().numpy(), color='red')\n", 449 | "plt.show()" 450 | ], 451 | "metadata": { 452 | "collapsed": false, 453 | "pycharm": { 454 | "name": "#%%\n" 455 | } 456 | } 457 | }, 458 | { 459 | "cell_type": "code", 460 | "execution_count": null, 461 | "outputs": [], 462 | "source": [], 463 | "metadata": { 464 | "collapsed": false, 465 | "pycharm": { 466 | "name": "#%%\n" 467 | } 468 | } 469 | } 470 | ], 471 | "metadata": { 472 | "kernelspec": { 473 | "display_name": "Python 3", 474 | "language": "python", 475 | "name": "python3" 476 | }, 477 | "language_info": { 478 | "codemirror_mode": { 479 | "name": "ipython", 480 | "version": 2 481 | }, 482 | "file_extension": ".py", 483 | "mimetype": "text/x-python", 484 | "name": "python", 485 | "nbconvert_exporter": "python", 486 | "pygments_lexer": "ipython2", 487 | "version": "2.7.6" 488 | } 489 | }, 490 | "nbformat": 4, 491 | "nbformat_minor": 0 492 | } -------------------------------------------------------------------------------- /02_MNIST_classification.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "source": [ 6 | "# Pytorch入门实战（2）-使用BP神经网络实现MNIST手写数字识别" 7 | ], 8 | "metadata": { 9 | "collapsed": false, 10 | "pycharm": { 11 | "name": "#%% md\n" 12 | } 13 | } 14 | }, 15 | { 16 | "cell_type": "markdown", 17 | "metadata": { 18 | "collapsed": true, 19 | "pycharm": { 20 | "name": "#%% md\n" 21 | } 22 | }, 23 | "source": [ 24 | "# 涉及知识点\n", 25 | "\n", 26 | "[Pytorch nn.Module的基本使用](https://blog.csdn.net/zhaohongfei_358/article/details/122797244)\n", 27 | "\n", 28 | "[Pytorch nn.Linear的基本用法](https://blog.csdn.net/zhaohongfei_358/article/details/122797190)\n", 29 | "\n", 30 | "[PytorchVision Transforms的基本使用](https://blog.csdn.net/zhaohongfei_358/article/details/122799782)\n", 31 | "\n", 32 | "[Pytorch中DataLoader的基本用法](https://blog.csdn.net/zhaohongfei_358/article/details/122742656)\n", 33 | "\n", 34 | "[Pytorch详解NLLLoss和CrossEntropyLoss](https://blog.csdn.net/qq_22210253/article/details/85229988)\n", 35 | "\n", 36 | "[如何确定神经网络的层数和隐藏层神经元数量](https://zhuanlan.zhihu.com/p/100419971)" 37 | ] 38 | }, 39 | { 40 | "cell_type": "markdown", 41 | "source": [ 42 | "# 本文内容\n", 43 | "\n", 44 | "本文将会使用BP神经网络（就是最普通的神经网络）实现一个MNIST手写数据集的实现。话不多说，直接开始。\n", 45 | "\n", 46 | "本文所使用到的环境如下:\n", 47 | "\n", 48 | "```\n", 49 | "python==3.8.5\n", 50 | "torch==1.10.2\n", 51 | "torchvision==0.11.3\n", 52 | "matplotlib==3.2.2\n", 53 | "```\n", 54 | "\n", 55 | "首先先导入需要的包:" 56 | ], 57 | "metadata": { 58 | "collapsed": false, 59 | "pycharm": { 60 | "name": "#%% md\n" 61 | } 62 | } 63 | }, 64 | { 65 | "cell_type": "code", 66 | "execution_count": 3, 67 | "outputs": [], 68 | "source": [ 69 | "import os\n", 70 | "import torch\n", 71 | "import matplotlib.pyplot as plt\n", 72 | "from time import time\n", 73 | "from torchvision import datasets, transforms\n", 74 | "from torch import nn, optim" 75 | ], 76 | "metadata": { 77 | "collapsed": false, 78 | "pycharm": { 79 | "name": "#%%\n" 80 | } 81 | } 82 | }, 83 | { 84 | "cell_type": "markdown", 85 | "source": [ 86 | "定义transform对象，其定义了数据集中的图片应该做怎样的处理：" 87 | ], 88 | "metadata": { 89 | "collapsed": false, 90 | "pycharm": { 91 | "name": "#%% md\n" 92 | } 93 | } 94 | }, 95 | { 96 | "cell_type": "code", 97 | "execution_count": 4, 98 | "outputs": [], 99 | "source": [ 100 | "transform = transforms.Compose([transforms.ToTensor(),\n", 101 | " transforms.Normalize((0.5,), (0.5,)),])" 102 | ], 103 | "metadata": { 104 | "collapsed": false, 105 | "pycharm": { 106 | "name": "#%%\n" 107 | } 108 | } 109 | }, 110 | { 111 | "cell_type": "markdown", 112 | "source": [ 113 | "加载和下载训练数据集，这里使用pytorch提供的API进行下载。如果你下载不下来，可以使用[百度网盘链接](https://pan.baidu.com/s/1NmxIlPhaeKSz_kFwCOn6rA?pwd=6hfa)进行下载，然后解压即可。" 114 | ], 115 | "metadata": { 116 | "collapsed": false, 117 | "pycharm": { 118 | "name": "#%% md\n" 119 | } 120 | } 121 | }, 122 | { 123 | "cell_type": "code", 124 | "execution_count": 6, 125 | "outputs": [ 126 | { 127 | "data": { 128 | "text/plain": "Dataset MNIST\n Number of datapoints: 60000\n Root location: train_set\n Split: Train\n StandardTransform\nTransform: Compose(\n ToTensor()\n Normalize(mean=(0.5,), std=(0.5,))\n )" 129 | }, 130 | "execution_count": 6, 131 | "metadata": {}, 132 | "output_type": "execute_result" 133 | } 134 | ], 135 | "source": [ 136 | "train_set = datasets.MNIST('train_set', # 下载到该文件夹下\n", 137 | " download=not os.path.exists('train_set'), # 是否下载，如果下载过，则不重复下载\n", 138 | " train=True, # 是否为训练集\n", 139 | " transform=transform # 要对图片做的transform\n", 140 | " )\n", 141 | "train_set" 142 | ], 143 | "metadata": { 144 | "collapsed": false, 145 | "pycharm": { 146 | "name": "#%%\n" 147 | } 148 | } 149 | }, 150 | { 151 | "cell_type": "markdown", 152 | "source": [ 153 | "等待一段时间下载成功后，可以看到训练集中一共有6w个数据，接下来下载测试数据集：" 154 | ], 155 | "metadata": { 156 | "collapsed": false, 157 | "pycharm": { 158 | "name": "#%% md\n" 159 | } 160 | } 161 | }, 162 | { 163 | "cell_type": "code", 164 | "execution_count": 7, 165 | "outputs": [ 166 | { 167 | "name": "stdout", 168 | "output_type": "stream", 169 | "text": [ 170 | "Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz\n", 171 | "Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz to test_set\\MNIST\\raw\\train-images-idx3-ubyte.gz\n" 172 | ] 173 | }, 174 | { 175 | "data": { 176 | "text/plain": " 0%| | 0/9912422 [00:00", 347 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAOJ0lEQVR4nO3df6xU9Z3G8edZrP4BDQHvFQkF6TYaNRuXkhFNNA2bSv2FwZp0U/6oGM1SE1RqatC4RjRqghtbc42GeF1/4EqpNZSIBt0SrD/4p3E0LOISlUVoQbxcYgw0QbvgZ/+4h80V73znMnPmB3zfr+RmZs4z554PAw9n7py5cxwRAnDi+7tODwCgPSg7kAnKDmSCsgOZoOxAJk5q58Z6enpi+vTp7dwkkJUdO3Zo3759Hilrquy2L5PUJ2mMpH+PiGWp+0+fPl3VarWZTQJIqFQqNbOGn8bbHiPpMUmXSzpX0nzb5zb6/QC0VjM/s8+StC0itkfE3yT9VtK8csYCULZmyj5F0l+G3d5VLPsa2wttV21XBwcHm9gcgGY0U/aRXgT4xntvI6I/IioRUent7W1icwCa0UzZd0maOuz2dyR90tw4AFqlmbK/LelM29+1fbKkn0paW85YAMrW8KG3iDhk+yZJ/6mhQ29PRcT7pU0GoFRNHWePiHWS1pU0C4AW4u2yQCYoO5AJyg5kgrIDmaDsQCYoO5AJyg5kgrIDmaDsQCYoO5AJyg5kgrIDmaDsQCYoO5AJyg5kgrIDmaDsQCYoO5AJyg5kgrIDmaDsQCYoO5AJyg5kgrIDmaDsQCYoO5AJyg5kgrIDmaDsQCYoO5CJpk7ZbHuHpAOSDks6FBGVMoYCUL6myl74p4jYV8L3AdBCPI0HMtFs2UPSH2y/Y3vhSHewvdB21XZ1cHCwyc0BaFSzZb8oImZKulzSIts/OPoOEdEfEZWIqPT29ja5OQCNaqrsEfFJcblX0hpJs8oYCkD5Gi677bG2v33kuqQfSdpS1mAAytXMq/GTJK2xfeT7/CYiXi1lKgCla7jsEbFd0j+WOAuAFuLQG5AJyg5kgrIDmaDsQCYoO5CJMn4RBuhKW7bUftvHpk2bkuvefPPNyfzw4cPJfP/+/cm8E9izA5mg7EAmKDuQCcoOZIKyA5mg7EAmKDuQCY6zn+Bee+21ZF6tVpP5kiVLyhznaz799NNk/sEHHyTz559/PpmvXLmyZtbscfDzzz8/me/evTuZT5kypantN4I9O5AJyg5kgrIDmaDsQCYoO5AJyg5kgrIDmeA4+wngmWeeqZk99NBDyXXrHe9dtGhRMn/llVeS+VtvvVUze/bZZ5Prfv7558m8lcaPH5/MV69encw7cRy9HvbsQCYoO5AJyg5kgrIDmaDsQCYoO5AJyg5kguPsx4Gnn346mS9evLhmduDAgeS627dvT+aVSiWZf/zxx8n8yy+/TOYpp59+ejIfGBhI5hFRM+vp6Umuu27dumQ+derUZN6N6u7ZbT9le6/tLcOWTbS93vZHxeWE1o4JoFmjeRr/jKTLjlp2h6QNEXGmpA3FbQBdrG7ZI+JNSZ8dtXiepBXF9RWSri53LABla/QFukkRsUeSisvTat3R9kLbVdvVwcHBBjcHoFktfzU+IvojohIRld7e3lZvDkANjZZ9wPZkSSou95Y3EoBWaLTsayUtKK4vkPRiOeMAaJW6x9ltr5I0W1KP7V2SlkpaJul3tm+Q9GdJP2nlkMe7Q4cOJfPrrrsumb/00kvJvN6x9JSDBw8m823btiXzU089NZlfeeWVNbMbb7wxue769euT+V133ZXMJ06cWDNbs2ZNct16nwt/PKpb9oiYXyP6YcmzAGgh3i4LZIKyA5mg7EAmKDuQCcoOZIJfcW2Da6+9NpmvWrWqZds+44wzkvntt9+ezGfOnJnML7jggmOe6Yjly5cn876+vmQ+adKkZP7EE0/UzOr9uU5E7NmBTFB2IBOUHcgEZQcyQdmBTFB2IBOUHcgEx9nb4Kyzzmpq/dmzZyfza665pmY2b9685LrTpk1rZKRRS30k87333ptcd+/e9GeiPPjgg8l8zpw5NbNTTjklue6JiD07kAnKDmSCsgOZoOxAJig7kAnKDmSCsgOZcOq0tmWrVCpRrVbbtr1uUe8xPnz4cDIfM2ZMMrd9zDOVpd7f59y5c2tm9U65vGzZsmS+ZMmSZN7Jx6VTKpWKqtXqiH9w9uxAJig7kAnKDmSCsgOZoOxAJig7kAnKDmSC32dvg3rHe086qXv/Gnbv3p3Mb7vttmSeOpZ+4YUXJte99dZbk3mOx9GbUXfPbvsp23ttbxm27B7bu21vKr6uaO2YAJo1mqfxz0i6bITlD0fEjOKr9seRAOgKdcseEW9K+qwNswBooWZeoLvJ9ubiaf6EWneyvdB21XZ1cHCwic0BaEajZV8u6XuSZkjaI+lXte4YEf0RUYmISm9vb4ObA9CshsoeEQMRcTgivpL0hKRZ5Y4FoGwNld325GE3fyxpS637AugOdQ/w2l4labakHtu7JC2VNNv2DEkhaYekn7duRLTSzp07k/n999+fzN94441knvrM+xdeeCG57sknn5zMcWzqlj0i5o+w+MkWzAKghXi7LJAJyg5kgrIDmaDsQCYoO5CJ7v3dSpTi4MGDyfyWW25J5mvXrk3m48ePT+Z9fX01s56enuS6KBd7diATlB3IBGUHMkHZgUxQdiATlB3IBGUHMsFx9hPco48+mszrHUev9zHXS5cuTebnnXdeMkf7sGcHMkHZgUxQdiATlB3IBGUHMkHZgUxQdiATHGc/DkREMn/99ddrZkuWLGlq23Pnzk3m9U6rjO7Bnh3IBGUHMkHZgUxQdiATlB3IBGUHMkHZgUxwnP04sH79+mR+6aWXNvy9zz777GT+3HPPNfy90V3q7tltT7X9R9tbbb9ve3GxfKLt9bY/Ki4ntH5cAI0azdP4Q5J+GRHnSLpQ0iLb50q6Q9KGiDhT0obiNoAuVbfsEbEnIt4trh+QtFXSFEnzJK0o7rZC0tUtmhFACY7pBTrb0yV9X9KfJE2KiD3S0H8Ikk6rsc5C21Xb1cHBwSbHBdCoUZfd9jhJqyX9IiL2j3a9iOiPiEpEVHp7exuZEUAJRlV229/SUNFXRsTvi8UDticX+WRJe1szIoAy1D30ZtuSnpS0NSJ+PSxaK2mBpGXF5YstmTADH374YTK/7777Gv7el1xySTJ/+OGHk/nYsWMb3ja6y2iOs18k6WeS3rO9qVh2p4ZK/jvbN0j6s6SftGRCAKWoW/aI2CjJNeIfljsOgFbh7bJAJig7kAnKDmSCsgOZoOxAJvgV1zao9zbhBx54IJlv3Lgxmc+YMaNm9sgjjyTXPeecc5I5Thzs2YFMUHYgE5QdyARlBzJB2YFMUHYgE5QdyATH2Uvw6quvJvPHHnssmb/88svJfObMmcn88ccfr5lxHB1HsGcHMkHZgUxQdiATlB3IBGUHMkHZgUxQdiATHGcfpf7+/prZ3XffnVx3YGAgmV988cXJfOXKlcl82rRpyRyQ2LMD2aDsQCYoO5AJyg5kgrIDmaDsQCYoO5CJ0ZyffaqkZyWdLukrSf0R0Wf7Hkn/IunIh6LfGRHrWjVoq+3cuTOZL168uGb2xRdfJNedM2dOMl++fHky5zg6yjCaN9UckvTLiHjX9rclvWN7fZE9HBEPtW48AGUZzfnZ90jaU1w/YHurpCmtHgxAuY7pZ3bb0yV9X9KfikU32d5s+ynbE2qss9B21Xa13mmQALTOqMtue5yk1ZJ+ERH7JS2X9D1JMzS05//VSOtFRH9EVCKi0tvb2/zEABoyqrLb/paGir4yIn4vSRExEBGHI+IrSU9ImtW6MQE0q27ZbVvSk5K2RsSvhy2fPOxuP5a0pfzxAJRlNK/GXyTpZ5Les72pWHanpPm2Z0gKSTsk/bwF87XN5s2bk3nq8Nr111+fXLevry+Zjxs3LpkDZRjNq/EbJXmE6Lg9pg7kiHfQAZmg7EAmKDuQCcoOZIKyA5mg7EAm+CjpwlVXXZXMI6JNkwCtwZ4dyARlBzJB2YFMUHYgE5QdyARlBzJB2YFMuJ3Hj20PShr+mc09kva1bYBj062zdetcErM1qszZzoiIET//ra1l/8bG7WpEVDo2QEK3ztatc0nM1qh2zcbTeCATlB3IRKfL3t/h7ad062zdOpfEbI1qy2wd/ZkdQPt0es8OoE0oO5CJjpTd9mW2P7C9zfYdnZihFts7bL9ne5Ptaodnecr2Xttbhi2baHu97Y+KyxHPsdeh2e6xvbt47DbZvqJDs021/UfbW22/b3txsbyjj11irrY8bm3/md32GEkfSpojaZektyXNj4j/busgNdjeIakSER1/A4btH0j6q6RnI+IfimX/JumziFhW/Ec5ISJu75LZ7pH0106fxrs4W9Hk4acZl3S1pOvUwccuMdc/qw2PWyf27LMkbYuI7RHxN0m/lTSvA3N0vYh4U9JnRy2eJ2lFcX2Fhv6xtF2N2bpCROyJiHeL6wckHTnNeEcfu8RcbdGJsk+R9Jdht3epu873HpL+YPsd2ws7PcwIJkXEHmnoH4+k0zo8z9Hqnsa7nY46zXjXPHaNnP68WZ0o+0inkuqm438XRcRMSZdLWlQ8XcXojOo03u0ywmnGu0Kjpz9vVifKvkvS1GG3vyPpkw7MMaKI+KS43CtpjbrvVNQDR86gW1zu7fA8/6+bTuM90mnG1QWPXSdPf96Jsr8t6Uzb37V9sqSfSlrbgTm+wfbY4oUT2R4r6UfqvlNRr5W0oLi+QNKLHZzla7rlNN61TjOuDj92HT/9eUS0/UvSFRp6Rf5/JP1rJ2aoMdffS/qv4uv9Ts8maZWGntb9r4aeEd0g6VRJGyR9VFxO7KLZ/kPSe5I2a6hYkzs028Ua+tFws6RNxdcVnX7sEnO15XHj7bJAJngHHZAJyg5kgrIDmaDsQCYoO5AJyg5kgrIDmfg/SP9Cn800eXAAAAAASUVORK5CYII=\n" 348 | }, 349 | "metadata": { 350 | "needs_background": "light" 351 | }, 352 | "output_type": "display_data" 353 | } 354 | ], 355 | "source": [ 356 | "plt.imshow(images[0].numpy().squeeze(), cmap='gray_r');" 357 | ], 358 | "metadata": { 359 | "collapsed": false, 360 | "pycharm": { 361 | "name": "#%%\n" 362 | } 363 | } 364 | }, 365 | { 366 | "cell_type": "markdown", 367 | "source": [ 368 | "到这里，前期准备工作就结束了。\n", 369 | "\n", 370 | "---\n", 371 | "\n", 372 | "开始定义神经网络" 373 | ], 374 | "metadata": { 375 | "collapsed": false, 376 | "pycharm": { 377 | "name": "#%% md\n" 378 | } 379 | } 380 | }, 381 | { 382 | "cell_type": "code", 383 | "execution_count": 10, 384 | "outputs": [], 385 | "source": [ 386 | "class NerualNetwork(nn.Module):\n", 387 | "\n", 388 | " def __init__(self):\n", 389 | " super().__init__()\n", 390 | "\n", 391 | " \"\"\"\n", 392 | " 定义第一个线性层，\n", 393 | " 输入为图片（28x28），\n", 394 | " 输出为第一个隐层的输入，大小为128。\n", 395 | " \"\"\"\n", 396 | " self.linear1 = nn.Linear(28 * 28, 128)\n", 397 | " # 在第一个隐层使用ReLU激活函数\n", 398 | " self.relu1 = nn.ReLU()\n", 399 | " \"\"\"\n", 400 | " 定义第二个线性层，\n", 401 | " 输入是第一个隐层的输出，\n", 402 | " 输出为第二个隐层的输入，大小为64。\n", 403 | " \"\"\"\n", 404 | " self.linear2 = nn.Linear(128, 64)\n", 405 | " # 在第二个隐层使用ReLU激活函数\n", 406 | " self.relu2 = nn.ReLU()\n", 407 | " \"\"\"\n", 408 | " 定义第三个线性层，\n", 409 | " 输入是第二个隐层的输出，\n", 410 | " 输出为输出层，大小为10\n", 411 | " \"\"\"\n", 412 | " self.linear3 = nn.Linear(64, 10)\n", 413 | " # 最终的输出经过softmax进行归一化\n", 414 | " self.softmax = nn.LogSoftmax(dim=1)\n", 415 | "\n", 416 | " # 上述动作可以直接使用nn.Sequential写成如下形式：\n", 417 | " self.model = nn.Sequential(nn.Linear(28 * 28, 128),\n", 418 | " nn.ReLU(),\n", 419 | " nn.Linear(128, 64),\n", 420 | " nn.ReLU(),\n", 421 | " nn.Linear(64, 10),\n", 422 | " nn.LogSoftmax(dim=1)\n", 423 | " )\n", 424 | "\n", 425 | " def forward(self, x):\n", 426 | " \"\"\"\n", 427 | " 定义神经网络的前向传播\n", 428 | " x: 图片数据, shape为(64, 1, 28, 28)\n", 429 | " \"\"\"\n", 430 | " # 首先将x的shape转为(64, 784)\n", 431 | " x = x.view(x.shape[0], -1)\n", 432 | "\n", 433 | " # 接下来进行前向传播\n", 434 | " x = self.linear1(x)\n", 435 | " x = self.relu1(x)\n", 436 | " x = self.linear2(x)\n", 437 | " x = self.relu2(x)\n", 438 | " x = self.linear3(x)\n", 439 | " x = self.softmax(x)\n", 440 | "\n", 441 | " # 上述一串，可以直接使用 x = self.model(x) 代替。\n", 442 | "\n", 443 | " return x" 444 | ], 445 | "metadata": { 446 | "collapsed": false, 447 | "pycharm": { 448 | "name": "#%%\n" 449 | } 450 | } 451 | }, 452 | { 453 | "cell_type": "code", 454 | "execution_count": 11, 455 | "outputs": [], 456 | "source": [ 457 | "model = NerualNetwork()" 458 | ], 459 | "metadata": { 460 | "collapsed": false, 461 | "pycharm": { 462 | "name": "#%%\n" 463 | } 464 | } 465 | }, 466 | { 467 | "cell_type": "markdown", 468 | "source": [ 469 | "神经网络定义完后，开始定义损失函数，这里选用**负对数似然**损失函数（`NLLLoss`， [negative log likelihood loss](https://pytorch.org/docs/stable/generated/torch.nn.NLLLoss.html)），其常用于分类任务。[详情可参考链接](https://blog.csdn.net/qq_22210253/article/details/85229988)" 470 | ], 471 | "metadata": { 472 | "collapsed": false, 473 | "pycharm": { 474 | "name": "#%% md\n" 475 | } 476 | } 477 | }, 478 | { 479 | "cell_type": "code", 480 | "execution_count": 12, 481 | "outputs": [], 482 | "source": [ 483 | "criterion = nn.NLLLoss()" 484 | ], 485 | "metadata": { 486 | "collapsed": false, 487 | "pycharm": { 488 | "name": "#%%\n" 489 | } 490 | } 491 | }, 492 | { 493 | "cell_type": "markdown", 494 | "source": [ 495 | "接下来定义优化器，这里使用随机梯度下降法，学习率设置为0.003，momentum取默认的0.9（用于防止过拟合）" 496 | ], 497 | "metadata": { 498 | "collapsed": false, 499 | "pycharm": { 500 | "name": "#%% md\n" 501 | } 502 | } 503 | }, 504 | { 505 | "cell_type": "code", 506 | "execution_count": 13, 507 | "outputs": [], 508 | "source": [ 509 | "optimizer = optim.SGD(model.parameters(), lr=0.003, momentum=0.9)" 510 | ], 511 | "metadata": { 512 | "collapsed": false, 513 | "pycharm": { 514 | "name": "#%%\n" 515 | } 516 | } 517 | }, 518 | { 519 | "cell_type": "markdown", 520 | "source": [ 521 | "准备工作完毕，开始训练数据集：" 522 | ], 523 | "metadata": { 524 | "collapsed": false, 525 | "pycharm": { 526 | "name": "#%% md\n" 527 | } 528 | } 529 | }, 530 | { 531 | "cell_type": "code", 532 | "execution_count": 14, 533 | "outputs": [ 534 | { 535 | "name": "stdout", 536 | "output_type": "stream", 537 | "text": [ 538 | "Epoch 0 - Training loss: 0.6294137474252726\n", 539 | "Epoch 1 - Training loss: 0.27885234054884933\n", 540 | "Epoch 2 - Training loss: 0.2180362274207032\n", 541 | "Epoch 3 - Training loss: 0.17646600610650043\n", 542 | "Epoch 4 - Training loss: 0.14901786734228895\n", 543 | "Epoch 5 - Training loss: 0.12897429347081957\n", 544 | "Epoch 6 - Training loss: 0.11274210547309504\n", 545 | "Epoch 7 - Training loss: 0.10064082649717135\n", 546 | "Epoch 8 - Training loss: 0.09091206552532277\n", 547 | "Epoch 9 - Training loss: 0.08191311532861865\n", 548 | "Epoch 10 - Training loss: 0.07508732156971021\n", 549 | "Epoch 11 - Training loss: 0.07009464516681728\n", 550 | "Epoch 12 - Training loss: 0.0649078527074764\n", 551 | "Epoch 13 - Training loss: 0.06004000982112769\n", 552 | "Epoch 14 - Training loss: 0.054164604703361575\n", 553 | "\n", 554 | "Training Time (in minutes) = 0.9925608317057292\n" 555 | ] 556 | } 557 | ], 558 | "source": [ 559 | "time0 = time() # 记录下当前时间\n", 560 | "epochs = 15 # 一共训练15轮\n", 561 | "for e in range(epochs):\n", 562 | " running_loss = 0 # 本轮的损失值\n", 563 | " for images, labels in train_loader:\n", 564 | " # 前向传播获取预测值\n", 565 | " output = model(images)\n", 566 | "\n", 567 | " # 计算损失\n", 568 | " loss = criterion(output, labels)\n", 569 | "\n", 570 | " # 进行反向传播\n", 571 | " loss.backward()\n", 572 | "\n", 573 | " # 更新权重\n", 574 | " optimizer.step()\n", 575 | "\n", 576 | " # 清空梯度\n", 577 | " optimizer.zero_grad()\n", 578 | "\n", 579 | " # 累加损失\n", 580 | " running_loss += loss.item()\n", 581 | " else:\n", 582 | " # 一轮循环结束后打印本轮的损失函数\n", 583 | " print(\"Epoch {} - Training loss: {}\".format(e, running_loss/len(train_loader)))\n", 584 | "\n", 585 | "# 打印总的训练时间\n", 586 | "print(\"\\nTraining Time (in minutes) =\",(time()-time0)/60)" 587 | ], 588 | "metadata": { 589 | "collapsed": false, 590 | "pycharm": { 591 | "name": "#%%\n" 592 | } 593 | } 594 | }, 595 | { 596 | "cell_type": "markdown", 597 | "source": [ 598 | "最终在我这台机器上，花费了2分多钟完成了训练。可以看到，损失是越来越小的。" 599 | ], 600 | "metadata": { 601 | "collapsed": false, 602 | "pycharm": { 603 | "name": "#%% md\n" 604 | } 605 | } 606 | }, 607 | { 608 | "cell_type": "markdown", 609 | "source": [ 610 | "接下来进行模型的评估" 611 | ], 612 | "metadata": { 613 | "collapsed": false, 614 | "pycharm": { 615 | "name": "#%% md\n" 616 | } 617 | } 618 | }, 619 | { 620 | "cell_type": "code", 621 | "execution_count": 15, 622 | "outputs": [ 623 | { 624 | "name": "stdout", 625 | "output_type": "stream", 626 | "text": [ 627 | "Number Of Images Tested = 10000\n", 628 | "\n", 629 | "Model Accuracy = 0.97\n" 630 | ] 631 | } 632 | ], 633 | "source": [ 634 | "correct_count, all_count = 0, 0\n", 635 | "model.eval() # 将模型设置为评估模式\n", 636 | "\n", 637 | "# 从test_loader中一批一批加载图片\n", 638 | "for images,labels in test_loader:\n", 639 | " # 循环检测这一批图片\n", 640 | " for i in range(len(labels)):\n", 641 | " logps = model(images[i]) # 进行前向传播，获取预测值\n", 642 | " probab = list(logps.detach().numpy()[0]) # 将预测结果转为概率列表。[0]是取第一张照片的10个数字的概率列表（因为一次只预测一张照片）\n", 643 | " pred_label = probab.index(max(probab)) # 取最大的index作为预测结果\n", 644 | " true_label = labels.numpy()[i]\n", 645 | " if(true_label == pred_label): # 判断是否预测正确\n", 646 | " correct_count += 1\n", 647 | " all_count += 1\n", 648 | "\n", 649 | "print(\"Number Of Images Tested =\", all_count)\n", 650 | "print(\"\\nModel Accuracy =\", (correct_count/all_count))" 651 | ], 652 | "metadata": { 653 | "collapsed": false, 654 | "pycharm": { 655 | "name": "#%%\n" 656 | } 657 | } 658 | }, 659 | { 660 | "cell_type": "markdown", 661 | "source": [ 662 | "最终，本次训练在测试数据集上的精准率为97.41%" 663 | ], 664 | "metadata": { 665 | "collapsed": false, 666 | "pycharm": { 667 | "name": "#%% md\n", 668 | "is_executing": true 669 | } 670 | } 671 | }, 672 | { 673 | "cell_type": "markdown", 674 | "source": [ 675 | "# 参考资料\n", 676 | "\n", 677 | "[Handwritten Digit Recognition Using PyTorch — Intro To Neural Networks](https://towardsdatascience.com/handwritten-digit-mnist-pytorch-977b5338e627)： https://towardsdatascience.com/handwritten-digit-mnist-pytorch-977b5338e627" 678 | ], 679 | "metadata": { 680 | "collapsed": false, 681 | "pycharm": { 682 | "name": "#%% md\n" 683 | } 684 | } 685 | }, 686 | { 687 | "cell_type": "code", 688 | "execution_count": null, 689 | "outputs": [], 690 | "source": [], 691 | "metadata": { 692 | "collapsed": false, 693 | "pycharm": { 694 | "name": "#%%\n" 695 | } 696 | } 697 | } 698 | ], 699 | "metadata": { 700 | "kernelspec": { 701 | "display_name": "Python 3", 702 | "language": "python", 703 | "name": "python3" 704 | }, 705 | "language_info": { 706 | "codemirror_mode": { 707 | "name": "ipython", 708 | "version": 2 709 | }, 710 | "file_extension": ".py", 711 | "mimetype": "text/x-python", 712 | "name": "python", 713 | "nbconvert_exporter": "python", 714 | "pygments_lexer": "ipython2", 715 | "version": "2.7.6" 716 | } 717 | }, 718 | "nbformat": 4, 719 | "nbformat_minor": 0 720 | } -------------------------------------------------------------------------------- /03_cnn_image_classification.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": { 6 | "collapsed": true, 7 | "pycharm": { 8 | "name": "#%% md\n" 9 | } 10 | }, 11 | "source": [ 12 | "# Pytorch入门实战（3）：使用简单CNN实现物体分类" 13 | ] 14 | }, 15 | { 16 | "cell_type": "markdown", 17 | "source": [ 18 | "# 本文涉及知识点\n", 19 | "[PytorchVision Transforms的基本使用](https://blog.csdn.net/zhaohongfei_358/article/details/122799782)\n", 20 | "\n", 21 | "\n", 22 | "[Pytorch nn.Linear的基本用法\n", 23 | "](https://blog.csdn.net/zhaohongfei_358/article/details/122797190)\n", 24 | "\n", 25 | "[nn.Conv2d官方文档](https://pytorch.org/docs/stable/generated/torch.nn.Conv2d.html)" 26 | ], 27 | "metadata": { 28 | "collapsed": false, 29 | "pycharm": { 30 | "name": "#%% md\n" 31 | } 32 | } 33 | }, 34 | { 35 | "cell_type": "markdown", 36 | "source": [ 37 | "# 本文内容\n", 38 | "\n", 39 | "使用自己随便构造的一个简单的CNN网络，使用CIFAR10数据集（包含10种类别，图片大小为32x32）训练一个分类网络。\n", 40 | "\n", 41 | "本文所使用到的环境如下:\n", 42 | "\n", 43 | "```\n", 44 | "torch==1.10.2\n", 45 | "```" 46 | ], 47 | "metadata": { 48 | "collapsed": false, 49 | "pycharm": { 50 | "name": "#%% md\n" 51 | } 52 | } 53 | }, 54 | { 55 | "cell_type": "markdown", 56 | "source": [ 57 | "# 数据预处理" 58 | ], 59 | "metadata": { 60 | "collapsed": false, 61 | "pycharm": { 62 | "name": "#%% md\n" 63 | } 64 | } 65 | }, 66 | { 67 | "cell_type": "markdown", 68 | "source": [ 69 | "首先导入需要的包：" 70 | ], 71 | "metadata": { 72 | "collapsed": false, 73 | "pycharm": { 74 | "name": "#%% md\n" 75 | } 76 | } 77 | }, 78 | { 79 | "cell_type": "code", 80 | "execution_count": 1, 81 | "outputs": [], 82 | "source": [ 83 | "import torch\n", 84 | "import torchvision\n", 85 | "import torchvision.transforms as transforms\n", 86 | "import torch.nn as nn\n", 87 | "import torch.optim as optim\n", 88 | "\n", 89 | "import matplotlib.pyplot as plt\n", 90 | "import numpy as np" 91 | ], 92 | "metadata": { 93 | "collapsed": false, 94 | "pycharm": { 95 | "name": "#%%\n" 96 | } 97 | } 98 | }, 99 | { 100 | "cell_type": "markdown", 101 | "source": [ 102 | "定义transform，表明图片的处理方式和batch_size：" 103 | ], 104 | "metadata": { 105 | "collapsed": false, 106 | "pycharm": { 107 | "name": "#%% md\n" 108 | } 109 | } 110 | }, 111 | { 112 | "cell_type": "code", 113 | "execution_count": 2, 114 | "outputs": [], 115 | "source": [ 116 | "# 如果你的内存不够的话，可以减小batch_size\n", 117 | "# 一般batch_size越大，模型越稳定，训练速度越快。（但也不是越大越好）\n", 118 | "batch_size = 16\n", 119 | "\n", 120 | "transform = transforms.Compose(\n", 121 | " [transforms.ToTensor(), # 将图片转为Tensor类型\n", 122 | " # 对图片进行正则化。第一个参数为mean(均值)，第二个为std(方差)。每个参数之所以有三个0.5，是因为有RGB三个通道。\n", 123 | " # 综上，这句就是把图片的RGB三个通道都正则化到均值为0.5，方差为0.5的分布上。\n", 124 | " transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])" 125 | ], 126 | "metadata": { 127 | "collapsed": false, 128 | "pycharm": { 129 | "name": "#%%\n" 130 | } 131 | } 132 | }, 133 | { 134 | "cell_type": "markdown", 135 | "source": [ 136 | "之后开始准备数据集，这里直接使用官方提供的数据集。如果你的网够快，可以把`download=False`改为`True`，或者通过[百度网盘](https://pan.baidu.com/s/120cx2BA2hZ2nB_qaBgekdA?pwd=6aof)下载，然后解压到`data`目录下" 137 | ], 138 | "metadata": { 139 | "collapsed": false, 140 | "pycharm": { 141 | "name": "#%% md\n" 142 | } 143 | } 144 | }, 145 | { 146 | "cell_type": "code", 147 | "execution_count": 4, 148 | "outputs": [ 149 | { 150 | "name": "stdout", 151 | "output_type": "stream", 152 | "text": [ 153 | "Downloading https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz to ./data\\cifar-10-python.tar.gz\n" 154 | ] 155 | }, 156 | { 157 | "data": { 158 | "text/plain": " 0%| | 0/170498071 [00:00", 240 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAB4CAYAAADrPanmAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAADkYElEQVR4nOz9Waxta5bfCf2+Zrar3Wt3Z+/T3zb6zHC4yi4blxOMqXqBopBAgEAgIeUTD0g8YPHmt3pC4tUSoEIqCQpBQYEsMCoKl22yj4zmNnHb05/d79XP9mt4+OZae5/IiIzAmSicqjOuzt3nrL2aueb85vjG+I//+A/hveetvbW39tbe2l89k7/pA3hrb+2tvbW39i9nbx34W3trb+2t/RW1tw78rb21t/bW/oraWwf+1t7aW3trf0XtrQN/a2/trb21v6L21oG/tbf21t7aX1H7CzlwIcS/LYT4TAjxpRDiH/xlHdRbe2tv7a29tV9t4l+WBy6EUMDnwN8HXgJ/BPx3vPef/OUd3lt7a2/trb21X2Z/kQj8Xwe+9N5/7b1vgP8d8O/85RzWW3trb+2tvbVfZfov8Nq7wItb/34J/I0/7wV5nvvxePwX+Mi39tbe2lv7z5+dnJxceu/3f/7xv4gDF7/gsT+Dxwghfhf4XYDRaMTv/u7v/gU+8q29tbf21v7zZ//wH/7DZ7/o8b+IA38J3L/173vA659/kvf+HwH/COD4+NgD7O7uMhqNAFBKIaVASoEQAqRCCImUEiEAD1IItNZIuUF8wi88HnB478B7vAcfHsI7sDicc3jvtz+ttW88dvuP9571es1isQCgFT2UjmialrqpcbYlH+5y5977RHGOkAqB2B6PEB5uPSIQ+O02Fx5HEL6XCM9RwhMrTxxLjHU4371SgHMe2xTYpsJbg3cGZ1q8azk9fYa1luuV5fW1AS8Q8tYb0+2mt2ocHv9zW6zg1gHePGfzWyFuPc6t13qECNdLdJ8nbn3u5gV5Ao/2w4NTd4yKUkzbsF5cY6oFSRqDcDhb422LljH7R+8idMT1xXPK5RXCN8Ras1yt8UKQZn36gzGjwYDzkxcsl0uibMBk75g7dx/RH/Q4P33G9OqUYrXCWUeWD6jbJVE6IMlHWNOwmp9hzRqVTDgcJCRaUlUFr18/wVpLUdQ0JqwrrSN6d76Le/h38TLCi7A2U+X5/kNg9pynn/yYq9PXeO+IkxTvHdYZhFBIpREIqqoAIdkZ7XCws8+d8T5aaJ68/JTeqCTNQGmJ92FNO3tzOp1TrNpjAH7wg+8zHI1w3iOAxju0ECAlDjAeIufR+Jtr4jf3DVgBHkH4tUDcXhT+1gq49drNo0KIbkn5myd4i3ceqSI+/+wzTk9PwIOqDYd39mlbi7UWaxuKcs2do3uM9/bQsQ5rzN+6X7r39N5hWkPbNnjX3QvW0hpDVZYIAUmaUjc1y/WKpqkZDoY4a2nqBu8dSZKyv79Hv9/frt+maXj14oR5Fe7x6WzKer1GCNH5ouBjrHVYa3DO43F4F07I5p4S3pErTRLFIKFxjrJtaJxDq4hIa6QUOGdpm3b7vuEe9PjOX0l5cz795noIgfee4XDIZLLDn2d/EQf+R8D7QojHwCvgvw38d3+dF47HY46PjxFCEEURSgmUkmGhd3+k0uAdzjiEEKRJhpSqcyp++0dIjxAO0Z1k78Bbj7fQeou1Zuu0rbUYYzDGdAvKvvGYc+GzNg68aEB5QV1DXQukihkPjti9/z2SfISSGuEFoZRgO+ccLoCE7a3hfXDI4XcgZPi7kp5EOnLtkNrSWo8XEUJI8J6mLKhXF5hijjPByZmmAtdwfvYCi2VZOr4+M2GjkxKEwN92vN6x8du+Wzg3N5/izyZS3ZM3x4oIW+XP5VZh01Xb54TvJW6e5z2TPjzqkr6l2ycSA2qz5HKxYj2tGI01ThhMs8abijQakO33SNMh8+ols+kK367pZwmXV1O80gxGCqeHRInk9HzKdDZjsBOTjFNEfod4OKI+OWNVW+bLgrY2DF3KqlySjVIGiaZtas6vL2mqGSKT7GR7JFpSNxWvXn9FUzVcTwvKxoL3pGmP3egDzIN3cXoXJ2OEFAxjw99712FeWtZ//Hu8fvUM6wxp1sc5i7UtUmmUikEIlosZQsfgHIeDCfvZmFRnfLX6/5DuLBlkgihWOGdx1mKM314dY6OtA//gm9/k6N4x1jkkMC9KRNMQZRkySXBSkliPCl5n4xIJDkjSCrDd5i09v9SB+25P3jibsHHLLlByN0vHGZz1qCjh4uqK09MTBJ7IeQ53xrS1xdgWY2uW2vPo7l3uvfuYOE+3jk15EdZ998HOWYr1mun1lMV8SVXWVFWFtQ3lckmWpewd3aGxBqSnKAR7exPqsmK9WuOsZTgc8P6773JwcLB1vKvVmvWs2DrwxWLJ5dUlUkq01iilAMJm0bbBbziLsx7vNw7coZ1npDVZHCMjRSsFpbMUrcE5iLRGSYm1LU3ToLV+I2jc3Iubx73z3SbB1oF7z///HLj33ggh/sfA/4PgCf7X3vuPf53XSilRKjhjKVXnwBVCKZAKj8A4qEpHUVY4C1kWDlXgwFuE8AgJcazJco3SDiVDtN7tobcihZuTopTi55k3zrntQtpEnR64vHqFFBLhNWncY3/vgMf3HrDTy4kShRQenMVaDyI4fyFFl0EEZ2aNx7vOnYtuw5ESJUMElypL5Cuml9egE9LeCCUVbV1SX72munqGb+c4GzYY7yxKwOZqh89Sb0RYGwsxv8SLboF4tt9dbNOAzYZ48yo20fUmsmYTIb15DUOWJG82LSlwzm2fo1SXDgFaSlzbYtsGbw3L5YIs0xhXYeo1uIZIplyev2DMHlIalBZUZYsnJkljRJQQxxHGGE5OTpkv1yRJTr/fR+K4On/F4voV15ev0MKSxgKsB1cjvCPCELULfDVDVzPapmRVn+KOhkAE3Q1mjMW64PCk1ugkRdAgpy/w4zFexzgpcM6TG0NhfTiNUiC8p20bnLPd1uew3uC8xzmPEgJvHcI6NAodJ0jdEqeKKJZI1W3CHnTktzQDbxSsw9+Njml1EtYtgsX8iumzF4z39xke7pONBohIYqy9uaTbu0dR0UXgCIQLj96Kp28c+PbRbu04362XLlDqgigvNCiQMsaius8U6CQl7cdkvZCxSCU4koccHh+yd2ePtJchlQqr1AuElNvYrG1a1BWcvX7J5599wunZOfP5nOVqwevXL3n46D57h0MGOzu0ckRcJoz3xixmcywWnKM/7rN/vM/hncPt3bBYLOiP+ohLsXWo1lqEEFuHvfEJ3nuklOF5WNwmahaglcK1hqKpSfKU3s6Y/Z1dSmM5ef2auqlpb/kepRTW2jfQgM3vhBC4DVrgbt+fv5oh+BeJwPHe/2PgH/9LvA7v/Y2z9OCFoDWG1XrJy9ennF/OWa2hrCzGCoTMsDYLu7RwSAVRpMh7GTs7fcaDlN1xzt4oY6evkcLgNzvdrc/8RX9+yUFy+fILxqN99nfvMNkbMNkd8+hoj4ODnCgSIUNwAmPDBZVCIrsowuGx3tNagXcK68AYQ1XN8VVDEmlE3VCs51ydn/LkyXPSPOfxu4/o9VKqckl1dYY0ayLtcEpgJRiz8d2b4xYg1K0bUGydLt2RhBdsHKu8iXTe/MLbn11u02UX4cbapHabVDfckfrGeXdwipI3G6QUHmgAGPcyqsbghSWLJVoJ0iTBGEHZ1Ogo5t7dx6BzXr98Tl1cUCzmrOYzmqYmimNc05D1us31eoZHM9k9YDDo01Rrnn75Kb0kIU8jrEvQ5MRSE8uUfPd99u8cs5xeUEzPqBeCuH/MIIqRKtqeAWs9rQPbhZ8hwIjCOb76Gqvv4rIIlKRpS37vP/vPyOvnLBZLjFNYpxgNx2BaGttFvd5RVgWoGNMKnO023lhRY5DaouMIpXUHnbVIGSPVrXWqo+2VioQjkg6LRQnB0U7G7/+f/wXPX79G5hn333+XDx6/z70H99nZ2QnOxjmcB4HEW4cXEikUWmqU6pzFrQ3bdtewNS3GGIQArWMk0WYZAR7nHR4TVo8BsQkjPRgnmc6maB1jXYCFellGGmVIrxBObjcEvMAYx/XVNbOra67Oznn+1RN+/1/8C7766iuuZ9cU5ZrWVCgF33zvCN8sSfUuR/t3KKoGheBgcsD+eA/vPXEco1DhszqHKLzcYEjhbunW78astdvgRCm1/e5KdY7chwBMSolQEu01Qim0jvnww++we+eITz75CV988TMuLi+3G8DGeW8d9q1IHAJEtHHe/7/YX8iB/8vaFj/tTpxUGmM9T54954c/+jGfff4lRWWZL8ARE8U9st4+SXJMnPeRWoAMmLNUS1I1JVeCYU9wuJ/w3jsTPnz/EKUEwkvErRP3y47h538PIIuWtG842Mm592CPyd6Ew0nE7qAlZFoC0HgE3tmAPjiHc+AEtM4j8dT1ima5ZD675OriFcLUREpjTU1TFZSrJVQlcVyzPKtouug+8o4oAik11juk8yitaY1l666lQmmNs5t0ufse3VcJi8QS8pKNQwhR8w2k7dlkLA5w4lZ0QNj8uuWPFAIlRIC6Okhri/mLG/w+ROQ3IG5VXDOdLihWc5pqRZ4nKBmB1KSpIEsT9vYecvzgMe6nf8jZ6xWVSoiTnPl8SdbrESUprbGoNmDLWT7CeclsOu0chCVyCU1p0ZHG+QbjKi4u5uzd+4Cr5ZKr60umixmVcfSQ2C0cF7K3KFKIepNZuODAowytc5wAqTxOeaTyZNRcffaHrPycSCr2D48Q0vPOw8fYxlBZTV21tE2J8waHJE767A379Ce7lHjqcsne0SMO7k3QUUlVXOKKWYCnumsiZIQWfbjo1iWSLm5FeFAJtG7JF09+ysuzM+Q/T5jkEw7u3Ofv/hd/h+999zscH98NAY0D7yxVa3HOE0lNnGg2dQ3VwQjWWYy1WGOp6hJrHVrH5Nlg6+hD6u9wtqEoKrzX1GXZ3dOSg9EO3/n2N/E4lqsVZd0wHO1wcP8OcZKGbMN72rrhxfMXfPGzz/jqy694/eo1lxeXLBdzzs/OWSwXVNUa01bE2vHXf/A93nlwhCnXyLZmNBzx9PPPWKxWfOvDDxHC463BiZSmLkG6m9qaCjWG7dLnJsDbrHmEuFVv2yCK2zso/DtPEQqUECRRStob0ljNcm359vf+Gk6AcR+zmM+32f8vsm09zjqscyipfuHzfpn9Rhw4cMtxglSK169f8+nPPuOjjz6htY75smRdGOK0j5aKql2DbDHFAkuN9wYIkTjGEnmII8vL145nLyMa+9t8+73HHRZ985m/yGn//J+NKVchmhW74wHf+OADju/eI80GRDokx9Z7bIf/eQ/FusCjEDLCeMG6rljNZ1TzKc1ySrOeQbPGe0PjQyqNNaSRpzdJyHsSSY1vBE6E5YLWON92uL5HIIjUjdMRhMW2hTp+bhPamEcFiMC7LU69cb4B97xJoEV3YSRyu7hvQ0s2vNvNed1+tnzjHEp5s2gvzp5yfTXHNBWRlgz7vW3BbzgaMhqNGIz3GQwnPHz4mLq4xjYFsZLUZU1Vtui4R9t6hGgRSoVCkxfUdYFpSwQGYQVVXTOaTNBK4UXLfHFBvj7CJRmld5goQfd3UEmPolrhbkU+clsA7rIzIUDFOJXjZLhdUuVIdUNen+Lra7JRTBRn7MYDxpMR++Oc64sVurTEtsb4Gqskg50dRqMxom2RsqK2C+bLKVKuSdJ7CBHqJd5apBZYY7s0PkaoZHuMddNSVi3Oe7SAF198ycWrV+iyZozE1ZammvHx2TVt01CsCv7m3/w3uHN0BMIRKY33ktYEB11viqbOEUURcRxjCdivVppYxzS+xTQta7vuYAaxReC08DizgRhcdx4lg36fg4M7OAyjnR2MJQRgSQJSYo1lMZ3x5Muv+KPf/wNePHnKfDZnsVwwny+Yzmasy4LWtDhvSbOY/d0BvV6Pjz/+HCGe8d6Hc773W9/nw/ffxyLZ299FeIv3Dh3FjCc7oGQIsrzoSAK3MtCwwMG5AF0BzobaxwYXxwT4yEmH8OCwtK7FWDjeP2B/Z5+d0R5xnGKsR6mUvb07XO1dMbueI5Xoru2bBIFNdB7WWYAlhRQ3NaVfci/ftt9YBN79pUsnPC9fvebZsxdMZzNG4wlVXaAjSLKGKK5o3Qrj16xnC1ozR2tI0wQrLOvlijSKybKY1sKyMMR/opn0++zvjtFKbb3Nn3Xiv/xEWcA5S5pljHZ2GexMaFoo25BK2S4NMo2hqWrOLy4AidIpDsFivWB5fQ71ElGvwZTEWgAuYKxCEGmBlJoohjgWWOO3lW8nfMAxXdihcR4hBVFXPNx8ASUDzt098Ge+h+cmkpZe4m5FIJvXbJy2Et114aaotHHvAY7qNg65wc83xa7NDR2i9HBtbz5nPj2hWBVID5FMiaMeSMFg2GNvf5/JZJ+8PwKpGO3skPUGxEmGloK93UOurq/xPsAZSZKhtcdGlvF4SFkomlqAr8HVtMYiZUQUJ6E+ISXFaoXsj0BFRPkAqXPSNGG+mrHd0bwH68C6jvUTCrpCdPUaAUJ6ct2SyjXN4gWnpy/R+oA41ozyHkeHE5RZIKWnKRZU8ym2XuPTnHR/h1HuWF6tKKsVKwzLxSV5NsO3DQ6LbWxXVwkYdrjRQaibW7Uq1qwWS4x1aNvys5/8hOnFBam37EYRTeMwwrNqS06efsUnvR79LCeOE/b2d1BCBCDEeVphccZ2hTrHlsol6aBKSaQlAkXjPc62NLUJjkYJpBIoGdaNkDcbIECUpOS9IchQA/BCgU62jrIsSk5fn/DTH/+EP/3hnzK9uAwbStuwLgrmywWNMVgMSSzJBjmD0Yir6zmvX1/j0BSVQUcx/+Z/6e9x99G7gdmCC+VapYizHIQMRILuO3NrwxZ4xGaj7gKV29BqqJk5THfPeBE2KWNavJHEyYDheJ/xzh4yiqjriuViSRrn7E72eZE8p6xKlLrB3G9H+xvixLZ+tmHj3S5K/Dn2G43Aw30uKauaJ0+fcXJ2FjZEAUW5ZHc3J9IFzrUhFRIjri+/pqkuGAwSUr1L29Ys5ufI0S7jyRF5b8B67fnDP/mYo90Rf/3732EyHocdWHj8Jt+XILoaW2Cy/IIIvD/BxwmtV6waw8WqwhiFteBdi7ctzrQ0xZr59RWvXr/CWU+sY7RWzOdXNOsZ/VQTS4/EEak4BNbC3xQCpUDIm51YSHA2RMp1bTr4IjBtNBJ962YW3c79Bv1rSw30CKXCzbNBtrsT/PP4f/juEqE2sEj3nwwF2YD3+64s55HiVjH4VoVTbjZEcePgAcpyHpgJOkLJcCxRrJjsDbl7/4jx5ICqljSuJUoS0qxHFGdYZ3n44B2a1uKEJE0yJjt74BVCKA4PdihW1zTVHG8q1usVQjVMdu6ho4gkXtNWMJ8XqNUCGSvSrI/SnkQ5cObWDQWRgHiTLgvQSpIlmkHc4hOLzhvyvMTVc55ePOf5p5/RmIo7d/bYmQyIpWN+fUlrcubXMxbnp7h6TTbZhabA1pLl9Jy2sPiyomymZOmKtlrhvcHWDTjLegUeQ5oGmu3tskW1mLKKFHVjscspH//pH1Os5yTCEUuHtzVxkvFwr8e6bXn95Av+0BqyNOFv/J2/gdIa6UEJhxMBesIa8B7bhqxQR6C1AgtaCpQCFUvqxtF4g5IaqUCqwIRRSkBEBy2GkymURicJQuruvlbBiSPwxjK9vOLJ11/zyUcfc/rqNev1Gg+0xlDWJWVTY53FCkOiM1QaY5F8/vUL5vOSNM2xT5/ghOXR++/xjd/+a9BRkrfrUIUsUmzKQM7jjLntibaLd1Pf2WRfmyLmBm50bGp3HtcavItRKiWOc6IkAe8olzPWi2tGwwE7wx0mkz2ePXsaHA6bTCdAoLcz28AgC+f8z63N/Zz95iJwGQqXIop59uQZX3zxgtevzzG2YTq7pizmVKljflWwXK0oasnD9ydYt6KsrmlawapcUFYLnGvREnbGI/RgwHDQo64q/sk//RfIOOZ73/kme7tjXNWGz5WbHc6DMAjMLzhGye/8u/89PvvR7/Piao769Ev25w2D3i4ChWlWNOWMcnnN7OKS9XyKaSu8dWgpSCKNtyWjOEF7h20M1llkGqOUQEZqy3XfcAC8F0jpu5TeYTtc23uPkA6lgiO39s1zqTpneUP32hbzgZDOOrEpUoFEE7jGNwtls1AhFGID5heiewFIocMxipv9YZOBhF2wg3Po/LZ4U6dBqQhnahAWqUJRSCvIYkUcK4SW1KVhmPYZZGMePl5i6xUvn1zRz3a5c1wxm81QKGKVsrd3jzhJ6eUe7RtqYVGiz8HBY9LeHvuHd2jahrIqiL4P//j/9h/x+vknRGmf4WhCvz9gvTinrBbBgQFZEvHeo32ulg51usBZw3g85vHjA955J6Y/stx5eM6sXfDp8zmvXMH06orTNOHdh0fsDBLq1RXL2RQfRSgVotAoSXj4+B7f+MZjZssp06KimDeouEeSRvT2DtG9MbZZ4vGsVi2vnq1I+p7d3RTla2RUAzkArlphi4imrLl+9YyLV89INPjWUduG2lZEDoZRj1G/R+08l6+e8B//H/4Dosjw+J13GQ0naKUxpsbUVciiPLRtS21bsiiiPxoGaESEoqaxjqaxCO/QCDSE6NU4lNeh1nSLOSGkRkUJQlqsdxBWFAK4nk757Gc/4+OffBSc93LFslzjfMg6m7ahaRtaZ7HCYZZr1mXNuZqC9XglsaKlaRYsrl7z+Sc/5m/9zt8lyXKUViilUFH0Rp1HesB62qq5ySQ7FtdNQHlDAXDO0TTNFuvHb7g7Cm8D/i29pVwvmJqawWDA1flr5rMZ9x7cZzgecf/+fV6+fIE1LdaZDmIS27tzA6Vs4Jo8z1mv11hrb93Bv9x+QxH4JnJztMbxJz/8CV9//YLXr86xrgLlMG3B7PwcZ1yIRmVEMZ+SJGMWzCjXC9blHCEMWkkWy4rZYk2cFURJxs7+Llenr/m9P/gTrq/nfOeb30B6x/27hyFt6jBhvO6O5zZTIzih8f6E/jDl/ORr5tcn3Dl6yGg4IY0Vri2R3qDwuLplmMWQAs4igSSOQ8XbeUCSJmmXiobd1yGwpsXjkFoiRYTv7iKBCwVaCV5YhA83jbOWqjEYp25hjQKpFH5LhRRdMdXTWoPHh+KcCJHcjaOX291/U/iUSt4s4i0+Hv53OzvxXXrnlML7cB43hU7JBlbpaGGd7e0fM5te0ZRryrJAyJi2LPiiWHJ+esZgckTSmzD4oEe1kiwWFat1QVGs+Oyzr+ntZ3gUV9cXLJZTzi5fYpymqs5ZL14iXMGoN+Deg7/Oo0d7VOWaolxSVmvyfkaURVRlgTWePEohz5BK8+3v/DZ5HgPhhuxpjRMVTSJYG8W4l3Fnb4+d0R4tS5786D/BeEliYx701vxIKYxruLw8J08cwzwhycbY1lFUBbP5DOUMh8WENE/ZFznv76esdEMUr2iFYDX1XPhzvFtjVyXZFfxb+RHlfsLU1pSXFsMK2AHg6vw11XrKcl3w1ScfU8ynxFjyJGV8eMBICubTK5RvSYRjOMgROma6KvkP//3/DTuTPb7xjW/xzuN32dvfw2AxzmyxVwkI6/nq689Zm5ooz0h6OTrJaJ3GOkGkFJEMdQ6BJBYRKotwpg3X3znmV2ecvvicOO2R5EOy/hAVaYrlgh//8If88I//mC8//5zpfMqqXtPahoBaCaSO6A+GlMWaxra41tHUNcZXSDwq1jgvOJzs8/DePik1/+n/9T9C6Zjxzpije/d49N4HoCPAU67XtFXF7Oqaqplu16XUGhnFgWYLHbx4mx3iOlilc+A+hCnOWnQiWC1nFItr2qpif/+A9XpNlqaslzPmixkX0ylJHLMump+j8AZTSpEkCVrrLXyjOtbQr4Oh/OYgFMKFuri44OOPP2G1KlEq7qhcDW3j8NbRS0fESQ+PwtQ1ef8AJXsICqQvUd6jhMKaltn0Cuscad4n7/XI05jFfMGPf/QRz756wc5oh//Gf/Xvk6XBybkNi8QLoH7j+Lz3/OSP/inXJ0+piyXCeWYXL4l1TC9PiXUo7sRRHPBfL5EYhv2c4WCAt4K6rYmVJIr1NnpujQUfIaXFu1CIjUJnD2XZIKRHaYlQAucaymKNNS2RUh07wLAsGpzL2SR9UoY0cevAu+NHBYav6Ngjmz8etgtEdJi3hEBj67j0m6o7BAzcbaJ1zxavc/ibcPwWjBw6BN+Eo+4e32cyGnFx9pqTV69oa8+on7OeFSymc/o7cw7vvsMrrbi+XvD5559ydf4EaSoO9xTT2RlXF+cILL1+TtqLieMd2qalqkpsM8M0BWl2SlMp6mZF06xRyrG7Nw61BSLKpmW6mOGlZG/3gP29+2h/DTTUTcur1xeMjaZZrBBZihCW9XrKn/74K9J4xcunXyHSEXp0xGrZMtnpYZ3lk0+/5umzF+zv9Hnw4C537z5CK0LB2VrW65JV0WIbw7ox1NaRSEh6gjgV6CwEKnVpWRc19nif5OCQ/uKarKiQZLzqEsU00eAbZtfnfP31V1jvuPv4MXeP7zIcj3HC8cVHP2b+8hUKi/ItCsUgkbjKsr6+5Cd/+Ps8//wz7j94wN17hyAccRzWsmkbiumC519/xbKp0GlCfzRi9+AOO4f30L1BwL1xSBxeKDQRIpZUZQGAMS3Pv/ohfzq65J0Pvs/Rgw+RWlHVaz7+6Kd89JMf8/rlCxaLGUVTUrumi1BDeOW8AGvJtGS33yNLNP0sZTgYYJzlxekFdWvwDoqyoiwXnL36DITn+iLn6uIJp6++QiUpSgvatsJhKMuCy+UTELtAyAx1lGBtgNKSKMU7gzEtZrMZbfpHrMe4EEg5D1EuWa/nNHVNua5YlSVZllI1NeuzU9ZlRW0sUaRDHWkbBIktPPNm96fd8tCVUoGl8yvsN+LAPWw3l7OzM16/fk1V1SA0WgswAiliUJLBcJ+dyRE6jpmvWiKdoVTaAf0t0ncFFGspV0tM3aDjGVm/zyBLaOuGpmp5zTnvPHof5wMWtW3BF7cZGLeO0XueffkJbbnEmRrpPWlb4ZMETAT5AKdCW6+SkqZqwRuU2KWXJlgVdmknAOdwHWfWtA7tJTqR6CQOzUdS4JxEKIt1Lc4YhPPUZUFbVeF9ZOhW9cLT1A2QQecklVTBCcuw+LflzK5gKfxNcVGKDlEXHTbY4SpSiHAsahN932aTC8Rt9gpsP2vzyKb25ZzvMHa6zwrmnEerCCUkbd2AFgifUpVrfFVhkWS9AfiW+XzJYvYKU08ZpJZ+3nBxMsW3FVkWMcojxn2J1pZq1dAoQYXG1NBUJZflKy4vX2NdSZZJ2maMcyOkimibNbUpaW2FdbBYlCR5gKech8Yrsv1dIgNZv0eW51TlmhevZox7hqpaEwlN2i65M4r59jfv8tmXZ0znSxYLR12UREqzv3tAlkXIvQGiafDWcn15jdaOGkUtFKWxJK2lqR1NYyjXFdPzFcV1wVdihhKgXU3fKVJ1wwPPsgx8S2taVkXB/cfv8M3f+i329g/I8h7OtdSrBZ+dX4b6ig9ZXawlqRI4IairgtlFgzQNslmideDlIwRNXTO/vub01Utab1FaU86m1PMFl+fXjO/dZ/9wjzQOUJwTDvDY1mPcBo70GFuR9TT9YY8kjbGm5frygo9+8hOefv0111dX1HXVcdR9YBVtGs4A5S0Hu0Me39tj3EsY9jIGwxGt9aRpzOX1Eq0kTWVYrRfovMFJh7UrzHJNaecoFSE0OAxCeZq2ZVGek+STsK47+MI6iXMBMsM71usl6/WKOIpRWiE8NKqllQbThghdK0lVlZRlRVUbVBRRG4NpDWVV0RiD0hFCpOBdyG46yMQFUn53b9iOF24ZDkcopUJ7/7+yEXjnCbz3nJ6eMptOqSqL7KJMpWKiKANvSfMRw9EBeT9nVbwg0hodRQgp8cbiXdflZT22rGiLCi8kxWpN1csQQlHXBpymsYraQOo83lu8MDhuKsubY9rYxflrlAgNKYmWRFHCoJ+ikIz6PUBR1y1xJHENlFVL07QYY0OhQoSVGJyaxzqPcYJYSNI0J80itFZYPI3xpDqhrktMW9PWFcW6QYnQnWabFkONUGAas3WmQScmNBEpBdZuMgs62uCGgyq2C0gQCqX+lu6KlJv2+A2s4reXavO6bd4k6J53s8A2ztu7UAba4O0bW68KvKloqoq2LpEerKmpqjVOKEScUJQLrGswpiaLS7KRY7cvmIzXXE4LkrFk0I/Z3YuY7DpgiasLtJEsfUZZQNtU1GXB9eVrEDXOxGhVofs9olhRWxMYO7KlrAuqszPG92JiJRFSEaU9eseH9K0gzXKSSFGu11xeniJqTRQJctUwYM7OaELvg2OePL8G72law3pdspwvaKoleaoYxAOkaahqTzG/IutnOAlOQd3WUEga51GJZX5VcXGyYj2vKYoz9OqMw0HOcT4iiVKIMoBtpJymCcPxiG9893u8881vkSR5aCRzLffuP+TFpz+jruvuWngiKUkk6DgmkRLTWor5lAtRk8YqvK8QtK1hPp/RVKuu4OlwK8eirlm8PGNUlvRHOb18jJISh0ShAPcGRTfOU+4+fsjenQPiJGG1XPHi6TN+9sknnLx6TVkUtG2L8B6JwDkwHXVSSU8SC+4eT/jW+/fYG2X00og4TmmsxOE5u1xQlw2RAlxLnCWQS4QGLxtqOQ/CMM7jhUd6QWMMtV2xIWVq3X1vKTEt7OyMwVlwhrosiLQmiSNwIJ1AeYmVviMONNRVSVW3WCRCR8zmc8qixPvw3joKLJ1YKSLdPaYUQmocnnbbnRlkPYbDPkmSbRuKfpX9ZiLwDqO13nFyckJd17StAyO3TkTrFLxltarx4pK8yDBtg5aCKIpROqE1CV4orNNI0xXikKHS3SqKwqGiCB2NyPMxKp/w8nyFuJMSxyIUBGkDs8i9WdQDaKoqtLynEUma0Bum7Ix79OOY/d09lIyoa0Oe53gnefbihCgKTTcIj1RhcRtUKJRoReQhHfQY7O2i4xjfQRHSenrOk5mGcr5genFJ3dYM+xneNjRmRluHhpDKeHyyYawIlFbkMSRJ0G0JbLCQugfedtR1eXVVbxGcrfO3WubfJI1s8UDnPQqJ2rBR2Dz3zSffNGMHuMV5v6VeAdjWsl4uWMyuqVZzKrGkqubMV3N0liN7OToVWF/hKDjcTxinY/ZHhvFkTX8oMcaiREUctaT5kjgxjHorFqOI01cxL57XrGcX1I2nKuao2GGMp24k0jXECUTGIZXB+BWr4hLlU5zbB2KkEKQ6RjvHII3RytNUay7OLrg8PyGqctJEonoFjWuYRwq0Yjzqs16vqUqH0pLBIKbHMpT5hEdEkKqWycDgWNJzMyK/QktN2fQp2h7JAGxT411NmWjOYsGgmBMngqGKiBsJ4+DA27pi0It5eO+Y/N/829y/f48oim/qFSh29w/Zu3OHq/NzXFdYl1rhWoWUECUaq0PDjq0rGiOwVXBYAN7W4Fu0kGipiBVoDNfLmj/9k4+59/gBh7sTRnmP1rlQhwHi7vUIIIrp7eyA1hRFyenrc370xz/k5fPnzKdTmqoOkeemAN41L4EjjRV3j3a4d2dElgp6PUUSQV3OaBt4794e337vMb61NOWahopklJPs55AqQonLYxuLkALrHdZbyrIh62fbdZnneYBuvQff0ssi6rVBWgeNYbW6xiQJ3rqAj4vQkZrlfa5nS5bLAushyYLwXVU1gEJFMrCg0ojjowNEU5BKQyQ8URTRH01Qccyri3NWZcGqrFgXhjRJGI1HFGVJFN1kXb/MfiMO3LkgINU4y3Q6xXRRkaDrYnTBEUg0q6JkVb9CXBt6ekBdlfR6E+Kkh3SWOIqRXqClRMcxcZKRZDlSx504VgQyQqoEEWX8wY8vObvKuXecsjOWKOGhtVh7o0q4sYPBkNFkFAqKwhILH3bdumY+dWRZTpxkeG9xXuNEQ9VWiHWNEDVpb4BNMhaNpy4MdVNhnMNflbgnF10jUJeCeoerWwZ5TKIk9XrNTz/5Ke88vk85n5JJy+4w5fhgjyRP+fRSYULmuq2OJ5FGCUnTOpqqplpPacyM0e4jpExwnsD3JWiyKPQtR2xvXZ+wWMOi7zD0juHyizpWu0MAwoYiHEHb4ZYDb5uKxfya1fIaawrapub6uqW2jsEO2KahWi+pqoLl/IKdLEMPInJriaIztG5Ruu7ElyTFSrKczREI8uSI8SjhPKl48uxrjNWs10tiF5HkCT2ZYEyLViE7qIo15XqFNSf00zHtwyEQdzS6ltOXl1ws1wjbUJYFF9MlkYwoVhWpk1Te8rJoOHtyjY4SpHXkkQYXk+cZj+6PuX8nY71sqctQ/BqONPfuplxfrTBqRatWSJUwnRsu12sUKYlbkGlHHGtqJJlMuFwXFLMlSa3Jv38EgKlLSDz9VDN+dIy1YIsCJ24K0yqKuf/OI0xTUy5XHc3ZkKhQKLd+k4kJpPBoLQNkI0PxvJ8qTCMDQ8cD3qGV5mBnyI8+OeX505fcPzjkeLRLbWqcgEhpdBc1OmeZrp/zBz/6v/PB+z8glWMuLmY8f/oZzboCLwJkYgxCdZ2lPhxPqLN4Yi2oizknpysaM2Y0HBBLSVmV+Krhzr5klMeoNGI6t3z+01N2Hh2yc29MPFRUrgDZNd0R5BFaC8YRonagrmvW6wrhg97S1cUZ56/PWM0XNHWDM4ZaSLSSIMF6F3ozZOgeb1uDFwLT1nhnibWmaYK43GDQ51vf/Rb/7n/tv87q6ozf+0//CU8+/xnr5YKdyZwPv/VNfvtb32JdFZycn/P05ekWbx+NxltZjj/PfiMO3FpH07bUbcNysaBtKzyS0FbZHbTzWA9emJD+KYttGy7OLxju3WVnfJ80GZDlPaJIg5IIGVQMhdZ4D8qqEJGjsELipWJRwOdP1ixWC+4eao73E5Rz2LbZCttszBQlot8LtDfpSWREohSxlqAcjgrnDMt1S2s1yKC/YmxD2VhclBCPcnwU0cqW0pdIBVVVoWWnQobFGEvTOJqyRUiF7kdkWR7kLAnUI531SYY7JL0Bh3d2+Oz6usPWwVhP1UBSlSghES5E9UbGrMoad/2SOOkjoxypE4SQgVVuXeC8doXOgPjcaKFsJEcDN3cjlRbapDd0wa0z3wLvobkjRhHpGwdeNyXr5ZyqWOJdA75BWssoG6KspLieM8vOQ5ppLcWq5Kqt8G2DiSoWpcRaTZ4m9Hs94qjH1VnKq9cv6fXXoZYhHbZds1haausQkcYSoZMhTWNp6hZpQ3ZWlmuMaZHG4mwoVkWx4vAgx1vHQaJo2xixtAy85Tv37vDixQXNYkq9KqlpWNYCIUty6fHOEClBL4bdnkBrR9aDJAmd3JESaF/jvaAkp/QeYRXGWtp2zWLakKkWnMZLjasajDTMqpKLosaaiB9s7h9jaZoG4x1GBGoeG2njLqOSCnYmu1wOhpiqpKnLUDgTIfOUQna9EB4lPN63GGsQXhJFmkEvoa5114cQunKlAB0JJIazkzMuL694eOcOramwApzSOGu26yCKI6r6ghcvf4hrely+bilWKxIdU+gIr1T4TGNIlcaoOmiGSNBRTJL3UVGPpllzdnrB1dWUOM0pa8erkyvG40sOd/foJykvX7yklY7+g72wmoUm1inet1jXBhaWioljRS6X2+VaFCXX0zk4R12tcLaiLkps04bGOQTGGcoNJCrCvWF9g0fiRWhENK1AS0G/lzNrF0EF1Tm80KxacOkE3z+iVKecLedcr694dfH73Lt3yN7uDkJJemmP0XDMel1yenpOnmX0svTP9aW/EQdujKGpGuqmplwv8bYJMqrIbUEN2cEA3oANwj2WlqqaoVc9pEqJowGeDFSG1BohNWzlLg3OB9WIgLAJhJMYBOs64eyyxtQGW0n2JwrhDd6F1uKN9bIetrG0tg2aJP2Ypq4pVi2xVmRZTJYnYRHKlMEgD3Q+70mSjHiwg8oHKKNQrkK1liRVNG2L0lFQpvOgrMW6IkiP6pgkTkm9IdZR6ALUCY1TFBZqJalu8UPDhhEkY5uyJlIeqVKiKELoFJ1OaExNY1YIWSBVRBSloFIkGiduFT27n3LjBLrWOtV1qd3g3xvxru51onu1h9Z4jAnvqW615bXVkrqcU1frgPG3hihOGOQZxnloS4QpsVYG7jEtJvLIyKKjhszvodSQNInJkohI5wgpOL94Ta9q6fUMWoOOwgZrCIJN1lqcqRFWIeoK3Rji1mMacK3FqnoLm2UJPHgosK3Aec3ppUVKwaCf8Hd+MOKfuRWfL67RkWLSz5CVZ1G01F2hOdaSUaoY6hZp1qQqNH0EjFdSL69ZzATLSlCZGI0jjQQHI02sHJEA0KxNhFtZRGRRwiMTiY9v0mlPp6LnXFdncXhhw/Xa1Dh8wLqlEjhvsaYhihSSICkrfOjsDZu3wZmgNy+9RKsYKVPABaaU8AGftpIoE/R6muViyWw6p6gK8AaLx4v2VgDkccLSuJJVNaNerri4LLm8OqeoLQaDkx4nOxEn54iEwqsAQaqOiVJ5iSZC43BWUKwarmdrTi/nvDyb81VyQj/LEd7x7jcfMtwdkPQVKnZIL8CqjuMukISO7AhF0612JRWRjhEeitWS1WoVGn1caHq7EanoAEK/YWWF70hHCvDW0tQVg36P9XIV/EbTslisePnqFC01rdM4lWKI8M5iyobXr86oy5okS6mtD0qY1tE0LfG/qhCKtSGCaOoaZ01oue6GIXgnoNMOCGl9V4TwHisbTLuiWF+BiJAiReoe6CRgfEjCrOUN5c3jRVfS69gnock2Ylk4bNOEm1tG9FKL8ma7AAWwMxnhjKMqLViLaQ2LVYU1Fi0kTZvSmgAzDIY5/V6G6vAEGee4KMc6ttKisZbkcUSlgxZ4uOEkQkpqJTEqNBUooVGAkhFK5+hYgtIYEePjBJHGW2/roRNyEhjjiVRwYkJERI0nTsfU9RJrKnzTIqixTQkqdJFJFSNUhFShuKLULcGqrlj5hm5N58AjpbaFzi2s4gV1bakJGiX6lgOvyxmmXuFtjfeOxhj6vT5Kebxrkd6gfUNrRCfvabq03ZLGDUmUkPbu4DE4s8LZBU42lK0hajy9XJIkMZFS5KnG1EFFzrsGmhV5khJnEava45qWqHW0hk3BAIAkhod3BNYImkYwmzfkiWFvJ+HDB4qPfuaJYtgZJRwdDbleOU4vS85PK5I4fG4vUZi6oVp5tO5YPj44kOszy+Ja0dYSrAuUyMjTz0B4ixIOi2TVhsJ8pDypCNg1It6ey1D08tuOPrzAC7eFUITwCBsYSK1pMW2Dty0q7rKszb0mCMV832JNHSiBUhE0Q6IA8fkgDGU7pki/79nfHbAuaxbzJUVRkMYh47OI0PDSmcNhJTSuYV3VzBZzrqeXNJXCEO5NLwXWBy0V5QWpjnA4tAxaLaXxxF1m7YGyqrmarilrx3K5xrsFWZpyuL9PbzgmyhOIHU65Tryr67Z1ILEY7/BssMfQuaBk0O5O04zlYkODvRHTFd09KrhNdNiEPR3vy1uqsmDQH6KkQCsZghwE15dX4AWr1QpjbKc3EPKadVkhpnOiosIqTVlWoXP19kf8OfYbi8Drusa0hjSJiSOF9SFSdjak9kKGk3zTbKJwzmBNSVMt8D5ItEoVB1K/dGgdoj4pukW+0d/u3Lb3my0hwvmYVe1prlY4WfHw0JHp0C3ZfSCTnQECyWoZUdZrVm1Ds14x7PWoG4MxFVXtQhEj30VLTZ5EocNN9rhuBNaswuLx0I8lvVhgE0VrQlu/kKEjsdESFwWZ1aCHrhAiR0cTRG6JYkl/mNIbjtg/3EPKS+gW4rZ2oBOSXkScplgf0nNjk6Cz7i3OVNhmhTVz6sUFQiZESZ847ZP2+vSyPmkaBUcON5112yYfsdW7kGIjF0sX9QVHUkWSorTUNrShb6xcXeNMiRIeJQV1a+lLQVWtEL4lUjHCVzgnQ+bkwLcOmpq+rhGipTfoMV+dMV9+iakvqFuI4xVZukMvS4m1RAtPP45ojcUJi3YVyizY2z8iinZ4USwomwbRWIQTCH8jy6Ul7PTAOsHpecNsvsJbz6ODHrFfUZcrhj3Jw7s93nk04nTq6Kcp60VJEnl6qSaONS8vKpq6RW8keDq2T2MLWgORCnQ+BUQy6FkqETRPWjSZCv0FqRbkMmhEN7cmJ20oZ3bD5SdQIG/6AATWQ9M2FGVB01QIbxAorDUgAuSykUgwtsXZGqWDTj0YWmswXtB6j2nDQAMlW/LdhkePjvjoszPmiyXL1ZJ0p8f2prmdHQJEmhZP2TSs1wXFeknTCGScorQi8gpbt5jWIIQkz4OMLtJTlzWr0qCVwAmNFoJ13bBYNSAS0qxrqIk0cdrDGMlyVeKygOWjPZVrcLbGthYlNNZJSlFvHZ8xhqZpSZOE4WDEYn5JVbkQWAJSKxIdY9p2WwvYFnw6C8wuT1WuaeoKKSBLU3aGQ/Z2diiXC9bLJZcnzykWV0hC0Bo6/TWtddRFRYtguVzT6w+3m8avst+oA3fOMRwOSZOEKkBOCMRWO5dbEd7mp20bvChCJ6MLVECVQR7dwcvQ8iuExqPDBdiKn3SfvanUC4VTCUY4nl8uSIRhf9Swaav3Hn74w58wmewhpUBpGPYTRsMxeRRhGkPbGFpb4VEsFktGgyFp2kemI3r5Hqkek8QpSkmSRJMkEtusiB7cBRkRZz2iJMOLwCd1BAy+rSrmVzM+XK9pLYzHI5T0xEqA0/Tz4facONuEaj6WNt8hyYf08wwpJb1UMcgNrTEkSUoUR0jhKVaXnL54wsnZC2bLM5SKafoj9P4+4+yIYW9ElqTE2yjb4zvPLWQYAycImUKYwhP0juvaEEcGIS2ubLk9EyxONVVdsFhMaVtD0wburrINaSRJk4CtaiFxUiFcUNpLpGSv75kunmLbXYS8JMtPUNkJcSw4y1MS7Yi1J0k9PW3RssG1DUZ6YuGJfUtVrFhLS2UtTkcQaXB22wEH0NSGV8+ucc7yz3405XzWcjhOMMuKP/jDOZ9+sWSUasrrkmfFCbN1KAgrXzGME4YpeAxfnbY0VUosNyJgdAhTgKBuQKuAKyvhSXSnI6I0lXK4ShOpGu0Ny0Zw3pjt/MIgi6pQdNo2BEgraHaE7NNYC0J2OuMSax1tU+GsQUVxKJq70HDT1mWAKp3CmlAXsSLhalWGTmjn8cYivKGsCt798Lf46vmC2WzJq9en7I/fCdku4pZzEwiZgk4oK0NRWZwR5ElGrmG8t4/SCVXVMr2YMzcLPGDwKBsqjeWqoCwNjW1ROnT8NnWoV0gh8F51QlqK9aLkk59+yXS9w933x+ze75HtSNA1QrVIZQlTuwSWGwceRYosj0mTCIki7+UY0wT+u1Ls7e3y8OEDLk7PuLq8oliv2cgq051/Ibpz3racnb7GWk+cppTlimdPvmC9KljNZphy1Y0O9B1UrEBIVBQHjSBgNp2SJClKyl9LWvZXOnAhxH3gfwvcIYR8/8h7/78UQkyA/z3wCHgK/Le899Nf+YkEvYWyLG+mX3RsDFAdBAJv7OS39Do8Fm+CszPCUKwbri4ihIrp9xUyjrBW4JyGTmskQIPdZkDQ9RAyOB9UimslV9NXpFFFFrXbz72eVVzNXoEQJKnioOzxzffucjVdUlWbMVIxuwd3GUyOqfWYqR0Qt0NyN2aQTHAdhcnVNsz3Mw274x12Jnvk/T5JFhyrUl3S5jxFUXBxdkFZFTx/8QwhKqJIE2lNXVYs5+ubc6IUOlG01ZLp+RPM6iX7k30mOwekeZ90JyGLYnZGPfIsAuFZrYc8Pj7g7OwepycnnJ+fcjV9xWz2kouTZ7z37je4d/c+WX8MwmGNw3oZqu5NgJKssbRN0Ktomjp0n61K1k2N8RFRkjPu3ywv1SmtNbZltV5hUTT1GuUMaZSSJopBL2axavG+BdkQx45+5tjp19T1OU7tce9wxGj4DWS9x8svp5ykjtPZnOV6SdzzGF8gG8s4T5F5SpTFRFHQUD+4t4/UETKeM50VYFuiuGQT6KxXhp98Oueq9Pz0Zc2kp2iU5bPPlrRGIJynaT1lZUmEJ9fhRtzJJf0YxomgFytiPMPY0Ys7rRvXJexCdpjztkTcKS0EFpXqhiRoX/G9oSTT4YbLtWC/f3M/RDoijmJQGtnBi0qHZq6t4p0U6EhxcHSAqRdMzwvWbYtrDKoORAJjGtq2wjY1UknqXs5kb4+kP+TlxZIXV0vee/QIYWrq9SKM+CtKdgdDJvu7nF1O+emnX/Dt9x8RRfINvDhwXGO8TXFtQy/WPLw74Di7Q1tbeoM+rbXMZkteK00aRTTeksURmpCZDId9dg4nXFxekKYpSZrjHHz19AWXFzOa1uK9xTiHECXMPPbrhroxVCXce2+EyhVRJhBxA9ZjrEc6veVKDPopaaqItKZpa3b3JrQdFzfPU77xrQ949O4D9o92ODu55OTVGSevXiNd0FYRHTvLd7UkY1qsE7iqoGkKrq9Ogj6SC1Op0GJb+NQdy2Qw6DPcGSO04umLl0zaHaJIEEV/ORG4Af6n3vsfCiEGwJ8IIf6fwP8Q+E+89/+eEOIfAP8A+J/9Gu/H1dUVVVVR1zWz2QznuohbBA0Q0S3kN1tBgkkszgfJTW+Detp6cU4aDVDWIXo1UvfBp2iZksQJcayIOtEkrRTCBx0F6z2taanLmPWyZl3VKGHY7nta4rynMYbVsmZZ1XiZkeZDhsN9RoMx/cGQOBvjoz6FiBE2QTcJ5Uqwrha0xqGVIE8EvRTyOEboHjLKUDpGCNUVXQMWfH15xcnrM16+OGF2NSNPktAlaBrW6yVl0fLOw72tA1cqIs36pFGKqyPqesn1dEXbSgbDlqyXIzJHWQq80wjpqcsWa2F355DJcIf7x0d88eRzPn/yNVVVUFdrVqslxrku7V3ROk9rg3ZNaDq4+Wm2g6FlgLSSHHT2xuVbrZYorel36aFHIYUgFhH3H9zl0cO7pGnEcnXRydECPmDRRSHp9/bZf/SI8cEIJRdcPr9GIdmdwOurFZfTFqs9lJa4bshUN/klyam95b17D/j+v/YDnjx7heMrrpcvMLbhcDcO3b/AphBbtZ7dvuagr9jJBLHwCCWY9KIwMFgLsliSqOCO93oRWSToJ5BGgXGSacijwPro+ri69mlPNzUtQHrb6Uo3DlrT0f38Rp1HIFXTzTYCpXSYpejCTEs8CH0jo+C8QwhJrDxRFOOcp6gCZNkUJbu7R/SHOR5PXRdgDVmvx+TgkIPjY3SasvzoZ/ivXvLw3fcZpJrZ5RlPPv8Z1bqAtuXxw3vMVwXPX7zi5eklH7z3EOmDhGuwgJ1705JFCjXqkeQ9BvdG2ArKpmRdlkRxgpARO7sHtKYhktBPE/IkwpqG8d6Qd995gI5TiqLm6bMXlEURfEB36qyxVFSojjAoEDhjaJuSvXsD9u72UIlASItSNshLd6dbStE1D2qkteR5n0gnNLJBEgZyXF5eg1Dk/T69wRqlNb4xQeitY4JtxOIkInRf+1CclYSCrFKhlwIhcN4jRYDNUIo4z8iHAzxQVyVFsSKK06BF/ivsVz7De38CnHR/XwohPgXuAv8O8Dvd0/594P/Nr+nAT09POT09pWka5vP5BijctrYLcWs5+xsIJTi5BuMs0sVbMNaUc6rZK1JRo/0SnYxADsnzA/b3RgyHPbI8Ik4U0jnaxlCWFYvlmqv5NdV6iTBrqqYii8INAqE4aBHIKCbJYuJ0gMjvkUzukI33SHsDdJpSW4lpVGhEUg6lW1oraLWiNY48DSmSJVDEKiMpakPrCrQOg529a1jOrzl9fcr52QXX13OsMSRJ6OYq6jVlsUJKQ9ve6LZIKYl1BErjlQ/Te/AUjcOt1tTGUpUl6/WSSIe0r6lbmrYlS1OGvZT+YMTO5ID41RleKBZFCVeXyOmM5WJOVa5xMu4msusu0uucbEfTREhUHKGjFBVlSBXhxU02U9cNIImTNGjnS0HTtPR6GUd373L/wX0W8+vQAIUgaNRIWispm30Gw32y+ADX1qzLKVeXl6yKirgv0HFDXTcspoI8ifEtRFZQlAZrC1rn+MFwzCBLSZMI5y3L9QxDybe/8c2tmJUQEGtBHgtipRklgiwKWL+QnjyNqK1H6aAaGalQgchjRaQIXbvSoxUkmq3Yk++kBZwPzjs48I0mDV2dfgNBdIMwfKDRblBuIdzWgdMxhJTclOFEoNHeuk9Cp3LDelkEjR2hyfsZs+mcx3v7HN19ENL8KtQlsqzHYDJhMN7Bec/k5IpYS4ajMffu3mEymVCsVly8eMLV5Tl3Dt5jf3fM2clLPvrZV9w5PmKn34087G5c5xqcgThK0DFoERgfiY4opiUeSJKE/YN9PDHz6TXYml4SkSWK1bImUpLRaEzZWBbrGWcXl5RldUNS6BoCW2eoEVvpiEA6qPEmNOvkI4mOA8VVKoE3m/tHh0Y1FSEwpGkenLnUCKGoKsNyXhCnCc6BlAqtNaa1HVsujD3xmwVEJ2nhXRDp7FrnoygmS1O897RNg2nC1VRaoyKNUBLbDVevqirIKf9ls1CEEI+A7wN/ABx2zh3v/YkQ4uCXvOZ3gd8FGI1GAJyfn/PFF1+EL6AUSkm68QnhBpY32ho39OKQHpqmxFqD0CaAhjLC2RV2bWijFY2cYpsBOtmll2Q8OLrL3n5OFHmss9TrBaVbUTVTiutTzp48pW4d4/2MurW0RpGqwNEuyhKV5AzHE0a7R4z277N3+IgkG4WOS2DZtKzXRVdEUkRKkMQaJTKSeEiSKHq9hDxLidMIGSuK2uCXBd63eOeIZERdzXn98hnXV1Pqug7Raay2bext21KVJXHkWa4W2yaZcAOHOpmMM6I4C9oprqVsDVVbIHwnOuSD9rM1DU1ryLKM3fEwcE3VgF5/n8ZZFpWjMHOcbamWM5xtifIdojxDxwOk0gG/k6GFP1ToVeDgd0U0LwIevDEpFUpHaOeJrAACVao3GLB7sM9kf4/lahb0XpwAHyFIcCQ0fg/EOyxnkvLiKdP5p1yenjC9FJRWo7NOl2ItSKMMpMU5SbEqWRhD6wU4z/X5GYvpNYvFNdPpGf2R4m/84B1OTlPKKsQDSQSTHIQPzSVShIKadI44UkgNWgVdmxA9hwwLIcOkdyDWklh2qn7+hsEjpA88/e66IQIHG9iKjIVz1UHJNmxmvsPPt/dU97+N5K8UYT7j7VoRCGbX15ydXVDVLaOdPXb39nj56oTxwSGPv/kdRrt7VE2NFgKtY5AKLyVFsWYy2SWNFM47hjt77Ex2Wa/XTM9PePLka75370Pu7O/wdZbwpx/9jG9++0N66d0bboYH29a4psVKj7ca0xRUC0/sUq6urymqKmwsvTFJMuT66pq6LLFtgWkVxtaUdUN7NePkYsrT5y95+vQ5TWvCIG+CAw8SDg4IQ4mlB2EtpmqQPqI/7OOsor+jQn3sVmYY6ZhIhCi8bU0HTyVEUU2a5OTZgCTJCVt10OJP05RVVW/ZV867G4qh6KDebm3IrsFwMByxs7ODs4bFdMqibhAIdKSJoigwjTqd8qZuwmSkv0wHLoToA/9H4H/ivV/8OuN+woX0/wj4RwDHx8ceAgbeNOEgrbXEcUxRNsExY9BKdx2A3Ugv5zteuKNtK8JUerpdzKCEQUcrki4Cr0pJ3QwRNmPcO6Bdn/LJl5/y/PlTrs5PwbasV0um19dcXV6yf+cx/b1/jcYoWiPZCCWcXV3x7gfHHN39kL2jd8l27iCijMo2NE0H4TSOugk3n6RFpppISwZZzN5Oj8mkRy9LkTJoCGulAmZcVXjhKNYFr56+Yr24xLQVWmsirVDaIwiOyLSeum6YzxesV3O8s7R6B7ZT+kIY5zcptwwRRJiI7juBFIcjYIbIGJkIGq85XwjkqsWjGR98EMbTChGYLW0NLqIq58TpiKy/R5SPNisi4Le3ilbc+ttGG3xjR3eOKcuK5WLBxfkZi9k03EhSh+xBKryQQRjfAJagaZGkyOFjKvOI869/SG2fUjVnXJ5F/PBHBiNrIh2RZYpBpsjiHiMpyZKYRIE3LfOyZnZ5gbZhWoqpC7Q0PHpwwIPjCdfXnrIKuLQSlkwFssFmErmzAd0dpQTmCqE4bjzgoBcrGqnCMAFv2M3ErSxyczY8kRLbR4KcRPjdFibpajUbkTCtgz5ImIp+48A3KnbeO3S3iYZBBreicODzLz7n9PyCO/v7fPe732H34ICffv4zLmYL5mVFhsBHSWBmCYVpXafHLRiPd0jSlKvpjOl8ycH+Hg/f/YDT51/zxz/+lN7xF0Rpj3ce3eOPfvQxL54/Z5zqIEoXbnyasqWtwBmHkjGuTrh49gKzqFksFyyWS5yX9AcTkmzEZ599yeXZC/qZ5PHDY77/27/FbFHx0z/4E04vrlmuC2zbksYxsQpFzdqDsQKDp7Em1BOMJRYKIWF5suZp8pq6HbLn+kRpRFVZ0g7p6Q16ZGko+idpzKDX5/r8jF6ecnR0xN//t/4rRGnC1fUF0+tLzk5O8NbyxWxx2zcioRPickRJwkZtcNDrsX/ngHv377N3cECxWvL8y6+YXU/DxqkUu5Ndjh/co65rPv7oE5RSpGlCFP8lOXAhRERw3v+B9/7/1D18JoQ46qLvI+D813kvCGpqvV6Puq6D9m1lsHWFaS1eNURpj0gpWh/0d8OcQHC2RSpJnERIqbCuBVuiZIKSGucC7zVUgyrq+ozZ9Annp6/4p/+vf8JnH/+YJNIcHd0lSnLiSNEf7bB39JA0fYiSBiFqNm3lca5YVXOKcklTLVBLH7BbnYBKEGiUEAx6CUmSMBpm7AxT9kY5B5MRrTHsTnYQwuJcQ9uUzK4vMaVld3fEdHbN6ck506sVWkfEcRZokNJCaEUJY69ay2K54vzyiuuLC4SPOHw0CnKTXYp2e0zUJlrznVB9lKahtkCXjjsIUfJWsgpEYB9sRr9sylGjnUNsWwJdo1TXmhw49m8yfMTmrcKaIbpFIzRtCb4lSRQ7OyNirZjsjhju9LHWcnU1J0qGHN/v0bYW06zDtYgsRaOIkwijDFUjubhO+OGPGv7kR4D2HB8ahjns72qUSoicIk5T+sMeAyWIp1e0Bto28OWTeKMZvc/XpzVlE3XrPEAf3oXTsBETcj5Ezz23Ge4MznfhhQOhZMep7sTDnCe6dUmkFEQdJ952m54QAXLZ7IGb6SxKBozUdtSVMKhF4PTNeY6iiCRJUD6wQzaX0HVDB6y1nJyf83t//MccH0y499777N17gPUwObrH6XTK9XzKzsFuh/+HKDCNY/ACZyKygzv0xxMur6aUZYmUiny4w3f+9b/NT756xY9++hGP3nmHR/ePWM3n7PQTXFuHeZLdd94Z9ugPGoz3SAxNW3O1mPHk41cM8wQpPMYYXs1PWFUnvD4/w1QrsmxMlGSUteGjz77ks6+eUzcWJRWxjpBCk6U5SZSxKkqaYk3rajxN0BryCtfNv/ROsL5umb1ukRiSgaKqJGnHfExiRZpprLWdImXDu+8/QgvJcDjCeUPTOLTWTHZ3EXiuLi5o2gYdJaRpGrDrtgHbAoIkiRFSkWYpB3fu8N6HH+ClYG0Ny7qi8hYRBW2kuqpZTWesBoNA9yXASkmSolWEc3922Mxt+3VYKAL4XwGfeu//F7d+9R8D/wPg3+t+/l9+1Xtt7ODggKqqOD09ZbVahYJZ23Si6iIMvo1TlNTYDTOpU+xKkpS8F3cFGBtoUbqPlzmWFEWC0gmonFXhWKwM/eEB3/2tv0lTOQbDEf3hLjoZImQPZI8436cooKfBZwHOEQL2dg85PDymn0W0xRWr6Uuuri7J0iFRnBFFCXGS0Ov38GnGUI2QyQBlx2gvuby65vz1cyaTIVJYptNzPvv0I/Kkx2DQoyxLiqLBy7SDkTTPnz+lqlf0+xkP7h/jsTx7/jUvX75gPp+R5nmYKLS5Pt1/dIUxgE2OuMFIhbdsXPKmGQc8XnZRomeL39H9W3RFZC8lMunxRldB17SzwWy7dwvda9t6oL/hiQNQMJtdspivKIsagWT/cMj+JEA4Qih6/R3SLAtOy9WYdk3TllydLTD916yrmvMLePpM8PS1Qec9qqpkuTJEwjHMg/b6emExtUUvS7wWaBUxOriPSqFe1HgXs79zyNHhA6p2gvPF9ppLKdEyYMiyK0xtdqU3myu6foON7r7YNHmE8yK4UYKUXSrtEQjnAitqcxrDqQp1ge6cy9sfI28+b2PGGNqmxeAQ3aYSNtbwvLpp+elHH4OUvPfhNzg4ukvjwEvJ7p1jLn/2FVVdY40Jevbehy5k2w0bcR7jBcZLoiRFdhRFFUUcHt/n/W9+m598/gQpFe88fsyDoyMGg5wsSThdLQgplEBEGpkKfNvQNA1F5ajqinVRMUzDhlGblov5nItFTV03xCoiiTOEV1yeT7m4uMYYu2Xu4D1JHJNlObPZnKqqaOsKL7pOXBUwaKEVMk5AadrSsLyqyHo5cRoRyZv29Hv3jrl775imaVks5jx7+ozJ7pgkTtA6YjafhsYeoTCm5Xq64NXJGWjNB9/7NkeHd9AqYj6f8/GnH7Farfje977DzmRCFCegJGmW4EXIjisd+Pc6TTi+d4+ry0tOTk+ZzhekWR+84uH9h9w9PsI5y2w2/zP+87b9OhH43wb++8BPhRA/6h77nxMc938ohPgfAc+B/+av8V5AwMLv3r2LUoqnT592hRu5jRKNCRO3ZZQgZByG9vowdy+KFDoK0bYUFufDYmmdpjYx3qREUY7SAxqTMlt6RqMBB3e/xTd+KybKegiVYV2MtTHGxrRGUxZn2Ny/AQnk6YBIakxd4E2LsZ1egmmJ46QbxaQolik60pSrnOl5xvmgx8vRiPmiYLUsGY2GRFrRmhrnJFVrqa6modDhBdCilGe1WvDFl59Rlise3L/Hg/v3uL4+59mzJ5RlSdRpR6zXNzRCuHHcoovWuopK9+9upHHnoG8c+C3uQweqbl1EF2Zu/PptN3zr6Td4JzdO5M88uTMpG9p2RVnNWRcl3imuLiIe3jsk0pIoCilvFLugPYNC6wSExdsYax1lobm4VJxfaIQece/hfV69fElrFqzLjicvHU4qjAfbBKqn11CpiKqtObues14uGQ4i3r+/x7inWUpBy+acSZT23aSZ7vt1mYriZmxdKBRuulRvbVybc+rElmmyATXC+fS3z3TY6LwMAwxuDaa+4XSHN3S3NsO2aajroNsnvQ+dzDK0yFsL8/mKi7Nz3n/nHY6Pjsl7PVzn6A8O7vCzT59QVZa29fT78QZ3CxmflLSm5eTiEhWlPHjwkH5/uNnO0Tri4ePHfPXynMAzTzjcnWC9IZIapdbbc1A2nqoRWK9xTuG9JNaKPNYcTEb0s5TVuuRqvqBpCrwL0ecg7zPqD1GqKxZ261ZLQRJHjHfGxFFEc1FT1QXW1GgtGKQpw16fLMqIdYqMQkG/qQ1yXVGuG3qVA3fD2Bn0++zv71HXDVoJXr96yc5kh36/h5S60+0O8gKLxQJrHcvVmt2DA377r/+Ao8M7xDphMZ9TNmt++tFPOTo65PG779LrD2hs2829DRHWSZqyvr7mNNJ863vf4eLsnFcvXjK7nrNcVWRJxjuPH3F0/w7T2ewv7sC99/+c29v/m/b3ftXrf5FlWcbe3h5aa+bzOVdXV7eUAAXGBB0BjUdrgdoMGdAStaXWdMNFfVBJqa3Ht+AagdMRaZTSmISreRiGkOcTdu99G4umaQW2ChoideNYFxVlsQCbhOJcZ6PBCK0kdbXGOY+KUoSUARulxVmDNQ5jg15E3SxZzDRXSUyaBQe/nBdcXobOuiRNOTg8ROqYYr1ASxUGxwJawfTygufPntK0NTs7Y0zT8vz5c87Ozxn2cnq9HrP5nPV61cm8drZx4F1RCyFuOZSNZKzgDS+zOYfcet4bv7n5/wY2QGw+6haWu3kP//NkuFt7IdDrJYxGGdbWSGGpa0tRLMB74kiTxArjgiqlcRbvg8C+EII0T9Eixtqc1TJhtUro9QccHh1xPZ1SLgrWZYvC4kRLrHOscxhjMBh8JLkuK4ypOL+6pllOOdod8fBORD6qOX3lKLvjlKKrjcvbe3mISgO7MdBP/TZreePE3HxxQac7cnNSPJ5bu0LAt2VX+aJjqrAJIsQNh9xvJMaCGdNiWrl14NCGrmMRUVSG84tLTGv4xvvvMeoPkULiXYsQjv29A5Ikp64NZdmyt5sESt4mLZCCsm158uIl+/sHvPPOOwxHIxCbVnLHvXsP2Nv9EqkUZVXTu3NAUZdopbfdg95DY6BpZGCNGYFC009T7h/u8ujeHbI44nq+4Hw6RZ9f0pow1i5LEsbDIXESGCuxCi2tSRIzGPTZ2w8F1cbUtG2FEo5BlnFnd5ed8QSlYpwLuuZ1HdRDReEpVhXFssK2Hnr+jXXcTRtEdRCtjkI7e5gPGyQmqqpitVrRtC0PH93n8PgOg8GIWEUoJblzfIdPPv0YY02Yb9nLiV2LUqFZTCnFqpcTR4HJdXB0h/5gSN00LBYrFosVu3t77O1PmExGVHXFr7LfSCemlJI0TZlMJty7d4+Liwtcd8P5bkEaY2jqEms8URTGPWktUUpijLkZmIDD+pbGN9hWYaTCyQahWpww2Isls2WAXjwiiC01Fms2DrilLNZUq2sE+6gOhxLAu4+PqBrDyek509kMqSOQCbGKECq0zCdpRG8wxnpLL43J0og0iUhjxc54yPx61clPBk2LZVGyf2engxwalAjyoRLH+elrlosZKgo7/+XlBZ9+8ileQJqkpHGMEmGD29zPG72SjYXUv4v5/M9Nt/Z+m8ptH9p6l85Zb8P5jcMJm0JgU/yS8Hr74WwdOdBpbgR75/E9jo8mFEXJclmwXBaslwV7e2MGg4wojqjbMBmnaU0QmzKBmRTFkjRJiKZ98H3wA3qDSbhRYo1FUZV1GF6gKybjjMV8QVEWQepUCw4ur/B4VsUKbZbEQpCqS3YnPjAwbg2QE12A5sO0DxChBmOl356qwOEOejZBpkF0p7CLzNnIIneR9fZahPKu9wEP9zZQBJVUQe63gws3AzTCpvhmSVh1g0+8C+ugbVuMC5nLcr7k5OQ1+/t77O/tBmpaGzYOJSWTyYQ7h4eYtmUxn8PdY5SQIaPVwdmuVgueP3/K3/nb/wWOj+6E8XnWbcutx/ce8Oidd5jNF1zPZqTZB7TeEkm9rRsIIcjSjEg61osVynoSFzHp53zzBw+4e7BLXRUoLTi6s8vTswvqZkXbrPCuppfHHN89ZnfSp25rdBQxHI3Y299n/3CPL76aUdsKoRyjXs6ju0e8+/gxe/tHFHXL9WzJxdUV5WJB0w0xXkxX6CRGqJjhJJzL2eyak5NQdJwv5njvmM6uOTs/ZblcYU3YrNvG8uLFS05OTokixeHhPk++/hprDN66QCwwDVkv4yc//SnXszm7+3tBmTOOaduWKNJMr6acnZ/Tti2vX58wGo0YjceMJyOWqwWjcY+6KVitF9R1ya+y39hU+tva2zs7E4QQrNerILWqReA+F0FkJzhaTRQLdnaPKMo1TVPTNnXQQqYCEQECYSWtUegGWhMKJ3USGnoQElcHAfi6XlBVM4rqiqIoONh7TJYekMSaQIEQ/LVvv8s/+8OPWK9rysoilaHfz2iamrJY4awN8pvJFXEasYoUSRSRxlFoSpnlDAY79PsZURTjhWRVFMSiJko9bW2wxuAsNLahbkq0VigtWa6m/PPf++eUVc3RnWMGgzGDXspwOCDv99GR7s5lKLBtpbf9Tay2cd7bnwQVNSfevBbds2+ee5PJB1ErGZz4ZkN4Y1P4M9c2vJXfgLydjYY5B/vDMIxDRbSNwVrHZGePJM5xXlJWLUImWBuGFrRtRWtKGrMGYzFuhnGhCzdOImbzU3TUgrQ0zlK3hub8lNOz19i2DdF0pIn7PSLlqAuLNgJtY6plzB/94Zr0ScG6uE+YSL09hbQtNzNEfeCAWLfhegu0lkjZDefz/kYiQ4TaQ6Ql1oE1QVFgM1zabqLsTbYUNF2BwPs2vlPA8wSxLXFzTje2naPoAzVUEBrUrPeUVcF6veQbH36A94RhxSKM3fMevDF865vv8+TJUy4vTpleH5BEGgRESeit6GUJ3/+t73F0uI9pNsyoCB2nSB2R5n3ee+99PvvsMxbTKWmaIjqJ4U3zifeeqigxlSLXKc44Em/4xoeP+e6j77Ber7iYXrMyhv5kxf69Q5ZtjZ0vkL5lMsz45geP+P53P+Du0SV5r8dgOCbJhjih+dFPfoJznt3dPT549IjvffNDHt27j5cRz16dcXI+5fLqisvLC/IoD53MUWCZIDWb6DtJE/JeStu27OyMODzcJ4o0xliqqsG0ljhOOTk55fTshP4g5+/9l/8uf+tv/RssljOMafHWbrfauq355OMv6PUHfPDBB+S9BClloAZLydfiK169eMnd+/f51ne+TRzHHB4ccLC/z3g84s7RHe7cOUDHuqNK/vn2G3PgWofden9/nyRJKYo1FxcXvD55yXw+JY1ipNBUlaFpbNAM94I4imhNRGsU1oYF6VyLCMIOeFvh2pa6mKKifigOed9NdW9oypq2bbEE2EPFktFol/vH+/TzZMvLxXs++uhjTl+/ZrVY0TaB3uhiQIaqvcoUUZTS649ovUGYEueD1jnAenXJ2fkVWRKFhS0jeoMBuaqxbYtSYUyblBFlu2ayO+To+A6L1ZKr60vqsmI0HIG3SDzDfsbR/pjWtixsFw95wIei2c9TO38R1bNzGTe2ieTfQGrpYJiNk+EN5/2LHLh/883CxnIrAvcerHHgW5xyCKFIYkldLSjWCxaLkidPzxgPj3j3vW8RRV3qGkWkIgZREeeCJA84ubMVg37Ccm6xtsX5FoSgsTZMPUmCIpxSAq08Go+xcLy3wyjuc3w0YtkOqUsNTm+P3rjgqDd6dZthztY5lA6Ca57AP97UAmx7qzgsgiqkM+H7+w5OCTNJg+6JcwE+sR1/OQwxCLuecKFqYTuIxm/P/c25NsbQtiJAEx22JaXCWkuWJRwdHzCZjLdp+4Z2CGCN4XB/B+EN1gaRNQk0TRCz8k6gpOTe8RHeWZrahklYUUwqIJKScrVkfzLEvfOApqxomiZkLbeO0TnHdLakdyaItcdXBo+hd1ehRUuiFGmUoZMeLkvoPzgkWS6ItOLu3WPee/SIw70d/u3f+VtU9QorPFUL07nh9dkcLSWj4YC9yZjDo30G4z4qVjx/fcKT5895/vIF59dXXTOeDxmwkggZirx5h+qHkWoa722YJyA3G7QiSSKEEEwmY5bLBY8ePeDevWM++OA94kTT93kYFKJlGLsmPd/9re/y8tUFvUGP3b0JB4e7tG1NXbdhJujViMOjQx48fMx7H36AM+EaPrx/n3t37zJfzHnnnffJ8oymEXz5xbM/c6/dtt+YA990KGVZhveQpilxHJOmEc9fwGw2JUkCZUhJQ11XeG9pmhrrgk6zjjR13eK9RwmNVilKRggB1hQIlqHttq2R3qKFRSlNMsiJszFpPiTrjYmSHloLpHDbTjIPvHx9jseR5xohHVVVcXl1io4TlFYorYmjNHRtKUhjTRpHxDoiUpo8z1CRZ5DFYXp2bTBeURmB9zoI+gtFqgXL5QKlJKPhEOsdZVUQJYELKrFhxJVtSKTFtyXQY4vhbeZeip8/z9u/bUpo3OKa/Jlrsn3uLUhX/KIq5i98PTfHs0Vhbj4pS/IgVStFYDXQsdidpa5qlssZ8/kZxbrG45nOr2mahiTNmUwG9AbgrECpCGctJ69ekGYpq/kK24b5gVJKlIZEC/Z3RvTyHKkU66oFGVG3JaNcsjdJODzuM3eGximiWzi+dQLb8f+cvQ0rhUG0oRkqfC8ngzyxNQEz39RwnQfZDbHtTgTOd0qZXZfeZvKUc+DtZojGpirxRiXh1v9/2XkP3z3VmvF4B6l1KPR7h2SDSYfr74xFSM94PMQ72TWJNXhvcSZAMcaaQIXbBjOA62BHFwaQCNMy7qW4NGa9WuG6bM0Ys/282WxBeuWIpUe2QNxNxXFNGKLha4RocJFlKWp8rsnoM9odMxz1kVgOJwOMS6ms5WpZcn51ybOXTxHCMN7JGY1zkjQCpTi5vOTLZ8/5+uVLXl9esipLYiVpfYMhoqgLrmdT6sYxPmS7sSqpiKN42xy3mY8bROjiLiMzHB7eCSPPRqNQWO7WiZQKqcMmcHx8l+9+93uMxzu0bdshDUGfpixLtNYc373Lo8ePcc4RxRGx0sQ6CrIg6w1sHFhuv8p+Yw5cSon3Hq31NhpXWhHHUZjTZx111d44VeFp24qqqhBKBPqTgIISby1Yj4rC4zqKaFqJ6nQnpI+JJKRaopOMKOujohwV50TpIFATVY2Sltvx6dVsBcKjpSeNBdZ41sUS5w3SSISQ1LIMaawUpFlCv5eTJSk2ihiOB/R6EZNRH49Ar2pmawM6QwiB7ZTeaFvmsynWtGGOXhyGREi1mUUZ2pJNU2CbFC3aLWa9idCc625WcQOZvAGPiA5L/QXe+5afwfNzDn4T+m298q+Ylf0L4RmIo3TrwJVWOGtDcdoHudReL2F3d0BVWa6un/Pq5ITlokTJnPF4zHBH46xnNl1SrEuur6/JspyqrBAEbneUKNJU089j3nl4n73dCUpHXE4X5IM+l+cFVhpUqukNY+YzaFuP3miT0EXfG773tp4Y9K5FJ/K/Keg638FSLmyQrnuTUOpjwyykawQN8sddVO5dKFAaJzqmQ0fLvHX+1C3nfbtu8fMmhUSrCC/CwOPhYLA9/pvr0G3y3oOzpFEEBPXBYr0iTjStYKtrE2u9WVzhu3ZTqwIs1OC9J+50AVaLBa4b3GtvOfCqqlktW/I4YhD1GPQn9AYjvBI0vqJsl6ybGat6ymx9hVeWwaTPeHdAPsgQygcKHhG+aTGzNSeXZ3z+5HNqV5LlmjRTCOUpq5rnz0/48slLXl1cMl+tcd6QZSk6FRCF4RLrWtM0crvOhZChnT6M0cR7h3WBUqpVYM60jaGuWvq9AePxiCzrhZZ6GeGEQkmFFBEexXi0y3e+/R3qpqFpWoqi6hQvFUVRI4Rmf/+Aw4MDhA8UV601WkYMx2OEUCwWa0Y7e//qQiiblG4jG+tcwBi1jhmOIt5JUzySk9cnlEUZpDMjxXodpl4Mh0OUVlS1RHgZJrarNZGKkGlG1usTM0ALw3DQI8/SEBlHGiFS6sqyWC1YlyvW5Zr7dw852EmIVMCjpQ5OfF1WWNsQ6cAWybOYqqoR0oPvpF9tjWkqWmMQUtDr5fR6OVmW4nzNZNwnTUKrsrE2TG3XaRA5cjWmKanLgrOzU7SKaBqDEIooToliwLqglucNbVtQ1zGDYYaogrPwrpMO9e5GNwOgg1Q2mhvA7dB4+++tv/D8Uue+kTHonvbnOvDbTvv2+ykVhSynE7o3oiGNE2Kt6Pd6DId99vcmXE8XvD45o98TFOuW66tTXr54gY4cSkWsVgWLxQLTNtTdsOW0nxOnmjyPGYxS9idDvv3Nb3B855AkSbmYLrlinxfPLyhqQ9FKLBG29TQWkh5I1cE81ndddDdfwFoXJsdLgSY4bQ/4reP33RCAYG7TRez81gf6jbP3phuWEYqejQXvVdBMYeN0A9wifQfDdJDN9rx2tDSxiRQ7PZ2yKjEmQHMSgbdhSrzoOjSdsaG9v8PYvXc4a1kt5wzVMMgzEzoInQnMFGddh7dbmi1P3G2vr3OOxWKJThJEHN+aKRuyJXAM+mMeHDzg/XvvcnT0AMyapV9zWp7ycvaSi8tXVLMZSZRwfDzh8GhCb5gS5wnedOvOrpkVKz5//jVfvPqK/YMh/WRAkoISjsV8wWeffc2zl2csmxqDQceQjWJGozQwi1KLj2yYzNOZFAopus5vKbZZh1IKrWK8U1xezKgrS57F5FmPNE5D5J5rjDEBDegKuFo43n2nz+nZOavVksV8zZ07d8BryrJFCMVkss9otIMHkjgJMJcQSKHQccL19ZSj47tvFK5/mf1GHHgURcRxvG2nlx3PGxGwvF4+4BsffpvhcMzZ2SnT6ZSiKNjwZNM0FJzWq3WHq1aUZkpTV5TViqJak2QDEg2x9jjTsPSeuq5Jkx5nJ6dcXZ7QmoLDox3+2nfuMOxpwARxIWLAh1FpQgZ5yio0n2ipWRUFrXEoren1cnZ2dog7hkiaJkRREM6fXpwwu4AXz76mbluKoiGN++yPe2RZymDQBzzPXrzg8y++ZmdnF63jwDsWoa05j1LSNCZJFWmuGYwHoQOzYxgZ5yhrE9JB3NaB/qLhwx0w/UY6/oucdvjFhgPB1mNvJtiHcP42mr6RRxUdxzxEkvr2JwnRbWAmDNQVgtY0eNs5KwS9LONgb48H945YrtZczxacn085Pbnk/Oycq6s5JvGocYZUPQbDAWkak/cyBoOc4bDHZJxzsD9hZzwmSQKVM++n9M2In+US4zNalbKyipOzFziRkT0YoFXoxrQ24N1SiK4pNXQCK7nh0xMKxc5ju47NjdpgaF4iQDEEBYPAKCHMUBDgTBhEsiF4Wy9C43CH4fiupmE9Ha68KY/d2CZbdaF/u9sIWjwho1VKYV2XuUrFttco8CCDDLIOOK9Sksn+JEyVv8WUCUmB+P+y92exlmVpfh/2W9OeznjnmDIjcqiu7q7q6oFdpDi3RNi0admyQFswDAMybKDfDMGGIFF+44MBAgZs680gDBiC/SIDfiD8YkumQZGSyGazuqqrhxoyKzMjImO447ln2tOa/LD2OfdGd7OrCDRdhNE7EZmRN849se/Ze3/rW//vPyCHoaQf/swYPdDi0km5PmXJFkVOkecotQFAa8WzZ0/54GsVX3n2Ie+dvsdBMWdze8Nq84bf/uJ3+Z0ffZ/z62skBV9/cMZsPuff+Oa/xs++9wxTKGSRk+UTbNexvL7hfLVgG1se/cxjjg9KHs7HPDk85LCsuLmuKauOYtQRKk/QEp1LJic5h8cjqipPwhqhqVd397T3fmC1BVzvhnNPA40QUoP0o88+J/hAWeQUhaHvGxJ+rgnR4W3EObs3n1JKUuSa7RZurhc8fPCIm+sbri6vmc/nnJ6e7uvebjYBYLKM07NTlstlCt74Y/dd6fipFfCyLNFaY61lvdkm60V5t8XP84wnT54wn8+4vb3l+vqam5sbVqsVy+WSvuvYbodgWiSEGm9bbLumXl8idIYeKHVxyA+MIfLBh09ptwsIa8pMMK0OwdcgSqSQexpUjFA37TCcGmLGpMB6iwvDBW9auu2S9eKK+eEhx4dHOBGxbaRtGpzfJfwkC9wQAl28pV8myliUgrazXF4vMEXJartOEmk10LFEYDw7YDYeMx0rxuMSlZVIqfd7/hAidjDh3z+kAANccj8ObbdFj/dohv/8I94V8T/EKye1n/cK+P679qhNRGQWquGa51myE3AW6/wgN07DsxjCPdhHIpVkfjBlejDmyXunWPsR11e33NwsWK03tF1PBEajkqLMGI0qiiIn0zqJPUyGyQzeebquw/pIWUx59mjK7dWGGC1vL855+eoLLi5qDg9PKIoyDStjxN6FMiFkHGiaAzVw6L4DAsIO3oipcw+p+Ckt6F1EMhRtAJ8Sf2K8J7EXidWiVXqXEO9QCxAMUafpuPcs2+DpvcPHZOKVVG4So4v9dRU+2aqKwRM//UDJ/E0ohVCpuEspGGWGbvAn8gOEIkmxepk27CLcohB4AdlgwOR9ACU5LR8iM41t+zQw3P11EkK0fPLF93j15kvGxYQYei6vv+D86hK04fTsEVoZbNPztfcf83A+wwDNekuhKnQuqSO8WVzz+uYcUXp+4ZeecjIbI5otk1JzfFBy+vCAWjeYzxTr3hKUQGcCowW36yUyP0TGAjz0td1vJXdsuLQ4mTSjGQprCJHF4obvfPtb/Mqf+RXKKkPI5NevtUosoeGZEwis8ymLVgryIjWBr1+/5tmzZ7x48YKiKDg7O+Pw8HBPBb1PuyRGnj57j3/yT17xxfMfcX7x491JfqoQihCC8XhM3bR4H/YdQJL0pg90NptRliXT6ZSjoyNubm5YLBZ3XTns+bBEiNEm/q0V+B2HYviXlJLV4i1lkXPw8ITxeMzR0REqykSTupfOAhCjGArwzkfED4HEOx6uSsWHyOJ2wXazRrBLqLkLnc1MogYKwPVJXODDruSm7bAp74anidtNCkiWInU2MuKsY1t3lCN97xzvPi+4V8CF+AMF/Q67RSYMkB9XwNm91f1FYQeiDO6R7D/gd74vcsebhgQrhB34IEiDaJE8RKROhkwhBHpn0cMW13tLP3Q2B0cjikpw0I7ouh7rPXlmqEYlWWYGPxjQMvHzlVJ7q07tA2jHh+8dsBo5oMNomB/M+N3ff7FnDe3OPz1LOzx6d2MJECH5pIjUpcUAzofBkVHssW4pBJmM6EGKHwAb0ntqOViPJYvB5A+uFCEqCIlrnaTraQgZh05aZsXe2zGEBN/5GDEqYbAR9vx/N2RBGpPtB8bJpyXg3LAAhYgYIBM/3MNCKZRIAIMadiTB75as9HH0Xb+3PQWGxkvihmt3B6FApjIynWGjpYspiCVGTyMM5eyIkZL4kJTFIVhUlaGqnGwyojQlUUm2bcP3Pvs+L65e0oia6rCgiw2qmoDSLPoVtxe3CKnZii2Hj0cc6RylNUKmxeb66hYRU3QhPoK6S4oSQ45ojBFr7R7aEkJge8fFxQVCwmhUIYTAWZegEmUw2aDUjCCk2M/zwgBd9V3Pq1ev+O53v8vz58/52te+xuHhIUVR7BeNpmn2+pd0/VKQhPcuxbj9mOOnUsB3hxBiYJ4UWOveKeBxCB+NUe0hl6IoGI/Haag1nTKqRlxfX7PdrAnW3uG0Q+GQgpS7N3BAi6JgPp8xn04Zj8ZU1ZiqGqHQe5/fHSYPpG1lGLgBw9ZWSImWchBrDJ1siEMSR8KiEy4dcN6npifKfT6z9ZF2EAdIIdHaUJYFeV6Q5fl+W2u0RhtJZhRlmVOYxC/ubKCUdzmO6Xz9HTYt2J/Xuxh3vOuOf4KsveENhn/fDTJ3X3kXCN9BKu8uCEHdPcwpTmxw0SNgnSP6FHws1IDp+uTBTggp8cdbvHfDlD8p5IQS5IXBh7Sdz/JswGjTX29kMjpTKsFyQkqECkQsJ4cFk/wg+VRHx8OHJ4Ni9u7ziFEOg3NA7HDp9HO5sFv/BnTy3khhB6XgBd4zhFIPw+OhWDvPkHkaB+744H4SJcgsDd0JgxjLDElVPi30+o6RIOJuoDpMRgccPg4ZnLthq4w7pSb71n/vnRPZP29igC53GytB8sQPfvBHuTfIjs7hHIOJ1sBuEgKcGgrRzltRoCIoDFJnSaEZ9PBZVJTlGKUl1na0bcAYQdcFrhdrgjdkuqGz19Q+8snzzzhfXONjoCpKetcmg80oqXtH3WyR2uAAXRTkeYnJ8pQ4bwyQ0bbJvjV6gczC/trt2EtpUQxonewxQgj0fU/bJlU0pGLb98nXRSlFXuT72LPUdCWrC+881rqhMfVcXl7SdcmB9ObmZv+ZA2y3W6y1SJFIEFVZMJlMqKqKLFv8mOfzp5iJ2fd7e/ph2hv3tByl5LBCpV8hpi4lMxmz6YyT4xNOT0+5Prnm/O1brq+v6JtmTwPabXmlUmRZRp7nFGWZiv90SlWNE1tFmTSBNvoPYcZCCI6Pj/duYEIkZzPn07ByP0gScl/LQnADPhqw1mFdKuBaaUQUBB8JPg2DlEjewkYbTJ6RZYq8yIgxoKQkM4YsF0wmU/JyRD7EK9kgseHusmmZwph3nfvdZp9UZ4et2b7ZHmrtTumXjndQ8T/0+3dhcvGHXzYUhD9YwIt7nU7fhwHTlYQIXd8jYqDM8lSDRLr+Sd0Y6K0bOsPUHQmZbE5j1IAeKHmG4HY5p+kBIqSOSu42ZUEkul60KXS6GhFDTu96Tk4f8MEHH5AXqThGofHZPCXEq8Q8SXUvkqT9qSmQ94ZdUe4gpgSBDCHoCA1e3RXwqFJhRw/FNkAMaZcSdI7OUg6iFMkiQgiddig+WdpGdVfA69Ym+9IY6GRyIQ9xJ/AR+2ep03czkfvPlxA78VEiD0i1G6LG/SWUwiV82NqhKKeby7vUXO0DKWIKVBBaI2NyfNzfm8LguzypH0NaUGMISBfQKkN4EK7FRFDaUa8kL18uOS96gpDcNi1r71ivbog2IEVJoccY1ePqDB80nY20TmJkhkAihULEHBlytDTIoBmVY5RI9FTvI+Uo39++bduyXK5TJm2IZFmGlDrtdrdb2jaRJtabzdBkQtM0xMhQW7KBx8++ow4hDX6325o8L2jbpCS9vr6h63uKvNjvGrfbGu8dWivG4xGz6TQtKD9GMLd/8n6SF/1JHY8ePYq//uu//v+zv+9Pjz89/vT40+P/H46//bf/9rdijL/6B7/+k+6l//T40+NPjz89/vT4V+z4qUAov/mtb/H9H34y0HQ8RW7Ii5IoZzhmKOXw9uUwER6SRgAiKCGJ3iFiGHiwkTLLBnxXkGnFfFxgtOJ203NxvaK1DqU1VVWQYozScLQaj3FKse17XO/wjefx6Qk/8/H7CCH4t//tv5lcxaoSo00KYV5sWDc9fUh4du891kfqjefyfIlzCWqxzr+ToqJEwntV4pKhpRzk2JGqlPzc1x9zfDpPUv4Y8UGwbD2fv7rBDXxnJdLrtRJ8/5/9Z7i+I+pAzPzASAnv4GthmCe0bfvOuYDYm4LFeI+9EmLCo9OUEawjuh5CoN2sic4htSIbTZBVhReCnfNTmr8EfLxjlGQq46A4AuDsZxbILA0rFSKFGQg5eJsrFBWVeMDx5CmZMTjfs12vuL665vz8HFSgmGrKeaSaeoo8IGKk6xz1GtqNxNYaPFwvrvnyyzeMp4c8fPgeB/NDurYDBdfbL7jdvKFuVsyKQ3wnGOXvo2TBZtvyyadvybWgXi34mQ+f8ZWPn/Hk8Rl5ITk9PSUzSZm32yZXZQUM8BCBEFPqkSDgbJ9wf6kSNTCA7TvqzS3etklfYDJiiEnVW4wwRUkk0DU3rK4vabY13qZQgd/4Ilm1fuvb/wChWo5PJxSVpGk2ECKnp8eMRxVd1/HixTm3S4vRia7bNS25Unzjqx/z6L3HZGXFpu54+eotL198hi5h3W2RuWE0GXM4G/Pw8AEvP31Ns1ozGRV88OwJWhX81m/+EO8iVVUmz5/VIvnj65LpyVOKySF9t+Vb//D/QnCK6URhcklAYF1gUkpmkxQerE1BVo0oplMurlqaPilUd1ASYphjsHPUTBChHOwDxTBARuwsqdXe0O0O5hw8wgcBIUKg4wMA/uvf/G1++KPn6fUBovdokwaggWT3nKCVAkhK07ZvMINXSRoQ7wgLu5lYGiInk6s0q5D6vlPju6jHDrmNJN8cKdJQ+ud/9iP+3K9+7Y+tpT+VAq4Gwybb9tR1Td8LVF3jqLHhFmMkedEjRaQsNFWVcin3D763+L7F1Ru6ekVpFNtesm16FIGMBkHg7XXD5WKLUIaj40OOjuYQI9PZnGfPPiCvKr7/+XNeLxaE3hNqRzebAqmgHR4eYoyhqkryLMf7QGYWsFhhvcBHQR8iddujVETpEd6nKXYyEUr4vhRpSp34zp7SKOaTEiWS0LmqNI+ezMlyuafUuSAx28DlKtA4AVKgdEo8N1qxD49N9W8ouiRqm4j7ghxDwOMTo2F4uVSJQ+wZhBsi+V8ImfxDTBRp+BUcfbMhOotYLXBNg/WeLjOoyQQ5mUE5QtwzMLp/c95NByCqFpUnL3ejksowCoUDQpCIAIIOmfd4ZantkovNW168ecHyZklZ5RyMxozzgvFMM58abNdz/mZNXfdsVpJuk5OpDOsaOtswHpwes1zhQ0TnkYmWhFyiG0GVtUSnEU0yPzFacTQvmZcZ5emYr3z0mGdPTzk5O6QYFVTVeGBnqEHQGMilwLsOqWNSlso0J+m6Fl0ZBIqu85y/veL2dokQnjwLlIWiykom44osL9OAXBukSR43TXD0qiaIDVEEjLl7VG+uFzh3ixYNT98/pveJvne7vaLFoDOFGnu6RU2ZJ2qc7x3NdkmzvcZ2FVK7ZMYWepytIQu03ZK+DTT9mrJ4gHUTEB1FCbNpxnxWsF332G5D2zq08uRZibcNfV1TVdMkkScVtb6+oq41rpPkhUQnvzlshF5tISuJrsD7LSrzFLmk6fyeKaNEmi/souJ2OgQhJCGIe18TA+ddgFKIIAlSIqNCpoFPSlCSaRERUu4L33bbcH1zm+woBkW3VBAJCCU4PDgkz4theGlpupa6aymqYhBoJRpiGFh0g2ILBvl8DAElBHleJEbSQJJ4t9m6E2ulxUsjhGS7/RN0IxRCKOCfAa9ijP+mEOIQ+E+BZ8AXwL8TY/zxY1NgPJlwfHLCer0hhIAeRCvRbnGuRokMlY9R0TMqDafHM548+4DS5IkKFBztesH64hWLiw3FKCduM5q+I4aWTDtE6GjbBuscVVExnU05Pj1CxMB77z/j61//BkpnfHlxSXB++DDfNebf0Qp3lEClVPIoNskbJSDRcWe1GSnyksDOl8ShlSA3KnXdA1NACs/xpOLkcEymNUpKjDFoDXWzJQg/dB2CLEsmOV0IBBlTtzrEd91dl+FfA/VJ7KxldwPM3Wu4oxVqKZO4Q0qMSA6KZZanLsQHRNMSuw5fbwnbNcH1mHYLbYOzDleDb7ZIH8l0htAGLwQ2OOBOURjuFfPFVU0xEeSlIs9AiEBE4sTwcNESZYZqBL13LJY3fPnmS56/ek7oYc4YVc+p2inOjREip+s6vvzygjfPV2yWghhHzKYppSfGmIbERic2iWoRxlFmQFGSjRyZSqyM5o0gdJAZwaPTko/PzjidjTk9mTM/nFLNcvLxCOJOqDTI4F0g9D1ds0JpyHKByQS4Bt9vMcWEpm548/qG737ne1xeXJDl8OTJEY8fnzIeJX+Y3Ch6a7F9j28DUTjazQ22XRH6BhEFkrshZnDQbS3dqiP3GZ0vWIeW9e2GLBiOzw6Ynh1xu0oCG6xGekfteppuQ90sCSLSd4JgPVIYlPTkxmDbnr629E1y4yvLgnIy4mg+pyor1rcdRZFhrU/Pi5SYLMM222S9eC9su8gUTSuom4hzgbyAokjDfG97gkze6i5Asy2YHKT0eRdDYrrs79o/qh7dm6PHgSkVk+YjysTAiUOsGmGnfE0RjSIGdmLMffHfNx+REEBpyWQ84uOPPuLs7JTr6xvevHnDxbWl82mx2BXtAESZOngxiLzeCVzRKoUWDwX+D7qEQrzj2kdBIBEZ/rkiu3vHv0gH/u8B3wOmw///LeDvxxj/jhDibw3//x/+JG+kjMGUJZnzCLlGyoD0HouFYPHe01MgnUd4QSEjh5OCg+khmTFoFVlfSd5szmlpkbGFaABJpiRHo0CpAj98GRKvVQqCdyxur1NRLHLGkzEpwcSnMGXPAMncTdGLIq281ibi/47Rcng4o2larAu4IFAyY9s4Ghv3365UuoFztVPaJV5Xpg2H81H6syzZgEYkbWNpa49UEW3U4JImk9Wl8FjS90cC1vt7n2aihe25uvLuZpW7rSSCnYpVydRt++AxyjArRxxOphxN5yghWJ5fcPP2nM3lG9rNAms7CMnvONeaUTlCmZx2WyObjhkSXYzolGKxXeMJuBgIBPw9N8JPfveSonKUE0k+EiAsUgvIDTJLgbJ91bC2F2zbnuvrWy6vrlh2C8p8TK9h2TrkooWsIcoJ5xfXfPv3fsSnv32FbSKHD6Y8eH9Ot4ooVw2BGRohBV5t6OSKIBuEtBgpQQaESgwXSItKlTX84s++x6Qo8HgQnkjA933KKdRiTxnVOpIbaOtu4IdnZFmOVjnaQIyKFxfn/PZ3vsc//Sffoe9ahHY4/zEHh2dM5w/IMo2LnrprkvjL9gjh6LY3hL6l0AIlNZ2T+3uzKgpiXyC8pK8dvlFc3GxpM8tUG06zGbPJhPUysL2+JdYdhsjDk2Oca+jtBuM0fStYLdb0nUTmkTIfIWSOtdBsLXXdMJ+MGWcZZVngXMTayHQ2TmpVlWT1mckIOhnG7Qu4gDKX1EVgtYS6jugtTCYROZZk2iFTfDxCCrbrDcdnE0ZVpHOR3u5c0O/Mw+L+fk8sHnmviHsYeO1yz3tPBVvu+bUJQhkaGzUsD8PzwFD0pQJjNMfHR3z88Uf81V/7y3zlKx/y8uWX/KN/9I9ovlOzrrf0XT80aul9lVRoJRAu0Pv+zm5gMG9zAx1zd167DnyX7ITYGZ2ln0iKu4DqP+74SUONnwD/HeB/A/yvhi//W8CvDb//T4B/wE9YwFfXV1x9+QLvAtttTanikAk5nDigVMSgkMHR315w9clv0Y1GKCnIM4W3HdKvmczGBBS5dFSZIzeCrCxRZJwcFGhjkSaD6FkuV6gsBZDa3uEDtE0LMRJcSmFPEtbhPFerpE7zfh9CkWc5SkCZKaLv2GxWXN4uMeUBUpSgNEiJUYZMC0oDs3FJZvTAhYa+7WjWa8bjgtG4AjQ3iw3W9hSlSYo3H7labFi3Dd0uv2EwWVL3O/B0gfYF+w9ct2F2EAc1qcRIzbgcU+Q583LMfDRhUlYpn7Dr+PLNb3H5o9+nXd2gTfKIiETyasbRgyecvvcBBw+fcvniORcvf4TuWsxqQVZURCRbpWgGHFjdM+P57m++wNsNJo+YSqCyQHmgGB8WTA9LZkcT3GHEuWvWtxs2N1vabU9RgBkFYtHjtGfVWtxVw9auuLxcIkcRXUmkgvmJ5vCp4M0XNWJTokyiaUqhWHXXtPYt1tskPmEQdhiJHD7XIjc8PT3hg6cPCF7QC4UVQwJP2xO8p6m3aKMoqyJ1t8DseIyzNtH+UCBLbm9XfOvbv8lvfev7/OjTc5QYMTua03U1n3y+wPGSycFHfOWr72HtktYmGqKWChF8cq5s+5RPiaPxDkjimrLUZGrCZDJmtemZjKYQr3Be0LWB2+stzaYFHckKRZVPmGUlk7zg7fULmmbFqBqhMPS15e2rmni5YnZsqEYVuTJcXdxQ5YZKG0KWrCG265ovX7xicXtLZnKkFPRdh5GKUVEiiKh7Rcf6QFVG2h5qC6s1tF3auXSuY9pLJpNAOVZsNz23VxmqGFHkir4Lex/1RFu8E7ohxN5Tfd+J7xmxQ07oAE3IQTiWnpFd6MY9aC8k8yolFQeHBzx98h4nx0d881f/DH/5L/9Fjk8O+cf/+L/i61/7WZa3V7x+8yWffvYZRTXBu8FrXSU3QjlAokqINB8iokwScMR7sMo7xfseDMS98/sjrTD+iOMn7cD/D8B/AEzufe0sxvgGYEimP/0J3wslNZnJIRPoTY2SUBYFJoBUFqEkD2cPid5RsEVpSWZGdG2PFxDMjGI8YTY7Y3wWEMpwYDXWR7SCWZXENqMnnqb3BCGQ2mB9GkwUxYSbm1us84xHI4xSbLoG33TY/q6AW2uBtLgIIYf/T4NIASxuLvjWt3+Df/SP/yFPnv083/iVv8Th6WPKfIRRIuG8MaB0QWYi3jtW63XKuROCbWvI1hrvYb1Ksv2xqyhsSZQZ680a1/cIYZIab+hI3rlow8oNvKOC23UhOw8LgaAqCo7nRzx7+JRZWVEYPWzV0tBShCHAITpwloggz3M65+i2W1zXEayj32y5evEJq4tX9L5HKkU1mXD60dfI8xGXbVKkanG3oES1xoVN4nV3ChVBbjQiBqSLiF7i1zEFM7Q9oXPIGNFKofEQLF3dsV5scB6KIqPzNVkeODw22I1GU+C2SWeQKTEIuAyRyOp2w6JbYK1DRDBSYJSATDO3Hk2SxzdBcnVzThYVwpSovEQVBmTHzfUN9XbLbDZnUuVJqSsg2p6wXkL09HXG85sl//e/959xc7PhzfkN51e3hHjNLxz9LM2mZ7WqadxnZOMJ48MpB2NJ6By+7/DRYV1N42pqV+Otx9tA3UsgDd462+P6jj5kZBnEec/T6SnX21u60LNcXBCCwvY9Z8WMpw8f8d7JEfX6lm03pxCBg3JMLEsWp45PXt6y3EYOTwrm0xnaaBa3C+bVBGMdeVCELsnZP3/9Cmctjx4+JgRBvd7i+o7QdORFRmZ7chLEtFgHohTYkPoaZQS3G0FtPdMWto2n7XsOg0DonPPzG2ZHkkxXjEvFemMB+c5sJckNUvEO+9oc9/f8jo+9ex7kIBLbDdp3Ssu7x0cO3bPm8PCI8XTGarPlsy+ec3x8wjfM1ziYH6GF5s//2T/P0cExs9Gc//o3fhPKIsEnPhCcxzpLDIFATAPOobPeGZHt8PJdsxVJvPPdOXmXGkXvB0z8J6B4/ySp9P8mcBFj/JYQ4td+7Dv+4e//deDXIYUZAwg9wlSnmDynWLXgWpAmeXXrEaNxyccffh1bbxC0TErJdD5icfWa69sNVkIRJAHJclmjjaNtHX5YeXOTFI7GFGR5RV4W5EWRvKRFyuq7vLklhkBZFHSbLX29SUKbeFcEjVZEkjhCIJPcP0TcIB+21rNZr3n76guWqy0IxTd++c/x4YcfMy5LgkvDMec91qdpdtf31G1your7Lg2+AIFCZ1naHkaPiI5RYWj7SB8lPiZBjhKRXL2rowkxyacJg3DDu8FeQBBcCvYdlSUn0yPeO37IVOfowY9Em7Q4CB2JPqJNipza7SR88MnP2Pcsr87xtkebz7h4/ilNU+OCTyrZvOTg4IRJMaV927Jse2Kw+6CbqgLtBUIzwBbpvz4Eus4h1zV919MPE/3o067BSInHo12yGV7ddqwXFu8kWQWzY4ltk0lUfQvL1YJN0/L+iUk/ixRY79iu16zrLXaIo5NCIImoTDOuPFpB1/U8f7ngxahEtw2bVY0uKk4eP0QXis16ndS5ZYGtV/TNhiAFolmhuib9XcHz+RcvOX91w6PTJ9hGcHV5y3KzpHM9X75+hfKCXErevPic/89/1vNX/uIvoanxocWFni60bIWlVTHZHhhFMAqWu4dKURYV89kYNbHkDwwmGthY1ptIvbX0veBoesyj8TGPDk45nY5ZiJbxckLpIlMzImrDaGKIqgepiSiU0RSVJlOSqcnRfYuvLVvb8+WrC5quQ2cGkxcIF2nYJutU1wMG/87uMNJ1gugGzxcNnsiqGeYvEnINJnOUlSHQYJst2UhiTIFQyXjrHTfM1JnsO2wYhKJDUd8zohjESz7sX78bgsZ411hkJmNUjtBa40NgtV7Tdy1v3rzl888/5/33nvDVr3yVvmtQUjCfTDk+OGLbtvz2pz9MStWdbXAYGiZSALZADrJ60gAz3FlJ7Ng0+7COyB0xgTDg4feh0j/6+ElT6f97Qoi/QXocp0KI/ytwLoR4OHTfD4E/0nklxvh3gb8LScgDIEyBzOeovMBkY7re4oKkKEsm1Zj5wRyjx0QZyUcHTA9GzA9KWmvZvt2iGoeXHh/harFFSs1mW+Ns8vPWOm0FT05G5JMRWTUmLwqklLS9Y7le03UOoxM9r6troutRKttDOQBFng90MTEoAl0qZoMhxk6JttlsWG8tJv8ux0eHPD475OHhM4JPGKExelC7yb3Hrx8sYImKfMDW8zxPr5XJ/zw3ikJLhE8J5AKJkYJc3oUFhLhzxku0OgWpgPctoetxISniDsYzjqZzRsqwPn+Dtw2cnTGeH1AUo/RAaUOWlyk6yyS1WurgA947mu0K2yWl2fr2ht4Pbi5CEgNkeYVSBtM74nqD1dl+z2aUGAy8UhBCYioGohPQJphKKYEl7jur5DkhMF1KBfcE2rqn3lialScrFcFraHOU1/Q+crlYEzSYh3nyjJcCby1t29O1yUgrxjSrCNGjtMbnaajVtT1v37zii1JR2ZbN4pZqNKbMFR2wXm+YzaYUWmNkTMNRIsZ2SAJBpwX36mpF6CMFhpHMqLIcWyWst+taDrKCkYLYbPnB7/42P/+VEw6mGoSlDz02utTdI/c+JfZe7mmIEanTfRO1hSyghCZzisIbYtB0XcQozaQaMyoqCmOoyozDgxmmDhhh6FMoJ0Iki1vbW9q2SWZq3hPbDhE8XdOybj1121JVJXFI/5E+XXvvHS6GwSTtboiuErSMCAzGXhEvIr2XbFuR2EgKdBZR2mIC2KZNjUyRaHfR7aAG9sPx3dCPuxq+/+0gPH73i/eYWQlBv+fXYgx5niOE2PuSKCFomprl7YKry3M+fP8J27YlKsF0NOKbv/or/PXnf43z9S2vLy7obZeGlwPTa0BLEDumidyvKMNp7Ww4BnVs3PHD3qmc+7nWH3f8JKn0/xHwHwEMHfi/H2P8nwgh/rfAvwv8neG/f+/H/m3DIU0BxtH2IESR+LE+MisKHp6dMDs44eLtFV3TcKQfcFQcMD6YceS2uO+/pGt6sjwjKwxlYbA24Vs+SIgSKQ0mGzGdn3B0csJoVCXZalOzXK25uLhhPh0xm5QEH3BtT55psnw08D3TUWTZvYQRsILh4ic+ttIaneU0fep433z5OZ9/esz7j075+Y8/Rqv0XkLEfcr5zpu8txElFXmeMxmPk9/LUHBiDCmkltRxawKZ0hityJRERXsH+cWU4h5iQO1W8eCh2eCXN3hZMD15yoOjUw5GY7rtki++/23a9ZL+46/y5MOvkJ/lSJOhtKasxuTliDYriMETvBtuzJiiy7yj7zqsc9jIgHkmapbzns16gb1aYBeX+LyAh+k8gwPbp6ixKBPlDudBRVzv6AeOvNvdzFIghwAIIwVae6JKxcvkgk55ujqwuhGMTIbRGRJB33qycUZZVuRFjhDgvMP5iPeJURR8JPpA5z1CJsk+gLU9l28veF5JHhQa6ToUBaHvOL9acXlxzYOHZ0TncW1NXkq8c5RSEPMMHNxsW65vNhgM9e0a27SM84LJ0Zw8L5hNJxzlJYejkkLC9fU1y8UVha7Qyg8ZqQEtNLGRNE1LHxxW3rFQnHdYfIL0Wk/f9Og82U8YbchmJevNmrqtB0566vK0MpwczfHKEnykblq6uqNQCisD/bZmfe0IjUZby/Z6gZmN6fuWznry3FCOj1ltt6w3W1QaMe7ZRmrvlTOwW4Ug14LgkoQ9DIyTIDTb3ifbgSjIMsh0R8gEnbFkhcVkASFTyk3qUneDzFSR48AySX1uKopBxOFr3CvuYmjbh6QpsevU05G8khI8ut3W+OCZT6YE17PdLPnkB99jXGiInkwbjo+P+eArH/M3/+Z/n+99/hn/4B/+l1xuUqhLouom2CQEPzz7O1JBGBrvIcyG1LFHvwuRifdYWzts/08OA/+jjr8D/N+EEP9z4AXwP/xJv7GaHCKvHD/6/m9RsESolDpfFQXjqqQsM37vd3+PZb3ly5vXrDaXHBQ/Tz4qcdqwsYGjcszP/eyHzKcTtjUEkRGEHqAYQ5bnjEcjqqpCS0nbNHz2/Atev/yS589f8uzpQ6r8DNtbtMk5OTnBZCPGozuYv23b5JyYEhWQArJMD/CM4aA/5OTBI2Q2pms2eNvx4rMf8P2DOf/aL3+TyfiIzGhCSAG+LnikiMxnU7xP3XyiKfrEdsksWisQAh8SlUoRKY0iM5IikxilsN2dj0wMMSVjD/P3CEjvUd2WuLqgGD3gg4ePeXh0Quw63lycc/3qc9bLG5p6jdKa6cEh4+kMsIxnM/LRGLXMsX239+PIhm2sd0MQcwQX00KW5RmT6Yz17YLPv/d9rt98ge1qquOz/Xl2jaftUuetEyEer1LnG2WiYYUINgbQ6SFgJ3zKJUGnLE2pNflEIZSia0EKQ1lWTIsJuTGYkaFnw3w+oSyLBGu4FqUKhDIMoGXa/Xhwzu87tswYHp4esGkbtnnF2fEBWVlysbjkzcWSL19csW0Ei2XLbFZQjBLOXhiBEpLeRs5vGr71T7/LNJ8ji5zYKqyIjPKM8XjE0/efErc1265j3TVkecGoLPFdT9/VdHVDvempJkcsrxsubq7Z9A2xLDAHw2oooOs7bhdL+uslk/cnQypMKhZaabIsQyuJ9/0gLoqsbreovMA7QT/40+ug+OjwhJWs6ewCtWyQreBBPkJ1llwbnPUomSx+b5uGnsCm2yI8KJ9EKtK5O59cBhppiISYdh5px5P81hNWDLaXBA9GWgwCoyNaeCZTQWU0tRWkO31X3JKl7T54Ow5h2zuW1UACE/f+iXsmVBzwcd4p4IhkGNZ2CdZcr1cE29Osb/nisx/yD2Pgv/rgGX/5L/0FPvrwA/LC8Pb8LT/79Z/l3/0f/49SZu7tLdv1lp1n6f1jZ4qVgqMTNVAOHkrISG/t3qL4zlKZP/Q+/7zjX6iAxxj/AYltQozxGvhr/yLfvz9CeogPpob55JQnT5/RdIKqLJkdzsjLgmp8CGaUeNda4m2NV5pyNGN+MObBh4+oTk94cb7gxRcXHB08ZDKaUlaafDTiyeP3KAvNpt6y3m5YbRP39eOvPGNcZcwOxowmBYieh++dJiYKLT7uhAiRm+WS8XiUUqX7jsXNguVqydXVNc5ZnHdcLm4oyildnUzeV7e3fPbpD/lnv/lP+PrXfiVRl/RgsCUkUQgyI5FFCbveQUj6wRjfuuS+l2UZ0+mYpulZrVNSt+scq+3mHdc3GCitw33quh6xWiJWKzSSk/kB752eMa0qts4ihoVIa41ttmyX19SrW+ZHZ2RGU40nmGqEMBmx7yA4xG7L6f0QHJ0wuqrMOT455eTBY6bHD7l++Zyb159yu1yQFwUmv8ddbkF0aRstRUzm2FIQ8QSZ6FYMSUj7qYBU6ExhkuoIF1LHh3KgBeN5RVkcMM6mjExFWWQcV6dsmysODw8o8hyI2Nhjipw8lKAVBBBRYKylaer9tH88Lnn0y1+lr2vq5TWXXcPS9XgXudnWXK06lvUN1WVDUUmUcVTTEUUuqUpD03Y8f3HBpz96zt/4b32d+dkRN7GlvfZ0y1uOj495+/Yto7xgOp6QacUXzz/l5csLjqcmsZq2Nde3LYvPliw2W5z0FCPDLB/vP0uhJJnKkivn+IyLt+c0sWU2HyGE4vL8NS+/XPPs4RnX1xe8sD3rqmJbr6mbiF1L8kyDjJggOc5yjuYlLuZI1aFNwAjFdDZhlI0I3Qbf9wQPorEcjafUdYe3nmAtdVMnuCNZBA7PD9SNQBqBUAL80IUHATGFTas05mfTeK6WUGaeycyjtCfLAplLnawQer+L2LkfhhCGrjvdKzLKFHBBgksGEjl3BMS77jaEu+IoBGgtKYqM7XabdpsCuhBorKNrauof/JAH773H8cNHnEpNZ3s2m5o/80u/yK/9xb/A7dUV3/72dxEyOUjuFtJdHYHEdmEwD4sihcGkr92xZO4fchDW/bjjp+NGaC0mK3nvK7+I6zeo4pjg1jgEQUBeFFTTOX65ocozZtWE6XiExzMajXjw+BGzwwN623Hx+kvevn7D21dfMp6OODg44OT0IaM858vXL7le3iA0GK15+fxLzs/fMq5KqjDBhQxpSvKyBDzT8RHzgwNg2PoYzYs3r2maBmIkzzTb7YrPP/+U29sF603N9eKW9WJLvemJLtBlljeX13zvh9/jZ3/+F8izHIRPJXAIldhlgSqdoXSGVEkw1A3QRAhuuOW22D4xMDbbW968fcF3vvMdJpPpnZvjwCmVIhVDVd8SVtf49TJtn11Pvbol9i7BJgNbQsaAEuBsT9NsabsGISQmM1TjKfloQt82EHzix3tHcINVpo8453n86DEfff2XqSaHbJcr2uU1mYYqzzBZ9k7aiNCK2DFM6GUqpGL4TBAEKcEoVJ5kzELEJHLKDDrXeBxKgjIZeVZSmgkmLzFGoqUiU4Esi1QjQ1bMGI1KVKYIwmFp0IWkkiNEO+RxitQ9uxj29EsXI2tr2W7WdNsapwWFScO969WWtYvIEGmip/KCqjIEGSh1RfCadd3w6vqW1kdOTk6S5sAo5qMRZTVO1EVtOJgdMBlP2DYNt7c159cNrveEvqGua+pOcLPwbL2kmGuymSGfiH3QhHeWIJPbvYoCbERokZw1Zbqvsixjs12ztBbTt7SmRMkM1+c4m9HWNS60hOjJ9M5pMWkSiOmad+stzbYnIBgZQ54VHBRTVl3PrYg46QmZRlWOaB1G7CKUU7lsGsAm+ND5JLSRIkEfMSTxngesh86B1gEfe0LsiaFHkBHFHRZ8h4Hfw7pj8tX3O2RbCMS9VO64b7vvdgfhnhIu3qP1Jbpw4m1LpUEqsmQnynLb0IWIzDJ0loK1R1XBX/u1v8p2vWG1XPH9H36KyUbsMfr9XzIMY+8PYgf7bLhrWHaMMiHEkBP8r2gmZogR68A5yeZ2S6ZzovBgRBLViBTJ5H2HALRKD9ntZkNnOzbbLU0zwpiCcZUzKgWX17cE0VKODJkRXF6+4fd/8Lu0tmE6HTGdTri9veL86hZ7LIk3WzqfcTSfMjs8od6uMXmJ0olrG2Pk+Zcv+MEPfoC1PeOq4vjoiL7rcCEwOzjAZAXbukUJhdFpEGKt53qx5Hd+//f52td/wLP332dUFQM0IgkiTca984RoE3NGOqJPHs4ipiGptSFZq0ZBV2949fJzfvDD3+PTT3+fw6MzDmYP9lSk3SRb+B6xXBDWK2zdEoZQCO8szXZNvbyl264Qwe3zFL0fUmtsj5SSPMuYHR6zXa1wXUdfr4h4XKwJfcQ6j/PJwrWazJkenqB0xs3bN3jbkmlFWeToIsdk2f6am2mBli1CREymMXni5qdAaInQqXibMkfpNGqSUqZAhiyR/JQYYfSYTI3I1RiTC1RZI1WLkhGjBLqIVMWUrMiIMtDHHicdphBU2QipFN5ZZAw463GM9g+KtZ6b2yX9eovbNvQqeev4KLhYLhHllBgiFkfvYSLHFFlOZaoh41RhvWA6mzGeTFiub1ivtzRNTzlSCJkxmsw5OXtAkWXcrjcpo7MLaGWRIWIHi9SrxRafRRQ5UUW8dEmtAkgXiLGjb7b00jGqEsvK95JN37PeOMbVmPnIQGho/RYTAoXU2D4pIdvO0bseqTxlmSO83ePNIaThtY3pNdoYyizjoKwYz4643W45GY/TfQc0rmW73pApjRxV+2sehcTbgFTpPYkRoyTOg1QCEeLemt5Hhh2op+stXd/j3GDRsB/o3Su8MbFc4n5/mIbfDHF9d77n7xbwPetjOJwL9L3DOY/3kRSwksJAdtHHUaR7w4cUXi5lkrorIfnoow/4K3/lL1I3WyDy4suLgRCTirL39z3x74Klwz4dmzQMF+nPhUpe9kml9CcwxPyXcbihy1kvV2xv35AJy/xwNuCtKYk+yyJK2oTlZYLeRy4Wa5q+5urmhvF0zGx2wHR2yHx+zWK1QUpJZnLGozFvXr/i7fmXmExRFuB6QdduaDvPunVYXxMZ8eDBezx89JTzN69AGnafd4yR3/u93+M7v/0dMqM4PT6m2dZIZQhRcHx4xGhkWa1qiixHiQl5Yej6hrbr+Pz5c37r2/+M6DoenBwxHo/JihJj0mDNh0gMlmgtMYIkcVEJaZodBml6FiO3F+d88ckn/OiHP2C5uEaZjPn0lPtmkpGIaBrcYkG3WuO6PiUBSYUgYm1P19X0bU30dmCWePo+RWlZ16Nksgo4PD7B9T0xBNY3Gt8Yts7iY5eGgQGCkIl1IhW+a2lXNxAShp/nOSpL4RS7ozoeEcskfDBFRp4bCAkKEkohjUZlhrzIyXOTHgCRYCedKRSKzJ2g/AgRCzQFko6s0MgiIpVDC41UkbGuUEZiY0/ra4K0qDxSigKlJN72CG9ppaXSJmWMAn1nefvlDcpb6D1eS6SMtM5xWzc8fv8h3kZs3WIdxFiQZxOU1Inr61PK+MFshtaam9sVV4slm7pj7MFFiTQF1WSKloLOWSyR1ntUH4aU94LOt9ys12QTwygqXJR0zg0yHjBRgHW4rkaWgpkeYXPJxXrD9W3DctVyPJ8zn44I7RbfW3onUb0fhrmO3qUhojEKZYDegveJKRSSeEkM91WMieGUa8XRZESZa5QxKVVLQgiem5sleaZ5HSbcDs+QUILo2WdLCkgU2CwV8BiT0GYgXRFjwHaeuraUtcVFTxRibyq1QxnuBvhDJ07q5BMAvktnJX3P8ML78vUdhC5IFN/epp2l9xExpMxrnTIKvEzQRwiJ671L21Eqlc75fMYv/9IvUgwD87/3//h/sVjcDnDRzj/9Xv+/+yHSWI3RaERZlsQIq/WWEEBIRRx8in7c8VMp4JvlBYvzL/G2QYuG4CukHKNEwkhHVc7h4YhMn3FyfMzJ8RHLHi5vN2RlhTQZXhX0ouRqC41X6KykKmcUZkrfel69eYFSkkwafO9YLm5YL5fIkCG9pBwVzGYz5oenPH3/AUbmXF9dwj1s+ZMf/pBXr18zH1eo6FktFjiX7rYvX76krlsWi1ukUkyqGU+ePGKz3XBze03fd/zgBz/g8ssveHx2wqNHj3j03vs8fv8ZZTXDu9Rh71bkzCjcrjMXgyugFGyur/jR7/4Orz5/znq1TUOPcCc22jmsxeBw6yXr6wX9ZoMUkSKrKIoCNWCHUkQkAdd3id/dtrR1TdvUOGsJMiKV4vDkhLIaMZ0d8frlZywvXiYYZmhkYtoWUZQ5rm/pNmv6ekWIDpRCBYmMkejuzvPw4RFlXyBERBuNNiYtXsShy87QRpNlhmo0Qko9NE8RlSlMX9CfV2yXFts3VKXGBk8+qdCZRxceFXMiqYja2NP3DVu3xckOmZEom6IAY8B3aYeTG6RPiT6ucyxeLRhXBWVeIqRJw1VhMZXl8YdHWFvz+otbrl8tIPYUY43dbpFKUdcdRhgkgXq75c3FJdvWkpUjVFaw3ra8ubjicDZhVObYGFCZYV1vcFZQFSVK5qy7QOssRmm00cSo2GwCB6P0WeYmR8SMSVHy7MkjtjFw3i7pNh3NumO7aVHhgmm+ZVw4Ap66CXRrTTlOkXTOOfLcMJ+PyJSlHWY4AkUUgxmb0oyrHGstbd+xWi0ZlSVFVVKVOiXSKE30nklZ4VzPzdrsA7f7sEv7GXxIiJQmUhiB0HJPUdUItAZEpO8D9cZSlB2hKImovfMj3HGnd/f+ffdNYDc9fScO+A8JYt6BUCD6YfoZ0zDa6JTq5L1LwvwocdZh+z4FXIS73UCMcHBwwJ/7s9/kgw+eEZD85//53+fq6oZ629L3bWKeKEGWmeFXto+U/LPf/FU++uhDutbyj/7Lf8IPfvg5Iujk1HkPq//nHT+VAh7dFuySTIHMknVoU2+ZjAqm8zkPHz3k9PETCIHxeExRlHTW8vTD97m8vuZ2XdMHeP7yLedvr9isWqrRnI8++Iiz4xNWqy3vv/8BDx6+PxD5PSF4fuZnj6iqI6bTKVVVUVUVZS6oG4tROQaJ3HkYCMG//lf/GkrnvP7yc1588TnC9WipsSjq1rKuO9bbhrZ3IALLekvwHinh7OSYD5494/r1Z/zwB9/lRz/8HcajGQcnD5g/eML85ITj04ccH54ynRxQjCcsl2u2bUeIAaMlIjh+67/4+9y+PUeIDFEesLy84GygMgL7sGUfAn69pGu2RBK8cfz++0zOHrFcXCVr0uU1dnNDsB3RgcfiXJe8PZwnqiQLLsvkvphlOSaTfLq9RakkVxdaICPMZ2Mevv8MrXK23RW4DiEgU5rQ1wQH3t2xZY5Pjul8SYw+xajpDNv3RBHRWYbJc1RmEpWxLMnk3QNSmYybzxXnL9+yuLrFWcdsPmc6m7O61hxOK5ROROHoHL3vaGOCT9rQ4GlAJivQMHR8UmTo3COjQoT0vWWe8ZUnZwnjJOJioLep+8vliKzs+ejrJzx8VnH1usb4I15+8ZLf/6ffJXqJzgvMqOLBwRwlAzI6JlVGORpzfDTndrmmzBSu69j2Pc16y6So9lltMihiNCwWW/q2x1CiQyA0Hdb3sCvgWpGJnGlWMDYZAUfY9PhtD60jQ2AMtP2a4+mMHIFvBCLL8N5hrR30CTnz+RzfbanDGhkT3c27QO88RVGgtaFte9q2xtueTEf0rQYlyKuScjQmNzm99fR9SxOPgQkxQtclCMPEHX0uKaFDCHib5hyZhrIAY1Ln3HaRZmvpxh0m80MRk3+IofEHi/juEAz0vJ0qeSe7v3eIewphbz22G/DokHB01/ukrLQWEYcs284RXUQrw2g0TtGAuzo+KCsfPXzIf/Dv/y/5c3/uV/nub/8eL1+8ZLOtmU3nHBxOePT4ISfHx8xmU7IsAyH44IOnHM7nXF5e8+TJQ/53//v/I1FkRMDcS2H65x0/lQKupSDTCffMy5IHDx/y8Ve+wtOnT3j85AGnJ8dkeYEUUBRJ4CKUJHjP+fkVi+WazbZhs9kylpHVouLJoweMRmO6xnJ7u+Gjj96nHE/RpsD5QNP2NH1MNEMC1vX0ztKulsQYab3F4dH3Fr333nufv/7X/zqvXz3n+aff55Pf+RZVVdF6SWOXKWmeiPWOGD0XV1cIIloKbFsj7ZpcdkjfogV09Zarm1vsJ5/jjGQ+P+L09BFPHj3l2YcfMZ7NcdanIZVt2S4u+OHvfovj+TE2CDZthwsOZ2/fuYGlkIS6oVkuB4giCXGW257V1uJ7y+rqDd3qEuE7oEfJksOH73P2/kfMTx4MKrEEssaYqFUuOlSWU4wmjCdTfNukSCwRePz4MccPH9NuGpTK0HlFlDLZ/WYKrEthwsOR5Rn4HkHK+RSmYt1ElFGYLBsi7jRCGcrMkKkkuOhjQDnP9atb6m2zpxeeX16w9ZGH4yMCWcKJo0UKhxQ9XezxMaAUCLKkNIwNaeFLvtGFHBNCRDo5qOkciDUiG3H64JRt27C6rWk7h28b6npBNj4gjwrWYC3oWYkaFYhokCYnaoXMFcjAuMwp84zReALR4dot83HJrCxotjWbxS1nR8ccHxyR4RBktF1Eo9FEdAwUSCY6I5o7fcLRKKNwPSWBm8sr8gcHnB0ec7voWYYekRkePz4g1x2Z1AjvCc7StTV9lzzU81yjzZRqPKJ26bpopXEkHxtnk8jMuUjXWZyzjKqMo6Mptm4xRY7INZ6e9bZFYhIc44f8VwRCGFrbY2NiPkkBQQiCAmvTzEftGRupgLsINoiUPRwTfz9F5SW2VoB9xy2QyV3wPiuQtAjFnS+9ENwv72n46e+xVAQSld5DiQTluJiYUVEQXcQ5x3Q84/T4lNPjU6aTCcbcZWEi7oQ5R4dz/tv/zf8Gf+nP/4UUr+ZDaohygzZ6n0R/5xQa6boGgeNgPiHLNMmxQ+xDqv/YWvpjX/Ev4UgrlyTPCz7+ysf8/Ne/ztNn73N6epL4u0NQrRCJPSKVHDilyUReCMWkGuNmMx4dTRHRUxQ5y1XN6/oSaz29C0yzAp2V4AMqCKRraZs1QiYc0A2sir7vk4ez69/x78iynMPDY3KjMAQWF6/ZbjdomWxK88KkFds5euuI3mIyTWYUUji6dpsECTE5byfOvqFHUG87mm3N8mbB5es3PH/xOYenD8izDC1AeoutFzjhaKOlsx5va4x2TCbv5hKLGBHWMT84oVUZYfBkWC+u+OJ73yVTAr9dEF2ThpdCUU1nfPDVn+fk/Q+ppge4EBI7aLjBdvxV6zzC5JTTKa5vkErQe3j8wceMpgdEBzrPUCYjiIBwlrIakbUOJQ27HlxpjRLJWrfIDaoosXSIwXDK6CTfFtpQ5BojBY6A85YoIqb0nD56gJYVdb3l937nt+D8Jd3RCNtJcg8iS9YFQfhkos8OFMjJjcBr6F1INDIhUgCxc3cWsSLQq4ZQZWSnBbETiEKQFRpV9EQhkXJENRoxnmgWFx1d22BGFdFrBBqtBCcHc25ubvj8+UukNhwfeZz3PH/+gvlkjJ3P6V3Pqt6y6HqCUhxUBVIomtZzs7yhKOBwXjEuDCOj0TqjHj7L45Eh7zWVlhid+N8GiXCR2DnI0gLlrGS1adCth34IILEKax0mSzdQ8AHnPEqawWwpYcFx55/tkxWslIrJdMJ0WtF4x+xwRswNtbMsuw1FVSRGUKOGAh5xIe14vN8xpFPVlIngQfACFyTWqxQ8LWTyOpKSKBQhhn2O545qJ6IYAsVTJ70PeuDuZTHe8caJ95w692dw9/pkqTD4lYQBHw9x8MdXOAJFXvLBsw949PAR08kUo807NMH02wQHSQHjUUFZFPcyfpMdR/rL7+8GdoKfgBKKtm4S7Bh3oeX3gaA/+vjppNILQVYUPHj0iK9/4xf42i98jdOzU8bjEXlmEMET/G4iHobpdMB5lyTbWlNogxlXGD0jM4pt3dB1fqAAKbZ1y6jrETbivKft0sPWNVuklggRid7RbrdcXV1TVkl5yDsXV5FlOWoy4/j4lMPTh2y/fIGIkizPqEIyW4rB07QB72EyGVGVOcE2SDH4MDAMw0VSwxVqRHQC5y3tesnltubq5oLqzStGZZmEOyLguhXrfku7gmXrCf2KXFvG4z/cdRRZyfGHX6XbbNgsrtKv5YLbty8oM0mmkrQ5xpiwzdmcsyfvc3ByRtSGTZ2Gr1YM9ppAb3u2bUPtHLIcMT48Jh+NcFFy8t4zsqJEZ1u0yZAmQwYHMlCO54xUixAZ18M5ZlmGCBlKRcoiwxQ5NpYJM1cKoxSZlkhjEnVQCkT0BJWcP6eHOfPiK2R6ymJxzcvnnxKXl9imx3c5BJlglKDwAmQcbEejAGHIpEQZQWcDbmgGQgBvh9YQUEpQTjWuBJ8FlFSUsSBTikiHEBLXJxMrJVqksEgpGE2ndE3ENpZoHcfTGecXF7x6c05eliid7dPJZQw0bUPnksPg5WLJqus5m03QQmCtpXc9Z8cTZtOSKteUWmEysy/gByODURIlw3BuyeHSdz2xd4QgaJc1BkcWPLnKyEcZqz5ZFienyIjE47qWrutAa2xr6azDegdSJhtU2xODw+QZs1nqECkyZtMxclSQW4ttHFVuUFogu3v48p6zfYcx36XsgAgSFxR1m+jDQkTQgoAEmX6uZNDm94+l2BVxkd5r1/uyUzwyDCr3ffe7RlipT77rfnIlqbRMjcpAYxd+gLSIyBCYz6d8+OxDjo5O0TrDWkfX9iitBsprevcQHIJUsJWUaLUrxHefxx31kb1DaKbTLKFpO5z37MfHf0JeKH/ih84MJ2en/Mo3f5U/86u/wuHxAfN5khoLwNsWbxm2boOUOwq8C9jOImJMbAejKQtN0/asNw3OB0xmKMqCetNwvbgdpsuD8EVEtAalAkoJOtezWrzli08/4aOPP2JUaKq9+CR1IEZKgk1uhuP5Mfntkr7rqaoRShmstWRG0bYKa1sePDhhMh5xc3VOtM3QDQBSIrRGZRllPiGPOSH0BJ/8z2vbcfn2FbdKUmpBJhzr7Q3L9QZkusBaR8qyp8zd3W0RI1pIjk8e8XMffozte95+8QlvfvR9tO/p2jWEHqlNEj7EiFQ51WxOXhRJgBFTSo91jm3bYqRCqiSNX6yWXCxvmGQF49EEozQuwPzkIUPTMrBIMmLbIPOS2fEjRvIG19h9Aa/KkiAjSkaKzGCynKine+6ukgJjBEqbxI+XoAYwQQvD6VnFeyc/R98pIoKvfPVnWX4ecT4SPckLRMWk6iRiRNzfN1EojBDkskRrcD7Jnp33WCGxw9a8yDMOn5xy0TvWi2XCyYNGSUNuMvousLru6Kxlc1szygsODg7xtuDmcp1CGOqaaTXiRz/6ESG4wQ0veYZoo3DBUXctvQ94pdg6R71Y0NVbNOnzOTyacnwyoygVxojhnr3r3A6nyXKh6y3KdRDHrJdrfG/JAGygfbOkHCnOHhzw+PSU0oz44tO3rG+vUTqQ6YCmx7crum4LqmBta5q+xfmeojBkeUZTL0H05GXBeFYRwg4SyCgmEyol8a0j1skzX94TzBijcNGDT145qZHxeJd+HikFzsFq7Wmdx0gYFYrgBUIkHnQkea+nd0yy+Sh20WhJmbkr3u904sPvdxzv4WEZKIR3rzscZbx3WFA3lrbT9HaIBXQ9kYAMkkcnx7z/3vuMxlOazrGpN1RlTjUqU7SaSnrQrmsxSlMUyYN+v2zc0byTANhHvAtk2dA4CINWOUJobCQx2oPfWzz8sbX0x77iX8LxM1/9Cvl4zle++vOMqjGZMmgpyXTqnvvg6ZqGruvoe4u1FtunX0bC8cGI6TRHKcWrNze8eH1F21m2TcNqvcU6y3Jzi40BrTVayxSQoCVJLxLxvsd1G3zfcHpyxNnhlOOjGcEF1quatP3qUUGSCZAh0tY1QkSmswnVaEJbJz9w7xsIjoNpweOTisPDOQ8OMy4vXrG4usTGQIgCZ1sulkvgLWU+oixyijwjLzJyMibWY21L125Y1SuuFjdc3a7QOmc+LTmaG8o80vVLtEneCjIKSp3z4XvPePTkPbq6pr56w7USaeCUabq2JwaPi4m+mJUluqxYLJc0DmRWYMoSYwyb1YrWezyBxlnebm9pVKAsR5jpIbNilIy/tKZtGi4vLri+vqbvE6d9fvqI6ewEdbPCNuv9Nc9yg8gqpBwEOiZnXozxsU8aABlBJoYFMk+cbpWEHyYYTp88ZHVT8+bNBZdXbzBVQTfJMUWgmhaMpyWm9MQ4LEjR4306JylNeo6Exug04JMC2q5DSo0bdhxZpnh4PIdFx+/+4AXBZ0yrAw7HB7TLwCevv+D6umF2MCXPDa1viVEynx+CyzEhQ/Udo3LCdDzjm9/8VfJqRJ7neO8ZjapkOGYyNotb6q5HFSVVXjCtSsa5YTIqOD2ZMZkU5HlGWeYYHZPvyfC0Hh6Oycee7aZjtaqpwgEneUl+dkw7m+NdChM5PtI8enzI/GBOCIqbNzc4V9Nbh8CgYo9vN2zrFX3oWG1rrO2QOlAajTSarqsxGczmJfODCToKZAj0vceut3TR025XyfBbgHcFUCSoQHj6LuB9KrxKRZRKj5ZzIKQnEOh8oFkLKgOZlEiRUxRj8lGO2NymDn4PU8DOaoL7xXvAle97bMMdA+WdQae4K4zf+LmH/MoHE+rOstl6trWlbhuarqVtO2wf+fiDMw6nBd73XFytuDx/w6SqODo52g8kI5HNZk2ZZ0wmnrzIBzGOSM6aWhBFGqTX24blcs10MmU0GqF1QiSOT8+IytC1LuXL+n9FaYQCSd85Xr++ZDadkGWS7VqAj1RFRbSCzaqmaYZtfWfx3pEbxWyavIK7tubiesOnX1xwu1qz2Wyo6xY7UNOKUYGMDhECIkhkkIgg0wOkDG1T02y2SCF4/PiMjz56ymxcsVgshwIuIBiECAgsWbZlOr2l7W55e7llvZE4p1NhxrHtNljn+fL1mropOT4qePaR4sGTaSoqQ5fX9Z7oNCEIYuggtqlIKU2mdfKh6nLW2wmjueK4OSAEmE0Kzk4rHj2pKMeetpF7Wp+KktlsihaRt29ecfPqJfXNFa7vaJoGSUDGhBfmWjCdVown4+T2pzUmywasWGAyw/XthmVbs/WWnoCuJjQI6hCpBFRVGqhppaiqnE1u2IRAJFBMZmlwFHYOD+kIyiFkwvskyRNeGQ0k7m2UEhsDRjP4v2u0TpxcGSTnr17z7W9/n09+8AmX55cYVRJd5M/+G9/g9MmM+WGB9y1aGXyQhODobYvwDq1yVEw86zBAWiAQXkJ0d9tzrTHzKf76DQfViHYbiU3Hqr3l+s2KWX6GtCNyJhxPxigR+NZv/C6T6SOMMJTFCAfcdh1b6zmYnVCWqWvdrG8ZVQdslkt654g2Mi0myEzgrMW3ns5FVJRsS0upDWvVo5GMK43Sd4yE+fSQqSxpp47ZtKGcjnn24ISutdR1R9tailGFGYPJko1DEIKvfvSMH31+yfnVDQFB5zyL2xU3V9e0fcQ5IMpUkLxktdlgg+DB4Qmnp2cYpanXG84OjhjPJnQEmrpPtr1FKkJmbaBJH2mZC3wl6IZwht3Tv+tIkw1yTF4gMSlNWy9YbCPnC8dhlg9QcPL52XG+07ZP3OvASSHn4p7X966w31M/pm8R3FfyvP/+KY/mD5OaMyS41fkk7uk6R3CCanIG7obr85q6d2y3W7brW7x3+N5jdDK9a7qa0SRjW28pyoIizzEm4eUmM0gpWa/XXF5esVjccnb6AOcc1ajC9o6+68F5hA+JffOvKo1wtVpzu7nm7cU1RaZQ4glhPMSaOZGMbjpLVzfYriNEP2B+kcViwfl5y3rTcnNbc7WsaZsmDSWtG0zcBRqBig4VBSqmAs6QZRdjsvVs2hYfYTwacXgwoSqKxHTYHSFZfFq3xcdr5tM1VaGQuuDFl5bblaPpQkpPCWmA5YkIZZkcZBwd54QwGqS7gz9wSHFQibV0l4WHD+A7nBNkW4/H0vUtQmukNORZQKgek5fkhaZrB1zdJxwT7+m3G15/8Qm3F1/iug0xBLreUu7S4CVoJcm1JPiEtWYxBUT43uKdSyZHXcu2a2gGDC6IlK5S257GW8blCGKy2T04PUUbQ1FUvHnxGfl4SlaMifMDhLu3BZSwy501MmBMQGmb3AylIQqNwg/h0IP6dpj7BCU4eS9j/ClI6Wnrlq1rmB2ccPxgxvygoqqyFI4tNDFIfHTp73TdXszhhR1SgpJCTog0F9g9Jq21fH55xfd/9IKTyUN0Fem2lvXilm7bY1RBc9vwYr3i8rXg0YMjKgw0PavVmk3doCTcNlvWdcsBCtsF1qs1by+uOD4+pt52iODoth0iwHw8QUXBaPAS8d5zu6hZ3Vzz8HSG9HOkL8lyAYMdyrZuKDKH1obpzKALjc41Mbo04J/kZEVJPs4RKgzQhSBTkYP5jMvrW3rn6T1okxF9ZDYuKMoyRak1PRLYNh0qKzk6fcDR8QmQjKhabxkrSVkU6Dwjl5JuU6Myg6x37AzBqDQgHEpB18cBuko/gx8EPgluTgwoHwR16zi/2mCjwCLQeYJdBuj4HkBzx1xJs8o7NgvD9b4bFO3MopIH/H1WoTKavCyGqLWkTI4k2DVZPqdGo2mvua0jmy4ShWI+n6GVoG+2WNEMQqVIu+1pG0vXWNq8x5iMGANt3xBioG1bNpsNbddyYDtW62VCDG5vWS+uGClPZlL3beS/oh14CJHNZsvm/IL5dMJsUrBZrRmVK8bVBKUMV9c32K5FikQFcwEWiw2v37zh9uaWbd1hXUQXBUomDwYRk/2qEomqmIRBqetOFRMQ0LeWzXrFZr0mosm0IjMKrf6AgYwA6xtW63OWy1d4v0LgMVqi9Y6SJHBeEmMyhLfOYx2pA+w7EAGQSJIDWXLi24l/07YqBhBBEJ1ITBUBLqbC1vUKPeSAlmWS3Qt5d9mc7WldT7NZkwfL5Zsvub1dYK1NYczOIXK9L1I7DwkfYgqWqGtsFDgfqJuadVPT2J5+8HjeFVEXI713tLYnhJIQPFFE8mqEHJLaL87P0SZDlSP8/BDsXQGXUmC0RomAUjtJMmnarxRRqsQGiBYtBWbwQ3GkkNrxPEMgiU4SncD2HXlVUo1yikIncybSgEqENOgKYngYXSAKj4s+JaUgCaS0Ja3vwmN767m5XHF53TPKFGWmkbkgqg6TK7SMyTtmu6FdOZR1xC6glAVXY7stViR1n9Ga0Pc0oWWzXlBvVmwyTfDJCtjFQO8sxvaMqzFTnRF8oOk7Nk3NZnXFdJTTNY7epMHjroDXdUdhe4xJN4vI0yDd4gkqoozAhg5pBTpqhBjCBEJgPBqhdFITRyQmrxhVYx48PGA2H7O4WXN5kVSVQQrikLY0nc0oNIQu3RvbtqFQAq0URVHQNS15VaJWen+fVYUaFkmBNgHrUuGOQSRvcBfBRUIUe5aZdYHVpseFDTrPePSkQpCUw+n6kuCZKO/xud+FS3ZfuRtaxuHRv/Oa3x0esARkTPUiaXzEoHsYdmsyBax0Xct6bUFlPPvgKeOqom8a+rbFW5u44QM/vFG7sHLDerPmsy++YNMkGmxR5BweHeBcT9vGBG92DbPS8AsfPx5YcZGz4zE/7vipFPCyTPjeq1evePXqS05ODoneYZRmVI4pyhHrzZLCKMZVhtGCut3yg08+43s/+ITbxS3eWvKi4PGTJ5wcHw8raKL0aCX2BTytycmMOwzUwbrecH11ye1yy3g8w2gBA/+VewMPIS3r9VveXv6Ii4vPaLcLmsaxWGc0jcK5XUcdk/zYpaFr3yvq2uP9lpROb1BSoYQc4s+GRBopkTJl6anhl9QgysSYGE0MTZucy4xOdptSpV3GDp5wtse2CxbXlxTxgNvFglXdEr1IBTTGxJ8NASJ4KehdwKNo65aGW1TbYmOkaWrqtqH1dh/Flh6YNPH3IcVGWdsTnMN5h9YZuhhRTQOoAWvWGWJ6AHdCzGHGIVEqDZHjYK2JTG51QkH0YVBSQj5ght6nIamQktWipV47gk1+HkZrlEqUNKUVGQobYsqrDaAxaaEJDSE6fIyEYTHdFQIt1R4v9S6yrh1Cz1jWAZlpdCkpph4ZUz5o13dpxxIEl+cLSiHIq4Isc2jV0/cOFSPTqqTe3NB1HU1dI2LHZnXJaDQBZYhdGmi67ZKxSn4eSiV6qmscy80mMRJCJLiI7e4WQ2cdje1oRLJAmGgwlNiQrpd3yUsk6y1VOUoLsLN0Ng7CuAKT5Wk3V46ZT4949v5DDg4LrsY5Wgmc0/RRUXctQhmKsuJwkhOdT776mzWdsykKUST4Kx9VKJNYI0IIqkpjjCIvItb7xCRzQNBYewdTbJvEOvNRpI1o7wmh4/x8xfHphDLXxLgr4jta4SDPiXJI6eEOA2foyIf7d8dJSSHH7xZwGx2tT82YDhJ5T6WZqIqBGHpMMUEoaPuWzrbM5zMmZcZls2C7uaBeLxNdUJl9lyRVssd49fo1/+//4h/z9nJBVhQ8ee8J3/zmr1A/fkCRF+AcOjg+eHTEv/7nv07bbnEhEPP7CZZ/9PFTKeDz2QyhS/rekxcFby/OuTy/xPWOIi8ZjyeUZcbp4QFdq2m7mi+ev+Db3/kOfZ+4zLZLKS4iBs6OJiiSZFfJlHG3o/ek5GqPdx11U3N1fcPNzYLFYoUQkvlsynxWoYYU6HdW57jkk89+g8+/+C2ub17StT0XF2uWa896o+isIsUzWXT0qLxhVGXMDzKKQqJMwJgMrQxa6WRSs0uHV8k5TovEj5YigYIxxCHRPSW7R7XrJXxaqb0jDpa3AF3fs14seP3ll3zw+CGmnGD1ltbVZLGjLAoiEmd7pIgoLehdT0DSWMvm5oY2RoISuJQim7jAw51uJAjvUUJRSUmOIFiH3db0MTJCJcVYCEiZQhECHjEeIUOETQeAljEtrFqhdKAUHikcfQh4NyyAviOqglEMZCoDZQgDN3xxs+LFFy+5urih6xydDVy+uiCqkOxDRSTTkoyEg0obEFEjTI6UDoXDBYUkzR9EUCjtSX1/uupSaMb5IWe/ULC8XRGixQWBzkpinjxUblZrUJJqOiY3htXla/rFOVJnjEaGUWHw7QbX1/zo8y8IMTKZTJhNR6xXK/L8kNYnBsbRaIxB8OrFC14rQTkaU1YjQnQEEWj6NqWlK4lzfk9+q6ox2jrqeptmOeeRR0+fYlSO7fvEhsgyqvGM8WRKXddsmhaQnD08Zfr5a+YHc44OpqgYWQY4PT6gqBx5Nmc6HnGzsLw6X7G47ejanq7tqJUHKajGI2zwbOuazWrNtBql7NF41/9IAfNJhs5Kgki+3MnxT5OZCkQYrChqzs83vD1v2DSJAQIR62G96lndOqoHOVFGgutIM5NUIGMIw/vKAQcHkOlZGs5jP4cJw/9F0sB8OKL0BCPQQhEJCBILJvgkZddaDOfkkpgwRG5vblkvV0zyOd6t2W7ecHX5CkkgG3JYd46Cxhi6zTmvX/2Qz17cYH3kzdtXzOcF00nGw7MnzKoxhYqcHVX8ws88IoQOT+TtJvJy8cfX0p9KAVcKJuOCp08fgpAczEYcz8bJJGaIHStyQ99b3py/5fXr13z++WfU2xXzWYkUApdLmqbj+uocZz8ky5JRkZSpSAkhIVq22yXL5YrrmyWL2zXVqOTs7IinTx8zHo84PT3h0aMzhFQprWVP3Ym8evsdXl/8NqvNOZtNx9uLDevaEgGdB4TuiXiU7JmUiqfvTXj0eMTBQY5SAR8cKY3HDYvKrmMXoDQhSmxMogUVU3GXKm3xZRTJWyQ6IkMn9kdAYlEpnNYsNluyasx0VDHJJNIGNBJNSdO29F1HZkQyigodt8tz6mJGHRUWkFInsdEgAlJSUChFFTzaevJcM8pyqrwAFG9Wt4iXLzk6PWN6ckZWjXj09Clog5ICNarIhYTN5XCmgRiSh3oUEV1IDBYJw8KhmGZpqCXxdCEOnbmC6Ll8vmR5uWa73qRgYgTRe6qRQZm0izEyeaO7KAlDZyYUKF8N3XiGDXFY+GNS/A3WvQCbdc1vf/d7/Npf+wa/8stf5cXzS96+vKZetPitQ2WabetQmSELCmcDLy+uefT4Ad12Sd925ELRn51wdXPJxfU1QqRYNO88b1+9JgZBCyhleHJ6ynE14cnjB9gq49XFOS9fvKTb1lSVphhlOCyreknXeY6G86xmY8ZA0eTgZqy9ZT6f4K3DkozGpNFEGTEmzY6IHm0kWCiKMdNMUsaGpunQRhClIM9LMpNjpKfIAl8+vyBsOmJjid4jpEbo5E2ihGQyGqOlJHSWzfUNJmhcmwNJDDY7PEQZkxKFXIqzw2uKKmc8KSDC6QPHg4c17y9aPvn8gvPLLW2X2ETOeW5uWk7OHqCMIOBwtkczcMTjjvN9jydOBHHHSBke5Xcas3DvQbK9w3YBnZcJ2vI2OYG6NHjNjSZKj1R1ypU9OAAr2Vwv2WjIZM/psWZSzcDHfWziXf4mjEcP+O/+jb/Eb37nOcu14/TshF/+pY8pRMf1izfY+SHTeUGmWixdaqBiSpB698z/8PETFXAhxBz4PwFfTx8H/zPgB8B/CjwDvgD+nRjjj1kvhs8zQnI4syidY3vLeFQwHVdMRhXBRy4uLzm/vOHq8oLLizesVzdUuabKTcKIB6x6u22wtkeJEj10uMGl6LS351es1jUxQFGUPHv6hCdPzjg5PmQ0GlGUReKeC4nte4gC5wY5OfD67e+wrc9ZrW5ZLnusTUb4IXqiHuAaoxiPCh49qDg7KxmVEqm6ZD0pQCm5Dy7ekfdRKg0yB3TOxUgY8LydzWva8wXwcc8lD3EXgnrXYZg8Z3J4zPTwmKYPHBwdoRXUmzU3l69Y3lzj0v6cKATWQeYFXb2l0SOCSdBOmvSnrWeUEhMjZe9QbYPvWyJjZFURtGLZdlxtNuTbFfHFhthsmD39iAdPnrLZpER69gq1dHgfCFGhAomq2UKuJaVJxl0BjZaSXGlMVmBFeoBj+rg4f3NJvWqwfY8PgaLIGY3K9LAOH1iIYkj3SbsaoQIqeEJILo8yGV0gicRoUCKnaTd396UHu478/ndecv665vZ2y+q2xjUe5SVaJfvjyXhCNlb44DmbP+b48IyrxTUhKMqywGvN4YMzngTQSjMpK4QPVMrw/tNnfP/5F7y+POfq+pLFwREPHz7gcD5DhEC/3fJl31CMC6SRmEIzKitGey9CWG3XmNxjspzW1gipubq+xjYd0XqMVGkxrhXr1Yq6run7nuOTE7RSaAmZFmmYbRQPHp1y9vABItTDQq/JZoniKqIfxDyBoizpraeua8bjMUpKbNezXi65XS4RusBaBSh8jFxtOnyscTEQvB98iRTbxrLc5sTBF4UA4wPJE1+BjNzcdGy3Du8di5str77ccvogIy/GOLfAhYAQMiHb+5nkgHgP5I09JzxG5P25pbhTZu6+y/oAXUduTEqbCgl21FolLrpwBNcjRUFVFhwcClCwabf0/QbnO6JMqlEhJVExaBDSKelC85WfeUznNNc3jmp0wGx8QqUdi26DlFPyPKcsDokqAzrwO3bUn0ABB/5j4P8ZY/wfCCEyoAL+18DfjzH+HSHE3wL+FvAf/iRvprTGeWi7nlKCtYGmTsEFXVtjrefq8pr1cslqecN6dYvtWqajGaNRgRCCtu3obc8OXgihp95aNtua1WrD7XJNiIJRVTGbTTk8mnFycsjZ6Qnj0QhtzOADLZIDYEimRc4PfKcYOT//nOurCzabeuDWagIOIZOQqCgM43HBwTzj6EhTlQIlBix9EBtIKYebTAyFVyAG/+kYUgFPA5YwbAMHpdlw0+0GpZF7FKn7F9AYSp0SdF69vWR8eMxkNme7XtHalsUyccaJGZKI9YGud4S6JpQ9aJOoGPdMvBhc4oJ3ib3hHC4EmhBonWNle9oQsFqRtS366hKlMybvfYDJEt1tZ1WwOxITR6FEmk0QIpEiBUbIxP3ukBgCWuRpSMUggBGKrvF0vcN7UNqQZyVt2+N7SwyDahfAD1L64R8pkg+7UBElejQRG6EXEiUUmcmQIl1zJRSFHvHm5Zbz85ama3HWolGcHiQ7YRwUGGa6QivFUTVlsVzi2oASOVLlvL68QUlN7x19b3FdT+gtpwdHmCzDaAUi0riWkAtW2zV919G0HUYMKt+qRMik6ivKAqXLvRKzsZa171BB0Gxr2hjYtB190yEjjKsReZmTZzld3yVTJpESr5a1G8RUmiw3tL1nPE4mY7Z3eBtJ5lFplpRlCuc6tpsN223SXiBIQ3Ih8c7jUhoETdsO1zwnxshyWydP+qFdTjYO6fXW22G3Gwc6qyLL4OgoQymB1j2rVU/TNJy/uUTrGYfHBm0Kur7eK9J3DKPhoWV4xN4p4Ltj11jcC6VPw3QlESq5I7oh6Sel/iVcRkjwIT3XUhlMobHB0jmf4MjgQUJUKRUoxDvxUOKeRcaTjCePD8hzj3cVfSMYzxTKdNiwputHVOUYnY1wtkvOoffO8593/NgCLoSYAn8F+J+mzyP2QC+E+LeAXxte9p+QotZ+ogKeF0ViXSAJwdLUPW0N64GZ0PeOeltj+5Z6u6Gut4AnyxQmL9NDLyw7mXrXNtyEns1mw3K5ZbPpEErz8NEZ7z95yNnpIfODKaNxybiaIJUmDJzP4HzKq3QusTPcnXx1s7miazqUFEymGq0U1nmMVuSFYTSumE/HzGclWe4gpGISoieKMDBgdjuOOxXYDneNhIEvPQxnBImvHbmXKiL3N2FakHe9fDqSLaVhXdd879NP+cbPfMBsPMaMVmQ3F4jLc1TrUbsUlBDpfES1LaJvIS+ImLsYquEIQC9AG42zmk4mpWZoW1pn8QScMaycQzQN8tWXqMkMM5nivccN4qv9ecpEh9RKkkkHMaBUmfBykSNkjhMCH1tUHIouIhVZMcZQUZQj8qJBCklVjinKHCV1WiAHsVSqFR1CKHaFKMaIigEjPSqEFH5LCgMwWg3CjhQyXRUTrt5esmmXWN+hRGRcFOhjzbLtkUgyZah0xigvCLnkixdfIDPNdD6jHI948/Y1ZVlibcC1HZs+UWLn80Nu6y2eZCdQqpwnj864vVjw6s0lvQtYCUVeMK4qMq2TNYTUaHXXgbsoafpAbC3bTUsXLMpobNcjhQQUKs+ojME6izEpw3U8niCurjAqkmXJwiCEiMkkTVPj2g4RBRJJ09YgIkVp8L5ntV5SLhTT6YwYInXTpOiwmEQ0eVVw3+UvRmj7DqFTkyTZEQoStBiC3/MChRC4Pn3vZKzQWqB1ghu3m57bxQ0mi0g14fgsI1Lv+fwy7mnl6b6NMQ049wX83iMjUkt8n62ilMKY9GxHvyveYs87l0PYQrJlCCACOhP40BFFlt5TAkGm7hv2JINdA+JjQBOZTzO8jaw3mq62hIlC6Z6mvYFlmiWpfELvrtP7/gmZWX0IXAL/ZyHELwLfAv494CzG+CZdrPhGCHH6R32zEOLXgV8HmM1mAIyqgmlWcLucY5sVl5cXhODQWg8hwJEYHcSOtt3StR15kTEaV3ResV3X9G2fOloiL1+/4vZ2Q9c6sqzgyZNH/OIv/TzPnp4xnYzI8wJjMpAqbb0CiZWw56JGOuvStP9eAR+NDI8fPx4uskCrgOvbxHjIFNmQJG8QgwGQxHlBDAqkH/L/4nBjBSAMXNN4z5UsJIx7eJ0Uco+Bx5ic2ELcFfr7HywQ0+ud93z2xXOur2959vgh84MT1FjA9AB9dEy4aBC2T3S9zODyAkFA2pawS8VOrUx6aymJUmC1oo6RoBS91rjeEqwdPkNHjJJ1OcbKGrddop5/xsnXfokYIq6374QvF1mJUQqjJYXOCRGUztO1lgapDEpAbzs616FlkQa/oqSMjzgu3uPnfuZrZOoz1qsNR8cn/Oo3f5HT0ydp3uAjCjHgh/1QG4YCHtJOQhmF80kwYoPHu4Zc6f3PLZWkKIrBRjaFN0slkAH6tmfb1KSJRKC3PYpA3bW02yXPjj/iw2cfMz44YLNsiSJwdvoI1/VslkvOW8dV03D59g1tvaYqcz589IC/+I1v8N3f/SHPX5xzuVwjM8Px8Zj5aMy0GqEwODv4YpTDh6kqlJQE12J9TWc9pTRImeFDYLlp6GPAq6RELsqS8XjMeDQikzdUhUGpxMqyzqJ0pO9avPUYmREi3FwviNEzGmVEPE1Ts15niSLXNHuKniIZalX5hOnogNeXGazSH2p1b1c3dLX3IQ+phuDhYZElgpSB8UhSFjl5pnjxfMlqbbm8uCVER1kdkuWa3vfIKIkIduvGjjW1Z0+xY5LsCrwA8W5ba5SmMCZ12yHZFqQE+RRmroTE02O0onWJ3lTmGSEkXYDOU0Sds5YoUrP17u4A8IG+65BRUJVpQRMizWKC6Fiub7harNlsYD4vaJ1FBZegP/74WLWfpIBr4FeA/0WM8TeEEP8xCS75iY4Y498F/i7Ao0ePEvPHdQQ80m1RSjOeHfPm/C31YkH0FuEtJyeHifrUdTjvmOYVIFgu1vR9PwRaSJabmu7LwJPHj/jgFx/z9OkjHj86ZT6fUJQlWmf7CKQQIs46Opv8R0JITmm7btG78M62//hoQpwrovAI4ZB48NmA3VkEHoUl7ckCLkBAgUwmNTEm6lQcijckkRGkVVrs1GQD7n1nWh+Hrw+FRSaaWRxYKumD3f0ngpSM59OhaAkubm+52W64sQ5x8igFKNebdA5FiZzMiH3qIKLe4cgDO0br/dBHAH0MBKEH4dGQH7jL9hIeRWBrFPV4wqqtEcsFpcnwrif4uwIuSB1KEJqeFFMVvEzJLAJ8sAP+H7G+S4tXNGRixkQ95MnJQx79jROiNOi84nB+gIqe6vSQ2/AZjT/HyY7oA9avcTElB8WQ3CBbBHQ5EZPOJULnVvTO4EMKmNZacXA45vh2QlUoYrTJ2dFHtr1lUdcEH1i0FrPaUmjFy5sXrLuOxjmarqOynl/9xi/hRfJWb7c1V+cXeK9Zby1BW2xfY1vHqzfw4uWXvDk/p2lbiIEy0zw4OuLJwzMyIgKJcyD1XeH5/PkVcyyTTLKpO2ywaJWllJxhCx8iLBZLRqOK6Swpbd+8fsNmueT4cMbpyQwlBY6cosyQMmKKimBhu2lYrjbkRUaWKybTkpOTQx4/eUjbJvOryXSK7S3ttibGyOzggNgz3Mvq3g2aAMI7kY1+R+a+dwoUQyEe8A2tA6dnmiyf8uJ5w+2tZbVoefvlimcfTdCyH2inEMOdZewdwfZup+rv3c9C7vN60v0eQPvU8AYfkdYn91OR6Jg+elCe/2975xIrWZLe9d8XcV75unnvrVvv6h63Z8aDLAHGIDYgbywBns3AAsm7EbI0G0CwYGHkjWcHSLBFAoFkIQsL8ZC9BCEkxMYwA+PxmNYw43Hj7pnuet1X3sw8j4j4WESczKzqqu5C3V236vb5S1WZ9+TrfCdOfPHF9/h/jav54IMTzhfKaDxBncddzJnPPWUeqEwZ9QxKsNElG1LXKw2K+AyjHVUJWeEIoaauDd478iKg4lgsL1jWDmgoMZvuYB+FF1Hg7wHvqervpr//LVGB3xeR28n6vg08eIHvAtKkCo6MjgcPHqJY7tyYA3Pqpubs5JS8sDFV0LU419I2MZXJt56qyGL5dRC+8jNv8ZUvvcWdOzc42N9jOhszmUwoqxFZlsf3qcYu8s7jOhcLXILG7ADvabvYocS58EQPu65tUhVYQESxkhGcEjRGvft0JfGeTluCGFTSJQ2ezrexiEQ9SEBMQNREBZgKEfpoufcem2WIOqJdEbtbd76v2LQEbHLFbG3xoAFvApNrc64dHvJ4vaBdnrJo1jQEKApk/xrM9iLJvbFoXqLlBOfiRk+92zRTlRQI6meE6HaB6bsHbW//NAFF8MZwEpT754+5Oz/A+xbvO0jBN8WhKJ1vaUOkms0ZUdo82hgacCiSZTi3Tr5tsGbMarokWDh+8JDZdI+D2ZRSAqdnp9z8qTdp2zlNfczKHZOLgK/pfGxKELwnMw5FYqcgieRURmwclxCAEjAE9XTdBeJWzEeGLK9AqlhijZJVBVmWk08m+CLnovO8++iUm9ePCFnOw5NT7j96zGK55Cf3H3Dv3pvcu32X8XjGxXJNNhnjjOPoxnW0W/HBw8f8t2/9L5raURYZs+mEo6ND3rp7h8PZhK5Z0zYd9arDrhzT+U0AXKM8Wix4HBpCaMiywHgUC5hC8LFrkwaoHcEZREo0ZHjnyUYzxpXBVrN4L2UNJyfnvPOjD5iO9xCp6JyloeT+40dYExAD00nNatmwf3AQS8PznMzG9NdIlSrR1++jXd7jSU6SZIV/TGCuT/5TbZnNhS98cUZ1v+birGW1WNNclBRVbLW2W51Jum2f+Ftj27Z+xyCqyaET4V2gWcfdlu8cRZale1IQ7fC+xVgll4z5dEaZC2WVsTg9xoQ14iSlLAuooetiEDi3eXSnSHLPGo2kiipk2Ljzb1uCauxBYFqa1SMaJ8xmM6x0L3CdXkCBq+oHIvKuiHxFVb8P/CLwv9O/rwP/ID3+9sf+Wg+JfBfjUclsUsRskbPHW4Xmo29vcX7Ger2OFJtdSwiBLDNYa5hMRuzvT7l545C7t25GHvHRiLIsyYqYTO9cDDxESzsqZ9Vo/ToflbdzsQTfOUeXiOu3sicVtfFfh2SFRqVuJfrE1AU8AcnivkA1luSCYK1s+KdFUj15ykmFTc7/E8FDrwEh9sWMK3n87b6eLJ3d5v+ggU4ineqjehGDRMEn350gWZ4qxGKgVhAC8kTH776zff/1pi9N3kmK2WxDdx8kcSlLnFDrJnYUCgTcTns6pcF5Q8DjQ0uRVViTE0u8LdYKK9cHbgMSPPgWHxyubjg/Pma5WNAsL1ifn7M3P2C2f8Bqdcq6OaXtlmA6itJTZJF2GFWcKplCo0ouDRlt7HxDjqjFpIrMKIti88DdGweRWtgGsHGnt2oCe7OC84s1q7WJbcAkR9ViTEHderr2jIvlgkfnxyzOLyJvdfCUeUHTrKlxTA4n3Lh1hMHRtY73HxxTZRV70yn7e3tcO9xnWpWxriEvwEuqiN0qnbwqqGtDs/aICgfVmCLxhvQFW0LfMSYaOvWq5cGjY3w2RbE8PDnDe0/XtgS3AN/Q1Kf4kNP6Ak/B2dIxLondpxYrHj86wdqMi+XFJs85K3LEJNpn/JNuvqcmfa9cex0uTysojWPQvxdigdd4qhxpRlVCt4xNuIuq96pv58L26W6cqFfeO9+7YwB5H2hqR71ekxnDdDSO1zqV4ts8Bpx9sIyrkqrMKSvI6ZiNpuRFi0nxuFh6Dz6EVIQVkjs4JF70gA/x3KxoigXEjmF4Bz4wzvcppKJzbTImP7kLBeBvA7+ZMlB+BPwNouv+34jIrwB/DPz1F/yuGHXOYwPb+XzG2dk5x8cn1E1L8CF2bQ8dy+WSuon+zK6LpPjjcWwCeng45/bta9y+ccRsNomsbXnq0ZeCkT7EDvB9mW7ftTqoxi7UziXL29G1HRfLJV0XNpdlu+1LHUhVMSYFwIJP32e2LpAUqtmQyQOgye0WOcjjFnFnQQhRUUb3SSKiT5/vO9RHX3+vvJ9Mg+pvUKeBWj3apQVod25IDBjHc4gv+Y1PfeuYDBoi7a3IMybidsGht9L7wicT26z13O3eezrvcSFmJEB0kQSveHUE7bBGQMuYh60FhgyDp8OmxSMQgqPp1ly0Mf+7aWpWbcvqfEHX1Mz25ywvTqn9OZ4aMQrakIujtDYWAaUFS7SL7bw0Fma44CGM01KWLpMRqrFllu/T1jWdb8AoWZ5zdFiRF/Duj+/j2g7v1hirZJLjO0/dxh3H4/MTTpYLysKwXi94+PB9cpvRdWtMFrBmTF5YsixjtDfl/vtnzCYFB/t7HM5n7M9GVJnBqMdkGSE30Q2xo8BtYcnGOY6S4FqK0SgSkvVGArJpSGDERtdg17BcrqklZjStViuC91RVQaae/ekMRGg6wXfCxbql9TDNS8RmdCFwsVxhH58QdE0IgaoaMbY2uW1iSz6edDHHHVw6d+0VqOwq8Z3ba2Od7ypmxWSBySx2DKozQSTWDWxKMD98mz4xBZ4w+nerjYDgY7oi6jFiYtAyuDSHfZqzggRDlY8wWUmWB/I9YVRWOH+aUiSjwt74ttNc6NkRvSb3Z+oglFmDV0FCH18NGHGUeQzcN0kPfhxeSIGr6neAP/eMl37xRT7/NKzNKPKCIivYm065d+8O1loePHjI8fEJi8WSto2KVUikObHrKQf7c64fHXLj+j7XjvZjk9UyvS4Sm/W66ItzIUWEQ19SG7c5LpEGbZR4F5X348fHiOTMZjHYaixoSKW6BARPT2XivKZgR3SLQExLi/6+aD179bGYx5CCNbsdQXRTNNS3Z/K+D2ZGp50qKZqecqPiWCQ/dI+edS1Z/cly6AMpqmB0Mzswu3mwO0p+Q9Kmsdg8MrhEGCMbSs9tpV3sjNKfr1iDxWKtoeti7KLt3MaA8N5FfzIOYxUjLaJrOqfUUhJ8QVCHZlU8RxXatqVdnNF1j3GpyUcIHkegrdd0bc364gxfrjC5RzKoncNYT2ZtDIiLIbiW3HQ0LqMj5pe3bYuVHLNN8om7woOKvMuxC0HWAQ2OSZHz5S+9yZu3DziohPsPz1hexLEdFwVdV9OqxeFYuRrJDdeP9tFVw3J5husCQR2z8R6ocnJ2SlEaTG4pqpIbN69xNJ8yn5TMxpbxKK7zKhYTLHj/hGtPMo0cMKVltVoyStSlRR4b5mZZFn2reewl2dQ1ClRVxWqd2PaaJo3tiFm1x/Vr89hUoA0cn9ecX5xQlAX7h3Nm05yiMLignJ6eMt+vNh2brLHkpSHLC0JmNtkfMX1ua2P3vN1BiAH03l23EWpXcffHortRNdLQjieGKrfRKEM/ZMhsbml98pXtQpHmwVMK3BCYjMsYsGzrSLkg0V9uVDGSA4YiL7B5hYhjNI0Nupulp+saCLElXtxIp5Rgy0aB46KBEOdI7FlA63AqhBBjMhQZxgaMjW6rTysL5VNHluUURUVRVDjXMZtMyN+4y9G1A87Pzzk+OWO1akAj2VBZ5FRVxf7+jFs3o697VJVkeRZTAhXa1hP5THjC4u4Z/za51CFa83F1DHjvOD0554MHD7m4WDMZz5j1FAQamRA3SlF6K9NB8ouLsiVv18RRjEelA6IyiX70aMlL8sgppCyUZJ1oH+zUzb0ckmUfW0hZUJsm8o76ll4pkyz2HQ6ThCDbDj4b3giTApfaGyVRxphPrf2eI+bJpvJknxohRDeHbP0/AKLYzFAWFu861m3NuvNkSYF3PoDtMCZEKlmTJVcUdFojto5EXVicRka3i1PH+rhlcvAWe/M91Ne0Me4LRjg7P8WP1/h8jceBFwIlOTGA2aJglHGuWFtxVuc0XcCHmi40dL7GSZVy1MHkQrFX8ON3jplVM6xO0HqNNi0T43jjzUNG/ox3K8ODM0ftp8z2r3Ph7+NtQ+M8B3aOCcKNwxF+PePspKVbdRweHXHj1nXe/eAD6oePyIqAVbhxa8JbX7zOpLDMJwXzacV4UuKxPDiuuVg2rJqAkazvaUwlgTIDbEbe5hzszTEEmraNi17bMh5XZLllsVhQliWTyZRiJJytT8hyy+hwD2szqrKgAC7O13jnWLUdF01LmcGt60fcu3vAeFzE1y4umM1G3Lt3i/Pzc9qmw3ulXawxRaD1gbbL6OMe8rSFIFs1+iSep6h6f/rW0W0TkVwIfTf65ErcMVj+f2Azi83j/a1BY9l85N+ISlyjMTEaVYnPu0XFMR3vs1ic0rYtwbs0l3xqmh0D2NbaWBncOWyeYRWMycmLEQf7exyf1biQoVqACr7McV1gMqvIyz3W2nK8/miBLkWBGzEUecVksod3kZ3NiqEqCuazKXdv38KHPpBHTPIvC6oyukistdEClphf6f1O26SkjFQ1ETJt4s+IiUqo38d1neP05JTHJyecnS959PCEa4dw4/r1eKLaYvtsEE3ujxDPyZrt9wYfX+9cmw4FFAf4mJa0sUUMfbGQakBTgLVrW7IsMrf5kLZipNXbxCBR9F/3PsJdBR7lCj76veMGeqtYdfd9T00goZ9Xggm962fzgcjMmBZEDIiYWPKvW4NJjdlcbwNURmKzjOCeoDPeK2es1NH5BmMCHS0mEzJTYrQDH7N5MgN7+Yiz3LEWj7qAsdHCm0z3KIoS9YEyy8jyHG9XOHGg/Q4APIHGxVJ5VaETg/oMNEPDCguMswlLVVYuklxBZFw8rRu+/fbb3Nq7xZfufIEbR4eUeE5rR1h58tkt8kVBd3rCyXlHZivme/vIuMOWGVYq2uWSpm6h2GNS5djas+7OOQ6KL8dkYtif5bxxZw/1K0y+JsuvEXzBaqk4v6b2notlh5GM8ajA7PCB5wVI58HDbDZiuVzEGoW2jQFM72lbS+eaTQf6+LmS6/NxzOxyHms8IwuuacFktHWNcy25FQ5vTvnTf+pnqEaW5TI2jihtxXg8Zr1aIyk+0zQd1uT4Rjg9XdLomG2+Y1rjn6EDPt62TO8S2TgNk9cu3X/x6O4i0fen/Hglvv1MEwLLzieSK6GwELqtK92qErqWojGYbE1WlpSTkrPzx/zxO/8XlQXWuEgJ4QwYsLndlNP3C0ORFwhQVbE/5vnijAcP38cWK/LMpEy56DruujwaiS+wGl1OT8x0sWxWUJQjurZNAkciqj69SNCNf9ja2DwBF1nxRALGBbIsDqLCZjsfO6zHAgNr8813K7GZsSZ/3vn5gsXZKUXwZKs1/vExPi82pxmcg03fvXjnSLKQjZHkC9eUEhedDn0Wh6gQNOZ898FHKxYRmxSyT0FWhzGRZjN+L9DHzZM/WTWmLUYjuY+p76hh2foWt67EPiVxxxfY+9o1XsO+4Ue0vqNFrqKbRWFTzWZ6siClrznaLCpsf7NAyUTw3tGFgMPQX82j7ovouKUxC1o5A04ojMdaD5LhyWP7OslwKligMgWhmFMWFW05JrQxXY40qeq6o1u1lGOTIv8eMQVdiOefEYm52gC1DzutuWKej0XJTY3vl9susFjW3L75Jneu3ebw2nWm1YgsBNbNmj/5J/4seTlm9kc/prPf56J+j/OzBeNRS2mUohDKIqPKK9paOD1pme9NuXFzxqopeHz+kOkk58Z8xo2jioNrls4rLAMaDI2DZtWhywaxwriKgc1ARt36TYvbclRSjCHLS+bzQ5rjU1ZnZ+R5xZiK4ANZZvHBYUSimyOzBNcyn07xlaFzLl4JK6ydMqoyRtUUbxSTG6bTKUeHIxRP8AXeCdbk1OslJo90wJ0PtE2gmo5wQelaT5Cw0Sq6mZf9hNrcrps7eNdLIP3NtHO/9jtEgaf7Ace/N6Z3mqX9l2x2hvJE6P9pR03bOlbrLsaGAjS2jxVFq9xqQDtP3Z5ishVZU5C3MTffh5oQuk3lb9cFOtdgbKrAli2Nb24z0MBkMo0t1fyS05OHYJeR3sDmZFKS5yOKzLKoO7rOww6FwrNwKQr87Pws0a7WdG1N3fqNn3qjhdgstUiQyGyHJ5acJz+bgDV93rZsFFufWiTGYI3bZregMaUnBJq6JXgly0pKI8wPPWAZ7+1tznO9/umkDLf+OlHdMB32i4ZqtMKVXunFAvkQYoJpn0LlxW5cJiGEVJ4bULNN39tayVGBhxA2nOGKbj6PpkCrGjJfYNSTpa3m06yK8X3xOm6Y9zRu4dldDqS3dGRT4Yay+d5+eOIlEWJmTPoGUTKgdjleDWos2c7sPHnYYUpwJqelQplijSAmo+9YZEVQo5F3u81pl9CthQcPH9CsV3RNsznf0DnWYYWEjmIlZKViTQz2+SD4EAtEghpCyAiJqlZDGYuiVHAeLBZ10c9jyJjZQ8obh8wnc/JshCMjEOhQPjipKUpYexjt7XHz3nWm+2OKiScfQVbmsWBs5PFjz6jw5PmUshrT+ZLZrESCZT6pmE4yykLJdY4HjM7QYAjeoXTYzKDlGC1ijroRt1Hga5nTGY+VHPVTOgvrPEsuubiw9hZysD4t3BItczNCs0CwiasDoa06XF7ELCQDkgmOgh88jo2fm9bQtgXOWVwbKMgQY+iaVJlcj/AhsJJ9OhN3Csbk3L37c+lefuLG2cypnYftzbUbnpEUt+q/5UMu763fRJ86/CTfSe+X3967/e+uGosP6a7XNL+kp2FI88XHIKSYgLEdNgsEr7SNJWhFaiuBD+CcjamF6QdCyiyzMcmdZTCcu9ji8LzJUUZRT4nFmoyiMNTasVrDsnl6Fn8Yoi9gpn9auHPnjn7jG994ab83YMCAAVcB3/zmN7+tqh9KJHkBupQBAwYMGPAq4qVa4CLyEFgCj17aj74aOGKQ+fOAQebPBy5D5i+o6vWnD75UBQ4gIt961lbgKmOQ+fOBQebPB14lmQcXyoABAwa8phgU+IABAwa8prgMBf7PLuE3LxuDzJ8PDDJ/PvDKyPzSfeADBgwYMODTweBCGTBgwIDXFC9NgYvIXxGR74vID1MT5CsJEXlHRH5fRL4jIt9Kxw5F5D+JyA/S48Fln+cngYj8SxF5ICLf2zn2XBlF5O+ncf++iPzlyznrT4bnyPzrIvLjNNbfEZGv7rx2FWR+Q0T+i4i8LSJ/ICJ/Jx2/smP9ETK/mmPdExF9lv+IlGJ/SOyvWQC/B/zsy/jtl/0PeAc4eurYPwJ+NT3/VeAfXvZ5fkIZf4HYZu97Hycj8LNpvEvgrXQf2MuW4VOS+deBv/eM914VmW8DP5+ez4D/k2S7smP9ETK/kmP9sizwPw/8UFV/pLGr/W8BX3tJv/0q4GvAb6TnvwH81cs7lU8OVf2vwPFTh58n49eA31LVRlX/CPgh8X54rfAcmZ+HqyLz+6r6P9PzBfA2cJcrPNYfIfPzcKkyvywFfhd4d+fv9/joi/I6Q4H/KCLfFpGe+OWmqr4P8QYBblza2X12eJ6MV33s/5aIfDe5WHpXwpWTWUR+CvgzwO/yORnrp2SGV3CsX5YCfxat1lVNf/kLqvrzwC8Bf1NEfuGyT+iScZXH/p8CXwR+Dngf+Mfp+JWSWUSmwL8D/q6qnn/UW59x7LWU+xkyv5Jj/bIU+HvAGzt/3wN+8pJ++6VCVX+SHh8A/4G4nbovIrcB0uODyzvDzwzPk/HKjr2q3ldVr5EY/Z+z3TpfGZlFJCcqst9U1X+fDl/psX6WzK/qWL8sBf4/gC+LyFupMfIvA7/zkn77pUFEJiIy658Dfwn4HlHWr6e3fR347cs5w88Uz5Pxd4BfFpFSRN4Cvgz890s4v08dvRJL+GvEsYYrIrNEUut/Abytqv9k56UrO9bPk/mVHeuXGN39KjGi+4fAr112tPkzkvGniRHp3wP+oJcTuAb8Z+AH6fHwss/1E8r5r4nbyI5ogfzKR8kI/Foa9+8Dv3TZ5/8pyvyvgN8HvkucyLevmMx/kegO+C7wnfTvq1d5rD9C5ldyrIdKzAEDBgx4TTFUYg4YMGDAa4pBgQ8YMGDAa4pBgQ8YMGDAa4pBgQ8YMGDAa4pBgQ8YMGDAa4pBgQ8YMGDAa4pBgQ8YMGDAa4pBgQ8YMGDAa4r/B+fFbHI53WcGAAAAAElFTkSuQmCC\n" 241 | }, 242 | "metadata": { 243 | "needs_background": "light" 244 | }, 245 | "output_type": "display_data" 246 | }, 247 | { 248 | "name": "stdout", 249 | "output_type": "stream", 250 | "text": [ 251 | "car car plane truck horse horse dog horse car plane dog bird frog deer bird horse\n" 252 | ] 253 | } 254 | ], 255 | "source": [ 256 | "# 获取一些图片及其对应的标签\n", 257 | "dataiter = iter(trainloader)\n", 258 | "\"\"\"\n", 259 | "这里images是一个tensor，shape为(16, 3, 32, 32)\n", 260 | "\t其中16为batch_size，即16张图片，3为RGB通道，图片大小为32x32\n", 261 | "labels也是tensor，shape为(16)，为每个图片的对应的标签。\n", 262 | "\"\"\"\n", 263 | "images, labels = dataiter.next()\n", 264 | "\n", 265 | "\"\"\"\n", 266 | "make_grid：制作表格。即把多张图片拼到一张中去。\n", 267 | "\t\t nrow=8，表示生成表格的列数。\n", 268 | "从下面的输出可以看到，images的16张图片被make_grid\n", 269 | "按照2x8的表格拼成了一张大图片。该方法方便人们进行\n", 270 | "图像展示。\n", 271 | "\"\"\"\n", 272 | "imshow(torchvision.utils.make_grid(images, nrow=8))\n", 273 | "# print labels\n", 274 | "print(' '.join(f'{classes[labels[j]]:5s}' for j in range(batch_size)))" 275 | ], 276 | "metadata": { 277 | "collapsed": false, 278 | "pycharm": { 279 | "name": "#%%\n" 280 | } 281 | } 282 | }, 283 | { 284 | "cell_type": "markdown", 285 | "source": [ 286 | "# 定义CNN分类模型" 287 | ], 288 | "metadata": { 289 | "collapsed": false, 290 | "pycharm": { 291 | "name": "#%% md\n" 292 | } 293 | } 294 | }, 295 | { 296 | "cell_type": "code", 297 | "execution_count": 8, 298 | "outputs": [], 299 | "source": [ 300 | "class Net(nn.Module):\n", 301 | " def __init__(self):\n", 302 | " super().__init__()\n", 303 | " \"\"\"\n", 304 | " \t定义卷积层, nn.Conv2d官方文档：https://pytorch.org/docs/stable/generated/torch.nn.Conv2d.html\n", 305 | " \tnn.Conv2d包含三个重要参数：\n", 306 | " \t\tin_channels: 输入的通道数\n", 307 | " \t\tout_channels: 输出的通道数\n", 308 | " \t\tkernel_size: 卷积核的大小\n", 309 | " \t\tstride: 步长，默认为1\n", 310 | " \t\tpadding: 填充，默认为0，即不进行填充\n", 311 | "\n", 312 | " \t补充：这里卷积层2d的意思是数据是“2维的”，\n", 313 | " \t\t 例如图片就是2维数据（长×宽）。同理也有Conv1d，\n", 314 | " \t\t 是针对于文本、信号等1维数据，也有Conv3d\n", 315 | " \t\t 是针对视频等这种3维数据。\n", 316 | " \t\"\"\"\n", 317 | " self.classifier = nn.Sequential(\n", 318 | " nn.Conv2d(3, 6, 5),\n", 319 | " # 激活函数\n", 320 | " nn.ReLU(),\n", 321 | " # 使用MaxPool进行下采样。\n", 322 | " nn.MaxPool2d(2, 2),\n", 323 | " nn.Conv2d(6, 16, 5),\n", 324 | " nn.ReLU(),\n", 325 | " nn.MaxPool2d(2, 2),\n", 326 | " # 当完成卷积后，使用flatten将数据展开\n", 327 | " # 即将tensor的shape从(batch_size, c, h, w)变成(batch_size, c*h*w)，这样才能送给全连接层\n", 328 | " nn.Flatten(),\n", 329 | " # 最后接全连接层。\n", 330 | " # 计算方式可以参考：https://blog.csdn.net/zhaohongfei_358/article/details/123269313\n", 331 | " nn.Linear(16 * 5 * 5, 120),\n", 332 | " nn.ReLU(),\n", 333 | " nn.Linear(120, 84),\n", 334 | " nn.ReLU(),\n", 335 | " nn.Linear(84, 10)\n", 336 | " # 注意这里并没有调用Softmax，也不能调Softmax\n", 337 | " # 这是因为Softmax被包含在了CrossEntropyLoss损失函数中\n", 338 | " # 如果这里调用的话，就会调用两遍，最后网络啥都学不着\n", 339 | " )\n", 340 | "\n", 341 | " def forward(self, x):\n", 342 | " return self.classifier(x)" 343 | ], 344 | "metadata": { 345 | "collapsed": false, 346 | "pycharm": { 347 | "name": "#%%\n" 348 | } 349 | } 350 | }, 351 | { 352 | "cell_type": "code", 353 | "execution_count": 9, 354 | "outputs": [], 355 | "source": [ 356 | "net = Net()\n", 357 | "# 使用简单的CrossEntorpyLoss作为损失函数，一般多分类问题都用这个\n", 358 | "criterion = nn.CrossEntropyLoss()\n", 359 | "# 使用简单的SGD作为优化器\n", 360 | "optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9)" 361 | ], 362 | "metadata": { 363 | "collapsed": false, 364 | "pycharm": { 365 | "name": "#%%\n" 366 | } 367 | } 368 | }, 369 | { 370 | "cell_type": "markdown", 371 | "source": [ 372 | "# 训练网络" 373 | ], 374 | "metadata": { 375 | "collapsed": false, 376 | "pycharm": { 377 | "name": "#%% md\n" 378 | } 379 | } 380 | }, 381 | { 382 | "cell_type": "markdown", 383 | "source": [ 384 | "开始训练网络，由于网络较小，这里直接用cpu进行训练" 385 | ], 386 | "metadata": { 387 | "collapsed": false, 388 | "pycharm": { 389 | "name": "#%% md\n" 390 | } 391 | } 392 | }, 393 | { 394 | "cell_type": "code", 395 | "execution_count": 10, 396 | "outputs": [ 397 | { 398 | "name": "stdout", 399 | "output_type": "stream", 400 | "text": [ 401 | "[1, 2000] loss: 2.113\n", 402 | "[2, 2000] loss: 1.567\n", 403 | "[3, 2000] loss: 1.406\n", 404 | "[4, 2000] loss: 1.280\n", 405 | "[5, 2000] loss: 1.194\n", 406 | "[6, 2000] loss: 1.121\n", 407 | "[7, 2000] loss: 1.063\n", 408 | "[8, 2000] loss: 1.017\n", 409 | "[9, 2000] loss: 0.966\n", 410 | "[10, 2000] loss: 0.924\n", 411 | "[11, 2000] loss: 0.885\n", 412 | "[12, 2000] loss: 0.848\n", 413 | "[13, 2000] loss: 0.825\n", 414 | "[14, 2000] loss: 0.787\n", 415 | "[15, 2000] loss: 0.758\n", 416 | "[16, 2000] loss: 0.735\n", 417 | "[17, 2000] loss: 0.698\n", 418 | "[18, 2000] loss: 0.679\n", 419 | "[19, 2000] loss: 0.655\n", 420 | "[20, 2000] loss: 0.630\n", 421 | "Finished Training\n" 422 | ] 423 | } 424 | ], 425 | "source": [ 426 | "# 把所有训练样本看过一遍称为1个epoch\n", 427 | "# 简单起见，这里只训练20个epochs\n", 428 | "epochs = 20\n", 429 | "\n", 430 | "for epoch in range(epochs):\n", 431 | "\t# 记录一下损失\n", 432 | " running_loss = 0.0\n", 433 | " for i, data in enumerate(trainloader):\n", 434 | "\t\t# trainloader返回的是tuple，第一个是图片数，第二个对应的labels\n", 435 | " inputs, labels = data\n", 436 | "\n", 437 | " # 清除之前的梯度\n", 438 | " optimizer.zero_grad()\n", 439 | "\n", 440 | " # 进行前向传播\n", 441 | " outputs = net(inputs)\n", 442 | " # 计算损失\n", 443 | " loss = criterion(outputs, labels)\n", 444 | " # 反向传播\n", 445 | " loss.backward()\n", 446 | " # 更新参数\n", 447 | " optimizer.step()\n", 448 | "\n", 449 | " # 记录损失，每2000次打印一次损失\n", 450 | " running_loss += loss.item()\n", 451 | " if i % 2000 == 1999:\n", 452 | " print(f'[{epoch + 1}, {i + 1:5d}] loss: {running_loss / 2000:.3f}')\n", 453 | " running_loss = 0.0\n", 454 | "\n", 455 | "print('Finished Training')" 456 | ], 457 | "metadata": { 458 | "collapsed": false, 459 | "pycharm": { 460 | "name": "#%%\n" 461 | } 462 | } 463 | }, 464 | { 465 | "cell_type": "markdown", 466 | "source": [ 467 | "在训练了20个epoch后，损失降到了0.648。可以看到损失还在下降，大家可以尝试多训练一段时间。" 468 | ], 469 | "metadata": { 470 | "collapsed": false, 471 | "pycharm": { 472 | "name": "#%% md\n" 473 | } 474 | } 475 | }, 476 | { 477 | "cell_type": "markdown", 478 | "source": [ 479 | "# 测试模型" 480 | ], 481 | "metadata": { 482 | "collapsed": false, 483 | "pycharm": { 484 | "name": "#%% md\n" 485 | } 486 | } 487 | }, 488 | { 489 | "cell_type": "markdown", 490 | "source": [ 491 | "测试下模型总的精准率：" 492 | ], 493 | "metadata": { 494 | "collapsed": false, 495 | "pycharm": { 496 | "name": "#%% md\n" 497 | } 498 | }, 499 | "outputs": [ 500 | { 501 | "ename": "SyntaxError", 502 | "evalue": "invalid character in identifier (, line 1)", 503 | "output_type": "error", 504 | "traceback": [ 505 | "\u001B[1;36m File \u001B[1;32m\"\"\u001B[1;36m, line \u001B[1;32m1\u001B[0m\n\u001B[1;33m 测试下模型总的精准率：\u001B[0m\n\u001B[1;37m ^\u001B[0m\n\u001B[1;31mSyntaxError\u001B[0m\u001B[1;31m:\u001B[0m invalid character in identifier\n" 506 | ] 507 | } 508 | ], 509 | "execution_count": 11 510 | }, 511 | { 512 | "cell_type": "code", 513 | "execution_count": 12, 514 | "outputs": [ 515 | { 516 | "name": "stdout", 517 | "output_type": "stream", 518 | "text": [ 519 | "Accuracy of the network on the 10000 test images: 64 %\n" 520 | ] 521 | } 522 | ], 523 | "source": [ 524 | "correct = 0 # 记录正确的数量\n", 525 | "total = 0 # 记录总数\n", 526 | "\n", 527 | "# torch.no_grad表示不需要计算梯度。\n", 528 | "with torch.no_grad():\n", 529 | " for data in testloader:\n", 530 | " images, labels = data\n", 531 | "\t\t# 前向传播\n", 532 | " outputs = net(images)\n", 533 | " \"\"\"\n", 534 | "\t\toutputs.shape为(16, 10)，batch_size为16, 10为类别\n", 535 | "\t\toutput这16张图片的各个类别的可能性（未经Softmax处理）\n", 536 | "\t\t所以通过torch.max找到最大的那个。\n", 537 | "\t\ttorch.max接受两个参数，第一个是tensor，第二个是dim（维度）\n", 538 | "\t\t\t\t 这里传1，意思是在类别这个维度上取最大的\n", 539 | "\t\ttorch.max有两个输出，values和indexes，\n", 540 | "\t\t\t\t values就是最大的数是什么，\n", 541 | "\t\t\t\t indexes是这些最大的数的index是什么\n", 542 | "\t\t这里我们只需要index即可，所以忽略第一个参数\n", 543 | "\t\t\"\"\"\n", 544 | " _, predicted = torch.max(outputs, 1)\n", 545 | " # 记录总数量\n", 546 | " total += labels.size(0)\n", 547 | " # 计算正确数量\n", 548 | " correct += (predicted == labels).sum().item()\n", 549 | "\n", 550 | "print(f'Accuracy of the network on the 10000 test images: {100 * correct // total} %')" 551 | ], 552 | "metadata": { 553 | "collapsed": false, 554 | "pycharm": { 555 | "name": "#%%\n" 556 | } 557 | } 558 | }, 559 | { 560 | "cell_type": "markdown", 561 | "source": [ 562 | "接下来计算下每个类别精准率：" 563 | ], 564 | "metadata": { 565 | "collapsed": false, 566 | "pycharm": { 567 | "name": "#%% md\n" 568 | } 569 | } 570 | }, 571 | { 572 | "cell_type": "code", 573 | "execution_count": 13, 574 | "outputs": [ 575 | { 576 | "name": "stdout", 577 | "output_type": "stream", 578 | "text": [ 579 | "Accuracy for class: plane is 80.0 %\n", 580 | "Accuracy for class: car is 70.5 %\n", 581 | "Accuracy for class: bird is 53.7 %\n", 582 | "Accuracy for class: cat is 46.3 %\n", 583 | "Accuracy for class: deer is 55.6 %\n", 584 | "Accuracy for class: dog is 50.8 %\n", 585 | "Accuracy for class: frog is 77.8 %\n", 586 | "Accuracy for class: horse is 67.4 %\n", 587 | "Accuracy for class: ship is 73.6 %\n", 588 | "Accuracy for class: truck is 70.9 %\n" 589 | ] 590 | } 591 | ], 592 | "source": [ 593 | "# 统计每个类别的正确数量和总数量\n", 594 | "correct_pred = {classname: 0 for classname in classes}\n", 595 | "total_pred = {classname: 0 for classname in classes}\n", 596 | "\n", 597 | "# again no gradients needed\n", 598 | "with torch.no_grad():\n", 599 | " for data in testloader:\n", 600 | " images, labels = data\n", 601 | " outputs = net(images)\n", 602 | " _, predictions = torch.max(outputs, 1)\n", 603 | " # collect the correct predictions for each class\n", 604 | " for label, prediction in zip(labels, predictions):\n", 605 | " if label == prediction:\n", 606 | " correct_pred[classes[label]] += 1\n", 607 | " total_pred[classes[label]] += 1\n", 608 | "\n", 609 | "\n", 610 | "# print accuracy for each class\n", 611 | "for classname, correct_count in correct_pred.items():\n", 612 | " accuracy = 100 * float(correct_count) / total_pred[classname]\n", 613 | " print(f'Accuracy for class: {classname:5s} is {accuracy:.1f} %')" 614 | ], 615 | "metadata": { 616 | "collapsed": false, 617 | "pycharm": { 618 | "name": "#%%\n" 619 | } 620 | } 621 | }, 622 | { 623 | "cell_type": "markdown", 624 | "source": [ 625 | "# 参考文献\n", 626 | "\n", 627 | "[pytorch官方CNN分类样例](https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html): https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html" 628 | ], 629 | "metadata": { 630 | "collapsed": false, 631 | "pycharm": { 632 | "name": "#%% md\n" 633 | } 634 | } 635 | }, 636 | { 637 | "cell_type": "code", 638 | "execution_count": null, 639 | "outputs": [], 640 | "source": [], 641 | "metadata": { 642 | "collapsed": false, 643 | "pycharm": { 644 | "name": "#%%\n" 645 | } 646 | } 647 | } 648 | ], 649 | "metadata": { 650 | "kernelspec": { 651 | "display_name": "Python 3", 652 | "language": "python", 653 | "name": "python3" 654 | }, 655 | "language_info": { 656 | "codemirror_mode": { 657 | "name": "ipython", 658 | "version": 2 659 | }, 660 | "file_extension": ".py", 661 | "mimetype": "text/x-python", 662 | "name": "python", 663 | "nbconvert_exporter": "python", 664 | "pygments_lexer": "ipython2", 665 | "version": "2.7.6" 666 | } 667 | }, 668 | "nbformat": 4, 669 | "nbformat_minor": 0 670 | } -------------------------------------------------------------------------------- /04_LSTM_sentiment_analysis.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": { 6 | "collapsed": true, 7 | "pycharm": { 8 | "name": "#%% md\n" 9 | }, 10 | "id": "mMulrrXe6_0i" 11 | }, 12 | "source": [ 13 | "# Pytorch入门实战（4）：基于LSTM实现文本的情感分析" 14 | ] 15 | }, 16 | { 17 | "cell_type": "markdown", 18 | "source": [ 19 | "# 本文涉及知识点\n", 20 | "\n", 21 | "[Pytorch nn.Module的基本使用](https://blog.csdn.net/zhaohongfei_358/article/details/122797244)\n", 22 | "\n", 23 | "[Pytorch nn.Linear的基本用法](https://blog.csdn.net/zhaohongfei_358/article/details/122797190)\n", 24 | "\n", 25 | "[Pytorch中DataLoader的基本用法](https://blog.csdn.net/zhaohongfei_358/article/details/122742656)\n", 26 | "\n", 27 | "[Pytorch nn.Embedding的基本使用](https://blog.csdn.net/zhaohongfei_358/article/details/122809709)\n", 28 | "\n", 29 | "[详解torch.nn.utils.clip_grad_norm_ 的使用与原理](https://blog.csdn.net/zhaohongfei_358/article/details/122820992)" 30 | ], 31 | "metadata": { 32 | "collapsed": false, 33 | "pycharm": { 34 | "name": "#%% md\n" 35 | }, 36 | "id": "TyKcUptm6_0k" 37 | } 38 | }, 39 | { 40 | "cell_type": "markdown", 41 | "source": [ 42 | "# 本文内容\n", 43 | "\n", 44 | "本文基于文章[Long Short-Term Memory: From Zero to Hero with PyTorch](https://blog.floydhub.com/long-short-term-memory-from-zero-to-hero-with-pytorch/)的代码，对该文章代码进行了一些修改和注释添加。该文章详细的介绍了LSTM，如果对LSTM不熟悉的朋友，可以先看下改文章。\n", 45 | "\n", 46 | "本文使用的亚马逊评论数据集，训练一个可以判别文本情感的分类器。\n", 47 | "\n", 48 | "数据集如下：\n", 49 | "\n", 50 | "```\n", 51 | "链接：https://pan.baidu.com/s/1cK-scxLIliTsOPF-6byucQ\n", 52 | "提取码：yqbq\n", 53 | "```" 54 | ], 55 | "metadata": { 56 | "collapsed": false, 57 | "pycharm": { 58 | "name": "#%% md\n" 59 | }, 60 | "id": "YTCs253y6_0l" 61 | } 62 | }, 63 | { 64 | "cell_type": "markdown", 65 | "source": [ 66 | "# 数据预处理\n", 67 | "\n", 68 | "首先导入要使用的包：" 69 | ], 70 | "metadata": { 71 | "collapsed": false, 72 | "pycharm": { 73 | "name": "#%% md\n" 74 | }, 75 | "id": "rv0IV3qr6_0l" 76 | } 77 | }, 78 | { 79 | "cell_type": "code", 80 | "execution_count": 1, 81 | "outputs": [], 82 | "source": [ 83 | "import bz2 # 用于读取bz2压缩文件\n", 84 | "from collections import Counter # 用于统计词频\n", 85 | "import re # 正则表达式\n", 86 | "import nltk # 文本预处理\n", 87 | "import numpy as np" 88 | ], 89 | "metadata": { 90 | "pycharm": { 91 | "name": "#%%\n" 92 | }, 93 | "id": "dcJoLvsE6_0m" 94 | } 95 | }, 96 | { 97 | "cell_type": "markdown", 98 | "source": [ 99 | "将数据样本解压到当前目录的data目录下，其中包含两个文件：train.ft.txt.bz2”和“test.ft.txt.bz2”\n", 100 | "\n", 101 | "解压后，读取训练数据和测试数据：" 102 | ], 103 | "metadata": { 104 | "collapsed": false, 105 | "pycharm": { 106 | "name": "#%% md\n" 107 | }, 108 | "id": "RhSZFZBQ6_0m" 109 | } 110 | }, 111 | { 112 | "cell_type": "code", 113 | "execution_count": 2, 114 | "outputs": [ 115 | { 116 | "output_type": "stream", 117 | "name": "stdout", 118 | "text": [ 119 | "/usr/local/lib/python3.7/dist-packages/gdown/cli.py:131: FutureWarning: Option `--id` was deprecated in version 4.3.1 and will be removed in 5.0. You don't need to pass it anymore to use a file ID.\n", 120 | " category=FutureWarning,\n", 121 | "Downloading...\n", 122 | "From: https://drive.google.com/uc?id=1BKma83La6Cx3m1rWmnQScHjTWj-z65NP\n", 123 | "To: /content/data.zip\n", 124 | "100% 517M/517M [00:05<00:00, 90.6MB/s]\n" 125 | ] 126 | } 127 | ], 128 | "source": [ 129 | "!gdown --id '1BKma83La6Cx3m1rWmnQScHjTWj-z65NP' --output data.zip" 130 | ], 131 | "metadata": { 132 | "pycharm": { 133 | "name": "#%%\n" 134 | }, 135 | "id": "87YmVQWf6_0n", 136 | "outputId": "d07d229e-4760-48cb-a97f-b126d80eaf96", 137 | "colab": { 138 | "base_uri": "https://localhost:8080/" 139 | } 140 | } 141 | }, 142 | { 143 | "cell_type": "code", 144 | "execution_count": 3, 145 | "outputs": [ 146 | { 147 | "output_type": "stream", 148 | "name": "stdout", 149 | "text": [ 150 | "Archive: data.zip\n", 151 | " inflating: test.ft.txt.bz2 \n", 152 | " inflating: train.ft.txt.bz2 \n" 153 | ] 154 | } 155 | ], 156 | "source": [ 157 | "!unzip data.zip" 158 | ], 159 | "metadata": { 160 | "pycharm": { 161 | "name": "#%%\n" 162 | }, 163 | "id": "l6pid1tY6_0o", 164 | "outputId": "4e23b4b6-1e5f-44d8-b09e-f9a10cf4af6e", 165 | "colab": { 166 | "base_uri": "https://localhost:8080/" 167 | } 168 | } 169 | }, 170 | { 171 | "cell_type": "code", 172 | "execution_count": 5, 173 | "outputs": [ 174 | { 175 | "output_type": "stream", 176 | "name": "stdout", 177 | "text": [ 178 | "b'__label__2 Stuning even for the non-gamer: This sound track was beautiful! It paints the senery in your mind so well I would recomend it even to people who hate vid. game music! I have played the game Chrono Cross but out of all of the games I have ever played it has the best music! It backs away from crude keyboarding and takes a fresher step with grate guitars and soulful orchestras. It would impress anyone who cares to listen! ^_^\\n'\n" 179 | ] 180 | } 181 | ], 182 | "source": [ 183 | "train_file = bz2.BZ2File('train.ft.txt.bz2')\n", 184 | "test_file = bz2.BZ2File('test.ft.txt.bz2')\n", 185 | "train_file = train_file.readlines()\n", 186 | "test_file = test_file.readlines()\n", 187 | "print(train_file[0])" 188 | ], 189 | "metadata": { 190 | "pycharm": { 191 | "name": "#%%\n" 192 | }, 193 | "id": "SAsgBBHj6_0o", 194 | "outputId": "359aba04-75cc-4cb9-dec2-cf8d9cd059d9", 195 | "colab": { 196 | "base_uri": "https://localhost:8080/" 197 | } 198 | } 199 | }, 200 | { 201 | "cell_type": "markdown", 202 | "source": [ 203 | "从上面打印的数据可以看到，每条数据由两部分组成，*Label*和*Data*。其中：\n", 204 | "\n", 205 | "- `__label__1` 代表差评，之后将其编码为0\n", 206 | "- `__label__2` 代表好评，之后将其编码为1\n", 207 | "\n", 208 | "由于数据量太大，所以这里只取100w条记录进行训练，训练集和测试集按照*8:2*进行拆分：" 209 | ], 210 | "metadata": { 211 | "collapsed": false, 212 | "pycharm": { 213 | "name": "#%% md\n" 214 | }, 215 | "id": "_BxmKkli6_0o" 216 | } 217 | }, 218 | { 219 | "cell_type": "code", 220 | "execution_count": 6, 221 | "outputs": [], 222 | "source": [ 223 | "num_train = 800000\n", 224 | "num_test = 200000\n", 225 | "\n", 226 | "train_file = [x.decode('utf-8') for x in train_file[:num_train]]\n", 227 | "test_file = [x.decode('utf-8') for x in test_file[:num_test]]" 228 | ], 229 | "metadata": { 230 | "pycharm": { 231 | "name": "#%%\n" 232 | }, 233 | "id": "uL3MhKtc6_0p" 234 | } 235 | }, 236 | { 237 | "cell_type": "markdown", 238 | "source": [ 239 | "> 这里使用decode('utf-8')是因为源文件是以二进制类型存储的，从上面的`b''`可以看出\n", 240 | "\n", 241 | "源文件中，数据和标签是在一起的，所以要将其拆分开：" 242 | ], 243 | "metadata": { 244 | "collapsed": false, 245 | "pycharm": { 246 | "name": "#%% md\n" 247 | }, 248 | "id": "4utoGSoz6_0p" 249 | } 250 | }, 251 | { 252 | "cell_type": "code", 253 | "execution_count": 7, 254 | "outputs": [], 255 | "source": [ 256 | "# 将__label__1编码为0（差评），__label__2编码为1（好评）\n", 257 | "train_labels = [0 if x.split(' ')[0] == '__label__1' else 1 for x in train_file]\n", 258 | "test_labels = [0 if x.split(' ')[0] == '__label__1' else 1 for x in test_file]\n", 259 | "\n", 260 | "\"\"\"\n", 261 | "`split(' ', 1)[1]`：将label和data分开后，获取data部分\n", 262 | "`[:-1]`：去掉最后一个字符(\\n)\n", 263 | "`lower()`: 将其转换为小写，因为区分大小写对情感识别帮助不大，且会增加编码难度\n", 264 | "\"\"\"\n", 265 | "train_sentences = [x.split(' ', 1)[1][:-1].lower() for x in train_file]\n", 266 | "test_sentences = [x.split(' ', 1)[1][:-1].lower() for x in test_file]" 267 | ], 268 | "metadata": { 269 | "pycharm": { 270 | "name": "#%%\n" 271 | }, 272 | "id": "2JlI_ZXE6_0p" 273 | } 274 | }, 275 | { 276 | "cell_type": "markdown", 277 | "source": [ 278 | "在对数据拆分后，对数据进行简单的数据清理：\n", 279 | "\n", 280 | "由于数字对情感分类帮助不大，所以这里将所有的数字都转换为0：" 281 | ], 282 | "metadata": { 283 | "collapsed": false, 284 | "pycharm": { 285 | "name": "#%% md\n" 286 | }, 287 | "id": "Nk04Vc-y6_0p" 288 | } 289 | }, 290 | { 291 | "cell_type": "code", 292 | "execution_count": 8, 293 | "outputs": [], 294 | "source": [ 295 | "for i in range(len(train_sentences)):\n", 296 | " train_sentences[i] = re.sub('\\d','0',train_sentences[i])\n", 297 | "\n", 298 | "for i in range(len(test_sentences)):\n", 299 | " test_sentences[i] = re.sub('\\d','0',test_sentences[i])" 300 | ], 301 | "metadata": { 302 | "pycharm": { 303 | "name": "#%%\n" 304 | }, 305 | "id": "07AVnZC16_0p" 306 | } 307 | }, 308 | { 309 | "cell_type": "markdown", 310 | "source": [ 311 | "数据集中还存在包含网站的样本，例如：`Welcome to our website: www.pohabo.com`。对于这种带有网站的样本，网站地址会干扰数据处理，所以一律处理成：`Welcome to our website: `：" 312 | ], 313 | "metadata": { 314 | "collapsed": false, 315 | "pycharm": { 316 | "name": "#%% md\n" 317 | }, 318 | "id": "abM6rblh6_0q" 319 | } 320 | }, 321 | { 322 | "cell_type": "code", 323 | "execution_count": 9, 324 | "outputs": [], 325 | "source": [ 326 | "for i in range(len(train_sentences)):\n", 327 | " if 'www.' in train_sentences[i] or 'http:' in train_sentences[i] or 'https:' in train_sentences[i] or '.com' in train_sentences[i]:\n", 328 | " train_sentences[i] = re.sub(r\"([^ ]+(?<=\\.[a-z]{3}))\", \"\", train_sentences[i])\n", 329 | "\n", 330 | "for i in range(len(test_sentences)):\n", 331 | " if 'www.' in test_sentences[i] or 'http:' in test_sentences[i] or 'https:' in test_sentences[i] or '.com' in test_sentences[i]:\n", 332 | " test_sentences[i] = re.sub(r\"([^ ]+(?<=\\.[a-z]{3}))\", \"\", test_sentences[i])" 333 | ], 334 | "metadata": { 335 | "pycharm": { 336 | "name": "#%%\n" 337 | }, 338 | "id": "oiHNzDE56_0q" 339 | } 340 | }, 341 | { 342 | "cell_type": "markdown", 343 | "source": [ 344 | "数据清理结束后，我们需要将**文本进行分词**，并**将仅出现一次的单词丢掉**，因为它们参考价值不大：" 345 | ], 346 | "metadata": { 347 | "collapsed": false, 348 | "pycharm": { 349 | "name": "#%% md\n" 350 | }, 351 | "id": "pVoDvGR16_0q" 352 | } 353 | }, 354 | { 355 | "cell_type": "code", 356 | "source": [ 357 | "nltk.download('punkt') # 使用nltk.work_tokenize前，需要下载`punkt`" 358 | ], 359 | "metadata": { 360 | "id": "0SerO5AE7-pw", 361 | "outputId": "6337ca5e-754a-410a-bdee-8bbda9f3e0b5", 362 | "colab": { 363 | "base_uri": "https://localhost:8080/" 364 | } 365 | }, 366 | "execution_count": 12, 367 | "outputs": [ 368 | { 369 | "output_type": "stream", 370 | "name": "stderr", 371 | "text": [ 372 | "[nltk_data] Downloading package punkt to /root/nltk_data...\n", 373 | "[nltk_data] Unzipping tokenizers/punkt.zip.\n" 374 | ] 375 | }, 376 | { 377 | "output_type": "execute_result", 378 | "data": { 379 | "text/plain": [ 380 | "True" 381 | ] 382 | }, 383 | "metadata": {}, 384 | "execution_count": 12 385 | } 386 | ] 387 | }, 388 | { 389 | "cell_type": "code", 390 | "execution_count": 13, 391 | "outputs": [ 392 | { 393 | "output_type": "stream", 394 | "name": "stdout", 395 | "text": [ 396 | "0.0% done\n", 397 | "25.0% done\n", 398 | "50.0% done\n", 399 | "75.0% done\n", 400 | "100% done\n" 401 | ] 402 | } 403 | ], 404 | "source": [ 405 | "words = Counter() # 用于统计每个单词出现的次数\n", 406 | "for i, sentence in enumerate(train_sentences):\n", 407 | " words_list = nltk.word_tokenize(sentence) # 将句子进行分词\n", 408 | " words.update(words_list) # 更新词频列表\n", 409 | " train_sentences[i] = words_list # 分词后的单词列表存在该列表中\n", 410 | "\n", 411 | " if i % 20000 == 0: # 每2w打印一次进度\n", 412 | " print(str((i*100)/num_train) + \"% done\")\n", 413 | "print(\"100% done\")" 414 | ], 415 | "metadata": { 416 | "pycharm": { 417 | "name": "#%%\n" 418 | }, 419 | "id": "uUIDMkDt6_0q", 420 | "outputId": "b3db9b70-b558-4f14-9278-e8ef4f6a3872", 421 | "colab": { 422 | "base_uri": "https://localhost:8080/" 423 | } 424 | } 425 | }, 426 | { 427 | "cell_type": "markdown", 428 | "source": [ 429 | "移除仅出现一次的单词：" 430 | ], 431 | "metadata": { 432 | "collapsed": false, 433 | "pycharm": { 434 | "name": "#%% md\n" 435 | }, 436 | "id": "iWS8bm1g6_0q" 437 | } 438 | }, 439 | { 440 | "cell_type": "code", 441 | "execution_count": 14, 442 | "outputs": [], 443 | "source": [ 444 | "words = {k:v for k,v in words.items() if v>1}" 445 | ], 446 | "metadata": { 447 | "pycharm": { 448 | "name": "#%%\n" 449 | }, 450 | "id": "AeW4L89X6_0r" 451 | } 452 | }, 453 | { 454 | "cell_type": "markdown", 455 | "source": [ 456 | "将words按照出现次数由大到小排序，并转换为list，**作为我们的词典**，之后**对于单词的编码会基于该词典**：" 457 | ], 458 | "metadata": { 459 | "collapsed": false, 460 | "pycharm": { 461 | "name": "#%% md\n" 462 | }, 463 | "id": "F_twWbLr6_0r" 464 | } 465 | }, 466 | { 467 | "cell_type": "code", 468 | "execution_count": 15, 469 | "outputs": [ 470 | { 471 | "output_type": "stream", 472 | "name": "stdout", 473 | "text": [ 474 | "['.', 'the', ',', 'i', 'and', 'a', 'to', 'it', 'of', 'this']\n" 475 | ] 476 | } 477 | ], 478 | "source": [ 479 | "words = sorted(words, key=words.get,reverse=True)\n", 480 | "print(words[:10]) # 打印一下出现次数最多的10个单词" 481 | ], 482 | "metadata": { 483 | "pycharm": { 484 | "name": "#%%\n" 485 | }, 486 | "id": "7xGQFepO6_0r", 487 | "outputId": "2e1e1eb0-9f5d-41c4-a2ef-784e9d849b30", 488 | "colab": { 489 | "base_uri": "https://localhost:8080/" 490 | } 491 | } 492 | }, 493 | { 494 | "cell_type": "markdown", 495 | "source": [ 496 | "向词典中增加一个单词：\n", 497 | "\n", 498 | "- `_PAD`：表示填充，因为后续会固定所有句子长度。过长的句子进行阶段，过短的句子使用该单词进行填充" 499 | ], 500 | "metadata": { 501 | "collapsed": false, 502 | "pycharm": { 503 | "name": "#%% md\n" 504 | }, 505 | "id": "R-kILnqA6_0r" 506 | } 507 | }, 508 | { 509 | "cell_type": "code", 510 | "execution_count": 16, 511 | "outputs": [], 512 | "source": [ 513 | "words = ['_PAD'] + words" 514 | ], 515 | "metadata": { 516 | "pycharm": { 517 | "name": "#%%\n" 518 | }, 519 | "id": "mkBOi5Np6_0r" 520 | } 521 | }, 522 | { 523 | "cell_type": "markdown", 524 | "source": [ 525 | "整理好词典后，对**单词进行编码**，即**将单词映射成数字**，这里直接使用单词所在的数字下表作为单词的编码值：" 526 | ], 527 | "metadata": { 528 | "collapsed": false, 529 | "pycharm": { 530 | "name": "#%% md\n" 531 | }, 532 | "id": "n35FzP_M6_0r" 533 | } 534 | }, 535 | { 536 | "cell_type": "code", 537 | "execution_count": 17, 538 | "outputs": [], 539 | "source": [ 540 | "word2idx = {o:i for i,o in enumerate(words)}\n", 541 | "idx2word = {i:o for i,o in enumerate(words)}" 542 | ], 543 | "metadata": { 544 | "pycharm": { 545 | "name": "#%%\n" 546 | }, 547 | "id": "NBWVkmM_6_0r" 548 | } 549 | }, 550 | { 551 | "cell_type": "markdown", 552 | "source": [ 553 | "映射字典准备完毕后，就可以将`train_sentences`中存储的单词转化为数字了：" 554 | ], 555 | "metadata": { 556 | "collapsed": false, 557 | "pycharm": { 558 | "name": "#%% md\n" 559 | }, 560 | "id": "FieIwn2u6_0r" 561 | } 562 | }, 563 | { 564 | "cell_type": "code", 565 | "execution_count": 18, 566 | "outputs": [], 567 | "source": [ 568 | "for i, sentence in enumerate(train_sentences):\n", 569 | " train_sentences[i] = [word2idx[word] if word in word2idx else 0 for word in sentence]\n", 570 | "\n", 571 | "for i, sentence in enumerate(test_sentences):\n", 572 | " test_sentences[i] = [word2idx[word.lower()] if word.lower() in word2idx else 0 for word in nltk.word_tokenize(sentence)]" 573 | ], 574 | "metadata": { 575 | "pycharm": { 576 | "name": "#%%\n" 577 | }, 578 | "id": "mOCna9Q56_0r" 579 | } 580 | }, 581 | { 582 | "cell_type": "markdown", 583 | "source": [ 584 | "> 上面的`else 0`表示：如果单词没有在字典中出现过，则使用编码0，对应上面的`_PAD`\n", 585 | "\n", 586 | "为了方便构建模型，需要固定所有句子的长度，这里选择200作为句子的固定长度，对于长度不够的句子，在前面填充`0`(`_PAD`)，超出长度的句子进行从后面截断：" 587 | ], 588 | "metadata": { 589 | "collapsed": false, 590 | "pycharm": { 591 | "name": "#%% md\n" 592 | }, 593 | "id": "G95_HFbD6_0s" 594 | } 595 | }, 596 | { 597 | "cell_type": "code", 598 | "execution_count": 19, 599 | "outputs": [], 600 | "source": [ 601 | "def pad_input(sentences, seq_len):\n", 602 | " \"\"\"\n", 603 | " 将句子长度固定为`seq_len`，超出长度的从后面阶段，长度不足的在前面补0\n", 604 | " \"\"\"\n", 605 | " features = np.zeros((len(sentences), seq_len),dtype=int)\n", 606 | " for ii, review in enumerate(sentences):\n", 607 | " if len(review) != 0:\n", 608 | " features[ii, -len(review):] = np.array(review)[:seq_len]\n", 609 | " return features\n", 610 | "\n", 611 | "# 固定测试数据集和训练数据集的句子长度\n", 612 | "train_sentences = pad_input(train_sentences, 200)\n", 613 | "test_sentences = pad_input(test_sentences, 200)" 614 | ], 615 | "metadata": { 616 | "pycharm": { 617 | "name": "#%%\n" 618 | }, 619 | "id": "chNfmNcv6_0s" 620 | } 621 | }, 622 | { 623 | "cell_type": "markdown", 624 | "source": [ 625 | "上述方法除了固定长度外，还顺便将数字转化为了numpy数组。Label数据集也需要转换一下：" 626 | ], 627 | "metadata": { 628 | "collapsed": false, 629 | "pycharm": { 630 | "name": "#%% md\n" 631 | }, 632 | "id": "nqRUKb6s6_0s" 633 | } 634 | }, 635 | { 636 | "cell_type": "code", 637 | "execution_count": 20, 638 | "outputs": [], 639 | "source": [ 640 | "train_labels = np.array(train_labels)\n", 641 | "test_labels = np.array(test_labels)" 642 | ], 643 | "metadata": { 644 | "pycharm": { 645 | "name": "#%%\n" 646 | }, 647 | "id": "01OAgkzX6_0s" 648 | } 649 | }, 650 | { 651 | "cell_type": "markdown", 652 | "source": [ 653 | "到这里，数据预处理的工作基本完成，接下来该PyTorch登场了" 654 | ], 655 | "metadata": { 656 | "collapsed": false, 657 | "pycharm": { 658 | "name": "#%% md\n" 659 | }, 660 | "id": "-SKNTwas6_0s" 661 | } 662 | }, 663 | { 664 | "cell_type": "markdown", 665 | "source": [ 666 | "# 模型构建\n", 667 | "\n", 668 | "首先导出Pytorch需要用到的包" 669 | ], 670 | "metadata": { 671 | "collapsed": false, 672 | "pycharm": { 673 | "name": "#%% md\n" 674 | }, 675 | "id": "bJWix0sq6_0s" 676 | } 677 | }, 678 | { 679 | "cell_type": "code", 680 | "execution_count": 21, 681 | "outputs": [], 682 | "source": [ 683 | "import torch\n", 684 | "from torch.utils.data import TensorDataset, DataLoader\n", 685 | "import torch.nn as nn" 686 | ], 687 | "metadata": { 688 | "pycharm": { 689 | "name": "#%%\n" 690 | }, 691 | "id": "CSdCaxjs6_0s" 692 | } 693 | }, 694 | { 695 | "cell_type": "markdown", 696 | "source": [ 697 | "构建训练数据集和测试数据集的DataLoader，同时**定义BatchSize为200**:" 698 | ], 699 | "metadata": { 700 | "collapsed": false, 701 | "pycharm": { 702 | "name": "#%% md\n" 703 | }, 704 | "id": "ngzQUMlL6_0s" 705 | } 706 | }, 707 | { 708 | "cell_type": "code", 709 | "execution_count": 22, 710 | "outputs": [], 711 | "source": [ 712 | "batch_size = 200\n", 713 | "\n", 714 | "train_data = TensorDataset(torch.from_numpy(train_sentences), torch.from_numpy(train_labels))\n", 715 | "test_data = TensorDataset(torch.from_numpy(test_sentences), torch.from_numpy(test_labels))\n", 716 | "\n", 717 | "train_loader = DataLoader(train_data, shuffle=True, batch_size=batch_size)\n", 718 | "test_loader = DataLoader(test_data, shuffle=True, batch_size=batch_size)" 719 | ], 720 | "metadata": { 721 | "pycharm": { 722 | "name": "#%%\n" 723 | }, 724 | "id": "rB8exua36_0s" 725 | } 726 | }, 727 | { 728 | "cell_type": "markdown", 729 | "source": [ 730 | "如果有条件，建议使用显卡来加速计算：" 731 | ], 732 | "metadata": { 733 | "collapsed": false, 734 | "pycharm": { 735 | "name": "#%% md\n" 736 | }, 737 | "id": "vR-y2-by6_0s" 738 | } 739 | }, 740 | { 741 | "cell_type": "code", 742 | "execution_count": 23, 743 | "outputs": [], 744 | "source": [ 745 | "device = torch.device('cuda') if torch.cuda.is_available() else torch.device(\"cpu\")" 746 | ], 747 | "metadata": { 748 | "pycharm": { 749 | "name": "#%%\n" 750 | }, 751 | "id": "-ltzuAyQ6_0s" 752 | } 753 | }, 754 | { 755 | "cell_type": "code", 756 | "execution_count": 24, 757 | "outputs": [], 758 | "source": [ 759 | "class SentimentNet(nn.Module):\n", 760 | " def __init__(self, vocab_size):\n", 761 | " super(SentimentNet, self).__init__()\n", 762 | " self.n_layers = n_layers = 2 # LSTM的层数\n", 763 | " self.hidden_dim = hidden_dim = 512 # 隐状态的维度，即LSTM输出的隐状态的维度为512\n", 764 | " embedding_dim = 400 # 将单词编码成400维的向量\n", 765 | " drop_prob=0.5 # dropout\n", 766 | "\n", 767 | " # 定义embedding，负责将数字编码成向量，详情可参考：https://blog.csdn.net/zhaohongfei_358/article/details/122809709\n", 768 | " self.embedding = nn.Embedding(vocab_size, embedding_dim)\n", 769 | "\n", 770 | " self.lstm = nn.LSTM(embedding_dim, # 输入的维度\n", 771 | " hidden_dim, # LSTM输出的hidden_state的维度\n", 772 | " n_layers, # LSTM的层数\n", 773 | " dropout=drop_prob,\n", 774 | " batch_first=True # 第一个维度是否是batch_size\n", 775 | " )\n", 776 | "\n", 777 | "\n", 778 | "\n", 779 | " # LSTM结束后的全连接线性层\n", 780 | " self.fc = nn.Linear(in_features=hidden_dim, # 将LSTM的输出作为线性层的输入\n", 781 | " out_features=1 # 由于情感分析只需要输出0或1，所以输出的维度是1\n", 782 | " )\n", 783 | " self.sigmoid = nn.Sigmoid() # 线性层输出后，还需要过一下sigmoid\n", 784 | "\n", 785 | " # 给最后的全连接层加一个Dropout\n", 786 | " self.dropout = nn.Dropout(drop_prob)\n", 787 | "\n", 788 | " def forward(self, x, hidden):\n", 789 | " \"\"\"\n", 790 | " x: 本次的输入，其size为(batch_size, 200)，200为句子长度\n", 791 | " hidden: 上一时刻的Hidden State和Cell State。类型为tuple: (h, c),\n", 792 | " 其中h和c的size都为(n_layers, batch_size, hidden_dim), 即(2, 200, 512)\n", 793 | " \"\"\"\n", 794 | " # 因为一次输入一组数据，所以第一个维度是batch的大小\n", 795 | " batch_size = x.size(0)\n", 796 | "\n", 797 | " # 由于embedding只接受LongTensor类型，所以将x转换为LongTensor类型\n", 798 | " x = x.long()\n", 799 | "\n", 800 | " # 对x进行编码，这里会将x的size由(batch_size, 200)转化为(batch_size, 200, embedding_dim)\n", 801 | " embeds = self.embedding(x)\n", 802 | "\n", 803 | " # 将编码后的向量和上一时刻的hidden_state传给LSTM，并获取本次的输出和隐状态（hidden_state, cell_state）\n", 804 | " # lstm_out的size为 (batch_size, 200, 512)，200是单词的数量，由于是一个单词一个单词送给LSTM的，所以会产生与单词数量相同的输出\n", 805 | " # hidden为tuple(hidden_state, cell_state)，它们俩的size都为(2, batch_size, 512), 2是由于lstm有两层。由于是所有单词都是共享隐状态的，所以并不会出现上面的那个200\n", 806 | " lstm_out, hidden = self.lstm(embeds, hidden)\n", 807 | "\n", 808 | " # 接下来要过全连接层，所以size变为(batch_size * 200, hidden_dim)，\n", 809 | " # 之所以是batch_size * 200=40000，是因为每个单词的输出都要经过全连接层。\n", 810 | " # 换句话说，全连接层的batch_size为40000\n", 811 | " lstm_out = lstm_out.contiguous().view(-1, self.hidden_dim)\n", 812 | "\n", 813 | " # 给全连接层加个Dropout\n", 814 | " out = self.dropout(lstm_out)\n", 815 | "\n", 816 | " # 将dropout后的数据送给全连接层\n", 817 | " # 全连接层输出的size为(40000, 1)\n", 818 | " out = self.fc(out)\n", 819 | "\n", 820 | " # 过一下sigmoid\n", 821 | " out = self.sigmoid(out)\n", 822 | "\n", 823 | " # 将最终的输出数据维度变为 (batch_size, 200)，即每个单词都对应一个输出\n", 824 | " out = out.view(batch_size, -1)\n", 825 | "\n", 826 | " # 只去最后一个单词的输出\n", 827 | " # 所以out的size会变为(200, 1)\n", 828 | " out = out[:,-1]\n", 829 | "\n", 830 | " # 将输出和本次的(h, c)返回\n", 831 | " return out, hidden\n", 832 | "\n", 833 | " def init_hidden(self, batch_size):\n", 834 | " \"\"\"\n", 835 | " 初始化隐状态：第一次送给LSTM时，没有隐状态，所以要初始化一个\n", 836 | " 这里的初始化策略是全部赋0。\n", 837 | " 这里之所以是tuple，是因为LSTM需要接受两个隐状态hidden state和cell state\n", 838 | " \"\"\"\n", 839 | " hidden = (torch.zeros(self.n_layers, batch_size, self.hidden_dim).to(device),\n", 840 | " torch.zeros(self.n_layers, batch_size, self.hidden_dim).to(device)\n", 841 | " )\n", 842 | " return hidden" 843 | ], 844 | "metadata": { 845 | "pycharm": { 846 | "name": "#%%\n" 847 | }, 848 | "id": "MvOKKm2a6_0t" 849 | } 850 | }, 851 | { 852 | "cell_type": "markdown", 853 | "source": [ 854 | "模型定义完毕，构建模型对象：" 855 | ], 856 | "metadata": { 857 | "collapsed": false, 858 | "pycharm": { 859 | "name": "#%% md\n" 860 | }, 861 | "id": "XMh1JBZL6_0t" 862 | } 863 | }, 864 | { 865 | "cell_type": "code", 866 | "execution_count": 25, 867 | "outputs": [ 868 | { 869 | "output_type": "execute_result", 870 | "data": { 871 | "text/plain": [ 872 | "SentimentNet(\n", 873 | " (embedding): Embedding(221604, 400)\n", 874 | " (lstm): LSTM(400, 512, num_layers=2, batch_first=True, dropout=0.5)\n", 875 | " (fc): Linear(in_features=512, out_features=1, bias=True)\n", 876 | " (sigmoid): Sigmoid()\n", 877 | " (dropout): Dropout(p=0.5, inplace=False)\n", 878 | ")" 879 | ] 880 | }, 881 | "metadata": {}, 882 | "execution_count": 25 883 | } 884 | ], 885 | "source": [ 886 | "model = SentimentNet(len(words))\n", 887 | "model.to(device)" 888 | ], 889 | "metadata": { 890 | "pycharm": { 891 | "name": "#%%\n" 892 | }, 893 | "id": "pqZmoWar6_0t", 894 | "outputId": "12839ec8-c4b3-4d86-c4b7-aff71dc7c066", 895 | "colab": { 896 | "base_uri": "https://localhost:8080/" 897 | } 898 | } 899 | }, 900 | { 901 | "cell_type": "markdown", 902 | "source": [ 903 | "接下来定义损失函数，由于是二分类问题，所以使用**交叉熵（Binary Cross Entropy，BCE）**：" 904 | ], 905 | "metadata": { 906 | "collapsed": false, 907 | "pycharm": { 908 | "name": "#%% md\n" 909 | }, 910 | "id": "sb3eb6bZ6_0t" 911 | } 912 | }, 913 | { 914 | "cell_type": "code", 915 | "execution_count": 26, 916 | "outputs": [], 917 | "source": [ 918 | "criterion = nn.BCELoss()" 919 | ], 920 | "metadata": { 921 | "pycharm": { 922 | "name": "#%%\n" 923 | }, 924 | "id": "Wg2L1kOQ6_0t" 925 | } 926 | }, 927 | { 928 | "cell_type": "markdown", 929 | "source": [ 930 | "优化器选用Adam优化器：" 931 | ], 932 | "metadata": { 933 | "collapsed": false, 934 | "pycharm": { 935 | "name": "#%% md\n" 936 | }, 937 | "id": "bvvd1SYa6_0t" 938 | } 939 | }, 940 | { 941 | "cell_type": "code", 942 | "execution_count": 27, 943 | "outputs": [], 944 | "source": [ 945 | "lr = 0.005\n", 946 | "optimizer = torch.optim.Adam(model.parameters(), lr=lr)" 947 | ], 948 | "metadata": { 949 | "pycharm": { 950 | "name": "#%%\n" 951 | }, 952 | "id": "_6P6YNYM6_0t" 953 | } 954 | }, 955 | { 956 | "cell_type": "markdown", 957 | "source": [ 958 | "接下来定义训练代码：" 959 | ], 960 | "metadata": { 961 | "collapsed": false, 962 | "pycharm": { 963 | "name": "#%% md\n" 964 | }, 965 | "id": "sY7N__vE6_0t" 966 | } 967 | }, 968 | { 969 | "cell_type": "code", 970 | "execution_count": 28, 971 | "outputs": [ 972 | { 973 | "output_type": "stream", 974 | "name": "stdout", 975 | "text": [ 976 | "Epoch: 1/2... Step: 1000... Loss: 0.268714...\n", 977 | "Epoch: 1/2... Step: 2000... Loss: 0.187919...\n", 978 | "Epoch: 1/2... Step: 3000... Loss: 0.215379...\n", 979 | "Epoch: 1/2... Step: 4000... Loss: 0.195820...\n", 980 | "Epoch: 2/2... Step: 5000... Loss: 0.130096...\n", 981 | "Epoch: 2/2... Step: 6000... Loss: 0.110538...\n", 982 | "Epoch: 2/2... Step: 7000... Loss: 0.198314...\n", 983 | "Epoch: 2/2... Step: 8000... Loss: 0.233867...\n" 984 | ] 985 | } 986 | ], 987 | "source": [ 988 | "epochs = 2 # 一共训练两轮\n", 989 | "counter = 0 # 用于记录训练次数\n", 990 | "print_every = 1000 # 每1000次打印一下当前状态\n", 991 | "\n", 992 | "for i in range(epochs):\n", 993 | " h = model.init_hidden(batch_size) # 初始化第一个Hidden_state\n", 994 | "\n", 995 | " for inputs, labels in train_loader: # 从train_loader中获取一组inputs和labels\n", 996 | " counter += 1 # 训练次数+1\n", 997 | "\n", 998 | " # 将上次输出的hidden_state转为tuple格式\n", 999 | " # 因为有两次，所以len(h)==2\n", 1000 | " h = tuple([e.data for e in h])\n", 1001 | "\n", 1002 | " # 将数据迁移到GPU\n", 1003 | " inputs, labels = inputs.to(device), labels.to(device)\n", 1004 | "\n", 1005 | " # 清空模型梯度\n", 1006 | " model.zero_grad()\n", 1007 | "\n", 1008 | " # 将本轮的输入和hidden_state送给模型，进行前向传播，\n", 1009 | " # 然后获取本次的输出和新的hidden_state\n", 1010 | " output, h = model(inputs, h)\n", 1011 | "\n", 1012 | " # 将预测值和真实值送给损失函数计算损失\n", 1013 | " loss = criterion(output, labels.float())\n", 1014 | "\n", 1015 | " # 进行反向传播\n", 1016 | " loss.backward()\n", 1017 | "\n", 1018 | " # 对模型进行裁剪，防止模型梯度爆炸\n", 1019 | " # 详情请参考：https://blog.csdn.net/zhaohongfei_358/article/details/122820992\n", 1020 | " nn.utils.clip_grad_norm_(model.parameters(), max_norm=5)\n", 1021 | "\n", 1022 | " # 更新权重\n", 1023 | " optimizer.step()\n", 1024 | "\n", 1025 | " # 隔一定次数打印一下当前状态\n", 1026 | " if counter%print_every == 0:\n", 1027 | " print(\"Epoch: {}/{}...\".format(i+1, epochs),\n", 1028 | " \"Step: {}...\".format(counter),\n", 1029 | " \"Loss: {:.6f}...\".format(loss.item()))" 1030 | ], 1031 | "metadata": { 1032 | "pycharm": { 1033 | "name": "#%%\n" 1034 | }, 1035 | "id": "Gzc82i5j6_0t", 1036 | "outputId": "3c74bd52-f2f9-4e76-84bc-f667605b86dd", 1037 | "colab": { 1038 | "base_uri": "https://localhost:8080/" 1039 | } 1040 | } 1041 | }, 1042 | { 1043 | "cell_type": "markdown", 1044 | "source": [ 1045 | "> 如果这里抛出了`RuntimeError: CUDA out of memory. Tried to allocate ...`异常，可以将batch_size调小，或者清空gpu中的缓存（`torch.cuda.empty_cache()`）" 1046 | ], 1047 | "metadata": { 1048 | "collapsed": false, 1049 | "pycharm": { 1050 | "name": "#%% md\n" 1051 | }, 1052 | "id": "bDVxnKpc6_0t" 1053 | } 1054 | }, 1055 | { 1056 | "cell_type": "markdown", 1057 | "source": [ 1058 | "经过一段时间的训练，现在来评估一下模型的性能：" 1059 | ], 1060 | "metadata": { 1061 | "collapsed": false, 1062 | "pycharm": { 1063 | "name": "#%% md\n" 1064 | }, 1065 | "id": "AZSIMcDN6_0t" 1066 | } 1067 | }, 1068 | { 1069 | "cell_type": "code", 1070 | "execution_count": 29, 1071 | "outputs": [ 1072 | { 1073 | "output_type": "stream", 1074 | "name": "stdout", 1075 | "text": [ 1076 | "Test loss: 0.202\n", 1077 | "Test accuracy: 92.487%\n" 1078 | ] 1079 | } 1080 | ], 1081 | "source": [ 1082 | "test_losses = [] # 记录测试数据集的损失\n", 1083 | "num_correct = 0 # 记录正确预测的数量\n", 1084 | "h = model.init_hidden(batch_size) # 初始化hidden_state和cell_state\n", 1085 | "model.eval() # 将模型调整为评估模式\n", 1086 | "\n", 1087 | "# 开始评估模型\n", 1088 | "for inputs, labels in test_loader:\n", 1089 | " h = tuple([each.data for each in h])\n", 1090 | " inputs, labels = inputs.to(device), labels.to(device)\n", 1091 | " output, h = model(inputs, h)\n", 1092 | " test_loss = criterion(output.squeeze(), labels.float())\n", 1093 | " test_losses.append(test_loss.item())\n", 1094 | " pred = torch.round(output.squeeze()) # 将模型四舍五入为0和1\n", 1095 | " correct_tensor = pred.eq(labels.float().view_as(pred)) # 计算预测正确的数据\n", 1096 | " correct = np.squeeze(correct_tensor.cpu().numpy())\n", 1097 | " num_correct += np.sum(correct)\n", 1098 | "\n", 1099 | "print(\"Test loss: {:.3f}\".format(np.mean(test_losses)))\n", 1100 | "test_acc = num_correct/len(test_loader.dataset)\n", 1101 | "print(\"Test accuracy: {:.3f}%\".format(test_acc*100))" 1102 | ], 1103 | "metadata": { 1104 | "pycharm": { 1105 | "name": "#%%\n" 1106 | }, 1107 | "id": "fa8y_0t86_0t", 1108 | "outputId": "4d613e01-71d7-4347-fcfc-d926143678ae", 1109 | "colab": { 1110 | "base_uri": "https://localhost:8080/" 1111 | } 1112 | } 1113 | }, 1114 | { 1115 | "cell_type": "markdown", 1116 | "source": [ 1117 | "最终，经过训练后，可以得到90%以上的准确率。\n", 1118 | "\n", 1119 | "我们来实际尝试一下，定义一个`predict(sentence)`函数，输入一个句子，输出其预测结果：" 1120 | ], 1121 | "metadata": { 1122 | "collapsed": false, 1123 | "pycharm": { 1124 | "name": "#%% md\n" 1125 | }, 1126 | "id": "lT3opySB6_0u" 1127 | } 1128 | }, 1129 | { 1130 | "cell_type": "code", 1131 | "execution_count": 30, 1132 | "outputs": [], 1133 | "source": [ 1134 | "def predict(sentence):\n", 1135 | " # 将句子分词后，转换为数字\n", 1136 | " sentences = [[word2idx[word.lower()] if word.lower() in word2idx else 0 for word in nltk.word_tokenize(sentence)]]\n", 1137 | "\n", 1138 | " # 将句子变为固定长度200\n", 1139 | " sentences = pad_input(sentences, 200)\n", 1140 | "\n", 1141 | " # 将数据移到GPU中\n", 1142 | " sentences = torch.Tensor(sentences).long().to(device)\n", 1143 | "\n", 1144 | " # 初始化隐状态\n", 1145 | " h = (torch.Tensor(2, 1, 512).zero_().to(device),\n", 1146 | " torch.Tensor(2, 1, 512).zero_().to(device))\n", 1147 | " h = tuple([each.data for each in h])\n", 1148 | "\n", 1149 | " # 预测\n", 1150 | " if model(sentences, h)[0] >= 0.5:\n", 1151 | " print(\"positive\")\n", 1152 | " else:\n", 1153 | " print(\"negative\")" 1154 | ], 1155 | "metadata": { 1156 | "pycharm": { 1157 | "name": "#%%\n" 1158 | }, 1159 | "id": "q56TaJZO6_0u" 1160 | } 1161 | }, 1162 | { 1163 | "cell_type": "code", 1164 | "execution_count": 31, 1165 | "outputs": [ 1166 | { 1167 | "output_type": "stream", 1168 | "name": "stdout", 1169 | "text": [ 1170 | "negative\n", 1171 | "negative\n" 1172 | ] 1173 | } 1174 | ], 1175 | "source": [ 1176 | "predict(\"The film is so boring\")\n", 1177 | "predict(\"The actor is too ugly.\")" 1178 | ], 1179 | "metadata": { 1180 | "pycharm": { 1181 | "name": "#%%\n" 1182 | }, 1183 | "id": "qil4MtPi6_0u", 1184 | "outputId": "4df7aab6-0d64-493f-ee03-374f30856f44", 1185 | "colab": { 1186 | "base_uri": "https://localhost:8080/" 1187 | } 1188 | } 1189 | }, 1190 | { 1191 | "cell_type": "markdown", 1192 | "source": [ 1193 | "# 参考资料\n", 1194 | "\n", 1195 | "[Long Short-Term Memory: From Zero to Hero with PyTorch](https://blog.floydhub.com/long-short-term-memory-from-zero-to-hero-with-pytorch/): https://blog.floydhub.com/long-short-term-memory-from-zero-to-hero-with-pytorch/" 1196 | ], 1197 | "metadata": { 1198 | "collapsed": false, 1199 | "pycharm": { 1200 | "name": "#%% md\n" 1201 | }, 1202 | "id": "1fQ0sKb26_0u" 1203 | } 1204 | }, 1205 | { 1206 | "cell_type": "code", 1207 | "execution_count": null, 1208 | "outputs": [], 1209 | "source": [ 1210 | "" 1211 | ], 1212 | "metadata": { 1213 | "pycharm": { 1214 | "name": "#%%\n" 1215 | }, 1216 | "id": "gL4V6ekz6_0u" 1217 | } 1218 | } 1219 | ], 1220 | "metadata": { 1221 | "kernelspec": { 1222 | "display_name": "Python 3", 1223 | "language": "python", 1224 | "name": "python3" 1225 | }, 1226 | "language_info": { 1227 | "codemirror_mode": { 1228 | "name": "ipython", 1229 | "version": 2 1230 | }, 1231 | "file_extension": ".py", 1232 | "mimetype": "text/x-python", 1233 | "name": "python", 1234 | "nbconvert_exporter": "python", 1235 | "pygments_lexer": "ipython2", 1236 | "version": "2.7.6" 1237 | }, 1238 | "colab": { 1239 | "provenance": [], 1240 | "machine_shape": "hm" 1241 | }, 1242 | "accelerator": "GPU", 1243 | "gpuClass": "standard" 1244 | }, 1245 | "nbformat": 4, 1246 | "nbformat_minor": 0 1247 | } -------------------------------------------------------------------------------- /101_mosaic_video.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "colab": { 6 | "provenance": [] 7 | }, 8 | "kernelspec": { 9 | "name": "python3", 10 | "display_name": "Python 3" 11 | }, 12 | "language_info": { 13 | "name": "python" 14 | } 15 | }, 16 | "cells": [ 17 | { 18 | "cell_type": "code", 19 | "source": [ 20 | "# Install libraries\n", 21 | "!pip install ultralytics\n", 22 | "!pip install ffmpeg-python" 23 | ], 24 | "metadata": { 25 | "id": "WJTaMsEd09Vt" 26 | }, 27 | "execution_count": null, 28 | "outputs": [] 29 | }, 30 | { 31 | "cell_type": "code", 32 | "source": [ 33 | "# Download model and video.\n", 34 | "!gdown 1qcr9DbgsX3ryrz2uU8w4Xm3cOrRywXqb\n", 35 | "!wget https://github.com/iioSnail/pytorch_deep_learning_examples/raw/refs/heads/main/asserts/mp4/kunkun.mp4" 36 | ], 37 | "metadata": { 38 | "id": "eXGr2Fl1Wpdc" 39 | }, 40 | "execution_count": null, 41 | "outputs": [] 42 | }, 43 | { 44 | "cell_type": "code", 45 | "execution_count": null, 46 | "metadata": { 47 | "id": "x3QIKG890rxR" 48 | }, 49 | "outputs": [], 50 | "source": [ 51 | "import ffmpeg\n", 52 | "import cv2\n", 53 | "from numpy import ndarray\n", 54 | "from ultralytics import YOLO\n", 55 | "from tqdm import tqdm" 56 | ] 57 | }, 58 | { 59 | "cell_type": "code", 60 | "source": [ 61 | "# Apply mosaic to an image\n", 62 | "def mosaic_image(model, image:ndarray, mosaic_scale = 10) -> ndarray:\n", 63 | " results = model(image, verbose=False)\n", 64 | " results[0].boxes\n", 65 | "\n", 66 | " boxes = results[0].boxes.xyxy\n", 67 | " for i in range(len(boxes)):\n", 68 | " x1, y1, x2, y2 = boxes[i].int()\n", 69 | " roi = image[y1:y2, x1:x2]\n", 70 | "\n", 71 | " h, w = roi.shape[:2]\n", 72 | " small_roi = cv2.resize(roi, (w // mosaic_scale, h // mosaic_scale), interpolation=cv2.INTER_LINEAR)\n", 73 | " mosaic_roi = cv2.resize(small_roi, (w, h), interpolation=cv2.INTER_NEAREST)\n", 74 | " image[y1:y2, x1:x2] = mosaic_roi\n", 75 | "\n", 76 | " return image" 77 | ], 78 | "metadata": { 79 | "id": "ys1gOBSBWY6C" 80 | }, 81 | "execution_count": 4, 82 | "outputs": [] 83 | }, 84 | { 85 | "cell_type": "code", 86 | "source": [ 87 | "# Define filepaths.\n", 88 | "input_video = \"kunkun.mp4\"\n", 89 | "tmp_audio = \"tmp.wav\"\n", 90 | "tmp_video = \"tmp_kunkun.mp4\"\n", 91 | "output_video = \"mosaic_kunkun.mp4\"\n", 92 | "\n", 93 | "model = YOLO(\"yolov8n-face.pt\")" 94 | ], 95 | "metadata": { 96 | "id": "js-8koFq04mb" 97 | }, 98 | "execution_count": 5, 99 | "outputs": [] 100 | }, 101 | { 102 | "cell_type": "code", 103 | "source": [ 104 | "# Extract audio from the video.\n", 105 | "ffmpeg.input(input_video).output(tmp_audio, format='wav').run(overwrite_output=True)" 106 | ], 107 | "metadata": { 108 | "colab": { 109 | "base_uri": "https://localhost:8080/" 110 | }, 111 | "id": "Q270pQcA1JiA", 112 | "outputId": "9669c593-23b2-437b-9701-47845588cca9" 113 | }, 114 | "execution_count": 6, 115 | "outputs": [ 116 | { 117 | "output_type": "execute_result", 118 | "data": { 119 | "text/plain": [ 120 | "(None, None)" 121 | ] 122 | }, 123 | "metadata": {}, 124 | "execution_count": 6 125 | } 126 | ] 127 | }, 128 | { 129 | "cell_type": "code", 130 | "source": [ 131 | "# Play mosaic frame by frame and generate the output video with mosaic.\n", 132 | "cap = cv2.VideoCapture(input_video)\n", 133 | "if not cap.isOpened():\n", 134 | " print(\"Error: Could not open video file.\")\n", 135 | " exit(0)\n", 136 | "\n", 137 | "width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))\n", 138 | "height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))\n", 139 | "fps = cap.get(cv2.CAP_PROP_FPS)\n", 140 | "n_frames = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))\n", 141 | "\n", 142 | "fourcc = cv2.VideoWriter_fourcc(*'mp4v')\n", 143 | "out = cv2.VideoWriter(tmp_video, fourcc, fps, (width, height))\n", 144 | "\n", 145 | "pro_bar = tqdm(total=n_frames)\n", 146 | "while True:\n", 147 | " ret, frame = cap.read()\n", 148 | "\n", 149 | " if not ret:\n", 150 | " break\n", 151 | "\n", 152 | " frame = mosaic_image(model, frame)\n", 153 | " out.write(frame)\n", 154 | "\n", 155 | " pro_bar.update(1)\n", 156 | "\n", 157 | "cap.release()\n", 158 | "out.release()\n", 159 | "pro_bar.close()" 160 | ], 161 | "metadata": { 162 | "id": "WkKbG2uw1mnJ" 163 | }, 164 | "execution_count": null, 165 | "outputs": [] 166 | }, 167 | { 168 | "cell_type": "code", 169 | "source": [ 170 | "# Merge the video and audio.\n", 171 | "video_stream = ffmpeg.input(tmp_video)\n", 172 | "audio_stream = ffmpeg.input(tmp_audio)\n", 173 | "ffmpeg.output(video_stream, audio_stream, output_video, vcodec=\"copy\", acodec='aac').run(overwrite_output=True)" 174 | ], 175 | "metadata": { 176 | "id": "GiYXjgvQ1p8F" 177 | }, 178 | "execution_count": null, 179 | "outputs": [] 180 | }, 181 | { 182 | "cell_type": "code", 183 | "source": [ 184 | "# Show the result video.\n", 185 | "from IPython.display import HTML\n", 186 | "from base64 import b64encode\n", 187 | "import os\n", 188 | "\n", 189 | "# Compressed video path\n", 190 | "compressed_path = \"./compressed.mp4\"\n", 191 | "os.system(f\"ffmpeg -i {output_video} -vcodec libx264 {compressed_path}\")\n", 192 | "\n", 193 | "# Show video\n", 194 | "mp4 = open(compressed_path,'rb').read()\n", 195 | "data_url = \"data:video/mp4;base64,\" + b64encode(mp4).decode()\n", 196 | "HTML(\"\"\"\n", 197 | "\n", 200 | "\"\"\" % data_url)" 201 | ], 202 | "metadata": { 203 | "id": "AabemWblYA50" 204 | }, 205 | "execution_count": null, 206 | "outputs": [] 207 | } 208 | ] 209 | } -------------------------------------------------------------------------------- /README.en.md: -------------------------------------------------------------------------------- 1 |

中文 | English

2 | 3 | # Python Deep Learning Examples 4 | 5 | During my journey of learning deep learning, I encountered numerous challenges. For me, the two biggest obstacles were: 6 | 7 | 1. Most tutorials focus heavily on theory, but without hands-on practice, it’s hard to truly grasp the concepts. 8 | 2. Practical projects on GitHub are often too comprehensive, including many aspects unrelated to understanding theory, such as parallel computing and model optimization. Furthermore, many projects are provided as `.py` files instead of Jupyter notebooks, which makes it even harder for beginners like me to read and understand. 9 | 10 | To address these two issues, I built this project with a focus on using the simplest possible code to demonstrate deep learning concepts in practice. The goal is to help others understand the theory through hands-on experience. 11 | 12 | # Project Structure 13 | 14 | | Series Name | Project Title | Blog Links | Google Colab | 15 | |--|--|--|--| 16 | | PyTorch Beginner's Tutorial | 01. Implementing Linear Regression | [Blogger](https://iiosnail.blogspot.com/2024/11/pytorch-en-01.html) | [Open In Colab](https://colab.research.google.com/github/iioSnail/pytorch_deep_learning_examples/blob/main/01_linear_regression.ipynb) | 17 | || 02. Using a BP Neural Network to Recognize MNIST Handwritten Digits | [Blogger](https://iiosnail.blogspot.com/2024/11/pytorch-en-02.html) | [Open In Colab](https://colab.research.google.com/github/iioSnail/pytorch_deep_learning_examples/blob/main/02_MNIST_classification.ipynb) | 18 | || 03. Object Classification with a Simple CNN | [Blogger](https://iiosnail.blogspot.com/2024/11/pytorch-en-03.html) | [Open In Colab](https://colab.research.google.com/github/iioSnail/pytorch_deep_learning_examples/blob/main/03_cnn_image_classification.ipynb) | 19 | || 04. Sentiment Analysis of Text Using LSTM | [Blogger](https://iiosnail.blogspot.com/2024/11/pytorch-en-04.html) | [Open In Colab](https://colab.research.google.com/github/iioSnail/pytorch_deep_learning_examples/blob/main/04_LSTM_sentiment_analysis.ipynb) | 20 | || 05. Machine Translation Using nn.Transformer (English to Chinese) | [Blogger](https://iiosnail.blogspot.com/2024/11/pytorch-en-05.html) | [Open In Colab](https://github.com/iioSnail/chaotic-transformer-tutorials/blob/master/en_to_zh_demo.ipynb) | 21 | || 06. Using GAN to Generate Simple Anime Character Avatar | [Blogger](https://iiosnail.blogspot.com/2024/11/pytorch-en-06.html) | [Open In Colab](https://colab.research.google.com/github/iioSnail/pytorch_deep_learning_examples/blob/main/06_GAN_image_generation.ipynb) | 22 | || 07. Textual Metaphor Binary Classification with BERT | [Blogger](https://iiosnail.blogspot.com/2024/11/pytorch-en-07.html) | [Open In Colab](https://github.com/iioSnail/chaotic-transformer-tutorials/blob/master/bert_classification_demo.ipynb) | 23 | || 08. Few-shot Learning for Image Classification | [Blogger](https://iiosnail.blogspot.com/2024/11/pytorch-en-08.html) | [Open In Colab](https://colab.research.google.com/github/iioSnail/pytorch_deep_learning_examples/blob/main/08_few_shot_learning.ipynb) | 24 | | PyTorch Applications | 01. Video Face Mosaic Processing with YOLO | [Blogger](https://iiosnail.blogspot.com/2024/12/mosaic-en.html) | [Open In Colab](https://colab.research.google.com/github/iioSnail/pytorch_deep_learning_examples/blob/main/101_mosaic_video.ipynb) | 25 | || 02. Generate Lip-syncing Video with Wav2Lip | TODO | [Open In Colab](https://colab.research.google.com/github/iioSnail/pytorch_deep_learning_examples/blob/main/102_Wav2Lip_Inference.ipynb) | 26 | | LLM Applications | 01. A Native RAG Example with LangChain | [Blogger](https://iiosnail.blogspot.com/2025/04/native-rag.html) | [Open In Colab](https://colab.research.google.com/github/iioSnail/pytorch_deep_learning_examples/blob/main/201_native_rag.ipynb) | 27 | 28 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 |

中文 | English

2 | 3 | # Python项目实战 4 | 5 | 在学习深度学习的过程中有许多坎坷，对我来说，最大的困难有两点： 6 | 7 | 1. 大多的教学都是偏向理论，但如果不结合实践，很难吃透理论。 8 | 2. Github上的实战项目都是大而全的，里面包含了许多与理解理论无关的东西，例如并行计算、模型优化等等，而且很多都是py文件，并非jupyter，这样对于我这种菜鸟来说，增加了阅读难度。 9 | 10 | 针对这两个痛点，我构建了这个项目，致力于使用最简单的代码，来进行深度学习实战，帮助大家理解理论。 11 | 12 | # 项目目录 13 | 14 | | 系列名称 | 项目名称 | 博客地址 | Google Colab | 15 | |--|--|--|--| 16 | | Pytorch入门实战 | 01. 实现线性回归 | [CSDN](https://blog.csdn.net/zhaohongfei_358/article/details/121418622), [Blogger](https://iiosnail.blogspot.com/2024/10/pytorch1.html) | [Open In Colab](https://colab.research.google.com/github/iioSnail/pytorch_deep_learning_examples/blob/main/01_linear_regression.ipynb) | 17 | || 02. 使用BP神经网络实现MNIST手写数字识别 | [CSDN](https://blog.csdn.net/zhaohongfei_358/article/details/122800647), [Blogger](https://iiosnail.blogspot.com/2024/10/pytorch2.html) | [Open In Colab](https://colab.research.google.com/github/iioSnail/pytorch_deep_learning_examples/blob/main/02_MNIST_classification.ipynb) | 18 | || 03. 使用简单CNN实现物体分类 | [CSDN](https://blog.csdn.net/zhaohongfei_358/article/details/125020186), [Blogger](https://iiosnail.blogspot.com/2024/10/pytorch3.html) | [Open In Colab](https://colab.research.google.com/github/iioSnail/pytorch_deep_learning_examples/blob/main/03_cnn_image_classification.ipynb) | 19 | || 04. 基于LSTM实现文本的情感分析 | [CSDN](https://blog.csdn.net/zhaohongfei_358/article/details/122838743), [Blogger](https://iiosnail.blogspot.com/2024/10/pytorch4.html) | [Open In Colab](https://colab.research.google.com/github/iioSnail/pytorch_deep_learning_examples/blob/main/04_LSTM_sentiment_analysis.ipynb) | 20 | || 05. 基于nn.Transformer实现机器翻译（英译汉） | [CSDN](https://blog.csdn.net/zhaohongfei_358/article/details/126175328), [Blogger](https://iiosnail.blogspot.com/2024/10/pytorch5.html) | [Open In Colab](https://github.com/iioSnail/chaotic-transformer-tutorials/blob/master/en_to_zh_demo.ipynb) | 21 | || 06. 基于GAN生成简单的动漫人物头像 | [CSDN](https://blog.csdn.net/zhaohongfei_358/article/details/125675557), [Blogger](https://iiosnail.blogspot.com/2024/10/pytorch6.html) | [Open In Colab](https://colab.research.google.com/github/iioSnail/pytorch_deep_learning_examples/blob/main/06_GAN_image_generation.ipynb) | 22 | || 07. 基于BERT实现文本隐喻二分类（Kaggle入门题目） | [CSDN](https://blog.csdn.net/zhaohongfei_358/article/details/126426855), [Blogger](https://iiosnail.blogspot.com/2024/10/pytorch7.html) | [Open In Colab](https://github.com/iioSnail/chaotic-transformer-tutorials/blob/master/bert_classification_demo.ipynb) | 23 | || 08.小样本学习(Few-shot Learning)实现图片分类 | [CSDN](https://blog.csdn.net/zhaohongfei_358/article/details/126453857), [Blogger](https://iiosnail.blogspot.com/2024/10/pytorch8.html) | [Open In Colab](https://colab.research.google.com/github/iioSnail/pytorch_deep_learning_examples/blob/main/08_few_shot_learning.ipynb) | 24 | | Pytorch应用实战 | 01. 基于YOLO的视频人脸马赛克处理 | [CSDN](https://iio-snail.blog.csdn.net/article/details/144245634), [Blogger](https://iiosnail.blogspot.com/2024/12/mosaic.html) | [Open In Colab](https://colab.research.google.com/github/iioSnail/pytorch_deep_learning_examples/blob/main/101_mosaic_video.ipynb) | 25 | || 02. 基于Wav2Lip的对嘴型（Lip-syncing）视频生成 | [CSDN](https://iio-snail.blog.csdn.net/article/details/146425716) | [Open In Colab](https://colab.research.google.com/github/iioSnail/pytorch_deep_learning_examples/blob/main/102_Wav2Lip_Inference.ipynb) | 26 | | LLM应用实战 | 01. 基于LangChain的Native RAG简单样例 | [CSDN](https://iio-snail.blog.csdn.net/article/details/147148936), [Blogger](https://iiosnail.blogspot.com/2025/04/native-rag.html) | [Open In Colab](https://colab.research.google.com/github/iioSnail/pytorch_deep_learning_examples/blob/main/201_native_rag.ipynb) | 27 | -------------------------------------------------------------------------------- /asserts/images/01_01.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/iioSnail/pytorch_deep_learning_examples/d6ed972dcf00e7a38296e46ef2d295002f61751b/asserts/images/01_01.png -------------------------------------------------------------------------------- /asserts/images/01_02.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/iioSnail/pytorch_deep_learning_examples/d6ed972dcf00e7a38296e46ef2d295002f61751b/asserts/images/01_02.png -------------------------------------------------------------------------------- /asserts/images/06_01.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/iioSnail/pytorch_deep_learning_examples/d6ed972dcf00e7a38296e46ef2d295002f61751b/asserts/images/06_01.png -------------------------------------------------------------------------------- /asserts/images/101_01.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/iioSnail/pytorch_deep_learning_examples/d6ed972dcf00e7a38296e46ef2d295002f61751b/asserts/images/101_01.png -------------------------------------------------------------------------------- /asserts/images/101_02.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/iioSnail/pytorch_deep_learning_examples/d6ed972dcf00e7a38296e46ef2d295002f61751b/asserts/images/101_02.png -------------------------------------------------------------------------------- /asserts/images/101_03.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/iioSnail/pytorch_deep_learning_examples/d6ed972dcf00e7a38296e46ef2d295002f61751b/asserts/images/101_03.png -------------------------------------------------------------------------------- /asserts/images/101_04.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/iioSnail/pytorch_deep_learning_examples/d6ed972dcf00e7a38296e46ef2d295002f61751b/asserts/images/101_04.png -------------------------------------------------------------------------------- /asserts/images/101_05.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/iioSnail/pytorch_deep_learning_examples/d6ed972dcf00e7a38296e46ef2d295002f61751b/asserts/images/101_05.png -------------------------------------------------------------------------------- /asserts/images/101_06.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/iioSnail/pytorch_deep_learning_examples/d6ed972dcf00e7a38296e46ef2d295002f61751b/asserts/images/101_06.png -------------------------------------------------------------------------------- /asserts/mp4/kunkun.mp4: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/iioSnail/pytorch_deep_learning_examples/d6ed972dcf00e7a38296e46ef2d295002f61751b/asserts/mp4/kunkun.mp4 --------------------------------------------------------------------------------