├── .gitignore ├── AI从入门到放弃：《BP神经网络算法推导及代码实现笔记》 ├── AI从入门到放弃：BP神经网络算法推导及代码实现笔记.html ├── AI从入门到放弃：BP神经网络算法推导及代码实现笔记.pdf ├── AI从入门到放弃：BP神经网络算法推导及代码实现笔记.png ├── NeuralNetwork.py └── resources │ ├── 10DA03D78180BD0686C0CB9E691BF459.png │ ├── 32F79E55C7B0FF938AA82D2D69E3F97B.png │ ├── 41D98819C2BD4A40EF5FEC7B2221FE29.png │ ├── 43A0B27F2362DA18F8AEF4E007D6D2DD.png │ ├── 46B66C214BF95AC6EADC4F4120729397.jpg │ ├── 49839165F7B45F38E876D865D80D8E04.jpg │ ├── 5D2344074BC281BA3B5582A65BCCC4B5.jpg │ ├── 62990B82CC910688B813103D80FCB509.jpg │ ├── 6EC3CD5AC3B342C739D267519B2BEA57.png │ ├── 837AA91C395EAE98C77FE7B75E4A6100.jpg │ ├── 89D527539F0A7055D369F32C68B1F981.png │ ├── 8B74CF49BA388F18402444C8A7AAAC4F.png │ ├── AF2E4DBD05E044A145DBA1E169E7594C.jpg │ ├── D53615C119E3112D9200732F7DDC5643.png │ ├── E0D33265A98E034A16574FA99388225A.png │ └── E68468BFF0B3210F75F5293104007E66.jpg ├── AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》 ├── AI从入门到放弃：CNN的导火索，用MLP做图像分类识别？--html │ ├── AI从入门到放弃：CNN的导火索，用MLP做图像分类识别？.html │ └── resources │ │ ├── 18911F9E66B3C88DEC4109AA4E8ED5FB.png │ │ ├── 31E1ECF3F19D72C84CA3DA925C0CB9DD.png │ │ ├── 5352CC63BE1E522FF056853AA1A26253.png │ │ ├── 630243EC928C70E4417306DFBB39A9D1.jpg │ │ ├── A3E4523C6F2874F6555CD9A1EAEBE5B0.png │ │ ├── BCDD7CADA14ADFC183B9AFBCCB0B8A1C.png │ │ ├── C557C61880CBC43A9C4E776802CC06D0.jpg │ │ └── CBCBF0C20D17CDDAC3EE968D43FC0B54.png ├── AI从入门到放弃：CNN的导火索，用MLP做图像分类识别？.pdf ├── cat_pics_my.png ├── cat_pics_my_predict.png ├── cat_pics_test.png ├── cat_pics_test_predict.png ├── cat_pics_train.png ├── cat_pics_train_predict.png ├── datasets │ ├── my_cat_misu.h5 │ ├── test_catvnoncat.h5 │ └── train_catvnoncat.h5 ├── misu.jpg └── picture_classification.py ├── AI从入门到放弃：《手撕神经卷积网络（CNN）》 ├── AI从入门到放弃：手撕卷积神经网络（CNN）.pdf ├── CNN-Network │ ├── .idea │ │ ├── CNN-Network.iml │ │ ├── inspectionProfiles │ │ │ └── Project_Default.xml │ │ ├── misc.xml │ │ ├── modules.xml │ │ └── workspace.xml │ ├── Data │ │ └── mnist │ │ │ ├── get_mnist.sh │ │ │ ├── t10k-images-idx3-ubyte │ │ │ ├── t10k-labels-idx1-ubyte │ │ │ ├── train-images-idx3-ubyte │ │ │ └── train-labels-idx1-ubyte │ ├── Layers │ │ ├── Convolution.py │ │ ├── Flatten.py │ │ ├── FullConnection.py │ │ ├── MnistInput.py │ │ ├── NetworkBuilder.py │ │ ├── Pooling.py │ │ ├── Relu.py │ │ ├── SoftmaxLoss.py │ │ └── __init__.py │ ├── Model │ │ └── model.h5 │ ├── OutputImages │ │ ├── conv1_output.png │ │ ├── conv2_output.png │ │ ├── fc1_output.png │ │ ├── fc2_output.png │ │ ├── flatten_output.png │ │ ├── mnist_data_output.png │ │ ├── pool1_output.png │ │ ├── pool2_output.png │ │ ├── relu1_output.png │ │ ├── softmax_loss_output.png │ │ └── train_pics.png │ └── main.py ├── Html版本 │ ├── AI从入门到放弃：手撕卷积神经网络（CNN）.html │ └── resources │ │ ├── 04C9E0B3DDFFAE6DBF0692451F6992D1.gif │ │ ├── 08E51AD720981A718F79635F3C175469.jpg │ │ ├── 173BE7B5673987828414CBAB75111379.jpg │ │ ├── 1E97985A1B2BB3FE1B5C98D77AC5A403.png │ │ ├── 24170F1D134952DFD522B4CADC1FFD8F.jpg │ │ ├── 29CED44A9D37FEC25A2448EE9B041256.jpg │ │ ├── 2B3B1E8B6725BE2CBEDC4466384042A7.jpg │ │ ├── 2FA771C954BE667E95561579518D46F0.jpg │ │ ├── 3BA9904D9A93E639A6577E317A295DD0.jpg │ │ ├── 45C1395B84E554772D1AFB28C80EE7C9.jpg │ │ ├── 49FDC432141B5F7D59B17F93E594A777.jpg │ │ ├── 53081124276177707263BF45B962832D.jpg │ │ ├── 705305FEE5A050575544C64067405FCE.gif │ │ ├── 7CED4CBD723175E6811557CA99B83837.jpg │ │ ├── 85182DA6962E2C26664D03DC1647AB31.jpg │ │ ├── 8645FA96B78267F6B995D7909C263CE4.gif │ │ ├── 8D72F4E6CDD66BC654D440390E42F922.jpg │ │ ├── 965FBFC2D682036ED178B7DD1EDD50FA.gif │ │ ├── 9EEC442D8B126FD3F3B53E19704201A9.jpg │ │ ├── A5BFC0E4FDD254F431C6B702CFB6E463.jpg │ │ ├── ACB9841BC14C57C09CA27F6ED6BBEC5C.jpg │ │ ├── B9663FCC0A0B42FA0E7082FFC1BFF9E3.jpg │ │ ├── BAAF3CFE34D7CFE2F69F7F8AE0D42735.jpg │ │ ├── BCBCEFAF634DEA7AEA8B53898F1BA866.jpg │ │ ├── CA203A8C36C14AE9C2DC0870C1ECE8E6.jpg │ │ ├── CB3DF2830392E247AA78C4F33230483C.jpg │ │ ├── D6893085EC1FE4A0AD20F68589BE1768.gif │ │ ├── DC6921D73DAB14ED98A26032A73F0AA9.gif │ │ ├── E351E2E6EDC5210A4C60E2E09DC1C43C.jpg │ │ ├── E5189072F39325215235950E90F6F9EF.jpg │ │ ├── E61DF3C6A20A5F78F43CE5789F7FD9FF.jpg │ │ ├── EBD4E3242A6B9AB67BE41F808DD2FD8E.jpg │ │ ├── FA8B0532561AD9A74CAAF78BD7175437.jpg │ │ ├── FB2E612B54BB961753B87FDCBC9D1E68.jpg │ │ ├── FD3AADAD7C3BF1F8C190E84BCAB47DF9.jpg │ │ ├── FDE7CEEA3FB146BDDCA545DB6BD3B6CC.jpg │ │ ├── FE0DB3B95EB7135EB37A8EE627A6FE08.jpg │ │ └── FFE5AA9D3CA5F3A1248F531F72E20B1A.jpg └── Markdown版本 │ ├── AI从入门到放弃：手撕卷积神经网络（CNN）.md │ └── resources │ ├── 04C9E0B3DDFFAE6DBF0692451F6992D1.gif │ ├── 08E51AD720981A718F79635F3C175469.jpg │ ├── 173BE7B5673987828414CBAB75111379.jpg │ ├── 1E97985A1B2BB3FE1B5C98D77AC5A403.png │ ├── 24170F1D134952DFD522B4CADC1FFD8F.jpg │ ├── 29CED44A9D37FEC25A2448EE9B041256.jpg │ ├── 2B3B1E8B6725BE2CBEDC4466384042A7.jpg │ ├── 2FA771C954BE667E95561579518D46F0.jpg │ ├── 3BA9904D9A93E639A6577E317A295DD0.jpg │ ├── 45C1395B84E554772D1AFB28C80EE7C9.jpg │ ├── 49FDC432141B5F7D59B17F93E594A777.jpg │ ├── 53081124276177707263BF45B962832D.jpg │ ├── 705305FEE5A050575544C64067405FCE.gif │ ├── 7CED4CBD723175E6811557CA99B83837.jpg │ ├── 85182DA6962E2C26664D03DC1647AB31.jpg │ ├── 8645FA96B78267F6B995D7909C263CE4.gif │ ├── 8D72F4E6CDD66BC654D440390E42F922.jpg │ ├── 965FBFC2D682036ED178B7DD1EDD50FA.gif │ ├── 9EEC442D8B126FD3F3B53E19704201A9.jpg │ ├── A5BFC0E4FDD254F431C6B702CFB6E463.jpg │ ├── ACB9841BC14C57C09CA27F6ED6BBEC5C.jpg │ ├── B9663FCC0A0B42FA0E7082FFC1BFF9E3.jpg │ ├── BAAF3CFE34D7CFE2F69F7F8AE0D42735.jpg │ ├── BCBCEFAF634DEA7AEA8B53898F1BA866.jpg │ ├── CA203A8C36C14AE9C2DC0870C1ECE8E6.jpg │ ├── CB3DF2830392E247AA78C4F33230483C.jpg │ ├── D6893085EC1FE4A0AD20F68589BE1768.gif │ ├── DC6921D73DAB14ED98A26032A73F0AA9.gif │ ├── E351E2E6EDC5210A4C60E2E09DC1C43C.jpg │ ├── E5189072F39325215235950E90F6F9EF.jpg │ ├── E61DF3C6A20A5F78F43CE5789F7FD9FF.jpg │ ├── EBD4E3242A6B9AB67BE41F808DD2FD8E.jpg │ ├── FA8B0532561AD9A74CAAF78BD7175437.jpg │ ├── FB2E612B54BB961753B87FDCBC9D1E68.jpg │ ├── FD3AADAD7C3BF1F8C190E84BCAB47DF9.jpg │ ├── FDE7CEEA3FB146BDDCA545DB6BD3B6CC.jpg │ ├── FE0DB3B95EB7135EB37A8EE627A6FE08.jpg │ └── FFE5AA9D3CA5F3A1248F531F72E20B1A.jpg └── README.md /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # C extensions 7 | *.so 8 | 9 | # Distribution / packaging 10 | .Python 11 | build/ 12 | develop-eggs/ 13 | dist/ 14 | downloads/ 15 | eggs/ 16 | .eggs/ 17 | lib/ 18 | lib64/ 19 | parts/ 20 | sdist/ 21 | var/ 22 | wheels/ 23 | *.egg-info/ 24 | .installed.cfg 25 | *.egg 26 | MANIFEST 27 | 28 | # PyInstaller 29 | # Usually these files are written by a python script from a template 30 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 31 | *.manifest 32 | *.spec 33 | 34 | # Installer logs 35 | pip-log.txt 36 | pip-delete-this-directory.txt 37 | 38 | # Unit test / coverage reports 39 | htmlcov/ 40 | .tox/ 41 | .coverage 42 | .coverage.* 43 | .cache 44 | nosetests.xml 45 | coverage.xml 46 | *.cover 47 | .hypothesis/ 48 | .pytest_cache/ 49 | 50 | # Translations 51 | *.mo 52 | *.pot 53 | 54 | # Django stuff: 55 | *.log 56 | local_settings.py 57 | db.sqlite3 58 | 59 | # Flask stuff: 60 | instance/ 61 | .webassets-cache 62 | 63 | # Scrapy stuff: 64 | .scrapy 65 | 66 | # Sphinx documentation 67 | docs/_build/ 68 | 69 | # PyBuilder 70 | target/ 71 | 72 | # Jupyter Notebook 73 | .ipynb_checkpoints 74 | 75 | # pyenv 76 | .python-version 77 | 78 | # celery beat schedule file 79 | celerybeat-schedule 80 | 81 | # SageMath parsed files 82 | *.sage.py 83 | 84 | # Environments 85 | .env 86 | .venv 87 | env/ 88 | venv/ 89 | ENV/ 90 | env.bak/ 91 | venv.bak/ 92 | 93 | # Spyder project settings 94 | .spyderproject 95 | .spyproject 96 | 97 | # Rope project settings 98 | .ropeproject 99 | 100 | # mkdocs documentation 101 | /site 102 | 103 | # mypy 104 | .mypy_cache/ 105 | -------------------------------------------------------------------------------- /AI从入门到放弃：《BP神经网络算法推导及代码实现笔记》/AI从入门到放弃：BP神经网络算法推导及代码实现笔记.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zsysuper/AI_Notes/3f321cf266f2fb270f02c6b440a27843b43d5a18/AI从入门到放弃：《BP神经网络算法推导及代码实现笔记》/AI从入门到放弃：BP神经网络算法推导及代码实现笔记.pdf -------------------------------------------------------------------------------- /AI从入门到放弃：《BP神经网络算法推导及代码实现笔记》/AI从入门到放弃：BP神经网络算法推导及代码实现笔记.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zsysuper/AI_Notes/3f321cf266f2fb270f02c6b440a27843b43d5a18/AI从入门到放弃：《BP神经网络算法推导及代码实现笔记》/AI从入门到放弃：BP神经网络算法推导及代码实现笔记.png -------------------------------------------------------------------------------- /AI从入门到放弃：《BP神经网络算法推导及代码实现笔记》/NeuralNetwork.py: -------------------------------------------------------------------------------- 1 | #coding:utf-8 2 | import sklearn.datasets 3 | import sklearn.linear_model 4 | import matplotlib 5 | import matplotlib.font_manager as fm 6 | import matplotlib.pyplot as plt 7 | import numpy as np 8 | 9 | np.random.seed(1) 10 | 11 | font = fm.FontProperties(fname='/System/Library/Fonts/STHeiti Light.ttc') 12 | matplotlib.rcParams['figure.figsize'] = (10.0, 8.0) 13 | 14 | def sigmoid(input_sum): 15 | """ 16 | 函数： 17 | 激活函数Sigmoid 18 | 输入： 19 | input_sum: 输入，即神经元的加权和 20 | 返回： 21 | output: 激活后的输出 22 | input_sum: 把输入缓存起来返回 23 | """ 24 | output = 1.0/(1+np.exp(-input_sum)) 25 | return output, input_sum 26 | 27 | 28 | def sigmoid_back_propagation(derror_wrt_output, input_sum): 29 | """ 30 | 函数： 31 | 误差关于神经元输入的偏导: dE／dIn = dE/dOut * dOut/dIn 参照式（5.6） 32 | 其中： dOut/dIn 就是激活函数的导数 dy=y(1 - y)，见式（5.9） 33 | dE/dOut 误差对神经元输出的偏导，见式（5.8） 34 | 输入： 35 | derror_wrt_output：误差关于神经元输出的偏导: dE/dyⱼ = 1/2(d(expect_to_output - output)**2/doutput) = -(expect_to_output - output) 36 | input_sum: 输入加权和 37 | 返回： 38 | derror_wrt_dinputs: 误差关于输入的偏导，见式（5.13） 39 | """ 40 | output = 1.0/(1 + np.exp(- input_sum)) 41 | doutput_wrt_dinput = output * (1 - output) 42 | derror_wrt_dinput = derror_wrt_output * doutput_wrt_dinput 43 | 44 | return derror_wrt_dinput 45 | 46 | 47 | def relu(input_sum): 48 | """ 49 | 函数： 50 | 激活函数ReLU 51 | 输入： 52 | input_sum: 输入，即神经元的加权和 53 | 返回： 54 | outputs: 激活后的输出 55 | input_sum: 把输入缓存起来返回 56 | """ 57 | output = np.maximum(0, input_sum) 58 | return output, input_sum 59 | 60 | 61 | def relu_back_propagation(derror_wrt_output, input_sum): 62 | """ 63 | 函数： 64 | 误差关于神经元输入的偏导: dE／dIn = dE/dOut * dOut/dIn 65 | 其中： dOut/dIn 就是激活函数的导数 66 | dE/dOut 误差对神经元输出的偏导 67 | 输入： 68 | derror_wrt_output：误差关于神经元输出的偏导 69 | input_sum: 输入加权和 70 | 返回： 71 | derror_wrt_dinputs: 误差关于输入的偏导 72 | """ 73 | derror_wrt_dinputs = np.array(derror_wrt_output, copy=True) 74 | derror_wrt_dinputs[input_sum <= 0] = 0 75 | 76 | return derror_wrt_dinputs 77 | 78 | 79 | def tanh(input_sum): 80 | """ 81 | 函数： 82 | 激活函数 tanh 83 | 输入： 84 | input_sum: 输入，即神经元的加权和 85 | 返回： 86 | output: 激活后的输出 87 | input_sum: 把输入缓存起来返回 88 | """ 89 | output = np.tanh(input_sum) 90 | return output, input_sum 91 | 92 | 93 | def tanh_back_propagation(derror_wrt_output, input_sum): 94 | """ 95 | 函数： 96 | 误差关于神经元输入的偏导: dE／dIn = dE/dOut * dOut/dIn 97 | 其中： dOut/dIn 就是激活函数的导数 tanh'(x) = 1 - x² 98 | dE/dOut 误差对神经元输出的偏导 99 | 输入： 100 | derror_wrt_output：误差关于神经元输出的偏导: dE/dyⱼ = 1/2(d(expect_to_output - output)**2/doutput) = -(expect_to_output - output) 101 | input_sum: 输入加权和 102 | 返回： 103 | derror_wrt_dinputs: 误差关于输入的偏导 104 | """ 105 | output = np.tanh(input_sum) 106 | doutput_wrt_dinput = 1 - np.power(output, 2) 107 | derror_wrt_dinput = derror_wrt_output * doutput_wrt_dinput 108 | 109 | return derror_wrt_dinput 110 | 111 | 112 | def activated(activation_choose, input): 113 | """把正向激活包装一下""" 114 | if activation_choose == "sigmoid": 115 | return sigmoid(input) 116 | elif activation_choose == "relu": 117 | return relu(input) 118 | elif activation_choose == "tanh": 119 | return tanh(input) 120 | 121 | return sigmoid(input) 122 | 123 | def activated_back_propagation(activation_choose, derror_wrt_output, output): 124 | """包装反向激活传播""" 125 | if activation_choose == "sigmoid": 126 | return sigmoid_back_propagation(derror_wrt_output, output) 127 | elif activation_choose == "relu": 128 | return relu_back_propagation(derror_wrt_output, output) 129 | elif activation_choose == "tanh": 130 | return tanh_back_propagation(derror_wrt_output, output) 131 | 132 | return sigmoid_back_propagation(derror_wrt_output, output) 133 | 134 | class NeuralNetwork: 135 | """ 136 | 神经网络 137 | 支持深度网络，例如，设计一个5层网络，则layers_strcuture=[2,10,7,5,2] 138 | """ 139 | def __init__(self, layers_strcuture, print_cost = False): 140 | self.layers_strcuture = layers_strcuture 141 | self.layers_num = len(layers_strcuture) 142 | 143 | # 除掉输入层的网络层数，因为其他层才是真正的神经元层 144 | self.param_layers_num = self.layers_num - 1 145 | 146 | self.learning_rate = 0.0618 147 | self.num_iterations = 2000 148 | self.x = None 149 | self.y = None 150 | self.w = dict() 151 | self.b = dict() 152 | self.costs = [] 153 | self.print_cost = print_cost 154 | 155 | self.init_w_and_b() 156 | 157 | def set_learning_rate(self, learning_rate): 158 | """设置学习率""" 159 | self.learning_rate = learning_rate 160 | 161 | def set_num_iterations(self, num_iterations): 162 | """设置迭代次数""" 163 | self.num_iterations = num_iterations 164 | 165 | def set_xy(self, input, expected_output): 166 | """设置神经网络的输入和期望的输出""" 167 | self.x = input 168 | self.y = expected_output 169 | 170 | def init_w_and_b(self): 171 | """ 172 | 函数: 173 | 初始化神经网络所有参数 174 | 输入: 175 | layers_strcuture: 神经网络的结构，例如[2,4,3,1]，4层结构: 176 | 第0层输入层接收2个数据，第1层隐藏层4个神经元，第2层隐藏层3个神经元，第3层输出层1个神经元 177 | 返回: 神经网络各层参数的索引表，用来定位权值 wᵢ 和偏置 bᵢ，i为网络层编号 178 | """ 179 | np.random.seed(3) 180 | 181 | # 当前神经元层的权值为 n_i x n_(i-1)的矩阵，i为网络层编号，n为下标i代表的网络层的节点个数 182 | # 例如[2,4,3,1]，4层结构：第0层输入层为2，那么第1层隐藏层神经元个数为4 183 | # 那么第1层的权值w是一个 4x2 的矩阵，如： 184 | # w1 = array([ [-0.96927756, -0.59273074], 185 | # [ 0.58227367, 0.45993021], 186 | # [-0.02270222, 0.13577601], 187 | # [-0.07912066, -1.49802751] ]) 188 | # 当前层的偏置一般给0就行，偏置是个1xnᵢ的矩阵，nᵢ为第i层的节点个数，例如第1层为4个节点，那么： 189 | # b1 = array([ 0., 0., 0., 0.]) 190 | 191 | for l in range(1, self.layers_num): 192 | self.w["w" + str(l)] = np.random.randn(self.layers_strcuture[l], self.layers_strcuture[l-1])/np.sqrt(self.layers_strcuture[l-1]) 193 | self.b["b" + str(l)] = np.zeros((self.layers_strcuture[l], 1)) 194 | 195 | return self.w, self.b 196 | 197 | def layer_activation_forward(self, x, w, b, activation_choose): 198 | """ 199 | 函数： 200 | 网络层的正向传播 201 | 输入： 202 | x: 当前网络层输入（即上一层的输出），一般是所有训练数据，即输入矩阵 203 | w: 当前网络层的权值矩阵 204 | b: 当前网络层的偏置矩阵 205 | activation_choose: 选择激活函数 "sigmoid", "relu", "tanh" 206 | 返回: 207 | output: 网络层的激活输出 208 | cache: 缓存该网络层的信息，供后续使用： (x, w, b, input_sum) -> cache 209 | """ 210 | 211 | # 对输入求加权和，见式（5.1） 212 | input_sum = np.dot(w, x) + b 213 | 214 | # 对输入加权和进行激活输出 215 | output, _ = activated(activation_choose, input_sum) 216 | 217 | return output, (x, w, b, input_sum) 218 | 219 | def forward_propagation(self, x): 220 | """ 221 | 函数: 222 | 神经网络的正向传播 223 | 输入: 224 | 225 | 返回: 226 | output: 正向传播完成后的输出层的输出 227 | caches: 正向传播过程中缓存每一个网络层的信息： (x, w, b, input_sum),... -> caches 228 | """ 229 | caches = [] 230 | 231 | #作为输入层，输出 = 输入 232 | output_prev = x 233 | 234 | #第0层为输入层，只负责观察到输入的数据，并不需要处理，正向传播从第1层开始，一直到输出层输出为止 235 | # range(1, n) => [1, 2, ..., n-1] 236 | L = self.param_layers_num 237 | for l in range(1, L): 238 | # 当前网络层的输入来自前一层的输出 239 | input_cur = output_prev 240 | output_prev, cache = self.layer_activation_forward(input_cur, self.w["w"+ str(l)], self.b["b" + str(l)], "relu") 241 | caches.append(cache) 242 | 243 | output, cache = self.layer_activation_forward(output_prev, self.w["w" + str(L)], self.b["b" + str(L)], "sigmoid") 244 | caches.append(cache) 245 | 246 | return output, caches 247 | 248 | def show_caches(self, caches): 249 | """显示网络层的缓存参数信息""" 250 | i = 1 251 | for cache in caches: 252 | print("%dtd Layer" % i) 253 | print(" input: %s" % cache[0]) 254 | print(" w: %s" % cache[1]) 255 | print(" b: %s" % cache[2]) 256 | print(" input_sum: %s" % cache[3]) 257 | print("----------") 258 | i += 1 259 | 260 | def compute_error(self, output): 261 | """ 262 | 函数: 263 | 计算档次迭代的输出总误差 264 | 输入: 265 | 266 | 返回: 267 | 268 | """ 269 | 270 | m = self.y.shape[1] 271 | 272 | # 计算误差，见式(5.5): E = Σ1/2(期望输出-实际输出)² 273 | #error = np.sum(0.5 * (self.y - output) ** 2) / m 274 | 275 | # 交叉熵作为误差函数 276 | error = -np.sum(np.multiply(np.log(output),self.y) + np.multiply(np.log(1 - output), 1 - self.y)) / m 277 | error = np.squeeze(error) 278 | 279 | return error 280 | 281 | def layer_activation_backward(self, derror_wrt_output, cache, activation_choose): 282 | """ 283 | 函数: 284 | 网络层的反向传播 285 | 输入: 286 | derror_wrt_output: 误差关于输出的偏导 287 | cache: 网络层的缓存信息 (x, w, b, input_sum) 288 | activation_choose: 选择激活函数 "sigmoid", "relu", "tanh" 289 | 返回: 梯度信息，即 290 | derror_wrt_output_prev: 反向传播到上一层的误差关于输出的梯度 291 | derror_wrt_dw: 误差关于权值的梯度 292 | derror_wrt_db: 误差关于偏置的梯度 293 | """ 294 | input, w, b, input_sum = cache 295 | output_prev = input # 上一层的输出 = 当前层的输入; 注意是'输入'不是输入的加权和（input_sum） 296 | m = output_prev.shape[1] # m是输入的样本数量，我们要取均值，所以下面的求值要除以m 297 | 298 | # 实现式（5.13）-> 误差关于权值w的偏导数 299 | derror_wrt_dinput = activated_back_propagation(activation_choose, derror_wrt_output, input_sum) 300 | derror_wrt_dw = np.dot(derror_wrt_dinput, output_prev.T) / m 301 | 302 | # 实现式 （5.32）-> 误差关于偏置b的偏导数 303 | derror_wrt_db = np.sum(derror_wrt_dinput, axis=1, keepdims=True)/m 304 | 305 | # 为反向传播到上一层提供误差传递，见式（5.28）的 （Σδ·w） 部分 306 | derror_wrt_output_prev = np.dot(w.T, derror_wrt_dinput) 307 | 308 | return derror_wrt_output_prev, derror_wrt_dw, derror_wrt_db 309 | 310 | def back_propagation(self, output, caches): 311 | """ 312 | 函数: 313 | 神经网络的反向传播 314 | 输入: 315 | output：神经网络输 316 | caches：所有网络层（输入层不算）的缓存参数信息 [(x, w, b, input_sum), ...] 317 | 返回: 318 | grads: 返回当前迭代的梯度信息 319 | """ 320 | grads = {} 321 | L = self.param_layers_num # 322 | output = output.reshape(output.shape) # 把输出层输出输出重构成和期望输出一样的结构 323 | 324 | expected_output = self.y 325 | 326 | # 见式(5.8) 327 | #derror_wrt_output = -(expected_output - output) 328 | 329 | # 交叉熵作为误差函数 330 | derror_wrt_output = - (np.divide(expected_output, output) - np.divide(1 - expected_output, 1 - output)) 331 | 332 | # 反向传播：输出层 -> 隐藏层，得到梯度：见式(5.8), (5.13), (5.15) 333 | current_cache = caches[L - 1] # 取最后一层,即输出层的参数信息 334 | grads["derror_wrt_output" + str(L)], grads["derror_wrt_dw" + str(L)], grads["derror_wrt_db" + str(L)] = \ 335 | self.layer_activation_backward(derror_wrt_output, current_cache, "sigmoid") 336 | 337 | # 反向传播：隐藏层 -> 隐藏层，得到梯度：见式 (5.28)的(Σδ·w), (5.28), (5.32) 338 | for l in reversed(range(L - 1)): 339 | current_cache = caches[l] 340 | derror_wrt_output_prev_temp, derror_wrt_dw_temp, derror_wrt_db_temp = \ 341 | self.layer_activation_backward(grads["derror_wrt_output" + str(l + 2)], current_cache, "relu") 342 | 343 | grads["derror_wrt_output" + str(l + 1)] = derror_wrt_output_prev_temp 344 | grads["derror_wrt_dw" + str(l + 1)] = derror_wrt_dw_temp 345 | grads["derror_wrt_db" + str(l + 1)] = derror_wrt_db_temp 346 | 347 | return grads 348 | 349 | def update_w_and_b(self, grads): 350 | """ 351 | 函数: 352 | 根据梯度信息更新w，b 353 | 输入: 354 | grads：当前迭代的梯度信息 355 | 返回: 356 | 357 | """ 358 | # 权值w和偏置b的更新，见式:（5.16),(5.18) 359 | for l in range(self.param_layers_num): 360 | self.w["w" + str(l + 1)] = self.w["w" + str(l + 1)] - self.learning_rate * grads["derror_wrt_dw" + str(l + 1)] 361 | self.b["b" + str(l + 1)] = self.b["b" + str(l + 1)] - self.learning_rate * grads["derror_wrt_db" + str(l + 1)] 362 | 363 | def training_modle(self): 364 | """训练神经网络模型""" 365 | 366 | np.random.seed(5) 367 | for i in range(0, self.num_iterations): 368 | # 正向传播，得到网络输出，以及每一层的参数信息 369 | output, caches = self.forward_propagation(self.x) 370 | 371 | # 计算网络输出误差 372 | cost = self.compute_error(output) 373 | 374 | # 反向传播，得到梯度信息 375 | grads = self.back_propagation(output, caches) 376 | 377 | # 根据梯度信息，更新权值w和偏置b 378 | self.update_w_and_b(grads) 379 | 380 | # 当次迭代结束，打印误差信息 381 | if self.print_cost and i % 100 == 0: 382 | print ("Cost after iteration %i: %f" % (i, cost)) 383 | if self.print_cost and i % 100 == 0: 384 | self.costs.append(cost) 385 | 386 | # 模型训练完后显示误差曲线 387 | if False: 388 | plt.plot(np.squeeze(self.costs)) 389 | plt.ylabel(u'神经网络误差', fontproperties = font) 390 | plt.xlabel(u'迭代次数 (*100)', fontproperties = font) 391 | plt.title(u"学习率 =" + str(self.learning_rate), fontproperties = font) 392 | plt.show() 393 | 394 | return self.w, self.b 395 | 396 | def predict_by_modle(self, x): 397 | """使用训练好的模型（即最后求得w，b参数）来决策输入的样本的结果""" 398 | output, _ = self.forward_propagation(x.T) 399 | output = output.T 400 | result = output / np.sum(output, axis=1, keepdims=True) 401 | return np.argmax(result, axis=1) 402 | 403 | 404 | def plot_decision_boundary(xy, colors, pred_func): 405 | # xy是坐标点的集合，把集合的范围算出来 406 | # 加减0.5相当于扩大画布的范围，不然画出来的图坐标点会落在图的边缘，逼死强迫症患者 407 | x_min, x_max = xy[:, 0].min() - 0.5, xy[:, 0].max() + 0.5 408 | y_min, y_max = xy[:, 1].min() - 0.5, xy[:, 1].max() + 0.5 409 | 410 | # 以h为分辨率，生成采样点的网格，就像一张网覆盖所有颜色点 411 | h = .01 412 | xx, yy = np.meshgrid(np.arange(x_min, x_max, h), np.arange(y_min, y_max, h)) 413 | 414 | # 把网格点集合作为输入到模型，也就是预测这个采样点是什么颜色的点，从而得到一个决策面 415 | Z = pred_func(np.c_[xx.ravel(), yy.ravel()]) 416 | Z = Z.reshape(xx.shape) 417 | 418 | # 利用等高线，把预测的结果画出来，效果上就是画出红蓝点的分界线 419 | plt.contourf(xx, yy, Z, cmap=plt.cm.Spectral) 420 | 421 | # 训练用的红蓝点点也画出来 422 | plt.scatter(xy[:, 0], xy[:, 1], c=colors, marker='o', cmap=plt.cm.Spectral, edgecolors='black') 423 | 424 | 425 | if __name__ == "__main__": 426 | plt.figure(figsize=(16, 32)) 427 | 428 | # 用sklearn的数据样本集，产生2种颜色的坐标点，noise是噪声系数，噪声越大，2种颜色的点分布越凌乱 429 | xy, colors = sklearn.datasets.make_moons(60, noise=1.0) 430 | 431 | # 因为点的颜色是1bit，我们设计一个神经网络，输出层有2个神经元。 432 | # 标定输出[1,0]为红色点，输出[0,1]为蓝色点 433 | expect_outputed = [] 434 | for c in colors: 435 | if c == 1: 436 | expect_outputed.append([0,1]) 437 | else: 438 | expect_outputed.append([1,0]) 439 | 440 | expect_outputed = np.array(expect_outputed).T 441 | 442 | # 设计3层网络，改变隐藏层神经元的个数，观察神经网络分类红蓝点的效果 443 | hidden_layer_neuron_num_list = [1,2,4,10,20,50] 444 | for i, hidden_layer_neuron_num in enumerate(hidden_layer_neuron_num_list): 445 | plt.subplot(5, 2, i + 1) 446 | plt.title(u'隐藏层神经元数量: %d' % hidden_layer_neuron_num, fontproperties = font) 447 | 448 | nn = NeuralNetwork([2, hidden_layer_neuron_num, 2], True) 449 | 450 | # 输出和输入层都是2个节点，所以输入和输出的数据集合都要是 nx2的矩阵 451 | nn.set_xy(xy.T, expect_outputed) 452 | nn.set_num_iterations(25000) 453 | nn.set_learning_rate(0.1) 454 | w, b = nn.training_modle() 455 | plot_decision_boundary(xy, colors, nn.predict_by_modle) 456 | 457 | plt.show() -------------------------------------------------------------------------------- /AI从入门到放弃：《BP神经网络算法推导及代码实现笔记》/resources/10DA03D78180BD0686C0CB9E691BF459.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zsysuper/AI_Notes/3f321cf266f2fb270f02c6b440a27843b43d5a18/AI从入门到放弃：《BP神经网络算法推导及代码实现笔记》/resources/10DA03D78180BD0686C0CB9E691BF459.png -------------------------------------------------------------------------------- /AI从入门到放弃：《BP神经网络算法推导及代码实现笔记》/resources/32F79E55C7B0FF938AA82D2D69E3F97B.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zsysuper/AI_Notes/3f321cf266f2fb270f02c6b440a27843b43d5a18/AI从入门到放弃：《BP神经网络算法推导及代码实现笔记》/resources/32F79E55C7B0FF938AA82D2D69E3F97B.png -------------------------------------------------------------------------------- /AI从入门到放弃：《BP神经网络算法推导及代码实现笔记》/resources/41D98819C2BD4A40EF5FEC7B2221FE29.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zsysuper/AI_Notes/3f321cf266f2fb270f02c6b440a27843b43d5a18/AI从入门到放弃：《BP神经网络算法推导及代码实现笔记》/resources/41D98819C2BD4A40EF5FEC7B2221FE29.png -------------------------------------------------------------------------------- /AI从入门到放弃：《BP神经网络算法推导及代码实现笔记》/resources/43A0B27F2362DA18F8AEF4E007D6D2DD.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zsysuper/AI_Notes/3f321cf266f2fb270f02c6b440a27843b43d5a18/AI从入门到放弃：《BP神经网络算法推导及代码实现笔记》/resources/43A0B27F2362DA18F8AEF4E007D6D2DD.png -------------------------------------------------------------------------------- /AI从入门到放弃：《BP神经网络算法推导及代码实现笔记》/resources/46B66C214BF95AC6EADC4F4120729397.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zsysuper/AI_Notes/3f321cf266f2fb270f02c6b440a27843b43d5a18/AI从入门到放弃：《BP神经网络算法推导及代码实现笔记》/resources/46B66C214BF95AC6EADC4F4120729397.jpg -------------------------------------------------------------------------------- /AI从入门到放弃：《BP神经网络算法推导及代码实现笔记》/resources/49839165F7B45F38E876D865D80D8E04.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zsysuper/AI_Notes/3f321cf266f2fb270f02c6b440a27843b43d5a18/AI从入门到放弃：《BP神经网络算法推导及代码实现笔记》/resources/49839165F7B45F38E876D865D80D8E04.jpg -------------------------------------------------------------------------------- /AI从入门到放弃：《BP神经网络算法推导及代码实现笔记》/resources/5D2344074BC281BA3B5582A65BCCC4B5.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zsysuper/AI_Notes/3f321cf266f2fb270f02c6b440a27843b43d5a18/AI从入门到放弃：《BP神经网络算法推导及代码实现笔记》/resources/5D2344074BC281BA3B5582A65BCCC4B5.jpg -------------------------------------------------------------------------------- /AI从入门到放弃：《BP神经网络算法推导及代码实现笔记》/resources/62990B82CC910688B813103D80FCB509.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zsysuper/AI_Notes/3f321cf266f2fb270f02c6b440a27843b43d5a18/AI从入门到放弃：《BP神经网络算法推导及代码实现笔记》/resources/62990B82CC910688B813103D80FCB509.jpg -------------------------------------------------------------------------------- /AI从入门到放弃：《BP神经网络算法推导及代码实现笔记》/resources/6EC3CD5AC3B342C739D267519B2BEA57.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zsysuper/AI_Notes/3f321cf266f2fb270f02c6b440a27843b43d5a18/AI从入门到放弃：《BP神经网络算法推导及代码实现笔记》/resources/6EC3CD5AC3B342C739D267519B2BEA57.png -------------------------------------------------------------------------------- /AI从入门到放弃：《BP神经网络算法推导及代码实现笔记》/resources/837AA91C395EAE98C77FE7B75E4A6100.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zsysuper/AI_Notes/3f321cf266f2fb270f02c6b440a27843b43d5a18/AI从入门到放弃：《BP神经网络算法推导及代码实现笔记》/resources/837AA91C395EAE98C77FE7B75E4A6100.jpg -------------------------------------------------------------------------------- /AI从入门到放弃：《BP神经网络算法推导及代码实现笔记》/resources/89D527539F0A7055D369F32C68B1F981.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zsysuper/AI_Notes/3f321cf266f2fb270f02c6b440a27843b43d5a18/AI从入门到放弃：《BP神经网络算法推导及代码实现笔记》/resources/89D527539F0A7055D369F32C68B1F981.png -------------------------------------------------------------------------------- /AI从入门到放弃：《BP神经网络算法推导及代码实现笔记》/resources/8B74CF49BA388F18402444C8A7AAAC4F.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zsysuper/AI_Notes/3f321cf266f2fb270f02c6b440a27843b43d5a18/AI从入门到放弃：《BP神经网络算法推导及代码实现笔记》/resources/8B74CF49BA388F18402444C8A7AAAC4F.png -------------------------------------------------------------------------------- /AI从入门到放弃：《BP神经网络算法推导及代码实现笔记》/resources/AF2E4DBD05E044A145DBA1E169E7594C.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zsysuper/AI_Notes/3f321cf266f2fb270f02c6b440a27843b43d5a18/AI从入门到放弃：《BP神经网络算法推导及代码实现笔记》/resources/AF2E4DBD05E044A145DBA1E169E7594C.jpg -------------------------------------------------------------------------------- /AI从入门到放弃：《BP神经网络算法推导及代码实现笔记》/resources/D53615C119E3112D9200732F7DDC5643.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zsysuper/AI_Notes/3f321cf266f2fb270f02c6b440a27843b43d5a18/AI从入门到放弃：《BP神经网络算法推导及代码实现笔记》/resources/D53615C119E3112D9200732F7DDC5643.png -------------------------------------------------------------------------------- /AI从入门到放弃：《BP神经网络算法推导及代码实现笔记》/resources/E0D33265A98E034A16574FA99388225A.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zsysuper/AI_Notes/3f321cf266f2fb270f02c6b440a27843b43d5a18/AI从入门到放弃：《BP神经网络算法推导及代码实现笔记》/resources/E0D33265A98E034A16574FA99388225A.png -------------------------------------------------------------------------------- /AI从入门到放弃：《BP神经网络算法推导及代码实现笔记》/resources/E68468BFF0B3210F75F5293104007E66.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zsysuper/AI_Notes/3f321cf266f2fb270f02c6b440a27843b43d5a18/AI从入门到放弃：《BP神经网络算法推导及代码实现笔记》/resources/E68468BFF0B3210F75F5293104007E66.jpg -------------------------------------------------------------------------------- /AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》/AI从入门到放弃：CNN的导火索，用MLP做图像分类识别？--html/resources/18911F9E66B3C88DEC4109AA4E8ED5FB.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zsysuper/AI_Notes/3f321cf266f2fb270f02c6b440a27843b43d5a18/AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》/AI从入门到放弃：CNN的导火索，用MLP做图像分类识别？--html/resources/18911F9E66B3C88DEC4109AA4E8ED5FB.png -------------------------------------------------------------------------------- /AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》/AI从入门到放弃：CNN的导火索，用MLP做图像分类识别？--html/resources/31E1ECF3F19D72C84CA3DA925C0CB9DD.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zsysuper/AI_Notes/3f321cf266f2fb270f02c6b440a27843b43d5a18/AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》/AI从入门到放弃：CNN的导火索，用MLP做图像分类识别？--html/resources/31E1ECF3F19D72C84CA3DA925C0CB9DD.png -------------------------------------------------------------------------------- /AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》/AI从入门到放弃：CNN的导火索，用MLP做图像分类识别？--html/resources/5352CC63BE1E522FF056853AA1A26253.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zsysuper/AI_Notes/3f321cf266f2fb270f02c6b440a27843b43d5a18/AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》/AI从入门到放弃：CNN的导火索，用MLP做图像分类识别？--html/resources/5352CC63BE1E522FF056853AA1A26253.png -------------------------------------------------------------------------------- /AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》/AI从入门到放弃：CNN的导火索，用MLP做图像分类识别？--html/resources/630243EC928C70E4417306DFBB39A9D1.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zsysuper/AI_Notes/3f321cf266f2fb270f02c6b440a27843b43d5a18/AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》/AI从入门到放弃：CNN的导火索，用MLP做图像分类识别？--html/resources/630243EC928C70E4417306DFBB39A9D1.jpg -------------------------------------------------------------------------------- /AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》/AI从入门到放弃：CNN的导火索，用MLP做图像分类识别？--html/resources/A3E4523C6F2874F6555CD9A1EAEBE5B0.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zsysuper/AI_Notes/3f321cf266f2fb270f02c6b440a27843b43d5a18/AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》/AI从入门到放弃：CNN的导火索，用MLP做图像分类识别？--html/resources/A3E4523C6F2874F6555CD9A1EAEBE5B0.png -------------------------------------------------------------------------------- /AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》/AI从入门到放弃：CNN的导火索，用MLP做图像分类识别？--html/resources/BCDD7CADA14ADFC183B9AFBCCB0B8A1C.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zsysuper/AI_Notes/3f321cf266f2fb270f02c6b440a27843b43d5a18/AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》/AI从入门到放弃：CNN的导火索，用MLP做图像分类识别？--html/resources/BCDD7CADA14ADFC183B9AFBCCB0B8A1C.png -------------------------------------------------------------------------------- /AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》/AI从入门到放弃：CNN的导火索，用MLP做图像分类识别？--html/resources/C557C61880CBC43A9C4E776802CC06D0.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zsysuper/AI_Notes/3f321cf266f2fb270f02c6b440a27843b43d5a18/AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》/AI从入门到放弃：CNN的导火索，用MLP做图像分类识别？--html/resources/C557C61880CBC43A9C4E776802CC06D0.jpg -------------------------------------------------------------------------------- /AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》/AI从入门到放弃：CNN的导火索，用MLP做图像分类识别？--html/resources/CBCBF0C20D17CDDAC3EE968D43FC0B54.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zsysuper/AI_Notes/3f321cf266f2fb270f02c6b440a27843b43d5a18/AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》/AI从入门到放弃：CNN的导火索，用MLP做图像分类识别？--html/resources/CBCBF0C20D17CDDAC3EE968D43FC0B54.png -------------------------------------------------------------------------------- /AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》/AI从入门到放弃：CNN的导火索，用MLP做图像分类识别？.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zsysuper/AI_Notes/3f321cf266f2fb270f02c6b440a27843b43d5a18/AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》/AI从入门到放弃：CNN的导火索，用MLP做图像分类识别？.pdf -------------------------------------------------------------------------------- /AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》/cat_pics_my.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zsysuper/AI_Notes/3f321cf266f2fb270f02c6b440a27843b43d5a18/AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》/cat_pics_my.png -------------------------------------------------------------------------------- /AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》/cat_pics_my_predict.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zsysuper/AI_Notes/3f321cf266f2fb270f02c6b440a27843b43d5a18/AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》/cat_pics_my_predict.png -------------------------------------------------------------------------------- /AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》/cat_pics_test.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zsysuper/AI_Notes/3f321cf266f2fb270f02c6b440a27843b43d5a18/AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》/cat_pics_test.png -------------------------------------------------------------------------------- /AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》/cat_pics_test_predict.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zsysuper/AI_Notes/3f321cf266f2fb270f02c6b440a27843b43d5a18/AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》/cat_pics_test_predict.png -------------------------------------------------------------------------------- /AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》/cat_pics_train.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zsysuper/AI_Notes/3f321cf266f2fb270f02c6b440a27843b43d5a18/AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》/cat_pics_train.png -------------------------------------------------------------------------------- /AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》/cat_pics_train_predict.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zsysuper/AI_Notes/3f321cf266f2fb270f02c6b440a27843b43d5a18/AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》/cat_pics_train_predict.png -------------------------------------------------------------------------------- /AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》/datasets/my_cat_misu.h5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zsysuper/AI_Notes/3f321cf266f2fb270f02c6b440a27843b43d5a18/AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》/datasets/my_cat_misu.h5 -------------------------------------------------------------------------------- /AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》/datasets/test_catvnoncat.h5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zsysuper/AI_Notes/3f321cf266f2fb270f02c6b440a27843b43d5a18/AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》/datasets/test_catvnoncat.h5 -------------------------------------------------------------------------------- /AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》/datasets/train_catvnoncat.h5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zsysuper/AI_Notes/3f321cf266f2fb270f02c6b440a27843b43d5a18/AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》/datasets/train_catvnoncat.h5 -------------------------------------------------------------------------------- /AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》/misu.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zsysuper/AI_Notes/3f321cf266f2fb270f02c6b440a27843b43d5a18/AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》/misu.jpg -------------------------------------------------------------------------------- /AI从入门到放弃：《CNN的导火索，用MLP做图像分类识别？》/picture_classification.py: -------------------------------------------------------------------------------- 1 | #coding:utf-8 2 | import h5py 3 | import matplotlib.font_manager as fm 4 | import matplotlib.pyplot as plt 5 | import numpy as np 6 | from NeuralNetwork import * 7 | 8 | font = fm.FontProperties(fname='/System/Library/Fonts/STHeiti Light.ttc') 9 | 10 | def load_Cat_dataset(): 11 | train_dataset = h5py.File('datasets/train_catvnoncat.h5', "r") 12 | train_set_x_orig = np.array(train_dataset["train_set_x"][:]) 13 | train_set_y_orig = np.array(train_dataset["train_set_y"][:]) 14 | 15 | test_dataset = h5py.File('datasets/test_catvnoncat.h5', "r") 16 | test_set_x_orig = np.array(test_dataset["test_set_x"][:]) 17 | test_set_y_orig = np.array(test_dataset["test_set_y"][:]) 18 | 19 | mycat_dataset = h5py.File('datasets/my_cat_misu.h5', "r") 20 | mycat_set_x_orig = np.array(mycat_dataset["mycat_set_x"][:]) 21 | mycat_set_y_orig = np.array(mycat_dataset["mycat_set_y"][:]) 22 | 23 | classes = np.array(test_dataset["list_classes"][:]) 24 | 25 | train_set_y_orig = train_set_y_orig.reshape((1, train_set_y_orig.shape[0])) 26 | test_set_y_orig = test_set_y_orig.reshape((1, test_set_y_orig.shape[0])) 27 | mycat_set_y_orig = mycat_set_y_orig.reshape((1, mycat_set_y_orig.shape[0])) 28 | 29 | return train_set_x_orig, train_set_y_orig, test_set_x_orig, test_set_y_orig, mycat_set_x_orig, mycat_set_y_orig,classes 30 | 31 | 32 | def predict_by_modle(x, y, nn): 33 | 34 | m = x.shape[1] 35 | p = np.zeros((1,m)) 36 | 37 | output, caches = nn.forward_propagation(x) 38 | 39 | for i in range(0, output.shape[1]): 40 | if output[0,i] > 0.5: 41 | p[0,i] = 1 42 | else: 43 | p[0,i] = 0 44 | 45 | # 预测出来的结果和期望的结果比对，看看准确率多少： 46 | # 比如100张预测图片里有50张猫的图片，只识别出40张，那么识别率就是80% 47 | print(u"识别率: " + str(np.sum((p == y)/float(m)))) 48 | return np.array(p[0], dtype=np.int), (p==y)[0], np.sum((p == y)/float(m))*100 49 | 50 | 51 | def save_imgs_to_h5file(h5_fname, x_label, y_label, img_paths_list, img_label_list): 52 | data_imgs = np.random.rand(len(img_paths_list), 64, 64, 3).astype('int') 53 | label_imgs = np.random.rand(len(img_paths_list), 1).astype('int') 54 | 55 | for i in range(len(img_paths_list)): 56 | data_imgs[i] = np.array(plt.imread(img_paths_list[i])) 57 | label_imgs[i] = np.array(img_label_list[i]) 58 | 59 | f = h5py.File(h5_fname, 'w') 60 | f.create_dataset(x_label, data=data_imgs) 61 | f.create_dataset(y_label, data=label_imgs) 62 | f.close() 63 | 64 | return data_imgs, label_imgs 65 | 66 | if __name__ == "__main__": 67 | # 图片label为1代表这是一张喵星人的图片，0代表不是 68 | #save_imgs_to_h5file('datasets/my_cat_misu.h5', 'mycat_set_x', 'mycat_set_y', ['misu.jpg'],[1]) 69 | 70 | train_set_x_orig, train_set_y_orig, test_set_x_orig, test_set_y_orig, mycat_set_x_orig, mycat_set_y_orig, classes = load_Cat_dataset() 71 | 72 | train_x_flatten = train_set_x_orig.reshape(train_set_x_orig.shape[0], -1).T 73 | test_x_flatten = test_set_x_orig.reshape(test_set_x_orig.shape[0], -1).T 74 | mycat_x_flatten = mycat_set_x_orig.reshape(mycat_set_x_orig.shape[0], -1).T 75 | train_set_x = train_x_flatten / 255. 76 | test_set_x = test_x_flatten / 255. 77 | mycat_set_x = mycat_x_flatten / 255. 78 | 79 | print(u"训练图片数量: %d" % len(train_set_x_orig)) 80 | print(u"测试图片数量: %d" % len(test_set_x_orig)) 81 | 82 | plt.figure(figsize=(10, 20)) 83 | plt.subplots_adjust(wspace=0,hspace=0.15) 84 | for i in range(len(train_set_x_orig)): 85 | plt.subplot(21,10, i+1) 86 | plt.imshow(train_set_x_orig[i],interpolation='none',cmap='Reds_r',vmin=0.6,vmax=.9) 87 | plt.xticks([]) 88 | plt.yticks([]) 89 | plt.savefig("cat_pics_train.png") 90 | plt.show() 91 | 92 | plt.figure(figsize=(8, 8)) 93 | plt.subplots_adjust(wspace=0, hspace=0.1) 94 | for i in range(len(test_set_x_orig)): 95 | ax = plt.subplot(8, 8, i + 1) 96 | im = ax.imshow(test_set_x_orig[i], interpolation='none', cmap='Reds_r', vmin=0.6, vmax=.9) 97 | plt.xticks([]) 98 | plt.yticks([]) 99 | 100 | plt.savefig("cat_pics_test.png") 101 | plt.show() 102 | 103 | plt.figure(figsize=(2, 2)) 104 | plt.subplots_adjust(wspace=0, hspace=0) 105 | for i in range(len(mycat_set_x_orig)): 106 | ax = plt.subplot(1, 1, i + 1) 107 | im = ax.imshow(mycat_set_x_orig[i], interpolation='none', cmap='Reds_r', vmin=0.6, vmax=.9) 108 | plt.xticks([]) 109 | plt.yticks([]) 110 | 111 | plt.savefig("cat_pics_my.png") 112 | plt.show() 113 | 114 | # 用训练图片集训练模型 115 | layers_dims = [12288, 20, 7, 5, 1] 116 | nn = NeuralNetwork(layers_dims, True) 117 | nn.set_xy(train_set_x, train_set_y_orig) 118 | nn.set_num_iterations(10000) 119 | nn.set_learning_rate(0.0075) 120 | nn.training_modle() 121 | 122 | # 结果展示说明： 123 | # 【识别正确】： 124 | # 1.原图是猫，识别为猫 --> 原图显示 125 | # 2.原图不是猫，识别为不是猫 --> 降低显示亮度 126 | 127 | # 【识别错误】： 128 | # 1.原图是猫，但是识别为不是猫 --> 标红显示 129 | # 2.原图不是猫， 但是识别成猫 --> 标红显示 130 | 131 | # 训练用的图片走一遍模型，观察其识别率 132 | plt.figure(figsize=(10, 20)) 133 | plt.subplots_adjust(wspace=0, hspace=0.15) 134 | 135 | pred_train, true, accuracy = predict_by_modle(train_set_x, train_set_y_orig, nn) 136 | 137 | for i in range(len(train_set_x_orig)): 138 | ax = plt.subplot(21, 10, i + 1) 139 | 140 | x_data = train_set_x_orig[i] 141 | if pred_train[i] == 0 and train_set_y_orig[0][i] == 0: 142 | x_data = x_data/5 143 | 144 | if true[i] == False: 145 | x_data[:, :, 0] = x_data[:, :, 0] + (255 - x_data[:, :, 0]) 146 | 147 | im = plt.imshow(x_data,interpolation='none',cmap='Reds_r',vmin=0.6,vmax=.9) 148 | 149 | plt.xticks([]) 150 | plt.yticks([]) 151 | 152 | plt.suptitle(u"Num Of Pictrues: %d\n Accuracy: %.2f%%" % (len(train_set_x_orig), accuracy), y=0.92, fontsize=20) 153 | plt.savefig("cat_pics_train_predict.png") 154 | plt.show() 155 | 156 | # 不属于训练图片集合的测试图片，走一遍模型，观察其识别率 157 | plt.figure(figsize=(8, 8)) 158 | plt.subplots_adjust(wspace=0, hspace=0.1) 159 | 160 | pred_test, true, accuracy = predict_by_modle(test_set_x, test_set_y_orig, nn) 161 | 162 | for i in range(len(test_set_x_orig)): 163 | ax = plt.subplot(8, 8, i + 1) 164 | 165 | x_data = test_set_x_orig[i] 166 | if pred_test[i] == 0 and test_set_y_orig[0][i] == 0: 167 | x_data = x_data/5 168 | 169 | if true[i] == False: 170 | x_data[:, :, 0] = x_data[:, :, 0] + (255 - x_data[:, :, 0]) 171 | 172 | im = ax.imshow(x_data, interpolation='none', cmap='Reds_r', vmin=0.6, vmax=.9) 173 | 174 | plt.xticks([]) 175 | plt.yticks([]) 176 | 177 | plt.suptitle(u"Num Of Pictrues: %d\n Accuracy: %.2f%%" % (len(mycat_set_x_orig), accuracy), fontsize=20) 178 | plt.savefig("cat_pics_test_predict.png") 179 | plt.show() 180 | 181 | # 用我家主子的照骗，走一遍模型，观察其识别率，因为只有一张图片，所以识别率要么 100% 要么 0% 182 | plt.figure(figsize=(2, 2.6)) 183 | plt.subplots_adjust(wspace=0, hspace=0.1) 184 | 185 | pred_mycat, true, accuracy = predict_by_modle(mycat_set_x, mycat_set_y_orig, nn) 186 | for i in range(len(mycat_set_x_orig)): 187 | ax = plt.subplot(1, 1, i+1) 188 | 189 | x_data = mycat_set_x_orig[i] 190 | if pred_mycat[i] == 0 and mycat_set_y_orig[0][i] == 0: 191 | x_data = x_data/5 192 | 193 | if true[i] == False: 194 | x_data[:, :, 0] = x_data[:, :, 0] + (255 - x_data[:, :, 0]) 195 | 196 | im = ax.imshow(x_data, interpolation='none', cmap='Reds_r', vmin=0.6, vmax=.9) 197 | 198 | plt.xticks([]) 199 | plt.yticks([]) 200 | 201 | if pred_mycat[i] == 1: 202 | plt.suptitle(u"我：'我主子是喵星人吗？'\nA I :'是滴'", fontproperties = font) 203 | else: 204 | plt.suptitle(u"我：'我主子是喵星人吗？'\nA I :'唔系~唔系~'", fontproperties = font) 205 | plt.savefig("cat_pics_my_predict.png") 206 | plt.show() -------------------------------------------------------------------------------- /AI从入门到放弃：《手撕神经卷积网络（CNN）》/AI从入门到放弃：手撕卷积神经网络（CNN）.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zsysuper/AI_Notes/3f321cf266f2fb270f02c6b440a27843b43d5a18/AI从入门到放弃：《手撕神经卷积网络（CNN）》/AI从入门到放弃：手撕卷积神经网络（CNN）.pdf -------------------------------------------------------------------------------- /AI从入门到放弃：《手撕神经卷积网络（CNN）》/CNN-Network/.idea/CNN-Network.iml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | -------------------------------------------------------------------------------- /AI从入门到放弃：《手撕神经卷积网络（CNN）》/CNN-Network/.idea/inspectionProfiles/Project_Default.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 35 | -------------------------------------------------------------------------------- /AI从入门到放弃：《手撕神经卷积网络（CNN）》/CNN-Network/.idea/misc.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 6 | 7 | -------------------------------------------------------------------------------- /AI从入门到放弃：《手撕神经卷积网络（CNN）》/CNN-Network/.idea/modules.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | -------------------------------------------------------------------------------- /AI从入门到放弃：《手撕神经卷积网络（CNN）》/CNN-Network/.idea/workspace.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 31 | 32 | 33 | 34 | 35 | true 36 | DEFINITION_ORDER 37 | 38 | 39 | 40 | 41 | 42 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 |