├── .gitignore ├── Keras 入门课0 -- 目录.md ├── README.md ├── class_1.ipynb ├── class_2.ipynb ├── class_3.ipynb ├── class_4.ipynb ├── class_5.ipynb ├── class_6.ipynb ├── convert_flowers17.py ├── images ├── flowers.jpg ├── lstm.png ├── mlp_model.png ├── resnetblock.png ├── tensorboard_hist.png └── tensorboard_scalars.png └── tlmodel.png /.gitignore: -------------------------------------------------------------------------------- 1 | # Created by .ignore support plugin (hsz.mobi) 2 | *.zip 3 | /class_*/ 4 | *-checkpoint.ipynb 5 | *.h5 6 | /flowers17/ 7 | 8 | -------------------------------------------------------------------------------- /Keras 入门课0 -- 目录.md: -------------------------------------------------------------------------------- 1 | # Keras 入门课0 -- 目录 2 | 3 | 网络上Keras入门的课程或教程都有很多，基本上都是一些最简单的例子，而自己真正去使用的时候，发现需要学习的内容还有很多，看官方文档的时候，效率也是比较低下的。所以才有了这个系列的课程。通过一些例子，逐渐深入的去学习Keras，每节课一个例子，遇到的新的知识点都会拿出来进行分析。这样就会形成一个知识点目录，之后想使用某个知识点的时候，可以很方便的根据知识点进行回看，查询对应的example，这样对于一些方法的使用和拓展都有很大的好处。 4 | 5 | 本系列主要针对一些Keras官方的一些examples来学习Keras的使用。课程使用的例子都来自这里 https://github.com/keras-team/keras/tree/master/examples 6 | 7 | 官方给的examples有一些地方和最新的Keras版本不符合，我都相应做了修改。同时有一些累赘的地方也加了一些修改，删除。尽可能在每一课中做到代码清晰易懂，不重复。 8 | 9 | **课程系列目前还在更新中。。。** 10 | 11 | ## 文章目录 12 | 1. [Keras入门课1 -- 用MLP识别mnist手写字符][class_1] 13 | 2. [Keras入门课2 -- 使用CNN识别mnist手写数字][class_2] 14 | 3. [Keras入门课3 -- 使用CNN识别cifar10数据集][class_3] 15 | 4. [Keras入门课4 -- 使用ResNet识别cifar10数据集][class_4] 16 | 5. [Keras入门课5 -- 网络可视化及训练监控][class_5] 17 | ## 知识点目录 18 | 19 | 点击对应知识点可以直接跳转到指定文章，方便速查 20 | 21 | ### 数据处理相关 22 | 1. [载入Keras中预置的数据库及数据库数据的基本变换][class_1] 23 | 1. [根据不同的模型数据要求，给原始数据图像增加维度][class_2] 24 | 1. [使用Keras内置的ImageDataGenerator来做数据增强][class_3] 25 | 26 | ### 模型相关（Model） 27 | 1. [Sequential模型的定义，以及如何添加层][class_1] 28 | 1. [另一种使用Sequential()构建模型的方法，更加的简单快捷][class_3] 29 | 1. [使用函数式模型（Functional）更加自由的构建模型][class_4] 30 | 1. [将通过Functional方式定义的层初始化为一个模型（Model）][class_4] 31 | 1. [使用compile对网络进行配置][class_1] 32 | 1. [使用fit方法来对小数据库进行训练，这里的小数据库指的是所有数据可以一次性载入到内存][class_1] 33 | 1. [使用fit_generator来进行针对增强数据的训练][class_3] 34 | 1. [使用evaluate方法来对模型进行效果评估][class_1] 35 | 1. [保存模型][class_3] 36 | 37 | ### 网络层相关（Layers） 38 | 1. [如何对Dense层及Dropout层进行基本的配置][class_1] 39 | 1. [Conv2D卷积层和MaxPooling2D池化层的使用][class_2] 40 | 1. [使用keras.layers.add方法，将两个一模一样的张量进行相加][class_4] 41 | 42 | ### 经典网络 43 | 1. [搭建一个精简版的ResNet网络][class_4] 44 | 45 | ### 训练技巧 46 | 1. [在训练中调用回调函数][class_4] 47 | 1. [使用LearningRateScheduler在训练过程中动态的调节学习率][class_4] 48 | 1. [使用ModelCheckpoint保存checkpoint][class_4] 49 | 1. [使用ReduceLROnPlateau在训练进入平台期的时候动态调节学习率][class_4] 50 | 51 | ### 其他 52 | 1. [何用TensorBoard监控训练过程][class_5] 53 | 2. [如何保存网络结构图][class_5] 54 | 55 | 56 | 57 | 58 | 59 | [class_1]:http://blog.csdn.net/tsyccnh/article/details/78834171 60 | [class_2]:http://blog.csdn.net/tsyccnh/article/details/78835384 61 | [class_3]:http://blog.csdn.net/tsyccnh/article/details/78838005 62 | [class_4]:http://blog.csdn.net/tsyccnh/article/details/78865167 63 | [class_5]:http://blog.csdn.net/tsyccnh/article/details/78867562 -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Keras-Tutorials 2 | 3 | 本系列文章keras请使用2.1.2版本。 4 | -------------------------------------------------------------------------------- /class_1.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# 第一节课 使用Keras写一个mlp" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "mlp就是multilayer perceptron，多层感知机。数据集用的是经典的mnist，数字分类问题。\n", 15 | "\n", 16 | "首先导入keras的各种模块\n", 17 | "\n", 18 | "keras.datasets 里面包含了多种常用数据集，如mnist，cifar10等等，可以实现自动下载和解析等等。\n", 19 | "\n", 20 | "keras.models 里面有最核心的模型结构，如顺序模型结构Sequential\n", 21 | "\n", 22 | "keras.layers 里面有一些常用的层结构，如全连接层Dense\n", 23 | "\n", 24 | "keras.optimizers 里面有一些常用优化函数，如adam等" 25 | ] 26 | }, 27 | { 28 | "cell_type": "code", 29 | "execution_count": null, 30 | "metadata": {}, 31 | "outputs": [], 32 | "source": [ 33 | "import keras\n", 34 | "from keras.datasets import mnist \n", 35 | "from keras.models import Sequential \n", 36 | "from keras.layers import Dense,Dropout\n", 37 | "from keras.optimizers import RMSprop" 38 | ] 39 | }, 40 | { 41 | "cell_type": "markdown", 42 | "metadata": {}, 43 | "source": [ 44 | "载入mnist数据，第一次会自动下载，之后运行会载入本地文件。" 45 | ] 46 | }, 47 | { 48 | "cell_type": "code", 49 | "execution_count": null, 50 | "metadata": { 51 | "collapsed": true 52 | }, 53 | "outputs": [], 54 | "source": [ 55 | "(x_train,y_train),(x_test,y_test)=mnist.load_data()" 56 | ] 57 | }, 58 | { 59 | "cell_type": "markdown", 60 | "metadata": {}, 61 | "source": [ 62 | "↓查看一下数据格式，训练集一共有6万张，大小是28*28,单通道灰度图，测试集是1000张。标签是列向量" 63 | ] 64 | }, 65 | { 66 | "cell_type": "code", 67 | "execution_count": null, 68 | "metadata": {}, 69 | "outputs": [], 70 | "source": [ 71 | "print(x_train.shape,y_train.shape)\n", 72 | "print(x_test.shape,y_test.shape)" 73 | ] 74 | }, 75 | { 76 | "cell_type": "markdown", 77 | "metadata": {}, 78 | "source": [ 79 | "↓可视化一些图片" 80 | ] 81 | }, 82 | { 83 | "cell_type": "code", 84 | "execution_count": null, 85 | "metadata": {}, 86 | "outputs": [], 87 | "source": [ 88 | "import matplotlib.pyplot as plt\n", 89 | "im = plt.imshow(x_train[0],cmap='gray')\n", 90 | "plt.show()\n", 91 | "im2 = plt.imshow(x_train[1],cmap='gray')\n", 92 | "plt.show()" 93 | ] 94 | }, 95 | { 96 | "cell_type": "markdown", 97 | "metadata": {}, 98 | "source": [ 99 | "由于mlp的输入是一维向量，所以要转换\n", 100 | "\n", 101 | "将每一幅图像都转换为一个长向量，大小为28*28=784" 102 | ] 103 | }, 104 | { 105 | "cell_type": "code", 106 | "execution_count": null, 107 | "metadata": {}, 108 | "outputs": [], 109 | "source": [ 110 | "x_train = x_train.reshape(60000,784)\n", 111 | "x_test = x_test.reshape(10000,784)\n", 112 | "# x_train = x_train.astype('float32')\n", 113 | "# x_train = x_train.astype('float32')\n", 114 | "print(x_train.shape)" 115 | ] 116 | }, 117 | { 118 | "cell_type": "markdown", 119 | "metadata": {}, 120 | "source": [ 121 | "归一化，将图像的像素归到0~1" 122 | ] 123 | }, 124 | { 125 | "cell_type": "code", 126 | "execution_count": null, 127 | "metadata": { 128 | "collapsed": true 129 | }, 130 | "outputs": [], 131 | "source": [ 132 | "x_train = x_train/255\n", 133 | "x_test = x_test/255" 134 | ] 135 | }, 136 | { 137 | "cell_type": "markdown", 138 | "metadata": {}, 139 | "source": [ 140 | "将label也转换成One-hot标签，这里直接用keras的预置的一个函数 keras.utils.to_categorical" 141 | ] 142 | }, 143 | { 144 | "cell_type": "code", 145 | "execution_count": null, 146 | "metadata": {}, 147 | "outputs": [], 148 | "source": [ 149 | "print(y_train[0:10])# 查看原始标签 0~9" 150 | ] 151 | }, 152 | { 153 | "cell_type": "code", 154 | "execution_count": null, 155 | "metadata": {}, 156 | "outputs": [], 157 | "source": [ 158 | "y_train = keras.utils.to_categorical(y_train,10)\n", 159 | "y_test = keras.utils.to_categorical(y_test,10)\n", 160 | "\n", 161 | "print(y_train[0:10])#查看转换完毕的标签" 162 | ] 163 | }, 164 | { 165 | "cell_type": "markdown", 166 | "metadata": {}, 167 | "source": [ 168 | "开始构建模型，模型分包含两个隐层和一个输出层,都是全连接层，使用Sequential构建\n", 169 | "\n", 170 | "其中隐层输出采用ReLU激活函数，Sequential的第一层要指定input_shape，要注意，这里的input_shape 是不包含batch大小的，就只是后面几维" 171 | ] 172 | }, 173 | { 174 | "cell_type": "code", 175 | "execution_count": null, 176 | "metadata": { 177 | "collapsed": true 178 | }, 179 | "outputs": [], 180 | "source": [ 181 | "model = Sequential()\n", 182 | "model.add(Dense(512,activation='relu',input_shape=(784,)))\n", 183 | "model.add(Dropout(0.2))\n", 184 | "model.add(Dense(512,activation='relu'))\n", 185 | "model.add(Dropout(0.2))\n", 186 | "model.add(Dense(10,activation='softmax'))" 187 | ] 188 | }, 189 | { 190 | "cell_type": "code", 191 | "execution_count": null, 192 | "metadata": {}, 193 | "outputs": [], 194 | "source": [ 195 | "model.summary()#这一句用来输出网络结构" 196 | ] 197 | }, 198 | { 199 | "cell_type": "markdown", 200 | "metadata": {}, 201 | "source": [ 202 | "配置模型，主要包括\n", 203 | "loss：loss计算方法（损失函数）\n", 204 | "\n", 205 | "optimizer：优化函数\n", 206 | "\n", 207 | "metrics：指定哪些量需要在训练及测试中关注，一般都会写accuracy" 208 | ] 209 | }, 210 | { 211 | "cell_type": "code", 212 | "execution_count": null, 213 | "metadata": {}, 214 | "outputs": [], 215 | "source": [ 216 | "model.compile(loss='categorical_crossentropy',\n", 217 | " optimizer=RMSprop(),\n", 218 | " metrics=['accuracy'])" 219 | ] 220 | }, 221 | { 222 | "cell_type": "markdown", 223 | "metadata": {}, 224 | "source": [ 225 | "开始训练。这里使用的是model对象的fit方法。前两个参数分别是**完整的训练数据和训练标签**\n", 226 | "\n", 227 | "batch_size 表示每一次塞入多少张图片\n", 228 | "\n", 229 | "epochs 表示训练几轮\n", 230 | "\n", 231 | "verbose 表示用何种方式显示输出信息，0表示不输出，1表示在一直输出更新，2表示每一个epoch才输出一次。\n", 232 | "\n", 233 | "validation_data 表示验证集，格式和训练集一样，如果此参数不为空的话，每一个epoch过后就会输出验证集的loss和accuracy" 234 | ] 235 | }, 236 | { 237 | "cell_type": "code", 238 | "execution_count": null, 239 | "metadata": {}, 240 | "outputs": [], 241 | "source": [ 242 | "model.fit(x_train,y_train,batch_size=64,epochs=2,verbose=1,\n", 243 | " validation_data=(x_test,y_test))" 244 | ] 245 | }, 246 | { 247 | "cell_type": "markdown", 248 | "metadata": {}, 249 | "source": [ 250 | "测试结果，输出为loss以及其他之前compile模型时指定过的metrics的值" 251 | ] 252 | }, 253 | { 254 | "cell_type": "code", 255 | "execution_count": null, 256 | "metadata": {}, 257 | "outputs": [], 258 | "source": [ 259 | "score = model.evaluate(x_test,y_test,verbose=1)\n", 260 | "print('Test loss:',score[0])\n", 261 | "print('Test accuracy',score[1])" 262 | ] 263 | }, 264 | { 265 | "cell_type": "markdown", 266 | "metadata": {}, 267 | "source": [ 268 | "## 总结\n", 269 | "\n", 270 | "本文主要写了一个最简单的多层感知机模型，目的是熟悉keras最基本的操作。\n", 271 | "\n", 272 | "知识点：\n", 273 | "\n", 274 | "1. 学习载入Keras中预置的数据库及数据库数据的基本变换\n", 275 | "1. Sequential模型的定义，以及如何添加层\n", 276 | "1. 如何对Dense层及Dropout层进行基本的配置\n", 277 | "1. 学习使用compile对网络进行配置\n", 278 | "1. 使用fit方法来对小数据库进行训练，这里的小数据库指的是所有数据可以一次性载入到内存\n", 279 | "1. 使用evaluate方法来对模型进行效果评估\n", 280 | "\n", 281 | "参考：\n", 282 | "> https://github.com/keras-team/keras/tree/master/examples" 283 | ] 284 | }, 285 | { 286 | "cell_type": "code", 287 | "execution_count": null, 288 | "metadata": { 289 | "collapsed": true 290 | }, 291 | "outputs": [], 292 | "source": [] 293 | } 294 | ], 295 | "metadata": { 296 | "kernelspec": { 297 | "display_name": "Python 3", 298 | "language": "python", 299 | "name": "python3" 300 | }, 301 | "language_info": { 302 | "codemirror_mode": { 303 | "name": "ipython", 304 | "version": 3 305 | }, 306 | "file_extension": ".py", 307 | "mimetype": "text/x-python", 308 | "name": "python", 309 | "nbconvert_exporter": "python", 310 | "pygments_lexer": "ipython3", 311 | "version": "3.6.2" 312 | } 313 | }, 314 | "nbformat": 4, 315 | "nbformat_minor": 2 316 | } 317 | -------------------------------------------------------------------------------- /class_2.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Keras入门课2：使用CNN识别mnist手写数字" 8 | ] 9 | }, 10 | { 11 | "cell_type": "code", 12 | "execution_count": null, 13 | "metadata": {}, 14 | "outputs": [], 15 | "source": [ 16 | "import keras\n", 17 | "from keras.datasets import mnist\n", 18 | "from keras.models import Sequential\n", 19 | "from keras.layers import Dense, Dropout, Flatten\n", 20 | "from keras.layers import Conv2D, MaxPooling2D\n", 21 | "from keras import backend as K" 22 | ] 23 | }, 24 | { 25 | "cell_type": "code", 26 | "execution_count": null, 27 | "metadata": {}, 28 | "outputs": [], 29 | "source": [ 30 | "(x_train,y_train),(x_test,y_test) = mnist.load_data() # out: np.ndarray\n", 31 | "print(x_train.shape,y_train.shape)\n", 32 | "print(x_test.shape,y_test.shape)" 33 | ] 34 | }, 35 | { 36 | "cell_type": "markdown", 37 | "metadata": {}, 38 | "source": [ 39 | "↓可视化一些图片" 40 | ] 41 | }, 42 | { 43 | "cell_type": "code", 44 | "execution_count": null, 45 | "metadata": {}, 46 | "outputs": [], 47 | "source": [ 48 | "import matplotlib.pyplot as plt\n", 49 | "im = plt.imshow(x_train[0],cmap='gray')\n", 50 | "plt.show()\n", 51 | "im2 = plt.imshow(x_train[1],cmap='gray')\n", 52 | "plt.show()" 53 | ] 54 | }, 55 | { 56 | "cell_type": "code", 57 | "execution_count": null, 58 | "metadata": {}, 59 | "outputs": [], 60 | "source": [ 61 | "print(K.image_data_format())" 62 | ] 63 | }, 64 | { 65 | "cell_type": "markdown", 66 | "metadata": {}, 67 | "source": [ 68 | "这里用卷积神经网络来对图像做特征处理，一般来说，输入到网络的图像格式有以下两种：\n", 69 | "1. channels_first (batch_size,channels,width,height)\n", 70 | "1. channels_last (batch_size,width,height,channels)\n", 71 | "\n", 72 | "这里channels指的是通道数，灰度图是单通道channels=1，彩色图是三通道channels=3，需要注意的是，即使图像是单通道的，输入数据的维度依然是4维。反观我们的mnist图像数据，只有三维，所以我们要手动把channels这个维度加上。由于Keras使用不同后端的时候，数据格式不一样，所以要分情况进行维度增加\n", 73 | "\n", 74 | "值得注意的是，reshape函数第一个参数为-1，意思为保持当前维度不变" 75 | ] 76 | }, 77 | { 78 | "cell_type": "code", 79 | "execution_count": null, 80 | "metadata": {}, 81 | "outputs": [], 82 | "source": [ 83 | "\n", 84 | "if K.image_data_format()=='channels_first':\n", 85 | " x_train = x_train.reshape(-1,1,28,28)\n", 86 | " x_test = x_test.reshape(-1,1,28,28)\n", 87 | " input_shape = (1,28,28)\n", 88 | "else:\n", 89 | " x_train = x_train.reshape(-1,28,28,1)\n", 90 | " x_test = x_test.reshape(-1,28,28,1)\n", 91 | " input_shape = (28,28,1)" 92 | ] 93 | }, 94 | { 95 | "cell_type": "code", 96 | "execution_count": null, 97 | "metadata": {}, 98 | "outputs": [], 99 | "source": [ 100 | "print(x_train.shape,x_test.shape)" 101 | ] 102 | }, 103 | { 104 | "cell_type": "markdown", 105 | "metadata": {}, 106 | "source": [ 107 | "↓数据归一化" 108 | ] 109 | }, 110 | { 111 | "cell_type": "code", 112 | "execution_count": null, 113 | "metadata": {}, 114 | "outputs": [], 115 | "source": [ 116 | "x_train = x_train/255\n", 117 | "x_test = x_test/255" 118 | ] 119 | }, 120 | { 121 | "cell_type": "code", 122 | "execution_count": null, 123 | "metadata": {}, 124 | "outputs": [], 125 | "source": [ 126 | "y_train = keras.utils.to_categorical(y_train,10)\n", 127 | "y_test = keras.utils.to_categorical(y_test,10)" 128 | ] 129 | }, 130 | { 131 | "cell_type": "markdown", 132 | "metadata": {}, 133 | "source": [ 134 | "↓构建网络模型" 135 | ] 136 | }, 137 | { 138 | "cell_type": "code", 139 | "execution_count": null, 140 | "metadata": {}, 141 | "outputs": [], 142 | "source": [ 143 | "model = Sequential()\n", 144 | "model.add(Conv2D(filters = 32,kernel_size=(3,3),\n", 145 | " activation='relu',input_shape = input_shape))\n", 146 | "model.add(Conv2D(64,(3,3),activation='relu'))\n", 147 | "model.add(MaxPooling2D(pool_size=(2,2)))\n", 148 | "model.add(Dropout(0.25))#25%的参数会被舍弃\n", 149 | "model.add(Flatten())\n", 150 | "model.add(Dense(128,activation='relu'))\n", 151 | "model.add(Dropout(0.5))\n", 152 | "model.add(Dense(10,activation='softmax'))" 153 | ] 154 | }, 155 | { 156 | "cell_type": "code", 157 | "execution_count": null, 158 | "metadata": {}, 159 | "outputs": [], 160 | "source": [ 161 | "model.summary()" 162 | ] 163 | }, 164 | { 165 | "cell_type": "code", 166 | "execution_count": null, 167 | "metadata": {}, 168 | "outputs": [], 169 | "source": [ 170 | "model.compile(loss = keras.losses.categorical_crossentropy,\n", 171 | " optimizer = keras.optimizers.Adadelta(),\n", 172 | " metrics=['accuracy'])" 173 | ] 174 | }, 175 | { 176 | "cell_type": "code", 177 | "execution_count": null, 178 | "metadata": {}, 179 | "outputs": [], 180 | "source": [ 181 | "model.fit(x_train,y_train,batch_size=64,epochs=2\n", 182 | " ,verbose=1,validation_data=(x_test,y_test))" 183 | ] 184 | }, 185 | { 186 | "cell_type": "code", 187 | "execution_count": null, 188 | "metadata": {}, 189 | "outputs": [], 190 | "source": [ 191 | "score = model.evaluate(x_test, y_test, verbose=0)\n", 192 | "print('Test loss:', score[0])\n", 193 | "print('Test accuracy:', score[1])" 194 | ] 195 | }, 196 | { 197 | "cell_type": "markdown", 198 | "metadata": {}, 199 | "source": [ 200 | "## 总结\n", 201 | "\n", 202 | "1. 学习了如何根据不同的模型数据要求，给原始数据图像增加维度\n", 203 | "2. 学习了Conv2D卷积层和MaxPooling2D池化层的使用\n", 204 | "\n", 205 | "本文代码地址：https://github.com/tsycnh/Keras-Tutorials/blob/master/class_2.ipynb\n", 206 | "\n", 207 | "参考：\n", 208 | "> https://github.com/keras-team/keras/tree/master/examples" 209 | ] 210 | }, 211 | { 212 | "cell_type": "code", 213 | "execution_count": null, 214 | "metadata": { 215 | "collapsed": true 216 | }, 217 | "outputs": [], 218 | "source": [] 219 | } 220 | ], 221 | "metadata": { 222 | "kernelspec": { 223 | "display_name": "Python 3", 224 | "language": "python", 225 | "name": "python3" 226 | }, 227 | "language_info": { 228 | "codemirror_mode": { 229 | "name": "ipython", 230 | "version": 3 231 | }, 232 | "file_extension": ".py", 233 | "mimetype": "text/x-python", 234 | "name": "python", 235 | "nbconvert_exporter": "python", 236 | "pygments_lexer": "ipython3", 237 | "version": "3.6.2" 238 | } 239 | }, 240 | "nbformat": 4, 241 | "nbformat_minor": 2 242 | } 243 | -------------------------------------------------------------------------------- /class_3.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Keras入门课3：使用CNN识别cifar10数据集" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "cifar10是一个日常物品的数据集，一共有10类，属于是比较小的数据集。这次用一个4个卷积层加2个全连接层的典型CNN网络来进行分类" 15 | ] 16 | }, 17 | { 18 | "cell_type": "code", 19 | "execution_count": null, 20 | "metadata": {}, 21 | "outputs": [], 22 | "source": [ 23 | "import keras\n", 24 | "from keras.datasets import cifar10\n", 25 | "from keras.preprocessing.image import ImageDataGenerator\n", 26 | "from keras.models import Sequential\n", 27 | "from keras.layers import Dense, Dropout, Activation, Flatten\n", 28 | "from keras.layers import Conv2D, MaxPooling2D" 29 | ] 30 | }, 31 | { 32 | "cell_type": "markdown", 33 | "metadata": {}, 34 | "source": [ 35 | "↓首先载入cifar10数据集，和mnist数据集的载入方法一致，本地没有数据的话会先下载" 36 | ] 37 | }, 38 | { 39 | "cell_type": "code", 40 | "execution_count": null, 41 | "metadata": {}, 42 | "outputs": [], 43 | "source": [ 44 | "(x_train,y_train),(x_test,y_test) = cifar10.load_data()" 45 | ] 46 | }, 47 | { 48 | "cell_type": "markdown", 49 | "metadata": {}, 50 | "source": [ 51 | "cifar10数据集图像大小是32*32的3通道彩图，训练集5万张，测试集1万张。和之前的mnist数据集不同，由于是彩色的，所以样本直接就是4维的。" 52 | ] 53 | }, 54 | { 55 | "cell_type": "code", 56 | "execution_count": null, 57 | "metadata": {}, 58 | "outputs": [], 59 | "source": [ 60 | "print(x_train.shape,y_train.shape)\n", 61 | "print(x_test.shape,y_test.shape)" 62 | ] 63 | }, 64 | { 65 | "cell_type": "code", 66 | "execution_count": null, 67 | "metadata": {}, 68 | "outputs": [], 69 | "source": [ 70 | "import matplotlib.pyplot as plt\n", 71 | "plt.imshow(x_train[0])\n", 72 | "plt.show()\n", 73 | "plt.imshow(x_train[1])\n", 74 | "plt.show()" 75 | ] 76 | }, 77 | { 78 | "cell_type": "markdown", 79 | "metadata": {}, 80 | "source": [ 81 | "可以看到数据读入没有问题，第一张是蛤蟆，第二张是一个卡车。\n", 82 | "\n", 83 | "↓规范化数据" 84 | ] 85 | }, 86 | { 87 | "cell_type": "code", 88 | "execution_count": null, 89 | "metadata": { 90 | "collapsed": true 91 | }, 92 | "outputs": [], 93 | "source": [ 94 | "x_train = x_train/255\n", 95 | "x_test = x_test/255\n", 96 | "y_train = keras.utils.to_categorical(y_train,10)\n", 97 | "y_test = keras.utils.to_categorical(y_test,10)" 98 | ] 99 | }, 100 | { 101 | "cell_type": "markdown", 102 | "metadata": {}, 103 | "source": [ 104 | "↓构建模型。之前构建模型都是先生成一个model，然后使用add方法来一层一层的加，现在用另一种更方便的方法。直接在Sequential初始化的时候按数组一个一个写进去就可以了。" 105 | ] 106 | }, 107 | { 108 | "cell_type": "code", 109 | "execution_count": null, 110 | "metadata": {}, 111 | "outputs": [], 112 | "source": [ 113 | "model = Sequential([\n", 114 | " Conv2D(32,(3,3),padding='same',input_shape=(32,32,3),activation='relu'),\n", 115 | " Conv2D(32,(3,3),activation='relu'),\n", 116 | " MaxPooling2D(pool_size=(2,2)),\n", 117 | " Dropout(0.25),\n", 118 | " \n", 119 | " Conv2D(64,(3,3),padding='same',activation='relu'),\n", 120 | " Conv2D(64,(3,3),activation='relu'),\n", 121 | " MaxPooling2D(pool_size=(2,2)),\n", 122 | " Dropout(0.25),\n", 123 | " \n", 124 | " Flatten(),\n", 125 | " Dense(512,activation='relu'),\n", 126 | " Dropout(0.5),\n", 127 | " Dense(10,activation='softmax') \n", 128 | "])" 129 | ] 130 | }, 131 | { 132 | "cell_type": "code", 133 | "execution_count": null, 134 | "metadata": {}, 135 | "outputs": [], 136 | "source": [ 137 | "model.summary()" 138 | ] 139 | }, 140 | { 141 | "cell_type": "markdown", 142 | "metadata": {}, 143 | "source": [ 144 | "↓指定优化函数的参数" 145 | ] 146 | }, 147 | { 148 | "cell_type": "code", 149 | "execution_count": null, 150 | "metadata": { 151 | "collapsed": true 152 | }, 153 | "outputs": [], 154 | "source": [ 155 | "opt = keras.optimizers.rmsprop(lr=0.0001,decay=1e-6)" 156 | ] 157 | }, 158 | { 159 | "cell_type": "code", 160 | "execution_count": null, 161 | "metadata": { 162 | "collapsed": true 163 | }, 164 | "outputs": [], 165 | "source": [ 166 | "model.compile(loss='categorical_crossentropy',\n", 167 | " optimizer=opt,\n", 168 | " metrics=['accuracy'])" 169 | ] 170 | }, 171 | { 172 | "cell_type": "markdown", 173 | "metadata": {}, 174 | "source": [ 175 | "### 至此直接调用fit方法就可以进行训练了。但是为了模型更快的收敛以及更好的泛化性能，往往我们会对图像做一些变换，比如缩放、平移、旋转等等。下面我们要用keras自带的图像增强来对图像做一些变换" 176 | ] 177 | }, 178 | { 179 | "cell_type": "markdown", 180 | "metadata": {}, 181 | "source": [ 182 | "↓这里生成了一个数据增强器，包含了范围20°内的随机旋转，±15%的缩放以及随机的水平翻转。可调的参数还有很多，具体的可以查看文档。" 183 | ] 184 | }, 185 | { 186 | "cell_type": "code", 187 | "execution_count": null, 188 | "metadata": { 189 | "collapsed": true 190 | }, 191 | "outputs": [], 192 | "source": [ 193 | "datagen = ImageDataGenerator(\n", 194 | " rotation_range = 20,\n", 195 | " zoom_range = 0.15,\n", 196 | " horizontal_flip = True,\n", 197 | ")" 198 | ] 199 | }, 200 | { 201 | "cell_type": "code", 202 | "execution_count": null, 203 | "metadata": {}, 204 | "outputs": [], 205 | "source": [ 206 | "# datagen.fit(x_train) 只有使用featurewise_center，featurewise_std_normalization或zca_whitening时需要此函数" 207 | ] 208 | }, 209 | { 210 | "cell_type": "markdown", 211 | "metadata": {}, 212 | "source": [ 213 | "↓通过ImageDataGenerator生成的数据需要使用model的fit_generator方法来进行训练，其中的workers参数表示多线程运算。\n", 214 | "\n", 215 | "datagen的flow方法可以按批次的生成训练所需数据，注意这里生成的数据都是经过了数据增强的，并且是实时的。" 216 | ] 217 | }, 218 | { 219 | "cell_type": "code", 220 | "execution_count": null, 221 | "metadata": { 222 | "scrolled": true 223 | }, 224 | "outputs": [], 225 | "source": [ 226 | "model.fit_generator(datagen.flow(x_train,y_train,batch_size=64),steps_per_epoch = 1000,epochs = 2,\n", 227 | " validation_data=(x_test,y_test),workers=4,verbose=1)" 228 | ] 229 | }, 230 | { 231 | "cell_type": "markdown", 232 | "metadata": {}, 233 | "source": [ 234 | "↓保存模型，包括了模型的结构以及参数。后缀用h5" 235 | ] 236 | }, 237 | { 238 | "cell_type": "code", 239 | "execution_count": null, 240 | "metadata": {}, 241 | "outputs": [], 242 | "source": [ 243 | "model.save('cifar10_trained_model.h5')" 244 | ] 245 | }, 246 | { 247 | "cell_type": "code", 248 | "execution_count": null, 249 | "metadata": {}, 250 | "outputs": [], 251 | "source": [ 252 | "scores = model.evaluate(x_test,y_test,verbose=1)\n", 253 | "print('Test loss:',scores[0])\n", 254 | "print('Test accuracy:',scores[1])" 255 | ] 256 | }, 257 | { 258 | "cell_type": "markdown", 259 | "metadata": {}, 260 | "source": [ 261 | "### 总结\n", 262 | "\n", 263 | "1. 学习了一种新的使用Sequential()构建模型的方法，更加的简单快捷\n", 264 | "1. 学习了使用Keras内置的ImageDataGenerator来做数据增强的方法\n", 265 | "1. 调用model的fit_generator来进行针对增强数据的训练\n", 266 | "1. 学习了如何保存模型\n", 267 | "\n", 268 | "本文代码链接：https://github.com/tsycnh/Keras-Tutorials/blob/master/class_3.ipynb\n", 269 | "\n", 270 | "参考\n", 271 | "> https://github.com/keras-team/keras/blob/master/examples\n", 272 | "> https://keras-cn.readthedocs.io/en/latest/preprocessing/image/" 273 | ] 274 | } 275 | ], 276 | "metadata": { 277 | "kernelspec": { 278 | "display_name": "Python 3", 279 | "language": "python", 280 | "name": "python3" 281 | }, 282 | "language_info": { 283 | "codemirror_mode": { 284 | "name": "ipython", 285 | "version": 3 286 | }, 287 | "file_extension": ".py", 288 | "mimetype": "text/x-python", 289 | "name": "python", 290 | "nbconvert_exporter": "python", 291 | "pygments_lexer": "ipython3", 292 | "version": "3.6.2" 293 | } 294 | }, 295 | "nbformat": 4, 296 | "nbformat_minor": 2 297 | } 298 | -------------------------------------------------------------------------------- /class_4.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Keras入门课4：使用ResNet识别cifar10数据集" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "前面几节课都是用一些简单的网络来做图像识别，这节课我们要使用经典的ResNet网络对cifar10进行分类。\n", 15 | "\n", 16 | "ResNet是何凯明大神提出的残差网络，具体论文见此 \n", 17 | "\n", 18 | "ResNet v1 \n", 19 | "Deep Residual Learning for Image Recognition \n", 20 | "https://arxiv.org/pdf/1512.03385.pdf \n", 21 | "ResNet v2 \n", 22 | "Identity Mappings in Deep Residual Networks \n", 23 | "https://arxiv.org/pdf/1603.05027.pdf \n", 24 | "\n", 25 | "这一节课，我们只动手实现v1的一个精简版本（因为数据集cifar10的数据比较小）" 26 | ] 27 | }, 28 | { 29 | "cell_type": "code", 30 | "execution_count": null, 31 | "metadata": { 32 | "collapsed": true 33 | }, 34 | "outputs": [], 35 | "source": [ 36 | "import keras\n", 37 | "from keras.layers import Dense, Conv2D, BatchNormalization, Activation\n", 38 | "from keras.layers import AveragePooling2D, Input, Flatten\n", 39 | "from keras.optimizers import Adam\n", 40 | "from keras.regularizers import l2\n", 41 | "from keras import backend as K\n", 42 | "from keras.models import Model\n", 43 | "from keras.datasets import cifar10\n", 44 | "from keras.callbacks import ModelCheckpoint, LearningRateScheduler\n", 45 | "from keras.callbacks import ReduceLROnPlateau\n", 46 | "import numpy as np\n", 47 | "import os" 48 | ] 49 | }, 50 | { 51 | "cell_type": "code", 52 | "execution_count": null, 53 | "metadata": { 54 | "collapsed": true 55 | }, 56 | "outputs": [], 57 | "source": [ 58 | "(x_train, y_train), (x_test, y_test) = cifar10.load_data()" 59 | ] 60 | }, 61 | { 62 | "cell_type": "code", 63 | "execution_count": null, 64 | "metadata": { 65 | "collapsed": true 66 | }, 67 | "outputs": [], 68 | "source": [ 69 | "x_train = x_train/255\n", 70 | "x_test = x_test/255\n", 71 | "y_train = keras.utils.to_categorical(y_train,10)\n", 72 | "y_test = keras.utils.to_categorical(y_test,10)" 73 | ] 74 | }, 75 | { 76 | "cell_type": "markdown", 77 | "metadata": {}, 78 | "source": [ 79 | "↓构建模型基本模块，ResNet Block\n", 80 | "\n", 81 | "这里没有用Sequential模型，而是用了另外一种构建模型的方法，即函数式模型（Functional)\n", 82 | "Sequential模型有一个缺陷，即网络只能一层一层的堆叠起来，无法处理分支网络的情况。比如ResNet或GoogleNet中的Inception模块。使用Functional模型，构建起模型来十分自由，可以组合成各种各样的网络，可以说Sequential模型是Functional模型的一个子集。\n", 83 | "\n", 84 | "使用函数式模型很简单，直接在网络层模块后写一个括号，参数就是当前层的输入值，返回值就是当前层的输出值，比如：net = Conv2D(...)(inputs)" 85 | ] 86 | }, 87 | { 88 | "cell_type": "markdown", 89 | "metadata": {}, 90 | "source": [ 91 | "![](./images/resnetblock.png)" 92 | ] 93 | }, 94 | { 95 | "cell_type": "markdown", 96 | "metadata": {}, 97 | "source": [ 98 | "↓首先构建一个基本的block模块，就是上图的weight layer，这个模块包含了一个卷积层，一个BN层，一个激活层。可以看到上图下面那个layer没有激活层，所以函数内做了一个判断\n", 99 | "\n", 100 | "BN层的作用是对输出参数做归一化，可以有效使网络更易训练。一般来说，加了BN层的网络，可以不必再用Dropout层。\n", 101 | "同时这一次我们在卷积层中加入了L2正则化，目的是提升模型的泛化能力。" 102 | ] 103 | }, 104 | { 105 | "cell_type": "code", 106 | "execution_count": null, 107 | "metadata": { 108 | "collapsed": true 109 | }, 110 | "outputs": [], 111 | "source": [ 112 | "#ResNet Block\n", 113 | "def resnet_block(inputs,num_filters=16,\n", 114 | " kernel_size=3,strides=1,\n", 115 | " activation='relu'):\n", 116 | " x = Conv2D(num_filters,kernel_size=kernel_size,strides=strides,padding='same',\n", 117 | " kernel_initializer='he_normal',kernel_regularizer=l2(1e-4))(inputs)\n", 118 | " x = BatchNormalization()(x)\n", 119 | " if(activation):\n", 120 | " x = Activation('relu')(x)\n", 121 | " return x" 122 | ] 123 | }, 124 | { 125 | "cell_type": "markdown", 126 | "metadata": {}, 127 | "source": [ 128 | "↓这里构建整个网络。由于我们要处理的图像较小，所以ResNet用的也是一个20层的小号版。\n", 129 | "总体上分为五大部分。上面那张图我们称之为一个building block\n", 130 | "\n", 131 | "输入层 \n", 132 | "↓ \n", 133 | "6层 filter大小为16的building block \n", 134 | "↓ \n", 135 | "6层 filter大小为32的building block \n", 136 | "↓ \n", 137 | "6层 filter大小为64的building block \n", 138 | "↓ \n", 139 | "一层全连接\n", 140 | "一层输出层 \n", 141 | "第2~7层属于一个很规整的层叠加，每一个循环里都是在搭建一个building block \n", 142 | "第8~13层里面的首层的strides=2，这样输出就是16*16*32大小的张量，而输入是32*32*16大小的张量，所以对输入又做了一个卷积操作，使得其shape和正常卷积层的输出一直，这样才可以执行add操作。\n", 143 | "第14~19层套路一样\n", 144 | "\n", 145 | "返回为通过Model初始化过的一个模型" 146 | ] 147 | }, 148 | { 149 | "cell_type": "code", 150 | "execution_count": null, 151 | "metadata": { 152 | "collapsed": true 153 | }, 154 | "outputs": [], 155 | "source": [ 156 | "# 建一个20层的ResNet网络 \n", 157 | "def resnet_v1(input_shape):\n", 158 | " inputs = Input(shape=input_shape)# Input层，用来当做占位使用\n", 159 | " \n", 160 | " #第一层\n", 161 | " x = resnet_block(inputs)\n", 162 | " print('layer1,xshape:',x.shape)\n", 163 | " # 第2~7层\n", 164 | " for i in range(6):\n", 165 | " a = resnet_block(inputs = x)\n", 166 | " b = resnet_block(inputs=a,activation=None)\n", 167 | " x = keras.layers.add([x,b])\n", 168 | " x = Activation('relu')(x)\n", 169 | " # out：32*32*16\n", 170 | " # 第8~13层\n", 171 | " for i in range(6):\n", 172 | " if i == 0:\n", 173 | " a = resnet_block(inputs = x,strides=2,num_filters=32)\n", 174 | " else:\n", 175 | " a = resnet_block(inputs = x,num_filters=32)\n", 176 | " b = resnet_block(inputs=a,activation=None,num_filters=32)\n", 177 | " if i==0:\n", 178 | " x = Conv2D(32,kernel_size=3,strides=2,padding='same',\n", 179 | " kernel_initializer='he_normal',kernel_regularizer=l2(1e-4))(x)\n", 180 | " x = keras.layers.add([x,b])\n", 181 | " x = Activation('relu')(x)\n", 182 | " # out:16*16*32\n", 183 | " # 第14~19层\n", 184 | " for i in range(6):\n", 185 | " if i ==0 :\n", 186 | " a = resnet_block(inputs = x,strides=2,num_filters=64)\n", 187 | " else:\n", 188 | " a = resnet_block(inputs = x,num_filters=64)\n", 189 | "\n", 190 | " b = resnet_block(inputs=a,activation=None,num_filters=64)\n", 191 | " if i == 0:\n", 192 | " x = Conv2D(64,kernel_size=3,strides=2,padding='same',\n", 193 | " kernel_initializer='he_normal',kernel_regularizer=l2(1e-4))(x)\n", 194 | " x = keras.layers.add([x,b])# 相加操作，要求x、b shape完全一致\n", 195 | " x = Activation('relu')(x)\n", 196 | " # out:8*8*64\n", 197 | " # 第20层 \n", 198 | " x = AveragePooling2D(pool_size=2)(x)\n", 199 | " # out:4*4*64\n", 200 | " y = Flatten()(x)\n", 201 | " # out:1024\n", 202 | " outputs = Dense(10,activation='softmax',\n", 203 | " kernel_initializer='he_normal')(y)\n", 204 | " \n", 205 | " #初始化模型\n", 206 | " #之前的操作只是将多个神经网络层进行了相连，通过下面这一句的初始化操作，才算真正完成了一个模型的结构初始化\n", 207 | " model = Model(inputs=inputs,outputs=outputs)\n", 208 | " return model" 209 | ] 210 | }, 211 | { 212 | "cell_type": "code", 213 | "execution_count": null, 214 | "metadata": { 215 | "collapsed": true 216 | }, 217 | "outputs": [], 218 | "source": [ 219 | "model = resnet_v1((32,32,3))\n", 220 | "print(model)" 221 | ] 222 | }, 223 | { 224 | "cell_type": "code", 225 | "execution_count": null, 226 | "metadata": { 227 | "collapsed": true 228 | }, 229 | "outputs": [], 230 | "source": [ 231 | "model.compile(loss='categorical_crossentropy',\n", 232 | "optimizer=Adam(),\n", 233 | "metrics=['accuracy'])\n", 234 | "\n", 235 | "model.summary()" 236 | ] 237 | }, 238 | { 239 | "cell_type": "markdown", 240 | "metadata": {}, 241 | "source": [ 242 | "↓callbacks \n", 243 | "model的.fit方法有一个参数是callbacks，这个参数可以传入一些其他待执行的函数，在训练过程中，每一个epoch会调用一次列表中的callbacks \n", 244 | "\n", 245 | "本次课程用到的几个回调函数 \n", 246 | "ModelCheckpoint：用来每个epoch存储一遍模型 \n", 247 | "LearningRateScheduler:用来动态调整学习率。其输入为一个函数，该函数的输入为当前epoch数，返回为对应的学习率 \n", 248 | "ReduceLROnPlateau:用来在训练停滞不前的时候动态降低学习率。" 249 | ] 250 | }, 251 | { 252 | "cell_type": "code", 253 | "execution_count": null, 254 | "metadata": { 255 | "collapsed": true 256 | }, 257 | "outputs": [], 258 | "source": [ 259 | "checkpoint = ModelCheckpoint(filepath='./cifar10_resnet_ckpt.h5',monitor='val_acc',\n", 260 | " verbose=1,save_best_only=True)\n", 261 | "def lr_sch(epoch):\n", 262 | " #200 total\n", 263 | " if epoch <50:\n", 264 | " return 1e-3\n", 265 | " if 50<=epoch<100:\n", 266 | " return 1e-4\n", 267 | " if epoch>=100:\n", 268 | " return 1e-5\n", 269 | "lr_scheduler = LearningRateScheduler(lr_sch)\n", 270 | "lr_reducer = ReduceLROnPlateau(monitor='val_acc',factor=0.2,patience=5,\n", 271 | " mode='max',min_lr=1e-3)\n", 272 | "callbacks = [checkpoint,lr_scheduler,lr_reducer]" 273 | ] 274 | }, 275 | { 276 | "cell_type": "code", 277 | "execution_count": null, 278 | "metadata": { 279 | "collapsed": true 280 | }, 281 | "outputs": [], 282 | "source": [ 283 | "model.fit(x_train,y_train,batch_size=64,epochs=200,validation_data=(x_test,y_test),verbose=1,callbacks=callbacks)" 284 | ] 285 | }, 286 | { 287 | "cell_type": "code", 288 | "execution_count": null, 289 | "metadata": { 290 | "collapsed": true 291 | }, 292 | "outputs": [], 293 | "source": [ 294 | "scores = model.evaluate(x_test,y_test,verbose=1)\n", 295 | "print('Test loss:',scores[0])\n", 296 | "print('Test accuracy:',scores[1])" 297 | ] 298 | }, 299 | { 300 | "cell_type": "markdown", 301 | "metadata": {}, 302 | "source": [ 303 | "通过了200个批次的训练，训练集的准确率已经达到了100%，测试集达到了82.44%。这还是没有使用数据增强的效果，如果使用数据增强，准确率可以达到90+%" 304 | ] 305 | }, 306 | { 307 | "cell_type": "markdown", 308 | "metadata": {}, 309 | "source": [ 310 | "## 总结\n", 311 | "1. 学习了一种新的构建模型的方法，函数式模型（Functional），更自由灵活\n", 312 | "1. 学习了如何将通过Functional方式定义的层初始化为一个模型（Model）\n", 313 | "1. 使用keras.layers.add方法，可以将两个一模一样的张量进行相加\n", 314 | "1. 搭建了一个精简版的ResNet网络\n", 315 | "1. 学习了如何在训练中调用回调函数\n", 316 | "1. 学习了在训练过程中动态的调节学习率（使用LearningRateScheduler）\n", 317 | "1. 学习了保存checkpoint（使用ModelCheckpoint）\n", 318 | "1. 使用ReduceLROnPlateau在训练进入平台期的时候动态调节学习率\n", 319 | "\n", 320 | "\n", 321 | "参考：\n", 322 | "> https://github.com/keras-team/keras/blob/master/examples/cifar10_resnet.py" 323 | ] 324 | } 325 | ], 326 | "metadata": { 327 | "anaconda-cloud": {}, 328 | "kernelspec": { 329 | "display_name": "Python 3", 330 | "language": "python", 331 | "name": "python3" 332 | }, 333 | "language_info": { 334 | "codemirror_mode": { 335 | "name": "ipython", 336 | "version": 3 337 | }, 338 | "file_extension": ".py", 339 | "mimetype": "text/x-python", 340 | "name": "python", 341 | "nbconvert_exporter": "python", 342 | "pygments_lexer": "ipython3", 343 | "version": "3.6.2" 344 | } 345 | }, 346 | "nbformat": 4, 347 | "nbformat_minor": 2 348 | } 349 | -------------------------------------------------------------------------------- /class_5.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Keras入门课5：网络可视化及训练监控\n", 8 | "\n", 9 | "本节专注于Keras中神经网络的可视化，包括网络结构可视化以及如何使用TensorBoard来监控训练过程。 \n", 10 | "这里我们借用第2课的代码内容来进行示例和讲解。\n", 11 | "\n", 12 | "网络前面的定义、数据初始化都一样，主要是fit函数\n", 13 | "\n", 14 | "#### 启用TensorBoard\n", 15 | "在model的fit函数中加入TensorBoard的回调函数即可，训练数据就会自动保存在log_dir指定的目录内，然后在命令行启动命令 tensorboard --logdir=./log 即可。TensorBoard会记录loss及model.metrics里面的值，本例中即acc,loss,val_acc,val_loss四个值，每个epoch更新一次。 \n", 16 | "除了这些SCALARS，还会记录网络的GRAPH，直接可视化网络结构，但是相比用原生TensorFlow生成的图而言，相差还是比较大的，比较难看，所以不推荐在Keras中使用TensorBoard查看网络结构。\n", 17 | "\n", 18 | "我只训练了2个epoch，所以只记录下了两个值。曲线图如下\n", 19 | "![](./images/tensorboard_scalars.png)\n", 20 | "\n", 21 | "↓直方图，用来统计参数的分布\n", 22 | "\n", 23 | "![](./images/tensorboard_hist.png)\n", 24 | "\n", 25 | "In[1]:" 26 | ] 27 | }, 28 | { 29 | "cell_type": "code", 30 | "execution_count": null, 31 | "metadata": {}, 32 | "outputs": [], 33 | "source": [ 34 | "import keras\n", 35 | "from keras.datasets import mnist\n", 36 | "from keras.models import Sequential\n", 37 | "from keras.layers import Dense, Dropout, Flatten\n", 38 | "from keras.layers import Conv2D, MaxPooling2D\n", 39 | "from keras import backend as K\n", 40 | "# 引入Tensorboard\n", 41 | "from keras.callbacks import TensorBoard\n", 42 | "from keras.utils import plot_model\n", 43 | "\n", 44 | "(x_train,y_train),(x_test,y_test) = mnist.load_data() # out: np.ndarray\n", 45 | "\n", 46 | "x_train = x_train.reshape(-1,28,28,1)\n", 47 | "x_test = x_test.reshape(-1,28,28,1)\n", 48 | "input_shape = (28,28,1)\n", 49 | "\n", 50 | "x_train = x_train/255\n", 51 | "x_test = x_test/255\n", 52 | "y_train = keras.utils.to_categorical(y_train,10)\n", 53 | "y_test = keras.utils.to_categorical(y_test,10)" 54 | ] 55 | }, 56 | { 57 | "cell_type": "code", 58 | "execution_count": null, 59 | "metadata": { 60 | "collapsed": true 61 | }, 62 | "outputs": [], 63 | "source": [ 64 | "model = Sequential()\n", 65 | "model.add(Conv2D(filters = 32,kernel_size=(3,3),\n", 66 | " activation='relu',input_shape = input_shape,name='conv1'))\n", 67 | "model.add(Conv2D(64,(3,3),activation='relu',name='conv2'))\n", 68 | "model.add(MaxPooling2D(pool_size=(2,2),name='pool2'))\n", 69 | "model.add(Dropout(0.25,name='dropout1'))\n", 70 | "model.add(Flatten(name='flat1'))\n", 71 | "model.add(Dense(128,activation='relu'))\n", 72 | "model.add(Dropout(0.5,name='dropout2'))\n", 73 | "model.add(Dense(10,activation='softmax',name='output'))" 74 | ] 75 | }, 76 | { 77 | "cell_type": "code", 78 | "execution_count": null, 79 | "metadata": { 80 | "collapsed": true 81 | }, 82 | "outputs": [], 83 | "source": [ 84 | "plot_model(model,to_file='model.png')" 85 | ] 86 | }, 87 | { 88 | "cell_type": "markdown", 89 | "metadata": {}, 90 | "source": [ 91 | "↑keras的utils里面专门有一个plot_model函数是用来可视化网络结构的，为了保证格式美观，我们在定义模型的时候给每个层都加了一个名字。 \n", 92 | "对于大多数的Keras的layers，都有name这一参数。\n", 93 | "使用plot_model就可以生成类似下图的一张图片，相比TensorBoard的Graph要清晰明了很多。所以在Keras中打印图结构还是推荐使用Keras自带的方法。" 94 | ] 95 | }, 96 | { 97 | "cell_type": "markdown", 98 | "metadata": {}, 99 | "source": [ 100 | "![](./images/mlp_model.png)" 101 | ] 102 | }, 103 | { 104 | "cell_type": "code", 105 | "execution_count": null, 106 | "metadata": { 107 | "collapsed": true 108 | }, 109 | "outputs": [], 110 | "source": [ 111 | "model.compile(loss = keras.losses.categorical_crossentropy,\n", 112 | " optimizer = keras.optimizers.Adadelta(),\n", 113 | " metrics=['accuracy'])" 114 | ] 115 | }, 116 | { 117 | "cell_type": "markdown", 118 | "metadata": {}, 119 | "source": [ 120 | "TensorBoard接口函数，有很多参数可选，具体细节可以参看官方文档。相比TensorFlow中的summary保存而言，keras中的TensorBoard使用更为简单，但是灵活性较差，只适合一些最基础的使用。" 121 | ] 122 | }, 123 | { 124 | "cell_type": "code", 125 | "execution_count": null, 126 | "metadata": { 127 | "collapsed": true 128 | }, 129 | "outputs": [], 130 | "source": [ 131 | "tb = TensorBoard(log_dir='./logs', # log 目录\n", 132 | " histogram_freq=1, # 按照何等频率（epoch）来计算直方图，0为不计算\n", 133 | " batch_size=32, # 用多大量的数据计算直方图\n", 134 | " write_graph=True, # 是否存储网络结构图\n", 135 | " write_grads=False, # 是否可视化梯度直方图\n", 136 | " write_images=False,# 是否可视化参数\n", 137 | " embeddings_freq=0, \n", 138 | " embeddings_layer_names=None, \n", 139 | " embeddings_metadata=None)\n", 140 | "callbacks = [tb]" 141 | ] 142 | }, 143 | { 144 | "cell_type": "code", 145 | "execution_count": null, 146 | "metadata": { 147 | "collapsed": true 148 | }, 149 | "outputs": [], 150 | "source": [ 151 | "model.fit(x_train,y_train,batch_size=64,epochs=2\n", 152 | " ,verbose=1,validation_data=(x_test,y_test),\n", 153 | " callbacks=callbacks)" 154 | ] 155 | }, 156 | { 157 | "cell_type": "markdown", 158 | "metadata": {}, 159 | "source": [ 160 | "## 总结\n", 161 | "\n", 162 | "1. 学习了如何用TensorBoard监控训练过程\n", 163 | "1. 学习了如何使用keras自带的save_model函数来保存网络图\n", 164 | "\n", 165 | "\n", 166 | "在使用plot_model绘制图片的时候还遇到了一些错误，如果你也报错，可以参看我的另一篇文章尝试解决：[ Mac下使用Keras plot_model函数时出错的解决办法](http://blog.csdn.net/tsyccnh/article/details/78866976)\n", 167 | "\n", 168 | "本文代码地址：https://github.com/tsycnh/Keras-Tutorials/blob/master/class_5.ipynb\n", 169 | "\n", 170 | "参考：\n", 171 | "> https://keras.io/visualization/\n" 172 | ] 173 | } 174 | ], 175 | "metadata": { 176 | "kernelspec": { 177 | "display_name": "Python 3", 178 | "language": "python", 179 | "name": "python3" 180 | }, 181 | "language_info": { 182 | "codemirror_mode": { 183 | "name": "ipython", 184 | "version": 3 185 | }, 186 | "file_extension": ".py", 187 | "mimetype": "text/x-python", 188 | "name": "python", 189 | "nbconvert_exporter": "python", 190 | "pygments_lexer": "ipython3", 191 | "version": "3.6.2" 192 | } 193 | }, 194 | "nbformat": 4, 195 | "nbformat_minor": 2 196 | } 197 | -------------------------------------------------------------------------------- /class_6.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Keras 入门课6：使用Inception V3模型进行迁移学习\n", 8 | "\n", 9 | "深度学习可以说是一门数据驱动的学科，各种有名的CNN模型，无一不是在大型的数据库上进行的训练。像ImageNet这种规模的数据库，动辄上百万张图片。对于普通的机器学习工作者、学习者来说，面对的任务各不相同，很难拿到如此大规模的数据集。同时也没有谷歌，Facebook那种大公司惊人的算力支持，想从0训练一个深度CNN网络，基本是不可能的。但是好在已经训练好的模型的参数，往往经过简单的调整和训练，就可以很好的迁移到其他不同的数据集上，同时也无需大量的算力支撑，便能在短时间内训练得出满意的效果。这便是迁移学习。究其根本，就是虽然图像的数据集不同，但是底层的特征却是有大部分通用的。 \n", 10 | "\n", 11 | "迁移学习主要分为两种\n", 12 | "\n", 13 | "* 第一种即所谓的transfer learning，迁移训练时，移掉最顶层，比如ImageNet训练任务的顶层就是一个1000输出的全连接层，换上新的顶层，比如输出为10的全连接层，然后训练的时候，只训练最后两层，即原网络的倒数第二层和新换的全连接输出层。可以说transfer learning将底层的网络当做了一个特征提取器来使用。 \n", 14 | "* 第二种叫做fine tune，和transfer learning一样，换一个新的顶层，但是这一次在训练的过程中，所有的（或大部分）其它层都会经过训练。也就是底层的权重也会随着训练进行调整。\n", 15 | "\n", 16 | "一个典型的迁移学习过程是这样的。首先通过transfer learning对新的数据集进行训练，训练过一定epoch之后，改用fine tune方法继续训练，同时降低学习率。这样做是因为如果一开始就采用fine tune方法的话，网络还没有适应新的数据，那么在进行参数更新的时候，比较大的梯度可能会导致原本训练的比较好的参数被污染，反而导致效果下降。\n", 17 | "\n", 18 | "本课，我们将尝试使用谷歌提出的Inception V3模型来对一个花朵数据集进行迁移学习的训练。" 19 | ] 20 | }, 21 | { 22 | "cell_type": "markdown", 23 | "metadata": {}, 24 | "source": [ 25 | "数据集为17种不同的花朵，每种有80张样本，一共1360张图像，属于典型的小样本集。数据下载地址：http://www.robots.ox.ac.uk/~vgg/data/flowers/17/ \n", 26 | "官方没有给出图像对应的label，我写了一段代码，把每张图像加上标签，https://gist.github.com/tsycnh/177bbf7d93adc6207242fd334ce3bb60 \n", 27 | "同时，Keras对于数据的格式要求如下：我也写了一个脚本来做转换https://gist.github.com/tsycnh/1b35103adec1ad2be5090c486354859f \n", 28 | "```\n", 29 | "data/\n", 30 | " train/\n", 31 | " class1/\n", 32 | " img1\n", 33 | " img2\n", 34 | " ...\n", 35 | " class2/\n", 36 | " img1\n", 37 | " ...\n", 38 | " validation/\n", 39 | " class1/\n", 40 | " img1\n", 41 | " img2\n", 42 | " ...\n", 43 | " class2/\n", 44 | " img1\n", 45 | " ...\n", 46 | " test/\n", 47 | " class1/\n", 48 | " img1\n", 49 | " img2\n", 50 | " ...\n", 51 | " class2/\n", 52 | " img1\n", 53 | " ...\n", 54 | "```\n", 55 | "这个脚本我将训练集划分为800张，验证集和测试集分别为260张，图片顺序做了随机打乱\n", 56 | "\n", 57 | "如果你懒得自己转换，可以直接下载我已经把处理好的数据：https://download.csdn.net/download/tsyccnh/10641502\n", 58 | "\n", 59 | "请注意，这里的花朵识别仍属于最简单的单分类任务，样张如下" 60 | ] 61 | }, 62 | { 63 | "cell_type": "markdown", 64 | "metadata": {}, 65 | "source": [ 66 | "![](./images/flowers.jpg)" 67 | ] 68 | }, 69 | { 70 | "cell_type": "code", 71 | "execution_count": null, 72 | "metadata": {}, 73 | "outputs": [], 74 | "source": [ 75 | "from keras.preprocessing.image import ImageDataGenerator\n", 76 | "from keras.applications.inception_v3 import InceptionV3,preprocess_input\n", 77 | "from keras.layers import GlobalAveragePooling2D,Dense\n", 78 | "from keras.models import Model\n", 79 | "from keras.utils.vis_utils import plot_model\n", 80 | "from keras.optimizers import Adagrad\n", 81 | "# 数据准备\n", 82 | "train_datagen = ImageDataGenerator(\n", 83 | " preprocessing_function=preprocess_input,# ((x/255)-0.5)*2 归一化到±1之间\n", 84 | " rotation_range=30,\n", 85 | " width_shift_range=0.2,\n", 86 | " height_shift_range=0.2,\n", 87 | " shear_range=0.2,\n", 88 | " zoom_range=0.2,\n", 89 | " horizontal_flip=True,\n", 90 | ")\n", 91 | "val_datagen = ImageDataGenerator(\n", 92 | " preprocessing_function=preprocess_input,\n", 93 | " rotation_range=30,\n", 94 | " width_shift_range=0.2,\n", 95 | " height_shift_range=0.2,\n", 96 | " shear_range=0.2,\n", 97 | " zoom_range=0.2,\n", 98 | " horizontal_flip=True,\n", 99 | ")" 100 | ] 101 | }, 102 | { 103 | "cell_type": "markdown", 104 | "metadata": {}, 105 | "source": [ 106 | "这里用到的数据集和之前都不同，之前用的是一些公共的、Keras内置的数据集，这次用到的是自己准备的数据集。由于数据的图像大小比较大，不适合一次全部载入到内存中，所以使用了flow_from_directory方法来按批次从硬盘读取图像数据，并实时进行图像增强" 107 | ] 108 | }, 109 | { 110 | "cell_type": "code", 111 | "execution_count": null, 112 | "metadata": {}, 113 | "outputs": [], 114 | "source": [ 115 | "train_generator = train_datagen.flow_from_directory(directory='./flowers17/train',\n", 116 | " target_size=(299,299),#Inception V3规定大小\n", 117 | " batch_size=64)\n", 118 | "val_generator = val_datagen.flow_from_directory(directory='./flowers17/validation',\n", 119 | " target_size=(299,299),\n", 120 | " batch_size=64)" 121 | ] 122 | }, 123 | { 124 | "cell_type": "markdown", 125 | "metadata": {}, 126 | "source": [ 127 | "首先我们需要加载骨架模型，这里用的InceptionV3模型，其两个参数比较重要，一个是weights，如果是'imagenet',Keras就会自动下载已经在ImageNet上训练好的参数，如果是None，系统会通过随机的方式初始化参数，目前该参数只有这两个选择。另一个参数是include_top，如果是True，输出是1000个节点的全连接层。如果是False，会去掉顶层，输出一个8 \\* 8 \\* 2048的张量。 \n", 128 | "\n", 129 | "ps：在keras.applications里还有很多其他的预置模型，比如VGG，ResNet，以及适用于移动端的MobileNet等。大家都可以拿来玩玩。\n", 130 | "\n", 131 | "一般我们做迁移训练，都是要去掉顶层，后面接上各种自定义的其它新层。这已经成为了训练新任务惯用的套路。\n", 132 | "输出层先用GlobalAveragePooling2D函数将8 \\* 8 \\* 2048的输出转换成1 \\* 2048的张量。后面接了一个1024个节点的全连接层，最后是一个17个节点的输出层，用softmax激活函数。" 133 | ] 134 | }, 135 | { 136 | "cell_type": "code", 137 | "execution_count": null, 138 | "metadata": { 139 | "collapsed": true 140 | }, 141 | "outputs": [], 142 | "source": [ 143 | "# 构建基础模型\n", 144 | "base_model = InceptionV3(weights='imagenet',include_top=False)\n", 145 | "\n", 146 | "# 增加新的输出层\n", 147 | "x = base_model.output\n", 148 | "x = GlobalAveragePooling2D()(x) # GlobalAveragePooling2D 将 MxNxC 的张量转换成 1xC 张量，C是通道数\n", 149 | "x = Dense(1024,activation='relu')(x)\n", 150 | "predictions = Dense(17,activation='softmax')(x)\n", 151 | "model = Model(inputs=base_model.input,outputs=predictions)\n", 152 | "# plot_model(model,'tlmodel.png')" 153 | ] 154 | }, 155 | { 156 | "cell_type": "markdown", 157 | "metadata": {}, 158 | "source": [ 159 | "构建完新模型后需要进行模型的配置。下面的两个函数分别对transfer learning和fine tune两种方法分别进行了配置。每个函数有两个参数，分别是model和base_model。这里可能会有同学有疑问，上面定义了model，这里又将base_model一起做配置，对base_model的更改会对model产生影响么？ \n", 160 | "答案是会的。如果你debug追进去看的话，可以看到model的第一层和base_model的第一层是指向同一个内存地址的。这里将base_model作为参数，只是为了方便对骨架模型进行设置。\n", 161 | "\n", 162 | "setup_to_transfer_learning： 这个函数将骨架模型的所有层都设置为不可训练\n", 163 | "setup_to_fine_tune：这个函数将骨架模型中的前几层设置为不可训练，后面的所有Inception模块都设置为可训练。 \n", 164 | "这里面的GAP_LAYER需要配合打印图和调试的方法确认正确的值，感兴趣具体怎么操作的同学，可以私信我，以后看有没有必要把这个点写成教程。" 165 | ] 166 | }, 167 | { 168 | "cell_type": "code", 169 | "execution_count": null, 170 | "metadata": { 171 | "collapsed": true 172 | }, 173 | "outputs": [], 174 | "source": [ 175 | "'''\n", 176 | "这里的base_model和model里面的iv3都指向同一个地址\n", 177 | "'''\n", 178 | "def setup_to_transfer_learning(model,base_model):#base_model\n", 179 | " for layer in base_model.layers:\n", 180 | " layer.trainable = False\n", 181 | " model.compile(optimizer='adam',loss='categorical_crossentropy',metrics=['accuracy'])\n", 182 | "\n", 183 | "def setup_to_fine_tune(model,base_model):\n", 184 | " GAP_LAYER = 17 # max_pooling_2d_2\n", 185 | " for layer in base_model.layers[:GAP_LAYER+1]:\n", 186 | " layer.trainable = False\n", 187 | " for layer in base_model.layers[GAP_LAYER+1:]:\n", 188 | " layer.trainable = True\n", 189 | " model.compile(optimizer=Adagrad(lr=0.0001),loss='categorical_crossentropy',metrics=['accuracy'])" 190 | ] 191 | }, 192 | { 193 | "cell_type": "markdown", 194 | "metadata": {}, 195 | "source": [ 196 | "下面开始训练，这段代码也演示了如何在全部训练过程中改变模型。" 197 | ] 198 | }, 199 | { 200 | "cell_type": "code", 201 | "execution_count": null, 202 | "metadata": {}, 203 | "outputs": [], 204 | "source": [ 205 | "setup_to_transfer_learning(model,base_model)\n", 206 | "history_tl = model.fit_generator(generator=train_generator,\n", 207 | " steps_per_epoch=800,#800\n", 208 | " epochs=2,#2\n", 209 | " validation_data=val_generator,\n", 210 | " validation_steps=12,#12\n", 211 | " class_weight='auto'\n", 212 | " )\n", 213 | "model.save('./flowers17_iv3_tl.h5')\n", 214 | "setup_to_fine_tune(model,base_model)\n", 215 | "history_ft = model.fit_generator(generator=train_generator,\n", 216 | " steps_per_epoch=800,\n", 217 | " epochs=2,\n", 218 | " validation_data=val_generator,\n", 219 | " validation_steps=1,\n", 220 | " class_weight='auto')\n", 221 | "model.save('./flowers17_iv3_ft.h5')" 222 | ] 223 | }, 224 | { 225 | "cell_type": "markdown", 226 | "metadata": {}, 227 | "source": [ 228 | "可以看到经过两个epoch的transfer learning后，验证集准确率达到89.1%。再经过两个epoch的fine tune后验证集准确率达96.88%。可以看到迁移学习的效果还是很好的。" 229 | ] 230 | }, 231 | { 232 | "cell_type": "markdown", 233 | "metadata": {}, 234 | "source": [ 235 | "## 总结\n", 236 | "1. 学习了两种常用迁移学习方法（tranfer learning,fine tune)及训练技巧\n", 237 | "1. 学习了使用自己的数据样本进行训练\n", 238 | "1. 学习了加载Keras预置的经典模型\n", 239 | "1. 学习了如何在预置模型顶部添加新的层\n", 240 | "1. 学习了如何设置层的参数为不可训练\n", 241 | "\n", 242 | "\n", 243 | "\n", 244 | "### 参考\n", 245 | "> https://deeplearningsandbox.com/how-to-use-transfer-learning-and-fine-tuning-in-keras-and-tensorflow-to-build-an-image-recognition-94b0b02444f2" 246 | ] 247 | } 248 | ], 249 | "metadata": { 250 | "kernelspec": { 251 | "display_name": "Python 3", 252 | "language": "python", 253 | "name": "python3" 254 | }, 255 | "language_info": { 256 | "codemirror_mode": { 257 | "name": "ipython", 258 | "version": 3 259 | }, 260 | "file_extension": ".py", 261 | "mimetype": "text/x-python", 262 | "name": "python", 263 | "nbconvert_exporter": "python", 264 | "pygments_lexer": "ipython3", 265 | "version": "3.6.2" 266 | } 267 | }, 268 | "nbformat": 4, 269 | "nbformat_minor": 2 270 | } 271 | -------------------------------------------------------------------------------- /convert_flowers17.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import os 3 | from shutil import copyfile,rmtree 4 | # 数据预处理，将原始花朵数据集转换成keras要求的格式,如下.每一个子文件夹代表一类 5 | ''' 6 | data/ 7 | train/ 8 | class1/ 9 | img1 10 | img2 11 | ... 12 | class2/ 13 | img1 14 | ... 15 | validation/ 16 | class1/ 17 | img1 18 | img2 19 | ... 20 | class2/ 21 | img1 22 | ... 23 | test/ 24 | class1/ 25 | img1 26 | img2 27 | ... 28 | class2/ 29 | img1 30 | ... 31 | ''' 32 | 33 | data_folder = '/Users/shidanlifuhetian/All/data/flowers17/'#原始图像目录 34 | new_data_folder = './flowers17/'#新的文件夹 35 | 36 | # add label first 37 | np.random.seed(0)# 统一seed，保证每次随机结果都一样 38 | 39 | file_a = open(data_folder+'files.txt',mode='r') 40 | text = file_a.readlines() 41 | file_a.close() 42 | labels = [] 43 | file_b = open('./newtext.txt',mode='a') 44 | for i,item in enumerate(text): 45 | if i%80 ==0: 46 | class_num = i//80 47 | t = item.split('\n') 48 | newtext = t[0]+' flower_'+chr(65+class_num)+'\n' 49 | print(newtext) 50 | file_b.write(newtext) 51 | file_b.close() 52 | # split dataset 53 | if os.path.exists(new_data_folder): 54 | rmtree(new_data_folder) 55 | train_size = 800 56 | val_size = int((1360-train_size)/2) 57 | test_size = int(val_size) 58 | 59 | label_file = open('newtext.txt') 60 | 61 | labels = label_file.readlines() 62 | np.random.shuffle(labels) 63 | 64 | def save_images(current_i,phase,d_size): 65 | 66 | if phase == 'train': 67 | dst_folder = new_data_folder+'train/' 68 | elif phase == 'test': 69 | dst_folder = new_data_folder+'test/' 70 | elif phase == 'validation': 71 | dst_folder = new_data_folder+'validation/' 72 | else: 73 | print('phase error') 74 | exit() 75 | 76 | for i in range(current_i,current_i+d_size): 77 | item = labels[i] 78 | r = item.split(' ') 79 | img_full_path = data_folder+r[0] 80 | img_class = r[1].split('\n')[0] 81 | img_new_path = dst_folder+img_class+'/'+r[0] 82 | 83 | if not os.path.exists(dst_folder+img_class): 84 | os.makedirs(dst_folder+img_class) 85 | copyfile(img_full_path,img_new_path) 86 | print(img_new_path,' copied') 87 | current_i = i 88 | 89 | return current_i 90 | 91 | new_i = save_images(current_i=0,phase='train',d_size=train_size) 92 | new_i = save_images(current_i=new_i,phase='test',d_size=test_size) 93 | new_i = save_images(current_i=new_i,phase='validation',d_size=val_size) 94 | -------------------------------------------------------------------------------- /images/flowers.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tsycnh/Keras-Tutorials/4b026e64b57281a9a7212e220d6d4995770ac8cd/images/flowers.jpg -------------------------------------------------------------------------------- /images/lstm.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tsycnh/Keras-Tutorials/4b026e64b57281a9a7212e220d6d4995770ac8cd/images/lstm.png -------------------------------------------------------------------------------- /images/mlp_model.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tsycnh/Keras-Tutorials/4b026e64b57281a9a7212e220d6d4995770ac8cd/images/mlp_model.png -------------------------------------------------------------------------------- /images/resnetblock.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tsycnh/Keras-Tutorials/4b026e64b57281a9a7212e220d6d4995770ac8cd/images/resnetblock.png -------------------------------------------------------------------------------- /images/tensorboard_hist.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tsycnh/Keras-Tutorials/4b026e64b57281a9a7212e220d6d4995770ac8cd/images/tensorboard_hist.png -------------------------------------------------------------------------------- /images/tensorboard_scalars.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tsycnh/Keras-Tutorials/4b026e64b57281a9a7212e220d6d4995770ac8cd/images/tensorboard_scalars.png -------------------------------------------------------------------------------- /tlmodel.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/tsycnh/Keras-Tutorials/4b026e64b57281a9a7212e220d6d4995770ac8cd/tlmodel.png --------------------------------------------------------------------------------