├── 01_python ├── Anaconda的介绍、安装和环境管理.md ├── IDE之PyCharm的设置和Debug入门.md ├── Jupyter的介绍、安装及使用.md ├── conda和pip常用命令速查.md ├── python_basic │ ├── Python一键安装所有第三方库.md │ ├── Python内置库和函数使用及常见功能实现记录.md │ ├── Python基础入门笔记（一）.md │ └── Python基础入门笔记（二）.md ├── python_library │ ├── Python常用科学计算库快速入门(NumPy、SciPy、Pandas、Matplotlib、Scikit-learn).md │ ├── 科学计算库之matplotlib的使用.md │ ├── 科学计算库之numpy的使用.md │ ├── 科学计算库之pandas的使用.md │ └── 科学计算库之scikit-learn的使用.md └── python_library_ImageProcessing │ └── Python图像处理笔记.md ├── 02_basic ├── deep_learning │ ├── 01-深度学习要点梳理和个人理解.md │ ├── cnn_models │ │ └── ResNet.md │ ├── 《深度学习xx问》.md │ ├── 卷积神经网络以及图像分割论文整理和解读.md │ ├── 我的理解：神经网络参数改变过程？.md │ └── 激活函数之Sigmoid&ReLU&Softmax.md ├── dl-papers.md ├── machine_learning │ └── 机器学习.md └── math_basic │ └── 数学基础.md ├── 03_dl_framework ├── keras │ ├── keras-learning.md │ └── readme.md ├── pytorch │ ├── notes │ │ ├── 01-pytorch简介、安装(Linux、Windows、Mac).md │ │ └── pytorch的坑.md │ └── pytorch学习.md ├── tensorflow │ ├── [整理]TensorFlow的API详解和记录.md │ ├── [转]TensorFlow的理解和总结.md │ ├── readme.md │ ├── tensorflow中使用指定的GPU及显存分析.md │ ├── tensorflow优化器参数详解.md │ └── 《深度学习框架Tensorflow学习与应用》笔记 │ │ ├── code │ │ ├── week01-src │ │ │ └── 01-Tensorflow简介，Anaconda安装，Tensorflow的CPU版本安装.md │ │ ├── week02-src │ │ │ └── 02-Tensorflow的基础使用，包括对图(graphs),会话(session),张量(tensor),变量(Variable)的一些解释和操作.md │ │ ├── week03-src │ │ │ ├── 3-1非线性回归.ipynb │ │ │ ├── 3-1非线性回归.py │ │ │ ├── 3-2MNIST数据集分类简单版本.ipynb │ │ │ ├── 3-2MNIST数据集分类简单版本.py │ │ │ └── MNIST_data │ │ │ │ ├── t10k-images-idx3-ubyte.gz │ │ │ │ ├── t10k-labels-idx1-ubyte.gz │ │ │ │ ├── train-images-idx3-ubyte.gz │ │ │ │ └── train-labels-idx1-ubyte.gz │ │ ├── week04-src │ │ │ ├── 4-1交叉熵.ipynb │ │ │ ├── 4-1交叉熵.py │ │ │ ├── 4-2Dropout.ipynb │ │ │ ├── 4-2Dropout.py │ │ │ ├── 4-3优化器.ipynb │ │ │ └── 4-3优化器.py │ │ ├── week05-src │ │ │ ├── 5-1第四周作业.ipynb │ │ │ ├── 5-1第四周作业.py │ │ │ ├── 5-2tensorboard网络结构.ipynb │ │ │ ├── 5-2tensorboard网络结构.py │ │ │ ├── 5-3tensorboard网络运行.ipynb │ │ │ ├── 5-3tensorboard网络运行.py │ │ │ ├── 5-4tensorboard可视化.ipynb │ │ │ ├── 5-4tensorboard可视化.py │ │ │ └── mnist_10k_sprite.png │ │ ├── week06-src │ │ │ ├── 6-1卷积神经网络应用于MNIST数据集分类.ipynb │ │ │ └── 6-1卷积神经网络应用于MNIST数据集分类.py │ │ ├── week07-src │ │ │ ├── 7-1第六周作业.ipynb │ │ │ ├── 7-1第六周作业.py │ │ │ ├── 7-2递归神经网络RNN.ipynb │ │ │ └── 7-2递归神经网络RNN.py │ │ ├── week08-src │ │ │ ├── 8-1saver_save.ipynb │ │ │ ├── 8-1saver_save.py │ │ │ ├── 8-2saver_restore.ipynb │ │ │ ├── 8-2saver_restore.py │ │ │ ├── 8-3下载google图像识别网络inception-v3并查看结构.ipynb │ │ │ ├── 8-3下载google图像识别网络inception-v3并查看结构.py │ │ │ ├── 8-4使用inception-v3做各种图像的识别.ipynb │ │ │ ├── 8-4使用inception-v3做各种图像的识别.py │ │ │ └── images │ │ │ │ ├── lion.jpg │ │ │ │ ├── panda.jpg │ │ │ │ └── rabbit.jpg │ │ ├── week09-src │ │ │ ├── retain │ │ │ │ ├── 9-1测试训练好的模型.ipynb │ │ │ │ ├── 9-1测试训练好的模型.py │ │ │ │ └── retrain.bat │ │ │ └── slim │ │ │ │ ├── 9-2生成tfrecord.ipynb │ │ │ │ ├── 9-2生成tfrecord.py │ │ │ │ ├── datasets │ │ │ │ ├── dataset_factory.py │ │ │ │ └── myimages.py │ │ │ │ └── train.bat │ │ ├── week10-src │ │ │ ├── 10-1验证码生成.ipynb │ │ │ ├── 10-1验证码生成.py │ │ │ ├── 10-2生成tfrecord文件.ipynb │ │ │ ├── 10-2生成tfrecord文件.py │ │ │ ├── 10-3验证码识别.ipynb │ │ │ ├── 10-3验证码识别.py │ │ │ ├── 10-4captcha_test.ipynb │ │ │ ├── 10-4captcha_test.py │ │ │ └── nets │ │ │ │ ├── __init__.py │ │ │ │ ├── __init__.pyc │ │ │ │ ├── __pycache__ │ │ │ │ ├── __init__.cpython-35.pyc │ │ │ │ ├── alexnet.cpython-35.pyc │ │ │ │ ├── cifarnet.cpython-35.pyc │ │ │ │ ├── inception.cpython-35.pyc │ │ │ │ ├── inception_resnet_v2.cpython-35.pyc │ │ │ │ ├── inception_utils.cpython-35.pyc │ │ │ │ ├── inception_v1.cpython-35.pyc │ │ │ │ ├── inception_v2.cpython-35.pyc │ │ │ │ ├── inception_v3.cpython-35.pyc │ │ │ │ ├── inception_v4.cpython-35.pyc │ │ │ │ ├── lenet.cpython-35.pyc │ │ │ │ ├── nets_factory.cpython-35.pyc │ │ │ │ ├── overfeat.cpython-35.pyc │ │ │ │ ├── resnet_utils.cpython-35.pyc │ │ │ │ ├── resnet_v1.cpython-35.pyc │ │ │ │ ├── resnet_v2.cpython-35.pyc │ │ │ │ └── vgg.cpython-35.pyc │ │ │ │ ├── alexnet.py │ │ │ │ ├── alexnet.pyc │ │ │ │ ├── alexnet_test.py │ │ │ │ ├── cifarnet.py │ │ │ │ ├── cifarnet.pyc │ │ │ │ ├── inception.py │ │ │ │ ├── inception.pyc │ │ │ │ ├── inception_resnet_v2.py │ │ │ │ ├── inception_resnet_v2.pyc │ │ │ │ ├── inception_resnet_v2_test.py │ │ │ │ ├── inception_utils.py │ │ │ │ ├── inception_utils.pyc │ │ │ │ ├── inception_v1.py │ │ │ │ ├── inception_v1.pyc │ │ │ │ ├── inception_v1_test.py │ │ │ │ ├── inception_v2.py │ │ │ │ ├── inception_v2.pyc │ │ │ │ ├── inception_v2_test.py │ │ │ │ ├── inception_v3.py │ │ │ │ ├── inception_v3.pyc │ │ │ │ ├── inception_v3_test.py │ │ │ │ ├── inception_v4.py │ │ │ │ ├── inception_v4.pyc │ │ │ │ ├── inception_v4_test.py │ │ │ │ ├── lenet.py │ │ │ │ ├── lenet.pyc │ │ │ │ ├── nets_factory.py │ │ │ │ ├── nets_factory.pyc │ │ │ │ ├── nets_factory_test.py │ │ │ │ ├── overfeat.py │ │ │ │ ├── overfeat.pyc │ │ │ │ ├── overfeat_test.py │ │ │ │ ├── resnet_utils.py │ │ │ │ ├── resnet_utils.pyc │ │ │ │ ├── resnet_v1.py │ │ │ │ ├── resnet_v1.pyc │ │ │ │ ├── resnet_v1_test.py │ │ │ │ ├── resnet_v2.py │ │ │ │ ├── resnet_v2.pyc │ │ │ │ ├── resnet_v2_test.py │ │ │ │ ├── vgg.py │ │ │ │ ├── vgg.pyc │ │ │ │ └── vgg_test.py │ │ ├── week11-src │ │ │ ├── 11-1第十周作业-验证码识别.ipynb │ │ │ ├── 11-2第十周作业-验证码测试.ipynb │ │ │ ├── 11-3simple_word2vec.ipynb │ │ │ └── cnn-text-classification-tf-master │ │ │ │ ├── .DS_Store │ │ │ │ ├── .gitignore │ │ │ │ ├── LICENSE │ │ │ │ ├── README.md │ │ │ │ ├── data │ │ │ │ ├── .DS_Store │ │ │ │ └── rt-polaritydata │ │ │ │ │ ├── rt-polarity.neg │ │ │ │ │ └── rt-polarity.pos │ │ │ │ ├── data_helpers.py │ │ │ │ ├── eval.py │ │ │ │ ├── text_cnn.py │ │ │ │ ├── train.ipynb │ │ │ │ └── train.py │ │ └── week12-src │ │ │ ├── 12-1第十一周作业.ipynb │ │ │ ├── 12-1第十一周作业.py │ │ │ ├── 12声音分类.ipynb │ │ │ ├── 12声音分类.py │ │ │ └── text_cnn.py │ │ └── notes │ │ ├── 01-Tensorflow简介，Anaconda安装，Tensorflow的CPU版本安装.md │ │ ├── 02-Tensorflow的基础使用，包括对图(graphs),会话(session),张量(tensor),变量(Variable)的一些解释和操作.md │ │ ├── 03-Tensorflow线性回归以及分类的简单使用.md │ │ ├── 04-softmax，交叉熵(cross-entropy)，dropout以及Tensorflow中各种优化器的介绍.md │ │ ├── 05-使用Tensorboard进行结构可视化，以及网络运算过程可视化.md │ │ ├── 06-卷积神经网络CNN的讲解，以及用CNN解决MNIST分类问题.md │ │ ├── 07-递归神经网络LSTM的讲解，以及LSTM网络的使用.md │ │ ├── 08-保存和载入模型，使用Google的图像识别网络inception-v3进行图像识别.md │ │ ├── 09-Tensorflow的GPU版本安装。设计自己的网络模型，并训练自己的网络模型进行图像识别.md │ │ ├── 10-使用Tensorflow进行验证码识别.md │ │ ├── 11-Tensorflow在NLP中的使用(一).md │ │ ├── 12-Tensorflow在NLP中的使用(二).md │ │ └── tf.nn.softmax_cross_entropy_with_logits的用法问题.md ├── 深度学习框架对比.md └── 深度学习硬件选购及tensorflow各系统下的环境搭建.md ├── 04_computer_vision ├── [转]关于神经网络模型&TensorFlow学习&目标检测模型等内容的系列文章.md ├── object_detection │ ├── img │ │ ├── 《A Survey of Deep Learning-based Object Detection》综述小总结.png │ │ └── 深度学习目标检测发展史.jpg │ ├── object_detection.md │ ├── 深度学习-目标检测.md │ └── 目标检测发展.md ├── semantic_segmentation │ ├── 01-图像分割(含语义实例全景分割).md │ ├── readme.md │ ├── semantic_segmentation.md │ ├── 全景分割模型.md │ ├── 图像分割(语义分割、实例分割、全景分割).md │ ├── 实例分割模型.md │ ├── 计算机视觉笔记及资料整理（含图像分割、目标检测）.md │ ├── 语义分割总结.md │ ├── 语义分割模型.md │ └── 语义分割评价准则.md ├── 图像标注工具.md ├── 如何将图像分割数据集制作为TFRecord文件.md ├── 摘入一些对人工智能的的看法和思考的内容.md └── 深度学习问题汇集.md ├── 05_journals ├── readme.md ├── 参考文献格式.md ├── 文献管理软件Zotero使用.md └── 期刊和会议.md ├── README.md ├── assets └── images │ ├── 2018爱分析·中国人工智能创新企业榜.png │ ├── [转]企业中人工智能相关岗位对硕士的要求.png │ ├── [转]对计算机视觉方向的看法.jpg │ └── 玩转AI面试.jpg ├── dl_examples └── MNIST │ ├── MNIST数据集二进制格式转换为图片.md │ ├── keras-mnist │ ├── mnist-keras.py │ └── mnist.npz │ └── 手写数字识别MNIST讲解.md └── other ├── matlab学习.md └── resources └── 学习资源.md /01_python/IDE之PyCharm的设置和Debug入门.md: -------------------------------------------------------------------------------- 1 | ### PyCharm 设置 2 | 3 | - [pycharm快捷键及一些常用设置](http://blog.csdn.net/fighter_yy/article/details/40860949) 4 | - [配置PyCharm(背景色+字体大小+解释器选择)](http://blog.csdn.net/vernice/article/details/50934869) 5 | - [PyCharm选择性忽略PEP8代码风格警告信息]() 6 | 7 | 本人对 PyCharm 一般会进行如下一些设置： 8 | 9 | - 主题更换为 PyCharm-monokai。可以去下载：[PyCharm 2018.2.1 使用 sublime-monokai 配色方案]()，源码地址： 10 | 11 | 导入下载的主题方法是：File -> Import Settings 选择下载的主题 jar 文件即可导入。 12 | 13 | - View -> Toolbar 勾选 14 | 15 | - 字体设置：File -> Settings -> Editor -> Font：Size 设为 14 或 15。或使用 Color Scheme 下的导入的主题 `PyCharm-monokai.jar` 的字体的设置。 16 | 17 | 18 | 19 | 20 | ### PyCharm 调试 21 | 22 | - [最全Pycharm教程（10）——Pycharm调试器总篇](https://blog.csdn.net/u013088062/article/details/50214459) 23 | 24 | 25 | ### PyCharm 代码调试快捷键： 26 | 27 | - F8：单步 28 | - F7：单步，进函数里面（无函数时同 F8） 29 | - Shift+F8：单步跳出 30 | - Alt+F9：运行到光标所在位置处 31 | - Alt+F8：测试语句 32 | - F9：重新运行程序（实际是到下个断点） 33 | - Ctrl+F8：切换断点 34 | - Ctrl+F8：查看断点 35 | 36 | Debug 相关功能 37 | 38 | - F8：step over 单步 39 | 40 | 遇到断点后，程序停止运行，按F8单步运行。 41 | 42 | - F7：step into 进入 43 | 44 | 配合F8使用。单步调试 F8 时，如果某行调用其他模块的函数，在此行 F7，可以进入函数内部，如果是 F8 则不会进入函数内容，直接单步到下一行。 45 | 46 | - Alt+shift+F7：step into mycode 47 | 48 | 个人理解 F8 和 F7 的综合。1、没遇到函数，和 F8 一样；2、遇到函数会自动进入函数内部，和 F8 时按 F7 类似的 49 | 50 | - shift+F8：跳出 51 | 52 | 调试过程中，F7进入函数内后，shift+F8跳出函数，会回到进入前调用函数的代码。不是函数地方shift+F8跳出，怎么用没太明白，但最终会执行到结束。 53 | 54 | - F9：resume program 55 | 56 | 按翻译是重启程序 ，实际是 下个断点，当打多个断点是，F9会到下一个断点 57 | 58 | 常用：F8，F9，其次 Alt+shift+F7，或 F7，shift+F8 59 | 60 | 61 | 62 | ### PyCharm 快捷键 63 | 64 | #### PyCharm for Windows 65 | 66 | ![](https://img-1256179949.cos.ap-shanghai.myqcloud.com/20190702172245.jpg) 67 | 68 | 69 | 70 | #### PyCharm for Mac 71 | 72 | ![](https://img-1256179949.cos.ap-shanghai.myqcloud.com/20190702171340.jpg) 73 | 74 | -------------------------------------------------------------------------------- /01_python/Jupyter的介绍、安装及使用.md: -------------------------------------------------------------------------------- 1 | ## 一、什么是 Jupyter Notebook？ 2 | 3 | ### 1. 简介 4 | 5 | Jupyter Notebook 是基于网页的用于交互计算的应用程序。其可被应用于全过程计算：开发、文档编写、运行代码和展示结果。——from：[Jupyter Notebook官方介绍]() 自行选择对应平台 Anaconda 下载安装。如果还不知道什么 Anaconda，建议网上找下资料了解下。Anaconda 的安装我就不多说了，这里贴个链接： 44 | 45 | 安装完 Anaconda 发行版时已经自动为你安装了 Jupyter Notebook 的，但如果没有自动安装，那么就在终端（Linux或macOS的“终端”，Windows的“Anaconda Prompt”，以下均简称“终端”）中输入以下命令安装： 46 | 47 | ``` python 48 | conda install jupyter notebook 49 | ``` 50 | 51 | **（2）使用 pip 安装** 52 | 53 | 如果你是有经验的 Python 玩家，可以使用 pip 命令来安装 Jupyter Notebook。下面的命令输入都是在终端当中进行的。 54 | 55 | 1. 把 pip 升级到最新版本 56 | 57 | - Python 3.x：`pip3 install --upgrade pip` 58 | - Python 2.x：`pip install --upgrade pip` 59 | 60 | 注意：老版本的 pip 在安装 Jupyter Notebook 过程中或面临依赖项无法同步安装的问题。因此强烈建议先把 pip 升级到最新版本。 61 | 62 | 2. 安装 Jupyter Notebook 63 | 64 | - Python 3.x：`pip3 install jupyter` 65 | - Python 2.x：`pip install jupyter` 66 | 67 | 68 | 69 | ## 三、运行和使用 Jupyter Notebook 70 | 71 | ### 1. 运行 72 | 73 | （1）在默认端口启动：`jupyter notebook` 74 | 75 | 执行命令之后，在终端中将会显示一系列 notebook 的服务器信息，同时浏览器将会自动启动 Jupyter Notebook。 76 | 77 | 注意：之后在 Jupyter Notebook 的所有操作，都请保持终端不要关闭，因为一旦关闭终端，就会断开与本地服务器的链接，你将无法在 Jupyter Notebook 中进行其他操作啦。 78 | 79 | 浏览器地址栏中默认地将会显示：`http://localhost:8888`。其中，“localhost”指的是本机，“8888”则是端口号。 80 | 81 | 如果你同时启动了多个 Jupyter Notebook，由于默认端口“8888”被占用，因此地址栏中的数字将从“8888”起，每多启动一个Jupyter Notebook数字就加 1，如“8889”、“8890”…… 82 | 83 | （2）如果你想自定义端口号来启动 Jupyter Notebook，可以在终端中输入以下命令：`jupyter notebook --port ` 84 | 85 | 其中，“”是自定义端口号，直接以数字的形式写在命令当中，数字两边不加尖括号“<>”。如：`jupyter notebook --port 9999`，即在端口号为“9999”的服务器启动 Jupyter Notebook。 86 | 87 | （3）如果你只是想启动 Jupyter Notebook 的服务器但不打算立刻进入到主页面，那么就无需立刻启动浏览器。在终端中输入：`jupyter notebook --no-browser` 88 | 89 | 此时，将会在终端显示启动的服务器信息，并在服务器启动之后，显示出打开浏览器页面的链接。当你需要启动浏览器页面时，只需要复制链接，并粘贴在浏览器的地址栏中，轻按回车变转到了你的 Jupyter Notebook 页面。 90 | 91 | ### 2. 使用 92 | 93 | 当执行完启动命令之后，浏览器将会进入到 Jupyter 的主页面。接下来就可以使用 Jupyter 了。 94 | 95 | 使用 Jupyter 进行基本的代码编写等工作，网上找下文章看下马上就能玩，相信不难，先暂时不记录了… 96 | 97 | （待更新… 98 | 99 | 以上内容大部分来源：[Jupyter Notebook介绍、安装及使用教程 - 知乎]() 100 | 101 | 102 | 103 | ## 四、修改 Jupyter Notebook 的默认工作目录 104 | 105 | 第一次打开 Anaconda 中自带的 Jupyter，默认路径是 `C:\Users\用户名\`，如果不想保存在该目录，可以进行修改，更换到别的目录。 106 | 107 | （1）在 Anaconda Prompt 中生成配置文件 108 | 109 | 打开 Anaconda Prompt，输入如下命令：`jupyter notebook --generate-config` 110 | 111 | （2）找到生成的 `jupyter_notebook_config.py` 文件(可以使用 everything 搜索)，文本打开后找到`#c.NotebookApp.notebook_dir = ''` ，将 `#` 注释标记去掉，填入自己的工作路径，如： 112 | 113 | ``` xml 114 | c.NotebookApp.notebook_dir = u'D:\jupyter-code' 115 | ``` 116 | 117 | 保存。（注意：工作路径不能出现中文，否则无法打开 Jupyter Notebook） 118 | 119 | 上面两步完成之后，打开 Jupyter，若没有成功更改工作目录，继续下面这一步。 120 | 121 | （3）修改 JupyterNotebook 快捷方式的目标属性 122 | 123 | 右击 JupyterNotebook 快捷方式，选择【属性】，删除【目标】属性中的【%USERPROFILE%】，点击【应用】–【确定】。 124 | 125 | 再次打开 JupyterNotebook 发现工作目录已经修改为我们自己的工作目录。 126 | 127 | ——from：[修改Jupyter Notebook的默认工作目录]() 128 | 129 | 130 | 131 | 132 | 133 | -------------------------------------------------------------------------------- /01_python/conda和pip常用命令速查.md: -------------------------------------------------------------------------------- 1 | 2 | 3 | ## conda常用命令速查 4 | 5 | 6 | 7 | ``` 8 | conda -V 或 conda --version #查看版本 9 | conda --help / conda -h 如：conda env -h 10 | 11 | conda update conda #更新conda 12 | 13 | conda create --name #创建环境，如 conda create --n mykeras python=3.6.4 14 | conda remove --name --all #删除环境 15 | 16 | conda info --envs 或 conda env list #显示已创建环境 17 | activate env-name #Linux/Mac: source activate # 进入环境 18 | deactivate #Linux/Mac: source deactivate 退出 19 | 20 | conda create --name flowers --clone snowflakes #制作环境的完整副本。这里我们将克隆snowflakes创建一个名为flowers的精确副本： 21 | 22 | #更换源 23 | conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/ 24 | conda config --set show_channel_urls yes 25 | 26 | conda search numpy # 查找package信息 27 | conda search --full-name python #查找全名为“python”的包有哪些版本可供安装。 28 | 29 | conda install -n py3 numpy # 安装package到名为py3环境,如果不用-n指定环境名称，则被安装在当前活跃环境 30 | conda install scipy # 安装scipy到当前活跃环境 31 | conda update --all 或者 conda upgrade --all # 更新所有包 32 | conda update -n py3 numpy # 更新名为py3环境的package 33 | conda remove -n py3 numpy # 删除名为py3环境的package 34 | 35 | ##conda 将 conda、python 等都视为 package： 36 | conda update conda # 更新conda，保持conda最新 37 | conda update anaconda # 更新anaconda 38 | conda update python # 更新python 39 | 40 | 分享环境： 41 | conda env export > environment.yml 42 | 小伙伴拿到environment.yml文件后，将该文件放在工作目录下，可以通过以下命令从该文件创建环境： 43 | conda env create -f environment.yml 44 | 45 | 46 | conda list #列举当前活跃环境下的所有包 47 | conda list -n your_env_name #列举一个非当前活跃环境下的所有包 48 | conda install -n env_name package_name #为指定环境安装某个包 49 | ``` 50 | 51 | 52 | 53 | ## pip 常用命令速查 54 | 55 | 56 | 57 | ``` 58 | pip install # 59 | 60 | 61 | ``` 62 | 63 | -------------------------------------------------------------------------------- /01_python/python_basic/Python一键安装所有第三方库.md: -------------------------------------------------------------------------------- 1 | # 如何一键安装所有第三方库文件？ 2 | 3 | ## pip freeze 4 | 5 | 6 | 7 | 在查看别人的 Python 项目时，经常会看到一个 `requirements.txt` 文件，里面记录了当前程序的所有依赖包及其精确版本号。这个文件有点类似与 Rails 的 Gemfile。其作用是用来在另一台 PC 上重新构建项目所需要的运行环境依赖。 8 | 9 | `requirements.txt` 用来记录项目所有的依赖包和版本号，只需要一个简单的 pip 命令就能完成。 10 | 11 | 进入到需要导出所有 Python 库的那个环境，然后使用那个环境下的 pip ： 12 | 13 | ``` python 14 | pip freeze > requirements.txt 15 | ``` 16 | 17 | > requirement.txt 文件默认输出在桌面。 18 | > 19 | > 注：查看源文件，pip 的 freeze 命令用于生成将当前项目的 pip 类库列表生成 requirements.txt 文件。 20 | 21 | 然后就可以用： 22 | 23 | ``` python 24 | pip install -r requirements.txt 25 | ``` 26 | 27 | 来一次性安装 `requirements.txt` 里面所有的依赖包，真是非常方便。 28 | 29 | 30 | 31 | `requirements.txt` 文件类似如下： 32 | 33 | ``` python 34 | Django=1.3.1 35 | South>=0.7 36 | django-debug-toolbar 37 | ``` 38 | 39 | 将模块放在一个列表中，每一行只有一项。 40 | 41 | --- 42 | 43 | 44 | 45 | ## pipreqs 46 | 47 | ### pipreqs 的作用 48 | 49 | > 一起开发项目的时候总是要搭建环境和部署环境的，这个时候必须得有个 python 第三方包的 list，一般都叫做requirements.txt。 如果一个项目使用时 virtualenv 环境，还好办 `pip freeze` 就可以解决，但是如果一个项目的依赖 list 没有维护，而且又是环境混用，那就不好整理的呀，不过，这里安利一个工具 pipreqs，可以自动根据源码生成 `requirements.txt`。 50 | 51 | pip freeze 命令： 52 | 53 | ``` python 54 | pip freeze > requirements.txt 55 | ``` 56 | 57 | 这种方式配合 virtualenv 才好使，否则把整个环境中的包都列出来了。 58 | 59 | 60 | 61 | ### pipreqs 的使用 62 | 63 | pipreqs 这个工具的好处是可以通过对项目目录的扫描，自动发现使用了那些类库，自动生成依赖清单。缺点是可能会有些偏差，需要检查并自己调整下。 64 | 65 | pipreqs 的安装：`pip install pipreqs` 66 | 67 | 使用方式也比较简单，直接进入项目下然后使用 `pipreqs ./` 命令即可，如： 68 | 69 | ``` 70 | pipreqs ./ 71 | ``` 72 | 73 | 如果是 Windows 系统，会报编码错误 (UnicodeDecodeError: 'gbk' codec can't decode byte 0xa8 in position 24: illegal multibyte sequence) 。这是由于编码问题所导致的，加上 encoding 参数即可，如下： 74 | 75 | ``` python 76 | pipreqs ./ --encoding=utf-8 77 | ``` 78 | 79 | 生成 `requirements.txt` 文件后，可以根据这个文件下载所有的依赖。 80 | 81 | ``` python 82 | pip install -r requriements.txt 83 | ``` 84 | 85 | 附： 86 | 87 | ``` xml 88 | 详细用法： 89 | pipreqs [options] 90 | 91 | 选项： 92 | --use-local仅使用本地包信息而不是查询PyPI 93 | --pypi-server 使用自定义PyPi服务器 94 | --proxy 使用Proxy，参数将传递给请求库。你也可以设置 95 | 96 | 终端中的环境参数： 97 | $ export HTTP_PROXY =“http://10.10.1.10:3128” 98 | $ export HTTPS_PROXY =“https://10.10.1.10:1080” 99 | --debug打印调试信息 100 | --ignore ...忽略额外的目录 101 | --encoding 使用编码参数打开文件 102 | --savepath 保存给定文件中的需求列表 103 | --print输出标准输出中的需求列表 104 | --force覆盖现有的requirements.txt 105 | --diff 将requirements.txt中的模块与项目导入进行比较。 106 | --clean 通过删除未在项目中导入的模块来清理requirements.txt。 107 | ``` 108 | 109 | 110 | 111 | 112 | 113 | ## 参考文章： 114 | 115 | - [python 批量导出项目所依赖的所有库文件及安装的方法（包导出与导入）]() 116 | - [Python使用requirements.txt安装类库](https://www.cnblogs.com/zknublx/p/5953921.html) 117 | - [浅谈pipreqs组件(自动生成需要导入的模块信息)](https://www.cnblogs.com/fu-yong/p/9213723.html) 118 | 119 | -------------------------------------------------------------------------------- /01_python/python_library/科学计算库之matplotlib的使用.md: -------------------------------------------------------------------------------- 1 | # matplotlib 学习 2 | 3 | -------------------------------------------------------------------------------- /01_python/python_library/科学计算库之pandas的使用.md: -------------------------------------------------------------------------------- 1 | # pandas 学习 2 | 3 | -------------------------------------------------------------------------------- /01_python/python_library/科学计算库之scikit-learn的使用.md: -------------------------------------------------------------------------------- 1 | # scikit-learn 学习 2 | 3 | ## 1. sklearn.preprocessing.LabelEncoder 4 | 5 | sklearn.preprocessing.LabelEncoder()：标准化标签，将标签值统一转换成range(标签值个数-1)范围内 6 | 7 | 以数字标签为例： 8 | 9 | ``` xml 10 | In [1]: from sklearn import preprocessing 11 | ...: le = preprocessing.LabelEncoder() 12 | ...: le.fit([1,2,2,6,3]) 13 | ...: 14 | Out[1]: LabelEncoder() 15 | ``` 16 | 17 | 参考：[sklearn.preprocessing.LabelEncoder]() 18 | 19 | LabelEncoder可以将标签分配一个0—n_classes-1之间的编码。将各种标签分配一个可数的连续编号： 20 | 21 | ``` python 22 | >>> from sklearn import preprocessing 23 | >>> le = preprocessing.LabelEncoder() 24 | >>> le.fit([1, 2, 2, 6]) 25 | LabelEncoder() 26 | >>> le.classes_ 27 | array([1, 2, 6]) 28 | >>> le.transform([1, 1, 2, 6]) # Transform Categories Into Integers 29 | array([0, 0, 1, 2], dtype=int64) 30 | >>> le.inverse_transform([0, 0, 1, 2]) # Transform Integers Into Categories 31 | array([1, 1, 2, 6]) 32 | ``` 33 | 34 | 35 | 36 | ``` python 37 | >>> le = preprocessing.LabelEncoder() 38 | >>> le.fit(["paris", "paris", "tokyo", "amsterdam"]) 39 | LabelEncoder() 40 | >>> list(le.classes_) 41 | ['amsterdam', 'paris', 'tokyo'] 42 | >>> le.transform(["tokyo", "tokyo", "paris"]) # Transform Categories Into Integers 43 | array([2, 2, 1], dtype=int64) 44 | >>> list(le.inverse_transform([2, 2, 1])) #Transform Integers Into Categories 45 | ['tokyo', 'tokyo', 'paris'] 46 | ``` 47 | 48 | ——from：[使用sklearn之LabelEncoder将Label标准化]() 49 | 50 | 51 | 52 | ## 2. 53 | 54 | 55 | 56 | ## 3. 57 | 58 | 59 | 60 | -------------------------------------------------------------------------------- /02_basic/deep_learning/cnn_models/ResNet.md: -------------------------------------------------------------------------------- 1 | ## （1） 2 | 3 | 参考：[残差网络ResNet解读(原创)](https://zhuanlan.zhihu.com/p/32702162) 4 | 5 | 何恺明提出了一种残差结构来实现上述恒等映射(图1)：整个模块除了正常的卷积层输出外，还有一个分支把输入直接连到输出上，该输出和卷积的输出做算术相加得到最终的输出，用公式表达就是H(x)=F(x)+x，x是输入，F(x)是卷积分支的输出，H(x)是整个结构的输出。可以证明如果F(x)分支中所有参数都是0，H(x)就是个恒等映射。残差结构人为制造了恒等映射，就能让整个结构朝着恒等映射的方向去收敛，确保最终的错误率不会因为深度的变大而越来越差。如果一个网络通过简单的手工设置参数值就能达到想要的结果，那这种结构就很容易通过训练来收敛到该结果，这是一条设计复杂的网络时百试不爽的规则。回想一下BN中为了在BN处理后恢复原有的分布，使用了y=rx+delta公式， 当手动设置r为标准差，delta为均值时，y就是BN处理前的分布，这就是利用了这条规则。 6 | 7 | ![](https://pic3.zhimg.com/80/v2-6d080f8db3d814c41d968a054132a44e_hd.jpg) 8 | 9 | 10 | 11 | [深度学习——--残差网络(ResNet)](https://statusrank.xyz/2018/09/03/ResNet/)（大部分内容也都讲到了，不过这里提一点，大部分文章，关于参数的计算都少了算了偏置） 12 | 13 | ![](https://img-1256179949.cos.ap-shanghai.myqcloud.com/20181228211241.png) 14 | 15 | 很明显，该网络是带有跳跃式结构的，残差网络借鉴了高速网络的跨层链接思想,但对其进行改进(残差项原本是带权值的，但是ResNet采用恒等映射代替) 16 | 假定某段神经网络的输入时x,期望输出是H(x),即H(x)是期望的复杂潜在映射,如果要是学习这样的模型,则训练的难度会比较大。回想我们前面的假设,如果学习到较饱和的准确率(或者发现下层的误差变大时),那么接下来的学习目标就会转变为恒等学习,也就是使输入x近似于输出H(x)，以保持在后面的层次中不会造成精度下降。 17 | 在上图的残差网络结构中,通过“shortcut connections(捷径连接)”的方式,直接把输入传到输出作为初始结果,使输出结果为H(x) = F(x) + x,当F(x) = 0,那么H(x) = x,也就是上面的恒等映射。于是,ResNet相当于将学习目标改变了,不再是学习一个完整的输出,而是目标值H(x)和x的差值,也就是所谓的残差**F(x) = H(x) - x**,因此后面的目标就是将残差结果逼近于0,使随着网络加深,准确率不下降。 18 | 19 | 关于残差连接的理解：[resnet中的残差连接，你确定真的看懂了？](https://zhuanlan.zhihu.com/p/42833949) 20 | 21 | 蒋竺波：[深度学习_残差网络 ResNet](http://i.youku.com/i/UNTU0NjkwNTQyNA==)，关于梯度弥散问题这里有讲解一番。 22 | 23 | ![](https://pic4.zhimg.com/80/v2-eb44512243a8e59c1fe02a7e6bd481db_hd.jpg) 24 | 25 | --- 26 | 27 | 28 | 29 | ## （2） 30 | 31 | 32 | 33 | 34 | 35 | -------------------------------------------------------------------------------- /02_basic/deep_learning/《深度学习xx问》.md: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | ## 什么学习中网络不收敛指的是什么？ 6 | 7 | 1、误差一直来回波动，进入不到容忍度内。 8 | 9 | 2、跟迭代不收敛或者系统不稳定差不多，上下波动不能趋近一个定值。 10 | 11 | 12 | 13 | ## 强化学习/增强学习(Reinforce Learning) 14 | 15 | 强化学习（Reinforcement Learning）的输入数据作为对模型的反馈，强调如何基于环境而行动，以取得最大化的预期利益。与监督式学习之间的区别在于，它并不需要出现正确的输入/输出对，也不需要精确校正次优化的行为。强化学习更加专注于在线规划，需要在探索（在未知的领域）和遵从（现有知识）之间找到平衡。——from： 16 | 17 | 强化学习是一类算法, 是让计算机实现从一开始什么都不懂, 脑袋里没有一点想法, 通过不断地尝试, 从错误中学习, 最后找到规律, 学会了达到目的的方法. 这就是一个完整的强化学习过程. 实际中的强化学习例子有很多. 比如近期最有名的 Alpha go, 机器头一次在围棋场上战胜人类高手, 让计算机自己学着玩经典游戏 Atari, 这些都是让计算机在不断的尝试中更新自己的行为准则, 从而一步步学会如何下好围棋, 如何操控游戏得到高分。——from：[什么是强化学习 - 知乎]() 18 | 19 | 20 | 21 | ## 生成对抗网络 GAN 22 | 23 | GAN：一种概率生成模型。简单说， **概率生成模型的目的，就是找出给定观测数据内部的统计规律，并且能够基于所得到的概率分布模型，产生全新的，与观测数据类似的数据**。 24 | 25 | 举个例子，概率生成模型可以用于自然图像的生成。假设给定1000万张图片之后，生成模型可以自动学习到其内部分布，能够解释给定的训练图片，并同时生成新的图片。 26 | 27 | 与庞大的真实数据相比，概率生成模型的参数个数要远远小于数据的数量。因此，在训练过程中，生成模型会被强迫去发现数据背后更为简单的统计规律，从而能够生成这些数据。——from：[深度学习新星：GAN的基本原理、应用和走向 | 硬创公开课 | 雷锋网](https://www.leiphone.com/news/201701/Kq6FvnjgbKK8Lh8N.html) 28 | 29 | 30 | 31 | ## 迁移学习 32 | 33 | 迁移学习（Transfer learning）顾名思义就是就是把已学训练好的模型参数迁移到新的模型来帮助新模型训练数据集。——from：https://feisky.xyz/machine-learning/transfer-learning.html 34 | 35 | 迁移学习(Transfer learning) 顾名思义就是就是把已学训练好的模型参数迁移到新的模型来帮助新模型训练。考虑到大部分数据或任务是存在相关性的，所以通过迁移学习我们可以将已经学到的模型参数（也可理解为模型学到的知识）通过某种方式来分享给新模型从而加快并优化模型的学习效率不用像大多数网络那样从零学习（starting from scratch，tabula rasa）。——from：[刘诗昆的回答 - 知乎](https://www.zhihu.com/question/41979241/answer/123545914) 36 | 37 | 近年来，我们越来越擅长训练深度神经网络，使其能从大量的有标签的数据中学习非常准确的输入到输出的映射，无论它们是图像、语句还是标签预测。 38 | 39 | 我们的模型仍旧极度缺乏泛化到不同于训练的环境的能力。什么时候需要这种能力呢？就是你每一次将你的模型用到现实世界，而不是精心构建的数据集的时候。现实世界是混乱的，并且包含大量全新的场景，其中很多是你的模型在训练的时候未曾遇到的，因此这又使得模型不足以做出好的预测。将知识迁移到新环境中的能力通常被称为迁移学习（transfer learning），这就是本文将讨论的内容。——from：[机器之心的回答 - 知乎](https://www.zhihu.com/question/41979241/answer/208177153) 40 | 41 | 42 | 43 | ## 什么是什么是微调（Fine-tuning） 44 | 45 | 针对于某个任务，自己的训练数据不多，那怎么办？ 没关系，我们先找到一个同类的别人训练好的模型，把别人现成的训练好了的模型拿过来，换成自己的数据，调整一下参数，再训练一遍，这就是微调（fine-tune）。 46 | 47 | **为什么要微调？** 48 | 49 | 1. 对于数据集本身很小（几千张图片）的情况，从头开始训练具有几千万参数的大型神经网络是不现实的，因为越大的模型对数据量的要求越大，过拟合无法避免。这时候如果还想用上大型神经网络的超强特征提取能力，只能靠微调已经训练好的模型。 50 | 2. 可以降低训练成本：如果使用导出特征向量的方法进行迁移学习，后期的训练成本非常低，用 CPU 都完全无压力，没有深度学习机器也可以做。 51 | 3. 前人花很大精力训练出来的模型在大概率上会比你自己从零开始搭的模型要强悍，没有必要重复造轮子。 52 | 53 | **迁移学习 Transfer Learning：** 54 | 55 | 总是有人把 迁移学习和神经网络的训练联系起来，这两个概念刚开始是无关的。 迁移学习是机器学习的分支，现在之所以 迁移学习和神经网络联系如此紧密，现在图像识别这块发展的太快效果也太好了，所以几乎所有的迁移学习都是图像识别方向的，所以大家看到的迁移学习基本上都是以神经网络相关的计算机视觉为主，本文中也会以这方面来举例子 56 | 57 | 迁移学习初衷是节省人工标注样本的时间，让模型可以通过一个已有的标记数据的领域向未标记数据领域进行迁移从而训练出适用于该领域的模型，直接对目标域从头开始学习成本太高，我们故而转向运用已有的相关知识来辅助尽快地学习新知识 58 | 59 | 举一个简单的例子就能很好的说明问题，我们学习编程的时候会学习什么？ 语法、特定语言的API、流程处理、面向对象，设计模式，等等 60 | 61 | 这里面语法和 API 是每一个语言特有的，但是面向对象和设计模式可是通用的，我们学了JAVA，再去学C#，或者Python，面向对象和设计模式是不用去学的，因为原理都是一样的，甚至在学习C#的时候语法都可以少学很多，这就是迁移学习的概念，把统一的概念抽象出来，只学习不同的内容。 62 | 63 | 迁移学习按照学习方式可以分为基于样本的迁移，基于特征的迁移，基于模型的迁移，以及基于关系的迁移，这里就不详细介绍了。 64 | 65 | > 其实 "Transfer Learning" 和 "Fine-tune" 并没有严格的区分，含义可以相互交换，只不过后者似乎更常用于形容迁移学习的后期微调中。 我个人的理解，微调应该是迁移学习中的一部分。微调只能说是一个trick。 66 | 67 | 参考：[zergtant/pytorch-handbook：4.1-fine-tuning.ipynb]() -------------------------------------------------------------------------------- /02_basic/deep_learning/卷积神经网络以及图像分割论文整理和解读.md: -------------------------------------------------------------------------------- 1 | # 卷积神经网络发展 2 | 3 | ## 2012 AlexNet 4 | 5 | 6 | 7 | ## 2014 VGGNet 8 | 9 | 10 | 11 | ## 2014 SPPNet 12 | 13 | 14 | 15 | ## 2014 GoogLeNet(Inception v1) 16 | 17 | - [GoogLeNet 之 Inception(V1-V4)](https://blog.csdn.net/hejin_some/article/details/78636586) 18 | - [Inception的前世今生（一）--GoogLeNet](https://blog.csdn.net/shwan_ma/article/details/78933055) 19 | 20 | 21 | 22 | ## 2015 Inception v2 23 | 24 | 25 | 26 | ## 2015 Inception v3 27 | 28 | 29 | 30 | ## 2015 ResNet 31 | 32 | 33 | 34 | ## 2016 Inception v4 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | ## 2016 Xception 43 | 44 | 45 | 46 | ## 2016 DenseNet 47 | 48 | 49 | 50 | 51 | 52 | ## 参考资料 53 | 54 | - [CNN的发展历史（LeNet,Alexnet,VGGNet,GoogleNet,ReSNet）](https://blog.csdn.net/Liu941027/article/details/78297462) 55 | - [Lenet、Alexnet 、VGG、 GoogleNet、ResNet模型](https://blog.csdn.net/u013989576/article/details/71600795) -------------------------------------------------------------------------------- /02_basic/deep_learning/我的理解：神经网络参数改变过程？.md: -------------------------------------------------------------------------------- 1 | CSDN：[反向传播算法的直观理解](https://blog.csdn.net/mao_xiao_feng/article/details/53048213) 2 | 3 | 知乎：[AI从入门到放弃：BP神经网络算法推导及代码实现笔记](https://zhuanlan.zhihu.com/p/38006693) 4 | 5 | 看完两篇文章之后，对常见的神经网络代码运行过程的理解，以 MNIST 为例子： 6 | 7 | ![](https://img-1256179949.cos.ap-shanghai.myqcloud.com/20190302204703.png) 8 | 9 | 上面这个是没有使用 CNN 网络模型实现的。 10 | 11 | **其中可以看到框出来的代码，表示每次读取 batch_size=100 张图片作为输入，即100x28x28 作为输入 --> 进行 n_batch =55000//100 个次 --> 结束之后就把所有数据训练了一遍 --> 再把如上过程进行了22 个epoch（即22个轮回）。** 12 | 13 | 如果该过程拿到常见的**神经网络模型代码来讲**，大概是这样的： 14 | 15 | 1. 代码开始运行，整个神经网络的权值参数（即 filter 参数）会进行一次初始化； 16 | 17 | 2. **【进行第一个epoch】**进行第一个 batch_size 的数据作为输入，**进行一次正向传播（使用第一步初始化的参数值）**，得到的结果再与真实数据（与label）比较"差距"，理解为损失函数 loss，接下来就要最小化 loss 的值，怎么最小化呢？ 18 | 19 | 3. 进行反向传播，反向传播的细节参考最前面两篇文章看看。【我记录下重点】以开头第一篇文章为例，反向传播之后，最后更新参数，如下： 20 | 21 | ![](https://img-1256179949.cos.ap-shanghai.myqcloud.com/20190302205000.png) 22 | 23 | 这样误差反向传播法就完成了，这样我们得到了一**组新的参数值**（而不是之前最开始初始化的参数值）。 24 | 25 | 4. 再进行第二个 batch_size 的数据作为输入，**进行一次正向传播（注意：这里使用的是上一步得到的新的参数值）**，得到的结果再与真实数据（与label）进行比较"差距"，最小化 loss，同上过程一样，再然后进行反向传播，**得到新的的参数值；** 26 | 27 | 5. 进行第三个、第四个、第五个 batch_size...... 直到所有数据都有作为输入数据； 28 | 29 | 6. **【再进行第二个epoch】**在这个过程的的第一个 batch_size 过程中使用的参数为：**第一个 epoch 中的最后一个 batch_size 后得到的参数进行正向传播**。 30 | 31 | 7. **【再进行第三、第四、第五个epoch。。。。。】**直到结束！ 32 | 33 | 8. 注意：其实这里也有可能在第一个 epoch 完毕，得到的参数就是最优的。 34 | -------------------------------------------------------------------------------- /02_basic/deep_learning/激活函数之Sigmoid&ReLU&Softmax.md: -------------------------------------------------------------------------------- 1 | # （1） 2 | 3 | - Rectified Linear Unit(ReLU) - 用于隐层神经元输出 4 | - Sigmoid - 用于隐层神经元输出 5 | - Softmax - 用于多分类神经网络输出 6 | - Linear - 用于回归神经网络输出（或二分类问题） 7 | 8 | Softmax 激活函数只用于多于一个输出的神经元，它保证所以的输出神经元之和为1.0，所以一般输出的是小于1的概率值，可以很直观地比较各输出值。 9 | 10 | 11 | 12 | ## 为什么选择ReLU？ 13 | 14 | 深度学习中，我们一般使用 ReLU 作为中间隐层神经元的激活函数，AlexNet 中提出用 ReLU 来替代传统的激活函数是深度学习的一大进步。我们知道，sigmoid 函数的图像如下： 15 | 16 | ![](https://img-1256179949.cos.ap-shanghai.myqcloud.com/20190424093949.png) 17 | 18 | 而一般我们优化参数时会用到误差反向传播算法，即要对激活函数求导，得到sigmoid函数的瞬时变化率，其导数表达式为：![](https://img-1256179949.cos.ap-shanghai.myqcloud.com/20190424094005.png) 19 | 20 | 对应的图形如下： 21 | 22 | ![](https://img-1256179949.cos.ap-shanghai.myqcloud.com/20190424094019.png) 23 | 24 | 由图可知，导数从0开始很快就又趋近于0了，易造成“梯度消失”现象，而ReLU的导数就不存在这样的问题，它的导数表达式如下： 25 | 26 | ![](https://img-1256179949.cos.ap-shanghai.myqcloud.com/20190424094039.png) 27 | 28 | Relu函数的形状如下（蓝色）： 29 | 30 | ![](https://img-1256179949.cos.ap-shanghai.myqcloud.com/20190424094056.png) 31 | 32 | 对比sigmoid类函数主要变化是：1）单侧抑制 2）相对宽阔的兴奋边界 3）稀疏激活性。这与人的神经皮层的工作原理接近。 33 | 34 | ## 为什么需要偏移常量？ 35 | 36 | 通常，要将输入的参数通过神经元后映射到一个新的空间中，我们需要对其进行加权和偏移处理后再激活，而不仅仅是上面讨论激活函数那样，仅对输入本身进行激活操作。比如sigmoid激活神经网络的表达式如下： 37 | 38 | ![](https://img-1256179949.cos.ap-shanghai.myqcloud.com/20190424094129.png) 39 | 40 | x是输入量，w是权重，b是偏移量（bias）。这里，之所以会讨论sigmoid函数是因为它能够很好地说明偏移量的作用。 41 | 42 | 权重w使得sigmoid函数可以调整其倾斜程度，下面这幅图是当权重变化时，sigmoid函数图形的变化情况： 43 | 44 | ![](https://img-1256179949.cos.ap-shanghai.myqcloud.com/20190424094158.png) 45 | 46 | 上面的曲线是由下面这几组参数产生的： 47 | 48 | ![](https://img-1256179949.cos.ap-shanghai.myqcloud.com/20190424094229.png) 49 | 50 | 我们没有使用偏移量b（b=0），从图中可以看出，无论权重如何变化，曲线都要经过（0,0.5）点，但实际情况下，我们可能需要在x接近0时，函数结果为其他值。下面我们改变偏移量b，它不会改变曲线大体形状，但是改变了数值结果： 51 | 52 | ![](https://img-1256179949.cos.ap-shanghai.myqcloud.com/20190424094242.png) 53 | 54 | 上面几个sigmoid曲线对应的参数组为： 55 | 56 | ![](https://img-1256179949.cos.ap-shanghai.myqcloud.com/20190424094254.png) 57 | 58 | 这里，我们规定权重为 1，而偏移量是变化的，可以看出它们向左或者向右移动了，但又在左下和右上部位趋于一致。 59 | 60 | 当我们改变权重 w 和偏移量 b 时，可以为神经元构造多种输出可能性，这还仅仅是一个神经元，在神经网络中，千千万万个神经元结合就能产生复杂的输出模式。 61 | 62 | 参考： 63 | 64 | - [深度学习常用激活函数之— Sigmoid & ReLU & Softmax](https://blog.csdn.net/Leo_Xu06/article/details/53708647) 65 | - [交叉熵代价函数（cross-entropy cost function）]() 66 | 67 | #（2） 68 | 69 | > 只是映射了之后，相对大小没发生变化。该大的还是大，该小的还是小。人们为了方便当做概率罢了。 70 | > 71 | > ——from：通过逻辑回归的sigmoid函数把线性回归转化到[0,1]之间，这个值为什么可以代表概率？ - zkjjj的回答 - 知乎 72 | > https://www.zhihu.com/question/41647192/answer/221104515 73 | 74 | > 怎么说呢，就拿一个简单LR二分类举例吧，识别是否为西瓜。 75 | > 我们拿掉sigmoid，结果输出为一个值，例如100，然后模型训练结果是大于60就属于西瓜，没问题。 76 | > 现在加上sigmoid，整个输出值的范围从之前的正负无穷压缩到0到1，因为我们不能取到0和1，只是不断趋近，那么越接近1就说明原来的值越接近无限大，也就是离分割线（超平面）越远，那么属于正例的可能自然无限大了。 77 | > 简单来说，sigmoid只是把可能性大小压缩到0到100%之内，方便计算而已。 78 | > 79 | > ——from：https://www.zhihu.com/question/41647192/answer/216501244 80 | 81 | 82 | 83 | 本质上来说，Softmax 属于离散概率分布而 Sigmoid 是非线性映射。分类其实就是设定一个阈值，然后我们将想要分类的对象与这个阈值进行比较，根据比较结果来决定分类。**Softmax 函数能够将一个K维实值向量归一化**，所以它主要被用于多分类任务；**Sigmoid 能够将一个实数归一化**，因此它一般用于二分类任务。特别地，当 Softmax 的维数 K=2 时，Softmax 会退化为 Sigmoid 函数。——from：[快速理解Softmax和Sigmoid]([https://lolimay.cn/2019/01/14/%E5%BF%AB%E9%80%9F%E7%90%86%E8%A7%A3-Softmax-%E5%92%8C-Sigmoid/](https://lolimay.cn/2019/01/14/快速理解-Softmax-和-Sigmoid/)) -------------------------------------------------------------------------------- /02_basic/dl-papers.md: -------------------------------------------------------------------------------- 1 | ## Basic 2 | 3 | - 《A guide to convolution arithmetic for deep》[[Paper](https://arxiv.org/abs/1603.07285)] 4 | - 《Bag of Tricks for Image Classification with Convolutional Neural Networks》[[Paper](https://arxiv.org/abs/1812.01187)] 5 | 6 | 7 | 8 | 9 | 10 | ## CNN 11 | 12 | - [1989] LeNet：《Gradient-Based Learning Applied to document Recognition》[[Paper](http://vision.stanford.edu/cs598_spring07/papers/Lecun98.pdf)] 13 | 14 | - [2012] AlexNet：《ImageNet Classification with Deep Convolutional 15 | Neural Networks》[[Paper](https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf)] 16 | 17 | - [2014] Inception v1：《Going deeper with convolutions》[[Paper](https://arxiv.org/abs/1409.4842)] 18 | 19 | 注：先前叫 GoogLeNet，现在简单地被称为 Inception vN，其中 N 指的是由 Google 定的版本号。 20 | 21 | - [2014] VGGNet：《Very Deep Convolutional Networks for Large-Scale Image Recognition》[[Paper](https://arxiv.org/abs/1409.1556v6)] 22 | 23 | - [2015] Inception v2：《Batch Normalization Accelerating Deep Network Training by Reducing Internal Covariate Shift》[[Paper](https://arxiv.org/abs/1502.03167)] 24 | 25 | - [2015] Inception v3：《Rethinking the Inception Architecture for Computer Vision》[[Paper](https://arxiv.org/abs/1512.00567)] 26 | 27 | - [2015] ResNet：《Deep Residual Learning for Image Recognition》[[Paper](https://arxiv.org/abs/1512.03385)] 28 | 29 | - [2016] Inception v4：《Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning》[[Paper](https://arxiv.org/abs/1602.07261)] -------------------------------------------------------------------------------- /02_basic/machine_learning/机器学习.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/02_basic/machine_learning/机器学习.md -------------------------------------------------------------------------------- /02_basic/math_basic/数学基础.md: -------------------------------------------------------------------------------- 1 | ## 统计学概率论基础认识 2 | 3 | ### 什么是正太分布？高斯分布？ 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | -------------------------------------------------------------------------------- /03_dl_framework/keras/readme.md: -------------------------------------------------------------------------------- 1 | # keras 学习 2 | 3 | - [【笔记】Keras 学习笔记.md](./keras-learning.md) [荐]★★★ 4 | - [主页 - Keras 中文文档]() 5 | - [详解keras的model.summary()输出参数Param计算过程]() - 关于 console 台打印的参数数量的计算。 6 | - [tensorflow - What does 'non-trainable params' mean? - Stack Overflow]() - console 台打印的 non-trainable params 的理解。 7 | 8 | 9 | 10 | ## 模型可视化 11 | 12 | - [模型可视化 - Keras中文文档]() | [可视化 Visualization - Keras 中文文档]() 13 | 14 | ## 迁移学习/预训练 15 | 16 | - [预训练模型Application - Keras中文文档]() 17 | 18 | ## 导入模型测试 19 | 20 | 1、[Keras加载预训练模型 - 豌豆ip代理]() [荐] 21 | 22 | ``` 23 | 比如训练模型的时候用到了自定义的模块AttentionLayer，那么在加载模型的时候需要在custom_objects的参数中声明对应的字典项，否则将会报模块未定义的错误。 24 | 25 | model = load_model('./model1/GRUAttention( 0.8574).h5', custom_objects={'AttentionLayer': AttentionLayer}) 26 | 在训练的过程中有时候也会用到自定义的损失函数，这时候如果你加载模型知识为了进行预测不再其基础上再进行训练，那么加载模型的时候就没有必要在custom_objects参数中声明对应的字典项，只需要将compile参数设为False即可： 27 | 28 | model = load_model('./model1/GRUAttention(0.8574).h5', compile=False}) 29 | 如果此时你好需要在加载后的模型上继续进行训练，那么声明损失函数对应的字典项就是必须的： 30 | 31 | model = load_model('./model1/GRUAttention(0.8574).h5', compile=True, custom_objects={'focal_loss_fixed':focal_loss}) 32 | ``` 33 | 34 | 35 | 36 | ## 使用多 GPU 37 | 38 | - [如何让keras训练深度网络时使用两张显卡？ - 知乎]() 39 | - [keras 关于使用多个 gpu]() | [Keras同时用多张显卡训练网络 - 简书]() 40 | - [Keras多GPU及分布式]() - 有两种方法可以在多张 GPU 上运行一个模型：数据并行/设备并行。大多数情况下，你需要的很可能是“数据并行”。 41 | 42 | ## 相关文章 43 | 44 | 1、[Keras中的多分类损失函数categorical_crossentropy]() 45 | 46 | ``` 47 | 注意：当使用`categorical_crossentropy`损失函数时，你的标签应为多类模式，例如如果你有 10 个类别，每一个样本的标签应该是一个 10 维的向量，该向量在对应有值的索引位置为 1 其余为 0。 48 | 49 | 可以使用这个方法进行转换： 50 | 51 | from keras.utils.np_utils import to_categorical 52 | categorical_labels = to_categorical(int_labels, num_classes=None) 53 | ``` 54 | 55 | 2、[keras中的keras.utils.to_categorical方法]() - `to_categorical(y, num_classes=None, dtype='float32')` 56 | 57 | ``` 58 | 将整型标签转为 onehot。y 为 int 数组，num_classes 为标签类别总数，大于 max(y)（标签从0开始的）。 59 | 60 | 返回：如果 num_classes=None，返回 len(y)*[max(y)+1]（维度，m*n表示m行n列矩阵，下同），否则为 len(y)*num_classes。说出来显得复杂，请看下面实例。 61 | ``` 62 | 63 | 3、[keras中的回调函数]() 64 | 65 | 4、[为何Keras中的CNN是有问题的，如何修复它们？ - 知乎]() - 关于参数初始化的问题。 66 | 67 | > 我们证明，初始化是模型中特别重要的一件事情，这一点你可能经常忽略。此外，文章还证明，即便像 Keras 这种卓越的库中的默认设置，也不能想当然拿来就用。 -------------------------------------------------------------------------------- /03_dl_framework/pytorch/notes/pytorch的坑.md: -------------------------------------------------------------------------------- 1 | ### 1. nn.Module.cuda() 和 Tensor.cuda() 的作用效果差异 2 | 3 | 无论是对于模型还是数据，cuda()函数都能实现从CPU到GPU的内存迁移，但是他们的作用效果有所不同。 4 | 5 | 对于 nn.Module： 6 | 7 | ```numpy 8 | model = model.cuda() 9 | model.cuda() 10 | ``` 11 | 12 | 上面两句能够达到一样的效果，即对model自身进行的内存迁移。 13 | 14 | **对于Tensor:** 15 | 16 | 和nn.Module不同，调用tensor.cuda()只是返回这个tensor对象在GPU内存上的拷贝，而不会对自身进行改变。因此必须对tensor进行重新赋值，即tensor=tensor.cuda(). 17 | 18 | **例子:** 19 | 20 | ```text 21 | model = create_a_model() 22 | tensor = torch.zeros([2,3,10,10]) 23 | model.cuda() 24 | tensor.cuda() 25 | model(tensor) # 会报错 26 | tensor = tensor.cuda() 27 | model(tensor) # 正常运行 28 | ``` 29 | 30 | 31 | 32 | ### 2. PyTorch 0.4 计算累积损失的不同 33 | 34 | 以广泛使用的模式total_loss += loss.data[0]为例。Python0.4.0之前，loss是一个封装了(1,)张量的Variable，但Python0.4.0的loss现在是一个零维的标量。对标量进行索引是没有意义的（似乎会报 invalid index to scalar variable 的错误）。使用loss.item()可以从标量中获取Python数字。所以改为： 35 | 36 | ```text 37 | total_loss += loss.item() 38 | ``` 39 | 40 | 如果在累加损失时未将其转换为Python数字，则可能出现程序内存使用量增加的情况。这是因为上面表达式的右侧原本是一个Python浮点数，而它现在是一个零维张量。因此，总损失累加了张量和它们的梯度历史，这可能会产生很大的autograd 图，耗费内存和计算资源。 41 | 42 | 43 | 44 | ### 4. torch.Tensor.detach()的使用 45 | 46 | detach()的官方说明如下： 47 | 48 | > Returns a new Tensor, detached from the current graph. The result will never require gradient. 49 | 50 | 假设有模型A和模型B，我们需要将A的输出作为B的输入，但训练时我们只训练模型B. 那么可以这样做： 51 | 52 | ```text 53 | input_B = output_A.detach() 54 | ``` 55 | 56 | 它可以使两个计算图的梯度传递断开，从而实现我们所需的功能。 57 | 58 | 59 | 60 | ### 5. ERROR: Unexpected bus error encountered in worker. This might be caused by insufficient shared memory (shm) 61 | 62 | 出现这个错误的情况是，在服务器上的docker中运行训练代码时，batch size设置得过大，shared memory不够（因为docker限制了shm）.解决方法是，将Dataloader的num_workers设置为0. 63 | 64 | 65 | 66 | ### 6. pytorch中loss函数的参数设置 67 | 68 | 以CrossEntropyLoss为例： 69 | 70 | ```text 71 | CrossEntropyLoss(self, weight=None, size_average=None, ignore_index=-100, reduce=None, reduction='elementwise_mean') 72 | ``` 73 | 74 | - 若 reduce = False，那么 size_average 参数失效，直接返回向量形式的 loss，即batch中每个元素对应的loss. 75 | 76 | - 若 reduce = True，那么 loss 返回的是标量： 77 | 78 | - - 如果 size_average = True，返回 loss.mean(). 79 | - 如果 size_average = False，返回 loss.sum(). 80 | 81 | - weight : 输入一个1D的权值向量，为各个类别的loss加权，如下公式所示： 82 | 83 | ![](https://pic4.zhimg.com/80/v2-edc541e40d4e082f91ee9a10273ef673_hd.png) 84 | 85 | - ignore_index : 选择要忽视的目标值，使其对输入梯度不作贡献。如果 size_average = True，那么只计算不被忽视的目标的loss的均值。 86 | - reduction : 可选的参数有：‘none’ | ‘elementwise_mean’ | ‘sum’, 正如参数的字面意思，不解释。 87 | 88 | 89 | 90 | ### 7. 在做一些metric learning时候，可能提高模型准确率的技巧 91 | 92 | 来源于知乎大佬的评论区： 93 | 94 | - 背景：BFE-net 做reid时候backbone选择resnet50，但是没有进行warm-up 后准确率没有达到预期，大佬们的对话和解决方法： 95 | 96 | ![](https://pic2.zhimg.com/80/v2-1b36f9ad21cfd8ac1443d80338647851_hd.jpg) 97 | 98 | 找到作者git commit的记录: 99 | 100 | ```text 101 | if ep < 50: 102 | lr = 1e-4*(ep//5+1) 103 | elif ep < 200: 104 | lr = 1e-3 105 | elif ep < 300: 106 | lr = 1e-4 107 | ``` 108 | 109 | 结论：在2月份新加了2行代码,简单来说就是在前50个epoch 用较低的learning rate 去预热，后面慢慢恢复正常的lr 110 | 111 | ### 8. 使用nn.Dataparallel 数据不在同一个gpu上 112 | 113 | - 背景：pytorch 多GPU训练主要是采用数据并行方式： 114 | 115 | ```text 116 | model = nn.DataParallel(model) 117 | ``` 118 | 119 | - 问题：但是一次同事训练基于光流检测的实验时发现 data not in same cuda,做代码review时候，打印每个节点tensor，cuda里的数据竟然没有分布在同一个gpu上 120 | - 解决：最终解决方案是在数据，吐出后统一进行执行.cuda()将数据归入到同一个cuda流中解决了该问题。 121 | 122 | 123 | 124 | ### **9**.pytorch model load可能会踩到的坑： 125 | 126 | 如果使用了nn.Dataparallel 进行多卡训练在读入模型时候要注意加.module， 代码如下: 127 | 128 | ```text 129 | def get_model(self): 130 | if self.nGPU == 1: 131 | return self.model 132 | else: 133 | return self.model.module 134 | ``` 135 | 136 | 137 | 138 | ### **10**.pytorch .h5 数据读入问题： 139 | 140 | - 背景：我们知道Torch框架需要符合其自身规格的输入数据的格式，在图像识别中用到的是以.t7扩展名的文件类型，同时也有h5格式类型，这种类型的和t7差不多，均可被torch框架使用，但在读入时候有个官方BUG 141 | - 问题：[DataLoader, when num_worker >0, there is bug]() 读入.h5 数据格式时候如果dataloader>0 内存会占满，并报错 142 | - 问题解决： 143 | 144 | ```text 145 | # 测试数据 146 | f = h5py.File('test.h5') 147 | for i in range(256): 148 | f['%s/data' % i] = np.random.uniform(0, 1, (1024, 1024)) 149 | f['%s/target' % i] = np.random.choice(1000) 150 | ``` 151 | 152 | 解决： 153 | 154 | ```text 155 | # 错误发生 156 | dataloader = torch.utils.data.DataLoader( 157 | H5Dataset('test.h5'), 158 | batch_size=32, 159 | num_workers=8, 160 | shuffle=True 161 | ) 162 | # 解决，num_workers 改为0 163 | dataloader = torch.utils.data.DataLoader( 164 | H5Dataset('test.h5'), 165 | batch_size=32, 166 | num_workers=0, 167 | shuffle=True 168 | ) 169 | ``` 170 | 171 | *——from：* 172 | 173 | 174 | 175 | --- 176 | 177 | 178 | 179 | -------------------------------------------------------------------------------- /03_dl_framework/pytorch/pytorch学习.md: -------------------------------------------------------------------------------- 1 | # 学习记录 2 | 3 | 深度学习框架 PyTorch 学习记录。 4 | 5 | PyTorch 中文文档： [荐] 6 | 7 | 8 | 9 | ## 我的笔记 10 | 11 | - [01-pytorch简介、安装(Linux、Windows、Mac.md)](./notes/01-pytorch简介、安装(Linux、Windows、Mac).md) 12 | - 13 | 14 | 15 | 16 | ## pytorch博文 17 | 18 | - [PyTorch的4分钟教程，手把手教你完成线性回归 - 云+社区 - 腾讯云]() 19 | - [PyTorch使用tensorboardX - 知乎]() 20 | 21 | 22 | 23 | 24 | 25 | ## pytorch的坑 26 | 27 | - [PyTorch踩过的坑 - 知乎]() 28 | 29 | 30 | 31 | 32 | 33 | ## 学习资料 34 | 35 | 书籍： 36 | 37 | - 廖星宇《深度学习入门之PyTorch》 [[书籍代码仓库地址](https://github.com/L1aoXingyu/code-of-learn-deep-learning-with-pytorch)] 38 | - 陈云《深度学习框架PyTorch：入门与实践》 [[书籍代码仓库地址](https://github.com/chenyuntc/pytorch-book)] 39 | - Eli Stevens《Deep Learning with PyTorch》（中文名：《利用PyTorch进行深度学习》）[[下载](https://www.manning.com/books/deep-learning-with-pytorch)] | [[书籍代码仓库地址](https://github.com/svishnu88/DLwithPyTorch)] 40 | 41 | GitHub： 42 | 43 | - [INTERMT/Awesome-PyTorch-Chinese]() 44 | 45 | > 【干货】史上最全的PyTorch学习资源汇总 [http://www.pytorchchina.com](http://www.pytorchchina.com/) 46 | 47 | - [zergtant/pytorch-handbook](https://github.com/zergtant/pytorch-handbook/blob/master/chapter1/1.1-pytorch-introduction.md) 48 | 49 | > 这是一本开源的书籍，目标是帮助那些希望和使用PyTorch进行深度学习开发和研究的朋友快速入门。 50 | > 51 | > 由于本人水平有限，在写此教程的时候参考了一些网上的资料，在这里对他们表示敬意，我会在每个引用中附上原文地址，方便大家参考。 52 | > 53 | > 深度学习的技术在飞速的发展，同时PyTorch也在不断更新，且本人会逐步完善相关内容。 54 | 55 | 56 | 视频： 57 | 58 | - 《深度学习与PyTorch入门实战教程》：[YouTube在线观看](https://www.youtube.com/watch?v=rDhBcP4ikpA&list=PLDzdzeKX7DWep2KyJwJ-BmYciXTARvZO6) - 来源网易云课堂视频课程。 59 | 60 | 61 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week01-src/01-Tensorflow简介，Anaconda安装，Tensorflow的CPU版本安装.md: -------------------------------------------------------------------------------- 1 | ## Anaconda安装 2 | 3 | 1）Window、MacOS、Linux 都已支持 Tensorflow。 4 | 5 | 2）Window 用户只能使用 python3.5(64bit)。MacOS、Linux 支持 python2.7 和 python3.3+。 6 | 7 | 3）有 GPU 可以安装带 GPU 版本的，没有 GPU 就安装 CPU 版本的。 8 | 9 | 4） 推荐安装 Anaconda，pip 版本大于 8.1。 10 | 11 | 在学习过程中，建议使用 Jupyter Notebook 编程（当然也可以用其他工具，如 PyCharm），因安装完 Anaconda 自带 Jupyter Notebook，可以找到直接打开即可开始。 12 | 13 | 打开之后会打开默认浏览器，地址`http://localhost:8888`，然后就可以在浏览器下面新建文件进行代码编写等操作了，其中，默认保存的路径为 C 盘用户文件夹下，如：`C:\Users\用户名`。我们可以修改路径为我们自己的想要的目录之下，修改操作： 14 | 15 | 1. 打开 cmd 输入命令`jupyter notebook --generate-config`，可以看到生成文件的路径（可以看到默认在`C:\Users\用户名\.jupyter`文件夹下），这个就是生成的配置文件`jupyter_notebook_config.py` 16 | 17 | ![](https://img-1256179949.cos.ap-shanghai.myqcloud.com/18-10-8-14115072.jpg) 18 | 19 | 2. 然后打开这个配置文件，找到`#c.NotebookApp.notebook_dir = ' '`，把它改成： 20 | 21 | ``` xml 22 | c.NotebookApp.notebook_dir = '你想要设置的路径' 23 | ``` 24 | 25 | 如：`c.NotebookApp.notebook_dir = 'D:/Python/jupyter'`，那么以后再 Jupyter 上保存的文件就在`D:/Python/jupyter`文件夹里了。 26 | 27 | 注1：如果想要检测是否修改成功，可以在你设置的目录下里添加一个文件夹，例如：我在 jupyter 文件夹里添加了文件夹 text，那么在 Jupyter（浏览器上）就可以看到该文件夹了，则表明修改路径成功！ 28 | 29 | 注2：在进行如上修改之后，我发现未成功。网上找到了篇文章（[Anaconda Jupyter默认路径及修改无效解决方案](https://blog.csdn.net/mirrorui_/article/details/80605613)）也是遇到同样问题，按照文章解决方法，最后修改路径总算成功。其解决方法如下： 30 | 31 | 1. 找到 Jupyter 快捷方式，右键属性，并修改起始位置为你设置的路径，即刚才的「你想要设置的路径」 32 | 2. 在 Jupyter 的快捷方式属性中，有栏叫`目标`，将这栏最后的 `%USERPROFILE%` 去掉。 33 | 34 | 另外：如果需要检测 TensorFlow 是否安装成功，可以打开 Jupyter 新建 python 文件，输入 `import tensorflow as tf` 运行，看是否报错。 35 | 36 | 37 | 38 | ## TensorFlow安装 39 | 40 | 1、Windows安装TensorFlow 41 | 42 | - CPU版本，管理员方式打开命令提示符，输入命令：`pip install tensorflow` 43 | - GPU版本，管理员方式打开命令提示符，输入命令：`pip install tensorflow-gpu` 44 | 45 | 更新 TensorFlow： 46 | 47 | ``` python 48 | pip uninstall tensorflow 49 | pip install tensorflow 50 | ``` 51 | 52 | 注意，如果在安装过程中提示需要`MSVCP140.DLL`，则下载安装： 53 | 54 | ``` xml 55 | TensorFlow requires MSVCP140.DLL, which may not be installed on your system. If, 56 | when you import tensorflow as tf, you see an error about No module named 57 | "_pywrap_tensorflow" and/or DLL load failed, check whether MSVCP140.DLL is in 58 | your %PATH% and, if not, you should install the Visual C++ 2015 redistributable (x64 59 | version). 60 | ``` 61 | 62 | 2、Linux和MacOS安装Tensorflow 63 | 64 | - CPU版本 65 | - Python 2.7用户：`pip install tensorflow` 66 | - Python3.3+用户：`pip3 install tensorflow` 67 | - GPU版本 68 | - Python 2.7用户：`pip install tensorflow-gpu` 69 | - Python3.3+用户：`pip3 install tensorflow-gpu` 70 | 71 | 72 | 73 | 74 | 75 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week03-src/3-1非线性回归.py: -------------------------------------------------------------------------------- 1 | 2 | # coding: utf-8 3 | 4 | # In[1]: 5 | 6 | import tensorflow as tf 7 | import numpy as np 8 | import matplotlib.pyplot as plt 9 | 10 | 11 | # In[6]: 12 | 13 | #使用numpy生成200个随机点 14 | x_data = np.linspace(-0.5,0.5,200)[:,np.newaxis] 15 | noise = np.random.normal(0,0.02,x_data.shape) 16 | y_data = np.square(x_data) + noise 17 | 18 | #定义两个placeholder 19 | x = tf.placeholder(tf.float32,[None,1]) 20 | y = tf.placeholder(tf.float32,[None,1]) 21 | 22 | #定义神经网络中间层 23 | Weights_L1 = tf.Variable(tf.random_normal([1,10])) 24 | biases_L1 = tf.Variable(tf.zeros([1,10])) 25 | Wx_plus_b_L1 = tf.matmul(x,Weights_L1) + biases_L1 26 | L1 = tf.nn.tanh(Wx_plus_b_L1) 27 | 28 | #定义神经网络输出层 29 | Weights_L2 = tf.Variable(tf.random_normal([10,1])) 30 | biases_L2 = tf.Variable(tf.zeros([1,1])) 31 | Wx_plus_b_L2 = tf.matmul(L1,Weights_L2) + biases_L2 32 | prediction = tf.nn.tanh(Wx_plus_b_L2) 33 | 34 | #二次代价函数 35 | loss = tf.reduce_mean(tf.square(y-prediction)) 36 | #使用梯度下降法训练 37 | train_step = tf.train.GradientDescentOptimizer(0.1).minimize(loss) 38 | 39 | with tf.Session() as sess: 40 | #变量初始化 41 | sess.run(tf.global_variables_initializer()) 42 | for _ in range(2000): 43 | sess.run(train_step,feed_dict={x:x_data,y:y_data}) 44 | 45 | #获得预测值 46 | prediction_value = sess.run(prediction,feed_dict={x:x_data}) 47 | #画图 48 | plt.figure() 49 | plt.scatter(x_data,y_data) 50 | plt.plot(x_data,prediction_value,'r-',lw=5) 51 | plt.show() 52 | 53 | 54 | # In[ ]: 55 | 56 | 57 | 58 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week03-src/3-2MNIST数据集分类简单版本.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 2, 6 | "metadata": { 7 | "collapsed": false 8 | }, 9 | "outputs": [], 10 | "source": [ 11 | "import tensorflow as tf\n", 12 | "from tensorflow.examples.tutorials.mnist import input_data" 13 | ] 14 | }, 15 | { 16 | "cell_type": "code", 17 | "execution_count": 3, 18 | "metadata": { 19 | "collapsed": false 20 | }, 21 | "outputs": [ 22 | { 23 | "name": "stdout", 24 | "output_type": "stream", 25 | "text": [ 26 | "Extracting MNIST_data\\train-images-idx3-ubyte.gz\n", 27 | "Extracting MNIST_data\\train-labels-idx1-ubyte.gz\n", 28 | "Extracting MNIST_data\\t10k-images-idx3-ubyte.gz\n", 29 | "Extracting MNIST_data\\t10k-labels-idx1-ubyte.gz\n", 30 | "Iter 0,Testing Accuracy 0.8304\n", 31 | "Iter 1,Testing Accuracy 0.8708\n", 32 | "Iter 2,Testing Accuracy 0.8808\n", 33 | "Iter 3,Testing Accuracy 0.8878\n", 34 | "Iter 4,Testing Accuracy 0.8942\n", 35 | "Iter 5,Testing Accuracy 0.8967\n", 36 | "Iter 6,Testing Accuracy 0.8996\n", 37 | "Iter 7,Testing Accuracy 0.902\n", 38 | "Iter 8,Testing Accuracy 0.9037\n", 39 | "Iter 9,Testing Accuracy 0.9055\n", 40 | "Iter 10,Testing Accuracy 0.907\n", 41 | "Iter 11,Testing Accuracy 0.9074\n", 42 | "Iter 12,Testing Accuracy 0.9073\n", 43 | "Iter 13,Testing Accuracy 0.9084\n", 44 | "Iter 14,Testing Accuracy 0.9102\n", 45 | "Iter 15,Testing Accuracy 0.9112\n", 46 | "Iter 16,Testing Accuracy 0.9119\n", 47 | "Iter 17,Testing Accuracy 0.912\n", 48 | "Iter 18,Testing Accuracy 0.9131\n", 49 | "Iter 19,Testing Accuracy 0.9125\n", 50 | "Iter 20,Testing Accuracy 0.9134\n" 51 | ] 52 | } 53 | ], 54 | "source": [ 55 | "#载入数据集\n", 56 | "mnist = input_data.read_data_sets(\"MNIST_data\",one_hot=True)\n", 57 | "\n", 58 | "#每个批次的大小\n", 59 | "batch_size = 100\n", 60 | "#计算一共有多少个批次\n", 61 | "n_batch = mnist.train.num_examples // batch_size\n", 62 | "\n", 63 | "#定义两个placeholder\n", 64 | "x = tf.placeholder(tf.float32,[None,784])\n", 65 | "y = tf.placeholder(tf.float32,[None,10])\n", 66 | "\n", 67 | "#创建一个简单的神经网络\n", 68 | "W = tf.Variable(tf.zeros([784,10]))\n", 69 | "b = tf.Variable(tf.zeros([10]))\n", 70 | "prediction = tf.nn.softmax(tf.matmul(x,W)+b)\n", 71 | "\n", 72 | "#二次代价函数\n", 73 | "loss = tf.reduce_mean(tf.square(y-prediction))\n", 74 | "#使用梯度下降法\n", 75 | "train_step = tf.train.GradientDescentOptimizer(0.2).minimize(loss)\n", 76 | "\n", 77 | "#初始化变量\n", 78 | "init = tf.global_variables_initializer()\n", 79 | "\n", 80 | "#结果存放在一个布尔型列表中\n", 81 | "correct_prediction = tf.equal(tf.argmax(y,1),tf.argmax(prediction,1))#argmax返回一维张量中最大的值所在的位置\n", 82 | "#求准确率\n", 83 | "accuracy = tf.reduce_mean(tf.cast(correct_prediction,tf.float32))\n", 84 | "\n", 85 | "with tf.Session() as sess:\n", 86 | " sess.run(init)\n", 87 | " for epoch in range(21):\n", 88 | " for batch in range(n_batch):\n", 89 | " batch_xs,batch_ys = mnist.train.next_batch(batch_size)\n", 90 | " sess.run(train_step,feed_dict={x:batch_xs,y:batch_ys})\n", 91 | " \n", 92 | " acc = sess.run(accuracy,feed_dict={x:mnist.test.images,y:mnist.test.labels})\n", 93 | " print(\"Iter \" + str(epoch) + \",Testing Accuracy \" + str(acc))" 94 | ] 95 | }, 96 | { 97 | "cell_type": "code", 98 | "execution_count": null, 99 | "metadata": { 100 | "collapsed": true 101 | }, 102 | "outputs": [], 103 | "source": [] 104 | } 105 | ], 106 | "metadata": { 107 | "anaconda-cloud": {}, 108 | "kernelspec": { 109 | "display_name": "Python [default]", 110 | "language": "python", 111 | "name": "python3" 112 | }, 113 | "language_info": { 114 | "codemirror_mode": { 115 | "name": "ipython", 116 | "version": 3 117 | }, 118 | "file_extension": ".py", 119 | "mimetype": "text/x-python", 120 | "name": "python", 121 | "nbconvert_exporter": "python", 122 | "pygments_lexer": "ipython3", 123 | "version": "3.5.2" 124 | } 125 | }, 126 | "nbformat": 4, 127 | "nbformat_minor": 1 128 | } 129 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week03-src/3-2MNIST数据集分类简单版本.py: -------------------------------------------------------------------------------- 1 | 2 | # coding: utf-8 3 | 4 | # In[2]: 5 | 6 | import tensorflow as tf 7 | from tensorflow.examples.tutorials.mnist import input_data 8 | 9 | 10 | # In[3]: 11 | 12 | #载入数据集 13 | mnist = input_data.read_data_sets("MNIST_data",one_hot=True) 14 | 15 | #每个批次的大小 16 | batch_size = 100 17 | #计算一共有多少个批次 18 | n_batch = mnist.train.num_examples // batch_size 19 | 20 | #定义两个placeholder 21 | x = tf.placeholder(tf.float32,[None,784]) 22 | y = tf.placeholder(tf.float32,[None,10]) 23 | 24 | #创建一个简单的神经网络 25 | W = tf.Variable(tf.zeros([784,10])) 26 | b = tf.Variable(tf.zeros([10])) 27 | prediction = tf.nn.softmax(tf.matmul(x,W)+b) 28 | 29 | #二次代价函数 30 | loss = tf.reduce_mean(tf.square(y-prediction)) 31 | #使用梯度下降法 32 | train_step = tf.train.GradientDescentOptimizer(0.2).minimize(loss) 33 | 34 | #初始化变量 35 | init = tf.global_variables_initializer() 36 | 37 | #结果存放在一个布尔型列表中 38 | correct_prediction = tf.equal(tf.argmax(y,1),tf.argmax(prediction,1))#argmax返回一维张量中最大的值所在的位置 39 | #求准确率 40 | accuracy = tf.reduce_mean(tf.cast(correct_prediction,tf.float32)) 41 | 42 | with tf.Session() as sess: 43 | sess.run(init) 44 | for epoch in range(21): 45 | for batch in range(n_batch): 46 | batch_xs,batch_ys = mnist.train.next_batch(batch_size) 47 | sess.run(train_step,feed_dict={x:batch_xs,y:batch_ys}) 48 | 49 | acc = sess.run(accuracy,feed_dict={x:mnist.test.images,y:mnist.test.labels}) 50 | print("Iter " + str(epoch) + ",Testing Accuracy " + str(acc)) 51 | 52 | 53 | # In[ ]: 54 | 55 | 56 | 57 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week03-src/MNIST_data/t10k-images-idx3-ubyte.gz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week03-src/MNIST_data/t10k-images-idx3-ubyte.gz -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week03-src/MNIST_data/t10k-labels-idx1-ubyte.gz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week03-src/MNIST_data/t10k-labels-idx1-ubyte.gz -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week03-src/MNIST_data/train-images-idx3-ubyte.gz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week03-src/MNIST_data/train-images-idx3-ubyte.gz -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week03-src/MNIST_data/train-labels-idx1-ubyte.gz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week03-src/MNIST_data/train-labels-idx1-ubyte.gz -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week04-src/4-1交叉熵.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "metadata": { 7 | "collapsed": true 8 | }, 9 | "outputs": [], 10 | "source": [ 11 | "import tensorflow as tf\n", 12 | "from tensorflow.examples.tutorials.mnist import input_data\n", 13 | "\n", 14 | "#载入数据集\n", 15 | "mnist = input_data.read_data_sets(\"MNIST_data\",one_hot=True)\n", 16 | "\n", 17 | "#每个批次的大小\n", 18 | "batch_size = 100\n", 19 | "#计算一共有多少个批次\n", 20 | "n_batch = mnist.train.num_examples // batch_size\n", 21 | "\n", 22 | "#定义两个placeholder\n", 23 | "x = tf.placeholder(tf.float32,[None,784])\n", 24 | "y = tf.placeholder(tf.float32,[None,10])\n", 25 | "\n", 26 | "#创建一个简单的神经网络\n", 27 | "W = tf.Variable(tf.zeros([784,10]))\n", 28 | "b = tf.Variable(tf.zeros([10]))\n", 29 | "prediction = tf.nn.softmax(tf.matmul(x,W)+b)\n", 30 | "\n", 31 | "#二次代价函数\n", 32 | "# loss = tf.reduce_mean(tf.square(y-prediction))\n", 33 | "loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=y,logits=prediction))\n", 34 | "#使用梯度下降法\n", 35 | "train_step = tf.train.GradientDescentOptimizer(0.2).minimize(loss)\n", 36 | "\n", 37 | "#初始化变量\n", 38 | "init = tf.global_variables_initializer()\n", 39 | "\n", 40 | "#结果存放在一个布尔型列表中\n", 41 | "correct_prediction = tf.equal(tf.argmax(y,1),tf.argmax(prediction,1))#argmax返回一维张量中最大的值所在的位置\n", 42 | "#求准确率\n", 43 | "accuracy = tf.reduce_mean(tf.cast(correct_prediction,tf.float32))\n", 44 | "\n", 45 | "with tf.Session() as sess:\n", 46 | " sess.run(init)\n", 47 | " for epoch in range(21):\n", 48 | " for batch in range(n_batch):\n", 49 | " batch_xs,batch_ys = mnist.train.next_batch(batch_size)\n", 50 | " sess.run(train_step,feed_dict={x:batch_xs,y:batch_ys})\n", 51 | " \n", 52 | " acc = sess.run(accuracy,feed_dict={x:mnist.test.images,y:mnist.test.labels})\n", 53 | " print(\"Iter \" + str(epoch) + \",Testing Accuracy \" + str(acc))" 54 | ] 55 | } 56 | ], 57 | "metadata": { 58 | "anaconda-cloud": {}, 59 | "kernelspec": { 60 | "display_name": "Python [default]", 61 | "language": "python", 62 | "name": "python3" 63 | }, 64 | "language_info": { 65 | "codemirror_mode": { 66 | "name": "ipython", 67 | "version": 3 68 | }, 69 | "file_extension": ".py", 70 | "mimetype": "text/x-python", 71 | "name": "python", 72 | "nbconvert_exporter": "python", 73 | "pygments_lexer": "ipython3", 74 | "version": "3.5.2" 75 | } 76 | }, 77 | "nbformat": 4, 78 | "nbformat_minor": 1 79 | } 80 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week04-src/4-1交叉熵.py: -------------------------------------------------------------------------------- 1 | 2 | # coding: utf-8 3 | 4 | # In[ ]: 5 | 6 | import tensorflow as tf 7 | from tensorflow.examples.tutorials.mnist import input_data 8 | 9 | #载入数据集 10 | mnist = input_data.read_data_sets("MNIST_data",one_hot=True) 11 | 12 | #每个批次的大小 13 | batch_size = 100 14 | #计算一共有多少个批次 15 | n_batch = mnist.train.num_examples // batch_size 16 | 17 | #定义两个placeholder 18 | x = tf.placeholder(tf.float32,[None,784]) 19 | y = tf.placeholder(tf.float32,[None,10]) 20 | 21 | #创建一个简单的神经网络 22 | W = tf.Variable(tf.zeros([784,10])) 23 | b = tf.Variable(tf.zeros([10])) 24 | prediction = tf.nn.softmax(tf.matmul(x,W)+b) 25 | 26 | #二次代价函数 27 | # loss = tf.reduce_mean(tf.square(y-prediction)) 28 | loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=y,logits=prediction)) 29 | #使用梯度下降法 30 | train_step = tf.train.GradientDescentOptimizer(0.2).minimize(loss) 31 | 32 | #初始化变量 33 | init = tf.global_variables_initializer() 34 | 35 | #结果存放在一个布尔型列表中 36 | correct_prediction = tf.equal(tf.argmax(y,1),tf.argmax(prediction,1))#argmax返回一维张量中最大的值所在的位置 37 | #求准确率 38 | accuracy = tf.reduce_mean(tf.cast(correct_prediction,tf.float32)) 39 | 40 | with tf.Session() as sess: 41 | sess.run(init) 42 | for epoch in range(21): 43 | for batch in range(n_batch): 44 | batch_xs,batch_ys = mnist.train.next_batch(batch_size) 45 | sess.run(train_step,feed_dict={x:batch_xs,y:batch_ys}) 46 | 47 | acc = sess.run(accuracy,feed_dict={x:mnist.test.images,y:mnist.test.labels}) 48 | print("Iter " + str(epoch) + ",Testing Accuracy " + str(acc)) 49 | 50 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week04-src/4-2Dropout.py: -------------------------------------------------------------------------------- 1 | 2 | # coding: utf-8 3 | 4 | # In[3]: 5 | 6 | import tensorflow as tf 7 | from tensorflow.examples.tutorials.mnist import input_data 8 | 9 | #载入数据集 10 | mnist = input_data.read_data_sets("MNIST_data",one_hot=True) 11 | 12 | #每个批次的大小 13 | batch_size = 100 14 | #计算一共有多少个批次 15 | n_batch = mnist.train.num_examples // batch_size 16 | 17 | #定义两个placeholder 18 | x = tf.placeholder(tf.float32,[None,784]) 19 | y = tf.placeholder(tf.float32,[None,10]) 20 | keep_prob=tf.placeholder(tf.float32) 21 | 22 | #创建一个简单的神经网络 23 | W1 = tf.Variable(tf.truncated_normal([784,2000],stddev=0.1)) 24 | b1 = tf.Variable(tf.zeros([2000])+0.1) 25 | L1 = tf.nn.tanh(tf.matmul(x,W1)+b1) 26 | L1_drop = tf.nn.dropout(L1,keep_prob) 27 | 28 | W2 = tf.Variable(tf.truncated_normal([2000,2000],stddev=0.1)) 29 | b2 = tf.Variable(tf.zeros([2000])+0.1) 30 | L2 = tf.nn.tanh(tf.matmul(L1_drop,W2)+b2) 31 | L2_drop = tf.nn.dropout(L2,keep_prob) 32 | 33 | W3 = tf.Variable(tf.truncated_normal([2000,1000],stddev=0.1)) 34 | b3 = tf.Variable(tf.zeros([1000])+0.1) 35 | L3 = tf.nn.tanh(tf.matmul(L2_drop,W3)+b3) 36 | L3_drop = tf.nn.dropout(L3,keep_prob) 37 | 38 | W4 = tf.Variable(tf.truncated_normal([1000,10],stddev=0.1)) 39 | b4 = tf.Variable(tf.zeros([10])+0.1) 40 | prediction = tf.nn.softmax(tf.matmul(L3_drop,W4)+b4) 41 | 42 | #二次代价函数 43 | # loss = tf.reduce_mean(tf.square(y-prediction)) 44 | loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=y,logits=prediction)) 45 | #使用梯度下降法 46 | train_step = tf.train.GradientDescentOptimizer(0.2).minimize(loss) 47 | 48 | #初始化变量 49 | init = tf.global_variables_initializer() 50 | 51 | #结果存放在一个布尔型列表中 52 | correct_prediction = tf.equal(tf.argmax(y,1),tf.argmax(prediction,1))#argmax返回一维张量中最大的值所在的位置 53 | #求准确率 54 | accuracy = tf.reduce_mean(tf.cast(correct_prediction,tf.float32)) 55 | 56 | with tf.Session() as sess: 57 | sess.run(init) 58 | for epoch in range(31): 59 | for batch in range(n_batch): 60 | batch_xs,batch_ys = mnist.train.next_batch(batch_size) 61 | sess.run(train_step,feed_dict={x:batch_xs,y:batch_ys,keep_prob:0.7}) 62 | 63 | test_acc = sess.run(accuracy,feed_dict={x:mnist.test.images,y:mnist.test.labels,keep_prob:1.0}) 64 | train_acc = sess.run(accuracy,feed_dict={x:mnist.train.images,y:mnist.train.labels,keep_prob:1.0}) 65 | print("Iter " + str(epoch) + ",Testing Accuracy " + str(test_acc) +",Training Accuracy " + str(train_acc)) 66 | 67 | 68 | # In[ ]: 69 | 70 | Iter 29,Testing Accuracy 0.9727,Training Accuracy 0.995655 71 | Iter 30,Testing Accuracy 0.9722,Training Accuracy 0.995782 72 | 73 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week04-src/4-3优化器.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "metadata": { 7 | "collapsed": true 8 | }, 9 | "outputs": [], 10 | "source": [ 11 | "import tensorflow as tf\n", 12 | "from tensorflow.examples.tutorials.mnist import input_data" 13 | ] 14 | }, 15 | { 16 | "cell_type": "code", 17 | "execution_count": 2, 18 | "metadata": { 19 | "collapsed": false 20 | }, 21 | "outputs": [ 22 | { 23 | "name": "stdout", 24 | "output_type": "stream", 25 | "text": [ 26 | "Extracting MNIST_data\\train-images-idx3-ubyte.gz\n", 27 | "Extracting MNIST_data\\train-labels-idx1-ubyte.gz\n", 28 | "Extracting MNIST_data\\t10k-images-idx3-ubyte.gz\n", 29 | "Extracting MNIST_data\\t10k-labels-idx1-ubyte.gz\n", 30 | "Iter 0,Testing Accuracy 0.9165\n", 31 | "Iter 1,Testing Accuracy 0.9266\n", 32 | "Iter 2,Testing Accuracy 0.9257\n", 33 | "Iter 3,Testing Accuracy 0.9304\n", 34 | "Iter 4,Testing Accuracy 0.9307\n", 35 | "Iter 5,Testing Accuracy 0.931\n", 36 | "Iter 6,Testing Accuracy 0.9301\n", 37 | "Iter 7,Testing Accuracy 0.932\n", 38 | "Iter 8,Testing Accuracy 0.9316\n", 39 | "Iter 9,Testing Accuracy 0.9294\n", 40 | "Iter 10,Testing Accuracy 0.9338\n", 41 | "Iter 11,Testing Accuracy 0.9313\n", 42 | "Iter 12,Testing Accuracy 0.9295\n", 43 | "Iter 13,Testing Accuracy 0.9306\n", 44 | "Iter 14,Testing Accuracy 0.9312\n", 45 | "Iter 15,Testing Accuracy 0.9297\n", 46 | "Iter 16,Testing Accuracy 0.9317\n", 47 | "Iter 17,Testing Accuracy 0.9317\n", 48 | "Iter 18,Testing Accuracy 0.932\n", 49 | "Iter 19,Testing Accuracy 0.9317\n", 50 | "Iter 20,Testing Accuracy 0.9316\n" 51 | ] 52 | } 53 | ], 54 | "source": [ 55 | "#载入数据集\n", 56 | "mnist = input_data.read_data_sets(\"MNIST_data\",one_hot=True)\n", 57 | "\n", 58 | "#每个批次的大小\n", 59 | "batch_size = 100\n", 60 | "#计算一共有多少个批次\n", 61 | "n_batch = mnist.train.num_examples // batch_size\n", 62 | "\n", 63 | "#定义两个placeholder\n", 64 | "x = tf.placeholder(tf.float32,[None,784])\n", 65 | "y = tf.placeholder(tf.float32,[None,10])\n", 66 | "\n", 67 | "#创建一个简单的神经网络\n", 68 | "W = tf.Variable(tf.zeros([784,10]))\n", 69 | "b = tf.Variable(tf.zeros([10]))\n", 70 | "prediction = tf.nn.softmax(tf.matmul(x,W)+b)\n", 71 | "\n", 72 | "#二次代价函数\n", 73 | "# loss = tf.reduce_mean(tf.square(y-prediction))\n", 74 | "loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=y,logits=prediction))\n", 75 | "#使用梯度下降法\n", 76 | "# train_step = tf.train.GradientDescentOptimizer(0.2).minimize(loss)\n", 77 | "train_step = tf.train.AdamOptimizer(1e-2).minimize(loss)\n", 78 | "\n", 79 | "#初始化变量\n", 80 | "init = tf.global_variables_initializer()\n", 81 | "\n", 82 | "#结果存放在一个布尔型列表中\n", 83 | "correct_prediction = tf.equal(tf.argmax(y,1),tf.argmax(prediction,1))#argmax返回一维张量中最大的值所在的位置\n", 84 | "#求准确率\n", 85 | "accuracy = tf.reduce_mean(tf.cast(correct_prediction,tf.float32))\n", 86 | "\n", 87 | "with tf.Session() as sess:\n", 88 | " sess.run(init)\n", 89 | " for epoch in range(21):\n", 90 | " for batch in range(n_batch):\n", 91 | " batch_xs,batch_ys = mnist.train.next_batch(batch_size)\n", 92 | " sess.run(train_step,feed_dict={x:batch_xs,y:batch_ys})\n", 93 | " \n", 94 | " acc = sess.run(accuracy,feed_dict={x:mnist.test.images,y:mnist.test.labels})\n", 95 | " print(\"Iter \" + str(epoch) + \",Testing Accuracy \" + str(acc))" 96 | ] 97 | }, 98 | { 99 | "cell_type": "code", 100 | "execution_count": null, 101 | "metadata": { 102 | "collapsed": true 103 | }, 104 | "outputs": [], 105 | "source": [] 106 | } 107 | ], 108 | "metadata": { 109 | "anaconda-cloud": {}, 110 | "kernelspec": { 111 | "display_name": "Python [default]", 112 | "language": "python", 113 | "name": "python3" 114 | }, 115 | "language_info": { 116 | "codemirror_mode": { 117 | "name": "ipython", 118 | "version": 3 119 | }, 120 | "file_extension": ".py", 121 | "mimetype": "text/x-python", 122 | "name": "python", 123 | "nbconvert_exporter": "python", 124 | "pygments_lexer": "ipython3", 125 | "version": "3.5.2" 126 | } 127 | }, 128 | "nbformat": 4, 129 | "nbformat_minor": 1 130 | } 131 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week04-src/4-3优化器.py: -------------------------------------------------------------------------------- 1 | 2 | # coding: utf-8 3 | 4 | # In[1]: 5 | 6 | import tensorflow as tf 7 | from tensorflow.examples.tutorials.mnist import input_data 8 | 9 | 10 | # In[2]: 11 | 12 | #载入数据集 13 | mnist = input_data.read_data_sets("MNIST_data",one_hot=True) 14 | 15 | #每个批次的大小 16 | batch_size = 100 17 | #计算一共有多少个批次 18 | n_batch = mnist.train.num_examples // batch_size 19 | 20 | #定义两个placeholder 21 | x = tf.placeholder(tf.float32,[None,784]) 22 | y = tf.placeholder(tf.float32,[None,10]) 23 | 24 | #创建一个简单的神经网络 25 | W = tf.Variable(tf.zeros([784,10])) 26 | b = tf.Variable(tf.zeros([10])) 27 | prediction = tf.nn.softmax(tf.matmul(x,W)+b) 28 | 29 | #二次代价函数 30 | # loss = tf.reduce_mean(tf.square(y-prediction)) 31 | loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=y,logits=prediction)) 32 | #使用梯度下降法 33 | # train_step = tf.train.GradientDescentOptimizer(0.2).minimize(loss) 34 | train_step = tf.train.AdamOptimizer(1e-2).minimize(loss) 35 | 36 | #初始化变量 37 | init = tf.global_variables_initializer() 38 | 39 | #结果存放在一个布尔型列表中 40 | correct_prediction = tf.equal(tf.argmax(y,1),tf.argmax(prediction,1))#argmax返回一维张量中最大的值所在的位置 41 | #求准确率 42 | accuracy = tf.reduce_mean(tf.cast(correct_prediction,tf.float32)) 43 | 44 | with tf.Session() as sess: 45 | sess.run(init) 46 | for epoch in range(21): 47 | for batch in range(n_batch): 48 | batch_xs,batch_ys = mnist.train.next_batch(batch_size) 49 | sess.run(train_step,feed_dict={x:batch_xs,y:batch_ys}) 50 | 51 | acc = sess.run(accuracy,feed_dict={x:mnist.test.images,y:mnist.test.labels}) 52 | print("Iter " + str(epoch) + ",Testing Accuracy " + str(acc)) 53 | 54 | 55 | # In[ ]: 56 | 57 | 58 | 59 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week05-src/5-1第四周作业.py: -------------------------------------------------------------------------------- 1 | 2 | # coding: utf-8 3 | 4 | # In[2]: 5 | 6 | import tensorflow as tf 7 | from tensorflow.examples.tutorials.mnist import input_data 8 | 9 | #载入数据集 10 | mnist = input_data.read_data_sets("MNIST_data",one_hot=True) 11 | 12 | #每个批次的大小 13 | batch_size = 100 14 | #计算一共有多少个批次 15 | n_batch = mnist.train.num_examples // batch_size 16 | 17 | #定义两个placeholder 18 | x = tf.placeholder(tf.float32,[None,784]) 19 | y = tf.placeholder(tf.float32,[None,10]) 20 | keep_prob=tf.placeholder(tf.float32) 21 | lr = tf.Variable(0.001, dtype=tf.float32) 22 | 23 | #创建一个简单的神经网络 24 | W1 = tf.Variable(tf.truncated_normal([784,500],stddev=0.1)) 25 | b1 = tf.Variable(tf.zeros([500])+0.1) 26 | L1 = tf.nn.tanh(tf.matmul(x,W1)+b1) 27 | L1_drop = tf.nn.dropout(L1,keep_prob) 28 | 29 | W2 = tf.Variable(tf.truncated_normal([500,300],stddev=0.1)) 30 | b2 = tf.Variable(tf.zeros([300])+0.1) 31 | L2 = tf.nn.tanh(tf.matmul(L1_drop,W2)+b2) 32 | L2_drop = tf.nn.dropout(L2,keep_prob) 33 | 34 | W3 = tf.Variable(tf.truncated_normal([300,10],stddev=0.1)) 35 | b3 = tf.Variable(tf.zeros([10])+0.1) 36 | prediction = tf.nn.softmax(tf.matmul(L2_drop,W3)+b3) 37 | 38 | #交叉熵代价函数 39 | loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=y,logits=prediction)) 40 | #训练 41 | train_step = tf.train.AdamOptimizer(lr).minimize(loss) 42 | 43 | #初始化变量 44 | init = tf.global_variables_initializer() 45 | 46 | #结果存放在一个布尔型列表中 47 | correct_prediction = tf.equal(tf.argmax(y,1),tf.argmax(prediction,1))#argmax返回一维张量中最大的值所在的位置 48 | #求准确率 49 | accuracy = tf.reduce_mean(tf.cast(correct_prediction,tf.float32)) 50 | 51 | with tf.Session() as sess: 52 | sess.run(init) 53 | for epoch in range(51): 54 | sess.run(tf.assign(lr, 0.001 * (0.95 ** epoch))) 55 | for batch in range(n_batch): 56 | batch_xs,batch_ys = mnist.train.next_batch(batch_size) 57 | sess.run(train_step,feed_dict={x:batch_xs,y:batch_ys,keep_prob:1.0}) 58 | 59 | learning_rate = sess.run(lr) 60 | acc = sess.run(accuracy,feed_dict={x:mnist.test.images,y:mnist.test.labels,keep_prob:1.0}) 61 | print ("Iter " + str(epoch) + ", Testing Accuracy= " + str(acc) + ", Learning Rate= " + str(learning_rate)) 62 | 63 | 64 | # In[ ]: 65 | 66 | 67 | 68 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week05-src/5-2tensorboard网络结构.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "metadata": { 7 | "collapsed": true 8 | }, 9 | "outputs": [], 10 | "source": [ 11 | "import tensorflow as tf\n", 12 | "from tensorflow.examples.tutorials.mnist import input_data" 13 | ] 14 | }, 15 | { 16 | "cell_type": "code", 17 | "execution_count": 2, 18 | "metadata": { 19 | "collapsed": false 20 | }, 21 | "outputs": [ 22 | { 23 | "name": "stdout", 24 | "output_type": "stream", 25 | "text": [ 26 | "Extracting MNIST_data\\train-images-idx3-ubyte.gz\n", 27 | "Extracting MNIST_data\\train-labels-idx1-ubyte.gz\n", 28 | "Extracting MNIST_data\\t10k-images-idx3-ubyte.gz\n", 29 | "Extracting MNIST_data\\t10k-labels-idx1-ubyte.gz\n", 30 | "Iter 0,Testing Accuracy 0.8234\n" 31 | ] 32 | } 33 | ], 34 | "source": [ 35 | "#载入数据集\n", 36 | "mnist = input_data.read_data_sets(\"MNIST_data\",one_hot=True)\n", 37 | "\n", 38 | "#每个批次的大小\n", 39 | "batch_size = 100\n", 40 | "#计算一共有多少个批次\n", 41 | "n_batch = mnist.train.num_examples // batch_size\n", 42 | "\n", 43 | "#命名空间\n", 44 | "with tf.name_scope('input'):\n", 45 | " #定义两个placeholder\n", 46 | " x = tf.placeholder(tf.float32,[None,784],name='x-input')\n", 47 | " y = tf.placeholder(tf.float32,[None,10],name='y-input')\n", 48 | "\n", 49 | " \n", 50 | "with tf.name_scope('layer'):\n", 51 | " #创建一个简单的神经网络\n", 52 | " with tf.name_scope('wights'):\n", 53 | " W = tf.Variable(tf.zeros([784,10]),name='W')\n", 54 | " with tf.name_scope('biases'): \n", 55 | " b = tf.Variable(tf.zeros([10]),name='b')\n", 56 | " with tf.name_scope('wx_plus_b'):\n", 57 | " wx_plus_b = tf.matmul(x,W) + b\n", 58 | " with tf.name_scope('softmax'):\n", 59 | " prediction = tf.nn.softmax(wx_plus_b)\n", 60 | "\n", 61 | "#二次代价函数\n", 62 | "# loss = tf.reduce_mean(tf.square(y-prediction))\n", 63 | "with tf.name_scope('loss'):\n", 64 | " loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=y,logits=prediction))\n", 65 | "with tf.name_scope('train'):\n", 66 | " #使用梯度下降法\n", 67 | " train_step = tf.train.GradientDescentOptimizer(0.2).minimize(loss)\n", 68 | "\n", 69 | "#初始化变量\n", 70 | "init = tf.global_variables_initializer()\n", 71 | "\n", 72 | "with tf.name_scope('accuracy'):\n", 73 | " with tf.name_scope('correct_prediction'):\n", 74 | " #结果存放在一个布尔型列表中\n", 75 | " correct_prediction = tf.equal(tf.argmax(y,1),tf.argmax(prediction,1))#argmax返回一维张量中最大的值所在的位置\n", 76 | " with tf.name_scope('accuracy'):\n", 77 | " #求准确率\n", 78 | " accuracy = tf.reduce_mean(tf.cast(correct_prediction,tf.float32))\n", 79 | "\n", 80 | "with tf.Session() as sess:\n", 81 | " sess.run(init)\n", 82 | " writer = tf.summary.FileWriter('logs/',sess.graph)\n", 83 | " for epoch in range(1):\n", 84 | " for batch in range(n_batch):\n", 85 | " batch_xs,batch_ys = mnist.train.next_batch(batch_size)\n", 86 | " sess.run(train_step,feed_dict={x:batch_xs,y:batch_ys})\n", 87 | " \n", 88 | " acc = sess.run(accuracy,feed_dict={x:mnist.test.images,y:mnist.test.labels})\n", 89 | " print(\"Iter \" + str(epoch) + \",Testing Accuracy \" + str(acc))" 90 | ] 91 | }, 92 | { 93 | "cell_type": "code", 94 | "execution_count": null, 95 | "metadata": { 96 | "collapsed": true 97 | }, 98 | "outputs": [], 99 | "source": [] 100 | } 101 | ], 102 | "metadata": { 103 | "kernelspec": { 104 | "display_name": "Python [default]", 105 | "language": "python", 106 | "name": "python3" 107 | }, 108 | "language_info": { 109 | "codemirror_mode": { 110 | "name": "ipython", 111 | "version": 3 112 | }, 113 | "file_extension": ".py", 114 | "mimetype": "text/x-python", 115 | "name": "python", 116 | "nbconvert_exporter": "python", 117 | "pygments_lexer": "ipython3", 118 | "version": "3.5.2" 119 | } 120 | }, 121 | "nbformat": 4, 122 | "nbformat_minor": 1 123 | } 124 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week05-src/5-2tensorboard网络结构.py: -------------------------------------------------------------------------------- 1 | 2 | # coding: utf-8 3 | 4 | # In[1]: 5 | 6 | import tensorflow as tf 7 | from tensorflow.examples.tutorials.mnist import input_data 8 | 9 | 10 | # In[2]: 11 | 12 | #载入数据集 13 | mnist = input_data.read_data_sets("MNIST_data",one_hot=True) 14 | 15 | #每个批次的大小 16 | batch_size = 100 17 | #计算一共有多少个批次 18 | n_batch = mnist.train.num_examples // batch_size 19 | 20 | #命名空间 21 | with tf.name_scope('input'): 22 | #定义两个placeholder 23 | x = tf.placeholder(tf.float32,[None,784],name='x-input') 24 | y = tf.placeholder(tf.float32,[None,10],name='y-input') 25 | 26 | 27 | with tf.name_scope('layer'): 28 | #创建一个简单的神经网络 29 | with tf.name_scope('wights'): 30 | W = tf.Variable(tf.zeros([784,10]),name='W') 31 | with tf.name_scope('biases'): 32 | b = tf.Variable(tf.zeros([10]),name='b') 33 | with tf.name_scope('wx_plus_b'): 34 | wx_plus_b = tf.matmul(x,W) + b 35 | with tf.name_scope('softmax'): 36 | prediction = tf.nn.softmax(wx_plus_b) 37 | 38 | #二次代价函数 39 | # loss = tf.reduce_mean(tf.square(y-prediction)) 40 | with tf.name_scope('loss'): 41 | loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=y,logits=prediction)) 42 | with tf.name_scope('train'): 43 | #使用梯度下降法 44 | train_step = tf.train.GradientDescentOptimizer(0.2).minimize(loss) 45 | 46 | #初始化变量 47 | init = tf.global_variables_initializer() 48 | 49 | with tf.name_scope('accuracy'): 50 | with tf.name_scope('correct_prediction'): 51 | #结果存放在一个布尔型列表中 52 | correct_prediction = tf.equal(tf.argmax(y,1),tf.argmax(prediction,1))#argmax返回一维张量中最大的值所在的位置 53 | with tf.name_scope('accuracy'): 54 | #求准确率 55 | accuracy = tf.reduce_mean(tf.cast(correct_prediction,tf.float32)) 56 | 57 | with tf.Session() as sess: 58 | sess.run(init) 59 | writer = tf.summary.FileWriter('logs/',sess.graph) 60 | for epoch in range(1): 61 | for batch in range(n_batch): 62 | batch_xs,batch_ys = mnist.train.next_batch(batch_size) 63 | sess.run(train_step,feed_dict={x:batch_xs,y:batch_ys}) 64 | 65 | acc = sess.run(accuracy,feed_dict={x:mnist.test.images,y:mnist.test.labels}) 66 | print("Iter " + str(epoch) + ",Testing Accuracy " + str(acc)) 67 | 68 | 69 | # In[ ]: 70 | 71 | 72 | 73 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week05-src/5-3tensorboard网络运行.py: -------------------------------------------------------------------------------- 1 | 2 | # coding: utf-8 3 | 4 | # In[1]: 5 | 6 | import tensorflow as tf 7 | from tensorflow.examples.tutorials.mnist import input_data 8 | 9 | 10 | # In[2]: 11 | 12 | #载入数据集 13 | mnist = input_data.read_data_sets("MNIST_data",one_hot=True) 14 | 15 | #每个批次的大小 16 | batch_size = 100 17 | #计算一共有多少个批次 18 | n_batch = mnist.train.num_examples // batch_size 19 | 20 | #参数概要 21 | def variable_summaries(var): 22 | with tf.name_scope('summaries'): 23 | mean = tf.reduce_mean(var) 24 | tf.summary.scalar('mean', mean)#平均值 25 | with tf.name_scope('stddev'): 26 | stddev = tf.sqrt(tf.reduce_mean(tf.square(var - mean))) 27 | tf.summary.scalar('stddev', stddev)#标准差 28 | tf.summary.scalar('max', tf.reduce_max(var))#最大值 29 | tf.summary.scalar('min', tf.reduce_min(var))#最小值 30 | tf.summary.histogram('histogram', var)#直方图 31 | 32 | #命名空间 33 | with tf.name_scope('input'): 34 | #定义两个placeholder 35 | x = tf.placeholder(tf.float32,[None,784],name='x-input') 36 | y = tf.placeholder(tf.float32,[None,10],name='y-input') 37 | 38 | with tf.name_scope('layer'): 39 | #创建一个简单的神经网络 40 | with tf.name_scope('wights'): 41 | W = tf.Variable(tf.zeros([784,10]),name='W') 42 | variable_summaries(W) 43 | with tf.name_scope('biases'): 44 | b = tf.Variable(tf.zeros([10]),name='b') 45 | variable_summaries(b) 46 | with tf.name_scope('wx_plus_b'): 47 | wx_plus_b = tf.matmul(x,W) + b 48 | with tf.name_scope('softmax'): 49 | prediction = tf.nn.softmax(wx_plus_b) 50 | 51 | #二次代价函数 52 | # loss = tf.reduce_mean(tf.square(y-prediction)) 53 | with tf.name_scope('loss'): 54 | loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=y,logits=prediction)) 55 | tf.summary.scalar('loss',loss) 56 | with tf.name_scope('train'): 57 | #使用梯度下降法 58 | train_step = tf.train.GradientDescentOptimizer(0.2).minimize(loss) 59 | 60 | #初始化变量 61 | init = tf.global_variables_initializer() 62 | 63 | with tf.name_scope('accuracy'): 64 | with tf.name_scope('correct_prediction'): 65 | #结果存放在一个布尔型列表中 66 | correct_prediction = tf.equal(tf.argmax(y,1),tf.argmax(prediction,1))#argmax返回一维张量中最大的值所在的位置 67 | with tf.name_scope('accuracy'): 68 | #求准确率 69 | accuracy = tf.reduce_mean(tf.cast(correct_prediction,tf.float32)) 70 | tf.summary.scalar('accuracy',accuracy) 71 | 72 | #合并所有的summary 73 | merged = tf.summary.merge_all() 74 | 75 | with tf.Session() as sess: 76 | sess.run(init) 77 | writer = tf.summary.FileWriter('logs/',sess.graph) 78 | for epoch in range(51): 79 | for batch in range(n_batch): 80 | batch_xs,batch_ys = mnist.train.next_batch(batch_size) 81 | summary,_ = sess.run([merged,train_step],feed_dict={x:batch_xs,y:batch_ys}) 82 | 83 | writer.add_summary(summary,epoch) 84 | acc = sess.run(accuracy,feed_dict={x:mnist.test.images,y:mnist.test.labels}) 85 | print("Iter " + str(epoch) + ",Testing Accuracy " + str(acc)) 86 | 87 | 88 | # In[ ]: 89 | 90 | # for i in range(2001): 91 | # #m每个批次100个样本 92 | # batch_xs,batch_ys = mnist.train.next_batch(100) 93 | # summary,_ = sess.run([merged,train_step],feed_dict={x:batch_xs,y:batch_ys}) 94 | # writer.add_summary(summary,i) 95 | # if i%500 == 0: 96 | # print(sess.run(accuracy,feed_dict={x:mnist.test.images,y:mnist.test.labels})) 97 | 98 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week05-src/5-4tensorboard可视化.py: -------------------------------------------------------------------------------- 1 | 2 | # coding: utf-8 3 | 4 | # In[1]: 5 | 6 | import tensorflow as tf 7 | from tensorflow.examples.tutorials.mnist import input_data 8 | from tensorflow.contrib.tensorboard.plugins import projector 9 | 10 | 11 | # In[2]: 12 | 13 | #载入数据集 14 | mnist = input_data.read_data_sets("MNIST_data/",one_hot=True) 15 | #运行次数 16 | max_steps = 1001 17 | #图片数量 18 | image_num = 3000 19 | #文件路径 20 | DIR = "D:/Tensorflow/" 21 | 22 | #定义会话 23 | sess = tf.Session() 24 | 25 | #载入图片 26 | embedding = tf.Variable(tf.stack(mnist.test.images[:image_num]), trainable=False, name='embedding') 27 | 28 | #参数概要 29 | def variable_summaries(var): 30 | with tf.name_scope('summaries'): 31 | mean = tf.reduce_mean(var) 32 | tf.summary.scalar('mean', mean)#平均值 33 | with tf.name_scope('stddev'): 34 | stddev = tf.sqrt(tf.reduce_mean(tf.square(var - mean))) 35 | tf.summary.scalar('stddev', stddev)#标准差 36 | tf.summary.scalar('max', tf.reduce_max(var))#最大值 37 | tf.summary.scalar('min', tf.reduce_min(var))#最小值 38 | tf.summary.histogram('histogram', var)#直方图 39 | 40 | #命名空间 41 | with tf.name_scope('input'): 42 | #这里的none表示第一个维度可以是任意的长度 43 | x = tf.placeholder(tf.float32,[None,784],name='x-input') 44 | #正确的标签 45 | y = tf.placeholder(tf.float32,[None,10],name='y-input') 46 | 47 | #显示图片 48 | with tf.name_scope('input_reshape'): 49 | image_shaped_input = tf.reshape(x, [-1, 28, 28, 1]) 50 | tf.summary.image('input', image_shaped_input, 10) 51 | 52 | with tf.name_scope('layer'): 53 | #创建一个简单神经网络 54 | with tf.name_scope('weights'): 55 | W = tf.Variable(tf.zeros([784,10]),name='W') 56 | variable_summaries(W) 57 | with tf.name_scope('biases'): 58 | b = tf.Variable(tf.zeros([10]),name='b') 59 | variable_summaries(b) 60 | with tf.name_scope('wx_plus_b'): 61 | wx_plus_b = tf.matmul(x,W) + b 62 | with tf.name_scope('softmax'): 63 | prediction = tf.nn.softmax(wx_plus_b) 64 | 65 | with tf.name_scope('loss'): 66 | #交叉熵代价函数 67 | loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=y,logits=prediction)) 68 | tf.summary.scalar('loss',loss) 69 | with tf.name_scope('train'): 70 | #使用梯度下降法 71 | train_step = tf.train.GradientDescentOptimizer(0.5).minimize(loss) 72 | 73 | #初始化变量 74 | sess.run(tf.global_variables_initializer()) 75 | 76 | with tf.name_scope('accuracy'): 77 | with tf.name_scope('correct_prediction'): 78 | #结果存放在一个布尔型列表中 79 | correct_prediction = tf.equal(tf.argmax(y,1),tf.argmax(prediction,1))#argmax返回一维张量中最大的值所在的位置 80 | with tf.name_scope('accuracy'): 81 | #求准确率 82 | accuracy = tf.reduce_mean(tf.cast(correct_prediction,tf.float32))#把correct_prediction变为float32类型 83 | tf.summary.scalar('accuracy',accuracy) 84 | 85 | #产生metadata文件 86 | if tf.gfile.Exists(DIR + 'projector/projector/metadata.tsv'): 87 | tf.gfile.DeleteRecursively(DIR + 'projector/projector/metadata.tsv') 88 | with open(DIR + 'projector/projector/metadata.tsv', 'w') as f: 89 | labels = sess.run(tf.argmax(mnist.test.labels[:],1)) 90 | for i in range(image_num): 91 | f.write(str(labels[i]) + '\n') 92 | 93 | #合并所有的summary 94 | merged = tf.summary.merge_all() 95 | 96 | 97 | projector_writer = tf.summary.FileWriter(DIR + 'projector/projector',sess.graph) 98 | saver = tf.train.Saver() 99 | config = projector.ProjectorConfig() 100 | embed = config.embeddings.add() 101 | embed.tensor_name = embedding.name 102 | embed.metadata_path = DIR + 'projector/projector/metadata.tsv' 103 | embed.sprite.image_path = DIR + 'projector/data/mnist_10k_sprite.png' 104 | embed.sprite.single_image_dim.extend([28,28]) 105 | projector.visualize_embeddings(projector_writer,config) 106 | 107 | for i in range(max_steps): 108 | #每个批次100个样本 109 | batch_xs,batch_ys = mnist.train.next_batch(100) 110 | run_options = tf.RunOptions(trace_level=tf.RunOptions.FULL_TRACE) 111 | run_metadata = tf.RunMetadata() 112 | summary,_ = sess.run([merged,train_step],feed_dict={x:batch_xs,y:batch_ys},options=run_options,run_metadata=run_metadata) 113 | projector_writer.add_run_metadata(run_metadata, 'step%03d' % i) 114 | projector_writer.add_summary(summary, i) 115 | 116 | if i%100 == 0: 117 | acc = sess.run(accuracy,feed_dict={x:mnist.test.images,y:mnist.test.labels}) 118 | print ("Iter " + str(i) + ", Testing Accuracy= " + str(acc)) 119 | 120 | saver.save(sess, DIR + 'projector/projector/a_model.ckpt', global_step=max_steps) 121 | projector_writer.close() 122 | sess.close() 123 | 124 | 125 | # In[ ]: 126 | 127 | 128 | 129 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week05-src/mnist_10k_sprite.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week05-src/mnist_10k_sprite.png -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week06-src/6-1卷积神经网络应用于MNIST数据集分类.py: -------------------------------------------------------------------------------- 1 | import tensorflow as tf 2 | from tensorflow.examples.tutorials.mnist import input_data 3 | 4 | mnist = input_data.read_data_sets('MNIST_data', one_hot=True) 5 | 6 | # 每个批次的大小 7 | batch_size = 100 8 | # 计算一共有多少个批次 9 | n_batch = mnist.train.num_examples // batch_size 10 | 11 | 12 | # 初始化权值 13 | def weight_variable(shape): 14 | initial = tf.truncated_normal(shape, stddev=0.1) # 生成一个截断的正态分布 15 | return tf.Variable(initial) 16 | 17 | 18 | # 初始化偏置 19 | def bias_variable(shape): 20 | initial = tf.constant(0.1, shape=shape) 21 | return tf.Variable(initial) 22 | 23 | 24 | # 卷积层 25 | def conv2d(x, W): 26 | # x input tensor of shape '[batch,in_height,in_width,in_channles]' 27 | # W filter / kernel tensor of shape [filter_height,filter_width,in_channels,out_channels] 28 | # `strides[0] = strides[3] = 1`. strides[1]代表x方向的步长，strides[2]代表y方向的步长 29 | # padding: A `string` from: `"SAME", "VALID"` 30 | return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME') # 2d的意思是二维的卷积操作 31 | 32 | 33 | # 池化层 34 | def max_pool_2x2(x): 35 | # ksize [1,x,y,1] 36 | return tf.nn.max_pool(x, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME') 37 | 38 | 39 | # 定义两个placeholder 40 | x = tf.placeholder(tf.float32, [None, 784]) # 28*28 41 | y = tf.placeholder(tf.float32, [None, 10]) 42 | 43 | # 改变x的格式转为4D的向量[batch, in_height, in_width, in_channels]` 44 | x_image = tf.reshape(x, [-1, 28, 28, 1]) 45 | 46 | # 初始化第一个卷积层的权值和偏置 47 | W_conv1 = weight_variable([5, 5, 1, 32]) # 5*5的采样窗口，32个卷积核从1个平面抽取特征 48 | b_conv1 = bias_variable([32]) # 每一个卷积核一个偏置值 49 | 50 | # 把x_image和权值向量进行卷积，再加上偏置值，然后应用于relu激活函数 51 | h_conv1 = tf.nn.relu(conv2d(x_image, W_conv1) + b_conv1) 52 | h_pool1 = max_pool_2x2(h_conv1) # 进行max-pooling 53 | 54 | # 初始化第二个卷积层的权值和偏置 55 | W_conv2 = weight_variable([5, 5, 32, 64]) # 5*5的采样窗口，64个卷积核从32个平面抽取特征 56 | b_conv2 = bias_variable([64]) # 每一个卷积核一个偏置值 57 | 58 | # 把h_pool1和权值向量进行卷积，再加上偏置值，然后应用于relu激活函数 59 | h_conv2 = tf.nn.relu(conv2d(h_pool1, W_conv2) + b_conv2) 60 | h_pool2 = max_pool_2x2(h_conv2) # 进行max-pooling 61 | 62 | # 28*28的图片第一次卷积后还是28*28（数组变小了，但是图像大小不变），第一次池化后变为14*14 63 | # 第二次卷积后为14*14（卷积不会改变平面的大小），第二次池化后变为了7*7 64 | # 进过上面操作后得到64张7*7的平面 65 | 66 | # 初始化第一个全连接层的权值 67 | W_fc1 = weight_variable([7 * 7 * 64, 1024]) # 上一层有7*7*64个神经元，全连接层有1024个神经元 68 | b_fc1 = bias_variable([1024]) # 1024个节点 69 | 70 | # 把池化层2的输出扁平化为1维 71 | h_pool2_flat = tf.reshape(h_pool2, [-1, 7 * 7 * 64]) 72 | # 求第一个全连接层的输出 73 | h_fc1 = tf.nn.relu(tf.matmul(h_pool2_flat, W_fc1) + b_fc1) 74 | 75 | # keep_prob用来表示神经元的输出概率 76 | keep_prob = tf.placeholder(tf.float32) 77 | h_fc1_drop = tf.nn.dropout(h_fc1, keep_prob) 78 | 79 | # 初始化第二个全连接层 80 | W_fc2 = weight_variable([1024, 10]) 81 | b_fc2 = bias_variable([10]) 82 | 83 | # 计算输出 84 | prediction = tf.nn.softmax(tf.matmul(h_fc1_drop, W_fc2) + b_fc2) 85 | 86 | # 交叉熵代价函数 87 | cross_entropy = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=y, logits=prediction)) 88 | 89 | # 使用AdamOptimizer进行优化 90 | train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy) 91 | 92 | # 结果存放在一个布尔列表中 93 | correct_prediction = tf.equal(tf.argmax(prediction, 1), tf.argmax(y, 1)) # argmax返回一维张量中最大的值所在的位置 94 | 95 | # 求准确率 96 | accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32)) 97 | 98 | with tf.Session() as sess: 99 | sess.run(tf.global_variables_initializer()) 100 | for epoch in range(21): 101 | for batch in range(n_batch): 102 | batch_xs, batch_ys = mnist.train.next_batch(batch_size) 103 | sess.run(train_step, feed_dict={x: batch_xs, y: batch_ys, keep_prob: 0.7}) 104 | 105 | acc = sess.run(accuracy, feed_dict={x: mnist.test.images, y: mnist.test.labels, keep_prob: 1.0}) 106 | print("Iter " + str(epoch) + ", Testing Accuracy= " + str(acc)) -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week07-src/7-2递归神经网络RNN.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "metadata": { 7 | "collapsed": true 8 | }, 9 | "outputs": [], 10 | "source": [ 11 | "import tensorflow as tf\n", 12 | "from tensorflow.examples.tutorials.mnist import input_data" 13 | ] 14 | }, 15 | { 16 | "cell_type": "code", 17 | "execution_count": 2, 18 | "metadata": { 19 | "collapsed": false 20 | }, 21 | "outputs": [ 22 | { 23 | "name": "stdout", 24 | "output_type": "stream", 25 | "text": [ 26 | "Extracting MNIST_data/train-images-idx3-ubyte.gz\n", 27 | "Extracting MNIST_data/train-labels-idx1-ubyte.gz\n", 28 | "Extracting MNIST_data/t10k-images-idx3-ubyte.gz\n", 29 | "Extracting MNIST_data/t10k-labels-idx1-ubyte.gz\n", 30 | "Iter 0, Testing Accuracy= 0.7221\n", 31 | "Iter 1, Testing Accuracy= 0.8016\n", 32 | "Iter 2, Testing Accuracy= 0.8763\n", 33 | "Iter 3, Testing Accuracy= 0.9103\n", 34 | "Iter 4, Testing Accuracy= 0.9223\n", 35 | "Iter 5, Testing Accuracy= 0.9311\n" 36 | ] 37 | } 38 | ], 39 | "source": [ 40 | "#载入数据集\n", 41 | "mnist = input_data.read_data_sets(\"MNIST_data/\",one_hot=True)\n", 42 | "\n", 43 | "# 输入图片是28*28\n", 44 | "n_inputs = 28 #输入一行，一行有28个数据\n", 45 | "max_time = 28 #一共28行\n", 46 | "lstm_size = 100 #隐层单元\n", 47 | "n_classes = 10 # 10个分类\n", 48 | "batch_size = 50 #每批次50个样本\n", 49 | "n_batch = mnist.train.num_examples // batch_size #计算一共有多少个批次\n", 50 | "\n", 51 | "#这里的none表示第一个维度可以是任意的长度\n", 52 | "x = tf.placeholder(tf.float32,[None,784])\n", 53 | "#正确的标签\n", 54 | "y = tf.placeholder(tf.float32,[None,10])\n", 55 | "\n", 56 | "#初始化权值\n", 57 | "weights = tf.Variable(tf.truncated_normal([lstm_size, n_classes], stddev=0.1))\n", 58 | "#初始化偏置值\n", 59 | "biases = tf.Variable(tf.constant(0.1, shape=[n_classes]))\n", 60 | "\n", 61 | "\n", 62 | "#定义RNN网络\n", 63 | "def RNN(X,weights,biases):\n", 64 | " # inputs=[batch_size, max_time, n_inputs]\n", 65 | " inputs = tf.reshape(X,[-1,max_time,n_inputs])\n", 66 | " #定义LSTM基本CELL\n", 67 | " lstm_cell = tf.contrib.rnn.core_rnn_cell.BasicLSTMCell(lstm_size)\n", 68 | " # final_state[0]是cell state\n", 69 | " # final_state[1]是hidden_state\n", 70 | " outputs,final_state = tf.nn.dynamic_rnn(lstm_cell,inputs,dtype=tf.float32)\n", 71 | " results = tf.nn.softmax(tf.matmul(final_state[1],weights) + biases)\n", 72 | " return results\n", 73 | " \n", 74 | " \n", 75 | "#计算RNN的返回结果\n", 76 | "prediction= RNN(x, weights, biases) \n", 77 | "#损失函数\n", 78 | "cross_entropy = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=prediction,labels=y))\n", 79 | "#使用AdamOptimizer进行优化\n", 80 | "train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)\n", 81 | "#结果存放在一个布尔型列表中\n", 82 | "correct_prediction = tf.equal(tf.argmax(y,1),tf.argmax(prediction,1))#argmax返回一维张量中最大的值所在的位置\n", 83 | "#求准确率\n", 84 | "accuracy = tf.reduce_mean(tf.cast(correct_prediction,tf.float32))#把correct_prediction变为float32类型\n", 85 | "#初始化\n", 86 | "init = tf.global_variables_initializer()\n", 87 | "\n", 88 | "with tf.Session() as sess:\n", 89 | " sess.run(init)\n", 90 | " for epoch in range(6):\n", 91 | " for batch in range(n_batch):\n", 92 | " batch_xs,batch_ys = mnist.train.next_batch(batch_size)\n", 93 | " sess.run(train_step,feed_dict={x:batch_xs,y:batch_ys})\n", 94 | " \n", 95 | " acc = sess.run(accuracy,feed_dict={x:mnist.test.images,y:mnist.test.labels})\n", 96 | " print (\"Iter \" + str(epoch) + \", Testing Accuracy= \" + str(acc))" 97 | ] 98 | }, 99 | { 100 | "cell_type": "code", 101 | "execution_count": null, 102 | "metadata": { 103 | "collapsed": true 104 | }, 105 | "outputs": [], 106 | "source": [] 107 | } 108 | ], 109 | "metadata": { 110 | "anaconda-cloud": {}, 111 | "kernelspec": { 112 | "display_name": "Python [default]", 113 | "language": "python", 114 | "name": "python3" 115 | }, 116 | "language_info": { 117 | "codemirror_mode": { 118 | "name": "ipython", 119 | "version": 3 120 | }, 121 | "file_extension": ".py", 122 | "mimetype": "text/x-python", 123 | "name": "python", 124 | "nbconvert_exporter": "python", 125 | "pygments_lexer": "ipython3", 126 | "version": "3.5.2" 127 | } 128 | }, 129 | "nbformat": 4, 130 | "nbformat_minor": 1 131 | } 132 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week07-src/7-2递归神经网络RNN.py: -------------------------------------------------------------------------------- 1 | 2 | # coding: utf-8 3 | 4 | # In[1]: 5 | 6 | import tensorflow as tf 7 | from tensorflow.examples.tutorials.mnist import input_data 8 | 9 | 10 | # In[2]: 11 | 12 | #载入数据集 13 | mnist = input_data.read_data_sets("MNIST_data/",one_hot=True) 14 | 15 | # 输入图片是28*28 16 | n_inputs = 28 #输入一行，一行有28个数据 17 | max_time = 28 #一共28行 18 | lstm_size = 100 #隐层单元 19 | n_classes = 10 # 10个分类 20 | batch_size = 50 #每批次50个样本 21 | n_batch = mnist.train.num_examples // batch_size #计算一共有多少个批次 22 | 23 | #这里的none表示第一个维度可以是任意的长度 24 | x = tf.placeholder(tf.float32,[None,784]) 25 | #正确的标签 26 | y = tf.placeholder(tf.float32,[None,10]) 27 | 28 | #初始化权值 29 | weights = tf.Variable(tf.truncated_normal([lstm_size, n_classes], stddev=0.1)) 30 | #初始化偏置值 31 | biases = tf.Variable(tf.constant(0.1, shape=[n_classes])) 32 | 33 | 34 | #定义RNN网络 35 | def RNN(X,weights,biases): 36 | # inputs=[batch_size, max_time, n_inputs] 37 | inputs = tf.reshape(X,[-1,max_time,n_inputs]) 38 | #定义LSTM基本CELL 39 | lstm_cell = tf.contrib.rnn.core_rnn_cell.BasicLSTMCell(lstm_size) 40 | # final_state[0]是cell state 41 | # final_state[1]是hidden_state 42 | outputs,final_state = tf.nn.dynamic_rnn(lstm_cell,inputs,dtype=tf.float32) 43 | results = tf.nn.softmax(tf.matmul(final_state[1],weights) + biases) 44 | return results 45 | 46 | 47 | #计算RNN的返回结果 48 | prediction= RNN(x, weights, biases) 49 | #损失函数 50 | cross_entropy = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=prediction,labels=y)) 51 | #使用AdamOptimizer进行优化 52 | train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy) 53 | #结果存放在一个布尔型列表中 54 | correct_prediction = tf.equal(tf.argmax(y,1),tf.argmax(prediction,1))#argmax返回一维张量中最大的值所在的位置 55 | #求准确率 56 | accuracy = tf.reduce_mean(tf.cast(correct_prediction,tf.float32))#把correct_prediction变为float32类型 57 | #初始化 58 | init = tf.global_variables_initializer() 59 | 60 | with tf.Session() as sess: 61 | sess.run(init) 62 | for epoch in range(6): 63 | for batch in range(n_batch): 64 | batch_xs,batch_ys = mnist.train.next_batch(batch_size) 65 | sess.run(train_step,feed_dict={x:batch_xs,y:batch_ys}) 66 | 67 | acc = sess.run(accuracy,feed_dict={x:mnist.test.images,y:mnist.test.labels}) 68 | print ("Iter " + str(epoch) + ", Testing Accuracy= " + str(acc)) 69 | 70 | 71 | # In[ ]: 72 | 73 | 74 | 75 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week08-src/8-1saver_save.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "metadata": { 7 | "collapsed": true 8 | }, 9 | "outputs": [], 10 | "source": [ 11 | "import tensorflow as tf\n", 12 | "from tensorflow.examples.tutorials.mnist import input_data" 13 | ] 14 | }, 15 | { 16 | "cell_type": "code", 17 | "execution_count": 2, 18 | "metadata": { 19 | "collapsed": false 20 | }, 21 | "outputs": [ 22 | { 23 | "name": "stdout", 24 | "output_type": "stream", 25 | "text": [ 26 | "Extracting MNIST_data\\train-images-idx3-ubyte.gz\n", 27 | "Extracting MNIST_data\\train-labels-idx1-ubyte.gz\n", 28 | "Extracting MNIST_data\\t10k-images-idx3-ubyte.gz\n", 29 | "Extracting MNIST_data\\t10k-labels-idx1-ubyte.gz\n", 30 | "Iter 0,Testing Accuracy 0.8234\n", 31 | "Iter 1,Testing Accuracy 0.878\n", 32 | "Iter 2,Testing Accuracy 0.9003\n", 33 | "Iter 3,Testing Accuracy 0.905\n", 34 | "Iter 4,Testing Accuracy 0.9085\n", 35 | "Iter 5,Testing Accuracy 0.9109\n", 36 | "Iter 6,Testing Accuracy 0.912\n", 37 | "Iter 7,Testing Accuracy 0.9134\n", 38 | "Iter 8,Testing Accuracy 0.9152\n", 39 | "Iter 9,Testing Accuracy 0.9162\n", 40 | "Iter 10,Testing Accuracy 0.9169\n" 41 | ] 42 | } 43 | ], 44 | "source": [ 45 | "#载入数据集\n", 46 | "mnist = input_data.read_data_sets(\"MNIST_data\",one_hot=True)\n", 47 | "\n", 48 | "#每个批次100张照片\n", 49 | "batch_size = 100\n", 50 | "#计算一共有多少个批次\n", 51 | "n_batch = mnist.train.num_examples // batch_size\n", 52 | "\n", 53 | "#定义两个placeholder\n", 54 | "x = tf.placeholder(tf.float32,[None,784])\n", 55 | "y = tf.placeholder(tf.float32,[None,10])\n", 56 | "\n", 57 | "#创建一个简单的神经网络，输入层784个神经元，输出层10个神经元\n", 58 | "W = tf.Variable(tf.zeros([784,10]))\n", 59 | "b = tf.Variable(tf.zeros([10]))\n", 60 | "prediction = tf.nn.softmax(tf.matmul(x,W)+b)\n", 61 | "\n", 62 | "#二次代价函数\n", 63 | "# loss = tf.reduce_mean(tf.square(y-prediction))\n", 64 | "loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=y,logits=prediction))\n", 65 | "#使用梯度下降法\n", 66 | "train_step = tf.train.GradientDescentOptimizer(0.2).minimize(loss)\n", 67 | "\n", 68 | "#初始化变量\n", 69 | "init = tf.global_variables_initializer()\n", 70 | "\n", 71 | "#结果存放在一个布尔型列表中\n", 72 | "correct_prediction = tf.equal(tf.argmax(y,1),tf.argmax(prediction,1))#argmax返回一维张量中最大的值所在的位置\n", 73 | "#求准确率\n", 74 | "accuracy = tf.reduce_mean(tf.cast(correct_prediction,tf.float32))\n", 75 | "\n", 76 | "saver = tf.train.Saver()\n", 77 | "\n", 78 | "with tf.Session() as sess:\n", 79 | " sess.run(init)\n", 80 | " for epoch in range(11):\n", 81 | " for batch in range(n_batch):\n", 82 | " batch_xs,batch_ys = mnist.train.next_batch(batch_size)\n", 83 | " sess.run(train_step,feed_dict={x:batch_xs,y:batch_ys})\n", 84 | " \n", 85 | " acc = sess.run(accuracy,feed_dict={x:mnist.test.images,y:mnist.test.labels})\n", 86 | " print(\"Iter \" + str(epoch) + \",Testing Accuracy \" + str(acc))\n", 87 | " #保存模型\n", 88 | " saver.save(sess,'net/my_net.ckpt')" 89 | ] 90 | }, 91 | { 92 | "cell_type": "code", 93 | "execution_count": null, 94 | "metadata": { 95 | "collapsed": true 96 | }, 97 | "outputs": [], 98 | "source": [] 99 | } 100 | ], 101 | "metadata": { 102 | "anaconda-cloud": {}, 103 | "kernelspec": { 104 | "display_name": "Python [default]", 105 | "language": "python", 106 | "name": "python3" 107 | }, 108 | "language_info": { 109 | "codemirror_mode": { 110 | "name": "ipython", 111 | "version": 3 112 | }, 113 | "file_extension": ".py", 114 | "mimetype": "text/x-python", 115 | "name": "python", 116 | "nbconvert_exporter": "python", 117 | "pygments_lexer": "ipython3", 118 | "version": "3.5.2" 119 | } 120 | }, 121 | "nbformat": 4, 122 | "nbformat_minor": 1 123 | } 124 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week08-src/8-1saver_save.py: -------------------------------------------------------------------------------- 1 | 2 | # coding: utf-8 3 | 4 | # In[1]: 5 | 6 | import tensorflow as tf 7 | from tensorflow.examples.tutorials.mnist import input_data 8 | 9 | 10 | # In[2]: 11 | 12 | #载入数据集 13 | mnist = input_data.read_data_sets("MNIST_data",one_hot=True) 14 | 15 | #每个批次100张照片 16 | batch_size = 100 17 | #计算一共有多少个批次 18 | n_batch = mnist.train.num_examples // batch_size 19 | 20 | #定义两个placeholder 21 | x = tf.placeholder(tf.float32,[None,784]) 22 | y = tf.placeholder(tf.float32,[None,10]) 23 | 24 | #创建一个简单的神经网络，输入层784个神经元，输出层10个神经元 25 | W = tf.Variable(tf.zeros([784,10])) 26 | b = tf.Variable(tf.zeros([10])) 27 | prediction = tf.nn.softmax(tf.matmul(x,W)+b) 28 | 29 | #二次代价函数 30 | # loss = tf.reduce_mean(tf.square(y-prediction)) 31 | loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=y,logits=prediction)) 32 | #使用梯度下降法 33 | train_step = tf.train.GradientDescentOptimizer(0.2).minimize(loss) 34 | 35 | #初始化变量 36 | init = tf.global_variables_initializer() 37 | 38 | #结果存放在一个布尔型列表中 39 | correct_prediction = tf.equal(tf.argmax(y,1),tf.argmax(prediction,1))#argmax返回一维张量中最大的值所在的位置 40 | #求准确率 41 | accuracy = tf.reduce_mean(tf.cast(correct_prediction,tf.float32)) 42 | 43 | saver = tf.train.Saver() 44 | 45 | with tf.Session() as sess: 46 | sess.run(init) 47 | for epoch in range(11): 48 | for batch in range(n_batch): 49 | batch_xs,batch_ys = mnist.train.next_batch(batch_size) 50 | sess.run(train_step,feed_dict={x:batch_xs,y:batch_ys}) 51 | 52 | acc = sess.run(accuracy,feed_dict={x:mnist.test.images,y:mnist.test.labels}) 53 | print("Iter " + str(epoch) + ",Testing Accuracy " + str(acc)) 54 | #保存模型 55 | saver.save(sess,'net/my_net.ckpt') 56 | 57 | 58 | # In[ ]: 59 | 60 | 61 | 62 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week08-src/8-2saver_restore.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "metadata": { 7 | "collapsed": true 8 | }, 9 | "outputs": [], 10 | "source": [ 11 | "import tensorflow as tf\n", 12 | "from tensorflow.examples.tutorials.mnist import input_data" 13 | ] 14 | }, 15 | { 16 | "cell_type": "code", 17 | "execution_count": 2, 18 | "metadata": { 19 | "collapsed": false 20 | }, 21 | "outputs": [ 22 | { 23 | "name": "stdout", 24 | "output_type": "stream", 25 | "text": [ 26 | "Extracting MNIST_data\\train-images-idx3-ubyte.gz\n", 27 | "Extracting MNIST_data\\train-labels-idx1-ubyte.gz\n", 28 | "Extracting MNIST_data\\t10k-images-idx3-ubyte.gz\n", 29 | "Extracting MNIST_data\\t10k-labels-idx1-ubyte.gz\n", 30 | "0.098\n", 31 | "0.9169\n" 32 | ] 33 | } 34 | ], 35 | "source": [ 36 | "#载入数据集\n", 37 | "mnist = input_data.read_data_sets(\"MNIST_data\",one_hot=True)\n", 38 | "\n", 39 | "#每个批次100张照片\n", 40 | "batch_size = 100\n", 41 | "#计算一共有多少个批次\n", 42 | "n_batch = mnist.train.num_examples // batch_size\n", 43 | "\n", 44 | "#定义两个placeholder\n", 45 | "x = tf.placeholder(tf.float32,[None,784])\n", 46 | "y = tf.placeholder(tf.float32,[None,10])\n", 47 | "\n", 48 | "#创建一个简单的神经网络，输入层784个神经元，输出层10个神经元\n", 49 | "W = tf.Variable(tf.zeros([784,10]))\n", 50 | "b = tf.Variable(tf.zeros([10]))\n", 51 | "prediction = tf.nn.softmax(tf.matmul(x,W)+b)\n", 52 | "\n", 53 | "#二次代价函数\n", 54 | "# loss = tf.reduce_mean(tf.square(y-prediction))\n", 55 | "loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=y,logits=prediction))\n", 56 | "#使用梯度下降法\n", 57 | "train_step = tf.train.GradientDescentOptimizer(0.2).minimize(loss)\n", 58 | "\n", 59 | "#初始化变量\n", 60 | "init = tf.global_variables_initializer()\n", 61 | "\n", 62 | "#结果存放在一个布尔型列表中\n", 63 | "correct_prediction = tf.equal(tf.argmax(y,1),tf.argmax(prediction,1))#argmax返回一维张量中最大的值所在的位置\n", 64 | "#求准确率\n", 65 | "accuracy = tf.reduce_mean(tf.cast(correct_prediction,tf.float32))\n", 66 | "\n", 67 | "saver = tf.train.Saver()\n", 68 | "\n", 69 | "with tf.Session() as sess:\n", 70 | " sess.run(init)\n", 71 | " print(sess.run(accuracy,feed_dict={x:mnist.test.images,y:mnist.test.labels}))\n", 72 | " saver.restore(sess,'net/my_net.ckpt')\n", 73 | " print(sess.run(accuracy,feed_dict={x:mnist.test.images,y:mnist.test.labels}))" 74 | ] 75 | }, 76 | { 77 | "cell_type": "code", 78 | "execution_count": null, 79 | "metadata": { 80 | "collapsed": true 81 | }, 82 | "outputs": [], 83 | "source": [] 84 | } 85 | ], 86 | "metadata": { 87 | "kernelspec": { 88 | "display_name": "Python [default]", 89 | "language": "python", 90 | "name": "python3" 91 | }, 92 | "language_info": { 93 | "codemirror_mode": { 94 | "name": "ipython", 95 | "version": 3 96 | }, 97 | "file_extension": ".py", 98 | "mimetype": "text/x-python", 99 | "name": "python", 100 | "nbconvert_exporter": "python", 101 | "pygments_lexer": "ipython3", 102 | "version": "3.5.2" 103 | } 104 | }, 105 | "nbformat": 4, 106 | "nbformat_minor": 1 107 | } 108 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week08-src/8-2saver_restore.py: -------------------------------------------------------------------------------- 1 | 2 | # coding: utf-8 3 | 4 | # In[1]: 5 | 6 | import tensorflow as tf 7 | from tensorflow.examples.tutorials.mnist import input_data 8 | 9 | 10 | # In[2]: 11 | 12 | #载入数据集 13 | mnist = input_data.read_data_sets("MNIST_data",one_hot=True) 14 | 15 | #每个批次100张照片 16 | batch_size = 100 17 | #计算一共有多少个批次 18 | n_batch = mnist.train.num_examples // batch_size 19 | 20 | #定义两个placeholder 21 | x = tf.placeholder(tf.float32,[None,784]) 22 | y = tf.placeholder(tf.float32,[None,10]) 23 | 24 | #创建一个简单的神经网络，输入层784个神经元，输出层10个神经元 25 | W = tf.Variable(tf.zeros([784,10])) 26 | b = tf.Variable(tf.zeros([10])) 27 | prediction = tf.nn.softmax(tf.matmul(x,W)+b) 28 | 29 | #二次代价函数 30 | # loss = tf.reduce_mean(tf.square(y-prediction)) 31 | loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=y,logits=prediction)) 32 | #使用梯度下降法 33 | train_step = tf.train.GradientDescentOptimizer(0.2).minimize(loss) 34 | 35 | #初始化变量 36 | init = tf.global_variables_initializer() 37 | 38 | #结果存放在一个布尔型列表中 39 | correct_prediction = tf.equal(tf.argmax(y,1),tf.argmax(prediction,1))#argmax返回一维张量中最大的值所在的位置 40 | #求准确率 41 | accuracy = tf.reduce_mean(tf.cast(correct_prediction,tf.float32)) 42 | 43 | saver = tf.train.Saver() 44 | 45 | with tf.Session() as sess: 46 | sess.run(init) 47 | print(sess.run(accuracy,feed_dict={x:mnist.test.images,y:mnist.test.labels})) 48 | saver.restore(sess,'net/my_net.ckpt') 49 | print(sess.run(accuracy,feed_dict={x:mnist.test.images,y:mnist.test.labels})) 50 | 51 | 52 | # In[ ]: 53 | 54 | 55 | 56 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week08-src/8-3下载google图像识别网络inception-v3并查看结构.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "metadata": { 7 | "collapsed": true 8 | }, 9 | "outputs": [], 10 | "source": [ 11 | "import tensorflow as tf\n", 12 | "import os\n", 13 | "import tarfile\n", 14 | "import requests" 15 | ] 16 | }, 17 | { 18 | "cell_type": "code", 19 | "execution_count": 2, 20 | "metadata": { 21 | "collapsed": false 22 | }, 23 | "outputs": [ 24 | { 25 | "name": "stdout", 26 | "output_type": "stream", 27 | "text": [ 28 | "download: inception-2015-12-05.tgz\n", 29 | "finish: inception-2015-12-05.tgz\n" 30 | ] 31 | } 32 | ], 33 | "source": [ 34 | "#inception模型下载地址\n", 35 | "inception_pretrain_model_url = 'http://download.tensorflow.org/models/image/imagenet/inception-2015-12-05.tgz'\n", 36 | "\n", 37 | "#模型存放地址\n", 38 | "inception_pretrain_model_dir = \"inception_model\"\n", 39 | "if not os.path.exists(inception_pretrain_model_dir):\n", 40 | " os.makedirs(inception_pretrain_model_dir)\n", 41 | " \n", 42 | "#获取文件名，以及文件路径\n", 43 | "filename = inception_pretrain_model_url.split('/')[-1]\n", 44 | "filepath = os.path.join(inception_pretrain_model_dir, filename)\n", 45 | "\n", 46 | "#下载模型\n", 47 | "if not os.path.exists(filepath):\n", 48 | " print(\"download: \", filename)\n", 49 | " r = requests.get(inception_pretrain_model_url, stream=True)\n", 50 | " with open(filepath, 'wb') as f:\n", 51 | " for chunk in r.iter_content(chunk_size=1024):\n", 52 | " if chunk:\n", 53 | " f.write(chunk)\n", 54 | "print(\"finish: \", filename)\n", 55 | "#解压文件\n", 56 | "tarfile.open(filepath, 'r:gz').extractall(inception_pretrain_model_dir)\n", 57 | " \n", 58 | "#模型结构存放文件\n", 59 | "log_dir = 'inception_log'\n", 60 | "if not os.path.exists(log_dir):\n", 61 | " os.makedirs(log_dir)\n", 62 | "\n", 63 | "#classify_image_graph_def.pb为google训练好的模型\n", 64 | "inception_graph_def_file = os.path.join(inception_pretrain_model_dir, 'classify_image_graph_def.pb')\n", 65 | "with tf.Session() as sess:\n", 66 | " #创建一个图来存放google训练好的模型\n", 67 | " with tf.gfile.FastGFile(inception_graph_def_file, 'rb') as f:\n", 68 | " graph_def = tf.GraphDef()\n", 69 | " graph_def.ParseFromString(f.read())\n", 70 | " tf.import_graph_def(graph_def, name='')\n", 71 | " #保存图的结构\n", 72 | " writer = tf.summary.FileWriter(log_dir, sess.graph)\n", 73 | " writer.close()" 74 | ] 75 | }, 76 | { 77 | "cell_type": "code", 78 | "execution_count": null, 79 | "metadata": { 80 | "collapsed": false 81 | }, 82 | "outputs": [], 83 | "source": [] 84 | }, 85 | { 86 | "cell_type": "code", 87 | "execution_count": null, 88 | "metadata": { 89 | "collapsed": true 90 | }, 91 | "outputs": [], 92 | "source": [] 93 | } 94 | ], 95 | "metadata": { 96 | "anaconda-cloud": {}, 97 | "kernelspec": { 98 | "display_name": "Python [default]", 99 | "language": "python", 100 | "name": "python3" 101 | }, 102 | "language_info": { 103 | "codemirror_mode": { 104 | "name": "ipython", 105 | "version": 3 106 | }, 107 | "file_extension": ".py", 108 | "mimetype": "text/x-python", 109 | "name": "python", 110 | "nbconvert_exporter": "python", 111 | "pygments_lexer": "ipython3", 112 | "version": "3.5.2" 113 | } 114 | }, 115 | "nbformat": 4, 116 | "nbformat_minor": 1 117 | } 118 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week08-src/8-3下载google图像识别网络inception-v3并查看结构.py: -------------------------------------------------------------------------------- 1 | 2 | # coding: utf-8 3 | 4 | # In[1]: 5 | 6 | import tensorflow as tf 7 | import os 8 | import tarfile 9 | import requests 10 | 11 | 12 | # In[2]: 13 | 14 | #inception模型下载地址 15 | inception_pretrain_model_url = 'http://download.tensorflow.org/models/image/imagenet/inception-2015-12-05.tgz' 16 | 17 | #模型存放地址 18 | inception_pretrain_model_dir = "inception_model" 19 | if not os.path.exists(inception_pretrain_model_dir): 20 | os.makedirs(inception_pretrain_model_dir) 21 | 22 | #获取文件名，以及文件路径 23 | filename = inception_pretrain_model_url.split('/')[-1] 24 | filepath = os.path.join(inception_pretrain_model_dir, filename) 25 | 26 | #下载模型 27 | if not os.path.exists(filepath): 28 | print("download: ", filename) 29 | r = requests.get(inception_pretrain_model_url, stream=True) 30 | with open(filepath, 'wb') as f: 31 | for chunk in r.iter_content(chunk_size=1024): 32 | if chunk: 33 | f.write(chunk) 34 | print("finish: ", filename) 35 | #解压文件 36 | tarfile.open(filepath, 'r:gz').extractall(inception_pretrain_model_dir) 37 | 38 | #模型结构存放文件 39 | log_dir = 'inception_log' 40 | if not os.path.exists(log_dir): 41 | os.makedirs(log_dir) 42 | 43 | #classify_image_graph_def.pb为google训练好的模型 44 | inception_graph_def_file = os.path.join(inception_pretrain_model_dir, 'classify_image_graph_def.pb') 45 | with tf.Session() as sess: 46 | #创建一个图来存放google训练好的模型 47 | with tf.gfile.FastGFile(inception_graph_def_file, 'rb') as f: 48 | graph_def = tf.GraphDef() 49 | graph_def.ParseFromString(f.read()) 50 | tf.import_graph_def(graph_def, name='') 51 | #保存图的结构 52 | writer = tf.summary.FileWriter(log_dir, sess.graph) 53 | writer.close() 54 | 55 | 56 | # In[ ]: 57 | 58 | 59 | 60 | 61 | # In[ ]: 62 | 63 | 64 | 65 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week08-src/8-4使用inception-v3做各种图像的识别.py: -------------------------------------------------------------------------------- 1 | 2 | # coding: utf-8 3 | 4 | # In[1]: 5 | 6 | import tensorflow as tf 7 | import os 8 | import numpy as np 9 | import re 10 | from PIL import Image 11 | import matplotlib.pyplot as plt 12 | 13 | 14 | # In[2]: 15 | 16 | class NodeLookup(object): 17 | def __init__(self): 18 | label_lookup_path = 'inception_model/imagenet_2012_challenge_label_map_proto.pbtxt' 19 | uid_lookup_path = 'inception_model/imagenet_synset_to_human_label_map.txt' 20 | self.node_lookup = self.load(label_lookup_path, uid_lookup_path) 21 | 22 | def load(self, label_lookup_path, uid_lookup_path): 23 | # 加载分类字符串n********对应分类名称的文件 24 | proto_as_ascii_lines = tf.gfile.GFile(uid_lookup_path).readlines() 25 | uid_to_human = {} 26 | #一行一行读取数据 27 | for line in proto_as_ascii_lines : 28 | #去掉换行符 29 | line=line.strip('\n') 30 | #按照'\t'分割 31 | parsed_items = line.split('\t') 32 | #获取分类编号 33 | uid = parsed_items[0] 34 | #获取分类名称 35 | human_string = parsed_items[1] 36 | #保存编号字符串n********与分类名称映射关系 37 | uid_to_human[uid] = human_string 38 | 39 | # 加载分类字符串n********对应分类编号1-1000的文件 40 | proto_as_ascii = tf.gfile.GFile(label_lookup_path).readlines() 41 | node_id_to_uid = {} 42 | for line in proto_as_ascii: 43 | if line.startswith(' target_class:'): 44 | #获取分类编号1-1000 45 | target_class = int(line.split(': ')[1]) 46 | if line.startswith(' target_class_string:'): 47 | #获取编号字符串n******** 48 | target_class_string = line.split(': ')[1] 49 | #保存分类编号1-1000与编号字符串n********映射关系 50 | node_id_to_uid[target_class] = target_class_string[1:-2] 51 | 52 | #建立分类编号1-1000对应分类名称的映射关系 53 | node_id_to_name = {} 54 | for key, val in node_id_to_uid.items(): 55 | #获取分类名称 56 | name = uid_to_human[val] 57 | #建立分类编号1-1000到分类名称的映射关系 58 | node_id_to_name[key] = name 59 | return node_id_to_name 60 | 61 | #传入分类编号1-1000返回分类名称 62 | def id_to_string(self, node_id): 63 | if node_id not in self.node_lookup: 64 | return '' 65 | return self.node_lookup[node_id] 66 | 67 | 68 | #创建一个图来存放google训练好的模型 69 | with tf.gfile.FastGFile('inception_model/classify_image_graph_def.pb', 'rb') as f: 70 | graph_def = tf.GraphDef() 71 | graph_def.ParseFromString(f.read()) 72 | tf.import_graph_def(graph_def, name='') 73 | 74 | 75 | with tf.Session() as sess: 76 | softmax_tensor = sess.graph.get_tensor_by_name('softmax:0') 77 | #遍历目录 78 | for root,dirs,files in os.walk('images/'): 79 | for file in files: 80 | #载入图片 81 | image_data = tf.gfile.FastGFile(os.path.join(root,file), 'rb').read() 82 | predictions = sess.run(softmax_tensor,{'DecodeJpeg/contents:0': image_data})#图片格式是jpg格式 83 | predictions = np.squeeze(predictions)#把结果转为1维数据 84 | 85 | #打印图片路径及名称 86 | image_path = os.path.join(root,file) 87 | print(image_path) 88 | #显示图片 89 | img=Image.open(image_path) 90 | plt.imshow(img) 91 | plt.axis('off') 92 | plt.show() 93 | 94 | #排序 95 | top_k = predictions.argsort()[-5:][::-1] 96 | node_lookup = NodeLookup() 97 | for node_id in top_k: 98 | #获取分类名称 99 | human_string = node_lookup.id_to_string(node_id) 100 | #获取该分类的置信度 101 | score = predictions[node_id] 102 | print('%s (score = %.5f)' % (human_string, score)) 103 | print() 104 | 105 | 106 | # In[ ]: 107 | 108 | 109 | 110 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week08-src/images/lion.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week08-src/images/lion.jpg -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week08-src/images/panda.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week08-src/images/panda.jpg -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week08-src/images/rabbit.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week08-src/images/rabbit.jpg -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week09-src/retain/9-1测试训练好的模型.py: -------------------------------------------------------------------------------- 1 | 2 | # coding: utf-8 3 | 4 | # In[1]: 5 | 6 | import tensorflow as tf 7 | import os 8 | import numpy as np 9 | import re 10 | from PIL import Image 11 | import matplotlib.pyplot as plt 12 | 13 | 14 | # In[2]: 15 | 16 | lines = tf.gfile.GFile('retrain/output_labels.txt').readlines() 17 | uid_to_human = {} 18 | #一行一行读取数据 19 | for uid,line in enumerate(lines) : 20 | #去掉换行符 21 | line=line.strip('\n') 22 | uid_to_human[uid] = line 23 | 24 | def id_to_string(node_id): 25 | if node_id not in uid_to_human: 26 | return '' 27 | return uid_to_human[node_id] 28 | 29 | 30 | #创建一个图来存放google训练好的模型 31 | with tf.gfile.FastGFile('retrain/output_graph.pb', 'rb') as f: 32 | graph_def = tf.GraphDef() 33 | graph_def.ParseFromString(f.read()) 34 | tf.import_graph_def(graph_def, name='') 35 | 36 | 37 | with tf.Session() as sess: 38 | softmax_tensor = sess.graph.get_tensor_by_name('final_result:0') 39 | #遍历目录 40 | for root,dirs,files in os.walk('retrain/images/'): 41 | for file in files: 42 | #载入图片 43 | image_data = tf.gfile.FastGFile(os.path.join(root,file), 'rb').read() 44 | predictions = sess.run(softmax_tensor,{'DecodeJpeg/contents:0': image_data})#图片格式是jpg格式 45 | predictions = np.squeeze(predictions)#把结果转为1维数据 46 | 47 | #打印图片路径及名称 48 | image_path = os.path.join(root,file) 49 | print(image_path) 50 | #显示图片 51 | img=Image.open(image_path) 52 | plt.imshow(img) 53 | plt.axis('off') 54 | plt.show() 55 | 56 | #排序 57 | top_k = predictions.argsort()[::-1] 58 | print(top_k) 59 | for node_id in top_k: 60 | #获取分类名称 61 | human_string = id_to_string(node_id) 62 | #获取该分类的置信度 63 | score = predictions[node_id] 64 | print('%s (score = %.5f)' % (human_string, score)) 65 | print() 66 | 67 | 68 | # In[ ]: 69 | 70 | 71 | 72 | 73 | # In[ ]: 74 | 75 | 76 | 77 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week09-src/retain/retrain.bat: -------------------------------------------------------------------------------- 1 | python D:/Tensorflow/tensorflow-master/tensorflow/examples/image_retraining/retrain.py ^ 2 | --bottleneck_dir bottleneck ^ 3 | --how_many_training_steps 200 ^ 4 | --model_dir D:/Tensorflow/inception_model/ ^ 5 | --output_graph output_graph.pb ^ 6 | --output_labels output_labels.txt ^ 7 | --image_dir data/train/ 8 | pause 9 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week09-src/slim/datasets/dataset_factory.py: -------------------------------------------------------------------------------- 1 | # Copyright 2016 The TensorFlow Authors. All Rights Reserved. 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at 6 | # 7 | # http://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | # ============================================================================== 15 | """A factory-pattern class which returns classification image/label pairs.""" 16 | 17 | from __future__ import absolute_import 18 | from __future__ import division 19 | from __future__ import print_function 20 | 21 | from datasets import cifar10 22 | from datasets import flowers 23 | from datasets import imagenet 24 | from datasets import mnist 25 | from datasets import myimages 26 | 27 | datasets_map = { 28 | 'cifar10': cifar10, 29 | 'flowers': flowers, 30 | 'imagenet': imagenet, 31 | 'mnist': mnist, 32 | 'myimages':myimages, 33 | } 34 | 35 | 36 | def get_dataset(name, split_name, dataset_dir, file_pattern=None, reader=None): 37 | """Given a dataset name and a split_name returns a Dataset. 38 | 39 | Args: 40 | name: String, the name of the dataset. 41 | split_name: A train/test split name. 42 | dataset_dir: The directory where the dataset files are stored. 43 | file_pattern: The file pattern to use for matching the dataset source files. 44 | reader: The subclass of tf.ReaderBase. If left as `None`, then the default 45 | reader defined by each dataset is used. 46 | 47 | Returns: 48 | A `Dataset` class. 49 | 50 | Raises: 51 | ValueError: If the dataset `name` is unknown. 52 | """ 53 | if name not in datasets_map: 54 | raise ValueError('Name of dataset unknown %s' % name) 55 | return datasets_map[name].get_split( 56 | split_name, 57 | dataset_dir, 58 | file_pattern, 59 | reader) 60 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week09-src/slim/datasets/myimages.py: -------------------------------------------------------------------------------- 1 | # Copyright 2016 The TensorFlow Authors. All Rights Reserved. 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at 6 | # 7 | # http://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | # ============================================================================== 15 | """Provides data for the flowers dataset. 16 | 17 | The dataset scripts used to create the dataset can be found at: 18 | tensorflow/models/slim/datasets/download_and_convert_flowers.py 19 | """ 20 | 21 | from __future__ import absolute_import 22 | from __future__ import division 23 | from __future__ import print_function 24 | 25 | import os 26 | import tensorflow as tf 27 | 28 | from datasets import dataset_utils 29 | 30 | slim = tf.contrib.slim 31 | 32 | _FILE_PATTERN = 'image_%s_*.tfrecord' 33 | 34 | SPLITS_TO_SIZES = {'train': 1000, 'test': 500} 35 | 36 | _NUM_CLASSES = 5 37 | 38 | _ITEMS_TO_DESCRIPTIONS = { 39 | 'image': 'A color image of varying size.', 40 | 'label': 'A single integer between 0 and 4', 41 | } 42 | 43 | 44 | def get_split(split_name, dataset_dir, file_pattern=None, reader=None): 45 | """Gets a dataset tuple with instructions for reading flowers. 46 | 47 | Args: 48 | split_name: A train/validation split name. 49 | dataset_dir: The base directory of the dataset sources. 50 | file_pattern: The file pattern to use when matching the dataset sources. 51 | It is assumed that the pattern contains a '%s' string so that the split 52 | name can be inserted. 53 | reader: The TensorFlow reader type. 54 | 55 | Returns: 56 | A `Dataset` namedtuple. 57 | 58 | Raises: 59 | ValueError: if `split_name` is not a valid train/validation split. 60 | """ 61 | if split_name not in SPLITS_TO_SIZES: 62 | raise ValueError('split name %s was not recognized.' % split_name) 63 | 64 | if not file_pattern: 65 | file_pattern = _FILE_PATTERN 66 | file_pattern = os.path.join(dataset_dir, file_pattern % split_name) 67 | 68 | # Allowing None in the signature so that dataset_factory can use the default. 69 | if reader is None: 70 | reader = tf.TFRecordReader 71 | 72 | keys_to_features = { 73 | 'image/encoded': tf.FixedLenFeature((), tf.string, default_value=''), 74 | 'image/format': tf.FixedLenFeature((), tf.string, default_value='png'), 75 | 'image/class/label': tf.FixedLenFeature( 76 | [], tf.int64, default_value=tf.zeros([], dtype=tf.int64)), 77 | } 78 | 79 | items_to_handlers = { 80 | 'image': slim.tfexample_decoder.Image(), 81 | 'label': slim.tfexample_decoder.Tensor('image/class/label'), 82 | } 83 | 84 | decoder = slim.tfexample_decoder.TFExampleDecoder( 85 | keys_to_features, items_to_handlers) 86 | 87 | labels_to_names = None 88 | if dataset_utils.has_labels(dataset_dir): 89 | labels_to_names = dataset_utils.read_label_file(dataset_dir) 90 | 91 | return slim.dataset.Dataset( 92 | data_sources=file_pattern, 93 | reader=reader, 94 | decoder=decoder, 95 | num_samples=SPLITS_TO_SIZES[split_name], 96 | items_to_descriptions=_ITEMS_TO_DESCRIPTIONS, 97 | num_classes=_NUM_CLASSES, 98 | labels_to_names=labels_to_names) 99 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week09-src/slim/train.bat: -------------------------------------------------------------------------------- 1 | python D:/Tensorflow/slim/train_image_classifier.py ^ 2 | --train_dir=D:/Tensorflow/slim/model ^ 3 | --dataset_name=myimages ^ 4 | --dataset_split_name=train ^ 5 | --dataset_dir=D:/Tensorflow/slim/images ^ 6 | --batch_size=10 ^ 7 | --max_number_of_steps=10000 ^ 8 | --model_name=inception_v3 ^ 9 | pause -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/10-1验证码生成.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "metadata": { 7 | "collapsed": false 8 | }, 9 | "outputs": [ 10 | { 11 | "name": "stdout", 12 | "output_type": "stream", 13 | "text": [ 14 | ">> Creating image 10000/10000\n", 15 | "生成完毕\n" 16 | ] 17 | } 18 | ], 19 | "source": [ 20 | "# 验证码生成库\n", 21 | "from captcha.image import ImageCaptcha # pip install captcha\n", 22 | "import numpy as np\n", 23 | "from PIL import Image\n", 24 | "import random\n", 25 | "import sys\n", 26 | " \n", 27 | "number = ['0','1','2','3','4','5','6','7','8','9']\n", 28 | "# alphabet = ['a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z']\n", 29 | "# ALPHABET = ['A','B','C','D','E','F','G','H','I','J','K','L','M','N','O','P','Q','R','S','T','U','V','W','X','Y','Z']\n", 30 | "\n", 31 | "def random_captcha_text(char_set=number, captcha_size=4):\n", 32 | " # 验证码列表\n", 33 | " captcha_text = []\n", 34 | " for i in range(captcha_size):\n", 35 | " #随机选择\n", 36 | " c = random.choice(char_set)\n", 37 | " #加入验证码列表\n", 38 | " captcha_text.append(c)\n", 39 | " return captcha_text\n", 40 | " \n", 41 | "# 生成字符对应的验证码\n", 42 | "def gen_captcha_text_and_image():\n", 43 | " image = ImageCaptcha()\n", 44 | " #获得随机生成的验证码\n", 45 | " captcha_text = random_captcha_text()\n", 46 | " #把验证码列表转为字符串\n", 47 | " captcha_text = ''.join(captcha_text)\n", 48 | " #生成验证码\n", 49 | " captcha = image.generate(captcha_text)\n", 50 | " image.write(captcha_text, 'captcha/images/' + captcha_text + '.jpg') # 写到文件\n", 51 | "\n", 52 | "#数量少于10000，因为重名\n", 53 | "num = 10000\n", 54 | "if __name__ == '__main__':\n", 55 | " for i in range(num):\n", 56 | " gen_captcha_text_and_image()\n", 57 | " sys.stdout.write('\\r>> Creating image %d/%d' % (i+1, num))\n", 58 | " sys.stdout.flush()\n", 59 | " sys.stdout.write('\\n')\n", 60 | " sys.stdout.flush()\n", 61 | " \n", 62 | " print(\"生成完毕\")" 63 | ] 64 | }, 65 | { 66 | "cell_type": "code", 67 | "execution_count": null, 68 | "metadata": { 69 | "collapsed": true 70 | }, 71 | "outputs": [], 72 | "source": [] 73 | }, 74 | { 75 | "cell_type": "code", 76 | "execution_count": null, 77 | "metadata": { 78 | "collapsed": true 79 | }, 80 | "outputs": [], 81 | "source": [] 82 | } 83 | ], 84 | "metadata": { 85 | "anaconda-cloud": {}, 86 | "kernelspec": { 87 | "display_name": "Python [default]", 88 | "language": "python", 89 | "name": "python3" 90 | }, 91 | "language_info": { 92 | "codemirror_mode": { 93 | "name": "ipython", 94 | "version": 3 95 | }, 96 | "file_extension": ".py", 97 | "mimetype": "text/x-python", 98 | "name": "python", 99 | "nbconvert_exporter": "python", 100 | "pygments_lexer": "ipython3", 101 | "version": "3.5.2" 102 | } 103 | }, 104 | "nbformat": 4, 105 | "nbformat_minor": 2 106 | } 107 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/10-1验证码生成.py: -------------------------------------------------------------------------------- 1 | 2 | # coding: utf-8 3 | 4 | # In[1]: 5 | 6 | # 验证码生成库 7 | from captcha.image import ImageCaptcha # pip install captcha 8 | import numpy as np 9 | from PIL import Image 10 | import random 11 | import sys 12 | 13 | number = ['0','1','2','3','4','5','6','7','8','9'] 14 | # alphabet = ['a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z'] 15 | # ALPHABET = ['A','B','C','D','E','F','G','H','I','J','K','L','M','N','O','P','Q','R','S','T','U','V','W','X','Y','Z'] 16 | 17 | def random_captcha_text(char_set=number, captcha_size=4): 18 | # 验证码列表 19 | captcha_text = [] 20 | for i in range(captcha_size): 21 | #随机选择 22 | c = random.choice(char_set) 23 | #加入验证码列表 24 | captcha_text.append(c) 25 | return captcha_text 26 | 27 | # 生成字符对应的验证码 28 | def gen_captcha_text_and_image(): 29 | image = ImageCaptcha() 30 | #获得随机生成的验证码 31 | captcha_text = random_captcha_text() 32 | #把验证码列表转为字符串 33 | captcha_text = ''.join(captcha_text) 34 | #生成验证码 35 | captcha = image.generate(captcha_text) 36 | image.write(captcha_text, 'captcha/images/' + captcha_text + '.jpg') # 写到文件 37 | 38 | #数量少于10000，因为重名 39 | num = 10000 40 | if __name__ == '__main__': 41 | for i in range(num): 42 | gen_captcha_text_and_image() 43 | sys.stdout.write('\r>> Creating image %d/%d' % (i+1, num)) 44 | sys.stdout.flush() 45 | sys.stdout.write('\n') 46 | sys.stdout.flush() 47 | 48 | print("生成完毕") 49 | 50 | 51 | 52 | 53 | # 交替训练：不同任务有不同训练集， 54 | # 联合训练：不同任务有相关数据集。（验证码识别） 55 | 56 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/10-2生成tfrecord文件.py: -------------------------------------------------------------------------------- 1 | 2 | # coding: utf-8 3 | 4 | # In[1]: 5 | 6 | import tensorflow as tf 7 | import os 8 | import random 9 | import math 10 | import sys 11 | from PIL import Image 12 | import numpy as np 13 | 14 | 15 | # In[2]: 16 | 17 | #验证集数量 18 | _NUM_TEST = 500 19 | 20 | #随机种子 21 | _RANDOM_SEED = 0 22 | 23 | #数据集路径 24 | DATASET_DIR = "D:/Tensorflow/captcha/images/" 25 | 26 | #tfrecord文件存放路径 27 | TFRECORD_DIR = "D:/Tensorflow/captcha/" 28 | 29 | 30 | #判断tfrecord文件是否存在 31 | def _dataset_exists(dataset_dir): 32 | for split_name in ['train', 'test']: 33 | output_filename = os.path.join(dataset_dir,split_name + '.tfrecords') 34 | if not tf.gfile.Exists(output_filename): 35 | return False 36 | return True 37 | 38 | #获取所有验证码图片 39 | def _get_filenames_and_classes(dataset_dir): 40 | photo_filenames = [] 41 | for filename in os.listdir(dataset_dir): 42 | #获取文件路径 43 | path = os.path.join(dataset_dir, filename) 44 | photo_filenames.append(path) 45 | return photo_filenames 46 | 47 | def int64_feature(values): 48 | if not isinstance(values, (tuple, list)): 49 | values = [values] 50 | return tf.train.Feature(int64_list=tf.train.Int64List(value=values)) 51 | 52 | def bytes_feature(values): 53 | return tf.train.Feature(bytes_list=tf.train.BytesList(value=[values])) 54 | 55 | def image_to_tfexample(image_data, label0, label1, label2, label3): 56 | #Abstract base class for protocol messages. 57 | return tf.train.Example(features=tf.train.Features(feature={ 58 | 'image': bytes_feature(image_data), 59 | 'label0': int64_feature(label0), 60 | 'label1': int64_feature(label1), 61 | 'label2': int64_feature(label2), 62 | 'label3': int64_feature(label3), 63 | })) 64 | 65 | #把数据转为TFRecord格式 66 | def _convert_dataset(split_name, filenames, dataset_dir): 67 | assert split_name in ['train', 'test'] 68 | 69 | with tf.Session() as sess: 70 | #定义tfrecord文件的路径+名字 71 | output_filename = os.path.join(TFRECORD_DIR,split_name + '.tfrecords') 72 | with tf.python_io.TFRecordWriter(output_filename) as tfrecord_writer: 73 | for i,filename in enumerate(filenames): 74 | try: 75 | sys.stdout.write('\r>> Converting image %d/%d' % (i+1, len(filenames))) 76 | sys.stdout.flush() 77 | 78 | #读取图片 79 | image_data = Image.open(filename) 80 | #根据模型的结构resize 81 | image_data = image_data.resize((224, 224)) 82 | #灰度化 83 | image_data = np.array(image_data.convert('L')) 84 | #将图片转化为bytes 85 | image_data = image_data.tobytes() 86 | 87 | #获取label 88 | labels = filename.split('/')[-1][0:4] 89 | num_labels = [] 90 | for j in range(4): 91 | num_labels.append(int(labels[j])) 92 | 93 | #生成protocol数据类型 94 | example = image_to_tfexample(image_data, num_labels[0], num_labels[1], num_labels[2], num_labels[3]) 95 | tfrecord_writer.write(example.SerializeToString()) 96 | 97 | except IOError as e: 98 | print('Could not read:',filename) 99 | print('Error:',e) 100 | print('Skip it\n') 101 | sys.stdout.write('\n') 102 | sys.stdout.flush() 103 | 104 | #判断tfrecord文件是否存在 105 | if _dataset_exists(TFRECORD_DIR): 106 | print('tfcecord文件已存在') 107 | else: 108 | #获得所有图片 109 | photo_filenames = _get_filenames_and_classes(DATASET_DIR) 110 | 111 | #把数据切分为训练集和测试集,并打乱 112 | random.seed(_RANDOM_SEED) 113 | random.shuffle(photo_filenames) 114 | training_filenames = photo_filenames[_NUM_TEST:] 115 | testing_filenames = photo_filenames[:_NUM_TEST] 116 | 117 | #数据转换 118 | _convert_dataset('train', training_filenames, DATASET_DIR) 119 | _convert_dataset('test', testing_filenames, DATASET_DIR) 120 | print('生成tfcecord文件') 121 | 122 | 123 | # In[ ]: 124 | 125 | 126 | 127 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/__init__.py: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/__init__.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/__init__.pyc -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/__pycache__/__init__.cpython-35.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/__pycache__/__init__.cpython-35.pyc -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/__pycache__/alexnet.cpython-35.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/__pycache__/alexnet.cpython-35.pyc -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/__pycache__/cifarnet.cpython-35.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/__pycache__/cifarnet.cpython-35.pyc -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/__pycache__/inception.cpython-35.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/__pycache__/inception.cpython-35.pyc -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/__pycache__/inception_resnet_v2.cpython-35.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/__pycache__/inception_resnet_v2.cpython-35.pyc -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/__pycache__/inception_utils.cpython-35.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/__pycache__/inception_utils.cpython-35.pyc -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/__pycache__/inception_v1.cpython-35.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/__pycache__/inception_v1.cpython-35.pyc -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/__pycache__/inception_v2.cpython-35.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/__pycache__/inception_v2.cpython-35.pyc -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/__pycache__/inception_v3.cpython-35.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/__pycache__/inception_v3.cpython-35.pyc -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/__pycache__/inception_v4.cpython-35.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/__pycache__/inception_v4.cpython-35.pyc -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/__pycache__/lenet.cpython-35.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/__pycache__/lenet.cpython-35.pyc -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/__pycache__/nets_factory.cpython-35.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/__pycache__/nets_factory.cpython-35.pyc -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/__pycache__/overfeat.cpython-35.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/__pycache__/overfeat.cpython-35.pyc -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/__pycache__/resnet_utils.cpython-35.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/__pycache__/resnet_utils.cpython-35.pyc -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/__pycache__/resnet_v1.cpython-35.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/__pycache__/resnet_v1.cpython-35.pyc -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/__pycache__/resnet_v2.cpython-35.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/__pycache__/resnet_v2.cpython-35.pyc -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/__pycache__/vgg.cpython-35.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/__pycache__/vgg.cpython-35.pyc -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/alexnet.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/alexnet.pyc -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/cifarnet.py: -------------------------------------------------------------------------------- 1 | # Copyright 2016 The TensorFlow Authors. All Rights Reserved. 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at 6 | # 7 | # http://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | # ============================================================================== 15 | """Contains a variant of the CIFAR-10 model definition.""" 16 | 17 | from __future__ import absolute_import 18 | from __future__ import division 19 | from __future__ import print_function 20 | 21 | import tensorflow as tf 22 | 23 | slim = tf.contrib.slim 24 | 25 | trunc_normal = lambda stddev: tf.truncated_normal_initializer(stddev=stddev) 26 | 27 | 28 | def cifarnet(images, num_classes=10, is_training=False, 29 | dropout_keep_prob=0.5, 30 | prediction_fn=slim.softmax, 31 | scope='CifarNet'): 32 | """Creates a variant of the CifarNet model. 33 | 34 | Note that since the output is a set of 'logits', the values fall in the 35 | interval of (-infinity, infinity). Consequently, to convert the outputs to a 36 | probability distribution over the characters, one will need to convert them 37 | using the softmax function: 38 | 39 | logits = cifarnet.cifarnet(images, is_training=False) 40 | probabilities = tf.nn.softmax(logits) 41 | predictions = tf.argmax(logits, 1) 42 | 43 | Args: 44 | images: A batch of `Tensors` of size [batch_size, height, width, channels]. 45 | num_classes: the number of classes in the dataset. 46 | is_training: specifies whether or not we're currently training the model. 47 | This variable will determine the behaviour of the dropout layer. 48 | dropout_keep_prob: the percentage of activation values that are retained. 49 | prediction_fn: a function to get predictions out of logits. 50 | scope: Optional variable_scope. 51 | 52 | Returns: 53 | logits: the pre-softmax activations, a tensor of size 54 | [batch_size, `num_classes`] 55 | end_points: a dictionary from components of the network to the corresponding 56 | activation. 57 | """ 58 | end_points = {} 59 | 60 | with tf.variable_scope(scope, 'CifarNet', [images, num_classes]): 61 | net = slim.conv2d(images, 64, [5, 5], scope='conv1') 62 | end_points['conv1'] = net 63 | net = slim.max_pool2d(net, [2, 2], 2, scope='pool1') 64 | end_points['pool1'] = net 65 | net = tf.nn.lrn(net, 4, bias=1.0, alpha=0.001/9.0, beta=0.75, name='norm1') 66 | net = slim.conv2d(net, 64, [5, 5], scope='conv2') 67 | end_points['conv2'] = net 68 | net = tf.nn.lrn(net, 4, bias=1.0, alpha=0.001/9.0, beta=0.75, name='norm2') 69 | net = slim.max_pool2d(net, [2, 2], 2, scope='pool2') 70 | end_points['pool2'] = net 71 | net = slim.flatten(net) 72 | end_points['Flatten'] = net 73 | net = slim.fully_connected(net, 384, scope='fc3') 74 | end_points['fc3'] = net 75 | net = slim.dropout(net, dropout_keep_prob, is_training=is_training, 76 | scope='dropout3') 77 | net = slim.fully_connected(net, 192, scope='fc4') 78 | end_points['fc4'] = net 79 | logits = slim.fully_connected(net, num_classes, 80 | biases_initializer=tf.zeros_initializer(), 81 | weights_initializer=trunc_normal(1/192.0), 82 | weights_regularizer=None, 83 | activation_fn=None, 84 | scope='logits') 85 | 86 | end_points['Logits'] = logits 87 | end_points['Predictions'] = prediction_fn(logits, scope='Predictions') 88 | 89 | return logits, end_points 90 | cifarnet.default_image_size = 32 91 | 92 | 93 | def cifarnet_arg_scope(weight_decay=0.004): 94 | """Defines the default cifarnet argument scope. 95 | 96 | Args: 97 | weight_decay: The weight decay to use for regularizing the model. 98 | 99 | Returns: 100 | An `arg_scope` to use for the inception v3 model. 101 | """ 102 | with slim.arg_scope( 103 | [slim.conv2d], 104 | weights_initializer=tf.truncated_normal_initializer(stddev=5e-2), 105 | activation_fn=tf.nn.relu): 106 | with slim.arg_scope( 107 | [slim.fully_connected], 108 | biases_initializer=tf.constant_initializer(0.1), 109 | weights_initializer=trunc_normal(0.04), 110 | weights_regularizer=slim.l2_regularizer(weight_decay), 111 | activation_fn=tf.nn.relu) as sc: 112 | return sc 113 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/cifarnet.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/cifarnet.pyc -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/inception.py: -------------------------------------------------------------------------------- 1 | # Copyright 2016 The TensorFlow Authors. All Rights Reserved. 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at 6 | # 7 | # http://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | # ============================================================================== 15 | """Brings all inception models under one namespace.""" 16 | 17 | from __future__ import absolute_import 18 | from __future__ import division 19 | from __future__ import print_function 20 | 21 | # pylint: disable=unused-import 22 | from nets.inception_resnet_v2 import inception_resnet_v2 23 | from nets.inception_resnet_v2 import inception_resnet_v2_arg_scope 24 | from nets.inception_v1 import inception_v1 25 | from nets.inception_v1 import inception_v1_arg_scope 26 | from nets.inception_v1 import inception_v1_base 27 | from nets.inception_v2 import inception_v2 28 | from nets.inception_v2 import inception_v2_arg_scope 29 | from nets.inception_v2 import inception_v2_base 30 | from nets.inception_v3 import inception_v3 31 | from nets.inception_v3 import inception_v3_arg_scope 32 | from nets.inception_v3 import inception_v3_base 33 | from nets.inception_v4 import inception_v4 34 | from nets.inception_v4 import inception_v4_arg_scope 35 | from nets.inception_v4 import inception_v4_base 36 | # pylint: enable=unused-import 37 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/inception.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/inception.pyc -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/inception_resnet_v2.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/inception_resnet_v2.pyc -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/inception_utils.py: -------------------------------------------------------------------------------- 1 | # Copyright 2016 The TensorFlow Authors. All Rights Reserved. 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at 6 | # 7 | # http://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | # ============================================================================== 15 | """Contains common code shared by all inception models. 16 | 17 | Usage of arg scope: 18 | with slim.arg_scope(inception_arg_scope()): 19 | logits, end_points = inception.inception_v3(images, num_classes, 20 | is_training=is_training) 21 | 22 | """ 23 | from __future__ import absolute_import 24 | from __future__ import division 25 | from __future__ import print_function 26 | 27 | import tensorflow as tf 28 | 29 | slim = tf.contrib.slim 30 | 31 | 32 | def inception_arg_scope(weight_decay=0.00004, 33 | use_batch_norm=True, 34 | batch_norm_decay=0.9997, 35 | batch_norm_epsilon=0.001): 36 | """Defines the default arg scope for inception models. 37 | 38 | Args: 39 | weight_decay: The weight decay to use for regularizing the model. 40 | use_batch_norm: "If `True`, batch_norm is applied after each convolution. 41 | batch_norm_decay: Decay for batch norm moving average. 42 | batch_norm_epsilon: Small float added to variance to avoid dividing by zero 43 | in batch norm. 44 | 45 | Returns: 46 | An `arg_scope` to use for the inception models. 47 | """ 48 | batch_norm_params = { 49 | # Decay for the moving averages. 50 | 'decay': batch_norm_decay, 51 | # epsilon to prevent 0s in variance. 52 | 'epsilon': batch_norm_epsilon, 53 | # collection containing update_ops. 54 | 'updates_collections': tf.GraphKeys.UPDATE_OPS, 55 | } 56 | if use_batch_norm: 57 | normalizer_fn = slim.batch_norm 58 | normalizer_params = batch_norm_params 59 | else: 60 | normalizer_fn = None 61 | normalizer_params = {} 62 | # Set weight_decay for weights in Conv and FC layers. 63 | with slim.arg_scope([slim.conv2d, slim.fully_connected], 64 | weights_regularizer=slim.l2_regularizer(weight_decay)): 65 | with slim.arg_scope( 66 | [slim.conv2d], 67 | weights_initializer=slim.variance_scaling_initializer(), 68 | activation_fn=tf.nn.relu, 69 | normalizer_fn=normalizer_fn, 70 | normalizer_params=normalizer_params) as sc: 71 | return sc 72 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/inception_utils.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/inception_utils.pyc -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/inception_v1.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/inception_v1.pyc -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/inception_v2.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/inception_v2.pyc -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/inception_v3.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/inception_v3.pyc -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/inception_v4.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/inception_v4.pyc -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/lenet.py: -------------------------------------------------------------------------------- 1 | # Copyright 2016 The TensorFlow Authors. All Rights Reserved. 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at 6 | # 7 | # http://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | # ============================================================================== 15 | """Contains a variant of the LeNet model definition.""" 16 | 17 | from __future__ import absolute_import 18 | from __future__ import division 19 | from __future__ import print_function 20 | 21 | import tensorflow as tf 22 | 23 | slim = tf.contrib.slim 24 | 25 | 26 | def lenet(images, num_classes=10, is_training=False, 27 | dropout_keep_prob=0.5, 28 | prediction_fn=slim.softmax, 29 | scope='LeNet'): 30 | """Creates a variant of the LeNet model. 31 | 32 | Note that since the output is a set of 'logits', the values fall in the 33 | interval of (-infinity, infinity). Consequently, to convert the outputs to a 34 | probability distribution over the characters, one will need to convert them 35 | using the softmax function: 36 | 37 | logits = lenet.lenet(images, is_training=False) 38 | probabilities = tf.nn.softmax(logits) 39 | predictions = tf.argmax(logits, 1) 40 | 41 | Args: 42 | images: A batch of `Tensors` of size [batch_size, height, width, channels]. 43 | num_classes: the number of classes in the dataset. 44 | is_training: specifies whether or not we're currently training the model. 45 | This variable will determine the behaviour of the dropout layer. 46 | dropout_keep_prob: the percentage of activation values that are retained. 47 | prediction_fn: a function to get predictions out of logits. 48 | scope: Optional variable_scope. 49 | 50 | Returns: 51 | logits: the pre-softmax activations, a tensor of size 52 | [batch_size, `num_classes`] 53 | end_points: a dictionary from components of the network to the corresponding 54 | activation. 55 | """ 56 | end_points = {} 57 | 58 | with tf.variable_scope(scope, 'LeNet', [images, num_classes]): 59 | net = slim.conv2d(images, 32, [5, 5], scope='conv1') 60 | net = slim.max_pool2d(net, [2, 2], 2, scope='pool1') 61 | net = slim.conv2d(net, 64, [5, 5], scope='conv2') 62 | net = slim.max_pool2d(net, [2, 2], 2, scope='pool2') 63 | net = slim.flatten(net) 64 | end_points['Flatten'] = net 65 | 66 | net = slim.fully_connected(net, 1024, scope='fc3') 67 | net = slim.dropout(net, dropout_keep_prob, is_training=is_training, 68 | scope='dropout3') 69 | logits = slim.fully_connected(net, num_classes, activation_fn=None, 70 | scope='fc4') 71 | 72 | end_points['Logits'] = logits 73 | end_points['Predictions'] = prediction_fn(logits, scope='Predictions') 74 | 75 | return logits, end_points 76 | lenet.default_image_size = 28 77 | 78 | 79 | def lenet_arg_scope(weight_decay=0.0): 80 | """Defines the default lenet argument scope. 81 | 82 | Args: 83 | weight_decay: The weight decay to use for regularizing the model. 84 | 85 | Returns: 86 | An `arg_scope` to use for the inception v3 model. 87 | """ 88 | with slim.arg_scope( 89 | [slim.conv2d, slim.fully_connected], 90 | weights_regularizer=slim.l2_regularizer(weight_decay), 91 | weights_initializer=tf.truncated_normal_initializer(stddev=0.1), 92 | activation_fn=tf.nn.relu) as sc: 93 | return sc 94 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/lenet.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/lenet.pyc -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/nets_factory.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/nets_factory.pyc -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/nets_factory_test.py: -------------------------------------------------------------------------------- 1 | # Copyright 2016 Google Inc. All Rights Reserved. 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at 6 | # 7 | # http://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | # ============================================================================== 15 | 16 | """Tests for slim.inception.""" 17 | 18 | from __future__ import absolute_import 19 | from __future__ import division 20 | from __future__ import print_function 21 | 22 | import tensorflow as tf 23 | 24 | from nets import nets_factory 25 | 26 | slim = tf.contrib.slim 27 | 28 | 29 | class NetworksTest(tf.test.TestCase): 30 | 31 | def testGetNetworkFn(self): 32 | batch_size = 5 33 | num_classes = 1000 34 | for net in nets_factory.networks_map: 35 | with self.test_session(): 36 | net_fn = nets_factory.get_network_fn(net, num_classes) 37 | # Most networks use 224 as their default_image_size 38 | image_size = getattr(net_fn, 'default_image_size', 224) 39 | inputs = tf.random_uniform((batch_size, image_size, image_size, 3)) 40 | logits, end_points = net_fn(inputs) 41 | self.assertTrue(isinstance(logits, tf.Tensor)) 42 | self.assertTrue(isinstance(end_points, dict)) 43 | self.assertEqual(logits.get_shape().as_list()[0], batch_size) 44 | self.assertEqual(logits.get_shape().as_list()[-1], num_classes) 45 | 46 | def testGetNetworkFnArgScope(self): 47 | batch_size = 5 48 | num_classes = 10 49 | net = 'cifarnet' 50 | with self.test_session(use_gpu=True): 51 | net_fn = nets_factory.get_network_fn(net, num_classes) 52 | image_size = getattr(net_fn, 'default_image_size', 224) 53 | with slim.arg_scope([slim.model_variable, slim.variable], 54 | device='/CPU:0'): 55 | inputs = tf.random_uniform((batch_size, image_size, image_size, 3)) 56 | net_fn(inputs) 57 | weights = tf.get_collection(tf.GraphKeys.GLOBAL_VARIABLES, 'CifarNet/conv1')[0] 58 | self.assertDeviceEqual('/CPU:0', weights.device) 59 | 60 | if __name__ == '__main__': 61 | tf.test.main() 62 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/overfeat.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/overfeat.pyc -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/resnet_utils.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/resnet_utils.pyc -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/resnet_v1.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/resnet_v1.pyc -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/resnet_v2.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/resnet_v2.pyc -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/vgg.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week10-src/nets/vgg.pyc -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week11-src/cnn-text-classification-tf-master/.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week11-src/cnn-text-classification-tf-master/.DS_Store -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week11-src/cnn-text-classification-tf-master/.gitignore: -------------------------------------------------------------------------------- 1 | *.npy 2 | runs/ 3 | 4 | # Created by https://www.gitignore.io/api/python,ipythonnotebook 5 | 6 | ### Python ### 7 | # Byte-compiled / optimized / DLL files 8 | __pycache__/ 9 | *.py[cod] 10 | *$py.class 11 | 12 | # C extensions 13 | *.so 14 | 15 | # Distribution / packaging 16 | .Python 17 | env/ 18 | build/ 19 | develop-eggs/ 20 | dist/ 21 | downloads/ 22 | eggs/ 23 | .eggs/ 24 | lib/ 25 | lib64/ 26 | parts/ 27 | sdist/ 28 | var/ 29 | *.egg-info/ 30 | .installed.cfg 31 | *.egg 32 | 33 | # PyInstaller 34 | # Usually these files are written by a python script from a template 35 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 36 | *.manifest 37 | *.spec 38 | 39 | # Installer logs 40 | pip-log.txt 41 | pip-delete-this-directory.txt 42 | 43 | # Unit test / coverage reports 44 | htmlcov/ 45 | .tox/ 46 | .coverage 47 | .coverage.* 48 | .cache 49 | nosetests.xml 50 | coverage.xml 51 | *,cover 52 | 53 | # Translations 54 | *.mo 55 | *.pot 56 | 57 | # Django stuff: 58 | *.log 59 | 60 | # Sphinx documentation 61 | docs/_build/ 62 | 63 | # PyBuilder 64 | target/ 65 | 66 | 67 | ### IPythonNotebook ### 68 | # Temporary data 69 | .ipynb_checkpoints/ 70 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week11-src/cnn-text-classification-tf-master/README.md: -------------------------------------------------------------------------------- 1 | **[This code belongs to the "Implementing a CNN for Text Classification in Tensorflow" blog post.](http://www.wildml.com/2015/12/implementing-a-cnn-for-text-classification-in-tensorflow/)** 2 | 3 | It is slightly simplified implementation of Kim's [Convolutional Neural Networks for Sentence Classification](http://arxiv.org/abs/1408.5882) paper in Tensorflow. 4 | 5 | ## Requirements 6 | 7 | - Python 3 8 | - Tensorflow > 0.12 9 | - Numpy 10 | 11 | ## Training 12 | 13 | Print parameters: 14 | 15 | ```bash 16 | ./train.py --help 17 | ``` 18 | 19 | ``` 20 | optional arguments: 21 | -h, --help show this help message and exit 22 | --embedding_dim EMBEDDING_DIM 23 | Dimensionality of character embedding (default: 128) 24 | --filter_sizes FILTER_SIZES 25 | Comma-separated filter sizes (default: '3,4,5') 26 | --num_filters NUM_FILTERS 27 | Number of filters per filter size (default: 128) 28 | --l2_reg_lambda L2_REG_LAMBDA 29 | L2 regularizaion lambda (default: 0.0) 30 | --dropout_keep_prob DROPOUT_KEEP_PROB 31 | Dropout keep probability (default: 0.5) 32 | --batch_size BATCH_SIZE 33 | Batch Size (default: 64) 34 | --num_epochs NUM_EPOCHS 35 | Number of training epochs (default: 100) 36 | --evaluate_every EVALUATE_EVERY 37 | Evaluate model on dev set after this many steps 38 | (default: 100) 39 | --checkpoint_every CHECKPOINT_EVERY 40 | Save model after this many steps (default: 100) 41 | --allow_soft_placement ALLOW_SOFT_PLACEMENT 42 | Allow device soft device placement 43 | --noallow_soft_placement 44 | --log_device_placement LOG_DEVICE_PLACEMENT 45 | Log placement of ops on devices 46 | --nolog_device_placement 47 | 48 | ``` 49 | 50 | Train: 51 | 52 | ```bash 53 | ./train.py 54 | ``` 55 | 56 | ## Evaluating 57 | 58 | ```bash 59 | ./eval.py --eval_train --checkpoint_dir="./runs/1459637919/checkpoints/" 60 | ``` 61 | 62 | Replace the checkpoint dir with the output from the training. To use your own data, change the `eval.py` script to load your data. 63 | 64 | 65 | ## References 66 | 67 | - [Convolutional Neural Networks for Sentence Classification](http://arxiv.org/abs/1408.5882) 68 | - [A Sensitivity Analysis of (and Practitioners' Guide to) Convolutional Neural Networks for Sentence Classification](http://arxiv.org/abs/1510.03820) 69 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week11-src/cnn-text-classification-tf-master/data/.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week11-src/cnn-text-classification-tf-master/data/.DS_Store -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week11-src/cnn-text-classification-tf-master/data_helpers.py: -------------------------------------------------------------------------------- 1 | #coding:utf-8 2 | import numpy as np 3 | import re 4 | import itertools 5 | from collections import Counter 6 | 7 | 8 | def clean_str(string): 9 | """ 10 | Tokenization/string cleaning for all datasets except for SST. 11 | Original taken from https://github.com/yoonkim/CNN_sentence/blob/master/process_data.py 12 | """ 13 | # 不是特定字符都变成空格 14 | string = re.sub(r"[^A-Za-z0-9(),!?\'\`]", " ", string) 15 | # 加空格 16 | string = re.sub(r"\'s", " \'s", string) 17 | string = re.sub(r"\'ve", " \'ve", string) 18 | string = re.sub(r"n\'t", " n\'t", string) 19 | string = re.sub(r"\'re", " \'re", string) 20 | string = re.sub(r"\'d", " \'d", string) 21 | string = re.sub(r"\'ll", " \'ll", string) 22 | string = re.sub(r",", " , ", string) 23 | string = re.sub(r"!", " ! ", string) 24 | string = re.sub(r"\(", " \( ", string) 25 | string = re.sub(r"\)", " \) ", string) 26 | string = re.sub(r"\?", " \? ", string) 27 | # 匹配2个或多个空白字符变成一个" "空格 28 | string = re.sub(r"\s{2,}", " ", string) 29 | # 去掉句子首尾的空白符，再转小写 30 | return string.strip().lower() 31 | 32 | 33 | def load_data_and_labels(positive_data_file, negative_data_file): 34 | """ 35 | Loads MR polarity data from files, splits the data into words and generates labels. 36 | Returns split sentences and labels. 37 | """ 38 | # Load data from files 39 | positive_examples = list(open(positive_data_file, "r", encoding="utf-8").readlines()) 40 | positive_examples = [s.strip() for s in positive_examples] 41 | negative_examples = list(open(negative_data_file, "r", encoding="utf-8").readlines()) 42 | negative_examples = [s.strip() for s in negative_examples] 43 | # Split by words 44 | x_text = positive_examples + negative_examples 45 | x_text = [clean_str(sent) for sent in x_text] 46 | # Generate labels 47 | positive_labels = [[0, 1] for _ in positive_examples] 48 | negative_labels = [[1, 0] for _ in negative_examples] 49 | y = np.concatenate([positive_labels, negative_labels], 0) 50 | return [x_text, y] 51 | 52 | 53 | def batch_iter(data, batch_size, num_epochs, shuffle=True): 54 | """ 55 | Generates a batch iterator for a dataset. 56 | """ 57 | data = np.array(data) 58 | data_size = len(data) 59 | num_batches_per_epoch = int((len(data)-1)/batch_size) + 1 60 | print("num_batches_per_epoch:",num_batches_per_epoch) 61 | for epoch in range(num_epochs): 62 | # Shuffle the data at each epoch 63 | if shuffle: 64 | shuffle_indices = np.random.permutation(np.arange(data_size)) 65 | shuffled_data = data[shuffle_indices] 66 | else: 67 | shuffled_data = data 68 | for batch_num in range(num_batches_per_epoch): 69 | start_index = batch_num * batch_size 70 | end_index = min((batch_num + 1) * batch_size, data_size) 71 | yield shuffled_data[start_index:end_index] 72 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week11-src/cnn-text-classification-tf-master/eval.py: -------------------------------------------------------------------------------- 1 | #! /usr/bin/env python 2 | 3 | import tensorflow as tf 4 | import numpy as np 5 | import os 6 | import time 7 | import datetime 8 | import data_helpers 9 | from text_cnn import TextCNN 10 | from tensorflow.contrib import learn 11 | import csv 12 | 13 | # Parameters 14 | # ================================================== 15 | 16 | # Data Parameters 17 | tf.flags.DEFINE_string("positive_data_file", "./data/rt-polaritydata/rt-polarity.pos", "Data source for the positive data.") 18 | tf.flags.DEFINE_string("negative_data_file", "./data/rt-polaritydata/rt-polarity.neg", "Data source for the positive data.") 19 | 20 | # Eval Parameters 21 | tf.flags.DEFINE_integer("batch_size", 64, "Batch Size (default: 64)") 22 | tf.flags.DEFINE_string("checkpoint_dir", "", "Checkpoint directory from training run") 23 | tf.flags.DEFINE_boolean("eval_train", False, "Evaluate on all training data") 24 | 25 | # Misc Parameters 26 | tf.flags.DEFINE_boolean("allow_soft_placement", True, "Allow device soft device placement") 27 | tf.flags.DEFINE_boolean("log_device_placement", False, "Log placement of ops on devices") 28 | 29 | 30 | FLAGS = tf.flags.FLAGS 31 | FLAGS._parse_flags() 32 | print("\nParameters:") 33 | for attr, value in sorted(FLAGS.__flags.items()): 34 | print("{}={}".format(attr.upper(), value)) 35 | print("") 36 | 37 | # CHANGE THIS: Load data. Load your own data here 38 | if FLAGS.eval_train: 39 | x_raw, y_test = data_helpers.load_data_and_labels(FLAGS.positive_data_file, FLAGS.negative_data_file) 40 | y_test = np.argmax(y_test, axis=1) 41 | else: 42 | x_raw = ["a masterpiece four years in the making", "everything is off."] 43 | y_test = [1, 0] 44 | 45 | # Map data into vocabulary 46 | vocab_path = os.path.join(FLAGS.checkpoint_dir, "..", "vocab") 47 | vocab_processor = learn.preprocessing.VocabularyProcessor.restore(vocab_path) 48 | x_test = np.array(list(vocab_processor.transform(x_raw))) 49 | 50 | print("\nEvaluating...\n") 51 | 52 | # Evaluation 53 | # ================================================== 54 | checkpoint_file = tf.train.latest_checkpoint(FLAGS.checkpoint_dir) 55 | graph = tf.Graph() 56 | with graph.as_default(): 57 | session_conf = tf.ConfigProto( 58 | allow_soft_placement=FLAGS.allow_soft_placement, 59 | log_device_placement=FLAGS.log_device_placement) 60 | sess = tf.Session(config=session_conf) 61 | with sess.as_default(): 62 | # Load the saved meta graph and restore variables 63 | saver = tf.train.import_meta_graph("{}.meta".format(checkpoint_file)) 64 | saver.restore(sess, checkpoint_file) 65 | 66 | # Get the placeholders from the graph by name 67 | input_x = graph.get_operation_by_name("input_x").outputs[0] 68 | # input_y = graph.get_operation_by_name("input_y").outputs[0] 69 | dropout_keep_prob = graph.get_operation_by_name("dropout_keep_prob").outputs[0] 70 | 71 | # Tensors we want to evaluate 72 | predictions = graph.get_operation_by_name("output/predictions").outputs[0] 73 | 74 | # Generate batches for one epoch 75 | batches = data_helpers.batch_iter(list(x_test), FLAGS.batch_size, 1, shuffle=False) 76 | 77 | # Collect the predictions here 78 | all_predictions = [] 79 | 80 | for x_test_batch in batches: 81 | batch_predictions = sess.run(predictions, {input_x: x_test_batch, dropout_keep_prob: 1.0}) 82 | all_predictions = np.concatenate([all_predictions, batch_predictions]) 83 | 84 | # Print accuracy if y_test is defined 85 | if y_test is not None: 86 | correct_predictions = float(sum(all_predictions == y_test)) 87 | print("Total number of test examples: {}".format(len(y_test))) 88 | print("Accuracy: {:g}".format(correct_predictions/float(len(y_test)))) 89 | 90 | # Save the evaluation to a csv 91 | predictions_human_readable = np.column_stack((np.array(x_raw), all_predictions)) 92 | out_path = os.path.join(FLAGS.checkpoint_dir, "..", "prediction.csv") 93 | print("Saving evaluation to {0}".format(out_path)) 94 | with open(out_path, 'w') as f: 95 | csv.writer(f).writerows(predictions_human_readable) -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week11-src/cnn-text-classification-tf-master/text_cnn.py: -------------------------------------------------------------------------------- 1 | #coding:utf-8 2 | import tensorflow as tf 3 | import numpy as np 4 | import pickle 5 | 6 | 7 | class TextCNN(object): 8 | """ 9 | A CNN for text classification. 10 | Uses an embedding layer, followed by a convolutional, max-pooling and softmax layer. 11 | """ 12 | # sequence_length-最长词汇数 13 | # num_classes-分类数 14 | # vocab_size-总词汇数 15 | # embedding_size-词向量长度 16 | # filter_sizes-卷积核尺寸3，4，5 17 | # num_filters-卷积核数量 18 | # l2_reg_lambda-l2正则化系数 19 | def __init__( 20 | self, sequence_length, num_classes, vocab_size, 21 | embedding_size, filter_sizes, num_filters, l2_reg_lambda=0.0): 22 | 23 | # Placeholders for input, output and dropout 24 | self.input_x = tf.placeholder(tf.int32, [None, sequence_length], name="input_x") 25 | self.input_y = tf.placeholder(tf.float32, [None, num_classes], name="input_y") 26 | self.dropout_keep_prob = tf.placeholder(tf.float32, name="dropout_keep_prob") 27 | 28 | # Keeping track of l2 regularization loss (optional) 29 | l2_loss = tf.constant(0.0) 30 | 31 | # Embedding layer 32 | with tf.device('/cpu:0'), tf.name_scope("embedding"): 33 | self.W = tf.Variable( 34 | tf.random_uniform([vocab_size, embedding_size], -1.0, 1.0), 35 | name="W") 36 | # [batch_size, sequence_length, embedding_size] 37 | self.embedded_chars = tf.nn.embedding_lookup(self.W, self.input_x) 38 | # 添加一个维度，[batch_size, sequence_length, embedding_size, 1] 39 | self.embedded_chars_expanded = tf.expand_dims(self.embedded_chars, -1) 40 | 41 | # Create a convolution + maxpool layer for each filter size 42 | 43 | pooled_outputs = [] 44 | for i, filter_size in enumerate(filter_sizes): 45 | with tf.name_scope("conv-maxpool-%s" % filter_size): 46 | # Convolution Layer 47 | filter_shape = [filter_size, embedding_size, 1, num_filters] 48 | W = tf.Variable(tf.truncated_normal(filter_shape, stddev=0.1), name="W") 49 | b = tf.Variable(tf.constant(0.1, shape=[num_filters]), name="b") 50 | conv = tf.nn.conv2d( 51 | self.embedded_chars_expanded, 52 | W, 53 | strides=[1, 1, 1, 1], 54 | padding="VALID", 55 | name="conv") 56 | # Apply nonlinearity 57 | h = tf.nn.relu(tf.nn.bias_add(conv, b), name="relu") 58 | # Maxpooling over the outputs 59 | pooled = tf.nn.max_pool( 60 | h, 61 | ksize=[1, sequence_length - filter_size + 1, 1, 1], 62 | strides=[1, 1, 1, 1], 63 | padding='VALID', 64 | name="pool") 65 | pooled_outputs.append(pooled) 66 | 67 | # Combine all the pooled features 68 | num_filters_total = num_filters * len(filter_sizes) 69 | self.h_pool = tf.concat(pooled_outputs, 3) 70 | # 把池化层输出变成一维向量 71 | self.h_pool_flat = tf.reshape(self.h_pool, [-1, num_filters_total]) 72 | 73 | # Add dropout 74 | with tf.name_scope("dropout"): 75 | self.h_drop = tf.nn.dropout(self.h_pool_flat, self.dropout_keep_prob) 76 | 77 | # Final (unnormalized) scores and predictions 78 | with tf.name_scope("output"): 79 | W = tf.get_variable( 80 | "W", 81 | shape=[num_filters_total, num_classes], 82 | initializer=tf.contrib.layers.xavier_initializer()) 83 | b = tf.Variable(tf.constant(0.1, shape=[num_classes]), name="b") 84 | l2_loss += tf.nn.l2_loss(W) 85 | l2_loss += tf.nn.l2_loss(b) 86 | self.scores = tf.nn.softmax(tf.nn.xw_plus_b(self.h_drop, W, b, name="scores")) 87 | self.predictions = tf.argmax(self.scores, 1, name="predictions") 88 | 89 | # CalculateMean cross-entropy loss 90 | with tf.name_scope("loss"): 91 | losses = tf.nn.softmax_cross_entropy_with_logits(logits=self.scores, labels=self.input_y) 92 | self.loss = tf.reduce_mean(losses) + l2_reg_lambda * l2_loss 93 | 94 | # Accuracy 95 | with tf.name_scope("accuracy"): 96 | correct_predictions = tf.equal(self.predictions, tf.argmax(self.input_y, 1)) 97 | self.accuracy = tf.reduce_mean(tf.cast(correct_predictions, "float"), name="accuracy") 98 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/code/week12-src/text_cnn.py: -------------------------------------------------------------------------------- 1 | #coding:utf-8 2 | import tensorflow as tf 3 | import numpy as np 4 | import pickle 5 | 6 | 7 | class TextCNN(object): 8 | """ 9 | A CNN for text classification. 10 | Uses an embedding layer, followed by a convolutional, max-pooling and softmax layer. 11 | """ 12 | # sequence_length-最长词汇数 13 | # num_classes-分类数 14 | # vocab_size-总词汇数 15 | # embedding_size-词向量长度 16 | # filter_sizes-卷积核尺寸3，4，5 17 | # num_filters-卷积核数量 18 | # l2_reg_lambda-l2正则化系数 19 | def __init__( 20 | self, sequence_length, num_classes, vocab_size, 21 | embedding_size, filter_sizes, num_filters, l2_reg_lambda=0.0): 22 | 23 | # Placeholders for input, output and dropout 24 | self.input_x = tf.placeholder(tf.int32, [None, sequence_length], name="input_x") 25 | self.input_y = tf.placeholder(tf.float32, [None, num_classes], name="input_y") 26 | self.dropout_keep_prob = tf.placeholder(tf.float32, name="dropout_keep_prob") 27 | 28 | with open('embeddings.pickle','rb') as f: 29 | embeddings = pickle.load(f) 30 | 31 | # Keeping track of l2 regularization loss (optional) 32 | l2_loss = tf.constant(0.0) 33 | 34 | # Embedding layer 35 | with tf.device('/cpu:0'), tf.name_scope("embedding"): 36 | self.W = tf.Variable(embeddings, name='W') 37 | # self.W = tf.Variable(tf.random_uniform([vocab_size, embedding_size], -1.0, 1.0),name="W") 38 | # [batch_size, sequence_length, embedding_size] 39 | self.embedded_chars = tf.nn.embedding_lookup(self.W, self.input_x) 40 | # 添加一个维度，[batch_size, sequence_length, embedding_size, 1] 41 | self.embedded_chars_expanded = tf.expand_dims(self.embedded_chars, -1) 42 | 43 | # Create a convolution + maxpool layer for each filter size 44 | 45 | pooled_outputs = [] 46 | for i, filter_size in enumerate(filter_sizes): 47 | with tf.name_scope("conv-maxpool-%s" % filter_size): 48 | # Convolution Layer 49 | filter_shape = [filter_size, embedding_size, 1, num_filters] 50 | W = tf.Variable(tf.truncated_normal(filter_shape, stddev=0.1), name="W") 51 | b = tf.Variable(tf.constant(0.1, shape=[num_filters]), name="b") 52 | conv = tf.nn.conv2d( 53 | self.embedded_chars_expanded, 54 | W, 55 | strides=[1, 1, 1, 1], 56 | padding="VALID", 57 | name="conv") 58 | # Apply nonlinearity 59 | h = tf.nn.relu(tf.nn.bias_add(conv, b), name="relu") 60 | # Maxpooling over the outputs 61 | pooled = tf.nn.max_pool( 62 | h, 63 | ksize=[1, sequence_length - filter_size + 1, 1, 1], 64 | strides=[1, 1, 1, 1], 65 | padding='VALID', 66 | name="pool") 67 | pooled_outputs.append(pooled) 68 | 69 | # Combine all the pooled features 70 | num_filters_total = num_filters * len(filter_sizes) 71 | self.h_pool = tf.concat(pooled_outputs, 3) 72 | # 把池化层输出变成一维向量 73 | self.h_pool_flat = tf.reshape(self.h_pool, [-1, num_filters_total]) 74 | 75 | # Add dropout 76 | with tf.name_scope("dropout"): 77 | self.h_drop = tf.nn.dropout(self.h_pool_flat, self.dropout_keep_prob) 78 | 79 | # Final (unnormalized) scores and predictions 80 | with tf.name_scope("output"): 81 | W = tf.get_variable( 82 | "W", 83 | shape=[num_filters_total, num_classes], 84 | initializer=tf.contrib.layers.xavier_initializer()) 85 | b = tf.Variable(tf.constant(0.1, shape=[num_classes]), name="b") 86 | l2_loss += tf.nn.l2_loss(W) 87 | l2_loss += tf.nn.l2_loss(b) 88 | self.scores = tf.nn.softmax(tf.nn.xw_plus_b(self.h_drop, W, b, name="scores")) 89 | self.predictions = tf.argmax(self.scores, 1, name="predictions") 90 | 91 | # CalculateMean cross-entropy loss 92 | with tf.name_scope("loss"): 93 | losses = tf.nn.softmax_cross_entropy_with_logits(logits=self.scores, labels=self.input_y) 94 | self.loss = tf.reduce_mean(losses) + l2_reg_lambda * l2_loss 95 | 96 | # Accuracy 97 | with tf.name_scope("accuracy"): 98 | correct_predictions = tf.equal(self.predictions, tf.argmax(self.input_y, 1)) 99 | self.accuracy = tf.reduce_mean(tf.cast(correct_predictions, "float"), name="accuracy") 100 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/notes/01-Tensorflow简介，Anaconda安装，Tensorflow的CPU版本安装.md: -------------------------------------------------------------------------------- 1 | ## 1. Anaconda安装 2 | 3 | 1）Window、MacOS、Linux 都已支持 Tensorflow。 4 | 5 | 2）Window 用户只能使用 python3.5(64bit)。MacOS、Linux 支持 python2.7 和 python3.3+。 6 | 7 | 3）有 GPU 可以安装带 GPU 版本的，没有 GPU 就安装 CPU 版本的。 8 | 9 | 4） 推荐安装 Anaconda，pip 版本大于 8.1。 10 | 11 | 在学习过程中，建议使用 Jupyter Notebook 编程（当然也可以用其他工具，如 PyCharm），因安装完 Anaconda 自带 Jupyter Notebook，可以找到直接打开即可开始。 12 | 13 | 打开之后会打开默认浏览器，地址 `http://localhost:8888`，然后就可以在浏览器下面新建文件进行代码编写等操作了，其中，默认保存的路径为 C 盘用户文件夹下，如：`C:\Users\用户名`。我们可以修改路径为我们自己的想要的目录之下，修改操作： 14 | 15 | 1. 打开 cmd 输入命令 `jupyter notebook --generate-config`，可以看到生成文件的路径（可以看到默认在 `C:\Users\用户名\.jupyter` 文件夹下），这个就是生成的配置文件 `jupyter_notebook_config.py` 16 | 17 | ![](https://img-1256179949.cos.ap-shanghai.myqcloud.com/18-10-8-14115072.jpg) 18 | 19 | 2. 然后打开这个配置文件，找到 `#c.NotebookApp.notebook_dir = ' '`，把它改成： 20 | 21 | ``` xml 22 | c.NotebookApp.notebook_dir = '你想要设置的路径' 23 | ``` 24 | 25 | 如：`c.NotebookApp.notebook_dir = 'D:/Python/jupyter'`，那么以后再 Jupyter 上保存的文件就在`D:/Python/jupyter`文件夹里了。 26 | 27 | 注1：如果想要检测是否修改成功，可以在你设置的目录下里添加一个文件夹，例如：我在 jupyter 文件夹里添加了文件夹 text，那么在 Jupyter（浏览器上）就可以看到该文件夹了，则表明修改路径成功！ 28 | 29 | 注2：在进行如上修改之后，我发现未成功。网上找到了篇文章（[Anaconda Jupyter默认路径及修改无效解决方案](https://blog.csdn.net/mirrorui_/article/details/80605613)）也是遇到同样问题，按照文章解决方法，最后修改路径总算成功。其解决方法如下： 30 | 31 | 1. 找到 Jupyter 快捷方式，右键属性，并修改起始位置为你设置的路径，即刚才的「你想要设置的路径」 32 | 2. 在 Jupyter 的快捷方式属性中，有栏叫 `目标`，将这栏最后的 `%USERPROFILE%` 去掉。 33 | 34 | 另外：如果需要检测 TensorFlow 是否安装成功，可以打开 Jupyter 新建 python 文件，输入 `import tensorflow as tf` 运行，看是否报错。 35 | 36 | 37 | 38 | ## 2. TensorFlow安装 39 | 40 | 1、Windows安装TensorFlow 41 | 42 | - CPU版本，管理员方式打开命令提示符，输入命令：`pip install tensorflow` 43 | - GPU版本，管理员方式打开命令提示符，输入命令：`pip install tensorflow-gpu` 44 | 45 | 更新 TensorFlow： 46 | 47 | ``` python 48 | pip uninstall tensorflow 49 | pip install tensorflow 50 | ``` 51 | 52 | 注意，如果在安装过程中提示需要 `MSVCP140.DLL`，则下载安装： 53 | 54 | ``` xml 55 | TensorFlow requires MSVCP140.DLL, which may not be installed on your system. If, 56 | when you import tensorflow as tf, you see an error about No module named 57 | "_pywrap_tensorflow" and/or DLL load failed, check whether MSVCP140.DLL is in 58 | your %PATH% and, if not, you should install the Visual C++ 2015 redistributable (x64 59 | version). 60 | ``` 61 | 62 | 2、Linux和MacOS安装Tensorflow 63 | 64 | - CPU版本 65 | - Python 2.7用户：`pip install tensorflow` 66 | - Python3.3+用户：`pip3 install tensorflow` 67 | - GPU版本 68 | - Python 2.7用户：`pip install tensorflow-gpu` 69 | - Python3.3+用户：`pip3 install tensorflow-gpu` 70 | 71 | 关于 pip 和 pip3 的区别： 72 | 73 | - [python3中的pip和pip3](https://segmentfault.com/q/1010000010354189) 74 | - [安装python3后使用pip和pip3的区别](https://zhidao.baidu.com/question/494182519781589612.html?qbl=relate_question_1) 75 | 76 | > `pip`和`pip3`都在`Python36\Scripts\`目录下，如果同时装有 python2 和 python3，pip 默认给`python2` 用，`pip3`指定给 python3 用。如果只装有 python3，则`pip`和`pip3`是等价的。 77 | 78 | > 1、其实这两个命令效果是一样的，没有区别： 79 | > 80 | > （1）比如安装库 numpy，pip3 install numpy 或者 pip install numpy：只是当一台电脑同时有多个版本的 Python 的时候，用 pip3 就可以自动区分用 Python3 来安装库。是为了避免和 Python2 发生冲突的。 81 | > 82 | > （2）如果你的电脑只安装了 Python3，那么不管用 pip 还是 pip3 都一样的。 83 | > 84 | > 2、安装了 python3 之后，会有 pip3 85 | > 86 | > （1）使用 pip install XXX ： 87 | > 88 | > 新安装的库会放在这个目录下面：python2.7/site-packages； 89 | > 90 | > （2）使用 pip3 install XXX ： 91 | > 92 | > 新安装的库会放在这个目录下面：python3.6/site-packages； 93 | > 94 | > （3）如果使用 python3 执行程序，那么就不能 import python2.7/site-packages 中的库。 95 | 96 | 97 | 98 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/notes/11-Tensorflow在NLP中的使用(一).md: -------------------------------------------------------------------------------- 1 | 2 | 3 | ## TensorFlow在NLP的使用（一） 4 | 5 | ### 一、Word2vec 6 | 7 | word2vec 是 Google 于 2013 年开源的一个用于获取词向量的工具包，作者是 Tomas Mikolov。 8 | 9 | 它是一个将词表示为一个向量的工具，通过该向量表示，可以用来进行更深入的自然语言处理，比如机器翻译等。 10 | 11 | 网上资料： 12 | 13 | - [自己动手写word2vec 系列文章](https://blog.csdn.net/u014595019/article/details/51884529) 14 | - [word2vec 笔记](https://blog.csdn.net/zhangxb35/article/details/74716245) 15 | - ...... 16 | 17 | 当我们分析图片或者语音的时候，我们通常都是在分析密集的，高纬度的数据集。我们所需的全部信息都储存在原始数据中。 18 | 19 | ![](https://img-1256179949.cos.ap-shanghai.myqcloud.com/20181012143644.png) 20 | 21 | 当我们处理自然语言问题的时候，我们通常会做分词，然后给每一个词一个编号，比如猫的编号是 120，狗的编号是 343。比如女生的编号是 1232，女王的编号是 2329。这些编号是没有规律，没有联系的，我们从编号中不能得到词与词之间的相关性。 22 | 23 | 例如：How are you？ 24 | 25 | How : 234 26 | Are : 7 27 | you : 987 28 | 29 | 如果把上面转换为 one-hot 方式： 30 | 31 | ``` xml 32 | 000…1000000… 33 | 00000001000… 34 | 000…0000010 35 | ``` 36 | 37 | 但是你从编号中看不到词与此之间什么相关性。 38 | 39 | ### 二、Word2vec两种模型 40 | 41 | 1）连续词袋模型（CBOW） 42 | 43 | 根据词的上下文词汇来预测目标词汇，例如上下文词汇是“今天早餐吃__”，要预测的目标词汇可能是“面包”。 44 | 45 | 2）Skip-Gram模型 46 | 47 | Skip-Gram模型刚好和CBOW相反，它是通过目标词汇来预测上下文词汇。例如目标词汇是“早餐”，上下文词汇可能是“今天”和“吃面包”。 48 | 49 | ![](https://img-1256179949.cos.ap-shanghai.myqcloud.com/20181012144601.png) 50 | 51 | 对于这两种模型的训练，我们可能容易想到，使用 softmax 作为输出层来训练网络。这个方法是可 52 | 行的，只不过使用 softmax 作为输出层计算量将会是巨大的。假如我们已知上下文，需要训练模型 53 | 预测目标词汇，假设总共有 50000 个词汇，那么每一次训练都需要计算输出层的 50000 个概率值。 54 | 55 | 所以训练 Word2vec 模型我们通常可以选择使用噪声对比估计（Noise Contrastive Estimation） 56 | 。NCE 使用的方法是把上下文 h 对应地正确的目标词汇标记为正样本（D=1），然后再抽取一些错 57 | 误的词汇作为负样本（D=0）。然后最大化目标函数的值。 58 | 59 | ![](https://img-1256179949.cos.ap-shanghai.myqcloud.com/20181012144725.png) 60 | 61 | 当真实的目标单词被分配到较高的概率，同时噪声单词的概率很低时，目标函数也就达到最大值 62 | 了。计算这个函数时，只需要计算挑选出来的k个噪声单词，而不是整个语料库。所以训练速度会 63 | 很快。 64 | 65 | ### 三、Word2vec图形化 66 | 67 | ![](https://img-1256179949.cos.ap-shanghai.myqcloud.com/20181012144910.png) 68 | 69 | ### 四、CNN在自然语言处理的应用 70 | 71 | 说到 CNN 我们首先可能会想到 CNN 在计算机视觉中的应用。近几年 CNN 也开始应用于自然语言处 72 | 理，并取得了一些引人注目的成绩。 73 | 74 | CNN 应用于 NLP 的任务，处理的往往是以矩阵形式表达的句子或文本。矩阵中的每一行对应于一 75 | 个分词元素，一般是一个单词，也可以是一个字符。也就是说每一行都是一个词或者字符的向量 76 | （比如前面说到的 word2vec）。假设我们一共有 10 个词，每个词都用 128 维的向量来表示，那么 77 | 我们就可以得到一个 10×128 维的矩阵。这个矩阵就相当于是一副“图像”。 78 | 79 | ![](https://img-1256179949.cos.ap-shanghai.myqcloud.com/20181012145126.png) 80 | 81 | GitHub 上的例子：[cnn-text-classification-tf](https://github.com/dennybritz/cnn-text-classification-tf) 82 | 83 | 84 | 85 | -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/notes/12-Tensorflow在NLP中的使用(二).md: -------------------------------------------------------------------------------- 1 | ## TensorFlow在NLP中的使用（二） 2 | 3 | 1、声音信号 4 | 5 | 将 N 个采样点集合成一个观测单位，成为帧。通常 N 的值为 256 或 512，覆盖的时间约为 20-30ms 左右。为了避免两帧之间变化过大，因此会让两相邻帧之间有一段重叠区域。通常语音识别所采用的语音信号的采样频率为 8KHz 或 16KHz。 6 | 7 | ![](https://img-1256179949.cos.ap-shanghai.myqcloud.com/20181012145910.png) 8 | 9 | 2、梅尔频率倒谱系数（MFCC） 10 | 11 | MFCC 是一种广泛使用的语音特征，在 1980 年由 Davis 和 Mermelstein 研究出来。 12 | 13 | 3、声谱图 14 | 15 | 语音被分为很多帧，每帧语音都对应于一个频谱（通过 FFT 计算得到），频谱表示频率与能量的关 16 | 系。 17 | 18 | ![](https://img-1256179949.cos.ap-shanghai.myqcloud.com/20181012150052.png) 19 | 20 | 4、频谱图 21 | 22 | 我们先将一帧语音的频谱通过坐标表示出来，如左图。再将图旋转90度，如中间的图。然后把这些幅度映射到一个灰度级表示。 23 | 24 | ![](https://img-1256179949.cos.ap-shanghai.myqcloud.com/20181012150136.png) 25 | 26 | 5、spectrogram声谱图 27 | 28 | ![](https://img-1256179949.cos.ap-shanghai.myqcloud.com/20181012150208.png) 29 | 30 | 6、共振峰 31 | 32 | 下面是一个语音的频谱图。峰值就表示语音的主要频率成分，这些峰值成为共振峰。共振峰携带了声音的辨识属性。用它就可以识别不同的声音。 33 | 34 | ![](https://img-1256179949.cos.ap-shanghai.myqcloud.com/20181012150244.png) 35 | 36 | 7、包络 37 | 38 | 我们需要把共振峰提取出来，不仅要提取共振峰的位置，还要提取它们转变的过程，也就是频谱的包络。包络就是一条连接这些共振峰点的平滑曲线。 39 | 40 | ![](https://img-1256179949.cos.ap-shanghai.myqcloud.com/20181012150331.png) 41 | 42 | 8、分离包络和频谱的细节 43 | 44 | ![](https://img-1256179949.cos.ap-shanghai.myqcloud.com/20181012150401.png) 45 | 46 | 9、Mel频率分析 47 | 48 | 刚刚我们得到了频谱包络，不过人类听觉感知实验表明，人类的听觉的感知只聚焦在某些特定的区域，而不是整个频谱包络。Mel 频率分析就是基于人类听觉感知实验的。人耳就像一个滤波器组，它只关注某些特定频率的分量，也就是说它只让某些频率的信号通过。并且在低频区域由很多的滤波器，分布比较密集，在高频区域，滤波器比较少，也比较稀疏。 49 | 50 | ![](https://img-1256179949.cos.ap-shanghai.myqcloud.com/20181012150452.png) 51 | 52 | 10、Mel频率分析 53 | 54 | 人的听觉系统是一个特殊的非线性系统，它响应不同频率信号的灵明度是不同的。在语音特征的提取上，人类的听觉系统非常好，它不仅能提取出语义信息，而且能提取出说话人的个人特征。所以语音识别系统中能模拟人类听觉感知处理的特点，就有可能提高语音的识别率。 55 | 56 | MFCC 考虑到了人类的听觉特征，将线性频谱映射到基于听觉感知的Mel非线性频谱中。 57 | 58 | 11、语音处理流程 59 | 60 | ![](https://img-1256179949.cos.ap-shanghai.myqcloud.com/20181012150554.png) 61 | 62 | 12、ffmpeg 63 | 64 | - `conda install -c conda-forge ffmpeg` 65 | - Windows 安装方式：http://www.bubuko.com/infodetail-786878.html 66 | - Ubuntu 安装方式：http://blog.csdn.net/u012386199/article/details/51188988 -------------------------------------------------------------------------------- /03_dl_framework/tensorflow/《深度学习框架Tensorflow学习与应用》笔记/notes/tf.nn.softmax_cross_entropy_with_logits的用法问题.md: -------------------------------------------------------------------------------- 1 | 2 | 3 | 发现个问题：在【[04-softmax，交叉熵(cross-entropy)，dropout以及Tensorflow中各种优化器的介绍](./Notes/04-softmax，交叉熵\(cross-entropy\)，dropout以及Tensorflow中各种优化器的介绍.md)】中（三）节开始的代码`4-1交叉熵.py`，可以注意到如下代码： 4 | 5 | ``` python 6 | # 创建一个简单的神经网络 7 | W = tf.Variable(tf.zeros([784, 10])) 8 | b = tf.Variable(tf.zeros([10])) 9 | prediction = tf.nn.softmax(tf.matmul(x, W) + b) 10 | 11 | # 二次代价函数 12 | # loss = tf.reduce_mean(tf.square(y-prediction)) 13 | # 这里使用对数释然代价函数tf.nn.softmax_cross_entropy_with_logits()来表示跟softmax搭配使用的交叉熵 14 | loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=y, logits=prediction)) 15 | ``` 16 | 17 | 我觉得这里是有问题的，问题在哪呢？先看[【TensorFlow】tf.nn.softmax_cross_entropy_with_logits的用法](https://blog.csdn.net/zj360202/article/details/78582895)该文，可以了解到 `tf.nn.softmax_cross_entropy_with_logits` 函数的 logits 参数传入的是未经过 softmax 的 label 值。 18 | 19 | ``` python 20 | import tensorflow as tf 21 | 22 | #our NN's output 23 | logits=tf.constant([[1.0,2.0,3.0],[1.0,2.0,3.0],[1.0,2.0,3.0]]) 24 | ``` 25 | 26 | ``` python 27 | #step1:do softmax 28 | y=tf.nn.softmax(logits) 29 | #true label 30 | y_=tf.constant([[0.0,0.0,1.0],[0.0,0.0,1.0],[0.0,0.0,1.0]]) 31 | #step2:do cross_entropy 32 | cross_entropy = -tf.reduce_sum(y_*tf.log(y)) 33 | ``` 34 | 35 | 两步可以用这一步代替： 36 | 37 | ``` python 38 | #do cross_entropy just one step 39 | cross_entropy2=tf.reduce_sum(tf.nn.softmax_cross_entropy_with_logits(logits, y_))#dont forget tf.reduce_sum()!! 40 | ``` 41 | 42 | 但`prediction = tf.nn.softmax(tf.matmul(x, W) + b)`这里的 prediction 已经经历了一次 softmax，然后又经过了 tf.nn.softmax_cross_entropy_with_logits 函数，这相当于经过两个 softmax 了（虽然大的值的概率值还是越大，这点上倒是没影响。） 43 | 44 | 关于 softmax_cross_entropy_with_logits，这篇文章也有提到【[TensorFlow入门](https://statusrank.xyz/articles/31693f5f.html)】： 45 | 46 | > 这个函数内部自动计算 softmax 函数，然后在计算代价损失,所以logits必须是未经 softmax 处理过的函数，否则会造成错误。 47 | 48 | 注1：好像后面的笔记中程序代码都是这样的，我觉得可能视频讲解老师没注意到这点问题。另外，在该文 [06-卷积神经网络CNN的讲解，以及用CNN解决MNIST分类问题](./Notes/06-卷积神经网络CNN的讲解，以及用CNN解决MNIST分类问题.md) 的笔记中，我也记录了该问题。 49 | 50 | 注2：对 softmax、softmax loss、cross entropy 不了解，推荐看这几篇：**[卷积神经网络系列之softmax，softmax loss和cross entropy的讲解](https://blog.csdn.net/u014380165/article/details/77284921)** | [CNN入门讲解：我的Softmax和你的不太一样]()，讲解的非常易懂。【荐】 51 | 52 | 另外，关于在 TensorFlow 中有哪些损失函数的实现呢？看看该文：[tensorflow API:tf.nn.softmax_cross_entropy_with_logits()等各种损失函数](https://blog.csdn.net/NockinOnHeavensDoor/article/details/80139685) -------------------------------------------------------------------------------- /03_dl_framework/深度学习框架对比.md: -------------------------------------------------------------------------------- 1 | # 深度学习框架对比 2 | 3 | ## torch 4 | 5 | 模块名：Embeding layer 6 | 7 | 调用名： 8 | 9 | ``` python 10 | torch.nn.Embedding( 11 | num_embeddings, 12 | embedding_dim, 13 | padding_idx=None, 14 | max_norm=None, 15 | norm_type=2, 16 | scale_grad_by_freq=False, 17 | sparse=False) 18 | ``` 19 | 20 | 参数： 21 | 22 | ``` python 23 | num_embeddings--字典大小（vocab_size） 24 | embedding_dim--嵌入维度（embed_dim） 25 | padding_idx--如果提供的话，输出遇到此下标时用零填充， 26 | max_norm--如果提供的话，会重新归一化词嵌入，使它们的范数小于提供的值 27 | norm_type--对于max_norm选项计算p范数时的p, 28 | scale_grad_by_freq--如果提供的话，会根据字典中单词频率缩放梯度, 29 | sparse=False 30 | ``` 31 | 32 | 预训练： 33 | 34 | ``` python 35 | nn.Embedding.from_pretrained(weight) 36 | ``` 37 | 38 | 实例： 39 | 40 | ``` python 41 | import torch 42 | embedding = torch.nn.Embedding(10, 3) 43 | input = torch.LongTensor([[1,2,4,5],[4,3,2,9]]) 44 | embedding(input) 45 | result: 46 | tensor([[[-0.0251, -1.6902, 0.7172], 47 | [-0.6431, 0.0748, 0.6969], 48 | [ 1.4970, 1.3448, -0.9685], 49 | [-0.3677, -2.7265, -0.1685]], 50 | 51 | [[ 1.4970, 1.3448, -0.9685], 52 | [ 0.4362, -0.4004, 0.9400], 53 | [-0.6431, 0.0748, 0.6969], 54 | [ 0.9124, -2.3616, 1.1151]]]) 55 | ``` 56 | 57 | 58 | 59 | ## tensorflow 60 | 61 | 模块名：Embeding layer 62 | 63 | 调用名： 64 | 65 | ``` python 66 | tf.nn.embedding_lookup( 67 | params, 68 | ids, 69 | partition_strategy='mod', 70 | name=None, 71 | max_norm=None) 72 | ``` 73 | 74 | 参数： 75 | 76 | ``` xml 77 | params--输入类型（vocab_size,embed_dim） 78 | ids--输入数据 input 数据 79 | partition_strategy--指定切分策略的字符串,在 len(params) > 1 的情况下使用., 80 | name--名字, 81 | max_norm--如果提供该参数,embedding 值将被 L2-normalize 为 max_norm 的值 82 | ``` 83 | 84 | 预训练： 85 | 86 | ``` python 87 | 如果使用预训练直接把params设置为预训练向量 88 | ``` 89 | 90 | 91 | 92 | ## keras 93 | 94 | 模块名：Embeding layer 95 | 96 | 调用名： 97 | 98 | ``` python 99 | keras.layers.Embedding( input_dim,output_dim, 100 | embeddings_initializer='uniform',embeddings_regularizer=None, 101 | activity_regularizer=None,embeddings_constraint=None, 102 | mask_zero=False, 103 | input_length=None） 104 | ``` 105 | 106 | 参数： 107 | 108 | ``` xml 109 | input_dim, output_dim, 110 | embeddings_initializer='uniform', embeddings_regularizer=None, 111 | activity_regularizer=None, embeddings_constraint=None, 112 | mask_zero=False, 113 | input_length=None 114 | ``` 115 | 116 | *——from：某个群文件* 117 | 118 | 119 | 120 | -------------------------------------------------------------------------------- /04_computer_vision/[转]关于神经网络模型&TensorFlow学习&目标检测模型等内容的系列文章.md: -------------------------------------------------------------------------------- 1 | ## 关于神经网络模型&TensorFlow学习&目标检测模型等内容的系列文章 2 | 3 | 在开源中国看到一份系列文章，推荐看： 4 | 5 | - [大话卷积神经网络（CNN）](https://my.oschina.net/u/876354/blog/1620906) 6 | - [大话循环神经网络（RNN）](https://my.oschina.net/u/876354/blog/1621839) 7 | - [大话深度残差网络（DRN）](https://my.oschina.net/u/876354/blog/1622896) 8 | - [大话深度信念网络（DBN）](https://my.oschina.net/u/876354/blog/1626639) 9 | - [大话CNN经典模型：LeNet](https://my.oschina.net/u/876354/blog/1632862) 10 | - [大话CNN经典模型：AlexNet](https://my.oschina.net/u/876354/blog/1633143) 11 | - [大话CNN经典模型：VGGNet](https://my.oschina.net/u/876354/blog/1634322) 12 | - [大话CNN经典模型：GoogLeNet](https://my.oschina.net/u/876354/blog/1637819) 13 | - [浅说“迁移学习”](https://my.oschina.net/u/876354/blog/1614883) 14 | - [浅说“强化学习”](https://my.oschina.net/u/876354/blog/1614879) 15 | - …… 16 | - [【AI实战】快速掌握TensorFlow（一）：基本操作](https://my.oschina.net/u/876354/blog/1930175) 17 | - [【AI实战】快速掌握TensorFlow（二）：计算图、会话](https://my.oschina.net/u/876354/blog/1930490) 18 | - [【AI实战】快速掌握TensorFlow（三）：激励函数](https://my.oschina.net/u/876354/blog/1937296) 19 | - [【AI实战】快速掌握TensorFlow（四）：损失函数](https://my.oschina.net/u/876354/blog/1940819) 20 | - [【AI实战】搭建基础环境](https://my.oschina.net/u/876354/blog/1924805) 21 | - …… 22 | - [机器学习的五大流派](https://my.oschina.net/u/876354/blog/1676029) 23 | - [CNN进化史](https://my.oschina.net/u/876354/blog/1797489) 24 | - [大话目标检测经典模型（RCNN、Fast RCNN、Faster RCNN）](https://my.oschina.net/u/876354/blog/1787921) 25 | - [大话CNN经典模型：GoogLeNet（从Inception v1到v4的演进）](https://my.oschina.net/u/876354/blog/1637819) 26 | - …… 27 | 28 | > 作者：雪饼 [博文目录](https://my.oschina.net/u/876354)（推荐花点时间全部看完~） -------------------------------------------------------------------------------- /04_computer_vision/object_detection/img/《A Survey of Deep Learning-based Object Detection》综述小总结.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/04_computer_vision/object_detection/img/《A Survey of Deep Learning-based Object Detection》综述小总结.png -------------------------------------------------------------------------------- /04_computer_vision/object_detection/img/深度学习目标检测发展史.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/04_computer_vision/object_detection/img/深度学习目标检测发展史.jpg -------------------------------------------------------------------------------- /04_computer_vision/object_detection/object_detection.md: -------------------------------------------------------------------------------- 1 | # object_detection 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | ## Models 10 | 11 | - R-CNN：《Rich feature hierarchies for accurate object detection and semantic segmentation》[[Paper](https://arxiv.org/abs/1311.2524)] 12 | - Fast R-CNN：《Fast R-CNN》 [[Paper](https://arxiv.org/abs/1504.08083)] 13 | - Faster R-CNN：《Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks》 [[Paper](https://arxiv.org/abs/1506.01497)] 14 | - Yolo 15 | - SSD 16 | - Mask R-CNN ：《Mask R-CNN》 [[Paper](https://arxiv.org/abs/1703.06870)] -------------------------------------------------------------------------------- /04_computer_vision/object_detection/深度学习-目标检测.md: -------------------------------------------------------------------------------- 1 | # 目标检测(Object Detection) 2 | 3 | 4 | 5 | -------------------------------------------------------------------------------- /04_computer_vision/object_detection/目标检测发展.md: -------------------------------------------------------------------------------- 1 | # 目标检测发展 2 | 3 | ## 目标检测发展史 4 | 5 | ![](./img/深度学习目标检测发展史.jpg) 6 | 7 | 8 | 9 | ## 目标检测综述 10 | 11 | ![](./img/《A Survey of Deep Learning-based Object Detection》综述小总结.png) -------------------------------------------------------------------------------- /04_computer_vision/semantic_segmentation/readme.md: -------------------------------------------------------------------------------- 1 | #### (1) 图像分割基础 2 | 3 | ①什么是图像分割？ 4 | 5 | - [图像分割 传统方法 整理](https://zhuanlan.zhihu.com/p/30732385) [荐看完] 6 | 7 | 图片分割根据灰度、颜色、纹理、和形状等特征将图像进行划分区域，让区域间显差异性，区域内呈相似性。主要分割方法有： 8 | 9 | ``` xml 10 | 基于阈值的分割 11 | 基于边缘的分割 12 | 基于区域的分割 13 | 基于图论的分割 14 | 基于能量泛函的分割 15 | ``` 16 | 17 | - [十分钟看懂图像语义分割技术 | 雷锋网](https://www.leiphone.com/news/201705/YbRHBVIjhqVBP0X5.html) [荐看完] 18 | 19 | ②综述类/总结类： 20 | 21 | - [从全连接层到大型卷积核：深度学习语义分割全指南](https://mp.weixin.qq.com/s?__biz=MzA3MzI4MjgzMw==&mid=2650728920&idx=4&sn=3c51fa0a95742d37222c3e16b77267ca&scene=21#wechat_redirect) 22 | 23 | - [分割算法——可以分割一切目标（各种分割总结）](https://mp.weixin.qq.com/s/KcVKKsAyz-eVsyWR0Y812A) [荐] 24 | 25 | 深度学习最初流行的分割方法是，打补丁式的分类方法 (patch classification) 。逐像素地抽取周围像素对中心像素进行分类。由于当时的卷积网络末端都使用全连接层 (full connected layers) ，所以只能使用这种逐像素的分割方法。 26 | 27 | 但是到了 2014 年，来自伯克利的 Fully Convolutional Networks（FCN）卷积网络，去掉了末端的全连接层。随后的语义分割模型基本上都采用了这种结构。除了全连接层，语义分割另一个重要的问题是池化层。池化层能进一步提取抽象特征增加感受域，但是丢弃了像素的位置信息。但是语义分割需要类别标签和原图像对齐，因此需要从新引入像素的位置信息。有两种不同的架构可以解决此像素定位问题。 28 | 29 | 第一种是`编码-译码架构`。编码过程通过池化层逐渐减少位置信息、抽取抽象特征；译码过程逐渐恢复位置信息。一般译码与编码间有直接的连接。该类架构中 U-net 是最流行的。 30 | 31 | 第二种是`膨胀卷积` (dilated convolutions) 【这个核心技术值得去阅读学习】，抛弃了池化层。 32 | 33 | - [一文概览主要语义分割网络：FCN,SegNet,U-Net...](https://www.tinymind.cn/articles/410) 34 | 35 | 该文为译文，介绍了很多语义分割的深度学习模型，包括半监督下的语义分割，可以大致看下。 36 | 37 | ③深度学习语义分割模型的介绍： 38 | 39 | - [语义分割(semantic segmentation) 常用神经网络介绍对比-FCN SegNet U-net DeconvNet](https://blog.csdn.net/zhyj3038/article/details/71195262) 40 | - [深度学习（十九）——FCN, SegNet, DeconvNet, DeepLab, ENet, GCN](https://blog.csdn.net/antkillerfarm/article/details/79524417) 41 | 42 | ④图像分割的衡量指标： 43 | 44 | - [图像分割的衡量指标详解](https://blog.csdn.net/qq_37274615/article/details/78957962) 45 | 46 | 语义分割其实就是对图片的每个像素都做分类。其中，较为重要的语义分割数据集有：VOC2012 以及 MSCOCO。 47 | 48 | #### (2) 图像分割仓库 49 | 50 | - [semseg](https://github.com/guanfuchen/semseg) 51 | 52 | > 常用的语义分割架构结构综述以及代码复现 53 | 54 | - [DeepNetModel](https://github.com/guanfuchen/DeepNetModel) 55 | 56 | > 记录每一个常用的深度模型结构的特点（图和代码） 57 | > 58 | > 大佬的博客：[计算机视觉相关资源整理](https://guanfuchen.github.io/post/markdown_blog_ws/markdown_blog_2017_11/%E8%AE%A1%E7%AE%97%E6%9C%BA%E8%A7%86%E8%A7%89%E7%9B%B8%E5%85%B3%E8%B5%84%E6%BA%90%E6%95%B4%E7%90%86/) 59 | 60 | - [Semantic-Segmentation-Suite](https://github.com/GeorgeSeif/Semantic-Segmentation-Suite) 61 | 62 | > Semantic Segmentation Suite in TensorFlow. Implement, train, and test new Semantic Segmentation models easily! 63 | 64 | - [mrgloom/awesome-semantic-segmentation](https://github.com/mrgloom/awesome-semantic-segmentation)（图像分割论文下载及实现可以在这里找到~） 65 | 66 | #### (3) 图像分割论文及最新研究 67 | 68 | 论文汇集： 69 | 70 | - [语义分割 - Semantic Segmentation Papers](https://blog.csdn.net/zziahgf/article/details/72639791) 71 | 72 | 73 | 74 | #### (4) 图像分割讲解视频 75 | 76 | - [浙大博士生刘汉唐：带你回顾图像分割的经典算法](http://www.mooc.ai/course/414/learn#lesson/2266)（需要注册才能观看~） 77 | - [197期\_张觅\_基于深度卷积网络的遥感影像语义分割层次认知方法](https://www.bilibili.com/video/av24599502?from=search&seid=11210211322309323243)（关于遥感图像语义分割的，但听得不是很清楚~） 78 | - [【 计算机视觉 】深度学习语义分割Semantic Segmentation（英文字幕）（合辑）_哔哩哔哩]() -------------------------------------------------------------------------------- /04_computer_vision/semantic_segmentation/semantic_segmentation.md: -------------------------------------------------------------------------------- 1 | # Semantic segmentation 2 | 3 | ## 语义分割 4 | 5 | - [语义分割 - Semantic Segmentation Papers - CSDN博客](https://blog.csdn.net/zziahgf/article/details/72639791) 6 | - [语义分割 - Semantic Segmentation Papers - AIUAI](https://www.aiuai.cn/aifarm62.html) 7 | - [分类 语义分割 下的文章 - AIUAI](https://www.aiuai.cn/category/segmentation/) 8 | - [Segmentation - handong1587](https://handong1587.github.io/deep_learning/2015/10/09/segmentation.html) 9 | - [mrgloom/awesome-semantic-segmentation](https://github.com/mrgloom/awesome-semantic-segmentation) - 语义分割、实例分割、数据集、Benchmark、标注工具、医学图像分割、卫星图像分割... 10 | 11 | - [GeorgeSeif/Semantic-Segmentation-Suite](https://github.com/GeorgeSeif/Semantic-Segmentation-Suite) 12 | 13 | - [guanfuchen/semseg](https://github.com/guanfuchen/semseg) 14 | 15 | - [AstarLight/Satellite-Segmentation](https://github.com/AstarLight/Satellite-Segmentation) 16 | 17 | 18 | 19 | 20 | 21 | ## Models 22 | 23 | 24 | - **FCN：**《Fully Convolutional Networks for Semantic Segmentation》 [[Paper-v1](https://arxiv.org/abs/1411.4038v1)] [[Paper-v2](https://arxiv.org/abs/1411.4038v2)]（最新提交时间：2015.03.08） 25 | - **U-Net：**《U-Net: Convolutional Networks for Biomedical Image Segmentation》[[Paper](https://arxiv.org/abs/1505.04597)]（最新提交时间：2015.05.18） 26 | - **SegNet：**《SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation》[[Paper-v1](https://arxiv.org/abs/1511.00561v1)] [[Paper-v2](https://arxiv.org/abs/1511.00561v2)] [[Paper-v3](https://arxiv.org/abs/1511.00561v3)]（最新提交时间：2016.11.10） 27 | - Dilated Convolutions：《Multi-Scale Context Aggregation by Dilated Convolutions》[[Paper-v1](https://arxiv.org/abs/1511.07122v1)] [[Paper-v2](https://arxiv.org/abs/1511.07122v2)] [[Paper-v3](https://arxiv.org/abs/1511.07122v3)]（最新提交时间：2016.04.30） 28 | - DeconvNet：《Learning Deconvolution Network for Semantic Segmentation》[[Paper](https://arxiv.org/abs/1505.04366)]（最新提交时间：2015.05.17） 29 | - RefineNet：《RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation》[[Paper-v1](https://arxiv.org/abs/1611.06612v1)] [[Paper-v2](https://arxiv.org/abs/1611.06612v2)] [[Paper-v3](https://arxiv.org/abs/1611.06612v3)]（最新提交时间：2016.11.25） 30 | - PSPNet：《Pyramid Scene Parsing Network》[[Paper-v1](https://arxiv.org/abs/1612.01105v1)] [[Paper-v2](https://arxiv.org/abs/1612.01105v2)]（最新提交时间：2017.04.27） 31 | - Large Kernel Matters：《Large Kernel Matters -- Improve Semantic Segmentation by Global Convolutional Network》[[Paper](https://arxiv.org/abs/1703.02719)]（最新提交时间：2017.03.08） 32 | - **DeepLab 系列：** 33 | - DeepLab v1：《Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs》[[Paper-v1](https://arxiv.org/abs/1412.7062v1)] [[Paper-v2](https://arxiv.org/abs/1412.7062v2)] [[Paper-v3](https://arxiv.org/abs/1412.7062v3)] [[Paper-v4](https://arxiv.org/abs/1412.7062v4)]（最新提交时间 ：2016.06.07） 34 | - DeepLab v2：《DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs》[[Paper-v1](https://arxiv.org/abs/1606.00915v1)] [[Paper-v2](https://arxiv.org/abs/1606.00915v2)]（最新提交时间：2017.05.12） 35 | - DeepLab v3：《Rethinking Atrous Convolution for Semantic Image Segmentation》[[Paper-v1](https://arxiv.org/abs/1706.05587v1)] [[Paper-v2](https://arxiv.org/abs/1706.05587v2)] [[Paper-v3](https://arxiv.org/abs/1706.05587v3)]（最新提交时间：2017.12.05） 36 | - DeepLab v3+：《Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation》[[Paper-v1](https://arxiv.org/abs/1802.02611v1)] [[Paper-v2](https://arxiv.org/abs/1802.02611v2)] [[Paper-v3](https://arxiv.org/abs/1802.02611v3)]（最新提交时间：2018.08.22） 37 | - NAS：《Searching for Efficient Multi-Scale Architectures for Dense Image Prediction》[[Paper-v1](https://arxiv.org/abs/1809.04184)]（提交时间：2018.09.11） 相关文章：[语义分割领域开山之作：Google提出用神经网络搜索实现语义分割 | 雷锋网](https://www.leiphone.com/news/201810/hPe93A6N0YSQPF7y.html) -------------------------------------------------------------------------------- /04_computer_vision/semantic_segmentation/全景分割模型.md: -------------------------------------------------------------------------------- 1 | # 全景分割模型 2 | 3 | -------------------------------------------------------------------------------- /04_computer_vision/semantic_segmentation/实例分割模型.md: -------------------------------------------------------------------------------- 1 | # 实例分割 2 | 3 | ## 2017 Mask R-CNN《Mask R-CNN》 4 | 5 | 6 | 7 | ## 2019《Mask Scoring R-CNN》 -------------------------------------------------------------------------------- /04_computer_vision/semantic_segmentation/语义分割总结.md: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | ``` 6 | - 什么是超像素、语义分割、实例分割、全景分割？ 7 | - 什么是同物异谱、同谱异物？ 8 | - RGB图像、全色图像、多光谱图像、高光谱图像？ 9 | - ... 10 | - 语义分割发展和历史 11 | - 2000年之前，数字图像处理时我们采用方法基于几类：阈值分割、区域分割、边缘分割、纹理特征、聚类等 12 | - 2000年到2010年期间， 主要方法有四类：基于图论、聚类、分类以及聚类和分类结合。 13 | - 2010年至今，神经网络模型的崛起和深度学习的发展，主要涉及到几种模型 14 | 发展历程： 15 | - 2014年 FCN 模型，主要贡献为在语义分割问题中推广使用端对端卷积神经网络，使用反卷积进行上采样 16 | - 2015年 U-net 模型，构建了一套完整 的编码解码器 17 | - 2015年 SegNet 模型，将最大池化转换为解码器来提高分辨率 18 | - 2015年 Dilated Convolutions（空洞卷积），更广范围内提高了内容的聚合并不降低分辨率 19 | - 2016年 DeepLab v1&v2 20 | - 2016年 RefineNet 使用残差连接，降低了内存使用量，提高了模块间的特征融合 21 | - 2016年 PSPNet 模型 22 | - 2017年 Large Kernel Matters 23 | - 2017年 DeepLab V3 24 | 以上几种模型可以按照语义分割模型的独有方法进行分类，如专门池化（PSPNet、DeepLab），编码器-解码器架构（SegNet、E-Net），多尺度处理（DeepLab）、条件随机场（CRFRNN）、空洞卷积（DiatedNet、DeepLab）和跳跃连接（FCN）。 25 | 26 | - 前DL时代的语义分割： 27 | - Grab cut是微软剑桥研究院于2004年提出的著名交互式图像语义分割方法。与N-cut一样，grab cut同样也是基于图划分，不过grab cut是其改进版本，可以看作迭代式的语义分割算法。Grab cut利用了图像中的纹理（颜色）信息和边界（反差）信息，只要少量的用户交互操作即可得到比较好的前后背景分割结果。 28 | 。。。 29 | - DL时代 30 | ``` 31 | 32 | 33 | 34 | 35 | 36 | ``` 37 | 语义分割是对图像的一种更精细的推断与理解，由粗到细为： 38 | 39 | - 图像分类 - 初级的图片理解，其对整张图片进行整体理解. 40 | - 目标定位与检测 - 不仅提供图像内的类别，还包括相对于物体类别的空间为位置信息. 41 | - 语义分割 - 对每个图像像素进行密集预测，得到像素类别信息. 42 | 43 | 图像的语义分割是将输入图像中的每个像素分配一个语义类别，以得到像素化的密集分类。 44 | 45 | 虽然自 2007 年以来，语义分割/场景解析一直是计算机视觉社区的一部分，但与计算机视觉中的其他领域很相似，自 2014 年 Long 等人首次使用全卷积神经网络对自然图像进行端到端分割，语义分割才有了重大突破。 46 | 47 | ——from：https://www.aiuai.cn/aifarm602.html#E-Net%E5%92%8CLink-Net 48 | ``` 49 | 50 | -------------------------------------------------------------------------------- /04_computer_vision/semantic_segmentation/语义分割模型.md: -------------------------------------------------------------------------------- 1 | ## 语义分割模型 2 | 3 | ### 2014 FCN 4 | 5 | 6 | 7 | ### 2014 DeepLabv1 8 | 9 | 10 | 11 | ### 2015 DeconvNet 12 | 13 | ### 2015 Dilated Convolutions 14 | 15 | 16 | 17 | ### 2015 SegNet 18 | 19 | 20 | 21 | 22 | 23 | 关于 segnet 模型的代码，有很多没看到实现池化索引，网上看到篇文章提到 keras 如何实现的，看看：[keras SegNet 使用池化索引（pooling indices）]() 24 | 25 | 另外，github 上有个 segnet 模型的代码也有实现：[ykamikawa/tf-keras-SegNet]() 26 | 27 | ### 2015 U-Net 28 | 29 | ### 2016 DeepLab v2 30 | 31 | ### 2016 PSPNet 32 | 33 | 34 | 35 | ### 2016 RefineNet 36 | 37 | 38 | 39 | ### 2017 DeepLab v3 40 | 41 | 42 | 43 | ### 2017 DeepLab v3+ 44 | 45 | ### 2017 Large Kernel Matters 46 | 47 | 48 | 49 | 50 | 51 | ### 2017 MobileNets 52 | 53 | 54 | 55 | ### 2018 全景分割《Panoptic Segmentation》 56 | 57 | 58 | 59 | ### 2018 神经网络搜索实现语义分割《Searching for Efficient Multi-Scale Architectures for Dense Image Prediction》 60 | 61 | 62 | 63 | ### 2019 NAS《Auto-DeepLab Hierarchical Neural Architecture Search for Semantic Image Segmentation》 64 | 65 | 66 | 67 | ### 2019 全景FPN《Panoptic Feature Pyramid Networks》 68 | 69 | 70 | 71 | ### 2019《TensorMask A Foundation for Dense Object Segmentation》 72 | 73 | 74 | 75 | 76 | 77 | ### 78 | 79 | -------------------------------------------------------------------------------- /04_computer_vision/semantic_segmentation/语义分割评价准则.md: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 参考资料： 6 | 7 | - [语义分割 | 发展综述](https://zhuanlan.zhihu.com/p/37618829) 8 | - [图像分割的衡量指标详解](https://blog.csdn.net/qq_37274615/article/details/78957962) 9 | 10 | -------------------------------------------------------------------------------- /04_computer_vision/图像标注工具.md: -------------------------------------------------------------------------------- 1 | # labelme 语义分割 2 | 3 | Windows 下安装： 4 | 5 | ``` 6 | pip install pyqt5 # pyqt5 can be installed via pip on python3 7 | pip install labelme 8 | ``` 9 | 10 | 如上默认安装的是 labelme 最新版，此时我的版本是：`3.14.1` 11 | 12 | 注1：本人使用 labelme 进行进行标注得到 json 文件，然后使用 `labelme_json_to_dataset` 转换的时候，得到的 label.png 为彩色，而非黑色图像，看评论有人说是版本问题… 13 | 14 | 注2：然后我安装了 labelme 旧版 `2.9.0`，`pip install labelme==2.9.0`，发现这个版本 `labelme_json_to_dataset` 命令生成的 `label.png` 文件确实是全黑色，并且是 **16 位深度**的。 15 | 16 | 然后我使用 `cv2.imread(“label.png”)` 读取发现得到的数值最小最大都是 0；使用 cv2.imread(label.png”, 2) 读取发现得到的数值最小是 0，最大是 1，为什么呢？后来知道了。先看 [opencv imread()方法第二个参数介绍]() | [opencv中imread第二个参数的含义]()，可以说，`imread(const string& filename, int flag=1)`，filename 指图像名称，flag 指读取图像颜色类型。 17 | 18 | - flag=-1时，8位深度，原通道 19 | - flag=0，8位深度，1通道 20 | - flag=1, 8位深度 ，3通道 21 | - flag=2，原深度，1通道 22 | - flag=3, 原深度，3通道 23 | - flag=4，8位深度 ，3通道 24 | 25 | 我解释下：因为 label.png 是 16 位的，默认 flag=1，按上可以看到只读取到了图像的 8 位，得到 3 通道，得到的全是 0；若 flag=2，按原深度即读取了图像位深度 16 位，得到了数值 1。 26 | 27 | 我的理解：本质原因在于 imread 读取了图像的多少位。另外注意，如果本来是 1 个通道的图像，imread 第二个参数选择了返回 3 个通道的，那么第一个通道读取的数值，在相同像素的位置另外两个通道也会有同样数值。 28 | 29 | 注：可以使用 matlab 软件 imread(imagepath) 读取图像，点击打开工作区的 ans，可以看到图像数值以及是通道数量。 30 | 31 | 32 | 33 | 相关阅读： 34 | 35 | - [wkentaro/labelme]() 36 | - [图片标注工具Labelme的安装及使用方法]() 37 | - [labelme标注的数据分析]() 38 | - [图像语义分割标注工具labelme制作自己的数据集用于mask-rcnn训练]() 39 | - [图片标注工具Labelme的安装及使用方法]() 40 | - [labelme标注的数据分析]() 41 | - [图像语义分割标注工具labelme制作自己的数据集用于mask-rcnn训练]() 42 | 43 | 44 | 45 | # labelme 目标检测 46 | 47 | - [LabelImg图片标注]() 48 | 49 | -------------------------------------------------------------------------------- /04_computer_vision/如何将图像分割数据集制作为TFRecord文件.md: -------------------------------------------------------------------------------- 1 | 2 | 3 | 参考： 4 | 5 | - [针对分割问题（图片）的TFRecord使用方法](https://zhuanlan.zhihu.com/p/32078191) 6 | - [如何制作将分割数据集封装成tfrecords](https://blog.csdn.net/qwe2508/article/details/81180613#commentBox) 7 | 8 | 9 | 10 | -------------------------------------------------------------------------------- /04_computer_vision/摘入一些对人工智能的的看法和思考的内容.md: -------------------------------------------------------------------------------- 1 | ## 摘入一些看过的觉得有用和值得思考的内容 2 | 3 | 这篇文章【[让算法解放算法工程师——NAS 综述](https://www.leiphone.com/news/201809/snezJyk32bUpwKgW.html?type=preview)】说到： 4 | 5 | > 构建一个典型的机器学习项目，一般分成以下步骤：收集原始数据（标注数据）、清洗数据、特征工程、模型构建、超参数调优、模型验证和模型部署。整个过程中，模型构建最能体现创造力，而最耗时的，要数特征工程和超参数调优。 6 | 7 | 8 | 9 | -------------------------------------------------------------------------------- /05_journals/readme.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/05_journals/readme.md -------------------------------------------------------------------------------- /05_journals/参考文献格式.md: -------------------------------------------------------------------------------- 1 | 国家期刊出版格式要求在中图分类号的下面应标出文献标识码，规定如下： 2 | 3 | （1）作者可从下列A、B、C、D、E中选用一种标识码来揭示文章的性质： 4 | 5 | - A—理论与应用研究学术论文（包括综述报告）； 6 | - B—实用性成果报告（科学技术）、理论学习与社会实践总结（科技）； 7 | - C—业务指导与技术管理的文章（包括特约评论）； 8 | - D—一般性通讯、报导、专访等； 9 | - E—文件、资料、人物、书刊、知识介绍等。 10 | 11 | 注：英文的文献标识码应与中文对应。 12 | 13 | （2）[参考文献类型标识码] 14 | 15 | - M——专著 Monograph ； 16 | - C——-论文集Collection ； 17 | - N——报纸文章 News； 18 | - J——期刊文章 Journal； 19 | - D——学位论文 Degree； 20 | - R——报告 Report； 21 | - S——标准 Standard； 22 | - P——专利 Patent； 23 | - A——专著、论文集中的析出文献 Article； 24 | - Z——其它末说明文献 25 | 26 | （1）文献类型标识：专著[M]；期刊[J]；论文集[c]；学位论文[D]； 27 | 标准[S]；报告[R]；专利[P]；报纸[N]； 28 | 29 | （2）电子文献类型标识：数据库[DB]；计算机程序[CP]；电子公告[EB]； 30 | 31 | （3）电子文献的载体类型及其标识：联机网上数据库[DB／OL]； 32 | 33 | 34 | 35 | 参考资料： 36 | 37 | - [论文参考文献中J、M等是什么意思 - 百度知道]() 38 | - [参考文献标准格式 - 知乎]() 39 | - [参考文献格式详细解释（常见） - 知乎]() 40 | 41 | -------------------------------------------------------------------------------- /05_journals/文献管理软件Zotero使用.md: -------------------------------------------------------------------------------- 1 | 简单来说，Zotero 是一款文献管理工具。帮助你收集、管理和引用具有参考价值的论文、期刊、书籍等知识性资源。初期的 Zotero 是基于 Firefox 浏览器的一款插件，目前已有独立单机版可下载。 2 | 3 | Zotero 的软件界面是基于浏览器内核的，支持 Windows, MacOS, Linux 多种操作系统。作为一款协助科研工作者收集、管理以及引用研究资源的免费软件，如今已被广泛使用。 4 | 5 | - 要是在安装完 Zotero 后，若未自动安装对应的 Word 插件，可以尝试该方式：[zotero自动安装word插件失败]()，本人在使用 Office2013 时，按照文中步骤操作可解决问题。在使用 Office2016 时，我的操作是：仅仅把 zotero 安装目录下的这个文件夹 `\Zotero\extensions\zoteroWinWordIntegration@zotero.org\install\` 下的 `Zotero.dotm` 复制到 `\Microsoft Office\Office16\STARTUP\`，然后打开 Word 找到 “选项” -> "信任中心" -> "信任中心设置" -> "宏设置" -> 选择第四项"启用所有宏..." 即可解决。 6 | 7 | 浏览器扩展：Chrome 下的安装 [Zotero Connector](https://chrome.google.com/webstore/detail/zotero-connector/ekhagklcjbdpajgpjgmbionohlpdbjgc/related?utm_source=chrome-ntp-icon)，这个插件安装后，会在浏览器中产生一个按钮（支持Chrome，Firefox 等），当你在线搜索到一篇或一系列文献时，这个按钮会检测到你目前的页面中所包含的文献信息和 pdf 附件。只要轻点按钮，就可以把这些信息和所有的附件下载到本地的 Zotero 软件中。 8 | 9 | 如何分享引用研究资源、插入文献、坚果云同步文献：[科研工作者的神器-zotero论文管理工具](https://mp.weixin.qq.com/s/rUv0d05xJgMxm2YoefWH5A)（我使用的该方式） | [标准论文参考文献添加方法——Zotero入门使用教程](https://blog.csdn.net/misayaaaaa/article/details/53559689) | [如何在Zotero中设置webdav连接到坚果云？](http://help.jianguoyun.com/?p=3168) 10 | 11 | - 注：插入文献时候，如果需删除「域」，若直接删除要是卡顿，可以先「解除域」再删除，这样会很快。解除域的快捷键为 `Ctrl＋Shift＋F9`。参考：[怎样全部去掉word中的域_百度知道](https://zhidao.baidu.com/question/166021389.html)。如果文件已进行了保存，想要还原域是没办法了，否则可以通过撤销上一步操作还原。 12 | 13 | 详细的：[Zotero（1）：文献管理软件Zotero基础及进阶示范 - 阳志平的网志](https://www.yangzhiping.com/tech/zotero1.html) 14 | 15 | 16 | 17 | 18 | 19 | -------------------------------------------------------------------------------- /assets/images/2018爱分析·中国人工智能创新企业榜.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/assets/images/2018爱分析·中国人工智能创新企业榜.png -------------------------------------------------------------------------------- /assets/images/[转]企业中人工智能相关岗位对硕士的要求.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/assets/images/[转]企业中人工智能相关岗位对硕士的要求.png -------------------------------------------------------------------------------- /assets/images/[转]对计算机视觉方向的看法.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/assets/images/[转]对计算机视觉方向的看法.jpg -------------------------------------------------------------------------------- /assets/images/玩转AI面试.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/assets/images/玩转AI面试.jpg -------------------------------------------------------------------------------- /dl_examples/MNIST/keras-mnist/mnist-keras.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | np.random.seed(1337) 3 | from keras.datasets import mnist 4 | from keras.utils import np_utils 5 | from keras.models import Sequential 6 | from keras.layers import Dense, Activation 7 | from keras.optimizers import RMSprop 8 | 9 | # 下载数据集 10 | (X_train, y_train), (X_test, y_test) = mnist.load_data('./mnist.npz') # 下载不来，可以手动下载下来后缀npz的数据，下载地址：https://s3.amazonaws.com/img-datasets/mnist.npz 11 | 12 | # 数据预处处理 13 | X_train = X_train.reshape(X_train.shape[0], -1) / 255. 14 | X_test = X_test.reshape(X_test.shape[0], -1) / 255. 15 | y_train = np_utils.to_categorical(y_train, num_classes=10) 16 | y_test = np_utils.to_categorical(y_test, num_classes=10) 17 | 18 | # 不使用model.add()，用以下方式也可以构建网络 19 | model = Sequential([ 20 | Dense(400, input_dim=784), 21 | Activation('relu'), 22 | Dense(10), 23 | Activation('softmax'), 24 | ]) 25 | 26 | # 定义优化器 27 | rmsprop = RMSprop(lr=0.001, rho=0.9, epsilon=1e-08, decay=0.0) 28 | model.compile(optimizer=rmsprop, 29 | loss='categorical_crossentropy', 30 | metrics=['accuracy']) # metrics赋值为'accuracy'，会在训练过程中输出正确率 31 | 32 | # 这次我们用fit()来训练网路 33 | print('Training ------------') 34 | model.fit(X_train, y_train, epochs=4, batch_size=32) 35 | 36 | print('\nTesting ------------') 37 | # 评价训练出的网络 38 | loss, accuracy = model.evaluate(X_test, y_test) 39 | 40 | print('test loss: ', loss) 41 | print('test accuracy: ', accuracy) -------------------------------------------------------------------------------- /dl_examples/MNIST/keras-mnist/mnist.npz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jayboxyz/deeplearning-cv-notes/2e368164fd9eccfeb8ead3ccc6413111ed8c87cd/dl_examples/MNIST/keras-mnist/mnist.npz -------------------------------------------------------------------------------- /dl_examples/MNIST/手写数字识别MNIST讲解.md: -------------------------------------------------------------------------------- 1 | (1) [CNN实现MNIST手写数字识别](https://blog.csdn.net/lijiecao0226/article/details/78379110) 2 | 3 | 通过该文重点体会下网络结构卷积、池化过程，feature map 以及深度的变化。 4 | 5 | 如果不清楚，可以看下该文【[从AlexNet理解卷积神经网络的一般结构](https://blog.csdn.net/chaipp0607/article/details/72847422)】，现摘入部分内容： 6 | 7 | CNN 中的卷积层操作与图像处理中的卷积是一样的，都是一个卷积核对图像做自上而下，自左而右的加权和操作，不同指出在于，在传统图像处理中，我们人为指定卷积核，比如 Soble，我们可以提取出来图像的水平边缘和垂直边缘特征。而在 CNN 中，卷积核的尺寸是人为指定的，但是卷积核内的数全部都是需要不断学习得到的。比如一个卷积核的尺寸为`3×3×3`，分别是宽，高和厚度，那么这一个卷积核中的参数有 27 个。在这里需要说明一点： 8 | 9 | - 卷积核的厚度=被卷积的图像的通道数 10 | - 卷积核的个数=卷积操作后输出的通道数 11 | 12 | 这两个等式关系在理解卷积层中是非常重要的！！举一个例子，输入图像尺寸`5×5×3`（宽/高/通道数）,卷积核尺寸：`3×3×3`（宽/高/厚度），步长：1，边界填充：0，卷积核数量：1。用这样的一个卷积核去卷积图像中某一个位置后，是将**该位置上宽 3，高 3，通道 3 上 27 个像素值分别乘以卷积核上 27 个对应位置的参数**，得到一个数，依次滑动，得到卷积后的图像，**这个图像的通道数为 1**（与卷积核个数相同），图像的高宽尺寸如下公式：![](https://img-1256179949.cos.ap-shanghai.myqcloud.com/20181105201546.png)所以，卷积后的图像尺寸为：`3×3×1`（宽/高/通道数） 13 | 14 | 15 | 16 | (2) [TensorFlow入门1-CNN网络及MNIST例子讲解](https://blog.csdn.net/zgzczzw/article/details/79897956) 17 | 18 | 该文讲解的挺全面的，值得一看。现摘入其中部分内容： 19 | 20 | 全连接层，顾名思义，指的是这一层的每个节点都跟上一层的全部节点有连接。 21 | 22 | ![](https://img-1256179949.cos.ap-shanghai.myqcloud.com/20181105203937.png) 23 | 24 | 比如图中全连接层的节点 a1 与上一层的所有节点 x1、x2、x3 有联系，a2 也与上一层的所有节点有联系，就这样，少一个也不行。因此解释全连接层的时候，图就是上面这样一个网状结构。 25 | 26 | 假设 x1，x2，x3 是全连接层的输入值，那么在全连接层的 a1 可以表示为以下的形式。 27 | 28 | ![](https://img-1256179949.cos.ap-shanghai.myqcloud.com/20181105204240.png) 29 | 30 | 从这个公式可以看出，a1 与 x1，x2，x3 都有联系，只是输入节点的权重值不同。同理，a2 和 a3 也可以表示为以下的形式。 31 | 32 | ![](https://img-1256179949.cos.ap-shanghai.myqcloud.com/20181105204304.png) 33 | 34 | 由此就可以从数学层面去理解什么叫全连接层，但是这个全连接层在实际应用中表示得意义是什么呢，简单来说，它可以作为一个分类网络，还是上面那个图，输入层是 3 个值，全连接层是 3 个节点，就代表这个全连接层可以把上一层的特征值分类为三种特征。当然，输入层和全连接层的节点数并不一定相同，比如下面这个结构。 35 | 36 | ![](https://img-1256179949.cos.ap-shanghai.myqcloud.com/20181105204341.png) 37 | 38 | 最后的全连接层是 10 个节点，全连接层的上一层是 15 个特征值，这样就把 15 个特征值归类为 10 种特征。举例来说，比如输出的 15 个特征标记位 t1-15，而 t1，t3，t5 三种特征可以认为都属于某一种输出 o1。而 t2，t6 两种特征可以认为属于输出 o2，以此类推，就可以把 15 个特征分类为 10 个输出。 39 | 40 | 前面提到的 MNIST 问题中，全连接层有两层，第一层是 1024 个节点，第二层是 10 个节点。一般情况下，最靠近用户的输出层就是用户所期待的结果类别数，MNIST 问题中，用户期待的输出是 0-9 一共 10 个数字也就是 10 种类别，所以第二层全连接层是 10 个节点。再往前看，卷积层在每一个像素点上提取 64 个特征值，所以整个图片可以有`高x宽`x64个特征值，也就是 `7x7x64` 个特征，第一个全连接层是 1024 个节点，表示希望这个网络将这 7*7*64 个特征归类为 1024 个分类。至于为什么第一个连接层是 1024 个节点，是一个经验值的问题，这个节点的个数是可以调整的，后面会讲到全连接层对模型体积的影响，也会再说到这个经验值的问题。那么全连接层是如何分类的，一起来看一下。（剩下内容略……） 41 | 42 | 个人补充：关于 softmax 函数。 43 | 44 | > 我们知道 MNIST 的结果是 0-9，我们的模型可能推测出一张图片是数字 9 的概率是 80%，是数字 8 的概率是 10%，然后其他数字的概率更小，总体概率加起来等于 1。这是一个使用 softmax 回归模型的经典案例。softmax 模型可以用来给不同的对象分配概率。![](https://img-1256179949.cos.ap-shanghai.myqcloud.com/20181105205325.png) 45 | > 46 | > 比如输出结果为[1, 5, 3]： 47 | > 48 | > ![](https://img-1256179949.cos.ap-shanghai.myqcloud.com/20181105205403.png) 49 | > 50 | > softmax 多用于多分类神经网络输出。 51 | 52 | 53 | 54 | (3) [Tensorflow 实现 MNIST 手写数字识别](https://blog.csdn.net/u010858605/article/details/69830657)：通过该文重点体会下神经网络最后是如何做到数字分类的。 -------------------------------------------------------------------------------- /other/resources/学习资源.md: -------------------------------------------------------------------------------- 1 | # 资源 2 | 3 | ## 视频 4 | 5 | - [Video]偏科普入门，莫烦机器学习教程： 6 | - [Video]适合入门，吴恩达机器学习课程：、或 B 站： 7 | - [Video]吴恩达深度学习课程：（中英文字幕） 8 | - [Video]林轩田《机器学习基石》，B 站观看： 9 | - [Video]林轩田《机器学习技法》，B 站观看： 10 | - [Video]李宏毅《一天搞懂深度学习》，B 站观看： [荐] 11 | - [Video]李宏毅_机器学习，B 站观看： 12 | - [Video]李宏毅_深度学习，B 站观看： 13 | - [Video]深度学习计算机视觉课程，李飞飞_斯坦福 CS231n 课程，B 站观看：（中文字幕） 14 | - [Videos]李沐《动手学深度学习》：，书籍预览版：[《动手学深度学习》](https://zh.d2l.ai/)，代码GitHub地址：[d2l-ai/d2l-zh](https://github.com/d2l-ai/d2l-zh) 15 | 16 | 17 | 18 | 台大教授李宏毅的机器学习课程经常被认为是中文开放课程中的首选。李教授的授课风格风趣幽默，通俗易懂，其课程内容中不仅有机器学习、深度学习的基础知识，也会介绍 ML 领域里的各种最新技术。近日，2019 版的课程资料与视频终于上线了。 19 | 20 | - 课程资料链接：http://speech.ee.ntu.edu.tw/~tlkagk/courses_ML19.html 21 | - 课程视频（Bilibili）：https://www.bilibili.com/video/av46561029/ 22 | - YouTube 链接：https://www.youtube.com/playlist?list=PLJV_el3uVTsOK_ZK5L0Iv_EQoL1JefRL4 23 | 24 | 李宏毅（Hung-yi Lee）目前任台湾大学电机工程学系和电机资讯学院的助理教授，他曾于 2012 年获得台湾大学博士学位，并于 2013 年赴麻省理工学院（MIT）计算机科学和人工智能实验室（CSAIL）做访问学者。他的研究方向主要是机器学习（深度学习）和语音识别。此前，他的 2016 版《机器学习》课程视频曾是流行学习资料。 25 | 26 | 27 | 28 | ## GitHub 29 | 30 | - [apachecn/AiLearning](https://github.com/apachecn/AiLearning) 31 | - [DeepLearning-500-questions](https://github.com/scutan90/DeepLearning-500-questions) - 《深度学习500问》 32 | - [AI初学者--（机器学习爱好者）](http://www.ai-start.com/) 33 | - [daily-paper-computer-vision](https://github.com/amusi/daily-paper-computer-vision) 34 | - [机器学习笔记 - feisky]() 35 | - - 李航《统计学习方法》的代码实现。 36 | 37 | ## 书籍 38 | 39 | **1.机器学习** 40 | 41 | - 《写给人类的机器学习》译者：[飞龙](https://github.com/wizardforcel)（等）；原书：[Machine Learning for Humans](https://medium.com/machine-learning-for-humans/) 42 | - 周志华《机器学习》，2016年1月 43 | - Peter Harrington 《机器学习实战》，中文版译者：李锐/李鹏/曲亚东/王斌 ，2013年6月 [[GitHub代码仓库](https://github.com/pbharrin/machinelearninginaction)] 44 | 45 | **2.深度学习** 46 | 47 | - Michael Nielsen[《Neural Networks and Deep Learning》](http://neuralnetworksanddeeplearning.com/index.html)，中文版《神经网络与深度学习》 48 | - 弗朗索瓦•肖莱 《Python深度学习》，中文版译者：张亮，2018年8月 49 | - 张玉宏《深度学习之美：AI时代的数据处理与最佳实践》，2018年6月 50 | - 张平《图解深度学习与神经网络：从张量到TensorFlow实现》，2018年09月 51 | - 李沐、Aston Zhang 等人《动手学深度学习》预览版：[《动手学深度学习》](https://zh.d2l.ai/) 52 | - 邱锡鹏《神经网络与深度学习》：[在线阅读](https://nndl.github.io/) 53 | 54 | **3.深度学习框架** 55 | 56 | - 泽宇/顾思宇 《Tensorflow：实战Google深度学习框架》 57 | - 黄文坚/唐源《TensorFlow实战》 58 | - 廖星宇《深度学习入门之PyTorch》 [[代码仓库](https://github.com/L1aoXingyu/code-of-learn-deep-learning-with-pytorch)] 59 | - 陈云《深度学习框架PyTorch：入门与实践》 [[代码仓库](https://github.com/chenyuntc/pytorch-book)] 60 | 61 | 62 | 63 | ## 博客&专栏 64 | 65 | - [Poll的笔记 - 博客园](http://www.cnblogs.com/maybe2030) 66 | - [Charlotte77 - 博客园](https://www.cnblogs.com/charlotte77/) 67 | - [刘建平Pinard - 博客园](https://www.cnblogs.com/pinard/) 68 | 69 | 70 | 71 | # 学习路线和资源 72 | 73 | - [重磅 | 完备的 AI 学习路线，最详细的资源整理！](https://mp.weixin.qq.com/s/3_KT7WZvvTGts3wprlpFPA) --------------------------------------------------------------------------------