├── .gitignore ├── CHANGELOG.md ├── LICENSE ├── README.EN.md ├── README.md ├── docs ├── imgs │ ├── acc.png │ └── loss.png ├── index.md ├── overall-structure.md ├── 数据集.md ├── 求解器.md ├── 示例.md ├── 网络层.md └── 网络模型.md ├── examples ├── 2_nn_mnist.py ├── 3_nn_cifar10.py ├── 3_nn_iris.py ├── 3_nn_mnist.py ├── 3_nn_orl.py ├── lenet5_mnist.py └── nin_cifar10.py ├── imgs ├── logo.png └── logo2.png ├── mkdocs.yml ├── pynet ├── .gitignore ├── __init__.py ├── models │ ├── AlexNet.py │ ├── FCNet.py │ ├── LeNet5.py │ ├── NIN.py │ ├── Net.py │ ├── ThreeLayerNet.py │ ├── TwoLayerNet.py │ ├── __init__.py │ ├── __pycache__ │ │ ├── AlexNet.cpython-37.pyc │ │ ├── NIN.cpython-37.pyc │ │ ├── Net.cpython-37.pyc │ │ ├── ThreeLayerNet.cpython-37.pyc │ │ ├── TwoLayerNet.cpython-37.pyc │ │ ├── __init__.cpython-37.pyc │ │ └── utils.cpython-37.pyc │ └── utils.py ├── nn │ ├── BN.py │ ├── Conv2d.py │ ├── CrossEntropyLoss.py │ ├── Dropout.py │ ├── Dropout2d.py │ ├── FC.py │ ├── GAP.py │ ├── Layer.py │ ├── MaxPool.py │ ├── ReLU.py │ ├── Softmax.py │ ├── __init__.py │ ├── __pycache__ │ │ ├── CrossEntropyLoss.cpython-37.pyc │ │ ├── Dropout.cpython-37.pyc │ │ ├── Dropout2d.cpython-37.pyc │ │ ├── FC.cpython-37.pyc │ │ ├── GAP.cpython-37.pyc │ │ ├── Layer.cpython-37.pyc │ │ ├── ReLU.cpython-37.pyc │ │ ├── __init__.cpython-37.pyc │ │ ├── im2row.cpython-37.pyc │ │ ├── pool2row.cpython-37.pyc │ │ └── utils.cpython-37.pyc │ ├── im2row.py │ ├── pool2row.py │ └── utils.py ├── optim │ ├── __init__.py │ ├── __pycache__ │ │ ├── __init__.cpython-37.pyc │ │ └── sgd.cpython-37.pyc │ ├── lr_scheduler │ │ ├── LRScheduler.py │ │ ├── StepLR.py │ │ └── __init__.py │ ├── optimizer.py │ └── sgd.py ├── requirements.txt ├── solver.py └── vision │ ├── __init__.py │ ├── __pycache__ │ └── __init__.cpython-37.pyc │ ├── accuracy.py │ ├── data │ ├── __init__.py │ ├── __pycache__ │ │ ├── __init__.cpython-37.pyc │ │ ├── cifar.cpython-37.pyc │ │ ├── iris.cpython-37.pyc │ │ ├── mnist.cpython-37.pyc │ │ ├── orl.cpython-37.pyc │ │ ├── utils.cpython-37.pyc │ │ └── xor.cpython-37.pyc │ ├── cifar.py │ ├── iris.py │ ├── mnist.py │ ├── orl.py │ ├── utils.py │ └── xor.py │ └── draw.py └── requirements.txt /.gitignore: -------------------------------------------------------------------------------- 1 | model/ 2 | 3 | plt/ 4 | 5 | data/__pycache__/ 6 | src/__pycache__/ 7 | nn/__pycache__/ 8 | .idea/ 9 | -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | # [](https://github.com/zjZSTU/PyNet/compare/v0.4.0...v) (2020-05-05) 2 | 3 | 4 | 5 | # [](https://github.com/zjZSTU/PyNet/compare/v0.3.0...v) (2019-09-11) 6 | 7 | 8 | 9 | # [0.3.0](https://github.com/zjZSTU/PyNet/compare/v0.2.1...v0.3.0) (2019-09-11) 10 | 11 | 12 | ### Features 13 | 14 | * **model:** 实现自定义层数和大小的神经网络FCNet ([c80706d](https://github.com/zjZSTU/PyNet/commit/c80706d)) 15 | * **nn:** 实现批量归一化层，在FCNet上实现BN，使用3层网络进行测试 ([c31a014](https://github.com/zjZSTU/PyNet/commit/c31a014)) 16 | 17 | 18 | 19 | ## [0.2.1](https://github.com/zjZSTU/PyNet/compare/v0.2.0...v0.2.1) (2019-07-03) 20 | 21 | 22 | 23 | # [0.2.0](https://github.com/zjZSTU/PyNet/compare/v0.1.0...v0.2.0) (2019-07-02) 24 | 25 | 26 | ### Bug Fixes 27 | 28 | * **optim:** 修复学习率调度器失败 ([c483601](https://github.com/zjZSTU/PyNet/commit/c483601)) 29 | 30 | 31 | 32 | # [0.1.0](https://github.com/zjZSTU/PyNet/compare/dba4057...v0.1.0) (2019-06-29) 33 | 34 | 35 | ### Bug Fixes 36 | 37 | * **LeNet5:** 批量大小为1时LeNet5前向操作中C5->F6步骤出错 ([48b5a46](https://github.com/zjZSTU/PyNet/commit/48b5a46)) 38 | * **MaxPool:** 变量名出错 input->inputs ([af183c2](https://github.com/zjZSTU/PyNet/commit/af183c2)) 39 | * **models:** 修复加载错误模型 ([d8c35c8](https://github.com/zjZSTU/PyNet/commit/d8c35c8)) 40 | 41 | 42 | ### Features 43 | 44 | * **data:** vision包下新增data包，用于数据加载 ([5d82ac0](https://github.com/zjZSTU/PyNet/commit/5d82ac0)) 45 | * **data:** 修改.gitignore，添加data目录 ([a243e20](https://github.com/zjZSTU/PyNet/commit/a243e20)) 46 | * **dropout:** 实现随机失活操作，修改ThreeNet和LeNet5实现随机失活操作 ([eaef5d9](https://github.com/zjZSTU/PyNet/commit/eaef5d9)) 47 | * **GAP:** 实现全局平均池化层操作 ([cecfe69](https://github.com/zjZSTU/PyNet/commit/cecfe69)) 48 | * **layer:** 添加softmax评分类实现 ([ab85f10](https://github.com/zjZSTU/PyNet/commit/ab85f10)) 49 | * **layers:** 添加动量更新 ([b8e1e60](https://github.com/zjZSTU/PyNet/commit/b8e1e60)) 50 | * **LeNet-5:** 取出卷积层失活操作 ([57b8e5e](https://github.com/zjZSTU/PyNet/commit/57b8e5e)) 51 | * **nets:** LeNet-5模型添加动量更新功能 ([b9cac97](https://github.com/zjZSTU/PyNet/commit/b9cac97)) 52 | * **NIN:** 实现NIN类 ([f06b1b6](https://github.com/zjZSTU/PyNet/commit/f06b1b6)) 53 | * **nn:** 实现2层神经网络类和LeNet-5 ([a846c20](https://github.com/zjZSTU/PyNet/commit/a846c20)) 54 | * **nn:** 实现conv class、maxpool class、fc class、relu class和CrossEntropyLoss class ([dba4057](https://github.com/zjZSTU/PyNet/commit/dba4057)) 55 | * **nn:** 实现Nesterov加速梯度 ([580d0a6](https://github.com/zjZSTU/PyNet/commit/580d0a6)) 56 | * **nn:** 实现三层神经网络的随机失活，分离前向训练和预测 ([60f6f65](https://github.com/zjZSTU/PyNet/commit/60f6f65)) 57 | * **refactor:** 重构网络类和模型设计，参考Pytorch/models结构，使用面向对象方法 ([5f0ecec](https://github.com/zjZSTU/PyNet/commit/5f0ecec)) 58 | * **src:** 网络测试 ([05fb51d](https://github.com/zjZSTU/PyNet/commit/05fb51d)) 59 | 60 | 61 | 62 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "[]" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright [yyyy] [name of copyright owner] 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | -------------------------------------------------------------------------------- /README.EN.md: -------------------------------------------------------------------------------- 1 | # PyNet 2 | 3 | ![](./imgs/logo.png) 4 | 5 | [![Documentation Status](https://readthedocs.org/projects/zj-pynet/badge/?version=latest)](https://zj-pynet.readthedocs.io/zh_CN/latest/?badge=latest) [![standard-readme compliant](https://img.shields.io/badge/standard--readme-OK-green.svg?style=flat-square)](https://github.com/RichardLitt/standard-readme) [![Conventional Commits](https://img.shields.io/badge/Conventional%20Commits-1.0.0-yellow.svg)](https://conventionalcommits.org) [![Commitizen friendly](https://img.shields.io/badge/commitizen-friendly-brightgreen.svg)](http://commitizen.github.io/cz-cli/) 6 | 7 | [中文版本（Chinese version）](./README.md) 8 | 9 | > Numpy-based deep learning library 10 | 11 | Implementation of deep learning based on numpy, modular design guarantees easy implementation of the model, which is suitable for the introduction of junior researchers in deep learning. 12 | 13 | ## Table of Contents 14 | 15 | - [PyNet](#pynet) 16 | - [Table of Contents](#table-of-contents) 17 | - [Background](#background) 18 | - [Badge](#badge) 19 | - [Install](#install) 20 | - [Usage](#usage) 21 | - [CHANGELOG](#changelog) 22 | - [TODO](#todo) 23 | - [Maintainers](#maintainers) 24 | - [Thanks](#thanks) 25 | - [Contributing](#contributing) 26 | - [License](#license) 27 | 28 | ## Background 29 | 30 | Systematic learning convolution neural network has been nearly half a year.Using librarys such as pytorch can't understand the implementation in depth. So I plan to complete a deep learning framework from scratch.The initial implementation will refer to the operation of cs231n, and then we will implement it in the form of computational graphs. I hope this project can improve my programming ability and help others at the same time. 31 | 32 | ## Badge 33 | 34 | If you use PyNet, add the following Badge 35 | 36 | [![pynet](https://img.shields.io/badge/pynet-ok-brightgreen)](https://github.com/zjZSTU/PyNet) 37 | 38 | To add in Markdown format, use this code: 39 | 40 | ``` 41 | [![pynet](https://img.shields.io/badge/pynet-ok-brightgreen)](https://github.com/zjZSTU/PyNet) 42 | ``` 43 | 44 | ## Install 45 | 46 | PyNet need the following prerequisites 47 | 48 | * python3.x 49 | * numpy 50 | * opencv3.x 51 | 52 | ## Usage 53 | 54 | Refer to the sample code under the [example](https://github.com/zjZSTU/PyNet/tree/master/examples) folder 55 | 56 | Full version reference [releases](https://github.com/zjZSTU/PyNet/releases) 57 | 58 | Realized Network Model（Located in [pynet/models](https://github.com/zjZSTU/PyNet/tree/master/pynet/models) folder）： 59 | 60 | * 2-Layer Neural Network 61 | * 3-Layer Neural Network 62 | * LeNet-5 63 | * AlexNet 64 | * NIN 65 | 66 | Realized Network Layer（Located in [pynet/nn](https://github.com/zjZSTU/PyNet/tree/master/pynet/nn) folder）： 67 | 68 | * Convolution Layer (Conv2d) 69 | * Fully-Connected Layer (FC) 70 | * Max-Pooling layer (MaxPool) 71 | * ReLU Layer (ReLU) 72 | * Random Dropout Layer (Dropout/Dropout2d) 73 | * Softmax 74 | * Cross Entropy Loss 75 | * Gloabl Average Pool (GAP) 76 | 77 | ## CHANGELOG 78 | 79 | see the [CHANGELOG](./CHANGELOG.md) on this repository. 80 | 81 | ## TODO 82 | 83 | * Realization of batch normalization 84 | * Realization of Computational Graph 85 | 86 | ## Maintainers 87 | 88 | * zhujian - *Initial work* - [zjZSTU](https://github.com/zjZSTU) 89 | 90 | ## Thanks 91 | 92 | * [cs231n](http://cs231n.github.io/) 93 | * [PyTorch](https://pytorch.org/) 94 | 95 | ## Contributing 96 | 97 | Anyone's participation is welcome! Open an [issue](https://github.com/zjZSTU/PyNet/issues) or submit PRs. 98 | 99 | Small note: 100 | 101 | * Git submission specifications should be complied with [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0-beta.4/) 102 | * If versioned, please conform to the [Semantic Versioning 2.0.0](https://semver.org) specification 103 | * If editing the README, please conform to the [standard-readme](https://github.com/RichardLitt/standard-readme) specification. 104 | 105 | ## License 106 | 107 | [Apache License 2.0](LICENSE) © 2019 zjZSTU 108 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 2 | # PyNet 3 | 4 | ![](./imgs/logo.png) 5 | 6 | [![Documentation Status](https://readthedocs.org/projects/zj-pynet/badge/?version=latest)](https://zj-pynet.readthedocs.io/zh_CN/latest/?badge=latest) [![standard-readme compliant](https://img.shields.io/badge/standard--readme-OK-green.svg?style=flat-square)](https://github.com/RichardLitt/standard-readme) [![Conventional Commits](https://img.shields.io/badge/Conventional%20Commits-1.0.0-yellow.svg)](https://conventionalcommits.org) [![Commitizen friendly](https://img.shields.io/badge/commitizen-friendly-brightgreen.svg)](http://commitizen.github.io/cz-cli/) 7 | 8 | > 基于Numpy的深度学习库 9 | 10 | 基于`Numpy`的深度学习实现，模块化设计保证模型的轻松实现，适用于深度学习初级研究人员的入门 11 | 12 | [使用示例](https://zj-pynet.readthedocs.io/zh_CN/latest/%E7%A4%BA%E4%BE%8B/) 13 | 14 | ***这个项目不再继续了。最开始的想法很简单，就是要深入学习卷积神经网络。不过随着项目的深入会发现确实有很多的约束。不管这么样，对于刚刚开始入门深度学习的童鞋们来说，看看这里的源码还是很有帮助的*** 15 | 16 | ## 内容列表 17 | 18 | - [PyNet](#pynet) 19 | - [内容列表](#内容列表) 20 | - [背景](#背景) 21 | - [徽章](#徽章) 22 | - [安装](#安装) 23 | - [用法](#用法) 24 | - [版本更新日志](#版本更新日志) 25 | - [待办事项](#待办事项) 26 | - [主要维护人员](#主要维护人员) 27 | - [致谢](#致谢) 28 | - [参与贡献方式](#参与贡献方式) 29 | - [许可证](#许可证) 30 | 31 | ## 背景 32 | 33 | 系统性的学习卷积神经网络也快半年了，使用`pytorch`等库不能很好的深入理解实现，所以打算从头完成一个深度学习框架。最开始的实现会参考`cs231n`的作业，之后会以计算图的方式实现。希望这个项目能够切实提高自己的编程能力，同时也能够帮助到其他人 34 | 35 | ## 徽章 36 | 37 | 如果你使用了`PyNet`，请添加以下徽章 38 | 39 | [![pynet](https://img.shields.io/badge/pynet-ok-brightgreen)](https://github.com/zjZSTU/PyNet) 40 | 41 | Markdown格式代码如下： 42 | 43 | ``` 44 | [![pynet](https://img.shields.io/badge/pynet-ok-brightgreen)](https://github.com/zjZSTU/PyNet) 45 | ``` 46 | 47 | ## 安装 48 | 49 | ``` 50 | # 文档工具依赖 51 | $ pip install -r requiremens.txt 52 | # PyNet库依赖 53 | $ cd pynet 54 | $ pip install -r requirements.txt 55 | ``` 56 | 57 | ## 用法 58 | 59 | 有两种文档使用方式 60 | 61 | 1. 在线浏览文档：[PyNet](https://zj-pynet.readthedocs.io/zh_CN/latest/) 62 | 63 | 2. 本地浏览文档，实现如下： 64 | 65 | ``` 66 | $ git clone https://zj-pynet.readthedocs.io/zh_CN/latest/ 67 | $ cd PyNet 68 | $ mkdocs serve 69 | ``` 70 | 启动本地服务器后即可登录浏览器`localhost:8000` 71 | 72 | ## 版本更新日志 73 | 74 | 请参阅仓库中的[CHANGELOG](./CHANGELOG.md) 75 | 76 | ## 待办事项 77 | 78 | * 计算图实现 79 | 80 | ## 主要维护人员 81 | 82 | * zhujian - *Initial work* - [zjZSTU](https://github.com/zjZSTU) 83 | 84 | ## 致谢 85 | 86 | * [cs231n](http://cs231n.github.io/) 87 | * [PyTorch](https://pytorch.org/) 88 | 89 | ## 参与贡献方式 90 | 91 | 欢迎任何人的参与！打开[issue](https://github.com/zjZSTU/PyNet/issues)或提交合并请求。 92 | 93 | 注意: 94 | 95 | * `git`提交请遵守[Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0-beta.4/) 96 | * 如果进行版本化，请遵守[Semantic Versioning 2.0.0](https://semver.org)规范 97 | * 如果修改README，请遵守[standard-readme](https://github.com/RichardLitt/standard-readme)规范 98 | 99 | ## 许可证 100 | 101 | [Apache License 2.0](LICENSE) © 2019 zjZSTU -------------------------------------------------------------------------------- /docs/imgs/acc.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deep-learning-algorithm/PyNet/354c7ee88a712a1f5069d58a0be4a6cbfaeab861/docs/imgs/acc.png -------------------------------------------------------------------------------- /docs/imgs/loss.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deep-learning-algorithm/PyNet/354c7ee88a712a1f5069d58a0be4a6cbfaeab861/docs/imgs/loss.png -------------------------------------------------------------------------------- /docs/index.md: -------------------------------------------------------------------------------- 1 | 2 | # PyNet 3 | 4 | `PyNet`是一个纯`Numpy`实现的卷积神经网络库。最初的实现目的就是为了更深入的学习和理解卷积神经网络的原理和实现，其结构简单，示例完整，适用于入门 5 | 6 | ## 文档实现流程 7 | 8 | 1. [介绍`PyNet`整体实现架构](./overall-structure.md) 9 | 2. [介绍实现的网络层](./网络层.md) 10 | 3. [介绍实现的网络模型](./网络模型.md) 11 | 4. [介绍实现的求解器](./求解器.md) 12 | 5. [介绍使用的数据集](./数据集.md) 13 | 6. [介绍实现的示例](./示例.md) 14 | 15 | ## 实现原理 16 | 17 | 对于卷积神经网络的原理，参考 18 | 19 | * [机器学习](https://zj-image-processing.readthedocs.io/zh_CN/latest/algorithm/machine-learning/) 20 | * [深度学习](https://zj-image-processing.readthedocs.io/zh_CN/latest/algorithm/deep-learning/) 21 | 22 | ## 致谢 23 | 24 | 在`PyNet`最开始的时候，参考了[PyTorch](https://pytorch.org/)的实现思路；在后续的发展过程中，进一步参考了[CS231n](http://cs231n.github.io/)课程作业中使用的架构 -------------------------------------------------------------------------------- /docs/overall-structure.md: -------------------------------------------------------------------------------- 1 | 2 | # 整体架构 3 | 4 | ## 整个仓库 5 | 6 | ``` 7 | ├── CHANGELOG.md 8 | ├── docs 9 | ├── examples 10 | ├── imgs 11 | │   ├── logo2.png 12 | │   └── logo.png 13 | ├── LICENSE 14 | ├── mkdocs.yml 15 | ├── pynet 16 | ├── README.EN.md 17 | ├── README.md 18 | └── requirements.txt 19 | ``` 20 | 21 | * `PyNet`实现代码位于`pynet` 22 | * `PyNet`示例代码位于`examples` 23 | * 说明文档位于`docs` 24 | 25 | ## pynet 26 | 27 | ``` 28 | ├── __init__.py 29 | ├── models 30 | ├── nn 31 | ├── optim 32 | ├── __pycache__ 33 | ├── requirements.txt 34 | ├── solver.py 35 | └── vision 36 | ``` 37 | 38 | * 层的实现位于`nn`，比如`Conv2d/MaxPool/Dropout/BN/ReLU/...` 39 | * 模型的实现位于`models`，比如`FCNet/LeNet5/AlexNet/NIN` 40 | * 优化器和学习率调度器的实现位于`optim`，比如`SGD/StepLR` 41 | * 数据加载和训练结果绘制的实现位于`vision` 42 | * 求解器实现位于`solver.py` 43 | 44 | ## examples 45 | 46 | ``` 47 | ├── 2_nn_mnist.py 48 | ├── 3_nn_cifar10.py 49 | ├── 3_nn_iris.py 50 | ├── 3_nn_mnist.py 51 | ├── 3_nn_orl.py 52 | ├── lenet5_mnist.py 53 | └── nin_cifar10.py 54 | ``` 55 | 56 | 测试了`4`个网络模型：`2`层`/3`层神经网络、`LeNet5`和`NIN`；以及`4`个数据集：`MNIST、CIFAR10、iris`和`ORL` 57 | 58 | *`Note`：使用`NIN`的使用就很慢了* -------------------------------------------------------------------------------- /docs/数据集.md: -------------------------------------------------------------------------------- 1 | 2 | # 数据集 3 | 4 | 在`PyNet`中提供了多个数据集的预处理。包含 5 | 6 | 1. `xor` 7 | 2. `mnist` 8 | 3. `cifar10` 9 | 4. `orl` 10 | 5. `iris` 11 | 12 | 相应的实现代码位于`pynet/vision/data`目录 -------------------------------------------------------------------------------- /docs/求解器.md: -------------------------------------------------------------------------------- 1 | 2 | # Sovler 3 | 4 | 模型的整个训练在`solver.py`文件中实现 5 | 6 | ``` 7 | class Solver(object): 8 | 9 | def __init__(self, model, data, criterion, optimizer, **kwargs): 10 | 11 | def _reset(self): 12 | 13 | def _step(self, X_batch, y_batch): 14 | 15 | def check_accuracy(self, X, y, num_samples=None, batch_size=8): 16 | 17 | def train(self): 18 | ``` 19 | 20 | ## __init__ 21 | 22 | * 必选参数 23 | * `model`：网络模型 24 | * `data`：包含了训练和测试数据集 25 | * `criterion`：评价函数 26 | * `optimizer`：优化器 27 | * 可选参数 28 | * `lr_scheduler`：学习率调度器，默认为`None` 29 | * `batch_size`：单次处理大小，默认为`8` 30 | * `num_epochs`：迭代周期次数，默认为`10` 31 | * `reg`：正则化因子，默认为`1e-3` 32 | * `print_every`：每隔多少论打印一次信息，默认为`1` -------------------------------------------------------------------------------- /docs/示例.md: -------------------------------------------------------------------------------- 1 | 2 | # 使用示例 3 | 4 | 使用`PyNet`非常简单，类似`PyTorch`实现方式 5 | 6 | * 第一步：创建数据集并进行预处理 7 | * 第二步：创建网络模型对象 8 | * 第三步：创建评价函数对象 9 | * 第四步： 创建优化器对象 10 | * 第五步：创建求解器对象，设置训练参数，训练 11 | * 第六步：打印和绘制训练结果 12 | 13 | 以`examples/2_nn_mnist.py`为例 14 | 15 | ## 头文件加载 16 | 17 | ``` 18 | import pynet 19 | import pynet.models as models 20 | import pynet.optim as optim 21 | import pynet.nn as nn 22 | from pynet.vision.data import mnist 23 | from pynet.vision import Draw 24 | ``` 25 | 26 | ## 创建数据集并进行预处理 27 | 28 | ``` 29 | data_path = '~/data/decompress_mnist' 30 | 31 | x_train, x_test, y_train, y_test = mnist.load_mnist(data_path, shuffle=True, is_flatten=True) 32 | 33 | x_train = x_train / 255 - 0.5 34 | x_test = x_test / 255 - 0.5 35 | 36 | data = { 37 | 'X_train': x_train, 38 | 'y_train': y_train, 39 | 'X_val': x_test, 40 | 'y_val': y_test 41 | } 42 | ``` 43 | 44 | ## 创建网络模型对象 45 | 46 | ``` 47 | model = models.TwoLayerNet(num_in=784, num_hidden=200, num_out=10) 48 | ``` 49 | 50 | ## 创建评价函数对象 51 | 52 | ``` 53 | criterion = nn.CrossEntropyLoss() 54 | ``` 55 | 56 | ## 创建优化器对象 57 | 58 | ``` 59 | optimizer = optim.SGD(model.params) 60 | ``` 61 | 62 | ## 创建求解器对象，设置训练参数，训练 63 | 64 | ``` 65 | solver = pynet.Solver(model, data, criterion, optimizer, batch_size=128, num_epochs=10) 66 | solver.train() 67 | ``` 68 | 69 | ## 打印和绘制训练结果 70 | 71 | ``` 72 | plt = Draw() 73 | plt(solver.loss_history) 74 | plt.multi_plot((solver.train_acc_history, solver.val_acc_history), ('train', 'val'), 75 | title='准确率', xlabel='迭代/次', ylabel='准确率') 76 | print('best_train_acc: %f; best_val_acc: %f' % (solver.best_train_acc, solver.best_val_acc)) 77 | ``` 78 | 79 | 训练日志如下 80 | 81 | ``` 82 | $ python 2_nn_mnist.py 83 | epoch: 1 time: 1.08 loss: 0.684346 84 | train acc: 0.8880; val_acc: 0.8914 85 | epoch: 2 time: 1.00 loss: 0.337981 86 | train acc: 0.9131; val_acc: 0.9162 87 | epoch: 3 time: 1.05 loss: 0.284188 88 | train acc: 0.9320; val_acc: 0.9320 89 | epoch: 4 time: 1.01 loss: 0.249591 90 | train acc: 0.9448; val_acc: 0.9439 91 | epoch: 5 time: 0.99 loss: 0.227722 92 | train acc: 0.9528; val_acc: 0.9493 93 | epoch: 6 time: 0.99 loss: 0.213616 94 | train acc: 0.9592; val_acc: 0.9547 95 | epoch: 7 time: 1.00 loss: 0.204579 96 | train acc: 0.9640; val_acc: 0.9594 97 | epoch: 8 time: 1.04 loss: 0.198835 98 | train acc: 0.9685; val_acc: 0.9642 99 | epoch: 9 time: 1.03 loss: 0.195210 100 | train acc: 0.9715; val_acc: 0.9669 101 | epoch: 10 time: 1.01 loss: 0.193020 102 | train acc: 0.9738; val_acc: 0.9684 103 | best_train_acc: 0.973767; best_val_acc: 0.968400 104 | ``` 105 | 106 | ![](./imgs/loss.png) 107 | 108 | ![](./imgs/acc.png) -------------------------------------------------------------------------------- /docs/网络层.md: -------------------------------------------------------------------------------- 1 | 2 | # 网络层 3 | 4 | 当前实现了以下网络层： 5 | 6 | * 卷积层 7 | * 有权重参数 8 | * 卷积层（`Conv2d`） 9 | * 全连接层（`FC`） 10 | * 无权重参数 11 | * 批量归一化层（`BN`） 12 | * 随机失活层（`Dropout`） 13 | * 全局平均池化层（`GAP`） 14 | * 最大池化层（`MaxPool`） 15 | * 激活函数 16 | * 整流线性函数（`ReLU`） 17 | * 损失函数 18 | * 交叉熵损失（`CrossEntropyLoss`） 19 | * 评分函数 20 | * `Softmax` -------------------------------------------------------------------------------- /docs/网络模型.md: -------------------------------------------------------------------------------- 1 | 2 | # 网络模型 3 | 4 | 当前实现了以下网络模型： 5 | 6 | * 神经网络 7 | * 二层神经网络（`TwoLayerNet`） 8 | * 三层神经网络（`ThreeLayerNet`） 9 | * 神经网络（`FCNet`，根据输入参数设置网络层数） 10 | * 卷积神经网络 11 | * `LeNet5` 12 | * `AlexNet` 13 | * `NIN` 14 | 15 | 在每个网络模型中 16 | 17 | 1. 实现了前向计算（`forward`）和反向梯度计算（`backward`） 18 | 2. 对于添加了随机失活的模型，额外设置了`train()/eval()` -------------------------------------------------------------------------------- /examples/2_nn_mnist.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-6-30 下午3:13 4 | # @Author : zj 5 | 6 | import pynet 7 | import pynet.models as models 8 | import pynet.optim as optim 9 | import pynet.nn as nn 10 | from pynet.vision.data import mnist 11 | from pynet.vision import Draw 12 | 13 | data_path = '~/data/decompress_mnist' 14 | 15 | if __name__ == '__main__': 16 | x_train, x_test, y_train, y_test = mnist.load_mnist(data_path, shuffle=True, is_flatten=True) 17 | 18 | x_train = x_train / 255 - 0.5 19 | x_test = x_test / 255 - 0.5 20 | 21 | data = { 22 | 'X_train': x_train, 23 | 'y_train': y_train, 24 | 'X_val': x_test, 25 | 'y_val': y_test 26 | } 27 | 28 | model = models.TwoLayerNet(num_in=784, num_hidden=200, num_out=10) 29 | criterion = nn.CrossEntropyLoss() 30 | optimizer = optim.SGD(model.params) 31 | 32 | solver = pynet.Solver(model, data, criterion, optimizer, batch_size=128, num_epochs=10) 33 | solver.train() 34 | 35 | plt = Draw() 36 | plt(solver.loss_history) 37 | plt.multi_plot((solver.train_acc_history, solver.val_acc_history), ('train', 'val'), 38 | title='准确率', xlabel='迭代/次', ylabel='准确率', save_path='acc.png') 39 | print('best_train_acc: %f; best_val_acc: %f' % (solver.best_train_acc, solver.best_val_acc)) 40 | -------------------------------------------------------------------------------- /examples/3_nn_cifar10.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-7-1 下午6:53 4 | # @Author : zj 5 | 6 | import numpy as np 7 | import pynet.models as models 8 | import pynet 9 | import pynet.optim as optim 10 | from pynet.vision.data import cifar 11 | import pynet.nn as nn 12 | from pynet.vision import Draw 13 | 14 | data_path = '~/data/cifar_10/cifar-10-batches-py' 15 | 16 | if __name__ == '__main__': 17 | data_dict = cifar.get_CIFAR10_data(data_path) 18 | 19 | model = models.FCNet([2000, 800], input_dim=3072, num_classes=10, normalization='batchnorm') 20 | criterion = nn.CrossEntropyLoss() 21 | optimizer = optim.SGD(model.params, lr=1e-3) 22 | 23 | solver = pynet.Solver(model, data_dict, criterion, optimizer, batch_size=256, num_epochs=250, 24 | reg=1e-3, print_every=1) 25 | solver.train() 26 | 27 | plt = Draw() 28 | plt(solver.loss_history) 29 | plt.multi_plot((solver.train_acc_history, solver.val_acc_history), ('train', 'val'), 30 | title='准确率', xlabel='迭代/次', ylabel='准确率', save_path='acc.png') 31 | print('best_train_acc: %f; best_val_acc: %f' % (solver.best_train_acc, solver.best_val_acc)) 32 | -------------------------------------------------------------------------------- /examples/3_nn_iris.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-7-1 下午4:39 4 | # @Author : zj 5 | 6 | 7 | import numpy as np 8 | import pynet 9 | import pynet.optim as optim 10 | import pynet.models as models 11 | import pynet.nn as nn 12 | from pynet.vision.data import iris 13 | from pynet.vision import Draw 14 | 15 | data_path = '~/data/iris-species/Iris.csv' 16 | 17 | if __name__ == '__main__': 18 | x_train, x_test, y_train, y_test = iris.load_iris(data_path, shuffle=True, tsize=0.8) 19 | 20 | mean = np.mean(x_train, axis=0, keepdims=True) 21 | x_train -= mean 22 | x_test -= mean 23 | 24 | data = { 25 | 'X_train': x_train, 26 | 'y_train': y_train, 27 | 'X_val': x_test, 28 | 'y_val': y_test 29 | } 30 | 31 | model = models.ThreeLayerNet(num_in=4, num_h1=40, num_h2=20, num_out=3, dropout=0.5) 32 | criterion = nn.CrossEntropyLoss() 33 | optimizer = optim.SGD(model.params, lr=1e-3, momentum=0.5, nesterov=True) 34 | 35 | solver = pynet.Solver(model, data, criterion, optimizer, batch_size=120, num_epochs=50000, 36 | reg=1e-3, print_every=500) 37 | solver.train() 38 | 39 | plt = Draw() 40 | plt(solver.loss_history) 41 | plt.multi_plot((solver.train_acc_history, solver.val_acc_history), ('train', 'val'), 42 | title='准确率', xlabel='迭代/次', ylabel='准确率', save_path='acc.png') 43 | print('best_train_acc: %f; best_val_acc: %f' % (solver.best_train_acc, solver.best_val_acc)) 44 | -------------------------------------------------------------------------------- /examples/3_nn_mnist.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-6-30 下午4:26 4 | # @Author : zj 5 | 6 | import pynet 7 | import pynet.models as models 8 | import pynet.optim as optim 9 | import pynet.nn as nn 10 | from pynet.vision.data import mnist 11 | from pynet.vision import Draw 12 | 13 | data_path = '~/data/decompress_mnist' 14 | 15 | if __name__ == '__main__': 16 | x_train, x_test, y_train, y_test = mnist.load_mnist(data_path, shuffle=True, is_flatten=True) 17 | 18 | x_train = x_train / 255 - 0.5 19 | x_test = x_test / 255 - 0.5 20 | 21 | data = { 22 | 'X_train': x_train, 23 | 'y_train': y_train, 24 | 'X_val': x_test, 25 | 'y_val': y_test 26 | } 27 | 28 | model = models.ThreeLayerNet(num_in=784, num_h1=1200, num_h2=200, num_out=10, dropout=0.5) 29 | criterion = nn.CrossEntropyLoss() 30 | optimizer = optim.SGD(model.params, lr=1e-3) 31 | 32 | solver = pynet.Solver(model, data, criterion, optimizer, batch_size=256, num_epochs=10, print_every=1, reg=1e-3) 33 | solver.train() 34 | 35 | plt = Draw() 36 | plt(solver.loss_history) 37 | plt.multi_plot((solver.train_acc_history, solver.val_acc_history), ('train', 'val'), 38 | title='准确率', xlabel='迭代/次', ylabel='准确率', save_path='acc.png') 39 | print('best_train_acc: %f; best_val_acc: %f' % (solver.best_train_acc, solver.best_val_acc)) 40 | -------------------------------------------------------------------------------- /examples/3_nn_orl.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-7-1 下午6:40 4 | # @Author : zj 5 | 6 | import numpy as np 7 | import pynet 8 | import pynet.models as models 9 | import pynet.nn as nn 10 | import pynet.optim as optim 11 | from pynet.vision.data import orl 12 | from pynet.vision import Draw 13 | 14 | data_path = '~/data/att_faces_png' 15 | 16 | if __name__ == '__main__': 17 | x_train, x_test, y_train, y_test = orl.load_orl(data_path, shuffle=True) 18 | 19 | x_train = x_train / 255 - 0.5 20 | x_test = x_test / 255 - 0.5 21 | 22 | data = { 23 | 'X_train': x_train, 24 | 'y_train': y_train, 25 | 'X_val': x_test, 26 | 'y_val': y_test 27 | } 28 | 29 | model = models.ThreeLayerNet(num_in=644, num_h1=2000, num_h2=800, num_out=40) 30 | criterion = nn.CrossEntropyLoss() 31 | optimizer = optim.SGD(model.params, lr=2e-2) 32 | 33 | solver = pynet.Solver(model, data, criterion, optimizer, batch_size=4, num_epochs=100, 34 | reg=1e-3, print_every=1) 35 | solver.train() 36 | 37 | plt = Draw() 38 | plt(solver.loss_history) 39 | plt.multi_plot((solver.train_acc_history, solver.val_acc_history), ('train', 'val'), 40 | title='准确率', xlabel='迭代/次', ylabel='准确率', save_path='acc.png') 41 | print('best_train_acc: %f; best_val_acc: %f' % (solver.best_train_acc, solver.best_val_acc)) 42 | -------------------------------------------------------------------------------- /examples/lenet5_mnist.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-7-2 上午11:05 4 | # @Author : zj 5 | 6 | import pynet 7 | import pynet.models as models 8 | import pynet.optim as optim 9 | import pynet.nn as nn 10 | from pynet.vision.data import mnist 11 | from pynet.vision import Draw 12 | 13 | data_path = '~/data/decompress_mnist' 14 | 15 | if __name__ == '__main__': 16 | x_train, x_test, y_train, y_test = mnist.load_mnist(data_path, dst_size=(32, 32), shuffle=True) 17 | 18 | x_train = x_train / 255 - 0.5 19 | x_test = x_test / 255 - 0.5 20 | 21 | data = { 22 | 'X_train': x_train, 23 | 'y_train': y_train, 24 | 'X_val': x_test, 25 | 'y_val': y_test 26 | } 27 | 28 | model = models.LeNet5(in_channels=1, out_channels=10, dropout=0.5) 29 | criterion = nn.CrossEntropyLoss() 30 | optimizer = optim.SGD(model.params, lr=1e-3, momentum=0.9, nesterov=True) 31 | stepLR = optim.StepLR(optimizer, 5, gamma=0.1) 32 | 33 | solver = pynet.Solver(model, data, criterion, optimizer, 34 | lr_scheduler=stepLR, batch_size=128, num_epochs=10, print_every=1) 35 | solver.train() 36 | 37 | plt = Draw() 38 | plt(solver.loss_history) 39 | plt.multi_plot((solver.train_acc_history, solver.val_acc_history), ('train', 'val'), 40 | title='准确率', xlabel='迭代/次', ylabel='准确率', save_path='acc.png') 41 | print('best_train_acc: %f; best_val_acc: %f' % (solver.best_train_acc, solver.best_val_acc)) 42 | -------------------------------------------------------------------------------- /examples/nin_cifar10.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-6-21 下午2:45 4 | # @Author : zj 5 | 6 | from pynet import nn, models, vision 7 | import pynet.models.utils as utils 8 | from pynet.vision.data import cifar 9 | import numpy as np 10 | import time 11 | 12 | data_path = '~/data/decompress_cifar_10' 13 | 14 | epochs = 100 15 | batch_size = 128 16 | momentum = 0.9 17 | learning_rate = 1e-3 18 | reg = 1e-3 19 | p_h = 0.5 20 | 21 | 22 | def nin_train(): 23 | x_train, x_test, y_train, y_test = cifar.load_cifar10(data_path, shuffle=True) 24 | 25 | # 标准化 26 | x_train = x_train / 255.0 - 0.5 27 | x_test = x_test / 255.0 - 0.5 28 | 29 | net = models.nin(in_channels=3, p_h=p_h) 30 | criterion = nn.CrossEntropyLoss() 31 | 32 | accuracy = vision.Accuracy() 33 | 34 | loss_list = [] 35 | train_list = [] 36 | test_list = [] 37 | best_train_accuracy = 0.995 38 | best_test_accuracy = 0.995 39 | 40 | range_list = np.arange(0, x_train.shape[0] - batch_size, step=batch_size) 41 | for i in range(epochs): 42 | total_loss = 0 43 | num = 0 44 | start = time.time() 45 | for j in range_list: 46 | data = x_train[j:j + batch_size] 47 | labels = y_train[j:j + batch_size] 48 | 49 | scores = net(data) 50 | loss = criterion(scores, labels) 51 | total_loss += loss 52 | num += 1 53 | 54 | grad_out = criterion.backward() 55 | net.backward(grad_out) 56 | net.update(lr=learning_rate, reg=reg) 57 | end = time.time() 58 | print('one epoch need time: %.3f' % (end - start)) 59 | print('epoch: %d loss: %f' % (i + 1, total_loss / num)) 60 | loss_list.append(total_loss / num) 61 | 62 | if (i % 20) == 19: 63 | # # 每隔20次降低学习率 64 | # learning_rate *= 0.5 65 | 66 | train_accuracy = accuracy.compute_v2(x_train, y_train, net, batch_size=batch_size) 67 | test_accuracy = accuracy.compute_v2(x_test, y_test, net, batch_size=batch_size) 68 | train_list.append(train_accuracy) 69 | test_list.append(test_accuracy) 70 | 71 | print(loss_list) 72 | print(train_list) 73 | print(test_list) 74 | if train_accuracy > best_train_accuracy and test_accuracy > best_test_accuracy: 75 | path = 'nin-epochs-%d.pkl' % (i + 1) 76 | utils.save_params(net.get_params(), path=path) 77 | break 78 | 79 | draw = vision.Draw() 80 | draw(loss_list, xlabel='迭代/20次') 81 | draw.multi_plot((train_list, test_list), ('训练集', '测试集'), 82 | title='精度图', xlabel='迭代/20次', ylabel='精度值', save_path='acc.png') 83 | 84 | 85 | if __name__ == '__main__': 86 | start = time.time() 87 | nin_train() 88 | end = time.time() 89 | print('training need time: %.3f' % (end - start)) 90 | -------------------------------------------------------------------------------- /imgs/logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deep-learning-algorithm/PyNet/354c7ee88a712a1f5069d58a0be4a6cbfaeab861/imgs/logo.png -------------------------------------------------------------------------------- /imgs/logo2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deep-learning-algorithm/PyNet/354c7ee88a712a1f5069d58a0be4a6cbfaeab861/imgs/logo2.png -------------------------------------------------------------------------------- /mkdocs.yml: -------------------------------------------------------------------------------- 1 | # 站点名称 2 | site_name: 'PyNet' 3 | # 仓库链接 4 | repo_url: https://github.com/zjZSTU/PyNet.git 5 | # 作者 6 | site_author: 'zhujian' 7 | # 版权信息 8 | copyright: '2019, zhujian' 9 | # 源文件目录 10 | docs_dir: 'docs' 11 | # 生成静态文件目录 12 | site_dir: 'site' 13 | # 额外信息 14 | extra: 15 | # 版本号 16 | version: 0.1.0 17 | # 主题 18 | theme: 19 | # name: 'readthedocs' 20 | # name: 'mkdocs' 21 | name: 'material' 22 | # markdown扩展 23 | markdown_extensions: 24 | - toc: 25 | permalink: true 26 | - pymdownx.arithmatex 27 | 28 | extra_javascript: 29 | - 'https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-MML-AM_CHTML' 30 | # 导航 31 | nav: 32 | - Home: index.md 33 | - 整体架构: overall-structure.md 34 | - 网络层: 网络层.md 35 | - 网络模型: 网络模型.md 36 | - 求解器: 求解器.md 37 | - 数据集: 数据集.md 38 | - 使用示例: 示例.md -------------------------------------------------------------------------------- /pynet/.gitignore: -------------------------------------------------------------------------------- 1 | __pycache__/ 2 | -------------------------------------------------------------------------------- /pynet/__init__.py: -------------------------------------------------------------------------------- 1 | from .solver import Solver 2 | -------------------------------------------------------------------------------- /pynet/models/AlexNet.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-6-20 下午2:22 4 | # @Author : zj 5 | 6 | from pynet import nn 7 | from .Net import Net 8 | from .utils import load_params 9 | 10 | __all__ = ['AlexNet', 'alexnet'] 11 | 12 | model_urls = { 13 | 'alexnet': '' 14 | } 15 | 16 | 17 | class AlexNet(Net): 18 | """ 19 | AlexNet模型 20 | """ 21 | 22 | def __init__(self, in_channels, out_channels, momentum=0, nesterov=False): 23 | super(AlexNet, self).__init__() 24 | self.conv1 = nn.Conv2d(in_channels, 11, 11, 96, stride=4, padding=0, momentum=momentum, nesterov=nesterov) 25 | self.conv2 = nn.Conv2d(96, 5, 5, 256, stride=1, padding=2, momentum=momentum, nesterov=nesterov) 26 | self.conv3 = nn.Conv2d(256, 3, 3, 384, stride=1, padding=1, momentum=momentum, nesterov=nesterov) 27 | self.conv4 = nn.Conv2d(384, 3, 3, 384, stride=1, padding=1, momentum=momentum, nesterov=nesterov) 28 | self.conv5 = nn.Conv2d(384, 3, 3, 256, stride=1, padding=1, momentum=momentum, nesterov=nesterov) 29 | 30 | self.maxPool1 = nn.MaxPool(3, 3, 96, stride=2) 31 | self.maxPool2 = nn.MaxPool(3, 3, 256, stride=2) 32 | self.maxPool3 = nn.MaxPool(3, 3, 256, stride=2) 33 | self.fc1 = nn.FC(9216, 4096, momentum=momentum, nesterov=nesterov) 34 | self.fc2 = nn.FC(4096, 4096, momentum=momentum, nesterov=nesterov) 35 | self.fc3 = nn.FC(4096, out_channels, momentum=momentum, nesterov=nesterov) 36 | 37 | self.relu1 = nn.ReLU() 38 | self.relu2 = nn.ReLU() 39 | self.relu3 = nn.ReLU() 40 | self.relu4 = nn.ReLU() 41 | self.relu5 = nn.ReLU() 42 | self.relu6 = nn.ReLU() 43 | self.relu7 = nn.ReLU() 44 | 45 | self.dropout = nn.Dropout() 46 | 47 | self.U1 = None 48 | self.U2 = None 49 | 50 | def __call__(self, inputs): 51 | return self.forward(inputs) 52 | 53 | def forward(self, inputs): 54 | # inputs.shape = [N, C, H, W] 55 | assert len(inputs.shape) == 4 56 | x = self.relu1(self.conv1(inputs)) 57 | x = self.maxPool1(x) 58 | x = self.relu2(self.conv2(x)) 59 | x = self.maxPool2(x) 60 | x = self.relu3(self.conv3(x)) 61 | x = self.relu4(self.conv4(x)) 62 | x = self.relu5(self.conv5(x)) 63 | x = self.maxPool3(x) 64 | 65 | # (N, C, 1, 1) -> (N, C) 66 | x = x.reshape(x.shape[0], -1) 67 | x = self.relu6(self.fc1(x)) 68 | self.U1 = self.dropout(x.shape, 0.5) 69 | x *= self.U1 70 | x = self.relu7(self.fc2(x)) 71 | self.U2 = self.dropout(x.shape, 0.5) 72 | x *= self.U2 73 | 74 | x = self.fc3(x) 75 | return x 76 | 77 | def backward(self, grad_out): 78 | da10 = self.fc3.backward(grad_out) 79 | da10 *= self.U2 80 | 81 | dz10 = self.relu7.backward(da10) 82 | da9 = self.fc2.backward(dz10) 83 | da9 *= self.U1 84 | 85 | dz9 = self.relu6.backward(da9) 86 | da8 = self.fc1.backward(dz9) 87 | 88 | # [N, C] -> [N, C, 1, 1] 89 | N, C = da8.shape[:2] 90 | dz8 = da8.reshape(N, C, 1, 1) 91 | 92 | da7 = self.maxPool3.backward(dz8) 93 | dz7 = self.relu5.backward(da7) 94 | da6 = self.conv5.backward(dz7) 95 | 96 | dz6 = self.relu4.backward(da6) 97 | da5 = self.conv4.backward(dz6) 98 | 99 | dz5 = self.relu3.backward(da5) 100 | dz4 = self.conv3.backward(dz5) 101 | 102 | da3 = self.maxPool2.backward(dz4) 103 | dz3 = self.relu2.backward(da3) 104 | dz2 = self.conv2.backward(dz3) 105 | 106 | da1 = self.maxPool1.backward(dz2) 107 | dz1 = self.relu1.backward(da1) 108 | self.conv1.backward(dz1) 109 | 110 | def update(self, lr=1e-3, reg=1e-3): 111 | self.fc3.update(learning_rate=lr, regularization_rate=reg) 112 | self.fc2.update(learning_rate=lr, regularization_rate=reg) 113 | self.fc1.update(learning_rate=lr, regularization_rate=reg) 114 | self.conv5.update(learning_rate=lr, regularization_rate=reg) 115 | self.conv4.update(learning_rate=lr, regularization_rate=reg) 116 | self.conv3.update(learning_rate=lr, regularization_rate=reg) 117 | self.conv2.update(learning_rate=lr, regularization_rate=reg) 118 | self.conv1.update(learning_rate=lr, regularization_rate=reg) 119 | 120 | def predict(self, inputs): 121 | # inputs.shape = [N, C, H, W] 122 | assert len(inputs.shape) == 4 123 | x = self.relu1(self.conv1(inputs)) 124 | x = self.maxPool1(x) 125 | x = self.relu2(self.conv2(x)) 126 | x = self.maxPool2(x) 127 | x = self.relu3(self.conv3(x)) 128 | x = self.relu4(self.conv4(x)) 129 | x = self.relu5(self.conv5(x)) 130 | x = self.maxPool3(x) 131 | 132 | # (N, C, 1, 1) -> (N, C) 133 | x = x.reshape(x.shape[0], -1) 134 | x = self.relu6(self.fc1(x)) 135 | x = self.relu7(self.fc1(x)) 136 | 137 | x = self.fc2(x) 138 | return x 139 | 140 | def get_params(self): 141 | pass 142 | 143 | def set_params(self, params): 144 | pass 145 | 146 | 147 | def alexnet(pretrained=False, **kwargs): 148 | """ 149 | 创建模型对象 150 | """ 151 | 152 | model = AlexNet(**kwargs) 153 | if pretrained: 154 | params = load_params(model_urls['alexnet']) 155 | model.set_params(params) 156 | return model 157 | -------------------------------------------------------------------------------- /pynet/models/FCNet.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-7-8 下午1:36 4 | # @Author : zj 5 | 6 | """ 7 | 参考 cs231n assignment2 FullyConnectedNet，实现自定义层数和大小的神经网络 8 | 9 | 网络结构为 10 | 11 | {FC - [batch/layer norm] - RELU - [dropout]} * (L - 1) - FC 12 | """ 13 | 14 | __all__ = ['FCNet'] 15 | 16 | import numpy as np 17 | from pynet import nn 18 | from .Net import Net 19 | 20 | 21 | class FCNet(Net): 22 | """ 23 | 神经网络 24 | """ 25 | 26 | def __init__(self, hidden_dims, input_dim=3 * 32 * 32, num_classes=10, normalization=None, 27 | dropout=1.0, seed=None, weight_scale=1e-2, dtype=np.double): 28 | super(FCNet, self).__init__() 29 | self.hidden_dims = hidden_dims 30 | self.input_dim = input_dim 31 | self.num_classes = num_classes 32 | self.normalization = normalization 33 | self.weight_scale = weight_scale 34 | self.dtype = dtype 35 | 36 | self.use_dropout = dropout != 1 37 | self.num_layers = 1 + len(hidden_dims) 38 | self.relu = nn.ReLU() 39 | 40 | self.fcs = self._get_fcs() 41 | self.params = self._get_params() 42 | self.caches = self._get_caches() 43 | 44 | self.use_dropout = dropout != 1.0 45 | self.dropout_param = {} 46 | if self.use_dropout: 47 | self.dropout = nn.Dropout() 48 | 49 | self.dropout_param['mode'] = 'train' 50 | self.dropout_param['p'] = dropout 51 | if seed is not None: 52 | self.dropout_param['seed'] = seed 53 | 54 | self.bn_params = [] 55 | if self.normalization == 'batchnorm': 56 | self.bn = nn.BN() 57 | 58 | for i in range(len(hidden_dims)): 59 | self.params['gamma%d' % (i + 1)] = np.ones(hidden_dims[i]) 60 | self.params['beta%d' % (i + 1)] = np.zeros(hidden_dims[i]) 61 | self.bn_params.append({'mode': 'train'}) 62 | 63 | # 转换可学习参数为指定数据类型 64 | for k, v in self.params.items(): 65 | self.params[k] = v.astype(dtype) 66 | 67 | def __call__(self, inputs): 68 | return self.forward(inputs) 69 | 70 | def forward(self, inputs): 71 | inputs = inputs.reshape(inputs.shape[0], -1) 72 | inputs = inputs.astype(self.dtype) 73 | 74 | x = None 75 | for i in range(self.num_layers): 76 | w = self.params['W%d' % (i + 1)] 77 | b = self.params['b%d' % (i + 1)] 78 | 79 | if i == 0: 80 | x = inputs 81 | self.caches['z%d' % (i + 1)], self.caches['z%d_cache' % (i + 1)] = self.fcs[i].forward(x, w, b) 82 | 83 | if i != (self.num_layers - 1): 84 | z = self.caches['z%d' % (i + 1)] 85 | 86 | if self.normalization == 'batchnorm': 87 | gamma = self.params['gamma%d' % (i + 1)] 88 | beta = self.params['beta%d' % (i + 1)] 89 | bn_param = self.bn_params[i] 90 | 91 | y, self.caches['y%d_cache' % (i + 1)] = self.bn(z, gamma, beta, bn_param) 92 | z = y 93 | 94 | x = self.relu(z) 95 | 96 | if self.use_dropout: 97 | x, self.caches['dropout%d_cache' % (i + 1)] = self.dropout(x, self.dropout_param) 98 | 99 | return self.caches['z%d' % self.num_layers] 100 | 101 | def backward(self, grad_out): 102 | grad = dict() 103 | 104 | da = None 105 | for i in reversed(range(self.num_layers)): 106 | z = self.caches['z%d' % (i + 1)] 107 | z_cache = self.caches['z%d_cache' % (i + 1)] 108 | 109 | if i == (self.num_layers - 1): 110 | dz = grad_out 111 | else: 112 | if self.use_dropout: 113 | dropout_cache = self.caches['dropout%d_cache' % (i + 1)] 114 | da = self.dropout.backward(da, dropout_cache) 115 | 116 | dz = self.relu.backward(da, z) 117 | 118 | if self.normalization == 'batchnorm': 119 | y_cache = self.caches['y%d_cache' % (i + 1)] 120 | dy = dz 121 | dz, grad['gamma%d' % (i + 1)], grad['beta%d' % (i + 1)] = self.bn.backward(dy, y_cache) 122 | 123 | grad['W%d' % (i + 1)], grad['b%d' % (i + 1)], da = self.fcs[i].backward(dz, z_cache) 124 | 125 | self.caches = self._get_caches() 126 | return grad 127 | 128 | def _get_fcs(self): 129 | fcs = list() 130 | if self.hidden_dims is None: 131 | fcs.append(nn.FC(self.input_dim, self.num_classes, weight_scale=self.weight_scale)) 132 | else: 133 | for i in range(self.num_layers): 134 | if i == 0: 135 | num_in = self.input_dim 136 | else: 137 | num_in = self.hidden_dims[i - 1] 138 | 139 | if i == (self.num_layers - 1): 140 | num_out = self.num_classes 141 | else: 142 | num_out = self.hidden_dims[i] 143 | 144 | fcs.append(nn.FC(num_in, num_out)) 145 | 146 | return fcs 147 | 148 | def _get_params(self): 149 | params = dict() 150 | for i, fc in enumerate(self.fcs): 151 | params['W%d' % (i + 1)], params['b%d' % (i + 1)] = fc.get_params() 152 | return params 153 | 154 | def _get_caches(self): 155 | caches = dict() 156 | for i in range(1, self.num_layers): 157 | caches['z%d' % i] = None 158 | caches['z%d_cache' % i] = None 159 | if i != (self.num_layers - 1): 160 | if self.normalization == 'batchnorm': 161 | caches['y%d_cache' % (i + 1)] = None 162 | if self.use_dropout: 163 | caches['dropout%d_cache' % (i + 1)] = None 164 | 165 | return caches 166 | 167 | def train(self): 168 | if self.use_dropout: 169 | self.dropout_param['mode'] = 'train' 170 | if self.normalization == 'batchnorm': 171 | for i in range(self.num_layers - 1): 172 | self.bn_params[i]['mode'] = 'train' 173 | 174 | def eval(self): 175 | if self.use_dropout: 176 | self.dropout_param['mode'] = 'test' 177 | if self.normalization == 'batchnorm': 178 | for i in range(self.num_layers - 1): 179 | self.bn_params[i]['mode'] = 'test' 180 | -------------------------------------------------------------------------------- /pynet/models/LeNet5.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-7-2 上午10:14 4 | # @Author : zj 5 | 6 | 7 | from pynet import nn 8 | from .Net import Net 9 | from .utils import load_params 10 | 11 | __all__ = ['LeNet5'] 12 | 13 | model_urls = { 14 | 'lenets': '' 15 | } 16 | 17 | 18 | class LeNet5(Net): 19 | """ 20 | LeNet-5网络 21 | """ 22 | 23 | def __init__(self, in_channels=1, out_channels=10, dropout=1.0, weight_scale=1e-2): 24 | super(LeNet5, self).__init__() 25 | self.conv1 = nn.Conv2d2(in_channels, 5, 5, 6, stride=1, padding=0, weight_scale=weight_scale) 26 | self.conv2 = nn.Conv2d2(6, 5, 5, 16, stride=1, padding=0, weight_scale=weight_scale) 27 | self.conv3 = nn.Conv2d2(16, 5, 5, 120, stride=1, padding=0) 28 | 29 | self.maxPool1 = nn.MaxPool2(2, 2, 6, stride=2) 30 | self.maxPool2 = nn.MaxPool2(2, 2, 16, stride=2) 31 | self.fc1 = nn.FC(120, 84) 32 | self.fc2 = nn.FC(84, out_channels) 33 | 34 | self.relu = nn.ReLU() 35 | self.z1 = None 36 | self.z1_cache = None 37 | self.z2 = None 38 | self.z2_cache = None 39 | self.z3 = None 40 | self.z3_cache = None 41 | self.z4 = None 42 | self.z4_cache = None 43 | self.z5 = None 44 | self.z5_cache = None 45 | self.z6 = None 46 | self.z6_cache = None 47 | # self.z7 = None 48 | self.z7_cache = None 49 | 50 | self.use_dropout = dropout != 1.0 51 | self.dropout_param = {} 52 | if self.use_dropout: 53 | self.dropout_param['mode'] = 'train' 54 | self.dropout_param['p'] = dropout 55 | self.dropout = nn.Dropout() 56 | self.U1 = None 57 | 58 | self.params = self._get_params() 59 | 60 | def __call__(self, inputs): 61 | return self.forward(inputs) 62 | 63 | def forward(self, inputs): 64 | # inputs.shape = [N, C, H, W] 65 | assert len(inputs.shape) == 4 66 | self.z1, self.z1_cache = self.conv1(inputs, self.params['W1'], self.params['b1']) 67 | a1 = self.relu(self.z1) 68 | self.z2, self.z2_cache = self.maxPool1(a1) 69 | 70 | self.z3, self.z3_cache = self.conv2(self.z2, self.params['W2'], self.params['b2']) 71 | a3 = self.relu(self.z3) 72 | self.z4, self.z4_cache = self.maxPool2(a3) 73 | 74 | self.z5, self.z5_cache = self.conv3(self.z4, self.params['W3'], self.params['b3']) 75 | a5 = self.relu(self.z5) 76 | 77 | # (N, C, 1, 1) -> (N, C) 78 | a5 = a5.reshape(a5.shape[0], -1) 79 | 80 | self.z6, self.z6_cache = self.fc1(a5, self.params['W4'], self.params['b4']) 81 | a6 = self.relu(self.z6) 82 | if self.use_dropout and self.dropout_param['mode'] == 'train': 83 | self.U1 = self.dropout(a6.shape, self.dropout_param['p']) 84 | a6 *= self.U1 85 | 86 | z7, self.z7_cache = self.fc2(a6, self.params['W5'], self.params['b5']) 87 | 88 | return z7 89 | 90 | def backward(self, grad_out): 91 | grad = dict() 92 | 93 | grad['W5'], grad['b5'], da6 = self.fc2.backward(grad_out, self.z7_cache) 94 | if self.use_dropout and self.dropout_param['mode'] == 'train': 95 | da6 *= self.U1 96 | 97 | dz6 = self.relu.backward(da6, self.z6) 98 | grad['W4'], grad['b4'], da5 = self.fc1.backward(dz6, self.z6_cache) 99 | 100 | # [N, C] -> [N, C, 1, 1] 101 | N, C = da5.shape[:2] 102 | da5 = da5.reshape(N, C, 1, 1) 103 | 104 | dz5 = self.relu.backward(da5, self.z5) 105 | grad['W3'], grad['b3'], dz4 = self.conv3.backward(dz5, self.z5_cache) 106 | 107 | da3 = self.maxPool2.backward(dz4, self.z4_cache) 108 | dz3 = self.relu.backward(da3, self.z3) 109 | grad['W2'], grad['b2'], da2 = self.conv2.backward(dz3, self.z3_cache) 110 | 111 | da1 = self.maxPool1.backward(da2, self.z2_cache) 112 | dz1 = self.relu.backward(da1, self.z1) 113 | grad['W1'], grad['b1'], da0 = self.conv1.backward(dz1, self.z1_cache) 114 | 115 | return grad 116 | 117 | def _get_params(self): 118 | params = dict() 119 | params['W1'], params['b1'] = self.conv1.get_params() 120 | params['W2'], params['b2'] = self.conv2.get_params() 121 | params['W3'], params['b3'] = self.conv3.get_params() 122 | params['W4'], params['b4'] = self.fc1.get_params() 123 | params['W5'], params['b5'] = self.fc2.get_params() 124 | 125 | return params 126 | 127 | def train(self): 128 | if self.use_dropout: 129 | self.dropout_param['mode'] = 'train' 130 | 131 | def eval(self): 132 | if self.use_dropout: 133 | self.dropout_param['mode'] = 'test' 134 | -------------------------------------------------------------------------------- /pynet/models/NIN.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-6-21 上午11:00 4 | # @Author : zj 5 | 6 | 7 | import pynet.nn as nn 8 | from .Net import Net 9 | from .utils import load_params 10 | 11 | __all__ = ['NIN', 'nin'] 12 | 13 | model_urls = { 14 | 'nin': '' 15 | } 16 | 17 | 18 | class NIN(Net): 19 | """ 20 | NIN网络 21 | """ 22 | 23 | def __init__(self, in_channels=1, out_channels=10, momentum=0, nesterov=False, p_h=1.0): 24 | super(NIN, self).__init__() 25 | self.conv1 = nn.Conv2d(in_channels, 5, 5, 192, stride=1, padding=2, momentum=momentum, nesterov=nesterov) 26 | self.conv2 = nn.Conv2d(96, 5, 5, 192, stride=1, padding=2, momentum=momentum, nesterov=nesterov) 27 | self.conv3 = nn.Conv2d(192, 3, 3, 192, stride=1, padding=1, momentum=momentum, nesterov=nesterov) 28 | 29 | self.mlp1 = nn.Conv2d(192, 1, 1, 160, stride=1, padding=0, momentum=momentum, nesterov=nesterov) 30 | self.mlp2 = nn.Conv2d(160, 1, 1, 96, stride=1, padding=0, momentum=momentum, nesterov=nesterov) 31 | 32 | self.mlp2_1 = nn.Conv2d(192, 1, 1, 192, stride=1, padding=0, momentum=momentum, nesterov=nesterov) 33 | self.mlp2_2 = nn.Conv2d(192, 1, 1, 192, stride=1, padding=0, momentum=momentum, nesterov=nesterov) 34 | 35 | self.mlp3_1 = nn.Conv2d(192, 1, 1, 192, stride=1, padding=0, momentum=momentum, nesterov=nesterov) 36 | self.mlp3_2 = nn.Conv2d(192, 1, 1, out_channels, stride=1, padding=0, momentum=momentum, nesterov=nesterov) 37 | 38 | self.maxPool1 = nn.MaxPool(2, 2, 96, stride=2) 39 | self.maxPool2 = nn.MaxPool(2, 2, 192, stride=2) 40 | 41 | self.gap = nn.GAP() 42 | 43 | self.relu1 = nn.ReLU() 44 | self.relu2 = nn.ReLU() 45 | self.relu3 = nn.ReLU() 46 | self.relu4 = nn.ReLU() 47 | self.relu5 = nn.ReLU() 48 | self.relu6 = nn.ReLU() 49 | self.relu7 = nn.ReLU() 50 | self.relu8 = nn.ReLU() 51 | self.relu9 = nn.ReLU() 52 | 53 | self.dropout = nn.Dropout2d() 54 | 55 | self.p_h = p_h 56 | self.U1 = None 57 | self.U2 = None 58 | 59 | def __call__(self, inputs): 60 | return self.forward(inputs) 61 | 62 | def forward(self, inputs): 63 | # inputs.shape = [N, C, H, W] 64 | assert len(inputs.shape) == 4 65 | x = self.relu1(self.conv1(inputs)) 66 | x = self.relu2(self.mlp1(x)) 67 | x = self.relu3(self.mlp2(x)) 68 | x = self.maxPool1(x) 69 | self.U1 = self.dropout(x.shape, self.p_h) 70 | x *= self.U1 71 | 72 | x = self.relu4(self.conv2(x)) 73 | x = self.relu5(self.mlp2_1(x)) 74 | x = self.relu6(self.mlp2_2(x)) 75 | x = self.maxPool2(x) 76 | self.U2 = self.dropout(x.shape, self.p_h) 77 | x *= self.U2 78 | 79 | x = self.relu7(self.conv3(x)) 80 | x = self.relu8(self.mlp3_1(x)) 81 | x = self.relu9(self.mlp3_2(x)) 82 | 83 | x = self.gap(x) 84 | return x 85 | 86 | def backward(self, grad_out): 87 | # grad_out.shape = [N, C] 88 | assert len(grad_out.shape) == 2 89 | da11 = self.gap.backward(grad_out) 90 | 91 | dz11 = self.relu9.backward(da11) 92 | da10 = self.mlp3_2.backward(dz11) 93 | dz10 = self.relu8.backward(da10) 94 | da9 = self.mlp3_1.backward(dz10) 95 | dz9 = self.relu7.backward(da9) 96 | da8 = self.conv3.backward(dz9) 97 | 98 | da8 *= self.U2 99 | da7 = self.maxPool2.backward(da8) 100 | dz7 = self.relu6.backward(da7) 101 | da6 = self.mlp2_2.backward(dz7) 102 | dz6 = self.relu5.backward(da6) 103 | da5 = self.mlp2_1.backward(dz6) 104 | dz5 = self.relu4.backward(da5) 105 | da4 = self.conv2.backward(dz5) 106 | 107 | da4 *= self.U1 108 | da3 = self.maxPool1.backward(da4) 109 | dz3 = self.relu3.backward(da3) 110 | da2 = self.mlp2.backward(dz3) 111 | dz2 = self.relu2.backward(da2) 112 | da1 = self.mlp1.backward(dz2) 113 | dz1 = self.relu1.backward(da1) 114 | da0 = self.conv1.backward(dz1) 115 | 116 | def update(self, lr=1e-3, reg=1e-3): 117 | self.mlp3_2.update(learning_rate=lr, regularization_rate=reg) 118 | self.mlp3_1.update(learning_rate=lr, regularization_rate=reg) 119 | self.conv3.update(learning_rate=lr, regularization_rate=reg) 120 | 121 | self.mlp2_2.update(learning_rate=lr, regularization_rate=reg) 122 | self.mlp2_1.update(learning_rate=lr, regularization_rate=reg) 123 | self.conv2.update(learning_rate=lr, regularization_rate=reg) 124 | 125 | self.mlp2.update(learning_rate=lr, regularization_rate=reg) 126 | self.mlp1.update(learning_rate=lr, regularization_rate=reg) 127 | self.conv1.update(learning_rate=lr, regularization_rate=reg) 128 | 129 | def predict(self, inputs): 130 | # inputs.shape = [N, C, H, W] 131 | assert len(inputs.shape) == 4 132 | x = self.relu1(self.conv1(inputs)) 133 | x = self.relu2(self.mlp1(x)) 134 | x = self.relu3(self.mlp2(x)) 135 | x = self.maxPool1(x) 136 | 137 | x = self.relu4(self.conv2(x)) 138 | x = self.relu5(self.mlp2_1(x)) 139 | x = self.relu6(self.mlp2_2(x)) 140 | x = self.maxPool2(x) 141 | 142 | x = self.relu7(self.conv3(x)) 143 | x = self.relu8(self.mlp3_1(x)) 144 | x = self.relu9(self.mlp3_2(x)) 145 | 146 | x = self.gap(x) 147 | return x 148 | 149 | def get_params(self): 150 | out = dict() 151 | out['conv1'] = self.conv1.get_params() 152 | out['conv2'] = self.conv2.get_params() 153 | out['conv3'] = self.conv3.get_params() 154 | 155 | out['mlp1'] = self.mlp1.get_params() 156 | out['mlp2'] = self.mlp2.get_params() 157 | out['mlp2_1'] = self.mlp2_1.get_params() 158 | out['mlp2_2'] = self.mlp2_2.get_params() 159 | out['mlp3_1'] = self.mlp3_1.get_params() 160 | out['mlp3_2'] = self.mlp3_2.get_params() 161 | 162 | out['p_h'] = self.p_h 163 | 164 | return out 165 | 166 | def set_params(self, params): 167 | self.conv1.set_params(params['conv1']) 168 | self.conv2.set_params(params['conv2']) 169 | self.conv3.set_params(params['conv3']) 170 | 171 | self.mlp1.set_params(params['mlp1']) 172 | self.mlp2.set_params(params['mlp2']) 173 | self.mlp2_1.set_params(params['mlp2_1']) 174 | self.mlp2_2.set_params(params['mlp2_1']) 175 | self.mlp3_1.set_params(params['mlp3_1']) 176 | self.mlp3_2.set_params(params['mlp3_1']) 177 | 178 | self.p_h = params.get('p_h', 1.0) 179 | 180 | 181 | def nin(pretrained=False, **kwargs): 182 | """ 183 | 创建模型对象 184 | """ 185 | 186 | model = NIN(**kwargs) 187 | if pretrained: 188 | params = load_params(model_urls['nin']) 189 | model.set_params(params) 190 | return model 191 | -------------------------------------------------------------------------------- /pynet/models/Net.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-6-20 下午2:06 4 | # @Author : zj 5 | 6 | from abc import ABCMeta, abstractmethod 7 | 8 | __all__ = ['Net'] 9 | 10 | 11 | class Net(metaclass=ABCMeta): 12 | """ 13 | 所有神经网络模型基类 14 | """ 15 | 16 | @abstractmethod 17 | def forward(self, inputs): 18 | pass 19 | 20 | @abstractmethod 21 | def backward(self, grad_out): 22 | pass 23 | -------------------------------------------------------------------------------- /pynet/models/ThreeLayerNet.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-6-30 下午4:20 4 | # @Author : zj 5 | 6 | 7 | from pynet import nn 8 | from .Net import Net 9 | 10 | __all__ = ['ThreeLayerNet'] 11 | 12 | 13 | class ThreeLayerNet(Net): 14 | """ 15 | 实现2层神经网络 16 | """ 17 | 18 | def __init__(self, num_in=0, num_h1=0, num_h2=0, num_out=0, dropout=1.0, weight_scale=1e-2): 19 | super(ThreeLayerNet, self).__init__() 20 | self.fc1 = nn.FC(num_in, num_h1, weight_scale=weight_scale) 21 | self.fc2 = nn.FC(num_h1, num_h2, weight_scale=weight_scale) 22 | self.fc3 = nn.FC(num_h2, num_out, weight_scale=weight_scale) 23 | 24 | self.relu = nn.ReLU() 25 | self.z1_cache = None 26 | self.z1 = None 27 | self.z2_cache = None 28 | self.z2 = None 29 | self.z3_cache = None 30 | 31 | self.use_dropout = dropout != 1.0 32 | self.dropout_param = {} 33 | if self.use_dropout: 34 | self.dropout_param['mode'] = 'train' 35 | self.dropout_param['p'] = dropout 36 | self.dropout = nn.Dropout() 37 | self.U1 = None 38 | self.U2 = None 39 | 40 | self.params = self._get_params() 41 | 42 | def __call__(self, inputs): 43 | return self.forward(inputs) 44 | 45 | def forward(self, inputs): 46 | inputs = inputs.reshape(inputs.shape[0], -1) 47 | 48 | self.z1, self.z1_cache = self.fc1(inputs, self.params['W1'], self.params['b1']) 49 | a1 = self.relu(self.z1) 50 | if self.use_dropout and self.dropout_param['mode'] == 'train': 51 | self.U1 = self.dropout(a1.shape, self.dropout_param['p']) 52 | a1 *= self.U1 53 | 54 | self.z2, self.z2_cache = self.fc2(a1, self.params['W2'], self.params['b2']) 55 | a2 = self.relu(self.z2) 56 | if self.use_dropout and self.dropout_param['mode'] == 'train': 57 | self.U2 = self.dropout(a2.shape, self.dropout_param['p']) 58 | a2 *= self.U2 59 | 60 | z3, self.z3_cache = self.fc3(a2, self.params['W3'], self.params['b3']) 61 | 62 | return z3 63 | 64 | def backward(self, grad_out): 65 | grad = dict() 66 | grad['W3'], grad['b3'], da2 = self.fc3.backward(grad_out, self.z3_cache) 67 | if self.use_dropout and self.dropout_param['mode'] == 'train': 68 | da2 *= self.U2 69 | 70 | dz2 = self.relu.backward(da2, self.z2) 71 | grad['W2'], grad['b2'], da1 = self.fc2.backward(dz2, self.z2_cache) 72 | if self.use_dropout and self.dropout_param['mode'] == 'train': 73 | da2 *= self.U2 74 | 75 | dz1 = self.relu.backward(da1, self.z1) 76 | grad['W1'], grad['b1'], da0 = self.fc1.backward(dz1, self.z1_cache) 77 | 78 | return grad 79 | 80 | def _get_params(self): 81 | params = dict() 82 | params['W1'], params['b1'] = self.fc1.get_params() 83 | params['W2'], params['b2'] = self.fc2.get_params() 84 | params['W3'], params['b3'] = self.fc3.get_params() 85 | return params 86 | 87 | def train(self): 88 | if self.use_dropout: 89 | self.dropout_param['mode'] = 'train' 90 | 91 | def eval(self): 92 | if self.use_dropout: 93 | self.dropout_param['mode'] = 'test' 94 | -------------------------------------------------------------------------------- /pynet/models/TwoLayerNet.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-6-29 下午8:28 4 | # @Author : zj 5 | 6 | 7 | from pynet import nn 8 | from .Net import Net 9 | from .utils import load_params 10 | 11 | __all__ = ['TwoLayerNet'] 12 | 13 | model_urls = { 14 | 'two_layer_net': '' 15 | } 16 | 17 | 18 | class TwoLayerNet(Net): 19 | """ 20 | 实现2层神经网络 21 | """ 22 | 23 | def __init__(self, num_in=0, num_hidden=0, num_out=0, dropout=1.0, weight_scale=1e-2): 24 | super(TwoLayerNet, self).__init__() 25 | self.fc1 = nn.FC(num_in, num_hidden, weight_scale) 26 | self.fc2 = nn.FC(num_hidden, num_out, weight_scale) 27 | 28 | self.relu = nn.ReLU() 29 | self.z1 = None 30 | self.z1_cache = None 31 | self.z2_cache = None 32 | 33 | self.use_dropout = dropout != 1.0 34 | self.dropout_param = {} 35 | if self.use_dropout: 36 | self.dropout_param['mode'] = 'train' 37 | self.dropout_param['p'] = dropout 38 | self.dropout = nn.Dropout() 39 | self.U1 = None 40 | 41 | self.params = self._get_params() 42 | 43 | def __call__(self, inputs): 44 | return self.forward(inputs) 45 | 46 | def forward(self, inputs): 47 | # inputs.shape = [N, D_in] 48 | assert len(inputs.shape) == 2 49 | self.z1, self.z1_cache = self.fc1(inputs, self.params['W1'], self.params['b1']) 50 | a1 = self.relu(self.z1) 51 | if self.use_dropout and self.dropout_param['mode'] == 'train': 52 | self.U1 = self.dropout(a1.shape, self.dropout_param['p']) 53 | a1 *= self.U1 54 | 55 | z2, self.z2_cache = self.fc2(a1, self.params['W2'], self.params['b2']) 56 | 57 | return z2 58 | 59 | def backward(self, grad_out): 60 | grad = dict() 61 | grad['W2'], grad['b2'], da1 = self.fc2.backward(grad_out, self.z2_cache) 62 | if self.use_dropout and self.dropout_param['mode'] == 'train': 63 | da1 *= self.U1 64 | 65 | dz1 = self.relu.backward(da1, self.z1) 66 | grad['W1'], grad['b1'], da0 = self.fc1.backward(dz1, self.z1_cache) 67 | 68 | return grad 69 | 70 | def _get_params(self): 71 | params = dict() 72 | params['W1'], params['b1'] = self.fc1.get_params() 73 | params['W2'], params['b2'] = self.fc2.get_params() 74 | return params 75 | 76 | def train(self): 77 | if self.use_dropout: 78 | self.dropout_param['mode'] = 'train' 79 | 80 | def eval(self): 81 | if self.use_dropout: 82 | self.dropout_param['mode'] = 'test' 83 | 84 | 85 | def two_layer_net(pretrained=False, **kwargs): 86 | """ 87 | 创建模型对象 88 | """ 89 | 90 | model = TwoLayerNet(**kwargs) 91 | if pretrained: 92 | params = load_params(model_urls['alexnet']) 93 | model.set_params(params) 94 | return model 95 | -------------------------------------------------------------------------------- /pynet/models/__init__.py: -------------------------------------------------------------------------------- 1 | from .TwoLayerNet import * 2 | from .ThreeLayerNet import * 3 | from .FCNet import * 4 | from .LeNet5 import * 5 | from .AlexNet import * 6 | from .NIN import * 7 | 8 | from .utils import * 9 | -------------------------------------------------------------------------------- /pynet/models/__pycache__/AlexNet.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deep-learning-algorithm/PyNet/354c7ee88a712a1f5069d58a0be4a6cbfaeab861/pynet/models/__pycache__/AlexNet.cpython-37.pyc -------------------------------------------------------------------------------- /pynet/models/__pycache__/NIN.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deep-learning-algorithm/PyNet/354c7ee88a712a1f5069d58a0be4a6cbfaeab861/pynet/models/__pycache__/NIN.cpython-37.pyc -------------------------------------------------------------------------------- /pynet/models/__pycache__/Net.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deep-learning-algorithm/PyNet/354c7ee88a712a1f5069d58a0be4a6cbfaeab861/pynet/models/__pycache__/Net.cpython-37.pyc -------------------------------------------------------------------------------- /pynet/models/__pycache__/ThreeLayerNet.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deep-learning-algorithm/PyNet/354c7ee88a712a1f5069d58a0be4a6cbfaeab861/pynet/models/__pycache__/ThreeLayerNet.cpython-37.pyc -------------------------------------------------------------------------------- /pynet/models/__pycache__/TwoLayerNet.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deep-learning-algorithm/PyNet/354c7ee88a712a1f5069d58a0be4a6cbfaeab861/pynet/models/__pycache__/TwoLayerNet.cpython-37.pyc -------------------------------------------------------------------------------- /pynet/models/__pycache__/__init__.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deep-learning-algorithm/PyNet/354c7ee88a712a1f5069d58a0be4a6cbfaeab861/pynet/models/__pycache__/__init__.cpython-37.pyc -------------------------------------------------------------------------------- /pynet/models/__pycache__/utils.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deep-learning-algorithm/PyNet/354c7ee88a712a1f5069d58a0be4a6cbfaeab861/pynet/models/__pycache__/utils.cpython-37.pyc -------------------------------------------------------------------------------- /pynet/models/utils.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-6-20 下午2:16 4 | # @Author : zj 5 | 6 | 7 | import pickle 8 | 9 | 10 | def save_params(params, path='params.pkl'): 11 | with open(path, 'wb') as f: 12 | pickle.dump(params, f, -1) 13 | 14 | 15 | def load_params(path='params.pkl'): 16 | with open(path, 'rb') as f: 17 | param = pickle.load(f) 18 | return param 19 | 20 | 21 | if __name__ == '__main__': 22 | a = {'a': 2341, 'b': 'adfadfa'} 23 | save_params(a) 24 | b = load_params() 25 | print(b) 26 | -------------------------------------------------------------------------------- /pynet/nn/BN.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-7-4 下午2:45 4 | # @Author : zj 5 | 6 | import numpy as np 7 | 8 | __all__ = ['BN'] 9 | 10 | 11 | class BN: 12 | """ 13 | batch normalization layer 14 | 批量归一化层 15 | """ 16 | 17 | def __call__(self, inputs, gamma, beta, bn_param): 18 | return self.forward(inputs, gamma, beta, bn_param) 19 | 20 | def forward(self, inputs, gamma, beta, bn_param): 21 | # inputs.shape == [N, C] 22 | assert len(inputs.shape) == 2 23 | 24 | mode = bn_param['mode'] 25 | eps = bn_param.get('eps', 1e-5) 26 | momentum = bn_param.get('momentum', 0.9) 27 | 28 | N, D = inputs.shape 29 | running_mean = bn_param.get('running_mean', np.zeros(D, dtype=inputs.dtype)) 30 | running_var = bn_param.get('running_var', np.zeros(D, dtype=inputs.dtype)) 31 | 32 | out, cache = None, None 33 | if mode == 'train': 34 | sample_mean = np.mean(inputs, axis=0, keepdims=True) 35 | sample_var = np.var(inputs, axis=0, keepdims=True) 36 | x_norm = (inputs - sample_mean) / np.sqrt(sample_var + eps) 37 | 38 | out = x_norm * gamma + beta 39 | cache = (sample_mean, sample_var, x_norm, gamma, eps, inputs) 40 | 41 | running_mean = momentum * running_mean + (1 - momentum) * sample_mean 42 | running_var = momentum * running_var + (1 - momentum) * sample_var 43 | elif mode == 'test': 44 | stds = np.sqrt(running_var + eps) 45 | out = gamma / stds * inputs + (beta - gamma * running_mean / stds) 46 | else: 47 | raise ValueError('Invalid forward batchnorm mode "%s"' % mode) 48 | 49 | # Store the updated running means back into bn_param 50 | bn_param['running_mean'] = running_mean 51 | bn_param['running_var'] = running_var 52 | 53 | return out, cache 54 | 55 | def backward(self, grad_out, cache): 56 | sample_mean, sample_var, x_norm, gamma, eps, x = cache 57 | m = x.shape[0] 58 | 59 | dgamma = np.sum(grad_out * x_norm, axis=0) 60 | dbeta = np.sum(grad_out, axis=0) 61 | dnorm = grad_out * gamma 62 | 63 | dvar = dnorm * (x - sample_mean) * (-0.5) * (sample_var + eps) ** (-1.5) 64 | dvar = np.sum(dvar, axis=0, keepdims=True) 65 | dmean = np.sum(dnorm * (-1) / np.sqrt(sample_var + eps), axis=0, keepdims=True) 66 | dmean += dvar * np.sum(-2 * (x - sample_mean), axis=0, keepdims=True) / m 67 | 68 | dx = dnorm / np.sqrt(sample_var + eps) + dvar * 2 * (x - sample_mean) / m + dmean / m 69 | 70 | return dx, dgamma, dbeta 71 | -------------------------------------------------------------------------------- /pynet/nn/Conv2d.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-7-2 上午9:53 4 | # @Author : zj 5 | 6 | 7 | from .utils import * 8 | from .im2row import * 9 | from .pool2row import * 10 | from .Layer import * 11 | 12 | __all__ = ['Conv2d'] 13 | 14 | 15 | class Conv2d: 16 | """ 17 | convolutional layer 18 | 卷积层 19 | """ 20 | 21 | def __init__(self, in_c, filter_h, filter_w, filter_num, stride=1, padding=0, weight_scale=1e-2): 22 | """ 23 | :param in_c: 输入数据体通道数 24 | :param filter_h: 滤波器长 25 | :param filter_w: 滤波器宽 26 | :param filter_num: 滤波器个数 27 | :param stride: 步长 28 | :param padding: 零填充 29 | :param weight_scale: 30 | """ 31 | super(Conv2d, self).__init__() 32 | self.in_c = in_c 33 | self.filter_h = filter_h 34 | self.filter_w = filter_w 35 | self.filter_num = filter_num 36 | self.stride = stride 37 | self.padding = padding 38 | self.weight_scale = weight_scale 39 | 40 | def __call__(self, inputs, w, b): 41 | return self.forward(inputs, w, b) 42 | 43 | def forward(self, inputs, w, b): 44 | # input.shape == [N, C, H, W] 45 | assert len(inputs.shape) == 4 46 | N, C, H, W = inputs.shape[:4] 47 | out_h = int((H - self.filter_h + 2 * self.padding) / self.stride + 1) 48 | out_w = int((W - self.filter_w + 2 * self.padding) / self.stride + 1) 49 | 50 | a = im2row_indices(inputs, self.filter_h, self.filter_w, stride=self.stride, padding=self.padding) 51 | z = a.dot(w) + b 52 | 53 | out = conv_fc2output(z, N, out_h, out_w) 54 | cache = (a, inputs.shape, w, b) 55 | return out, cache 56 | 57 | def backward(self, grad_out, cache): 58 | assert len(grad_out.shape) == 4 59 | 60 | a, input_shape, w, b = cache 61 | 62 | dz = conv_output2fc(grad_out) 63 | grad_W = a.T.dot(dz) 64 | grad_b = np.sum(dz, axis=0, keepdims=True) / dz.shape[0] 65 | 66 | da = dz.dot(w.T) 67 | return grad_W, grad_b, row2im_indices(da, input_shape, field_height=self.filter_h, 68 | field_width=self.filter_w, stride=self.stride, padding=self.padding) 69 | 70 | def get_params(self): 71 | return self.weight_scale * np.random.normal(loc=0, scale=1.0, size=( 72 | self.filter_h * self.filter_w * self.in_c, self.filter_num)), \ 73 | np.zeros((1, self.filter_num)) 74 | -------------------------------------------------------------------------------- /pynet/nn/CrossEntropyLoss.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-6-20 下午3:12 4 | # @Author : zj 5 | 6 | 7 | from .pool2row import * 8 | 9 | __all__ = ['Conv2d'] 10 | 11 | 12 | class CrossEntropyLoss(object): 13 | 14 | def __call__(self, scores, labels): 15 | return self.forward(scores, labels) 16 | 17 | def forward(self, scores, labels): 18 | # scores.shape == [N, score] 19 | # labels.shape == [N] 20 | assert len(scores.shape) == 2 21 | assert len(labels.shape) == 1 22 | scores -= np.max(scores, axis=1, keepdims=True) 23 | expscores = np.exp(scores) 24 | probs = expscores / np.sum(expscores, axis=1, keepdims=True) 25 | 26 | N = labels.shape[0] 27 | correct_probs = probs[range(N), labels] 28 | loss = -1.0 / N * np.sum(np.log(correct_probs)) 29 | return loss, probs 30 | 31 | def backward(self, probs, labels): 32 | assert len(probs.shape) == 2 33 | assert len(labels.shape) == 1 34 | grad_out = probs 35 | N = labels.shape[0] 36 | 37 | grad_out[range(N), labels] -= 1 38 | return grad_out 39 | -------------------------------------------------------------------------------- /pynet/nn/Dropout.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-6-20 下午2:56 4 | # @Author : zj 5 | 6 | import numpy as np 7 | 8 | 9 | class Dropout(object): 10 | 11 | def __call__(self, inputs, dropout_param): 12 | return self.forward(inputs, dropout_param) 13 | 14 | def forward(self, inputs, dropout_param): 15 | p, mode = dropout_param['p'], dropout_param['mode'] 16 | if 'seed' in dropout_param: 17 | np.random.seed(dropout_param['seed']) 18 | 19 | mask = None 20 | out = None 21 | 22 | if mode == 'train': 23 | mask = (np.random.ranf(inputs.shape) < p) / p 24 | out = inputs * mask 25 | elif mode == 'test': 26 | out = inputs 27 | 28 | cache = (dropout_param, mask) 29 | # print(mask.shape) 30 | out = out.astype(inputs.dtype, copy=False) 31 | 32 | return out, cache 33 | 34 | def backward(self, dout, cache): 35 | dropout_param, mask = cache 36 | mode = dropout_param['mode'] 37 | 38 | dx = None 39 | if mode == 'train': 40 | dx = dout * mask 41 | elif mode == 'test': 42 | dx = dout 43 | 44 | return dx 45 | -------------------------------------------------------------------------------- /pynet/nn/Dropout2d.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-6-20 下午3:37 4 | # @Author : zj 5 | 6 | import numpy as np 7 | 8 | 9 | class Dropout2d(object): 10 | 11 | def __call__(self, shape, p): 12 | return self.forward(shape, p) 13 | 14 | def forward(self, shape, p): 15 | assert len(shape) == 4 16 | N, C, H, W = shape[:4] 17 | U = (np.random.rand(N * C, 1) < p) / p 18 | res = np.ones((N * C, H * W)) 19 | res *= U 20 | 21 | if np.sum(res) == 0: 22 | return 1.0 / p 23 | return res.reshape(N, C, H, W) 24 | -------------------------------------------------------------------------------- /pynet/nn/FC.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-6-30 上午10:12 4 | # @Author : zj 5 | 6 | from .pool2row import * 7 | from .Layer import * 8 | 9 | __all__ = ['FC'] 10 | 11 | 12 | class FC: 13 | """ 14 | fully connected layer 15 | 全连接层 16 | """ 17 | 18 | def __init__(self, num_in, num_out, weight_scale=1e-2): 19 | """ 20 | :param num_in: 前一层神经元个数 21 | :param num_out: 当前层神经元个数 22 | """ 23 | super(FC, self).__init__() 24 | assert isinstance(num_in, int) and num_in > 0 25 | assert isinstance(num_out, int) and num_out > 0 26 | 27 | self.num_in = num_in 28 | self.num_out = num_out 29 | self.weight_scale = weight_scale 30 | 31 | def __call__(self, inputs, w, b): 32 | return self.forward(inputs, w, b) 33 | 34 | def forward(self, inputs, w, b): 35 | # inputs.shape == [N, num_in] 36 | assert len(inputs.shape) == 2 and inputs.shape[1] == self.num_in 37 | assert w.shape == (self.num_in, self.num_out) 38 | assert b.shape == (1, self.num_out) 39 | 40 | z = inputs.dot(w) + b 41 | cache = (inputs, w, b) 42 | return z, cache 43 | 44 | def backward(self, grad_out, cache): 45 | inputs, w, b = cache 46 | grad_w = inputs.T.dot(grad_out) 47 | grad_b = np.sum(grad_out, axis=0, keepdims=True) / grad_out.shape[0] 48 | 49 | grad_in = grad_out.dot(w.T) 50 | return grad_w, grad_b, grad_in 51 | 52 | def get_params(self): 53 | return self.weight_scale * np.random.normal(loc=0, scale=1.0, size=(self.num_in, self.num_out)), \ 54 | np.zeros((1, self.num_out)) 55 | -------------------------------------------------------------------------------- /pynet/nn/GAP.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-6-21 上午11:18 4 | # @Author : zj 5 | 6 | 7 | import numpy as np 8 | from .Layer import * 9 | 10 | __all__ = ['GAP'] 11 | 12 | 13 | class GAP(Layer): 14 | """ 15 | global average pooling layer 16 | 全局平均池化层 17 | """ 18 | 19 | def __call__(self, inputs): 20 | return self.forward(inputs) 21 | 22 | def forward(self, inputs): 23 | # input.shape == [N, C, H, W] 24 | assert len(inputs.shape) == 4 25 | N, C, H, W = inputs.shape[:4] 26 | 27 | z = np.mean(inputs.reshape(N, C, -1), axis=2) 28 | 29 | cache = (inputs.shape) 30 | return z, cache 31 | 32 | def backward(self, grad_out, cache): 33 | input_shape = cache 34 | N, C, H, W = input_shape[:4] 35 | dz = grad_out.reshape(N * C, -1) 36 | da = np.repeat(dz, H * W, axis=1) 37 | 38 | return da.reshape(N, C, H, W) 39 | 40 | 41 | if __name__ == '__main__': 42 | gap = GAP() 43 | 44 | inputs = np.arange(36).reshape(2, 2, 3, 3) 45 | res, cache = gap(inputs) 46 | print(res) 47 | 48 | grad_out = np.arange(4).reshape(2, 2) 49 | da = gap.backward(grad_out, cache) 50 | print(da) 51 | -------------------------------------------------------------------------------- /pynet/nn/Layer.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-6-20 下午2:24 4 | # @Author : zj 5 | 6 | from abc import ABCMeta, abstractmethod 7 | 8 | __all__ = ['Layer'] 9 | 10 | 11 | class Layer(metaclass=ABCMeta): 12 | 13 | @abstractmethod 14 | def forward(self, inputs): 15 | pass 16 | 17 | @abstractmethod 18 | def backward(self, grad_out, cache): 19 | pass 20 | -------------------------------------------------------------------------------- /pynet/nn/MaxPool.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-7-2 上午10:06 4 | # @Author : zj 5 | 6 | from .utils import * 7 | from .pool2row import * 8 | from .Layer import * 9 | 10 | __all__ = ['MaxPool'] 11 | 12 | 13 | class MaxPool: 14 | """ 15 | max pool layer 16 | 池化层，执行max运算 17 | """ 18 | 19 | def __init__(self, filter_h, filter_w, filter_num, stride=2): 20 | super(MaxPool, self).__init__() 21 | self.filter_h = filter_h 22 | self.filter_w = filter_w 23 | self.filter_num = filter_num 24 | self.stride = stride 25 | 26 | def __call__(self, inputs): 27 | return self.forward(inputs) 28 | 29 | def forward(self, inputs): 30 | # input.shape == [N, C, H, W] 31 | assert len(inputs.shape) == 4 32 | N, C, H, W = inputs.shape[:4] 33 | out_h = int((H - self.filter_h) / self.stride + 1) 34 | out_w = int((W - self.filter_w) / self.stride + 1) 35 | 36 | a = pool2row_indices(inputs, self.filter_h, self.filter_w, stride=self.stride) 37 | z = np.max(a, axis=1) 38 | 39 | arg_z = np.argmax(a, axis=1) 40 | input_shape = inputs.shape 41 | a_shape = a.shape 42 | cache = (arg_z, input_shape, a_shape) 43 | 44 | return pool_fc2output(z, N, out_h, out_w), cache 45 | 46 | def backward(self, grad_out, cache): 47 | arg_z, input_shape, a_shape = cache 48 | 49 | dz = pool_output2fc(grad_out) 50 | da = np.zeros(a_shape) 51 | da[range(a_shape[0]), arg_z] = dz 52 | 53 | return row2pool_indices(da, input_shape, field_height=self.filter_h, field_width=self.filter_w, 54 | stride=self.stride) 55 | -------------------------------------------------------------------------------- /pynet/nn/ReLU.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-6-30 上午10:33 4 | # @Author : zj 5 | 6 | import numpy as np 7 | from .Layer import * 8 | 9 | __all__ = ['ReLU'] 10 | 11 | 12 | class ReLU(object): 13 | 14 | def __call__(self, inputs): 15 | return self.forward(inputs) 16 | 17 | def forward(self, inputs): 18 | return np.maximum(inputs, 0) 19 | 20 | def backward(self, grad_out, inputs): 21 | assert inputs.shape == grad_out.shape 22 | 23 | grad_in = grad_out.copy() 24 | grad_in[inputs < 0] = 0 25 | return grad_in 26 | -------------------------------------------------------------------------------- /pynet/nn/Softmax.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-6-20 下午8:07 4 | # @Author : zj 5 | 6 | import numpy as np 7 | 8 | __all__ = ['Softmax'] 9 | 10 | 11 | class Softmax(object): 12 | """ 13 | softmax评分 14 | """ 15 | 16 | def __init__(self): 17 | pass 18 | 19 | def __call__(self, scores): 20 | return self.forward(scores) 21 | 22 | def forward(self, scores): 23 | # scores.shape == [N, C] 24 | assert len(scores.shape) == 2 25 | scores -= np.max(scores, axis=1, keepdims=True) 26 | expscores = np.exp(scores) 27 | probs = expscores / np.sum(expscores, axis=1, keepdims=True) 28 | 29 | return probs 30 | -------------------------------------------------------------------------------- /pynet/nn/__init__.py: -------------------------------------------------------------------------------- 1 | from .Layer import Layer 2 | from .FC import FC 3 | from .ReLU import ReLU 4 | from .Dropout import Dropout 5 | from .Dropout2d import Dropout2d 6 | from .CrossEntropyLoss import CrossEntropyLoss 7 | from .Conv2d import Conv2d 8 | from .MaxPool import MaxPool 9 | from .GAP import GAP 10 | from .BN import BN 11 | -------------------------------------------------------------------------------- /pynet/nn/__pycache__/CrossEntropyLoss.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deep-learning-algorithm/PyNet/354c7ee88a712a1f5069d58a0be4a6cbfaeab861/pynet/nn/__pycache__/CrossEntropyLoss.cpython-37.pyc -------------------------------------------------------------------------------- /pynet/nn/__pycache__/Dropout.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deep-learning-algorithm/PyNet/354c7ee88a712a1f5069d58a0be4a6cbfaeab861/pynet/nn/__pycache__/Dropout.cpython-37.pyc -------------------------------------------------------------------------------- /pynet/nn/__pycache__/Dropout2d.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deep-learning-algorithm/PyNet/354c7ee88a712a1f5069d58a0be4a6cbfaeab861/pynet/nn/__pycache__/Dropout2d.cpython-37.pyc -------------------------------------------------------------------------------- /pynet/nn/__pycache__/FC.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deep-learning-algorithm/PyNet/354c7ee88a712a1f5069d58a0be4a6cbfaeab861/pynet/nn/__pycache__/FC.cpython-37.pyc -------------------------------------------------------------------------------- /pynet/nn/__pycache__/GAP.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deep-learning-algorithm/PyNet/354c7ee88a712a1f5069d58a0be4a6cbfaeab861/pynet/nn/__pycache__/GAP.cpython-37.pyc -------------------------------------------------------------------------------- /pynet/nn/__pycache__/Layer.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deep-learning-algorithm/PyNet/354c7ee88a712a1f5069d58a0be4a6cbfaeab861/pynet/nn/__pycache__/Layer.cpython-37.pyc -------------------------------------------------------------------------------- /pynet/nn/__pycache__/ReLU.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deep-learning-algorithm/PyNet/354c7ee88a712a1f5069d58a0be4a6cbfaeab861/pynet/nn/__pycache__/ReLU.cpython-37.pyc -------------------------------------------------------------------------------- /pynet/nn/__pycache__/__init__.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deep-learning-algorithm/PyNet/354c7ee88a712a1f5069d58a0be4a6cbfaeab861/pynet/nn/__pycache__/__init__.cpython-37.pyc -------------------------------------------------------------------------------- /pynet/nn/__pycache__/im2row.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deep-learning-algorithm/PyNet/354c7ee88a712a1f5069d58a0be4a6cbfaeab861/pynet/nn/__pycache__/im2row.cpython-37.pyc -------------------------------------------------------------------------------- /pynet/nn/__pycache__/pool2row.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deep-learning-algorithm/PyNet/354c7ee88a712a1f5069d58a0be4a6cbfaeab861/pynet/nn/__pycache__/pool2row.cpython-37.pyc -------------------------------------------------------------------------------- /pynet/nn/__pycache__/utils.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deep-learning-algorithm/PyNet/354c7ee88a712a1f5069d58a0be4a6cbfaeab861/pynet/nn/__pycache__/utils.cpython-37.pyc -------------------------------------------------------------------------------- /pynet/nn/im2row.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-5-25 下午4:17 4 | # @Author : zj 5 | 6 | from builtins import range 7 | import numpy as np 8 | import time 9 | 10 | 11 | def get_im2row_indices(x_shape, field_height, field_width, padding=1, stride=1): 12 | # First figure out what the size of the output should be 13 | N, C, H, W = x_shape 14 | assert (H + 2 * padding - field_height) % stride == 0 15 | assert (W + 2 * padding - field_height) % stride == 0 16 | out_height = int((H + 2 * padding - field_height) / stride + 1) 17 | out_width = int((W + 2 * padding - field_width) / stride + 1) 18 | 19 | # 行坐标 20 | i0 = stride * np.repeat(np.arange(out_height), out_width) 21 | i1 = np.repeat(np.arange(field_height), field_width) 22 | i1 = np.tile(i1, C) 23 | 24 | # 列坐标 25 | j0 = stride * np.tile(np.arange(out_width), out_height) 26 | j1 = np.tile(np.arange(field_width), field_height * C) 27 | 28 | i = i0.reshape(-1, 1) + i1.reshape(1, -1) 29 | j = j0.reshape(-1, 1) + j1.reshape(1, -1) 30 | 31 | k = np.repeat(np.arange(C), field_height * field_width).reshape(1, -1) 32 | 33 | return (k, i, j) 34 | 35 | 36 | def im2row_indices(x, field_height, field_width, padding=1, stride=1): 37 | # Zero-pad the input 38 | p = padding 39 | x_padded = np.pad(x, ((0, 0), (0, 0), (p, p), (p, p)), mode='constant') 40 | 41 | k, i, j = get_im2row_indices(x.shape, field_height, field_width, padding, stride) 42 | 43 | rows = x_padded[:, k, i, j] 44 | C = x.shape[1] 45 | # 逐图像采集 46 | rows = rows.reshape(-1, field_height * field_width * C) 47 | return rows 48 | 49 | 50 | def row2im_indices(rows, x_shape, field_height=3, field_width=3, padding=1, stride=1, isstinct=False): 51 | N, C, H, W = x_shape 52 | H_padded, W_padded = H + 2 * padding, W + 2 * padding 53 | x_padded = np.zeros((N, C, H_padded, W_padded), dtype=rows.dtype) 54 | k, i, j = get_im2row_indices(x_shape, field_height, field_width, padding, 55 | stride) 56 | rows_reshaped = rows.reshape(N, -1, C * field_height * field_width) 57 | np.add.at(x_padded, (slice(None), k, i, j), rows_reshaped) 58 | 59 | if isstinct: 60 | # 计算叠加倍数，恢复原图 61 | x_ones = np.ones(x_padded.shape) 62 | rows_ones = x_ones[:, k, i, j] 63 | x_zeros = np.zeros(x_padded.shape) 64 | np.add.at(x_zeros, (slice(None), k, i, j), rows_ones) 65 | x_padded = x_padded / x_zeros 66 | 67 | if padding == 0: 68 | return x_padded 69 | 70 | return x_padded[:, :, padding:-padding, padding:-padding] 71 | -------------------------------------------------------------------------------- /pynet/nn/pool2row.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-5-26 下午1:45 4 | # @Author : zj 5 | 6 | from builtins import range 7 | import numpy as np 8 | import time 9 | 10 | 11 | def get_pool2row_indices(x_shape, field_height, field_width, stride=1): 12 | # First figure out what the size of the output should be 13 | N, C, H, W = x_shape 14 | assert (H - field_height) % stride == 0 15 | assert (W - field_height) % stride == 0 16 | out_height = int((H - field_height) / stride + 1) 17 | out_width = int((W - field_width) / stride + 1) 18 | 19 | # 行坐标 20 | i0 = stride * np.repeat(np.arange(out_height), out_width) 21 | i0 = np.tile(i0, C) 22 | i1 = np.repeat(np.arange(field_height), field_width) 23 | 24 | # 列坐标 25 | j0 = stride * np.tile(np.arange(out_width), out_height * C) 26 | j1 = np.tile(np.arange(field_width), field_height) 27 | 28 | i = i0.reshape(-1, 1) + i1.reshape(1, -1) 29 | j = j0.reshape(-1, 1) + j1.reshape(1, -1) 30 | 31 | k = np.repeat(np.arange(C), out_height * out_width).reshape(-1, 1) 32 | 33 | return (k, i, j) 34 | 35 | 36 | def pool2row_indices(x, field_height, field_width, stride=1): 37 | k, i, j = get_pool2row_indices(x.shape, field_height, field_width, stride) 38 | rows = x.copy()[:, k, i, j] 39 | 40 | return rows.reshape(-1, field_height * field_width) 41 | 42 | 43 | def row2pool_indices(rows, x_shape, field_height=2, field_width=2, stride=2, isstinct=False): 44 | N, C, H, W = x_shape 45 | x = np.zeros(x_shape, dtype=rows.dtype) 46 | k, i, j = get_pool2row_indices(x_shape, field_height, field_width, stride) 47 | rows_reshaped = rows.reshape(N, -1, field_height * field_width) 48 | np.add.at(x, (slice(None), k, i, j), rows_reshaped) 49 | 50 | if isstinct and (stride < field_height or stride < field_width): 51 | x_ones = np.ones(x.shape) 52 | rows_ones = x_ones[:, k, i, j] 53 | x_zeros = np.zeros(x.shape) 54 | np.add.at(x_zeros, (slice(None), k, i, j), rows_ones) 55 | return x / x_zeros 56 | 57 | return x 58 | 59 | 60 | if __name__ == '__main__': 61 | # x = np.arange(32).reshape(2, 1, 4, 4) 62 | # print(x) 63 | x = np.ones((128, 3, 32, 32)) 64 | rows = pool2row_indices(x, 4, 4, stride=2) 65 | # print(rows) 66 | print(rows.shape) 67 | pool = row2pool_indices(rows, x.shape, field_height=4, field_width=4, stride=2, isstinct=True) 68 | # print(pool) 69 | print(np.sum(x == pool)) 70 | -------------------------------------------------------------------------------- /pynet/nn/utils.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-5-27 上午11:16 4 | # @Author : zj 5 | 6 | 7 | def conv_fc2output(inputs, batch_size, out_height, out_width): 8 | output = inputs.copy() 9 | # [N*H*W, C] 10 | local_connect_size, depth = output.shape[:2] 11 | # [N*H*W, C] -> [N, H, W, C] 12 | output = output.reshape(batch_size, out_height, out_width, depth) 13 | # [N, H, W, C] -> [N, C, H, W] 14 | return output.transpose((0, 3, 1, 2)) 15 | 16 | 17 | def conv_output2fc(inputs): 18 | output = inputs.copy() 19 | # [N, C, H, W] 20 | num, depth, height, width = output.shape[:4] 21 | 22 | # [N,C,H,W] —> [N,C,H*W] 23 | output = output.reshape(num, depth, -1) 24 | # [N,C,H*W] -> [N,H*W,C] 25 | output = output.transpose(0, 2, 1) 26 | # [N,H*W,C] -> [N*H*W,C] 27 | return output.reshape(-1, depth) 28 | 29 | 30 | def pool_fc2output(inputs, batch_size, out_height, out_width): 31 | output = inputs.copy() 32 | # [N*C*H*W] -> [N, C, H, W] 33 | return output.reshape(batch_size, -1, out_height, out_width) 34 | 35 | 36 | def pool_output2fc(inputs): 37 | return inputs.copy().reshape(-1) 38 | -------------------------------------------------------------------------------- /pynet/optim/__init__.py: -------------------------------------------------------------------------------- 1 | from .optimizer import Optimizer 2 | from .sgd import SGD 3 | 4 | from .lr_scheduler import * -------------------------------------------------------------------------------- /pynet/optim/__pycache__/__init__.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deep-learning-algorithm/PyNet/354c7ee88a712a1f5069d58a0be4a6cbfaeab861/pynet/optim/__pycache__/__init__.cpython-37.pyc -------------------------------------------------------------------------------- /pynet/optim/__pycache__/sgd.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deep-learning-algorithm/PyNet/354c7ee88a712a1f5069d58a0be4a6cbfaeab861/pynet/optim/__pycache__/sgd.cpython-37.pyc -------------------------------------------------------------------------------- /pynet/optim/lr_scheduler/LRScheduler.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-7-2 下午4:28 4 | # @Author : zj 5 | 6 | 7 | from abc import ABCMeta, abstractmethod 8 | from pynet.optim import Optimizer 9 | 10 | 11 | class LRScheduler(metaclass=ABCMeta): 12 | 13 | def __init__(self, optimizer, last_epoch=-1): 14 | if not isinstance(optimizer, Optimizer): 15 | raise TypeError('{} is not an Optimizer'.format(type(optimizer).__name__)) 16 | self.optimizer = optimizer 17 | if last_epoch == -1: 18 | for group in optimizer.optim_configs.values(): 19 | group.setdefault('initial_lr', group['lr']) 20 | else: 21 | for i, group in enumerate(optimizer.optim_configs.values()): 22 | if 'initial_lr' not in group: 23 | raise KeyError("param 'initial_lr' is not specified " 24 | "in param_groups[{}] when resuming an optimizer".format(i)) 25 | self.base_lrs = list(map(lambda group: group['initial_lr'], optimizer.optim_configs.values())) 26 | self.step(last_epoch + 1) 27 | self.last_epoch = last_epoch 28 | 29 | @abstractmethod 30 | def get_lr(self): 31 | pass 32 | 33 | def step(self, epoch=None): 34 | if epoch is None: 35 | epoch = self.last_epoch + 1 36 | self.last_epoch = epoch 37 | for param_group, lr in zip(self.optimizer.optim_configs.values(), self.get_lr()): 38 | param_group['lr'] = lr 39 | -------------------------------------------------------------------------------- /pynet/optim/lr_scheduler/StepLR.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-7-2 下午4:27 4 | # @Author : zj 5 | 6 | from .LRScheduler import LRScheduler 7 | 8 | 9 | class StepLR(LRScheduler): 10 | 11 | def __init__(self, optimizer, step_size, gamma=0.1, last_epoch=-1): 12 | self.step_size = step_size 13 | self.gamma = gamma 14 | 15 | super(StepLR, self).__init__(optimizer, last_epoch) 16 | 17 | def get_lr(self): 18 | return [base_lr * self.gamma ** (self.last_epoch // self.step_size) for base_lr in self.base_lrs] 19 | -------------------------------------------------------------------------------- /pynet/optim/lr_scheduler/__init__.py: -------------------------------------------------------------------------------- 1 | from .LRScheduler import LRScheduler 2 | from .StepLR import StepLR 3 | -------------------------------------------------------------------------------- /pynet/optim/optimizer.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-7-2 下午4:30 4 | # @Author : zj 5 | 6 | 7 | from abc import ABCMeta, abstractmethod 8 | 9 | __all__ = ['Optimizer'] 10 | 11 | 12 | class Optimizer(metaclass=ABCMeta): 13 | 14 | def __init__(self, params, defaults): 15 | self.params = params 16 | 17 | self.optim_configs = {} 18 | for p in params.keys(): 19 | # d = {k: v for k, v in defaults.items()} 20 | d = defaults.copy() 21 | self.optim_configs[p] = d 22 | 23 | @abstractmethod 24 | def step(self, grad): 25 | pass 26 | -------------------------------------------------------------------------------- /pynet/optim/sgd.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-7-1 下午8:04 4 | # @Author : zj 5 | 6 | import numpy as np 7 | from .optimizer import Optimizer 8 | 9 | 10 | class SGD(Optimizer): 11 | 12 | def __init__(self, params, lr=1e-3, momentum=0, nesterov=False): 13 | if isinstance(lr, float) and lr < 0.0: 14 | raise ValueError("Invalid learning rate: {}".format(lr)) 15 | if momentum < 0.0: 16 | raise ValueError("Invalid momentum value: {}".format(momentum)) 17 | if nesterov is True and momentum < 0.0: 18 | raise ValueError("Invalid momentum value: {}".format(momentum)) 19 | 20 | # self.params = params 21 | self.use_momentum = momentum != 0 22 | self.nesterov = nesterov 23 | defaults = dict(lr=lr, momentum=momentum) 24 | 25 | super(SGD, self).__init__(params, defaults) 26 | 27 | # self.optim_configs = {} 28 | # for p in params.keys(): 29 | # d = {k: v for k, v in defaults.items()} 30 | # self.optim_configs[p] = d 31 | 32 | def step(self, grad): 33 | assert isinstance(grad, dict) 34 | for p, w in self.params.items(): 35 | dw = grad[p] 36 | config = self.optim_configs[p] 37 | next_w, next_config = self._sgd(w, dw, config, is_weight=('W' in p)) 38 | self.params[p] = next_w 39 | self.optim_configs[p] = next_config 40 | 41 | def _sgd(self, w, dw, config, is_weight=True): 42 | 43 | if self.use_momentum and is_weight: 44 | v_prev = config.get('velocity', np.zeros_like(w)) 45 | 46 | if self.nesterov: 47 | v = config['momentum'] * v_prev - config['lr'] * dw 48 | next_w = w + (1 + config['momentum']) * v - config['momentum'] * v_prev 49 | else: 50 | v = config['momentum'] * v_prev - config['lr'] * dw 51 | next_w = w + v 52 | 53 | config['velocity'] = v 54 | else: 55 | next_w = w - config['lr'] * dw 56 | 57 | return next_w, config 58 | -------------------------------------------------------------------------------- /pynet/requirements.txt: -------------------------------------------------------------------------------- 1 | # Requirements automatically generated by pigar. 2 | # https://github.com/damnever/pigar 3 | 4 | # vision/draw.py: 7 5 | matplotlib == 3.1.2 6 | 7 | # models/FCNet.py: 16 8 | # nn/BN.py: 6 9 | # nn/Dropout.py: 6 10 | # nn/Dropout2d.py: 6 11 | # nn/GAP.py: 7 12 | # nn/ReLU.py: 6 13 | # nn/Softmax.py: 6 14 | # nn/im2row.py: 7 15 | # nn/pool2row.py: 7 16 | # optim/sgd.py: 6 17 | # solver.py: 6 18 | # vision/accuracy.py: 7 19 | # vision/data/cifar.py: 6 20 | # vision/data/iris.py: 7 21 | # vision/data/mnist.py: 8 22 | # vision/data/orl.py: 7 23 | # vision/data/xor.py: 7 24 | numpy == 1.17.2 25 | 26 | # vision/data/iris.py: 6 27 | # vision/data/orl.py: 9 28 | pandas == 0.25.3 29 | 30 | # vision/data/cifar.py: 8 31 | # vision/data/iris.py: 8,9 32 | # vision/data/orl.py: 10 33 | scikit_learn == 0.21.3 34 | -------------------------------------------------------------------------------- /pynet/solver.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-6-29 下午8:33 4 | # @Author : zj 5 | 6 | import numpy as np 7 | import time 8 | 9 | __all__ = ['Solver'] 10 | 11 | 12 | class Solver(object): 13 | 14 | def __init__(self, model, data, criterion, optimizer, **kwargs): 15 | self.model = model 16 | self.X_train = data['X_train'] 17 | self.y_train = data['y_train'] 18 | self.X_val = data['X_val'] 19 | self.y_val = data['y_val'] 20 | self.criterion = criterion 21 | self.optimizer = optimizer 22 | 23 | self.lr_scheduler = kwargs.pop('lr_scheduler', None) 24 | self.batch_size = kwargs.pop('batch_size', 8) 25 | self.num_epochs = kwargs.pop('num_epochs', 10) 26 | self.reg = kwargs.pop('reg', 1e-3) 27 | self.use_reg = self.reg != 0 28 | 29 | self.print_every = kwargs.pop('print_every', 1) 30 | 31 | if len(kwargs) > 0: 32 | extra = ', '.join('"%s"' % k for k in list(kwargs.keys())) 33 | raise ValueError('未识别参数: %s' % extra) 34 | 35 | self._reset() 36 | 37 | def _reset(self): 38 | self.current_epoch = 0 39 | self.best_train_acc = 0 40 | self.best_val_acc = 0 41 | self.best_params = {} 42 | self.loss_history = [] 43 | self.train_acc_history = [] 44 | self.val_acc_history = [] 45 | 46 | def _step(self, X_batch, y_batch): 47 | scores = self.model.forward(X_batch) 48 | loss, probs = self.criterion.forward(scores, y_batch) 49 | if self.use_reg: 50 | for k in self.model.params.keys(): 51 | if 'W' in k: 52 | loss += 0.5 * self.reg * np.sum(self.model.params[k] ** 2) 53 | 54 | grad_out = self.criterion.backward(probs, y_batch) 55 | grad = self.model.backward(grad_out) 56 | if self.use_reg: 57 | for k in grad.keys(): 58 | if 'W' in k: 59 | grad[k] += self.reg * self.model.params[k] 60 | 61 | self.optimizer.step(grad) 62 | 63 | return loss 64 | 65 | def check_accuracy(self, X, y, num_samples=None, batch_size=8): 66 | """ 67 | 精度测试，如果num_samples小于X长度，则从X中采样num_samples个图片进行检测 68 | """ 69 | N = X.shape[0] 70 | if num_samples is not None and N > num_samples: 71 | mask = np.random.choice(N, num_samples) 72 | N = num_samples 73 | X = X[mask] 74 | y = y[mask] 75 | 76 | if N < batch_size: 77 | batch_size = N 78 | num_batches = N // batch_size 79 | if N % batch_size != 0: 80 | num_batches += 1 81 | 82 | y_pred = [] 83 | for i in range(num_batches): 84 | start = i * batch_size 85 | end = (i + 1) * batch_size 86 | scores = self.model.forward(X[start:end]) 87 | y_pred.extend(np.argmax(scores, axis=1)) 88 | acc = np.mean(y_pred == y) 89 | 90 | return acc 91 | 92 | def train(self): 93 | num_train = self.X_train.shape[0] 94 | iterations_per_epoch = max(num_train // self.batch_size, 1) 95 | 96 | for i in range(self.num_epochs): 97 | self.current_epoch = i + 1 98 | start = time.time() 99 | total_loss = 0. 100 | self.model.train() 101 | for j in range(iterations_per_epoch): 102 | idx_start = j * self.batch_size 103 | idx_end = (j + 1) * self.batch_size 104 | X_batch = self.X_train[idx_start:idx_end] 105 | y_batch = self.y_train[idx_start:idx_end] 106 | 107 | total_loss += self._step(X_batch, y_batch) 108 | end = time.time() 109 | if self.lr_scheduler is not None: 110 | self.lr_scheduler.step() 111 | 112 | if self.current_epoch % self.print_every == 0: 113 | avg_loss = total_loss / iterations_per_epoch 114 | self.loss_history.append(float('%.6f' % avg_loss)) 115 | print('epoch: %d time: %.2f loss: %.6f' % (self.current_epoch, end - start, avg_loss)) 116 | 117 | self.model.eval() 118 | train_acc = self.check_accuracy(self.X_train, self.y_train, batch_size=self.batch_size) 119 | val_acc = self.check_accuracy(self.X_val, self.y_val, batch_size=self.batch_size) 120 | self.train_acc_history.append(train_acc) 121 | self.val_acc_history.append(val_acc) 122 | print('train acc: %.4f; val_acc: %.4f' % (train_acc, val_acc)) 123 | 124 | if val_acc >= self.best_val_acc and train_acc > self.best_train_acc: 125 | self.best_train_acc = train_acc 126 | self.best_val_acc = val_acc 127 | self.best_params = dict() 128 | for k, v in self.model.params.items(): 129 | self.best_params[k] = v.copy() 130 | 131 | # At the end of training swap the best params into the model 132 | self.model.params = self.best_params 133 | -------------------------------------------------------------------------------- /pynet/vision/__init__.py: -------------------------------------------------------------------------------- 1 | from .draw import Draw 2 | from .accuracy import Accuracy 3 | -------------------------------------------------------------------------------- /pynet/vision/__pycache__/__init__.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deep-learning-algorithm/PyNet/354c7ee88a712a1f5069d58a0be4a6cbfaeab861/pynet/vision/__pycache__/__init__.cpython-37.pyc -------------------------------------------------------------------------------- /pynet/vision/accuracy.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-6-20 下午3:30 4 | # @Author : zj 5 | 6 | 7 | import numpy as np 8 | 9 | 10 | class Accuracy(object): 11 | 12 | def __init__(self): 13 | pass 14 | 15 | def __call__(self, scores, labels): 16 | return self.forward(scores, labels) 17 | 18 | def forward(self, scores, labels): 19 | predicted = np.argmax(scores, axis=1) 20 | return np.mean(predicted == labels), predicted 21 | 22 | def check_accuracy(self, data_array, labels_array, net, batch_size=128): 23 | total_accuracy = 0.0 24 | num = 0 25 | 26 | range_list = np.arange(0, data_array.shape[0] - batch_size, step=batch_size) 27 | for i in range_list: 28 | data = data_array[i:i + batch_size] 29 | labels = labels_array[i:i + batch_size] 30 | 31 | scores = net.predict(data) 32 | predicted = np.argmax(scores, axis=1) 33 | total_accuracy += np.mean(predicted == labels) 34 | num += 1 35 | 36 | return total_accuracy / num 37 | -------------------------------------------------------------------------------- /pynet/vision/data/__init__.py: -------------------------------------------------------------------------------- 1 | from .xor import * 2 | from .iris import * 3 | from .cifar import * 4 | from .orl import * 5 | from .mnist import * 6 | 7 | """ 8 | 加载图像数据统一格式为 [N, C, H, W] 9 | 10 | 图像数据集加载完成后分为训练集、验证集和测试集 11 | """ 12 | -------------------------------------------------------------------------------- /pynet/vision/data/__pycache__/__init__.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deep-learning-algorithm/PyNet/354c7ee88a712a1f5069d58a0be4a6cbfaeab861/pynet/vision/data/__pycache__/__init__.cpython-37.pyc -------------------------------------------------------------------------------- /pynet/vision/data/__pycache__/cifar.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deep-learning-algorithm/PyNet/354c7ee88a712a1f5069d58a0be4a6cbfaeab861/pynet/vision/data/__pycache__/cifar.cpython-37.pyc -------------------------------------------------------------------------------- /pynet/vision/data/__pycache__/iris.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deep-learning-algorithm/PyNet/354c7ee88a712a1f5069d58a0be4a6cbfaeab861/pynet/vision/data/__pycache__/iris.cpython-37.pyc -------------------------------------------------------------------------------- /pynet/vision/data/__pycache__/mnist.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deep-learning-algorithm/PyNet/354c7ee88a712a1f5069d58a0be4a6cbfaeab861/pynet/vision/data/__pycache__/mnist.cpython-37.pyc -------------------------------------------------------------------------------- /pynet/vision/data/__pycache__/orl.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deep-learning-algorithm/PyNet/354c7ee88a712a1f5069d58a0be4a6cbfaeab861/pynet/vision/data/__pycache__/orl.cpython-37.pyc -------------------------------------------------------------------------------- /pynet/vision/data/__pycache__/utils.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deep-learning-algorithm/PyNet/354c7ee88a712a1f5069d58a0be4a6cbfaeab861/pynet/vision/data/__pycache__/utils.cpython-37.pyc -------------------------------------------------------------------------------- /pynet/vision/data/__pycache__/xor.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/deep-learning-algorithm/PyNet/354c7ee88a712a1f5069d58a0be4a6cbfaeab861/pynet/vision/data/__pycache__/xor.cpython-37.pyc -------------------------------------------------------------------------------- /pynet/vision/data/cifar.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-6-20 下午7:22 4 | # @Author : zj 5 | 6 | import numpy as np 7 | import os 8 | from sklearn.model_selection import train_test_split 9 | from .utils import * 10 | 11 | train_list = ['data_batch_1', 'data_batch_2', 'data_batch_3', 'data_batch_4', 'data_batch_5'] 12 | 13 | test_batch = 'test_batch' 14 | 15 | 16 | def load_CIFAR_batch(file_path): 17 | """ load single batch of cifar """ 18 | data_dict = load_pickle(file_path) 19 | data = data_dict['data'] 20 | labels = data_dict['labels'] 21 | 22 | data = data.reshape(10000, 3, 32, 32).astype("float") 23 | labels = np.array(labels) 24 | return data, labels 25 | 26 | 27 | def load_CIFAR10(file_dir): 28 | """ load all of cifar """ 29 | xs = [] 30 | ys = [] 31 | for filename in train_list: 32 | file_path = os.path.join(file_dir, filename) 33 | data, labels = load_CIFAR_batch(file_path) 34 | xs.append(data) 35 | ys.append(labels) 36 | x_train = np.concatenate(xs) 37 | y_train = np.concatenate(ys) 38 | 39 | x_test, y_test = load_CIFAR_batch(os.path.join(file_dir, test_batch)) 40 | 41 | return x_train, y_train, x_test, y_test 42 | 43 | 44 | def get_CIFAR10_data(cifar_dir, val_size=0.05, normalize=True): 45 | """ 46 | 加载CIFAR10数据，从训练集中分类验证集数据 47 | :param cifar_dir: cifar解压文件路径 48 | :param val_size: 浮点数，表示验证集占整个训练集的百分比 49 | :param normalize: 是否初始化为零均值，1方差 50 | :return: dict，保存训练集、验证集以及测试集的数据和标签 51 | """ 52 | x_train, y_train, x_test, y_test = load_CIFAR10(cifar_dir) 53 | 54 | x_train, x_val, y_train, y_val = train_test_split(x_train, y_train, test_size=val_size, shuffle=False) 55 | 56 | # Normalize the data: subtract the mean image and divide the variance 57 | if normalize: 58 | x_train = x_train / 255 - 0.5 59 | x_val = x_val / 255 - 0.5 60 | x_test = x_test / 255 - 0.5 61 | 62 | # Package data into a dictionary 63 | return { 64 | 'X_train': x_train, 'y_train': y_train, 65 | 'X_val': x_val, 'y_val': y_val, 66 | 'X_test': x_test, 'y_test': y_test, 67 | } 68 | -------------------------------------------------------------------------------- /pynet/vision/data/iris.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-6-20 下午4:25 4 | # @Author : zj 5 | 6 | import pandas as pd 7 | import numpy as np 8 | from sklearn import utils 9 | from sklearn.model_selection import train_test_split 10 | 11 | 12 | # iris_path = '/home/zj/data/iris-species/Iris.csv' 13 | 14 | 15 | def load_iris(iris_path, shuffle=True, tsize=0.8): 16 | """ 17 | 加载iris数据 18 | """ 19 | data = pd.read_csv(iris_path, header=0, delimiter=',') 20 | 21 | if shuffle: 22 | data = utils.shuffle(data) 23 | 24 | species_dict = { 25 | 'Iris-setosa': 0, 26 | 'Iris-versicolor': 1, 27 | 'Iris-virginica': 2 28 | } 29 | data['Species'] = data['Species'].map(species_dict) 30 | 31 | data_x = np.array( 32 | [data['SepalLengthCm'], data['SepalWidthCm'], data['PetalLengthCm'], data['PetalWidthCm']]).T 33 | data_y = data['Species'] 34 | 35 | x_train, x_test, y_train, y_test = train_test_split(data_x, data_y, train_size=tsize, test_size=(1 - tsize), 36 | shuffle=False) 37 | 38 | return x_train, x_test, y_train, y_test 39 | -------------------------------------------------------------------------------- /pynet/vision/data/mnist.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-6-20 下午7:50 4 | # @Author : zj 5 | 6 | from .utils import * 7 | import os 8 | import numpy as np 9 | 10 | 11 | def load_mnist(mnist_path, dst_size=None, shuffle=True, is_flatten=False): 12 | """ 13 | 加载mnist数据 14 | """ 15 | train_dir = os.path.join(mnist_path, 'train') 16 | test_dir = os.path.join(mnist_path, 'test') 17 | 18 | x_train = [] 19 | x_test = [] 20 | y_train = [] 21 | y_test = [] 22 | train_file_list = [] 23 | cate_list = list(range(10)) 24 | for i in cate_list: 25 | data_dir = os.path.join(train_dir, str(i)) 26 | file_list = os.listdir(data_dir) 27 | for filename in file_list: 28 | file_path = os.path.join(data_dir, filename) 29 | train_file_list.append(file_path) 30 | 31 | # 读取测试集图像 32 | data_dir = os.path.join(test_dir, str(i)) 33 | file_list = os.listdir(data_dir) 34 | for filename in file_list: 35 | file_path = os.path.join(data_dir, filename) 36 | img = read_image(file_path, is_gray=True) 37 | if img is not None: 38 | if dst_size is not None: 39 | img = resize_image(img, dst_size=dst_size) 40 | if is_flatten: 41 | x_test.append(img.reshape(-1)) 42 | else: 43 | h, w = img.shape[:2] 44 | x_test.append(img.reshape(1, h, w)) 45 | y_test.append(i) 46 | 47 | train_file_list = np.array(train_file_list) 48 | if shuffle: 49 | np.random.shuffle(train_file_list) 50 | 51 | # 读取训练集图像 52 | for file_path in train_file_list: 53 | img = read_image(file_path, is_gray=True) 54 | if img is not None: 55 | if dst_size is not None: 56 | img = resize_image(img, dst_size=dst_size) 57 | 58 | if is_flatten: 59 | x_train.append(img.reshape(-1)) 60 | else: 61 | h, w = img.shape[:2] 62 | x_train.append(img.reshape(1, h, w)) 63 | y_train.append(int(os.path.split(file_path)[0].split('/')[-1])) 64 | 65 | return np.array(x_train), np.array(x_test), np.array(y_train), np.array(y_test) 66 | -------------------------------------------------------------------------------- /pynet/vision/data/orl.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-6-20 下午7:37 4 | # @Author : zj 5 | 6 | from .utils import * 7 | import numpy as np 8 | import os 9 | import pandas as pd 10 | from sklearn.model_selection import train_test_split 11 | import warnings 12 | 13 | warnings.filterwarnings('ignore') 14 | 15 | 16 | def load_orl(orl_path, shuffle=True, dst_size=(23, 28)): 17 | """ 18 | 加载ORL数据 19 | """ 20 | cate_list = [] 21 | data_list = [] 22 | cate_dict = dict() 23 | 24 | num = 0 25 | file_list = os.listdir(orl_path) 26 | for file_name in file_list: 27 | file_dir = os.path.join(orl_path, file_name) 28 | img_list = os.listdir(file_dir) 29 | for img_name in img_list: 30 | img_path = os.path.join(file_dir, img_name) 31 | img = read_image(img_path, is_gray=True) 32 | resized_img = resize_image(img, dst_size=dst_size) 33 | 34 | data_list.append(resized_img.reshape(-1)) 35 | # data_list.append(img.reshape(-1)) 36 | cate_list.append(file_name) 37 | cate_dict[file_name] = num 38 | num += 1 39 | 40 | x_train, x_test, y_train, y_test = train_test_split(np.array(data_list), np.array(cate_list), train_size=0.8, 41 | test_size=0.2, shuffle=shuffle) 42 | 43 | y_train_labels = pd.DataFrame(y_train)[0].map(cate_dict).values 44 | y_test_labels = pd.DataFrame(y_test)[0].map(cate_dict).values 45 | 46 | return x_train, x_test, y_train_labels, y_test_labels 47 | -------------------------------------------------------------------------------- /pynet/vision/data/utils.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-6-20 下午7:25 4 | # @Author : zj 5 | 6 | 7 | import cv2 8 | import pickle 9 | 10 | 11 | def read_image(img_path, is_gray=False): 12 | if is_gray: 13 | return cv2.imread(img_path, cv2.IMREAD_GRAYSCALE) 14 | else: 15 | return cv2.imread(img_path) 16 | 17 | 18 | def resize_image(src, dst_size): 19 | if src.shape == dst_size: 20 | return src 21 | return cv2.resize(src, dst_size) 22 | 23 | 24 | def load_pickle(file_path): 25 | with open(file_path, 'rb') as f: 26 | data_dict = pickle.load(f, encoding='latin1') 27 | return data_dict 28 | -------------------------------------------------------------------------------- /pynet/vision/data/xor.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-6-20 下午4:18 4 | # @Author : zj 5 | 6 | 7 | import numpy as np 8 | 9 | 10 | def load_xor(): 11 | xor_data = np.array([[0, 0], [0, 1], [1, 0], [1, 1]]) 12 | xor_labels = np.array([0, 1, 1, 0]) 13 | 14 | return xor_data, xor_labels 15 | -------------------------------------------------------------------------------- /pynet/vision/draw.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # @Time : 19-6-20 下午3:40 4 | # @Author : zj 5 | 6 | 7 | import matplotlib.pyplot as plt 8 | 9 | plt.rcParams['font.sans-serif'] = ['simhei'] # 用来正常显示中文标签 10 | plt.rcParams['axes.unicode_minus'] = False # 用来正常显示负号 11 | 12 | 13 | class Draw(object): 14 | 15 | def __call__(self, values, title='损失图', xlabel='迭代/次', ylabel='损失值'): 16 | self.forward(values, title='损失图', xlabel='迭代/次', ylabel='损失值') 17 | 18 | def forward(self, values, title='损失图', xlabel='迭代/次', ylabel='损失值', save_path='./loss.png'): 19 | assert isinstance(values, list) 20 | f = plt.figure() 21 | plt.title(title) 22 | plt.ylabel(ylabel) 23 | plt.xlabel(xlabel) 24 | plt.plot(values) 25 | plt.savefig(save_path) 26 | plt.show() 27 | 28 | def multi_plot(self, values_list, labels_list, title='损失图', xlabel='迭代/次', ylabel='损失值', save_path='./loss.png'): 29 | assert isinstance(values_list, tuple) 30 | assert isinstance(labels_list, tuple) 31 | assert len(values_list) == len(labels_list) 32 | 33 | f = plt.figure() 34 | plt.title(title) 35 | plt.ylabel(ylabel) 36 | plt.xlabel(xlabel) 37 | for i in range(len(values_list)): 38 | plt.plot(values_list[i], label=labels_list[i]) 39 | plt.legend() 40 | plt.savefig(save_path) 41 | plt.show() 42 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | mkdocs==1.0.4 2 | mkdocs-material==4.6.0 3 | mkdocs-minify-plugin==0.2.1 4 | Markdown==3.1.1 5 | markdown-katex==201912.11b0 --------------------------------------------------------------------------------