├── .idea └── vcs.xml ├── Books ├── PyTorch 模型训练实用教程(余霆嵩).pdf ├── PyTorch框架班作业汇总（第一期）.docx ├── PyTorch框架班作业汇总（第一期）.pdf ├── PyTorch框架班作业汇总（第二期）.docx ├── PyTorch框架班作业汇总（第二期）.pdf ├── PyTorch框架班作业（第一期）-问题汇总.docx └── PyTorch框架班作业（第一期）-问题汇总.pdf ├── README.md ├── docs ├── chapter1 │ ├── 1.1-pytorch-introduction.md │ ├── 1.2-pytorch-installation.md │ ├── 1.3-deep-learning-with-pytorch-60-minute-blitz.md │ ├── 1.4-pytorch-resource.md │ ├── 1_tensor_tutorial.ipynb │ ├── 2_autograd_tutorial.ipynb │ ├── 3_neural_networks_tutorial.ipynb │ ├── 4_cifar10_tutorial.ipynb │ ├── 5_data_parallel_tutorial.ipynb │ ├── autograd_tutorial.ipynb │ ├── cifar10_tutorial.ipynb │ ├── data_parallel_tutorial.ipynb │ ├── neural_networks_tutorial.ipynb │ ├── pic1.png │ ├── readme.md │ └── tensor_tutorial.ipynb ├── chapter2 │ ├── 1.png │ ├── 10.png │ ├── 11.png │ ├── 12.png │ ├── 2.1.1.pytorch-basics-tensor.ipynb │ ├── 2.1.2-pytorch-basics-autograd.ipynb │ ├── 2.1.3-pytorch-basics-nerual-network.ipynb │ ├── 2.1.4-pytorch-basics-data-lorder.ipynb │ ├── 2.2-deep-learning-basic-mathematics.ipynb │ ├── 2.3-deep-learning-neural-network-introduction.ipynb │ ├── 2.4-cnn.ipynb │ ├── 2.5-rnn.ipynb │ ├── 2.png │ ├── 3.png │ ├── 4.png │ ├── 5.png │ ├── 6.png │ ├── 7.png │ ├── 8.png │ ├── 9.gif │ ├── alexnet.png │ ├── cnn.png │ ├── googlenet.png │ ├── gru.gif │ ├── lenet5.jpg │ ├── lstm.gif │ ├── median_benchmark.csv │ ├── readme.md │ ├── resnet.png │ ├── resnet18.jpg │ └── vgg16.png ├── chapter3 │ ├── 3.1-logistic-regression.ipynb │ ├── 3.2-mnist.ipynb │ ├── 3.3-rnn.ipynb │ ├── german.data-numeric │ └── readme.md ├── chapter4 │ ├── 1.PNG │ ├── 1280px-Felis_silvestris_catus_lying_on_rice_straw.jpg │ ├── 2.PNG │ ├── 3.PNG │ ├── 4.1-fine-tuning.ipynb │ ├── 4.2.1-visdom.ipynb │ ├── 4.2.2-tensorboardx.ipynb │ ├── 4.2.3-cnn-visualizing.ipynb │ ├── 4.3-fastai.ipynb │ ├── 4.5-multiply-gpu-parallel-training.ipynb │ ├── readme.md │ └── static.zip └── chapter5 │ ├── 5.1-kaggle.md │ ├── kaggle.png │ └── readme.md ├── requirements.txt ├── screenshots ├── cuda_version.png ├── python_version.png └── test_torch.png └── src ├── PyTorch_Tutorial ├── Code │ ├── 1_data_prepare │ │ ├── 1_1_cifar10_to_png.py │ │ ├── 1_2_split_dataset.py │ │ ├── 1_3_generate_txt.py │ │ ├── 1_3_mydataset.py │ │ ├── 1_5_compute_mean.py │ │ └── 1_5_compute_mean_bak.py │ ├── 2_model │ │ ├── 2_finetune.py │ │ └── net_params.pkl │ ├── 3_optimizer │ │ ├── 3_1_lossFunction │ │ │ ├── 1.jpg │ │ │ ├── 1_L1Loss.py │ │ │ ├── 2_MSELoss.py │ │ │ ├── 3_CrossEntropyLoss.py │ │ │ ├── 4_NLLLoss.py │ │ │ ├── 5_PoissonNLLLoss.py │ │ │ └── 6_KLDivLoss.py │ │ └── 3_2_optimizer │ │ │ ├── 1_param_groups.py │ │ │ ├── 2_zero_grad.py │ │ │ ├── 3_state_dict.py │ │ │ ├── 4_load_state_dict.py │ │ │ ├── 5_add_param_group.py │ │ │ └── net_params.pkl │ ├── 4_viewer │ │ ├── 1_tensorboardX_demo.py │ │ ├── 2_visual_weights.py │ │ ├── 3_visual_featuremaps.py │ │ ├── 4_hist_grad_weight.py │ │ └── 5_Show_ConfMat.py │ ├── main_training │ │ ├── main.py │ │ └── main_bak.py │ └── utils │ │ ├── __init__.py │ │ └── utils.py ├── Data │ ├── cat.png │ ├── cover.png │ ├── net_params_72p.pkl │ ├── train.txt │ ├── valid.txt │ └── visual.txt ├── PyTorch_tutorial_0.0.5_余霆嵩.pdf ├── Result │ └── all_scalars.json └── requirements.txt ├── Week1 ├── Day1 │ ├── 0.开发环境准备.pdf │ ├── 1.自动求导机制.ipynb │ ├── 2.CUDA语义.ipynb │ └── 3.多进程最佳实践.ipynb ├── Day2 │ ├── 5.序列化语义.ipynb │ ├── 6.torch - PyTorch中文文档.pdf │ └── 6.torch-Learning-1-9.ipynb ├── Day3-Day5 │ └── 6 │ │ ├── 6.torch - PyTorch中文文档.pdf │ │ ├── 6.torch-Learning-10-20.ipynb │ │ ├── 6.torch-Learning-21-56.ipynb │ │ ├── 6.torch-Learning-57-85.ipynb │ │ └── tensor操作参考 │ │ ├── 1.tensor基础数据类型.pdf │ │ ├── 2.创建tensor.doc │ │ ├── 2.创建tensor.pdf │ │ ├── 3.索引和切片.doc │ │ ├── 3.索引和切片.pdf │ │ ├── 4.tensor变换.doc │ │ ├── 4.tensor变换.pdf │ │ ├── 5.Concat-split.pdf │ │ ├── 5.concat-split.doc │ │ ├── 6.math operation.pdf │ │ ├── 7.statisics.pdf │ │ ├── 9.tensor advanced.pdf │ │ └── 9.tensor_advanced.doc └── lesson1.py ├── Week2 ├── Day08 │ ├── 14.torch.cuda.ipynb │ └── 8.torch.Storage.ipynb ├── Day09~Day10 │ ├── 16.torch.utils.data.ipynb │ ├── 17.torch.utils.model_zoo .ipynb │ ├── 18.torchvision.datasets.ipynb │ ├── 19.torchvision.models.ipynb │ ├── 20.torchvision.transforms.ipynb │ └── 21.torchvision.utils.ipynb └── Day11~Day12 │ ├── 10.torch.nn.functional.ipynb │ └── 9.torch.nn - PyTorch中文文档.pdf ├── Week3 ├── Day15~Day16 │ └── 12.torch.nn.init.ipynb └── Day19 │ ├── 11.torch.optim.ipynb │ └── linear.pth └── Week5 ├── Day30 └── LogistRegression.py ├── Day31 └── MLP.py ├── Day32 └── CNN-Classification.py └── Day33 └── RNN-LSTM.py /.idea/vcs.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | -------------------------------------------------------------------------------- /Books/PyTorch 模型训练实用教程(余霆嵩).pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/Books/PyTorch 模型训练实用教程(余霆嵩).pdf -------------------------------------------------------------------------------- /Books/PyTorch框架班作业汇总（第一期）.docx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/Books/PyTorch框架班作业汇总（第一期）.docx -------------------------------------------------------------------------------- /Books/PyTorch框架班作业汇总（第一期）.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/Books/PyTorch框架班作业汇总（第一期）.pdf -------------------------------------------------------------------------------- /Books/PyTorch框架班作业汇总（第二期）.docx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/Books/PyTorch框架班作业汇总（第二期）.docx -------------------------------------------------------------------------------- /Books/PyTorch框架班作业汇总（第二期）.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/Books/PyTorch框架班作业汇总（第二期）.pdf -------------------------------------------------------------------------------- /Books/PyTorch框架班作业（第一期）-问题汇总.docx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/Books/PyTorch框架班作业（第一期）-问题汇总.docx -------------------------------------------------------------------------------- /Books/PyTorch框架班作业（第一期）-问题汇总.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/Books/PyTorch框架班作业（第一期）-问题汇总.pdf -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # 《Pytorch框架班》训练营 2 | 3 | ## 课程资料 4 | - [Pytorch中文文档地址](https://github.com/zergtant/pytorch-handbook)：本文档介绍了最新的Pytorch1.1的API 5 | - [《Pytorch模型训练实用教程》中配套代码](https://github.com/tensor-yu/PyTorch_Tutorial) 6 | - 本训练营的学习安排与课程任务：详见文件夹Books中的《PyTorch框架班作业（第一期）》doc文档 7 | 8 | ## 课程安排 9 | **总课时：6 周** 10 | 11 | ### 第一周 12 | - 1 Pytorch简介，配置电脑环境 13 | - 2 学习序列化模型、torch接口 14 | - 3 深入了解torch接口 15 | 16 | ### 第二周 17 | - 4 torch.Storage、torch.cuda操作 18 | - 5 数据读取、数据扩增 19 | - 6 构建网络模型 20 | 21 | ### 第三周 22 | - 7 网络模型参数初始化 23 | - 8 损失函数 24 | - 9 优化算法torch.optim 25 | 26 | ### 第四周 27 | - 10 设置学习率 28 | - 11 数据可视化 29 | 30 | ### 第五周 31 | - 12 Logistics分类实战 32 | - 13 多层感知机MLP实战 33 | - 14 卷积神经网络CNN：Resnet18 34 | - 15 递归神经网络RNN-LSTM：词性预测 35 | 36 | ### 第六周 37 | - 16 CNN卷积神经网络：定位检测实战 38 | - 17 GAN项目实战 39 | - 18 开始一个简单的图像分类比赛(kaggle 猫狗识别) 40 | - 19 期末总结 41 | 42 | ## 总结 43 |   老师给的是Pytorch0.4的API资料，配合Pytorch1.1的代码，能更好的学习API的使用。第五周的课程，如果采用单机运行，可能由于算力不够，需要等待很长的时间。建议搭建Google Colab，需要采用科学上网。 44 |   由于笔者还不会深度学习，只能先通过查看Pytorch的API示例，学习框架的使用，并逐步改写代码。目标下一次要完成《吴恩达深度学习教程》和《深度学习》花书。 45 |   requirements.txt文件是笔者自己的环境，配置环境如下： 46 | - CUDA安装的是10.0版本 47 | ![image](screenshots/cuda_version.png) 48 | - torch版本 49 | ![image](screenshots/test_torch.png) 50 | - python版本 51 | ![image](screenshots/python_version.png) -------------------------------------------------------------------------------- /docs/chapter1/1.1-pytorch-introduction.md: -------------------------------------------------------------------------------- 1 | # 1.1 Pytorch 简介 2 | 3 | ## 1.1.1 PyTorch的由来 4 | 很多人都会拿PyTorch和Google的Tensorflow进行比较，这个肯定是没有问题的，因为他们是最火的两个深度学习框架了。但是说到PyTorch，其实应该先说[Torch](http://torch.ch)。 5 | 6 | ## 1.1.2 Torch是什么？ 7 | 8 | **Torch英译中：火炬** 9 | 10 | A Tensor library like Numpy, unlike Numpy it has strong GPU support. Lua is a wrapper for Torch (Yes! you need to have a good understanding of Lua), and for that you will need LuaRocks package manager. 11 | [1](https://stackoverflow.com/questions/44371560/what-is-the-relationship-between-pytorch-and-torch) 12 | 13 | Torch是一个与Numpy类似的张量（Tensor）操作库，与Numpy不同的是Torch对GPU支持的很好，Lua是Torch的上层包装。 14 | 15 | Torch is not going anywhere. PyTorch and Torch use the same C libraries that contain all the performance: TH, THC, THNN, THCUNN and they will continue to be shared. 16 | We still and will have continued engineering on Torch itself, and we have no immediate plan to remove that. 17 | [2](https://discuss.pytorch.org/t/roadmap-for-torch-and-pytorch/38) 18 | 19 | PyTorch和Torch使用包含所有相同性能的C库：TH, THC, THNN, THCUNN，并且它们将继续共享这些库。 20 | 21 | 这样的回答就很明确了，其实PyTorch和Torch都使用的是相同的底层，只是使用了不同的上层包装语言。 22 | 23 | 注：LUA虽然快，但是太小众了，所以才会有PyTorch的出现。 24 | 25 | ## 1.1.3 重新介绍 PyTorch 26 | PyTorch is an open source machine learning library for Python, based on Torch, used for applications such as natural language processing. It is primarily developed by Facebook's artificial-intelligence research group, and Uber's "Pyro" software for probabilistic programming is built on it. 27 | [3](https://en.wikipedia.org/wiki/PyTorch) 28 | 29 | PyTorch是一个基于Torch的Python开源机器学习库，用于自然语言处理等应用程序。 它主要由Facebook的人工智能研究小组开发。Uber的"Pyro"也是使用的这个库。 30 | 31 | PyTorch is a Python package that provides two high-level features: 32 | 33 | Tensor computation (like NumPy) with strong GPU acceleration 34 | 35 | Deep neural networks built on a tape-based autograd system 36 | You can reuse your favorite Python packages such as NumPy, SciPy and Cython to extend PyTorch when needed. 37 | [4](https://github.com/pytorch/pytorch) 38 | 39 | PyTorch是一个Python包，提供两个高级功能： 40 | * 具有强大的GPU加速的张量计算（如NumPy） 41 | * 包含自动求导系统的的深度神经网络 42 | 43 | ## 1.1.4 对比PyTorch和Tensorflow 44 | 没有好的框架，只有合适的框架， [这篇知乎文章](https://zhuanlan.zhihu.com/p/28636490)有个简单的对比，所以这里就不详细再说了。 45 | 并且技术是发展的，知乎上的对比也不是绝对的，比如Tensorflow在1.5版的时候就引入了Eager Execution机制实现了动态图，PyTorch的可视化,windows支持，沿维翻转张量等问题都已经不是问题了。 46 | 47 | ## 1.1.5 再次总结 48 | 49 | - PyTorch算是相当简洁优雅且高效快速的框架 50 | - 设计追求最少的封装，尽量避免重复造轮子 51 | - 算是所有的框架中面向对象设计的最优雅的一个，设计最符合人们的思维，它让用户尽可能地专注于实现自己的想法 52 | - 大佬支持,与google的Tensorflow类似，FAIR的支持足以确保PyTorch获得持续的开发更新 53 | - 不错的的文档（相比FB的其他项目，PyTorch的文档简直算是完善了，参考Thrift），PyTorch作者亲自维护的论坛 供用户交流和求教问题 54 | - 入门简单 55 | 56 | 所以如果以上信息有吸引你的内容，那么请一定要读完这本书:) 57 | -------------------------------------------------------------------------------- /docs/chapter1/1.2-pytorch-installation.md: -------------------------------------------------------------------------------- 1 | # 1.2 Pytorch环境搭建 2 | PyTorch的安装十分简单，根据[PyTorch官网](https://pytorch.org/)，对系统选择和安装方式等灵活选择即可。 3 | 这里以[anaconda](https://www.anaconda.com/)为例，简单的说一下步骤和要点。 4 | 国内安装anaconda建议使用[中科大](http://mirrors.ustc.edu.cn/help/anaconda.html)镜像，快的不是一点半点。 5 | 6 | 目前中科大、清华镜像都已经关闭。直接通过prompt下载很慢，并且经常会出现HTTPERROR导致下载失败。 7 | 所以可以提前下载好压缩包，然后离线下载，这样就不容易出现下载到一半下载失败令人苦恼的情况。 8 | 9 | # 1.2.1 安装Pytorch 10 | anaconda安装完成后可以开始创建环境，这里以win10 系统为例。打开Anaconda Prompt 11 | ```bash 12 | #pytorch为环境名，这里创建python3.6版。 13 | conda create -n pytorch python=3.6 14 | # 切换到pytorch环境 15 | activate pytorch 16 | # ***以下为1.0版本安装*** 17 | #安装GPU版本，根据cuda版本选择cuda80，cuda92，如果cuda是9.0版，则不需要 18 | #直接conda install pytorch -c pytorch 即可 19 | # win下查看cuda版本命令nvcc -V 20 | conda install pytorch cuda92 -c pytorch 21 | # cpu版本使用 22 | # conda install pytorch-cpu -c pytorch 23 | 24 | # torchvision 是torch提供的计算机视觉工具包，后面介绍 25 | pip install torchvision 26 | 27 | 28 | # *** 官方更新了1.01 所以安装方式也有小的变更 29 | # torchversion提供了conda的安装包，可以用conda直接安装了 30 | # cuda支持也升级到了10.0 31 | # 安装方式如下： 32 | # cpu版本 33 | conda install pytorch-cpu torchvision-cpu -c pytorch 34 | # GPU版 35 | conda install pytorch torchvision cudatoolkit=10.0 -c pytorch 36 | # cudatoolkit后跟着相应的cuda版本 37 | # 目前测试 8.0、9.0、9.1、9.2、10.0都可安装成功 38 | 39 | 40 | ``` 41 | 42 | 验证输入python 进入 43 | ```python 44 | import torch 45 | torch.__version__ 46 | # 得到结果'0.4.1' 47 | ``` 48 | 49 | ## 1.2.2 配置 Jupyter Notebook 50 | 新建的环境是没有安装安装ipykernel的所以无法注册到Jupyter Notebook中，所以先要准备下环境 51 | ```bash 52 | #安装ipykernel 53 | conda install ipykernel 54 | #写入环境 55 | python -m ipykernel install --name pytorch --display-name "Pytorch for Deeplearning" 56 | ``` 57 | 下一步就是定制 Jupyter Notebook 58 | ```bash 59 | #切换回基础环境 60 | activate base 61 | #创建jupyter notebook配置文件 62 | jupyter notebook --generate-config 63 | ## 这里会显示创建jupyter_notebook_config.py的具体位置 64 | ``` 65 | 打开文件，修改 66 | ``` 67 | c.NotebookApp.notebook_dir = '' 默认目录位置 68 | c.NotebookApp.iopub_data_rate_limit = 100000000 这个改大一些否则有可能报错 69 | ``` 70 | 71 | ## 1.2.3 测试 72 | 至此 Pytorch 的开发环境安装完成，可以在开始菜单中打开Jupyter Notebook 在New 菜单中创建文件时选择`Pytorch for Deeplearning` 创建PyTorch的相关开发环境了 73 | 74 | ## 1.2.4 问题解决 75 | 76 | ### 问题1：启动python提示编码错误 77 | 78 | 删除 .python_history [来源](http://tantai.org/posts/install-keras-pytorch-jupyter-notebook-Anaconda-window-10-cpu/) 79 | ### 问题2 默认目录设置不起效 80 | 打开快捷方式，看看快捷方式是否跟这个截图一样，如果是则删除 `%USERPROFILE%` 改参数会覆盖掉notebook_dir设置，导致配置不起效 81 | 82 | ![Alt text](pic1.png) 83 | 84 | 如果你还发现其他问题，请直接留言 85 | 86 | 87 | -------------------------------------------------------------------------------- /docs/chapter1/1.3-deep-learning-with-pytorch-60-minute-blitz.md: -------------------------------------------------------------------------------- 1 | # PyTorch 深度学习:60分钟快速入门 （官方） 2 | 3 | 本章为官方网站的 4 | [Deep Learning with PyTorch: A 60 Minute Blitz] 5 | (https://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html) 6 | 的中文翻译，目前在网上看到所有中文翻译版本都已经过时了，所以才又从新翻译了一遍，确保与官方同步 7 | 8 | ## 目录 9 | 10 | 1. [张量](1_tensor_tutorial.ipynb) 11 | 2. [Autograd: 自动求导](2_autograd_tutorial.ipynb) **本章是冲突的重灾区，建议阅读** 12 | 3. [神经网络](3_neural_networks_tutorial.ipynb) 13 | 4. [训练一个分类器](4_cifar10_tutorial.ipynb) 14 | 5. [选读：数据并行处理(多GPU)](5_data_parallel_tutorial.ipynb) 15 | 16 | ## 说明 17 | 本章中的所有图片均来自于PyTorch官网，版权归PyTorch所有. -------------------------------------------------------------------------------- /docs/chapter1/1.4-pytorch-resource.md: -------------------------------------------------------------------------------- 1 | # 相关资源列表 -------------------------------------------------------------------------------- /docs/chapter1/autograd_tutorial.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "metadata": { 7 | "collapsed": false 8 | }, 9 | "outputs": [], 10 | "source": [ 11 | "%matplotlib inline" 12 | ] 13 | }, 14 | { 15 | "cell_type": "markdown", 16 | "metadata": {}, 17 | "source": [ 18 | "\nAutograd: Automatic Differentiation\n===================================\n\nCentral to all neural networks in PyTorch is the ``autograd`` package.\nLet\u2019s first briefly visit this, and we will then go to training our\nfirst neural network.\n\n\nThe ``autograd`` package provides automatic differentiation for all operations\non Tensors. It is a define-by-run framework, which means that your backprop is\ndefined by how your code is run, and that every single iteration can be\ndifferent.\n\nLet us see this in more simple terms with some examples.\n\nTensor\n--------\n\n``torch.Tensor`` is the central class of the package. If you set its attribute\n``.requires_grad`` as ``True``, it starts to track all operations on it. When\nyou finish your computation you can call ``.backward()`` and have all the\ngradients computed automatically. The gradient for this tensor will be\naccumulated into ``.grad`` attribute.\n\nTo stop a tensor from tracking history, you can call ``.detach()`` to detach\nit from the computation history, and to prevent future computation from being\ntracked.\n\nTo prevent tracking history (and using memory), you can also wrap the code block\nin ``with torch.no_grad():``. This can be particularly helpful when evaluating a\nmodel because the model may have trainable parameters with `requires_grad=True`,\nbut for which we don't need the gradients.\n\nThere\u2019s one more class which is very important for autograd\nimplementation - a ``Function``.\n\n``Tensor`` and ``Function`` are interconnected and build up an acyclic\ngraph, that encodes a complete history of computation. Each tensor has\na ``.grad_fn`` attribute that references a ``Function`` that has created\nthe ``Tensor`` (except for Tensors created by the user - their\n``grad_fn is None``).\n\nIf you want to compute the derivatives, you can call ``.backward()`` on\na ``Tensor``. If ``Tensor`` is a scalar (i.e. it holds a one element\ndata), you don\u2019t need to specify any arguments to ``backward()``,\nhowever if it has more elements, you need to specify a ``gradient``\nargument that is a tensor of matching shape.\n\n" 19 | ] 20 | }, 21 | { 22 | "cell_type": "code", 23 | "execution_count": null, 24 | "metadata": { 25 | "collapsed": false 26 | }, 27 | "outputs": [], 28 | "source": [ 29 | "import torch" 30 | ] 31 | }, 32 | { 33 | "cell_type": "markdown", 34 | "metadata": {}, 35 | "source": [ 36 | "Create a tensor and set requires_grad=True to track computation with it\n\n" 37 | ] 38 | }, 39 | { 40 | "cell_type": "code", 41 | "execution_count": null, 42 | "metadata": { 43 | "collapsed": false 44 | }, 45 | "outputs": [], 46 | "source": [ 47 | "x = torch.ones(2, 2, requires_grad=True)\nprint(x)" 48 | ] 49 | }, 50 | { 51 | "cell_type": "markdown", 52 | "metadata": {}, 53 | "source": [ 54 | "Do an operation of tensor:\n\n" 55 | ] 56 | }, 57 | { 58 | "cell_type": "code", 59 | "execution_count": null, 60 | "metadata": { 61 | "collapsed": false 62 | }, 63 | "outputs": [], 64 | "source": [ 65 | "y = x + 2\nprint(y)" 66 | ] 67 | }, 68 | { 69 | "cell_type": "markdown", 70 | "metadata": {}, 71 | "source": [ 72 | "``y`` was created as a result of an operation, so it has a ``grad_fn``.\n\n" 73 | ] 74 | }, 75 | { 76 | "cell_type": "code", 77 | "execution_count": null, 78 | "metadata": { 79 | "collapsed": false 80 | }, 81 | "outputs": [], 82 | "source": [ 83 | "print(y.grad_fn)" 84 | ] 85 | }, 86 | { 87 | "cell_type": "markdown", 88 | "metadata": {}, 89 | "source": [ 90 | "Do more operations on y\n\n" 91 | ] 92 | }, 93 | { 94 | "cell_type": "code", 95 | "execution_count": null, 96 | "metadata": { 97 | "collapsed": false 98 | }, 99 | "outputs": [], 100 | "source": [ 101 | "z = y * y * 3\nout = z.mean()\n\nprint(z, out)" 102 | ] 103 | }, 104 | { 105 | "cell_type": "markdown", 106 | "metadata": {}, 107 | "source": [ 108 | "``.requires_grad_( ... )`` changes an existing Tensor's ``requires_grad``\nflag in-place. The input flag defaults to ``False`` if not given.\n\n" 109 | ] 110 | }, 111 | { 112 | "cell_type": "code", 113 | "execution_count": null, 114 | "metadata": { 115 | "collapsed": false 116 | }, 117 | "outputs": [], 118 | "source": [ 119 | "a = torch.randn(2, 2)\na = ((a * 3) / (a - 1))\nprint(a.requires_grad)\na.requires_grad_(True)\nprint(a.requires_grad)\nb = (a * a).sum()\nprint(b.grad_fn)" 120 | ] 121 | }, 122 | { 123 | "cell_type": "markdown", 124 | "metadata": {}, 125 | "source": [ 126 | "Gradients\n---------\nLet's backprop now\nBecause ``out`` contains a single scalar, ``out.backward()`` is\nequivalent to ``out.backward(torch.tensor(1))``.\n\n" 127 | ] 128 | }, 129 | { 130 | "cell_type": "code", 131 | "execution_count": null, 132 | "metadata": { 133 | "collapsed": false 134 | }, 135 | "outputs": [], 136 | "source": [ 137 | "out.backward()" 138 | ] 139 | }, 140 | { 141 | "cell_type": "markdown", 142 | "metadata": {}, 143 | "source": [ 144 | "print gradients d(out)/dx\n\n\n" 145 | ] 146 | }, 147 | { 148 | "cell_type": "code", 149 | "execution_count": null, 150 | "metadata": { 151 | "collapsed": false 152 | }, 153 | "outputs": [], 154 | "source": [ 155 | "print(x.grad)" 156 | ] 157 | }, 158 | { 159 | "cell_type": "markdown", 160 | "metadata": {}, 161 | "source": [ 162 | "You should have got a matrix of ``4.5``. Let\u2019s call the ``out``\n*Tensor* \u201c$o$\u201d.\nWe have that $o = \\frac{1}{4}\\sum_i z_i$,\n$z_i = 3(x_i+2)^2$ and $z_i\\bigr\\rvert_{x_i=1} = 27$.\nTherefore,\n$\\frac{\\partial o}{\\partial x_i} = \\frac{3}{2}(x_i+2)$, hence\n$\\frac{\\partial o}{\\partial x_i}\\bigr\\rvert_{x_i=1} = \\frac{9}{2} = 4.5$.\n\n" 163 | ] 164 | }, 165 | { 166 | "cell_type": "markdown", 167 | "metadata": {}, 168 | "source": [ 169 | "You can do many crazy things with autograd!\n\n" 170 | ] 171 | }, 172 | { 173 | "cell_type": "code", 174 | "execution_count": null, 175 | "metadata": { 176 | "collapsed": false 177 | }, 178 | "outputs": [], 179 | "source": [ 180 | "x = torch.randn(3, requires_grad=True)\n\ny = x * 2\nwhile y.data.norm() < 1000:\n y = y * 2\n\nprint(y)" 181 | ] 182 | }, 183 | { 184 | "cell_type": "code", 185 | "execution_count": null, 186 | "metadata": { 187 | "collapsed": false 188 | }, 189 | "outputs": [], 190 | "source": [ 191 | "gradients = torch.tensor([0.1, 1.0, 0.0001], dtype=torch.float)\ny.backward(gradients)\n\nprint(x.grad)" 192 | ] 193 | }, 194 | { 195 | "cell_type": "markdown", 196 | "metadata": {}, 197 | "source": [ 198 | "You can also stop autograd from tracking history on Tensors\nwith ``.requires_grad=True`` by wrapping the code block in\n``with torch.no_grad()``:\n\n" 199 | ] 200 | }, 201 | { 202 | "cell_type": "code", 203 | "execution_count": null, 204 | "metadata": { 205 | "collapsed": false 206 | }, 207 | "outputs": [], 208 | "source": [ 209 | "print(x.requires_grad)\nprint((x ** 2).requires_grad)\n\nwith torch.no_grad():\n\tprint((x ** 2).requires_grad)" 210 | ] 211 | }, 212 | { 213 | "cell_type": "markdown", 214 | "metadata": {}, 215 | "source": [ 216 | "**Read Later:**\n\nDocumentation of ``autograd`` and ``Function`` is at\nhttps://pytorch.org/docs/autograd\n\n" 217 | ] 218 | } 219 | ], 220 | "metadata": { 221 | "kernelspec": { 222 | "display_name": "Python 3", 223 | "language": "python", 224 | "name": "python3" 225 | }, 226 | "language_info": { 227 | "codemirror_mode": { 228 | "name": "ipython", 229 | "version": 3 230 | }, 231 | "file_extension": ".py", 232 | "mimetype": "text/x-python", 233 | "name": "python", 234 | "nbconvert_exporter": "python", 235 | "pygments_lexer": "ipython3", 236 | "version": "3.6.7" 237 | } 238 | }, 239 | "nbformat": 4, 240 | "nbformat_minor": 0 241 | } -------------------------------------------------------------------------------- /docs/chapter1/pic1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/docs/chapter1/pic1.png -------------------------------------------------------------------------------- /docs/chapter1/readme.md: -------------------------------------------------------------------------------- 1 | # PyTorch 中文手册第一章 ： PyTorch入门 2 | 3 | ## 目录 4 | 5 | 1. [PyTorch 简介](1.1-pytorch-introduction.md) 6 | 2. [PyTorch 环境搭建](1.2-pytorch-installation.md) 7 | 3. [PyTorch 深度学习：60分钟快速入门（官方）](1.3-deep-learning-with-pytorch-60-minute-blitz.md) 8 | - [张量](1_tensor_tutorial.ipynb) 9 | - [Autograd：自动求导](2_autograd_tutorial.ipynb) 10 | - [神经网络](3_neural_networks_tutorial.ipynb) 11 | - [训练一个分类器](4_cifar10_tutorial.ipynb) 12 | - [选读：数据并行处理（多GPU）](5_data_parallel_tutorial.ipynb) 13 | 4. [相关资源介绍](1.4-pytorch-resource.md) 14 | -------------------------------------------------------------------------------- /docs/chapter2/1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/docs/chapter2/1.png -------------------------------------------------------------------------------- /docs/chapter2/10.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/docs/chapter2/10.png -------------------------------------------------------------------------------- /docs/chapter2/11.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/docs/chapter2/11.png -------------------------------------------------------------------------------- /docs/chapter2/12.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/docs/chapter2/12.png -------------------------------------------------------------------------------- /docs/chapter2/2.1.2-pytorch-basics-autograd.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# PyTorch 基础 : 自动求导\n", 8 | "深度学习的算法本质上是通过反向传播求导数，而PyTorch的autograd模块则实现了此功能。在Tensor上的所有操作，autograd都能为它们自动提供微分，避免了手动计算导数的复杂过程。\n", 9 | "\n", 10 | "***从0.4起, Variable 正式合并入Tensor, Variable 本来实现的自动微分功能，Tensor就能支持。读者还是可以使用Variable(tensor), 但是这个操作其实什么都没做。***\n", 11 | "\n", 12 | "所以，以后的代码建议直接使用Tensor，因为官方文档中已经将Variable设置成过期模块\n", 13 | "\n", 14 | "要想使得Tensor使用autograd功能，只需要设置tensor.requries_grad=True" 15 | ] 16 | }, 17 | { 18 | "cell_type": "code", 19 | "execution_count": 1, 20 | "metadata": {}, 21 | "outputs": [ 22 | { 23 | "data": { 24 | "text/plain": [ 25 | "'1.0.0'" 26 | ] 27 | }, 28 | "execution_count": 1, 29 | "metadata": {}, 30 | "output_type": "execute_result" 31 | } 32 | ], 33 | "source": [ 34 | "# 首先要引入相关的包\n", 35 | "import torch\n", 36 | "#打印一下版本\n", 37 | "torch.__version__" 38 | ] 39 | }, 40 | { 41 | "cell_type": "markdown", 42 | "metadata": {}, 43 | "source": [ 44 | "在张量创建时，通过设置 requires_grad 标识为Ture来告诉Pytorch需要对该张量进行自动求导，PyTorch会记录该张量的每一步操作历史并自动计算" 45 | ] 46 | }, 47 | { 48 | "cell_type": "code", 49 | "execution_count": 2, 50 | "metadata": {}, 51 | "outputs": [ 52 | { 53 | "data": { 54 | "text/plain": [ 55 | "tensor([[0.7966, 0.6401, 0.1673, 0.4988, 0.3223],\n", 56 | " [0.8881, 0.4756, 0.4391, 0.8762, 0.5714],\n", 57 | " [0.9581, 0.3330, 0.0885, 0.5103, 0.7473],\n", 58 | " [0.9493, 0.4851, 0.8115, 0.0083, 0.2560],\n", 59 | " [0.3152, 0.0132, 0.0858, 0.0406, 0.1677]], requires_grad=True)" 60 | ] 61 | }, 62 | "execution_count": 2, 63 | "metadata": {}, 64 | "output_type": "execute_result" 65 | } 66 | ], 67 | "source": [ 68 | "x = torch.rand(5, 5, requires_grad=True)\n", 69 | "x" 70 | ] 71 | }, 72 | { 73 | "cell_type": "code", 74 | "execution_count": 3, 75 | "metadata": {}, 76 | "outputs": [ 77 | { 78 | "data": { 79 | "text/plain": [ 80 | "tensor([[0.7349, 0.7580, 0.2178, 0.1207, 0.9278],\n", 81 | " [0.8113, 0.3937, 0.9049, 0.0681, 0.0810],\n", 82 | " [0.5836, 0.2981, 0.7601, 0.6309, 0.3913],\n", 83 | " [0.4454, 0.2274, 0.4062, 0.5707, 0.7378],\n", 84 | " [0.4096, 0.7709, 0.7267, 0.4186, 0.1841]], requires_grad=True)" 85 | ] 86 | }, 87 | "execution_count": 3, 88 | "metadata": {}, 89 | "output_type": "execute_result" 90 | } 91 | ], 92 | "source": [ 93 | "y = torch.rand(5, 5, requires_grad=True)\n", 94 | "y" 95 | ] 96 | }, 97 | { 98 | "cell_type": "markdown", 99 | "metadata": {}, 100 | "source": [ 101 | "我们看到 该张量的grad_fn已经被赋予了一个新的函数。下面我们来调用反向传播函数，计算其梯度" 102 | ] 103 | }, 104 | { 105 | "cell_type": "code", 106 | "execution_count": 4, 107 | "metadata": {}, 108 | "outputs": [ 109 | { 110 | "data": { 111 | "text/plain": [ 112 | "tensor(24.0248, grad_fn=)" 113 | ] 114 | }, 115 | "execution_count": 4, 116 | "metadata": {}, 117 | "output_type": "execute_result" 118 | } 119 | ], 120 | "source": [ 121 | "z=torch.sum(x+y)\n", 122 | "z" 123 | ] 124 | }, 125 | { 126 | "cell_type": "markdown", 127 | "metadata": {}, 128 | "source": [ 129 | "## 简单的自动求导" 130 | ] 131 | }, 132 | { 133 | "cell_type": "code", 134 | "execution_count": 5, 135 | "metadata": {}, 136 | "outputs": [], 137 | "source": [ 138 | "z.backward()" 139 | ] 140 | }, 141 | { 142 | "cell_type": "code", 143 | "execution_count": 6, 144 | "metadata": {}, 145 | "outputs": [ 146 | { 147 | "name": "stdout", 148 | "output_type": "stream", 149 | "text": [ 150 | "tensor([[1., 1., 1., 1., 1.],\n", 151 | " [1., 1., 1., 1., 1.],\n", 152 | " [1., 1., 1., 1., 1.],\n", 153 | " [1., 1., 1., 1., 1.],\n", 154 | " [1., 1., 1., 1., 1.]]) tensor([[1., 1., 1., 1., 1.],\n", 155 | " [1., 1., 1., 1., 1.],\n", 156 | " [1., 1., 1., 1., 1.],\n", 157 | " [1., 1., 1., 1., 1.],\n", 158 | " [1., 1., 1., 1., 1.]])\n" 159 | ] 160 | } 161 | ], 162 | "source": [ 163 | "#看一下x和y的梯度\n", 164 | "print(x.grad,y.grad)" 165 | ] 166 | }, 167 | { 168 | "cell_type": "markdown", 169 | "metadata": {}, 170 | "source": [ 171 | "## 复杂的自动求导" 172 | ] 173 | }, 174 | { 175 | "cell_type": "code", 176 | "execution_count": 7, 177 | "metadata": {}, 178 | "outputs": [ 179 | { 180 | "data": { 181 | "text/plain": [ 182 | "tensor([[0.7252, 1.7343, 0.0371, 0.3972, 0.7656],\n", 183 | " [0.1452, 0.6768, 0.7937, 0.4231, 0.5193],\n", 184 | " [1.6275, 0.2478, 0.0672, 0.8888, 0.5644],\n", 185 | " [1.3385, 0.1239, 1.0143, 0.0052, 1.0832],\n", 186 | " [0.9570, 0.2122, 0.5613, 0.1223, 0.3774]], grad_fn=)" 187 | ] 188 | }, 189 | "execution_count": 7, 190 | "metadata": {}, 191 | "output_type": "execute_result" 192 | } 193 | ], 194 | "source": [ 195 | "x = torch.rand(5, 5, requires_grad=True)\n", 196 | "y = torch.rand(5, 5, requires_grad=True)\n", 197 | "z= x**2+y**3\n", 198 | "z" 199 | ] 200 | }, 201 | { 202 | "cell_type": "code", 203 | "execution_count": 8, 204 | "metadata": {}, 205 | "outputs": [ 206 | { 207 | "name": "stdout", 208 | "output_type": "stream", 209 | "text": [ 210 | "tensor([[0.2330, 1.9195, 0.2359, 1.2472, 1.7473],\n", 211 | " [0.7365, 0.5059, 1.7816, 1.0653, 1.4377],\n", 212 | " [1.8439, 0.7431, 0.5183, 1.5588, 1.4149],\n", 213 | " [1.4076, 0.3347, 1.6417, 0.1141, 0.7594],\n", 214 | " [1.8638, 0.5341, 0.3941, 0.6932, 0.6859]])\n" 215 | ] 216 | } 217 | ], 218 | "source": [ 219 | "#我们的返回值不是一个scalar，所以需要输入一个大小相同的张量作为参数，这里我们用ones_like函数根据x生成一个张量\n", 220 | "z.backward(torch.ones_like(x))\n", 221 | "print(x.grad)" 222 | ] 223 | }, 224 | { 225 | "cell_type": "markdown", 226 | "metadata": {}, 227 | "source": [ 228 | "我们可以使用`with torch.no_grad()`上下文管理器临时禁止对已设置requires_grad=True的张量进行自动求导。这个方法在测试集计算准确率的时候会经常用到，例如" 229 | ] 230 | }, 231 | { 232 | "cell_type": "code", 233 | "execution_count": 9, 234 | "metadata": {}, 235 | "outputs": [ 236 | { 237 | "name": "stdout", 238 | "output_type": "stream", 239 | "text": [ 240 | "False\n" 241 | ] 242 | } 243 | ], 244 | "source": [ 245 | "with torch.no_grad():\n", 246 | " print((x +y*2).requires_grad)" 247 | ] 248 | } 249 | ], 250 | "metadata": { 251 | "kernelspec": { 252 | "display_name": "Python 3", 253 | "language": "python", 254 | "name": "python3" 255 | }, 256 | "language_info": { 257 | "codemirror_mode": { 258 | "name": "ipython", 259 | "version": 3 260 | }, 261 | "file_extension": ".py", 262 | "mimetype": "text/x-python", 263 | "name": "python", 264 | "nbconvert_exporter": "python", 265 | "pygments_lexer": "ipython3", 266 | "version": "3.6.7" 267 | } 268 | }, 269 | "nbformat": 4, 270 | "nbformat_minor": 2 271 | } 272 | -------------------------------------------------------------------------------- /docs/chapter2/2.1.4-pytorch-basics-data-lorder.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# PyTorch 基础 :数据的加载和预处理\n", 8 | "PyTorch通过torch.utils.data对一般常用的数据加载进行了封装，可以很容易地实现多线程数据预读和批量加载。\n", 9 | "并且torchvision已经预先实现了常用图像数据集，包括前面使用过的CIFAR-10，ImageNet、COCO、MNIST、LSUN等数据集，可通过torchvision.datasets方便的调用" 10 | ] 11 | }, 12 | { 13 | "cell_type": "code", 14 | "execution_count": 1, 15 | "metadata": {}, 16 | "outputs": [ 17 | { 18 | "data": { 19 | "text/plain": [ 20 | "'1.0.1.post2'" 21 | ] 22 | }, 23 | "execution_count": 1, 24 | "metadata": {}, 25 | "output_type": "execute_result" 26 | } 27 | ], 28 | "source": [ 29 | "# 首先要引入相关的包\n", 30 | "import torch\n", 31 | "#打印一下版本\n", 32 | "torch.__version__" 33 | ] 34 | }, 35 | { 36 | "cell_type": "markdown", 37 | "metadata": {}, 38 | "source": [ 39 | "## Dataset\n", 40 | "Dataset是一个抽象类, 为了能够方便的读取，需要将要使用的数据包装为Dataset类。\n", 41 | "自定义的Dataset需要继承它并且实现两个成员方法：\n", 42 | "1. `__getitem__()` 该方法定义用索引(`0` 到 `len(self)`)获取一条数据或一个样本\n", 43 | "2. `__len__()` 该方法返回数据集的总长度\n", 44 | "\n", 45 | "下面我们使用kaggle上的一个竞赛[bluebook for bulldozers](https://www.kaggle.com/c/bluebook-for-bulldozers/data)自定义一个数据集，为了方便介绍，我们使用里面的数据字典来做说明（因为条数少）" 46 | ] 47 | }, 48 | { 49 | "cell_type": "code", 50 | "execution_count": 2, 51 | "metadata": {}, 52 | "outputs": [], 53 | "source": [ 54 | "#引用\n", 55 | "from torch.utils.data import Dataset\n", 56 | "import pandas as pd" 57 | ] 58 | }, 59 | { 60 | "cell_type": "code", 61 | "execution_count": 3, 62 | "metadata": {}, 63 | "outputs": [], 64 | "source": [ 65 | "#定义一个数据集\n", 66 | "class BulldozerDataset(Dataset):\n", 67 | " \"\"\" 数据集演示 \"\"\"\n", 68 | " def __init__(self, csv_file):\n", 69 | " \"\"\"实现初始化方法，在初始化的时候将数据读载入\"\"\"\n", 70 | " self.df=pd.read_csv(csv_file)\n", 71 | " def __len__(self):\n", 72 | " '''\n", 73 | " 返回df的长度\n", 74 | " '''\n", 75 | " return len(self.df)\n", 76 | " def __getitem__(self, idx):\n", 77 | " '''\n", 78 | " 根据 idx 返回一行数据\n", 79 | " '''\n", 80 | " return self.df.iloc[idx].SalePrice" 81 | ] 82 | }, 83 | { 84 | "cell_type": "markdown", 85 | "metadata": {}, 86 | "source": [ 87 | "至此，我们的数据集已经定义完成了，我们可以实例话一个对象访问他" 88 | ] 89 | }, 90 | { 91 | "cell_type": "code", 92 | "execution_count": 4, 93 | "metadata": {}, 94 | "outputs": [], 95 | "source": [ 96 | "ds_demo= BulldozerDataset('median_benchmark.csv')" 97 | ] 98 | }, 99 | { 100 | "cell_type": "markdown", 101 | "metadata": {}, 102 | "source": [ 103 | "我们可以直接使用如下命令查看数据集数据\n" 104 | ] 105 | }, 106 | { 107 | "cell_type": "code", 108 | "execution_count": 5, 109 | "metadata": {}, 110 | "outputs": [ 111 | { 112 | "data": { 113 | "text/plain": [ 114 | "11573" 115 | ] 116 | }, 117 | "execution_count": 5, 118 | "metadata": {}, 119 | "output_type": "execute_result" 120 | } 121 | ], 122 | "source": [ 123 | "#实现了 __len__ 方法所以可以直接使用len获取数据总数\n", 124 | "len(ds_demo)" 125 | ] 126 | }, 127 | { 128 | "cell_type": "code", 129 | "execution_count": 6, 130 | "metadata": {}, 131 | "outputs": [ 132 | { 133 | "data": { 134 | "text/plain": [ 135 | "24000.0" 136 | ] 137 | }, 138 | "execution_count": 6, 139 | "metadata": {}, 140 | "output_type": "execute_result" 141 | } 142 | ], 143 | "source": [ 144 | "#用索引可以直接访问对应的数据, 对应 __getitem__ 方法\n", 145 | "ds_demo[0]" 146 | ] 147 | }, 148 | { 149 | "cell_type": "markdown", 150 | "metadata": {}, 151 | "source": [ 152 | "自定义的数据集已经创建好了，下面我们使用官方提供的数据载入器，读取数据\n", 153 | "## Datalorder\n", 154 | "DataLoader为我们提供了对Dataset的读取操作，常用参数有：batch_size(每个batch的大小), shuffle(是否进行shuffle操作), num_workers(加载数据的时候使用几个子进程)，下面做一个简单的操作" 155 | ] 156 | }, 157 | { 158 | "cell_type": "code", 159 | "execution_count": 7, 160 | "metadata": {}, 161 | "outputs": [], 162 | "source": [ 163 | "dl = torch.utils.data.DataLoader(ds_demo, batch_size=10, shuffle=True, num_workers=0)" 164 | ] 165 | }, 166 | { 167 | "cell_type": "markdown", 168 | "metadata": {}, 169 | "source": [ 170 | "DataLoader返回的是一个可迭代对象，我们可以使用迭代器分次获取数据" 171 | ] 172 | }, 173 | { 174 | "cell_type": "code", 175 | "execution_count": 8, 176 | "metadata": {}, 177 | "outputs": [ 178 | { 179 | "name": "stdout", 180 | "output_type": "stream", 181 | "text": [ 182 | "tensor([24000., 24000., 24000., 24000., 24000., 24000., 24000., 24000., 24000.,\n", 183 | " 24000.], dtype=torch.float64)\n" 184 | ] 185 | } 186 | ], 187 | "source": [ 188 | "idata=iter(dl)\n", 189 | "print(next(idata))" 190 | ] 191 | }, 192 | { 193 | "cell_type": "markdown", 194 | "metadata": {}, 195 | "source": [ 196 | "常见的用法是使用for循环对其进行遍历" 197 | ] 198 | }, 199 | { 200 | "cell_type": "code", 201 | "execution_count": 9, 202 | "metadata": {}, 203 | "outputs": [ 204 | { 205 | "name": "stdout", 206 | "output_type": "stream", 207 | "text": [ 208 | "0 tensor([24000., 24000., 24000., 24000., 24000., 24000., 24000., 24000., 24000.,\n", 209 | " 24000.], dtype=torch.float64)\n" 210 | ] 211 | } 212 | ], 213 | "source": [ 214 | "for i, data in enumerate(dl):\n", 215 | " print(i,data)\n", 216 | " # 为了节约空间, 这里只循环一遍\n", 217 | " break" 218 | ] 219 | }, 220 | { 221 | "cell_type": "markdown", 222 | "metadata": {}, 223 | "source": [ 224 | "我们已经可以通过dataset定义数据集，并使用Datalorder载入和遍历数据集，除了这些以外，PyTorch还提供能torcvision的计算机视觉扩展包，里面封装了\n", 225 | "## torchvision 包\n", 226 | "torchvision 是PyTorch中专门用来处理图像的库，PyTorch官网的安装教程中最后的pip install torchvision 就是安装这个包。" 227 | ] 228 | }, 229 | { 230 | "cell_type": "markdown", 231 | "metadata": {}, 232 | "source": [ 233 | "### torchvision.datasets\n", 234 | "torchvision.datasets 可以理解为PyTorch团队自定义的dataset，这些dataset帮我们提前处理好了很多的图片数据集，我们拿来就可以直接使用：\n", 235 | "- MNIST\n", 236 | "- COCO\n", 237 | "- Captions\n", 238 | "- Detection\n", 239 | "- LSUN\n", 240 | "- ImageFolder\n", 241 | "- Imagenet-12\n", 242 | "- CIFAR\n", 243 | "- STL10\n", 244 | "- SVHN\n", 245 | "- PhotoTour\n", 246 | "我们可以直接使用，示例如下：" 247 | ] 248 | }, 249 | { 250 | "cell_type": "code", 251 | "execution_count": 10, 252 | "metadata": {}, 253 | "outputs": [], 254 | "source": [ 255 | "import torchvision.datasets as datasets\n", 256 | "trainset = datasets.MNIST(root='./data', # 表示 MNIST 数据的加载的目录\n", 257 | " train=True, # 表示是否加载数据库的训练集，false的时候加载测试集\n", 258 | " download=True, # 表示是否自动下载 MNIST 数据集\n", 259 | " transform=None) # 表示是否需要对数据进行预处理，none为不进行预处理\n" 260 | ] 261 | }, 262 | { 263 | "cell_type": "markdown", 264 | "metadata": {}, 265 | "source": [ 266 | "### torchvision.models\n", 267 | "torchvision不仅提供了常用图片数据集，还提供了训练好的模型，可以加载之后，直接使用，或者在进行迁移学习\n", 268 | "torchvision.models模块的 子模块中包含以下模型结构。\n", 269 | "- AlexNet\n", 270 | "- VGG\n", 271 | "- ResNet\n", 272 | "- SqueezeNet\n", 273 | "- DenseNet" 274 | ] 275 | }, 276 | { 277 | "cell_type": "code", 278 | "execution_count": 11, 279 | "metadata": {}, 280 | "outputs": [], 281 | "source": [ 282 | "#我们直接可以使用训练好的模型，当然这个与datasets相同，都是需要从服务器下载的\n", 283 | "import torchvision.models as models\n", 284 | "resnet18 = models.resnet18(pretrained=True)" 285 | ] 286 | }, 287 | { 288 | "cell_type": "markdown", 289 | "metadata": {}, 290 | "source": [ 291 | "### torchvision.transforms\n", 292 | "transforms 模块提供了一般的图像转换操作类，用作数据处理和数据增强" 293 | ] 294 | }, 295 | { 296 | "cell_type": "code", 297 | "execution_count": 12, 298 | "metadata": {}, 299 | "outputs": [], 300 | "source": [ 301 | "from torchvision import transforms as transforms\n", 302 | "transform = transforms.Compose([\n", 303 | " transforms.RandomCrop(32, padding=4), #先四周填充0，在把图像随机裁剪成32*32\n", 304 | " transforms.RandomHorizontalFlip(), #图像一半的概率翻转，一半的概率不翻转\n", 305 | " transforms.RandomRotation((-45,45)), #随机旋转\n", 306 | " transforms.ToTensor(),\n", 307 | " transforms.Normalize((0.4914, 0.4822, 0.4465), (0.229, 0.224, 0.225)), #R,G,B每层的归一化用到的均值和方差\n", 308 | "])" 309 | ] 310 | }, 311 | { 312 | "cell_type": "markdown", 313 | "metadata": {}, 314 | "source": [ 315 | "肯定有人会问：(0.485, 0.456, 0.406), (0.2023, 0.1994, 0.2010) 这几个数字是什么意思？\n", 316 | "\n", 317 | "官方的这个帖子有详细的说明:\n", 318 | "https://discuss.pytorch.org/t/normalization-in-the-mnist-example/457/21\n", 319 | "这些都是根据ImageNet训练的归一化参数，可以直接使用，我们认为这个是固定值就可以" 320 | ] 321 | }, 322 | { 323 | "cell_type": "markdown", 324 | "metadata": {}, 325 | "source": [ 326 | "我们已经完成了Python的基本内容的介绍，下面我们要介绍神经网络的理论基础，里面的公式等内容我们都使用PyTorch来实现" 327 | ] 328 | }, 329 | { 330 | "cell_type": "code", 331 | "execution_count": null, 332 | "metadata": {}, 333 | "outputs": [], 334 | "source": [] 335 | } 336 | ], 337 | "metadata": { 338 | "kernelspec": { 339 | "display_name": "Python 3", 340 | "language": "python", 341 | "name": "python3" 342 | }, 343 | "language_info": { 344 | "codemirror_mode": { 345 | "name": "ipython", 346 | "version": 3 347 | }, 348 | "file_extension": ".py", 349 | "mimetype": "text/x-python", 350 | "name": "python", 351 | "nbconvert_exporter": "python", 352 | "pygments_lexer": "ipython3", 353 | "version": "3.6.8" 354 | } 355 | }, 356 | "nbformat": 4, 357 | "nbformat_minor": 2 358 | } 359 | -------------------------------------------------------------------------------- /docs/chapter2/2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/docs/chapter2/2.png -------------------------------------------------------------------------------- /docs/chapter2/3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/docs/chapter2/3.png -------------------------------------------------------------------------------- /docs/chapter2/4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/docs/chapter2/4.png -------------------------------------------------------------------------------- /docs/chapter2/5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/docs/chapter2/5.png -------------------------------------------------------------------------------- /docs/chapter2/6.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/docs/chapter2/6.png -------------------------------------------------------------------------------- /docs/chapter2/7.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/docs/chapter2/7.png -------------------------------------------------------------------------------- /docs/chapter2/8.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/docs/chapter2/8.png -------------------------------------------------------------------------------- /docs/chapter2/9.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/docs/chapter2/9.gif -------------------------------------------------------------------------------- /docs/chapter2/alexnet.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/docs/chapter2/alexnet.png -------------------------------------------------------------------------------- /docs/chapter2/cnn.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/docs/chapter2/cnn.png -------------------------------------------------------------------------------- /docs/chapter2/googlenet.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/docs/chapter2/googlenet.png -------------------------------------------------------------------------------- /docs/chapter2/gru.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/docs/chapter2/gru.gif -------------------------------------------------------------------------------- /docs/chapter2/lenet5.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/docs/chapter2/lenet5.jpg -------------------------------------------------------------------------------- /docs/chapter2/lstm.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/docs/chapter2/lstm.gif -------------------------------------------------------------------------------- /docs/chapter2/readme.md: -------------------------------------------------------------------------------- 1 | # Pytorch 中文手册第二章 ： 基础 2 | 3 | ## 目录 4 | 5 | ### 第一节 PyTorch 基础 6 | 1. [张量](2.1.1.pytorch-basics-tensor.ipynb) 7 | 2. [自动求导](2.1.2-pytorch-basics-autograd.ipynb) 8 | 3. [神经网络包nn和优化器optm](2.1.3-pytorch-basics-nerual-network.ipynb) 9 | 4. [数据的加载和预处理](2.1.4-pytorch-basics-data-lorder.ipynb) 10 | 11 | ### 第二节 深度学习基础及数学原理 12 | [深度学习基础及数学原理](2.2-deep-learning-basic-mathematics.ipynb) 13 | 14 | ### 第三节 神经网络简介 15 | [神经网络简介](2.3-deep-learning-neural-network-introduction.ipynb) 16 | 17 | ### 第四节 卷积神经网络 18 | [卷积神经网络](2.4-cnn.ipynb) 19 | 20 | ### 第五节 循环神经网络 21 | [循环神经网络](2.5-rnn.ipynb) -------------------------------------------------------------------------------- /docs/chapter2/resnet.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/docs/chapter2/resnet.png -------------------------------------------------------------------------------- /docs/chapter2/resnet18.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/docs/chapter2/resnet18.jpg -------------------------------------------------------------------------------- /docs/chapter2/vgg16.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/docs/chapter2/vgg16.png -------------------------------------------------------------------------------- /docs/chapter3/3.1-logistic-regression.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "metadata": {}, 7 | "outputs": [ 8 | { 9 | "data": { 10 | "text/plain": [ 11 | "'1.0.0'" 12 | ] 13 | }, 14 | "execution_count": 1, 15 | "metadata": {}, 16 | "output_type": "execute_result" 17 | } 18 | ], 19 | "source": [ 20 | "import torch\n", 21 | "import torch.nn as nn\n", 22 | "import numpy as np\n", 23 | "torch.__version__" 24 | ] 25 | }, 26 | { 27 | "cell_type": "markdown", 28 | "metadata": {}, 29 | "source": [ 30 | "# 3.1 logistic回归实战\n", 31 | "在这一章里面，我们将处理一下结构化数据，并使用logistic回归对结构化数据进行简单的分类。\n", 32 | "## 3.1.1 logistic回归介绍\n", 33 | "logistic回归是一种广义线性回归（generalized linear model），与多重线性回归分析有很多相同之处。它们的模型形式基本上相同，都具有 wx + b，其中w和b是待求参数，其区别在于他们的因变量不同，多重线性回归直接将wx+b作为因变量，即y =wx+b,而logistic回归则通过函数L将wx+b对应一个隐状态p，p =L(wx+b),然后根据p 与1-p的大小决定因变量的值。如果L是logistic函数，就是logistic回归，如果L是多项式函数就是多项式回归。\n", 34 | "\n", 35 | "说的更通俗一点，就是logistic回归会在线性回归后再加一层logistic函数的调用。\n", 36 | "\n", 37 | "logistic回归主要是进行二分类预测，我们在激活函数时候讲到过 Sigmod函数，Sigmod函数是最常见的logistic函数，因为Sigmod函数的输出的是是对于0~1之间的概率值，当概率大于0.5预测为1，小于0.5预测为0。\n", 38 | "\n", 39 | "下面我们就来使用公开的数据来进行介绍" 40 | ] 41 | }, 42 | { 43 | "cell_type": "markdown", 44 | "metadata": {}, 45 | "source": [ 46 | "## 3.1.2 UCI German Credit 数据集\n", 47 | "\n", 48 | "UCI German Credit是UCI的德国信用数据集，里面有原数据和数值化后的数据。\n", 49 | "\n", 50 | "German Credit数据是根据个人的银行贷款信息和申请客户贷款逾期发生情况来预测贷款违约倾向的数据集，数据集包含24个维度的，1000条数据，\n", 51 | "\n", 52 | "在这里我们直接使用处理好的数值化的数据，作为展示。\n", 53 | "\n", 54 | "[地址](https://archive.ics.uci.edu/ml/machine-learning-databases/statlog/german/)" 55 | ] 56 | }, 57 | { 58 | "cell_type": "markdown", 59 | "metadata": {}, 60 | "source": [ 61 | "## 3.2 代码实战\n", 62 | "我们这里使用的 german.data-numeric是numpy处理好数值化数据，我们直接使用numpy的load方法读取即可" 63 | ] 64 | }, 65 | { 66 | "cell_type": "code", 67 | "execution_count": 2, 68 | "metadata": {}, 69 | "outputs": [], 70 | "source": [ 71 | "data=np.loadtxt(\"german.data-numeric\")" 72 | ] 73 | }, 74 | { 75 | "cell_type": "markdown", 76 | "metadata": {}, 77 | "source": [ 78 | "数据读取完成后我们要对数据做一下归一化的处理" 79 | ] 80 | }, 81 | { 82 | "cell_type": "code", 83 | "execution_count": 3, 84 | "metadata": {}, 85 | "outputs": [], 86 | "source": [ 87 | "n,l=data.shape\n", 88 | "for j in range(l-1):\n", 89 | " meanVal=np.mean(data[:,j])\n", 90 | " stdVal=np.std(data[:,j])\n", 91 | " data[:,j]=(data[:,j]-meanVal)/stdVal" 92 | ] 93 | }, 94 | { 95 | "cell_type": "markdown", 96 | "metadata": {}, 97 | "source": [ 98 | "打乱数据" 99 | ] 100 | }, 101 | { 102 | "cell_type": "code", 103 | "execution_count": 4, 104 | "metadata": {}, 105 | "outputs": [], 106 | "source": [ 107 | "np.random.shuffle(data)" 108 | ] 109 | }, 110 | { 111 | "cell_type": "markdown", 112 | "metadata": {}, 113 | "source": [ 114 | "区分训练集和测试集，由于这里没有验证集，所以我们直接使用测试集的准确度作为评判好坏的标准\n", 115 | "\n", 116 | "区分规则：900条用于训练，100条作为测试\n", 117 | "\n", 118 | "german.data-numeric的格式为，前24列为24个维度，最后一个为要打的标签（0，1），所以我们将数据和标签一起区分出来" 119 | ] 120 | }, 121 | { 122 | "cell_type": "code", 123 | "execution_count": 5, 124 | "metadata": {}, 125 | "outputs": [], 126 | "source": [ 127 | "train_data=data[:900,:l-1]\n", 128 | "train_lab=data[:900,l-1]-1\n", 129 | "test_data=data[900:,:l-1]\n", 130 | "test_lab=data[900:,l-1]-1" 131 | ] 132 | }, 133 | { 134 | "cell_type": "markdown", 135 | "metadata": {}, 136 | "source": [ 137 | "下面我们定义模型，模型很简单" 138 | ] 139 | }, 140 | { 141 | "cell_type": "code", 142 | "execution_count": 6, 143 | "metadata": {}, 144 | "outputs": [], 145 | "source": [ 146 | "class LR(nn.Module):\n", 147 | " def __init__(self):\n", 148 | " super(LR,self).__init__()\n", 149 | " self.fc=nn.Linear(24,2) # 由于24个维度已经固定了，所以这里写24\n", 150 | " def forward(self,x):\n", 151 | " out=self.fc(x)\n", 152 | " out=torch.sigmoid(out)\n", 153 | " return out\n", 154 | " " 155 | ] 156 | }, 157 | { 158 | "cell_type": "markdown", 159 | "metadata": {}, 160 | "source": [ 161 | "测试集上的准确率" 162 | ] 163 | }, 164 | { 165 | "cell_type": "code", 166 | "execution_count": 7, 167 | "metadata": {}, 168 | "outputs": [], 169 | "source": [ 170 | "def test(pred,lab):\n", 171 | " t=pred.max(-1)[1]==lab\n", 172 | " return torch.mean(t.float())" 173 | ] 174 | }, 175 | { 176 | "cell_type": "markdown", 177 | "metadata": {}, 178 | "source": [ 179 | "下面就是对一些设置" 180 | ] 181 | }, 182 | { 183 | "cell_type": "code", 184 | "execution_count": 8, 185 | "metadata": {}, 186 | "outputs": [], 187 | "source": [ 188 | "net=LR() \n", 189 | "criterion=nn.CrossEntropyLoss() # 使用CrossEntropyLoss损失\n", 190 | "optm=torch.optim.Adam(net.parameters()) # Adam优化\n", 191 | "epochs=1000 # 训练1000次\n" 192 | ] 193 | }, 194 | { 195 | "cell_type": "markdown", 196 | "metadata": {}, 197 | "source": [ 198 | "下面开始训练了" 199 | ] 200 | }, 201 | { 202 | "cell_type": "code", 203 | "execution_count": 9, 204 | "metadata": {}, 205 | "outputs": [ 206 | { 207 | "name": "stdout", 208 | "output_type": "stream", 209 | "text": [ 210 | "Epoch:100,Loss:0.6313,Accuracy：0.76\n", 211 | "Epoch:200,Loss:0.6065,Accuracy：0.79\n", 212 | "Epoch:300,Loss:0.5909,Accuracy：0.80\n", 213 | "Epoch:400,Loss:0.5801,Accuracy：0.81\n", 214 | "Epoch:500,Loss:0.5720,Accuracy：0.82\n", 215 | "Epoch:600,Loss:0.5657,Accuracy：0.81\n", 216 | "Epoch:700,Loss:0.5606,Accuracy：0.81\n", 217 | "Epoch:800,Loss:0.5563,Accuracy：0.81\n", 218 | "Epoch:900,Loss:0.5527,Accuracy：0.81\n", 219 | "Epoch:1000,Loss:0.5496,Accuracy：0.80\n" 220 | ] 221 | } 222 | ], 223 | "source": [ 224 | "for i in range(epochs):\n", 225 | " # 指定模型为训练模式，计算梯度\n", 226 | " net.train()\n", 227 | " # 输入值都需要转化成torch的Tensor\n", 228 | " x=torch.from_numpy(train_data).float()\n", 229 | " y=torch.from_numpy(train_lab).long()\n", 230 | " y_hat=net(x)\n", 231 | " loss=criterion(y_hat,y) # 计算损失\n", 232 | " optm.zero_grad() # 前一步的损失清零\n", 233 | " loss.backward() # 反向传播\n", 234 | " optm.step() # 优化\n", 235 | " if (i+1)%100 ==0 : # 这里我们每100次输出相关的信息\n", 236 | " # 指定模型为计算模式\n", 237 | " net.eval()\n", 238 | " test_in=torch.from_numpy(test_data).float()\n", 239 | " test_l=torch.from_numpy(test_lab).long()\n", 240 | " test_out=net(test_in)\n", 241 | " # 使用我们的测试函数计算准确率\n", 242 | " accu=test(test_out,test_l)\n", 243 | " print(\"Epoch:{},Loss:{:.4f},Accuracy：{:.2f}\".format(i+1,loss.item(),accu))" 244 | ] 245 | }, 246 | { 247 | "cell_type": "markdown", 248 | "metadata": {}, 249 | "source": [ 250 | "训练完成了，我们的准确度达到了80%" 251 | ] 252 | }, 253 | { 254 | "cell_type": "code", 255 | "execution_count": null, 256 | "metadata": {}, 257 | "outputs": [], 258 | "source": [] 259 | } 260 | ], 261 | "metadata": { 262 | "kernelspec": { 263 | "display_name": "Python 3", 264 | "language": "python", 265 | "name": "python3" 266 | }, 267 | "language_info": { 268 | "codemirror_mode": { 269 | "name": "ipython", 270 | "version": 3 271 | }, 272 | "file_extension": ".py", 273 | "mimetype": "text/x-python", 274 | "name": "python", 275 | "nbconvert_exporter": "python", 276 | "pygments_lexer": "ipython3", 277 | "version": "3.6.7" 278 | } 279 | }, 280 | "nbformat": 4, 281 | "nbformat_minor": 2 282 | } 283 | -------------------------------------------------------------------------------- /docs/chapter3/readme.md: -------------------------------------------------------------------------------- 1 | # Pytorch 中文手册第三章 ： 实践 2 | 3 | ## 目录 4 | 5 | ### 3.1 logistic回归 6 | [logistic回归二元分类](3.1-logistic-regression.ipynb) 7 | 8 | ### 3.2 CNN:MNIST数据集手写数字识别 9 | [CNN:MNIST数据集手写数字识别](3.2-mnist.ipynb) 10 | 11 | ### 3.3 RNN实例：通过Sin预测Cos 12 | [RNN实例：通过Sin预测Cos](3.3-rnn.ipynb) 13 | -------------------------------------------------------------------------------- /docs/chapter4/1.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/docs/chapter4/1.PNG -------------------------------------------------------------------------------- /docs/chapter4/1280px-Felis_silvestris_catus_lying_on_rice_straw.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/docs/chapter4/1280px-Felis_silvestris_catus_lying_on_rice_straw.jpg -------------------------------------------------------------------------------- /docs/chapter4/2.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/docs/chapter4/2.PNG -------------------------------------------------------------------------------- /docs/chapter4/3.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/docs/chapter4/3.PNG -------------------------------------------------------------------------------- /docs/chapter4/4.2.1-visdom.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "metadata": {}, 7 | "outputs": [ 8 | { 9 | "data": { 10 | "text/plain": [ 11 | "'1.0.0'" 12 | ] 13 | }, 14 | "execution_count": 1, 15 | "metadata": {}, 16 | "output_type": "execute_result" 17 | } 18 | ], 19 | "source": [ 20 | "import torch\n", 21 | "import math\n", 22 | "import numpy as np\n", 23 | "from visdom import Visdom\n", 24 | "import time\n", 25 | "torch.__version__" 26 | ] 27 | }, 28 | { 29 | "cell_type": "markdown", 30 | "metadata": {}, 31 | "source": [ 32 | "# 4.2.1 使用Visdom在 PyTorch 中进行可视化\n", 33 | "Visdom是Facebook在2017年发布的一款针对PyTorch的可视化工具。[官网](https://github.com/facebookresearch/visdom),visdom由于其功能简单，一般会被定义为服务器端的matplot，也就是说我们可以直接使用python的控制台模式进行开发并在服务器上执行，将一些可视化的数据传送到Visdom服务上，通过Visdom服务进行可视化" 34 | ] 35 | }, 36 | { 37 | "cell_type": "markdown", 38 | "metadata": {}, 39 | "source": [ 40 | "## 安装\n", 41 | "Visdom的安装很简单，直接使用命令`pip install visdom`安装即可。\n", 42 | "在安装完成后，使用命令`python -m visdom.server` 在本地启动服务器，启动后会提示`It's Alive! You can navigate to http://localhost:8097` 这就说明服务已经可用，我们打开浏览器，输入`http://localhost:8097` 即可看到页面。\n", 43 | "\n", 44 | "端口8097是默认的端口可以在启动命令后加 `-port`参数指定端口，常用的参数还有 `--hostname`，`-base_url`等" 45 | ] 46 | }, 47 | { 48 | "cell_type": "markdown", 49 | "metadata": {}, 50 | "source": [ 51 | "## 坑\n", 52 | "Visdom的服务在启动时会自动下载一些静态文件，这里坑就来了，因为某些无法描述的原因，导致下载会失败，比如类似这样的提示 `ERROR:root:Error 404 while downloading https://unpkg.com/layout-bin-packer@1.4.0` 就说明静态文件没有下载完全，这样有可能就会打不开或者页面中没有菜单栏，那么需要手动进行下载，这里我打包了一份正常的静态文件，直接复制到`Lib\\site-packages\\visdom`中即可。\n", 53 | "\n", 54 | "如果不知道conda的环境目录在哪里，可以使用`conda env list` 查看\n", 55 | "\n", 56 | "感谢CSDN的伙伴提供的缺失文件，原文[这里](https://blog.csdn.net/qq_36941368/article/details/82288154)" 57 | ] 58 | }, 59 | { 60 | "cell_type": "markdown", 61 | "metadata": {}, 62 | "source": [ 63 | "## 基本概念\n", 64 | "### Environments\n", 65 | "Environments的作用是对可视化区域进行分区，每个用户都会有一个叫做main的默认分区，如图所示:\n", 66 | "![](1.PNG)\n", 67 | "在程序指定的情况下，默认的图表都会放到这里面" 68 | ] 69 | }, 70 | { 71 | "cell_type": "markdown", 72 | "metadata": {}, 73 | "source": [ 74 | "## Panes\n", 75 | "Panes是作为每一个可视化图表的容器，可以使用生成的图表，图片，文本进行填充，我们可以对Panes进行拖放，删除，调整大小和销毁等操作：\n", 76 | "![](2.PNG)\n", 77 | "Panes和Environments是一对多的关系，即一个Environments可以包含多个Panes\n" 78 | ] 79 | }, 80 | { 81 | "cell_type": "markdown", 82 | "metadata": {}, 83 | "source": [ 84 | "### VIEW\n", 85 | "在对Panes进行调整后，可以通过VIEW对状态进行管理：\n", 86 | "![](3.PNG)" 87 | ] 88 | }, 89 | { 90 | "cell_type": "markdown", 91 | "metadata": {}, 92 | "source": [ 93 | "## 可视化接口\n", 94 | "Visdom是由Plotly 提供的可视化支持，所以提供一下可视化的接口:\n", 95 | "- vis.scatter : 2D 或 3D 散点图\n", 96 | "- vis.line : 线图\n", 97 | "- vis.stem : 茎叶图\n", 98 | "- vis.heatmap : 热力图\n", 99 | "- vis.bar : 条形图\n", 100 | "- vis.histogram: 直方图\n", 101 | "- vis.boxplot : 箱型图\n", 102 | "- vis.surf : 表面图\n", 103 | "- vis.contour : 轮廓图\n", 104 | "- vis.quiver : 绘出二维矢量场\n", 105 | "- vis.image : 图片\n", 106 | "- vis.text : 文本\n", 107 | "- vis.mesh : 网格图\n", 108 | "- vis.save : 序列化状态" 109 | ] 110 | }, 111 | { 112 | "cell_type": "markdown", 113 | "metadata": {}, 114 | "source": [ 115 | "## 使用\n", 116 | "### 绘制简单的图形\n", 117 | "这里我们使用官方的DEMO来做样例" 118 | ] 119 | }, 120 | { 121 | "cell_type": "code", 122 | "execution_count": 2, 123 | "metadata": {}, 124 | "outputs": [], 125 | "source": [ 126 | "env = Visdom() \n", 127 | "assert env.check_connection() #测试一下链接，链接错误的话会报错" 128 | ] 129 | }, 130 | { 131 | "cell_type": "markdown", 132 | "metadata": {}, 133 | "source": [ 134 | "这里生成sin和cos两条曲线数据" 135 | ] 136 | }, 137 | { 138 | "cell_type": "code", 139 | "execution_count": 3, 140 | "metadata": {}, 141 | "outputs": [], 142 | "source": [ 143 | "Y = np.linspace(0, 2 * math.pi, 70)\n", 144 | "X = np.column_stack((np.sin(Y), np.cos(Y)))" 145 | ] 146 | }, 147 | { 148 | "cell_type": "markdown", 149 | "metadata": {}, 150 | "source": [ 151 | "使用茎叶图展示" 152 | ] 153 | }, 154 | { 155 | "cell_type": "code", 156 | "execution_count": 4, 157 | "metadata": { 158 | "scrolled": true 159 | }, 160 | "outputs": [ 161 | { 162 | "data": { 163 | "text/plain": [ 164 | "'window_36f18bc34b4992'" 165 | ] 166 | }, 167 | "execution_count": 4, 168 | "metadata": {}, 169 | "output_type": "execute_result" 170 | } 171 | ], 172 | "source": [ 173 | "env.stem(\n", 174 | " X=X,\n", 175 | " Y=Y,\n", 176 | " opts=dict(legend=['Sine', 'Cosine'])\n", 177 | " )" 178 | ] 179 | }, 180 | { 181 | "cell_type": "markdown", 182 | "metadata": {}, 183 | "source": [ 184 | "可以通过env参数指定Environments，如果名称包含了下划线`_`那么visdom会跟根据下划线分割并自动分组" 185 | ] 186 | }, 187 | { 188 | "cell_type": "code", 189 | "execution_count": 5, 190 | "metadata": { 191 | "scrolled": true 192 | }, 193 | "outputs": [], 194 | "source": [ 195 | "envtest = Visdom(env='test_mesh')\n", 196 | "assert envtest.check_connection()" 197 | ] 198 | }, 199 | { 200 | "cell_type": "markdown", 201 | "metadata": {}, 202 | "source": [ 203 | "生成一个网格图" 204 | ] 205 | }, 206 | { 207 | "cell_type": "code", 208 | "execution_count": 6, 209 | "metadata": {}, 210 | "outputs": [ 211 | { 212 | "data": { 213 | "text/plain": [ 214 | "'window_36f18bc533e990'" 215 | ] 216 | }, 217 | "execution_count": 6, 218 | "metadata": {}, 219 | "output_type": "execute_result" 220 | } 221 | ], 222 | "source": [ 223 | "x = [0, 0, 1, 1, 0, 0, 1, 1]\n", 224 | "y = [0, 1, 1, 0, 0, 1, 1, 0]\n", 225 | "z = [0, 0, 0, 0, 1, 1, 1, 1]\n", 226 | "X = np.c_[x, y, z]\n", 227 | "i = [7, 0, 0, 0, 4, 4, 6, 6, 4, 0, 3, 2]\n", 228 | "j = [3, 4, 1, 2, 5, 6, 5, 2, 0, 1, 6, 3]\n", 229 | "k = [0, 7, 2, 3, 6, 7, 1, 1, 5, 5, 7, 6]\n", 230 | "Y = np.c_[i, j, k]\n", 231 | "envtest.mesh(X=X, Y=Y, opts=dict(opacity=0.5))" 232 | ] 233 | }, 234 | { 235 | "cell_type": "markdown", 236 | "metadata": {}, 237 | "source": [ 238 | "## 更新损失函数" 239 | ] 240 | }, 241 | { 242 | "cell_type": "markdown", 243 | "metadata": {}, 244 | "source": [ 245 | "在训练的时候我们每一批次都会打印一下训练的损失和测试的准确率，这样展示的图表是需要动态增加数据的，下面我们来模拟一下这种情况" 246 | ] 247 | }, 248 | { 249 | "cell_type": "code", 250 | "execution_count": 7, 251 | "metadata": {}, 252 | "outputs": [], 253 | "source": [ 254 | "x,y=0,0\n", 255 | "env2 = Visdom()\n", 256 | "pane1= env2.line(\n", 257 | " X=np.array([x]),\n", 258 | " Y=np.array([y]),\n", 259 | " opts=dict(title='dynamic data'))" 260 | ] 261 | }, 262 | { 263 | "cell_type": "code", 264 | "execution_count": 8, 265 | "metadata": {}, 266 | "outputs": [ 267 | { 268 | "name": "stdout", 269 | "output_type": "stream", 270 | "text": [ 271 | "0 0.0\n", 272 | "1 1.5\n", 273 | "3 5.25\n", 274 | "6 12.375\n", 275 | "10 24.5625\n", 276 | "15 44.34375\n", 277 | "21 75.515625\n", 278 | "28 123.7734375\n", 279 | "36 197.66015625\n", 280 | "45 309.990234375\n" 281 | ] 282 | } 283 | ], 284 | "source": [ 285 | "for i in range(10):\n", 286 | " time.sleep(1) #每隔一秒钟打印一次数据\n", 287 | " x+=i\n", 288 | " y=(y+i)*1.5\n", 289 | " print(x,y)\n", 290 | " env2.line(\n", 291 | " X=np.array([x]),\n", 292 | " Y=np.array([y]),\n", 293 | " win=pane1,#win参数确认使用哪一个pane\n", 294 | " update='append') #我们做的动作是追加，除了追加意外还有其他方式，这里我们不做介绍了" 295 | ] 296 | }, 297 | { 298 | "cell_type": "markdown", 299 | "metadata": {}, 300 | "source": [ 301 | "在运行完上述程序时，切换到visdom，看看效果吧\n", 302 | "\n", 303 | "visdom的基本用法介绍完毕，下一节介绍更加强大的 tensorboardx" 304 | ] 305 | }, 306 | { 307 | "cell_type": "code", 308 | "execution_count": null, 309 | "metadata": {}, 310 | "outputs": [], 311 | "source": [] 312 | } 313 | ], 314 | "metadata": { 315 | "kernelspec": { 316 | "display_name": "pytorch 1.0", 317 | "language": "python", 318 | "name": "pytorch1" 319 | }, 320 | "language_info": { 321 | "codemirror_mode": { 322 | "name": "ipython", 323 | "version": 3 324 | }, 325 | "file_extension": ".py", 326 | "mimetype": "text/x-python", 327 | "name": "python", 328 | "nbconvert_exporter": "python", 329 | "pygments_lexer": "ipython3", 330 | "version": "3.6.7" 331 | } 332 | }, 333 | "nbformat": 4, 334 | "nbformat_minor": 2 335 | } 336 | -------------------------------------------------------------------------------- /docs/chapter4/readme.md: -------------------------------------------------------------------------------- 1 | # Pytorch 中文手册第四章 ： 提高 2 | 3 | ## 目录 4 | 5 | ## 第一节 Fine-tuning 6 | 7 | [Fine-tuning](4.1-fine-tuning.ipynb) 8 | 9 | ## 第二节 可视化 10 | 11 | [visdom](4.2.1-visdom.ipynb) 12 | 13 | [tensorboardx](4.2.2-tensorboardx.ipynb) 14 | 15 | [可视化理解卷积神经网络](4.2.3-cnn-visualizing.ipynb) 16 | 17 | ## 第三节 Fast.ai 18 | [Fast.ai](4.3-fastai.ipynb) 19 | 20 | ## 第五节 多GPU并行训练 21 | [多GPU并行计算](4.5-multiply-gpu-parallel-training.ipynb) -------------------------------------------------------------------------------- /docs/chapter4/static.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/docs/chapter4/static.zip -------------------------------------------------------------------------------- /docs/chapter5/5.1-kaggle.md: -------------------------------------------------------------------------------- 1 | # 5.1 kaggle介绍 2 | ## 5.1.1 Kaggle 3 | ### 平台简介 4 | Kaggle是由联合创始人、首席执行官安东尼·高德布卢姆（Anthony Goldbloom）2010年在墨尔本创立的，主要为开发商和数据科学家提供举办机器学习竞赛、托管数据库、编写和分享代码的平台。该平台已经吸引了80万名数据科学家的关注，这些用户资源或许正是吸引谷歌的主要因素。 5 | 6 | 本段摘自[百度百科](https://baike.baidu.com/item/Kaggle/10449376) 7 | 8 | 通俗的说，Kaggle上面有着各种高质量的数据集，并且基于这些数据集会有一些大的公司进行赞助，来举办一些算法的竞赛，在竞赛时还可通过社区进行讨论（包括比赛中的分享、答疑，和比赛后的top solution分享）。这样就使得全球的顶尖高手会对参加比赛产生兴趣，因为收获到的不仅有奖金还有名气。同时kaggle对萌新也很友好，在社区中可以学到很多的知识和解决方案（top solution）。 9 | 10 | ### 比赛介绍 11 | kaggle 网站是纯英文的，我们可以把比赛分为2类: 12 | 13 | 1. 竞赛competitions：竞赛的目的很简单，就是要求在指定时间内（一般是2-3个月）使用出题方的数据完成指定的任务，如果有幸赢得比赛，不但可以获得奖金，模型也可能会被竞赛赞助商应用到商业实践中、获奖者直接进入赞助企业工作或者获得一些重大会议邀请、发表论文等 14 | 2. 数据集datasets：数据集是赞助商为了解决某些问题，免费公开了自己的内部的一些脱敏的数据，所有人都可以使用这些数据集进行研究，来改进现有的模型或者优化现有问题的解决方案，但是这些模型或解决方案可以不公开，所以这部分是不提供奖金的，只是能够与别人的方案做对比。 15 | 16 | ## 5.1.2 Kaggle板块介绍 17 | ### Data 18 | 19 | 这个不用多介绍，就是我们所使用的数据。 20 | 21 | 这个模块需要认值阅读，它介绍数据的产生方式、存储形式、每个字段的含义等。如果数据规模很大，你没有内存足够大的服务器可以hold住，可能就没法打这个比赛或者使用这个数据集；我们还要注意他的数据类型什么，比如图像数据那就得用深度神经网络，如果对这方面不熟或者没有GPU可用，肯定是没有办法参加这个比赛，不要妄想使用GPU来做CNN计算，至少目前来说是不可能的。 22 | 我们可将 kaggle 平台上的比赛分成一下4类： 23 | 24 | 1. 挖掘：面对的是结构化数据，也就是表格数据，包括了各式各样的预测问题(预测销量、点击率、推荐排序等)，主要的共性就是理解数据，理解问题，从数据中找到有用的信息用来预测，这类问题胜负更多的是在特征上，所以这部分是树形模型的天下比如有名的xgboost。 25 | 2. 图像：图像的话那肯定需要CNN了，这部分虽然可以用迁移学习来做，但是要想获得排名大量的算力也是必须的，因为要验证微调的次数也会很多。 26 | 3. 语音：这部分不太了解，就不献丑了 27 | 4. NLP ：BERT出来以后各种预训练的权重会对这方面的比赛帮助很大 28 | 29 | ### Rules 30 | 31 | 比赛规则。这个必须要看，无论是萌新还是大神，比赛的规则还是要遵守的。 32 | 33 | 提交次数：这里会写明每天允许的最大提交次数，一般是 5次，假如一个比赛持续时间是三个月，那么总提交次数差不多就是 5×90=450 次。 34 | 35 | 很多人为了躲过提交次数的限制或者“节省提交次数”，专门注册了小号，这被称为 multiple accounts，是会被 kaggle 的反作弊系统侦察出来的。在比赛结束后，会先公布初步排名，然后 kaggle 平台反作弊系统开始运行，大约两三天后，凡是被判为作弊的队伍直接从排名中移除，几个月的努力就打水漂了！所以这个操作一定要禁止。 36 | 37 | 另一个是组外私自分享代码和结果，这也是明令禁止的。组队之后队员之间可以分享，或者通过公开的 kernel或discussion区分享。同样，如果被检测出不同队伍或个人间有相似的结果，也会被移除最终榜单。 38 | ### Team 39 | 40 | 在参加比赛的时候可以向别人发起组队邀请，或者接受别人的邀请，三个臭皮匠顶个诸葛亮，组队的好处就不用说了，每个队伍最多4个人，并且是可以共享代码的，这样相当于算力提高了4倍。 41 | 42 | 另外就是一定要给队伍起个骚气的名字。 43 | 44 | ### Kernels 45 | 46 | 。。。(不知道怎么翻译，总之就是核心代码)。支持 Python 语言的脚本 .py 和 .ipynb，和 R 语言的脚本 .R 和 .ipynb。 47 | 48 | 分 public kernel 和 private kernel。 49 | 50 | public kernel是公开的，大家都可以看到，从这里可以学到非常多的东西，当然你自己也可以通过公开自己的 kernel 分享解决方案或观点。 51 | 52 | private kernel是你自己的，别人看不见，你可以分享给组内成员。 53 | 54 | 为方便大家打比赛，kaggle 提供了一些运算资源。kaggle 用户的每个 kernel 可以有 16G 的内存和 4 核CPU，这足够打多数比赛了。另外，提供了 GPU，在新建 kernel 的时候可以选择开启 GPU，但当打开 GPU 时，CPU 和内存资源会少一些，这个后面的薅羊毛指南中会详细介绍。 55 | 56 | 57 | ### Discussion 58 | 59 | 讨论区，这个区大家会分享观点、讨论问题、甚至寻找组队队友。 60 | 61 | kaggle 的分享氛围非常好，对萌新也非常友好。在整个比赛进程中大家不断地分享自己的新发现，很多有用的信息都是在这里获取的。 62 | 对于一个新手而言，每天做好 kernel 区和 discussion区的跟踪，有充足的时间尝试他们的想法，应该可以获得一个不错的排名。 63 | 比赛结束后，一些大牛甚至会将自己获胜用到的方法、小技巧（tricks）全部分享出来。 64 | 65 | ### Leaderboard 66 | 67 | 排名区，分 public LB 和 private LB。比赛方会将 test 数据集中一部分(比如 30%)拿出来做为 public LB 评分和排名，剩下的部分作为 private LB（也就是最终结果）的评分和排名。 68 | 69 | 你每天都可以提交并查看自己的答案在 public LB 的得分和排名情况，在比赛结束前需要选择两个提交作为自己的最终答案，比赛结束后，平台会计算你的答案的 private LB 得分并自动挑选得分高的一个作为你的最终成绩。 70 | 71 | 在讨论区你会经常听到大家讨论 CV score、LB score，指的就是你模型本地交叉验证的得分和提交后的 public LB 得分。 72 | 73 | shake up：public LB 得分可能会和 private LB 得分差别很大，比赛结果公布前你可能排名前十，比赛结果公布后发现自己跌到上千名了，这就是所谓的 shake up，一般是模型过拟合了，这个需要在训练的时候就要注意。 74 | 75 | 76 | ## 5.1.3 Kaggle竞赛的排名机制 77 | 一般情况下，都是以准确率为导向的排名，毕竟我们的模型计算的准确是第一标准。 78 | 79 | 针对于比赛而言，在比赛结束之前，参赛者每天最多可以提交5次测试集的预测结果。每一次提交结果都会获得最新的临时排名成绩，直至比赛结束获得最终排名，Kaggle将参赛者每次提交的结果取出25%-33%，并依照准确率进行临时排名。在比赛结束时，参赛者可以指定几个已经提交的结果，Kaggle从中去除之前用于临时排名的部分，用剩余数据的准确率综合得到最终排名。 80 | 81 | 所以，比赛过程中用于最终排名的那部分数据，参赛者是始终得不到关于准确率的反馈的。这样一定程度避免参赛模型的过拟合，保证评选出兼顾准确率和泛化能力的模型。 82 | 83 | ## 5.1.4 Kaggle薅羊毛指南 84 | Kaggle提供免费访问内核中的NVidia K80 GPU。我的个人理解相当于1060或者1070的水平吧。 85 | 我们可以在Kernel的界面右侧底端，Setting选项中将GPU开启，这样就可以使用免费的GPU资源了 86 | 87 | ![GPU设置](kaggle.png) 88 | 89 | Kaggle只提供6小时的连续GPU使用时间（印象中，待确认），虽然对于大数据量计算来说根本就不够用，但是对于研究，这些已经够了，例如我们在图像识别中经常用到的CIFAR-10 图片分类，一般训练2个小时左右就能够得到比较准确的模型了，这对于入门学习来说已经沟通了。 90 | 91 | ## 5.1.5 其他的一些数据竞赛平台 92 | 除了Kaggle，其实还有不少类似的平台； 93 | 94 | [DrivenData]( https://www.drivendata.org/) 95 | 96 | [CrowdANALYTIX]( https://www.crowdanalytix.com/community) 97 | 98 | [InnoCentive]( https://www.innocentive.com/our-solvers/) 99 | 100 | [TundIT]( https://towardsdatascience.com/top-competitive-data-science-platforms-other-than-kaggle-2995e9dad93c) 101 | 102 | [Codalab]( https://competitions.codalab.org/) 103 | 104 | [Analytics Vidhya]( https://datahack.analyticsvidhya.com/) 105 | 106 | [CrowdAI]( https://www.crowdai.org/challenges) 107 | 108 | [Numerai]( https://numer.ai/rounds) 109 | 110 | [Data Science Challenge]( https://www.datasciencechallenge.org/) 111 | 112 | [KDD Cup]( https://www.kdd.org/kdd2019/kdd-cup) 113 | 114 | [天池]( https://tianchi.aliyun.com/competition/gameList/activeList) 115 | 116 | [腾讯广告算法大赛]( https://algo.qq.com) 117 | -------------------------------------------------------------------------------- /docs/chapter5/kaggle.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/docs/chapter5/kaggle.png -------------------------------------------------------------------------------- /docs/chapter5/readme.md: -------------------------------------------------------------------------------- 1 | # Pytorch 中文手册第五章 ： 应用 2 | 3 | ## 目录 4 | 5 | ## 第一节 Kaggle介绍 6 | [Kaggle介绍](5.1-kaggle.md) 7 | ## 第二节 结构化数据 8 | ## 第三节 计算机视觉 9 | ## 第四节 自然语言处理 10 | ## 第五节 协同过滤 -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | absl-py==0.7.1 2 | astor==0.7.1 3 | backcall==0.1.0 4 | bleach==3.0.2 5 | certifi==2019.3.9 6 | chardet==3.0.4 7 | colorama==0.4.1 8 | cycler==0.10.0 9 | Cython==0.29.11 10 | decorator==4.3.0 11 | defusedxml==0.5.0 12 | entrypoints==0.2.3 13 | future==0.17.1 14 | gast==0.2.0 15 | google-pasta==0.1.7 16 | graphviz==0.10.1 17 | grpcio==1.16.0 18 | h5py==2.8.0 19 | idna==2.8 20 | imageio==2.5.0 21 | imageio-ffmpeg==0.3.0 22 | ipykernel==5.1.1 23 | ipython==7.2.0 24 | ipython-genutils==0.2.0 25 | ipywidgets==7.4.2 26 | itchat==1.2.32 27 | jedi==0.13.2 28 | jieba==0.39 29 | Jinja2==2.10.1 30 | jsonschema==2.6.0 31 | jupyter==1.0.0 32 | jupyter-client==5.2.4 33 | jupyter-console==6.0.0 34 | jupyter-contrib-core==0.3.3 35 | jupyter-core==4.4.0 36 | jupyter-nbextensions-configurator==0.4.1 37 | kafka-python==1.4.5 38 | Keras-Applications==1.0.6 39 | Keras-Preprocessing==1.0.5 40 | kiwisolver==1.0.1 41 | lightgbm==2.2.3 42 | Markdown==3.0.1 43 | MarkupSafe==1.1.0 44 | matplotlib==2.2.3 45 | mistune==0.8.4 46 | moviepy==1.0.0 47 | nbconvert==5.4.0 48 | nbextensions==0.1.0 49 | nbformat==4.4.0 50 | notebook==5.7.8 51 | numpy==1.16.3 52 | opencv-python==3.4.3.18 53 | pandas==0.23.4 54 | pandocfilters==1.4.2 55 | parso==0.3.1 56 | pickleshare==0.7.5 57 | Pillow==5.2.0 58 | proglog==0.1.9 59 | prometheus-client==0.5.0 60 | prompt-toolkit==2.0.7 61 | protobuf==3.8.0 62 | py4j==0.10.7 63 | pycocotools==2.0 64 | Pygments==2.3.1 65 | PyHDFS==0.2.1 66 | pyparsing==2.2.0 67 | pypng==0.0.19 68 | PyQRCode==1.2.1 69 | pyspark==2.3.3 70 | python-dateutil==2.7.3 71 | python-votesmart==0.3.3 72 | pytz==2018.5 73 | pywinpty==0.5.5 74 | PyYAML==5.1.1 75 | pyzmq==17.1.2 76 | qtconsole==4.4.3 77 | requests==2.21.0 78 | scikit-learn==0.20.2 79 | scipy==1.2.0 80 | seaborn==0.9.0 81 | Send2Trash==1.5.0 82 | simplejson==3.16.0 83 | six==1.11.0 84 | sklearn==0.0 85 | tensorboard==1.14.0 86 | tensorboardX==1.7 87 | tensorflow==1.14.0 88 | tensorflow-estimator==1.14.0 89 | termcolor==1.1.0 90 | terminado==0.8.1 91 | testpath==0.4.2 92 | torch==1.1.0 93 | torchvision==0.3.0 94 | tornado==5.1.1 95 | tqdm==4.32.2 96 | traitlets==4.3.2 97 | urllib3==1.25.3 98 | wcwidth==0.1.7 99 | webencodings==0.5.1 100 | Werkzeug==0.15.4 101 | widgetsnbextension==3.4.2 102 | wrapt==1.11.2 103 | wxpy==0.3.9.8 104 | xgboost==0.81 105 | -------------------------------------------------------------------------------- /screenshots/cuda_version.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/screenshots/cuda_version.png -------------------------------------------------------------------------------- /screenshots/python_version.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/screenshots/python_version.png -------------------------------------------------------------------------------- /screenshots/test_torch.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/screenshots/test_torch.png -------------------------------------------------------------------------------- /src/PyTorch_Tutorial/Code/1_data_prepare/1_1_cifar10_to_png.py: -------------------------------------------------------------------------------- 1 | # coding:utf-8 2 | """ 3 | 将cifar10的data_batch_12345 转换成 png格式的图片 4 | 每个类别单独存放在一个文件夹，文件夹名称为0-9 5 | """ 6 | 7 | import numpy as np 8 | import os 9 | import pickle 10 | 11 | from imageio import imwrite 12 | 13 | data_dir = '../../Data/cifar-10-batches-py/' 14 | train_o_dir = '../../Data/cifar-10-png/raw_train/' 15 | test_o_dir = '../../Data/cifar-10-png/raw_test/' 16 | 17 | Train = False # 不解压训练集，仅解压测试集 18 | 19 | 20 | # 解压缩，返回解压后的字典 21 | def unpickle(file): 22 | fo = open(file, 'rb') 23 | dict_ = pickle.load(fo, encoding='bytes') 24 | fo.close() 25 | return dict_ 26 | 27 | 28 | def my_mkdir(my_dir): 29 | if not os.path.isdir(my_dir): 30 | os.makedirs(my_dir) 31 | 32 | 33 | # 生成训练集图片， 34 | if __name__ == '__main__': 35 | if Train: 36 | for j in range(1, 6): 37 | data_path = data_dir + "data_batch_" + str(j) # data_batch_12345 38 | train_data = unpickle(data_path) 39 | print(data_path + " is loading...") 40 | 41 | for i in range(0, 10000): 42 | img = np.reshape(train_data[b'data'][i], (3, 32, 32)) 43 | img = img.transpose(1, 2, 0) 44 | 45 | label_num = str(train_data[b'labels'][i]) 46 | o_dir = os.path.join(train_o_dir, label_num) 47 | my_mkdir(o_dir) 48 | 49 | img_name = label_num + '_' + str(i + (j - 1) * 10000) + '.png' 50 | img_path = os.path.join(o_dir, img_name) 51 | imwrite(img_path, img) 52 | print(data_path + " loaded.") 53 | 54 | print("test_batch is loading...") 55 | 56 | # 生成测试集图片 57 | test_data_path = data_dir + "test_batch" 58 | test_data = unpickle(test_data_path) 59 | for i in range(0, 10000): 60 | img = np.reshape(test_data[b'data'][i], (3, 32, 32)) 61 | img = img.transpose(1, 2, 0) 62 | 63 | label_num = str(test_data[b'labels'][i]) 64 | o_dir = os.path.join(test_o_dir, label_num) 65 | my_mkdir(o_dir) 66 | 67 | img_name = label_num + '_' + str(i) + '.png' 68 | img_path = os.path.join(o_dir, img_name) 69 | imwrite(img_path, img) 70 | 71 | print("test_batch loaded.") 72 | -------------------------------------------------------------------------------- /src/PyTorch_Tutorial/Code/1_data_prepare/1_2_split_dataset.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | """ 3 | 将原始数据集进行划分成训练集、验证集和测试集 4 | """ 5 | 6 | import os 7 | import glob 8 | import random 9 | import shutil 10 | 11 | dataset_dir = '../../Data/cifar-10-png/raw_test/' 12 | train_dir = '../../Data/train/' 13 | valid_dir = '../../Data/valid/' 14 | test_dir = '../../Data/test/' 15 | 16 | train_per = 0.8 17 | valid_per = 0.1 18 | test_per = 0.1 19 | 20 | 21 | def makedir(new_dir): 22 | if not os.path.exists(new_dir): 23 | os.makedirs(new_dir) 24 | 25 | 26 | if __name__ == '__main__': 27 | 28 | for root, dirs, files in os.walk(dataset_dir): 29 | for sDir in dirs: 30 | imgs_list = glob.glob(os.path.join(root, sDir)+'/*.png') 31 | random.seed(666) 32 | random.shuffle(imgs_list) 33 | imgs_num = len(imgs_list) 34 | 35 | train_point = int(imgs_num * train_per) 36 | valid_point = int(imgs_num * (train_per + valid_per)) 37 | 38 | for i in range(imgs_num): 39 | if i < train_point: 40 | out_dir = train_dir + sDir + '/' 41 | elif i < valid_point: 42 | out_dir = valid_dir + sDir + '/' 43 | else: 44 | out_dir = test_dir + sDir + '/' 45 | 46 | makedir(out_dir) 47 | out_path = out_dir + os.path.split(imgs_list[i])[-1] 48 | shutil.copy(imgs_list[i], out_path) 49 | 50 | print('Class:{}, train:{}, valid:{}, test:{}'.format(sDir, train_point, valid_point-train_point, imgs_num-valid_point)) 51 | -------------------------------------------------------------------------------- /src/PyTorch_Tutorial/Code/1_data_prepare/1_3_generate_txt.py: -------------------------------------------------------------------------------- 1 | # coding:utf-8 2 | import os 3 | ''' 4 | 为数据集生成对应的txt文件 5 | ''' 6 | 7 | train_txt_path = '../../Data/train.txt' 8 | train_dir = '../../Data/train/' 9 | valid_txt_path = '../../Data/valid.txt' 10 | valid_dir = '../../Data/valid/' 11 | 12 | 13 | def gen_txt(txt_path, img_dir): 14 | f = open(txt_path, 'w') 15 | 16 | for root, s_dirs, _ in os.walk(img_dir, topdown=True): # 获取 train文件下各文件夹名称 17 | for sub_dir in s_dirs: 18 | i_dir = os.path.join(root, sub_dir) # 获取各类的文件夹 绝对路径 19 | img_list = os.listdir(i_dir) # 获取类别文件夹下所有png图片的路径 20 | for i in range(len(img_list)): 21 | if not img_list[i].endswith('png'): # 若不是png文件，跳过 22 | continue 23 | label = img_list[i].split('_')[0] 24 | img_path = os.path.join(i_dir, img_list[i]) 25 | line = img_path + ' ' + label + '\n' 26 | f.write(line) 27 | f.close() 28 | 29 | 30 | if __name__ == '__main__': 31 | gen_txt(train_txt_path, train_dir) 32 | gen_txt(valid_txt_path, valid_dir) -------------------------------------------------------------------------------- /src/PyTorch_Tutorial/Code/1_data_prepare/1_3_mydataset.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | from PIL import Image 3 | from torch.utils.data import Dataset 4 | 5 | 6 | class MyDataset(Dataset): 7 | # 从我们准备好的 txt 里获取图片的路径和标签， 并且存储在 self.imgs 8 | def __init__(self, txt_path, transform=None, target_transform=None): 9 | fh = open(txt_path, 'r') 10 | imgs = [] 11 | for line in fh: 12 | line = line.rstrip() 13 | words = line.split() 14 | # (路径,标签) 15 | imgs.append((words[0], int(words[1]))) 16 | 17 | # 最主要就是要生成这个list， 然后DataLoader中给index，通过getitem读取图片数据 18 | self.imgs = imgs 19 | self.transform = transform 20 | self.target_transform = target_transform 21 | 22 | def __getitem__(self, index): 23 | # fn-路径，label-标签 24 | fn, label = self.imgs[index] 25 | 26 | # 利用Image.open对图片进行读取 27 | # 像素值 0~255，在transfrom.totensor会除以255，使像素值变成 0~1 28 | img = Image.open(fn).convert('RGB') 29 | 30 | if self.transform is not None: 31 | # 在这里做transform，转为tensor等等 32 | # PyTorch 做数据增强的方法是在原始图片上进行的，并覆盖原始图片，这一点需要注意 33 | img = self.transform(img) 34 | 35 | return img, label 36 | 37 | def __len__(self): 38 | return len(self.imgs) -------------------------------------------------------------------------------- /src/PyTorch_Tutorial/Code/1_data_prepare/1_5_compute_mean.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | 3 | import numpy as np 4 | import cv2 5 | import random 6 | 7 | """ 8 | 随机挑选CNum张图片，进行按通道计算均值mean和标准差std 9 | 先将像素从0～255归一化至 0-1 再计算 10 | """ 11 | 12 | 13 | train_txt_path = '../../Data/train.txt' 14 | 15 | CNum = 2000 # 挑选多少图片进行计算 16 | 17 | img_h, img_w = 32, 32 18 | imgs = np.zeros([img_w, img_h, 3, 1]) 19 | means, stdevs = [], [] 20 | 21 | with open(train_txt_path, 'r') as f: 22 | lines = f.readlines() 23 | random.shuffle(lines) # shuffle , 随机挑选图片 24 | 25 | for i in range(CNum): 26 | img_path = lines[i].rstrip().split()[0] 27 | 28 | img = cv2.imread(img_path) 29 | img = cv2.resize(img, (img_h, img_w)) 30 | 31 | img = img[:, :, :, np.newaxis] 32 | imgs = np.concatenate((imgs, img), axis=3) 33 | print(i) 34 | 35 | imgs = imgs.astype(np.float32)/255. 36 | 37 | 38 | for i in range(3): 39 | pixels = imgs[:,:,i,:].ravel() # 拉成一行 40 | means.append(np.mean(pixels)) 41 | stdevs.append(np.std(pixels)) 42 | 43 | means.reverse() # BGR --> RGB 44 | stdevs.reverse() 45 | 46 | print("normMean = {}".format(means)) 47 | print("normStd = {}".format(stdevs)) 48 | print('transforms.Normalize(normMean = {}, normStd = {})'.format(means, stdevs)) 49 | 50 | -------------------------------------------------------------------------------- /src/PyTorch_Tutorial/Code/1_data_prepare/1_5_compute_mean_bak.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | 3 | import numpy as np 4 | import cv2 5 | import random 6 | 7 | """ 8 | 随机挑选CNum张图片，进行按通道计算均值mean和标准差std 9 | 先将像素从0～255归一化至 0-1 再计算 10 | """ 11 | 12 | 13 | train_txt_path = 'D:\\BehaviorAnalysis\\trunk\\src\\poseEstimation1.0.1\\data\\temp_image\\train.txt' 14 | 15 | CNum = 20000 # 挑选多少图片进行计算 16 | 17 | img_h, img_w = 32, 32 18 | imgs = np.zeros([img_w, img_h, 3, 1]) 19 | means, stdevs = [], [] 20 | 21 | with open(train_txt_path, 'r') as f: 22 | lines = f.readlines() 23 | random.shuffle(lines) # shuffle , 随机挑选图片 24 | 25 | for i in range(CNum): 26 | img_path = lines[i].rstrip().split()[0] 27 | 28 | img = cv2.imread(img_path) 29 | img = cv2.resize(img, (img_h, img_w)) 30 | 31 | img = img[:, :, :, np.newaxis] 32 | imgs = np.concatenate((imgs, img), axis=3) 33 | print(i) 34 | 35 | imgs = imgs.astype(np.float32)/255. 36 | 37 | 38 | for i in range(3):#三通道 39 | pixels = imgs[:,:,i,:].ravel() # 拉成一行 40 | means.append(np.mean(pixels)) 41 | stdevs.append(np.std(pixels)) 42 | 43 | means.reverse() # BGR --> RGB 44 | stdevs.reverse() 45 | 46 | print("normMean = {}".format(means)) 47 | print("normStd = {}".format(stdevs)) 48 | print('transforms.Normalize(normMean = {}, normStd = {})'.format(means, stdevs)) 49 | 50 | -------------------------------------------------------------------------------- /src/PyTorch_Tutorial/Code/2_model/2_finetune.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | 3 | import torch 4 | from torch.utils.data import DataLoader 5 | import torchvision.transforms as transforms 6 | import numpy as np 7 | import os 8 | from torch.autograd import Variable 9 | import torch.nn as nn 10 | import torch.nn.functional as F 11 | import torch.optim as optim 12 | import sys 13 | from Code.utils.utils import MyDataset, validate, show_confMat 14 | from datetime import datetime 15 | 16 | sys.path.append("..") 17 | 18 | train_txt_path = '../../Data/train.txt' 19 | valid_txt_path = '../../Data/valid.txt' 20 | 21 | classes_name = ['plane', 'car', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck'] 22 | 23 | train_bs = 16 24 | valid_bs = 16 25 | lr_init = 0.001 26 | max_epoch = 10 27 | 28 | # log 29 | result_dir = '../../Result/' 30 | 31 | now_time = datetime.now() 32 | time_str = datetime.strftime(now_time, '%m-%d_%H-%M-%S') 33 | 34 | log_dir = os.path.join(result_dir, time_str) 35 | if not os.path.exists(log_dir): 36 | os.makedirs(log_dir) 37 | 38 | # -------------------------------------------- step 1/5 : 加载数据 ------------------------------------------- 39 | 40 | # 数据预处理设置 41 | normMean = [0.4948052, 0.48568845, 0.44682974] 42 | normStd = [0.24580306, 0.24236229, 0.2603115] 43 | normTransform = transforms.Normalize(normMean, normStd) 44 | trainTransform = transforms.Compose([ 45 | transforms.Resize(32), 46 | transforms.RandomCrop(32, padding=4), 47 | transforms.ToTensor(), 48 | normTransform 49 | ]) 50 | 51 | validTransform = transforms.Compose([ 52 | transforms.ToTensor(), 53 | normTransform 54 | ]) 55 | 56 | # 构建MyDataset实例,读取数据路径和标签 57 | train_data = MyDataset(txt_path=train_txt_path, transform=trainTransform) 58 | valid_data = MyDataset(txt_path=valid_txt_path, transform=validTransform) 59 | 60 | # 构建DataLoder ,生成器 61 | train_loader = DataLoader(dataset=train_data, batch_size=train_bs, shuffle=True) 62 | valid_loader = DataLoader(dataset=valid_data, batch_size=valid_bs) 63 | 64 | 65 | # ------------------------------------ step 2/5 : 定义网络 ------------------------------------ 66 | 67 | 68 | class Net(nn.Module): 69 | def __init__(self): 70 | super(Net, self).__init__() 71 | self.conv1 = nn.Conv2d(3, 6, 5) 72 | self.pool1 = nn.MaxPool2d(2, 2) 73 | self.conv2 = nn.Conv2d(6, 16, 5) 74 | self.pool2 = nn.MaxPool2d(2, 2) 75 | self.fc1 = nn.Linear(16 * 5 * 5, 120) 76 | self.fc2 = nn.Linear(120, 84) 77 | self.fc3 = nn.Linear(84, 10) 78 | 79 | def forward(self, x): 80 | x = self.pool1(F.relu(self.conv1(x))) 81 | x = self.pool2(F.relu(self.conv2(x))) 82 | x = x.view(-1, 16 * 5 * 5) 83 | x = F.relu(self.fc1(x)) 84 | x = F.relu(self.fc2(x)) 85 | x = self.fc3(x) 86 | return x 87 | 88 | # 定义权值初始化 89 | def initialize_weights(self): 90 | for m in self.modules(): 91 | if isinstance(m, nn.Conv2d): 92 | torch.nn.init.xavier_normal_(m.weight.data) 93 | if m.bias is not None: 94 | m.bias.data.zero_() 95 | elif isinstance(m, nn.BatchNorm2d): 96 | m.weight.data.fill_(1) 97 | m.bias.data.zero_() 98 | elif isinstance(m, nn.Linear): 99 | torch.nn.init.normal_(m.weight.data, 0, 0.01) 100 | m.bias.data.zero_() 101 | 102 | 103 | net = Net() # 创建一个网络 104 | 105 | # ================================ # 106 | # finetune 权值初始化 107 | # ================================ # 108 | 109 | # load params 110 | pretrained_dict = torch.load('net_params.pkl') 111 | 112 | # 获取当前网络的dict 113 | net_state_dict = net.state_dict() 114 | 115 | # 剔除不匹配的权值参数 116 | pretrained_dict_1 = {k: v for k, v in pretrained_dict.items() if k in net_state_dict} 117 | 118 | # 更新新模型参数字典 119 | net_state_dict.update(pretrained_dict_1) 120 | 121 | # 将包含预训练模型参数的字典"放"到新模型中 122 | net.load_state_dict(net_state_dict) 123 | 124 | # ------------------------------------ step 3/5 : 定义损失函数和优化器 ------------------------------------ 125 | # ================================= # 126 | # 按需设置学习率 127 | # ================================= # 128 | 129 | # 将fc3层的参数从原始网络参数中剔除 130 | ignored_params = list(map(id, net.fc3.parameters())) 131 | base_params = filter(lambda p: id(p) not in ignored_params, net.parameters()) 132 | 133 | # 为fc3层设置需要的学习率 134 | optimizer = optim.SGD([ 135 | {'params': base_params}, 136 | {'params': net.fc3.parameters(), 'lr': lr_init * 10}], lr_init, momentum=0.9, weight_decay=1e-4) 137 | 138 | criterion = nn.CrossEntropyLoss() # 选择损失函数 139 | scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size=50, gamma=0.1) # 设置学习率下降策略 140 | 141 | # ------------------------------------ step 4/5 : 训练 -------------------------------------------------- 142 | 143 | for epoch in range(max_epoch): 144 | 145 | loss_sigma = 0.0 # 记录一个epoch的loss之和 146 | correct = 0.0 147 | total = 0.0 148 | scheduler.step() # 更新学习率 149 | 150 | for i, data in enumerate(train_loader): 151 | # 获取图片和标签 152 | inputs, labels = data 153 | inputs, labels = Variable(inputs), Variable(labels) 154 | 155 | # forward, backward, update weights 156 | optimizer.zero_grad() 157 | outputs = net(inputs) 158 | loss = criterion(outputs, labels) 159 | loss.backward() 160 | optimizer.step() 161 | 162 | # 统计预测信息 163 | _, predicted = torch.max(outputs.data, 1) 164 | total += labels.size(0) 165 | correct += (predicted == labels).squeeze().sum().numpy() 166 | loss_sigma += loss.item() 167 | 168 | # 每10个iteration 打印一次训练信息，loss为10个iteration的平均 169 | if i % 10 == 9: 170 | loss_avg = loss_sigma / 10 171 | loss_sigma = 0.0 172 | print("Training: Epoch[{:0>3}/{:0>3}] Iteration[{:0>3}/{:0>3}] Loss: {:.4f} Acc:{:.2%}".format( 173 | epoch + 1, max_epoch, i + 1, len(train_loader), loss_avg, correct / total)) 174 | print('参数组1的学习率:{}, 参数组2的学习率:{}'.format(scheduler.get_lr()[0], scheduler.get_lr()[1])) 175 | # ------------------------------------ 观察模型在验证集上的表现 ------------------------------------ 176 | loss_sigma = 0.0 177 | cls_num = len(classes_name) 178 | conf_mat = np.zeros([cls_num, cls_num]) # 混淆矩阵 179 | net.eval() 180 | for i, data in enumerate(valid_loader): 181 | 182 | # 获取图片和标签 183 | images, labels = data 184 | images, labels = Variable(images), Variable(labels) 185 | 186 | # forward 187 | outputs = net(images) 188 | outputs.detach_() 189 | 190 | # 计算loss 191 | loss = criterion(outputs, labels) 192 | loss_sigma += loss.item() 193 | 194 | # 统计 195 | _, predicted = torch.max(outputs.data, 1) 196 | # labels = labels.data # Variable --> tensor 197 | 198 | # 统计混淆矩阵 199 | for j in range(len(labels)): 200 | cate_i = labels[j].numpy() 201 | pre_i = predicted[j].numpy() 202 | conf_mat[cate_i, pre_i] += 1.0 203 | 204 | print('{} set Accuracy:{:.2%}'.format('Valid', conf_mat.trace() / conf_mat.sum())) 205 | print('Finished Training') 206 | 207 | # ------------------------------------ step5: 绘制混淆矩阵图 ------------------------------------ 208 | 209 | conf_mat_train, train_acc = validate(net, train_loader, 'train', classes_name) 210 | conf_mat_valid, valid_acc = validate(net, valid_loader, 'valid', classes_name) 211 | 212 | show_confMat(conf_mat_train, classes_name, 'train', log_dir) 213 | show_confMat(conf_mat_valid, classes_name, 'valid', log_dir) 214 | -------------------------------------------------------------------------------- /src/PyTorch_Tutorial/Code/2_model/net_params.pkl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/src/PyTorch_Tutorial/Code/2_model/net_params.pkl -------------------------------------------------------------------------------- /src/PyTorch_Tutorial/Code/3_optimizer/3_1_lossFunction/1.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/src/PyTorch_Tutorial/Code/3_optimizer/3_1_lossFunction/1.jpg -------------------------------------------------------------------------------- /src/PyTorch_Tutorial/Code/3_optimizer/3_1_lossFunction/1_L1Loss.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | 3 | import torch 4 | import torch.nn as nn 5 | 6 | # ----------------------------------- L1 Loss 7 | 8 | # 生成网络输出 以及 目标输出 9 | output = torch.ones(2, 2, requires_grad=True)*0.5 10 | target = torch.ones(2, 2) 11 | 12 | # 设置三种不同参数的L1Loss 13 | reduce_False = nn.L1Loss(reduction='none') 14 | size_average_True = nn.L1Loss(reduction='mean') 15 | size_average_False = nn.L1Loss(reduction='sum') 16 | 17 | o_0 = reduce_False(output, target) 18 | o_1 = size_average_True(output, target) 19 | o_2 = size_average_False(output, target) 20 | 21 | print('\nreduce="none", 输出同维度的loss:\n{}\n'.format(o_0)) 22 | print('reduction="mean"，\t求平均:\t{}'.format(o_1)) 23 | print('reduction="sum"，\t求和:\t{}'.format(o_2)) 24 | -------------------------------------------------------------------------------- /src/PyTorch_Tutorial/Code/3_optimizer/3_1_lossFunction/2_MSELoss.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | 3 | import torch 4 | import torch.nn as nn 5 | 6 | # ----------------------------------- MSE loss 7 | 8 | # 生成网络输出 以及 目标输出 9 | output = torch.ones(2, 2, requires_grad=True) * 0.5 10 | target = torch.ones(2, 2) 11 | 12 | # 设置三种不同参数的L1Loss 13 | reduce_False = nn.MSELoss(reduction='none') 14 | size_average_True = nn.MSELoss(reduction='mean') 15 | size_average_False = nn.MSELoss(reduction='sum') 16 | 17 | o_0 = reduce_False(output, target) 18 | o_1 = size_average_True(output, target) 19 | o_2 = size_average_False(output, target) 20 | 21 | print('\nreduce=False, 输出同维度的loss:\n{}\n'.format(o_0)) 22 | print('size_average=True，\t求平均:\t{}'.format(o_1)) 23 | print('size_average=False，\t求和:\t{}'.format(o_2)) 24 | -------------------------------------------------------------------------------- /src/PyTorch_Tutorial/Code/3_optimizer/3_1_lossFunction/3_CrossEntropyLoss.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | 3 | import torch 4 | import torch.nn as nn 5 | import numpy as np 6 | import math 7 | 8 | # ----------------------------------- CrossEntropy loss: base 9 | 10 | loss_f = nn.CrossEntropyLoss(weight=None, reduction='none') 11 | # 生成网络输出 以及 目标输出 12 | output = torch.ones(2, 3, requires_grad=True) * 0.5 # 假设一个三分类任务，batchsize=2，假设每个神经元输出都为0.5 13 | target = torch.from_numpy(np.array([0, 1])).type(torch.LongTensor) 14 | 15 | loss = loss_f(output, target) 16 | 17 | print('--------------------------------------------------- CrossEntropy loss: base') 18 | print('loss: ', loss) 19 | print('由于reduce=False，所以可以看到每一个样本的loss，输出为[1.0986, 1.0986]') 20 | 21 | 22 | # 熟悉计算公式，手动计算第一个样本 23 | output = output[0].detach().numpy() 24 | output_1 = output[0] # 第一个样本的输出值 25 | target_1 = target[0].numpy() 26 | 27 | # 第一项 28 | x_class = output[target_1] 29 | # 第二项 30 | exp = math.e 31 | sigma_exp_x = pow(exp, output[0]) + pow(exp, output[1]) + pow(exp, output[2]) 32 | log_sigma_exp_x = math.log(sigma_exp_x) 33 | # 两项相加 34 | loss_1 = -x_class + log_sigma_exp_x 35 | print('--------------------------------------------------- 手动计算') 36 | print('第一个样本的loss：', loss_1) 37 | 38 | 39 | # ----------------------------------- CrossEntropy loss: weight 40 | 41 | weight = torch.from_numpy(np.array([0.6, 0.2, 0.2])).float() 42 | loss_f = nn.CrossEntropyLoss(weight=weight, reduction='none') 43 | output = torch.ones(2, 3, requires_grad=True) * 0.5 # 假设一个三分类任务，batchsize为2个，假设每个神经元输出都为0.5 44 | target = torch.from_numpy(np.array([0, 1])).type(torch.LongTensor) 45 | loss = loss_f(output, target) 46 | print('\n\n--------------------------------------------------- CrossEntropy loss: weight') 47 | print('loss: ', loss) # 48 | print('原始loss值为1.0986, 第一个样本是第0类，weight=0.6,所以输出为1.0986*0.6 =', 1.0986*0.6) 49 | 50 | # ----------------------------------- CrossEntropy loss: ignore_index 51 | 52 | loss_f_1 = nn.CrossEntropyLoss(weight=None, reduction='none', ignore_index=1) 53 | loss_f_2 = nn.CrossEntropyLoss(weight=None, reduction='none', ignore_index=2) 54 | 55 | output = torch.ones(3, 3, requires_grad=True) * 0.5 # 假设一个三分类任务，batchsize为2个，假设每个神经元输出都为0.5 56 | target = torch.from_numpy(np.array([0, 1, 2])).type(torch.LongTensor) 57 | 58 | loss_1 = loss_f_1(output, target) 59 | loss_2 = loss_f_2(output, target) 60 | 61 | print('\n\n--------------------------------------------------- CrossEntropy loss: ignore_index') 62 | print('ignore_index = 1: ', loss_1) # 类别为1的样本的loss为0 63 | print('ignore_index = 2: ', loss_2) # 类别为2的样本的loss为0 64 | -------------------------------------------------------------------------------- /src/PyTorch_Tutorial/Code/3_optimizer/3_1_lossFunction/4_NLLLoss.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | 3 | import torch 4 | import torch.nn as nn 5 | import numpy as np 6 | 7 | # ----------------------------------- log likelihood loss 8 | 9 | # 各类别权重 10 | weight = torch.from_numpy(np.array([0.6, 0.2, 0.2])).float() 11 | 12 | # 生成网络输出 以及 目标输出 13 | output = torch.from_numpy(np.array([[0.7, 0.2, 0.1], [0.4, 1.2, 0.4]])).float() 14 | output.requires_grad = True 15 | target = torch.from_numpy(np.array([0, 0])).type(torch.LongTensor) 16 | 17 | 18 | loss_f = nn.NLLLoss(weight=weight, reduction='none') 19 | loss = loss_f(output, target) 20 | 21 | print('\nloss: \n', loss) 22 | print('\n第一个样本是0类，loss = -(0.6*0.7)={}'.format(loss[0])) 23 | print('\n第二个样本是0类，loss = -(0.6*0.4)={}'.format(loss[1])) -------------------------------------------------------------------------------- /src/PyTorch_Tutorial/Code/3_optimizer/3_1_lossFunction/5_PoissonNLLLoss.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | 3 | import torch 4 | import torch.nn as nn 5 | import numpy as np 6 | 7 | # ----------------------------------- Poisson NLLLoss 8 | 9 | # 生成网络输出 以及 目标输出 10 | log_input = torch.randn(5, 2, requires_grad=True) 11 | target = torch.randn(5, 2) 12 | 13 | loss_f = nn.PoissonNLLLoss() 14 | loss = loss_f(log_input, target) 15 | print('\nloss: \n', loss) 16 | 17 | -------------------------------------------------------------------------------- /src/PyTorch_Tutorial/Code/3_optimizer/3_1_lossFunction/6_KLDivLoss.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | 3 | import numpy as np 4 | import torch 5 | import torch.nn as nn 6 | 7 | # ----------------------------------- KLDiv loss 8 | 9 | loss_f = nn.KLDivLoss(reduction='none') 10 | loss_f_mean = nn.KLDivLoss(reduction='batchmean') 11 | 12 | # 生成网络输出 以及 目标输出 13 | output = torch.from_numpy(np.array([[0.1132, 0.5477, 0.3390]])).float() 14 | output.requires_grad = True 15 | target = torch.from_numpy(np.array([[0.8541, 0.0511, 0.0947]])).float() 16 | 17 | loss_1 = loss_f(output, target) 18 | loss_mean = loss_f_mean(output, target) 19 | 20 | print('\nloss: ', loss_1) 21 | print('\nloss_mean: ', torch.div(loss_mean, 3)) 22 | 23 | 24 | # 熟悉计算公式，手动计算样本的第一个元素的loss，注意这里只有一个样本，是 element-wise计算的 25 | 26 | output = output[0].detach().numpy() 27 | output_1 = output[0] # 第一个样本的第一个元素 28 | target_1 = target[0][0].numpy() 29 | 30 | loss_1 = target_1 * (np.log(target_1) - output_1) 31 | 32 | print('\n第一个样本第一个元素的loss：', loss_1) 33 | 34 | 35 | 36 | -------------------------------------------------------------------------------- /src/PyTorch_Tutorial/Code/3_optimizer/3_2_optimizer/1_param_groups.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | 3 | import torch 4 | import torch.optim as optim 5 | 6 | w1 = torch.randn(2, 2) 7 | w1.requires_grad = True 8 | 9 | w2 = torch.randn(2, 2) 10 | w2.requires_grad = True 11 | 12 | w3 = torch.randn(2, 2) 13 | w3.requires_grad = True 14 | 15 | # 一个参数组 16 | optimizer_1 = optim.SGD([w1, w3], lr=0.1) 17 | print('len(optimizer.param_groups): ', len(optimizer_1.param_groups)) 18 | print(optimizer_1.param_groups, '\n') 19 | 20 | # 两个参数组 21 | optimizer_2 = optim.SGD([{'params': w1, 'lr': 0.1}, 22 | {'params': w2, 'lr': 0.001}]) 23 | print('len(optimizer.param_groups): ', len(optimizer_2.param_groups)) 24 | print(optimizer_2.param_groups) 25 | -------------------------------------------------------------------------------- /src/PyTorch_Tutorial/Code/3_optimizer/3_2_optimizer/2_zero_grad.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | 3 | import torch 4 | import torch.optim as optim 5 | 6 | # ----------------------------------- zero_grad 7 | 8 | w1 = torch.randn(2, 2) 9 | w1.requires_grad = True 10 | 11 | w2 = torch.randn(2, 2) 12 | w2.requires_grad = True 13 | 14 | optimizer = optim.SGD([w1, w2], lr=0.001, momentum=0.9) 15 | 16 | optimizer.param_groups[0]['params'][0].grad = torch.randn(2, 2) 17 | 18 | print('参数w1的梯度：') 19 | print(optimizer.param_groups[0]['params'][0].grad, '\n') # 参数组，第一个参数(w1)的梯度 20 | 21 | optimizer.zero_grad() 22 | print('执行zero_grad()之后，参数w1的梯度：') 23 | print(optimizer.param_groups[0]['params'][0].grad) # 参数组，第一个参数(w1)的梯度 24 | -------------------------------------------------------------------------------- /src/PyTorch_Tutorial/Code/3_optimizer/3_2_optimizer/3_state_dict.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | 3 | import torch.nn as nn 4 | import torch.nn.functional as F 5 | 6 | 7 | # ----------------------------------- state_dict 8 | class Net(nn.Module): 9 | def __init__(self): 10 | super(Net, self).__init__() 11 | self.conv1 = nn.Conv2d(3, 1, 3) 12 | self.pool = nn.MaxPool2d(2, 2) 13 | self.fc1 = nn.Linear(1 * 3 * 3, 2) 14 | 15 | def forward(self, x): 16 | x = self.pool(F.relu(self.conv1(x))) 17 | x = x.view(-1, 1 * 3 * 3) 18 | x = F.relu(self.fc1(x)) 19 | return x 20 | 21 | 22 | net = Net() 23 | 24 | # 获取网络当前参数 25 | net_state_dict = net.state_dict() 26 | 27 | print('net_state_dict类型：', type(net_state_dict)) 28 | print('net_state_dict管理的参数: ', net_state_dict.keys()) 29 | for key, value in net_state_dict.items(): 30 | print('参数名: ', key, '\t大小: ', value.shape) 31 | -------------------------------------------------------------------------------- /src/PyTorch_Tutorial/Code/3_optimizer/3_2_optimizer/4_load_state_dict.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | 3 | import torch 4 | import torch.nn as nn 5 | import torch.nn.functional as F 6 | 7 | 8 | # ----------------------------------- load_state_dict 9 | 10 | class Net(nn.Module): 11 | def __init__(self): 12 | super(Net, self).__init__() 13 | self.conv1 = nn.Conv2d(3, 1, 3) 14 | self.pool = nn.MaxPool2d(2, 2) 15 | self.fc1 = nn.Linear(1 * 3 * 3, 2) 16 | 17 | def forward(self, x): 18 | x = self.pool(F.relu(self.conv1(x))) 19 | x = x.view(-1, 1 * 3 * 3) 20 | x = F.relu(self.fc1(x)) 21 | return x 22 | 23 | def zero_param(self): 24 | for m in self.modules(): 25 | if isinstance(m, nn.Conv2d): 26 | torch.nn.init.constant_(m.weight.data, 0) 27 | if m.bias is not None: 28 | m.bias.data.zero_() 29 | elif isinstance(m, nn.Linear): 30 | torch.nn.init.constant_(m.weight.data, 0) 31 | m.bias.data.zero_() 32 | net = Net() 33 | 34 | # 保存，并加载模型参数(仅保存模型参数) 35 | torch.save(net.state_dict(), 'net_params.pkl') # 假设训练好了一个模型net 36 | pretrained_dict = torch.load('net_params.pkl') 37 | 38 | # 将net的参数全部置0，方便对比 39 | net.zero_param() 40 | net_state_dict = net.state_dict() 41 | print('conv1层的权值为:\n', net_state_dict['conv1.weight'], '\n') 42 | 43 | # 通过load_state_dict 加载参数 44 | net.load_state_dict(pretrained_dict) 45 | print('加载之后，conv1层的权值变为:\n', net_state_dict['conv1.weight']) 46 | -------------------------------------------------------------------------------- /src/PyTorch_Tutorial/Code/3_optimizer/3_2_optimizer/5_add_param_group.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | 3 | import torch 4 | import torch.optim as optim 5 | 6 | # ----------------------------------- add_param_group 7 | 8 | w1 = torch.randn(2, 2) 9 | w1.requires_grad = True 10 | 11 | w2 = torch.randn(2, 2) 12 | w2.requires_grad = True 13 | 14 | w3 = torch.randn(2, 2) 15 | w3.requires_grad = True 16 | 17 | # 一个参数组 18 | optimizer_1 = optim.SGD([w1, w2], lr=0.1) 19 | print('当前参数组个数: ', len(optimizer_1.param_groups)) 20 | print(optimizer_1.param_groups, '\n') 21 | 22 | # 增加一个参数组 23 | print('增加一组参数 w3\n') 24 | optimizer_1.add_param_group({'params': w3, 'lr': 0.001, 'momentum': 0.8}) 25 | 26 | print('当前参数组个数: ', len(optimizer_1.param_groups)) 27 | print(optimizer_1.param_groups, '\n') 28 | 29 | print('可以看到，参数组是一个list，一个元素是一个dict，每个dict中都有lr, momentum等参数，这些都是可单独管理，单独设定，十分灵活！') 30 | 31 | -------------------------------------------------------------------------------- /src/PyTorch_Tutorial/Code/3_optimizer/3_2_optimizer/net_params.pkl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/src/PyTorch_Tutorial/Code/3_optimizer/3_2_optimizer/net_params.pkl -------------------------------------------------------------------------------- /src/PyTorch_Tutorial/Code/4_viewer/1_tensorboardX_demo.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | import torch 3 | import torchvision.utils as vutils 4 | import numpy as np 5 | import torchvision.models as models 6 | from torchvision import datasets 7 | from tensorboardX import SummaryWriter 8 | 9 | resnet18 = models.resnet18(False) 10 | writer = SummaryWriter('../../Result/runs') 11 | sample_rate = 44100 12 | freqs = [262, 294, 330, 349, 392, 440, 440, 440, 440, 440, 440] 13 | 14 | true_positive_counts = [75, 64, 21, 5, 0] 15 | false_positive_counts = [150, 105, 18, 0, 0] 16 | true_negative_counts = [0, 45, 132, 150, 150] 17 | false_negative_counts = [0, 11, 54, 70, 75] 18 | precision = [0.3333333, 0.3786982, 0.5384616, 1.0, 0.0] 19 | recall = [1.0, 0.8533334, 0.28, 0.0666667, 0.0] 20 | 21 | for n_iter in range(100): 22 | s1 = torch.rand(1) # value to keep 23 | s2 = torch.rand(1) 24 | # add_scalar()：在一个图表中记录一个标量的变化，常用于 Loss 和 Accuracy 曲线的记录 25 | # data grouping by `slash` 26 | writer.add_scalar('data/scalar_systemtime', s1[0], n_iter) 27 | # data grouping by `slash` 28 | writer.add_scalar('data/scalar_customtime', s1[0], n_iter, walltime=n_iter) 29 | 30 | # add_scalars()：在一个图表中记录多个标量的变化，常用于对比，如 trainLoss 和 validLoss 的比较等。 31 | writer.add_scalars('data/scalar_group', {"xsinx": n_iter * np.sin(n_iter), 32 | "xcosx": n_iter * np.cos(n_iter), 33 | "arctanx": np.arctan(n_iter)}, n_iter) 34 | x = torch.rand(32, 3, 64, 64) # output from network 35 | if n_iter % 10 == 0: 36 | # torchvision.utils.make_grid()：将一组图片拼接成一张图片，便于可视化。 37 | x = vutils.make_grid(x, normalize=True, scale_each=True) 38 | # add_image()：绘制图片，可用于检查模型的输入，监测 feature map 的变化，或是观察 weight。 39 | writer.add_image('Image', x, n_iter) # Tensor 40 | # writer.add_image('astronaut', skimage.data.astronaut(), n_iter) # numpy 41 | # writer.add_image('imread', 42 | # skimage.io.imread('screenshots/audio.png'), n_iter) # numpy 43 | x = torch.zeros(sample_rate * 2) 44 | for i in range(x.size(0)): 45 | # sound amplitude should in [-1, 1] 46 | x[i] = np.cos(freqs[n_iter // 10] * np.pi * 47 | float(i) / float(sample_rate)) 48 | writer.add_audio('myAudio', x, n_iter) 49 | writer.add_text('Text', 'text logged at step:' + str(n_iter), n_iter) 50 | writer.add_text('markdown Text', '''a|b\n-|-\nc|d''', n_iter) 51 | 52 | # add_histogram()：绘制直方图和多分位数折线图，常用于监测权值及梯度的分布变化情况，便于诊断网络更新方向是否正确。 53 | for name, param in resnet18.named_parameters(): 54 | if 'bn' not in name: 55 | writer.add_histogram(name, param, n_iter) 56 | 57 | # add_pr_curve()：绘制 PR 曲线 58 | writer.add_pr_curve('xoxo', np.random.randint(2, size=100), np.random.rand( 59 | 100), n_iter) # needs tensorboard 0.4RC or later 60 | # add_pr_curve_raw()：从原始数据上绘制 PR 曲线 61 | writer.add_pr_curve_raw('prcurve with raw data', true_positive_counts, 62 | false_positive_counts, 63 | true_negative_counts, 64 | false_negative_counts, 65 | precision, 66 | recall, n_iter) 67 | 68 | # export_scalars_to_json()：将 scalars 信息保存到 json 文件，便于后期使用 69 | # export scalar data to JSON for external processing 70 | writer.export_scalars_to_json("../../Result/all_scalars.json") 71 | 72 | # add_embedding()：在三维空间或二维空间展示数据分布，可选 T-SNE、 PCA 和 CUSTOM 方法。 73 | # 展示 Mnist 中的 100 张图片的三维数据分布 74 | dataset = datasets.MNIST('../../Data/mnist', train=False, download=True) 75 | images = dataset.data[:100].float() 76 | label = dataset.targets[:100] 77 | features = images.view(100, 784) 78 | writer.add_embedding(features, metadata=label, label_img=images.unsqueeze(1)) 79 | 80 | writer.add_embedding(features, global_step=1, tag='noMetadata') 81 | dataset = datasets.MNIST('../../Data/mnist', train=True, download=True) 82 | images_train = dataset.data[:100].float() 83 | labels_train = dataset.targets[:100] 84 | features_train = images_train.view(100, 784) 85 | 86 | all_features = torch.cat((features, features_train)) 87 | all_labels = torch.cat((label, labels_train)) 88 | all_images = torch.cat((images, images_train)) 89 | dataset_label = ['test'] * 100 + ['train'] * 100 90 | all_labels = list(zip(all_labels, dataset_label)) 91 | 92 | writer.add_embedding(all_features, metadata=all_labels, label_img=all_images.unsqueeze(1), 93 | metadata_header=['digit', 'dataset'], global_step=2) 94 | 95 | #GRAPH 96 | # add_graph()：绘制网络结构拓扑图 97 | dummy_input = torch.rand(6, 3, 224, 224) 98 | writer.add_graph(resnet18, dummy_input) 99 | 100 | # VIDEO 101 | vid_images = dataset.data[:16 * 48] 102 | vid = vid_images.view(16, 1, 48, 28, 28) # BxCxTxHxW 103 | 104 | writer.add_video('video', vid_tensor=vid) 105 | writer.add_video('video_1_fps', vid_tensor=vid, fps=1) 106 | 107 | writer.close() 108 | -------------------------------------------------------------------------------- /src/PyTorch_Tutorial/Code/4_viewer/2_visual_weights.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | import torch 3 | import torchvision.utils as vutils 4 | from tensorboardX import SummaryWriter 5 | import torch.nn as nn 6 | import torch.nn.functional as F 7 | 8 | 9 | class Net(nn.Module): 10 | def __init__(self): 11 | super(Net, self).__init__() 12 | self.conv1 = nn.Conv2d(3, 6, 5) 13 | self.pool1 = nn.MaxPool2d(2, 2) 14 | self.conv2 = nn.Conv2d(6, 16, 5) 15 | self.pool2 = nn.MaxPool2d(2, 2) 16 | self.fc1 = nn.Linear(16 * 5 * 5, 120) 17 | self.fc2 = nn.Linear(120, 84) 18 | self.fc3 = nn.Linear(84, 10) 19 | 20 | def forward(self, x): 21 | x = self.pool1(F.relu(self.conv1(x))) 22 | x = self.pool2(F.relu(self.conv2(x))) 23 | x = x.view(-1, 16 * 5 * 5) 24 | x = F.relu(self.fc1(x)) 25 | x = F.relu(self.fc2(x)) 26 | x = self.fc3(x) 27 | return x 28 | 29 | # 定义权值初始化 30 | def initialize_weights(self): 31 | for m in self.modules(): 32 | if isinstance(m, nn.Conv2d): 33 | torch.nn.init.xavier_normal_(m.weight.data) 34 | if m.bias is not None: 35 | m.bias.data.zero_() 36 | elif isinstance(m, nn.BatchNorm2d): 37 | m.weight.data.fill_(1) 38 | m.bias.data.zero_() 39 | elif isinstance(m, nn.Linear): 40 | torch.nn.init.normal_(m.weight.data, 0, 0.01) 41 | m.bias.data.zero_() 42 | 43 | 44 | net = Net() # 创建一个网络 45 | pretrained_dict = torch.load('../2_model/net_params.pkl') 46 | net.load_state_dict(pretrained_dict) 47 | 48 | writer = SummaryWriter(logdir='../../Result/visual_weights') 49 | params = net.state_dict() 50 | for k, v in params.items(): 51 | if 'conv' in k and 'weight' in k: 52 | 53 | c_int = v.size()[1] # 输入层通道数 54 | c_out = v.size()[0] # 输出层通道数 55 | 56 | # 以feature map为单位，绘制一组卷积核，一张feature map对应的卷积核个数为输入通道数 57 | for j in range(c_out): 58 | print(k, v.size(), j) 59 | kernel_j = v[j, :, :, :].unsqueeze(1) # 压缩维度，为make_grid制作输入 60 | kernel_grid = vutils.make_grid(kernel_j, normalize=True, scale_each=True, nrow=c_int) # 1*输入通道数, w, h 61 | writer.add_image(k+'_split_in_channel', kernel_grid, global_step=j) # j 表示feature map数 62 | 63 | # 将一个卷积层的卷积核绘制在一起，每一行是一个feature map的卷积核 64 | k_w, k_h = v.size()[-1], v.size()[-2] 65 | kernel_all = v.view(-1, 1, k_w, k_h) 66 | kernel_grid = vutils.make_grid(kernel_all, normalize=True, scale_each=True, nrow=c_int) # 1*输入通道数, w, h 67 | writer.add_image(k + '_all', kernel_grid, global_step=666) 68 | writer.close() -------------------------------------------------------------------------------- /src/PyTorch_Tutorial/Code/4_viewer/3_visual_featuremaps.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | import torch 3 | import torchvision.utils as vutils 4 | import numpy as np 5 | from tensorboardX import SummaryWriter 6 | import torch.nn.functional as F 7 | import torchvision.transforms as transforms 8 | import sys 9 | from Code.utils.utils import MyDataset, Net, normalize_invert 10 | from torch.utils.data import DataLoader 11 | 12 | sys.path.append("..") 13 | 14 | vis_layer = 'conv1' 15 | log_dir = '../../Result/visual_featuremaps' 16 | txt_path = '../../Data/visual.txt' 17 | pretrained_path = '../../Data/net_params_72p.pkl' 18 | 19 | net = Net() 20 | pretrained_dict = torch.load(pretrained_path) 21 | net.load_state_dict(pretrained_dict) 22 | 23 | # 数据预处理 24 | normMean = [0.49139968, 0.48215827, 0.44653124] 25 | normStd = [0.24703233, 0.24348505, 0.26158768] 26 | normTransform = transforms.Normalize(normMean, normStd) 27 | testTransform = transforms.Compose([ 28 | transforms.Resize((32, 32)), 29 | transforms.ToTensor(), 30 | normTransform 31 | ]) 32 | # 载入数据 33 | test_data = MyDataset(txt_path=txt_path, transform=testTransform) 34 | test_loader = DataLoader(dataset=test_data, batch_size=1) 35 | img, label = iter(test_loader).next() 36 | 37 | x = img 38 | writer = SummaryWriter(logdir=log_dir) 39 | for name, layer in net._modules.items(): 40 | 41 | # 为fc层预处理x 42 | x = x.view(x.size(0), -1) if "fc" in name else x 43 | 44 | # 对x执行单层运算 45 | x = layer(x) 46 | print(x.size()) 47 | 48 | # 由于__init__()相较于forward()缺少relu操作，需要手动增加 49 | x = F.relu(x) if 'conv' in name else x 50 | 51 | # 依据选择的层，进行记录feature maps 52 | if name == vis_layer: 53 | # 绘制feature maps 54 | x1 = x.transpose(0, 1) # C，B, H, W ---> B，C, H, W 55 | img_grid = vutils.make_grid(x1, normalize=True, scale_each=True, nrow=2) # B，C, H, W 56 | writer.add_image(vis_layer + '_feature_maps', img_grid, global_step=666) 57 | 58 | # 绘制原始图像 59 | img_raw = normalize_invert(img, normMean, normStd) # 图像去标准化 60 | img_raw = np.array(img_raw * 255).clip(0, 255).squeeze().astype('uint8') 61 | writer.add_image('raw img', img_raw, global_step=666) # j 表示feature map数 62 | 63 | writer.close() 64 | -------------------------------------------------------------------------------- /src/PyTorch_Tutorial/Code/4_viewer/4_hist_grad_weight.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | 3 | import torch 4 | from torch.utils.data import DataLoader 5 | import torchvision.transforms as transforms 6 | import numpy as np 7 | import os 8 | from torch.autograd import Variable 9 | import torch.nn as nn 10 | import torch.optim as optim 11 | import sys 12 | sys.path.append("..") 13 | from Code.utils.utils import MyDataset, validate, show_confMat, Net 14 | from tensorboardX import SummaryWriter 15 | from datetime import datetime 16 | 17 | train_txt_path = '../../Data/train.txt' 18 | valid_txt_path = '../../Data/valid.txt' 19 | 20 | classes_name = ['plane', 'car', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck'] 21 | 22 | train_bs = 16 23 | valid_bs = 16 24 | lr_init = 0.001 25 | max_epoch = 100 26 | 27 | # log 28 | log_dir = '../../Result/hist_grad_weight' 29 | 30 | writer = SummaryWriter(logdir=log_dir) 31 | 32 | # ------------------------------------ step 1/4 : 加载数据------------------------------------------------- 33 | 34 | # 数据预处理设置 35 | normMean = [0.4948052, 0.48568845, 0.44682974] 36 | normStd = [0.24580306, 0.24236229, 0.2603115] 37 | normTransform = transforms.Normalize(normMean, normStd) 38 | trainTransform = transforms.Compose([ 39 | transforms.Resize(32), 40 | transforms.RandomCrop(32, padding=4), 41 | transforms.ToTensor(), 42 | normTransform 43 | ]) 44 | 45 | validTransform = transforms.Compose([ 46 | transforms.ToTensor(), 47 | normTransform 48 | ]) 49 | 50 | # 构建MyDataset实例 51 | train_data = MyDataset(txt_path=train_txt_path, transform=trainTransform) 52 | valid_data = MyDataset(txt_path=valid_txt_path, transform=validTransform) 53 | 54 | # 构建DataLoder 55 | train_loader = DataLoader(dataset=train_data, batch_size=train_bs, shuffle=True) 56 | valid_loader = DataLoader(dataset=valid_data, batch_size=valid_bs) 57 | 58 | # ------------------------------------ step 2/4 : 网络初始化---------------------------------------------- 59 | 60 | net = Net() # 创建一个网络 61 | net.initialize_weights() # 初始化权值 62 | 63 | # ------------------------------------ step 3/4 : 定义损失函数和优化器 ------------------------------------ 64 | 65 | criterion = nn.CrossEntropyLoss() # 选择损失函数 66 | optimizer = optim.SGD(net.parameters(), lr=lr_init, momentum=0.9, dampening=0.1) # 选择优化器 67 | scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size=50, gamma=0.1) # 设置学习率下降策略 68 | 69 | # ------------------------------------ step 4/4 : 训练 -------------------------------------------------- 70 | 71 | for epoch in range(max_epoch): 72 | 73 | loss_sigma = 0.0 # 记录一个epoch的loss之和 74 | correct = 0.0 75 | total = 0.0 76 | scheduler.step() # 更新学习率 77 | 78 | for i, data in enumerate(train_loader): 79 | # 获取图片和标签 80 | inputs, labels = data 81 | inputs, labels = Variable(inputs), Variable(labels) 82 | 83 | # forward, backward, update weights 84 | optimizer.zero_grad() 85 | outputs = net(inputs) 86 | loss = criterion(outputs, labels) 87 | loss.backward() 88 | optimizer.step() 89 | 90 | # 统计预测信息 91 | _, predicted = torch.max(outputs.data, 1) 92 | total += labels.size(0) 93 | correct += (predicted == labels).squeeze().sum().numpy() 94 | loss_sigma += loss.item() 95 | 96 | # 每10个iteration 打印一次训练信息，loss为10个iteration的平均 97 | if i % 10 == 9: 98 | loss_avg = loss_sigma / 10 99 | loss_sigma = 0.0 100 | print("Training: Epoch[{:0>3}/{:0>3}] Iteration[{:0>3}/{:0>3}] Loss: {:.4f} Acc:{:.2%}".format( 101 | epoch + 1, max_epoch, i + 1, len(train_loader), loss_avg, correct / total)) 102 | 103 | # 每个epoch，记录梯度，权值 104 | for name, layer in net.named_parameters(): 105 | writer.add_histogram(name + '_grad', layer.grad.cpu().data.numpy(), epoch) 106 | writer.add_histogram(name + '_data', layer.cpu().data.numpy(), epoch) 107 | 108 | print('Finished Training') 109 | -------------------------------------------------------------------------------- /src/PyTorch_Tutorial/Code/4_viewer/5_Show_ConfMat.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | import numpy as np 3 | import os 4 | import matplotlib.pyplot as plt 5 | 6 | 7 | def show_confMat(confusion_mat, classes_name, set_name, out_dir): 8 | """ 9 | 可视化混淆矩阵，保存png格式 10 | :param confusion_mat: nd-array 11 | :param classes_name: list,各类别名称 12 | :param set_name: str, eg: 'valid', 'train' 13 | :param out_dir: str, png输出的文件夹 14 | :return: 15 | """ 16 | # 归一化 17 | confusion_mat_N = confusion_mat.copy() 18 | for i in range(len(classes_name)): 19 | confusion_mat_N[i, :] = confusion_mat[i, :] / confusion_mat[i, :].sum() 20 | 21 | # 获取颜色 22 | cmap = plt.cm.get_cmap('Greys') # 更多颜色: http://matplotlib.org/examples/color/colormaps_reference.html 23 | plt.imshow(confusion_mat_N, cmap=cmap) 24 | plt.colorbar() 25 | 26 | # 设置文字 27 | xlocations = np.array(range(len(classes_name))) 28 | plt.xticks(xlocations, classes_name, rotation=60) 29 | plt.yticks(xlocations, classes_name) 30 | plt.xlabel('Predict label') 31 | plt.ylabel('True label') 32 | plt.title('Confusion_Matrix_' + set_name) 33 | 34 | # 打印数字 35 | for i in range(confusion_mat_N.shape[0]): 36 | for j in range(confusion_mat_N.shape[1]): 37 | plt.text(x=j, y=i, s=int(confusion_mat[i, j]), va='center', ha='center', color='red', fontsize=10) 38 | # 保存 39 | plt.savefig(os.path.join(out_dir, 'Confusion_Matrix_' + set_name + '.png')) 40 | plt.close() 41 | 42 | 43 | if __name__ == '__main__': 44 | print('QQ group: {}, password: {}'.format(671103375, 2018)) 45 | -------------------------------------------------------------------------------- /src/PyTorch_Tutorial/Code/main_training/main.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | 3 | import torch 4 | from torch.utils.data import DataLoader 5 | import torchvision.transforms as transforms 6 | import numpy as np 7 | import os 8 | from torch.autograd import Variable 9 | import torch.nn as nn 10 | import torch.nn.functional as F 11 | import torch.optim as optim 12 | import sys 13 | 14 | sys.path.append("..") 15 | from Code.utils.utils import MyDataset, validate, show_confMat 16 | from tensorboardX import SummaryWriter 17 | from datetime import datetime 18 | 19 | train_txt_path = '../../Data/train.txt' 20 | valid_txt_path = '../../Data/valid.txt' 21 | 22 | classes_name = ['plane', 'car', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck'] 23 | 24 | train_bs = 16 25 | valid_bs = 16 26 | lr_init = 0.001 27 | max_epoch = 1 28 | 29 | # log 30 | result_dir = '../../Result/' 31 | 32 | now_time = datetime.now() 33 | time_str = datetime.strftime(now_time, '%m-%d_%H-%M-%S') 34 | 35 | log_dir = os.path.join(result_dir, time_str) 36 | if not os.path.exists(log_dir): 37 | os.makedirs(log_dir) 38 | 39 | writer = SummaryWriter(logdir=log_dir) 40 | 41 | # ------------------------------------ step 1/5 : 加载数据------------------------------------ 42 | 43 | # 数据预处理设置 44 | # 前三行设置均值，标准差，以及数据标准化 45 | normMean = [0.4948052, 0.48568845, 0.44682974] 46 | normStd = [0.24580306, 0.24236229, 0.2603115] 47 | normTransform = transforms.Normalize(normMean, normStd) 48 | # 用 transforms.Compose 将所需要进行的处理给 compose 起来 49 | trainTransform = transforms.Compose([ 50 | transforms.Resize(32), 51 | # 随机裁剪 52 | # 在裁剪之前先对图片的上下左右均填充上 4 个 pixel，值为0，即变成一个 40*40 的数据，然后再随机进行 32*32 的裁剪。 53 | transforms.RandomCrop(32, padding=4), 54 | # Totensor 55 | transforms.ToTensor(), 56 | # 数据标准化(减均值，除以标准差) 57 | normTransform 58 | ]) 59 | 60 | validTransform = transforms.Compose([ 61 | transforms.ToTensor(), 62 | normTransform 63 | ]) 64 | 65 | # 构建MyDataset实例 66 | # 从 MyDataset 类中初始化 txt， txt 中有图片路径和标签 67 | train_data = MyDataset(txt_path=train_txt_path, transform=trainTransform) 68 | valid_data = MyDataset(txt_path=valid_txt_path, transform=validTransform) 69 | 70 | # 构建DataLoder 71 | # 初始化 DataLoder 时，将 train_data 传入，从而使 DataLoder 拥有图片的路径 72 | train_loader = DataLoader(dataset=train_data, batch_size=train_bs, shuffle=True) 73 | valid_loader = DataLoader(dataset=valid_data, batch_size=valid_bs) 74 | 75 | 76 | # ------------------------------------ step 2/5 : 定义网络------------------------------------ 77 | 78 | # 1.必须继承 nn.Module 这个类，要让 PyTorch 知道这个类是一个 Module 79 | class Net(nn.Module): 80 | # 2.在__init__(self)中设置好需要的“组件" 81 | def __init__(self): 82 | super(Net, self).__init__() 83 | 84 | self.conv1 = nn.Conv2d(3, 6, 5) 85 | self.pool1 = nn.MaxPool2d(2, 2) 86 | self.conv2 = nn.Conv2d(6, 16, 5) 87 | self.pool2 = nn.MaxPool2d(2, 2) 88 | self.fc1 = nn.Linear(16 * 5 * 5, 120) 89 | self.fc2 = nn.Linear(120, 84) 90 | self.fc3 = nn.Linear(84, 10) 91 | 92 | # 3.在 forward(self, x)中用定义好的“组件”进行组装 93 | def forward(self, x): 94 | # 第1-2行:x 经过 conv1，然后经过激活函数 relu，再经过 pool1 操作 95 | x = self.pool1(F.relu(self.conv1(x))) 96 | x = self.pool2(F.relu(self.conv2(x))) 97 | # 第3行:表示将 x 进行 reshape，为了后面做为全连接层的输入 98 | x = x.view(-1, 16 * 5 * 5) 99 | # 第4-5行:先经过全连接层 fc，然后经过 relu； 100 | x = F.relu(self.fc1(x)) 101 | x = F.relu(self.fc2(x)) 102 | # 第6行:模型的最终输出是 fc3 输出 103 | x = self.fc3(x) 104 | return x 105 | 106 | # 定义权值初始化 107 | def initialize_weights(self): 108 | for m in self.modules(): 109 | # 判断各层属于什么类型，然后根据不同类型的层，设定不同的权值初始化方法 110 | if isinstance(m, nn.Conv2d): 111 | torch.nn.init.xavier_normal_(m.weight.data) 112 | if m.bias is not None: 113 | m.bias.data.zero_() 114 | elif isinstance(m, nn.BatchNorm2d): 115 | m.weight.data.fill_(1) 116 | m.bias.data.zero_() 117 | elif isinstance(m, nn.Linear): 118 | torch.nn.init.normal_(m.weight.data, 0, 0.01) 119 | m.bias.data.zero_() 120 | 121 | 122 | net = Net() # 创建一个网络 123 | net.initialize_weights() # 初始化权值 124 | 125 | # ------------------------------------ step 3/5 : 定义损失函数和优化器 ------------------------------------ 126 | 127 | criterion = nn.CrossEntropyLoss() # 选择损失函数 128 | optimizer = optim.SGD(net.parameters(), lr=lr_init, momentum=0.9, dampening=0.1) # 选择优化器 129 | scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size=50, gamma=0.1) # 设置学习率下降策略 130 | 131 | # ------------------------------------ step 4/5 : 训练 -------------------------------------------------- 132 | 133 | for epoch in range(max_epoch): 134 | 135 | loss_sigma = 0.0 # 记录一个epoch的loss之和 136 | correct = 0.0 137 | total = 0.0 138 | scheduler.step() # 更新学习率 139 | 140 | # 在一个iteration进行时，才读取一个batch的图片数据enumerate()函数会返回可迭代数据的一个“元素” 141 | # 在这里data是一个batch的图片数据和标签， data是一个list 142 | for i, data in enumerate(train_loader): 143 | # if i == 30 : break 144 | # 获取图片和标签 145 | inputs, labels = data 146 | # 将图片数据转换成 Variable 类型，然后称为模型真正的输入 147 | inputs, labels = Variable(inputs), Variable(labels) 148 | 149 | # forward, backward, update weights 150 | optimizer.zero_grad() 151 | outputs = net(inputs) 152 | loss = criterion(outputs, labels) 153 | loss.backward() 154 | optimizer.step() 155 | 156 | # 统计预测信息 157 | _, predicted = torch.max(outputs.data, 1) 158 | total += labels.size(0) 159 | correct += (predicted == labels).squeeze().sum().numpy() 160 | loss_sigma += loss.item() 161 | 162 | # 每10个iteration 打印一次训练信息，loss为10个iteration的平均 163 | if i % 10 == 9: 164 | loss_avg = loss_sigma / 10 165 | loss_sigma = 0.0 166 | print("Training: Epoch[{:0>3}/{:0>3}] Iteration[{:0>3}/{:0>3}] Loss: {:.4f} Acc:{:.2%}".format( 167 | epoch + 1, max_epoch, i + 1, len(train_loader), loss_avg, correct / total)) 168 | 169 | # 记录训练loss 170 | writer.add_scalars('Loss_group', {'train_loss': loss_avg}, epoch) 171 | # 记录learning rate 172 | writer.add_scalar('learning rate', scheduler.get_lr()[0], epoch) 173 | # 记录Accuracy 174 | writer.add_scalars('Accuracy_group', {'train_acc': correct / total}, epoch) 175 | 176 | # 每个epoch，记录梯度，权值 177 | for name, layer in net.named_parameters(): 178 | writer.add_histogram(name + '_grad', layer.grad.cpu().data.numpy(), epoch) 179 | writer.add_histogram(name + '_data', layer.cpu().data.numpy(), epoch) 180 | 181 | # ------------------------------------ 观察模型在验证集上的表现 ------------------------------------ 182 | if epoch % 2 == 0: 183 | loss_sigma = 0.0 184 | cls_num = len(classes_name) 185 | conf_mat = np.zeros([cls_num, cls_num]) # 混淆矩阵 186 | net.eval() 187 | for i, data in enumerate(valid_loader): 188 | 189 | # 获取图片和标签 190 | images, labels = data 191 | images, labels = Variable(images), Variable(labels) 192 | 193 | # forward 194 | outputs = net(images) 195 | outputs.detach_() 196 | 197 | # 计算loss 198 | loss = criterion(outputs, labels) 199 | loss_sigma += loss.item() 200 | 201 | # 统计 202 | _, predicted = torch.max(outputs.data, 1) 203 | # labels = labels.data # Variable --> tensor 204 | 205 | # 统计混淆矩阵 206 | for j in range(len(labels)): 207 | cate_i = labels[j].numpy() 208 | pre_i = predicted[j].numpy() 209 | conf_mat[cate_i, pre_i] += 1.0 210 | 211 | print('{} set Accuracy:{:.2%}'.format('Valid', conf_mat.trace() / conf_mat.sum())) 212 | # 记录Loss, accuracy 213 | writer.add_scalars('Loss_group', {'valid_loss': loss_sigma / len(valid_loader)}, epoch) 214 | writer.add_scalars('Accuracy_group', {'valid_acc': conf_mat.trace() / conf_mat.sum()}, epoch) 215 | print('Finished Training') 216 | 217 | # ------------------------------------ step5: 保存模型 并且绘制混淆矩阵图 ------------------------------------ 218 | net_save_path = os.path.join(log_dir, 'net_params.pkl') 219 | # 保存模型 220 | torch.save(net.state_dict(), net_save_path) 221 | 222 | conf_mat_train, train_acc = validate(net, train_loader, 'train', classes_name) 223 | conf_mat_valid, valid_acc = validate(net, valid_loader, 'valid', classes_name) 224 | 225 | show_confMat(conf_mat_train, classes_name, 'train', log_dir) 226 | show_confMat(conf_mat_valid, classes_name, 'valid', log_dir) 227 | -------------------------------------------------------------------------------- /src/PyTorch_Tutorial/Code/main_training/main_bak.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | 3 | import torch 4 | from torch.utils.data import DataLoader 5 | import torchvision.transforms as transforms 6 | import numpy as np 7 | import os 8 | from torch.autograd import Variable 9 | import torch.nn as nn 10 | import torch.nn.functional as F 11 | import torch.optim as optim 12 | import sys 13 | from Code.utils.utils import MyDataset, validate, show_confMat 14 | from tensorboardX import SummaryWriter 15 | from datetime import datetime 16 | 17 | sys.path.append("..") 18 | # ------------------------------------ step 0/5 : 设置基本参数------------------------------------------------------ 19 | train_txt_path = '../../Data/train.txt' 20 | valid_txt_path = '../../Data/valid.txt' 21 | 22 | classes_name = ['plane', 'car', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck'] 23 | 24 | train_bs = 16 # batch_size 25 | valid_bs = 16 26 | lr_init = 0.001 27 | max_epoch = 100 28 | 29 | # log 保存日志 30 | result_dir = '../../Result/' 31 | 32 | now_time = datetime.now() 33 | time_str = datetime.strftime(now_time, '%m-%d_%H-%M-%S') 34 | 35 | log_dir = os.path.join(result_dir, time_str) 36 | if not os.path.exists(log_dir): 37 | os.makedirs(log_dir) 38 | 39 | writer = SummaryWriter(log_dir=log_dir) 40 | 41 | # ------------------------------------ step 1/5 : 加载数据------------------------------------ 42 | 43 | # 数据预处理设置 44 | normMean = [0.4948052, 0.48568845, 0.44682974] 45 | normStd = [0.24580306, 0.24236229, 0.2603115] 46 | normTransform = transforms.Normalize(normMean, normStd) 47 | trainTransform = transforms.Compose([ 48 | transforms.Resize(32), 49 | transforms.RandomCrop(32, padding=4), 50 | transforms.ToTensor(), 51 | normTransform 52 | ]) 53 | 54 | validTransform = transforms.Compose([ 55 | transforms.ToTensor(), 56 | normTransform 57 | ]) 58 | 59 | # 构建MyDataset实例 60 | train_data = MyDataset(txt_path=train_txt_path, transform=trainTransform) 61 | valid_data = MyDataset(txt_path=valid_txt_path, transform=validTransform) 62 | 63 | # 构建DataLoder 64 | train_loader = DataLoader(dataset=train_data, batch_size=train_bs, shuffle=True) 65 | valid_loader = DataLoader(dataset=valid_data, batch_size=valid_bs) 66 | 67 | 68 | # ------------------------------------ step 2/5 : 定义网络------------------------------------ 69 | 70 | 71 | class Net(nn.Module): 72 | def __init__(self): 73 | super(Net, self).__init__() 74 | self.conv1 = nn.Conv2d(3, 6, 5) 75 | self.pool1 = nn.MaxPool2d(2, 2) 76 | self.conv2 = nn.Conv2d(6, 16, 3) 77 | self.pool2 = nn.MaxPool2d(2, 2) 78 | self.conv3 = nn.Conv2d(16, 64, 3) 79 | self.pool3 = nn.MaxPool2d(2, 1) 80 | self.fc1 = nn.Linear(64 * 3 * 3, 120) 81 | self.fc2 = nn.Linear(120, 84) 82 | self.fc3 = nn.Linear(84, 10) 83 | 84 | def forward(self, x): 85 | x = self.pool1(F.relu(self.conv1(x))) 86 | x = self.pool2(F.relu(self.conv2(x))) 87 | x = self.pool3(F.relu(self.conv3(x))) 88 | x = x.view(-1, 64 * 3 * 3) 89 | x = F.relu(self.fc1(x)) 90 | x = F.relu(self.fc2(x)) 91 | x = self.fc3(x) 92 | return x 93 | 94 | # 定义权值初始化 95 | def initialize_weights(self): 96 | for m in self.modules(): 97 | if isinstance(m, nn.Conv2d): 98 | torch.nn.init.xavier_normal_(m.weight.data) 99 | if m.bias is not None: 100 | m.bias.data.zero_() 101 | elif isinstance(m, nn.BatchNorm2d): 102 | m.weight.data.fill_(1) 103 | m.bias.data.zero_() 104 | elif isinstance(m, nn.Linear): 105 | torch.nn.init.normal_(m.weight.data, 0, 0.01) 106 | m.bias.data.zero_() 107 | 108 | 109 | net = Net() # 创建一个网络 110 | net.initialize_weights() # 初始化权值 111 | 112 | # ------------------------------------ step 3/5 : 定义损失函数和优化器 ------------------------------------ 113 | 114 | criterion = nn.CrossEntropyLoss() # 选择损失函数 115 | # optimizer = optim.SGD(net.parameters(), lr=lr_init, momentum=0.9, dampening=0.1) # 选择优化器 116 | optimizer = optim.Adam(net.parameters(), lr=lr_init) 117 | # scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size=50, gamma=0.1) # 设置学习率下降策略 118 | scheduler = torch.optim.lr_scheduler.CosineAnnealingLr(optimizer, T_max=50) 119 | 120 | # ------------------------------------ step 4/5 : 训练 -------------------------------------------------- 121 | 122 | for epoch in range(max_epoch): 123 | 124 | loss_sigma = 0.0 # 记录一个epoch的loss之和 125 | correct = 0.0 126 | total = 0.0 127 | scheduler.step() # 更新学习率 128 | 129 | for i, data in enumerate(train_loader): 130 | # if i == 30 : break 131 | # 获取图片和标签 132 | inputs, labels = data 133 | inputs, labels = Variable(inputs), Variable(labels) 134 | 135 | # forward, backward, update weights 136 | optimizer.zero_grad() 137 | outputs = net(inputs) 138 | loss = criterion(outputs, labels) 139 | loss.backward() 140 | optimizer.step() 141 | 142 | # 统计预测信息 143 | _, predicted = torch.max(outputs.data, dim=1) 144 | total += labels.size(0) 145 | correct += (predicted == labels).squeeze().sum().numpy() 146 | loss_sigma += loss.item() 147 | 148 | # 每10个iteration 打印一次训练信息，loss为10个iteration的平均 149 | if i % 10 == 9: 150 | loss_avg = loss_sigma / 10 151 | loss_sigma = 0.0 152 | print("Training: Epoch[{:0>3}/{:0>3}] Iteration[{:0>3}/{:0>3}] Loss: {:.4f} Acc:{:.2%}".format( 153 | epoch + 1, max_epoch, i + 1, len(train_loader), loss_avg, correct / total)) 154 | 155 | # 记录训练loss 156 | writer.add_scalars('Loss_group', {'train_loss': loss_avg}, epoch) 157 | # 记录learning rate 158 | writer.add_scalar('learning rate', scheduler.get_lr()[0], epoch) 159 | # 记录Accuracy 160 | writer.add_scalars('Accuracy_group', {'train_acc': correct / total}, epoch) 161 | 162 | # 每个epoch，记录梯度，权值 163 | for name, layer in net.named_parameters(): 164 | writer.add_histogram(name + '_grad', layer.grad.cpu().data.numpy(), epoch) 165 | writer.add_histogram(name + '_data', layer.cpu().data.numpy(), epoch) 166 | 167 | # ------------------------------------ 观察模型在验证集上的表现 ------------------------------------ 168 | if epoch % 2 == 0: 169 | loss_sigma = 0.0 170 | cls_num = len(classes_name) 171 | conf_mat = np.zeros([cls_num, cls_num]) # 混淆矩阵 172 | net.eval() 173 | for i, data in enumerate(valid_loader): 174 | 175 | # 获取图片和标签 176 | images, labels = data 177 | images, labels = Variable(images), Variable(labels) 178 | 179 | # forward 180 | outputs = net(images) 181 | outputs.detach_() 182 | 183 | # 计算loss 184 | loss = criterion(outputs, labels) 185 | loss_sigma += loss.item() 186 | 187 | # 统计 188 | _, predicted = torch.max(outputs.data, 1) 189 | # labels = labels.data # Variable --> tensor 190 | 191 | # 统计混淆矩阵 192 | for j in range(len(labels)): 193 | cate_i = labels[j].numpy() 194 | pre_i = predicted[j].numpy() 195 | conf_mat[cate_i, pre_i] += 1.0 196 | 197 | print('{} set Accuracy:{:.2%}'.format('Valid', conf_mat.trace() / conf_mat.sum())) 198 | # 记录Loss, accuracy 199 | writer.add_scalars('Loss_group', {'valid_loss': loss_sigma / len(valid_loader)}, epoch) 200 | writer.add_scalars('Accuracy_group', {'valid_acc': conf_mat.trace() / conf_mat.sum()}, epoch) 201 | print('Finished Training') 202 | 203 | # ------------------------------------ step5: 保存模型 并且绘制混淆矩阵图 ------------------------------------ 204 | net_save_path = os.path.join(log_dir, 'net_params.pkl') 205 | torch.save(net.state_dict(), net_save_path) 206 | 207 | conf_mat_train, train_acc = validate(net, train_loader, 'train', classes_name) 208 | conf_mat_valid, valid_acc = validate(net, valid_loader, 'valid', classes_name) 209 | 210 | show_confMat(conf_mat_train, classes_name, 'train', log_dir) 211 | show_confMat(conf_mat_valid, classes_name, 'valid', log_dir) 212 | -------------------------------------------------------------------------------- /src/PyTorch_Tutorial/Code/utils/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/src/PyTorch_Tutorial/Code/utils/__init__.py -------------------------------------------------------------------------------- /src/PyTorch_Tutorial/Code/utils/utils.py: -------------------------------------------------------------------------------- 1 | # coding: utf-8 2 | from PIL import Image 3 | from torch.utils.data import Dataset 4 | import numpy as np 5 | import torch 6 | from torch.autograd import Variable 7 | import os 8 | import matplotlib.pyplot as plt 9 | import torch.nn as nn 10 | import torch.nn.functional as F 11 | 12 | 13 | class Net(nn.Module): 14 | def __init__(self): 15 | super(Net, self).__init__() 16 | self.conv1 = nn.Conv2d(3, 6, 5) 17 | self.pool1 = nn.MaxPool2d(2, 2) 18 | self.conv2 = nn.Conv2d(6, 16, 5) 19 | 20 | self.pool2 = nn.MaxPool2d(2, 2) 21 | self.fc1 = nn.Linear(16*5*5, 120) 22 | self.fc2 = nn.Linear(120, 84) 23 | self.fc3 = nn.Linear(84, 10) 24 | 25 | def forward(self, x): 26 | x = self.pool1(F.relu(self.conv1(x))) 27 | x = self.pool2(F.relu(self.conv2(x))) 28 | x = x.view(-1, 16* 5* 5) 29 | x = F.relu(self.fc1(x)) 30 | x = F.relu(self.fc2(x)) 31 | x = self.fc3(x) 32 | return x 33 | 34 | # 定义权值初始化 35 | def initialize_weights(self): 36 | for m in self.modules(): 37 | if isinstance(m, nn.Conv2d): 38 | torch.nn.init.xavier_normal_(m.weight.data) 39 | if m.bias is not None: 40 | m.bias.data.zero_() 41 | elif isinstance(m, nn.BatchNorm2d): 42 | m.weight.data.fill_(1) 43 | m.bias.data.zero_() 44 | elif isinstance(m, nn.Linear): 45 | torch.nn.init.normal_(m.weight.data, 0, 0.01) 46 | m.bias.data.zero_() 47 | 48 | class MyDataset(Dataset): 49 | def __init__(self, txt_path, transform = None, target_transform = None): 50 | fh = open(txt_path, 'r') 51 | imgs = [] 52 | for line in fh: 53 | line = line.rstrip() 54 | words = line.split() 55 | imgs.append((words[0], int(words[1]))) 56 | 57 | self.imgs = imgs # 最主要就是要生成这个list， 然后DataLoader中给index，通过getitem读取图片数据 58 | self.transform = transform 59 | self.target_transform = target_transform 60 | 61 | def __getitem__(self, index): 62 | fn, label = self.imgs[index] 63 | img = Image.open(fn).convert('RGB') # 像素值 0~255，在transfrom.totensor会除以255，使像素值变成 0~1 64 | 65 | if self.transform is not None: 66 | img = self.transform(img) # 在这里做transform，转为tensor等等 67 | 68 | return img, label 69 | 70 | def __len__(self): 71 | return len(self.imgs) 72 | 73 | 74 | def validate(net, data_loader, set_name, classes_name): 75 | """ 76 | 对一批数据进行预测，返回混淆矩阵以及Accuracy 77 | :param net: 78 | :param data_loader: 79 | :param set_name: eg: 'valid' 'train' 'tesst 80 | :param classes_name: 81 | :return: 82 | """ 83 | net.eval() 84 | cls_num = len(classes_name) 85 | conf_mat = np.zeros([cls_num, cls_num]) 86 | 87 | for data in data_loader: 88 | images, labels = data 89 | images = Variable(images) 90 | labels = Variable(labels) 91 | 92 | outputs = net(images) 93 | outputs.detach_() 94 | 95 | _, predicted = torch.max(outputs.data, 1) 96 | 97 | # 统计混淆矩阵 98 | for i in range(len(labels)): 99 | cate_i = labels[i].numpy() 100 | pre_i = predicted[i].numpy() 101 | conf_mat[cate_i, pre_i] += 1.0 102 | 103 | for i in range(cls_num): 104 | print('class:{:<10}, total num:{:<6}, correct num:{:<5} Recall: {:.2%} Precision: {:.2%}'.format( 105 | classes_name[i], np.sum(conf_mat[i, :]), conf_mat[i, i], conf_mat[i, i] / (1 + np.sum(conf_mat[i, :])), 106 | conf_mat[i, i] / (1 + np.sum(conf_mat[:, i])))) 107 | 108 | print('{} set Accuracy:{:.2%}'.format(set_name, np.trace(conf_mat) / np.sum(conf_mat))) 109 | 110 | return conf_mat, '{:.2}'.format(np.trace(conf_mat) / np.sum(conf_mat)) 111 | 112 | 113 | def show_confMat(confusion_mat, classes, set_name, out_dir): 114 | 115 | # 归一化 116 | confusion_mat_N = confusion_mat.copy() 117 | for i in range(len(classes)): 118 | confusion_mat_N[i, :] = confusion_mat[i, :] / confusion_mat[i, :].sum() 119 | 120 | # 获取颜色 121 | cmap = plt.cm.get_cmap('Greys') # 更多颜色: http://matplotlib.org/examples/color/colormaps_reference.html 122 | plt.imshow(confusion_mat_N, cmap=cmap) 123 | plt.colorbar() 124 | 125 | # 设置文字 126 | xlocations = np.array(range(len(classes))) 127 | plt.xticks(xlocations, list(classes), rotation=60) 128 | plt.yticks(xlocations, list(classes)) 129 | plt.xlabel('Predict label') 130 | plt.ylabel('True label') 131 | plt.title('Confusion_Matrix_' + set_name) 132 | 133 | # 打印数字 134 | for i in range(confusion_mat_N.shape[0]): 135 | for j in range(confusion_mat_N.shape[1]): 136 | plt.text(x=j, y=i, s=int(confusion_mat[i, j]), va='center', ha='center', color='red', fontsize=10) 137 | # 保存 138 | plt.savefig(os.path.join(out_dir, 'Confusion_Matrix' + set_name + '.png')) 139 | plt.close() 140 | 141 | 142 | def normalize_invert(tensor, mean, std): 143 | for t, m, s in zip(tensor, mean, std): 144 | t.mul_(s).add_(m) 145 | return tensor 146 | -------------------------------------------------------------------------------- /src/PyTorch_Tutorial/Data/cat.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/src/PyTorch_Tutorial/Data/cat.png -------------------------------------------------------------------------------- /src/PyTorch_Tutorial/Data/cover.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/src/PyTorch_Tutorial/Data/cover.png -------------------------------------------------------------------------------- /src/PyTorch_Tutorial/Data/net_params_72p.pkl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/src/PyTorch_Tutorial/Data/net_params_72p.pkl -------------------------------------------------------------------------------- /src/PyTorch_Tutorial/Data/visual.txt: -------------------------------------------------------------------------------- 1 | ../../Data/cat.png 3 -------------------------------------------------------------------------------- /src/PyTorch_Tutorial/PyTorch_tutorial_0.0.5_余霆嵩.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/src/PyTorch_Tutorial/PyTorch_tutorial_0.0.5_余霆嵩.pdf -------------------------------------------------------------------------------- /src/PyTorch_Tutorial/requirements.txt: -------------------------------------------------------------------------------- 1 | absl-py==0.7.1 2 | astor==0.7.1 3 | backcall==0.1.0 4 | bleach==3.0.2 5 | certifi==2019.3.9 6 | chardet==3.0.4 7 | colorama==0.4.1 8 | cycler==0.10.0 9 | Cython==0.29.11 10 | decorator==4.3.0 11 | defusedxml==0.5.0 12 | entrypoints==0.2.3 13 | future==0.17.1 14 | gast==0.2.0 15 | google-pasta==0.1.7 16 | graphviz==0.10.1 17 | grpcio==1.16.0 18 | h5py==2.8.0 19 | idna==2.8 20 | imageio==2.5.0 21 | imageio-ffmpeg==0.3.0 22 | ipykernel==5.1.1 23 | ipython==7.2.0 24 | ipython-genutils==0.2.0 25 | ipywidgets==7.4.2 26 | itchat==1.2.32 27 | jedi==0.13.2 28 | jieba==0.39 29 | Jinja2==2.10.1 30 | jsonschema==2.6.0 31 | jupyter==1.0.0 32 | jupyter-client==5.2.4 33 | jupyter-console==6.0.0 34 | jupyter-contrib-core==0.3.3 35 | jupyter-core==4.4.0 36 | jupyter-nbextensions-configurator==0.4.1 37 | kafka-python==1.4.5 38 | Keras-Applications==1.0.6 39 | Keras-Preprocessing==1.0.5 40 | kiwisolver==1.0.1 41 | lightgbm==2.2.3 42 | Markdown==3.0.1 43 | MarkupSafe==1.1.0 44 | matplotlib==2.2.3 45 | mistune==0.8.4 46 | moviepy==1.0.0 47 | nbconvert==5.4.0 48 | nbextensions==0.1.0 49 | nbformat==4.4.0 50 | notebook==5.7.8 51 | numpy==1.16.3 52 | opencv-python==3.4.3.18 53 | pandas==0.23.4 54 | pandocfilters==1.4.2 55 | parso==0.3.1 56 | pickleshare==0.7.5 57 | Pillow==5.2.0 58 | proglog==0.1.9 59 | prometheus-client==0.5.0 60 | prompt-toolkit==2.0.7 61 | protobuf==3.8.0 62 | py4j==0.10.7 63 | pycocotools==2.0 64 | Pygments==2.3.1 65 | PyHDFS==0.2.1 66 | pyparsing==2.2.0 67 | pypng==0.0.19 68 | PyQRCode==1.2.1 69 | pyspark==2.3.3 70 | python-dateutil==2.7.3 71 | python-votesmart==0.3.3 72 | pytz==2018.5 73 | pywinpty==0.5.5 74 | PyYAML==5.1.1 75 | pyzmq==17.1.2 76 | qtconsole==4.4.3 77 | requests==2.21.0 78 | scikit-learn==0.20.2 79 | scipy==1.2.0 80 | seaborn==0.9.0 81 | Send2Trash==1.5.0 82 | simplejson==3.16.0 83 | six==1.11.0 84 | sklearn==0.0 85 | tensorboard==1.14.0 86 | tensorboardX==1.7 87 | tensorflow==1.14.0 88 | tensorflow-estimator==1.14.0 89 | termcolor==1.1.0 90 | terminado==0.8.1 91 | testpath==0.4.2 92 | torch==1.1.0 93 | torchvision==0.3.0 94 | tornado==5.1.1 95 | tqdm==4.32.2 96 | traitlets==4.3.2 97 | urllib3==1.25.3 98 | wcwidth==0.1.7 99 | webencodings==0.5.1 100 | Werkzeug==0.15.4 101 | widgetsnbextension==3.4.2 102 | wrapt==1.11.2 103 | wxpy==0.3.9.8 104 | xgboost==0.81 105 | -------------------------------------------------------------------------------- /src/Week1/Day1/0.开发环境准备.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/src/Week1/Day1/0.开发环境准备.pdf -------------------------------------------------------------------------------- /src/Week1/Day1/1.自动求导机制.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# 自动求导机制\n", 8 | "本说明将概述Autograd如何工作并记录操作。了解这些并不是绝对必要的，但我们建议您熟悉它，因为它将帮助您编写更高效，更简洁的程序，并可帮助您进行调试。 \n", 9 | "## 从后向中排除子图\n", 10 | "每个变量都有两个标志： **requires_grad 和 volatile** 。它们都允许从梯度计算中精细地排除子图，并可以提高效率。 \n", 11 | "\n", 12 | "### requires_grad\n", 13 | "如果有一个单一的输入操作需要梯度，它的输出也需要梯度。相反，只有所有输入都不需要梯度，输出才不需要。如果其中所有的变量都不需要梯度进行，后向计算不会在子图中执行。" 14 | ] 15 | }, 16 | { 17 | "cell_type": "code", 18 | "execution_count": 1, 19 | "metadata": {}, 20 | "outputs": [ 21 | { 22 | "data": { 23 | "text/plain": [ 24 | "False" 25 | ] 26 | }, 27 | "execution_count": 1, 28 | "metadata": {}, 29 | "output_type": "execute_result" 30 | } 31 | ], 32 | "source": [ 33 | "import torch\n", 34 | "from torch import nn\n", 35 | "from torch.autograd import Variable\n", 36 | "\n", 37 | "x = Variable(torch.randn(5, 5))\n", 38 | "y = Variable(torch.randn(5, 5))\n", 39 | "z = Variable(torch.randn(5, 5), requires_grad=True) \n", 40 | "a = x + y \n", 41 | "a.requires_grad " 42 | ] 43 | }, 44 | { 45 | "cell_type": "code", 46 | "execution_count": 2, 47 | "metadata": {}, 48 | "outputs": [ 49 | { 50 | "data": { 51 | "text/plain": [ 52 | "True" 53 | ] 54 | }, 55 | "execution_count": 2, 56 | "metadata": {}, 57 | "output_type": "execute_result" 58 | } 59 | ], 60 | "source": [ 61 | "b = a + z\n", 62 | "b.requires_grad" 63 | ] 64 | }, 65 | { 66 | "cell_type": "markdown", 67 | "metadata": {}, 68 | "source": [ 69 | "这个标志特别有用，当您想要冻结部分模型时，或者您事先知道不会使用某些参数的梯度。 \n", 70 | "\n", 71 | "例如，如果要对预先训练的CNN进行优化，只要切换冻结模型中的 requires_grad 标志就足够了，直到计算到最后一层才会保存中间缓冲区，\n", 72 | "其中的仿射变换将使用需要梯度的权重并且网络的输出也将需要它们。" 73 | ] 74 | }, 75 | { 76 | "cell_type": "code", 77 | "execution_count": 3, 78 | "metadata": {}, 79 | "outputs": [], 80 | "source": [ 81 | "import torchvision\n", 82 | "from torch import optim\n", 83 | "\n", 84 | "model = torchvision.models.resnet18(pretrained=True) \n", 85 | "for param in model.parameters(): \n", 86 | " param.requires_grad = False\n", 87 | "# Replace the last fully-connected layer\n", 88 | "# Parameters of newly constructed modules have requires_grad=True by default \n", 89 | "model.fc = nn.Linear(512, 100)\n", 90 | "\n", 91 | "# Optimize only the classifier \n", 92 | "optimizer = optim.SGD(model.fc.parameters(), lr=1e-2, momentum=0.9)" 93 | ] 94 | }, 95 | { 96 | "cell_type": "markdown", 97 | "metadata": {}, 98 | "source": [ 99 | "### volatile（现在已经不用了，这个在0.41版中已经被标注为过期了，现在可以直接使用Tensor）" 100 | ] 101 | }, 102 | { 103 | "cell_type": "markdown", 104 | "metadata": {}, 105 | "source": [ 106 | "**纯粹的inference模式下推荐使用 volatile**，当你确定你甚至不会调用 .backward() 时,它比任何其他自动求导的设置更有效\n", 107 | "\n", 108 | "它将使用绝对最小的内存来评估模型。 volatile 也决定了 require_grad is False 。\n", 109 | "\n", 110 | "**volatile 不同于 require_grad** 的传递。\n", 111 | "如果一个操作甚至只有有一个 volatile 的输入，它的输出也将是 volatile 。 \n", 112 | "\n", 113 | "Volatility 比“不需要梯度”更容易传递——只需要一个 volatile 的输入即可得到一个 volatile 的输出，相对的，需要所有的输入“不需要梯度”才能得到不需要梯度的输出。\n", 114 | "\n", 115 | "使用volatile标志，您不需要更改模型参数的任何设置来用于inference。创建一个 volatile 的输入就够了，这将保证不会保存中间状态。" 116 | ] 117 | }, 118 | { 119 | "cell_type": "markdown", 120 | "metadata": {}, 121 | "source": [ 122 | "### 张量（Tensor）\n", 123 | "--------\n", 124 | "\n", 125 | "``torch.Tensor``是这个包的核心类。如果设置\n", 126 | "``.requires_grad`` 为 ``True``，那么将会追踪所有对于该张量的操作。 \n", 127 | "当完成计算后通过调用 ``.backward()``，自动计算所有的梯度，\n", 128 | "这个张量的所有梯度将会自动积累到 ``.grad`` 属性。\n", 129 | "\n", 130 | "要阻止张量跟踪历史记录，可以调用``.detach()``方法将其与计算历史记录分离，并禁止跟踪它将来的计算记录。\n", 131 | "\n", 132 | "为了防止跟踪历史记录（和使用内存），可以将代码块包装在``with torch.no_grad()：``中。\n", 133 | "在评估模型时特别有用，因为模型可能具有`requires_grad = True`的可训练参数，但是我们不需要梯度计算。\n", 134 | "\n", 135 | "在自动梯度计算中还有另外一个重要的类``Function``.\n", 136 | "\n", 137 | "``Tensor`` and ``Function`` are interconnected and build up an acyclic\n", 138 | "graph, that encodes a complete history of computation. Each tensor has\n", 139 | "a ``.grad_fn`` attribute that references a ``Function`` that has created\n", 140 | "the ``Tensor`` (except for Tensors created by the user - their\n", 141 | "``grad_fn is None``).\n", 142 | "\n", 143 | "``Tensor`` 和 ``Function``互相连接并生成一个非循环图，它表示和存储了完整的计算历史。\n", 144 | "每个张量都有一个``.grad_fn``属性，这个属性引用了一个创建了``Tensor``的``Function``（除非这个张量是用户手动创建的，即，这个张量的\n", 145 | "``grad_fn`` 是 ``None``）。\n", 146 | "\n", 147 | "如果需要计算导数，你可以在``Tensor``上调用``.backward()``。 \n", 148 | "如果``Tensor``是一个标量（即它包含一个元素数据）则不需要为``backward()``指定任何参数，\n", 149 | "但是如果它有更多的元素，你需要指定一个``gradient`` 参数来匹配张量的形状。\n" 150 | ] 151 | }, 152 | { 153 | "cell_type": "code", 154 | "execution_count": 4, 155 | "metadata": {}, 156 | "outputs": [ 157 | { 158 | "name": "stdout", 159 | "output_type": "stream", 160 | "text": [ 161 | "tensor([[1., 1.],\n", 162 | " [1., 1.]], requires_grad=True)\n" 163 | ] 164 | } 165 | ], 166 | "source": [ 167 | "x = torch.ones(2, 2, requires_grad=True)\n", 168 | "print(x)" 169 | ] 170 | }, 171 | { 172 | "cell_type": "code", 173 | "execution_count": 5, 174 | "metadata": {}, 175 | "outputs": [ 176 | { 177 | "name": "stdout", 178 | "output_type": "stream", 179 | "text": [ 180 | "tensor([[1., 1.],\n", 181 | " [1., 1.]], requires_grad=True)\n" 182 | ] 183 | } 184 | ], 185 | "source": [ 186 | "x = torch.ones(2, 2, requires_grad=True)\n", 187 | "print(x)" 188 | ] 189 | }, 190 | { 191 | "cell_type": "markdown", 192 | "metadata": {}, 193 | "source": [ 194 | "## 自动求导如何编码历史信息\n", 195 | "每个变量都有一个 .creator 属性，它指向把它作为输出的函数。\n", 196 | "\n", 197 | "这是一个由 Function 对象作为节点组成的有向无环图（DAG）的入口点，它们之间的引用就是图的边。每次执行一个操作时，一个表示它的新 Function 就被实例化，它的 forward() 方法被调用，并且它输出的 Variable 的创建者被设置为这个 Function 。然后，通过跟踪从任何变量到叶节点的路径，可以重建创建数据的操作序列，并自动计算梯度。\n", 198 | "\n", 199 | "**需要注意的一点是**，整个图在每次迭代时都是从头开始重新创建的，这就允许使用任意的 Python控制流语句，这样可以在每次迭代时改变图的整体形状和大小。在启动训练之前不必对所有可能的路径进行编码—— what you run is what you differentiate.\n" 200 | ] 201 | }, 202 | { 203 | "cell_type": "markdown", 204 | "metadata": {}, 205 | "source": [ 206 | "## Variable上的In-place操作\n", 207 | "在自动求导中支持in-place操作是一件很困难的事情，我们在大多数情况下都不鼓励使用它们。\n", 208 | "\n", 209 | "Autograd的缓冲区释放和重用非常高效，并且很少场合下in-place操作能实际上明显降低内存的使用量。除非您在内存压力很大的情况下，否则您可能永远不需要使用它们。\n", 210 | "\n", 211 | "限制in-place操作适用性主要有两个原因：\n", 212 | "１．覆盖梯度计算所需的值。这就是为什么变量不支持 log_ 。它的梯度公式需要原始输入， 而虽然通过计算反向操作可以重新创建它，但在数值上是不稳定的，并且需要额外的工作， 这往往会与使用这些功能的目的相悖。\n", 213 | "\n", 214 | "２．每个in-place操作实际上需要实现重写计算图。不合适的版本只需分配新对象并保留对旧 图的引用，而in-place操作则需要将所有输入的 creator 更改为表示此操作的 Function 。这就 比较棘手，特别是如果有许多变量引用相同的存储（例如通过索引或转置创建的），并且如 果被修改输入的存储被任何其他 Variable 引用，则in-place函数实际上会抛出错误" 215 | ] 216 | }, 217 | { 218 | "cell_type": "markdown", 219 | "metadata": {}, 220 | "source": [ 221 | "## In-place正确性检查\n", 222 | "每个变量保留有version counter，它每次都会递增，当在任何操作中被使用时。当 Function 保存任何用于后向的tensor时，还会保存其包含变量的version counter。一旦访问 self.saved_tensors ，它将被检查，如果它大于保存的值，则会引起错误。" 223 | ] 224 | }, 225 | { 226 | "cell_type": "code", 227 | "execution_count": null, 228 | "metadata": {}, 229 | "outputs": [], 230 | "source": [] 231 | } 232 | ], 233 | "metadata": { 234 | "kernelspec": { 235 | "display_name": "Python 3", 236 | "language": "python", 237 | "name": "python3" 238 | }, 239 | "language_info": { 240 | "codemirror_mode": { 241 | "name": "ipython", 242 | "version": 3 243 | }, 244 | "file_extension": ".py", 245 | "mimetype": "text/x-python", 246 | "name": "python", 247 | "nbconvert_exporter": "python", 248 | "pygments_lexer": "ipython3", 249 | "version": "3.7.2" 250 | }, 251 | "toc": { 252 | "base_numbering": 1, 253 | "nav_menu": {}, 254 | "number_sections": true, 255 | "sideBar": true, 256 | "skip_h1_title": false, 257 | "title_cell": "Table of Contents", 258 | "title_sidebar": "Contents", 259 | "toc_cell": false, 260 | "toc_position": { 261 | "height": "calc(100% - 180px)", 262 | "left": "10px", 263 | "top": "150px", 264 | "width": "273.188px" 265 | }, 266 | "toc_section_display": true, 267 | "toc_window_display": true 268 | } 269 | }, 270 | "nbformat": 4, 271 | "nbformat_minor": 2 272 | } 273 | -------------------------------------------------------------------------------- /src/Week1/Day1/2.CUDA语义.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# CUDA语义\n", 8 | "torch.cuda 会记录当前选择的GPU，并且分配的所有CUDA张量将在上面创建。可以使用 torch.cuda.device 上下文管理器更改所选设备。\n", 9 | "\n", 10 | "但是，一旦张量被分配，您可以直接对其进行操作，而不考虑所选择的设备，结果将始终放 在与张量相同的设备上。\n", 11 | "\n", 12 | "默认情况下，不支持跨GPU操作，唯一的例外是 copy_() 。 除非启用对等存储器访问，否则 对分布不同设备上的张量任何启动操作的尝试都将会引发错误。\n", 13 | "下面你可以找到一个展示如下的小例子：" 14 | ] 15 | }, 16 | { 17 | "cell_type": "code", 18 | "execution_count": 1, 19 | "metadata": {}, 20 | "outputs": [ 21 | { 22 | "name": "stdout", 23 | "output_type": "stream", 24 | "text": [ 25 | "tensor([1.2096])\n", 26 | "1.209649682044983\n" 27 | ] 28 | } 29 | ], 30 | "source": [ 31 | "import torch\n", 32 | "\n", 33 | "x = torch.randn(1)\n", 34 | "print(x)\n", 35 | "print(x.item())" 36 | ] 37 | }, 38 | { 39 | "cell_type": "code", 40 | "execution_count": 2, 41 | "metadata": { 42 | "scrolled": true 43 | }, 44 | "outputs": [ 45 | { 46 | "name": "stdout", 47 | "output_type": "stream", 48 | "text": [ 49 | "tensor([2.2096], device='cuda:0')\n", 50 | "tensor([2.2096], dtype=torch.float64)\n" 51 | ] 52 | } 53 | ], 54 | "source": [ 55 | "# is_available 函数判断是否有cuda可以使用\n", 56 | "# ``torch.device``将张量移动到指定的设备中\n", 57 | "if torch.cuda.is_available():\n", 58 | " device = torch.device(\"cuda\") # a CUDA 设备对象\n", 59 | " y = torch.ones_like(x, device=device) # 直接从GPU创建张量\n", 60 | " x = x.to(device) # 或者直接使用``.to(\"cuda\")``将张量移动到cuda中\n", 61 | " z = x + y\n", 62 | " print(z)\n", 63 | " print(z.to(\"cpu\", torch.double)) # ``.to`` 也会对变量的类型做更改" 64 | ] 65 | }, 66 | { 67 | "cell_type": "markdown", 68 | "metadata": {}, 69 | "source": [ 70 | "## 最佳实践 \n", 71 | "使用固定的内存缓冲区\n", 72 | "当副本来自固定（页锁）内存时，主机到GPU的复制速度要快很多。\n", 73 | "\n", 74 | "CPU张量和存储开放了 一个 pin_memory() 方法，它返回该对象的副本，而它的数据放在固定区域中。\n", 75 | "\n", 76 | "另外，一旦固定了张量或存储，就可以使用异步的GPU副本。只需传递一个额外的 async=True 参数到 cuda() 的调用。\n", 77 | "\n", 78 | "这可以用于将数据传输与计算重叠。通过将 pin_memory=True 传递给其构造函数，可以使 DataLoader 将batch返回到固定内存中。" 79 | ] 80 | }, 81 | { 82 | "cell_type": "markdown", 83 | "metadata": {}, 84 | "source": [ 85 | "## 使用 nn.DataParallel 替代 multiprocessing CUDA语义\n", 86 | "\n", 87 | "大多数涉及批量输入和多个GPU的情况应默认使用 DataParallel 来使用多个GPU。尽管有GIL的存在，单个python进程也可能使多个GPU饱和。\n", 88 | "\n", 89 | "从0.1.9版本开始，大量的GPU(8+)可能未被充分利用。然而，这是一个已知的问题，也正在积极开发。和往常一样，测试你的用例吧。\n", 90 | "\n", 91 | "调用 multiprocessing 来利用CUDA模型存在重要的注意事项；使用具有多处理功能的CUDA模型有重要的注意事项；除非就是需要谨慎地满足数据处理需求，否则您的程序很可能会出现错误或未定义的行为。" 92 | ] 93 | } 94 | ], 95 | "metadata": { 96 | "kernelspec": { 97 | "display_name": "Python 3", 98 | "language": "python", 99 | "name": "python3" 100 | }, 101 | "language_info": { 102 | "codemirror_mode": { 103 | "name": "ipython", 104 | "version": 3 105 | }, 106 | "file_extension": ".py", 107 | "mimetype": "text/x-python", 108 | "name": "python", 109 | "nbconvert_exporter": "python", 110 | "pygments_lexer": "ipython3", 111 | "version": "3.7.2" 112 | }, 113 | "toc": { 114 | "base_numbering": 1, 115 | "nav_menu": {}, 116 | "number_sections": true, 117 | "sideBar": true, 118 | "skip_h1_title": false, 119 | "title_cell": "Table of Contents", 120 | "title_sidebar": "Contents", 121 | "toc_cell": false, 122 | "toc_position": {}, 123 | "toc_section_display": true, 124 | "toc_window_display": true 125 | } 126 | }, 127 | "nbformat": 4, 128 | "nbformat_minor": 2 129 | } 130 | -------------------------------------------------------------------------------- /src/Week1/Day1/3.多进程最佳实践.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# 多进程最佳实践\n", 8 | "torch.multiprocessing 是Python multiprocessing 的替代品。它支持完全相同的操作，但扩展了它以便通过 multiprocessing.Queue 发送的所有张量将其数据移动到共享内存中，并且只会向其他进程发送一个句柄。\n", 9 | "
  • 注意
    • \n", 10 | "
    当一个张量Tensor发送到另一个进程时， 张量Tensor数据将被共享。如果torch.Tensor.grad不是None，它也是共享的。在Tensor没有torch.Tensor.grad字段发送到另一个进程之后，它会创建一个标准的特定。grad Tensor于流程的流程，不会在所有流程中自动共享，这与 Tensor数据的共享方式不同。
    \n", 11 | "这允许实现各种训练方法，如Hogwild，A3C或任何其他需要异步操作的方法。" 12 | ] 13 | }, 14 | { 15 | "cell_type": "markdown", 16 | "metadata": {}, 17 | "source": [ 18 | "## 共享CUDA张量\n", 19 | "\n", 20 | "仅在Python 3中使用 spawn 或 forkserver 启动方法才支持在进程之间共享CUDA张量。Python 2中的 multiprocessing 只能使用 fork 创建子进程，并且不被CUDA运行时所支持。 \n", 21 | "\n", 22 | "
  • 需要注意的地方
    • \n", 23 | "CUDA API要求导出到其他进程的分配，只要它们被使用就要一直保持有效。您应该小心，确保您共享的CUDA张量只要有必要就不要超出范围。这不是共享模型参数的问题， 但传递其他类型的数据应该小心。注意，此限制不适用于共享CPU内存。\n", 24 | "\n", 25 | "参考：使用 nn.DataParallel 替代 multiprocessing " 26 | ] 27 | }, 28 | { 29 | "cell_type": "markdown", 30 | "metadata": {}, 31 | "source": [ 32 | "## 最佳实践和提示 \n", 33 | "### 避免和抵制死锁\n", 34 | "\n", 35 | "当一个新进程被产生时，有很多事情可能会出错，最常见的死锁原因是后台线程。\n", 36 | "如果有任何线程持有锁或导入模块，并且 fork 被调用，则子进程很可能处于损坏的状态，并以不同的方式死锁或失败。\n", 37 | "\n", 38 | "注意，即使您没有，Python内置的库也可能会这样做 —— 不需要看得比 multiprocessing 更远。 \n", 39 | "\n", 40 | "multiprocessing.Queue 实际上是一个非常复杂的类，它产生用于序列化，发送和接收对象的多个线程，它们也可能引起上述问题。如果您发现自己处于这种情 况，请尝试使用 multiprocessing.queues.SimpleQueue ，这不会使用任何其他线程。\n", 41 | "\n", 42 | "我们正在竭尽全力把它设计得更简单，并确保这些死锁不会发生，但有些事情无法控制。如果有任何问题您无法一时无法解决，请尝试在论坛上提出，我们将看看是否可以解决问题" 43 | ] 44 | }, 45 | { 46 | "cell_type": "markdown", 47 | "metadata": {}, 48 | "source": [ 49 | "### 重用经过队列的缓冲区\n", 50 | "\n", 51 | "记住每次将 Tensor 放入 multiprocessing.Queue 时，必须将其移动到共享内存中。\n", 52 | "\n", 53 | "如果它已经被共享，它是一个无效的操作，否则会产生一个额外的内存副本，这会减缓整个进程。\n", 54 | "\n", 55 | "即使你有一个进程池来发送数据到一个进程，使它返回缓冲区 —— 这几乎是免费的，并且允许你在 发送下一个batch时避免产生副本。\n", 56 | "\n", 57 | "### 异步多进程训练（例如Hogwild）\n", 58 | "\n", 59 | "使用 torch.multiprocessing ，可以异步地训练模型，参数可以一直共享，也可以定期同步。在第一种情况下，我们建议发送整个模型对象，而在后者中，我们建议只发送 state_dict() 。\n", 60 | "\n", 61 | "我们建议使用 multiprocessing.Queue 来在进程之间传递各种PyTorch对象。例如， 当使用fork 启动方法时，可能会继承共享内存中的张量和存储器，但这是非常容易出错的，应谨慎使用，而且只能由高级用户使用。\n", 62 | "\n", 63 | "队列虽然有时是一个较不优雅的解决方案，但基本上能在所 有情况下正常工作。\n", 64 | "\n", 65 | "
  • 需要注意的一点
    • \n", 66 | "你应该注意有关全局语句，它们没有被 if __name__ == '__main__' 保护。如果使用 与 fork 不同的启动方法，则它们将在所有子进程中执行。" 67 | ] 68 | }, 69 | { 70 | "cell_type": "code", 71 | "execution_count": 1, 72 | "metadata": {}, 73 | "outputs": [], 74 | "source": [ 75 | "import torch.nn as nn\n", 76 | "import torch.nn.functional as F\n", 77 | "import torch.multiprocessing as mp\n", 78 | "\n", 79 | "class Net(nn.Module):\n", 80 | " def __init__(self):\n", 81 | " super(Net, self).__init__()\n", 82 | " self.conv1 = nn.Conv2d(1, 10, kernel_size=5)\n", 83 | " self.conv2 = nn.Conv2d(10, 20, kernel_size=5)\n", 84 | " self.conv2_drop = nn.Dropout2d()\n", 85 | " self.fc1 = nn.Linear(320, 50)\n", 86 | " self.fc2 = nn.Linear(50, 10)\n", 87 | "\n", 88 | " def forward(self, x):\n", 89 | " x = F.relu(F.max_pool2d(self.conv1(x), 2))\n", 90 | " x = F.relu(F.max_pool2d(self.conv2_drop(self.conv2(x)), 2))\n", 91 | " x = x.view(-1, 320)\n", 92 | " x = F.relu(self.fc1(x))\n", 93 | " x = F.dropout(x, training=self.training)\n", 94 | " x = self.fc2(x)\n", 95 | " return F.log_softmax(x, dim=1)" 96 | ] 97 | }, 98 | { 99 | "cell_type": "code", 100 | "execution_count": 2, 101 | "metadata": {}, 102 | "outputs": [], 103 | "source": [ 104 | "# Hogwild\n", 105 | "# 在examples repository中可以找到具体的Hogwild实现，可以展示代码的整体结构。下面一个小例子(自己去找替换一个model)：\n", 106 | "\n", 107 | "import torch.multiprocessing as mp\n", 108 | "\n", 109 | "def train(model):\n", 110 | " # Construct data_loader, optimizer, etc.\n", 111 | " for data, labels in data_loader:\n", 112 | " optimizer.zero_grad()\n", 113 | " loss_fn(model(data), labels).backward()\n", 114 | " optimizer.step() # This will update the shared parameters\n", 115 | "\n", 116 | "if __name__ == '__main__':\n", 117 | " num_processes = 4\n", 118 | " model = Net()\n", 119 | " # NOTE: this is required for the ``fork`` method to work\n", 120 | " model.share_memory()\n", 121 | " processes = []\n", 122 | " for rank in range(num_processes):\n", 123 | " p = mp.Process(target=train, args=(model,))\n", 124 | " p.start()\n", 125 | " processes.append(p)\n", 126 | " for p in processes:\n", 127 | " p.join()" 128 | ] 129 | } 130 | ], 131 | "metadata": { 132 | "kernelspec": { 133 | "display_name": "Python 3", 134 | "language": "python", 135 | "name": "python3" 136 | }, 137 | "language_info": { 138 | "codemirror_mode": { 139 | "name": "ipython", 140 | "version": 3 141 | }, 142 | "file_extension": ".py", 143 | "mimetype": "text/x-python", 144 | "name": "python", 145 | "nbconvert_exporter": "python", 146 | "pygments_lexer": "ipython3", 147 | "version": "3.7.2" 148 | }, 149 | "toc": { 150 | "base_numbering": 1, 151 | "nav_menu": {}, 152 | "number_sections": true, 153 | "sideBar": true, 154 | "skip_h1_title": false, 155 | "title_cell": "Table of Contents", 156 | "title_sidebar": "Contents", 157 | "toc_cell": false, 158 | "toc_position": { 159 | "height": "calc(100% - 180px)", 160 | "left": "10px", 161 | "top": "150px", 162 | "width": "273.188px" 163 | }, 164 | "toc_section_display": true, 165 | "toc_window_display": true 166 | } 167 | }, 168 | "nbformat": 4, 169 | "nbformat_minor": 2 170 | } 171 | -------------------------------------------------------------------------------- /src/Week1/Day2/5.序列化语义.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# 序列化语义 \n", 8 | "## 最佳实践\n", 9 | "## 保存模型的推荐方法\n", 10 | "这主要有两种方法序列化和恢复模型。 " 11 | ] 12 | }, 13 | { 14 | "cell_type": "markdown", 15 | "metadata": {}, 16 | "source": [ 17 | "### 第一种（推荐）只保存和加载模型参数： \n", 18 | "torch.save(the_model.state_dict(), PATH)" 19 | ] 20 | }, 21 | { 22 | "cell_type": "markdown", 23 | "metadata": {}, 24 | "source": [ 25 | "
    \n",
26 |     "the_model = TheModelClass(*args, **kwargs) \n",
27 |     "the_model.load_state_dict(torch.load(PATH))\n",
28 |     "
    " 29 | ] 30 | }, 31 | { 32 | "cell_type": "markdown", 33 | "metadata": {}, 34 | "source": [ 35 | "## 第二种保存和加载整个模型： \n", 36 | "torch.save(the_model, PATH)" 37 | ] 38 | }, 39 | { 40 | "cell_type": "markdown", 41 | "metadata": {}, 42 | "source": [ 43 | "
    \n",
44 |     "the_model = torch.load(PATH)\n",
45 |     "
    " 46 | ] 47 | }, 48 | { 49 | "cell_type": "markdown", 50 | "metadata": {}, 51 | "source": [ 52 | "然而，在这种情况下，序列化的数据被绑定到特定的类和固定的目录结构，所以当在其他项目中使用时，或者在一些严重的重构器之后它可能会以各种方式break。" 53 | ] 54 | } 55 | ], 56 | "metadata": { 57 | "kernelspec": { 58 | "display_name": "Python 3", 59 | "language": "python", 60 | "name": "python3" 61 | }, 62 | "language_info": { 63 | "codemirror_mode": { 64 | "name": "ipython", 65 | "version": 3 66 | }, 67 | "file_extension": ".py", 68 | "mimetype": "text/x-python", 69 | "name": "python", 70 | "nbconvert_exporter": "python", 71 | "pygments_lexer": "ipython3", 72 | "version": "3.7.2" 73 | }, 74 | "toc": { 75 | "base_numbering": 1, 76 | "nav_menu": {}, 77 | "number_sections": true, 78 | "sideBar": true, 79 | "skip_h1_title": false, 80 | "title_cell": "Table of Contents", 81 | "title_sidebar": "Contents", 82 | "toc_cell": false, 83 | "toc_position": {}, 84 | "toc_section_display": true, 85 | "toc_window_display": false 86 | } 87 | }, 88 | "nbformat": 4, 89 | "nbformat_minor": 2 90 | } 91 | -------------------------------------------------------------------------------- /src/Week1/Day2/6.torch - PyTorch中文文档.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/src/Week1/Day2/6.torch - PyTorch中文文档.pdf -------------------------------------------------------------------------------- /src/Week1/Day3-Day5/6/6.torch - PyTorch中文文档.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/src/Week1/Day3-Day5/6/6.torch - PyTorch中文文档.pdf -------------------------------------------------------------------------------- /src/Week1/Day3-Day5/6/tensor操作参考/1.tensor基础数据类型.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/src/Week1/Day3-Day5/6/tensor操作参考/1.tensor基础数据类型.pdf -------------------------------------------------------------------------------- /src/Week1/Day3-Day5/6/tensor操作参考/2.创建tensor.doc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/src/Week1/Day3-Day5/6/tensor操作参考/2.创建tensor.doc -------------------------------------------------------------------------------- /src/Week1/Day3-Day5/6/tensor操作参考/2.创建tensor.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/src/Week1/Day3-Day5/6/tensor操作参考/2.创建tensor.pdf -------------------------------------------------------------------------------- /src/Week1/Day3-Day5/6/tensor操作参考/3.索引和切片.doc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/src/Week1/Day3-Day5/6/tensor操作参考/3.索引和切片.doc -------------------------------------------------------------------------------- /src/Week1/Day3-Day5/6/tensor操作参考/3.索引和切片.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/src/Week1/Day3-Day5/6/tensor操作参考/3.索引和切片.pdf -------------------------------------------------------------------------------- /src/Week1/Day3-Day5/6/tensor操作参考/4.tensor变换.doc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/src/Week1/Day3-Day5/6/tensor操作参考/4.tensor变换.doc -------------------------------------------------------------------------------- /src/Week1/Day3-Day5/6/tensor操作参考/4.tensor变换.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/src/Week1/Day3-Day5/6/tensor操作参考/4.tensor变换.pdf -------------------------------------------------------------------------------- /src/Week1/Day3-Day5/6/tensor操作参考/5.Concat-split.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/src/Week1/Day3-Day5/6/tensor操作参考/5.Concat-split.pdf -------------------------------------------------------------------------------- /src/Week1/Day3-Day5/6/tensor操作参考/5.concat-split.doc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/src/Week1/Day3-Day5/6/tensor操作参考/5.concat-split.doc -------------------------------------------------------------------------------- /src/Week1/Day3-Day5/6/tensor操作参考/6.math operation.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/src/Week1/Day3-Day5/6/tensor操作参考/6.math operation.pdf -------------------------------------------------------------------------------- /src/Week1/Day3-Day5/6/tensor操作参考/7.statisics.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/src/Week1/Day3-Day5/6/tensor操作参考/7.statisics.pdf -------------------------------------------------------------------------------- /src/Week1/Day3-Day5/6/tensor操作参考/9.tensor advanced.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/src/Week1/Day3-Day5/6/tensor操作参考/9.tensor advanced.pdf -------------------------------------------------------------------------------- /src/Week1/Day3-Day5/6/tensor操作参考/9.tensor_advanced.doc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/src/Week1/Day3-Day5/6/tensor操作参考/9.tensor_advanced.doc -------------------------------------------------------------------------------- /src/Week1/lesson1.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # encoding: utf-8 3 | """ 4 | @author: HuRuiFeng 5 | @file: lesson1.py 6 | @time: 2019/6/6 10:07 7 | @desc: 验证安装的环境 8 | """ 9 | import torch 10 | 11 | print(torch.__version__) 12 | print(torch.cuda.is_available()) 13 | print('hello, world.') -------------------------------------------------------------------------------- /src/Week2/Day08/14.torch.cuda.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# torch.cuda\n", 8 | "该包增加了对CUDA张量类型的支持，实现了与CPU张量相同的功能，但使用GPU进行计算。\n", 9 | "它是懒惰的初始化，所以你可以随时导入它，并使用is_available()来确定系统是否支持CUDA。" 10 | ] 11 | }, 12 | { 13 | "cell_type": "code", 14 | "execution_count": 1, 15 | "metadata": {}, 16 | "outputs": [ 17 | { 18 | "data": { 19 | "text/plain": [ 20 | "620199616" 21 | ] 22 | }, 23 | "execution_count": 1, 24 | "metadata": {}, 25 | "output_type": "execute_result" 26 | } 27 | ], 28 | "source": [ 29 | "import torch\n", 30 | "\n", 31 | "#返回cublasHandle_t指针，指向当前cuBLAS句柄\n", 32 | "torch.cuda.current_blas_handle() " 33 | ] 34 | }, 35 | { 36 | "cell_type": "code", 37 | "execution_count": 2, 38 | "metadata": {}, 39 | "outputs": [ 40 | { 41 | "data": { 42 | "text/plain": [ 43 | "0" 44 | ] 45 | }, 46 | "execution_count": 2, 47 | "metadata": {}, 48 | "output_type": "execute_result" 49 | } 50 | ], 51 | "source": [ 52 | "#返回当前所选设备的索引\n", 53 | "torch.cuda.current_device()" 54 | ] 55 | }, 56 | { 57 | "cell_type": "code", 58 | "execution_count": 3, 59 | "metadata": {}, 60 | "outputs": [ 61 | { 62 | "data": { 63 | "text/plain": [ 64 | "" 65 | ] 66 | }, 67 | "execution_count": 3, 68 | "metadata": {}, 69 | "output_type": "execute_result" 70 | } 71 | ], 72 | "source": [ 73 | "#返回一个当前所选的 Stream\n", 74 | "torch.cuda.current_stream() " 75 | ] 76 | }, 77 | { 78 | "cell_type": "markdown", 79 | "metadata": {}, 80 | "source": [ 81 | "### torch.cuda.device(idx) \n", 82 | "上下文管理器，可以更改所选设备。 \n", 83 | "参数：idx(int)表示设备索引选择。如果这个参数是负的，则是无效操作" 84 | ] 85 | }, 86 | { 87 | "cell_type": "code", 88 | "execution_count": 4, 89 | "metadata": {}, 90 | "outputs": [ 91 | { 92 | "data": { 93 | "text/plain": [ 94 | "" 95 | ] 96 | }, 97 | "execution_count": 4, 98 | "metadata": {}, 99 | "output_type": "execute_result" 100 | } 101 | ], 102 | "source": [ 103 | "torch.cuda.device(0)" 104 | ] 105 | }, 106 | { 107 | "cell_type": "code", 108 | "execution_count": 5, 109 | "metadata": {}, 110 | "outputs": [ 111 | { 112 | "data": { 113 | "text/plain": [ 114 | "1" 115 | ] 116 | }, 117 | "execution_count": 5, 118 | "metadata": {}, 119 | "output_type": "execute_result" 120 | } 121 | ], 122 | "source": [ 123 | "#返回可得到的GPU数量。\n", 124 | "torch.cuda.device_count()" 125 | ] 126 | }, 127 | { 128 | "cell_type": "code", 129 | "execution_count": 6, 130 | "metadata": {}, 131 | "outputs": [ 132 | { 133 | "data": { 134 | "text/plain": [ 135 | "True" 136 | ] 137 | }, 138 | "execution_count": 6, 139 | "metadata": {}, 140 | "output_type": "execute_result" 141 | } 142 | ], 143 | "source": [ 144 | "#返回一个bool值，指示CUDA当前是否可用\n", 145 | "torch.cuda.is_available()" 146 | ] 147 | }, 148 | { 149 | "cell_type": "markdown", 150 | "metadata": {}, 151 | "source": [ 152 | "设置当前设备。 \n", 153 | "不鼓励使用此函数来设置。在大多数情况下，最好使用CUDA_VISIBLE_DEVICES环境变量。 \n", 154 | "参数：device(int)表示所选设备。如果此参数为负，则此函数是无效操作。" 155 | ] 156 | }, 157 | { 158 | "cell_type": "code", 159 | "execution_count": 7, 160 | "metadata": {}, 161 | "outputs": [], 162 | "source": [ 163 | "device = torch.cuda.set_device(0)" 164 | ] 165 | }, 166 | { 167 | "cell_type": "markdown", 168 | "metadata": {}, 169 | "source": [ 170 | "选择给定流的上下文管理器。 \n", 171 | "在其上下文中排队的所有CUDA核心将在所选流上入队。 \n", 172 | "参数：stream(Stream)表示所选流。如果是None，则这个管理器是无效的。" 173 | ] 174 | }, 175 | { 176 | "cell_type": "code", 177 | "execution_count": 8, 178 | "metadata": {}, 179 | "outputs": [ 180 | { 181 | "data": { 182 | "text/plain": [ 183 | "" 184 | ] 185 | }, 186 | "execution_count": 8, 187 | "metadata": {}, 188 | "output_type": "execute_result" 189 | } 190 | ], 191 | "source": [ 192 | "stream = torch.cuda.current_stream() \n", 193 | "torch.cuda.stream(stream)" 194 | ] 195 | }, 196 | { 197 | "cell_type": "markdown", 198 | "metadata": {}, 199 | "source": [ 200 | "等待当前设备上所有流中的所有核心完成。" 201 | ] 202 | }, 203 | { 204 | "cell_type": "code", 205 | "execution_count": 9, 206 | "metadata": {}, 207 | "outputs": [], 208 | "source": [ 209 | "torch.cuda.synchronize() " 210 | ] 211 | }, 212 | { 213 | "cell_type": "markdown", 214 | "metadata": {}, 215 | "source": [ 216 | "### torch.cuda.comm.reduce_add(inputs, destination=None) \n", 217 | "将来自多个GPU的张量相加，所有输入应具有匹配的形状 \n", 218 | "参数：\n", 219 | "- inputs(Iterable[Tensor])：要相加张量的迭代 \n", 220 | "- destination(int, optional)：将放置输出的设备（默认值：当前设备） \n", 221 | "\n", 222 | "返回： 一个包含放置在destination设备上的所有输入的元素总和的张量。" 223 | ] 224 | }, 225 | { 226 | "cell_type": "markdown", 227 | "metadata": {}, 228 | "source": [ 229 | "### torch.cuda.comm.scatter(tensor, devices, chunk_sizes=None, dim=0, streams=None) \n", 230 | "打散横跨多个GPU的张量。\n", 231 | "参数： \n", 232 | "- tensor(Tensor)：要分散的张量\n", 233 | "- devices(Iterable[int]) int的迭代，指定哪些设备应该分散张量。 \n", 234 | "- chunk_sizes(Iterable[int], optional)：要放置在每个设备上的块大小。它应该匹配devices的长度并且总和为tensor.size(dim)。如果没有指定，张量将被分成相等的块。\n", 235 | "- dim(int, optional)：沿着这个维度来chunk张量\n", 236 | "\n", 237 | "返回：包含tensor块的元组，分布在给定的devices上。" 238 | ] 239 | }, 240 | { 241 | "cell_type": "markdown", 242 | "metadata": {}, 243 | "source": [ 244 | "### torch.cuda.comm.gather(tensors, dim=0, destination=None) \n", 245 | "从多个GPU收集张量。\n", 246 | "张量尺寸在不同于 dim 的所有维度上都应该匹配。\n", 247 | "参数：\n", 248 | "- tensors(Iterable[Tensor])：要收集的张量的迭代。 \n", 249 | "- dim(int)：沿着此维度张量将被连接。 \n", 250 | "- destination(int, optional)：输出设备（-1表示CPU，默认值：当前设备）。\n", 251 | "\n", 252 | "返回： 一个张量位于destination设备上，这是沿着dim连接tensors的结果。" 253 | ] 254 | }, 255 | { 256 | "cell_type": "markdown", 257 | "metadata": {}, 258 | "source": [ 259 | "### 流和事件\n", 260 | "### class torch.cuda.Stream：CUDA流的包装 \n", 261 | "参数：\n", 262 | "- device(int, optional)：分配流的设备。 \n", 263 | "- priority(int, optional)：流的优先级。较低的数字代表较高的优先级。\n", 264 | "\n", 265 | "### query() \n", 266 | "检查所有提交的工作是否已经完成。 \n", 267 | "返回： 一个布尔值，表示此流中的所有核心是否完成。 " 268 | ] 269 | }, 270 | { 271 | "cell_type": "markdown", 272 | "metadata": {}, 273 | "source": [ 274 | "### record_event(event=None)\n", 275 | "记录一个事件\n", 276 | "参数：event(Event, optional)：要记录的事件。如果没有给出，将分配一个新的。 \n", 277 | "返回：记录的事件\n", 278 | "\n", 279 | "### synchronize()\n", 280 | "等待此流中的所有核心完成。\n", 281 | "\n", 282 | "### wait_event(event)\n", 283 | "将所有未来的工作提交到流等待事件。\n", 284 | "参数：event(Event)：等待的事件\n", 285 | "\n", 286 | "### wait_stream(stream)\n", 287 | "与另一个流同步，提交到此流的所有未来工作将等待直到所有核心在调用完成时提交给给定的流" 288 | ] 289 | }, 290 | { 291 | "cell_type": "markdown", 292 | "metadata": {}, 293 | "source": [ 294 | "### class torch.cuda.Event(enable_timing=False, blocking=False, interprocess=False, _handle=None) \n", 295 | "CUDA事件的包装。\n", 296 | "参数： \n", 297 | "- enable_timing(bool)：指示事件是否应该测量时间（默认值：False） \n", 298 | "- blocking(bool)：如果为true，wait()将被阻塞（默认值：False） \n", 299 | "- interprocess(bool)：如果为true， 则可以在进程之间共享事件（默认值：False）\n", 300 | "\n", 301 | "### elapsed_time(end_event)\n", 302 | "返回事件记录之前经过的时间。\n", 303 | "\n", 304 | "### ipc_handle()\n", 305 | "返回此事件的IPC句柄。\n", 306 | "\n", 307 | "### query()\n", 308 | "检查事件是否已被记录。 \n", 309 | "返回： 一个布尔值，指示事件是否已被记录。\n", 310 | "\n", 311 | "### record(stream=None)\n", 312 | "记录给定流的事件。\n", 313 | "\n", 314 | "### synchronize()\n", 315 | "与事件同步。\n", 316 | "\n", 317 | "### wait(stream=None)\n", 318 | "使给定的流等待事件。" 319 | ] 320 | } 321 | ], 322 | "metadata": { 323 | "kernelspec": { 324 | "display_name": "Python 3", 325 | "language": "python", 326 | "name": "python3" 327 | }, 328 | "language_info": { 329 | "codemirror_mode": { 330 | "name": "ipython", 331 | "version": 3 332 | }, 333 | "file_extension": ".py", 334 | "mimetype": "text/x-python", 335 | "name": "python", 336 | "nbconvert_exporter": "python", 337 | "pygments_lexer": "ipython3", 338 | "version": "3.7.2" 339 | }, 340 | "toc": { 341 | "base_numbering": 1, 342 | "nav_menu": {}, 343 | "number_sections": false, 344 | "sideBar": true, 345 | "skip_h1_title": false, 346 | "title_cell": "Table of Contents", 347 | "title_sidebar": "Contents", 348 | "toc_cell": false, 349 | "toc_position": {}, 350 | "toc_section_display": true, 351 | "toc_window_display": false 352 | } 353 | }, 354 | "nbformat": 4, 355 | "nbformat_minor": 2 356 | } 357 | -------------------------------------------------------------------------------- /src/Week2/Day08/8.torch.Storage.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# torch.Storage \n", 8 | "一个 torch.Storage 是一个单一数据类型的连续一维数组。 \n", 9 | "\n", 10 | "每个 torch.Tensor 都有一个对应的、相同数据类型的存储。 \n", 11 | "\n", 12 | "class torch.FloatStorage \n", 13 | "byte()：将此存储转为byte类型 \n", 14 | "\n", 15 | "char()：将此存储转为char类型 \n", 16 | "\n", 17 | "double()： 将此存储转为double类型 \n", 18 | "\n", 19 | "short():将此存储转为short类型\n", 20 | "\n", 21 | "long():将此存储转为long类型 \n", 22 | "\n", 23 | "int()：将此存储转为int类型\n", 24 | "\n", 25 | "clone()：返回此存储的一个副本 \n", 26 | "\n", 27 | "copy_()：（待补充）\n", 28 | "\n", 29 | "cpu()：如果当前此存储不在CPU上，则返回一个它的CPU副本 " 30 | ] 31 | }, 32 | { 33 | "cell_type": "markdown", 34 | "metadata": {}, 35 | "source": [ 36 | "new():（待补充） \n", 37 | "\n", 38 | "pin_memory()：如果此存储当前未被锁定，则将它复制到锁定内存中。 resize_() share_memory_()\n", 39 | "\n", 40 | "size（）:（待补充） \n", 41 | "\n", 42 | "resize_() :（待补充） \n", 43 | "\n", 44 | "share_memory_()：将此存储移动到共享内存中。 对于已经在共享内存中的存储或者CUDA存储，这是一条空指令，它们不需要移动就能在进程 间共享。共享内存中的存储不能改变大小。 返回：self" 45 | ] 46 | }, 47 | { 48 | "cell_type": "markdown", 49 | "metadata": {}, 50 | "source": [ 51 | "cuda(device=None, async=False)：返回此对象在CUDA内存中的一个副本。 \n", 52 | "如果此对象已在CUDA内存中且在正确的设备上，那么不会执行复制操作，直接返回原对象。 \n", 53 | "\n", 54 | "参数：\n", 55 | "device (int) - 目标GPU的id。默认值是当前设备。 \n", 56 | "\n", 57 | "async (bool) -如果值为True，且源在锁定内存中，则副本相对于宿主是异步的。否则此参数不起效果。" 58 | ] 59 | }, 60 | { 61 | "cell_type": "markdown", 62 | "metadata": {}, 63 | "source": [ 64 | "data_ptr()：（待补充）" 65 | ] 66 | }, 67 | { 68 | "cell_type": "markdown", 69 | "metadata": {}, 70 | "source": [ 71 | "element_size() :（待补充）\n", 72 | "\n", 73 | "fill_():（待补充）\n" 74 | ] 75 | }, 76 | { 77 | "cell_type": "markdown", 78 | "metadata": {}, 79 | "source": [ 80 | "tolist():返回一个包含此存储中元素的列表 \n", 81 | "\n", 82 | "type(new_type=None, async=False) 将此对象转为指定类型。\n", 83 | "如果已经是正确类型，不会执行复制操作，直接返回原对象。 \n", 84 | "\n", 85 | "参数：\n", 86 | "new_type (type or string) -需要转成的类型 \n", 87 | "\n", 88 | "async (bool) -如果值为True，且源在锁定内存中而目标在GPU中——或正好相反，则复制操作相对于宿主异步执行。否则此参数不起效果。" 89 | ] 90 | } 91 | ], 92 | "metadata": { 93 | "kernelspec": { 94 | "display_name": "Python 3", 95 | "language": "python", 96 | "name": "python3" 97 | }, 98 | "language_info": { 99 | "codemirror_mode": { 100 | "name": "ipython", 101 | "version": 3 102 | }, 103 | "file_extension": ".py", 104 | "mimetype": "text/x-python", 105 | "name": "python", 106 | "nbconvert_exporter": "python", 107 | "pygments_lexer": "ipython3", 108 | "version": "3.7.2" 109 | }, 110 | "toc": { 111 | "base_numbering": 1, 112 | "nav_menu": {}, 113 | "number_sections": true, 114 | "sideBar": true, 115 | "skip_h1_title": false, 116 | "title_cell": "Table of Contents", 117 | "title_sidebar": "Contents", 118 | "toc_cell": false, 119 | "toc_position": {}, 120 | "toc_section_display": true, 121 | "toc_window_display": false 122 | } 123 | }, 124 | "nbformat": 4, 125 | "nbformat_minor": 2 126 | } 127 | -------------------------------------------------------------------------------- /src/Week2/Day09~Day10/16.torch.utils.data.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# torch.utils.data" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "### class torch.utils.data.Dataset\n", 15 | "表示Dataset的抽象类。\n", 16 | "所有其他数据集都应该进行子类化。所有子类应该override __len__ 和 __getitem__ ，前者提供了数据集的大小，后者支持整数索引，范围从0到len(self)。\n", 17 | "\n", 18 | "### class torch.utils.data.TensorDataset\n", 19 | "class torch.utils.data.TensorDataset(data_tensor, target_tensor) \n", 20 | "包装数据和目标张量的数据集。\n", 21 | "通过沿着第一个维度索引两个张量来恢复每个样本。 \n", 22 | "参数：\n", 23 | "- data_tensor (Tensor) － 包含样本数据\n", 24 | "- target_tensor (Tensor) － 包含样本目标（标签）" 25 | ] 26 | }, 27 | { 28 | "cell_type": "markdown", 29 | "metadata": {}, 30 | "source": [ 31 | "### class torch.utils.data.DataLoader\n", 32 | "class torch.utils.data.DataLoader(dataset, batch_size=1, shuffle=False, sampler=None, num_workers=0, collate_fn=< function default_collate>, pin_memory=False, drop_last=False)\n", 33 | "\n", 34 | "数据加载器。组合数据集和采样器，并在数据集上提供单进程或多进程迭代器。 \n", 35 | "参数：\n", 36 | "- dataset (Dataset) – 加载数据的数据集。\n", 37 | "- batch_size (int, optional) – 每个batch加载多少个样本(默认: 1)。\n", 38 | "- shufØe (bool, optional) – 设置为 True 时会在每个epoch重新打乱数据(默认: False).\n", 39 | "- sampler (Sampler, optional) – 定义从数据集中提取样本的策略。如果指定，则忽略shuffle 参数。\n", 40 | "- num_workers (int, optional) – 用多少个子进程加载数据。0表示数据将在主进程中加载(默认: 0)\n", 41 | "- collate_fn (callable, optional)\n", 42 | "- pin_memory (bool, optional)\n", 43 | "- drop_last (bool, optional) – 如果数据集大小不能被batch size整除，则设置为True后可删除最后一个不完整的batch。如果设为False并且数据集的大小不能被batch size整除，则最后一个batch将更小。(默认: False)" 44 | ] 45 | }, 46 | { 47 | "cell_type": "markdown", 48 | "metadata": {}, 49 | "source": [ 50 | "### class torch.utils.data.sampler.Sampler\n", 51 | "class torch.utils.data.sampler.Sampler(data_source) \n", 52 | "所有采样器的基础类。 \n", 53 | "每个采样器子类必须提供一个 __iter__ 方法，提供一种迭代数据集元素的索引的方法，以及返回迭代器长度的 __len__ 方法" 54 | ] 55 | }, 56 | { 57 | "cell_type": "markdown", 58 | "metadata": {}, 59 | "source": [ 60 | "### class torch.utils.data.sampler.SequentialSampler\n", 61 | "class torch.utils.data.sampler.SequentialSampler(data_source) \n", 62 | "样本元素顺序排列，始终以相同的顺序。 \n", 63 | "参数： \n", 64 | "- data_source (Dataset) – 采样的数据集" 65 | ] 66 | }, 67 | { 68 | "cell_type": "markdown", 69 | "metadata": {}, 70 | "source": [ 71 | "### class torch.utils.data.sampler.RandomSampler\n", 72 | "class torch.utils.data.sampler.RandomSampler(data_source) \n", 73 | "样本元素随机，没有替换。 \n", 74 | "参数： \n", 75 | "- data_source (Dataset) – 采样的数据集" 76 | ] 77 | }, 78 | { 79 | "cell_type": "markdown", 80 | "metadata": {}, 81 | "source": [ 82 | "### class torch.utils.data.sampler.SubsetRandomSampler\n", 83 | "class torch.utils.data.sampler.SubsetRandomSampler(indices) \n", 84 | "样本元素从指定的索引列表中随机抽取，没有替换。 \n", 85 | "参数： \n", 86 | "- indices (list) – 索引的列表" 87 | ] 88 | }, 89 | { 90 | "cell_type": "markdown", 91 | "metadata": {}, 92 | "source": [ 93 | "### class torch.utils.data.sampler.WeightedRandomSampler\n", 94 | "class torch.utils.data.sampler.WeightedRandomSampler(weights, num_samples, replacement=True) \n", 95 | "样本元素来自于[0,..,len(weights)-1]，给定概率（weights）。 \n", 96 | "参数： \n", 97 | "- weights (list) – 权重列表。没必要加起来为1 \n", 98 | "- num_samples (int) – 抽样数量" 99 | ] 100 | } 101 | ], 102 | "metadata": { 103 | "kernelspec": { 104 | "display_name": "Python 3", 105 | "language": "python", 106 | "name": "python3" 107 | }, 108 | "language_info": { 109 | "codemirror_mode": { 110 | "name": "ipython", 111 | "version": 3 112 | }, 113 | "file_extension": ".py", 114 | "mimetype": "text/x-python", 115 | "name": "python", 116 | "nbconvert_exporter": "python", 117 | "pygments_lexer": "ipython3", 118 | "version": "3.7.2" 119 | }, 120 | "toc": { 121 | "base_numbering": 1, 122 | "nav_menu": {}, 123 | "number_sections": false, 124 | "sideBar": true, 125 | "skip_h1_title": false, 126 | "title_cell": "Table of Contents", 127 | "title_sidebar": "Contents", 128 | "toc_cell": false, 129 | "toc_position": {}, 130 | "toc_section_display": true, 131 | "toc_window_display": false 132 | } 133 | }, 134 | "nbformat": 4, 135 | "nbformat_minor": 2 136 | } 137 | -------------------------------------------------------------------------------- /src/Week2/Day09~Day10/18.torchvision.datasets.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# torchvision.datasets\n", 8 | "torchvision.datasets 中包含了以下数据集 \n", 9 | "- MNIST COCO（用于图像标注和目标检测）(Captioning and Detection) \n", 10 | "- LSUN Classication \n", 11 | "- ImageFolder \n", 12 | "- Imagenet-12 \n", 13 | "- CIFAR10 \n", 14 | "- CIFAR100 \n", 15 | "- STL10" 16 | ] 17 | }, 18 | { 19 | "cell_type": "markdown", 20 | "metadata": {}, 21 | "source": [ 22 | "Datasets 拥有以下 API : \\_\\_getitem__ \\_\\_len__\n", 23 | "由于以上 Datasets 都是 torch.utils.data.Dataset 的子类，所以，他们也可以通过 torch.utils.data.DataLoader 使用多线程（python的多进程）。 \n", 24 | "举例说明： torch.utils.data.DataLoader(coco_cap, batch_size=args.batchSize, shuffle=True, num_workers=args.nThreads)在构造函数中，不同的数据集直接的构造函数会有些许不同，但是他们共同拥有 keyword 参数。\n", 25 | "- transform - 一个函数，原始图片作为输入，返回一个转换后的图片。 （详情请看下面关于 torchvision-tranform 的部分）\n", 26 | "- target_transform - 一个函数，输入为 target ，输出对其的转换。例子，输入的是图片标注 的 string ，输出为 word 的索引。" 27 | ] 28 | }, 29 | { 30 | "cell_type": "markdown", 31 | "metadata": {}, 32 | "source": [ 33 | "```python\n", 34 | "import torchvision.datasets as dset\n", 35 | "import torchvision.transforms as transforms\n", 36 | "cap = dset.CocoCaptions(root = 'dir where images are', \n", 37 | " annFile = 'json annotation file',\n", 38 | " transform=transforms.ToTensor())\n", 39 | "\n", 40 | "print('Number of samples: ', len(cap))\n", 41 | "img, target = cap[3] # load 4th sample\n", 42 | "\n", 43 | "print(\"Image Size: \", img.size())\n", 44 | "print(target)\n", 45 | "```" 46 | ] 47 | }, 48 | { 49 | "cell_type": "markdown", 50 | "metadata": {}, 51 | "source": [ 52 | "```python\n", 53 | "#检测:\n", 54 | "dset.CocoDetection(root=\"dir where images are\", annFile=\"json annotation file\", [transform, target_transform]) #需要路径\n", 55 | "```" 56 | ] 57 | }, 58 | { 59 | "cell_type": "markdown", 60 | "metadata": {}, 61 | "source": [ 62 | "## LSUN\n", 63 | "dset.LSUN(db_path, classes='train', [transform, target_transform]) \n", 64 | "参数说明： \n", 65 | "- db_path = 数据集文件的根目录 \n", 66 | "- classes = ‘train’ (所有类别, 训练集), ‘val’ (所有类别, 验证集), ‘test’ (所有类别, 测试集)\n", 67 | "- [‘bedroom_train’, ‘church_train’, …] - 需要加载的类别列" 68 | ] 69 | }, 70 | { 71 | "cell_type": "markdown", 72 | "metadata": {}, 73 | "source": [ 74 | "## ImageFolder\n", 75 | "一个通用的数据加载器，数据集中的数据以以下方式组织： \n", 76 | "root/dog/xxx.png、root/dog/xxy.png、root/dog/xxz.png \n", 77 | "root/cat/123.png、root/cat/nsdf3.png、root/cat/asd932_.png \n", 78 | "\n", 79 | "dset.ImageFolder(root=\"root folder path\", [transform, target_transform]) \n", 80 | "有以下成员变量: \n", 81 | "- self.classes - 用一个list保存 类名\n", 82 | "- self.class_to_idx - 类名对应的 索引\n", 83 | "- self.imgs - 保存(img-path, class) tuple的list\n", 84 | "```python\n", 85 | "img_dir='.\\\\img'\n", 86 | "#img_dir='E:\\\\mtcnn-onet\\\\train'\n", 87 | "data = ImageFolder(img_dir,transform=transforms.Compose(\n", 88 | " [transforms.Resize(224), #Bnnet 改成64,mobilenet,resnet改成224\n", 89 | " transforms.RandomHorizontalFlip(),\n", 90 | " transforms.ToTensor(),\n", 91 | " #transforms.Normalize(mean=(0.034528155, 0.033598177, 0.009853649), std=(0.15804708, 0.16410254, 0.0643605))\n", 92 | " ]))\n", 93 | "```" 94 | ] 95 | }, 96 | { 97 | "cell_type": "markdown", 98 | "metadata": {}, 99 | "source": [ 100 | "## CIFAR\n", 101 | "dset.CIFAR10(root, train=True, transform=None, target_transform=None, download=False) \n", 102 | "dset.CIFAR100(root, train=True, transform=None, target_transform=None, download=False) \n", 103 | "参数说明： \n", 104 | "- root : cifar-10-batches-py 的根目录 \n", 105 | "- train : True = 训练集, False = 测试集 - \n", 106 | "- download : True = 从互联上下载数据，并将其放在root目录下" 107 | ] 108 | }, 109 | { 110 | "cell_type": "markdown", 111 | "metadata": {}, 112 | "source": [ 113 | "## STL10\n", 114 | "dset.STL10(root, split='train', transform=None, target_transform=None, download=False) \n", 115 | "参数说明： \n", 116 | "- root : stl10_binary的根目录 \n", 117 | "- split : 'train' = 训练集, 'test' = 测试集, 'unlabeled' = 无标签数据集, 'train+unlabeled' = 训练 + 无标签数据集 (没有标签的标记为-1) \n", 118 | "- download : True = 从互联上下载数据，并将其放在root目录下" 119 | ] 120 | } 121 | ], 122 | "metadata": { 123 | "kernelspec": { 124 | "display_name": "Python 3", 125 | "language": "python", 126 | "name": "python3" 127 | }, 128 | "language_info": { 129 | "codemirror_mode": { 130 | "name": "ipython", 131 | "version": 3 132 | }, 133 | "file_extension": ".py", 134 | "mimetype": "text/x-python", 135 | "name": "python", 136 | "nbconvert_exporter": "python", 137 | "pygments_lexer": "ipython3", 138 | "version": "3.7.2" 139 | }, 140 | "toc": { 141 | "base_numbering": 1, 142 | "nav_menu": {}, 143 | "number_sections": true, 144 | "sideBar": true, 145 | "skip_h1_title": false, 146 | "title_cell": "Table of Contents", 147 | "title_sidebar": "Contents", 148 | "toc_cell": false, 149 | "toc_position": { 150 | "height": "calc(100% - 180px)", 151 | "left": "10px", 152 | "top": "150px", 153 | "width": "273.188px" 154 | }, 155 | "toc_section_display": true, 156 | "toc_window_display": true 157 | } 158 | }, 159 | "nbformat": 4, 160 | "nbformat_minor": 2 161 | } 162 | -------------------------------------------------------------------------------- /src/Week2/Day09~Day10/19.torchvision.models.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# torchvision.models\n", 8 | "\n", 9 | "torchvision.models模块的 子模块中包含以下模型结构。\n", 10 | "- AlexNet\n", 11 | "- DenseNet \n", 12 | "- Inception3\n", 13 | "- ResNet\n", 14 | "- SqueezeNet\n", 15 | "- VGG\n", 16 | "- alexnet\n", 17 | "- densenet\n", 18 | "- densenet121\n", 19 | "- densenet161\n", 20 | "- densenet169 \n", 21 | "- densenet201 \n", 22 | "- inception\n", 23 | "- inception_v3 \n", 24 | "- resnet \n", 25 | "- resnet101\n", 26 | "- resnet152\n", 27 | "- resnet18 \n", 28 | "- resnet34\n", 29 | "- resnet50\n", 30 | "- squeezenet\n", 31 | "- squeezenet1_0\n", 32 | "- squeezenet1_1\n", 33 | "- vgg\n", 34 | "- vgg11\n", 35 | "- vgg11_bn \n", 36 | "- vgg13 \n", 37 | "- vgg13_bn\n", 38 | "- vgg16 \n", 39 | "- vgg16_bn\n", 40 | "- vgg19" 41 | ] 42 | }, 43 | { 44 | "cell_type": "code", 45 | "execution_count": 1, 46 | "metadata": {}, 47 | "outputs": [], 48 | "source": [ 49 | "#你可以使用随机初始化的权重来创建这些模型。\n", 50 | "import torchvision.models as models\n", 51 | "resnet18 = models.resnet18()\n", 52 | "alexnet = models.alexnet()\n", 53 | "squeezenet = models.squeezenet1_0()\n", 54 | "densenet = models.densenet161()" 55 | ] 56 | }, 57 | { 58 | "cell_type": "code", 59 | "execution_count": 2, 60 | "metadata": {}, 61 | "outputs": [ 62 | { 63 | "name": "stderr", 64 | "output_type": "stream", 65 | "text": [ 66 | "Downloading: \"https://download.pytorch.org/models/alexnet-owt-4df8aa71.pth\" to C:\\Users\\Hurf/.cache\\torch\\checkpoints\\alexnet-owt-4df8aa71.pth\n", 67 | "100.0%\n" 68 | ] 69 | } 70 | ], 71 | "source": [ 72 | "#我们也提供了预训练(pre-trained)的模型\n", 73 | "import torchvision.models as models\n", 74 | "#pretrained=True就可以使用预训练的模型\n", 75 | "resnet18 = models.resnet18(pretrained=True)\n", 76 | "alexnet = models.alexnet(pretrained=True)" 77 | ] 78 | }, 79 | { 80 | "cell_type": "markdown", 81 | "metadata": {}, 82 | "source": [ 83 | "### AlexNet 模型结构 \n", 84 | "torchvision.models.alexnet(pretrained=False, ** kwargs) \n", 85 | "参数： \n", 86 | "- pretrained (bool) – True, 返回在ImageNet上训练好的模型。\n", 87 | "\n", 88 | "### ResNet18 模型 \n", 89 | "torchvision.models.resnet18(pretrained=False, ** kwargs) \n", 90 | "参数： \n", 91 | "- pretrained (bool) – True, 返回在ImageNet上训练好的模型。 \n", 92 | "\n", 93 | "### ResNet-34 模型 \n", 94 | "torchvision.models.resnet34(pretrained=False, ** kwargs) \n", 95 | "参数： \n", 96 | "- pretrained (bool) – True, 返回在ImageNet上训练好的模型。\n", 97 | "\n", 98 | "### ResNet-50 模型\n", 99 | "torchvision.models.resnet50(pretrained=False, ** kwargs)\n", 100 | "参数：\n", 101 | "- pretrained (bool) – True, 返回在ImageNet上训练好的模型。\n", 102 | "\n", 103 | "### ResNet-101 模型\n", 104 | "torchvision.models.resnet101(pretrained=False, ** kwargs) \n", 105 | "参数： \n", 106 | "- pretrained (bool) – True, 返回在ImageNet上训练好的模型。\n", 107 | "\n", 108 | "### ResNet-152 模型\n", 109 | "torchvision.models.resnet152(pretrained=False, ** kwargs) \n", 110 | "参数： \n", 111 | "- pretrained (bool) – True, 返回在ImageNet上训练好的模型。\n", 112 | "\n", 113 | "### VGG 11-layer 模型\n", 114 | "torchvision.models.vgg11(pretrained=False, ** kwargs) \n", 115 | "VGG 11-layer model (configuration “A”) \n", 116 | "参数： \n", 117 | "- pretrained (bool) – True, 返回在ImageNet上训练好的模型。\n", 118 | "\n", 119 | "### BN VGG 11-layer 模型\n", 120 | "torchvision.models.vgg11_bn(** kwargs) \n", 121 | "VGG 11-layer model (configuration “A”) with batch normalization\n", 122 | "\n", 123 | "### VGG 13-layer 模型\n", 124 | "torchvision.models.vgg13(pretrained=False, ** kwargs) \n", 125 | "VGG 13-layer model (configuration “B”) \n", 126 | "参数： \n", 127 | "- pretrained (bool) – True, 返回在ImageNet上训练好的模型。\n", 128 | "\n", 129 | "### BN VGG 13-layer 模型\n", 130 | "torchvision.models.vgg13_bn(** kwargs) \n", 131 | "VGG 13-layer model (configuration “B”) with batch normalization\n", 132 | "\n", 133 | "### VGG 16-layer 模型\n", 134 | "torchvision.models.vgg16(pretrained=False, ** kwargs) \n", 135 | "VGG 16-layer model (configuration “D”) \n", 136 | "参数： \n", 137 | "- pretrained (bool) – If True, returns a model pre-trained on ImageNet\n", 138 | "\n", 139 | "### BN VGG 16-layer 模型\n", 140 | "torchvision.models.vgg16_bn(** kwargs) \n", 141 | "VGG 16-layer model (configuration “D”) with batch normalization\n", 142 | "\n", 143 | "### VGG 19-layer 模型\n", 144 | "torchvision.models.vgg19(pretrained=False, ** kwargs) \n", 145 | "VGG 19-layer model (configuration “E”) \n", 146 | "参数： \n", 147 | "- pretrained (bool) – True, 返回在ImageNet上训练好的模型。\n", 148 | "\n", 149 | "### BN VGG 19-layer 模型\n", 150 | "torchvision.models.vgg19_bn(** kwargs) \n", 151 | "VGG 19-layer model (configuration ‘E’) with batch normalizatio" 152 | ] 153 | }, 154 | { 155 | "cell_type": "code", 156 | "execution_count": null, 157 | "metadata": {}, 158 | "outputs": [], 159 | "source": [] 160 | } 161 | ], 162 | "metadata": { 163 | "kernelspec": { 164 | "display_name": "Python 3", 165 | "language": "python", 166 | "name": "python3" 167 | }, 168 | "language_info": { 169 | "codemirror_mode": { 170 | "name": "ipython", 171 | "version": 3 172 | }, 173 | "file_extension": ".py", 174 | "mimetype": "text/x-python", 175 | "name": "python", 176 | "nbconvert_exporter": "python", 177 | "pygments_lexer": "ipython3", 178 | "version": "3.7.2" 179 | }, 180 | "toc": { 181 | "base_numbering": 1, 182 | "nav_menu": {}, 183 | "number_sections": false, 184 | "sideBar": true, 185 | "skip_h1_title": false, 186 | "title_cell": "Table of Contents", 187 | "title_sidebar": "Contents", 188 | "toc_cell": false, 189 | "toc_position": {}, 190 | "toc_section_display": true, 191 | "toc_window_display": false 192 | } 193 | }, 194 | "nbformat": 4, 195 | "nbformat_minor": 2 196 | } 197 | -------------------------------------------------------------------------------- /src/Week2/Day09~Day10/20.torchvision.transforms.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# torchvision transform\n", 8 | "对PIL.Image进行变换 \n", 9 | "class torchvision.transforms.Compose(transforms) \n", 10 | "将多个transform组合起来使用。 \n", 11 | "参数：\n", 12 | "- transforms： 由transform构成的列表. \n", 13 | "\n", 14 | "例子： \n", 15 | "```python\n", 16 | "transforms.Compose([\n", 17 | " transforms.CenterCrop(10),\n", 18 | " transforms.ToTensor(),\n", 19 | " ])\n", 20 | "```" 21 | ] 22 | }, 23 | { 24 | "cell_type": "markdown", 25 | "metadata": {}, 26 | "source": [ 27 | "## torchvision.transforms.Scale\n", 28 | "class torchvision.transforms.Scale(size, interpolation=2) \n", 29 | "将输入的 PIL.Image 重新改变大小成给定的 size，size 是最小边的边长。举个例子，如果原图的 height>width ，那么改变大小后的图片大小`(size*height/width, size)`。\n", 30 | "**用例:** \n", 31 | "```python\n", 32 | "from torchvision import transforms\n", 33 | "from PIL import Image\n", 34 | "crop = transforms.Scale(12)\n", 35 | "img = Image.open('test.jpg')\n", 36 | "\n", 37 | "print(type(img))\n", 38 | "print(img.size)\n", 39 | "\n", 40 | "croped_img=crop(img)\n", 41 | "print(type(croped_img))\n", 42 | "print(croped_img.size)\n", 43 | "```\n", 44 | " \n", 45 | "(10, 10) \n", 46 | " \n", 47 | "(12, 12) " 48 | ] 49 | }, 50 | { 51 | "cell_type": "markdown", 52 | "metadata": {}, 53 | "source": [ 54 | "## torchvision.transforms.CenterCrop\n", 55 | "class torchvision.transforms.CenterCrop(size) \n", 56 | "将给定的PIL.Image进行中心切割，得到给定的size，size可以是tuple，(target_height, target_width)。size也可以是一个Integer，在这种情况下，切出来的图片的形状是正方形。\n", 57 | "\n", 58 | "## torchvision.transforms.RandomCrop\n", 59 | "class torchvision.transforms.RandomCrop(size, padding=0) \n", 60 | "切割中心点的位置随机选取。size可以是tuple也可以是Integer。\n", 61 | "\n", 62 | "## torchvision.transforms.RandomHorizontalFlip\n", 63 | "class torchvision.transforms.RandomHorizontalFlip \n", 64 | "随机水平翻转给定的PIL.Image,概率为0.5。即：一半的概率翻转，一半的概率不翻转。 \n", 65 | "\n", 66 | "## torchvision.transforms.RandomSizedCrop\n", 67 | "class torchvision.transforms.RandomSizedCrop(size, interpolation=2) \n", 68 | "先将给定的PIL.Image随机切，然后再resize成给定的size大小。" 69 | ] 70 | }, 71 | { 72 | "cell_type": "markdown", 73 | "metadata": {}, 74 | "source": [ 75 | "## torchvision.transforms.Pad\n", 76 | "class torchvision.transforms.Pad(padding, fill=0) \n", 77 | "将给定的PIL.Image的所有边用给定的pad value填充。 padding：要填充多少像素 fill：用什么值填充 \n", 78 | "\n", 79 | "例子：\n", 80 | "```\n", 81 | "from torchvision import transforms\n", 82 | "from PIL import Image\n", 83 | "padding_img = transforms.Pad(padding=10, fill=0)\n", 84 | "img = Image.open('test.jpg')\n", 85 | "\n", 86 | "print(type(img))\n", 87 | "print(img.size)\n", 88 | "\n", 89 | "padded_img=padding(img)\n", 90 | "print(type(padded_img))\n", 91 | "print(padded_img.size)\n", 92 | "```\n", 93 | " \n", 94 | "(10, 10) \n", 95 | " \n", 96 | "(30, 30) #由于上下左右都要填充10个像素，所以填充后的size是(30,30) " 97 | ] 98 | }, 99 | { 100 | "cell_type": "markdown", 101 | "metadata": {}, 102 | "source": [ 103 | "## 对Tensor进行变换\n", 104 | "class torchvision.transforms.Normalize(mean, std) \n", 105 | "给定均值：(R,G,B) 方差：（R，G，B），将会把Tensor正则化。即：Normalized_image=(image-mean)/std。 \n", 106 | "\n", 107 | "## torchvision.transforms.ToTensor\n", 108 | "class torchvision.transforms.ToTensor \n", 109 | "把一个取值范围是[0,255]的PIL.Image或者shape为(H,W,C)的numpy.ndarray，转换成形状为[C,H,W]，取值范围是[0,1.0]的torch.FloadTensor" 110 | ] 111 | }, 112 | { 113 | "cell_type": "code", 114 | "execution_count": 1, 115 | "metadata": {}, 116 | "outputs": [ 117 | { 118 | "name": "stdout", 119 | "output_type": "stream", 120 | "text": [ 121 | "(10, 10, 3)\n", 122 | "tensor([[[214, 131, 237, 29, 229, 94, 13, 98, 213, 138],\n", 123 | " [119, 233, 39, 139, 163, 253, 220, 121, 39, 26],\n", 124 | " [ 9, 152, 131, 220, 209, 221, 239, 9, 134, 130],\n", 125 | " [177, 106, 192, 145, 63, 138, 18, 124, 104, 119],\n", 126 | " [ 51, 166, 224, 47, 25, 65, 128, 99, 11, 247],\n", 127 | " [137, 101, 17, 242, 13, 83, 118, 112, 19, 253],\n", 128 | " [202, 93, 171, 5, 223, 96, 170, 14, 192, 135],\n", 129 | " [ 41, 153, 216, 239, 81, 56, 227, 41, 157, 21],\n", 130 | " [147, 4, 119, 152, 175, 224, 219, 106, 168, 158],\n", 131 | " [214, 173, 172, 192, 182, 160, 211, 88, 41, 152]],\n", 132 | "\n", 133 | " [[155, 138, 50, 179, 202, 54, 9, 21, 97, 174],\n", 134 | " [152, 27, 246, 54, 95, 132, 30, 165, 80, 27],\n", 135 | " [180, 84, 107, 60, 32, 131, 60, 108, 47, 42],\n", 136 | " [160, 27, 28, 114, 55, 130, 221, 80, 39, 47],\n", 137 | " [139, 61, 66, 181, 182, 121, 58, 50, 67, 211],\n", 138 | " [ 79, 234, 195, 153, 179, 94, 197, 235, 61, 5],\n", 139 | " [221, 187, 103, 8, 39, 227, 133, 227, 235, 25],\n", 140 | " [ 61, 55, 193, 58, 254, 50, 45, 14, 135, 123],\n", 141 | " [216, 177, 31, 185, 94, 178, 87, 198, 122, 169],\n", 142 | " [195, 149, 235, 119, 6, 58, 65, 247, 108, 106]],\n", 143 | "\n", 144 | " [[ 83, 112, 26, 247, 63, 222, 115, 141, 52, 249],\n", 145 | " [ 83, 67, 155, 190, 62, 48, 229, 40, 126, 247],\n", 146 | " [ 51, 216, 100, 102, 225, 237, 169, 176, 165, 112],\n", 147 | " [ 23, 20, 159, 43, 116, 131, 238, 222, 176, 168],\n", 148 | " [210, 106, 51, 30, 243, 8, 248, 209, 120, 105],\n", 149 | " [183, 99, 82, 142, 199, 46, 15, 26, 241, 14],\n", 150 | " [ 71, 19, 204, 149, 235, 59, 249, 158, 85, 126],\n", 151 | " [ 5, 68, 21, 159, 225, 24, 105, 78, 43, 253],\n", 152 | " [156, 194, 233, 0, 24, 220, 230, 66, 219, 202],\n", 153 | " [ 34, 60, 149, 192, 82, 192, 157, 148, 9, 69]]],\n", 154 | " dtype=torch.int32)\n" 155 | ] 156 | } 157 | ], 158 | "source": [ 159 | "import numpy as np\n", 160 | "import torchvision\n", 161 | "\n", 162 | "data = np.random.randint(0, 255, size=300)\n", 163 | "img = data.reshape(10,10,3)\n", 164 | "print(img.shape)\n", 165 | "img_tensor = torchvision.transforms.ToTensor()(img) # 转换成tensor\n", 166 | "print(img_tensor)" 167 | ] 168 | }, 169 | { 170 | "cell_type": "markdown", 171 | "metadata": {}, 172 | "source": [ 173 | "## torchvision.transforms.ToPILImage\n", 174 | "class torchvision.transforms.ToPILImage \n", 175 | "将shape为(C,H,W)的Tensor或shape为(H,W,C)的numpy.ndarray转换成PIL.Image，值不变。" 176 | ] 177 | }, 178 | { 179 | "cell_type": "markdown", 180 | "metadata": {}, 181 | "source": [ 182 | "**通用变换**\n", 183 | "## torchvision.transforms.Lambda\n", 184 | "class torchvision.transforms.Lambda(lambd) \n", 185 | "使用lambd作为转换器。 \n", 186 | "具体详情见余霆嵩老师的教程，比较全面" 187 | ] 188 | } 189 | ], 190 | "metadata": { 191 | "kernelspec": { 192 | "display_name": "Python 3", 193 | "language": "python", 194 | "name": "python3" 195 | }, 196 | "language_info": { 197 | "codemirror_mode": { 198 | "name": "ipython", 199 | "version": 3 200 | }, 201 | "file_extension": ".py", 202 | "mimetype": "text/x-python", 203 | "name": "python", 204 | "nbconvert_exporter": "python", 205 | "pygments_lexer": "ipython3", 206 | "version": "3.7.2" 207 | }, 208 | "toc": { 209 | "base_numbering": 1, 210 | "nav_menu": {}, 211 | "number_sections": true, 212 | "sideBar": true, 213 | "skip_h1_title": false, 214 | "title_cell": "Table of Contents", 215 | "title_sidebar": "Contents", 216 | "toc_cell": false, 217 | "toc_position": {}, 218 | "toc_section_display": true, 219 | "toc_window_display": true 220 | } 221 | }, 222 | "nbformat": 4, 223 | "nbformat_minor": 2 224 | } 225 | -------------------------------------------------------------------------------- /src/Week2/Day09~Day10/21.torchvision.utils.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# torchvision.utils\n", 8 | "## torchvision.utils.make_grid\n", 9 | "torchvision.utils.make_grid(tensor, nrow=8, padding=2, normalize=False, range=None, scale_each=False) \n", 10 | "猜测，用来做雪碧图的（sprite image）。 \n", 11 | "给定 4D mini-batch Tensor， 形状为$B \\times C \\times H \\times W$,或者一个a list of image，做成一个size为(B / nrow, nrow)的雪碧图。 \n", 12 | "参数： \n", 13 | "- normalize=True ，会将图片的像素值归一化处理\n", 14 | "- range - 如果 range=(min, max)， min和max是数字，那么min，max用来规范化image\n", 15 | "- scale_each=True ，每个图片独立规范化，而不是根据所有图片的像素最大最小值来规范化\n", 16 | "\n", 17 | "## torchvision.utils.save_image\n", 18 | "torchvision.utils.save_image(tensor, filename, nrow=8, padding=2, normalize=False, range=None, scale_each=False) \n", 19 | "将给定的Tensor保存成image文件。如果给定的是mini-batch tensor，那就用make-grid做成雪碧图，再保存。" 20 | ] 21 | }, 22 | { 23 | "cell_type": "code", 24 | "execution_count": null, 25 | "metadata": {}, 26 | "outputs": [ 27 | { 28 | "name": "stdout", 29 | "output_type": "stream", 30 | "text": [ 31 | "Populating the interactive namespace from numpy and matplotlib\n", 32 | "Downloading https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz to .//train//data\\cifar-10-python.tar.gz\n" 33 | ] 34 | }, 35 | { 36 | "name": "stderr", 37 | "output_type": "stream", 38 | "text": [ 39 | "D:\\MyPythonWork\\Pytorch-Camp\\venv\\lib\\site-packages\\torchvision\\transforms\\transforms.py:208: UserWarning: The use of the transforms.Scale transform is deprecated, please use transforms.Resize instead.\n", 40 | " warnings.warn(\"The use of the transforms.Scale transform is deprecated, \" +\n", 41 | "12.2%" 42 | ] 43 | } 44 | ], 45 | "source": [ 46 | "import torch\n", 47 | "import torchvision.datasets as dset\n", 48 | "import torchvision.transforms as transforms\n", 49 | "import matplotlib.pyplot as plt\n", 50 | "import torch.nn as nn\n", 51 | "import torchvision\n", 52 | "import numpy as np\n", 53 | "%pylab inline\n", 54 | "dataset = dset.CIFAR10(root='.//train//data', download=True, \n", 55 | " transform=transforms.Compose([\n", 56 | " transforms.Resize(200),\n", 57 | " transforms.ToTensor(),\n", 58 | " transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)),\n", 59 | " \n", 60 | " ]))\n", 61 | "img = torchvision.utils.make_grid(dataset[1][0]).numpy()\n", 62 | "plt.imshow(np.transpose(img,(1,2,0)))\n", 63 | "plt.show()" 64 | ] 65 | }, 66 | { 67 | "cell_type": "code", 68 | "execution_count": null, 69 | "metadata": {}, 70 | "outputs": [], 71 | "source": [] 72 | } 73 | ], 74 | "metadata": { 75 | "kernelspec": { 76 | "display_name": "Python 3", 77 | "language": "python", 78 | "name": "python3" 79 | }, 80 | "language_info": { 81 | "codemirror_mode": { 82 | "name": "ipython", 83 | "version": 3 84 | }, 85 | "file_extension": ".py", 86 | "mimetype": "text/x-python", 87 | "name": "python", 88 | "nbconvert_exporter": "python", 89 | "pygments_lexer": "ipython3", 90 | "version": "3.7.2" 91 | }, 92 | "toc": { 93 | "base_numbering": 1, 94 | "nav_menu": {}, 95 | "number_sections": true, 96 | "sideBar": true, 97 | "skip_h1_title": false, 98 | "title_cell": "Table of Contents", 99 | "title_sidebar": "Contents", 100 | "toc_cell": false, 101 | "toc_position": {}, 102 | "toc_section_display": true, 103 | "toc_window_display": false 104 | } 105 | }, 106 | "nbformat": 4, 107 | "nbformat_minor": 2 108 | } 109 | -------------------------------------------------------------------------------- /src/Week2/Day11~Day12/9.torch.nn - PyTorch中文文档.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/src/Week2/Day11~Day12/9.torch.nn - PyTorch中文文档.pdf -------------------------------------------------------------------------------- /src/Week3/Day19/linear.pth: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Relph1119/Pytorch-Camp/e4cf0e2e12350f88f63a6fc00b13de7f329db276/src/Week3/Day19/linear.pth -------------------------------------------------------------------------------- /src/Week5/Day30/LogistRegression.py: -------------------------------------------------------------------------------- 1 | import torch 2 | from torch import nn, optim 3 | from torch.autograd import Variable 4 | from torch.utils.data import DataLoader 5 | from torchvision import transforms 6 | from torchvision import datasets 7 | import time 8 | 9 | # 定义超参数 10 | batch_size = 32 11 | learning_rate = 1e-3 12 | # TODO: num_epoches = 100, 由于单机算力不足，故降至10 13 | num_epoches = 10 14 | 15 | rootPath = '../data' 16 | 17 | # 下载训练集 MNIST 手写数字训练集 18 | train_dataset = datasets.MNIST( 19 | root=rootPath, train=True, transform=transforms.ToTensor(), download=True) 20 | 21 | test_dataset = datasets.MNIST( 22 | root=rootPath, train=False, transform=transforms.ToTensor()) 23 | 24 | train_loader = DataLoader(train_dataset, batch_size=batch_size, shuffle=True) 25 | test_loader = DataLoader(test_dataset, batch_size=batch_size, shuffle=False) 26 | 27 | 28 | # 定义 Logistic Regression 模型 29 | class Logstic_Regression(nn.Module): 30 | def __init__(self, in_dim, n_class): 31 | super(Logstic_Regression, self).__init__() 32 | self.logstic = nn.Linear(in_dim, n_class) 33 | 34 | def forward(self, x): 35 | out = self.logstic(x) 36 | return out 37 | 38 | 39 | model = Logstic_Regression(28 * 28, 10) # 图片大小是28x28 40 | use_gpu = torch.cuda.is_available() # 判断是否有GPU加速 41 | if use_gpu: 42 | print("利用GPU进行模型训练") 43 | model = model.cuda() 44 | # 定义loss和optimizer 45 | criterion = nn.CrossEntropyLoss() 46 | optimizer = optim.SGD(model.parameters(), lr=learning_rate) 47 | 48 | # 开始训练 49 | for epoch in range(num_epoches): 50 | print('*' * 10) 51 | print('epoch {}'.format(epoch + 1)) 52 | since = time.time() 53 | running_loss = 0.0 54 | running_acc = 0.0 55 | for i, data in enumerate(train_loader, 1): 56 | img, label = data 57 | img = img.view(img.size(0), -1) # 将图片展开成 28x28 58 | if use_gpu: 59 | img = Variable(img).cuda() 60 | label = Variable(label).cuda() 61 | else: 62 | img = Variable(img) 63 | label = Variable(label) 64 | # 向前传播 65 | out = model(img) 66 | loss = criterion(out, label) 67 | running_loss += loss.item() * label.size(0) 68 | _, pred = torch.max(out, 1) 69 | num_correct = (pred == label).sum() 70 | running_acc += num_correct.item() 71 | # 向后传播 72 | optimizer.zero_grad() 73 | loss.backward() 74 | optimizer.step() 75 | 76 | if i % 300 == 0: 77 | print('[{}/{}] Loss: {:.6f}, Acc: {:.6f}'.format( 78 | epoch + 1, num_epoches, running_loss / (batch_size * i), 79 | running_acc / (batch_size * i))) 80 | print('Finish {} epoch, Loss: {:.6f}, Acc: {:.6f}'.format( 81 | epoch + 1, running_loss / (len(train_dataset)), running_acc / (len( 82 | train_dataset)))) 83 | model.eval() 84 | eval_loss = 0. 85 | eval_acc = 0. 86 | for data in test_loader: 87 | img, label = data 88 | img = img.view(img.size(0), -1) 89 | if use_gpu: 90 | img = img.cuda() 91 | label = label.cuda() 92 | 93 | out = model(img) 94 | loss = criterion(out, label) 95 | eval_loss += loss.item() * label.size(0) 96 | _, pred = torch.max(out, 1) 97 | num_correct = (pred == label).sum() 98 | eval_acc += num_correct.item() 99 | print('Test Loss: {:.6f}, Acc: {:.6f}'.format(eval_loss / (len( 100 | test_dataset)), eval_acc / (len(test_dataset)))) 101 | print('Time:{:.1f} s'.format(time.time() - since)) 102 | print() 103 | 104 | # 保存模型 105 | torch.save(model.state_dict(), './logstic.pth') 106 | -------------------------------------------------------------------------------- /src/Week5/Day31/MLP.py: -------------------------------------------------------------------------------- 1 | import torch.nn as nn 2 | import torch.optim as optim 3 | from torch.utils.data import DataLoader 4 | from torchvision import datasets, transforms 5 | 6 | batch_size = 200 7 | learning_rate = 0.01 8 | epochs = 10 9 | 10 | rootPath = '../data' 11 | 12 | train_loader = DataLoader( 13 | datasets.MNIST(rootPath, train=True, download=True, 14 | transform=transforms.Compose([ 15 | transforms.ToTensor(), 16 | transforms.Normalize((0.1307,), (0.3081,)) 17 | ])), 18 | batch_size=batch_size, shuffle=True) 19 | test_loader = DataLoader( 20 | datasets.MNIST(rootPath, train=False, transform=transforms.Compose([ 21 | transforms.ToTensor(), 22 | transforms.Normalize((0.1307,), (0.3081,)) 23 | ])), 24 | batch_size=batch_size, shuffle=True) 25 | 26 | 27 | class MLP(nn.Module): 28 | 29 | def __init__(self): 30 | super(MLP, self).__init__() 31 | 32 | self.model = nn.Sequential( 33 | nn.Linear(784, 200), 34 | nn.ReLU(inplace=True), 35 | nn.Linear(200, 200), 36 | nn.ReLU(inplace=True), 37 | nn.Linear(200, 10), 38 | nn.ReLU(inplace=True), 39 | ) 40 | 41 | def forward(self, x): 42 | x = self.model(x) 43 | 44 | return x 45 | 46 | 47 | net = MLP() 48 | optimizer = optim.SGD(net.parameters(), lr=learning_rate) 49 | criteon = nn.CrossEntropyLoss() 50 | 51 | for epoch in range(epochs): 52 | 53 | for batch_idx, (data, target) in enumerate(train_loader): 54 | data = data.view(-1, 28 * 28) 55 | 56 | logits = net(data) 57 | loss = criteon(logits, target) 58 | 59 | optimizer.zero_grad() 60 | loss.backward() 61 | # print(w1.grad.norm(), w2.grad.norm()) 62 | optimizer.step() 63 | 64 | if batch_idx % 100 == 0: 65 | print('Train Epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}'.format( 66 | epoch, batch_idx * len(data), len(train_loader.dataset), 67 | 100. * batch_idx / len(train_loader), loss.item())) 68 | 69 | test_loss = 0 70 | correct = 0 71 | for data, target in test_loader: 72 | data = data.view(-1, 28 * 28) 73 | logits = net(data) 74 | test_loss += criteon(logits, target).item() 75 | 76 | pred = logits.data.max(1)[1] 77 | correct += pred.eq(target.data).sum() 78 | 79 | test_loss /= len(test_loader.dataset) 80 | print('\nTest set: Average loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)\n'.format( 81 | test_loss, correct, len(test_loader.dataset), 82 | 100. * correct / len(test_loader.dataset))) 83 | -------------------------------------------------------------------------------- /src/Week5/Day32/CNN-Classification.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import torch 3 | from torch import nn 4 | import torch.nn.functional as F 5 | from torchvision.datasets import CIFAR10 6 | from torch.utils.data import DataLoader 7 | 8 | rootPath = '../data' 9 | 10 | 11 | def conv3x3(in_channel, out_channel, stride=1): 12 | return nn.Conv2d(in_channel, out_channel, 3, stride=stride, padding=1, bias=False) 13 | 14 | 15 | # 实现残差块 16 | class residual_block(nn.Module): 17 | def __init__(self, in_channel, out_channel, same_shape=True): 18 | super(residual_block, self).__init__() 19 | self.same_shape = same_shape 20 | stride = 1 if self.same_shape else 2 21 | 22 | self.conv1 = conv3x3(in_channel, out_channel, stride=stride) 23 | self.bn1 = nn.BatchNorm2d(out_channel) 24 | 25 | self.conv2 = conv3x3(out_channel, out_channel) 26 | self.bn2 = nn.BatchNorm2d(out_channel) 27 | if not self.same_shape: 28 | self.conv3 = nn.Conv2d(in_channel, out_channel, 1, stride=stride) 29 | 30 | def forward(self, x): 31 | out = self.conv1(x) 32 | out = F.relu(self.bn1(out), True) 33 | out = self.conv2(out) 34 | out = F.relu(self.bn2(out), True) 35 | 36 | if not self.same_shape: 37 | x = self.conv3(x) 38 | return F.relu(x + out, True) 39 | 40 | 41 | """ 42 | 测试一下一个 residual block 的输入和输出 43 | """ 44 | # 1.输入输出形状相同 45 | test_net = residual_block(32, 32) 46 | test_x = torch.zeros(1, 32, 96, 96) 47 | print('input: {}'.format(test_x.shape)) 48 | test_y = test_net(test_x) 49 | print('output: {}'.format(test_y.shape)) 50 | 51 | # 2.输入输出形状不同 52 | test_net = residual_block(3, 32, False) 53 | test_x = torch.zeros(1, 3, 96, 96) 54 | print('input: {}'.format(test_x.shape)) 55 | test_y = test_net(test_x) 56 | print('output: {}'.format(test_y.shape)) 57 | 58 | 59 | # 实现一个 ResNet，它就是 residual block 模块的堆叠 60 | class resnet(nn.Module): 61 | def __init__(self, in_channel, num_classes, verbose=False): 62 | super(resnet, self).__init__() 63 | self.verbose = verbose 64 | 65 | self.block1 = nn.Conv2d(in_channel, 64, 7, 2) 66 | 67 | self.block2 = nn.Sequential( 68 | nn.MaxPool2d(3, 2), 69 | residual_block(64, 64), 70 | residual_block(64, 64) 71 | ) 72 | 73 | self.block3 = nn.Sequential( 74 | residual_block(64, 128, False), 75 | residual_block(128, 128) 76 | ) 77 | 78 | self.block4 = nn.Sequential( 79 | residual_block(128, 256, False), 80 | residual_block(256, 256) 81 | ) 82 | 83 | self.block5 = nn.Sequential( 84 | residual_block(256, 512, False), 85 | residual_block(512, 512), 86 | nn.AvgPool2d(3) 87 | ) 88 | 89 | self.classifier = nn.Linear(512, num_classes) 90 | 91 | def forward(self, x): 92 | x = self.block1(x) 93 | if self.verbose: 94 | print('block 1 output: {}'.format(x.shape)) 95 | x = self.block2(x) 96 | if self.verbose: 97 | print('block 2 output: {}'.format(x.shape)) 98 | x = self.block3(x) 99 | if self.verbose: 100 | print('block 3 output: {}'.format(x.shape)) 101 | x = self.block4(x) 102 | if self.verbose: 103 | print('block 4 output: {}'.format(x.shape)) 104 | x = self.block5(x) 105 | if self.verbose: 106 | print('block 5 output: {}'.format(x.shape)) 107 | x = x.view(x.shape[0], -1) 108 | x = self.classifier(x) 109 | return x 110 | 111 | 112 | def data_tf(x): 113 | x = x.resize((96, 96), 2) # 将图片放大到 96 x 96 114 | x = np.array(x, dtype='float32') / 255 115 | x = (x - 0.5) / 0.5 # 标准化，这个技巧之后会讲到 116 | x = x.transpose((2, 0, 1)) # 将 channel 放到第一维，只是 pytorch 要求的输入方式 117 | x = torch.from_numpy(x) 118 | return x 119 | 120 | 121 | train_set = CIFAR10(rootPath, train=True, transform=data_tf, download=True) 122 | train_data = DataLoader(train_set, batch_size=64, shuffle=True) 123 | test_set = CIFAR10(rootPath, train=False, transform=data_tf, download=True) 124 | test_data = DataLoader(test_set, batch_size=128, shuffle=False) 125 | 126 | net = resnet(3, 10) 127 | optimizer = torch.optim.SGD(net.parameters(), lr=0.01) 128 | criterion = nn.CrossEntropyLoss() 129 | 130 | 131 | # 数据处理 132 | def data_tf(x): 133 | x = x.resize((96, 96), 2) # 将图片放大到 96 x 96 134 | x = np.array(x, dtype='float32') / 255 135 | x = (x - 0.5) / 0.5 # 标准化，这个技巧之后会讲到 136 | x = x.transpose((2, 0, 1)) # 将 channel 放到第一维，只是 pytorch 要求的输入方式 137 | x = torch.from_numpy(x) 138 | return x 139 | 140 | 141 | # 读取数据 142 | train_set = CIFAR10(rootPath, train=True, transform=data_tf) 143 | train_data = torch.utils.data.DataLoader(train_set, batch_size=64, shuffle=True) 144 | test_set = CIFAR10(rootPath, train=False, transform=data_tf) 145 | test_data = torch.utils.data.DataLoader(test_set, batch_size=128, shuffle=False) 146 | 147 | net = resnet(3, 10) 148 | # 优化算法 149 | optimizer = torch.optim.SGD(net.parameters(), lr=0.01) 150 | # 损失函数 151 | criterion = nn.CrossEntropyLoss() 152 | epochs = 20 153 | for epoch in range(epochs): 154 | 155 | net.train() 156 | loss_sigma = 0.0 157 | correct = 0.0 158 | total = 0.0 159 | 160 | for i, (data, label) in enumerate(train_data): 161 | if torch.cuda.is_available(): 162 | data = data.cuda() 163 | label = label.cuda() 164 | net = net.cuda() 165 | optimizer.zero_grad() 166 | outputs = net(data) 167 | loss = criterion(outputs, label) 168 | loss.backward() 169 | optimizer.step() 170 | _, predicted = torch.max(outputs.data, dim=1) 171 | total += label.size(0) 172 | correct += (predicted == label).squeeze().sum().cpu().numpy() 173 | loss_sigma += loss.item() 174 | # 每10个iteration 打印一次训练信息，loss为10个iteration的平均 175 | if i % 2 == 0: 176 | loss_avg = loss_sigma / 10 177 | loss_sigma = 0.0 178 | print("Training: Epoch[{:0>3}/{:0>3}] Iteration[{:0>3}/{:0>3}] Loss: {:.4f} Acc:{:.2%}" \ 179 | .format(epoch + 1, epochs, i + 1, len(train_data), loss_avg, correct / total)) 180 | 181 | # ------------------------------------ 观察模型在验证集上的表现 ------------------------------------ 182 | if epoch % 1 == 0: 183 | loss_sigma = 0.0 184 | 185 | net.eval() 186 | for i, (data, label) in enumerate(test_data): 187 | # forward 188 | data = data.to("cuda") 189 | label = label.to("cuda") 190 | outputs = net(data) 191 | outputs.detach_() 192 | # 计算loss 193 | loss = criterion(outputs, label) 194 | loss_sigma += loss.item() 195 | # 统计 196 | _, predicted = torch.max(outputs.data, 1) 197 | total += label.size(0) 198 | correct += (predicted == label).squeeze().sum().cpu().numpy() 199 | loss_sigma += loss.item() 200 | # 每10个iteration 打印一次训练信息，loss为10个iteration的平均 201 | if i % 2 == 0: 202 | loss_avg = loss_sigma / 10 203 | loss_sigma = 0.0 204 | print("test: Epoch[{:0>3}/{:0>3}] Iteration[{:0>3}/{:0>3}] Loss: {:.4f} Acc:{:.2%}" \ 205 | .format(epoch + 1, epochs, i + 1, len(test_data), loss_avg, correct / total)) 206 | 207 | print('Finished Training') 208 | 209 | torch.save(net, 'net.pkl' + str(epochs)) # 保存整个神经网络的结构和模型参数 210 | torch.save(net.state_dict(), 'net_params_BN1_Nopool.pkl' + str(epoch)) # 只保存神经网络的模型参数 211 | -------------------------------------------------------------------------------- /src/Week5/Day33/RNN-LSTM.py: -------------------------------------------------------------------------------- 1 | # 词性预测 2 | import torch 3 | from torch import nn 4 | 5 | # 训练数据 6 | training_data = [("The dog ate the apple".split(), 7 | ["DET", "NN", "V", "DET", "NN"]), 8 | ("Everybody read that book".split(), 9 | ["NN", "V", "DET", "NN"])] 10 | 11 | # 接下来我们需要对单词和标签进行编码 12 | word_to_idx = {} 13 | tag_to_idx = {} 14 | for context, tag in training_data: 15 | for word in context: 16 | if word.lower() not in word_to_idx: 17 | word_to_idx[word.lower()] = len(word_to_idx) 18 | for label in tag: 19 | if label.lower() not in tag_to_idx: 20 | tag_to_idx[label.lower()] = len(tag_to_idx) 21 | 22 | # 可以查看word_to_idx和tag_to_idx对应的数 23 | print(word_to_idx) 24 | print(tag_to_idx) 25 | 26 | # 然后我们对字母进行编码 27 | alphabet = 'abcdefghijklmnopqrstuvwxyz' 28 | char_to_idx = {} 29 | for i in range(len(alphabet)): 30 | char_to_idx[alphabet[i]] = i 31 | # 查看字母对应的数字 32 | print(char_to_idx) 33 | 34 | 35 | # 构建训练数据 36 | def make_sequence(x, dic): # 字符编码 37 | idx = [dic[i.lower()] for i in x] 38 | idx = torch.LongTensor(idx) 39 | return idx 40 | 41 | 42 | # 构建单个字符的 lstm 模型 43 | class char_lstm(nn.Module): 44 | def __init__(self, n_char, char_dim, char_hidden): 45 | super(char_lstm, self).__init__() 46 | 47 | self.char_embed = nn.Embedding(n_char, char_dim) 48 | self.lstm = nn.LSTM(char_dim, char_hidden) 49 | 50 | def forward(self, x): 51 | x = self.char_embed(x) 52 | out, _ = self.lstm(x) 53 | return out[-1] # (batch, hidden) 54 | 55 | 56 | # 构建词性分类的 lstm 模型 57 | class lstm_tagger(nn.Module): 58 | def __init__(self, n_word, n_char, char_dim, word_dim, 59 | char_hidden, word_hidden, n_tag): 60 | super(lstm_tagger, self).__init__() 61 | self.word_embed = nn.Embedding(n_word, word_dim) 62 | self.char_lstm = char_lstm(n_char, char_dim, char_hidden) 63 | self.word_lstm = nn.LSTM(word_dim + char_hidden, word_hidden) 64 | self.classify = nn.Linear(word_hidden, n_tag) 65 | 66 | def forward(self, x, word): 67 | char = [] 68 | for w in word: # 对于每个单词做字符的 lstm 69 | char_list = make_sequence(w, char_to_idx) 70 | char_list = char_list.unsqueeze(1) # (seq, batch, feature) 满足 lstm 输入条件 71 | char_infor = self.char_lstm(char_list) # (batch, char_hidden) 72 | char.append(char_infor) 73 | char = torch.stack(char, dim=0) # (seq, batch, feature) 74 | 75 | x = self.word_embed(x) # (batch, seq, word_dim) 76 | x = x.permute(1, 0, 2) # 改变顺序 77 | x = torch.cat((x, char), dim=2) # 沿着特征通道将每个词的词嵌入和字符 lstm 输出的结果拼接在一起 78 | x, _ = self.word_lstm(x) 79 | 80 | s, b, h = x.shape 81 | x = x.view(-1, h) # 重新 reshape 进行分类线性层 82 | out = self.classify(x) 83 | return out 84 | 85 | 86 | net = lstm_tagger(len(word_to_idx), len(char_to_idx), 10, 100, 50, 128, len(tag_to_idx)) 87 | criterion = nn.CrossEntropyLoss() 88 | optimizer = torch.optim.SGD(net.parameters(), lr=1e-2) 89 | # 开始训练 90 | for e in range(300): 91 | train_loss = 0 92 | for word, tag in training_data: 93 | word_list = make_sequence(word, word_to_idx).unsqueeze(0) # 添加第一维 batch 94 | tag = make_sequence(tag, tag_to_idx) 95 | 96 | # 前向传播 97 | out = net(word_list, word) 98 | loss = criterion(out, tag) 99 | train_loss += loss.item() 100 | # 反向传播 101 | optimizer.zero_grad() 102 | loss.backward() 103 | optimizer.step() 104 | if (e + 1) % 50 == 0: 105 | print('Epoch: {}, Loss: {:.5f}'.format(e + 1, train_loss / len(training_data))) 106 | 107 | # 看看预测的结果 108 | net = net.eval() 109 | test_sent = 'Everybody ate the apple' 110 | test = make_sequence(test_sent.split(), word_to_idx).unsqueeze(0) 111 | out = net(test, test_sent.split()) 112 | print('out=', out) 113 | print('tag_to_idx=', tag_to_idx) 114 | 115 | """ 116 | out= tensor([[-1.2255, 1.6552, -0.3672], 117 | [-0.9423, -0.1564, 1.2740], 118 | [ 1.6619, -0.7175, -0.7306], 119 | [-0.5624, 1.7271, -0.9589]], grad_fn=) 120 | tag_to_idx= {'det': 0, 'nn': 1, 'v': 2} 121 | 122 | 最后可以得到上面的结果，因为最后一层的线性层没有使用 softmax，所以数值不太像一个概率， 123 | 但是每一行数值最大的就表示属于该类，可以看到第一个单词 'Everybody' 属于 nn， 124 | 第二个单词 'ate' 属于 v，第三个单词 'the' 属于det，第四个单词 'apple' 属于 nn，所以得到的这个预测结果是正确的 125 | """ 126 | --------------------------------------------------------------------------------