├── images └── resnet101_deepNetworkDesigner.png ├── resnet101Example.m ├── LICENSE.txt ├── readme.md ├── assembleResNet101.m └── resnet101Layers.m /images/resnet101_deepNetworkDesigner.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/matlab-deep-learning/resnet-101/HEAD/images/resnet101_deepNetworkDesigner.png -------------------------------------------------------------------------------- /resnet101Example.m: -------------------------------------------------------------------------------- 1 | %% Classify Image Using ResNet-101 2 | % This example shows how to classify an image using the ResNet-101 3 | % pretrained convolutional neural network. 4 | 5 | % Copyright 2019 The MathWorks, Inc. 6 | 7 | % Read an example image. 8 | img = imread("peppers.png"); 9 | 10 | % The image that you want to classify must have the same size as the input 11 | % size of the network. Resize the image to be 224-by-224 pixels, the input 12 | % size of ResNet-101. 13 | img = imresize(img,[224 224]); 14 | 15 | % Assemble the pretrained ResNet-101 network. Alternatively, you can create 16 | % a pretrained ResNet-101 network by installing the Deep Learning Toolbox 17 | % Model for ResNet-101 Network support package from the Add-On Explorer 18 | % using the resnet101 function. 19 | net = assembleResNet101; 20 | 21 | % Analyze the network architecture. 22 | analyzeNetwork(net) 23 | 24 | % Classify the image using the network. 25 | label = classify(net,img); 26 | 27 | % Display the image together with the predicted label. 28 | figure 29 | imshow(img) 30 | title(string(label)) -------------------------------------------------------------------------------- /LICENSE.txt: -------------------------------------------------------------------------------- 1 | Copyright (c) 2019, The MathWorks, Inc. 2 | 3 | Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 4 | 5 | 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 6 | 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 7 | 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. 8 | 9 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. -------------------------------------------------------------------------------- /readme.md: -------------------------------------------------------------------------------- 1 | # Overview 2 | 3 | ResNet-101 is a convolutional neural network that is trained on more than a million images from the ImageNet database. As a result, the network has learned rich feature representations for a wide range of images. The network can classify images into 1000 object categories, such as keyboard, mouse, pencil, and many animals. 4 | 5 | The network has an image input size of 224-by-224-by-3. 6 | 7 | # Usage 8 | 9 | This repository requires [MATLAB](https://www.mathworks.com/products/matlab.html) (R2018b and above) and the [Deep Learning Toolbox](https://www.mathworks.com/products/deep-learning.html). 10 | 11 | This repository provides three functions: 12 | - resnet101Layers: Creates an untrained network with the network architecture of ResNet-101 13 | - assembleResNet101: Creates a ResNet-101 network with weights trained on ImageNet data 14 | - resnet101Example: Demonstrates how to classify an image using a trained ResNet-101 network 15 | 16 | To construct an untrained ResNet-101 network to train from scratch, type the following at the MATLAB command line: 17 | ```matlab 18 | lgraph = resnet101Layers; 19 | ``` 20 | The untrained network is returned as a `layerGraph` object. 21 | 22 | To construct a trained ResNet-101 network suitable for use in image classification, type the following at the MATLAB command line: 23 | ```matlab 24 | net = assembleResNet101; 25 | ``` 26 | The trained network is returned as a `DAGNetwork` object. 27 | 28 | To classify an image with the network: 29 | ```matlab 30 | img = imresize(imread("peppers.png"),[224 224]); 31 | predLabel = classify(net, img); 32 | imshow(img); 33 | title(string(predLabel)); 34 | ``` 35 | 36 | # Documentation 37 | 38 | For more information about the ResNet-101 pre-trained model, see the [resnet101](https://www.mathworks.com/help/deeplearning/ref/resnet101.html) function page in the [MATLAB Deep Learning Toolbox documentation](https://www.mathworks.com/help/deeplearning/index.html). 39 | 40 | # Architecture 41 | 42 | ResNet-101 is a residual network. A residual network is a type of DAG network that has residual (or shortcut) connections that bypass the main network layers. Residual connections enable the parameter gradients to propagate more easily from the output layer to the earlier layers of the network, which makes it possible to train deeper networks. This increased network depth can result in higher accuracies on more difficult tasks. 43 | 44 | The network is 101 layers deep. The network depth is defined as the largest number of sequential convolutional or fully connected layers on a path from the input layer to the output layer. In total, ResNet-101 has 347 layers. 45 | 46 | You can explore and edit the network architecture using [Deep Network Designer](https://www.mathworks.com/help/deeplearning/ug/build-networks-with-deep-network-designer.html). 47 | 48 | ![ResNet-101 in Deep Network Designer](images/resnet101_deepNetworkDesigner.png "ResNet-101 in Deep Network Designer") 49 | 50 | # ResNet-101 in MATLAB 51 | 52 | This repository demonstrates the construction of a residual deep neural network from scratch in MATLAB. You can use the code in this repository as a foundation for building residual networks with different numbers of residual blocks. 53 | 54 | You can also create a trained ResNet-101 network from inside MATLAB by installing the Deep Learning Toolbox Model for ResNet-101 Network support package. Type `resnet101` at the command line. If the Deep Learning Toolbox Model for ResNet-101 Network support package is not installed, then the function provides a link to the required support package in the Add-On Explorer. To install the support package, click the link, and then click Install. 55 | 56 | Alternatively, you can download the ResNet-101 pre-trained model from the MathWorks File Exchange, at [Deep Learning Toolbox Model for ResNet-101 Network](https://www.mathworks.com/matlabcentral/fileexchange/65678-deep-learning-toolbox-model-for-resnet-101-network). 57 | 58 | You can create an untrained ResNet-101 network from inside MATLAB by importing a trained ResNet-101 network into the Deep Network Designer App and selecting Export > Generate Code. The exported code will generate an untrained network with the network architecture of ResNet-101. 59 | -------------------------------------------------------------------------------- /assembleResNet101.m: -------------------------------------------------------------------------------- 1 | function net = assembleResNet101() 2 | % assembleResNet101 Assemble ResNet-101 network 3 | % 4 | % net = assembleResNet101 creates a ResNet-101 network with weights trained 5 | % on ImageNet. You can load the same ResNet-101 network by installing the 6 | % Deep Learning Toolbox Model for ResNet-101 Network support package from 7 | % the Add-On Explorer and then using the resnet101 function. 8 | 9 | % Copyright 2019 The MathWorks, Inc. 10 | 11 | % Download the network parameters. If these have already been downloaded, 12 | % this step will be skipped. 13 | % 14 | % The files will be downloaded to a file "resnet101Params.mat", in a 15 | % directory "ResNet101" located in the system's temporary directory. 16 | dataDir = fullfile(tempdir, "ResNet101"); 17 | paramFile = fullfile(dataDir, "resnet101Params.mat"); 18 | downloadUrl = "http://www.mathworks.com/supportfiles/nnet/data/networks/resnet101Params.mat"; 19 | 20 | if ~exist(dataDir, "dir") 21 | mkdir(dataDir); 22 | end 23 | 24 | if ~exist(paramFile, "file") 25 | disp("Downloading pretrained parameters file (160 MB).") 26 | disp("This may take several minutes..."); 27 | websave(paramFile, downloadUrl); 28 | disp("Download finished."); 29 | else 30 | disp("Skipping download, parameter file already exists."); 31 | end 32 | 33 | % Load the network parameters from the file resNet101Params.mat. 34 | s = load(paramFile); 35 | params = s.params; 36 | paramNames = fields(params); 37 | 38 | % Create a layer graph with the network architecture of ResNet-101. 39 | lgraph = resnet101Layers; 40 | 41 | % Create a cell array containing the layer names. 42 | layerNames = {lgraph.Layers(:).Name}'; 43 | 44 | % Specify the average image of the image input layer. To specify properties 45 | % of layers in the layer graph, you must create a copy of the layer, 46 | % specify the property of the new layer, and then replace the old layer 47 | % with the new one. 48 | layer = lgraph.Layers(1); 49 | layer.AverageImage = params.AverageImage; 50 | lgraph = replaceLayer(lgraph,layerNames{1},layer); 51 | 52 | % Specify the weights of the first convolutional layer. 53 | layer = lgraph.Layers(2); 54 | layer.Weights = params.conv1.weights; 55 | layer.Bias = zeros(1,1,size(params.conv1.weights,4)); 56 | lgraph = replaceLayer(lgraph,layerNames{2},layer ); 57 | 58 | % Specify the weights of the final fully connected layer at the end of the 59 | % network. The last fully connected layer outputs class scores. 60 | layer = lgraph.Layers(end-2); 61 | layer.Weights = params.fc1000.weights; 62 | layer.Bias = params.fc1000.biases; 63 | lgraph = replaceLayer(lgraph,layerNames{end-2},layer); 64 | 65 | % Specify the classes of the classification layer. 66 | layer = classificationLayer("Classes",params.categoryNames,"Name",layerNames{end}); 67 | lgraph = replaceLayer(lgraph,layerNames{end},layer); 68 | 69 | % Specify the parameters of the convolutional and batch normalization 70 | % layers. 71 | learnableLayerNames = intersect(layerNames,paramNames); 72 | for i = 1:numel(learnableLayerNames) 73 | name = learnableLayerNames{i}; 74 | idx = strcmp(layerNames,name); 75 | layer = lgraph.Layers(idx); 76 | 77 | if isa(layer,"nnet.cnn.layer.Convolution2DLayer") 78 | % Convolutional layers have learned Weights parameters. The Bias 79 | % parameters are all zero. 80 | layerParams = params.(name); 81 | layer.Weights = layerParams.weights; 82 | layer.Bias = zeros(1,1,size(layerParams.weights,4)); 83 | 84 | elseif isa(layer,"nnet.cnn.layer.BatchNormalizationLayer") 85 | % Batch normalization layers have TrainedMean and TrainedVariance 86 | % variables and learned Offset and Scale parameters. 87 | trainedVars = params.(name); 88 | layer.TrainedMean = reshape(trainedVars.trainedMean,1,1,[]); 89 | layer.TrainedVariance = reshape(trainedVars.trainedVariance,1,1,[]); 90 | 91 | % Learned Offset and Scale parameters are stored as a vector e.g 92 | % with size 256x1. Reshape to a three-dimensional array e.g. with 93 | % size 1x1x256. 94 | learnedParams = params.(replace(name,"bn","scale")); 95 | layer.Offset = reshape(learnedParams.offset,1,1,[]); 96 | layer.Scale = reshape(learnedParams.scale,1,1,[]); 97 | end 98 | 99 | lgraph = replaceLayer(lgraph,name,layer); 100 | end 101 | 102 | % Assemble the network. 103 | net = assembleNetwork(lgraph); 104 | 105 | end -------------------------------------------------------------------------------- /resnet101Layers.m: -------------------------------------------------------------------------------- 1 | function lgraph = resnet101Layers() 2 | % resnet101Layers ResNet-101 layer graph 3 | % 4 | % lgraph = resnet101Layers creates a layer graph with the network 5 | % architecture of ResNet-101. The layer graph contains no weights. 6 | 7 | % Copyright 2019 The MathWorks, Inc. 8 | 9 | % Create the first five layers of the network. The network has an image 10 | % input size of 224-by-224-by-3. 11 | initialSection = [ 12 | imageInputLayer([224 224 3],"Name","data") 13 | convolution2dLayer(7,64,"Stride",2,"Padding",3,"BiasLearnRateFactor",0,"Name","conv1") 14 | batchNormalizationLayer("Name","bn_conv1") 15 | reluLayer("Name","conv1_relu") 16 | maxPooling2dLayer(3,"Stride",2,"Padding",[0 1 0 1],"Name","pool1")]; 17 | lgraph = layerGraph(initialSection); 18 | 19 | % Use the function addAndConnectResNetSection to create a ResNet section 20 | % and connect it to the end of the layer graph. 21 | lgraph = addAndConnectResNetSection(lgraph, ... 22 | lgraph.Layers(end).Name, ... % Layer to connect to 23 | "res2", ... % Name of ResNet section 24 | 64, ... % Number of filters in the first 1-by-1 convolutions 25 | 64, ... % Number of filters in the 3-by-3 convolutions 26 | 256, ... % Number of filters in the last 1-by-1 convolutions 27 | 1, ... % Stride of the first convolution of the section 28 | {'a','b','c'}); % Names of the residual units 29 | 30 | % Add and connect three more ResNet sections to the network. 31 | lgraph = addAndConnectResNetSection(lgraph, ... 32 | lgraph.Layers(end).Name, ... 33 | "res3", ... 34 | 128, ... 35 | 128, ... 36 | 512, ... 37 | 2, ... 38 | {'a','b1','b2','b3'}); 39 | 40 | lgraph = addAndConnectResNetSection(lgraph, ... 41 | lgraph.Layers(end).Name, ... 42 | "res4", ... 43 | 256, ... 44 | 256, ... 45 | 1024, ... 46 | 2, ... 47 | {'a','b1','b2','b3','b4','b5','b6','b7','b8','b9','b10','b11','b12',... 48 | 'b13','b14','b15','b16','b17','b18','b19','b20','b21','b22'}); 49 | 50 | lgraph = addAndConnectResNetSection(lgraph, ... 51 | lgraph.Layers(end).Name, ... 52 | "res5", ... 53 | 512, ... 54 | 512, ... 55 | 2048, ... 56 | 2, ... 57 | {'a','b','c'}); 58 | 59 | % Create, add, and connect the final ResNet section of the network. 60 | finalSection = [ 61 | averagePooling2dLayer(7,"Stride",7,"Name","pool5") 62 | fullyConnectedLayer(1000,"Name","fc1000") 63 | softmaxLayer("Name","prob") 64 | classificationLayer("Name","ClassificationLayer_predictions")]; 65 | 66 | lastLayerName = lgraph.Layers(end).Name; 67 | lgraph = addLayers(lgraph,finalSection); 68 | lgraph = connectLayers(lgraph,lastLayerName,"pool5"); 69 | 70 | end 71 | 72 | 73 | function lgraph = addAndConnectResNetSection( ... 74 | lgraph, ... 75 | layerToConnectFrom, ... 76 | sectionName, ... 77 | numF1x1first, ... 78 | numF3x3, ... 79 | numF1x1last, ... 80 | firstStride, ... 81 | residualUnits) 82 | % addAndConnectResNetSection Creates a ResNet section and connects it to 83 | % the specified layer of a layer graph. 84 | % 85 | % This function connects a new ResNet section to the layer 86 | % layerToConnectFrom in the layer graph lgraph. 87 | % 88 | % sectionName is the name of the new network section. 89 | % 90 | % numF1x1first is the number of filters in the first 1-by-1 convolution of 91 | % the residual units of the section. 92 | % 93 | % numF3x3 is the number of filters in the 3-by-3 convolution of the 94 | % residual units of the section. 95 | % 96 | % numF1x1last is the number of filters in the last 1-by-1 convolution of 97 | % the residual units of the section. 98 | % 99 | % firstStride is the stride of the first convolution of the section. 100 | % 101 | % residualUnits contains the names of the residual units of the section. 102 | % This argument determines the number of residual units of the 103 | % section. 104 | stride = firstStride; 105 | for i = 1:length(residualUnits) 106 | layerRoot = sectionName+residualUnits{i}; 107 | bnRoot = "bn"+extractAfter(sectionName,"res")+residualUnits{i}; 108 | 109 | % Create the block of layers for the residual unit and add it to the 110 | % layer graph. 111 | resnetBlock = [ 112 | convolution2dLayer(1,numF1x1first,"Stride",stride,"Name",layerRoot+"_branch2a","BiasLearnRateFactor",0) 113 | batchNormalizationLayer("Name",bnRoot+"_branch2a") 114 | reluLayer("Name",layerRoot+"_branch2a_relu") 115 | 116 | convolution2dLayer(3,numF3x3,"Padding",1,"Name",layerRoot+"_branch2b","BiasLearnRateFactor",0); 117 | batchNormalizationLayer("Name",bnRoot+"_branch2b") 118 | reluLayer("Name",layerRoot+"_branch2b_relu") 119 | 120 | convolution2dLayer(1,numF1x1last,"Name",layerRoot+"_branch2c","BiasLearnRateFactor",0) 121 | batchNormalizationLayer("Name",bnRoot+"_branch2c") 122 | 123 | additionLayer(2,"Name",layerRoot) 124 | reluLayer("Name",layerRoot+"_relu")]; 125 | 126 | lgraph = addLayers(lgraph,resnetBlock); 127 | 128 | % For the first residual unit of the section, add a residual connection 129 | % with projection layers and connect to the previous section. 130 | % Projection layers contain a 1-by-1 convolutional layer that projects 131 | % the input into a space with a larger number of channels. This 132 | % projection is necessary because the activations in different sections 133 | % of the network have a different number of channels. For the other 134 | % residual units, connect to the previous residual unit. 135 | if i == 1 136 | projectionLayers = [ 137 | convolution2dLayer(1,numF1x1last,"Stride",stride,"Name",layerRoot+"_branch1","BiasLearnRateFactor",0) 138 | batchNormalizationLayer("Name",bnRoot+"_branch1")]; 139 | lgraph = addLayers(lgraph,projectionLayers); 140 | 141 | lgraph = connectLayers(lgraph,layerToConnectFrom,layerRoot+"_branch2a"); 142 | lgraph = connectLayers(lgraph,layerToConnectFrom,layerRoot+"_branch1"); 143 | lgraph = connectLayers(lgraph,bnRoot+"_branch1",layerRoot+"/in2"); 144 | else 145 | lgraph = connectLayers(lgraph,sectionName+residualUnits{i-1}+"_relu",sectionName+residualUnits{i}+"_branch2a"); 146 | lgraph = connectLayers(lgraph,sectionName+residualUnits{i-1}+"_relu",sectionName+residualUnits{i}+"/in2"); 147 | end 148 | 149 | stride = 1; 150 | end 151 | end --------------------------------------------------------------------------------