├── README.md ├── TestImages ├── set07_V007_I00269.jpg ├── set07_V007_I01169.jpg ├── set07_V008_I00719.jpg └── set07_V008_I01379.jpg ├── deepPed_demo.m ├── fetch_models.sh └── model_def └── alexnet_deploy_fc7_CAFFE.prototxt /README.md: -------------------------------------------------------------------------------- 1 | ## DeepPed: *Deep Convolutional Neural Networks for Pedestrian Detection* 2 | 3 | Created by Denis Tomè, Federico Monti, Luca Baroffio and Luca Bondi. 4 | 5 | ### Introduction 6 | 7 | DeepPed is a state-of-the-art pedestrian detector that extends R-CNN work done by Girshick et al. combining region proposals with rich features computed by a convolutional neural network. This method achieves 19.90% log-average-miss-rate on the Caltech Pedestrian Dataset. 8 | 9 | DeepPed is described in an [arXiv tech report](http://arxiv.org/abs/1510.03608) and will appear in Elsevier Journal of Signal Processing. 10 | 11 | ### Citing R-CNN 12 | 13 | If you find R-CNN useful in your research, please consider citing: 14 | 15 | @article{tome2015Deep, 16 | author = {Tomè, Denis and Monti, Federico and Baroffio, Luca and Bondi, Luca and Tagliasacchi, Marco and Tubaro, Stefano}, 17 | title = {Deep convolutional neural networks for pedestrian detection}, 18 | journal = {arXiv preprint arXiv:1510.03608}, 19 | year = {2015} 20 | } 21 | } 22 | 23 | ### License 24 | 25 | DeepPed is released under the Simplified BSD License (refer to the 26 | LICENSE file for details). 27 | 28 | ### Installing R-CNN 29 | 30 | 0. **Prerequisites** 31 | 0. MATLAB (tested with 2015a on 64-bit Linux) 32 | 0. Caffe's [prerequisites](http://caffe.berkeleyvision.org/installation.html#prequequisites) 33 | 0. **Install Caffe and R-CNN** 34 | 0. Download [Caffe](https://github.com/BVLC/caffe) (version described in R-CNN instructions) 35 | 0. Download R-CNN and follow the [instructions](http://github.com/rbgirshick/rcnn) 36 | 0. **Install DeepPed** 37 | 0. Change into the R-CNN source code directory: `cd rcnn` 38 | 0. Get the DeepPed source code by cloning the repository: `git clone https://github.com/DenisTome/DeepPed.git` 39 | 0. Get the Piotr's Image & Video Matlab Toolbox by cloning the repository: `git clone https://github.com/pdollar/toolbox.git` 40 | 0. From the `R-CNN` folder, run the model fetch script: `./DeepPed/fetch_models.sh`. 41 | 0. Open the `startup.m` matlab file, adding the two commands `addpath(genpath('DeepPed'));` and `addpath(genpath('toolbox'));` at the end of the file. 42 | 43 | ### Running DeepPed on an image 44 | 45 | 1. Change to where you installed R-CNN: `cd rcnn`. 46 | 2. Start MATLAB `matlab`. 47 | * **Important:** if you don't see the message `R-CNN startup done` when MATLAB starts, then you probably didn't start MATLAB in `rcnn` directory. 48 | 3. Run the demo: `>> deepPed_demo` 49 | -------------------------------------------------------------------------------- /TestImages/set07_V007_I00269.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DenisTome/DeepPed/42d13ad35f211fd04452ce9564551617fca1279c/TestImages/set07_V007_I00269.jpg -------------------------------------------------------------------------------- /TestImages/set07_V007_I01169.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DenisTome/DeepPed/42d13ad35f211fd04452ce9564551617fca1279c/TestImages/set07_V007_I01169.jpg -------------------------------------------------------------------------------- /TestImages/set07_V008_I00719.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DenisTome/DeepPed/42d13ad35f211fd04452ce9564551617fca1279c/TestImages/set07_V008_I00719.jpg -------------------------------------------------------------------------------- /TestImages/set07_V008_I01379.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DenisTome/DeepPed/42d13ad35f211fd04452ce9564551617fca1279c/TestImages/set07_V008_I01379.jpg -------------------------------------------------------------------------------- /deepPed_demo.m: -------------------------------------------------------------------------------- 1 | function deepPed_demo() 2 | 3 | imgPath = 'DeepPed/TestImages'; 4 | listImages = bbGt('getFiles',{imgPath}); 5 | 6 | % load and adjust the LDCF detector 7 | load('toolbox/detector/models/LdcfCaltechDetector.mat'); 8 | pModify = struct('cascThr',-1,'cascCal',.025); 9 | detector = acfModify(detector,pModify); 10 | 11 | % load the trained SVM 12 | SVM = load('data/rcnn_models/DeepPed/SVM_finetuned_alexnet.mat'); 13 | PersonW = SVM.W; 14 | PersonB = SVM.b; 15 | 16 | % load the trained svm of level 2 17 | cl2 = load('data/rcnn_models/DeepPed/SVM_level2.mat'); 18 | 19 | %load the finetuned AlexNet 20 | rcnn_model_file = 'data/rcnn_models/DeepPed/finetuned_alexNet.mat'; 21 | use_gpu = 1; %to change to zero if caffe compiled without CUDA support 22 | rcnn_model = rcnn_load_model(rcnn_model_file, use_gpu); 23 | thresh = 2; 24 | 25 | for i = 1 : length(listImages) 26 | img = imread(listImages{i}); 27 | % detect possible pedestrians with LDCF 28 | bbs = acfDetect(img,detector); 29 | dt_ldcf = bbs; 30 | 31 | % evaluate BBs retrieved by LDCF with our finetuned AlexNet 32 | bbs(:,3) = bbs(:,1) + bbs(:,3); 33 | bbs(:,4) = bbs(:,2) + bbs(:,4); 34 | bbs(:,5) = []; 35 | feat = rcnn_features(img, bbs, rcnn_model); 36 | scores_cnn = feat*PersonW + PersonB; 37 | 38 | % use second level SVM 39 | scores = [dt_ldcf(:,5) scores_cnn]*cl2.W+cl2.b; 40 | 41 | % discard BBs with too low score and apply NMS 42 | I = find(scores(:) > thresh); 43 | scored_boxes = cat(2, bbs(I, :), scores(I)); 44 | keep = nms(scored_boxes, 0.3); 45 | dets = scored_boxes(keep, :); 46 | dets(:,3) = dets(:,3) - dets(:,1); 47 | dets(:,4) = dets(:,4) - dets(:,2); 48 | 49 | % show the final obtained results 50 | figure(1); 51 | imshow(img); 52 | hold on 53 | for k = 1 : size(dets,1) 54 | rectangle('Position',dets(k,1:4),'EdgeColor', 'r','LineWidth',3); 55 | end 56 | hold off 57 | title('Press a button to continue'); 58 | waitforbuttonpress(); 59 | end 60 | 61 | end 62 | 63 | -------------------------------------------------------------------------------- /fetch_models.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" 4 | cd $DIR 5 | 6 | FILE=DeepPed.tar.gz 7 | CHECKSUM=deddc4c230904f726f9ccc865bc747b1 8 | 9 | if [ -f $FILE ]; then 10 | echo "File already exists. Checking md5..." 11 | os=`uname -s` 12 | if [ "$os" = "Linux" ]; then 13 | checksum=`md5sum $FILE | awk '{ print $1 }'` 14 | elif [ "$os" = "Darwin" ]; then 15 | checksum=`cat $FILE | md5` 16 | fi 17 | if [ "$checksum" = "$CHECKSUM" ]; then 18 | echo "Model checksum is correct. No need to download." 19 | exit 0 20 | else 21 | echo "Model checksum is incorrect. Need to download again." 22 | fi 23 | fi 24 | 25 | echo "Downloading precomputed DeepPed model (500 MB)..." 26 | 27 | wget ftp://ftp.elet.polimi.it/users/Luca.Bondi/deepped/$FILE 28 | 29 | echo "Unzipping..." 30 | 31 | tar zxvf $FILE -C ../. 32 | 33 | echo "Done. Please run this command again to verify that checksum = $CHECKSUM." 34 | -------------------------------------------------------------------------------- /model_def/alexnet_deploy_fc7_CAFFE.prototxt: -------------------------------------------------------------------------------- 1 | name: "CaltechNet" 2 | input: "data" 3 | input_dim: 256 4 | input_dim: 3 5 | input_dim: 227 6 | input_dim: 227 7 | layers { 8 | name: "conv1" 9 | type: CONVOLUTION 10 | blobs_lr: 1 11 | blobs_lr: 2 12 | weight_decay: 1 13 | weight_decay: 0 14 | convolution_param { 15 | num_output: 96 16 | kernel_size: 11 17 | stride: 4 18 | } 19 | bottom: "data" 20 | top: "conv1" 21 | } 22 | layers { 23 | name: "relu1" 24 | type: RELU 25 | bottom: "conv1" 26 | top: "conv1" 27 | } 28 | layers { 29 | name: "norm1" 30 | type: LRN 31 | lrn_param { 32 | local_size: 5 33 | alpha: 0.0001 34 | beta: 0.75 35 | } 36 | bottom: "conv1" 37 | top: "norm1" 38 | } 39 | layers { 40 | name: "pool1" 41 | type: POOLING 42 | pooling_param { 43 | pool: MAX 44 | kernel_size: 3 45 | stride: 2 46 | } 47 | bottom: "norm1" 48 | top: "pool1" 49 | } 50 | layers { 51 | name: "conv2" 52 | type: CONVOLUTION 53 | blobs_lr: 1 54 | blobs_lr: 2 55 | weight_decay: 1 56 | weight_decay: 0 57 | convolution_param { 58 | num_output: 256 59 | pad: 2 60 | kernel_size: 5 61 | group: 2 62 | } 63 | bottom: "pool1" 64 | top: "conv2" 65 | } 66 | layers { 67 | name: "relu2" 68 | type: RELU 69 | bottom: "conv2" 70 | top: "conv2" 71 | } 72 | layers { 73 | name: "norm2" 74 | type: LRN 75 | lrn_param { 76 | local_size: 5 77 | alpha: 0.0001 78 | beta: 0.75 79 | } 80 | bottom: "conv2" 81 | top: "norm2" 82 | } 83 | layers { 84 | name: "pool2" 85 | type: POOLING 86 | pooling_param { 87 | pool: MAX 88 | kernel_size: 3 89 | stride: 2 90 | } 91 | bottom: "norm2" 92 | top: "pool2" 93 | } 94 | layers { 95 | name: "conv3" 96 | type: CONVOLUTION 97 | blobs_lr: 1 98 | blobs_lr: 2 99 | weight_decay: 1 100 | weight_decay: 0 101 | convolution_param { 102 | num_output: 384 103 | pad: 1 104 | kernel_size: 3 105 | } 106 | bottom: "pool2" 107 | top: "conv3" 108 | } 109 | layers { 110 | name: "relu3" 111 | type: RELU 112 | bottom: "conv3" 113 | top: "conv3" 114 | } 115 | layers { 116 | name: "conv4" 117 | type: CONVOLUTION 118 | blobs_lr: 1 119 | blobs_lr: 2 120 | weight_decay: 1 121 | weight_decay: 0 122 | convolution_param { 123 | num_output: 384 124 | pad: 1 125 | kernel_size: 3 126 | group: 2 127 | } 128 | bottom: "conv3" 129 | top: "conv4" 130 | } 131 | layers { 132 | name: "relu4" 133 | type: RELU 134 | bottom: "conv4" 135 | top: "conv4" 136 | } 137 | layers { 138 | name: "conv5" 139 | type: CONVOLUTION 140 | blobs_lr: 1 141 | blobs_lr: 2 142 | weight_decay: 1 143 | weight_decay: 0 144 | convolution_param { 145 | num_output: 256 146 | pad: 1 147 | kernel_size: 3 148 | group: 2 149 | } 150 | bottom: "conv4" 151 | top: "conv5" 152 | } 153 | layers { 154 | name: "relu5" 155 | type: RELU 156 | bottom: "conv5" 157 | top: "conv5" 158 | } 159 | layers { 160 | name: "pool5" 161 | type: POOLING 162 | pooling_param { 163 | pool: MAX 164 | kernel_size: 3 165 | stride: 2 166 | } 167 | bottom: "conv5" 168 | top: "pool5" 169 | } 170 | layers { 171 | name: "fc6" 172 | type: INNER_PRODUCT 173 | blobs_lr: 1 174 | blobs_lr: 2 175 | weight_decay: 1 176 | weight_decay: 0 177 | inner_product_param { 178 | num_output: 4096 179 | } 180 | bottom: "pool5" 181 | top: "fc6" 182 | } 183 | layers { 184 | name: "relu6" 185 | type: RELU 186 | bottom: "fc6" 187 | top: "fc6" 188 | } 189 | layers { 190 | name: "drop6" 191 | type: DROPOUT 192 | dropout_param { 193 | dropout_ratio: 0.5 194 | } 195 | bottom: "fc6" 196 | top: "fc6" 197 | } 198 | layers { 199 | name: "fc7" 200 | type: INNER_PRODUCT 201 | blobs_lr: 1 202 | blobs_lr: 2 203 | weight_decay: 1 204 | weight_decay: 0 205 | inner_product_param { 206 | num_output: 4096 207 | } 208 | bottom: "fc6" 209 | top: "fc7" 210 | } 211 | layers { 212 | name: "relu7" 213 | type: RELU 214 | bottom: "fc7" 215 | top: "fc7" 216 | } 217 | --------------------------------------------------------------------------------